Yesterday, I talked about trust as the essence of agile software development. Today, I want to talk about a lack of trust as the scourge of traditional heavyweight software engineering methods.
When we started out to create MSF for CMMI Process Improvement, it was known by the codename "MSF Formal". The business plan called for two methods - one agile, one CMMI Level 3. These had been identified by market research as areas of interest to our core customers for Visual Studio products - IT departments and CIO's. It was assumed (we now know wrongly) that they were two ends of a spectrum - opposites. Agile wasn't formal and formal wasn't agile. That of course is correct. We associated formal with American aerospace and defense contracting and government contracting. We associated it with existing CMMI adopters. And hence assumed "formal" and CMMI were synonymous. Of course, we don't see aerospace and defense as a big market for Visual Studio Team System this time around - v2.0 in 2007 maybe ;-). Sure there is Microsoft platform development in that world but as a percentage of market share for Team System, it is a small market. Our core market is IT departments and outsource vendors.
What we assumed was that those who wanted to adopt formal quality assurance, needed a formal method like those used in American defense and government contracting. We even had internal evangelization presentations that described "MSF Formal" as having "all the knobs on 11" - where the knobs were documentation, gates and signoffs, verification steps and so forth. We identified these as aspects of a trustless environment. We talked of "MSF Formal" as a "low trust method". Where there is low trust, there must be documentation, verification, audit and potentially punishment. Where there is a lack of trust, there is fear. Where there is fear, there is a lot of people taking cover and protecting themselves. Such protection is waste - muda. It takes the form of overly long schedules, overly detailed specifications, precise plans and elaborate change procedures to track all of the changes, audit trails and assigned accountability and responsibility - there must be a scapegoat.
We all know that waterfall is bad. I've been preaching that large batch sizes are bad for years now on this site. However, not all formal development is waterfall in nature but it can still be bad from an economic perspective because it is riddled with fear and fear breeds lots of waste. We anecdotally describe these processes as "heavyweight". What we really mean is "lacking in trust." Methodologies can be iterative and have lots of feedback and understand people and their psychology but if they encourage and breed fear and leave out trust then there are still likely to be heavyweight and wasteful. By holding a mirror to the problem, we see in reflection, trust as the essence of agile.
Luckily, we had an epiphany and gained a deeper, better understanding of the CMMI before it was too late. We didn't go and create a heavyweight, formal process with all the trust taken out. That wouldn't have been doing our market any favors. We wouldn't have been adding value. No CIO in his right mind wants a wasteful heavyweight process even if it does gain him a CMMI Level 3 badge to brag about on the golf course on Saturday mornings. What we are delivering shows that formal quality assurance and agility are compatible because trust can be part of it. But we should have known that already. It was Deming's mantra - drive out fear! Drive out fear! DRIVE OUT FEAR!
