Saturday, June 05, 2004

Classifiying Uncertainty

[Sometimes I just get it wrong. This is one of these times. The 2x2 grid below is nonsense. See my updated diagram in a more recent blog entry representing my more recent thinking on how to map De Meyer et al to Deming. I’m leaving the original as a historical record. DON’T READ THIS ]

I didn’t use this diagram in my Cutter IT Journal article though I created it with the intention of needing it. This is a supplement to the content of Chapter 4 - Dealing with Uncertainty. The diagram reflects the 4-types of variation identified by De Meyer et al in their article from Winter 2002 edition of MIT Sloan Management Review, “Managing Project Uncertainty from Variation to Chaos”.

We can use this model to better understand a system for software engineering in a given market. This matrix can be mapped against the Wheeler matrix more or less quadrant for quadrant. Variation is where assigned cause has been eliminated and only chance cause exists within a known and understood system. Foreseen Uncertainty is where there are identifiable risks and understood issues which affect the delivery of the project but the basic market for the deliverable is understood and the business model or go-to-market strategy is understood. Unforeseen Uncertainty is a land where the system is not understood well enough to be under control. It will feel out of control and what gets delivered won’t be exactly what the customer wanted or when they wanted it. This could be because the software development is happening with a new paradigm of tools or method - when teams started using 4GLs for example - but it may also occur in new markets where the model is not understood and the degree of variation cannot be predicted. Finally, there is Chaos, the land where we don’t know what we don’t know and we are trying to find out - neither the market, the business model, the customer base, the product features, or the technology are understood.

From a project management perspective, knowing where our project lies in this matrix is important for setting expectations. With Variation we can simply create a common cause buffer based on existing process control limits. With Foreseen Uncertainty, we create a risk management plan which may contain some mitigation approaches which amount to further buffer. In some industries this is called insurance. Insurance works just like common cause buffering but the aggregation is at a higher level. Say for example, a risk mitigation was to bring on more staff to a project, then those staff must be coming from a pool of people elsewhere in the organization. In other words, the wider organization is underwriting the insurance for the project. Unforeseen Uncertainty is a land of high risk - we don’t know what we know, or our system is not understood. Playing in this quadrant could also be called gambling. As every gambler will tell you - you win some and you lose some. Many VC’s will tell you that they lose up to 19 for every one they win. The Unforeseen Uncertainty quadrant is the land of venture capital. Finally, Chaos is a land where only the research budget should venture. It’s extreme gambling. It’s adventure rather than merely venture. It’s a world where you assume you lost your money and nothing got delivered. Delivery is a bonus. It’s like winning the lottery. The project management objectives for Unforeseen Uncertainty are simple - try to move the project into the Foreseen Uncertainty quadrant before the money runs out - then ask for more. With Chaos it is similar - try to move the project into either the Unforeseen Uncertainty of Foreseen Uncertainty quadrants before the money runs out. It is unreasonable to buffer a plan in the right hand quadrant. It simply costs too much. Better not to buffer, to get as far as you can with what you’ve got and demonstrate that you’ve moved to a world of greater certainty before asking for more resources.

It’s easy to see from this matrix that the more uncertain, the greater agility is needed to react to change. In the right hand column, the iteration cycle must be short, the feedback loop very tight. By understanding where our project lies in this matrix we can make decisions about what represents the best iteration cycle and make informed guesses about how much we want to invest in requirements and analysis versus code, test and refactor.