Thursday, October 21, 2004

Getting Back to FDD’s Roots

Over the course of this summer I’ve given 3 talks about The Coad Method’s Contribution to Agile and presented both on the evolution of Coad’s domain modeling techniques and the evolution of management techniques in FDD at the Borland Conference. FDD’s roots in the The Coad Method as described in Peter Coad’s 1995/1997 book Object Model: Strategies, Patterns and Applications has tended to be glossed over in recent years since the Agile Manifesto was written. For a while, any form of analysis and design was out of favor in the agile community. However, Eric Evans with his tomb, Domain Driven Design, has made the whole idea of domain modeling respectable in the agile community again.

However, it isn’t so much the domain modeling which I think is the unique innovation from the Coad Method in FDD but the very notion of a Feature as a fine-grained piece of client-valued functionality which can be developed in a short period of time - in my experience anything from 4 man hours to 8 man days with the mean around 10 man hours on projects of any size or scale beyond one or two developers. Features use a language template of <action> <result> <object>. This format lends itself to object-oriented implementation. The <object> is the class which will implement the method whose name is <action> and the <result> is the return value from the method. Every Feature maps to a single UML Sequence diagram (whether you choose to draw it or not). Features are very tightly defined.

As such Features enable a lot of things. Features are predictable and repeatable within a reasonable tolerance. Features are approximately the same size and both the effort involved, and the likelihood of defects can be estimated fairly accurately. Features are also a unique measure of work-in-process inventory. They enable a style of management and a mechanism for reporting which isn’t possible with tasks where there is no codification scheme. Features enable end-to end traceability. In the most agile of FDD implementations, Features are the requirements, and yet they map on to the analysis, and directly to a sequence diagram in the design. They name a method in the code and identify a class in the domain model. Features are transparent. Features are loosely coupled. Without Features, FDD is like any other iterative and incremental method of software development. With Features it is more like an MRP system for software engineering management whilst double timing as an architectural approach to building systems.

FDD is perhaps the first example of a domain model-driven agile process. As we move to a world of model driven design (Borland Together already does this and the forthcoming Microsoft Class Designer and Software Factories apporach follow the same lead) and its alternative model-driven architecture, FDD seems to be the only agile method prepared for the paradigm shift. This wouldn’t be possible without its roots in The Coad Method.

There is a lot more in FDD which dates to Peter Coad’s work in the mid-nineties. For example, the notion of “frequent, tangible, working results” was Peter’s slogan for much of the decade and it translated into the rule that an iteration should not be more than 3 months long without releasing working code. The notion of consensus dates to Coad’s work too - this time with Fred Racey as documented in The Coad Letter - Lessons Learned from Fred. Fred Racey had a big influence on FDD. The adoption of the Tuckman model for team working of “forming”, “storming”, “norming” and “performing” came from him. So did much of the advice on facilitation of domain modeling sessions, including team working norms and the use of “pluses and deltas” at the end of daily sessions.

Peter Coad was also teaching the use of Kano modeling for Feature prioritization. For some reason this didn’t get published until I included it in my own book but the idea of using a scale of 0..5 for how much you desire the inclusion of something and 0..5 for how badly disappointed you would be if it were missing was something Peter taught us in Singapore in September 1997.

Some people have suggested that FDD doesn’t need Peter Coad’s modeling techniques in order to work. They worry that inclusion of domain modeling is a barrier to entry and a turn off for the prospective agile developer. I just can’t agree with that point of view. I think it is true that any method which asks you to make a fine-grained plan, to batch the plan into work packages of approximately two weeks and to build the project incrementally from there, will lead you to success. However, FDD without domain modeling and without the Feature template of <action> <result> <object> looks like an asynchronous rendition of Scrum. As communicating around an asynchronous schedule is harder, it creates a barrier to adoption. It would be better to use Scrum and be done with it. The true innovation in FDD and the enabler that makes it so incredibly productive and such a great platform for continuous improvement is The Feature and Features date from The Coad Method in the mid-nineties. With the growing popularity of domain driven design, hopefully the agile community will take a closer look at FDD and try to understand it better - try to understand the value of its roots in The Coad Method and the agility that it enables.

Sunday, October 17, 2004

Back to my Roots

I’ve been thinking a lot recently about my start in the software business as a games programmer. A few themes keep coming up in the agile community which for me are simply being rediscovered.

Pair Programming - yep, I was doing that in 1983, but we only used it on new stuff, stuff with high uncertainty. What might be called “research” - faster methods for scrolling the display, masked sprites, 3-D graphics engines, multi-channel sound.

Very Low Defect Software - the audience for computer games in the early 1980’s was teenage boys and they just didn’t tolerate defects. In fact most of the “journalists” were teenage boys too and they didn’t tolerate defects and had the power to kill sales game with a poor review. In fact the defect rate was close to zero per game though I do know of at least 2 of the twenty or so games I published that shipped with a defect.

Short Lead Times - this Crash interview from 1985 demonstrates that it only took a small handful of us - only 2 or 3 developers - to ship games with up to 60,000 lines of code every month. Games were a lot simpler in those days - and a lot less expensive to produce.

Re-usable Code - one way that we met these incredibly short lead times was excessive re-use of code. Towards the end of the 1980’s we had refined this into what amounted to an object-oriented language executed through the use of macros in the assembler. Each object (a data structure) had sets of behavior (assembly procedures) which could be run against it. There was effectively encapsulation and polymorphism but no inheritance.

Perhaps what was most odd - looking back - was that we took it all for granted. It was reasonable to develop 80 lines of code per day (as an aggregate average over 3 years) with near zero defects. In the games world, we were lean and agile or we ceased to exist. None of us had any comprehension of the inefficiency in what we came to know as information technology.

I have many memories from those days but one that stands out for me, was the day that I took my watch off and didn’t wear it for a couple of years. Why? Well I was too troubled with clock watching whilst I debugged problems. As I was being paid on completion, time spent fixing bugs was eating into my hourly rate - yes, even at 17 years old I was worrying about this stuff. I felt that clock watching and stress were counter-productive to software quality so I took my watch off and just didn’t wear it. I measured time with Mazlov’s hierarchy issues like “Am I tired?”, “Am I hungry?”

So it is surprising to me now, to encounter the SEI’s Personal Software Process (PSP) which is all about time management and both estimating and tracking of time spent on specific development tasks in order to improve quality. There does seem to be anecdotal evidence to suggest that the method works. My employer’s IT division are adopting it. There are other case studies. Watts Humphrey has run scientific studies on it. As a result of work I’m doing writing a formal software development method, I’ll be studying PSP and its big brother TSP (the Team Software Process) a lot more. This means I’ll be writing more about them here at agilemanagement.net over the coming months.

I come to this with a feeling that although PSP/TSP can be shown to work, I am skeptical that they are the only true path to almost zero defect software development. It is unreasonable to correlate time management and time based estimation and tracking as the only way to achieve high quality in software engineering. I’m personally very familiar with someone with a track record of delivering defects in the order of 1 per 10,000 lines of assembly language without the use of anything other than the sun in the sky and an empty stomach to estimate the passing of time. It’s important that we don’t forget that the existence of one set of empirical data does not prove that a method is correct to the exclusion of others, it merely proves that a method is effective in a set of circumstances. PSP seems to be effective but to my mind, is not the only way to achieve low defect software, nor is it the only way to achieve a CMM(I) rating of level 5. Do you have experience with PSP? Please share!

Tuesday, October 12, 2004

Developer As Executive

Writing in “The Effective Executive” (1966) Peter Drucker expresses that all knowledge workers are de facto executives.

“I have called ‘executives’ those knowledge workers, managers, or individual professionals who are expected by virtue of their position or their knowledge to make decisions in the normal course of their work that have impact on the performance and results of the whole.”

By this token, all software developers are ‘executives’ in Drucker’s definition. Why is this true? And, what does this mean for software development management?

Every line of code a developer writes affects the performance (functional or non-functional) of the product or system in development. In the example of a software product company, the competitiveness of the business relies on the performance of its product. More than that, the competitiveness of the business requires the software development group to be capable of responding to market demands with appropriate functionality, quality and timeliness. Every design decision potentially affects the ability of the business to respond to those market demands. If developers are left to simply design, develop and test on their own without supervision then every day, individual developers are making executive decisions, on their own, in open loop.

It isn’t good enough to demonstrate that code meets the functional requirements because the working code disguises the many executive decisions that the developer made whilst producing it. The effect of these ‘executive decisions’ may not be felt for many months to come. Cruft in the code can lead to long lead times, poor quality and dissatisfied customers. The introduction of design and code inspections based on established guidelines and best practices effectively elevates the ‘executive decisions’ to a group or communal level. The guidelines should encode the ‘executive decisions’ which have been made concerning tradeoffs like flexibility versus simplicity in design. The guideline documents surface the executive decisions made by the technical team so that management can agree that the decisions are properly aligned with the strategic direction of the business. The inspection process is there to verify that executive direction was carried out correctly.

Processes such as FDD, which ask for designs to be completed by Feature Teams and code to be reviewed by a Feature Team whilst a Chief Programmer is held accountable for the execution of the process and the quality of the product, remove the executive decisions from the individual. Individual developers are not empowered to make executive decisions on their own, rather they are enabled to contribute to them as part of a team. In this respect the managers of the business can rest a little easier that some control is being exercised over the strategic direction and its alignment with tactical decisions.

The term ‘technical debt’ has crept in to the vernacular of the agile movement recently. Technical debt is the notion that code is left in a state unfit for long term maintenance - it works now, but only just. In order to continue using and re-using the code in future iterations, some refactoring will be needed. Technical debt is the stuff of executive decisions. It should be surfaced and visible to management that a decision was made to leave a cruft debt in the code base. It should be part of the guidelines, i.e. if YAGNI is your mantra then your guidelines should explain what that means and provide examples. Management can then decide whether the choices are aligned with the direction of the business and needs of the market. If cruft exists because developers were left to create it on their own - making executive decisions in open loop - then the business will ultimately get what it deserves - beaten in a competitive market!

The message from Drucker is clear - knowledge workers make executive decisions all the time in their daily work. So pay attention to the little things. Understand that each knowledge decision is an executive decision and put processes in place to control the quality of those decisions and insure that they are aligned with the strategic direction of the business.

Friday, October 08, 2004

Drucker on Teamwork

In the agile movement, we talk a lot about the power of team work and agile methods spend a lot of energy on team working and organizational structures and communication plans for team working. Peter Drucker agrees. Writing in “Management in a Time of Great Change” (1995) he says,

“In the knowledge society, it is not the individual who performs. The individual is a cost center rather than a performance center. It is the organization that performs.”

I’ve mentioned before why individual measurement is bad but Drucker seems to go further. He thinks it is pointless.

Wednesday, October 06, 2004

TOC ICO Conference - Miami Oct 25th

I’m going to be speaking at Eli Goldratt’s Theory of Constraints International Certification Organization (TOC ICO) conference in Miami at the end of this month. It’s a real privilege to be invited as there are only 14 speakers over the 4 days of the event. My talk will cover a summary of my book and my recent work on Lean and the Theory of Variation / Six Sigma. There will be slides and a paper published in the conference proceedings. I’ll make them available here later in the year. If you are attending the event in Miami, please make a point of introducing yourself.