It seems to be the season for the academic community to publish papers about FDD. Here is another recent one by Serguei Khramtchenko of Harvard University, Comparing eXtreme Programming and Feature Driven Development in Academic and Regulated Environments.

What encourages me most about these academic papers is that the authors seem to have captured a better understanding of FDD from reading the literature and without exposure to the originators than some authors in the agile field. One key element these authors have understood which is often misquoted is that modeling comes before the feature list in FDD. Yes! Modeling is used as a discovery process. Something else Serguei really latches on to is the power of color modeling and its vital role in making FDD extremely lean, "While they are not expected to become UML gurus I witnessed amazingly quick learning of color UML."

|

Jianxiong Pang and Lynne Blair show in this new academic paper, Refining Feature Driven Development - a methodology for early aspects, how to adapt FDD for use with Aspect Oriented Software Development (AOSD). It shows clearly that Features readily represent an aspect-oriented requirements engineering approach but that the design-by-feature and build-by-feature steps can be refined for use with aspects to develop more easily reusable features - separated as individual aspects.

By and large I wholeheartedly agree with this paper. It's a pity that so much FDD related material hasn't been very explicitly documented. For example, the authors suggest the idea of a persistency (sic) feature set which would contain features to describe persistent storage. Actually Peter Coad wrote about this concept in 1997 when he described 4 types of features PD (problem domain), HI (human interface), SI (system interface) and DM (data management) where the DM features are equivalent of the persistency features suggested here. Equally, the compatibility features described could be argued are covered under the heading of SI.

However, I have argued previously and documented in my writing that there are some non-functional requirements not covered in the original FDD literature that do deserve to be documented as features. Although these do not represent "client-valued functionality", they are "client valued non-functionality". They represent some differentiating ability for which the customer will pay (i.e. associates a tangible value to them) such as a performance or speed improvement.

What I like most about adapting FDD for aspect-oriented programming is the clear alignment and ease of traceability. Every line item on the Feature List, now has a specific aspect in the code. Talk about properly aligning a value chain - every developer can speak concisely and articulately about what they are doing and why it is valuable to the client, business or end consumer - whilst management can (automatically) track progress towards the goal by monitoring the delivery of aspects. Now that really is the basis for agile management!

|

If we mix the topics from yesterday and the day before to get FeatureComplexityPointDays, we have created a control metric or "smell test" for our development team. If a Feature with Complexity Points of 2 is supposed to take 1 man day and we desire that Lead Time should never exceed 2 weeks (10 man days) then an under control FDD process should have a FeatureComplexityPointDays per Developer score of 20 or less.

For example if a Feature Team consisting of 1 Chief Programmer and 2 Developers has a Chief Programmer Work Package with 15 Features with average complexity of 2 then they have a CPWP with an estimated FeatureComplexityPointDays total of 30. This CPWP should typically complete in 5 working days. If it takes longer or shorter then it might just show that the estimate was wrong. This is normal common cause variation. However, if the actual FeatureComplexityPointDays number exceeds some control limit, say 60 [a precise calculation of a control limit needs a set of real data sampled daily over a short period to get an average moving range], then it's a strong indication that something is wrong - an identifiable, assignable cause. Identifiable, assignable causes can and should be addressed by management intervention.

Metrics cannot tell you the root cuase of the problem. The manager needs to act as a facilitator to help find the cause, then use his or her judgement to implement action to remove the assignable cause which took the process out of control.

|

Over the last few years, I've written several times about my 5 point scale for Feature Complexity in FDD. As my colleague Daniel Vacanti tells me,

"All features are not born equal"

and neither they are!

In fact the size or complexity of a Feature is a particularly important thing to know in two situations. The first is when you are working on a short iteration cycle anything from 2 to 8 weeks. The second is where you are close to complete - only a few Features to go, but equally only a few days on the schedule remaining and you don't want to be even one day late. Feature complexity is not important on larger projects such as the original FDD project in Singapore - about 15 months, 2000 business logic features and as many more again for UI and interfaces to legacy systems. We have shown over the years that a statistical sample of Features tends to show a very normal distribution of complexity and there is indeed the concept of an "average Feature". Hence, when trying to estimate an FDD project it is often easier to do it for a 500 or 1000 Feature project than it is for a 80 Feature project.

So Feature Complexity Points provides a way of managing FDD projects accurately in smaller iterations. I use a 5 point scale because psychologically it is easy to categorize things into 5 buckets. However, what are those five buckets. Well that hasn't ever been adequately written down. It's heuristic. I once wrote a suggestion that it is roughly based on the number of classes touched by the Feature and this is still a good rule of thumb. The more classes required, the more complex. Features which touch only a single class such as enquiry on a boolean variable - is it set of not? - are so simple that they are definitely complexity 1 out of 5. A Feature with a great deal of algorithmic complexity or involving complex business rules may be a 5 on the scale.

As a default, I then use a power law to translate the Feature Complexity Points into effort. 1 = 0.5 days, 2 = 1 day, through to 5 which is 8 days or more. Remember in FDD a Feature should never take more than 10 days. If it's so big that you can't envisage completing it in 10 man days then analyze it more and change it to more, smaller Features.

I'm beginning to build a list of heuristics for Feature Complexity and I will publish it later this year. Meanwhile, inventory measured as Feature Complexity Points that are tracked on a CFD chart will tend to give more accurate measures than simple Feature tracking.

|

It's a commonly held misconception that my book has a heavy focus on financial accounting for software engineering. The root of this misconception is in the difference between management accounting and financial accounting. The two are not the same and for good reasons.

Financial accounting is something performed to agreed standards and principles (GAAP) designed to fairly and accurately report the true worth of a business to its owners and its true profits to the tax collector. Management accounting, on the other hand, is a mechanism for using a financial metric (dollars) as a normalizing mechanism for making decision about (often vastly) different choices and alternatives. Management accounting helps you decide whether it is better to add 900 customer account advocates in a call center in Ireland, against staffing a user experience and technical writing department in St. Louis, MO, and investing heavily in intuitive products. Management accounting helps you make decisions. It helps you to do the right thing.

Hence, my book does use management accounting as a mechanism for evaluating the merits of different aspects of the software project lifecycle and different styles of development process. The Throughput Accounting model in my book is designed to help you make decisions, for example, should you invest more in requirements, or spend more on development?

To really emphasize that management accounting has nothing to do with financial reporting, Eli Goldratt in The Haystack Syndrome, introduced a metric he called DollarDays. This is the multiplication of the value of inventory held by the number of days it has been held. Clearly, DollarDays has nothing to do with the financial worth of the inventory. It does not double in value every few days. The purpose of DollarDays is to underscore a simple lesson - holding inventory is bad practice! By using a DollarDays metric as a replacement for Investment, it makes the ROI on slow turning inventory look very bad. The purpose is simple. Amplify the valuable information and improve the signal to noise ratio from management accounting.

We could try something similar in agile development. How about StoryPointDays - multiple of the number of WIP Story Points by the number of days they have been in development - or in FDD, FeatureDays? The target would be to minimize the number. Each team in a larger organization could report its FeatureDays score each day. Management could see at a glance whether the team was making a good job of keeping batch size (Chief Programmer Work Package) small and lead time short. The lower your StoryPointDays score, the more agile you are!

|

You Aren't Going To Need It In This Iteration Or Release!

Following up on yesterday's post on refactoring, I'd like to characterize FDD's approach to the XP practice of YAGNI. To use XP terminology for a moment, FDD has an architecture spike at the beginning - build a domain model - followed by a planning spike - build a Feature List followed by a loose scheduling process which accepts variance and uncertainty. Only then does the hard work of design and coding begin - typically one week into the project rather than one hour or one day.

FDD is saying that investing up to one week to gain information about an iteration or release is worth it. In other words, there is a return on investment due to reduced refactoring. Hence, FDD does not follow the YAGNI rule on a per Feature basis. Instead it says, find the simplest thing which makes sense for the whole iteration - based on the architecture work and modeling done in the first week - and build that. Does that mean in week 2, you might be building a class which is needed in week 4? Yes, it does! Isn't that risky? No, it isn't! Unless you are living in a world where the requirements suffer significant change and the business model of the customer changes at such a rate that the half-life of the requirements is less than a month, then it is not risky to build code which was identified during the initial architecture spike or domain modeling.

|

It's funny how life can have its little coincidences. There has been a fairly active conversation in my Yahoo! group about how to deal with refactoring and whether or not it constitutes "dark matter" which should be tracked on the Feature List and on the Cumulative Flow Diagram. Similar questions have been asked over at Jeff De Luca's site this month. It may take several blog entries to fully cover this topic but here goes for the first one.

As Jeff says in his reply refactoring seems to mean many things to many people. Brad Appleton has pointed out to me that Martin Fowler is now using two terms - refactoring and restructuring. Refactoring is the idea that as you grow an architecture and add classes and methods, you have to make changes to existing classes and methods and the tests which validate them. These changes are small and are more like adding associations and perhaps changing method signatures. Then there is restructuring which is outright changing of the model because it couldn't cope with a new requirement.

So first let me say that I wholly agree with Jeff De Luca's opinion, that refactoring to use this new and refined definition, is "business as usual" for FDD. FDD embraces the idea of refactoring without making a song and dance about it. Refactoring is part and parcel of the Design-By-Feature activity and any refactoring of code and tests needed on existing classes would be hidden as part of the Build-By-Feature activity. There is no need to show this as additional Features and it is not "dark matter".

Restructuring on the other hand, for example, taking a class inheritance hierarchy and reorganizing it using <<Role>> classes using object inheritance to provide additional flexibility and re-use, that would be rework of existing code which is already credited in the earned value percentage for the project or the in flow of value recorded on the cumulative flow diagram. Such a change, even if it gets called "refactoring" would definitely want to be identified as "dark matter" and listed on the Feature List of the project. Why?

Figure 1. Class Inheritance for Bank Account Borrowing and Depositing

Simply put, restructuring represents rework which eats into the productivity of the developers. It reduces the velocity on the visible client-valued functionality. If restructuring remains hidden and invisible then it simply makes the team look slower. However, if the restructuring is surfaced as a Feature and there is transparency on it, then questions can be asked. Why is this restructuring taking place? What is its value? The answer might be simple. I want to restructure the class hierarchy in order to make re-use out of what is current a sub-class.

Figure 2. Object Inheritance for Bank Account Borrowing and Depositing

In the example shown in Figure 1 and 2, the behavior associated with borrowing and depositing money from/to a bank is restructured into an object inheritance model using <<Role>> classes. This allows the Depositor and Borrower behavior to be reused by both the individual Person class and the corporate Organization class. Hence, the restructuring is being justified on the grounds of re-use.

Note that this restructuring probably has a payback. The "dark matter" Features for the restructuring work may have an immediate impact because Features associated with borrowing and depositing by an Organization may be dropped from scope or replaced with the restructuring Features.

|

I've talked about "dark matter" before - emergent Features which are missed in the initial Feature List analysis but discovered as the team gets into the Design-By-Feature, Build-By-Feature cycle. Dark matter is not scope creep a.k.a. change requests - those should be plotted differently. I thought I'd show you an example. In this diagram, initial scope starts to fall as Features are pushed from the current iteration to a future iteration. The team gradually discovers that the customer didn't really need those features at this time. Meanwhile, their own analysis gradually reveals the dark matter. These previously hidden features get added to the feature list and plotted as part of the Feature Inventory in a Cumulative Flow Diagram [see the CFD for the same project]. Then a change request arrives. It is modeled and sized and it too is added to the overall scope. Later it is discovered that this change isn't really needed until much later and it is dropped from the Feature Inventory for this iteration, being placed into the future feature backlog.

[Update: There has been a longer, deeper thread of discussion on this topic in the agile management Yahoo! group]

|

My blog on Tribalism seems to have caught the attention of at least a few notables from what they want to be known as the "XP School of Thought". My intent with the piece seems to be misinterpreted by those who see it as critical of XP. So now it is time to give credit where it is due.

eXtreme Programming deserves credit for its leadership - for its role as vanguard of a revolution. I'm sure that Kent Beck, Martin Fowler, Ward Cunningham, Ron Jeffries and others did not set out to create a cult, tribe or religion. Their intent was to preach a better way of building software and to show that software development could deliver value and delight customers.

Where XP seems to have scored is with its attractiveness to developers. It appealed to their psyche. It is almost a side-effect that it happens to be more productive in many circumstances. It was XP which gave the agile movement momentum. In the popular press, such as Business Week, you never see "agile" mentioned, you see "extreme programming" used as synonymous with the whole agile movement. The enlightened amongst us know that is wrong. But in my view it is all right. If the wider world wants to call it XP because it gives them a handle with which to grapple then that is fine. I'm not in love with a brand. Like many other thought leaders in this space, what I want to see is a more economically effective software industry.

In the longer term, political devices such as polarization of a debate amongst artisans or intellectuals will fade. There is no need to demonize an enemy when the enemy is already you. So too will branding of prescriptive solutions. What will prevail will be the lessons and the principles. We are only just beginning to get to grips with these, to abstract out from the prescriptive solutions what is truly useful. I believe that this shows a growing maturity. Perhaps we can leave blind faith and tribal affinity behind and adopt a more scientific approach to achieving the goal of more economically effective software production.

In years, or probably decades, to come, software developers will accept the humanity in knowledge work as a basic tenet and understand that to compensate for human frailty it is necessary to do things in small batches, focus on high quality and highly effective communication. All of these things will be taken for granted. At the root of this revolution in how software is done, will have been eXtreme Programming , first and foremost and above everything else that was once called "agile".

|

Writing about my Irish project, the other day, prompted me to examine the quality produced by well run and well executed FDD projects. The project in Ireland produced a very low defect rate. The business logic features had only 5 bugs from 153 features. This number resonated with me. 5 bugs in every 100 features sounds like a typical result. I've been doing FDD projects in telecom companies for the past 5 years and I've seen this similar level of quality. I'm not talking about released bugs. I'm talking about code handed off from development to quality assurance. The release code is typically perfect or near perfect.

The key seems to be close adherence to the process. Batch sizes must be kept small - Chief Programmer Work Packages must be capable of completing in two weeks or less. A proper domain model must be created. Each of the 6 feature milestones must be conducted properly with particular emphasis on design review and code review. When all of this is done properly coupled with unit testing* then the quality of the code is near perfect.

Quality of 5 bugs per hundred features is comparable with CMM Level 5 quality levels typically reported as bugs per function point. There is a strong correlation between function points and features. I would go as far as to describe features as object-oriented function points. The metrics for the telecom industry state that a function point averages 125 lines of code. This is very consistent with the size of a feature. Features lead directly to sequence diagrams for design and hence to methods on classes. They are functions.

It's interesting to reflect that FDD teams which would pass a CMM Level 2 audit and at a stretch might get to Level 3 can produce CMM Level 5 quality almost straight out of the gate.

[*Note unit testing was always part of the process but Jeff De Luca chose not to define when it should be done and to leave that to the individual implementation. Hence, contrary to popularly held belief, test first is allowed in FDD]

|

A sense of belonging can go too far and become problematic. It goes too far when it becomes tribalism rather than mere affinity and shared experience. The human being is a naturally tribal animal. The tribal system seems have evolved for our survival. But deep in our genetic make-up their seems to be a core conflict. Women are genetically programmed to breed-out, to maintain the breadth and health of the gene pool. Men on the other hand are genetically programmed to protect their own genetic code and this manifests itself in a tendency to beat the livin bejeezus out of anything carrying a significantly different set of genetic code. So women make love with other tribes and make war against them. This is a core conflict for which I doubt the ability of The Theory of Constraints to find an injection. In modern society, most of us ease our inner tribal anxiety with sport. In the UK and much of the rest of the world it is football (soccer) and in the United States, it is college sport - anything will do, baseball, basketball, football (sic), etc. This is usually accompanied by a spouse sitting rooting for the other team just to annoy us (men).

However, we see tribalism elsewhere in society too. I get asked about it all the time...

"Anderson, eh? What's your clan?"
"Anderson!"
"Hmmm. That so! You have your own clan! Do you have a tartan?"
"Yes!"
"A kilt?"
"Yes, but I never get a chance to wear it in the United States."

Political scientists will tell you its a 101 first year college credit to understand why politics tends to polarize into two camps. In the United States your either a Donkey or an Elephant or a laughing stock Naderite who "wasted his vote". In software lifecycle process in this modern decade, you are either an eXtreme Programmer or you are a traditional software engineer. Or so a large element of the XP camp would have us think. The truth is that this form of tribalism is as useful as 30,000 foul mouthed neds shouting racial abuse at a black player from the terracing of Glasgow football stadium [Ed. They are all seater stadiums these days].

I wrote my feelings about this loss of focus on what is truly important before. Asking are you extreme or not? is not the right question. Asking, how extreme is it? is still the wrong question. The right question is asking what is the best thing to do to maximize the profit and return on investment now and in the future for the stockholders. The answer will vary according to the business - both the business model and the maturity of the industry. Understanding the principles behind eXtreme Programming is what is fundamentally important then knowing when to apply those principles appropriately is what follows.

If you must insist on being tribal and in some virtual form of painting your face with blue die, wrapping your body in ten yards of plaid in a set pattern and playing the bagpipes for the neighboring bullies, is your thing, then remember that your tribe had better stick to its own territory or it might just find itself in a battle it can't win.

Of course, denial ain't just a river. We don't fight wars over software process. There is no contest to win or lose. When we don't want to be objective, we humans tend to bury ourselves in denial and support our egos with belief. Sometimes the tribalism associated with XP goes further. It becomes a religion - a blind faith, a bellief without objectivity. I once had lunch with a staff member to get to know him. I asked what he thought of agile development and he replied, "Well I'm an eXtreme Programmer, so...." When I relayed the tale to our CFO, he said, "So do we need to rehabilitate this person?" I'll leave discussion on XP as religion for another time.

|

While I'm on the topic of feeling a sense of belonging to a team, I thought I relate this other tale which dates back to the summer of 1999. I was working Dublin, Ireland at the time. 1999 was a great time to be alive. It was the bubble. It was a great time to be in Ireland. The 90's had been good to the country. A combination of a pro-business tax regime, demographics which were producing a large number of young graduates, inward investment and the spoils from years of EU handouts, were turning Dublin into a boom town. It was impossible to get service in a restaurant because all the would-be waiters were working as web developers.

I was working at a company still known as Trinity Commerce though it was already part of Telecom Eireann which was due to be sold off by the Irish government in an IPO. Brían O'Byrne, now of Statesoft, was another member of the team. It was in fact the project where we first created the state machine execution engine for Statechart models. And in some ways, that is at the root of this story.

Earlier in the year, I had analyzed the project and sized it as 153 business logic features. The business logic was being written in PL/SQL for Oracle 8. I had no control over that. But nevertheless we ran the project as an FDD project. A team of 7 people spent 11 weeks building the business logic. They were very professional. Everything got designed, tested and reviewed. And the quality was very high - only 5 bugs in total. Meanwhile, the customer had refused to descope some of the UI requirements. We had pleaded with them that their specification was for a GUI app but they had asked for it to run in 3.x web browsers. They refused to budge. In the end, we were forced into building the state machine engine to track the state of the UI. This caused a delay in the UI development. A long delay. It took 12 weeks to build the first state machine engine. However, the advantage, if there was one, was that we had a complete UI design and a full statechart model before we started coding the UI proper.

So we were in a world where the project wasn't yet late. The business logic was complete but almost no UI existed except the infrastructure. Still nothing to show anyone.

Remember, it's the bubble. You can't get served in a restaurant and hiring developers is difficult. To fill the skills gap, the principals of the company had visited a local college and hired 5 fresh graduates. We had three of them on our team. Now these people are probably all experienced professionals today but in those days they were as green as the grass rustling in the fields near the Sandyford Industrial Estate office where we were working. So we were frighteningly close to being late and half the UI development team was straight out of college.

In order to make significant progress, it was agreed that we would work a weekend. Hey, I was a contractor, I got paid by the hour! I agreed to buy the pizza out of my own pocket as a gesture to the team. There were 7 team members altogether including the 2 guys who coded the framework. In addition, one of the business logic team agreed to come in, to deal with any issues. His name was Brian.

Brian soon got very bored with his weekend. After all, the business logic was bullet proof and the system had been designed with statecharts. Every single call we needed from the UI already existed and it all worked. Brian had little to do but wonder around and gaze out of the window. Luckily he was gifted in an Irish way and kept us all amused with jokes and idle chit-chat.

The application was being deployed in Oracle Application Server which predated EJB 1.0. It was a whole different product in those days. One peculiarity about it was that it only had a single global logging queue and we relied heavily on logging instrumentation for debug information. We had seven people writing View and Controller classes - each one of them with a piece of the statechart model to code. In order to test, a developer had to obtain a "lock" on the app server, so that no one else would run the code and corrupt the log file. The team were sitting in a U-shaped corner about 4 feet apart. The "lock" was obtained by asking "Can I clear the log file?" This shortened to "Clear the logs?" by mid-afternoon on the Saturday. Heads down at the keyboard coding, any conversation was interrupted every few minutes with a cry of "Clear the logs?" Every few minutes a new feature was being tested by someone on the team. The team morale was high. Everyone new progress was being made. They could hear it. They could see it - every time they ran a test. In such a structured framework, it was easy to make rapid, accurate, bug free progress. It was good to be alive. It was good to be in Ireland in the bubble and it was good to be on the UI team of the EIBS project that weekend.

On the following morning, Brian was wandering around the office building bored beyond belief and feeling decided spare. All thanks to the wonderful quality of the code he and his colleagues had written. He really had nothing to do. He was a PL/SQL programmer. The UI was written in Java. There was nothing he could do. He took to raising his arms and shouting "Clear the logs!" "Clear the logs!" After a while, we called him over and made him an honorary member of the UI team. He was now officially allowed to cry "Clear the logs!" whenever he liked. He was one of us. He belonged!

|

It's not uncommon for groups within big organizations to feel lost. They sit and wonder what the meaning of their work is. They have no idea how it is aligned with the corporate goals or how they can influence the stock price. There are many big companies which have lost their way and lack vision or leadership at the top. I even worked for one where the two top leaders were later dismissed and investigated by the tax authorities for illegal dealings - allegedly, they more worried about their own wealth than performing their fiduciary duty. In such circumstances  it can be hard to motivate a team of developers. Many of them need a bigger picture to frame their work and a goal or a dream to believe.

Back at Sprintpcs.com, my boss had handed me down his own localized dream for a future for the company. He called it the Mobile Portal and my team which contained some big brains, turned it into a vision for a platform for mobile web services - an operating system for the Internet. But all of that came later. In the first instance I had to lead and motivate a team with a portfolio of lack luster projects which had been going nowhere - bogged down in requirements paralysis. So I tied the dream to a tangible sense of identity and branded it with an identity and word mark devised for me by a friend in the User Experience team. The symbol signifies the "knowledge of the world in the palm of your hand" - though some critics said it looks like a broken propeller. As an identity symbol, it is not world beating. It doesn't work well in inverse colors, it's too complicated and doesn't work so well in black and white. Compare it with the agileman logo for this site.

To be sure, identity marks within a larger organization are divisive. And I admit, I was rallying the team initially around internal rivalry. We were determined to show the IT division that we could develop software faster, better and cheaper than they could. I cast the bogeyman as another part of the company. Later that would change and the enemy would be portal companies - AOL, Microsoft, Yahoo!. Meanwhile, we started a trend with our identity mark and other groups in the company began to have logos too. About a year later, the Marketing Communications unit banned internal identity marks as divisive - because they were!

However, the team rallied around the vision for Mobile Portal and they rallied around the change to Feature Driven Development and they identified with the team logo. They printed their names on cards with the logo on it and placed them on their cubes for all to see. They were learning a new professionalism and learning that projects could be delivered on-time and on-budget with the agreed function. We even delivered one project without defects. For all of us it was a crucible experience - a coming of age as engineers. For me personally it was a huge roller-coaster ride but the team were largely sheltered from that. It's a tribute to how good it feels to work on a great agile team, and how good it feels to work with colleagues that you know well and have come to trust through the experience of delivering for each other, that the team, more than 2 years after I left still feels that sense of identity and sense of belonging. As the months pass, they are breaking apart - set asunder by corporate reorganizations and better job offers outside. But they still want to be together and as a result they started their own Yahoo! group - an exclusive club just for the 30 or so of them to keep in touch as the years pass - and they still identify with their very own mark. How cool is that?

|

A good post yesterday by Keith Ray, pointing at other writing by Johanna Rothman, about the role of testers on agile projects. What I like best about the trend towards test first and automated testing is the elevation of testing as a profession. Now testers are developers just like the product developers. They need the same skill set and they often get paid on the same pay scale. This blunts the snooty high nosed view that some developers hold towards testers. I even worked for guy a year or two ago who believed that no one really wanted to be a tester and that is why we had high turnover in our test department. He said, "All testers are wannabe developers". How sad is that?

There is another hidden benefit to having real development skills on the QA team - it makes developers and testers interchangeable. This is fantastic for dealing with bottlenecks in a process and providing a flexible workforce which can respond to change. Agile test methods lead to more agile development organizations and that's great!

|

Recently, I've had some mail about FDD and specifically how my description of it is "in conflict" with the Palmer and Felsing book which much closer reflects the original recipe from the Modeling in Color book. I wouldn't characterize the differences as "conflict" but it is true that there are differences. I think this is inevitable. Why should something stay the same over a 6 year period? Does that imply it was "finished" and "perfect"? Personally, I would expect things to change and I'd be suspicious if they didn't.

Over the years, at Sprint and Motorola I tried a few changes to FDD, or my team tried them for me. One of these changes made it into my book - Design By Feature Set. It is now time to "out" this change as problematic and no longer recommended. Yes! That means the book is wrong and will need changed in a future revision. It also means that in this instance, the original recipe is still best.

I have the data. Unfortunately, I can't share it with you - my readers - but it is convincing. The team using the Design by Feature Set approach described in Chapter 23 is definitely producing larger batch sizes, leading to longer lead times, with poorer quality and a slower overall productivity than another team doing similar work who are sticking to the original recipe of designing the Features in single Chief Programmer Work Packages which are batches of Features that can be completed in 2 weeks or less - a lead time of 10 working days or less. There seems to be little upside to the Design By Feature Set idea. It doesn't reduce refactoring - as far as I can see. It doesn't lead to a better architecture. It doesn't improve communication and it doesn't serve to improve planning. On the other hand, it does increase work-in-process and consequently the complexity being held in the minds of the developers. It does lengthen the lead time on design and this seems to have a direct effect on quality. It also seems to encourage reduced team working. Because the batch size is larger, their is a tendency to carve it up into pieces and have individual developers design on their own.

The original 2 weeks or less "size" for a Chief Programmer Work Package was chosen for psychological reasons - primarily that developers deserved to have the endorphin high from being "complete" at least once every two weeks. Agile management is so heavily influenced psychology. I'll give Jeff De Luca the final word on this subject. His very own Pareto Principle...

"Software development is 20% technology and 80% psychology!", Jeff De Luca

|

I met with a company this week where they really get the transparency religion. They have a project knowledge base built from Bugzilla code and a source code repository in CVS. They allow their clients to have password controlled access to all of this. The client can see the source code. There is total transparency. The company provides access to CVS and the client can pull down newly built code every week. There is a conference call with the programmers to discuss "what's new this week". It truly is eXtreme Programming at its best - and its verified CMM Level 2 compliant just for good measure.

What impressed me most with the management was that they also understood that they were providing transparent access into the data ocean of their projects. The chances that an executive from their client companies would be able to interpret all the data remote to non-existent. So transparency for them had somewhat reached its own limit to growth. I showed them some material from the book, particularly Chapter 14 Operations Review and we discussed how Feature Driven Development uses a roll up parking lot diagram and how TOC provides a way of distilling out the vital leveraging information such as buffer usage in a Critical Chain plan. I then showed how I had merged the parking lot with the buffer usage to provide a truly accurate objective parking lot with drill down into specific projects.

Eli Goldratt writing in The Haystack Syndrome described data as the input to a question and information as the answer to a question. If you are to provide useful transparency then you must provide information (transparently) which answers the questions in the viewers mind. Do you need to be a mind reader? Not quite! Just put yourself in their shoes and ask, what would I need to know about this project, if I were the CEO of the client?

|

In the interests of fair and balanced reporting, this website brings you the Denver Post review of Eli Goldratt's Viable Vision Tour...

The surly wonk swaggered in a white shirt, unbuttoned at the top. Once a soldier in the Israeli army, he raised his voice like a commander, his mouth on a wireless mike and his finger on the button of a slideshow projector. He began his points with maverick bravado and ended them with rhetorical demands for agreement: 'Is that understood? Yes or no?'

Many cried back a hearty 'yes,' proving the maxim that the more you pay consultants, the more you'll let them patronize you.

As for the comments about sales in Japan, I can confirm that Tokyo bookstores are carrying large volumes of Goldratt's work and I have both The Goal and It's not Luck in Japanese on my bookshelf. Most Tokyo bookstores have entire shelf sections dedicated to TOC and often a whole book case. There are many native Japanese writing books on TOC. So there is a mild truth to Eli's claims. However, better than Hari Potta? You have to be kidding! Have you checked out The Economist worldwide best sellers big book list recently? Harii Pottaa (in Japanese) is down from no.1 to no.9 this month.

|

Systems Thinking goes mainstream in the ITV show The Planman starring Robbie Coltrane. I guess it is better to be late to the party than never. Thanks to my Tivo looking after my cultural soul for recording the show, and to public service broadcaster KCTS for airing it, we in America get to see systems thinking in action in a mainstream TV drama just a year after our British friends. Coltrane's character is a devoted reader of The Art of Systems Thinking by O'Connor and McDermott. A text which was a big inspiration for me too. However, if you've seen my royalty checks, you'd know that I didn't use it to pull off a daylight robbery. Just one more thing - Glasgow and Edinburgh look great in this show. It reminds me how long it is since I've been back and actually walked around to see the changes.

|

Martin Fowler asks, should their be agile certification? and speculates that perhaps people wouldn't pay for it. This is a theme which Steve McConnell touched on in his book Professional Software Development where in Chapter 19 "Stinking Badges", he takes the idea further than certification and talks openly about licensing - ah hem just like real engineers.

The real issue with this is economics - the market currently does not demand it. It really requires some firms to be visibly beating out competitors through their use of more agile teams before the market will start to demand certification or licensing. I feel that the enabler to this is greater transparency and that has to be requested by the stockholders in the form of the market analysts who work for the firms which do the trading on their behalf. The Throughput Accounting measures I propose in Agile Management would be one way of doing this. It would measure the effectiveness of agile teams.

Then you need to compare the effectiveness against the competency and see if their is a relationship. Competency can only be measured by some form of certification. The prolonged certification that Martin Fowler suggests is required sounds more like an apprenticeship to me. Yes, agile development is a craft and perhaps a journeyman scale and a craftsman's certification model is what is required. Initially, this starts small because the number of experts and masters in the craft is small. Hence, the number of journeymen is small and the number of apprentices not quite so small. However, gradually the numbers would swell.

How does a craftsman report his/her status in the profession/craft? Perhaps like this...

David Anderson trained under Peter Coad and Jeff De Luca. Worked as journeyman at <insert employers> in the following <capacity>. Achieved his master craftsman status after delivering/building <project> or publishing <work>. Finally he achieved his expert status when invited to write and later publish "Agile Management for Software Engineering" in Peter Coad's series for Prentice Hall PTR - a work which extended the state of the art in software lifecycle management through application of the Theory of Constraints.

Everyone in the industry would have to produce a resume which looked like this and was somehow rooted in the core experts of agile (or some other branch of software engineering) as the original mentors. For example, my staff from Sprint in Kansas City would list that they trained under me and that I in turn had trained under Peter Coad and Jeff De Luca. Everyone would have a pedigree.

|

Eli Goldratt's writing on the Theory of Constraints contains a lot of use of the Socratic method - the style of teaching adopted by Socrates which uses questions and asks the pupil to devise the answers for themselves. Goldratt goes into some depth in What is this thing called the Theory of Constraints and how should it be implemented to explain why the Socratic method works - pages 16 through 20. However, I've learned the hard way that it is problematic and more recently I've learned that Jonahs (qualified practitioners of TOC) all over the World have experienced the same issues and the problem is well known and discussed. Well evidently not well enough known. So here I am writing it down in a place where Google will find it for the benefit of those who come after me.

First, the problem statement. Generally, Goldratt recommends the Socratic method when a systems thinker is trying to persuade a cause-effect thinker that a course of action is correct. The questions are designed to reveal the layers of depth in the system to be analyzed and allow the cause-effect thinker to "see" the system and realize the deeply obscured cause and effect despite time delays and derivative actions between cause and its effect. Peter Senge called this point of linkage between cause and effect "the leverage point" and seeing it can be difficult. However, my experience with the Socratic method is very negative. In systems thinking and the patterns movement, they say you have to see something 3 times before you cease to think of it as coincidence. And I've seen this problem enough now to realize it is real. So what is it?

Simply put the subject being questioned feels manipulated by the questioner. As layers of questions unveil layers of the system being analyzed, the subject becomes increasingly irritated and objects. They feel manipulated. They feel like the questioner is leading them down a path and that other paths are somehow being obscured from them. That it is all a trick. And most of all, they resent that the questioner already knows the answer.

When I first saw this, I put it down to West Coast, liberal, conspiracy theorist tendencies. However, I've recently heard that the problem is seen all over the World. At last week's Puget Sound TOC Learning Event, in Bellevue WA, coach Fran Fisher, who was an invited speaker, nailed the problem. Yes, the subject feels manipulated and the reason why is simple. They know that the questioner knows the answer. The conversation is not one between peers. When the questioner is a knowledgeable expert in a field and the subject an apprentice then the relationship will breakdown if the apprentice feels manipulated. She recommended that the relationship which works is a coaching relationship. In this case, the relationship is one of peers. The person being coached is an expert in their field. The person doing the coaching is only able to ask the right questions but does not know the answers. Indeed, this is truly a description of the relationship between Jonah and Alex Rogo in Eli Goldratt's novels. Jonah is not (or pretends very well not to be) an expert in Alex's field. He simply asks the right questions and forces Alex to come up with the answers for himself.

The revealing detail was finally offered from the back of the room, and I confess to forgetting who offered it. Apparently, Socrates used his method in two ways. The first was to ask of the subject a series of questions which led them to a conclusion, then to ask some more which led to a conflicting conclusion. Socrates would expose the internal inconsistency and show people that by their own logic, their proposal was invalid. He would also use the questioning method in collaboration with others when neither he nor his colleagues knew the answer. It was a journey of discovery for them all.

Hence, the advice seems to be, make careful use of the Socratic method, as it is considered dangerous when the inquisitor is an expert and the inquisition is designed to educate the subject of the inquisition. Better to use Socrates' method of questioning when no one knows the answer. Using it to flush out internal inconsistency risks embarrassing someone, so it is best used in that circumstance only in private. Using Socratic method as a teaching tool is to be avoided.

|