I wasn't surprised by the theme in this interview with Steve Ballmer. Recently posters proclaiming Good to Great have appeared around the campus. The top management are clearly Jim Collins fans. They've constructed the company's new vision statement with its mission, values, and core tenets. The new mission is to help companies and people around the world to realize their potential. No difficulty then for me in identifying my day-to-day work with the company's mission. My job is to help software developers and their management around the world better realize their potential by building better software, faster, and cheaper. For me, it is as Steve says, "galvanizing". Unlike the tone in the article, I don't see it as a problem. It's part of the reason I joined Microsoft.

It's not surprising that the founders of Microsoft have turned their attention to leaving a long term legacy. It'll be interesting for all of us to follow this over the next twenty years. Is it possible to imagine a Microsoft without Bill and Steve? I'd love to speculate but I've decided to keep those thoughts to myself.

As promised yesterday, another example of bad leadership, this time from the most recent episode of The Apprentice (now in the third season). In this episode, Verna walked out. Yip! She packed her bags and walked off early on the 2nd morning. It required Carolyn to show what a true leader would do under these circumstances. She got in her car and went to look for her.

There's a lesson that Michael could have learned from a former military leader like last season's winner, Kelly - you never leave anyone behind. Where was Michael after Verna disappeared?

Something which led to Verna's departure was her lack of sleep. In fact, both teams seemed to pass on sleeping at all in order to maximize their input. The leaders were managing hours of effort rather than effectiveness. Something else military people understand is that some sleep is essential. When a platoon sleeps, two soldiers keep watch. After an hour or two, they wake colleagues and swap. In a critical situation, more stay awake, but the platoon commander always ensures that every team member gets a couple of hours of sleep. We seldom, if ever see this on The Apprentice.

And finally, a theme which comes up again and again throughout the three seasons - food. Every good military manager knows that an army marches on its stomach. It's a key part of management and leadership to insure that workers are well fed. If a manager ask staff to work overtime, then the manager shows up and makes the tea, fetches the pizza or whatever it takes to keep people productive. Again and again on this show, we see project managers neglect to insure that their team get fed.

These are such basic mistakes. Every manager should adopt the three golden rules we can learn from this...

• No one gets left behind. It's very bad for morale. When the team sees that management will abandon a weak member it makes them feel insecure. Deming said that we should "drive out fear". You can't eliminate fear if you show that you'll leave someone behind.
• Everyone gets some sleep. People start to make mistakes very quickly when they are tired. Mistakes cause trouble and agitate others. It becomes a vicious cycle as tired people irritate each other more and more. More mistakes get made. More time is wasted. Focus on the task is lost.
• Everyone gets to eat a sufficiency. Without food people don't think straight or act rationally. Mistakes get made. This irritates others on the team and undermines trust amongst the team members. They see others at their worst and they react to it. A team can fracture into a group of individuals because trust breaks down when mistakes are made due to basic elements at the top of Mazlov's hierarchy.

Today's blog entry was inspired by my wife, whilst we watched an episode of The Apprentice last season. Elizabeth was having yet another crying scene and my wife blurted out, "Ahhh, Elizabeth is just such a woman!" I almost choked on my fruit tea. So Elizabeth was showing her emotional and sensitive side - I guess these are attributes more commonly found in women. Of all the people I know, perhaps only my home-maker wife can get away with a comment like this. It's certainly not something an American manager can explicitly express in the workplace. And it made me think. Although I don't see such sentiment expressed explicitly, I do see managers complain about behavior in men and women which could be described as merely "human." Just as in the show with Elizabeth, these expressions of humanity are often looked upon negatively - as though it is a failing. So to pick on the specific example - do we really want all women in the knowledge workplace to be hard, unemotional and thick skinned? Do we want them to be able to take everything in their stride - all business, totally focused. Personally, I find this idea a bit frightening.

People come in a whole range of varieties and managers have to understand that. Some don't cope well with change, or uncertainty, or lack of direction. People feel pressure in different ways - worry about things which often aren't worth worrying about. It's the manager's job to understand and assign them roles and responsibilities appropriately. Furthermore, it's the manager's responsibility to control chaos, bring stability to an organization and to act rationally in the face of variation - to understand what is normal variation and what represents exceptional circumstances that require intervention. Stability helps staff to function better regardless of their emotional or cultural disposition. In the event that one of the team does have a breakdown, it's the manager's job to fix it.

The implication that someone "can't handle it" and there is "no room on the team for someone who can't pull themselves together" is really a manager treating the symptom - taking the easy way out. It's easier to say, "I don't want this person around." The harder job is to look for the root cause and deal with that. Take away the circumstances which caused the emotional breakdown so that it doesn't happen again. It was a lack of leadership which failed Elizabeth and the nature of this game show which has one person eliminated each week didn't help. Elizabeth identified as a weak link quickly became an outcast - someone who could be sacrificed for the survival of the others. This made the situation worse. It became a vicious cycle as she felt more and more isolated.

This is where a show like The Apprentice helps us all by highlighting some bad examples of poor leadership. More tomorrow...

In manufacturing quality assurance, they measure something called defects per opportunity. It is always a number less than 1. For rivet you pop, it's either a good rivet or its not. Terms like Six Sigma meaning less than 4 defects per million opportunities are born out of this paradigm. However, in software we have a bigger problem. In software, defects per opportunity can be greater than one. Even using the SEI's measurement of defects per line of code, it is possible to insert more than one defect per line - unless you are working in machine code.

It's for this reason that reducing defects, eliminating the opportunity for them, pays such a high dividend in software development. In some of my recent presentations, I talk about reducing defect rates from 3:1 to 2:100 and I'm not making this data up. It is real data from real projects. I did see a team inserting around 3 defects per feature and another team sitting on the same floor deliver a 7 week iteration with as few as 2 escaped defects per 100 Features. This represents a staggering improvement. Capers Jones has data that suggests the poorest of the poor insert about 5 defects per function point. If we take this as equivalent of 1:1 in manufacturing then 2 per 100 is equivalent to 3 sigma or 3 per thousand in manufacturing.

Now these numbers are still nowhere close to some other human activities such as landing airliners - around 7 sigma. However, considering software development is design work where no two features (or function points) are really the same, then it's remarkably good.

Now let's assume that a QA department was as good at catching escaped defects through integration, system and product testing, then we could assume that only 2 escaped defects per hundred would actually escape to the customer. That's only 4 for every 10,000 features. Using my manufacturing equivalency its about 49 per million or considerably better than 5 sigma.

Hmmm...

What's your personal defect rate?

Do you ever stop to think about how many opportunities there are for defect insertion in knowledge work and do you measure how accurate your performance intent against action actually is? Do you study, even subjectively how this fluctuates with tiredness, distraction and at different times of the day?

Recently, I was in a meeting in my office with my graphic designer. I offered her a drink which she accepted. I said, I'd go to the kitchen as I wanted a cup of tea and I'd fetch the soda for her too. While I was at it, I would also photocopy the design specs that I'd drafted and give her a copy. So I set off down the corridor. Soon I realized that I'd forgotten my insulated mug. Oh well, I thought, no tea for me - lowering my standards to maintain conformance with specification for the trip ;-) Next I walked straight past the copy room and into the gents toilet and was surprised to find that there wasn't a photocopy machine in there. 180 degree turn and in to the copy room. I made a copy. Went to the kitchen. Collected the drink for my colleague and returned to my office - only to realize that I'd left the master of the design spec in the photocopy room.

In the space of two minutes I had inserted 3 defects into my day. It was only 3.30pm and it was only Wednesday. So no excuse for being tired from a long week at work. Now it might just be me! I might be so totally absent minded that I shouldn't be trusted with my own house keys. However, I fear that I am not unique. How many day dreamers do we have in high technology?

And people wonder why pair programming is effective?

...An industry turns its lonely eyes to you (woo woo woo)

It took a pack of lawyers for Simon and Garfunkel to persuade Joe Dimaggio that they weren't criticizing him or suggesting that he'd disappeared from public sight or that the shine had come off the public image of a man who married Marilyn Monroe. In fact, as the lawyers explained, Joe had to get used to the idea that he was being used as a metaphor.

Recently, Martin Fowler has been offering us advice about how dangerous metaphors can be. They are confusing. The problem is two fold. Firstly, they don't stretch well (as Alan Cooper famously explained) and secondly not everyone can see the abstraction and subsequent concrete example in another field. Joe Dimaggio was a metaphor for the all-American hero. Here Martin offers us a warning about Lean as the all-Agile productivity hero.

Where I want to differ with Martin's view is in his characterization of Lean as a metaphor. Lean Thinking offers us a set of principles. Principles are abstract concepts rather than concrete ones. The all-American hero is an abstract concept. Joe Dimaggio on the other hand was a decidedly real example of a baseball player. As I explained before, the English language offers us 3 words for mapping concepts like this, which from strongest to weakest are: abstraction; analogy; and metaphor. Lean is an abstraction rather than a metaphor. Martin Fowler made his name teaching us about abstraction in his seminal work - Analysis Patterns .

At the heart of Martin's complaint with the Poppendiecks promotion of Lean is the suggestion that "inventory" in software development manifests itself as documentation. As zero (or very low) inventory is a goal in manufacturing then zero documentation should be a goal in software development. Hmmm. Sounds metaphorical to me. So, let's examine this. First off, inventory is an abstract concept which has a specific example in Lean Manufacturing confusingly often called inventory or goods-on-hand. Inventory comes in three forms: raw material, work-in-progress; and finished goods. All three of these terms can apply to the abstract or concrete. So, it's tricky! The Poppendiecks thinking in metaphors (rather than abstractions) said, "Gee! Inventory in a factory is a pile of stuff. What looks like a pile of stuff in software development? Hmmm. How about a stack of documents?" It didn't take long for a few of us to correct this notion and I notice that recently in the Lean Development Yahoo! group no one has been suggesting that "inventory is documentation". So Martin is behind the times though doubtless countless others who've read some of the Poppendieck articles or book will have learned the same notion - that "inventort" is documentation. Both Clarke and Jason explain why the Lean abstraction does work and how to truly represent inventory in a software development process. All of this begs the question from the title...

Where have you gone Martin Fowler (international man of abstraction)? an industry still turns its lonely eyes to you...

Martin Fowler blogs about a recent workshop where people were asked to rank principles for layered architectures. Interestingly, the concept of organizing teams around layers in the architecture emerges as an anti-pattern. I've heard this before often from people who should know better. I've also watched a colleague (back in 1999) - Peter Durcansky (some of you will know him as the developer of Playground, the Coad-Yourdon Notation modeling tool) - who had to rescue a project which went wrong mostly for this very reason - teams were organized to code in vertical slices. The code was poorly coupled and lacked cohesion.

Many agile developers believe that vertical slices are best. In fact the standard advice of "pick a pair, pick a story, TDD-it, and deliver it working" suggests vertical development across layers in the architecture. So why, when I generally believe in agile approaches, do I believe this is an anti-pattern?

Firstly, I do admit it works on smaller projects and for small tasks such as change requests. At 4thpass/Motorola in Seattle we did assign individuals or teams to do vertically partitioned development for change requests and minor upgrades. However, I do believe it is an anti-pattern on larger projects with more than one team (i.e. more than 6 people) on a project intended to run for quarters or years where the software will grow into a large code-base. Why? There are two reasons.

The first is rooted in a pattern identified as good in the ranking, "low coupling between layers, high cohesion within layers." Equally, it's good organizational skill to create teams which are highly cohesive and loosely coupled. Hence, if the teams doing the work are highly cohesive and loosely coupled, and the layers are highly cohesive and loosely coupled, shouldn't these be aligned? If the team spans the layers then it encourages poor cross layer cohesion (between code from different teams) and poor coupling between layers (as teams won't feel compelled to enforce loose coupling within their own code). [XP with its refactoring and shared code ownership allows an anyone goes anywhere policy and this in theory fixes the problem. However, with large code bases, teams tend not to wander all over the code base, refactoring other teams' code. So the problem remains.] Code cohesion will suffer when the organization and the architecture do not align. This is yet further evidence that to be an effective manager of software development, you must understand how code is built.

The second reason relates to scheduling and the uncertainty or variation implied by another principle agreed as good in Fowler's survey, namely "separation of concerns". The reason this makes sense is that it allows things to vary independently and the degree of variation, change and uncertainty will be different for each class or component. We "separate concerns" because we know they vary independently. There is another advantage to understanding and separating out variation. It lies in Critical Chain theory. The size of a buffer can be smaller when the uncertainty is smaller. Hence, in a project schedule for a network of components being developed in parallel by more than one team, the schedule can be shorter if the uncertainty is isolated effectively. Reinertsen explained, in Managing the Design Factory, that coupled tasks assume the uncertainty of the greatest element. Hence, cross-layer tasking maximizes the effect of uncertainty. Separating teams across layers, having them work only on code in that layer, facilitates minimal uncertainty. You achieve minimal uncertainty with a schedule that has tasks for each layer in the system, as shown here.

It is certainly true that an immature team cannot run with a Critical Chain schedule. The concept of tightly synchronous iterations such as Scrum Sprints works much better. However, once you've controlled chaos, the next step is to look for continuous improvement. One way to achieve better throughput is to switch to asynchronous scheduling and reduce buffering by separating out differing degrees of variation in the architecture and the task network for the schedule. Hence, small teams doing vertical slices is probably the right answer for chaotic teams on smaller projects. Longer term, the most productive answer is to separate out teams across layers.

Line managers complain to me that they are never sure whether or not they are chickens or pigs. Should they even be in the morning standup meeting? I've vacillated on this before too. My opinion is that when everything is running well then the line manager is pretty much superfluous unless they also play another role such as architect or domain expert. However, under chaotic conditions, the line manager is a useful member of the scrum. So, is a line manager a chicken or a pig?

Line managers are seldom the scrum master. Scrum masters usually report to line managers, in my experience. So in theory the line manager is a chicken - merely involved, someone who receives notification, output and occasionally is consulted for input. However, when a special cause event happens and an escalation is needed in order to remove a blockage and get the backlog moving again, then suddenly the line manager is a pig - committed, doing the work, both accountable and responsible.

Hence, line managers live in a world of oscillation between two states. Never quite stable as a chicken or a pig. They are chiglets! If we're not happy with the idea that a chiglet isn't permanently committed but sometimes merely involved then I suggest we designate the role and responsibility relationship of the chiglet as "facilitates".

Hence, a agile ARCI model would have three designations

• F - facilitates
• C - committed
• I - involved

Ken Schwaber introduced the concept of chickens and pigs with the daily scrum. Each attendee is either a chicken or a pig. The metaphor comes from the idea of a project to make breakfast of bacon and eggs. In such a project, the pig is said to be committed but the chicken is merely involved.

Committed and involved could be mapped to the ARCI designations for roles and responsibilities. This might help to map an agile process into a more formal or traditional organization.

• Committed (pig) is the equivalent of A(ccountable) and R(esponsible)
• Involved (chicken) is the equivalent of C(onsults) and I(nformed)

This CI or pig and chicken designation removes the problem of command and control by merging the doing, with the giving of orders. Committed designation allows for empowerment and its alter-ego self-organization. Committed allows for shared responsibility (in the common sense) and shared ownership. Everyone on the team, doing the work is committed and they are all jointly accountable.

RACI is an acronym used in formal software engineering processes to designate roles against responsibilities. The letters mean

• R - Responsible (does the work)
• A - Accountable (blamed when the work is not completed satisfactorily)
• C - Consults (provides input)
• I - Informed (receives the output or a facsimile or summary thereof)

The problem with RACI is that it sounds nice but it warps the focus. It ought to be ARCI - perhaps not so English language friendly but it more accurately puts "accountability" first on the list. Too often I see process specifications where the author (in a hurry) omits the A and specifies only who is responsible to do the work - fine you might think. Well the problem with this is that in a command and control world, if there is no one assigned to be accountable and give out the orders then the work will never get started.

ARCI is firmly a facet of the command and control management style and it belongs in the traditional software engineering world. But what might an agile ARCI look like? More tomorrow...

There has been a growing interest in FDD recently. Partly because Jeff De Luca is now offering certification, partly because there are now at least two knowledge base tools on the market which support it, FDD Tracker and FDD Manager, and partly because there is a growing body of corporate IT departments taking a look at agile methods but they want to have a firm grip on architecture and aren't prepared to go the whole extreme way to emergent design.

As a result of this growing demand, I've been asked to make it easier to find FDD material on this site. Apparently some other FDD sites aren't very easy to browse and discover newer material which isn't in the Palmer & Flesing book. So in response to this, I've created Channel FDD. You will find it on the RHS navigation of the home page and on any FDD article page. There are over 30 papers and blog entries featured. If you have any more suggestions as to how I can make this site more useful to FDDers around the world, please comment...

This past fall season has seen a TV ratings war between the King of Manhattan real estate, Donald Trump, and the mildly eccentric (superficially) Sir Richard Branson. Trump had a second season of his show The Apprentice whilst Branson launched his copycat show Rebel Billionaire.

What was most interesting to watch was the very obvious difference in styles. Trump seems to breed confrontation and his organization is very hierarchical.A hierarchy reinforced with the trappings of power. The final two contestants got to boss around former colleagues whilst they were driven in a chauffeured Maybach limousine and their lackies followed behind in a minivan. Trump dismisses the losers with his punch line "You're Fired!" The entire season was filled with bickering, Machiavellian intrigue and outright bitchiness amongst the colleagues - particularly the women.

Meanwhile, Branson's style is collaborative. He allows the losing team to debate who should be up for his "elimination challenge". Branson encourages consensus whilst Trump encourages finger pointing and dissent. Whilst Trump looks for loyalty to the defeated leader, Branson looks for objectivity in analysis of the defeat. Branson then has the losing elimination pair challenge each other. In the case of an outright loser then he doesn't have to fire anyone - they self-selected such as the guy who fell asleep during the night whilst camping out in the savannah of Africa. An emotional Branson hugged the guy as he handed him his ticket home on the tarmac before they boarded their next flight. He was evidently sorry to lose such a strong candidate.

The final dismissal with Branson is again non-confrontational - the uncharitable might call it passive aggressive. He simply confronts the two losers on the tarmac at the steps to the plane and hands them a ticket each. One ticket allows someone to board whilst the other is sent on a different flight home to the USA.

One final key difference is that Branson never asks his elimination contestants to do anything he wouldn't do himself and he often joins them. With Branson its all casual clothing, breaking bread around the table, hugs and emotional support. It wouldn't be a stretch to imagine his organization is much flatter than Trump's and his senior managers don't enjoy the trappings of power because he recognizes that the only power they wield is the power to influence through respect.

When I'm watching The Apprentice, I can't help but remind myself that it is entertainment and that much of it is setup for the viewer. It isn't real. It isn't reality. It's fake! However, I feel that the differences in style between Trump and Branson are real and really are reflected in the nature of the two game shows. I have made up my mind who I'd prefer to work for. Have you?

How do you change the encumbant management paradigm in an industry? How do you generate economic improvement from chaos? Answer: one manager at a time.

So, it always gives me great pleasure to read blogs like this one - where the reader clearly took the main message from Agile Management without having to trudge through all 330 pages. We all know that most people don't have time to read in-depth cover-to-cover, so it reassures me that at least one reader managed to internalize my message - especially when I spent so long figuring out how to articulate it.

I started to wonder how this could be done in my current team. Actually I started to wonder if we were already doing it. It occurred to me that we have a fixed number of people, split into broad categories of analyst, developer and tester. If analysis is our constraint, then by the TOC I should reduce the flow of work to accomodate that. The problem is - what do I do with my developers and testers? If I just leave them idle then I'm realising no benefit, since I have to pay them the same amount anyway (fixed cost). I'm assuming that I don't have the capacity to loan them to other projects (partly 'cos I'd never get them back, and partly 'cos of the return ramp-up cost). Then it dawned on me that if I had developers who could act as analysts, or somehow shift some of the analysis work to the developers - then I'd be able to speed up and keep everyone busy.

Recently Joel has expressed his opinions about the efficacy of Microsoft Solution Framework and started a long thread of debate on his site. [This is slightly old news but I'm just back from an emotional zone-out]. Amongst the many replies, I particularly liked this one.

Joel makes a good basic point. If you believe the spin that accompanied MSF v3.0 then you'd believe that it captures the Microsoft development process and as Joel points out, Microsoft has a unique culture and unique position in the World. It hires really great people, empowers them, makes everyone individually accountable for their own actions, arms them with lots of resources and then allows the results to emerge with a very hands off management style. It's the combination of the culture, the leadership and the above average quality of the people that make the recipe work. In Joel's opinion - as a former Microsoft employee who now runs his own firm - that recipe isn't easily scaled across the industry. And he'd be right! Though there is a lot of value that companies can derived from trying to emulate the Microsoft culture of leadership, empowerment, trust and accountability, without a similar culture Microsoft's processes are unlikely to succeed elsewhere.

What Joel didn't realize when he wrote his piece is that MSF v4.0 is being written by me and Randy Miller - co-author of A Practical Guide to Extreme Programming and Advanced Use Case Modeling. Randy used to work at Togethersoft for Peter Coad and between us, we have a bit of a clue about software process and what it takes to succeed in a less egalitarian environment than the Redmond campus. MSF v4.0 is actually a tooling framework enabled in the Visual Studio Team System product. It can be configured to deliver any process. In fact, we've already seen a version of Scrum built on the early CTP release of VSTS. Out of the box, VSTS will ship with two processes - MSF Agile - and it's bigger brother to be named later that will be CMMI level 3 compliant. A stretch-to-fit agile process which will meet most of the process areas in CMMI Level 2 and 3. [Note: CMMI v1.1 not SW-CMM which the SEI has recently deprecated!!!]

I'll be talking a lot more about MSF v4.0 and Microsoft's entry into the agile methodology space throughout this year.

Sometimes, objectivity, transparency and effective up-management will have no effect. Why? Because senior management chooses to squeeze the line management to temporarily compensate for its own failure.

Recently, a friend of mine has quit his second job within 12 months. Why? Because "I'm sick of being set up for failure" and "it was the same old b***s*** over the schedule." In short, a manager being squeezed.

While I was back in Scotland in October, I had dinner with an old college buddy and his wife who had just quit her job in the airline business as a no.1 flight attendant - the cabin crew line management position. Why? Because she was being asked to sacrifice quality of service (i.e. her and her staff's pride in their job) and staff break times in order to sell more merchandize. In short, the business model of her employer was broken. They couldn't make enough money from selling airline tickets so they had to try and plug the gap by spending more and more staff time pushing goods on to a captive market. The ever increasing retail sales targets meant that on shorter flights, service and staff breaks had to be sacrificed or the targets had to be missed. A line manager wishing to keep her job had to make herself unpopular with her staff and her customers. Of course, the longer term effect is to damage the brand and reputation of the airline - but what the heck, the executive management will have retired by then, but line managers will still be trying to pay off their mortgage doing the same old job.

The line manager squeeze - setting the line manager an impossible task - is another example of management misdirection. The senior management conveniently divert attention and deflect blame and criticism on to the junior line management. It's yet another example of why the line management job isn't to be envied. No wonder so many senior techies prefer to stick to architecture and development and don't volunteer for leadership training and management positions.

I'm hoping to get this blog back on the rails this month. I know a few people have been missing it and wondering whether I gave up. The truth is this past quarter has been challenging for me personally. My father passed away on the day I spoke at the TOCICO conference in Miami. Since then I've had to sell the family home in Scotland and overcome a backlog of work at my day job after 1 month on bereavement leave. This past week, my wife gave birth to our second child - Natalie - and we'll be moving soon to a bigger house, that we just agreed to purchase. So I've been busy and focused on family rather than career this past while. Now it's time to get back to work.

For 2005, I'm intending to write another book. I've actually got three projects on the go. One or two of them will turn into books over the next year or so. This year will see the launch of Microsoft Visual Studio Team System and the new MSF v4.0 processes (including the CMMI Level 3 version) for which I'm responsible. I'll be doing a book about managing .NET projects using VSTS and its reporting mechanisms which include cumulative flow diagrams as an out-of-the-box report. So VSTS enables "agile management". I've also got a book on color modeling in my sights. It's time to bring back The Coad Method and take it to the next level. The third book is a much longer term project. More on that another time.