Wednesday, May 24, 2006

No Trust without Transparency

Is your project naked?

Is your agile development group hiding information about their work in progress? If you a project manager or customer, have you heard, “trust us we are doing agile?” If you are sitting with a project plan that says work will be done in 6 four week iterations and all you have is a loose idea of what is planned for each iteration, how do you manage risk? Traditionally, you’d ask for a more detailed plan. You’d push for predictable big planning upfront. However, the reality is that the team probably can’t make a more detailed plan due to uncertainties. So you do you manage risk?

If you are in the development team, it isn’t enough to say, “We have an iteration plan and we’ll deliver working code at the end of the iteration, trust us!” Trust isn’t enough. If a project plan schedules 6 months in 6 four week iterations, then it isn’t enough to give the project manager only six data points to show progress. Some transparency in to the progress within the iteration is required and it is essential that this data is shared with the project manager and the customer. What is being reported out must be the same data and information being used to run the project internally. At Microsoft we call this concept “Trustworthy transparency.”

Transparency is the antidote to requests for big planning upfront. Transparency offers project managers the ability to do risk management through the project life time. If you aren’t reporting cumulative flow, or burn down or burn up, or (gulp) earned value, every day to every stakeholder then you aren’t embracing the core foundation of agile development techniques - high trust.

Try running a naked project. You’ll find it liberating. Technorati tag: Agile, David+Anderson

What I learned this past week

So I’m back in Seattle after traveling to Orlando for VSLive! and Washington D.C. for IRMA. I’ve also been visiting a customer where I’m helping them with a long term case study for VSTS and MSF. I’ve got a single takeaway from these engagements - work item type definition is hard - really HARD! Walking through a workflow, optimizing it and transcribing it in to a state model for encoding in to a Team Foundation Server work item type is hard. I need to think more about this and publish some guidance. For now here a brief description of how I do it…

(1) I start by asking the business owners to sketch the flow of work using stick figures and little stacks of work to be processed. I get them to map the flow with arrows between stations and activities for the stick figures scribbled on to a white board

(2) I then make a statechart model in Visio and create a state for each station in the flow - typically identified by a stick figure performing an activity. I then create a state for each queue in front of each station.

(3) I then map all the possible transitions between states. Remembering to think about return transitions. “Ooops, I didn’t mean to close that, I need to re-open it.”

(4) I then ask for all the reasons that each transition can happen. This activity usually flushes out a few missing transitions and even an occasional missing state. So I then rework steps 2 and 3 and complete the reasons

(5) I then transcribe the Visio statechart in to the WITD XML using an XML editor. It is a good idea to start with the WITDs that are shipped with MSF and edit them.

We’re not done at that point.

(6) Next we need to identify all the data fields required and a form layout for those fields. We need to decide which of the fields will be used in reports.

(7) Now using the XML editor we add these to the WITD file.

(8) Now we need to go through each transition and identify required fields that must be completed as a pre-requisite for that transition.

(9) Finally we edit the subtle aspects of the WITD file to include the required fields for specific transitions.

We’ll be posting some white papers on customization of MSF process templates very soon now and I will alert you when they are on-line. Technorati tag: Agile, David+Anderson, MSF, Vistual+Studio

Tuesday, May 16, 2006

Another MSF node in the Blogosphere

My colleague Sam Guckenheimer, Group Product Planner for Visual Studio and Team System, has just launched his blog. He also has a book out, Software Engineering With Visual Studio Team System with contributions from Juan Perez of Personify Design. I personally think that this book is a great addition to the agile literature even though it is billed as a software engineering book for Microsoft platform developers. I read the first full draft returning from Tokyo on April 10th 2005. I think Sam did a marvelous job with the book that took him almost three years to write.

Plus, check this out. I have a quote on the back cover along with my good process buddies Bill Curtis from Borland and Francis Delgado from Avanade. The back cover reads like a good group at an SEPG cocktail party. Cups up!

“This is first and foremost a book about software engineering. In discussing flash points such as planning, documentation, governance, auditability, and organization, Sam presents the case for both agile and more formal practices, as well as describing the optimal conditions for each. Even though the material is presented in the context of VSTS, the guidance is universal.”

-Dr. Bill Curtis, chief process officer, Borland Software Corporation

“Sam Guckenheimer ushers in the era of trustworthy transparency that will revolutionize the way we manage software development projects.”

-David J. Anderson, author of Agile Management for Software Engineering

“This book is an eye opener: a door to a new era of software engineering.”

-Francis T. Delgado, senior program manager, Avanade Technorati tag: Agile, David+Anderson, Bill+Curtis, VSTS, MSF, Sam+Guckenheimer, Visual+Studio

Thursday, May 11, 2006

Measuring Bike To Work Month

The month of May is “Bike to Work Month” in Seattle. This sponsored event tries to encourage people to take up their bikes and ride to work to alleviate traffic congestion in a city that is continuing to attract a lot of immigration. With firms like Microsoft hiring over 100 people every week, the roads only get more and more congested. This year Starbucks is sponsoring Bike to Work Day on May 19th that tries to encourage people to try it just once and see how it feels. The enticement is a free breakfast.

But this isn’t a post about biking. It’s a post about metrics. The goal for the metropolitan traffic authorities is to minimize congestion and maximize road user satisfaction. They do this primarily by promoting the use of public transport and ride sharing in higher occupancy vehicles - for the foreign audience, a high occupancy vehicle in the United States has two people in it :-O.

With Bike to Work Month cyclists are encourage to log miles commuted. This is a macho metric for the avid biker. So far I’ve ridden 6 days this month for a total of 136 miles. But gaming the metric is easy. On Friday night I rode home with my friend Diane and her friend Nate around the top of Lake Washington via the Sammamish River trail and the Burke-Gilman trail for a total of 31 miles. It took two hours to get home when the direct way over the SR-520 floating bridge takes me just over 1 hour. Miles ridden is not a good metric to judge whether biking is truly helping congestion.

In the WSDOT (Washington State Department of Transport) Commute Survey that Microsoft had me complete online at the start of the month, I was asked how many journey’s I’d replaced by biking and other combinations like how many times I rode the bus. Well the truth is that a bike ride replaces a drive or two bus rides - the no.48 and the no.545. But, to get over the SR-520 floating bridge I still need to put my bike on a bus. So one of the bus rides is not completely eliminated but I am less discriminating about the bus I ride - any bus will do so long as it is crossing the lake. SO number of journeys replaced - the officially captured metric - does not dirve the correct behavior either.

The real metric that drives customer satisfaction is journey time. How long did it take me to get from home to work or work to home? Though there are other parameters in the equation such as “riding is a workout and saves time going to the gym”, the journey time is the main one. In other walks of life we might generalize this journey time metric as Lead Time. I’ve written and talked before about how important Lead Time is as a metric to drive the correct behavior. However, it has a problem. It is a lagging indicator. It acts as a good report card but not as a good control or management metric.

What affects my lead time from home to work? If I’m biking it takes 22-27 minutes to get to the freeway bus stop. I then have to wait for a bus with a free bike rack spot. This can take from 1 minute to 25 minutes. The length of the line of waiting bikers has a big effect. In other words, it is the inventory of bikers at the bus stop that most affects my total journey time. The ride on the East-side of the lake to the Microsoft campus is again fairly low variation (though I do have a set of options for which stop to get off and the remaining ride varies accordingly.) On the way home I ride the whole way except for the lake crossing. Again, the inventory of waiting riders is the biggest factor in my journey time. With no inventory of riders waiting I am home in 1 hour and 5 minutes give or take a couple of minutes. On the other hand, I can wait 30 minutes for a bus with a free spot adding 50% to my journey time and making me dissatisfied with biking as an option. The other options - riding the bus: 90 minutes; driving my car: from 45 to 100 minutes - begin to look more attractive.

So here is my point. If you want to understand how flow of traffic and demand of commuters affects satisfaction with the whole system, you need to understand the inventory levels of waiting passengers at key points in the city, and the flow of traffic at those points, throughout the day and understand how it varies across the days of the week and across months and at different times of year. The key driver of correct management decisions would be a metric on queuing time and length of queues. And it is just this that is currently hard to gather because no one is spying at the bus stops. Hence, the data that is really needed to drive customer satisfaction isn’t being captured.

Are you listening Jim Benson, I just gave you a great new product idea for Grayhill Solutions!!!

And the moral of this story - measuring the queue of work and managing that queue is one of the most useful things you can do day-to-day to drive customer satisfaction because lead time is such a big part of customer satisfaction.

Wednesday, May 10, 2006

Adapting the Organization Structure to Change

More on organization structure following the thread I started yesterday. I probably won’t get this finished today and it probably won’t be totally coherent and thought through. It is after all a blog and not a serious article.

Yesterday, I talked about architecture and coupling and cohesion and object-orientation and distributed components and aspects and services, and how ideally we want a loosely coupled, highly cohesive organization mapped against our product architecture. This provides an encapsulation that is empowering to a team and leads to high productivity with low overhead. In Lean terms - less waste. However, the problem is that we don’t do aspect-oriented organizations very well. Anything that has a cross-cutting concern tends to get bogged down in politics and negotiation over who owns it.

I’m not talking about matrixed organizations. We have that fairly set already. At Microsoft we have Program Managers (in the rest of the world you’d call us Project Managers) and resources from functional managers are matrixed to the PM to do projects - Features in products. This works nicely. The Feature Team is hopefully cohesive and loosely coupled through the PM as the communication point. But it doesn’t work well when the Feature is cross-cutting of others. This leads to a lot of extra communication - waste - and slows things down.

There is a second problem mapping architecture to organization. Architecture has uncertainties at the start of a project just at the time when you want to lock the organizational structure. Furthermore, we tend to do architecture top-down and this usually means that we get it wrong first time and several iterations are needed before it settles down. This leaves us with an organization that is mismatched to the underlying architecture of the product with the resultant overhead of extra communication, uncertainty, several teams accessing the same code and a general slow down in productivity. While developing Team System v1.0 we saw just these kinds of problems. The MSF team was moved around a bit and parts of the process template and work item tracking were also moved as Rick La Plante looked for the best organizational structure to match the emerging architecture of the product.

However, there is a better way. Bottom-up architecture driven off a domain model has been shown to facilitate late definition of the component or service boundaries. This postponement of decision making fits with the Lean idea of making decisions at the last responsible moment. Bottom-up architecture solves the problems of top-down architecture and the rework and confusion it can cause.

A few years ago, I worked on a project at another company. I was leading the team in Seattle. The architecture was being done in Mountain View, CA. While the other development teams were in Beijing, Bangalore and St. Petersburg. The architects analyzed the system and then divided the work out between the four teams. What happened next? The teams spent months holding weekly global teleconferences to resolve issues with the interfaces between components and the roles and responsibilities between the teams. This led to poor productivity and lots of stress and confusion. A direct result of poor top-down architecture and something that could have been solved by bottom-up domain-driven architecture.

So here are the challenges I see still in organizational structure. How do you organize for the early lifecycle while you are building the domain model and the deciding the architectural separation of concerns? How do you allow for postponed, last responsible moment architectural decisions and reconcile that with optimal, loosely-coupled, high cohesive organization design? How do you organize for cross cutting concerns? What does an aspect-oriented organization look like? Technorati tag: Agile, David+Anderson, Lean