CNN Money has an article about Outsource World Expo in New York.... 

"Workers, from software developers to system administrators to engineers, are very frightened about what this trend means about the future of the industry and the future of their jobs," Joshua Sperry, an organizer with the Communications Workers of America, said this week in a press release announcing a protest of offshore outsourcing.

Hal Macomber is discussing good and bad variation. Variation is getting attention from a some other bloggers too. Why is variation important? Because the more of it you have the more it costs to get work completed.

If project promises are to be met and software delivered on-time, then schedules must be buffered against common cause variation. This requires an understanding of the performance of a team in the past, and a prediction that their performance will be similar in the future. For example, if historically it has taken 2.2 days +/- 1.0 days to complete Features of complexity rating 3, then it is likely to take a similar time in future.

To be sure of delivery a manager will need to buffer 1.0 days over the mean 2.2 days for such a Feature. A whole project of similar Features would require a buffer determined from the square root of the sum of the squares of those 2.0 individual buffers. For 50 Features, the required project buffer is 7 days. The total project length is 110 days + 7 days = 117 days.

If the manager improves the team skill level with mentored training in analysis, then variation may fall. Say it fell to only +/- 0.4 days. The result would be a requirement for a buffer of only 3 days for 50 Features. Saving 4 days of effort.

Reducing variation, through better, more repeatable analysis of software deliverables, saves money by reducing the time to deliver projects.

Introducing a new policy can move the system capacity constrained resource (CCR) and create problems throughout an organization. These problems are foreseeable when a manager understands the effect of policies on the CCR. Policies are a type of constraint. The introduction of a new policy might make it the overall system constraint.

For example, let us imagine that the chief architect is becoming uncomfortable with the quality of emerging code and feels that the original architectural intent is not being achieved. With a simple announcement, he introduces a new policy that all designs must be reviewed by him before coding commences. In an instant, the chief architect is now the overall system constraint. A one-man bottleneck. It doesn't matter where the constraint was before - UI design, Testing, Usability, Requirements or Coding - the new policy moved the constraint.

What are the effects of moving the constraint? People downstream in the process find themselves idle for periods. Work upstream begins to stock pile. People producing designs can't get to "complete" without a long wait. The project schedule can be abandoned. It too needs to be re-written. The PMO (if there is one) needs to re-write the master schedule. Customers get frustrated and their expectations need to be managed.

When the CCR moves, everything changes. New policies can move the CCR. Changes in CCR should be planned and the overall system-wide effect understood in advance, otherwise chaos ensues, leading to accusations, blame and loss of trust. The introduction of new policies cannot be made locally. New policies should be agreed system-wide after being fully understood by the value-chain of line managers affected.

Steve Norrie links to this article [PDF] about the effect of 'net negative programmers'. I've known a few of these. They take two main forms - the overly technical, resume building, next big thing developer, and the spoiled child, my-work-is-my-art prima donna.

The first type slow everyone else down by using convoluted architectures or half-baked, new, high risk technologies which are unproven for the application scale and performance requirements. This can result in a need to train lots of engineers in a new technique. It increases the defect rate, and may result in a poor product which fails to meet customer expectations.

The second type just refuse to change, to grow, to learn more and to challenge themselves professionally. They aren't humble. In fact they are downright arrogant. They think they know it all and their performance is already good enough. They act as a "fifth column" and undermine managerial initiatives to create a culture of continuous improvement. These 'net negative programmers' are one man 'limits to growth' - individual constraints.

Net negative programmers can increase project risk, increase costs, increase defects and reduce throughput, profit and ROI for the whole team and business. It requires strong management to overcome this. It requires managers prepared to be unpopular and prepared to make the hard decisions. Sometimes, eliminating a team member who is otherwise well respected for their intellect may result in a net improvement in performance.

Frank Patrick has a really colorful post explaining Critical Chain scheduling and buffer usage. The 2 charts of what is essentially the same project with different amounts of buffer tell a very revealing story. Good project management is all about probability and learning to live with uncertainty. Deterministic dependency is where Critical Path techniques breakdown.

Translating to project management, the deterministic line is equivalent to completed work. Looking into the future, we move from points in time to ranges of time within which we can expect completion of a project. The further out we look -- the more work subject to uncertainty that remains -- the wider the range of promises needs to be.

Dell's CIO, Randy Mott really seems to understand the essence of Agile Management according to this article in Information Week.

For Dell's IT staff, the challenge is executing on that visionary ambition within intensely practical performance goals. Their five-year objectives include reducing IT spending to less than 1% of revenue and cutting the application development cycle to nine months or less. And to do that while delivering 100% of projects on time with a measurable bottom-line improvement.

CNN has picked up on a report by the New York Federal Reserve Bank that suggests that the recovery is jobless because there is a restructuring beginning to happen.

In a recent report, economists at the New York Fed suggest that what is happening is structural. In past recessions job losses were far more cyclical: The economy turned down, your company laid you off, but as soon as things got better you got hired back.

I've referred to this as the "silicon rust belt" effect. The idea that many knowledge worker jobs will go off-shore just as heavy industrial jobs moved to lower cost economies in the 1970's and 1980's. This will leave an aging workforce of software engineers who can't be easily retrained. They will then spend their twilight career years pushing carts for Home Depot.

It's time for the software development business to get lean and focus on delivering real ROI faster. Read the whole CNN article...

Martin Fowler says that we "cannot measure productivity". I disagree! Much of my book is about precisely how to measure productivity - as Throughput - or the rate of delivered value. This can be done by tracking client-valued functionality delivered as working code. In FDD that means Features delivered in a given time period with Throughput defined as the value of those Features.

Actually, if you read the posting careful through to the end, you will see that Fowler actually agrees with me ;-)

Bart Perkins has a very intelligently written article about some of the hazards of outsourcing in this week's Computerworld.

Misunderstandings about outsourcing can generate bad publicity. One company's outsourcing plans were leaked to a state senator, who was told that the company was bringing in illegal aliens under false visas to replace U.S. citizens. The senator was prepared to take the issue to the statehouse, until the company cleared up the misunderstanding.

Dave Thomas highlights the problem with business rules in requirements and implementing software systems. Dave has all the questions, luckily Nicola, Mayfield and Abney have the answers. Maybe I should recommend Dave gets a copy of Streamlined Object Modeling :-).

Meanwhile, my own copy of Principles of the Business Rule Approach turned up today. Like a bad student, I started reading at page 109, chapter 8 that describes how to define rules and gives advice on good and bad style. Having read one chapter and flicked through the rest, I'm convinced this is a technique that, when coupled to advanced analysis and design techniques will reduce variation throughout the lifecycle and improve repeatability. The effect will be to reduce required buffers and allow projects to be delivered faster with higher quality.

Coincidentally, I'm actually working on modeling a system of reasonable size - about 6 months work for a team of 10 - this week. [It's good for a manager to do real work from time to time]. I am using business rules for real, as part of step 2 in FDD - Build a Feature List. I'll post more comments when I've read the rest of the book and developed some experience using what I learned.

Johanna Rothman writes in her blog that managers still matter even with agile development processes.

"But what about managers? Earlier, I said that managers matter too. Here's why. Good people can triumph over inadequate process or inadequate management. But even the best people with the best process can't triumph over bad management. Bad management trumps everything else. I've worked for bad managers, and I bet you have too, so you've seen the damage bad managers can cause."

This topic is close to my heart. It has a lot to do with why I wrote the book. Bad managers kill software productivity. I recall a conversation I had with the (then) CIO of Sprint PCS - Jerry Batt. We both agreed that management in IT was universally poor on the average. I said I thought it was because enough managers hadn't been programmers - often getting into a PM track too early in their career and as a result developers didn't respect them. He replied that he believed the reason that managers were so poor was that too many of them were programmers. We held the same view, agreed on its effect but disagreed on its cause.

I now realize that I was talking about leadership whilst he was (as a trained senior manager) talking about management - target setting, measurement, analysis of feedback, reaction and investment. In Leading Geeks, Paul Glen states that he believes management and leadership cannot be separated with respect to knowledge work. I disagree. I wrote a book about management. He wrote one (primarily) about leadership. Both are important. Both matter. Motivating a team of developers requires both skills.

Scott Ambler writes about his experience at the Agile Development conference in his column at Software Development magazine [requires registration]

"In June, I went to the first Agile Development Conference in Salt Lake City, and it was one of the best conferences I've ever attended - partly due to the insights I gained from the Executive Summit track."

In his excellent book, Managing the Design Factory, Donald G. Reinertsen, observes that architectural partitioning in product design, reduces risk by segregating variability and thus reducing uncertainty and the necessary buffering in the schedule.

It now seems possible to separate concerns in the requirements stage. Facts, Rules and Tasks have different amounts of uncertainty and are prone to different levels of change. Separating them out allows parallel development with reduced buffering and shorter schedules.

Facts about a domain (Features in FDD or Information in Goldratt's, The Haystack Syndrome) have a very low uncertainty. Facts are seldom likely to change. The only issue at variance is whether a computer system needs to include them or not. Hence, domain modeling and fact identification (Feature Listing) - steps 1 and 2 in FDD - have very low uncertainty. Completing these represents real progress.

The separate identification of business rules allows them to vary independently of facts. Rules are likely to change. Rules represent business policies such as "You must be 18 years of age to contract for a service plan" and "New service plans include a 2 year commitment with penalties for early cancellation". Such policies can be changed by marketing managers at short notice.

Tasks are the least well understood. Often UI designers talk of "a-modal design". The purpose of which is to allow a task to be performed in any order. This prevents introducing a design constraint on usage. Different people like to perform tasks differently. Capturing task descriptions too early or too rigorously risks problems with the user interface design late in the cycle. It may even involve rejection by the user.  Tasks, procedures and UI designs have a high variability and are subject to late change. They need larger buffers. Keeping them separate from the rest of a system is more effective.

The problem with Use Cases is that they encourage the coupling of all three concerns - facts, rules and tasks. Use Cases couple all three types of requirement. This can lead to poor quality architecture. I fear that the mental model encouraged by Use Cases and their coupled approach to requirements leads to problems throughout the lifecycle. Plans based on requirements as Use Cases will need more buffering, will be subject to more late change and will carry greater uncertainty and risk throughout the lifetime of a project.

[I must thank both Martin Geddes and Philip Bradley whose thoughts contributed to this post]

Seattle has a ride-free bus service within a limited zone around downtown. It operates from 6am until 7pm every day. People have speculated that it is a county government perk for tourists, or perhaps for office workers, or maybe the politicians just want to be nice to the voters! I tend to think this is unlikely. Perhaps systems thinking and the Theory of Constraints offer a better explanation.

Public transport systems can become virtuous or vicious cycles - the more they get used the more provision of service, and the more available service, the more usage. Equally, the corollary is true, the less usage, the less service is provided which leads to less usage. There is a subtle tipping point between the two and a stable system seems hard to achieve. If service is unreliable and infrequent then the paying public will go elsewhere and usage will fall. This will lead the supplier to reduce service and soon there is no benefit from a public transport system at all.

So it is desirable to have both a timely and frequent service. If buses are to run on-time then uncertainty surrounding the scheduling of the timetable must be reduced. Processing large batches of passengers at a few stops in downtown has more irregularity than processing smaller batches at the many stops in the urban neighborhoods and suburbs. However, if passengers don't have to mess around finding change and drivers don't have to collect fares then the irregularity associated with large numbers of people entering or leaving a bus is reduced. Making the downtown area ride free improves the likelihood that the timetable can be met. It helps to keep buses moving and to keep traffic flowing in downtown. The systemic effect encourages usage through improved quality of service.

All that was required to make this happen was a change to policy constraints - the rider must always pay and that payment must always be made on entry. As soon as these policies were waived in favor of a policy where riders going downtown, pay on entry and those going out-of-town pay on exit, then the problem of irregular flow is reduced and buses are more likely to run on time. The side-effect and added passenger benefit is the emergence of a ride-free zone. The metro transit company is trading off the localized revenue lost in downtown against the gain from increased usage of the system by longer distance travelers.

Often the throughput or effectiveness of a system element can be limited by a policy constraint. The policy may be having a negative effect on the whole system. Eliminating the policy may produce both a local improvement and a global improvement - as is the case with the King County Metro Ride-Free Zone - buses run on-time but even better more overall use is made of the bus service.

I have been asked to provide some software engineering examples of the common cause and special cause variation from Don't apologize be on-time!

Common cause variation in software development would be things like the size and complexity of Features or Stories or Use Cases estimates. For example, an estimate could be under or over and it is likely that these will vary within reasonably bounded limits. It might also be things like staff vacation, sickness due to common illness like influenza or cold. The amount of staff downtime is likely to be bounded to a number of days per year. Whether or not the current project overlaps with those downtime days is the issue at variance.

Special cause variation would be things like office downtime due to the power outage caused by the 20 year ice storm. Special cause would be your Indian chief architect announcing he will be out for one month unpaid leave to get married. He only gets married once. Therefore it won't happen again.

There are gray areas. For example, some special causes may be seen as common causes under different circumstances. For example, let's imagine that not only is the chief architect Indian but half the team of developers is also Indian and male and single. The chance that one of the them will take a month off to get married during any given project is high. Therefore for a manager with a team of this description, a one man month outage due to marriage is common, not special. Hence, this should be buffered as common cause.

Another example might be that your middleware supplier issues an upgrade which subsequently breaks all of the your code due to a bug. If that supplier was previously reliable then this should be treated as special cause variation. However, it may turn out to be an initial instance of a problem which will become more regular. Once a new pattern of behavior is recognized then such variation should be treated as common cause.

I discovered that there is a whole community around the Business Rules theory. It seems there was a resurgence in these ideas in 1997 - about the time that I was busy helping to create FDD. Hence, it's explainable why I'm only just discovering it.

It seems this work is driven by the data modeling community - authors such as Terry Halpin - and there seems to be little involvement from the OO community. I wonder if they even know about the work of Nicola, Mayfield and Abney?

I had lunch today with 4 developers. Despite the high geek quotient around the table in the Chinese restaurant of Seattle's International District, the conversation was particularly eclectic. However, a thread on economics and politics inevitably led to the debate on the economics of out-sourcing.

At one point, I said, "If costs in Bangalore are about 1/4 those in Seattle then all you need to do is improve productivity 4 fold to compete." One of the developers was stunned by this. He is a hard working, smart, diligent guy. How could he possible work 4 times harder? Good question.

I replied that if you have an initial quality metric of 3 bugs per Feature and you cut that to say 0.5 bugs per Feature by spending 15% of your effort on design reviews, code reviews and unit tests, then you will increase productivity 4 fold - easy! And this is just scraping the surface of what is possible. Eliminate all sorts of waste such as - queuing and waiting time, conflicts, use automation on repeating process and non-value-added activities such as reporting, improve analysis techniques to focus just enough and no more, improve flow in the value chain and reduce work-in-process. With all of these things it is possible to see up to 10 fold improvements with initially immature teams.

I was impressed with this article, Business Rules Connect First, by Ronald G. Ross in Computerworld. It is based on his book Principles of the Business Rules Approach. What I like about it most is that it offers a lean, succinct method for completely defining a set of requirements in an FDD project. Ross says requirements are 3-sided: Facts, Processes, and Rules. His definition of Facts is almost identical to the FDD definition of Features, which in turn share a striking resemblance to Eli Goldratt's definition of Information (as opposed to data) from The Haystack Syndrome. Processes can be easily modeled using a chain of Peter Coad's <<Moment-Interval>> archetypes. Hence, FDD has these two sides of Ross' requirements pretty well nailed.

What Ross seems to be offering is a solution to the 3rd side - the rules which determine the nature of the business logic implemented in the code. Nicola, Mayfield and Abney have already explained how these should be modeled and coded in Streamlined Object Modeling. Ross seems to offer a simple, clean method for capturing the requirements which can be performed by business owners but leads directly to a clean implementation by developers. It's all value-add and no waste - an ideal Lean development process.

As all rules are by definition constraints, I am wondering if the process of discovering the rules using Ross' method would encourage an examination of these policy constraints - resulting in a simplification of the business rules (removal of constraints) before the system is designed and coded. This would lead to simpler systems which require less effort to build. Sounds even leaner to me.

I've ordered the book!

Back in the early 1990's, I was once lucky enough to tour the Basingstoke, UK office of a well known Fortune 100 technology company, on a weekend. I was accompanied by a Senior Director, who attempting to find a machine with which to demo a new product, chanced upon a staff member's desk where some documents had been left in full view. He picked them up and said he would lock them in his own office, so that the staffer wasn't fined. Apparently, company policy at the time was to enforce the clean desk policy through an automatic payroll deduction. :-O

More recently, I have encountered a policy on intellectual property protection which goes beyond clean desks but includes clean whiteboards, clean notice boards, clean walls - pretty much everywhere clean that isn't specifically locked and has limited access. The thing is I don't feel such cleanliness policies are compatible with lean software development.

If you visit the site office for a construction site, you expect to see drawings, plans and project plans lying around. The team doing the building needs to be able to view the blueprints. The same is true in Lean manufacturing. They use a technique called visual control. Visual control means that the intellectual property is in full view, pinned to the wall and openly manipulated by the workforce as they work. Software engineering is best done this way too. Models should be on the wall as reference and tribute to work completed. Feature lists, progress charts, production graphs and defect counts should also be in full view. Group accountability is insured by providing transparency and openness and encouraging debate over the work product. When you lock everything out of sight, you discourage debate and group responsibility for results. People need to hold meetings to share information and there is a tendency towards "knowledge is power" information hiding. The result is poorer productivity at the expense of happy company lawyers.

Clean desk IP protection policies, like all policies are constraints. Sometimes the constraining influence of a policy is not immediately obvious. It requires systems thinking to analyze effect on the software development ecosystem and determine whether a policy is truly advantageous.

"Don't apologize, be on-time" was advice given by Sean Connery to Wesley Snipes in Rising Sun. It was one of the few true insights into Japanese culture in an otherwise poor movie.

Hence, it was with some surprise that I stood on the sidewalk outside my office last Friday waiting for my (Japanese) wife who was late. When I called her she was "2 minutes away". 10 minutes later she showed up with 20 minutes to take me to an appointment which, at best, was 20 minutes away.

It turned out that she setout in plenty of time but got stuck behind a truck which had grounded itself turning off Ballard Bridge into Nickerson. On the return trip we hit the same jam, as emergency services tried to clear the mess with a heavy duty tow truck. In the end we were 10 minutes late. My wife's journey which would normally have taken 20 minutes each way, actually took over an hour. Is it reasonable to attribute blame in this kind of situation?

There is considerable variance possible between my home and my office. The draw-bridge can be up - typical delay 5 minutes. There are numerous traffic lights. The level-crossing can be closed for a train - typical delay 10 minutes but alternative route available. A typical journey takes 20 minutes. A good time is 15 minutes. If most traffic lights are against, the bridge is up and the train is coming then it might take 30 minutes. However, if there is an accident or a fire (as happened last year) then the journey could take much longer.

These latter events are known in quality assurance as special cause. The others - train, lights, bridge - are common cause. To buffer for special cause events may allow for a face-saving delivery on-time but it is very expensive. Most estimates would be way over and projects would look far too expensive. For the collect husband and take him to appointment project a reasonable estimate for the round trip might have been 50 minutes. This would have absorbed all reasonable common cause variance with an almost 100% certainty of completing the job on-time.

Any reasonable senior manager should expect line managers to buffer common cause variance appropriately whilst accepting that unforeseeable special causes can still make a delivery late without attributing blame or fault.