Microsoft is releasing an agile flavor of the forthcoming MSF V4.0. Download a sneak preview from this GotDotNet Workspaces site. [You need a Microsoft Passport to get in and you may need to be logged in to passport before you select the link to be successful].

|

This blog is one year old today. I started it as a way of publicizing my book. I can't be more pleased with the results. The book has outsold my expectations for it and has been very well received. Now, in summer 2004, it seems widely accepted in the agile community that the Theory of Constraints has a role to play in the software engineering process. One year ago, that really wasn't true. Putting the book aside though, this blog has been very successful too. During the year I have posted 205 blog entries (including this one) and 15 other articles. Perhaps it is the content which keeps people coming back. With around 500 page views per day and over 10,000 RSS feeds per week, I'm amazed at how the audience has grown. Thanks - it makes it all worthwhile to know that people actually read the material. Most curious has to be the popularity of the overview slides. These receive around 100 downloads per day. I'd love to know why...

|

Clarke Ching has started a Yahoo! group and weblog for virtual team to work together collaboratively to apply the TOC Thinking Processes to the problems in software project management. This isn't your usual kind of group chat or publishing weblog, it's real interactive work with a specific goal in mind. Learn more at TOC Software.

|

Drucker on Refactoring - no really! Here is what Peter Drucker might have had to say about refactoring were it available as a practice when he wrote these words in 1954.

"A favorite story at management meetings is that of the three stonecutters who were asked what they were doing. The first replied, 'I am making a living.' The second kept on hammering while he said, 'I am doing the best job of stonecutting in the entire country.' The third one looked up with a visionary gleam in his eyes and said, 'I am building a cathedral.'

The third man is, of course, the true 'manager.' The first man knows what he wants to get out of the work and manages to do so. He is likely to give a "fair day's work for a fair day's pay."

It is the second man who is a problem. Workmanship is essential; without it no business can flourish; in fact, an organization becomes demoralized if it does not demand of its members the most scrupulous workmanship they are capable of. But there is always a danger that the true workman, the true professional, will believe that he is accomplishing something when in effect he is just polishing stones or collecting footnotes. Workmanship must be encouraged in the business enterprise. But it must always be related to the needs of the whole.

... The tendency to make the craft or function an end in itself [in future] will therefore be even more marked than it is today. ... The new technology will need both the drive for excellence in workmanship and the consistent direction of managers at all levels toward the common goal."

What Drucker is telling us is that craft workmanship such as zealous refactoring is important both to an organization's morale - it is important that people can take pride in their work - but also to the quality of what is being produced. However, over-zealous refactoring destroys shareholder value. So beware of the cruft polisher on your team. Refactoring should be done for the right reasons - to eliminate technical debt, to rebalance the books, and facilitate future iterations. It should never be done simply because a developer doesn't like the architecture or the implementation. If you can't answer in the positive to this question - does this code prevent us from efficacious delivery of future functionality? - then any refactoring is to use Drucker's term merely "polishing stones."

|

I've had a lot to say on the difference between agile development methods and their predecessors is a focus on effectiveness rather than cost accounting style efficiency. Others in the community have picked up on this theme too and the relationship of efficiency and cost accounting back to Frederick Taylor and time'n'motion studies from the turn of the 20th Century have been drawn. Taylorism was about specialization and efficiency of specialists - local optimum hoping for a global optima - whilst modern lean/agile thinking is about effectiveness of adaptable generalists and systems thinking with alignment around a global optimum.

Peter Drucker has had quite a bit to say about effectiveness and knowledge workers. Most of what follows is taken from "The Effective Executive" from 1966 - note a clear 3 years before The Psychology of Computer Programming by Gerald Weinberg. In the mid-Sixties, software engineering had something to learn from general management science but at the time the link had never been made.

"To be effective is the job of the knowledge worker."

"people of high effectiveness are conspicuous by their absence in knowledge jobs. High intelligence is common enough amongst knowledge workers. Imagination is far from rare. The level of knowledge tends to be high. But there seems to be little correlation between a man's effectiveness and his intelligence, his imagination, or his knowledge. Brilliant men are often strikingly ineffectual; they fail to realize that the brilliant insight is not by itself achievement."

"Intelligence, imagination, and knowledge are essential resources, but only effectiveness converts them into results."

He then separates the Taylor era from the modern knowledge worker era so effectively with this definition...

"For manual work, we need only efficiency, that is, the ability to do things right rather than the ability to get the rights things done."

Efficiency = Do Things Right
Effectiveness = Do The Right Things

"Increasing effectiveness may well be the only area where we can hope significantly to raise the level of the knowledge worker's performance, achievement, and satisfaction."

"One of the weaknesses of young, highly educated people today ... is that they are satisfied to be versed in one narrow specialty and affect contempt for the other areas."

Drucker comments that finding effective people is difficult and makes an observation that today we would summarize as  - There is No 'Effective' Myers-Briggs Type.

"there is no 'effective personality.'"

He goes on to observe that effectiveness is not achieved from principles but from successful execution of practices. If we are to learn from Drucker, it is that we cannot merely talk about agile development in abstract principles - even though this is the simplest way to gain consensus in the community - in order to show effectiveness we must articulate practices which deliver productivity.

"Effectiveness is a habit; that is, a complex of practices. And practices can be learned."

Hence, Drucker leaves us with hope - the hope that all we have to do is to teach, coach, mentor and instill the correct set of practices in the individual knowledge worker and effectiveness will follow. What is largely lacking in the teaching of software engineering as a profession is just that - the set of practices which lead to effectiveness. If we can fix that both at the academic level and within industry itself then we can start to deliver the value that shareholders deserve from their expensive knowledge workers.

|

I feel that more and more what I and others are doing with agile comes back to Boehm's Spiral Model first postulated in 1986 [1] and refined in 1988 [2]. Boehm described Spiral as evolutionary rather than incremental. The purpose was risk management and the enabler was the ability to analyze the whole system but only in the briefest of detail to identify the highest priority features for an initial iteration. On delivery of the iteration, customer feedback would be used to prioritize the next set of features and make refinements and add detail to the initial analysis. It was this feedback and refinement which separated it from merely incremental development - presumably the assumption in 1986 was that incremental meant, big analysis up-front, divide system into set of iterations, perform each iteration in open loop without accepting changes.

I've been reflecting on a big FDD project I was involved in during the first half of the year which started its discovery phase in the Q4 last year. Without realizing it was significant, I now reflect on the fact that we did two passes of modeling - this is not part of the standard FDD prescription. The domain was so new that no one really knew what they wanted or what the system should do. We did a first pass domain modeling session with all the marketing and project management folks as well as architects and chief programmers. This initial pass was very thin - it was this way because we knew so little about the domain. However, this served its purpose to flush out areas for further discussion and as an enabler to allow the marketing folks to define their requirements. This first pass modeling session gave us a scope, a list of business processes and an outline of the subject areas in the domain. It was enough for us to identify rough outlines for 6 iterations.

The second pass of modeling - in greater detail, but no greater than a typical FDD project - then happened on each iteration and typically took 4 to 5 days. For a 7 week iteration, this is a lot more time spent modeling than Jeff De Luca or Peter Coad would typically recommend (1 week for every 3 months), but the domain was new and uncertain which probably explains the extra effort required. A Feature List would then be written and the iteration scoped in detail. The Design-By-Feature, Build-by-Feature cycle would then start and each Feature would be designed with a UML sequence diagram. Hence, there were three passes at modeling and on each pass more information was obtained. This is one more than a typical FDD project and it begins to sound more and more like Boehm's Spiral Model.

When I first heard about Spiral Model, I was dismissive. The key to it is an ability to examine a problem in only enough detail to move around the loop. It's been widely identified that over-modeling, or drilling deep is a common problem with developers. The natural tendency is towards "big design up-front." What makes the Spiral Model hard to achieve in practice is that discipline or craft tool which enables a wide but not deep pass of entire scope in order to manage risk and prioritize deliverables. At the time, I didn't believe that a suitable technique - a suitable enabler - existed to make Spiral Model real in practice. However, I am now wondering whether Coad's domain modeling in color archetypes is such a tool and an enabler of a true spiral approach to software engineering.

I ask myself would I do it again - domain model in two passes for a multi-iteration project? The answer is definitely, "Yes!" Do I think that developers can be taught to model just wide enough, to embrace uncertainty and to understand when they have just enough information to move forward? The answer to this is also "Yes!" but there is a caveat. This skill - of knowing when to stop - is learned through experience. Hence, the skill can only be passed from individual to individual through real world work experience and direct observation. It truly is a craft. Could we (the experts in domain modeling in color) codify that experience so that it might be taught in a classroom? I don't have an answer for that yet...

References
[1] Barry Boehm, "A Spiral Model of Software Development and Enhancement", ACM SIGSOFT Software Engineering Notes, August 1986.
[2] Barry Boehm "A Spiral Model of Software Development and Enhancement", IEEE Computer, vol.21, #5, May 1988, pp 61-72.

|

There has been a lot of debate in the agile community over this last 3 years about self-organization and whether there is need for any management at all in software development. Peter Drucker offers us this advice in Management Challenges for the 21st Century (1999)...

One does not "manage" people.
The task is to lead people.
And the goal is to make productive the specific strengths and knowledge of each individual. [Emphasis original]

In my book and again in an article in the current edition of Cutter IT Journal, I argue for splitting project management and development (or engineering) management into two distinct roles filled by different people. Why?

Drucker wrote these words (above) in the context of an old assumption that there is one right way to manage people (or there should be). He argues that this is inaccurate and that whilst there may be one right way to manage things, people can not be managed by prescription. We could interpret this for software engineering by saying that whilst there may be one right way to manage projects, there is no right way to manage the people engineering the project - those people must be led. In simple terms, the project manager is responsible for managing things - the issues and day-to-day problems which arise in a project, for removing the obstacles and to facilitating the flow of work. The engineering manager has a different job - to lead the people, to organize them into a system for software design and construction and to motivate them to do good work as a team and to build that team into a learning organization which is always looking for better ways to become more productive. It's a structure with Engineering Manager as leader, Project Manager as manager, and staff reporting to the leader rather than the manager.

|

August is Peter Drucker Month at agilemanagement.net. Peter Drucker is considered the father of modern management science. He started studying and writing about management in the 1940's and was the academic who made it fashionable. In his early years he worked (or at least shared an office with) Edwards Deming. Later in life Deming and Drucker had some professional disagreements, primarily over Drucker's "Management By Objectives" (MBO) which Deming deeply disagreed with. Drucker is in his late 90's now but as recently as 5 years ago was still publishing new material.

To kick things off, here is an observation from Drucker's 1992 book, Managing for the Future.

"Managing the knowledge worker for productivity is the next great challenge for American management."

Writing in the same year, Ed Yourdon predicted The Decline and Fall of the American programmer as he felt sure their jobs would be outsourced offshore to low cost economies. I recall giving a recruitment presentation for interns in 1994 at Napier University in Edinburgh and predicting that "programming is a dead end profession when your job as a programmer can be done in India for one fifth of the cost." I was offering to give the interns exposure to "more than just programming" and an "opportunity to learn skills which will protect your career." Of course, Yourdon and I  were both wrong - what saved the industry was the Internet bubble - demand exceeded supply and cost was no object of concern until the bubble burst. Here in 2004, we're learning it wasn't a cure but a stay of execution and once again knowledge worker productivity is the next great challenge for the American manager.

So what does Peter Drucker have to say about productivity of knowledge workers and what might we in the software engineering profession learn from it? [More to come throughout August...]

 

|