The team initially valued "thinking ahead," "subtle but extensible designs," and "my code is private."
The team, largely under the impetus of Kent Beck, rebuilt itself with the core values "make it simple and clear," "you don't really need that subtle item," "all code is public,” and “any pair of people sitting together may change anything." With these shifts, the same people also adopted a different and highly disciplined set of practices.
Those caveats having been placed, I do notice people having certain kinds of "failure modes." I regularly see methodologies and projects fail for not taking these human characteristics into account. We can build systems of people that are less likely to fail by explicitly taking these characteristics into account.
The five failure modes to take into account are people
· Making mistakes
· Preferring to fail conservatively
· Inventing instead of researching
· Being creatures of habit
· Being inconsistent
Making Mistakes
That people make mistakes is, in principle, no surprise to us. Indeed, that is exactly why iterative and incremental development were invented.
Iterative refers to a scheduling and staging strategy that allows rework of pieces of the system.
Iterative development lets the team learn about the requirements and design of the system. Grady Booch calls this sort of learning “gestalt, round-trip design” (Booch 1994), a term that emphasizes the human characteristic of learning by completing.
Iterative schedules are difficult to plan, because it is hard to guess in advance how many major learnings will take place. To get past this difficulty, some planners simply fix iterations at three [levels? points? arbitrary milestones?]: draft design, major design, and tested design.
Incremental refers to a scheduling and staging strategy in which pieces of the system are developed at different rates or times and integrated as they are developed.
Incremental development lets the team learn about its own development process as well as about the system being designed. After a section of the system is built, the team members examine their working conventions to find out what should be improved. They might change the team structure, the techniques, or the deliverables.
Incremental is the simpler of the two methods to learn, because cutting the project into subprojects is not as tricky as deciding when to stop improving the product. Incremental development is a critical success factor for modern projects (Cockburn ). [Insert date]
The very reason for incremental and iterative strategies is to allow for people’'s inevitable mistakes to be discovered relatively early and repaired in a tidy manner.
That people make mistakes should really not be any surprise to us. And yet, some managers seem genuinely surprised when the development team announces a plan to work according to an incremental or iterative process. I have heard of managers saying things like,
“What do you mean, you don’t know how long it will take?” or “What do you mean, you plan to do it wrong the first time? I can go out and hire someone who will promise to do it right the first time.” In other words, the manager is saying that he expects the development team not to make any major mistakes or to learn anything new on the project.
One can find people who promise to get things right the first time, but one is unlikely to find people who actually get things right the first time. People make mistakes in estimation, requirements, design, typing, proofreading, installing, testing, ... and everything else they do. There is no escape. We must accept that mistakes will be made and use processes that adjust to the fact of mistakes.
Given how obvious it is that people make mistakes, the really surprising thing is that managers still refuse to use incremental and iterative strategies. I will argue that this is not as surprising as it appears, because it is anchored in two failure modes of humans: preferring to fail conservatively rather than risk succeeding differently; and having difficulty changing working habits.
Preferring to Fail Conservatively
There is evidence that people generally are risk-averse when they have something in their hands that they might lose and risk-accepting if they are in the process of losing something and may have a chance to regain it (Piattelli-Palmarini 1996).
The Piattelli-Palmarini study descibes a number of experiments involving risks and rewards. The interesting thing is that even when the outcomes are mathematically identical, the results are different depending on how the situation is presented. Illusions of Choice
Piattelli-Palmarini cites a dual experiment. In the first, people are given $300 and then have to choose between a guaranteed $100 more or a 50/50 chance at $200 more. People prefer to take the guaranteed $100. In the second, people are given $500 and then have to choose between having $100 taken away from them or a 50/50 chance of having $200 taken away from them.
People prefer to risk having $200 taken from them.
(Piattelli-Palmarini, p. 58) Mathematically, all outcomes are equal. What is interesting is the difference in the outcomes depending on how the problem is stated.
Piattelli-Palmarini sums up the aspect relevant to project managers: We are risk-averse when we might gain.
Consider a manager faced with changing from waterfall to incremental or iterative scheduling. The waterfall strategy is accepted as a normal, conservative way of doing business, even though some people think it is faulty. The manager has used this strategy several times, with varying success. Now, one of his junior people comes to him with a radically different approach. He sees some significant dangers in the new approach. His reputation is riding on this next project. Does he use the normal, conservative strategy or try out the risky new strategy?
Odds are that he will use the normal, conservative strategy, a "guaranteed" standard outcome, rather than one that might work but might blow up in strange ways.
This characteristic, "preferring to fail conservatively rather than to risk succeeding differently," gets coupled with people's fear of rejection and the difficulty they have in building new work habits. The three together explain (to me) why managers continue to use the long-abused one-pass waterfall development process. Based on this line of thinking, I expect that people will continue to use the waterfall process even in the presence of mounting evidence against it and increasing evidence supporting incremental and iterative development. The line of thinking that perpetuates the use of the waterfall process is anchored in a failure mode.
In keeping with variations among people, some people have the opposite tendency. Often, though, the most adventuresome people are those who have little to lose personally if the project fails.
The good news is that there are opportunities for both sorts of people. The bad news is that these people probably find themselves on the same project.
Inventing Rather than Researching
This behavioral mode may be peculiar to American and European software developers (I don't have enough experience with Indian and Asian developers to comment on their habits). It is the tendency to avoid researching previous solutions to a problem and just invent a new solution on the spot.
This tendency is usually described as a sickness, the Not-Invented-Here (NIH) syndrome. I prefer not to view it as a disease but rather as a natural outgrowth of cultural pressures. One might instead call it the Invent-Here-Now Imperative. It grows in the following way:
From earliest school days, students are instructed not to copy other people's work, to not help each other, and to be as original as possible in all but rote memory acts. They are given positive marks for originality and punished for using other people's solutions. (Recently, a fourth grade teacher told her students not to call each other to discuss homework problems—not even to ask for which problems to do!).
Through the university level, assignments are designed to produce grades for individual work, not for teamwork. This reaches a culmination in the Ph.D. dissertation, where originality is a core requirement.
Somewhere in these years of schooling, some people join the profession of "programmer," a person whose job is to program and who advances in the profession by writing harder and more uniquely original programs.
Under these circumstances, it is hardly surprising that the people internalize the Invent-Here-Now Imperative.
Upon showing up at work, though, these same people are told by the business owners that they should not write new programs but should scavenge solutions created throughout the industry over the history of the field. They should use as many existing solutions as possible, without violating intellectual property rights.
The rewards offered for this behavior are meager. People continue to receive low evaluations for reusing code instead of writing new code. Promotion comes to those who do the most and the best programming, not those who successfully hook together existing components. Technical authors still refer to people who do such work as low-level "component assemblers."
In one survey, education and attitude showed the greatest correlation with increased reuse (Frakes 1995). Just being shown that reuse was culturally considered a preferable outcome over developing new solutions [this phrase, which is not yet a complete sentence.]. Reward structures did not show a significant effect, nor did object-oriented technology, CASE tools, or a myriad of other factors.
Texas Instruments fought its "Not Invented Here" syndrome with an unusual award, the "Not Invented Here But I Did It Anyway" award (Dixon 2000). This NIHBIDIA award not only rewards people who make use of previous results, but it pokes fun at people caught up in the NIH syndrome at the same time. In this way, it creates a social effect of the type Frakes and Fox were referring to.
People who are professionals in some different field do practice effective reuse. These people, using the computer to accomplish some assignment of value in that other field, develop their sense of accomplishment from the program's effect in that other field, not from the cleverness of the programming. They are therefore motivated to put the software together to get on with their other work. They happily accept a less glamorous design if it can be put into use quickly.
Inconsistent Creatures of Habits
Asking a person to change his habits or to be consistent in action are the two most difficult requests I can think of. We are creatures of habit who resist learning new behaviors, and at the same time we tend toward inconsistency.
This may seem like a harsh judgement, and so I illustrate it with a conversation I heard between four people. Each was a senior manager or had a Ph.D., and so these were people you would most expect to be able to handle changing habits and being consistent. The Clean Desk Technique