"Becoming Self-Adapting" describes a technique for evolving a light-but-sufficient, project-personal methodology quickly enough to be useful to the project. The key idea is to reflect every few weeks on what works well and what should be changed.
The theory so far seems to say that we should use a mostly oral tradition to bind the huge amount of information generated within the project.
Common sense tells us that oral tradition is insufficient. Looking for Documentation
A programmer told of his company rewriting their current core product because there is no documentation, no one left who knows how the system was built, and they are unable to make their next changes. He said he hopes there will be documentation after the project, this time. Another told of three projects, each of which will build on the previous. The three are not at the same location. He said that they can't possibly work on a strictly oral basis.
It is possible to have too little stickiness in the information at hand. It is time to revisit the Cooperative Game principle:
The primary goal is to deliver software; the secondary goal is to set up for the following game.
Reaching the primary goal is clear: if you don't deliver the software, it won't matter how nicely you have set up for the following game.
If, on the other hand, you deliver the software and do a poor job of setting set up for the following game, you jeopardize that game.
The two are competing activities. Balancing the two competing activities relies on two arts.
The first art is guessing how to allocate resources to each goal. Ideally, documentation activities are deferred as long as possible, and then made as small as possible. Excessive documentation done too early delays the delivery of the software. If, however, too little documentation is done too late, the person who knows something needed for the next project has already vanished.
The second art is guessing how much can be bound in your group's oral tradition and how much has to be committed to archival documentation. Recall that we don't care, care, past a certain point, whether the models and other documentation are complete, correctly match the "real" world (whatever that is), or are up-to-date with the current version of the code. We care whether the people receiving them find them useful for their specific needs. The correct documentation is exactly that needed for the receiver to make her next move in the game. Any effort to make the models complete, correct and current past that point is a waste of money.
Usually, the people on the successful projects I have interviewed felt that they had succeeded "despite the obviously incomplete documents and sloppy processes" (their words, not mine). Viewed in our current light, however, we can guess that they succeeded exactly because the people made good choices in stopping work on certain communications as soon as they reached sufficiency and before diminishing returns set in. They made the paperwork adequate, they didn't polish it.
Adequate is a great condition if the team is in a race for an end goal and short on resources. Recall the programmer who said, "It is clear to me as I start creating use cases, object models and the like, that the work is doing some good. But at some point, it stops being useful, and starts being both drudgery and a waste of effort. I can't detect when that point is crossed, and I have never heard it discussed. It is frustrating, because it turns a useful activity into a wasteful activity."
We are seeking that point, the one at which useful work becomes wasteful. That is the second art.
Barely sufficient
I don't think I need to give examples of overly heavy or overly light methodologies. Most people have seen or heard enough of these.
"Just-barely-too-light" methodologies, on the other hand, are hard to find, and very informative. They are the ones that help us understand what barely sufficient means.
Two such project stories are given earlier in the book: "Just Never Documentation" in Chapter 1, and "Sticking Thoughts on the Wall" in Chapter 3. In each, an otherwise well-run project ran below the level of sufficiency at a key moment. Just Never Documentation (Recapped) This team followed all of the XP practices, and delivered software in a timely manner to a receptive customer. At the end of several years, the sponsoring executives slowed and eventually stopped new development. Once the team members dispersed, there was no archived documentation on the system, and no team of people conversant with its structure. The formerly sufficient oral culture was now insufficient.
In this story, the team reached the first goal of the game, delivering a running system. They failed to set up for the next game, maintenance and evolution.
Using my own logic against me, one could argue that the documentation was exactly and perfectly sufficient for the needs of the company: The project was canceled, never to be restarted, and so the correct, minimal amount of documentation was zero! However, drawing on Naur's "programming as theory building," we can see that the team had successfully built up their own "theory" during the creation of the software, but they left insufficient tracks for the next team to benefit from the lessons they had learned. Sticking Thoughts on the Wall (Recapped) The analysts could not keep track of the domain in their heads, it was so complex. However, they had just switched from a heavy process to XP, and thought they were forbidden from producing any paperwork. As the months went by, they found it increasingly hard to decide what to develop next, and to determine the implications of their decisions. They were running below the threshold of sufficiency for their portion of the game. Rather than less, they needed more documentation to make their project work.
They eventually recognized their situation, and started inventing information holders so that their communications would reach sufficiency.
What we should see is that "insufficiency" lies, not in the methodology, but in the fit between the methodology and the project as ecosystem. What is barely sufficient for one team may be overly sufficient or insufficient for another. Insufficiency occurs when team members do not communicate well enough for other team members to carry out their work.
The ideal quantity, "barely sufficient," varies by time and place within any one project. The same methodology may be overly sufficient at one moment on a project and insufficient at another moment.
That second art mentioned above is finding the point of "barely sufficient," and then finding it again when it moves.
Recommendations for Documentation
This leads us to a set of recommendations: Don't ask for requirements to be perfect, design documents to be up-to-date with the code, project plan to match the state of the project. Ask, instead, that the requirements gatherers capture just enough to communicate with the designers.
Ask them to replace typing with faster communications media where possible, including visits in person or short video clips. If the designers happen all to be expert and sitting close by each other, ask to dispense with design documentation beyond whiteboard sketches, and then capture the whiteboard drawings with photos or printing whiteboards. Bear in mind that there will be other people coming after this design team, people who will, indeed, need more design documentation. Run that as a parallel and resource-competing thread of the project, instead of forcing it into the linear path of the project's development process. Be as inventive as possible about ways to reach the two goals adequately, dodging the impracticalities of being perfect.
Find (using exaggerated adjectives for a moment) the lightest, sloppiest methodology possible for the situation. Make sure it is just rigorous enough that the communication actually is sufficient.
Agile implies being effective and manoeverable. An agile process is both light and sufficient. The lightness is a means of staying manoeverable. The sufficiency is a matter of staying in the game.
The question for using agile methodologies is not to ask, "Can an agile methodology be used in this situation" but "How can we remain agile in this situation?"
A 40-person team won't be as agile as a six-person colocated team. However, each can maximize its use of the agile methodology principles, and run as light and fast as they can creatively make their circumstances allow. The 40-person team will use a heavier-agile methodology, the six-person team will use a lighter-agile one. Each team will focus on communications, community, frequent wins and feedback.
If they are paying attention, they will reflect periodically about the fit of their methodology to their ecology, and keep finding where the point "barely sufficient" has moved itself to.
Sweet Spots
Part of getting to agile is identifying the sweet spots of effective software development and moving the project as close as possible to those sweet spots.
A team that can arrange to land on any of those sweet spots, gets to take advantage of some extra efficient mechanism. To the extent the team can't arrange to land in a sweet spot, it must use less efficient mechanisms. At that point, the team should think creatively to see how to get to the sweet spot, and to deal with not being there.
Here are a selection of five sweet spots:
Two to eight people in one room
Information moves the fastest in this sweet spot. The people ask each other questions without overly raising their voices. They are aware of when the other people are available to answer questions. They overhear relevant conversations without pausing their work. They keep the design ideas and project plan on the board, in ready sight.
People repeatedly tell me said that while the environment can get noisy, they have never been on a more effective project than when their small team sat in the same room.
With leaving this sweet spot, the cost of moving information goes up very fast. Every doorway, corner and elevator multiplies that cost.
The story, "e-Presence and e-Awareness" (Chapter 3), tells of one team not being able to land in this sweet spot. They used web cams on their workstations to get some of the presence and awareness of sitting in the same room. They used chat boxes to get answers to they very many small questions that constantly arise. They were creative in mimicking the sweet spot in an otherwise unsweet situation.
On-site usage experts
Having a usage expert available at all times means that the feedback time from imagined to evaluated solution is as short as possible, often just minutes to a few hours.
Such rapid feedback means that the development team grows a deeper understanding of the needs and habits of the users, and start making fewer mistakes coming up with new ideas. They try more ideas, making for a better final product. With a good sense of collaboration, the programmers will test the usage experts' ideas and offer counter-proposals. This will sharpen the customers' own ideas for the how the new system should look.
The cost of missing this sweet spot is a lowered probability of making a really useable product, and a much higher cost for running all the experiments.
There are many alternative, if less effective, mechanisms you can use when you can't land on this sweet spot. They have been well documented over the years: weekly interview sessions with the users; ethnographic studies of the user community; surveys; friendly alpha-test groups. There are certainly others.
Missing this sweet spot does not excuse you from getting good user feedback. It just means you have to work harder for it.
One-Month increments
There is no substitute for rapid feedback, both on the product and on the development process itself. Incremental development is perfect for providing feedback points. Short increments help the both the requirements and the process itself gets repaired quickly. The question is, how long should the delivery increments be?