Waterfall
and we don't mean Niagara
Mostly when we hear the term Waterfall, it is uttered along the lines of an admonition of something to be avoided, the proverbial Titanic headed for catastrophe. How did such a terrible fate befall a word that embodies one of the most wondrous phenomena of the natural world? Have you ever stood at the foot of Niagara or Iguazu, soaked from the mist, in awe of the unceasing thunder?
In software development, Waterfall describes a somewhat less awesome process of trying to achieve predictable outcome by controlling all of the project variables.
It’s not that Waterfall is a bad process, it’s just that it doesn’t match up to the kind of project we typically face in software development. Waterfall can be perfectly appropriate, for example, for relatively small projects in a well know technology and domain space, for a client that has a definite objective, especially if they are heavily committed to a fixed budget.
If the project involves some new technology, an unfamiliar problem domain, or the requirements are otherwise likely to evolve during the process, then you won’t be able to control all the variables. You need to adopt a scheduling practice that allows you to somehow keep resource allocation aligned with shifting and emerging objectives, the proverbial changing of the wheel on a moving carriage; that’s more along the lines of what most of us face in the workplace.
Adapting to change is one of the central tenants of Agile.
Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
— The Agile Manifesto
The alternative leaves us building what was asked for, at the expense of what is needed.
-
The perception of the customer’s need is changed as soon as they first see executable software, so we fully anticipate that the requirements we start with are only an indication of what is needed.
-
As a specification is built, the technology impacts the design in unanticipated ways; when aspects of the original specification prove impractical, or unexpected opportunities emerge.
-
The commercial, regulatory or other context that the product is intended for changes while the work is in progress.
These and other reasons are the motives for why practices designed to accommodate change are to the advantage of the customer.
Within the context of an Agile project, there is frequent debate about what is the appropriate level of definition before stories are ready to be committed to work. At one extreme we find a simple statement of business objective expressed in a story format, leaving the technical details unspecified:
As <some persona>
I want <some outcome>
So that I can <achieve some objective>.
Other circumstances need a detailed specification complete with mockups, all the way down to the CSS classes and javascript behaviors. There’s no right answer on this kind of thing, other than to look at the team, the work, the client and the context. Going too far in the direction of specification ends up being a sort of mini-waterfall process.
Mature agencies can educate their clients about the process, and then structure their contract in a way that is consistent with an Agile team. Typically this means that the budget is scoped for some number of months, and then billed at increments corresponding to deliverables. The contract states business objectives to accomplish, without a lot of detail about exactly what it will look like in the end. The promise of regular deliverables balances out the initial lack of specificity, and the lack of specificity is what gives the team the opportunity to make the most of the resources that they have toward the stated goals.
Large organizations can be somewhat schizophrenic, with Agile teams and Waterfall budgets, driven by a painful budgetary process that is more interested in controlling all of the project variables than anything else. The organization risks motivating the team to build what was asked for rather than what is needed.
Fixed budgets are not incompatible with Agile projects, but the project goals need to be stated in terms of business outcomes without specification of the solution, and governance needs to be adapted to measure objectives. Effectively, the Agile team working against a fixed budget is asked to get as far down the road as they possibly can toward the stated goals within the resources available, keeping in mind the mandate to realize the customer’s competitive advantage.
Let's agree to define productivity in terms of throughput. We can debate the meaning of productivity in terms of additional measurements of the business value of delivered work, but as Eliyahu Goldratt pointed out in his critique of the Balanced Scorecard, there is a virtue in simplicity. Throughput doesn’t answer all our questions about business value, but it is a sufficient metric for the context of evaluating the relationship of practices with productivity.