Agile Coach Online

Expert support on demand

Agile Coach Online Logo

Whatever happened to… requirements, specification etc?

There are certain things that are essential in every software project such as: understanding what is needed; building something that solves the need; ensuring the solution is fit for purpose.

If an approach failed to do these basic things, then it would not be a valid approach to developing software. So it should be obvious that these things are all part of Agile or it would not be as successful as it is. In fact, Agile was introduced to make all these things better than traditional waterfall approaches. People who complain about Agile are often complaining because they do not realise how these things are handled now. Stuff might look different, but the results are more effective than waterfall approaches. As with much of Agile, a better understanding of what it is really about can help alleviate fears and concerns.

So, let’s start with a few terms that are familiar to everyone who has ever been involved in traditional waterfall software design.

Note that I am not saying that any of these things are bad or should not be done. They have evolved over many years as a way of developing software. The single biggest problem is that these things are done in sequence (which is why we call it waterfall) with limited opportunity to revise and adapt. There are lots of other issues with the approach. You can find plenty of articles about the advantages and disadvantages of waterfall, so I do not need to cover them again here

Now let's add a few words from project management

All those things (or equivalents) are present in Agile. Is that a surprise? If they weren’t possible in Agile then it would not be an effective approach. These things are handled in different (and generally better) ways, though. A big part of the purpose of Agile was to improve these things. Older approaches were designed when computer time was expensive and people were cheap. Since the 1970’s this balance has changed completely. There are other factors too, but that is sufficient for this article.

Let's explore a few of the issues arising from these approaches before we look at how Agile can help us deal with them

1. Issues for traditional approaches

Perhaps the hardest thing for people to accept about traditional approaches is that those approaches were often dependent upon guesswork, wishful thinking, concealment and sometimes downright lies.

So how does that relate to Waterfall and Project Management ideas?

Waterfall

In the waterfall list, the basic problem is assuming all these steps are sequential. Even a small amount of thought shows that this does not make sense. It could take months or even years to create the documents for each stage. The likelihood that you have accurately maintained information over that time, so that you deliver what was originally wanted is vanishingly small.

The most obvious issue is ‘requirements’. Can you really capture everything that is needed for a solution in one go without building anything or showing options to stakeholders? The answer is no for anything except the simplest projects, so we had a proliferation of huge ‘specification’ documents which tried to cover all the bases and were never possible to read and implement in detail. There are times when up-front requirements can work – e.g. making an interface between program A and program B or duplicating the functionality of legacy system Z with new technology X - but these are rare occurences. Sadly, a lot of companies – especially big consultancies – still promote up-front requirements documents so they can do the work and get it signed off (so they get paid) even though it is if limited real value.

Project Management

For the Project Management side, I blame MacProject (well, not exclusively). It came out in 1984 as a graphical way to plan projects on the computer, with visualisations like Gantt, PERT, timelines and so on. Microsoft Project followed 6 years later and eventually became widely used. The problem with these tools is that they give you very nice professional looking views on a project, so it is easy to think what they say is ‘true’ rather than just a guess. You can put bad information in and still get a realistic looking (but not actually realistic) project plan. These tools create a distance between the actual work and the management – and the further the management team is from the actual work, the more they are dealing with a ‘fantasy’ work model.

Even without those tools, there is a problem. You can produce monthly reports, charts for timelines, Gannt and PERT, RAG status lists, Risk Assessment matrices and so on but they are problematic for 3 obvious reasons:

  1. Bad data makes visualisations unreliable
  2. Data filtered up a hierarchy loses context and becomes mis-interpreted (e.g. a ‘maybe’ date somehow becomes a definite commitment)
  3. In middle management (particularly) it is possible to lose focus and think that the important thing is to deliver the right graphs and reports, not the right project

Sorry, but they are all real and happen a lot. I am not trying to destroy the idea of project management or question the value of staff in these areas. It is just that, as with other aspects of software development, there are potential problems that arise and which Agile tries to address by changing some of these processes.

2. Agile is Empirical

The examples above illustrate part of why Agile was developed. We knew these things did not work. They were not compatible with modern IT Infrastructure, were based on unrealistic models and encouraged passing off responsibility rather than owning what you do. They led to poor and uninformed decision-making, poor planning, poor progress tracking and a permanent ‘fire-fighting’ model of development.

A key driver of Agile approaches is making empirically based decisions - based on hard facts and sufficient information rather than opinions or 'gut feelings'. Many aspects of Agile follow from this one basic point.

Here are a few common things to all Agile approaches:

An Agile team is not a team of programmers. They do not write stuff to a specification and hand it over. They are integral in every part from initial concept to supporting the live product. They have many skills which are not ‘code monkey’ work and participate in requirements, specification, design, testing and delivery. Mostly all at the same time. An Agile team has devolved responsibility and can make many decisions for themselves. If you are not Agile yourself, then you still have to learn how to work with these teams, because it redefines what you do

Let’s revisit those old words and see how they relate to Agile.

3. Agile approaches to Waterfall issues

All these things still happen, but we recognise that is was wrong to do them sequentially, so we do just enough up front and then continuously refine and improve them as the work progresses.

4. Agile Approaches to Project Management

Project Management looks different but still deals with the same essential issues as it always did. Learning to think in this way is one of the hardest things for people in ‘traditional’ roles, but is also an excellent path to career enhancement.