Misconceptions about Agile

These are just a few of the most common ideas which people can hold about agile and which can inhibit developing good agile. If you think about it, Agile would not be much good as a way to develop solutions if any of these mis-conceptions were true. This section is not trying to explain the detail of how to deal with these issues, but to make clear why they are issues and why they need to be thought about.

• Agile is Anarchy
• There is no planning in Agile
• Agile is incompatible with large organisations
• Agile does not involve requirements, specification, design
• Agile is a magic fix for everything

Agile is Anarchy

To an observer outside an Agile team, things can look like anarchy. There is no strict hierarchical decision-making and individuals often make their own decisions about what to do next and how to do it. It can look a bit like each person is doing what they want.

In addition, one of the big areas where Agile adds value is by being responsive to changes in user needs and business requirements. Instead of developing a complete, up-front, detailed specification of what is being delivered, we get 'just enough' detail and continuously adjust and improve what we are delivering as we learn more and get feedback from users and stakeholders. This can easily be mis-interpreted as allowing anyone to change anything at any time.

This is not dis-organisation, lack of discipline or anarchy. Teams can work in this way because there are very clear approaches to embracing and managing change in the work and requirements. They look different from more traditional approaches but are an essential part of having a successful team delivering well. While the details vary depending on which agile methodologies are being used in a particular case, there are some key ideas that we stick to:

• Everyone in the team takes joint responsibility for delivering effective and valuable solutions. There is no concept of "I did my bit" so it is "someone else's problem"
• Decision-making is devolved to the most appropriate level (often individuals). Different decisions may be devolved in different ways
• Everyone shares information with the rest of the team and knows when to escalate decisions to larger groups
• A team plans work and reviews progress frequently (typically in 2 week periods) and at a fine level of detail. Everyone should be aware of what everyone else is working on at any given time
• Teams get frequent feedback from users and stake-holders and adjust what is being delivered accordingly
• Any proposed changes must be justified and assessed by the team to understand their implications
• Teams work to a capacity. They make commitments based upon that capacity. If any changes are proposed which could impact that capacity (e.g. additional work, changes to structure, changes to priority) then the team will review the implications and make adjustments (e.g. changing current commitments, re-prioritising work items) accordingly

This works because Agile teams are self-managing and working towards shared goals with shared responsibility. No single person can over-ride the decisions and commitments that the team make together. Teams will have clear processes in place for refining work requirements, planning ahead, making near-future commitments and managing change. It is essential that stakeholders working with the team understand these processes and work with them. This is how we maintain well-structured and responsive working. The result is an effective, distributed and structured system for planning and delivering solutions.

Teams are normally supported in achieving these things and keeping on track by a few 'special' roles - such as Scrum Master or non-scrum equivalent (representing good Agile working), Product Owner (representing the business), Technical Lead (representing good techical decisions such as architecture and standards). These roles are not the same as managers in those areas making decisions on behalf of the team, they are roles which inform and guide the team in delivering effectively. Events such as 'stand-ups', 'planning sessions', 'review sessions' and 'issue-based workshops' are examples of how we support these team processes.

There is no planning in Agile

There are rare occasions where a single team is working on a 'blue sky' project with no constraints on time, resource or what is delivered. For everything else, large scale planning is essential.

Fine grain planning for the immediate future is handled by the team as described in the previous section. It is part of being self-managing. There is no need for anyone outside the team to look 'inside' the team processes for that level of planning unless something goes wrong. Trust the team to do their job or escalate an issue if they cannot resolve it. This is much like "Management by Exception" - a technique that has been around for almost 100 years. It is also important to note that the fine-grain units of work a team uses (e.g. stories, epics, tasks) are not the appropriate things to use when dealing with higher level and longer range planning. These small work items are continuously being created, modified, split, joined, resized or abandoned. That is a natural part of agile processes and good work management.

In any but the most trivial case, higher level plans are needed. These plans can be used for many things including:

• Identifying deliverable outcomes and setting timescales for the work
• Managing resources
• Managing dependencies between teams or pieces of work
• Facilitating tracking of progress and early identification of any issues which may impact upon delivery

For a stakeholder, the goal of these plans can be concisely summarised as:

"I want to know that the solution being delivered is appropriate, will be delivered in a timely manner and will be delivered within the available resources"

This is all essential stuff. No-one who has worked in a large organisation would question the necessity for doing these things. Agile happily embraces them, but applies Agile thinking to look at them from a different perspective. In particular by being responsive to change, recognising that a complete up-front specification of a solution is rarely valid, and making good use of the team expertise lead to achieving these things in different ways than a traditional hierarchical and waterfall approach. Here are some key differences.

• Involve the team(s) right from the start of planning. Plans should be based on real information and expertise - and the best place to find that is in the teams themselves
• Use real deadlines (where essential) and plan to deliver within those. We do not use arbitrary deadlines as progress tracking mechanisms. That is handled in a different way
• Do 'just enough' planning at each stage and refine the plan as the work progresses. The plan gets more accurate over time.
• Go into 'just enough' detail to answer the sort of questions above. There is no advantage in trying to go into all the detail up front (and quite a lot of disadvantages)
• Set a scope for the work early, at a high level. Subsequently we add more detail, but this scope should stay stable in almost every case
• Planning and progress tracking is done in large, stable units. Examples include 'feature sets' or 'functional groups'. These are at a higher (less granular) level than stories, tasks or epics
• Use plans to estimate effort. This is not the same as setting a timeline. Estimates of effort need to be combined with knowledge of the capacity and velocity of actual teams before you can create timelines
• Progress can be reported and measured against the high level items on the plan. They are not 'workpackages' though - so do not expect them to be completed in sequence. The team may work on several at once or jump between different pieces of work in whichever way is most effective
• Agile plans are subject to revison and refinement as information improves and a project progresses. They become more accurate as the team(s) complete more of the work and understand the remainder better. This is preferable to having a detailed, up-front plan based on pure guesswork and constantly trying to justify why delivery is not according to the plan

The above approach can satisfy the need for forward planning in the organisation without inhibiting the flexibility and effectiveness of Agile. Failing to do any of the above will reduce the agility and effectiveness of the teams.

Agile is incompatible with large organisations

Much of the early work on Agile focused on small teams working independently. The team takes responsibility for everything that they do. We want to maintain this as much as possible, but when a team is part of a larger organisation and/or set of teams working together there are clearly interactions between an individual team and the wider context. It is necessary to balance certain areas between centralised/organisation-wide control and monitoring and the devolved responibility of the teams. Areas where we have to do this might include:

• Setting and maintaining organisational standards (e.g. solution architecture, design consistency)
• Compliance - areas such as data privacy, security and regulated activities in each sector
• Business values - Agile focuses on delivering what is of value. In a large organisation we need to capture the idea of what is valuable (to the business and to the users) early on so this can inform planning and delivery by teams
• Dependencies - teams need help with the 'bigger picture' of what is going on and particularly in managing pieces of work involving multiple teams with dependencies between them which affect delivery
• Escalating Issues - sometimes issues arise which have implications outside the team or which cannot be resolved at the team-to-team level. When these things happen we need a mechanism to escalate these issues and respond to them at an organisational level

Each organisation needs to find the best way to deal with these things while retaining the value that Agile adds. Deconstructing the existing hierarchy is one of the most difficult areas of change for a large organisation. There are examples of it being done successfully at highly regulated organisations (e.g. finance sector) and traditionally highly centralised organisations (e.g. central government), so it is certainly possible in pretty much all environments. If these sectors can do it, anyone can. The key is to work out what needs to be managed or monitored centrally and keeping that to the viable minimum. Distinguishing what needs to be done centrally, as opposed to what is currently done centrally is a significant change in thinking.

Here are a few aspects to consider:

• Devolve - it is essential to find a way to devolve power and responsibility from a central and controlling structure. However, it is also essential to ensure that the right things are devolved in the most appropriate ways so that there are benefits from the changes. That involves a lot of understanding of the organisation
• Develop trust - the organisation has to develop new models of working in which teams trust the more central roles and vice versa. This has to work both ways. Nothing is gained by devolving things and then retaining the old working practices as well (which some organisations try to do as a 'safety net'). Removing the older practices is essential but can only work when mutual trust develops. This does not happen overnight. A plan for building that trust in new ways of working is integral to the success of moving towards an agile organisation
• Do "just enough" planning and reporting - the key is that each part of the organisation needs to understand just how much information it needs and when. As we devolve and build trust, fewer of the older planning mechanisms are needed, and many that stay in place can become much 'lighter touch'. Done well, this can bring significant benefit to the organisation and everyone within it. Similarly, good tools to support agile enable us to have a "single source of truth" about what is going on at all levels. This means we can make use of reporting based on actual data - an empirical base is important for agile - and live dashboards. This contrasts with spending time creating hierarchies of manual reports which can become progressively more abstract and distant from reality in the organisation

Agile does not involve requirements, specification, design

At one extreme we have very traditional models in which reaching a solution involves capturing all requirements, then specifying a solution in full detail, then giving that specification to teams to develop, and then testing it. There are lots of reasons why this is ineffective and sometimes plain wrong.

At the other extreme, we can imagine a model where we do not do these things at all, or do them only at the last minute as we are building a solution. This is a caricature of what some people imagine happens in Agile. It is not how Agile really works.

The key is to make decisions at the latest sensible time. This is the time that satisfies the needs of all the participants in the most effective way. It is different for different decisions on each piece of work. Requirements, Specification and Design are all part of this process.

• We need to know what we are trying to achieve right at the start of the project. We can capture this by putting effort into developing the scope for our project. This does not mean that we try and work out all the details of everything in advance, but just enough for our initial purposes of estimating the effort in the work and what we expect the solution to do in high level terms
• Early on, we are likely to have to identify some more detail in what constrains our solution. Major architectural decisions, standards to comply with and interactions between parts of the solution at a high level are examples of what might be needed here. We make an initial pass but they are all subject to revision up to a point where we make an organisational commitment to them
• Value of work can be determined by a team in small groups. In a large organisation we need to capture some idea of value to the organisation (business value, value to customers). This can then be used to help guide the teams in making value-based decisions around what to do, when to do it and how to do it
• After these early stages we work in a short cycle of iteration. Requirements capture, specification, solution design, development and testing all happen concurrently in small units. We engage closely with our stakeholders in each iteration. This gives us the advantage of being able to adjust the solution as we go along so that we: deliver what is actually needed; work effectively; build stakeholder engagement; can respond to changing environments. These are all key benefits of being more agile
• As we progress through these iterations we understand the issues and the solution better. This enables us to re-examine and refine our plans continuously throughout the project. It keeps us grounded in real information about the work so that we always have the most accurate possible view of progress

Agile is a magic fix for everything

While most of the above mis-conceptions concern negative views about Agile, it is also sometimes the case that people go the other way and 'over-sell' Agile as being the solution to everything. This is not the case. Deciding to become Agile is the start of a journey, not the end. It requires thinking about the specifics of a given organisation, team and product and working out the best approach to realise the benefits in that context. It is also an ongoing process of continuous improvement and learning. There is no 'end point' to an Agile journey.

There is no single 'right way' to be Agile, but there are plenty of wrong things that should be avoided. These are some ways organisations reduce or nullify the benefits of a more Agile approach:

• Using the words of Agile, but not making the changes in thinking and process to turn those words into real changes and benefits
• Imposing Agile top-down rather than helping it grow and evolve in the organisation. Doing this prevents effective ownership by everyone and true adoption of Agile as an ongoing process change
• Copying an Agile approach wholesale from somewhere else. Someone else's Agile is not your Agile. Be inspired by stories and approaches from elsewhere, but look at how they relate to your context. Do not assume duplicating them will bring success
• Changing people's titles but not re-educating them and redesigning their roles to be effective
• Trying to retain a traditional hierarchical structure of control and reporting on top of the Agile approaches
• Deciding Agile is a change for 'them' (often just the development teams) not needed for everyone. It is not like that. Effective change requires the whole organisation to start thinking and understanding in an Agile way. Even parts of the organisation that are not becoming Agile need to adapt in order to work effectively alongside those parts that are more Agile