Everything you always wanted to know about Agile but were afraid to ask
This is a concise summary aimed primarily at people who are working with Agile teams, but are outside them. Loosely speaking, these are "stakeholders" such as change owners in the business, separate UAT teams or Subject Matter experts. This includes managers, planners and senior management to one degree or another.
Audience
You are probably part of a large organisation, have requirements to satisfy business needs, and need a development team to make this happen. The organisation has Agile teams but you are not part of the Agile thing (yet). You most likely work to budgets, deadlines and resource commitments. It is quite likely that you understand traditional “project management” but do not see how it relates to this Agile stuff.
Contents:
1. Introduction
Agile works brilliantly in small companies or companies which have always worked that way (e.g. Google, Spotify). A big company can often see the advantages of Agile and will move their developers that way, but they commonly fail to recognise that this means other parts of the company also have to change in the way they work with these teams. Agileness will expand from the developers but, until you become Agile across the organisation, it is important to understand the relationship between agile and non-agile people.
Agile is a way of working which is efficient, continuously improving, and responsive to change (among other things). It delivers the solutions people actually need. My experience is that people who are 'anti-agile' are often reacting against what they think Agile is – not what it actually is.
First, let's get rid of some mis-conceptions
- Agile is anarchy. Not true. It often looks like anarchy from the outside, because it is non-hierarchical with all team members making decisions and sharing responsibility for them. For this to work well, there have to be very well defined agile processes which everyone knows and follows. They are not always obvious to people outside the team, but if we do not follow these processes, chaos ensues
- Agile does not do deadlines. Not true. Of course it does. What sort of process approach would it be if it could not deliver to deadlines. However, it only accepts 'essential' deadlines. Artificial deadlines created as a way of managing development (a common technique) are not applicable to Agile. We have different mechanisms for prioritising and managing work
- You cannot plan in Agile. Not true. Of course you can - and in a big organisation it is essential. What we avoid is a) planning based on unfounded assumptions b) sticking to one plan when circumstances change. Plans help us manage resources, dependencies and deadlines. They will be quite accurate short term, but deliberately vaguer as we look further ahead. This is because they are grounded in actual knowledge rather than speculation about the future. Saying "We will do X on day Y in 2 years time" is obviously meaningless. It gives an impression of knowledge and planning accuracy which is unfounded
- Progress reporting does not work. It does, if you do it right. Because we are responsive, we are constantly looking at the balance between work requested, resource available and timelines. The work is divided into small units (often called stories) with which we plan in short time periods and then review progress. This period is typically about 2 weeks. This iterative process enables us to provide a sound empirical basis for progress tracking. As part of this we need to encourage everyone to move away from written reports and start learning to interpret live dashboards of progress. These dashboards must be based on stable parts of the agile model - not stories and epics which are always subject to change. In that way we are basing our decision-making on real information, not subjective views
Mis-conceptions are explored more in another article, but these are probably the most common ones.
2. What is an Agile team?
An Agile team is not a team of developers. If you are still thinking about capturing requirements, specifying solutions, then 'throwing them over the wall' to a team of developers, you are missing the point (and the benefits) of Agile. One common example of this is people who try to get all the 'stories' for a project written in advance (often by a third party) and then hand them over to the team. This is just a traditional approach done in smaller pieces. The stories must evolve and change as the team progresses. This is part of the fluidity that makes agile effective.
The core of an Agile team is:
- Product Owner(PO) - A key role. The Product Owner is the primary interface between the development team and the Business. They need skills and knowledge on both sides. They take the lead in co-ordinating requests from the business, planning work (both short and long-term), prioritising activities, escalating issues from the team to the business and more. They are not a Project Manager (because the team is self-managing) but they are as close as it gets
- Developers - In the simplest model, everyone who is not the Product Owner is a developer. These are not developers as you might think of them traditionally. You do not give these people specifications and they program them like a bunch of monkeys. These people are active participants in requirements capture, specification, development and testing. They are also all expert in the agile processes and in contributing to the management and improvement of the team
In addition to these people, you are quite likely to find:
- Scrum Master - Helps the team be Agile. This includes ensuring everyone knows and follows the appropriate processes, advising how to be agile in novel situations, reflecting the behaviour of the team back to the team to assist continuous improvement, helping to unblock any blocked issues. In an organisation this scope needs to extend beyond the Agile team itself to help set the expectations of the business, push back on unreasonable requests, create dashboards to enable people outside the team to see appropriate information and advise them on how to interpret this information. The scrum master will work closely with the Product Owner to ensure interaction between the business and the team is working smoothly and appropriately
- Technical Lead – Often useful is to have one senior developer who represents the others. They may have more expertise (particularly at the architectural level) than the other members of the team, or it may simply be that the team find it useful to have a single technical touchpoint. The Lead helps resolve architectural decisions, ensures consistency across the team approach, works to provide technical advice to the Product Owner and the business, plus drives the unblocking of items which are blocked for technical reasons
These are not essential in a mature team because these roles will be shared between all team members. They are very useful in developing teams.
3. Organising work
Work is broken down into small pieces which are kept, in a prioritised order, in the 'backlog'. This is reviewed continuously (or at least once per sprint) to make sure it remains current. Work can be added, removed or modified. Although the backlog idea is slightly different, you can think of it as a plan. I encourage teams to tentatively allocate backlog items into sprints up to 3 ahead of the current sprint. If you allocated more than that then you would be going back to waterfall and become unresponsive. This process is called "Backlog Maintenance" or "Backlog Grooming" and is led by the Product Owner.
You may hear these things in the backlog referred to by many names - tasks, stories, issues, items, PBI (Product Backlog Item), ticket, Jira. They are not all the same. The three most important are:
- Story – an item which is small enough to be done in a sprint (see below). It must be well defined and testable. It will include a reason for why it is needed in terms of user or stake-holder requirements. It may have other requirements too, depending on how the team chooses to work, but we generally describe stories as "Ready" when they are in a state that a developer could potentially pick the story up and implement the required solution (including testing). Nothing will be picked up by the team unless it is ready
- Task - another sort of item which is small enough to be done in a sprint. The difference between this and a story is that a task does not have a 'reason' within it. It is normally a thing which is needed to support a story, but not something that a user requests. For example, there may be a user story which is "As a user I want my interaction to be secure", but there may be a task like "Implement SSL protocols". A user woud never ask for that task - they do not even know what it means - but it is something that must be done to deliver that story
- Epic – an epic is a way to group stories which achieve a specific objective but are spread over several sprints. It should not be particularly big and it must still be clear when it is 'Done'. Epics are not deliverables, workstreams, themes, workpackages, feature sets or anything else. We have different ways of handling those things
Epics, stories and tasks will be created, modified, removed or implemented over the course of a project. The complete set of epics and stories will not exist until the project is over. Many items in the backlog will not be "Ready". That is natural because developing the stories to readiness is a collaborative process and so 'skeleton' and 'partial' stories are an integral part of that backlog. Helping to bring stories into a state of readiness is an important part of what members of the business do to support successful Agile.
For planning and progress tracking, I recommend using higher level groupings of epics (such as Feature Sets). These can generally be mapped out at the start of the project and should remain fairly stable. They satisfy the progress and reporting needs of most people outside the team. They can also be used for high level estimation as well as managing deadlines and dependencies. They are monitored and updated as the project progresses to ensure that they are as accurate as they can be given the information available at any given time.
We can also group these stories in various other ways. Choosing the best way to group them is an important aspect of setting up a project.
4. Interacting with the team
The Product Owner is the primary interface between the Agile team and the business. Do not approach other members of the agile team directly to ask for work to be done, to change priorities or to get updates on progress. These should all go via the Product Owner, who has overall responsibility for these things. We need to maintain this agile model if we are going to work in a controlled, systematic and effective way. This does not mean that there cannot be direct interaction between members of the business (e.g. Subject matter experts) and the developers about other things. Discussions, conversation, clarification are all part of the Agile process and should be done as directly as possible and with as little overhead as possible - though with the awareness of the Product Owner. Creating stories and making them "ready" is the key platform where these direct discussions can take place.
Almost all Agile teams work on the "Scrum" approach – though this is not the only one. The key to Scrum is the idea of a "sprint".
A sprint is a fixed time period. The team agree what work will be achieved during a sprint and commit to delivering it. They then do that work and (hopefully) achieve it all by the end of the sprint. A sprint:
- Starts with a sprint planning session in which the Product Owner proposes work to the team and they discuss it, agreeing on what they will commit to deliver in the upcoming sprint
- Ends with some sort of 'demo' to the stakeholders to show what has been achieved and get feedback on how the project is progressing. This is generally followed by a retrospective session in which the team look at how they worked in the sprint and see what can be improved or changed
It is not generally acceptable to ask the team to take on new work within a sprint, or to change the requirements for something which is already in the sprint. If this is essential (which does happen occasionally) then technically the sprint is stopped and re-planned. The Product Owner leads in justifying why the new work is necessary and deciding what should be removed from the sprint so the team can still deliver within capacity. Directly approaching the team members with changes is never acceptable.
- Talk to the PO if you want to request work or prioritise and re-prioritise work
- Talk to the PO if it is essential to change the content of the sprint that is in progress
- Work with the PO and any relevant team members to create and 'make ready' the stories in the backlog. This is an ongoing process, but the Product Owner will normally invite you to participate in more formal "backlog refinement" sessions to check that stories are ready and agree priorities across conflicting needs
- Talk to team members as requested during a sprint to clarify details on stories (but not to change Requirements or Acceptance Criteria)
- Participate in the Sprint Demo session when something of relevance to you has been in the sprint so that you can provide feedback to the team
You will not participate in Sprint Planning sessions. The Product Owner acts as your representative during those sessions.
You do not need to know what goes on inside a sprint. Trust the team. If something happens that will affect you, then the Product Owner and/or Scrum Master will flag it up to you