Backlogs - a key component of Agile

Backlogs are probably the most important part of Agile, and also the most misunderstood.

They provide an integrated set of information about what is planned, possible or happening at all levels of a project or organisation. They are also one of the easiest things to get wrong - so they give you no advantage if you do not use them well.

A backlog is not a ‘to do’ list. It replaces long-term and short-term planning. It captures possibilities and ideas. It is not a ‘pipeline’ of work that will be done, but a list of possibilities and proposals. It is a way to monitor progress and continuously improve what we do and how we do it.

It does not matter what approaches to Agile you adopt - good backlogs and backlog maintenance are essential.

Some basic ideas

Except in trivially small cases (like 3 people delivering a very simple thing), we give structure to work by breaking it down into a hierarchy. This is not done (in detail) in advance, but just to the level we can be sure of based on our current data. Hierarchies of work and backlogs are closely related

For hierarchies

• Work - is organised into a hierarchy which reflects the fact that smaller pieces of work contribute to larger pieces of work. This applies to most things, but not everything fits into a larger piece of the hierarchy (e.g. bug fixes do not have to have a hierarchical parent)
• Not everything is in this hierarchy - values, objectives for improvement/change (e.g. {{KPIs}}), organisational structure (e.g. who is doing what) are not part of this hierarchy - though they will probably be linked to it. Getting the right things (and only the right things) in the hierarchy is essential If it is not an activty that progresses through stage over time, them it does not belong in this hierarchy
• The hierarchy evolves over time. We can often do the highest level very early in a project, but we only do the details Just In Time (JIT) to maintain our flexibility
• Different people create and use different levels of the hierarchy, but they must all share awareness of the whole thing and inform each other about plans, progress and decisions at each level
• Putting something into the hierarchy is not a commitment to do it. This is not a pipeline of work - it can include ideas, thoughts, explorations, alternative ways to tackle problems and all sorts of other stuff - may of which will never be 'deivered'.That is a good thing, not a problem
• As we get down to further detail (finer granularity) in the hierarchy, more flexibility happens and more people can contribute. At the finest granularity, anyone can add to the hierarchy

{{Hmm. An article on hierarchies is needed}}

For Backlogs

• A backlog is (typically) a slice through the hierarchy looking at 1-3 levels of activity which are focused on the intended audience (e.g. a team, a department, senior management)
• It consists of a set of ‘items’
• Items can be of different types - they represent possible activities that may (or may not) be done - including delivery work, research, ideas and alternative ways of doing things
• A backlog item can be in different states - from a skeleton of a thought to a fully developed piece of work which we can commit to delivering
• Any item we are working on is no longer in the backlog. Once we have started work on something, it can never go back to the backlog
• We do not commit to delivering any backlog item until it is ‘ready’ for us to commit to (which includes having sufficient detail, an estimate of resource required and other things). What ‘ready’ means depends on the level of hierarchy and the nature of the team responsible for it
• We have a regular and frequent process to look at all backlog items. This includes adding/removing them, committing resource to making them ready, estimating (and revising the estimate) on how much effort they will take, prioritising them and making a commitment to delivering them. This is called backlog maintenance
• Backlog maintenance replaces ‘planning’ in traditional approaches. But this can only work if we have a well structured hierarchy of work. It is a whole team activity - no-one can define a backlog and hand it to a team because the team has to own it and define it themselves at the relevant levels
• A backlog also has ways to structure it which facilitate planning ahead and progress tracking. For example, we will set a priority of work, but in sprint approaches we might (tentatively) assign items to future sprints. We might also schedule work based on d ependencies and (real) deadlines. How we capture data about items so that we can ‘slice’ the data in different ways (and for different audiences) is essential

Why do backlogs matter?

Backlogs are a very participatory way to organise work. They help break down silos of work and, in particular, eliminate the “plan, build, deliver” cycle of older approaches which negate the advantages of being user-centric, value-orientated and responsive.They give us a very visible, shared set of information about what we are planning and doing, and basing that on real information. This all prevents problems that can arise when organisations separate planning from delivery, or fail to be responsive to the needs of their users and stakeholders.

Used properly, then provide value to everyone at all levels, minimise the reporting overhead and ensure accuracy in the information we share about the work we are planning or doing.

{{More on value}}

Some practical backlog things

Here are some practical approaches to making good use of a backlog.

Backlog items

At each level of backlog, and for each audience who will use it, we need to ensure we have the right information and relationships captured in a backlog item to enable everyone to see what is happening and how it relates to other things in the work hierarchy. There can be lots of things in here, but there must be at least:

• A unique identifier - so you can tell the difference between this and other items
• A sensible title - which means something to every audience. Do not use technical terms or abbreviations alone
• A summary of what it is - short and in meaningful language
• A status - which tells us where this item is in the process of developing the backlog
• An owner - does not have to be the person doing the work, but it is the person who is responsible for making this item progress. Without that, things can drop out of the process unnoticed
• Links to where the item sits in the hierarchy - parents and children as appropriate
• Links to other items - e.g. dependencies and risks
• Effort - not needed until we start work on this item, but we cannot start a piece of work until we know how much effort is involved (as an estimation)
• Priority - how important this work is compared to the other work in the backlog

They are only the essentials. There is much more information that we might add, depending on its relevance. This will help people view the backlog in different ways.

Backlog structure

A backlog is not a simple list of things to do. It is much cleverer than that.

As we have already discussed, most work in a backlog has a hierarchical relationship to parent and children items. This enables us to make more accurate assessments of effort and progress. It also helps us relate a piece of work (at any level) to the value it has, the priority for its delivery and the risks and dependencies which are impacted by changes in this work.

There is more to a well structured backlog than a hierarchy of work. We can (and should) categorise items on the backlog in a variety of ways so that it is easier for different audiences to see only the things they need to see in the backlog by filtering it. These will differ for different sorts of work and different organisations, but will often include:

• Responsible teams (matters more at higher levels) - but basically about which team is contributing what to each piece of work
• Types of work (mostly predictable project work vs. demand driven work from users such as bug fixes and support activities) or technical debt (work the delivery team wants/needs to do in the future)
• Status - mentioned above, but distinguishing things such as ideas, things to research, things being refined to and things which are ready
• Departments/regions - who is affected by the work. Important for large companies but, obviously, knowing what is relevant to my department is important for efficient delivery of information to stakeholders

There can be others. These categories are probably more relevant to people outside the team than people within it, but enable efficient communication by allowing people to ‘slice’ the data in different ways and see only what is relevant to them.

Backlog usage and activities

Since the backlog is central to what we do (at all levels) it must be maintained and reviewed on a frequent basis. This is commonly referred to as “backlog maintenance”.

• The whole team must be aware of and look at the backlog which is relevant to them. This should be a team activity to ensure everyone is aware of what the whole team is doing
• All items in the backlog must be regularly reviewed and decisions made about them including:
  • Removing items that are no longer relevant or needed
  • Deciding to do work to refine a backlog item (from ‘skeleton’ to ‘ready’) and assigning someone to do that work
  • Grouping or splitting backlog items as needed
  • Refining the data for the item
  • Propagating relevant changes up and down the hierarchy of work
• Ensuring that there are enough items in a ‘ready’ state (including estimation and prioritisation) to enable the team (or an individual) to choose the next items to work on. This might inform ‘sprint planning’ in scrum teams or work selection in Kanban and Lean teams.

Tell me more

{{To do - links to more MEC details and other resources}}