This article revisits, revises and extends the original text from 2014, updating it to include the topic of dependencies. 

Software quality is often related to batch size and the time taken to complete a piece of work. The relationship is known to be non-linear, i.e. the defect rate rises faster and accelerates away as the batch size or length of time extends. Hence, smaller batches, completed in short periods of time lead to dramatically fewer defects. So, in theory, small batches of work are desirable.

As a result of all the advantages of smaller batches – higher quality, more frequent interaction between customers and the delivery organization, and the potential gains from earlier delivery of valuable, working software (also known as the avoidance of opportunity cost of delay), organizations have been under pressure to pursue shorter and shorter sprint lengths.

On the face of it, smaller timeboxes and hence smaller batch sizes for the sprint backlog, are a good thing. However, smaller timeboxes create three types of pressure that are often difficult to cope with and adjust to: firstly, smaller batches require an ever more detailed approach to requirements analysis and development – the need to write ever more fine-grained story which can be completed within the smaller time window; secondly, the need for an ever more accurate approach to estimation, so that a realistic commitment can be made; and finally, the need to know whether a piece of work will be affected by dependencies, and if so, a need to track and manage such dependencies, between teams and multiple sprint backlogs, and potentially, across sprint boundaries. A failure at any of these three things causes the timebox approach to break down and fail spectacularly. The timebox approach is inherently fragile on a large scale. Let´s examine why…

Developing capability with a new and hopefully better requirements analysis technique, designed to provide the granularity to ensure work items are sufficiently small as to fit neatly into a short timebox, is hard, even when you know with confidence which technique you should adopt. The challenge with very short sprints is that there is, still after 20 years of Agile, little or no solid guidance on writing fine-grained, consistently small user stories. Even if there is a good method that your team is comfortable using, then (a) you now have a lot of analysis upfront, before sprint planning, and (b) you have introduced a peer-to-peer, or parent-child dependency management problem, that becomes especially acute as soon as you are unable to select all of the peers for the same sprint, with the same team.

Fine-grained requirements analysis coupled to short timeboxed sprints introduces a dependency management problem to Agile methodologies.

Alternatively, in the absence of strong guidance on requirements analysis, anxiety leads to undesirable effects such as breaking stories up into functional units based on information discovery activities such as “architecture story”, “design story”, “development story,” “test story” where the real customer value may now span across multiple sprints and peer-to-peer dependencies between stories across sprints is now a tracking requirement. This type of breakdown defeats the purpose of the ever-smaller timeboxes and creates a false sense of agility when in fact, customer value and quality are not improving, perhaps the opposite may even be true. A failure to instrument for and track customer lead time disguises the problem, and Agile teams focused locally and only on themselves, merrily ignore that they are not delivering customer value.

The concept of a ¨Design Sprint¨ (or other similar, upstream, pre-commitment information discovery) is now a recognized thing and you can take classes in it. Be under no illusion, design sprints are an anti`-pattern, they are an indicator that timeboxed increments in Agile methods are a dead-end – they do not scale! To be Agile you need to be small – small teams, small products, small codebase. To scale, you need to abandon agility and use much larger timeboxes.

 

As sprints become shorter and shorter, the challenge of knowing whether a work item can be completed within the time limit becomes more and more important. Consequently, over the last 10-15 years, I have watched on the sidelines as the Agile community has pursued ever more elaborate means of estimation. The shorter the timebox, the more upfront effort is required to estimate whether the work will fit into the available time. Big estimation up front, is an anti-pattern and destroys the concept of agility – you need to estimate everything in your backlog first to play some magical game of Tetris trying to fit odd-shaped, fine-grained requirements into the sprint backlog. As we shrink the timebox, we increase the economic costs (the transaction costs of sprint planning) and decrease the efficiency of sprints. It is neither Agile or Lean.

What if a story in our sprint backlog gets blocked because of a dependency? That might prevent it from completing on time. And hence, the shorter the sprint timebox, the greater the need to identify dependencies upfront. This requires yet more analysis of as yet uncommitted items in the product backlog. And what if there are dependencies? Agile methodologies resort to organization design to solve this – the magical cross-functional team. The idea that everyone on the team can do everything that might be required to produce a top-quality product. For any firm that aspires to market leadership, this has never been true – you don´t win at anything with generalists, winning requires specialists. A small band of elite generalists can and does happen but again it doesn´t scale. You don´t staff a 500 person IT department, or a 600-person business unit with elite generalists.

All too often we see organizations, like one of those in a case study we use daily in our training classes – they had 5 development teams, 3 data-base administrators, and 2 user-experience designers. So either, you put the two UX guys in each of 2 teams, and the 3 database guys in the other 3 teams and now you have 2 front-end teams and 3 back-end teams, and you try to plan the sprints such that those teams only pick front-end or back-end stories, and oh whoops, you are disconnected from the customer requested functionality and have created dependencies, or you accept the truth and you use the dba´s and the UX people as shared services, and once again you have dependencies.

Dependencies are a fact of life in all but very small-scale software development, or mediocre development done by largely dilettante generalists. If you are doing anything at scale, or anything that aspires to market leadership then you have specialists, and you have dependencies. The Agile movement has been in denial of this basic truth – to be Agile you have to aspire to be mediocre!

The Scaled Agile Framework (SAFe) approach to dependencies, is to reverse agility and instead use 3-month timeboxes called product increments. Effectively rolling the clock back to 1994, a big planning up-front approach, involving big analysis and big estimation up front – and lots of red string – to produce incredibly brittle plans that are unlikely to survive the first few of the 13 weeks intended. Scaled Agile Framework and its product increments and PI Planning quite simply aren´t Agile. To cope with the challenges and anxieties of agilists everywhere, it simply gives them permission to stop being Agile and do everything in quarterly time-boxes. It is an elegant solution but it isn´t Agile, rather it is the antithesis of Agile and the antithesis of the motivation for the lightweight methods movement of the late 1990s that evolved into the Agile movement.

If you want to be Agile beyond the team level, beyond maturity level 1, then you have to remove the constraint of timeboxes to control batch size. A time constraint is a fantastic way to take a chaotic organization from the anarchy of maturity level 0 to the control and predictability of maturity level 1, but that is where it ends. Time-constrained sprints are an evolutionary cul-de-sac, a dead-end. You cannot scale Agile using timeboxed sprints!

The answer is to focus on quality and short delivery times, by using a WIP constraint, rather than a time constraint. Only the Kanban Method offers this for organizations struggling to scale Agile.

I am now convinced that this pattern, taken from Kanban Maturity Model 2nd edition page 133, is a necessary step to large scale agility.

For each Scrum team, leave everything else alone, except you remove the sprint timebox – stories started dont have to finish within 2 weeks. Instead of limiting the time, limit the WIP, use a constant work-in-progress (CONWIP) limit. Initially, replenishment cadence and delivery cadence can be left synchronized with the former sprint cadence, eventually, you are likely to tune these to the needs of the business domain, the transaction, and coordination costs of holding the meetings, and the arrival rate of new information, that affects selection, sequencing, scheduling, and priority. This is your first step on a journey of evolutionary change, and your first step on the road to large scale business agility.

This change frees you from the tyranny of the timebox, and its three dysfunctions of upfront analysis, excessive and costly estimation, and heavy-weight dependency management.

If things take, however long they take, then there is no need for upfront analysis to break an item into smaller items, no need to track complicated dependencies between sprints or across teams. Instead, let the work break down happen naturally once the work is committed for delivery. Track that work with a 2-tiered kanban board, parking lots, and avatars showing the involvement of shared services.

There is no need for awkward elaborate, big analysis upfront, estimation efforts. Simply track historical lead time through your workflow and use the lead time distribution chart to determine probabilistically how long something might take to complete.