An introduction to Lean Kanban software development

Digital product delivery is evolving, thanks to the speed of change in technology. Digital teams realise that maintaining a competitive edge today means adapting their processes and methods to effect change faster. Lean Kanban is an increasingly popular way in which digital teams are looking to achieve this.

This guide provides an overview of the Lean Kanban approach to software development, and explains the key benefits it offers. The approach can be applied to both project work and maintenance work, aka ‘business as usual’ (BAU).

What does Lean mean?

Lean Kanban brings together two fields of process improvement called Lean and Kanban. Let’s look at each of them in turn.

Lean development is born from Lean manufacturing, a process pioneered by Ford more than 100 years ago which gained prominence in the late 1940s when it was adopted, and further developed, by Toyota.

Lean manufacturing aims to:

• Reduce waste. This makes abundant sense for a repetitive, repeatable process.
• Increase efficiency. This is typically achieved by buying newer, more up-to-date machinery.
• Eliminate bottlenecks. This is achieved by restructuring the manufacturing workflow.

When it comes to applying this in software development, we need to think a little differently to get maximum value from these underlying Lean principles, and to make the terminology more palatable to digital teams.

What is Lean software development?

Lean software development has the same basic underlying principles as Lean manufacturing, but their application is different:

• Identify potential for improvement. Rather than ‘reducing waste’ (the first principle of Lean manufacturing), Lean software development looks at potential areas for improvement. Unlike manufacturing, the process of developing software can’t be characterised as a repeatable, stable process (even if it can feel repetitive at times), and so the Lean principle of reducing waste must be applied differently in software development.
• Increase efficiency. Applied to software development, this principle focuses on improving the process, rather than the people. As W. Edwards Deming put it, ‘85% of the reasons for failure are deficiencies in the process, rather than the employee. Management’s role, therefore, is to change the process, rather than ‘badgering individuals to do better’.
• Eliminate bottlenecks. This involves identifying steps in the software development process where work that needs attention is queuing up at too great a volume for the work to keep flowing smoothly. The objective is to try and eliminate such bottlenecks where a buffer between steps is regularly too full, and to work out how to reduce this.

Okay, so what about the ‘Kanban’ part of Lean Kanban software development?

What is Kanban software development?

As digital businesses shift away from large, one-off digital projects with defined finish lines, towards programmes of continuous improvement, the general practices of Kanban software development are helping digital teams to work in an agile and iterative way:

• Visualise progress. The scope and progress of current development work is visualised using a Kanban board. Capturing the Git workflow in a diagram is another good example of visualisation.
• Limit work in progress (WIP). The digital team sets limits for their work-in-progress, with the aim of maximising flow and reducing the average time tickets take to complete (we’ll explore this in more detail later in the article).
• Manage the workflow. The use of a pull system means that the team focuses on finishing actions (also explored later).
• Make policies explicit. Policies, such as the definition of ‘ready’, or the definition of ‘done’, should be used sparingly for maximum impact, and can be either visual, or verbal. They’re used to ensure common understanding across both the delivery team and the business stakeholders.
• Implement feedback loops. Transparency and clear communication channels are established through the likes of a daily sync between stakeholders, weekly progress checkpoint, quality assurance (QA), and user acceptance testing (UAT).
• Improve and evolve collaboratively. This recognises that progress is a journey, not a destination. For example, WIP limits are seldom set once. It’s far more common to adjust them as the size of the team and the nature of the work it does changes.

Kanban illustration courtesy of agilemarketing.net

Kanban is defined as a WIP-limited pull system. To understand this, let’s first explore what’s meant by a pull system. The easiest way of doing this is to start by explaining the far-more-common push system.

What is a push system in software development?

In a push system, when you complete a ticket you push it into the next column in your Jira board (or whatever type of board you use for managing the work in progress). But that next column is used for work that needs to be worked on by someone else on the team. As an example, you might be moving a ticket directly from the ‘In progress’ column to ‘Tech review’.

The pros of the push system mean that you require fewer columns on the board, because there are fewer separate columns for queues. And for a tool like Jira, this is a big plus. A Jira board fits to the size of the window it's in, so as you have more columns, the width of those columns decreases, making the tickets harder to read.

On the other hand, just because you’ve moved a ticket along to the next person doesn’t mean that someone is free to work on your ticket. Pushing work into the next column in this way can make it difficult to see the size of the work queue, and how much work is actually in progress as opposed to queuing up, ready to be worked on. It also means that individuals on the receiving end of the column can feel under pressure as the ball is now perceived to be in their court. 

An example of what a push system looks like in Jira

So now we know what a push system involves, what does a pull system in Kanban look like?

What is a pull system in Kanban?

In a pull system, when you complete a ticket you push it into the next column on your Kanban board. But that next column acts as a buffer. For example, you might move a ticket from ‘In progress’ to ‘Ready for tech review’. When someone becomes available to do the tech review, they then pull the ticket into the corresponding tech review column.

The advantages of this approach are that it doesn’t matter if a ticket moves but a person isn’t available to work on it immediately. It’s much easier to see how big the queue is and how much is in progress, and no one’s responsible for the receiving column, which makes for a less pressured working environment.

A pull system allows the team to analyse the time a bit of work spends in the buffer area to help improve the process – without asking people to work harder or longer. The down side of this is that it requires more columns on the board, which is bad for Jira, as we explored in the previous section. As mentioned earlier, the more columns you have in Jira the narrower they become.

An illustration of a Kanban pull system in Jira

Now that we know what the pull system part of WIP-limited pull system means, let’s take a look at what WIP means.

How can you limit work in progress in Kanban?

One of the general practices in Kanban software development is to control and limit work in process or work in progress (WIP). These synonyms are used interchangeably in Kanban to reference how much work is active at any one time.

So why is controlling WIP important in Kanban? Well, we know that when we overload any system, quality and production suffer. And when we overload people with work, completion slows and defects rise; rework increases and increases the load on the system.

Controlling WIP is a lever we can adjust to help balance the development workflow with quality. Also, delivering each item more quickly reduces the cost of delay and can increase competitive edge.

In software development, tickets are a way of managing tasks in the development lifecycle using a system like Jira. There are many possible ways to control WIP, but Jira only readily facilitates controlling WIP with per-column limits as follows:

• Step A. The team sets maxima (and minima) for the columns within their control
• Step B. The team reviews how effective the limits are in practice at, for example, a regular retrospective meeting
• Step C. If the limits aren’t working well the team adjusts them, before going back to Step B

Column limits help digital teams to control the WIP for the entire process. As we have now adopted a pull system, a member of the team can only PULL a ticket into a column and start work when the relevant column is under its WIP limit.

The WIP limits then act together as a lever, and the sum of the column limits is more important than the individual ones. Reducing the overall total will increase the throughput of work done by the team. This needs to be done whilst maintaining or improving quality.

So to wrap up what we’ve already covered, Lean is a set of principles. Kanban, on the other hand, has a set of foundational principles, general practices, agendas and values. And neither Lean nor Kanban is a process that can simply be replicated from one project to the next. 

Now that we’ve looked at all the constituent parts, it’s time to sum up what we mean by Lean Kanban.

What is Lean Kanban software development?

Lean Kanban is designed to drive efficiencies in software development and create a continuous flow of delivery. It uses a ‘pull system’ to optimise the development workflow and places limits on the work in progress based on the team’s capacity. In this way, it emphasises data and resources over assumptions, and balances the development workflow with quality.

So what are the business benefits of Lean Kanban software development?

1. More time to focus on development

Businesses spend significantly less time on planning when they use Lean Kanban software development.

One way is to evolve the desired solution in phases. For example, in a phased delivery approach you might start by building a skeleton, then put flesh on the bones to achieve a minimum viable product (MVP), and finish by expanding that into a minimum marketable product (MMP). This approach allows the development team to focus on the tasks required to achieve the phase goal or milestone.

Lean Kanban does away with assigning story points to a user story, and then breaking down the story into subtasks that are each assigned a time estimate. Instead the epics and features are broken down into tickets that are roughly the same size, or ‘just right’ as Goldilocks would say. The team decides what size they want to aim for, and are free to adjust this as the development progresses.

Having most of the tickets the same size helps them to flow more smoothly across the board. It also means that once a history of throughput has built up it can be leveraged when planning.

Significantly less time is also spent on estimation, because Lean Kanban’s ‘Goldilocks’ estimation process is quicker, easier, and more accurate than task breakdown time estimation.

Scrum is currently the most popular Agile development practice, but, by comparing Scrum ceremonies with Lean Kanban ceremonies in the example below, it’s clear that, with Lean Kanban, the development team has more than 10% more time to get stuff done. In Scrum terms, this equates to an extra day each for the whole team – every sprint!

Scrum ceremonies (2-week sprint)

• Sprint day 1 (7.5 hours):
  • Sprint review (including demo, if possible)
  • Retrospective
  • Sprint planning part 1 (Story pointing)
  • Sprint planning part 2 (task breakdown, time estimates)
  • Sprint commitment meeting with PO
• Example workshop (3.75 hours)
  • Backlog refinement
• Sprint review preparation (1 hour)
• Daily standup (9 x 15 minutes = 2.25 hours)
  • Yesterday’s accomplishments
  • Today’s planned accomplishments
  • Blockers

Grand total = 14.5 hours per sprint, or 7.25 hours per week, which represents 19.3% of a 37.5 hour week.

Lean Kanban ceremonies (1-week iteration)

• Weekly checkpoint aka ‘4 Rs’ (Up to 2 hours) 
  • Review (including demo, if possible)
  • Risks
  • Retrospective
  • Refresh backlog
• Review preparation (0.5 hours)
• Daily sync (4 x 7.5 minutes = 0.5 hours)
  • Walk the board top to bottom (swimlanes), right to left (columns)
  • Any other business (AOB)
  • Optional daily demo

Grand total = up to 3 hours per week (8%).

2. Less risk

Lean Kanban significantly reduces project risks in the following ways:

• Just-in-time ticket definition means that there is less re-work when requirements change. If a full backlog of tickets is created at the start of the project, those tickets need to be deleted or rewritten when the agreed approach or requirements change. The Lean Kanban approach negates the need for this re-work by not having a full backlog of tickets defined upfront.
• The phased approach mentioned above delays commitment to the specifics of project goals. This allows the developed solution to better accommodate the remaining budget.
• Much shorter feedback loops (daily/weekly, rather than fortnightly) significantly reduce the risk of the team going in the wrong direction.
• Weekly risk assessment: risks are one of the ‘4Rs’ and, while this need only be a short part of the weekly checkpoint, it serves to keep risks actively on everyone’s radar.

3. Better predictability

Once development work has begun and the team start capturing delivery metrics, each week’s work will give more tangible information on how well the team is working together.  This enables the team to better predict future throughput based on past experience. As the team gets better at Goldilocks estimation so do the predictions based on experience. This opens the door to probabilistic forecasting using tools such as the Monte Carlo method.

4. Better metrics

An API can be used to extract the necessary data from Jira to build a progress dashboard that has tangible metrics for:

• Throughput
• Average development time, distribution, and probability
• Average cycle time, distribution, and probability
• Overall progress at epic level
• Process improvement potential
• Net flow and predictability
• Project burndown and burnup
• Net backlog growth, with forecasts based on previous backlog growth

As an added bonus, there is less opportunity to negatively gamify these metrics. For example, if the team deliberately reduces the size of its tickets to increase perceived throughput this will reduce cycle time and therefore increase the actual throughput. While it’s not impossible to gamify in a negative way, with the metrics in place it’s much easier to spot occasions of this.

The beauty of these metrics is that they measure the performance, be it good or not so good, of the team as a whole. As W. Edwards Deming puts it: ‘I am not reporting things about people. I am reporting things about practices’.

5. Happiness

Lean Kanban can make a client happy for the following reasons: 

• The client sees new work coming into user acceptance testing (UAT) every day
• They feel part of the daily sync, weekly checkpoint, and the team
• They have access to clear, accurate metrics to understand progress and predict the rate of future development
• By focusing on throughput a business can realise an earlier return on investment or minimise the cost of delay for a feature.  For high-value features this could have a significant financial benefit for the business

And from the delivery team’s perspective, Lean Kanban gives plenty of other reasons to be happy: 

• Less time spent in ceremonies; more time building great software
• Less working through a predefined, detailed scope; more opportunity to design solutions and react to change.
• Less about detailed (story point and time) estimates; more about throughput (Goldilocks estimation) and flow
• A sense of achievement every day

Why does this matter? Well we know that a happy working environment leads to improved quality, better collaboration, and more innovative solutions. And, to quote W Edwards Deming one last time, ‘innovation comes from people who take joy in their work’.

That’s alI for now! I hope you’ve enjoyed this introduction to Lean Kanban and now have a better understanding of this increasingly popular approach to software development, and the business value it can create. Don't miss my follow-up post on implementing Lean Kanban software development.