Measuring the ability of Scrum teams to create value
Suppose that Scrum teams do not produce real value for the customer as measured by CV or UV. What should they do? Should they immediately focus on releasing lots of new product developments that can deliver value to customers? In short, the answer is probably not, and for many organisations, clearly not. Why is this the case? The answer is connected to the delivery speed and delivery efficiency.
If a team can't deliver potential improvements to CV in quick succession, it can't get feedback from customers and it can't respond quickly to feedback. It is working almost blind to the value it delivers. For the team to improve its value delivery performance, it must first improve its ability to deliver value. This has two aspects: the speed of delivery and the team's effectiveness in delivering on each expenditure.
Measuring the transport speed
The delivery speed (also called Time to Market or T2M in EBM) is measured by a variant of cycle time. Most organisations, using an agile approach, are almost exclusively concerned with speed of delivery. Delivery speed is certainly important, and organisations are rightly concerned with improving speed, not because it is valuable in itself, but because faster T2M enables the team to deliver new capabilities faster, measure their value, and adapt based on the customers’ feedback.
The most comprehensive measure of T2M is the time taken to process feedback, as it measures the time it takes to get from the idea of improving value to the point where the team can validate whether the idea has indeed improved the value experienced by the customer.
Less comprehensive metrics such as speed of delivery, or the time it takes from the start of the team's work to the point where it is available to the customer, ignore the discussions and analysis of customer needs early in the delivery process, and the time it takes to gather feedback. Measurements that ignore brainstorming at the beginning and feedback gathering at the end can give a false sense of 'speed' as important parts of the feedback loop, such as feedback evaluation, are left out of the measurement.
For a team to improve CV, they need to be able to test new ideas quickly in order to know that they are working on things that are valuable to customers. Teams with slow T2M will spend too much time on things that are not valuable and will not be able to quickly improve on the current value.
It's worth seeing that simply delivering quickly is not enough; teams that don't measure the value of what they deliver can't benefit from quick feedback loops because they don't get feedback. Teams that focus only on fast performance but not on the value of performance are fooling themselves that they are working agile. Agility involves responding quickly to feedback, because without feedback a team can never improve. If a team ignores feedback, it is like delivering in long cycles, because most of what is delivered is likely to become waste.
Measuring the effectiveness of Scrum teams
But speed is not everything. To get a full picture of a team's ability to deliver, we also need to look at the impact of the release on the actual score. Measuring the change in CV resulting from a release is ultimately the most important, but the time required to measure it and the many factors that influence it can obscure easily noticeable improvements.
Let us illustrate this with an example. Consider two teams: one with a cycle time of one month and one with a cycle time of two months. It would be tempting to say that the first team performs better, but consider an additional piece of information: the first team's product releases consist almost entirely of bug fixes and minor improvements that do not change the CV much, while the second team's releases consist almost entirely of new product improvements that significantly improve the CV. Which team performs better?
The example illustrates the importance of efficiency in improving the value provided by a product, also known in EBM as innovation capability or A2I. The ability of Scrum teams to focus on improving the customer experience, avoid interruptions, and avoid multi-tasking and non-value-added discussions all contributes to efficiency. That's why this team will perform better than a team that focuses only on speed. Speed is important, but only if the customer experience improves with each release.
To quickly improve the current value, Scrum teams must have a fast T2M but also a high A2I. They need to deliver in fast cycles, but each release should also include valuable capabilities that customers value, or at least thoughtful attempts at how the team can improve the CV of the product. The Lean Flow metrics mentioned above provide some insight into where the team can improve the working flow and reduce the waste of time and resources caused by inefficient ways of working, but measuring efficiency is often much simpler:
How much time do team members spend in meetings? How many of these meetings are related to a particular product versus other organizational needs? How many times a day do they interrupt their work to do other things?
Improving team performance for Scrum teams
Deciding what to focus on can be a challenge for the teams. They'll always be under pressure to deliver more product development, but if they can't measure what they're delivering in fast cycles, there's a very high chance they'll produce features that don't help them improve their delivered products.
In this model, although value (improving CV and reducing UV) is the key to achieving strategic goals, if a team cannot test new ideas quickly, they may focus on poor product development. To improve their ability to deliver value, they should first focus on improving A2I and T2M.
So focus on improving A2I first, as this will improve the team's ability to focus on improving T2M and ultimately CV/UV. Things that can help improve A2I include:
These improvements can also improve T2M by reducing interruptions and allowing the team to focus properly. If T2M is still not fast enough, the team can try some things that can help improve delivery speed, including:
As the A2I and T2M of Scrum teams improve, they will be more effective in conducting experiments focused on improving the value they deliver, as they will be able to gather and act on customer feedback more quickly. These experiments typically focus on testing possible solutions to meet customer needs and gathering information that will help them better understand the customer, their desired outcomes and their current satisfaction.
Conclusion
Many organisations are pursuing agile approaches because they are interested in improving speed of delivery; they instinctively feel they need to be more responsive to changes in customer preferences and threats from competitors. While speed of delivery is important, focusing solely on speed overlooks the greater opportunity that lies in building deeper relationships with customers. The real benefit of improving delivery speed is to increase the frequency with which customer reviews are measured.
In addition to speed, many organisations are also focusing on increasing team productivity. While well-intentioned, this focus is also largely misdirected. Instead of looking at how much output the team produces, they should focus on how much value the team produces. Because from the customer's perspective there is no link between output and perceived value.
To better understand unmet customer needs, teams need to be able to quickly run experiments on value, gather information on whether those experiments met customer needs, and then quickly adjust their computational plans. In this context, it is not the speed of delivery that is important, but the speed of learning. Teams can improve their performance in this respect by focusing on reducing waste and interruptions that distract them from the learning objectives they have set, as expressed in the sprint objectives.
Over time, achieving these value-oriented, learning-driven sprint goals will help Scrum teams move towards intermediate goals and ultimately towards strategic goals, which usually translate into meeting customer needs.
