Waterfall to Scrum: How to Measure the Value of Agility with Sam Falco Agile Coaches' Corner

This week, Dan and Sam are answering another fascinating listener question. This listener wrote in, “We are in an organization that has been going through an Agile transformation. Our leaders have been asking for metrics to compare Waterfall vs. Scrum. They want to know if we are delivering more with Scrum. How do we measure this?”
 
Navigating how to measure values when transitioning from Waterfall to Scrum can be a difficult challenge. How do you measure whether something is a successful initiative or not? What are the key differences in what metrics you should be measuring between Waterfall and Scrum? How do you measure the value that Scrum brings? What are some of the key metrics you should be measuring? Tune in to find out!
 
If you have a question of your own (or would like to share your thoughts on today’s episode), you can email the podcast at Podcast@AgileThought.com or Tweet @AgileThought using #AgileThoughtPodcast!
 
Key Takeaways
How do you measure the value your organization gets from Scrum vs. Waterfall?
Typically what is measured in Waterfalls is “iron triangle” stuff (i.e. “Were we on time? Did we stay on budget? Did we complete the scope?”) However, these things don’t really indicate whether or not you are actually getting value out of the effort
“Too often, what we’re actually measuring in Waterfall was: ‘Did we get all the work done in a certain amount of time and within budget?’ And that’s not what we’re interested in, in [an] Agile effort.” — Sam Falco
Find different ways to measure the value, whether it’s in actual dollars earned or cost savings or goals achieved
It is more important to measure the return on investment and the value it brought to customers rather than “Did we push it across the finish line?”
Measure based on value rather than on velocity
It’s not uncommon for those from a Waterfall world to measure how many tasks were completed rather than delivering valuable increments (which is critical with Scrum)
Regardless of whether it is Waterfall or Scrum, you might measure: generated revenue, saved spending, etc.
Evidence-based management as a method for measuring the success of an Agile effort:
Reference The Evidence-Based Management Guide from Scrum.org
It will provide you with a number of key value areas to help determine whether you are actually adding value, as well as some suggestions for key value measures for each of those areas
Listen to episode 107 of the Agile Coaches’ Corner, “Evidence-Based Management 101 with Sam Falco”
Think about the metrics you have in place now and whether or not they apply now (i.e. “Does this metric we’re using help us determine the current value for our products?”, “Does it help us determine time-to-market?”, “Does it help us determine our ability to innovate?” And if not, ask, “Do we really need it?”, “Are we measuring something that matters and drives results?”)
Measure lead time (if it shrinks dramatically, this is a good sign that this is going to work for your organization)
“Take a look at what metrics you’re measuring now. And if those were valuable in telling you whether or not your efforts were worthwhile, they might be appropriate now.” — Sam Falco
Even though your process is different, you can still measure using a lot of the same metrics — you just have to look at them in a different way
The most important things to measure are around the value (“Is the stuff we’re producing actually valuable to people?”)
What are some of the ways we can measure value?
Quality metrics (Has your quality increased?) — Agility tends to come with increases in quality because of a limit of work-in-progress and helping the teams focus
Look at trends; not single points in time
Have your escape defects gone down? Has your technical debt gone down?
Look at things like, how long does it take you to get a release ready for production? Is it taking less time