Evidence Based Management (EBM) for Organizations: Gathering the metrics

• Gathering the metrics
  • Gathering the metrics for Evidence-based Management in software organizations can be a strenuous task
    • what to collect and from where
  • organizations recognize agility as a means to improve their value, yet tend to lack focus and commitment to moving towards it
    • entrenched traditional ideas and a resistance to change
    • if it ain’t broke, don’t fix it” mentality
    • : “That will not work here”
    • In the successful enterprise its worse
      • “We are fantastic and we know best as we have been doing it this way for many years with success”
  • traditionally measure of success
    • does not take into account how much ‘value’ is delivered to the customer or users
    • they measure
      • delivered on time
      • delivered on budget
      • delivered on scope: with the features that were required upfront
  • try to encourage the organization to start measuring the value that they are delivering rather than how they are delivering it
  • EBM provide organizations with some standard agile metrics
  • Most of the scaling Scrum methodologies out there miss the boat on measurements. Not only that they provide too much rigor in the wrong places for a “one size fits all” solution
  • Useful for
    • Organizations
      • to understand ROI for their hard-earned cash spent on improving their process and practices
      • can use these metrics to demonstrate the value that their investments in agility with training and consulting are bringing them
    • good consultants
      • to understand value that they bring to their customers so they can inspect and adapt as well
  • We need to gather these metrics at a point in time as soon as possible to provide a baseline for where the organization currently is. We can then reassess the organization at intervals, probably no more than quarterly, which will allow us to look at the trends over time
  • Scrum.org suggests tracking many of these metrics and monitoring the trend to help inform your additions towards agility
• Why Evidence Based Management can be hard
  • people tend to ignore evidence when that evidence clashes with their believes or observations
  • So, what do you think what is most responsible? Leader? People themselves?
    • Autor said says it is related to people:
      • teams will bring about great leaders and not vice versa
      • The often-incorrect linking of a team’s performance to the leader’s style or traits is called the “leader attribution error”
      • Changing the team lead for example would probably be a futile action because people can only motivate themselves
      • The desire to do something cannot be impose
      • What you can do is align personal goals with organizational goals by using the employee's own need for fulfillment as the motivator
    • what can we do?
      • create the conditions necessary for great team performance and great leaders to emerge
      • collect evidence about results and act based on that evidence
• Measuring agility
  • organizations in the midst of an agile transformation, they question “how do we know if/when we’re agile?”
  • When quizzed about why they want to be agile, they respond that they “want to go faster”, or maybe even that they “want to be more responsive
  • Speed Matters, but only so much, one candidate: Customer Cycle Time
    • The amount of time from when work starts on a release until the point where it is actually released
    • This measure helps reflect an organization’s ability to reach its customer
    • is important because it helps an organization streamline its means of delivering value to customers
    • however, doesn’t consider the time it takes to gather feedback and formulate a response
    • ignore time spent deciding how to respond by only starting the clock once the decision of what to do has been made
  • A measure that better reflects an organization’s ability to be responsive: Time to Pivot
    • A measure of true business agility that presents the elapsed time between when an organization receives feedback or new information and when it responds to that feedback
    • for example, the time between when it finds out that a competitor has delivered a new market-winning feature to when the organization responds with matching or exceeding new capabilities that measurably improve customer experience
    • reflects all the things that are done between the receipt of information that triggers a need to respond, and the actual response itself, including the time spent deciding how to respond
  • even Time to Pivot ignores something important - the effectiveness of the response, 2 additional measures (called KVAs in EBM) are useful for doing this: Unrealized Value, and the Ability to Innovate
    • ineffective: responses that don’t contribute to reducing this satisfaction gap
    • NOTE: NOT about speed, if deliver frequently, but each release low Q/Q little to reduce the satisfaction gap
      • Low quality: the potential for each response to reduce Unrealized Value
      • Low quantity: few innovations on each response
  • Conclusion
    • SPEED:
      • when measuring speed, make sure the clock starts when you get information that requires a response
      • Starting it when you finally get around to doing something ignores the time you spend deciding what to do
    • QUALITY
      • you need to measure the effectiveness of your product releases, not just their frequency
      • Product releases that don’t reduce a satisfaction gap aren’t effective
      • you need to measure the results to know what worked and what didn’t
    • QUANTITY
      • you need to understand how much of what you do is really valuable
      • People may look busy, but if most of what they do is unproductive then the organization won’t be able to reduce the satisfaction gap very quickly, no matter how frequently it delivers new releases to customers
  • NOTE
    • CCT: time between
      • 1. Work starts on a release
      • 2. Released
    • T2L: time between
      • 5. Idea
      • 1. Build
      • 2. Deploy
      • 3. Usage
      • 4. Learn
    • T2P: time between
      • 1. Receive feedback or info
      • 2. Learn & Idea: time deciding how to respond
      • 3. Build
      • 4. release