Evidence Based Management (EBMgt) for Organizations: Concepts

• Concept
  • A process for measuring, diagnosing and improving the value an organization derives from its software investments
  • It improves an organization’s ability to compete on its software capabilities
  • by using evidence to focus investments on areas that will create the highest value for the entire organization, and monitoring their effect on the organization
  • Its iterative, incremental approach and helps organizations control the risk
  • help organizations focus their efforts on most valuable improvements and monitor their effect on the organization
  • with empirical Iterative and incremental approach that helps to control risk
• Empiricism, Hypothesis, and Knowledge
  • is also a pillar of ‘Evidence-Based Management’
  • Inspired by evidence-based decisions in medical practice, in empirical management, evidence is also used for better decision-making
  • In empiricism, experience is seen as the primary -if not only- source of knowledge
  • In a context of high unpredictability and considerable change, information and insights gained from work are infinitely more valuable than any upfront theory, assumption or prediction
  • In the absence of observable work, all preliminary information is to be considered a hypothesis, not knowledge or evidence
  • Without knowledge no informed decision about the future can be made
  • Knowledge is gained when an actual working result can be compared against a stated hypothesis and the observers capitalize on the findings that emerge from the comparison
• Repeated cycles
  • (1) stating a hypothesis, (2) working on it for a limited time and (3) subsequent verification on the doing
  • is a highly dynamic and safe way to gather evidence of what does and does not work
  • enables learning and improving while effectively making progress
• Scrum
  • ‘Sprint Planning’, serves to state a hypothesis, made up of a forecast of work for achieving a business objective, the ‘Sprint Goal’
  • ‘Increment’, is weighed against expectations, objectives and changed circumstances
  • Time-boxing a Sprint limits the risk of needlessly accumulating unidentified variations in the outcome
  • Working on sprints takes away the usual blindfold for reality, yet still allows focus for the people doing the work: Within the Sprint all attention goes to working on the hypothesis, although the actual implementation is open for evolutions through daily optimizations against the Sprint Goal
  • To assure that the reality check at the end of the Sprint (a ‘Sprint Review’ event), results in reliable findings and learning
  • Scrum requires the delivery of a “Done” Increment, no unknown work remain hidden. Undone work is a major unknown variable that strongly degrades the value of the evidence
  • Every Sprint is a ‘project’, an investment in a workable and observable outcome, leaving the option to change course and direction
• Culture
  • Traditional culture:
    • plan-driven, command and control management
    • nobody moves until all is completely known and planned, after which the plan is blindly executed
    • the plan has priority over the changing reality
    • the benefits from Scrum are real, but limited if Scrum is only used to iteratively deliver according to a preset plan
  • Scrum thrives best in a culture…
    • where well-considered experimental discovery is combined with fact-based decisions
    • with courage to act in the moment, to change direction as needed, supported by knowledge and evidence reflecting reality, no matter what that reality or the hard truth is
    • that values knowledge from actual experience more than a predictive plan and self-generated or imaginary ‘facts’
  • Empowering the Scrum process fully
    • might be an indication that management understands that time-boxed efforts deliver more valuable information than any prediction is able to
    • managers embrace to act and decide upon proven results, evidence, and actual facts
    • In return they get a high level of flexibility, while risks are controlled from having releasable work, every 30 days or less
• EBM and scrum
  • evidence is the best-suited base for informed decisions
  • EBM aligns greatly with the empirical foundations of Scrum, but does not in itself require Scrum
  • covers the principles of the overall managerial culture within an organization, not its choice for a specific product development process
  • evidence-based decisions in product development is only one organizational area where a better fit, scrum for product development is great…
  • However, evidence-based decisions in managerial decisions informed by the best available evidence matched with practical experience and appraisal of context, ethics and human values plainly works better
  • E-B decision works better IN ALL DOMAINS OF AN ORGANIZATION
• What Management does
  • sets company goals
  • and to best achieve these goals, provides and promotes
    • organizational practices
    • strategies
    • tactics
  • Their decisions impact the internal organization, users and customers, and -indirectly- an organization’s market position
  • An evidence-based manager observes and verifies the outcomes of delivered work against assumed results to adapt and improve managerial decisions
  • EBM does not inhibit a manager from using experience and intuition, it rather frames it
• Advantages of EBM (by using the Feedback loops which offers updated evidence)
  • improves the quality of managerial decision-making through the integration of current best evidence with practical experience and human values
  • The gathering, appraisal and application of evidence permit to better forecast the likelihood of business success and of achieving company goals
  • It permits organizational optimizations through the ongoing discovery of better practices and the removal of dysfunctional practice
  • helps managers justify decisions about how to organize, structure, invest in and deliver services
  • help managers redesign organizational practices to better suit changing needs and circumstances
  • increases the legitimacy of decisions in the organization and the value a manager represents for the organization
  • provides structure for otherwise seemingly arbitrary decisions and gut-feel opinions
  • limits the risks with a cyclic approach in which evidence is regularly revisited and updated, so decisions can be revised
• The possible sources of ‘evidence’
  • Evidence in the managerial domain primarily serves to support or to reject a managerial hypothesis, an assertion that certain organizational practices are beneficial to the organization, and that other practices hinder or limit the organization
  • Internal
    • organizational data, metrics, internal research, reviews, surveys, measured effectiveness of processes and procedures. Internal evidence requires an unbiased, scientific approach to gather it
  • External
    • books, public research, studies, documented practices, experiments, benchmarks and industry comparison
    • External evidence needs appraisal to the organization’s specific context and values
  • The strength of evidence stretches between 2 extremities
    • Direct or primary evidence
      • the strongest type. It contains direct, clear and indisputable proof or contradiction of the truth or validity of an assertion
    • Circumstantial or secondary evidence
      • much weaker. It might show consistency with an expected result but does not necessarily rule out other, possibly contradictory assertions. Circumstantial evidence might point to alternative explanations
  • Evidence is created and accumulated through observations in controlled conditions
• EBM Applied
  • use evidence about
    • the value the organization actually delivers
    • its ability to deliver value
    • its ability to innovate
  • There is no widely spread and accepted external evidence on the effectiveness of organizational practices in software organizations
  • Each organization picking up on EBM identifies the right internal evidence, at home, from the inside
  • Levels on which Evidence is captured
    • Organizational value
      • Captured through 3 metrics (primary and direct evidence)
        • Current Value
        • Time to market
        • Ability to innovate
    • Departmental value
    • Product value
  • Evidence in software organizations is often and wrongly collected at individual team level
    • via productivity, burn-down charts, velocity or similar derivatives, and other team parameters that are no direct evidence of value of the delivered services of the organization