Strategyzer webinar learnings: Designing experiments that matter

Strategyzer put on a free webinar to discuss how to design business experiments that matter, with David Bland & Alex Osterwalder providing the discussion.  I stumbled on these guys after being recommended the book 'Business Model Generation' (, and subsequently reading the follow-up book 'Value Proposition Design' (my present CTO is encouraging VPD practices) - plus using many of their resources.

(Will add a link to the recording here when it arrives...)

Big lessons

  1. When companies start seeing failure as success, culture is moving.
    • e.g. I am so glad we learned there was nothing in this idea; we saved millions of investment dollars
  2. The ramifications of incentivizing output instead of outcome
    • Current software development methods reward output (deliver by a certain date, deliver a feature or product) instead of rewarding the outcome
    • This leads to focusing on building instead of learning
  3. Learn / Build / Measure loop
    • Start with learn!  Almost everyone starts w. build
    • Tightly couple learning with building


  • Output vs. outcome:  How does this fit into or change the Agile w. DevOps view?
    • How do you balance getting things done with experimentation?
      • Is 'getting things done' a preconceived notion that traps us?
  • What needs to change to allow (enforce?) 'experiments with evidence'?
  • Fear of failure is an easy theme here - how, practically, can this be addressed and mitigated?

Other notes

  • Are you focusing on building? or learning?
  • Are you throwing away things you've built?
    • If not, the things you are building are too expensive!
  • Lean is about deferring commitment
    • Be crafty/scrappy/creative in your experiments
    • Gets very hard to throw away things that have a big investment in infra/code
  • In environments where regulatory or legal barriers are present - don't assume commissions/legal depts automatically want to shut you down
    • Talk to them!
    • Find out what you can and can't do

Design thinking notes

  • Three parts that must not be separated - they inform each other:
    • Desirable - do they?
    • Viable - should we?
    • Feasible - can we?
  • Multi-disciplinary teams help keep them together
    • e.g. Engineering, sales, business, etc
  • Desirability
    • Start with customer interviews in teams of two
    • Landing page test
      • Use the words of the customers
  • Pricing tests also should not be ignored - treat the same way

Further reading

  • Express Executive Education -
    • Despite the name, a lot of valuable lessons for anyone trying to make things better
  • (added to my future book list...)


Popular posts from this blog

Learning through failure - a keyboard creation journey

Learning Opportunities - Watching/listening list

DFSR - eventid 4312 - replication just won't work