ENABLING

INNOVATION
Using Lean Startup &
Innovation Accounting
LUKAS FITTL
@lfittl

spark59.com
Lean Startup is trademarked by Eric Ries and used with permission.
Business Model Canvas is created by Alex Osterwalder and licensed under CC-BY-SA.

About Myself
Founded 2 tech startups,
worked with many others

&

Spark59

Innovation in an
Agile Environment

Waterfall

Risk

D
L
I
BU
D
L
I
BU
D
L
I
U
B

Time
Release!

Agile / XP / SCRUM

Risk

BUILD

BUILD

BUILD

BUILD

Release! Release! Release! Release!

Time

Are we making our

product
better or worse?

Are we efficient?
vs

Are we effective?

The Definition of Done

should include testing
customer behaviour

Validated Learning

IDEAS

LEARN

BUILD

DATA

PRODUCT

MEASURE

Experiment

State, before you do the work,

this is what I believe will happen,
and when you do the the work
see whether it happens or not.
- Jabe Bloom

Hypotheses
instead of
Requirements

Requirement:
Users can create 3D avatars.

Hypothesis:
We believe that users will create a 3D
avatar and spend more time on other
users profiles.

Falsifiable Hypothesis:
50% of newly signed up users will create a 3D avatar,
Avg time spent on user profiles will go from 30s to 1min+

Be closer to reality,
and avoid
bias & comfort zones

User Stories != Experiments

Missing the Measure Stage
No Way to Validate the Learning

Deploy != Launch
if $rollout.active?(:chat, current_user)
...
else
...
end

Deploy Every Commit

Release To A Small Set of Customers
Launch The New Feature When Ready

Experiments need to be
shown to actual customers
Not just to your team!

Tools & Ta
c

Lean Stack:
Experiment Reports

tics

EXPERIMENT REPORT

Author: [NAME]

Title: [TITLE]

Background
What are you trying to learn or achieve?

Created: [DATE]

Results
Enter the data.

Falsifiable Hypothesis
Your list of statements on the expected outcome of the experiment.
Use this format:
[Specific Repeatable Action] will [Expected Measurable Outcome]

Validated Learning

VALIDATED or INVALIDATED

Summarize your learning from the experiment.

Experiment Scope
How many and/or how long will the experiment last?

Minimum Build Plan

List the build plan step by step to get your measurement and experiment.

Next Action
Whats the next experiment?

Lean Stack by Spark59.com

EXPERIMENT REPORT

Author: [NAME]

Title: [TITLE]

Background
What are you trying to learn or achieve?

Created: [DATE]

Results
Enter the data.

Falsifiable Hypothesis
Your list of statements on the expected outcome of the experiment.
Use this format:
[Specific Repeatable Action] will [Expected Measurable Outcome]

Validated Learning

VALIDATED or INVALIDATED

Summarize your learning from the experiment.

Experiment Scope
How many and/or how long will the experiment last?

Minimum Build Plan

List the build plan step by step to get your measurement and experiment.

Next Action
Whats the next experiment?

Lean Stack by Spark59.com

Background
What are you trying to learn or achieve?

Give Context.
Why is this relevant?

Falsifiable Hypothesis
Your list of statements on the expected outcome of
the experiment.
Use this format:
[Specific Repeatable Action] will
[Expected Measurable Outcome]

Quantify Your Goals

Experiment Scope
How long will the experiment last?

Minimum Build Plan

List the build plan step by step to get your
measurement and experiment.

Set Scope & Timebox

Planning an Experiment
1. Context - Why is this relevant?
2. Quantify Your Goals
3. Set Scope & Timebox

Test your
Value and your Growth
Hypothesis first

Validated Learning:
Define Hypothesis,
Build & Launch Experiment,
Learn from the Results

Concepts for making this work:

1. Product Ownership
2. Shared Understanding
3. Accountability
4. Scoping & Timeboxing

Hierarchy
Anarchy

Lean Startup

1. Product Ownership
2. Shared Understanding
3. Accountability
4. Scoping & Timeboxing

1. Product Ownership
2. Shared Understanding
3. Accountability
4. Scoping & Timeboxing

Centralized Product Ownership

Cross-Functional
Team

Product
Owner

Customers

Distributed Product Ownership

Business
Stakeholder

Cross-Functional
Team

Customers

Product Ownership is about

Defining Customer Value,
not Aesthetics

Business Experiments
instead of ( just)
Development Tasks

Experiments are
Owned By
A Team Member

Draws Expert
Resources as needed
(e.g. through pairing)

1. Product Ownership

2. Shared Understanding
3. Accountability
4. Scoping & Timeboxing

Why?
are we doing
what were doing

Team needs to
Understand The Impact
of their work

Shared understanding is about

getting everyone to walk in the
other persons shoes a little.
Bill Scott - UX at Netflix, PayPal
This includes the business owners shoes!

Are we making progress

towards our business goal?

Tools & Ta
c

Company-Wide
Metrics Dashboard

tics

Actionable
Accessible
Auditable

Actionable

Tie specific repeatable actions to

observed results.

Accessible

Let everyone in the company

see and understand whats going on.

Auditable

Tie the numbers back to actual

customers, so you can verify the metric
and believe in it.

You need all three

ACTIONABLE

AUDITABLE

ACCESSIBLE

Unit Economics
instead of
Gross Revenue

Dave McClures AARRR Metrics

ACQUISITION

How do users find you?

ACTIVATION

Do users have a reat first experience?

RETENTION

Do users come back?

REVENUE
REFERRAL

How do you make money?

Do users tell others?

Value Metrics
How do users find you?

ACQUISITION

Do users have a great first experience?

ACTIVATION

Do users come back?

RETENTION

How do you make money?

Do users tell others?

REVENUE
REFERRAL

Growth Metrics
How do users find you?

ACQUISITION

Do users have a great first experience?

ACTIVATION

Do users come back?

RETENTION

How do you make money?

Do users tell others?

REVENUE
REFERRAL

Tools & Ta
c

Cross-Functional
Pairing

tics

Tools & Ta
c

Weekly Strategy Meeting

Part 1: Ideation

tics

Avoid Brainstorming.
It causes
Premature Convergence
Around One Idea.

By Daniel Cook
http://www.lostgarden.com/2010/08/visualizing-creative-process.html

Design Studio
by Will Evans
@semanticwill

Observe & Explain

To The Group

Ideate
Alone

Sketch &
Write down!

Discuss
In the Group

Review Experiments
Only In Weekly Meeting
Focus On Efficiency
In Daily Stand-ups

1. Product Ownership
2. Shared Understanding

3. Accountability
4. Scoping & Timeboxing

Tools & Ta
c

Lean Stack:
Validated Learning Board

tics

Kan-ban Board
Build

IDEAS

LEARN

BUILD

Measure

Learn

DATA

PRODUCT

MEASURE

Build

Measure

Learn

1 experiment
per team member

Separate Task Board

for tracking the details
Swimlane for each Experiment

Tools & Ta
c

Weekly Strategy Meeting

Part 2: Review

tics

Review Experiment Results

Challenge the Interpretation
Kill / Restart overdue Experiments

EXPERIMENT REPORT

Author: [NAME]

Title: [TITLE]

Background
What are you trying to learn or achieve?

Created: [DATE]

Results
Enter the data.

Falsifiable Hypothesis
Your list of statements on the expected outcome of the experiment.
Use this format:
[Specific Repeatable Action] will [Expected Measurable Outcome]

Validated Learning

VALIDATED or INVALIDATED

Summarize your learning from the experiment.

Experiment Scope
How many and/or how long will the experiment last?

Minimum Build Plan

List the build plan step by step to get your measurement and experiment.

Next Action
Whats the next experiment?

Lean Stack by Spark59.com

Results
Enter the data.

Actual data you Measured,

based on your Hypothesis.

Validated Learning
Summarize your learning from the experiment.

Validated
Invalidated
Inconclusive

Why?

Next Action
Whats the next experiment?

What will you do next

based on your learning?


Failure is good.
Validated Learning is our
Measurement of Progress.

1. Product Ownership
2. Shared Understanding
3. Accountability

4. Scoping & Timeboxing

Timeboxing

Build

Measure

Learn

Experiment Scope

Build

Measure

Learn

Startups dont starve,

they drown.
Open-ended learning is dangerous,
always timebox.

Recommended Timeboxes:
Interviews/Prototypes:
Qualitative Tests:
Quantitative Tests:

1-2 Weeks
1 Month
2-3 Months

Customer Interview
(Paper) Prototype
MVP
The Perfect Product
Inside-the-Building Comfort Zone

Get-out-of-the-Building Reality
Customer Interview
(Paper) Prototype
MVP
The Perfect Product

Justify the Cost of MVPs

with Proof

Understand Problem:
Explorative Customer Interviews
Define Solution:
User Testing With Prototypes
Validate Qualitatively:
Release MVP and hand-collect feedback
10s to 100s of customers
Verify Quantitatively:
Partial Rollout or Split Test and measure improvement
100s to 10,000s of customers

In Summary

0. Validated Learning
1. Product Ownership
2. Shared Understanding
3. Accountability
4. Scoping & Timeboxing

Tools & Ta
c

Experiment Reports
Company-wide Dashboard
Validated Learning Board
Weekly Strategy Meeting

tics

Thanks!
leanstack.com
lukas@spark59.com