Measurement

April 12, 2017

Measurement and ideas
Measurement can turn a vague idea into a good one

● Do back-of-the-envelope calculations
● Focus on what will make a great product
● Note: don’t worry about measuring total market size
○ “Could this eventually be huge?” is good enough
Product: What should you measure?
Product: What should you measure?

● Users getting value today

○ Active users
○ Revenue
○ Transactions
Retention: usage over time by cohort
The “ring of fire”

● If cohorts keep declining without a lower bound, eventually:

○ You will burn through all your existing users
○ You will exhaust the supply of new users
The “ring of fire”
Increasing retention over time

● Your goal should be to get cohorts to stay above some line

● If cohorts increase over time, you are in a very strong position
○ Whatsapp
○ Uber riders
○ Facebook
Retention predicts high valuations - Tom Tunguz
 MEASURE
RETENTION!
You want exponential growth
Measure your % growth per week
Iteration drives progress

● Iteration speed is a huge advantage over bigger companies

● Measurement is crucial to iterating in the right direction
● Measure your iteration speed itself
How seriously should you take your metrics?

1. Not seriously enough

○ It’s easy to fool yourself and go in circles
2. Very seriously
○ You make smart, informed decisions
3. Too seriously
○ You increase the metric but not the underlying value
Reality can be painful

● Metrics reduce delusion, but sometimes that will hurt

● Most leaders rely on delusion to keep followers going
● You want to be:
○ Learning from the past
○ Grounded in the reality of the present
○ Optimistic about the future
Q&A