Digital Product

Management
From measuring success to launching products
 Previously on Digital Product Management...
1. A product vision is about understanding how the future we are aiming for should be
2. A product strategy is about laying the path to reach that future
3. Product objectives are about quantifying how fast are we moving towards that future
4. Product decisions is about creating the process to analyse and execute on the available
information to shorten that path
5. Product roadmap is about communicating the barriers and aligning everyone towards
a unified goal

Andre Albuquerque - Digital Product Management

 So what are we covering today?

1. Measuring success
2. Using metrics to act
3. Launching products
4. Growing products

Andre Albuquerque - Digital Product Management

 Measuring success
“If you can’t measure it, you can’t improve it”

- Peter Drucker

André Albuquerque - Digital product Management

There are many ways to measure success

Amazon stock price, December 27th 2018 DAU/MAU, Facebook Q3Y18 earnings

User retention analysis, Snapchat v TikTok % of feature adoption

But measuring success in product is hard
 This is the reality

Communications team Finance team cuts Company is features It’s the end of the Operations increases
sends a huge email the prices by 15% to on national news month and sales support team and
campaign get more buyers getting massive team pushes to hit answers all pending
attention their bonus emails & messages

Andre Albuquerque - Digital Product Management

 What really matters when defining
measurement
1. Understand what business you’re in.
- Your business model defines what needs to be measured: Subscription, Transactional, Game,
App/In-App, etc. Each model has more suitable OMTMs at any moment.
2. Understand at what stage you are.
- Different stages have different goals Common stages are MVP validation, Feature optimisation, Business
model optimisation.
3. Know why you’re measuring something.
- Consider the reason for measurement. Is it to optimise a feature? Is it to track business adoption? Is it to
make decisions?

Andre Albuquerque - Digital Product Management 9

 What good metrics look like:

Specific – target a specific area for improvement.

Measurable – quantify or at least suggest an indicator of progress.
Assignable – specify who will do it.
Realistic – state what results can realistically be achieved, given available resources.
Time-related – specify when the result(s) can be achieved.

Andre Albuquerque - Digital Product Management 10

 4 frameworks to create some harmony in
success measurement chaos

😇 Loyalty metrics (Net Promoter Score)

🤺 Utilization metrics (Product metrics)
🙏 Alignment metrics (OKRs)
🌪 Funnel metrics (AARRR)

Andre Albuquerque - Digital Product Management

 Loyalty metrics
NPS

André Albuquerque - Digital product Management

 A strong success metric is satisfaction (NPS)
● “How likely is it that you would recommend our company/product/service to” a friend or colleague?”
● Studies show correlation with revenue evolution
● Simple approach, easily implementable, available at any stage, as long as you have customers/users

NPS = % Promoters - % Detractors

Andre Albuquerque - Digital Product Management

 Which industry has the
lowest average NPS and
why?

Andre Albuquerque - Digital Product Management

 Internet/Telecom Providers

Andre Albuquerque - Digital Product Management Satmetrix NICE 2018 average NPS by industry report
 Utilization metrics
Product Metrics

André Albuquerque - Digital product Management

 Common product metrics that inform OKRs
MAU/DAU - Monthly / Daily Active Users, social metrics
GMV | AOV - Gross Merchandising Value, Average Order Value, ecommerce/D2C metrics
LTV - Lifetime Value, repeatable purchase metric
CPA | CPC | CPL - Cost per acquisition/click/lead, acquisition funnel metrics
ARPU - Average revenue per user, B2C metrics
Contribution/Take rates - % of kept sales, net revenue/total sale, marketplace/ecommerce
Accounting metrics (Gross/Net Revenue | Profit, EPS…) - Overall company metrics
Sales - Individual units sold / transacted, commonly adapted to the industry/model (eg: nights)
Retention Rate | Churn Rate - Retained and lost customers. SaaS/Subscription metrics
ARR/MRR - Annualised/Monthly Recurring Revenue, SaaS/Subscription metrics

Andre Albuquerque - Digital Product Management

 Product metrics split by operating models

E-commerce Marketplace Social SaaS Hardware

GMV Transactions Renewal Rate

MAU/DAU Sales
AOV Take rate Churn Rate
ARPU Transactions
Transactions Booking volume ARR/MRR

Andre Albuquerque - Digital Product Management

 BREAKOUT ROOM TIME
1. 7 minutes
2. Random groups, everyone says which one is the 1st app on their dock
(from the left). The app with more votes is chosen. If none has more
votes, choose one.
3. What are metrics of success for this app?
4. Will be picking some groups to demonstrate

Andre Albuquerque - Digital Product Management

 Alignment metrics
Objectives and Key Results (OKRs)

André Albuquerque - Digital product Management

 OKRs - Objectives and Key Results

Objectives describe the outcome you want to achieve, the future

🎯 Objective state you want to get to. Not a number or a measurement.

Key Results are lag measures (something you don’t control),

🔑 Key Result that when achieved, get you to the objective.

Initiatives are lead measures (something you have direct

💡 Initiative control over) that should influence your Key Results.

Andre Albuquerque - Digital Product Management

 OKRs - Objectives and Key Results

🎯 Objective Increase overall customer happiness

Specific metric Descriptive
Time-boxed

Hit 80 NPS during Q2

🔑 Key Result
Get >9/10 ratings in the April sales
Actionable

Follow up 100% of customers that bought last quarter by EoM

💡 Initiative Increase customer support team by 2x during Q2

Controllable

Andre Albuquerque - Digital Product Management

 OKRs trickle down enabling alignment
CEO’s objective VP’s Objective Director’s Objective

90% Answers to
questions under
24h
15% conversion
to premium
30% add credit
card details in
first 7 days
Achieve traction
with the MVP

80% users accept

push notifications
40% of users come
back two times in
one week
50% of users
deposit 10€ on
Andre Albuquerque - Digital Product Management
their first usage
 What is (likely) an OKR
at these companies?

Andre Albuquerque - Digital Product Management

 What is (likely) an OKR
at these companies?

O: Improve DAU by 20%

KR: Increase mobile users in
emerging countries by 50%

Andre Albuquerque - Digital Product Management

 What is (likely) an OKR
at these companies?

O: Improve DAU by 20% O: Achieve 300B GMV in Y20

KR: Increase mobile users in KR: Grow prime users by 30%
emerging countries by 50% in India

Andre Albuquerque - Digital Product Management 26

 What is (likely) an OKR
at these companies?

O: Improve DAU by 20% O: Achieve 300B GMV in Y20 O: Get 200M subscribers
KR: Increase mobile users in KR: Grow prime users by 30% KR: Launch 100 original shows
emerging countries by 50% in India in 2020

Andre Albuquerque - Digital Product Management 27

 How do you use metrics to
make product decisions?
What are Uber’s main metrics?

André Albuquerque - Digital product Management

 Uber’s key metrics

Average waiting time % utilised supply

Andre Albuquerque - Digital Product Management

 UBER has a balancing challenge
● 2-sided-marketplace
○ Riders (users riding an uber) is the Demand
○ Drivers (cars taking passengers) is the Supply
● Each side needs to be balanced:
○ Over-demand: Too many riders, not enough drivers
■ “Average Waiting Time” up
○ Over-supply: Too many drivers, not enough riders
■ “% Utilised Supply” down

Andre Albuquerque - Digital Product Management

 PO
What happens in each scenario? L
TI L
ME

Andre Albuquerque - Digital Product Management

 PO
What happens in each scenario? L
TI L
ME

Avg Waiting Time ↗

Andre Albuquerque - Digital Product Management
 PO
What happens in each scenario? L
TI L
ME

Avg Waiting Time ↗ % Supply Utilization ↘

Andre Albuquerque - Digital Product Management
 How to get AWT↗
and %SU↘?

Andre Albuquerque - Digital Product Management

 Surge pricing

● If there is too much demand and not enough supply:

a. Supply needs to be encouraged to increase
b. Demand needs to be discouraged from using the service

● Surge Pricing v1 comes in as a product solution to solve the problems:

a. Higher prices reduces passengers, balancing demand
b. Higher earnings increase volume of more drivers in the street

Andre Albuquerque - Digital Product Management

 But what about in this situation?

Andre Albuquerque - Digital Product Management

 Both metrics are going into the wrong place

% Supply Utilization ↘
Andre Albuquerque - Digital Product Management Avg Waiting Time ↗
 You create incentives to move your variables
Surge Pricing v2 adds area bounds:

● As supply is dynamic, increased earnings in a

new area makes them move
● Too much demand in a specific area makes
prices go up, reducing riders in that area
● Balanced zones are not affected, so local
demand can still grow

Andre Albuquerque - Digital Product Management

 Launching products

”All too often, shipping product is seen as an end. We believe the opposite
is true, that shipping is a beginning.”
- Paul Adams, VP Product, Intercom
-

André Albuquerque - Digital product Management

Andre Albuquerque - Digital Product Management
 SHIPPING

Launch

Launch

Launch

What companies think What theories say

Reality
launching product is launching product is

Andre Albuquerque - Digital Product Management

 Efficiency is defined as the ability to
accomplish something with the least
amount of waste.
Effectiveness is defined as the degree
to which something is successful in
producing a desired result.

Growth is about effectiveness.

Andre Albuquerque - Digital Product Management

 Do Things That Don’t Scale - 3 examples

Recruit users manually

Do stuff under the hood
Make users feel
Tinder launched local unique
Expensify said they had AI for
parties where users had
expenses but were using
to download to enter for Producthunt sent
mechanical turk to make it
free custom stickers and
“automatic”
letters to their top
users
Andre Albuquerque - Digital Product Management 43
 Growth is a continuous loop as was discovery

Look for customers, activate Scale revenue, scale

visitors and get them to pay acquisition, scale value

Andre Albuquerque - Digital Product Management 44

 1. Hubspot - Free tools

Andre Albuquerque - Digital Product Management 45

 2. Shave Dollar Club - Viral buzz

Andre Albuquerque - Digital Product Management 46

 3. Uniplaces - Ambassador programs

Andre Albuquerque - Digital Product Management 47

 But at a certain point of a product’s
growth, doing things that don’t scale…
doesn’t scale

Andre Albuquerque - Digital Product Management

3 methods to cross the chasm

49
 1. Piggybacking

Andre Albuquerque - Digital Product Management 50

 Eg: How Uniplaces piggybacked OLX

Andre Albuquerque - Digital Product Management

 2. Finding a replicable growth model

Andre Albuquerque - Digital Product Management

 Funnel metrics
AARRR metrics

André Albuquerque - Digital product Management

 The AARRR metrics

1 2 3 4 5

Acquisition Activation Retention Referral Revenue

How do customers How many are How many are How many are How many are
find you? getting to “Aha”? coming back? advocating for paying you?
you?

Andre Albuquerque - Digital Product Management

 Explaining AARRR using
Acquisition - Why are coming to my product?
● Define your acquisition metric
○ Is it a download? A registration? Is it a profile?
● Define your acquisition strategy
○ Find the acquisition channels (Tinder: Word of Mouth,
Referral, SEM)
○ Know the metrics that define those channels
○ Where will you invest? Use ROI on channel
○ What can you do to improve?

Andre Albuquerque - Digital Product Management

 Explaining AARRR using
Activation - Why are users engaging proactively?
● Define your activation metric
○ Maybe it’s not registering, it’s actually acting. (Tinder:
first swipe). Use a ratio to cover over time
● Define your activation strategy
○ Why aren’t users activating?
○ What prevents most users from that activation action?
○ What can you do to facilitate users to do the action?
○ How can we make it occur faster?

Andre Albuquerque - Digital Product Management

 Explaining AARRR using
Retention - Why are users leaving or staying?
● Define your retention metric
○ Use the activation metric as denominator
○ Use cohorts using this metric
● Define your retention strategy
○ Discover what makes users stay?
○ What are the ones leaving not experiencing?
○ What is the bottleneck to retain unretained?

Andre Albuquerque - Digital Product Management

 Explaining AARRR using
Revenue - Why are users paying, and why are they not?
● Define your revenue metric
○ Use ratio to compare paying vs free usage
○ Sales, ARPU, Connect it to your pricing and model
● Define your revenue strategy
○ What is most used? Can we make a premium version?
Can we expose users to this premium feature?
○ What’s my monetization strategy?
○ What will you test to make users pay?

Andre Albuquerque - Digital Product Management

 Explaining AARRR using
Referral - Why are users telling about my product to others?
● Define your referral metric
○ K-viral coefficient, referral attribution
○ Referral Conversion from campaigns
● Define your referral strategy
○ Those that came from WoM, why were they referred?
What do users love the most?
○ Make it easy to refer (contact integration)
○ Make it interesting to refer (incentives)

Andre Albuquerque - Digital Product Management

 3x BREAKOUT ROOM TIME
1. 5 minutes
2. Choose last app of your dock
3. Room 1: Acquisition > Metric + Strategy
4. Room 2: Activation > Metric + Strategy
5. Room 3: Referral > Metric + Strategy
6. Will be picking some groups to demonstrate

Andre Albuquerque - Digital Product Management

 For the
Product Pitch

Andre Albuquerque - Digital Product Management

 Present your final product
● Video (max 7 minute video) + presentation
a. Presentation (max 10 slides)
i. Business model
ii. Success measurement (OKRs and metrics)
iii. Launch strategy (using Funnel Metrics)
iv. Final prototype
b. Video presentation
i. Everything you find relevant to tell your story
ii. Upload to Youtube and submit on Moodle
iii. Not Loom, make it fun

Andre Albuquerque - Digital Product Management