The product manager’s guide

to customer-centered growth
Contents
03 INTRO
04 SECTION 1: BEFORE YOU START
The two approaches to product growth
Why you need an experimentation process
Set your strategic goals
The four key documents you need

08 SECTION 2: THE EXPERIMENTATION PROCESS

Step 1: Generate data-driven experiment ideas
Step 2: Prioritize your hypotheses
Step 3: Design and implement your experiments
Step 4: Analyze your test results and extract the lessons learned
Step 5: Systematize your results

13 SECTION 3: SUSTAIN GROWTH OVER TIME

Repeat the process
Improve the process
Customer experience is the foundation of growth

The product manager’s guide to customer-centered growth 02

The key to driving Intro
sustainable growth Most products don’t fail today because their teams can’t write enough code. They fail because they can’t

is using customer acquire and retain enough users.

insights to discover And that’s why so many people are looking for the one tactic, secret, or hack that will skyrocket their growth
curve. But we believe that focusing on specific tactics is the wrong approach.

the tactics that work In this eBook, we’ll make the case that, instead, you should focus on implementing a rigorous process of

best for you. experimentation. And that the key to driving sustainable growth is using customer insights to discover the
unique combination of tactics that will work for your product.

The material we’ll discuss is best suited for PMs working on products that have already found product/market
fit and have a good handle on retention. You need to have built a “must have” experience that customers love
before you transition into the growth phase.

Ideally, you already have a substantial amount of traffic, you’re ready to scale, and you want to make sure you do
it right—by putting the user at the center of the process because that’s what pays off in the long run.

And while every company will adapt this process to fit their own needs, our goal is to give you a
step-by-step guide for implementing a highly experimental, customer-centered growth process into
your organization. In this eBook, you’ll learn how to:

IMPLEMENT A PROVEN FIVE-STEP EXPERIMENTATION PROCESS

USE CUSTOMER INSIGHTS TO DISCOVER GROWTH OPPORTUNITIES
EXTRACT MAXIMUM VALUE OUT OF EACH TEST YOU RUN
INCREASE THE ACCURACY OF YOUR TESTS OVER TIME

The product manager’s guide to customer-centered growth 03

Section 1: Before you start
THE TWO APPROACHES TO PRODUCT GROWTH
“Growth is a system,
not a bag of tricks.”
There are two approaches to growing a product. - Andrew Chen,
Head of Supply Growth at Uber
1 You A/B test a handful of tactics you think will work based on your opinion, and hope they improve
your growth curve.
2 You start by looking at your analytics to identify the steps of your product flow where you’re seeing
the biggest dropoffs. Once you know where the problems are, you gather qualitative data to get
insight into what’s causing them. Then, you turn those insights into hypotheses and design
experiments to test them empirically. And finally, you feed the lessons you learn about your
product, customers, and channels back into the process to guide which tests you run next.

It’s a no-brainer. Your growth efforts will be much more effective if you establish a continuous, consistent
cadence of experimentation to identify the leaks in your funnel and come up with test ideas based on
multiple sources of data.

Peep Laja uses a great metaphor: “Would you rather have a doctor operate on you based on an opinion,
or careful examination and tests?” You’re like a growth doctor, and your product is your patient.

WHY YOU NEED AN EXPERIMENTATION PROCESS

When you focus on tactics first, you run into three core problems:

What works for other
products won’t necessarily
work for yours.
Your business model,
product, and audience are
fundamentally different
from anyone else’s.
Distribution channels are
changing faster than ever.
Channels that work for you
today will likely be saturated
and far less effective 12 months
from now. To stay ahead of
the curve, you have to always
be experimenting to discover
emerging channels.
There are no silver bullets.
No one-off tactic is going
to solve all your growth
problems. The tactics that
do produce explosive growth
are usually the result of
a combination of lessons
learned from previous
experiments.

The product manager’s guide to customer-centered growth 04

Tactics are important. But instead of focusing on them first, focus on
establishing a systematic and repeatable experimentation process.
This will lead you to discover the combination of tactics that result in
sustainable growth for your product.

Your growth efforts will be more productive and successful once you
move away from testing random tactics and start approaching it as the
scientific process it is:

Set strategic
goals “ There is no one right or perfect growth
process. The important part is just to have
Systematize Gather and one, stick to it, and improve it over time.”
analyze data
- Brian Balfour, VP of Growth, HubSpot

Analyze Generate
results hypothesis

Design and Prioritize

implement tests experiments

In the following sections, you’ll learn an experimentation process originally

developed by Brian Balfour. We combined it with ideas from Peep Laja’s
ResearchXL framework, and added our own touches as well.

This process isn’t perfect, but it will give you a solid foundation to build on.
Take it, copy it, and adapt it to your own team. But whatever you do, use it.

The product manager’s guide to customer-centered growth 05

SET YOUR STRATEGIC GOALS
Once you’ve determined what area to focus on, you need to set a time
Before diving into the process, you need to figure out three things: frame for how long you’ll focus on it. We recommend a period of time
between 30 and 90 days. Anything less than that won’t be enough to
What do you want to accomplish? make substantial progress. And anything more will start to drag on,
making it easy for your team to lose motivation.
When will you want to accomplish it by?

How will you measure success? Next, you have to decide how you’ll measure success. What quantifiable
results will you use to measure whether or not you have achieved your
Start by taking a step back and setting your high-level strategic goals. goals? For example, if you decided to focus on improving your activation
Ask yourself this question: rate, the quantifiable results you measure might look like this:

What’s the one area of the customer lifecycle that, if you 35% activation rate
made substantial improvements, would have the biggest
impact on your growth curve right now? 40% of users come back two times per week after week 1

Is it acquisition, activation, retention, referral, or monetization? There’s no

right or wrong choice, and the answer will be different for every product. 15% of users come back two times per week after week 10

Objectives and key results

The Objective and Key Result (OKR) framework is a goal-setting method used by many growth teams in Silicon Valley.
It was first invented by Intel and popularized at high-growth tech companies like Google, Zynga, and LinkedIn.

If you’d like to learn more about how to set OKRs, we highly recommend Christina Wodtke’s guide.

The product manager’s guide to customer-centered growth 06

THE FOUR KEY DOCUMENTS YOU’LL NEED

There are four key documents you’ll use throughout this process to
keep it driving forward:

1 Backlog
The purpose of the backlog is to provide a public place for your
team to dump and record all the growth ideas they come up with
as they go through the experimentation process so they can
come back to them later.

2 Pipeline
This is a list of all the experiments your team has previously
run, are currently running, and are going to run next. It allows
anyone on your team, including your new hires, to see every
single experiment you’ve run and the lessons you’ve learned
along the way.

3 Experiment docs
Every experiment gets its own document that outlines the objective,
hypothesis, experiment design, results, analysis, and lessons
learned. Experiment docs force your team members to critically
think through their ideas to implement them successfully. To
download an example experiment doc template, click here.

4 Playbooks
These are step-by-step guides that anyone can follow to repeat
what you’ve learned without having to talk to you.

Now, let’s dive into the details of each step of the experimentation
process.

The product manager’s guide to customer-centered growth 07

Where’s your funnel Section 2: The experimentation process
leaking? What’s STEP 1: GENERATE DATA-DRIVEN EXPERIMENT IDEAS
causing people The first step in the growth process is about collecting quantitative behavioral data and qualitative user
to have doubts, insights to identify the biggest issues your customers and potential customers are experiencing—and then
using those findings to guide your experiments.
uncertainties, Fixing these problems will help you meet your customer’s needs and expectations, craft an experience
and hesitations? they’ll love, and improve your growth curve along the way.

But not all data is equally useful. You don’t just need any data; you need actionable data that gives you insight
into how to optimize your specific area of focus. For example, if you’re focused on improving activation,
break down your product’s onboarding and activation flow into its individual steps. And then collect data for
each step.

Narrowing your scope down to the individual steps allows you to gather the right data that will give you insight
into which levels you can pull to impact your area of focus—and prevent you from getting overwhelmed by
having too much irrelevant data.

Start by gathering quantitative data that shows you where the problem is—where the biggest dropoffs are
and where your funnel is leaking.

Once you know where the issues are happening, you can gather qualitative insight to understand why.
What’s causing people to have doubts, uncertainties, and hesitations?

Once you know the source of the problem, you can come up with hypotheses to test about what you think
would remove the friction.

The product manager’s guide to customer-centered growth 08

Here are 11 data sources you can use to generate data-driven test ideas. You don’t have to use every single one—choose a combination of three to five quantitative
and qualitative sources that make the most sense for what you’re trying to learn. As you do your research, document all the issues you identify and test ideas you
come up with in your backlog.

Web analytics On-page surveys & live chat Product comparison

Identify specific pages and steps in your product
flow where you’re losing customers and potential
customers.
Technical analysis
Identify annoying cross-device and cross-browser
issues that are hurting your conversion rates.
Fix issues if the added value is greater than the
value of the time it will take your developers to
identify and fix them.
Form analytics
Know exactly which form fields people hesitate
the most to fill out, which fields they leave blank
even though they are required, and which ones
cause the most error messages.
User session replays
Anonymously record video of everything people
do as they interact with your product. Watch
how people respond to your product while risking
their real money or personal information.
Place surveys on low-performing pages and
ask people why they aren’t converting.
Chat transcripts give you insight into common
frustrations and where people get confused.
User testing
Observe people while they use your product
and speak their thoughts out loud. Hear and
see where people get stuck, confused, and
frustrated.
Mouse tracking
Use click maps, attention maps, and scroll maps
to record what people do on your product pages
with their mouse or fingers.
Product walkthrough
Hour-long team exercise. Review your product
page by page with your team and identify where
you can make the experience more relevant,
clear, and value-adding—and where you can
reduce friction and distraction.
Twenty-minute team exercise. Everyone picks
one or two products from your competitive and
noncompetitive space that they use actively.
Walk through their product flow and discuss
what they do well, what they could do better,
and how you can adapt it to your own product
and audience.
Growth network
Talk to other growth professionals on the phone
on a regular basis. Exchange ideas, share
experiments, and discuss what worked, what
didn’t, and why you think you got the results
you did. Adapt ideas to your own product.
Question brainstorm
Hour-long team exercise. Spend the first 20
minutes asking nothing but “why,” “what if,” and
“what about,” questions. No discussion or answers.
Write your question on a sticky note, announce
it to the team, and put it on the whiteboard. This
reveals what you don’t know and generates
discussion.

As you gather and analyze the data you collect, you should be dumping all the issues you identify and experiment ideas you come up with into your backlog.
The next step in the process is to translate those findings into a prioritized pipeline of hypotheses to test.

These data sources are a combination of ideas from a book called The Innovator’s Solution by Clayton Christensen and the ResearchXL framework, originally developed by Peep Laja.

The product manager’s guide to customer-centered growth 09

STEP 2: PRIORITIZE YOUR HYPOTHESES
Now that you have a backlog full of potential tests to run, the next
step is to turn your findings into prioritized hypotheses.
Every test idea needs to have its own experiment doc. And in this
step, you’ll use it to document five things: the objective, hypothesis,
probability of success, resources needed, and potential impact if the
test is successful. The purpose of this exercise is to make you think
through why you want to run this experiment and whether or not it’s
actually a good idea.
First, describe what you’re testing, and why you’re testing it, in one
or two sentences. This is your objective. Describe it so that someone
else on your team will understand what you did if they read it a year
from now.
Then, you need to write down a hypothesis for every experiment.
Surprisingly, most people don’t do this. Here’s an example of what
your hypothesis should look like:
If successful, [variable X] will increase
by [impact] because [assumptions].
You’re basically predicting how much you think the variable you’re
targeting will increase, and then you’re laying out the underlying
assumptions of your prediction. This allows other people on the team
to challenge your assumptions and, hopefully, help you run more
accurate tests over time.
Once you’ve crafted the hypothesis, the next step is to prioritize each
experiment based on three core factors, and, of course, to document
them in your experiment doc.
• Probability
What’s the probability that this experiment will be successful? Use
a rough estimation of low = 20%, medium = 50%, and high = 80%.
You might give your test a high probability of success if it’s based
on something you learned from a previous experiment. But if
you’re testing a new channel that you have no experience in, you
might rank it low.
• Resources
Roughly estimate how much time you’ll need from your design,
engineering, and marketing resources to implement the experiment
(i.e., 1 hour, 1 day, 1 week, 1 month, etc.).
• Impact
If this test is successful, what’s the total impact you expect to see?
What do you predict the results will be? How much will it improve
the targeted metric?
Once you’ve documented everything, you can compare experiment
docs, weigh them against the other tests in your pipeline, and make
an objective decision about which experiments you think will help you
make the most progress toward your strategic goals.
The product manager’s guide to customer-centered growth 10
Growth experiments build on themselves. The more experiments you
run, the more you learn—which allows your experiments to become more
accurate over time. That’s why we recommend starting with experiments
that offer high business value and aren’t very resource-intensive. Here’s a
matrix that really helps simplify the prioritization process:
VALUE VS. COMPLEXITY QUADRANT
HIGH
Business
1 2
value
? Okay, now that you’ve outlined your prioritized experiment, then next
LOW HIGH
Complexity / effort
- Values vs. complexity quadrant. Jim Semick, the founder of ProductPlan.
Finally, once you’ve prioritized the list of hypotheses in your backlog,
move the highest priority experiments that you plan to run next into your
pipeline.
STEP 3: DESIGN AND IMPLEMENT YOUR EXPERIMENTS
Once you’ve dropped your prioritized hypotheses into the pipeline, the
next step is to design and implement your controlled experiments.
Keep it as simple as possible. Think about it like this: how can you spend
the minimum amount of time and resources to gather the data you need
to understand whether or not your hypothesis is valid?
Spend five to ten minutes outlining the steps needed to implement the
test, and document it in your experiment doc. The outline doesn’t have to
be very detailed, but going through this process will do two things.
1 It forces you to think through the experiment beforehand and
prevents you from making small mistakes and avoidable errors
2
that would force you to rerun the test.
If anyone else on your team ever wants to repeat the test or
iterate off it, they can understand exactly what you did without
having to talk to you.
step is to run it. There are a few things you need to do to make sure your
tests are statistically significant:
Test with the appropriate sample size
Test for an entire business cycle
Don’t end your test based on confidence intervals
We won’t dive too deep into the statistics here because a number of
great resources covering the subject have already been created. Here
are three articles that will teach you everything you need to know:
• How Not to Run an A/B Test
• Crash Course on A/B Testing Statistics
• 12 A/B Split Testing Mistakes I See Businesses Make All The Time
The product manager’s guide to customer-centered growth 11
STEP 4: ANALYZE YOUR TEST RESULTS AND EXTRACT THE LESSONS LEARNED

Once you’ve run your controlled experiment, the next step is to analyze the results and extract the lessons learned.
As you determine the success or failure of an experiment, look at three factors and document them in the experiment doc: “Why?”
Impact. What were the results of the experiment? is by far the most important
question you can ask. The
Accuracy. How close were the results to your hypothesis? answers you come up with
will lead you to new ideas
Why? Why do you think you saw the results you did?
and experiments to run next.
“Why?” is by far the most important question you can ask because the answers you come up with will lead you to new
ideas and experiments to run next. Use this question to extract as much insight as possible and, again, document all the
lessons you learned in the experiment doc.

Once you capture what you learned, make a list of action items that consists of tests you want to run next. And then transfer
those ideas into your backlog. Now that you’ve gained a deeper understanding of your customer, product, and channel, go
back to the experiments in your pipeline and adjust the predictions you made based on what you know now.

Feeding what you learn back into your backlog and predictions is the most critical part of this experimentation process—
it’s what allows your experiments to become increasingly accurate and successful over time. If you skip this step, you’ll fail
to build on your experience and you’ll constantly have to start over from square one. You’ll essentially be throwing away
the value you’ve created.

STEP 5: SYSTEMATIZE YOUR RESULTS

This is the final step in the process. Once you discover a successful experiment, try to implement it as a new product
feature or initiative. For example, if you only ran your test with 15% of your audience, then launch it to all of your users as a
new part of the product experience.

But sometimes your team will discover things that are more manual and can’t be productized—for example when a growth
marketer on your team discovers an effective method from promoting a piece of content to a specific segment of your
audience. If that’s the case, turn it into a playbook, which is a step-by-step guide that anyone can use to repeat what you
did and get the same result. Make your playbooks publicly available to everyone on the team to read, comment, and
collaborate on.

This allows you to capture, retain, and share the knowledge and value you create with successful experiments.

The product manager’s guide to customer-centered growth 12

Feeding the lessons Section 3: Sustain growth over time
you learn back into REPEAT THE PROCESS

your backlog and Repeat steps 1-5 as many times as you feasibly can in your 30-90 day period. Take the lessons you learn from
analyzing experiment results and continuously feed them back into your backlog, and use them to refine the
using them to adjust predictions in your prioritized experiments.

your predictions is This feedback loop plays a critical role in making your process more accurate with each successive cycle.

what allows this IMPROVE THE PROCESS

process to become Once you go through this 30-90 day process a few times, you need to take an even bigger step back to examine

increasingly accurate the effectiveness of your efforts from a macro level and think about how you can optimize them. You need to
look at three things in particular:

and successful over 1 Batting average

time. What’s your team’s ratio of successful experiments to failures? This number should be getting better over
time.

2 Accuracy
Are your hypotheses getting more accurate as you gain more experience in a channel? The longer you
work in a channel, the more accurate your experiments should get.

3 Throughput
How many experiments you do you run in a given period of time? How can you increase the velocity?

You can improve those three things by constantly optimizing your process, your team, and the tools you use.
And as a result, your experimentation efforts will become more accurate and more successful over time.

The product manager’s guide to customer-centered growth 13


If you don’t deliver a “must have”
experience, it won’t matter what
process or growth tactics you use.
CUSTOMER EXPERIENCE IS THE FOUNDATION OF
GROWTH
As you run experiments and optimize the funnel, remember that
customer experience and growth are inextricably tied. None of your
efforts will matter if you don’t deliver an experience people love.
The key to this process is that your team puts the customer at the center
of it—constantly gaining a deeper understanding of your customers,
how they use your product, and which channels are effective for reaching
them. As Sean Ellis eloquently said:
“I found that the key to sustaining rapid growth is
understanding your ‘must have’ experience and then
aligning the entire business around that experience.
This includes aligning the product roadmap, funnel
optimization, and messaging.”
- Sean Ellis, founder of Qualaroo and GrowthHackers.com
The most important factor for sustainable growth is whether or not your
product meets your customers’ needs, solves their problems, and produces
their desired outcome in a way that they enjoy and want more of.
If you don’t deliver a “must have” experience, it won’t matter what process
or growth tactics you use. It’s hard to justify investing in top-of-the-funnel
acquisition if every new user eventually leaves. Bottom line: the better
your customer experience is, the more customers you’ll retain, the more
people they’ll share your product with, and the easier it will be to grow.
The product manager’s guide to customer-centered growth 14
 Create great experiences
UserTesting is the fastest and most advanced user experience research platform on the market. We give marketers,
product managers, and UX teams on-demand access to people in their target audience who deliver audio, video,
and written feedback on websites, mobile apps, prototypes, and even physical products and locations.

2672 Bayshore Parkway, Mountain View, CA 94043

www.usertesting.com | 1.800.903.9493 | sales@usertesting.com