Welcome to

Product School!
MODULE 5 - VALIDATING AN OPPORTUNITY HYPOTHESIS
Agenda

WEEK 1 WEEK 2 WEEK 3

Understand Customers & Problems

Course Introductions Competitiveness, primary & secondary activities, customer Validate an Opportunity Hypothesis
Instructor & trainee Intros journey maps Effort vs. user value, A/B testing & customer interviews

Introduction to Product Management Create an Opportunity Hypothesis Define Product Requirements

PM skills, goals & methodologies Qualitative & quantitative methods, setting goals PRD, MVP & roadmaps

Set Product Objectives

User personas, metrics & use cases

WEEK 4 WEEK 5 WEEK 6

Start Building Develop Your Product II Get Hired

Design processes, product vs. design & sketching MVC, API design & team management Retrospectives, public speaking & resume reviewing

Develop Your Product I

Market Your Product Deliver and Present
Development methodologies, engineers & product, design
Channels, messaging & insights Capstone Project
patterns
 Agenda Accelerated

DAY 1 DAY 2 DAY 3 DAY 4 DAY 5

Strategy & Discovery Research & Define Design & Develop Launch Present

Create an Opportunity Hypothesis Start Building

Course Introductions Market your Product Deliver & Present
Qualitative & quantitative methods, Design processes, product vs. design
Instructor & trainee Intros Channels, messaging & insights Capstone Project
setting goals & sketching

Introduction to Product
Management
PM skills, goals & methodologies
Validate an Opportunity
Hypothesis
Effort vs. user value, A/B testing &
customer interviews
Get Hired
Develop Your Product I Retrospectives, public speaking &
Development methodologies, resume reviewing
engineers & product, design patterns

Develop Your Product II

Set Product Objectives Define Product Requirements MVC, API design & team
User personas, metrics & use cases PRD, MVP & roadmaps management

Understand Customers & Problems

Competitiveness, primary &
secondary activities, customer
journey maps
Index > Session

Agenda

1. Internal Validation

2. Validating with Experiments

3. Customer Interviews
Module 5 > Section 1 > Internal Validation

Internal Validation
Module 5 > Section 1 > Internal Validation

Internal questions

Internal validation simply involves stepping

back, looking at the opportunity, and asking
questions about it.

Here are some sample questions:

● Is this in line with our vision?

● Is it on our roadmap?
● Can we do it well?
● Can we design it so that the cost is less
than the benefit?
Module 5 > Section 1 > Internal Validation

Bugs & Sugs

Sometimes you have known issues, bugs or

feature requests (sugs).

These can be semi-quantitative if many people

ask for it or determine the impact on your
product.
Module 5 > Section 1 > Internal Validation

Effort vs. user value

How much will your customers value the

feature?

Low effort, high-value = best

VALUE
1. High-effort, low-value = worst

2. But it’s not a perfect test

a. Sometimes you have to do high-

effort, low-value things like
refactoring code to pay off
technical debt.
EFFORT
Module 5 > Section 1 > Internal Validation
Should we work on this?
Checklist of 12 questions to help your internal validation for a
new opportunity hypothesis.
1. Is this opportunity in line with our vision?
2. Does it support the product’s vision and core function, or does
it dilute the product’s value?
3. Can we do it well with our capabilities (or is it feasible and
desirable to expand our capabilities to meet the opportunity)?`
4. How does it contribute to our key metrics?
Internal Validation Checklist Template here.
5. Do we have any data be it from analytics, surveys, or bug
reports to support this opportunity?
6. Is it required to meet a critical business initiative?
7. How does it contribute to our users’ winning?
8. Is it on our roadmap for this year? Or is something else on
our roadmap that makes this idea irrelevant soon?
9. Will it matter in two years? (It’s OK if the feature is to
address an immediate need, but you’ll want to limit those, as
you want to prioritize things that have a higher value over
time.)
10. Will everyone benefit? If it only helps a niche set of
customers, is it worth the cost?
11. If it succeeds, can we support it?
12. Can we design it so that the cost is less than the benefit?
Module 5 > Section 1 > Internal Validation

Strengths, Weaknesses,
Opportunities and Threats (SWOT)

Goal Goal
S W O
What SDo You What Do You T
Strengths
Strengths Weaknesses Opportunities Threats
Want? Want?

Another useful tool to do an internal validation is to do a SWOT analysis.

Here, you write down attributes for your product for each category.
Module 5 > Section 1 > Internal Validation

SWOT

Goal Goal
S W O
What SDo You What Do You T
Strengths
Strengths Weaknesses Opportunities Threats
Want? Want?

Strengths
Internal factors that benefit a product
(anything that aligns in the company’s favor, such as skill sets, culture, etc.)
Module 5 > Section 1 > Internal Validation

SWOT

Goal Goal
W O
What SDo You What Do You T
Strengths Weaknesses Opportunities Threats
Want? Want?

Weaknesses
Internal/What would the company struggle with
(e.g., a software-only web co would struggle to make a physical product)
Module 5 > Section 1 > Internal Validation

SWOT

Goal
W O
What SDo You T
Strengths Weaknesses Opportunities Threats
Want?

Opportunities
External factors that increase need of product
 Module 5 > Section 1 > Internal Validation

SWOT

Goal Goal
W O
What SDo You What Do You T
Strengths Weaknesses Opportunities Threats
Want? Want?

Threats
External challenges a product faces (industry/societal forces opposing product)
1. See if the opportunity hypothesis helps limit a weakness or take advantage of an opportunity
2. The hypothesis might also depend on something you listed under weakness = hint it won’t go smoothly

SWOT Template here.

Module 5 > Section 1 > Internal Validation

Saying NO

Often the issue isn’t finding opportunities but figuring out

what to focus on.

This often means saying NO.

Ideas come from everywhere!
● Remember, you’re going to say no a lot as a PM!

● Do internal validation first since it’s fast, and it will let

you filter out lots of ideas.
Module 5 > Section 1 > Internal Validation

Soft influence

The problem with saying no often is that they’ll think you

don’t listen & stop coming to you with ideas.

To mitigate this, use internal validation to quickly provide

intelligent reasons for saying no:

● Helps with your soft power

● Lets others feel appreciated because it shows them you

thought about their idea

Once you’ve gone through the Internal Validation Checklist you might find the Internal Validation Template useful.
Module 5 > Section 1 > Internal Validation

Key takeaways

1. Get internal validation done early on!

2. Get internal validation done before proceeding.

Index > Session

Agenda

1. Internal Validation

2. Validating with Experiments

3. Customer Interviews
Module 5 > Section 1 > Internal Validation

Validating
with Experiments
Module 5 > Section 1 > Internal Validation

A/B Tests

One of the most common experiments is an A/B Test.

In an A/B Test, you make multiple variants of

whatever you’re trying to test.

For example, if you want to experiment with the labels

of your buttons on your mobile app or website, you
can run variants of your UI with percentages of your
user base.

However, you can’t do an A/B test with everything!

OPTION OPTION
Module 5 > Section 1 > Internal Validation

Minimum Viable Product

(MVP)

Most people talk about the type of product you’ll

actually build/ship, and we’ll cover that in another
session.

But there are very simple MVPs that you can

inexpensively create to help validate your hypothesis.
Module 5 > Section 1 > Internal Validation

Pre-order MVP

The simplest is a pre-order MVP. This helps you evaluate if

people are interested enough in your idea to pay for it.
Create a fake marketing website for your product,
describing it as if it already existed with the features you
feel are important, and add a “Buy” button. Then, market
the product like it actually exists: take out different display
ads and see how many people click Buy.
To avoid being completely deceptive, we recommend
adding a note when someone clicks Buy that says you
haven’t finished building this product, or are currently
working on it, and they can enter their email address to be
updated when you have new information.
Module 5 > Section 1 > Internal Validation

Concierge MVP

Manually work with your customers just like a real concierge

would perform a task. The task is the opportunity you think
you can do for them.

Blue Apron could’ve done a concierge MVP by manually

asking customers which meal they’d like to cook, assembling
the ingredients, and giving a box to customers.
Module 5 > Section 1 > Internal Validation

Wizard of Oz MVP

In Wizard of Oz MVPs you manually do your core task for

customers.

In a concierge, it’s clear you’re manually doing it—you

interact with customers. In a Wizard of Oz MVP, you make
the process look automated even though you’re doing it
manually.

Example: This is like in the early days of Netflix DVDs

Module 5 > Section 1 > Internal Validation

Fake Door MVP

The objective here is to integrate a user experience (UX)

trigger for a feature you're considering, without actually
developing the feature. Monitor and count the number of users
who interact with this trigger. To prevent users from
continually attempting to use this non-functional feature, it's
important to have some sort of feedback mechanism in place,
even if it's just a simple error message.
Module 5 > Section 1 > Internal Validation

Testing MVPs

Remember, none of these MVPs scale.

These are “testing” MVPs, ways to validate your product idea

with minimal work. They also don’t let you understand why
people do/don’t do certain things, but they let you see how
customers will react to the idea.
Module 5 > Section 1 > Internal Validation

Prioritization

After we’ve decided that yes, we should pursue this

opportunity, we need to make sure it’s the best thing to do
next.

Check out the Validating with Experiments Template here.

Discover advanced
prioritization frameworks
and how to apply them in
Product on our Product
Leader Certification.

Live Module 2: Translate Your

Product Strategy into Action
Module 5 > Section 1 > Internal Validation

Will it help us achieve our goal?

If not, even if it's a great

idea… don’t work on it
next.
Module 5 > Section 1 > Internal Validation

USER
SATISFACTION
DELIGHTER
Kano model (WOW)

PERFORMANCE
A way to gauge how critical a feature is (MORE IS BETTER)

for your user.

LESS INVESTMENT INVESTMENT

MUST HAVE

USER
DISSATISFACTION

Kano Template here.

Module 5 > Section 1 > Internal Validation

Value over time

Important to remember that business value changes

over time.

Fixing a bug in Product X is less important if Product

Y is about to be released.

Consider how long this opportunity will be useful to

you.
Module 5 > Section 1 > Internal Validation

Opportunity cost

It is present in everything & there is no perfect answer.

By strategically thinking about what to do next, you’ll

be on track to make good decisions.
Module 5 > Section 1 > Internal Validation

Key takeaways

1. Many ways to validate your product via

experiments.

2. Different MVPs help validate many

assumptions without expending valuable
engineering resources.

3. You can be more iterative about the products

you roll out.
Module 5 > Section 1 > Internal Validation

Time for a break

Stretch, breathe,
grab a drink

Estimated time: 5 minutes

Index > Session

Agenda

1. Internal Validation

2. Validating with Experiments

3. Customer Interviews
Module 5 > Section 3> Customer Interviews

Customer Interviews
Module 5 > Section 3> Customer Interviews

How do customer
interviews help Product
Managers?
Module 5 > Section 3> Customer Interviews

Customer interviews

Analytics Interviews Surveys

Analytics tell us what
customers are doing but not
why (signal)
Interviews are ways to talk
directly to a small # of
customers to understand them
(validation)
Surveys are in the middle
ground: we can get some
insight into lots of customers
and gather data around the
things we heard in interviews
(validation at scale and
quicker).
Module 5 > Section 3> Customer Interviews

Customer interviews

It’s a way to validate if the person you think is It’s a conversation to help you see if people
truly the right customer and seeing if you’re on truly have the problem you’re trying to solve
the right path for them with your product. and if they’d use your solution.

This includes understanding:

● What problems they have

● How they’re solving them right now

● How those solutions fall short

● How they decide if a new product’s

worth it
Module 5 > Section 3> Customer Interviews

Tips for a good customer interview…

Before anything, collaborate

with your user researcher.
They will lead the way and if
you don’t have one, you can
be your own user researcher.
Module 5 > Section 3> Customer Interviews

Tip #1: Set a clear

hypothesis

● This allows us to write our thoughts

I believe that <personas> experience
around who we should talk to, the
<pain> when doing <task> because
problems we think they have and what
of <limitation> and alleviating that
we think will help.
pain would let customers <achieve
gain>, although they’d have to
<accept this limitation>
Module 5 > Section 3> Customer Interviews

Tip #2: Know where

to find customers

● The company might have key ● Go where they are (gym, cafe, etc.)
customers already to reach out to. Talk
● Social networks
to the sales/support teams
● Aim to find extreme to non-extreme
● The company might have a beta
customers so that you get a range of
program for early adopters willing to
views!
test and tolerate bugs

● Run SQL queries on customer

databases and send emails/call them
Module 5 > Section 3> Customer Interviews

Tip #3: Find your user segment

Aim to identify from your user the following:

Behavior Motivation Goals Frustration

Using open-ended questions helps you to get a STORY

Module 5 > Section 3> Customer Interviews

Tip #4: Plan your interview questions

& stick to the script!

● You’ll also want to plan out some questions to

ask during the interview in advance.

● It’s easy to ask bad questions: good questions

can be hard!
Module 5 > Section 3> Customer Interviews

Tip #5: Close with an

important question

● This is a magic question worth asking at the

end of the interview. It’s a way to account for
things you might not have planned in advance. Is there anything else about <this
topic> that I should have asked
you about?
Module 5 > Section 3> Customer Interviews
Example: Understanding how users
buy homes in the UK
● User researchers drove the recruiting using a
third party vendor in London.
● User researchers prepared a script which they
used to drive the interviews.
● Share the nuances/similarities based on the
trends.
HYPOTHESIS
“Users buying homes in the UK have very
different needs than the those buying in the
US”
USER SEGMENT
Worked with user research to find the right
segments based on the market
Module 5 > Section 3> Customer Interviews

Key takeaways

1. Customer interviews are super important, use them to

validate or invalidate a hypothesis.

2. Stick to the scripts and use a user researcher.

3. Come back with conclusions to influence your

strategy.
Module 5 > Section 3> Customer Interviews > Exercise Unmute and share Estimated time: 110 minutes

Breakout Group Discussion:

Kano model/SWOT/ A/B Test plan

Validate your opportunity hypothesis by:

1. Using the Kano model to segment the product’s

features by listing its attributes (Basic Expectations,
Satisfier or Delighter).

2. Doing a SWOT Analysis to organize the product’s

strengths, weaknesses, opportunities & threats.

3. Running an A/B test to validate your opportunity

hypothesis.

4. Sharing!
Remember to use the Kano and SWOT Templates already
provided and the A/B Testing Template here.
Q&A