Problem Solving PME

14/11/2022, 21:33 You are the PM for a streaming video service.
treaming video service. You come into the office and see that one key metric has dropped by 80%. What will you do? — Problem Solving Question | PM Exercises
Go Back
You are the PM for a streaming video service. You come into the office and see that one key metric
has dropped by 80%. What will you do?
74.6k
Problem Solving
Asked at
Netflix
Amazon
& 2 other
Request views
companies
Answer
Share
Interview Guide
How to answer Problem Solving questions

Save
Mark as Read
Answers (28) Popular Post Answer
Top Problem Solving
interview questions
Liked by PM Exercises
You launched a new signup

PM Team
P
Platinum PM flow to encourage new users to
P
add more profile information.
Aug 12, 2018
A/B test results indicate that
the % of people that added
This is the type of Netflix problem solving interview more information increased by
8%. However, 7 day retention
question that has more than one answer. This makes it
decreased by 2%. What do you
critical to establish the right framework from the start.
do? 20 answers | 18.9k views

Drivers are dropping out of a

80% is a huge drop in any key metric.
city on Lyft. How do you figure

out what's going on? 15
I would try to first narrow down exactly what metric this answers | 10k views
is so I would ask the interviewer if they could tell me if

Show all questions
the metric is new user retention, churn, monetization etc.
Second, I would try to understand if this was a sudden or
gradual drop. for 80% definitely sounds like a sudden
drop or else somebody would’ve said something already.
If it’s a sudden drop, I would try to pinpoint around what
time this drop occured and figure out if there were any
internal/external factors that could have caused it.
internal factors include: new feature was released, server
went down, a new bug became prevalent. For the last
two, you can segment it by region, browser/device type,
and OS type. The issue could also be that the metrics we
are grabbing is incorrect.
external factors include: a new competitor has joined
into the market, bad PR, maybe a firmware was pushed
outside of your control. It could also be due to
seasonality or a major temporary event. If it’s a major
temporary event, you should see KPIs begin to return to
their normal state shortly.
Third, I would try to see if any other relational KPI drops.
It’s easier to know what KPI it is before, but we can go
along the user journey and see if any KPI before it
dropped.
https://www.productmanagementexercises.com/393/streaming-video-service-office-that-metric-dropped-what-will 1/28
14/11/2022, 21:33 You are the PM for a streaming video service. You come into the office and see that one key metric has dropped by 80%. What will you do? — Problem Solving Question | PM Exercises
IE: A user signs up for the service -> enters in a credit
card for payment (optional) -> clicks on a video to watch
-> Watches the video -> chooses another video to watch
This is important in narrowing down exactly when the
problem first starts. For example, if a key KPI is number
of videos watched, perhaps the sign in is where most
people are failing.
If the issue is a feature, I would try to clarify what the
goal of the feature is. It could be possible that we
started doing targeted ads and conversion dropped but
the first time purchase after clickthrough increased. It
would be important to understand if the goal of the
feature change was met even with this big of a KPI drop.
If I can ascertain the exact issue, I would work with Sys
Ops, Engineering, and other people on my team to try to
address it. If the issue is a bug, we would have to issue a
hotfix. If the issue is a server, than sys ops can look into
it. If the issue was due to a feature release, we should
probably look into either fixing it or reversing it quickly.
If the issue is external, this would be harder to solve
immediately and would often require going through the
normal cycle of product development to address them.
So in summary, I would first make sure we can ascertain
if the drop was temporary or permanent, gradual or
sudden and if the KPI drop may have occured elsewhere
int he user funnel. I would look at internal and external
factors to try to see if I can pinpoint the issue. Third, if
the issue can be fixed immediately, I would contact my
team to put out a hotfix or roll back a change that we
may have made. If not, we should understand the issue
thoroughly before acting and let people in the company
know of our findings.
A 225
S Mvotes | 6 feedback
Vote Give feedback
6 Feedbacks
Shashidhar Shivayogi
Bronze PM
B
S
Jan 2, 2022
Absolutely great answer. You have used a great
approach, asked the right questions, investigated
so well and provided a great summary. Given a 30
minute interview, it would be extremely hard to
beat such an answer. I also see some great points
in the feedback but I believe it is owing to such a
great start by you that makes it easier to build on
it and make it better. Well done!!
2 Vote Reply
Mat Griffiths
Bronze PM
B
M
Jan 19, 2022
Good answer, I would add one thing to the end: if
the problem was internal I would put a process in
place to prevent the same issue happening
again.
For example, if it was a bug that went undetected
in a release then we can put an automated test in
place that flags the issue.
9 Vote Reply
Show more feedback 2 of 6
IngaM
Platinum PM
P
I
Jun 3, 2020
Framework/steps: describe the product, clarify the
question, define the problem statement, draw the user
journey and ask a series of questions to zoom in on
which part of the funnel results in the problem, and
provide a quick solution (optional).
Describe the feature/product:
The company's primary business is its subscription-
based streaming service which offers online streaming
of a library of films and television programs, including
those produced in-house. We offer video streaming in
the US and internationally. Is this correct?
Interviewer: Yes.
Clarifying questions.
Is this a new product (post launch)?
Interviewer: No, we've had this service for years.
What's the absolute number of the drop (sometimes the
absolute number is small but percentage seems large)?
Interviewer: Assume that the absolute number is also
large.
Does this drop affect US, international or both markets?
Interviewer: All markets.
Are internal or external stakeholders affected by the
drop?
Interviewer: External.
So, the drop is affecting our end users, ie customers?
I: Yes.
Device, mobile or desktop?
Interviewer: Mobile.
Do we offer it on all types of phones (eg, iphone,
android)?
I: Yes.
Did the drop was device specific (e.g. iphone only)?
I: No.
Let me restate the problem and continue with more
questions.
We've been providing streaming video service to
customers in the US and internationally for years. The
product is available on desktop and on mobile (all
devices). Today we discovered an 80% drop in key metric
on mobile platforms (all devices) that affects external
customers in all markets (US and internationally). I'm in
charge of investigating the problem and finding the right
course of actions. Is this correct?
I: Yes.
As this is affecting our external users, i.e. customers, I'd
like to draw the funnel/user journey and try to identify
where the drop occurred. Does this sound good?
I: Yes.
As a user of the video service, I download the app on
mobile to watch the content in the car/etc, I create the
account/pay subscription fee, log in, search through
database of videos, filter the output, find the video I
want to watch, watch it, log out, repeat. Is this the right
sequence? Did I miss anything significant?
I: Yes, it's correct and no, you didn't miss anything.
As you mentioned the problem only affected mobile
users, is % of app downloads the same?
I: Yes.
What about % of paid mobile subscribers?
I: The drop is in paid mobile subscribers.
Is it a monthly/quarterly or daily subscribers?
I: Monthly.
With 80% in paid mobile monthly subscription, I'd need
to investigate further.
Internal:
Did we change how we calculate this metric?
I: No.
Has the drop happened in the past (seasonality)?
I: No, that's the first time the drop is that dramatic.
Was there a technical incident, bug, server issue that
affected the number of downloads this month?
I: No.
Did we release a new version that's lagging technically or
on UX?
I: No
Did we release a new free feature that's cannibalizing our
paid service?
I: No
External:
Did the % of mobile devices sold drop this
year/quarter/month?
I: No
Are there any micro/macro economic factors that
affecting the decline in % of paid mobile subscribers?
I: No.
Is there a competitor that's taking our market share?
I: Bingo.
I assume the competitor is focused on both the US and
international market and specifically on mobile. Is the
competitor providing free service or a major discount?
I: Free video content on mobile apps.
I'll restate the problem:
We've been providing a paid streaming video service to
customers in the US and internationally for years. The
product is available on desktop and on mobile (all
devices). This month we've discovered an 80% drop in
monthly paid subscription on mobile platforms (all
devices) in all markets (US and internationally). The
reason for the drop is a new competitor who entered a
video mobile market and offers a free video content.
151 votes | 9 feedback
Vote Give feedback
9 Feedbacks
Bijan
Platinum PM
P
Jun 12, 2020
Things you did well
Clarifying questions: Great set of initial
questions to help you understand the
problem statement.
Good diagnosis: You asked really good
questions to discover the root cause of the
problem
Things to improve
Solutions: After discovering the potential
root cause, I think you can continue with a
recommendation on how to deal with the
drop. If the question was asking "figure out
why", it would be ok to stop after
discovering the root cause. But since it's
asking "what will you do?" it expects you to
come up with some ideas to deal with the
drop
Great work. I enjoyed reading your answer.
39 Vote Reply
Igal Pronin
PM
PM
May 18, 2021
An outstanding answer, very clear and structured.
2 Vote Reply
Palash Kala
Platinum PM
P
Aug 27, 2020
As a PM for a streaming video service, I will have
different kind of metrics
North star metric being
Total number of subscribers
If I divide the metrics into different parts, they will
be along with major metrics
Awareness
Number of new visits per day (divided by
channel)
Acquisition
# of signups per day
Activation
We can measure quick activation
% of users who have watched at
least one episode by the number of
accounts which were created in the
last 24 hours
Engagement
DAU, WAU, MAU
Active user is
Watched at least one show per
day
Time spent by the activated users on the
product on average
Number of episodes/movies watched per
activated user
Retention
Stickiness; DAU/MAU
Cohort analysis
Daily retention by cohort when they
first logged in
Referral
# of users who share the
Revenue
Average customer lifetime value
Churn
# of subscribers not renewing or
canceling subscription by total
number of subscribers
Performance metrics
Page load timing
# of buffering required per video
Based on the metric which has dropped, I will
further solve the problem
Let’s say DAU dropped by 80%
What can be the different reasons
I will not check the metrics which
are optimizing ones, will only check
the ones that can impact the most
First, I will speak to users to find out
the reasons and find which part of it
the reason might be
Check the social media
comments
Content
A new show which is launched
and is available on competitors
is not available on netflix
Channel
Content takes a lot of time to
load (buffering)
Site/app crash
Network issues
Faced throughout the user
group
Other issues
Public relations issue?
What are people saying on
Social Media
Are people banning our
product for some reason
Based on the reason, I will then, solve the
problem working with the engineering team
P 29 votes | 1 feedback
Vote Give feedback
1 Feedback
Prashant Ranjan
Bronze PM
B
P
Oct 22, 2022
Nice approach.
0 Vote Reply
solders15
P
Platinum PM
S
Aug 12, 2018
1) Did this key metric drop for a particular market
(US/UK)?
2) Did this key metric drop on a specific platform (PC,
mobile (android, iOS..))?
3) If this metric is usage related, did the flow from
source/referrers (SEO, SEM, partners, FB..) to our
streaming service change? If so, which one?
4) Is this fall in key metric seasonal? Has it happened
before? Over what time frame has this key metric fallen?
5) If this metric is usage related, did the streaming video
usage go down for ALL services, or was it just ours?
6) Was this drop in the metric related to problems of
internet connectivity? DNS failure? Data center down? if
related to a specific market, did the ISPs go down?
7) Was there a natural calamity that has caused this
metric to go down?
8) Is the metric going down a telemetry related issue?
9) If this metric is usage related, is the metric going
down a result of a recent app redesign ? (user error)
10) Is the metric going down a result of the app
misbehaving? (app error)
11) Have the users gone to our competition?
11
P votes | 2 feedback
Vote Give feedback
2 Feedbacks
pankaj kumar
PM
PM
Aug 7, 2022
These are some really good questions you have
asked. I would really appreaciate if you can also
provide your solution if any of those issues
happens.
0 Vote Reply
Prashant Ranjan
Bronze PM
B
P
Oct 22, 2022
Nice clarifying questions, but I would like to know
how would you approach to find the root
problem.
0 Vote Reply
PMhummingbird
Gold PM
G
P
Aug 12, 2018
Any drop of 80% is a big drop.
Start by alerting your teammates: Customer support
team, Leadership, IT, Security, Dev team about this issue.
Setup a quick call so that people can jump into a
teleconference line to discuss the issue.
Send a reminder for the Customer Service, IT, Security &
Dev team leads to join immediately.
Start the call & inform everyone about the issue and ask
them to start looking into their respective dashboards
for any additional clues.
Meanwhile, you should start looking at user feedback
and other dashboards that might indicate an issue.
If the impacted conversion is something like Checkout,
then try to purchase an item and see if it goes through.
Try to access the site from inside & outside the network.
Within 30 minutes, if folks on the call haven’t been able
to figure out the issue, then drop a note to leadership
informing them about who all is on the call & are
investigating. Let them know that you will provide the
next update within 30 mins.
Ongoing updates do not need to be every 30 mins, but
you do need to keep digging in to figure out the problem.
That covers the few main areas in a Crisis:
1) Informing quickly
2) Escalating
3) Problem solving
Generally such a big drop usually means:
> DoS attack
> Problems with a new rollout
> Problem with the backend system or machine
Your team should be able to help you narrow it down to a
solution.
Once the team has figured out a solution, continue to
keep everyone informed and then address the fix with
the right team and see it to completion.
Once fix is implemented, thank everyone, especially the
team that figure out the problem & fixed it quickly.
Y 9 votes | 3 feedback
Vote Give feedback
3 Feedbacks
yunusshamu
PM
PM
Y
Jun 24, 2021
Your answer is good, but you are focusing on the
technical aspects of the potential issue. It could
be non-technical (eg. marketing campaign
change), possibly outside of our control (eg.
competitor activity, CDN partner making
changes) or even expected (eg. business model
change from ad supported/free service to paid
model). It will be better to investigate different
aspects of the problem.
2 Vote Reply
Sharayu Kulkarni
PM
PM
May 30, 2022
Your answer covers what,why and how about the
given problem. Informing quickly and escalation
can be solved with proper communication and
multiple follow ups. For problem solving you may
want to update more the team specific tasks too.
For ex- if it is a bug then provide a hot fix would
be a next action point and tracking the behaviour
of the hot fix would be a last mile.
0 Vote Reply
Varsha Datta
Bronze PM
B
Aug 22, 2020
You are the PM for a streaming video service. You come
into the office and see that one key metric has dropped
by 80%. What will you do?
Clarifying questions
1. What kind of streaming video service?
2. Is this a new product or service?
3. What is the key metric? Viewership, revenue, user
base
4. What is the platform for this service? Web, Phone
app, TV app etc.? And is the drop in one or all
platforms?
5. What is the end objective? Is it to find the root
cause or also propose a solution?
6. Is certain part of the geography affected?
7. Is the drop temporary or permanent, gradual or
sudden?
8. What's the absolute number of the drop (sometimes
the absolute number is small but percentage seems
large)?
Assumptions
1. There are different types – Pay by month, pay by
use, free. Assuming this is Pay-by-month
2. This product has been there for years
3. Assuming key metric is viewership
4. Assuming the drop is in TV apps
5. Assuming US only is affected
6. The drop is not on a major US holiday
7. Both absolute and % dropped majorly
Goal: Find the root cause and propose a

solution
Framework
Define the metric and verify the calculation
Since when is the decline happening?
Is the drop temporary or permanent, gradual or
sudden?
Factors that might impact
Geography
ISP Health
Competition (Price, content)
Content
Is that key metric impacting other metrics as well?
Issues with internal service
Seasonality
Implementation/Design
Metric: Number of viewers per day or week
Timeframe: 1 day
Factors:
External Factors:
Competition and Content: New
competitor in the market/new show in the
market/season finale/major shows
External server issue
Weather/natural calamity
Firmware was pushed outside of your
control.
Internal Factors:
New version release had a bug
Internal Server issue
Content: Some famous shows were
removed in the past day and the new
content was not appealing
Conclusion
Using the above framework, perform the analysis
with the above data and identify the root cause
If this is an internal issue
Add quality/appealing content: Time to
resolution will be longer since we have to do
market analysis and come up with a new
improvised content
Conduct periodic market studies to
understand competitor content
Resolve bugs by working with the IT/server
team – Time to resolution is immediate
If this is an external issue
We can only work with external server team to
resolve the issue asap
F 7 votes
P | 4 feedback
Vote Give feedback
4 Feedbacks
Floydimus
PM
PM
Sep 16, 2020
I really liked the structure but felt this was half
way done and seems like you missed some
critical pointers.
0 Vote Reply
Varsha Datta
Bronze PM
B
Sep 16, 2020
Thank you for the feedback. Would you be able to
provide some key critical points that I've missed?
This will help me structure my thoughts and
future responses. Thanks in advance
2 Vote Reply
SPMdavid
P
Platinum PM
S
Aug 12, 2018
We need to have a complete understanding of the key
metric before determining next steps. In this example,
the interviewer is asking about streaming service. The
best way to start to think about a popular streaming
service like Netflix and the relevant metrics.
More information is needed-
What is the frequency of measurement
(daily/weekly/monthly/quarterly/yearly)?
How is data collected and processed for this key
metric)?
When is the data presented?
Is this metric is related to acquisition, activation,
retention, revenue?
Is this metric specific for a device, show, region,
language, genre?
Depending on the answers, we can take specific steps.
Few scenarios-
1. New user signup in the USA using a mobile device
dropped 80% compared to the previous day. This metric
is measured daily, data collection is fully automated, and
data is presented daily with zero latency.
I will first rule out few reasons because Product
managers are supposed to know what is going on with
the product.
• I am ruling out any issues with cloud services (AWS or
own data centers) because everybody should have heard
about it even before checking key metric
• Everybody should have heard if the entire streaming
service is down for a significant amount of time either
due to a new deployment, dos attack during the previous
day.
• Somebody hacked and stole user data
Now, we can start identifying probable root causes.
• Is this normal? Did it happen before?
80% is a big drop but who knows if this has happened
before. Let’s assume not many new users sign up for
Netflix on super bowl Sunday or some big event.
• We are comparing 2 days of data. Let’s say we are on
day 4 of the week comparing the data for day 2 and day
3. Let’s analyze the data for day 1 to check for any
significant increase in user signup on day 2 compared to
day 1. This may happen if Netflix has released a new
season of a popular show.
• Was there any disruption in the signup service and
third-party credit card system?
Even if the streaming service was working fine there may
be issues with specific system components related to
sign up.
• Check the news for anything special happened
yesterday?
Few examples, Winter storm for most parts of the
country, terrorist attack, stock market tanking 3000
points
2. Revenue related to independent shows for last quarter
dropped 80% in the USA?
This metric is measured on a quarterly basis, data
collection is not fully automatic, and data is presented a
month after the quarter ends. This is tricky, but Netflix
produces the movies shows and make the shows
available free. They should have a way to calculate the
revenue tied to the following
• New users signed up because of the new show
• Drop in user churn rate because of the new show
This is key but complex metric requires extensive
analysis related to cost of the shows, no of shows, user
growth for the quarter. As data collection is not
automated and straightforward, there is a need to check
the assumptions and calculation model.
Vote Give feedback
rahul_g
Platinum PM
P
R
Apr 7, 2021
Step 1: Confirm if there's actually drop
Clarify: When you say 80% is it DoD, assuming the
answer is yes
Data sanity:
Did we make any changes to metric definition
Is the data complete
was there a surge yesterday which is making
the drop seem like 80% DoD(unlikely though)-
check against the trend last week, assuming it
confirm there's indeed a drop
Are we seeing a corresponding drop in other
related metrics
Is it continue or are we seeing a recovery
today
Isolate:
Internal factors
At what point did the start showing, did
anything change in that hour of the day
Was there any major app release
Was there any major instrumentation
changes release
Did we make any changes to event
definition
Was there any outage with any internal
systems/are we seeing any spike in error
rates if yes which system do they belong
to
User funnel: Is it because fewer users are
opening or same users are opening/where
is it that 80% drop is concentrated
Engg systems funnel: Is there a particular
step in the funnel where calls are
suddenly dropping
See drop split by
OS: android, ios, android TV etc
Platform: Mobile, app, web, TV
App version
Data center in case there are
multiple data center
Country/region
Connection type: wifi, web, mobile
User type: old vs new
Who has dropped off? isolate the
most loyal set of users among them
who have dropped off and see if
there's a pattern among them
Extenal
Reviews: are the users suddenly
complaining about something
Is anybody reporting issues with the
measurement tool we are using to
measure the metrics
Is anybody from the competition
reporting a similar issue - what are they
saying
Was there a major internet or electricity
outage in any of the major cities
Was there a major event yesterday that
lead to a drop For example a lot of people
are out on xmas so they dont watch TV as
much
I think among the above we should we be able to isolate
the issue
Once we have issue isolated (if there's one) if client side:
we'll have to push a hot fix + nudge all impacted users to
upgrade and recovery will mimic the upgrade rate curve
if server side then we should see a recovery immediately
post the the fix
Vote Give feedback
Patikoo
Silver PM
S
P
Apr 11, 2020
Me: Well 80% dropdown on a key metric? It is definitely
concerning. Before I dig in tell me, is this metrics relating
to revenue or engagement?
Interviewer: Revenue.
Me: Revenue is great challenge to solve but let focus on
an engagement metrics for retention and a long term
perspective. Does it make sense?
Interviewer: Yes.
Me: Great.
The first thing I will do is break down different data
dimensions so that I can isolate a cause. These data
factors are all around 3 main dimensions: 1- Users
property / 2- Features Attributes / 3- Funnel
1- User Attributes: let us investigate which user is
affected. Here we have two categories to consider: End-
user - Consumers Vs Video Creator / Admin.
For End-users, I will question some attributes
Is there a particular Geo/location or region that is
affected
Which OS or browser is the most affected
Language
For the Video Creator / Admin
Is there something wrong with the admin tool
App version
2- Feature property
What types of video is affected (Live or Past videos
Which Privacy settings are implemented
Is there a particular Geo/location or region that is
affected
Which OS or browser is the most affected
Language
3- As for the Funnel, I want to question how people
come to view the videos.
Do we have a problem with our feed impression?
Do we have a big in the video creation / posting
workflow?
Is the video notification feature working properly?
Are the video post URL broken/
Is the Share button down?
At the end, if I have all these information, I know a lot of
things and I will be able to work with developers and
engineers to isolate the problem and provide a solution
2P votes | 2 feedback
Vote Give feedback
2 Feedbacks
Bijan
Platinum PM
P
Apr 13, 2020
Hey there,
Great set of questions. Couple feedback:
- I would ask what metric was affected right
away. That's improtant for you to know. Once you
know the metric, it gets easier to continue your
investigation.
- Consider asking questions to ensure you know
when the drop happened? Was it day over day?
was there seasonality? Also, what if a promo was
just over?
- Consider separating internal and external
factors from each other and then continue
I hope it helps.
3 Vote Reply
Patikoo
Silver PM
S
Apr 13, 2020
You're right I could have asked more questions on
the clarification steps instead of guessing. Great
feedback thanks.
0 Vote Reply
Devashishv2012
Silver PM
S
D
Aug 7, 2022
1. Clarifying questions(I am assuming we are talking
about a platform like netflix or hotstar)
1. What is the key metrics we are talking about →

is average watch time per users— Yes
2. I would like to understand if this drop is abrupt
or gradual – Abrupt
3. I believe this streaming service is available in
multiple geographies. Is this drop specific to
any geographies or all geographies → APAC

4. I will further deep dive and try to understand if
this drop is specific to any device type i.e iOS,
android, mobile, web etc → uniform, agnostic

of platform
I will further deep dive and try to understand if there is
any external/internal factor which impacted this. Thing I
can think of for external factors are
2. External
1. Any new releases we have done – No
2. Any change in the technology side where users
areant able to browse the content . database
error etc → no
3. Any bad press which resulted into people
leaving our platform in order to protest → No

4. Any new competition on the market which has
resulted into majority of consumers moving
onto competitor platform → this should not

make any difference for existing paid users
unless the competitor has posted content
which wasnt perviously avilable on any other
digital platform
5. Is there any event going in the region i.e IPL ,
football world cup where most of the users are
engaging in watching this –No
3. Internal
1. Are we sure if analytics tool is reporting the
data correctly –Yes
2. A platform like netflix relies a lot on
personalisation. Have we changed anything on
the personalisation logic and people are not
able to find their most preferred content
3. Any experiment we are doing in home page i.e
moving recently watched, where you left
section to bottom in order to promote new
content → yes we pushed this a bit down as

we recently partnered with studio to show 5
new releases.
1. Do we know if the watchtime impact is
happening for customers who browse
most of the content from where you left
section–Yes 80% of customers continue
from where they left
2. This means the drop happened because
of the change in home page redesign
4. In order to fix this I would not launch this for
broader set of customers rather experiment
with smaller subset i.e 1-2% using A/B test.
During A/B we should get this insights and
page redesign should be done keeping in mind
the results from A/B test
1 vote | 0 feedback
Vote Give feedback
productmanager123
Gold PM
G
P
Jun 12, 2021
What is the streaming video service? Is it a B2C service
such as Youtube, Netflix? Or B2B service - where a cloud
provider is offering streaming service for companies
such as Disney to offer the actual streaming service.
Analyze the metric:
Assuming its a B2C service. What is the service? Assume
its a major broadcaster such as Disney, HBO who is
offering OTT service.
Which metric dropped by 80%? Revenue, monthly
subscriptions, MAUs, DAUs, WAUs? Number of hrs of
content streamed - monthly, weekly, daily? Lets assume
WAUs.
What is the baseline for the drop? Is it WoW or same
week last month? Assume WoW.
Is it seasonal? Did such drops happen around same
month last year? No
Has this metric been volatile all the time? For e.g, Did we
see many major WoW WAU drops (> 40% ) in the last 6
months? Basically want to understand if the metric is
volatile or has been steady. Not volatile.
Summary: B2C service. WoW WAU metric dropped. Not
related to seasonality. Not volatile.
Now lets look at who are the users and how they access
the service. These are users with active subscription who
turn on the app on phone, desktop or TV to watch.
Is the drop happening in a particular geo? US, RoW etc?
No. Common across all.
Did drop happen for users from a particular device? iOS,
android, smart TVs, web? No. Common across all.
Have the number of subscribers gone down WoW? Yes.
We need to further diagnose the reason.
Did we run a promo which expired. For e.g, Free access
for one month.
Did major content expire. For e.g, a popular show or
movie. Did that go offline on the service?
Has any competitor offered a new service ? Same type of
content, but half the price?
Lets assume promo has expired. Now what to do?
The fact that subscribers signed up during the promo,
but dropped off when it expired means, that there is
some reason why people signed up in the first place and
then dropped off because they didnt find any value.
What was the promo? Understand why people signed up.
Why are they now dropping off.
1 vote | 0 feedback
Vote Give feedback
Shashank Gupta
PM
PM
May 10, 2022
As a PM I Have my Dashboard that contains a few
important KPIs that help me with the feeling of the
health of the overall business.
80% drop is a hugely significant number so here are a
few steps that I would take -
1. Analyse the health of the app (Diagnosis of the
problem)
- Check if the data is being captured, and reported in a
correct manner. Doing a quick diagnosis of the overall
analytics part of the app.
- Check for internal factors - if there's any latest release
that has caused this issue, and identify bugs or code
incompatibility with the latest release or server outages.
- Check for the external factors that might have caused
to drop in this number say for ex - a very bad PR might
have caused this.
Internal factors can be resolved, external factors may
take time to resolve and if nothing major has happened
externally, then it will return to normalcy soon.
If the health of the app is good, we proceed to the next
step -
2. Funnel analysis of all the step that leads to that
metric. This will directly tell us which step is causing the
biggest drop and help us narrow the problem statement.
There might be that the fall may happen across the
steps not just on one particular step.
3. Once we have identified the step of the funnel causing
an anomaly in the drop, we start with the user
segmentation.
- Is this drop observed in a particular segment of users
or it has happened across the board?
- Is the drop the same across our casual, Core, and
power users?
- New users vs Old users - If new users have higher drops
then there might be a problem with the acquisition i.e.
we may be acquiring the wrong traffic
Depending upon what the diagnosis yields, we build our
solution accordingly. If there's a bug in the app, we get it
fixed immediately, if the new release has caused this
issue then we roll it back to fix it immediately. If there's a
problem across the funnel then we solve it for user flow
and usability via features improvement etc.
1 vote | 0 feedback
Vote Give feedback
Hari Govind
PM
PM
Apr 13, 2022
Assumption: Since the product in consideration is a
streaming service, I am assuming it to have a business
model similar to that of Netflix. (Subscription Model).
Moreover, the 80% drop seems to be a sudden drop
First I would deep-dive and identify the metric. I would
classify the metrics into -
Acquisition metrics:
No of new users logging-in and opting for
subscription
No of users making successful payment for 1st
time
Engagement metrics:
CTR of a button
Average Watch minutes per user
Conversion from Home Page to Detail Page
Retention Metrics:
No of old users coming on the landing page
No of successful renewals
I would take different cuts of the above based on
Geography, Device Type, Operating Systems, Time of the
Day. Moreover, I would see if any new release went live
the previous day and if all our necessary servers were
working as expected.
I would also consider externals factors -
Was the previous day a national holiday?
Has a competitor launched a new show?
Were there internet issues in any particular part of
the world where we are a major player?
After identifying the problem, will try to understand how
we can solve this as soon as possible; and will push the
dev team to send a Hotfix for the same
Vote Give feedback
Scott Thomas, MBA, PMP, Ent Architect, INCOSE CSEP
S
PM
PM
Jun 23, 2022
The first thing that comes to mind is that this is a very
substantial decrease and it sounds like it happened
abruptly so it's not a trend that they were tracking. It
says key metric but is unclear about what metric they are
speaking about so clarifying questions need to be asked.
Is it a service outage? Is it a drop in subscription rates?
Viewership during certain times of the day? Let's say it
was a service outage-related issue coming from a
certain region. In my experience, many service outages
are caused by network issues. A normal response could
be using a phone bridge to gather together with key IT
individuals and go through some troubleshooting steps.
I've found that the OSI model is a great tool to try to
isolate the problem, Based on the scenario my first
question might be to look at how the traffic is being
routed, is a device offline, could a port be blocked, did
someone change a firewall rule recently and is blocking
traffic and things like that. If it's subscriber-based then
you need to gather the right people and try to find the
root cause by performing an analysis. This might involve
customer questionnaires or even gathering a focus
group. It's even possible that some payment processing
services experienced a glitch and canceled everyone's
service. What time of the month is it? When do payment
processing and validation of customer purchases
happen? Is it an internal part of the company or an
external vendor doing this work?
Vote Give feedback
Sagar Anand
Silver PM
S
Jun 14, 2022
Q- A video streaming service , a key metric has dropped
by 80%, Abruptly in a day.
Clarifying Questions -
1- What is the type of streaming services - Free
trials (Netflix, Prime), Freemium, Content tiers, Free
2-In what all platforms the product is available? Is the
drop for any particular platform - Mobile (Android + IOS),
Desktop , TV, etc.
3-In which all geographies our product is available? Is
there any specific geography where we see the drop? -
All over India
4- What is the metric? Can we assume it to be a total
number of new subscriptions ?
5- Is the drop compared to yesterday's numbers or last
month's avg daily new subscription ? It could be possible
that today is normal but yesterday was a spike.
3- Users who complete payments but opt out of
subscription on the same day or later are counted in the
new subscription. Yes
After clarification -
There could be 3 broad categories of reasons for this
drop.
1- Internal to the company- Like app updates, glitches,
app slowness, advertisements, changes in the available
content, etc.
2- External - Like competitors, public sentiments,
calamities, events
3- Others - Its seasonal fluctuation, industry-wide
observation, calculation error, the metric was changed,
the metric is captured by a third party vendor and there
is a bug in their system.
Let me first clarify if any of the points from the 3rd
reason is applicable, if not, then I go looking for issues in
1 or 2.
Define the problem area. One of the ways is to check the
drop rate at major steps in the user journey and then dig
deeper into it.
1- Leads: Web + App (Paid and Unpaid)
It includes web traffic associated with
organic users, Ads, social media, affiliates, new app
downloads, etc.
Do we see any drop in the total number of leads?
2- Platform open rate (App + website) as % of 1
Do we see any drop in platform open rate?
3- User clicks on Buying option as % of 2
Do we see any drop in the buy option-click rate?
4- User what goes to the payment page as % of 3
Do we see any drop in the payment initiation rate?
5- Users who complete subscription purchase as % of 4
Do we see any drop in the subscription completion rate?
Reasons
There must be a drop in one or more of the above
numbers. Drop at a step could be due to these reasons
A - Changes before this and after the preceding step.
-It can be UI changes
-Dynamic variables change like content, text, images,
links, dropdown options, etc.
-Glitches, app slowness
B - Changes before the preceding steps
1- Change in composition of users basis Gender,
Geography, language, age, etc at step 1
2- Changes in the composition of users basis platforms
where they arrive like [Android app, IOS app, or
website] at step 1
3- Changes in the user composition basis device where
they arrive like a mobile, desktop, or tablet at step 1

C- External factors
1- Is the change due to external forces, like content,
prices, or specific shows in other sources.
2- It could be due to the availability of complementary
features such as coupons, cashback by affiliates, etc.
D - If not, draw a detailed user journey from the previous
step to this step, validate with the interviewer, and then
find specific issues around them.
Vote Give feedback
JK2
PM
PM
J
Apr 25, 2022
1. Point out that 80% is a significant drop.
2. Ask: Which metric dropped?
Engagement - Weekly watch time
Monetization - Monthly paid subscriber, monthly
advertisement revenue.
Growth - New users, daily active users
Interviewer: Assume the metric is weekly watch time.
3. Ask: Which user segments?
By geography - Asia, US, Europe
By age groups - Teenagers, working adults, mature
By platform - mobile, TV, desktop
Interviewer: Assume the users are teenagers.
4. Ask: Is this due to seasonal reasons?
Interviewer: No, this is not.
5. Ask: Is this due to any of the following reasons?
1. New competitors that took away teenagers’ time
consumption. For example, TikTok has recently
garnered a lot of attention.
2. Internal causes
Bugs.
New features.
Software performance issues such as slow loading
time.
3. Shows
Canceled popular shows.
A popular show just ended.
Interviewer: Let’s assume that a popular show just
ended.
6. There are 2 potential solutions.
1. Try to push for a new episode for that
particular show.
2. Recommend new shows that are similar and
resonate well with teenagers.
Interviewer: Let’s go with solution b
7. Let’s also set the metrics funnel to find out the
show that will increase the watch time for
teenagers.
1. Reach - number of users who see the thumbnail for
at least 30 seconds.
2. Reach - number of users who watch the trailers.
3. Engagement - number of users who watch 50% of
the show.
4. Engagement - number of users who watch 100% of
the show.
Vote Give feedback
Show more answers (12)
PREVIOUS
How would you improve Googl...
NEXT
If you were the PM for the ...
Companies
Google Product Manager Interview Questions Facebook Product Manager Interview Questions
Amazon Product Manager Interview Questions Microsoft Product Manager Interview Questions
Uber Product Manager Interview Questions Lyft Product Manager Interview Questions
Show More
Interview Questions
Product Management Product Design Interview Questions Product Management Product Strategy Interview
Questions
Product Management Product Improvement Interview Product Management Estimation Interview Questions
Questions
Product Management Behavioral Interview Questions Product Management Metrics Interview Questions
Show More
hello@productmanagementexercises.com
Terms & Conditions
Privacy Policy
Cookies
Copyright © 2021 PM Exercises

Problem Solving PME

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Problem Solving PME

Uploaded by

Copyright:

Available Formats

14/11/2022, 21:33 You are the PM for a streaming video service.

How to answer Problem Solving questions

You launched a new signup

do? 20 answers | 18.9k views

Drivers are dropping out of a

city on Lyft. How do you figure

is so I would ask the interviewer if they could tell me if

Second, I would try to understand if this was a sudden or

gradual drop. for 80% definitely sounds like a sudden

drop or else somebody would’ve said something already.

If it’s a sudden drop, I would try to pinpoint around what

internal/external factors that could have caused it.

internal factors include: new feature was released, server

went down, a new bug became prevalent. For the last

two, you can segment it by region, browser/device type,

and OS type. The issue could also be that the metrics we

are grabbing is incorrect.

external factors include: a new competitor has joined

into the market, bad PR, maybe a firmware was pushed

outside of your control. It could also be due to

seasonality or a major temporary event. If it’s a major

temporary event, you should see KPIs begin to return to

their normal state shortly.

Third, I would try to see if any other relational KPI drops.

It’s easier to know what KPI it is before, but we can go

along the user journey and see if any KPI before it

IE: A user signs up for the service -> enters in a credit

card for payment (optional) -> clicks on a video to watch

-> Watches the video -> chooses another video to watch

This is important in narrowing down exactly when the

problem first starts. For example, if a key KPI is number

of videos watched, perhaps the sign in is where most

people are failing.

If the issue is a feature, I would try to clarify what the

goal of the feature is. It could be possible that we

started doing targeted ads and conversion dropped but

the first time purchase after clickthrough increased. It

would be important to understand if the goal of the

If I can ascertain the exact issue, I would work with Sys

Ops, Engineering, and other people on my team to try to

address it. If the issue is a bug, we would have to issue a

it. If the issue was due to a feature release, we should

probably look into either fixing it or reversing it quickly.

If the issue is external, this would be harder to solve

immediately and would often require going through the

normal cycle of product development to address them.

So in summary, I would first make sure we can ascertain

if the drop was temporary or permanent, gradual or

sudden and if the KPI drop may have occured elsewhere

int he user funnel. I would look at internal and external

factors to try to see if I can pinpoint the issue. Third, if

the issue can be fixed immediately, I would contact my

team to put out a hotfix or roll back a change that we

may have made. If not, we should understand the issue

thoroughly before acting and let people in the company

know of our findings.

Vote Give feedback

Absolutely great answer. You have used a great

approach, asked the right questions, investigated

so well and provided a great summary. Given a 30

minute interview, it would be extremely hard to

beat such an answer. I also see some great points

in the feedback but I believe it is owing to such a