Decision Sheet: Measuring ROI

Protagonist: Sunil is the newly hired data scientist at Bazaar.com.

Case Facts:

1. Bazaar.com is a leading online retailer in the United States.

2. They use display advertising and search engine advertising (paid search ads on Google and Bing).

3. Keywords are classified into branded and nonbranded categories.

4. this discussion focuses on sponsored search advertising for branded keywords on Google.

5. The ROI calculation provided by Bob shows a high ROI of 320.0%.

Problems:

1. Concerns regarding the necessity of showing sponsored ads to users who already search for the
brand name.

2. Uncertainty about the potential impact of stopping sponsored search advertising for branded
keywords.

3. Lack of certainty whether the data from the first nine weeks is comparable to the data from the
last three weeks.

4. Time constraints due to an impending meeting with the advertising agency representative.

Problem Statement:

How can Sunil effectively analyze the ROI of sponsored search advertising for branded keywords on
Google considering the concerns raised by Bob and Myra, and the available data limitations?

Decision Statement:

Sunil needs to decide on the most appropriate analysis to determine the impact of sponsored search
advertising for branded keywords on Google and whether it should be continued.

Criteria:

1. Accuracy of the analysis.

2. Relevance to the concerns raised by Bob and Myra.

3. Ability to control for potential confounding variables.

4. Feasibility within the given time constraints.

Alternatives:

1. Compare the data from the first nine weeks to the data from the last three weeks to understand
the impact of stopping sponsored search advertising.
2. Conduct a regression analysis to control for potential confounding variables and determine the
true impact of sponsored search advertising.

3. Run a randomized controlled trial to directly assess the effectiveness of sponsored search
advertising for branded keywords.

Analysis:

1. Comparing Data: This approach may provide insights into the immediate impact of stopping
sponsored search advertising but may not account for potential seasonal variations or other factors
affecting website traffic.

2. Regression Analysis: This method allows for controlling variables and might provide a more
accurate understanding of the impact of sponsored search advertising.

3. Randomized Controlled Trial: While ideal for establishing causality, it might not be feasible within
the given time frame and resources.

Decision:

Considering the time constraints and the need for a comprehensive analysis, Sunil decides to conduct
a regression analysis to control for potential confounding variables and determine the true impact of
sponsored search advertising for branded keywords on Google. He will carefully interpret the results,
considering the concerns raised by Bob and Myra, to provide actionable insights for the marketing
team.