MKTG 4110 Class 2

Case study 1: Email marketing at a European sporting goods retailer

 Pentathlon is a sporting goods retailer
 The firm currently sends out promotional emails to its customers

Anna Quintero (digital marketing director):

 Sending too many would annoy customers
 They might ignore it or become hostile to the brand
 Therefore we should send a certain amount for optimization

Francois Cabret (department head):

 Should send more emails and not limit the numbers based on survey

1. How do you reconcile the evidence?

a. If you get a lot of spam but it does not indicate whether or not you order
b. You don’t know why the emails are varied
 Therefore the evidence cannot be correctly interpreted and it might be a reverse causality.
 For the Reverse Causality, if you buy more stuff you get more emails.
o We therefore do not if the emails are causing people to order more.
 There might be a Sample Selection Issue
c. The evidence was conducted based on the email list the firm had
d. Therefore you are left with a pool of complainers or enjoyers or emails
e. The people who are annoyed already blocked or were off the email list
f. Therefore the 72% should be way higher than what is shown
2. Given the evidence, would you impose a limit on promotional email activity?
 Send more or less? Is there an optimal point?
 Decrease and test and experiment see the difference?
ESTABLISHING CAUSALITY
- The measurement of nothing happening and something happening
- A situation where (A) something happens and another where (B) nothing happens
- One they see the ad and the other where they don’t
- If all conditions are equal,1 we can conclude the effect of the effect in the outcome
- But it is very difficult to make all things equal, as things are always different
o Culture, characteristics, personalities, timing, experience

Probabilistically equivalent
- Therefore continuous randomization with a large enough amount will average it out
- No systematic differences between groups (on how they would respond to ads)
- Any observed outcome will be due to the Ad

The # 1 reason why analytics goes bad

- data not generated as part of an experiment are presented or interpreted as if they were!
- This is especially caused by historical data without doing an experiment
- As you do not and have not equalize the factors

Always Experiment
- Business make targeted decisions that maximize profits
- Ads are targeted to maximize profit and Emails are targeted to maximize profits
- The gold standard in analytics is experimentation
- Ads are assigned randomly and Emails are assigned randomly

Uncover bad analytics (No to All. Yes to Any)

1. Are there pre-existing differences between groups?

Could groups be probabilistically not equivalent?
o There is no probabilistically equivalent as they weren’t randomly assigned
2. Is there a confound that could explain the outcome?
What else might be going on?
o People might already be interested in buying a car
3. Is there a common driver of both managerial decisions and outcome?
Decisions and outcomes both responding to a common factor?
o Such as consumer loyalty which affect your purchases and email amounts
4. Is there reverse causality?
• Outcomes -> Managerial decisions?
o The more you look for product you see more ads of that product

1
Ceteris paribus
Case study 2: Advertising Effectiveness
 Harmony makes ads for car dealers
 Advertise in Google for the car dealers and manufacturers
 Paul Rogan, finds that showing ads increase the purchase rate individually
 Imagine seeing a dealer ad on the banner of a sport site or a manufacturor ad

1. How would you summarize the lessons about advertising on Google to the
manufacturers?
- General interest vs specific interest
- Google is too general of a forum to find useful data
- If someone is interested in buying a car, they already bought it
- What is it actually based on?
2. Could you use the conclusion (on the right) to underline the value of this data?
 There are too many variables not provided in the data
 The data is garbage, as there is no way for us to identify the effect of the ad