102 views

Uploaded by dickson_anane

save

You are on page 1of 4

Information Management Online, June 21, 2002

David S. Coppock

"Lift" is probably the most commonly used metric to measure the performance of targeting

models in marketing applications. This article is a short lesson on what lift is, why it is

important and some pitfalls to avoid.

The purpose of a simple targeting model is to identify a subgroup (target) from a larger

population. The target members selected are those likely to respond positively to a marketing

offer. A model is doing a good job if the response within the target is much better than

average for the population as a whole. Lift is simply the ratio of these values: target response

divided by average response.

Lift is usually quantified by dividing the population into deciles – ten even groups – into

which population members are placed, based on their predicted probability of response. The

highest responders are put into decile 1, etc. Figure 1 shows a typical model applied against a

population with an average response of 5 percent.

One thousand total offers were made, so each decile contains one hundred members. In the

top decile, there were sixteen responders for a response rate of 16 percent. Compared to the

average response rate of 5 percent, this gives a lift of 3.20 for decile 1. Each successive decile

has a lower response rate. The deciles start performing worse than average after decile 4.

For each person targeted there is a cost of making the offer, and a corresponding profit if a

positive response is obtained. The marketer can calculate the profit of targeting each decile

and simply include each decile down to the last decile that is profitable. The cumulative

response rate and lift will then show the average performance of the model for everybody in

the target.

This information is often presented graphically:

We have total responders of 50. The

top two deciles have a total

response of 16+12 = 28. This makes

56 % of the total responders simply

because 28/50 = 56 %

predict the probability of response

for each individual. The highest

responders are put into decile 1, etc

built we will be able to target the

most likely people to respond with

less data. That is we will be able to

select only those that will likely

respond – which is less data-

compared to doing that at random

which per definition is linear. In this

case using the model we would need

to send adverts to only 20% to Figure 2: Lift Chart

target 56% of the likely responders. Another useful chart compares the cumulative percent of responses

With the model, it means working at

captured as each decile is added to the target. In the current example,

random and 20 % at random will the top two deciles capture about 55% of the responders. This is

target only 20% of the likely compared to a random baseline where two deciles (20% of the

responders. population) would capture 20% of the responders. This result is not

quite the "80/20" rule, but it is much better than not targeting. The

The next question is we need to find greater the area between the two lines, the more the model was able

the members that make up the top to concentrate responders in the top deciles.

20 percentile.

the percentage of the likely

responders. 100% will correspond to

the target 50.

the members in the database to

reach 100% that is the 50 likely

responders. From the graph using

the model we will need about 9

percentile.

marketing department may just

target the 5 or 6

percentile(corresponding to 80 0r Figure 3: Cumulative Percent of Responses Captured

90% of the target so about 40 to 45

It’s hard to say what level of lift represents "good" model

of the people likely to respond) as all

the other percentiles doesn’t help

performance, because the potential for predictive targeting varies

much based on the cost involved. widely between applications. A model predicting who will buy an

American-made car could be highly predictive (potentially a lift of

However, at random the 5 percentile 4+ in the top two deciles), because purchase patterns tend to be stable

will target only 50 % of those likely over time. A model predicting who will buy a white car may have a

to respond = 25 people likely to top lift of around 1.5 if color choice is more variable and random.

respond

Each of these may be "good" models in the sense that they do as good

a job of predicting as is possible. However, the model with the higher

lift will probably be much more useful in marketing applications.

Rather than using lift to evaluate a model in isolation, it is often more

useful to use lift to evaluate the relative performance of alternative

models. If a tree model provides higher lift than a neural net on the same data, this provides a

key factor in choosing between the models.

I will mention two things to watch out for when reviewing lift metrics. First, model lift

should be calculated on a holdout sample that was not used to estimate the model.

"Overfitting" can cause the model to predict well for the data set on which the model was

estimated. But this performance needs to carry over into new data sets.

Second, when calculating lift, always ask the question: Lift versus what? Model performance

should be compared against the marketing strategy that would occur if the model were not

available. The model should not be compared against an unreasonably pessimistic alternative.

Consider the following results of a direct mail program to sell magazine subscriptions.

As would be expected, the current subscribers had a much higher response rate for magazine

renewals than did prospects for new subscriptions. In the absence of modeling, current

subscribers and prospects would be treated as two separate groups. Targeting models, if used,

should be built separately for each group and evaluated only within that group.

My own comments: which percentile in each group (current subscriber and prospects

on their own right) should we target? That is perform modelling and analysis for each

group

Suppose, however, that the two groups were combined and treated as one overall group for

modeling purposes. The simplest model would use only one predictive variable – whether or

not the individual was a current subscriber or a prospect. Clearly, the model would put the

1,000 current subscribers in the top decile, and all the prospects in the bottom nine deciles. A

naïve calculation of lift for the top decile would divide the current subscriber response rate of

50 percent into the total response rate of 9.5 percent for a lift of 5.26. But the model is simply

taking credit for something that was obvious anyway. As usual, if it sounds too good to be

true, it probably isn’t.

Further comments

Starting point: What is the total number of possible responders from the

population? The answer to this can come from past campaigns.

Our aim is to have a model to (help us) build a customer segmentation arranged

in percentiles of the number of people with a certain probability to respond to

the advertisement. The top segment or percentile 1 is made up of the people

with the highest probability or likelihood to respond.

So as expected the people in the top segment will have some common profile

and as such will have similar values for their attributes to values to their

attributes

Among the total number of potential responders we would expect to have more

people to be in the segment 1/percentile1 and followed by segment

2/percentile2 etc...

The lift is defined as the number of possible responder in each segment divided

by the anticipated total number of possible responders for the whole customers

in the database (the population)

The lift and profit charts are ordered by decreasing probability of the target. The chart moves "up" when a

prediction is corrent and doesn't when a prediction is wrong. The predict probability in the legend indicates

the predict probability of the selection in the chart. That is, if you have clicked somewhere on the chart and

the legend shows predict probability = 52%, then everything to the left of the bar has a PredictProbability of

52% or greater and everything to the right has less than 52%.

This is very useful for profit charts, where you can determine the probability threshold you should use to

select candidates based on the predict probability value at the peak of the curve.

- Data ModelingUploaded byarjun2u
- Data Modeling PatternsUploaded byBhaswar Goswami
- Dimensional Data Modeling 3Uploaded byAnkal Rao
- 7336533 Data Modeling ERUploaded byhaseebsql
- Data Modeling BasicUploaded bys_iska
- Data ModelingUploaded byManikandan Suriyanarayanan
- Data ModelingUploaded bysreeiim
- Data ModelingUploaded byRaman Brar
- ERDDataModelingUploaded byAshish Chowdhary
- Data ModelingUploaded byJamal Aruna
- Dimensional Data Modeling IntroductionUploaded byChun Hsien Lee
- H10 Data ModelingUploaded byadnan8086
- Data ModelingUploaded bynaveen.lawrence
- DataModeling_Uploaded byמוחמד טאהא
- Data Modeling BasicsUploaded byrj0680
- Data Modeling -DaywiseUploaded bymaheshumbarkar
- Logical Data Modeling GuideUploaded byBoyapally Ravikanth Reddy
- Data ModelingUploaded byashishmtiwari6483
- Data Modeling By SatishUploaded byYellanki
- Conceptual Data ModelingUploaded byTaqi Shah
- Logical Data modelingUploaded bysayyan
- Best Books Data ModelingUploaded bySean Rout
- Data Modeling ContentsUploaded bygcarthick
- Data Modeling MIT2Uploaded byskyjj8
- Data ModelingUploaded bySanjana Garg
- Data Modeling and MDMUploaded byNawinprabhu_Ja_5348
- Semantic Data ModelingUploaded byschow906291
- Data ModelingUploaded bySaeed Oktar Jawanda
- Data ModelingUploaded byGHIELANG
- UML and Data ModelingUploaded byDede Ziber

- Le marché de l'art contemporain 2015, selon ArtpriceUploaded byFrance Culture
- Business Case OutlineUploaded byAmbITion
- Fijacion de Precios InternacionalesUploaded byMau Contreras
- Distribucion de Planta VatUploaded byYoha Olivos
- Assignment 1Uploaded byNAbihah AZman
- Pas 96Uploaded byINGACASILLAS
- Mercados y supermecados.pdfUploaded byAngelo Ascona
- GstUploaded byArun N Sankar
- final paper OIUploaded byReyhan Prasidya
- INDITEXUploaded byKanon Raven
- Unidad 4 - Métodos de Transporte y AsignaciónUploaded byDiana Concepción Mex Alvarez
- Plan de Produccion Bakhos Bordados 2Uploaded byCoralySilvaRios
- Monografía de la calidad total.Uploaded byCarmen Rosa
- Ducati CaseUploaded byChiradeep Majumder
- Colgate Case WriteupUploaded byRishit Shah
- Necanko, Inc.Uploaded byRizki Setyo Pratomo
- Matriz Aodf Cabo San ViejoUploaded byKarla Alvarenga
- Coste por impresión, coste por clic, coste por lead y coste por aUploaded byZoubair El Ahrach
- Chap 7 Writing Business PlanUploaded byVinesh Ramanathan
- Titan ProjectUploaded bySourav Pathak
- balance-scorecard-1223824921045112-8.pptUploaded byTania Rodriguez Incer
- LadrilloCV AdoquinC IIIUploaded byKenneth Barros
- cCGUploaded byMd Mostafa Kamal Faisal
- Ejerc.CosteoDirecto19JunioUploaded byHéctor Cortéz
- Mercadotecnia las armas del futuroUploaded byjose
- marches_ publicsMaroc.pdfUploaded byCrystal Kline
- Consolidated Scripts for Translation _Jul 2017Uploaded byJeenesh Suresh
- Proctor & Gamble Powerpoint Strategic OverviewUploaded byccollins1721
- 1_Compañia Minera MilpoUploaded byRony HP
- Advanced Microeconomics Petri Ch 5Uploaded byKarla Treminio