Why Lift?

Data Modeling and Mining

Information Management Online, June 21, 2002
David S. Coppock
"Lift" is probably the most commonly used metric to measure the performance of targeting
models in marketing applications. This article is a short lesson on what lift is, why it is
important and some pitfalls to avoid.
The purpose of a simple targeting model is to identify a subgroup (target) from a larger
population. The target members selected are those likely to respond positively to a marketing
offer. A model is doing a good job if the response within the target is much better than
average for the population as a whole. Lift is simply the ratio of these values: target response
divided by average response.
Lift is usually quantified by dividing the population into deciles – ten even groups – into
which population members are placed, based on their predicted probability of response. The
highest responders are put into decile 1, etc. Figure 1 shows a typical model applied against a
population with an average response of 5 percent.

Figure 1: Response Rate and Lift Calculations

One thousand total offers were made, so each decile contains one hundred members. In the
top decile, there were sixteen responders for a response rate of 16 percent. Compared to the
average response rate of 5 percent, this gives a lift of 3.20 for decile 1. Each successive decile
has a lower response rate. The deciles start performing worse than average after decile 4.
For each person targeted there is a cost of making the offer, and a corresponding profit if a
positive response is obtained. The marketer can calculate the profit of targeting each decile
and simply include each decile down to the last decile that is profitable. The cumulative
response rate and lift will then show the average performance of the model for everybody in
the target.
This information is often presented graphically:
We have total responders of 50. The
top two deciles have a total
response of 16+12 = 28. This makes
56 % of the total responders simply
because 28/50 = 56 %

Remember we used the model to

predict the probability of response
for each individual. The highest
responders are put into decile 1, etc

So the point is when a good model is

built we will be able to target the
most likely people to respond with
less data. That is we will be able to
select only those that will likely
respond – which is less data-
compared to doing that at random
which per definition is linear. In this
case using the model we would need
to send adverts to only 20% to Figure 2: Lift Chart
target 56% of the likely responders. Another useful chart compares the cumulative percent of responses
With the model, it means working at
captured as each decile is added to the target. In the current example,
random and 20 % at random will the top two deciles capture about 55% of the responders. This is
target only 20% of the likely compared to a random baseline where two deciles (20% of the
responders. population) would capture 20% of the responders. This result is not
quite the "80/20" rule, but it is much better than not targeting. The
The next question is we need to find greater the area between the two lines, the more the model was able
the members that make up the top to concentrate responders in the top deciles.
20 percentile.

Vertical axis of the graph represents

the percentage of the likely
responders. 100% will correspond to
the target 50.

So @ random we need to contact all

the members in the database to
reach 100% that is the 50 likely
responders. From the graph using
the model we will need about 9
percentile.

Based on cost benefit analysis the

marketing department may just
target the 5 or 6
percentile(corresponding to 80 0r
90% of the target so about 40 to 45
of the people likely to respond) as all
the other percentiles doesn’t help
much based on the cost involved.
However, at random the 5 percentile
will target only 50 % of those likely
to respond = 25 people likely to
respond
Figure 3: Cumulative Percent of Responses Captured
It’s hard to say what level of lift represents "good" model
performance, because the potential for predictive targeting varies
widely between applications. A model predicting who will buy an
American-made car could be highly predictive (potentially a lift of
4+ in the top two deciles), because purchase patterns tend to be stable
over time. A model predicting who will buy a white car may have a
top lift of around 1.5 if color choice is more variable and random.
Each of these may be "good" models in the sense that they do as good
a job of predicting as is possible. However, the model with the higher
lift will probably be much more useful in marketing applications.
Rather than using lift to evaluate a model in isolation, it is often more
useful to use lift to evaluate the relative performance of alternative
models. If a tree model provides higher lift than a neural net on the same data, this provides a
key factor in choosing between the models.
I will mention two things to watch out for when reviewing lift metrics. First, model lift
should be calculated on a holdout sample that was not used to estimate the model.
"Overfitting" can cause the model to predict well for the data set on which the model was
estimated. But this performance needs to carry over into new data sets.
Second, when calculating lift, always ask the question: Lift versus what? Model performance
should be compared against the marketing strategy that would occur if the model were not
available. The model should not be compared against an unreasonably pessimistic alternative.
Consider the following results of a direct mail program to sell magazine subscriptions.

Figure 4: Magazine Campaign Performance

As would be expected, the current subscribers had a much higher response rate for magazine
renewals than did prospects for new subscriptions. In the absence of modeling, current
subscribers and prospects would be treated as two separate groups. Targeting models, if used,
should be built separately for each group and evaluated only within that group.
My own comments: which percentile in each group (current subscriber and prospects
on their own right) should we target? That is perform modelling and analysis for each
group
Suppose, however, that the two groups were combined and treated as one overall group for
modeling purposes. The simplest model would use only one predictive variable – whether or
not the individual was a current subscriber or a prospect. Clearly, the model would put the
1,000 current subscribers in the top decile, and all the prospects in the bottom nine deciles. A
naïve calculation of lift for the top decile would divide the current subscriber response rate of
50 percent into the total response rate of 9.5 percent for a lift of 5.26. But the model is simply
taking credit for something that was obvious anyway. As usual, if it sounds too good to be
true, it probably isn’t.

Further comments

Starting point: What is the total number of possible responders from the
population? The answer to this can come from past campaigns.
Our aim is to have a model to (help us) build a customer segmentation arranged
in percentiles of the number of people with a certain probability to respond to
the advertisement. The top segment or percentile 1 is made up of the people
with the highest probability or likelihood to respond.

So as expected the people in the top segment will have some common profile
and as such will have similar values for their attributes to values to their
attributes

Among the total number of potential responders we would expect to have more
people to be in the segment 1/percentile1 and followed by segment
2/percentile2 etc...

The lift is defined as the number of possible responder in each segment divided
by the anticipated total number of possible responders for the whole customers
in the database (the population)

The lift and profit charts are ordered by decreasing probability of the target. The chart moves "up" when a
prediction is corrent and doesn't when a prediction is wrong. The predict probability in the legend indicates
the predict probability of the selection in the chart. That is, if you have clicked somewhere on the chart and
the legend shows predict probability = 52%, then everything to the left of the bar has a PredictProbability of
52% or greater and everything to the right has less than 52%.

This is very useful for profit charts, where you can determine the probability threshold you should use to
select candidates based on the predict probability value at the peak of the curve.