Uplift modelling

Setup#

pip install numpy scikit-learn

Verified with numpy 1.26.4 and scikit-learn 1.7.2.

The T-learner: two models, one subtraction#

Simulated randomised campaign: loyalty drives buying regardless; the coupon works only on price-sensitive newcomers. Because we built the world, each customer's true effect is known — so the targeting strategies can be scored honestly.

import numpy as np
from sklearn.ensemble import GradientBoostingClassifier

rng = np.random.default_rng(22)
n = 20_000
loyal = rng.random(n)                       # 0 = brand new, 1 = die-hard regular
sensitive = rng.random(n)                   # how much price moves this person
coupon = rng.random(n) < 0.5                # a properly randomised campaign

# loyal customers buy anyway; the coupon only moves price-sensitive newcomers
lift = 0.25 * sensitive * (1 - loyal)
buy = rng.random(n) < (0.15 + 0.55 * loyal + coupon * lift)

X = np.column_stack([loyal, sensitive])
m1 = GradientBoostingClassifier(random_state=0).fit(X[coupon], buy[coupon])
m0 = GradientBoostingClassifier(random_state=0).fit(X[~coupon], buy[~coupon])
uplift = m1.predict_proba(X)[:, 1] - m0.predict_proba(X)[:, 1]
likely = m0.predict_proba(X)[:, 1]          # "who buys anyway" — the naive target

top_uplift = uplift >= np.quantile(uplift, 0.8)
top_likely = likely >= np.quantile(likely, 0.8)
print(f"true effect on the top-20% by uplift score:  {lift[top_uplift].mean():.3f}")
print(f"true effect on the top-20% likely buyers:    {lift[top_likely].mean():.3f}")
print(f"true effect on a random 20%:                 {lift.mean():.3f}")

true effect on the top-20% by uplift score:  0.129
true effect on the top-20% likely buyers:    0.018
true effect on a random 20%:                 0.062

The walkthrough#

Uplift targeting doubles random; likely-buyer targeting falls below random. Per coupon: 0.129 extra purchases for uplift targeting, 0.062 for random spray — and 0.018 for the "target our best customers" strategy. Same budget, same models, different question asked.

Why do likely buyers underperform a coin toss? The likeliest buyers are the loyal regulars — exactly the people whose behaviour the coupon cannot change (the (1 - loyal) factor in lift zeroes them out). Random spray at least hits some persuadables by accident; likely-buyer targeting systematically selects the unmovable. This is the sweet-shop aunty receiving charity from the till, at scale.

This two-model recipe is the T-learner ("T" for two). Simple, uses any classifier (gradient boosting here), and its weakness is visible in the code: each model optimises its own arm's accuracy, and their errors do not cancel when subtracted. Small individual errors can dominate the small differences being estimated. The S-, X- and R-learner variants (researcher block) attack exactly this.

Real campaigns cannot print lift. True effects stay hidden; evaluation uses held-out randomised data ranked by score — checking that top-decile customers show the largest treated-versus-control gap. The standard picture is the Qini curve, cumulative incremental buys against people targeted.

Common mistakes#

Training on observational campaign data. If last year's coupons went to customers marketing chose, the two models learn the choosing policy's fingerprints — confounding inside a subtraction. The randomised requirement is not negotiable without the heavy machinery of earlier lessons.

Evaluating with accuracy or AUC. Uplift has no observable label — no customer reveals both futures. Model-quality claims need uplift-specific tools (Qini/AUUC on randomised holdout), not classification metrics.

Reading per-person scores as precise. Individual uplift estimates are noisy differences of noisy predictions. Ranking and decile statements are defensible; "Ramesh's uplift is exactly 0.40" is not.

Ignoring negative uplift. The do-not-disturb segment justifies the whole method in retention settings — skipping the bottom of the ranking is often worth more than refining the top.

Try it yourself#

Add a do-not-disturb segment: subtract 0.3 * (loyal > 0.9) * coupon-style annoyance from the buying probability and retrain. Check the bottom decile's true effect turns negative — and compute how much the campaign gains by excluding them versus targeting the top only.

What to learn next#

• Double machine learning — the orthogonal machinery under the R-learner.
• Potential outcomes and counterfactuals — the two-futures bookkeeping uplift operationalises.
• Recommender A/B testing — where uplift thinking meets ranking products.