Churn Prediction: Find Customers at Risk, and the Ones You Can Still Save
Who is likely to leave, who can be saved, and where to act
Career figures, not model results. What your own model achieves is shown by backtests and a holdout test on your data.
You Can't Predict Churn Until You Define It
Churn prediction estimates the chance that a customer stops buying within a set future window, early enough to act while the relationship is still worth saving. But “churn” means two very different things, and each needs a different method.
Somebody presses "cancel"
Subscriptions, SaaS, memberships. Cancellation is a dated, visible event, so it's a clean prediction problem, and the model can also estimate how soon it's likely to happen.
Nobody cancels, they just stop
Retail, e-commerce, DTC. There's no label, so churn must be defined from your own buying rhythm, or modelled directly as the probability that a customer has quietly dropped out.
A fixed cut-off mislabels healthy customers
Why Accuracy Is the Wrong Way to Judge a Churn Model
Churners are usually a small minority. That makes accuracy a trap: a “model” that simply says nobody will leave scores very well while catching no one.
"Accuracy" of predicting that nobody leaves, while catching none of the 5 customers who actually go. That's why accuracy is never the headline number here.
Precision and recall, not accuracy
Recall: how many leavers you caught. Precision: how many flagged customers really left. The trade-off between them is a business decision.
The precision-recall curve
On rare events the usual "area under the curve" score looks flattering. A precision-recall curve focuses on the rare group that matters.
A cut-off set on real costs
Offer cost, customer value and the cost of discounting people who'd stay decide the cut-off. The default 50% is rarely right.
Imbalance handled carefully
Weighting rare cases is reliable. Tricks that invent synthetic churners can inflate offline scores, so every gain is checked on clean, unseen data.
If the model says 30% risk, roughly 30 in 100 such customers should actually leave. A model that ranks well but isn't calibrated is unsafe for deciding how much to spend on each customer.
What Actually Predicts Churn
Trends and ratios beat raw counts. The best churn signals describe how a customer is changing compared with their own normal behaviour, plus the moments where something went wrong.
Engagement trajectory
Whether activity is rising or falling matters more than the raw count. Recent activity is compared with the customer's own past average.
Recency vs their own rhythm
Silence is judged against each customer's normal gap between orders, not one universal number.
Friction events
Support tickets, failed payments, delivery problems, returns, downgrades. Often the strongest signals, and often missing because they live in other systems.
Leakage: the silent model killer
A signal like "received a win-back email" only exists because churn was already happening. Every input must be something the model would genuinely know at the moment of prediction, or offline results collapse in real use.
Fixed windows, written down
What the model looks at, and how far ahead it predicts, are defined explicitly:
How the Model Is Proven Before Anyone Uses It
A churn model is tested the way it will be used: trained on the past, then asked to predict a later period it has never seen. And it has to beat the simplest rule you could write.
Time-based split
Mirrors real life: learn from what happened, predict what comes next. Reported as precision and recall at the chosen cut-off, the precision-recall curve, and a calibration check.
Random split
Mixes future and past together, so the model quietly "sees the future". Scores look great offline and fall apart once the model goes live.
The benchmark is a simple recency rule ("hasn't bought in X days"). Many churn models barely beat it. If machine learning can't beat a rule you could write in one line, the honest recommendation is to use the rule.
Why discounting your riskiest customers wastes money
A short walkthrough of the three groups hidden inside every "at-risk" list, and how a holdout group shows which customers a retention offer actually saves.
A Risk Score Is Not a Targeting List
Discounting the top 10% riskiest customers sounds logical and usually wastes budget, because risk answers “who will leave?”, not “who can we persuade?”. Every at-risk list hides three groups:
Will leave without help, and stay with it. The only group worth the retention budget.
- Action
- Send: the offer or the call
Would have stayed anyway. A discount here is margin given away, then credited to the campaign.
- Action
- Send: normal messaging
The decision is made. Spend is wasted, and unwanted messages can even speed up the exit.
- Action
- Send: nothing, save the budget
Uplift: predicting the difference an offer makes
Instead of "who is risky", the model estimates who changes their mind because of the intervention. That needs a deliberate holdout group that gets no offer. It costs some preventable churn, and it buys real evidence instead of a campaign taking credit for customers who'd have stayed.
Same risk, same period, one difference: the offer. That gap is the only honest measure of what retention spend achieved.
Risk × value: who is worth saving
Paired with predicted LTV, two customers with the same risk can deserve very different treatment.
Where Churn Scores Go, and What They Change
A score sitting in a spreadsheet changes nothing. Each customer’s risk, value and main reason are sent into the tools your team already uses, on a schedule. Clean data flow into ad platforms is covered under Signal Engineering.
Lifecycle messaging
Risk tiers pushed into your email or CRM tool, so flows trigger on modelled risk instead of fixed timers. Usually the fastest win.
Human outreach
A prioritised call queue for customer success, weighted by customer value, not just risk.
Paid media
At-risk segments targeted for retention; lost causes suppressed. Suppression is often the cheaper gain.
Product & operations
The reasons behind churn sent to the teams that own them. Fixing failed payments or late deliveries often beats any discount.
What drives the predictions often points to a fixable operational problem. Sometimes the right answer isn't a campaign at all, but a payment retry, a courier change or a clearer cancellation flow.
Platform Churn Scores vs a Custom Churn Model
Many tools already offer a churn score, and for some mid-market brands that’s enough. A custom churn model, built on your raw data with advanced data science and supervised by me, earns its cost when the predictors live in systems a platform can’t see, or when the retention budget is big enough to justify uplift targeting.
Platform churn scores are compared in detail on CRM & Marketing Automation SaaS vs Cognitive Intelligence.
| CRM / email platform score | Product analytics tools | DIY in your warehouse | Custom model (Cognitive Intelligence) | |
|---|---|---|---|---|
| What you get | A built-in risk score, nothing to build | Strong usage cohorts and early warnings for apps | Your definition, inputs and cut-off, if you have the team | Churn defined from your data, cut-off set on your economics, uplift with a holdout |
| Where it stops | Vendor's churn definition, limited inputs, cut-off not yours | Weak on billing and support data, often the best predictors | You own pipelines, retraining, leakage checks and monitoring | Needs enough churn events, and a willingness to hold some customers out |
| Who owns the model | The vendor | Shared, generic | You | You, with code and outputs |
| Who reviews it | Nobody | Nobody | Your own data team | Me, at every retrain |
When to stay with the platform score: your churn definition is standard, the key signals live inside that platform, and the retention budget is modest. The audit tells you which side of that line you're on.
What Data a Churn Model Needs
The binding constraint is the number of churn events, not the number of customers. A big customer base with very few leavers can still be too thin to model.
Event history with timestamps
Orders, sessions or usage, with customer IDs and dates, and enough history for both the look-back and prediction windows.
Friction data
Tickets, failed payments, refunds, returns, delivery exceptions, downgrades. Often the strongest signals, and the most often missing.
Persistent identity
Recognising the same customer across sessions and channels. Guest checkout breaks the trajectory the model needs.
With too few churn events, a supervised model can't learn reliably. A dropout-probability approach or a well-tuned recency rule will serve you better, and the audit will say so.
Who Churn Prediction Is Built For (and Not For)
Churn modelling pays off where keeping a customer is worth real money, and where there’s a budget and a team ready to act on the scores.
Good fit
- Subscription and SaaS, where cancellation is visible
- Replenishment e-commerce: beauty, supplements, grocery, pet
- Memberships and services: gyms, insurance, utilities, telecoms
- Teams with a real retention budget to spend wisely
- Businesses already modelling LTV, or planning to
- Organisations willing to run a holdout
Not a fit yet
- Very few churn events per period
- No customer IDs or timestamps on orders
- One-off purchases with no repeat pattern
- No budget or team to act on the scores
What Actually Changes in the Numbers
No outcome is promised. What changes is where retention money goes, and the holdout makes the programme prove its own worth, including the possibility that it’s worth less than it costs.
Margin stops leaking
The biggest saving is not discounting customers who'd have stayed. That leak is invisible in standard campaign reports.
Human time follows value
Ranking by risk × predicted value changes who gets a personal call and who gets an automated email.
Root causes become visible
The drivers behind the scores often point to fixable operational failures worth more than any campaign.
Where AI Agents Fit Into Churn Prevention
A SaaS tool is someone else’s generic model. An AI agent is your own model, kept running: scoring on schedule, watching for drift and catching problems between reviews. The agents are autonomous, built from data science and marketing work, and every action is reviewed. More on the AI agents hub.
What the agents watch
- Spots a shift in the churn rate before the monthly report does
- Flags rising friction: a spike in failed payments, delivery issues or tickets
- Watches predicted vs actual churn, so a drifting model is retrained early
- Checks every scoring run: calibration, missing data and holdout integrity, before scores reach your tools
What stays with a person
- Defining churn. A commercial judgement that changes every number downstream.
- Setting the budget and the offer. An optimiser pointed only at retention will always recommend giving more away.
- Protecting the holdout. There's constant pressure to shrink the control group. Someone has to say no.
- Reading the model as diagnosis. Operational findings must reach the team that owns them.
Churn Prediction: Frequently Asked Questions
What is churn prediction?
Churn prediction estimates the chance that a customer stops buying within a set future window, so you can act while the relationship is still worth saving.
How do you define churn when nobody cancels?
From your own data: a threshold based on each customer’s normal gap between orders, or a model of the probability that a customer has quietly dropped out. Never a round number picked in advance.
How accurate are churn models?
Accuracy is the wrong yardstick, because churners are rare. Models are judged on precision, recall, the precision-recall curve and calibration. Any accuracy figure quoted before seeing your data is a sales claim.
Why not just discount everyone who is flagged?
The flagged group mixes customers who can be persuaded with those who would stay anyway and those already gone. Uplift targeting with a holdout separates them, so the budget goes only where it changes the outcome.
Why do we need a holdout group?
Without untreated customers you can’t know whether retention efforts worked. The cost of some preventable churn buys evidence you can trust.
Do you use a churn tool or platform?
No. Each model is custom-built on your raw data with advanced data science, checked on later periods it hasn’t seen, and supervised and reviewed by me at every retrain. You own the model, the code and the outputs.
What data is needed?
Timestamped orders or usage with customer IDs, friction data such as tickets and failed payments, and a way to recognise the same customer over time. The number of churn events is the main constraint.
Can we just use our CRM or email platform's churn score?
Often yes for mid-market brands with standard definitions. A custom build pays off when key predictors sit outside that platform, or the retention budget justifies uplift targeting.
Do you work with brands outside Pakistan?
Yes. Delivery is remote from Lahore, for brands across Pakistan, the UK, USA, UAE, Canada and Turkey, with cut-offs and offers set per market.
AI-driven digital marketing intelligence consultant. 12+ years in digital marketing, 100+ clients across 15 industries and 6 markets, a Master's in Computer Science and MPhil / MS Data Science (AI). Every churn model is built, checked and reviewed by me personally, with no hand-off to an account manager.
Where to Go Next
Find Out Whether Your Data Can Support a Churn Model
A data audit checks your churn definition, event volume and friction data, then tells you plainly whether a custom model would beat a simple recency rule, including when the rule is the better choice.