Designing Renewal Prediction for Insurance

Renewal prediction in insurance is a well-defined ML problem with a surprisingly high rate of poor implementations. This post walks through the architecture and modelling decisions that actually matter, based on the patterns we see across general and life insurance products.

What Makes Insurance Renewal Different From Generic Churn?

Most churn models treat every lost customer the same. Insurance doesn't work that way.

A policyholder who lapses because a competitor offered a lower premium behaves very differently from one who lapses because a claim was mishandled, or one who simply forgot to renew. These aren't variations on a single signal — they're separate exit paths with different lead times, different feature signals, and different intervention strategies. Building one model and calling it "renewal prediction" is where a lot of teams go wrong.

The other structural difference is the policy lifecycle. Unlike SaaS, where churn is continuous, insurance has a hard renewal date. This concentrates risk into a narrow window — typically 30 to 60 days before expiry — but it also gives you a predictable label: renewed or lapsed at a known timestamp. That makes the supervised learning setup cleaner than most churn problems, if you frame it correctly.

A common framing mistake is defining the prediction target too late. If your model only scores policyholders two weeks before renewal, the retention team has no time to act on medium-confidence cases. You want to score at 90 days out, with a refresh at 45 days. This requires two things: a feature set that's informative that far in advance, and calibrated probabilities rather than raw scores, because the business needs to reason about likelihood, not ranking.

How Do You Build a Feature Set That's Actually Predictive?

The features that matter most tend to fall into four groups.

Policy economics — premium change at renewal (year-on-year delta), claims history (frequency, severity, recency), no-claims discount status, and coverage changes. Premium uplift is typically the single strongest predictor of lapse in motor and home lines. A 15% increase correlates with a significant drop in retention, though the exact threshold varies by segment.

Engagement signals — login frequency on the customer portal, whether the customer uses a mobile app or only interacts at renewal, mid-term contact history (calls, emails, web chat), and document download events. Customers who haven't logged in for 180 days and haven't called are cold. Customers who downloaded their policy wording 10 days ago are probably shopping.

Claim experience — not just whether a claim was filed, but how long it took to settle, whether it required escalation, and what the final outcome was. A customer whose claim took 45 days to settle and was partially disputed is a very different risk from one whose claim closed in 7 days with full payment.

Contextual and competitive signals — these are harder to get but worth it where possible. Postcode-level competitor pricing data (from comparison sites or licensed datasets), economic indicators like CPI changes in the policyholder's region, and vehicle or property age all add signal. They're also relatively stable, which helps at the 90-day horizon.

Feature engineering matters more than model choice at this stage. The ratio of current premium to market benchmark, the delta in claims frequency over a rolling 24-month window, days since last voluntary contact — these constructed features typically outperform raw inputs by a measurable margin in validation.

Choosing the Right Model Architecture

For a tabular dataset of the size most insurers have (hundreds of thousands to low millions of policies per year), gradient-boosted trees are the right starting point. XGBoost 2.x and LightGBM 4.x both perform well. The choice between them is usually about training speed and integration constraints rather than accuracy, since they're within a few percentage points of each other on most insurance datasets.

Calibration is non-negotiable. A model that outputs a score of 0.7 needs to mean "70% of policyholders with this score renew," not just "this person is ranked higher than someone with 0.5." Use Platt scaling or isotonic regression on a held-out calibration set. Validate with reliability diagrams. Under-calibrated models produce terrible expected value calculations downstream.

Neural approaches (tabular transformers, TabNet) occasionally outperform boosted trees on datasets with strong interaction effects between high-cardinality categoricals. In practice, the improvement rarely justifies the added complexity in a production renewal scoring pipeline. The exception is if you're incorporating unstructured data — call centre transcripts, claims notes — in which case a two-tower architecture that separately encodes structured and text features, then combines them, can push AUC meaningfully.

Multi-model Segmentation

A single model trained on the full book will underperform models trained on natural segments. At minimum, separate motor, home, and health lines. Within motor, private car versus commercial fleets behave differently enough to warrant separate models. The operational overhead of maintaining five models instead of one is real, but so is the accuracy gap.

Handling Class Imbalance

Renewal rates for most lines sit between 75% and 88%. That's not severe imbalance, so SMOTE and aggressive resampling are usually unnecessary. Class-weighted loss functions are sufficient. If you're working a specialty line with 60% renewal rates, resampling starts to help, but evaluate carefully — synthetic minority samples can introduce artefacts that hurt calibration.

What Does a Production Renewal Scoring Pipeline Look Like?

The scoring pipeline runs on a schedule tied to the policy renewal date, not a calendar schedule. Every policy gets scored at T-90 and T-45 days before expiry. The pipeline queries a feature store for pre-computed features, calls the model inference endpoint, writes scores and confidence intervals to a central store, and triggers downstream workflows — CRM queuing, outbound contact prioritisation, email personalisation.

A few implementation details that matter in practice:

• Feature freshness: if the feature store hasn't been updated in 48 hours, fail loudly. Stale features are worse than no features because they create silent errors.
• Score versioning: store the model version alongside every score. When you retrain, you need to know which predictions were made with which model for back-testing.
• Feedback loop: renewal outcomes need to flow back into the training dataset within two weeks of the renewal date. Longer than that and your model starts drifting without you noticing.

Monitoring should track PSI (Population Stability Index) on input features and expected calibration error on a rolling window of resolved predictions. If PSI exceeds 0.2 on a key feature, that's a signal to investigate before the next retraining cycle.

Does the Model Actually Change Renewal Rates?

This is the question that matters, and it's separate from model accuracy.

A model with 0.82 AUC that scores policyholders accurately is useless if the retention team's interventions don't work. The model is only as valuable as the action it enables. Before building, confirm that your retention team has differentiated interventions for different risk segments — a discount offer for price-sensitive lapsers, a claims experience call for dissatisfied customers, a simple reminder for high-intent customers who just need a nudge.

Measure lift against a holdout control group, not against the pre-model baseline. This is the only clean way to attribute improvement to the model rather than to the retention team's general activity.

If you're not ready to run controlled experiments, or if the retention team doesn't have bandwidth to act on model outputs, building the model first is the wrong sequence. Define the intervention playbook, then build the scoring system to feed it.

Conclusion

Start with the 90-day scoring horizon and calibrated probabilities. Get your feature engineering right before you tune the model. Run a proper holdout experiment before declaring success.

If you're at the point of deciding whether to build this in-house or bring in a team that's done it before, the honest answer is: the modelling itself is tractable for a competent ML team. The harder parts are data pipeline reliability, feedback loop infrastructure, and integrating scores into CRM and contact workflows in a way that actually changes behaviour. Those are integration problems as much as ML problems.

We're happy to talk through your specific data setup if you're in early design stages.

---

FAQ

How far in advance should we score policies for renewal prediction? Score at 90 days before expiry and refresh at 45 days. Scoring earlier than 90 days introduces noise because too much can change. Scoring later than 45 days leaves the retention team too little time to act on mid-confidence cases, especially for customers who need outreach rather than just an email.

What's the minimum dataset size needed to train a reliable renewal model? You need enough resolved policy cycles to have at least 5,000 to 10,000 lapse events in your training set after segmentation. Below that, a simple logistic regression with well-engineered features will usually outperform a boosted tree because there isn't enough data to fit the non-linearities reliably.

Should we use a single model or separate models per product line? Separate models per major product line. Motor, home, and health policies have different feature distributions and different lapse drivers. A joint model will find a mediocre average. The maintenance overhead of separate models is manageable with a standard ML platform and worth the accuracy improvement.

How do we measure whether the model is actually improving retention? Use a randomised holdout experiment. Score all policies, then randomly assign a proportion of high-risk policyholders to a control group that receives standard treatment. Compare renewal rates between the model-guided group and the control group. Without this, you cannot separate model value from general retention activity.

What happens when the model's predictions drift over time? Monitor Population Stability Index on key features on a weekly basis. Set a retraining trigger at PSI above 0.2 or when expected calibration error degrades beyond an agreed threshold. Most renewal models need retraining every six to twelve months due to shifts in pricing, product mix, and competitive environment.

Have a project in mind? Contact Sodio Technologies to discuss your requirements and explore the right technology solution for your business.