What is model drift?
A model learns patterns from a period of history. When the present stops looking like that history, its predictions get worse, usually slowly enough that nobody notices until a decision goes wrong.
Marketing data drifts more than most. Platforms rename fields and change attribution, privacy rules cut signals, new channels appear, and customer behavior moves with prices and seasons.
What are the types of model drift?
| Type | What changes | Marketing example |
|---|---|---|
| Data drift | The inputs the model sees | A new market launch shifts the audience mix |
| Concept drift | The relationship between inputs and outcome | A competitor's price cut changes how spend turns into sales |
| Upstream change | A field or definition the model depends on | A platform changes how it counts a conversion |
| Feedback drift | The model's own decisions change its future data | A bidding model shifts spend, which changes the data it learns from |
Detection usually combines two checks: statistical tests that compare recent inputs with the training data, and accuracy tracking once actual outcomes arrive.
What is an example of model drift?
A forecast that stopped fitting
A demand forecast tracked closely for a year. After a platform changed its conversion window, reported conversions dropped 15% overnight, though actual sales didn't move.
The model read it as falling demand and recommended cutting spend. A check on the input data flagged the sudden shift, and the team traced it to the definition change before acting on the forecast.
Data drift vs. concept drift
| Data drift | Concept drift | |
|---|---|---|
| What changed | Who or what the model sees | How the world responds |
| Detectable before outcomes arrive | Yes, by watching inputs | Usually not, it shows up as falling accuracy |
| Typical fix | Retrain on recent data | Retrain, and sometimes rethink the model |
Why does model drift matter?
Drift turns a trusted model into a confident wrong one. Because the model keeps producing numbers that look normal, the problem usually surfaces as a bad decision.
- Watch the inputs. Flag sudden shifts in volume, mix, or definitions before they reach a forecast.
- Track accuracy. Compare predictions with what actually happened as soon as outcomes arrive.
- Set a retrain rule. Decide what level of drift or error triggers retraining, and who signs off.
- Separate data problems from real change. A broken feed and a real market shift can look the same in a chart.
How does Alli do this?
Alli Alerts runs anomaly detection models built by PMG's data science team, which judge each metric against its own historical pattern and flag both unusual drops and unusual gains, alongside Custom Alerts for budget caps, pacing, and feed health.
Alli Datasources shows connection status and feed health for every source in one place, so teams can tell a stale feed from a real performance change, and Alli Semantic Layer keeps the definitions models depend on in one governed place.
Explore Alli AlertsModel drift FAQ
What causes model drift?
Changes in the real world that the model's training data didn't include, such as new audiences, new channels, price changes, seasonality, or changes to how data is defined or collected.
What is the difference between data drift and concept drift?
Data drift is a change in the inputs a model sees. Concept drift is a change in the relationship between those inputs and the outcome the model predicts.
How do you detect model drift?
Compare recent input data with the training data using statistical tests, and track prediction accuracy as actual outcomes come in. Alert when either moves beyond a set threshold.
How often should a model be retrained?
When monitoring shows drift or accuracy loss beyond an agreed threshold, or on a regular schedule for fast-changing data. Marketing models often need more frequent refreshes than models in stable domains.
Last updated Back to the glossary