A machine learning model is trained on a snapshot of the world. But the world doesn't sit still. Data drift is the quiet erosion of accuracy when the data your model meets in production no longer resembles the data it learned from.
Every model is trained on data collected during a specific window of time. The model learns the statistical patterns of that window — and freezes them. But the real world keeps changing. Customers evolve. Sensors age. Seasons turn. The gap between the photograph and the river is drift.
A model learns the relationship between features X and targets Y from historical data. The data defines everything the model knows.
The model is shipped to production. Predictions start flowing. Everything looks fine — for a while. But the input distribution has already begun to shift, slowly, invisibly.
Without warning, accuracy degrades. The model is asked to make predictions on data it has never seen — and its assumptions no longer hold. The cost of silence grows.
Below is a binary classifier trained on two Gaussian classes. Use the sliders to shift the production data and watch drift metrics respond in real time. Red circles are misclassified points.
The input features change distribution, but the relationship between features and target stays the same. A spam filter trained on emails from 2019 sees emojis it never learned.
The relationship between features and target changes. What was true yesterday is false today. Interest rates rise, and suddenly yesterday's loan-risk model is wrong about everyone.
The proportion of classes shifts. A fraud-detection model trained on 1% fraud rate now sees 8% — the prior is wrong, and calibrated probabilities no longer match reality.
A recommender trained on summer browsing patterns sees winter search behavior. Shorts become coats. Click rates collapse.
A recession changes which income brackets default. The model still expects 2019 default rates — and underprices risk everywhere.
A new MRI scanner produces images with different contrast. The model trained on the old scanner misclassifies healthy tissue as lesions.
Press play and watch twelve months of a model's life. Drift accumulates silently. Detection thresholds fire. Without intervention, accuracy decays toward random chance.
Buckets both distributions into the same bins, compares the share of points in each. PSI is symmetric and forgiving — the workhorse of industry drift monitoring.
Computes the empirical CDF of each distribution and finds the largest vertical gap between them. Distribution-free, exact for 1D, and excellent for continuous features.
Measures the information-theoretic cost of using distribution Q when the truth is P. Asymmetric — useful when you care about a specific reference direction.
Track PSI, KS, and prediction distributions continuously. Alert before accuracy drops.
Identify which features drifted, when it started, and whether upstream data sources changed.
Add recent labeled data to your training set. Weight newer samples if drift is severe.
Online learning, domain adaptation, or rolling training windows keep models current.