Loading
Model drift is the gradual decline in an AI or machine learning model's predictive accuracy after deployment, as the statistical patterns in live, real-world data diverge from the patterns the model was originally trained on. It is not a bug or a one-off failure but an expected, ongoing property of any model operating in a changing environment, from a fraud-detection system built on pre-pandemic spending patterns to a credit model whose definition of "creditworthy" shifts with the economic cycle. Left undetected, drift silently erodes accuracy, fairness, and reliability long after a model has passed its initial validation and gone into production. Because drift accumulates gradually rather than announcing itself with an outage, prudential regulators including Australia's APRA and the US Federal Reserve have made continuous, drift-aware monitoring, not one-off, point-in-time testing, an explicit supervisory expectation for AI and other probabilistic models.
Run the free AI Health CheckModel Drift, the degradation of an AI model's performance over time as the real-world data distribution diverges from the distribution on which the model was trained.
Model drift takes two forms. Data drift occurs when the statistical properties of inputs change (e.g., a fraud model trained pre-pandemic facing post-pandemic transaction patterns). Concept drift occurs when the relationship between inputs and the target itself changes (e.g., what counts as creditworthy changes with macroeconomic conditions). Both require ongoing monitoring and a retraining cadence: APRA's 30 April 2026 letter to all APRA-regulated entities warned that "few entities had continuous validation or monitoring in place to detect issues such as model drift, bias, failure modes, or control breakdowns in a timely manner," and that point-in-time, sample-based assurance is "ill suited to probabilistic models that learn, adapt and degrade over time."
Source: APRA 30 April 2026 industry letter; Federal Reserve SR 26-2 (17 April 2026), superseding SR 11-7
Data drift (covariate shift)
The statistical distribution of the model's input features changes even though the underlying relationship between inputs and outcomes stays the same. Example: a fraud model trained on pre-pandemic transaction volumes and channels facing a post-pandemic surge in online payments.
Concept drift
The relationship between inputs and the target outcome itself changes, so the same inputs now imply a different result. Example: the factors that once predicted creditworthiness shift as interest rates and household budgets move.
Label / prediction drift
The distribution of the model's own outputs, or of the ground-truth labels used to check it, changes over time. Often a downstream symptom of data or concept drift, and frequently the first signal to show up on a monitoring dashboard.
Upstream / schema drift
A change in the data pipeline itself, such as a source system that silently renames, rescales, or stops sending a field. It can look like model drift in the metrics but is actually a data-engineering fault that needs a different fix.
Two documents anchor current supervisory thinking. In the United States, the Federal Reserve's SR 26-2, Revised Guidance on Model Risk Management (issued 17 April 2026), supersedes SR 11-7 (2011) and the more recent SR 21-8 (2021). While SR 26-2 shifts toward a more tailored, materiality-based approach, it explicitly retains conceptual soundness, performance analysis, and ongoing monitoring as core validation disciplines that must be maintained across a model's life, not just at launch.
In Australia, APRA's 30 April 2026 letter to all APRA-regulated entities was more direct: supervisory work found that few entities had continuous validation or monitoring in place to detect issues such as model drift, bias, failure modes, or control breakdowns in a timely manner. APRA said point-in-time, sample-based assurance methods are ill suited to probabilistic models that learn, adapt, and degrade over time, and it expects monitoring to be continuous and proportionate to the criticality of the use case, from pre-deployment assessment through the full operating lifecycle.
Neither document treats drift as a purely technical nuisance. Both frame undetected drift as an operational and prudential risk deserving the same board and accountable-executive attention as any other control failure, which is why drift monitoring is increasingly written into model risk management frameworks and AI inventories rather than left solely to MLOps dashboards.
Statistical monitoring
Track input distributions and output or error metrics over time using measures such as population stability index (PSI), KL/KS divergence, or rolling accuracy against delayed ground truth, with alert thresholds calibrated to the model's criticality.
Champion-challenger testing and a retraining cadence
Run a candidate model alongside the production model on live data, and set a retraining or recalibration schedule, whether periodic or triggered by a monitoring threshold, instead of waiting for a scheduled annual revalidation.
Named ownership and a model inventory
Assign a named accountable owner to each production model and track it in a model inventory, so a drift alert has a clear escalation path rather than being lost between data science and the business line.
Documented human oversight and response plan
Define in advance who can pause, roll back, or retrain a drifting model, and what evidence, drift metrics, incident log, revalidation sign-off, needs to be captured for the audit trail.
What is model drift in simple terms?
It is when an AI model gets less accurate over time because the real-world data it sees in production no longer matches the data it was trained on, not because the model's code changed, but because the world around it did.
What's the difference between data drift and concept drift?
Data drift is a change in the inputs themselves, such as the mix of customers, transactions, or images looking different. Concept drift is a change in the relationship between those inputs and the correct answer, so the same inputs now mean something different. A model can experience either, or both, at the same time.
Is model drift the same thing as model decay?
The terms overlap in everyday use. 'Model decay' usually describes the general performance decline that results, while data drift and concept drift are the two specific mechanisms that cause it. When reading a vendor or regulator document, check which mechanism is actually being described.
How often should AI models be monitored for drift?
There is no single universal legal frequency, but supervisors including APRA now expect continuous, proportionate monitoring rather than a one-off validation at launch or a purely annual check, with the depth and frequency scaled to how critical the model is and how fast its operating environment changes.
What do the Federal Reserve and APRA require about model drift?
The Federal Reserve's SR 26-2 (17 April 2026, superseding SR 11-7) keeps ongoing performance monitoring as a core model-validation discipline for the banks it supervises. APRA's 30 April 2026 letter to all APRA-regulated entities is more explicit, stating that point-in-time, sample-based assurance is unsuited to models that learn, adapt, and degrade over time, and expecting continuous, risk-proportionate monitoring across the AI lifecycle.
Can model drift cause fairness or compliance problems, not just lower accuracy?
Yes. As a model drifts it can start producing systematically different outcomes for different groups of customers even though nothing about its code or documented rules changed, which is why drift monitoring increasingly sits alongside bias and fairness testing in AI governance programs rather than being treated as a pure engineering metric.
Last reviewed July 2026
This page is general information about What Is Model Drift?, not legal, regulatory, or professional advice, and does not capture every nuance or exception. Requirements change and can be fact-specific. Always verify against primary sources and your own qualified legal counsel before relying on it.