What is model drift and why does it matter for AI governance?
Model drift is the degradation of an AI model's accuracy and reliability over time as the real-world conditions it was trained on change. There are three types: data drift (input distributions shift), concept drift (the relationship between inputs and outputs changes), and label drift (the meaning of target variables evolves). Model drift is one of the most common causes of AI governance failure in production, and most organisations have no monitoring for it.
Model drift is not a failure of the model, it is an expected, inevitable property of any model deployed in a changing environment. The governance question is not whether drift will occur, but whether you will detect it before it causes harm.
Types of drift
Data drift (covariate shift). The statistical properties of the input data change. Customer demographics shift, product mix changes, market conditions evolve. The model receives inputs it wasn't designed for.
Concept drift. The relationship between inputs and outputs changes. What constituted fraud in 2024 looks different in 2026. Customer satisfaction drivers evolve. The "right answer" the model should produce has changed even though the inputs look similar.
Label drift. The target variable's distribution changes. Default rates shift across economic cycles. Claim frequencies change. The outcome the model predicts becomes more or less common.
Why governance must address drift
Regulatory expectations now explicitly include ongoing monitoring. APRA (Australian Prudential Regulation Authority)'s 30 April 2026 letter specified "continuous validation" rather than point-in-time assurance. The Federal Reserve's SR 26-2 (17 April 2026) replaces annual revalidation with risk-based monitoring. The ECB's July 2025 Supervisory Guide on Internal Models extends MRM expectations, including drift monitoring, to all models including AI/ML. ISO/IEC 42001 expects ongoing performance management. The EU AI Act requires post-market monitoring for high-risk AI systems.
Undetected drift creates escalating risk: degrading accuracy leads to worse decisions, which leads to customer harm, regulatory exposure, and financial loss, all while the model continues to operate and the organisation believes it's performing as designed.
Detecting drift
Statistical monitoring. Track distributional characteristics of inputs and outputs over time. Population Stability Index (PSI), Kolmogorov-Smirnov tests, and Jensen-Shannon divergence are common statistical measures. Set thresholds that trigger review when exceeded.
Performance monitoring. Track the model's prediction accuracy against actual outcomes. Accuracy, precision, recall, F1 score, AUC-ROC for classification; MAE, RMSE for regression. Compare current performance against the baseline established at validation.
Demographic monitoring. Track performance across demographic groups separately. Drift may affect some groups more than others, a credit model that remains accurate overall but degrades for a specific demographic group creates bias and regulatory exposure.
Responding to drift
When drift is detected: assess whether it materially affects decisions; retrain the model on current data if appropriate; revalidate against updated benchmarks; document the drift event, assessment, and remediation in the AI risk register; report to the AI governance committee and board if the drift affected a high-risk system or produced adverse outcomes.
Primary sources: NIST AI RMF · Federal Reserve SR 26-2
