本文目前仅提供英文版本。
AI Model Risk Controls: Validation, Monitoring, and What Regulators Actually Expect
Model risk management frameworks — originally designed for quantitative financial models — are being extended to cover AI. Here is the AI model risk control framework that financial services regulators and internal audit functions expect to see.
Key Takeaways
Model risk management (MRM) frameworks — established by US Federal Reserve SR 11-7 (superseded April 2026 by SR 26-2), APRA CPG 220, and analogous guidance globally — apply to AI models used in regulated financial activities. These are not aspirational standards; they are regulatory expectations.
The three core MRM controls are model development (rigorous development process with documented methodology, assumptions, limitations, and testing), model validation (independent review of the model before deployment and after significant changes), and model use (ongoing monitoring, performance review, and limitation awareness by model users).
AI-specific MRM challenges beyond traditional quantitative models: explainability (AI models may produce accurate outputs through mechanisms that cannot be explained); concept drift (AI model performance degrades as the real-world relationships it learned change); adversarial vulnerability (AI models can be manipulated through carefully crafted inputs); and data dependency (AI model quality depends entirely on training data quality, which may not be apparent until the model fails).
Validation scope for AI models should extend beyond accuracy testing to include: demographic disparity testing (does the model produce biased outcomes across protected groups?); adversarial robustness testing (how does the model respond to inputs designed to manipulate it?); out-of-distribution testing (how does the model perform on inputs that differ from its training data?); and interpretability assessment (can the model's outputs be explained to the required degree for the use case?).
Ongoing monitoring for AI models must go beyond traditional model monitoring. In addition to accuracy and stability metrics, AI model monitoring should include: demographic performance monitoring (has accuracy changed differently for different groups?); input distribution monitoring (are the inputs the model receives changing in ways that may indicate concept drift?); and output distribution monitoring (are the model's outputs changing in ways that may indicate something unexpected is happening?).
Third-party AI models — AI components embedded in vendor products — require MRM treatment equivalent to internally developed models where they are used in regulated decisions. 'We bought it from a vendor' is not an exemption from model risk management requirements.
"仅供参考。本文不构成法律、监管、财务或专业建议。如需具体指导，请咨询合格专家。"
Why MRM frameworks apply to AI
Model risk management frameworks were developed primarily for quantitative financial models — credit scoring models, market risk models, pricing models, and actuarial models. AI models used in equivalent functions — credit underwriting, fraud detection, market risk assessment, insurance pricing — are subject to the same regulatory expectations. Regulators have been explicit about this: the US Federal Reserve's SR 11-7 guidance on model risk management states that it "applies to quantitative models as well as AI and machine learning models." APRA's CPG 220 model risk guidance similarly extends to AI models.
This matters because MRM frameworks are not aspirational guidance — they represent regulatory expectations that are tested in supervisory examinations and that form the basis for regulatory enforcement where deficiencies cause harm. Organisations that treat AI models as outside their MRM framework are taking regulatory risk.
The three core MRM controls for AI
Model development: a rigorous AI model development process requires documented methodology (what modelling approach was chosen and why), data documentation (what training data was used, where it came from, how it was cleaned and prepared, and what its limitations are), assumption documentation (what assumptions does the model make about the world, and how sensitive are its outputs to those assumptions), testing documentation (what tests were conducted during development, what the results were, and how limitations were addressed), and limitations documentation (what situations is the model not designed for, and what are the consequences of using it outside its intended scope).
Model validation: independent validation — conducted by someone who was not involved in developing the model — is the second core control. Validation should assess: whether the model's methodology is appropriate for its intended use; whether the training data is appropriate and the data preparation process is sound; whether the model performs as documented in testing; whether the model's limitations are accurately documented; and whether the model produces unbiased outcomes. Validation should be conducted before deployment and repeated when the model is significantly changed or when performance monitoring indicates possible degradation.
Model use: the third core MRM control addresses how models are used in practice. Model users — whether human decision-makers using model outputs or automated systems — should understand the model's purpose, its limitations, and what it cannot tell them. Human overrides of model recommendations should be documented and monitored. Model outputs should not be mechanically applied without the judgement that the model's limitations require.
AI-specific validation extensions
Traditional model validation methodology covers accuracy, stability, and conceptual soundness. AI models require additional validation dimensions.
Demographic disparity testing: does the model produce materially different outcomes for demographic groups that should be treated equivalently? This goes beyond testing whether protected characteristics are explicit model inputs — it requires testing whether the model's outputs are demographically neutral after controlling for legitimate risk factors.
Adversarial robustness: AI models can be manipulated through carefully crafted inputs — a phenomenon called adversarial attack. For AI used in fraud detection, credit assessment, or other consequential decisions, validation should include testing whether the model produces stable outputs when inputs are perturbed in ways consistent with potential manipulation.
Out-of-distribution testing: AI models are trained on historical data and may perform poorly when they encounter inputs that differ significantly from their training distribution. Validation should include testing the model's performance on scenarios that differ from training data — including stress scenarios and edge cases.
Interpretability assessment: for AI used in regulated decisions, the degree to which model outputs can be explained to required stakeholders (regulators, customers, internal reviewers) should be assessed. Where interpretability is insufficient for the use case, either the model should be modified to improve interpretability or the use case should be reconsidered.