Risk Concept

What Is Model Risk?

Model risk is the risk of adverse consequences from decisions based on incorrect, misused, or misunderstood quantitative models. Developed as a formal risk category in financial services, model risk management is now being extended to AI and machine learning systems — creating new governance obligations for APRA-regulated entities and, increasingly, any organisation deploying consequential AI.

Origin: financial model risk

Model risk governance originated in financial services following a series of high-profile failures in which quantitative models — credit scoring, risk measurement, derivative pricing — produced incorrect outputs with material financial consequences. Regulators including the US Federal Reserve (SR 11-7, superseded by SR 26-2 in April 2026), the European Banking Authority, and APRA developed frameworks requiring financial institutions to validate, monitor, and control the models they use in consequential decisions.

Components of model risk

Model error

The model produces incorrect outputs due to flawed design, incorrect assumptions, errors in the mathematical framework, or bugs in implementation. The model does not do what it is supposed to do.

Model misuse

The model is applied in contexts for which it was not designed, or inputs are provided outside the model's valid range. A model validated for one population may not generalise to another.

Model drift

The relationship between inputs and outputs the model learned from historical data changes over time. A model that performed well when trained may degrade without recalibration as the underlying environment changes.

Data quality risk

The model produces incorrect outputs because the input data is incomplete, stale, miscoded, or not representative of the actual population to which it is being applied.

Implementation risk

The model logic is correctly specified but incorrectly implemented in production systems — through coding errors, incorrect data pipelines, or configuration errors that cause the deployed model to behave differently from the validated version.

APRA expectations in Australia

APRA-regulated entities — banks, insurers, superannuation funds — are expected to manage model risk within their enterprise risk management frameworks under CPS 220. APRA's supervisory guidance has increasingly flagged that model risk management frameworks must extend to AI and machine learning systems, including: large language models used in customer communications; machine learning models in credit, pricing, and underwriting decisions; AI tools in claims handling and fraud detection; and third-party AI services accessed through APIs as material operational dependencies under CPS 230 (in force from July 2025).

AI makes model risk harder

Traditional model risk management was developed for relatively interpretable quantitative models — regression equations, actuarial tables, option pricing formulae. AI and machine learning models are more complex: they may have billions of parameters, non-linear relationships that cannot be directly inspected, and emergent behaviour that was not designed into the system. Standard validation techniques must be adapted or supplemented. The field of explainable AI (XAI) exists partly to address this gap — making AI model outputs interpretable enough to be validated and challenged.

What is model drift? AI for GRC teams in Australia