What Is...
Governance Concept
What Is AI Explainability?
AI explainability is the ability to explain, in human-understandable terms, why an AI system produced a particular output. It is distinct from interpretability (understanding the model's internal workings) and is a legal compliance requirement in multiple jurisdictions.
Legal requirements by jurisdiction
GDPR Article 22
EU / UK
Meaningful information about the logic involved in automated decisions with legal or significant effects. Must be specific to the individual decision.
EU AI Act Article 13
EU
High-risk AI systems must be transparent, enabling deployers to understand how the AI works, interpret its outputs, and implement appropriate oversight.
ECOA / Reg B
US
Specific principal reasons for adverse credit decisions, algorithmic scores without reasons do not satisfy the obligation.
Privacy Act / APPs
Australia
Right to access personal data including AI-generated assessments; right to seek correction of inaccurate information.
Common explainability techniques
SHAP values
Assigns each feature a contribution score for a specific prediction. Most widely used in regulated industry contexts.
LIME
Local Interpretable Model-agnostic Explanations, fits a simple interpretable model around a specific prediction.
Counterfactual explanations
"What would need to change for the outcome to be different?", most useful for actionable explanations to individuals.
Attention visualisation
Shows which parts of the input the model weighted most heavily, common for text and image models.
Key distinction: Explainability techniques explain specific decisions to specific people. Interpretability means understanding what a model computes internally. Both matter for governance, they are not interchangeable.