The Complete AI Governance Glossary: 80 Terms Every Professional Needs to Know

A

Accountability (AI) — The principle that organisations and individuals must be answerable for AI system decisions and outcomes. Distinguished from responsibility (who has the obligation) and liability (who faces legal consequences). Accountability requires both that there is a named person answerable for each AI system's outcomes and that they have sufficient information and authority to genuinely discharge that accountability. Nominal accountability — where a person is named but lacks meaningful oversight — does not satisfy regulatory accountability requirements.

Adverse action notice — A legally required notification given to an individual when an automated decision goes against them, particularly in credit, employment, and insurance contexts. In the US, the Equal Credit Opportunity Act and Fair Credit Reporting Act require adverse action notices for credit decisions. The notice must provide specific reasons for the adverse decision — "the algorithm decided" does not satisfy the requirement. See also: explainability, automated decision-making.

AI Governance — The policies, structures, processes, and controls that enable organisations to develop and use AI responsibly. Distinct from AI ethics (the principles) and AI compliance (the legal obligations), though related to both. Effective AI governance is operational — it shapes real decisions about AI deployment, not just documentation produced for auditors.

AI Lifecycle — The stages through which an AI system progresses: design and scoping, data collection and preparation, model development and training, validation and testing, deployment, monitoring, and decommissioning. Governance obligations apply at each stage and differ between stages. Pre-deployment governance (risk assessment, bias testing, documentation) is different from post-deployment governance (monitoring, incident response, performance review).

AI literacy — The knowledge and skills needed to understand, evaluate, and use AI effectively. Distinct from AI technical expertise (the ability to build AI systems). AI governance requires a specific form of AI literacy: understanding how AI systems work at a level sufficient to assess their risks, evaluate their governance, and engage meaningfully with technical experts about their design and operation.

AI system — The EU AI Act defines an AI system as "a machine-based system designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments." This definition is intentionally broad — it captures most software that uses machine learning, not just systems explicitly called "AI."

Alignment — The property of an AI system pursuing the objectives that its developers and users actually intend, rather than proxy metrics or unintended objectives. Alignment failures can occur when: the objective function specified does not capture the true objective (reward hacking), the AI system behaves differently in training versus deployment (distributional shift), or the AI system's behaviour changes as capabilities increase (emergent misalignment). Alignment is primarily a technical AI safety research concept but has direct enterprise governance implications.

Algorithmic bias — Systematic and unfair discrimination produced by an AI algorithm. Can arise from: biased training data that reflects historical discrimination, feature selection that proxies for protected characteristics, objective functions that optimise for metrics that disadvantage protected groups, or feedback loops that amplify initial disparities. Algorithmic bias can be direct (the algorithm explicitly uses a protected characteristic) or indirect (the algorithm uses proxies correlated with protected characteristics). Legal liability does not require intent — disparate impact is sufficient.

Algorithmic transparency — The principle that AI systems and their decision-making processes should be understandable and open to scrutiny. Distinct from explainability (the ability to explain a specific decision) and interpretability (the ability to understand a model's internal workings). Full algorithmic transparency — publishing model weights and training data — is rarely achievable due to IP protection and privacy obligations; the governance question is what level of transparency is appropriate for each context.

Agentic AI — AI systems configured to take sequences of autonomous actions in pursuit of goals, rather than responding to individual prompts and producing single outputs. Agentic AI can browse the web, write and execute code, send communications, and interact with external systems with limited human oversight of individual actions. Agentic deployment creates specific governance challenges around human oversight, action scope limitation, and accountability for agent-initiated actions.

Audit trail (AI) — A record of AI system inputs, processing, and outputs sufficient to reconstruct the basis for any significant decision and to support post-hoc review, regulatory examination, and incident investigation. EU AI Act Article 12 requires logging of high-risk AI system operation to the extent technically feasible. Audit trails must be maintained for a specified period (at least six months under EU AI Act) and must be accessible to relevant authorities on request.

B

Bias testing — The systematic assessment of an AI system for unfair outcomes across different groups of people. Includes testing for disparate impact (different outcomes for different groups without adequate justification) and disparate treatment (different processing of inputs from different groups). Bias testing requires: demographic data about the population affected, defined fairness metrics, statistical methodology to assess significance of disparities, and documentation of both the methodology and the results. Bias testing before deployment and ongoing monitoring after deployment are both required for high-risk AI.

Black box AI — Colloquial term for AI systems whose internal operations cannot be directly observed or understood, even by the people who built them. Complex neural networks are often described as black boxes because their decision-making process cannot be explained in human-understandable terms. The governance implication: black box AI used in high-stakes decisions creates specific challenges for explainability obligations, bias auditing, and human oversight. See also: interpretability, explainability.

C

Conformity assessment — The process by which high-risk AI systems under the EU AI Act are assessed for compliance with the Act's requirements before being placed on the market or put into service. For most Annex III high-risk AI systems, conformity assessment can be conducted by the provider themselves (self-assessment). For biometric identification systems and some critical infrastructure AI, third-party notified body assessment is required. Self-assessment does not mean lightweight — it requires comprehensive documentation against all applicable requirements.

Content moderation AI — AI systems used to identify and remove or restrict access to content that violates platform policies or legal requirements. Content moderation AI creates governance challenges around: accuracy (both false positives and false negatives have significant consequences), bias (moderation systems may treat different languages, dialects, or cultural contexts differently), and accountability (automated removal of content implicates freedom of expression obligations).

Credit scoring AI — AI systems used to assess the creditworthiness of loan applicants or existing credit customers. Credit scoring AI is high-risk AI under EU AI Act Annex III and is subject to specific regulatory requirements in most jurisdictions: adverse action notice requirements in the US, Article 22 GDPR rights in the EU/UK, and responsible lending obligations in Australia. Credit scoring AI has a documented history of discriminatory outcomes through proxy variables.

D–G

Data governance — The policies, processes, and standards for managing data quality, availability, and appropriate use. For AI governance, data governance addresses: training data quality and provenance, data subject rights in relation to AI training data, data minimisation in AI systems, and ongoing data quality monitoring for deployed AI. Distinct from AI governance but foundational to it — AI systems are only as good as the data they are trained on and operated with.

Deployer (EU AI Act) — An organisation or individual that uses a high-risk AI system under its own authority. Distinguished from the provider (who develops or places on the market). Deployers have specific obligations under the EU AI Act including: using AI within its intended purpose, implementing human oversight, ensuring appropriate monitoring, reporting serious incidents, and maintaining logs. Deployer obligations cannot be transferred to the provider through contract.

Disparate impact — When a neutral policy or practice has a disproportionately adverse effect on a protected group, even without discriminatory intent. Disparate impact is actionable under anti-discrimination law in Australia (indirect discrimination), the EU (indirect discrimination), and the US (disparate impact doctrine under employment and fair lending law). AI systems can produce disparate impact through proxy variables, biased training data, or optimisation for metrics that correlate with protected characteristics.

Explainability — The ability to explain the basis for a specific AI decision in human-understandable terms. Distinguished from interpretability (understanding a model's internal workings) and transparency (openness about the AI system's existence and operation). Explainability is a legal requirement in many jurisdictions: EU AI Act Article 13 requires transparency of high-risk AI; GDPR Article 22 requires meaningful information about the logic of automated decisions; US financial regulation requires specific adverse action reasons. The challenge: some AI systems that produce accurate outputs cannot provide reliable explanations of those outputs.

Foundation model — A large AI model trained on broad data at scale, capable of being adapted to a wide range of downstream tasks. GPT-4, Claude 3, and Gemini Ultra are examples. Foundation models are governed under the EU AI Act's GPAI (General Purpose AI) provisions, with additional obligations for models with systemic risk (trained above 10^25 FLOPs). Enterprise organisations that deploy foundation models through APIs are deployers, not providers, but have deployer obligations under the EU AI Act.

Frontier AI — The most capable AI systems at any given time, operating at or near the boundary of what is technically possible. Frontier AI creates governance challenges that existing frameworks were not designed for: emergent capabilities, agentic deployment potential, and scale of potential impact. The UK AI Safety Institute and the US AI Safety and Security Board specifically focus on frontier AI governance.

GPAI (General Purpose AI) — AI models that can be used for a range of purposes, as distinct from AI developed for a specific use case. Under the EU AI Act, GPAI models have specific transparency and safety obligations. Models with "systemic risk" — trained with more than 10^25 FLOPs — have additional obligations including adversarial testing, incident reporting, and cybersecurity measures.

H–M

Hallucination — When an AI language model generates confident-sounding but factually incorrect or fabricated content. Hallucination is not a bug in the traditional sense — it is a characteristic of how large language models generate text, producing statistically plausible rather than necessarily accurate content. Enterprise governance implications: AI outputs used in consequential contexts (legal documents, medical advice, financial analysis, regulatory submissions) must be verified by humans with appropriate expertise before reliance.

High-risk AI — Under the EU AI Act, AI systems that pose significant risks to health, safety, or fundamental rights, listed in Annex III. Includes AI in biometric identification, critical infrastructure, education, employment, essential private and public services, law enforcement, border control, and administration of justice. High-risk AI providers and deployers have the most demanding compliance obligations under the EU AI Act.

Human oversight — Mechanisms enabling humans to monitor, understand, and when necessary intervene in AI system operations. EU AI Act Article 14 requires deployers of high-risk AI to implement human oversight measures. Human oversight must be genuine — nominal review without real capacity to assess or intervene does not satisfy the requirement. The specific implementation of human oversight depends on the AI system's decision speed, volume, and consequence profile.

Interpretability — The property of an AI model that allows its internal workings to be understood directly. Distinct from explainability (explaining specific outputs) and transparency (openness about the system's existence). Logistic regression models are fully interpretable — the relationship between inputs and outputs can be precisely described. Deep neural networks are generally not interpretable at the level of individual computations, though interpretability research is producing techniques that provide partial insight.

Model card — Structured documentation of an AI model's intended uses, training data, performance characteristics, limitations, and ethical considerations. Model cards are a best practice in responsible AI development and are becoming a regulatory requirement. EU AI Act technical documentation requirements align closely with model card content. Model cards support the explainability obligations that apply to high-risk AI and provide the baseline information for bias auditing.

Model drift — The degradation of an AI model's performance over time as the statistical distribution of real-world data diverges from the training data distribution. Model drift is a governance concern because it can cause an AI system that passed pre-deployment testing to perform poorly in production — including producing biased or inaccurate outputs. Monitoring for model drift is a requirement under EU AI Act human oversight provisions and a component of responsible post-deployment AI governance.

Model risk management — The framework for identifying, measuring, monitoring, and controlling the risks posed by AI and statistical models used in decision-making. Originated in financial services regulation (US Federal Reserve SR 11-7 guidance) and extended to AI/ML models by financial regulators globally. Model risk management requires: model inventory, independent validation, performance monitoring, model owner accountability, and governance of model changes.

N–R

Notifiable Data Breach (AI) — A data breach triggered by an AI system failure — for example, an AI that returns other users' personal data, a model inversion attack that recovers training data, or an AI-assisted cyberattack that results in data exfiltration. Notifiable data breach obligations under the Privacy Act (Australia) and GDPR (EU/UK) apply to AI-related breaches on the same terms as other data breaches. The 30-day notification clock under the Australian scheme runs from when the organisation becomes aware of the breach.

Post-quantum cryptography — Cryptographic algorithms designed to be secure against attacks by both classical and quantum computers. NIST finalised post-quantum cryptography standards in August 2024 (FIPS 203, 204, 205). Enterprise organisations with data that must remain confidential beyond 2030-2035 should begin migrating to post-quantum cryptography now, due to the "harvest now, decrypt later" threat.

Proxy variable — A variable used in an AI model that is correlated with a protected characteristic (race, gender, age, disability) without explicitly referencing it. Postcode can be a proxy for race in housing markets. Purchase history can be a proxy for gender in retail. Credit history can be a proxy for age. Using proxy variables can produce discriminatory outcomes that are legally actionable as indirect discrimination even where the protected characteristic is not explicitly included in the model.

Provider (EU AI Act) — An organisation or individual that develops or places on the market an AI system or general purpose AI model. Distinguished from the deployer (who uses the AI). Providers of high-risk AI have the most extensive obligations: conformity assessment, technical documentation, registration in the EU AI Act database, post-market monitoring, and serious incident reporting. Providers of GPAI models have obligations around transparency, copyright compliance, and for systemic risk models, additional safety requirements.

S–Z

Safety case — A structured argument, supported by evidence, that an AI system is acceptably safe for its intended use in a specific operational context. Safety cases are required for AI in high-consequence industries including aviation, nuclear, medical devices, and autonomous vehicles. The safety case methodology — developed in aerospace and nuclear before AI — provides a rigorous framework for demonstrating safety that is increasingly being applied to AI in other high-stakes contexts.

Systemic risk (AI) — The risk that an AI system could have significant negative effects on a large number of people, disrupt critical systems, or cause widespread harm. Under the EU AI Act, GPAI models trained above 10^25 FLOPs are presumed to have systemic risk and are subject to additional obligations. Systemic risk is also relevant to AI in financial services (where AI failures can amplify market instability), critical infrastructure (where AI failures can affect essential services), and information ecosystems (where AI-generated content can influence public opinion at scale).

Technical documentation (AI) — The detailed documentation required for high-risk AI systems under the EU AI Act, demonstrating compliance with the Act's requirements. Must include: description of the system and its intended purpose, design specifications, training data documentation, performance metrics, bias testing results, cybersecurity measures, and human oversight mechanisms. Technical documentation must be created before deployment and maintained throughout the system's lifecycle.

Training data — The data used to train an AI model. Training data quality, provenance, and composition fundamentally determine the model's performance, biases, and limitations. Key governance obligations relating to training data: lawful basis for using personal data in training (GDPR/Privacy Act), copyright compliance for training on creative works, bias in training data creating discriminatory model outputs, and documentation of training data composition in technical documentation.