For informational purposes only. Definitions are written as general educational summaries. Legal and regulatory terms carry precise meanings in their source instruments, always refer to the primary regulation or standard for authoritative definitions. Regulatory requirements change; definitions are reviewed regularly but may not reflect the latest developments.
Want to explore a term in more depth or see how it applies to your specific situation?
Accountability (AI)
The principle that a specific person, role, or organisation must be answerable for an AI system's decisions and impacts. Effective AI governance requires named accountability, not shared responsibility across teams, so that when an AI system fails, the responsible party is clearly identified and able to act.
Algorithmic Discrimination
When an AI system produces outcomes that unfairly disadvantage individuals or groups based on protected characteristics, race, gender, age, disability, national origin, even if those characteristics are not explicitly included in the model's inputs. Algorithmic discrimination can arise from biased training data, proxy variables, or feedback loops.
Algorithmic Impact Assessment
A structured evaluation of the potential harms, benefits, and fairness implications of an AI system before and during deployment. Similar to an environmental impact assessment, it documents who is affected by AI decisions, in what ways, and what safeguards are in place.
Auditability
The capacity of an AI system to be examined, reconstructed, and verified after the fact. An auditable AI system has sufficient logging, documentation, and traceability that an independent reviewer can determine what inputs led to what outputs and why. Auditability is a core requirement under the EU AI Act for high-risk systems.
Automated Decision-Making
The use of an AI or algorithmic system to make or significantly influence decisions affecting individuals, employment screening, credit scoring, insurance pricing, parole recommendations, without meaningful human review of each decision. Multiple regulatory frameworks impose specific obligations on organisations using automated decision-making in high-stakes contexts.
Bias (AI)
Systematic and unfair skewing in AI outputs resulting from flawed training data, model design, or deployment context. AI bias can be statistical (inaccuracy for certain groups), representation bias (underrepresentation of certain groups in training data), or measurement bias (using flawed proxies for real-world attributes). Bias testing is increasingly required by regulation.
Black Box AI
An AI system whose internal decision-making process is not interpretable or explainable, inputs go in, outputs come out, but the reasoning is opaque. Black box AI systems create governance challenges because decisions affecting people cannot be explained, challenged, or audited. Distinguished from 'white box' or interpretable AI.
Conformity Assessment
The process of verifying that a high-risk AI system meets the requirements of the EU AI Act before it is placed on the EU market. Depending on the risk category, conformity assessment may be conducted by the provider (self-assessment) or by a notified third-party body. Successful completion is documented in a declaration of conformity.
Data Governance
The set of policies, processes, and structures that govern how data is collected, stored, processed, shared, and used. In the context of AI, data governance covers training data quality, provenance documentation, privacy compliance, and access controls, all of which directly affect the reliability and legal compliance of AI systems.
Data Provenance
The documented history of where training data came from, how it was collected, what processing it underwent, and what rights or restrictions apply to its use. Data provenance is a critical component of AI governance because undisclosed or improperly acquired training data creates IP, privacy, and regulatory liability.
Deepfake
AI-generated synthetic media, video, audio, or images, that realistically depicts people doing or saying things they did not actually do or say. Deepfakes present significant governance challenges around misinformation, reputational harm, and fraud. The EU AI Act explicitly addresses transparency requirements for synthetic media.
Deployer (EU AI Act)
Under the EU AI Act, a deployer is any natural or legal person who uses an AI system under their own authority in a professional context. Deployers of high-risk AI systems have specific obligations including conducting fundamental rights impact assessments, implementing human oversight, and monitoring AI performance, distinct from the obligations of providers who develop and sell AI systems.
EU AI Act
Regulation (EU) 2024/1689 - the world's first comprehensive AI regulation, in force from August 2024. The EU AI Act takes a risk-based approach: prohibited AI practices are banned outright; high-risk AI systems face mandatory requirements for risk management, technical documentation, human oversight, and conformity assessment; limited and minimal risk systems face lighter obligations. Maximum penalties reach €35 million or 7% of global annual turnover.
Explainability
The capacity to describe, in terms meaningful to a human, why an AI system produced a particular output or decision. Explainability is distinct from interpretability (understanding the model's internal mechanics) and is the more governance-relevant concept, the ability to tell a person affected by an AI decision why that decision was made.
Fairness (AI)
The absence of unjustified disparate treatment or impact across different groups in AI system outputs. AI fairness is not a single concept but a family of mathematical definitions, equal accuracy, equal false positive rates, equalised odds, that are often mutually incompatible. Governance frameworks require organisations to define which fairness criteria apply to their context and demonstrate compliance.
Foundation Model
A large AI model trained on broad data at scale, designed to be adapted for a wide range of downstream tasks. GPT-4, Claude, Llama, and Gemini are foundation models. The EU AI Act imposes specific obligations on providers of 'general-purpose AI systems' that function as foundation models, particularly those with systemic risk designation.
General-Purpose AI System (GPAI)
Under the EU AI Act, a GPAI system is an AI model trained on large amounts of data that exhibits significant generality and is capable of performing a wide range of distinct tasks. Large language models and multimodal models are typically GPAI systems. Providers of GPAI systems must maintain technical documentation, publish training data summaries, and comply with EU copyright law.
Governance Framework (AI)
A structured set of policies, processes, roles, and controls that governs how an organisation develops, deploys, and monitors AI systems. An AI governance framework typically covers risk classification, accountability structures, human oversight requirements, monitoring cadences, and incident response, providing the operational infrastructure that turns AI ethics principles into practice.
Hallucination
The tendency of large language models to generate plausible-sounding but factually incorrect outputs, citing non-existent sources, attributing statements to people who never made them, or inventing statistics. Hallucination is a structural property of current language models, not a bug to be eliminated. Governance controls typically require human verification of AI-generated content before professional use.
High-Risk AI System
Under the EU AI Act, AI systems that pose significant risks to health, safety, or fundamental rights and are subject to mandatory compliance obligations. High-risk categories include AI used in: employment and recruitment, credit scoring, educational assessment, law enforcement, migration and asylum, critical infrastructure management, and administration of justice. High-risk systems require risk management systems, technical documentation, data governance, human oversight, and accuracy and robustness testing.
Human-in-the-Loop
A design approach where a human reviews, approves, or can override AI decisions before they take effect. Human-in-the-loop requirements are mandated for high-risk AI systems under the EU AI Act and are considered best practice for any AI system making consequential decisions about individuals. Distinguished from 'human-on-the-loop' (monitoring without per-decision review) and 'human-out-of-the-loop' (fully automated).
Human Oversight
The organisational and technical mechanisms that enable humans to monitor, understand, correct, and if necessary suspend or stop an AI system. The EU AI Act requires deployers of high-risk AI systems to implement effective human oversight, not merely nominal oversight, ensuring that the humans responsible have the information, authority, and tools to intervene.
Impact Assessment (AI)
A structured evaluation, conducted before deployment, of the potential effects of an AI system on individuals, groups, and society. The EU AI Act requires fundamental rights impact assessments for certain high-risk AI deployers, particularly public bodies. Privacy impact assessments are also required where AI processes personal data.
Incident Response (AI)
The documented process for identifying, escalating, investigating, and remediating AI system failures, unexpected outputs, or harms to individuals. Effective AI incident response defines what constitutes an incident, who is notified, how the system is isolated or corrected, and how affected parties are informed. The EU AI Act requires providers to report serious incidents to market surveillance authorities.
Interpretability
The degree to which the internal mechanics of an AI model can be understood by humans. A logistic regression model is highly interpretable, each input's contribution to the output is mathematically transparent. A deep neural network is typically low-interpretability. High interpretability facilitates auditing, debugging, and regulatory review, but may come at the cost of model performance.
ISO 31000
The international standard for risk management principles and guidelines (ISO 31000:2018). Provides a universal framework for identifying, assessing, treating, and monitoring risk. Widely adopted across regulated industries as the foundation for enterprise risk management programs. In AI governance, ISO 31000 provides the risk management methodology that underpins AI-specific frameworks including the AIRA Framework.
ISO 42001
ISO/IEC 42001:2023 - the world's first international standard for AI management systems, published in December 2023. Specifies requirements for establishing, implementing, maintaining, and continually improving an Artificial Intelligence Management System (AIMS) using the Plan-Do-Check-Act methodology. The only certifiable AI management system standard. Organisations certified under ISO 42001 can demonstrate structured AI governance to investors, regulators, and enterprise customers.
Key Risk Indicator (KRI)
A measurable metric that provides early warning of increasing AI risk before an incident occurs. Examples include model accuracy degradation over time, distribution shift in input data, increasing rates of human overrides, and complaint volumes. Effective AI governance programs define KRIs for each AI system and establish thresholds that trigger escalation or review.
Large Language Model (LLM)
A type of foundation model trained on large quantities of text data to generate, summarise, classify, and reason about language. ChatGPT, Claude, Gemini, Llama, and Mistral are large language models. LLMs exhibit hallucination, can encode training data biases, and produce outputs that vary across runs, governance frameworks must account for these properties when deploying LLMs in consequential contexts.
Model Card
A structured document accompanying an AI model that describes its intended use, performance characteristics, evaluation results, and known limitations. Model cards, introduced by Google researchers in 2018, are now considered best practice for AI transparency and are required or expected under multiple governance frameworks including the EU AI Act's technical documentation requirements.
Model Drift
The degradation of an AI model's performance over time as the real-world data it encounters diverges from the data on which it was trained. A credit scoring model trained on pre-2020 data may underperform following economic shifts; a hiring model trained on historical applicant pools may produce biased outputs as applicant demographics change. Model drift monitoring is a core requirement of ongoing AI governance.
Model Risk
The potential for adverse outcomes, financial loss, regulatory penalty, reputational damage, or harm to individuals, arising from decisions based on AI model outputs. Model risk management, particularly in financial services (where it is regulated through frameworks like SR 26-2 in the US — which superseded SR 11-7 in April 2026), encompasses model development standards, independent validation, ongoing monitoring, and governance of model changes.
NIST AI RMF
The National Institute of Standards and Technology AI Risk Management Framework (2023), a voluntary US framework for managing AI risks across four core functions: Govern (establishing AI risk governance structures), Map (identifying and categorising AI risks), Measure (analysing and assessing risk), and Manage (prioritising and treating risk). Widely adopted in US federal agencies and regulated industries.
Post-Market Monitoring
The ongoing surveillance of an AI system's performance, outputs, and impacts after deployment. The EU AI Act requires providers of high-risk AI systems to implement post-market monitoring plans that detect issues that may emerge over time. Post-market monitoring is distinct from pre-deployment testing, it covers the full production lifecycle.
Privacy by Design
The principle that privacy protections should be embedded into AI systems from the earliest design stage, rather than added as an afterthought. Applied to AI governance, privacy by design requires data minimisation in training sets, purpose limitation in data use, access controls from the outset, and impact assessment before personal data is processed for AI training or inference.
Prohibited AI Practices
Under the EU AI Act, AI applications that are banned outright because their risks are considered unacceptable. These include: subliminal manipulation of behaviour without awareness; exploitation of vulnerabilities; social scoring by governments; real-time remote biometric identification in public spaces (with narrow exceptions); emotion recognition in workplace and education; and AI systems that create untargeted facial recognition databases.
Provider (EU AI Act)
Under the EU AI Act, a provider is any entity that develops or has an AI system developed and places it on the market or puts it into service under its own name or trademark. Providers of high-risk AI systems bear the heaviest obligations under the Act, including conformity assessment, technical documentation, EU database registration, and ongoing market surveillance.
Red-Teaming (AI)
A structured adversarial testing process where a team deliberately attempts to elicit harmful, biased, or unexpected outputs from an AI system before deployment. AI red-teaming tests for failure modes that standard benchmarks miss, prompt injection, jailbreaking, demographic bias in edge cases, and behaviour under adversarial inputs. Increasingly required by AI governance frameworks for high-risk applications.
Regulatory Sandbox (AI)
A controlled environment in which organisations can test innovative AI systems under regulatory supervision, with reduced compliance obligations, before market deployment. The EU AI Act mandates member states to establish national AI regulatory sandboxes. Sandboxes provide a pathway for responsible innovation while maintaining regulatory oversight.
Risk Appetite (AI)
The level and type of AI risk an organisation is willing to accept in pursuit of its objectives. AI risk appetite statements define the boundaries within which AI systems may operate without escalation, specifying, for example, that AI systems may not autonomously make decisions affecting employment without human review, or that AI systems may not process sensitive personal data without explicit approval. Risk appetite must be approved at board level.
Risk Classification (AI)
The process of categorising AI systems by the level and type of risk they present, to calibrate governance and control requirements. The EU AI Act uses four categories: prohibited, high-risk, limited risk, and minimal risk. Organisational frameworks typically use three tiers (High / Elevated / Standard) based on factors including decision impact, data sensitivity, affected populations, and reversibility of outcomes.
Sensitive Data (AI)
Categories of personal data whose processing in AI systems requires particular safeguards due to the significant privacy risks their disclosure creates. Under GDPR and the EU AI Act, sensitive data includes racial or ethnic origin, political opinions, religious beliefs, trade union membership, genetic data, biometric data, health data, and data about sex life or sexual orientation. Training AI systems on sensitive data without adequate legal basis and safeguards is a common compliance failure.
Shadow AI
The use of AI tools by employees without the knowledge, approval, or oversight of their organisation's IT, legal, or governance functions. Shadow AI is endemic in workplaces, individuals using personal ChatGPT subscriptions for work tasks, uploading confidential documents to consumer AI tools, or using unsanctioned automation tools. It represents one of the primary unmanaged AI risk vectors for most organisations.
Systemic Risk (GPAI)
Under the EU AI Act, certain general-purpose AI models with particularly large training compute (above 10^25 FLOPS as a threshold indicator) are designated as posing systemic risk. Providers of GPAI models with systemic risk face additional obligations including adversarial testing, incident reporting to the European AI Office, and cybersecurity measures. This designation is currently applied to the largest frontier AI models.
Technical Documentation
The body of written evidence required under the EU AI Act for high-risk AI systems, demonstrating regulatory compliance before market placement. Technical documentation covers: system description and intended purpose; training data and data governance; performance metrics; risk management processes; human oversight measures; and cybersecurity specifications. Must be maintained and updated throughout the system's lifecycle.
Training Data
The data used to train an AI model, the information from which the model learns patterns, relationships, and behaviours. Training data quality directly determines model quality: biased, incomplete, or inaccurately labelled training data produces flawed AI outputs. Training data provenance, documenting where data came from and whether its use is legally authorised, is a core AI governance requirement.
Transparency (AI)
The principle that AI systems and their use should be disclosed to the people they affect. Transparency obligations under the EU AI Act require: disclosure when consumers interact with AI systems; labelling of AI-generated content; and publication of training data summaries for general-purpose AI systems. Transparency is a foundational principle of trustworthy AI across all major governance frameworks.
Trustworthy AI
A characterisation of AI systems that are lawful (complying with applicable laws), ethical (aligned with ethical principles and values), and robust (technically and socially reliable). The EU's 2019 Guidelines for Trustworthy AI identified seven key requirements: human agency and oversight, technical robustness and safety, privacy and data governance, transparency, diversity and fairness, societal wellbeing, and accountability.
Use Case Assessment
A structured evaluation of a proposed AI application before development or procurement, examining intended purpose, affected populations, risk classification, legal basis for data use, and governance requirements. Use case assessment is the earliest intervention point in the AI lifecycle governance process, preventing problematic AI applications from being built rather than detecting issues after deployment.
Validation (AI)
The independent process of confirming that an AI model performs as intended across a representative range of conditions, including edge cases and adversarial inputs. Model validation in regulated industries (particularly financial services) is a formal function typically conducted by a party independent of the development team. Validation produces a validation report that forms part of the model's technical documentation.
This glossary is maintained by the AIRiskAware research team and updated regularly as the regulatory landscape evolves.