Governance Concept

What Is AI Bias?

AI bias is not a single phenomenon, it encompasses multiple distinct types with different causes, different tests, and different legal implications. Disparate impact bias is actionable under anti-discrimination law without proof of discriminatory intent.

Definition

AI Bias, systematic and unfair differences in AI system outputs that disadvantage particular individuals or groups, often correlated with protected characteristics.

AI bias is rarely the result of intent. It typically arises from biased training data, flawed proxy variables, or feedback loops that amplify existing inequities. Most jurisdictions treat AI-driven indirect discrimination as unlawful regardless of intent: the EU AI Act, the Equality Act 2010, US Title VII, and Australian anti-discrimination law all extend to AI-mediated decisions. Bias auditing is increasingly mandated (e.g., NYC Local Law 144).

Source: NIST SP 1270; EU AI Act, Article 10

Four types of AI bias

Historical bias

Training data reflects historical discrimination, the model learns and perpetuates discriminatory patterns from the past.

Representation bias

Training data underrepresents certain groups, the model performs worse for those groups in production.

Proxy variable bias

The model uses variables correlated with protected characteristics to produce discriminatory outcomes without explicitly using those characteristics.

Feedback loop bias

The model's outputs feed back into training data, amplifying initial biases over time.

Complete bias testing guide

Legal implications

Disparate impact, where an apparently neutral AI produces outcomes that disproportionately disadvantage a protected group, is actionable under anti-discrimination law in Australia, the EU, UK, and US without proof of discriminatory intent.

Read the legal analysis