What Is Fairness Metric?
Fairness Metric is a quantitative measure used to evaluate whether an AI system produces equitable outcomes across different demographic or protected groups.
Fairness Metric, a quantitative measure used to evaluate whether an AI system produces equitable outcomes across different demographic or protected groups.
Common fairness metrics include: demographic parity (equal positive prediction rate across groups), equalised odds (equal true positive and false positive rates), and individual fairness (similar individuals receive similar outputs). A fundamental challenge — proven mathematically by the Impossibility Theorem (Chouldechova, 2017; Kleinberg et al., 2016) — is that these metrics cannot all be satisfied simultaneously except in trivial cases. This means fairness is a values question, not a pure technical problem: organisations must explicitly choose which fairness definition to optimise for, and document the choice.
Source: Chouldechova (2017); Kleinberg et al. (2016); NYC LL144 audit requirements
Plain-language explanation
Common fairness metrics include: demographic parity (equal positive prediction rate across groups), equalised odds (equal true positive and false positive rates), and individual fairness (similar individuals receive similar outputs). A fundamental challenge — proven mathematically by the Impossibility Theorem (Chouldechova, 2017; Kleinberg et al., 2016) — is that these metrics cannot all be satisfied simultaneously except in trivial cases. This means fairness is a values question, not a pure technical problem: organisations must explicitly choose which fairness definition to optimise for, and document the choice.
See where you stand on AI governance
Take the free 7-question maturity assessment and get a personalised action plan.
Free assessment, 3 minutes →