Fairness metrics

Definitions#

Let $A$ be the protected attribute, $Y \in {0,1}$ the true outcome, $R \in [0,1]$ the model score and $\hat{Y} = \mathbb{1}[R \ge t]$ the thresholded decision.

Independence (demographic parity). $\hat{Y} \perp A$, i.e. $P(\hat{Y} = 1 \mid A = a)$ is constant in $a$. Relaxations: the difference $\max_{a,a'} |P(\hat Y=1 \mid a) - P(\hat Y=1 \mid a')|$, or the ratio, which underlies the four-fifths rule of thumb used in US employment screening.

Separation (equalised odds). $\hat{Y} \perp A \mid Y$. Equivalently $P(\hat Y = 1 \mid Y = y, A = a)$ is constant in $a$ for both $y \in {0,1}$ — equal TPR and equal FPR. Equal opportunity (Hardt et al., 2016) is the relaxation to $y = 1$ only.

Sufficiency (predictive parity, calibration). $Y \perp A \mid R$. Equivalently $P(Y = 1 \mid R = r, A = a)$ is constant in $a$: a score of 0.7 means the same thing for everyone.

Barocas, Hardt and Narayanan, Fairness and Machine Learning, organise the entire literature under these three, and it is the cleanest available framing.

The impossibility results#

Chouldechova (2017) proves the constraint that the developer block demonstrates. Writing $p$ for the base rate $P(Y=1 \mid A=a)$, $\text{PPV}$ for precision, and $\text{FPR}$, $\text{FNR}$ for the two error rates:

$$ \text{FPR} = \frac{p}{1-p} \cdot \frac{1 - \text{PPV}}{\text{PPV}} \cdot (1 - \text{FNR}) $$

Every quantity is per group. Fix PPV equal across groups and let $p$ differ; then FPR and FNR cannot both be equal. Equal calibration plus unequal prevalence forces unequal error rates. There is no algorithmic escape.

Kleinberg, Mullainathan and Raghavan (2016) prove the companion result for scores rather than decisions: calibration within groups, balance for the positive class, and balance for the negative class are jointly satisfiable only when base rates are equal or the predictor is perfect.

Corbett-Davies and Goel (2018), The Measure and Mismeasure of Fairness, argue from the decision-theoretic side that all three criteria can be satisfied by degrading a group's outcomes, and that the constraints therefore cannot be treated as goals in themselves. The infra-marginality critique matters: comparing rates across groups with different risk distributions can indicate a difference in the distributions rather than in the treatment.

Interventions, by pipeline stage#

Pre-processing. Reweighting (Kamiran and Calders, 2012); learning fair representations (Zemel et al., 2013) minimising $I(Z; A)$ while preserving $I(Z; Y)$; disparate-impact removal (Feldman et al., 2015) by per-group quantile alignment of features. Model-agnostic, and weakest in guarantee.

In-processing. Constrained ERM — Agarwal et al. (2018), A Reductions Approach to Fair Classification, reduces fairness-constrained classification to a sequence of cost-sensitive problems solved by any base learner, with provable convergence to the constrained optimum. Adversarial debiasing (Zhang et al., 2018) trains a predictor against an adversary that predicts $A$ from the prediction. Strongest guarantees, requires retraining.

Post-processing. Hardt et al. (2016) derive the optimal equalised-odds predictor as a per-group randomised threshold, solvable by a small linear program over the group ROC curves. The achievable region is the intersection of the group ROC hulls, so the fair optimum is bounded by the worst group's curve. Cheap, needs $A$ at inference time, and often unlawful to use for that reason.

Individual and causal fairness#

Group metrics are averages and can be satisfied by a policy that treats similar individuals differently. Dwork et al. (2012) propose Lipschitz individual fairness: $D(f(x), f(x')) \le d(x, x')$ for a task-specific similarity metric $d$. The metric $d$ is the whole difficulty, and constructing it is equivalent to solving the fairness problem.

Counterfactual fairness (Kusner et al., 2017) requires the prediction to be unchanged in the counterfactual world where $A$ is different, under a specified structural causal model. It makes assumptions explicit and testable, and it requires a causal graph you cannot validate from observational data alone.

Reporting protocol#

• Report all four rates per group, with $n$ and bootstrap intervals.
• Report the intersecting cells, not only marginal groups, with counts.
• State the threshold and the split the threshold was chosen on.
• State which criterion the system is committed to and why, before results are available.

Fairlearn and AIF360 implement most of the above. Both are wrappers around the arithmetic here; the choice of criterion remains outside the library.

Papers#

• Hardt, Price and Srebro, Equality of Opportunity in Supervised Learning, 2016 — arxiv.org/abs/1610.02413
• Chouldechova, Fair Prediction with Disparate Impact, 2017 — arxiv.org/abs/1703.00056
• Kleinberg, Mullainathan and Raghavan, Inherent Trade-Offs in the Fair Determination of Risk Scores, 2016 — arxiv.org/abs/1609.05807
• Agarwal et al., A Reductions Approach to Fair Classification, 2018 — arxiv.org/abs/1803.02453
• Corbett-Davies and Goel, The Measure and Mismeasure of Fairness, 2018 — arxiv.org/abs/1808.00023
• Kusner et al., Counterfactual Fairness, 2017 — arxiv.org/abs/1703.06856

What to learn next#

• Explainability — once a gap is found, working out what drives it.
• Model cards and documentation — where these numbers get published.
• Model evaluation — the base rates and confusion matrices behind all of it.