Model cards and documentation

The proposals and what each covers#

Model Cards (Mitchell et al., FAT* 2019) specify nine sections: model details, intended use, factors, metrics, evaluation data, training data, quantitative analyses, ethical considerations, caveats and recommendations. The methodological contribution is the insistence on disaggregated evaluation across intersectional groups, motivated directly by Buolamwini and Gebru's Gender Shades finding that aggregate accuracy concealed a 30-point error gap.

Datasheets for Datasets (Gebru et al., 2018) covers the dataset: motivation, composition, collection process, preprocessing, uses, distribution, maintenance. Adapted from the electronics component datasheet, where the analogy is exact — the part works within stated operating conditions and not outside them.

Data Statements (Bender and Friedman, TACL 2018) target NLP specifically, with schedules for speaker and annotator demographics, curation rationale and language variety.

FactSheets (Arnold et al., IBM 2019) extend toward supplier's declarations of conformity, adding provenance, safety and lineage attributes for procurement contexts.

System Cards describe the deployed system rather than the model — the pre- and post-processing, the human review layer, the thresholds. The distinction matters because harm is a property of the system, and a model card that omits the threshold and the human loop describes something nobody deployed.

The empirical case for disaggregation#

Aggregate metrics are averages over the deployment population, so they are dominated by the majority slice. The relevant results:

• Buolamwini and Gebru (2018): commercial gender classifiers at high aggregate accuracy with error rates of 0.8% on lighter-skinned men and up to 34.7% on darker-skinned women. The worst cell was intersectional, not visible on either marginal.
• Koenecke et al. (2020): commercial speech recognisers with roughly double the word error rate for Black speakers versus white speakers, aggregate performance reported as strong.
• Oakden-Rayner et al. (2020): "hidden stratification" in medical imaging — clinically critical subclasses with far worse performance than the aggregate, occurring at rates too low to move the headline number.

Oakden-Rayner et al. give the sharpest framing: the dangerous slices are precisely those rare enough not to affect aggregate metrics and severe enough to matter clinically. Slice discovery, not slice reporting, is the harder open problem, and automated approaches (Eyuboglu et al., 2022, Domino) remain research tools.

Reporting uncertainty#

Slice metrics are estimates from small samples and should carry intervals. For a proportion $\hat{p}$ from $n$ samples, the Wilson score interval is preferred over the normal approximation, which is badly behaved near 0 and 1 and at small $n$:

$$ \frac{\hat{p} + \frac{z^2}{2n} \pm z\sqrt{\frac{\hat{p}(1-\hat{p})}{n} + \frac{z^2}{4n^2}}}{1 + \frac{z^2}{n}} $$

with $z$ the standard normal quantile (1.96 for 95%). For composite metrics such as F1 or AUC, bootstrap over the slice.

With many slices, multiple comparisons apply. Reporting the worst of twenty slices without correction will find an apparent disparity in a fair model at a predictable rate. Pre-register the slices, or report corrected intervals.

Documentation as a governance artefact#

The regulatory direction is toward mandated technical documentation for higher-risk systems. The EU AI Act's Annex IV lists the technical documentation requirements for high-risk systems, and the NIST AI RMF's Measure and Manage functions assume the existence of documented evaluation. Model cards and datasheets are close in shape to what these frameworks call for, and adopting them early is cheaper than retrofitting.

Two honest limitations. Documentation is self-reported, so it constrains a careless team more than an adversarial one; the enforcement mechanism is external audit, which requires access these documents alone do not grant. And Bommasani et al. (2023), the Foundation Model Transparency Index, found published disclosure for large models to be sparse and inconsistent — the practice lags the proposal by a wide margin.

Papers#

• Mitchell et al., Model Cards for Model Reporting, FAT* 2019 — arxiv.org/abs/1810.03993
• Gebru et al., Datasheets for Datasets, CACM 2021 — arxiv.org/abs/1803.09010
• Bender and Friedman, Data Statements for NLP, TACL 2018
• Oakden-Rayner et al., Hidden Stratification Causes Clinically Meaningful Failures, CHIL 2020 — arxiv.org/abs/1909.12475
• Koenecke et al., Racial disparities in automated speech recognition, PNAS 2020
• Raji et al., Closing the AI Accountability Gap, FAccT 2020 — arxiv.org/abs/2001.00973
• Eyuboglu et al., Domino: Discovering Systematic Errors with Cross-Modal Embeddings, ICLR 2022

What to learn next#

• AI regulation — the rules that increasingly expect this document.
• Deploying responsibly — what happens after the card is written.
• CI/CD for machine learning — wiring card generation into the pipeline.