Deepfake detection

Problem framing#

Deepfake detection is binary classification under covariate shift where the shift is adversarial and unbounded. Formally, training draws from manipulation families $\mathcal{M}{\text{train}}$ and deployment draws from $\mathcal{M}{\text{test}}$, with no guarantee that $\mathcal{M}{\text{test}} \subseteq \mathcal{M}{\text{train}}$ and an adversary actively selecting $\mathcal{M}_{\text{test}}$.

This rules out standard i.i.d. generalisation guarantees. The relevant evaluation is therefore leave-one-manipulation-out cross-dataset AUC, not within-dataset accuracy.

Artefact families#

Upsampling and spectral traces. Transposed convolution and nearest/bilinear upsampling impose periodic structure on the spectrum. Zhang et al. (2019), Detecting and Simulating Artifacts in GAN Fake Images, characterise it; Frank et al. (ICML 2020) build a DCT-domain classifier on it; Durall et al. (CVPR 2020) show the azimuthal spectral falloff differs systematically. Corvi et al. (ICASSP 2023) extend the analysis to diffusion models and find weaker but still present traces.

Blending boundaries. Face X-ray (Li et al., CVPR 2020, arxiv.org/abs/1912.13458) predicts the blending mask rather than a fake/real label, on the argument that almost every face-swap pipeline composites a generated region into a real frame. It is trained on self-blended images and generalises better than label-supervised baselines, because the supervision target is method-agnostic.

Identity inconsistency. ICT (Dong et al., CVPR 2022) compares inner-face and outer-face identity, exploiting that a swap changes one and not the other. Requires a reference identity, which restricts the setting.

Temporal and physiological. Blink rate, rPPG consistency across facial regions, and head-pose/landmark dynamics. All were effective and all were closed once published, which is the structural problem with any published cue.

Benchmarks#

The DFDC result is the sobering one. The winning solution in an open competition with a large purse reached a level that is far from usable as automated evidence, on a held-out set drawn from the same collection effort.

Generalisation strategies#

• Method-agnostic supervision. Face X-ray's blending mask, and self-supervised blending augmentation (SBI, Shiohara and Yamasaki, CVPR 2022), synthesise training fakes from real images alone, removing the dependence on any particular generator.
• Frozen foundation encoders with minimal adaptation. Yermakov et al. (WACV 2026) fine-tune only LayerNorm parameters of a pre-trained vision encoder — roughly 0.03% of weights — with L2-normalised features and a metric-learning objective, reporting state-of-the-art average cross-dataset AUROC across fourteen benchmarks from 2019 to 2025. The finding that training on older, more diverse manipulation sets generalises better than training on recent ones is worth internalising.
• Reconstruction and one-class methods. Model the real-image manifold and flag deviation, avoiding the need for fake examples entirely. Weaker in absolute terms, more robust to novel generators.

Adversarial robustness#

Detectors are differentiable classifiers and inherit every adversarial vulnerability. Carlini and Farid (CVPR Workshops 2020), Evading Deepfake-Image Detectors with White- and Black-Box Attacks, reduce a detector's AUC from 0.95 to below 0.1 with perturbations invisible to a human, in both white-box and black-box settings. Any deployed detector should be assumed evadable by a motivated adversary.

Provenance as the structural answer#

Detection is a losing asymmetry: the defender must generalise to all future generators, the attacker needs one that works. Provenance inverts the burden.

C2PA (Coalition for Content Provenance and Authenticity) specifies cryptographically signed manifests binding capture device, edit history and assertions to the asset. Capture-time signing is shipping in some camera hardware. The limitations are real and should be stated: an unsigned file is not evidence of manipulation, signatures are stripped by most platform processing pipelines, and a signed image of a screen showing a fake is a valid signature of a false scene.

Regulation is moving in the same direction. Article 50 of the EU AI Act, applicable from 2 August 2026, requires providers of generative systems to mark synthetic output in a machine-readable form and requires deployers to disclose deep fakes. That obligation was left in place by the AI Digital Omnibus (Regulation (EU) 2026/1744), which deferred the high-risk obligations but not the transparency ones. Marking is a labelling duty on compliant actors; it does not constrain a non-compliant one.

Papers#

• Rössler et al., FaceForensics++, ICCV 2019 — arxiv.org/abs/1901.08971
• Li et al., Celeb-DF, CVPR 2020 — arxiv.org/abs/1909.12962
• Li et al., Face X-ray for More General Face Forgery Detection, CVPR 2020 — arxiv.org/abs/1912.13458
• Wang et al., CNN-generated images are surprisingly easy to spot... for now, CVPR 2020 — arxiv.org/abs/1912.11035
• Frank et al., Leveraging Frequency Analysis for Deep Fake Image Recognition, ICML 2020 — arxiv.org/abs/2003.08685
• Carlini and Farid, Evading Deepfake-Image Detectors, CVPR Workshops 2020 — arxiv.org/abs/2004.00622
• Dolhansky et al., The DeepFake Detection Challenge Dataset, 2020 — arxiv.org/abs/2006.07397
• Shiohara and Yamasaki, Detecting Deepfakes with Self-Blended Images, CVPR 2022 — arxiv.org/abs/2204.08376
• Yermakov et al., Deepfake Detection that Generalizes Across Benchmarks, WACV 2026 — arxiv.org/abs/2508.06248

What to learn next#

• GAN — the generator architecture whose upsampling leaves the traces above.
• Diffusion models — the newer family, with fainter traces and a different failure profile.
• Adversarial attacks — why every detector is evadable by a determined attacker.