Face embeddings and ArcFace

From softmax to angular margin#

Standard softmax cross-entropy on the final layer is

$$ L = -\log \frac{e^{W_{y}^{\top} x + b_{y}}}{\sum_{j=1}^{C} e^{W_{j}^{\top} x + b_{j}}} $$

$x \in \mathbb{R}^d$ is the embedding, $W_j$ the $j$-th class weight column, $b_j$ its bias, $C$ the number of identities, $y$ the true label. The decision boundary depends on both magnitude and angle, so nothing constrains the geometry of the learned features.

Setting $b = 0$ and normalising both sides, $\lVert W_j \rVert = 1$ and $\lVert x \rVert = s$, reduces the logit to $s\cos\theta_j$ where $\theta_j$ is the angle between $x$ and $W_j$. All information now lives on a hypersphere of radius $s$.

ArcFace (Deng, Guo, Xue, Zafeiriou, CVPR 2019, arxiv.org/abs/1801.07698) adds an additive angular margin to the target logit:

$$ L_{\text{arc}} = -\log \frac{e^{s\cos(\theta_y + m)}}{e^{s\cos(\theta_y + m)} + \sum_{j \neq y} e^{s\cos\theta_j}} $$

$m$ is the margin in radians, paper default $0.5$; $s$ the scale, paper default $64$. Because $\cos$ is monotonically decreasing on $[0, \pi]$, adding $m$ inside strictly lowers the target logit, so the network must reduce $\theta_y$ below what the unmargined objective would demand.

The margin family, compared#

ArcFace's argument for the additive angular form is geometric: the margin is a constant arc length on the hypersphere, so the induced separation is uniform across the space. CosFace's margin, being additive in cosine, produces angular separation that varies with $\theta$. SphereFace's multiplicative margin is difficult to optimise and needed an annealing schedule.

Empirically the three are close on saturated benchmarks and separate on hard ones. Do not expect large differences on LFW, which sits above 99.8% for all of them.

Why the scale is necessary#

With $\lVert x \rVert = 1$, logits lie in $[-1, 1]$ and the softmax over $C$ classes cannot approach a one-hot distribution. The lower bound on the loss is roughly $\log C - 2$, and gradients stay small regardless of correctness.

Wang et al. (2017), NormFace, analyse this and derive a lower bound on $s$ as a function of $C$ and the target confidence. For $C \sim 10^5$ identities, $s$ in the range 30 to 64 is standard.

Complexity#

The head is a single matrix product, $O(BCd)$ for batch $B$, embedding dimension $d$ and $C$ classes. With $C = 10^7$ identities and $d = 512$, the weight matrix alone is 20 GB in fp32, which dominates everything.

Two standard responses:

• Partial FC (An et al., CVPR 2022, arxiv.org/abs/2203.15565) samples a subset of negative class centres per step. Sampling 10% of classes loses negligible accuracy and cuts the head's memory by an order of magnitude, which is what made ten-million-identity training practical.
• Sub-centre ArcFace (Deng et al., ECCV 2020) gives each identity $K$ sub-centres, tolerating label noise by letting outlier images attach to a minority sub-centre instead of dragging the main one.

Evaluation, and what the numbers mean#

LFW is saturated; treat any LFW number above 99.5% as uninformative. The benchmarks that still discriminate are IJB-B and IJB-C, reported as TAR at FAR $10^{-6}$ — true accept rate when only one impostor pair in a million is accepted. IJB-C provides roughly 15.6 million impostor comparisons, which is what makes a FAR of $10^{-6}$ measurable at all.

At those operating points the differences between recent methods are typically one to two percentage points, and are sensitive to training set, backbone and alignment. Compare methods only when all three are held fixed.

Embeddings are biometric data#

Template inversion is a solved problem in the research literature: Mai et al. (TPAMI 2019), On the Reconstruction of Face Images from Deep Face Templates, recover recognisable faces from embeddings alone, including from models they did not have access to. Later work with generative priors has improved this considerably.

The practical consequences are concrete. An embedding database is a biometric database under GDPR Article 9 and under India's DPDP Act, and is a "biometric identifier" under Illinois BIPA. Protection strategies exist — cancellable biometrics, homomorphic matching, secure enclaves — and each costs accuracy or latency. See privacy in ML for the wider picture on what leaks out of a stored representation.

Papers#

• Liu et al., SphereFace: Deep Hypersphere Embedding for Face Recognition, CVPR 2017 — arxiv.org/abs/1704.08063
• Wang et al., NormFace: L2 Hypersphere Embedding for Face Verification, ACM MM 2017 — arxiv.org/abs/1704.06369
• Wang et al., CosFace: Large Margin Cosine Loss, CVPR 2018 — arxiv.org/abs/1801.09414
• Deng et al., ArcFace: Additive Angular Margin Loss for Deep Face Recognition, CVPR 2019 — arxiv.org/abs/1801.07698
• Mai et al., On the Reconstruction of Face Images from Deep Face Templates, TPAMI 2019 — arxiv.org/abs/1703.00832
• An et al., Killing Two Birds with One Stone: Partial FC, CVPR 2022 — arxiv.org/abs/2203.15565

What to learn next#

• Embeddings — the same idea, applied to words instead of faces.
• Loss functions — where the margin lives and what it does to gradients.
• Vector databases — searching millions of embeddings without comparing every one.