Autoencoders

Definition and objective#

An autoencoder is a pair of parameterised maps trained to approximate the identity through a constrained intermediate space:

f_θ : R^n → R^k      encoder
g_φ : R^k → R^n      decoder

θ*, φ*  =  argmin  E_{x ~ p_data} [ L( x, g_φ(f_θ(x)) ) ]

• n — input dimension; k — code dimension
• L — reconstruction loss: squared error for continuous data, Bernoulli cross-entropy for data in [0,1]
• h = f_θ(x) ∈ R^k — the code, or latent representation

An autoencoder is undercomplete when k < n, which is the constraint that forces structure to be learned. Overcomplete autoencoders with k ≥ n need a different regulariser, since the identity map is available and optimal.

The linear case is exactly PCA#

Baldi & Hornik (1989) analysed the linear autoencoder g(f(x)) = W_2 W_1 x under squared loss. Every critical point corresponds to projection onto a subspace spanned by some subset of eigenvectors of the data covariance. All are saddle points except the global minimum, which projects onto the subspace of the top k eigenvectors — the PCA subspace.

The solution is unique only up to an invertible k × k transform: W_1 → A W_1, W_2 → W_2 A^{-1} leaves the reconstruction unchanged. So a linear autoencoder recovers the principal subspace, not the ordered principal axes. The Developer section measures the consequence: identical reconstruction error to four decimals, from different weights.

Kunin et al. (2019) show that adding L2 regularisation to a linear autoencoder breaks this symmetry and recovers the ordered principal directions, which explains why the two methods are not always as interchangeable as the theorem suggests.

Non-linear encoders and decoders generalise this to non-linear manifolds, and the factor-of-ten error reduction on the spiral is that generalisation observed directly.

Regularised variants#

Denoising autoencoder (Vincent et al., 2008). Corrupt the input and reconstruct the clean original: minimise E_{x, x̃ ~ q(x̃|x)} [ L(x, g(f(x̃))) ]. The identity map is no longer a solution even when overcomplete. Vincent et al. (2010) show that for small additive Gaussian corruption the objective is equivalent to penalising the Frobenius norm of the encoder Jacobian, and Alain & Bengio (2014) show the trained model estimates the score ∂ log p(x) / ∂x — the connection that leads directly to score-based diffusion models.

Contractive autoencoder (Rifai et al., 2011). Add λ ‖ ∂f(x)/∂x ‖_F^2 explicitly, making the representation locally insensitive to input perturbation while reconstruction keeps it from collapsing.

Sparse autoencoder. Penalise the average activation of code units, via an L1 term or a KL penalty towards a low target rate. Recently prominent again for a different purpose: sparse autoencoders trained on the residual stream of language models to extract interpretable, monosemantic features (Bricken et al., 2023; Templeton et al., 2024).

The variational autoencoder#

Kingma & Welling (2013) replace the deterministic code with a latent-variable model p_θ(x) = ∫ p_θ(x|z) p(z) dz, with p(z) = N(0, I). The marginal is intractable, so introduce a variational posterior q_φ(z|x) and optimise the evidence lower bound:

log p_θ(x)  ≥  E_{q_φ(z|x)} [ log p_θ(x|z) ]  −  KL( q_φ(z|x) ‖ p(z) )  =  ELBO(x)

• q_φ(z|x) — the encoder, typically N(μ_φ(x), diag(σ²_φ(x)))
• p_θ(x|z) — the decoder likelihood
• The first term is reconstruction; the second is the regulariser that fills the latent space in
• The gap between log p_θ(x) and the ELBO is exactly KL(q_φ(z|x) ‖ p_θ(z|x)), so maximising the bound both fits the data and sharpens the posterior approximation

The reparameterisation trick makes this trainable: sample z = μ_φ(x) + σ_φ(x) ⊙ ε with ε ~ N(0, I). The randomness is moved to an input, so the gradient flows into μ_φ and σ_φ with variance far lower than a score-function estimator gives.

The KL term is precisely what a plain autoencoder lacks, and precisely what the Developer section's latent-hole measurement diagnoses. Overlapping posteriors leave no unclaimed regions in latent space.

Two well-documented pathologies:

• Posterior collapse. With a sufficiently expressive decoder — an autoregressive one especially — the model can maximise the ELBO by setting q_φ(z|x) ≈ p(z), driving the KL term to zero and ignoring z entirely. Remedies: KL annealing, the free-bits constraint (Kingma et al., 2016), or weakening the decoder.
• Blur. A Gaussian decoder likelihood makes the reconstruction term a squared error, whose optimum under uncertainty is the conditional mean. Averaging over plausible completions is a smudge. This is the mirror image of the GAN's mode collapse: mode-covering blurs, mode-seeking sharpens.

β-VAE (Higgins et al., 2017) scales the KL term by β > 1, trading reconstruction for disentangled factors. Locatello et al. (2019) subsequently proved that unsupervised disentanglement is impossible without inductive biases or supervision, and that reported results were highly seed-dependent — a useful corrective to that literature.

Discrete latents and the modern role#

VQ-VAE (van den Oord, Vinyals & Kavukcuoglu, 2017) replaces the continuous code with a lookup into a learned codebook e_1..e_K, z_q = e_{argmin_j ‖ z_e − e_j ‖}. The argmin is non-differentiable, so the straight-through estimator copies the decoder gradient to the encoder, and two auxiliary terms keep the codebook and encoder aligned. Discrete codes avoid posterior collapse and let a separate autoregressive prior be fitted over the code grid afterwards.

This is where autoencoders sit in current systems. VQGAN (Esser, Rombach & Ommer, 2021) adds a patch discriminator and a perceptual loss to the VQ-VAE, giving sharp reconstructions at high compression. Latent diffusion (Rombach et al., 2022) then runs the entire diffusion process inside that latent space rather than on pixels, cutting compute by roughly an order of magnitude at 512×512. Stable Diffusion is a diffusion model over an autoencoder's code space.

So the autoencoder is not the generator. It is the coordinate system the generator works in, and the adversarial term from the GAN lesson is what keeps its decoder from blurring.

Masked autoencoders (He et al., 2022) mask 75 percent of image patches and reconstruct them with an asymmetric encoder-decoder, where the encoder sees only visible patches. High masking ratios make the task non-trivial and the asymmetry makes pretraining cheap. BERT (Devlin et al., 2019) is the same idea for text, and both are denoising autoencoders under a specific corruption process.

Autoencoders for anomaly detection, with caveats#

Score by reconstruction error, s(x) = ‖ x − g(f(x)) ‖², thresholded on held-out normal data. The Developer section demonstrates both the strength and the failure mode.

Two documented caveats matter in production.

Generalisation to anomalies. A sufficiently expressive autoencoder often reconstructs unseen anomalous inputs well, because the learned mapping is not restricted to the training manifold. Memory-augmented variants (Gong et al., 2019) restrict the decoder input to a convex combination of stored prototypes, forcing anomalies to be reconstructed from normal patterns and inflating their error.

Out-of-distribution likelihood. Nalisnick et al. (2019) showed deep generative models can assign higher likelihood to out-of-distribution inputs than to their own training data — a VAE trained on CIFAR-10 scoring SVHN higher. Reconstruction error is not likelihood, but the result is a standing warning against treating any single generative score as an anomaly detector without validation.

The practical consequence is the point made with data above: an autoencoder flags inputs that violate learned relationships, not inputs that are only rare. Pair it with a marginal density check or a per-feature range check.

Cost#

Forward       O( |θ| + |φ| )  per sample; roughly 2× a classifier of the same width
Latent size   k << n is the constraint doing the work
Latent diffusion  operating at 64×64×4 instead of 512×512×3 is a ~48× reduction
                  in the elements the diffusion model must process

Papers#

• Baldi, P. & Hornik, K. (1989). Neural Networks and Principal Component Analysis. Neural Networks 2(1).
• Hinton, G. & Salakhutdinov, R. (2006). Reducing the Dimensionality of Data with Neural Networks. Science 313(5786).
• Vincent, P. et al. (2008). Extracting and Composing Robust Features with Denoising Autoencoders. ICML.
• Rifai, S. et al. (2011). Contractive Auto-Encoders. ICML.
• Kingma, D. P. & Welling, M. (2013). Auto-Encoding Variational Bayes. — arxiv.org/abs/1312.6114
• Alain, G. & Bengio, Y. (2014). What Regularized Auto-Encoders Learn from the Data-Generating Distribution. JMLR 15.
• Higgins, I. et al. (2017). beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework. ICLR.
• van den Oord, A., Vinyals, O. & Kavukcuoglu, K. (2017). Neural Discrete Representation Learning (VQ-VAE). NeurIPS — arxiv.org/abs/1711.00937
• Nalisnick, E. et al. (2019). Do Deep Generative Models Know What They Don't Know? ICLR — arxiv.org/abs/1810.09136
• Locatello, F. et al. (2019). Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations. ICML.
• Esser, P., Rombach, R. & Ommer, B. (2021). Taming Transformers for High-Resolution Image Synthesis (VQGAN). CVPR — arxiv.org/abs/2012.09841
• He, K. et al. (2022). Masked Autoencoders Are Scalable Vision Learners. CVPR — arxiv.org/abs/2111.06377
• Rombach, R. et al. (2022). High-Resolution Image Synthesis with Latent Diffusion Models. CVPR — arxiv.org/abs/2112.10752

What to learn next#

• Transformers — the architecture that now sits on top of these code spaces.
• Diffusion models — the generator that runs inside an autoencoder's latent space.
• Clustering — what to do with the codes once you have them.