Non-negative matrix factorisation
NMF breaks data into parts that can only add up, never cancel out — which is why its topics, ingredients and building blocks are ones a human can actually read.
- 7 min read
- 3 reading levels
- Published
Read these first
On this page 5
One lesson, three depths. Pick the one that fits you today — you can switch any time.
Beginner — No maths. Plain English.
NMF breaks your data into building blocks that are only ever added together, never subtracted.
That one restriction is what forces the blocks to be things you can recognise.
Think of a thali. Every meal on the menu is some combination of dishes: two scoops of dal, one roti, some rice, a little pickle. Crucially, a thali can only add dishes. There is no such thing as minus-one-roti. Describe a hundred meals this way and the dishes themselves — the reusable, recognisable parts — emerge naturally.
NMF decomposes data under exactly that rule. Every row must be rebuilt as a sum of parts, with non-negative amounts. That one restriction changes everything about how the parts look.
Why it exists
PCA and truncated SVD allow negative amounts. Their components cancel each other out — one component adds "cricket words", another subtracts a bit of them back. The rebuild is accurate, but each individual component is a strange object: "plus stadium, minus dollar, plus half of rupee". Useful mathematically, unreadable humanly.
Much real data cannot be negative anyway: word counts, pixel brightness, sound energy, purchase quantities, gene activity. For such data, "parts that only add" match how the data was physically made. A document adds words from its topics. A face image adds light from eyes, nose, mouth. NMF's parts line up with those real ingredients, and that is its entire appeal: the output reads like an ingredient list.
How it works
parts (topics)
documents = mix of x word recipe of
each topic each topic
"wicket fell..." = 0.8 x [cricket topic] + 0.0 x [finance topic]
"rupee slides..." = 0.0 x [cricket topic] + 0.7 x [finance topic]
"team owner = 0.2 x [cricket topic] + 0.8 x [finance topic]
buys stake..."NMF produces two small tables: how much of each part every row uses, and what each part is made of. Both tables contain no negative numbers, so every cell answers a plain question: "how much?".
A real example you have seen
Music apps describe your taste as amounts of moods — 40% workout energy, 30% late-night calm, 30% 90s nostalgia. Nobody's taste is minus 20% romantic. Additive parts are how people naturally describe mixtures, and NMF is the maths that produces descriptions in that shape.
Remember this
- NMF rebuilds every row as a non-negative sum of non-negative parts.
- The no-subtraction rule is why parts look like real topics or ingredients.
- Use it when your data cannot be negative and you need readable components.
What to learn next
- Truncated SVD and LSA — the unconstrained cousin, and what cancellation costs.
- Matrix factorization for recommenders — the same factorisation selling you films.
- Clustering — hard assignments where NMF gives soft mixtures.
Developer — Code and libraries.
Setup
pip install scikit-learnOutputs verified with scikit-learn 1.7.2.
Seven headlines, two ingredients
import numpy as np
from sklearn.decomposition import NMF
from sklearn.feature_extraction.text import CountVectorizer
docs = [
"cricket bat ball wicket over",
"wicket ball cricket stadium",
"bat ball over stadium crowd",
"rupee dollar market stocks",
"market stocks rally rupee",
"dollar market slide stocks",
"cricket stocks: team owner buys market stake", # a genuine mix
]
vec = CountVectorizer()
X = vec.fit_transform(docs) # word counts: all non-negative
nmf = NMF(n_components=2, random_state=0)
W = nmf.fit_transform(X) # how much of each topic per doc
H = nmf.components_ # how much of each word per topic
words = vec.get_feature_names_out()
for k, row in enumerate(H):
top = row.argsort()[-4:][::-1]
print(f"topic {k}: {', '.join(words[t] for t in top)}")
print("topic mix per document:")
print(W.round(2))topic 0: stocks, market, rupee, dollar topic 1: ball, cricket, over, bat topic mix per document: [[0. 0.96] [0.01 0.76] [0. 0.85] [0.7 0. ] [0.64 0. ] [0.64 0. ] [0.79 0.16]]
Exact decimals can vary slightly across sklearn versions and platforms; the clean two-topic split is the stable part.
The walkthrough
Read the last row first. The mixed headline — a cricket team owner buying market stake — scores 0.79 finance and 0.16 cricket. NMF did not force it into one bucket; it reported the blend. Rows are mixtures, and the numbers are proportions you can quote in a meeting.
W and H are the two tables. W (7 documents x 2 topics) says how much of each topic each document uses. H (2 topics x vocabulary) says which words make up each topic. Their product approximately rebuilds the original count matrix.
Zeros are the interpretability. Look how many entries are exactly 0. The no-negatives rule cannot use cancellation, so the factorisation prefers genuinely sparse, part-like structure. Compare truncated SVD on the same data, whose components mix positive and negative weights across all words.
Counts or tf-idf both work — both are non-negative. Counts keep the "amounts of ingredients" reading cleanest; tf-idf sharpens topics when common words blur them.
Common mistakes
Feeding negative values. Standardised columns contain negatives, and NMF raises ValueError immediately. If you have standardised your data, you have chosen the wrong tool or the wrong scaling — use min-max scaling to [0, 1] instead.
Expecting the same answer every run. NMF's optimisation has many valid local answers; different seeds give slightly different topics. Fix random_state for reproducibility, and for stability-critical work run several seeds and check the topics agree in substance.
Treating n_components as discoverable by formula. Two topics was our choice, not the data's. Try a range, inspect topics for coherence, and watch the reconstruction error curve for an elbow. Ten topics on seven documents would yield beautiful garbage.
Comparing W values across different n_components runs. The scale of W and H trade off against each other (doubling one and halving the other changes nothing). Compare proportions within a run, not raw magnitudes across runs.
Try it yourself
Set n_components=3 and rerun. Inspect the third topic's words: with only two real themes present, watch how NMF invents a third from leftovers — the standard failure smell when k is too big.
What to learn next
- Truncated SVD and LSA — the unconstrained cousin, and what cancellation costs.
- Matrix factorization for recommenders — the same factorisation selling you films.
- Clustering — hard assignments where NMF gives soft mixtures.
Researcher — Mathematics and papers.
The optimisation problem
Given non-negative X (n x d), find W >= 0 (n x k), H >= 0 (k x d) minimising a divergence D(X || W H). Standard choices: squared Frobenius ||X - WH||_F^2, and generalised Kullback-Leibler sum_ij (X_ij log(X_ij / (WH)_ij) - X_ij + (WH)_ij), the latter matching Poisson-count likelihoods and hence text. Unlike truncated SVD, no closed form exists: the problem is convex in W or H separately, non-convex jointly, and NP-hard in general (Vavasis, 2009, SIAM J. Optim.).
Algorithms
Lee and Seung's multiplicative updates (1999 Nature; 2001 NeurIPS) — e.g. for Frobenius, H <- H * (W^T X) / (W^T W H) elementwise — preserve non-negativity automatically and monotonically decrease the objective, though convergence can stall. sklearn defaults to coordinate descent (solver="cd") with nndsvd-based initialisation (Boutsidis and Gallopoulos, 2008), which seeds factors from an SVD and largely determines which local optimum you reach — the practical reason runs differ mainly by seed and init. solver="mu" remains available for KL loss. Per-iteration cost is O(nnz(X) * k) on sparse input.
Why parts emerge
Lee and Seung's famous demonstration: on aligned face images, NMF learns localised parts (eyes, noses, moustaches) where PCA learns holistic "eigenfaces". Non-negativity forbids cancellation, so overlapping full-face components cannot be corrected by subtraction and are penalised; sparse, non-overlapping parts win. The effect is real but conditional — on poorly aligned or non-compositional data, NMF's parts blur. Donoho and Stodden (2003) give the geometric condition: the parts-based solution is unique when the data satisfies a separability condition (each part appears somewhere nearly alone). Separability-based algorithms (Arora et al., 2012) achieve provable factorisation under that assumption — anchor words in topic-model language.
Relatives
With KL loss, NMF is algebraically near-identical to probabilistic latent semantic analysis (Hofmann, 1999; equivalence shown by Gaussier and Goutte, 2005); LDA (Blei, Ng and Jordan, 2003) adds Dirichlet priors and a full generative story. Sparse coding replaces non-negativity with an L1 penalty; archetypal analysis constrains parts to be convex combinations of data points. In recommenders, non-negative variants of matrix factorisation keep latent user tastes additive and explainable — the same interpretability dividend in a different currency.
What to learn next
- Truncated SVD and LSA — the unconstrained cousin, and what cancellation costs.
- Matrix factorization for recommenders — the same factorisation selling you films.
- Clustering — hard assignments where NMF gives soft mixtures.