Content-based filtering

The representation#

Classic content-based recommenders represent item $i$ as a weighted term vector over a vocabulary $V$:

$$ w_{t,i} = \underbrace{\mathrm{tf}(t, i)}{\text{count in item } i} \times \underbrace{\log \frac{|I|}{\mathrm{df}(t)}}{\text{rarity across the catalogue}} $$

Where $t \in V$ is a term, $\mathrm{tf}(t,i)$ its frequency in item $i$'s document, $\mathrm{df}(t)$ the number of items containing $t$, and $|I|$ the catalogue size. Scikit-learn's default uses the smoothed form $\log\frac{1+|I|}{1+\mathrm{df}(t)} + 1$, whose additive $+1$ is precisely why a term occurring in every item retains non-zero weight.

Vectors are then L2-normalised and compared with cosine similarity:

$$ \mathrm{sim}(i, j) = \frac{\mathbf{w}_i \cdot \mathbf{w}_j}{|\mathbf{w}_i| \, |\mathbf{w}_j|} $$

The user profile#

The Rocchio formulation (Rocchio, 1971) builds a profile from positive and negative feedback:

$$ \mathbf{p}_u = \alpha \mathbf{p}_u^{(0)} + \frac{\beta}{|D_u^+|}\sum_{i \in D_u^+} \mathbf{w}_i - \frac{\gamma}{|D_u^-|}\sum_{i \in D_u^-} \mathbf{w}_i $$

Where $D_u^+$ and $D_u^-$ are the sets of items $u$ liked and disliked, $\mathbf{p}_u^{(0)}$ any prior profile, and $\alpha, \beta, \gamma \ge 0$ weight the three terms. Typical settings weight positives far above negatives ($\beta \gg \gamma$), which reflects an empirical asymmetry: in recommendation, a positive signal is specific and a negative signal is diffuse.

The profile-as-centroid assumption is the method's real limitation. For a multimodal preference distribution the centroid lies in a low-density region and matches nothing. Modelling $\mathbf{p}u$ as a mixture, or keeping a per-item nearest-neighbour score $\max{j \in D_u^+} \mathrm{sim}(i,j)$ instead of a centroid, both address this and both are cheap.

Where the modern version diverges#

Two changes account for most of the improvement since 2015.

Learned item representations. Rather than hand-built term vectors, encode item content with a trained model: a sentence encoder for text, a CNN or ViT for images, CLIP-style joint text-image encoders for both. Van den Oord, Dieleman and Schrauwen (2013), Deep content-based music recommendation, is the canonical demonstration: they trained a CNN on audio spectrograms to predict the latent factors of a collaborative filtering model, giving usable factors for tracks with zero plays.

Learned mapping instead of cosine. Cosine assumes item space and user space are the same space. Two-tower architectures learn separate encoders $f(u)$ and $g(i)$ into a shared space with a trained objective. Content features enter through $g$, so cold items get a position without any interaction history. This is now the default construction for cold-start-tolerant retrieval, and it is covered in two-tower retrieval models.

Over-specialisation, formally#

Let $\mathcal{S}_u = { i : \mathrm{sim}(i, \mathbf{p}_u) > \tau }$ be the reachable set at threshold $\tau$. Since $\mathbf{p}_u$ is a convex combination of vectors in $D_u^+$, and cosine similarity is continuous, $\mathcal{S}_u$ is contained in a neighbourhood of the convex hull of the consumed items. Any item whose representation lies outside that neighbourhood is unreachable, regardless of the user's true utility for it.

Collaborative signals are the standard escape, because co-occurrence is not constrained by content geometry. This is the formal reason hybrids are not a convenience but a requirement. Adomavicius and Tuzhilin (2005) state the argument in its original form.

Evaluation caveat#

Content-based methods look strong on offline metrics computed against logged data, and part of that is circular. Logged interactions were generated by a system that already ranked similar items highly, so items resembling the user's history are over-represented in the test set. Held-out accuracy therefore partly measures agreement with the previous policy. Report catalogue coverage and intra-list diversity alongside accuracy, or the number will flatter the method.

Papers#

• Rocchio (1971), Relevance Feedback in Information Retrieval, in The SMART Retrieval System.
• Salton and Buckley (1988), Term-weighting approaches in automatic text retrieval, Information Processing and Management.
• Adomavicius and Tuzhilin (2005), Toward the Next Generation of Recommender Systems, IEEE TKDE — the survey that named the limitations precisely.
• Van den Oord, Dieleman and Schrauwen (2013), Deep content-based music recommendation, NIPS.
• Lops, de Gemmis and Semeraro (2011), Content-based Recommender Systems: State of the Art and Trends, in the Recommender Systems Handbook.

What to learn next#

• Two-tower retrieval models — learned encoders replacing cosine on hand-built vectors.
• Hybrid recommenders — the formal escape from over-specialisation.
• Embeddings — how the item representation is learned.