Learn AI Create the Future — with Pranay
Main site

Semantic Search and Reranking

FAISS

FAISS is the library most vector search systems are built on or benchmarked against. This lesson builds a real IVF index, tunes its speed-versus-accuracy knob, and saves it to disk.

Read these first

On this page 5
  1. Why it exists
  2. How it works
  3. A real example you have seen
  4. Remember this
  5. What to learn next

One lesson, three depths. Pick the one that fits you today — you can switch any time.

Beginner — No maths. Plain English.

FAISS is a library that stores millions of vectors and finds the closest ones to a query, fast.

Picture a huge warehouse with a million packages, sorted into labelled zones instead of piled in one giant heap. A forklift driver looking for a specific package does not check every crate in the building. They go straight to the right zone, then search only there. FAISS builds exactly that kind of zoned, searchable warehouse. The "packages" are numeric vectors, and the "zones" are found automatically by the library itself.

FAISS (Facebook AI Similarity Search) is an open-source library, built by Meta. It stores large collections of vectors and quickly answers "which of these are closest to this new vector?" It implements the HNSW and IVF ideas from the previous lesson as ready-to-use, heavily optimised code.

Why it exists

Understanding how IVF and HNSW work conceptually is one thing. Implementing either one correctly, fast, and without subtle bugs, is a serious undertaking. Most teams should not repeat that engineering effort from scratch. FAISS exists so nobody has to. It has been battle-tested across some of the largest vector search deployments in the world. It has become close to a de facto standard. Other vector databases often measure themselves against it.

How it works

build an index once:
  choose an index type (flat/exact, IVF, HNSW, or combinations)
  hand it your vectors -- FAISS organises them internally
        |
        v
search it many times:
  hand it a query vector, ask for the top-k closest
  FAISS returns them, using whatever internal structure
  you chose, in milliseconds
        |
        v
save the built index to a file, reload it later --
no need to rebuild it every time your program restarts

The choice of index type is the main decision FAISS asks of you. Pick exact search for small collections, or when you need guaranteed correctness. Pick IVF or HNSW for large collections, where approximate, fast answers are the better trade.

A real example you have seen

Product pages that say "customers who liked this also liked..." often return results instantly, from a catalogue of millions of items. That page is very likely backed by a FAISS index, or one of its direct descendants. This core library sits behind a large share of production similarity search worldwide.

Remember this

  • FAISS is a library, not a full database — it stores and searches vectors fast, and you build the rest of the system around it.
  • It implements the IVF and HNSW ideas from the previous lesson as production-grade, optimised code.
  • An index can be built once, saved to disk, and reloaded — no need to rebuild it on every restart.

What to learn next

Developer — Code and libraries.

Setup

bash
pip install faiss-cpu numpy

Outputs verified with faiss-cpu 1.14.3 on CPU.

Building an IVF index, tuning nprobe, and saving it

faiss_ivf.py
import time
import numpy as np
import faiss

rng = np.random.default_rng(0)
n_vectors, dim, n_queries = 50_000, 384, 200
vectors = rng.standard_normal((n_vectors, dim)).astype("float32")
queries = rng.standard_normal((n_queries, dim)).astype("float32")

flat_index = faiss.IndexFlatL2(dim)
flat_index.add(vectors)
_, exact_ids = flat_index.search(queries, 10)

nlist = 100  # number of coarse clusters ("cells") to split the data into
quantizer = faiss.IndexFlatL2(dim)
ivf_index = faiss.IndexIVFFlat(quantizer, dim, nlist)

t0 = time.perf_counter()
ivf_index.train(vectors)   # learns the nlist cluster centres with k-means
ivf_index.add(vectors)
build_time = time.perf_counter() - t0

for nprobe in [1, 8, 32]:
    ivf_index.nprobe = nprobe   # how many of the 100 cells to actually search
    t0 = time.perf_counter()
    _, ivf_ids = ivf_index.search(queries, 10)
    search_time = time.perf_counter() - t0
    overlaps = [len(set(exact_ids[i]) & set(ivf_ids[i])) for i in range(n_queries)]
    recall = sum(overlaps) / (n_queries * 10)
    print(f"nprobe={nprobe:>3}  search={search_time*1000:>6.1f} ms  recall@10={recall:.2%}")

print(f"\none-time train+add cost: {build_time:.2f} s for {n_vectors:,} vectors")

faiss.write_index(ivf_index, "products.index")
reloaded = faiss.read_index("products.index")
print("reloaded index has", reloaded.ntotal, "vectors, ready to search with no retraining")
Output
nprobe=  1  search=   3.4 ms  recall@10=3.75%
nprobe=  8  search=  17.9 ms  recall@10=21.35%
nprobe= 32  search=  68.8 ms  recall@10=62.85%

one-time train+add cost: 0.17 s for 50,000 vectors
reloaded index has 50000 vectors, ready to search with no retraining

Timings and exact recall numbers depend on the machine this ran on and, as in the previous lesson, on the fact that random Gaussian vectors are a genuinely hard case for any ANN method — expect meaningfully higher recall on real embeddings at the same nprobe settings. The trend itself, more cells searched trading speed for recall, holds regardless of hardware.

The walkthrough

quantizer = faiss.IndexFlatL2(dim) is IVF's coarse map, not a second search index you interact with directly. IVF needs some way to decide which cells are closest to an incoming query. Handing it a flat (brute-force) index over only the nlist cluster centroids — 100 of them here, not 50,000 — makes that coarse lookup itself essentially instant.

ivf_index.train(vectors) runs k-means once to find the cell centroids; .add(vectors) then assigns every vector to its nearest cell. These are two separate calls because you can, in principle, train on a representative sample and add a much larger full dataset afterward — useful when the full collection is too large to run k-means over directly.

nprobe is the one knob to reach for first when tuning IVF. The table shows it directly: nprobe=1 searches a single cell and is fastest but weakest; nprobe=32 searches nearly a third of all cells, far slower, far higher recall. Production systems typically start around nprobe in the low tens and tune from measured recall on their own data.

faiss.write_index / faiss.read_index persist the whole trained structure — the learned cluster centroids and every vector's assignment — so a real service loads a pre-built index at startup rather than retraining on every deploy.

Common mistakes

Calling .add() before .train(). IVF needs trained cluster centroids to know where to place incoming vectors — adding to an untrained index raises an error, on purpose.

Choosing nlist far too large or far too small for the data size. As a rough starting point, FAISS's own documentation suggests nlist on the order of sqrt(n) to a few times sqrt(n) for n vectors — too few cells makes each cell too large to search quickly; too many makes the coarse quantizer itself slow and each cell too sparse to be statistically meaningful.

Forgetting the index needs to be rebuilt, not only re-added-to, after major data changes. IVF's cell boundaries are learned once from the data present at .train() time. If the underlying data distribution shifts substantially afterward, a stale set of cell centroids can degrade recall — periodic retraining is a real operational concern at scale.

Try it yourself

Change nlist from 100 to 20 and to 400, re-run the nprobe sweep at each, and compare. A smaller nlist needs a smaller nprobe to reach comparable recall, because each cell already covers more of the space.

What to learn next

Researcher — Mathematics and papers.

Index types and when to choose each

  • IndexFlatL2 / IndexFlatIP — exact brute-force search, Euclidean or inner-product distance. O(n) per query, zero approximation error. The right choice below roughly a few hundred thousand vectors, or as ground truth for measuring recall of an approximate index, exactly as used throughout this section.
  • IndexIVFFlat — the coarse-quantizer-plus-cell-search structure demonstrated above. Cell assignment is exact within a probed cell; approximation comes entirely from which cells get probed.
  • IndexIVFPQ — combines IVF's cell partitioning with Product Quantization (Jégou, Douze & Schmid, 2011): each vector is compressed by splitting it into sub-vectors and quantizing each sub-vector independently against a small learned codebook, shrinking memory use by an order of magnitude or more at some further accuracy cost. This is the combination that lets FAISS scale to billions of vectors on commodity hardware.
  • IndexHNSWFlat — the graph-based structure from the previous lesson, also available directly through FAISS.

GPU acceleration

FAISS's original paper (Johnson, Douze & Jégou, 2017/2019) is explicitly framed around GPU-accelerated similarity search, reporting billion-scale nearest-neighbour search substantially faster on GPU than the best contemporary CPU implementations, through custom CUDA kernels for both exact and IVF-based search. faiss-gpu (a separate package from faiss-cpu) exposes this directly, with an index built on CPU transferable to GPU memory via faiss.index_cpu_to_gpu.

Distance metrics and normalisation, precisely

IndexFlatL2 computes squared Euclidean distance: ||q - v||^2. IndexFlatIP computes the raw inner product q . v. Neither is cosine similarity unless vectors are pre-normalised to unit length — a common source of silently wrong results, since FAISS performs no automatic normalisation. For normalised vectors, ranking by inner product and ranking by cosine similarity are identical, and ranking by L2 distance and by cosine similarity are monotonically related but not numerically identical, which matters when raw distance values (not only ranking) are consumed downstream.

Key references

  • Johnson, J., Douze, M. & Jégou, H. (2017, revised 2019). Billion-scale similarity search with GPUs. arXiv:1702.08734
  • Jégou, H., Douze, M. & Schmid, C. (2011). Product quantization for nearest neighbor search. IEEE TPAMI 33(1).
  • Douze, M. et al. (2024). The Faiss library. arXiv:2401.08281

Current state

FAISS remains the reference implementation most managed vector database products (Pinecone, Weaviate, Qdrant, Milvus, and others) either wrap directly or benchmark their own custom index implementations against. The 2024 library paper documents substantial newer additions — better GPU support, additional quantization schemes, and improved index-combination utilities — while the core IVF, HNSW and product-quantization building blocks covered here have remained conceptually stable since the library's original release.

What to learn next

Related terms

What to learn next

These follow on from what you just read.

  • Semantic Search and Reranking

    Query rewriting and expansion

    Add extra terms to a query using what its own top results already have in common. It genuinely improves recall, and it can also backfire by pulling in an off-topic result — this lesson shows both happening in the same example.

  • Semantic Search and Reranking

    Filtered vector search

    Combining a similarity search with hard constraints like category or price sounds straightforward, until pre-filtering and post-filtering are shown to return completely different, sometimes wrong, results on the exact same query.

  • Semantic Search and Reranking

    ColBERT and late interaction

    Instead of squashing a whole document into one vector, keep one vector per word and match each query word to its single best-matching document word — a genuine middle ground between bi-encoders and cross-encoders.