Stephen Robertson, Okapi, and BM25: Ranking Documents by Probabilistic Relevance
BM25 grew from Stephen Robertson's probabilistic retrieval research and the Okapi system, combining term rarity, saturating term frequency and document-length normalization.
Probabilistic retrieval asked which documents are most likely to be relevant
Information retrieval researchers in the 1970s developed probabilistic models that treated relevance as uncertain evidence rather than geometric similarity alone. Stephen Robertson became a central figure in this tradition, working on the Probability Ranking Principle and models that estimate how term occurrence should affect the odds that a document satisfies an information need.[1]
The eventual BM25 function emerged from this longer program. It should not be described as a single formula invented suddenly in 1994; it was the product of years of probabilistic modeling and Okapi system experiments.
Okapi turned probabilistic theory into a working retrieval system
At City University London, Robertson and colleagues built and evaluated the Okapi retrieval system. The software gave researchers a practical place to test term weighting, document-length normalization, relevance feedback and ranking functions against real collections.[2]
Stephen Walker implemented important parts of the system, and City later described Robertson and Walker’s work as the path that produced BM25.[3]
BM means Best Match rather than Bayesian Model
City’s historical account explains that BM stood for ‘Best Match’ and that 25 was the function number in the evolving Okapi software. The name therefore preserves the experimental engineering history of the model.[3]
Term frequency helped until repeated occurrences stopped adding much evidence
BM25 rewards a query term appearing multiple times in a document, but the gain saturates. The first few occurrences provide substantial evidence; repeating the same term dozens more times should not increase the score linearly.
The k1 parameter controls this saturation behavior. The design captures a practical retrieval insight: a document discussing a term repeatedly is probably more focused on it than a document mentioning it once, but repetition has diminishing returns.[1]
Document length normalization corrected a systematic ranking bias
Long documents naturally contain more term occurrences simply because they contain more text. BM25 adjusts term-frequency evidence according to document length relative to the collection average.[1]
The b parameter controls how strongly this normalization operates. A value near zero largely ignores length; larger values increasingly discount occurrences in unusually long documents.
Length normalization encodes an assumption about verbosity
The model distinguishes, imperfectly, between a document that is long because it covers many topics and one that is long because it gives more detail on the same topic. The normalization is a useful approximation rather than a semantic understanding of document structure.
Inverse document frequency retained the value of rare query terms
BM25 includes a collection-frequency component related to inverse document frequency. Terms appearing in few documents contribute more ranking evidence than terms appearing nearly everywhere.[1]
This connects Robertson’s probabilistic framework with the term-specificity lineage associated with Karen Spärck Jones. Modern BM25 therefore embodies several generations of information-retrieval research rather than one isolated trick.
TREC made ranking functions compete on common large test collections
The Text REtrieval Conference created by NIST in the 1990s gave research groups large collections, shared topics and common evaluation measures. The Okapi team’s TREC-3 report documented its 1994 experiments and weighting modifications.[4]
City’s retrospective account identifies the 1994 TREC period as the breakthrough that demonstrated BM25’s effectiveness strongly enough for the function to spread far beyond the original Okapi project.[3]
Evaluation infrastructure can make an algorithm historically visible
A ranking function becomes much easier to trust when independent teams can compare it against alternatives on shared data. TREC helped turn retrieval research into cumulative experimental engineering.
BM25 escaped Okapi and became a default baseline
Robertson and Hugo Zaragoza’s later review describes BM25 as one of the most successful text-retrieval algorithms produced by the probabilistic relevance framework.[1]
Open-source search libraries and engines adopted variants of BM25, and researchers routinely use it as a lexical baseline even when evaluating neural retrieval methods.
A strong lexical baseline remains hard to displace
Even modern neural-search evaluations routinely compare against BM25 because its term matching, saturation and length normalization provide a powerful baseline without requiring labeled training data.
Why BM25 remains one of search’s most durable ranking functions
BM25 combines three modest ideas remarkably well: rare terms are more informative, repeated occurrences have diminishing returns, and document length should affect how term counts are interpreted. The result is compact, fast and difficult to beat without additional signals or training data.[1][5]
Its history also illustrates how search advances are made: probabilistic theory, iterative software experiments and shared evaluation infrastructure all contributed to a ranking function that outlived the system in which it was born.
Works Cited
- 01
- 02Stephen Robertson — Publications and Okapi Research Archive staff.city.ac.uk
- 03
- 04NIST TREC-3 — Robertson et al., Okapi at TREC-3 (1994) pages.nist.gov
- 05Stanford IR Book — Okapi BM25: a non-binary model nlp.stanford.edu
CodeHistory is a living archive. Citations document the evidence used for this edition; later evidence may refine the account.
Submit a research lead