What is BM25 keyword search?
Understand BM25 keyword relevance, term statistics and document length, and distinguish lexical retrieval from hybrid search and later reranking.
BM25 is a lexical relevance-scoring method: it helps rank documents using matching query terms and statistics about those terms and documents. Its main ideas are term frequency, inverse document frequency and document-length normalization. It is useful background for understanding keyword search, but a BM25 score is not an AI visibility score, a citation probability or a measure of whether a document is factually correct.
The three ideas behind the score
Microsoft's BM25 relevance documentation describes how matching terms contribute to full-text search relevance. Term frequency concerns occurrences within a document. Inverse document frequency reflects how common a term is across the indexed collection. Length normalization helps account for differences in document length rather than rewarding a long document simply for containing more words.
These factors work together. Repeating a keyword is not a reliable publishing strategy, and a numerical score has meaning within the query and index that produced it. The scoring function is one part of a search system; its text analysis, filters, boosts and later ranking stages are separate configuration choices.
What lexical matching means
Lexical retrieval works with terms derived from text. The surrounding product can perform language analysis or other transformations before scoring, so lexical does not always mean matching raw character strings exactly. However, BM25 itself does not turn a query into a dense vector or compare documents by embedding similarity.
For a hypothetical help-centre query such as “Bluetooth pairing code,” matching the important terms can be useful. A differently worded page might still contain the answer, which is one reason a team might compare lexical retrieval with other methods. This example illustrates the distinction; it is not an observed ranking test or a claim about a particular engine's results.
BM25, hybrid search and reranking are different
A hybrid system can combine keyword and vector retrieval. In Azure AI Search's documented hybrid approach, those retrieval paths run together and their result lists are fused. A later semantic ranker is a different step. Applying a reranker to keyword results alone does not, by itself, make the retrieval hybrid.
Our guide to semantic versus hybrid search separates those stages. The reranker guide explains why a later ranking stage depends on the candidates it receives. None of these descriptions establishes the undisclosed implementation of an external search or answer engine.
A documented implementation, not a universal configuration
Azure's relevance guide explains where BM25 is used in that product and how its scores can be influenced by searchable fields and scoring configuration. Those product details should not be generalized to every search platform. Consult the documentation for the engine and version you actually use before choosing fields, parameters or result limits.
Keep retrieval depth separate from response pagination: a limit on how many results an API returns at once is not evidence that the system searched only that many documents. Likewise, a large raw score does not make a page universally better across unrelated queries or indexes.
A practical evaluation checklist
- Choose representative queries, including terms and identifiers that genuinely matter to the task.
- Decide what makes a result relevant before comparing configurations.
- Record the index, searchable fields, analysis settings and ranking configuration.
- Check both the candidates found and their ordering; they answer different questions.
- Compare a hybrid or reranking addition with the simpler baseline using the same evaluation definitions.
- Report missing or failed measurements separately rather than inventing a score.
What this means for a publisher
Use clear terminology and accurate descriptions that help readers identify the subject. Do not infer that repeating keywords, copying a scoring formula or adding a markup field will secure an AI citation. BM25 explains one approach to lexical relevance; it does not supply a universal formula for ranking in every search or answer system.
Related guides
Sources and further reading
Provider documentation can change. Check the current guidance before changing crawler or search settings.