How do reranker models work?

A reranker model takes a user query and a pre-retrieved list of candidate documents, then assigns new relevance scores to reorder that list. It does not fetch new documents or expand the pool. In Azure AI Search, the semantic ranker runs as a secondary stage over results first scored by BM25 or by Reciprocal Rank Fusion (RRF), rescoring up to the top 50 items and returning captions and optional answers extracted verbatim from your content (Semantic Ranking Overview - Azure AI Search).

Retrieval vs reranking: different jobs

Retrieval finds candidates; reranking orders them. Retrieval quality determines candidate recall (which items make the list). Reranking determines ordering (which of those items appear first). If retrieval fails to include a relevant document, a reranker cannot recover it later (Azure semantic ranker).

In Azure, initial retrieval uses BM25 (keyword) or RRF over hybrid/vector inputs; the semantic ranker then applies language understanding to reorder that preranked set (What is BM25 keyword search?, RRF vs reranking).

How a reranker works in practice (Azure example)

Inputs are consolidated and text-only

• Stage: Always L2 (second-level) ranking over a preranked set; reranking uses only text, even for hybrid/vector queries (Azure semantic ranker).
• Candidate cap: Only the top 50 results from the initial ranking proceed to reranking (Azure semantic ranker).
• Summarization: For each document, Azure summarizes specified fields to fit the model’s input size. Typical field priorities are title, keywords, then content; title and keywords accept up to 128 tokens each, with the remainder allocated to content. The summarization model accepts up to about 2,000 tokens per document, and the generated summary string passed to the ranker can be up to 2,048 tokens (as of November 2024). Excess text is trimmed (Azure semantic ranker).
• Synonyms: If you configure synonym maps for fields included in the semantic configuration, those synonyms are applied during reranking (Azure semantic ranker).

Scoring and outputs

• Scoring: The system assigns a @search.rerankerScore from 0 to 4, where higher means more semantically relevant to the query. Minor distribution shifts can occur over time; avoid overly granular thresholds (Azure semantic ranker).
• Outputs: The response includes the rescored list plus a semantic caption per document (plain and highlighted). You can also request an optional semantic answer if the query is question-like and a suitable passage exists. Captions and answers are extracted verbatim from your content; no generative model creates new text in this workflow (Azure semantic ranker).
• Query rewrite (preview): You can enable a request-time feature that expands a text query into multiple variants (for example, correcting typos). Those variants run through retrieval, then the reranker rescoring applies to the resulting set. Rewrite influences recall before reranking; the reranker still only reorders the returned candidates (Azure semantic ranker).

What rerankers can and cannot do

They can

• Promote results that better match the query’s context and intent compared with the initial keyword or fused ranking (Azure semantic ranker).
• Return helpful captions and, when found, direct answer passages copied from your indexed content (Azure semantic ranker).

They cannot

• Rerun the query over the whole corpus or retrieve items omitted by the initial stage; reranking works only over the existing candidate set, up to 50 documents in Azure (Azure semantic ranker).
• Create new information or paraphrase passages; captions and answers are verbatim extracts (Azure semantic ranker).
• Guarantee an answer if your content lacks answer-like text for the question format (Azure semantic ranker).

Relevance vs correctness and grounding

A reranker measures semantic relevance to the query, not factual correctness. In a retrieval-augmented generation (RAG) pipeline, reranking can improve which sources are shown first, but it does not by itself establish factual grounding or “truth.” You still need grounding practices and evaluation against known ground truth (What is RAG?, Ground truth in AI).

When a reranker helps most

Azure notes the strongest gains on information-rich, prose-style content such as knowledge bases and documentation, where semantic understanding can disambiguate terms and surface contextually relevant passages more effectively than keyword signals alone (Azure semantic ranker).

Practical implementation tips

• Maximize recall first: tune your retriever (BM25, hybrid, RRF fusion) so genuinely relevant items make the candidate set. A reranker cannot fix omissions (RRF vs reranking).
• Prioritize semantic fields: configure title, keywords, and content in priority order; long fields will be trimmed to fit token budgets (Azure semantic ranker).
• Interpret scores robustly: use @search.rerankerScore as a ranking signal and avoid brittle cutoffs because score distributions can vary slightly (Azure semantic ranker).
• Use captions and answers to aid UX: show highlighted passages and, when available, the direct answer extract to speed user judgment (Azure semantic ranker).
• Consider query rewrite carefully: it can improve recall before reranking by generating up to 10 variants, but it does not change the reranker’s scope or behavior (Azure semantic ranker).

Actionable checklist

• Define retrieval first: choose BM25, vector, or hybrid, and decide if you’ll fuse with RRF (RRF vs reranking).
• Set semantic configuration: list fields in priority order; enable synonym maps where helpful (Azure semantic ranker).
• Guard input sizes: structure documents so key content fits within summarization and 2,048-token summary limits (Azure semantic ranker).
• Log candidate coverage: ensure target answers appear in the top 50 before relying on reranking.
• Expose captions/answers: present the extracted passages to improve clarity in results.
• Monitor score shifts: keep thresholds coarse and test across updates.

FAQ

Can a reranker help if my retriever misses documents?
No. In Azure, the semantic ranker only rescors the top 50 results from the initial ranking; it cannot search the full corpus or add missing items (Azure semantic ranker).

Does reranking replace RRF?
No. RRF is a fusion method at retrieval time; reranking is a later semantic ordering stage. See our comparison: RRF vs reranking.

Can a reranker generate new answers?
No. Captions and optional answers are verbatim extracts from your indexed content (Azure semantic ranker).

How many candidates does Azure’s reranker consider?
Up to 50 top results from the initial BM25 or RRF ranking proceed to the semantic reranking stage (Azure semantic ranker).