/agentsop-reranker-stage

name: agentsop-reranker-stage
version: 0.1.0
description: >-
  Enhancement-overlay SOP for the reranker stage of a RAG pipeline — the "retrieve wide,
  rerank narrow" discipline. Activate when a calling agent owns a retrieval pipeline whose
  answers have plateaued: top-k contains the right document but it is buried below noise, or
  the context window is under pressure from too many marginal chunks. Encodes the one non-
  negotiable insight — a cheap bi-encoder retrieves *wide* for recall, then a more expensive
  cross-encoder (which reads query + document *together*) reranks *narrow* for precision;
  keep top-N=20-50, rerank to top-k=3-5. Covers when to add a reranker (and when not to),
  N-vs-k tuning, model choice (Cohere/Voyage API vs bge-reranker local vs
  SentenceTransformer cross-encoder), latency/cost budgeting, and the cross-framework
  mapping (LlamaIndex node postprocessors, LangChain ContextualCompressionRetriever,
  Cohere/Voyage rerank APIs, local cross-encoders). This is an ENHANCE overlay over the per-
  framework skills — cross-link `[[llamaindex]]` and `[[agentsop-hybrid-retrieval]]` for the
  deep API. Search keywords: cross-encoder, Cohere rerank, bge reranker, rerank RAG,
  ColBERT, retrieval reranking, improve RAG precision, retrieve wide rerank narrow.trigger_keywords:
  - "reranker"
  - "rerank"
  - "cross-encoder"
  - "CohereRerank"
  - "bge-reranker"
  - "SentenceTransformerRerank"
  - "node postprocessor"
  - "ContextualCompressionRetriever"
  - "voyage rerank"
  - "top_n after retrieval"
  - "retrieve wide rerank narrow"
when_to_use:
  - "a RAG pipeline's answer quality has plateaued and the relevant doc is present in top-k but not ranked first"
  - "the LLM context window is pressured by too many marginal chunks and you want fewer, higher-precision chunks"
  - "retrieval recall is already good (hit-rate high) but MRR / top-1 precision is low"
  - "a user asks where to add a reranker, how to tune N vs k, or which rerank model to use (API vs local)"
  - "reviewing a pipeline with `similarity_top_k=20` and no postprocessor — a reranker is missing"
when_not_to_use:
  - "retrieval RECALL is the problem (the right doc is NOT in top-N at all) — fix retrieval/chunking/hybrid first"
  - "top-k already small (<=5) and answers are correct — no plateau, no lever to pull"
  - "hard real-time path where the extra rerank round-trip blows the latency budget and quality is already acceptable"
overlay: true
cross_links: [llamaindex, hybrid-retrieval]

Reranker Stage · SOP

> Third-person analytical view of how a mature RAG pipeline *thinks* about the > reranker. The skill is for an LLM agent that writes / reviews / debugs > retrieval code — it teaches the cross-framework reranking discipline, not one > vendor's API. For the per-framework API, descend to `[[llamaindex]]` > (node postprocessors) or `[[agentsop-hybrid-retrieval]]` (the recall stage that feeds > the reranker).

This is the **C4 gap skill** in the Phase-D enhance pass. The reranker SOP existed only buried inside `[[llamaindex]]` (`OP-03 AddReranker`, Stage 3 step 7, anti-pattern A6). It is the **highest-ROI single addition** to a naive RAG pipeline, so it earns a standalone overlay.

---

1 · 何时激活 (Activation Rules)

Activate when **any** holds:

1. A RAG pipeline's answer quality has **plateaued** after the cheap knobs (prompt, embedding model, chunk size) are exhausted — `[[llamaindex]]` Stage 3 lists reranking as the **last** optimization step, deliberately. 2. Diagnostics show the **relevant document is in top-k but buried** — high hit-rate, low MRR, wrong top-1. This is LlamaIndex failure modes **#1 / #10** ([[llamaindex]] `OP-03`). 3. The LLM **context window is under pressure** — too many marginal chunks inflate cost, latency, and "lost-in-the-middle" degradation. A reranker lets you retrieve 50 and feed 5. 4. A user asks **where to add a reranker, how to tune N vs k, or API vs local**.

Do **not** activate (boundary — see §6):

• **Recall is the bottleneck**: the right doc is *not in top-N at all*. A

reranker can only reorder what retrieval already found — fix retrieval, hybrid (`[[agentsop-hybrid-retrieval]]`), or chunking first.

• top-k is already small (≤5) and answers are correct — no plateau.
• A hard sub-100ms path where the extra round-trip is unaffordable and quality

is already acceptable.

---

2 · 核心心智模型 (Core Mental Model)

The one sentence

> **Retrieve wide for recall with a cheap bi-encoder; rerank narrow for > precision with an expensive cross-encoder that sees query + document > together — something the bi-encoder structurally could not do.**

Why two stages exist at all

The retriever (bi-encoder / vector search) embeds the query and every document **separately, offline**. Similarity is a dot product of two vectors that never met. This is *fast* (vectors are precomputed; ANN search is sub-linear) but *lossy*: the document's vector is a single "topic average" computed without knowledge of the query.

A **cross-encoder** takes `[query, document]` as a **single joint input** and runs full attention across both, emitting one relevance score. It sees exactly which query token matches which document token. This is far more accurate — and far more expensive: it cannot be precomputed, so it runs **once per (query, candidate) pair at query time**. Scoring 1M docs this way is infeasible; scoring **20-50** is cheap.

  query ─┐                               query ─┐
         ├─ dot product (precomputed)            ├─► [CROSS-ENCODER] ─► score
  doc  ─┘   ← bi-encoder, FAST, lossy     doc  ─┘   joint attention, SLOW, sharp
       RECALL stage (retrieve top-50)         PRECISION stage (rerank → top-5)

The reranker is the bridge: it spends cross-encoder accuracy on a small candidate set the bi-encoder produced cheaply. **Wide net, sharp knife.**

The order law (inherited from `[[llamaindex]]` Stage 3)

> Prompts first, reranking last. Reranki