/agentsop-reranker-stage

name: agentsop-reranker-stage
version: 0.1.0
description: >-
  Adds and tunes a reranker stage for RAG using the retrieve-wide,
  rerank-narrow pattern. Use when relevant documents appear in the initial
  top-N but are buried by noise, top-1 precision or MRR is low despite adequate
  recall, or too many marginal chunks consume context. Covers cross-encoder,
  API, and local rerankers; N-to-k selection; and latency/cost tradeoffs. Do not
  use when retrieval recall itself is failing.
when_to_use:
  - "a RAG pipeline's answer quality has plateaued and the relevant doc is present in top-k but not ranked first"
  - "the LLM context window is pressured by too many marginal chunks and you want fewer, higher-precision chunks"
  - "retrieval recall is already good (hit-rate high) but MRR / top-1 precision is low"
  - "a user asks where to add a reranker, how to tune N vs k, or which rerank model to use (API vs local)"
  - "reviewing a pipeline with `similarity_top_k=20` and no postprocessor — a reranker is missing"
when_not_to_use:
  - "retrieval RECALL is the problem (the right doc is NOT in top-N at all) — fix retrieval/chunking/hybrid first"
  - "top-k already small (<=5) and answers are correct — no plateau, no lever to pull"
  - "hard real-time path where the extra rerank round-trip blows the latency budget and quality is already acceptable"
overlay: true
cross_links: [llamaindex, hybrid-retrieval]

Reranker Stage · SOP

> Third-person analytical view of how a mature RAG pipeline *thinks* about the > reranker. The skill is for an LLM agent that writes / reviews / debugs > retrieval code — it teaches the cross-framework reranking discipline, not one > vendor's API. For the per-framework API, descend to `[[llamaindex]]` > (node postprocessors) or `[[agentsop-hybrid-retrieval]]` (the recall stage that feeds > the reranker).

This is the **C4 gap skill** in the Phase-D enhance pass. The reranker SOP existed only buried inside `[[llamaindex]]` (`OP-03 AddReranker`, Stage 3 step 7, anti-pattern A6). It is the **highest-ROI single addition** to a naive RAG pipeline, so it earns a standalone overlay.

---

1 · 何时激活 (Activation Rules)

Activate when **any** holds:

1. A RAG pipeline's answer quality has **plateaued** after the cheap knobs (prompt, embedding model, chunk size) are exhausted — `[[llamaindex]]` Stage 3 lists reranking as the **last** optimization step, deliberately. 2. Diagnostics show the **relevant document is in top-k but buried** — high hit-rate, low MRR, wrong top-1. This is LlamaIndex failure modes **#1 / #10** ([[llamaindex]] `OP-03`). 3. The LLM **context window is under pressure** — too many marginal chunks inflate cost, latency, and "lost-in-the-middle" degradation. A reranker lets you retrieve 50 and feed 5. 4. A user asks **where to add a reranker, how to tune N vs k, or API vs local**.

Do **not** activate (boundary — see §6):

• **Recall is the bottleneck**: the right doc is *not in top-N at all*. A

reranker can only reorder what retrieval already found — fix retrieval, hybrid (`[[agentsop-hybrid-retrieval]]`), or chunking first.

• top-k is already small (≤5) and answers are correct — no plateau.
• A hard sub-100ms path where the extra round-trip is unaffordable and quality

is already acceptable.

---

2 · 核心心智模型 (Core Mental Model)

The one sentence

> **Retrieve wide for recall with a cheap bi-encoder; rerank narrow for > precision with an expensive cross-encoder that sees query + document > together — something the bi-encoder structurally could not do.**

Why two stages exist at all

The retriever (bi-encoder / vector search) embeds the query and every document **separately, offline**. Similarity is a dot product of two vectors that never met. This is *fast* (vectors are precomputed; ANN search is sub-linear) but *lossy*: the document's vector is a single "topic average" computed without knowledge of the query.

A **cross-encoder** takes `[query, document]` as a **single joint input** and runs full attention across both, emitting one relevance score. It sees exactly which query token matches which document token. This is far more accurate — and far more expensive: it cannot be precomputed, so it runs **once per (query, candidate) pair at query time**. Scoring 1M docs this way is infeasible; scoring **20-50** is cheap.

  query ─┐                               query ─┐
         ├─ dot product (precomputed)            ├─► [CROSS-ENCODER] ─► score
  doc  ─┘   ← bi-encoder, FAST, lossy     doc  ─┘   joint attention, SLOW, sharp
       RECALL stage (retrieve top-50)         PRECISION stage (rerank → top-5)

The reranker is the bridge: it spends cross-encoder accuracy on a small candidate set the bi-encoder produced cheaply. **Wide net, sharp knife.**

The order law (inherited from `[[llamaindex]]` Stage 3)

> Prompts first, reranking last. Reranking is high-impact but expensive — > exhaust the cheap knobs (prompt, embed model, chunk size, hybrid) before > spending per-query cross-encoder latency. But once those are spent, the > reranker is usually the **single biggest remaining lever** (5-15pp > faithfulness lift on noisy corpora — [[llamaindex]] `OP-03`).

What a reranker is NOT

• Not a recall fix — it reorders, never retrieves (§6).
• Not a chunking fix — it scores whole candidates, it does not resize them.
• Not free — every reranked candidate is an inference (local) or a billed unit

(API).

---

3 · SOP 工作流 (Agentic Protocol)

Each stage gates the next. Never skip the baseline measurement.

Stage 0 — Confirm the lever is real

Before adding anything, prove the symptom is *precision*, not *recall*:

1. Run the existing pipeline against ~30-50 labeled QA pairs. 2. Record **hit-rate@N** (is the gold doc in top-N?) and **MRR** (how high?). 3. **If hit-rate is low** → recall problem → STOP, fix retrieval / hybrid (`[[agentsop-hybrid-retrieval]]`) / chunking. A reranker will not help. 4. **If hit-rat