/agentsop-multiscale-chunking

name: agentsop-multiscale-chunking
version: 0.1.0
description: >-
  Designs multiscale chunking for RAG by embedding small units for retrieval
  precision and returning larger context for synthesis. Use when fixed-size
  chunks either lose surrounding context or dilute relevance in long documents,
  manuals, filings, or codebases. Covers sentence-window and parent-child or
  auto-merging strategies, base chunk sizing, and evaluation. Do not use when a
  single chunk scale already meets retrieval and generation needs.

Multi-scale Chunking · C5 Enhancement Overlay

> Overlay on top of [[llamaindex]]. The base skill teaches the 5-layer RAG > pipeline and lists `DecoupleChunkScope` as one optimization knob among many. > This overlay zooms in on that single knob and turns it into a standalone > recipe: **how to resolve the chunk paradox when one chunk size is provably > not enough.** Third-person analytical view for an agent writing / reviewing > RAG ingestion code — not an end-user tutorial.

---

1. 何时激活 (Activation Rules)

Activate this overlay when **all three** RAG preconditions hold and the chunk paradox has actually surfaced:

1. The corpus is **long documents** — prose manuals, financial filings, legal contracts, research papers, codebases — where a single answer-bearing fact sits inside a larger context that the LLM needs to interpret it. 2. A **chunk-size sweep has stalled**: small chunks (128–256) win retrieval precision but the LLM answers from fragments; large chunks (1024–2048) give rich context but recall on specific queries drops because the embedding becomes a "topic average". The official failure-mode checklist documents both poles as *separate* failures — #2 (wrong chunk from too-small) and #6 (context overflow / dilution from too-large) (cited in [[llamaindex]] R3). 3. Faithfulness or relevancy is **plateauing below target** and bumping `chunk_size` only moves the failure from one pole to the other.

Concrete triggers:

• "Answers are technically retrieved but the model lacks context to explain them."
• "I keep retuning chunk_size and it never wins on both faithfulness and recall."
• A reviewer sees `SentenceSplitter(chunk_size=4096)` shipped as the fix for

"incomplete answers" (this is anti-pattern A1 in [[llamaindex]]).

Do **not** activate when:

• The corpus is short/static (<100k tokens) — prompt-stuff with caching; multi-scale chunking is over-engineering (§6).
• The chunk-size sweep *did* converge on a single winner (e.g. 1024 for prose) — pin it and stop.
• Retrieval quality is fine and the bottleneck is orchestration or synthesis.

---

2. 核心心智模型 (Core Mental Model)

> **Decouple the embed-unit from the return-unit. Embed small for retrieval > precision; return large for generation context.**

The naive assumption is that the unit you index *is* the unit you feed the LLM. That single identity is the source of the paradox: it forces one chunk size to serve two opposing jobs.

NAIVE (one unit, two jobs)          MULTI-SCALE (two units, one job each)
─────────────────────────          ─────────────────────────────────────
       [ chunk ]                    embed unit  →  small  (precision job)
      /         \                          │
 embed it     feed it                   match
 (wants       (wants                       │
  small)       large)                 return unit →  large  (context job)
   ↓             ↓                          ▲
  CONFLICT — pick one,                   expand from
  lose the other                        match → parent / window

Three load-bearing sub-principles:

1. **A Node is a graph node, not a chunk.** In LlamaIndex a `Node` carries `relationships` (PREV/NEXT/PARENT/CHILD). Those links are exactly what let you store a small node for matching and *resolve* it to a larger node for return ([[llamaindex]] Principle 2). Multi-scale chunking is "build a chunk-graph", not "split into chunks".

2. **Two geometries of expansion.** Once embed ≠ return, you must choose *how* the small match expands into the large return:

• **Horizontal** — return N adjacent sentences around the matched sentence

(sentence-window). The expansion is *positional*.

• **Vertical** — return the parent chunk when enough sibling children match

(auto-merging / parent-child). The expansion is *hierarchical*.

3. **Match the geometry to the document, not to taste.** Flat narrative prose → horizontal. Documents with real structure (headings, sections, tables of contents) → vertical. This is the central dilemma case (§5.1).

The overlay's promise: this *strictly dominates* a compromise chunk size when the sweep frontier is non-flat — you no longer average two bad sizes.

---

3. SOP 工作流 (Decision Protocol)

A three-gate protocol. Do **not** skip Gate 0 — multi-scale chunking is only justified once a single chunk size has been proven insufficient.

Gate 0 — Pick the base chunk size first (and try to stop here)

Run the canonical sweep from [[llamaindex]] OP-02 *before* reaching for any multi-scale machinery:

from llama_index.core.evaluation import (
    FaithfulnessEvaluator, RelevancyEvaluator,
)
# 1. ~20 eval QA pairs via DatasetGenerator.from_documents(docs)
# 2. sweep:
for cs in (128, 256, 512, 1024, 2048):           # overlap = 0.1–0.2 × cs
    idx = build_index(docs, chunk_size=cs, overlap=int(0.15 * cs))
    record(cs, faithfulness=eval_f(idx), relevancy=eval_r(idx), p95=latency(idx))

• If a **single chunk size dominates** both faithfulness and relevancy → pin it

and **stop**. LlamaIndex's own Uber 10-K study peaked at **1024** for prose; code lands at **80–160** tokens (§5.2).

• If the **frontier is non-flat** (small wins precision, large wins context,

no single winner) → proceed to Gate 1. *Do not compromise on a middle size.*

Gate 1 — Choose the expansion strategy by document structure

| Document shape | Strategy | Geometry | |---|---|---| | F