/agentsop-multiscale-chunking

name: agentsop-multiscale-chunking
version: 0.1.0
description: >-
  Enhancement-overlay (C5) for RAG over long documents — the chunk-paradox resolution.
  Activate when a single fixed chunk size cannot satisfy both retrieval precision (small
  chunks) and generation context (large chunks): small chunks lose surrounding context,
  large chunks dilute embedding relevance into "topic averages". Encodes the core flip —
  decouple the embed-unit from the return-unit: embed small for retrieval precision, return
  large for synthesis context — and the SOP to pick a base chunk size, choose a horizontal
  (sentence-window) vs vertical (auto-merging / parent-child) expansion strategy, and
  measure the lift. Cross-links [[llamaindex]] for the full RAG SOP; this overlay supplies
  the missing "chunk-paradox-resolution" recipe that the framework docs
  (HierarchicalNodeParser, SentenceWindow) only describe in fragments. Medium-frequency for
  any RAG over long prose, manuals, filings, or codebases. Search keywords: chunk size,
  chunking strategy, parent document retriever, sentence window, small-to-big retrieval,
  hierarchical chunking, optimal chunk size.

Multi-scale Chunking · C5 Enhancement Overlay

> Overlay on top of [[llamaindex]]. The base skill teaches the 5-layer RAG > pipeline and lists `DecoupleChunkScope` as one optimization knob among many. > This overlay zooms in on that single knob and turns it into a standalone > recipe: **how to resolve the chunk paradox when one chunk size is provably > not enough.** Third-person analytical view for an agent writing / reviewing > RAG ingestion code — not an end-user tutorial.

---

1. 何时激活 (Activation Rules)

Activate this overlay when **all three** RAG preconditions hold and the chunk paradox has actually surfaced:

1. The corpus is **long documents** — prose manuals, financial filings, legal contracts, research papers, codebases — where a single answer-bearing fact sits inside a larger context that the LLM needs to interpret it. 2. A **chunk-size sweep has stalled**: small chunks (128–256) win retrieval precision but the LLM answers from fragments; large chunks (1024–2048) give rich context but recall on specific queries drops because the embedding becomes a "topic average". The official failure-mode checklist documents both poles as *separate* failures — #2 (wrong chunk from too-small) and #6 (context overflow / dilution from too-large) (cited in [[llamaindex]] R3). 3. Faithfulness or relevancy is **plateauing below target** and bumping `chunk_size` only moves the failure from one pole to the other.

Concrete triggers:

• "Answers are technically retrieved but the model lacks context to explain them."
• "I keep retuning chunk_size and it never wins on both faithfulness and recall."
• A reviewer sees `SentenceSplitter(chunk_size=4096)` shipped as the fix for

"incomplete answers" (this is anti-pattern A1 in [[llamaindex]]).

Do **not** activate when:

• The corpus is short/static (<100k tokens) — prompt-stuff with caching; multi-scale chunking is over-engineering (§6).
• The chunk-size sweep *did* converge on a single winner (e.g. 1024 for prose) — pin it and stop.
• Retrieval quality is fine and the bottleneck is orchestration or synthesis.

---

2. 核心心智模型 (Core Mental Model)

> **Decouple the embed-unit from the return-unit. Embed small for retrieval > precision; return large for generation context.**

The naive assumption is that the unit you index *is* the unit you feed the LLM. That single identity is the source of the paradox: it forces one chunk size to serve two opposing jobs.

NAIVE (one unit, two jobs)          MULTI-SCALE (two units, one job each)
─────────────────────────          ─────────────────────────────────────
       [ chunk ]                    embed unit  →  small  (precision job)
      /         \                          │
 embed it     feed it                   match
 (wants       (wants                       │
  small)       large)                 return unit →  large  (context job)
   ↓             ↓                          ▲
  CONFLICT — pick one,                   expand from
  lose the other                        match → parent / window

Three load-bearing sub-principles:

1. **A Node is a graph node, not a chunk.** In LlamaIndex a `Node` carries `relationships` (PREV/NEXT/PARENT/CHILD). Those links are exactly what let you store a small node for matching and *resolve* it to a larger node for return ([[llamaindex]] Principle 2). Multi-scale chunking is "build a chunk-graph", not "split into chunks".

2. **Two geometries of expansion.** Once embed ≠ return, you must choose *how* the small match expands into the large return:

• **Horizontal** — return N adjacent sentences around the matched sentence

(sentence-window). The expansion is *positional*.

• **Vertical** — return the parent chunk when enough sibling children match

(auto-merging / parent-child). The expansion is *hierarchical*.

3. **Match the geometry to the document, not to taste.** Flat narrative prose → horizontal. Documents with real structure (headings, sections, tables of contents) → vertical. This is the central dilemma case (§5.1).

The overlay's promise: this *strictly dominates* a compromise chunk size when the sweep frontier is non-flat — you no longer average two bad sizes.

---

3. SOP 工作流 (Decision Protocol)

A three-gate protocol. Do **not** skip Gate 0 — multi-scale chunking is only justified once a single chunk size has been proven insufficient.

Gate 0 — Pick the base chunk size first (and try to stop here)

Run the canonical sweep from [[llamaindex]] OP-02 *before* reaching for any multi-scale machinery:

from llama_index.core.evaluation import (
    FaithfulnessEvaluator, RelevancyEvaluator,
)
# 1. ~20 eval QA pairs via DatasetGenerator.from_documents(docs)
# 2. sweep:
for cs in (128, 256, 512, 1024, 2048):           # overlap = 0.1–0.2