/agentsop-llamaindex

name: agentsop-llamaindex
description: |
  Operating-system distillation of LlamaIndex — the leading RAG / document-agent
  framework. Activate when the calling agent must build, debug, harden, or evaluate
  a Retrieval-Augmented Generation pipeline over unstructured/private data, decide
  between RAG primitives (Index types, retrievers, query engines, routers, agents),
  or pick LlamaIndex vs LangChain / Haystack / raw vector store for a coding task.
  Encodes the 5-layer mental model (Documents → Nodes → Indices → Retrievers →
  Query Engines / Response Synthesizers), the canonical RAG bootstrap SOP from
  baseline `VectorStoreIndex` through hybrid + reranker + eval-loop hardening,
  the official 13-failure-mode checklist, and 5 dilemma cases distilled from docs,
  GitHub issues, and 2025 production post-mortems.
version: 0.1.0

LlamaIndex · SOP

> Third-person analytical view of how LlamaIndex *thinks* about turning private > documents into a grounded answering system. The skill is for an LLM agent that > writes / reviews / debugs RAG code — not for an end user reading docs.

---

何时激活 (Activation Rules)

Activate this skill when any of the following holds:

1. The user's request involves building, modifying, or debugging a **RAG pipeline** (retrieval over private/unstructured data + LLM synthesis). 2. The user mentions **LlamaIndex** (`from llama_index...`), **LlamaParse**, **LlamaCloud**, or a LlamaIndex-style primitive (`VectorStoreIndex`, `SummaryIndex`, `IngestionPipeline`, `QueryEngine`, `SubQuestionQueryEngine`, `RouterQueryEngine`, `Settings`, `Workflows`). 3. The user is **comparing RAG frameworks** (LlamaIndex vs LangChain vs Haystack vs raw vector store). 4. The user is choosing between **stuffing context, RAG, or an agent** for a knowledge task. 5. The user is debugging retrieval quality (hallucinations, wrong chunks, stale data, embedding drift) — even if the codebase predates LlamaIndex, the failure-mode taxonomy applies. 6. The user is **evaluating** a RAG system (faithfulness, relevancy, MRR, hit-rate).

Do **not** activate when:

• The task is pure agent orchestration with no retrieval (use LangGraph/CrewAI skill instead).
• The corpus is tiny (<100k tokens, static) and prompt-stuffing is the correct answer.
• The data is pure SQL/tabular with no unstructured component.

---

核心心智模型 (Core Mental Model)

LlamaIndex's design rests on three principles that distinguish it from "vector DB SDK + custom glue":

Principle 1 — The Index is a noun, not a verb

> In LangChain, "indexing" is something you do to a vector store. In LlamaIndex, an `Index` is a first-class typed object with its own retrieval semantics. Picking the right Index is half the architecture decision.

The 5-layer pipeline:

Documents → Nodes → Index → Retriever → Query Engine → Response
   ↓         ↓        ↓         ↓             ↓
parsing   chunking  storage   filters    synthesis
metadata  graph     primitive  rerank    (refine/tree_sum/compact)

Each layer has a **distinct failure mode** and a **distinct optimization knob**. See `references/R1-architecture.md` for the layer-failure-knob mapping.

Principle 2 — A Node is a graph node, not a chunk

A `Node` carries: `text`, `metadata`, `embedding`, **`relationships`** (PREV/NEXT/PARENT/CHILD links), and lifecycle ids. The `relationships` field is what enables Hierarchical, Auto-Merging, and Sentence-Window retrieval. The mental flip: **don't think "split into chunks", think "build a chunk-graph"**.

Principle 3 — Indices are not interchangeable

| Index | Pick when | |---|---| | `VectorStoreIndex` | Default; semantic Q&A over chunks; ~90% of RAG cases | | `SummaryIndex` | "Summarize this whole doc" — small, fan-out synthesis | | `TreeIndex` | Hierarchical content with progressive zoom-in | | `KeywordTableIndex` | Keyword-heavy queries, no embeddings budget | | `PropertyGraphIndex` | Multi-hop reasoning over entities | | `DocumentSummaryIndex` | Mixed corpora needing document-level routing first |

A `RouterQueryEngine` over multiple per-task indices is often the correct top-level shape, not a single monolithic `VectorStoreIndex`.

The 2025 shift

LlamaIndex now positions as **"the leading document agent and OCR platform"** (README). `LlamaParse v2` + `Workflows 1.0` (June 2025) + `LlamaCloud` mark a strategic move from "RAG framework" to "platform between messy documents and document-grounded agents". For a coder agent: assume Workflows for any new agentic code (QueryPipeline is deprecated).

---

SOP 工作流 (Agentic Protocol)

The protocol every RAG implementation must walk through. Each stage gates on the next.

Stage 0 — Frame the problem

Before code, answer:

1. Is the corpus **unstructured + non-trivial size (>100k tokens) + growing**? If not → see `R4` boundaries; LlamaIndex may be the wrong tool. 2. Is **retrieval quality** the bottleneck (not orchestration)? If orchestration dominates → LangGraph leads, LlamaIndex becomes a retrieval tool *inside* it. 3. What is the **query distribution**? (lookup-only / summary / compare-contrast / mixed). This decides whether a single Index or a Router is needed.

Stage 1 — Baseline (cheap, fast, observable)

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.core.node_parser import SentenceSplitter

Settings.llm        = OpenAI(model="gpt-4o-mini")
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")
Settings.node_parser = SentenceSplitter(chunk_size=1024, chunk_overlap=20)

docs  = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(docs)
qe    = index.as_query_engine(similarity_top_k=4)

Pin `Settings` **once at app boot**, never inline. This eliminates the entire embedding-mismatch failure class (failure #4).

Stage 2 — Build the eval loop **before** optimizing anything

from llama_index.core.evaluation import (
    DatasetGenerator, Faith