/agentsop-framework-selection

name: agentsop-framework-selection
version: 0.1.0
description: |
  Neutral, framework-agnostic decision tree for project kickoff: "which agent /
  RAG / LLM framework should I reach for?" Synthesizes the ecosystem sections of
  7 landmark-project SOPs (LangGraph, LlamaIndex, DSPy, CrewAI, vLLM, Aider, Dify)
  into one layered rubric. Core stance: frameworks are LAYERS, not competitors —
  a real project usually combines DSPy (compile) + LlamaIndex (retrieve) +
  LangGraph (orchestrate) + vLLM (serve), and you choose ONE per layer, not one
  to rule all. Use when starting any LLM/agent/RAG project, or whenever
  the "which framework?" question is asked. Deliberately neutral — unlike vendor
  docs and the LangChain-biased `framework-selection` on skill.sh, this skill has
  no horse in the race.
overlay: true
cross_links: [llm-engine-selection, agent-topology-selection, repo-state-gating]

Framework-Fit Decision Tree at Project Kickoff · SOP (ENHANCE overlay)

> Overlay posture: this is the **capstone** Phase-D skill — the most-cited entry > at any project kickoff. It decides *which layer(s) you need* and *which > framework owns each layer*. It does **not** teach any framework's API; for that, > descend to the per-framework SOPs (`langgraph-sop`, `llamaindex-sop`, > `dspy-sop`, `crewai-sop`, `vllm-sop`, `aider-sop`, `dify-sop`). Every > load-bearing claim carries an inline source tag resolving in > `references/R1-decision-tree.md`. > > Neutrality note: vendor pages each claim the center of the universe > (LangChain: "use LangGraph for production"; LlamaIndex: "the document agent > platform"; Dify: "scaffolding is the bottleneck"). This skill quotes those > claims but does not adopt any of them. The 7 SOPs *disagree* on the crossover > points; we surface the disagreements rather than papering over them.

---

1. 何时激活 (When to Activate)

Activate when **any** of the following fire:

• A new LLM / agent / RAG project is starting and no framework has been chosen yet.
• Someone asks "which framework should I use?" / "LangChain or LlamaIndex?" /

"LangGraph vs CrewAI?" / "do we need a framework at all?"

• A coder is about to `pip install` an orchestration / RAG / agent framework

before having articulated *what layers the project needs*.

• A project already picked one framework "for everything" and is now fighting it

in a layer it was never good at (e.g., doing deep RAG inside CrewAI, or hand-rolling retrieval inside LangGraph).

• A no-code / visual builder (Dify, Flowise, LangFlow) has hit a complexity

ceiling and the team is asking "do we rewrite in code?"

Do **not** re-run this skill mid-implementation for a layer already chosen — that is churn. Run it once at kickoff, and again only when a *new layer* appears (e.g., "we now need to self-host the model" → triggers `[[agentsop-llm-engine-selection]]`).

> Mental check: *the wrong framework is the single highest-cost decision in the > project — it is a one-week-to-reverse mistake, sometimes a one-month one.* > `crewai-sop · OP-1`, `vllm-sop · OP-7`. Spend 20 minutes on this tree before > opening any tutorial.

---

2. 核心心智模型 (Core Mental Model)

**Frameworks are layers, not competitors.** The single most common kickoff error is treating "LangChain vs LlamaIndex vs DSPy vs CrewAI vs vLLM" as a horse race with one winner. They are not on the same axis. A mature LLM system is a *stack*:

┌─────────────────────────────────────────────────────────────┐
│  L7 App platform / UI / Auth   │ Dify, Flowise, LangFlow      │  ship-fast scaffolding
├─────────────────────────────────────────────────────────────┤
│  L6 Serving / inference        │ vLLM, SGLang, llama.cpp …    │ → see [[agentsop-llm-engine-selection]]
├─────────────────────────────────────────────────────────────┤
│  L3 Orchestration / control    │ LangGraph, CrewAI, Workflows │ → see [[agentsop-agent-topology-selection]]
├─────────────────────────────────────────────────────────────┤
│  L2 Retrieval / context        │ LlamaIndex, Haystack         │  ingestion, index, query
├─────────────────────────────────────────────────────────────┤
│  L1 Modeling / prompt-compile  │ DSPy, Outlines, Guidance     │  the LM call itself
├─────────────────────────────────────────────────────────────┤
│  Coding-agent surface (cross)  │ Aider, Cline, Cursor …       │  end-user product, not a layer
└─────────────────────────────────────────────────────────────┘

DSPy's own ecosystem doc draws this layering explicitly — DSPy "sits *underneath* LangChain, LlamaIndex, LangGraph as a compiler for individual LM calls" `dspy-sop · R5`. LlamaIndex's doc says "many production systems use both: LlamaIndex as the retrieval layer, LangGraph as the orchestration layer" `llamaindex-sop · R5`. Dify's doc describes the hybrid "Dify for frontend/RAG/auth

• LangGraph for core agent logic behind HTTP" `dify-sop · R5`. The convergence is

unanimous: **choose per layer, then check interop.**

Three corollaries:

1. **You may not need every layer.** A static-corpus Q&A bot needs L1 only (stuff the context window). A RAG chatbot needs L1+L2. A durable multi-step agent needs L1+L2+L3. Only self-hosting adds L6. Only mixed-role teams add L7. 2. **The cleanest combinations are additive; the awkward ones are same-layer.** DSPy+LangGraph (compile-inside-node) and LlamaIndex+LangGraph (retrieve-then- orchestrate) are textbook `dspy-sop · R5`. DSPy+LangChain or DSPy+CrewAI are a "smell" — both are L1-ish prompt strategies fighting for the same slot `dspy-sop · R5`. 3. **"No framework" is a legitimate answer for ≥1 layer.** Frameworks earn their dependency surface only past a complexity threshold (§4 gate G0).

---

3. SOP (The Procedure)

The kickoff procedure runs as numbered steps (Pass A = Steps 1–6, Pass B = Step 7, Pass C = Step 8):

**Pass A — Identify the layers you actually need.** Walk the stack top to bottom and mark each layer needed / not-needed for *this* project:

1. L1 Mode