/agentsop-prompt-history-inspect

name: agentsop-prompt-history-inspect
version: 0.1.0
description: |
  Tool skill — the *first move* in any LM-debugging session: dump the actual rendered prompt
  the framework sent to the model, before changing anything else. Activate when an LM call
  produced an unexpected output (wrong answer, schema violation, refusal, truncation,
  cost spike, latency spike, infinite loop, "model got dumber after upgrade"). The skill
  enforces a 30-second inspect step BEFORE any prompt edit, model swap, retry, or temperature
  tweak. Cross-framework cheat sheet: DSPy `inspect_history`, LangGraph `get_state_history`,
  CrewAI `step_callback`, LangChain `set_debug`/`set_verbose`, Aider `/diff`+`--verbose`,
  raw OpenAI/Anthropic via `OPENAI_LOG=debug`/`ANTHROPIC_LOG=debug` or HTTPX event hooks.
  Do NOT activate for first-time prompt authoring, exploratory prompt design, or non-LM bugs.

Prompt-History Inspect — First Move Before Anything Else

> *"The prompt you wrote is not the prompt the model received."* > — Operating axiom for every framework that templates, injects few-shots, appends tool definitions, or wraps system messages.

---

1. 何时激活 (When to activate)

Activate this skill the **moment** an LM call surprises you, BEFORE any other debug move.

| Trigger | Signal | |---|---| | Output wrong | "Why did it answer X?" / hallucinated fact / wrong format / refusal | | Output truncated | mid-sentence cut, partial JSON, missing fields | | Output empty / repeats | model returns `""`, repeats the same token, loops | | Behaviour changed | "It worked yesterday" / "It worked on GPT-4o but not on Llama-3" | | Cost / latency spike | tokens jumped 3× without code change → something got injected | | Tool call wrong | wrong tool picked, args malformed, tool call missing | | Schema validation failed | Pydantic / Outlines / guidance grammar refused output | | Eval regression | metric dropped after upgrading framework version | | Production bug | a user-facing thread produced a wrong answer — need to see what the LM saw |

**Do NOT activate** when:

• You are *authoring* a new prompt for the first time (no rendered prompt exists yet).
• The bug is clearly outside the LM call (retriever returned empty, API key invalid, network down).
• The framework hasn't even been called yet (e.g., import error, schema validation pre-call).

**The trigger is universal across the stack.** Any framework that *templates* a prompt — DSPy, LangChain, LangGraph, CrewAI, LlamaIndex, Aider, Guidance, Outlines — has a layer between "what you wrote" and "what the model received." This skill is the first-line probe into that gap.

---

2. 核心心智模型 (Core mental model)

┌──────────────────┐    ┌──────────────────┐    ┌──────────────────┐
│ What you wrote   │ ≠  │ What was rendered│ ≠  │ What the LM saw  │
│ (template,       │    │ (after few-shot  │    │ (after provider  │
│  signature, etc) │    │  injection, tool │    │  reformatting,   │
│                  │    │  defs appended,  │    │  message squash, │
│                  │    │  system msg,     │    │  token truncation)
│                  │    │  history, etc)   │    │                  │
└──────────────────┘    └──────────────────┘    └──────────────────┘
       layer 1                layer 2                  layer 3
       (your code)         (framework render)      (provider transport)

**Three mental shifts** the agent must internalize:

1. **The rendered prompt is the ground truth, not your source code.** Frameworks silently inject system messages, append tool JSON-schemas, deduplicate messages, summarise history, truncate context, reorder fields. The *only* trustworthy artefact is what was sent on the wire.

2. **Inspect first, change nothing.** Premature prompt edits hide the bug. If you "fix" the prompt before seeing the rendered output, you're optimising against a hallucination of the problem. The discipline is: **dump → diff against expectation → identify which layer diverged → fix at that layer.**

3. **The inspect command is framework-specific but the SOP is universal.** DSPy gives you `inspect_history(n=1)`. LangChain gives you `set_debug(True)`. Raw SDKs give you `OPENAI_LOG=debug` or HTTPX event hooks. You learn one cheat sheet (§7) once and the SOP applies everywhere.

**The 30-second test.** Before you do *anything* else, you should be able to print the exact final prompt text in <30 seconds. If you can't — you're not set up to debug LMs. Fix that first.

---

3. SOP 工作流 (SOP workflow)

A four-step ritual. Do them in order. Do not skip ahead.

Step 1 — Dump the rendered prompt (≤30 sec)

Look up the framework on the §7 cheat sheet. Run the inspect command. Get the actual text of:

• The system message (if any)
• The user message
• Any few-shot demos that were injected
• Tool / function definitions (if any)
• Assistant prefill (if any)
• The model's raw response

For DSPy: `dspy.inspect_history(n=1)` [[dspy.ai/api/utils/inspect_history/](https://dspy.ai/api/utils/inspect_history/)]. For LangChain: `from langchain.globals import set_debug; set_debug(True)` [[python.langchain.com/api_reference/core/globals/langchain_core.globals.set_debug.html](https://python.langchain.com/api_reference/core/globals/langchain_core.globals.set_debug.html)]. For raw SDKs: `export OPENAI_LOG=debug` or `ANTHROPIC_LOG=debug` [[github.com/openai/openai-python](https://github.com/openai/openai-python), [github.com/anthropics/anthropic-sdk-python](https://github.com/anthropics/anthropic-sdk-python)].

Step 2 — Diff against expectation

Write down (mentally or in a scratch file): *what did I expect the prompt to contain?* Then compare to the dump. Look for:

• **Extra content** you didn't intend (auto-injected system message, hidden few-shot, tool JSON-schema bloat).
• **Missing content** you did intend (your variable interpolation rendered as `None` / empty / `{var}` literal).
• **Wrong order** (assistant message before system, tool call before tool result).
• **T