/agentsop-streaming-output

name: agentsop-streaming-output
description: |
  Enhancement-overlay decision protocol for STREAMING the output of long-running
  LLM / agent runs from the *backend*, not just wiring a typing animation in the
  UI. Activates when a coder agent must stream final tokens to a chat client,
  surface intermediate agent steps (which tool, which node, partial reasoning),
  emit custom tool-progress events, choose a transport (SSE vs WebSocket), or
  decide what to do when the client disconnects mid-stream. The langchain /
  langgraph skills mention stream modes but stop at "you can stream"; this skill
  encodes *what to stream, over what transport, and how to fail safely*.
version: 0.1.0

Streaming Tool/Agent Output · SOP (Enhancement Overlay)

> Source posture: every non-trivial claim is cited inline. Short tags like > `[lg/stream]`, `[lc/astream-events]`, `[oai/stream]`, `[anthropic/stream]`, > `[mdn/sse]` resolve against `references/R1-source-evidence.md`. > > This is an **ENHANCE overlay**: it sits on top of `[[agentsop-langgraph]]` (which > names the four stream modes but treats streaming as one of ten operations) and > `[[langchain]]`. Read those for the orchestration; read this for the > streaming SOP. Cross-link: `[[agentsop-langgraph]]` OP-8.

---

何时激活 (Activation Rules)

Activate when **any** of these fire:

• The run is **long** (multi-second to multi-minute agent loop, RAG over many

docs, multi-tool chain) and the user is **waiting** — perceived latency, not total latency, is the product metric.

• The user asks to "stream the response", "show a typing effect", "show progress",

"show which tool the agent is running", or "show the chain of thought".

• You are building a **chat** surface (stream final tokens) OR an **agent** surface

(stream intermediate steps: node entered, tool called, partial state) OR a **long task** surface (stream custom progress like "embedded 40/200 docs").

• You must pick a **transport**: Server-Sent Events (SSE) vs WebSocket vs plain

chunked HTTP, and handle **client disconnect** / cancellation cleanly.

• You're wiring `graph.stream(...)` / `astream_events` / OpenAI `stream=True` /

Anthropic `client.messages.stream` and need to know *which mode* and *what to forward to the client*.

Do **not** activate for: a single fast (<1s) completion, a batch/offline job with no waiting human, or a pure front-end animation question (that's CSS, not a backend SOP). Streaming a 300ms call adds protocol overhead for zero UX gain — see *反模式*.

---

核心心智模型 (Core Mental Model)

**Stream what the user needs to *see*, not everything the engine *emits*.** A backend stream is a curated projection of the run's internal event firehose onto exactly three audiences:

1. **Chat audience → final tokens.** A human reading prose wants character-by- character output of the *final* assistant message. In LangGraph this is `stream_mode="messages"` (LLM tokens + metadata); in raw SDKs it's `stream=True` / `.messages.stream` `[lg/stream]` `[oai/stream]` `[anthropic/stream]`. They do **not** want to see tool JSON or scratch nodes.

2. **Agent audience → intermediate updates.** A developer (or a power-user UI) watching an agent work wants "entered node `planner`", "calling tool `search`", "got 5 results" — the state diffs *between* steps. LangGraph: `stream_mode="updates"` (per-node diffs) `[lg/stream]`. LangChain LCEL: `astream_events` (a typed event stream: `on_chat_model_stream`, `on_tool_start`, `on_tool_end`) `[lc/astream-events]`.

3. **Progress audience → custom events.** Work happening *inside* one tool/node (a loop, a long embed, a download) is invisible to the framework's automatic events. You must **emit** progress yourself: LangGraph `stream_mode="custom"` via `get_stream_writer()` `[lg/stream]`; LCEL via custom callback / dispatched events `[lc/astream-events]`.

The load-bearing insight from the LangGraph docs: **stream modes are composable — pass a list** (`stream_mode=["messages","updates","custom"]`) and demultiplex on the client by the tuple tag `[lg/stream]`. So the real design question is never "can I stream" but **"which projection(s) does this surface need, and how do I tag them on one wire?"**

Second axiom: **a stream is a contract with a client that can vanish.** Networks drop, users close tabs, browsers cap connections. The backend must decide, *up front*, whether a disconnect should **cancel** the run (stop burning tokens) or **detach** and let it finish (so a reconnect can replay). That decision is part of the design, not an afterthought — see *困境 Case 2*.

---

SOP 工作流 (Agentic Protocol)

Walk top-down. Each step has a gate.

Step 1 · Confirm streaming is warranted

Gate: is a human **waiting** on a run that takes **>~1–2s**? If no (batch job, sub- second call), **don't stream** — return the whole payload. Streaming a fast call adds SSE/WebSocket framing, reconnect logic, and partial-parse bugs for no UX win `[mdn/sse]`. Exit here for fast paths.

Step 2 · Classify the surface → pick the projection

Map the surface to one (or more) of the three audiences:

| Surface | Primary projection | LangGraph mode | LangChain | |---|---|---|---| | Chat / prose | final tokens | `messages` | `astream_events` → `on_chat_model_stream` | | Agent inspector / dev UI | step updates | `updates` | `astream_events` (`on_tool_*`, `on_chain_*`) | | Full-state replay / resume | snapshots | `values` | n/a (rebuild from events) | | Long in-tool work | custom progress | `custom` | dispatched custom events | | Debug everything | raw firehose | `debug` | `astream_events` (all) |

`values` emits the **full state** after each step (heavy, good for resume); `updates` emits only the **diff** (light, good for live UI) `[lg/stream]`. Default a chat agent to `["messages","updates"]` and add `"custom"` only when a tool has internal progress worth surfacing `[lg/stream]` (= `[[agentsop-langgraph]]` OP-8).

Step 3 · Pick the