/braintrust-tracing

name: braintrust-tracing
description: Braintrust tracing for Claude Code - hook architecture, sub-agent correlation, debugging
user-invocable: false

Braintrust Tracing for Claude Code

Comprehensive guide to tracing Claude Code sessions in Braintrust, including sub-agent correlation.

Architecture Overview

                         PARENT SESSION
                    +---------------------+
                    |  SessionStart       |
                    |  (creates root)     |
                    +----------+----------+
                               |
                    +----------v----------+
                    |  UserPromptSubmit   |
                    |  (creates Turn)     |
                    +----------+----------+
                               |
          +--------------------+--------------------+
          |                    |                    |
+---------v--------+  +--------v--------+  +--------v--------+
| PostToolUse      |  | PostToolUse     |  | PreToolUse      |
| (Read span)      |  | (Edit span)     |  | (Task - inject) |
+------------------+  +-----------------+  +--------+--------+
                                                    |
                                         +----------v----------+
                                         |   SUB-AGENT         |
                                         |   SessionStart      |
                                         |   (NEW root_span_id)|
                                         +----------+----------+
                                                    |
                                         +----------v----------+
                                         |   SubagentStop      |
                                         |   (has session_id)  |
                                         +---------------------+

Hook Event Flow

| Hook | Trigger | Creates | Key Fields | |------|---------|---------|------------| | **SessionStart** | Session begins | Root span | `session_id`, `root_span_id` | | **UserPromptSubmit** | User sends prompt | Turn span | `prompt`, `turn_number` | | **PreToolUse** | Before tool runs | (modifies Task prompts) | `tool_input.prompt` | | **PostToolUse** | After tool runs | Tool span | `tool_name`, `input`, `output` | | **Stop** | Turn completes | LLM spans | `model`, `tokens`, `tool_calls` | | **SubagentStop** | Sub-agent finishes | (no span) | `session_id` of sub-agent | | **SessionEnd** | Session ends | (finalizes root) | `turn_count`, `tool_count` |

Trace Hierarchy

Session (task span) - root_span_id = session_id
|
+-- Turn 1 (task span)
|   |
|   +-- claude-sonnet (llm span) - model call with tool_use
|   +-- Read (tool span)
|   +-- Edit (tool span)
|   +-- claude-sonnet (llm span) - response after tools
|
+-- Turn 2 (task span)
|   |
|   +-- claude-sonnet (llm span)
|   +-- Task (tool span) -----> [Sub-agent session - SEPARATE trace]
|   +-- claude-sonnet (llm span)
|
+-- Turn 3 ...

Sub-Agent Tracing: What Works and What Doesn't

What Doesn't Work

**SessionStart doesn't receive the Task prompt.**

We tried injecting trace context into Task prompts via PreToolUse:

# PreToolUse hook injects:
[BRAINTRUST_TRACE_CONTEXT]
{"root_span_id": "abc", "parent_span_id": "xyz", "project_id": "123"}
[/BRAINTRUST_TRACE_CONTEXT]

But SessionStart only receives session metadata, not the modified prompt. The injected context is lost.

What DOES Work

**Task spans in parent session contain everything:**

• `agentId` - identifier for the sub-agent run
• `totalTokens`, `totalToolUseCount` - metrics
• `content` - full agent response/summary
• `tool_input.prompt` - original task prompt
• `tool_input.subagent_type` - agent type (e.g., "oracle")

**SubagentStop hook receives the sub-agent's `session_id`:**

• This equals the sub-agent's orphaned trace `root_span_id`
• Allows correlation between parent Task span and child trace

The Correlation Pattern

**Current state:** Sub-agents create orphaned traces (new `root_span_id`).

**Correlation method:** 1. Query parent session's Task spans for agent metadata 2. Match `agentId` or timing with orphaned traces 3. Sub-agent's `session_id` = its trace's `root_span_id`

**Future solution (not yet implemented):**

SubagentStop fires -> writes session_id to temp file
PostToolUse (Task) -> reads temp file -> adds child_session_id to Task span metadata

This would link: `Task.agentId` + `Task.child_session_id` -> orphaned trace `root_span_id`

State Management

Per-Session State Files

~/.claude/state/braintrust_sessions/
  {session_id}.json       # Per-session state

Each session file contains:

{
  "root_span_id": "abc-123",
  "project_id": "proj-456",
  "turn_count": 5,
  "tool_count": 23,
  "current_turn_span_id": "turn-789",
  "current_turn_start": 1703456789,
  "started": "2025-12-24T10:00:00.000Z",
  "is_subagent": false
}

Global State

~/.claude/state/braintrust_global.json   # Cached project_id
~/.claude/state/braintrust_hook.log      # Debug log

Debugging Commands

Check if Tracing is Active

# View hook logs in real-time
tail -f ~/.claude/state/braintrust_hook.log

# Check if session has state
cat ~/.claude/state/braintrust_sessions/*.json | jq -s '.'

# Verify environment
echo "TRACE_TO_BRAINTRUST=$TRACE_TO_BRAINTRUST"
echo "BRAINTRUST_API_KEY=${BRAINTRUST_API_KEY:+set}"

Query Braintrust Directly

# List recent sessions
uv run python -m runtime.harness scripts/braintrust_analyze.py --sessions 5

# Analyze last session
uv run python -m runtime.harness scripts/braintrust_analyze.py --last-session

# Replay specific session
uv run python -m runtime.harness scripts/braintrust_analyze.py --replay <session-id>

# Find sub-agent traces (orphaned roots)