/exploring-llm-traces

name: exploring-llm-traces
description: >
  Debug and inspect LLM/AI agent traces using PostHog's MCP tools.
  Use when the user pastes a trace or session URL (e.g. /ai-observability/traces/<id> or /ai-observability/sessions/<id>),
  asks to debug a trace, figure out what went wrong, check if an agent used a tool correctly,
  verify context/files were surfaced, inspect subagent behavior, investigate LLM decisions,
  or analyze token usage and costs. Also use when raw SQL/HogQL against
  `events.properties.$ai_input` / `$ai_output_choices` returns empty — message content lives only
  on the dedicated `posthog.ai_events` table.

Exploring LLM traces with MCP tools

PostHog captures LLM/AI agent activity as traces. Each trace is a tree of events representing a single AI interaction — from the top-level agent invocation down to individual LLM API calls.

Available tools

| Tool | Purpose | | ------------------------------- | ------------------------------------------------------------- | | `posthog:query-llm-traces-list` | Search and list traces; can return large multi-trace payloads | | `posthog:query-llm-trace` | Get a single trace by ID with full event tree | | `posthog:read-data-schema` | Discover custom event/person properties before filtering | | `posthog:execute-sql` | Ad-hoc SQL for complex trace analysis |

Event hierarchy

See the [event reference](./references/events-and-properties.md) for the full schema.

$ai_trace (top-level container)
  └── $ai_span (logical groupings, e.g. "RAG retrieval", "tool execution")
        ├── $ai_generation (individual LLM API call)
        └── $ai_embedding (embedding creation)

Events are linked via `$ai_parent_id` → parent's `$ai_span_id` or `$ai_trace_id`.

Workflow: debug a trace or session from a URL

Step 1 — Classify the URL

First inspect the path. Do not treat every UUID-looking value as a trace ID.

• `/ai-observability/traces/<trace_id>` or legacy `/llm-analytics/traces/<trace_id>` / `/llm-observability/traces/<trace_id>` is a single trace. Fetch it with `posthog:query-llm-trace`.
• `/ai-observability/sessions/<session_id>` or legacy `/llm-analytics/sessions/<session_id>` is an AI session, not a trace. Fetch traces with `posthog:query-llm-traces-list` filtered by event property `$ai_session_id`.

Preserve `date_from` / `date_to` query parameters from the URL when present. If none are present but the URL has a `timestamp` query parameter, use that timestamp as the anchor and query an absolute window around it, for example `timestamp - 36h` to `timestamp + 36h`. This handles exact session links whose UI timestamp may be offset from the stored event timestamps while keeping the query bounded. If the URL has neither explicit dates nor `timestamp`, use a safe default like `{"date_from": "-7d"}`.

For exact trace and session URLs, skip schema discovery for the standard `$ai_*` fields used below. These are AI observability built-ins, not project-specific custom properties.

Step 2 — Browse trace summaries

Explicitly set `detail: "summary"` when browsing traces. This keeps metadata and short content previews without spending context on full prompts and outputs. Omitting `detail` still returns full detail for compatibility with existing callers.

For a trace URL, call `posthog:query-llm-trace` with:

{
  "traceId": "<trace_id>",
  "detail": "summary",
  "dateRange": { "date_from": "-7d" }
}

For a session URL, call `posthog:query-llm-traces-list` with:

{
  "detail": "summary",
  "dateRange": { "date_from": "<timestamp_minus_36h>", "date_to": "<timestamp_plus_36h>" },
  "filterTestAccounts": false,
  "limit": 20,
  "properties": [{ "type": "event", "key": "$ai_session_id", "value": ["<session_id>"], "operator": "exact" }]
}

Use the URL's `date_from` / `date_to` values in the session query if present. If the URL only has `timestamp`, calculate the absolute date range from that timestamp instead of using a relative range like `-1h`. Set `filterTestAccounts: false` for an exact URL so the requested trace is not hidden by account filters.

The result contains trace and event metadata with previews of prompts, outputs, span states, and custom properties. A trace with `_detail: { "mode": "summary" }` contains previews, not the complete content.

From the result you get:

• Every event with its type (`$ai_span`, `$ai_generation`, etc.)
• Span names (`$ai_span_name`) — these are the tool/step names
• Latency, error flags, models used
• Parent-child relationships via `$ai_parent_id`
• `_posthogUrl` — **always include this in your response** so the user can click through to the UI

Step 3 — Read the content needed for the investigation

Once you have selected a trace, request `posthog:query-llm-trace` with `detail: "full"` before inspecting exact tool arguments, checking which context the model received, or searching conversation content:

{
  "traceId": "<trace_id>",
  "detail": "full",
  "dateRange": { "date_from": "-7d" }
}

Preserve the date range from the original URL or discovery query instead of copying the example range. Keep relevant property filters to narrow the read. If the user already identified the trace and needs exact content, you can request full detail directly.

Both modes enforce response size limits. Check truncation markers before drawing conclusions: an omitted event or a keyword missing from a preview is not evidence that it was absent from the trace. If full detail is still truncated, narrow the query to the relevant events or open `_posthogUrl` for the complete data.

Step 4 — Parse large full-detail results with scripts

When the result is persisted to a file (large traces with full `$ai_input`/`$ai_output_choices`), use the [parsing scripts](./scripts/) to explore it.

**Start with the summary** to g