/debug-codex-session

name: debug-codex-session
description: Investigate a specific Codex CLI session by session ID — correlate the local rollout transcript (`~/.codex/sessions/YYYY/MM/DD/rollout-*-<SESSION_ID>.jsonl`) with the router's production logs to understand what Codex rendered vs. what the upstream served on the /v1/responses path. Use when given a session ID and asked "why did X render?" (missing thinking, missing routing marker, wrong model, tool-call weirdness) for a Codex conversation routed through the router.

Debugging a Codex session

Given a Codex **session ID**, pull the local rollout transcript (what the client saw) and the corresponding production cloud logs (which model/provider served it), then correlate them. The rollout `.jsonl` is ground truth for *what rendered*; the cloud logs confirm *what the router decided and what the upstream sent*; `internal/translate/responses*.go` + `internal/proxy/service.go` explain *why the wire shape looks that way*.

The sibling skill [debug-claude-session](../debug-claude-session/SKILL.md) is the Claude Code counterpart. The workflow is the same shape; the transcript format, the id conventions, and the correlation key are all different — do not carry Claude assumptions over.

Setup: Cloud deployment config

Before starting, create a gitignored config file with your deployment's cloud logging details:

cat > .claude/skills/debug-codex-session/.deployment.json <<'EOF'
{
  "cloud_provider": "gcp",
  "project_id": "your-project-id",
  "region": "us-central1",
  "service_name": "router",
  "log_command_template": "gcloud logging read ... --project {project_id} --format=json"
}
EOF

If `.deployment.json` is missing, prompt the user for these details and walk them through creating it. The file is gitignored and contains no secrets — just the service/project/region names needed to construct cloud log queries.

Critical gotchas (read first)

• **Correlate by `client_session_id`, not by time.** Codex 0.149+ sends the session ID as the `Session-Id` header (and `Thread-Id` with the same value on the main thread); `sessionIDFromHeaders` ([internal/proxy/client_identity.go](../../../internal/proxy/client_identity.go)) picks it up and `bindRequestLogger` binds it as `client_session_id` on every log line. So the Codex session ID from the transcript filename is a **direct** log filter. This is the big difference from the Claude skill, which has to correlate by time + model.
• **The transcript does NOT record the served model.** `turn_context.model` and `thread_settings_applied.model` are the model Codex *requested* (e.g. `gpt-5.6-sol`). The router ignores the request's `model` field for routing. The **served** model appears only in (a) the injected Weave routing marker, when one was emitted, and (b) `decision_model` in cloud logs. Never report `turn_context.model` as the model that served the turn.
• **Empty `summary: []` on a reasoning item is not lost thinking.** Native Codex reasoning items carry opaque `encrypted_content` (replay state for the next Responses request) and a *public* `summary` array. With `summary: "auto"` the upstream frequently returns zero summary text while still billing `reasoning_output_tokens`. Check `event_msg/token_count → info.total_token_usage.reasoning_output_tokens` before concluding thinking was dropped — nonzero tokens with empty summaries means the model reasoned and the upstream chose not to surface a summary.
• **Codex subscription turns take the native passthrough path — the router does not translate them.** `codexResponsesRequest` ([internal/proxy/service.go](../../../internal/proxy/service.go)) detects a ChatGPT JWT + `ChatGPT-Account-ID`; the original Responses body is preserved and `ResponsesWriter.SetPassthrough()` / `SetPassthroughBadge()` ([internal/translate/responses.go](../../../internal/translate/responses.go)) forward upstream bytes verbatim. On that path, "the router dropped it" is almost never the answer — verify passthrough before blaming translation.
• **`ProxyOpenAIChatCompletion` in the logs does not mean Codex used chat completions.** `ProxyOpenAIResponses` delegates into the shared chat proxy for routing/billing/telemetry, so the completion log line is named `ProxyOpenAIChatCompletion complete` and `ingress="openai_chat_completions"` even for `/v1/responses` traffic. Check `path` on the access-log line to see the real surface.
• **Short base36 item ids are router-minted; long hex ids are upstream.** `newResponsesID` produces `msg_23aljo`-style ids, so a synthetic routing-badge assistant item looks like `msg_s6w20w`, while a genuine upstream assistant message looks like `msg_09cf01fda402...`. Same for `rs_`, `ctc_`, `fc_` prefixes. This is the fastest way to tell injected content from served content.
• **The routing marker is a separate assistant item, and it is stripped on the way back up.** `StripRoutingBadgeFromResponsesInput` removes the badge from replayed input so router text never reaches the model. A missing marker on a turn is often expected (sticky same-model turns emit nothing) — see [test-codex-locally](../test-codex-locally/SKILL.md) for the `PriorServedModel == ServedIdentity()` rule.
• **The rollout file mixes client-side and upstream events.** `response_item` entries are conversation items (both directions); `event_msg` entries are client-side UI/telemetry events. `developer`/`user` role messages with UUID-shaped ids are locally constructed, not served.

Workflow

- [ ] 1. Locate the local rollout transcript
- [ ] 2. Get the session shape (event histogram + session_meta)
- [ ] 3. Read turn_context: requested model, effort, summary mode
- [ ] 4. Extract the items showing the symptom
- [ ] 5. Check token accounting before calling anything "missing"
- [ ] 6. Fetch cloud logs filtered by client_session_id
- [ ] 7. Correlate transcript + cloud logs
- [ ] 8. Trace to the responses code path

1. Locate the local rollout transcript

Codex writes one rollout file