/debug-with-trail

name: debug-with-trail
description: Debug runtime issues for servers, daemons, workers, and test runs by querying captured stdout/stderr via trail's MCP tools. Walks through prerequisites check, optional targeted instrumentation, log analysis, and mandatory cleanup of any added logs.
when_to_use: User asks to "debug this", "why is this failing", "what's in the logs", "why did it crash", "investigate this error", "trace the bug", "what's happening when I do X", "this test is failing", "why does this test fail", "fix the failing test", "the test suite is broken", pastes an error message / test failure output and asks for help diagnosing it, or volunteers a trail session id (e.g. "my server is running with trail id <uuid>", "the API is captured under trail, session <uuid>", or pastes the `capturing → <uuid>` line).
allowed-tools: Bash Read Edit Grep Glob mcp__plugin_trail_trail__list_sessions mcp__plugin_trail_trail__get_logs

Debug with trail

Use this workflow when investigating a runtime issue with the help of `trail` — a local CLI that captures process stdout/stderr into per-session JSONL files and exposes them via the MCP tools `list_sessions` and `get_logs`.

The workflow applies to two broad cases:

1. **Long-running processes** — servers, daemons, workers, anything that stays up and emits logs continuously. The user reproduces the issue by sending a request / triggering the code path, and you query the captured stream. 2. **Test failures** — a test (or a small subset) is failing and you need to figure out why. The "process" trail captures is the test invocation itself (`trail run -- go test ./...`, `trail run -- npm test`, `trail run -- pytest path/to/test.py`). The test runner's stdout/stderr (assertion failures, panics, framework output, plus any `print`/`console.log`/`fmt.Println` from the code under test) lands in a session you can query exactly the same way. Instrumentation goes into either the test file or the production code the test exercises; the marker-based cleanup rules are identical.

Most phases below are written with a server in mind because that's the more common case, but the test-debugging variant is called out wherever the flow differs (mainly around how the session is launched and how iteration works without a long-running daemon).

The workflow has six phases. Steps 1, 2, 4, 5, and 6 are **mandatory**. Step 3 is conditional — only do it when the existing logs don't already explain the issue. **Cleanup (Phase 6) only happens after the fix is verified in Phase 5** — never clean up instrumentation while the user might still want to keep digging or while a fix is unverified.

**Supporting files:**

• `reference.md` — auto-reloader / test-runner watcher tables, framework "ready" indicators, per-language instrumentation syntax. Load when the inline shortlists don't cover the user's stack.
• `examples.md` — three end-to-end worked traces (existing-logs diagnosis, closed-loop variant, failing test). Read when you want a concrete picture of phase compression.

---

Working ledger (`debug-notes.md`) — maintained throughout the session

From the moment a debugging task begins, keep a running scratchpad in a temp file so both you and the user can see how the investigation is evolving. This is your *mental ledger* — it stops you from re-testing the same hypothesis twice, makes it obvious what you've ruled out, and gives the user a chance to redirect early if you're chasing the wrong thread.

**Location.** Write it to a temp directory that is clearly disposable — e.g. `/tmp/trail-debug-<marker>.md` (use the same `TRAIL-DEBUG-<4-CHAR>` marker from Phase 3.1 if instrumentation has started, otherwise pick a short slug). Never put it in the project repo — it must not get committed.

**Structure.** Plain markdown, kept short. Append as you go; don't rewrite history. Suggested sections:

# Debug ledger — <one-line task summary>

## Problem statement
<exact failure as the user described it: error message, command, expected vs actual>

## Data & repro material
- Trail session id(s):
- Repro command (curl / test invocation / enqueue):
- Relevant files identified so far:
- Existing log lines that look suspicious:

## Hypotheses
1. [ ] <hypothesis> — basis: <why you think so> — how to test: <query / probe / fix attempt>
2. [ ] ...

## Test ledger
| # | Hypothesis | What I did | Finding | Verdict |
|---|------------|------------|---------|---------|
| 1 | H1 | added probe at orders.js:23, ran curl X | probe never fired | ruled out — code path not reached |
| 2 | H2 | queried level=error duration=5m | TypeError on req.body.user | likely root cause |

## Open questions / things still to try
- ...

**When to update.** At minimum:

• **Phase 1**, right after the active session is identified — write the problem statement, the repro command, and the session id.
• **Phase 2**, after reading existing logs and the suspect code — append the initial **Hypotheses** list. Tell the user what hypotheses you're forming before you start testing them, so they can correct course.
• **Phase 3.4 / Phase 5.2**, after each query of your marker output — append a row to the **Test ledger** with the hypothesis, what you did, what you found, and a verdict (confirmed / ruled out / inconclusive). Update or add hypotheses as new evidence comes in.
• **Phase 4**, before presenting analysis — re-read the ledger to make sure the hypothesis you're presenting is actually the one best supported, not just the most recent one.

**Deletion.** The ledger is removed in **Phase 6** after the user confirms satisfaction with the fix. See Phase 6.5.

---

Closing the loop: hot-reload + self-driven repro (strongly recommended setup)

The default flow asks the user to manually `Ctrl+C`, re-run `trail run -- ...`, and reproduce the issue every time you add instrumentation (Phase 3.3) or apply a fix (Phase 5.2). That's three human-in-the-loop steps per iteration, which makes long debugging sessions painfu