/exploring-scouts

name: exploring-scouts
description: >
  How to explore and make sense of PostHog Signals scouts — the scheduled agents that scan a
  project and write reports into the Signals inbox. Use when a user wants to understand what
  scouts they have, how each one is behaving, and whether the fleet is actually working. Covers
  surveying the fleet and its schedules, reading recent scout runs and drilling into a single
  run's reasoning, inspecting the durable scratchpad memory the fleet has built up, tracing a
  run to the reports it wrote or edited, and assessing a scout's health and performance over time
  (cadence, success rate, report rate, signal-to-noise). Read-only and exploratory — to write or
  tune a scout, use `authoring-scouts` instead. Trigger on "what are my scouts doing",
  "how is my <x> scout performing", "show me recent scout runs", "why did this scout find/report
  nothing", "what has the fleet learned", "explore scout run <id>", "is my scout working".
metadata:
  owner_team: signals

Exploring Signals scouts

A **scout** is a scheduled agent that wakes on its own interval, looks at one PostHog project, decides what's genuinely worth surfacing, and either writes it into the Signals inbox as a **report** or closes out empty (a real, valid outcome). PostHog ships a fleet of canonical scouts — a cross-product generalist (`signals-scout-general`) plus per-surface specialists (error tracking, logs, AI observability, experiments, feature flags, session replay, web analytics, surveys, and more). A project may also have **custom scouts** beyond the canonical fleet — any `signals-scout-*` skill a team authored (e.g. `-brand-mentions`, `-mcp-feedback`) shows up here too, so don't assume a fixed roster: `scout-config-list` is the authoritative roster for a project. (One caveat: a just-authored scout has no config row until the coordinator's next tick auto-registers one — or until someone registers it via the write-side `scout-config-create` — so a brand-new scout may briefly be missing from the list.)

This skill helps you **understand and explore what a project's scouts are doing and how they're performing** — entirely through read-only MCP tools. It is the observability counterpart to the `authoring-scouts` skill (which teaches writing and tuning) and to the `inbox-exploration` skill (which covers the inbox reports scouts feed into). (The scout tools were recently renamed from `signals-scout-*` to `scout-*`; if a `scout-*` name comes back unknown, the server may still expose it under the legacy `signals-scout-*` name — search the tool catalog and call whichever name it returns.)

Governed metric first

When the question asks for scout failure rate or cost per run, call `posthog:metric-list` before the scout tools and look for `scout_run_fail_pct` or `scout_cost_per_run`. Run an approved, non-drifted match with `posthog:data-catalog-metric-run` for the canonical headline. If the user also asks which scouts, runs, or failure modes drive the result, answer the headline first, then use the workflows below for a noncanonical breakdown. If no governed metric matches, say so and label the derived measure noncanonical.

**A scout's output is inbox reports, written 1:1.** Scouts list `emit_report` / `edit_report` in their `allowed_tools` and **author or edit inbox reports directly**; a run's output shows up as **`emitted_report_ids`** (reports it authored) and **`edited_report_ids`** (reports it updated). The run rows also carry `emitted_count` / `emitted_finding_ids` — **legacy fields from the deprecated signal-emitting channel** (weak `emit_signal` findings a pipeline consolidated). On a report-channel scout they stay `0` / empty even on a productive run; a non-zero tally means the run came from a scout still on the legacy channel (an old custom scout, or a canonical scout not yet ported) — real output for that run, not noise. When unsure of a scout's channel, check its `allowed_tools` via `skill-get`. **Never read `emitted_count: 0` as "did nothing"** — check the report columns and the run summary first. A scout whose config carries a `structured_output_schema` has a third output channel next to reports: schema-validated **measurement records**, recorded as `$scout_structured_output` events in the project (only _scalar_ top-level payload keys flatten to `output_<key>` properties — object and array fields live solely inside the full `output` property, so a missing `output_<key>` is not a missing value; `subject` names the judged entity) rather than as run-row columns. The events are the ground truth — `metadata.derived.has_structured_output` says the run had at least one batch **accepted**, which is a fast per-run screen but not delivery confirmation (a rare capture failure after acceptance leaves it true with fewer or no events behind it), so count the events when the number of records matters. See [`references/scout-data-model.md`](references/scout-data-model.md) for the event shape. Each run also carries a `metadata` map. Top-level: the provenance set `harness_prompt_version` / `report_channel` (`none`, `emit`, `edit`, or `both`) / `skill_origin` / `github_guidance`, saying which instructions the run was given; plus routing keys (`model` / `runtime_adapter` / `reasoning_effort`) only when a gate or pin overrode the default. Nested under `metadata.derived`: booleans the harness computes at the end of the run (`has_emit_report`, `has_edit_report`, `has_self_improvement`, `has_chart`, `has_self_validation`, `has_structured_output`). When comparing runs (before/after a prompt change, one model against another), segment on all four provenance values first: runs differing on any of `harness_prompt_version`, `report_channel`, `skill_origin`, or `github_guidance` were given different instructions and aren't a like-for-like population. Runs predating this field have none of them, so treat missing provenance as unknown and exclude those runs from a comparison rather than pooling them. F