/creating-replay-vision-scanners

name: creating-replay-vision-scanners
description: "Guides agents through creating and safely sizing a Replay Vision scanner: choosing the scanner type (monitor/classifier/scorer/summarizer), shaping the RecordingsQuery that selects sessions, and — crucially — estimating the credits it will spend and checking the org's remaining budget before creating, so a broad scanner doesn't exhaust the budget on its first scheduled sweep.\nTRIGGER when: user asks to create, set up, or configure a Replay Vision scanner, OR when you are about to call vision-scanners-create, OR when widening an existing scanner's query, sampling_rate, or sampling_mode (or moving it to a pricier model) via vision-scanners-update.\nDO NOT TRIGGER when: only reading scanners or observations, deleting a scanner, or running an existing scanner against sessions on demand (vision-scanners-scan-session/-scan-sessions). For a one-off question about sessions you already have, use vision-scanners-inline-scan rather than creating a scanner — the skill's first section covers when that applies."

Creating Replay Vision scanners

A scanner is a standing LLM probe over session recordings. Once created and enabled, it runs on a **Temporal schedule that sweeps every 5 minutes**, applying its prompt to each new matching recording and recording the result as an observation (a queryable `$recording_observed` event). Each observation spends **credits** (1 credit = $0.01) from the org's budget for the current billing period, and an observation's price depends on the scanner's `model` — so size spend in credits, and convert any observation count a user gives you into credits at that model's price.

That schedule is exactly why creation needs a gut-check: a scanner with a permissive query and full sampling starts spending automatically and can drain the whole period's budget within its first few sweeps. Creation itself does **not** check quota — that protection only kicks in at observation time, by which point the budget may already be gone.

First: is a scanner even the right thing?

A scanner is a **standing watch over future recordings**. If the user has specific sessions in front of them and a question about those sessions, they don't want a scanner at all — they want `vision-scanners-inline-scan`, which takes `session_ids` plus a `prompt`, saves nothing, and schedules nothing.

Use an inline scan when the sessions are already known: "what went wrong in these five recordings", "did any of yesterday's checkout sessions hit the coupon bug", anything you'd otherwise answer by creating a scanner and deleting it afterwards. It costs the same credits per session and reuses answers when the same question is asked twice, so re-asking is cheap.

Create a scanner only when the user wants recordings that **haven't happened yet** to be scanned automatically. If you find yourself planning to create a scanner, read its results once, and delete it, stop and run an inline scan instead — a throwaway scanner leaves a scheduled sweep running against every future recording that matches its query.

Core principle: size before you ship

Never create an enabled scanner blind. Estimate its monthly credit spend, check the remaining credit budget, and — when the projected spend is a meaningful fraction of what's left — show the user the numbers and get confirmation before creating. This is the heart of the skill; the rest is supporting detail.

The flow

Step 1: What should the scanner do?

If the question, type, or prompt is still vague, run [[designing-replay-vision-scanners]] first. It turns a wish into one visible question with a fixed answer shape, a proof rule, and permission to say inconclusive. Come back here once those are settled.

Pick a `scanner_type` and write its `scanner_config`. Every type needs a `prompt`; the rest is type-specific:

| Type | What it produces | `scanner_config` shape | | ------------ | ----------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `monitor` | Open-ended observation against a prompt (e.g. "flag rage clicks") | `{"prompt": "..."}`; optional `"allow_inconclusive": true` (off by default, so the model must answer yes or no) | | `classifier` | Assigns tags from a fixed label set | `{"prompt": "...", "tags": ["tag-a", "tag-b"]}` — `tags` needs ≥1 entry; optional `"multi_label": false` (defaults to true), `"allow_freeform_tags": true` (off by default) | | `scorer` | Numeric score on a rubric | `{"prompt": "...", "scale": {"min": 1, "max": 5, "label": "frustration"}}` — `min` < `max`; `label` optional | | `summarizer` | Free-text summary | `{"prompt": "..."}`; optional `"length": "short" \| "medium" \| "long"` (default `"medium"`). Embeddings are always on |

`scanner_type` is **locked after creation** — to change it you delete and recreate, so confirm the type is right up front, and get the `scanner_config` shape right (a wrong shape is a create error, not a silent default — unknown keys are rejected too).

If the user's intent makes the type and prompt obvious, just proceed — don't interrogate them. When they only have a goal ("find where users get stuck in onboarding"), `vision-scanners-draft` drafts the type, config, query, model and sampling for you to review; it saves nothing. Ask the user how much it should use, and off