/designing-replay-vision-scanners

name: designing-replay-vision-scanners
description: "Designs a Replay Vision scanner that produces trustworthy observations: one visible question per scanner, a type chosen from the answer shape, a query that selects only sessions able to answer it, a prompt that demands on-screen proof and allows no or inconclusive, a model picked by the cost of a wrong answer, and a first-batch test pass. Sizing and the create call stay in creating-replay-vision-scanners.\nTRIGGER when: user wants to design, draft, or improve a Replay Vision scanner or its prompt, asks what to scan for, asks why a scanner returns vague observations, or asks an agent to propose scanners for a product.\nDO NOT TRIGGER when: the question and prompt are settled and the job is to size and create (use creating-replay-vision-scanners), the scanner targets one experiment (use scanning-experiments-with-replay-vision), or the job is to read existing observations (use exploring-replay-vision-observations)."

Designing Replay Vision scanners

A scanner is one prompt applied to one recording at a time. It watches the recording and writes an observation. Replay Vision fixes the watching part. It does not fix the thinking part. The thinking is the design work this skill covers.

Every scanner that produces useful observations shares three properties:

1. One focused question that only the recording can answer. 2. A recording query that selects only sessions able to answer it. 3. Permission to answer no or inconclusive.

Design in this order: question, type, query, prompt, model, first-batch test. Then hand off to [[creating-replay-vision-scanners]] for sizing and the create call.

Step 1: Test the question

Write the question in one sentence before you touch the API. Then run three checks.

**Only the recording can answer it.** If an event already answers the question, build an insight instead. A rage click event already proves the rage click. The scanner question must add something the event cannot: "did the clicked control actually fail?"

**One recording answers it.** A scanner sees one recording and cannot compare it with recordings it has never seen. Push every cross-session question to a scout, a digest, or product analytics. Comparisons between experiment variants, trends over time, and "how common is this" all live outside the scanner prompt.

**The answer has a fixed shape.** A yes or no with proof. One tag from a short list. A score on a stated scale. A summary with fixed labeled lines. If you cannot name the answer shape, you do not have a question yet.

Reject these questions and rewrite them:

• "Flag anything interesting." The model then decides what matters. That is the user's job.
• "Summarize what happened." Nobody reads a pile of plausible session summaries.
• "Compare the variants." One recording holds one variant.
• "Explain why they gave a low score." The recording shows behavior, not motive.

If the user has specific sessions in front of them and a one-off question, they do not need a scanner. Use `vision-scanners-inline-scan` instead.

Step 2: Pick the type from the answer shape

| Answer shape | Type | Design rule | | ------------------------------------------------ | ------------ | ------------------------------------------------------------------------------------------------------------------------------------- | | Yes or no, with proof | `monitor` | Set `allow_inconclusive: true` when many matched sessions never reach the flow in question. Yes only when the proof is on screen. | | One label from a short list | `classifier` | Keep 5 to 9 tags. Always include an escape tag. `multi_label` defaults to true; set it false unless one session carries several jobs. | | A number on a rubric | `scorer` | Use when the distribution matters more than any single observation. Define both ends of the scale in the prompt. | | A fixed set of labeled lines about one recording | `summarizer` | Name the lines in the prompt (for example Journey, Friction, Outcome, Evidence). Never ask for free prose. |

`scanner_type` is locked after creation. Get this right before the create call.

Escape tags for classifiers are not optional. A classifier must pick a tag. Without `nothing-broken`, `never-reached`, `cant-tell`, or `inconclusive` in the list, the model invents friction on sessions that contain none. Make the escape tag exclusive: the prompt must say it never combines with a defect tag.

Use `allow_freeform_tags: true` only when the goal is taxonomy discovery, and say so in the prompt: "invent a short snake_case tag only when the recording clearly shows a job outside this list". The tag vocabulary lives in `tags`, so the prompt describes the dimension and does not need to restate the list. If the prompt does define each tag, keep the two lists identical. Freeform tags hide drift between them, so re-read both together after every edit.

Step 3: Aim the query

The query decides which recordings deserve a judgment. The prompt decides what judgment to make. A better query improves quality more than another paragraph of prompt.

Build the query from the project's real data. Call `read-data-schema` with `{"query": {"kind": "events"}}`, or inspect existing scanners. Never invent an event name.

A query has one required part and two optional parts:

1. **The premise clause, required.** One predicate that proves the person used the surface in question, built from verified events or URLs, with explicit AND or OR where the premise needs more than one. Examples: a filter change, a saved object, an export, a survey submission, a wizard step. This clause establishes what the prompt may assume. 2. **A minimum active duration, opti