/creating-experiments

name: creating-experiments
description: "Guides agents through experiment creation: reading the project's setup with experiment-setup-context, defining the hypothesis, configuring rollout and bucketing, setting up analytics and running time, and reporting which choices are guesses. Delegates rollout decisions to configuring-experiment-rollout and metric setup to configuring-experiment-analytics.\nTRIGGER when: user asks to create a new experiment or A/B test, OR when you are about to call experiment-create.\nDO NOT TRIGGER when: user is updating an existing experiment, managing lifecycle, or only browsing experiments."

Creating experiments

This skill walks through creating a new A/B test experiment: read the project, then the 3-step flow, then report the draft.

Core principle: draft first, iterate on details

Create the experiment as a draft quickly, then iterate on metrics and configuration. The user gets a tangible draft immediately and can refine it. Choose settings from the project's facts rather than asking, and say which choices are guesses. Bucketing is the exception: the facts cannot decide it, so ask before you create the experiment.

Step 0: Read the project

The right bucketing, metric and running time depend on the project: who sees the page, which SDKs evaluate the flag, how often the metric event happens. Read that before configuring anything.

1. From the request, infer the **target** (the event that marks someone reaching the change, usually `$pageview` for a page) and a **candidate metric event**. Confirm both exist with `read-data-schema`. Don't ask the user for event names you can find. For a page, read `event_property_values` for `$host` and `$pathname` on the target event as well. The response samples the values rather than listing them all, so read it for the shape the project records (a trailing slash, a `www.` prefix, the casing) rather than as proof that a value is absent. An exact filter has to carry that shape: `/pricing` matches nothing where every pageview says `/pricing/`, and `example.com` matches nothing where the host is `www.example.com`. 2. If the `experiment-setup-context` tool is available, call it once with `target_event` and `metric_event`. For a web surface, add `target_properties` with an exact `$host`. Add an exact `$pathname` as well when the surface is one page. `target_url_contains` is a substring match on `$current_url`. A bare domain matches any host that contains it, `notexample.com` included, and a homepage path matches every page under it. Both overstate the traffic and the exposure rate. For a surface that spans several pages, keep the exact `$host` and put only the path fragment in `target_url_contains`. Add `metric_properties` in the same call when the candidate metric counts only some occurrences of its event. Pass `previous_experiments_limit: 5`. Five experiments are enough to read a precedent, and the default of 10 roughly doubles the response for no more signal. If the tool is not available, continue without it and treat every choice below as a best guess. Never call a tool you can't see.

Each filter needs a `type` of `event` or `person`, a `key`, an `operator` and a `value`. The call rejects the `flag_evaluates_to` operator with a 400 that names it. For the homepage of one domain:

   {
     "target_event": "$pageview",
     "target_properties": [
       { "key": "$host", "type": "event", "operator": "exact", "value": ["www.example.com"] },
       { "key": "$pathname", "type": "event", "operator": "exact", "value": ["/"] }
     ],
     "metric_event": "your_conversion_event",
     "previous_experiments_limit": 5
   }

3. Apply [references/setup-decisions.md](references/setup-decisions.md) to the result. It maps each fact to a choice (bucketing, where the flag is evaluated, exposure, primary metric, running time, precedent) and to a tier for the summary. 4. Carry those choices into steps 1 to 3.

Ask the user only when a choice is "not decided" and the user is in the conversation. If the user said they are away, or asked for a draft without questions, decide with the defaults and list the open points in the report.

The 3-step creation flow

Step 1: What are we testing?

Gather these before calling `experiment-create`:

• **Experiment name** — descriptive, inferred from context when possible
• **Hypothesis** — what you expect to happen (goes in `description`)
• **Feature flag key** — kebab-case. Infer a new key unless the user names an existing flag.

The flag is auto-created — do NOT create one separately. The one exception is device-id bucketing, which `experiment-create` cannot set (see the recipe in `configuring-experiment-rollout`).

• **Type** — leave empty (will internally default to `"product"`. The `"web"` value is reserved for no-code experiments configured visually with the PostHog

toolbar in a browser; it cannot be meaningfully driven via MCP. If a user asks for a no-code/toolbar experiment, point them to the PostHog UI instead of creating one here.)

If the user gives enough context to infer these, don't ask — just proceed.

Step 2: Who sees what variant?

This is about rollout configuration.

**Before asking any rollout question, load `configuring-experiment-rollout`.** The disambiguation wording, recommendations, and post-answer branches live there — do not formulate rollout questions yourself, and do not assume an example you remember covers the user's path.

Key decision points (covered in detail by `configuring-experiment-rollout`):

• Variant split (how many variants, what percentage each)
• Overall rollout percentage (what % of all users enter the experiment)
• Bucketing and whether to persist the flag across authentication steps — the user chooses, because the project's facts cannot decide it

If the user doesn't mention rollout specifics, use defaults: 50/50 control/test, 100% rollout.

Step 3: How to measu