/managing-experiment-lifecycle

name: managing-experiment-lifecycle
description: "Guides experiment state transitions: launching, pausing, resuming, freezing/unfreezing exposure, ending, shipping variants, archiving, resetting, duplicating, and copying to another project. Covers preconditions, implications for variant assignment and analysis, and the decision framework for when to use each action.\nTRIGGER when: user asks to launch, pause, resume, end, ship, archive, reset, duplicate, or copy an experiment to another project, or to freeze/unfreeze exposure (stop enrolling new users while metrics keep flowing, or reopen enrollment).\nDO NOT TRIGGER when: user is creating an experiment (use creating-experiments), configuring rollout (use configuring-experiment-rollout), or setting up metrics (use configuring-experiment-analytics)."

Managing experiment lifecycle

This skill covers experiment state transitions — what each action does, when to use it, and how it affects variant assignment and analysis.

State diagram

draft ──launch──▶ running ──end──▶ stopped ──archive──▶ archived
                  │ │   ▲              │
                  │ pause resume  ship_variant
                  │ │   │         (also ends if running)
                  │ ▼   │
                  │ paused (flag inactive, still "running" status)
                  │
                  ├─freeze_exposure──▶ exposure_frozen ──unfreeze_exposure──▶ running
                  │                    (enrollment closed, metrics keep flowing)

Any non-draft state ──reset──▶ draft

Actions and their implications

For each action, the two key questions:

1. **Who sees what variant?** (user perspective) 2. **Who is in my analysis?** (statistical perspective)

Launch (`experiment-launch`)

Transitions draft → running. Activates the feature flag and sets `start_date`.

• **Preconditions**: must be in draft, flag needs 2-20 multivariate variants (no specific key required; the baseline defaults to "control" when present, else the first variant)
• **Pre-launch checklist**: has at least one metric? Variants correct? Flag implemented in code?
• **Variants**: users start being bucketed into variants based on the configured split
• **Analysis**: data collection begins from `start_date`

No request body needed.

One optional item worth a single mention at launch, when the change is user-facing and substantial: a short survey, shown when users finish the experimented flow (e.g. triggered by the form's submit event), collects qualitative feedback (a rating, an optional comment) alongside the metrics, from day one. Offer it once as setup advice, drop it if declined, and never let it delay the launch. Do **not** raise it at end or ship-variant time — there it reads as a gate on rolling out. → See [`references/qualitative-feedback.md`](../diagnosing-experiment-results/references/qualitative-feedback.md) in [[diagnosing-experiment-results]]

Pause (`experiment-pause`)

Deactivates the feature flag. Users fall back to the default experience (typically control).

• **Preconditions**: must be running and not already paused
• **Variants**: flag is not returned by `/decide` — no new exposure events recorded
• **Analysis**: no new data while paused, but existing data is preserved. Experiment stays "running".

No request body. Use `experiment-resume` to reactivate.

Resume (`experiment-resume`)

Reactivates the feature flag after a pause. Users are re-bucketed deterministically into the same variants.

• **Preconditions**: must be paused
• **Variants**: same assignment as before pause — deterministic bucketing
• **Analysis**: exposure tracking resumes

No request body.

Freeze exposure (`experiment-freeze-exposure`)

Stops enrolling **new** users while everything else keeps going: already-enrolled users keep their variant, metrics keep flowing, and `end_date` stays null. Snapshots the already-exposed users into a static cohort and narrows every release condition on the feature flag to that cohort. Status becomes `exposure_frozen`.

Use for long-horizon metrics (revenue, LTV, retention, renewals) when the sample is big enough and you want to stop adding users without stopping measurement. Neither end nor pause fits that job: end stops measurement at `end_date`, and pause deactivates the flag for everyone.

• **Preconditions**: must be running (not draft, stopped, paused, or already frozen), flag linked and not deleted, at least one release condition
• **Variants**: enrolled users keep their variant (deterministic bucketing); new users no longer match the flag
• **Analysis**: exposures stop growing (a flat exposure curve is expected), but metric data keeps accumulating for enrolled users

**Timing**: the exposure scan and cohort snapshot run synchronously inside the API call, and duration scales with the number of exposed persons — an experiment with tens of thousands of exposed users can take on the order of tens of seconds. Set expectations with the user, wait for the response, and don't treat a slow call as a failure or retry it.

**Not applicable (400) for**:

• **Group-aggregated experiments** — the flag targets groups, not persons, and a person cohort can't freeze group-based matching
• **Experiments in a holdout** — holdout assignment is evaluated before release conditions, so new users would keep entering the holdout
• **Flags with early access conditions** — also evaluated before release conditions, so freezing can't stop new enrollment
• **Mostly-anonymous exposure** (e.g. experiments on logged-out surfaces) — anonymous "personless" users can never match a person cohort and would silently lose their variant, so freezes with more than a small unresolved share are rejected
• **Very large exposed sets** — the exposure scan is bounded by a person cap and a timeout; over either bound the API returns a clean 400 rather than freezing

When a freeze is rejected, explain which limitation applies rather than retrying — these are structural, not transi