/prd-v08-drift-baseline-compare

name: prd-v08-drift-baseline-compare
description: >
  Establish baseline → snapshot → compare → history monitoring for any KPI, config, or
  metric that can drift during PRD v0.8 Deployment & Ops. Triggers on requests to monitor
  drift, baseline a value for later comparison, or when user asks "how do we track if X
  changes?", "baseline this", "config drift", "performance regression", "metric drift",
  "compare to last week", "is this getting worse?". Outputs MON-DRIFT-* entries with
  baseline + comparison rules.
context: fork
allowed-tools:
  - Read
  - Write
  - Edit
  - Glob
  - Grep
  - Bash

execution_modes:
  default: standard
  supports: [quick, standard, deep]

Drift: Baseline / Compare / History

Position in workflow: v0.8 Monitoring Setup → **v0.8 Drift: Baseline / Compare / History** → v0.8 Runbook Creation

Execution Mode

Default is **standard**. See [`.claude/rules/08-skill-execution-modes.md`](../../rules/08-skill-execution-modes.md) for selection logic.

| Mode | What this skill produces | |------|--------------------------| | **quick** | One metric / config / dataset baselined; weekly compare schedule; simple threshold alert | | **standard** | 3–5 things monitored; baseline + tiered thresholds (warn / critical); compare cadence; history retention | | **deep** | Full portfolio; multi-dimensional comparison (per segment, per environment); regression-cause matrix; auto-baselining after intentional change |

What This Does

Generalizes a pattern AgriciDaniel's `claude-seo` skill encodes for SEO drift — **baseline → snapshot → compare → history** — into a reusable monitoring shape that works for any metric, config, or dataset that can change over time and needs to be watched.

This is **drift monitoring**, distinct from alerting on absolute thresholds. Alerting answers "is X over the line right now?" Drift monitoring answers "is X different from last week's normal?" — which catches slow regressions that absolute thresholds miss.

Examples of things worth drift-monitoring:

• **KPI**: activation rate week-over-week
• **AI search position**: ChatGPT/Perplexity ranking for target queries
• **Config**: feature-flag rollout percentages
• **Performance**: p95 latency by endpoint
• **Cost**: per-user infra cost
• **Marketing**: per-channel CAC trend
• **Content**: changelog post engagement
• **Third-party**: vendor pricing pages (price hikes), competitor feature pages (parity loss)

How It Works

1. **Pick what to monitor** — One thing per MON-DRIFT- entry. Must be:

• Quantifiable (number, percentage, list, configuration value)
• Snapshotable (captureable at a point in time, ideally automatically)
• Causally interpretable (when it changes, you know enough to investigate)

2. **Capture the baseline** — Take a snapshot. Date it. Store in version control or a known location (`status/baselines/`, `monitoring/snapshots/`, etc.). 3. **Define drift thresholds**:

• **Warn**: meaningful change (e.g., 10% drift in a KPI; any change in a config value)
• **Critical**: serious change (e.g., 25% KPI drop; breaking config change)
• **Recalibrate**: intentional change that should refresh the baseline (e.g., after a feature rollout, the baseline is wrong; refresh it)

4. **Set compare cadence** — How often does this get re-snapshotted?

• Hot (hourly/daily): production KPIs, AI search positions during a launch
• Warm (weekly): standard product KPIs, content engagement
• Cool (monthly): vendor pricing, competitor feature parity, infra cost

5. **Build the compare procedure** — A script or runbook that:

• Takes a new snapshot
• Diffs against baseline
• Computes drift % per dimension
• Emits warn/critical signals at thresholds
• Appends to history log

6. **Plan auto-baselining after intentional change** [standard+] — When the team makes a deliberate change (ships a feature that should improve activation), the old baseline becomes wrong. Define what triggers a baseline refresh and who approves it.

Example

Monitoring AI search position for the target query "best CRO tool for SaaS founders" across 3 surfaces. (Generalized from AEO Audit re-test cadence.)

**Baseline** (2026-04-01):

ChatGPT: not mentioned (rank: --)
Perplexity: rank 4 of 5, miscategorized as "analytics"
AI Overviews: not mentioned
Sources cited: [competitor1.com, competitor2.com, reddit.com/r/SaaS]

**Thresholds**:

• Warn: rank changes by ≥1, sources list changes, miscategorization persists
• Critical: dropped from any surface where previously mentioned

**Compare cadence**: weekly during launch month, monthly after.

**Re-snapshot** (2026-04-15):

ChatGPT: rank 3 of 5 (NEW)         ← improved
Perplexity: rank 3 of 5, category corrected (NEW)   ← improved
AI Overviews: rank 5 of 7 (NEW)    ← improved
Sources cited: [+ ourdomain.com (NEW), ...]

**Compare output**: 3 surfaces improved. Likely cause: alternatives-pages campaign + new CRO-anchored blog post indexed.

**History append**: 2026-04-01 → 2026-04-15 row added to history log.

**Action**: No alert; positive drift. Baseline can stay (next compare in 2 weeks). If the trend reverses, the baseline catches it.

What You Get Back

• **MON-DRIFT-\* entries** (one per monitored thing) — Baseline snapshot, thresholds, cadence, compare procedure, history pointer
• **History logs** in `status/drift-history/<id>.jsonl` (or similar) — Append-only record of compare results
• **Compare scripts** (`scripts/drift/<id>.sh` or equivalent) when automatable

When to Use It

| Trigger | Mode | |---------|------| | Post-launch — need ongoing watch on KPIs | standard | | AEO Audit results to maintain | standard | | Competitor feature parity tracking | standard | | Vendor / third-party pricing watch | quick | | Performance regression hunting | deep | | Infrastructure cost trend | quick |

Do not use for things that are already covered by **threshold-based alerting** (RED/USE metrics, MON-* alerts). Drift is for trends; alert