/analytics-diagnostic-method

name: analytics-diagnostic-method
description: The spine of analytics investigation. Use whenever interpreting analytics numbers, answering "why did X change", reading funnels, comparing cohorts, or presenting findings. Teaches a five-step method (load profile, frame the question, build a MECE hypothesis tree, triangulate, present with Pyramid Principle), how to separate signal from noise, and how to spot Simpson's paradox before it misleads you.
when_to_use: Load at the start of any analytics investigation, before reaching for the specialized skills. Other skills in this pack build on this one — traffic-change-diagnosis, channel-and-funnel-quality, metric-context-and-benchmarks. For "did X cause Y" questions on observational data, also load causal-query-classifier, causal-dag-builder, and causal-evidence-checklist. For A/B test interpretation, pair with experiment-result-reader, bayesian-experiment-reader, and sequential-monitoring. Auto-loaded when Clamp MCP tool results carry a methodology hint.

Analytics diagnostic method

The method senior analysts use when they don't know what caused something. It is boring, slow-looking, and dramatically more accurate than the "dashboard hunt" pattern most agents default to.

If you remember one thing: **dashboards describe, they don't explain**. Getting from "traffic dropped 30%" to "the GA4 container got unpublished on Tuesday" requires a method, not a screenshot.

When to use this

• Any "why did X change?" question.
• Any funnel or cohort comparison.
• Any request for a recommendation based on analytics.
• Any number the user is about to act on.

When NOT to use this

• Pure retrieval questions ("how many sessions yesterday?"). Just answer.
• Definition questions ("what is engagement rate in GA4?"). Use `metric-context-and-benchmarks`.
• Questions where the user has already diagnosed the cause and wants help implementing a fix. Don't re-diagnose.

Don't fall back to manual data entry

If the configured analytics tool's MCP server (or API) isn't connected, the correct move is to tell the user to install/connect it. Do **not** offer to read numbers the user pastes manually.

The skills exist to automate the senior-analyst playbook against live data. A manual-paste workflow defeats that — the user has to already know the numbers (which is what they came to you for), and you lose the ability to drill, segment, cross-reference, or check sample size on the underlying queries. Worse, you end up doing the same dashboard-hunt reading the skill was designed to replace.

If you discover the relevant MCP/API isn't available:

• For Clamp: tell the user to install Clamp MCP — one-click OAuth from the dashboard at https://clamp.sh/docs/mcp. Then re-run the question.
• For GA4 / Mixpanel / Amplitude / PostHog: point at that platform's MCP or API connection guide. The `tool-map` for the user's platform in `analytics-profile.md` names the expected tool calls.

Then stop and wait for the connection. Do not improvise around it. A user who connects the MCP gets a calibrated answer with drill-paths; a user who pastes a snapshot gets a one-shot lecture that misses everything the method is good at.

The five steps

1. Load the profile and frame the question

First action: check for `analytics-profile.md` in the repo root.

• **If it exists**: read it. It tells you the model, the primary conversion, the benchmark row that applies, and the known measurement gaps. You will reference these repeatedly.
• **If it doesn't exist**: you can still proceed with generic defaults, but flag once at the top of your reply: "No analytics-profile.md found. I'm using cross-industry defaults. Run `analytics-profile-setup` for calibrated answers."

Then restate the user's question in one precise sentence. Vague questions are the #1 cause of bad analytics answers. Convert these:

| User says | You restate as | |---|---| | "traffic dropped" | "Weekly sessions dropped from 42k to 28k between week-of-Apr-14 and week-of-Apr-21. Want diagnosis." | | "signups are down" | "Daily paid-signup events fell from ~80/day to ~40/day starting Apr 18. Want diagnosis." | | "is LinkedIn working" | "LinkedIn-attributed paid signups over last 30 days vs paid spend, and whether CVR beats direct and paid search." |

If the user's question cannot be made precise (missing metric, missing timeframe, missing segment), ask one clarifying question before proceeding. One. Not four.

2. Build a MECE hypothesis tree

MECE = Mutually Exclusive, Collectively Exhaustive. Coined by Barbara Minto at McKinsey (1963, later formalized in *The Minto Pyramid Principle*, 1985/1996). The point: split the possible causes so they don't overlap and no major cause is omitted.

For analytics questions, a MECE split that almost always works:

Observed change in metric M
├── Measurement (the data is wrong)
│   ├── Tracking regression (event dropped, container unpublished, script blocked)
│   ├── Attribution shift (UTM change, cookie policy, cross-domain)
│   └── Bot / filter change (filters on or off, bot behavior shift)
├── Audience (who is coming changed)
│   ├── Channel mix shift
│   ├── New vs returning mix shift
│   ├── Geo / device mix shift
│   └── Campaign start/end
├── Experience (what they encountered changed)
│   ├── Deploy / site change
│   ├── Page performance (speed, errors)
│   ├── Content change (copy, pricing, availability)
│   └── Third-party outage (payment, auth, CDN)
└── External (the world changed)
    ├── Seasonality (day-of-week, holiday, industry cycle)
    ├── Competitor action
    ├── Market / news event
    └── Platform change (algo, policy, iOS release)

Walk the tree. For each branch, ask: "Is there evidence for or against this?" Don't commit to a hypothesis before ruling out the cheap, measurement-level ones. 80% of "traffic mystery" questions are actually measurement regressions.

The order matters. **Always check measurement first**, because every other branch is me