/product-analytics

name: product-analytics
description: Product analytics and growth expert. Use when designing event tracking, defining metrics, running A/B tests, or analyzing retention. Covers AARRR framework, funnel analysis, cohort analysis, and experimentation.

Product Analytics

Core Principles

• **Metrics over vanity** — Focus on actionable metrics tied to business outcomes
• **Data-driven decisions** — Hypothesize, measure, learn, iterate
• **User-centric measurement** — Track behavior, not just pageviews
• **Statistical rigor** — Understand significance, avoid false positives
• **Privacy-first** — Respect user data, comply with GDPR/CCPA
• **North Star focus** — Align all teams around one key metric

---

Hard Rules (Must Follow)

> These rules are mandatory. Violating them means the skill is not working correctly.

No PII in Events

**Events must NEVER contain personally identifiable information.**

// ❌ FORBIDDEN: PII in event properties
track('user_signed_up', {
  email: 'user@example.com',     // PII!
  name: 'John Doe',              // PII!
  phone: '+1234567890',          // PII!
  ip_address: '192.168.1.1',     // PII!
  credit_card: '4111...',        // NEVER!
});

// ✅ REQUIRED: Anonymized/hashed identifiers only
track('user_signed_up', {
  user_id: hash('user@example.com'),  // Hashed
  plan: 'pro',
  source: 'organic',
  country: 'US',                       // Broad location OK
});

// Masking utilities
const maskEmail = (email) => {
  const [name, domain] = email.split('@');
  return `${name[0]}***@${domain}`;
};

Object_Action Event Naming

**All event names must follow the object_action snake_case format.**

// ❌ FORBIDDEN: Inconsistent naming
track('signup');                    // No object
track('newProject');                // camelCase
track('Upload File');               // Spaces and PascalCase
track('user-created');              // kebab-case
track('BUTTON_CLICKED');            // SCREAMING_CASE

// ✅ REQUIRED: object_action snake_case
track('user_signed_up');
track('project_created');
track('file_uploaded');
track('payment_completed');
track('checkout_started');

Actionable Metrics Only

**Track metrics that drive decisions, not vanity metrics.**

// ❌ FORBIDDEN: Vanity metrics without context
track('page_viewed');               // No insight
track('button_clicked');            // Too generic
track('app_opened');                // Doesn't indicate value

// ✅ REQUIRED: Actionable metrics tied to outcomes
track('feature_activated', {
  feature: 'dark_mode',
  time_to_activation_hours: 2.5,
  user_segment: 'power_user',
});

track('checkout_completed', {
  order_value: 99.99,
  items_count: 3,
  payment_method: 'credit_card',
  coupon_applied: true,
});

Statistical Rigor for Experiments

**A/B tests must have proper sample size and significance thresholds.**

// ❌ FORBIDDEN: Drawing conclusions too early
// "After 100 users, variant B has 5% higher conversion!"
// This is not statistically significant.

// ✅ REQUIRED: Proper experiment setup
const experimentConfig = {
  name: 'new_checkout_flow',
  hypothesis: 'New flow increases conversion by 10%',

  // Statistical requirements
  significance_level: 0.05,      // 95% confidence
  power: 0.80,                   // 80% power
  minimum_detectable_effect: 0.10, // 10% lift

  // Calculated sample size
  sample_size_per_variant: 3842,

  // Guardrails
  max_duration_days: 14,
  stop_if_degradation: -0.05,    // Stop if 5% worse
};

---

Quick Reference

When to Use What

| Scenario | Framework/Tool | Key Metric | |----------|---------------|------------| | Overall product health | North Star Metric | Time spent listening (Spotify), Nights booked (Airbnb) | | Growth optimization | AARRR (Pirate Metrics) | Conversion rates per stage | | Feature validation | A/B Testing | Statistical significance (p < 0.05) | | User engagement | Cohort Analysis | Day 1/7/30 retention rates | | Conversion optimization | Funnel Analysis | Drop-off rates per step | | Feature impact | Attribution Modeling | Multi-touch attribution | | Experiment success | Statistical Testing | Power, significance, effect size |

---

North Star Metric

Definition

A North Star Metric is the **one metric** that best captures the core value your product delivers to customers. When this metric grows sustainably, your business succeeds.

Characteristics of Good NSMs

✓ Captures product value delivery
✓ Correlates with revenue/growth
✓ Measurable and trackable
✓ Movable by product/engineering
✓ Understandable by entire org
✓ Leading (not lagging) indicator

Examples by Company

| Company | North Star Metric | Why It Works | |---------|------------------|--------------| | **Spotify** | Time Spent Listening | Core value = music enjoyment | | **Airbnb** | Nights Booked | Revenue driver + value delivered | | **Slack** | Daily Active Teams | Engagement = product stickiness | | **Facebook** | Monthly Active Users | Network effect foundation | | **Amplitude** | Weekly Learning Users | Value = analytics insights | | **Dropbox** | Active Users Sharing Files | Core product behavior |

NSM Framework

North Star Metric
       ↓
┌──────┴──────┬──────────┬──────────┐
│             │          │          │
Input 1    Input 2   Input 3   Input 4
(Supporting metrics that drive NSM)

Example: Spotify
NSM: Time Spent Listening
├── Daily Active Users
├── Playlists Created
├── Songs Added to Library
└── Share/Social Actions

How to Define Your NSM

1. **Identify core value proposition**

• What job does your product do for users?
• When do users get "aha!" moment?

2. **Find the metric that represents this value**

• Transaction completed? (e.g., Nights Booked)
• Time engaged? (e.g., Time Listening)
• Content created? (e.g., Messages Sent)

3. **Validate it correlates with business success**

• Does NSM increase → revenue increases?