/algo-ad-ctr

name: "\"algo-ad-ctr\""
description: "\"Build CTR prediction models for estimating ad click-through rates from features. Use this skill when the user needs to predict click probability, build an ad ranking model, or evaluate ad creative performance — even if they say 'predict click rate', 'ad relevance scoring', or 'which ad will get more clicks'.\"."
allowed-tools: Read, Glob, Grep

CTR Prediction Model

Overview

CTR prediction estimates the probability that a user clicks on an ad given context (user, query, ad, position). Forms the core of ad ranking: AdRank = Bid × pCTR. Typically uses logistic regression or gradient-boosted trees. Training on billions of impressions.

When to Use

**Trigger conditions:**

• Building or improving an ad ranking system
• Predicting click probability for bid optimization
• Evaluating ad creative effectiveness from feature analysis

**When NOT to use:**

• When predicting post-click conversions (use conversion rate model)
• When setting bid amounts (use bidding strategy skill)

Algorithm

IRON LAW: A CTR Model Must Be CALIBRATED
Predicting relative ranking is insufficient. The predicted probability
must MATCH actual click frequency (e.g., predicted 5% → 5 clicks per
100 impressions). Without calibration, bid optimization breaks:
  Expected Value = Bid × pCTR × pConversion
  If pCTR is off by 2x, bids are wrong by 2x.

Phase 1: Input Validation

Collect impression logs with: user features, ad features, query features, position, click label (0/1). Handle class imbalance (CTR typically 1-5%). **Gate:** Sufficient volume (100K+ impressions), click labels verified, no data leakage from position.

Phase 2: Core Algorithm

1. Feature engineering: user demographics, ad category, query-ad match, historical CTR, time/device features 2. Train model: logistic regression (interpretable) or GBDT (higher accuracy) 3. Calibrate predictions: Platt scaling or isotonic regression on holdout set 4. Evaluate: log-loss (calibration) + AUC (ranking quality)

Phase 3: Verification

Check calibration: bucket predictions into deciles, compare predicted vs actual CTR per bucket. Plot reliability diagram. **Gate:** Calibration curve close to diagonal, AUC > 0.70.

Phase 4: Output

Return predicted CTR with confidence interval and top contributing features.

Output Format

{
  "prediction": {"ctr": 0.035, "confidence_interval": [0.028, 0.042]},
  "top_features": [{"feature": "query_ad_match", "importance": 0.32}],
  "metadata": {"model": "gbdt", "auc": 0.78, "log_loss": 0.21, "calibration_error": 0.008}
}

Examples

Sample I/O

**Input:** Trained logistic regression with 3 features and these coefficients:

intercept: -3.0
position_1:  0.8
query_ad_match: 1.5
user_is_mobile: 0.3

Features for current request: position_1=1, query_ad_match=1, user_is_mobile=1

**Expected:** logit = -3.0 + 0.8 + 1.5 + 0.3 = -0.4 pCTR = sigmoid(-0.4) = 1/(1 + e^0.4) ≈ 0.401 → **40.1%**

Verify: for features all 0 (baseline), pCTR = sigmoid(-3.0) ≈ 0.047 (4.7%). Calibration is checked by bucketing predictions and comparing to actual CTR in each bucket.

Edge Cases

| Input | Expected | Why | |-------|----------|-----| | New ad, no history | Use ad category average | Cold start for features | | Position 1 vs position 4 | Different CTR, same relevance | Position bias inflates top-slot CTR | | Very rare query | Low confidence | Insufficient training data for that query |

Gotchas

• **Position bias**: Ads in position 1 get more clicks regardless of relevance. Train on position-debiased data or include position as a feature and normalize at inference.
• **Data freshness**: CTR patterns change rapidly (seasonality, trends). Retrain daily or use online learning.
• **Feature leakage**: Including click-derived features (e.g., historical CTR of this exact ad-query pair) creates leakage if not handled carefully with time-based splits.
• **Class imbalance**: 97% no-click, 3% click. Use proper evaluation metrics (log-loss, AUC), not accuracy. Consider downsampling negatives during training.
• **Multi-task learning**: CTR and conversion rate are related but different. Joint models can improve both by sharing lower layers.

References

• For feature engineering best practices, see `references/feature-engineering.md`
• For position debiasing techniques, see `references/position-debiasing.md`