/filtering-bot-traffic

name: filtering-bot-traffic
description: 'Identify, measure, and exclude bot / crawler / AI-agent traffic in PostHog web and product analytics using the traffic classification surface (the isLikelyBot / getTrafficType HogQL functions and the $virt_* virtual properties). Use when the user asks to "exclude bots", "filter out crawlers", "remove bot traffic from my numbers", "how much of my traffic is bots / AI crawlers", "is GPTBot / ChatGPT / Claude hitting my site", "break down traffic by human vs bot", or wants clean human-only counts in an insight or dashboard. For the real-time Live tab bot tiles, use exploring-live-traffic instead.'

Filtering and measuring bot traffic

PostHog classifies every request by user agent so you can tell humans apart from bots, crawlers, and AI agents anywhere HogQL runs — the SQL editor, insights, trends, and Web analytics breakdowns. This skill teaches you (the agent) how to use that classification to:

• exclude bots so analytics reflect human traffic only
• measure how much traffic is automated, and which bots / operators are responsible
• separate AI-agent traffic (worth measuring) from noise (worth dropping)
• pick the right surface — virtual properties for the insight builder, functions for raw SQL

For real-time ("right now", last 30 min) bot questions and the Live tab tiles, use the **exploring-live-traffic** skill instead. This skill is for historical windows, saved insights, dashboards, and filtering.

When to use this skill

Use it when the user wants to:

• exclude or filter out bots ("remove bots from my pageviews", "humans only")
• quantify automated traffic ("what % of traffic is bots?", "how much is AI crawlers?")
• find which bots hit them ("which crawlers visit us?", "is ChatGPT reading our docs?")
• break a trend down by traffic type or bot name
• measure AI-agent / AI-search traffic specifically (AEO / answer-engine visibility)

Do **not** use it for the Live tab, real-time numbers, or the per-minute bot charts — that is exploring-live-traffic.

The classification surface

Two equivalent ways to reach the same classification. Prefer **virtual properties** in the insight builder and filters; use **functions** in hand-written SQL or when you need a value the virtual properties don't expose.

Virtual properties (insight builder, filters, breakdowns)

These read the user agent for you (falling back from `$raw_user_agent` to `$user_agent`), so you don't pass anything in. Available wherever you pick an event property.

| Property | Value | | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `$virt_is_bot` | boolean — `true` for bots / crawlers / automation | | `$virt_traffic_type` | `Regular`, `AI Agent`, `Bot`, or `Automation` | | `$virt_traffic_category` | finer category, e.g. `ai_crawler`, `ai_search`, `ai_assistant`, `search_crawler`, `seo_crawler`, `social_crawler`, `monitoring`, `http_client`, `headless_browser`, `no_user_agent`, `regular` | | `$virt_bot_name` | display name, e.g. `Googlebot`, `GPTBot`, `ClaudeBot` | | `$virt_bot_operator` | company behind the bot, e.g. `Google`, `OpenAI`, `Anthropic` |

HogQL functions (raw SQL)

Pass the user agent explicitly. Use `coalesce(nullIf(properties.$raw_user_agent, ''), properties.$user_agent)` to cover both server-side (`$raw_user_agent`) and JS SDK (`$user_agent`) captures. The `nullIf` keeps an empty `$raw_user_agent` from shadowing a real `$user_agent` and being misread as a bot — this mirrors the expression the virtual properties use internally.

| Function | Returns | | ------------------------ | ---------------------------------------------------------------------------- | | `isLikelyBot(ua)` | `true` if the UA matches a bot/automation pattern (empty UA counts as a bot) | | `getTrafficType(ua)` | `AI Agent` / `Bot` / `Automation` / `Regular` | | `getTrafficCategory(ua)` | subcategory; `regular` for humans | | `getBotType(ua)` | same subcategory but empty string for humans — handy for filtering | | `getBotName(ua)` | bot name; empty for humans | | `getBotOperator(ua)` | operator/company; empty for humans |

Traffic types — what to keep vs drop

`getTrafficType` / `$virt_traffic_type` sorts every request into four buckets. The default move differs per bucket — don't treat them all as noise:

| Type | What it is | Default move | | ------------ | --------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | | `Regular` | Human visitors