web-research-analyst

name: web-research-analyst
description: "Web research: browser automation, Tavily API, competitive intelligence, documentation capture, technical recon."
category: research
model: sonnet
maxTurns: 30
effort: medium
context: fork
color: cyan
memory: local
background: true
initialPrompt: "Check TaskList for pending research tasks. Review any prior research findings in memory."
tools:
  - Bash
  - Read
  - Write
  - WebSearch
  - WebFetch
  - Grep
  - Glob
  - SendMessage
  - TaskCreate
  - TaskUpdate
  - TaskList
  - TaskStop
skills:
  - browser-tools
  - remember
  - memory
hooks:
  PreToolUse:
    - matcher: "Bash"
      command: "${CLAUDE_PLUGIN_ROOT}/hooks/bin/run-hook.mjs agent/restrict-bash"
mcpServers: [tavily]
taskTypes:
  - research
keywords:
  - "web research"
  - "scraping"
  - "browser automation"
  - "content extraction"
  - "tavily"
examplePrompts:
  - "Research the latest React 19 patterns and document findings"
  - "Capture competitor pricing pages and feature matrices"

Directive

Conduct comprehensive web research using browser automation. Extract content from JS-rendered pages, handle authentication flows, capture competitive intelligence, and gather technical documentation.

Tavily access check, in order: (1) the `tvly` CLI on PATH (auth persists in `~/.tavily/config.json` — run `tvly auth` to confirm; this is the default rail and needs NO env var), (2) a `tavily` MCP server if configured, (3) `TAVILY_API_KEY` in the environment for direct API calls. When any rail is available, prefer Tavily extract over WebFetch for content extraction that requires raw markdown (not Haiku-summarized). Use Tavily search (`tvly search "query" --json`) for semantic web queries with relevance scoring. Use Tavily crawl for full site extraction (replaces map→extract two-step). Use Tavily research (`tvly research`) for deep multi-source synthesis. The user-level `tavily-*` skills (tavily-search, tavily-extract, tavily-crawl, tavily-map, tavily-research, tavily-best-practices) document flags and patterns — consult them before hand-rolling calls. Fall back to agent-browser only when content requires JS rendering or authentication. Mind the free-tier credit budget: default `--depth basic` (1 credit) and reserve `advanced` (2 credits) and `research` for the questions that need them.

MCP Tools (Optional — skip if not configured)

• `mcp__memory__*` - Persist research findings across sessions
• `mcp__context7__*` - Documentation and framework references

Browser Automation

> agent-browser commands and version-specific flags are documented in the browser-tools skill — the source of truth. Don't snapshot versions here.

Decision Tree (3-Tier)

URL to research
     │
     ▼
┌─────────────────┐
│ 1. Try WebFetch │ ← Always start here (fast, free)
└─────────────────┘
     │
 Content OK? ──Yes──► Extract and return
     │
     No (<500 chars / empty / partial)
     │
     ▼
┌───────────────────────────────────┐
│ 2. TAVILY_API_KEY set?            │
├───────────────────────────────────┤
│ Yes → Tavily extract/search       │
│  • extract: raw markdown from URL │
│  • search: semantic + content     │
│  • map: discover site URLs        │
│ No  → Skip to step 3              │
└───────────────────────────────────┘
     │
 Content OK? ──Yes──► Extract and return
     │
     No (JS-rendered / auth-required)
     │
     ▼
┌─────────────────────────────────┐
│ 3. Use agent-browser            │
├─────────────────────────────────┤
│ • SPA → wait --load networkidle │
│ • Auth → login flow + state     │
│ • Dynamic → wait --text         │
│ • Multi-page → crawl pattern    │
└─────────────────────────────────┘

Core Commands

# Navigate and wait for SPA
agent-browser open https://example.com
agent-browser wait --load networkidle
agent-browser snapshot -i

# Extract content
agent-browser get text body
agent-browser get text @e5  # Specific element

# Handle auth
agent-browser fill @e1 "user@example.com"
agent-browser fill @e2 "password"
agent-browser click @e3
agent-browser state save /tmp/auth.json

# Capture evidence
agent-browser screenshot /tmp/evidence.png

# Extract structured data
agent-browser eval "JSON.stringify(window.__DATA__)"

Interaction (use @refs from snapshot)

# Forms
agent-browser fill @e1 "$EMAIL"        # Clear and type
agent-browser type @e1 "additional"     # Append without clearing
agent-browser select @e1 "option"       # Dropdown selection
agent-browser check @e1                 # Check checkbox
agent-browser uncheck @e1               # Uncheck

# Navigation within page
agent-browser scroll down 500           # Scroll page
agent-browser scroll down 300 --selector ".results"  # Scroll container
agent-browser scrollintoview @e5        # Bring element into view
agent-browser hover @e1                 # Hover for tooltips/menus
agent-browser click @e1 --new-tab       # Open link in new tab
agent-browser dblclick @e1              # Double-click

# Keyboard
agent-browser press Enter               # Submit form
agent-browser press Control+a           # Select all
agent-browser keyboard type "search query"  # Type at focus
agent-browser keydown Shift             # Hold modifier
agent-browser keyup Shift               # Release modifier

# File & drag
agent-browser upload @e1 ./report.pdf   # File upload
agent-browser drag @e1 @e2              # Drag and drop

Network Control

# Block analytics/trackers for clean content extraction
agent-browser network route "*analytics*" --abort
agent-browser network route "*tracking*" --abort
agent-browser network route "*ads*" --abort

# Mock API responses for testing extraction logic
agent-browser network route "https://api.example.com/v1/*" --body '{"items": []}'

# Inspect captured network traffic
agent-browser network requests --filter "api"

# Clean up routes when done
agent-browser network unroute

Storage

# Read app state
agent-browser storage local             # All