/agentation-self-driving

name: agentation-self-driving
description: Autonomous design critique mode using the Agentation annotation toolbar. Use when the user asks to "critique this page," "add design annotations," "review the UI," "self-driving mode," "auto-annotate," or wants an AI agent to autonomously add design feedback annotations to a web page via the browser. Requires the Agentation toolbar to be installed on the target page and agent-browser skill to be available.
allowed-tools: Bash(agent-browser:*)

Agentation Self-Driving Mode

Autonomously critique a web page by adding design annotations via the Agentation toolbar — in a visible headed browser so the user can watch the agent work in real time, like watching a self-driving car navigate.

Launch — Always Headed

The browser MUST be visible. Never run headless. The user watches you scan, hover, click, and annotate.

**Preflight**: Verify `agent-browser` is available before anything else:

command -v agent-browser >/dev/null || { echo "ERROR: agent-browser not found. Install the agent-browser skill first."; exit 1; }

**Launch**: Try opening directly first. Only close an existing session if the open command fails with a stale session error — this avoids killing a browser someone else is using:

# Try to open. If it fails (stale session), close first then retry.
agent-browser --headed open <url> 2>&1 || { agent-browser close 2>/dev/null; agent-browser --headed open <url>; }

Then verify the Agentation toolbar is present and expand it:

# 1. Check toolbar exists on the page. Agentation 3.1+ renders inside a shadow root on <agentation-toolbar>,
#    so query through it (the fallback to document keeps older versions working)
agent-browser eval "(document.querySelector('agentation-toolbar')?.shadowRoot || document).querySelector('[data-feedback-toolbar]') ? 'toolbar found' : 'NOT FOUND'"
# If "NOT FOUND": Agentation is not installed on this page — stop and tell the user

# 2. Expand ONLY if collapsed (clicking when already expanded collapses it)
agent-browser eval "(document.querySelector('agentation-toolbar')?.shadowRoot || document).querySelector('[data-feedback-toolbar][class*=expanded]') ? 'already expanded' : ((document.querySelector('agentation-toolbar')?.shadowRoot || document).querySelector('[class*=toggleContent]')?.click(), 'expanding')"

# 3. Verify: take a snapshot and look for toolbar controls
agent-browser snapshot -i
# If expanded: you'll see "Block page interactions" checkbox, color buttons (Purple, Blue, etc.)
# If collapsed: you'll only see the small toggle button — retry step 2

"Block page interactions" must be checked (default: on).

> **eval quoting rule**: Always use `[class*=toggleContent]` (no quotes around the attribute value) in eval strings. Do not use double-bang in eval because bash treats it as history expansion. Do not use backslash-escaped inner quotes either, as they break unpredictably across shells.

Critical: How to Create Annotations

**Standard element clicks (`click @ref`) do NOT trigger annotation dialogs.** The Agentation overlay intercepts pointer events at the coordinate level. Use coordinate-based mouse events — this also makes the interaction visible in the browser as the cursor moves across the page.

> **`@ref` compatibility**: Only `click`, `fill`, `type`, `hover`, `focus`, `check`, `select`, `drag` support `@ref` syntax. The commands `scrollintoview`, `get box`, and `eval` do NOT — they expect CSS selectors. Use `eval` with `querySelector` for scrolling and position lookup.

# 1. Take interactive snapshot — identify target element and build a CSS selector
agent-browser snapshot -i
# Example: snapshot shows  heading "Point at bugs." [ref=e10]
# Derive a CSS selector: 'h1', or more specific: 'h1:first-of-type'

# 2. Scroll the element into view via eval (NOT scrollintoview @ref — that breaks)
agent-browser eval "document.querySelector('h1').scrollIntoView({block:'center'})"

# 3. Get its bounding box via eval (NOT get box @ref — that also breaks)
agent-browser eval "((r) => r.x+','+r.y+','+r.width+','+r.height)(document.querySelector('h1').getBoundingClientRect())"
# Returns: "383,245,200,40"  (parse these as x,y,width,height)

# 4. Move cursor to element center, then click
#    centerX = x + width/2,  centerY = y + height/2
agent-browser mouse move <centerX> <centerY>
agent-browser mouse down left
agent-browser mouse up left

# 5. Get the annotation dialog refs — read the FULL snapshot output
#    Dialog refs appear at the BOTTOM of the list, don't truncate with head/tail
agent-browser snapshot -i
# Look for: textbox "What should change?" and "Cancel" / "Add" buttons

# 6. Type critique — fill and click DO support @ref
agent-browser fill @<textboxRef> "Your critique here"

# 7. Submit (Add button enables after text is filled)
agent-browser click @<addRef>

If no dialog appears after clicking, the toolbar may have collapsed. Re-expand (only if collapsed) and retry:

agent-browser eval "(document.querySelector('agentation-toolbar')?.shadowRoot || document).querySelector('[data-feedback-toolbar][class*=expanded]') ? 'ok' : ((document.querySelector('agentation-toolbar')?.shadowRoot || document).querySelector('[class*=toggleContent]')?.click(), 'expanded')"

Building CSS selectors from snapshots

The snapshot shows element roles, names, and refs. Map them to CSS selectors:

| Snapshot line | CSS selector | |--------------|-------------| | `heading "Point at bugs." [ref=e10]` | `h1` or `h1:first-of-type` | | `button "npm install agentation Copy" [ref=e15]` | `button:has(code)` or by text content via eval | | `link "Star on GitHub" [ref=e28]` | `a[href*=github]` | | `paragraph (long text...) [ref=e20]` | Target by section: `section:nth-of-type(2) p` |

When in doubt, use a broader selector and verify with eval:

agent-browser eval "document.querySelector('h2').textContent"

The Loop

Work top-to-bottom through the page. For