/agent-experience

name: agent-experience
description: "Audit the developer experience of a product, SDK, docs site, or SKILL.md by dropping multiple Claude subagents at it with only a tiny task prompt and real tools (WebFetch, Bash, Write). Agents must discover the docs themselves, install deps, ask for credentials if needed, and attempt real execution. The skill captures each agent's trace — tool calls, retries, wall time, errors — and scores on Setup Friction, Speed, Efficiency, Error Recovery, and Doc Quality, then emits an HTML report with an A–F grade and concrete fixes. Use when the user asks to audit agent experience, test a skill, audit docs for agents, check if a SDK is agent-friendly, validate a SKILL.md, measure agent DX, or benchmark how painful onboarding is for an AI agent. Triggers: 'audit agent experience', 'test this skill', 'audit docs for agents', 'is my SDK agent-friendly', 'run a DX audit', 'agent experience test', 'test my docs', 'how do agents do with my product'."
compatibility: "Requires an agent harness with subagent (Agent tool), WebFetch, and Bash support, plus network access. No CLI install needed; audited products may require their own credentials at runtime."
license: MIT
metadata:
  author: jay-sahnan
  version: "1.4.0"
allowed-tools: Read WebFetch Write Bash AskUserQuestion Agent

Audit Agent Experience

Evaluate how well a product/SDK/docs surface works when an AI agent actually tries to onboard and do a realistic task — **starting from a short one-sentence prompt**, with nothing pasted in. The agent must find the docs, install what it needs, and attempt real work. That's the only honest test of agent DX.

The skill spawns multiple subagents in parallel, captures each one's tool-call trace, and scores the experience using the same dimensions as the Skill Test Arena dashboard: Setup Friction, Speed, Efficiency, Error Recovery, Doc Quality.

Core principle

**Do not spoonfeed.** The subagent gets a tiny prompt like *"Get started with {product} and {do its primary thing}"*. It must discover the docs, choose the path, and hit real failures. A good doc survives this; a bad doc does not.

Workflow

Execute these steps in order. Do not skip ahead.

Step 1 — Identify the target and define the abstract goal

Resolve what the user is asking to audit. The target may arrive in one of three forms:

• **URL** — a docs site or product page (e.g., `https://docs.example.com`). This is the *seed* the subagents start from.
• **Repo / file path** — for SKILL.md audits or SDK repos.
• **Product name** — if the user is vague ("test my product"), ask via `AskUserQuestion` for the URL or repo.

**This skill is product-agnostic. Never assume what the user wants to audit.** Do not infer a target from environment signals (operator's email domain, git remote, repo name, recent files, memory, CLAUDE.md). Even if context strongly suggests a particular company, the user-facing question must NOT pre-fill or default to any specific product, URL, or company name. Ask open-endedly with neutral options only: e.g., "Paste a URL", "Paste a local path", "Type a product name". If the user did not name a target in their invocation, ask them — start fresh, no priors.

**Research lightly** *after* the user has named a target. 1 WebFetch max, enough to confirm: what is this product, and does it have a getting-started guide? You're identifying *that there is a flow to follow*, not extracting the steps. The whole point is to let the docs dictate the path.

**Define ONE abstract goal, not a step-by-step checklist.** The goal should be at the level of "complete the onboarding" or "make the product do its primary thing once" — NOT a list of specific actions.

Why: prescriptive checklists steer agents. If you tell them "navigate to example.com" but the docs' quickstart navigates to a different URL, the agent is torn between your instruction and the docs. That pollutes the test.

Examples of good abstract goals (the target product is supplied by the user — the examples below are illustrative only, not defaults):

• A search API → *"Complete the getting-started guide. Success = your code successfully calls the API and prints whatever the docs treat as a meaningful result."*
• A payments API → *"Complete the getting-started flow for making a test charge. Success = you have a charge ID or equivalent confirmation."*
• A browser-automation SDK → *"Complete the getting-started guide end-to-end. Success = you have code that runs a cloud browser session using whatever approach the docs recommend."*
• A SKILL.md → *"Follow the skill's instructions and produce a successful outcome for its advertised job."*

Examples of BAD goals (too prescriptive — don't do this):

• ~~"Navigate to https://example.com"~~ (steers — the docs may pick a different URL)
• ~~"Use Playwright"~~ (the docs may recommend Stagehand or Selenium)
• ~~"Print the page title"~~ (the docs may print session ID, response body, anything)

The subagent will self-report against the abstract goal: *did I complete the onboarding as the docs described?* (yes / no / partial). The concrete sub-outcomes the agent *actually achieved* live in their trace under `primary_outcome_achieved`, not in a pre-defined checklist.

If the target has no clear getting-started flow (rare — even a README is a flow), ask the user what "done" means before continuing.

Step 2 — Gather audit config via AskUserQuestion

Use `AskUserQuestion` in a **single call with 4 questions**. Options: max 4 per question.

1. **Test depth** (single-select, header: `"Depth"`):

• `5 agents (Recommended)` — balanced coverage
• `3 agents` — quick sanity check
• `10 agents` — thorough, higher cost

2. **Programming languages** (multiSelect, header: `"Languages"`): pick up to 4 — `Python`, `TypeScript`, `Go`, `Shell/Bash` (let user deselect).

3. **Personas** (multiSelect, header: `"Personas"`):

• `Standard (Recommended)` — neutral baseline, no behavioral flavoring. Just "do the task.