/expo-skill-eval

name: expo-skill-eval
description: Evaluate Expo skills in this repo end-to-end - trigger accuracy, generated code quality, and runtime screenshots on iOS simulator and Android emulator via Expo Go (web optional). Use when the user wants to eval an Expo skill, test that a skill produces working code, benchmark a skill with device screenshots, or verify a skill's output renders correctly.
version: 1.0.0
license: MIT
allowed-tools: "Read(~/.cache/expo-skill-eval/**), Read(/tmp/expo-skill-eval-*/**), Read(/private/tmp/expo-skill-eval-*/**), Write(/tmp/expo-skill-eval-*/**), Write(/private/tmp/expo-skill-eval-*/**), Edit(/tmp/expo-skill-eval-*/**), Edit(/private/tmp/expo-skill-eval-*/**), Bash(python3 /tmp/expo-skill-eval-*), Bash(python3 /private/tmp/expo-skill-eval-*), Bash(python3 *expo-skill-eval/scripts/*), Bash(tee /tmp/expo-skill-eval-*), Bash(tee /private/tmp/expo-skill-eval-*), Bash(bash *expo-skill-eval/scripts/*)"

Expo Skill Eval

Evaluates skills in `plugins/expo/skills/` for trigger accuracy, generated code quality, and/or runtime rendering in Expo Go.

Requirements: macOS with Xcode (iOS simulators), Android SDK with at least one AVD, and `bun`. No other device tooling is assumed.

Workspace root: `/private/tmp/expo-skill-eval-<skill-name>/iteration-N/` (e.g. `/private/tmp/expo-skill-eval-expo-ui/iteration-4/`).

Before starting — clarify scope

**Confirm all of the following up front, before any pipeline work — don't skip any** (only skip a given item if the request already states that choice). Batch them into `AskUserQuestion` calls of ≤4 questions each, in this order:

1. **Which skill** to eval (if not clear from the request). 2. **Prompts** — which prompts drive the eval. Built-in prompts (from the skill's eval cases) are **pre-selected all**; drop any, add a custom text prompt, or **build from an uploaded screenshot** (a target UI the skill must reproduce). See **Prompts** below. 3. **What to verify** — one multi-select of three options: Runtime + screenshots / Trigger accuracy / Code checks (no device). See *What to verify* below. 4. **Expo SDK** — latest (default, auto-detected) or a pinned version. 5. **Runner** — Expo Go (default) or development build. 6. **Platforms** — iOS / Android / web (always offer all three). 7. **Permission flag** for `claude -p` — skip-permissions (default) or accept-edits. 8. **Viewer delivery** — local only (default) or publish a shareable Artifact. 9. **If trigger accuracy is selected** — confirm the published `expo` plugin is disabled (or not installed).

Each is detailed below. Items 4–6 (SDK, runner, platforms) fit naturally in one `AskUserQuestion` call.

**If the skill to eval is not clear from the request**, list available skills from `plugins/expo/skills/` and ask which one to evaluate.

**How the skill under test is loaded — two mechanisms, one per phase** (don't pick one globally): executor runs reference it by **file path** (`SKILL_PATH = plugins/expo/skills/<skill>/SKILL.md`, read explicitly), while the trigger eval loads it as a **plugin** (`--plugin-dir plugins/expo`, so the model can auto-select it from its description). Both point at the *local, in-repo* version — that's what you're evaluating. You do **not** need any special flag to *launch* the harness session itself (the harness finds the skill by repo path); the mechanisms apply to the `claude -p` subprocesses it spawns. See steps 1 and 3 for why each phase differs. **One pre-run check (required when the trigger eval is in scope):** if the *published* `expo` plugin is installed/enabled, disable it (via `/plugin`) **before launching** the harness and re-enable after. A single disable is a global-config change that both this session and the spawned `claude -p` subprocesses inherit. Why it's required for the trigger eval: that phase loads the local skill via `--plugin-dir`, and a second installed `expo` collides with it — the model may trigger the *published* `expo:expo-ui`, and since detection only sees the tool-call name you'd silently score the published description instead of your local edits (the collision could also just error). The **executor / runtime / static** phases are *not* affected — they read the skill under test by its local `SKILL_PATH` with no `--plugin-dir` — so a run with no trigger eval can skip the disable. Disabling `expo` does **not** disable `expo-skill-eval` (a standalone project skill, not part of the `expo` plugin), so the harness stays available.

**Surface this to the user as an explicit up-front confirmation** — the same way you confirm which skill to eval. When the trigger eval is in scope, ask the user to confirm the published `expo` plugin is disabled (or not installed) *before* you start step 1; if it's still enabled, pause and have them disable it via `/plugin`. Don't run the trigger eval until they confirm — the harness can't reliably detect installed plugins on its own (reading the global plugin config or `claude plugin list` would prompt), so this is a manual confirmation, not an auto-check.

**Pick the prompts — built-in, custom, or a target screenshot.** The prompts are the *inputs* that drive the executor (with-skill and without-skill); they are separate from what you *verify*. Confirm them with `AskUserQuestion` (skip if the request already names a prompt):

• **Built-in prompts** — representative prompts you generate by reading the skill under test (its `SKILL.md` + `references/`) and `references/runtime-matrix.md`, covering the skill's standard use cases. (If the skill already ships eval cases under `evals/evals.json`, fold their `prompt` fields in too — but most skills don't, so you usually derive them.) **Pre-select all** so the default run exercises the skill's standard cases; let the user deselect any.
• **Custom text prompt** — a one-off prompt the user types. Don't spend a dedicated option slot on this: `AskUserQuestion` auto-adds a **"Type something"** / Other entry, and anything typed there becomes a custom