/builder-smoke-test

name: builder-smoke-test
description: Smoke test the Agent Builder feature branch end-to-end against a hermetic project scaffolded by the skill (linked to the current worktree). Covers workspace reconciliation, stored agents/skills CRUD, ownership, visibility, stars, registry/library Copy flow, picker allowlists, model policy, RBAC role gating, role impersonation UI, builder defaults, infrastructure diagnostics, channels, and Studio + Agent Builder UI. Trigger when validating the agent-builder feature branch, PRs that touch packages/server, packages/playground, packages/playground-ui agent-builder routes, or builder EE code paths.

Builder Smoke Test

End-to-end smoke testing of the Agent Builder feature set against a hermetic project the skill scaffolds at `~/mastra-builder-smoke-tests/builder-smoke` (configurable). The project links to the current worktree via `pnpm` `link:` overrides, so changes to packages under `packages/`, `stores/`, `auth/`, `channels/`, `observability/`, `browser/`, and `client-sdks/` take effect on the next `mastra dev` restart.

This skill is for **branch QA** — it complements the release-time `mastra-smoke-test`. It exercises the Builder EE surface (stored entities, RBAC, registry, infra, channels) using a minimal, predictable project rather than the kitchen-sink `examples/agent`.

⚠️ Mandatory Test Checklist

**Use `task_write` to track progress.** Run ALL sections unless `--test` or `--scope` narrows the run.

**Do not skip sections unless you hit an actual blocker.** "Seemed complex" or "I'll come back to it" are not valid reasons. Attempt every step — only stop when you literally cannot proceed. Report what you tried and what blocked you.

| # | Section | Reference | When required | | --- | ---------------------- | -------------------------------- | ----------------------------------------------------------------------- | | 1 | **Setup** | `references/setup.md` | Always | | 2 | **Workspace** | `references/workspace.md` | `--test workspace` or full | | 3 | **Reconciliation** | `references/reconciliation.md` | Steps 1 + 5 only; steps 2/3/4/6 are out of smoke-test scope (see below) | | 4 | **Defaults** | `references/defaults.md` | `--test defaults` or full | | 5 | **Model Policy** | `references/model-policy.md` | `--test model-policy` or full | | 6 | **Skills** | `references/skills.md` | `--test skills` or full | | 7 | **Registry** | `references/registry.md` | `--test registry` or full | | 8 | **Agents** | `references/agents.md` | `--test agents` or full | | 9 | **Picker Allowlists** | `references/picker-allowlist.md` | `--test pickers` or full | | 10 | **Favorites** | `references/favorites.md` | `--test favorites` or full (formerly `stars`) | | 11 | **Permissions / RBAC** | `references/permissions.md` | `--test permissions` or full | | 12 | **Infrastructure** | `references/infrastructure.md` | `--test infrastructure` or full | | 13 | **Channels** | `references/channels.md` | `--test channels` or full | | 14 | **UI** | `references/ui.md` | `--test ui` or full | | 15 | **Auth** | `references/auth.md` | `--test auth` or `--auth on` |

Execution flow

1. **Confirm the project directory.** Before scaffolding, ask the user where they want `$PROJECT_DIR` to live. Offer the default (`~/mastra-builder-smoke-tests/builder-smoke`) as a suggestion. Skip the question if they already passed `--dir` or have `$BUILDER_SMOKE_TEST_DIR` exported. See `references/setup.md` step 0. 2. **Read the reference file** for each section you're about to run. 3. **Under `--auth on`, extract the session cookie before running any other section.** The WorkOS cookie is `httpOnly`, so `curl` cannot mint it and `document.cookie` cannot read it. The scaffold ships a debug route at `GET /smoke-test/cookie` gated by `SMOKE_TEST_COOKIE_LEAK=1`. Follow the **"Extracting the session cookie for curl (auth on)"** section below before touching any auth-on endpoint. **Do not pivot to UI-only testing because curl is "blocked" — the cookie route is the unblock path.** 4. **Seed non-owner data after the server has booted at least once.** A fresh scaffold has no skills authored by anyone other than the test user, which makes non-owner / Library Copy / non-owner visibility / non-admin stars flows untestable. Run `bash .claude/skills/builder-smoke-test/scripts/seed-multi-user.sh` (or with `--dir $PROJECT_DIR`) before sections 6 (Skills), 7 (Registry), and 10 (Stars). The script is idempotent and bypasses RBAC by writing directly to libsql, so it works regardless of `--auth` mode or current role. **Do not mark non-owner steps as "blocked" without running this first.** 5. **Execute the steps** — use `curl` for API checks (with `-H "Cookie: $COOKIE"` under `--auth on`), whichever browser tool the harness has wired up (Stagehand, Chrome MCP, etc.) for UI checks. 6. **Record results** in the summary table. 7. **Mark the section complete** with `task_write` before moving to the next.

Partial testing (`--test`)

If `-