/mastra-smoke-test

name: mastra-smoke-test
description: Smoke test Mastra projects locally or deploy to staging/production. Tests Studio UI, agents, tools, workflows, traces, memory, and more. Supports both local development and cloud deployments.

Mastra Smoke Test

Comprehensive smoke testing for Mastra projects.

Release smoke workflows

Use progressive disclosure: stay in this file until the workflow branches, then read only the reference for the branch you are on. `references/release-smoke.md` is a short index if you need the full map.

Alpha release branch point

Before alpha smoke testing, identify the alpha versioning PR state. Prefer the standard Changesets release branch:

gh pr view changeset-release/main \
  --json number,title,state,url,headRefName,baseRefName,isDraft,mergeable,reviewDecision,updatedAt,mergedAt,mergeCommit

Expected shape:

title: chore: version packages (alpha)
head: changeset-release/main
base: main

If that branch lookup fails, search open and recently merged PRs:

gh pr list --state open --search 'version packages alpha in:title' --limit 20
gh pr list --state merged --search 'version packages alpha in:title' --limit 20

Then branch:

• If the versioning PR is **open**, read `references/alpha-versioning-pr.md`.
• If the versioning PR is **merged**, read `references/alpha-publish.md`.
• If no versioning PR exists, report that and wait for the scheduled alpha versioning flow or user direction.

Do not create the alpha smoke-test project until the automatic alpha publish workflow has completed and the intended packages are installable.

Stable release branch point

If the user is running the stable/full release workflow, read `references/stable-release-smoke.md`. If that workflow fails after some packages publish, switch to `references/stable-partial-publish-recovery.md`.

Scope and targeted checks

After the release package is published and before running smoke tests, read `references/release-scope-discovery.md` to identify changed features. Use the default generated project for the baseline checklist, then add targeted checks for changed features the generated project does not exercise.

When scope discovery identifies a branch:

• For general changed-feature coverage, read `references/targeted-feature-smoke.md`.
• For storage/provider schema or migration changes, read `references/storage-provider-migration-smoke.md`.

⚠️ Mandatory Test Checklist

**Use `task_write` to track progress.** Run ALL tests unless `--test` specifies otherwise.

**Do not skip tests unless you hit an actual blocker.** "Seemed complex" or "wasn't sure" are not valid reasons. Attempt everything - only stop a test when you literally cannot proceed. Report what you tried and what blocked you.

| # | Test | Reference | When Required | | --- | ----------------- | --------------------------------- | ---------------------------------- | | 1 | **Setup** | `references/tests/setup.md` | Always | | 2 | **Agents** | `references/tests/agents.md` | `--test agents` or full | | 3 | **Tools** | `references/tests/tools.md` | `--test tools` or full | | 4 | **Workflows** | `references/tests/workflows.md` | `--test workflows` or full | | 5 | **Traces** | `references/tests/traces.md` | `--test traces` or full | | 6 | **Scorers** | `references/tests/scorers.md` | `--test scorers` or full | | 7 | **Memory** | `references/tests/memory.md` | `--test memory` or full | | 8 | **MCP** | `references/tests/mcp.md` | `--test mcp` or full | | 9 | **Errors** | `references/tests/errors.md` | `--test errors` or full | | 10 | **Experiments** | `references/tests/experiments.md` | `--test experiments` or local full | | 11 | **Studio Deploy** | `references/tests/studio.md` | `--test studio` (cloud only) | | 12 | **Server Deploy** | `references/tests/server.md` | `--test server` (cloud only) |

Full staging and production runs skip **Experiments** as not applicable; the companion worker is a local build artifact. A targeted `--test experiments` run therefore requires `--env local`.

Execution Flow

1. **Read the reference file** for each test you're about to run 2. **Execute the steps** in that reference file 3. **Mark the test complete** before moving to the next

Partial Testing (`--test`)

If `--test` is provided:

1. Always run **Setup** (step 1) 2. Run **only** the specified test(s) 3. Skip other tests

Example: `--test agents,traces` → Run steps 1, 2, and 5 only.

Local Studio Browser Smoke

For local release smoke tests, do **both** API/curl checks and a Studio browser pass unless `--skip-browser` is explicitly requested or browser access is genuinely blocked. API checks prove runtime endpoints work; browser checks prove the Playground/Studio UI can load, submit forms, and display results.

Before opening the browser:

1. Confirm the dev server is alive on the expected port:

   curl -s -o /dev/null -w '%{http_code}\n' http://localhost:4111
   lsof -i :4111 || true

2. If the process died, restart it from the generated project and wait for readiness:

   cd "$SMOKE_DIR/smoke-project"
   pnpm run dev > "$SMOKE_DIR/logs/dev-server-browser.log" 2>&1 &

   for i in {1..60}; do
     code=$(curl -s -o /dev/null -w '%{http_code}' http://localhost:4111 || true)
     [ "$code" = 200 ] && break
     sleep 1
   done

3. Use browser tools to navigate to `http://localhost:4111`. If `networkidle` times out but `domcontentloaded` succeeds and the UI is usable, continue and note the timeout.

Recommended browser task list:

1. Verify Studio shell loads
2. Smoke test agent chat UI
3. Smoke test tools UI
4. Smok