/test-mcp-server

name: test-mcp-server
description: Use when testing MCP servers -- e2e tests with the sunpeak inspector, visual regression testing, live testing against real ChatGPT, multi-model evals, Playwright configuration, or scaffolding test infrastructure with "sunpeak test init". Works with any MCP server (Python, Go, TypeScript, etc.), not just sunpeak projects.

Test MCP Server

sunpeak includes a testing framework that works with any MCP server, regardless of language or framework. It provides four testing layers: e2e tests (inspector-based), visual regression, live tests (against real ChatGPT), and evals (multi-model tool calling).

For sunpeak app projects, testing integrates automatically. For non-sunpeak MCP servers (Python, Go, Rust, etc.), `sunpeak test init` scaffolds a self-contained test directory.

Getting Started

sunpeak test init                        # Interactive setup (detects project type)
sunpeak test init --server http://localhost:8000/mcp  # URL-based server
sunpeak test init --server "python server.py"         # Command-based server

`sunpeak test init` detects three project types:

• **sunpeak projects** -- Adds `defineConfig()` and eval boilerplate
• **JS/TS projects** -- Adds Playwright config, smoke test, and evals at project root
• **Non-JS projects** -- Creates a self-contained `tests/sunpeak/` directory with its own `package.json`

Getting Reference Code

Clone the sunpeak repo for working test examples:

git clone --depth 1 https://github.com/Alignbase/sunpeak /tmp/sunpeak

Test examples live at `/tmp/sunpeak/packages/sunpeak/template/tests/`. This includes e2e tests, simulations, evals, and live tests.

Commands

sunpeak inspect              # Inspect any MCP server in the inspector (standalone)
sunpeak test                 # Run unit (if configured) + e2e tests
sunpeak test --e2e           # Run e2e tests only (Playwright)
sunpeak test --visual        # Run e2e tests with visual regression comparison
sunpeak test --visual --update  # Update visual regression baselines
sunpeak test init            # Scaffold test infrastructure into a project
sunpeak test --live          # Run live tests against real ChatGPT (requires tunnel + browser session)
sunpeak test --eval          # Run evals against multiple LLM models (requires API keys)
sunpeak test --unit          # Run unit tests (sunpeak app framework only, not standalone)

Flags are additive: `--e2e --live --eval` runs all three. `--update` implies `--visual`. `--eval` and `--live` are never included in the default run (they cost money). `--unit` is only for sunpeak app framework projects (created with `sunpeak new`), not standalone testing.

E2E Tests with the `mcp` and `inspector` Fixtures

Import `test` and `expect` from `sunpeak/test`. The `mcp` fixture provides protocol-level methods, and the `inspector` fixture handles rendering, double-iframe traversal, URL construction, and host selection. Tests run automatically across ChatGPT and Claude hosts via Playwright projects.

import { test, expect } from 'sunpeak/test';

test('renders weather card', async ({ inspector }) => {
  const result = await inspector.renderTool('show-weather');
  const app = result.app();
  await expect(app.locator('h1')).toHaveText('Austin');
});

test('renders in dark mode', async ({ inspector }) => {
  const result = await inspector.renderTool('show-weather', {}, { theme: 'dark' });
  const app = result.app();
  await expect(app.locator('h1')).toBeVisible();
});

test('loads without console errors', async ({ inspector }) => {
  const errors: string[] = [];
  inspector.page.on('console', (msg) => {
    if (msg.type() === 'error') errors.push(msg.text());
  });

  const result = await inspector.renderTool('show-weather', {}, { theme: 'dark' });
  const app = result.app();
  await expect(app.locator('h1')).toBeVisible();

  const unexpectedErrors = errors.filter(
    (e) =>
      !e.includes('[IframeResource]') &&
      !e.includes('mcp') &&
      !e.includes('PostMessage') &&
      !e.includes('connect')
  );
  expect(unexpectedErrors).toHaveLength(0);
});

test('pip mode (skip on Claude)', async ({ inspector }) => {
  test.skip(inspector.host === 'claude', 'Claude does not support PiP');
  const result = await inspector.renderTool('show-weather', {}, { displayMode: 'pip' });
  await expect(result.app().locator('h1')).toBeVisible({ timeout: 5000 });
});

`mcp` Fixture API

Protocol-level methods (raw MCP data, no rendering):

| Method | Description | |--------|-------------| | `listTools()` | List all tools from the server. Returns `Tool[]`. | | `callTool(name, input?)` | Call a tool, return the raw MCP result. No rendering. | | `listResources()` | List all resources from the server. Returns `Resource[]`. | | `readResource(uri)` | Read a resource by URI. Returns the content string. |

`inspector` Fixture API

Rendering methods and properties:

| Method | Description | |--------|-------------| | `renderTool(name, input?, options?)` | Render a tool result in the inspector, return `InspectorResult` |

| Property | Description | |----------|-------------| | `page` | Raw Playwright `Page` for chrome-level assertions | | `host` | Current host ID (`'chatgpt'` or `'claude'`) from Playwright project |

`InspectorResult` API

| Property/Method | Description | |--------|-------------| | `app()` | Get FrameLocator for rendered resource UI (handles double-iframe) | | `content` | Raw MCP content items | | `structuredContent` | Structured content from tool response | | `isError` | Whether the tool returned an error | | `source` | Where the data came from: `'fixture'` or `'server'` | | `screenshot(name?, options?)` | Take a screenshot and compare against a baseline (only runs with `--visual`) |

MCP-Native Matchers

| Matcher | Description | |---------|-------------| | `expect(result).toBeError()` | Assert tool result is an error | | `expect(result).toHaveTextCon