testdriver-agent
How the TestDriver agent behaves on GitHub issues, pull requests, and @mentions
Configure TestDriver sandbox options and environment settings
Claude Code
$ npx -y skills add testdriverai/testdriverai --skill testdriver-customizing-devices --agent claude-code
Codex
$ npx skills add testdriverai/testdriverai --skill testdriver-customizing-devices -a codex -y
Cursor
$ npx skills add testdriverai/testdriverai --skill testdriver-customizing-devices -a cursor -y
Gemini CLI
$ npx skills add testdriverai/testdriverai --skill testdriver-customizing-devices -a gemini-cli -y
GitHub Copilot
$ npx skills add testdriverai/testdriverai --skill testdriver-customizing-devices -a github-copilot -y
OpenCode
$ npx skills add testdriverai/testdriverai --skill testdriver-customizing-devices -a opencode -y
Windsurf
$ npx skills add testdriverai/testdriverai --skill testdriver-customizing-devices -a windsurf -y
name: testdriver:customizing-devices description: Configure TestDriver sandbox options and environment settings
<!-- Generated from customizing-devices.mdx. DO NOT EDIT. -->
Configure the TestDriver behavior with options that you give to the `TestDriver()` function:
const testdriver = TestDriver(context, {
// === Sandbox & Connection ===
newSandbox: true, // Force creation of a new sandbox (default: true)
reconnect: false, // Reconnect to last sandbox (default: false)
keepAlive: 60000, // Keep sandbox alive after disconnect in ms (default: 60000)
os: "linux", // 'linux' | 'windows' (default: 'linux')
resolution: "1366x768", // Sandbox resolution (e.g., '1920x1080')
ip: "203.0.113.42", // Direct IP for self-hosted sandbox
sandboxAmi: "ami-1234", // Custom AMI ID (AWS deployments)
sandboxInstance: "i3.metal", // EC2 instance type (AWS deployments)
// === Preview & Debugging ===
preview: "browser", // "browser" | "ide" | "none" (default: "browser")
headless: false, // @deprecated - use preview: "none" instead
debugOnFailure: false, // Keep sandbox alive on test failure for debugging
// === Caching ===
cache: true, // Enable element caching (default: true)
// Or use advanced caching config:
// cache: {
// enabled: true,
// thresholds: {
// find: { screen: 0.05, element: 0.8 },
// assert: 0.05
// }
// },
cacheKey: "my-test", // Cache key for element finding operations
// === Recording & Screenshots ===
dashcam: true, // Enable/disable Dashcam video recording (default: true)
autoScreenshots: true, // Capture screenshots before/after each command (default: false)
// === AI Configuration ===
ai: { // Global AI sampling configuration
temperature: 0, // 0 = deterministic, higher = more creative
top: {
p: 0.9, // Top-P nucleus sampling (0-1)
k: 40, // Top-K sampling (1 = most likely, 0 = disabled)
},
},
// === Screen Change Detection ===
redraw: true, // Enable redraw detection (default: true)
// Or use advanced redraw config:
// redraw: {
// enabled: true,
// thresholds: {
// screen: 0.05, // Pixel diff threshold (0-1), false to disable
// network: false, // Monitor network activity (default: false)
// }
// },
// === Logging & Analytics ===
logging: true, // Enable console logging output (default: true)
analytics: true, // Enable analytics tracking (default: true)
// === Advanced ===
apiRoot: "https://...", // API endpoint URL (for self-hosted deployments)
environment: {}, // Additional environment variables for the sandbox
});Control how test execution is visualized. The `preview` option determines where the live debugger view opens:
const testdriver = TestDriver(context, {
preview: "browser", // Opens in default browser (default)
});| Value | Description | |-------|-------------| | `"browser"` | Opens debugger in default browser (default) | | `"ide"` | Opens preview in IDE panel (VSCode, Cursor - requires TestDriver extension) | | `"none"` | Headless mode, no visual preview |
**IDE Preview**
For the best development experience, use `preview: "ide"` with the TestDriver extension for VSCode or Cursor:
const testdriver = TestDriver(context, {
preview: "ide", // Opens preview in IDE panel
});**Headless Mode**
Run tests without any visual preview. Useful for CI/CD pipelines:
const testdriver = TestDriver(context, {
preview: "none", // No visual preview (headless)
});<Note> The legacy `headless: true` option still works for backward compatibility and maps to `preview: "none"`. </Note>
Keep the sandbox alive when a test fails so you can reconnect and debug interactively. The sandbox ID is printed to the console along with instructions for reconnecting via MCP.
const testdriver = TestDriver(context, {
debugOnFailure: true,
});If self-hosting TestDriver, use `ip` to specify the device IP. See [Self-Hosting TestDriver](../self-hosting.md) for details.
const testdriver = TestDriver(context, {
ip: "203.0.113.42", // Your allowlisted IP
});Set the `os` property to run tests on a specific operating system. Available options are `linux` (default) and `windows`.
const testdriver = TestDriver(context, {
os: "windows", // Run on Windows sandbox
});You can make the operating system configurable via environment variables. This requires adding code to read from `process.env` in your test:
const testdriver = TestDriver(context, {
os: process.env.TD_OS || "linux", // Read from env, default to Linux
});Then pass the variable when running tests:
# Run tests on Windows TD_OS=windows vitest run # Run tests on Linux (default) TD_OS=linux vitest run
This pattern is useful for running the same test suite across multiple operating systems in CI/CD:
# Example GitHub Actions matrix
strategy:
matrix:
os: [linux, windows]
steps:
- run: TD_OS=${{ matrix.os }} vitest runDashcam video recording is enabled by default. Disable it to skip recording:
const testdriver = TestDriver(context, {
dashcam: false,
});Screenshots are automatically captured before and after every command (click, type, find, assert, etc.) by default. Each screenshot filename includes the line number from your test file.
Disable automatic screenshots:
const testdriver = TestDriver(context, {
autoScreenshots: false,
});How the TestDriver agent behaves on GitHub issues, pull requests, and @mentions
Deploy TestDriver on your AWS infrastructure using CloudFormation
How TestDriver learns your app and caches what it discovers for instant, deterministic replays
Produce the "Playwright 1.xx API Review" artifact for a range of commits - the public API additions and the breaking or behavior…
Write, refine, run, and QA promptfoo evaluation suites: promptfooconfig.yaml, prompts, providers, vars, tests, assertions,…
DeepEval evaluation workflow for AI agents and LLM applications. TRIGGER when the user wants to evaluate or improve an AI agent,…