/testing-core-processors

name: testing-core-processors
description: Use when writing or debugging integration tests for error processors in packages/core/src/processors/. Covers the MockLanguageModelV2 pattern for simulating API errors and verifying retry behavior, plus the build prerequisites for focused vitest runs.

Testing Core Error Processors

How to write integration tests for error processors in `packages/core/src/processors/`.

Prerequisites

• Build core before running tests: `pnpm build:core` (from repo root)
• If focused vitest runs fail to resolve `@internal/test-utils/setup`, the build step was skipped

Mock Model Pattern

Use `MockLanguageModelV2` from `@internal/ai-sdk-v5/test` to simulate API errors and verify retry behavior.

import { APICallError } from '@internal/ai-sdk-v5';
import { convertArrayToReadableStream, MockLanguageModelV2 } from '@internal/ai-sdk-v5/test';

// Track calls and captured prompts
let callCount = 0;
const receivedPrompts: any[] = [];

const model = new MockLanguageModelV2({
  doGenerate: async ({ prompt }) => {
    callCount++;
    receivedPrompts.push(JSON.parse(JSON.stringify(prompt)));
    if (callCount === 1) {
      throw new APICallError({
        message: '...',
        url: '...',
        requestBodyValues: {},
        statusCode: 400,
        responseBody: '...',
        isRetryable: false,
      });
    }
    return {
      rawCall: { rawPrompt: null, rawSettings: {} },
      finishReason: 'stop',
      usage: { inputTokens: 10, outputTokens: 20, totalTokens: 30 },
      content: [{ type: 'text', text: 'response' }],
      warnings: [],
    };
  },
  doStream: async ({ prompt }) => {
    // Same error logic as doGenerate
    // IMPORTANT: Stream response must include all event types:
    return {
      rawCall: { rawPrompt: null, rawSettings: {} },
      warnings: [],
      stream: convertArrayToReadableStream([
        { type: 'stream-start', warnings: [] },
        { type: 'response-metadata', id: 'id-0', modelId: 'mock-model', timestamp: new Date(0) },
        { type: 'text-start', id: 'text-1' },
        { type: 'text-delta', id: 'text-1', delta: 'response text' },
        { type: 'text-end', id: 'text-1' },
        { type: 'finish', finishReason: 'stop', usage: { inputTokens: 10, outputTokens: 20, totalTokens: 30 } },
      ]),
    };
  },
});

Stream Mock Gotcha

The stream mock format requires `stream-start`, `response-metadata`, `text-start`, `text-delta`, `text-end`, and `finish` events. Using only `text-delta` + `finish` (the minimal format) will result in empty text output because the AI SDK expects the full event sequence. See `prefill-error-recovery.test.ts` for the reference pattern.

Test Structure

For each error processor, write at minimum:

1. **Happy path**: Processor catches the target error, modifies messages, retries successfully

• Assert: `agent.generate()` succeeds, mock called 2x, retry prompt has expected modifications

2. **Control test**: Same scenario without the processor — error propagates

• Assert: `agent.generate()` throws the expected error

3. **Selectivity test**: Processor ignores unrelated errors (e.g. rate limit 429)

• Assert: Error propagates, mock called only 1x

Passing Messages with Tool Calls

When seeding conversation history for tool-related tests, pass messages as the second argument to `agent.generate()` or `agent.stream()` using the AI SDK message format:

const messages = [
  { role: 'user', content: 'Do something' },
  { role: 'assistant', content: [{ type: 'tool-call', toolCallId: 'some-id', toolName: 'myTool', args: {} }] },
  { role: 'tool', content: [{ type: 'tool-result', toolCallId: 'some-id', toolName: 'myTool', result: 'done' }] },
];
await agent.generate(messages);

Running Tests

# Run focused processor tests
npx vitest run packages/core/src/processors/my-processor.test.ts

# Run all processor tests
npx vitest run packages/core/src/processors/

# Full core test suite (slower)
pnpm test:core

Devin Secrets Needed

None for mock-based integration tests. For live API tests:

• `ANTHROPIC_API_KEY` — for testing against real Anthropic API
• `OPENROUTER_API_KEY` — for testing provider switching scenarios