/caliber-testing

name: caliber-testing
description: Writes Vitest tests following project patterns: __tests__/ directories, vi.mock() for module mocking with vi.hoisted() for test-time factories, global LLM mock from src/test/setup.ts, environment variable save/restore in beforeEach/afterEach, vi.clearAllMocks() lifecycle, and test file organization. Use when user says 'write tests', 'add test coverage', 'test this', creates *.test.ts files, or when test failures appear in CI. Do NOT use for non-test code or for debugging without writing tests.
paths:
  - src/**/__tests__/*.test.ts

Caliber Testing

Critical

• All test files MUST be placed in `__tests__/` directories parallel to source files: `src/[module]/__tests__/[module].test.ts`
• Register any new `__tests__/` directories in `vitest.config.ts`'s `include` glob (already configured: `src/**/*.test.ts`)
• NEVER mock `src/test/setup.ts` — it is the global LLM provider mock already applied to all tests
• Environment variable tests MUST save `process.env` in `beforeEach`, restore in `afterEach`, and explicitly delete env vars to test absence
• Temporary file/directory cleanup MUST happen in `afterEach`, not in individual test cleanup. Use `fs.rmSync(dir, { recursive: true, force: true })`
• When a test requires unmocking modules mocked in global setup, call `vi.unmock('../module.js')` BEFORE the import statement
• Run `pnpm test` locally and `pnpm test:coverage` before committing to verify coverage thresholds (lines: 50, functions: 50, branches: 50, statements: 50)

Instructions

Step 1: Create the test file in the correct directory

Create `src/[module]/__tests__/[module].test.ts`. The parent source file is `src/[module]/[module].ts`.

**Verify**: The `__tests__` directory exists at the same level as the source file being tested.

Step 2: Import test framework

At the top of every test file, import from `vitest`:

import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';

Add additional imports based on what you're testing:

• File system: `import fs from 'fs'; import path from 'path'; import os from 'os';`
• Temporary files: Use `fs.mkdtempSync(path.join(os.tmpdir(), 'caliber-prefix-'))`
• Exec: `import { execSync } from 'child_process';`

**Verify**: All required test utilities are imported before test definitions.

Step 3: Set up module mocking (if needed)

If testing a module that imports other modules you want to mock:

vi.mock('../config.js', () => ({
  loadConfig: vi.fn(),
  writeConfigFile: vi.fn(),
}));

For complex mocks with test-time factory functions (hoisted):

const { mockLoadConfig } = vi.hoisted(() => ({
  mockLoadConfig: vi.fn(),
}));

vi.mock('../config.js', () => ({
  loadConfig: () => mockLoadConfig(),
}));

For unmocking global setup mocks (e.g., to test llm/index.js itself):

vi.unmock('../index.js');

Place all `vi.mock()` and `vi.unmock()` calls BEFORE importing the module under test.

**Verify**: Mock declarations appear before the import of the module being tested.

Step 4: Organize tests in describe blocks

Group related tests with `describe()`:

describe('functionName', () => {
  it('returns X when Y', () => {
    // test body
  });
});

**Verify**: Each `it()` test has a clear, complete assertion.

Step 5: Manage environment and process state

For tests that modify `process.env` or `process.argv`:

describe('config tests', () => {
  const originalEnv = process.env;
  const originalArgv = process.argv;

  beforeEach(() => {
    process.env = { ...originalEnv }; // Copy, not reference
    process.argv = [...originalArgv];
    delete process.env.SPECIFIC_VAR; // Explicitly remove vars to test absence
  });

  afterEach(() => {
    process.env = originalEnv;
    process.argv = originalArgv;
  });

  it('tests env var behavior', () => {
    process.env.MY_VAR = 'test';
    // test code
  });
});

**Verify**: `beforeEach` creates a copy of env/argv; `afterEach` restores originals; unused env vars are explicitly deleted with `delete process.env.VAR`.

Step 6: Manage temporary files and directories

For file system tests, create temporary directories and clean them up:

describe('file tree', () => {
  const dirs: string[] = [];
  
  afterEach(() => {
    for (const d of dirs) {
      try { fs.rmSync(d, { recursive: true, force: true }); } catch {}
    }
    dirs.length = 0;
  });

  it('processes files', () => {
    const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'caliber-test-'));
    dirs.push(tmp);
    // use tmp directory
  });
});

**Verify**: All temporary directories are pushed to a cleanup array and removed in `afterEach`.

Step 7: Handle mock state cleanup

Before each test, clear mock call history to avoid pollution between tests:

beforeEach(() => {
  vi.clearAllMocks(); // Reset all mock call counts and return values
});

afterEach(() => {
  vi.restoreAllMocks(); // Restore original implementations
  vi.resetModules(); // Reset cached module imports (if you reload modules)
});

**Verify**: `beforeEach` calls `vi.clearAllMocks()` for providers and `afterEach` calls `vi.restoreAllMocks()`.

Step 8: Test assertions

Write assertions using `expect()`. Match the patterns from existing tests:

// Simple checks
expect(value).toBe(expected);
expect(array).toContain(item);
expect(fn).toThrow('error message');

// Instance checks
expect(obj).toBeInstanceOf(ClassName);

// Mock checks
expect(mockFn).toHaveBeenCalledTimes(1);
expect(mockFn).toHaveBeenCalledWith(arg);

// File system checks
expect(fs.existsSync(path)).toBe(true);

**Verify**: Each test has at least one assertion and uses appropriate `expect()` matchers.

Step 9: Run tests

Run tests locally before committing:

pnpm test                # Run all tests in watch mode
pnpm test:coverage       # Check coverage thresholds
pnpm test -- src/my/path/__tes