/agent-qa-authoring

name: agent-qa-authoring
description: Use when creating, editing, validating, or running agent-qa tests, suites, or hooks. Prefer agent-qa MCP tools, enforce canonical agent-qa IDs, and use the bundled schema reference to avoid hallucinated config keys or YAML fields.
metadata:
  short-description: Author agent-qa tests safely

agent-qa Authoring

Workflow

1. Discover the local surface with `agent_qa_discover`. 2. Inspect active config with `agent_qa_get_config`, especially targets, devices, providers, and `services.mcp`. 3. Load `references/agent-qa-contracts.json` when you need exact schema fields or ID contracts. 4. Generate every new ID with agent-qa tooling:

• MCP: `agent_qa_generate_id`
• CLI fallback: `agent-qa ids generate <test|suite|hook|run|observation>`
• npx fallback: `npx --yes agent-qa ids generate <type>`

5. Never hand-write IDs. Validate existing IDs with `agent_qa_validate_id` or `agent-qa ids validate <type> <id> --json`. 6. Validate definitions before saving:

• Tests: `agent_qa_validate_test` or `agent_qa_validate_definition` with `kind: "test"`
• Suites: `agent_qa_validate_suite` or `agent_qa_validate_definition` with `kind: "suite"`
• Hooks: `agent_qa_validate_definition` with `kind: "hooks"`

7. Prefer MCP authoring mutations:

• Tests: `agent_qa_create_test`, `agent_qa_update_test`, `agent_qa_delete_test`
• Suites: `agent_qa_create_suite`, `agent_qa_update_suite`, `agent_qa_delete_suite`
• Hooks: `agent_qa_create_hook`, `agent_qa_update_hook`, `agent_qa_delete_hook`

8. Use CLI/YAML fallback only when MCP is unavailable. Keep file paths matched by `workspace.testMatch` or `workspace.suiteMatch`.

Required ID Contracts

• Test IDs: `t_` + 10 id-agent words.
• Suite IDs: `s_` + 10 id-agent words.
• Hook IDs: `h_` + 10 id-agent words.
• Run IDs: `r_` + 10 id-agent words.
• Observation IDs: `obs_` + 10 id-agent words.

Before Running

• Validate YAML first.
• Prefer `agent_qa_enqueue_test_run` and `agent_qa_enqueue_suite_run` over shelling out.
• If using CLI fallback, run only after validation succeeds.

Do Not

• Do not invent config keys.
• Do not use legacy root config buckets.
• Do not hand-write IDs.
• Do not mutate files outside the configured workspace patterns.