/visual-qa

name: visual-qa
description: |
  Visual quality assurance: analyze game screenshots for defects, compare against reference, check motion in frame sequences.
  Supports runtime-native inspection plus Gemini or OpenAI API-backed VQA.

Visual QA

$ARGUMENTS

<!-- agent-operations: inspect_images, read_project_config, read_skill_reference, run_shell_command -->

Resolve bundled prompts, criteria, and scripts relative to this skill's published directory using the active runtime's `read_skill_reference` mapping. Call that resolved directory `<visual-qa-skill-dir>`; do not assume a runtime-specific skill-root environment variable.

CRITICAL: Find acceptance-blocking problems. Do not rationalize defects that block the caller-provided Task Context.

CRITICAL: When Task Context is provided, use its `Verify:` criteria and any design rules it lists as the gate. A reference image is provenance and auxiliary context only — never a pixel-perfect or style-matching bar. Do not fail a check for pure reference mismatch (palette, capitalization, wording, roundedness, spacing, polish, composition) unless it breaks a stated design rule or the `Verify:` criteria, blocks operation, destabilizes layout, or makes the visible state logically false.

CRITICAL: When the caller lists design rules, answer every one of them by `rule_id` and report verdict, evidence, and confidence. Do not assign severity and do not decide what blocks acceptance — `tools/design_rules.py grade` derives that from the rule text. See `scripts/criteria.md` § "Design Rule Checks".

Execution Steps

Step 1 - Parse Arguments

If arguments include `--log <path>`, set `VQA_LOG` to that path and remove those tokens before mode detection.

Reject unsupported argv shapes such as `--screenshot <file> --requirements "..."`.

Step 2 - Detect Mode

Pick the mode from caller args by matching the first row whose precondition holds. If no row matches, STOP and tell the caller their args are malformed.

| Mode | Precondition | Required argv shape | |---|---|---| | Static | `references/<ref>.png` path AND exactly 1 screenshot path | `Check references/<ref>.png against <screenshot.png> - Goal: ... Requirements: ... Verify: ...` | | Dynamic | `references/<ref>.png` path AND 2 or more frame paths | `Check references/<ref>.png against <frame_glob> - Goal: ... Requirements: ... Verify: ...` | | Question | No `references/` path; caller asks a question about screenshots | `--question "..." <screenshot.png> [...]` |

If a reference path appears in the args but the file does not exist on disk, STOP. Return `verdict: error` with `reason: "reference file missing: <path>"`.

Screenshot paths for Static/Dynamic mode may come from `e2e/screenshots/`, `reports/fixgap-visual/`, or `reports/verifier-temp/`.

Each call takes one scene's reference plus that scene's screenshot or frame paths. Reject a single stitched, montage, or contact-sheet image supplied in place of per-scene paths.

Step 3 - Select Backend

Read `.godotmaker/config.yaml` for model selection:

• `vqa_model`: primary VQA backend. Supported values are `native`, `codex`, `gemini:<model>`, and `openai:<model>`.
• `vqa_fallback_model`: fallback when the primary backend is unavailable. Supported values are `native`, `codex`, and `none`.

If `.godotmaker/config.yaml` is missing, `vqa_model` is missing, or `vqa_model` is empty, STOP with `verdict: error`.

Runtime-native VQA means direct image inspection by the selected runtime provider. `native` uses the active agent runtime. `codex` uses Codex image inspection, including from a Claude Code orchestration when Codex is available. API-backed VQA runs `<visual-qa-skill-dir>/scripts/visual_qa.py` with `--model <vqa_model>`.

Do not silently switch providers except for the explicit `vqa_fallback_model` path.

Step 4A - Runtime-Native Execution

Use this path when `vqa_model` is `native` or `codex`, or when an API-backed model fails and `vqa_fallback_model` is `native` or `codex`.

1. Confirm the mode detected in Step 2. 2. Read the mode prompt:

• Static: `scripts/static_prompt.md`
• Dynamic: `scripts/dynamic_prompt.md`
• Question: `scripts/question_prompt.md`

3. For Static and Dynamic, read `scripts/criteria.md`. 4. Combine the mode prompt, criteria, and Task Context. 5. Read every image file referenced in the arguments using the active runtime image-reading path. 6. Analyze only the supplied images and constructed prompt. Never inspect game code. 7. Produce the required output format. 8. Append the debug log entry.

For `native`, use the active agent runtime's image inspection. For `codex`, use the active runtime mapping's `inspect_images` invocation path. Non-Codex runtimes must use the mapped `codex exec` image-input path rather than pretending their own model inspected the files.

Step 4B - API-Backed Execution

Use this path when `vqa_model` is `gemini:<model>` or `openai:<model>`.

Parse the arguments to construct the command. The script is at `<visual-qa-skill-dir>/scripts/visual_qa.py`.

Detect the available Python command: run `python3 --version` and `python --version`, then use whichever succeeds. Cache the result for the session.

Model selection:

• If `vqa_model` is API-backed, pass it as `--model <value>`.
• If `vqa_model` is `native` or `codex`, use Step 4A.
• If `vqa_model` is missing or empty, STOP with `verdict: error`.

VQA_MODEL=$(grep -oP 'vqa_model:\s*\K\S+' .godotmaker/config.yaml 2>/dev/null || echo "")
MODEL_FLAG=""
case "$VQA_MODEL" in
  "") echo "Missing vqa_model in .godotmaker/config.yaml" >&2; exit 1 ;;
  native|codex) echo "Use Runtime-Native Execution for vqa_model: $VQA_MODEL" >&2; exit 1 ;;
  gemini:*|openai:*) MODEL_FLAG="--model $VQA_MODEL" ;;
  *) echo "Unsupported vqa_model for API-backed visual_qa.py: $VQA_MODEL" >&2; exit 1 ;;
esac

# Static
PYTHON "<visual-qa-skill-dir>/scripts/visual_qa.py" --log ${VQA_LOG:-.vqa.log} $MODEL_FLAG [--context "Goal: ... Requirements: ... Veri