/flame-harness-evaluator

name: flame-harness-evaluator
description: Phase 6 — skeptical QA. Run the game, watch it, then judge against the contract. Default = functional check; --strict adds quality and edge-case passes.
argument-hint: ""
allowed-tools: [Agent, Read, Write, Edit, Bash, Glob, Grep]

flame-harness-evaluator

Phase 6 of the flutter-flame-harness pipeline. Skeptical QA gate that decides PASS or FAIL against the negotiated contract. Default mode runs the functional check (6.1) only; `--strict` adds quality scoring (6.2) and an agent-team edge-case sweep (6.3).

All file schemas (`config.md`, `state.md`, `handoff/round-N-gen.md`, `feedback/round-N-qa.md`, `build-log.md`) and the phase transition table are defined in `docs/harness-protocol.md` — that document is the single source of truth (§1 for `config.md`; §2 for `state.md`; §3 for `contract.md`; §4 for handoff layout; §5 for feedback layout; §6 for log schemas; §7 for the `evaluator → admob / evaluator → build / evaluator → generator` transitions). Do not redefine schemas here.

---

Critical Rule

**"Run the code, see the app, then judge." Never PASS on code review alone. Execute commands, launch the game on a simulator, play the core loop, capture and study screenshots. Stub detected = automatic FAIL, no exceptions.**

---

Setup — Read Inputs

Before any check, load:

1. `docs/harness/state.md` — extract `current_round` (integer ≥ 1) and confirm `next_role: evaluator`. 2. `docs/harness/config.md` — extract `app_slug`, `strict_mode` (bool), `max_rounds` (int). 3. `docs/harness/contract.md` — parse `## Mandatory Hard Gates` and `## Functional Criteria`. Confirm `## Status: AGREED` is present; if missing, abort with: `flame-harness-evaluator: contract not AGREED — run flame-harness-contract first`. 4. `docs/harness/handoff/round-<N>-gen.md` (where N = `current_round`) per protocol §4. If the file is missing, abort with: `flame-harness-evaluator: handoff for round <N> not found — generator must run first`.

---

6.1 Functional Check (default — always runs)

Run every step in order. A failure on any **Mandatory Hard Gate** from `contract.md` is an immediate FAIL — do not continue checking remaining criteria.

Step 1 — Static analysis

cd <projects-dir>/<app_slug>
flutter analyze

Required result: **0 issues**. If any issues are reported, record each with file, line, and message. This is a Mandatory Hard Gate.

Step 2 — Tests

cd <projects-dir>/<app_slug>
flutter test

Required result: **0 failures**. Capture the full output. This is a Mandatory Hard Gate.

Step 3 — Stub / TODO grep

grep -rn "TODO\|stub\|placeholder\|스텁\|미구현" \
  <projects-dir>/<app_slug>/lib/ --include="*.dart"

Any match is an **automatic FAIL**. No exceptions (per `docs/harness-protocol.md` §3 Hard Gate 3). Record each match with file path, line number, and the matched text.

Step 4 — game_config.dart centralization

grep -rn "[0-9]\{3,\}\.\?[0-9]*" \
  <projects-dir>/<app_slug>/lib/game/ --include="*.dart" \
  | grep -v "game_config.dart"

Magic numbers (3+ digits) in game logic files other than `game_config.dart` are a Hard Gate failure (per protocol §3 Hard Gate 4). Exclude intentional non-tuning constants (e.g., HTTP status codes) if clearly justified in a comment; document any exclusion in the feedback file.

Step 5 — l10n completeness

cd <projects-dir>/<app_slug>
flutter gen-l10n

Then confirm that every configured `lib/l10n/app_<locale>.arb` (the project's `default_language`, plus `app_en.arb` when `default_language` ≠ `en`) contains identical key sets:

python3 -c "
import glob, json
arbs = glob.glob('lib/l10n/app_*.arb')
keysets = {f: set(json.load(open(f))) for f in arbs}
allkeys = set().union(*keysets.values()) if keysets else set()
bad = {f: sorted(allkeys - ks) for f, ks in keysets.items() if allkeys - ks}
if len(arbs) < 1: print('NO ARB FILES'); exit(1)
if bad: print('MISSING KEYS:', bad); exit(1)
print('l10n OK', list(keysets))
"

Missing keys in either locale are a Hard Gate failure (protocol §3 Hard Gate 6).

Step 5a — Platform-robustness gates

Verify the `## Platform-Robustness Gates` (R1–R11) from `contract.md` / `docs/game-gotchas.md`:

# R1 audio: frequent SFX pooled + audio calls guarded; BGM stop on teardown
grep -rn "AudioPool" lib/ || echo "WARN: no AudioPool — frequent SFX may stutter"
grep -rn "FlameAudio\|AudioPool\|\.bgm" lib/ | grep -i "try\|catch" >/dev/null || echo "CHECK: audio not in try/catch"
grep -rn "bgm.stop\|\.stop()" lib/ || echo "CHECK: BGM stop on teardown/background"
# R2 haptics (if used)
ls lib/systems/haptics.dart 2>/dev/null && grep -nE "kIsWeb|isIOS|isAndroid|enabled|Stopwatch|elapsed" lib/systems/haptics.dart
# R3 lifecycle
grep -rn "WidgetsBindingObserver\|didChangeAppLifecycleState\|pauseEngine" lib/ || echo "FAIL: no lifecycle pause"
# R4 performance: no per-frame whereType in update hot paths
grep -rn "whereType" lib/ | grep -i "update" && echo "CHECK: whereType inside update — verify it's cached, not per-frame"
# R5 branding: custom icon + splash + localized display name (not defaults)
ls assets/icons/icon.png 2>/dev/null || echo "FAIL: no custom app icon"
grep -q "flutter_launcher_icons" pubspec.yaml && grep -q "flutter_native_splash" pubspec.yaml || echo "FAIL: launcher-icons/native-splash not configured"
sips -g hasAlpha assets/icons/icon.png 2>/dev/null | grep -qi "hasAlpha: no" || echo "CHECK: icon may have alpha (App Store rejects)"
# R5 Android adaptive icon: padded foreground + monochrome configured & emitted (not the full-bleed iOS icon → avoids the cropped/"zoomed" launcher look)
grep -q "adaptive_icon_foreground" pubspec.yaml || echo "CHECK: no adaptive_icon_foreground — Android reuses full-bleed icon and looks cropped/zoomed"
grep -q "adaptive_icon_monochrome" pubspec.yaml || echo "CHECK: no adaptive_icon_monochrome — Android 13+ themed icon will look off"
ls android/app/src/ma