/day0-release

name: day0-release
description: Deterministic end-to-end driver for day-0 quantized-checkpoint releases — chains PTQ → evaluation → comparison with enforced gates between stages (the evaluation stage deploys the checkpoint itself), and returns a publish decision (ACCEPT / REGRESSION / ANOMALOUS / INFEASIBLE). Use when the user asks to "release a model at day-0", "quantize and validate model X is within N% of baseline and tell me if it's publishable", or "run the full day-0 workflow". Do NOT use for single-stage requests — quantizing only (use ptq), serving only (use deployment), evaluating only (use evaluation), or comparing two existing runs (use compare-results).
license: Apache-2.0

Day-0 Release

Drive a model from a pretrained checkpoint to a publish decision for a quantized checkpoint, in a fixed sequence with a gate after every stage. This skill is a **conductor**: it sequences the existing domain skills and enforces the gates — it does not re-implement quantization, serving, evaluation, or comparison.

**Goal (the default day-0 criterion):** a quantized checkpoint smaller than the source, with accuracy drop within the threshold (default <1%) on the standard benchmark set versus the matching baseline, plus a publish recommendation.

When to use

Use only for the full goal-driven release. For a single stage, route to the domain skill directly: quantize → **ptq**, serve → **deployment**, evaluate → **evaluation**, compare two existing runs → **compare-results**.

Inputs

Resolve these before starting (ask the user for anything missing):

• **Model** — HF handle or checkpoint path.
• **Recipe / qformat** — e.g. `nvfp4`, `fp8`, or a recipe path. One candidate for v1.
• **Cluster / launcher** — from `clusters.yaml` (see the common skill's

`environment-setup.md`).

• **Eval set** — defaults to the evaluation skill's AA suite

(`recipes/tasks/aa/`).

• **Threshold** — max accuracy drop; default `0.01` (1%).

The chain

setup ─▶ PTQ ─▶ canary ─▶ baseline-eval ─▶ quantized-eval ─▶ compare ─▶ verbosity ─▶ closeout
          │        │           │                │              │           │
       gate_ptq  /health    gate_run         gate_run    gate_compare  gate_verbosity
                 + 1 gen

The **evaluation** skill deploys the model it evaluates (it stands up its own endpoint per run), so there is no separate deploy *stage* — a serving failure during evaluation surfaces through the eval gate (`DEPLOYMENT_HEALTH_FAILED`) and triages to the **deployment** skill (see Step 4). The Step 2b **canary** is not that: it runs *before* any evaluation precisely so an unservable checkpoint is caught in ~15 min rather than after a multi-hour eval.

Accuracy (Step 5) and verbosity (Step 5b) are **independent gates**; closeout requires both.

Run each stage by invoking the domain skill, then run its gate before proceeding. **Do not advance past a failed gate.** Copy this checklist and track progress:

- [ ] Step 0: Resolve inputs; confirm threshold and eval set
- [ ] Step 1: Setup gate — creds present, cluster reachable
- [ ] Step 2: PTQ (ptq skill) → gate_ptq.py
- [ ] Step 2b: Serving canary — /health + one generation (deployment skill)
- [ ] Step 3: Baseline eval (evaluation skill, deploys source) → gate_run.py
- [ ] Step 4: Quantized eval (evaluation skill, deploys candidate) → gate_run.py
- [ ] Step 5: Compare (compare-results skill) → external sanity → gate_compare.py → decision
- [ ] Step 5b: Verbosity gate → gate_verbosity.py
- [ ] Step 6: Closeout — report + publish recommendation

Step 1 — Setup gate

Use the common skill's `credentials.md` and `remote-execution.md` to confirm credentials and cluster reachability. If either fails, stop with `SYSTEMIC` — do not start PTQ.

Step 2 — PTQ

Invoke the **ptq** skill to produce the quantized checkpoint. Then gate:

# The ptq skill's post-PTQ validation produces a validation-summary JSON (size
# ratio + layer-precision counts + metadata diffs; see the ptq skill's
# references/checkpoint-validation.md). v1 gates on that summary:
python "$SKILL_DIR/scripts/gate_ptq.py" --summary <validation-summary.json>
#   add `--recipe <qformat>` to override the recipe recorded in the summary

`gate_ptq.py` returns JSON `{pass, failure_class, detail}`. On `pass: false`, branch on `failure_class` (see **Triage** below). Do not evaluate an unvalidated checkpoint.

Step 2b — serving canary (MANDATORY before Step 3)

The canary itself is **already specified** by the ptq skill: see `ptq/references/checkpoint-validation.md` (required gate, canary query and the `Serving canary` row of its report table) and `ptq/SKILL.md`. Run it there rather than re-deriving it here — `gate_ptq.py` checks size, coverage and metadata, not whether the checkpoint *loads*, and skipping the canary has cost a full baseline eval against a checkpoint the serving stack could never load.

On failure use `failure_class: CHECKPOINT_NOT_SERVABLE` and drop to the **deployment** skill; do not proceed to Step 3.

**What that spec does not cover: writing a canary that cannot lie.** Both of these produced a wrong verdict on a large MoE, in opposite directions:

• **Poll ceiling > load time**, with headroom. A 50 min poll against a 51 min load

reported failure for a checkpoint that serves fine. Large MoE loads are CPU-bound fp8 dequant (~50 min); 0% GPU during *load* is normal.

• **Print an explicit `RESULT:` on every path** and exit non-zero on failure. A

poll loop that falls through to the generation test exits 0 and reads as PASS.

• Server log on shared storage, not node-local `/tmp`.
• Canary the **as-exported** artifact, not a copy you modified to make it work.

Step 3 — Baseline eval

The baseline is the **source** (pre-quantization) model on the same task set and sampling params. Always run a fresh baseline via the **evaluation** skill, which deploys the source model itself. Gate with `gate_run.py`.

Step 4 — Quanti