/cdf-anomalygen

name: cdf-anomalygen
description: >-
  Use when running the CDF AnomalyGen pipeline over a defect dataset — mask placement,
  fine-tuning, synthetic defect-image generation (SDG), evaluation, quality refinement, and
  pseudo-labeling — even when the user names only one stage.
license: Apache-2.0
compatibility: >-
  Requires a CUDA GPU and the built env (native .venv or the Docker image); Hugging Face auth +
  network access to download the model checkpoints.
metadata:
  owner: NVIDIA
  service: anomaly generation
  version: 1.2.0
  author: NVIDIA <nvidia@nvidia.com>
  tags:
      - physical-ai
      - anomaly-generation
      - cosmos
      - fine-tuning
  languages: [python]
  frameworks: [pytorch, cosmos, diffusion]

Skill: cdf-anomalygen

Purpose

AnomalyGen fine-tunes the frozen **Cosmos** base into an **anomaly-generation** model: given a clean image + defect mask, it paints a realistic defect into that region (synthetic data for detectors). Pick a mode, collect its params, run the steps — **all commands from the repo root.**

When to Use This Skill

Use for the CDF AnomalyGen pipeline over a defect dataset — any single stage (mask placement, fine-tuning, generation, evaluation, refinement, or pseudo-labeling) or the full run.

• "Run the AnomalyGen pipeline on phone_screen"
• "Fine-tune an anomaly-generation model for PCB defects"
• "Generate 25 synthetic PCB anomaly images"
• "Place defect masks", "evaluate quality", or "pseudo-label the generated defects"

Do not use this skill for generic image generation, or for synthetic data that isn't defect images built from the `{texture}/{defect}` clean-image + defect-mask contract.

Prerequisites

• A **CUDA GPU** and the project's built env (native `.venv` or the Docker image).
• **Hugging Face auth** — run `hf auth login` — plus network access; the downloader fetches

the model checkpoints.

• A **`dataset_dir`** in the Step 1 contract layout (clean images + `{defect}` masks; plus

`defect_spec.jsonl` **only when `mask_backend=amp`** — the default `amg` never reads it, and nothing after Step 2 does either).

Preflight the env + checkpoints (GPU, Python, CUDA/torch, deps, HF auth, model checkpoints):

scripts/preflight_env_ckpt.sh

**Read which group failed before reacting** — the summary separates them, and only one is fixed by downloading:

• *missing checkpoints* → `scripts/download_checkpoints.sh` (run `hf auth login` first).
• *env / Python / deps* → rebuild the environment (`bash scripts/env_setup.sh`, then activate it).

**Third group — model access for mask placement (checked HERE, never mid-flow).** The default `mask_backend=amg` needs a vision model, and a run that discovers this at Step 2 has already spent the user's time. A **VLM API is a base, a key and a model name** — and all three live in ONE place: **`~/.anomalygen/endpoints.yaml`**, the model registry (nothing is built into the code; the template is `scripts/agentic_mask_generation_serving/endpoints.example.yaml`). It sits under `$HOME`, not in the repo, so it works from any directory, serves every checkout, and is writable inside the read-only product containers. The moment inputs are locked with `mask_backend=amg` (the default), probe:

[ -f ~/.anomalygen/endpoints.yaml ] && echo "amg model: ~/.anomalygen/endpoints.yaml present" \
    || echo "amg model: no ~/.anomalygen/endpoints.yaml — NO model can resolve; copy scripts/agentic_mask_generation_serving/endpoints.example.yaml there"

**The model resolves in the registry → use it.** That is the whole check: `--gen-model` and `--judge-model` are REQUIRED flags (there is no default model), and every name they carry must have a registry entry (`model` + `base_url` + `key`) — the entry point refuses to start when one does not; there is no environment fallback. A user's own OpenAI-compatible endpoint is the same case: one `endpoints.yaml` entry with their documented base URL, their key, and their model name in `--gen-model`/`--judge-model`.

**Anything less → stop and ask the user now**, before Step 1 — unless the user's request already names the model (e.g. "use local Gemma"): that IS the pick, act on it without re-asking. Three options — whichever it is, set `AMG_MODEL` to the chosen name; the Step 2 command passes it to both required flags:

• **(a) supply an API** — one `endpoints.yaml` entry: a model name, a base and a key

(e.g. `AMG_MODEL=azure/openai/gpt-6-astra`).

• **(b) let the skill start the model service** — in a container, as a background process inside it;

on a dev host, from `scripts/agentic_mask_generation_serving/`. Needs a GPU; the weights are a default checkpoint — `scripts/download_checkpoints.sh gemma` fetches them and preflight verifies them (the group argument fetches only this model; bare, the script fetches all eleven). `mkdir -p ~/.anomalygen && cp scripts/agentic_mask_generation_serving/endpoints.example.yaml ~/.anomalygen/endpoints.yaml` wires it: the template's gemma entry already routes `AMG_MODEL=nvidia/google/gemma-4-31b-it` to `http://localhost:8902/v1`, and the run carries `--beam-multi 4 --beam-pairwise --exemplar-captions --exemplar-visual-prompt --coverage-note` — the local model always pairs with the in-loop beam and the three information channels. Bootstrap, lifecycle and the why: [`agentic-mask-generation.md`](references/agentic-mask-generation.md).

• **(c) set `mask_backend=amp`** for this run.

Never fall back silently. Whichever they pick, the run's own capability probe confirms the model can take images and answer in JSON before Step 2 spends anything.

Inputs

Collect the run's inputs from the user before starting. **If the `mode` or any input it requires is missing, ask the user first — never guess a required input or silently fall back to its default.**

**Lock the `mode` first** — it fixes which steps run and which inputs are required (ask for any not provided):

• **`full`** — train + generate, Steps **1 → 7