/nvflare-autofl

name: nvflare-autofl
description: "Use for agent-assisted Auto-FL optimization of an existing NVFLARE job in simulation, POC, or production. Do not use for code conversion, diagnosis-only work, or deployment setup."
license: Apache-2.0
compatibility: "Requires NVFLARE 2.9.0+, Python, and permission to run NVFLARE jobs in the selected environment."
metadata:
  version: "0.1.0"
  author: "NVIDIA FLARE Team <federatedlearning@nvidia.com>"
  tags: "nvflare, federated-learning, optimization"
  min-flare-version: "2.9.0"
  blast-radius: submits_production
  category: Optimization

NVFLARE Auto-FL

**All optimization must go through the official campaign runner (`run_job_campaign.py`): never optimize or edit the user's project directly outside a prepared candidate.**

Purpose

Improve a measured objective for an existing NVFLARE job through isolated, reproducible candidate changes while the campaign runner preserves the live best source, comparison budget, metric semantics, state, and evidence.

Available Scripts

| Script | Purpose | Arguments | | --- | --- | --- | | `scripts/run_job_campaign.py` | Authoritative campaign lifecycle runner | `ACTION JOB` plus action-specific flags | | `scripts/campaign_guard.py` | Read-only ledger diagnostics | `[RESULTS]`, `--mode`, and diagnostic thresholds | | `scripts/plot_progress.py` | Render campaign progress | `[RESULTS]`, `--output`, `--metric`, `--mode` | | `scripts/job_importer.py` | Import library used by the campaign runner | Not a standalone CLI |

Run the bundled CLIs directly with Python. This skill has no NVFLARE or agent `run_script()` helper; do not invent or call one. Resolve each script relative to this `SKILL.md`.

Inputs

• **Required**: an existing NVFLARE `job.py`, the optimization objective or metric, and `sim`, `poc`, or `prod`.
• **Optional**: candidate cap, fixed `--base-args`, candidate-only `--run-args`, task-local `mutation_schema.yaml`, and declared simulator environment-variable names.
• **Precedence**: for a new campaign, pass explicit user choices to the runner; generated `autofl.yaml` and state then become authoritative. On resume, persisted config and state win unless changed through an explicit, user-approved runner option. Never infer a cap from ambient variables.

Instructions

Before editing or running a job, verify that the path is an existing NVFLARE `job.py` and the user requested an optimization objective. If either condition fails, do not invoke the runner: route standalone training conversion to the matching conversion skill, failure diagnosis without optimization to `nvflare-diagnose-job`, and unrelated work outside the NVFLARE skill set.

Resolve [run_job_campaign.py](scripts/run_job_campaign.py) relative to this `SKILL.md`, store its absolute path as `RUNNER`, and initialize the campaign:

python "$RUNNER" initialize ./job.py [--metric <metric>] --env <sim|poc|prod> [--max-candidates <n>]

Campaign direction comes from `job.py` `key_metric_mode` or a same-metric `stop_cond`; NVFLARE defaults to `max`. Declare raw loss metrics with `key_metric_mode="min"`; explicitly negated metrics remain ordinary `max` metrics. For conditional recipes, safe refusals, and unnamed simulator roots, read the [job import contract](references/job-import-contract.md).

Read `autofl.yaml` and the JSON response, then prepare an agent-authored candidate with a short hypothesis and optional candidate-only arguments:

python "$RUNNER" prepare ./job.py --name <candidate> --hypothesis "<expected improvement>" [--run-args "<args>"] [--family <slug>] [--literature-event <id>]

Pass `--family <slug>` and `--literature-event <id>` when the candidate develops a recorded literature review; both persist in the manifest and as ledger columns.

Edit only the returned candidate source directory. Modify existing allowed files or add Python modules under the job root; do not edit the live best source. Then evaluate:

python "$RUNNER" evaluate ./job.py --manifest <candidate_manifest.json>

Simulation evaluation runs the candidate immediately. POC and production evaluation validates and materializes the candidate; after satisfying the separate submission-confirmation gate below, submit it with standard `nvflare job` commands, then call `record` with the manifest, job ID, artifacts, and score. Use `abandon` to restore a pending candidate. Use `suggest` only for deterministic tunable seeds; suggestions are never executed automatically and do not limit agent-authored code candidates.

If the job directory contains a task-local `mutation_schema.yaml`, treat its `comparison_budget_args.default_candidate_budget` and mutation bounds as authoritative. Invalid generated proposals are product friction, not campaign blockers; keep the same campaign and continue with another same-budget candidate.

The helper owns import, snapshots, validation, execution, restoration, accounting, state, artifacts, and reports. After each action, read `.nvflare/autofl/campaign_state.json`; finalize only when `final_response_allowed=true`. Read [continuous campaigns](references/continuous-campaigns.md) for long-running behavior and [experiment comparability](references/experiment-comparability.md) for budgets and reruns.

Use the ledger score extracted per `objective.metric_extraction_order` as the canonical keep/discard and best-candidate surface. Before interpreting fallback or cross-site scores, read the exact selection and fallback rules in [experiment comparability](references/experiment-comparability.md).

Read `autofl.yaml` and show the user a concise campaign summary:

• **Editable**: metric, environment, budget, tunables, artifacts, `objective.optimization_metric`, metric source, source hash, and importer version.
• **Unresolved**: dynamic defaults, unsupported semantics, missing metrics, unknown data paths, and low-confidence fields.
• **Allowed**: edit/create paths, fixed-budget and metric invariants, and environment