/by-campaign-optimizer

id: "skill_6cd1e0f4c6864ff89ff64d873441900f"
name: "by-campaign-optimizer"
display-name: "BY Campaign Optimizer (Active Learning)"
short-description: "Trains a Random Forest on scored designs from prior rounds and proposes parameter adjustments for the next round. Use when a multi-round campaign has at least 10 scored designs and you need data-driven suggestions for thresholds, design counts, and feature targets."
category: "optimization"
keywords: "active learning, random forest, optimizer, campaign, multi-round, parameter tuning, feature importance, acquisition function, scikit-learn"
version: "1.0"
last-updated: "2026-05-20"

BY Campaign Optimizer (Active Learning)

Multi-round design campaigns generate scored designs at every iteration. This skill turns that scoring history into actionable parameter changes for the next round — training a lightweight Random Forest on the designs you already have, ranking which features actually discriminate good from bad, and proposing thresholds and design counts that target the most promising regions of feature space.

The optimizer is the **iteration loop**: it runs *after* the diagnosis step (`by-failure-diagnosis`) has identified which features matter, and decides what parameters to push or relax for the next round. Diagnosis answers "what went wrong?"; this skill answers "what should we try next?".

---

When to Use This Skill

✅ **Use this skill when:**

• A design round has completed and scores are written to `*_scores.json` files
• The campaign has **≥10 scored designs total** across all rounds
• You are deciding how to configure the next round of a multi-round campaign
• `by-failure-diagnosis` has identified discriminating features and you need

concrete threshold values

• You want a feature-importance ranking to decide what to optimize for
• The user asks "what should I change for the next round?" or "tune my campaign"

❌ **Do NOT use this skill when:**

• The campaign has fewer than 10 scored designs → use rule-based defaults from

`by-campaign-manager` instead

• This is the first round (no prior scores exist) → use `by-research` and

`by-design-workflow` to set initial parameters

• Designs are failing for non-statistical reasons (e.g., compute errors, bad

target structure) → run `by-failure-diagnosis` first to confirm the signal is real

• The campaign is a single-shot screen with no planned iterations
• You only need a "why did designs fail?" report → use `by-failure-diagnosis`

(the diagnostic counterpart to this optimizer)

---

Quick Start

from proteus_cli.campaign.active_learning import (
    has_enough_data,
    suggest_from_campaign,
)

campaign_dir = "campaigns/tnfa/campaign_20260520_001"

if has_enough_data(campaign_dir):
    result = suggest_from_campaign(campaign_dir)
    print(f"Source: {result.source}")                  # "active_learning"
    print(f"Confidence: {result.confidence}")          # "high" if >30 designs
    print(f"Top feature: {result.feature_importances[0]}")
    print(f"Recommendations: {result.recommended_parameters}")

Or via the CLI scripts shipped with this skill:

# Step 1: train RF, write optimizer_output.json
python scripts/optimize_from_csv.py \
    --scores campaign/round_1/scores.csv \
    --output campaign/round_1/optimizer_output.json

# Step 2: write next-round YAML config
python scripts/propose_next_round.py \
    --optimizer-output campaign/round_1/optimizer_output.json \
    --previous-config campaign/round_1/config.yaml \
    --output campaign/round_2/config.yaml

---

Installation

| Package | Version | License | Commercial Use | Install | |---------|---------|---------|----------------|---------| | scikit-learn | ≥1.3 | BSD-3-Clause | ✅ Permitted | `pip install scikit-learn` | | pandas | ≥2.0 | BSD-3-Clause | ✅ Permitted | `pip install pandas` | | numpy | ≥1.24 | BSD-3-Clause | ✅ Permitted | `pip install numpy` | | PyYAML | ≥6.0 | MIT | ✅ Permitted | `pip install pyyaml` |

Or install the ML extras bundle:

pip install by-agent[ml]

**License Compliance:** All packages permit commercial use in AI applications.

---

Inputs

**Required:**

• **Scored designs**: either
• A campaign directory containing `*_scores.json` files (Python API), or
• A flat CSV with one row per design (CLI scripts)
• Each scored design must contain numeric values for some subset of:

`ipsae` (or `ipsae_min`), `iptm`, `plddt`, `rmsd`, `liabilities`, `cdr3_length`, `net_charge`, `hydrophobic_fraction`

• A `status` column (`PASS` / `FAIL`) is optional but improves recommendations

**Optional:**

• `min_designs` (default 10) — the active-learning threshold
• A previous-round YAML config to use as a template for the next round
• A target feature to optimize (defaults to `ipsae`)

See [references/active-learning.md](references/active-learning.md) for sample-size guidance and [references/random-forest-approach.md](references/random-forest-approach.md) for feature-engineering details.

---

Outputs

**Primary results:**

• `optimizer_output.json` — feature importances, recommended thresholds,

exploration regions, confidence level, and a human-readable explanation

• `OptimizationResult` dataclass (Python API) with the same content

**Next-round config:**

• `config_round_N+1.yaml` — design parameters with adjusted thresholds,

updated `num_designs`, and feature targets carried over from the optimizer

**Diagnostic fields in `optimizer_output.json`:**

• `source` — `"active_learning"` if RF trained, `"rule_based"` if fallback
• `confidence` — `"low"` / `"medium"` / `"high"` based on training-set size
• `feature_importances` — sorted list of `(feature, importance)` tuples
• `recommended_parameters` — concrete values for the next round
• `exploration_regions` — feature-space regions to sample more densely
• `warnings` — any files skipped or features dropped

---

Clarification Questions

**⚠️ CRITICAL: ASK THIS FIRST.** Always confirm the user has at l