/by-scoring

id: "skill_c170f6b9f5464c23b9969f7ca78fa96b"
name: "by-scoring"
display-name: "BY Scoring"
short-description: "Interpret and apply BY scoring metrics — ipSAE, ipTM, pLDDT, RMSD, liability counts, and the composite ranking formula. Use when scoring designs, interpreting screening output, troubleshooting metric disagreements, or advising on candidate ranking."
category: "scoring"
keywords: "ipSAE, ipTM, pLDDT, composite score, scoring, ranking, PAE, Protenix, BoltzGen, antibody, nanobody, design"
version: "1.0"
last-updated: "2026-05-20"
mcp_tools: ["mcp__by-screening__score_ipsae", "mcp__by-screening__score_ipsae_multi_seed", "mcp__by-screening__compute_composite", "mcp__by-screening__screen_liabilities"]

BY Scoring Skill

Interpret and apply BY custom scoring metrics for protein and antibody design. This skill covers ipSAE (interface Predicted Structural Accuracy Error) — the primary custom metric that differentiates BY from generic structure prediction tools — along with ipTM, pLDDT, RMSD, liability scoring, and the BY composite ranking formula.

ipSAE uses the open-source DunbrackLab formula (Dunbrack et al. 2025) with no proprietary dependencies. Use this skill whenever you need to score designs, interpret scoring output, troubleshoot disagreements between metrics, or advise on candidate ranking.

---

When to Use This Skill

**Use this skill when:**

• ✅ Scoring designs after Protenix refolding — you have NPZ or confidence JSON output
• ✅ Computing the BY composite score on a panel of screened designs
• ✅ Explaining why two metrics (ipSAE vs ipTM) disagree on a candidate
• ✅ Setting modality-specific thresholds (antibody vs nanobody vs de novo)
• ✅ Diagnosing why an entire panel failed screening (zero LAB-READY)
• ✅ Auditing whether a multi-seed result is stable or driven by one lucky seed
• ✅ Interpreting asymmetric ipSAE (`dt >> td` or `td >> dt`)
• ✅ Selecting the right PAE cutoff (10 A Protenix/AF3 vs 15 A AF2)

**Don't use this skill for:**

• ❌ Generating designs — use the **by-design-workflow** skill instead
• ❌ Running liability or developability screening directly — use the **by-screening** skill
• ❌ Choosing which target to design against — use the **by-research** skill
• ❌ Submitting candidates to a lab — use the **by-lab** agent (triple-gated)
• ❌ Predicting raw structures — use the **protenix** skill

---

Quick Start

Compute ipSAE for a single Protenix output:

# JSON confidence file (Protenix /summary_confidence.json)
python scripts/calc_ipsae.py \
    --pae confidence.json \
    --chains confidence.json \
    --design A --target B \
    --pae-cutoff 10.0

# Or run the worked numerical example from references/
python scripts/calc_ipsae.py --example
# Expected: ipsae_min = 0.0396

Rank a panel by composite score:

python scripts/composite_score.py \
    --input scored.csv \
    --output ranked.csv \
    --modality nanobody

# Expected verification:
# ✓ Composite scoring completed: 95 rows -> ranked.csv
#   LAB-READY=10  FOLLOW-UP=18  BORDERLINE=32  NOT-VIABLE=20  FILTER-FAIL=15

Visualize score distributions:

python scripts/plot_score_distributions.py \
    --input ranked.csv \
    --output distributions.pdf \
    --modality nanobody

Via MCP (preferred inside the agent loop):

Tool: mcp__by-screening__score_ipsae
Args: { "npz_path": "/path/to/protenix.npz",
        "design_chain_ids": [0],
        "target_chain_ids": [1] }

---

Explicit Tool Naming

ALWAYS name tools explicitly when discussing workflows:

• "Protenix" for structure prediction/refolding (not "the structure predictor")
• "BoltzGen" for antibody/nanobody design generation (not "the design tool")
• "PXDesign" for de novo binder design (not "the binder generator")
• "ipSAE" by name (not "the scoring metric") — capital S-A-E
• "ipTM" by name (not "the confidence score")

When explaining pipeline stages, always say which tool does each step:

• "BoltzGen generates backbone structures" (not "designs are generated")
• "Protenix refolding validates the designs" (not "structures are validated")
• "ipSAE scores rank the candidates" (not "candidates are scored")

---

Inputs

**Required for scoring a single design:**

• **Protenix NPZ output** (`*.npz`) with `pae` key of shape `[N_sample, N_token, N_token]` and `token_asym_id` key of shape `[N_token]`.
• **OR Protenix confidence JSON** with `pae` (or `predicted_aligned_error`) and `token_chain_ids` keys, plus optional `iptm`, `ptm`, `plddt`.
• **Design chain ID(s)** — integer asym_ids (NPZ) or chain letters (JSON). Convention: antibody/VHH first, antigen last.
• **Target chain ID(s)** — same format as design chains.

**Required for composite ranking:**

• **Scored CSV** with columns: `design_id`, `ipsae_min`, `iptm`. Optional: `liability_count`, `plddt_mean`, `ca_rmsd`, `scaffold`.

**Required for multi-seed scoring:**

• **Directory of Protenix outputs** — one NPZ or confidence JSON per seed. Minimum 20 seeds for VHH/scFv, 10 for de novo.

**Optional:**

• `--pae-cutoff`: 10.0 (default, Protenix/AF3) or 15.0 (AF2).
• `--modality`: `antibody`, `nanobody`, `vhh`, `denovo`, `bispecific`, `peptide`. Switches hard-filter thresholds. See [references/thresholds-by-modality.md](references/thresholds-by-modality.md).
• `--weight-ipsae` / `--weight-iptm` / `--weight-liability`: override default composite weights `(0.50, 0.30, 0.20)`. Weights must sum to 1.0. See [references/composite-score.md](references/composite-score.md).

---

Outputs

**Primary results (per design):**

• `design_to_target_ipsae` (dt_ipsae) — directional ipSAE, design as source frame, in `[0.0, 1.0)`.
• `target_to_design_ipsae` (td_ipsae) — directional ipSAE, target as source frame.
• `ipsae_min` — `min(dt, td)`, the primary ranking metric.
• `iptm` — global inter-chain confidence from Protenix.
• `plddt_mean` — mean pLDDT over the **design chain only** (never over both chains).
• `ca_rmsd` — CA-RMSD between predicted and refolded de