/pentest-engagement

name: pentest-engagement
description: Run a professional penetration engagement OR a network vulnerability scan from a scope. WEB mode (apex domains / app URLs) — mandatory surface expansion, systematic OWASP attack-class coverage, reversible active exploitation, authoritative validation, Transilience PDF. NETWORK mode (a list of IPs/CIDRs, e.g. 1500 hosts) — machine-prudent BATCHED tool-first nmap sweep (host discovery + bounded common+less-common port/service scan + CVE surfacing) with a uniform per-IP output tree, then bounded deep-dives on the highest-value hosts. The general (non-CTF) analogue of hackthebox/htb-solve.
context: fork

Pentest Engagement

Orchestrates a scoped pentest end-to-end via the `pentest-engagement` workflow. It is the breadth-complete, coverage-gated counterpart to the flag-shaped `htb-solve` — same engine (`coordinator-loop`, now with **interleaved per-finding validation** built into the loop), but driven by an attack-class **coverage matrix** instead of a flag, with surface expansion and root-cause severity baked in.

When to use

A real (non-CTF) engagement defined by a **scope** — either:

• **WEB** — web / API / cloud apps defined by apex domains / asset URLs, **or**
• **NETWORK** — a list of **IPs / CIDRs / ranges** (e.g. 1500 hosts) to scan for live services and vulnerabilities.

The workflow **auto-detects** the mode in Setup (`engagement_kind`): predominantly IPs/CIDRs → network; apex domains / app URLs → web. For HackTheBox/CTF use `hackthebox` (htb-solve) instead.

Run it

**WEB** — from a scope file or inline:

Workflow('pentest-engagement', { scope_file: 'projects/pentest/<engagement>-scope.md' })
Workflow('pentest-engagement', { scope: { engagement_name, apex_domains:[], assets:[...], creds_env:[...], roe, business_tier } })

**NETWORK** — inline IP/CIDR list or a scope file containing one (a plain newline list of IPs/CIDRs is accepted):

Workflow('pentest-engagement', { targets: ['10.0.0.0/24', '192.0.2.0/24', '198.51.100.7'] })
Workflow('pentest-engagement', { scope_file: 'projects/pentest/<engagement>-ips.txt', scan_profile: 'standard' })

Options (shared): `maxConcurrent` (default = **prudent, derived from CPU cores** — ~half the cores, capped 2–8; never hundreds/thousands of parallel tasks), `dryRun`, `max_experiments`, `business_tier`, `report` (default true). Options (network): `scan_profile` `light` (bounded `1-1024` + curated less-common, for large/fast sweeps) / **`standard`** (full-range `-p-` on every reachable host, DEFAULT; two-stage SYN→`-sV` on found-open ports, host-count-guarded, message-bus + non-443-TLS aware) / `full` (`-p-` + bounded UDP); `udp` (top-50 UDP; off for light/standard, auto-on for full); `slice_size` (hosts per scan worker, auto ≈64 IP-list / 2 CIDR-heavy); `deepen_top` (deep-dive the N highest-value hosts, default 10, `0` to skip); `geo_vantages` (≤2 gcp zones for the 2nd-vantage allowlist re-probe; overrides the US+EU default), `auto_provision` (default true; `false` = detect+flag only, no cloud spend). On a source-IP/geo-allowlist signature the workflow auto-provisions a 2nd-geography vantage and re-probes the filtered hosts before concluding "no surface."

Write the scope file per [`reference/scope-file-format.md`](reference/scope-file-format.md). Credentials are referenced by **env-var name only** and read from the repo `.env` via `python3 tools/env-reader.py` — never inline secret values.

Phases (what the workflow does)

1. **Setup** — `env-reader` creds, parse scope, **classify kind** (web|network), **read CPU cores → prudent parallel-task cap**, `OUTPUT_DIR = projects/pentest/<date>_<engagement>/`, STARTED Slack (gated). 2. **Expand** *(WEB, the #1 fix)* — MANDATORY CT-log / passive-DNS / origin-discovery across every in-scope apex (`crt.sh`, `certspotter`, `subfinder`, origin-discovery for CDN/WAF-fronted hosts). Scope = the **discovered** surface, not the handoff. Builds the per-asset work list + seeds each asset's coverage matrix. **Scan** *(NETWORK, replaces Expand)* — slice the IP/CIDR set into machine-prudent batches; **one nmap worker per slice** (agents scale with *slices* ≈ dozens, never with IP count) runs the SAME pipeline: host discovery (reachability is unknown) → bounded common+less-common port/service scan → CVE surfacing (`nmap --script vulners`, `nuclei`, each CVE-ID enriched via `tools/nvd-lookup.py`) → writes a **uniform per-IP tree** `hosts/<ip>/{recon,host.json,findings}` + a merged `recon/inventory/`. 3. **Assess** *(single interleaved stage — no separate downstream validation pass)* — WEB: each asset → `coordinator-loop` (**coverage mode**), which validates **each candidate the instant it is materialized** on fresh blind agents (strict per-finding cure/drop loop) before search continues. NETWORK: bounded `coordinator-loop` deep-dives on **only** the `deepen_top` highest-value hosts, same interleaved per-finding validation (everything else is the uniform tool-scan, not a per-host agent). **Coverage-by-VALID:** a class is covered only by a `VALID`/`REPAIRED` finding, a justified N/A, or a genuine negative — a class whose candidates were all rejected/dropped stays `pending` and search keeps going. 4. **Correlate** — `attack-path-stitcher` + `risk-prioritiser` across all validated findings → ranked org roadmap. 5. **Report** *(deterministic — no agent authors the report)* — JS hands the resolved engagement block + exact commands to ONE finalize runner: `tools/report_data_build.py` merges the namespaced interim finding-JSONs into the canonical `report_data.json` (the sole-owner assembly), then the **format-dispatched** renderer runs — `transilience` → the canonical `generate_report.py` PDF skill, `custom` → `custom_report_cmd` (or a Markdown fallback). JS then **hard-gates**: `report_data` assembled ∧ (transilience: `WROTE` ∧ `bytes>0` ∧ the `[assets: …/formats/transilience-report-style]` provenance tag), retry-once → `BLOCKED`. Only `VALID`/`REPAIRED` fin