agent-opfor

OPFOR is short for Opposition Force — a military term for the unit that plays the enemy in training, so the rest of the army learns what real attacks feel like before they come. We named the tool after that idea: to defend AI agents better, you have to attack them first.

Why KeyValue built this

We've shipped 130 products for 90 startups over the last ten years. In the last 18 months, almost every one of them had an AI agent in it — and every one of those teams hit the same wall when it came to testing.

So we built OPFOR. For ourselves first. Now open source.

Apache 2.0. Built from India.

Quick Start

npm install -g @keyvaluesystems/agent-opfor-cli
export OPENAI_API_KEY=your-key    # or GEMINI_API_KEY, ANTHROPIC_API_KEY, etc.

One-shot — runs the setup wizard and immediately starts the scan:

opfor run

Two-step — save a config you can reuse or commit to CI:

opfor setup                               # wizard saves a config to .opfor/configs/
opfor run --config .opfor/configs/<file>  # run any time against the saved config

https://github.com/user-attachments/assets/a6a3cff2-2cf9-4486-944e-ac0163e7ea04

What opfor does

Opfor red-teams the full AI agent surface — prompts, tools, MCP servers, memory, and multi-turn reasoning. It generates targeted attacks for OWASP LLM Top 10, OWASP Agentic AI Top 10, OWASP MCP Top 10, OWASP API Security, and EU AI Act bias suites, fires them at your target, and judges each response with an LLM.

Most red-team tooling in this space is excellent at one thing — a probe library, a developer evaluator, a programmatic framework. Opfor covers more ground in one tool:

• Browser extension for non-developers — anyone on your team can red-team a deployed chatbot, no code, no env vars, no YAML
• Run opfor as an MCP server — let your AI coding agent in Cursor or Claude Desktop red-team your other agents through natural language
• Full OWASP coverage in one tool — LLM Top 10, Agentic AI Top 10, MCP Top 10, API Security Top 10
• No black box — every attack prompt, request, response, and judge verdict is logged; reproducible, auditable, forkable
• Built for agents, not just models — designed for tool calls, MCP, memory, and multi-turn state from day one
• Trace-aware — integrates with Langfuse and Netra so the LLM judge sees what your agent did internally, not just what it said

Five ways to run opfor

Different people on your team need different entry points. Opfor ships five.

All five share the same evaluators, attack templates, and judge logic.

→ CLI reference · Browser extension setup · MCP setup · Skills setup · SDK reference · Session handling

How it works

When you run a scan, opfor:

1. Fetches target info — connects to your agent, detects available tools, MCP endpoints, capabilities
2. Plans attacks per category — generates targeted prompts for each evaluator in your selected suite
3. Emulates the attack — runs multi-turn adversarial conversations (real requests, real responses)
4. Evaluates with a judge — an LLM judge classifies each response with pass/fail + reasoning
5. Generates a report — HTML for browsing, JSON for CI/CD, all artifacts logged for reproducibility

Each run lands in its own subfolder under .opfor/reports/run-report-<compactTs>-<slug>-<shortId>/ containing <slug>-report.html and <slug>-report.json. Autonomous opfor hunt runs use the same layout under hunt-report-<compactTs>-<slug>-<shortId>/.

Testing cost

Every run reports what its instrumented LLM calls cost, broken down by model:

Token usage: 51,323 input / 6,057 output (57,380 total)
Testing cost: $0.18
   deepseek/deepseek-v4-pro [attacker]: $0.037
   anthropic/claude-opus-5 [judge]: $0.14

This is opfor's own spend — the attacker and judge LLMs. It excludes your target's inference cost, which opfor cannot see from the outside. The per-model split is the useful part: the judge is often the bigger share, and pointing it at a cheaper model is usually the easiest saving.

Prices come from a snapshot of LiteLLM's public price map that ships with the package, so runs work offline and a report re-rendered later produces the same figure. Cached input is billed at the provider's cache rate — multi-turn attacks re-send the conversation each turn, and that repeated prefix is often ~100× cheaper than fresh text, so the figure tracks the real bill rather than a worst case. Caveats worth knowing:

• Caching is only credited when it's reported. A provider that doesn't break out cached tokens, or a model with no published cache rate, is charged at the full input rate — an over-estimate, chosen over quietly under-reporting.
• Unknown models are never counted as free. A model missing from the price table is reported as unpriced and the total is marked a lower bound, rather than silently reading as $0.
• A few helper calls aren't metered yet. Trace curation, session summarisation and one JSON helper don't report token usage, so their spend is missing from the total. Treat the figure as a floor.

→ Token usage and testing cost

Evaluator coverage

Opfor ships with curated suites that map to industry standards. Pick a suite or run individual evaluators.

→ Full evaluator reference and OWASP mapping

Trace-aware testing

Plug opfor into your observability stack and the LLM judge sees not just the final response — but every tool call, retrieval, and intermediate reasoning step. Out of the box, opfor integrates with Langfuse and Netra.

"telemetry": {
  "provider": "langfuse",
  "langfuse": { "baseUrl": "https://cloud.langfuse.com" }
}

This catches what input/output testing misses — PII that leaks into a tool call but never reaches the user, scope escalations in MCP that don't change the response text, agents that retrieve unauthorized data but render a clean reply.

→ Trace-aware testing guide

Autonomous Red-Teaming

opfor hunt skips the config file entirely. Give it an endpoint and an objective, and a multi-agent system — commander, operators, scout — runs an adaptive attack campaign on its own: recon, strategy, multi-turn probing, report. Unlike opfor run, the agents run on Claude only (via a Claude API key, claude setup-token, or your local claude login session) — your target can be anything.

opfor hunt \
  --endpoint "https://your-agent.com/v1/chat" \
  --objective "Find jailbreaks, system-prompt leakage, and safety bypasses."

Add --ui to watch the attack tree unfold in a live dashboard.

→ Full reference

Browser extension — red-team a chatbot

The browser extension is opfor's no-code path. Install from the Chrome Web Store, open any chat interface, click the opfor icon, pick a suite, and watch it run.

https://github.com/user-attachments/assets/80c2692f-b18b-4899-99df-e7eb8d50b02a