/llm-app-audit

description: Black-box audit of a DEPLOYED AI agent (not the MCP server) — tool-call hijacking, cross-session memory poisoning, confused-deputy via connected tools, agent-to-agent IDOR, excessive agency, privilege compromise. Usage: /llm-app-audit <agent-url-or-product>

/llm-app-audit

Audit a live AI agent product as a black box: you are trying to make it *act* with its privileges on someone else's behalf, not just say something. Loads the `skills/agentic-app-audit` playbook.

When to use

• The target is a deployed assistant/agent with tools (bookings, email, payments, file/RAG, browsing).
• You can interact with it but cannot read its MCP server / tool definitions (for that, use `/mcp` + `skills/mcp-server-audit`).
• For pure text-behavior attacks on a chat box, use `/llm-redteam` instead.

What it checks (ASI-mapped)

| Attack | ASI | Deterministic oracle | |---|---|---| | Tool-call hijacking | ASI02 | OOB callback — the tool reaches your collaborator host (`tools/oob_listener.py`) | | Cross-session memory poisoning | ASI06 | plant canary as identity A, re-read as identity B | | Confused-deputy via tools | ASI02/03 | privileged record returned that the user's role can't access | | Agent-to-agent IDOR | ASI07 | agent A returns agent B's context | | Excessive agency | ASI08 | destructive action completes without the required confirmation | | Privilege compromise | ASI03 | invoke a tool outside the user's role |

Supporting tools

tools/llm_redteam.py --url <endpoint> --category excessive-agency --json   # tool-abuse probes
tools/oob_listener.py --listen                                             # confirm tool reach-out
tools/verifiers/ (xss)                                                      # confirm insecure-output XSS at the sink

Turn a hit into a report

ASI alone is **Informational**. Chain to impact: injection → memory poisoning → affects every user; tool misuse → SSRF/IMDS with returned data; insecure output → stored XSS → session theft. Use TWO identities for anything cross-tenant and record the full turn sequence so triage can replay it verbatim.