/prompt-injection

name: prompt-injection
description: "Audit applications for AI prompt injection, agent security, and LLM permission boundary vulnerabilities. Use when the user mentions 'prompt injection,' 'LLM security,' 'AI security,' 'jailbreak,' 'indirect prompt injection,' 'prompt leaking,' 'AI red team,' 'LLM vulnerabilities,' 'AI input validation,' 'system prompt extraction,' 'agent security,' 'MCP security,' 'AI permissions,' 'AI privilege escalation,' or needs to secure any application with AI features, AI agents, or LLM integrations."
allowed-tools: Read, Grep, Glob, Bash, Write, WebSearch

Prompt Injection — AI/LLM Security Audit

Audit applications that use AI features, LLM integrations, or AI agents for prompt injection, privilege escalation, and authorization bypass vulnerabilities.

Cross-references: `threat-modeling` for design-time AI risk modeling on new AI features (before this skill applies); `owasp-audit` for the XSS / output-rendering patterns that overlap when LLM output reaches the browser (sanitize on render, JSON-LD breakout); `api-audit` for the API surface that LLM tools and MCP servers expose; `ai-risk-management` for the broader governance frame this skill sits within — prompt injection is the security slice of AI risk; AI RMF covers the rest (fairness, robustness, transparency, drift, lifecycle).

Background

Prompt injection is the #1 vulnerability in LLM-integrated applications (OWASP Top 10 for LLMs, LLM01). It occurs when untrusted input influences the instructions an LLM follows, causing it to ignore its system prompt, leak secrets, or take unauthorized actions.

**Three attack classes:**

• **Direct injection:** Attacker provides malicious input directly to the LLM (e.g., chat input, form field processed by AI)
• **Indirect injection:** Attacker plants malicious instructions in data the LLM will later consume (e.g., web pages, emails, documents, database records, tool outputs, RAG chunks)
• **Cross-privilege injection:** Lower-privileged user plants injection in shared data that a higher-privileged user's AI session consumes, escalating privileges through the AI layer

Methodology

Step 1: Map the AI Attack Surface

Identify every place the application uses AI. This includes direct LLM API calls AND higher-level AI features:

Grep for LLM API calls:
- openai, anthropic, cohere, replicate, ollama
- ChatCompletion, messages.create, generate, complete
- langchain, llamaindex, autogen, crewai

Also look for AI features that may not be obvious LLM calls:
- AI-powered search or recommendations
- AI content generation (summaries, descriptions, emails)
- AI chatbots or copilots embedded in the app
- AI-assisted form completion or auto-fill
- AI moderation or classification
- AI-driven workflow automation
- MCP (Model Context Protocol) servers and tool registrations

For each AI integration, document: 1. **What is the system prompt?** Read it fully. 2. **What user input reaches the prompt?** Trace every variable interpolated into the prompt template. 3. **What external data reaches the prompt?** (RAG results, tool outputs, web scrapes, database records, file contents, emails) 4. **What actions can the LLM take?** (tool/function calls, code execution, database writes, API calls, email sending) 5. **How is the LLM output used downstream?** (rendered as HTML, executed as code, used in SQL, passed to another LLM) 6. **What user role/permissions context does the AI operate under?** (its own service account? the requesting user's session? an admin context?)

Step 2: Audit Prompt Construction

Check how prompts are assembled. Look for:

**Unsanitized interpolation:**

# VULNERABLE — user input directly in prompt
prompt = f"Summarize this: {user_input}"

# VULNERABLE — external data injected without marking
prompt = f"Answer based on this context: {rag_results}"

**Missing input/output boundaries:**

# BETTER — clear delimiters separating instructions from data
prompt = f"""Summarize the text between the <document> tags.
<document>
{user_input}
</document>"""

**Secrets in system prompts:**

# VULNERABLE — API keys, database credentials, or internal URLs in system prompt
system = f"You are a helper. Use API key {API_KEY} to call..."

Check for these patterns:

• User input concatenated or f-string interpolated into prompts without delimiters
• RAG/retrieval results injected without sanitization or boundary markers
• Tool/function outputs fed back into prompts without validation
• System prompts containing secrets, internal URLs, or sensitive business logic
• Chain-of-thought or scratchpad content exposed to the user

Step 3: Audit Output Handling

Check what happens with LLM responses:

**Rendered as HTML (XSS via LLM):**

// VULNERABLE — LLM output rendered as raw HTML
<div dangerouslySetInnerHTML={{ __html: llmResponse }} />

If the LLM can be tricked into outputting `<script>` tags or event handlers, and the output is rendered unsanitized, this is XSS.

**Executed as code:**

# VULNERABLE — LLM output passed to eval/exec
exec(llm_response)

**Used in database queries:**

# VULNERABLE — LLM output used in raw SQL
cursor.execute(f"SELECT * FROM {llm_response}")

**Passed to another LLM (chained injection):** If LLM A's output becomes input to LLM B, an attacker can inject instructions that propagate through the chain.

Step 4: Audit Tool/Function Calling and AI Agents

If the LLM has access to tools, function calls, or operates as an autonomous agent:

**Tool inventory and validation:** 1. **What tools are available?** List every tool/function the LLM can invoke. 2. **Are tool arguments validated?** The LLM may be tricked into passing malicious arguments. 3. **Are destructive tools gated?** (delete, send email, transfer funds, modify records) 4. **Is there human-in-the-loop for high-risk actions?**

# VULNERABLE — LLM can call any tool without validation
result =