/rag-data-exposure-audit

name: rag-data-exposure-audit
description: "Assesses document authorization, tenant isolation, raw-chunk fields, citations, caches, exports, and sensitive-source leakage in retrieval systems. Use when an authorized RAG assessment must determine what data each identity can retrieve or disclose."
metadata:
  portability: universal
  discipline: penetration-testing
  domain: ai-systems
  tags: [rag, data-exposure, authorization, multi-tenant, provenance]

RAG data exposure audit

Measure whether retrieval and generation preserve document- and tenant-level authorization. Use controlled fixtures and restricted evidence handling so the audit does not amplify any sensitive material it discovers.

When to Use

• Testing document authorization and tenant isolation in an authorized RAG or

enterprise-search deployment.

• Reviewing raw retrieval fields, citations, exports, caches, logs, or fallback

paths for unintended disclosure.

• Reproducing a suspected source-policy mismatch with controlled documents.

When NOT to Use

• Do not enumerate real sensitive documents or another tenant's corpus without

explicit written authority.

• Do not place real credentials, personal data, or production secrets into test

documents.

• Do not quote discovered sensitive content when a redacted evidence reference

and digest are sufficient.

Exposure states

Keep these states separate: `metadata-visible`, `candidate-returned`, `chunk-retrieved`, `context-present`, `response-disclosed`, `citation-exposed`, `exported`, and `downstream-propagated`. One state does not prove the next.

Workflow

1. Bind the audit to exact tenants, identities, roles, corpora, connectors, routes, allowed data classes, query budget, monitoring, and cleanup duties. 2. Build an authorization matrix from authoritative source policy: identity × document or collection × operation. Preserve unknown policy as unknown. 3. Establish controlled fixtures with unique inert markers across allowed, denied, tenant-specific, revoked, deleted, and stale states where authorized. 4. Record source ACL, ingestion identity, index namespace, metadata filters, retrieval identity, model identity, cache keying, and response channel for each fixture. 5. Establish same-identity baselines for ordinary allowed and denied requests. 6. Exercise only approved identity and query pairs. Capture target-origin query, retrieval, context, citation, response, export, cache, and policy receipts. 7. Test exposure surfaces independently:

• source-to-index ACL propagation;
• tenant and namespace filters;
• raw chunk, metadata, debug, and citation fields;
• query-time versus ingestion-time authorization;
• revoked, deleted, stale, and cached content;
• summaries, follow-up questions, exports, logs, and fallback sources;
• downstream tools or agents that consume retrieved content.

8. Compare observed exposure with the authoritative matrix. Distinguish policy mismatch from stale policy, cache, route, index, or identity ambiguity. 9. Grade each pair `enforced`, `exposed`, `inconclusive`, `blocked`, or `not-tested`. Missing telemetry cannot produce `enforced`. 10. Remove fixtures, verify source and derived-store cleanup, and record residual retrieval or cached disclosure.

Deliverable

Return the scope, authorization matrix, controlled-fixture register, tested identity pairs, exposure-state receipts, policy mismatches, alternative explanations, restricted evidence references, cleanup status, and owned fixes.

Rationalizations to reject

• **“The model refused to quote it, so access control worked.”** Retrieval,

context, citations, or debug fields may already have exposed the content.

• **“The source ACL is correct.”** Verify propagation through connectors,

indexes, filters, caches, and response paths.

• **“A title is harmless metadata.”** Titles and metadata can disclose sensitive

existence, ownership, or project context.

• **“The document was deleted.”** Derived stores and caches require separate

cleanup evidence.

• **“An admin account can see everything, so the result is expected.”** Compare

the tested identity with explicit authorized purpose and tenant scope.

• **“One denied query proves isolation.”** Coverage requires the relevant

identities, routes, fields, and lifecycle states.