Guides › Secure RAG › Checklist

RAG Security Checklist

A stage-by-stage checklist for a retrieval pipeline handling sensitive content. Grouped by where each control sits, because position determines what a control can catch. If you only do three things, do the ones marked as prerequisites.

Prerequisites

• Log retrieval provenance. Record which chunks were returned, for which query, to which identity, at what time. Everything else on this list is unverifiable without it.
• Know what is in the index. Maintain an inventory of source systems, connectors and the identity each connector authenticates as.
• Measure a baseline leak rate. Run the self-test before changing anything, so improvements are demonstrable.

Ingestion

• Run an explicit eligibility gate before indexing, and record the decision with the artifact.
• Parse untrusted files in a worker holding no ambient credentials and no metadata service access.
• Disable template evaluation in data fields and constrain external references in formats that support them.
• Scan for hidden text in metadata, alt attributes and comment fields, which survive extraction but are invisible to a human reviewer.
• Transform sensitive content rather than removing it, so downstream copies inherit the reduced exposure without breaking the documents.
• Record provenance for every artifact, and pin external content to a digest rather than a mutable URL.

Chunking and embedding

• Propagate permission metadata from document to every chunk. Storing only a source URI leaves the retriever nothing to filter on.
• Default unlabeled content to closed and surface it for review rather than treating it as public.
• Register chunk identifiers that match exactly what the retrieval path reports, since that string is the join key for every later audit.
• Treat the embedding store as a second copy of the content, not as a derived artifact with lower sensitivity.

Retrieval

• Filter before scoring. Constrain the candidate set by entitlement before similarity search, not after. Post-filtering degrades ranking and leaks existence through counts and latency.
• Resolve entitlements against the requesting identity on every query, never against the indexing service account.
• Refresh permissions on ACL change events, not on the re-index schedule. Offboarding is the case that matters.
• Re-run the leak test after every re-index, because chunk permission metadata frequently fails to survive a rebuild.

Context boundary

• Mark retrieved content as data and keep it syntactically separate from instructions.
• Put tool calls that a retrieved document appears to have triggered on a different authorization path than user-initiated ones.
• Treat every writable source that feeds the index as an injection surface, including ticket queues and shared mailboxes.
• Assume some injections get through the classifier and size the blast radius accordingly.

Derived copies

• Inventory caches, conversation history, agent memory, evaluation sets and debug logs. Each inherits content and rarely inherits permissions.
• Confirm deletion propagates from the source system through to the index. Orphaned chunks are a common finding during a data subject request.
• Apply retention to conversation history holding retrieved content, on the same clock as the source data.

Governance

• Name an owner for the retrieval pipeline who has a security reporting line. This usually sits unowned between data engineering, security and the ML team.
• Gate changes to parsing, chunking and retrieval logic behind review.
• Keep the entitlement matrix and query set fixed so the leak rate stays comparable across runs.
• Map controls to the frameworks you will be assessed against, principally EU AI Act Articles 10 and 12, NIST AI RMF and ISO/IEC 42001.

Frequently asked questions

What order should we work through this in?

Prerequisites first, then retrieval, then ingestion. Logging makes everything measurable, retrieval filtering closes the largest category of leak events, and ingestion controls are the durable fix that reduces what a future failure costs.

How long does a first pass take?

Provenance logging is usually days. Moving to pre-filtering is typically weeks, and the work is rarely in the database, it is in getting current permission attributes onto every chunk. Ingestion transformation is a project rather than a task.

Which items map to EU AI Act obligations?

Provenance logging and retention map to Article 12's automatic logging requirement. Eligibility gating, permission propagation and ingestion controls map to Article 10 on data governance. Injection resistance and pipeline integrity sit under Article 15.

Is this checklist enough on its own?

It covers retrieval. It does not cover model memorization, which is training data leakage, corpus tampering, which is data poisoning, or the build pipeline, which is AI data pipeline security.