Is a RAG security checklist enough on its own?

It covers retrieval. It does not cover model memorization, corpus tampering, or the build pipeline, each of which needs its own assessment.

Guides › Secure RAG › Checklist

AI Knowledge Base Security Checklist

Hunter Moore, Founder and CTO · Updated August 4, 2026

A stage-by-stage checklist for an AI knowledge base handling sensitive content. Classic RAG over a vector store is the most common shape, but the stages below apply to any corpus your AI retrieves from at query time. The controls are grouped by where each one sits, because position determines what a control can catch. If you only do three things, do the ones marked as prerequisites.

At a glance

Grouped by stage. Every control is listed where it sits in the pipeline, because position determines what it can catch.
Prerequisites come first. Provenance logging, an index inventory and a baseline leak rate are what make every other item verifiable.
Retrieval closes the most. Constraining the candidate set by entitlement before similarity search removes the largest category of leak events.

Applies to any AI knowledge base

None of this is specific to one architecture. It shows up in classic RAG over a vector store, in agentic retrieval where the model writes its own queries and decides what to fetch, in GraphRAG and hybrid keyword-plus-vector setups, in MCP and connector-based assistants wired into SharePoint, Confluence or Drive, and in enterprise AI search platforms like Microsoft Copilot and Glean. If a model sees internal documents at query time, there's a knowledge base behind it, and it can return content the person asking was never entitled to.

Agentic setups tend to be worse rather than better. When the model writes its own queries, the set of documents it can reach is wider than anything a product spec described, and usually nobody has written down what it should never touch.

Prerequisites

Log retrieval provenance. Record which chunks were returned, for which query, to which identity, at what time. Everything else on this list is unverifiable without it.
Know what is in the index. Maintain an inventory of source systems, connectors and the identity each connector authenticates as.
Measure a baseline leak rate. Run the self-test before changing anything, so improvements are demonstrable.

Twenty-nine items in six groups. The grouping is the point: a control is only as good as its position in the pipeline.

Ingestion

Run an explicit eligibility gate before indexing, and record the decision with the artifact.
Parse untrusted files in a worker holding no ambient credentials and no metadata service access.
Disable template evaluation in data fields and constrain external references in formats that support them.
Scan for hidden text in metadata, alt attributes and comment fields, which survive extraction but are invisible to a human reviewer.
Transform sensitive content rather than removing it, so downstream copies inherit the reduced exposure without breaking the documents.
Record provenance for every artifact, and pin external content to a digest rather than a mutable URL.

Common mistake

Starting here. Ingestion controls are the durable fix, but they are a project rather than a task, and without provenance logging and a baseline leak rate measured first there is no way to demonstrate that anything improved.

Chunking and embedding

Propagate permission metadata from document to every chunk. Storing only a source URI leaves the retriever nothing to filter on.
Default unlabeled content to closed and surface it for review rather than treating it as public.
Register chunk identifiers that match exactly what the retrieval path reports, since that string is the join key for every later audit.
Treat the embedding store as a second copy of the content, not as a derived artifact with lower sensitivity.

Retrieval

Filter before scoring. Constrain the candidate set by entitlement before similarity search, not after. Post-filtering degrades ranking and leaks existence through counts and latency.
Resolve entitlements against the requesting identity on every query, never against the indexing service account.
Refresh permissions on ACL change events, not on the re-index schedule. Offboarding is the case that matters.
Re-run the leak test after every re-index, because chunk permission metadata frequently fails to survive a rebuild.

Context boundary

Mark retrieved content as data and keep it syntactically separate from instructions.
Put tool calls that a retrieved document appears to have triggered on a different authorization path than user-initiated ones.
Treat every writable source that feeds the index as an injection surface, including ticket queues and shared mailboxes.
Assume some injections get through the classifier and size the blast radius accordingly.

Derived copies

Inventory caches, conversation history, agent memory, evaluation sets and debug logs. Each inherits content and rarely inherits permissions.
Confirm deletion propagates from the source system through to the index. Orphaned chunks are a common finding during a data subject request.
Apply retention to conversation history holding retrieved content, on the same clock as the source data.

A checklist that is not measured decays. Fixing the inputs makes the leak rate comparable over time, which is what turns the work into something reportable.

Governance

Name an owner for the retrieval pipeline who has a security reporting line. This usually sits unowned between data engineering, security and the ML team.
Gate changes to parsing, chunking and retrieval logic behind review.
Keep the entitlement matrix and query set fixed so the leak rate stays comparable across runs.
Map controls to the frameworks you will be assessed against, principally EU AI Act Articles 10 and 12, NIST AI RMF and ISO/IEC 42001.

PrerequisitesLog retrieval provenance, inventory the index, and measure a baseline leak rate. Usually days of work, and it is what makes everything after it measurable.
RetrievalFilter before scoring and resolve entitlements against the requesting identity. Typically weeks, and the effort goes into getting current permission attributes onto every chunk.
IngestionEligibility gating, isolated parsing and content transformation. A project rather than a task, and the durable fix that reduces what a future failure costs.

Frequently asked questions

What order should we work through this in?

Prerequisites first, then retrieval, then ingestion. Logging makes everything measurable, retrieval filtering closes the largest category of leak events, and ingestion controls are the durable fix that reduces what a future failure costs.

How long does a first pass take?

Provenance logging is usually days. Moving to pre-filtering is typically weeks, and the work is rarely in the database, it is in getting current permission attributes onto every chunk. Ingestion transformation is a project rather than a task.

Which items map to EU AI Act obligations?

Provenance logging and retention map to Article 12's automatic logging requirement. Eligibility gating, permission propagation and ingestion controls map to Article 10 on data governance. Injection resistance and pipeline integrity sit under Article 15.

Is this checklist enough on its own?

It covers retrieval. It does not cover model memorization, which is training data leakage, corpus tampering, which is data poisoning, or the build pipeline, which is AI data pipeline security.

Sources

European Union, Regulation (EU) 2024/1689, the AI Act, Articles 10, 12 and 15. · NIST, AI Risk Management Framework. · ISO/IEC, ISO/IEC 42001.