Guides › Secure RAG › Preventing leakage

How to Prevent RAG Data Leakage

Hunter Moore, Founder and CTO · Updated August 4, 2026

Seven controls prevent a retrieval pipeline from returning content the person asking should not see. They are listed in pipeline order, because where a control sits determines what it can and cannot catch, and most teams reach for the last one first.

At a glance
  • Order matters. The seven controls are listed in pipeline order, because where a control sits determines what it can and cannot catch.
  • Retrieval carries the most weight. Per-query entitlement enforcement against the requesting identity is the one control nothing else substitutes for.
  • Ingestion is second. Handle content before text becomes an embedding, because every downstream copy inherits whatever was ingested.
  • Output filters come last. Most teams reach for them first, and by then the content has already been retrieved and put in a prompt.

Which control matters most?

Per-query entitlement enforcement at retrieval. Nothing else substitutes for it. If the retriever does not evaluate the requesting identity against chunk-level permissions at the moment of the query, every other control is compensating for a decision that was never made.

Every other control is compensating for a decision that was never made.

The second most important is handling content at ingestion, before text becomes an embedding, because every downstream copy inherits whatever was ingested.

WHERE EACH CONTROL HAS TO SIT TO WORK Ingestion Eligibility gate before indexing Parse in a credential-free sandbox Chunk and embed Propagate the source ACL to every chunk Transform sensitive values before embedding Retrieval Filter by entitlement before scoring Log which chunks were returned to whom Context boundary Keep retrieved text syntactically separate from instructions A control placed downstream of the stage it is meant to govern can only report, not prevent. This is why response filtering does not substitute for retrieval filtering, and why retrieval filtering does not substitute for ingestion.
Seven controls, positioned by stage. Position determines capability: a control can only act on data that has not already passed it.

1. Decide what is eligible before you index

Not everything in a corpus belongs in a retrieval index. Run the eligibility decision as an explicit gate, record it alongside the artifact, and make the record auditable. Deciding later means deciding after the embedding already exists.

2. Transform sensitive content rather than removing it

Removing sensitive values breaks documents. A contract with holes in it stops answering questions about the contract, which is why exclusion lists get quietly relaxed six weeks after launch.

Transformation modifies the data so sensitive content cannot be encoded into weights or surfaced through retrieval, while structure, relationships and analytical shape survive. Applied at ingestion, the chunk, the embedding, the cache entry, the summary and any future fine-tuning example all inherit it. This is the layer Hardshell operates on, inside the customer's environment and upstream of the index.

Removing the values

what you have to maintain

  • An exclusion list that someone has to keep current.
  • A fresh decision every time a new source is added.
  • Pressure to narrow the list each time an answer comes back incomplete.

Transforming the content

what you have to maintain

  • One set of rules, applied before anything is indexed.
  • New sources pick up the same rules without a separate decision.
  • No incomplete answers, so no pressure to widen what is exposed.

3. Carry permission metadata through chunking

When a 90-page document becomes 400 chunks, each chunk needs the access decision that governed the original. Many pipelines store only the source URI, which leaves the retriever with no attribute to evaluate.

Watch for content that never had meaningful permissions to inherit. Scanned PDFs in a shared folder, exports in object storage, channels that were public by default. A pipeline that only propagates existing labels has nothing to work with there, and those files are frequently the most sensitive in the corpus.

4. Enforce entitlements as a pre-filter, not a post-filter

Common mistake

Post-filtering retrieves the top-k chunks and then drops the ones the user cannot access. It is the common shortcut and it leaks three ways. Ranking quality degrades because the good results were discarded after selection. A reranking step often still processes the dropped text. And result counts or latency differences can confirm the existence of documents the user was never supposed to know about.

A pre-filter constrains the candidate set before similarity search, so unentitled chunks are never scored. The mechanics are covered in RAG permissions.

5. Refresh permissions as fast as they change

Access control lists copied at ingestion go stale the moment someone changes teams or a document is reclassified. Nightly re-indexing leaves a day in which a revoked user still matches chunks they can no longer open. Offboarding is the sharp case, because the source system revokes instantly and the index does not. Event-driven updates on ACL changes are the minimum; live entitlement checks at query time are better.

6. Treat retrieved content as untrusted at the context boundary

Retrieved documents should be structurally separated from instructions and marked as data for the life of the request. Tool calls that a retrieved document appears to have triggered deserve a different authorization path than tool calls the user made. That failure mode is indirect prompt injection, and it is a separate problem from entitlements.

ORDER OF WORK Phase 1 Retrieval provenance logging typical elapsed time: days Records which chunks went to which identity, for which query. Prevents nothing on its own. Closes The measurement gap Phase 2 Entitlement pre-filtering typical elapsed time: weeks Resolves permissions against the person asking, before similarity search runs. Closes Most live leak events Phase 3 Ingestion transformation typical elapsed time: a project Reduces what a chunk carries, so the next entitlement mistake costs less. Closes What the residue is worth Each phase depends on the one before it. Pre-filtering without logging cannot be shown to have worked.
Sequencing matters more than completeness. Logging makes the problem visible, retrieval filtering closes most of it, and ingestion work decides what the remainder is worth.

7. Log provenance, then measure

Record which chunks were retrieved for which query by which identity. Without that you cannot audit, investigate, or answer a regulator. With it you can run a leakage self-test and track a leak rate over time.

Output inspection belongs here too, as a backstop rather than a control. By the time text reaches a response filter the sensitive content has already been retrieved, placed in a prompt, and processed by a model that may sit outside your boundary.

  1. Decide what is eligible before you indexRun the decision as an explicit gate, recorded alongside the artifact and auditable.
  2. Transform sensitive content rather than removing itStructure, relationships and analytical shape survive, and downstream copies inherit the change.
  3. Carry permission metadata through chunkingEach chunk needs the access decision that governed the original document.
  4. Enforce entitlements as a pre-filterConstrain the candidate set before similarity search, so unentitled chunks are never scored.
  5. Refresh permissions as fast as they changeEvent-driven updates on ACL changes are the minimum; live checks at query time are better.
  6. Treat retrieved content as untrustedSeparate documents from instructions and mark them as data for the life of the request.
  7. Log provenance, then measureRecord which chunks were retrieved for which query by which identity, then track a leak rate.

Frequently asked questions

What is the fastest meaningful improvement?

Turn on retrieval provenance logging, then move entitlement enforcement from post-filter to pre-filter. The first takes hours and unlocks every audit you will ever run. The second closes the largest single category of leak events.

Can we just exclude sensitive documents from the index?

You can, and it usually makes the assistant useless for the questions people actually ask. Exclusion also tends to erode, because the first team that cannot get an answer requests an exception. Transformation keeps the documents retrievable while changing what they can expose.

Does row-level security in the vector database solve this?

It helps if the filter is applied before similarity search and the permission attributes on each chunk are current. Both conditions fail often. The database enforces whatever attributes you wrote; it has no view of whether those attributes still match the source system.

How do we prove any of this to an auditor?

With provenance logs and a repeatable measurement. Article 12 of the EU AI Act requires automatic event logging for high-risk systems, and Article 10 covers data governance. A leak rate produced by a documented method, re-run on a schedule, is the kind of evidence that satisfies both.

Sources

European Union, Regulation (EU) 2024/1689, the EU AI Act, 2024. Article 10 on data governance and Article 12 on record-keeping.