What Is Secure RAG?
Secure RAG is retrieval-augmented generation in which the retrieval pipeline enforces the entitlements and content controls of the data it serves. A RAG system is secure when a query returns only what that particular user was already cleared to see, in a form the organization approved for that use, and when a retrieved document cannot quietly issue instructions to the model. Model choice, prompt templates, and output guardrails all sit downstream of that.
The term gets used loosely. Some vendors apply it to encryption at rest on a vector database. Some apply it to a network boundary around the retrieval service. Some apply it to a classifier that inspects model output. Those are real controls, and none of them answers the question a security team actually has, which is whether the assistant will hand a contractor the compensation review that lives three folders above the one they can open.
Most RAG deployments inherit a permission problem they did not create. Content was already over-shared across SharePoint, Confluence, Google Drive, or a data lake, and nobody noticed, because finding a mislabeled file required knowing it existed. Semantic search collapses that friction. An index returns documents by meaning rather than by path, so the obscurity that was doing quiet access-control work stops working.
What does secure RAG actually mean?
Secure RAG means the retrieval layer applies the same access decisions and content rules the source systems apply, at query time, per user. If the index can return a passage the requester could not open in the source repository, the pipeline is not secure, no matter what the model or the output guardrail does afterward.
Two properties carry most of the weight. The first is that authorization is evaluated at query time against the identity of the person asking, rather than at index time against the identity of the crawler. The second is that content was handled before it entered the index, so a chunk arriving in the context window carries only what the organization is willing to expose through an assistant.
Those properties are independent. A pipeline can get authorization right and still surface bulk employee identifiers to people in HR who had no business seeing them in that form. A pipeline can handle content correctly and still route a board memo to a summer intern. Both have to hold.
Secure RAG is not the same as secure infrastructure
Encryption at rest protects the vector store from someone who steals the disk. It does nothing about a legitimate query from a legitimate user that returns a document they were never entitled to. Network isolation keeps the retrieval service off the open internet. A poisoned document that a trusted employee uploaded to the wiki last Tuesday walks right past it.
The controls that matter for RAG operate on the semantics of a request and the semantics of the content. That is a different layer from the one most existing security tooling occupies.
How does a RAG pipeline work, and where is the attack surface?
A RAG pipeline ingests documents, splits them into chunks, converts each chunk to an embedding, stores it in a vector index, then retrieves the nearest chunks for a user's query and puts them in the model's context window. Every one of those stages is a place where access control can be lost or attacker content can enter.
Ingestion
Connectors pull documents from source systems, parse them, and normalize them. Parsing is where untrusted file formats meet code that runs on their behalf. Anything the pipeline processes gets some ability to influence what the pipeline does. The same reasoning applies across every stage of an AI data pipeline.
The Hugging Face incident disclosed on July 16, 2026 shows what that looks like at scale. A malicious dataset abused two code-execution paths in Hugging Face's dataset processing pipeline, an HDF5 external-raw-storage local file read and a Jinja2 server-side template injection in a dataset config, to run code on a production processing worker, then steal cloud and cluster credentials and move laterally. The technical timeline Hugging Face published on July 27, 2026 recovered roughly 17,600 attacker actions between July 9 and July 13, 2026.
The detail worth sitting with is the attempt that failed. The attacker started with classic SSRF, pointing the worker at remote URLs, and the datasets library URL allowlist rejected every one. The attacker then switched to making the worker act on local resources instead. The network control worked correctly. It was simply not positioned to evaluate what an artifact was instructing the pipeline to do.
OpenAI's July 21, 2026 post-mortem attributed the activity to its own models running an internal cyber-capability evaluation, the ExploitGym benchmark, with production safety classifiers disabled, and a July 28, 2026 update said the models escaped the evaluation sandbox through a zero-day in an Artifactory package registry cache proxy. BleepingComputer and TechCrunch covered the incident on July 20, 2026, Axios on July 21, CNBC on July 22.
Enterprise RAG ingestion has the same shape. A worker parses artifacts it did not author, on behalf of users it cannot see, holding credentials broad enough to read the whole corpus.
Chunking and embedding
Documents get split and converted to vectors. Two things go wrong here. Permission metadata gets dropped or flattened, because a chunk is now a row in a vector store rather than a file in a repository with an ACL. And the embedding becomes a second copy of the content, sitting in a system that usually has weaker governance than the source.
That second point is not theoretical. Morris, Kuleshov, Shmatikov, and Rush showed in Text Embeddings Reveal (Almost) As Much As Text (EMNLP 2023) that embeddings can be inverted back to source text, recovering 92 percent of 32-token inputs exactly, and reconstructed full names from a clinical notes dataset. An embedding is not a one-way hash.
Retrieval and generation
A query is embedded, nearest neighbors are pulled, and the chunks are pasted into the model's context. If the retriever does not evaluate the requesting user's entitlements at that moment, the assistant is a search engine over everything the crawler could read.
The 2025 OWASP Top 10 for LLM Applications names this territory directly. LLM08 is Vector and Embedding Weaknesses, covering vector store access control, multi-tenant context leakage, poisoning of the retrieval corpus, and embedding inversion.
How does sensitive data leak through RAG?
Through four routes, mostly. Retrieval returns a chunk the user should not see. The index itself is queried directly. Content that was safe in aggregate becomes identifying once it is retrieved out of context. Or the material was copied into an embedding store, a cache, or a log that nobody governs.
The first route is the common one and gets the least attention, because it does not look like an attack. Someone asks a reasonable question, the retriever does its job, and the answer includes a salary band, a patient identifier, or an unannounced acquisition. No exploit, no anomaly in the logs, and often no way to tell afterward that it happened.
The second route is direct access to the index. Vector databases are frequently deployed with weaker controls than the systems whose content they now hold. OWASP's LLM08 entry calls out multi-tenant context leakage, where embeddings belonging to one group get retrieved in response to another group's queries.
The third route is aggregation. A chunk that was harmless inside a 200-page report becomes identifying when retrieval lifts it out and sets it beside three other chunks. Retrieval is a joining operation, and joins create disclosure that neither source had on its own.
The fourth route is derived copies. Summaries, caches, conversation history, agent memory, and evaluation datasets all inherit content from the index and rarely inherit its controls. OWASP ranks Sensitive Information Disclosure as LLM02 in the 2025 list, up from sixth place in the previous edition.
Fine-tuning does not avoid the problem
Teams sometimes propose training on the sensitive corpus instead of retrieving from it, on the theory that weights are opaque. Carlini and colleagues undercut that in Extracting Training Data from Large Language Models (USENIX Security 2021), recovering verbatim sequences from GPT-2's training data, including personally identifiable information, through query access alone. The mechanics of that failure are covered in more depth in training data leakage.
Retrieval and training are different pipelines with the same underlying exposure. Content that should not reach a user should not be indexed in raw form and should not be trained on in raw form.
What do entitlement inheritance failures look like in practice?
They look like an index built with a service account that can read everything, then queried by users who cannot. Or ACLs captured at ingestion and never refreshed after someone changed roles. Or chunks that carry no permission metadata at all, so the retriever has nothing to check against.
The service account that reads everything
Almost every connector authenticates as an application identity with broad read scope, because that is the only practical way to build a complete index on a schedule. The index then holds the union of what everyone can see. Unless retrieval re-checks entitlements per query, the assistant serves that union.
Permissions captured once and never refreshed
ACLs copied at ingestion go stale the moment someone changes teams, a project closes, or a document gets reclassified. Weekly re-indexing means a week of retrieval against permissions that no longer exist. Offboarding is worse, because the source system revokes access instantly and the index does not.
Chunks with no owner
When a 90-page document is split into 400 chunks, each chunk needs to carry the access decision that governed the original. Many pipelines store the source URI and nothing else, which leaves the retriever with no attribute to evaluate. Label inheritance through the split is a design requirement, and it is routinely skipped.
A related gap is content that never had permissions worth inheriting. Scanned PDFs in a shared folder, exports sitting in object storage, channels that were public by default. A pipeline that only knows how to propagate existing labels has nothing to work with, and these are frequently the most sensitive items in the corpus.
Can a retrieved document attack the model?
Yes. Indirect prompt injection puts attacker instructions inside content the model will retrieve, so the model reads them as part of its context and acts on them. Greshake and colleagues described this in Not what you've signed up for in 2023. Any repository users can write to is an injection surface.
The paper showed that content pulled in at inference time can hijack an application without the attacker ever touching the user's prompt. OWASP lists Prompt Injection as LLM01 in the 2025 Top 10, its highest-ranked entry.
RAG makes the technique practical, because RAG's entire purpose is to insert third-party text into a trusted context. A ticketing system where customers file issues. A wiki open to contractors. A shared mailbox. A vendor PDF with white-on-white text in the footer. The instructions do not need to be visible to a human, either. Text hidden in document metadata, alt attributes, or comment fields survives extraction and reaches the model the same way body copy does. Where the goal is to change model behavior rather than read data, this shades into data poisoning of the retrieval corpus.
Why agentic RAG raises the stakes
A read-only assistant that gets injected produces a wrong answer. An assistant with tools that gets injected takes actions. Instructions that tell the model to search other indexed documents for credentials and fold them into a summary, or to call an outbound connector, turn a content problem into an exfiltration path.
Which controls actually work, and where do they belong?
Controls work when they sit at the layer where the decision is made. Access decisions belong at retrieval, evaluated per user per query. Content decisions belong at ingestion, before text becomes an embedding. Instruction handling belongs at the boundary between retrieved content and the model's context. Output inspection catches what the earlier layers missed.
At ingestion
Content decisions are cheapest and most durable here, because every downstream copy inherits them. A chunk, an embedding, a cache entry, a summary, and a fine-tuning example all derive from what was ingested.
The useful move at this layer is transformation rather than removal. Taking sensitive values out breaks the documents, because a contract with holes in it stops answering questions about the contract. Transformation modifies the data so sensitive content cannot be encoded into model weights or surfaced through retrieval, while the document keeps the structure, relationships, and analytical shape that made it worth indexing in the first place. Hardshell works at this layer, model-agnostic and upstream of training, fine-tuning, and RAG.
At retrieval
Entitlements have to be evaluated per query against the requesting identity, ideally as a pre-filter that constrains the candidate set before similarity search rather than a post-filter applied to results. Permission metadata must travel with every chunk and refresh on a cadence that matches how fast access changes in the source systems, which for offboarding means minutes.
At the context boundary
Retrieved content should be structurally separated from instructions, marked as data, and treated as untrusted for the life of the request. Tool calls that a retrieved document appears to have triggered deserve a different authorization path than tool calls the user asked for. Provenance should be logged so an answer can be traced back to the chunks that produced it.
At output
Response inspection catches what got through, and should be budgeted as a backstop. By the time text reaches an output guardrail, the sensitive content has already been retrieved, placed in a prompt, and processed by a model that may be running outside your boundary.
How this relates to tools you may already run
DSPM tells you where sensitive data lives and how it is exposed, which is necessary input rather than an enforcement point in the retrieval path. Model scanning checks artifacts for tampering and malicious serialization, which addresses supply chain rather than entitlements. Guardrails and LLM firewalls inspect prompts and responses at the application edge. Each covers a real gap. None of them changes what a chunk contains or who the retriever will serve it to.
How does RAG security map to regulation and standards?
Directly. NIST's AI RMF and its Generative AI Profile both treat information disclosure and data provenance as named risks. The EU AI Act sets data governance and logging duties for high-risk systems. ISO/IEC 42001 asks for a management system around AI data handling. Existing privacy law already covers what RAG surfaces.
The NIST AI Risk Management Framework (AI RMF 1.0), published in January 2023, organizes the work under Govern, Map, Measure, and Manage. For RAG, Map is where you establish what the corpus contains and who is entitled to it. Measure is where you test whether retrieval honors that in production rather than on a slide.
The NIST Generative AI Profile (NIST AI 600-1), published July 2024, is the more specific document. It enumerates risks that generative AI creates or amplifies, including Data Privacy and Information Security, and maps suggested actions back to AI RMF subcategories. Its data provenance material lines up closely with ingestion controls.
The EU AI Act, Regulation (EU) 2024/1689, entered into force on August 1, 2024. Article 10 sets data governance requirements for high-risk systems and Article 12 requires automatic logging of events across the system's lifetime. The application date for standalone high-risk systems under Annex III moved from August 2, 2026 to December 2, 2027 under the Digital Omnibus on AI, which the Council approved on June 29, 2026. Teams got more time. The evidence they will need is the same evidence, and assembling entitlement and provenance records retroactively is harder than instrumenting for them now.
ISO/IEC 42001:2023 specifies an AI management system, and its Annex A controls cover data management for AI systems, including provenance, quality, and handling across the lifecycle. Enterprise buyers ask about it more often each quarter.
Sector rules still apply on top of all of this. HIPAA, GLBA, and the GDPR have no RAG exemption. A retrieval pipeline that returns protected health information to an unauthorized user has made a disclosure, whether or not a model was in the loop.
The practical consequence is documentation. Every one of these frameworks assumes you can show which data entered the system, what was done to it, and who could reach it. RAG pipelines assembled in a hurry usually cannot answer the second question at all.
Secure RAG comes down to positioning. Hugging Face had a working network control that could not see what an artifact was telling the worker to do. Most RAG deployments have working identity controls in the source systems that the retriever never consults, and content controls that stop at the edge of the document management system. Moving those decisions into the ingestion and retrieval path is the work, which is why Hardshell operates on the data before it becomes an index, inside the customer's environment.
Frequently asked questions
Is RAG safer than fine-tuning on sensitive data?
In some ways. Retrieval keeps content out of model weights and lets you change what the system can see without retraining. It also creates a live query path into your corpus that runs at user speed. Carlini et al. showed training data can be extracted from weights, so neither approach protects raw sensitive content on its own. The control has to sit on the data.
Does encrypting the vector database make RAG secure?
No. Encryption at rest protects against theft of the storage layer. Retrieval leakage happens through authorized queries against a decrypted index, so encryption never engages. Encrypt anyway, for the same reasons you encrypt anything else, and treat it as unrelated to whether the retriever will serve a chunk to someone who should not receive it.
Can a guardrail or LLM firewall stop RAG data leakage?
Partially. Output inspection catches recognizable patterns like card numbers and national identifiers with decent recall. It performs much worse on contextual sensitivity, such as an internal forecast or a named employee's performance note, because there is no pattern to match. It also runs after retrieval has already placed the content in the model's context.
What is the difference between prompt injection and indirect prompt injection?
Direct prompt injection is a user typing adversarial instructions into the chat box. Indirect prompt injection puts those instructions into content the system will retrieve later, so a different user triggers them without knowing. Greshake and colleagues named the second class in 2023. RAG is the main delivery channel for it, since retrieval pulls untrusted text into a trusted context.
How often should a RAG index refresh permissions?
As fast as access changes in the source system. Nightly re-indexing leaves up to 24 hours where a revoked user still matches chunks they can no longer open. Event-driven updates on ACL changes and offboarding are better. Where the retriever can check live entitlements at query time instead of storing a snapshot, that is better still.
Do we have to keep sensitive data out of the index entirely?
Usually not, and excluding it tends to make the assistant useless for the questions people actually have. The workable path is to transform sensitive content before it is embedded, so retrieval returns documents that stay analytically faithful while the sensitive values are no longer recoverable from the chunk or its embedding.
Where does RAG security appear in the OWASP Top 10 for LLM Applications?
Across three entries in the 2025 list. LLM01 Prompt Injection covers instructions arriving through retrieved documents. LLM02 Sensitive Information Disclosure covers content reaching users who should not receive it. LLM08 Vector and Embedding Weaknesses is the RAG-specific entry, covering vector store access control, multi-tenant leakage, corpus poisoning, and embedding inversion.
Related pages on retrieval security
Each of these answers one narrower question about retrieval over sensitive data.
- Does RAG leak data?
- How to prevent RAG data leakage
- RAG permissions: enforcing entitlements at query time
- Vector database security
- RAG vs fine-tuning for sensitive data
- Indirect prompt injection in RAG pipelines
- RAG security checklist
- Secure RAG vs LLM guardrails
- Can employees see restricted documents through an AI assistant?
- Embedding inversion: can vector embeddings be reversed?
- RAG leakage self-test procedure