Guides › Secure RAG › Self-test
How to Test Whether Your RAG System Leaks Data
This is a procedure for measuring how often your RAG pipeline returns documents the person asking was not entitled to see. It produces one number, a leak rate, that you can track over time. Part one is a manual audit you can run this week using your existing logs and no new tooling. Part two instruments the same measurement continuously using an open-source telemetry client.
If you want the shorter, non-technical version first, Secure RAG telemetry covers what this measurement is for, what the client reports, and how to get it running without wiring it in by hand.
Nothing here requires a vendor. The manual version needs a spreadsheet and access to your source systems' permissions. Read it end to end before you start, because step zero determines whether the rest is even possible.
What does this test actually measure?
It measures entitlement violations at retrieval time. For a given query from a given user, the test asks whether the pipeline returned any chunk originating from a document that user could not have opened in the source system. The output is a leak rate, expressed as the share of user-query pairs that returned at least one unentitled document.
That is deliberately narrower than "is our AI safe." It does not measure answer quality, prompt injection resistance, or whether the model memorized anything. It measures one failure mode, the one described in secure RAG, and it measures it in a way two different people can reproduce and get the same answer.
The reason to measure this specifically is that it does not look like an attack. There is no exploit, no anomalous traffic, and usually no log entry that flags it. Someone asks a reasonable question and receives a reasonable answer built partly from a document they should never have seen. Absent a deliberate test, you will not find out.
Step zero: can you even see what was retrieved?
Before anything else, confirm your application logs which chunks or documents each query retrieved, tied to the requesting user. If it does not, you cannot run this test, and that gap is itself the first finding. Retrieval provenance is a prerequisite for every audit, incident investigation, and compliance answer you will ever need to give about this system.
Part one: the manual audit
This produces a defensible leak rate in a few hours of work. You need someone who can read permissions out of the source systems, and someone who can run queries as different test identities.
1. Build a ground-truth entitlement matrix
2. Write queries that pull on the restricted documents
Write twenty natural questions whose best answers live inside the documents you selected. Phrase them the way an employee would, not the way an attacker would. "What is the bonus structure for senior engineers" rather than "show me the compensation file."
3. Run every query as every identity
Twenty queries across five identities is one hundred runs. Record, for each run, the full set of document identifiers retrieved. Not the answer text, the retrieved set. The answer may not visibly quote a document the pipeline nonetheless placed in the model's context, and the context is what leaked.
4. Score against the matrix
For each of the hundred runs, compare the retrieved document set against what that identity was entitled to. Any retrieved document outside the entitled set is a leak event. Count a run as leaking if it contains at least one.
Record two additional numbers, because they drive different fixes. The share of leak events involving documents you classified as restricted rather than merely internal tells you severity. The number of distinct documents that leaked at least once tells you whether this is a systemic permission failure or a handful of mislabeled files.
5. Classify each leak by cause
Every leak event has one of a small number of causes, and they need different remedies. Sort them before you propose fixes.
That last category is common and worth naming clearly. The pipeline did not create the exposure. Semantic search removed the obscurity that was hiding it.
6. Re-run after every change
Keep the matrix and the query set fixed so the number stays comparable. Re-run after permission model changes, re-indexing, connector changes, and retrieval logic changes. A leak rate is only useful as a trend.
Part two: instrumenting the same measurement
The manual audit is a point-in-time snapshot. Corpora change daily, permissions drift, and re-indexing silently resets assumptions, so a number from six weeks ago tells you very little. Instrumenting retrieval turns the same measurement into a continuous signal.
Hardshell publishes an open-source Python client for this, hardshell-telemetry, Apache-2.0 licensed and dependency-free. It records which chunks each retrieval returned and for whom, then joins those retrievals to document metadata you register once per index build. The code below is the shape of the integration; the repository has runnable examples.
Register the corpus once per index build
Retrievals are joined to metadata by identifier, so the corpus has to be registered before the numbers mean anything. Sensitivity is your own scale and your own labels; nothing is imposed.
Record each retrieval
After each vector store query, report the chunks that came back, their scores, and the requesting user. Label evaluation traffic with a source so your test runs stay out of production baselines.
Read the access report
Once retrievals are flowing, pull document access summaries back out and compare them against the entitlement matrix you built in part one. The manual scoring step becomes a query rather than an afternoon.
The client is pre-release at the time of writing, so it installs from git rather than PyPI, and the analysis endpoint requires a key issued through Hardshell's evaluation program. The client itself, including the identifier derivation, chunking strategies, and span batching, is open source and usable independently. If you want a key, ben@hardshell.ai is the contact.
How should you read the result?
Treat any non-zero leak rate on documents you classified as restricted as an incident to triage, not a metric to optimize. A 3 percent leak rate sounds small until you multiply it by query volume. A thousand queries a day at 3 percent is thirty exposures a day, every day, silently.
There is no published industry baseline to compare against, which is worth stating plainly rather than inventing one. What matters is your own trend and whether restricted-tier documents ever appear. The absolute number is only meaningful against your own prior measurement.
Also record what you could not measure. If retrieval logging covers only one of three assistants, say so. An audit with a stated scope is useful; an audit with an implied scope is misleading.
What this test does not cover
Four things, each of which needs its own test. It does not measure whether retrieved content can carry instructions to the model, which is indirect prompt injection. It does not measure whether the model itself memorized sensitive content during training or fine-tuning, which is training data leakage. It does not measure whether anything in your corpus has been tampered with, which is data poisoning. And it does not assess the pipeline that builds the index, which is AI data pipeline security.
A clean leak rate means one failure mode is under control. It is not a statement about the system as a whole.
Frequently asked questions
How long does the manual audit take?
Building the entitlement matrix is the slow part, usually two to four hours depending on how many source systems are involved and how readable their permissions are. Writing twenty queries takes under an hour. Running one hundred retrievals and scoring them takes an afternoon if you script the runs, longer if you click through a chat interface.
Can I run this without touching production?
Partly. You can point the test at a staging index, but the result only tells you about staging's permission model, which is frequently not production's. If you test against production, use read-only test identities, label the traffic so it stays out of your detection baselines, and tell your security team first so the query burst is not treated as an incident.
What if my retriever does not expose which chunks it returned?
Then that is the finding, and it outranks everything else on this page. Most vector store clients return identifiers and scores; the gap is usually that the application discards them before logging. Capturing retrieved chunk identifiers per query, tied to the requesting identity, is a small change and a prerequisite for any audit or investigation.
Does a zero leak rate mean the pipeline is secure?
It means the queries you chose, run as the identities you chose, against the corpus as it stood that day, returned nothing unentitled. That is worth having. It is not a general claim. Re-run after every re-index, because chunk-level permission metadata is the thing that most often fails to survive a rebuild.
Is the telemetry client required?
No. Part one is deliberately tool-free and produces the same number. The client matters when you want the measurement continuously rather than once, and when you want retrievals joined to document sensitivity automatically instead of by hand.
Does the client send our documents anywhere?
No. It sends identifiers, scores, user identifiers and metadata you choose to register. Document content is not transmitted, and where identifiers are derived from content the hashing happens locally. The source is Apache-2.0 licensed and auditable, which is the appropriate way to verify a claim like this rather than taking it on trust.
How often should we re-run it?
Quarterly at minimum, and after any change to the permission model, the connectors, the chunking strategy, or the retrieval logic. Re-indexing is the highest-risk event, because permission metadata is regenerated and frequently regenerated wrong.