AI Knowledge Base Security

An AI knowledge base is whatever corpus your AI retrieves from at query time. Classic RAG over a vector store is the most common shape, but agentic retrieval, GraphRAG, connector-based assistants and enterprise AI search all have one. That corpus is a security boundary, and almost nobody treats it as one, because it wasn't built by a security team. It was built by an application team as a product feature, and the questions asked about a product feature are about latency and answer quality rather than about who is allowed to see what.

What counts as an AI knowledge base

The term covers more ground than RAG does, which is the reason to use it. If a model reads internal documents at query time, there's a knowledge base behind it, whatever the team that built it calls the thing. The architectures differ in how the corpus gets assembled and searched, and those differences change the shape of the exposure rather than removing it.

Most organizations are running several of these at once. A company with a Copilot tenant, a team-built retrieval service over a vector store and an agent wired into a few MCP servers has three knowledge bases and three sets of retrieval decisions, usually owned by three different groups. There's rarely one place to look.

Why it is a security boundary

Retrieval decides, per query, which documents a model sees. Nothing else in the stack makes that call. The identity of the person asking, the sensitivity of the material and the scope of what they're entitled to all have to be reconciled at that moment, and in a default pipeline none of them are. What decides is the distance between the question and a chunk.

So an access-control decision is being made by a similarity function, which has no way to represent the idea that this identity may read a document and that one may not. A vector store doesn't inherit your access model just because the documents came from a system that had one. The permissions stayed behind.

The other half of the problem is that the failure is quiet. Someone asks a reasonable question. The assistant gives a reasonable answer. Part of that answer came from a document the person asking was never allowed to open. No exploit ran, no alert fired, and in most deployments there's no log entry that would tell you it happened. Application logs hold the question, the answer, the latency and the token count. They don't hold which documents were reached, or by whom. A permissions bug in a document management system leaves a trail somebody can find later. This one produces a helpful answer and no trace.

The four failure modes

Nearly everything sorts into four. They have different causes and different owners, and the fix for one does nothing for the other three.

Entitlements ignored at retrieval time

The index carries no per-document permission data, or it carries it and the query path never uses it. Ask a question whose answer lives in a restricted document and the restricted document comes back. It's the most common mode and the easiest to demonstrate. Does RAG leak data? walks the routes it happens through, can employees see restricted documents through an AI assistant is the form the question usually takes in a security review, and RAG permissions covers why trimming results after the model has already read them is not the same as filtering before the search runs.

Permissions that went stale after a revocation

Someone leaves a team. A matter closes. A contractor's access ends. The source system reflects all of it. The index doesn't, because the index was populated by a job that last ran on Sunday. Access that was correctly removed stays live in retrieval until the next sync, and the length of that window is a security property almost nobody has written down. How to prevent RAG data leakage takes the controls in pipeline order, including where permission sync belongs.

Content that is sensitive in itself

Sometimes the person asking is entitled to the answer but not to every field in the document that carries it. A clinician needs the treatment history without the identifiers attached to it. An analyst needs the finding without the source. Entitlement enforcement does nothing here, because the entitlement check passes. The exposure is in the content, and it is also in the vectors, which are not one-way. Vector database security covers what encryption at rest doesn't reach, embedding inversion covers what research has recovered from embeddings alone, and RAG vs fine-tuning for sensitive data covers the same question when the corpus goes into weights instead of an index.

Retrieved documents that carry instructions

A retrieved chunk is not inert. If it contains text shaped like an instruction, the model may act on it, which makes your corpus a delivery channel rather than only a data source. The surface widens as the corpus takes in material you didn't write, like a customer's ticket or a page an agent fetched mid-task. Agentic retrieval sharpens it further, because the agent chooses what to read next and can be talked into reading something. Indirect prompt injection in RAG pipelines covers the delivery paths and which mitigations hold up. Secure RAG vs LLM guardrails covers why inspecting prompts and responses at the application edge never reaches the retrieval decision underneath.

How to measure your exposure

Measurement comes before controls, for a practical reason. Until you know how often retrieval returns something the person asking wasn't entitled to, you can't tell which of the four modes you have, you can't size the work, and you can't answer the question a security review is actually asking.

The procedure isn't complicated. Build a ground-truth entitlement matrix from your source systems. Write queries whose answers live in documents only some identities may open. Run each query as several test identities spanning the entitlements you need to tell apart. Compare what came back against what each identity was entitled to. The output is a leak rate you can track over time. The full self-test procedure has the detail, including a manual version you can run this week against logs you already have.

If you'd rather instrument it than run it by hand, Hardshell publishes a free, open-source telemetry client that records which chunks were retrieved and for whom, then reports the pattern. Content is hashed locally and never leaves your environment. The reporting endpoint runs through Hardshell's Enterprise Evaluation Program, which is where a scoped evaluation key comes in. The telemetry page has the client, the setup notes and the key request.

Where the controls belong

Once you have a number, the work sorts by cause, and most of it isn't in the AI layer. Three controls do the bulk of it, in this order.

The RAG security checklist is the stage-by-stage version of all three, from ingestion through retrieval to the model context boundary. What is secure RAG? covers what the term means and where each control point sits. The short version is that retrieval should make the same access decision your source systems already make, on every query, and you should be able to show a reviewer that it does.

Standards and regulation

No framework in this space names AI knowledge base security as a control family. What they create is an obligation to know what your system exposes, which is the same measurement problem arriving from a different direction.

NIST's AI Risk Management Framework is voluntary and organizes the work into governing, mapping, measuring and managing risk. Retrieval leakage sits in the measure function, and the framework's value here is that it asks for evidence rather than assurances. The companion Generative AI Profile, AI 600-1, treats information security and data privacy as generative-AI risk categories and lists suggested actions against them.

ISO/IEC 42001 is a management-system standard for AI, which means it's certifiable. It doesn't prescribe a retrieval control. It asks you to decide your position, apply it, and keep records that show you did. For a knowledge base that means a written statement of what may be indexed and who may reach it, plus evidence that retrieval honors it.

The EU AI Act's heavier obligations attach to systems classified as high risk. For those, Article 10 covers data and data governance and Article 12 covers automatic recording of events over the system's lifetime. Whether an internal assistant falls in that category depends on what it's used for, and many won't. The logging expectation is worth reading either way, because retrieval events are precisely what most deployments don't record.

OWASP's Top 10 for LLM Applications maps most directly. LLM01 covers prompt injection, LLM02 covers sensitive information disclosure and LLM08 covers vector and embedding weaknesses. Three of the ten are about the knowledge base, which is a fair indication of where the risk sits.