Guides › MITRE ATLAS › Posture assessment

How to Assess Your AI Security Posture With MITRE ATLAS

This is a procedure for turning MITRE ATLAS into an assessment of AI security posture and data security posture that you can run in-house, repeat each quarter, and hand to an auditor. It produces one artifact: a coverage record listing every data-layer ATLAS technique in scope, what evidence you hold for it, and where the gaps are. It does not produce a score, because ATLAS does not define one, and any number you attached would be yours rather than MITRE's. Expect two to three weeks the first time for a mid-size estate, most of it in steps one and four, and a few days for each repeat.

Step 1: Inventory the AI systems and the data behind them

Start with the data, not the models. For every assistant, copilot, agent or trained model in use, record the corpus it retrieves from and which source systems feed it through which service account, the training or fine-tuning sets it was built on, who can query it (a named group, all employees, contractors, customers or the public), and which sensitive categories are in scope. Include the systems nobody on the team calls RAG: Copilot over SharePoint, a support bot over the ticketing system, an enterprise search product wired into Drive. If a model sees internal documents when it answers, it belongs on the list. The AI knowledge base security guide has an architecture table that helps classify each one, and the three-minute AI data risk assessment on this site is a fast way to draft the inventory before you formalize it. ATLAS describes this artifact as mitigation AML.M0023, AI Bill of Materials.

Step 2: Scope the ATLAS techniques that apply

Not all 114 techniques. Use the data-layer table in the MITRE ATLAS guide as the starting set and prune by architecture. If no system trains or fine-tunes on internal data, the training poisoning and extraction rows fall out. If no assistant has tool access, Exfiltration via AI Agent Tool Invocation (AML.T0086) falls out. If everything in the inventory is retrieval, the retrieval leakage, indirect injection and knowledge base poisoning rows are the whole scope. Write down what you excluded and why. An excluded technique with a reason is a scoping decision; an omitted one is a gap. A typical mid-size estate ends up with 12 to 18 techniques in scope.

Step 3: Set an evidence standard for each technique

Every technique in scope will end up in one of four states, and agreeing on their definitions before testing is what keeps the record honest.

• Tested. You attempted the technique against the system, by self-test, red team or exposure test, and recorded the outcome and the date.
• Detected. You have telemetry that would show the technique if it happened, and you have verified that it fires. Retrieval telemetry keyed to identity and returned chunks is the example for LLM Data Leakage.
• Mitigated. A control that ATLAS or the crosswalk maps to the technique is in place and you can point to it. Mitigated without Tested is a claim, not evidence, and should be labeled that way.
• Not assessed. The honest default for anything you have not yet reached.

"Not applicable" is not a fifth state. It is the job of step two, and it belongs in the scoping notes with a reason attached.

Step 4: Run the tests

Group the in-scope techniques by the test that exercises them. Most estates need five or six test types, not one per technique.

• Retrieval leakage (AML.T0057, AML.T0085.000, AML.T0082). Run the RAG leakage self-test: build an entitlement matrix, run twenty queries as five identities, and score the retrieved documents against the matrix to get a leak rate. For credential harvesting, search the index itself for key and token patterns rather than waiting for a query to surface one.
• Indirect prompt injection (AML.T0051.001, AML.T0066, AML.T0086). Plant a benign canary instruction in a document each test identity can reach and check whether the assistant follows it. If the agent has tools, check whether it attempts an outbound action. The indirect prompt injection guide covers what holds and what does not.
• Knowledge base poisoning (AML.T0070, AML.T0071, AML.T0080). Trace how content enters the index, test whether an unprivileged user can get a document indexed, and check that provenance metadata survives a re-index. Agent memory counts as part of the corpus.
• Training data poisoning (AML.T0020, AML.T0115.000, AML.T0010.002). Scan training and fine-tuning sets for anomalous samples and label flips before training, and verify the provenance and hashes of any public dataset the pipeline pulls. The data poisoning and pipeline security guides cover the five entry points.
• Training data extraction (AML.T0024.000, AML.T0024.001, AML.T0057). Run membership inference and extraction probes against models trained on internal data, seeded with canary strings you planted before training. See training data leakage.
• Embedding inversion and index exposure (AML.T0024.001, AML.T0007). Check who and what can read the vector index directly, and whether embeddings leave the environment. See embedding inversion and vector database security.

Test against production where you can, because a staging index frequently has a different permission model. Use read-only test identities, label the traffic so it stays out of your detection baselines, and tell the security team first so the query burst is not treated as an incident.

Step 5: Record findings with technique IDs and the ATLAS version

One record per technique per system. The fields that matter: system, date, ATLAS release, technique ID and name, primary tactic, state, outcome, a pointer to the evidence, a severity you assign, and an owner. If you report to more than one audience, add the OWASP, NIST AI RMF, ISO/IEC 42001 and EU AI Act references from the crosswalk to the same record rather than keeping a second document. Store it somewhere versioned. An example record looks like this.

Step 6: Map mitigations and identify gaps

For each technique in scope, list the ATLAS mitigations it maps to and the controls you actually run. The ATLAS guide lists the six mitigations that carry most of the data layer: access control on data at rest (AML.M0005), training data sanitization (AML.M0007), dataset provenance (AML.M0025), telemetry logging (AML.M0024), guardrails (AML.M0020) and the AI bill of materials (AML.M0023). A technique that is Not assessed, or Tested with a finding, and has no control mapped to it is a gap. Prioritize gaps three ways: by ATLAS maturity, with Realized techniques first because they have been observed in the wild; by the sensitivity of the data from step one; and by who can reach the system, with customer-facing and all-employee deployments ahead of named groups. How to prevent RAG data leakage and RAG permissions cover the retrieval-side controls in pipeline order.

Step 7: Build the coverage record and set the cadence

Roll the records up into one table: techniques in scope, the count in each state, and the gap list with owners and dates. That table is the posture assessment. Reported through the crosswalk, the same table becomes Map and Measure evidence for NIST AI RMF, risk assessment and treatment evidence for ISO/IEC 42001, and Article 10, 12 and 15 evidence for anyone with a high-risk system under the EU AI Act. Re-run quarterly, and after any re-index, connector change, permission model change, new assistant, or ATLAS release that renames a technique you use. Keep the query set and the test identities fixed between runs so the numbers stay comparable. A first-pass summary for a small estate might look like this.

Where Hardshell fits

Steps four and five are where most teams stall, because running the tests well takes tooling and time the team does not have. Hardshell's live-fire exposure testing runs those steps against your systems and returns each finding tagged with the MITRE ATLAS technique IDs it demonstrates, along with the evidence behind it and the crosswalk references. Knowledge base telemetry keeps the Detected column true between assessments, with detections that carry the same IDs. The open-source telemetry client is free and is enough to run the retrieval rows of this procedure yourself.

Frequently asked questions

Can we run this assessment without MITRE ATLAS?

Yes, and the tests are the same. What you lose is comparability. Findings labeled in your own words cannot be lined up against last quarter, against another team's assessment, or against the ATT&CK-based detections your SOC already runs, and every reader has to learn your taxonomy before they can use the record. ATLAS IDs cost nothing to add and remove that problem.

How is this different from the AI Data Risk Assessment on this site?

The three-minute assessment is a questionnaire. It gives you a risk read and a reading list based on what you say you have. This procedure is evidence-based and gives you a coverage record based on what you tested. Run the quick one first to scope the inventory, then run this one to find out whether the answers were right.

How do we turn the coverage record into a score?

Carefully, and with the definition attached. ATLAS does not score, so any number is yours. If leadership needs one figure, report coverage: the share of in-scope techniques that hold Tested or Detected evidence. Report the gap count next to it. Resist weighting schemes that produce a single risk number, because they hide the one thing the record is good at showing, which is exactly where the evidence stops.

Does this satisfy NIST AI RMF, ISO/IEC 42001 or the EU AI Act?

It produces evidence those frameworks ask for; it does not on its own satisfy any of them. NIST AI RMF's Map and Measure functions, ISO/IEC 42001's risk assessment and treatment requirements, and Articles 10, 12 and 15 of the EU AI Act for high-risk systems all expect you to have identified the relevant threats and tested for them. The coverage record is a direct answer to that expectation. The framework crosswalk shows which column each row reports into.