Guides › Sovereign AI deployment

Sovereign AI Deployment: On-Prem, Private Cloud and Disconnected Options for Regulated and Federal Teams

Where an AI system runs decides who can reach the data inside it. This guide compares the three deployment patterns available to a regulated or federal program, lists what a sovereign deployment has to cover beyond the model itself, and sets out the design principles that hold up in a denied environment.

Three deployment patterns and what each one settles

Most organizations run more than one. The useful exercise is to classify workloads by the data they touch, assign each to the least restrictive pattern that satisfies its obligations, and then make sure the AI data layer behaves the same way in all of them so controls and evidence don't fork.

What a sovereign deployment has to cover

"Sovereign" tends to get applied to the model. The data layer is where most of the exposure lives.

• Model weights. Self-hosted or vendor-served. If fine-tuned on your data, the weights are your data in another form. See training data leakage.
• The corpus and its index. Whatever an assistant retrieves from. If it lives in a vendor-hosted index, the documents have left. See vector database security.
• Embeddings. Vectors aren't one-way hashes; inversion recovers meaningful text from them. An embedding store outside the boundary is a copy of the corpus outside the boundary. See embedding inversion.
• Retrieval and inference telemetry. Logs of queries, returned chunks and answers. Keep them keyed to opaque IDs so they can be shipped to a SOC without shipping content.
• The vendor's own paths home. License checks, update channels, usage reporting, crash reports. In a denied environment every one of these is an outbound dependency that either fails closed or leaks.
• Entitlement and identity. Permissions enforced at query time need an identity source inside the boundary. A cloud identity provider is another outbound dependency.

The federal and defense case

Federal programs rarely get to pick a pattern. The data classification picks it for them.

• CUI. NIST SP 800-171 governs its protection in nonfederal systems, CMMC assesses that protection across the defense industrial base, and DFARS 252.204-7012 expects FedRAMP Moderate or equivalent from any cloud service that stores, processes or transmits it. Retrieval context carrying CUI to a commercial AI service is a transmission.
• Export-controlled technical data. ITAR and EAR constrain access by foreign persons, which rules out services where foreign access can't be excluded.
• DoD impact levels. IL4, IL5 and IL6 set where DoD data may be processed. A model provider's commercial endpoint isn't inside those boundaries unless specifically authorized.
• Classified enclaves and denied environments. No outbound path at all. Enforcement, detection and licensing have to run locally, through jamming, link loss and partition, and entitlement has to be an offline artifact rather than a callback.

The last case is the design constraint that forces the right architecture. If the data layer works with no network, it works everywhere else too. The MITRE ATLAS mapping gives program security the technique IDs to report findings against.

The regulated commercial case

Health systems, banks, credit unions and insurers mostly land on private cloud. PHI, nonpublic personal information and examiner expectations make a vendor-hosted knowledge base hard to defend, while fully disconnected operation is more than the workload needs. The question that remains is what crosses to the model provider on each request. HIPAA's minimum-necessary standard, GLBA safeguards and a growing set of state privacy laws all treat that crossing as a disclosure to a vendor. Scoped retrieval and substitution before the boundary are how a private-cloud deployment keeps a frontier model in the loop without disclosing identifiers. The third-party platforms page covers the mechanics and the main guide maps the regulations.

Design principles that hold in every pattern

• Leave data where it lives. Connect to source systems and inherit their permissions. Don't build an AI-ready copy.
• Bring a scoped view to the model. Retrieval is an access decision. Make it per caller, at query time.
• Substitute before the boundary. Real identifiers never cross to a model provider. Restore them for entitled readers only.
• No outbound dependency for enforcement. Detection, enforcement and entitlement run inside the boundary, and a cut link fails closed.
• Telemetry without content. Record every retrieval keyed to opaque IDs so the evidence can travel and the data can't.
• Re-test utility after hardening. Security that quietly degrades the model gets turned off. Measure answer quality against your own benchmarks after every change.
• Evidence in a shared vocabulary. Tag findings and detections with MITRE ATLAS technique IDs so program security, auditors and vendors read the same record.

Where Hardshell sits

Hardshell's data layer runs as a cloud service, inside a customer's private cloud, or as an offline bundle that installs inside an enclave with no phone-home, no telemetry egress and no cloud dependency. Entitlement is by contract and an offline artifact rather than a callback, detection and enforcement run local, and hardened data is re-tested for utility so protection never quietly degrades the model in use. The same platform secures AI knowledge bases, model training and fine-tuning across all three patterns, so a program can move a workload between them without changing its controls or its evidence.

Frequently asked questions

Is on-prem the only sovereign option?

It's the only option where nothing can leave. Private cloud with scoped retrieval, substitution and a per-request record is sovereign in the operational sense for most regulated commercial workloads, and it keeps frontier models available. The data classification decides.

Can we use a frontier model inside a disconnected enclave?

Only if the provider ships weights you can run locally, which the largest frontier models' providers generally don't. Disconnected deployments run open-weight or custom models, and the data layer has to protect the training data and the corpus those models use.

What does "no phone home" cover?

License validation, update checks, usage reporting, crash reporting and any vendor-side analytics. A sovereign deployment should be able to run indefinitely with the network cable unplugged, with licensing proven by an artifact you hold rather than a callback.

Does running in our private cloud make the model provider's terms irrelevant?

No. If your private-cloud application calls out to a model provider, the provider's terms govern what you send. The private cloud settles the storage and index questions. What crosses on each request is still your decision to make and record.