Guides › Data silos

Data Silos and Enterprise AI: Why Connecting Them Leaks Data, and How to Do It Safely

A data silo is a store of data that one part of the organization can reach and the rest can't. For twenty years the advice was to break silos down, and for twenty years most organizations didn't, because the silos were doing a job nobody had written down: they kept people away from data they had no business seeing. Enterprise AI assistants and connectors finish the job in a single query. This guide explains what a silo was quietly doing for you, why “connect everything to the assistant” leaks, why the migrate-first and block-everything answers both fail, and what a design that unlocks data where it lies looks like in practice.

What a data silo is, and what it was quietly doing for you

A data silo is any store of data that one team, system or business unit can reach and the others effectively can't. Silos form the same way everywhere. A department buys its own tool, an acquired company arrives with its own stack, a regulated workflow gets its own database, and a decade of SaaS purchasing leaves customer data spread across a CRM, a support desk, a billing system and a few hundred spreadsheets. Practitioners interviewed by InformationWeek in 2025 named legacy infrastructure, organizational culture, inconsistent data standards, and mergers and acquisitions as the usual causes, and noted that older organizations and serial acquirers have it worst.

The data management literature treats silos purely as a cost, and they are one. Analysts work from fragments, models train on partial views, and the same customer exists in three systems under three spellings. That framing misses what silos were doing on the security side. A silo enforces access by friction. Nobody outside finance opened the budget share, because nobody outside finance knew it existed, had the link, or had a reason to go looking. The site's permissions might have granted every employee read access, and it didn't matter, because discovery was hard.

Security teams call this security through obscurity, and the field has spent decades explaining why it isn't a real control. That's correct, and it misses the practical point. The friction was carrying real weight, and it held for as long as a human had to find, open and read each document one at a time. Every oversharing audit that has ever surprised a CISO is a measure of how much weight it was carrying.

Why “break down the silos” became the enterprise AI playbook

The AI readiness argument is now standard. Gartner's February 2025 survey found that 63% of organizations either lack AI-ready data management practices or aren't sure they have them, and predicted that through 2026, organizations will abandon 60% of AI projects that aren't supported by AI-ready data. Dun & Bradstreet's 2026 AI Momentum Survey of 10,000 businesses put it more bluntly: 97% report active AI initiatives, 5% say their data is ready to support them, half say access to data is a problem, and 38% cite a lack of integration across systems.

Two industries answer that gap. Data platform vendors sell the unified layer: a lakehouse, a fabric, a semantic layer, something that pulls every silo into one governed store before a model touches it. The AI assistant vendors sell the reverse: leave the data where it is and connect the assistant to it. Microsoft 365 Copilot reads across SharePoint, OneDrive, Teams and Exchange through the user's own identity. ChatGPT Enterprise ships connectors for SharePoint, Google Drive, Box, Dropbox, GitHub, HubSpot, Notion and an MCP server for anything else. Claude connects to Google Workspace, Microsoft 365 and custom MCP servers. Enterprise search products index all of it.

The second path wins on time to value, which is why most organizations are on it. It is also the interesting one for this guide, because each of those connectors is a silo collapse. Nothing moves. Every store that used to require its own login, its own search box and its own reason to visit now answers the same question in the same chat window, joined.

What happens when an assistant collapses the silos

Every major assistant vendor says the same thing about permissions, and it is true. Microsoft's architecture documentation states that Copilot “can only summarize or reference content that the user is authorized to access.” OpenAI says source permissions still apply through connectors. Anthropic says Claude mirrors your existing permissions. The assistant bypasses nothing. The problem is the word “authorized.” In most tenants, what a user is technically authorized to open and what they were meant to see diverged years ago, and the silo friction covered the difference.

Three distinct things go wrong when that friction disappears.

Over-permissioned sources become reachable

A SharePoint site created in 2019 with “Everyone except external users” read access, a shared drive with a link-sharing default, a group that inherited membership through a reorg: none of this was a problem while nobody looked. The assistant looks on every query. Microsoft's own tooling describes the scale. Restricted SharePoint Search, the control Microsoft shipped so tenants could cap Copilot to an allow-list of sites while permissions were reviewed, holds at most 100 sites, is described by Microsoft as not a security boundary, and from July 31, 2026 can no longer be newly enabled, with customers pointed to Restricted Content Discovery and SharePoint Advanced Management instead. A stopgap with a 100-site cap exists because the number of over-shared sites in a typical tenant is larger than 100.

Joins create data that no single silo held

The second failure has no equivalent in the pre-AI world. A user with legitimate access to the org chart, the travel calendar and the expense system could in principle have worked out who was interviewing at a competitor. Nobody did, because it took an afternoon. An assistant that retrieves across all three answers the question in a sentence. Each retrieved document passes its own permission check. The inference that results was never a document, so nothing checked it. This is the mechanism the AI knowledge base security guide describes for a single retrieval corpus, applied across every connector at once.

The collapsed data leaves your boundary

Connecting silos to an internal assistant is one problem. Connecting them to a frontier assistant run by a third party is that problem plus a data sovereignty problem. Whatever the connector retrieves goes to the model as context, with real names, account numbers and record contents intact, and is held under the provider's retention terms. Enterprise plans fix what the provider does afterward: no training by default, a defined retention window. They don't change what crosses. The third-party platforms guide covers that route in detail.

The three answers that don't work

Migrate everything first

The lakehouse answer says: consolidate the silos into one governed store, build the permission model once, then connect the assistant. In practice this takes eighteen months to several years, creates a new copy of every sensitive record under a new permission model that has to be reconciled with the old ones, and the silos re-form in the meantime because the business keeps buying tools. By the time the unified store is governed, the assistant has already been connected to the ungoverned originals, because the business couldn't wait. Consolidation is a reasonable data strategy. It is a poor security control, because the exposure exists on day one and the control arrives at the end.

Block the assistants

Disabling connectors, banning the tools and writing DLP rules reduces what the sanctioned assistant can reach. It does nothing for the employee who pastes the same document into a personal account, and it gives up the productivity gain that justified the project. Organizations that chose this answer in 2024 and 2025 generally got both outcomes: shadow AI use rose, and the sanctioned rollout happened anyway, later, under pressure, with less time for the permission work.

Allow-list the sources

Capping the assistant to a short list of reviewed sites is the current default for Copilot deployments, and it is a sensible first week. Microsoft's own guidance is that it is temporary. The cap limits what the assistant can see, which also limits what the assistant is useful for, and it does nothing about what the assistant can reach inside the allowed sources. It is friction, reintroduced by hand, with an expiry date.

The design that holds: unlock data where it lies

The design that works keeps what the silos were good at, which is that data stays in a system that already governs it, and replaces the friction with a real control at the moment of retrieval. It has five properties, and they are the same five the sovereignty guide arrives at from the other direction.

Data silos and data sovereignty are one problem seen from two sides

Silos are the internal version of the question: who inside the organization can reach which data, and whether the answer stays true once an assistant does the reaching. Sovereignty is the external version: which data leaves the organization's control, to whom, and under whose rules. Connecting silos to a third-party assistant raises both at once, in one configuration screen, and the same five properties answer both. The sovereignty vs residency page covers the definitions, and the deployment page covers where the assistant should run once the data question is settled.

Where Hardshell sits

Hardshell is a data security layer between an organization's existing systems and the AI platforms it uses. It connects to the stores where data already lives without migrating them, inherits their permissions, limits each retrieval to what the requesting user is entitled to see, substitutes stand-ins for real identifiers before content reaches a model provider, and records every retrieval under opaque IDs. Exposure testing measures how much a connected assistant leaks before rollout, and findings carry MITRE ATLAS technique IDs. It runs as a cloud service, in a private cloud, or as an offline bundle for disconnected environments. The practical sequence for a first connection is on the how to safely connect enterprise data to AI page, and the connector-specific failure modes are on the AI connector and MCP security page.

Frequently asked questions

Are data silos bad for AI?

For model quality and analytics, yes: fragmented data produces partial models and inconsistent answers. For security, silos were doing unplanned work, and removing them without replacing that work is how most AI data exposure now happens. The goal is to connect the data with a real control at retrieval time. Preserving the silos isn't the point, and neither is removing them blind.

Does Microsoft 365 Copilot respect existing permissions?

Yes. Microsoft states Copilot can only summarize or reference content the user is authorized to access, and that holds. The exposure comes from what those permissions already allow. Microsoft's own remediation guidance, covering SharePoint Advanced Management, Restricted Content Discovery and sensitivity labels, exists because the gap between “authorized” and “intended” is large in most tenants.

Do we need a data lake or warehouse before we can use AI on our data?

No. Consolidation helps analytics and may be worth doing for its own reasons. It is slow, it creates a second copy of sensitive data under a second permission model, and the assistant will be connected to the originals long before it finishes. Permission-scoped retrieval against the systems you already have puts the control in place on day one.

What is the difference between data silos and data sovereignty?

Silos describe who inside the organization can reach which data. Sovereignty describes which data leaves the organization's control and under whose rules. Connecting silos to a third-party AI platform is where the two meet, and the same controls answer both: retrieval scoped to the asker, substitution before the boundary, and a record of every retrieval.

How do we find out what our AI assistant can already reach?

Run an oversharing audit on each connected source under a sample of real user identities, then run the assistant itself against a set of questions it should refuse and count the documents it returns. The leakage self-test describes the procedure, and retrieval telemetry turns it into a continuous measurement after rollout.