AI models are becoming short-term tenants in long-lived enterprise data estates. Yet many organizations are remodeling the building around each new occupant.
The market keeps asking which model enterprises should trust: open or closed, domestic or foreign, one frontier provider or another. These are legitimate questions. They are also questions with a very short shelf life. Model rankings change by the month. Providers change terms, regulators redraw boundaries and capabilities that looked scarce six months ago become commodities.
In my experience, enterprise data doesn’t move at that speed. Neither do the controls, applications and business processes built around it. That mismatch is why the most important AI architecture decision for CIOs is no longer which model to choose. It is whether the enterprise can absorb continuous change without fragmenting its data estate or surrendering control.
The winners in enterprise AI will be those who build a data foundation extensible enough to accommodate it ,build a data foundation extensible enough to accommodate that ongoing change and sovereign enough to remain in control.
Extensibility is now a production requirement
During experimentation, architectural limits are easy to hide. A team can connect a new model to a copied dataset, prove a use case and call the pilot successful. Production exposes the real architecture: identity, permissions, data residency, lineage, latency, recovery and cost.
Every time those controls are rebuilt around a model, the model becomes a dependency. Adding or changing capabilities can mean moving data, recreating access policies, revalidating pipelines and reopening compliance reviews. What looked like speed at the pilot stage becomes a maze of point-to-point architecture that is difficult to extend and even harder to govern.
This isn’t a hypothetical. A Harris Poll surveying 600 CIOs found that 81% expect to rely on two or more large language model providers in 2026, and 93% say different models perform better for different use cases, requiring ongoing evaluation. More than half also noted they’ve switched providers at least once already, mostly to cut costs. This is consistent with our observations in the field: Multi-model is not a future scenario CIOs plan around. It is the operating condition they’re already managing now, whether or not the architecture underneath was built for it.
The gap between expectation and architecture is where the real cost surfaces. When a model is deeply embedded in application logic, data pipelines and prompt structures, replacing it with another provider turns into a multi-quarter engineering project rather than a simple configuration change. Teams end up paying twice, once to adopt the new model and yet again to unwind everything wired to the old one.
This helps explain why AI experimentation remains far ahead of AI value. CIO.com’s 2026 State of the CIO survey found that only 19% of respondents said their AI initiatives had met or exceeded business goals. Architecture is not the only reason, but systems optimized for the first model demo rarely become durable production systems by accident.
I recommend that CIOs add extensibility and control to the familiar measures of performance, security and cost. Can a team introduce a new model without relocating governed data? Can it change the deployment environment without rebuilding the application? Can it add a new way to query data without creating another synchronized copy? If the answer is no, the enterprise is extending its AI capability by fragmenting its control.
Put control where the durable value lives
Models are increasingly interchangeable. Enterprise context is not. The distinctive asset is the governed body of customer history, operational state, intellectual property and institutional knowledge that gives a model something useful to reason over.
That makes the data layer the durable control point. Identity, access, locality, audit and policy should be enforced as close to the data as possible, inside an environment the enterprise can operate, extend and move. The model can then be inspected, sandboxed or replaced within those boundaries.
A common pattern shows why. A team wires retrieval and access controls directly against one model’s embedding format and API to get to production faster. A year or two later, a new regulatory requirement or a stronger-performing model prompts a switch. Governance fenced around the original model will result in rebuilding access policies, revalidating pipelines and, ultimately, reproving compliance inside the new vendor’s environment. Governance enforced at the data layer, on the other hand, allows the new model to inherit the operating policies and audit trail from day one, maintaining the existing fence even as the model changes.
This does not make model provenance or security irrelevant. It makes those risks containable. Governance attached separately to every model is a collection of fences that must be rebuilt whenever the vendor changes. Governance anchored in the data layer is the property line: The tenant can change and the building can expand without changing who controls the grounds.
That is what sovereignty means in operational terms. It is not only where data is hosted but who can govern, move and extend the environment on their own terms.
Ask what the enterprise still controls
Before approving the next AI platform, CIOs should ask three questions:
- Can this architecture absorb a new model or workload without moving governed data?
- Which governance controls remain ours if the provider, region or deployment model changes?
- Does adding this capability extend the foundation or create another data copy, control plane and dependency?
These questions become more urgent as agents move into production. Agents multiply quickly; they act across systems and they need the governed data the enterprise already holds. When controls are enforced where that data lives, every new agent inherits them by default. Without that foundation, each agent brings its own pipeline, permission model and place to store what it needs, and the governance burden grows with every agent added.
IBM’s Institute for Business Value surveyed 2,000 technology executives in 2026 and found that businesses building control into their AI systems deploy 16 times more AI agents than those relying on manual governance, while spending a fraction of the budget to get there. This research is consistent with what I’ve seen in the field. In our research report with MIT Technology Review, we found that 95% of enterprises are planning to establish their own AI and data platforms within the next three years, and the 13% most committed to sovereignty are already delivering 2x more agentic and generative AI applications in mainstream production.
Businesses designing for adaptability early on are more likely to keep their workloads portable and have an easier time replacing models, instead of getting locked into hard dependencies. These businesses also reported a meaningfully higher return on their AI investment. There’s a clear gap that traces back to architecture. Whichever model a business happens to select matters less than whether the foundation underneath can absorb the next one.
The next model announcement will attract the headlines. The more consequential work sits underneath it: building an extensible data foundation that can accommodate the next model, the next regulation and the next workload while keeping data under consistent enterprise control.
The smartest AI strategy is not a perfect bet on today’s model. It is a data foundation that can keep extending without ceding authority over the data it serves. The building should outlive the tenant, and no model or platform should ever hold the deed to the enterprise’s data.