As agentic AI moves from experimentation to real-world deployment, Gopi Duddi, Chief Technology Officer at Couchbase, explains why enterprise data architecture must evolve to support it.

Enterprise technology estates rarely fail because of one bad architectural decision. More often, complexity builds gradually: a database here, an integration layer there, a new cloud environment after an acquisition, and another data platform for the next transformation programme.

The result is an enterprise ‘Frankenstack’: an architecture assembled over years to support successive generations of applications and workloads. While that complexity has often been manageable for traditional applications, AI agents are changing the equation.

As enterprises move from AI experimentation to production-scale agentic applications, the data infrastructure beneath them is becoming a critical constraint. Agents need access to multiple sources of operational data, real-time context and persistent memory to operate across distributed environments while maintaining governance and predictable performance.

AI is turning data fragmentation from an architectural inconvenience into a business and operational risk.

AI exposes the cost of fragmentation

Enterprise data is no longer concentrated in a handful of systems. Cloud migration, acquisitions, SaaS adoption and successive transformation projects have left many organisations managing information across legacy databases, cloud services, data warehouses, object stores and specialist platforms. These systems may also sit in multiple regions and infrastructure environments.

Traditional application architectures could accommodate this fragmentation through carefully defined integrations. Workloads were generally bounded: an application accessed a known set of systems to execute a defined business process. Agentic applications are less constrained.

An AI agent may need to retrieve a customer’s current account information, historical interactions, product data and operational status before deciding what action to take. It may then need to write information back to a transactional system, invoke another service and retain the relevant context for a future interaction.

Increasingly, that context isn’t just structured records. It’s unstructured and semi-structured data that agents need to search semantically, not just query by key or field. An agent reasoning over support tickets, product documentation or prior conversations may require vector search capable of retrieving relevant context quickly and reflecting changes in the underlying data in real time.

The challenge is to connect these systems with sufficient speed and resilience for an autonomous workload operating at enterprise scale.

Adding more integration points can compound the problem; each additional connector, API, data pipeline or specialised store introduces another dependency to operate, secure and govern. The architecture can become more complex while the underlying data experience becomes less coherent.

The data plane becomes part of the AI architecture

The AI stack is often described in terms of models, GPUs and orchestration. While those components are important, they represent only part of what is required to run an agent reliably. Agents also require a data plane capable of providing persistent memory, retrieving relevant context, connecting to operational systems and enforcing governance across those interactions. This is driving a shift in how organisations think about AI infrastructure, away from treating data as a downstream resource for an AI model.

Memory is sometimes treated as a solved problem – a vector store bolted onto an application. But agent memory at production scale is an infrastructure problem, not simply a feature. It needs to persist across sessions, scale to support large numbers of concurrent agent interactions and stay consistent with the operational systems an agent is simultaneously reading from and writing to. Where memory lives in a separate system from the operational data it references, synchronisation lag and stale information can undermine an agent’s decisions.

For an agent, data is part of the execution environment. The quality, availability and latency of that data directly influence the agent’s ability to make decisions and take action. For customer-facing agents, data issues can quickly affect the customer experience: stale context can produce irrelevant responses, while slow retrieval or unavailable data can delay or prevent actions from being completed.

An AI data plane therefore needs to support capabilities that have traditionally been distributed across multiple infrastructure layers: operational data access, memory, contextual retrieval, governance and observability. The objective is to provide a consistent way for AI workloads to access and operate on distributed data.

Governance must extend to the agent

The expansion of AI agents also changes the governance model. Traditional application permissions are typically designed around human users and relatively predictable application workflows. Agents introduce a new class of actor: software that can retrieve information, make decisions and execute actions across multiple systems.

Organisations need to establish what information an agent can access, what actions it can perform, which identity or permissions it operates under and how those activities can be monitored and audited. Those controls must remain effective as data moves between systems and workloads operate across cloud, on-premises and edge environments.

The challenge becomes greater when an agent combines information from multiple sources. Data that appears innocuous in isolation can become sensitive when correlated with other enterprise information.

Latency and resilience are now data problems

While the performance of an AI application is often discussed in terms of model inference speed, in production environments the model is only one part of the latency equation. An agent may need to retrieve multiple pieces of context, query operational systems, access its memory and invoke tools before producing a response or taking an action.

If every step requires a separate network call or data movement between services, latency can quickly become the limiting factor. The same applies to resilience. An agent dependent on a chain of loosely connected services inherits the failure characteristics of that chain. A failure or degradation in one component can prevent the agent from completing an otherwise straightforward task.

This makes data locality, availability, replication and predictable access increasingly important design considerations for AI workloads. In some environments, particularly at the edge, moving data to a central location simply to make it available to an AI workload may be neither practical nor desirable.

The infrastructure therefore needs to bring computation and data access closer together where the workload requires it, while maintaining a consistent governance model across locations. Semantic retrieval can add another hop to that chain. If vector search runs in a separate system from the transactional and contextual data an agent also needs, decisions may require stitching results across systems, potentially compounding latency and the risk of inconsistency.

The AI era requires a different data foundation

The shift to agentic AI should not be interpreted as a requirement to replace every existing database or consolidate every enterprise system onto a single platform. Most organisations will continue to operate heterogeneous environments for the foreseeable future. The more important question is whether those environments can provide a coherent foundation for AI, allowing vector search, operational data access and memory to work with consistent, up-to-date information without unnecessary integration overhead.

That means addressing the technical debt that has accumulated across data estates: duplicated or out-of-sync information, inconsistent interfaces, brittle integrations, fragmented security policies and data that remains technically accessible but operationally difficult to use. When data is not synchronised across environments, agents risk acting on incomplete or outdated information. These weaknesses become more obvious as agents require broader, more dynamic access to enterprise information than traditional applications.

This creates an opportunity for infrastructure leaders: rather than treating AI as another workload to be layered onto the existing technology estate, organisations can use it as a forcing function to modernise the underlying data architecture. The priority should be creating a data foundation that allows AI workloads to access relevant information wherever it resides, with the necessary performance, resilience and controls.

The enterprise Frankenstack was built for an era of applications, integrations and human-driven workflows. Agentic AI demands something different: a data architecture designed for continuous access to context, persistent memory and autonomous action.