Mikael Holmberg, Distinguished Engineer and Member of the Office of the CTO at Extreme Networks, explains why even the most powerful AI infrastructure will struggle to deliver if the network cannot keep pace.

Most data centre networks were built on a reasonable assumption: not everything needs to talk to everything else at once. AI has challenged that assumption.

Networks today need to be built for the realities of modern inference, including variable demand, always-on services and strict requirements for consistency and control. As enterprises move towards supporting AI, the underlying economics are also shifting, with token-based pricing, data egress fees and the rapid growth of GPU-intensive workloads all influencing infrastructure decisions.

Every GPU is communicating continuously with many others in lockstep, and when communication slows, everything drags. Because those GPUs are costly, every dropped packet or moment of latency can translate into longer training times and higher costs.

The network now plays a direct role in determining the performance, efficiency and return on investment of AI infrastructure.

AI changes the nature of data movement

If one GPU falls behind, the entire workload can suffer. A dropped packet can disrupt communication between nodes, and that delay can ripple across the cluster. When hundreds of high-value accelerators are left idle because of a network bottleneck, the cost quickly adds up. Every extra minute of AI training time comes at a price.

Traditional data centres were built primarily for north-south traffic: users and the internet coming in, and responses going back out.

With AI, a much greater proportion of traffic moves east-west, going from server to server, VM to VM and container to container, all inside the data centre. The network is no longer simply a transport layer. To keep up with the scale of AI deployments, data centre network architectures are evolving.

For the most demanding production AI environments, this can mean using a high-performance network fabric designed to minimise packet loss, reduce latency and provide east-west bandwidth at scale. The distinction between the traditional data centre network and the AI fabric is therefore becoming increasingly important.

Why traditional designs fall short

Many traditional data centre networks rely on oversubscription, which made sense when workloads were independent and traffic was intermittent.

But in training environments, each GPU continuously exchanges data with many others. The assumption that only a portion of the network will be active at any given time no longer holds in the same way. Oversubscription ratios of 3:1 or 5:1 that were once acceptable can introduce bottlenecks that degrade performance across the whole system.

Bandwidth is also feeling the strain. As AI workloads scale, requirements are rising rapidly, from 400G to 800G, with 1.6T now on the horizon. At the same time, there is far less tolerance for latency and packet loss.

AI environments demand consistency and reliability at a level that many traditional networks were not designed to provide. Technologies such as RDMA over Converged Ethernet (RoCE) are consequently moving from an advanced option to an increasingly important design consideration.

Towards an AI-ready network

Meeting these demands requires more than faster pipes. It requires rethinking what a data centre network is actually designed to do.

The starting point is capacity. Networks need to support sustained, simultaneous, full-throughput communication rather than being designed solely around statistical averages.

Reliability follows from that. Predictable performance means low latency, minimal jitter and near-lossless operation. Best-effort delivery was acceptable when the network was viewed primarily as transport. It’s less acceptable when the network sits in the critical path of a significant AI investment.

The deeper shift is architectural. AI systems are not simply collections of servers running independent tasks. They are coordinated, distributed systems in which compute and communication are inseparable. The network has to reflect that.

This is what the idea of an ‘AI fabric’ means: not as a marketing term, but as a recognition that the network is part of the application itself.

Planning for what comes next

For many organisations, the challenge is not just building new infrastructure, but adapting existing environments to support AI workloads alongside traditional ones. Not every workload needs the same level of performance, but those that do will quickly expose weaknesses in the underlying network. Identifying where those pressures will emerge and designing for them early is critical.

This applies to inference as well as training. As organisations move from experimenting with AI to running it in production, inference workloads can place sustained demands on the network.

What this means for AI investment

McKinsey estimates that data centres worldwide could require $6.7 trillion in capital expenditure by 2030, including $5.2 trillion for infrastructure capable of supporting AI workloads.

The data centre has always evolved in response to what runs on it. AI is not the first workload to force a rethink, but it is one of the clearest examples of network and application performance becoming directly coupled.

That changes the calculation for anyone investing in AI. You can have the best model, huge amounts of compute and an ambitious use case, but none of it really matters if the network cannot keep up. Infrastructure decisions that might once have seemed secondary are now central to whether AI delivers on its promises.

The network is not behind the scenes anymore. It’s central to how the show runs.