Ian Bitterlin, DCA Consultant Partner & Industry Expert, argues that separating AI training from inference is essential to understanding where the real opportunities – and infrastructure challenges – lie.

Government announcements describing ambitions to ‘lead the world in AI’ rely on an understanding of what ‘AI’ means – a hurdle to be cleared. An example of this was Linda McMahon, US Education Secretary, declaring that “A-one should be taught in schools from an early age”. But what do we mean by ‘AI’? Is it the creation or the use of AI?

Creating – ‘Training’

The current GPU-based technology being used in the creation of AI depends upon copying the most complex machine we know in the universe – the neural network of the mammalian cortex – by using vast arrays of microprocessors to run algorithms (written by humans) that ‘read’ vast quantities of digitised text and create links between them to answer specific, multifaceted questions. If the question is poorly phrased, the output can either fail or result in ‘boilerplate’.

The more thought that goes into the question, the better the quality of the answer. So, the user who is experienced in the search process will benefit the most from AI.

A good place to start reading about the pursuit of artificial intelligence is The Singularity Is Near by Ray Kurzweil, the ‘singularity’ being the point where a single machine would reach the cerebral cortex capacity of a single human. When the book was written in 2005, the fastest supercomputer in the world had just about reached the cortex capacity of a honeybee, and even with the historical exponential growth of computations/US$, it will still take decades to reach a single human cortex unless we break free of silicon-based transistors.

The human brain has 86 billion neurons, each with 7,000 synaptic connections to other neurons – synapses. A three-year-old child has 10^15 synapses, which decline by adulthood to 100–500 trillion. These work in parallel – which explains why our attempts so far result in huge, single-site data centres. The age of the 10 GW facility, enough power for 10 million people, is already in sight.

The link between transistors and neurons is often oversimplified to the ‘single unit of compute’, with comparisons using the single transistor as the base unit versus a single neuron. However, a single neuron can form a complete system with multiple inputs, both analogue and digital, and multiple outputs.

On the scale of neurons in one creature, we humans are not at the top either in total neurons or in the number in the cerebral cortex attributed to intelligence, and so ‘AI’ can be achieved at many levels:

• The fruit fly has 100,000 neurons, but only 2,500 in its cortex for intelligence
• A mouse has 14 million neurons for intelligence
• An elephant has 5.6 billion
• A gorilla has 9 billion
• A human brain has 16 billion
• But the orca has 43 billion neurons

As the number of neurons increases, so does the ‘intelligence’, but one big problem is that a human brain uses a few watts whilst our GPU solution may need 10 GW, so we are currently using a huge hammer to crack the very small ‘nut’ that nature gave us.

The fundamental difference between electronic computers and the cortex is that our traditional computer design relies on very fast sequential computation, whereas the mammalian brain carries out billions of digital and analogue computations in parallel.

It is highly likely that the future of AI computing will be dominated by quantum and/or biological technology purely on energy grounds… but the race is now on.

The human-written algorithms that AI uses to create new texts can save the user vast amounts of time but can end up producing nothing better than one human could achieve, given access and sufficient time. The results of an AI search depend totally on the quality of the question.

These are the so-called AI ‘training’ facilities, where large language models (LLMs) are trained in ever-increasing numbers of data centres that consume more electrical energy than large cities in a single location.

The GPUs have to be as close to one another as possible, limited by the speed of light, and there have to be many hundreds of thousands of them. So, planned facilities have quickly moved from 0.5 GW (enough for a city of 1 million people) to 1 GW and, more recently, to 3 GW – enough for a major capital city.

A single 3 GW training facility is 10% of the UK national grid – negatively impacting all sustainability goals and placing huge pressure on the locations where the infrastructure might be suitable. Even in the USA, with a grid capacity 10 times that of the UK, the solution for AI training has quickly generated off-grid, natural gas-fuelled energy solutions to produce a quick-to-market result. In economies that are prepared to pursue dominance in AI training regardless of climate change, we can witness numerous plans to build off-grid using fossil fuels.

So, the most likely answer to the question ‘will the UK attract GW-scale AI training facilities?’ is ‘no’.

The creation of AI training, with GPUs, power, cooling and software infrastructure solutions (all imported), is probably beyond the capability of an economy the size of the de-industrialised UK. The biggest AI players, all based on NVIDIA GPUs, have already invested trillions of US dollars in the race to be ‘first’ and dominate the predicted demand for AI, whilst the UK does not have the grid capacity or any location for just one >3 GW substation within four years, nor the human resources, hardware manufacturing base or capital to catch up. It also has a government that is supporting the highest electricity prices in the world, and a belief in ‘net zero’.

So, for AI training facilities to be built anywhere:

• A minimum of 1 GW can be connected within two years
• Probably off-grid, natural gas (nuclear is the lowest-carbon solution, but has too long a gestation period)
• Emergency back-up generation has taken a step backwards and is seen as an option
• Full-scale 2-4-hour BESS is thought of as a substitute for emergency power generation, but the generation has to be able to recharge it
• Whether the energy plan is environmentally sustainable is a question for each country, but GW-scale continuous fossil-fuelled generation is back on many agendas
• The cost of energy has been a key issue in the recent past, but the present GPU solution is driven by power availability, yet the UK market price is a barrier to adoption

Using – ‘Inference’

Using AI is enabled through inference data centres, close to the paying users, with low latency and most likely to be ‘edge’ facilities at 30–50 kW each. They will deliver the tools and offer software packages for the application of AI in a wide variety of use cases.

Using AI to enhance the economy does not require the country to house training facilities. Used with skill, AI inference will dramatically enhance the productivity of people and systems, bringing AI into commercial, personal and industrial areas that will help companies grow, produce skilled jobs and add to national GDP.

The economy of the country will become AI-ready without risky investment or inordinate delay.

As AI continues to evolve, probably towards quantum solutions, single cabinets will increase in compute capacity and decline in power consumption.

The inability to host, or the unaffordability of hosting, training facilities in-country also has the potential upside of de-risking the future of AI training if it evolves into quantum and/or biological computing, with lower power requirements by several orders of magnitude.

The solution is truly ‘computing at the edge’, with distributed micro-data centres offering all the benefits of AI without the delay, high capex and high environmental impact of training facilities. Each metropolitan area will have many hundreds of AI inference edge facilities, and the revenue streams from connectivity and usage will outstrip those from training.

So, in conclusion, AI inference facilities can be built anywhere, with the following advantages:

• Low-latency connectivity housed in existing buildings, with rooftop 5G/6G for mobile applications, leveraging existing fibre
• Power capacity for 30–50 kW applications, with time-to-market measured in months, not years
• No supply chain shortages for large electrical power and mechanical cooling plant
• The opportunity to utilise existing building emergency generation, or avoid generation completely, relying on overlapping cells
• Liquid cooling enables 100% reuse of waste heat, with sustainability not requiring district heat networks