Phil Burr, Head of Product at Lumai, makes the case for optical computing in data centres, and how it can help facilitate the increased demands from AI in a sustainable way.
The AI boom is creating a performance demand which is far outpacing the capabilities of present processor technology in data centres. The current answer is to add more silicon area, more cost and more power. It’s a strategy that is chasing diminishing returns.
The cost of trying to maximise performance in current AI accelerator products can be incredible. As one example, a few months ago, Nvidia’s CEO Jensen Huang revealed the company had spent $10bn developing its Blackwell GPU, with the product costing $30,000 to $40,000. Alongside the large infrastructure costs needed for tasks like power-delivery and cooling, combined with the immense environmental impact of data centres, it’s clear the industry is creaking under the pressure of finding a new approach to tackle its power, cost and sustainability challenges.
From innovative cooling techniques to green energy development, there are a range of advancements attempting to improve sustainability and efficiency. But when it comes to accelerators, one innovation is set to transform how the sector achieves a leap in AI inference performance and energy efficiency while also reducing the total cost of ownership (TCO). This is 3D (“free-space”) optics.
Why use 3D optics?
The maths that underscores AI is perfectly suited to optical compute – if leveraged in the right way, it can deliver immense enhancements to performance and efficiency. There are two optics solutions in the tech world: integrated photonics and 3D optics.
Integrated photonics has been trialled with AI processing, but the companies trialling such an approach have now focused their technology on other areas (interconnect or switching). The technology has exciting potential to overcome limitations in what are called electronic-only interconnect solutions. But when it comes to processing, integrated photonics falls short of delivering the performance needed,owing to factors like poor scalability and low compute precision.
The use of 3D optics overcomes the limitations of electronic-only AI solutions. By computing with photons instead of electrons, and performing highly parallel computing, 3D optics can deliver the jump in inference performance while using approximately 10% of the power of a GPU.
Why it’s perfect for data centres
An optical AI accelerator can harness the key benefits already seen in everyday optical communications. These include wavelength multiplexing, rapid clock speeds and negligible energy consumption. As AI applications rapidly grow in size, these accelerators are substantially more scalable than current ‘2D’ chips as they use all three spatial dimensions to perform computations.
How exactly does it do this?
Matrix multiplication, the maths process behind processing, has three elements: copying, multiplying and adding. The optical accelerator performs these tasks by manipulating millions of individual beams of light – a process called matrix-vector multiplication (MVM) – with data encoded in the laser beams. The entire MVM, encompassing millions of parallel operations, can take place in a single clock cycle while using very minimal energy. Brilliantly, the system actually develops more power efficiency as its performance increases because of its quadratic scaling advantage.
Without a considered design, accelerator performance can be hindered by memory bandwidth. But with an optical processor, this memory can be spread out across the width of the vector, allowing for greater memory bandwidth without the use of costly HBM.
The types of components used in these optical processors have already been proven in data centres. In fact, Google has been using these in its Optical Circuit Switch for many years now, illustrating the reliability and effectiveness of using similar technology within the exacting data centre requirements.
Sustainability and TOC priorities
In the face of these new challenges, as an industry we should remember that between 2015-2019, despite data centre workloads nearly tripling, power demand “remained flattish”. It shows it can be done, and we now need to discover innovative ways to facilitate the AI revolution without devouring evermore energy, especially as power demand is currently set to grow by 160% by 2030 due to AI.
The sustainability issue of course extends beyond energy generation. Each extra Watt of power used means more power and cooling infrastructure is required and more emissions are produced as a result of this. By reducing the power needed for AI accelerators, we can further the lifespan of datacentres, resulting in fewer new constructions and fewer emissions generated from this.
Sustainability often works in line with reducing TCO. The great aspect of a 3D optical processor is that it can use standard optical and electronic components already present in datacentres – they’re just re-engineered. There’s no need for the latest silicon technology or costly HBM memory. Consequently, manufacturing costs are massively reduced as well as the capital cost for operators. If we combine these savings with a lower need for power and cooling architecture, the TCO that emerges is a fraction of a GPU.
Time to embrace new techniques and technologies
From a technical viewpoint, the surge in AI performance demand is creating a need for new solutions to solve limitations that the industry is facing. But the mission goes beyond creating better performance alone.
We’ve seen this month with Google just how much emissions have risen as a result of AI development. It’s a necessary and sharp reminder that we have to do more, and this comes down to a willingness to adopt new techniques and technologies.
Policymakers have a major part to play in creating sustainable infrastructure and energy sources that datacentres can harness for their own needs. But the industry needs a hardware solution to power an AI revolution without adding to its energy consumption – and the answer could lie in 3D optics.
