Gordon Johnson, Senior CFD Manager at Subzero Engineering, discusses how CFD simulations can help optimise cooling and improve energy efficiency in data centres.
The recent rise in AI has led to vast amounts of computational power and data storage being required. This has led to a significant increase in the need for high-performance servers capable of handling complex tasks, pushing data centres to adopt a more robust infrastructure. However, promises to invest in AI products are clashing directly with assurances to reduce CO2 emissions.
Even tech giants such as Google and Microsoft admit their net-zero targets are in jeopardy. In fact, Google recently revealed that its greenhouse gas emissions have risen 48% over the past five years and that electricity consumption by data centres, alongside supply chain emissions, were the primary cause of the increase, endangering its goal of being carbon-neutral by 2030.
The International Energy Agency has estimated that data centres’ total electricity consumption could double from 2022 levels to 1,000TWh (terawatt hours) in 2026, with AI resulting in data centres using 4.5% of global energy generation by 2030, according to calculations by research firm SemiAnalysis.
As power and heat loads continue to rise, efficient cooling in the data centre is becoming more and more challenging – cooling is considered responsible for up to 40% of a data centre’s total energy consumption. In addition, data centre thermal management is often a reactive process, meaning we tend to wait for server alarms or increased server inlet temperatures before we react.
What happens if critical components within the data centre, such as cooling units, fail unexpectedly? Even worse, what happens if all cooling simultaneously fails due to a power outage or other significant event such as a failure at the chilled water plant?
The answer? Modeling and simulation with Computational Fluid Dynamics (CFD), which includes software specific to the data centre environment.
Additional key benefits
CFD software creates a 3D model or digital twin of the physical attributes of the data centre, including the performance characteristics of the cooling units, ITE, and other significant equipment, described by some as an MRI of what is really happening inside the data centre. CFD provides a detailed understanding of the complex interactions between airflow, heat removal from the ITE, and cooling systems, enabling data centre operators to make informed decisions to reduce energy consumption and improve overall cooling efficiency.
The key benefits of using CFD include, but are not limited, to:
- Identifying potential server hot spots
- Validating the pre-construction phase of a data centre
- Correctly placing additional ITE and cooling units in existing data centres
- Optimising airflow and reducing excess and unnecessary cooling
- Simulating via transient modeling the ITE ride out time due to power and/or cooling system failure.
CFD can aid in the design phase of data centres by simulating a variety of layout configurations. It looks at the placement of servers, racks, cooling units and airflow containment systems to ensure optimal thermal performance starting with the design phase down to the operational phase of the data centre. This means that designs can be validated by avoiding and preventing costly mistakes.
However, CFD is not just for new design. Legacy data centres can also take advantage of the benefits that CFD provides, including simulating the results of proposed changes to the existing room layout such as capacity redistribution or ITE expansion. In all cases, a CFD is always beneficial because it assists data centre operators to make data-based informed decisions while minimising or eliminating critical potential equipment downtime.
A CFD also accurately simulates and provides a detailed analysis of airflow and temperature distribution at all stages of the cooling process. These patterns include temperature distribution in both cold and hot aisles, in addition to the temperature across the server inlets from the bottom to the top of the racks. CFD simulations further evaluates the performance of different cooling strategies. For example, it can simulate air cooling strategies deployed in conjunction with liquid cooling designs, such as those using air cooling to remove the supplemental ITE heat not removed via the circulating liquid in direct-to-chip cold plate cooling.
Additionally, CFD now has the ability to accurately reduce risk from power and/or cooling system failure with transient modelling, showing via time increments what will happen to the ITE when either cooling is lost or both cooling and fan power (airflow) fail. When both cooling and fan power fails throughout the data centre, cooling is not lost immediately since the fans will take some time to come to a complete stop. CFD can simulate this time period, showing how hot the ITE server inlets will get and how soon this will happen before power is restored to the data centre.
The carbon footprint
The growing focus on sustainability has highlighted our need to work towards energy-efficient data centre designs, recognising the need to reduce our energy footprint while maintaining safe and optimal operating temperatures at the ITE. CFD assists with this important endeavor by identifying server hot spots before they occur, resulting in reduced downtime and improving overall ITE reliability.
This further aids in extending the lifespan of the ITE by ensuring it operates within ASHRAEs (American Society of Heating, Refrigeration, and Air-Conditioning Engineers) recommended thermal temperature guidelines. This way, potential issues can be addressed before they lead to costly problems, improving both the overall reliability and uptime of the data centre.
Using CFD is one of, if not the, most important step in determining data centre energy efficiency since it provides a detailed understanding of the complex interactions between airflow, heat and cooling systems. This enables data centre operators to make informed decisions to reduce both energy consumption and the data centre’s carbon footprint.
The challenges
Implementing CFD into existing data centre management processes is not without its challenges. As with any analytical tool, the accuracy of assessments and simulations depends on the quality of the data used. For accurate modelling, a baseline of how the data centre is currently operating is crucial for reliable results, and this should include current rack kW, cooling unit supply and return temperatures, rack and room leakage, as well as other additional room and airflow characteristics.
Data centre experts agree that all new data centres should always perform a CFD before construction, while existing data centres should also perform a CFD before making any ITE and cooling changes for design validation.
What’s next?
Real-world case studies and success stories highlight the impact of CFD in data centre optimisation by demonstrating significant improvements in energy efficiency, cooling effectiveness and operational performance. In the future, we expect to see CFD used extensively with the integration of machine learning, AI, ITE, and hybrid cooling infrastructure designs, with a continued priority on sustainability.
CFD modelling is a powerful tool for designing and validating efficient technology solutions for new and legacy data centre environments by providing predictive results that bridge high-performance computer server operations with the critical mechanical system. CFD also safeguards the data centre by simulating failure scenarios that enable data centre managers to better understand and design the appropriate redundancy for power and cooling failure scenarios.
Armed with data from the CFD analysis, a suitably experienced technology partner can advise customers on impactful data centre design changes, upgrades, or tweaks to improve operating improvements – all of which will help optimise and reduce energy usage while safeguarding the data centre.
