Should I deploy water cooling or air cooling in my datacentre?

Datacentres have employed cooling solutions ever since the first one got too hot and someone opened a window. Going back even further to the era of Colossus and the Bletchley Park codebreakers, water-cooling was used to stop valves from warping and cracking.

Today the sensitive and sophisticated demands of datacentre environments have given rise to ingenious cooling innovations that occupy two camps: those that directly cool the air of the datacentre environment, and those that use water as a cooling medium.

Each has its own set of advantages and disadvantages that buyers must weigh up before making their choice of cooling strategy. This guide spells them out.

Why is cooling strategy such a critical issue?
What are water cooling and air cooling anyway?
What factors should be considered when choosing between the two?
What happens when you add free cooling into the equation?
Water cooling
Air cooling
Glossary of terms

Why is cooling strategy such a critical issue?

Cooling is a strategic consideration, rather than a tactical one, for datacentre operators for three principal reasons:

Cooling costs are a huge part of the datacentre energy bill

The world and its 7.5 billion people consume a lot of energy, around 3% of which is used exclusively by datacentres.

Around 50% of that energy is spent powering cooling systems to ensure that datacentre environments don’t overheat.

Anyone who operates a datacentre will be familar with PUE (Power Usage Effectiveness) ratings and the target of getting this as close to 1.0 as possible. Doing so not only drastically reduces the cost of operating a datacentre, but also its impact on the planet.    

IT infrastructure components are highly sensitive to extreme temperatures

Leave your smartphone or tablet in the sun for too long and it will turn itself off in an attempt to protect its components from irreparable damage. Consumer IT devices are far tougher than their datacentre counterparts because they are designed to withstand a broad spectrum of operating temperatures. Tiny wafers of plastic and metal (technically, silicon is a metalloid) not only stop working in extreme heat; they distort and melt. As the miniaturisation of processing power into smaller and smaller form factors reaches its vanishing point (many scientists believe that Moore’s Law is no longer sustainable), the physical limitations of heat tolerance don’t become any easier.

Uptime is an absolute business imperative

The prospect of some unplanned system downtime has never been enjoyed or encouraged by any organisation that relies on IT. Yet today’s total reliance on IT systems for communications, business processes and productivity makes downtime unthinkable. Lost seconds can lose millions for some organisations. This places extraordinary focus upon the physical layer of datacentre environments; how they are managed, controlled and optimised. Already conspicuous because of its significance to the overall energy bill, cooling is right under the spotlight.

What are water cooling and air cooling anyway?

It’s easy to get confused about the differences between water cooling and air cooling, because the objective of all datacentre cooling is to cool air. Confusion can also be compounded by the fact that air-based cooling systems invariably use some form of liquid refrigerant. For the purposes of this guide, the distinction will be as follows:

Water cooling is where water enters the environment to create a cooling effect on datacentre infrastructure

Air cooling is where cool air is used to directly cool infrastructure within the datacentre environment 

The principal reason for making the distinction along these lines is to denote the kinds of solutions that must be implemented to realise each strategy. In water cooling, a water supply must be pumped under pressure inside the datacentre environment. In air cooling, fans, vents and physical barriers will be key features of the solution.  

What factors should be considered when choosing between the two?

Datacentres are dynamic environments with ever-changing demands that are heavily influenced by disruptive new digital technologies. These, in turn, impact upon the cooling requirements that organisations face. The chief factor, therefore, is assessing not only what present demands are, but also how these are likely to change in the next 3–5 years.

The super-dense datacentre

Space is a premium within on-premise and collocated datacentre environments. This, combined with the evolutionary path of semiconductor technology and the onset of IT virtualisation, has led computing infrastructure to become progressively more dense.

All of this adds to the power budget which, in turn, turns up the temperature gauge as emitted heat.

The question is, how much denser (and therefore hotter) can your datacentre get? How rapidly will power budgets increase? These are key considerations when forming a cooling strategy.  

Exponential data growth

All of us are creating and consuming more information. Organisations are increasingly required to archive information for regulatory compliance purposes.

Cisco predicts global IP traffic to triple from 96 Exabytes per month in 2016 to 278 Exabytes per month in 2021.

FTTH and next generation broadband are bringing higher data transport speeds to homes and businesses.  Ethernet bandwidth within the datacentre has jumped from 1Gbps to 100Gbps in less than a decade, with 400Gbps the next standard on the cards.  All of this is adding the scale of datacentre infrastructure, and with it the designs necessary to deliver the optimum cooling capacity.  Once again the questions are how much and how fast.

The impact of cloud computing trends

Cloud computing is having a disruptive effect on cooling strategies, but not necessarily in the way you might expect. While cloud is undoubtedly feeding and being fed by exponential data growth (and contributing to the pressure on datacentre density), it is also changing how organisations use their own datacentre assets.

Some datacentres are shrinking rather than growing, as businesses elect to switch their private-cloud oriented IT consumption models to a public cloud or hybrid approach.

The rate of larger, new build datacentres is slowing, though all this data has to go somewhere which explains the rise of mega datacentres such as those owned by Microsoft (Azure), Amazon (AWS) and Google (GCE).  Many organisations are looking at how best to retrofit new cooling strategies into an existing environment rather than wait until the next greenfield project to start afresh.  

Revolution vs. evolution

Prudent organisations looking at their future cooling strategy will be mindful of what cooling investments they currently have in place. This should inform whether a root and branch rip-out and replace approach is warranted, or else some subtler tweaks.  Perhaps the change should be immediate in order to obtain the greatest benefits? Perhaps it should be a steadier migration, in line with projected new demands and expectations?

There is no doubt that introducing an entirely new cooling methodology will require careful planning to mitigate risks and safeguard investments.

This would certainly be the case when replacing a mature air-based cooling system with a new water-based solution.  Cases of water-based systems being replaced by air-based solutions are far rarer, but would be equally disruptive if not managed appropriately..

What happens when you add free cooling into the equation?

The normal business case for air or water-based cooling would typically come down to:

  • How much cooling load can be achieved
  • How efficiently it can be achieved (i.e. how much energy is consumed)
  • How much the solution costs to procure, install and maintain

Free cooling introduces the principle of using ‘freely available’ cool air from the atmosphere and/or a cool water source such as a deep lake or lagoon to support datacentre cooling objectives.  This can have a significant impact on the business case, as it radically increases the efficiency of cooling because energy costs have been circumvented.  

Some free cooling solutions can even be a net contributor to the business case, adding value (effectively, revenue) rather than only consuming cost.

This is achieved by diverting heated air and/or water into the heating system of the building in which the datacentre is housed. Cooling solutions that do not use ‘free cooling’ principles can also achieve this, although the effective on the business case is lessened by the expense of consuming more energy.

However, it is important to note that free cooling solutions are not entirely ‘free’.  

Any free cooling system still needs to be designed, built, tested and maintained.  Free cooling will also likely play a contributory or partial role in the overall cooling objective, rather than taking on the full cooling load. Rather like an electric fuel cell in a hybrid vehicle, a combustion engine is still used to help achieve the required performance, though the fuel cell facilitates this extremely efficiently.  

Water cooling

Water cooling introduces water into the datacentre environment rather than using chilled air to circulate around equipment.  There are different approaches to water cooling, distinguished by their proximity to physical IT equipment. Highly-targeted water cooling systems have been developed to run water inside CPUs, while others are designed to attach to the rear panels of server racks where heat is expelled.

Pros

Heat conduction

Water has a thermal conductivity of around 0.6, which doesn’t sound all that impressive until you compare it with air which – at normal atmospheric pressure – comes out at just 0.02.  Put cold water down a copper pipe (thermal conductivity = 400) and you have a good means of performing efficient heat exchange.  

Like its air-based equivalent, water-based cooling solutions are available as targeted solutions that sit within rows of cabinets (In-Row cooling) or attached to 1U racks inside the cabinet itself.  These close-coupled cooling approaches use CRAHs to bring chilled water into very close proximity to the hot IT components, enabling efficient heat exchange to take place across short distances.  

Exciting new free cooling applications

It isn’t just cold outside air which is driving free cooling solutions; water-based innovations are getting in on the act with some high profile examples. These include the cloud computing division of Chinese IT giant Alibaba which has a datacentre in Shanghai that pumps water in from the nearby, 120m deep Lake Qiandao. Google’s sea-water cooled datacentre in Hamina takes its supply from the Gulf of Finland which has an average temperature of 0oC in winter.  Others pump water up from naturally cool groundwater aquifers.

Cons

Leakages could be catastrophic

Needless to say, water and electronics don’t mix.  Many water-based cooling systems now feature safety measures to mitigate the risk of leakage.  However, there is no getting away from the fact that water can be a dangerous and volatile substance prone to turning to gas or solid at extreme temperatures, corroding metal, plastics and even rock.  It can also harbour micro-organisms that multiply and cause blockages, hydraulic expansion and public health problems.  Such effects are unlikely but must be managed and considered as part of any water-based cooling investment.

Potentially disruptive retrofits

This disadvantage could be levelled at both forms of cooling approach, however it is more likely that an existing datacentre will already use an air-based system. Replacement of a water-based cooling system with an air-based system is a rare occurrence indeed.  Any decision to retrofit a water-based system into a datacentre that currently uses an air-based system may have any number of complications, depending on the solution chosen. These can be offset by the strength of the overall business case making sound financial sense, and by the presence of redundant cooling and power units to enable a staged migration process that does not result in planned or unplanned downtime.

    Air Cooling

    Air cooling is a more rudimentary approach to managing datacentre temperatures and is most likely to be employed by the smallest and simplest environments. Solutions range in complexity from perimeter-based approaches that situate CRACs around the walls of a datacentre, to more targeted solutions positioning between or even inside rows of cabinets. All air cooling approaches endeavour to separate hot air from cold air, and designs are heavily dependent upon room layout and the specific positioning and orientation of IT equipment.

    Pros

    Safety

    Air-based cooling systems are fundamentally safe and use basic principles familiar to every school pupil, let alone qualified datacentre professionals. Even the invisibility of air has now been revealed through computational fluid dynamics (CFD) and thermal modelling systems that allow legacy cooling solutions to be optimise for clear efficiency improvements.  This is a mature technology with advanced capabilities such as hot/cold-aisle containment (H/CACS) that are still innovating at pace.

    Excellent for lower density deployments

    While water-based cooling arguably has the greater case for the most dense and demanding datacentre environments, air-based systems still deliver excellent performance at the lower density range.  Even without adding complementary free cooling capabilities, an air-cooled system using CRACs in an In-Row or In-Rack format – potentially as part of a H/CACS deployment – can deliver good cooling results and PUE.

    Proven free cooling benefits

    The availability of ‘free’ cool air in most climates (at least for part of the year) has allowed free cooling solutions that utilise air intakes to become the predominant form of the technology. Installation and maintenance of these solutions can be comparatively simple, depending on the sophistication of the components and the load stress placed on the system. Such solutions integrate logically with conventional CRAC-based cooling solutions.

    Cons

    Energy intensiveness

    Air is a gas, giving it an immediate disadvantage to liquids and solids in its comparative ineffectiveness as a heat exchange medium. This is a significant contributor to the complaint that air-based cooling solutions are not as efficient as water-based solutions at high IT equipment densities.  It simply requires too much energy to cool the air surrounding hot IT equipment to an extent sufficient to support a cool datacentre environment.  

    The trend toward warmer datacentre environments has alleviated this somewhat, making air-based systems appear less inefficient in those deployments where datacentre operators are happy to run higher temperatures without it having a meaningful effect on rates of equipment failure. However, the same efficiency gains can be made with water-based systems, by setting the water intake temperature at a higher level, thereby reducing load on the water chillers.

     

    Glossary of Terms

    CACS

    Cold-Aisle Containment System.  An enclosed datacentre environment that uses physical barriers to prevent hot and cold air mixing so that the cooling process is as efficient as possible.  In CACS, the cold aisle between the front sides of two inward facing rows of server cabinets is contained, leaving the rest of the datacentre space warm/hot.

    CFD

    Computational Fluid Dynamics – a method of applying the laws of physics to model and visually represent the present or anticipated behaviour of a gas or liquid as it interacts with objects or other gases/liquids.  Used in datacentres to model flows of hot/cool air.

    CRAC

    Computer Room Air Conditioning unit – operates on the same principle as everyday air conditioning systems, in that it takes hot air in and expels cold air out.  However, unlike a traditional air conditioning unit, a CRAC can control humidity and is designed to be precision managed and aligned to cope with the specific requirements of an IT infrastructure environment.

    CRAH

    Computer Room Air Handler – similar to a CRAC but uses a water chiller to remove heat.

    Ethernet

    A system for connecting computer systems to form an area network using standardised protocols for the high speed transmission of large quantities of data.

    Exabyte

    1 quintillion bytes of information.

    Free cooling

    The concept of cooling a datacentre environment using freely accessible air and/or water instead of, or in addition to, conventional chillers.

    HACS

    Hot-Aisle Containment System.  An enclosed datacentre environment that uses physical barriers to prevent hot and cold air mixing so that the cooling process is as efficient as possible.  In HACS, the hot aisle between the rear sides of two outward facing rows of server cabinets is contained, leaving the rest of the datacentre space cool.

    Heat exchange

    A core principle of thermodynamics that allows a solid and/or fluid to transfer heat to another.

    Hybrid cloud

    An IT platform leveraging resources from numerous different private and public cloud service models.

    Private cloud

    Cloud computing used by and deployed at a single organisation, typically within an on-premise datacentre environment

    Public cloud

    Multi-tenanted, shared IT resources made available by a public cloud service provider on a pay-by-use or subscription model (if not for free) under the auspices of a service level agreement.

    PUE

    Power Usage Effectiveness is a ratio expressing the efficiency of total power delivered to a datacentre facility to be used by the computing equipment within in.  The lower the PUE (1:1 would be the lowest theoretically possible), the more efficient the datacentre is at converting its electricity consumption into value-generating IT-driven activity.  Cooling datacentre IT equipment is typically the greatest challenge to achieving a low PUE.

     

    Here to help

    Let’s talk about your digital aspirations and the next steps to take you on your journey to hybrid cloud. Get in touch and we’ll show you the way.