Applied Energy 89 (2012) 150–155

Contents lists available at ScienceDirect

Applied Energy
journal homepage: www.elsevier.com/locate/apenergy

Computational fluid dynamic investigation of liquid rack cooling in data centres

Ali Almoli a, Adam Thompson a, Nikil Kapur a, Jonathan Summers a, Harvey Thompson a,⇑, George Hannah b
a
Institute of Engineering Thermofluids, Surfaces & Interfaces (iETSI), School of Mechanical Engineering, University of Leeds, Leeds, LS2 9JT, United Kingdom
b
Airedale International Air Conditioning Ltd, Leeds, United Kingdom

a r t i c l e i n f o a b s t r a c t

Article history: Relying on thermal air management in a data centre is becoming less effective as heat densities from the
Received 12 July 2010 Information Technology (IT) equipment continue to rise. Direct liquid cooling is more efficient at trans-
Received in revised form 21 January 2011 ferring the waste heat, but requires liquid loops passing as close as possible to the heat source. A new
Accepted 3 February 2011
Computational Fluid Dynamics (CFD) strategy is developed for data centre scenarios where a liquid loop
Available online 25 February 2011
heat exchanger is attached at the rear of server racks (back doors), which can avoid the need to separate
the cold and hot air streams in traditional hot/cold aisle arrangements. The effectiveness of additional
Keywords:
fans in the back door heat exchangers is investigated using the three-dimensional CFD model of a sim-
Data centres
Liquid cooling
plified three-aisle, six-rack data centre configuration.
Heat transfer Ó 2011 Elsevier Ltd. All rights reserved.
CFD modelling

1. Introduction In this paper we focus on thermal air flow management and

The rapidly increasing energy demand of data centres is pre-
senting industry and governments with an energy supply problem.
It is clear that the current roadmap of higher density servers with
no radical changes in computing technology on the horizon will
lead to year-on-year increases in energy requirements since data
centre power consumption has doubled in the last 5 years and is
likely to double again in the next 5 years to over 100 billion
kW h [1]. Within the UK, the issue of thermal management in data
centres is becoming critical since power supply is becoming re-
stricted in key data centre locations such as London Docklands,
The Thames Valley and Manchester [2].
There are two main inefficiencies leading to such enormous en-
ergy requirements: the Information Technology (IT) hardware inef-
ficiencies and the cooling requirements, each accounting for
roughly 40% of the total energy usage, with the result that each
kW h of energy for data processing requires a further kW h for
cooling. The energy-efficient design of data centres is a truly mul-
ti-disciplinary problem. IT load inefficiencies can be addressed by
improved semiconductor technologies [2] and server virtualisa-
tion, while cooling of the electronics in data centres can be
achieved in a number of ways, by far the most popular at present
being via cold air, but there is a trend to adopt direct liquid cooling
[3] of the servers by, for example tube and fin heat exchangers at-
tached to the back of the server rack or, yet to be fully utilised,
dielectric liquid immersion cooling and on-chip spray cooling [4].
cooling in data centres, issues which are particularly challenging
since Computer Room Air Conditioning (CRAC) systems have to
maintain temperatures and humidity levels in narrow bands in or-
der to avoid catastrophic data losses due to over-heating servers,
hygroscopic dust or electric discharge failures. Strategies are usu-
ally based on the separation of hot and cold air via a layout of
hot and cold aisles, but efficiencies can be gained with either of
the two aisles contained. The CRAC units usually supply cold air
into data centres through raised floor tiles and the cold air passes
through the server racks, cools the electronic equipment and
emerges from the back of the servers as a hot air stream. Compu-
tational Fluid Dynamics (CFD) is increasingly being used to im-
prove air flow design in such systems [5,6]. The objectives of this
paper are to: (i) develop a new CFD modelling strategy to investi-
gate the effectiveness of liquid loop heat exchangers mounted to
the rear of the IT server racks, referred to as a back door cooler,
and (ii) apply the CFD model to a simplified three-row, six-rack
high-density data centre scenario where the back door cooler
either contains a series of fans (active) or no fans (passive), the lat-
ter relying entirely on the fans in the servers to push the air
through the back door cooler.
2. CFD modelling of thermal air flow in data centres
2.1. Mathematical model

Although Computational Fluid Dynamics (CFD) is now used to

⇑ Corresponding author. Tel.: +44(0) 113 343 2136; fax: +44(0)113 343 2150. analyse and design thermal air flows in data centres, they are still
E-mail address: h.m.thompson@leeds.ac.uk (H. Thompson). largely unverified with respect to the accuracy of their thermal

0306-2619/$ - see front matter Ó 2011 Elsevier Ltd. All rights reserved.
doi:10.1016/j.apenergy.2011.02.003
 A. Almoli et al. / Applied Energy 89 (2012) 150–155 151

predictions for large data centres [6]. In addition, the modelling investigations into the most appropriate turbulence modelling ap-
methodologies adopted for the all-important flow through server proaches are required, however this issue is not pursued here. The
racks have not been described adequately [1], making comparison two transport equations are:
with previous studies extremely difficult. In this paper we briefly  
@k 1 lt 2l
describe a modelling methodology for the thermal air flow in data þ r  ðkUÞ ¼ r  rðkÞ þ t Sij  Sij  e ð3Þ
centres with a particular focus on the representation of the server @t q qk q
racks and back door cooling units.  
Thermal air flows in data centres are usually complex, recircu- @e 1 lt e e2
þ r  ðeUÞ ¼ r  rðeÞ þ C 1e 2lt Sij  Sij  C 2e ð4Þ
lating air flows characterised by a hierarchy of different length @t q qe kq k
scales. A typical Reynolds number, Re, based on an air inlet velocity
with the turbulent viscosity defined via
from the supply vents of 1 m/s and a rack length scale of 2.4 m,
leads to an estimated Re  105 indicating the turbulent flow re- k
2

gime. Most previous CFD studies of data centre air flows have used lt ¼ qC l ð5Þ
e
Reynolds Averaged Navier–Stokes (RANS) models, see e.g. the very
recent study of Cho et al. [1]. The governing continuity and and the Sij terms are the deformation tensor. Five empirical con-
momentum equations, written in the RANS format, are: stants qk, qe, C1e, C2e, and Cl in Eqs. (3)–(5) are set equal to 1, 1.3,
1.44, 1.92, and 0.09 respectively (Boulet et al. [7]).
rU ¼0 ð1Þ The energy equation is also solved and takes the form:
  
@U 1 1 @T v vT 1
þ r  ðUUÞ ¼ r  ðr  qU 0 U 0 Þ þ S ð2Þ þ r  ðTUÞ ¼ r  þ rðTÞ þ SQ ð6Þ
@t q q @t Pr PrT qC p

where r ¼ PI þ lðrðUÞ þ ½rðUÞT Þ is the Newtonian stress tensor
and l is the air viscosity, q its density (defined using the ideal gas
law with constant pressure 1.013  105 Pa), U and U 0 are the aver-
age and turbulent fluctuation velocity vectors respectively, P is
the pressure and I is the unit tensor. The vector S represents the
additional momentum sources, which will be discussed in greater
detail below, and the qU 0 U 0 term is the so-called Reynolds stress
tensor that requires additional model equations.
Following Cho et al. [1], the CFD models developed here use the
standard k–e model where the turbulence is described with two
additional variables k (turbulent kinetic energy) and e (turbulent
dissipation). This model requires the flow to be fully turbulent,
which may not be the case in all areas of the air flow within the
data centre, but the model is well tested and popular for flows that
have complex geometries and heat transfer. Clearly, more detailed
where T and v are the temperature and dynamic viscosity respec-
tively and Pr is the Prandtl number defined by
v k
Pr ¼ where a ¼ ; ð7Þ
a qC p
k is the thermal conductivity and Cp is the air’s specific heat transfer
capacity. The subscript T indicates the turbulent flow and SQ is the
source term of the energy equation, described below.
2.2. Server rack and back door cooler modelling
The simplified data centre configuration considered is shown in
Fig. 1. It consists of two rows of three server racks where cold air is
supplied from six floor vents into a cold aisle, passes the rack
mounted IT equipment (servers), absorbs the heat generated by

Fig. 1. Plan view of three-aisle, six-rack data centre.

152 A. Almoli et al. / Applied Energy 89 (2012) 150–155
them, and the resultant hot air flows into a hot aisle. Many data
centres recirculate the hot air that flows into the hot aisle through
CRAC units that cool the air before it flows back out into the cold
aisle through the supply vents. In this paper we investigate cases
where liquid loop heat exchanger units are attached to the back
of the servers so that the hot air is cooled before it flows into the
‘hot’ aisle.
Previous CFD studies of data centre air flows that have appeared
in the literature have provided very little explanation of the way
the flow through server racks are modelled. These omissions make
it very difficult to carry out detailed comparison with previous CFD
studies.
The modelling strategy adopted here is summarised in Fig. 2.
The cold aisle is represented by region B, where the inlet air veloc-
ity and temperature is specified at each of the supply vents and the
governing RANS, continuity, energy, k and e equations are solved
simultaneously. The region to the left of region B is composed of
a server rack (rack 1), back door cooler (back door cooler 1) and
a hot aisle (region A). In practise, rack 1 forms a complex geomet-
rical obstacle to the air passing through it and the processors act as
sources of thermal energy into the air stream [5]. Previous CFD
studies of flow through the servers have ignored the small-scale
geometric features and modelled them in a highly averaged sense
[6]. In the present study we treat the rack as a porous medium by
Fig. 2. The solution procedure of the governing equations and the boundary condition.
defining the following source terms in the momentum and energy
equations respectively:
Q rack
S ¼ KU and SQ ¼ ð8Þ
V rack
where K ij ¼  alij and aij are elements of the permeability tensor. The
coordinate system and hence the momentum source terms are
aligned with the principal axes of the porous medium and therefore
the elements in Kij are zero apart from the diagonal ones, K11, K22
and K33 (see [8]). Note that the momentum source term assumed
here is linear; however more complex nonlinear models for porous
media flows [9] can be imposed if necessary.
In Eq. (8) a11 is the permeability of the rack in the direction of
flow (front to back), Qrack is the rate of heat generation inside the
rack and Vrack is the volume of the rack. The other two non-zero
terms, a22 and a33 are both set to 1020 m2 to ensure that flow is
primarily through the servers from front to back. In practise, a11
can be estimated experimentally by measuring the pressure drop
across the rack for a range of flow rates, but its value would be
dependent on the type of IT equipment. The rate of energy gener-
ation by the IT equipment can be estimated at a particular server
load from manufacturers’ specifications. Results presented here
have used an estimated a11 = 1.8  105 m2, which is calculated
by equating the pressure drop in a Hagen–Poiseuille flow in a pipe
 A. Almoli et al. / Applied Energy 89 (2012) 150–155 153

Table 1 The hot air emerging from the rear of rack 1 passes into an air
Heat load per rack and the source term. gap within which the same RANS, continuity, energy, k and e equa-
Heat load per rack (kW) Source term per rack (W/m3) tions as in region B, are solved. The air then flows into the back
15 7102.2 door cooler 1, which constitutes a further obstruction to the air
25 11,837 flow and which is again modelled by an additional porous medium
30 14204.5 by adding the source term in Eq. (8). In the illustrative results pre-
sented here, the permeability values, aij, for the back door cooler
are set to the same values as for the racks.
The energy removing properties of the back door cooler are
Table 2
Variation in rack intake temperature as a function of number of mesh cells.
modelled by treating it as a radiator which removes energy from
the air stream at a rate given by
Number of Cells Intake temperature of the rack (K)
68,788 294.1
78,336 294.5 Q_ bdc ¼ mhðjUjÞAðT
_ ref  T air Þ
87,884 295.0
169,442 295.0
417,358 295.0 where m _ is the mass flow rate of air across the back door cooler,
hðjUjÞ and A are its average convective heat transfer coefficient at
airspeed jUj and cross-sectional surface area respectively, Tair is
whose diameter is equal to the hydraulic diameter of a typical 1U the temperature of the air stream and Tref is the set point tempera-
server. The heat load and the source term for each case are shown ture of the back door cooler. In practise the heat transfer coefficient,
below in Table 1. h, for the back door cooler will be a function of air speed through
Each server rack is split into four sections representing a cluster the cooler. The required empirical relationship of the form hðjUjÞ
of rack servers or a blade cluster of approximately 10U. Since the can be determined by using experimental measurements of the dif-
air flow through the rack is aided by server fans at its rear (indi- ference between air on and air off temperatures as a function of air
cated by the left orange line in Fig. 2) this is modelled by a pressure speed. Following Tang et al’s [11] results for a plane fin and tube
drop across the rear rack boundary, which is given as a cubic poly- heat exchanger, in all calculations presented here, the value
nomial in terms of the speed of air flow normal (given by vector n ^) h ¼ 900jUj0:71 is used based on a best-fit against experimental data
to the boundary and is based on a Comair Rotron fan [10], for a passive back door cooler.
Back door cooler 1 is an example of a passive back door cooler
^ and
V ¼U n since the air flow through it is not driven by mechanical means,
DP ¼ 948:4  131:39V þ 16:2V 2  1:154V 3 ðfor the three lower server clustersÞ but relies on the pressure generated by the fans of the IT equip-
DP ¼ 1481:87  164:23V þ 16:2V 2  0:923V 3 ðfor the upper server clusterÞ ment. Back door cooler 2, however, is an example of an active back
door cooler which has additional fans at its rear to increase the
The above fan curve expressions can be adjusted based on fan speed of air flow through the back door cooler. Although this
speed using a relationship between
 2 the fan speed and the static clearly requires additional energy to power the fans, it can be seen
pressure that is given as PP12 ¼ NN12 , where N1 and N2 are the first from the relationship between h and |U| that the increased air flow
and second fan speeds (RPM) respectively and P1 and P2 are the will lead to a higher average heat transfer coefficient, and hence
associated static pressures (Pa) at these fan speeds respectively. higher heat extraction rate, from the air stream.

Fig. 3. Velocity profile (left), colour indicates airflow magnitude with yellow being the highest speed; and temperature profile (right), blue for cool and red for hot for the
active back door cooler at 30 kW. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Table 3
_ CRAC and m
Effect of using back door coolers on the cooling load of CRAC units with six 30 kW racks, where m _ BD are the mass flow rate through the
CRAC and back door respectively.

Configuration Heat load on CRAC Heat load on back door

CRAC 180 kW 0 kW
CRAC + passive m_ CRAC C p ðT OUT  T IN Þ ¼ 14:5 kW m_ BD C p ðT B4  T AFTER Þ ¼ 164:8 kW
CRAC + active m_ CRAC C p ðT OUT  T IN Þ ¼ 11:5 kW m_ BD C p ðT B4  T AFTER Þ ¼ 170:8 kW
154 A. Almoli et al. / Applied Energy 89 (2012) 150–155
Table 4
The intake and exhaust temperatures (T) for both active (black curves) and passive (red curves) back door coolers for different heat loads (kW). (For interpretation of the
references to colour in this figure legend, the reader is referred to the web version of this article.)
Heat dissipation load (kW) Rack intake temperature
15 kW
25 kW
30 kW
3. Results and discussion
The CFD model for the server rack in a simplified data centre
configuration as described above is solved using the commercial
CFD package, ANSYS FLUENT 12.0.1. Within the CFD mesh, surfaces
and volumes are represented using quadrilateral and hexahedral
elements respectively. Results were obtained on five different grid
sizes using between 68,788 and 417,358 cells as highlighted in Ta-
ble 2. The table demonstrates that the results are mesh indepen-
dent when using a mesh with 87,884 cells as it can be seen that
for larger mesh sizes there is no variation in the rack intake tem-
perature. Steady state solutions were obtained using the pressure
based method for solving the governing equations using the de-
fault relaxation factors. A brief analysis of the effectiveness of both
the active and passive back door coolers is presented in Table 3.
3.1. Air flow within the data centre
Air enters the data centre via the floor vents where an inlet tem-
perature, TIN, is maintained by the CRAC unit supplying cool air to
the under floor plenum. The cool air rises in the cold aisle, which is
in front of the server racks and is then drawn through the server
Rack exhaust temperature
racks, where it is heated by the IT loads within the servers and then
passes into the hot aisles at the back of the two rows of servers. Fi-
nally the hot air in the hot aisles is extracted via the outlet vents in
the roof of the simplified data centre. Fig. 3 (left) shows the air flow
via colour coded velocity vectors and it is possible to see that the
air speeds are greatest in front of the servers where the air is pro-
pelled into the room through the flow tiles, and in the case of active
back doors, the air speeds to the rear of the server racks is in-
creased by the additional fans in the back door cooler. Fig. 3 (right)
shows the corresponding temperature distribution.
3.2. Comparison of passive and active back door coolers
Table 3 shows an example of how the use a back door cooler can
reduce the cooling load for a CRAC unit. The data shown is for a
case in which each rack adds 30 kW of thermal power into the
air streams. When CRAC units are used without the assistance of
a back door cooler, all of the heat transferred into the air stream
has to be removed by the CRAC unit, namely 180 kW (six racks
at 30 kW each).
When the passive back door coolers are used, these remove
approximately 165 kW from the air streams with the result that
 A. Almoli et al. / Applied Energy 89 (2012) 150–155
the load on the CRAC units is reduced to only 14.5 kW. Finally,
when an active back door cooler is used, the higher heat transfer
coefficient that results from higher air speeds through it enables
the load on the CRAC units to be reduced further to only
11.5 kW. Clearly the precise values of these benefits will depend
critically on the back door cooler specifications. The cost benefit
analysis of using liquid cooling via the back door scenarios will
need to take account of the need to reduce the temperature of
the water before it returns to the rear of the rack, but there are
benefits in using liquid in that its specific heat capacity is much
greater than air and the temperature ranges could make use of free
cooling in certain geographical locations.
Table 4 shows CFD predictions of the rack intake and exhaust
temperatures for the passive and active back door cooler specifica-
tions described earlier, for heat dissipation rates of 15, 25 and
30 kW per rack. As expected, the exhaust air temperatures leaving
the active back door cooler are lower than those associated with
the passive cooler. For the parameters used here, the exhaust tem-
peratures with the active back door cooler are typically between 1
and 2 °C less than for the passive case with a heat dissipation load
per rack of 30 kW.
4. Conclusions
Relying solely on air as the primary heat transfer medium in data
centres is becoming increasingly problematical due to inexorable
increases in power densities from IT equipment. Although CFD is
increasingly being used to improve air flow design in data centres,
few details of the means by which the all-important flow through
server racks is modelled have appeared in the literature to date. This
is an important omission since it makes comparison and validation
with other studies difficult. A new CFD methodology has been devel-
oped for simulating data centre scenarios where a liquid loop heat
155
exchanger is attached at the rear of the server racks. This enables
the potential benefits of liquid back door cooling versus traditional
air cooling in terms of reduced load on the CRAC units to be assessed
against the need to cool the recirculating water, which could in prin-
ciple be achieved by free cooling in cold climates. The additional
benefits of using active versus passive back door cooling can also
be assessed by comparing the further reduced load on the CRAC
units against the energy needed by the active back door cooler’s
fans. Detailed validation of the CFD methodology is required before
it can be used in practical data centre design scenarios.
References
[1] Cho J, Lim T, Sean Kim B. Measurements and predictions of the air distribution
systems in high compute density (Internet) data centers. Energy Build
2009;41:1107–15.
[2] Cushing R, Doherty J. Next generation data centres. MSC Business
Development Ltd.; 2009.
[3] Cader T, Westra L, Sorel V, Marquez A. Liquid cooling in data centers. ASHRAE
Trans 2009;115(pt 1):231–41.
[4] Anandan SS, Ramalingam V. Thermal management of electronics: a review of
literature. Therm Sci 2008;12(2):5–26.
[5] Choi J, Kim Y, Srebric ASivasubramaniamJ, Wang Q, Lee J. A CDF-based tool for
studying temperature in rack-mounted servers. IEEE Transactions on
Computers 2008;57(8):1129–42.
[6] Shrivastava SK, Iyengar M, Sammakia BG, Schmidt R, VanGilder JW.
Experimental-numerical comparison for a high-density data center: hot spot
heat fluxes in excess of 500W/ft2. IEEE Trans Compon Packag Technol
2009;32(1):166–72.
[7] Boulet M, Marcos B, Dostie M, Moresoli C. CFD modeling of heat transfer and
flow field in a bakery pilot oven. J. Food Eng 2010;97(3):393–402.
[8] Liakopoulos AC. Darcy’s coefficients of permeability as symmetric tensor of
second rank. J Hydrol Sci 1965;10(3):41–8.
[9] Bear J. Dynamics of fluids in porous media. Dover Books; 1988.
[10] Comair rotron company. Fan specification based on Tarzan AC – TNE2A –
020174 (http://www.comairrotron.com).
[11] Tang LH, Zeng M, Wang QW. Experimental and numerical investigation on air-
side performance of fin-and-tube heat exchangers with various fin patterns.
Exp Therm Fluid Sci 2009;33:818–27.