Trading TCO for PUE?

Prepared for our customers by Romonet Ltd

Version: 1.0
Document Status: Final
Confidentiality: Non-confidential
Author: Liam Newcombe
1 Document information
1.2 Version History
Version Description Version Updates Date
Collated and expanded from initial blogs and edited 7/July/2013
0.1 Draft
to White Paper format

1.3 Release History

Version Description Authoriser Date

1.0 Final Liam Newcombe 10/July/2013

Rom onet and the 4 circles logo m ark are trad em arks of Rom onet Ltd .
Other brand s and trad em arks are acknow led ged . Patents Pend ing. Cop yright Rom onet 2013. All rights reserved .

Rom onet Ltd T. UK 0844 257 1992
Corinthian H ou se T. Int +44 (0)20 8256 0250
17 Lansd ow ne Road F. + 44 (0) 20 8626 7053
Croyd on, Su rrey, CR0 2BX UK email info@rom onet.com
w w w .rom onet.com
Rom onet Inc. T:+1 (415) 658 5763
2121 N . California Blvd .
Su ite 290
Walnu t Creek, CA 94596, USA
 Rom onet Ltd Trad ing TCO for PUE 3 of 16
DC-FVER
Liam N ew combe

2 Contents
3 Summary .............................................................................................................................................................. 4
4 The example data center..................................................................................................................................... 5
4.1 Free cooling hours comparison ......................................................................................................................... 6
4.2 PUE comparison .................................................................................................................................................. 7
4.3 Water consumption ............................................................................................................................................. 8
4.4 TCO over 10 years ............................................................................................................................................... 9
5 What about investing in solar instead? .......................................................................................................... 10
5.1 Financial assessment of Solar PV .................................................................................................................... 11
6 What about water consumption? .................................................................................................................... 12
6.1 Comparing overall energy and water consumption .................................................................................... 14
7 Conclusions ........................................................................................................................................................ 15
 Rom onet Ltd Trad ing TCO for PUE 4 of 16
DC-FVER
Liam N ew combe

3 Summary
There is no shortage of d ata center p rod u cts w hich claim to offer hu ge red u ctions in PUE and an
associated large ROI. The problem is that, in m any cases, chasing greater and greater PUE red u ctions is
the w rong thing to d o, both financially and environm enta lly.

In this White Pap er, w e w ill exam ine one com m on d ata center d esign op tion that m any w ou ld exp ect to
not only resu lt in a low er PUE, bu t to also save a lot of energy and m oney. Su rp risingly, w hen a d etailed
analysis is carried ou t there is no financial or environm ental benefit d esp ite red u cing the PUE. In fact, for
a $500k initial investm ent the 10 year ROI evalu ates to a loss of rou ghly $400k, the reasons w hy the
ap p arently obviou s choice d oesnt w ork are d iscu ssed below .

The key take-aw ay from this sp ecific case shou ld not be that this typ e of cooling system is u nsu itable, as
this is not necessarily a general resu lt. What w e can learn how ever, is that assessm ent of TCO and ROI for
d ata center investm ent choices shou ld not be based on w eak p roxy ind icators su ch as free cooling
hou rs as this is likely to inclu d e errors w hich are larger than the estim ated savings. Effective analysis of
the financial as w ell as the engineering elem ents is the key to avoid ing costly m istakes.

To fu rther evalu ate the environm ental cred entials w e w ill also look at:

What hap p ens to the TCO if w e sp end the $500k on renew able energy from solar p anels instead ?
Does ou r d ata center u se m ore or less w ater overall w ith the ad iabatic op tion?

A sp ecific ou tcom e is that the p roliferation of w eb based and p re-sales tools w hich rely on w eak
ind icators, or w orse, d o not d escribe their m ethod ology, are su bject to su ch large errors that op erators
shou ld think very carefu lly before basing any d ecision on their ou tp u t.
 Rom onet Ltd Trad ing TCO for PUE 5 of 16
DC-FVER
Liam N ew combe

4 The example data center

The exam p le w e w ill u se is a new -bu ild 1.2MW IT load d ata center w ith d irect ou tsid e air econom isers
and tw o reasonable, bu t m od ern, IT equ ip m ent environm ental ranges. The d esign is a Tier III d ata center
u sing good qu ality com p onents w hich achieves a reasonable balance of cap ital cost and Total Cost of
Ow nership . The qu estion is w hether it is w orth ad d ing ad iabatic (evaporative cooling and
hu m id ification) sections to the Air H and ling Units (AH Us), w hich w ou ld allow the site to op erate on
ou tsid e air instead of m echanical cooling for m any ad d itional hou rs p er year.

The instinctive resp onse is that ad iabatic cooling shou ld show a hu ge energy efficiency
ad vantage in a d ry clim ate su ch as N ew Mexico as long as w e contain the relatively sm all w ater
cost.
The p roblem is that these ad iabatic com p onents are relatively exp ensive to p u rchase and
continu e to cost m oney to m aintain.

Ou r m od ern d esign site has tw o choices of su p p ly air control ranges to keep the IT equ ip m ent w ithin the
ASH RAE Class A1 range. The table below su m m arises the control bound aries for those w ho w ant to
know how the analysis w as configu red :

Parameter Non-Contained Air Flow Contained Air Flow

Max IT Inlet Temperature 21C / 70F 24C / 75F
Supply Air Temperature 14.5C / 58F 24C / 75F
Minimum Humidity 20% RH at 21C 20% RH at 24C
Maximum Humidity 17C dew point (~75% RH) 17C dew point (~65% RH)
Table 1 Env ironment al cont rol ranges

Ou r d ry d esign can run entirely free cooling only w hen the ou tsid e air is betw een the hu m id ity lim its
and below the target su p p ly tem p eratu re. The p roblem for the non -ad iabatic d esign is that the clim ate in
N ew Mexico is too cold and d ry in the w inter w ith too little hu m id ity in the ou tsid e air, or hot and d ry in
the su m m er, too hot to ru n free cooling. Und er these cond itions su rely it w ou ld m ake good sense to u se
ad iabatic u nits to hu m id ify the air w hen it is too d ry and to cool it for free w hen it is too hot?
 Rom onet Ltd Trad ing TCO for PUE 6 of 16
DC-FVER
Liam N ew combe

4.1 Free cooling hours comparison

It is common to see people try to assess the performance of a free cooling system by comparing how many free
cooling hours per year it achieves. Intuitively it seems that this would be a good indicator of the performance, more
free hours must be better after all.

The table and graph below shows the percentage of the typical year that each version will spend in full free cooling,
partial free cooling and full mechanical cooling:

Condition Normal Adiabatic 75F Normal 58F Adiabatic

75F 58F
Full free cooling (no compressor) 30% 96% 21% 69%
Partial free cooling (part compressor) 14% 2% 20% 15%
Full mechanical cooling 56% 2% 59% 16%
Table 2 Free cooling hours

Looking at the resu lts of the free cooling analysis for ou r site in the d ry N ew Mexico clim ate, it w ou ld
seem that the ad iabatic op tion is a no-brainer and m u st hu gely ou tp erform the stand ard ou tsid e air
system .
If w e w ere to try to base an op erational co st savings estim ate on this free cooling hou rs analysis w e
w ou ld need estim ates of:

Base p ow er cost $0.058 / kWh

IT kW load 1,000kW average
Cooling overhead fu d ge factor 0.25 * cooling load

If w e u se this ap p roach to d eterm ine the op erational cost savings then ou r ou tp u t m ight look like:

75F Normal 75F Adiabatic 58F Normal 58F Adiabatic

Mech cooling hours 6,090 380 6,929 2,747
Mech cooling kWh 1,522,500 95,000 1,732,250 686,750
Mech cooling $ $88,305 $5,510 $100,471 $39,832
Estimated annual saving $82,795 $60,639
Estimated 10 year saving $827,950 $606,390
Table 3 Flaw ed cost sav ings analysis

Desp ite ap p earing to be d erived u sing a reasonable m ethod , the savings estim ates above are com p letely
w rong. As w e w ill show in the next cou p le of sections, d u e to factors su ch as the interactions of clim ate,
varying m echanical com p ressor efficiency w ith external tem p eratu re and p art load ing, free cooling hou rs
 Rom onet Ltd Trad ing TCO for PUE 7 of 16
DC-FVER
Liam N ew combe

are not a u sefu l ind icator of energy or cost. This is d escribed in m ore d etail in ou r cooling analysis p ap er
d evelop ed for the EU Cod e of Cond u ct on Data Centers.

4.2 PUE comparison

If instead of sim p ly cou nting the free cooling hou rs w e carry ou t a m ore d etailed assessm ent of the
achieved p erform ance w e can get a better u nd erstand ing of w hat is going on. The ou tp u t below is from a
fu ll hou rly sim u lation of both norm al (red ) and ad iabatic (blu e) d esigns op erating at both su p p ly air
tem p eratu res w hich takes into accou nt cooling load , varying DX com p ressor p erform ance etc. (note that
for read ability the PUE axis starts at 1.0, not 0)

From the left hand chart, at 75F su p p ly, w ith ad iabatic cooling (blu e) w e can ru n alm ost the w hole year
w ithou t starting ou r m echanical com p ressors and they d o not ru n at fu ll load for m any hou rs at all
resu lting in the sm all sp ike in Ju ly - Sep tem ber. Withou t the ad iabatic cooling w e have m ore hou rs of fu ll
m echanical cooling, bu t these are p red om inantly in the w inter w hen the ou tsid e air is too d ry to su p p ly
to the d ata hall.

In the right hand charts, at 58F even w ith ad iabatic cooling the d ata center relies heavily on m echanical
cooling in the su m m er w here the ou tsid e air is too hot or too hu m id . The norm al, d ry cooling op tion at
58F on the other hand , has few d ays w here it d oes not u se the m echanical cooling, qu ite a few d ays of
m echanical cooling d u ring the d ay in the hot su m m er p lu s the d ays of fu ll m echanical cooling in the cold -
d ry w inter cond itions.
 Rom onet Ltd Trad ing TCO for PUE 8 of 16
DC-FVER
Liam N ew combe

4.3 Water consumption

Given the selection of N ew Mexico, a region w ith little w ater and therefore relatively high w ater p rices,
w e need to check the w ater consu m p tion in ad d ition to the energy p rofile to d eterm ine the environm ental
and financial p erform ance of ou r d esign op tions. The ad iabatic hu m id ifier sections u se w ater in tw o
op erating m od es for ou r site:

H u m id ification of ou tsid e air that is too d ry to m eet the m inim u m IT hu m id ity target
Cooling of ou tsid e air that is too hot to m eet the m axim u m IT su p p ly tem p eratu re target

The w ater consu m p tion is calcu lated by d eterm ining the w ater requ irem ent to m eet the change in air
m oistu re cond itions. A 50% overhead is then ad d ed to this to accou nt for w ater lost from the system d u e
to flu shing and other p rocesses; this is qu ite conservative bu t d ep end s greatly u p on the ind ivid u al u nits
in u se - anything u p to 200% overhead is realistic. With a w ater cost of $22 p er 1,000 Gallons as an
ind icative sam p le p rice, the w ater consu m p tion to achieve the m inim u m su p p ly hu m id ity and ad iabatic
cooling targets is show n below :

75F Adiabatic 58F Adiabatic

Humidification hours 4,657 3,376
Annual litres H20 1,551,000 696,000
Adiabatic cooling hours 1,226 2,122
Annual litres H20 493,000 357,000
Total annual H20 2,045,000 1,053,000
Annual H20 Including losses 3,067,000 1,580,000
Annual water cost $17,805 $9,170
Table 4 Annual w at er consumpt ion of adiabat ic humidificat ion and cooling
 Rom onet Ltd Trad ing TCO for PUE 9 of 16
DC-FVER
Liam N ew combe

4.4 TCO over 10 years

Given the relatively sm all w ater cost there is still som e hop e for a good TCO ou tcom e on the ad iabatic
op tion. To p erform the TCO analysis w e w ill:

Evalu ate both op tions over 10 years w ith a 7% (ad ju sted ) d iscou nt rate
Assu m e flat (real) w ater and p ow er costs over the p eriod
Ap p ly a $500,000 first cap ital cost d ifference to bu ild w ith the ad iabatic sections in the AH Us
based on actu al p rice d ata
Ap p ly a $10,000 p er annu m op erational and m aintenance cost for the cleaning and treatm ent of
the ad iabatic cooling cham bers
Allow the IT pow er d raw to rise over the first fou r years as 250kW, 500kW, 750kW and then
1,000kW for the rem aining years
$0.058 p er kWh average u tility energy cost

Pu tting these nu m bers into the TCO analysis w e can now p lot ou r TCO over 10 years to see w hat overall
saving w e are able to recover against the initial cap ital investm ent from red u ced op erational cost:

Unfortunately, as the chart show s, the sm all red u ction in op erational energy cost sim p ly cannot offset the
initial cap ital cost. Desp ite d eliberately selecting a clim ate to favou r the engineering d ecision there is still
no financial argu m ent to su p p ort the ad d itional cap ital exp end itu re.

Capital Cost 10 Year PV Saving

75F Adiabatic $500,000 -$415,000
58F Adiabatic $500,000 -$421,000
Table 5 Real 10 y ear TCO of adiabat ic opt ion
 Rom onet Ltd Trad ing TCO for PUE 10 of 16
DC-FVER
Liam N ew combe

5 What about investing in solar instead?

H aving d iscou nted the u se of ad iabatic coolers w hat w ou ld be the im p act if, having shaved $500k off ou r
cap ital bu d get, w e sp ent it on solar p hotovoltaic renew able energy instead ?

To assess this w e need som e d ata abou t the solar p anels and how m u ch su nlight w e can exp ect throu gh
the year.

Solar p anel installations are frequ ently qu oted as $ p er p eak kW, w hat this m eans is that ou r
cap ital cost for installation gives u s a p eak kW cap acity w hen the s u n is overhead on a clear d ay.
For this analysis I have u sed a cost of $2,000 p er kW p eak (kWP) 1, u sing ou r fu ll $500k this
allow s for a p eak solar cap acity of 250kW.
We w ont get 250kW for m u ch of the tim e, certainly not overnight w hen it is d ark, w e w ill get
less in the m orning and evening and less w hen it is clou d y (w hich it isnt very often in N ew
Mexico). Fortu nately the sam e clim ate d ata w e u sed for the cooling p erform ance analysis carries
a typ ical year of solar rad iation w hich w e can u se for a d ecent ap p roxim ation of the annu al
energy ou tp u t of the PV farm .

As before, w e are consid ering both th e 58F and 75F (contained airflow ) su p p ly tem p eratu re control
op tions for the d ata center, each of w hich is consid ered in d ry and ad iabatic ou tsid e air variants. The
chart below show s the d aily total kWh d em and from ou r p reviou s analysis for each of the fou r op tions. In
ad d ition the available d aily energy ou tp u t of the 250kWP solar p anel farm is show n to allow for
com p arison, in the best case the contribu tion is ~ 4.5%.

Figure 1 Cont ribut ion of 250kW P solar PV farm t o t ot al daily demand

There is a com m on saying in the UK that som ething is abou t as m u ch u se as a chocolate teapot (d u e to
the obviou s therm al stability p roblem of chocolate to hot tea); in this case w e m ight consid er u p d ating it
to abou t as m u ch u se as a solar p anel on a d ata center.

1Based on continued oversupply of solar PV to the market from China and resulting falling prices, see
http://www.forbes.com/sites/peterdetwiler/2012/12/11/solars-steady-march/ for a US market summary
 Rom onet Ltd Trad ing TCO for PUE 11 of 16
DC-FVER
Liam N ew combe

5.1 Financial assessment of Solar PV

Whilst the contribu tion to the overall energy consu m p tion of the d ata center is sm all w e shou ld also
assess the solar PV for TCO contribu tion as it d oes d irectly red u ce electricity costs to the site. To d o this
w e m u st m ake som e assu m p tions abou t how the cost of electricity is going to change over the tim e w e
op erate ou r investm ent. Most p red ictions for the US show p ow er cost staying low for som e tim e d u e to
the influ ence of shale gas and other m arket forces, how ever environm ental or energy secu rity p ressu res
m ay cau se the valu e of solar generated pow er to increase over this tim e fram e.

One of the od d ities for this investm ent is that in m any cases the op erator w ou ld be better off locating the
solar p anels in an area w ith a feed -in tariff stru ctu re and selling the p rod u ced p ow er to the grid as this
w ou ld yield a greater incom e than the savings on energy achieved w ith the p anels collocated w ith the
d ata center and no available su rp lu s to feed back.

To get a sim p le estim ate of the valu e w e keep energy costs flat and the sam e 7% d iscou nt rate w e u sed in
the original TCO assessm ent. Looking ou t over a fu ll 20 years (investm ents in energy generation are not
short term ):

Year Annual PV kWh Cost per kWh Annual Saving PV NPV

1 472,490 $0.058 $27,704 -$472,596
2 472,490 $0.058 $25,612 -$446,984
3 472,490 $0.058 $23,936 -$423,048
4 472,490 $0.058 $22,370 -$400,678
5 472,490 $0.058 $20,907 -$379,771
10 472,490 $0.058 $14,906 -$294,049
15 472,490 $0.058 $10,628 -$232,931
20 472,490 $0.058 $7,578 -$189,354
Table 6 - Solar PV Inv estment at 0% pow er cost increase

As show n in the table, the ROI is better than the ad iabatic coolers bu t still not p ositive. To break even at
the 20 year p oint w e w ou ld need the valu e of the pow er generated by the solar p anels to increase,
sp ecifically an annu al increase in valu e of 0.9% less than ou r investm ent d iscou nt rate, or 6.9%. This is
p ossible w ith legislative ch anges su ch as feed -in tariffs or a governm ent ap p lied cost of carbon bu t it is
qu estionable w hether this investm ent analysis shou ld be p art of the d ata center or a sep arate
u nd ertaking.
 Rom onet Ltd Trad ing TCO for PUE 12 of 16
DC-FVER
Liam N ew combe

6 What about water consumption?

An area of grow ing concern for d ata centers is w ater consu m p tion; this has a d ire ct (and likely to rise)
cost im p act w hich w as d ealt w ith in the TCO analysis. The second ary im p act is the associated
environm ental im p act of w ater consu m p tion, p articu larly in areas of increasing w ater scarcity w hich
inclu d es m any p arts of the USA. To p rop erly assess the u tility of u sing w ater for free cooling of the
d ata center w e shou ld consid er w hether there is a net increase or red u ction in the w ater consu m p tion
associated w ith the d ata center.

This analysis is not as sim p le as it seem s, as w e cannot u sefu lly consid er the w ater consu m p tion of the
d ata center w ithou t inclu d ing the w ater u sed in the energy su p p ly. Sp ecifically, the WUE often qu oted in
m arketing and green d ashboard s is of little relevance w ithou t u nd erstand ing the w ater consu m p tion
involved in the electricity su p p ly to the d ata center. This is inclu d ed in the WUEsource variant of the m etric
w hich is p rovid es a m echanism for u s to com pare the overall w ater im p act. Obviou sly ou r d ata center
u ses m ore w ater than a d ry cooled d ata center bu t su bstantially less than a d ata center w hich u ses c ooling
tow ers. The qu estion is - d oes it u se less overall?

We have an analysis of the w ater requ irem ent of the ad iabatic system along w ith an estim ate that 1.5
tim es this m u ch w ater is actu ally u sed from the su p p ly. The table in the Green Grid WUE 2 p ap er gives an
Energy Water Intensity Factor (EWIF) of 2.38 litres p er kWh for N ew Mexico. Ap plying this to the d aily
energy consu m p tion w e can d erive the total w ater consu m p tion and WUE source for the fou r d ata center
op tions, this is show n in Figu re 2 below :

Figure 2 Daily W UE (Source) for all four opt ions

It is clear from Figu re 2 that there is no su bstantial d ifference betw een the op tions; ind eed the ord er of
p reference varies from su m m er to w inter as w ater consu m p tion in the ad iabatic hu m id ifiers is trad ed
w ith w ater consu m p tion at the p ow er station for m echanical cooling.

2 http://www.thegreengrid.org/~/media/WhitePapers/WUE
 Rom onet Ltd Trad ing TCO for PUE 13 of 16
DC-FVER
Liam N ew combe

Figure 3 Annual t ot al lit res of w ater - dat a center and energy supply

Figu re 3 show s how this ad d s u p over a typ ical year, and w e can see that the w ater consu m p tion of the
ad iabatic op tions is higher overall. This is p erhap s not su rp rising as it follow s from the sam e reason that
the ad iabatic cooling d oesnt save m oney. The DX m echanical cooling p rovid es anything u p to 7kW of
cooling for every 1kW of p ow er consu m ed , this m eans that the ad iaba tic cooling on the d ata center
w ou ld need to be 3.5 tim es 3 m ore efficient than the cooling tow ers at the p ow er station to even keep u p .

This ou tcom e u nd erlines the necessity to com p are like w ith like and u se realistic nu m bers w hen
evalu ating op tions for a d ata center. We have been generou s to the ad iabatic p lant, in m any cases the
w ater treatm ent and losses cou ld be 2x or m ore th e actu al ad iabatic requ irem ent. This analysis has u sed
an estim ate of 1.5 and still the ad iabatic op tion cam e ou t w orse. Other locations w ill have d ifferent w ater
intensities of p ow er generation, m any low er. It is also not u ncom m on to see the case stu d y for this typ e of
d evice m ad e by com p aring the p rop osed system w ith an u nrealistically p oor baseline u sing u nnecessarily
restrictive controls and neglecting the su bstantial im p rovem ents in m echanical cooling efficiency
d elivered in recent years.

3Thermoelectric generation based on steam turbines is generally ~ 33% efficient producing electricity rejecting the
remaining 67% source energy as heat, so the power station must reject twice as much heat energy as it supplies in
electricity to power our DX chillers.
 Rom onet Ltd Trad ing TCO for PUE 14 of 16
DC-FVER
Liam N ew combe

6.1 Comparing overall energy and water consumption

Unfortunately there is still no u sefu l m ethod of com p aring w ater w ith sou rce energy in m ost cases as the
relationship betw een w ater and sou rce energy is both com p lex and location d ep end ent. In an area su ch as
the Pacific N orth West USA or N orthern Eu rop e, w ater is easily available at very low energy and
environm ental im p act, w hile in the Sou th Western USA w ater availability is becom ing a seriou s p roblem .
This m eans that w e cannot sensibly try to com p are the WUE of d ata centers u nless they have the sam e
PUE as w e d o not know how m u ch WUE it w ou ld be w orth trad ing for a change of, say 0.1 in the PUE .
 Rom onet Ltd Trad ing TCO for PUE 15 of 16
DC-FVER
Liam N ew combe

7 Conclusions
Whilst ru nning the analyses and com p iling the ou tp u t for this White Pap er m any of the resu lts have
su rp rised u s both in their d irection and m agnitu d e. H op efu lly these exam p les have p rovid ed a good
exam p le of w hy sim p le ru les of thu m b or free cooling hou rs estim ates are not u sefu l for assessing d ata
center p erform ance and that in m any cases the real behaviors can be qu ite cou nter-intu itive.

In other w ord s, before com m itting you rself to a m ajor investm ent in a d ata center, no m atter how
straightforw ard the benefit it w ill p rovid e ap p ears to be, it is alw ays w orth closely m od eling how it w ill
im p act you r TCO.
 Rom onet Ltd Trad ing TCO for PUE 16 of 16
DC-FVER
Liam N ew combe

Rom onet and the 4 circles logo m ark are trad em arks of Rom onet Ltd .
Other brand s and trad em arks are acknow led ged . Patents Pend ing. Cop yright Rom onet 2013. All rights reserved .

Rom onet Ltd T. UK 0844 257 1992
Corinthian H ou se T. Int +44 (0)20 8256 0250
17 Lansd ow ne Road F. + 44 (0) 20 8626 7053
Croyd on, Su rrey, CR0 2BX UK email info@rom onet.com
w w w .rom onet.com
Rom onet Inc. T:+1 (415) 658 5763
2121 N . California Blvd .
Su ite 290
Walnu t Creek, CA 94596, USA