Accredited Tier Designer

http://uptimeinstitute.
com
©~zo~1s~u-pu_m_el-nst-itu-te-.L-Lc----=====~·uptime Instituter----==~~::::=:::===:::::;:;;;::=:::::=====~
Copyright ©2009-2015 by Uptime Institute, LLC
20 West 37th Street - 6th Floor
New York, NY 10018
All rights reserved.

The Uptime Institute's (Institute) Publications are protected by international
copyright law. The Institute requires written requests at each and every occasion that
the Institute's intellectual property or portions of the Institute's intellectual property
are reproduced or used. The Institute copyright extends to all media-paper,
electronic, and video content-and includes use in other publications, internal
company distribution, company Web sites and marketing materials, and handouts for
seminars and courses. For more information, please visit
www.uptimeinstitute.com/resources to download a Copyright Reprint Permission
Request Form.
Uptime Institute 20 West 37u' Street, 61h Floor

New York. NY 10018
+1 206.706.4149
http://uptimeinstitute.com
Uptin1e Institute
accredited tier designerT..
Table of Contents
Tier Standards and Technical Papers ............................................ Tab 1
Course Schedule ........................................................................... Tab 2
Session 1 Notes ............................................................................. Tab 3
Session 3 Notes .............................................................................. Tab 5
Session 5 Notes .............................................................................. Tab 7
Session 8 Notes ............................................................................ Tab 10
Abbreviations ............................................................................... Tab 11
Uptime Institute Contacts ........................................................... Tab 12

'
Uptime Institute''
Uptime Institute~
Abstract
The Uptime Institute Tier Standard: Topology is an objective basis for comparing the functionality, capacity,
and expected availability (or performance) of a particular site infrastructure design topology against other sites,
or for comparing a group of sites. This Standard describes criteria to differentiate four classifications of site
infrastructure topology based on increasing levels of redundant capacity components and distribution paths.
This Standard focuses on the definitions of the four Tiers and the performance confirmation tests for determining
compliance to the definitions . The Commentary, in a separate section, provides practical examples of site
infrastructure system designs and configurations that fulfill the Tier definitions as a means to clarify the Tier
classification criteria.
Keywords
ambient temperatures, autonomous response, availability, classification, Compartmentalization, Concurrent
Maintenance, Concurrently Maintainable, Continuous Cooling, data center, dry bulb, dual power, Fault Tolerance,
Fault Tolerant, functionality, infrastructure, metrics, Operational Sustainability, performance, redundant, reliability,
Tier, Tier level, Tiers, topology, wet bulb
Copyrights
This document is copyrighted by the Uptime Institute, LLC . Uptime Institute-in making this document available
as a reference to governmental agencies, public institutions, and private users-does not waive any rights in
copyright to this document.
Uptime Institute's Publications are protected by international copyright law. Uptime Institute requires written
requests at each and every occasion that Uptime Institute's intellectual property or portions of the intellectual
property are reproduced or used. Uptime Institute copyright extends to all media- paper, electronic, and video
content-and includes use in other publications, internal company distribution, company Web sites and marketing
materials, and handouts for seminars and courses.
For more information, please visit www.uptimeinstitute.com/publications to download a Copyright Reprint

Permission Request Form.
Uptimelnstitutec'
Introduction
This introduction is not part of Uptime Institute Data Center Site Infrastructure Tier Standard: Topology. It provides
the reader with context for the application of the Standard.
This Institute Data Center Site Infrastructure Tier Standard: Topology is a restatement of the content previously
published as Uptime Institute publication Tier Classifications Define Site Infrastructure Performance. Selected
content of this publication has been reedited into an ANSI Standards Model format. Future updates or changes to
Uptime Institute Tier Standard: Topology shall be accomplished through a review and recommendation process
consistent with other recognized Standards bodies.
The Tier Classifications were created to consistently describe the site-level infrastructure required to sustain data
center operations, not the characteristics of individual systems or subsystems. Data centers are dependent
upon the successful and integrated operation of electrical, mechanical, and building systems. Every
subsystem and system must be consistently deployed with the same site uptime objective to satisfy the
distinctive Tier requirements. The most critical decision-making perspective owners and designers must consider,
when making inevitable tradeoffs, is what effect does the decision have on the life-cycle-integrated operation of
the Information Technology (IT) environment in the computer room.
Simply put, the Tier topology rating for an entire site is constrained by the rating of the weakest subsystem that
will impact site operation. For example, a site with a robust Tier IV UPS configuration combined with a Tier II
chilled water system yields a Tier II site rating.
This very stringent definition is driven by senior executives who have approved multi-million dollar investments for
an objective report of actual site capabilities. Any exceptions and exclusions footnoted in the approval documents
will be quickly lost and forgotten. If a site has been advertised within an organization as being Fault Tolerant
(Tier IV), it will be inconsistent to have to plan a site shutdown at any time in the future-regardless of any "fine
print" exclusions that diligently identified the risk. For this reason, there are no partial or fractional Tier ratings.
A site's Tier rating is not the average of the ratings for the critical site infrastructure subsystems. The site's Tier
rating is the lowest of the individual subsystem ratings.
Similarly, the Tier rating cannot be claimed by using calculated mean time between failures (MTBF)
component statistical reliability to generate a predictive availability and then using that number to match the
empirical availability results with those of sites representing the different Tier classifications. Statistically valid
component values are not available, partly because product life cycles are getting shorter and no independent,
industry-wide database exists to collect failure data.
Finally, this Standard focuses on the topology and performance of an individual site. High levels of end-user
availability may be attained through the integration of complex IT architectures and network configurations that
take advantage of synchronous applications running on multiple sites. However, this Standard is independent of
the IT systems operating within the site.
2
Uptime Institute''
Additional Factors and Exposures

Uptime Institute Tier Standard: Topology and Tier Standard: Operational Sustainability establish a consistent
set of performance criteria that can be satisfied, and adjudicated, worldwide. For the data center design,
implementation, and sustained operation to be successful, additional factors and exposures must also be
considered by the owner and project team. Many of these will be dictated by the site location as well as local,
national, or regional considerations and/or regulations. For example, building codes and Authorities Having
Jurisdiction (AHJs) ; seismic, extreme weather (high winds, tornado) ; flooding ; adjacent property uses ; union
or other organized labor force; and/or physical security (either as corporate policy or warranted by immediate
surroundings).
Due to the many design and management options that may be dictated by the owner, regulated by local
government, recommended by industry groups, or followed as a general practice, it is not feasible for Tier
Standard: Topology and Tier Standard: Operational Sustainability to establish criteria for these additional factors
and exposures worldwide. And, the Uptime Institute does not wish to displace or confuse the guidance of local
experts, which are key for timely project delivery, regulatory compliance, and implementation of best practices.
For a successful project, Uptime Institute recommends that the project team create a comprehensive catalogue of
project requirements, which incorporates Tier Standard: Topology, Tier Standard: Operational Sustainability, and
carefully considered mitigation measures of these additional factors and exposures. This approach will ensure the
project meets the compliance objectives of Uptime Institute's international standards, as well as local constraints
and owner's business case.
3
Uptime Institute~
Contents
1.0 Overview .............. ... .. ................. .... .......... .. ..... ... ............ .... ... ........ ......... ..... ................ .......... ........ 5
1.1 Scope ... ..... ...... ......... .. .... .......... ...... ..... ......... ... ..................... ....... ...... ..... ... .... .... ...................... 5
1.2 Purpose ....... ........ .. .. ...... .. ................. ... .. .. .. ....... ...... ...... ..... .. ................. .. ......... ..... ... ......... ....... 5
1.3 References ........................................ ... .... ..... ..... ........... .. ..... ........ .. .. ... .. ... ..... .. .... .. ...... .... ........ 5
1.4 Related Publications .. .. .. .. .. .... ........................... .... ................ ...... .. ... .. .. ... ................ .. .. .. ....... ..... 5
2.0 Tier Classification Definitions ......... .. ......... .... ... ......... ..... ..... ....... .... ............ .......... ... ..... ........... ..... 5
2.1 Tier I - Basic Data Center Site Infrastructure ........ .. ........ ... ....... .. .......... .......... ......................... 5
2.2 Tier II - Redundant Site Infrastructure Capacity Components .. ........ .. .. ... ...... .. .. ........ .. ............ 6
2.3 Tier Ill - Concurrently Maintainable Site Infrastructure .. ..... ... .. ...... .... ......... ........ ........ ............ 6
2.4 Tier IV- Fault Tolerant Site Infrastructure ........................ .... .... ... ... ...... .. .. .. ... .. .. .. .. .. .... .. ........... 7
2.5 Engine-Generator Systems ....................... .. ... ... ......... .... .. ... ......... ..... ... ... ....... ..... ........ ............ 8
2.6 Ambient Temperature Design Points .............. .... ..... .. ......... .... ... .. .... .... .. ......... ..... .... .. ... ............ 8
2.7 Communications Routing ......................... ...................... .... ............................. .... ............. ....... 8
2.8 Makeup Water ................. .. .... ......... .................. ...................... ...... ...... ... ............... .. ..... ............ 9
2.9 Tier Requirements Summary .......... .... ....... ... ..... .. ... ............................. .. .. .. ............. ..... .... ..... .... 9
3.0 Commentary for Application of the Tier Standard: Topology ......... .... ........ ........ .. ....... .. ............ 9
3.1 Outcome-Based Tier Standard ... ............. .. ... ..... ...... .. .............. .... .. ... .... .... ........... ..... ... .. ........... 9
3.2 Impact of Ambient Design Conditions ... .... .. ............... .. ............ ...... ..... .... ..... ...... .......... ........... 9
3.3 Restrictions Against Engine-Generator Runtime Limitations (Tier Ill and Tier IV) ........ .......... 10
3.4 Tier Functionality Progression .... .. ........... ......... ............. .............. ........................................... 10
3.5 Fractional or Incremental Tier Classification .... ............ ......................................................... 11
3.6 Non-Compliance Trends .... .. ............................... ......... .......................................................... 12
Modifications ..... .......... ........ ...... ....................................... ............ ... ...... .. ...................... ............ ............ 12
4
Uptime Institute~
1. Overview
1.1 Scope
This Standard establishes four distinctive definitions of data center site infrastructure Tier classifications
(Tier I, Tier II, Tier Ill, Tier IV), and the performance confirmation tests for determining compliance to
the definitions. The Tier classifications describe the site-level infrastructure topology required to sustain
data <::enter operations, not the characteristics of individual systems or subsystems. This Standard is
predicated on the fact that data centers are dependent upon the successful and integrated operation of
several separate site infrastructure subsystems, the number of which is dependent upon the individual
technologies (e-g., power generation, refrigeration, uninterruptible power sources, etc.) selected to sustain
the operation.
Every subsystem and system integrated into the data center site infrastructure must be consistently
deployed with the same site uptime objective to satisfy the distinctive Tier requirements .
Compliance with the requirements of each Tier is measured by outcome-based confirmation tests and
operational impacts. This method of measurement differs from a prescriptive design approach or a
checklist of required equipment.
Commentary on this Standard is in a separate section that provides examples for the design and
configuration of facility systems for each Tier topology level. The commentary section also offers guidance
in the application and implementation of the Tier definitions. In addition, the commentary section includes
discussion and examples to aid in understanding Tier concepts as well as information on common design
topology shortfalls.
1.2 Purpose
The purpose of this Standard is to equip design professionals, data center operators, and non-technical
managers with an objective and effective means for identifying the anticipated performance of different
data center site infrastructure design topologies.
1.3 References
American Society of Heating, Refrigerating, and Air-Cond itioning Engineers, ASHRAE Handbook-
Fundamentals (Latest Version).
Uptime Institute Fault Tolerant Power Compliance Specification, Version 2.0.
1.4 Relationship between Tiers and Operational Sustainability

Accredited Tier Designer Technical Paper Series
Further information can be found at www.uptimeinstitute.com.
2. Site Infrastructure Tier Standards

2.1 Tier 1: Basic Site Infrastructure
2.1.1 The fundamental requirement:
a) A Tier I basic data center has non-redundant capacity components and a single,
non-redundant distribution path serving the critical environment. Tier I infrastructure includes:
a dedicated space for IT Systems; a UPS to filter power spikes, sags, and momentary
outages; dedicated cooling equipment; and an engine generator to protect IT functions from
extended power outages.
b) Twelve hours of on-site fuel storage for engine generator(s).
2.1.2 The performance confirmation tests:
a) There is sufficient capacity to meet the needs of the site.

b) Planned work will require most or all of the site infrastructure systems to be shut down
affecting critical environment, systems, and end users.
5
Uptime Institute''
2.1.3 The operational impacts:
a) The site is susceptible to disruption from both planned and unplanned activities. Operation
(Human) errors of site infrastructure components will cause a data center disruption.
b) An unplanned outage or failure of any capacity system, capacity component, or distribution
element will impact the critical environment.
c) The site infrastructure must be completely shut down on an annual basis to safely perform
necessary preventive maintenance and repair work. Urgent situations may require more
frequent shutdowns. Failure to regularly perform maintenance significantly increases the risk
of unplanned disruption as well as the severity of the consequential failure.
2.2 Tier II: Redundant Site Infrastructure Capacity Components

2.2.1 The fundamental requirement:
a) A Tier II data center has redundant capacity components and a single, non-redundant
distribution path serving the critical environment. The redundant components are extra engine
generators, UPS modules and energy storage, chillers, heat rejection equipment, pumps,
cooling units, and fuel tanks.
b) Twelve hours of on-site fuel storage for 'N' capacity.
a) Redundant capacity components can be removed from service on a planned basis without
causing any of the critical environment to be shut down.
b) Removing distribution paths from service for maintenance or other activity requires shutdown
of critical environment.
c) There is sufficient permanently installed capacity to meet the needs of the site when
redundant components are removed from service for any reason.
a) The site is susceptible to disruption from both planned activities and unplanned events.
Operation (Human) errors of site infrastructure components may cause a data
center disruption.
b) An unplanned capacity component failure may impact the critical environment. An unplanned
outage or failure of any capacity system or distribution element will impact the
critical environment.
c) The site infrastructure must be completely shut down on an annual basis to safely perform
preventive maintenance and repair work. Urgent situations may require more frequent
shutdowns. Failure to regularly perform maintenance significantly increases the risk of
unplanned disruption as well as the severity of the consequential failure.
2.3 Tier Ill: Concurrently Maintainable Site Infrastructure

2.3.1 The fundamental requirements:
a) A Concurrently Maintainable data center has redundant capacity components and multiple
independent distribution paths serving the critical environment. Only one distribution path is
required to serve the critical environment at any time.
b) All IT equipment is dual powered as defined by Uptime Institute's Fault Tolerant Power
Compliance Specification, Version 2.0 and installed properly to be compatible with the
topology of the site's architecture. Transfer devices, such as point-of-use switches, must be
incorporated for critical environment that does not meet this specification.
c) Twelve hours of on-site fuel storage for 'N' capacity.
a) Each and every capacity component and element in the distribution paths can be removed
from service on a planned basis without impacting any of the critical environment.
b) There is sufficient permanently installed capacity to meet the needs of the site when
redundant components are removed from service for any reason.
6
Uptime Institute'
a) The site is susceptible to disruption from unplanned activities. Operation errors of site
infrastructure components may cause a computer disruption.
b) An unplanned outage or failure of any capacity system will impact the critical environment.
c) An unplanned outage or failure of a capacity component or distribution element may impact
the critical environment.
d) Planned site infrastructure maintenance can be performed by using the redundant capacity
components and distribution paths to safely work on the remaining equipment.
e) During maintenance activities, the risk of disruption may be elevated. (This maintenance
condition does not defeat the Tier rating achieved in normal operations.)
2.4 Tier IV: Fault Tolerant Site Infrastructure

2.4.1 The fundamental requirements :
a) A Fault Tolerant data center has multiple, independent, physically isolated systems that
provide redundant capacity components and multiple, independent, diverse, active distribution
paths simultaneously serving the critical environment. The redundant capacity components
and diverse distribution paths shall be configured such that 'N' capacity is providing power and
cooling to the critical environment after any infrastructure failure.
b) All IT equipment is dual powered as defined by Uptime Institute's Fault Tolerant Power
Compliance Specification, Version 2.0 and installed properly to be compatible with the
topology of the site's architecture. Transfer devices, such as point-of-use switches, must be
incorporated for critical environment that does not meet this specification.
c) Complementary systems and distribution paths must be physically isolated from one another
(compartmentalized) to prevent any single event from simultaneously impacting both systems
or distribution paths.
d) Continuous Cooling is required .
e) Twelve hours of on-site fuel storage for 'N' capacity.
2.4.2 The performance confirmation tests :
a) A single failure of any capacity system, capacity component, or distribution element will not
impact the critical environment.
b) The infrastructure controls system demonstrates autonomous response to a failure while
sustaining the critical environment.
c) Each and every capacity component and element in the distribution paths can be removed
from service on a planned basis without impacting any of the critical environment.
d) There is sufficient capacity to meet the needs of the site when redundant components or
distribution paths are removed from service for any reason .
a) The site is not susceptible to disruption from a single unplanned event.

b) The site is not susceptible to disruption from any planned work activities.
c) The site infrastructure maintenance can be performed by using the redundant capacity
components and distribution paths to safely work on the remaining equipment.
d) During maintenance activity where redundant capacity components or a distribution path shut
down, the critical environment is exposed to an increased risk of disruption in the event a
failure occurs on the remaining path. This maintenance configuration does not defeat the Tier
rating achieved in normal operations.
e) Operation of the fire alarm, fire suppression, or the emergency power off (EPO) feature may
cause a data center disruption.
7
Uptime Institute~
2.5 Engine-Generator Systems

Engine-generator systems are considered the primary power source for the data center. The local power
utility is an economic alternative. Disruptions to the utility power are not considered a failure, but rather an
expected operational condition for which the site must be prepared. Accordingly, engine generators must
automatically start and assume load upon loss of utility.
2.5.1 Site on Engine-Generator Power
A Tier Ill or IV engine-generator system, along with its power paths and other supporting
elements, shall meet the Concurrently Maintainable and/or Fault Tolerant performance
confirmation tests while they are carrying the site on engine-generator power.
2.5.2 Manufactures' Runtime Limitation
Engine generators for Tier Ill and IV sites shall not have a limitation on consecutive hours of
operation when loaded to 'N' demand. Engine generators that have a limit on consecutive hours
of operation at 'N' demand are appropriate for Tier I or II.
2.5.3 Regulatory Runtime Limitation
Engine-generator systems often have an annual regulatory limit on operating hours driven
by emissions. These environmental limits do not impact the consecutive hours of operation
constraint established in this section.
2.6 Ambient Temperature Design Points

The effective capacity for data center facilities infrastructure equipment shall be determined at the
peak demand condition based on the climatological reg ion and steady state operating set points for the
data center. All manufactures' equipment capacities shall be adjusted to reflect the ex1reme observed
temperatures and altitude at which the equipment will operate to support the data center.
2.6.1 Ex1reme Annual Design Conditions
The capacity of all equipment that rejects heat to the atmosphere shall be determined at the
Ex1reme Annual Design Conditions that best represents the data center location in the most
recent edition of the ASHRAE Handbook- Fundamentals. (Each ASH RAE Handbook is
revised and published every 4 years.) The design Wet Bulb (WB) temperature shall be the
listed Extreme Max WB value and the design Dry Bulb (DB) temperature for design shall be the
"N=20 years" value.
2.6.2 Computer Room Set Points
The capacity for computer room cooling equipment shall be determined at the return air
temperature, and relative humidity established by the owner for steady state data center
operations.
2.7 Communications Routing

Conveyance for fiber or communications connections from off site to data center communication
demarcation must be in accordance with Concurrently Maintainable requirements for Tier Ill and Fault
Tolerant, Compartmentalized requirements for Tier IV.
2.8 Makeup Water

On-site, backup makeup water storage is required for Tier Ill and Tier IV sites using evaporative cooling.
Accordingly, the makeup water system must also be Concurrently Maintainable and Fault Tolerant as
required to the point of delivery for a minimum duration of 12 hours.
8
Uptime Institute~
2.9 Tier Requirements Summary

A summary of the preceding requirements defining the four distinct Tier classification levels is in Table 1.
Toerl Tier II Tocr Ill Toer IV
Active Capacity Components N

N N+1 N+1
to Support the IT Load After any Failure
Distribution Paths 1 1 1 Active and 1 Alternate 2 Simultaneously Active
Concurrently Maintainable No No Yes Yes
Fault Tolerance No No No Yes
Compartmentalization No No No Yes
Continuous Cooling No No No Yes
Table 1: Tier Requirements Summary
3. Commentary for Application of the Tier Standard: Topology

This Commentary is not part of the Data Center Site Infrastructure Tier Standard: Topology. It provides the reader
with context for the application of the Standard.
3.1 Outcome-Based Tier Standard

The definitions used in Uptime Institute's Tier Standard are necessarily and intentionally very broad to
allow innovation and client manufacture and equipment preferences in achieving the desired level of
site infrastructure performance or uptime. The individual Tiers represent categories of site infrastructure
topology that address increasingly sophisticated operating concepts, leading to increased site
infrastructure availability.
The operational performance outcomes that define the four Tiers of site infrastructure are very
straightforward. Many designs that pass a checklist approach will fail an operational performance
requirements approach. This means that, in addition to the rigorous application of engineering principles,
there is still considerable judgment and flexibility in the design for uptime and how subsystems are
integrated to allow for multiple operating modes.
3.2 Impact of Ambient Design Conditions

The sustainable effective capacity of most cooling and power generating equipment is impacted by
the actual ambient conditions in which it operates. These components typically require more energy to
operate and provide less usable capacity as altitude and ambient air temperatures rise.
A common practice for conventional facilities is to select design values applicable to most but not all
anticipated hours of operation of that facility. Th is results in an economical cho ice of equipment that meets
requirements most of the time. This is not appropriate for data centers that are expected to operate on a
24 x Forever basis.
Using a DB temperature for design that is exceeded 2% of the time results in selection of a component
that is undersized 175 hours of the year. Although this may seem to imply that the owner runs an
operational risk for a little over one week each year, these hours actually occur incrementally spread over
several days. The 2% design value could result in actual conditions exceeding the design parameters
of the equipment several hours every afternoon for a 1· to 2-month period. A 0.4% value, considered
conservative by many design professionals, still results in equipment performing below requirements
approximately 35 hours each year.
Another example concerning ambient conditions arises when selecting heat rejection systems for split
system direct expansion cooling system. Many manufactures provide product selection tables based on
95°F/35°C ambient outside conditions. These components will only produce the nominal capacity listed
when operating in up to 95°F/35°C outside air. These component capacities must be adjusted downward
to provide the required capacity when temperatures exceed 95°F/35°C.
3.3 Restrictions Against Engine-Generator Runtime Limitation (Tier Ill and Tier IV)
The intent of the restriction against engine-generator runtime limitation is to ensure the engine-generator
plant is capable of supporting the site load on a continuous basis. Tier topology requires that the load
capacity of engine generators bearing one of the three main ISO 8528-1 ratings (Continuous, Prime,
Standby) must be considered differently, based on the specific rating.
9
Uptime Institute··
a) Continuous-rated engine generators can be run for an unlimited number of hours at the
rated kW.
b) Prime-rated engine generators can be run for a !irnitOO number of hours at the rated kW. This
capacity does not meet the intent of Section 2.5. As stated in ISO 8528-1, the capacity of a
Prime-rated engine generator must be reduced to 70% (derated) to operate on an unlimited
basis. Some manufactures state a different reduced capacity (may be more or less than
70%) at which the engine generator can operate on an unlimited basis either in the product
specification, or by separate letter. The manufactures' certification of capacity at an unlimited
duration will be used to determine compliance with lier requirements .
c) Standby engine generators are, by definition, held to an annual run-hour limitation. This
limitation does not meet the intent of Section 2.5. Some manufactures state a different,
reduced capacity at which the engine generator can operate on an unlimited basis either in the
product specification, or by separate letter. The manufactures' certification of capacity at an
unlimited duration will be used to determine compliance with lier requirements.
3.4 Tier Functionality Progression

Owners who select Tier I and lier II solutions to support current IT technology are typically seeking
a solution to short-term requirements. Both Tier I and lier II are usually tactical solutions, i.e., driven
by first-cost and time-to-market more than life-cycle cost and uptime (or availability) requirements.
Rigorous uptime requirements and long-term viability usually lead to the strategic solutions found more
often in lier Ill and lier IV site infrastructure. lier Ill and Tier IV site infrastructure solutions have an
effective life beyond the current IT requirement. Strategic site infrastructure solutions enable the owner
to make strategic business decisions concerning growth and technology, unconstrained by current site
infrastructure topology.
3.4.1 lier I
lier I solutions acknowledge the owner's desire for dedicated site infrastructure to support IT
systems. lier I infrastructure provides an improved environment over that of an ordinary office
setting and includes: a dedicated space for IT systems; a UPS to filter power spikes, sags, and
momentary outages; dedicated cooling equipment not shut down at the end of normal office
hours; and an engine generator to protect IT functions from extended power outages.
3.4.2 lier II
lier II solutions include redundant critical power and cooling capacity components to provide an
increased margin of safety against IT process disruptions due to site infrastructure equipment
failures. The redutndant components are typically extra UPS modules, chillers, heat rejection
equipment, pumps, cooling units, and engine generators. A malfunction or normal maintenance
will result in loss of a capacity component.
3.4.3 lier Ill

lier Ill site infrastructure adds the concept of Concurrent Maintenance beyond what is available
in liar I and Tier II solutions. Concurrent Maintenance means that each and every capacity or
distribution component necessary to support the IT processing environment can be maintained
on a planned basis without impact to the IT environment. The effect on the site infrastructure
topology is that a redundant delivery path for power and cooling is added to the redundant
critical components of Tier II. Maintenance allows the equipment and distribution paths to be
returned to 'like-new' condition on a frequent and regular basis.
Thus, the system will reliably and predictably perform as originally intended. Moreover, the
ability to concurrently allow site infrastructure maintenance and IT operation requires that
each and every system or component that supports IT operations must be able to be taken
offline for scheduled maintenance without impact to the IT environment. This concept extends
to important subsystems such as control systems for the mechanical plant, start systems for
engine generators, EPO controls, power sources for cooling equipment and pumps, isolation
valves, and others.
10
Uptime Institute''
3.4.4 Tier IV
Tier IV site infrastructure builds on Tier Ill, adding the concept of Fault Tolerance to the site
infrastructure topology. Similar to the application of Concurrent Maintenance concepts, Fault
Tolerance extends to each and every system or component that supports IT operations. Tier IV
considers that any one of these systems or components may fail or experience an unscheduled
outage at any time. The Tier IV definition of Fault Tolerance is based on a single component or
path failure .
However, the site must be designed and operated to tolerate the cumulative impact of every site
infrastructure component, system, and distribution path disrupted by the failure . For example,
the failure of a single switchboard will affect every subpanel and equipment component deriving
power from the switchboard. A Tier IV facility will tolerate these cumulative impacts without
affecting the operation of the computer room.
3.5 Fractional or Incremental Tier Classification

The four Tier Standard Classifications address topology, or configuration, of site infrastructure, rather
than a prescriptive list of components to achieve a desired operational outcome. For example, the same
number of chillers and UPS modules can be arranged on single power and cooling distribution paths
resulting in a Tier II solution (Redundant Components), or on two distribution paths that may result in a
Tier Ill solution (Concurrently Maintainable)
Consistent, across-the-board application of Tier topology concepts for electrical, mechanical, automation,
and other subsystems is required for any site to satisfy the Tier standards defining any classification level.
Selecting the appropriate topology solution based on the IT availability requirements to sustain well-
defined business processes, and the substantial financial consequences for downtime, provides the best
foundation for investment in data center facilities. It is preferable for the owner's focus during the data
center design and delivery process to be on the consistent application of the Tier Performance Standard
rather than on the details that make up the data center site infrastructure.
However, site infrastructure has been occasionally described by others in the industry in terms of
fractional Tiers (e.g. , Tier 2.5), or incremental Tiers (Tier Ill+, Enhanced Tier Ill, or Tier IV-Iite). Fractional
or incremental descriptions for site infrastructure are not appropriate and are misleading. Including a
criteria or an attribute of a higher Tier Classification in the design does not increase the overall Tier
Classification. However, deviation from the Tier objective in any subsystem will prevent a site from being
Certified at that Tier.
a) A site that has an extra (redundant) UPS module but needs all the installed cooling units
running to keep the computer room temperature within limits does not meet the redundancy
requirements for Tier II.
b) A switchboard that cannot be shut down without affecting more than the redundant number
of secondary chilled water pumps (reducing the available capacity to less than N) is not
Concurrently Maintainable and will not be Certified as Tier Ill.
c) Including a UPS system patterned after a Tier IV system within a site having a Tier II power
rlistribution backbone yields a Tier II Certification.
11
Uptime Institute"'
3.6 Non-Compliance Trends

The most significant deviations from the Tier Standard found in most sites can be summarized as
inconsistent solutions. Frequently, a site will have a robust, Fault Tolerant electrical system patterned after
a Tier IV solution, but will utilize a Tier II mechanical system that cannot be maintained without interrupting
computer room operations. This results in an overall Tier II site rating .
Most often, the mechanical system fails Concurrent Maintenance criteria because of inadequate
coordination between the number and location of isolation valves in the chilled water distribution path.
Another common oversight is branch circuiting of mechanical components, which results in having to
shut down the entire mechanical system to perform electrical maintenance. If more than the redundant
number of chillers, towers, or pumps is de-energized for electrical maintenance, computer-room cooling is
impacted.
Electrical systems often fail to achieve Tier Ill or Tier IV criteria due to design choices made in the
UPS and the critical power distribution path. UPS configurations that utilize common input and output
switchgear are almost always unmaintainable without critical environment outages and will fail the Tier
Ill requirements even after spending many hundreds of thousands of dollars. Topologies that include
static transfer switches in the critical power path for single-corded IT devices will likely fail both the Fault
Tolerance criteria and the Concurrent Maintenance criteria.
Consistent application of standards is necessary to have an integrated solution for a specific data center.
It is clear that the IT organization invests heavily in the features offered by newer critical environment
technology. Often, as the electrical and mechanical infrastructures are defined and the facility operations
are established, there is a growing degree of inconsistency in the solutions incorporated in a site. An
investment in one segment must be met with a similar investment in each of the other segments if any
of the elements in the combined solution are to have the desired effect on IT availability. A well-executed
data center master plan or strategy should consistently resolve the entire spectrum of IT and facility
requirements.
Modifications
This Standard incorporates the 2010 voting results of the Owners Advisory Committee.
The engine-generator fuel storage requirements is effective 1 May 2010.
The changes incorporated are a result of the 2012 discussion and voting by the Owners Advisory Committee. All
updates specific to this version are effective 1 August 2012.
About the Uptime Institute

Uptime Institute is an unbiased advisory organization focused on improving the performance, efficiency, and
reliability of business critical infrastructure through innovation, collaboration , and independent certifications.
Uptime Institute serves all stakeholders responsible for IT service availability through industry leading standards,
education, peer-to-peer networking, consulting, and award programs delivered to enterprise organizations and
third-party operators, manufacturers, and providers. Uptime Institute is recognized globally tor the creation and
administration of the Tier Standards & Certifications for Data Center Design, Construction, and Operational
Sustainability along with its Management & Operations reviews, FORCSSTM methodology, and energy
efficiency initiatives.
Questions?
Please contact your regional representative online: http://uptimeinstitute.com/contact-us,
or email us at: info@uptimeinstitute.com
Uptime Institute is a division of The 451 Group, a leading technology

industry analyst and data company Uptime Institute has oHice
locations in the U S., Mexico, Costa Rica, Brazil, U K., Spain, U A.E ,
Russia, Taiwan, Singapore, and Malaysia.
Visit www.uptimeinstitute com for more information.
© 2009-2014 Uptime Institute, LLC All rights reserved

00001 A
12
Uptime Institutee
)
Uptime Institute·
Abstract
The Uptime Institute Tier Standard: Operational Sustainability is an objective methodology for data center
owners to align the facility management program with the specific Tier of installed site infrastructure in order to
achieve the organization's business objectives or mission imperatives. Tier Standard: Operational Sustainability
establishes the behaviors and risks beyond the Tier Classification System (I, II, Ill, and IV) that impact long-term
data center performance. Tier Standard: Operational Sustainability unifies site management behaviors with the
Tier functionality of the site infrastructure.
Keywords
data center, infrastructure, Tier, Classification , Tiers, Tier level, topology, availability, reliability, redundant,
Concurrent Maintenance, Concurrently Maintainable, Fault Tolerance, Fault Tolerant, Operational Sustainability,
functionality, performance, metrics, Tier Standard, behaviors, risks , business objectives, mission imperatives,
maintenance, failure response , critical load, capital investment, elements, Management & Operations, Building
Characteristics, Site Location, design, construction, commissioning, transition-to-operations, Basic Capacity,
Redundant Capacity, human error, staffing, organization, housekeeping, maintenance management system,
service level agreements, life cycle, training, on-the-job training, planning, coordination, management,
site policies, financial management, site infrastructure library, building features, design principles,
operating conditions, natural disasters, man-made disasters, Abnormal Incident Reports database,
method of procedure, failure analysis, preventive maintenance, predictive maintenance, deferred maintenance,
quality control, site configuration procedures, standard operating procedures, emergency operating procedures,
purpose built, security, access, setback, set point, flood plain, seismic zone, risk evaluation
Copyrights
This document is copyrighted by the Uptime Institute, LLC. Uptime Institute-in making this document available
as a reference to governmental agencies, public institutions, and private users-does not waive any rights in
copyright to this document.
Uptime Institute's Publications are protected by international copyright law. Uptime Institute requires written
requests at each and every occasion that Uptime Institute's intellectual property or portions of the intellectual
property are reproduced or used. Uptime Institute copyright extends to all media-paper, electronic, and video
content-and includes use in other publications, internal company distribution, company Web sites and marketing
materials, and handouts for seminars and courses.
For more information, please visit www.uptimeinstitute.com to download a Copyright Reprint

Permission Request Form.
Uptime Institute'
Introduction
This introduction is not part of the Data Center Site Infrastructure Tier Standard: Operational Sustainability.
It provides the reader with context for the application of the Standard.
Tier Standard: Topology (available separately) describes the functionality requirements of the site infrastructure
to meet the specific business objective or mission imperative. Long-term availability of data center infrastructure
is not guaranteed by Tier alone. Tier Standard: Operational Sustainability defines the behaviors and risks beyond
Tier that impact the ability of a data center to meet its uptime objectives over the long term. The uptime of a data
center is the resultant combination of both Tier of the site infrastructure and Operational Sustainability. This
Standard is a tool to help owners maximize infrastructure investment. Additionally, this Standard facilitates
comparison of data centers from an operational perspective.
Similar to Tier of installed infrastructure equipment, the rigor and sophistication of Operational Sustainability site
management concepts and methodologies are established by the business requirements of the site. A Tier Ill
performance requirement results in a more complex site infrastructure than a Tier I. Similarly, a Tier Ill data center
requires more comprehensive behaviors and more rigor in mitigating risks than a Tier I. Therefore, Operational
Sustainability behaviors and risk identification & mitigation are directly tied to the Tier Classification System.
The three elements of Operational Sustainability, in order of decreasing impact to operations, are Management &
Operations, Building Characteristics, and Site Location. Each of these three elements has multiple categories and
components with associated behaviors and risk . The Institute Abnormal Incident Reports (AIRs) database reveals
that the leading cause of reported data center outages are directly attributable to shorfalls in management, staff
activities, and operations procedures. Accordingly, Management & Operations is the most influential element to
sustain operations.
Finally, Tier Standard: Operational Sustainability defines behaviors that result in more efficient operation of the
data center, thereby offering opportunities to increase energy efficiency.
Additional Factors and Exposures

Uptime Institute Tier Standard: Topology and Tier Standard: Operational Sustainability establish a consistent
set of performance criteria that can be satisfied , and adjudicated , worldwide. For the data center design,
implementation, and sustained operation to be successful, additional factors and exposures must also be
considered by the owner and project team. Many of these will be dictated by the site location as well as local,
national, or regional considerations and/or regulations. For example, building codes and Authorities Having
Jurisdiction (AHJs); seismic, extreme weather (high winds, tornado); flooding; adjacent property uses; union
or other organized labor force; and/or physical security (either as corporate policy or warranted by immediate
surroundings).
Due to the many design and management options that may be dictated by the owner, regulated by local
government, recommended by industry groups, or followed as a general practice, it is not feasible for Tier
Standard: Topology and Tier Standard: Operational Sustainability to establish criteria for these additional factors
and exposures worldwide. And, the Uptime Institute does not wish to displace or confuse the guidance of local
experts, which are key for timely project delivery, regulatory compliance, and implementation of best practices.
For a successful project, Uptime Institute recommends that the project team create a comprehensive catalogue of
project requirements, which incorporates Tier Standard: Topology, Tier Standard: Operational Sustainability, and
carefully considered mitigation measures of these additional factors and exposures . This approach will ensure the
project meets the compliance objectives of Uptime Institute's international standards, as well as local constraints
and owner's business case.
2
Uptime Institute"
Contents
1.0 Overview ... ........... .. ... ...... ........ ..... .... ...... .. .... .. ....... ......... .. ........ ............. .............................................. . 4
1.1 Scope ...... ...... .... ....... .................... ........... .... .. ......... ...... ... ............. .............................................. 4
1.2 Purpose ........................ .... ....................... ........ .... .... ...... ..... ............. ,............ .......... .................... 4
1.3 Tier Standard: Topology ...... ....... ................ .......... ............ ...... ................................................. 4
1.4 Relationship between Tiers and Operational Sustainability ..................................................... 5
1.5 Exclusions from Operational Sustainability .......... .... .......... .... ................................................... 5
1.6 Reference ............ .... .............. ....... .... ............ ..... .. ........ ... ...... ...... ............................................... 5
2.0 Elements Of Operational Sustainability ....... .. .. .. ..... ...... ........... .. .. ..... ........... .. ................................. 5
2.1 Management & Operations ............. ....... ............ ... ..... ........... ........ .............. ................................ 5
2.2 Building Characteristics .............. .......... ....... ... ..... ........... ...... ... ................................................. .. 5
2.3 Site Location ... .. ..... ... ... ........... ..... ............... ...... .. .... ..... ......... ................................................. ..... 6
3.0 Topology Enhancements ...... ....... ... .. ................. .. .... ......... ... .... ....... .... .............................................. 6
4.0 Behaviors and Risks .. ....... ... ........ ....... .. ........... ..................... ......... ...... ............ -................................. 6
4.1 Table Organization ..... ........... ..... .... ........................... ..... ........... .................................................. 6
4.2 Evaluating Effectiveness ..... .. .......... ..... ..... ... ............. .. .................. .................................. - .......... 6
4.3 Prioritization .................. ............................. .... .. ...... .... ............ ...... ............. ......... .............. ........... 7
5.0 Summary ........... .... .... ........... ... .. .. .... ..... ........ ......... ........ ... ........................................................... 7
6.0 Certification ........ ..... ...... .... .......... ........ .... ........ .......... ........ ... ... ...... ....................... ............... ........ 7
Modifications ... .. ... .................. .... ............. ... ... ..... ............. .... ......... ......... ... .... ................. ....................... ...... 7
Table 1.1 Management & Operations-Staffing and Organization ... .. .. ................................................ 8
Table 1.2 Management & Operations-Maintenance ................... ..... ............................... ...................... 9
Table 1.3 Management & Operations-Training ........ .. ....... ............. .... ................................................... 10
Table 1.4 Management & Operations-Planning, Coordination, and Management.. .... ..................... 11
Table 1.5 Management & Operations-Operating Conditions ...... ....... .... ...... .... ................................... 12
Table 2.1 Building Characteristics-Pre-Operational ...... .......... ......... ...... .. .. ...... ... ......... ....................... 12
Table 2.2 Building Characteristics-Building Features ................................ ... .. ................... ................ 13
Table 2.3 Building Characteristics-Infrastructure .. .............. ...... .... .... ........ .... ... ... ................. .............. 14
Table 3.1 Site Location - Natural Disaster Risk ................. ...... .. ...... ....... .. ...... ............... ......................... 15
Table 3.2 Site Location- Man-Made Disaster Risk ........ .......................... ...... ..... ...... ........ ..................... 15
3
Uptime Institute~
1. Overview
1.1 Scope
This Standard establishes the Uptime Institute Data Center Site Infrastructure
Tier Standard: Operational Sustainability. This Standard establishes the behaviors and risks beyond Tier
of installed infrastructure that impact the ability of a data center to meet its business objectives or mission
imperatives over the long term.
This owner's Standard is complementary to the Tier Standard: Topology. Tier Standard: Topology establishes the
performance requirements for configuration of the power and cooling equipment, including the capability
for redundancy, planned maintenance work, or failure response without impacting the critical load. Tier
Standard: Topology does not prescribe or constrain solutions. Rather the intent of Tier Standard: Topology is to
provide the framework to align site infrastructure capital investment with the business objective(s) or
mission imperative(s) that the data center supports.
Tier Standard: Operational Sustainability provides the site management behaviors and risks-contingent upon
achievement of the site infrastructure. Consistent with the progressive nature of Tier Standard: Topology,
Operational Sustainability behaviors increase in complexity and comprehensiveness as
Tier increases.
The three elements of Operational Sustainability are Management & Operations, Building Characteristics,
and Site Location. Each of these three elements has multiple categories and components with associated
behaviors or risks. The specific behaviors, prioritized so that owners may address
highest risks first, are presented in table format in this Standard.
The benefits of Operational Sustainability behaviors are fully realized when incorporated early into the
project in the conceptual planning. Then, carried through design, construction, commissioning, and
transition-to-operations-and ultimately addressed on a persistent basis during the operational life
of the data center.
1.2 Purpose
Tier Standard: Operational Sustainability provides data center owners, operators, and managers with
the prioritized behaviors and risks intrinsic to data center operations. Adherence to the recommended
behaviors will assist in attaining the full performance potential of the installed infrastructure. This
Standard is a tool to help owners maximize infrastructure investment. Additionally, this Standard facilitates
comparison of data centers from an operational perspective. Tier Standard: Operational Sustainability
establishes a baseline of site management behaviors by Tier.
1.3 Tier Standard: Topology

Tier Standard: Topology establishes four distinctive definitions of data center site infrastructure using the
Tier Classifications (I, II, Ill, and IV) and the performance confirmation tests for determining compliance to
the definitions. The Tier Classification System describes the site-level infrastructure topology required to
sustain data center operations, not the characteristics of individual systems or subsystems.
For informational purposes, the following is a brief summary of each Tier from the
Tier Standard: Topology.
· Tier I- Basic Capacity: Site-wide shutdowns are required for maintenance or repair work.
Capacity or distribution failures will impact the site.
• Tier II - Redundant Capacity components: Site-wide shutdowns for maintenance are still required.
Capacity failures may impact the site. Distribution failures will impact the site.
• Tier Ill- Concurrently Maintainable : Each and every capacity component and distribution path
in a site can be removed on a planned basis for maintenance or replacement without impacting
operations. The site is still exposed to a equipment failure or operator error.
· Tier IV- Fault Tolerant: An individual equipment failure or distribution path interruption will not
impact operations. A Fault Tolerant site is also Concurrently Maintainable .
4
Uptime Institute~
1.4 Relationship between Tiers and Operational Sustainability

Similar to Tier of installed infrastructure, the rigor and sophistication of Operational Sustainability site
management concepts and methodologies are driven by the business requirements of the site. All three
Operational Sustainability elements impact the performance potential of the Tier topology of the installed
infrastructure, yet the Management & Operations Element has the largest impact on long-term availability.
Staffing levels, the approach to maintenance, and the number and details of processes and procedures
are illustrative categories of the Management & Operations Element that are directly related to each Tier
level.
1.5 Exclusions from Operational Sustainability

Safety, environmental, and personnel management are not addressed in Tier Standard: Operati@oal
Sustainabi/ity. Failure to address any of these will add significant risk to data center operations. Yet, these
items are excluded from the Standard because they are under the purview of a) management or internal
corporate compliance audit groups and/or b) external enforcement and regulatory agencies.
1.6 References and Related Publications

Data Center Site Infrastructure Tier Standard: Topology
Natural Disaster Risk Profiles for Data Centers
Further information can be found at www.uptimeinstitute.com.
2. Elements Of Operational Sustainability

2.1 Management & Operations
Analysis of the Institute AIRs database reveals that the majority of the reported data center outages are
directly attributable to human error. Human error includes operato r error-but more importantly, speaks
to management decisions regarding staffing, maintenance, training, and overall rigor of the operation.
The right number of qualified people is critical to meeting long-term performance objectives. Without the
right number of qualified employees organized correctly, a data center does not have the resources to be
successful.
After correct staffing, a comprehensive approach to maintaining a data center is an absolute requirement
to achieve the uptime objective. An effective maintenance program encompasses increasingly rigorous
preventive maintenance (PM), housekeeping policies, maintenance management system (MMS) to track
work, and service level agreements (SLAs). As the performance objective increases, the requirements for
documentation, complexity, and detail for each of these items increases.
In addition, a comprehensive training program ensures consistent operations and maintenance of a data
center's infrastructure. All personnel must understand policies, procedures, and unique requirements of
work in the data center to avoid unplanned outages and respond to anticipated events.
2.2 Building Characteristics

Buildiny Characteristics include commissioning, building features, and infrastructure that can potentially
affect attainment of availability objectives.
An extensive commissioning program is critical to a data center achieving a desired uptime objective.
Comprehensive commissioning is the only way to ensure that the site infrastructure functions according
to the design. It also provides the data center operators the opportunity to operate and test procedures
without impacting the critical environment. Commissioning should exercise the equipment enough to
identify potential manufacturing defects.
Building features can positively or negatively impact the availability objectives. Building features that
support performance objectives include purpose-built data centers, adequate space for support and
specialty spaces, and controlled access areas.
Additional infrastructure beyond just providing power and cooling is necessary to support the operation
of a data center. Mechanical support systems such as chemical treatment and fuel scrubbing extend
the life of a system and decrease the risk of failure. Having adequate space to safely conduct normal
maintenance activities also reduce the risk of human error. The space, power, and cooling exhaust points
must be aligned and monitored to avoid wasted capital expenditures.
5
Uptime Institute"
2.3 Site Location

The highest level of functionality in a data center can be easily defeated by a local or regional disaster
whether relating to natural occurrences or man-made factors. The site selection process for a new data
center should evaluate the risks of these types of disasters, For new or existing data centers, these risks
must be well documented, signed off by management, and with the proper level of mitigation in place.
Thus, both management expectations and the likely impact of the event on availability are accounted for.
Depending on the performance objective, mitigation actions may be required.
3. Topology Enhancements
It is the data center owner's prerogative to enhance topology above that required for a specific Tier.
Therefore, topology enhancements are not one of the Operational Sustainability Building Characteristics
behaviors in this Standard. However, topology enhancements can significantly increase the performance
potential by providing redundant capacity/distribution paths or Fault Tolerance above that required for a
specific Tier. Having redundant components in a Tier IV System+System configuration reduces the risk
of human error. Owners should consider practical topology enhancements for critical systems to support
their Operational Sustain ability program. The evaluation of topology enhancements should balance the
synergies of having greater operational flexibility with greater operational complexity.
4. Behaviors & Risks

4.1 Table Organization
The tables in this Standard list and categorize the Operational Sustainability behaviors and risks. As
the uptime objective (Tier) of a data center increases so does the number of Operational Sustainability
behaviors required to achieve that objective. The behaviors applicable for each specific Tier are marked
with a check .
The Site Location tables list the risks that should be evaluated. The tables also provide specific criteria to
determine the magnitude of the risk. Develop mitigation plans for each risk identified.
4.2 Evaluating Effectiveness

The tables provide behaviors not requirements. There are many different ways to achieve these
behaviors. It is important the behaviors both exist and are effective in order for the data center to achieve
it uptime objective. There are three core principles to determine Operational Sustainability effectiveness:
Proactive, Prac tieed, and lnfoJmed. Evidence of all three principles must be present for a behavior to be
considered effective.
4.2.1 Proactive - Is there a continuous improvement component present to ensure the processes and
procedures are always being improved and kept updated? Behaviors are anticipated and ap-
propriate processes and procedures are in place in advance Evidence of a Proactive principle
includes well-documented processes for all existing and anticipated ac~ivities with procedures
in place for regular review and update
4.2.2 Practiced -Are processes and procedures always followed? Having processes and
procedures alone will not enhance Operational Sustainability unless all data center personnel
consistently follow a disciplined approach. A task or procedure always accomplished the same
way, no matter who is performing it, is evidence of th is principle.
4.2.3 Informed - Is the knowledge to achieve a behavior held by the organization or an individual?
Do all staff have knowledge of and access to all processes and procedures for any activity they
might be required to perform? For example, does the maintenance technician required to
perform a specific activity: 1) know there is a method of procedure (MOP) available for that
activity, 2) where to find it, and 3) is granted access to it.
6
Uptime Institute~
4.3 Prioritization
The prioritization of the Management & Operations and Building Characteristic behaviors are based on
analysis of the AIRs database. Within each element, the categories and components are listed in the
tables in order of decreasing importance.
Site Location risks are of equal importance but specific criteria identifies the risk scale as higher or lower
based on the magnitude of potential impact. The level of mitigation in place will reduce potential impact to
operations.
5. Summary
The behaviors established in Tier Standard: Operational Sustainability combined with the infrastructure
requirements in the Tier Standard: Topology are essential for a site to achieve its uptime potential.
The installed infrastructure alone cannot ensure the long-term viability of the site unless Operational
Sustainability behaviors are addressed. Site management teams that incorporate the principles of both
Standards will have notably better results in realizing or exceeding the full uptime potential of the installed
infrastructure.
6. Certification
The Uptime Institute reserves the exclusive right to rate and Certify data centers according to
Tier Standard: Topology and Tier Standard: Operational Sustainability.
Please refer to www.uptimeinstitute.com.
Modifications
This Standard incorporates wording and organizational changes to clarify select behaviors.
The Operational Sustainability Rating information is available at www.uptimeinstitute.com.
7
Uptime Institute·
support the business objective <I'
3. 24 x 7 staff presence: minimum of 1 qualified FTE <I'
4. 2 qualified FTEs of facility support per shift <I'
5. Total FTE count numerically matches load requirements <I' <I' <I'
6. Escalation and call-out procedures are in place for assigned staff and specified
<I' <I' <I'
vendor support for designated critical systems and equipment
7. Engineering trade (e.g. , electrical, mechanical, controls, building management
system [BMS], etc.) coverage split by shift based on operations and maintenance <I' <I'
requirements
Qualifications 1. Appropriate staff trade licenses required by governmental regulation <I' <I' <I' <I'
2. Experience and technical training required to properly maintain and operate the
<I' <I'
installed infrastructure
3. Shift personnel qualified for specific shift operations individually and as a shift team <I' <I'
Organization 1. Organization chart showing reporting chain and all interfaces between the Facility,
<I' <I' <I' <I'
Engineering, Information Technology (IT), and Security groups
job descriptions-available and in use <I' <I' <I'
<I' <I'
<I'
5. Integrated approach to operational management, including all facets of the data

<I' <I'
center operation (Facilities, IT, and Security)
Table 1.1 Management & Operations-Staffing and Organization Category
8
Uptime Institute"
Preventive 1. Effective preventive maintenance (PM) program including list of maintenance

Maintenance
Program
a due and record of let ion "' "' "' "'
2. PM program encompasses original equipment manufacturer (OEM) maintenance
recommendations "' "' "'
3. Detailed procedures for switching between redundant equipment- available and in
use "' "'
4. Fully scripted preventive maintenance activities (PMs) (e.g., through a method of
"' "'
5. Quality control process in place that validates a) the proper completion of and b) the
of the PM "' "'
Housekeeping
Policies
1. Computer room floor and underfl of dirt and debris
2. Data center free of combustibles, cleaning equipment, shipping boxes, or personal
"' "' "' "'
conveniences items , coffee "' "' "'
3. Housekeeping policies available and enforced to ensure a contaminant free data
center environment "' "'
Maintenance 1. Effective maintenance management system ([MMS] paper or computerized) to track
Management
System
status of all maintenance
activities-available and in use "' "' "' "'
2. Maintains list of installed equ ipment (make, model, year of manufacture, year of
installation, o rating specifications, warranty information, "' "' "'
3. Work orders list special tools or parts required to complete PMs
4. Maintains performance/trend data on equipment and history of maintenance
activities
Vendor Support e for normal and

2. Service level agreements (SLA) outlining scope of work, PM schedule, training
"' "'
uirements, and nse times for all critical "' "' "'
3. Vendor call-in process and po ints-of-contact for pre-approved and qualified
technician "' "'
Deferred
Maintenance
90%
"'
Program
3. Process for tracking deferred maintenance and executing it during a maintenance
"' "'
window "' "'
Predictive
Maintenance
Program
1. Effective predictive maintenance program
"' "'
Life-Cycle 1. Effective process for planning, scheduling, and funding the life-cycle replacement of
Planning major infrastructure components "' "'
Failure Analysis 1. Maintains list of all outages including dates, times, infrastructure equipment/
Program systems involved and specific computing outages, root-cause analysis, and lessons
learned "' "' "'
2. Effective process to determine root cause, identify lessons learned, and implement
corrective actions "' "'
3. Trend analysis process v-·
Table 1.2 Management & Operations-Maintenance Category
9
Uptime Institute"
Training Category Applicable for Tier
Component Behavior I II Ill IV
Data Center 1. On-the-job training (OJT) program for each new employee on a) the system(s) they
Staff Training will be responsible for operating and maintaining, and b) the rules of working in the ./ ./ ./ ./
data center
2. Documented formal classroom, operational demonstrations, and/or shift drills
./ ./
covering the following:
. All policies, processes, and procedures for the operation and maintenance of data
center systems
. Site Configuration Procedures (SCPs)-how the infrastructure is configured for
normal operation
. Standard Operating Procedures (SOPs)-how the infrastructure configuration is
changed during normal operations
. Emergency Operating Procedures (EOPs)-how the site is controlled and operated
during abnormal circumstances or emergency situations
. MOPs
. MMS Procedures
3. Training programs include training schedule, lesson plans, required reference ./ ./
materials, and records of attendance
4. Formal qualification program for designated personnel performing data center
./ ./
operations
Vendor Training 1. List of training required before a vendor is allowed to work in the data center ./ ./ ./ ./
(Part-time 2. Briefing on data center processes and procedures with respect to the work to be
Support) ./ ./ ./
performed
3. Formal training covering the appropriate subset of training received by the data ./ ./
center staff
4. Training programs include training schedule, lesson plans, required reference ./ ./
materials, and records of attendance
Table 1.3 Management & Operations-Training Category
10
Uptime Institute~
Planning, Coordination, and Management Category Applicable for Tier
Site Policies 1. Formal documented policies and procedures for the following:
.
../ ../ ../
Site staff performs all site infrastructure operations (e.g., configuration changes
and operations under normal, emergency, or abnormal conditions)
. Site Configuration: site infrastructure configuration for normal operations
. Standard Operations: changes to normal operating configuration (e.g., shifting
chillers)
. Emergency Operations: control of the site during abnormal circumstances or event
. Change Management: a) review and approval of changes to the site baseline and b)
evaluation of risk as related to planned changes
. Mitigation plans for site risks
~
Financial 1. Process to ensure that operating and capital funding levels are consistently sufficient
../ ../ ../
Process and available to support the business objective
2. Operating and capital budgets managed separately from non-critical facilities and are
../
not pooled with other buildings or groups of buildings
Referen ce 1. The following reference and record documents available for use (off site or on site): ../ ../ ../ ../
Library . As-built drawings
. Operation and maintenance documentation
. Studies (e.g. , soils, structural, electrical, mechanical, breaker, circuit, etc.)
. Commissioning reports
. Warranty documentation and pre-purchased maintenance agreements
. Written automation sequences of operation
2. The previously listed reference and record documents available on site at all times ../ ./
3. Reference documents located in centralized location (library) available to site

./ ./
operational personnel
4. Process ensuring master copies are maintained current with additional copies
./ ./
available to site operational personnel , vendors, designers, Atr. .
Capacity 1. Process for managing the installation and removal of IT equipment from the
./ ./ ./ ./
Management computer room
2. Computer room floor plan-developed and regularly reviewed/updated ./ ./ ./
3. Process for forecastino futurA s~or.A, ~nwA r, ond cooling growth requirements on a
./ ./
periodic basis (e.g., 1/6/12/24/36 month)
4. Tracking mechanism for current space, power, and cooling capacity and utilization
./ ./
reviewed periodically
5. Effective process for a) computer room airflow management and b) electrical power
./ ./
monitoring, management, and analysis
Table 1.4 Management & Operations - Planning, Coordination, and Management Category
11
Uptime Institute®
Load 1. Process to ensure the maximum loads are not exceeded and capacity is reserved for
Management switching between components
Operating Set 1. Consistent operating set points (e.g., temperature, pressure, volumetric flow, etc.)
Points established based on both risk to continuous availability and cost of operation
Rotating
1. Effective process for alternating the use of redundant infrastructure equipment as
Redundant
part of the site maintenance program
Equipment
Table 1.5 Management & Operations-Operating Conditions Category
2. Re critical infrastructure components

3. Functional testing, critical infrastructure stand-alone testing, and pre-system startup
configuration
4. System start, OEM test, and individual test (1ST)
5. Integrated systems operational test (ISOT)
Table 2.1 Building Characteristics-Pre-Operational Category
12
Uptime Institute®
I
2. Single-purpose facility to support IT equipment operations ~ ~
Stand-alone building physically separated from other corporate facilities on the site ~ ~
4. Data center built to standards exceeding local building codes to ensure continued
~ ~
operations following a natural event
Support and 1. Adequate space separate from computer room for IT hardware receiving, storing,
~ ~ ~
Specialty staging, building, and testing
Spaces 2. Adequate space separate from computer room for the following functions: ~ ~
. BMS/Building Automation System (BAS) control center

. Command Center/Disaster Recovery
. Parts and tool storage
. Engineering and Facility shop activities
. Meeting and training purposes
Security and 1. Controlled access to all computer rooms and support spaces ~ ~ ~
Access
2. Controlled building access ~ ~
3. Periodic review of access ~ ~
4. Controlled site access V'
Setbacks Adequate space around the data center to minimize impacts from adjacent facilities ~ ~
Table 2.2 Building Characteristics-Building Features Category
13
Uptime Institute®
Infrastructure Category Applicable for Tier
Flexibility for 1. Designed and constructed so that computer room space can be reconfigured with
Incremental reasonable effort, and incremental increases in space, power, and cooling can be ~ ~
Capacity accomplished with minimal risk to the existing critical load

Increases
2. Connection points for future/temporary extensions or capacity units ~ ~
Infrastructure 1. Mechanical support systems available (e.g., chemical treatment, fuel scrubbing, etc.)
~ ~
to Support to extend the life of or protect the infrastructure
Operations 2. Mechanical systems installed to facilitate ease of operations ~ ~
3. Consistent labeling of infrastructure equipment and standardized sizes ~ ~
4. Electrical systems installed to facilitate ease of operations ~ ~
Ease of 1. Adequate space for the safe conduct of all normal maintenance activities on
~ ~ ~
Maintenance infrastructure equipment
2. Adequate space (sufficient swing radii, lifting points, and in/out pathways) for the
~ ~
safe conduct of rapid removal and replacement on infrastructure equipment
3. Equipment access provided to facilitate delivery and installation of motors or other
~ ~
large components
Space, Power,
and Cooling 1. Data center design coordinated space, power, and cooling capacity exhaust points ~ ~
Exhaust Points
Table 2.3 Building Characteristics-Infrastructure Category
14
Uptime Institute"
Natural Disaster Risk Category Scale of Risk'
Component Higher Lower
Flooding (river, lake, reservoir, canal, pond, etc.)

< 100-YearFiood Plain > 100-Year Flood Plain
and Tsunami 2
Hurricanes, Tornadoes, and Typhoons High Medium
Seismic Activity 3 > 0.8 m/s2 < 0.8 m/s 2
Active Volcanoes High Medium
Table 3.1 Site Location-Natural Disaster Risk Category
Man-Made Disaster Risk Category Scale of Risk'
Component Higher Lower
Airport /Military Airfield < 3 miles from any active > 3 miles from any active
runway; inside a 1 x5-mile runway; outside a 1 x5-mile
runway extension runway extension
Adjacent Properties Exposures Chemical plant, fireworks Office building,
factory, etc. undeveloped land, etc.
Transportation Corridors < 1 mile > 1 mile
Table 3.2 Site Location-Man-Made Disaster Risk Category
1
The level of mitigation in place will reduce potential impact to operations.
2
Risk evaluation from the regional or local flood plain map or international equivalent.
3
Peak Ground Acceleration (meters per second squared [m/s')) that can be expected during the next 50 years with 10% probability.

reliability of business critical infrastructure through innovation, collaboration, and independent certifications.
Uptime Institute serves all stakeholders responsible for IT service availability through industry leading standards,
education, peer-to-peer networking, consulting, and award programs delivered to enterprise organizations and
third-party operators, manufacturers, and providers. Uptime Institute is recognized globally for the creation and
admini stration of the Ti er Stand ard s & Certifications for Data Center Design, Co nstruction, and Operational
Sustainability along with its Management & Operations reviews, FORCSSTM methodology, and energy
efficiency initiatives.
Questions?
Please contact your regional representative online: http://uptimeinstitute.com/contact-us,
or email us at: info@uptimeinstitute.com
Uptime Institute is a division of The 451 Group, a leading technology

industry analyst and data company Uptime Institute has office locations
in the U.S , Mexico, Costa Rica, Brazil, UK., Spain, U A E, Russia,
Taiwan, Singapore, and Malaysia.
Visit www.uptimeinstitute.com for more information.
© 2010·2014 Uptime Institute , LLC All rights reserved

15 00002 A
Uptime Institute®
Uptime Institute~
This technical paper provides additional detail regarding the Tier consequences
of engine-generators and their ratings. This technical paper was prompted by the
interactions at Accredited Tier Designer sessions and industry comments and queries.
Tier Requirements
The core premise, as set forth in the Tier Standard: Topology, is that the only reliable source of power for a
data center is the engine-generator plant. Although the purchase of power from the local utility is an economic
alternative, there are no considerations of this utility power that affect the owner 's target Tier objective.
Instead, for Tier Ill (Concurrently Maintainable) and Tier IV (Fault Tolerant) functionality objectives, the Tier
Standard: Topology states:
"Engine generators for Tier Ill and IV sites shall not have a limitation on consecutive hours of operation
when loaded to 'N' demand. Engine generators that have a limit on consecutive hours of operation at N
demand are appropriate for Tier I or II. "
There are two key aspects of this requirement: 1) disruptions to the utility power are not considered a
failure, but an anticipated operational condition for which the site must be prepared , and 2) a Tier Ill or IV
engine-generator system, along with its power paths and other supporting elements, shall meet the
Concurrently Maintainable and/or Fault Tolerant performance confirmation tests while they are carrying the
site on engine-generator power.
Two scenarios of operation of the engine-generator plant for an extended period (weeks to months) are the
loss of the local utility due to malfunctions within the utility system - resulting in extended outages-or the
catastrophic malfunction of the UPS system . The latter requires that the engine-generator plant be run to
ensure the most reliable and stable power is being delivered to the IT critical environment. If the local utility
is utilized during a UPS outage, then any perturbation or loss of the utility will impact the computer room
operation and potentially result in an outage .
Uptime Institute field experience and member data shows that the availability and reliability of the
infrastructure is paramount to achieving the business objectives or mission imperative for the data center.
Accordingly, engine generators m1,1st have no runtime limitations at N units.
Ratings
Engine generators and their ratings are governed by International Organization for Standardization (ISO)"
Standard 8528-1. This standard covers Reciprocating Internal Combustion (RIC) engines, alternating
current (AC) generators , and associated systems. The three principal ratings as defined in the standard are
Emergency Standby, Prime, and Continuous.
"Emergency Standby Power: The maximum power for which an engine-generator is capable of
delivering for up to 200 hours per year. The allowable average power output over a 24-hour run
period is 70% of the standby rating unless otherwise agreed to by the RIC manufacturer."
"Prime Power: The maximum power for which an engine-generator is capable of delivering continuously
with a variable load for an unlimited number of hours. The allowable average power output over a
24-hour run period is 70% of the prime rating unless otherwise agreed to by the RIC manufacturer."
"Continuous Power: The maximum power for which an engine-generator is capable of delivering
continuously for a constant load for an unlimited number of hours."
When practically applying these definitions and the requirement for no runtime limitations at N demand ,
standby-rated units as defined with limited run hours do not comply with Tier Ill and IV. Standby units-
allowed to run for limited durations at constrained capacities-do not afford the data center owner the
capability to run the engine-generator plant at capacity for extended periods to support operations during
critical events and do not meet Tier requirements. Some manufacturers allow only up to 500 hours per year
for certain units. However, a standby-rated unit can comply with Tier Ill and Tier IV requirements with proper
manufacturer documentation that establishes the unlimited run hour capacity of the unit at the site conditions.
2
Uptime Institute"
Prime-rated units, per their definition, have more robustness than standby units. Many manufacturers offer
the same unit with both standby and prime ratings. However, in order to comply with the no runtime limitations
at N-load requirement , these units must be de-rated to 70% of their prime rating. Note, however, that some
manufacturers will offer a de-rating of more or less than 70% of the prime rating. It is important to work with
the manufacturer to commit to writing the specific allowance for runtimes and capacities. Continuous is the
only rating that complies with the requirement without any de-rating.
In summary, only continuous ratings, de-rated prime ratings or standby ratings with no runtime limitations
qualify for the Tier Ill or IV requirement for engine-generators.
Option 1: 70% of nameplate capacity

Option 2: Larger capacity than Option 1
No additional requirement for hours of
with manufacturer letter
operation limitations
Can be used tor Tier Ill and Tier IV with
manufacturer letter; Tier Certification
capacity dependent on manufacturer letter
Additional derating may be required due to site conditions (ambient temperatures,
elevation)-consult manufacturer requirements
Table 1: Tier Requirements Summary-Engine Generators
Related Publications
Tier Standard: Topology
Further information can be found at www.uptimeinstitute.com

reliability of business critical infrastructure through innovation, collaboration, and independent certifications.
Uptime Institute serves all stakeholders responsible for IT service availability through industry leading
standards, education, peer-to-peer networking, consulting, and award programs delivered to enterprise
organizations and third-party operators, manufacturers, and providers . Uptime Institute is recognized
globally for the creation and administration of the Tier Standards & Certifications for Data Center Design,
Construction, and Operational Sustainability along with its Management & Operations reviews, FORCSSTM
methodology, and energy efficiency initiatives.
Questions?
Please contact your regional representative: http://uptimelnstitute.com/contact-us
or email us at: info@uptlmeinstitute.com.
Uptime Institute is a division of The 451 Group. a leading technology
industry analyst and data company Uptime Institute has office
locations in the US •. Mexico. Costa Rica. Brazil. U.K.. Spain. U A.E.
Russia. Taiwan. Singapore. and Malaysia.
Visit www.uptimeinstitute.com for more information.
© 2014 Uptime lnsl ilule. LLC All rig hts reserved

00019 A
3
Uptime Institute$
Uptime Institute~
An objective of the Uptime Institute's Data Center Site Infrastructure Tier Standard
Program is the consistent application of infrastructure availability concepts to facility
systems supporting data center critical environments. This technical paper provides
a summary and follow-up to questions and discussion during the Accredited Tier
Designer (ATD) curriculum presentations in September 2009 concerning alternate
sources of water necessary to sustain operation of evaporative condenser water
towers. Consistent with the concept that the loss of Public Utility electrical power is
a design condition rather than a failure mode, the loss of the Public Utility domestic
water must also be addressed as a design condition.
In response to a question during the presentation of the ATD curriculum, consider the impact of providing
a Concurrently Maintainable source of water supply for evaporative cooling towers without regard to the
availability of the community or district vvater mains (Public Utility domestic water). This is primarily an issue
for evaporative cooling situations because of the volume of water required to maintain functional water level
in cooling towers, but applicable to any facility infrastructure that depends on the availability of water to
operate the cooling system .
As with many Tier-based design considerations, providing a Concurrently Maintainable source of water for
evaporative cooling towers can be accomplished through adaptation of traditional design solutions rather than
inclusion of unique or complex design elements. Commonly accepted "rules of thumb" provide a means to
develop boundaries of the solution, although specific engineering is required for each application based on
load and ambient conditions.
Example
Assumptions:
a. Each 1,000 kilowatts (kW) of cooling load (approximately 285 refrigeration tons) requires about 855
gallons per minute (gpm) of condenser water flow through the evaporative cooling towers, at 3 gpm of
condenser water per ton of cooling.
b. For the purposes of estimating water requirements, evaporation consumes about 1% of condenser
water flow, and drift consumes another 0.5%. During periods of emergency conditions, cooling tower
"blow-down" (purge flow to remove mineral buildup in condenser water) will be suspended to conserve
condenser water. Thus a source of water is required to replenish, or "makeup" about 1.5% of condenser
water flow to sustain evaporative cooling process.
Using the above assumptions, the amount of makeup water necessary to sustain evaporative cooling for
1,000-kW load for 24 hours is:
(855 gpm) X (60 minutes/hour) X (24 hours/day) X (1 .5%) or approximately 18,500 gallons.
This volume of water is equivalent to 18,500 gallons /7.5 gallons per ft3, or about 2,500 ft3 of water.
Thus, the evaporative cooling tower makeup water for a "megawatt-day" (1 ,000 kW for 24 hours) is
approximately 18,500 gallons (2,500 ft3) of water depending on local ambient conditions.
2
Uptime Institute"
A design based on (3) 150-ton cooling towers could provide an N+ 1 solution for the 285-ton cooling
requirement mentioned in Assumption (a.), and with the appropriate piping configuration could meet
Concurrently Maintainable topology requirements. Moreover, an on-site, Concurrently Maintainable makeup
water supply for the evaporative cooling towers could result from a topology that included a 9,250-gallon or
1,250-ft3 condenser water storage tank in the form of a sump for each cooling tower.
Considering the footprint of some popular 150-ton evaporative cooling tower choices, many of these products
are typically 8 to 9 feet wide and 10 to 12 feet long. A sump with a footprint about the same size as a 150-ton
cooling tower (9 feet X 12 feet) and about 12 feet deep has a volume of approximately 1,250 ft3 and will
contain 9,250 gallons.
Tier Requirements
Consistent application of Tier concepts requires that the amount of makeup water for evaporative cooling
towers stored on site provide an equal duration of operation as the fuel supply for the engine-generator
system. The preceding example discusses the volume of makeup water for a megawatt-day. The volume must
be adjusted to meet the site cooling load and operating duration necessary to meet project requirements. A
megawatt-day of makeup water will support a 2-megawatt load for 12 hours, while supporting a 4-megawatt
load for 12 hours will require 2 megawatt-days of makeup water.
A recently constructed Tier Ill Certified data center has 8 in-ground condenser water sumps, one for every
chilled water machine and condenser tower. Each condenser water sump (condenser water storage tank)
is 14 feet wide X 16 feet long with over 12.5 feet of useable water depth, resulting in over 160,000 gallons
of storage integrated into the condenser water system. Using the above example, this provides nearly 9
megawatt-days of makeup water.
Moreover, with adequate planning, a thermal energy storage tank integrated into the chilled water system as
part of a Continuous Cooling solution, may also provide a large volume of readily available water stored on
site to sustain evaporative cooling processes during a disruption of Public Utility domestic water.
While detailed calculations considering the rise and fall of wet-bulb temperatures over a 24-hour period may
indicate that less makeup water is required to sustain a specific load in a specific location, such calculations
are beyond the intent of this paper. This paper demonstrates that meeting the requirement to provide on-site
storage of makeup water for evaporative cooling towers is in fact a straightforward and manageable solution.
Questions?
Please contact your regional representative: http://uptimeinstitute.com/contact-us
or email us at: info@uptimeinstitute.com.
Uptime Institute is a division of The 451 Group. a leading lechnology

industry analyst and data company. Uptime Institute has office
locations in the U.S.. Mexico. Costa Rica, Brazil, U,K . Spain, U,A E..
Russia, Taiwan, Singapore, and Malaysia
Visit www~ uptimeinstitute . com for more information.
© 2014 Uptime lnstitu1e, LLC All rights reserved

00020 A
3
Uptime Institute®
Uptime Institute·
This technical paper clarifies the requirements for Continuous Cooling in the conte xt
of the Uptime Institute's Tier Standard: Topology. Tier IV is the only Tier that requires
Continu ous Cooling. Additiona lly, this paper serves to recommend Continuous Cooling
at densities beyond 4 kilowatts (kW)/rack, regardless of Tier .
As the power densities in the data center increase, the need for Continuous Cooling becomes more
profound. The risk of the loss of cooling during a UPS ride-though event can be catastrophic to a business . IT
equipment may fail or become "wounded."
However, depending on the cooling or UPS technology deployed, the definition and requirement for
Continuous Cooling can widely differ. This paper clarifies the definition of Continuous Cooling and details its
deployment with varying types of techno logy choices.
Regardless of technology, Continuous Cooling is defined as thfl flhility to rrovirlfl stflhlfl r.ooling to the IT
and UP-8 environment \"Jithout any interruption . Continuous Cooling provides thi s stable cooling capability for
the duration of the UPS ride-through time. For example, for a static UPS system with 15 minutes of battery,
the Continuous Cooling will need to provide stable cooling for 15 minutes. Note, however, that if there are
redundant modules and battery strings available, the 15 minute ride-through time could become 30 minutes
or more (depending on redundancies). Careful consideration should be taken by the owner to determine if
the stated ride-through time or the available ride-through time should be the requirement for the Continuous
Cooling solution.
Thfl AmArican Society of Heating, Refrigerating, and Air-Conditioning Engineers (ASH RAE), in concert with
major IT equipment manufacturers, established The rmal Guidelines for Da ta Processing Environments that
includes recommended computer equipment inlet air temperatures necessary to enable reliable operation
of servers, storage, and network devices. As of 2010, the ASH RAE guideline (accepted on a global basis)
recommends that the device inlet be between 66-81 oF (18-27"C) and 20-80% relative humidity to meet the
manufacturer's established criteria.
As a point of reference, Uptime Institute conducted a demonstration on a 6-kW/rack average computer room.
Intake air temperatures in computer rooms will exceed the top value in this range within 60 seconds after a
loss of cooling or even air movement.
Consider the scenario of a utility failure , during which the UPS continues to power the IT devices, but
mechanical plant operation is interrupted. Depending on the technology of the cooling deployed, this
interruption may continue for several minutes . During this time, elevated temperatures in the computer room
may damage IT equipment. Continuous Cooling provides the bridge to enable stable cooling to continue until
the mechanical or other cooling resumes .
Tier Requirements
Tier IV is the only Tier that requires Continuous Cooling.
Continuous Cooling for a chilled water system is generally accomplished with thermal energy storage (TES)
capability (also known as chilled water storage) . Secondary pumps and computer room air handlers (CRAHs)
are required to be on UPS. This can be the IT UPS or a separate, Concurrently Maintainable and Fault
Tolerant , mechanical UPS system. If the cooling system is in a primary-direct configuration , then the primary
pumps are required to be on UPS .
Continuous Cooling for direct exchange (OX) systems requires both the computer room air conditioners
(CRAGs) and the external condensers to be on a Concurrently Maintainable and Fault Tolerant UPS system.
The same requirement exists for split- system air conditioning units .
Continuous Cooling for 100% outside air systems that can provid e cooling throughout the year require th e
fans (or the system that delivers the air to the computer room) to be on UPS.
2
Uptime Institute'
When rotary UPS systems are deployed as the IT UPS, then the cooling system must be on the no-break
bus. This allows provision of cooling throughout a UPS ride-through event. If there is a chilled water system
deployed in tandem with a rotary UPS with no batteries, typically no TES is required . However, each specific
case should be reviewed to ensure stable cooling is provided during a loss of power event.
Regardless of any of the cooling or UPS technology deployed in a particular data center, consideration
must be given to the time required to restore mechanical cooling. For example, in a chilled water system,
although the engine generators may assume the electrical load within seconds after loss of utility, the result
of the momentary loss of power to the chillers may require a restart cycle lasting up to 15 minutes or even
more. Although manufacturers are reducing the restart times, the interval between the loss of power and a
resumption of the systems ability to produce stable cooling needs to be incorporated as a data point in the
ride-through time. For example, if a UPS ride-through time is set at 5 minutes, yet it takes 15 minutes to
resume stable mechanical cooling after a loss of power, then the TES must be able to provide 15 minutes
ofchilled water storage.
Providing thermal stability to the IT and UPS environment during the transition from utility outage to
engine-generator power, Continuous Cooling ensures that a utility event does not result in costly heat
damage to IT hardware or critical equipment. A requirement only for Tier IV, but justifiable for average
densities above 4 kW in light of potential damage to facilities and IT investment.
Accredited Tier Designer Technical Paper Series : Engine-Generator Ratings
Accredited Tier Designer Technical Paper Series : Makeup Water
Questions?
Please contact your regional representative : http :// uptlmelnstitute.com/contact-us
or email us at: lnfo @uptlmeinstitute.com .
Uptime Institute is a division of The 451 Group. a leading technology

industry analyst and data company. Uptime Institute has office
locations in the U.S.. Mexico. Costa Rica. Brazil. U K . Spain. U.A.E..
Russia. Taiwan. Singapore. and Malaysia
Visit www uptimeinstitute.com for more information.
© 20 14 Uptime lnslilute. LLC All rig hts reserved

000 18A
3
Select portions of the curriculum presentation are not provided in this booklet
due to permissions granted or withheld by companies providing actual examples
for educational purposes only. Please refrain from taking pictures or videotaping
any portion of this course without the instructor's prior written permission.
Copyright ©2009-2015 by Uptime Institute, LLC

20 West 37th Street - 6th Floor
New York, NY 10018
All rights reserved.

The Uptime Institute's (Institute) Publications are protected by international
copyright law. The Institute requires written requests at each and every occasion that
the Institute's intellectual property or portions of the Institute's intellectual property
are reproduced or used. The Institute copyright extends to all media-paper,
electronic, and video content-and includes use in other publications, internal
company distribution, company Web sites and marketing materials, and handouts for
seminars and courses. For more information, please visit
www.uptimeinstitute.com/resources to download a Copyright Reprint Permission
Request Form.
Uptin1elnstitute 20 West 37th Street. 6thFloor

New York, NY 10018
+1 206.706.4149
http://uptimeinstitute.com
s:::
c
(/,
1/,
Cl:
Cl.
Uptime Institute
Thank You
The Uptime Institute (Institute) acknowledges these

organizations for their valuable contribution to the
Accredited Tier Designer (ATD) Curriculum:
Caterpillar, Peoria, Illinois
Consulting Engineers, Inc., Tulsa, Oklahoma
Tier IV Group, Lee's Summit, Missouri
Titan Security Vault Services, Tulsa, Oklahoma
PCL Canada
eBRC, Luxembourg
Plus others wishing to remain unnamed
- - - - - - - -Uptimelnstitute.·- - - - -sessian
- -Page2- 1
Uptime Institute Tier Standard
Owners Advisory Committee Members
• Owner Advisory Committee (OAC) is a consortium
of data center owners/operators having received
Uptime Institute Tier Certifications
> OAC is a formally organized group created to validate and
endorse the contents and direction of the Tier Standards
• OAC represents global leaders in the Financial,
Healthcare, Insurance, Manufacturing, Retail, and
Government industries
• OAC members are worldwide: Australia, Brazil,
Canada, Costa Rica, India, Kingdom of Saudi
Arabia, Luxembourg, Russia, South Africa, Spain,
Switzerland, Taiwan, UAE, UK, and the U.S.
Uptime Institute Sess1on I - Page J
Tier Standards
• Tier Standard: Topology

> Defines Tier Classification System
> Performance Requirements for each functionality
objective
• Tier Standard: Operational Sustainability

> Defines Expected Operational Behaviors
> Aligned by Tier
Both Are Owner Standards

Uptimelnstitute- - - - - - - se:.~\00 1 Page 6
Distinction between ATD and ATS Course
• Accredited Tier Designer (ATD) course is for

Professional Engineers (PE) that generate designs
> Course totally dedicated to the Tier Standard: Topology
> Detailed presentation of the Institute Tier topology concepts
> "Deep dive" into proper application of Tiers
• Accredited Tier Specialist (ATS) course is for data
center owners, operators, and outside resources
> Covers both Tier Classification System and Operational
Sustainability
> Concept review of the Tier topology concepts
> Management practices critical to Operational Sustainability
- - - - - - - -Uptime Institute St>SSlon 1- Pa!jc 7
A TO Course Goals
• Enhance Design Professionals' understanding of

the practical requirements of Institute Tier-based
design
• Facilitate consistent application of Institute Tier
topology concepts
• Provide valuable education and recognition for
Industry participants in the Institute Tier Program
Uptimelnstitute- - - - - - - - sc::.slon 1 - PageS

Content and Context
• Accomplished Design Professionals and Engineers

> Tier Standard augments your knowledge for data center design
> Local codes and requirements form the foundation for design
> Examples based on minimum Tier criteria, not Best Practices
• The concept of "N"
> Definition of "N" sensitive to the context it is used
• Refers to required number of components to meet need
• Refers to capacity of system in kW when discussing the
design load or design demand
• Accreditation is not an endorsement for Certification
Uptimelnstitut · Se$mn l - Page 9
UptimeInstitute·- - - - - - - -
Tier Classification Genesis
• An Owner' s Request
Data Center Performance and Investment Criteria
• An Industry Solution
Tier Classifications Define Site Infrastructure Performance
• An International Standard
Data Center Site Infrastructure Tier Standard
-------- Uptime lnstitute~--------

1 - Page 14 s~slO/l
Tier Classification Objectives
• Provide a common understanding and language of

data center infrastructure concepts
• Identify expected data center performance by
differences in topology
> Recognize that all data centers are not alike
- nor need to be
> Refers to a single operations site
• Tier concepts are simple; application requires
extreme diligence
- - - - - - - - Uptimelnstitute:- - - - - sc~SIO/l 1 - Pa~(.' 15

Tier Topology Categories
• Tier Classifications represent broad topology

categories
> Redundant capacity components
> Redundant (diverse) distribution paths
> Classification based on Maintenance opportunity and Failure
response
• Fractional "concepts" are not rationalized
> No Standard for Tier 111.6
> Tier Ill + is undefined
> Site Classification based on lowest system rating
• Differentiation within a Tier enhances Operational
Sustainability
Se~swn 1 Pa,ge 16
Key Tier Topology Principles
• Begin and end at the IT Critical Environment

• Build upon the previous Tier
• Provide facility operation and maintenance
opportunities
• Considers only the built environment
Uptime Institute SessiOn 1 PaP,Q 17

Tier Classifications
• Tier I - Basic Capacity

• Tier II - Redundant Components
• Tier Ill- Concurrently Maintainable
> Applies to Each and Every component and path
• Tier IV- Fault Tolerant
> Considers a Single event, but Consequential impact
Uptimelnstitut · Sesswn 1 - Page IS
Tier I - Basic Capacity
• Summary
> Non-redundant capacity components ("N" only)
• Critical Environment power and cooling systems
> Single distribution path
• Operations and Maintenance Considerations
> Site infrastructure and Critical Environments must be shut down
for annual maintenance and repair work
> Installation or construction of capacity may disrupt the Critical
Environment
--------Uptime lnstitute~--------
session 1 - 19 P<~IJ<:>
..
Tier I - Operational Risks
• Any capacity component or distribution path element

failure will disrupt the Critical Environment
• All or portions of the Critical Environment are
susceptible to disruption due to planned and
unplanned activities
• Operations (Human) errors have high likelihood of
site disruption
• Deferred maintenance to avoid downtime increases
the risk and severity of disruptions in the Critical
Environment
Uptimelnstitute·- - - - - - - se~SlOn 1 - Paee 20
Tier II- Redundant Components
• Summary
> Redundant capacity components (N+R)
• Engine generators, UPS modules, IT and UPS cooling
> Single distribution path
>~capacity components can be maintained or repaired with
limited impact to the Critical Environment
> Site infrastructure and Critical Environments must be shut down
for annual maintenance and repair work
> Installation or replacement of capacity components may disrupt
the Critical Environment
Uptimelnstitut · Sesswn 1 - Paee 21

Tier II - Operational Risks
• A capacity component failure may disrupt the Critical

Environment
• A distribution path element failure will disrupt the Critical
Environment
• All or portions of the Critical Environment are susceptible
to disruption due to planned and unplanned activities
• Operations (Human) errors have high likelihood of site
disruption
• Deferred maintenance to avoid downtime increases the
risk and severity of disruptions in the Critical Environment
Uptimelnstitute- - - - - - - - s~siOn 1 Pa~c2Z
Tier Ill - Concurrently Maintainable
• Summary /
> Redundant capacity components and independent distribution
paths (transformers and transfer switches are path elements)
> Some elementes of a distribution path may be inactive
> Predicated on dual-cord IT equipment
> No runtime limits on engine-generator capacity at design load
> Each and Every capacity component and distribution path
element can be taken out of service for maintenance, repair, or
replacement without impacting the Critical Environment or IT
processes
- - - - - - - -Uptimelnstitute- - - - - - - - SC"..sionl Pi'~eB

Tier Ill- Practical Insight
• Each and Every extends to:

> Valves and fittings
> Switchgear and panels
• Maintenance focus requires:
> Dead lugs for safety during electrical activities
> Dry pipes to avoid liquid spills
• Single Points-of-Failure are not eliminated
Uptimelnstitute- - - - - - - -Se-;swn I- Page 2<1
Tier Ill - Operational Risks
• All or portions of the Critical Environment are

susceptible to disruption due to failures or
unplanned activities
• Scheduled maintenance activities occur on
redundant components, distribution paths, and
systems-which will reduce redundancy and may
elevate risk of disruption
• Operations (Human) errors may lead to site
disruption
• Single-cord IT equipment or incorrect installation
may defeat Tier Ill infrastructure
UptimeInstitute S~wnJ-PaP,c2S
Tier IV- Fault Tolerant
• Summary
> Redundant capacity components
> Redundant active distribution paths
> Compartmentalization of both capacity components and
distribution paths S , .A. _
., 't\~
> "N" after any 'failure
> Continuous Cooling for critical IT and UPS systems
> No runtime limits on engine-generator capacity at design load
> Each and Every capacity component and distribution path
element can sustain a failure, error, planned, or unplanned event
without impacting the Critical Environment or IT processes
Uptimelnstitute"- - - - - - - - - sc~s,on I Pa~e 16
Tier IV- Practical Insight
• Single event with consequential impact

> Loss of a switchboard impacts everything "downstream"
powered by that switchboard
> Replacing a valve requires a dry pipe on both sides
• Design considerations for Continuous Cooling are
consistent with UPS for IT equipment power
• Most human errors are considered failure events
> Exceptions:
• Emergency Power Off (EPO) Activations
• Fire Suppression Activations
• Failure to properly connect IT loads
- - - - - - - -Uptimelnstitute- - - - - - - - - Sc~sion 1 .,a~c 27

Tier IV - Operational Risks
• The Critical Environment is not susceptible to disruption

due to failure of any single capacity component,
distribution element, site infrastructure system, or single
human error
• Scheduled maintenance activities occur on redundant
components, elements, and systems- which may create
a risk of disruption
• Operation of the EPO system, activation of the fire
protection system, or malicious human interaction may
lead to site disruption
• Single-cord IT equipment or incorrect installation may
defeat Tier IV infrastructure
Uptime Institute:- - - - Sesswn 1 - Page 28
§ 2.5 Engine-Generator Systems
• Concurrent Maintenance and Fault Tolerance

> The engine-generator system, along with its power paths and other
supporting elements, shall meet the applicable Concurrently
Maintainable and Fault Tolerant criteria while they are canying the site
load on engine-generator power
• Manufacturers' Runtime Limits
> Manufactures' limitation on consecutive hours of operation at the design
load is not permissible for Tier Ill and IV
> Prime and Standby engine-generator ratings include runtime and load
limits
• Regulatory Runtime Limits
> Annual Authority Having Jurisdiction (AHJ) regulatory limits on operating
hours driven by emissions do not affect the capacity or redundancy of
engine generators in Tier design or Certification
Uptime Institute Sess1on 1 - Pa~e 19

§ 2.6 Ambient Temperature Design Points
• Power and cooling equipment capacities shall be

determined at extreme recorded outdoor
tern peratu res
• The ASH RAE Handbook- Fundamentals lists
appropriate outdoor design points
> The design Wet Bulb (WB) temperature shall be
"Extreme Max WB"
> The design Dry Bulb (DB) temperature shall be
"n=20 years" value
• Cooling coil capacity shall be determined at
anticipated computer room or critical environment
return air conditions
Uptime Institute S~swnl Pagc30
§ 2. 7 Communications Routing
• Communications conveyance (vaults, cable ways,

conduit) must be explicitly designed for Concurrent
Maintenance (Tier Ill) or Fault Tolerance (Tier IV)
from the property line to the data center
communication demarcation point
• Points beyond the property line are beyond control
of the owner
• Distribution within the data center is adequately
covered by other standards
Uptime Institute Sf"'.Jo<'n I Page 31

§ 2.8 Makeup Water
• Applies to direct and indirect evaporative cooling

solutions
• Must provide for 12 hours consumption from on-site
sources at "N" demand
• Makeup water system shall meet other Tier criteria
(as required for Tier II, Tier Ill, Tier IV) to the point of
consumption
• More information in Session 4 and detailed
examples address water volumes involved in
Accredited Tier Designer Technical Paper Series:
Makeup Water
Uptime Institute- Se!.Sion 1 Page 32
Public Utility Systems
• The number or lack of utility service entrances does

not affect Tier classification
• The engine-generator system is the source of power
for site availability and reliability
• The loss of any public utility (power, water, natural
gas, etc.) is not considered a failure
Uptime Institute- - - - - - Session I Palje )]

Operational Sustainability
• Complements Tier Standard: Topology

• Differentiates the value of investment within a Tier
level
• Focuses on the effectiveness of investment over the
life of the facility
• Aggregates Positive and Negative impact of design
and management choices
> Management and Operations
> Built Environment
> Site Location
- - - - - - - -Uptimelnstitute:-------- sc~sion l - Page- 34
Tier Classification Recap
• The four Tiers are progressive

> Tier I is the foundation
> Tier II includes Tier I, adds redundant components
> Etc.
• Each and Every considerations are exhaustive
• Engine-generator ratings are more restrictive for
Tier Ill and Tier IV
• Tier IV requires Continuous Cooling and
Compartmentalization
- - - - - - - - Uptimelnstitute:-------- 5€:'SSlOn 1- Page 35

Uptime Institute
Mechanical Systems
DX Split System
- - - - - - - . . . . . -Uptime Institute
Nominal Capacity: 300 GPM
N= 300 GPM
(gallons per minute)
Basic
Uptime Institute Se;s1on2- Page 3
Redundant Components: 300 GPM
J. IJ.
N= 300 GPM N + 1= 300 GPM, Redundant
.L L.L.L
N + 1= 300 GPM, Redundant
Basic Redundant
-----·-- -- - - -Uptimelnstitut · Scs~JCn 1- Page 4
Single Distribution Path
•
{A
Basic
Capacity
N= 300 GPM
•
Redundant
Components
I
:A I I'
Uptimelnstitute. - - - - session 2 - Pa~e 5
Redundant Distribution Path
Concurrently
Maintainable
_r
.A.
N= 300 GPM ~
Uptimelnstitute- - - - - - - -
.,..••
k:.~,
Redundant Distribution Path
Fault
Tolerant
au ·ton ·o·mous: (adj.) Not controlled by others or outside forces;

Independent
Uptime Institute S('~ton 2 - Page 7
Compromised Distribution Path
Basic ~ }
Capacity
N= 330 GPM
,A .A
~ ~
~
Uptimelnstitute:- - - - - - - - Se!:~IOn Z PaR" 8

Valves are a Risk
• A common "Rule of Thumb" is to avoid valves

whenever possible
> Engineers for Nuclear Power Plants and Submarines minimize
valves where possible
• Both Nuclear Plants and Subs are shut down for an
extended period on a regular basis for maintenance
(Equivalent to Tier II)
• Tier Ill and Tier IV Data Centers are NOT shut
down on a regular basis for maintenance!
Uptimelnstitute. - - - - - - Sc>slan2 Pa8"9
Dual Distribution Path
Concurrently
Maintainable
0
Fault 0
3
Tolerant "0
~
(I)
N= 300 GPM ~
~
0
"
System A System B
Uptimelnstitute'-------- s~sion2 P<'BC II

Basic Capacity
Computer Room Cooling
N= 400 kW
(6 X 70 kW= 420 kW)
- - - - - - - Uptimelnstitute-'- - - - - Sc~wnl - Pag~ 12
Redundant Components
Computer Room Cooli
f.i
DE]
...
fl
m. §
DE]
...
§
m. §
N= 400 kW
(6 CRAH Required)
f! § fi §
OQ§ . .
9 CEf CEf
Uptime Institute Se:.SIOI11 - Pa~;<.> I)
Concurrently Maintainable
Concurrently Maintainable Loop

N=400kW
- - - - - - - - Uptime Inst itute. ':.csswn 2- I!CI!JC 14
System A
Concurrently Maintainable System
N= 400 kW
System B
- - - - - - - -Uptimeinst itutA<·. . . - - . - - - - -
sessiM 2- Page 15
System A
System B
Concurrently Maintainable Dual Coil N= 400 kW
Uptimelnstitute,__..._ __
Se-.swn 2 - Pa£c 16
System A
System B
Concurrently Maintainable 4-Pipe N= 400 kW
Uptimelnstitut · Sc<..sion2- Pal!e 17

Fault Tolerant
System A
N= 400 kW
System B
- - - - - - - Uptime Institute- - SC'Ssion1 Pag~ 18
Tier IV Chilled Water Distribution
• The topology mav look like Tier Ill solutions

• Tier IV must autonomously respond to failures-
such as leaky valves or fittings-to prevent further
impact to the site
• Compartmentalization of chilled water pipes and
control elements outside the computer room is
required
- - - - - - -Uptime Institute:------~ Sess1on 2 Paga 19

i...
en
......, 0
N
c (1)
Oil
ro
(]) a...
. c I
~~~!!
• N
.
0
0
0
•
0
0
c..
c:
0
·v;
I
I
I T
I E
0
()
"'
(1)
Vl
~
-
(L) z+
..c
co E
c ........
CD
IT~I
·- I
co en
c >.
+-'
·- C/)
L~ ~
~~-;-.,.~
®(1)
.w-s
co s...... -+-J
00.
~ (L) ~
+-' ~
>.CO
~s
<D-e
s
.,.,...c
~
t: (])
::l -
- f! ~
(.)·-
..c
c
0
()
()
~ ~ ~ ~
t
... ,.£. ,.£. A
Chilled Water System
. ,. • I --·--....--
~
.. .
~
N+1 Components
UptimeInstitute Sess1on 2- Paee 20
Fault Tolerant
Chilled Water System
a
System A eE:;J
System 8 r:::=J
~-------------------------- '
N+N Components
Uptime lnstitute"!-----~---
PaEe21 s~sion2-
Glycol Systems
0
8
- - - - - - -Uptimel n s t i t u t e " - - - -2 -Pagt--22-

se~ion
Glycol System Tier Progression
Basic Capacity
N Components
- - - - - - -UptimeInstitute", Session 2 Paa;c 23

Redundant
Components
+I
+
I
I I,
N+1 Components
---------Uptimelnstitute..·~-------
Sesslon 2 Page 24 ~
• ~ ·• 1,... .., I •
Concurrently ·r ..- · ·- ·-r·-_ .
Larger Capacity
Maintainable
.r!-:r--tl .
Pumps
1 1 ~RM 1 ~ cB:I
Add Capacity N+1 Components
- - - - - - - - -Uptimelnstitute·- - - - - - - - s(";sion 2- Pagc-25

"' I •
~._.· r· - ·• ·1---• ·r·-_____J
• I•
~ ~
Fault Tolerant?
Requires:
... r .l~----·r·---·~-~
,.., • .... .. J ~.
• Compartmentalization
• Autonomous Operation N+1 Components
• Continuous Cooling
Uptimelnstitute- - - - - - - - SE'!.SIOn2 Page26
Split System CRAG Cooling
• Avoids issues of multiple valves

and piping systems
• Integrity of power sources is a
significant design and
Certification consideration
(Session 3)
• Ambient (outdoor) design
temperatures have a large affect
on capacity (Session 5)
• Continuous Cooling, required for
Tier IV, has proven difficult to
achieve
Uptimelnstitut · Scss10n 2 Pall<:> 27
Continuous Cooling
• Continuous Cooling is the capability to maintain

steady state in the critical environments during a
UPS discharge when neither utility nor engine-
generator power is available
> Computer rooms, Network rooms, UPS rooms ...
• Corollary to Uninterrupted Power for IT devices
• Continuous Cooling is required to meet Tier IV
criteria
> Part of the Uptime Institute Tier Standard
Uptimelnstitut ·
Why Continuous Cooling?
• Provides IT devices thermal stability for the duration

of UPS discharge
.....+- - - - - : -
uts occur frequently
> Chillers I CRAGS I Switchgear Relays
> Consider a UPS battery plant is often designed for 15 minutes or
more
• "Normal Sequence of Operations" can take several
minutes
> Programmed engine-generator start delay
> Start and parallel sequence
> Switchgear transfer
> Chiller or CRAC restart sequence
- - - - - -Uptimelnstitute.·- - - - - - - - - - sc~ion2 Page29
Continuous Cooling Case Study
ASHRAE 2011 Recommendations

• Recommended operating envelope for IT equipment
> Temperature
64.4°F- 80.6°F Dry Bulb
(18°C- 27"C)
> Temperature Rate-of-Rise
9oF per hour
(5°C)
> Moisture
s 60% Relative Humidity (RH) and
41 .9°F- 59°F Dew Point
(5.5°C -15°C)
Uptimelnstitute Sesion2 Pa!J~ 30
Temperature Rate-of-Rise Test
Test Protocol:
• Steady state configuration and conditions
> Load =250 cabinets at 6 kW average or 1,500 kW
> Temperature sensors indicate::; 80nF (2rC)
> Power and cooling system nominal
• Shut down all (14) CRAH units
• Record temperature rate-of-rise to gooF (32°C)
• Recover
Uptime Institute sc.,.,•on2 P"r.c 31

Temperature Rate-of-Rise Results
• First Data Point - 1 Minute

> Temperature sensor at 7 feet AFF = gooF (32°C}
> Temperature sensor at ceiling= 110°F (43°C)
> Owner aborts test
• Last Data Point- 20 Minutes
> Temperature sensor at 7 feet AFF returns to 78°F (25.SOC)
Uptimelnstitute:-------- Sesslon 2- Page 32
Tier IV Continuous Cooling
• Cooling Technology drives the solution

• Requires, as appropriate for cooling solution
> Uninterruptible power for cooling fans
> Uninterruptible power secondary chilled water pumps or glycol
system pumps
> Thermal storage or uninterruptible refrigeration
• Uninterruptible CRAC-based Continuous Cooling is
Conceivable t' •. L . u . ·)
'- a ~~~ e.• V"""'L-
---------- Uptime lnstitute~·---~ Sess1on 2- Page 33

Non-Critical Functions
• Some facility infrastructure components do not

directly impact availability of Critical Environments
• These systems may be isolated on failure or for
maintenance for extended periods
• Duration of "extended period" is equivalent to on-
site stores of engine-generator fuel and makeup
water
Uptime Institute
Common Non-Critical Functions
• Building pressurization (Makeup air systems)

• Humidification
• Fuel polishing (fuel storage filtration systems)
• Reverse-osmosis systems ( iJJ ~
• "Free cooling" or economizer systems ~
• Building automation (Tier I, Tier II, Tier Ill)
• Support area comfort HVAC
• Battery room ventilation
Uptimelnstitute- - - - - - - se~ion Z Pa!Je )5

Unintended Impacts
• Non-critical system connections to critical systems

may create non-compliant topology
• Anecdotal cautions
> Chilled water connections for building air handlers
> Free cooling heat exchanger integration
> Fuel polishing system isolation
UptimeInstitute-'- - - - - - - - sc~SIOO 2- Page 36
Mechanical System Recap
• Chilled water and glycol systems require careful

attention to piping and valve configurations
• Direct expansion (OX) systems simplify (avoid)
piping challenges, but require careful attention to
power and Continuous Cooling solutions
• Integration of non-critical systems can create
Concurrent Maintainability and Fault Tolerance
discrepancies
- - - - - - - -Uptimelnstitute,-------- 5€'~~100 Z- Page 37

Uptimelnstitute-
Critical Electrical Systems
• Critical Distribution - Computer Room Power

> UPS output to server cabinets
> Includes other Critical Environments
• UPS Systems
• Power Backbone
> Site power source to UPS input and mechanical systems
• Engine Generators
• Utility Power Source
Uptimelnstitute- - - . - - - -se-..sion
----
3- Pa~::e 2
Single Power Path
Risks of Failure
• UPS system level failure
• Major circuit breaker
• Minor circuit breaker
• Plug or receptacle
• Electrical connection
• Human error
• EPO
L--· - ·-·- ·-· • Fire
Uptime Institute Scs~wn l - Page 3
Dual Power Path
Risks of Failure
- • AlB circuiting error

• UPS step load failure
• EPO
• Fire
I
L.- . - ·- ·- ·-· L--·-·-·-·-·
Uptimelnstitut · Sc~s1on J - Pal}(' 4

Static Transfer Switches (STS)
400-V STS
• Better product than ever
• Large impact upon failure
• Input to computer
hardware is not
> Concurrently
Maintainable
> Fault Tolerant
• Ensure STS source-switch
algorithm will not transfer
a fault to the alternate
sourceJ
Uptimelnstitut · Session 3- Page 5
Point-of-Use Transfer Switch (PTS)
• Tier-compliant
- accommodation
• Provides redundancy
close to load
• Flexible solution added
or removed as required
• Risk transferred to IT
architecture
- - - - - - -.........uptime Institute- - - - - -Session

-3-- -
Page 6
Critical IT Loads
• Computer room loads provide the precedent

• Network Infrastructure I Switch Rooms
• Network Operations Center
> Consider impact to system continuous availability requirements
for IT operations
• Building Automation or Management Systems
> Essential to autonomous Tier IV system response
Uptime Instit ute
UPS Capacity -
"Need" versus Redundant
~I
E1J
Basic "m·'"
N= 400 kW
- -
rn
U__L_l__j Redundant
N=400kW
:o:t l!b !"!
mJ !Dill ~
•
liDo
'--- -
- - - - - Uptime Institute·--=== Setsion 3 P.:aec:- 8

Basic Capacity
UPS and Critical Distribution
• N= 3 UPS modules
• Single path
Uptime Institute
• Single path
• Adds redundant
UPS module
,-c:;o;;o~>;.....,.. .- ·. ·· - ··
j BwA
- - - - - - - - -UptimeInstitute· - - - - - - - - "'~) p • &P

Basic Capacity
:-··-·· -··-··-··-·· -·· -.. _. __,-p-
LV A Source
'" I
I UPS A Input Bus
:
I + + + ;:c: :
:
I • N= 3 UPS modules
I
• Single path
~L .·UPSAParalleiBus
- ·· - .. - · · - .. - ··- .. - ·
._ .L ..C ...:
j UPS A Output Bus

:I * *
i-~~i~~i 5isiributian· ·- · ·;;- ·· - ·l
··- · ·- ·
600-V
48Q.V
~ ?~? ?????! Device
-
PDUAx
c::=======:::::::::=======================-Uptime Institute · session 3 - P••• 9

LVASource ~
_ ___ ___ .. _ ___ ·- ·· 1
i -~~;; ~~~;B-~s- .. - .. v ! • N= 3 UPS modules
I
• Single path
• Adds redundant
UPS module
~
:. ___,_. __*__,_. :
UPS A Parallel Bus
L -- - ·· - ·· - ·· -· · - · ·- ·· - ··
! UPS A Output Bus

j
_ . .L .. C _;
r> ~
- .. - .. - .. - .. - .. - . ·r- .. - .. - .. - .. -
: Critical Distribution 1.., :
! Bus A ,) I 600-V
480-V
I Device
PDUAx
~::;==========:=:=:~~Uptime Institute session 3- Page 10 '

LV EGA Source
LV A Source
;- ·· - ··- ·· - ·· - ·· - ··- ·· - ·· - ·· - " - ?- ''1
I UPS A Input Bus
:
I - "'
:
I
i) ) ) ))
1 A A .:
!
• Adds second critical
ciistribution path
• )~dds means to
~L UPS A Parallel Bus f .. C _;
.. - .. - .. _ , _ .. - · · - .. _ , _ , · - rnaintain UPS
j UPS A Output Bus ~;ystem
:I * * .. LV EG B Source
:-~~;~; oo;,o;;;o" ~-;j~ ~ >->lJ E~

rl; f;
fp ~-?. ?6;.., ~~,;~~:~:
~,___________. -r-H_H_H_ - -H-H~
.r ·WJJ
-~( - ·i
! rli)l i
-
PDUAx PDUBx
~~===============:::::::==========Uptime Institute- session 3 - Page 11

Fault Tolerant
LV EG A Source LV EG B source
LV A Source LV B Source
UPS A Input Bus UPS B Input Bus
: UPS A Parallel Bus

L .. - .. - .. - . . - .. - . . - .. - . . - . ·•- . .
·- ·· - ·· - ·· - ·· - ·· ,
I UPS A Output Bus UPS B Output Bus i
i *
:-criiicai oisin"buiian· · - · ·- · · - · · - · · Critical Distribution :

! BusA 600-V
480-V
Bus B !
I Device I
~ ·· - ·· - · · - ·· - ·· - ·· - · · - ·
• Dual path
~~ ®
~u~
~~~~~~~~~~~i~e lnst~~~~~~~~se-
~o~
n3-~~12
• Adds second critical
distribution path
• Adds means to
maintain UPS
LVEGBSource - - - - ,
system
PDU M POUBx
Uptimelnstitute·- - - - - -3 - - scsSIOn Page 11
Fault Tolerant
N= 3 UPS modules
Dual path
Uptimelnstitut e'
Tier Ill and IV Shortfall -
Main-Tie-Main
!- -- >
--- --- ·-- ··- ··- ··- ·-,
> ~
UPS B Output Bus
i :
: !
L ,. _ . - ·· - ··- ·· - ·•- •· - •·.. ,;1
PDUAx POUBx
Uptim eInstitute Se-..s!on 3 Pa~e 13
Tier Ill and IV Method -

Main-Tie-Tie-Main
PDUAx PDU Bx
outo o atiDIII D Oosin o into a Dault will Cb D pro D ise uer ITil
Uptimelnstitut:e·- - - - - - - - sc~sion J Page 14

Electrical Backbone
• Includes the elements that distribute power from the

site power sources (Utility and Engine-Generator
systems) to the UPS input and mechanical and
building loads
• Tier concepts apply similarly to both medium- and
low-voltage segments
• Most discrepancies occur in Concurrent
Maintainability and Fault Tolerance of the low-
voltage segment
,- = - - - - - - - -Uptimeinstitute
Basic Capacity
Cooling Components Power
Mechanical Distribution A
~ · · - ·· - ·· - ~ · -
: Computer Room Cooling A
·· - ·· - ·· - ··-··I
I
~~
00
N= 3 Cooling Units
Uptimel nstitute:- - - - - - - s~sion3 Paoo 16
Cooling Components Power
Mechanical Distribution A
r - · · - · · - · · - · · - · · - · · - ··-·· - ··
: Computer Room Cooling A I
00
~ 00
~
N= 3 Cooling Units Redundant Capacity Components
UptimeInstitute Se~mn 3 Page 17
Concurrently Maintainable & Fault Tolerant

Cooling Com nts Power
Mechanical Distribulfon A Mechan1cal DistnbultOn 6
-··- ··-· ·- ·· - ·'

-··-··-··-··-
: Computer Room Cooling A
·· -· - ·· -· · ...., .·-··- ·· Computer Room Cooling B j
'--~~--~~~~--~
L.. ••
Tier IV rerulres autonomous

response to failure
Redundant Capacity Components Redundant Capacity Components
N= 3 Cooling Units
Uptime lnstitute~--------
scmon J- Page IS
Alternate Power
Mechanical Distnbulion A Mechanical Distribution B
~ --·· - ··-··-··-
Computer Room Cooling B j
· · -··..J
Tier IV requires autonomous

response to failure
Redundant Capacity
Component
N= 3 Cooling Units
Uptimelnstitut.e - · - - - - - - - - sc~SJOn 3- Pa~c 19

Chilled Water Plant Power
Mechanical Dislnbution A Mechanical 01slflbut1on B
Automatic Transfer Switch

Tier IV requires autonomous
response to failure
Redundant Capacity
Components
N= 3 Chillers
UptimeInstitute'
Basic Capacity
Electrical Backbone
Mech Distribution A
Main Distribution A .·-··-··-
.- .. - .. - .. - .. -.
Computer Room Cooling A 1
.-··-··-··-··-·
Chilled Water Plant A 1
.-··-··-··-··-·
House Power A 1
I.. -. ·- .. _,
"N" Engine-Generator
I j. ~~S.Inp.u~·~us ~.; =~ UPS System A H ~r~~ ~ist·B·~~ A ..~
System Capacity I
--===-=======-====Uptime Institute session 3- Page 21

Electrical Backbon1e
Mech Distribution A
Main Distribution A ··-··- ·· -
.-· · -··-· · - · ·-·
.-··-··-··-··-·
.- · ·-··-··-··-·
House Power A 1
I
··- · ·- · · .....
·- · ·-··-:,_... I · - · · - ·· -·
I ~ ~~S~p.u~ ~ ~.c:l UPS System A :i-_ ~r~t Dist.~u~ i ..
"N+" Engine-Generator
System Capacity
--------~-Uptime Institute session 3- Page 22

Electrical Backbone
Mech Dis1rilution A
Main Distribution A r- ·· -· ·- ··
r- ··-· ·, : • I ,-· ·- ··-· ·- ·· -·.
I ~--~---...;.- Cofr4)U18rRoomCooling A I
~- ·· -· · -· · - ·· -·.
1 I . ChiJied Water Plant A I
~- ·· -· · - ·· - ·· - · .
1------..;.- House Power A I
r · - ·· - ·· ~
] !
-·-··-·· --·-lr----·
~ ~~~lnpu;~sA·s
~
UPSSystemA .i....l~~~~:~~:; .i
System Capacity
.u. -· ·- ·· - ·· - ·· - ·
1'~ PI/I) VI )t ...,,_, •• 1--..o----~·- House Power 8 I
.... ~· ~- -· - ·· - ·· - ·· - ··
Pv- ft·c:.- ltt!' l--+----~·- Chilled Water Plant 8 I
~ 11 .,):, 't)e.., ~ '-· ·-··-· ·- ··- ··
~-v~~al~~ I._ .. _
1-..;....---~
· - ~Room eoonng 8 I
,, J
• I '- ·· - ··- ·· - ·· - ··
fo k_ J..o-v, r~s Main Dislnbution B L. . . - ·· -
Mach Dls1rtbu!lon B
··
~ cll'\11:,
-====================================::~Uptime Institute session 3 - Page n
Fault Tolerant
Electrical Backbon1e
Mech Dis1rtlutioo A
Mai" Distribution A .-··- ··- ··
r - ·· - ·· , : • I ~- ·· - · ·- · · - ·· - · .
I 1-----~.- ~RoomCoortng A 1
~- ·· - ·· - ·· - ·· - ·.
1-+------i-.- Chilled Water Plant A I
~- · · - ·· - ·· -· · - · .
1------.;..- House Power A I
_ .. _ .. _ _ 1.. - ...- .. :
~- ·· - · ·-·........: . ~-··-··-·.
1-~----;.: t::::J
UPS Input Bus A UPS System A ~ Crit Dist Bus A 1
- ·· -··- ·· - · · - ·· - ··
i.. .. _ .. _ .. l
®~
System Capacity ®
® :· 1.. - • . - .. ~
t--; - - - - - .·- U·P~ ~·a:~

1- · ·-··-·
5. UPS System B g-~ ~:~~; ·~
_ .. _ .. _ _ i.. .. _ .. _ .. :l ·-··-··- · ·
.u.
•• }-...;,..---~·- House Power B I
~- ·· - ·· -··- · ·- ··
I
0
·-
1--+----~ Chilled Water Plant B I
'-· ·- ·· - ·· -· ·- ··
1-......----:-·- CoJl11111Br Room Cooling B I
f._. , _ .. J
Main Distribution B .:. . .. _ _..
•..
Mech Dlstrhnlon B
I '-· ·-· ·- ·· -··- ··
c=::========.:Uptimelnstitute ---..Em
session 3 - Page 24
Minimum Fault Tolerant
Electrical System
Mech Distribution A
Main Distribution A
House Power A 1
'· ·- ·· - ·· -'
· . -U~~~s;e·m-A. '1
1-------....&.- ; - - ~ritica; ~istrib~t;o~B~: -; ·~
Side "An
---- -- u
Side "Bn
. . _u~~ Sys~e.m_s, ) Critical Distribution Bus B 1
,. ._.. _.. ..,

House Power B I
Computer Room Cooling B I

Main Distribution 8
Mech Distribution B
~=======~Uptiine Institute:--.-::::::::::::::~::=::::::::::::==:::::::=
Session 3 - Page 25
Basic Capacity
Electrical Backbone
Main Distribution A .·- ·· - ·· -·.

1-CI-~--·r-
·- ··Computer
- ·· -Room·· Cooling
- ·· -A·j
11-[}-j----j-
·- ·· Chlfted
- ·· Vllaltr
- ··-Plari·· - ·j
A.
·- ··- ·· -~
11-[}-L----j'-
·· - ·· - ·i tfQlM A
i ·- ·· - ·· --:
1-C~---r_·· -c;S·I~·~::E:i
., _ .. _ .. :
UPSS)"tltmA H ~~~ ;-i
I _ , _ ., _ _,
~w Engine-Genera lor
System Capacity
- ·· - ·· - ··
Uptimelnstitute·- - - - - -3-- - se~sion Page Zl
Redundant Comp<?nents
Electrical Backbone
Mtc:ll Olllb1butlon A
Main Distribution A
tcl-i---r-
·
·-· ·- ··- ·· - ··- ·j
Ch.ledW1\erPanl A
tQ....I----ir-
·- ·· - ·· - ·· - ·· - ·
··- ·· - -· - :
1
1-C~---t: -;;~;.;..·.;..-:.~ .....
ytWM A H~~;,.-;-.~;i
_ .. _ .. _ , :.. .. _ .. _ ..1 _ .. _ .. _ ,
:... .. - .. - ·
"N+ ~ Engine-Generator
System Capactty
Uptime Institute Se~sion 1 Pn!lr 77

Electrical Backbone
Syslem Capacity
Uptimelnstitut · S~s•on J- PagC' B
Fault Tolerant
Electrical Backbone
Syslem Capacity
Uptimelnstitut S~smnJ- Pag ~Z ·I

Minimum Fault Tolerant
Electrical System
.~~":
j i
=-~~~~!
:-. ·~:.;..·~~· ':- ;
10-T-----i-
-··- ··-··---·-··
!; __ _ __ _ __ .j
Side "A"
Side "B'
10-i---+-
- ·· - -· - ·· - ··- ··I
Hc>.JHPwW •
- ····--··-
··- ····--··-
··-···
I
j 1-D-+---+-
~ ..~~:~.- ~ ~_
_... ..... .. .... .. ,.... ,. _ _
I to-"---+ ~Rcx:n~B
I : _ .. _ , _ , _ ,, _ ,I
;;;~·
--·
Uptimelnstitut ·
§ 2.5 Engine-Generator Systems
• Consider engine generators as site power source

> Tier Ill requires Concurrent Maintenance while running
> Tier IV requires Fault Tolerance while running
• Manufacturers' Engine Ratings
> Tier Ill and Tier IV engine-generator capacity is based on
manufacturers' unlimited runtime capacity at site conditions
> Standby and non-de-rated Prime-rated units are acceptable for
Tier I and Tier II using the capacity at site conditions
• Runtime Limits
> Manufacturers' runtimes represent capability constraints
> EPA runtimes represent regulatory constraints
- - - - - - - -Uptime Institute
Definitions
• Definition of an Engine Generator

> A generating set consists of one or more Reciprocating Internal
Combustion (RIC) engines used to produce mechanical energy
and one or more generators to convert the mechanical energy
into electrical energy . The generating set includes any
components used for coupling the mechanical prime mover(s)
and electrical generator(s) (e.g., couplings, gearbox) and, where
applicable, any load-bearing and mounting components.
-ISO 8528-1, 2005
Uptime Institute- - - - - - - - sessiOn J - Pa!:e 27
Assumptions
• Applies only to Engine Generators as defined

• Purposes of an Engine-Generator Plant
> Provide reliable power for data centers
> Provide power when utility is not available
> Provide power during maintenance activities
> Provide power after infrastructure failure
• Loss of utility is not considered a failure but is a normal
operational condition
• Engine-Generator Plant could be the ONLY source
of power
• Capacity is a separate discussion from
environmental permitting constraints or emissions
----
ratings
- - -UptimeInstitute. - - - - - S~s1on 3 - Page 28
Engine-Generator Ratings
• International Standards Organization (ISO) 8528-1

is the governing document
• 4 rating classifications
> Emergency Standby
> Limited-Time Running
> Prime
> Continuous
• Major differences in operating hours and power
output capacities
Uptime Institute sc~sion J- Page 29
Standby Rating
• Definition
> The maximum power available during a variable electrical power
sequence, under the stated operating conditions, for which a
generating set is capable of delivering in the event of a utility
power outage or under test conditions for up to 200 CDurs of
operation per year -1so B52B-1
• Caveat for manufacturer's maintenance intervals
• But
> The permissible average ROWer output over 24 hours of
operation cannot exceed of the standby rating unless
otve~Vise agreed by t ~ manufacturer - 1so a52B-1
Uptimelnstitute·- - - - = = = - - - sessinn 1 P>1~P 1n

Prime Rating
• Definition
> The maximum power which a generating set is capable of
delivering continuously while supplying a variable electrical load
when operated for an unlimited number of hours per year -/so 8528-1
• But
> The permissible average power output over 24 hours of
operation cannot exceed 70% of the Prime rating unless
otherwise agreed by the manufacturer -1so 8528-1
Uptime Institute Session 3 - Pagf.;' 31
Continuous Rating
• Definition
> The maximum power which the generating set is capable of
delivering continuously while supplying a constant electrical load
when operated for an unlimited number of hours per year -1so 8528- 1
Uptime Institute. Se5sion 3 - PIIIJ~ 32

Manufacturer's Engine Generator
Rating Definitions
• ISO 8528 definitions are the default ratings for Tier

Certifications
• Manufacturers may attest that their units exceed the
ISO 8528 definitions for:
> Average power capability - higher than 70% for Standby and
Prime units
> Unlimited runtime- Standby units without hours limitations
• Manufacturers are moving to specific definitions for
data center applications
• Must utilize the site conditions including altitude and
ambient conditions
Uptime Institute S~s1 on 3 - PagC' H
Engine-Generator System Topology
• Tier I requires an engine-generator system providing

adequate capacity (N) for critical load
> At least one engine generator or equivalent
> No specific technology mandated
• Tier II requires at least N+1 capacity components
> At least two engine generators or equivalent
Uptime Institute
Engine-Generator Concepts
N= 3 Engine Generators
[Q]
Engine-Generator Engine-Generator
Paralleling Bus A Utility Bus _P~r.a~ng Bus B
1· · - · · 1· · - ··
.••
· · - · · ..J.. . . J L.- • •· - · · - · ·
.r---- :•• _________.

I
.··--------------------------------------------------------:
I I
.
r·-· ~
I
: -----------·--------·····--·-··-·····-······· ··-···•
I
:
I
I
I
I
Main Distribution A I I Main Distribution B
1··-· · ... - · · . - .}-
•
.. - . I .. - .. i-
I
UPS Input Bus A Mech Distribution Bus A Mech Distribution Bus B
c:=:=========:---;Uptime Institute session 3 ~Page 35

Engine-Generator Concepts (Alternate)
,··-·
Paralleling Bus A Utility Bus Paralleling Bus B
.. -.'1' . . J L- . . - I .. - • . I . . '1.- ..- ..

- .
'' ''
r··-- ': ------------------------------ - ~ ------------------ -•'
' ''
:' ·---------------------------------------------------------;
,··-··
Main Distribution A . '
''
··-···- · } - ·· -.
'
''
, -· - ··:--'
. · · - ·· · - ·
Main Distribution B
UPS Input Bus A Mech Distribution Bus A Mech Distribution Bus B
c::=::==================================--Uptillle Institute Session 3 - Page 36

Fault Tolerant
N= 3 Engine Generators Tier IV reouires autonomous
response to failure
Paralleling Bus A Utility Bus Paralleling Bus B
1 .. _,
.. - .. ~I .. J '-- . • I - •. - ••
I
I
I
I
~------- : ---------------J
I
..: .~---------------------------------------------------------:.
I I
r···· : ···---------------------------------------------···•
I I
• •
Main Distribution A
1··-·· ·· -
.
· i -··- , . . - . . :.- .. _ , ... _
:
_
Main Distribution B
UPS Input Bus A Mech Distribution Bus A
~===================Upti~ne lnstitute--=================
Session 3 - Page 37
Engine-Generator
.Pa~!~g Bus B
Main DlstrbuUon A
r - ··
t,,'l l..... a...A
- - - - - - - -Uptime Institute__._ __ SC!>SIOn)- PaEC 35
Engine-Generator Concepts (Alternate)
Uptime Institute-=== Sc<»ion l - Paee 36

Fault Tolerant
N= 3 Engine Generators Tier IV reCllires autonomous
response to failure
Englne-Genera\OT Engine-Gene111\or
PllliliUellf~!'. .~!i~BusB
Uptimelnstitute- - - Se-ssion) Page-37
Utility Power Source
• Provides an economical alternative to on-site power

generation
• Dual utility power solutions are not required to meet
criteria for any Tier
• Due diligence is required to ensure utility
transformer and bus configuration do not
compromise:
> Dual path requirements of Tier Ill
> Compartmentalization requirements of Tier IV
- - - - - - - -UptimeInstitute Se~wn 3 Page-38

Electrical System Recap
• Tier progression- Basic Capacity, Redundant

Components, Concurrent Maintenance, Fault
Tolerance
• Begin power path design in the computer room and
continue to the site power source, including Each
and Every component and element
• Electrical topology can defeat mechanical topoiogy
- - - - - - - -Uptimelnstitute- - - - - - - - SQ'OSIOA 3 Pa£e> 39

Uptime Institute'
Session 4 Overview
> Tier IV requirement
• Communications Conveyance
• Ancillary Systems
> Engine-generator fuel system
> Building automation
> Water sources
> Fire protection including EPO
- - - - - - - - -Uptimelnstitute.·- - - - -sessJon
-- -
Pa!Jc 3
-l
Other Ancillary Systems
• No Tier-level specific or Certification criteria

> Building pressurization (Makeup air systems)
> Battery room ventilation
> Reverse-osmosis systems
> "Free-Cooling" or economizer systems
> Lightning protection
> Grounding
> Load bank
• Integrate carefully!
Uptime Institute
§ 2.4.1.c) Compartmentalization
• Applies to complementary systems and distribution

paths in Tier IV topology
• Tier IV requires physical isolation to prevent a single
event from simultaneously impacting more than the
number of redundant components or systems
• Each compartment shall contain no more than the
number of redundant components
Equipment Compartmentalization
Secondary Chilled Waler Pumps N=3 (R=1)
Chilled Waler Machines and Primary Pumps N=4 (R=2)
Not Compliant with Tier IV requirements

Uptimelnstitute' ---- sc~lon 4 - Page b
Effective Compartmentalization
Secondary Chilled Water Pumps N=3 (R=1) Chilled Water Macllines and Primary Pumps N=4 (R=2)
Uptime Institute' - -
Electrical Compartmentalization
Switchboard A Switchboard 8
Not Compliant with Tier IV requirements
Uptime Institute Se;~wn -1 Pa~e 8
Electrical Compartmentalization
.Al [u U ·- 1 -A M ((· U-t " e. A

I o ernllllD I[I]]aJO'DDJJJ I I o e iTJIIDDI[I]]DDD'DIIIlD I
~&c..l~ ·- l(.l jo(~v/-4 ,- 2-(J.
I o e ITIIIl DI [I]]OOO'DIIIlD I I o ernllllD I[][ll]DO'OJIO I
Upti.melnstitut•e.·- - - -
,~
§ 2.7 Communication Routing
• Applies to communications connections from site

property line to data center communications point of
demarcation, including intermediate vaults, rooms,
and closets
• Tier Ill requires diverse conveyances, but may share
path
• Tier IV requires diverse conveyances and physical
isolation to prevent a single event from
simultaneously impacting more than the number of
redundant paths
- - - - - - - -Uptimelnstitute-"....____ _ Se-ssion 4- Page 10
Site Communications Path

I
•I -- - - --
Tier IV requirements
.9:-/currently Maintainable
·aths required for Tier Ill
- - - - - - - - -Uptimelnstitute·- - - - - - - Se<.sfon4 - P t~ge II

• . p ·cu rrently Maintainable

' ths required for Tier iii
,
r--,--~··
UptimeInstitute 5ession4 Page 12
-1--- -- --------·- - -- --r-r-"'"T

'
· itlii Tier IV requirements
rrently Maintainable
required for Tier Ill
• Common vault is not Fault
! Tolerant
t=
- - - - - - - -Uptimel n s t i t u t e - - - - -" -r.-agc-- -
se~~•oo 13
-1 - -- -- - ·- --
I
POP
• · p currently Maintainable
ths required for Tier Ill
·• Common vault is not Fault
Tolerant
r------- e-:;~,._-r- • Compartmentalized path
required for Tier IV
Uptimelnstitute:- - - - - - - - sess!on 4 - Page 14
Engine-Generator Fuel System
• Storage Tanks
• Piping Systems
• Pumps
• Day Tanks
• Controls
• Filtration
-------~Uptime lnstitute~--------
sessli'ln 4 Page- IS
Fuel System Tier Progression
• Tier I- 12 hours fuel storage for engine generator(s)

• Tier II -Redundant tanks and pumps for 12-hour
supply (at least 2)
• Tier Ill- Concurrently Maintainable 12-hour fuel
supply system to "N" engine generators while in
operation
• Tier IV - Autonomous control response to
component or path failure
Uptime Institute-----~ S~slol'\4 -Pasel6
Fuel System Tier Criteria
Bulk Fuel Storage -

No Topology Requirements
Tier 1-
Total of 12 Hours Fuel
Tier 11-
Redundant 12 Hours Fuel
-"""""""'
Fuel Supply
L-- -------- - ---------------· - - -!~::'O.,::::<::::::,:::,::::::::.:::::.<:,:::·::::.::::,::::=::::c:::,.:-:!

Tier Ill and Tier IV -
Other Fuel System Requirements
Uptimelnstitute·- - - . - - - - -r"e"
-- c;,.~~;n.," 17
Fuel System Consideration
Fuel Storage Tank 1A Fuel Storage Tank 18
• A single equalization valve is not Concurrently Maintainable

nor Fault Tolerant
• Use double valves for Concurrent Maintainability
Uptimelnstitute Session4- Page IB
Fuel System Consideration
Fuel Storage Tank 1A Fuel Storage Tank 18
• A single equalization valve is not Concurrently Maintainable

nor Fault Tolerant
• Use double valves for Concurrent Maintainability
• Provide Compartmentalization for Fault Tolerance
Manually operated "Normally Closed" valves avoid Fault 1.7
Tolerant automation requirements
- - - - - - - - -Uptimelnstitute- - - -.........- - - se~wn4- Page 19
Fuel Polishing Systems
• Fuel polishing (filtration and bio-treatment) is not

required for any Tier
> Fuel polishing is an Operational Sustainability issue
• Fuel polishing must be carefully integrated to avoid
compromising Tier Ill or Tier IV criteria
--------Uptimelnstitut:e·-----~4 Pa!ie 20 Sc~~1on
Fuel Polishing Configuration
• This configuration is limited to

supplying no more than the
redundant number of engine-
generator subsystems
Fuel Storage Tank
Uptimelnstitute, - - - - - ,,..~~·nn 4 -· P>~£:<> 21

Fuel Polishing Configuration
• This fuel filtration configuration

can be isolated without affecting
fuel supply to engine generators
Fuel Storage Tank
semon4 Pag"'22
Building Automation
• Includes:
> Supervisory Control and Data Acquisition (SCADA)
> Plant controls (BAS) (BMS)
> Emergency Power Off (EPO)
• Principal Tier IV Consideration
> Tier IV requires autonomous response to failure
Uptimelnstitute- - - - -Se!.Sion
-4 - -
Pagc-23
Tier IV Autonomous Response
• Operator intervention shall not be required to respond

to single system failure
• Control system failure shall not disrupt Critical
Environment
> Critical Environment must remain stable with failed control
system
• Tier IV data center facility infrastructure control
systems shall:
> Detect system failure
> Isolate and contain failure
> Sustain "N" capacity after failure of any component or path
Uptimelnstitut · 5~•on 4- Pag~ 24
Electrical System Control Considerations
• No special consideration for Tier I or Tier II

• Control maintenance or a single failure shall not
disrupt subordinate control systems (e.g., chiller
plant control)
• Control systems must comply with Tier criteria while
engine generators are supporting the site load
> Concurrently Maintainable for Tier Ill
> Fault Tolerant for Tier IV
> A single failure (including software instance) shall not prevent
autonomous engine-generator start/operation in Tier IV sites
- - - - - - - -UptimeI n s t i t u t e - - - - - -) - - <;,..-.~....,."
Plant and Temperature Controls
• Common names
> Building Automation System (BAS)
> Building Management Systems (BMS)
> Direct Digital Controls (DOC)
• Scope of system
> "Head-end" server
> Communications (which may include site network)
> Building node control panel, local control panel
> Sensor, actuator, input/output cards, power supplies
Uptimel n s t i t u t e - - - - - - - -
SI'~sion 4- Pi~§~ 26
Plant and Temperature Controls
• No special controls requirements for Tier I and Tier II

• Tier Ill requires that Critical Environment must remain stable
with any control element out of service for calibration, repair,
or replacement on a scheduled basis
• Tier IV data center facility infrastructure control systems shall:
> Detect system failure
> Isolate and contain failure
> Sustain "N" capacity after failure of any component or path
Uptimelnstitute:- - - - - - - S~Ion·l Pai!C27

Active Control Systems
• Active: Requires continuous modulation and

feedback to manage system
• Will fail
• Control elements that must change state in
response to failure must be physically independent
of failed component
> Power source
> Control cabinet or node
> Sensors
Uptime Institute Se5SIOn4 - rage 28
Active Control Impact
• Design represents traditional lfri!

Primary - Secondary loops
• "Normal" operation uses
I fri"l
valve "A" to control I fri1-
temperature in secondary
loop
IE51
• Variations in primary loop
pressure adversely impact
secondary loop temperature
stability
• "Abnormal" operation uses
valve "B" to control loop
temperature
• Control failure drives system
to full-cold-and loss of
sensible capacity
Uptimelnstitut:e·- - - - - - - - -
<;c> ....inn •I - ~-"eP 1Q
§ 2.8 Makeup Water System
• Backup on-site makeup water storage is required for

sites using evaporative cooling
• The makeup water system must meet applicable
topology criteria by Tier to point of delivery for a
minimum duration of 12 hours
> Identical to minimum fuel system storage requirement
• Technical Series Paper provides additional
information about typical water volumes
Uptimelnstitute-·- - - - - - - - se~510n 'I Page 30
Makeup Water Tier Criteria
Makeup water Slaraga
I
I
Tier Ill and Tier IV- I Bulk Water Storage-
. I
Other Condenser System Requirements 1 No Topology Requirements
- - - - - - - -UptimeInstitute- - - - SJco~1on 4- Page 31
Fire Protection Considerations
• Gaseous fire suppression is permissible but not

required by any Institute Tier Classification level
• Integration of fire detection system into other facility
infrastructure systems shall be
> Concurrently Maintainable (Tier Ill) or
> Fault Tolerant (Tier IV)
• Combined fire detection and EPO systems are
suspect
> e :.L w r i_ ~ us iii iiir:Jr _
d-e) ra..J-·..s ~ 0 p-t.J,,· 'D'1....) s~.~ .s +~~: l,r~d t -Jf.,
- - - - - - - -Uptimelnstitute- - - - - - - - sc~sion 4- P<~gc 32
EPO Considerations
• The Tier Standard does not require an EPO if not

required by AHJ
• Standing-voltage EPO systems have not been
shown to be compliant with Tier Ill or Tier IV
> Often integrated into distributed components
> Centralized Applied Voltage systems are effective
• The EPO system shall be
> Concurrently Maintainable for Tier Ill
> Fault Tolerant for Tier IV
-1 rase 33
St>$~10n
Ancillary Systems Recap
• Compartmentalization is Required for Tier IV

• Critical ancillary systems must meet same criteria as
power and cooling system at each Tier level
> Tier II - Redundant Components
> Tier Ill- Concurrently Maintainable
> Tier IV- Fault Tolerant
• Reminder: Some ancillary systems are not critical
but may inadvertently compromise Tier
Uptimelnstitute- - - - - - - - 35-
sc~510n 4 P~JIC'
Uptime Institute~--------
Common Tier Design Discrepancies
• Design point for capacity calculations

• Concurrently Maintainable and Fault Tolerant power
for mechanical systems
• Valve placement consistent with redundancy
- - - - - - - -Uptimelnstitute:- - - - - - - -
S~ion 5- Pagel
Design Point: Institute Tier Standard
§2.6 Ambient Temperature Design Points

"The effective capacity for data center facilities infrastructure
equipment shall be determined at the peak demand conditions
based on the climatological region and steady state operating
set points for the data center. All manufacturers' equipment
capacities shall be adjusted to reflect the extreme observed
temperatures and altitude at which the equipment will operate
to support the data center."
UptimeInstitute Session 5 Paljc 8
Design Point vs. Owners Objectives
Owners' initial expectations

• No water in the computer room
• Small infrastructure footprint
• Sustainable refrigerant
• 75°F (24 oc) return air temperature at 50% RH
Supports owners' planned 500-kW server room
- - - - - - - -Uptimelnstitute- - - - - - S- sossmn Pago 9

Design Point:
Manufacturer's Product Data
• Design Objective: 30-ton, air-cooled, DX unit
> Equivalent to 105 kW
• Largest capacity range of a popular provider

> Nominal Total capacity 104 kW- ~sible capacity 83 kW
• 75°F (24°C) DB return air at 50% RH '
> R22 refrigerant
> Semi-hermetic compressor
> Water-cooled compressor (120 GPM@ 85°F (29.4°C) EWT)
> "Capacity data is factory cerlified to be within 5% tolerance."
> Alternative Model Total capacity 101 kW- Sensible capacity 89 kW
• 75°F (24°C) DB at 45% RH
• 105-kW DX unit yields less than 85 kW useful cooling
Uptime Institute---==== Sccsion5 P<~gciO
Capacity Determination
• Capacity at ASH RAE rating criteria must be adjusted for

the project
--....--Uptimelnstitute..~-~~~---
.n ) }of'C.I~ p .,
ASHRAE Design Conditions
• Reference: 2009 ASH RAE Handboo k~dc:Vn entals

(Updated and published every 4 years)
~\S~~~ ,'~ CE~YW~~11' P· ~~ f}B~~ 6§ ~ arl~

Extreme Annual
Monthly Design Dry Bulb Temperatures Profile (July)
Design Conditions
5% 2% 0.4% N=20 years
91.4nF 94ZF 97.4"F 103SF
!~fDP~ \ ~ C) II 'I rf, I[Ciftftr l7i:.:>. 11111 1 D

\
Unlikely
30 hr 15 hr 3 hr
(20-year period)
-------~ Uptimelnstitute Se~swn 5 Page 12
ASHRAE Design Conditions
• Reference: 2009 ASH RAE Handbook-Dc:ndaD entals

(Updated and published every 4 years)
DODD DO DIIIIIJD DDDD[]]][]DD DIJDDDDDDD DDDDDDDDD DIJDCIIJ DDDDDDDDD
Extreme Annual
Monthly Design Dry Bulb Temperatures Profile (July)
Design Conditions
5% ; 2% 0.4% N=20 years
33°C 34.6°C 36.4°C 39.rc
DIJDD[ ICIJffiiJD JDDD DITIIIJDD

Unlikely
30 hr 15 hr 3 hr
(20-year period)
;"'-"\()n P'lg<:' I "3 ~
Capacity Validation
• Manufacturer's data for 30-ton nominal unit, R407

refrigerant, manufacturer's recommended air-cooled
refrigerant condenser
Outside DB Inside DB Sensible Sensible
Altitude Inside RH
Temperature Temperature Capacity (kW) Capacity (ton)
95•F (35.C) Sea Level so·F (26.7"C) 40% 93.7 26.6
• Manufacturer's data for N=20 Extreme Design

Condition, adjusted for altitude (above 5,000 feet),
using owners' set points
Outside DB Inside DB Sensible Sensible
Altitude Inside RH
Temperature Temperature Capacity (kW) Capacity (ton)
1o4· F (4o· cl 5,000 feet 75•F (24.C) 50% 64.9 18.5
Uptimelnstitute·- - - - - - - - sc·:lon:i l'tlP,CI·I
Design Point:
500-kW Computer Room Demand
• Standard Capacity- N=6 (562 kW) Redundant= 2

1 93.7kW I 1 93.7kW I 1 93.7kw l 1 93.7kW I
1 93.7kW I 1 93.7kW I 1 93.7kW I 1 93.7kW I
~\q -5, }
• Ambient Conditions - N=8 ( kW) Redundant = 0
1 64.9 kW I 1 64.9kW I 1 64.9 kW I 1 64.9 kW I
1 64.9kW I 1 64.9 kW I 1 64.9 kW I 1 64.9 kW I
(~)
~ o(ju~ ~ ~·cd Tcrf ~
Uptime Institute. Sc~~1on S P<~l)e I~

Design Point:
500-kW Computer Room Demand
• Standard Capacity - N=6 (562 kW) Redundant = 2

l e3.7kW I l e3.7kW I l e3.7kW I l e3.7kW I
l e3.7kW I l s3.7kW I 1 93.7 kW I l e3.7kW I
• 95% of Baseline- N=8+ (9J!G kW) Redundant= 0

l s1 .6kW I l s1 .6kW I l s1.6kW l l s1.6kW I
l s1 .6kw l l s1 .6kw l j s1.6kW l l s1 .skw l
• OOIIIJODIJO DIJOOO OIIIJOOOOOO[[[][l]OO DOD CDOIIIlDliiiO 0
0 OIIIJlliJDDrnJD 1IIJ ODD DDNominal DOD kW C3 0 DIJOOO
Uptimelnstitute.·- - - - - - - sc~SIOO ~ P<~se 16
Mechanical System Power
oocnn!IIIlrn JLLJc
-~ DDn lnrJDO[]]]JDJID 0 rnDI"IliJDDUIJ
UptimelnstitutP.·- -
Mechanical System Power
Sub 7 Line 1)
,__ [_______________
1__ /\
(---P-u-bli~Utiirty Sub 9 Line 4
Not l _JJ -roLNJ
D DOtiED[[[]DJ[[IJ[]]O
r~-~T --- -------------;---: r---;---------- 1
----- T~ -§ 1\J - --- /JcW}J~
1
» rnTTl D DITDDIIIIJDDITD
0 II II II '
l~~l__~_____ i_ ___} __ j L-~-----~--~---l~-~J
r::: '"' '~~-~ §!~-- -:

z:
I ::tl "< I
' N O'
I
l
I
l_?l____ ----- -----lL-_____._:- - -
~- -~,-----------------:
i--------------,--.. -,
~ 5: :
I'"'o
~ O :' UPS A Cooling See E203 ,
: ::tl
(')O :
I to:~ __:_________________I
I : UPS 8 Cooling See E204 : S::
~--------------. I~
iiJ?s;a-s;;e;~-1
~-=~-~:e_E402 : L:_cu I I
I<_ _ _ _ _ _ ~ ~~
I _ __
_ _ _ _ _ _ _._
I
L--------------J
: ~-~ r---- 5 ~ .p,.,._ tXt. ~~·\\IMJ t'\&" -C:f:h·cM
· ~o · --~~~r-r-r-r-r-~ ,r\v.. Q#.c ~ ~ VtJ"'IJ-~ ik~o~-
l~_~j__ __j __ ? ? ? ?. ?. ? ? ?
D{
- - - - - - - - - -Uptime Institute sessions- Page 11

Tell-Tale Panel Schedules
Panel UMHVAC2A Panel URHVAC14 Panel UMHVAC2B

Rating: 4aOY/277 3P 4W 600A 42 KAIC Rating: 480Y/277 3P 4W 400A 42 KAIC Rating: 480Y /277 3P 4W 600A 42 KAIC
Located: Mechanical Mezzanine Located: Roof Penthouse Located: Mechanical Mezzanine
Fed From: Fed From: Fed From:
Remarks: GND Bus Remarks: GND Bus Remarks: GND Bus
1 2 1 2 1 2
CRCU 01 3 4 CRCU 03 ACC 01 3 4 ACC02 CRCU 02 3 4 CRCU 04

5 6 5 6 5 6
7 8 7 8 7 8
CRCU 05 9 10 CRCU 07 ACC03 9 10 ACC04 CRCU 06 9 10 CRCU 08

12 12 12
" " "
13 14 13 14 13 14
CRCU 09 CRCU 11 CRCU10 CRCU12

15 16 ACC05 15 16 ACC06 15 16
(Future) (Future) (Future) (Future)
17 18 17 18 17 18
19 20 19 20 19 20
ACC09 ACC 11 ACC10 ACC12

21 22 ACC07 21 22 ACC08 21 22
(Future) (Future) (Future) (Future)
24 24
23
"4 23 23 24
~o\J~
• Current Tier functionality limited by Penthouse Panel
• Future plan impacted as well )r
{'\ D
,,el' p. ~
"\ \ e,.l
tl ef'"
c:::=:====::=::::::::=====::::::::::::::::::::::::::::::~~uptillle lnstitute- - __,n

Session 5 - Page 18
Piping Topology
=-.:r;
@
D
D
D
D
D
CT-2 CT-4 CT-5{futur~)
.T.
=
.r··- r -- ' . G~ 1.-,·M-
L-------~----~ OOQCTP ~
~
~1-3
.T
I I eo.-uA .s,...;. =
: !"~
n
J-
1
I.
1 ~ ~ Not Concurrently Maintainable nor Fault Tolerant

1 1....-J
41~
I _____________________,
(.C'~· ~-
'- ·· -----~--------------------------
· · - _.., ooooooo----------------------------~.---
co V\J. Rftl.
c=::========~Uptime lnstitute ~---:~:::'i"~--t:,
sessions- Page 19
Piping Topology
jo./-+ L
CRCU- 18 CRCU _OS CRCU- 16 ..,.) c:.. 0. Jc.D\
CRCU- 07 r- r- r--
.
.
--.r-..J-L....T---- o.. . ,. , ,.,,..
~ ··J'
.-- (Future)
I ~ '
-
I
(Future)
8 owo - - ~ i)J~c#
\~~
>tv-~~
8CJYJt ~ '~ 8 DW O - - ~ ~f~
CRCU- 19
(Future)
-
-
i
CRCU- 06 1---
r-----
I I
~ -7
1 1 CRCU- 17
(Future)
t-
r-----
I CRCU- 04
'-------
Co mputer Room
SARoo~--- ----- ------- -,- --- --- --------UPSBRoo~
u
o o I [] o
r:J
~
0
~ I ~
0 Cl
~ v
ucu- 27 ucu- 29
• Initial condition requires 4 redundant ComputE~r Room Cooling Units

• Future condition requires 8 redundant units (f'J+8!)
• UPS cooling is not Concurrently Maintainable nor Fault Tolerant
, .- -~c:~Uptime Institute~ -- session s- Page zo

Valve Placement Guidelines
N+1 Air Cooled Chillers N+1 Secondary ,...------, N+2 CRAH
Pumps
~ I' , , , , ,, ,,r•r
~ !' V// / / / / / 41
~
• N+1 devices in looped systems shall be separated by at least 2 valves
• N+2 devices in looped systems shall be separated by at least 1 valve
• Tier IV requires autonomous valve operation
t::::=:======::::::==='!Uptillle lnstitute.~·~~~====
Session 5- Page 21
Tell-Tale Panel Schedules
F«<FI'CO'I\:
Panel UMHVAC2A
~::.:::::::21(.\~
fl......,.. GHoe..
CRCU01 CRCUOJ
_ ACC01
Panel URHVAC14
......
:,~.:z:~.ooA.21WC
F..,F!Onl:
ACC02
FlldF"'""
RlnlaiiUoGND-
CRCU02
Panel UMHVAC2B
~~::::::=-n:2 KAJC
CRCU04
CRCU 0~ CRCU07 ACC03 ACC04 CRCUOG CRCUOB
CRCU09
(Fulure) .._,
C RCU ·,1
ACCO!i
••
ACC06
CR C U 10
iFt.illil•l
CRCU 12
(Ful ure)
ACC09 ACC •t ACC08

ACC 10 ACC ' 1
ACC07
jF.-U!t) 4f .cat FIAnl JF..Il'6~
• Current Tier functionality limited by Penthouse Panel

• Future plan impacted as well
Uptime lnstitute='''e'~' "'=~--~-·m.,.,.._,.,~ ~.,,,

Sc~~1on 5 Pagr: I 8
Piping Topology
Not Concurrently Maintainable nor Fault Tolerant
·~---------- con4Sup------------
--~-----------conoRtno-----------__,~....-
Uptimelnstitute- - - - -scmon
--- 5 f'~nc 19
Piping Topology
-,
I
I
~ ! lli ~'· ~
'
• Initial condition requires 4 redundant Computer Room Cooling Units
• Future condition requires 8 redundant units (N+S!)
• UPS cooling is not Concurrently Maintainable nor Fault Tolerant
Uptime Institute S~ion 5 - Page 20
Valve Placement Guidelines

N+1 Air Cooled Chillers N+1 Secondary ..-----,
0 ~ • IDWlLJ~·,.._•--,----, Pumps 1- ·...f~]~
~
. ~ --·
L
• N+1 devices in looped systems shall be separated by at least 2 valves
• N+2 devices in looped systems shall be separated by at least 1 valve
• Tier IV requires autonomous valve operation
- - - - - - - -Uptime Institute Session 5 Page 21

Condenser Water Schematic
2 valves separate like elements
8
N+1 Components
Uptime lnstitute.~-------
Sc-.sion S Pag~ 2J
Compartmentalization:
Institute Tier Standard
§2.4.1. d)
"Complementary systems and distribution paths must be
physically isolated from one another (Compartmentalized) to
prevent any single event from simultaneously impacting both
systems or distribution paths. "
Uptimelnstitute..........,-------
"('<,""'n" r~~ny;
Mechanical Equipment Power Path
Sub 7 Line 1) (---- Pu-bli;-UUI~y----. \ Sub 9 Line 4
rJo j- r~+ 1o/~ , __ ----------- __ /
DDDDDDrnJDDrnJDDDD
r;-~r------h
r~j ____;___ _j ____ } __ j
r-1---
l__ __} ____i__
~
~ ~31
[_~_
,:~~~....,.§J_ Cf ~-:
:~--rr·-~-TI,~z~~:
·-}-· .; 2 i
l~~l ___ }____f-----?.__---_
-l - _-_-_-...Jr---------- ___ r~l
_____L
,---r----~------------- ------- ----------~- ---
1 o:
~~ Cii l
!'
~l_ __ ?____] ____ 20...d ____ ] __l __l
IU~~-;;Eio~- j ..:
I"' s: ''
,-·-,----------- --- ----~ ~ ------- -- -----"l--
L,. _ _ _ _ _ _ _ _ _ _ _ _
_________ l ___ _ :, s:: I :
• ~ ()() I UPS A Cooling See E203 • It UPS B Cooling See E2Q.4 !t (')(') I
~ ~ i I ~
~
1 I
: UPS 2A See E402
~---------- ----J
I W' - ------I
'-'---.J----------
I
L.-------------- --·--J
I ~ ~
~-- ~ r----- lf-- --------- -----------·

: ~o: :
l~-~L __U _?__ ?__?__ ?__?__ _?__~_?___ j
, Uptime Institute session s- rage 26

Compartmentalizati~on:
Tier IV Requirement
- · - · - · - - - - - - - - - - ~~sA · rl~
,
ATSC l ljlll ATSD
~! ~o c ~ 0vFo 0
I
r-=l:t vFo A r-=l:t vFo 8 [ 1 i Secondary Chilled Water Pumps:
~ i ~ ] ~ i ~ i N=3 (R=1)
scHWPA li scHwP s j scHWPcj scHwPo j Chilled Water Machines, Primary Pumps:
· li i i i N=4 (R=2)
~~- - · - · · · -- - · · · · · · · · ·;:; · · · · · · · · · · · · · · · ·rr · · · · · · · - · - · · · - · · - · - · --
CHWM 1A CHWM 2A CHWM 18 CHWM 28 CHWM1C CHWM2C
I
I .
I . . . . .. . .. . . ... . .... _ . . . .. _ . . . . . . . . . . . . . . . . • . . . .. . . . . .. . ·- · I
• No more than the redundant number of like devices shall

be located in a single space
• Complementary power distribution components shall not
be located in the same physical space
r:::=:::===============================----Uptillle Institute~ m~
Session 5 - Page Z7
Mechanical Equipment Power Path
OIIl!JDJilllr:om:r:rrJO
Uptimelnstitut · kfpO.n ) , tltf 10
Tier IV Requirement
Secondary Chilled Water Pumps:
N=3 (R=1)
Chilled Water Machines, Primary Pumps:
N=4 (R=2)
• No more than the redundant number of like devices shall

be located in a single space
• Complementary power distribution components shall not
be located in the same physical space
Uptime Institute:- - - - S=l on ~ - r agc27
Design Discrepancies Recap
• Just like a utility outage, hot days are not an

accident
> Select equipment based on Extreme ambient conditions
• Integrate power topology with mechanical topology
> Redundant units are dependent on power source and path
• The number of isolation valves required is based on
equipment redundancy
> Ensure capacity is available with any element isolated
• Power path and mechanical equipment shall be
Compartmentalized for Tier IV
- - - - - - - -Uptimelnstitut:e·- - - - - - - - -
Pa13e 28 sc~ion 5
UptimeInstitute- - - - - - -
Operational Sustainability
• Management and Operations

• Building Characteristics
• Site Location
- - - - - - - -Uptime Institute Scs.sion6- rage1

Building Characteristics Categories
Infrastructure Pre-Operational
> Flexibility for Incremental > Lo~~t~~tot( .s:r;·o-'1•'""\
Capacity Increases (..)
> ;.,.Jv4f~ I.V'C.. - +a -~ of.s
> 6~ , J. ~- ~~
Building Features
> Pu rpose Built
> J ..J(>fo.-J 4 >f~J ~f~
> Security and Access
> Setback
Uptime Institute Sc:s10n6 Page 3
Infrastructure to Support Operations
• Simplicity of design has a major impact on

the availability of the data center by reducing
human error
> Ease to Operate and Manage
> Ease to Understand Configuration
> Ease to Troubleshoot
Uptime Institute'___._ _ _ _ _ __
Sc~w~m 6 Pa?,e 4
- - - - - - -Uptime Institute- - - - Session6- Palle 5
Uptime Institute Session 6- Page 6

Design Example: 2N Option
• Allows the UPSs and PDUs to be loaded at maximum of 50%

• Easiest layout to manage loads and connections
- - - - - - - -UptimeInstitute- - - - - - - - Session 6- Page 7

Design Example: 3 to Make 2 Option
-----------------------,
I
--, :------i
~ ~-~~-~-~-~----~ ----
I 1 I I
I I 1 I '
• Allows the UPSs and PDUs to be loaded at maximum of 66.6%

• Requires strict management of loads and connections
Uptimel n s t i t u t e - - - - - -Pa11c-B-
scs~ion 6-
Infrastructure Support to Operations
Labeling conventions
Color: All Infrastructure (Engine Generators, Panels,
Switchgear, Piping, Conduits or Cables)
Labels: Prefix AXXX
Suffix XXXA
System Labels System B XXXX
Identification Must:
1. Be easily understood
2. Identify system
3. Identify location
4. Sized for easy reading
5. Consistently located
- - - - - - - - Uptimelnstitute Session 6- Pa~e 9
Ease of Maintenance
• Maintenance space is necessary to enable proper

maintenance
> Different space requirements for maintenance, repairs, or
replacement
• Allows installation or replacement of critical components
without affecting operations
> Provides craftsmen the ability to work safely and with freedom of
motion, thus reducing the risk of human error
- - - - - - - -Uptime Institute Sc5~ion 6 -Page IZ

Ease of Maintenance
- - - - - - -Uptimelnstitut · Sc,s1on6- Pa~;:c 1<1
Ease of Maintenance
Ease of Maintenance
• Infrared scanning
windows allow
safe maintenance
practices while
under load
- - - - - - - -Uptimelnstitute':- - - - - - - - k • ILJII10 ,~11\
Support and Specialty Spaces
• Support and specialty spaces decrease risks

• IT hardware receiving, storage, and staging area
provides place for
> Uncrating equipment-keeps packing materials out of computer
room
> Testing equipment before putting it on critical power
> Storing supplies-keeps contaminants and combustibles out of
computer room and saves valuable computer room space
- - - - - - -..........uptimeInstitute- - - - Session6- Page 17

Support and Specialty Spaces
• Building automation system (BAS) control room

> Space for Facilities to monitor the data center infrastructure
• Command/Disaster Recovery center
> Dedicated space to work any data center issues
• Dedicated spaces for parts and tool storage
> Necessary to store critical spare parts and tools for quick response
> Allows for better management of these resources
• Engineering and Facility shop space
> Office space for Engineering support (Facilities, IT) is necessary to
enable quick response times to issues
> Shop space, if necessary, to repair equipment or build power whips
away from the computer room floor
- - - - - - - -Uptimelnstitut · Session 6- Page IB
Commissioning
• A process designed to test both individual

components and whole systems to ensure that they
operate as the designer intended
• Commissioning (Cx) is the 'official' beginning of the
facility hand off from the Contractor and Designer to
Maintenance and Operations
• A commissioning agent (CxA) is generally appointed
to lead the team through the commissioning process
Uptimelnstitut:e·- - - - - - - - ~' t~.L)

Commissioning
• Commissioning is not only for new data center

builds
• Scalable infrastructure designs need to be
commissioned when they are installed, even though
there is critical IT load
> This needs to be considered during the design phase in order to
minimize the risk to the critical IT load
• Re-commissioning should take place after major
repairs to critical systems
- - - - - - - -Uptime Institute·-.-....-------~ Session 6- Pil!!C11
CxA
• The CxA should be hired during the design phase to

begin coordinating and planning the commissioning
process
• Most effective when the CxA is an unbiased, third
party who works directly for the owner
• Responsible for the following
> Directing the planning of all phases of commissioning
> Scripting all tests for the Integrated Systems Testing (1ST)
> Gathering documentation from all sources and compiling into a
final commissioning report
Sc5SIOn 6 - Pilgc l2
Levels of Commissioning
• Level 1: Factory Test

> Factory engineers test equipment in a controlled environment
against national standards (e.g., ASHRAE, ARI, ANSI, NETA, etc.)
> Can be witnessed by the owner if desired
• Level 2: Receipt, Installation, and Pre-Functional
Testing
> Contractor installs the equipment and ensures that the individual
components are installed per the design drawings
• Level 3: Functional Testing
> Contractors test very basic operation of individual components
(i.e., power on, proper rotation, etc.)
Uptimelnstitute,-------- scssion 6- Pajle13
Levels of Commissioning
• Level4: System Start, OEM Testing

> Factory representatives come on site to start up and test
equipment against company standards
> Test and Balancing (TAB)
> Generally starts the warranty period
• Level 5: Integrated Systems Operational Test
> Full systems are tested against the documented Sequences of
Operation
> Tests prove that each and every component works within the
system as it is supposed to
> Should be tested against all possible operating conditions
/situations
UptimeInstitute- - - - Sc:sion 6- Pagc24

Commissioning
Bathtub Curve
Useful Life
Time
Commissioning must surpass the infant mortality period
- - - - - - - -UptimeInstitute- - - - - - - - Session 6 PagG"15
Problems Found in the Commissioning

of Tier Ill and Tier IV Data Centers
• Engine generator coolant and oil leaks
• Inability of engine generators to produce stated capacity
at ambient conditions
• UPS inverter explosion
• Loss of power source to mechanical controller enabled
the mechanical plant emergency off system
• Explosion of CT transformer in switchgear
• Air handling unit mode change from free cooling to
recirculation was not Fault Tolerant
Discussion Point: What have you found in

Commissioning of your data centers?
--------Uptime lnstitute-·~-- Se~s10n 6- Page 26

Data Center Site Selection
• Site Location issues include

> Risks posed by natural disasters
> Risks posed by man-made features, which can change over time
• Investigate site risks
> Online tools
> On site and area investigation
• Risk mitigation methodologies should be employed
when and where possible, consistent with business
objectives
Uptime Institute------~~~ Session 6- P<~ge 27
Operational Sustainability Summary
• Building Characteristics impact Operational

Sustainability
> Infrastructure to support operations
> Ease of maintenance
> Support and specialty spaces
• An intensive commissioning program is the key to
successfully bringing a new data center into
operation
• Site Location
> Identify and mitigate risk
- = - , _ - - - - - -Uptimelnstitut -· Sesswn 6- P<~ge 28

Uptime Institute
Tiers Does Not Require a Raised Floor
• The Uptime Institute:

> Choice of cooling delivery (underfloor or overhead) is an owner
decision based on operational preference
> Decisions such as: raised floor or on-slab, Cold Aisle/Hot Aisle,
containment of Cold Aisle/Hot Aisle, or gallery cooling can effect
the efficiency of computer room cooling, but none of them are
mandated by Tiers
- - - - - - - -Uptime Institute- - - - - - - - session 7- l'<~ge Z

For Tier Ill or IV, the Engine-Generator Plant
Must be Operational at all Times

> Tiers does not require that the engine-generator plant run at all
times
> Data centers will typically utilize the public utility a majority of
the time
> However, the engine-generator plant must be properly
configured and sized to carry the criticai ioad without runtime
limitations ( .Jll,.. Ill)
> Meeting the above criteria requires special attention to engine-
generator capacity ratings and power distribution
Uptime Institute'---== SE'~s10n 7- Page 3
Tiers Has No Physical Security Requirements

> The level of security of a data center is largely determined by
• Industry practices and requirements
• The criticality of IT functions preformed at the site
• Owner's security policies
> A number of physical security measures that are best practices
should not be confused with Tier requirements
> Access Control is an expected Behavior for Operational
Sustainability
Uptime Institute.·- - - -
/~;~,.de.
Tier IV is the Best

> Tiers reflects a performance-based business-objective driven,
data center topology benchmarking system
> An organization's tolerance for risk determines the appropriate
Tier to support the business objective
> If no business objective is defined, then often Tiers is misused
to rationalize unnecessary investment
Uptime Institute
Utility Feeds Determine Tier Level

> The number of utility feeds, substation, and power grids that
provide public power to a data center neither predicts nor
influences Tier level
> Utility power is not even required for Tiers
- - - - - - - -Uptime Institute SCS.Slcn 7 Pag~ 6

. ..
Component Count Determines Tier Level

> N+1, N+2, N+N, or 2(N+1) does not determine Tier level
> It is possible to achieve Tier IV with just N+1 components for
some systems
- - - - - - - -Uptimel n s t i t u t e - - - - - - - - Sessio'l7 Page 7
Site Location Affects Tier Level

> Although a critical consideration for the life-cycle operation of the
facility and in determining, evaluating, and mitigating risk to the
data center, the geographical location does not affect Tier
> Site location does impact Operational Sustainably
Uptimelnstitute:- - - - - - -c.,"'''""7
---- ,~,.
n
Uptime Institute
Question #1
• For a Tier-rated design, the useable sensible

capacity of a cooling unit must be determined
based on which of the following:
A. Altitude if the site is located above sea level
B. Actual computer room return air and ASHRAE (n=20) outside
air temperatures
C. Impact of local design features such as cooling equipment on
a roof
\I'D. All of the above
Uptime Institute Se-s~ion 7 Page 10

Question #2
• What is the MINIMUM possible number of UPS

modules for a Redundant Components or
Concurrently Maintainable UPS system :
A One
v B. Two
C. Four or more
D. Cannot determine with the information provided
- - - - - - - -Uptimelnstitut:e·- - - - - - - - - - - - sc~ioo 7 Page 12
Question #3
• During a planned maintenance activity, which

statement properly describes a Tier Ill site?
A The site reverts to a Tier II topology during maintenance
v' B. The site remains a Tier Ill, but operates at an increased risk
of disruption
C. The site represents Tier IV during approved maintenance
activities
D. The Tier classification does not apply during maintenance
Uptimelnstitute- - - - - - - - <;-qnn 7- ~i'le-:> 14

Question #4
Condenser Pump Topology
Condenser Pump
Given that two pumps are required to support the design load (N=2 pumps),
the configuration depicted in the Condenser Pump Topology figure above can
best be described as:
A FaultTolerant
B. Concurrently Maintainable
1../'t. Redundant Components
D. Basic Capacity
Uptimelnstitute- - - - - - - - sc~sion 7 ras.:- 16
Question #5
• Which elements of a site infrastructure can be

EXCLUDED from the "Each and Every" criteria for
Tier Ill?
A. Fuel oil piping
B. Battery room exhaust system
C. Chillers that can be temporarily connected to the infrastructure
V D. Band C
Uptimelnstitute - - - - - - - S<>SS10n7 Po!J"I8

Question #6
• Compartmentalization for a Tier IV (Fault Tolerant)

site infrastructure applies to which of the following
elements?
A. Power distribution routing
B. Security system conduit routing
C. Chilled water pipe routing
/ D. A and C
v
Uptimelnstitute- - - - - - - - 7-Pa11c
-20- Sc~ion
Question #7
• A proposed Tier IV system design includes six

engine generators in an N+2 configuration. Which
of the following solutions correctly addresses
Compartmentalization"
t/A. Provide three engine-generator rooms. Install two engine
generators in each room.
B. Split the engine generators equally with three engine
generators contained in each of two rooms.
C. Install four engine generators in one generator room . Install
the two redundant engine generators in a second, separate
generator room.
D. The N+2 topology doe not require Compartmentalization.
Uptimelnstitute- - - - -Session
-7 - -
ra&e 12
l
]
Uptimelnstitut.e>'..........- - - - - - -
Tier-Based Design Review
• Lesson Objectives
> This session will be a mentored exercise to review drawings
submitted for Institute Certification
> Individual drawings are used by the permission of the copyright
owner for the ATD course and shall not be removed from the
room
> Return all drawings at the completion of the exercise to Institute
staff
Uptimelnstitute- - - - - - -Paec2- Se!.!ilOil 8-

Exercise Format
• Form teams of 4 or 5
> Multiple discipline teams are best
• Review of 1st assigned system
> Allocate 25 minutes for review
> Take 5 minutes to confirm notes for presentation
• Present Summaries
> Each team allocated 10 minutes to present summary
• Review of 2nd assigned system
> 4 sets of drawings; allocate 25 minutes for each review;
> Take 5 minutes to confirm notes for presentation
• Present Summaries
> Each team allocated 10 minutes to present summary
• Discussion may continue after the session, but drawings must
remain in the room
Uptime Institute I P•tl' J
Review Objectives
1. Determine highest level of Tier compliance

2. Identify characteristics that must be modified to
comply with next Tier
3. Identify characteristics that exceed Tier level
determined in Step 1
Uptimelnstitute:- - - - - - - - sezs,on B Page4

fl
Uptiine Institute
Accredited Tier Designer
Abbreviation List
-A-
Acc Air Cooled Condenser
AFF Above Finished Floor
AHJ Authority Having Jurisdiction
ANSI American National Standards Institute
ASH RAE American Society of Heating, Refrigerating and Air-conditioning Engineers
ATD Accredited Tier Designer
ATS Automatic Transfer Switch
-8 -
BAS Building Automation System
BMS Building Management System
-C-
CRAC Computer Room Air Conditioner (OX)
CRAH Computer Room Air Handler (Chilled Water)
CRCU Computer Room Cooling Unit
-0-
DB Dry Bulb Temperature
DOC Direct Digital Controls
DX Direct Expansion
- E-
EPA Environmental Protection Agency (U.S.)
EPO Emergency Power Off
EWT Entering Water Temperature
-G-
GPM Gallons Per Minute
-H-
HVAC Heating, Ventilation, Air Conditioning
-I-
ISO International Organization for Standardization
IT Information Technology
Uptin1elnstitute
accredited tier designer""
-K -
kVA Kilovolt Amperes
kW Kilowatts
-M -
MCC Motor Control Center
MTS Manual Transfer Switch
-N-
N Nominal Capacity or Units
-P-
PDU Power Distribution Unit
POP Point of Presence
PTS Point-of-Use Transfer Switch
-R-
R Redundant Capacity or Units
RH Relative Humidity
-S-
scADA Supervisory Control and Data Acquisition
STS Static Transfer Switch
-T-
TST Thermal Storage Tank
-U-
UCU UPS Cooling Unit
UL Underwriters Laboratories
UPS Uninterruptible Power Source
-V-
VLA Vented Lead Acid (batteries)
VRLA Valve Regulated Lead Acid (batteries)
-W-
WB Wet Bulb Temperature
WH Water Heater
'

Accredited Tier Designer

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Accredited Tier Designer

Uploaded by

Copyright:

Available Formats

http://uptimeinstitute.

All rights reserved.

Uptime Institute 20 West 37u' Street, 61h Floor

Tier Standards and Technical Papers ............................................ Tab 1

Course Schedule ........................................................................... Tab 2

Session 1 Notes ............................................................................. Tab 3

Session 2 Notes ............................................................................. Tab 4

Session 3 Notes .............................................................................. Tab 5

Session 4 Notes ............................................................................. Tab 6

Session 5 Notes .............................................................................. Tab 7

Session 6 Notes ............................................................................. Tab 8

Session 7 Notes ............................................................................. Tab 9

Session 8 Notes ............................................................................ Tab 10

Abbreviations ............................................................................... Tab 11

Uptime Institute Contacts ........................................................... Tab 12

For more information, please visit www.uptimeinstitute.com/publications to download a Copyright Reprint

Additional Factors and Exposures

3.5 Fractional or Incremental Tier Classification .... ............ ......................................................... 11

3.6 Non-Compliance Trends .... .. ............................... ......... .......................................................... 12

1.4 Relationship between Tiers and Operational Sustainability

2. Site Infrastructure Tier Standards

2.1.2 The performance confirmation tests:

a) There is sufficient capacity to meet the needs of the site.

2.1.3 The operational impacts:

2.2 Tier II: Redundant Site Infrastructure Capacity Components

2.2.2 The performance confirmation tests:

2.2.3 The operational impacts:

2.3 Tier Ill: Concurrently Maintainable Site Infrastructure

2.3.2 The performance confirmation tests:

2.3.3 The operational impacts:

2.4 Tier IV: Fault Tolerant Site Infrastructure

2.4.2 The performance confirmation tests :

2.4.3 The operational impacts:

a) The site is not susceptible to disruption from a single unplanned event.

2.5 Engine-Generator Systems

2.5.1 Site on Engine-Generator Power

2.5.2 Manufactures' Runtime Limitation

2.5.3 Regulatory Runtime Limitation

2.6 Ambient Temperature Design Points

2.6.1 Ex1reme Annual Design Conditions

2.6.2 Computer Room Set Points

2.7 Communications Routing

2.8 Makeup Water

2.9 Tier Requirements Summary

Toerl Tier II Tocr Ill Toer IV

Active Capacity Components N

Distribution Paths 1 1 1 Active and 1 Alternate 2 Simultaneously Active

Concurrently Maintainable No No Yes Yes

Fault Tolerance No No No Yes

Continuous Cooling No No No Yes

Table 1: Tier Requirements Summary

3. Commentary for Application of the Tier Standard: Topology

3.1 Outcome-Based Tier Standard

3.2 Impact of Ambient Design Conditions

3.4 Tier Functionality Progression

3.4.3 lier Ill

3.5 Fractional or Incremental Tier Classification

3.6 Non-Compliance Trends

About the Uptime Institute

Uptime Institute is a division of The 451 Group, a leading technology

Visit www.uptimeinstitute com for more information.

© 2009-2014 Uptime Institute, LLC All rights reserved

For more information, please visit www.uptimeinstitute.com to download a Copyright Reprint

Additional Factors and Exposures