Professional Documents
Culture Documents
Many data center owners and others commonly believe that commissioning takes
place only in the last few days before the facility enters into operation. In reality, data
center commissioning is a continuous process that, when executed properly, helps
ensure that the systems will meet mission critical objectives, design intent, and
contract documents. The commissioning process should begin at project inception and
continue through the life of the data center.
Uptime Institute’s extensive global field experience reveals that many of the problems,
and subsequent consequences, observed in operational facilities could have been
identified and remediated during a thorough commissioning process. Rigorous,
comprehensive commissioning reduces initial failure rates, ensures that the data
center functions as designed, and verifies facility operations capabilities—setting up
Operations for success. At the outset of the commissioning program development, the
owner and commissioning agent (CxA) should identify the important elements and
benchmarks for each phase of the data center life cycle. Each element and benchmark
must be executed successfully during commissioning to ensure the data center is
rigorously examined prior to operations.
Uptime Institute wants to highlight the importance and benefits of commissioning for
data center owners and operators and clarify the goals, objectives, and process of
commissioning a data center.
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 1 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
This publication:
• Defines and reinforces the basic concepts of the five levels of commissioning
• Relates the levels of commissioning to the data center life cycle
• Presents technical considerations for commissioning activities associated witheach
phase
• Details the overall management of a commissioning program
• Identifies minimal roles and responsibilities for each stakeholder throughout the
commissioning process
Commissioning tests some of the most important operations a data center will perform
over its life and helps ease the transition between site development and daily
operations. Commissioning:
• Verifies that the equipment and systems operate as designed by the Engineer-of-
Record
• Provides a baseline for how the facility should perform throughout the rest of its life
• Affords the best opportunity for Operations to become familiar with how systems
operate and test and verify operational procedures without risking critical IT loads
• Determines the performance limits of a data center—the most overlooked benefit of
commissioning
In other words, commissioning highlights what a system can do and how it will respond
beyond the original requirements and design features if the process is executed to a
high degree of quality. Commissioning, like data center operations, must be considered
throughout the full life cycle of the data center (see Start with the End in Mind, The
Uptime Institute Journal Vol. 3 p. 104). Commissioning should first be considered, and
planned for, at a project’s inception and continue throughout the design, construction,
transition-to-operations, and ongoing operations where re-commissioning is
appropriate.
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 2 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
COMMISSIONING LEVELS
Over time, various organizations have defined the levels of commissioning. As a result,
a data center owner may encounter a number of variations when attempting to
understand and implement a commissioning program. With this publication, Uptime
Institute clarifies the purpose of each level. However, each and every data center
project is unique, which could mean that one or more of these activities might fit better
within a different level of commissioning for some projects. Table 1 is organized to
outline the process and sequence for commissioning, but the most important thing is
that all the activities are completed. The high reliability essential to mission critical
facilities requires that a rigorous and complete commissioning program includes all five
levels to ensure that capital investments are not wasted.
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 3 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 4 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
commissioning and testing a Tier I data center may be less than that of a Tier IV data
center—based on differences in the actual design complexity, topology, size,
components, and sequence of operations. However, the roles and responsibilities and
technical requirements for the commissioning team should not differ largely between
Tiers and should be just as rigorous and comprehensive for a Tier I as it would be for a
Tier IV.
COMMISSIONING STAKEHOLDERS
The most critical stakeholders involved on any project are listed below. They should
fulfill their major roles sequentially. Stakeholders with additional expertise or valuable
contributions should also participate. The CxA should ensure that the roles and
responsibilities of the commissioning stakeholders are balanced and well documented.
Owner: The owner should initiate the commissioning process at the project outset,
including identifying key stakeholders to take part in the program and communicating
expectations for the commissioning program. Owner’s personnel are typically
responsible for internal engineering, project management, and administration; however,
the IT end user may also be part of the owner’s team. When the owner’s personnel lack
the necessary experience for these activities, those responsibilities should be
delegated to an authorized third-party representative, typically referred to as an
owner’s representative. If the owner does not participate (or appoint a representative),
no one on the commissioning team will have the knowledge or perspective to represent
the owner’s interests. The end result could be a sub-standard facility, unnecessarily
vulnerable to outages and unable to support the business needs.
Contractor: The commissioning team always includes the general contractor and
specialty trade contractors, including mechanical, electrical, and controls, as well as
OEM vendors who will be bringing equipment on site and assisting with testing.
Without contractors, commissioning activities will be nearly impossible to complete
properly. Contractors often coordinate vendors and physically operate equipment
during commissioning procedures. Uptime Institute experience indicates that although
contractors are rarely excluded entirely from commissioning their input is sometimes
undervalued.
Architect and engineers: The Engineers-of-Record are legally responsible for the
design of the data center, including those responsible for mechanical and electrical
systems. Design intent may be compromised if designers do not participate in
commissioning. The design engineer specifies the sequence of operations and is the
only party who can confirm that the intent of the design was met.
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 5 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
CxA: Ideally a third party is the responsible authority for the planning and execution of
the entire commissioning process. The CxA may be an owner’s representative or a
qualified mechanical or electrical contractor. Trying to commission without a CxA could
result in poorly planned, undocumented, and unscripted commissioning activities.
Additionally, it makes it more difficult to close out construction and properly transition
to operations, which includes proper punch-list item closeout and execution of training
for operations staff.
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 6 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
At this time, the owner should hire the data center facility manager and one data center
facility supervisor to support the commissioning activities as representation for the
operations team. It is not necessary to build the entire operations team to support the
commissioning and construction activities.
CxA Selection
Selecting the CxA in the Pre-Design phase allows it to help develop the OPR and BOD,
the commissioning program, the budget, and schedule.
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 7 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
• Contracted directly to the owner to ensure the owner’s interests are held primary
• Optimally aligned to both the owner’s operations team and the owner’s design and
construction team to further align interests and gain efficiencies in coordinating
activities throughout commissioning
• An independent third party
Project Schedule
• Must include all commissioning activity time on the schedule to avoid project delays.
• Should allot sufficient time for correcting installation and performance deficiencies
revealed during commissioning.
• Should assess the requirement and/or capability for post-occupancy commissioning
activities. This can include provisions for seasonal commissioning to assess the
performance of critical components in a variety of ambient conditions.
At this point, the schedule should have significant flexibility and can be better defined
at each phase. Depending on the size, complexity, and sequence of operations
associated with a facility, a rigorous Level 5 commissioning schedule could take up to
20 working days or longer. Even for small and relatively basic data centers,
commissioning teams will find it challenging to complete Level 5 commissioning with a
high level of detail and rigor in less than a few days.
Commissioning Budget
• Should include a large contingency reserve for Level 4 and Level 5 commissioning
budgets • Should include all items and personnel required to support complete
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 8 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
commissioning
At this point in the process there are a considerable number of unknowns in the design,
construction, and commissioning requirements. Budgets for Level 4 and Level 5
commissioning should include a large contingency reserve to accommodate the
unknown parameters of the project. This contingency reserve can be reduced as
appropriate as the project moves along and more and more items are defined.
However, the final budget should still have a contingency reserve to account for
unforeseeable issues, such as additional load bank rental time in the event
commissioning takes longer than scheduled. Budgets need to include all items and
personnel required to support commissioning. This includes, but is not limited to, load
banks, calibrated measurement devices, data loggers, technician support, engineering
support, and consumables such as fuel for engine-generator sets.
ASHRAE Standard 202 defines a BOD as “a document that records the concepts,
calculations, decisions, and product selections used to meet the OPR and to satisfy
applicable regulatory requirements, standards, and guidelines. This includes both
narrative descriptions and lists of individual items that support the design process.”
The OPR and BOD are generated by the project owner and communicate important
expectations for the data center project. These documents should be revised and
updated at the end of each phase in the construction process. Specific to data center
commissioning, the OPR and BOD should:
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 9 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
The decision as to whether or not the design intent is to be scalable or use shared
infrastructure will have a large effect on subsequent implementation and
commissioning phases. Shared infrastructure systems in an incremental buildout can
potentially increase the risk associated with future commissioning phases. Careful
planning can mitigate this risk. Where shared infrastructure is to be used in a phased
implementation, the OPR or BOD should highlight the importance of including design
features that will allow a full and rigorous commissioning process at each phase of
project implementation.
The design and pre-construction phases are commonly blended into a design/build
format in which some activities are completed concurrently (see Table 3). The design
typically goes through multiple iterations between the Engineer-of-Record and owner;
each iteration identifies additional data center design details. Since the Engineer-of-
Record is responsible for creating the designs and ensuring their concurrence with
applicable commissioning documents, it becomes the responsibility of the other
stakeholders to verify compliance. The CxA and contractor should review the design
throughout this process, provide feedback based on their experience, and ensure
compliance with the OPR document.
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 10 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
Commissioning Plan
The commissioning plan is the heart of the commissioning program for a data center.
While the CxA will take the lead in its development, all other stakeholders should
review and participate in the approval of the final commissioning plan. Utilizing each
unique skillset in developing and reviewing the commissioning program will help ensure
a rigorous commissioning program.
Once appointed, the CxA must develop the overall commissioning plan, which
generally includes:
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 11 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
Design Review
• For concurrence with the OPR and planned operations • For commissioning
readiness
At this phase, appropriate stakeholders should develop plans, checklists, and reports
for Level 1, Level 2, and Level 3. All stakeholders must review these documents.
Additional technical requirements during this phase include:
• Review project schedule and budget to ensure the schedule continues to maintain
adequate resources and time to complete commissioning
• Verify adherence to OPR and BOD
• Amend documents as necessary in order to keep them up to date
• Add design elements as required to allow for the commissioning program to meet the
minimum OPR and meet commissioning readiness requirements
• If the design is scalable and to be implemented in future phases with shared
systems, ensure that the design allows for future commissioning by including
enhancements to reduce the risk to the active IT equipment.
• Ensure that equipment specifications identify that the specified capacity is net of
any deductions or tolerances allowed by national or international manufacturing
standards (verified during Level 1 and Level 3)
Typically, long-lead items are procured using an RFP process in parallel with the design
process. The commissioning requirements for this equipment should be included in the
RFP documents and adherence to these should be assessed throughout the
equipment delivery and installation. As part of these requirements, RFPs should
identify the requirement for on-site OEM technician support throughout Level 4 and
Level 5.
• OEMs should provide the written test procedure to the commissioning team for
approval prior to the Level 1 activities.
• Contractors, in conjunction with the CxA, should provide Level 2 Post-Installation
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 12 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
Throughout the data center construction, the CxA will monitor progress to ensure that
the installations conform to the OPR document. Additionally, the first three levels of
commissioning will take place and be overseen by the CxA (see Table 4).
During the construction phase, the focus moves from developing plans to execution,
with team members executing Level 1, Level 2, and Level 3 activities. At the same time,
the operations team and the CxA develop scripts for Level 4 and Level 5, which are to
be reviewed by all stakeholders.
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 13 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
• Review project schedule and budget to ensure the schedule continues to have
adequate time and budget to
complete commissioning
• Protect equipment stored on site awaiting installation from hazards (e.g., dirt and
construction debris,
impact, fire hazards) and maintain according to manufacturer’s recommendations
• Verify that circuit breakers are set in accordance with the short circuit and breaker
coordination and arc
flash study
• Repeat Level 1 procedures in the actual data center environment since factory
witness testing is
performed in ideal conditions rarely seen in practice in the data center, ensuring that
no equipment
damage occurred during transit and that the equipment performs at the same level as
when it was tested
at the factory
• Ensure that the building management system (BMS) functions at a basic level so
that it is ready to
support critical Level 4 and Level 5 commissioning activities
• Log critical asset information (e.g., make, model, serial number) into the
maintenance management
system (MMS) (or other suitable recordkeeping) as equipment is received on site to be
available for the
operations team
• Continue to submit formal reports to the owner detailing all items tested, steps
taken to test, and the
results as soon as reasonably possible
• Repeat entire testing procedures when programming or control wiring is altered to
correct a testing step
that does not complete successfully as it is possible to have an unexpected impact
Script development is the responsibility of the CxA, with support from all other team
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 14 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
• Under expected normal operations in the same manner that the operations team will
operate the
data center
• Under expected maintenance conditions in the same manner that the operations
team will maintain
the data center
• In manual operation as necessary to support future upgrades and replacements
In addition, commissioning should simulate system and component failures to test fault
tolerant features, even when fault tolerance may not be a specific design assumption
because it will inform the operations team on how the infrastructure responds when
failures inevitably occur. Uptime Institute recommends testing in as many additional
scenarios as possible that make sense for the design—even scenarios that may be
outside the scope of design—to provide key information to operations about how to
respond when the facility does not function and/or respond as designed.
Once construction of the data center has been substantially completed, the CxA will
lead the team through Levels 4 and 5 commissioning. The purpose of these activities is
to ensure that individual systems and the full data center ecosystem function as they
were intended in the design and OPR documents. This includes verifying
requiredcapacities and ensuring that equipment can be isolated as intended and that
the data center responds as expected to faults (see Table 5).
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 15 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
While weather cannot be anticipated, it can and will have an impact on the final
commissioning results. Results from Level 4 and 5 commissioning activities should be
extrapolated to predict how the equipment would perform at the actual design extreme
temperatures and conditions. However, commissioning activities should be scheduled
seasonally to verify system operation for the actual extreme and varying ambient
conditions for which it was designed—especially in the event economizers are utilized
in the data center.
Level 4 and 5 commissioning presents a unique opportunity to any data center, which
is to be able to fully test and practice equipment operation in any building condition
without any risk to critical IT load. Improvements in future operations can result when
stakeholders take advantage of this opportunity.
Level 4
• Level 4 must include load bank tests of the engine-generator sets, UPS, and UPS
battery systems at design
and rated capacities.
• Minimum continuous runtime durations of not less than eight hours are
recommended for all load bank
tests; however, continuous runtimes of up to 24 hours are considered a best practice.
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 16 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
• Ensure that load banks are distributed within critical areas to best simulate the
actual IT environment
distribution, ideally physically located within racks and with forced cooling on a
horizontal path, which
allows for more accurate and realistic mechanical system testing.
• Prior to commencing Level 5, building management and control system (BMCS)
graphics should be
completed to support the commissioning activities and to help commission the BMCS
because operations
staff will eventually rely on the BMCS including alarming and data trending features.
Level 5
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 17 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
Site Cleanup
• Replace the air filters for the electrical systems and heating, ventilation, and air
conditioning (HVAC)
systems following the conclusion of Level 5 commissioning
• Flush and clean piping and ductwork to ensure construction debris does not impact
future mechanical
plant performance
The turnover-to-operations includes all activities associated with formally turning the
facility over to the owner and Operations. Primarily, this includes completing the final
documentation associated from the commissioning activities Levels 1-5 and utilizing
the commissioning results to finalize SOPs, MOPs, and EOPs.
Needless to say, this is a critical juncture for the data center. Soon after Level 4 and 5,
the facility will become live and support critical IT infrastructure, which begins the
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 18 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
need to maintain the facility. At this time, Operations must take all of the lessons
learned and knowledge gained from the construction and commissioning phases to
finalize the maintenance and operations program. Operations must complete this work
in a relatively short amount of time in order to minimize risk to the data center. The
longer it takes to finalize all of the documentation and processes, the longer the facility
will be at risk. Operations needs full support during this transition to ensure the overall
uptime and success of the data center.
• Ensuring that all post-Level 5 punchlist items are successfully completed and closed
out
• Facilitating and/or coordinating infrastructure or OEM training to the operations staff
• Assisting as necessary the development of critical operating procedures.
The CxA is also responsible for gathering all testing reports and checklists from all five
levels of commissioning to create the final commissioning report. The final report to
the owner should include:
• Electrical and mechanical load and system condition readings taken at timely
intervals before major
actions are implemented in each test
• All steps, results, and system readings at every stage of the commissioning
The CxA should return to the site approximately one year following completion of
commissioning to review the building operations and to ensure there are no
outstanding items related to re-commissioning or seasonal commissioning efforts.
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 19 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
In data centers that are built to be scalable, it is imperative that commissioning be just
as rigorous for the follow-on infrastructure deployments to minimize risk to the facility.
Commissioning activities undoubtedly add risk to the data center, especially where
infrastructure systems are shared. However, this risk must be weighed against the risk
of performing the associated commissioning tests. If a component or system is not
going to perform as expected, the owner must decide if it is better to have this occur
during a planned commissioning activity or during an unplanned failure. While
performing a rigorous commissioning program during the initial buildout may prove the
concept of the design, the facility could potentially be at risk if all of the new
infrastructure components and systems are not tested rigorously.
These types of commissioning activities, by their very nature, occur while the systems
are supporting critical IT load. In these instances, the operations team best knows how
these activities may impact that mission critical load. During re-commissioning or
incremental commissioning, the operations team should be working in very close
collaboration with the commissioning team to ensure the integrity of the data center.
Additionally, if re-commissioning involves changes to the configuration of the data
center, Operations needs full awareness so that operating procedures that impact
maintenance and emergency activities can be updated and tested completely.
The best way to mitigate the risk of re-commissioning efforts or for follow-on phases is
to ensure that the facility is properly and extensively commissioned when it is originally
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 20 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
built. And, as part of a rigorous re-commissioning program, all of the points discussed
in this paper for standard commissioning apply to the re-commissioning efforts.
However, due to the higher level of risk with these activities, there are some additional
requirements.
• Adequate notice must be provided to service owners about the schedule, duration,
risk, and
countermeasures in place for the re-commissioning activities in order to gain
concurrence from IT
end users.
• For facilities that are based on a dual-corded IT equipment topology, the owner and
Operations
should verify that the existing critical load is appropriately dual corded where systems
that support
installed IT loads are to be commissioned.
• As load banks can introduce contaminates, load bank placement should be
considered carefully so as not
to impact the existing critical IT equipment.
• Detailed commissioning scripts must be prepared and followed during
commissioning to ensure minimal
risk to existing IT equipment. Priority should be given to the live production IT
environment, and back-out
procedures should be in place to ensure an optimal mean time to recovery (MTTR) in
case of a power
down event.
• Seasonal testing of the systems should be performed to verify performance in a
variety of climatic
conditions, including extreme ambient conditions. This also ensures that economizers,
where used, will
be tested properly.
CONCLUSION
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 21 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
Commissioning activities represent a unique opportunity for data center owners. The
ability to rigorously test the capabilities of the critical infrastructure that support the
data center without any risk to mission critical IT loads is an opportunity that should be
capitalized on to the maximum possible extent. Uptime Institute observes that this
critical opportunity is being wasted far too often in data center facilities, with not nearly
enough emphasis on the rigor and depth of the commissioning program required for a
mission critical facility until critical IT hardware is already connected.
A well-planned and executed commissioning program will help validate the capital
investment in the facility to date. It will also put the operations team in a far better
position to manage and operate the critical infrastructure for the rest of the data
center’s useful life, and ultimately ensure that the facility realizes its full potential.
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 22 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM
the faculty/staff delivering the Accredited Tier Designer (ATD) and Accredited Tier
Specialist (ATS) programs, and any other courses to be developed and delivered by
Uptime Institute.
https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 23 of 23