You are on page 1of 23

Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

Improve Project Success Through Mission Critical Commissioning

Rigorous testing of data center components should be a continuous process

By Ryan Orr, with Chris Brown and Ed Rafter

Many data center owners and others commonly believe that commissioning takes
place only in the last few days before the facility enters into operation. In reality, data
center commissioning is a continuous process that, when executed properly, helps
ensure that the systems will meet mission critical objectives, design intent, and
contract documents. The commissioning process should begin at project inception and
continue through the life of the data center.

Uptime Institute’s extensive global field experience reveals that many of the problems,
and subsequent consequences, observed in operational facilities could have been
identified and remediated during a thorough commissioning process. Rigorous,
comprehensive commissioning reduces initial failure rates, ensures that the data
center functions as designed, and verifies facility operations capabilities—setting up
Operations for success. At the outset of the commissioning program development, the
owner and commissioning agent (CxA) should identify the important elements and
benchmarks for each phase of the data center life cycle. Each element and benchmark
must be executed successfully during commissioning to ensure the data center is
rigorously examined prior to operations.

Uptime Institute wants to highlight the importance and benefits of commissioning for
data center owners and operators and clarify the goals, objectives, and process of
commissioning a data center.

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 1 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

This publication:

• Defines and reinforces the basic concepts of the five levels of commissioning
• Relates the levels of commissioning to the data center life cycle
• Presents technical considerations for commissioning activities associated witheach
phase
• Details the overall management of a commissioning program
• Identifies minimal roles and responsibilities for each stakeholder throughout the
commissioning process

Commissioning tests some of the most important operations a data center will perform
over its life and helps ease the transition between site development and daily
operations. Commissioning:

• Verifies that the equipment and systems operate as designed by the Engineer-of-
Record
• Provides a baseline for how the facility should perform throughout the rest of its life
• Affords the best opportunity for Operations to become familiar with how systems
operate and test and verify operational procedures without risking critical IT loads
• Determines the performance limits of a data center—the most overlooked benefit of
commissioning

In other words, commissioning highlights what a system can do and how it will respond
beyond the original requirements and design features if the process is executed to a
high degree of quality. Commissioning, like data center operations, must be considered
throughout the full life cycle of the data center (see Start with the End in Mind, The
Uptime Institute Journal Vol. 3 p. 104). Commissioning should first be considered, and
planned for, at a project’s inception and continue throughout the design, construction,
transition-to-operations, and ongoing operations where re-commissioning is
appropriate.

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 2 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

COMMISSIONING LEVELS
Over time, various organizations have defined the levels of commissioning. As a result,
a data center owner may encounter a number of variations when attempting to
understand and implement a commissioning program. With this publication, Uptime
Institute clarifies the purpose of each level. However, each and every data center
project is unique, which could mean that one or more of these activities might fit better
within a different level of commissioning for some projects. Table 1 is organized to
outline the process and sequence for commissioning, but the most important thing is
that all the activities are completed. The high reliability essential to mission critical
facilities requires that a rigorous and complete commissioning program includes all five
levels to ensure that capital investments are not wasted.

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 3 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

Table 1. Commissioning Levels 1-5

COMMISSIONING AND UPTIME INSTITUTE TIERS


Unlike Uptime Institute’s Operational Sustainability Standard, the rigor associated with
commissioning a data center has little relationship to its Tier level. The scope for

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 4 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

commissioning and testing a Tier I data center may be less than that of a Tier IV data
center—based on differences in the actual design complexity, topology, size,
components, and sequence of operations. However, the roles and responsibilities and
technical requirements for the commissioning team should not differ largely between
Tiers and should be just as rigorous and comprehensive for a Tier I as it would be for a
Tier IV.

COMMISSIONING STAKEHOLDERS
The most critical stakeholders involved on any project are listed below. They should
fulfill their major roles sequentially. Stakeholders with additional expertise or valuable
contributions should also participate. The CxA should ensure that the roles and
responsibilities of the commissioning stakeholders are balanced and well documented.

Owner: The owner should initiate the commissioning process at the project outset,
including identifying key stakeholders to take part in the program and communicating
expectations for the commissioning program. Owner’s personnel are typically
responsible for internal engineering, project management, and administration; however,
the IT end user may also be part of the owner’s team. When the owner’s personnel lack
the necessary experience for these activities, those responsibilities should be
delegated to an authorized third-party representative, typically referred to as an
owner’s representative. If the owner does not participate (or appoint a representative),
no one on the commissioning team will have the knowledge or perspective to represent
the owner’s interests. The end result could be a sub-standard facility, unnecessarily
vulnerable to outages and unable to support the business needs.

Contractor: The commissioning team always includes the general contractor and
specialty trade contractors, including mechanical, electrical, and controls, as well as
OEM vendors who will be bringing equipment on site and assisting with testing.
Without contractors, commissioning activities will be nearly impossible to complete
properly. Contractors often coordinate vendors and physically operate equipment
during commissioning procedures. Uptime Institute experience indicates that although
contractors are rarely excluded entirely from commissioning their input is sometimes
undervalued.

Architect and engineers: The Engineers-of-Record are legally responsible for the
design of the data center, including those responsible for mechanical and electrical
systems. Design intent may be compromised if designers do not participate in
commissioning. The design engineer specifies the sequence of operations and is the
only party who can confirm that the intent of the design was met.

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 5 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

Operations: The maintenance and operations managers, supervisors, and technicians


are ultimately responsible for the day-to-day operations of the data center and its
maintenance activities. This group may include owner’s personnelor a third party
contracted for ongoing operations. Excluding Operations is a huge missed opportunity
for training and compromises the team’s ability to verify maintenance and operations
procedures (SOPs, EOPs, MOPs, etc.). Without live training during commissioning to
verify effective procedures, the operations team will not be fully ready for maintenance
or failures when the facility is in operation.

CxA: Ideally a third party is the responsible authority for the planning and execution of
the entire commissioning process. The CxA may be an owner’s representative or a
qualified mechanical or electrical contractor. Trying to commission without a CxA could
result in poorly planned, undocumented, and unscripted commissioning activities.
Additionally, it makes it more difficult to close out construction and properly transition
to operations, which includes proper punch-list item closeout and execution of training
for operations staff.

PRE-DESIGN PHASE ELEMENTS AND BENCHMARKS


Pre-design phase commissioning immediately follows the approval of a data center
project and begins with selection of the CxA through a request for proposal (RFP)
process (see Table 2). During the pre-design phase the owner, Engineers-of-Record,
operations staff, and the CxA identify the owner’s project requirements (OPR) for the
data center. Table 2 lists each participant in data center design, construction, and
commissioning and denotes responsibility for particular tasks. In some projects, the
owner may elect to utilize an owner’s representative to manage the day-to-day
activities of the project on their behalf.

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 6 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

Table 2. Pre-Design Phase tasks

At this time, the owner should hire the data center facility manager and one data center
facility supervisor to support the commissioning activities as representation for the
operations team. It is not necessary to build the entire operations team to support the
commissioning and construction activities.

PRE-DESIGN PHASE COMMISSIONING TECHNICAL REQUIREMENTS


Tasks to complete during the pre-design phase include developing a project schedule
that includes commissioning, creating a budget, outlining a commissioning plan, and
documenting the OPR and basis of design (BOD).

Technical requirements during this phase include:

CxA Selection

Selecting the CxA in the Pre-Design phase allows it to help develop the OPR and BOD,
the commissioning program, the budget, and schedule.

The CxA should have and provide:

• Appropriate staff to support the technical requirements of the project


• Experience with mission critical/data center facility commissioning
• Experience with the project’s known topologies and technologies
• Sample commissioning documents (e.g., Commissioning Plan, Method Statements,

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 7 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

Commissioning Scripts, System Manual)


• Commissioning certifications
• Client referrals

The CxA should be:

• Contracted directly to the owner to ensure the owner’s interests are held primary
• Optimally aligned to both the owner’s operations team and the owner’s design and
construction team to further align interests and gain efficiencies in coordinating
activities throughout commissioning
• An independent third party

Ideally, the CxA should not be an employee of the construction contractors or


architect/engineering firms. When a third-party CxA is not a viable choice, the best
alternative would be a representative from the owner’s team when the technical
expertise is available within the company. When the owner’s team does not have the
technical expertise required, a third-party mechanical or electrical contractor with
commissioning experience could be utilized. Of course, cost is a factor in selecting a
CxA, but it should not be allowed to compromise quality and rigor.

Project Schedule

• Must include all commissioning activity time on the schedule to avoid project delays.
• Should allot sufficient time for correcting installation and performance deficiencies
revealed during commissioning.
• Should assess the requirement and/or capability for post-occupancy commissioning
activities. This can include provisions for seasonal commissioning to assess the
performance of critical components in a variety of ambient conditions.

At this point, the schedule should have significant flexibility and can be better defined
at each phase. Depending on the size, complexity, and sequence of operations
associated with a facility, a rigorous Level 5 commissioning schedule could take up to
20 working days or longer. Even for small and relatively basic data centers,
commissioning teams will find it challenging to complete Level 5 commissioning with a
high level of detail and rigor in less than a few days.

Commissioning Budget

• Should include a large contingency reserve for Level 4 and Level 5 commissioning
budgets • Should include all items and personnel required to support complete

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 8 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

commissioning

At this point in the process there are a considerable number of unknowns in the design,
construction, and commissioning requirements. Budgets for Level 4 and Level 5
commissioning should include a large contingency reserve to accommodate the
unknown parameters of the project. This contingency reserve can be reduced as
appropriate as the project moves along and more and more items are defined.
However, the final budget should still have a contingency reserve to account for
unforeseeable issues, such as additional load bank rental time in the event
commissioning takes longer than scheduled. Budgets need to include all items and
personnel required to support commissioning. This includes, but is not limited to, load
banks, calibrated measurement devices, data loggers, technician support, engineering
support, and consumables such as fuel for engine-generator sets.

OPR and BOD

American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc.


(ASHRAE) Standard 202 defines an OPR as “a written document that details the
requirements of a project and the expectations of how it will be used and operated.
This includes project goals, measurable performance criteria, cost considerations,
benchmarks, success criteria, and supporting information. (The term Project Intent or
Design Intent is used by some owners for their Commissioning Process Owner’s
Project Requirements.)”

ASHRAE Standard 202 defines a BOD as “a document that records the concepts,
calculations, decisions, and product selections used to meet the OPR and to satisfy
applicable regulatory requirements, standards, and guidelines. This includes both
narrative descriptions and lists of individual items that support the design process.”

The OPR and BOD are generated by the project owner and communicate important
expectations for the data center project. These documents should be revised and
updated at the end of each phase in the construction process. Specific to data center
commissioning, the OPR and BOD should:

• Comply with ASHRAE Standard 202, or similar


• Specify that the CxA is responsible for all commissioning, testing, and formal
reporting
• Identify whether the intent of the data center design is to be scalable with IT
requirements and whether the use of shared infrastructure is allowed and note any
subsequent design features necessary to commissioning

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 9 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

The decision as to whether or not the design intent is to be scalable or use shared
infrastructure will have a large effect on subsequent implementation and
commissioning phases. Shared infrastructure systems in an incremental buildout can
potentially increase the risk associated with future commissioning phases. Careful
planning can mitigate this risk. Where shared infrastructure is to be used in a phased
implementation, the OPR or BOD should highlight the importance of including design
features that will allow a full and rigorous commissioning process at each phase of
project implementation.

DESIGN AND PRE-CONSTRUCTION PHASE ELEMENTS AND BENCHMARKS

The design and pre-construction phases are commonly blended into a design/build
format in which some activities are completed concurrently (see Table 3). The design
typically goes through multiple iterations between the Engineer-of-Record and owner;
each iteration identifies additional data center design details. Since the Engineer-of-
Record is responsible for creating the designs and ensuring their concurrence with
applicable commissioning documents, it becomes the responsibility of the other
stakeholders to verify compliance. The CxA and contractor should review the design
throughout this process, provide feedback based on their experience, and ensure
compliance with the OPR document.

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 10 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

Table 3. Design and Pre-Construction Phase tasks

DESIGN AND PRE-CONSTRUCTION PHASE COMMISSIONING TECHNICAL


REQUIREMENTS

Commissioning Plan

The commissioning plan is the heart of the commissioning program for a data center.
While the CxA will take the lead in its development, all other stakeholders should
review and participate in the approval of the final commissioning plan. Utilizing each
unique skillset in developing and reviewing the commissioning program will help ensure
a rigorous commissioning program.

Once appointed, the CxA must develop the overall commissioning plan, which
generally includes:

• Scope of commissioning activities, including identification of any re-commissioning


requirements

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 11 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

• General schedule of commissioning


• Documentation requirements
• Risk identification and mitigation plans
• Required resources (e.g., tools, personnel, equipment)
• Identification of the means and methods for testing

Design Review

• For concurrence with the OPR and planned operations • For commissioning
readiness

At this phase, appropriate stakeholders should develop plans, checklists, and reports
for Level 1, Level 2, and Level 3. All stakeholders must review these documents.
Additional technical requirements during this phase include:

• Review project schedule and budget to ensure the schedule continues to maintain
adequate resources and time to complete commissioning
• Verify adherence to OPR and BOD
• Amend documents as necessary in order to keep them up to date
• Add design elements as required to allow for the commissioning program to meet the
minimum OPR and meet commissioning readiness requirements
• If the design is scalable and to be implemented in future phases with shared
systems, ensure that the design allows for future commissioning by including
enhancements to reduce the risk to the active IT equipment.
• Ensure that equipment specifications identify that the specified capacity is net of
any deductions or tolerances allowed by national or international manufacturing
standards (verified during Level 1 and Level 3)

Typically, long-lead items are procured using an RFP process in parallel with the design
process. The commissioning requirements for this equipment should be included in the
RFP documents and adherence to these should be assessed throughout the
equipment delivery and installation. As part of these requirements, RFPs should
identify the requirement for on-site OEM technician support throughout Level 4 and
Level 5.

Commissioning Plans and Scripts for Level 1, Level 2, and Level 3

• OEMs should provide the written test procedure to the commissioning team for
approval prior to the Level 1 activities.
• Contractors, in conjunction with the CxA, should provide Level 2 Post-Installation

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 12 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

checklists from the record drawings to verify installation of all equipment.


• The entire commissioning team should review all Level 2 checklists.
• OEMs, in conjunction with the CxA, should provide start-up and functional
checklists for Level 3.
• Where component-level functional testing is necessary beyond the OEM’s typical
scope of work, the
CxA shall create testing procedures that should be reviewed by the commissioning
team and executed by
the contractors.

CONSTRUCTION PHASE ELEMENTS AND BENCHMARKS

Throughout the data center construction, the CxA will monitor progress to ensure that
the installations conform to the OPR document. Additionally, the first three levels of
commissioning will take place and be overseen by the CxA (see Table 4).

Table 4. Construction Phase tasks

CONSTRUCTION PHASE COMMISSIONING TECHNICAL REQUIREMENTS

During the construction phase, the focus moves from developing plans to execution,
with team members executing Level 1, Level 2, and Level 3 activities. At the same time,
the operations team and the CxA develop scripts for Level 4 and Level 5, which are to
be reviewed by all stakeholders.

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 13 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

Technical requirements during this phase include:

• Review project schedule and budget to ensure the schedule continues to have
adequate time and budget to
complete commissioning
• Protect equipment stored on site awaiting installation from hazards (e.g., dirt and
construction debris,
impact, fire hazards) and maintain according to manufacturer’s recommendations
• Verify that circuit breakers are set in accordance with the short circuit and breaker
coordination and arc
flash study
• Repeat Level 1 procedures in the actual data center environment since factory
witness testing is
performed in ideal conditions rarely seen in practice in the data center, ensuring that
no equipment
damage occurred during transit and that the equipment performs at the same level as
when it was tested
at the factory
• Ensure that the building management system (BMS) functions at a basic level so
that it is ready to
support critical Level 4 and Level 5 commissioning activities
• Log critical asset information (e.g., make, model, serial number) into the
maintenance management
system (MMS) (or other suitable recordkeeping) as equipment is received on site to be
available for the
operations team
• Continue to submit formal reports to the owner detailing all items tested, steps
taken to test, and the
results as soon as reasonably possible
• Repeat entire testing procedures when programming or control wiring is altered to
correct a testing step
that does not complete successfully as it is possible to have an unexpected impact

In addition, Engineers-of-Record should provide a finalized sequence of operations


document to the CxA, so it can create the Level 4 and Level 5 commissioning scripts.

Commissioning Plans and Scripts for Level 4 and Level 5

Script development is the responsibility of the CxA, with support from all other team

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 14 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

members. Plans and scripts should:

• Identify every test and step to be taken to complete commissioning


• Identify and describe the anticipated results for each step of the test
• Identify responsible parties for each step of each test to ensure that everyone is
available and
prepared, and to assist with schedule and budget reviews
• Identify safety precautions and personal protection equipment (PPE) requirements
for all team members

Testing should be conducted:

• Under expected normal operations in the same manner that the operations team will
operate the
data center
• Under expected maintenance conditions in the same manner that the operations
team will maintain
the data center
• In manual operation as necessary to support future upgrades and replacements

In addition, commissioning should simulate system and component failures to test fault
tolerant features, even when fault tolerance may not be a specific design assumption
because it will inform the operations team on how the infrastructure responds when
failures inevitably occur. Uptime Institute recommends testing in as many additional
scenarios as possible that make sense for the design—even scenarios that may be
outside the scope of design—to provide key information to operations about how to
respond when the facility does not function and/or respond as designed.

LEVEL 4 AND 5 COMMISSIONING PHASE ELEMENTS AND BENCHMARKS

Once construction of the data center has been substantially completed, the CxA will
lead the team through Levels 4 and 5 commissioning. The purpose of these activities is
to ensure that individual systems and the full data center ecosystem function as they
were intended in the design and OPR documents. This includes verifying
requiredcapacities and ensuring that equipment can be isolated as intended and that
the data center responds as expected to faults (see Table 5).

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 15 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

Table 5. Commissioning Phase tasks

While weather cannot be anticipated, it can and will have an impact on the final
commissioning results. Results from Level 4 and 5 commissioning activities should be
extrapolated to predict how the equipment would perform at the actual design extreme
temperatures and conditions. However, commissioning activities should be scheduled
seasonally to verify system operation for the actual extreme and varying ambient
conditions for which it was designed—especially in the event economizers are utilized
in the data center.

LEVEL 4 and 5 COMMISSIONING PHASE TECHNICAL REQUIREMENTS

Level 4 and 5 commissioning presents a unique opportunity to any data center, which
is to be able to fully test and practice equipment operation in any building condition
without any risk to critical IT load. Improvements in future operations can result when
stakeholders take advantage of this opportunity.

All critical components and systems must be fully tested—representative testing


should not be acceptable for mission critical data centers.

Level 4

• Level 4 must include load bank tests of the engine-generator sets, UPS, and UPS
battery systems at design
and rated capacities.
• Minimum continuous runtime durations of not less than eight hours are
recommended for all load bank
tests; however, continuous runtimes of up to 24 hours are considered a best practice.

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 16 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

• Ensure that load banks are distributed within critical areas to best simulate the
actual IT environment
distribution, ideally physically located within racks and with forced cooling on a
horizontal path, which
allows for more accurate and realistic mechanical system testing.
• Prior to commencing Level 5, building management and control system (BMCS)
graphics should be
completed to support the commissioning activities and to help commission the BMCS
because operations
staff will eventually rely on the BMCS including alarming and data trending features.

Level 5

• Ensure that commissioning team members and contractors are positioned


strategically throughout the
data center to monitor all systems throughout Level 5
• Size load banks as small as reasonably possible for Level 5 activities to best
simulate the
actual IT environment for more accurate and realistic mechanical system testing
• Perform Level 5 tests with the fire detection and suppression systems active, rather
than in bypass to
ensure there is no adverse impact to the critical infrastructure
• Isolate equipment at the upstream circuit breaker when performing equipment
isolations to simulate
maintenance activities rather than at the local disconnect physically located on the unit
• Complete evaluations when changes are made throughout Level 5 to fix deficiencies
and determine
which, if any, tests must be repeated/redone
• Consider the need to retest more than once in order to ensure the successful test
was not an anomaly
when an initial test is not successfully completed
• Complete testing on both utility power and on engine-generator power (or other
alternative to utility
power source)
• When simulating faults, simulate multiple fault types across separate tests on each
piece of equipment
• Ensure that sensor failures are included in the testing scope when simulating faults
on highly automated

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 17 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

data centers that rely heavily upon field sensors


• Document, identify, and validate normal operating set points, alarms, and
component settings
• Monitor alarms that are generated in the BMCS and electrical power monitoring
system (EPMS) to
ensure that they are accurate and useful
• At a minimum, take electrical load and critical area temperature readings between
each discrete test;
where data loggers are used, measurements should be logged every 30-60 seconds
• Test a variety of load conditions—25%, 50%, 75%, and 100% step loads—in order to
simulate the actual
load conditions as a data center gradually increases its critical IT load
• Test (as possible without causing damage) emergency conditions—such as N-1 and
no cooling with
design load—to provide the information necessary for the operations team to structure
future
emergency operating practices and plan for staff appropriately
• Install aisle containment strategies that are to be utilized as part of the design to
ensure the aisle
containment strategies support the infrastructure as required

Site Cleanup

• Replace the air filters for the electrical systems and heating, ventilation, and air
conditioning (HVAC)
systems following the conclusion of Level 5 commissioning
• Flush and clean piping and ductwork to ensure construction debris does not impact
future mechanical
plant performance

TURNOVER-TO-OPERATIONS PHASE ELEMENTS AND BENCHMARKS

The turnover-to-operations includes all activities associated with formally turning the
facility over to the owner and Operations. Primarily, this includes completing the final
documentation associated from the commissioning activities Levels 1-5 and utilizing
the commissioning results to finalize SOPs, MOPs, and EOPs.

Needless to say, this is a critical juncture for the data center. Soon after Level 4 and 5,
the facility will become live and support critical IT infrastructure, which begins the

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 18 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

need to maintain the facility. At this time, Operations must take all of the lessons
learned and knowledge gained from the construction and commissioning phases to
finalize the maintenance and operations program. Operations must complete this work
in a relatively short amount of time in order to minimize risk to the data center. The
longer it takes to finalize all of the documentation and processes, the longer the facility
will be at risk. Operations needs full support during this transition to ensure the overall
uptime and success of the data center.

During the Turnover-to-Operations phase, the CxA duties can include:

• Ensuring that all post-Level 5 punchlist items are successfully completed and closed
out
• Facilitating and/or coordinating infrastructure or OEM training to the operations staff
• Assisting as necessary the development of critical operating procedures.

The CxA is also responsible for gathering all testing reports and checklists from all five
levels of commissioning to create the final commissioning report. The final report to
the owner should include:

• Electrical and mechanical load and system condition readings taken at timely
intervals before major
actions are implemented in each test
• All steps, results, and system readings at every stage of the commissioning

The CxA should return to the site approximately one year following completion of
commissioning to review the building operations and to ensure there are no
outstanding items related to re-commissioning or seasonal commissioning efforts.

RE-COMMISSIONING AND FUTURE INSTALLATION PHASE ELEMENTS AND


BENCHMARKS

Commissioning should be performed any time new infrastructure is installed or any


time there is a significant change to the configuration of existing infrastructure. This
could include planned expansion of the data center or major replacements (see Table
6).

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 19 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

Table 6. Re-Commissioning Phase tasks

In data centers that are built to be scalable, it is imperative that commissioning be just
as rigorous for the follow-on infrastructure deployments to minimize risk to the facility.
Commissioning activities undoubtedly add risk to the data center, especially where
infrastructure systems are shared. However, this risk must be weighed against the risk
of performing the associated commissioning tests. If a component or system is not
going to perform as expected, the owner must decide if it is better to have this occur
during a planned commissioning activity or during an unplanned failure. While
performing a rigorous commissioning program during the initial buildout may prove the
concept of the design, the facility could potentially be at risk if all of the new
infrastructure components and systems are not tested rigorously.

These types of commissioning activities, by their very nature, occur while the systems
are supporting critical IT load. In these instances, the operations team best knows how
these activities may impact that mission critical load. During re-commissioning or
incremental commissioning, the operations team should be working in very close
collaboration with the commissioning team to ensure the integrity of the data center.
Additionally, if re-commissioning involves changes to the configuration of the data
center, Operations needs full awareness so that operating procedures that impact
maintenance and emergency activities can be updated and tested completely.

The best way to mitigate the risk of re-commissioning efforts or for follow-on phases is
to ensure that the facility is properly and extensively commissioned when it is originally

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 20 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

built. And, as part of a rigorous re-commissioning program, all of the points discussed
in this paper for standard commissioning apply to the re-commissioning efforts.
However, due to the higher level of risk with these activities, there are some additional
requirements.

RE-COMMISSIONING AND FUTURE INSTALLATION PHASE TECHNICAL


REQUIREMENTS

All of the technical requirements previously provided apply to re-commissioning and


future installation phases. However, follow-up commissioning activities also require the
following special considerations by the CxA:

• Adequate notice must be provided to service owners about the schedule, duration,
risk, and
countermeasures in place for the re-commissioning activities in order to gain
concurrence from IT
end users.
• For facilities that are based on a dual-corded IT equipment topology, the owner and
Operations
should verify that the existing critical load is appropriately dual corded where systems
that support
installed IT loads are to be commissioned.
• As load banks can introduce contaminates, load bank placement should be
considered carefully so as not
to impact the existing critical IT equipment.
• Detailed commissioning scripts must be prepared and followed during
commissioning to ensure minimal
risk to existing IT equipment. Priority should be given to the live production IT
environment, and back-out
procedures should be in place to ensure an optimal mean time to recovery (MTTR) in
case of a power
down event.
• Seasonal testing of the systems should be performed to verify performance in a
variety of climatic
conditions, including extreme ambient conditions. This also ensures that economizers,
where used, will
be tested properly.

CONCLUSION

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 21 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

Commissioning activities represent a unique opportunity for data center owners. The
ability to rigorously test the capabilities of the critical infrastructure that support the
data center without any risk to mission critical IT loads is an opportunity that should be
capitalized on to the maximum possible extent. Uptime Institute observes that this
critical opportunity is being wasted far too often in data center facilities, with not nearly
enough emphasis on the rigor and depth of the commissioning program required for a
mission critical facility until critical IT hardware is already connected.

A well-planned and executed commissioning program will help validate the capital
investment in the facility to date. It will also put the operations team in a far better
position to manage and operate the critical infrastructure for the rest of the data
center’s useful life, and ultimately ensure that the facility realizes its full potential.

Ryan Orr joined Uptime Institute in 2012 and currently serves as a


senior consultant. He performs Design and Constructed Facility
Certifications, Operational Sustainability Certifications, and
customized Design and Operations Consulting and Workshops. Mr.
Ryan Orr Orr’s work in critical facilities includes responsibilities ranging from
project engineer on major upgrades for legacy enterprise data
centers, space planning for the design and construction of multiple new data center
builds, and data center Maintenance and Operations support.

Christopher Brown joined Uptime Institute in 2010 and currently


serves as Vice President, Global Standards and is the Global Tier
Authority. He manages the technical standards for which Uptime
Institute delivers services and ensures the technical delivery staff is
Chris Brown properly trained and prepared to deliver the services. Mr. Brown
continues to actively participate in the technical services delivery
including Tier Certifications, site infrastructure audits, and custom strategic-level
consulting engagements.

Edward P. Rafter has been a consultant to Uptime Institute


Professional Services (ComputerSite Engineering) since 1999 and
assumed a full-time position with Uptime Institute in 2013 as
principal of Education and Training. He currently serves as vice
Ed Rafter president-Technology. Mr. Rafter is responsible for the daily
management and direction of the professional education staff to
deliver all Uptime Institute training services. This includes managing the activities of

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 22 of 23
Improve Project Success Through Mission Critical Commissioning 15/12/15, 6:17 PM

the faculty/staff delivering the Accredited Tier Designer (ATD) and Accredited Tier
Specialist (ATS) programs, and any other courses to be developed and delivered by
Uptime Institute.

https://journal.uptimeinstitute.com/improve-project-success-through-mission-critical-commissioning/ Page 23 of 23

You might also like