EOP Training - Utility Loss MVSW With Tie Closure W - Attach 1-10-2020

EOP TRAINING SCENARIO
UTILITY LOSS MVSW WITH TIE CLOSURE
Scenario: At 10:45 PM while engineers are working on their inventories. UPS summary alarms start coming in
on the shift phone for all B-side electrical. There is currently no indication that generators have started. Security
has received UPS summery alarms and contacted GSOC to start a bridge call.
Conditions: It is Friday night and they are operating with only two engineers, the third is using PTO. Both
engineers are down in the shop working on inventories.
What is the first action necessary? (Verify site stability and determine what happened)
For this scenario one engineer will move to the console room while the other begins to check B side SCC &
UPS’s. Starting with the DH that is closest, in this case DH1 and then move to DH2. The other engineer has
moved into the console room and noticed on the G&W MVSW that the graphic is currently displaying an
abnormal configuration and the following displays.
1. MVSW-2B WAY 1 tie shows closed.
2. MVSW-1B WAY 2 shows open.
3. The voltage reading on CKT 839 shows 0 kV for all phases.
What is the next action? (Deep dive into system stability and Activate FREC Communication Protocol)
The engineer moving through B side UPS’s verifies that they are all in operation and that no faults still existing
on UPS’s or SCC. The engineer moving through B side passes on this information to the engineer in the console
room. Then visually verifies generator statuses.
The engineer in the console rooms confirms the UPS, MVSW, and generator status information on Power
Analytics then initiates a call to management tree to provide information (management may already have
called in due to GSOC establishing bridge call). The engineer in the console room relays the information to
management found on the G&W, UPS, and generator statuses.
What is the next action? ((Implement Generator run logs and rounds schedule, apply load banks if necessary
(Utilize Generator MOPs), to supplement load)) Contact OPPD to report outage/get status on power loss.
With the status of CKT 839 showing 0 kV for all three phases. Contact OPPD to report outage/get status on
power loss (should be done by management). OPPD responds that their medium voltage breaker serving CKT
839 has opened, that restoration of power has begun, root cause analysis is in progress, and that they will
contact us when work is complete.
What is the next action necessary? (Assess the situation, consult with management, and take action to keep
critical load online while inducement alarms continue on the SBYD Generator)
Wait for OPPD to notify management that work is complete. Then once management is notified and the G&W
shows kV back to normal condition (around 7.75 to 7.95 kV) and at the direction of management begin transfer
to generators for re-transfer back to normal configuration. *note the time of re-transfer will vary depending on
various factors i.e. the stock market is open etc.
Revision Date: 1/10/2020 Page 1 of 2 Electronic Copy – Controlled Printed Copy – Uncontrolled Fidelity Internal Information (http://infosecurity.fmr.com/spi/)
Next Steps – (Contact OPPD for status update. Once power work has been completed and the all-clear has
been given by OPPD for A-Side Power Stability, the engineer awaits a planned re-transfer window to be
coordinated by FREC Management with the business units).
Once power is returned and management has given a time and approval for transfer back to normal condition.
Use MOP-OPPD Line 839 for transfer back to normal configuration.
An event occurs during re-transfer back to normal power
During the re-transfer of DH1 B and DH2 B side back to normal power ATS DH2-B2 does not re-transfer. A red
light is illuminated on the closed transition alarms section of the ATS it is a “failure to synchronize” alarm. GEN
DH2 B1/2 has started and is running. You know the generator is in good working order because it had its semi-
annual PM just last week. You have already preformed closed transitions for GEN DH1-B1 and GEN DH2 B3/4 in
preparation to open the cross tie and return to normal and were finishing closed transitions with ATS DH2-B1 &
DH2-B2 when this occurred. What do you do now?
What is the first action?
To start, the first engineer notifies the management tree of the “failure to synchronize” problem. While this is
happening, the second engineer goes out to check engine and electrical data on the generator display. After
many minutes of going through the electrical and engine data the generator it is observed that the frequency
displayed in the electrical data is 59.8 Hz. The engineers realize that the normal setting on the generator
frequency dial is supposed to be around 60.1 Hz so that the generator always leads the utility. The
potentiometer below the generator display also shows that it is dialed in at 59.8Hz.
Next steps.
Noticing that the potentiometer is set incorrectly the engineers notify management of the problem and get
permission to adjust the potentiometer back to its original setting. Once permission is given the engineers
adjust the locking potentiometer back to the 60.1Hz setting.
What next?
After the settings have been set correctly, the engineers move back to the ATS to attempt to re-initiate transfer
to generator. Once the go ahead has been given by management the ATS transfer is re-done, transfer is
successful, and MOP-OPPD Line 839 is followed through to completion.
Discussion: - What steps need to be taken in what order?
How do you ensure communication occurs between the shifts at turn-over?
What can be learned from this scenario? (Things don’t always go as planned, and at times you must deviate
from MOP steps in a controlled fashion to react to problems that arise ((aka troubleshooting)). This happens in
conjunction with Management involvement and approval.
MOP
TRANSFERING BUILDING TO 1 FEED – OPPD LINE 839 OUTAGE
Description:
Transferring Building to 1 Feed from OPPD. For a planned OPPD Line 839 outage.
Change Management Number: ____________________________
Procedure:
Verify Utility is stable and ready
Building closed transition to generators
Open Feeder breaker from OPPD Line 839, Close Tie
Open Tie, Close Feeder breaker from OPPD Line 839
Transfer G&W back to normal configuration
PPE:
Safety shoes, safety glasses and hearing protection
Two Man Rule:

During all steps of the Method of Procedure, the two engineer rule must be followed. Specifically; one Engineer reads aloud the instruction and after
acknowledgement of understanding, the other engineer must perform the action and both engineers must acknowledge that each step is checked off
before proceeding to the next action.
LOTO:
Follow lock out/tag out procedure approved for Fidelity West Data Center
Contacts:
Engineering: Craig Ball (402-651-6703) DC OPS: Larry Kruse (402-938-9014)
Engineering: Kevin McColgan (508-294-5951) DC OPS: Bob Lobsiger (402-938-9007)
Engineering: Ray Brock (402-297-0248) DC OPS: Terry Schaeffer (402-938-9018)
Engineering: Brian Sweet (402-281-7214) Security: (402-938-9100)
☐ 1. Conduct safety meeting with all parties and discuss plan of work, MOP and safety countermeasures required.
☐ a. Complete safety sign-off sheet
☐ 2. Verify and document that all breaker positions and/or unit set points are in expected (normal operation) positions before beginning work
☐ 3. Any waste products will be removed by vendor at completion of job
☐ 4. Contact Security Office and notify them that system will be taken offline
☐ 5. Closed Transition to Generators for Utility Line 1 CKT 839 Feed and Utility Line 2 CKT 842
☐ a. Utility Line Feed 1, CKT 839 effects DH1 B-1, DH2 B1, B2, B3 and B4
☐ b. Utility Line Feed 2, CKT 842 effects DH1 A-1, DH2 A1, A2, A3 and A4 and SB
☐ b. See approved MOP’s for Closed Transition to Generators for effected electrical rooms
☐ 6. Verify all loads are carried by Generators for effected electrical rooms
☐ 7. At the G & W Workstation

☐ a. Double click on key to loop automation
☐ b. When prompted, enter (all Caps) SEL for user and SEL for password. Click Ok
☐ c. Click on button to disable loop automation
☐ 8. At MVSW 1B , Open Way 2 Utility line 1, CKT 839

☐ a. At the G&W main screen double click on MVSW 1B located at upper left side of screen
☐ b. On MVSG-1B operator screen double click on keyed icon Way 2 Utility Feed 1
☐ c. Click on open breaker Icon in the upper right-hand side of the screen.
☐ d. A password will be prompted to activate breaker. Password is SEL all caps enter to activate, close command will be sent
☐ 9. At MVSW 2B, Close “Way 1” (TIE)

☐ a. Double click on 2B MVSW located at upper left side of screen
☐ b. On MVSW-2B operator screen double click on keyed Way 1, Tie
☐ c. Close Way 1 SW 2B1 in upper right hand of screen
☐ d. A password will be prompted to activate breaker. Password is SEL all caps enter to activate, close command will be sent
☐ 10. Transfer Generators to Alternate (Utility Line 842)
☐ a. See approved MOP’s for returning to Utility from Generator
☐ 11. After OPPD work is completed and utility line 839 Feed is stable, closed transition to generator on Utility Feed 842
☐ b. See approved MOP’s for Closed Transition to Generators for effected electrical rooms
☐ 12. Follow Approved MOP for Return to Normal after Single OPPD Outage
NAME:___________________________________ DATE:___________________ TIME:_____________________
NAME:___________________________________ DATE:___________________ TIME:_____________________
SAFETY SIGN-OFF SHEET
MOP
TRANSFERING BUILDING TO 1 FEED – OPPD LINE 839 OUTAGE
PRINTED NAME COMPANY SIGNATURE DATE
MOP
SW4-DH1-B1 CLOSED TRANSITION TRANSFER TO GENERATOR WITH ADDITIONAL LOAD OPTION
Description:
This procedure will enable closed transition transfer of SW4-DH1-B1 from utility power to generator with the option of additional load
using permanent 700 kW load bank and then closed transition retransfer back to utility power. This procedure is also used for Storm
Avoidance
Procedure:
Verify SW4-DH1-B1 systems are in normal configuration
Transfer SW4-DH1-B1 to Generator
Additional Load Option
Retransfer
Return to Normal
PPE:
Safety glasses, leather gloves, safety shoes, hearing protection
Two Man Rule:

During all steps of the Method of Procedure, the two engineer rule must be followed. Specifically; one Engineer reads aloud the instruction
and after acknowledgement of understanding, the other engineer must perform the action and both engineers must acknowledge that
each step is checked off before proceeding to the next action
LOTO:
Contacts:
☐ 2. Verify and document that all breaker positions and/or unit set points are in expected (normal operation) positions before
beginning work
☐ 4. Contact Security Office and notify them that (SW4-DH1-B1) will be taken to generator and to expect alarms/notifications
☐ 5. (SW4-DH1-B1) normal configuration

☐ a. Verify no (SW4-DH1-B1) or (SW4-DH1-A1) alarms that would adversely affect critical systems during procedure
☐ b. (SW4-DH1-B1) normal breaker configuration and no (UPS-DH1-B1) (UPS-DH1-B1-2) (SCC) or (UPSM-DH1-B1) alarms that
would affect critical load
☐ 6. (GEN-DH1-B1) ready
☐ a. (GEN-DH1-B1) clear of personnel and safe to run
☐ b. Generator Run & Emissions Log entries
☐ 7. Before Generator Run

☐ a. The measuring stick and water indicator paste is on the inside South wall on each Generator
☐ b. Unscrew the white 4” pipe cap next to the fuel level indicator on the South side of the Generator Enclosure
☐ c. Take the measuring stick and add the water indicator paste on the bottom 6” of the stick, slide it down the pipe to get
accurate measure of fuel in the tank
☐ d. Fuel Level (inches) __________
☐ e. If water is detected on stick (paste will turn red) advise your Lead Engineer and Senior Manager of Engineering
☐ f. Wipe off the excess fuel and paste from the measuring stick and hang it back up
☐ g. Re-screw the 4”pipe cap back on the pipe
☐ h. Discard dirty fuel rag in oil rag containment
☐ 8. (SW4-DH1-B1) Closed Transition Transfer to Generator
☐ a. Verify operator at SW4-DH1-B1 remote Generator/Breaker Control Panel located outside Electrical Room B in passageway
☐ b. At (SW4-DH1-B1) remote Generator/Breaker Control Panel, turn the “Preferred Source Selector Switch” key from Utility
to the GEN position. Result: (GEN-DH1-B1) starts and Closed Transition Transfer is made. (SW4-DH1-B1) on generator
☐ c. Verify (SW4-DH1-B1) on generator at HMI/control panel
☐ 9. Review no alarms and take readings

☐ a. Verify no breaker trips on switchboards (SW4-DH1-B1), (UDP4-DH1-B1) and (UDPM4-DH1-B1)
☐ b. No alarms other than on generators, normal operation (UPS-DH1-B1-1) & (UPS-DH1-B1-2) and (UPSM-DH1-B1)
☐ c. Verify generator voltage, amps, kW, HZ at HMI
☐ d. At (GEN-DH1-B1) take Run Log readings right after start and every 30 - 60 minutes. Emissions Log - before and after run is
complete
Additional Load Option

This section of procedure may be used if the operator desires to put additional load on (GEN-DH1-B1)
☐ 1. At (GCB-DH1-B1) Switchboard, turn NORMAL / PARALLEL LOAD BANK selector switch to PARALLEL LOAD BANK
☐ 2. Add load bank from (GCB-DH1-B1)

☐ a. At (GCB-DH1-B1) Switchboard, turn (Operation Mode Selector Key Switch) from NORMAL to LB TEST position.
Note: Momentary key switch is spring loaded to return to the NORMAL position when released.
Result: Portable Generator / Load Bank Breaker CLOSES
☐ 3. Apply Load
☐ a. At (GCB-DH1-B1) load bank control panel, turn on control power by pressing Control Power On soft key.
Result: Load bank cooling fan will come on enabling load to be applied to load bank
☐ b. At (GCB-DH1-B1) load bank control panel, select MANUAL LOAD soft key
☐ c. At (GCB-DH1-B1) load bank control panel, select Master Load soft key
☐ d. At (GCB-DH1-B1) load bank control panel, select desired load by pressing individual load soft keys
☐ e. At (GCB-DH1-B1) load bank control panel, verify voltage, amperage and kW on metering screen
☐ 4. At (SW4-DH1-B1) Switchboard Controls HMI, verify load bank graphic ON and load bank circuit (RED)
☐ 5. At EPMS monitor, verify (GEN-DH1-B1) metering and alarms
☐ 6. At generator, record load bank log numbers every 30 – 60 minutes for duration of run
__________________________________________________________________________________________________________________
Remove Load
☐ 1. Remove load bank load from (GCB-DH1-B1)

☐ a. At (GCB-DH1-B1) load bank control panel, select MANUAL LOAD and remove load by pressing individual load soft keys
previously enabled on HMI until there is no load
☐ b. At (GCB-DH1-B1) load bank control panel, press Master Load soft key to disable all individual load soft keys
☐ c. At (GCB-DH1-B1) load bank control panel, verify 400 volts, no amperage and no kW on metering screen
☐ d. Load bank to run at no load for minimum of 5 minutes for cool down
☐ e. At completion of 5 minute load bank cool down, shut off load bank cooling fan by pressing Control Power Off soft key on
(GCB-DH1-B1) load bank controller HMI
☐ 2. At (GCB- DH1-B1) Switchboard, turn NORMAL / PARALLEL LOAD BANK selector switch to NORMAL
Result: Load Bank Breaker OPENS cutting off generator power to load bank
☐ 3. At (GCB-DH1-B1) Switchboard, press (END LOAD BANK TEST) button to OPEN (PORTABLE GENERATOR / LOAD BANK BREAKER).
☐ 10. Prepare for retransfer of (SW4-DH1-B1)
☐ a. Notify Security that SW4-DH1-B1 will be returned to Utility. Generator logs available for recording
☐ b. Verify (UPS-DH1-B1-1) (UPS-DH1-B1-2) (UPSM-DH1-B1) are online and no alarms present
☐ c. . For Storm Avoidance, confirm utility is stable, verify with OPPD. (800-554-6773) for representative or if no response
from Representative call Account Executive-Ron Mahoney on Cell (402-616-7237)
☐ d. For Storm Avoidance, verify G&W MV Switchgear is in Normal configuration and in AUTO, if not perform G&W (Return to
Verify Normal MOP)
☐ 11. (SW4-DH1-B1) Closed Transition Transfer back to Utility

☐ a. Verify operator at SW4-DH1-B1 remote Generator/Breaker Control Panel
☐ b. At (SW4-DH1-B1) remote Generator/Breaker Control Panel, turn the “Preferred Source Selector Switch” Key from Gen to
the Utility position
☐ c. Press the “Retransfer Initiate” button on control panel. This will start Sync Timer. Once the Generator and Utility are in
phase Utility Breaker will CLOSE and the Generator Breaker will OPEN. (If the sync time delay times out the HID will
display “Fail to Sync alarm”. Depress the (Alarm Reset) button to attempt another transfer)
☐ d. Verify (SW4-DH1-B1) is served from utility, generator on 5 minute cool down timer
☐ e. Verify (GEN-DH1-B1) has shut down after cool down timer has expired
☐ f. Check for HMI / control panel alarms and reset
☐ 12. Review alarms and return to normal ready state

☐ a. Switchboards (SW4-DH1-B1), (UP4-DH1-B1) and (UPM4-DH1-B1) no breaker trips or alarms
☐ b. (UPS-DH1-B1-1) (UPS-DH1-B1-2) and (UPSM-DH1-B1) no alarms
☐ c. EPMS
☐ d. BMS
☐ e. Verify (GEN-DH1-B1) has shut down after 5 minute engine cool down timer
☐ f. Complete Generator Run Log & Emissions Log on generator and BMS
☐ 13. After Generator Run

☐ a. Unscrew the white 4” pipe cap next to the fuel level indicator on the South side of the Generator Enclosure
☐ b. Take the measuring stick and slide it down the pipe to get accurate measure of fuel in the tank
☐ c. Fuel Level (inches) __________
☐ d. Wipe off the excess fuel from the measuring stick and hang it back up
☐ e. Re-screw the 4”pipe cap back on the pipe
☐ f. Discard dirty fuel rag in oil rag containment
☐ 14. Verify and document that all breaker positions and/or unit set points have been returned to expected (normal operation)
position
☐ 15. Contact Security Office and notify them of end of operations
NAME:___________________________________ DATE:___________________ TIME:_____________________
NAME:___________________________________ DATE:___________________ TIME:_____________________
MOP
MOP
SW4-DH2-B1/B2 QUARTERLY CLOSED TRANSITION TRANSFER TO GENERATOR
WITH FULL LOAD
Description:
This procedure will enable closed transition transfer of SW4-DH2-B1 and SW4-DH2-B2 from utility power to generator with full load using
permanent 700 kW load bank and then closed transition retransfer back to utility power.
Procedure:
Verify SW4-DH2-B1 and SW4-DH2-B2 systems are in normal configuration
Transfer SW4-DH2-B1 and SW4-DH2-B2 to Generator
Load Bank
Retransfer
Return to Normal
PPE:
Two Man Rule:

During all steps of the Method of Procedure, the two engineer rule must be followed. Specifically; one Engineer reads aloud the
instruction and after acknowledgement of understanding, the other engineer must perform the action and both engineers must
acknowledge that each step is checked off before proceeding to the next action
LOTO:
Contacts:
beginning work
☐ 5. Verify on ETROL & EPMS no alarms for DH2 that would affect operations
☐ 6. From BMS record (GEN-DH2-B1/2) fuel and DEF levels in Generator Emissions Run Log
☐ 7. At (GEN-DH2-B1/2) verify
☐ a. (GEN-DH2-B1/2) clear of personnel and safe to run

☐ 9. Switchboard kW and ATS verification
☐ a. Verify kW load at (SW4-DH2-B1) & (SW4-DH2-B2) switchboards power meters for load banking
☐ b. In (DH2-B1/2) Electrical Room, verify (ATS-DH2-B1) & (ATS-DH2-B2) normal status
☐ 10. (ATS-DH2-B1) Closed Transition Transfer to Generator

☐ a. At (ATS-DH2-B1) – Initiate transfer using the “Transfer Control” selector switch; turn the switch to the “Transfer
Test” position and hold for 15 sec
☐ b. Once (GEN-DH2-B1/2) is at rated voltage and frequency ATS will check sync between generator and utility then close
transition transfer from NORMAL to EMERGENCY
☐ c. At (ATS-DH2-B1) verify transfer switch connected to EMERGENCY; RED light is ON. Result: (ATS-DH2-B1) is now
served from (GEN-DH2-B1/2)
☐ 11. Verify alarms

☐ a. (UPS-DH2-B1) and (UPSM-DH2-B1) “On Generator” notice and no alarms that would affect data center operations
☐ b. Switchboards (SW4-DH2-B1), (UP4-DH2-B1) and (UPM4-DH2-B1) no breaker trips or alarms

☐ a. At (ATS-DH2-B2) – Initiate transfer using the “Transfer Control” selector switch; turn the switch to the “Transfer
Test” position and hold for 15 sec
☐ c. At (ATS-DH2-B2) verify transfer switch connected to EMERGENCY; RED light is ON. Result: (ATS-DH2-B2) is now
served from (GEN-DH2-B1/2)

☐ 14. (GEN-DH2-B1/2) LOGGING

☐ a. At generator controller check metering for 400 volts phase - phase, 60.10 Hz and no alarms. Note: Hearing protection
required
☐ b. Record engine and generator data for load bank log
____________________________________________________________________________________________________________
Load Bank
☐ 1. (GCB-DH2-B1/2) LOAD BANK BREAKER OPERATION

☐ a. At (GCB-DH2-B1/2) Switchboard, turn (Operation Mode Selector Switch) from NORMAL to LB TEST position
☐ b. At (GCB-DH2-B1/2) Switchboard, CLOSE (Portable Generator / Load Bank) breaker using OPEN / CLOSE switch.
Breaker is now closed and supplying power to load bank via (DH2-B1/2 B3/4-ATS)
☐ 2. At (DH2-B1/2 B3/4-ATS), open doors and verify transfer switch is connected to load
☐ a. If (Transfer Switch Connected to Emergency) light is illuminated close doors and proceed to next step. If not
connected to (Emergency Source) turn (Master Transfer Control) to (Transfer To Emergency). Both (Transfer Switch
Connected to Emergency) & (Emergency Source Accepted) lights should be illuminated indicating (GEN-DH2-B1/2)
power is transferred to (GEN-DH2-B1/2 B3/4-Load Bank)
☐ 3. APPLY LOAD
☐ a. At load bank control panel in (GCB-DH2-B1/2), turn on cooling fan by pressing CONTROL POWER ON soft key.
Result: Load bank cooling fan will come on enabling load to be applied to generator
☐ b. At load bank control panel, select MANUAL LOAD soft key
☐ i. Select MASTER LOAD soft key to apply load to generator
☐ ii. Select desired load by pressing individual load soft keys. NOTE: Maximum applied load 700 kW
☐ iii. At load bank control panel, verify voltage, amperage and kW on metering screen
☐ 4. VERIFY LOAD
☐ a. At (GEN-DH2-B1/2), record load bank log numbers every 30 minutes for duration of run
☐ b. At EPMS monitor, verify (GEN-DH2-B1/2) metering and alarms
____________________________________________________________________________________________________________
Remove Load
☐ 1. Remove load
☐ a. At load bank control panel in (GCB-DH2-B1/2), select (MANUAL LOAD) soft key and remove load by pressing
individual load soft keys previously enabled on HMI until there is no load
☐ b. Press (MASTER LOAD) soft key to disable all individual load keys
☐ c. Verify 400 volts, no amperage and no kW on metering screen
☐ d. Load bank to run no load for minimum of 5 minutes for cool down
☐ e. At completion of 5 minute load bank cool down, shut off load bank cooling fan by pressing Control Power Off soft
key on load bank controller HMI

☐ a. At (GCB-DH2-B1/2) switchboard, turn (Operation Mode Selector Switch) from “LB TEST” to NORMAL position
☐ b. At (GCB-DH2-B1/2) switchboard, Open (Portable Generator / Load Bank) breaker using OPEN / CLOSE switch.
Result: (Portable Generator / Load Bank) breaker OPENS cutting off generator power to (DH2-B1/2 B3/4-ATS) &
(GEN-DH2-B1/2 B3/4-Load Bank)
☐ 15. In DH2-B1 Electrical Room, prepare for retransfer

☐ a. Inform Security of retransfer
☐ b. UPS-DH2-B1, UPS-DH2-B2 and UPSM-DH2-B1, UPSM-DH2-B2 “On Generator” notice and no alarms that would affect
data center operations
☐ c. Switchboards (SW4-DH2-B1 and SW4-DH2-B2), (UP4-DH2-B1 and UP4-DH2-B2) and (UPM4-DH2-B1 and UPM4-DH2-B2)
no breaker trips or alarms
☐ 16. (ATS-DH2-B1) Close Transition Transfer back to utility from generator

☐ a. At (ATS-DH2-B1) verify (Normal Source Accepted) GREEN Light is ON
☐ b. Turn the “Transfer Control” Switch to the “Retransfer Delay Bypass” position
☐ c. ATS sync check will start and once sources are in acceptable range the ATS will automatically close transition
retransfer back to NORMAL. Generator 5 minute cool down timer started
☐ d. Verify at ATS the “Transfer Switch Connected To Normal”, GREEN light is ON
☐ e. In DH2-B1/2 Electrical Room, verify load at (SW4-DH2-B1 and SW4-DH2-B2) switchboard power meters
☐ f. In DH2-B1/2 Electrical Room, verify no alarms; (UPS-DH2-B1), (UPSM-DH2-B1)

☐ c. ATS sync check will start and once sources are in acceptable range the ATS will automatically close transition
retransfer back to NORMAL. Generator 5 minute cool down timer started
☐ 18. (GEN-DH2-B1/2) Final

☐ a. Perform final walkthrough inspection of (GEN-DH2-B1/2)
☐ b. No leaks, no breaker panel trips, no generator alarms, generator in AUTO
☐ c. At (GEN-DH2-B1/2) record final load bank and emissions log numbers
☐ 19. Verify on ETROL, BMS and EPMS that all alarms from load bank operations have cleared and data center returned to
normal ready state

☐ 21. On BMS, record final fuel and DEF level numbers for Generator Emissions Log
position
NAME:___________________________________ DATE:___________________ TIME:_____________________
NAME:___________________________________ DATE:___________________ TIME:_____________________
MOP
SW4-DH2-B1/B2 QUARTERLY CLOSED TRANSITION TRANSFER TO GENERATOR WITH FULL LOAD
MOP
SW4-DH2-B3/B4 QUARTERLY CLOSED TRANSITION TRANSFER TO
GENERATOR WITH FULL LOAD
Description:
This procedure will enable closed transition transfer of SW4-DH2-B3 and SW4-DH2-B4 from utility power to generator with full load using
permanent 700 kW load bank and then closed transition retransfer back to utility power.
Procedure:
Verify SW4-DH2-B3 and SW4-DH2-B4 systems are in normal configuration
Transfer SW4-DH2-B3 and SW4-DH2-B4 to Generator
Load Bank
Retransfer
Return to Normal
PPE:
Two Man Rule:

During all steps of the Method of Procedure, the two engineer rule must be followed. Specifically; one Engineer reads aloud the
instruction and after acknowledgement of understanding, the other engineer must perform the action and both engineers must
acknowledge that each step is checked off before proceeding to the next action
LOTO:
Contacts:
beginning work
☐ 5. Verify on ETROL & EPMS no alarms for DH2 that would affect operations
☐ 6. From BMS record (GEN-DH2-B3/4) fuel and DEF levels in Generator Emissions Run Log
☐ 7. At (GEN-DH2-B3/4) verify
☐ a. (GEN-DH2-B3/4) clear of personnel and safe to run

☐ 9. Switchboard kW and ATS verification
☐ a. Verify kW load at (SW4-DH2-B3) & (SW4-DH2-B4) switchboards power meters for load banking
☐ b. In (DH2-B3/4) Electrical Room, verify (ATS-DH2-B3) & (ATS-DH2-B4) normal status

☐ a. At (ATS-DH2-B3) – Initiate transfer using the “Transfer Control” selector switch; turn the switch to the “Transfer Test”
position and hold for 15 sec
☐ c. At (ATS-DH2-B3) verify transfer switch connected to EMERGENCY; RED light is ON. Result: (ATS-DH2-B3) is now served
from (GEN-DH2-B3/4)


☐ a. At (ATS-DH2-B4) – Initiate transfer using the “Transfer Control” selector switch; turn the switch to the “Transfer Test”
position and hold for 15 sec
☐ c. At (ATS-DH2-B4) verify transfer switch connected to EMERGENCY; RED light is ON. Result: (ATS-DH2-B4) is now served
from (GEN-DH2-B3/4)

☐ 14. (GEN-DH2-B3/4) LOGGING

☐ a. At generator controller check metering for 400 volts phase - phase, 60.10 Hz and no alarms. Note: Hearing protection
required
☐ b. Record engine and generator data for load bank log
__________________________________________________________________________________________
Load Bank

☐ a. At (GCB-DH2-B3/4) Switchboard, turn (Operation Mode Selector Switch) from NORMAL to LB TEST position
☐ b. At (GCB-DH2-B3/4) Switchboard, CLOSE (Portable Generator / Load Bank) breaker using OPEN / CLOSE switch.
Breaker is now closed and supplying power to load bank via (DH2-B1/2 B3/4-ATS)
☐ 2. At (DH2-B1/2 B3/4-ATS), open doors and verify transfer switch is connected to load
☐ a. If (Transfer Switch Connected to Normal) light is illuminated close doors and proceed to next step. If not
connected to (Normal Source) turn (Master Transfer Control) to (Transfer To Normal). Both (Transfer Switch
Connected to Normal) & (Normal Source Accepted) lights should be illuminated indicating (GEN-DH2-B3/4) power is
transferred to (GEN-DH2-B1/2 B3/4-Load Bank)
☐ 3. APPLY LOAD
☐ a. At load bank control panel in (GCB-DH2-B1/2), turn on cooling fan by pressing CONTROL POWER ON soft key.
Result: Load bank cooling fan will come on enabling load to be applied to generator
☐ b. At load bank control panel, select MANUAL LOAD soft key
☐ i. Select MASTER LOAD soft key to apply load to generator
☐ ii. Select desired load by pressing individual load soft keys. NOTE: Maximum applied load 700 kW
☐ iii. At load bank control panel, verify voltage, amperage and kW on metering screen
☐ 4. VERIFY LOAD
☐ a. At (GEN-DH2-B3/4), record load bank log numbers every 30 minutes for duration of run
☐ b. At EPMS monitor, verify (GEN-DH2-B3/4) metering and alarms
_____________________________________________________________________________________________________
Remove Load
☐ 1. Remove load
☐ a. At load bank control panel in (GCB-DH2-B1/2), select (MANUAL LOAD) soft key and remove load by pressing individual
load soft keys previously enabled on HMI until there is no load
☐ b. Press (MASTER LOAD) soft key to disable all individual load keys
☐ c. Verify 400 volts, no amperage and no kW on metering screen
☐ d. Load bank to run no load for minimum of 5 minutes for cool down
☐ e. At completion of 5 minute load bank cool down, shut off load bank cooling fan by pressing Control Power Off soft key
on load bank controller HMI

☐ a. At (GCB-DH2-B3/4) switchboard, turn (Operation Mode Selector Switch) from “LB TEST” to NORMAL position
☐ b. At (GCB-DH2-B3/4) switchboard, Open (Portable Generator / Load Bank) breaker using OPEN / CLOSE switch.
Result: (Portable Generator / Load Bank) breaker OPENS cutting off generator power to (DH2- B1/2 B3/4-ATS) &
(GEN-DH2-B1/2 B3/4-Load Bank)
☐ 15. In DH2-B3 Electrical Room, prepare for retransfer

☐ a. Inform Security of retransfer
☐ b. UPS-DH2-B3, UPS-DH2-B4 and UPSM-DH2-B3, UPSM-DH2-B4 “On Generator” notice and no alarms that would affect
data center operations
☐ c. Switchboards (SW4-DH2-B3 and SW4-DH2-B4), (UP4-DH2-B3 and UP4-DH2-B4) and (UPM4-DH2-B3 and UPM4-DH2-B4)
no breaker trips or alarms

☐ c. ATS sync check will start and once sources are in acceptable range the ATS will automatically close transition retransfer
back to NORMAL. Generator 5 minute cool down timer started

☐ c. ATS sync check will start and once sources are in acceptable range the ATS will automatically close transition retransfer
back to NORMAL. Generator 5 minute cool down timer started
☐ 18. (GEN-DH2-B3/4) Final

☐ a. Perform final walkthrough inspection of (GEN-DH2-B3/4)
☐ b. No leaks, no breaker panel trips, no generator alarms, generator in AUTO
☐ c. At (GEN-DH2-B3/4) record final load bank and emissions log numbers
☐ 19. Verify on ETROL, BMS and EPMS that all alarms from load bank operations have cleared and data center returned to normal
ready state

☐ 21. On BMS, record final fuel and DEF level numbers for Generator Emissions Log
position
NAME:___________________________________ DATE:___________________ TIME:_____________________
NAME:___________________________________ DATE:___________________ TIME:_____________________
MOP

EOP Training - Utility Loss MVSW With Tie Closure W - Attach 1-10-2020

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

EOP Training - Utility Loss MVSW With Tie Closure W - Attach 1-10-2020

Uploaded by

Copyright:

Available Formats

EOP TRAINING SCENARIO

UTILITY LOSS MVSW WITH TIE CLOSURE

How do you ensure communication occurs between the shifts at turn-over?

Change Management Number: ____________________________

Two Man Rule:

☐ 3. Any waste products will be removed by vendor at completion of job

☐ 7. At the G & W Workstation

☐ 8. At MVSW 1B , Open Way 2 Utility line 1, CKT 839

☐ 9. At MVSW 2B, Close “Way 1” (TIE)

NAME:___________________________________ DATE:___________________ TIME:_____________________

NAME:___________________________________ DATE:___________________ TIME:_____________________

PRINTED NAME COMPANY SIGNATURE DATE

Change Management Number: ____________________________

Two Man Rule:

☐ 3. Any waste products will be removed by vendor at completion of job

☐ 5. (SW4-DH1-B1) normal configuration

☐ 7. Before Generator Run

☐ 9. Review no alarms and take readings

Additional Load Option

☐ 2. Add load bank from (GCB-DH1-B1)

☐ 5. At EPMS monitor, verify (GEN-DH1-B1) metering and alarms

☐ 1. Remove load bank load from (GCB-DH1-B1)

☐ 11. (SW4-DH1-B1) Closed Transition Transfer back to Utility

☐ 12. Review alarms and return to normal ready state

☐ 13. After Generator Run

☐ 15. Contact Security Office and notify them of end of operations

NAME:___________________________________ DATE:___________________ TIME:_____________________

NAME:___________________________________ DATE:___________________ TIME:_____________________

PRINTED NAME COMPANY SIGNATURE DATE

Change Management Number: ____________________________

Two Man Rule:

☐ 3. Any waste products will be removed by vendor at completion of job

☐ 8. Before Generator Run

☐ 10. (ATS-DH2-B1) Closed Transition Transfer to Generator

☐ 11. Verify alarms

☐ 12. (ATS-DH2-B2) Closed Transition Transfer to Generator

☐ 13. Verify alarms

☐ 14. (GEN-DH2-B1/2) LOGGING

☐ 1. (GCB-DH2-B1/2) LOAD BANK BREAKER OPERATION

☐ 2. (GCB-DH2-B1/2) LOAD BANK BREAKER OPERATION

☐ 15. In DH2-B1 Electrical Room, prepare for retransfer

☐ 16. (ATS-DH2-B1) Close Transition Transfer back to utility from generator

☐ 17. (ATS-DH2-B2) Close Transition Transfer back to utility from generator

☐ 18. (GEN-DH2-B1/2) Final

☐ 20. After Generator Run

☐ 23. Contact Security Office and notify them of end of operations

NAME:___________________________________ DATE:___________________ TIME:_____________________

NAME:___________________________________ DATE:___________________ TIME:_____________________

PRINTED NAME COMPANY SIGNATURE DATE

Change Management Number: ____________________________

Two Man Rule:

☐ 3. Any waste products will be removed by vendor at completion of job

☐ 8. Before Generator Run

☐ 10. (ATS-DH2-B3) Closed Transition Transfer to Generator

☐ 11. Verify alarms

☐ 12. (ATS-DH2-B4) Closed Transition Transfer to Generator

☐ 13. Verify alarms

☐ 14. (GEN-DH2-B3/4) LOGGING

☐ 1. (GCB-DH2-B3/4) LOAD BANK BREAKER OPERATION

☐ 2. (GCB-DH2-B3/4) LOAD BANK BREAKER OPERATION

☐ 15. In DH2-B3 Electrical Room, prepare for retransfer

☐ 16. (ATS-DH2-B3) Close Transition Transfer back to utility from generator

☐ 17. (ATS-DH2-B4) Close Transition Transfer back to utility from generator

NAME:_________________________ DATE:_ TIME:_____________

NAME:_________________________ DATE:_ TIME:_____________

NAME:_________________________ DATE:_ TIME:_____________

NAME:_________________________ DATE:_ TIME:_____________

NAME:_________________________ DATE:_ TIME:_____________

NAME:_________________________ DATE:_ TIME:_____________

NAME:_________________________ DATE:_ TIME:_____________

NAME:_________________________ DATE:_ TIME:_____________