Professional Documents
Culture Documents
Process Operational Readiness and Operational Readiness Follow-On
Process Operational Readiness and Operational Readiness Follow-On
SSDC-39
SYSTEM SAFETY
DEVELOPMENT CENTER
Idaho
February 1987
UNITED STATES
DEPARTMENT OF ENERGY
D IS C L A IM E R
This report was prepared as an account of work sponsored by the United States
Government. Neither the United States nor the United States Department of
Energy nor any of their employees, nor any of their contractors, subcontractors,
or their employees, makes any warranty, expressed or implied, or assumes any
legal liability or responsibility for the accuracy, completeness, or usefulness of
any information, apparatus, product or process disclosed, or represents that its
use would not infringe privately owned rights.
Available from:
DE90 014196
Prepared by
R. J. Nertney
Published by
MASTER
DISTRIBUTION OF THIS DOCUMENT IS UNLIMITED
EXECUTIVE SUMMARY
INTRODUCTION ............................................................................................................................ 1
FIGURES
5. Development .................................................................................................................... 11
17. For Each Bottom Tier Hardware Item (O.R. Work Sheet) .......................... 35
v
\
FIGURES
(continued)
vi
PROCESS OPERATIONAL READINESS
INTRODUCTION
Operational Readiness
1. Functional Criteria
3
While the intent of both of these types of criteria is often the
same, applicability and relevance as related to specific systems are often
quite different in a practical sense.
• People
• Hardware Elements
4
Hardware-People-Procedures
Relationships
Do the
People procedures
match the people
who use them^
Do the
Hardware procedures
match the hardware?
Figure 1
How Do We Achieve Operational Readiness?
6
How do we achieve operational readiness?
Plant-personnel interlace
Personnel basically
compatible with
hardware system
T 4 Q to
actual plant systems and
Personnel conditions; hdwr accepted Plant hardware
to personnel requirements
Personnel actually
present
match hdwr
Personnel
plant in
terlace
Vcteared/
\ Go /
/ Go \
Proced
ural
system
cleared
Procedure on
lloor with
appropriate
supporting software
Intergroup liaison
field liaison &$& Plant-procedural
Personnel-procedural interlace
interface
Create detailed
procedures
S2 2630
Procedural systems
Figure 2
How Can We Prevent Oversights in Complex Systems?
8
vc
1.30
Figure 3
System
ready
l~"
Major Major Major
subsystem subsystem subsystem
#1 #2 #N
S 0853
Figure 4
Development
General
constraints Baseline configuration
• Hardware
General policies,
• Personnel
rules, standards,
procedures &
regulations, etc.
• Management
General studies, controls Safety analysis
development of reports
data, development • Preliminary
of method, etc. Methods and • Final
Information Mission specific
analysis
Current system
configuration
• Technical specs
• Operating limits Operational safety Operational
• Management controls assessment readiness
• QA requirements 10.
Specific operating
protocol
S2 1421
1.32
Fiqure 5
readiness based on the current system configuration and a specific
operating protocol indicated inside the dotted rectangle. Acceptability is
based on an operational safety assessment (Box 9) which utilizes
appropriate technical specifications, operating limits, management controls
and quality assurance criteria/information (Box 6).
Even situations in which conditions which were not under our control
were involved as causal factors can still indicate a lack of operational
readiness. For example, wind/lightning damage can indicate design
deficiencies in system protection from these natural hazards. Other
12
Case study # 1
Upstream Processes
The mishap is the result of “upstream processes”:
People
at the
job site
Procedures
and
management
controls relating
to the mishap
Hardware
Conditions
at time
of mishap
v___________________________ ______________
The “Upstream Processes” 1-2-21
S4 9235
Fiqure 6
situations may indicate that we failed to establish meteorological controls
over our operations in an adequate manner.
1. The processes are complex and involve many people and interfaces.
14
Preparation of Procedures
Prepare document
revision request
(DRR)
Obtain
field review
Obtain procedure
review board (PRB) Issue procedure
approval for field use
Maintain procedure
history file
INEL-S-32 657
1.33
Figure 7
The Hardware Configuration Control Model
1-2-25
S4 9238
Figure 8
Person removed from pool
Personnel Pool
Maintenance
Processes
1.2
Personnel Pool
(Job classification)
Selection Basic Training Examination-
RO SRO ss TS
Processes and Drill Certification
(Job) (Job) Processes it n a *.
Mgm*
Task-D ecision
Assignmen Processes
4.0
1
Task R elated
Machine
Training and Drill
-fen" S2 3967
Figure 9
Objective
Whath
LHWhath
pH What | O o O A
X o
O A O O A
UnwhatH
o A o
A O
A o o
A A A A
o O o O
INEL-S-32 716
1.8
Figure 10
Very often the manager or other reviewer will not be concerned with
"who" but only that the necessary tasks ("what") have been done; that is,
that all of the elements in each vertical column of the matrix have been
completed.
19
the complete jobs have been done or not. This means that certain
individuals must be responsible for tracking and knowing the status of work
completion.
These are not necessarily the same individuals who do the work. For
example, an installation of an instrument panel may involve electricians,
instrument personnel, pipefitters, laborers, painters, insulators and other
craftsmen and their supervisors. Each of these groups may report,
independently, that their particular work orders are complete and "clear".
The important question, however, is whether or not the control panel is
complete and functional in accordance with the intended design and
function. This may require additional inputs from the quality assurance
and operating groups, as well as "collective" overview reports from line
managers and from coordinators. Experience indicates that an important
part of the operational readiness process involves identification of
individuals who are responsible for knowing and reporting the readiness
status of each component and subsystem of the system.
Readiness Review
In any case, review should include scrutiny from two points of view by
personnel having different skills and knowledge. Review should include:
20
• Review by higher command level management to assure that the
overall system will do what is needed in the broader sense.
21
1. Use standardized trees and models for straightforward jobs.
• Person
• Hardware
• Procedures
Working to this level will often indicate the need to extend the
development further. For example, is the furniture polish referred to
above flammable or toxic? If this is the case, additional training for the
person, special operating procedures and storage cabinets (hardware) may be
necessary.
22
The Standard Tree
________
Procedures
1
............. " 1 ■ 1
6 9957
Figure 11
In all cases, the rule is to make the "punishment fit the crime" in
terms of necessary and sufficient development to provide adequate hazard
control.
24
• Reducing deficiencies, deviations, and unknowns to statements of
operational risk.
In the case of "green" items the risk (safety) analyses which have
been performed for the system are protected in terms of system design
specifications.
25
How do we use existing inspections,
reviews, checklists, etc.?
6 9963
Figure 12
For "red" items risk (safety) analyses must be reevaluated in the
context of the known variances. Acceptabi1ity of the system must be
determined in the context of the revised risk (safety) analyses.
"Blue" items are most difficult to deal with since the nature of
potential variances is unknown. The "true" but "unknown" status of the
"blue" item can lead to a very wide range of potential consequences:
27
PART II —FOLLOW-ON ACTIVITIES
• Plan
• Execution
28
MORT
ro
vo
Management
factors
Risk
implementation evaluation
Maintain a state of Design and plan of
operational readiness followup program 5 0846
Figure 13
What is operational readiness?
• Right people
• Right time
• Right place
• Right hardware
System
operationally
ready
• Initial
• Continuing
Procedures
Hardware Personnel and
management
Figure 14 controls 5 0847
Maintenance of Hardware
31
How do these Elements Fit Together
in Operational Mishaps
Program
defect
Failure to:
• Complete mission
• Within budget
• On schedule
• With acceptable side effects
5 0848
Figure 15
have a program defect leading to a specific program inadequacy which takes
the system out of its condition of operational readiness. A series of
events then carry the system through to the ultimate mishap. For example,
the specific failure could be failure to lubricate a coolant pump bearing.
The system is then "out of operational readiness" with a dry bearing. This
leads to overheating (the second event), to seizing of the bearing (event
three), and finally to loss of coolant which the pump was supposed to have
provided, with severe damage to components which were to have been cooled
by the pump.
Going back to the program defect (or, more often, defects), we find
that "upstream" defects leading to failure to lubricate the pump bearing
might be originating in a number of areas. As indicated in Figure 16, the
failure could be involved in design, fabrication or installation of the
pump in such a way that maintainability was compromised (Reference 7).
Lubrication of the pump could involve "maintenance" and/or "operational"
activities in a number of ways.
Finally, the ultimate disaster could have been averted had failure to
lubricate been detected and corrected prior to catastrophic damage (through
response to change). This entire sequence may be related to a typical
operational readiness work sheet shown in Figure 17 which includes a number
of maintenance related items.
This closes our loop back to Figures 16, 18 and 19 and indicates the
need not only to maintain our hardware systems but the need to maintain our
personnel pool with personnel who are up-to-date in their capabi1ity to
operate our systems adequately as indicated in Box 1.2 of Figure 18.
33
The Hardware Configuration Control Model
Follow-on
1-2-25
S4 9238
Figure 16
se
System
no.
Number assigned to description box
description
Brief job description using simplified nomenclature
Job
where possible
numerical
SDD
Numerical coding system for tree orientation
Responsible
person
Individual who knows when the job was done, what was
done and the final status
Installation
request
Status of and including as-built verification of installation
work
Item
processes
F ig u re
and calibration
maintenance
Required
17
Mgmt
co Task-D ecision
cn Assignmen Processes
4.0
1
Task R elated
Machine
Training and Drill
4.2 "
S2 3967
Figure 18
Preparation o Procedures
Obtain
field review
Obtain procedure
review board (PRB) Issue procedure
approval for field use
Maintain procedure
history file ^
INEL-S-32 657
1.33
Figure 19
Similarly, Figure 19 indicates, in Box 1, the requirement for a continuing
function to establish the need for new and revised procedures based on
current operational requirements.
38
REFERENCES
4
APPENDIX A
43
EXAMPLE 1
44
PAGE I
PURER EAClim
RE ADI EON
OCCUPANCY-IKE
•£»
cn
IMRIMAIIC/PROCESS
SVS1CHS MART
SIMIClIHUS/UMiaS (SClHHHIIAt tfCUC AIMINISIRAIIVE
PERSONNEE HEADY SAI'EIV SYSIEHS
uMiir Of (Ml ION OF All COHIRRE SYRENS
READY
READY
flHtlX fROiCSS
SISUHS)
PAGE 92
Figure Al
OtriNl Uf IACII CUL/UUMK AM A AS 1 PAGE 4)
^ SHOWN BY mass SVSUH KaMSYNIATION
ON PAGES SO AND SI.
IIARDWARi/PROCESS
SYSIEHS READY
EVAIUAIC EON All CINHICAIS/
(SEIRNNIIAL CYCLIC
FEED INCLUDING. BUI NOE
OPERAY ION OF All
I INI ICO 10. E.G.. EUll
PUREX PROCESS
EIEHENTS, RECOVERED
7
SIIVCR NIIHAtC. HYDRA/HIE,
SYSIEHS)
SAFEIV REVIEW
CaHHIIICE APPROVAL
SPECIAL PROCESS
PROCESS CEILS/ HISCELIANCOUS DECONIAHINATION PROJECTS COMPUTE
aiCHICAIS/ PROCESS SUPPURE
WURK ROOMS SUPPLIES SYSTEMS READY ON OCIEICO (SEE
FEEDS READY OPERAIIONS READY (DEFINE)
READY AVAII ABU N01E)
Figure A2
I’AGC 44
PKOCfSi CULS/
WORK ROOMS
READY
5
HAZARDOUS WORK
CONDITIONS
UEANIHfi AIIO
MIRK RRUCESS CORRECTED (I.E.,
DISPOSAI SYSTEM
SYSTEMS READY THROUGH IMPLEMEN
READY
TATION OF JSA
CORRECTIVE ACTION
SYSTEM)
2 k GO TO PUNCH LIST
UI.U
I PAGE 73
RAGE SS I PAGE SS PAGE 56 PAGE 79
Figure A3
CAM SS
■ mouciiw noiHiNC/
I BASIC SIBUCIUM/ I (QUIPMNI (WflHl riK rMiccnoN HONIIOBIMG
I CEIL MADE I Him (tisrcci to usi/ SVSIIH/CquiMCNI SVSIENS MAM
I lOCAlim) MAM MAM
Figure A4
EXAMPLE 2
49
oeJEcmt
FW3U7Y
REACT
UANACEM
cotm
STSTENS
READY
COUflSSONIKi OPERATWG
COUUISSONMG
TRWJTO TRAM ID
STORAGE
SUPERVISORY SIGNS,
READY
CONTROLS ALARUStOC-
COUUUMCADON
NOV. 11,1968
Figure A5
SAFETY / mmt \
-( COOES: SIAM)ARDS )
UTUTES STORAGE
SYSTEMS READY READY
READY Wrebjuuonsy
PAGE t
5 Ar^'e
FFE ft GAS
OPEMNGS
EVACUATION
km
EVACUATION GUARDS
ACTIONS
m r ADEQUACY'
OF STARS ft
COOH5
V LADDERS j
USE OF
PERSONNEL
HANOUNGJ
.PROTECTION,
ELECTRICAL
WARMNG ft SAFETY
, SONS y
HAZARD
PRCHART
NOV. 11,1906
Figure A6
PAOELT
UTILITIES
r
SEA POTABLE COMPRE&SED PURGE
H IV ELECTRICAL
WATER WATER AIR GAS
f
PAGE
CJ>
ro
HEAT AIR
SOURCE(S) DISTRIBUTION
I. /iL—
Figure A7
EXAMPLE 3
53
041WI»)t M*.
unc nucusnR mousTRies UN I-M-89 SI-8
□■a lauaa No.
6-01-84 41
SAFETY INSTRUCTION MANUAL 3uinfa»« laua Oataa
9-30-80
SwBtMt lauad ay
Safety Assessment and
READINESS REVIEWS - SAFETY Analysis
A8
F ia u re
</)
I
g
3
</»
(M.S000-080 (4.79)
54
unc nucienn ihdustriss
Ora In art
UNI-M-89
Paga No.
SI-8
6-01-84 43
SAFETY INSTRUCTION MANUAL Suoaraaaa laaua Onaa
9-30-80
lauaa By
Safety Assessment and
READINESS REVIEWS - SAFETY Analysis
A9
F ig u re
I
I
lI
1
i
55
EXAMPLE 4
This is the startup readiness tree for the Princeton Tokamak. Fusion
Test Reactor (TFTR).
56
1 1 1
TFTR EQUIP. A SOFTWARE EXPER. BLDGS & MGMT SYSTEMS A
PERSONNEL READY SAFETY READY
READY SERVICES READY PROCEDURES READY
WIS 7
J
I I
1-l| ORCAMZADON TOCAMAK 2-51 COOUNG
ANALYSIS INSffCnONS
fi
nsi cai
4.5 CICADA 5.1 CONRCUttAnO^
STRUCTUCt 5YJHM WATER SYSTEM CONTROL ROOM MGMl CONTRa
■-.E fUA ».i.i ■“ mic — ava( i....p
VA(W» Vtlltl
i.i.i • nits csit ewi
lACttllv lalftniMI A.1.1
I.I.I u
uucfiea ctiftBiA Import |r«uOk«M i.i.i - MAI a. a»t(* it niN ft«S I.I.I o»ta him itiAotifiw — falltllM A.I.T — movM *0afa
0
II CtUI.I^* «M I i.i.i
».i.»
ca»oa<*t coot IM
L. (ICAM lanafict
• .1.1 —
—
Mat raotldioo
tiCMtiaa taeo/iaon)
A.I.I - MM faoTICtlOB
A.I.A 1— LiaxnM
»» (till
A.I.t • <«*«(
II. I III
».l.l L- foafa
i t.• ** eic*»A 1*11 iMct
X2. RfCTR. POWfR 2.61 TORUS VACUUM S4 RAD. SAKTY 4J TEST CCU 4.6 nsr
TRAMNC 4 Dim. SYSTEM PUMnMG , MOMTORINO IASEMINI
fcpc hiuhno
TROCEPURU
1
I. aeafi"
>fiu>a
W»a>wa c
REVIEW (FPORR) REQUIREMENTS TREE Figure 2
Figure A10
EXAMPLE 5
58
I I I
Reactor Vessel 7.1 Rai 09070704100805041009110606060604050310
Elactrlcal Water______7J
Syatama PPS Plant Building
•nc Syatoma Protection
Vessel (to Secondary Primary System 7r and Auxiliary
include Bottom Coolant Coolant Systems
Commercial ATR Pneumatic
Head) 1 Power Systems ^ System System Emergency Irradiation
C Fire Water Heating and Facility
Top Head El Pressure Sub-system ^
2 4S0 V
System
Electrical indary Pipes Experiment
Thermal Shield Distribution SCS and Valves Reactor Inpile Tube
(Internal) 3 Controls Shutdown Facility ID-N
Diesel Power Primary System 2
Instrument System Cooling umps Colorimetric
Thimbles Tower Reactor Vessel ^ System
4 Emergency
(CAM)
Diesel Water Level Flow ^
9
Vessel Generator Cooling Exchanger2 Experiment
Penetrations S Tower Fans Neutron Level 10 Emergency Inpile Tube
and Motors ^Surpe Tankg Monitors (RAM) Facility IA-NE
4SO V Recirculation
Electrical Wide Range Flow 2
Dlatribution, PCS Controls 2 Neutron Level 11 Fire Protection NR Data System,
Chemical
Inlet Pressur^ System
Treatment
System
-
Emergency Fission Break
Generator Flow Detection Outlet
12
System UCW System 3 Subsystem Pressure 4
and Alarms
Experiment
System
PPS Power Inpile Tube
13
Hi 4 Bypass
Oamlnerallzer Mode Select 14
Total
Dilferential Structure
(Floor Loeds
Facility 2A-C
Experiment
Inpile Tube
Facility
1C-W
(AHTL)
Calm/Patm
Minor
Mods, on
All Lipt’s
Above
S2 2675
1.92
Figure All
EXAMPLE 6
60
Let’s look at an example of readiness
for a real life ground support facility.
Objective:
establish criteria,
package & let
RFP for AMO
building
>
>
>
1 II 1 1 IV
Figure A12
c'
CTl
r\>
610 044
Fiqure A13
Aircraft
Ground
Safety
X
Hangar
Future
space needs
adequate
Access Maintenance
shop
runway adequacy
Maneuvering Portable
space extinguishers
Location Material
loading
requirements
Tie downs
6 9954
EXAMPLE 7
64
ADEQUATE
& EFFECTIVE
RADIATION PROTECTION
PROGRAM
I
r~
ROUTINE EMERGENCY
OPERATIONS OPERATIONS
APPENDIX B
PROGRAM
RADIATION
111-12
PROTECTION
ORGANIZATION
Corporate
Staffing
Scope of Back-up Adequate ' Staff ^
Clearly
Responsibili Properly
for Respon Managed
Defined ties Adequate sibilities
A16
Figure
Clear Assign
Organization ment of Duties Personnel
Surveys RPM(s) Supervisors
Authorities tc Monitoring
Chart
Responsibilities
RWP Job
Issuance Technicians Clerks
Coverage
instrument Respiratory
Maintenance Protection
& Calibration
Special Multi
Chemistry Training
Assignments Discipline
Program
Management
RPCHART1 NOV 6, 86
66
APPENDIX B
APPENDIX B
M. G. Bullock
1. FACILITY MAINTENANCE
Performance Objective
1.1.1
1.1.2
69
1.1.3
1.1.4
1.2.1
1.2.2
70
1.2.3
1.2.4.
1.2.5
71
1.2.6
1.2.7
1.3.1
1.3.2
1.3.3
72
reviewed periodically to identify persistent or recurring problems and
adjustments made to preventive maintenance activities, maintenance
practices, and operating practices to further enhance plant safety and
reliabi1ity.
1.4.1
3. Equipment the operator must rely upon for plant assessment and
control.
1.4.2
1.4.3
73
requirements shall be based on manufacturer's recommendations, Technical
Specification requirements, past operating experiences, and good
maintenance practices, as applicable.
1.4.4
1.5.1
1.5.2
1.5.3
1.5.4
74
measurements made with that instrument during the interval of the out of
tolerance condition shall be evaluated and remeasurements performed, if
required. The results of the evaluation shall be documented and filed.
1.6.1
1.6.2
75
FACILITY MAINTENANCE
P t 0
Faculty Outstanding
Sub-Standard
Control of
Work Special
Control Processes
System
Administration
Assure Plant
Operating &
Design
Integrity Establish Assure
Provide a
Product
Training
Quality
Program
Establish
Approval &
Monitorilng
Controls
Evaluate 6
Provide Monitor Document Out of
PM Iblerance
Perform Status Conditions
Inspections &
Operational
Checkouts
Review of
Completed
Work Control
Documents
6 5687
Figure B1
COMPLETED SSDC PUBLICATIONS