December 1994
B.E. Goldberg, K. Everhart, R. Stevens,
N. Babbitt III, P. Clemens, and L. Stout
System Engineering “Toolbox” for
DesignOriented Engineers
National Aeronautics and Space Administration
Marshall Space Flight Center • MSFC, Alabama 35812
NASA Reference Publication 1358
December 1994
B.E. Goldberg
Marshall Space Flight Center • MSFC, Alabama
K. Everhart, R. Stevens, N. Babbitt III,
P. Clemens, and L. Stout
Sverdrup Technology, Inc.
System Engineering “Toolbox” for
DesignOriented Engineers
ii
ACKNOWLEDGMENTS
The authors are very grateful for the help received from the following persons in producing this
document. Becky Mohr contributed information and illustrations concerning preliminary hazard
analyses and failure modes and effects analyses. Bryan Bachman provided a thorough review of drafts of
the entire document. Larry Thomson prepared a figure in the system safety and reliability tools section.
Jimmy Howell verified all numerical calculations in the examples. The following persons reviewed the
indicated sections of this document and offered suggestions that greatly enhanced the discussions of the
tools and methodologies presented:
Bill Cooley DesignRelated Analytical Tools
Melissa Van Dyke Trend Analysis Tools
Karl Knight System Safety and Reliability Tools
Charles Martin Statistical Tools and Methodologies
Graphical Data Interpretation Tools
Ben Shackelford Case Study
iii
Figures provided courtesy of Sverdrup Technology, Inc., Tullahoma, Tennessee.
Figure 31 Figure 321
Figure 32 Figure 322
Figure 33 Figure 323
Figure 34 Figure 324
Figure 35 Figure 325
Figure 36 Figure 326
Figure 37 Figure 327
Figure 38 Figure 328
Figure 39 Figure 329
Figure 310 Figure 330
Figure 311 Figure 331
Figure 312 Figure 332
Figure 315 Figure 333
Figure 316 Figure 334
Figure 317 Figure 335
Figure 318 Figure 336
Figure 319 Figure 340
Figure 320 Figure 341
iv
TABLE OF CONTENTS
Page
1. INTRODUCTION .................................................................................................................. 11
1.1 Purpose ......................................................................................................................... 11
1.2 Scope ......................................................................................................................... 11
1.3 Relationship With Program or Project Phases.............................................................. 12
References .................................................................................................................... 18
2. CONCEPT DEVELOPMENT TOOLS .................................................................................. 21
2.1 Trade Studies ................................................................................................................ 21
2.1.1 Description ...................................................................................................... 21
2.1.2 Application...................................................................................................... 23
2.1.3 Procedures....................................................................................................... 23
2.1.4 Example .......................................................................................................... 26
2.1.5 Advantages...................................................................................................... 28
2.1.6 Limitations ...................................................................................................... 29
2.1.7 Bibliography.................................................................................................... 29
2.2 CostVersusBenefit Studies......................................................................................... 29
2.2.1 Description ...................................................................................................... 29
2.2.2 Application...................................................................................................... 29
2.2.3 Procedures....................................................................................................... 210
2.2.4 Example .......................................................................................................... 211
2.2.5 Advantages...................................................................................................... 213
2.2.6 Limitations ...................................................................................................... 213
2.2.7 Bibliography.................................................................................................... 213
References ...................................................................................................... 214
3. SYSTEM SAFETY AND RELIABILITY TOOLS ............................................................... 31
3.1 Risk Assessment Matrix ............................................................................................... 32
3.1.1 Description ...................................................................................................... 32
3.1.2 Application...................................................................................................... 35
3.1.3 Procedures....................................................................................................... 36
3.1.4 Example .......................................................................................................... 310
3.1.5 Advantages...................................................................................................... 312
3.1.6 Limitations ...................................................................................................... 312
3.1.7 Bibliography.................................................................................................... 312
v
TABLE OF CONTENTS (Continued)
Page
3.2 Preliminary Hazard Analysis........................................................................................ 312
3.2.1 Description ...................................................................................................... 312
3.2.2 Application...................................................................................................... 313
3.2.3 Procedures....................................................................................................... 313
3.2.4 Example .......................................................................................................... 316
3.2.5 Advantages...................................................................................................... 317
3.2.6 Limitations ...................................................................................................... 318
3.2.7 Bibliography.................................................................................................... 318
3.3 Energy Flow/Barrier Analysis .......................................................................................... 318
3.3.1 Description .......................................................................................................... 318
3.3.2 Application.......................................................................................................... 319
3.3.3 Procedures........................................................................................................... 319
3.3.4 Example .............................................................................................................. 319
3.3.5 Advantages.......................................................................................................... 321
3.3.6 Limitations .......................................................................................................... 321
3.3.7 Bibliography........................................................................................................ 321
3.4 Failure Modes and Effects (and Criticality) Analysis ...................................................... 321
3.4.1 Description .......................................................................................................... 321
3.4.2 Application.......................................................................................................... 322
3.4.3 Procedures........................................................................................................... 322
3.4.4 Example .............................................................................................................. 326
3.4.5 Advantages.......................................................................................................... 328
3.4.6 Limitations .......................................................................................................... 329
3.4.7 Bibliography........................................................................................................ 329
3.5 Reliability Block Diagram................................................................................................ 330
3.5.1 Description .......................................................................................................... 330
3.5.2 Application.......................................................................................................... 332
3.5.3 Procedures........................................................................................................... 332
3.5.4 Example .............................................................................................................. 333
3.5.5 Advantages.......................................................................................................... 334
3.5.6 Limitations .......................................................................................................... 334
3.5.7 Bibliography........................................................................................................ 335
3.6 Fault Tree Analysis .......................................................................................................... 335
3.6.1 Description .......................................................................................................... 335
3.6.2 Application.......................................................................................................... 335
3.6.3 Procedures........................................................................................................... 336
3.6.3.1 Fault Tree Generation .......................................................................... 337
3.6.3.2 Probability Determination.................................................................... 337
3.6.3.3 Identifying and Assessing Cut Sets...................................................... 341
3.6.3.4 Identifying Path Sets ............................................................................ 343
vi
TABLE OF CONTENTS (Continued)
Page
3.6.4 Examples............................................................................................................. 344
3.6.4.1 Fault Tree Construction and Probability Propagation.......................... 344
3.6.4.2 Cut Sets ................................................................................................ 345
3.6.4.3 Path Sets............................................................................................... 346
3.6.5 Advantages.......................................................................................................... 346
3.6.6 Limitations .......................................................................................................... 347
3.6.7 Bibliography........................................................................................................ 347
3.7 Success Tree Analysis ...................................................................................................... 348
3.7.1 Description .......................................................................................................... 348
3.7.2 Application.......................................................................................................... 348
3.7.3 Procedures........................................................................................................... 349
3.7.4 Example .............................................................................................................. 350
3.7.5 Advantages.......................................................................................................... 351
3.7.6 Limitations .......................................................................................................... 351
3.7.7 Bibliography........................................................................................................ 351
3.8 Event Tree Analysis.......................................................................................................... 351
3.8.1 Description .......................................................................................................... 351
3.8.2 Application.......................................................................................................... 352
3.8.3 Procedures........................................................................................................... 353
3.8.4 Example .............................................................................................................. 354
3.8.5 Advantages.......................................................................................................... 354
3.8.6 Limitations .......................................................................................................... 354
3.8.7 Bibliography........................................................................................................ 356
3.9 Fault Tree, Reliability Block Diagram, and Event Tree Transformations ....................... 356
3.9.1 Description .......................................................................................................... 356
3.9.2 Application.......................................................................................................... 356
3.9.3 Procedures........................................................................................................... 356
3.9.3.1 Fault Tree to RBD Transformation...................................................... 356
3.9.3.2 RBD and Fault TreetoEvent Tree Transformation............................ 356
3.9.3.3 RBD to Fault Tree Transformation...................................................... 358
3.9.3.4 Event Tree to RBD and Fault Tree Transformation ............................ 358
3.9.4 Example .............................................................................................................. 358
3.9.5 Advantages.......................................................................................................... 361
3.9.6 Limitations .......................................................................................................... 361
3.9.7 Bibliography........................................................................................................ 361
3.10 CauseConsequence Analysis........................................................................................... 361
3.10.1 Description .......................................................................................................... 361
3.10.2 Application.......................................................................................................... 362
3.10.3 Procedures........................................................................................................... 362
3.10.4 Example .............................................................................................................. 364
3.10.5 Advantages.......................................................................................................... 364
3.10.6 Limitations .......................................................................................................... 365
3.10.7 Bibliography........................................................................................................ 366
vii
TABLE OF CONTENTS (Continued)
Page
3.11 Directed Graphic (Digraph) Matrix Analysis ................................................................... 366
3.11.1 Description .......................................................................................................... 366
3.11.2 Application.......................................................................................................... 366
3.11.3 Procedures........................................................................................................... 367
3.11.4 Example .............................................................................................................. 369
3.11.5 Advantages.......................................................................................................... 370
3.11.6 Limitations .......................................................................................................... 372
3.11.7 Bibliography........................................................................................................ 372
3.12 Combinatorial Failure Probability Analysis Using Subjective Information..................... 372
3.12.1 Description .......................................................................................................... 372
3.12.2 Application.......................................................................................................... 373
3.12.3 Procedures........................................................................................................... 373
3.12.4 Example .............................................................................................................. 374
3.12.5 Advantages.......................................................................................................... 374
3.12.6 Limitations .......................................................................................................... 374
3.13 Failure Mode Information Propagation Modeling............................................................ 376
3.13.1 Description .......................................................................................................... 376
3.13.2 Application.......................................................................................................... 376
3.13.3 Procedures........................................................................................................... 376
3.13.4 Example .............................................................................................................. 377
3.13.5 Advantages.......................................................................................................... 378
3.13.6 Limitations .......................................................................................................... 378
3.14 Probabilistic Design Analysis........................................................................................... 378
3.14.1 Description .......................................................................................................... 378
3.14.2 Application.......................................................................................................... 378
3.14.3 Procedures........................................................................................................... 380
3.14.4 Advantages.......................................................................................................... 383
3.14.5 Limitations .......................................................................................................... 383
3.15 Probabilistic Risk Assessment.......................................................................................... 384
3.15.1 Description .......................................................................................................... 384
3.15.2 Application.......................................................................................................... 384
3.15.3 Procedures........................................................................................................... 384
3.15.4 Advantages.......................................................................................................... 385
3.15.5 Limitations .......................................................................................................... 385
References .......................................................................................................... 386
viii
TABLE OF CONTENTS (Continued)
Page
4. DESIGNRELATED ANALYTICAL TOOLS.......................................................................... 41
4.1 Sensitivity (Parametric) Analysis ..................................................................................... 41
4.1.1 Description .......................................................................................................... 41
4.1.2 Application.......................................................................................................... 41
4.1.3 Procedures........................................................................................................... 41
4.1.4 Example .............................................................................................................. 43
4.1.5 Advantages.......................................................................................................... 44
4.1.6 Limitations .......................................................................................................... 44
4.2 Standard Dimensioning and Tolerancing ......................................................................... 45
4.2.1 Description .......................................................................................................... 45
4.2.2 Application.......................................................................................................... 45
4.2.3 Procedures........................................................................................................... 45
4.2.4 Example .............................................................................................................. 46
4.2.5 Advantages.......................................................................................................... 47
4.2.6 Limitations .......................................................................................................... 47
4.3 Tolerance Stackup Analysis ............................................................................................. 47
4.3.1 Description .......................................................................................................... 47
4.3.2 Application.......................................................................................................... 47
4.3.3 Procedures........................................................................................................... 47
4.3.4 Example .............................................................................................................. 48
4.3.5 Advantages.......................................................................................................... 49
4.3.6 Limitations .......................................................................................................... 49
4.3.7 Bibliography........................................................................................................ 49
References .......................................................................................................... 410
5. GRAPHICAL DATA INTERPRETATION TOOLS................................................................. 51
5.1 Scatter Diagram................................................................................................................ 51
5.1.1 Description .......................................................................................................... 51
5.1.2 Application.......................................................................................................... 51
5.1.3 Procedures........................................................................................................... 53
5.1.4 Example .............................................................................................................. 53
5.1.5 Advantages.......................................................................................................... 53
5.1.6 Limitations .......................................................................................................... 53
5.2 Control Chart .................................................................................................................... 54
5.2.1 Description .......................................................................................................... 54
5.2.2 Application.......................................................................................................... 54
5.2.3 Procedures........................................................................................................... 55
5.2.4 Example .............................................................................................................. 55
5.2.5 Advantages.......................................................................................................... 56
5.2.6 Limitations .......................................................................................................... 56
ix
TABLE OF CONTENTS (Continued)
Page
5.3 Bar Chart........................................................................................................................... 56
5.3.1 Description .......................................................................................................... 56
5.3.2 Application.......................................................................................................... 56
5.3.3 Procedures........................................................................................................... 56
5.3.4 Example .............................................................................................................. 57
5.3.5 Advantages.......................................................................................................... 57
5.3.6 Limitations .......................................................................................................... 57
5.4 TimeLine Chart .............................................................................................................. 58
5.4.1 Description .......................................................................................................... 58
5.4.2 Application.......................................................................................................... 58
5.4.3 Procedures........................................................................................................... 58
5.4.4 Example .............................................................................................................. 58
5.4.5 Advantages.......................................................................................................... 58
5.4.6 Limitations .......................................................................................................... 59
5.5 Stratification Chart............................................................................................................ 59
5.5.1 Description .......................................................................................................... 59
5.5.2 Application.......................................................................................................... 59
5.5.3 Procedures........................................................................................................... 510
5.5.4 Example .............................................................................................................. 511
5.5.5 Advantages.......................................................................................................... 511
5.5.6 Limitations .......................................................................................................... 511
5.6 Pareto Chart ...................................................................................................................... 511
5.6.1 Description .......................................................................................................... 511
5.6.2 Application.......................................................................................................... 511
5.6.3 Procedures........................................................................................................... 511
5.6.4 Example .............................................................................................................. 512
5.6.5 Advantages.......................................................................................................... 513
5.6.6 Limitations .......................................................................................................... 513
5.6.7 Bibliography........................................................................................................ 513
5.7 Histograms........................................................................................................................ 513
5.7.1 Description .......................................................................................................... 513
5.7.2 Application.......................................................................................................... 513
5.7.3 Procedures........................................................................................................... 513
5.7.4 Example .............................................................................................................. 514
5.7.5 Advantages.......................................................................................................... 514
5.7.6 Limitations .......................................................................................................... 514
References........................................................................................................... 515
x
TABLE OF CONTENTS (Continued)
Page
6. STATISTICAL TOOLS AND METHODOLOGIES................................................................. 61
6.1 “Studentt” Analysis ......................................................................................................... 61
6.1.1 Description .......................................................................................................... 61
6.1.2 Application.......................................................................................................... 61
6.1.3 Procedures........................................................................................................... 63
6.1.4 Example .............................................................................................................. 63
6.1.5 Advantages.......................................................................................................... 63
6.1.6 Limitations .......................................................................................................... 64
6.1.7 Bibliography........................................................................................................ 64
6.2 Analysis of Variance......................................................................................................... 64
6.2.1 Description .......................................................................................................... 64
6.2.2 Application.......................................................................................................... 64
6.2.3 Procedures........................................................................................................... 64
6.2.4 Example .............................................................................................................. 66
6.2.5 Advantages.......................................................................................................... 67
6.2.6 Limitations .......................................................................................................... 67
6.3 Correlation Analysis ......................................................................................................... 67
6.3.1 Description .......................................................................................................... 67
6.3.2 Application.......................................................................................................... 67
6.3.3 Procedures........................................................................................................... 67
6.3.4 Example .............................................................................................................. 68
6.3.5 Advantages.......................................................................................................... 68
6.3.6 Limitations .......................................................................................................... 68
6.4 Factorial Analysis ............................................................................................................. 68
6.4.1 Description .......................................................................................................... 68
6.4.2 Application.......................................................................................................... 68
6.4.3 Procedures........................................................................................................... 69
6.4.4 Example .............................................................................................................. 610
6.4.5 Advantages.......................................................................................................... 612
6.4.6 Limitations .......................................................................................................... 612
6.5 Confidence/Reliability Determination and Analysis ........................................................ 612
6.5.1 Description .......................................................................................................... 612
6.5.2 Application.......................................................................................................... 612
6.5.3 Procedures........................................................................................................... 613
6.5.4 Example .............................................................................................................. 614
6.5.5 Advantages.......................................................................................................... 614
6.5.6 Limitations .......................................................................................................... 614
xi
TABLE OF CONTENTS (Continued)
Page
6.6 Regression Analysis.......................................................................................................... 614
6.6.1 Description .......................................................................................................... 614
6.6.2 Application.......................................................................................................... 615
6.6.3 Procedures........................................................................................................... 615
6.6.4 Example .............................................................................................................. 616
6.6.5 Advantages.......................................................................................................... 617
6.6.6 Limitations .......................................................................................................... 617
6.7 Response Surface Methodology ....................................................................................... 617
6.7.1 Description .......................................................................................................... 617
6.7.2 Application.......................................................................................................... 617
6.7.3 Procedures........................................................................................................... 617
6.7.4 Example .............................................................................................................. 618
6.7.5 Advantages.......................................................................................................... 619
6.7.6 Limitations .......................................................................................................... 619
References .......................................................................................................... 620
7. TOTAL QUALITY MANAGEMENT (TQM) TOOLS ............................................................. 71
7.1 Benchmarking................................................................................................................... 75
7.1.1 Description .......................................................................................................... 75
7.1.2 Application.......................................................................................................... 75
7.1.3 Procedures........................................................................................................... 75
7.1.4 Example .............................................................................................................. 76
7.1.5 Advantages.......................................................................................................... 76
7.1.6 Limitations .......................................................................................................... 76
7.2 Cause and Effect Diagrams (Also Known as Fishbone Diagrams
or Ishakawa Diagrams) ....................................................................................... 77
7.2.1 Description .......................................................................................................... 77
7.2.2 Application.......................................................................................................... 77
7.2.3 Procedures........................................................................................................... 77
7.2.4 Examples............................................................................................................. 78
7.2.5 Advantages.......................................................................................................... 78
7.2.6 Limitations .......................................................................................................... 710
7.2.7 Bibliography........................................................................................................ 710
7.3 Concurrent Engineering.................................................................................................... 710
7.3.1 Description .......................................................................................................... 710
7.3.2 Application.......................................................................................................... 710
7.3.3 Procedures........................................................................................................... 710
7.3.4 Example .............................................................................................................. 711
7.3.5 Advantages.......................................................................................................... 712
7.3.6 Limitations .......................................................................................................... 712
xii
TABLE OF CONTENTS (Continued)
Page
7.4 Cost of Quality.................................................................................................................. 712
7.4.1 Description .......................................................................................................... 712
7.4.2 Application.......................................................................................................... 712
7.4.3 Procedures........................................................................................................... 712
7.4.4 Example .............................................................................................................. 714
7.4.5 Advantages.......................................................................................................... 714
7.4.6 Limitations .......................................................................................................... 715
7.5 Design of Experiments ..................................................................................................... 715
7.5.1 Description .......................................................................................................... 715
7.5.2 Application.......................................................................................................... 715
7.5.3 Procedures........................................................................................................... 715
7.5.4 Example .............................................................................................................. 716
7.5.5 Advantages.......................................................................................................... 718
7.5.6 Limitations .......................................................................................................... 718
7.5.7 Bibliography........................................................................................................ 719
7.6 Evolutionary Operation .................................................................................................... 719
7.6.1 Description .......................................................................................................... 719
7.6.2 Application.......................................................................................................... 719
7.6.3 Procedures........................................................................................................... 719
7.6.4 Example .............................................................................................................. 720
7.6.5 Advantages.......................................................................................................... 723
7.6.6 Limitations .......................................................................................................... 723
7.7 Brainstorming ................................................................................................................... 723
7.7.1 Description .......................................................................................................... 723
7.7.2 Application.......................................................................................................... 723
7.7.3 Procedures........................................................................................................... 724
7.7.4 Example .............................................................................................................. 724
7.7.5 Advantages.......................................................................................................... 725
7.7.6 Limitations .......................................................................................................... 725
7.8 Checklists.......................................................................................................................... 726
7.8.1 Description .......................................................................................................... 726
7.8.2 Application.......................................................................................................... 726
7.8.3 Procedures........................................................................................................... 726
7.8.4 Example .............................................................................................................. 726
7.8.5 Advantages.......................................................................................................... 726
7.8.6 Limitations .......................................................................................................... 726
xiii
TABLE OF CONTENTS (Continued)
Page
7.9 Delphi Technique.............................................................................................................. 727
7.9.1 Description .......................................................................................................... 727
7.9.2 Application.......................................................................................................... 727
7.9.3 Procedures........................................................................................................... 727
7.9.4 Example .............................................................................................................. 728
7.9.5 Advantages.......................................................................................................... 729
7.9.6 Limitations .......................................................................................................... 729
7.10 Nominal Group Technique ............................................................................................... 730
7.10.1 Description .......................................................................................................... 730
7.10.2 Application.......................................................................................................... 730
7.10.3 Procedures........................................................................................................... 730
7.10.4 Example .............................................................................................................. 730
7.10.5 Advantages.......................................................................................................... 732
7.10.6 Limitations .......................................................................................................... 732
7.11 Force Field Analysis ......................................................................................................... 732
7.11.1 Description .......................................................................................................... 732
7.11.2 Application.......................................................................................................... 732
7.11.3 Procedures........................................................................................................... 734
7.11.4 Example .............................................................................................................. 734
7.11.5 Advantages.......................................................................................................... 735
7.11.6 Limitations .......................................................................................................... 735
7.12 Quality Function Deployment .......................................................................................... 735
7.12.1 Description .......................................................................................................... 735
7.12.2 Application.......................................................................................................... 736
7.12.3 Procedures........................................................................................................... 736
7.12.4 Example .............................................................................................................. 737
7.12.5 Advantages.......................................................................................................... 740
7.12.6 Limitations .......................................................................................................... 740
7.12.7 Bibliography........................................................................................................ 740
7.13 Quality Loss Function....................................................................................................... 741
7.13.1 Description .......................................................................................................... 741
7.13.2 Application.......................................................................................................... 742
7.13.3 Procedures........................................................................................................... 742
7.13.4 Example .............................................................................................................. 743
7.13.5 Advantages.......................................................................................................... 744
7.13.6 Limitations .......................................................................................................... 744
xiv
TABLE OF CONTENTS (Continued)
Page
7.14 Statistical Process Control ................................................................................................ 744
7.14.1 Description .......................................................................................................... 744
7.14.2 Application.......................................................................................................... 744
7.14.3 Procedures........................................................................................................... 744
7.14.4 Example .............................................................................................................. 746
7.14.5 Advantages.......................................................................................................... 748
7.14.6 Limitations .......................................................................................................... 749
7.14.7 Bibliography........................................................................................................ 749
7.15 Flowchart Analysis ........................................................................................................... 749
7.15.1 Description .......................................................................................................... 749
7.15.2 Application.......................................................................................................... 752
7.15.3 Procedures........................................................................................................... 752
7.15.4 Example .............................................................................................................. 752
7.15.5 Advantages.......................................................................................................... 752
7.15.6 Limitations .......................................................................................................... 752
7.16 Work Flow Analysis ......................................................................................................... 753
7.16.1 Description .......................................................................................................... 753
7.16.2 Application.......................................................................................................... 754
7.16.3 Procedures........................................................................................................... 754
7.16.4 Example .............................................................................................................. 754
7.16.5 Advantages.......................................................................................................... 755
7.16.6 Limitations .......................................................................................................... 755
References .......................................................................................................... 756
8. TREND ANALYSIS TOOLS..................................................................................................... 81
8.1 Performance Trend Analysis ............................................................................................ 84
8.1.1 Description .......................................................................................................... 84
8.1.2 Application.......................................................................................................... 84
8.1.3 Procedures........................................................................................................... 85
8.1.4 Example .............................................................................................................. 87
8.1.5 Advantages.......................................................................................................... 88
8.1.6 Limitations .......................................................................................................... 88
8.1.7 Bibliography........................................................................................................ 88
8.2 Problem Trend Analysis ................................................................................................... 88
8.2.1 Description .......................................................................................................... 88
8.2.2 Application.......................................................................................................... 89
8.2.3 Procedures........................................................................................................... 810
8.2.4 Example .............................................................................................................. 811
8.2.5 Advantages.......................................................................................................... 815
8.2.6 Limitations .......................................................................................................... 815
8.2.7 Bibliography........................................................................................................ 815
xv
TABLE OF CONTENTS (Concluded)
Page
8.3 Programmatic Trend Analysis .......................................................................................... 815
8.3.1 Description .......................................................................................................... 815
8.3.2 Application.......................................................................................................... 816
8.3.3 Procedures........................................................................................................... 816
8.3.4 Example .............................................................................................................. 818
8.3.5 Advantages.......................................................................................................... 818
8.3.6 Limitations .......................................................................................................... 818
8.3.7 Bibliography........................................................................................................ 818
8.4 Supportability Trend Analysis.......................................................................................... 819
8.4.1 Description .......................................................................................................... 819
8.4.2 Application.......................................................................................................... 820
8.4.3 Procedures........................................................................................................... 821
8.4.4 Example .............................................................................................................. 822
8.4.5 Advantages.......................................................................................................... 822
8.4.6 Limitations .......................................................................................................... 822
8.4.7 Bibliography........................................................................................................ 823
8.5 Reliability Trend Analysis................................................................................................ 824
8.5.1 Description .......................................................................................................... 824
8.5.2 Application.......................................................................................................... 824
8.5.3 Procedures........................................................................................................... 825
8.5.4 Example .............................................................................................................. 825
8.5.5 Advantages.......................................................................................................... 825
8.5.6 Limitations .......................................................................................................... 826
8.5.7 Bibliography........................................................................................................ 826
References........................................................................................................... 827
Appendix A ...................................................................................................................................... A1
Appendix B ...................................................................................................................................... B1
Appendix C ...................................................................................................................................... C1
Appendix D ...................................................................................................................................... D1
Appendix E ...................................................................................................................................... E1
Appendix F ....................................................................................................................................... F1
xvi
LIST OF ILLUSTRATIONS
Figure Title Page
21. Example utility functions ................................................................................... 27
22. Example weighted factor trade study summary table ........................................ 28
31. Risk plane ........................................................................................................... 35
32. Isorisk contour usage ........................................................................................ 36
33. Risk plane to risk matrix transformation ............................................................ 37
34. Helpful hints in creating a risk assessment matrix ............................................. 38
35. Typical risk assessment matrix ........................................................................... 310
36. Severity and probability interpretations ............................................................. 311
37. PHA process flowchart ...................................................................................... 314
38. Typical PHA ...................................................................................................... 316
39. Example of system breakdown and numerical coding ....................................... 323
310. FMECA process flowchart ................................................................................. 324
311. Typical FMECA worksheet ............................................................................... 326
312. Example of an FMECA ...................................................................................... 327
313. Typical complex RBD ........................................................................................ 331
314. Example RBD ..................................................................................................... 333
315. Fault tree construction process............................................................................ 339
316. Log average method of probability estimation ................................................... 339
317. Relationship between reliability and failure probability propagation................. 340
318. Failure probability propagation through OR and AND gates ............................. 340
319. Exact solution of OR gate failure probability propagation ................................ 341
320. Example fault tree .............................................................................................. 344
321. Example of determining cut sets ........................................................................ 345
322. Example of determining path sets ...................................................................... 346
323. Success tree construction process ...................................................................... 349
xvii
LIST OF ILLUSTRATIONS
Figure Title Page
324. Example success tree ......................................................................................... 350
325. Event tree (generic case) .................................................................................... 352
326. Event tree (Bernoulli model) .............................................................................. 353
327. Example ETA ..................................................................................................... 355
328. Fault tree to RBD transformation. ...................................................................... 357
329. Deriving cut and path sets from an RBD ........................................................... 357
330. RBD to event tree transformation ...................................................................... 358
331. RBD to fault tree transformation ....................................................................... 359
332. Event tree to fault tree transformation ............................................................... 359
333. Equivalent logic RBD and fault tree .................................................................. 360
334. Relationship between cause and consequence ................................................... 362
335. Causeconsequence analysis format ................................................................... 364
336. Example causeconsequence analysis ................................................................ 365
337. Comparison between digraph and fault tree logic gates .................................... 368
338. Construction of digraph adjacency matrix ......................................................... 369
339. Example digraph matrix analysis ....................................................................... 371
340. Example combinatorial failure probability analysis .......................................... 375
341. Example failure mode information propagation model ...................................... 379
342. Load and capability transfer functions ............................................................... 382
343. Interference between load and capability density functions .............................. 383
41. Example of dimensioning and tolerancing ......................................................... 46
42. Oring joint ......................................................................................................... 48
43. Oring joint components ..................................................................................... 48
51. Scatter diagram example..................................................................................... 54
52. Control chart example......................................................................................... 55
xviii
LIST OF ILLUSTRATIONS
Figure Title Page
53. Bar chart example ............................................................................................... 57
54. Timeline chart example ..................................................................................... 59
55. Stratification (histogram) chart example ............................................................ 510
56. Pareto chart example........................................................................................... 512
57. Histogram example ............................................................................................. 514
61. Line generated with least squares method .......................................................... 616
71. Comparative benchmarking ................................................................................ 76
72. Design rework cause and effect diagram............................................................ 78
73. Cause and effect diagram on receiving telephone messages .............................. 79
74. Concurrent engineering example ........................................................................ 711
75. Standard cost of quality curve............................................................................. 713
76. Factor/level effects graph.................................................................................... 718
77. EVOP example.................................................................................................... 720
78. Sample of a partial igniter subsystem fault tree.................................................. 728
79. Fault tree sample with estimates assigned .......................................................... 729
710. Force field analysis example............................................................................... 734
711. House of quality .................................................................................................. 735
712. QFD example on automobile industry ................................................................ 739
713. Traditional view to meeting specification........................................................... 741
714. Quality loss function for NIB.............................................................................. 742
715. Quality loss function example............................................................................. 743
716. Control chart showing mean deviation for each part .......................................... 747
717. Range chart showing mean range for each part .................................................. 747
718. Pareto chart showing mean deviation for each hole guide.................................. 748
719. Control chart showing mean deviation for hole guide 1..................................... 748
xix
LIST OF ILLUSTRATIONS
Figure Title Page
720. Example of topdown flowchart ......................................................................... 749
721. Example of detailed flowchart ........................................................................... 750
722. Common flowchart symbols .............................................................................. 751
723. Work flow diagram example .............................................................................. 753
724. WFA example .................................................................................................... 755
81. Performance trend analysis example .................................................................. 87
82. Problem trend analysis example ........................................................................ 812
83. Programmatic trend analysis example ............................................................... 819
84. Supportability trend analysis example ............................................................... 823
85 Reliability trend analysis example ..................................................................... 825
xx
LIST OF TABLES
Table Title Page
11. System engineering “toolbox” function matrix ................................................... 13
12. System engineering “toolbox” project phase matrix ............................................ 15
21. Concept development tools and methodologies ................................................... 22
22. Typical weighted trade study summary table ....................................................... 25
23. Example selection criteria for costversusbenefit analyses ................................ 210
31. Symbolic logic techniques ................................................................................... 32
32. System safety and reliability tools and methodologies ........................................ 33
33. Examples of strategies to manage harmful energy flow ...................................... 320
34 . Simple RBD construction ..................................................................................... 330
35. FTA procedures .................................................................................................... 336
36. Fault tree construction symbols ........................................................................... 338
37. Probability propagation expressions for logic gates ............................................ 342
38. Causeconsequence tree construction symbols .................................................... 363
39. Combinatorial failure probability analysis subjective scale ................................ 373
41. Designrelated analytical tools and methodologies ............................................. 42
42. Sensitivity analysis calculations ........................................................................... 44
51. Graphical data interpretation tools and methodologies ........................................ 52
61. Statistical tools and methodologies ...................................................................... 62
62. Factorial analysis factors and magnitudes ........................................................... 69
63. Factorial analysis example ................................................................................... 610
71. TQM tools and methodologies ............................................................................. 72
72. Month’s cost of quality ......................................................................................... 714
73. 2
3
factorial design data.......................................................................................... 716
74. Trial, effects, and results ....................................................................................... 716
xxi
LIST OF TABLES (Continued)
Table Title Page
75. Calculation of effects ........................................................................................... 717
76. EVOP cycle No. 1 data ........................................................................................ 720
77. EVOP cycle No. 2 data ........................................................................................ 721
78. Comparison of EVOP cycle No. 1 and cycle No. 2 data ..................................... 722
79. Motor postflight checklist ..................................................................................... 727
710. Replacement technology concerns ....................................................................... 731
711. Concerns with assigned weighting factors ........................................................... 733
712. QFD matrix sample calculations .......................................................................... 737
713. Nominal hole size deviations and drill guide positions ....................................... 746
81. Trend analysis tools and methodologies .............................................................. 83
xxii
ACRONYMS
AHP Analytical hierarchy process
AHPA Analytical hierarchy process approach
AIAA American Institute of Aeronautics and Astronomics
ANOVA Analysis of variance
B/C Benefittocost
CIL Critical items list
CIM Change in mean
CSF Compliance safety factor
DAS Data acquisition system
DOE Design of experiments
DOF Degreeoffreedom
DR Discrepancy report
EF External failure
ETA Event tree analysis
EVOP Evolutionary operation
FMEA Failure modes and effects analysis
FMECA Failure modes, effects, and criticality analysis
FTA Fault tree analysis
IF Internal failure
L(y) Loss function (quality)
LCL Lower control limits
LDL Lower decision line
LIB Larger is better
xxiii
ACRONYMS (Continued)
LSL Lower specification limit
MTBF Mean time between failures
MTBR Mean time between repairs
MTTR Mean time to repair
NASA National Aeronautics and Space Administration
NGT Nominal group technique
NIB Nominal is best
PDA Probabilistic design analysis
PHA Preliminary hazard analysis
PRA Probabilistic risk assessment
PRACA Problem reporting and corrective action
OSHA Occupational Safety and Health Administration
QFD Quality function deployment
RBD Reliability block diagram
RSM Response surface methodology
SE Standard error
SESTC System Effectiveness and Safety Technical Committee
SIB Smaller is better
SME Sum of mean error
SMQ Safety and mission quality
SMR Sum of mean replicate
SPC Statistical process control
SRM Solid rocket motor
xxiv
ACRONYMS (Continued)
SSE Sum of squares error
SSR Sum of squares replication
SST Total sum of squares
STA Success tree analysis
TQM Total quality management
UCL Upper control limit
UCLR Upper control limit range
UDL Upper decision line
USL Upper specification limit
WFA Work flow analysis
xxv
xxvi
REFERENCE PUBLICATION
SYSTEM ENGINEERING “TOOLBOX” FOR DESIGNORIENTED ENGINEERS
1. INTRODUCTION
1.1 Purpose
Many references are available on systems engineering from the project management perspective.
Too often, these references are of only limited utility from the designer’s standpoint. A practicing,
designoriented systems engineer has difficulty finding any ready reference as to what tools and
methodologies are available.
The purpose of this system engineering toolbox is to provide tools and methodologies available
to the designoriented systems engineer. A tool, as used herein, is defined as a set of procedures to
accomplish a specific function. A methodology is defined as a collection of tools, rules, and postulates to
accomplish a purpose. A thorough literature search was performed to identify the prevalent tools and
methodologies. For each concept addressed in the toolbox, the following information is provided: (1)
description, (2) application, (3) procedures, (4) example, if practical, (5) advantages, (6) limitations, and
(7) bibliography and/or references.
This toolbox is intended solely as guidance for potential tools and methodologies, rather than
direction or instruction for specific technique selection or utilization. It is left to the user to determine
which technique(s), at which level of detail are applicable, and what might be the expected “value
added” for their purposes. Caution should be exercised in the use of these tools and methodologies. Use
of the techniques for the sake of “using techniques” is rarely resourceeffective. In addition, while
techniques have been categorized for recommended areas of use, this is not intended to be restrictive.
Readers are encouraged to question, comment (app. A) and, in general, use this reference as one source
among many. The reader is also cautioned to validate results from a given tool to ensure accuracy and
applicability to the problem at hand.
1.2 Scope
The tools and methodologies available to the designoriented systems engineer can be
categorized in various ways depending upon the application. Concept development tools, section 2, are
useful when selecting the preferred option of several alternatives. Among these alternatives are such
things as cost, complexity, weight, safety, manufacturability, or perhaps determining the ratio of
expected future benefits to the expected future costs.
System safety and reliability tools, section 3, address the following areas of concern: (1) identify
and assess hazards, (2) identify failure modes and show their consequences or effects, and (3) symbolic
logic modeling tools used to understand the failure mechanisms of the system. These tools are also used
to determine the probability of failure occurring or the reliability that a component will operate success
fully, either in comparative or absolute terms, as applicable.
Designrelated analytical tools, section 4, are applied to show (1) which parameters affect a sys
tem the most or least, (2) a method for specifying dimensions and tolerances, and (3) the determination
of the possibility or probability of having form, fit, or function problems with a design, or to determine a
tolerance or dimension necessary to avoid these problems.
11
When there is a desire to monitor performance, identify relationships, or reveal the most
important variables in a set of data, graphical data interpretation tools are typically applied. These tools
are discussed in section 5. Statistical tools and methodologies, section 6, compare sample statistics and
population statistics. Variations are identified and mathematical relationships are determined. Many
excellent texts are available on statistical methods, as are software packages. For this reason, this
document touches only lightly on this area.
Total quality management (TQM) tools, section 7, are applied to continuously improve perfor
mance at all levels of operation, in all areas of an organization, using all available human and capital
resources. Finally, quantitative tools that are used to identify potentially hazardous conditions based on
past empirical data are trend analysis tools, section 8. The ultimate objective for these tools is to assess
the current status, and to forecast future events.
To assist in further defining optimal areas in which each technique may be useful, table 11
provides a functional matrix which categorizes the functionality of each tool or methodology into (1)
data analysis, (2) problem identification, (3) decision making, (4) modeling, (5) prevention, (6) creative,
and (7) graphical. These functionality categories are found in reference 1.1.
Extensive research was performed in order to identify all prevalent tools and methodologies
available to the designoriented systems engineer. Nevertheless, important tools or methodologies may
have been overlooked. If a tool or methodology should be considered for this toolbox, appendix A is
provided for the reader to complete and return to the individual indicated on the form.
To further illustrate how selected tools and methodologies in this toolbox are applied, and misap
plied, appendix B provides a case study illustrating the trials and tribulations of an engineer applying his
recently acquired knowledge of the techniques to a given work assignment.
Appendix C provides a glossary of terms applicable to the tools and methodologies in this toolbox.
1.3 Relationship With Program or Project Phases
Each tool or methodology may be performed in a minimum of one of the following phases, as
described in reference 1.2, of a project design cycle.
(1) Phase A (conceptual trade studies)—a quantitative and/or qualitative comparison of
candidate concepts against key evaluation criteria to determine the best alternative.
(2) Phase B (concept definition)—the establishment of system design requirements as well as
conceptually designing a mission, conducting feasibility studies and design tradeoff
studies.
(3) Phase C (design and development)—the initiation of product development and the
establishment of system specifications.
(4) Phase D (fabrication, integration, test, and evaluation)—system verification.
(5) Phase E (operations)—the deployment of the product and performance validation.
Table 12 provides a project phase matrix for all of the tools and methodologies identified in this
toolbox. An entry of (1) for the phase means the technique is primarily performed in that phase and an
entry of (2) means the technique is secondarily performed in that phase. Though the entries in this matrix
are a result of research by the authors, other phases should be considered by the user for a particular tool
or methodology.
12
Table 11. System engineering “toolbox” function matrix—Continued
Section Tool or Methodology Data
Analysis
Problem
Identification
Decision
Making
Modeling Prevention Creative Graphical
Concept development tools
2.1 Trade studies √ √
2.2 Costversusbenefit studies √ √
System safety and reliability tools
3.1 Risk assessment matrix √ √
3.2 Preliminary hazard analysis, √ √ √ √
3.3 Energy flow/barrier analysis √ √ √ √
3.4 Failure modes and effects analysis √ √ √ √
3.5 Reliability block diagram √ √ √ √
3.6 Fault tree analysis √ √ √ √ √ √
3.7 Success tree analysis √ √ √ √ √
3.8 Event tree analysis √ √ √ √ √
3.9 Fault tree/reliability block diagram/event tree
transformations
√ √ √ √ √
3.10 Causeconsequence analysis √ √ √ √ √ √
3.11 Directed graph (digraph) matrix analysis √ √ √ √ √
3.12 Combinatorial failure probability analysis using
subjective information
√ √ √
3.13 Failure mode information propagation modeling √ √ √ √
3.14 Probabilistic design analysis √ √
3.15 Probabilistic risk assessment √ √ √ √ √
Designrelated analytical tools
4.1 Sensitivity (parametric) analysis √ √
4.2 Standard dimensioning and tolerancing √
4.3 Tolerance stackup analysis √ √ √
Graphical data interpretation tools
5.1 Scatter diagram √
5.2 Control chart √ √ √ √
5.3 Bar chart √
5.4 Timeline chart √
5.5 Stratification chart √
5.6 Pareto chart √ √ √ √
5.7 Histograms √
Note: Functionality categories found in reference 1.1.
13
Table 11. System engineering “toolbox” function matrix—Continued.
Section Tool or Methodology Data
Analysis
Problem
Identification
Decision
Making
Modeling Prevention Creative Graphical
Statistical tools and methodologies
6.1 “Studentt” analysis √ √ √
6.2 Analysis of variance √ √ √
6.3 Correlation analysis √ √ √
6.4 Factorial arrays √ √ √
6.5 Confidence/reliability determination and analysis √ √ √
6.6 Regression analysis √ √ √ √
6.7 Response surface methodology √ √ √ √
TQM tools
7.1 Benchmarking √ √
7.2 Cause and effect diagrams √ √
7.3 Concurrent engineering √ √ √
7.4 Cost of quality √ √
7.5 Design of experiments √ √
7.6 Evolutionary operation √ √ √
7.7 Brainstorming √ √ √
7.8 Checklists √ √ √
7.9 Delphi technique √ √
7.10 Nominal group technique √ √ √
7.11 Force field analysis √ √ √
7.12 Quality function deployment √ √ √
7.13 Quality loss function √ √
7.14 Statistical process control √ √ √ √
7.15 Flowchart analysis √ √
7.16 Work flow analysis √ √
Trend analysis tools
8.1 Performance trend analysis √ √ √ √
8.2 Problem trend analysis √ √ √ √
8.3 Programmatic trend analysis √ √ √ √
8.4 Supportability trend analysis √ √ √ √
8.5 Reliability trend analysis √ √ √ √
Note: Functionality categories found in reference 1.1.
14
Table 12. System engineering “toolbox” project phase matrix—Continued
Code: 1—Primary
2—Secondary
Phase A Phase B Phase C Phase D Phase E
Section Tool or Methodology Conceptual
Trade Studies
Concept
Definition
Design and
Development
Fabrication,
Integration,
Test, and
Evaluation
Operations
Concept development tools
2.1 Trade studies 1 2
2.2 Cost versus benefit studies 1 2 2
System safety and reliability tools
3.1 Risk assessment matrix 2 1
3.2 Preliminary hazard analysis, 2 1
3.3 Energy flow/barrier analysis 2 1 2
3.4 Failure modes and effects analysis 1
3.5 Reliability block diagram 1
3.6 Fault tree analysis 1 2
3.7 Success tree analysis 1 2
3.8 Event tree analysis 1 2 1
3.9 Fault tree/reliability block diagram/event tree
transformations
2 1
3.10 Causeconsequence analysis 1 2 1
3.11 Directed graph (digraph) matrix analysis 1
3.12 Combinatorial failure probability analysis using
subjective information
1
3.13 Failure mode information propagation modeling 1 2
3.14 Probabilistic design analysis 1
3.15 Probabilistic risk assessment 1
Note: Phases discussed in reference 1.2.
15
Table 12. System engineering “toolbox” project phase matrix—Continued
Code: 1—Primary
2—Secondary
Phase A Phase B Phase C Phase D Phase E
Section Tool or Methodology Conceptual
Trade Studies
Concept
Definition
Design and
Development
Fabrication,
Integration,
Test, and
Evaluation
Operations
Designrelated analytical tools
4.1 Sensitivity (parameteric) analysis 1 1
4.2 Standard dimensioning and tolerancing 1 2
4.3 Tolerance stackup analysis 1 1
Graphical data interpretation tools
5.1 Scatter diagram 1
5.2 Control chart 1
5.3 Bar chart 1
5.4 Timeline chart 1
5.5 Stratification chart 1
5.6 Pareto chart 1
5.7 Histograms 1
Statistical tools and methodologies
6.1 “Studentt” analysis 2 1 2
6.2 Analysis of variance 2 1 2
6.3 Correlation analysis 2 1 2
6.4 Factorial arrays 1 2 2
6.5 Confidence/reliability determination and analysis 1 1 1
6.6 Regression analysis 1 2
6.7 Response surface methodology 1 1
Note: Phases discussed in reference 1.2.
16
Table 12. System engineering “toolbox” project phase matrix—Continued.
Code: 1—Primary
2—Secondary
Phase A Phase B Phase C Phase D Phase E
Section Tool or Methodology Conceptual
Trade Studies
Concept
Definition
Design and
Development
Fabrication,
Integration,
Test, and
Evaluation
Operations
TQM tools
7.1 Benchmarking 2 2 1
7.2 Cause and effect diagrams 2 2 1
7.3 Concurrent engineering 2 1
7.4 Cost of quality 1
7.5 Design of experiment 1 2
7.6 Evolutionary operation 2 1
7.7 Brainstorming 1 2
7.8 Checklists 2 2 1
7.9 Delphi technique 1 1 2
7.10 Nominal group technique 1 1 2
7.11 Force field analysis 1 1
7.12 Quality function deployment 2 2 1
7.13 Quality loss function 2 1
7.14 Statistical process control 1
7.15 Flowchart analysis 1 2
7.16 Work flow analysis 1
Trend analysis tools
8.1 Performance trend analysis 2 1
8.2 Problem trend analysis 2 1
8.3 Programmatic trend analysis 1
8.4 Supportability trend analysis 1
8.5 Reliability trend analysis 2 1
Note: Phases discussed in reference 1.2.
17
REFERENCES
1.1 Brocka, B., and Brocka, M.S.: “Quality Management, Implementing the Best Ideas of the
Masters.” Business One Irwin, Homewood, Illinois 60430.
1.2 “System Engineering Process (Short Course Lecture Notebook).” Center for Systems Management
(CSM), Santa Rosa, California, September 1991.
18
2. CONCEPT DEVELOPMENT TOOLS
Trade studies and costversusbenefit studies are presented in this section. These tools are used to
select the preferred option of several alternatives. Trade studies, section 2.1, are quantitative and/or
qualitative comparison techniques to choose an alternative when considering such items as cost,
complexity, weight, safety, manufacturability, etc. Costversusbenefit studies, section 2.2, provide a
method to assess alternatives by determining the ratio of expected future benefits to expected future
costs.
A summary of the advantages and limitations of each tool or methodology discussed in this
section is presented in table 21.
2.1 TRADE STUDIES
2.1.1 Description
In general, trade (or tradeoff) studies provide a mechanism for systematic depiction of both sys
tem requirements and system design options for achieving those requirements. Once tabulated, a
comparison of relevant data (cost, complexity, weight, safety, manufacturability, etc.) is then performed
to rank those candidate design options in order of desirability.
These studies are categorized as either a weighted factor trade study or an analytical hierarchy
trade study, with the latter being a special version of the former. These techniques are described in
reference 2.1. A trade tree can be generated with either of the above two options. A trade tree is simply a
pictorial representation of how highlevel alternatives (or issues) in the decision process are logically
resolved into decreasingly lower level alternatives (or issues). A trade tree may be presented without
results or simply as a representation of options.
A weighted factor trade study is usually performed when each of the options under consideration
is very well defined and there is good definition of the program requirements as well. All factors
(program requirements) that are determined to be important, are delineated with an associated weighting
factor. The options are then assessed with respect to each of the factors and an equation is developed that
weighs this assessment. The decision is then based upon the numerical results of the analysis.
The analytical hierarchy process (AHP) is a variation of the weighted factors analysis and is the
most complex of the trade studies presented here. This approach allows for delineation of the facts and
rationale that go into the subjective assessment of each of the options. Further, pseudoquantitative equa
tions may be developed (as in probabilistic assessment equations for failure causes in fault tree analyses)
to increase confidence in analysis results. The AHP provides a multicriteria analysis methodology that
employs a pairwise comparison process to compare options to factors in a relative manner. This is used
when subjective verbal expressions (equal, moderate, strong, very strong, etc.) are easier to develop than
numerical (3 versus 3.2, etc.) assessments. Pseudoquantitative numbers are then ascribed to the words
and a score developed for each of the options.
A key to any trade study is the initial selection and prioritization of specific desirable attributes.
This is often very difficult and the prioritization delineation may change during the early phases of the
program. It is very important, and often overlooked, that when the prioritization changes, a cursory look
at the significant, completed trades should be performed to determine any impacts to their conclusions.
21
Table 21. Concept development tools and methodologies.
Tool or Methodology Section Advantages Limitations
Trade studies 2.1 (1) Different kinds and/or levels of study allow flexi
bility in the depth of the review, i.e., resources
expended can be commensurate with the benefits of
the task.
(2) Adaptive to prioritization based upon programmatic
(cost, schedule) considerations as well as technical
(weight, reliability, etc.) ones.
(3) Identification of disadvantages of specific design
option may lead to the definition of effective
countermeasures if combined with other techniques.
(1) Very dependent upon the expertise of the analyst
and the amount of available accurate quantitative
data.
(2) Improper generation of selection criteria, weight
factors, and utility functions can prejudice the
assessment and lead to incorrect results.
(3) The number of alternatives which can be considered
is limited by the expenditure of resources required to
perform the analysis.
(4) Options evaluated are not determined as a result of
the study but must be decided upon prior to the
assessment by the operator.
(5) Weighting factors and advantages/disadvantages are
very subjective (although objective data may be
added which significantly complicates and enlarges
the study) and this subjectivism is very near to the
study conclusions.
Costversusbenefit studies 2.2 (1) By performing a costversusbenefit analysis, the
analyst can assess the cost effectiveness of several
alternatives over the entire life cycle of the proposed
system under consideration.
(2) Provides documentation of the parameters evaluated
and the prioritized options considered.
(1) The analysis is flawed if system requirements are
incomplete or inadequate. If the system operating
environment is not understood or accurately
characterized, the total costs can be underestimated.
If the system requirements are too general or vague,
the effectiveness of benefits can not be addressed in
specific, measurable terms.
(2) The analysis is only as good as the list of alter
natives considered. An incomplete list of alter
natives will lead to an incomplete analysis.
(3) The analysis is flawed if incomplete or inaccurate
cost estimates are used.
(4) The analyst must be able to quantify the value of
benefits, which are often intangible or insubstantial
and difficult to characterize in terms of monetary
value.
22
2.1.2 Application
These studies should typically be performed in phase A of NASA projects. However, trade
studies can also be performed in phase B, or whenever a method is needed to select alternatives, such as
selecting test methods, evaluating design change proposals, or performing makeorbuy decisions. A
trade study analysis allows a systematic approach to evaluation of design options with respect to
programmatic considerations or other, nonreliability related considerations (weight, maintainability,
manufacturability). These studies may also be used to help the designer delineate which system require
ments are most important (used in conjunction with the Pareto chart analysis, sec. 5.6).
2.1.3 Procedures
The procedures for performing a weighted trade study are presented below. By performing step
6, an AHP weighted trade study will be performed. These procedures are described in detail and were
adapted from reference 2.1.
(1) Define the mission objectives and requirements for the system under consideration. These
objectives and requirements should be clear, accurate, and specific. These requirements will
provide the scope of the assessment and the basis for the selection criteria. Prioritize the
objectives/requirements if possible; this will aid in the weight factors for the selection criteria.
(2) Identify credible alternative candidates for the system under consideration. These
alternatives can be imposed or obtained in brainstorming sessions (sec. 7.7). The list of
alternatives selected during brainstorming sessions may be reduced by eliminating
alternatives which do not appear capable of meeting requirements. The list may be reduced
further by eliminating alternatives with low probability of successful implementation or
those which are expected to exceed cost constraints. The remaining alternatives should be
described in sufficient detail that the relative merits between them can be ascertained.
(3) Develop a trade tree (optional). A trade tree is developed to graphically illustrate the
alternatives and how highlevel alternatives in the decision process are logically resolved
into decreasingly lower level alternatives. For large trade studies with many alternatives
and criteria attributes, create a trade tree to group alternatives with unique criteria
attributes. A large trade study may be resolved into several smaller trade studies with fewer
required total comparison evaluations. This will lead to fewer resources to conduct the
assessment without degradation of the results.
(4) Develop and specify the selection criteria to be used in the analysis. The selection criteria
are benchmarks to assess the effectiveness and applicability characteristics of the
alternatives to be considered. Ideally, the selection criteria should have the following
characteristics:
a. Be expressed in general terms that mean the same thing to every evaluator.
b. Be practical to measure or predict within acceptable uncertainty and cost limits.
c. Provide a distinction between alternatives without prejudice.
d. Correlate directly to the established requirements and high priority issues. (A
numbering system, showing the specific correlation, is often useful here.)
e. Be separate and independent from each of the other selection criterion in all aspects of
the assessment.
23
(5) Establish weights for the selection criteria. These weights should reflect the importance of
each criterion relative to its importance to the overall selection decision. The weights
should be given numerical values to accommodate objective comparisons between
unrelated criteria. The numerical values of the weight factors should sum to 100. The
weights should be predetermined by the person (or group) with the ultimate decision
authority, but not necessarily shared with the analysts to ensure that alternatives are
assessed against each criterion objectively. Each criterion may be resolved into several
levels of components to establish its weight. The degree to which the individual criterion is
resolved into components is dependent on how effective the criterion components can be
evaluated, and represents the resolution limit of the assessment.
Consult with the end user of the system (the internal or external customer) to verify that the
selection criteria and weights are compatible with his needs.
(6) Perform an analytical hierarchy process as described in reference 2.2 to establish weights
for the selection criteria (optional). This technique is beneficial for very complex trade
studies when operational data are not available and a subjective analysis is to be performed.
The following steps define this process:
a. Establish a scale of the relative level of significance to the system objectives between
two given criteria attributes. Establish three to five definitions to subjectively define
this scale of relative level of significance. Generate clarifications for each definition so
that qualified managers and engineers can subjectively use the definitions. If five
definitions are used, assign the numerical values 1, 3, 5, 7, and 9 to these definitions in
order of increasing diversity between the given two attributes. Reserve the numerical
values of 2, 4, 6, and 8 as values to be assigned when interpolating between two of the
definitions. If attribute n has a numerical value of relative level of significance of “j”
relative to attribute m, then attribute m has a numerical value of relative level of
significance of “1/j” relative to attribute n.
b. Survey a group of qualified managers and engineers (or customers) to establish a
consensus on the relative relationships between each attribute and the rest of the
attributes.
c. Create a normalized matrix (all the attributes versus all the attributes) with these
relationships. Note that all elements of the diagonal of this matrix equal 1.
d. Determine the relative weights for each criterion component by performing an
eigenvector analysis.
e. Determine the weight for all attributes by calculating the product of each individual
attribute weighing factor and its weights of associated category headings.
(7) Generate utility functions (optional). This technique is used to establish a consistent scale
for dissimilar criteria. A relationship is established between a measure of effectiveness for
each selection criterion and a common scale (for example, 0 to 10). The relationship may
be a continuous function (not necessarily a straight line) or discrete values. For attributes
other than technical, such as cost, schedule, risk, etc., a subjective verbal scale may be used
(i.e., high, medium, low).
(8) Assess each alternative relative to the selection criteria. First estimate the performance of
every alternative for a given criterion in terms of the measure of effectiveness used in gen
erating the utility functions. For the ideal situation, the analyst may use test data, vendor pro
vided data, similarity comparison, modeling, engineering experience, parametric
24
analysis, or other costeffective and reliable methods to generate the performance estimates.
In reality, this is often very difficult to perform objectively. It is worthwhile, however, even
when somewhat subjective (i.e., heavy use of engineering experience). If quantification of
qualitative ranking is required, use caution in drawing conclusions. Assume that a difference
in the conclusion of less than onehalf the quantified number of a onestep difference is an
equivalent answer. This corresponds to a confidence band for the evaluation.
Next, determine the score for each alternative relative to a given criterion by correlating the
estimate of performance for all the criteria to the mutual scale using the utility functions
generated in step 7. Next, multiply the scores for all alternatives by the weight factor for the
criterion (determined in steps 5 or 6) to determine the weighted score for all alternatives for
that criterion. Repeat this procedure for all criteria attributes.
(9) Tabulate the results. Generate a matrix of criteria versus alternatives to summarize the
results from the preceding steps. A typical table is illustrated in table 22 and was
generalized from an example presented in reference 2.1.
Table 22. Typical weighted trade study summary table.
2.1
Criteria Alternates, x
i
(x
1
through x
n
)
Criterion,
y
j
Weights
w
j
= 100 Alternate x
1
Alternate x
2
Alternate x
3
Through
Alternate x
n–1
Alternate x
n
Score
(0–10)
Weighted
Score
Score
(0–10)
Weighted
Score
Score
(0–10)
Weighted
Score
Score
(0–10)
Weighted
Score
y
1
w
1
s
11
w
1
s
11
s
21
w
1
s
21
s
n1
w
1
s
n1
y
2
w
2
s
12
w
2
s
12
s
22
w
2
s
22
s
n2
w
2
s
n2
y
3
w
3
s
13
w
2
s
13
s
23
w
2
s
23
s
n3
w
3
s
n3
y
4
to y
m–1
y
m
w
m
s
1m
w
m
s
1m
s
2m
w
2
s
2m
s
nm
w
m
s
nm
Total ∑(w
j
s
ij
) ∑(w
j
s
ij
)
j
∑(w
j
s
ij
)
j
(10) Perform a sensitivity analysis to evaluate the merit of the results relative to making an alter
nate selection. Examine the results of the weighted trade study to see if any total weighted
scores of any alternatives are closer in numerical value than is warranted in making a deci
sion due to the confidence levels of the performance estimates that had been used to estab
lished the scores. If this is the case, then gather more data to increase the confidence level of
the performance estimates, repeat the assessment, and regenerate the summary table for the
weighted trade study. If, after the analysis is repeated, the alternative numerical total score is
too close to make a decision, reconsider the selection criterion and weighting factors.
(11) Select the superior alternative. Select the alternative with the highest value of total
weighted scores.
25
2.1.4 Example
Problem:
Four alternatives for a new automobile design are being considered. The selection decision will
be based on comparing the four alternatives to the following criteria attributes and their associated
weight factors:
Item Criteria Attribute Weight Factor
1 Average fuel economy 20
2 Acceleration (0 to 60 mph) 15
3 Braking (70 to 0 mph) 15
4 Road handling 15
5 Implement new technology
risk
10
6 Cost 25
Total 100
Utility functions have been generated for each criteria attribute and are presented in figure 21.
The estimates for each alternative relative to each criteria attribute are listed below:
Item Criteria attribute
Measure of
Effectiveness
Alt.
A
Alt.
B
Alt.
C
Alt.
D
1 Average fuel economy miles per gallon 16 19 23 18
2 Acceleration (0 to 60 mph) seconds 7 9 10 12
3 Braking (70 to 0 mph) feet 180 177 190 197
4 Road handling
(300 ft dia. skidpad)
g 0.86 0.88 0.83 0.78
5 Implementing new
technology risks
 Low Avg. High Very
low
6 Cost Dollars, × 1,000 21 20 24 22
From the information given above, formulate a weighted factor trade study summary table, and
select the superior alternative.
26
10
5
0
Average Fuel Economy, mpg
25
Acceleration (0 to 60 mph), seconds
Implementing New Technology Risks
Road Handling, g
Cost, dollars × 1,000
15
10
5
0
10
5
0
10
5
0
10
5
0
5 15
0.7 0.9 15 25
Very
High
Braking (70 to 0 mph), feet
10
5
0
175 200
Very
Low
Avg.
Score
Score
Score
Score
Score Score
Figure 21. Example utility functions.
27
Solution:
Presented in figure 22 is the completed weighted factor trade study summary. Scores were
determined from effectiveness measures for all alternatives relative to all criteria attributes and the utility
functions. Based on the results of the trade study, alternative B is the preferred option.
Criteria Alternates, x
i
Attribute
Item
Weights
w
j
=100 Alternate A Alternate B Alternate C Alternate D
Score
(0–10)
Weighted
Score
Score
(0–10)
Weighted
Score
Score
(0–10)
Weighted
Score
Score
(0–10)
Weighted
Score
1 20 1 20 4 80 8 160 3 60
2 15 9 135 8 120 7.5 112.5 5 75
3 15 9.8 147 9.9 148.5 8.5 127.5 5 75
4 15 4.5 67.5 7 105 2.5 37.5 1.5 22.5
5 10 8 80 6 60 4 40 10 100
6 25 4 100 5 125 1 25 3 75
Total 549.5 638.5 502.5 407.5
Figure 22. Example weighted factor trade study summary table.
2.1.5 Advantages
The following advantages can be realized from performing trade studies:
(1) Different kinds and/or levels of study allow flexibility in the depth of the review, i.e.,
resources expended can be commensurate with the benefits of the task.
(2) This technique is adaptive to prioritization based upon programmatic considerations (cost,
schedule) as well as technical ones (weight, reliability, etc.).
(3) Identification of disadvantages of a specific design option may lead to the definition of
effective countermeasures if combined with other techniques.
(4) The method provides a clearly documented analysis in which the (a) prioritized objectives
and requirements, (b) considered alternatives, and (c) selection methodology are recorded.
28
2.1.6 Limitations
The following limitations are associated with performing trade studies:
(1) These techniques are very dependent upon the expertise of the analyst and the amount of
available accurate quantitative data.
2.1
(2) Improper generation of selection criteria, weight factors, and utility functions can prejudice
the assessment and lead to incorrect results.
2.1
(3) The number of alternatives which can be considered is limited by the expenditure of
resources to perform the analysis.
2.1
(4) Options evaluated are not determined as a result of the study but must be decided upon
prior to the assessment by the person (or group) with decision authority.
(5) Weighting factors and advantages/disadvantages are very subjective (although objective
data may be added in the analytical hierarchy process approach (AHPA), this significantly
complicates and enlarges the study) and this subjectivism significantly influences the study
conclusions.
2.1.7 Bibliography
Blanchard, B.S., and Fabreycky, W.J.: “System Engineering and Analysis.” Second edition, Englewood
Cliffs, Prentice Hall, New Jersey, pp. 67–72, 1990.
Cross, N.: “Engineering Design Methods.” John Wiley & Sons, pp. 101–121, 1989.
Saate, T. L.: “Analytical Hierarchy Process.” McGrawHill, 1980.
2.2 COSTVERSUSBENEFIT STUDIES
2.2.1 Description
Costversusbenefit studies are also known as benefitcost analyses,
2.32.4
benefitcost ratio
analyses,
2.5
and costbenefit analyses.
2.6
Costversusbenefit studies, as described in reference 2.5, pro
vide a method to assess alternates by determining the ratio of expected future benefits to expected future
costs. Both the expected future benefits and costs are expressed in terms of present value. The
alternatives are ranked in decreasing order with the preferred option being the alternative with the
highest benefittocost (B/C) ratio, while falling within overall cost restraints.
2.2.2 Application
Benefitcost analyses apply to the selection of projects
2.32.4
or machines or systems
2.52.6
based on
their relative B/C ratios. Costversusbenefit studies, as discussed in this section, will apply to the
selection of system or system element alternatives based on their relative B/C ratios. These studies
should typically be performed in phase A, however, they could also be performed in phases B or C.
These studies can be used when two or more alternatives are being considered with fixed cost
constraints, fixed desired results or benefits, or when both costs and desired results vary.
29
2.2.3 Procedures
The following procedures to perform costversusbenefit studies were adapted from references
2.3, 2.5, and 2.6.
(1) Define the requirements for the system or system element under consideration. These
requirements should be measurable and verifiable. Translate general and vague
requirements into specific, quantitative requirements in which system effectiveness can be
measured and assessed.
2.6
Prioritize these requirements, if possible.
(2) Define a list of credible, mutually exclusive alternatives; that is, if one alternative is selected,
the others are not to be implemented.
2.3
Each alternative should be characterized to a level of
completeness such that all substantial costs and benefits can be identified.
2.6
Note that the
alternatives require an implicit determination of technical and schedule viability.
(3) Develop and specify the selection criteria to be used in the analysis. The example selection
criteria presented in table 23 were adapted from reference 2.5.
Table 23. Example selection criteria for costversusbenefit analyses.
Condition or Circumstance Selection Criteria
Budget C is fixed Maximum B/C ratio.
Desired result B is fixed. Maximum B/C ratio.
Two alternatives are being considered with
neither budget C or desired result B fixed.
Calculate the ∆Bto∆C ratio between the
alternatives. Choose the lower cost
alternative, unless the ∆Bto∆C ratio is
≥1. Then choose the higher cost
alternative.
More than two alternatives are being
considered with neither budget C or desired
result B fixed.
Select alternative per benefitcost ratio
incremental analysis (sec. 2.2.3,
step 11).
(4) Identify the cost or savings for each alternative. The cost should include such items as initial
investment, and ongoing operating and maintenance expenses (including depreciation) for the
life of the system. The savings should include such items as residual or salvage values, etc.
2.3
(5) Identify the benefit or detriments for each alternative. The benefits might include such
items as increased performance, reduced operating times, compressed schedules, increased
reliability, increased safety, value added due to increase productivity, etc. The detriments
might include such items as loss of production time, increased schedules, increased
equipment operating costs, environmental impacts, reduced property value, etc.
2.3
The cost
risk and technical maturity for each alternative may be included as a multiplying factor (f)
for this analysis. Since it is subjective, use of only three factors—0.5, 0.75, or 1—is
probably as fine a distinction as is warranted.
(6) Specify the time interval (expected operating life of the system) to which the analysis is to
apply.
210
(7) Develop cost and savings estimates and the benefits and detriments estimates for each
alternative.
2.3
The estimates for each alternative should be for the same time interval
specified in step 6. Every attempt should be made to base cost and savings estimates on
actual historical cost data.
(8) Identify the interest rate that will be assumed for the analysis.
2.3
Convert all costs, savings,
benefits, and detriments estimates to present worth values.
2.5
(9) Determine the total cost for each alternative by algebraically summing all costs as positive
values and all savings as negative values.
(10) Determine the total benefit value for each alternative by algebraically summing all benefits
as positive values and all detriments as negative values.
(11) Calculate the B/C ratio for each alternative by dividing the total benefit (B) by the total cost
(C).
For cases with fixed cost restraints or fixed desired results or benefits, perform step 12.
2.5
(12) Rank the alternatives relative to their respective costtobenefit ratios, and select the
superior alternative based on selection criteria established in step 3.
For cases with cost restraints and desired results or benefits that vary, perform steps 13
through 17.
2.5
(13) If there exists any alternatives with a B/C ≥1, then do not give further consideration to
alternatives with a B/C <1.
(14) Order the remaining alternatives in sequence of increasing total C.
(15) Determine the incremental B/C ratio ∆B/C for each consecutive pair of alternatives with
increasing total cost.
∆B/C
i
= ∆B
i
/∆ C
i
, where ∆B
i
= B
i +1
– B
i
and ∆C
i
= C
i+1
– C
i
for each i
th
pair of (n – 1) pairs of n alternatives where alternative i = 1,2,...,n listed in order
of C.
(16) Next, examine each distinct increment of increased cost investment. If the ∆B/C
i
is <1, then
the increment is not beneficial. If the ∆B/C
i
is >1, then the increment is beneficial.
(17) The preferred alternative is the last alternate listed in order of increasing cost whose
incremental ∆B/C
i
is >1. Therefore, the preferred alternative may not necessarily have the
greatest B/C ratio.
2.2.4 Example
Problem:
Five data acquisition systems (DAS) are under consideration to acquire data for solid rocket
motor tests in a test stand over a 10yr time interval. Each system has a different total cost and the
capabilities of each system are different in terms of maximum number of channels, maximum sample
211
rates, required maintenance, data accuracy, turnaround time between tests, and mean time between
system failures. The present value of the estimated total cost and total value of combined benefits of the
system are presented below. The present values of cost and benefits were determined over a 10yr
expected system life, with an assumed annual interest rate of 10 percent. Perform a costversusbenefit
analysis to determine the best alternative.
System A B C D E
Total cost (dollars) 500k 300k 750k 800k 400k
Total benefits (dollars) 750k 400k 900k 750k 600k
B/C 1.50 1.33 1.20 0.93 1.50
Solution:
Step 1. Delete options with a B/C ratio <1. Since the B/C for system D is <1, this option will no
longer be considered.
Step 2. List the remaining options in order of increasing total cost.
System B E A C
Total cost (dollars) 300k 400k 500k 750k
Total benefits (dollars) 400k 600k 750k 900k
Step 3. Determine the incremental B/C ratio ∆B/C for each consecutive pair of alternatives with
increasing total cost.
Increment E–B A–E C–A
∆ Total cost (dollars) 100k 100k 250k
∆ Total benefits (dollars) 200k 150k 150k
∆ B/C 2.0 1.5 0.6
Step 4. Identify the preferred alternative as the last alternate listed in order of increasing cost
whose incremental ∆B/C
i
is >1.
By inspection, the last incremental ∆B/C with a value >1 is A–E. Therefore, the preferred
alternative is DAS A.
212
2.2.5 Advantages
The following advantages are realized by performing costversusbenefit analyses:
(1) The analyst can assess the cost effectiveness of several alternatives over the entire life cycle
of the proposed system under consideration.
(2) The method provides a clearly documented analysis in which the prioritized
objectives/requirements, the alternatives considered, and the selection methodology are
recorded.
2.2.6 Limitations
Costversusbenefit analyses possess the following limitations:
(1) The analysis is flawed if system requirements are incomplete or inadequate. If the system
operating environment is not understood or accurately characterized, the total costs can be
underestimated. If the system requirements are too general or vague, benefits cannot be
addressed in specific, measurable terms of effectiveness.
2.5
(2) The analysis is only as good as the list of alternatives considered. An incomplete list of
alternatives will lead to an incomplete analysis.
2.6
(3) The analysis is flawed if incomplete or inaccurate cost estimates are used.
2.6
(4) The analyst must be able to quantify the value of benefits, which are often intangible or
insubstantial and difficult to characterize in terms of monetary value.
2.3
(5) The analysis does not take into account technical complexity or maturity of an alternative,
except as a cost uncertainty factor. Further, system reliability and safety issues are not
treated except by the selection of the alternative. As cost is generally only one of many
factors, this tool is generally insufficient for selection of large, new design efforts, but more
appropriate to productionlevel design solutions.
2.2.7 Bibliography
Thuesen, G.J., and Fabrycky, W.J.: “Engineering Economy.” Seventh edition, Englewood Cliffs,
Prentice Hall, New Jersey, 1989.
213
REFERENCES
2.1 “System Engineering Management Guide.” Defense Systems Management College, January 1990.
2.2 Saaty, T.L.: “Priority Setting in Complex Problems.” IEEE Transactions, pp. 140–155, August
1983.
2.3 Jelen, F.C., and Black, J.H.: “Cost and Optimization Engineering.” Second edition, New York,
McGrawHill Book Company, 1983.
2.4 Beattie, C.J., and Reader, R.D.: “Quantitative Management in R & D.” London, Chaplin and Hall
Ltd., 1971.
2.5 Newnan, D.G.: “Engineering Economic Analysis.” Second edition, Engineering Press, Inc., San
Jose, California, 1983.
2.6 Chestnut, H.: “System Engineering Methods.” John Wiley & Sons Inc., New York, 1967.
214
3. SYSTEM SAFETY AND RELIABILITY TOOLS
This section describes several system safety and reliability tools available to the system engineer
analyst. The risk assessment matrix is discussed in section 3.1. This device supports a standard
methodology to subjectively evaluate hazards as to their risks. It is used in conjunction with hazard
analyses, such as the preliminary hazard analysis (PHA) technique discussed in section 3.2. The PHA
can be used to identify hazards and to guide development of countermeasures to mitigate the risk posed
by these hazards. The energy/flow barrier analysis discussed in section 3.3 is also a technique to identify
hazards and to evaluate their corresponding countermeasures.
Once hazards are identified, they can be further explored if failure modes of the elements of the
system are known. The failure modes and effects analysis (FMEA), discussed in section 3.4, can be used
to identify failure modes and their consequences or effects. Also discussed in section 3.4 is the failure
modes, effects, and criticality analysis (FMECA). The FMECA is similar to the FMEA but also
addresses the criticality, or risk, associated with each failure mode.
Several symbolic logic methods are presented in this section. These methods construct
conceptual models of failure or success mechanisms within a system. These tools are also used to
determine either the probability of failures occurring or the probability that a system or component will
operate successfully. The probability of a successful operation is the reliability. If the probability of
failure (P
F
) is examined, then the model is generated in the failure domain and if the probability of
success (P
S
) is examined, then the model is generated in the success domain. For convenience, the
analyst can model either in the failure or success domain (or both domains), then convert the final
probabilities to the desired domain using the following expression: P
F
+ P
S
= 1.
These models are developed using forward (bottomup) or backwards (topdown) logic. When
using forward logic the analyst builds the model by repeatedly asking, “What happens when a given fail
ure occurs?” The analyst views the system from a “bottomup” perspective. This means he starts by
looking at the lowest level elements in the system and their functions. Classically, the FMEA, for exam
ple, is a bottomup technique. When using backwards logic to build a model, the analyst repeatedly asks,
“What will cause a given failure to occur?” The analyst views the system from a “topdown”
perspective. This means he starts by looking at a high level system failure and proceeds down into the
system to trace failure paths. The symbolic logic techniques discussed in this section and their
characteristics are presented in table 31.
Each of the symbolic logic techniques has its own unique advantages and disadvantages.
Sometimes it is beneficial to construct a model using one technique, then transform that model into the
domain of another technique to exploit the advantages of both techniques. Fault trees are generated in
the failure domain, reliability diagrams are generated in the success domain, and event trees are
generated both in the success and failure domains. Methods are presented in section 3.9 to transform any
one of the above models into the other two by translating equivalent logic from the success to failure or
failure to success domains.
Probabilities are propagated through the logic models to determine the probability that a system
will fail or the probability the system will operate successfully, i.e., the reliability. Probability data may
be derived from available empirical data or found in handbooks. If quantitative data are not available,
then subjective probability estimates may be used as described in section 3.12. Caution must be
exercised when quoting reliability numbers. Use of confidence bands is important. Often the value is in a
comparison of numbers that allows effective resource allocation, rather than “exact” determination of
31
Table 31. Symbolic logic techniques.
Technique Section Success
Domain
Failure
Domain
Forward
(BottomUp)
Backwards
(TopDown)
Reliability block
diagram
3.5 √ √
Fault tree analysis 3.6 √ √
Success tree analysis 3.7 √ √
Event tree analysis 3.8 √ √ √
Causeconsequence
analysis
3.10 √ √ √ √
Directed graph matrix
analysis
3.11 √ √ √
expected reliability levels. Probabilistic design analysis (PDA) is discuss in section 3.14. This technique
uses advanced statistical methods to determine P
F
modes.
Failure mode information propagation modeling is discussed in section 3.13. This technique
allows the analyst to determine what information is needed, and how and where the information should
be measured in a system to detect the onset of a failure mode that could damage the system.
Finally, probabilistic risk assessment (PRA) is discussed in section 3.15. This is a general
methodology that shows how most of the techniques mentioned above can be used in conjunction to
assess risk with severity and probability.
A summary of the major advantages and limitations of each tool or methodology discussed in
this section is presented in table 32.
3.1 Risk Assessment Matrix
3.1.1 Description
The risk assessment matrix, as described in reference 3.1, is a tool to conduct subjective risk
assessments for use in hazard analysis. The definition of risk and the principle of the isorisk contour are
the basis for this technique.
Risk for a given hazard can be expressed in terms of an expectation of loss, the combined
severity and probability of loss, or the longterm rate of loss. Risk is the product of severity and
probability (loss events per unit time or activity). Note: the probability component of risk must be
attached to an exposure time interval.
The severity and probability dimensions of risk define a risk plane. As shown in figure 31, iso
risk contours depict constant risk within the plane.
32
Table 32. System safety and reliability tools and methodologies—Continued
Tool or Methodology Section Advantages Limitations
Risk assessment matrix 3.1 Provides standard tool to subjectively assess risk. Only used to assess risk of hazards, does not identify
hazards.
Preliminary hazard analysis 3.2 Identifies and provides inventory of hazards and
countermeasures.
Does not address coexisting system failure modes.
Energy flow/barrier analysis 3.3 Identify hazards associated with energy sources and
determines if barriers are adequate countermeasures.
(1) Does not address coexisting system failure modes.
(2) Fails to identify certain classes of hazards, e.g.,
asphyxia in oxygendeficient confined spaces.
Failure modes and effects (and
criticality) analysis
3.4 Thorough methods of identifying single point failures
and their consequences. A criticality analysis provides a
risk assessment of these failure modes.
Can be extremely labor intense. Does not address
coexisting system failure modes.
Reliability block diagram 3.5 A symbolic logic model that is relatively easy for the
analyst to construct. System reliability can be derived,
given component reliability.
Component reliability estimates may not be readily
available; total calculated reliability may be
unrealistically high.
Fault tree analysis 3.6 (1) Enables assessment of probabilities of coexisting
faults or failures.
(2) May identify unnecessary design elements.
(1) Addresses only one undesirable event or condition
that must be foreseen by the analyst.
(2) Comprehensive trees may be very large and
cumbersome.
Success tree analysis 3.7 Assesses probability of favorable outcome of system
operation.
(1) Addresses only one desirable event or condition that
must be foreseen by the analyst.
(2) Comprehensive trees may be very large and
cumbersome.
Event tree analysis 3.8 (1) Enables assessment of probabilities of coexisting
faults or failures.
(2) Functions simultaneously in failure and success
domain.
(3) End events need not be anticipated. Accident
sequences through a system can be identified.
(1) Addresses only one initiating challenge that must
be foreseen by the analyst.
(2) Discrete levels of success and failure are not
distinguishable.
Fault tree, reliability, block
diagram, and event tree
transformations
3.9 Allows the analyst to overcome weakness of one
technique by transforming a model of a system into an
equivalent logic model in another analysis technique.
This technique offers no additional information and is
only as good as the input model.
33
Table 32. System safety and reliability tools and methodologies—Continued.
Tool or Methodology Section Advantages Limitations
Causeconsequence analysis 3.10 (1) Enables assessment of probabilities of coexisting
faults or failures.
(2) End events need not be anticipated.
(3) Discrete levels of success and failure are
distinguishable.
(1) Addresses only one initiating challenge that must
be foreseen by the analyst.
(2) May be very subjective as to consequence severity.
Directed graph (digraph) matrix
analysis
3.11 (1) Allows the analyst to examine the fault propagation
through several primary and support systems.
(2) Minimal cut sets, singlepoint failure, and double
point failures can be determined with less computer
computation than with FTA.
(1) Trained analyst and computer codes and resources to
perform this technique may be limited.
(2) Only identifies single point (singleton) and dual
points (doubleton) of failure.
Combinatorial failure
probability analysis using
subjective information
3.12 Allows analyst to perform qualitative probabilistic risk
assessment based on the exercise of subjective
engineering judgment when no quantitative data is
available.
Use of actual quantitative data is preferred to this
method. Should only be used when actual quantitative
failure data is unavailable.
Failure mode information
propagation modeling
3.13 Measurement requirements can be determined that if
implemented can help safeguard a system in operation
by providing warning at the onset of a threatening
failure mode.
(1) This technique is only applicable if the system is
operating in a near normal range and for the instant
of time just before initiation of a failure.
(2) Data and results, unless used in a comparative
fashion, may be poorly received.
Probabilistic design analysis 3.14 (1) Allows the analyst a practical method of
quantitatively and statistically estimating the
reliability of a system during the design phase.
(2) Provides alternative to the traditional method of
imposing safety factors and margins to ensure
system reliability. That method might be flawed if
significant experience and historical data of similar
components are not available.
(1) Analyst must have significant experience in
probability and statistical methods to apply this
technique.
(2) Historical population data used must very close to
asplanned design population to be viable.
Extrapolation between populations can render
technique nonviable.
Probabilistic risk assessment 3.15 Provides methodology to assess overall system risks;
avoids accepting unknown, intolerable, and senseless
risk.
Performing the techniques of this methodology requires
skilled analysts. Techniques can be misapplied and
results misinterpreted.
34
PROBABILITY
S
E
V
E
R
I
T
Y L
I
K
E
L
Y
0
NEVER
CATACLYSMIC
I
n
c
r
e
a
s
i
n
g
R
i
s
k
Isorisk
Contours
R = P x S = K
1
R = K
2
> K
1
R = K
3
> K
2
RISK
is
CONSTANT
along any
ISORISK
CONTOUR.
PROBABILITY
is a function of
EXPOSURE
INTERVAL.
SEVERITY
and
PROBABILITY,
the
two variables
that
constitute risk,
define a
RISK PLANE.
Figure 31. Risk plane.
The concept of the isorisk contour is useful to provide guides, convention, and acceptance limits
for risk assessments (fig. 32).
Risk should be evaluated for worst credible case, not worst conceivable case, conditions. Failure
to assume credible (even if conceivable is substituted) may result in an optimistic analysis; it will result
in a nonviable analysis.
3.1.2 Application
The risk assessment matrix is typically performed in phase C but may also be performed in phase
A. This technique is used as a predetermined guide or criteria to evaluate identified hazards as to their
risks. These risks are expressed in terms of severity and probability. Use of this tool allows an organiza
tion to institute and standardize the approach to perform hazard analyses. The PHA, defined in section
3.2, is such an analysis.
35
PROBABILITY
S
E
V
E
R
I
T
Y
0
0
RISK ASSESSMENT
GUIDES: If Risk for a
given Hazard can be
assessed at any severity
level, an isorisk contour
gives its probability at all
severity levels. (Most,
but not all hazards
behave this way. Be wary
of exceptions — usually
highenergy cases.)
RISK ASSESSMENT
CONVENTION: If
possible, assess Risk for
the Worst Credible
Severity of outcome. (It’ll
fall at the top end of its
own isorisk contour.)
ACCEPTABLE
(de minimis)
PROVISIONALLY
ACCEPTABLE
NOT
ACCEPTABLE
1
2
ACCEPTANCE: Risk
Tolerance Boundaries
follow isorisk contours.
3
Figure 32. Isorisk contour usage.
3.1.3 Procedures
Procedures, as described in reference 3.1, for developing a risk assessment matrix are presented
below:
(1) Categorize and scale the subjective probability levels for all targets, such as frequent,
probable, occasional, remote, improbable, and impossible (adapted from MIL–STD–
882C).
3.2
Note: A target is defined as the “what” which is at risk. One typical breakout of
targets is personnel, equipment, downtime, product loss, and environmental effects.
(2) Categorize and scale the subjective severity levels for each target, such as catastrophic,
critical, marginal, and negligible.
(3) Create a matrix of consequence severity versus the probability of the mishap. Approximate
the continuous, isorisk contour functions in the risk plane with matrix cells (fig. 33).
These matrix cells fix the limits of risk tolerance zones. Note that not the analyst but man
agement establishes and approves the risk tolerance boundaries.
(4) The following hints will be of help when creating the matrix:
a. Increase adjacent probability steps by orders of magnitude. The lowest step,
“impossible,” is an exception (fig. 34(a)).
b. Avoid creating too many matrix cells. Since the assessment is subjective, too many
steps add confusion with no additional resolution (fig. 34(b)).
36
“Zoning” the Risk Plane into
judgmentally tractable cells
produces a Matrix.
Matrix cells approximate the
continuous, isorisk contour
functions in the Risk Plane.
Steps in the Matrix define Risk
Tolerance Boundaries.
PROBABILITY
S
E
V
E
R
I
T
Y
PROBABILITY
S
E
V
E
R
I
T
Y
A B C D E F
I
II
III
IV
Figure 33. Risk plane to risk matrix transformation.
c. Avoid discontinuities in establishing the risk zones, i.e., make sure every onestep path
does not pass through more than one zone (fig. 34(c)).
d. Establish only a few risk zones. There should only be as many zones, as there are
desired categories of resolution to risk issues, i.e., (1) unacceptable, (2) accepted by
waiver, and (3) routinely accepted (fig. 34(d)).
(5) Calibrate the risk matrix by selecting a cell and attaching a practical hazard scenario to it.
The scenario should be familiar to potential analysts or characterize a tolerable perceivable
threat. Assign its risk to the highest level severity cell just inside the acceptable risk zone.
This calibration point should be used as a benchmark to aid in evaluating other, less
familiar risks.
37
PROBABILITY
S
E
V
E
R
I
T
Y
1
2
3
A B C D E F
I
II
III
IV
Factors of 10 separate
adjacent Probability Steps.
D = 10 E
C = 10 D
B = 10 C
A = 10 B
…but F = 0 (“Impossible”)
(a) Useful conventions.
PROBABILITY
S
E
V
E
R
I
T
Y
1
2
3
A B C D E F
I
II
III
IV
PREFERRED
A 24cell Matrix can be resolved
into 9 levels of “priority,” or even
more. But what are the rational
functions for the many levels?
Three zones will usually suffice.
A Hazard’s Risk is either…
• (3) Routinely Accepted
• (2) Accepted by Waiver, or
• (1) Avoided.
PROBABILITY
S
E
V
E
R
I
T
Y
1
A B C D E F
I
II
III
IV
4
2 3 6
7
8
8
6
9 7
2 3
4
5 4 4
5
5
FLAWED
(b) Do not create too many cells.
Figure 34. Helpful hints in creating a risk assessment matrix—Continued
38
PROBABILITY
S
E
V
E
R
I
T
Y
1
2
3
A B C D E F
I
II
III
IV
Can a countermeasure make the
“leap” from Zone (1) to Zone (3)
in a single step?
PREFERRED
Make every onestep path
from a high Risk Zone (1)
to a lower Risk Zone (3)
pass through the
intermediate Zones (2).
PROBABILITY
S
E
V
E
R
I
T
Y
1
2
3
A B C D E F
I
II
III
IV
FLAWED
?
?
(c) Avoid discontinuities.
PROBABILITY
S
E
V
E
R
I
T
Y
1
2
3
A B C D E F
I
II
III
IV
PROBABILITY
S
E
V
E
R
I
T
Y
A B C D E F G K L H I J
I
II
III
IV
V
VI
VII
1
2
3
Subjective judgment can’t
readily resolve more than six
discrete probability steps.
Added steps become
confused/meaningless.
Keep it SIMPLE!
4 6 = 24 cells
is better than
7 12 = 84 cells PREFERRED
FLAWED
(d) Do not create too many zones.
Figure 34. Helpful hints in creating a risk assessment matrix—Continued.
39
3.1.4 Example
A typical risk assessment matrix, adapted from MIL–STD–882C,
3.2
is presented in figure 35. Example
interpretations of the severity and probability steps for this matrix are presented in figure 36.
I
CATASTROPHIC
III
MARGINAL
IV
NEGLIGIBLE
Severity
of
Consequences
II
CRITICAL
Probability of Mishap**
A
FREQUENT
C
OCCASIONAL
F
IMPOSSIBLE
E
IMPROBABLE
D
REMOTE
B
PROBABLE
3
2
1
*Adapted from MILSTD882C **Life Cycle = 25 yrs.
Risk Code/ Actions
NOTE: Personnel must not be exposed to hazards in Risk Zones 1 and 2.
3
Operation permissible.
2
Operation requires written, timelimited waiver, endorsed
by management.
1 Imperative to suppress risk to lower level.
Figure 35. Typical risk assessment matrix.
310
Provide
stepwise
scaling of
SEVERITY
levels for
each
TARGET.
Decide on
TARGETS.
Provide
stepwise
scaling of
PROBABILITY
levels for
all
TARGETS.
PROBABILITY
is a function of
EXPOSURE
INTERVAL.
Longterm (5 yrs or
greater) environ
mental damage or
requiring >$1M to
correct and/or in
penalties
Mediumterm (15
yrs) environmental
damage or requir
ing $250K$1M to
correct and/or in
penalties
Shortterm (<1 yr)
environmental dam
age or requiring
$1K$250K to cor
rect and/or in pen
alties
Minor environment
al damage, readily
repaired and/or
requiring <$1K to
correct and/or in
penalties
DESCRIPTIVE
WORD
DEFINITION
Probability of Mishap**
LEVEL
DOWN
TIME
>4 months
2 weeks
to
4 months
1 day
to
2 weeks
<1 day
EQUIPMENT
LOSS ($)**
>1M
250K
to
1M
1K
to
250K
<1K
PERSONNEL
ILLNESS/
INJURY
Death
Severe injury or
severe
occupational
illness
Minor
injury or
minor
occupation
al illness
No injury or
illness
CATEGORY/
DESCRIPTIVE
WORD
I
CATASTROPHIC
II
CRITICAL
III
MARGINAL
IV
NEGLIGIBLE
Severity of Consequences
A
B
C
D
E
F
Likely to occur
repeatedly in system
life cycle
Physically impossible
to occur
IMPOSSIBLE
IMPROBABLE
REMOTE
OCCASIONAL
PROBABLE
FREQUENT
PRODUCT
LOSS
Values as
for loss
Equipment
Loss
ENVIRONMENTAL
EFFECT
*Adapted from MILSTD882C **Life Cycle = 25 yrs.
Probability of
occurrence cannot be
distinguished from
zero
Likely to occur several
times in system life
cycle
Likely to occur
sometime in system
life cycle
Not likely to occur
repeatedly in system
life cycle, but possible
Figure 36. Severity and probability interpretations.
311
3.1.5 Advantages
The risk assessment matrix provides the following advantages:
3.1
(1) The risk matrix provides a useful guide for prudent engineering.
(2) The risk matrix provides a standard tool of treating the relationship between severity and
probability in assessing risk for a given hazard.
(3) Assessing risk subjectively avoids unknowingly accepting intolerable and senseless risk,
allows operating decisions to be made, and improves resource distribution for mitigation of
loss resources.
3.1.6 Limitations
The risk assessment matrix possesses the following limitations:
3.1
(1) The risk assessment matrix can only be used if hazards are already identified. This tool
does not assist the analyst in identifying hazards.
(2) This method is subjective without data and is a comparative analysis only.
3.1.7 Bibliography
Code of Federal Regulations, Medical devices, “PreMarket Notification.” sec. 807.90,
vol. 21.
Code of Federal Regulations, “Process Safety Management of Highly Hazardous Chemicals.” sec.
1910.119 (e), vol. 29.
Department of Defense Instruction, No. 5000.36. “System Safety Engineering & Management.”
NASA NHB 1700.1, vol. 3,“System Safety.”
NUREG/GR0005. “RiskBased Inspection – Development of Guidelines.”
3.2 Preliminary Hazard Analysis
3.2.1 Description
A PHA, as described in reference 3.3, produces a line item tabular inventory of nontrivial system
hazards, and an assessment of their remaining risk after countermeasures have been imposed. This
inventory includes qualitative, not quantitative, assessments of risks. Also, often included is a tabular
listing of countermeasures with a qualitative delineation of their predicted effectiveness. A PHA is an
early or initial system safety study of system hazards.
312
3.2.2 Application
PHA’s are best applied in phase C but may also be applied in phase B. This tool is applied to cover
wholesystem and interface hazards for all mission phases. A PHA may be carried out, however, at any
point in the life cycle of a system. This tool allows early definition of the countermeasure type and
incorporation of design countermeasures as appropriate.
3.2.3 Procedures
A flowchart describing the process to perform a PHA is presented in figure 37. Procedures for
performing PHA’s, as described in reference 3.3, are presented below:
(1) Identify resources of value, such as personnel, facilities, equipment, productivity, mission
or test objectives, environment, etc. to be protected. These resources are targets.
(2) Identify and observe the levels of acceptable risk that have been predetermined and
approved by management. These limits may be the risk matrix boundaries defined in a risk
assessment matrix (sec. 3.1).
(3) Define the extent of the system to be assessed. Define the physical boundaries and
operating phases (such as shakedown, activation, standard operation, emergency shutdown,
maintenance, deactivation, etc.). State other assumptions, such as if the assessment is
based on an asbuilt or asdesigned system, or whether current installed countermeasures
will be considered.
(4) Detect and confirm hazards to the system. Identify the targets threatened by each hazard. A
hazard is defined as an activity or circumstance posing “a potential of loss or harm” to a
target and is a condition required for an “undesired loss event.”
3.3
Hazards should be
distinguished from consequences and considered in terms of a source (hazard), mechanism
(process), and outcome (consequence). A team approach to identifying hazards, such as
brainstorming (sec. 7.7), is recommended over a single analyst. If schedule and resource
restraints are considerations, then a proficient engineer with knowledge of the system
should identify the hazards, but that assessment should be reviewed by a peer. A list of
proven methods
*
for finding hazards is presented below:
a. Use intuitive “engineering sense.”
b. Examine and inspect similar facilities or systems and interview workers assigned to
those facilities or systems.
c. Examine system specifications and expectations.
d. Review codes, regulations, and consensus standards.
e. Interview current or intended system users or operators.
f. Consult checklists (app. D).
g. Review system safety studies from other similar systems.
*
Provided courtesy of Sverdrup Technology, Inc., Tullahoma, Tennessee.
3.3
313
Figure 37. PHA Process flowchart
314
h. Review historical documents—mishap files, nearmiss reports, OSHArecordable injury
rates, National Safety Council data, manufacturer’s reliability analyses, etc.
i. Consider “external influences” like local weather, environment, or personnel
tendencies.
j. Consider all mission phases.
k. Consider “common causes.” A common cause is a circumstance or environmental
condition that, if it exists, will induce two or more fault/failure conditions within a
system.
l. Brainstorm (sec. 7.7)—mentally develop credible problems and play “whatif” games.
m. Consider all energy sources. What is necessary to keep them under control; what
happens if they get out of control?
(5) Assess worstcredible case (not the worstconceivable case) severity and probability for
each hazard and target combination. Keep the following considerations in mind during the
evaluation:
a. Remember that severity for a specific hazard varies as a function of targets and
operational phases.
b. A probability interval must be established before probability can be determined. This
interval can be in terms of time, or number of cycles or operations.
c. The assessment will underestimate the true risk if a shortterm probability interval is
used unless the risk acceptance criterion is adjusted accordingly. Probability intervals
expressed in hours, days, weeks, or months are too brief to be practical. The interval
should depict the estimated facility, equipment, or each human operator working life
span. An interval of 25 to 30 yr is typically used and represents a practical value.
d. The probability for a specific hazard varies as a function of exposure time, target,
population, and operational phase.
e. Since probability is determined in a subjective manner, draw on the experience of
several experts as opposed to a single analyst.
(6) Assess risk for each hazard using a risk assessment matrix (sec. 3.1). The matrix should be
consistent with the established probability interval and force or fleet size for this
assessment.
(7) Categorize each identified risk as acceptable or unacceptable, or develop countermeasures
for the risk, if unacceptable.
(8) Select countermeasures in the following descending priority order to optimize
effectiveness: (1) “design change,” (2) “engineering safety systems,” (3) “safety devices,”
(4) “warning devices,” and (5) “procedures and training.”
3.3
315
Note that this delineation, while in decreasing order of effectiveness, is also typically in
decreasing order of cost and schedule impact (i.e., design changes have the highest
potential for cost and schedule impact). A trade study (sec. 2.1) might be performed to
determine a countermeasure of adequate effectiveness and minimized program impact.
(9) Reevaluate the risk with the new countermeasure installed.
(10) If countermeasures are developed, determine if they introduce new hazards or intolerably
diminish system performance. If added hazards or degraded performance are unacceptable,
determine new countermeasures and reevaluate the risk.
3.2.4 Example
An example of a completed PHA worksheet
3.3
for a pressurized chemical intermediate transfer system is
presented in figure 38. (A blank form is included in appendix E.)
Brief Descriptive Title (Portion of System/Subsystem/Operational Phases covered by this analysis):
Probability Interval: 25 years
H
a
z
a
r
d
T
a
r
g
e
t
*
S
e
v
e
r
i
t
y
P
r
o
b
a
b
i
l
i
t
y
R
i
s
k
C
o
d
e
Description of Countermeasures
S
e
v
e
r
i
t
y
P
r
o
b
a
b
i
l
i
t
y
R
i
s
k
C
o
d
e
Approved by/Date:
Preliminary Hazard Analysis
Identify countermeasures by appropriate code letter(s):
D = Design Alteration E = Engineered Safety Feature
S = Safety Device W = Warning Device
P = Procedures/Training
Analysis: Initial
Revision Addition
System Number: __________
Date:
Hazard No. / Description
Risk
Before
Risk
After
Prepared by/Date: *Target Codes: P—Personnel
T—Downtime
E—Equipment
R—Product V—Environment
Pressurized UnFo Containment and Replenishment Reservoir and Piping / Startup, Routine Operation, Standard Stop, Emergency Shutdown
25 Feb. 1993
X
E
P
T
I
II
III
D
C
C
2
2
3
I
II
III
E
D
D
3
3
3
SrdA.a.042 — Flange Seal A29 leakage, releasing
pressurized UnFo
chemical intermediate from
containment system, producing toxic vapors and
attacking nearby equipment.
Surround flange with sealed annular stainless steel
catchment housing, with gravity runoff conduit led to
DetectoBox™ containing detector/alarm device and
chemical neutralizer (S/W). Inspect flange seal at 2month
intervals, and regasket during annual plant maintenance
shutdown (P). Provide personal protective equipment
(Schedule 4) and training for response/cleanup crew (S/P).
Show hazard alphanumeric
designator. Describe hazard
source mechanism, worst
credible outcome.
Assess worstcredible Severity, and
Probability for that outcome. Show
Risk (from assessment matrix) for
hazard “asis” — i.e., with no
added countermeasures.
SrdA (Chem/Int)
Identify target(s).
Describe newly proposed countermeasures to
reduce Probability/Severity.
NOTE: THESE COUNTERMEASURES MUST
BE IN PLACE PRIOR TO OPERATION.
Reassess Probability/Severity, and
show Risk (from assessment matrix)
for hazard, presuming new
countermeasures to be in place. If Risk
is not acceptable, new countermeasures
must be developed.
3
3
Figure 38. Typical PHA.
316
Note that the worksheet from this example contains the following information:
a. Brief description of the portion of the system, subsystem, or operation covered in the
analysis.
b. Declaration of the probability interval.
c. System number.
d. Date of analysis.
e. Hazard (description and identification number).
f. Hazard targets (check boxes for personnel, equipment, downtime, product
environment).
g. Risk assessment before countermeasures are considered; including severity level,
probability level, and risk priority code (zone from risk matrix, fig. 35).
h. Description of countermeasure (with codes for various types).
i. Risk assessment after countermeasures are considered, including severity level,
probability level, and risk priority code.
j. Signature blocks for the analyst and reviewers/approvers.
The PHA worksheet used in the example is typical. However, an organization may create
their own worksheet customized for their operation. For example, different target types may be listed. In
any case, great care should be given in designing the form to encourage effective usage. Although
helpful, a PHA is not a structured approach that assists the analyst in identifying hazards or threats.
3.2.5 Advantages
A PHA provides the following advantages:
3.3
(1) Identifies and provides a log of primary system hazards and their corresponding risks.
(2) Provides a logically based evaluation of a system’s weak points early enough to allow
design mitigation of risk rather than a procedural or inspection level approach.
(3) Provides information to management to make decisions to allocate resources and prioritize
activities to bring risk within acceptable limits.
(4) Provides a relatively quick review and delineation of the most significant risks associated
with a specific system.
317
3.2.6 Limitations
A PHA possesses the following limitations:
3.3
(1) A PHA fails to assess risks of combined hazards or coexisting system failure modes.
Therefore a false conclusion may be made that overall system risk is acceptable simply
because each hazard element risk identified, when viewed singularly, is acceptable.
(2) If inappropriate or insufficient targets or operational phases are chosen, the assessment will
be flawed. While on the other hand, if too many targets or operational phases are chosen,
the effort will become too large and costly to implement.
3.2.7 Bibliography
Air Force Systems Command Design Handbook DH 16, “System Safety.” December 1982.
Army Regulation 389516, “System Safety Engineering and Management.” May 1990.
Browning, R.L.: “The Loss Rate Concept in Safety Engineering.” Marcel Dekker, Inc., 1980.
Hammer, W.: “Handbook of System and Product Safety.” PrenticeHall, Inc., 1972.
Henley, E.J., and Kumamoto, H.: “Probabilistic Risk Assessment.” The Institute of Electrical and
Electronic Engineers, Inc., New York, 1991.
Malasky, S.W.: “System Safety: Technology and Application.” Garland STPM Press, 1982.
Raheja, D.G. “Assurance Technology and Application  Principles and Practices.” McGrawHill, 1991.
Roland, H.E., and Moriaty, B.: “System Safety Engineering and Management.” John Wiley & Sons,
Inc., 1983.
3.3 Energy Flow/Barrier Analysis
3.3.1 Description
The energy flow/barrier analysis, as described in reference 3.4, is a system safety analysis tool,
used to identify hazards and determine the effectiveness of countermeasures employed or suggested to
mitigate the risk induced by these hazards. This tool is also known as energy trace/barrier analysis. The
energy flow/barrier method is a useful supplement to the PHA discussed in section 3.2.
Energy sources such as electrical, mechanical, chemical, radiation, etc., are identified.
Opportunities for undesired energy flow between the sources and targets are assessed. Barriers are
countermeasures against hazards caused by flows from these energy sources to targets. Examples of
barriers include barricades, blast walls, fences, lead shields, gloves, safety glasses, procedures, etc.
318
3.3.2 Application
An energy flow/barrier analysis can be beneficially applied whenever assessments are needed to
assure an identified target is being safeguarded against a potential energy source that can impose harm.
This assessment can be applied during phase C but may also be applied in phase E or phase B. This
analysis can also be applied in failure investigations.
3.3.3 Procedures
Procedures to perform an energy flow/barrier analysis, as described in reference 3.4, are
presented below:
(1) Examine the system and identify all energy sources.
(2) Examine each potential energy flow path in the system. Consider the following for each
energy flow path:
a. What are the potential targets, such as personnel, facilities, equipment, productivity,
mission or test objectives, environment, etc.? Remember every energy source could
have multiple flow paths and targets.
b. Is the energy flow unwanted or detrimental to a target?
c. Are existing barriers sufficient countermeasures to mitigate the risk to the targets?
(3) Consider the following strategies extracted from reference 3.4 to control harmful energy
flow:
a. Eliminate energy concentrations.
b. Limit quantity and/or level of energy.
c. Prevent the release of energy.
d. Modify the rate of release of energy.
e. Separate energy from target in time and/or space.
f. Isolate by imposing a barrier.
g. Modify target contact surface or basic structure.
h. Strengthen potential target.
i. Control improper energy input.
3.3.4 Example
Examples of strategies to manage harmful energy flows are presented in table 33.
319
Table 33. Examples* of strategies to manage harmful energy flow.
Strategy Examples
Eliminate energy concentrations · Control/limit floor loading
· Disconnect/remove energy source from system
· Remove combustibles from welding site
· Change to nonflammable solvent
Limit quantity and/or level of energy · Store heavy loads on ground floor
· Lower dam height
· Reduce system design voltage/operating pressure
· Use small(er) electrical capacitors/pressure accumulators
· Reduce/ control vehicle speed
· Monitor/limit radiation exposure
· Substitute less energetic chemicals
Prevent release of energy · Heavywall pipe or vessels
· Interlocks
· Tagout – lockouts
· Doublewalled tankers
· Wheel chocks
Modify rate of release of energy · Flow restrictors in discharge lines
· Resistors in discharge circuits
· Fuses/circuit interrupters
Separate energy from target in time
and/or space
· Evacuate explosive test areas
· Impose explosives quantitydistance rules
· Install traffic signals
· Use yellow nopassing lines on highways
· Control hazardous operations remotely
Isolate by imposing a barrier · Guard rails
· Toe boards
· Hard hats
· Face shields
· Machine tool guards
· Dikes
· Grounded appliance frames/housing
· Safety goggles
Modify target contact surface or basic
structure
· Cushioned dashboard
· Fluted stacks
· Padded rocket motor test cell interior
· Whipple plate meteorite shielding
· Breakaway highway sign supports
· Foamed runways
Strengthen potential target · Select superior material
· Substitute forged part for cast part
· “Harden” control room bunker
· Crossbrace transmission line tower
Control improper energy input · Use coded, keyed electrical connectors
· Use matchthreaded piping connectors
· Use back flow preventors
*Examples provided courtesy of Sverdrup Technology Inc., Tullahoma, Tennessee.
320
3.3.5 Advantages
The energy flow/barrier analysis provides a systematic thought process to identify hazards
associated with energy sources and determines if current or planned barriers are adequate
countermeasures to protect exposed targets.
3.4
3.3.6 Limitations
The energy flow/barrier analysis possesses the following limitations.
3.4
(1) Even after a thorough analysis, all hazards might not be discovered. Like the PHA (sec.
3.2), an energy flow/barrier analysis fails to assess risks of combined hazards or coexisting
system failure modes.
(2) This tool also fails to identify certain classes of hazards, e.g., asphyxia in oxygendeficient
confined spaces.
(3) Due to design and performance requirements, it is not always obvious that energy may be
reduced or redirected. A reexamination of energy as heat, potential versus kinetic
mechanical energy, electrical, chemical, etc. may aid this thought process.
3.3.7 Bibliography
Department of Energy, DOD 76–451: SSDC–29, “Barrier Analysis.”
Haddon, W., Jr., M.D.: “Energy Damage and the Ten Countermeasure Strategies.” Human Factors
Journal, August 1973.
Johnson, W.G.: “MORT Safety Assurance Systems.” Marcel Dekker, Inc., 1980.
3.4 Failure Modes and Effects (and Criticality) Analysis
3.4.1 Description
An FMEA, as described in reference 3.5, is a forward logic (bottomup), tabular technique that
explores the ways or modes in which each system element can fail and assesses the consequences of
each of these failures. In its practical application, its use is often guided by topdown “screening” (as
described in sec. 3.4.3) to establish the limit of analytical resolution. An FMECA also addresses the
criticality or risk of individual failures. Countermeasures can be defined, for each failure mode, and
consequent reductions in risk can be evaluated. FMEA and FMECA are useful tools for cost and benefit
studies (sec. 2.2), to implement effective risk mitigation and countermeasure, and as precursors to a fault
tree analysis (FTA) (sec. 3.5).
321
3.4.2 Application
An FMEA can be used to call attention to system vulnerability to failures of individual compo
nents. Singlepoint failures can be identified. This tool can be used to provide reassurance that the cause,
effect, and associated risk (FMECA) of component failures have been appropriately addressed. These
tools are applicable within systems or at the systemsubsystem interfaces and can be applied at the
system, subsystem, component, or part levels.
These failure mode analyses are typically performed during phase C. During this phase, these
analyses can be done with or shortly after the PHA (sec. 3.2). The vulnerable points identified in the
analyses can aid management in making decisions to allocate resources in order to reduce vulnerability.
3.4.3 Procedures
Procedures for preparing and performing FMECA’s, as described in reference 3.5, are presented
below. Procedures for preparing an FMEA are the same with steps 8 through 12 omitted.
Steps prior to performing the FMEA or FMECA:
(1) Define the scope and boundaries of the system to be assessed. Gather pertinent information
relating to the system, such as requirement specifications, descriptions, drawings, compo
nents and parts lists, etc. Establish the mission phases to be considered in the analysis.
(2) Partition and categorize the system into advantageous and reasonable elements to be
analyzed. These system elements include subsystems, assemblies, subassemblies,
components, and piece parts.
(3) Develop a numerical coding system that corresponds to the system breakdown (fig. 39).
Steps in performing the FMEA or FMECA (see flowchart presented in fig. 310):
(4) Identify resources of value, such as personnel, facilities, equipment, productivity, mission
or test objectives, environment, etc. to be protected. These resources are targets.
(5) Identify and observe the levels of acceptable risk that have been predetermined and
approved by management. These limits may be the risk matrix boundaries defined in a risk
assessment matrix (sec. 3.2).
(6) By answering the following questions posed in reference 3.5, the scope and resources
required to perform a classical FMEA can be reduced, without loss of benefit:
a. Will a system failure render an unacceptable or unwanted loss?
If the answer is no, the analysis is complete. Document the results. (This has the addi
tional benefit of providing visibility of nonvalue added systems, or it may serve to
correct incomplete criteria being used for the FMEA.) If the answer is yes, ask the
following question for each subsystem identified in step 2 above.
322
Subassembly 1
Typical Coding System: Subsystem No.  Assembly No.  Subassembly No.  Component No.  Part No.
For example, code number for part 2 above is 0301030102
System
Subsystem 2 Subsystem 3 Subsystem 4
Assembly 1 Assembly 2 Assembly 3
Subassembly 2 Subassembly 3
Component 1 Component 2 Component 3
Part 1 Part 2
Part 3
Subsystem 1
Figure 39. Example of system breakdown and numerical coding.
3.5
b. Will a subsystem failure render an unacceptable or unwanted loss?
If the answer for each subsystem is no, the analysis is complete. Document the results.
If the answer is yes for any subsystem, ask the following question for each assembly of
those subsystems identified in step 2 above.
c. Will an assembly failure render an unacceptable or unwanted loss?
If the answer for each assembly is no, the analysis is complete. Document the results. If
the answer is yes for any assembly, ask the following question for each component of
those assemblies identified in step 2 above.
d. Will a subassembly failure render an unacceptable or unwanted loss?
If the answer for each subassembly is no, the analysis is complete. Document the
results. If the answer is yes for any subassembly, ask the following question for each
component of those subassemblies identified in step 2 above:
323
Figure 310. FMECA process flowchart.
324
e. Will a component failure render an unacceptable or unwanted loss?
If the answer for each component is no, the analysis is complete. Document the results.
If the answer is yes for any component, ask the following question for each part of
those components as identified in step 2 above.
f. Will a part failure render an unacceptable or unwanted loss?
(7) For each element (system, subsystem, assembly, subassembly, component, or part) for
which failure would render an unacceptable or unwanted loss, ask and answer the following
questions:
a. What are the failure modes (ways to fail) for this element?
b. What are the effects (or consequences) of each failure mode on each target?
(8) Assess worstcredible case (not the worstconceivable case) severity and probability for
each failure mode, effect, and target combination.
(9) Assess risk of each failure mode using a risk assessment matrix (sec. 3.1). The matrix
should be consistent with the established probability interval and force or fleet size for this
assessment.
(10) Categorize each identified risk as acceptable or unacceptable.
(11) If the risk is unacceptable, then develop countermeasures to mitigate the risk.
(12) Then reevaluate the risk with the new countermeasure installed.
(13) If countermeasures are developed, determine if they introduce new hazards or intolerable or
diminished system performance. If added hazards or degraded performance are unaccept
able, develop new countermeasures and reevaluate the risk.
(14) Document your completed analysis on an FMEA or FMECA worksheet. The contents and
formats of these worksheets vary among organizations. Countermeasures may or may not
be listed. Typically, the information found in an FMECA worksheet, according to reference
3.5, is presented in figure 311. A worksheet for an FMEA would be similar with the risk
assessment information removed. A sample FMEA worksheet is attached in appendix F.
325
FMEA NO: ________________________
PROJECT NO.: ______________________
SUBSYSTEM NO.: __________________
SYSTEM NO.: ______________________
PROB. INTERVAL: ___________________
FAILURE MODES, EFFECTS,
AND CRITICALITY ANALYSIS
WORKSHEET
SHEET ___ OF ____
DATE _________________________
PREPARED BY: __________________
REVIEWED BY: __________________
APPROVED BY:__________________
TARGET CODE: P – PERSONNEL / E – EQUIPMENT / T – DOWNTIME / R – PRODUCTS / D –DATA / V –ENVIRONMENT
Item/ Failure Failure Failure
T
a
r
Risk Assessment
Action Required/
Id. No. Functional
Ident.
Mode Cause Event g
e
t
S
e
v
P
r
o
b
R C
i o
s d
k e
Comments
Figure 311. Typical FMECA worksheet.
3.5
3.4.4 Example
An example FMECA
3.5
is illustrated in figure 312. The system being assessed is an automated
mountain climbing rig. A schematic of the system is presented in figure 312(a). Figure 312(b)
illustrates the breakdown and coding of the system into subsystem, assembly, and subassembly
elements. An FMECA worksheet for the control subsystem is presented in figure 312(c).
326
(a) System.
Subsystem Assembly Subassembly
Hoist (A) Motor (A01) Windings (A01a)
Inboard bearing (A01b)
Outboard bearing (A01c)
Rotor (A01d)
Stator (A01e)
Frame (A01f)
Mounting plate (A01g)
Wiring terminals (A01h)
Drum (A02)
External power source (B)
Cage (C) Frame (C01)
Lifting Lug (C02)
Cabling (D) Cable (D01)
Hook (D02)
Pulleys (D03)
Controls (E) Switch (E01) START (E01a)
FULL UP LIMIT (E01b)
Wiring (E01c)
(b) System breakdown and coding.
Figure 312. Example of an FMECA—Continued
327
FMEA NO: ________________________
PROJECT NO.:
______________________
SUBSYSTEM NO.: Controls
SYSTEM NO.: Mountain Climbing
Rig_____
PROB. INTERVAL: 30 years
___________
FAILURE MODES, EFFECTS,
AND CRITICALITY ANALYSIS
WORKSHEET
SHEET ___ OF ____
DATE _________________________
PREPARED BY: __________________
REVIEWED BY: __________________
APPROVED BY:__________________
TARGET CODE: P – PERSONNEL / E – EQUIPMENT / T – DOWNTIME / R – PRODUCTS / D – DATA / V – ENVIRONMENT
Item/ Failure Failure Failure
T
a
r
Risk Assessment Action Required/
Id. No. Functional
Ident.
Mode Cause Event g
e
t
S
e
v
P
r
o
b
R C
i o
s d
k e
Comments
E–01–a Start switch Switch fails
closed.
Mechanical
failure or
corrosion.
Cage will
not move.
P
E
T
IV
IV
IV
C
C
C
3
3
3
E–01–b Full up switch Switch fails
open.
Mechanical
failure or
corrosion.
Cage does
not stop.
P II A 1
E–02 Wiring Cut, dis
connected.
Varmint
invasion.
No
response a
switch.
Start switch
fails open.
Stop switch
fails
closed.
Cage stays
in safe
position.
P
E
T
IV
IV
IV
D
D
D
3
3
3
(c) Worksheet.
Figure 312. Example of an FMECA—Continued.
3.4.5 Advantages
Performing FMEA’s and FMECA’s provide the following advantages:
3.5
(1) Provides a mechanism to be exhaustively thorough in identifying potential singlepoint
failures and their consequences. An FMECA provides risk assessments of these failures.
(2) Results can be used to optimize reliability, optimize designs, incorporate “fail safe”
features into the system design, obtain satisfactory operation using equipment of “low
reliability,” and guide in component and manufacturer selection.
(3) Provide further analysis at the piecepart level for highrisk hazards identified in a PHA.
328
(4) Identify hazards caused by failures to be added to the PHA that may have been previously
overlooked in the PHA.
(5) Provides a mechanism for more thorough analysis than an FTA, since every failure mode of
each component of the system is assessed.
3.6
3.4.6 Limitations
The following limitations are imposed when performing FMEA’s and FMECA’s.
(1) Costly in manhour resources, especially when performed at the partscount level within
large, complex systems.
(2) Probabilities or the consequences of system failures induced by coexisting, multiple
element faults or failures within the system are not addressed or evaluated.
(3) Although systematic, and guidelines/check sheets are available for assistance, no check
methodology exists to evaluate the degree of completeness of the analysis.
(4) This analysis is heavily dependent upon the ability and expertise of the analyst for finding
all necessary modes.
(5) Human error and hostile environments frequently are overlooked.
(6) Failure probability data are often difficult to obtain for an FMECA.
(7) If too much emphasis is placed on identifying and eliminating singlepoint failures, then
focus on more severe system threats may be overlooked.
(8) An FMECA can be a very thorough analysis suitable for prioritizing resources to higher
risk areas if it can be performed early enough in the design phase. However, the level of
design maturity required for an FMECA is not generally achieved until late in the design
phase, often too late to guide this prioritization.
3.4.7 Bibliography
Layton, D.: “System Safety – Including DOD Standards.” Weber Systems Inc., Chester, OH, 1989.
Lees, F.P.: “Loss Prevention in the Process Industries.” 2 vols. Butterworths, London, 1980.
MIL–STD–1629A, “Procedures for Performing a Failure Modes, Effects, and Criticality Analysis.”
November 1980.
Raheja, D.G.: “Assurance Technologies – Principles and Practices.” McGrawHill, Inc., 1991.
Roberts, N.H., Vesely, W.E., Haasl, D.F., and Goldberg, F.F.: “Fault Tree Handbook.” NUREG–0492.
U.S. Government Printing Office, Washington, DC.
Roland, H.E., and Moriarty, B.: “System Safety Engineering and Management. ” John Wiley & Sons,
1982.
329
3.5 Reliability Block Diagram
3.5.1 Description
A reliability block diagram (RBD) is a backwards (topdown) symbolic logic model generated in
the success domain. The descriptions of RDB’s contained herein was obtained from references 3.7 and
3.8. Each RBD has an input and an output and flows left to right from the input to the output. Blocks
may depict the events or system element functions within a system. However, these blocks typically
depict system element functions only. A system element can be a subsystem, subassembly, component,
or part.
Simple RBD’s are constructed of series, parallel, or combinations of series and parallel elements
(table 34). Each block represents an event or system element function. These blocks are connected in
series if all elements must operate successfully for the system to operate successfully. These blocks are
connected in parallel if only one element needs to operate successfully for the system to operate success
fully. A diagram may contain a combination of series and parallel branches. The system operates if an
uninterrupted path exists between the input and output.
Table 34 . Simple RBD construction.
Type branch Block diagram representation System reliability #
Series
A B
R
S
= R
A *
R
B
Parallel
A
B
R
S
= 1 – (1–R
A
)(1–R
B
)
Seriesparallel
A
B
C
D
R
S
= (1 – (1–R
A)
(1–R
B
))
* (1 – (1–R
C)
(1–R
D
))
Parallelseries
A
C
B
D
R
S
= 1 – (1 – (R
A *
R
B
))
*
(1 – (R
C *
R
D
))
#
Assumes all components function independently of each other.
330
RBD’s illustrate system reliability. Reliability is the probability of successful operation during a
defined time interval. Each element of a block diagram is assumed to function (operate successfully) or
fail independently of each other element. The relationships between element reliability and system relia
bility for series and parallel systems are presented below, and their derivations are found in reference
3.8.
Series Systems:
R
S
= ∏
i
n
R
i
= R
1*
R
2
*
R
3 *
. . . .
Rn.
Parallel Systems:
R
S
= 1–∏
i
n
(1–R
i
) = [1–(1–R
1
)
*
(1–R
2
) (1–R
3
)
*
. . . .
(1–Rn )]
where
R
S
= system reliability,
R
i
= system element reliability, and
n = number of system elements (which are assumed to function independently).
Not all systems can be modeled with simple RBD’s. Some complex systems cannot be modeled
with true series and parallel branches. These systems must be modeled with a complex RBD. Such an
RBD is presented in figure 313. Notice in this example, if element E fails, then paths B, E, G and B, E,
H are not success paths, thus this is not a true series or parallel arrangement.
A
C
D
F
B E
G
H
Figure 313. Typical complex RBD.
331
3.5.2 Application
An RBD allows evaluation of various potential design configurations.
3.8
Required subsystem and
element reliability levels to achieve the desired system reliability can be determined. Typically, these
functions are performed during phase C. An RBD may also be used to identify elements and logic as a
precursor to performing an FTA (sec. 3.6).
3.5.3 Procedures
The procedures (adapted from reference 3.8) to generate a simple RBD are presented below:
(1) Divide a system into its elements. A functional diagram of the system is helpful.
(2) Construct a block diagram using the convention illustrated in table 34.
(3) Calculate system reliability band, R
SL
(low) to R
SH
(high), from each individual element’s
reliability band, R
iL
(low) to R
iH
(high), in the following manner:
a. For series systems with n elements that are to function independently,
R
SL
= ∏
i
n
(R
iL
) = R
1L *
R
2L *
R
3L *
. . . .
R
nL
R
SH
= ∏
i
n
(R
iH
) = R
1H *
R
2H *
R
3H *
. . . .
R
nH.
b. For parallel systems with n elements that are to function independently,
R
SL
= 1–∏
i
n
(1–R
pL
) = [1–(1–R
1L
)
*
(1–R
2L
) (1–R
3L
)
*
. . . .
(1–R
nL
)]
R
SH
= 1–∏
i
n
(1–R
pH
) = [1–(1–R
1H
)
*
(1–R
2H
) (1–R
3H
)
*
. . . .
(1–R
nH
)].
c. For seriesparallel systems, first determine the reliability for each parallel branch using
the equations in step 3b. Then treat each parallel branch as an element in a series branch
and determine the system reliability by using the equations in step 3a.
d. For parallelseries systems, first determine the reliability for each series branch using
the equations in step 3a. Then treat each series branch as an element in a parallel branch
and determine the system reliability by using the equations in step 3b.
332
e. For systems that are composed of the four above arrangements, determine the reliability
for the simplest branches. Then, treat these as branches within the remaining block dia
gram, and determine the reliability for the new simplest branches. Continue this process
until one of the above four basic arrangements remains, then determine the system
reliability.
3.5.4 Example
A system has two subsystems designated 1 and 2. Subsystem 2 is designed to be a backup for
subsystem 1. Subsystem 1 has three components and at least one of the three must function successfully
for the subsystem to operate. Subsystem 2 has three components that all need to function successfully
for the subsystem to operate. The estimated reliability band for each individual component over the
system’s estimated 10yr life interval is presented below:
Subsystem Component Reliability Bands
Low High
1 A 0.70 0.72
1 B 0.80 0.84
1 C 0.60 0.62
2 D 0.98 0.99
2 E 0.96 0.97
2 F 0.98 0.99
An RBD for the system is presented in figure 314. Note that the components for subsystem 1 are
in a parallel branch with the components of subsystem 2. Also, note that the components for subsystem 1
form a series branch and the components for subsystem 2 form a parallel branch.
D E F
A
B
0.70  0.72
0.80  0.84
0.980.99 0.960.97 0.980.99
C
0.60  0.62
Figure 314. Example RBD.
333
C alculations for subsystem and system reliabilities are presented below:
Subsystem 1: R
1L
= 1 – (1–0.70)(1–0.80)(1–0.60) = 0.976 (low band value)
R
1H
= 1 – (1–0.72)(1–0.84)(1–0.62) = 0.983 (high band value)
Subsystem 2: R
2L
= (0.98)(0.96)(0.98) = 0.922 (low band value)
R
2H
= (0.99)(0.97)(0.99) = 0.951 (high band value)
System: R
SL
= 1 – (1–0.976)(1–0.922) = 0.998 (low band value)
R
SH
= 1 – (10.983)(1–0.951) = 0.999 (high band value)
Therefore, the reliability band for the system is 0.998 to 0.999.
3.5.5 Advantages
An RBD provides the following advantages:
(1) Allows early assessment of design concepts when design changes can be readily and
economically incorporated.
3.8
(2) Tends to be easier for an analyst to visualize than other logic models, such as a fault tree.
3.7
(3) Blocks representing elements in an RBD can be arranged in a manner that represent how
these elements function in the system.
3.7
(4) Since RBD’s are easy to visualize, they can be generated prior to performing an FTA and
transformed into a fault tree by the method discussed in section 3.9.
3.5.6 Limitations
An RBD possesses the following limitations:
(1) Systems must be broken down into elements where reliability estimates can be obtained.
Such a breakdown for a large system can be a significant effort.
3.8
(2) System element reliability estimates might not be readily available for all elements. Some
reliability estimates may be very subjective, difficult to validate, and not be accepted by
others in the decision making process. If the element reliability values have different
confidence bands, this can lead to significant problems.
(3) Not all systems can be modeled with combinations of series, parallel, seriesparallel, or
parallelseries branches. These complex systems can be modeled with a complex RBD.
However, determining system reliability for such a system is more difficult than for a
simple RBD.
3.73.8
334
3.5.7 Bibliography
Pages, A. and Godran, M.: “System Preliminary Evaluation & Prediction in Engineering.” Springer
Verb, 1986.
3.6 Fault Tree Analysis
3.6.1 Description
A fault tree analysis (FTA), as described in reference 3.9, is a topdown symbolic logic model
generated in the failure domain. This model traces the failure pathways from a predetermined,
undesirable condition or event, called the TOP event, of a system to the failures or faults (fault tree
initiators) that could act as causal agents. Previous identification of the undesirable event also includes a
recognition of its severity. An FTA can be carried out either quantitatively or subjectively.
The FTA includes generating a fault tree (symbolic logic model), entering failure probabilities
for each fault tree initiator, propagating failure probabilities to determining the TOP event failure
probability, and determining cut sets and path sets. A cut set is any group of initiators that will, if they
all occur, cause the TOP event to occur. A minimal cut is a least group of initiators that will, if they all
occur, cause the TOP event to occur. A path set is a group of fault tree initiators that, if none of them
occurs, will guarantee the TOP event cannot occur.
The probability of failure for a given event is defined as the number of failures per number of
attempts. This can be expressed as:
P
F
= F/(S+F) , where F = number of failures and S = number of successes.
Since reliability for a given event is defined as the number of successes per number of attempts,
then the relationship between the probability of failure and reliability can be expressed as follows:
R = S/(S+F),
therefore
R + P
F
= S/(S+F) + F/(S+F) = 1
and
P
F
= 1–R.
3.6.2 Application
FTA’s are particularly useful for high energy systems (i.e., potentially high severity events), to
ensure that an ensemble of countermeasures adequately suppresses the probability of mishaps. An FTA
is a powerful diagnostic tool for analysis of complex systems and is used as an aid for design
improvement.
335
This type of analysis is sometimes useful in mishap investigations to determine cause or to rank
potential causes. Action items resulting from the investigation may be numerically coded to the fault tree
elements they address, and resources prioritized by the perceived highest probability elements.
FTA’s are applicable both to hardware and nonhardware systems and allow probabilistic assess
ment of system risk as well as prioritization of the effort based upon root cause evaluation. The
subjective nature of risk assessment is relegated to the lowest level (root causes of effects) in this study
rather than at the top level. Sensitivity studies can be performed allowing assessment of the sensitivity of
the TOP event to basic initiator probabilities.
FTA’s are typically performed in phase C but may also be performed in phase D. FTA’s can be
used to identify cut sets and initiators with relatively high failure probabilities. Therefore, deployment of
resources to mitigate risk of highrisk TOP events can be optimized.
3.6.3 Procedures
The procedures, as described in reference 3.9, for performing an FTA are presented below. These
procedures are divided into the four phases: (1) fault tree generation, (2) probability determination,
(3) identifying and assessing cut sets, and (4) identifying path sets. The analyst does not have to perform
all four phases, but can progress through the phases until the specific analysis objectives are met. The
benefits for each of the four phases are summarized in table 35.
Table 35. FTA procedures.
Section Procedures Benefits
3.6.3.1 Fault tree generation All basic events (initiators), intermediate events, and the TOP
event are identified. A symbolic logic model illustrating fault
propagation to the TOP event is produced.
3.6.3.2 Probability determination Probabilities are identified for each initiator and propagated to
intermediate events and the TOP event.
3.6.3.3 Identifying and assessing cut sets All cut sets and minimal cuts sets are determined. A cut set is
any group of initiators that will, if they all occur, cause the
TOP event to occur. A minimal cut set is a least group of
initiators that, if they all occur, will cause the TOP event to
occur. Analysis of a cut set can help evaluate the probability
of the TOP event, identify qualitative common cause
vulnerability, and assess quantitative common cause
probability. Cut sets also enable analyzing structural,
quantitative, and item significance of the tree.
3.6.3.4 Identifying path sets All path sets are determined. A path set is a group of fault tree
initiators that, if none of them occurs, will guarantee the TOP
event cannot occur.
336
3.6.3.1 Fault Tree Generation
Fault trees are constructed with various event and gate logic symbols, defined in table 36.
Although many event and gate symbols exist, most fault trees can be constructed with the following four
symbols: (1) TOP or Intermediate event, (2) inclusive OR gate, (3) AND gate, and (4) basic event. The
procedures, as described in reference 3.9, to construct a fault tree are illustrated in figure 315.
A frequent error in fault tree construction is neglecting to identify common causes. A common
cause is a condition, event, or phenomenon that will simultaneously induce two or more elements of the
fault tree to occur. A method for detecting common causes is described in section 3.6.3.3, step 8.
Sections 3.6.3.2 through 3.6.4.3 are included for completeness and to provide insight as to the
mathematics that takes place in the commercially available fault tree programs. All large trees are
typically analyzed using these programs; for small trees hand analysis may be practical.
3.6.3.2 Probability Determination
If a fault tree is to be used as a quantitative tool, the probability of failure must be determined for
each basic event or initiator. Sources for these failure probabilities may be found from manufacturer’s
data, industry consensus standards, MILstandards, historical evidence (of the same or similar systems),
simulation or testing, Delphi estimates, and the log average method. A source for human error probabili
ties is found in reference 3.10. The Delphi technique (sec. 7.9) derives estimates from the consensus of
experts. The log average method is useful when the failure probability cannot be estimated but credible
upper and lower boundaries can be estimated. This technique is described in reference 3.11 and is
illustrated in figure 316.
Failure probabilities can also be determined from a PDA as discussed in section 3.14.3, step 6.
Probabilities must be used with caution to avoid the loss of credibility of the analysis. In many
cases it is best to stay with comparative probabilities rather than the “absolute” values. Normalizing data
to a standard, explicitly declared meaningless value is a useful technique here. Also, confidence or error
bands, on each cited probability number, are required to determine the significance of any quantitatively
driven conclusion.
Once probabilities are estimated for all basic events or initiators, they are propagated through
logic gates to the intermediate events and finally the TOP event. The probability of failure of
independent inputs through an AND gate is the intersection of their respective individual probabilities.
The probability of failure of independent events through an OR (inclusive) gate is the union of their
respective individual probabilities. Propagation of confidence and error bands is performed simply by
propagation of minimum and maximum values within the tree.
The relationship between reliability and failure probability propagation of two and three inputs
through OR (inclusive) and AND gates is illustrated in figure 317. Propagation of failure probabilities
for two independent inputs through an AND and OR (inclusive) is conceptually illustrated in figure 3
18. As shown in figures 317, the propagation solution through an OR gate is simplified by the rare
event approximation assumption. The exact solution for OR gate propagation is presented in figure 319.
However, the use of this exact solution is seldom warranted.
The propagation equations for the logic gates, including the gates infrequently used, are
presented in table 37.
337
Table 36. Fault tree construction symbols.
Symbol Name Description
Event (TOP or
intermediate)
*
TOP Event – This is the conceivable, undesired event to which
failure paths of lower level events lead.
Intermediate Event – This event describes a system condition
produced by preceding events.
Inclusive OR gate
*
An output occurs if one or more inputs exist. Any single input is
necessary and sufficient to cause the output event to single occur.
Exclusive OR gate An output occurs if one, but only one input exists. Any single
input is necessary and sufficient to cause the output event to
occur.
M
Mutually exclusive
OR gate
An output occurs if one or more inputs exist. However, all other
inputs are then precluded. Any single input is necessary and
sufficient to cause the output event to occur.
AND gate
*
An output occurs if all inputs exist. All inputs are necessary and
sufficient to cause the output event to occur.
Priority AND gate An output occurs if all inputs exist and occur in a predetermined
sequence. All inputs are necessary and sufficient to cause the
output event to occur.
INHIBIT gate An output occurs if a single input event occurs in presence of an
enabling condition.
Basic event
*
An initiating fault or failure that is not developed further. These
events determine the resolution limit of the analysis. They are
also called leaves or initiators.
External event An event that under normal conditions is expected to occur.
Undeveloped event An event not further developed due to a lack of need, resources,
or information.
Conditioning Event These symbols are used to affix conditions, restraints, or
restrictions to other events.
*
Most fault trees can be constructed with these four logic symbols.
338
6. Repeat /
continue…
4. Identify secondlevel
contributors.
5. Link secondlevel
contributors to TOP by
logic gates.
1. Identify undesirable TOP
event.
3. Link contributors to TOP
by logic gates.
2. Identify firstlevel
contributors.
Basic Event…(“Leaf,”
“Initiator,” or “Basic”)
indicates limit of analytical
resolution.
Figure 315. Fault tree construction process.
• Estimate upper and lower credible bounds of probability for the phenomenon in question.
• Average the logarithms of the upper and lower bounds.
• The antilogarithm of the average of the logarithms of the upper and lower bounds is less
than the upper bound and greater than the lower bound by the same factor. Thus, it is
geometrically midway between the limits of estimation.
0.1
0.0316
+
0.01 0.02 0.03 0.04 0.05 0.07
PL
Lower
Probability
Bound
10
–2
PU
Upper
Probability
Bound
10
–1
Log P
L
+ Log P
U
2
(–2) + (–1)
2
Log Average = Antilog = Antilog = 10
–1.5
= 0.0316228
Note that, for the example shown, the arithmetic average would be…
i.e., 5.5 times the lower bound and 0.55 times the upper bound.
0.01 + 0.1
2
= 0.055
Figure 316. Log average method of probability estimation.
339
P
F
= 1 – R
T
P
F
= 1 – (R
A
R
B
)
P
F
= 1 – [(1 – P
A
)(1 – P
B
)]
R + P
F
≡ 1
OR Gate AND Gate
Either of two, independent,
element failures produces
system failure.
Both of two, independent
elements must fail to produce
system failure.
R
T
= R
A
+ R
B
– R
A
R
B
R
T
= R
A
R
B
P
F
= 1 – R
T
P
F
= 1 – (R
A
+ R
B
– R
A
R
B
)
P
F
= 1 – [(1 – P
A
) + (1 – P
B
) – (1 – P
A
)(1 – P
B
)]
P
F
= P
A
P
B
P
F
= P
A
+ P
B
– P
A
P
B
“Rare
Event
Approximation”
…for P
A,B
≤ 0.2
P
F
≅ P
A
+ P
B
with error ≤ 11%
P
F
= P
A
P
B
P
C Omit for
approximation
For 3 Inputs
For 2 Inputs
P
F
= P
A
+ P
B
+ P
C
– P
A
P
B
– P
A
P
C
– P
B
P
C
+ P
A
P
B
P
C
[Union / ] [Intersection / ]
Figure 317. Relationship between reliability and failure probability propagation.
Intersection /
P
T
= P
1
+ P
2
– P
1
P
2
P
T
= P
1
P
2
AND Gate…
TOP
P
1
P
2
P
T
= P
1
P
2
1 2
OR Gate…
TOP
P
1
P
2
P
T
P
1
+ P
2
1 2
1 & 2
are
INDEPENDENT
events
Usually negligible…
P
T
= P
e
P
T
P
e
Union /
Figure 318. Failure probability propagation through OR and AND gates.
340
The ip operator ( ) is the
cofunction of pi ( ∏ ). It
provides an exact solution
for propagating probabilities
through the OR gate. Its use
is rarely justifiable.
P
T
= P
e
= 1 – P (1 – P
e
)
P
T
= 1 – [(1 – P
1
) (1 – P
2
) (1 – P3 ) (1 – Pn
)] …
TOP
P
1
P
2
1 2
3
P
3
P
T
= P
e
TOP
P
1
P
2
1 2
3
P
3
P
T
= ?
TOP
1 2
3
P
T
= (1 – P
e
)
P
1
= (1 – P
1
)
P
2
= (1 – P
2
)
P
3
= (1 – P
3
)
Failure
Failure Success
Figure 319. Exact solution of OR gate failure probability propagation.
3.6.3.3 Identifying and Assessing Cut Sets
A cut set is any group of initiators that will produce the TOP event, if all the initiators in the
group occur. A minimal cut set is the smallest number (in terms of elements, not probability) of initiators
that will produce the TOP event, if all the initiators in the group occur. One method of determining and
analyzing cut sets is presented below. These procedures for determining cut sets are described in
reference 3.9 and are based on the MOCUS computer algorithm attributed to J.B. Fussell. Analysis of a
cut set can help evaluate the probability of the TOP event, identify common cause vulnerability, and
assess common cause probability. Cut sets also enable analyzing structural, quantitative, and item
significance of the tree.
Determining Cut Sets:
(1) Consider only the basic events or initiators (discarding intermediate events and the TOP
event).
(2) Assign a unique letter to each gate and a unique number to each initiator, starting from the
top of the tree.
(3) From the top of the tree downwards, create a matrix using the letters and numbers. The
letter for the gate directly beneath the TOP event will be the first entry in the matrix.
Proceed through the matrix construction by (1) substituting the letters for each AND gate
with letters for the gates and numbers of the initiators that input into that gate (arrange
these letters and numbers horizontally in the matrix rows), and (2) substituting the letters
for each OR gate with letters for the gates and numbers of the initiators that input into that
gate (arrange these letters and numbers vertically in the matrix columns).
341
Table 37. Probability propagation expressions for logic gates.
Symbol Name Venn Diagram Propagation Expressions
Inclusive OR
gate
‡
P1 P2
P
T
= P
1
+ P
2
– (P
1
*
P
2
)
P
T
= P
1
+ P
2
#
Exclusive OR
gate
P2 P1
P
T
= P
1
+ P
2
–2 (P
1
*
P
2
)
P
T
= P
1
+ P
2
#
M
Mutually
exclusive OR
gate
P1 P2
P
T
= P
1
+ P
2
AND gate
‡
and
(priority AND
gate)
P1 P2
P
T
= P
1 *
P
2
‡
Most fault trees can be constructed with these two logic gates.
#
Simplified expression for rare event approximation assumption.
(4) When all the gate’s letters have been replaced, a final matrix is produced with only
numbers of initiators. Each row of this matrix represents a Booleanindicated cut set.
(5) Visually inspect the final matrix and eliminate any row that contains all elements of a lesser
row. Next, through visual inspection, eliminate redundant elements within rows and rows
that repeat other rows. The remaining rows define the minimal cut sets of the fault tree.
Assessing Cut Sets:
(6) Since a cut set is any group of initiators that will produce the TOP event, if all the initiators
in the group occur, the cut set probability, P
K
(the probability that the cut set will induce the
TOP event) is mathematically the same as the propagation through an AND gate, expressed
as:
P
K
= P
1 *
P
2 *
P
3*
P
4
...
*
P
n
.
342
(7) Determine common cause vulnerability by uniquely assigning letter subscripts for common
causes to each numbered initiator (such as m for moisture, h for human operator, q for heat,
v for vibration, etc.). Note that some initiators may have more than one subscript, while
others will have none. Check to see if any minimal cut sets have elements with identical
subscripts. If that is the case, then the TOP event is vulnerable to the common cause the
subscript represents. This indicates that the probability number, calculated as above, may
be significantly in error, since the same event (the socalled common cause) could act to
precipitate each event, i.e., they no longer represent statistically independent events.
(8) Analyze the probability of each common cause at its individual probability level of both
occurring, and inducing all terms within the affected cut set.
(9) Assess the structural significance of the cut sets to provide qualitative ranking of contribu
tions to system failure. Assuming all other things are equal then:
a. A cut set with many elements indicates low vulnerability.
b. A cut set with few elements indicates high vulnerability.
c. Numerous cut sets indicates high vulnerability.
d. A cut set with a single initiator, called a singleton, indicates a potential singlepoint
failure.
(10) Assess the quantitative Importance, I
K
,
of each cut set, K. That is, determine the numerical
probability that this cut set induced the TOP event, assuming it has occurred.
I
K
= P
K
/P
T
where
P
K
= the probability that the cut set will occur (see step 6 above), and
P
T
= the probability of the TOP event occurring.
(11) Assess the quantitative importance, I
e,
of each individual initiator, e. That is, determine the
numerical probability that initiator e contributed to the TOP event, if it has occurred:
I
e
= I
K
e
e
N
e
∑
where
N
e
= number of minimal cut sets containing initiator e, and
I
K
e
= importance of the minimal cut sets containing initiator e.
3.6.3.4 Identifying Path Sets
A path set is a group of fault tree initiators that, if none of them occurs, ensures the TOP event
cannot occur. Path sets can be used to transform a fault tree into a reliability diagram (sec. 3.9). The
procedures to determine path sets are as follows:
(1) Exchange all AND gates for OR gates and all OR gates for AND gates on the fault tree.
343
(2) Construct a matrix in the same manner as for cut sets (sec. 3.6.3.3, steps 1–5).
Each row of the final matrix defines a path set of the original fault tree.
3.6.4 Examples
3.6.4.1 Fault Tree Construction and Probability Propagation
An example of a fault tree with probabilities propagated to the TOP event is presented in
figure 320. In this example the TOP event is the “artificial wakeup fails.” The system being examined
consists of alarm clocks used to awaken someone. In this example for brevity, only a nominal
probability value for each fault tree initiator is propagated through the fault tree to the TOP event.
However, for a thorough analysis, both low and high probability values that define a probability band for
each initiator could be propagated through the fault tree to determine a probability band for the TOP
event.
Faults/Operation……8x10
3
Rate, Faults/Year………2/1
Assume 260 Operations/Year
KEY:
ARTIFICIAL
WAKEUP
FAILS
3.34x10
4
ALARM
CLOCKS
FAIL
3.34x10
4
MAIN,
PLUGIN
CLOCK FAILS
1.82x10
2
POWER
OUTAGE
1x10
2
FAULTY
INNARDS
3x10
4
ELECTRICAL
FAULT
3x10
4
MECHANICAL
FAULT
8x10
8
HOUR
HAND
FALLS
OFF
4x10
4
HOUR
HAND
JAMS
WORKS
2x10
4
FORGET TO
SET
8x10
3
BACKUP
(WINDUP)
CLOCK FAILS
1.83x10
2
FAULTY
MECHANISM
4x10
4
FORGET TO
SET
8x10
3
FORGET TO
WIND
1x10
2
NOCTURNAL
DEAFNESS
Negligible
Approx. 0.1/yr
3/1 2/1 1/10 2/1
1/15
1/10
3/1
1/20
Figure 320. Example fault tree.
344
3.6.4.2 Cut Sets
An example of how to determine Booleanindicated minimal cut sets for a fault tree is presented
in figure 321.
PROCEDURE:
• Assign letters to
gates. (TOP gate
is “A.”) Do not
repeat letters.
• Assign numbers
to basic initiators.
If a basic appears
more than once,
represent it by the
same number at
each appearance.
TOP
C
D B
A
1
2 3
4 2
• Construct a matrix, starting with the TOP “A” gate…
2 2 3
1 2
1 4
2 4 3
1 2
2 3
1 4
A B D
C D
1 D
2 D 3
1 D
2 D 3
1 4
1 2
TOP event gate
is A, the
initial matrix
entry.
A is an AND
gate. B & D,
its inputs, re
place it hori
zontally.
B is an OR
gate. 1 & C,
its inputs, re
place it verti
cally. Each
requires a new
row.
C is an AND
gate. 2 & 3,
its inputs,
replace it
horizontally.
D (top row), is
an OR gate.
2 & 4, its in
puts, replace
it vertically.
Each requires a
new row.
D (2nd row), is
an OR gate.
Replace as
before.
These BooleanIndicated
Cut Sets…
…reduce to
these Minimal
Cut Sets.
Minimal Cut Set
rows are least
groups of
initiators which
will induce TOP.
Figure 321. Example of determining cut sets.
345
3.6.4.3 Path Sets
An example of how to determine path sets for a fault tree is presented in figure 322.
1 2
1 3
1 4
3 4 5 6
1 3
1 4
1 5
1 6
2 3 4
This
Fault
Tree
has…
…these Minimal
Cut Sets
…and these Path
Sets.
Path Sets are
least groups of
initiators which, if
they cannot
occur, guarantee
against TOP
occurring.
TOP
1
2
3
3
4
6
G
F
E
D
C B
A
5
4 1
“Barring” terms (n) denotes consideration
of their success properties.
Figure 322. Example of determining path sets.
3.6.5 Advantages
An FTA provides the following advantages:
3.9
(1) Enables assessment of probabilities of combined faults/failures within a complex system.
(2) Singlepoint and common cause failures can be identified and assessed.
(3) System vulnerability and lowpayoff countermeasures are identified, thereby guiding
deployment of resources for improved control of risk.
(4) This tool can be used to reconfigure a system to reduce vulnerability.
(5) Path sets can be used in trade studies to compare reduced failure probabilities with
increases in cost to implement countermeasures.
346
3.6.6 Limitations
A FTA possesses the following limitations:
(1) Address only one undesirable condition or event that must be foreseen by the analyst. Thus,
several or many FTA’s may be needed for a particular system.
(2) Fault trees used for probabilistic assessment of large systems may not fit or run on
conventional PCbased software.
(3) The generation of an accurate probabilistic assessment may require significant time and
resources. Caution must be taken not to “over work” determining probabilities or
evaluating the system, i.e., limit the size of the tree.
(4) A fault tree is not accurate unless all significant contributors of faults or failures are
anticipated.
(5) Events or conditions under the same logic gate must be independent of each other.
(6) A fault tree is flawed if common causes have not been identified.
(7) Events or conditions at any level of the tree must be independent and immediate
contributors to the next level event or condition.
(8) The failure rate of each initiator must be constant and predictable. Specific
(noncomparative) estimates of failure probabilities are typically difficult to find, to achieve
agreement on, and to successfully use to drive conclusions. Comparative analyses are
typically as valuable with better receptions from the program and design teams.
3.6.7 Bibliography
Crosetti, P.A.: “Reliability and Faulttree Analysis Guide.” Department of Energy No. DOE
7645/22, 1982.
Dillon, B.S., and Singh, C.: “Engineering Reliability – New Techniques and Applications.” John Wiley
& Sons, 1981.
Fussell, J.B., and Burdick, G.R.: “Nuclear Systems Reliability Engineering and Risk Assessment.”
Society for Industrial and Applied Mathematics, 1977.
Hammer, W: “Handbook of System and Product Safety.” Prentice Hall, 1972.
Henley, E.J., and Kumamoto, H.: “Probabilistic Risk Assessment.” The Institute of Electrical and
Electronic Engineers, Inc., New York, 1991.
Malsaky, S.W.: “System Safety: Technology and Application.” Garland Press, 1983.
Roberts, N.H., Vesely, W.E., Haasl, D.F., and Goldberg, F.F.: “Fault Tree Handbook.” NUREG–0492,
U.S. Government Printing Office, Washington, DC.
Roland, H.E., and Moriarty, B.: “System Safety Engineering and Management.” John Wiley & Sons,
1982.
347
William S.G., Riley, J., and Koren, J.M.: “A New Approach to the Analysis of Reliability Block
Diagrams.” Proceedings from Annual Reliability and Maintainability Symposium, SAIC, Los
Altos, New Mexico, 1990.
Wynholds, W., Potterfield, R., and Bass, L.: “Fault Tree Graphics – Application to System Safety.”
Proceedings of the Second International System Safety Conference, 1975.
3.7 Success Tree Analysis
3.7.1 Description
A success tree analysis (STA) is a backwards (topdown) symbolic logic model generated in the
success domain. This model traces the success pathways from a predetermined, desirable condition or
event (TOP event) of a system to the successes (success tree initiators) that could act as causal agents.
An STA is the compliment of an FTA (sec. 3.6) which is generated in the failure domain with failure
pathways from undesirable events.
The STA includes generating a success tree (symbolic logic model), determining success proba
bilities for each tree initiator, propagating each initiator probability to determining the TOP event
probability and determining cut sets and path sets. In the success domain, a cut set is any group of
initiators that will, if they all occur, prevent the TOP event from occurring. A minimal cut set is a least
group of initiators that will, if they all occur, prevent the TOP event from occurring. A path set is a
group of success tree initiators that, if all of them occur, will guarantee the TOP event occurs.
The probability of success for a given event is defined as the number of successes per number of
attempts. This can be expressed as:
P
s
= S/(S+F) , where S = number of successes and F = number of failures.
Since reliability for a given event is also defined as the number of successes per number of
attempts, then
R = P
S
.
3.7.2 Application
Particularly useful for high energy systems (i.e., potentially high severity events), to ensure that
an ensemble of countermeasures adequately leads to a successful top event. This technique is a powerful
diagnostic tool for analysis of complex systems and is used as an aid for design improvement and is
applicable both to hardware and nonhardware systems. This technique also allows probabilistic assess
ment of causal benefits as well as prioritization of effort based upon root cause evaluation. The
subjective nature of the probability assessment is relegated to the lowest level (root causes of effects) in
this study rather than at top level. Sensitivity studies can be performed allowing assessment of the
sensitivity of study results to subjective numbers.
348
The STA is typically applied in phase C but may also be applied in phase D. A success tree can
be used to verify the logic of a fault tree. Since a success tree is the logic compliment of a fault tree, if a
success tree is generated from a fault tree, the logic of the success tree needs to be valid if the logic of a
fault tree is to be valid.
3.7.3 Procedures
Success trees, like fault trees, are constructed with various event and gate logic symbols. These
symbols are defined in table 36. Although many event and gate symbols exist, most success trees can be
constructed with the following four symbols: (1) TOP or intermediate event, (2) inclusive OR gate, (3)
AND gate, and (4) basic event. The procedures, as described in reference 3.9, to construct a fault tree
also apply to success tree generation and are illustrated in figure 323. The commercial computer
programs are similar, as are the cautions for use of probability values.
6. Repeat / continue…
4. Identify secondlevel
contributors.
5. Link secondlevel
contributors to TOP by
logic gates.
1. Identify desirable TOP
event.
3. Link contributors to TOP
by logic gates.
2. Identify firstlevel
contributors.
Basic Event…(“Leaf,”
“Initiator,” or “Basic”)
indicates limit of analytical
resolution.
Figure 323. Success tree construction process.
A success tree can be constructed from a fault tree. Transform a success tree from a fault tree by
simply changing all AND gates to OR gates and OR gates to AND gates, and restating each initiator,
intermediate event, and top event as a success opposed to a failure.
Determine the probability of success (P
s)
for each basic event or initiator. Sources for these suc
cess probabilities may be found from manufacturer’s data, industry consensus standards, MIL standards,
historical evidence (of similar systems), simulation or testing. Delphi estimates, and the log average
method. The Delphi technique (sec. 7.9) derives estimates from the consensus of experts. Remember that
the probability of success equals reliability (R) and may be determined from (P
F
) as shown in the
following equation:
P
S
= 1 – P
F
.
Once probabilities are estimated for all basic events or initiators, propagate these probabilities
through logic gates to the intermediate events and finally the TOP event. Use the expressions presented
in table 37 to propagate probabilities through logic gates.
349
Generate cut sets and path sets in the same manner as for fault trees, as presented in sections
3.7.3.3 and 3.7.3.4, respectively.
3.7.4 Example
The compliment success tree for the fault tree presented in section 3.6.4 is presented in figure 3
24.
Artificial Wakeup
Succeeds
0.9996
Possess
Keen Hearing
1.000
Alarm Clock Works
Properly
0.9996
Main, Plugin
Clock Works
0.9806
Functioning Clock
Components
0.9997
Mechanical
Component
Success
Hour
Hand Does
Not Jam
Works
0.9998
Hour
Hand Stays
On
0.9996
Electrical
Components
Perform
Properly
0.9997
Uninterrupted
Power
0.9885
Remembered
to Set Alarm
0.9923
Backup Clock
(Windup) Performs
Properly
0.9805
Unflawed
Mechanism
0.9996
Remembered
to Set Backup
Alarm
0.9923
to Wind Clock
0.9885
1.000
Remembered
Figure 324. Example success tree.
350
3.7.5 Advantages
An STA provides the following advantages:
3.9
(1) Assesses probability of favorable outcome of system operation.
(2) Compliments the FTA by providing a method to verify the logic of the fault tree.
3.7.6 Limitations
An STA possesses the following limitations:
3.9
(1) Address only one desirable condition or event that must be foreseen by the analyst. Thus,
several or many STA’s may be needed for a particular system.
(2) Success trees used for probabilistic assessment of large systems may not fit/run on
conventional PCbased software.
(3) The generation of an accurate probabilistic assessment may require significant time and
resources. Caution must be taken not to overdo the number generation portion.
(4) A success tree is not accurate unless all significant contributors to system successes are
anticipated.
(5) Events or conditions under the same logic gate must be independent of each other.
(6) Events or conditions at any level of the tree must be independent and immediate
contributors to the next level event or condition.
(7) The probability of success (reliability) of each initiator must be constant and predictable.
3.7.7 Bibliography
Henley, E.J., and Kumamoto, H.: “Probabilistic Risk Assessment.” The Institute of Electrical and
Electronic Engineers, Inc., New York, 1991.
3.8 Event Tree Analysis
3.8.1 Description
An event tree analysis (ETA), as described in references 3.6 and 3.12, is a forward (bottomup)
symbolic logic modeling technique generated in both the success and failure domain. This technique
explores system responses to an initiating “challenge” and enables assessment of the probability of an
unfavorable or favorable outcome. The system challenge may be a failure or fault, an undesirable event,
or a normal system operating command.
351
A generic event tree portrays all plausible system operating alternate paths from the initiating
event. A generic event tree is illustrated in figure 325. A Bernoulli model event tree uses binary
branching to illustrate that the system either succeeds or fails at each system logic branching node. A
Bernoulli model event tree is illustrated in figure 326. A decision tree is a specialized event tree with
unity probability for the system outcome.
Portray all credible system operating permutations.
.
SUCCESS
FAILURE
INITIATION
OPERATION/
OUTCOME
D
E
C
I
S
I
O
N
/
A
C
T
I
O
N
D
/
A
O/O
A B C
N
1
2
3
n
OPERATION/
OUTCOME
D
E
C
I
S
I
O
N
/
A
C
T
I
O
N
D
E
C
I
S
I
O
N
/
A
C
T
I
O
N
FAILURE
FAILURE
FAILURE
SUCCESS
SUCCESS
SUCCESS
OPERATION/
OUTCOME
Trace each path to eventual success or failure.
Figure 325. Event tree (generic case).
3.8.2 Application
The ETA is particularly useful in analyzing commandstart or commandstop protective devices,
emergency response systems, and engineered safety features. The technique is useful in evaluating
operating procedures, management decision options, and other nonhardware systems. The ETA is also
useful in evaluating effect and benefit of subtiered or redundant design countermeasures for design
trades and assessment.
An ETA may be used in conjunction with an FTA to provide a technique sensitivity assessment.
However, success or failure probabilities used must be used with caution to avoid the loss of credibility
of the analysis. In many cases it is best to stay with comparative probabilities rather than the “absolute”
values. Normalizing data to a standard, explicitly declared meaningless value is a useful technique here.
Also, confidence or error bands, on each cited probability number, are required to determine the signifi
cance of any quantitatively driven conclusion.
An ETA may also be performed to compliment an FMEA. This technique is typically performed
in phase C or E but may also be performed in phase D.
352
Reduce tree to simplified representation of
system behavior. Use binary branching.
Lead unrecoverable failures and
undefeatable successes directly to final
outcomes.
A fault tree or other analysis
may be necessary to determine
probability of the initiating event
or condition. (Unity probability
may be assumed.)
INITIATION
SUCCESS
FAILURE
FAILURE
FAILURE
FAILURE
FAILURE
FAILURE
FAILURE
SUCCESS
SUCCESS
SUCCESS
SUCCESS
SUCCESS
Figure 326. Event tree (Bernoulli model).
3.8.3 Procedures
The procedures, as described in reference 3.12, for performing an ETA are presented below.
(1) Identify the initiating challenge to the system being examined.
(2) Determine the paths (alternate logic sequences) by answering the question, “What happens
when the system is challenged by the initiation event?” By convention, trace successful
paths upwards and failure paths downwards.
a. For the general event tree, trace all plausible system operating permutations to a success
or failure termination.
b. For the Bernoulli model event tree, use binary branching to show the system pathways.
Simplify the tree by pruning unnecessary alternate branches of nonrecoverable failures
or undefeatable successes.
(3) Determine the probability of the initiating event by applying a fault tree (sec. 3.6) or other
analysis. For a decision tree, assume the probability of the initiating event is one.
(4) Determine the probability of each potential path by multiplying the individual probabilities
of events making up the path.
(5) Determine the probability of the system success by adding the probabilities for all paths
terminating in success.
(6) Determine the probability of the system failure by adding the probabilities for all paths
terminating in failure.
353
3.8.4 Example
An example of an ETA is presented in figure 327. The example includes the system and
scenario being assessed and the resulting event tree. Note that in this example the probability of the
challenging initiator is assumed to be one and the tree has been pruned to its simplest form by using
engineering logic. For example, since failure of the float switch is a nonrecoverable failure, its path leads
directly to a final failure outcome with no alternate paths. In a similar manner since successful operation
of the pump is an undefeatable success, its path also leads to a final success outcome with no alternate
paths.
3.8.5 Advantages
An ETA provides the following advantages:
(1) Enables the assessment of multiple, coexisting system faults and failures.
(2) Functions simultaneously in the failure of success domain.
(3) End events need not be anticipated.
(4) Potential singlepoint failures, areas of system vulnerability, and lowpayoff countermea
sures are identified and assessed, thereby guiding deployment of resources for improved
control of risk and optimized utilization of limited resources.
(5) Failure propagation paths of a system can be identified and traced. This can be a “quick and
dirty” comparative technique and provides very clear visibility of ineffective countermea
sures.
3.8.6 Limitations
An ETA possesses the following limitations:
(1) Address only one initiating challenge. Thus, multiple ETA’s may be needed for a particular
system.
(2) The initiating challenge is not disclosed by the analysis, but must be foreseen by the
analyst.
(3) Operating pathways must be foreseen by the analyst.
(4) Although multiple pathways to system failure may be disclosed, the levels of loss
associated with particular pathways may not be distinguishable without additional analyses.
(5) Specific, noncomparative success or failure probability estimates are typically difficult to
find, difficult to achieve agreement on, and to successfully use to drive conclusions.
Comparative analyses are typically as valuable, with better reception from the program and
design teams.
354
KLAXON PUMP
K
P
S
B
BACKGROUND/PROBLEM — A subgrade compartment containing
important control equipment is protected against flooding by the system
shown. Rising flood waters close float switch S, powering pump P from
an uninterruptible power supply. A klaxon K is also sounded, alerting
operators to perform manual bailing, B, should the pump fail. Either
pumping or bailing will dewater the compartment effectively. Assume
flooding has commenced, and analyze responses available to the
dewatering system…
• Develop an event tree representing system responses.
• Develop a reliability block diagram for the system.
• Develop a fault tree for the TOP event Failure to Dewater.
SIMPLIFYING ASSUMPTIONS:
• Power is available full time.
• Treat only the four system components S , P , K, and B.
• Consider operator error as included within the bailing function, B .
EVENT TREE…
[P
B
P
P
– P
B
P
P
P
S
–
P
B
P
K
P
P
+
P
B
P
K
P
P
P
S
]
[1 – P
S
– P
P
+ P
P
P
S
]
[P
P
– P
P
P
S
–
P
K
P
P
+
P
K
P
P
P
S
]
[P
K
P
P
– P
K
P
P
P
S
]
[P
P
– P
P
P
S
]
Water Rises
(1.0)
Float
Switch
Succeeds
(1 – P
S
)
Float
Switch
Fails
(P
S
)
Pump Succeeds
(1 – P
P
)
Pump Fails
(P
P
)
Klaxon Succeeds
(1 – P
K
)
Klaxon Fails
(P
K
)
Bailing Succeeds
(1 – P
B
)
Bailing
Fails
(P
B
)
S
U
C
C
E
S
S
F
A
I
L
U
R
E
[P
S
]
P
SUCCESS
= 1 – P
S
– P
K
P
P
+ P
K
P
P
P
S
– P
B
P
P
+ P
B
P
P
P
S
+ P
B
P
K
P
P
– P
B
P
K
P
P
P
S
P
FAILURE
= P
S
+ P
K
P
P
– P
K
P
P
P
S
+ P
B
P
P
– P
B
P
P
P
S
– P
B
P
K
P
P
+ P
B
P
K
P
P
P
S
P
SUCCESS
+ P
FAILURE
= 1
[P
P
– P
P
P
S
– P
K
P
P
+ P
K
P
P
P
S
–
P
B
P
P
+ P
B
P
P
P
S
+ P
B
P
K
P
P
–
P
B
P
K
P
P
P
S
]
Figure 327. Example ETA.
355
3.8.7 Bibliography
Battelle Columbus Division, “Guidelines for Hazard Evaluation Procedures.” 1985.
Henley, E.J., and Kumamoto, H., “Reliability Engineering and Risk Assessment.” New York, 1981.
Lees, F.P. “Loss Prevention in the Process Industries.” 2 vols., Butterworths, London, 1980.
3.9 Fault Tree, Reliability Block Diagram, and Event Tree Transformations
3.9.1 Description
Fault trees (sec. 3.6), RBD’s (sec. 3.5), and event trees (sec. 3.7) are all symbolic logic models.
Fault trees are generated in the failure domain, reliability diagrams are generated in the success domain,
and event trees are generated in the success and failure domains. These techniques, described in
reference 3.13 and presented below, transform any one of the above models into the other two by
translating equivalent logic from the success to failure or failure to success domain.
3.9.2 Application
These techniques are applicable by the analyst that wishes to exploit the benefits of the fault tree,
RBD, and event tree. Fault trees offer the analyst comprehensive qualitative or quantitative analysis.
RBD’s offer the analyst a simplistic method to represent system logic. Event trees allow the analyst to
assess a system in both the success and failure domains. This technique is typically performed in phase
C but may also be performed in phase B.
3.9.3 Procedures
The procedures for transforming a fault tree, RBD, or event tree to either of the other two logic
models are presented in the following sections.
3.13
3.9.3.1 Fault Tree to RBD Transformation
An RBD represents system component functions that, if these functions prevail, produces
success in place of a TOP fault event. A fault tree can be transformed into a reliability diagram as
illustrated in figure 328.
3.9.3.2 RBD and Fault TreetoEvent Tree Transformation
An event tree represents path sets in the success branches of the tree and all the cut sets in the
failure branches of the tree. Therefore, if the path sets and cut sets of a system are known for a certain
challenge to a system (TOP event of a fault tree), then an event tree can be constructed.
Cut sets and path sets may be obtained from a reliability diagram as shown in figure 329.
For large complex fault trees, cut sets and path sets are obtainable using the MOCUS algorithm
described in sections 3.6.3.3 and 3.6.3.4, respectively.
356
!
1 2 3
4 & 5 & 6
Nastiness
4 5 6
The OR gate in this position
indicates a series string of
component functions.
1 3
4
5
6
2
!
This AND gate indicates a
paralleled set of component
functions in the series string.
Figure 328. Fault tree to RBD transformation.
Path
Sets
1 / 2 / 3 / 4
1 / 2 / 3 / 5
1 / 2 / 3 / 6
Minimal
Cut Sets
1
2
3
4 / 5 / 6
!
6
5
4
1 2 3
Figure 329. Deriving cut and path sets from an RBD.
357
To transform an RBD into an event tree, proceed as shown in figure 330. To transform a fault
tree into an event tree, first transform the fault tree into an RBD (sec. 3.9.3.1).
1 3
4
5
6
2
!
1
2
3
4
5
6
Success
Failure
All of these parallel
elements must fail to
produce system failure.
Failure of any one of
these series elements
makes system failure
irreversible.
Figure 330. RBD to event tree transformation.
3.9.3.3 RBD to Fault Tree Transformation
A fault tree represents system functions which, if they fail, produce TOP event fault in place of
the success to which the reliability block path lead. The series nodes of an RBD denote an OR gate
beneath the TOP event of a fault tree. The parallel paths in an RBD denote the AND gate for redundant
component functions in a fault tree. Therefore, a reliability diagram can be transformed into a fault tree
as shown in figure 331.
3.9.3.4 Event Tree to RBD and Fault Tree Transformation
An event tree represents path sets in the success branches of the tree and all the cut sets in the
failure branches of the tree. To transform an event tree into an RBD, reverse the process illustrated in
figure 330. Once the RBD is formed, a fault tree can be formed as illustrated in figure 331. Also, an
event tree can be transformed into a fault tree by inspection as shown in figure 332.
3.9.4 Example
An RBD and fault tree are transformed from the example event tree presented in figure 327, and
presented in figure 333(a) and (b), respectively. All three of the models represent equivalent logic of the
system.
358
These series nodes
indicate an OR gate
beneath TOP.
1
2
3
4
5
6
7
!
!
1
2 3 4 5
6 & 7
Evil
6 7
2 & 3
Grief
4 & 5
Woe
These parallel paths indicate
AND gates for redundant
component functions.
Figure 331. RBD to fault tree transformation.
SUCCESS
FAILURE A
1
FAILURE B
1
FAILURE B
2
FAILURE C
FAILURE B
3
FAILURE A
2
FAILURE D
SUCCESS
SUCCESS
SUCCESS
SUCCESS
SUCCESS
SUCCESS
SUCCESS
FAILURE
18
19
20
21
30
1
i
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
22
23
24
25
26
27
28
29
FAILURE
A12
FAILURE
A
1
FAILURE
A
2
16 7* 1* i 26 12 5* 2 3* i
*Note that not all events represented here are failures.
Figure 332. Event tree to fault tree transformation.
359
FLOAT
SWITCH
S
PUMP
P
KLAXON
K
BAILING
B
CUT
SETS
S
P / K
P / B
PATH
SETS
S / P
S / K / B
See Figure 327.
(a) RBD.
EXACT SOLUTION:
P
TOP
= P
S
+ P
P
P
K
– P
P
P
K
P
S
+ P
B
P
P
–
P
B
P
P
P
S
– P
B
P
K
P
P
+ P
B
P
K
P
P
P
S
RARE EVENT APPROXIMATION:
P
TOP
= P
S
+ P
P
P
K
+ P
P
P
B
CUT
SETS
S
P / K
P / B
PATH
SETS
S / P
S / K / B
COMMAND
FAILURE
RESPONSE
FAILURE
FAILURE
TO
DEWATER
S
FLOAT
SWITCH
FAILS
OPEN
WATER
REMOVAL
FAILS
P
PUMP
FAILS
MANUAL
REMOVAL
FAILS
K
KLAXON
FAILS
B
BAILING
FAILS
See Figure 327.
(b) Fault tree.
Figure 333. Equivalent logic RBD and fault tree.
360
3.9.5 Advantages
These techniques allow the analyst to overcome weaknesses of one analysis technique by trans
forming a system model into an equivalent logic model as another analysis technique. For example, a
complex system that may be hard to model as a fault tree might be easily modeled with an RBD. Then,
the RBD can be transformed into a fault tree, and extensive quantitative or pseudoquantitative analysis
can be performed.
3.9.6 Limitations
These techniques possess the following limitations:
(1) No new information concerning the system is obtained and the models are only as good as
the models being transformed.
(2) The cut sets and path sets required to perform these transformations for large complex
systems may require many manhours or extensive computer resources to determine.
3.9.7 Bibliography
Gough, W.S., Riley, J., and Koren, J.M.: “A New Approach to the Analysis of Reliability Block
Diagrams.” Proceedings from Annual Reliability and Maintainability Symposium, SAIC, Los
Altos, New Mexico, 1990.
3.10 CauseConsequence Analysis
3.10.1 Description
A causeconsequence analysis is a symbolic logic technique described in references 3.6 and 3.14,
and presented below. This technique explores system responses to an initiating “challenge” and enables
assessment of the probabilities of unfavorable outcomes at each of a number of mutually exclusive loss
levels. The analyst starts with an initiating event and performs a forward (bottomup) analysis using an
event tree (sec. 3.8). This technique provides data similar to that available with an event tree; however, it
affords two advantages over the event tree—time sequencing of events is better portrayed, and discrete,
staged levels of outcome are analyzed.
The cause portion of this technique is a system challenge that may represent either a desired or
undesired event or condition. The cause may be a fault tree TOP event and is normally, but not always,
quantified as to probability. The consequence portion of this technique yields a display of potential out
comes representing incremental levels of success or failure. Each increment has an associated level of
assumed or calculated probability, based on variations of response available within the system.
A conceptual illustration of how a cause is assessed to understand its consequences is presented
in figure 334. Note that the cause has an associated probability, and each consequence has an associated
severity and probability.
361
P
0
S
1
, P
C1
S
3
, P
C3
S
n
, P
Cn
S
2
, P
C2
CONSEQUENCE 3
CONSEQUENCE 2
CONSEQUENCE 1
CONSEQUENCE n
ANALYSIS
S
n
= Severity of
the n
th
Consequence
P
Cn
= Probability of
the n
th
Consequence
occurring
CAUSE
OVERPRESSURE
RELIEF FAILS
Y N
P
0
, probability of
the Cause, may be
determined by Fault
Tree Analysis.
Figure 334. Relationship between cause and consequence.
3.10.2 Application
This technique is typically applied in phase C or E but may also be applied in phase D. The
causeconsequence analysis is particularly useful in analyzing commandstart/commandstop protective
devices, emergency response systems, and engineered safety features. Causeconsequence analyses are
useful in evaluating operating procedures, management decision options, and other nonhardware
systems. Also, it will evaluate the effect/benefit of subtiered/redundant design countermeasures for
design trades and assessment. This technique may be used in conjunction with an FTA to provide a
technique sensitivity assessment. This technique may also be used to compliment an FMEA.
3.10.3 Procedures
The procedures, as described in references 3.6 and 3.14, for performing a causeconsequence
analysis are presented below.
(1) Identify the initiating event that challenges the system.
(2) Determine the probability, P
0,
that this event will occur. This probability may be
determined from an FTA (sec. 3.6.3.2) or assumed.
(3) Next, trace the possible consequences to the system from the initiating event. At various
levels the path may branch with two possible outcomes. Construct the consequence
diagram by asking the following questions:
3.6
a. What circumstances allow this event to proceed to subsequent events?
b. What other events may occur under different system operating circumstances?
362
c. What other system elements does this event influence?
d. What subsequent event could possibly result as an outcome of this event?
(4) Use the symbols presented in table 38 to construct the consequence diagram.
Table 38. Causeconsequence tree construction symbols.
3.14
AND
Gate
OR
Gate
Coexistence of all inputs opens gate and produces
an output.
Gate opens to produce output when any input
exists.
Branching
Operator
Event
Y N
Output is “Yes” if condition is met and “No” if it
is not met. Branching operator statement may be
written in either the fault or the success domain.
The outputs are mutually exclusive, therefore
P
Y
+P
N
= 1.
Basic
Event
An independent initiating event, representing the
lower resolution limit of the analysis.
Consequence
Descriptor
End event/condition to which analysis leads, with
the severity level stated.
Symbol Name Description
(5) The format of the consequence tree is presented in figure 335. Note that all paths lead into
branching operators or consequence descriptors. The branching operator always has one
input and two output paths (yes and no). The consequence descriptor has one input, no
outputs, and is a termination point in the diagram.
(6) For each branching operator, establish the probability, P
i
, that the event can happen.
Therefore, P
i
and (1–P
i
) are the probabilities for the yes and no paths from the branch
operator, respectively. This step is often difficult and subjective due to a scarcity of data.
Probability bands are often useful to provide an understanding of the analyst's confidence
in the delineated probabilities.
(7) Determine the probability of each consequence descriptor, P
ci
, by multiplying event
probabilities along the path that terminates at that consequence descriptor.
(8) Finally, determine the severity of each consequence descriptor, S
i
.
363
Note that, because the analysis is exhaustive…
(P
0
P
1
) + P
0
(1 – P
1
) (1 – P
2
) + P
0
(1 – P
1
) P
2
= P
0
CONSEQUENCE
DESCRIPTOR 1
CONSEQUENCE
DESCRIPTOR 2
CONSEQUENCE
DESCRIPTOR 3
BRANCHING
OPERATOR
Y N
BRANCHING
OPERATOR
Y N
P
0
P
1
P
0
P
1
INITIATING
CHALLENGE
P
0
(1 – P
1
)
P
2
P
0
(1 – P
1
) P
2
P
0
(1 – P
1
) (1 – P
2
)
Fault trees or other analyses
may be used to establish
probabilities for the Initiating
Challenge and for Branching
Operator Y/N outcomes.
Figure 335. Causeconsequence analysis format.
3.10.4 Example
*
Problem:
A copying machine uses an electrically heated drum to fix dry ink to copy paper. The drum
heater is thermostatically controlled. The drum is also equipped with an automatic overheat safety cutoff
to prevent damage to the copier. The probability of failure is finite for both the drum thermostat and the
overheat cutoff. Combustibles are often present in the copying room near the machine. Uncontrolled
drum temperature can rise high enough to ignite them. The room is equipped with an automatic sprinkler
system initiated by a heat detector. Employees frequent the room and can initiate an emergency response
alarm in the event of fire. After a delay, a fire brigade responds to extinguish the blaze.
The causeconsequence analysis for the above problem is presented in figure 336.
3.10.5 Advantages
Causeconsequence analyses provide the following advantages:
3.14
(1) The analysis is not limited to a “worstcredible case” consequence for a given failure.
Therefore, a less conservative, more realistic assessment is possible.
(2) Enable assessment of multiple, coexisting system faults and failures.
(3) End events need not be anticipated.
(4) The time order of events is examined.
*
This example was provided courtesy of Sverdrup Technology, Inc., Tullahoma, Tennessee.
3.1.
364
FIRE
BRIGADE
RESPONSE
FAILS
EMPLOYEE
DETECTION/
RESPONSE
FAILS
BUILDING DAMAGE
≈ $1.5M
WATER/FIRE/SMOKE
DAMAGE ≈ $50,000
EMERGENCY
RESPONSE
FAILS
Y N
NEARBY
COMBUSTIBLES
IGNITE
Y N
DRUM
OVERHEATS
DRUM
THERMOSTAT
FAILS
CLOSED
OVERHEAT
CUTOFF
FAILS
COMBUSTIBLES
PRESENT
NEARBY
IGNITION
TEMPERATURE
REACHED
COPIER DAMAGE
≈ $250
HEAT DETECTOR/
AUTO SPRINKLER
FAIL
Y N
P
0
P
1
P
0
(1 – P
1
)
P
2
MANUFACTURER'S
TEST DATA P
0
P
1
(1 – P
2
)
P
3
BUILDING LOSS
≈ $6.5M
P
0
P
1
P
2
P
3
P
0
P
1
P
2
(1 – P
3
)
Note that, because the analysis is exhaustive…
P
0
P
1
P
2
P
3
+ P
0
P
1
P
2
(1 – P
3
) + P
0
P
1
(1 – P
2
) + P
0
(1 – P
1
) = P
0
Figure 336. Example causeconsequence analysis.
(5) Probabilities of unfavorable system operating consequences can be determined for a
number of discrete, mutually exclusive levels of loss outcome. Therefore, the scale of
partial successes and failures is discernible.
(6) Potential singlepoint failures or successes, areas of system vulnerability, and lowpayoff
countermeasures are identified and assessed, thereby guiding deployment of resources for
improved control of risk and optimized utilization of limited resources.
3.10.6 Limitations
Causeconsequence analyses possess the following limitations:
3.14
(1) Address only one initiating challenge. Thus, multiple analyses may be needed for a
particular system.
(2) The initiating challenge is not disclosed by the analysis, but must be foreseen by the
analyst.
365
(3) Operating pathways must be foreseen by the analysts.
(4) The establishment of probabilities is often difficult and controversial.
(5) Determining the severity on consequences may be subjective and difficult for the analyst to
defend.
3.10.7 Bibliography
Battelle Columbus Division, “Guidelines for Hazard Evaluation Procedures.” 1985.
Burdic, G.R., and Fussell, J.B.: “On the Adaptation of CauseConsequence Analysis to U.S Nuclear
Power Systems Reliability and Risk Assessment, System Reliability and Risk Assessment.” JBF
Associates, Inc., Knoxville, Tennessee, 1983.
Greenberg, H.R., and Cramer, J.J. “Risk Assessment and Risk Management for the Chemical Process
Industry.” Van Nostrand Reinhold, 1991.
Lees, F.P.: “Loss Prevention in the Process Industries.” 2 vols, Butterworths, London, 1980.
3.11 Directed Graphic (Digraph) Matrix Analysis
3.11.1 Description
Directed graph (digraph) matrix analysis, as described in reference 3.15, is a technique using
matrix representation of symbolic logic models to analyze functional system interactions. Logic models
are first generated in the success domain, then converted into the failure domain. However, it should be
noted that models can be directly created in the failure domain, without first creating the model in the
success domain.
This technique consists of four phases. First, the analyst determines combinations of systems or
combinations of subsystems within a single system for thorough assessment. This phase is parallel to
determining failure propagation paths using an ETA (sec. 3.8). The second phase consists of
constructing a digraph model in the success domain, then converting this model to a digraph model in
the failure domain for each failure propagation path. The third phase consists of separating the digraph
models into independent models, then determining the singleton and doubleton minimal cutsets of each
failure propagation path. Finally, the fourth phase consists of an assessment of the minimal cut sets
relative to probability of occurrence.
3.11.2 Application
This technique, according to reference 3.15, can be used independently or as an element of a
PRA (sec. 3.15). If this technique is used as part of a PRA, then it is performed after the identification of
failure propagation paths by ETA but before FTA’s are begun.
3.15
This technique is applied to evaluate
the failure propagation paths involving several systems and their support systems, or within a single
system involving several system elements (subsystem, component, part, etc.) and is best applied in phase
B.
366
3.11.3 Procedures
Presented below is a summary of the detailed procedures found in reference 3.15 for performing
a digraph matrix analysis.
(1) Identify the associated group of systems (or associated system elements of a single system)
to be thoroughly evaluated. Use event trees (sec. 3.8) to identify failure propagation paths.
For a complete analysis, identify every credible initiator to an undesirable event and
prepare an event tree that illustrates each specific failure propagation path.
a. Acquire pertinent information concerning the collection of systems to be assessed, such
as design specifications and packages, safety assessment reports (such as PHA’s, sec.
3.2), and prior safety or reliability studies.
b. Study checklists of potential initiating challenges. From these checklists develop a list
of initiators that are applicable to the systems being studied.
c. Develop event trees for each initiating challenge to the system.
d. Prepare a list of failure propagation paths from step 1c. Assume unity probability for all
systems required to work in the failure propagation path. This simplifying assumption
leaves only failure propagation paths that are combinations of systems that must fail for
a serious threat to be posed.
(2) Construct a digraph model for each possible failure propagation path. Use a backward, top
down approach to construct a toplevel digraph, then expand each element into its own
digraph. Continue expanding the elements of new digraphs until the desired resolution level
of the analysis is reached. An outline of the steps involved in producing the digraphs is
presented below.
a. Create a success domain digraph model for each success path. Connect upstream
elements to a downstream element with an AND gate if the upstream element relies on
the successful operation of all the downstream components. Connect upstream elements
to a downstream element with an OR gate if the upstream element relies on the
successful operation of only one of two or more downstream elements. The symbols for
AND and OR gates for a digraph are different than those used for a fault tree, however
they represent the same logic as the fault tree symbols. A comparison between the
digraph and fault tree symbols is presented in figure 337.
b. Form a failure domain model by taking the model generated in step 2a and interchange
all AND gates with OR gates and all OR gates with AND gates. This failure domain
model represents a path for failure propagation.
c. Form an adjacency matrix that represents the digraph. The matrix is constructed by the
process illustrated in figure 338.
367
AND Gate OR Gate
Digraph
A
B
C
A
B
C
Fault tree
A B
C
A B
C
Represented logic Event C will occur only if
both event A and event B
occur.
Event C will occur only if
event A or event B occurs.
Figure 337. Comparison between digraph and fault tree logic gates.
d. Next link all connected elements in the adjacency matrix. This is accomplished by
processing the adjacency matrix with the reachability code. This code is described in
detail in reference 3.15. The output of this code will show all elements connected by a
path and illustrate which elements can be reached from a specific element, therefore all
possible paths between pairs of nodes in the network. Next, use this information to
determine singleton and doubleton cut sets.
e. Determine minimal singleton and doubleton cut sets from the cut sets determined in
step 2d.
(3) Subdivide the digraph into independent digraphs if the success domain digraph model
becomes too large to determine singleton and doubleton cut sets for the computer platform
being used. Then determine singleton and doubleton minimal cut sets of the smaller
independent digraphs.
(4) Assess the singleton and doubleton minimal cut sets. This assessment can be conducted in a
manner similar to that for a conventional PRA (sec. 3.15) in which risk is assessed with the
probability of the cut sets occurring and the severity of the consequence of the failure
propagation path.
368
Type Digraph Adjacency Matrix
Direct
connection
Element A Element B
A B
A 0 1
B0 0
AND gate
Element A
Element C
Element B
A B C
A 0 0 B
B0 0 A
C0 0 0
OR gate
Element C
Element A
Element B
A B C
A 0 0 1
B0 0 1
C0 0 0
Figure 338. Construction of digraph adjacency matrix.
3.11.4 Example
An example digraph matrix analysis, adapted from reference 3.15, for a simple system is
illustrated in figure 339. The system consists of two redundant power supplies to power a motor that
drives a pump. The success domain model of this system is presented in figure 339(a). Note that this
model represents the success path for successful operation of the pump. The failure domain model,
presented in figure 339(b), was generated by replacing the OR gate in the success domain model with
an AND gate. Inspection of the two models suggests that for simple systems the failure domain model
can easily be generated without first generating the success model. In cases with more complex systems,
first generating a success domain model may prove to be beneficial.
The adjacency matrix and adjacency elements are presented in figures 339(c) and (d),
respectively. The adjacency matrix illustrates whether there is a direct path from node i to node j. If
matrix element (i,j) = 1, there is a path from node i to node j. For example, element (M,P) = 1, which
means there is a straight (uninterrupted) and unconditional path between the motor and pump. If element
(i,j) = 0, there is no path from node i to j. For example, element (PS–1, PS–2) = 0, which means there is
a no straight path between the main power supply and the auxiliary power supply. If the adjacency
element (i,j) is ≠ 0 or 1, then there is a second component that must fail along with component i to cause
component j to fail. For example, adjacency element (PS–1, M) is equal to PS–2 (nonzero or 1 value).
369
This symbol represent the second component that must fail, given the failure of PS–1, to cause M to fail
to operate (i.e., failure of both the main and auxiliary power supplies will cause the motor not to
operate).
The reachability matrix and reachability elements are presented in figure 339(e) and (f), respec
tively. The methodology to generate the reachability matrix from the adjacency matrix is presented in
reference 3.15. Simply stated, the reachability matrix illustrates the pairs of nodes that a path exists
between, by connecting linked pairs from the adjacency matrix. Therefore the reachability matrix
illustrates the complete pathways (through linked node pairs) of the graphical model elements illustrated
by the adjacency matrix. Processing the adjacency matrix into the reachability matrix yields the paths
between all pairs of nodes. The reachability elements are derived from the reachability matrix in the
same manner that adjacency elements are derived from the adjacency matrix. Note, in this example, that
the reachability elements include all the adjacent elements and the new information that if both PS–1 and
PS–2 fail, then P will not operate (even though neither PS–1 or PS–2 are directly adjacent to P).
Therefore, the reachability matrix yielded the new information that if both power supplies failed, the
pump will not operate.
The summary matrix presented in figure 339(g) illustrates which components can lead to failure
of the pump, P. If an “*” is entered as a matrix element (i,j) and either i or j is a value of 1, then the other
corresponding component i or j is a singleton. The only singleton in this system is the motor, i.e., the
single failure of the motor will cause the pump not to operate. If a “*” is entered as a matrix element (i,j)
that corresponds to component i and component j, then component i and component j form a doubleton.
The only doubleton of this system is the pair of redundant power supplies, i.e., failure of both the main
and auxiliary power supplies will cause the pump not to operate.
Obviously, in this example the singletons (single point failures) and doubletons (double point
failures) could have easily been identified without performing a digraph matrix analysis. However, for
complex systems which are modeled with many nodes and logic gates, this technique allows
determination of singletons and doubletons which otherwise would not be as readily identified.
3.11.5 Advantages
The digraph matrix analysis provides the following advantages:
3.15
(1) The analysis allows the analyst to examine each failure propagation path through several
systems and their support systems in one single model. Unlike the FTA with failure propa
gation paths divided in accordance to arbitrarily defined systems, this approach allows
more rigorous subdividing of the independent subgraphs.
(2) Since the technique identifies singleton and doubleton minimal cut sets without first deter
mining all minimal cut sets, considerable computer resources can be saved over other
methods such as the FTA.
370
Motor, M
Main Power
Supply,
PS1
Auxiliary
Power
Supply,
PS2
Pump, P
Main Power
Supply,
PS1
Auxiliary
Power
Supply,
PS2
Motor, M
Pump, P
(a) Success domain model. (b) Failure domain model.
PS1 PS2 M P
PS1 0 0 PS2 0
PS2 0 0 PS1 0
M 0 0 0 1
P 0 0 0 0
PS1, M, PS2
PS2, M, PS1
M, P, 1
(c) Adjacency matrix. (d) Adjacency elements.
PS1 PS2 M P
PS1 0 0 PS2 PS2
PS2 0 0 PS1 PS1
M 0 0 0 1
P 0 0 0 0
PS1, M, PS2 (Adjacent)
PS1, P, PS2
PS2, M, PS1 (Adjacent)
PS2, P, PS1
M, P, 1 (Adjacent)
(e) Reachability matrix. (f) Reachability element.
Figure 339. Example digraph matrix analysis—Continued
371
1 PS1 PS2 M P
PS1  *   
PS2   *  
M *    
1    * 
P     
Singletons: M
Doubletons: PS1, PS2
(g) Summary matrix.
Figure 339. Example digraph matrix analysis—Continued.
3.11.6 Limitations
Digraph matrix analyses possess the following limitations.
3.15
(1) Trained analysts and computer codes to perform this technique may be limited.
(2) For particular types of logic models, complete treatment may require more computer
resources than FTA’s.
3.11.7 Bibliography
Grumman Space Station Division, “Digraph Analysis Assessment Report.” Reston Virginia, October
1991.
Kandel, A., and Avni, E.: “Engineering Risk and Hazard Assessment.” vol. 2, CRC Press Inc., Boca
Raton, Florida.
3.12 Combinatorial Failure Probability Analysis Using Subjective Information
3.12.1 Description
The combinatorial failure probability analysis using subjective information is described in refer
ence 3.16 and presented below. This technique was developed by the System Effectiveness and Safety
Technical Committee (SESTC) of the American Institute of Aeronautics and Astronomics (AIAA), in
1982. This technique provides the analyst a procedure to propagate probability data derived from the
subjective probability scales defined in MIL–STD–882C.
3.2
372
3.12.2 Application
This technique is typically performed in phase C and is applicable when no quantitative failure
probability data are available and may be used in conjunction with other analyses such as an RBD (sec.
3.5), FTA (sec. 3.6), STA (sec. 3.7), ETA (sec. 3.8), and causeconsequence analysis (sec. 3.10).
3.12.3 Procedures
The procedures, as described in reference 3.16, for a combinatorial failure probability analysis
using subjective information are presented below.
(1) Arbitrary, dimensionless “probability values” have been assigned to the probability incre
ments (frequent, probable, occasional, remote, and improbable) defined in MIL–STD–
882C.
3.2
The subjective scale for these arbitrary values is presented in table 39. Descriptive
words and definitions for the level of the scale are also given in this table.
Table 39. Combinatorial failure probability analysis subjective scale.
AIAA/SESTC MIL–STD–882C
Threshold
Levels
Probability
Level
*
Level
Descriptive Word
Definition
8×10
–2
to
1.00000
3×10
–1
A Frequent Likely to occur frequently.
8×10
–3
to
8×10
–2
3×10
–2
B Probable Will occur several times in life of an item.
8×10
–4
to
8×10
–3
3×10
–3
C Occasional Likely to occur sometime in life on an item.
8×10
–5
to
8×10
–4
3×10
–4
D Remote Unlikely but possible to occur in life of an
item.
0.00000 to
8×10
–5
3×10
–4
E Improbable So unlikely if can be assumed occurrence may
not be experienced.
*
Arbitrarily selected, dimensionless numbers.
Table provided courtesy of Sverdrup Technology, Inc., Tullahoma, Tennessee.
(2) Estimate subjective failure probabilities of contributor events or conditions using the scale
defined in MIL–STD–882C.
3.2
Select and consistently apply the same probability exposure
interval (operating duration or number of events) for every initiator probability estimate
used in the analysis.
(3) Correlate the subjective estimate (step 2) with the arbitrary, dimensionless values (step 1).
Propagate these values in the same manner as quantitative data is combined in classical
numerical methods (such as presented in figs. 318 and 319).
(4) Convert the final probability number resulting from propagation (step 3) back into the sub
jective scale defined in MIL–STD–882C.
3.2
373
3.12.4 Example
The following example
*
uses this subjective combinatorial technique in a fault tree problem.
Problem/Background:
• A large rotating machine has six mainshaft bearings. Replacement of a bearing costs $18,000
and requires 3 wk of down time.
• Each bearing is served by:
• pressurized lubrication oil
• a watercooled jacket
• a temperature sensing/alarm/shutdown system.
• In addition, there are sensing/alarm/shutdown systems for:
• lube pressure failure
• cooling water loss of flow.
• If they function properly, these systems will stop operation of the rotating machine early
enough to prevent bearing damage. (System sensitivity makes the necessary allowance for
machine “rollout” or “coasting.”)
• Failure records for the individual system components are not available, but probabilities can
be estimated using the subjective scale of MIL–STD–882C.
3.2
What is the probability that any one of the six bearings will suffer burnout during the coming
decade?
The system schematic and fault tree are presented in figure 340(a) and (b), respectively. Note
both the arbitrary subjective probability value and letter representing the relevant probability level from
table 39 are presented for each fault tree initiator.
3.12.5 Advantages
This technique allows the analyst to perform a probabilistic assessment based on the exercise of
subjective engineering judgment when no quantitative probability estimates are available.
3.12.6 Limitations
This technique should only be used when actual quantitative failure rate data is not available. The
use of actual quantitative data is preferred over this method. This tool should only be used for
comparative analysis only. Data and results, unless used in a comparative fashion, may be poorly
received.
*
This example was provided courtesy of Sverdrup Technology, Inc., Tullahoma, Tennessee.
3.16
374
Bearing Burnout Loss Penalty:
• $18,000 Replacement Costs
• 3Week Interruption of Use
UTILITY SUBSYSTEMS
PROTECTIVE FEATURES
• Sensing
• Alarm
• Shutdown
dT
/
dt
T
H
2
O
Flow
Lube
Pressure
(a) System schematic.
6x10
8
3x10
3 3x10 4
3x10
3
3x10
4
3x10
2
9x10
7
9x10
7
2x10
6
C D
B
E
D C
BEARING
BURNOUT
UNRESOLVED
UTILITY SERVICE
FAILURE
SHUTDOWN
FAILURE
UNRESOLVED
LUBE
FAILURE
UNRESOLVED
COOLANT
FAILURE
LUBE
PRESSURE
FAILURE
COOLING
WATER
FAILURE
SHUTDOWN
FAILURE
SHUTDOWN
FAILURE
Bearing burnout is “ Improbable” f or any one
of t he six bearings over t he 10year period.
10year f ailure probabilit y est imat es
are ent ered at t he lowest levels of t he
t ree and propagat ed upward.
Bearing Temperat ure
Sensing/ Alarm/ Shut down
Failure
Coolant Loss
Sensing/ Alarm/ Shut down
Failure
Lube Pressure
Sensing/ Alarm/ Shut down
Failure
(b) System fault tree.
Figure 340. Example combinatorial failure probability analysis.
375
3.13 Failure Mode Information Propagation Modeling
3.13.1 Description
Failure mode information propagation modeling is a qualitative analysis method described in
reference 3.17 and presented below. This technique involves separating a system into its basic functional
components and examines the benefit of measuring precedent failure information that may be
transmitted between components of a system. This information may be transmitted during the initial
outset of a variety of failure modes. The technique provides insight into both the types of information
that should be measured to safeguard the system, and location within the system at which sensors might
be appropriately positioned.
3.13.2 Application
This technique effectively directs resource deployment to optimally safeguard a system against
potential failures by identifying measurement requirements. These requirements are defined in terms of
measured parameter, sensor type, and sensor location. This technique is best applied in phase C but may
also be applied in phase D.
3.13.3 Procedures
The procedures, as described in reference 3.17, to perform failure mode information propagation
modeling are presented below.
(1) Divide the system into its principle functional components and assign a number to each
component. Like the FMEA (sec. 3.4), the resolution of this analysis is dependent upon the
level (i.e., subsystems, assemblies, subassemblies, or piece parts) to which the system
elements are resolved.
(2) Identify the physical links (energy flow and shared stress) between the components of the
system. These links include such items as electrical power, air flow, liquid flow, gas flow,
thermal heat transfer, friction, spring, rolling element, etc.
(3) Identify and record the failure modes for each component and assign a letter to each failure
mode for each component.
(4) Identify and record the flow of failure mode information at each physical link that is
available externally to each component and transmitted to one or more other components.
(5) Classify the failure mode information constituents by their signal characteristics (e.g.,
thermal, pressure, acceleration, etc.).
(6) Identify the minimal success sets of the sensor network. A minimal success set is a sensor
group that encompasses all failure modes.
(7) Assess the various minimal success sets in terms of feasibility, cost, and effectiveness. The
following questions should be asked:
a. Feasibility. Do the sensors currently exist or can they be developed? Can they be
obtained in time to satisfy schedule requirements?
376
b. Cost. Is the cost of installing, maintaining, and operating the sensor network less than
the cost of the failure that the system is being safeguarded against?
c. Effectiveness. Are there other preventive maintenance activities more effective than
installing a sensor network? Will the sensing network forewarn before the start of
system failures or does it just announce system crashes? Will the sensors impede
normal system operation? Will they degrade system performance? Will they pose any
new hazards to the system? Will the sensor network operate dependably? Will the
sensors have adequate sensor redundancy?
3.13.4 Example
The following example
*
uses failure mode information propagation modeling to a sensor
network success set for a system.
Problem:
Consider a ventilating fan powered by an electric motor through a belt drive. A common frame
structure supports both the motor and a bearing, through which power is delivered to the fan. (Consider
motor bearings as integral parts of the motor.) Assume a constant aerodynamic fan load. A schematic of
the system is presented in figure 341(a). Determine sensor network minimal success sets for the system.
Solution:
(1) Perform steps 1–5 identified in section 3.13.3. These steps are explained below and illus
trated in figure 341(b).
a. Step 1. Divide the system into its principle functional components and assign a number
to each component. These are the electrical motor, fan belt, fan, frame, and bearing.
b. Step 2. Identify the physical links (energy flow and shared stress) between the compo
nents of the system. The electric motor, for example, has electrical power input, is
linked to the fan belt by friction, and is mechanically and thermally linked to the frame.
c. Step 3. Identify and record the failure modes for each component and assign a letter to
each failure mode. For example, the failure modes for the fan include shaft or rotor
binding, bearing vibration, open winding, and shorted winding.
d. Step 4. Catalog the flow of failure mode information at each physical link that is
available externally to each component and transmitted to one or more other
components. For example, for the mechanical link between the electric motor and
frame, the failure information available includes electric motor bearing vibrations (1–
B), fan belt slipping and breaking (2–A/B), and bearing binding (5–A).
e. Step 5. Classify the failure mode information constituents by their signal
characteristics. For example, the electric motor bearing vibration (1–B) and fan bearing
vibration (5–B) can be monitored by an accelerometer at test point 4/1 (between frame,
component 1 and electric motor, component 4).
*
This example was provided courtesy of Sverdrup Technology, Inc., Tullahoma, Tennessee.
3.16
377
(2) From the information displayed in figure 341(b), construct a matrix of failure mode versus
sensor type (with each test point identified). Determine the minimum success sets of mea
surement sensors. These sets are sensor groups that encompass all failure modes. The
matrix and minimum success sets for this system are presented in figure 341(c).
3.13.5 Advantages
Information propagation modeling provides the following advantages:
3.17
(1) Allows the analyst to identify measurement requirements, that, if implemented, can help
safeguard a system by providing warnings at the onset of a failure mode that threatens the
system.
(2) Compliments an FMEA.
3.13.6 Limitations
Information propagation modeling possesses the following limitations:
3.17
(1) This technique is only applicable if the system is operating in a nearnormal range, and for
the instant of time immediately prior to the initiation of a failure mode.
(2) Externally induced and common cause faults are not identified or addressed.
(3) The risks of the failure modes are not quantified in terms of criticality and severity.
(4) The propagation of a failure through the system is not addressed.
3.14 Probabilistic Design Analysis
3.14.1 Description
A PDA, as described in references 3.8 and 3.18, is a methodology to assess relative component
reliability for given failure modes. The component is characterized by a pair of transfer functions that
represent the load (stress, or burden) that the component is placed under by a given failure mode, and
capability (strength) the component has to withstand failure in that mode. The variables of these transfer
functions are represented by probability density functions. Given that the probability distributions for
both the load and capability functions are independent, the interference area of these two probability
distributions is indicative of failure. Under these conditions, a point estimate for failure of the
component relative to the failure mode under consideration can be determined.
3.14.2 Application
A PDA can be used to analyze the reliability of a component during phase C of a program. The
PDA approach offers an alternative to the more traditional approach of using safety factors and margins
to ensure component reliability. This traditional approach is vulnerable if significant experience and
historical data are not available for components similar to that which is being considered.
3.8 3.18
378
Elements:
• Electric Motor
• Fan Belt
• Bearing
• Fan
• Frame
(a) System schematic.
Electric
Motor Fan Belt Fan
Frame Bearing
1 2 3
4 5
A. Binding
B. Vibration
A. Blade Damage
A. Slipping
B. Breaking
A. Shaft/Rotor Binding
B. Bearing Vibration
C. Open Winding
D. Shorted Winding
1A/C/D
2A/B
3A
5A
1A/C/D
2A/B
3A
5A
1A/C/D
2A/B
3A
5A
1A/C/D
2A/B
3A
5A
1B
2A/B
5A
1A/C/D
3A
5A/B
3A
5A/B
0/1 1/2
4/1
4/5
3/5
2/3
3/00
Accelerometer
3A
5B
Flow Monitor
1A/C/D
2A/B
3A
5A Accelerometer
1B
5B
Tachometer
1A/C/D
2A/B
3A
5A
Belt Slip Monitor
2A/B
Tachometer
1A/C/D
2A/B
3A
5A
Belt Slip
Monitor
2A/B
Power Monitor
1A/C/D
2A/B
3A
5A
Heat Flux Monitor
1A/C/D
Accelerometer
3A
5B
• • • • • • • • • •
Electrical Power
Friction
Air Flow
Rolling Element
Mechanical
Thermal
Spring
Gas Flow
Liquid Flow
(b)
Model.
Figure 341. Example failure mode information propagation model—Continued
379
Minimal Success Sets*
Power Monitor at 0/1
or
Tachometer at 1/2
or
Tachometer at 2/3
or
Flow Monitor at 3/00
and
Accelerometer at 4/1
*Sensor groups that envelope all
failure modes
1A
1B
1C
1D
2A
2B
3A
4
5A
5B
Failure
Mode
3/00 3/5 4/5 4/1 2/3 1/2 0/1
P
o
w
e
r
M
o
n
i
t
o
r
T
a
c
h
o
m
e
t
e
r
B
e
l
t
S
l
i
p
M
o
n
i
t
o
r
T
a
c
h
o
m
e
t
e
r
B
e
l
t
S
l
i
p
M
o
n
i
t
o
r
F
l
o
w
M
o
n
i
t
o
r
A
c
c
e
l
e
r
o
m
e
t
e
r
A
c
c
e
l
e
r
o
m
e
t
e
r
A
c
c
e
l
e
r
o
m
e
t
e
r
H
e
a
t
F
l
u
x
M
o
n
i
t
o
r
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
S
e
n
s
o
r
T
e
s
t
P
o
i
n
t
Not Applicable
(c) Minimal success sets.
Figure 341. Example failure mode information propagation model—Continued.
3.14.3 Procedures
The procedures, adapted from reference 3.8 and 3.18, for performing a PDA in the context of a
total design reliability program for a system are presented below.
(1) Specify the system design requirements. These requirements should be stated in clear and
concise terms that are measurable and verifiable.
(2) Identify variables and parameters that are related to the design.
(3) Identify the failure modes of the system by using a method such as a FMEA (sec. 3.4).
(4) Confirm the selection of critical design parameters.
(5) Establish relationships between the critical parameters and organizational, programmatic,
and established failure criteria.
(6) Ascertain the reliability associated with each critical failure mode with the following proba
bilistic analysis method:
a. Identify the random variables that effect the variation in the load to be imposed on the
component for the given failure mode. Incorporate these random variables into a
transfer function that represents this load (stress, or burden).
Load Transfer Function: L = f
L
(X
1
, X
2
, X
3
, ....X
n
).
380
b. Identify the random variables that affect the variation in the capability of the component
to withstand the load imposed for the given failure mode. Incorporate these random
variables into a transfer function that represent this capability (strength).
Capability Transfer Function: C = g
C
(Y
1
, Y
2
, Y
3
, ....Y
m
).
c. Gather data to perform the load and capability calculations.
d. Determine probability distributions of the load (stress, or burden) and capability
(strength) of the failure mode. Consider each variable of the transfer function as a prob
ability density function (illustrated in figure 342). The density function can be repre
sented as either a discrete variable distribution using empirical test data, or as a continu
ously variable form of the density function.
Note: The area under an entire probability density function curve is equal to a
probability of one, therefore a range between two values of the independent random
variable of a density function curve is equal to a probability less than or equal to one.
Probability density functions of both load and capability continuous random variables
for a given failure mode are presented in figure 343. Also illustrated in this figure is
the interference of the load and capability density functions. For independent load and
capability functions, this interference is indicative the failure mode will occur. In figure
343, both density functions are normal distributions with different means and
variances. However, generally one or both of these density functions may be an
exponential, log normal, gamma, Weibull, or other distribution.
e. Calculate the reliability (R) for the failure mode from the load and capability
distributions. Reliability is the probability that the failure mode will not occur. The
expression for reliability is:
R = 1 – P
F
.
The expression for P
F
, is dependent upon the type of load and capability distributions.
Expressions for P
F
for various distributions are found in most advanced statistics text
books and handbooks. Expressions for P
F
between combinations of exponential, log
normal, gamma, and Weibull distributions are found in reference 3.8.
(7) Assess the reliability for each critical failure mode, including load and capability in this
assessment, then modify the design to increase reliability. Repeat the process until the
design reliability goals or requirements are met.
(8) Perform trade studies (sec. 2.1) to reassess and optimize the design for performance, cost,
environmental issues, maintainability, etc.
(9) Repeat step 8 for each critical component for the system.
(10) Determine the relative reliability of the system.
(11) Repeat the above steps to optimize system reliability.
381
f1(X1)
X1
f2(X2)
X2
fn(Xn)
Xn
g1(Y1)
Y1
g2(Y2)
Y2
gm(Ym)
Ym
fL(L)
L
Load Transfer Function
L = fL(X1, X2, ....Xn)
gC(C)
C
Capability Transfer Function
C = g C(Y1, Y2, ....Ym)
Figure 342. Load and capability transfer functions.
3.8
382
Probability
Density
Function
Load
f (L)
L
Capability
fc (C)
f (L)
L
f (C), C
L C L , C
Overlap Indicative of Failure
Figure 343. Interference between load and capability density functions.
3.8
3.14.4 Advantages
A PDA provides the following advantages:
(1) Allows the analyst a practical method of quantitatively and statistically analyzing the
relative reliability of a system during the design phase.
3.8
Therefore PDA’s can be used to
determine valuable areas of the design and aid in determining the resource allocation during
the test and evaluation phase.
(2) This technique mandates that the analyst address and quantify the uncertainty of design
variables and understand its impact on system reliability of the design.
3.8
(3) The PDA approach offers a more accurate and truly quantitative alternative method to the
more traditional approach of using safety factors and margins to ensure component
reliability.
3.8 3.18
(4) The technique provides a more precise method for determining failure probabilities to
support FTA’s than does use of subjective methods.
3.14.5 Limitations
A PDA possesses the following limitations:
(1) The analyst must have experience in probability and statistical methods to apply this
technique.
3.8
(2) Determining the density functions of the random variables in the load and capability
transfer functions may be difficult.
3.18
(3) Historical population data used must be very close to the asplanned design population to
be viable. Extrapolation between populations can render the technique nonviable.
383
(4) This technique identifies the relative probabilities that various failure modes will occur, but
does not address the severity of the failure modes. Therefore, this technique should be used
as one element among other elements of a PRA (sec. 3.15) to assess the risk associated with
the various failure modes.
3.15 Probabilistic Risk Assessment
3.15.1 Description
A PRA is a general term given to methodologies that assess risk. Although PRA methods are
customarily thought of as being quantitative, these methods can be either subjective (as by use of the risk
assessment matrix, sec. 3.1), or quantitative in nature.
According to reference 3.6, a PRA generally consists of three phases. During phase 1, the system
is defined, hazards are identified, elements of the system vulnerable to hazards are identified, and the
overall scope of types of hazards to be assessed is defined. PHA’s (sec. 3.2), are typically performed
during phase 1.
During phase 2, the failure propagation paths and probabilities are established. ETA (sec. 3.8),
FTA (sec. 3.6), FMECA (sec. 3.4) and/or causeconsequence analysis (sec. 3.10) are performed.
Finally, during phase 3, a consequence analysis is performed. Severity is established. Then, an
assessment of risk is performed in terms of probability and severity, and by comparison to other societal
risks.
3.15.2 Application
A PRA is performed to identify consequence of failure in terms of potential injury to people,
damage to equipment or facilities, or loss of mission requirements. The PRA is typically performed in
phase C.
3.15.3 Procedures
The following procedures, adopted from reference 3.6, offer guidance in performing a
probabilistic risk assessment:.
(1) Phase 1 (activities performed during the preliminary design stage).
a. Define the system to be assessed, identify the elements (targets) of the systems that are
susceptible to hazards, and from an overall perspective identify potential hazards.
b. Perform a PHA (sec. 3.2). In performing a PHA, the analyst: (1) identifies targets, (2)
defines the scope of the system, (3) recognizes the acceptable risk limits, (4) identifies
hazards, (5) assesses the risk for each hazard and target combination in terms of proba
bility and severity, (6) and if the risk are unacceptable determines countermeasures to
mitigate the risk, and (7) and repeats the assessment with the countermeasures
incorporated.
384
(2) Phase 2 (activities initiated after accomplishing hardware and configuration selections).
a. Identify failure propagation paths with techniques such as an ETA (sec. 3.8). In
performing an ETA, the analyst (1) identifies an initiating challenge to the system, and
(2) determines the alternate logic paths from the initiating event.
b. Determine initiators and propagate probability of failure with methods such as FTA
(sec. 3.6). Probability of failure modes can also be determined with the probabilistic
analysis method presented in section 3.14.
c. A causeconsequence analysis (sec. 3.10) may be performed to establish both failure
propagation path and probabilities of causes and consequences.
d. A digraphmatrix analysis (sec. 3.11) may be performed after the ETA is complete and
before FTA’s have begun.
3.15
e. An FMECA (sec. 3.4) may be performed. Examine all failure modes and criticality
ranking of each system element.
(3) Phase 3 (perform a consequence analysis).
a. Establish the severity of the failure modes.
b. Assess risk of all failure modes in terms of severity and probability.
c. Calibrate the risk of the system being examined by comparing it to other known societal
risks.
3.15.4 Advantages
Assessing risk avoids unknowingly accepting intolerable and senseless risk, allows operating
decisions to be made, and improves resource distribution for control of loss resources.
3.1
3.15.5 Limitations
A PRA possesses the following limitations:
(1) Probabilistic risk assessment requires skilled analysts. If the analyst is untrained in the
various tools required, the tool could be misapplied or the results misinterpreted.
(2) Depending on the size and complexity of the system being assessed, significant manhour
and/or computer resources may be needed to complete.
(3) Sufficient information and data may not be available to perform a thorough assessment.
385
REFERENCES
3.1 Clemens, P.L.: “Working with the Risk Assessment Matrix.” Second edition, Lecture
Presentation, Sverdrup Technology, Inc., June 1993.
3.2 “System Safety Program Requirements.” MIL–STD–882C, January 1993.
3.3 Mohr, R.R.: “Preliminary Hazard Analysis.” Fourth edition, Lecture presentation, Sverdrup
Technology, Inc., July 1993.
3.4 Clemens, P.L.: “Energy Flow/Barrier Analysis.” Third edition, Lecture presentation, Sverdrup
Technology, Inc., June 1993.
3.5 Mohr, R.R.: “Failure Modes and Effects Analysis.” Sixth edition, Lecture presentation, Sverdrup
Technology, Inc., October 1992.
3.6 Henley, E.J., and Kumamoto, H.: “Probabilistic Risk Assessment.” The Institute of Electrical and
Electronic Engineers, Inc., NY, 1991.
3.7 Gough, W.S., Riley, J., and Koren, James M.: “A New Approach to the Analysis of Reliability
Block Diagrams.” Proceedings from Annual Reliability and Maintainability Symposium, SAIC,
Los Altos, NM, 1990.
3.8 Kampur, K.C., and Lamberson: “Reliability in Engineering Design.” John Wiley & Sons, NY,
1977.
3.9 Clemens, P.L.: “Fault Tree Analysis.” Fourth edition, Lecture presentation, Sverdrup Technology,
Inc., May 1993.
3.10 Swain, A.D., and Guttman, H.E.: “Handbook of Human Reliability Analysis with Emphasis on
Nuclear Power Plant Applications.” NUREG/CR–1278.
3.11 Briscoe, Glen J.: “Risk Management Guide.” System Safety Development Center, SSDC–11,
DOE 7645/11, September 1982.
3.12 Clemens, P.L.: “Event Tree Analysis.” Second edition, Lecture presentation, Sverdrup
Technology, Inc., June 1990.
3.13 Clemens, P.L.: “Transformations, Fault Tree/Reliability Block Diagram/Event Tree.” Lecture
presentation, Sverdrup Technology, Inc., November 1992.
3.14 Clemens, P.L.: “CauseConsequence Analysis.” Third edition, Lecture presentation, Sverdrup
Technology, Inc., December 1992.
3.15 Alesso, H.P., Sacks, I.J., and Smith, C.F.: “Initial Guidance on DigraphMatrix Analysis for
System Interaction Studies.” Lawrence Livermore National Laboratory, March 1983.
3.16 Clemens, P.L.: “Combinatorial Failure Probability Analysis Using MIL–STD–882B,” Fourth
edition, Lecture presentation, Sverdrup Technology, Inc., August 1991.
3.17 Clemens, P.L.: “Failure Information Propagation Modeling.” Second edition, Lecture
presentation, Sverdrup Technology, Inc., October 1989.
3.18 “Solid Propulsion Reliability Guidebook.” The Phillips Laboratory and the Engineering Society
for Advancing Mobility Land, Sea, Air, and Space (SAE), vol. 2, draft, June 1992.
386
4. DESIGNRELATED ANALYTICAL TOOLS
Two designrelated analytical tools (sensitivity analysis and tolerance stackup analysis) that can
be useful to systems engineering are discussed in this section. In addition, Geometric Dimensioning and
Tolerancing, ANSI–Y–14.5, is discussed. This section is included to give the systems engineer an under
standing of the standard methods of dimensioning and tolerancing.
A summary of the advantages and limitations of each tool or methodology discussed in this
section is presented in table 41.
4.1 Sensitivity (Parametric) Analysis
4.1.1 Description
In sensitivity analysis, sensitivity functions (or coefficients of influence) are generated by taking
partial derivatives with respect to each parameter that affects the outcome of a relationship.
4.1.2 Application
Sensitivity analysis typically should be performed in phase C or D. This analysis can be used for
nearly any type of relationship. Sensitivity analysis is especially useful when environmental conditions
can change, when factors such as age affect performance, or when manufacturing tolerances affect
performance. Sensitivity analysis can show which parameters affect a system the most or least. This can
facilitate optimizing a system, reducing variability, or adjusting a system for wear or changing
conditions. Typical examples of the use of sensitivity analysis are manufacturing formulation and
processes (e.g., bond strength, burn rate, erosion rate, or material strength).
4.1.3 Procedures
The procedure for obtaining the sensitivity of a relationship by analytical methods is as follows:
(1) Generate an equation for the relationship under consideration.
(2) Find the coefficients of influence
4.1
by taking the partial derivatives for each parameter
under consideration.
(3) Solve the equations for the coefficients of influence to find the sensitivity at given
conditions.
An alternate approach to approximate sensitivity is to assume a straightline relationship between
two points in the sample space of the relationship, and to solve the relationship for two conditions repre
sented by two values of the parameters in question. This method is often preferred for relationships with
parameters that are interrelated, such as throat area and exit pressure in the thrust equation.
41
42
Table 41. Designrelated analytical tools and methodologies.
Tool or Methodology Section Advantages Limitations
Sensitivity (parametric) analysis 4.1 The effect of each parameter can be assessed to determine
which parameters have the greatest effect on the
outcome of a process and which parameters can yield the
most benefit from adjustment.
It is often not easy to isolate a variable to obtain a
second derivative. For example, when obtaining the
sensitivity of thrust to throat diameter, changing a
throat diameter not only changes motor pressure, but
changes the nozzle expansion ratio and exit pressure.
The pressure ratio is typically found by iteration or by
tables. If the approximation approach above is taken,
care must be used to ensure a small enough range for
parameter values to achieve the desired accuracy.
Standard dimensioning and
tolerancing
4.2 Dimensioning and tolerancing per ANSI–Y–14.5 is
fairly standard. In addition, some aspects of dimen
sioning and tolerancing per ANSI–Y–14.5 are better
suited for production. For example, true positioning
allows for a circular tolerance zone, whereas putting
tolerances to rectangular coordinates allows a square
tolerance zone. Thus, a functional part that would
comply with true position tolerances may not comply
with rectangular tolerances. Dimensioning strategy can
minimize the cumulative tolerance stackup. This is
facilitated by following the dimensioning and
tolerancing system of ANSI–Y–14.5.
A moderate amount of training and practice is required to
effectively use standard dimensioning and tolerancing.
Tolerance stackup analysis 4.3 Worstcase tolerance analysis can simply determine the
envelope of possible form, fit, and function. Statistical
analysis can show that, even if exceeding a requirement
is possible, it may be extremely unlikely.
Worstcase tolerance analysis is conservative in that
when many tolerances combine, it becomes increasingly
unlikely that all dimensions will be simultaneously
worstcase. Statistical tolerance analysis usually
assumes a normal distribution of dimensions in the
tolerance zone, which may be unrealistic. In addition,
care must be exercised when combining tolerances, in
that:
(1) If some tolerances are much smaller than others,
their inclusion in tolerance stackup analysis is
superfluous. Consideration of significant digits
may be helpful, e.g., a 0.030 tolerance may have
a smallest unit of measurement greater than a
0.0005 tolerance.
(2) It may be superfluous to combine tolerances from
different manufacturing processes, e.g.,
machining and casting.
4.1.4 Example
In the following hypothetical example, the sensitivity of pressure with respect to throat area is
being determined. The equation for this analysis is the pressure equation. The equation for pressure is:
P
c
= r
b
C
*
g
A
s
A
*
(4.1)
where P
c
is the chamber pressure, r
b
is the propellant burn rate, C
*
is the propellant gas characteristic
exhaust velocity, is the propellant density, g is gravity, A
s
is the propellant burn surface area, and A
*
is
the throat area. To find the sensitivity of pressure to motor throat area, take the partial derivative of
equation (4.1) with respect to A
*
. P
c
is taken over a narrow range where r
b
is approximately constant.
P
c
A
*
= – r
b
C
*
g
A
s
¸
1
]
1
1
A
*
( )
2
(4.2)
where ∂ designates a partial derivative. The sensitivity is found by substituting values for the variability
into the partial derivative equation. Numbers can be substituted into equation (4.2) to obtain the slope at
a particular value of A
*
. It is intuitively obvious that the relationship between the partial derivative and
A
*
is both negative and inversely proportional to A
*2
.
Another example of the approximation method is the substitution of selected values into the
thrust equation (4.6). The sensitivity of thrust to throat area is to be investigated for a hypothetical motor
with the following characteristics:
A
s
= 300 in
2
A
*
= 1.9 in
2
,
2.1 in
2
A
e
= 10 in
2
= 0.06 lbm/in
3
γ = 1.2
r
b
= 0.5 in/s
C
*
= 5100 in/s
g = 386.40 in/s
2
.
The first step is to calculate the chamber pressure, substituting into equation (4.1), using the first
value of A
*
which is 1.9 in
2
. The next step is to calculate Mach number (M) iteratively from equation (4.3):
A
e
A
*
¸
¸
_
,
2
·
1
M
2
2
+1
1+
−1
2
M
2 ¸
¸
_
,
¸
1
]
1
+1
−1
.
(4.3)
43
The third step is to calculate nozzle exit plane pressure (P
e
) from equation (4.4).
P
e
P
c
·
1
1+
−1
2
M
2
¸
1
]
1
−1
(4.4)
The next step is to calculate the thrust coefficient (C
F)
from equation (4.5).
C
F
·
2
2
−1
2
+1
¸
¸
_
,
+1
−1
¸
1
]
1
1
1
1−
P
e
P
c
¸
¸
_
,
−1
¸
1
]
1
1
1
¹
'
¹
¹
¹
¹
;
¹
¹
¹
1
2
(4.5)
The final step is to calculate thrust (T) from equation (4.6).
T = C
F
A
*
P
c
. (4.6)
The above calculations should be performed again, using A
*
= 2.1 in
2
. The values obtained from
both calculations are shown in table 42.
Table 42. Sensitivity analysis calculations.
P
c
M P
e
C
F
T
A
*
= 1.9 in
2
62.52 2.82 1.87 1.50 177.62
A
*
= 2.1 in
2
56.57 2.75 1.93 1.47 174.60
Conclusion:
The thrust (T) is inversely proportional to the throat area (A
*
).
4.1.5 Advantages
The effect of each parameter can be assessed to determine which parameters have the greatest
effect on the outcome of a process, and which parameters can yield the most benefit for adjustment.
4.1.6 Limitations
It is often not easy to isolate a variable to obtain a second derivative. For example, when
obtaining the sensitivity of thrust to throat diameter, changing a throat diameter not only changes motor
pressure, but changes the nozzle expansion ratio and exit pressure. The pressure ratio is typically found
by iteration or by tables. If the approximation approach above is taken, care must be used to ensure a
small enough range for the parameter values to achieve the desired accuracy.
44
4.2 Standard Dimensioning and Tolerancing
4.2.1 Description
Dimensioning and tolerancing on drawings is complicated enough to yield confusion, unless
standardized methods are employed at all stages of a project life from design to manufacture. Standard
dimensioning and tolerancing per ANSI–Y–14.5 is an internationally recognized method of stating
dimensions and tolerances.
4.2.2 Application
Standard dimensioning and tolerancing is typically applied in phase C but the technique could
also be applied in phase D. Standard dimensioning and tolerancing allows the design engineer to
indicate how tolerances are to be applied. This information is understood by draftsmen, manufacturing
engineers, and machinists to assure the form, fit, and function intended by the design engineer (or
systems engineer). Some of the methods of specifying dimensions and tolerances are discussed here.
4.2.3 Procedures
This section explains how dimensions and tolerances are specified on design drawings.
4.3
Following is a list of feature controls used to specify how a tolerance is to be applied to a design feature,
from ANSI–Y–14.5:
st r aight ness
f l at ness
ci r cul ar i t y
cy l i ndr i ci t y
pr of i l e of a l i ne
pr of ile of a sur f ace
angul ar i t y
per pendi cul ar i t y
par al l el i sm
posi t i on
concent r i ci t y
ci r cul ar r unout
t ot al runout
 B
dat um ident if ying
l et t er
XXX
basic dimension (XXX) ref erence dimension
A basic dimension is contained in a box (unless otherwise specified on the drawing). Basic
dimensions are the controlling dimensions on a drawing, and have no tolerances associated with them.
Basic dimensions set up a dimensional pattern, such as a bolt pattern. The locations of the features in the
pattern (e.g., bolt holes or threads) are toleranced using trueposition tolerances. Often the title block of a
drawing will indicate standard tolerances peculiar to that drawing that will apply to all basic dimensions
shown without a specified tolerance. A tolerance is shown for each significant digit used in the basic
dimensions on the drawing. For example, a tolerance of ± 0.1 may apply to all basic dimensions with
one significant digit.
45
Reference dimensions are the result of basic dimensions. In the example below, an inner and
coincident outer diameter are specified; the thickness is a reference dimension. In this situation, the inner
and outer diameters are of primary importance; the thickness is of secondary importance.
A rectangular box is used as a feature control box. The symbol in the first section of the box is
for the type of tolerance (e.g., true position). The first symbol in the second section is the type of
measurement (a diametrical tolerance is shown in fig. 41). The number is the size of the tolerance. The
second symbol in the second section (a circle with the letter “M,” “L,” or “R”) specifies the relation of
the tolerance with the size of the feature. The third (and any subsequent) section specifies which data are
used (which feature or dimension the tolerances concern).
4.2.4 Example
Following is a hypothetical fixed nozzle assembly used to show the purpose of dimension and
tolerance methods:
3.000 ±.020
1.000 ±0.003
4.00
 A 
0.030 A R
0.020 A R
 B
0.010 A B
1.25 X 12 UNF
0.005 A B R
Figure 41. Example of dimensioning and tolerancing.
In this example, datum A is defined by the throat of the nozzle, thus datum A is the axis of the
throat. The nozzle exit is referenced to datum A. The true position of the exit is to be within ±0.030 of the
throat axis (datum A), and the exit plane is to be within a 0.020 tolerance zone perpendicular to the throat
axis. The true position tolerance is not affected by the feature size of the throat diameter. (The “R” inside
the circle indicates that the position tolerance is applied “regardless of feature size.” An “M” inside the
circle would denote that the position tolerance applies to “maximum material condition;” thus the
tolerance can be relaxed by an amount commensurate with the difference that the size of the feature is
less than the maximum allowable size. An “L” inside the circle would denote “least material condition”
where the tolerance applies to the smallest feature size allowable.) The exit plane also defines datum B.
The boss at the end of the nozzle is controlled by a total runout tolerance. The surface is to be
within a 0.010 tolerance zone perpendicular to the axis made by the throat and exit (datums A and B).
The threads of the nozzle are to be concentric to the throat and exit axis within 0.005, and the axis of the
threads is to be within ±0.015 of the throat axis. Note that for the profile type tolerance controls (e.g.,
46
runout or perpendicularity), the number defines a tolerance zone. This means that the total “width” of the
acceptable deviation is defined by the tolerance. Thus a tolerance zone of 0.020 is analogous to a ±0.010
tolerance. For position tolerances, the number call out is ±, thus the axis of the nozzle exit must fall
inside a circle of 0.030 radius around the throat axis.
Note that the tolerances in this example control the thrust vector. The length of the nozzle is con
trolled by a basic dimension. The exit is truepositioned to the basic dimension from the nozzle throat,
and the required perpendicularity to the throat axis is greater than the true position tolerance. The nozzle
exit is toleranced to keep the thrust vector in line (within a certain amount) with the throat axis. The
nozzle boss is controlled by runout to the axis defined by the throat and exit plane. The boss surface
tolerance is to facilitate a consistent seal with the motor. The thread is controlled by concentricity to the
same axis to keep the thrust axis in line with the motor axis. It can be seen that the thickness of the boss
is not a controlling dimension; it is a reference dimension. If this dimension were not specified, the form,
fit, or function of the component would not be affected.
4.2.5 Advantages
Dimensioning and tolerancing per ANSI–Y–14.5 is fairly standard. In addition, some aspects of
dimensioning and tolerancing per ANSI–Y–14.5 are better suited for production. For example, true posi
tioning allows for a circular tolerance zone, whereas putting tolerances to rectangular coordinates allows
a square tolerance zone. Thus, a functional part that would comply with true position tolerances may not
comply with rectangular tolerances. Dimensioning strategy can minimize the cumulative tolerance
stackup. This is facilitated by following the dimensioning and tolerancing system of ANSI–Y–14.5.
4.2.6 Limitations
A moderate amount of training and practice is required to effectively use standard dimensioning
and tolerancing.
4.3 Tolerance Stackup Analysis
4.3.1 Description
Tolerance stackup analysis determines if a form, fit, or function problem exists when
manufacturing tolerances combine in a finished part or assembly. Tolerance stackup analysis is typically
performed by either assuming worstcase allowable dimensions, or by using statistical analysis of
tolerances.
4.3.2 Application
Tolerance stackup analysis is typically performed in phase C or D. This technique is used to
determine the possibility or probability of having form, fit, or function problems with a design, or to
determine a tolerance or dimension necessary to avoid form, fit, or function problems.
4.3.3 Procedures
Three typical methods for tolerance stackup analysis are:
(1) Worstcase tolerance stackup analysis, used to determine size or position if all applicable
dimensions occur at the worstcase extremes of the tolerance zones simultaneously.
47
(2) Statistical analysis of tolerances, used where the expected standard deviations of tolerances
are combined to determine the probability of a final tolerance.
4.4
(3) Design using simulation methods, where a computer is used to do a Monte Carlo analysis
of the possible combinations of tolerances.
4.5
4.3.4 Example
In the following hypothetical Oring joint assembly (fig. 42), the tolerances of each component
are shown in figure 43. Find the maximum tolerance stackup possible to obtain the minimum squeeze,
and the probability that the squeeze will be less than 0.035. The nominal squeeze is 0.050 inches.
Figure 42. Oring joint.
±0.010
±0.010
±0.005
Figure 43. Oring joint components.
The probability of the squeeze being less than 0.035 is obtained by finding the distance from the
mean (in terms of standard deviations) that this condition represents. The standard deviation is assumed
to be one third of the tolerance on the parts (this means all parts will fall within 3 standard deviations of
the nominal dimension) and is therefore:
Component standard deviation = 0.010/3 = 0.0033
Oring standard deviation = 0.005/3 = 0.00167
and by summation of squares,
system standard deviation = (2(0.0033)
2
+ (0.00167)
2
)
.5
= 0.005.
For a squeeze of 0.035, the distance (in standard deviations) from the mean (z) is
z = (0.035–0.050)/0.005 = –3.0.
48
Using a table for the normal distribution function, the area under the half curve for z = ±3 is
0.4987. Since this is a onesided question (no interest in the squeeze being 0.065), the area under the
curve beyond z = 3 is (0.5–0.4987) = 0.0013. This value is interpreted as 0.13 percent probability that
the squeeze on the Oring will be 0.035 or less.
A close look at the example above will show that more sources of variation are possible than
those considered. For example, the surfaces compressing the Oring may not be flat or normal to the
direction of squeeze. Also, position tolerances are often determined at maximum material condition, thus
position can vary more when not at maximum material condition. It can be extremely cumbersome to
perform a statistical analysis of all the possible variations on some assemblies, so software exists to
perform the statistical analysis. A typical example of software is the “Variation Simulation Analysis”
4.5
that uses Monte Carlo methods to simulate the possible ways that the tolerances can stack up.
4.3 4.4
The
results can be used to determine probabilities that certain overall tolerances will exceed a critical value,
or which tolerances are most important to form, fit, or function.
4.3.5 Advantages
Worstcase tolerance analysis can simply determine the envelope of possible form, fit, and func
tion. Statistical analysis can show that, even if exceeding a requirement is possible, it may be extremely
unlikely.
4.3.6 Limitations
Worstcase tolerance analysis is conservative, in that when many tolerances combine, it becomes
increasingly unlikely that all dimensions will be worstcase simultaneously. Statistical tolerance analysis
usually assumes a normal distribution of dimensions in the tolerance zone, which may be unrealistic. In
addition, care must be exercised when combining tolerances, in that:
(1) If some tolerances are much smaller than others, their inclusion in tolerance stackup
analysis is superfluous. Consideration of significant digits may be helpful, e.g., a 0.030
tolerance may have a smallest unit of measurement greater than a 0.0005 tolerance.
(2) It may be superfluous to combine tolerances from different manufacturing processes, e.g.,
machining and casting.
4.3.7 Bibliography
Craig, M.: “Managing Variation by Design Using Simulation Methods.” Applied Computer Solutions,
Inc.
49
REFERENCES
4.1 “System Sensitivity Analysis.” Edited by J.B. Cruz, Jr., Dowden, Hutchinson and Rose, Inc.
Stroudsburg, PA.
4.2 Hill, P.G., and Peterson, C.R.: “Mechanics and Dynamics of Propulsion.” Third edition,
AddisonWesley Publishing Company, Reading, MA, November 1970.
4.3 “Dimensioning and Tolerancing.” ANSI–Y–14.5M, The American Society of Mechanical
Engineers, United Engineering Center, 345 East 47th Street, New York, NY 10017, 1982.
4.4 Shigley, J.E.: “Mechanical Engineering Design.” Third edition, McGrawHill.
4.5 “Variation Simulation Analysis Software, Getting Started Manual.” Applied Computer Solutions
Inc., 300 Maple Park Blvd., St. Clair Shores, MI 48081.
410
5. GRAPHICAL DATA INTERPRETATION TOOLS
There are numerous excellent texts on the appropriate use of graphical data interpretation tools.
While this section lists and briefly discusses some of the available tools, the neophyte reader is advised
to read and utilize standard handbook references when using these techniques in problem solving to
avoid misuse and error. This toolbox is to provide knowledge of the existence of these techniques, and
references for their appropriate application.
One way to analyze data is by graphical interpretation. The analysis can be used to monitor
performance, identify relationships, and reveal the most important variables in a set of data. The scatter
diagram, section 5.1, makes it possible to determine if any relationship exists between two variables.
The control chart, section 5.2, monitors the performance of a process with frequent outputs. Control
charts are useful in trend analysis, section 8, and statistical process control, section 7.14. The bar chart
compares quantities of data to help identify distribution patterns. This chart is discussed in section 5.3.
One of the most common data displays is the timeline chart, section 5.4. This chart displays
changes over time. Sorting data that share a common characteristic into different groups is often
accomplished with a stratification chart. This chart is discussed in section 5.5. A Pareto chart, section
5.6, is used typically when there is a need to know the relative importance of data or variables. This
chart will also identify the problems, causes, or conditions that occur most frequently. A histogram,
section 5.7, is a bar chart that shows a dispersion of data over a specified range. This type of chart is
commonly used in presentations to make data easier to interpret.
A summary of the advantages and limitations of each tool or methodology discussed in this
section is presented in table 51.
5.1 Scatter Diagram
5.1.1 Description
Scatter diagrams, also called XY graphs, plot raw data and allow the analyst to determine if any
relationship exists between two variables. No interpretation of the data should be attempted, but
correlations can be inferred.
5.1
5.1.2 Application
The graphic display of the scatter diagram can help one determine possible causes of problems,
even when the connection between two variables is unexpected. The direction and compactness of the
cluster of points gives a clue as to the strength of the relationship between the variables. The more that
this cluster resembles a straight line, the stronger the correlation between the variables. The scatter
diagram technique is best applied in phase E.
The scatter diagram displays one variable on the horizontal (X) axis and the other variable on the
vertical (Y) axis. If there is a correlation between the two variables, positive or negative, it can be
assumed if the data from one are changed, then this will effect the data from the other.
5.2
51
52
Table 51. Graphical data interpretation tools and methodologies.
Tool or Methodology Section Advantages Limitations
Scatter diagram 5.1 (1) The general relationship between two variables can
be determined at a glance.
(2) The graph can help determine a possible cause(s) of
problems by looking at correlations.
(1) The choice of scale for the graph can distort the
data, thus possibly giving the appearance of a
correlation that is better or worse than reality.
(2) The correlation does not prove a causeandeffect
relationship.
Control chart 5.2 (1) The control chart helps one understand the
capabilities of the process.
(2) The chart can prevent tampering with processes that
are under statistical control.
(3) The chart monitors the effects of process changes
that are aimed at improvement.
(4) Control charts can be used without extensive
knowledge of statistics.
(1) The control chart tells only if the process is in
control.
(2) The underlying causes are not determined.
Bar chart 5.3 (1) The bar chart tells its story at a glance.
(2) It makes graphic comparisons of quantity easy to
see.
A bar chart is limited in the number of data categories
that can be displayed at one time.
Timeline chart 5.4 (1) The timeline chart shows a “moving picture” of
fluctuations over time.
(2) Defect rates can be plotted on time lines in order to
identify trends.
The timeline chart shows the direction of change but it
gives no indication as to the reason for the change.
Stratification chart 5.5 The approach not only produces a priority ordering of
the problems but also identifies an improvement
strategy.
(1) The correct stratification variables for resolving a
problem are generally not known prior to data
collection.
(2) All potentially important stratification variables
cannot be determined without planning.
Pareto chart 5.6 (1) The pareto chart helps to identify the few areas of
concern that are most important.
(2) The chart is useful in analyzing defect data.
A poor pareto chart will result if the causes chosen to
study are wrong. Some preplanning needs to be done
before choosing categories.
Histograms 5.7 (1) A histogram helps identify changes in a process as
the data changes.
(2) A histogram helps establish standards for a process.
A histogram is not a good tool for computing process
capability.
5.1.3 Procedures
As described in reference 5.2, a scatter diagram is prepared in the following manner:
(1) Collect the two selected variables of each occurrence.
(2) Draw the horizontal and vertical scales with equal length.
(3) The dependent variable, the one that you can have an effect on, is assigned to the vertical
(Y) axis. The independent variable is assigned to the horizontal (X) axis. Set the scale
intervals and label.
(4) Plot each data point.
(5) A possible relationship can be determined by visual inspection of the graph.
5.1.4 Example
As adapted from reference 5.3, an aptitude test was given to 10 employees and the scores were
then compared to the production levels of these employees over a certain time period. The scatter
diagram, example shown in figure 51, would show if there were any relationship between the test scores
and the production levels.
Employee Test Score Production Level
1 27 120
2 13 80
3 8 60
4 37 150
5 32 135
6 10 70
7 17 95
8 22 105
9 6 50
10 7 55
This plot shows that the higher test scores result in higher production levels.
5.1.5 Advantages
(1) The general relationship between two variables can be determined at a glance.
(2) The graph can help determine a possible cause of problems by looking at correlations.
5.1.6 Limitations
(1) The choice of scale for the graph can distort the data, thus possibly giving the appearance
of a correlation that is better or worse than reality.
(2) The correlation does not prove a causeandeffect relationship.
53
•
•
•
•
•
•
•
•
•
•
0
20
40
60
80
100
120
140
160
0 5 10 15 20 25 30 35 40
P
r
o
d
u
c
t
i
o
n
L
e
v
e
l
Test Score
Figure 51. Scatter diagram example.
5.2 Control Chart
5.2.1 Description
A control chart monitors the performance of a process with frequent outputs. The chart shows a
pictorial representation of an ongoing process and determines whether or not the process is performing
within acceptable parameters. The control chart is based on four concepts:
(1) All processes change with time.
(2) Individual points of the process are unpredictable.
(3) A stable process changes randomly, and groups of points from a stable process tend to fall
within predictable bounds.
(4) An unstable process does not change randomly, and when changes occur they are generally
out of the range of normal operations.
5.2
5.2.2 Application
The control chart technique is best performed in phase E. As described in reference 5.2, control
charts are used to show the variation of several variables including average ( X ) and range (R) as well as
the number of defects (PN), percent defective (P), defects per variable unit (U), and defects per fixed
unit (C).
The upper control limits (UCL) and lower control limits (LCL) should not be confused with
specification limits. The control limits show the natural change of a process, such that points within the
limits generally indicate normal and expected change. Points that are outside of the control limits reveal
that something has occurred that requires special attention because the points are outside of the builtin
54
systematic cause of change in the process. One point that is outside of the control limits does not mean
the process is out of control but it should be explained.
The control chart can to be used continuously to determine whether the process remains within
established control limits. As new points are added, the chart can be monitored for points that may fall
outside of the limits and require causes to be identified.
Control charts are used in performing statistical process control (SPC) (sec. 7.14) and trend
analysis (sec. 8.).
5.2.3 Procedures
As described in reference 5.2, a control chart (fig. 52) is constructed in the following manner:
(1) Determine the control limits to show the expected change of the process.
(2) Gather data.
(3) Plot the data on the control chart to evaluate performance and identify the points outside of
the control limits.
(4) Determine why points are outside of the control limits.
(5) Find ways to identify causes of problem points, reduce the normal variation, and improve
the mean.
5.2.4 Example
0 5 10 15 20 25 30
0
0.1
0.2
0.3
0.4
0.5
0.6
Event
0
0.1
0.25
0.076
0.083
0.125
0.25
0.333
0.286
0.333
0.5
0.182
0.076
0.1
0.091
0.5
0.2
0.25
0.5
0.2
0 0
0.091
0.167
0 0 0
0.111
0
0.182
0
UCL = 0.55
LCL=0.15
Figure 52. Control chart example.
55
5.2.5 Advantages
(1) The control chart helps the analyst understand the capabilities of the process.
(2) The control chart can prevent tampering with processes that are under statistical control.
(3) The control chart monitors the effects of process changes that are aimed at improvement.
(4) Control charts can be used without extensive knowledge of statistics.
5.2.6 Limitations
(1) The control chart tells only if the process is in control.
(2) The control chart does not indicate the underlying cause unless data on outside processes
are included in the analysis.
5.3 Bar Chart
5.3.1 Description
Bar charts show a comparison of quantities of data to help identify quantity changes. The
quantities of data are depicted by the lengths of the bars that represent cost, percentage, or frequency of
events. The bars may be horizontal or vertical.
5.2
5.3.2 Application
Bar charts are one of the most common types of data display and this technique is typically
performed in phase E. Differences and similarities between and among selected categories are
emphasized by the heights of the columns. Bar charts can show double and triple bars to compare
different time periods or different populations.
5.3.3 Procedures
As described in reference 5.2, a bar chart (fig. 53) is constructed in the following manner:
(1) If necessary, raw data are entered on a checklist (sec. 7.8).
(2) List the categories across the horizontal scale at the bottom.
(3) Label the quantities on the vertical scale at the left. Make sure the scale is broad enough to
include the highest and lowest value in each category.
(4) Draw the bar according to the quantity of each category.
(5) Give the bar chart a legend to identify different colors or patterns.
56
5.3.4 Example
Sale of Household Appliances
1980 versus 1990
(in millions)
0
1
2
3
4
5
6
7
8
9
10
(Nominal Categories)
1980
1990
Figure 53. Bar chart example.
5.3.5 Advantages
(1) The bar chart tells its story at a glance.
(2) The bar chart makes graphic comparisons of quantity easy to see.
5.3.6 Limitations
A bar chart is somewhat limited in the number of data categories that can be displayed at one
time.
57
5.4 TimeLine Chart
5.4.1 Description
The timeline chart is among the most common types of data displays. The chart graphically
displays changes over a period of time.
5.4.2 Application
The timeline chart is a special case of XY plots where the independent variable is some time
value. The chart connects data points with line segments. The line segments connecting the points on the
chart give a clear picture of changes over time. The vertical scale is a quantity while the horizontal scale
is divided into time intervals such as “hours,” “days,” and “weeks.”
5.2
This technique is best performed
in phase E.
5.4.3 Procedures
As described in reference 5.2, a timeline chart (fig. 54) is prepared in the following manner:
(1) Enter the raw data on a checklist (sec. 7.8).
(2) Establish time intervals (usually hours, days, weeks, etc.) for the horizontal axis. The
intervals should be evenly spaced and labeled.
(3) Establish the quantities for the vertical axis and make them evenly spaced (e.g., 10, 20, 30,
etc.) and label the axis.
(4) Connect, with line segments, the quantities plotted for each successive interval.
(5) If the points are difficult to read, add horizontal and vertical grids.
(6) Title the chart to define the time period for which the data are displayed.
5.4.4 Example
A study was made comparing the average number of errors that were made per operator at
different times of the day over a certain time period (fig. 54).
5.4.5 Advantages
(1) The timeline shows a “moving picture” of fluctuations over time.
(2) Defect rates can be plotted on time lines in order to identify trends.
58
l
l
l
l
l
l
l
l l
0
9
:
0
0
:
0
0
A
M
1
0
:
0
0
:
0
0
A
M
1
1
:
0
0
:
0
0
A
M
1
2
N
o
o
n
0
1
:
0
0
:
0
0
P
M
0
2
:
0
0
:
0
0
P
M
0
3
:
0
0
:
0
0
P
M
0
4
:
0
0
:
0
0
P
M
0
5
:
0
0
:
0
0
P
M
0
1
2
3
4
5
6
Q
u
a
n
t
i
t
y
Time Intervals
l
No. of Errors
Figure 54. Timeline chart example.
5.4.6 Limitations
The timeline chart shows the direction of change but it gives no indication as to the reason for the
change.
5.5 Stratification Chart
5.5.1 Description
The term “stratification,” derived from “stratum,” is used in data analysis. Stratification is done by
sorting data into different groups that share a common characteristic. Some common stratification
variables are shift, operator, and machine.
5.5.2 Application
The stratification chart is best applied in phase E. Comparisons of different groups, units, or other types
of strata can often lead to suggesting an improvement strategy. For example, a process is incurring a 10
percent defect rate with a particular product. You can stratify by vendor, lot, operator, shift, time,
machine, etc. and compute a percent defective for each category (stratification variable).
The data can be depicted in graphic form for easy visual interpretation. Should the data not include a
significant problem, select other stratification variables and collect more data. The graph may show that
one category is producing a higher defect rate than others. This does not mean the “cause” of a problem
has been found. What has been found is where the problem is occurring the most. The cause has yet to
be determined.
5.4
59
5.5.3 Procedures
As described in reference 5.4, the stratification process (fig. 55) is performed in the following
manner:
(1) Choose the stratification variables.
(2) Gather data and record the potentially important stratification variables.
(3) Graph the data using one of a number of different tools, such as bar chart (sec. 5.3), Pareto
chart (sec. 5.6), and histograms (sec. 5.7).
(4) Analyze the data on the chosen stratification variables and compare to each other.
(5) Separate the possible problem areas into special and common cause problems.
(6) If no conclusions are found, choose different stratification variables.
(7) Determine the strategy to improve the problem.
History of Discrepancy Reports for a Solid Rocket Motor
0
2
4
6
8
10
12
14
16
18
20
Month
NOZZLE
CASE
JOINT
INSULATION
IGNITER
PROPELLANT
LEGEND
Figure 55. Stratification (histogram) chart example.
510
5.5.4 Example
Shown in figure 55 is a histogram of discrepancy reports for a solid rocket motor (SRM),
stratified by components.
5.5.5 Advantages
The approach not only produces a priority ordering of the problems but also identifies areas for
improvement.
5.5.6 Limitations
(1) The correct stratification variables for resolving a problem are generally not known prior to
data collection.
(2) All potentially important stratification variables cannot be determined without planning.
5.6 Pareto Chart
5.6.1 Description
When there is a need to know the relative importance of data or variables (problems, causes, or
conditions), a Pareto chart is often used. This chart helps to highlight the few data or variables that may
be vital. The Pareto chart also helps to identify which problems, causes, or conditions are the most
important or most frequent so they can be addressed first.
5.2
5.6.2 Application
The Pareto chart can be used to examine the “how,” “what,” “when,” “where,” and “why”
of a suspected problem cause. This technique is typically performed in phase E. The chart is an
illustration of the data as of a specific time period. The data are arranged in descending order with the
most important to the left. The Pareto chart is based on the “Pareto principle” which states that a few of
the causes often account for most of the effects.
5.5
Pareto charts are used in performing problem trend
analyses (sec. 8.2).
5.6.3 Procedures
As described in reference 5.2, a Pareto chart (fig. 56) is created in the following manner:
(1) Identify the most likely causes of a problem (take from the cause/effect diagram (sec. 7.2)).
(2) Gather the data on causes; if necessary, use a checklist (sec. 7.8).
(3) Summarize the numbers of observations and calculate the percentages of each cause.
(4) Set the right vertical scale from zero to 100 percent.
(5) Make the left vertical scale the same height as the right scale and set it from zero to the
number of observations.
511
Power
Supply
Machine
Calibration
Connection Electrical
Component
Feed
Transformer
Wrong
Connection
Operator
Training
100
80
60
40
20
0
20%
40%
60%
80%
100%
0%
H
2
O Filter
Figure 56. Pareto chart example.
(6) The columns are drawn using the left scale.
(7) The first point is plotted at the upper center of the first column.
(8) Calculate and add together the percentages of cause one and two. The second point,
corresponding to their sum, is plotted across from the right scale directly over the second
column. The third point is found by adding the percentage of cause three to the total of one
and two, and plot. The total of all columns added together should be 100 percent, and the
last point is at the 100percent point.
(9) The plotted points are then joined with line segments.
The chart in figure 56 reveals the slope is more radical over the first two bars (power supply and
machine calibration) and this means that the majority of the problems occur in these categories, i.e.,
areas to the left of the most radical slope are the most probable problem areas. This observation is even
more obvious when the heights of the bars are examined.
512
5.6.5 Advantages
(1) The Pareto chart helps to identify the few areas of concern that are most important.
(2) The chart is useful in analyzing defect data.
5.6.6 Limitations
A poor Pareto chart will result if the causes chosen to study are wrong. Some preplanning needs
to be done before choosing categories.
5.6.7 Bibliography
Cane, V.E.: “Defect Prevention, Use of Simple Statistical Tools.” Ford Motor Company, Livonia, MI,
1989.
Hines, W.W., and Montgomery, D.C.: “Probability and Statistics in Engineering and Management
Science.” John Wiley, New York, 1986.
Wadsworth, S. and Godfrey: “Modern Methods for Quality Control and Improvement.” John Wiley,
New York, 1986.
5.7 Histograms
5.7.1 Description
Histograms are bar charts that show a dispersion of data over a specified range. This spread of
data makes presentations easier to interpret.
5.1
5.7.2 Application
When data are plotted on histograms, many items tend to fall toward the center of the data
distribution. Fewer items fall on either side of the center. The bars are proportional in height to the
frequency of the group represented. Since group intervals are equal in size, the bars are of equal width.
5.4
The histogram is best applied in phase E.
5.7.3 Procedures
As described in reference 5.2, a histogram (fig. 57) is constructed in the following manner:
(1) Gather the data to be plotted and count the total number of data points.
(2) Find the range of the data by subtracting the smallest data point from the largest.
(3) The number of data bars in the graph should be limited to between 6 and 12. The width of
each bar is determined by dividing the range of data by the selected number of bars.
(4) Scale the groups of data on the horizontal axis.
513
(5) Scale the frequency of occurrence or the numbers on the vertical scale.
(6) Plot the frequency of occurrence of the numbers in ascending order.
(7) Draw the height of each bar to show the number or frequency of the group interval using
the scale on the vertical axis. Each bar, including all data points, is the same width.
5.7.4 Example
The chart in figure 57 displays a typical histogram.
0
1
2
3
4
5
6
7
8
Time To Complete Tasks (Hours)
0
1
2
3
4
5
6
7
8
010 1020 2030 3040 4050
Figure 57. Histogram example.
5.7.5 Advantages
(1) A histogram helps identify changes in a process as the data changes.
(2) A histogram helps establish standards for a process.
5.7.6 Limitations
A histogram is not a good tool for computing process capability.
514
REFERENCES
5.1 Brocka, B. and Brocka, M.S.: “Quality Management, Implementing the Best Ideas of the
Masters.” Business One Irwin, Homewood, IL 60430.
5.2 Hunt, V.D.: “Quality in America, How to Implement a Competitive Quality Program.”
HD62.15.H86, Business One Irwin, Homewood, IL 60430, 1992.
5.3 Lyonnet, P.: “Tools of Total Quality, An Introduction to Statistical Process Control.” Chapman
& Hall, 1991.
5.4 Cane, V.E.: “Defect Prevention, Use of Simple Statistical Tools.” Ford Motor Company,
Livonia, MI, 1989.
5.5 “Total Quality Management, A Guide for Implementation.” DOD 5000.51–G (Draft), February
15, 1989.
515
6. STATISTICAL TOOLS AND METHODOLOGIES
There are numerous, excellent and highly detailed texts on the appropriate use of statistical
techniques. While this section lists and briefly discusses some of the available tools, the novice
statistician is cautioned to read and utilize standard, handbook references when using these techniques in
problem solving. Use solely of this text might well result in misuse and error. This toolbox does provide
a suitable knowledge of the existence of these tools and references for their appropriate application.
In this section, the following typical statistical processes are discussed: “studentt” (t test)
analysis, analysis of variance (ANOVA), correlation analysis, factorial analysis, confidence analysis,
regression analysis, and response surface methodology.
In many of these analyses, a comparison of sample statistics and population statistics will be
made. Here, it is assumed that population statistics would be obtained if an infinite number of specimens
could be measured, or if the solution to a function for the probability distribution of points were
available. Sample statistics are made from actual measurements of a sample with a finite number of
specimens. When only sample statistics are available (as is usually the case in engineering applications),
there is a finite probability that they are “close” to the population statistics.
6.1
A summary of the advantages and limitations of each tool or methodology discussed in this
section is presented in table 61.
6.1 “Studentt” Analysis
6.1.1 Description
As described in reference 6.1, the “studentt” compares the sample statistic “t,” which is based on
the sample mean and standard deviation, to the tdistribution for the same sample size and a desired
significance (probability of error). The tdistribution is similar to the normal distribution in that with an
infinite sample size, the tdistribution is equivalent to the standard normal distribution. At sample sizes
lower than infinity, the tdistribution becomes “lower and flatter” than the normal distribution. The
output of the tdistribution chart is the probability (α) that t exceeds a certain t
on the ordinate of the t
distribution chart. However, usually the probability is chosen and t
is sought; a tdistribution table is
usually used to find t
.
The tdistribution was described in 1908 by W.S. Gosset under the pen name “student,” thus the
name “studentt” analysis.
6.1.2 Application
“Studentt” analyses, as described in reference 6.2, are used when sample sizes are low for the following
functions:
(1) Determine if a sample mean is equivalent to a population mean within a given probability
of error.
(2) Determine if two sample means are equivalent to each other within a given probability of
error.
This technique is typically applied in phase D but may also be performed in phase C or E.
61
62
Table 61. Statistical tools and methodologies.
Tool or Methodology Section Advantages Limitations
“Studentt” analysis 6.1 The procedure is relatively simple to apply. The parent distribution must be reasonably close to a
normal distribution.
ANOVA 6.2 Sources of variation can be found, random variation
isolated, or any chosen source of variability isolated.
The processes are timeconsuming and often
approximate.
Correlation analysis 6.3 The analysis is quite simple. A straightline relationship is assumed.
Factorial analysis 6.4 Sources of variation can be found, random variation
isolated, or any chosen source of variability isolated.
Also, interactions between variables can be isolated, and
large numbers of variables can be solved.
The processes in factorial analysis are more time
consuming than the analysis of variance. A full factorial
analysis does not solve for exponential or polynomial
effects. The fractional factorial analysis does not solve
for all effects and higher order effects separately.
Confidence/reliability
determination and analysis
6.5 This analysis can give a realistic probability of whether
or not a process may yield a value which is above or
below a requirement.
A sample statistic must be known or assumed, such as
the population standard deviation, before an analysis can
be performed.
Regression analysis 6.6 A mathematical relationship can be determined, by hand
or computer, when the relationship is not obvious by
inspection.
If the data are discrete (e.g., integer data), the actual line
generated will only approximate the actual relationship.
Response surface methodology 6.7 A mathematical relationship can be determined, by hand
or computer, when the relationship is not obvious by
inspection.
If the data are discrete (e.g., integer data), the actual line
generated will only approximate the actual relationship.
6.1.3 Procedures
The use of a ttest for determining if a sample mean is equal to a chosen population mean will be
shown here.
(1) Determine the target mean and significance level desired.
(2) Develop null and alternate hypotheses for the problem being investigated. If it is desired to
prove that the sample mean is on one particular side of the population mean, the null
hypothesis is that the sample and population mean are equal. The alternate hypothesis is
that the sample mean is on the particular side of the population mean. If it is desired to
prove that the sample mean is not on either side of the population mean, the null hypothesis
would be the same, but the two alternate hypotheses would be that the sample mean is
above or below the population mean. This latter situation would use a “twotailed”
analysis.
(3) Determine the mean and standard deviation of the sample.
(4) Determine the t value using equation (6.1).
t · sample mean – target mean
sample /( n)
1/2
(6.1)
(5) Compare t with t
α
for the desired significance and degreesoffreedom (DOF) (n–1).
If t is greater than t , the null hypothesis is disproved, i.e., it cannot be assumed with the chosen
confidence that the sample mean is equivalent to the target mean. For a twotailed analysis, if t is greater
than t
( /2)
(or t is less than – t
( /2)
), the null hypothesis is disproved.
6.1
6.1.4 Example
Pull tests of a propellant sample yielded the following strains before failure: 29, 31, 35, 34,
and 36 percent. The nominal strain capability is 34 percent. Determine with a 0.10 significance, if the
propellant batch is representative of the nominal propellant. Since the mean of the propellant batch could
be =, >, or <34 percent, a twotailed analysis will be done. Thus, α/2 will be used (0.05 significance).
The null hypothesis will be a strain capability equal to 34 percent.
The sample mean is 33 and the sample standard deviation is 2.915. Substituting into equation (6.1),
t = 0.1539. From the tdistribution table for 4 DOF, t
= 2.134.
If H
o
had been rejected, it could be stated that there was only one chance in ten that the null
hypothesis was rejected when it should not have been. This is referred to as a type I error.
If H
o
were not rejected, it could be stated that the null hypothesis could not be rejected at the
0.10 level of significance unless the probability of a type II error is determined. The determination of the
probability of a type II error is complicated and many texts consider it beyond their scope.
6.1.5 Advantages
The procedure is relatively simple to apply.
63
6.1.6 Limitations
The distribution of the parent population must be reasonably close to a normal distribution.
6.1.7 Bibliography
Crow, E.L., Davis, F.A. and Maxfield, M.W.: “Statistics Manual.” NAVORD Report 3369, NOTS 948.
Handbook 91, “Experimental Statistics.” U.S. Department of Commerce, National Bureau of Standards.
Mendenhall, W.: “Introduction to Probability and Statistics.” Fourth edition, Wadsworth Publishing
Company, Belmont, CA 94002, 1975.
6.2 Analysis of Variance
6.2.1 Description
ANOVA is a technique used in design of experiments (sec. 5.5) to compare sample statistics, to
determine if the variation of the mean and variance between two or more populations are attributable to
sources other than random variation.
6.1
6.2.2 Application
The ANOVA technique is typically performed in phase D but may also be performed in
phase C or E.
Some of the uses for analysis of variance are:
(1) Determining if two or more processes are producing products that are consistent with each
other.
(2) Determine which two or more processes are different if a difference in processes is
detected.
(3) Eliminate one source of variation to determine the effect of the others.
(4) Determining the significance of each factor.
6.2.3 Procedures
As described in reference 6.1, to determine if two or more samples have different sample
statistics, the following methods can be used to determine if the withinsample variation is greater than
the sampletosample variation. If only one source of variation is being investigated, a one way
classification is used. A factor F (equation (6.2)) is compared to F , a value that is related to the total
DOF, based on the number of samples (k) and the sample size (n).
F·
betweensample variance
mean ofwithinsample variance
(6.2)
64
The DOF of the number of samples is k–1, and the DOF of the sample size is n–1. The total DOF
is k*(n–1). If F exceeds F ,
then a difference exists between the samples that is not only due to random
variation. F is found from an F distribution table.
Rather than determining sample statistics for each sample, approximation formulas that use sums
and averages of squares, can be used.
F ·
SS(bs)/ (k −1)
SSE / k(n −1)
, (6.3)
where SS(bs) is the sum of squares (betweensample) and SSE is the sum of squares error. The SSE is
determined from the sum of squares total (SST) and SS(bs) by the formula
SSE = SST – SS(bs).
SST and SS(bs) can be found using the formulas
SST = ∑ (y
ij
)
2
– C, SS(bs) = ∑ (T
i
)
2
/n – C,
and
C = T
2
/(k*n)
where y
ij
= each data point, T = total of all data points, and T
i
= total for each sample.
If two sources of variation are being investigated, a twoway classification is used. Data can be
arranged in blocks representing one source of variation, and one data point from each sample
representing the other source of variation is put into each block (see example below). If two sources are
being investigated, the following approximation equations can be used:
F(bs1) ·
MS(bs1)
MSE
·
SS(bs1)/ a −1
SSE / K(a −1)(b −1)
(6.4)
and
F(bs2) ·
MS(bs2)
MSE
·
SS(bs2)/ b −1
SSE / K(a −1)(b −1)
(6.5)
where
SSE = SST – SS(bs1) – SS(bs2); SST = ∑(y
ij
)
2
– C;
MS = Mean square MSE = Mean square error
SS(bs1) = ∑(T
i
)
2
/b – C; SS(bs2) = ∑(T
j
)
2
/a – C; and
C = T
2
/(k*n),
where a = the number of samples of one source of variation and b = the number of samples of the other
source of variation.
65
Other methods exist to isolate more sources of variability simultaneously. The latin square
method eliminates three sources, and the GrecoLatin method eliminates four sources. These methods
must use n
2
observations.
Analysis of covariance is a similar technique used when conditions (such as environmental)
change. The effect of this change is accounted for by using regression. This involves partitioning a total
sum of products rather than squares.
6.2.4 Example
In the following hypothetical example, the effect of two parameters on the variability of strain
capability of a solid propellant will be investigated. The use of three lots of polymer (parameter A) and
two lots of curative (parameter B) will be investigated. Six batches of propellant are mixed and tested
with the following average results:
Polymer Curative Percent Strain
1 1 30
1 2 34
2 1 32
2 2 36
3 1 31
3 2 33
The following table is arranged with parameter A set up in columns and parameter B set up in rows:
Curative Lot 1 Curative Lot 2 Total for Polymer
Polymer lot 1 30 34 64
Polymer lot 2 32 36 68
Polymer lot 3 31 33 64
Total for curative 93 103 196
here
C = (196)
2
/6 = 6402.67,
SST = 30
2
+ 34
2
+ 32
2
+ 36
2
+ 31
2
+ 33
2
– 6402.67 = 6426 – 6402.67 = 23.33,
SS(bs1) = (64
2
+ 68
2
+ 64
2
)/2 – 6402.67 = 6408 – 6402.67 = 5.33,
SS(bs2) = (93
2
+ 103
2
)/3 – 6402.67 = 16.67,
MS(bs1) = 5.33/2 = 2.67,
MS(bs2) = 16.67/1 = 16.67,
MSE = 1.33/((3–1)(21)) = 0.67,
F(sb1) = 2.67/0.67 = 4.0, and
F(sb2) = 16.67/0.67 = 24.88.
66
Note that a = 3 is the number of sources of variation of parameter A (polymer), and b = 2 is the
number of sources of variation of parameter B. Since F(sb1) is less than F for a 0.05 significance
(F = 5.14), polymer has no effect on strain capability. Since F(sb1) for a 0.05 significance is greater
than 5.99, strain capability is affected by the curative lot.
6.2.5 Advantages
Sources of variation can be found, random variation isolated, or any chosen source of variability
isolated.
6.2.6 Limitations
The processes are timeconsuming and often approximate.
6.3 Correlation Analysis
6.3.1 Description
Correlation analysis measures the strength of a linear relationship between two sets of data.
6.3
6.3.2 Application
Correlation analysis can be used to determine if a relationship exists between two independent sets
of variables. This technique is best performed in phase D but may also be performed in phase C or E.
6.3.3 Procedures
The procedures, as found in reference 6.3, for determining if two sets of data are linearly related
is as follows:
(1) Determine the mean of each set of data.
(2) Determine the r value of the two sets of data using the following equation:
r ·
Σ(x
i
− x )(y
i
− y )
Σ x
i
− x ( )
2
( )
1/2
Σ y
i
− y ( )
2
( )
1/2
(6.6)
where x and y are the means of the first and second set of data respectively. The value of r will
be between –1 and 1. If r is close to 0, then no correlation is implied; if r is close to 1 (or –1) then
a high degree of correlation is implied.
(3) Determine the significance of the r value by using the following equation:
z ·
(n − 3)
2
1n
(1+ r)
(1− r)
(6.7)
67
(4) Look up the z value in a standard normal distribution table to determine the probability of
having a correlation.
6.3.4 Example
The following hypothetical sets of measurements were taken: 5.4, 6.2, 6.5, 7, and 7.5; and 2.3,
2.1, 2, 1.8, and 1.6. The mean of the two sets are 6.52 and 1.96, respectively . The deviations, products,
and squares of the deviations from the means are shown in the following.
x y dx dy dx * dy dx
2
dy
2
5.4 2.3 –1.12 0.34 –0.3808 1.25 0.1156
6.2 2.1 –0.32 0.14 –0.0448 0.1024 0.0196
6.5 2.0 –0.02 0.04 –0.0008 0.0004 0.0016
7.0 1.8 0.48 –0.16 –0.0768 0.2304 0.256
7.5 1.6 0.98 –0.36 –0.3528 0.9604 0.1296
summations –0.856 2.548 0.292
Using equation (6.6), the r value is 0.992. Using this value for n = 5, z is –3.938, thus there is
less than a 0.01 percent chance of these two data sets not being related.
6.3.5 Advantages
This analysis is simple to apply.
6.3.6 Limitations
A straightline relationship is assumed.
6.4 Factorial Analysis
6.4.1 Description
There are three types of factorial analysis described in this section—factorial analysis, full
factorial analysis, and fractional factorial analysis. Factorial analysis is similar to ANOVA in that the
analysis is based on sums of squares, however, factorial analysis further subdivides the treatment of
sums of squares into components and can show interaction effects between parameters.
6.1
6.4.2 Application
Factorial analysis is used for applications similar to those for which ANOVA is used, except that
factorial analysis deals with levels of variables. Factorial analysis is used with a small number of
variables (e.g., two to four). Full factorial analysis is performed for more variables, but only at two
levels for each variable. Fractional factorial analysis is used when so many variables are being
investigated that experimenting with them is unfeasible. For example, if five variables are being
investigated, 2
5
or 32 experiments would have to be performed. For six variables, the number would be
64, and this is without replication. Thus, fractional factorial analysis is often economically necessary.
6.1
This technique is typically performed in phase C but may also be performed in phase D or E .
68
6.4.3 Procedures
As described in reference 6.1, factorial analysis is performed the same as analysis of variance
except that an analysis of variance is performed for each variable against each other variable.
The procedure for performing a full factorial analysis will be discussed here. With factorial
analysis, 2
n
factorial experiments will be performed, and to account for experimental variability, r
replications will be performed. Here n will be the number of factors rather than the sample size (which is
effectively two). With factorial analysis, certain computational shortcuts can be applied when only two
levels of each variable are used, assuming straight line relationships. The following is the procedure for
using the factorial analysis where n = 3.
(1) Arrange the factors and magnitudes in a table such as the following:
Table 62. Factorial analysis factors and magnitudes.
A0,B0,C0 M1 M2 M3 total A0, B0,C0
A1,B0,C0 M1 M2 M3 total A1, B0,C0
A0,B1,C0 M1 M2 M3 total A0, B1,C0
A1,B1,C0 M1 M2 M3 total A1, B1,C0
A0,B0,C1 M1 M2 M3 total A0, B0,C1
A1,B0,C1 M1 M2 M3 total A1, B0,C1
A0,B1,C1 M1 M2 M3 total A0, B1,C1
A1,B1,C1 M1 M2 M3 total A1, B1,C1
etc. where the first column represents the experimental conditions and M1, M2, and M3 represent the
resulting magnitudes after the experiment for replication 1, 2, and 3. The last column is the total of all
replications of experiments for each experimental condition.
(2) Obtain a table of effects totals by removing the middle columns in the above table.
(3) Apply the method of Yates to this table as follows:
a. Add n (3) columns in the place of the middle columns and three columns to the right
side of the table (table 63).
b. Add the first two totals in the totals column to get the first element in column 1. Add
the third and fourth totals in the totals column to get the second element in column 1.
Continue in a like manner to get the third and fourth elements in column 1. Obtain the
fifth through eighth elements in column 1 the same way except that the totals are
subtracted (first value subtracted from the second). Column 2 is constructed the same
way from column 1 as column 1 was constructed from the totals column. Column 3 is
constructed the same way from column 2. Column 3 is the effect totals as in analysis of
variation. The notation in column n (3) and the sum of squares column is shortened;
2:1 means the first element of column 2.
c. Add a row for the error sum of squares and error mean square, determined as in
ANOVA.
69
Table 63. Factorial analysis example.
Exp.
Condition
Totals from
Above
1 2 3 Sum of
Squares
Mean of
Squares
F
1 A0, B0,C0 t1 t1 + t2 (t1 + t2) + (t3 + t4) 2:1 + 2:2
3:1/(r2
n
)
SS1/DOF MS1/SME
2 A1, B0,C0 t2 t3 + t4 (t5 + t6) + (t7 + t8) 2:3 + 2:4
3:1/(r2
n
)
SS2/DOF MS2/SME
3 A0, B1,C0 t3 t5 + t6 (t2 – t1) + (t4 – t3) 2:5 + 2:6
3:1/(r2
n
)
SS3/DOF MS3/SME
4 A1, B1,C0 t4 t7 + t8 (t6 – t5) + (t8 – t7) 2:7 + 2:8
3:1/(r2
n
)
SS4/DOF MS4/SME
5 A0, B0,C1 t5 t2 – t1 (t3 + t4) – (t1 + t2) 2:2 – 2:1
3:1/(r2
n
)
SS5/DOF MS5/SME
6 A1, B0,C1 t6 t4 – t3 (t7 + t8) – (t5 + t6) 2:4 – 2:3
3:1/(r2
n
)
SS6/DOF MS6/SME
7 A0, B1,C1 t7 t6 – t5 (t4 – t3) – (t2 – t1) 2:6 – 2:5
3:1/(r2
n
)
SS7/DOF MS7/SME
8 A1, B1,C1 t8 t8 – t7 (t8 – t7) – (t6 – t5) 2:8 – 2:7
3:1/(r2
n
)
SS8/DOF MS8/SME
summation SSE SME
To find:
2:1 + 2:2 = (t1 + t2) + (t3 + t4) + (t5 + t6) + (t7 + t8)
2:3 + 2:4 = (t2 – t1) + (t4 – t3) + (t6 – t5) + (t8 – t7)
2:2 – 2:1 = (t5 + t6) + (t7 + t8) – (t1 + t2) + (t3 + t4)
2:4 – 2:3 = (t6 – t5) + (t8 – t7) – (t2 – t1) + (t4 – t3)
(4) The sum of squares column is generated by dividing the square of each adjacent element in
column 3 by r * 2
n
.
(5) The mean of squares column is generated by dividing each adjacent element in the sum of
squares column by its respective DOF. The DOF will be 1 for each effect, but will be n–1
for the error row.
(6) Obtain each F by dividing each mean square by the error mean square.
(7) Compare each F to F for n–1, DOF. If any F exceeds F , that effect is significant.
A fractional factorial analysis is performed the same way as the full factorial analysis except the
analysis is split into fractions of (1/2)
p
. Thus, if a five variable investigation (32 experiments) is split
into 1/4, the number of experiments will be 2
n–p
(eight) experiments.
6.4.4 Example
The following are the results of a hypothetical experiment to determine if mix time, mix speed, and
mix vacuum affects the burn rate of a propellant. Two levels of each parameter were tested as follows:
Effect Parameter Low (0) High (1)
A mix time 2 hr 3 hr
B mix speed 1 rps 2 rps
C vacuum no vacuum 0.2 atm
610
Each effect was assigned a high and low level (e.g., 1 rps was assigned as low, 2 rps was
assigned as high). The high and low levels are designated as 0 and 1, respectively. Each experimental
condition was repeated three times with the following results:
Exp. Condition Rep 1 Rep 2 Rep 3 Total
A0 B0 C0 0.47 0.47 0.52 1.46
A1 B0 C0 0.46 0.46 0.51 1.43
A0 B1 C0 0.47 0.48 0.52 1.47
A1 B1 C0 0.48 0.50 0.50 1.48
A0 B0 C1 0.51 0.50 0.54 1.55
A1 B0 C1 0.49 0.52 0.54 1.55
A0 B1 C1 0.52 0.51 0.55 1.58
A1 B1 C1 0.50 0.52 0.54 1.56
The table is repeated with the replication columns deleted and replaced with the application of three
columns for the Method of Yates. Three additional columns are added, one for the sum of squares, one
for the mean square, and one for the F value for each effect.
Exp.
Condition
Total 1 2 3 Sum of
Squares
Mean of
Squares
DOF F
A0 B0 C0 1.46 2.89 5.84 12.08 6.0803 6.0803 1
A1 B0 C0 1.43 2.95 6.24 –0.04 0.000067 0.000067 1 0.2977
A0 B1 C0 1.47 3.10 –0.02 0.10 0.000417 0.000417 1 1.8616
A1 B1 C0 1.48 3.14 –0.02 0.02 0.000017 0.000017 1 0.0745
A0 B0 C1 1.55 –0.03 0.06 0.04 0.00667 0.00667 1 29.77
A1 B0 C1 1.55 0.01 0.04 0 0 0 1 0
A0 B1 C1 1.58 0 0.04 –0.02 0.000017 0.000017 1 0.0745
A1 B1 C1 1.56 –0.02 –0.02 –0.06 0.00015 0.00015 1 0.669
Replicates SSR SMR 0.00723 0.003615 2 16.138
error SSE SME 0.00157 0.000224 7
The correction term (C) is as follows:
C =
Sum of totals ( )
2
Number of effects ( ) Number of totals ( )
. (6.8)
The SST is as follows:
SST = Sum of each individual replication squared – C. (6.9)
The sum of squares treatment (SSTr) is as follows:
SSTr = [(Sum of each individual total squared)/Number of effects] – C. (6.10)
611
The sum of squares replication (SSR) is as follows:
SSR = [(Sum of vertical replication total squared)/Number of rows] – C. (6.11)
The sum of squares error (SSE) is as follows:
SSE = SST – SSTr – SSR. (6.12)
The sum of mean replicate (SMR) is as follows:
SMR = SSR/DOF. (6.13)
The sum of mean error (SME) is as follows:
SME = SSE/DOF. (6.14)
F
for a 0.05 confidence is 5.59, therefore effect C (vacuum) and replication have a significant
effect on the burn rate. (The third batch of propellant may have been different for another reason such as
contamination.) Note that since no values of F are greater than F for any conditions where two or more
effects are 1, then no interactions have a significant effect on burn rate. (For example, if the fourth line
had an F greater than F , then the interaction of mix time and mix speed would have a significant
interaction).
6.4.5 Advantages
Sources of variation can be found, random variation isolated, or any chosen source of variability
isolated. Also, interactions between variables can be isolated, and larger numbers of variables can be
solved for.
6.4.6 Limitations
The processes in factor analysis are more timeconsuming than the analysis of variance. A full
factorial analysis does not solve for exponential or polynomial effects. The fractional factorial analysis
does not solve for all effects and higher order effects separately.
6.5 Confidence/Reliability Determination and Analysis
6.5.1 Description
Confidence analysis compares sample values, means, or standard deviations with population
standard deviations to obtain a confidence interval, with a chosen significance.
6.5.2 Application
Confidence analysis is used to determine the interval of values that a data point could take, with a
chosen probability of being within that interval. Confidence analysis can be used with individual points,
means, standard deviations, regression lines, or reliability measurements such as mean time between
failures.
6.1
This technique is typically performed in phase C or E.
612
6.5.3 Procedures
As described in reference 6.1, the procedures for determining the confidence interval for the population
mean, given a sample mean, will be discussed here.
(1) Choose a confidence (α) level and obtain the α/2 term by dividing the confidence level by
2.
(2) Determine, from past experience (or by adjusting the sample standard deviation), the
population standard deviation.
(3) Obtain the z
(α/2)
value by looking up the z value for α/2 in a normal distribution table.
(4) The values for either end of the confidence interval is given by the equation:
Int = m
s
± z
(α/2)
* s
p
/n
1/2
(6.15)
where Int is the low or high confidence interval value, m
s
is the sample mean, s
p
is the population
standard deviation, and n is the sample size. For large n, the sample standard deviation can be used
instead of the population standard deviation.
The confidence interval for the population standard deviation, given the sample standard deviation, is
determined in the same way as above, except equation (6.16) is used.
Int ·
s
p
1 t z
( /2)
/ s
s
/ (2* n)
1/2
(6.16)
where s
s
is the sample standard deviation. For linear regression, the confidence for the equation of the
line is:
Int · (a + bx
o
) t t
/2
* s
e
*(1/ n + n(x
o
− m
s
)
2
/ S
xx
)
1/2
(6.17)
and for the y value:
Int · (a + bx
o
) t t
/2
* s
e
*(1+ 1/ n + n(x
o
− m
s
)
2
/ S
xx
)
1/2
(6.18)
where
se
2
·1/ (n − 2)Σ(yi − (a + bxi))
2
·
S
xx
*S
yy
−(S
xy
)
2
n(n−2)S
xx
where
S
xx
· n * Σx
i
2
− (Σx
i
)
2
, S
yy
· n * Σy
i
2
− (Σy
i
)
2
, and S
xy
· n * Σx
i
y
i
− Σx
i
( ) *( ΣY
i
) .
613
6.5.4 Example
Determine the confidence interval for insulation erosion at a station in the RSRM aft dome to
determine if the associated compliance safety factor (CSF) may actually fall below the 1.0 minimum
value, with a 95 percent confidence. The sample data for 18 flights (36 motors) is:
Erosion mean 1.112 in
Standard deviation 0.207 in (defined as known s
p
)
n 36
α/2 is (1–0.95)/2 = 0.025, therefore the z(
α/2
) term is 1.96. Entering the above values into equation
(6.15), the confidence interval is 1.112 ± 1.96 * 0.207/(36)
1/2
= from 1.042 to 1.182 for erosion.
The safety factor is then calculated using the maximum erosion value and is:
CSF = Min Ins t
Erosion+3s
p
+0.1
CSF = 3.36 = 1.766 .
1.182+3(0.207)+0.1
So, in this instance the confidence interval is used to calculate a safety value that can be compared to a
performance requirement.
6.5.5 Advantages
This analysis can give a realistic probability of whether or not a process may yield a value which
is above or below a requirement.
6.5.6 Limitations
A sample statistic must be known or assumed, such as the population standard deviation, before
an analysis can be performed.
6.6 Regression Analysis
6.6.1 Description
Regression analysis is a form of curve fitting to find a mathematical relationship for a group of
data. There are typically two types of regression: regression and multiple regression. Typical types of
relationships which are assumed for regression include linear (straight line), polynomial, and
exponential. A goodness of fit test is often performed to see how well the generated relationship fits the
data.
6.3
The method of least squares is probably the most frequently used method of regression. The
equation for the method of least squares is obtained by setting the derivative equal to zero of the
equation for the sum of the vertical distance from each y value to the mean y value.
614
6.6.2 Application
Regression, as described in reference 6.1, is typically used for three purposes:
(1) To find the mathematical relationship represented by a group of data points.
(2) To determine if the magnitude of a measurement is increasing or decreasing with time or
event.
Regression analysis is best applied in phase D but may also be applied in phase E. There are
several methods of regression. Multiple regression will be discussed in section 6.7. The least squares
method is a commonly used method of regression, and will be discussed here (assuming a straightline
relationship). The R
2
indicates the percent variation in the dependent variable that can be explained by
the independent variable.
6.6.3 Procedures
As described in reference 6.3, the use of the least squares method for finding the equation of a
line of the form
y = a + bx, (6.19)
is as follows:
(1) Determine the mean of the x
i
values ( x ) and y
i
values ( y ).
(2) Determine the deviation of each x
i
and y
i
value.
(3) Determine the slope of the trend line by dividing the summation of the multiple of the
deviations by the summation of the square of the x deviations (equation (6.19)).
b ·
Σ(x
i
− x )(y
i
− y )
Σ(x
i
− x )
2
(6.20)
(4) Determine the y intercept by subtracting the product of the slope and the mean x value from
the mean y value (equation (6.20)).
a = y – (b) x . (6.21)
The intercept and slope are used in equation (6.19) for a line representing the straightline
relationship. If the slope (b) is negative, then a decreasing trend may be indicated.
The explanatory power can be determined by R
2
as follows:
(1) Determine y values for each x value using the line generated above.
(2) Determine the deviation of each generated y value from the mean y.
615
(3) Obtain the R
2
value by dividing the sum of the square of the generated y deviations by the
sum of the square of the actual y deviations (equation (6.21)).
R
2
·
Σ(gen y
i
− y)
2
Σ(y
i
− y)
2
(6.22)
A good relationship is indicated by an R
2
value close to 1.
6.6.4 Example
As adapted from reference 6.3, assume the set of ordered pairs (1,4), (2,5), (3,6), (4,3), (5,5),
(6,5), (7,4), (8,6), (9,4), and (10,5). The following table shows summations, squares, and products that
go into the equations above:
x y (dx)
2
(dy)
2
(dx)(dy) y
g
dy
g
2
1 4 20.25 0.49 3.15 4.56 0.0196
2 5 12.25 0.09 –1.05 4.59 0.0121
3 6 6.25 1.69 –3.25 4.62 0.0064
4 3 2.25 2.89 2.55 4.65 0.0025
5 5 0.25 0.09 –0.15 4.68 0.0004
6 5 0.25 0.09 0.15 4.71 0.0001
7 4 2.25 0.49 –1.05 4.75 0.0025
8 6 6.25 1.69 3.25 4.78 0.0064
9 4 12.25 0.49 –2.45 4.81 0.0121
10 5 20.25 0.09 1.35 4.84 0.0196
summation 55 47 82.5 8.1 2.50 0.0817
where dx = xi – x , dy = yi – y , yg = generated points for each x, and dyg = yg – y . Using these data, the
mean x value is 5.5, the mean y value is 4.7, the slope (b) is 0.0303, and the y intercept (a) is 4.533. The
equation for the line is y = 0.0303(x) + 4.533. No significant relationship is indicated for this example,
R2 = 0.0101. Figure 61 shows the points and the generated line for this data.
10
9
8
7
6
5
4
3
2
1
0
1 3 4 5 6 7 8 9 10 0 2
generat ed line
Figure 61. Line generated with least squares method.
616
6.6.5 Advantages
A mathematical relationship can be determined, by hand or computer, when the relationship is
not obvious by inspection.
6.6.6 Limitations
If the data are discrete, e.g., integer data, the actual line generated will only approximate the
actual relationship.
6.7 Response Surface Methodology
6.7.1 Description
Response surface methodology is a method for surface fitting, much like regression is a method
for curve fitting. The surface can be a plane, using two independent variables and straightline
relationships, or it can be a more complex surface, using polynomial relationships. There are two
typically used methods for response surface analysis—multiple regression and factorial experimentation.
Factorial experimentation is discussed in section 6.4.
6.1
6.7.2 Application
Response surface analysis is typically used for the following purposes:
(1) To find the mathematical relationship represented by a group of data points.
(2) To optimize independent variables for maximum or minimum results.
This methodology is best performed in phase D or E.
6.7.3 Procedures
As described in reference 6.3, the least squares method of multiple regression, assuming a
straightline relationship, will be shown here. The basic form of the equation for a plane surface is y = a
+ b
1
x
1
+ b
2
x
2
+ b
3
x
3
+... + b
n
x
n
. This equation is minimized. After setting the derivative of the equation
for the sum of the vertical distances or ∑ (yi – (a + b
1
x
1
+ b
2
x
2
+ b
3
x
3
+... + b
n
x
n
))
2
to zero, the
equations for two independent variables are:
∑y = nb
0
+ b
1
* ∑x
1
+ b
2
* ∑x
2
,
∑(x
1
* y) = b
0
* ∑x
1
+ b
1
* ∑x
1
2
+ b
2
* ∑(x
1
* x
2
),
∑(x
2
* y) = b
0
* ∑x
2
+ b
1
* ∑(x
1
* x
2
) + b
2
* ∑x
2
2
. (6.23)
These equations are solved simultaneously for b
0
, b
1
, and b
2
.
617
Often, if the numbers are equally spaced, a set of numbers is coded. For example, the numbers
are substituted by assuming a smaller whole number for each original number. This practice makes
solving for the coefficients much easier with very little cost in accuracy.
6.7.4 Example
In the following hypothetical example, as adapted from reference 6.3, propellant was aged at
100°, 120°, and 140° for 1, 6, and 12 mo. Mean modulus of elasticity measurements are given for three
propellantaging temperatures and times. The columns for x
1
2
, x
2
2
, x
1
x
2
, x
1
y, and x
2
y and the bottom row
of summations are derived from the first two columns.
x
1
x
2
y x
1
2
x
2
2
x
1
x
2
x
1
y x
2
y
100 1 360 10,000 1 100 36,000 360
120 1 352 14,400 1 120 42,240 352
140 1 347 19,600 1 140 48,580 347
100 6 358 10,000 36 600 35,800 1,548
120 6 350 14,400 36 720 42,000 2,100
140 6 345 19,600 36 840 48,300 2,070
100 12 347 10,000 144 1,200 35,700 4,284
120 12 349 14,400 144 1,440 41,880 4,188
140 12 343 19,600 144 1,680 48,020 4,116
1,080 57 3,151 132,000 543 6,840 377,520 19,845
The equations for finding the constants are as follows:
From equation (6.23),
3,151 = 9b
0
+ b
1
1,080 + b
2
57
377,520 = b
0
1,080 + b
1
132,000 + b
2
6,840
19,845 = b
0
57 + b
1
6,840 + b
2
543
b
0
·
3,151 1, 080 57
37,7520 132, 000 6,840
19,845 6,840 543
9 1,080 57
1,080 132,000 6,840
57 6,840 543
b
1
and b
2
are calculated in the same manner. Solving the simultaneous equations (6.23), the constants are
b
0
= 383.98, b
1
= –0.25, and b
2
= –0.6117. Therefore the equation for modulus of elasticity for the
sample propellant is
y = 383.98 – 0.25 * x
1
– 0.6117 *
x
2
.
618
6.7.5 Advantages
A mathematical relationship can be determined, by hand or computer, when the relationship is not
obvious by inspection.
6.7.6 Limitations
If the data are discrete (e.g., integer data), the actual line generated will only approximate the actual
relationship.
619
REFERENCES
6.1 Miller, I. and Freund, J.E.: “Probability and Statistics for Engineers.” Second edition, Prentice
Hall, Inc., Englewood Cliffs, NJ 07632, 1977.
6.2 Hines, W.W. and Montgomery, D.C.: “Probability and Statistics in Engineering and Management
Science.” John Wiley and Sons, Inc., 1990.
6.3 NASA–STD–8070.5A, “Trend Analysis Techniques.” October 1990.
620
7. TOTAL QUALITY MANAGEMENT TOOLS
This section describes several TQM tools available to the system engineer analyst. TQM is
applied to continuously improve performance at all levels of operation, in all areas of an organization,
using all available human and capital resources. Improvement is addressed toward such areas as cost,
quality, market share, schedule and growth. TQM is an ongoing effort that demands commitment and
discipline.
A tool to assess an operation against other operations is the benchmarking technique which is
discussed in section 7.1. The cause and effect technique relates identified problems to their causes, and
this tool is discussed in section 7.2.
Concurrent engineering is more of an approach to quality management than a technique and it
is an interaction of disciplines during the design but before production. This approach is
discussed in section 7.3.
Three tools that attempt to improve the quality program are the cost of quality, design of
experiments (DOE), and evolutionary operation (EVOP). The cost of quality tracks a quality program
and attempts to identify ways to improve the program. This technique is discussed in section 7.4. Design
of experiments varies all possible combinations of factors and levels in an attempt to obtain the optimum
settings for a desired output. This technique is discussed in section 7.5. A methodology for improving
quality by looking at the production process is the evolutionary operation technique, and it is discussed
in section 7.6.
Group consensus techniques are often applied to solve problems. Three such tools are
brainstorming, Delphi, and nominal group technique (NGT). These techniques are discussed in sections
7.7, 7.9, and 7.10, respectively.
A methodology for collecting data quickly and easily in a simplified manner is the checklist
technique. This tool is discussed in section 7.8.
Another tool that might apply to the group consensus technique is the force field analysis. This
methodology counts the positive and negative forces, as well as their magnitudes, that effect the results
of a proposed solution or change in process. The force field analysis is discussed in section 7.11.
A methodology that is applied early in a design process is the quality function deployment
(QFD) technique which is discussed in section 7.12. This technique is used to solve problems before the
production phase begins and thus assists in the design of competitive products. By using a chart known
as the house of quality, priorities are given to the possible solutions as they relate to the identified
problems. Also, the product can be benchmarked against the competition in the areas of how well the
product stacks up against the competition as far as handling the identified problems, and how well the
product stacks up against the competition as far as meeting the appropriate engineering standards.
The final four tools that are discussed in this section are applied to improve a process. These
tools are quality loss function, SPC, flowchart analysis and work flow analysis (WFA). Quality loss
function, discussed in section 7.13, is a method of determining “loss to society” when a product is not at
the mean but is still within specification limits. SPC, discussed in section 7.14, is a process improvement
tool that helps identify problems quickly and accurately. The flowchart analysis, discussed in section
7.15, pictorially represents the steps of a process thus making it easier to eliminate nonvalued steps of
the process. Finally, the WFA, discussed in section 7.16, examines the work process for possible
improvements in performance and the quality of work life.
A summary of the advantages and limitations of each tool or methodology discussed in this
section is presented in table 71.
71
Table 71. TQM tools and methodologies—Continued
Tool or Methodology Section Advantages Limitations
Benchmarking 7.1 (1) Helps meet customer requirements.
(2) Helps establish goals and priorities.
(3) Helps determine true measures of productivity.
(4) Helps to attain and maintain a competitive
position.
(5) Helps identify and maintain awareness of industry
best practices.
(1) Must be continuous in order to keep up with the latest industry
changes.
(2) Determining industry “best practices” is difficult and often
subjective enough to be biased by the reviewing company’s
“wants” rather than the reviewing company’s customer’s wants.
Cause and effect diagrams 7.2 (1) Enables quality analysis groups to thoroughly
examine all possible causes or categories.
(2) Useful in analyzing statistical process control
(SPC) problems; SPC detects problems but can
poses no solutions.
Arriving at a group consensus is timeconsuming.
Concurrent engineering 7.3 (1) Shortens and makes more efficient the designto
development life cycle by employing the
interactions of functional disciplines by a cross
functional team.
(2) Reduces costs in the designtodevelopment life
cycle.
(1) The degree of success of this technique depends upon the degree
of cooperation between the multifunctional team members.
(2) Significant additional time, and associated funding, is required at
the front end of a program to perform the coordinated planning.
While time and money are saved overall within the effort, it is
often difficult to “frontload” large tasks.
(3) If design is pursued by projectized teams, the institutional
knowledge of the organization becomes very difficult to capture
or employ in the design decisions.
Cost of quality 7.4 (1) Helps to reveal and explain the more significant
costs.
(2) Activities and processes that need improvement can
be prioritized.
(3) Helps to reveal and explain the hidden costs of a
product or service.
(1) The technique itself can be expensive, thus making its goals of
saving/eliminating costs unachievable.
(2) Measurement for measurement's sake is an easy paradigm to fall
into; this technique is subject to misuse in this regard.
Design of experiments 7.5 The technique optimizes product and process design,
reduces costs, stabilizes production processes, and
desensitizes production variables.
(1) The performance of the analysis is time consuming and, the
results generally do not include parameter interactions.
Preknowledge of interaction significance is required to support
appropriate DOE technique selection.
(2) The DOE technique is often performed without a “verification
experiment” in which the predicted “optimized” parameters are
tested for performance (in agreement with the predictions). In,
addition a mistake is often made by taking the “best”
experiment’s parameters as an optimized set rather than an
interpolated set.
(3) Parameters must be interpolated from within the tested data set
rather than extrapolated beyond it.
72
Table 71. TQM tools and methodologies—Continued
Tool or Methodology Section Advantages Limitations
Evolutionary operation 7.6 (1) The cost is very low, so it can be run continuously.
(2) This technique increases a plant’s capacity and thus
profits will increase.
(3) The tool is simple and relatively straightforward.
EVOP is slow, so progress is slow.
Brainstorming 7.7 The technique takes advantage of the ideas of a group to
arrive at a quick consensus.
(1) The technique only proposes a solution but does not determine
one.
(2) The technique is limited by the ability of the group to achieve
consensus.
Checklists 7.8 (1) The tool is quick and easy to use.
(2) Checklists help to minimize errors and confusion.
Time must be taken to assemble a group to decide what data should
be collected.
Delphi technique 7.9 (1) Useful in eliminating personality clashes.
(2) Useful when powerful personalities are likely to
dominate the discussion.
(3) Inputs from experts unavailable for a single
meeting are included.
(1) Arriving at a group consensus is time consuming.
(2) Assembling the group participants is difficult/timeconsuming.
Nominal group technique 7.10 Very effective in producing many new ideas/solutions in
a short time.
(1) Assembling the group participants is difficult/timeconsuming.
(2) Divergence in weighting factors is common.
Force field analysis 7.11 Useful in determining which proposed solution, among
many, will meet the least resistance.
The technique is time consuming in arriving at a consensus on the
values (weights) of the forces, and is highly subjective.
Quality function deployment 7.12 (1) Helps organizations design more competitive,
higherquality, lowercost, products easier and
quicker.
(2) Helps ensure quality products and processes by
detecting and solving problems early.
(3) Engineering changes, design cycle, and startup
costs are reduced.
(4) Voice of the customer is heard.
(5) The technique is proactive, not reactive.
(6) Prevents problems from “falling through the crack.”
(7) The technique is costeffective.
(8) Easy to learn.
(1) Assembling the group participants is difficult/timeconsuming.
(2) The technique is not easy to perform.
73
Table 71. TQM tools and methodologies—Continued.
Tool or Methodology Section Advantages Limitations
Quality loss function 7.13 (1) Evaluates loss at earliest stage of product/process
development.
(2) Useful results obtained quickly and at low cost.
(1) It may be difficult to convince manufacturers to apply the
technique.
(2) It is often difficult to characterize the loss function.
Statistical process control 7.14 (1) This technique determines the cause of variation
based on a statistical analysis of the problem.
(2) The technique improves process performance.
(3) SPC helps identify problems quickly and
accurately.
SPC detects problems but poses no solutions.
Flowchart analysis 7.15 (1) Allows the examination of and understanding of
relationships in a process.
(2) Provides a stepbystep picture that creates a
common understanding about how the elements of
the process fit together.
(3) Comparing a flowchart to actual process activities
highlights areas where policies are unclear or are
being violated.
The development process is time consuming.
Work flow analysis 7.16 The technique increases productivity and improves
working conditions.
(1) The technique requires cooperation between employees and
management to be successful.
(2) The observed operation may not be fully representative of a
“typical” process that would occur without scrutiny.
74
7.1 Benchmarking
7.1.1 Description
Benchmarking, as described in reference 7.1, is a technique used to assess how an organization,
or process, is performing against internal guidelines, competitors, or even noncompetitors that may be
recognized as being superior. Benchmarking helps improve a process by recognizing priorities and
goals. The technique must be continuously applied in order to be effective because practices constantly
change (continuous improvement) affecting strategy. If the benchmarking process is performed once and
forgotten, then the operation may become inefficient by not keeping up with the latest industry best
practices.
7.1.2 Application
The benchmarking technique is typically performed in phase E but may also be performed in
phase A or B. This technique can be applied when it is desirable to know the strengths and weaknesses
of an organization’s own operation. These strengths and weaknesses can then be compared to internal
guidelines to evaluate the organization’s conformance to those guidelines.
Benchmarking can be applied to identify the strengths for products that directly compete with the
organization’s specific product under consideration. The manufacturers of those competing products are
probably using the same benchmarking technique to evaluate the competitors for their product. Once the
strengths and weaknesses of competing products are known, the company can attempt to differentiate
their capabilities in the marketplace.
By accomplishing this analysis, an organization can also incorporate the strengths of their
competitors that exist in certain areas.
7.1.3 Procedures
As adapted from reference 7.3, the basic elements of benchmarking include the following:
(1) Decide which process(es) or product(s) to benchmark.
(2) Determine the criteria to benchmark, i.e., benchmark internally against established
guidelines, benchmark against competitors, or benchmark against noncompetitors that are
considered industry leaders.
(3) Choose the particular characteristics of the operation or product to benchmark.
(4) Collect data on the processes or products that are being benchmarked.
(5) Analyze the data, prepare an action plan, and implement the plan.
(6) Assess the results of all the changes.
(7) Repeat the benchmarking technique, as necessary, in order to stay uptodate with the
applicable operation.
75
7.1.4 Example
The following illustration, adapted from reference 7.3, shows an example of comparative bench
marking between one company’s process and five competitors on a scale of 1 (worse) to 10 (better).
Better
Worse
10
9
8
7
6
5
4
3
2
1
Process
Competitor 2
Competitor 3
Competitor 1
Competitor 5
Competitor 4
Organization
Figure 71. Comparative benchmarking.
This illustration reveals that this company needs to look closely at the operations of competitors
2 and 3 and consider implementing into their process any strengths that are discovered. This company
should also look at those competitors rated lower on the scale and identify their weaknesses and ensure
that those weaknesses do not exist in their operation.
7.1.5 Advantages
(1) Benchmarking helps meet customer requirements.
(2) Benchmarking helps establish goals and priorities.
(3) Benchmarking helps determine true measures of productivity.
(4) Benchmarking helps to attain and maintain a competitive position.
(5) Benchmarking helps identify and maintain awareness of industry’s best practices.
7.1.6 Limitations
(1) The benchmarking process must be continuous in order to keep up with the latest industry
changes.
(2) Determining industry “best practices” is often difficult and subjective. The reviewing
company may well bias their results based on company “wants” rather than customer
“wants.”
76
7.2 Cause and Effect Diagrams (Also Known as Fishbone Diagrams
or Ishakawa Diagrams)
7.2.1 Description
The cause and effect diagram, as described in reference 7.3, graphically represents the
relationships between a problem (effect) and its possible causes. The development process is started in a
group session led by a trained facilitator. The problem is stated in terms acceptable to the group.
Possible causes are listed. The group then assigns priorities to the causes and action plans are developed.
When a cause and effect diagram is constructed, thinking is stimulated, thoughts are organized,
and discussions are begun. These discussions bring out many possible viewpoints on the subject. Once
all participants reach a similar level of understanding about an issue, an expansion of ideas can then be
examined.
Cause and effect diagrams are developed in a form, commonly referred to as “fish,” where the
effect is found in a box to the right which is the head of the fish. The bones of the fish show the
organized causes. The effects and causes can be expressed in words or data.
7.2.2 Application
As adapted from reference 7.3, cause and effect diagrams are used to examine many different
topics which include the following:
(1) The relationships between a known problem and the factors that might affect it.
(2) A desired future outcome and its related factors.
(3) Any event past, present, or future and its causal factors.
The cause and effect diagram is useful in examining processes such as SPC, SPC problems, (sec.
7.14) problems. The cause and effect diagram technique is best applied in phase E but may also be
applied in phase A or B. The technique is also useful in planning activities and brainstorming. The
diagram is basically a controlled way of gathering and using suggestions through group consensus.
7.2.3 Procedures
A cause and effect diagram, as adapted from reference 7.3, is developed in the following manner:
(1) Define the effect as clearly as is possible and place it at the head of the fish. This effect
represents the “problem” that is being investigated. As data are collected, the effect can be
redefined, if necessary.
(2) The group brainstorms the causes and lists them in no particular order. These causes are
then studied and the causes that affect these causes are identified. This will continue until
no new causes are thought of by the group.
(3) Once all causes are identified, list all categories, then display the categories on the diagram.
(4) The group then prioritizes the causes by multivoting. Each member of the group lists the
causes in order of significance. Votes are counted and a final list is written.
77
(5) The highest prioritized causes are listed on the diagram as the big bones. The next highest
prioritized causes will be listed on the diagram as the medium bones. Finally, the least
prioritized causes will be listed on the diagram as the small bones.
(6) As categories and causes are included on the diagram, thinking may be stimulated and new
causes may be identified.
(7) Teams are then formed to research and report on preventive (i.e., proactive) measures.
7.2.4 Examples
Example 1:
Assume the problem is design rework (fig. 72). The group fills in the probable root causes
through “brainstorming” ideas (sec. 7.7). When complete, the group prioritizes the causes using
multivoting. This is a technique where each person lists the causes in order of significance. Votes are
counted and a final list is written. Teams are formed to research and report on preventive measures. In
conclusion, a team has put their thoughts in writing and arrived at a consensus.
CHANGES SCHEDULE
No Second Shift
Under
Staffed
Lacking
Skills
Working
Outside
Discipline
INTERFACES
SKILL
SPECS
Discontented
Loanee
Doesn't Know
System
ByPassed
Outdated
Equipment
GRAPHICS
Poor
Notification
Try It Now,
Change Later
Tracking
Poor
Not Clear
Requires
Backtracking
Responsibility
Not Defined
Out of Sequence
Rushed
Late Start
Excessive
Meetings
Conflict
Not Clear
Cost Rules
REWORK
DESIGN
Figure 72. Design rework cause and effect diagram.
Example 2:
Figure 73 illustrates the resulting cause and effect diagram after the brainstorming session on
identifying problems in receiving telephone messages. The brainstorming effort for this problem is
covered in section 7.7.4.
7.2.5 Advantages
(1) The cause and effect diagram enables quality analysis groups to thoroughly examine all
possible causes or categories.
(2) The cause and effect diagram is useful in analyzing SPC problems. SPC detects a problem
but can pose no solution.
78
79
ENVIRONMENT
TRAINING METHOD
HUMAN ERROR HARDWARE
Message light not turned on
Criticality of message not identified (no guidelines)
Procedures
Messages are
notdelivered in a
timely manner
Message Taker responsibilities
Phone System Options
Inability to take long detailed message
No standard guidelines
for message takers
long detailed messages
Inadequate message delivery system
Employee
Unaware
ofmessage
No feedback of message
delivered
call pickup
No guideline for phone
system setup
Info not available to call recipient
Recipient doesn't know how
to obtain info employee
whereabouts
No guidelines for message takers
call transfer
call coverage
Employee forgets to sign out
Forget to undo call forward
Call recipient does not deliver message
distribution message misplaced
Lack of equipment to take long
detailed/technical messages
Not enough phones
Not enough trunk
lines
Lack of
interactiveautomated
directions to caller
Employee does not see light
Too small Poor contrast
Wrong message taken  incomplete message
rude caller
distractions
Employee fails to look at light
No identified
point of contact
No method to
reachemployee
notaccessible while off
i
Untimely delivery of message
Peak Activity
Number of calls
Figure 73. Cause and effect diagram on receiving telephone messages.
7.2.6 Limitations
The development of the cause and effect diagram can be timeconsuming in order to arrive at a
group consensus.
7.2.7 Bibliography
Kume, H.: “Statistical Methods for Quality Improvement.” The Association for Overseas Technical
Scholarships, 1985.
7.3 Concurrent Engineering
7.3.1 Description
Concurrent engineering is the interaction of technical disciplines during the design phase to
produce a robust design prior to production. This process is more of an engineering approach to quality
management than a technique.
7.1
The approach attempts to link and integrate, from the outset, all
elements of the product life cycle from conception through disposal.
Traditionally, quality, and producibility do not review an element until after the design has been
completed. Concurrent engineering, as described in reference 7.3, focuses on both the product and the
process simultaneously. One method of achieving this approach is by forming multifunction teams
consisting of engineers from several departments. This way, each department will follow the complete
process simultaneously rather than one department examining the design and then passing it on to the
next department and so on.
7.4
The concurrent engineering approach has been known for many years although its use is just
receiving widespread application in the United States.
7.5
7.3.2 Application
Because the concurrent engineering approach is used to address the product and process simulta
neously early in the design phase, it generally will save time and money. Through this technique, the
team will establish design goals as well as perform tradeoff analyses using such tools as QFD (sec.
7.12) and DOE (sec. 7.5). This technique is typically performed in phase C but may also be performed in
phase B.
7.3.3 Procedures
The basic elements involved in applying concurrent engineering include the following, as
adapted from reference 7.3:
(1) Establish multifunction teams which include members from design, quality, safety,
marketing, manufacturing, support, etc.
(2) Select and use design parameters that will help identify and reduce variability in the
production process.
(3) Use such techniques as DOE, QFD, computeraided design, robust design, group
technology, and value analysis to extend the traditional design approach.
710
7.3.4 Example
Figure 74 illustrates an example of how concurrent engineering is applied. By using
multifunctional teams, all phases of a product’s life cycle are simultaneously examined, thus making the
design process more efficient in terms of both cost and schedule.
INPUT
Cust omer
Needs
Development
Maint ainabilit y
Rel i abi l i t y
Saf et y
Verif icat ion
Logist ics
Manuf act uring
Training
Deployment
Operat ions
Support
Disposal
MULTIFUNCTIONAL
TEAMS
Sample Techniques
Cause and Ef f ect Diagram
( Sect ion 7.2)
Design of Experiment s, DOE
( Sect ion 7.5)
Brainst orming
( Sect ion 7.7)
Delphi Technique ( Sect ion 7.9)
Nominal Group Technique, NGT ( Sect ion 7.10)
Force Field Analysis ( Sect ion 7.11)
Qualit y Funct ion Deployment , QFD( Sect ion 7.12)
St at ist ical Process Cont rol, SPC ( Sect ion 7.14)
OUTPUT
BALANCED
PRODUCT
LIFE
CYCLE
Figure 74. Concurrent engineering example.
711
7.3.5 Advantages
(1) The concurrent engineering approach can be used to shorten and make more efficient the
designtodevelopment life cycle by employing the interactions of functional disciplines by
a crossfunctional team.
(2) The approach can also be applied to reduce costs in the designtodevelopment life cycle.
7.3.6 Limitations
(1) The degree of success of this technique depends upon the degree of cooperation between
the multifunctional team members.
(2) Significant additional time, and associated funding, is required at the front end of a
program to perform the coordinated planning. While time and money are saved overall
within the effort, it is often difficult to “frontload” large tasks.
(3) If design is pursued by projectized teams, the institutional knowledge of the organization
becomes very difficult to capture or employ in the design decisions.
7.4 Cost of Quality
7.4.1 Description
As described in reference 7.3, the cost of quality technique tracks the expense and benefit of a
quality program. This technique can identify the unwanted cost of not doing the job right the first time as
well as the cost of improving the job.
Cost of quality includes all of the costs associated with maintaining an acceptable quality program,
as well as the costs incurred as a result of failure to reach the acceptable quality level. This technique
allows the analyst to identify costs that are often hidden. Costs will not be reduced by merely tracking the
cost of quality but the technique may point out areas where a greater return on investment could be made.
7.4.2 Application
The cost of quality technique is best applied in phase E. This technique is applied to understand
the hidden costs of a product or service and to reduce or eliminate these costs. This technique can
identify the most significant costs and thus make it possible to prioritize the activities and/or processes
that may need improvement.
7.4.3 Procedures
The cost of quality technique is applied in the following manner:
(1) Collect cost data for the following categories;
a. Internal failure (IF) costs
b. External failure (EF) costs
c. Appraisal (A) costs
d. Prevention (P) costs
712
(2) Data are trended periodically on the standard cost of quality curve shown in figure 75:
TIME
$
A, P
IF, EF
Figure 75. Standard cost of quality curve.
As appraisal (reactive) and prevention efforts increase, failures decrease. A significant prevention effort
resulting in decreased failure warrants a decrease in appraisal (i.e., audits, inspections).
Prevention is the key. Concurrent engineering (sec. 7.3) helps achieve prevention. In some companies,
the suggestion system and/or savings shown in process improvement measures are considered
prevention.
Cost of quality programs requires a crossfunctional, interdepartment team to agree on what constitutes a
cost. Programs normally consist of three phases:
(1) Initiation.
(2) Development.
(3) Solidified gains.
Failures are indirectly proportional to the appraisals/preventions. As failures decrease, manpower
(reactive) should be decreased. Prevention costs run 2 percent or less of sales as a national average.
There are indications that, to optimize costbenefit relationships, it should be 10 percent. As the program
progresses, prevention costs (proactive) should increase.
Collection of data can be on a ROM basis and need not involve finances. Be careful not to create a
system and become so enamored with the system that the objective of savings is obscured.
Once data are collected and analyzed, they should be compared to a base. Examples are:
(1) Manhours per drawing.
(2) Direct cost per hour.
(3) Drawings per month.
713
7.4.4 Example
An example of a cost of quality data summary for a month is shown in table 72.
Table 72. Month’s cost of quality.
Cost Subj ect P A IF EF
32,000 Dr awi ng Er r or s X
2,000 Tr ai ni ng X
78,000
Erroneous
X
Inf or mat i on
18,000 War r ant y Claims X
10,000 Inspect ion/ Audit s
X
140,000 2,000 10,000 110,000 18,000
($)
The percentage breakdown is:
Prevention = 2,000/140,000 = 1.43 percent
Appraisal = 10,000/140,000 = 7.14 percent
Internal failure = 110,000/140,000 = 78.57 percent
External failure = 18,000/140,000 = 12.86 percent
100 percent
The total failure cost is $128,000 with only $2,000 spent on prevention. This example is 98.57
percent reactive and only 1.43 percent proactive.
7.4.5 Advantages
The following advantages were adapted from reference 7.6:
(1) The cost of quality technique helps to reveal and explain the more significant costs.
(2) Because of increased demands for time, energy, and money, it is helpful to develop a
quality technique whereby activities and processes that need improvement can be
prioritized. The cost of quality technique will accomplish this.
(3) The technique helps to reveal and explain the hidden costs of a product or service.
714
7.4.6 Limitations
(1) If not done as part of an overall plan, the cost of quality technique can be expensive, thus
making the goals of saving/eliminating costs unachievable.
(2) Measurement for measurement’s sake is an easy paradigm to fall into. This technique is
subject to misuse in this regard.
7.5 Design of Experiments
7.5.1 Description
The DOE technique is a control method of selecting factors, and levels of factors, in a predeter
mined way and varying possible combinations of these factors and levels. Quantitative results are
analyzed to show interactions and optimum settings of factors/levels to produce a desired output.
This technique may make the designtoproduction transition more efficient by optimizing the
product and process design, reducing costs, stabilizing production processes, and desensitizing
production variables.
7.3
7.5.2 Application
The design of experiments technique is typically performed in phase C but may also be
performed in phase D. This technique is used to achieve a robust design as an alternative to
experimenting in the production mode after the design has been completed. As described in reference
7.3, the following are among the applications for the DOE analysis:
(1) Compare two machines or methodologies.
(2) Examine the relative effects of various process variables.
(3) Determine the optimum values for process variables.
(4) Investigate errors in measurement systems.
(5) Determine design tolerances.
7.5.3 Procedures
As described in reference 7.3, the DOE technique is implemented as follows:
(1) Determine all of the pertinent variables whether they be product or process parameters,
material or components from suppliers, or environmental or measuring equipment factors.
(2) Separate the important variables which typically number no more than four.
(3) Reduce the variation on the important variables (including the control of interaction effects)
through redesign, close tolerance design, supplier process improvement, etc.
(4) Increase the tolerances on the less important variables to reduce costs.
715
7.5.4 Example
Data (yield in pounds) were recorded in table 73. For example, when A was at the low (A
1
)
level (10 °F), B was at the high (B
2
) level (60 psi), and C was at the low (C
1
) level (30 GPM), yield was
2.1 lbs.
Table 73. 2
3
factorial design data.
A
1
A
2
B
1
B
2
B
1
B
2
C
1
(1)
8.0
(7)
2.1
(6)
8.4
(4)
2.8
C
2
(5)
9.9
(3)
3.2
(2)
8.8
(8)
3.0
Numbers in parenthesis are standard cell designators. Normally four readings are averaged (e.g., 8.0 at
A
1
, B
1
, and C
1
, is an average of four data).
The orthogonal array is shown in table 74 along with the result of table 73. This array is used as
a “run recipe” in the actual conduct of the experiment. For example, all factors (A, B, C) are set at their
low level during trial 1.
Table 74. Trial, effects, and results.
Trial Main Effects SecondOrder Effects ThirdOrder
Effects
Results
A B C AB AC BC ABC
1 – – – + + + – 8.0
2 + – – – – + + 8.4
3 – + – – + – + 2.1
4 + + – + – – – 2.8
5 – – + + – – + 9.9
6 + – + – + – – 8.8
7 – + + – – + – 3.2
8 + + + + + + + 3.0
An example of the average of first order or main effects is shown using A
1
data and cells 1, 3, 5,
7; thus:
A
1 effects
=
8.0 + 3.2 + 9.9 + 2.1
4
· 5.80.
An example of a second order interaction (e.g., AB) is calculated by averaging data in the cells
where A and B are at like (L) levels and unlike (U) levels. They are:
AB
L
= cells 1,5, 4,8 ·
8.0 + 9.9 + 2.8 + 3.0
4
· 5.93.
716
AB
U
· cells 7,3,6,2 · 2.1+ 3.2 + 8.4 + 8.8
4
· 5.63.
An example of the third order interaction (i.e., ABC) is calculated using cell data where the
sum of the ABC subscripts are odd (O), then even (E). They are:
In cell #1, the factor levels are: A’s level is 1, B’s level is 1, and C’s level is 1. Therefore,
1+1+1 = 3, which is an odd number. The four cells having odd sums of levels are 1, 2, 3, 4.
In cell #5, the factor levels are: A’s level is 1, B’s level is 1, and C’s level is 2. Therefore,
1+1+2 = 4, which is an even number. The four cells having even sums of levels are 5, 6, 7, 8.
The calculations for all factors/levels are shown in table 75.
Table 75. Calculation of effects.
Summation Cells Computation Effect
A
1
1, 3, 5, 7 (8.0+3.2+9.9+2.1)/4 5.80
A
2
2, 4, 6, 8 (8.8+2.8+8.4+3.0)/4 5.75
B
1
1, 2, 5, 6 (8.0+8.8+9.9+8.4)/4 8.78
B
2
3, 4, 7, 8 (3.2+2.8+2.1+3.0)/4 2.78
C
1
1, 4, 6, 7 (8.0+2.8+8.4+2.1)/4 5.33
C
2
2, 3, 5, 8 (8.8+3.2+9.9+3.0)/4 6.23
AB
L
1, 4, 5, 8 (8.0+9.9+2.0+3.0)/4 5.725
AB
U
2, 3, 6, 7 (8.8+3.2+8.4+2.1)/4 5.63
AC
L
1, 2, 7, 8 (8.0+8.8+2.1+3.0)/4 5.48
AC
U
3, 4, 5, 6 (3.2+2.8+9.9+8.4)/4 6.08
BC
L
1, 3, 6, 8 (8.0+3.2+8.4+3.0)/4 5.65
BC
U
2, 4, 5, 7 (8.8+2.8+9.9+2.1)/4 5.90
ABC
O
1, 2, 3, 4 (8.0+8.8+3.2+2.8)/4 5.70
ABC
E
5, 6, 7, 8 (9.9+8.4+2.1+3.0)/4 5.85
Steps:
(1) Find C
Avg
:
This is the overall average of all data in all cells or,
C
Avg
=
8+9.9+2.1+3.2+8.4+8.8+2.8+3
8
· 5.78.
(2) Find an estimate of σ
c
;
Estimated σ
c
= (C
Avg)
1/2
/(4)
1/2
= (5.78)
1/2
/2 = 1.202.
717
(3) Ott
7.7
uses upper decision lines (UDL) and lower decision lines (LDL) instead of 3σ control
limits. The reason is that a decision of significant effects must be made when the plotted
data are beyond these lines. Ott also has a table called “exact factors for oneway analysis
of means, H
α
twosided.” H
.05
is found in the table. Then calculate the 95 percent UDL and
LDL, where α = .05, as follows:
UDL = C
Avg
+H
.05
(Estimated
c
) = 5.78+(1.39×1.188) = 7.43
LDL = C
Avg
–H
.05
(Estimated
c
) = 5.78–(1.39×1.188) = 4.13.
(4) The data from table 75, C
Avg
, UDL, and LDL are graphed in figure 76.
Yield
(pounds)
Effect
1
2
3
4
5
6
7
8
9
10
A
1
A
2
B
1
B
2
C
1
C
2
A
L
B A B A C
L U
A C B C
L
B C
U o
A B C ABC
E U
UDL=7.43
LDL=4.13
5.80
5.75
8.78
2.78
5.33
6.23
5.72
5.63
5.48
6.08
5.65
5.90
5.70
5.85
Figure 76. Factor/level effects graph.
Conclusion:
The main effect of B is very significant. Going from the high to the low level decreased yield 5
lbs. Raise B from 20 to 40 psi and run another experiment.
7.5.5 Advantages
This technique makes the designtoproduction transition more efficient by optimizing the product and
process design, reducing costs, stabilizing production processes, and desensitizing production
variables.
7.3
7.5.6 Limitations
(1) The performance of the analysis is timeconsuming and, if less than full factorial arrays are
employed, the results will not include all parametric interactions. Preknowledge of
interaction significance is required to support appropriate DOE technique selection.
718
(2) The DOE technique is often performed without a “verification experiment,” in which the
predicted “optimized” parameters are tested for performance (in agreement with the
predictions). In addition, a mistake is often made by taking the “best” experiment’s
parameters as an optimized set rather than an interpolated set.
(3) In order to perform the analysis, parameters must be interpolated from within the tested
data set rather than extrapolated beyond it.
7.5.7 Bibliography
Bhole, K.R.: “World Class Quality.” American Management Association, 1991.
7.6 Evolutionary Operation
7.6.1 Description
The EVOP technique is based on the idea that the production process reveals information on how
to improve the quality of a process. The technique has a minimal disruption to a process and creates
variation to produce data for analysis. Optimum control factor settings are identified for desired results.
Small, planned changes in the operating conditions are made and the results are analyzed. When
a direction for improvement is identified, process modifications can be made. The changes can continue
to be made until the rate of finding improvements decreases and then the changes can be applied to
different operating variables to identify more directions for improvement.
7.8
7.6.2 Application
The EVOP technique is best performed in phase E but may also be performed in phase D. This
technique is applied to reveal ways to improve a process. An experiment may use two or more control
factors (i.e., psi and degrees F are set) that produce a response (yield) known as response surface
methodology (RSM) (sec. 6.7). The question that may be asked is, “What are the degrees F and psi
settings that will produce maximum yield (pounds per batch)?”
Evolutionary operation works well with the SPC technique (sec. 7.14) in that SPC will monitor a
process and EVOP will reveal ways to improve the process.
7.6.3 Procedures
The EVOP technique is applied in the following manner:
(1) Choose two or three variables that are likely to affect quality.
(2) Make small changes to these variables according to a predetermined plan.
(3) Analyze the results and identify directions for improvement.
(4) Repeat until optimal conditions are found.
(5) The technique can then be applied to different variables.
719
7.6.4 Example
100
200
300
400
(Factor B)
F
20 30 40 50
60
PSI
(Factor A)
1 (B) 3 (D)
(Yield)
4 (E) 2 (C)
0 (A)
8 6
5 7
Cycle
#2
Cycle
#1
0
Figure 77. EVOP example.
Cycle No 1:
Per figure 77 above, select a reference point “0” (center of the box). The aim is to choose the psi
and degrees F that yield maximum output (body of the graph). Output (yield) can be volume, length, etc.
Corner No. 2 was maximum. Cycle No. 2 uses that corner as the reference point for the second box
(cycle). Actually, this is a simple 2
2
factorial experiment where the low and high levels of two factors,
i.e., degrees F and PSI were selected. Data for this example are shown in table 76.
Table 76. EVOP cycle No. 1 data.
RUN TIME ( A) TEMPERATURE ( B) POSITION YIELD
1
2
3
4

+

+


+
+
1
3
4
2
0
20
30
40
50
10
Legend:
" " = Low Level " +" = High Level
720
Main effects are calculated for A and B and second order interaction AB as follows:
A
EFFECT
(∑ High Levels–∑ Low Levels)/2
= [(30+50)–(20+40)]/2 = 10
B
EFFECT
= (∑ High Levels–∑ Low Levels)/2
= [(40+50)–(20+30)]/2 = 20
AB
INTERACTION
= (yield when A and B have like signs – yield when A
and B have unlike signs)/2
= [(20+50)–(30+40)]/2 = 0.
The change in mean (CIM) and 2 standard error (S.E.) cannot be calculated until two cycles are
complete. The S.E. is really a variation and encompasses 95percent confidence within the normal curve.
The 95percent is symmetrical with a 5percent level of significance, or a left and right tail of 2
1
/2
percent each. The CIM tells when a minimum or maximum occurs by comparing the results of the four
box corners to the reference point.
Cycle No. 2:
Corner No. 2 produced a maximum yield (i.e., 50) and becomes the new reference point. New
data were recorded as shown in table 77.
Table 77. EVOP cycle No. 2 data.
RUN TIME ( A) TEMPERATURE ( B) POSITION YIELD
1
2
3
4

+

+


+
+
5
7
8
6
0
26
32
38
48
18
Now, compare cycles (table 78).
721
Table 78. Comparison of EVOP cycle No. 1 and cycle No. 2 data.
CORNER SUBJECT
YIELD AT POSITION
0 6 7 8
A
B
C
D
E
F
Sum From Cycle No.1
Average From Cycle No.1
New Yield Dat a
B  C
New Sum = B + C
New Average = E/ n
10 20 30 40 50
10 20 30 40 50
18 26 32 38 48
 8  6  2 2 2
28 46 62 78 98
14 23 31 39 49
5
The new averages are used to calculate results. The levels of factors are determined by
examining the cycle No. 2 box of figure 77. For example, when A is at the high level, use corners 6 and
7. When A is high and B is low, use corner 7, etc.
A
EFFECT
= [(31+39)–(23+39)]/2 =4
B
EFFECT
= [(31+49)–(23+39)]/2 = 9
AB
INTERACTION
= [(23+31)–(39+49)]/2 = –17.
The CIM is calculated by multiplying the reference point data by 4 (now representative of four
corners) and letting the product be a sample, i.e., n = 1. The product is subtracted from the sum of the
four corners and divided by 5 (i.e., four corners are n = 4 + the reference point of n = 1):
23+31+39+49 = 142
4×14 = 56
86/5 = 17.2
The standard deviation and 2 S.E. when n = 2 are calculated using standard factors developed by
Box and Hunter.
7.9
They are K = 0.3, L = 1.41, and M = 1.26.
For the sample standard deviation:
s = K (corner “d” range)
s = 0.3 (–8 to +2) = 3
for 2 S.E. For new averages/effects:
L(s) = 1.41×3 = 4.23.
722
For CIM
M(s) = 1.26×3 = 3.78.
Results:
Psi limits are 4 t 4.23 = –0.23, 8.23
Temperature limits are 9 t 4.23 = 4.77, 13.23.
Conclusion:
Since the AB interaction = –17, there is a significant impact on the maximum yield. The psi can
be negative, positive, or nil. The temperature is borderline, but it should increase yield if it is decreased.
Select corner No. 7 and run a third cycle.
7.6.5 Advantages
The following advantages are adapted from reference 7.8:
(1) The cost of running EVOP is very low so it can be run continuously.
(2) EVOP will increase a plant’s capacity and thus profits will also increase.
(3) EVOP is simple and relatively straightforward.
7.6.6 Limitations
As described in reference 7.8, EVOP is slow, so progress is slow. If quick improvements are
needed, then this technique is inappropriate.
7.7 Brainstorming
7.7.1 Description
Brainstorming, as described in reference 7.3, is a group process wherein individuals quickly
generate ideas on a particular problem, free from criticism. The emphasis is on the quantity of ideas, not
the quality. In the end, the goal is to arrive at a proposed solution by group consensus. All members of
the group are equals and each is free to express ideas openly. The technique is an excellent way of
bringing out the creative thinking from a group.
7.7.2 Application
Brainstorming, as described in reference 7.1, is often used in business for such things as arriving at
compromises during union negotiations, coming up with advertising slogans, identifying root causes of a
problem, and finding solutions to a customer service problem.
If done properly, bashful yet creative people can be coaxed to propose good ideas. For some important
brainstorming sessions, a facilitator is necessary. The facilitator should be knowledgeable in the
brainstorming process and help as much as possible in the generation of ideas but should have no stake
in the outcome of the brainstorming session. This technique is typically performed in phase A but may
also be performed in phase C.
723
There are three phases of brainstorming, as adapted from reference 7.3:
(1) Generation phase—group members generate a list of ideas.
(2) Clarification phase—the group reviews the list of ideas to make sure all members
understand each one, discussions occur.
(3) Evaluation phase—the group eliminates duplication, irrelevancies, or issues that are off
limits.
7.7.3 Procedures
As described in reference 7.3, conduct a brainstorming session as follows:
(1) Clearly state the purpose of the brainstorming session.
(2) Group members can take turns expressing ideas, or a spontaneous discussion can occur.
(3) Discuss one topic at a time.
(4) Do not criticize ideas.
(5) Expand on ideas from others.
(6) Make the entire list of ideas available for all group members to review.
(7) After discussions and eliminations, arrive at a final proposed solution by group consensus.
7.7.4 Example
A group was assembled to brainstorm the causes for telephone messages not being received in a
timely manner. Each group member was given an opportunity to express ideas on the subject. A
spontaneous discussion developed, with some group members expanding on the ideas of others. The
following is a list of possible causes for the telephone message problem as a result of the brainstorming
session:
(1) Employee not at desk
(2) Secretary not available
(3) Volume of calls inhouse
(4) Too many incoming calls to receptionist
(5) Employee misses message
(6) Employee doesn’t see light or message
(7) Incomplete message taking
(8) Message mishandled
(9) Nonstandard message delivery system
(10) Employee offsite
(11) Criticality of message not identified
724
(12) Phone grouping not identified
(13) Whereabouts of employee unknown by call recipient
(14) Not utilizing available resources
(15) Caller leaves no message
(16) Message light not turned on
(17) Inadequate phone system
(18) No feedback of message delivered
(19) Lack of procedures
(20) No identified points of contact
(21) No answering machines
(22) Complicated phone system
(23) Forgetting to undo callforwarding
(24) People do not know how to use phone options
(25) Secretary does not deliver messages
(26) Secretary not in loop
(27) Cannot find known message in loop
(28) Wrong message taken
(29) Untimely delivery of message
(30) No guidelines for message taking
(31) Not enough phones
(32) Not enough trunk lines
(33) Volume of calls
(34) Congestion at receptionist’s desk
(35) Discontinuity at receptionist’s desk
(36) No beepers.
Following the brainstorming session for the causes of the problem, a cause and effect diagram
was developed as shown in section 7.2.4, example 2. Once this was completed and more discussions
were held, a proposed solution to the problem was presented.
7.7.5 Advantages
The technique takes advantage of the ideas of a group to arrive at a quick consensus.
7.7.6 Limitations
(1) Brainstorming only proposes a solution but does not determine one.
(2) The technique is limited by the ability of the group to achieve consensus.
725
7.8 Checklists
7.8.1 Description
A checklist, as described in reference 7.3, provides a list of checkoff items that enable data to be
collected quickly and easily in a simplified manner. The data are entered on a clear, orderly form. Proper
use of the checklist helps to minimize errors and confusion.
7.8.2 Application
Checklists should be laid out in advance or data may be omitted. If done right, the checklist will
be easy to complete and will allow for quick entry of data. One common method of data entry on a
checklist is hash marking.
Checklists are often used to collect data on such things as numbers of defective items, defect
locations, and defect causes. This technique is best applied in phase E but may also be applied in phase
A or B.
7.8.3 Procedures
As adapted from reference 7.3, a checklist is created in the following manner:
(1) A group should decide ahead of time what data should be collected.
(2) Make a draft of the checklist and ask the individuals who will fill out the form for input—
revise as necessary.
(3) Implement the checklist.
(4) As data are collected, review the results and, again, revise the checklist, as necessary, to
optimize use of the form.
7.8.4 Example
Table 79 illustrates a sample of the results of postflight hardware inspections for an imaginary
SRM. The listed defects occurred on the corresponding motor where checked.
7.8.5 Advantages
(1) The checklist is quick and easy to use.
(2) Checklists help to minimize errors and confusion.
7.8.6 Limitations
Time must be taken to assemble a group to decide what data should be collected.
726
Table 79. Motor postflight checklist.
Outer Igniter Joint
Discoloration
Aft Edge GEI
Insulation Chips
Water Under
Moisture Seal
Polysulfide
Porosity
Wet Soot on
Rubber
Edge
Insulation Exposure
Inhibitor Erosion
Defect
Description
Motor Number
01 02 03 04 05 06 07 08 09 10
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
7.9 Delphi Technique
7.9.1 Description
The Delphi technique, as described in reference 7.1, is an iterative process that results in a consensus by
a group of experts. The subject is presented to the experts. Without discussing the subject among
themselves, the experts send their comments to a facilitator. The facilitator reviews the comments and
eliminates those not applicable to the subject. Then, the comments are redistributed to the experts for
further review. This iteration is repeated until a consensus is reached.
7.9.2 Application
The Delphi technique is best performed in phase A or B but may also be performed in phase E. This
technique is a useful tool for finding a solution when personality differences exist between members of
involved technical areas. A group of experts can examine the problem and, through consensus, the
effects of the differences can be minimized. Another application for this technique is to allow all parties
to have equal input when one personality may otherwise overpower another in a discussion.
7.9.3 Procedures
As adapted from reference 7.1, the Delphi technique is applied in the following manner:
(1) Define the subject upon which the experts are to comment.
(2) Assemble a monitor group to determine task objectives, develop questionnaires, tabulate
results, etc.
(3) Choose the experts, making sure they have no vested interest in the outcome.
(4) Distribute the objectives, questionnaires, etc. to the experts for their initial set of opinions.
727
(5) The monitor team consolidates the opinions and redistributes the comments to the experts,
making sure that the comments remain anonymous.
(6) Repeat steps 4 and 5 until a group consensus is reached.
7.9.4 Example
The following example was adapted from reference 7.10:
A fault tree was generated for an SRM igniter, documenting all conceivable failure modes
associated with the subsystems. A sample of part of the fault tree is shown in figure 78. The fault tree
was then distributed to technical experts in the solid rocket industry. The expertise represented SRM
experience in design, structures, and processing. These experts were asked to assign subjective
estimations of failure probabilities of each mode and cause.
Igniter case
external insulation
fails to protect the
closure and
retaining ring 
structural, erosion
Inferior /
nonspec
insulation
Improper
material thickness
or geometry
Material
aging
degradation
Formulation
error
Improper
molding
Damage
to material or
threads due to
processing,
handling.
Poor
bonding of
material at adapter
Figure 78. Sample of a partial igniter subsystem fault tree.
The relative probabilities were based on a rating system which utilized a tailored version of
MIL–STD–882C
7.11
(sec. 3.12). The experts used letters to correspond to the descriptive words as
follows:
Level Descriptive Words Probability
A Infrequent 0.1
B Remote 0.01
C Improbable 0.001
D Very improbable 0.0001
E Almost nil 0.00001
728
Figure 79 shows an example of how the technical experts assigned estimations to each failure
level.
C E C
D
D D B B
Igniter case
external insulation
fails to protect the
closure and
retaining ring 
structural, erosion
Inferior /
nonspec
insulation
Improper
material thickness
or geometry
Material
aging
degradation
Formulation
error
Improper
molding
Damage
to material or
threads due to
processing,
handling.
Poor
bonding of
material at adapter
Figure 79. Fault tree sample with estimates assigned.
The team that generated the fault tree then took all the responses and assigned each failure level a failure
probability based on the letters assigned by the experts. An average was derived for each failure level
and applied to the fault tree. This labeled fault tree was distributed to the technical experts.
This process was repeated until a consensus on the assigned failure probabilities was arrived at by all of
the technical experts.
7.9.5 Advantages
(1) This technique can be useful in eliminating personality clashes.
(2) This technique can be useful when powerful personalities are likely to dominate the
discussion.
(3) Inputs from experts unavailable for a single meeting are included.
7.9.6 Limitations
(1) Arriving at a group consensus is timeconsuming.
(2) Assembling the group participants is difficult/timeconsuming.
729
7.10 Nominal Group Technique
7.10.1 Description
The NGT, as described in reference 7.1, is another tool used to reach a group consensus. When
priorities or rankings must be established, this decisionmaking process can be used. NGT is similar to
brainstorming (sec. 7.7) and the Delphi technique (sec. 7.9), but it is a structured approach that is
oriented toward more specialized problems. The group should be small (i.e., only 10 to 15 people), and
every member of the group is required to participate. This technique is often categorized as a silent
brainstorming session with a decision analysis process.
7.10.2 Application
The nominal group technique is an effective tool for producing many ideas and/or solutions in a
short time. The technique can be used for many of the same applications as brainstorming and the Delphi
technique. The NGT is best applied in phase A or B but may also be applied in phase E. Company
internal technical problems can be solved, personality clashes can be overcome, and NGT can be used to
develop new ideas to satisfy a particular problem.
7.3
7.10.3 Procedures
The NGT, as adapted from reference 7.1, is applied in the following manner:
(1) Generate the idea for discussion—a facilitator presents the problem and instructions to the
team.
(2) The team quietly generates ideas for 5 to 15 min—no discussion is allowed and no one
leaves until everyone is finished.
(3) The facilitator gathers the ideas roundrobin and posts them in no particular order on a flip
chart.
(4) The ideas are then discussed by the group; no arguments, just clarifications. Duplications
are eliminated.
(5) Each member of the group silently sets priorities on the ideas.
(6) The group votes to establish the priority or rank of each idea.
(7) The votes are tabulated and an action plan is developed.
7.10.4 Example
The following example was adapted from reference 7.12:
The overall objective of this task was to define an appropriate methodology for effective
prioritization of technology efforts required to develop replacement technologies (chemicals) mandated
by imposed and forecast legislation.
The methodology used was a semiquantitative approach derived from QFD techniques (sec.
7.12). This methodology aimed to weight the full environmental, cost, safety, reliability, and
programmatic implications of replacement technology development to an appropriate identification of
viable candidates and programmatic alternatives.
730
A list of concerns that needed to be addressed was developed as follows in table 710.
Table 710. Replacement technology concerns.
Chemical Concerns Environmental Concerns
(1) Number of sources (1) Clean air monitoring
(2) Limits of resources (2) Pollution prevention
(3) Availability (3) Toxic emissions
(4) Stability (4) Emissions control
(5) Drying ability (5) Ozone depletion potential
(6) Base material compatibility (6) Chemical storage availability
(7) Toxicity (7) Resource/ingredient recovery and
(8) Flash Point recycling
(9) Ease of maintenance (8) Hazardous waste management
(10) Historical data base
(11) Desirable reactivity Cost Concerns
(12) Undesirable reactivity (1) Manpower dollars
(13) Lottolot variability (2) Operations dollars
(14) Age sensitivity (3) Facilities dollars
(15) Shelf life (4) Materials dollars
(16) Bondline thickness (5) Chemical dollars
(6) Other hardware dollars
Process Concerns (7) Contracts dollars
(1) Contaminants removed (8) Change of specifications dollars
(2) Process steps (9) Specification verification dollars
(3) Parts processed at one time (10) Change of drawings dollars
(4) Required surface preparation (11) Development of procedure dollars
(5) Bondline thickness (12) Emissions control equipment dollars
(6) Process interaction (13) Emissions control testing dollars
(7) Bondline strength required
(8) Operator sensitivity Scheduling Concerns
(9) Lottolot variability Federal, State,and Local
(10) General cleaning ability (1) Research
(11) Surface requirements (2) Trade studies
(12) Possibility of stress corrosion cracking (3) Modification in planning
(13) Useful life of process part (4) Specification documentation
(14) Damage caused by process (5) Requirements documentation
(6) Drawing/design changes
Regulatory Concerns (7) Production time
(1) OSHA requirements (8) Testing
(2) State environmental laws (9) Vendor selection and certification
(3) Local environmental laws
(4) Federal environmental requirements Present Program Schedule
(5) Future federal regulations (1) Research
(2) Trade studies
Safety Concerns (3) Modification in planning
(1) Worker exposure limits (4) Specification documentation
(2) Spill response plans (5) Requirements documentation
(3) Fire response plans (6) Drawing/design changes
(4) Explosion response plans (7) Production time
(8) Testing
(9) Vendor selection and certification
731
A necessary step for developing a QFD matrix was to assign weighting factors to all of the
concerns. A group of 10–15 people knowledgeable in the subject of chemical replacement was
assembled to weight the concerns as follows:
(1) Each individual of the group, without discussion, generated ideas about the importance of
each concern.
(2) The facilitator collected the lists of ideas and posted them in no particular order.
(3) The ideas were discussed to clear up any misunderstandings.
(4) The group then voted on establishing the weighting factors on each concern.
Table 711 shows the list of assigned weighting factors based on a scale of 1 (least critical) to 20
(most critical).
7.10.5 Advantages
NGT is very effective in producing many new ideas/solutions in a short time.
7.10.6 Limitations
(1) Assembling the group participants is difficult/timeconsuming.
(2) Limiting discussion often limits full understanding of others ideas, with consequent
divergence of weighting factors as a likely result.
7.11 Force Field Analysis
7.11.1 Description
The force field analysis, as described in reference 7.1, is a technique that counts both the number
and magnitude of positive and negative forces that effect the results of a proposed solution or change in
process. The analysis of these positive and negative forces generally occurs after performing a
brainstorming session (sec. 7.7) or a cause and effect diagramming session (sec. 7.2).
This technique categorizes the identified forces as either positive or negative, and assigns a value
(weight) to each force. All positives and negatives are added and the more positive the total, the more
likely the proposed solution is the correct one. The more negative the total, the more likely the proposed
solution is not correct. A strategy is then developed to lessen the negative forces and enhance the
positive forces.
7.11.2 Application
The force field analysis is best applied in phase D or E. This analysis is often applied in
determining which proposed solution, among many, will meet the least resistance. The number of forces
should not be too high (i.e., < 20) or other more sophisticated approaches should be considered.
732
Table 711. Concerns with assigned weighting factors.
Chemical Concerns Environmental Concerns
(1) Number of sources 7 (1) Clean air monitoring 12
(2) Limits of resources 7 (2) Pollution prevention 12
(3) Availability 14 (3) Toxic emissions 15
(4) Stability 15 (4) Emissions control 12
(5) Drying ability 14 (5) Ozone depletor potential 15
(6) Base material compatibility 17 (6) Chemical storage availability 10
(7) Toxicity 13 (7) Resource/ingredient recovery 10
(8) Flash point 13 and recycling
(9) Ease of maintenance 8 (8) Hazardous waste management 12
(10) Historical data base 9
(11) Desirable reactivity 13 Cost Concerns
(12) Undesirable reactivity 13 (1) Manpower dollars 17
(13) Lottolot variability 11 (2) Facilities dollars 15
(15) Shelf life 9 (4) Materials dollars 14
(16) Bondline thickness 7 (5) Chemical dollars 16
(6) Other hardware dollars 14
Process Concerns (7) Contracts dollars 12
(1) Contaminants removed 15 (8) Change of specifications dollars 13
(2) Process steps 9 (9) Specification verification dollars 13
(3) Parts processed at one time 7 (10) Change of drawings dollars 11
(4) Required surface preparation 12 (11) Development of procedure dollars 12
(5) Bondline thickness 7 (12) Emissions control equipment dollars 15
(6) Process interaction 9 (13) Emissions control testing dollars 12
(7) Bondline strength required 9
(8) Operator sensitivity 12 Scheduling Concerns
(9) Lottolot variability 11 Federal, State, and Local
(10) General cleaning ability 13 (1) Research 9
(11) Surface requirements 14 (2) Trade studies 8
(12) Possible stress corr. crack. 14 (3) Modification in planning 9
(13) Useful life of process part 14 (4) Specification documentation 10
(14) Damage caused by process 13 (5) Requirements documentation 11
(6) Drawing/design changes 8
Regulatory Concerns (7) Production time 11
(1) OSHA requirements 13 (8) Testing 14
(2) State environmental laws 14 (9) Vendor selection & certification 12
(3) Local environmental laws 14
(4) Federal env. requirements 15 Present Program Schedule
(5) Future federal regulations 14 (1) Research 10
(2) Trade studies 11
Safety Concerns (3) Modification in planning 10
(1) Worker exposure limits 12 (4) Specification documentation 11
(2) Spill response plans 13 (5) Requirements documentation 11
(3) Fire response plans 14 (6) Drawing/design changes 10
(4) Explosion response plans 16 (7) Production time 11
(8) Testing 12
(9) Vendor selection & certification 11
733
Application of the force field analysis requires a proposed solution and inputs to the process.
These inputs might come from using group consensus techniques like those discussed in earlier sections.
Also, assigning the value (weight) to each force might also require group consensus techniques.
7.11.3 Procedures
The force field analysis, as adapted from reference 7.1, is performed in the following manner:
(1) Identify the proposed solution or change in process.
(2) Determine the forces, positive and negative, that might effect the implementation of this
proposed solution.
(3) Separate the forces into positive and negative lists and assign a value (weight) to each
force. Arriving at these values may be achieved by use of a group consensus technique like
the Delphi technique (sec. 7.9).
(4) Establish a strategy to lessen the negative forces and enhance the positive forces.
7.11.4 Example
Management met to discuss the possibility of approving a suggestion to allow employees to work
flextime. The group identified the positive and negative forces that will affect this decision as follows:
Positive forces Negative forces
Employees welcome change Employee accessible to customer
8 8
Increased production Employees present to receive messages
9 6
Coordinate hours to improve personal life Management aware of employee's
schedule
9 4
Total : 26 Total : 18
Figure 710. Force field analysis example.
The positive forces clearly outweighed the negative forces. Management developed a strategy to
lessen the magnitudes of the negative forces listed and thus enabled the proposal of flextime to be
approved.
734
7.11.5 Advantages
The force field analysis is useful in determining which proposed solution, among many, will
meet the least resistance.
7.11.6 Limitations
This technique is timeconsuming in arriving at a consensus on the values (weights) of the forces,
and is highly subjective.
7.12 Quality Function Deployment
7.12.1 Description
QFD, as described in reference 7.12, is a conceptual map that provides the means for cross
functional planning and communications. This technique is a method of turning the customer’s voice
into engineering language. A matrix is developed known as the “house of quality” and the main
elements of the matrix are the WHATs (customer concerns) and the HOWs (quantifiable solutions to the
concerns). The reason for the name “house” is because the matrix is shaped like a house and elements
are separated into rooms, as illustrated in figure 711.
MATRIX
RELATIONSHIP
HOWs
CORRELATION
MATRIX
WHATs
BENCH
MARKS
ENGINEERING
PARAMETERS
Figure 711. House of quality.
The other rooms of the house are defined as follows:
(1) Relationship matrix—This is the main body of the matrix, and it is the relationship between
each WHAT and HOW. These relationships are denoted by symbols or numbers which
correspond to weak, medium, and strong relationships.
Example : 1 = Weak
3 = Medium
9 = Strong.
735
(2) Correlation matrix—This is often called the “roof” of the house. The roof relates each of
the HOWs to each other and is also denoted by symbols or numbers which correspond to
strongpositive, mediumpositive, strongnegative and mediumnegative.
Example: ++ = Strongpositive
+ = Mediumpositive
– – = Strongnegative
– = Mediumnegative.
These data become important when the consideration of tradeoff factors is necessary.
(3) Benchmarks—This room is used to assess how well the product stacks up against the
competition.
(4) Engineering parameters—This room is used to assess how well the product stacks up to
applicable target values.
Inputs to the QFD matrix will require group sessions which will involve brainstorming (sec. 7.7), cause
and effect analysis (sec. 7.2) and other techniques that might help to gather information about customer
requirements
7.1
7.12.2 Application
The QFD technique is typically performed in phase C but may also be performed in phase A or B.
This technique may be used by every function in the producing organization and in every stage of product
development. The main focus is to implement change during design rather than during production.
Not only does the QFD matrix allow assessment of the product against the competition and other
benchmarks, it also enables a prioritization of the HOWs, i.e., the results of the QFD analysis can give
overall ratings for each quantifiable solution to the stated concerns. These ratings indicate which solutions
are most important and need to be considered first. The most important reason for the QFD analysis is to
identify the problem areas and the quantifiable solutions to these problems early in the design phase so
these issues will not have to be faced during production, which could lead to delays and higher costs.
7.12.3 Procedures
As adapted from reference 7.13, a QFD analysis is performed as follows:
(1) List and prioritize the WHATs that concern the customer. These items are generally very
vague and require further definition. This list will be placed in rows at the left side of the
house. Each item is weighted for importance to the customer.
(2) List the HOWs that address the WHATs. This list of quantifiable solutions to the WHATs
will be placed in columns and because the WHATs are so vague, one or more HOWs can
relate to each WHAT.
(3) Correlate the WHATs and HOWs. This correlation is entered into the main body of the
matrix (relationship matrix). These relationships are weighted as noted in section 7.12.1.
(4) List the benchmarks and perform an assessment. The assessment can be performed on both
the HOWs and the WHATs. Areas for improvement can easily be noted here by comparing
how well this product stacks up against the competition.
736
(5) Correlate the HOWs to each other as noted in section 7.12.1, step 2.
(6) Calculate the scores of the relationships. The score for each HOW as related to each
WHAT is determined by multiplying the weighting factor for each WHAT by the
corresponding value in the relationship matrix. The overall ratings for the values in table 7
12 are calculated as follows:
Table 712. QFD matrix sample calculations.
Concern 1
Concern 2
Concern 3
Overall Rating
W
H
A
T
S
H O S W
Solution 1 Solution 2 Solution 3
3 9 1
1
9
3
9 1
9
153 237 163
10
15
12
Weighting Factors
Solution 1 would have an overall rating of (10×3)+(15×1)+(12×9) = 30+15+108 = 153.
Solution 2 would have an overall rating of (10×9)+(15×9)+(12×1)= 90+135+12 = 237.
Solution 3 would have an overall rating of (10×1)+(15×3)+(12×9)= 10+45+108 = 163.
This example reveals that solution 2 is the most important HOW in achieving the collective
WHATs.
7.12.4 Example
A planning team for an automobile company performed a task of trying to anticipate problem
areas in a design so they can be improved upon or eliminated early. Six customer concerns (WHATs) for
an automobile were studied:
(1) Good performance
(2) Quiet
(3) Safe
(4) Good gas mileage
(5) Affordable
(6) Roomy.
737
Next, all possible solutions to these concerns (HOWs) were identified and they are:
(1) 0–60 (s)
(2) Fuel economy (mpg)
(3) Horsepower
(4) Weight (klbs)
(5) Emissions (ppm)
(6) Noise level (dB)
(7) Energy absorption rate (mph)
(8) Purchase price (k$)
(9) Maintenance cost ($)
(10) Head room (in)
(11) Elbow room (in)
(12) Leg room (in).
This automobile company was benchmarked (sec. 7.1) against three competitors as to how well
each company stacks up to meeting each WHAT. The benchmark rating scale used was from 1 (low) to
5 (high).
Engineering parameters were identified for each HOW. The first parameter for each was the
desired parameter for this company to target. The next row delineated the current company practice for
each parameter. A final entry for these parameters, was the percent difference between the company’s
present level and the desired target.
The roof was included which identified the relationships between the HOWs. The rating scale
used was as follows:
9 = Strong positive
3 = Medium positive
–3 = Medium negative
–9 = Strong negative.
Finally, weighting factors were given to each customer concern. That is, on a scale of 1 (low) to
10 (high), each concern was rated for importance. All of the data were coordinated and a QFD matrix
was developed as shown in figure 712.
738
Figure 712. QFD example on auto industry.
739
Conclusions:
(1) Looking at the overall ratings showed that the two most important solutions in achieving
the collective concerns were the horsepower rating followed by the time taken to get from 0
to 60 mph.
(2) The benchmarking of this company to the three main competitors revealed that, overall,
this company rated as well or better than the competitors. The matrix showed that this
company could stand to improve on achieving a quiet ride, getting better gas mileage, and
making the automobiles roomier.
7.12.5 Advantages
(1) The QFD technique helps organizations design more competitive, higherquality, and
lowercost products easier and quicker, and is aimed primarily at the development of new
products.
(2) This technique helps ensure quality products and processes by detecting and solving
problems early.
(3) Engineering changes are reduced.
(4) The design cycle is reduced.
(5) Startup costs are reduced.
(6) The voice of the customer is heard.
(7) The technique is proactive instead of reactive.
(8) The technique prevents problems from “falling through the crack.”
(9) The technique is economical.
(10) The QFD technique is easy to learn.
7.12.6 Limitations
(1) Assembling the group participants is difficult/timeconsuming.
(2) Even though the analysis is easy to learn, it is not easy to perform.
7.12.7 Bibliography
Hauser, J.R. and Clausing, D.: “The House of Quality.” Harvard Business Review, No. 88307, vol. 66 ,
No. 3, May–June 1988.
Sullivan, L.P.: “Quality Function Deployment.” Quality Process, June 1986.
740
7.13 Quality Loss Function
7.13.1 Description
The quality loss function technique is a Taguchi method of determining the “loss to society”
when a product is not at the true value (i.e., mean) although it still lies within specification limits.
As described in reference 7.14, in order to develop a function to quantify the loss incurred by
failure to achieve the desired quality, the following characteristics must be considered:
(1) Larger is better (LIB)—the target is infinity.
(2) Nominal is best (NIB)—a characteristic with a specific target value.
(3) Smaller is better (SIB)—the ultimate target is zero.
Traditionally, manufacturers have considered a product “perfect” if it lies between the lower and
upper specification limits as illustrated in figure 713.
LOSS LOSS
PERFECT
LSL N
USL
Figure 713. Traditional view to meeting specification.
where
LSL = Lower specification limit
N = Nominal
USL = Upper specification limit.
The problem with this approach is that when “tolerance stackup” (sec. 4.3) is considered,
difficulties arise. If two mating parts are being manufactured, they may fall at opposite ends of their
specific tolerance and they may not assemble properly.
741
7.13.2 Application
The quality loss function technique is typically performed in phase E but may also be performed in
phase D. This technique is used to improve a process, thus it can be used for productivity improvement
measurements. For each quality characteristic there is a function which defines the relationship between
economic loss (dollars) and the deviation of the quality characteristic from the nominal value.
7.14
The application of the quality loss function L(y) also reveals indications of customer
dissatisfaction. The further the characteristic lies from the nominal value, the more problems may arise
and thus more customer complaints. These complaints, in turn, will lead to a financial loss.
Of course, just because a characteristic meets the target value, it does not mean that the quality of
the product is adequate. The specification limits may be out of line.
7.13.3 Procedures
As described in reference 7.14, the L(y) around the target value n is given by:
L(y) = k (y–n)
2
(7.1)
where
L(y) = loss in dollars per unit product when the quality characteristic is equal to y.
y = the value of the quality characteristic, i.e., length, width, concentration,
surface finish, flatness, etc.
n = target value of y.
k = a proportionality constant.
By applying equation (7.1) and examining figure 714, it can be seen that L(y) is a minimum at
y = n and L(y) increases as y deviates from n.
L(y)
($)
Ao
∆o
n
y
Figure 714. Quality loss function for NIB.
742
where
Ao = consumer’s loss
and
∆o = tolerance.
To apply the quality loss function equation, proceed as follows:
(1) As given in equation (7.2):
L(y) = k (y – n)
2
(2) To calculate a dollar loss at some value (y), first calculate k.
k =
Ao
∆o
. (7.2)
(3) Calculate L(y).
7.13.4 Example
Determine the dollars lost at some value (y) per figure 715.
L(y)
($)
Ao
∆o
y
500 =
80 100
y = 85
120
Figure 715. Quality loss function example.
L(y) = k (y – n)
2
k =
Ao
∆o
·
500
(20)
2
·
500
400
= 1.25
L(y) = 1.25 (85 – 100)
2
= 1.25 (–15)
2
= 1.25 (225) = $281.25.
743
7.13.5 Advantages
(1) The quality loss function technique is an excellent tool for evaluating loss at the earliest
stage of the product/process development.
(2) Useful results can be obtained quickly and at low cost.
7.13.6 Limitations
(1) With many manufacturers following the guidelines that their product is adequate if certain
measurements are within the specification limits, it is difficult to convince them to apply
this technique.
(2) It is often very difficult to calculate the quality loss function for a given process. The
parameter y and the relationship to any A
o
are generally obscure.
7.14 Statistical Process Control
7.14.1 Description
SPC is a method of using statistics applied to the results of a process to control the process.
Historical data of the performance of the process (or operation of hardware) are statistically analyzed to
predict future performance or to determine if a process is “in control.” A process is defined as “in
control” if there are only random sources of variation present in the process and the associated data. In
these cases, the data can correctly be investigated with the standard methods of statistical analysis. If the
data are not “in control,” there is some special cause of variation present in the process, and this is
reflected in the data from that process. In these cases, this section on SPC assumes that the data
variability is still reasonably distributed around the mean, and these procedures are applicable. If these
procedures lead to a result of special cause variation at nearly every data point, these procedures cannot
be correctly applied.
7.14.2 Application
The SPC technique is best performed in phase E. This technique is used to determine if special
causes of variation are present in a process, or if all variation is random. In other words, SPC is used to
ensure that a product is being produced consistently, or is about to become inconsistent. Thus, SPC can
be used to isolate problems in a process before defective hardware is delivered. This technique can be
used for measurement type data (real numbers) or attribute data. There are two types of attribute data—
binomial data and poisson data. Binomial data have a given number of outcomes, e.g., three of four parts
on an assembly can be defective. Poisson data have an unlimited number of possible outcomes, e.g., a
yard of material may have 1, 10, or 100 flaws.
7.14.3 Procedures
The basic steps for conducting SPC are:
(1) Decide how to group the data. Subgroups should be chosen to show the performance of the
part or process of interest. For example, if a machine is producing several parts at a time,
the parts produced at one time will be a logical subgroup.
(2) Construct a control chart and range chart (see below).
744
(3) Determine and apply control limits to the data.
(4) Determine if any control limits are violated. If any control limits are violated, a special
cause is indicated. In addition to the specific control limits, the analyst must examine the
data plot for other visual indications of special causes in the data. Any particular pattern,
for example, would indicate a special cause is present. The use of engineering judgment is
critical to extracting the maximum amount of data from the SPC plots.
(5) Determine the special cause. This may require Pareto analysis or engineering judgment
using past experience.
(6) Implement a fix for the special cause of variation.
(7) Plot the data to ensure that the fix has been effective.
Control charts (sec. 5.2) are made as follows:
(1) A plot is made of the data, in temporal order of generation, on a scatter plot.
(2) If the data are subgrouped, the mean values of the subgroups are plotted.
(3) A range chart is made where the range is plotted for each subgroup. If the subgroup size is
one, a moving range chart is made. The moving range for an abscissa (“x” value) is the
absolute value of the difference of the ordinates for the abscissas and the previous abscissa.
(4) Determine control limits as discussed below.
(5) Apply appropriate rules to detect a lack of control (see below).
There are typically three control limits based on the population standard deviation of the process
(sec. 6). If negative values of data are possible, there are six control limits. They are the mean of data
plus or minus one, two, and three standard deviations. If one datum exceeds the mean plus three standard
deviations, a rule 1 violation exists. If two of three data points exceed the mean plus two standard
deviations, a rule 2 violation exists. If four of five consecutive data points exceed the mean plus one
standard deviation, a rule 3 violation exists. If eight consecutive points exceed the mean, a rule 4
violation exists. If negative values of data are possible, these rules apply if the values are below the
control limit.
For real number data, the population standard deviation is determined from the average of the
data by the equation:
s = R
m
/d
2
(7.3)
where s is the population standard deviation, R
m
is the mean of the subgroup ranges, and d
2
is a factor
for converting the mean range to the population standard deviation. The constant d
2
can be found in
reference 7.15. If the data are not subgrouped, the average moving range is used. The moving range is
the difference between a data point and the preceding point.
For binomial data, the population standard deviation is given by the equation
s = p
m
(1– p
m
)/ n
m
( )
0.5
(7.4)
where p
m
is the mean fraction defective, and n
m
is the number in each sample.
745
For Poisson data the population standard deviation is given by the equation
s = (C)
0.5
(7.5)
where C is the average number of nonconformities per subgroup.
The discussion in this section has thus far been centered on a violation of a control limit
indicating a special cause of variation being present. The special cause itself may be a shift in the entire
data pattern defined as a mean shift or population shift. In these cases, the limits should be modified or
recalculated to be appropriate for the subsequent data points. A mean shift is generally attributable to an
obvious special cause such as a change in process, material, operator, cutting head, or specification. Data
points immediately preceding and following a mean shift should not be grouped together for any other
analyses.
7.14.4 Example
A hypothetical drill jig is set up to drill five holes in a component. The five holes are of the same
size and have the same positional tolerance. Provide a control chart showing the performance of the drill
jig with the data below, and determine the source of any deviation from nominal hole position. Table 7
13 below shows the deviation from nominal hole size and position made by each drill guide for each
part.
Table 713. Nominal hole size deviations and drill guide positions.
Part # Tempora
l
Process
Order
Hole 1 Hole 2 Hole 3 Hole 4 Hole 5 Range Mean
2 1 2 1 2 3 1 2 1.8
1 2 1 2 3 4 3 3 2.6
4 3 3 3 1 2 2 2 2.2
5 4 2 2 2 3 1 3 2.0
3 5 4 2 3 2 2 2 2.6
6 6 2 1 2 3 1 2 1.8
7 7 6 3 1 2 3 5 3.0
10 8 7 2 2 1 3 6 3.0
8 9 9 3 2 2 2 7 3.6
9 10 10 2 1 3 4 9 4.0
range 9 2 2 2 2 5
mean 4.6 2.1 1.9 2.5 2.2 4.1 2.66
The mean and range for each part and each hole is shown in the table 713. Each part will be
considered to be a subgroup. If the variation between holes is of primary interest, it could be better to
treat each hole as a subgroup. However, the performance of the entire jig is of primary interest in this
example, so each part will be treated as a subgroup. The first control chart (fig. 716) shows the
performance of the jig with the mean plotted against the timephased process order. The UCL is shown.
The UCL is calculated using equation (7.6) to obtain the population standard deviation, multiplying it by
3 and adding it to the mean of the mean deviation. Notice that the mean measurement is increasing for
the last few parts, but no control limits are exceeded.
746
1 2 3 4 5 6 7 8 9 10
Mean
Deviation
6
5
4
3
2
1
UCL
Temporal Process Order
Figure 716. Control chart showing mean deviation for each part.
The second chart (fig. 717) is a range chart that shows the mean range for each part plotted
against part number (note that it remains in temporal order). Part number 9 exceeded the UCL range
(UCLR). UCLR is given by the equation:
UCLR = R
m
[1 + 3(d
3
/d
2
)] (7.6)
where R
m
is the mean range and d
3
is a factor for converting the mean range to the standard deviation of
the range. The constant d
3
can be found in reference 7.15. This shows that the withingroup variation is
increasing more that the grouptogroup variation.
2 3 4 5 6 7 8 9 10
10
9
8
7
6
5
4
3
2
1
Deviat ion
Range
UCLR
1
Figure 717. Range chart showing mean range for each part.
747
The third chart (fig. 718) shows a Pareto chart (sec. 5.6) where the mean deviation is plotted
against hole number. By examination, it can be seen that drill guide position 1 is producing holes with a
mean measurement that is higher than the other drill guide positions.
1 2 3 4 5
5
4
3
2
1
Mean
Deviat ion
Hole Number
Figure 718. Pareto chart showing mean deviation for each hole guide.
The fourth chart, figure 719, shows the deviation produced by hole guide 1 plotted against part
number. By examination, it can be seen that the deviation is increasing starting with part 7.
1 2 3 4 5 6 7 8 9 10
10
9
8
7
6
5
4
3
2
1
Part Number
Deviat ion
Figure 719. Control chart showing mean deviation for hole guide 1.
7.14.5 Advantages
(1) SPC is an excellent technique for determining the cause of variation based on a statistical
analysis of the problem.
(2) The technique improves process performance.
(3) SPC helps identify problems quickly and accurately.
748
7.14.6 Limitations
SPC detects problems but poses no solutions.
7.14.7 Bibliography
Miller, I., and Freund, J.E.: “Probability and Statistics for Engineers.” Second edition, Prentice Hall Inc.,
Englewood Cliffs, NJ 07632.
Nolan, T.W., and Provost, L.P.: “Understanding Variation.” Quality Progress, May 1990.
Weisbrodand, S., and McFarland, C.S., “A New Approach to Statistical Process Control in a Test
Environment: The Empirical Delta Control Chart.” IEEE Melbourne, FL, 1991.
7.15 Flowchart Analysis
7.15.1 Description
A flowchart, as described in reference 7.3, is a pictorial representation of the steps in a process
where each step is represented by a block. The review of a flowchart allows the elimination of nonvalue
added steps. When prepared by a group, the chart represents a consensus. The flowchart analysis is a
useful tool for determining how a process works. By studying how process steps relate to each other,
potential sources of problems can often be identified.
Many different types of flowcharts are useful in the continuous improvement process. Flowcharts
often used are the topdown flowchart, the detailed flowchart, and the work flow diagram. The topdown
flowchart, figure 720, presents only the major and most fundamental steps in a process. This chart
makes it easy to visualize the process in a single, simple flow diagram. Key actions associated with each
major activity are listed below their respective flow diagram steps. A topdown flowchart can be
constructed fairly quickly and easily. This type of flowchart is generally developed before attempting to
produce the detailed flowcharts for a process. By limiting the topdown flowchart to key actions, the
probability of becoming bogged down in the detail is reduced.
TQM
Exposure
Apply
TQM
Show
Purpose
TQM
Develop
Training
Procedures
Meet
Customer
Needs
Subordinates
• Develop
Implementation
Plan
First
Step
• Take
• Be
Committed
• Be
Consistent
• Examine
Your Use
• Develop
User's Guide
• Sell Idea
of
Implementation
• On the job
Training
• Provide
Training
Classes
• Use
Available
Resources
• Listen
to
Customer
• Understand
Customer
Needs
• Establish
Routine
Communication
with
Customer
• Become
Familiar
with TQM
• Familiarize
with TQM
of
Figure 720. Example of topdown flowchart.
749
The detailed flowchart, figure 721, gives specific information about process flow. At the most
detailed level, every decision point, feedback loop, and process step is represented. Detailed flowcharts
should only be used when the level of detail provided by the topdown or other simple flowcharts is
insufficient to support the analysis of the process.
Calculation
Are the bolts
strong enough?
Can they be
bigger?
Is there a
stronger material?
Can more bolts
be used?
End
Yes
Increase
bolt size
Yes
Select
stronger
material
Yes
Add more
bolts
Yes
No
No
No
No
End with
no satisfactory
answer
Do Bolt
Strength
Figure 721. Example of detailed flowchart.
The work flow diagram (section 7.15.4) is a graphic representation of how work flows through a facility.
This diagram is useful for analyzing flow processes, illustrating flow efficiency, and planning process
flow improvement. Figure 722 illustrates the most common flowchart symbols.
750
Activity Symbol
Decision Symbol
Terminal Symbol
Flow Line
Document Symbol
Data Base Symbol
On Page Connector
Off Page Connector
Brick Wall
Inspiration
Black Hole
Dead End
Magic Happens Here
DEAD
END
"POOF"
 Action that is taking place.
 Yes/No Decision.
 Beginning or end of
process.
 Shows direction of process flow.
 Indicates a document
source.
 Indicates a database
source.
 Indicates point elsewhere
on a large page where
process continues.
 Indicates point on
another page where
process continues.
 Shows obstacle beyond your control.
 Indicates a possible solution.
 Indicates a problem that
consumes all resources.
 Shows particular path of a process
has no acceptable solution.
 Indicates that, with a
breakthrough, we can
continue the process.
Figure 722. Common flowchart symbols.
751
7.15.2 Application
A flowchart is best applied in phase B but may also be applied in phase E. This chart is used to
provide a picture of the process prior to writing a procedure. Flowcharts should be created, then
procedures written to follow the flowchart. The chart should be included as an appendix in the
procedure. Flowcharts can be applied to anything from material flow to the steps it takes to service or
sell a product.
7.15.3 Procedures
A flowchart, as described in reference 7.1, is prepared in the following manner:
(1) A development team creates a diagram that defines the scope of the task to be undertaken.
Also identified are the major inputs and outputs.
(2) Create a data flow diagram. Start with executive level data that are involved in the process,
followed by department data and finally branch data.
(3) Using the data, create an initial model. The team should walk through the process and look
for any details that need to be clarified, added, or deleted.
(4) Make a data dictionary. This ensures that everyone involved in the project has a consistent
understanding of the terms and steps used.
(5) Add the process symbols.
(6) Revise, as necessary.
7.15.4 Example
The following example, figure 723, illustrates a work flow diagram for encountering problems
with a copy machine.
7.15.5 Advantages
The following advantages are adapted from reference 7.16:
(1) Flowcharts allow examination and understanding of relationships in a process.
(2) Flowcharts provide a stepbystep picture that creates a common understanding about how
the elements of a process fit together.
(3) Comparing a flowchart to actual process activities highlights areas where policies are
unclear or are being violated.
7.15.6 Limitations
The flowchart development process can be timeconsuming.
752
Start
Error
Light
Flashes
Clear Paper
Path
Still
Flashing
Press Help
Button
Follow
Directions
Call
Repairman
End
No
Yes
Yes
No
Yes
No Still
Flashing
Figure 723. Work flow diagram example.
7.16 Work Flow Analysis
7.16.1 Description
A WFA, as described in reference 7.1, examines the work process for possible improvements in
performance and quality of work life. This technique is really a special case of flowcharting (sec. 7.15).
The goal is to overcome the excuses for not changing work habits on the part of the employee as well as
management. Such excuses are, “It has always been done this way,” and “It’s not my responsibility.”
753
7.16.2 Application
A WFA is best applied in phase E. The analysis is performed in an employee/management
partnership, where the goal for each party is to improve productivity as well as the quality of work life.
The technique will work if executed by a partnership of management and employees.
7.16.3 Procedures
As adapted from reference 7.1, a WFA is performed in the following manner:
(1) Collect data concerning the operation being analyzed. This can be done by observing the
operation or asking questions, but not by reading an operations plan that would tell how the
operation is supposed to be done.
(2) Flowchart the process (sec. 7.15).
(3) Research and collect ideas on how to improve the operation from any sources available.
(4) Define the desired performance versus the actual performance.
(5) Identify the gaps in performance and propose changes to eliminate these gaps.
(6) Analyze these changes by using a multifunctional team.
(7) Once the changes are agreed upon, prototype them on a small basis in a certain area or shift.
(8) Once the bugs are ironed out and the changes are operating smoothly, implement them on a
largescale basis.
(9) Flowchart the new operation and revise the operating procedure documentation to reflect
the changes.
7.16.4 Example
An analysis team was assembled to analyze the food preparation process at a local fast food
restaurant in an attempt to find areas where the operation could be run more efficiently. The steps of the
analysis are as follows:
(1) The first step involved observing the operation and then flowcharting the process as shown
in figure 724 below.
(2) Members of the team then observed other restaurants to find ways of improving the process.
(3) Once the research was completed, the desired performance was identified and compared to
the actual process.
(4) The team, which involved management, employees, and outside consultants, then
developed a new plan for the process.
(5) This new process was first tried out during slow business hours to ensure the new process
ran smoothly.
(6) Once everyone agreed that the new process was more efficient, then it was implemented.
754
Start
No
Yes
Yes
No
Fry
Burger
Is meat
cooked enough?
Put on
Bun
Are
toppings
conveniently
located?
Reorganize
Serve
Figure 724. WFA example.
7.16.5 Advantages
The technique may increase productivity and improve working conditions.
7.16.6 Limitations
(1) The technique requires cooperation between employees and management to be most
successful.
(2) The observed operation may not be fully representative of a “typical” process that would
occur without scrutiny.
755
REFERENCES
7.1 Brocka, B., and M. S.: “Quality Management, Implementing the Best Ideas of the Masters.”
Business One Irwin, Homewood, IL 60430, 1992.
7.2 Camp, R.C.: “Benchmarking, The Search for Industry Best Practices that Lead to Superior
Performance.” Quality Press, 1989.
7.3 Hunt, V.D.: “Quality in America, How to Implement a Competitive Quality Program.” Business
One Irwin, Homewood, IL 60430, 1992.
7.4 Lake, J.G.: “Concurrent/Systems Engineering.” Defense Systems Management College, 1992.
7.5 “Total Quality Management – Executive Briefing.” American Supplier Institute, Dearborn, MI,
1990.
7.6 Choppin, J.: “Quality Through People, A Blueprint for Proactive Total Quality Management.”
IFS Publications, United Kingdom, 1991.
7.7 Ott, E.R., and Schilling E.G.: “Process Quality Control.” McGrawHill Book Company, 1990.
7.8 Juran, J.M., Gryna, F.M., Jr., and Bingham, R.S., Jr.: “Quality Control Handbook.” McGrawHill
Book Company, 1974.
7.9 Box, G.E.P., Hunter, W.G., and Hunter, J.S.: “Statistics For Engineers.” John Wiley and Sons,
Inc., 1978.
7.10 “Advanced Solid Rocket Motor Ignition Subsystem Assessment.” Larry Thomson, Kurt
Everhart, Curtis Ballard, Report No. 31400493259, August 1993.
7.11 “System Safety Program Requirements.” MIL–STD–882C, January 1993.
7.12 “A Comprehensive Tool for Planning and Development.” M.A. Schubert, Reliability Technology
Consultant, Inland Division of General Motors Corporation, Dayton, OH, 1989.
7.13 “Prioritization Methodology for Chemical Replacement.” W. Cruit, S. Schutzenhofer, B.
Goldberg, and K. Everhart, May 1993.
7.14 “Introduction to Quality Engineering – Four Day Workshop, Implementation Manual.”
American Supplier Institute, Dearborn, MI, 1989.
7.15 NASA–STD–8070.5A, “Trend Analysis Techniques.” October 1990.
7.16 Harrington, H.J.: “Business Process Improvement.” McGrawHill Book Company, TS5156
H338, 1991.
756
8. TREND ANALYSIS TOOLS
Trend analysis, as described in reference 8.1, is a quantitative tool used to identify potentially
hazardous conditions and cost savings based on past empirical data. Trend analysis evaluates variations
of data to find trends, with the ultimate objective of assessing current status and forecasting future
events. Trend analysis can be reactive or proactive. Data examined from past events can uncover a cause
of a problem or inefficiency in a product or operation. Also, realtime data can be tracked to detect
adverse trends that could indicate an incipient failure or can be used to reduce discrepancies in a product
or operation.
Program level trending exchanges data between organizations and correlates trends from the
various organizations to find relationships and allows integration of the trend analysis effort with any
planned TQM effort (sec. 7), such as SPC (sec. 7.14). It also allows upper level management to forecast
problems such as shortages, schedule delays, or failures. Finally, in starting a program level trending
effort early in the program, data collection will be more efficient and costeffective.
The use of trend analysis has several benefits. Among them are:
(1) Predicting system or process failure or violation of a process limit criterion.
(2) Indicating that a unit can remain in service longer than anticipated or projecting the service
life of a unit.
(3) Eliminating the need for some hardware inspections.
(4) Increase costeffectiveness by reducing variability in a process.
There are different levels of trend analysis parameter criticality based on the degree of the benefit
derived from the results of the trend analysis for that parameter. Some parameters have a direct effect on
system safety while others will have an impact on cost or timeliness of a process. Criticality levels have
an impact on the amount of trending to be performed, the level to which it is to be reported, the data that
are to be stored, and the time over which the trending is to be performed. Examples of criteria for levels
of requirements are:
(1) Parameters impacting personnel safety.
(2) Parameters impacting successful system performance.
(3) Parameters which could cause failure of a component that would not result in system
failure.
(4) Parameters impacting schedule of the system.
(5) Parameters impacting delivery schedule of components.
(6) Parameters impacting cost of manufacturing.
Trending can be used at levels from program management to component and system production
and vendors. Upper level management would conduct trending on program level issues, and individual
organizations would conduct trending on issues pertinent to that organization at a component/material,
subsystem, or system level.
81
Examples of trending activities are:
(1) Componentreceiving organizations can conduct trending on such things as would indicate
the quality of incoming components, materials, and problems of receiving them in proper
condition.
(2) Manufacturing can conduct trending on component and system requirements, and
production problems.
(3) Test, launch, and refurbishment organizations can conduct trending on performance, time
to conduct operations, and problems encountered.
Some trending results will be reported to upper level management, engineering, and the
customer, while other results would be for local use by the individual organizations.
Five trending analysis techniques will be discussed in this section. Performance trend analysis,
discussed in section 8.1, detects a degrading parameter prior to a potential failure as well as predicting
future parameter values.
Problem trend analysis, discussed in section 8.2, provides an early indicator of significant issues
in other types of trend analysis. Other applications of this analysis are to “examine the frequency of
problem occurrence, monitor the progress of problem resolution, uncover recurring problems, and assess
the effectiveness of recurrence control.”
8.2
A technique that provides visibility to determine the current/projected health of the human
support element is programmatic trend analysis. This analysis is discussed in section 8.3. A technique
that monitors the current health of support systems and forecasts support problems to enable resolution
with minimum adverse effect is supportability trend analysis. This analysis is discussed in section 8.4.
Finally, reliability trend analysis is discussed in section 8.5. This technique is similar to
performance trend analysis and problem trend analysis. Reliability trend analysis measures reliability
degradation or improvement and enables the prediction of a failure so action can be taken to avert the
failure.
There can be a high level of overlap for some of these types of trend analysis, depending on
individual definitions of performance, reliability, and problems. Since many tools are useful for all types
of trending and the trend analysis customer typically looks for known parameters, this overlap is not a
problem. Performance, problem, and reliability trend analyses are more directly applicable to the needs
of a system engineer, than programmatic or supportability trend analyses. However, the former two
types of trend analysis are presented here, since results from these analyses may impact the system for
which the system engineer is responsible.
A summary of the advantages and limitations of each tool or methodology discussed in this
section is presented in table 81.
82
Table 81. Trend analysis tools and methodologies.
Tool or Methodology Section Advantages Limitations
Performance trend analysis 8.1 (1) Detects a degrading parameter prior to a potential
failure.
(2) Predicts future parameter values or estimates the
longterm range of values of influential variables.
(3) The service life of systems or system elements can
be predicted.
(1) Parameter sensors may need to be installed to
obtain trending data; this can be costly.
(2) The operating state, output, or load, about/
through which a system/subsystem/component
fluctuates, often cannot be controlled to achieve
consistent trend data. (Data must be statistically
stable.)
(3) The slope and stability of the data approaching/
departing the recorded data point are not known
without using a data buffer.
(4) Data are not always easily quantifiable, limiting the
usefulness of the technique.
Problem trend analysis 8.2 (1) Provides an early indicator of significant issues in
other types of trend analysis.
(2) Examines the frequency of problem occurrence,
monitors the progress of problem resolution,
uncovers recurring problems and assesses the
effectiveness of recurrence control.
Candidate items should be chosen carefully because the
analysis can be costly if performed for all potential
problem areas.
Programmatic trend analysis 8.3 This technique monitors programmatic posture and
provides visibility to determine current/projected health
of the human support element.
The data collection process can be extensive because of a
potentially large and varied number of sources.
Supportability trend analysis 8.4 This technique monitors the current health of support
systems and forecasts support problems to enable
resolution with minimum adverse effect.
Determining the extent of analysis and identifying the
appropriate parameter variations that must be measured
can be difficult.
Reliability trend analysis 8.5 This technique measures reliability degradation or
improvement and enables the prediction of failures so
action can be taken to avert failure.
Candidate items must be chosen carefully because the
analysis can be costly if performed for all potential
parameters.
83
8.1 Performance Trend Analysis
8.1.1 Description
Performance trend analysis, as described in references 8.1 and 8.2, is a parametric assessment of
hardware and software operations to evaluate their status or to anticipate anomalies or possible
problems. This assessment not only includes operational performance, such as ballistics of an SRM but
also assesses hardware performance, such as insulation and inhibitor systems, the motor case, or the
nozzle system. For example, postflight measurements of insulation indicate the performance of the
insulation during motor operation. The independent variable in performance trend analysis can be time
or sequence. Some performance data, for example, that relating to safety, may be recorded and trended
on a realtime basis.
As an example, for an SRM, typical operational performance parameters to be trended could be
peak pressure, total impulse, ignition delay, thrust risetime characteristics, and propellant structural or
ballistic properties. Typical hardware performance parameters to be trended could include insulation
anomalies, structural factor of safety (calculated from asbuilt drawings), and seal performance (as
measured, i.e., from leak checks).
As described in reference 8.2, data sources for performance trend analysis might include new,
refurbished, and repaired component and subassembly acceptance inspection, checkout, and test data for
development and verification and production hardware including, but not limited to:
(1) Alignment data.
(2) Contamination data.
(3) Dimensional data.
(4) Nondestructive test data, e.g., magnetic particle, radiography, penetrant, and ultrasonic
data.
(5) Proof test data, e.g., leak check and hydroproof data.
(6) Functional or performance data, e.g., quantitative and qualitative data.
8.1.2 Application
Performance trend analysis is best applied in phase E but may also be applied in phase D. This
analysis can be used to identify certain parameters that will indicate that a system or system element
(i.e., subsystem, assembly, subassembly, component and piecepart) is degrading and will potentially
fail. These parameters can include, but are not limited to, the following:
8.2
(1) Direct measures of degradation, such as wear, erosion, pitting, and delamination.
(2) Measures of conditions that might introduce degradation, such as pressure anomalies,
temperature anomalies, vibration, friction, leakage, and contamination.
(3) Measures that indicate a shift in performance,such a changes in material properties,
calibrations, and electrical resistance.
Attendance to maintenance can help to detect degrading parameters which could lead to failure
or delay resulting from an exceedance of criteria.
84
8.1.3 Procedures
The procedures to apply performance trend analysis, adapted from references 8.1 and 8.2, are
presented below:
(1) Identify the elements of the system. Assess those hardware or software system elements to
identify items that could cause critical or costly failures. Each element of the system should
be considered, i.e., each subsystem, assembly, subassembly, component and piecepart. List
these system elements as candidates for performance trend analysis.
(2) From the list, select which items will be analyzed. Concerns (in terms of risk, safety, cost,
availability, or schedule) and expected benefits should be the basis for setting priorities
when considering which items to select for performance trend analysis.
(3) Determine the parameters that characterize the performance of the selected system
elements. Select parameters that will indicate performance deterioration of the given
system element in a timely manner for corrective actions to be approved by management
and implemented. Review the following to identify possible candidate parameters for
performance trending:
a. FMEA (sec. 3.4)/critical items list (FMEA/CIL).
b. Drawings and specifications.
c. Previous problem reports.
d. Equipment acceptance data.
e. Original equipment manufacturer’s data.
f. Operations manual.
(4) Establish the criticality of each selected parameter. The parameter criticality should be
based on the FMEA/CIL or other criteria that have been preapproved by management. The
criticality of the parameter will indicate the magnitude of the impact if an adverse trend is
detected and to what level of management that adverse trend is reported.
(5) Determine if the selected parameters can be quantified with obtainable data. A parameter
may be quantified with direct measured data (such as temperature, pressure, force, strain,
acceleration, heat flux, etc.) or by calculation involving two or more direct measurements
(such as specific impulse for rocket engines or compressor and turbine efficiencies for jet
engines). If data are not available, establish a system to acquire the data or drop the item
from trend analysis.
The availability of the data—the more available the data are, and assuming statistical
stability, the greater the likelihood of successful trending. Ten to twenty data points for a
parameter are desirable as a minimum.
(6) Develop acceptance levels for the parameters. These levels or limits become the basis for
determining if a parameter is in control or corrective actions are required. First, determine the
boundaries that define the required range for normal operation. These boundaries should be
identified for each parameter from a review of vendersupplied data, test or operational data,
or specifications or requirement documents. Next, determine action limits that fall within
these boundaries in which corrective action will be initiated if the action limits are exceeded.
85
Care should be taken in choosing the action limits so that (1) variation in normal acceptable
operation will not cause the action limits to be exceeded (causing unnecessary expenditure
of resources), and (2) corrective actions can be implemented promptly, once the action limit
is exceeded but before the boundaries for desired normal operation are exceeded. These
action limits should be taken from historical data that represent the same distribution for the
parameter as that in which future measurements will be recorded and tracked.
(7) Analyze the selected parameters for trends. Various statistical and graphical techniques for
performing trend analysis can be found in reference 8.3. Use graphical tools to transform
raw, measured, or calculated data into usable information. The graphical tools can include
scatter plots (sec. 5.1) and control charts (sec. 5.2). Use statistical tools, such as regression
analysis (sec. 6.6), to determine the trend line through a given set of performance data.
Determine how well the trend line fits the data by using techniques such as R
2
or Chi
Square measure of fit tests. These tests are described in detail in reference 8.3 and statistical
textbooks and handbooks. Use the trend line to detect if there is a trend that is approaching
or has exceeded the action limits determined in step 6.
(8) Resolve adverse trends. If an adverse trend is detected, determine the cause of the adverse
trend. Perform correlation analyses (sec. 6.3) to determine what other parameters (factors)
are contributing to the adverse trend. Once the cause of the adverse trend is identified,
propose a remedy to correct the problem before the boundaries for desired normal operation
are exceeded. Implement (management approval may be required) the remedy, then trend
future performance and assess the effectiveness of the remedy.
(9) Report the results. To maximize the benefits of the trend analysis effort, the results should
be documented and distributed to the appropriate levels of management and functional
organizations to ensure corrective actions are implemented in a timely manner once an
adverse trend is detected. Typically, these reports should contain the following items
(adapted from reference 8.2):
a. System element (from step 1).
b. Parameter identification (from step 3).
c. Criticality (from step 4).
d. Data source (from step 5).
e. Failure mode as described in the FMEA.
f. Baseline changes, if applicable.
g. Indication of excluded data, trends, their direction and disposition (adverse or
acceptable).
h. Corrective action used and its effectiveness, if applicable.
i. Need for additional data, if applicable.
j. Recommendations, as necessary.
k. Applicability to other types of trending.
l. Need for additional correlation analysis, if applicable.
86
8.1.4 Example
In a machine shop, the service life of saw blades was studied. The objectives of the study were to
determine the expected life of the blades and develop a methodology to determine when special causes
were effecting machine performance. Performance trend analysis was performed to address both these
questions. Blades are replaced when their performance degrades from 10 to 3 cuts per hour. First,
performance data were collected for 30 blades to statistically establish the expected service life and the
band for expected normal performance.
The daily average cuts per hour for each blade of the 30 blades were measured and recorded until
the 3 cutsperhour limit was reached. A linear regression analysis of these data was performed to
determine the relationship between the cuts per hour and work days. The variation of the 30 blades was
examined for each day of operation. This analysis revealed that the variation grew linearly with time. A
band was established from ± 3 standard deviations from the regression line for each day of operation. The
expected service life range for a given blade was expressed as the time range defined by the regression ± 3
standard deviation band of the regression intercepted the three cutsperhour replacement limit.
The lower (–3 standard deviation) limit of the band was defined as the action limit to ensure the
machine is operating properly. The daily average cuts per hour is tracked for a blade in operation. When
the action limit is exceed, the machine is examined to determine if there is a special cause that is
reducing the blade service life.
The expected band for normal operation and expected service life are illustrated on the performance
trend analysis plot presented in figure 81. The performance of a given blade that has just reached the
end of its service has been tracked on this chart. Note that the action limit is the lower limit of the
expected normal operation band.
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
0
2
4
6
8
10
12
0 5 10 15 20 25
Blade Service Life, Work Days
Blade Replacement Limit
± 3 Std. Dev. Band
(Historical Data)
Expected Service Life
Action Limit
Figure 81. Performance trend analysis example.
87
8.1.5 Advantages
(1) Performance trend analysis can be used to detect a degrading parameter prior to a potential
failure.
(2) This technique can predict future parameter values or estimate the longterm range of
values of influential variables.
(3) The service life of systems or system elements can be predicted.
8.1.6 Limitations
(1) Parameter sensors may need to be installed to obtain trending data—this can be costly.
(2) The operating state, output, or load, about/through which a system/subsystem/component
fluctuates, often cannot be controlled to achieve consistent trend data.
(3) The slope and stability of the data approaching/departing the recorded data point are not
known without using a data buffer.
(4) Data are not always easily quantifiable, limiting the usefulness of this technique.
8.1.7 Bibliography
“ASRM Trend Analysis Requirements Document.” Sverdrup Technology, Inc., Report No. 31400491
115, December 1991.
NASA Technical Memorandum 85840, “The Planning and Control of NASA Programs and Resources.”
NMI 1103.39, “Role and Responsibilities – Associate Administrator for Safety and Mission Quality
(SMQ).”
NMI 8070.3, “Problem Reporting, Corrective Action, and Trend Analysis Requirements.”
NMI 8621.1, “Mishap Reporting and Investigating.”
NHB 5300.4 (1A1), “Reliability Program Requirements for Aeronautical and Space System
Contractors.”
NHB 8070.TBD, “Significant Problem Reporting System.”
Special Study Z001U61, “Marshall Operations Reliability Trend Analysis Standard.” Sept. 16, 1991.
8.2 Problem Trend Analysis
8.2.1 Description
Problem trend analysis, as described in references 8.1 and 8.2, identifies repetitive problems and
assesses how often given problems occur. Also, problem trend analysis provides a mechanism to track
88
progress of problem resolution. Finally, problem trend analysis evaluates organizational proficiency in
preventing repetitive problems. Underlying causes can be uncovered when several problems are
compared. Problem trend analysis is often an early indicator of significant issues in other types of trend
analysis.
There are three basic objectives in problem trend analysis:
(1) Isolate problems to specific causes and examine the frequency of occurrence of these
causes. Problem trending is often initiated on a system level but finished on a component
(or lower) level.
(2) Track problems to determine if occurrence is increasing or decreasing, or if some problems
are affecting other parameters.
(3) Determine if baseline changes or corrective actions increase or decrease the frequency of
problem occurrence.
Data sources for problem trend analysis may include, but need not be limited to:
(1) Failure or problem reporting and corrective action systems such as Problem Reporting and
Corrective Action (PRACA).
8.2
(2) Discrepancy reports (DR’s).
(3) Problems identified by the other four types of trend analysis.
8.2.2 Application
As described in reference 8.2, problem trend analysis is used to identify recurring problems and
assesses the progress in resolving these problems and eliminating the recurrence of the problems. This
analysis is best applied in phase E but may also be applied in phase D. The main interest in this analysis
is locating where the key problems are occurring and the frequency of occurrence. Graphical techniques
such as the Pareto analysis (sec. 5.6) are useful in focusing attention and determining where other
analyses such as performance trend analysis (sec. 8.1) can be beneficial.
Problem trend analysis provides a historical overview of problems in an easytounderstand
graphical format. This overview assists in decisionmaking relative to design effectiveness, process, or
procedural changes over time. Problem trend analysis can be the first step in the initiation of corrective
action to improve system performance.
Basic criteria (from reference 8.2) for the selection of candidate items include:
(1) Problem frequency (establish from historical problem report databases).
(2) Criticality (usually determined from FMEA’s).
(3) Engineering judgment (by cognizant personnel familiar with both the hardware and
requirements).
(4) Unique program or project requirements (these requirements indicate more severe
consequences than normally associated with a given type of problem).
89
8.2.3 Procedures
Procedures (adapted from reference 8.2) to perform problem trend analysis are as follows :
(1) Gather pertinent data. Examine the applicable historical data base(s) and acquire the
appropriate data. These data bases contain information concerning problem reporting. The
data bases are usually maintained by the organization responsible for design and
manufacture of a system element or the operational organization that uses the system.
Typically, searches are made for rejection rates from acceptance testing, operation
problems, and configuration nonconformance. These searches should be performed for a
given time frame. The data bases should be searched for events, operating cycles, hardware
identification codes (i.e., system, subsystem, assembly, subassembly, component or piece
part number), failure mode codes from the FMEA, or key words for given hardware
failures or failure modes.
(2) Identify frequency of problems for the system element under consideration. The system
element may be the subsystem, assembly, subassembly, component or piecepart.
Determine the number of problems (without distinction of failure mode) associated with the
system element during given time periods (i.e., days, weeks, months, years, etc.). Next,
normalize these unrefined frequency data to the number of operations, cycles, missions, or
elements produced during the given time periods. Construct a bar chart (sec. 5.3) for both
the unrefined and normalized data. The unrefined data are plotted as a function of
occurrences versus time, while the normalized data are plotted as a function of occurrence
rates versus time.
(3) Identify primary causes of the problems. For each system element under consideration,
determine the categories of failure modes or causes that induced the problems identified in
step 2. Careful review of the problem reports should be performed to ensure that
inconsistent wording of problem reports by different authors does not mask the true value
of each failure mode or cause. Next, determine the number of occurrences for each failure
mode or cause. Construct a Pareto chart (sec. 5.6) of the number of occurrences versus
failure modes or causes and identify areas of concern. From the Pareto chart, identify the
failure modes or cause of consequence that require further assessment.
(4) Determine if a trend over time exists for each of the identified failure modes or cause of
consequence. Normalize the failure mode or cause as the problems were normalized in step
2 (i.e., normalized by the number of operations, cycles, missions, or elements produced
during the given time periods). Construct a bar chart (sec. 5.3) for each failure mode or
cause. These bar charts should present the total and normalized number of occurrences
versus time. Procedure, process, configuration or design changes and the time of their
implementation should be noted on these charts.
Once the bar chart is generated, fit the normalized failure mode or cause occurrences with
either a linear, exponential, power, logarithmic, or positive parabolic trend line. Determine
the goodness of fit for each trend line model to the data with such statistical methods as the
R
2
test. Refer to reference 8.3 or statistical textbooks or handbooks for details in fitting the
data with trend lines or testing for goodness of fit.
(5) Report the results. Prepare a summary assessment of the problem trend analysis, including:
a. System element (from step 2).
b. Data source, i.e., the historical problem report data base (from step 1).
810
c. Failure modes trended and total number of problem reports assessed.
d. Criticality (from FMEA) of the failure mode(s) and date of last occurrence.
e. Baseline procedure, process, configuration or design changes, if applicable.
f. Chief failure mode or cause of consequence.
g. Indication of trends, their direction and disposition (adverse or acceptable).
h. Corrective action used and its effectiveness, if applicable.
i. Need for additional data, if applicable.
j. Recommendations, as necessary
k. Applicability to other types of trending.
l. Need for additional correlation analysis, if applicable.
8.2.4 Example
The monthly rejection rate of wickets exceeded a company’s goal of 5 units per 1,000 units
produced (0.5 percent) during a 3mo period last year. A problem trend analysis effort was conducted to
understand the reason for the increased rejection rate and to formulate a plan to prevent future excessive
rejection rates. The manufacturing reports for a 1yr production of wickets were reviewed. The results
were summarized by month and are presented in figure 82(a). Also, the monthly production and
rejection rates are shown in figure 82(a).
The cause of each rejection was also identified from the manufacturing problem reports and was
categorized as being due to human error, inadequate properties of raw materials, production machine
malfunctions, or other miscellaneous causes. These results are presented for each month in figure 82(b).
The number of rejections and the rejection rates were plotted on a bar chart and the results are
presented in figure 82(c). The rejection rates were normalized to units produced monthly. As seen on
this chart, the rejection rate exceeded the company goal of 0.5 percent during August, September, and
October; therefore, this time period became the focus of the analysis.
Note from this figure that the normalized rejection rate data, not the absolute number of
rejections, indicate the time period of concern.
A Pareto chart (shown in figure 82(d)) was produced for the entire year to establish the
significance of each cause for rejection. This chart revealed that human error was the most significant
cause for rejection over the entire year period. However, a Pareto chart generated for the 3mo period of
concern, revealed that inadequate material properties was the most significant cause for unit rejection.
Comparing the two Pareto charts shows that inadequate material properties was a much more significant
problem during the 3mo period, and that human error was over the entire year. This chart for the 3mo
time period is presented in figure 82(e).
The number of rejections and the rejection rates due to inadequate properties of raw materials
were plotted on a bar chart and the results are presented in figure 82(f). The rejection rates were
normalized to units produced monthly. As seen on this chart, the increase in the rejection rate due to
inadequate material properties was the driving factor in exceeding of the maximum rejection goal.
811
Month Units Produced Units Rejected Rejection Rate
(Units/1,000 Produced)
Jan. 5,100 12 2.35
Feb. 4,600 21 4.28
Mar. 4,900 16 3.26
Apr. 2,900 12 4.14
May 3,150 13 4.12
Jun. 3,050 10 3.27
Jul. 3,000 12 4.00
Aug. 1,700 14 10.35
Sep. 1,400 14 9.65
Oct. 1,750 15 8.57
Nov. 3,100 9 2.90
Dec. 4,950 21 4.24
(a) History of unit rejections.
Causes for Unit Rejection
Month Human
Error
Inadequate
Material
Properties
Machine
Malfunction
Other Units
Rejected
Jan. 6 2 3 1 12
Feb. 10 4 5 2 21
Mar. 8 3 4 1 16
Apr. 6 3 3 0 12
May 6 2 4 1 13
Jun. 5 2 3 0 10
Jul. 6 2 2 2 12
Aug. 3 10 1 0 14
Sep. 3 9 1 1 14
Oct. 4 9 2 0 15
Nov. 5 2 2 0 9
Dec. 10 5 5 1 21
Total 72 53 35 9 169
(b) History of unit reject by cause.
Figure 82. Problem trend analysis example—Continued
812
J
A
N
F
E
B
M
A
R
A
P
R
M
A
Y
J
U
N
J
U
L
A
U
G
S
E
P
O
C
T
N
O
V
D
E
C
0
5
10
15
20
25
0
5
10
15
20
25
N
u
m
b
e
r
o
f
U
n
i
t
s
R
e
j
e
c
t
e
d
R
e
j
e
c
t
i
o
n
R
a
t
e
,
U
n
i
t
s
/
1
,
0
0
0
P
r
o
d
u
c
e
d
Time, Months
Units Rejected
Rejection Rate
(c) Total unit rejection and rejection rate versus time.
Human Error Inadequate
Material
Properties
Machine
Malfunctions
Other
0
20
40
60
80
100
0
20
40
60
80
100
N
u
m
b
e
r
o
f
U
n
i
t
R
e
j
e
c
t
i
o
n
s
P
e
r
c
e
n
t
Causes of Unit Rejections
(d) Pareto chart of causes for entire period.
Figure 82. Problem trend analysis example—Continued
813
Inadequate
Material
Properties
Human Error Machine
Malfunctions
Other
0
5
10
15
20
25
30
35
40
0
20
40
60
80
100
N
u
m
b
e
r
o
f
U
n
i
t
R
e
j
e
c
t
i
o
n
s
P
e
r
c
e
n
t
Causes of Unit Rejection
(for August, September, and October only)
(e) Pareto chart of causes for period of concern.
J
A
N
F
E
B
M
A
R
A
P
R
M
A
Y
J
U
N
J
U
L
A
U
G
S
E
P
O
C
T
N
O
V
D
E
C
0
2
4
6
8
10
12
0
2
4
6
8
10
12
N
u
m
b
e
r
o
f
U
n
i
t
s
R
e
j
e
c
t
e
d
R
e
j
e
c
t
i
o
n
R
a
t
e
,
U
n
i
t
s
/
1
,
0
0
0
P
r
o
d
u
c
e
d
Time, Months
Units Rejected
Rejection Rate
(f) Unit rejection and rejection rate (due to inadequate material properties) versus time.
Figure 82. Problem trend analysis example—Continued.
814
Further analysis showed that a statistically significant larger portion of the units rejected for
material properties came from one lot of materials used during the August to October period. This lot
met acceptance test criteria, however it was by a narrow margin. To avoid further high rejection rates,
the specifications for the raw material were tightened as were the corresponding acceptance tests.
8.2.5 Advantages
(1) Problem trend analysis can provide an early indicator of significant issues in other types of
trend analysis.
(2) This technique examines the frequency of problem occurrence, monitors the progress of
problem resolution, uncovers recurring problems, and assesses the effectiveness of
recurrence control.
8.2.6 Limitations
Candidate items should be chosen carefully because the analysis can be costly or noninformative
if performed for all potential problem areas.
8.2.7 Bibliography
“ASRM Trend Analysis Requirements Document.” Sverdrup Technology, Inc., Report No. 31400491
115, December 1991.
NASA Technical Memorandum 85840, “The Planning and Control of NASA Programs and Resources.”
NHB 5300.4 (1A1). “Reliability Program Requirements for Aeronautical and Space System
Contractors.”
NHB 8070.TBD, “Significant Problem Reporting System.”
NMI 1103.39, “Role and Responsibilities  Associate Administrator for Safety and Mission Quality
(SMQ).”
NMI 8070.3, “Problem Reporting, Corrective Action, and Trend Analysis Requirements.”
NMI 8621.1, “Mishap Reporting and Investigating.”
Special Study Z001U61, “Marshall Operations Reliability Trend Analysis Standard.” Sept. 16, 1991.
8.3 Programmatic Trend Analysis
8.3.1 Description
Programmatic trend analysis, as described in references 8.1 and 8.2, is concerned with
organizational or programmatic issues that may impact safety or system success. These issues include
general program health, schedule issues, overtime or sick time usage, production bottlenecks, accidents
or equipment damage, supply of critical skills (critical resource scheduling), cost of upkeep versus
redesign or reprocurement, noncompliances, and cost of replacement versus cost of repair.
815
8.3.2 Application
Programmatic trend analysis is best applied in phase E. The objective of programmatic trend
analysis is to provide management a status on programmatic issues or early warning of programmatic
problems. For example, warning of inappropriate demands on manpower, impending delays, mismatches
between demand and available expertise, alerting management on areas needing attention (e.g., damage,
injury or accident frequency), supporting program/project improvement changes, support management in
monitoring project management performance indicators over time to indicate endproduct safety and
reliability.
Common candidates for programmatic trend analysis include the following:
(1) “Manpower strength by speciality, experience, qualification, certification, and grade.”
8.2
(2) “Schedule changes/slippages or overages.”
8.2
(3) Accident or sick time frequency.
(4) “Overtime usage versus approved policy.”
8.2
(5) Labor problems.
(6) “Requirement changes, including waivers and deviations.”
8.2
(7) “System nonconformances and problems due to human error.”
8.2
(8) “Rework expenditures.”
8.2
(9) Time/cost considerations for redesign.
Concerns (in terms of risk, safety, cost, availability, or schedule) and expected benefits should be
the basis for setting priorities when considering using programmatic trend analysis.
8.2
Some programmatic trend data will be obtained from other parameters; however, some
parameters will be unique to programmatic trends. Trending parameters and supporting data to be
recorded and trended that would have a programmatic impact must be selected.
8.3.3 Procedures
As described in reference 8.2, apply the following steps to perform the programmatic trend
analysis:
(1) Determine the programmatic parameters to be assessed. Determine which programmatic
parameters will be trended. Common parameters that are trended are presented in section
8.3.2. However, the selection of parameters should be determined by the unique needs of
the organization or program involved. Maintain a list of parameters for which
programmatic data are to be supplied.
(2) Acquire and compile data for the selected parameters. Data sources (adapted from reference
8.2) for programmatic parameters include, but are not be limited to:
816
a. Planned versus actual cost reports (so that number and magnitudes of cost overruns
and underruns can be determined).
b. Planned versus actual schedule charts (so that the number and magnitude of schedule
delays and accelerations can be determined).
c. Quality assurance reports (documenting the number of noncompliances).
d. Development and verification status reports (documenting the success or failure in
verifying system requirements or specifications).
e. Inventory control records (documenting the number of times work was delayed due to
unavailable material).
f. Facility, equipment, and hardware problem and corrective action reports.
g. Acceptance records (documenting number of units produced that were accepted or not
accepted by the customer).
h. Shipping and receiving logs (including planned versus actual shipping and receiving
dates).
i. Work authorization and control documents.
j. Planned versus actual staff level reports.
k. Safety, mishap, or incident reports.
(3) Ensure the validity of the data. Care should be taken to ensure the data analyzed are
accurate and are an appropriate measure for the programmatic parameter being trended.
(4) Develop the required analytical techniques and controls (e.g., Pareto charts (sec. 5.6) and
histograms (sec. 5.7)). Action limits should be establish in which corrective action will be
initiated if the action limits are exceeded. Action limits can be set to ensure parameters stay
within the operating and administrative policies and procedures, work standards, and goals
of the organization.
(5) Determine the structure for project data collection, maintenance, and reporting. Identify the
organizations and personnel responsible for collecting, maintaining, assessing, and
reporting the data.
(6) Make data available to program management.
(7) Analyze the data for trends. Use control charts (sec. 5.2) to display the historical trends of
validated data for the programmatic parameters being measured, along with the realistic
action limits established.
(8) Resolve adverse trends. When an adverse trend has been identified, conduct an analysis of
that trend. Preparing a cause and effect diagram (sec. 7.2) may be useful in identifying the
root cause of the adverse trend. Once the cause of the adverse trend is identified, propose a
remedy to correct the problem before the boundaries for desired normal operation are
exceeded. Implement the remedy (management approval may be required), then trend
future performance for the programmatic parameter and assess the effectiveness of the
remedy.
817
(9) Report the results. The reports should be published at intervals that will allow management
to take prompt action to correct problems before they become unmanageable. The reports
should contain sufficient details so that management can accurately assess the risk
associated with an averse trend. Suggested reporting formats for common programmatic
parameters can be found in reference 8.2.
8.3.4 Example
At the start of a new program, candidate parameters were identified for programmatic trend
analysis. The list was reviewed by both the project team and management, and trending parameters were
selected. Arrangements were made for data to be collected and assessed for each parameter. Action
limits were determined from company policies and procedures and program requirements
documentation.
The following example illustrates how programmatic trend analysis was applied for a specific
programmatic parameter—overtime usage. Review of the company policy revealed that the average
overtime rate for a project with more than the equivalent of 100 fulltime workers should not exceed 10
percent per month. This particular program average staffing level was 125. An action limit of 8 percent
per month maximum overtime rate was established. If this action limit is approached or exceeded,
management should be notified and corrective action taken.
The actual overtime rate, expressed in percentage versus month worked, is presented for 1991 in
figure 83. As seen in this figure, the overtime rate exceeded the action limit in May. Management was
notified and overtime usage was reviewed. The cause for the increased rate was due to new negotiated
work to be performed. However, the scheduled completion date for the project had remained fixed.
Overtime projections revealed that the overtime rate would range from 10 to 13 percent for the
remainder of the calendar year.
Work was identified that could be subcontracted. This work was approximately 6 percent of the
total project. Management agreed to subcontract the work starting in midJune. Tracking the overtime
usage rate past the time the corrective action was implemented revealed that the fix of the programmatic
problem was effective (as shown in fig. 8.3).
8.3.5 Advantages
The programmatic trend analysis technique monitors programmatic posture and provides visibility to
determine the current/projected health of the human support element.
8.3.6 Limitations
The data collection process can be extensive because of a potentially large and varied number of
sources.
8.3.7 Bibliography
“ASRM Trend Analysis Requirements Document.” Sverdrup Technology, Inc., Report No. 31400491
115, December 1991.
NASA Technical Memorandum 85840, “The Planning and Control of NASA Programs and Resources.”
818
0
2
4
6
8
10
12
1991
Company Policy Limit
Action Limit
Surplus work
subcontracted.
Months
Figure 83. Programmatic trend analysis example.
NHB 5300.4 (1A–1), “Reliability Program Requirements for Aeronautical and Space System
Contractors.”
NHB 8070.TBD, “Significant Problem Reporting System.”
NMI 1103.39, “Role and Responsibilities – Associate Administrator for Safety and Mission Quality
(SMQ).”
NMI 8070.3, “Problem Reporting, Corrective Action, and Trend Analysis Requirements.”
NMI 8621.1, “Mishap Reporting and Investigating.”
Special Study Z001U61, “Marshall Operations Reliability Trend Analysis Standard.” Sept. 16, 1991.
8.4 Supportability Trend Analysis
8.4.1 Description
Supportability trend analysis, as described in references 8.1 and 8.2, is performed to evaluate the
proficiency of an organization at controlling the logistics factors supporting a program. Logistic concerns
likely to be trended are supplies of spare parts, replaceability, frequency of cannibalization, late deliveries,
819
shortages, maintenance, etc. Typically, data used for supportability trend analysis are not in a form that is
readily usable. Processing certain data is laborintensive or may not be feasible due to contractual
considerations. Sometimes indirect or related parameters may be used to indicate supportability.
8.4.2 Application
The supportability trend analysis technique is best applied in phase E. This analysis assesses the
effectiveness of logistics factors (extracted from reference 8.2) such as the following:
(1) Maintenance.
(2) Supply support.
(3) Facilities management and maintenance.
(4) Support personnel and training.
(5) Packaging, handling, storage, and transportation.
(6) Technical data support.
(7) Automated data processing hardware/software support.
(8) Logistics engineering support.
Supportability trend analysis monitors the current status of the support systems and forecasts the
future status in order to resolve problems with minimum adverse effect. The current support systems are
analyzed in order to estimate the future requirements of the systems. Also, support elements that can be
improved are identified and the effects on the supportability of other program factors are determined.
Another application of supportability trend analysis is to optimize system availability over
operating life. This is done by identifying the support elements that can be improved. Also, the effects
of system reliability and maintainability on supportability are measured, and areas for improvement are
identified.
Candidates used to evaluate system reliability/maintainability/availability support characteristics
include the following:
8.2
(1) Meantimebetweenfailures (MTBF)
(2) Meantimetorepair (MTTR)
(3) Meantimebetweenrepairs (MTBR).
Concerns (in terms of risk, safety, cost, availability, or schedule) and expected benefits should be
the basis for setting priorities when considering using supportability trend analysis.
8.2
Supportability trending parameters should be selected that indicate the effectiveness of the
support elements and the maintainability design factors. Operations and support systems should be
analyzed, if support degrades, to identify items that could lead to a system failure, schedule delay, or
cost increase.
820
8.4.3 Procedures
The procedures (adapted from reference 8.2) to perform supportability trend analysis are as
follows:
(1) Assess the overall operation. Identify parameters that could indicate impending system
failure, cost impacts, and schedule slippages if support functions deteriorate.
(2) Select parameters to be trended. Determine which parameters (identified in step 1) can best
be used to evaluate whether support functions are varying at a sufficient rate to require
management attention. Special consideration should be given to parameters that predict
system safety or success.
(3) Determine if quantitative data are available and adequately represent these parameters.
Supportability parameters may be derived directly from measurements or from calculations
involving two or more measurements. If measurement data are not available, develop a
system to measure the data or eliminate the parameter from the list to be trended.
(4) Establish acceptance limits for the selected parameters. These levels or limits become the
basis for determining if a parameter is in control or corrective action is required. First,
determine the acceptance levels and minimum baselines that define the required level of
support for normal operation. Acceptance limits and minimum support baselines should be
taken directly from program or project support requirements documentation. These
boundaries can also be determined from review of operation, maintenance, and logistics
manuals, and design requirements and specifications documents.
Next, determine action limits that fall within these boundaries, for which corrective action
will be initiated if the action limits are exceeded. Care should be taken in choosing the
action limits so that (1) variation in normal acceptable operation will not cause the action
limits to be exceeded (causing unnecessary expenditure of resources), and (2) corrective
actions can be implemented promptly, once the action limit is exceeded, but before the
boundaries for required support for normal operation are exceeded.
(5) Gather, measure, or calculate the data to be used to trend the selected supportability
parameters. Data sources (extracted from reference 8.2) for supportability trend analysis
may include, but need not be limited to:
a. Equipment problem reports.
b. Work authorization documents.
c. Contractual acceptance records.
d. Shipping and receiving reports.
e. Payment records for maintenance.
f. Transportation records.
g. Inventory records.
h. Issues and turnin records.
i. Training course attendance records.
821
j. Technical documentation error reporting.
k. Consumable replenishment records.
(6) Analyze the selected parameters for trends. Various statistical and graphical techniques for
performing supportability trend analysis can be found in reference 8.3. Use graphical tools
to transform raw, measured, or calculated data into usable information. These graphical
tools can include scatter plots (sec. 5.1), bar charts (sec. 5.3), and control charts (sec. 5.2).
Use statistical tools, such as regression analysis (sec. 6.6), to determine the trend line
through a given set of performance data. Determine how well the trend line fits the data by
using techniques such as R
2
or ChiSquare measure of fit tests. These tests are described in
detail in reference 8.3 and statistical textbooks and handbooks. Use the trend line to detect
if there is a trend that is approaching or has exceeded the action limits established in step 4.
(7) Resolve adverse trends. When an adverse trend has been identified, conduct an analysis for
that trend. A cause and effect diagram (sec. 7.2) may be useful in identifying the root cause
of the adverse trend. Once the cause of the adverse trend is identified, propose a remedy to
correct the problem before the boundaries for required support of normal operation are
exceeded. Implement the remedy (management approval may be required), then continue to
trend the supportability parameter and assess the effectiveness of the remedy.
(8) Report the results. The reports should be published at intervals that will allow management
to take prompt action to correct support problems before they become unmanageable. The
reports should contain sufficient details so that management can accurately assess the risk
to normal operation due to an adverse trend. Suggested reporting formats for common
supportability parameters can be found in reference 8.2.
8.4.4 Example
The following example illustrates supportability trend analysis for inventory control of a specific
spare part. Review of the project support requirements document revealed that at least eight spare parts
were always required. To ensure the inventory never reached this level, an action limit of 10 spare parts
was established. The inventory level for the parts for 11 months in 1988 and 1989 is presented in figure
84. As seen in this figure, the inventory level reached the action level in August 1988. Assessment of
the cause for the low inventory level revealed that usage of the spare parts did not increase, however,
more parts received from the vendor were being rejected in acceptance tests. The corrective action was
to change vendors for the parts. This occurred in September 1988. Tracking the inventory level past the
time the corrective action was implemented revealed that the fix to the support problem was effective.
8.4.5 Advantages
Supportability trend analysis monitors the current health of support systems and forecasts
support problems to enable resolution with minimum adverse effect.
8.4.6 Limitations
Determining the extent of analysis and identifying the appropriate parameter variations that must
be measured can be difficult.
822
M
A
R
A
P
R
M
A
Y
J
U
N
J
U
L
A
U
G
S
E
P
O
C
T
N
O
V
D
E
C
J
A
N
F
E
B
0
5
10
15
20
S
p
a
r
e
P
a
r
t
s
1988  1989
Action Limit
Minimum Required
Inventory Limit
New vendor acquired.
Months
Figure 84. Supportability trend analysis example.
8.4.7 Bibliography
“ASRM Trend Analysis Requirements Document.” Sverdrup Technology, Inc., Report No. 31400491
115, December 1991.
NASA Technical Memorandum 85840, “The Planning and Control of NASA Programs and Resources.”
NHB 5300.4 (1A–1), “Reliability Program Requirements for Aeronautical and Space System
Contractors.”
NHB 8070.TBD, “Significant Problem Reporting System.”
NMI 1103.39, “Role and Responsibilities – Associate Administrator for Safety and Mission Quality
(SMQ).”
NMI 8070.3, “Problem Reporting, Corrective Action, and Trend Analysis Requirements.”
NMI 8621.1, “Mishap Reporting and Investigating.”
Special Study Z001U61, “Marshall Operations Reliability Trend Analysis Standard.” Sept. 16, 1991.
823
8.5 Reliability Trend Analysis
8.5.1 Description
Reliability trend analysis, as described in reference 8.1, performs a parametric assessment of
factors affecting system reliability. The objectives of reliability trend analysis are to measure reliability
degradation or improvement, to predict an outofline failure, to verify design certification limits, to
determine life limits, and to evaluate inspection intervals. Although some parameters will be unique to
reliability, many parameters pertaining to reliability trending also pertain to performance or problem
trending.
Data sources for reliability trend analysis might include new, refurbished, and repaired
component and subassembly acceptance inspection, checkout, and test data for development and
verification and production hardware including, but not limited to:
(1) Alignment data.
(2) Contamination data.
(3) Dimensional data.
(4) Nondestructive test data, e.g., magnetic particle, radiography, penetrant, and ultrasonic
data.
(5) Proof test data, e.g., leak check and hydroproof data.
(6) Functional or performance data, e.g., quantitative and qualitative data.
8.5.2 Application
Reliability trend analysis is best applied in phase E but may also be applied in phase D.
Reliability trending parameters should be selected to indicate changes in the reliability of a system and
explain their causes. These parameters could also be performance or problem trending parameters or
strictly reliability parameters. The criteria for selecting parameters should consider criticality, problem
frequency, engineering judgment, etc. as deemed necessary. Trending parameters should be selected, as
applicable, for each system, subsystem, or component by:
(1) For each parameter, reviewing the FMEA/CIL, contract end item specification, limitedlife
items lists, previous problem reports, original equipment manufacturer’s data, equipment
acceptance data, operations manuals, etc. to determine if it is necessary or beneficial to
perform reliability trending.
(2) Determining the product life indicators necessary to determine the health of the system,
subsystem, or component, e.g., MTBF.
(3) Determining the failure modes pertinent to the system, subsystem, or component.
(4) Determining if time/cycle and failure data are available. Typically, at least 10 failures are
necessary to perform a reliability trend analysis; however, an action limit can be set to
indicate a “failure” data point. At least half of the time/cycle intervals should have an
average of at least one “failure” per time period. (For example, if six time intervals of 2 yr
are chosen, at least three intervals should have at least two failures.) Design/process change
data should be available.
824
(5) If necessary data are not available (e.g., failure and time/cycle data), consider the addition
of data sensors, obtaining alternate data, changing the parameter, or using engineering
judgment for the trend analysis.
(6) Determining if the parameter concerns reusability/repairability or a onetime failure.
8.5.3 Procedures
The only differences between performance and reliability trend analysis are the parameters
trended. Therefore, the procedures to perform reliability trend analysis are same as presented in section
8.1.3 for performance trend analysis.
8.5.4 Example
This example is a plot of reliability trending where the CSF is plotted for the first 20 flight
motors. The lines for the mean ± 3 standard deviations are based on the motors up to number 50 and
give an indication that the later motors have a higher CSF than the first motors.
Also plotted is the minimum CSF allowable by specification (1.5) that shows that this station is
well above the requirement. Most stations do not lie this far above the minimum CSF value.
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
2 4 6 8 10 12 14 16 18 20
0
5
10
15
20
Flight Sequence Number
Upper 3 Limit
Lower 3 Limit
Minimum Specification Limit
Mean
σ
σ
Figure 85. Reliability trend analysis example.
8.5.5 Advantages
Reliability trend analysis measures reliability degradation or improvement and enables the
prediction of possible failures so action can be taken to avert failure.
825
8.5.6 Limitations
Candidate items must be chosen carefully because the analysis can be costly if performed for all
potential problem areas.
8.5.7 Bibliography
“ASRM Trend Analysis Requirements Document.” Sverdrup Technology, Inc., Report No. 31400491
115, December 1991.
NASA Technical Memorandum 85840, “The Planning and Control of NASA Programs and Resources.”
NHB 5300.4 (1A–1), “Reliability Program Requirements for Aeronautical and Space System
Contractors.”
NHB 8070.TBD, “Significant Problem Reporting System.”
NMI 1103.39, “Role and Responsibilities – Associate Administrator for Safety and Mission Quality
(SMQ).”
NMI 8070.3, “Problem Reporting, Corrective Action, and Trend Analysis Requirements.”
NMI 8621.1, “Mishap Reporting and Investigating.”
Special Study Z001U61, “Marshall Operations Reliability Trend Analysis Standard.” Sept. 16, 1991.
826
REFERENCES
8.1 Babbitt, N.E., III: “Trend Analysis for Large Solid Rocket Motors.” AIAA 923357, Sverdrup
Technology, Inc., Huntsville, AL, 1992.
8.2 NHB 8070.5A, “NASA Trend Analysis and Reporting.” Draft, December 1991.
8.3 NASA–STD–8070.5A, “Trend Analysis Techniques.” NASA System Assessment and Trend
Analysis Division, Office of the Associate Administrator for Safety and Mission Quality, October
1990.
827
APPENDIX A
TOOL OR METHODOLOGY SUGGESTION FORM
A1
A2
Toolbox or Methodology Suggestion Form
A3
APPENDIX B
CASE STUDY:
TRIALS AND TRIBULATIONS OF USING SYSTEM ENGINEERING TOOLS
B1
B2
CASE STUDY:
TRIALS AND TRIBULATIONS
OF USING SYSTEM ENGINEERING TECHNIQUES
The Assignment
Charlie Smith came in to work early Monday morning. And why not? He was excited! He’d just
completed a course in System Engineering where they’d shown him all these “techniques” to make his
job easier and less subjective. He’d known about some of the techniques. But he wished he’d had the
course about 10 years ago—back when he was just starting as a systems engineer. Well, no matter...
Today his boss was going to give him a new assignment, and he’d show all of them his newlyfound
proficiency with the toolbox. This should be easy...
His boss, Mr. Jones came in about 9. It had been hard on Charlie, waiting that long, but he had
used the extra time to read his mail, do a little filing, return his phone messages, and write a report.
“Hmmm,” he thought, “maybe I came in a little too early...”
Aw well, his boss, Jones, had finally made it. Without exchanging pleasantries, Jones gave him a
package—“It’s your new assignment, but I don’t see how you’ll do it. The boss wants everything
measurable this time—wants to see how things were decided. Good luck—let me know how it goes.”
With that, Jones left and Smith tore open the package. “A Handson Science Museum display suitable
for grades K–12, for the Museum’s Chemistry Section.” Since Smith was a designer of aerospace
hardware, he really wasn’t sure about this one. What was he supposed to do? What were the ground
rules? Why get this assignment now, just when he was prepared to use all his training to really produce
real hardware, not a vague illdefined thing like this? Smith decided to talk to his boss—this day wasn’t
getting any better.
Jones’ secretary let him know that Jones was gone for the day. He went back to his desk and
found a message from the System Engineering class instructor, Ms. Doe. Puzzled, he called her back,
but he was so preoccupied with his new task that he started right in talking about it.
“Can you imagine, I thought I’d get to use those techniques to build something. Guess I’ll have
to finish this task first though. I think I’ll just pick something and press on. I don’t think the tools apply
here, you know? It’s not really defined enough and I don’t really think the data even could exist, much
less that I could get it. I mean, with a problem like this, there really aren’t any 'data’ to look at anyway!”
Charlie was getting himself kind of worked up.
Ms. Doe (Jane to her friends) almost laughed when she replied, “Buck (Smith’s boss—no one
knew his real name) asked me to call you because he thought you might react this way. Now remember
what we talked about in class. Every problem seems this way at first. The techniques really do add
value, as long as you think about why you’re using them. Tell you what, why don’t you look through the
phase A group, think about it, and we can talk it through tomorrow?”
B3
Phase A—If at First You Don’t Succeed...
After calming down a bit, Smith agreed that this was worth a try, but he really didn’t think it
would work. He hung up, asked his secretary to hold his calls (he liked doing that) and brought out his
matrix. “Let’s see... trade studies, costversusbenefit studies, risk assessment matrix...” No wait, that
risk assessment thing was a secondary for phase A. He carefully crossed it off his list and continued
“benchmarking, cause and effect, checklists, and quality function deployment,” all no good, they were
secondaries. That left brainstorming, Delphi technique, and nominal group technique. Well, that made
five techniques for him to use. Too bad about quality function deployment—he really liked that one, but
he’d follow the priorities Ms. Doe had set—after all, she ought to know.
Smith wanted to be systematic, so he placed the five techniques down in alphabetical order on a
piece of paper:
Brainstorming
Costversusbenefit studies
Delphi technique
Nominal group technique
Trade studies.
He’d start with brainstorming first. Jones was about to ask his secretary to call together his group
when he started feeling a little silly about asking for their help. After all, he wasn’t sure himself what he
wanted and didn’t want to look stupid to his employees. “If only this assignment had been better...,” he
thought. Anyway, (he just wanted to get this done!) he began to brainstorm by himself.
About an hour later he decided it wasn’t going very well. He had been to museums like the one
described in his project, and he was a chemical engineer by trade—but so far he just had a page listing of
potential chemistryrelated topics:
(A) Types of Matter:
Solid – materials
Liquid
Gas
Plasma
(B) Types of materials:
Metal
Ceramic
Polymer
Glass
(C) Building blocks for materials
Largest building blocks for materials
Molecules
Macromolecules, molecules
Smaller building blocks for materials
Atoms
Electrons
Neutrons
Protons
Subatomic particles
Quarks, etc.
B4
(D) Chemistry
(E) Designing Materials.
He didn’t know what kind of exhibit this might make, and brainstorming wasn’t going well. He
remembered from the class that brainstorming was best performed in a group, but he thought again about
looking foolish. Well, maybe he’d try the next technique—let’s see, that was costversusbenefit studies.
There was clearly no way to do costversusbenefit until he figured out what exhibit he was
going to build. He remembered from the class that that technique required data, and he didn’t have any.
He decided not to waste any more time on costversusbenefit—he’d be well prepared to argue with Jane
in the morning.
The next two methods, Delphi technique and nominal group technique, fell to similar fates. He
wasn’t really familiar with them. She must have taught them when he was out, checking in with his
office. That was OK because trade studies, the final technique, was one he liked and had used before.
Smith began by asking himself what the desirable features of a handson museum science exhibit for
chemistry were. He prepared a listing.
Features:
(1) Should be fun as well as educational.
(2) Should accommodate crowds—not just one person (or if one person, should have pretty
good throughput).
(3) Should be sturdy.
(4) Should have to do with chemistry (he’d almost forgotten that one!).
(5) Should fit in a space of...? (he’d have to find out).
(6) Must be ready by...? (he’d have to find out).
(7) Must cost no more than...? (he’d have to find out).
(8) Should be interesting to ages 5 through 18.
(9) Should have minimal consumables (he knew, from experience, that consumables could
destroy a budget).
After writing down these features, Smith realized that features 1 and 8 were kind of motherhood
statements, so he took another cut at figuring what he meant by these:
(1) Should be fun as well as educational.
What makes things fun for kids?
a. It should involve activity, not passive participation (nothing that the kids just watch).
The younger children might have more activity (crawling, climbing, jumping, running
etc.) than the older ones.
B5
b. It might involve winning or high scores or a competition of some sort.
c. It might involve their making something—visible accomplishment was usually fun.
d. It could involve testing of their physical or cognitive powers, but should have a
black/white answer.
e. It should not be perceived as childish—must be something an adult would do—don’t
insult the kids!!!
f. Moving parts were good—he might want to stay away from solely computerized
stuff.
g. High tech was good, maybe having to do with exploration or environmental themes—
tie it to something they knew something about.
h. If he wanted to get them to do it again and again, it should be something where they
might measure improvement, or at least get a different result with a different effort or
problem.
Smith sat back, well pleased with this listing. He realized that feature 8 was pretty well covered
by his current list and decided not to work on it separately. He wanted a little refresher on the trade study
methodology before he went on, so he pulled out his toolbox. Let’s see... page 23 (section 2.1.3) said to
“Define the mission objective and requirements for the system under consideration.” All right, he’d done
that, now what? “Identify credible alternative candidates for the system under consideration”—that’s
what the toolbox said, but how could he do that when he didn’t know what he was building? This
toolbox thing wasn’t as helpful as he thought. Smith packed up for the day and headed home—tomorrow
he was going to have a serious talk with Jane. She clearly hadn’t taught this stuff right and anyway, why
was Buck calling her about his new assignment, and why couldn’t it have been a better one, and... Oh
well, he’d save all that for tomorrow.
Phase A—...Try, Try Again
It was a bleak, rainy Tuesday morning. Smith’s brooding sleep had been interrupted often by the
sharp concussions of thunderstorms. He was going to be ready for Jane Doe! He arrived at work and
pulled together his files. His secretary had managed to get some additional information on the science
exhibit—the space allowable was approximately 3,000 ft
2
, and his timeframe was approximately 18 mo
until, it, the museum opened. She had left a note saying that there was still no hard data on his budget
but it would likely be on the order of $400,000. Well, that was something anyway. He checked his
calender and found that Jane Doe would be there in about 15 min. He used the time to prepare himself.
Jane arrived on time, wet, and most infuriating of all, cheerful. “So how did it go yesterday?”
Smith began, in a controlled but bitter tone: “Poorly. The tools didn’t work very well—there isn’t
enough data. I went systematically by your procedure—which by the way eliminated some things I
thought would be very useful—and I don’t think I’ve made any real progress. Another thing...”
B6
Jane interrupted him here, with that nowveryannoying laugh, “Slow down, slow down, let’s
take it one thing at a time. I’ve got all morning, and I think we can make this work. If not, I’ll talk to
Buck about it. Deal?”
Smith couldn’t say “no” to that. He figured with just one morning’s effort he’d be able to show
Jane that this wasn’t going to work—then it would be someone else’s problem. “Deal.”
They sat down at Smith’s desk. He asked his secretary to hold all calls. (He really liked doing
that.)
Smith showed Jane the results of his technique downselection—the alphabetized listing he’d
taken from the course toolbox. Jane began, “Charlie, you have to remember, the matrix on page 17 is
just a guide. You didn’t have to use all of those techniques that were marked priority 1, or ignore the
priority 2’s and the unmarked items for that matter. But, since that’s how you started, how did it go?”
Jane wasn’t wasting any time.
Smith told her of his difficulties in brainstorming and his concern for calling a team together
before he had anything planned out. She acknowledged that this was one of the shortcomings of the
brainstorming technique, and she understood—but didn’t seem to agree with—his reluctance to pull a
team together. She didn’t want to talk about costversusbenefit—she agreed that it wasn’t yet
appropriate and congratulated Smith on not trying to forcefit the technique to an inappropriate
application. This was not what Smith had expected. They skipped quickly over the next two techniques.
She explained they were sort of variations on brainstorming anyway, and got right to his trade study.
Smith was quite confident of his knowledge of this technique and felt secure that he could show Jane
once and for all that this project was just inappropriate for his newly mastered skills.
Jane read his nine features without comment. She then looked at his breakout for feature 1 and
frowned a bit. Smith didn’t want to lose his opportunity (she was turning out to be pretty hard to corner).
And he didn’t like that frown one bit. As soon as she looked up he let her have it. “You see, I followed
the guideline for trade studies—and by the way, I’ve been using them for many years—and couldn’t get
past the second step. How do I know what specifics to trade when I don’t have any specifics? And how
can I develop any specifics without data? I just don’t see how this thing is supposed to work!”
Jane’s response surprised Charlie. “These techniques are only to be used where they can help,
and you’re the only one who can decide where that is. They don’t replace data. In fact, many of them
may highlight where data are required, or just how much you don’t know. But, with your specific
problem, I have a few questions. I would have thought things like safety and access for handicapped
would be highpriority features. Also, what about education—you’ve analyzed some things that clarify
fun but what are the specific educational aspects that you’d like to focus on? I think a focus on that
might help a lot.”
Charlie knew that throughout the class she’d discussed using the toolbox as a guideline, and that
it wasn’t necessary to use each technique. He just hadn’t trusted his own knowledge of the toolbox well
enough to select against the toolbox guidance—cookbooks and gobys were a lot easier. OK, he’d give
her that one. That bit about safety and handicapped access—those were good and he added them to his
listing as features 10 and 11, respectively. As for the educational aspects, that was a great observation.
Together they began to make up a listing. It didn’t go very well at first, so they called in Dalton from the
Applied Chemistry department. After about an hour, they had a listing for the top 15 educational areas
that they wished to focus on:
B7
Educational Areas:
(1) Demonstrate units of mass, length, volume, temperature etc.
(2) Demonstrate intrinsic (color, viscosity, melting point, hardness, density...) versus extrinsic
properties (size, shape temperature...). Note intrinsic properties are also known as physical
properties.
(3) Demonstrate chemical properties (the tendency of the substance to change, through
interactions with other substances or singly).
(4) Demonstrate chemical change (new substance is formed) vs. physical change—include
exothermic and endothermic changes.
(5) Demonstrate elements, compounds, mixtures, and solutions.
(6) Demonstrate the states of matter: solid, liquid, gas, plasma.
(7) Demonstrate the laws of conservation of mass and energy.
(8) Provide a feel for Avogadro’s number.
(9) Demonstrate crystalline nature of many solids.
(10) Demonstrate the nature of polymer chains.
(11) Demonstrate the nature of metals and semiconductor materials.
(12) Demonstrate the principles of catalysis.
(13) Demonstrate the principles of combustion.
(14) Demonstrate the special nature of organic chemistry.
(15) Demonstrate the standard and quantum theory for the atom.
Smith knew from experience that the next thing to do was to combine these with the features
listing and see which areas were likely to make exhibits and which might combine, etc. But this sounded
like a combination of brainstorming and trade studies and checklist all twisted together. He asked Jane if
that was all right. She explained that there was no problem, as long as what they did was documented
and reasonably systematic. Charlie felt more like he had while in class—he was starting to get the hang
of this, again. They decided to brainstorm potential exhibits for each of the 15 specific educational areas,
and then use the features as a checklist to see if they were satisfied.
Charlie rewrote the features, renumbering and eliminating items, as appropriate:
(1) Should accommodate crowds—not just one person (or, if one person, should have pretty
good throughput).
(2) Should be sturdy.
B8
(3) Must be ready in 18 mo.
(4) Should be interesting to ages 5 through 18.
(5) Should have minimal consumables (he knew, from experience, that consumables could
destroy a budget).
(6) It should involve activity, not passive participation (nothing that the kids just watched). The
younger children might have more activity (crawling, climbing, jumping, running etc.) than
the older ones.
(7) It might involve winning or high scores or a competition of some sort.
(8) It might involve their making something—visible accomplishment was usually fun.
(9) It could involve testing of their physical or cognitive powers but should have a black/white
answer.
(10) It should not be perceived as childish—must be something an adult would do—don’t insult
the kids!!!
(11) Moving parts were good—he might want to stay away from solely computerized stuff.
(12) High tech was good, maybe having to do with exploration or environmental themes—tie it
to something they knew something about.
(13) If he wanted to get them to do it again and again, it should be something where they might
measure improvement, or at least get a different result with a different effort or problem.
(14) Must be safe.
(15) Should be handicappedaccessible.
He then rewrote the educational goals, indexing them by lettering them to avoid confusion with
the numbered features list:
a. Demonstrate units of mass, length, volume, temperature, etc.
b. Demonstrate intrinsic (color, viscosity, melting point, hardness, density...) versus extrinsic
properties (size, shape, temperature...). Note intrinsic properties are also known as physical
properties.
c. Demonstrate chemical properties (the tendency of the substance to change, through
interactions with other substances or singly).
d. Demonstrate chemical change (new substance is formed) versus physical change—include
exothermic and endothermic changes.
e. Demonstrate elements, compounds, mixtures, and solutions.
B9
f. Demonstrate the states of matter: solid, liquid, gas, plasma.
g. Demonstrate the laws of conservation of mass and energy.
h. Provide a feel for Avogadro’s number.
i. Demonstrate crystalline nature of many solids.
j. Demonstrate the nature of polymer chains.
k. Demonstrate the nature of metals and semiconductor materials.
l Demonstrate the principles of catalysis.
m. Demonstrate the principles of combustion.
n. Demonstrate the special nature of organic chemistry.
o. Demonstrate the standard and quantum theory for the atom.
Phase B—Starting to Have a Ball
Applied Chemist Dalton suggested that they divide the educational goals among several people.
Charlie agreed, and decided to work the problem with product development teams. He quickly formed
several teams and parcelled out the work. That took some explanations! He’d selected Dalton and Jane
for members of his team, along with design artist Mike Angelo, and a marketing executive who worked
their company’s precollege outreach efforts, Hewitt Wissard. Their task was to develop exhibits for
items h, j and k. Jane facilitated the brainstorming session, and by lunchtime they had several concepts
developed for each of the educational areas. Charlie copied the concept suggestions down from the
yellow stickies they’d used in brainstorming:
Provide a feel for Avogadro’s number (item h)
(1) Build a “ball pit” where the number of balls was some percentage of Avogadro’s number
and smaller kids could play. (Hewitt had seen something like this at a pizza place and his
daughter liked it a lot.)
(2) Have a sugar bed filled with grains of sugar that were some percentage of Avogadro’s
number. This could also be used for experiments (of some sort) and for microscopy when
discussing the crystal educational area. Maybe used for eating, too.
(3) Develop some kind of strengthtest thing where kids could compete to get close to
Avogadro’s Number on a scale or something. (Jane really wasn’t a scientist, but in
brainstorming, everyone’s input could be important).
B10
Demonstrate the nature of polymer chains (item j)
(1) Have microscopes set up to look at polymer crystals.
(2) Have a sort of maze set up that was partially amorphous and partially crystalline, like some
polymers are. Let the kids walk through it.
Demonstrate the nature of metals and semiconductor materials (item k)
(1) Have a large blast furnace that the kids could use to heattreat metals, and then measure the
resultant properties using an Instron tester. Also have water, oil, and salt quenching baths.
(2) Set up something where they could provide various amounts of dopant to semiconductor
crystals, and then measure the resistance etc.
(3) Have a display showing the crystal structure and how semiconductors and metals work
(electrically).
(4) Have polishing wheels set up with microscopes so they could polish specimens and look at
grain structure and stuff.
They were far from done, but it was a good start. When Jane asked Charlie if he still wanted her
to talk to Buck, he was surprised. He’d forgotten the deal during the long morning’s work. “No thanks, I
admit we’re making pretty good progress. I guess we might even start some phase B work this
afternoon, huh?”
“Sure,” replied Jane, “but you’re on your own for a while. I’ve got appointments for the next
couple of days. I think you’ll do fine, and if you run into problems, I’ll be back Thursday afternoon.
OK?”
“Not quite,” said Charlie with his own laugh, “I’ll see you Friday morning for a team meeting.
OK?”
“OK,” laughed Jane, and they all went out to lunch.
Friday came, and Charlie was impressed with the work they had accomplished. After several
revisions, they had centered on working on the Avogadro’s number thing, the one with the “ball pit.”
The decision had come after long discussion, and an aborted attempt to perform a weighted average
trade study to help quantify the results and the decisionmaking process. When Jane came in, Charlie
(and the rest of the group) was eager to find out what they had done wrong in using the trade study
methodology—although Charlie wasn’t interested in arguing this time. He was kind of looking forward
to working with Jane again. They showed Jane the brief attempt at a trade study that they had
formulated:
B11
Item Criteria Attribute Weight Factor
1 Avogadro’s number (H) 30
2 Fun (4,6,7,8, 30
9,10,11,12)
3 Cost (5) 25
4 Safe (14) 10
5 Variable outcome (13) 5
Charlie explained that the letter and numbers in parenthesis referred back to the specific feature
and the educational goal delineations they had previously produced. He was pleased with this, as he
thought it quite sensible to have tied the criteria attributes to the required features/goals that they had
agreed upon. Jane agreed that their approach did represent a very logical progression, but Charlie saw
her half smile again. By now he knew that meant they had made an error, and she had spotted it.
“Go on,” was all Jane said.
They let Jane know that they had spent a good deal of time discussing the weighting criteria; it
represented a group consensus. They then showed her the estimates for each alternative relative to each
criteria attribute. They had used the TQM techniques of ranking each of the qualitative items as 1, 5, or 9
to allow separation of results. These rankings, too, were consensus:
Item Criteria Attribute Weight Measure of Alt 1 Alt 2 Alt 3
Factor Effectiveness
1 Avogadro’s number 30 Obvious 9 9 9
2 Fun 30 See features 9 9 9
3 Cost 25 ROM estimate 9 9 5
4 Safe 10 Standard stuff 9 9
9
5 Variable outcome 05 Obvious 9 9 9
They hadn’t bothered to calculate the scores or weighted scores. It was obvious that it would come out a
wash.
Jane was wearing a large smile now. She said to Charlie, “I think you know where the problem
was, but I don’t think you recognize the value of what you did! Let’s start with the problem. Tell us
why you think it didn’t work.”
B12
Charlie was again caught off guard—he wished she’d stop doing that—but he answered her
question. “I don’t think our designs were well formulated enough to be able to evaluate and score. I
don’t think we did a very good job on defining quantitative, objective criteria attributes. But for this type
of effort, I’m not sure how to do any better. So how can you use a trade study then, when you’re at this
stage of a design? Why was it one of the recommended tools?”
Jane’s eyes glittered with her smile as she began, “You’re right about the criteria attributes. One
way might have been to simply count the features (an example: How many of the features defining fun
were incorporated under each alternate?) and quantify how many the alternate would have met. But that
wouldn’t have solved your fundamental problem. You’re right, the designs really weren’t mature enough
for this type of study to give a clear selection. To evaluate ‘Safe,’ a PHA would really be required,
which means you have to have at least a preliminary design. To evaluate ‘Fun’ and ‘Cost,’ the same
level of maturity is also necessary. But, what I don’t think you’ve realized is by performing this study,
you were able to identify that. At this stage of design maturity, no concepts were inappropriate. The fact
that nothing washed out gave you a valuable answer, and let you choose based on ‘gut feel’—what’s
often called ‘engineering’ or ‘programmatic judgement.’ Further, you elected to quantify your
qualitative feel for the alternate by using the 1,5,9 technique. I think you guys did just great! You
wouldn’t have tried to be specific about why you had selected one idea to pursue if you hadn’t had these
techniques—you knew intuitively that there weren’t enough data to use appropriate criteria. These
techniques won’t let you do things that can’t otherwise be accomplished. They’re just an aid. And I think
you did great. When it wasn’t helping, you tried something else. Which one did you wind up selecting,
anyway?”
“Alternate 1 the ball pit,” replied Charlie. “Now I thought we might flowchart the effort
required for phase B to figure out where we’re going with this. You know—the planyourwork and then
workyourplan kind of stuff.”
After some long discussions over the blank page that they were trying to use to do a flowchart,
Jane suggested that a good way to get started might be to flowchart what they had already done.
Although it seemingly added nothing to their completed tasks, she noted it was often easier to add to
something, and even easier to edit or change something, than it was to create it in the first place. Starting
the flowchart with the efforts they had already accomplished would give them a base to add upon, rather
than the nowbeginningtobeannoying blank page. They agreed and by the end of the day the group
had produced a reasonable flowchart (see figure B1.) Much of the day had been spent on trying to
define which tools would be used. This time they only used the toolbox as a guide and kept asking,
“Will this tool help me? Do I need it...?” Their flowchart represented their choices—to their surprise it
also provided some insights to the design process.
Many of the phase A decisions were management/programmatictype decisions that held
significant consequences for the remainder of the effort. It was also true that most of the data required to
support credible costversusbenefit or riskversusbenefit trade studies did not exist at this stage. Charlie
began to hold an even greater appreciation for the toolbox—not so much for the major type decisions,
but for the potential to reveal the more subtle ramifications of decisions that might otherwise go
unnoted. He spoke his thoughts to the group, noting that these subtleties were particularly critical at the
beginning of a project. He received the typical reaction by a group to someone who speaks the
obvious—they laughed.
B13
Prepare Requirement's Checklist:
Features and Goals
Form PDT and Divide Work
Brainstorm Concepts
Downselect Single Concept
Receive Assignment
Costvs.Benefit Studies
Delphi Technique
Nominal Group Techniques
Trade Studies
Select Tools for Phase A
Is the
technique
primarily
for Phase A?
Don't Use Technique
No
Yes
Attempt to use
the selected
technique
Selection Process that Was Used
Brainstorming
Toolbox Recommendations
Technique Familiarity (Personnal)
Judgement (schedule, criticality, resources etc...)
Yes
No
Review Potential tools
from Handbook for applicability
Will use of
the selected
technique
add value?
Attempt to use
the selected
technique
Don't Use Technique
Selection Process that Should Have Been Used
Prepare Features Prepare Goals
Brainstorming Brainstorming
Trade Study
B
B
Delineate Consolidated Checklist for Goals
and Features
A A C
Brainstorming
Brainstorming
WeightedAverage Trade Study
Phase A Flowchart: Note most decisions are
Programmatic / Management decisions with
a paucity of data and farreaching consequences.
Yes
No
Is the
Feature
quantifiable?
Additional Step that Should
Have Been Added
C
B
A
Yes
No
Are the
features well
defined and mutu
ally exclu
sive
Selection Process that Was Used
to determine Features / Goals
A
B
D D
E
(a) Phase A.
Figure B1. Flowchart—Continued
B14
E
Perform Preliminary Hazard Analysis
Flowchart the Proposed Process
Review Potential tools
from Toolbox for applicability
Toolbox Recommendations
Technique Familiarity (Personnal)
Judgement (schedule, criticality, resources etc...)
Yes
No
Will use of
the selected
technique
add value?
Include selected
technique in
flowchart
Don't Use Technique
Perform Proposed Studies on Selected Design, and Iterate Design
Perform Probabilistic Risk Assessment
Perform Reliability Block Diagram
Perform Digraph Analysis
Setup Strawman Design for Selected Concept
Use marble size balls
Pit is 20 x 20 x 3 foot deep
Use radium tracers for diffusion modeling
Use Radio wrist bands for children
Special pit area used for close packing demo
F
F
F
F
G
Yes
No
Yes
No
F
Does
technique
indicate a design
problem?
Have
all analyses
been completed?
G
G
H
Phase B Flowchart: Note from post "flowchart creation" step on, most
analyses and decisions are technical and disciplinespecialist oriented.
(b) Phase B.
Figure B1. Flowchart—Continued.
B15
Jane wasn’t laughing though. She noted that there was a change from the fundamental desirable
attributes of a phase A program manager to those of a phase B manager. Decisions at the phase A level
really required overall system experience and a capability for accepting available data, determining the
appropriate level for additional data, and decision selection from the minimum, adequate (often very
sparse) pool which that represented. Phase B efforts required a much more detaildriven selection
process which employed the talents and capabilities of discipline specialists—management did well as a
cheerleader here but might often leave the technical and some of the programmatic optimization
decisions to the concurrent engineering team working the problem. Phases C and D were similar in
nature to
phase A.
Charlie began to feel a little better about things. He also noted that brainstorming was a
technique that was seemingly suitable almost everywhere. He and Jane decided to use it to select a
restaurant for dinner, after the rest of the group had gone home.
Charlie had gotten home very late Friday night, he was still sleeping when his boss’s secretary,
Fawn Hunter, called Saturday morning to let him know that first thing Monday morning Buck wanted to
see what kind of progress they had made and discuss the project. He thanked Fawn for the information
and got up to do some more work on the flowchart, and to try to start some of the other tools. It was
looking like it was going to be a long weekend. About an hour later Jane called, volunteering to help.
Charlie was the one smiling now—the prospects for the weekend were looking up. He liked the phase A
and B flowcharts and added some of the symbols from the toolbox, just to key in on the difficulties for
those particular areas. He also added the “should have done” boxes in addition to the “as performed”
flows, and changed the phase B flowchart accordingly.
Charlie ran the calculations for the number of marbles in the ball pit:
Calculation of marble volume:
Volume of a sphere = (4/3) π r
3
r
marble
= 0.5 in
r
3
marble
= 0.125 in
3
V
marble
= 0.52 in
3
.
Calculation of ball pit size:
Assume 20 ft×20 ft×3 ft
V
ballpit
= 1,200 ft
3
×1,728 in/1 ft
3
= 2.07×10
6
in
3
.
The maximum packing density for spheres of a single diameter is 74 percent.
Calculation of number of marbles to “fill ball pit:”
No.
marbles
= (0.74) 2.07×10
6
in
3
/0.52 in
3
= 2.95×10
6
.
B16
Although that was a huge amount of marbles (Charlie started wondering about the feasibility and
cost of that many marbles), it didn’t begin to approach Avogadro’s number. Charlie was still at a loss for
how to relate the two, and the diffusion part was still evading him. But now that they had this much of a
start, he and Jane decided to perform the PHA. Once again it was time for brainstorming.
They filled up the first page with a list of 11 hazards (see figure B2), first listing all of them,
then identifying the targets for each of them, then working the severity and probability and risk factor. In
this way they were able to concentrate on the ideas first, without getting caught up in the specific
assessment issues or countermeasures. They used the toolbox risk assessment matrix (on page 310) that
had been adapted from MIL–STD–882C. Jane suggested that they only work countermeasures for those
items with a risk priority code of 1 or 2. There wasn’t any need to improve on those with a 3. Charlie
was quite surprised to find that the marbles represented such a significant risk. They settled on a design
change for that item.
After filling in the countermeasures and the new risk Priority Codes, they were left only with two
items of code level 2. Charlie didn’t know how to work these and neither did Jane. Jane did mention
though that they might just be a function of the integrated exhibit area (IEA)—disease transmission in
children’s museums was a common occurrence and wherever children jumped, someone was liable to
get jumped on. They decided to go eat a late lunch, pizza, and watch one of these ball pits in action.
After returning from lunch, Charlie did calculations for larger balls. He had gotten the idea of filling the
balls with sand or salt to better compare with Avogadro’s number. This also might be useful for partially
addressing the crystal educational goal. He and Jane worked the new calculations for the larger balls,
and for a salt filler.
Calculation of large ball volume:
volume of a sphere = (4/3) π r
3
r
ball
= 3.5 in
r
3
ball
= 4.29×10
1
in
3
V
ball
= 1.8×10
2
in
3
.
The maximum packing density for spheres of a single diameter is 74 percent.
Calculation of number of balls to “fill the ball pit:”
No.
balls
= (0.74) 2.07×10
6
in
3
/1.8×10
2
in
3
= 8.52×10
3
.
Volume of a grain of salt:
volume of a cube = abc
assume (a) is approximately equal to (b), which is approximately equal to (c)
assume a = 0.01 in.
B17
B18
Brief Descriptive Title (Portion of System/Subsystem/Operational Phases covered by this analysis):
Probability Interval: 25 years
H
a
z
a
r
d
T
a
r
g
e
t
*
S
e
v
e
r
i
t
y
P
r
o
b
a
b
i
l
i
t
y
R
i
s
k
C
o
d
e
Description of Countermeasures
S
e
v
e
r
i
t
y
P
r
o
b
a
b
i
l
i
t
y
R
i
s
k
C
o
d
e
Approved by/Date:
Preliminary Hazard Analysis
Identify countermeasures by appropriate code letter(s):
D = Design Alteration E = Engineered Safety Feature
S = Safety Device W = Warning Device
P = Procedures/Training
Analysis: Initial
Revision Addition System Number: __________
Date:
Hazard No. / Description
Risk
Before
Risk
After
Prepared by/Date: *Target Codes: P—Personnel
T—Downtime
E—Equipment
R—Product V—Environment
February 1994
X
2. Fall in pit. P III A 2
E
T
P  Have posted rules and trained monitors.
D  Use soft balls and padded surfaces.
P III B 2 III E 3
1. Cut hand on sharp edges.
D Place bumpers on all potential sharp edges.
P I D 2 I E 3 3. Suffocate under balls.
D  Use larger balls.
E  Limit use to children 3 feet tall or taller.
P I C 1
I F 3 4. Suffocate on marbles. D  Use larger balls.
P II E 3 5. Cancer risk from radium.
P III D 3 6. Health risk from radio waves.
P III A 2 III A
7. Children landing on each other.
P  Have posted rules and trained monitors. 2
P  Limit age of children in the pit to 7 years old or less.
P III D 3
8. Balls (marbles) breaking and leaving sharp
objects in pit.
9. Big children landing on small children or pit walls. P III B 2
IV C
T
P
III A 2
III A
10. Risk of disease communication. P  No very effective countermeasure .
2
P III A 2
III C 3
11. Balls becoming projectiles in and out of pit. P  Have posted rules and trained monitors.
Ball Pit for Science HandsOn Museum Display
T
C II 3
3
IV
IV
A
A
3
3
D  Use padded surfaces.
III D 3
IV B 3
IV C
C
3
3
IV
III C 3
B18
V
salt grain
= (0.01in)
3
= 1.0×10
–6
in
3
.
Calculation of the number of grains of salt to “fill ball pit:”
No.
salt
grains
= 2.07×10
6
in
3
/1.0×10
–6
in
3
= 2.07×10
12
.
The assumption was made that a ball has zero wall thickness and the salt grains will “magically
stack” in a sperical container.
Calculation of the number of grains of salt per ball:
No.
grains/ball
= (V
ball
/V
salt
) = (1.8×10
2
in
3
/1.0×10
–6
in
3
)
= 1.8×10
8
grains/ball
Calculation of number of grainfilled balls required to house Avogadro’s number of grains:
No.
balls
= Av= (1.8×10
8
grains/ball
) (x) = 6.02×10
23
= 3.34×10
15
.
The maximum packing density for spheres of a single diameter is 74 percent.
Calculation of required ball pit volume to contain 3.34×10
15
balls:
V
ballpit
= (V
ball
) (No.
balls
) = (1.8×10
2
in
3
) ( 3.34×10
15
)/0.74
= 8.12×10
17
in
3
= 3,195 mi
3
Calculation of cube side required to make a cube of volume = 2,364 miles
3
:
Side = (3,195 miles
3
)
1/3
= 17.97 mi.
There. They had made some major progress, and Charlie was beginning to visualize this exhibit. He
knew they were ready to talk with Buck on Monday. He did want to find out about using the PHA tool
for programmatic risk evaluation, and he had begun doodling with some small fault trees and was
impressed by what he could do with them. He had already pretty much decided not to do the PRA
assessment and....
Epilogue...Two (and a Half) for the Show
Opening day of the museum was a media event. There were speeches and interviews and plenty
of good words for all. Mr. and Mrs. Smith stayed in the background letting others soak up the limelight.
They were pleased and proud of what they had done, and excited that their soontobeborn child would
get to visit the museum often. Those lifetime passes for their family turned out to be a great wedding
gift! Charlie was putting together a short report on the lessons learned during those first few months of
the project—Jane was going to use it as a case study during her next class on the toolbox. He had left it
at home for Jane to read, she smiled again as she recalled the listing:
B19
(1) The toolbox is just a guide. Use techniques that have value specific to the requirements, not
simply because they are available or recommended.
(2) Don’t be afraid to use techniques that you’re unfamiliar with—but get expert help when
required! Anything can be misused.
(3) Expect to make mistakes enroute to success. Learn to recognize and correct them.
(4) Using the techniques does not mitigate the need for facts and data—rather it better defines
the need (garbage in—garbage out).
(5) Brainstorming is almost universally useful.
(6) When she smiles, my wife is always right.
B20
APPENDIX C
GLOSSARY OF TERMS
C1
C2
GLOSSARY OF TERMS
Te r m De f i n i t i o n
Analysis An examination of the elements of a system; separation of a whole
into its component parts. (Reference Section 4.1)
AND Gate A logic gate for which an output occurs if all inputs coexist. All
inputs are necessary and sufficient to cause the output to occur.
(Reference Section 3.5)
Backwards Logic The mental process in which an analyst models a system by
repeatedly asking the question, "What will cause a given failure to
occur?" Also called topdown logic. (Reference Section 3.0)
Barrier A countermeasure against hazards caused by a flow from an
energy source to a target. (Reference Section 3.3).
Basic Event An initiating fault or failure in a fault tree that is not developed
further. Also called an initiator or leaf. These events determine
the resolution limit for a fault tree analysis.
Cause The event or condition responsible for an action or result.
(Reference Section 3.10)
Common Cause A source of variation that is always present; part of the random
variation inherent in the process itself.
Consequence Something that follows from an action or condition; the relation of
a result to its cause. (Reference Section 3.10)
Control Limits Limits (also called action limits) set between the mean or nominal
values of a parameter and specification limits. If a control limit is
exceeded, corrective actions may need to be implemented before
the specification limit is exceeded. (Reference Section 5.2)
Countermeasure An action taken or a feature adopted to reduce the probability
and/or severity of risk for a hazard. (Reference Sections 3.2 and
3.4))
Creative Function The means of seeing new ways to perform work by breaking
through barriers that often stifle thinking. Some techniques that
are considered creative tools are evolutionary operation (Section
7.6), brainstorming (Section 7.7), and nominal group technique
(Section 7.10). (Reference Table 11)
C3
GLOSSARY OF TERMS
Te r m De f i n i t i o n
Critical Items List
(CIL)
A FMEAderived list (published as FMEA/CIL) containing system
items that have a criticality of 1 or 2, and items that are criticality
1R or 2R and fail redundancy screens. (Reference Section 3.4)
Criticality In reference to a parameter, criticality is the level of importance the
parameter has to the operation of the system. (Reference Section
3.4)
Customer The internal or external person or organization that is the user of a
product being produced or service being rendered. The immediate
customer is the user of the product or service in the next step of the
process.
Cut Set Any group of fault tree initiators which, if all occur, will cause the
TOP event to occur. (Reference Section 3.6)
Data Analysis
Function
The means of analyzing a process by using a data display. Some
techniques that are considered data analysis tools are checklists
(Section 7.8), control charts (Section 5.2), and force field analysis
(Section 7.11). (Reference Table 11)
Decision Making
Function
After analyzing all available data, a decision is made on how to
optimize the subject process. Some techniques that are considered
decision making tools are benchmarking (Section 7.1), nominal
group technique (Section 7.10), and force field analysis (Section
7.11). (Reference Table 11)
Degrees of Freedom The number of independent unknowns in the total estimate of a
factorial effect or a residual. (Reference Section 6.2)
Facilitator A person trained in group dynamics and problemsolving
structures who assumes the responsibility for ensuring a full
exchange of information between team members. (Reference
Section 7.2)
Factor A parameter or variable that affects product/process performance.
(Reference Section 6.2)
Fail Safe Proper function is impaired or lost but no further threat of harm
occurs. (Reference Section 3.4)
Failure A fault owing to breakage, wear out, compromised structural
integrity, etc. (Reference Section 3.4)
C4
GLOSSARY OF TERMS
Te r m De f i n i t i o n
Failure Domain In analysis work, failure domain refers to an analysis that seeks
the probability of a system not operating correctly. (Reference
Section 3.8)
Failure Mode The manner in which a failure occurs, i.e. the manner in which it
malfunctions. (Reference Section 3.4)
Failure Propagation
Path
The sequence of events that leads to an undesirable event or loss.
Also called an accident sequence.
Fault Inability to function in a desired manner, or operation in an
undesired manner, regardless of cause. (Reference Section 3.6)
Forward Logic The mental process in which an analyst models a system by
repeatedly asking the question, "What happens when a given
failure occurs?" Also called bottomup logic. (Reference Section
3.0)
Graphical Function The means of analyzing the data of a process by applying graphs
and/or charts. Some of the techniques that are considered
graphical tools are cause and effect diagram (Section 7.2), control
charts (Section 5.2), and quality function deployment (Section
7.12). (Reference Table 11)
Hazard An activity or condition which poses a threat of loss or harm; a
condition requisite to a mishap. (Reference Section 3.2)
Intermediate Event An event that describes a system condition produced by preceding
event and contributing to later events.
Mean The term used to describe a sample population average.
(Reference Section 6.1)
Mean Square
Deviation (MSD)
A measure of variability around the mean or target value.
Mishap An undesired loss event. (Reference Section 8.3)
Modeling Function The means of analyzing and modeling a process against standards
and/or other processes. Some of the techniques that are
considered modeling tools are benchmarking (Section 7.1), quality
function deployment (Section 7.12), and work flow analysis
(Section 7.16). (Reference Table 11)
C5
GLOSSARY OF TERMS
Te r m De f i n i t i o n
OR Gate A logic gate in which an output occurs if one or more inputs exist.
Any single input is necessary and sufficient to cause the output to
occur. (Reference Section 3.5)
Parameter The term applied to population or sample characteristics such as
the mean and standard deviation. (Reference Section 5.2)
Path Set A group of fault tree initiators which, if none of them occurs, will
guarantee that the TOP event cannot occur. (Reference Section
3.6)
Population The universe of data under investigation from which a sample will
be taken. (Reference Section 6.1)
Preliminary Coming before and usually forming a necessary prelude to
something. As in a preliminary hazard analysis, the analysis can
be performed in the design or preoperation phase, or it can be the
first analysis performed for a mature system. (Reference Section
3.2)
Prevention Function The means of analyzing data to be able to recognize potential
problems and prevent the process from heading in an adverse
direction. Some of the techniques that are considered preventive
tools are control charts (Section 5.2), Pareto analysis (Section
5.6), and design of experiments (Section 7.5). (Reference Table
11)
Probability The liklihood an event will occur within a defined time interval.
(Reference Section 3.14)
Problem
Identification
Function
The means of identifying potential problems from a data display as
a result of an analysis of the process. Some techniques that are
considered problem identification tools are control charts (Section
5.2), brainstorming (Section 7.7), and quality function
deployment (Section 7.12). (Reference Table 11)
Process A series of events progressively moving forward over time to
produce products or services for a customer. (Reference Section
7.1)
Project Phase A The conceptual trade studies phase of a project. Quantitative
and/or qualitative comparison of candidate concepts against key
evaluation criteria are performed to determine the best alternative.
(Reference Section 1.3)
C6
GLOSSARY OF TERMS
Te r m De f i n i t i o n
Project Phase B The concept definition phase of a project. The system mission and
design requirements are established and design feasibility studies
and design trade studies are performed during this phase.
(Reference Section 1.3)
Project Phase C The design and development phase of a project. System
development is initiated and specifications are established during
this phase. (Reference Section 1.3)
Project Phase D The fabrication integration, test, and evaluation phase of a project.
The system is manufactured and requirements verified during this
phase. (Reference Section 1.3)
Project Phase E The operations phase of a project. The system is deployed and
system performance is validated during this phase. (Reference
Section 1.3)
Qualitative Data that are not numerical in nature. (Reference Section 2.1)
Quantitative Data that are numerical in nature or can be described numerically.
(Reference Section 2.1)
Range A measure of the variation in a set of data. It is calculated by
subtracting the lowest value in the data set from the highest value
in that same set. (Reference Section 5.2)
Raw Data Data as measured or as taken directly from instruments or sensors.
(Reference Section 8.4)
Reliability The probability of successful operation of a system over a defined
time interval. (Reference Section 3.3)
Risk For a given hazard, risk is the longterm rate of loss; the product
of loss severity and loss probability. (Reference Section 3.1)
Sample One or more individual events or measurements selected from the
output of a process for purposes of identifying characteristics and
performance of the whole. (Reference Section 6.1)
Severity The degree of the consequence of a potential loss for a hazard.
(Reference Section 3.1)
Special Cause A source of variation that is intermittent, unpredictable, unstable;
sometimes called an assignment cause. It is signalled by a point
beyond the control limits. (Reference Section 8.1)
C7
GLOSSARY OF TERMS
Te r m De f i n i t i o n
Standard Deviation A measure of variability used in common statistical tests. The
square root of the variance. (Reference Section 6.1)
Subassembly A composite of components. (Reference Section 3.4)
Success Domain In analysis work, success domain refers to an analysis that seeks
the probability of a system operating correctly. (Reference Section
3.8)
System A composite of subsystems whose functions are integrated to
achieve a mission (includes materials, tools, personnel, facilities,
software, and equipment).
System Element A constituent of a system that may be a subsystem assembly,
component, or piecepart.
Target An object having worth that is threatened by a hazard. The object
may be personnel, equipment, downtime, product, data,
environment, etc. (Reference Section 3.1)
Threat A potential for loss. A hazard. (Reference Section 3.1)
TOP Event The conceivable, undesired event to which failure paths of lower
level events lead. (Reference Section 3.6)
Trends The patterns in a run chart or control chart that feature the
continued rise or fall of a series of points. Like runs, attention
should be paid to such patterns when they exceed a predetermined
number (statistically based). (Reference Section 8.0)
Upper Control Limit
Range
The upper control limit for the moving range chart for a set of data.
(Reference Section 7.14)
Variation The inevitable difference among individual outputs of a process.
The sources of variation can be grouped into two major classes:
Common Causes and Special Causes. (Reference Section 6.2)
Weighting Factor A method of rating the relative importance of a concern or selection
criterion as related to comparable concerns or selected criteria.
(Reference Sections 2.1 and 7.12)
C8
APPENDIX D
HAZARDS CHECKLIST
D1
D2
HAZARDS CHECKLIST
Notes:
1. Neither this nor any other hazards checklist should be considered complete. This list should be enlarged as
experience dictates. This list contains intentional redundant entries.
2. This checklist was extracted from "Preliminary Hazard Analysis (Lecture Presentation)", R.R. Mohr, Sverdrup
Technology, Inc., June 1993 (Fourth Edition).
D3
Electrical
_____ Shock
_____ Burns
_____ Overheating
_____ Ignition of Combustibles
_____ Inadvertent Activation
_____ Power Outage
_____ Distribution Backfeed
_____ Unsafe Failure to Operate
_____ Explosion/Electrical (Electrostatic)
_____ Explosion/Electrical (Arc)
Mechanical
_____ Sharp Edges/Points
_____ Rotating Equipment
_____ Reciprocating Equipment
_____ Pinch Points
_____ Lifting Weights
_____ Stability/Topping Potential
_____ Ejected Parts/Fragments
_____ Crushing Surfaces
Pneumatic/Hydraulic Pressure
_____ Overpressurization
_____ Pipe/Vessel/Duct Rupture
_____ Implosion
_____ Mislocated Relief Device
_____ Dynamic Pressure Loading
_____ Relief Pressure Improperly Set
_____ Backflow
_____ Crossflow
_____ Hydraulic Ram
_____ Inadvertent Release
_____ Miscalibrated Relief Device
_____ Blown Objects
_____ Pipe/Hose Whip
_____ Blast
Acceleration/Deceleration/Gravity
_____ Inadvertent Motion
_____ Loose Object Translation
_____ Impacts
_____ Falling Objects
_____ Fragments/Missiles
_____ Sloshing Liquids
_____ Slip/Trip
_____ Falls
Temperature Extremes
_____ Heat Source/Sink
_____ Hot/Cold Surface Burns
_____ Pressure Evaluation
_____ Confined Gas/Liquid
_____ Elevated Flammability
_____ Elevated Volatility
_____ Elevated Reactivity
_____ Freezing
_____ Humidity/Moisture
_____ Reduced Reliability
_____ Altered Structural Properties
(e.g., Embrittlement)
Radiation (Ionizing)
_____ Alpha
_____ Beta
_____ Neutron
_____ Gamma
_____ XRay
Radiation (NonIonizing)
_____ Laser
_____ Infrared
_____ Microwave
_____ Ultraviolet
HAZARDS CHECKLIST
Notes:
1. Neither this nor any other hazards checklist should be considered complete. This list should be enlarged as
experience dictates. This list contains intentional redundant entries.
2. This checklist was extracted from "Preliminary Hazard Analysis (Lecture Presentation)", R.R. Mohr, Sverdrup
Technology, Inc., June 1993 (Fourth Edition).
D4
Fire/Flammability  Presence of:
_____ Fuel
_____ Ignition Source
_____ Oxidizer
_____ Propellant
Explosives (Initiators)
_____ Heat
_____ Friction
_____ Impact/Shock
_____ Vibration
_____ Electrostatic Discharge
_____ Chemical Contamination
_____ Lightning
_____ Welding (Stray Current/Sparks)
Explosives (Effects)
_____ Mass Fire
_____ Blast Overpressure
_____ Thrown Fragments
_____ Seismic Ground Wave
_____ Meteorological Reinforcement
Explosives (Sensitizes)
_____ Heat/Cold
_____ Vibration
_____ Impact/Shock
_____ Low Humidity
_____ Chemical Contamination
Explosives (Conditions)
_____ Explosive Propellant Present
_____ Explosive Gas Present
_____ Explosive Liquid Present
_____ Explosive Vapor Present
_____ Explosive Dust Present
Leaks/Spills (Material Conditions)
_____ Liquid/Cryogens
_____ Gases/Vapors
_____ Dusts  Irritating
_____ Radiation Sources
_____ Flammable
_____ Toxic
_____ Reactive
_____ Corrosive
_____ Slippery
_____ Odorous
_____ Pathogenic
_____ Asphyxiating
_____ Flooding
_____ Run Off
_____ Vapor Propagation
Chemical/Water Contamination
_____ SystemCross Connection
_____ Leaks/Spills
_____ Vessel/Pipe/Conduit Rupture
_____ Backflow/Siphon Effect
Physiological (See Ergonomic)
_____ Temperature Extremes
_____ Nuisance Dusts/Odors
_____ Baropressure Extremes
_____ Fatigue
_____ Lifted Weights
_____ Noise
_____ Vibration (Raynaud's Syndrome)
_____ Mutagens
_____ Asphyxiants
_____ Allergens
_____ Pathogens
_____ Radiation (See Radiation)
_____ Cryogens
_____ Carcinogens
_____ Teratogens
_____ Toxins
_____ Irritants
HAZARDS CHECKLIST
Notes:
1. Neither this nor any other hazards checklist should be considered complete. This list should be enlarged as
experience dictates. This list contains intentional redundant entries.
2. This checklist was extracted from "Preliminary Hazard Analysis (Lecture Presentation)", R.R. Mohr, Sverdrup
Technology, Inc., June 1993 (Fourth Edition).
D5
Human Factors (See Ergonomic)
_____ Operator Error
_____ Inadvertent Operation
_____ Failure to Operate
_____ Operation Early/Late
_____ Operation Out of Sequence
_____ Right Operation/Wrong Control
_____ Operated Too Long
_____ Operate Too Briefly
Ergonomic (See Human Factors)
_____ Fatigue
_____ Inaccessibility
_____ Nonexistent/Inadequate "Kill"
Switches
_____ Glare
_____ Inadequate Control/Readout
Differentiation
_____ Inappropriate Control/Readout
Location
_____ Faulty/Inadequate
Control/Readout Labeling
_____ Faulty Work Station Design
_____ Inadequate/Improper Illumination
Control Systems
_____ Power Outage
_____ Interferences (EMI/ESI)
_____ Moisture
_____ Sneak Circuit
_____ Sneak Software
_____ Lightning Strike
_____ Grounding Failure
_____ Inadvertent Activation
Unannunciated Utility Outages
_____ Electricity
_____ Steam
_____ Heating/Cooling
_____ Ventilation
_____ Air Conditioning
_____ Compressed Air/Gas
_____ Lubrication Drains/Slumps
_____ Fuel
_____ Exhaust
Common Causes
_____ Utility Outages
_____ Moisture/Humidity
_____ Temperature Extremes
_____ Seismic Disturbance/Impact
_____ Vibration
_____ Flooding
_____ Dust/Dirt
_____ Faulty Calibration
_____ Fire
_____ SingleOperator Coupling
_____ Location
_____ Radiation
_____ WearOut
_____ Maintenance Error
_____ Vermin/Varmints/Mud Daubers
Contingencies (Emergency Responses by
System/Operators to "Unusual" Events):
_____ "Hard" Shutdowns/Failures
_____ Freezing
_____ Fire
_____ Windstorm
_____ Hailstorm
_____ Utility Outrages
_____ Flooding
_____ Earthquake
_____ Snow/Ice Load
HAZARDS CHECKLIST
Notes:
1. Neither this nor any other hazards checklist should be considered complete. This list should be enlarged as
experience dictates. This list contains intentional redundant entries.
2. This checklist was extracted from "Preliminary Hazard Analysis (Lecture Presentation)", R.R. Mohr, Sverdrup
Technology, Inc., June 1993 (Fourth Edition).
D6
Mission Phasing
_____ Transport
_____ Delivery
_____ Installation
_____ Calibration
_____ Checkout
_____ Shake Down
_____ Activation
_____ Standard Start
_____ Emergency Start
_____ Normal Operation
_____ Load Change
_____ Coupling/Uncoupling
_____ Stressed Operation
_____ Standard Shutdown
_____ Shutdown Emergency
_____ Diagnosis/Trouble Shooting
_____ Maintenance
APPENDIX E
EXAMPLE PRELIMINARY HAZARD ANALYSIS WORKSHEET
E1
E2
Example Preliminary Hazard Analysis Worksheet*
*This worksheet was extracted from “Preliminary Hazard Analysis (Lecture Presentation),” R.R. Mohr, Sverdrup Technology, Inc., June 1993.
E3
APPENDIX F
EXAMPLE FAILURE MODES AND EFFECTS ANALYSIS WORKSHEET
F1
F2
Example Failure Modes And Effects Analysis Worksheet*
*This worksheet was extracted from “Failure Modes and Effects Analysis (Lecture Presentation),” R.R. Mohr, Sverdrup Technology, Inc., July 1993.
F3
NASA Reference Publication 1358
System Engineering “Toolbox” for DesignOriented Engineers
B.E. Goldberg Marshall Space Flight Center • MSFC, Alabama K. Everhart, R. Stevens, N. Babbitt III, P. Clemens, and L. Stout Sverdrup Technology, Inc.
National Aeronautics and Space Administration Marshall Space Flight Center • MSFC, Alabama 35812
December 1994
ii
ACKNOWLEDGMENTS The authors are very grateful for the help received from the following persons in producing this document. Becky Mohr contributed information and illustrations concerning preliminary hazard analyses and failure modes and effects analyses. Bryan Bachman provided a thorough review of drafts of the entire document. Larry Thomson prepared a figure in the system safety and reliability tools section. Jimmy Howell verified all numerical calculations in the examples. The following persons reviewed the indicated sections of this document and offered suggestions that greatly enhanced the discussions of the tools and methodologies presented: Bill Cooley Melissa Van Dyke Karl Knight Charles Martin Ben Shackelford DesignRelated Analytical Tools Trend Analysis Tools System Safety and Reliability Tools Statistical Tools and Methodologies Graphical Data Interpretation Tools Case Study
iii
Figures provided courtesy of Sverdrup Technology, Inc., Tullahoma, Tennessee. Figure 31 Figure 32 Figure 33 Figure 34 Figure 35 Figure 36 Figure 37 Figure 38 Figure 39 Figure 310 Figure 311 Figure 312 Figure 315 Figure 316 Figure 317 Figure 318 Figure 319 Figure 320 Figure 321 Figure 322 Figure 323 Figure 324 Figure 325 Figure 326 Figure 327 Figure 328 Figure 329 Figure 330 Figure 331 Figure 332 Figure 333 Figure 334 Figure 335 Figure 336 Figure 340 Figure 341
iv
TABLE OF CONTENTS Page 1. INTRODUCTION .................................................................................................................. 1.1 1.2 1.3 Purpose ......................................................................................................................... Scope ......................................................................................................................... Relationship With Program or Project Phases.............................................................. References .................................................................................................................... 11 11 11 12 18 21 21 21 23 23 26 28 29 29 29 29 29 210 211 213 213 213 214 31 32 32 35 36 310 312 312 312
2. CONCEPT DEVELOPMENT TOOLS .................................................................................. 2.1 Trade Studies ................................................................................................................ 2.1.1 Description ...................................................................................................... 2.1.2 Application...................................................................................................... 2.1.3 Procedures....................................................................................................... 2.1.4 Example .......................................................................................................... 2.1.5 Advantages...................................................................................................... 2.1.6 Limitations ...................................................................................................... 2.1.7 Bibliography.................................................................................................... CostVersusBenefit Studies......................................................................................... 2.2.1 Description ...................................................................................................... 2.2.2 Application...................................................................................................... 2.2.3 Procedures....................................................................................................... 2.2.4 Example .......................................................................................................... 2.2.5 Advantages...................................................................................................... 2.2.6 Limitations ...................................................................................................... 2.2.7 Bibliography.................................................................................................... References ......................................................................................................
2.2
3. SYSTEM SAFETY AND RELIABILITY TOOLS ............................................................... 3.1 Risk Assessment Matrix ............................................................................................... 3.1.1 Description ...................................................................................................... 3.1.2 Application...................................................................................................... 3.1.3 Procedures....................................................................................................... 3.1.4 Example .......................................................................................................... 3.1.5 Advantages...................................................................................................... 3.1.6 Limitations ...................................................................................................... 3.1.7 Bibliography....................................................................................................
v
3 Procedures.5...............2 Preliminary Hazard Analysis.................. 3...............................................................5.......... 3.... 3...................................................................................................4.................................4 Example ............... 3.....................................4.................2......................... 3............... 3....................................................5..............6 vi ....................................................................................6 Limitations ...1 Description ..................4.............................................................................. 3..............................1 Fault Tree Generation .............................................................................................................. 312 312 313 313 316 317 318 318 318 318 319 319 319 321 321 321 321 321 322 322 326 328 329 329 330 330 332 332 333 334 334 335 335 335 335 336 337 337 341 343 3.......3.........................3...............................................5 3................................................................................. 3............. Fault Tree Analysis ...........................2 Application.........................3..............................3 Identifying and Assessing Cut Sets.................. Failure Modes and Effects (and Criticality) Analysis ............................................1 Description ...............6................................ 3.....................6...................................1 Description ................................ 3........................3...7 Bibliography...5....................... 3.................................................4 3...................................1 Description ......................................................5 Advantages.... 3.................2.....................................7 Bibliography... 3..........................................................................6.................. 3...........................3 Procedures........5 Advantages....3................................................3 Procedures............. 3................................3 3.......................................... 3.................................. 3........................ 3......... 3....... 3...............................................................1 Description ......................................2...............................................3 Procedures.... 3............................................... 3.......................................................................................................................................................5............4 Example ...........4........ 3...............................6 Limitations .....4............ 3...........................................3............................................2 Application............................ 3..............................2 Application.....................6...................................................4 Example ..........5 Advantages.............................................................................................................. 3.........................................................5........................6..........................4.................................................................................................2......................................... 3.................................................3 Procedures......................7 Bibliography............................................6 Limitations ........................................................ 3......................................................... 3............................................................................................................. Energy Flow/Barrier Analysis .........................................................4 Example .........3................................................................ 3.................................3........... Reliability Block Diagram......... 3.2 Application....2...................3........................................................................................................ 3............................6.6 Limitations .............3.........6...........................................................................................TABLE OF CONTENTS (Continued) Page 3.....................................2.............................................4 Identifying Path Sets ..............5........7 Bibliography.... 3..... 3....4...2.......... 3........................................5 Advantages......................................2 Application................................................................................................................2 Probability Determination..............................3.........................................................................................................................
............3 Procedures............7....................................................... 3...........................7...................................................................5 Advantages...........................7............................................................. 3...10.............................................. Reliability Block Diagram.......8.............................................................10.........2 Application..........................1 Fault Tree Construction and Probability Propagation......7 Bibliography...........3 Procedures.....9...........................................6...........................9.......... 3...................................... Fault Tree...........................................10........................1 Description .....................................7...............9......................................................................................... 3............................................10.............2 Application.... 3...................................................... vii ... 3.............................................................................7 Bibliography..........................................................6.......................... 3.............................................. Limitations ....4....................3...... 3............. Advantages..........................7.............................................................................2 Application.............8 3.....6 Limitations ............................................... 3.........4 Example ............................................................................................... 3... 3.......2 RBD and Fault TreetoEvent Tree Transformation.........3 Path Sets.............................................................7...................................................................... 3....................9......................... 3..........10 CauseConsequence Analysis...................................6................................................ 3.............4..............6..6................... 3....................4 Example ............................................................................ 3................................................................................... 3..............................9..............................6 Limitations .............. 3.........................3................................................... 3...........................................10.......................................... 3.................... 3................................................................................9....... Event Tree Analysis..............2 Application...........................................3 Procedures...............8................................................10.... 3................ Bibliography........................ 3.3 Procedures....................6 Limitations .................. 3.........7 Bibliography..........5 3........1 Description .. 3...............7 Success Tree Analysis ......................................9......... 3........... 3.....1 Fault Tree to RBD Transformation...........4 Example ......7...............5 Advantages................................................7 Bibliography...................8.....................................3 RBD to Fault Tree Transformation................ 3........................8.......................................................4 Example ..........................................3....6....6 3.... 3....................5 Advantages.......1 Description ........................................9.........................................................................................................4................................................ 3.........................6.............................................................. 3......................................TABLE OF CONTENTS (Continued) Page 3........................................................................9 3.......................8..................................................................................................................................................... 3.........2 Cut Sets ...........................................................................................................................................4 Event Tree to RBD and Fault Tree Transformation ........................................................................ 3................................................10...7 3......9................ 3..................................................8................ and Event Tree Transformations .............................................6 Limitations ...8.............................................................. 3...................................................................................9........................................................5 Advantages............................ 3..........1 Description ...............................................9........................................4 Examples.... 344 344 345 346 346 347 347 348 348 348 349 350 351 351 351 351 351 352 353 354 354 354 356 356 356 356 356 356 356 358 358 358 361 361 361 361 361 362 362 364 364 365 366 3.3....................
....11.........................................4 Advantages................ 3..............................................................................11 Directed Graphic (Digraph) Matrix Analysis ..................................3 Procedures........ 3............................................15........................... 3...........................12..................12............. 366 366 366 367 369 370 372 372 372 372 373 373 374 374 374 376 376 376 376 377 378 378 378 378 378 380 383 383 384 384 384 384 385 385 386 viii ......14...................14..................... 3.......1 Description ...........................2 Application......................13................1 Description .......................................................................................5 Limitations ............................5 Advantages...................................................................... 3............................................15...............................................14..........................................15 Probabilistic Risk Assessment..................................3 Procedures..........3 Procedures..6 Limitations ...........................4 Example ................ 3...................................................................................................................................................... 3................................. 3................ 3..5 Advantages.............. 3.... 3.................................................................................12...................................................................................................................5 Advantages.......11......3 Procedures............................. 3.............................................................................................................13........................................ 3....................13.................................. 3............. 3.......................... 3............................................................ 3.............. 3........13 Failure Mode Information Propagation Modeling...............................................4 Example .......................................................................................................................11.................................................................................................. 3.................................................6 Limitations .... 3..1 Description ........................................................................... 3......... References ...............................14 Probabilistic Design Analysis....4 Advantages............................ 3.....2 Application..........................................................................................................................5 Limitations .......................... 3............................................................................................................................................................................................................................... 3....................14..............TABLE OF CONTENTS (Continued) Page 3...............................................................................................15.......................................... 3.....................1 Description .............................12 Combinatorial Failure Probability Analysis Using Subjective Information......................................................................................................................................... 3.. 3....................12. 3.1 Description .........................11........13................6 Limitations ......12................13.2 Application.....................................12...........................15...................................................... 3................................................ 3...................................................................................14..................2 Application................................................................................................ 3.... 3................7 Bibliography.....................................4 Example ......................................................11.13.....................................................15.....................................................11..................11.................3 Procedures..................... 3.........2 Application..............................
............3..................2 Application................... 5..2 Application.....2.......................................................2............................1 Description ....2..................................................................................... 4............................................. 4....................... 4................................................... 4.................2........5 Advantages................................................................2 4............................................................................ 41 41 41 41 41 43 44 44 45 45 45 45 46 47 47 47 47 47 47 48 49 49 49 410 51 51 51 51 53 53 53 53 54 54 54 55 55 56 56 4.................................................... 4............................................................................... DESIGNRELATED ANALYTICAL TOOLS..........................................TABLE OF CONTENTS (Continued) Page 4.......... 5...........................................................................4 Example ............................................... 4......................................................... 4..........................6 Limitations ..................................................................................6 Limitations . 4.........................................................................................................................................3 5.5 Advantages................................................................................3..............................................................................................2..........................................2........................ 5.............................................................................4 Example ....................... 4.......................................... Control Chart ............................................................................. 4.....3...... 5............... 5.........5 Advantages.............................2..........1...........2.....1 Scatter Diagram ..............................................................6 Limitations ......................................................... 4....................1 Description .....................4 Example .....................................3...................................2 Application................. References ...1.................................5 Advantages.................................................................5 Advantages............................ 4......................2..............................1.........................................1..........2 Application......................3 Procedures.....................................................1....................3.......................................................... 5............................................................................ Tolerance Stackup Analysis .............................................................................. 4... 5........................1..........4 Example ..............2 Application.........................2.............................................. 5..... 5............................6 Limitations .............................................................. 5....................................................1 Description ............ 4.........3...... 4...........3..........1......... 4.................................................3 Procedures................................................................................................................ 5............................. 5...............................................6 Limitations .................................................................................................1.........1.........2 ix ..2....................................3 Procedures............................1 Sensitivity (Parametric) Analysis .................................................. 4..............1..............................7 Bibliography......1 Description ............................................................................................. 5.......................................................................................................... 4............................... GRAPHICAL DATA INTERPRETATION TOOLS................................................................................1...................................... Standard Dimensioning and Tolerancing ............................................................. 4................3 Procedures.......................1..............................2.......................3 Procedures............................... 4.1 Description .......................................................................................................................................................................................................................................4 Example ................... 5..................
.............................................................. 5.........................................................1 Description ........................6 5....... 5...............6 Limitations ..............................................3 Procedures.......... 5......................................................1 Description ..6....... 5...................3 Procedures........................3........... 5...............................5........................................................................5............................................................................................................................................ 5............................................................................. Histograms..................1 Description ............ 5.......................................1 Description ...........................................................................................................4........ 5................................................................................................. 5............................................................................ 5............................... 5.................................................3.....6.......................................................................................................................3............................................................................6 Limitations ...................... 5......7...................4 Example ..................................................1 Description .............4 Example .......................................................................................... References.......................................................................4............................ 5......6 Limitations .............2 Application..................................................7 Bibliography........................................................................................................4...................................4 Example .............. 5............................... Pareto Chart .............................5 Advantages.........................................................2 Application.............................................................5 5......................... 5......................... 5...........................7 x .............5.....................3 Bar Chart.......................................................................................7.... 5... 5...........................5..........................................................................................................6....................................................................................................................4 Example ................2 Application................3 Procedures................4 5..................................................4..............2 Application....... 5..............................................................................................................................................................................................................3.......................... Stratification Chart.............6 Limitations .................................. TimeLine Chart ..........6..........................3 Procedures.....5 Advantages....... 56 56 56 56 57 57 57 58 58 58 58 58 58 59 59 59 59 510 511 511 511 511 511 511 511 512 513 513 513 513 513 513 513 514 514 514 515 5..5......................... 5.................6.........4...............................................................................7.................................. 5...................................6................. 5...........................................................TABLE OF CONTENTS (Continued) Page 5................... 5............................................................7........................................ 5...........................................................3..............................................................................................................5 Advantages................. 5. 5................................................4 Example ........................3........5 Advantages.4.....7................5 Advantages....................................... 5.............................. 5...................5. 5.................................... 5.......... 5...........................................................................................................................................................7...6....3 Procedures.......................................................................................6 Limitations .....................2 Application...............................................................................
..................................... 6...................................................................6 Limitations ....................................................................................................................................1.......1..................................................................................................3.................................................................................1................................................................................... 6.............................................................................................................5 xi .........................................................................2........................................................6 Limitations ..............2 Application.......................................................3............2......... 6.................................1 Description .......6 Limitations ..........4 Example .2 Application................1 Description .................................5 Advantages.................................... 6............................................................4 Example ............ 6.............. 6..........................................3 Procedures...1 “Studentt” Analysis ............4...............5..............................6 Limitations .........2...1 Description .............................................................................................3.................................................................................................. 6........................7 Bibliography...................... 6..........................3 6..4 Example .....................TABLE OF CONTENTS (Continued) Page 6..................................................4......4 6......................2......3 Procedures..... Factorial Analysis ..............................................................................3 Procedures............................................................................................4 Example ....................................................... 6...................3......3................. 61 61 61 61 63 63 63 64 64 64 64 64 64 66 67 67 67 67 67 67 68 68 68 68 68 68 69 610 612 612 612 612 612 613 614 614 614 6....................................... Analysis of Variance..................... Confidence/Reliability Determination and Analysis .................................................................. 6.......................................................................... STATISTICAL TOOLS AND METHODOLOGIES.................................................. Correlation Analysis ......... 6..........................................................5 Advantages.................................6 Limitations ................ 6....................5............................5.............. 6.................4............................................................................4...1.....................1 Description .......................1 Description .........................3.................................................................. 6. 6................................................................3 Procedures.................. 6...............................................4 Example ..............................5.........................4.............................................................................................5 Advantages....... 6...................................2 Application...................................2 6.............................. 6.. 6. 6........................... 6...............2............................................................5................................................ 6................... 6..................................................................................................................... 6........................................................................................3 Procedures.... 6...................................................................1..5 Advantages.................................... 6.2 Application.................................................................2 Application.................................................4...................5 Advantages......................................................................2................................................................ 6.................................................................................................................................. 6........................... 6.... 6......................... 6.........................5......1........................................................1...................... 6...................
........... 6.... References ................... 6...............7..................3 xii ....................6............7......................................................7 Bibliography.. 7.......................................................................................................................................................................6 Limitations ............ 6.6......1 Benchmarking.....6 Limitations ............. Concurrent Engineering............................. 6..........................4 Example .........2.4 Examples................... 6............................. 6.6............................... Cause and Effect Diagrams (Also Known as Fishbone Diagrams or Ishakawa Diagrams) ......3 Procedures.................................................................................. 7.............................................................1 Description ................... 6.........3..........................................1....2 Application................2 7..............................................................................................1..........5 Advantages..........................1.................................................................3.....................................................................................................1................................................................................................................................................................................................6.............................................................................................................................. 7..5 Advantages.................................................1 Description .................6 Limitations ............................................................... 7..............3............................................................................................7.................................................................. 7.......................6..............3.....................................................................6 Regression Analysis.............3 Procedures.......... Response Surface Methodology ..................................................... 7................................................... 6..................................1 Description ........................... 7......................................1 Description ....1 Description ........................6...........2.......................................2 Application.................3... TOTAL QUALITY MANAGEMENT (TQM) TOOLS ...........7. 614 614 615 615 616 617 617 617 617 617 617 618 619 619 620 71 75 75 75 75 76 76 76 77 77 77 77 78 78 710 710 710 710 710 710 711 712 712 6..................................................................... 6...................................2.........................................................................................................5 Advantages............................. 7...................... 6..............2...........................................................6 Limitations .............................7.............................................6 Limitations ............... 7... 7... 7............................................................................................................................ 7..................................................... 6................................................ 7........ 6.....4 Example ...........................TABLE OF CONTENTS (Continued) Page 6.................... 7.....2 Application.....3 Procedures............................................................................................................................................... 7........................................3 Procedures......................................................................3................................5 Advantages.... 7................................................ 7.............................7 7...........................................2...........................................................................2 Application..........................................................................................................................................................4 Example ...................... 7............................................. 7........2.................................3 Procedures.2..........................................7................................................................................. 7........2 Application....... 7..................1.....5 Advantages.....4 Example .......................................................................................................................................................................................................................1......................................................................................
..............................3 Procedures....................................................................................................1 Description .........7................. Design of Experiments ...8..................................................5 Advantages....... 7....6.............................................................................. 7..4 Example ...............................................................................8 xiii .......6........................................................................6 Limitations ...........................................................4...................................................3 Procedures.........................................7..................................................5................3 Procedures........................................................2 Application.. 7................................................................................................... 7................. 7.......................................................................................6 Limitations ........ 7.......................2 Application...................................................... 7....................................................................................... 7..................................7.......4........................................... 7....................................... Brainstorming .........................6 Limitations ....... 7.... 7......................................................................................................................................5 Advantages...........8.5 Advantages............................................7.............................TABLE OF CONTENTS (Continued) Page 7........................6 7...................................................... 7.....................................6............................................. 7......................6............................................................................................................................................................ 7................. 7...........................5.........7 7.................................. 7..5 Advantages..3 Procedures............1 Description ........................................8..5...1 Description .................................................................................................... 712 712 712 712 714 714 715 715 715 715 715 716 718 718 719 719 719 719 719 720 723 723 723 723 723 724 724 725 725 726 726 726 726 726 726 726 7.......................................................................................................5 7............................................................................ 7....... 7..4 Example ......2 Application............................................7 Bibliography........................................................................................ 7.....................4 Cost of Quality................. Checklists................................................................ 7....................................................5................................ 7.....................................................................................4.......... 7..................................4 Example ................5........................................6 Limitations . 7....................... 7...........................................................................5............................................. 7.8...........6 Limitations ...........................................................3 Procedures............4......................................................................2 Application...8.................................................................................................1 Description ........6... Evolutionary Operation ............... 7................4 Example .......6......................................................................................................................................................................... 7.............................................7............5.....................................................................................2 Application.5 Advantages.... 7..........8...................................................................................................................... 7.................................................. 7.........................7........................4.....4.1 Description ......4 Example ...................................................................................................................................................... 7.........
.12.........................................2 Application...............4 Example ..........................................................9..........12......................... 7........................13. 7.........6 Limitations ..............................................................10 Nominal Group Technique .. 7..............5 Advantages............................................................................ 7.....6 Limitations .......................................................................................................................................................3 Procedures.................................................12....................5 Advantages.........................................................12....10............................................. 7....................................................................................... 7..............................2 Application.......................................12 Quality Function Deployment .............................................................2 Application..............................................................................6 Limitations ............11................... 7......................................................................4 Example ............................................................................................................................................................10....................9 Delphi Technique............................4 Example ......13..... 7........................................................9.......................5 Advantages.........................................................13 Quality Loss Function........................................................7 Bibliography.............12................................................................... 7..............11 Force Field Analysis ............3 Procedures........1 Description .........................1 Description ...................10............. 7..................................................... 7......12.. 7..........................................................................................3 Procedures.....................13................9.................................................................................... 7.......................... 7.................................. 7..........................11...........................9......................................10........... 7.........TABLE OF CONTENTS (Continued) Page 7........................................13...............................6 Limitations ..........11..............5 Advantages............................... 7.........1 Description ............................................................................... 7........................................ 7................................... 7.......1 Description ............... 7....................................................................................................................................................................... 7.............................................................................................................4 Example ......................................................... 7.......... 7.................................... 7.........11...10... 7........... 7..................................6 Limitations ................................................ 727 727 727 727 728 729 729 730 730 730 730 730 732 732 732 732 732 734 734 735 735 735 735 736 736 737 740 740 740 741 741 742 742 743 744 744 7..............4 Example .................................. xiv .......11........................................1 Description ........9.... 7.......................................... 7.................... 7...............................................................................................................................................................5 Advantages...................................................3 Procedures....... 7......12...... 7...............................13.........................................................................2 Application........10......2 Application.....................................................................................................................................................................................................................3 Procedures.................13................................................................................................ 7...9.............................. 7..........................................................................................................11...............
......... 7................ 8.............................................................................................................................. 8......................................................... 7....................... 7....................................................................... 7..................1...2...15.......3 Procedures.....................1...................................................................2 Application...............................TABLE OF CONTENTS (Continued) Page 7................................. 7.............................................................5 Advantages..... 8.................................2 Application.............6 Limitations ....................................................... 7..........1...................2.......................................... References .......... 7...............................1 Description ..................................................................... 7.................14 Statistical Process Control ............................7 Bibliography...................................................2........2.............................................................................................................4 Example .................................................. 7........................... 7...................7 Bibliography........................................................1 Description .........................................................................................16.................. 7.............................................14...........................................................................1 Description ...............................................................................1............... 7........2............. 8....2 Application..............2 Application......................14.................................................................................... 8..........1 Description ................................................... 7...................................... 8.............................................16...........14..............................................................................................................................................................................................................................................................................................2 Application..1 Performance Trend Analysis .........................16............................................................ 8..............2 xv ...... Problem Trend Analysis .........................................2.............. 8................................................ 744 744 744 744 746 748 749 749 749 749 752 752 752 752 752 753 753 754 754 754 755 755 756 81 84 84 84 85 87 88 88 88 88 88 89 810 811 815 815 815 8.........6 Limitations .............16 Work Flow Analysis .............................................................................................. 8..........................................15..............................................................................16.................. TREND ANALYSIS TOOLS...........2.................5 Advantages.............. 8...............4 Example ...............7 Bibliography.....................................................4 Example ................................................................................16....................... 7............................. 8.3 Procedures...............................................................................6 Limitations ..................................................................... 7.....1......................................6 Limitations ..........3 Procedures...................................3 Procedures.5 Advantages................................. 7.......................................................................................... 8................................... 7................5 Advantages....................15...14.....................................................................4 Example ..................15..........................15....................................................1 Description .........................14.........................................................6 Limitations ..................................................................... 8.....................5 Advantages....................15 Flowchart Analysis ......................................................................................................................16...................... 7...............................................................................................14..................................................................14..... 8................................... 7.............1.............3 Procedures...................................................... 8..............1....15........................................................................... 7........4 Example ............................... 8....................... 7......................................................................................................
...5....................................... 8.....3.. 8................................................................................ 8......4.................... Reliability Trend Analysis.................................................................3............4 Example ......................4............... 815 815 816 816 818 818 818 818 819 819 820 821 822 822 822 823 824 824 824 825 825 825 826 826 827 A1 B1 C1 D1 E1 F1 8.................................. 8......................................................................................3.........6 Limitations .................... 8........................................................................................................................5.......................................................................................5 Appendix A ......................2 Application................. 8................................................ 8.....................................4 Example .........7 Bibliography........4...................................................... 8............................ 8..............................5 Advantages.............................................................................................................................................................................................................................................................................3...........2 Application.................................................................................7 Bibliography......................................................................... Appendix C ...............................5........................................... 8..................................................3..........................................................................................5.....................1 Description ......1 Description ..................3......... Appendix D ...................................... 8.........................................................5...................TABLE OF CONTENTS (Concluded) Page 8................................................................................................................................ Appendix B ................................................................................................................................................................................................................ Supportability Trend Analysis......3 Procedures............................. References.............................7 Bibliography.......................................................... 8.................................... 8........................................................................................................................................3.................................................3 Procedures.............................................................................................................................3 Procedures............................6 Limitations ....................6 Limitations ...............................3 Programmatic Trend Analysis ............................... 8.........................................4...........................4........................................5..................................................................5 Advantages.................................................... 8............................................5 Advantages....................................................................................................... 8................... Appendix F .................................................4........................................................... 8.............................................. 8.... xvi ..4 8............................................................................. Appendix E .......................... 8...............................4.........................................5..............................................4 Example ..................1 Description .....2 Application............................................... 8............ 8......
31........ 37.................... 317....... 314..... 313.......................................................................................... Fault tree construction process................... Typical PHA ..... Title Example utility functions .................................. 22................ Example of determining cut sets ........... 310.................................. 32............................................. 315........................................................ Typical FMECA worksheet ...................................... Example fault tree ................................ 35..... 319........ 321............................. Example of system breakdown and numerical coding ..................................................................... 323..................... Isorisk contour usage ........................... Failure probability propagation through OR and AND gates ...... 318...................................................... Typical complex RBD ...... 34........................................... 322........................................................ FMECA process flowchart ........................ Page 27 28 35 36 37 38 310 311 314 316 323 324 326 327 331 333 339 339 340 340 341 344 345 346 349 xvii .................. Exact solution of OR gate failure probability propagation ..................... 33................... Severity and probability interpretations ................................................................................................................................................................................................................................................. Risk plane ........................................................................................................................ Helpful hints in creating a risk assessment matrix .............. Success tree construction process ......... 36........................................................... 38......................................................... Relationship between reliability and failure probability propagation.......................... Example RBD . 320....................................................................................... Example weighted factor trade study summary table ................ PHA process flowchart ...........................................................................LIST OF ILLUSTRATIONS Figure 21................................................................... Example of determining path sets ................ Risk plane to risk matrix transformation .............................. 312.................... Typical risk assessment matrix ........ 316.......................... Log average method of probability estimation .... 311..................... 39......... Example of an FMECA ..........................
...... Example causeconsequence analysis ......................... Relationship between cause and consequence ....................... Example of dimensioning and tolerancing ............................................................. 341............. ................................................... Oring joint components ................................................... Oring joint .. 42................................ Example digraph matrix analysis .... Scatter diagram example.................... 335............. 343...................... 43............................................... Event tree to fault tree transformation ............................................................................ 326................ 329............................ 333....................... 330... 51........... Example ETA ........................... 41..... Fault tree to RBD transformation..... 337................................... RBD to fault tree transformation .......... 334......... 339.............. Event tree (Bernoulli model) .................................................. Title Example success tree .......................................................... Example combinatorial failure probability analysis .......................................................... 328..................................................................... Interference between load and capability density functions ..................... 340........................... Causeconsequence analysis format ............................................................................................................................ 338........................................................ Equivalent logic RBD and fault tree ............................ 332....... 327............................................................. Comparison between digraph and fault tree logic gates ............................................................. RBD to event tree transformation .. Construction of digraph adjacency matrix ......................................................................... 336.......................................................................................... 331............... Event tree (generic case) ............................. 342........... Control chart example.............................. Example failure mode information propagation model ............ Deriving cut and path sets from an RBD .....................................................LIST OF ILLUSTRATIONS Figure 324.............................................. Load and capability transfer functions .............................................................................. xviii Page 350 352 353 355 357 357 358 359 359 360 362 364 365 368 369 371 375 379 382 383 46 48 48 54 55 .......... 325............ 52.....
.... 71..... Cause and effect diagram on receiving telephone messages ......................................... 714................. Traditional view to meeting specification..................... Concurrent engineering example ................ 713......... Range chart showing mean range for each part .................................... Standard cost of quality curve.......... Design rework cause and effect diagram .................................... QFD example on automobile industry .... 715.............. Quality loss function for NIB.. 710........................... 57................................................... Control chart showing mean deviation for hole guide 1........ House of quality .. 75....... 712......................................... Control chart showing mean deviation for each part ...................................................................... 72.. Stratification (histogram) chart example .................... 61............. Histogram example ........................................ 78............................................................ Page 57 59 510 512 514 616 76 78 79 711 713 718 720 728 729 734 735 739 741 742 743 747 747 748 748 xix ...........................................................................................LIST OF ILLUSTRATIONS Figure 53............................................................. 54....................................... 76.......................................................... 717.................... 55.............................. Title Bar chart example .................................. 56.. Force field analysis example... Pareto chart example........... 719....... Comparative benchmarking .................... 79............................................ Timeline chart example ............ 74........ 73........................................................ Factor/level effects graph......................................................................................................................................................................................................................................................................................................................................................... Line generated with least squares method ................................. Sample of a partial igniter subsystem fault tree......... 711................... Pareto chart showing mean deviation for each hole guide............................... Quality loss function example......................................... EVOP example.................................................................................................................................. 716................ Fault tree sample with estimates assigned .... 718........................................... 77..............
....................................... 85 Title Example of topdown flowchart ....... Supportability trend analysis example ......................... Common flowchart symbols .......... 81.. 83..................................... Example of detailed flowchart ......... 721................................................................................................................. 724........................... 82.......... Work flow diagram example ....................................................................... 722......LIST OF ILLUSTRATIONS Figure 720. Performance trend analysis example ........................................................ Reliability trend analysis example ............ WFA example .................................................. 723........................ Page 749 750 751 753 755 87 812 819 823 825 xx ......................................................................................................................... 84....................................................................................... Programmatic trend analysis example ............. Problem trend analysis example ...........................
.................... Month’s cost of quality .......................................... Concept development tools and methodologies ....................... 72....... FTA procedures .................................................................. 62................................................................................................................................... 42. 33...... 37................................................... Examples of strategies to manage harmful energy flow . 74.............................................................. Combinatorial failure probability analysis subjective scale ....................................... Trial...................................................................................... Causeconsequence tree construction symbols . 63........ Simple RBD construction ......................................... 31................................................ Statistical tools and methodologies .. and results ................................. 23 factorial design data..................................... Fault tree construction symbols ....... System engineering “toolbox” project phase matrix ................................................................ 32..................................................... Page 13 15 22 25 210 32 33 320 330 336 338 342 363 373 42 44 52 62 69 610 72 714 716 716 xxi ................................................. Designrelated analytical tools and methodologies .......... 61................. 41............................................................LIST OF TABLES Table 11.. 35................................................... 12............... effects................... 51........... 36.................................. 39... Title System engineering “toolbox” function matrix .... Graphical data interpretation tools and methodologies ....... Factorial analysis example .................................................................. 38........... Symbolic logic techniques ........................ Sensitivity analysis calculations ......................... Example selection criteria for costversusbenefit analyses ..................... 73.................................................. System safety and reliability tools and methodologies ............................... Probability propagation expressions for logic gates ....... Factorial analysis factors and magnitudes .................................................................. 34 . Typical weighted trade study summary table .......... 71........................................ 22.............................. 23.................. 21.. TQM tools and methodologies ...........
........ QFD matrix sample calculations .......... 2 data .......................... 81..... Trend analysis tools and methodologies .................... 76.. Nominal hole size deviations and drill guide positions .............................. 1 and cycle No................ Motor postflight checklist ...... Concerns with assigned weighting factors .............. Comparison of EVOP cycle No............................................................ 712................................... 78........................................ Replacement technology concerns ........................ EVOP cycle No.................................................................. Page 717 720 721 722 727 731 733 737 746 83 xxii ........................................ 2 data ..................... 713................ 1 data ........................................... 710................LIST OF TABLES (Continued) Table 75.............................................................................................. EVOP cycle No................................. 79......................................... Title Calculation of effects ..... 711....................... 77...............
effects.ACRONYMS AHP AHPA AIAA ANOVA B/C CIL CIM CSF DAS DOE DOF DR EF ETA EVOP FMEA FMECA FTA IF L(y) LCL LDL LIB Analytical hierarchy process Analytical hierarchy process approach American Institute of Aeronautics and Astronomics Analysis of variance Benefittocost Critical items list Change in mean Compliance safety factor Data acquisition system Design of experiments Degreeoffreedom Discrepancy report External failure Event tree analysis Evolutionary operation Failure modes and effects analysis Failure modes. and criticality analysis Fault tree analysis Internal failure Loss function (quality) Lower control limits Lower decision line Larger is better xxiii .
ACRONYMS (Continued) LSL MTBF MTBR MTTR NASA NGT NIB PDA PHA PRA PRACA OSHA QFD RBD RSM SE SESTC SIB SME SMQ SMR SPC SRM Lower specification limit Mean time between failures Mean time between repairs Mean time to repair National Aeronautics and Space Administration Nominal group technique Nominal is best Probabilistic design analysis Preliminary hazard analysis Probabilistic risk assessment Problem reporting and corrective action Occupational Safety and Health Administration Quality function deployment Reliability block diagram Response surface methodology Standard error System Effectiveness and Safety Technical Committee Smaller is better Sum of mean error Safety and mission quality Sum of mean replicate Statistical process control Solid rocket motor xxiv .
ACRONYMS (Continued) SSE SSR SST STA TQM UCL UCLR UDL USL WFA Sum of squares error Sum of squares replication Total sum of squares Success tree analysis Total quality management Upper control limit Upper control limit range Upper decision line Upper specification limit Work flow analysis xxv .
xxvi .
rules. The reader is also cautioned to validate results from a given tool to ensure accuracy and applicability to the problem at hand. and (3) symbolic logic modeling tools used to understand the failure mechanisms of the system. or function problems with a design. (6) limitations. fit. section 3. and (7) bibliography and/or references. rather than direction or instruction for specific technique selection or utilization. and (3) the determination of the possibility or probability of having form. A) and. this is not intended to be restrictive. Concept development tools. while techniques have been categorized for recommended areas of use. use this reference as one source among many. A practicing. at which level of detail are applicable. in general. In addition. are useful when selecting the preferred option of several alternatives. 1. section 2. (2) application. Too often. INTRODUCTION 1. if practical. section 4. is defined as a set of procedures to accomplish a specific function. and what might be the expected “value added” for their purposes. System safety and reliability tools. (5) advantages. (2) identify failure modes and show their consequences or effects. Use of the techniques for the sake of “using techniques” is rarely resourceeffective. (3) procedures. Caution should be exercised in the use of these tools and methodologies. This toolbox is intended solely as guidance for potential tools and methodologies. (2) a method for specifying dimensions and tolerances. For each concept addressed in the toolbox. It is left to the user to determine which technique(s). (4) example. either in comparative or absolute terms. designoriented systems engineer has difficulty finding any ready reference as to what tools and methodologies are available. manufacturability.1 Purpose Many references are available on systems engineering from the project management perspective. weight. as applicable. Designrelated analytical tools. A methodology is defined as a collection of tools. Readers are encouraged to question. or to determine a tolerance or dimension necessary to avoid these problems. or perhaps determining the ratio of expected future benefits to the expected future costs.REFERENCE PUBLICATION SYSTEM ENGINEERING “TOOLBOX” FOR DESIGNORIENTED ENGINEERS 1. are applied to show (1) which parameters affect a system the most or least. Among these alternatives are such things as cost. 11 .2 Scope The tools and methodologies available to the designoriented systems engineer can be categorized in various ways depending upon the application. safety. these references are of only limited utility from the designer’s standpoint. A thorough literature search was performed to identify the prevalent tools and methodologies. These tools are also used to determine the probability of failure occurring or the reliability that a component will operate successfully. complexity. comment (app. the following information is provided: (1) description. and postulates to accomplish a purpose. The purpose of this system engineering toolbox is to provide tools and methodologies available to the designoriented systems engineer. A tool. address the following areas of concern: (1) identify and assess hazards. as used herein.
Phase B (concept definition)—the establishment of system design requirements as well as conceptually designing a mission. Total quality management (TQM) tools. Many excellent texts are available on statistical methods. (3) (4) (5) Table 12 provides a project phase matrix for all of the tools and methodologies identified in this toolbox.When there is a desire to monitor performance. important tools or methodologies may have been overlooked. Phase E (operations)—the deployment of the product and performance validation. If a tool or methodology should be considered for this toolbox. using all available human and capital resources. graphical data interpretation tools are typically applied. this document touches only lightly on this area. in all areas of an organization. Extensive research was performed in order to identify all prevalent tools and methodologies available to the designoriented systems engineer. identify relationships. Statistical tools and methodologies. To further illustrate how selected tools and methodologies in this toolbox are applied.2. Phase D (fabrication. 1. and to forecast future events. These functionality categories are found in reference 1. table 11 provides a functional matrix which categorizes the functionality of each tool or methodology into (1) data analysis. The ultimate objective for these tools is to assess the current status.3 Relationship With Program or Project Phases Each tool or methodology may be performed in a minimum of one of the following phases. of a project design cycle. quantitative tools that are used to identify potentially hazardous conditions based on past empirical data are trend analysis tools. and evaluation)—system verification. Though the entries in this matrix are a result of research by the authors. For this reason. (3) decision making. conducting feasibility studies and design tradeoff studies. Variations are identified and mathematical relationships are determined. other phases should be considered by the user for a particular tool or methodology. (5) prevention. test. as are software packages. or reveal the most important variables in a set of data. Finally. 12 . integration. are applied to continuously improve performance at all levels of operation.1. section 7. (2) problem identification. appendix A is provided for the reader to complete and return to the individual indicated on the form. compare sample statistics and population statistics. (1) (2) Phase A (conceptual trade studies)—a quantitative and/or qualitative comparison of candidate concepts against key evaluation criteria to determine the best alternative. Nevertheless. (6) creative. as described in reference 1. To assist in further defining optimal areas in which each technique may be useful. appendix B provides a case study illustrating the trials and tribulations of an engineer applying his recently acquired knowledge of the techniques to a given work assignment. and (7) graphical. section 6. Phase C (design and development)—the initiation of product development and the establishment of system specifications. Appendix C provides a glossary of terms applicable to the tools and methodologies in this toolbox. section 8. An entry of (1) for the phase means the technique is primarily performed in that phase and an entry of (2) means the technique is secondarily performed in that phase. (4) modeling. and misapplied. These tools are discussed in section 5.
5 5.1 2.3 5.10 3. Energy flow/barrier analysis Failure modes and effects analysis Reliability block diagram Fault tree analysis Success tree analysis Event tree analysis Fault tree/reliability block diagram/event tree transformations Causeconsequence analysis Directed graph (digraph) matrix analysis Combinatorial failure probability analysis using subjective information Failure mode information propagation modeling Probabilistic design analysis Probabilistic risk assessment Designrelated analytical tools Sensitivity (parametric) analysis Standard dimensioning and tolerancing Tolerance stackup analysis Graphical data interpretation tools Scatter diagram Control chart Bar chart Timeline chart Stratification chart Pareto chart Histograms Note: Functionality categories found in reference 1.11 3.Table 11.15 4.5 3.8 3.1.7 Tool or Methodology Concept development tools Trade studies Costversusbenefit studies System safety and reliability tools Risk assessment matrix Preliminary hazard analysis.1 5.2 4.7 3.3 3.14 3.2 3.3 5. System engineering “toolbox” function matrix—Continued Section 2.1 3.6 3.6 5.13 3. Data Analysis √ √ √ √ √ √ √ √ √ √ √ √ Problem Identification Decision Making √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ Modeling Prevention Creative Graphical √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ 13 .1 4.12 3.4 3.2 5.2 3.9 3.4 5.
3 7.8 7.7 √ √ 7.13 7.4 7.2 7.7 7.2 8.5 7. Section Tool or Methodology Statistical tools and methodologies “Studentt” analysis Analysis of variance Correlation analysis Factorial arrays Confidence/reliability determination and analysis Regression analysis Response surface methodology TQM tools Benchmarking Cause and effect diagrams Concurrent engineering Cost of quality Design of experiments Evolutionary operation Brainstorming Checklists Delphi technique Nominal group technique Force field analysis Quality function deployment Quality loss function Statistical process control Flowchart analysis Work flow analysis Trend analysis tools Performance trend analysis Problem trend analysis Programmatic trend analysis Supportability trend analysis Reliability trend analysis Data Analysis √ √ √ √ √ √ √ Problem Identification √ √ √ √ √ √ √ Decision Making √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ Modeling Prevention Creative Graphical 6. 14 .4 8.5 √ √ √ √ √ √ √ √ √ √ Note: Functionality categories found in reference 1.9 7.10 7.11 7. System engineering “toolbox” function matrix—Continued.1 6.5 6.16 √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ 8.1 7.3 6.6 6.1 8.14 7.3 8.12 7.1.15 7.6 7.4 6.2 6.Table 11.
2.2 1 1 2 2 2 3.10 3. and Evaluation Phase E Operations 2.15 2 2 2 2 1 1 1 1 1 1 1 1 1 1 2 2 2 2 1 2 1 1 1 1 1 1 2 Note: Phases discussed in reference 1.8 3.Table 12.1 3. Energy flow/barrier analysis Failure modes and effects analysis Reliability block diagram Fault tree analysis Success tree analysis Event tree analysis Fault tree/reliability block diagram/event tree transformations Causeconsequence analysis Directed graph (digraph) matrix analysis Combinatorial failure probability analysis using subjective information Failure mode information propagation modeling Probabilistic design analysis Probabilistic risk assessment Phase A Conceptual Trade Studies Phase B Concept Definition Phase C Design and Development Phase D Fabrication.11 3.1 2.3 3. 15 . Integration.2 3.14 3.5 3.7 3.4 3.12 3. Test.13 3.9 3. System engineering “toolbox” project phase matrix—Continued Code: 1—Primary 2—Secondary Section Tool or Methodology Concept development tools Trade studies Cost versus benefit studies System safety and reliability tools Risk assessment matrix Preliminary hazard analysis.6 3.
2.6 5. Test. 16 .7 2 2 2 1 1 1 1 1 2 1 1 1 2 2 2 2 1 2 1 Note: Phases discussed in reference 1.5 6.4 6.3 1 1 1 5.Table 12. Integration.3 6.2 6. and Evaluation 1 2 1 Phase E Operations 4.1 4. System engineering “toolbox” project phase matrix—Continued Code: 1—Primary 2—Secondary Section Tool or Methodology Designrelated analytical tools Sensitivity (parameteric) analysis Standard dimensioning and tolerancing Tolerance stackup analysis Graphical data interpretation tools Scatter diagram Control chart Bar chart Timeline chart Stratification chart Pareto chart Histograms Statistical tools and methodologies “Studentt” analysis Analysis of variance Correlation analysis Factorial arrays Confidence/reliability determination and analysis Regression analysis Response surface methodology Phase A Conceptual Trade Studies Phase B Concept Definition Phase C Design and Development Phase D Fabrication.5 5.4 5.2 4.2 5.6 6.3 5.1 5.1 6.7 1 1 1 1 1 1 1 6.
2 8.13 7.8 7.1 7. and Evaluation Phase E Operations 7. Code: 1—Primary 2—Secondary Section Tool or Methodology TQM tools Benchmarking Cause and effect diagrams Concurrent engineering Cost of quality Design of experiment Evolutionary operation Brainstorming Checklists Delphi technique Nominal group technique Force field analysis Quality function deployment Quality loss function Statistical process control Flowchart analysis Work flow analysis Trend analysis tools Performance trend analysis Problem trend analysis Programmatic trend analysis Supportability trend analysis Reliability trend analysis Phase A Conceptual Trade Studies Phase B Concept Definition Phase C Design and Development Phase D Fabrication.11 7.5 7.7 7. System engineering “toolbox” project phase matrix—Continued.5 2 2 2 1 1 1 1 1 Note: Phases discussed in reference 1.2 7.6 7.3 7.10 7.16 2 2 2 2 2 1 1 1 1 1 2 2 1 1 2 2 1 1 1 2 1 1 2 1 1 2 2 2 1 1 1 2 1 2 1 8.3 8.Table 12.1 8. Integration. Test. 17 .9 7.2.4 8.4 7.12 7.14 7.15 7.
Santa Rosa. “System Engineering Process (Short Course Lecture Notebook).2 Brocka.” Business One Irwin. September 1991. 18 . and Brocka. B.REFERENCES 1. Homewood. Implementing the Best Ideas of the Masters. California. Illinois 60430.” Center for Systems Management (CSM).S. M.1 1..: “Quality Management.
safety. A key to any trade study is the initial selection and prioritization of specific desirable attributes. This is often very difficult and the prioritization delineation may change during the early phases of the program. A summary of the advantages and limitations of each tool or methodology discussed in this section is presented in table 21. section 2.2. are delineated with an associated weighting factor. Costversusbenefit studies.1. Pseudoquantitative numbers are then ascribed to the words and a score developed for each of the options. A trade tree can be generated with either of the above two options. These techniques are described in reference 2.2. are quantitative and/or qualitative comparison techniques to choose an alternative when considering such items as cost. etc. This approach allows for delineation of the facts and rationale that go into the subjective assessment of each of the options. safety. A trade tree is simply a pictorial representation of how highlevel alternatives (or issues) in the decision process are logically resolved into decreasingly lower level alternatives (or issues). very strong. All factors (program requirements) that are determined to be important. complexity. manufacturability. The options are then assessed with respect to each of the factors and an equation is developed that weighs this assessment. trade (or tradeoff) studies provide a mechanism for systematic depiction of both system requirements and system design options for achieving those requirements. Once tabulated. Trade studies.) assessments. 2. This is used when subjective verbal expressions (equal. completed trades should be performed to determine any impacts to their conclusions. pseudoquantitative equations may be developed (as in probabilistic assessment equations for failure causes in fault tree analyses) to increase confidence in analysis results. These tools are used to select the preferred option of several alternatives. It is very important. complexity.2.1. etc.1 TRADE STUDIES 2. A trade tree may be presented without results or simply as a representation of options. a comparison of relevant data (cost. provide a method to assess alternatives by determining the ratio of expected future benefits to expected future costs.) is then performed to rank those candidate design options in order of desirability. etc. The decision is then based upon the numerical results of the analysis. section 2. that when the prioritization changes. 21 . with the latter being a special version of the former.1. weight. The analytical hierarchy process (AHP) is a variation of the weighted factors analysis and is the most complex of the trade studies presented here. These studies are categorized as either a weighted factor trade study or an analytical hierarchy trade study. Further.) are easier to develop than numerical (3 versus 3. strong. The AHP provides a multicriteria analysis methodology that employs a pairwise comparison process to compare options to factors in a relative manner. and often overlooked. weight. A weighted factor trade study is usually performed when each of the options under consideration is very well defined and there is good definition of the program requirements as well. moderate.1 Description In general. a cursory look at the significant. CONCEPT DEVELOPMENT TOOLS Trade studies and costversusbenefit studies are presented in this section. etc. manufacturability.
(2) Improper generation of selection criteria.e. (4) Options evaluated are not determined as a result of the study but must be decided upon prior to the assessment by the operator.) ones. (3) The analysis is flawed if incomplete or inaccurate cost estimates are used. (2) Provides documentation of the parameters evaluated If the system requirements are too general or vague. the (1) The analysis is flawed if system requirements are analyst can assess the cost effectiveness of several incomplete or inadequate. measurable terms. (1) By performing a costversusbenefit analysis. (3) Identification of disadvantages of specific design option may lead to the definition of effective countermeasures if combined with other techniques. Costversusbenefit studies 2. reliability. and utility functions can prejudice the assessment and lead to incorrect results. the effectiveness of benefits can not be addressed in specific. Tool or Methodology Trade studies Section 2. which are often intangible or insubstantial and difficult to characterize in terms of monetary value. resources expended can be commensurate with the benefits of the task. i. (4) The analyst must be able to quantify the value of benefits. the total costs can be underestimated. characterized.2 22 .Table 21. (2) Adaptive to prioritization based upon programmatic (cost.. If the system operating alternatives over the entire life cycle of the proposed environment is not understood or accurately system under consideration. (2) The analysis is only as good as the list of alternatives considered. etc. (5) Weighting factors and advantages/disadvantages are very subjective (although objective data may be added which significantly complicates and enlarges the study) and this subjectivism is very near to the study conclusions.1 Advantages (1) Different kinds and/or levels of study allow flexibility in the depth of the review. weight factors. (3) The number of alternatives which can be considered is limited by the expenditure of resources required to perform the analysis. schedule) considerations as well as technical (weight. An incomplete list of alternatives will lead to an incomplete analysis. Concept development tools and methodologies. and the prioritized options considered. Limitations (1) Very dependent upon the expertise of the analyst and the amount of available accurate quantitative data.
The selection criteria are benchmarks to assess the effectiveness and applicability characteristics of the alternatives to be considered. These studies may also be used to help the designer delineate which system requirements are most important (used in conjunction with the Pareto chart analysis. Correlate directly to the established requirements and high priority issues. sec. showing the specific correlation. Provide a distinction between alternatives without prejudice. or performing makeorbuy decisions. trade studies can also be performed in phase B. manufacturability). Ideally. 2. or whenever a method is needed to select alternatives. an AHP weighted trade study will be performed. A large trade study may be resolved into several smaller trade studies with fewer required total comparison evaluations. this will aid in the weight factors for the selection criteria.6). The list may be reduced further by eliminating alternatives with low probability of successful implementation or those which are expected to exceed cost constraints. 23 (2) (3) (4) . However. 5. Be separate and independent from each of the other selection criterion in all aspects of the assessment. Develop a trade tree (optional). maintainability. Be practical to measure or predict within acceptable uncertainty and cost limits. create a trade tree to group alternatives with unique criteria attributes.1. and specific. evaluating design change proposals.2. A trade study analysis allows a systematic approach to evaluation of design options with respect to programmatic considerations or other. These alternatives can be imposed or obtained in brainstorming sessions (sec. (1) Define the mission objectives and requirements for the system under consideration. Develop and specify the selection criteria to be used in the analysis. These procedures are described in detail and were adapted from reference 2.1. A trade tree is developed to graphically illustrate the alternatives and how highlevel alternatives in the decision process are logically resolved into decreasingly lower level alternatives.1. Be expressed in general terms that mean the same thing to every evaluator. accurate. The list of alternatives selected during brainstorming sessions may be reduced by eliminating alternatives which do not appear capable of meeting requirements. For large trade studies with many alternatives and criteria attributes. d. Prioritize the objectives/requirements if possible. (A numbering system. such as selecting test methods. b.7).3 Procedures The procedures for performing a weighted trade study are presented below. c.2 Application These studies should typically be performed in phase A of NASA projects. The remaining alternatives should be described in sufficient detail that the relative merits between them can be ascertained. These objectives and requirements should be clear. Identify credible alternative candidates for the system under consideration. nonreliability related considerations (weight. By performing step 6. is often useful here. These requirements will provide the scope of the assessment and the basis for the selection criteria.) e. 7. the selection criteria should have the following characteristics: a. This will lead to fewer resources to conduct the assessment without degradation of the results.
Survey a group of qualified managers and engineers (or customers) to establish a consensus on the relative relationships between each attribute and the rest of the attributes. Establish a scale of the relative level of significance to the system objectives between two given criteria attributes. This technique is beneficial for very complex trade studies when operational data are not available and a subjective analysis is to be performed. The relationship may be a continuous function (not necessarily a straight line) or discrete values. If five definitions are used. Establish three to five definitions to subjectively define this scale of relative level of significance. risk. b. etc.(5) Establish weights for the selection criteria. The weights should be predetermined by the person (or group) with the ultimate decision authority. For attributes other than technical. low). The following steps define this process: a. (7) Generate utility functions (optional). modeling. Reserve the numerical values of 2. a subjective verbal scale may be used (i. and represents the resolution limit of the assessment. (6) Perform an analytical hierarchy process as described in reference 2. similarity comparison. A relationship is established between a measure of effectiveness for each selection criterion and a common scale (for example. 6. the analyst may use test data. such as cost. If attribute n has a numerical value of relative level of significance of “j” relative to attribute m. parametric 24 (8) . but not necessarily shared with the analysts to ensure that alternatives are assessed against each criterion objectively. This technique is used to establish a consistent scale for dissimilar criteria. The degree to which the individual criterion is resolved into components is dependent on how effective the criterion components can be evaluated. These weights should reflect the importance of each criterion relative to its importance to the overall selection decision. Generate clarifications for each definition so that qualified managers and engineers can subjectively use the definitions. and 9 to these definitions in order of increasing diversity between the given two attributes. Determine the relative weights for each criterion component by performing an eigenvector analysis. 4.. Each criterion may be resolved into several levels of components to establish its weight. engineering experience. Note that all elements of the diagonal of this matrix equal 1. 7. vendor provided data.. 5. The weights should be given numerical values to accommodate objective comparisons between unrelated criteria. Consult with the end user of the system (the internal or external customer) to verify that the selection criteria and weights are compatible with his needs. First estimate the performance of every alternative for a given criterion in terms of the measure of effectiveness used in generating the utility functions. d. Assess each alternative relative to the selection criteria. For the ideal situation. then attribute m has a numerical value of relative level of significance of “1/j” relative to attribute n. 0 to 10).e. e. medium.2 to establish weights for the selection criteria (optional). schedule. c. assign the numerical values 1. Determine the weight for all attributes by calculating the product of each individual attribute weighing factor and its weights of associated category headings. Create a normalized matrix (all the attributes versus all the attributes) with these relationships. and 8 as values to be assigned when interpolating between two of the definitions. The numerical values of the weight factors should sum to 100. 3. high.
(11) 25 . If.e. Typical weighted trade study summary table. This corresponds to a confidence band for the evaluation. however. Generate a matrix of criteria versus alternatives to summarize the results from the preceding steps. A typical table is illustrated in table 22 and was generalized from an example presented in reference 2. reconsider the selection criterion and weighting factors. Table 22.2. Select the superior alternative. multiply the scores for all alternatives by the weight factor for the criterion (determined in steps 5 or 6) to determine the weighted score for all alternatives for that criterion. the alternative numerical total score is too close to make a decision. determine the score for each alternative relative to a given criterion by correlating the estimate of performance for all the criteria to the mutual scale using the utility functions generated in step 7. Assume that a difference in the conclusion of less than onehalf the quantified number of a onestep difference is an equivalent answer. If quantification of qualitative ranking is required. and regenerate the summary table for the weighted trade study. (9) Tabulate the results. use caution in drawing conclusions. Next. If this is the case.1 Criteria Criterion. In reality. yj Weights w j = 100 Alternate x1 Alternates. this is often very difficult to perform objectively. after the analysis is repeated. Repeat this procedure for all criteria attributes. even when somewhat subjective (i. repeat the assessment. Examine the results of the weighted trade study to see if any total weighted scores of any alternatives are closer in numerical value than is warranted in making a decision due to the confidence levels of the performance estimates that had been used to established the scores. or other costeffective and reliable methods to generate the performance estimates. x i (x1 through xn) Alternate x2 Alternate x3 Through Alternate xn–1 Score (0–10) Weighted Score Alternate xn Score (0–10) Weighted Score Score (0–10) Weighted Score Score (0–10) Weighted Score y1 y2 y3 y4 to ym–1 ym Total w1 w2 w3 s11 s12 s13 w 1 s11 w 2 s12 w 2 s13 s21 s22 s23 w 1 s21 w 2 s22 w 2 s23 sn1 sn2 sn3 w 1 sn1 w 2 sn2 w 3 sn3 wm s1m w ms1m ∑(wj sij) s2m w 2 s2m ∑(wj sij)j snm w msnm ∑(wj sij)j (10) Perform a sensitivity analysis to evaluate the merit of the results relative to making an alternate selection. Next. then gather more data to increase the confidence level of the performance estimates. Select the alternative with the highest value of total weighted scores.1. heavy use of engineering experience)..analysis. It is worthwhile.
000 Alt. B 19 9 177 0. 26 . 20 Alt.2. and select the superior alternative.86 Low 21 Alt. C 23 10 190 0.1. D 18 12 197 0. skidpad) Implementing new technology risks Cost From the information given above. formulate a weighted factor trade study summary table. The selection decision will be based on comparing the four alternatives to the following criteria attributes and their associated weight factors: Item 1 2 3 4 5 6 Criteria Attribute Average fuel economy Acceleration (0 to 60 mph) Braking (70 to 0 mph) Road handling Implement new technology risk Cost Total Weight Factor 20 15 15 15 10 25 100 Utility functions have been generated for each criteria attribute and are presented in figure 21. A 16 7 180 0.4 Example Problem: Four alternatives for a new automobile design are being considered.78 Very low 22 Item 1 2 3 4 5 6 Criteria attribute Average fuel economy Acceleration (0 to 60 mph) Braking (70 to 0 mph) Road handling (300 ft dia.88 Avg. The estimates for each alternative relative to each criteria attribute are listed below: Measure of Effectiveness miles per gallon seconds feet g Dollars. × 1.83 High 24 Alt.
27 . mpg Very Very Avg.9 Road Handling.7 0. g 5 0 15 25 Cost. feet Score 10 5 0 0. seconds Score 10 0 175 200 Braking (70 to 0 mph). Example utility functions. High Low Implementing New Technology Risks 0 Score 10 5 Score 10 5 0 5 15 Acceleration (0 to 60 mph).000 Figure 21. dollars × 1.Score 10 5 Score 10 5 0 15 25 Average Fuel Economy.
alternative B is the preferred option.5 3 5 5 1. Based on the results of the trade study.5 4 1 160 112.5 37.).5 80 100 549. The method provides a clearly documented analysis in which the (a) prioritized objectives and requirements.5 Figure 22.5 Advantages The following advantages can be realized from performing trade studies: (1) (2) (3) (4) Different kinds and/or levels of study allow flexibility in the depth of the review.5 100 75 407.5 8 4 20 135 147 67.5 40 25 502. Criteria Attribute Item Weights wj =100 Alternate A Score (0–10) Weighted Score Alternates. Example weighted factor trade study summary table. schedule) as well as technical ones (weight. etc.5 2. (b) considered alternatives.e.9 7 6 5 80 120 148.5 105 60 125 638.1. i. and (c) selection methodology are recorded. x i Alternate B Score (0–10) Weighted Score Alternate C Score (0–10) Weighted Score Alternate D Score (0–10) Weighted Score 1 2 3 4 5 6 Total 20 15 15 15 10 25 1 9 9. 28 .Solution: Presented in figure 22 is the completed weighted factor trade study summary. resources expended can be commensurate with the benefits of the task.5 4 8 9.5 8 7.5 8. This technique is adaptive to prioritization based upon programmatic considerations (cost. 2.5 10 3 60 75 75 22.5 127. reliability. Scores were determined from effectiveness measures for all alternatives relative to all criteria attributes and the utility functions..8 4. Identification of disadvantages of a specific design option may lead to the definition of effective countermeasures if combined with other techniques.
52. this significantly complicates and enlarges the study) and this subjectivism significantly influences the study conclusions.” McGrawHill. Weighting factors and advantages/disadvantages are very subjective (although objective data may be added in the analytical hierarchy process approach (AHPA). while falling within overall cost restraints.5. 67–72.2.: “System Engineering and Analysis. 2. provide a method to assess alternates by determining the ratio of expected future benefits to expected future costs. as discussed in this section. 29 .: “Analytical Hierarchy Process. pp. will apply to the selection of system or system element alternatives based on their relative B/C ratios.1 Improper generation of selection criteria.2 COSTVERSUSBENEFIT STUDIES 2.2. 1990. Prentice Hall. 1980. as described in reference 2.S. 2. New Jersey. pp.2.2.” Second edition.” John Wiley & Sons. 2. Cross.2. L. 2. These studies can be used when two or more alternatives are being considered with fixed cost constraints. Costversusbenefit studies. The alternatives are ranked in decreasing order with the preferred option being the alternative with the highest benefittocost (B/C) ratio. however.1.32. These studies should typically be performed in phase A. Saate.4 or machines or systems2. Englewood Cliffs.6 Limitations The following limitations are associated with performing trade studies: (1) (2) (3) (4) (5) These techniques are very dependent upon the expertise of the analyst and the amount of available accurate quantitative data.. W.5 and costbenefit analyses. and utility functions can prejudice the assessment and lead to incorrect results.: “Engineering Design Methods.4 benefitcost ratio analyses. 2. N.1 Description Costversusbenefit studies are also known as benefitcost analyses.6 based on their relative B/C ratios. 1989. 101–121. they could also be performed in phases B or C.1.J. Both the expected future benefits and costs are expressed in terms of present value.2 Application Benefitcost analyses apply to the selection of projects 2.7 Bibliography Blanchard.32.2. fixed desired results or benefits. B. and Fabreycky.1 Options evaluated are not determined as a result of the study but must be decided upon prior to the assessment by the person (or group) with decision authority.1 The number of alternatives which can be considered is limited by the expenditure of resources to perform the analysis. or when both costs and desired results vary. weight factors. 2.6 Costversusbenefit studies. T.
3 Each alternative should be characterized to a level of completeness such that all substantial costs and benefits can be identified.3 Identify the benefit or detriments for each alternative. increased schedules. etc. reduced operating times. mutually exclusive alternatives. These requirements should be measurable and verifiable.6. etc. Translate general and vague requirements into specific.6 Note that the alternatives require an implicit determination of technical and schedule viability.2.3. (1) Define the requirements for the system or system element under consideration. value added due to increase productivity. if possible.2.2. Two alternatives are being considered with neither budget C or desired result B fixed. unless the ∆Bto∆C ratio is ≥1. Maximum B/C ratio. The benefits might include such items as increased performance. Since it is subjective. increased safety. Condition or Circumstance Budget C is fixed Desired result B is fixed. Then choose the higher cost alternative. the others are not to be implemented. 2.2. The example selection criteria presented in table 23 were adapted from reference 2. increased equipment operating costs. Calculate the ∆Bto∆C ratio between the alternatives. increased reliability. that is.5. The savings should include such items as residual or salvage values. and ongoing operating and maintenance expenses (including depreciation) for the life of the system. environmental impacts. Example selection criteria for costversusbenefit analyses.2.5. Selection Criteria Maximum B/C ratio.5.3 Procedures The following procedures to perform costversusbenefit studies were adapted from references 2. step 11). 210 (5) (6) .2. 0. Table 23. Choose the lower cost alternative.3. The detriments might include such items as loss of production time. and 2.6 Prioritize these requirements. (2) (3) Select alternative per benefitcost ratio More than two alternatives are being considered with neither budget C or desired incremental analysis (sec. compressed schedules. use of only three factors—0.3 The cost risk and technical maturity for each alternative may be included as a multiplying factor (f) for this analysis. quantitative requirements in which system effectiveness can be measured and assessed. 2. if one alternative is selected. result B fixed. reduced property value. Specify the time interval (expected operating life of the system) to which the analysis is to apply. The cost should include such items as initial investment. or 1—is probably as fine a distinction as is warranted.75. etc. Develop and specify the selection criteria to be used in the analysis. Define a list of credible. (4) Identify the cost or savings for each alternative.2. 2.
The preferred alternative is the last alternate listed in order of increasing cost whose incremental ∆B/Ci is >1. benefits.. Identify the interest rate that will be assumed for the analysis. the preferred alternative may not necessarily have the greatest B/C ratio. maximum sample 211 .(7) Develop cost and savings estimates and the benefits and detriments estimates for each alternative. then the increment is beneficial. 2.. perform steps 13 through 17. 2.5 Determine the total cost for each alternative by algebraically summing all costs as positive values and all savings as negative values. then the increment is not beneficial.3 The estimates for each alternative should be for the same time interval specified in step 6. For cases with cost restraints and desired results or benefits that vary.5 (8) (9) (10) (11) (12) Rank the alternatives relative to their respective costtobenefit ratios.3 Convert all costs. examine each distinct increment of increased cost investment. If the ∆B/Ci is <1. and select the superior alternative based on selection criteria established in step 3. Determine the total benefit value for each alternative by algebraically summing all benefits as positive values and all detriments as negative values.n listed in order of C.2. and detriments estimates to present worth values.2.2. Determine the incremental B/C ratio ∆B/C for each consecutive pair of alternatives with increasing total cost. (16) (17) Next. savings. If the ∆B/Ci is >1. where ∆Bi = Bi +1 – Bi and ∆Ci = Ci+1 – Ci for each ith pair of (n – 1) pairs of n alternatives where alternative i = 1. Each system has a different total cost and the capabilities of each system are different in terms of maximum number of channels.2. Therefore.. then do not give further consideration to alternatives with a B/C <1. Calculate the B/C ratio for each alternative by dividing the total benefit (B) by the total cost (C). Every attempt should be made to base cost and savings estimates on actual historical cost data.5 (13) (14) (15) If there exists any alternatives with a B/C ≥1.4 Example Problem: Five data acquisition systems (DAS) are under consideration to acquire data for solid rocket motor tests in a test stand over a 10yr time interval. ∆B/Ci = ∆Bi /∆ Ci .2. For cases with fixed cost restraints or fixed desired results or benefits..2. Order the remaining alternatives in sequence of increasing total C. perform step 12.
20 D 800k 750k 0. Perform a costversusbenefit analysis to determine the best alternative. Identify the preferred alternative as the last alternate listed in order of increasing cost whose incremental ∆B/Ci is >1.50 Step 3. Step 2. The present value of the estimated total cost and total value of combined benefits of the system are presented below. 212 . with an assumed annual interest rate of 10 percent. turnaround time between tests. the preferred alternative is DAS A. Therefore. List the remaining options in order of increasing total cost. this option will no longer be considered.rates. System Total cost (dollars) Total benefits (dollars) B 300k 400k E 400k 600k A 500k 750k C 750k 900k A 500k 750k 1.6 Step 4.0 A–E 100k 150k 1. Delete options with a B/C ratio <1. required maintenance. Determine the incremental B/C ratio ∆B/C for each consecutive pair of alternatives with increasing total cost. the last incremental ∆B/C with a value >1 is A–E.5 C–A 250k 150k 0. Increment ∆ Total cost (dollars) ∆ Total benefits (dollars) ∆ B/C E–B 100k 200k 2. The present values of cost and benefits were determined over a 10yr expected system life.50 B 300k 400k 1. System Total cost (dollars) Total benefits (dollars) B/C Solution: Step 1. Since the B/C for system D is <1. By inspection. data accuracy.93 E 400k 600k 1. and mean time between system failures.33 C 750k 900k 1.
: “Engineering Economy. 1989. system reliability and safety issues are not treated except by the selection of the alternative.J. (2) (3) (4) (5) 2.2.7 Bibliography Thuesen. An incomplete list of alternatives will lead to an incomplete analysis. but more appropriate to productionlevel design solutions. The method provides a clearly documented analysis in which the prioritized objectives/requirements. and Fabrycky.5 Advantages The following advantages are realized by performing costversusbenefit analyses: (1) (2) The analyst can assess the cost effectiveness of several alternatives over the entire life cycle of the proposed system under consideration. If the system operating environment is not understood or accurately characterized. measurable terms of effectiveness. 2. 213 .2.2.2. G. Englewood Cliffs.5 The analysis is only as good as the list of alternatives considered. except as a cost uncertainty factor.2. and the selection methodology are recorded. Prentice Hall.” Seventh edition.6 The analysis is flawed if incomplete or inaccurate cost estimates are used.2.J. If the system requirements are too general or vague. New Jersey.6 The analyst must be able to quantify the value of benefits.6 Limitations Costversusbenefit analyses possess the following limitations: (1) The analysis is flawed if system requirements are incomplete or inadequate. As cost is generally only one of many factors. new design efforts. benefits cannot be addressed in specific.2. this tool is generally insufficient for selection of large. the alternatives considered. which are often intangible or insubstantial and difficult to characterize in terms of monetary value.2.3 The analysis does not take into account technical complexity or maturity of an alternative. W.. Further. the total costs can be underestimated.
R.G. Inc. 140–155.4 2. McGrawHill Book Company. D.H.” Second edition. Chestnut. New York.6 “System Engineering Management Guide. 1983.J. pp.: “Quantitative Management in R & D. Beattie.5 2.L.: “Priority Setting in Complex Problems.” London.” Second edition.” IEEE Transactions. January 1990.. California..: “Engineering Economic Analysis. 214 . 1967.” John Wiley & Sons Inc.D. Newnan. Engineering Press. Chaplin and Hall Ltd. 1983..REFERENCES 2.1 2. Saaty. Jelen.” Defense Systems Management College. 1971. T.2 2.C..: “System Engineering Methods. F. J.: “Cost and Optimization Engineering..3 2. and Black. San Jose. New York. H. August 1983. and Reader. C.
Classically.4. The risk assessment matrix is discussed in section 3. then the model is generated in the failure domain and if the probability of success (PS ) is examined. These models are developed using forward (bottomup) or backwards (topdown) logic. If quantitative data are not available. the analyst can model either in the failure or success domain (or both domains). for example. When using forward logic the analyst builds the model by repeatedly asking. It is used in conjunction with hazard analyses. effects. Probabilities are propagated through the logic models to determine the probability that a system will fail or the probability the system will operate successfully.1. i. If the probability of failure (P F) is examined. then transform that model into the domain of another technique to exploit the advantages of both techniques. This means he starts by looking at a high level system failure and proceeds down into the system to trace failure paths. discussed in section 3. rather than “exact” determination of 31 . The symbolic logic techniques discussed in this section and their characteristics are presented in table 31. they can be further explored if failure modes of the elements of the system are known. the analyst repeatedly asks. can be used to identify failure modes and their consequences or effects. The PHA can be used to identify hazards and to guide development of countermeasures to mitigate the risk posed by these hazards. The energy/flow barrier analysis discussed in section 3. “What will cause a given failure to occur?” The analyst views the system from a “topdown” perspective. Once hazards are identified. Use of confidence bands is important. then subjective probability estimates may be used as described in section 3. reliability diagrams are generated in the success domain. the FMEA. When using backwards logic to build a model. Also discussed in section 3. the reliability.3 is also a technique to identify hazards and to evaluate their corresponding countermeasures. such as the preliminary hazard analysis (PHA) technique discussed in section 3. This means he starts by looking at the lowest level elements in the system and their functions. Methods are presented in section 3. and criticality analysis (FMECA). For convenience. The probability of a successful operation is the reliability.2. This device supports a standard methodology to subjectively evaluate hazards as to their risks. is a bottomup technique. or risk. Probability data may be derived from available empirical data or found in handbooks. and event trees are generated both in the success and failure domains. then the model is generated in the success domain. then convert the final probabilities to the desired domain using the following expression: PF + PS = 1. SYSTEM SAFETY AND RELIABILITY TOOLS This section describes several system safety and reliability tools available to the system engineer analyst.4 is the failure modes.e. Fault trees are generated in the failure domain. Often the value is in a comparison of numbers that allows effective resource allocation. Caution must be exercised when quoting reliability numbers.12. “What happens when a given failure occurs?” The analyst views the system from a “bottomup” perspective. The FMECA is similar to the FMEA but also addresses the criticality. Several symbolic logic methods are presented in this section. These tools are also used to determine either the probability of failures occurring or the probability that a system or component will operate successfully. associated with each failure mode.9 to transform any one of the above models into the other two by translating equivalent logic from the success to failure or failure to success domains. The failure modes and effects analysis (FMEA).. These methods construct conceptual models of failure or success mechanisms within a system. Sometimes it is beneficial to construct a model using one technique. Each of the symbolic logic techniques has its own unique advantages and disadvantages.3.
14.5 3.6 3.Table 31. the combined severity and probability of loss. 32 . Probabilistic design analysis (PDA) is discuss in section 3. Finally. Risk is the product of severity and probability (loss events per unit time or activity). probabilistic risk assessment (PRA) is discussed in section 3.7 3.10 3.8 3.1. As shown in figure 31. 3.1 Description The risk assessment matrix. The severity and probability dimensions of risk define a risk plane.13.1 Risk Assessment Matrix 3. and how and where the information should be measured in a system to detect the onset of a failure mode that could damage the system. as described in reference 3. Technique Reliability block diagram Fault tree analysis Success tree analysis Event tree analysis Causeconsequence analysis Directed graph matrix analysis Section 3. The definition of risk and the principle of the isorisk contour are the basis for this technique. A summary of the major advantages and limitations of each tool or methodology discussed in this section is presented in table 32. or the longterm rate of loss. Risk for a given hazard can be expressed in terms of an expectation of loss. is a tool to conduct subjective risk assessments for use in hazard analysis.11 Success Domain √ √ √ √ √ Failure Domain √ √ √ √ √ √ Forward (BottomUp) Backwards (TopDown) √ √ √ √ √ expected reliability levels. This is a general methodology that shows how most of the techniques mentioned above can be used in conjunction to assess risk with severity and probability. isorisk contours depict constant risk within the plane. This technique allows the analyst to determine what information is needed. Symbolic logic techniques. Note: the probability component of risk must be attached to an exposure time interval. Failure mode information propagation modeling is discussed in section 3. This technique uses advanced statistical methods to determine P F modes.15.1.
System safety and reliability tools and methodologies—Continued Tool or Methodology Risk assessment matrix Section 3. 33 . (2) May identify unnecessary design elements.9 Allows the analyst to overcome weakness of one technique by transforming a model of a system into an equivalent logic model in another analysis technique. A symbolic logic model that is relatively easy for the analyst to construct. Accident sequences through a system can be identified.7 Assesses probability of favorable outcome of system operation. (2) Functions simultaneously in failure and success domain. total calculated reliability may be unrealistically high. (2) Comprehensive trees may be very large and cumbersome.8 (1) Enables assessment of probabilities of coexisting faults or failures.3 (1) Does not address coexisting system failure modes. (2) Fails to identify certain classes of hazards. reliability. Does not address and their consequences. Does not address coexisting system failure modes. A criticality analysis provides a coexisting system failure modes. Identify hazards associated with energy sources and determines if barriers are adequate countermeasures.Table 32. and event tree transformations 3. (1) Addresses only one initiating challenge that must be foreseen by the analyst. (1) Addresses only one desirable event or condition that must be foreseen by the analyst. does not identify hazards. Energy flow/barrier analysis 3. Fault tree.2 Identifies and provides inventory of hazards and countermeasures. Reliability block diagram 3. (2) Discrete levels of success and failure are not distinguishable. Component reliability estimates may not be readily available. Limitations Only used to assess risk of hazards. Event tree analysis 3. Failure modes and effects (and criticality) analysis 3.4 Thorough methods of identifying single point failures Can be extremely labor intense.6 Success tree analysis 3. given component reliability. e. block diagram. asphyxia in oxygendeficient confined spaces.1 Advantages Provides standard tool to subjectively assess risk. Preliminary hazard analysis 3.5 Fault tree analysis 3. This technique offers no additional information and is only as good as the input model. (2) Comprehensive trees may be very large and cumbersome. (1) Enables assessment of probabilities of coexisting faults or failures.g. System reliability can be derived. risk assessment of these failure modes. (1) Addresses only one undesirable event or condition that must be foreseen by the analyst.. (3) End events need not be anticipated.
Tool or Methodology Causeconsequence analysis Section 3. (2) End events need not be anticipated. intolerable. (2) May be very subjective as to consequence severity. Extrapolation between populations can render technique nonviable.10 Advantages (1) Enables assessment of probabilities of coexisting faults or failures. (1) Analyst must have significant experience in probability and statistical methods to apply this technique. and senseless risk.14 (1) Allows the analyst a practical method of quantitatively and statistically estimating the reliability of a system during the design phase. Failure mode information propagation modeling 3. 34 .12 Allows analyst to perform qualitative probabilistic risk assessment based on the exercise of subjective engineering judgment when no quantitative data is available.15 Provides methodology to assess overall system risks. Techniques can be misapplied and results misinterpreted. may be poorly received. perform this technique may be limited. Performing the techniques of this methodology requires skilled analysts. (2) Provides alternative to the traditional method of imposing safety factors and margins to ensure system reliability. unless used in a comparative fashion. Probabilistic risk assessment 3. Should only be used when actual quantitative failure data is unavailable. (2) Data and results. Measurement requirements can be determined that if implemented can help safeguard a system in operation by providing warning at the onset of a threatening failure mode.13 (1) This technique is only applicable if the system is operating in a near normal range and for the instant of time just before initiation of a failure. (2) Only identifies single point (singleton) and dual points (doubleton) of failure. (3) Discrete levels of success and failure are distinguishable. That method might be flawed if significant experience and historical data of similar components are not available. singlepoint failure. Limitations (1) Addresses only one initiating challenge that must be foreseen by the analyst. and double point failures can be determined with less computer computation than with FTA. System safety and reliability tools and methodologies—Continued. avoids accepting unknown. Use of actual quantitative data is preferred to this method.11 (1) Allows the analyst to examine the fault propagation (1) Trained analyst and computer codes and resources to through several primary and support systems.Table 32. (2) Minimal cut sets. Combinatorial failure probability analysis using subjective information 3. Directed graph (digraph) matrix analysis 3. Probabilistic design analysis 3. (2) Historical population data used must very close to asplanned design population to be viable.
This technique is used as a predetermined guide or criteria to evaluate identified hazards as to their risks. Failure to assume credible (even if conceivable is substituted) may result in an optimistic analysis.SEVERITY SEVERITY and PROBABILITY. The PHA. The concept of the isorisk contour is useful to provide guides. 3. 32). defined in section 3. it will result in a nonviable analysis. conditions. the two variables that constitute risk.1. CATACLYSMIC R=K >K 3 In cr e R asi is n k g 2 R=K >K 2 1 R=PxS=K 1 LIKELY Isorisk Contours RISK is CONSTANT along any ISORISK CONTOUR. Risk should be evaluated for worst credible case. 0 NEVER PROBABILITY is a function of EXPOSURE PROBABILITY INTERVAL. Figure 31. 35 . Risk plane.2. convention. and acceptance limits for risk assessments (fig. is such an analysis. Use of this tool allows an organization to institute and standardize the approach to perform hazard analyses. define a RISK PLANE.2 Application The risk assessment matrix is typically performed in phase C but may also be performed in phase A. These risks are expressed in terms of severity and probability. not worst conceivable case.
an isorisk contour gives its probability at all severity levels. improbable. such as catastrophic. Be wary of exceptions — usually highenergy cases. such as frequent. The following hints will be of help when creating the matrix: a. Create a matrix of consequence severity versus the probability of the mishap. and impossible (adapted from MIL–STD– 882C). occasional. Categorize and scale the subjective severity levels for each target.1. assess Risk for the Worst. (2) (3) (4) 36 . (Most. Approximate the continuous. isorisk contour functions in the risk plane with matrix cells (fig. probable.” is an exception (fig.1. 33).3. for developing a risk assessment matrix are presented (1) Categorize and scale the subjective probability levels for all targets. 1 RISK ASSESSMENT GUIDES: If Risk for a given Hazard can be assessed at any severity level. b. The lowest step. 34(b)). as described in reference 3. “impossible. marginal. (It’ll fall at the top end of its own isorisk contour. too many steps add confusion with no additional resolution (fig. Increase adjacent probability steps by orders of magnitude. 34(a)). downtime. and negligible. equipment. 3. remote.) SEVERITY NOT ACCEPTABLE PROVISIONALLY ACCEPTABLE ACCEPTABLE (de minimis) 0 0 PROBABILITY Figure 32. Note that not the analyst but management establishes and approves the risk tolerance boundaries.2 Note: A target is defined as the “what” which is at risk.Credible Severity of outcome. Avoid creating too many matrix cells. and environmental effects.3 below: Procedures Procedures. One typical breakout of targets is personnel. These matrix cells fix the limits of risk tolerance zones.2 RISK ASSESSMENT CONVENTION: If possible. Isorisk contour usage. but not all hazards behave this way. Since the assessment is subjective.) 3 ACCEPTANCE: Risk Tolerance Boundaries follow isorisk contours. product loss. critical.
34(d)). Risk plane to risk matrix transformation. There should only be as many zones. 37 . Avoid discontinuities in establishing the risk zones. and (3) routinely accepted (fig. i. PROBABILITY Figure 33. d. make sure every onestep path does not pass through more than one zone (fig. Assign its risk to the highest level severity cell just inside the acceptable risk zone.e. i. (1) unacceptable.. (2) accepted by waiver. Establish only a few risk zones.e..S E V E R I T Y “Zoning” the Risk Plane into judgmentally tractable cells produces a Matrix. Steps in the Matrix define Risk Tolerance Boundaries. as there are desired categories of resolution to risk issues. less familiar risks. c. This calibration point should be used as a benchmark to aid in evaluating other. PROBABILITY S I E V E II R I T Y III IV F E D C B A Matrix cells approximate the continuous. isorisk contour functions in the Risk Plane. 34(c)). The scenario should be familiar to potential analysts or characterize a tolerable perceivable threat. (5) Calibrate the risk matrix by selecting a cell and attaching a practical hazard scenario to it.
A Hazard’s Risk is either… • (3) Routinely Accepted • (2) Accepted by Waiver. But what are the rational functions for the many levels? 8 6 5 4 4 IV 9 FLAWED 8 7 F I S E V II E R I T III Y IV E D C B A 1 PROBABILITY Three zones will usually suffice. Figure 34. Helpful hints in creating a risk assessment matrix—Continued 38 . or • (1) Avoided. F I S E V E II R I T Y III 6 E 5 D 4 C 3 B 2 A 1 7 5 4 3 2 A 24cell Matrix can be resolved into 9 levels of “priority.Factors of 10 separate adjacent Probability Steps.” or even more. D = 10 E C = 10 D B = 10 C A = 10 B …but F = 0 (“Impossible”) F I S E V II E R I T III Y IV E D C B A 1 2 3 PROBABILITY (a) Useful conventions. 2 3 PREFERRED PROBABILITY (b) Do not create too many cells.
F I S E V II E R I T III Y IV E D C B A 1 ? ? 3 2 Can a countermeasure make the “leap” from Zone (1) to Zone (3) in a single step? FLAWED F PROBABILITY I S E V II E R I T III Y IV E D C B A 1 Make every onestep path from a high Risk Zone (1) to a lower Risk Zone (3) pass through the intermediate Zones (2). L I S E V E R I T Y II III IV V VI VII K J I H G F E D C B 1 A 2 Subjective judgment can’t readily resolve more than six discrete probability steps. Added steps become confused/meaningless. 2 3 PREFERRED PROBABILITY (c) Avoid discontinuities. Figure 34. 39 . 3 FLAWED F PROBABILITY I S E V II E R I T III Y IV E D C B A 1 Keep it SIMPLE! 4 6 = 24 cells is better than 7 12 = 84 cells 2 3 PREFERRED PROBABILITY (d) Do not create too many zones. Helpful hints in creating a risk assessment matrix—Continued.
310 .3.3. Figure 35. Typical risk assessment matrix. Severity of Consequences I CATASTROPHIC II CRITICAL III MARGINAL IV NEGLIGIBLE Probability of Mishap** F IMPOSSIBLE E IMPROBABLE D REMOTE C OCCASIONAL B PROBABLE A FREQUENT 1 2 3 Risk Code/ 1 Actions Imperative to suppress risk to lower level.2 is presented in figure 35. timelimited waiver. endorsed by management. NOTE Personnel must not be exposed to hazards in Risk Zones 1 and 2. Operation requires written. Example interpretations of the severity and probability steps for this matrix are presented in figure 36. : *Adapted from MILSTD882C **Life Cycle = 25 yrs. adapted from MIL–STD–882C.4 Example A typical risk assessment matrix.1. 2 3 Operation permissible.
II CRITICAL Severe injury or severe occupational illness 250K to 1M 2 weeks to 4 months Values as for loss Equipment Loss Mediumterm (15 yrs) environmental damage or requiring $250K$1M to correct and/or in penalties Shortterm (<1 yr) B PROBABLE Likely to occur several times in system life cycle C OCCASIONAL Likely to occur sometime in system life cycle Provide stepwise scaling of PROBABILITY levels for all TARGETS.Severity of Consequences CATEGORY/ DESCRIPTIVE WORD I CATASTROPHIC Death >1M PERSONNEL ILLNESS/ INJURY EQUIPMENT LOSS ($)** DOWN TIME PRODUCT LOSS ENVIRONMENTAL EFFECT LEVEL Probability of Mishap** DESCRIPTIVE WORD DEFINITION >4 months Longterm (5 yrs or greater) environmental damage or requiring >$1M to correct and/or in penalties A FREQUENT Likely to occur repeatedly in system life cycle Provide stepwise scaling of SEVERITY levels for each TARGET. III MARGINAL Minor injury or minor occupation al illness 1K to 250K 1 day to 2 weeks environmental damage or requiring $1K$250K to correct and/or in penalties D REMOTE Not likely to occur repeatedly in system life cycle. *Adapted from MILSTD882C **Life Cycle = 25 yrs. 311 . Figure 36. F IMPOSSIBLE Physically impossible to occur Decide on TARGETS. Severity and probability interpretations. but possible IV NEGLIGIBLE Minor environmentNo injury or illness <1K E IMPROBABLE Probability of occurrence cannot be distinguished from zero <1 day al damage. readily repaired and/or requiring <$1K to correct and/or in penalties PROBABILITY is a function of EXPOSURE INTERVAL.
and an assessment of their remaining risk after countermeasures have been imposed. “RiskBased Inspection – Development of Guidelines.” 3.6 Limitations The risk assessment matrix possesses the following limitations:3. Code of Federal Regulations.1 (1) The risk matrix provides a useful guide for prudent engineering.3.“System Safety.” NUREG/GR0005. Also.1 Description A PHA.” sec.1. 5000. (2) (3) The risk matrix provides a standard tool of treating the relationship between severity and probability in assessing risk for a given hazard.5 Advantages The risk assessment matrix provides the following advantages: 3.2. No.” sec.1 (1) (2) The risk assessment matrix can only be used if hazards are already identified. Assessing risk subjectively avoids unknowingly accepting intolerable and senseless risk. not quantitative. This method is subjective without data and is a comparative analysis only. 807. 29. often included is a tabular listing of countermeasures with a qualitative delineation of their predicted effectiveness.3. 3. and improves resource distribution for mitigation of loss resources. as described in reference 3.1. 1910. “System Safety Engineering & Management.1. Medical devices.119 (e).7 Bibliography Code of Federal Regulations.1. This inventory includes qualitative. “PreMarket Notification. allows operating decisions to be made.” NASA NHB 1700. This tool does not assist the analyst in identifying hazards. “Process Safety Management of Highly Hazardous Chemicals. 21. Department of Defense Instruction. vol. 312 . vol. produces a line item tabular inventory of nontrivial system hazards.2 Preliminary Hazard Analysis 3. vol. 3.36. 3. A PHA is an early or initial system safety study of system hazards.90. assessments of risks.
. mechanism (process).3. Review codes. Define the extent of the system to be assessed.2. and outcome (consequence). State other assumptions. e. These resources are targets. to be protected. but that assessment should be reviewed by a peer. f. are presented below: (1) (2) Identify resources of value.3 313 . Detect and confirm hazards to the system. c.2.). d. or whether current installed countermeasures will be considered. This tool allows early definition of the countermeasure type and incorporation of design countermeasures as appropriate. Review system safety studies from other similar systems. then a proficient engineer with knowledge of the system should identify the hazards. as described in reference 3. etc.2 Application PHA’s are best applied in phase C but may also be applied in phase B. A hazard is defined as an activity or circumstance posing “a potential of loss or harm” to a target and is a condition required for an “undesired loss event. A list of proven methods* for finding hazards is presented below: a. such as brainstorming (sec.3 Hazards should be distinguished from consequences and considered in terms of a source (hazard).3. Identify the targets threatened by each hazard. environment. facilities. These limits may be the risk matrix boundaries defined in a risk assessment matrix (sec. equipment. Tennessee. 3.3 Procedures A flowchart describing the process to perform a PHA is presented in figure 37.3. If schedule and resource restraints are considerations. at any point in the life cycle of a system. Interview current or intended system users or operators. Use intuitive “engineering sense. Examine system specifications and expectations. and consensus standards. *Provided (3) (4) courtesy of Sverdrup Technology. regulations. deactivation. activation. 3. D). g. however. Tullahoma. This tool is applied to cover wholesystem and interface hazards for all mission phases. 7.” 3. mission or test objectives. productivity. Consult checklists (app. maintenance. is recommended over a single analyst. Define the physical boundaries and operating phases (such as shakedown. Inc. emergency shutdown. A PHA may be carried out.7). Identify and observe the levels of acceptable risk that have been predetermined and approved by management. Procedures for performing PHA’s. standard operation. such as if the assessment is based on an asbuilt or asdesigned system.1).” b. etc. A team approach to identifying hazards. Examine and inspect similar facilities or systems and interview workers assigned to those facilities or systems. such as personnel.
Figure 37. PHA Process flowchart 314 .
Probability intervals expressed in hours. (6) Assess risk for each hazard using a risk assessment matrix (sec. or number of cycles or operations. Keep the following considerations in mind during the evaluation: a. Categorize each identified risk as acceptable or unacceptable. if unacceptable. target. or months are too brief to be practical. The probability for a specific hazard varies as a function of exposure time. and operational phase.h.” (2) “engineering safety systems. days. The interval should depict the estimated facility. weeks.” (4) “warning devices. Since probability is determined in a subjective manner.” (3) “safety devices. Brainstorm (sec. draw on the experience of several experts as opposed to a single analyst. This interval can be in terms of time. environment. The matrix should be consistent with the established probability interval and force or fleet size for this assessment. or each human operator working life span. Remember that severity for a specific hazard varies as a function of targets and operational phases. Select countermeasures in the following descending priority order to optimize effectiveness: (1) “design change. An interval of 25 to 30 yr is typically used and represents a practical value. j. National Safety Council data. what happens if they get out of control? (5) Assess worstcredible case (not the worstconceivable case) severity and probability for each hazard and target combination. 7. or personnel tendencies.1). Review historical documents—mishap files. b.” and (5) “procedures and training. k. What is necessary to keep them under control. c. A probability interval must be established before probability can be determined. OSHArecordable injury rates. i. The assessment will underestimate the true risk if a shortterm probability interval is used unless the risk acceptance criterion is adjusted accordingly. will induce two or more fault/failure conditions within a system. population. Consider all mission phases. 3. etc. manufacturer’s reliability analyses. Consider all energy sources. l.” A common cause is a circumstance or environmental condition that. equipment. Consider “external influences” like local weather. d. e. or develop countermeasures for the risk. Consider “common causes. m.7)—mentally develop credible problems and play “whatif” games. if it exists. nearmiss reports.3 (7) (8) 315 .”3.
with no added countermeasures. If Risk is not acceptable. Identify target(s).e. while in decreasing order of effectiveness.1) might be performed to determine a countermeasure of adequate effectiveness and minimized program impact. 3.. Inspect flange seal at 2month intervals. Describe newly proposed countermeasures to reduce Probability/Severity. I E 3 II D 3 III D 3 Show hazard alphanumeric designator.3 for a pressurized chemical intermediate transfer system is presented in figure 38. If added hazards or degraded performance are unacceptable. NOTE: THESE COUNTERMEASURES MUST BE IN PLACE PRIOR TO OPERATION. / Description SrdA.4 Example An example of a completed PHA worksheet3. Standard Stop. E—Equipment V—Environment Approved by/Date: Assess worstcredible Severity. P E I II D 2 C 2 T III C 3 Surround flange with sealed annular stainless steel catchment housing. and show Risk (from assessment matrix) for hazard. Emergency Shutdown 3 Analysis: SrdA (Chem/Int) System Number: __________ Revision X Initial Addition Hazard Target* Probability Interval: 25 years Date: 25 Feb. design changes have the highest potential for cost and schedule impact). and Probability for that outcome. 2. Describe hazard source mechanism. Routine Operation.e. Reassess Probability/Severity. determine if they introduce new hazards or intolerably diminish system performance. new countermeasures must be developed. and regasket during annual plant maintenance shutdown (P).042 — Flange Seal A29 leakage. presuming new countermeasures to be in place. 1993 Risk Before Probability Risk Code Severity Description of Countermeasures Severity Identify countermeasures by appropriate code letter(s): D = Design Alteration E = Engineered Safety Feature S = Safety Device W = Warning Device P = Procedures/Training Risk After Probability Risk Code Hazard No. is also typically in decreasing order of cost and schedule impact (i. A trade study (sec.Note that this delineation.2.a. (A blank form is included in appendix E. Typical PHA.) Preliminary Hazard Analysis Brief Descriptive Title (Portion of System/Subsystem/Operational Phases covered by this analysis): Pressurized UnFo Containment and Replenishment Reservoir and Piping / Startup. Prepared by/Date: *Target Codes: P—Personnel T—Downtime R—Product Figure 38. determine new countermeasures and reevaluate the risk. If countermeasures are developed. producing toxic vapors and attacking nearby equipment.. Provide personal protective equipment (Schedule 4) and training for response/cleanup crew (S/P). Show Risk (from assessment matrix) for hazard “asis” — i. releasing pressurized UnFo 3 chemical intermediate from containment system. (9) (10) Reevaluate the risk with the new countermeasure installed. worstcredible outcome. with gravity runoff conduit led to DetectoBox™ containing detector/alarm device and chemical neutralizer (S/W). 316 .
Hazard targets (check boxes for personnel. Risk assessment after countermeasures are considered. product environment). fig. In any case. Date of analysis. including severity level. h. 35). Although helpful. or operation covered in the analysis. an organization may create their own worksheet customized for their operation.2. great care should be given in designing the form to encourage effective usage. Risk assessment before countermeasures are considered. f. System number. different target types may be listed. i. c. j. Provides a relatively quick review and delineation of the most significant risks associated with a specific system. probability level. including severity level. d. Provides a logically based evaluation of a system’s weak points early enough to allow design mitigation of risk rather than a procedural or inspection level approach. Hazard (description and identification number). and risk priority code. probability level. subsystem. 317 . Declaration of the probability interval. a PHA is not a structured approach that assists the analyst in identifying hazards or threats. Brief description of the portion of the system. equipment. Provides information to management to make decisions to allocate resources and prioritize activities to bring risk within acceptable limits. Description of countermeasure (with codes for various types).5 Advantages A PHA provides the following advantages:3.3 (1) (2) (3) (4) Identifies and provides a log of primary system hazards and their corresponding risks. Signature blocks for the analyst and reviewers/approvers. 3. g.Note that the worksheet from this example contains the following information: a. e. downtime. However. For example. The PHA worksheet used in the example is typical. b. and risk priority code (zone from risk matrix.
R. Raheja..2.G. and Kumamoto.2.: “Probabilistic Risk Assessment. Therefore a false conclusion may be made that overall system risk is acceptable simply because each hazard element risk identified. gloves. 1980. fences.: “System Safety Engineering and Management. when viewed singularly. Opportunities for undesired energy flow between the sources and targets are assessed. is acceptable.3 (1) A PHA fails to assess risks of combined hazards or coexisting system failure modes. Inc. H. The energy flow/barrier method is a useful supplement to the PHA discussed in section 3..3. lead shields. safety glasses.3 Energy Flow/Barrier Analysis 3. 1983.L. chemical..2. Browning. are identified. if too many targets or operational phases are chosen. radiation. blast walls.” McGrawHill.E. W. Inc. the assessment will be flawed. “Assurance Technology and Application . Barriers are countermeasures against hazards caused by flows from these energy sources to targets.W. etc. 1991.1 Description The energy flow/barrier analysis. used to identify hazards and determine the effectiveness of countermeasures employed or suggested to mitigate the risk induced by these hazards. (2) 3.7 Bibliography Air Force Systems Command Design Handbook DH 16..” Garland STPM Press. 3. Hammer. Inc. Henley. Roland. S. Examples of barriers include barricades.J.” Marcel Dekker.” PrenticeHall. the effort will become too large and costly to implement.: “The Loss Rate Concept in Safety Engineering. D.4..” May 1990. If inappropriate or insufficient targets or operational phases are chosen.” December 1982.” John Wiley & Sons. While on the other hand. This tool is also known as energy trace/barrier analysis. 1982. etc. Army Regulation 389516. 1972. New York. 1991. Malasky.. procedures.6 Limitations A PHA possesses the following limitations:3.Principles and Practices.: “System Safety: Technology and Application. and Moriaty. “System Safety. Inc. is a system safety analysis tool. E. Energy sources such as electrical.” The Institute of Electrical and Electronic Engineers.3. H. mechanical. as described in reference 3.: “Handbook of System and Product Safety.. 318 . “System Safety Engineering and Management. B.
This analysis can also be applied in failure investigations. 3. mission or test objectives. Are existing barriers sufficient countermeasures to mitigate the risk to the targets? (3) Consider the following strategies extracted from reference 3. i.3. This assessment can be applied during phase C but may also be applied in phase E or phase B. Control improper energy input. Is the energy flow unwanted or detrimental to a target? c.? Remember every energy source could have multiple flow paths and targets. Limit quantity and/or level of energy. g. c. Prevent the release of energy. Strengthen potential target.3. e.4. h. b.3. are presented below: (1) (2) Examine the system and identify all energy sources. facilities.4 to control harmful energy flow: a. Examine each potential energy flow path in the system. 319 . d. as described in reference 3. b. environment.2 Application An energy flow/barrier analysis can be beneficially applied whenever assessments are needed to assure an identified target is being safeguarded against a potential energy source that can impose harm.3 Procedures Procedures to perform an energy flow/barrier analysis. Separate energy from target in time and/or space. Modify the rate of release of energy. Modify target contact surface or basic structure. Consider the following for each energy flow path: a. Eliminate energy concentrations. Isolate by imposing a barrier. 3. productivity. etc. f. equipment.3. such as personnel. What are the potential targets.4 Example Examples of strategies to manage harmful energy flows are presented in table 33.
Examples* of strategies to manage harmful energy flow. Tennessee. Tullahoma.Table 33. keyed electrical connectors · Use matchthreaded piping connectors · Use back flow preventors Limit quantity and/or level of energy Prevent release of energy Modify rate of release of energy Separate energy from target in time and/or space Isolate by imposing a barrier Modify target contact surface or basic structure Strengthen potential target Control improper energy input *Examples provided courtesy of Sverdrup Technology Inc.. Strategy Eliminate energy concentrations Examples · Control/limit floor loading · Disconnect/remove energy source from system · Remove combustibles from welding site · Change to nonflammable solvent · Store heavy loads on ground floor · Lower dam height · Reduce system design voltage/operating pressure · Use small(er) electrical capacitors/pressure accumulators · Reduce/ control vehicle speed · Monitor/limit radiation exposure · Substitute less energetic chemicals · Heavywall pipe or vessels · Interlocks · Tagout – lockouts · Doublewalled tankers · Wheel chocks · Flow restrictors in discharge lines · Resistors in discharge circuits · Fuses/circuit interrupters · Evacuate explosive test areas · Impose explosives quantitydistance rules · Install traffic signals · Use yellow nopassing lines on highways · Control hazardous operations remotely · Guard rails · Toe boards · Hard hats · Face shields · Machine tool guards · Dikes · Grounded appliance frames/housing · Safety goggles · Cushioned dashboard · Fluted stacks · Padded rocket motor test cell interior · Whipple plate meteorite shielding · Breakaway highway sign supports · Foamed runways · Select superior material · Substitute forged part for cast part · “Harden” control room bunker · Crossbrace transmission line tower · Use coded. 320 .
This tool also fails to identify certain classes of hazards. to implement effective risk mitigation and countermeasure. 3.2).g.. and consequent reductions in risk can be evaluated.5 Advantages The energy flow/barrier analysis provides a systematic thought process to identify hazards associated with energy sources and determines if current or planned barriers are adequate countermeasures to protect exposed targets. In its practical application. 3. FMEA and FMECA are useful tools for cost and benefit studies (sec. Due to design and performance requirements. Inc.1 Description An FMEA.4. 3.7 Bibliography Department of Energy.D. August 1973.5. asphyxia in oxygendeficient confined spaces.: “Energy Damage and the Ten Countermeasure Strategies.4 (1) Even after a thorough analysis. 321 .3) to establish the limit of analytical resolution. “Barrier Analysis. as described in reference 3. An FMECA also addresses the criticality or risk of individual failures.4.3.4 Failure Modes and Effects (and Criticality) Analysis 3. it is not always obvious that energy may be reduced or redirected.” Haddon. electrical.4 3.2). Jr. 3.3.” Marcel Dekker. e. 1980. 2.” Human Factors Journal. potential versus kinetic mechanical energy. DOD 76–451: SSDC–29. Like the PHA (sec. tabular technique that explores the ways or modes in which each system element can fail and assesses the consequences of each of these failures. is a forward logic (bottomup).3. its use is often guided by topdown “screening” (as described in sec. etc. and as precursors to a fault tree analysis (FTA) (sec.. A reexamination of energy as heat. W. may aid this thought process. Johnson. for each failure mode..3.. W. M. an energy flow/barrier analysis fails to assess risks of combined hazards or coexisting system failure modes.G. all hazards might not be discovered.6 Limitations The energy flow/barrier analysis possesses the following limitations.: “MORT Safety Assurance Systems. chemical.5). Countermeasures can be defined.3. (2) (3) 3. 3.
4. These resources are targets. Establish the mission phases to be considered in the analysis. etc. By answering the following questions posed in reference 3. the analysis is complete. these analyses can be done with or shortly after the PHA (sec. etc. 310): (4) (5) Identify resources of value. Document the results. assemblies. subassemblies. components. These tools are applicable within systems or at the systemsubsystem interfaces and can be applied at the system. are presented below. These limits may be the risk matrix boundaries defined in a risk assessment matrix (sec.5. These system elements include subsystems. productivity. effect.) If the answer is yes. and associated risk (FMECA) of component failures have been appropriately addressed.3.2). subsystem. components and parts lists. (This has the additional benefit of providing visibility of nonvalue added systems. to be protected. and piece parts. (6) 322 . facilities. 3. Procedures for preparing an FMEA are the same with steps 8 through 12 omitted. such as personnel. The vulnerable points identified in the analyses can aid management in making decisions to allocate resources in order to reduce vulnerability. Singlepoint failures can be identified. 39).4. without loss of benefit: a. Develop a numerical coding system that corresponds to the system breakdown (fig.3 Procedures Procedures for preparing and performing FMECA’s. the scope and resources required to perform a classical FMEA can be reduced. ask the following question for each subsystem identified in step 2 above. These failure mode analyses are typically performed during phase C. During this phase.2 Application An FMEA can be used to call attention to system vulnerability to failures of individual components. equipment.5.2). drawings. or it may serve to correct incomplete criteria being used for the FMEA. Identify and observe the levels of acceptable risk that have been predetermined and approved by management. descriptions. Partition and categorize the system into advantageous and reasonable elements to be analyzed. environment. This tool can be used to provide reassurance that the cause. such as requirement specifications. or part levels. (2) (3) Steps in performing the FMEA or FMECA (see flowchart presented in fig. Steps prior to performing the FMEA or FMECA: (1) Define the scope and boundaries of the system to be assessed. as described in reference 3. 3. component. Will a system failure render an unacceptable or unwanted loss? If the answer is no. mission or test objectives. 3. Gather pertinent information relating to the system.
3. Will an assembly failure render an unacceptable or unwanted loss? If the answer for each assembly is no.Assembly No.System Subsystem 1 Subsystem 2 Subsystem 3 Subsystem 4 Assembly 1 Assembly 2 Assembly 3 Subassembly 1 Subassembly 2 Subassembly 3 Component 1 Component 2 Component 3 Part 1 Part 2 Part 3 Typical Coding System: Subsystem No. code number for part 2 above is 0301030102 Figure 39. Document the results. . ask the following question for each component of those subassemblies identified in step 2 above: 323 . If the answer is yes for any subsystem. d. . Document the results. Will a subsystem failure render an unacceptable or unwanted loss? If the answer for each subsystem is no. c.5 b. Document the results. Will a subassembly failure render an unacceptable or unwanted loss? If the answer for each subassembly is no. the analysis is complete. ask the following question for each component of those assemblies identified in step 2 above. . For example.Component No. . If the answer is yes for any assembly. the analysis is complete. the analysis is complete. If the answer is yes for any subassembly. ask the following question for each assembly of those subsystems identified in step 2 above. Example of system breakdown and numerical coding.Part No.Subassembly No.
324 . FMECA process flowchart.Figure 310.
A worksheet for an FMEA would be similar with the risk assessment information removed.5. Document your completed analysis on an FMEA or FMECA worksheet. If added hazards or degraded performance are unacceptable. component. If countermeasures are developed. Categorize each identified risk as acceptable or unacceptable. If the risk is unacceptable. develop new countermeasures and reevaluate the risk. the information found in an FMECA worksheet. the analysis is complete. Typically. Will a part failure render an unacceptable or unwanted loss? (7) For each element (system. effect. according to reference 3. or part) for which failure would render an unacceptable or unwanted loss.1). If the answer is yes for any component. The matrix should be consistent with the established probability interval and force or fleet size for this assessment. Then reevaluate the risk with the new countermeasure installed. What are the failure modes (ways to fail) for this element? b. Document the results. then develop countermeasures to mitigate the risk. f. ask and answer the following questions: a. and target combination. (10) (11) (12) (13) (14) 325 . is presented in figure 311. The contents and formats of these worksheets vary among organizations. assembly. A sample FMEA worksheet is attached in appendix F. Countermeasures may or may not be listed. 3. Will a component failure render an unacceptable or unwanted loss? If the answer for each component is no. determine if they introduce new hazards or intolerable or diminished system performance. ask the following question for each part of those components as identified in step 2 above. What are the effects (or consequences) of each failure mode on each target? (8) (9) Assess worstcredible case (not the worstconceivable case) severity and probability for each failure mode.e. Assess risk of each failure mode using a risk assessment matrix (sec. subsystem. subassembly.
A schematic of the system is presented in figure 312(a). Failure Mode Failure Cause Failure Event T a r g e t Risk Assessment S e v P r o b RC i o s d k e Action Required/ Comments Figure 311. 326 .FMEA NO: ________________________ PROJECT NO. and subassembly elements.5 is illustrated in figure 312.: __________________ SYSTEM NO.: ______________________ SUBSYSTEM NO.4 Example An example FMECA3. AND CRITICALITY ANALYSIS WORKSHEET SHEET ___ OF ____ DATE _________________________ PREPARED BY: __________________ REVIEWED BY: __________________ APPROVED BY:__________________ TARGET CODE: P – PERSONNEL / E – EQUIPMENT / T – DOWNTIME / R – PRODUCTS / D –DATA / V –ENVIRONMENT Id. The system being assessed is an automated mountain climbing rig. An FMECA worksheet for the control subsystem is presented in figure 312(c).3.4. Typical FMECA worksheet. INTERVAL: ___________________ FAILURE MODES. No. Figure 312(b) illustrates the breakdown and coding of the system into subsystem.: ______________________ PROB. EFFECTS. Item/ Functional Ident. assembly.5 3.
Example of an FMECA—Continued 327 .(a) System. Subsystem Hoist (A) Assembly Motor (A01) Subassembly Windings (A01a) Inboard bearing (A01b) Outboard bearing (A01c) Rotor (A01d) Stator (A01e) Frame (A01f) Mounting plate (A01g) Wiring terminals (A01h) Drum (A02) External power source (B) Cage (C) Frame (C01) Lifting Lug (C02) Cable (D01) Hook (D02) Pulleys (D03) Switch (E01) START (E01a) Cabling (D) Controls (E) FULL UP LIMIT (E01b) Wiring (E01c) (b) System breakdown and coding. Figure 312.
: Mountain Climbing Rig_____ PROB. INTERVAL: 30 years ___________ FAILURE MODES. disconnected.” and guide in component and manufacturer selection. Start switch fails open. Cut. Varmint invasion. (3) 328 . Example of an FMECA—Continued. incorporate “fail safe” features into the system design. No response a switch. Switch fails open.: Controls SYSTEM NO. Item/ Functional Ident. 3. An FMECA provides risk assessments of these failures. EFFECTS. (c) Worksheet. No.: ______________________ SUBSYSTEM NO.FMEA NO: ________________________ PROJECT NO. Results can be used to optimize reliability. optimize designs. Cage stays in safe position. Provide further analysis at the piecepart level for highrisk hazards identified in a PHA.5 (1) (2) Provides a mechanism to be exhaustively thorough in identifying potential singlepoint failures and their consequences. Start switch Full up switch Wiring Failure Mode Failure Cause Failure Event T a r g e t P E T P P E T Risk Assessment S e v IV IV IV II IV IV IV P r o b C C C A D D D RC i o s d k e 3 3 3 1 3 3 3 Action Required/ Comments E–01–a E–01–b E–02 Switch fails closed. Figure 312. obtain satisfactory operation using equipment of “low reliability. Mechanical failure or corrosion. Cage does not stop. Cage will not move.5 Advantages Performing FMEA’s and FMECA’s provide the following advantages: 3.4. Stop switch fails closed. AND CRITICALITY ANALYSIS WORKSHEET SHEET ___ OF ____ DATE _________________________ PREPARED BY: __________________ REVIEWED BY: __________________ APPROVED BY:__________________ TARGET CODE: P – PERSONNEL / E – EQUIPMENT / T – DOWNTIME / R – PRODUCTS / D – DATA / V – ENVIRONMENT Id. Mechanical failure or corrosion.
.. B. especially when performed at the partscount level within large. H. Effects.S. no check methodology exists to evaluate the degree of completeness of the analysis. “Procedures for Performing a Failure Modes. D. since every failure mode of each component of the system is assessed. 1989. 1991. DC. and Moriarty.4. F. W. Failure probability data are often difficult to obtain for an FMECA. This analysis is heavily dependent upon the ability and expertise of the analyst for finding all necessary modes.P. U.G.F. Provides a mechanism for more thorough analysis than an FTA. 3.H.: “Loss Prevention in the Process Industries.” NUREG–0492. OH. complex systems.3. Lees. multipleelement faults or failures within the system are not addressed or evaluated.” McGrawHill. D.” November 1980.6 3. Roland. MIL–STD–1629A.6 Limitations The following limitations are imposed when performing FMEA’s and FMECA’s. often too late to guide this prioritization. Chester. then focus on more severe system threats may be overlooked. Butterworths. 1980. Haasl. Inc.E.4. Raheja. Probabilities or the consequences of system failures induced by coexisting. Vesely.” 2 vols. D. Roberts. Human error and hostile environments frequently are overlooked.F. Washington. and Goldberg..: “System Safety Engineering and Management. (1) (2) (3) (4) (5) (6) (7) (8) Costly in manhour resources.: “Assurance Technologies – Principles and Practices.. If too much emphasis is placed on identifying and eliminating singlepoint failures. the level of design maturity required for an FMECA is not generally achieved until late in the design phase.7 Bibliography Layton..(4) (5) Identify hazards caused by failures to be added to the PHA that may have been previously overlooked in the PHA..E. Although systematic.” Weber Systems Inc. F.: “System Safety – Including DOD Standards. 329 . Government Printing Office. 1982. However. London. and guidelines/check sheets are available for assistance. ” John Wiley & Sons.: “Fault Tree Handbook. and Criticality Analysis. An FMECA can be a very thorough analysis suitable for prioritizing resources to higher risk areas if it can be performed early enough in the design phase. N.
Simple RBD’s are constructed of series. or combinations of series and parallel elements (table 34). Each RBD has an input and an output and flows left to right from the input to the output. The descriptions of RDB’s contained herein was obtained from references 3. Each block represents an event or system element function. subassembly. component. However. Type branch Series Block diagram representation System reliability # RS = RA * RB A B Parallel A RS = 1 – (1–RA)(1–RB) B Seriesparallel A C RS = (1 – (1–RA)(1–RB)) * (1 – (1–RC)(1–RD)) B Parallelseries A B D D RS = 1 – (1 – (RA * RB)) * (1 – (RC * RD)) C #Assumes all components function independently of each other. Simple RBD construction. or part. These blocks are connected in series if all elements must operate successfully for the system to operate successfully. Blocks may depict the events or system element functions within a system. parallel.1 Description A reliability block diagram (RBD) is a backwards (topdown) symbolic logic model generated in the success domain.5. These blocks are connected in parallel if only one element needs to operate successfully for the system to operate successfully.5 Reliability Block Diagram 3. 330 .3. Table 34 .7 and 3. these blocks typically depict system element functions only. A system element can be a subsystem. A diagram may contain a combination of series and parallel branches. The system operates if an uninterrupted path exists between the input and output.8.
. . H are not success paths. Such an RBD is presented in figure 313. then paths B. thus this is not a true series or parallel arrangement. 331 . G and B. . i n Parallel Systems: RS = 1– ∏ (1–Ri ) = [1–(1–R1 )*(1–R2 ) (1–R3 )* . These systems must be modeled with a complex RBD. Not all systems can be modeled with simple RBD’s. The relationships between element reliability and system reliability for series and parallel systems are presented below. . Some complex systems cannot be modeled with true series and parallel branches. (1–Rn )] i n where RS = system reliability. Series Systems: RS = ∏ Ri = R1* R2 * R3 * . . E.RBD’s illustrate system reliability. Ri = system element reliability. . if element E fails. Typical complex RBD. Rn. Each element of a block diagram is assumed to function (operate successfully) or fail independently of each other element. A D G B E C F H Figure 313. Notice in this example. E.8. and their derivations are found in reference 3. and n = number of system elements (which are assumed to function independently). Reliability is the probability of successful operation during a defined time interval.
RnH. Then treat each parallel branch as an element in a series branch and determine the system reliability by using the equations in step 3a. RSL = 1– ∏ (1–RpL ) = [1–(1–R1L )*(1–R2L ) (1–R3L )* . d. .3.5. b. .6).2 Application An RBD allows evaluation of various potential design configurations. . RSL = ∏ (RiL) = R1L *R2L *R3L * . these functions are performed during phase C. 332 . 3. . first determine the reliability for each series branch using the equations in step 3a. . (1–RnH)]. RiL (low) to RiH (high). Typically. .5. RSL (low) to RSH (high). For parallel systems with n elements that are to function independently. . For series systems with n elements that are to function independently. For seriesparallel systems. Construct a block diagram using the convention illustrated in table 34.3 Procedures The procedures (adapted from reference 3. . c. A functional diagram of the system is helpful. Then treat each series branch as an element in a parallel branch and determine the system reliability by using the equations in step 3b. RnL i n i n RSH = ∏ (RiH) = R1H *R2H *R3H * . For parallelseries systems.8 Required subsystem and element reliability levels to achieve the desired system reliability can be determined. from each individual element’s reliability band.8) to generate a simple RBD are presented below: (1) (2) (3) Divide a system into its elements. in the following manner: a. . Calculate system reliability band. 3. first determine the reliability for each parallel branch using the equations in step 3b.3. . . An RBD may also be used to identify elements and logic as a precursor to performing an FTA (sec. (1–RnL )] i n i n RSH = 1– ∏ (1–RpH) = [1–(1–R1H )*(1–R2H) (1–R3H )* . .
0.e. 0.80 0. The estimated reliability band for each individual component over the system’s estimated 10yr life interval is presented below: Subsystem 1 1 1 2 2 2 Component A B C D E F Reliability Bands Low High 0. treat these as branches within the remaining block diagram.0. and determine the reliability for the new simplest branches.97 0.0.960.4 Example A system has two subsystems designated 1 and 2. Example RBD. Continue this process until one of the above four basic arrangements remains. Then.96 0.72 0. determine the reliability for the simplest branches.980.84 0. 333 .99 0.60 0.72 A 0. For systems that are composed of the four above arrangements.97 0.99 E F D Figure 314.5.70 .98 0. 3.80 .99 0.99 An RBD for the system is presented in figure 314.84 B 0.60 . Subsystem 2 is designed to be a backup for subsystem 1.98 0.62 0. Subsystem 1 has three components and at least one of the three must function successfully for the subsystem to operate.70 0. then determine the system reliability. Also. Note that the components for subsystem 1 are in a parallel branch with the components of subsystem 2.980. note that the components for subsystem 1 form a series branch and the components for subsystem 2 form a parallel branch. Subsystem 2 has three components that all need to function successfully for the subsystem to operate.62 C 0.
If the element reliability values have different confidence bands.70)(1–0.7 Blocks representing elements in an RBD can be arranged in a manner that represent how these elements function in the system.98) = 0.9.976 R1H = 1 – (1–0.983 Subsystem 2: R2L = (0.983)(1–0.999.5 Advantages An RBD provides the following advantages: (1) (2) (3) (4) Allows early assessment of design concepts when design changes can be readily and economically incorporated.98)(0.998 to 0.8 System element reliability estimates might not be readily available for all elements.6 Limitations An RBD possesses the following limitations: (1) (2) Systems must be broken down into elements where reliability estimates can be obtained.73.72)(1–0. the reliability band for the system is 0.8 (3) 334 .84)(1–0.5.8 Tends to be easier for an analyst to visualize than other logic models. Not all systems can be modeled with combinations of series. 3.60) = 0. However. and not be accepted by others in the decision making process.998 RSH = 1 – (10.3. Such a breakdown for a large system can be a significant effort. parallel.3.5. These complex systems can be modeled with a complex RBD. this can lead to significant problems.3.C alculations for subsystem and system reliabilities are presented below: Subsystem 1: R1L = 1 – (1–0.976)(1–0. Some reliability estimates may be very subjective.951 System: RSL = 1 – (1–0.99) = 0.7 Since RBD’s are easy to visualize. or parallelseries branches. (low band value) (high band value) (low band value) (high band value) (low band value) (high band value) 3.62) = 0.951) = 0. determining system reliability for such a system is more difficult than for a simple RBD. they can be generated prior to performing an FTA and transformed into a fault tree by the method discussed in section 3.99)(0. seriesparallel.3. difficult to validate.80)(1–0.922) = 0. such as a fault tree.96)(0.97)(0.922 R2H = (0.999 Therefore.3.
Since reliability for a given event is defined as the number of successes per number of attempts.6.5. 1986. therefore R + PF = S/(S+F) + F/(S+F) = 1 and PF = 1–R. if none of them occurs. 3. A cut set is any group of initiators that will. cause the TOP event to occur. undesirable condition or event. The probability of failure for a given event is defined as the number of failures per number of attempts. as described in reference 3. 3. if they all occur.2 Application FTA’s are particularly useful for high energy systems (i.e. This can be expressed as: PF = F/(S+F) . to ensure that an ensemble of countermeasures adequately suppresses the probability of mishaps. Previous identification of the undesirable event also includes a recognition of its severity. 335 . M. An FTA is a powerful diagnostic tool for analysis of complex systems and is used as an aid for design improvement. potentially high severity events). This model traces the failure pathways from a predetermined.3. will guarantee the TOP event cannot occur. A path set is a group of fault tree initiators that. of a system to the failures or faults (fault tree initiators) that could act as causal agents. A minimal cut is a least group of initiators that will. called the TOP event.” SpringerVerb.9. cause the TOP event to occur. entering failure probabilities for each fault tree initiator. An FTA can be carried out either quantitatively or subjectively. The FTA includes generating a fault tree (symbolic logic model). is a topdown symbolic logic model generated in the failure domain. and determining cut sets and path sets. A.1 Description A fault tree analysis (FTA).6.: “System Preliminary Evaluation & Prediction in Engineering. where F = number of failures and S = number of successes. then the relationship between the probability of failure and reliability can be expressed as follows: R = S/(S+F). and Godran. propagating failure probabilities to determining the TOP event failure probability.6 Fault Tree Analysis 3..7 Bibliography Pages. if they all occur.
Sensitivity studies can be performed allowing assessment of the sensitivity of the TOP event to basic initiator probabilities. FTA’s can be used to identify cut sets and initiators with relatively high failure probabilities.9. deployment of resources to mitigate risk of highrisk TOP events can be optimized. FTA procedures. 3. A minimal cut set is a least group of initiators that. FTA’s are applicable both to hardware and nonhardware systems and allow probabilistic assessment of system risk as well as prioritization of the effort based upon root cause evaluation. for performing an FTA are presented below.4 Identifying path sets 336 . Therefore. These procedures are divided into the four phases: (1) fault tree generation. Table 35. 3. if they all occur. Probabilities are identified for each initiator and propagated to intermediate events and the TOP event.6. intermediate events.6. The analyst does not have to perform all four phases. A cut set is any group of initiators that will. A symbolic logic model illustrating fault propagation to the TOP event is produced. and assess quantitative common cause probability. if they all occur. (3) identifying and assessing cut sets. and (4) identifying path sets. and the TOP event are identified.3.6. Section 3. FTA’s are typically performed in phase C but may also be performed in phase D. and item significance of the tree. identify qualitative common cause vulnerability.3.3 Probability determination Identifying and assessing cut sets 3.3 Procedures The procedures. cause the TOP event to occur.6.3. All cut sets and minimal cuts sets are determined. Action items resulting from the investigation may be numerically coded to the fault tree elements they address. Analysis of a cut set can help evaluate the probability of the TOP event.2 3.1 Procedures Fault tree generation Benefits All basic events (initiators). A path set is a group of fault tree initiators that. as described in reference 3. The subjective nature of risk assessment is relegated to the lowest level (root causes of effects) in this study rather than at the top level. if none of them occurs. quantitative. The benefits for each of the four phases are summarized in table 35.3. but can progress through the phases until the specific analysis objectives are met.6. will guarantee the TOP event cannot occur. will cause the TOP event to occur.This type of analysis is sometimes useful in mishap investigations to determine cause or to rank potential causes. (2) probability determination. and resources prioritized by the perceived highest probability elements. All path sets are determined. Cut sets also enable analyzing structural.
the use of this exact solution is seldom warranted. The Delphi technique (sec. to construct a fault tree are illustrated in figure 315. A common cause is a condition. including the gates infrequently used.3.2 through 3. In many cases it is best to stay with comparative probabilities rather than the “absolute” values. step 6.3 are included for completeness and to provide insight as to the mathematics that takes place in the commercially available fault tree programs.3.6.2 Probability Determination If a fault tree is to be used as a quantitative tool.9) derives estimates from the consensus of experts. All large trees are typically analyzed using these programs. Although many event and gate symbols exist. are presented in table 37. A frequent error in fault tree construction is neglecting to identify common causes. However. historical evidence (of the same or similar systems).3. are required to determine the significance of any quantitatively driven conclusion. event. The procedures. 3. Failure probabilities can also be determined from a PDA as discussed in section 3. A source for human error probabilities is found in reference 3. Propagation of confidence and error bands is performed simply by propagation of minimum and maximum values within the tree. Sections 3. The probability of failure of independent inputs through an AND gate is the intersection of their respective individual probabilities.4. Normalizing data to a standard. step 8.3.6. Probabilities must be used with caution to avoid the loss of credibility of the analysis. Once probabilities are estimated for all basic events or initiators. (2) inclusive OR gate. Delphi estimates. Also. Propagation of failure probabilities for two independent inputs through an AND and OR (inclusive) is conceptually illustrated in figure 318. they are propagated through logic gates to the intermediate events and finally the TOP event. The propagation equations for the logic gates.6. This technique is described in reference 3. 7.11 and is illustrated in figure 316. most fault trees can be constructed with the following four symbols: (1) TOP or Intermediate event. or phenomenon that will simultaneously induce two or more elements of the fault tree to occur. the propagation solution through an OR gate is simplified by the rare event approximation assumption.14. MILstandards. and (4) basic event. explicitly declared meaningless value is a useful technique here. for small trees hand analysis may be practical. Sources for these failure probabilities may be found from manufacturer’s data. industry consensus standards. 337 . The probability of failure of independent events through an OR (inclusive) gate is the union of their respective individual probabilities.6. The exact solution for OR gate propagation is presented in figure 319.1 Fault Tree Generation Fault trees are constructed with various event and gate logic symbols.9. defined in table 36. A method for detecting common causes is described in section 3.3. as described in reference 3. confidence or error bands. As shown in figures 317. (3) AND gate. The log average method is useful when the failure probability cannot be estimated but credible upper and lower boundaries can be estimated. on each cited probability number. the probability of failure must be determined for each basic event or initiator.3.3. and the log average method.6.10. simulation or testing. The relationship between reliability and failure probability propagation of two and three inputs through OR (inclusive) and AND gates is illustrated in figure 317.
or restrictions to other events. Fault tree construction symbols. Intermediate Event – This event describes a system condition produced by preceding events. They are also called leaves or initiators. Symbol Name Event (TOP or intermediate)* Inclusive OR gate* Description TOP Event – This is the conceivable. Any single input is necessary and sufficient to cause the output event to occur. 338 . However. An output occurs if all inputs exist. Any single input is necessary and sufficient to cause the output event to single occur. INHIBIT gate An output occurs if a single input event occurs in presence of an enabling condition. resources. but only one input exists. Basic event* External event An initiating fault or failure that is not developed further.Table 36. Undeveloped event An event not further developed due to a lack of need. An event that under normal conditions is expected to occur. all other inputs are then precluded. Exclusive OR gate An output occurs if one. Conditioning Event These symbols are used to affix conditions. All inputs are necessary and sufficient to cause the output event to occur. restraints. Mutually exclusive OR gate M AND gate* An output occurs if one or more inputs exist. These events determine the resolution limit of the analysis. All inputs are necessary and sufficient to cause the output event to occur. Priority AND gate An output occurs if all inputs exist and occur in a predetermined sequence. * Most fault trees can be constructed with these four logic symbols. or information. An output occurs if one or more inputs exist. undesired event to which failure paths of lower level events lead. Any single input is necessary and sufficient to cause the output event to occur.
Basic Event…(“Leaf. • Estimate upper and lower credible bounds of probability for the phenomenon in question. Identify firstlevel contributors. = 0. Fault tree construction process.1.01 + 0.03 0.e. 339 .01 0.0316228 –1. 5. Link secondlevel contributors to TOP by logic gates.1 2 i.055 Figure 316.1 0. • Average the logarithms of the upper and lower bounds. Identify undesirable TOP event. 3. it is geometrically midway between the limits of estimation. Identify secondlevel contributors.5 times the lower bound and 0.” “Initiator.” or “Basic”) indicates limit of analytical resolution.07 0. Log average method of probability estimation.04 0. Link contributors to TOP by logic gates. Thus. for the example shown. 0.05 0. the arithmetic average would be… 0. 2..02 0.0316 + PL Lower Log PL+ Log P U = Antilog(–2) + (–1) Probability Log Average = Antilog 2 2 Bound –2 10 = 10 = 0. 4. 6.55 times the upper bound. 5. • The antilogarithm of the average of the logarithms of the upper and lower bounds is less than the upper bound and greater than the lower bound by the same factor.5 PU Upper Probability Bound 10–1 Note that. Repeat / continue… Figure 315.
R + PF ≡ 1 R T = RA RB R T = R A + R B – RAR B PF = 1 – R T P F = 1 – (R ARB) P F = 1 – [(1 – P A )(1 – P B)] PF = 1 – R T P F = 1 – (R A + R B – R AR B ) P F = 1 – [(1 – P A) + (1 – P B) – (1 – P [Union / ] A )(1 – P B)] ] PF = PA + PB – P A PB …for P A. Relationship between reliability and failure probability propagation. independent.For 2 Inputs OR Gate AND Gate Either of two.2 PF ≅ PA + PB with error ≤ 11% PF = P APB [Intersection / “Rare Event Approximation” PF = P A + P B + PC – P APB – PA PC – P BPC + P A PBP C For 3 Inputs Omit for approximation PF = P APB PC Figure 317. 340 . system failure. independent element failures produces elements must fail to produce system failure. AND Gate… TOP PT = Pe PT = P 1P 2 Intersection / OR Gate… TOP PT Union / Pe PT P1 + P 2 1 P1 2 P2 1&2 are INDEPENDENT events 1 P1 2 P2 P T = P 1 P2 P T = P 1 + P 2 – P 1 P2 Usually negligible… Figure 318. Both of two.B ≤ 0. Failure probability propagation through OR and AND gates.
3. Exact solution of OR gate failure probability propagation.3 Identifying and Assessing Cut Sets A cut set is any group of initiators that will produce the TOP event. Its use is rarely justifiable. if all the initiators in the group occur.9 and are based on the MOCUS computer algorithm attributed to J. These procedures for determining cut sets are described in reference 3. A minimal cut set is the smallest number (in terms of elements. Proceed through the matrix construction by (1) substituting the letters for each AND gate with letters for the gates and numbers of the initiators that input into that gate (arrange these letters and numbers horizontally in the matrix rows). starting from the top of the tree. Cut sets also enable analyzing structural. if all the initiators in the group occur. It P P T for propagating probabilities through the OR gate. The letter for the gate directly beneath the TOP event will be the first entry in the matrix.Failure TOP PT = ? Success TOP PT = Failure TOP PT = Pe (1 – P e ) 1 P1 2 P2 3 P3 1 2 3 1 P1 2 P2 3 P3 P1 = (1 – P 1 ) P3 = (1 – P 3 ) P2 = (1 – P 2 ) = P e = 1 – P (1 – P e ) The ip operator ( cofunction of pi ( provides an exact solution ) is the ∏ ). and assess common cause probability. T = 1 – [(1 – P 1) (1 – P 2 ) (1 – P 3 ) … (1 – Pn )] Figure 319. From the top of the tree downwards. and item significance of the tree. One method of determining and analyzing cut sets is presented below.B. 341 .3. and (2) substituting the letters for each OR gate with letters for the gates and numbers of the initiators that input into that gate (arrange these letters and numbers vertically in the matrix columns). Assign a unique letter to each gate and a unique number to each initiator. not probability) of initiators that will produce the TOP event. Analysis of a cut set can help evaluate the probability of the TOP event. create a matrix using the letters and numbers. Determining Cut Sets: (1) (2) (3) Consider only the basic events or initiators (discarding intermediate events and the TOP event). quantitative.6. Fussell. identify common cause vulnerability.
through visual inspection.. The remaining rows define the minimal cut sets of the fault tree. Visually inspect the final matrix and eliminate any row that contains all elements of a lesser row. Assessing Cut Sets: (6) Since a cut set is any group of initiators that will produce the TOP event. Probability propagation expressions for logic gates. expressed as: PK = P1 * P2 * P3* P4.. (4) (5) When all the gate’s letters have been replaced. 342 . eliminate redundant elements within rows and rows that repeat other rows. a final matrix is produced with only numbers of initiators.* Pn. Symbol Name Inclusive OR gate‡ P1 P2 Venn Diagram Propagation Expressions PT = P1 + P2 – (P1 * P2) PT = P1 + P2# Exclusive OR gate P1 P2 PT = P1 + P2 –2 (P1 * P2) PT = P1 + P2# M Mutually exclusive OR gate P1 P2 PT = P1 + P2 AND gate ‡ and (priority AND gate) P1 P2 PT = P1 * P2 ‡Most fault trees can be constructed with these two logic gates. PK (the probability that the cut set will induce the TOP event) is mathematically the same as the propagation through an AND gate. #Simplified expression for rare event approximation assumption. Next. if all the initiators in the group occur. the cut set probability. Each row of this matrix represents a Booleanindicated cut set.Table 37.
The procedures to determine path sets are as follows: (1) Exchange all AND gates for OR gates and all OR gates for AND gates on the fault tree. ensures the TOP event cannot occur. h for human operator. determine the numerical probability that initiator e contributed to the TOP event. A cut set with few elements indicates high vulnerability. 3. determine the numerical probability that this cut set induced the TOP event. Ie. c. then the TOP event is vulnerable to the common cause the subscript represents. etc. 343 .9). since the same event (the socalled common cause) could act to precipitate each event. assuming it has occurred. IK = PK/PT where PK = the probability that the cut set will occur (see step 6 above). Path sets can be used to transform a fault tree into a reliability diagram (sec. Note that some initiators may have more than one subscript. while others will have none. of each cut set. A cut set with a single initiator. I K.). and IKe = importance of the minimal cut sets containing initiator e. Numerous cut sets indicates high vulnerability. (8) (9) (10) Assess the quantitative Importance. of each individual initiator. This indicates that the probability number. may be significantly in error. Assess the structural significance of the cut sets to provide qualitative ranking of contributions to system failure. i. If that is the case.e. v for vibration. K. b. Analyze the probability of each common cause at its individual probability level of both occurring. indicates a potential singlepoint failure. and PT = the probability of the TOP event occurring.(7) Determine common cause vulnerability by uniquely assigning letter subscripts for common causes to each numbered initiator (such as m for moisture. q for heat. d. Check to see if any minimal cut sets have elements with identical subscripts. 3.3. Assuming all other things are equal then: a. That is. if none of them occurs. called a singleton.. they no longer represent statistically independent events. e. A cut set with many elements indicates low vulnerability. calculated as above. That is.6. if it has occurred: Ie = where ∑ IK e e Ne Ne = number of minimal cut sets containing initiator e. and inducing all terms within the affected cut set. (11) Assess the quantitative importance.4 Identifying Path Sets A path set is a group of fault tree initiators that.
for a thorough analysis. ARTIFICIAL WAKEUP FAILS 3.6.3.82x102 BACKUP (WINDUP) CLOCK FAILS 1.34x10 4 Approx. both low and high probability values that define a probability band for each initiator could be propagated through the fault tree to determine a probability band for the TOP event.6. 344 .1/yr ALARM CLOCKS FAIL 3. Example fault tree.1 Examples Fault Tree Construction and Probability Propagation An example of a fault tree with probabilities propagated to the TOP event is presented in figure 320.(2) Construct a matrix in the same manner as for cut sets (sec. In this example the TOP event is the “artificial wakeup fails. 3. steps 1–5). PLUGIN CLOCK FAILS 1. In this example for brevity.6.4 3. 3. However.” The system being examined consists of alarm clocks used to awaken someone.3. only a nominal probability value for each fault tree initiator is propagated through the fault tree to the TOP event.83x10 2 POWER OUTAGE FAULTY INNARDS 3x10 4 3/1 MECHANICAL FAULT 8x10 8 FORGET TO SET FAULTY MECHANISM FORGET TO SET FORGET TO WIND 1x102 8x10 3 2/1 4x104 1/10 8x103 2/1 1x102 3/1 ELECTRICAL FAULT 3x104 1/15 HOUR HAND FALLS OFF HOUR HAND JAMS WORKS KEY: Faults/Operation……8x10 3 Rate. 0.4. Faults/Year………2/1 Assume 260 Operations/Year 4x10 4 1/10 2x104 1/20 Figure 320.34x10 4 Negligible NOCTURNAL DEAFNESS MAIN. Each row of the final matrix defines a path set of the original fault tree.
is an OR gate. replace it horizontally. starting with the TOP “A” gate… A B D 1 D C D B is an OR gate. TOP PROCEDURE: • Assign letters to gates. • Assign numbers to basic initiators. 2 & 3. A B D 1 C 2 4 2 3 • Construct a matrix. 345 . B & D . replace it vertically. the initial matrix entry.2 Cut Sets An example of how to determine Booleanindicated minimal cut sets for a fault tree is presented in figure 321. These BooleanIndicated 1 2 Cut Sets… 2 2 3 1 4 …reduce to 2 4 3 these Minimal Cut Sets. D (2nd row).4. 1 2 2 3 1 4 Minimal Cut Set rows are least groups of initiators which will induce TOP. 1 D 2 D 3 C is an AND gate. its inputs. (TOP gate is “A. its inputs.”) Do not repeat letters. Each requires a new row. Replace as before. replace it vertically. 1 & C. Each requires a new row. represent it by the same number at each appearance. its inputs. 1 2 2 D 3 1 4 D (top row).3. If a basic appears more than once. A is an AND gate. 2 & 4. TOP event gate is A. Figure 321. replace it horizontally. Example of determining cut sets.6. its inputs. is an OR gate.
This tool can be used to reconfigure a system to reduce vulnerability.3.6. TOP A B C 1 D F 6 This Fault Tree has… 2 E 3 5 G Path Sets are least groups of initiators which. 6 1 1 1 1 2 3 4 5 6 3 4 “Barring” terms (n) denotes consideration of their success properties.5 Advantages An FTA provides the following advantages:3.9 (1) (2) (3) (4) (5) Enables assessment of probabilities of combined faults/failures within a complex system. guarantee against TOP occurring.4. thereby guiding deployment of resources for improved control of risk. 1 1 1 3 2 3 4 4 5 3 4 4 1 …these Minimal Cut Sets …and these Path Sets.6. 346 . if they cannot occur. 3. Figure 322. Path sets can be used in trade studies to compare reduced failure probabilities with increases in cost to implement countermeasures. Singlepoint and common cause failures can be identified and assessed.3 Path Sets An example of how to determine path sets for a fault tree is presented in figure 322. Example of determining path sets. System vulnerability and lowpayoff countermeasures are identified.
” Garland Press. The generation of an accurate probabilistic assessment may require significant time and resources. 347 . A fault tree is flawed if common causes have not been identified.J..S. The failure rate of each initiator must be constant and predictable. 1982. P.” Society for Industrial and Applied Mathematics. and Burdick. Roland.6.. Specific (noncomparative) estimates of failure probabilities are typically difficult to find. and to successfully use to drive conclusions. Roberts. G. J.” Prentice Hall.” Department of Energy No.: “System Safety Engineering and Management. B. limit the size of the tree. Events or conditions at any level of the tree must be independent and immediate contributors to the next level event or condition.: “Reliability and Faulttree Analysis Guide. H. S.e. (4) (5) (6) (7) (8) 3..: “Nuclear Systems Reliability Engineering and Risk Assessment. Comparative analyses are typically as valuable with better receptions from the program and design teams.A.6.B. Henley. 1991.. and Goldberg. Vesely. Inc. D.. and Singh.. E.: “Fault Tree Handbook.” John Wiley & Sons. DOE 7645/22.” NUREG–0492..: “Probabilistic Risk Assessment.E. Haasl. Events or conditions under the same logic gate must be independent of each other. 1983.E. i.W. Malsaky. H. W: “Handbook of System and Product Safety. N. C. B. 1972. 1981. 1982. Dillon. Washington.3.: “Engineering Reliability – New Techniques and Applications. Hammer. Caution must be taken not to “over work” determining probabilities or evaluating the system.R. Fault trees used for probabilistic assessment of large systems may not fit or run on conventional PCbased software.6 Limitations A FTA possesses the following limitations: (1) (2) (3) Address only one undesirable condition or event that must be foreseen by the analyst..F. U.H.” The Institute of Electrical and Electronic Engineers. 1977..S. Thus.F. F.” John Wiley & Sons. to achieve agreement on. A fault tree is not accurate unless all significant contributors of faults or failures are anticipated.7 Bibliography Crosetti. Fussell. W. New York.: “System Safety: Technology and Application. DC. and Kumamoto. several or many FTA’s may be needed for a particular system. Government Printing Office. and Moriarty.
propagating each initiator probability to determining the TOP event probability and determining cut sets and path sets. 348 . if they all occur.” Proceedings of the Second International System Safety Conference. prevent the TOP event from occurring. and Koren. L. to ensure that an ensemble of countermeasures adequately leads to a successful top event. R. 3. J. if they all occur. Wynholds. will guarantee the TOP event occurs.. This technique also allows probabilistic assessment of causal benefits as well as prioritization of effort based upon root cause evaluation.7. potentially high severity events). New Mexico. Riley. Los Altos. J. A minimal cut set is a least group of initiators that will.. 1975. This technique is a powerful diagnostic tool for analysis of complex systems and is used as an aid for design improvement and is applicable both to hardware and nonhardware systems.: “A New Approach to the Analysis of Reliability Block Diagrams. and Bass.e.M. The STA includes generating a success tree (symbolic logic model).G. SAIC.2 Application Particularly useful for high energy systems (i. desirable condition or event (TOP event) of a system to the successes (success tree initiators) that could act as causal agents.. A path set is a group of success tree initiators that. This can be expressed as: Ps = S/(S+F) . Since reliability for a given event is also defined as the number of successes per number of attempts.William S. An STA is the compliment of an FTA (sec.7 Success Tree Analysis 3. Potterfield.7. if all of them occur.: “Fault Tree Graphics – Application to System Safety.. Sensitivity studies can be performed allowing assessment of the sensitivity of study results to subjective numbers.. then R = PS. The subjective nature of the probability assessment is relegated to the lowest level (root causes of effects) in this study rather than at top level.1 Description A success tree analysis (STA) is a backwards (topdown) symbolic logic model generated in the success domain.” Proceedings from Annual Reliability and Maintainability Symposium. In the success domain. determining success probabilities for each tree initiator. This model traces the success pathways from a predetermined. 3. W. a cut set is any group of initiators that will. The probability of success for a given event is defined as the number of successes per number of attempts. 3. prevent the TOP event from occurring.6) which is generated in the failure domain with failure pathways from undesirable events. where S = number of successes and F = number of failures. 1990.
as are the cautions for use of probability values. Repeat / continue… 6.9. Identify firstlevel contributors. A success tree can be used to verify the logic of a fault tree. Transform a success tree from a fault tree by simply changing all AND gates to OR gates and OR gates to AND gates. 7. The commercial computer programs are similar. 349 . most success trees can be constructed with the following four symbols: (1) TOP or intermediate event. 3. Success tree construction process. Determine the probability of success (Ps) for each basic event or initiator. 3. as described in reference 3. the logic of the success tree needs to be valid if the logic of a fault tree is to be valid. (3) AND gate. Figure 323.” “Initiator. Remember that the probability of success equals reliability (R) and may be determined from (PF) as shown in the following equation: PS = 1 – PF . intermediate event. Basic Event…(“Leaf. 2. Since a success tree is the logic compliment of a fault tree. and the log average method. (2) inclusive OR gate. like fault trees.7. These symbols are defined in table 36. Identify secondlevel contributors. MIL standards. Link secondlevel contributors to TOP by logic gates. Sources for these success probabilities may be found from manufacturer’s data. Link contributors to TOP by logic gates. to construct a fault tree also apply to success tree generation and are illustrated in figure 323. and restating each initiator.3 Procedures Success trees. Once probabilities are estimated for all basic events or initiators.The STA is typically applied in phase C but may also be applied in phase D. Delphi estimates. propagate these probabilities through logic gates to the intermediate events and finally the TOP event. Identify desirable TOP event. The Delphi technique (sec. and top event as a success opposed to a failure. are constructed with various event and gate logic symbols. A success tree can be constructed from a fault tree. Although many event and gate symbols exist. 5. if a success tree is generated from a fault tree. 4.” or “Basic”) indicates limit of analytical resolution.9) derives estimates from the consensus of experts. and (4) basic event. The procedures. industry consensus standards. historical evidence (of similar systems). Use the expressions presented in table 37 to propagate probabilities through logic gates. 1. simulation or testing.
Example The compliment success tree for the fault tree presented in section 3.7.3.9885 0. 350 .9997 0.9996 0. as presented in sections 3.9996 Alarm Clock Works Possess Keen Hearing Properly 1.000 0.9996 Figure 324.9923 0.9997 Hour Hand Does Not Jam Works Hour Hand Stays On 0. Example success tree. respectively.4 24.3.3 and 3.9806 0.000 0.9885 Mechanical Component Success Electrical Components Perform Properly 1.7.9923 0.Generate cut sets and path sets in the same manner as for fault trees.4. Plugin Clock Works Backup Clock (Windup) Performs Properly 0.4 is presented in figure 3 Artificial Wakeup Succeeds 0.9998 0.9996 Main. 3.6.9805 Functioning Clock Components Uninterrupted Power Remembered to Set Alarm Unflawed Mechanism Remembered to Set Backup Alarm Remembered to Wind Clock 0.7.
New York. Thus. and Kumamoto. Compliments the FTA by providing a method to verify the logic of the fault tree. Inc. H.J.. 3. Events or conditions at any level of the tree must be independent and immediate contributors to the next level event or condition. or a normal system operating command.7..7 Bibliography Henley. Caution must be taken not to overdo the number generation portion.8.7. an undesirable event.1 Description An event tree analysis (ETA). Success trees used for probabilistic assessment of large systems may not fit/run on conventional PCbased software.7.” The Institute of Electrical and Electronic Engineers. is a forward (bottomup) symbolic logic modeling technique generated in both the success and failure domain. 1991. The generation of an accurate probabilistic assessment may require significant time and resources. The system challenge may be a failure or fault. 3.9 (1) (2) Assesses probability of favorable outcome of system operation.: “Probabilistic Risk Assessment. A success tree is not accurate unless all significant contributors to system successes are anticipated. 351 .9 (1) (2) (3) (4) (5) (6) (7) Address only one desirable condition or event that must be foreseen by the analyst. This technique explores system responses to an initiating “challenge” and enables assessment of the probability of an unfavorable or favorable outcome.3.8 Event Tree Analysis 3. E.6 Limitations An STA possesses the following limitations:3. 3. The probability of success (reliability) of each initiator must be constant and predictable. as described in references 3. several or many STA’s may be needed for a particular system.5 Advantages An STA provides the following advantages:3. Events or conditions under the same logic gate must be independent of each other.6 and 3.12.
Event tree (generic case). Trace each path to eventual success or failure. The ETA is also useful in evaluating effect and benefit of subtiered or redundant design countermeasures for design trades and assessment. Also. emergency response systems. An ETA may also be performed to compliment an FMEA. success or failure probabilities used must be used with caution to avoid the loss of credibility of the analysis. However.2 Application The ETA is particularly useful in analyzing commandstart or commandstop protective devices. and other nonhardware systems. Normalizing data to a standard. The technique is useful in evaluating operating procedures. are required to determine the significance of any quantitatively driven conclusion. Portray all credible system operating permutations. N D/A SUCCESS n FAILURE SUCCESS FAILURE O/O OPERATION/ OUTCOME OPERATION/ OUTCOME 1 INITIATION 3 DECISION/ ACTION C DECISION/ ACTION A DECISION/ ACTION B SUCCESS FAILURE SUCCESS FAILURE OPERATION/ OUTCOME 2 Figure 325. A Bernoulli model event tree is illustrated in figure 326. A decision tree is a specialized event tree with unity probability for the system outcome. A Bernoulli model event tree uses binary branching to illustrate that the system either succeeds or fails at each system logic branching node. management decision options. on each cited probability number. 3.A generic event tree portrays all plausible system operating alternate paths from the initiating event. . This technique is typically performed in phase C or E but may also be performed in phase D. 352 . confidence or error bands. In many cases it is best to stay with comparative probabilities rather than the “absolute” values. and engineered safety features. A generic event tree is illustrated in figure 325. An ETA may be used in conjunction with an FTA to provide a technique sensitivity assessment.8. explicitly declared meaningless value is a useful technique here.
(1) (2) Identify the initiating challenge to the system being examined. Event tree (Bernoulli model). Simplify the tree by pruning unnecessary alternate branches of nonrecoverable failures or undefeatable successes. for performing an ETA are presented below. trace all plausible system operating permutations to a success or failure termination. Determine the probability of the system failure by adding the probabilities for all paths terminating in failure. Use binary branching. SUCCESS FAILURE FAILURE SUCCESS FAILURE SUCCESS INITIATION FAILURE SUCCESS A fault tree or other analysis may be necessary to determine probability of the initiating event or condition. Lead unrecoverable failures and undefeatable successes directly to final outcomes. 3. assume the probability of the initiating event is one. b.3 Procedures The procedures. 353 . For a decision tree. as described in reference 3. (3) (4) (5) (6) Determine the probability of the initiating event by applying a fault tree (sec. Determine the probability of each potential path by multiplying the individual probabilities of events making up the path. use binary branching to show the system pathways. a. For the general event tree. (Unity probability may be assumed.Reduce tree to simplified representation of system behavior. For the Bernoulli model event tree.8. Determine the paths (alternate logic sequences) by answering the question.) SUCCESS FAILURE SUCCESS FAILURE FAILURE Figure 326. trace successful paths upwards and failure paths downwards.6) or other analysis. Determine the probability of the system success by adding the probabilities for all paths terminating in success. “What happens when the system is challenged by the initiation event?” By convention. 3.12.
Operating pathways must be foreseen by the analyst.5 Advantages An ETA provides the following advantages: (1) (2) (3) (4) Enables the assessment of multiple. 354 .6 Limitations An ETA possesses the following limitations: (1) (2) (3) (4) (5) Address only one initiating challenge.8.8. but must be foreseen by the analyst.4 Example An example of an ETA is presented in figure 327. Note that in this example the probability of the challenging initiator is assumed to be one and the tree has been pruned to its simplest form by using engineering logic. The example includes the system and scenario being assessed and the resulting event tree. multiple ETA’s may be needed for a particular system. noncomparative success or failure probability estimates are typically difficult to find. its path leads directly to a final failure outcome with no alternate paths. Potential singlepoint failures. This can be a “quick and dirty” comparative technique and provides very clear visibility of ineffective countermeasures. 3. the levels of loss associated with particular pathways may not be distinguishable without additional analyses. difficult to achieve agreement on. The initiating challenge is not disclosed by the analysis.8. Thus. thereby guiding deployment of resources for improved control of risk and optimized utilization of limited resources. areas of system vulnerability. (5) 3. End events need not be anticipated. In a similar manner since successful operation of the pump is an undefeatable success. Although multiple pathways to system failure may be disclosed. Specific. with better reception from the program and design teams. For example. and to successfully use to drive conclusions. its path also leads to a final success outcome with no alternate paths. and lowpayoff countermeasures are identified and assessed. Comparative analyses are typically as valuable. coexisting system faults and failures.3. since failure of the float switch is a nonrecoverable failure. Failure propagation paths of a system can be identified and traced. Functions simultaneously in the failure of success domain.
0) Pump Fails (P P) [PP – P P PS ] PKP P + PKP PP S] [PB PP – P BP PP S – Klaxon Fails (PK ) PB PK PP + PB PK PP PS ] Float Switch Fails (P S) [PKP P – P K PP PS ] [PS] PSUCCESS = 1 – P S – P K PP + P K PP PS – P BP P + P B PP PS + P B PK PP – P BP KP PP S PFAILURE = P S + P KP P – P K PP PS + P BP P – P BP PP S – P B PK PP + P BP KP PP S P SUCCESS + P FAILURE = 1 Figure 327. A klaxon K is also sounded. • Consider operator error as included within the bailing function. FAILURE Bailing Fails (PB) SUCCESS 355 . SIMPLIFYING ASSUMPTIONS: • Power is available full time. and B. P. and analyze responses available to the dewatering system… • Develop an event tree representing system responses. Example ETA. K. • Treat only the four system components S. Rising flood waters close float switch S . B . • Develop a reliability block diagram for the system. B.PUMP KLAXON P S K B BACKGROUND/PROBLEM — A subgrade compartment containing important control equipment is protected against flooding by the system shown. powering pump P from an uninterruptible power supply. Pump Succeeds (1 – P P) [1 – P S – P P + P PP S] Float Switch Succeeds (1 – P S) EVENT TREE… Klaxon Succeeds (1 – PK) [PP – P P PS – Bailing Succeeds (1 – P B ) [PP – P PP S – P K PP + P K PP PS – PB PP + P B PP PS + P BP KP P – PB PK PP PS ] Water Rises (1. alerting operators to perform manual bailing. Assume flooding has commenced. should the pump fail. Either pumping or bailing will dewater the compartment effectively. • Develop a fault tree for the TOP event Failure to Dewater.
“Reliability Engineering and Risk Assessment. 3. RBD’s offer the analyst a simplistic method to represent system logic. and event trees (sec. reliability diagrams are generated in the success domain.5)..1 Fault Tree to RBD Transformation An RBD represents system component functions that. “Guidelines for Hazard Evaluation Procedures. London. Reliability Block Diagram.9. 3. Fault trees are generated in the failure domain.P. 3. described in reference 3. RBD’s (sec. This technique is typically performed in phase C but may also be performed in phase B.” 2 vols. 3. 3. Butterworths.3. F. cut sets and path sets are obtainable using the MOCUS algorithm described in sections 3.6).2 RBD and Fault TreetoEvent Tree Transformation An event tree represents path sets in the success branches of the tree and all the cut sets in the failure branches of the tree.2 Application These techniques are applicable by the analyst that wishes to exploit the benefits of the fault tree. H.4. 356 .8. if the path sets and cut sets of a system are known for a certain challenge to a system (TOP event of a fault tree).. and Kumamoto.1 Description Fault trees (sec.” 1985.J. then an event tree can be constructed. RBD. For large complex fault trees.” New York.3.6.. and Event Tree Transformations 3.13 and presented below. Lees.9.3 Procedures The procedures for transforming a fault tree. or event tree to either of the other two logic models are presented in the following sections.13 3. produces success in place of a TOP fault event.3. Cut sets and path sets may be obtained from a reliability diagram as shown in figure 329. transform any one of the above models into the other two by translating equivalent logic from the success to failure or failure to success domain.9. 3. if these functions prevail. 1981.3. E. 1980. A fault tree can be transformed into a reliability diagram as illustrated in figure 328. and event tree. respectively. 3.7) are all symbolic logic models. 3.9. “Loss Prevention in the Process Industries. Event trees allow the analyst to assess a system in both the success and failure domains. Henley. Fault trees offer the analyst comprehensive qualitative or quantitative analysis.3.7 Bibliography Battelle Columbus Division. Therefore.9 Fault Tree. RBD.9.3 and 3.6. These techniques. and event trees are generated in the success and failure domains.
Deriving cut and path sets from an RBD.! The OR gate in this position indicates a series string of component functions. 4 1 Minimal Cut Sets 1 2 3 4/5/6 2 3 5 6 ! Path Sets 1/2/3/4 1/2/3/5 1/2/3/6 Figure 329. This AND gate indicates a paralleled set of component functions in the series string. Fault tree to RBD transformation. 357 . 1 2 3 4&5&6 Nastiness 4 5 6 1 2 3 4 5 6 ! Figure 328.
proceed as shown in figure 330. Also.9. To transform an event tree into an RBD. 3. All three of the models represent equivalent logic of the system. first transform the fault tree into an RBD (sec. reverse the process illustrated in figure 330.9. produce TOP event fault in place of the success to which the reliability block path lead. an event tree can be transformed into a fault tree by inspection as shown in figure 332.1). 358 . Failure Figure 330.3.4 Event Tree to RBD and Fault Tree Transformation An event tree represents path sets in the success branches of the tree and all the cut sets in the failure branches of the tree. Once the RBD is formed. To transform a fault tree into an event tree.3. 2 6 1 Success 4 3 5 ! All of these parallel elements must fail to produce system failure. respectively. 3.9. The series nodes of an RBD denote an OR gate beneath the TOP event of a fault tree. 4 1 2 3 5 6 Failure of any one of these series elements makes system failure irreversible. and presented in figure 333(a) and (b). 3.To transform an RBD into an event tree.9. 3.3.3 RBD to Fault Tree Transformation A fault tree represents system functions which. RBD to event tree transformation. if they fail. a fault tree can be formed as illustrated in figure 331. The parallel paths in an RBD denote the AND gate for redundant component functions in a fault tree. a reliability diagram can be transformed into a fault tree as shown in figure 331.4 Example An RBD and fault tree are transformed from the example event tree presented in figure 327. Therefore.
4 5 6 ! 7 These parallel paths indicate AND gates for redundant component functions. Figure 332. Event tree to fault tree transformation. 15 7 16 3 17 8 18 1 19 9 20 4 21 10 22 i 23 11 24 5 25 12 26 2 27 13 28 6 29 14 30 SUCCESS FAILURE A 1 SUCCESS FAILURE B 1 SUCCESS FAILURE B 2 SUCCESS FAILURE C SUCCESS FAILURE B 3 SUCCESS FAILURE A 2 SUCCESS FAILURE D FAILURE FAILURE A2 FAILURE A 12 SUCCESS FAILURE A 1 16 7* 3* 1* i 26 12 5* 2 i *Note that not all events represented here are failures. ! 1 2&3 Grief 4&5 Woe 6&7 Evil 2 3 4 5 6 7 Figure 331. 359 .2 1 3 These series nodes indicate an OR gate beneath TOP. RBD to fault tree transformation.
Equivalent logic RBD and fault tree. 360 . COMMAND FAILURE FAILURE TO DEWATER RESPONSE FAILURE EXACT SOLUTION: PTOP = PS + PP PK – PP PK PS + PBP P – PBPPP S – P BP KPP + PBP KPP PS RARE EVENT APPROXIMATION: PTOP = PS + PP PK + P PP B CUT SETS PATH SETS S/P S/K/B S P/K P/B S FLOAT SWITCH FAILS OPEN WATER REMOVAL FAILS P PUMP FAILS MANUAL REMOVAL FAILS K KLAXON FAILS B BAILING FAILS See Figure 327. (b) Fault tree. S P/K P/B (a) RBD.PUMP P FLOAT SWITCH S KLAXON K BAILING B CUT SETS PATH SETS S/P S/K/B See Figure 327. Figure 333.
9. 3. The analyst starts with an initiating event and performs a forward (bottomup) analysis using an event tree (sec. Los Altos.5 Advantages These techniques allow the analyst to overcome weaknesses of one analysis technique by transforming a system model into an equivalent logic model as another analysis technique. and extensive quantitative or pseudoquantitative analysis can be performed. Then. based on variations of response available within the system. 3.9.. A conceptual illustration of how a cause is assessed to understand its consequences is presented in figure 334. New Mexico. W.14.S. J. and discrete. the RBD can be transformed into a fault tree.10 CauseConsequence Analysis 3. 361 .9. J. The cause portion of this technique is a system challenge that may represent either a desired or undesired event or condition. This technique explores system responses to an initiating “challenge” and enables assessment of the probabilities of unfavorable outcomes at each of a number of mutually exclusive loss levels.1 Description A causeconsequence analysis is a symbolic logic technique described in references 3. however.7 Bibliography Gough.: “A New Approach to the Analysis of Reliability Block Diagrams.M. SAIC.6 and 3.” Proceedings from Annual Reliability and Maintainability Symposium. The cut sets and path sets required to perform these transformations for large complex systems may require many manhours or extensive computer resources to determine. 1990. Note that the cause has an associated probability. The cause may be a fault tree TOP event and is normally. and each consequence has an associated severity and probability. The consequence portion of this technique yields a display of potential outcomes representing incremental levels of success or failure. it affords two advantages over the event tree—time sequencing of events is better portrayed. quantified as to probability. 3. Riley. 3. but not always.3.6 Limitations These techniques possess the following limitations: (1) (2) No new information concerning the system is obtained and the models are only as good as the models being transformed. and Koren.8). For example. This technique provides data similar to that available with an event tree.. and presented below. staged levels of outcome are analyzed. Each increment has an associated level of assumed or calculated probability. a complex system that may be hard to model as a fault tree might be easily modeled with an RBD.10.
probability of the Cause. What circumstances allow this event to proceed to subsequent events? b.10.6 a. P C3 CONSEQUENCE n S n. P C1 CONSEQUENCE 2 CONSEQUENCE 3 S3 . P Cn S 2. and engineered safety features. This technique may be used in conjunction with an FTA to provide a technique sensitivity assessment.2) or assumed. This technique may also be used to compliment an FMEA. Construct the consequence diagram by asking the following questions:3. and other nonhardware systems. At various levels the path may branch with two possible outcomes. management decision options.6. Next. P C2 ANALYSIS Y N OVERPRESSURE RELIEF FAILS P Cn = Probability of the n th Consequence occurring S n = Severity of the n th Consequence P 0. 3.3.6 and 3. Determine the probability. trace the possible consequences to the system from the initiating event.2 Application This technique is typically applied in phase C or E but may also be applied in phase D. What other events may occur under different system operating circumstances? 362 .3 Procedures The procedures.14.10. as described in references 3.CONSEQUENCE 1 S 1. 3. The causeconsequence analysis is particularly useful in analyzing commandstart/commandstop protective devices. Causeconsequence analyses are useful in evaluating operating procedures. Also. Relationship between cause and consequence. CAUSE P0 Figure 334. P0. 3. may be determined by Fault Tree Analysis. that this event will occur. it will evaluate the effect/benefit of subtiered/redundant design countermeasures for design trades and assessment. emergency response systems. (1) (2) (3) Identify the initiating event that challenges the system. for performing a causeconsequence analysis are presented below. This probability may be determined from an FTA (sec.
Branching operator statement may be written in either the fault or the success domain. Si. End event/condition to which analysis leads. Finally. This step is often difficult and subjective due to a scarcity of data. Causeconsequence tree construction symbols. What other system elements does this event influence? d. with the severity level stated. Consequence Descriptor (5) The format of the consequence tree is presented in figure 335. Therefore. Determine the probability of each consequence descriptor. therefore PY+PN = 1. Table 38. For each branching operator. Note that all paths lead into branching operators or consequence descriptors. Pi .3. that the event can happen.14 Symbol Name OR Gate Description Gate opens to produce output when any input exists. The branching operator always has one input and two output paths (yes and no). P i and (1–Pi ) are the probabilities for the yes and no paths from the branch operator. establish the probability. (6) (7) (8) 363 . AND Gate Coexistence of all inputs opens gate and produces an output. Y Event N Branching Operator Output is “Yes” if condition is met and “No” if it is not met. determine the severity of each consequence descriptor. Basic Event An independent initiating event. by multiplying event probabilities along the path that terminates at that consequence descriptor. The consequence descriptor has one input. respectively. What subsequent event could possibly result as an outcome of this event? (4) Use the symbols presented in table 38 to construct the consequence diagram. representing the lower resolution limit of the analysis. The outputs are mutually exclusive. P ci. and is a termination point in the diagram.c. Probability bands are often useful to provide an understanding of the analyst's confidence in the delineated probabilities. no outputs.
The room is equipped with an automatic sprinkler system initiated by a heat detector. 3. The time order of events is examined. example was provided courtesy of Sverdrup Technology.4 Example* Problem: A copying machine uses an electrically heated drum to fix dry ink to copy paper. Combustibles are often present in the copying room near the machine.10. 364 . a less conservative. The probability of failure is finite for both the drum thermostat and the overheat cutoff. Uncontrolled drum temperature can rise high enough to ignite them. because the analysis is exhaustive… (P P ) + P (1 – P ) (1 – P ) + P (1 – P ) P = P P 0 1 0 1 2 0 1 2 0 0 Fault trees or other analyses may be used to establish probabilities for the Initiating Challenge and for Branching Operator Y/N outcomes. Causeconsequence analysis format.1. Tennessee. The drum is also equipped with an automatic overheat safety cutoff to prevent damage to the copier. a fire brigade responds to extinguish the blaze. After a delay. more realistic assessment is possible. Enable assessment of multiple.. Figure 335. End events need not be anticipated. Employees frequent the room and can initiate an emergency response alarm in the event of fire.10.CONSEQUENCE DESCRIPTOR 1 CONSEQUENCE DESCRIPTOR 2 CONSEQUENCE DESCRIPTOR 3 1 2 P P 0 1 P (1 – P ) (1 – P ) 0 P (1 – P ) P 0 1 2 Y N BRANCHING OPERATOR P 2 P 1 Y N BRANCHING OPERATOR P0 (1 – P 1 ) INITIATING CHALLENGE Note that. Inc.14 (1) (2) (3) (4) *This The analysis is not limited to a “worstcredible case” consequence for a given failure. Therefore.5 Advantages Causeconsequence analyses provide the following advantages:3. 3. 3. The drum heater is thermostatically controlled. Tullahoma. The causeconsequence analysis for the above problem is presented in figure 336. coexisting system faults and failures.
Note that. multiple analyses may be needed for a particular system.10.14 (1) (2) Address only one initiating challenge. Potential singlepoint failures or successes. areas of system vulnerability. (6) 3. Example causeconsequence analysis.5M P 0P 1P 2 (1 – P 3 ) P3 Y N EMERGENCY RESPONSE FAILS WATER/FIRE/SMOKE DAMAGE ≈ $50. mutually exclusive levels of loss outcome.000 MANUFACTURER'S TEST DATA EMPLOYEE DETECTION/ RESPONSE FAILS FIRE RESPONSE BRIGADE FAILS P2 Y N HEAT DETECTOR/ AUTO SPRINKLER FAIL P 0P 1 (1 – P 2) COPIER DAMAGE ≈ $250 Y N P1 NEARBY COMBUSTIBLES IGNITE DRUM OVERHEATS P 0 (1 – P 1) COMBUSTIBLES PRESENT NEARBY IGNITION TEMPERATURE REACHED P0 DRUM THERMOSTAT FAILS CLOSED OVERHEAT CUTOFF FAILS Figure 336. Thus. because the analysis is exhaustive… P0 P1 P2 P3 + P 0P 1P 2 (1 – P 3) + P 0P 1 (1 – P 2) + P 0 (1 – P 1) = P BUILDING LOSS ≈ $6. thereby guiding deployment of resources for improved control of risk and optimized utilization of limited resources. and lowpayoff countermeasures are identified and assessed. Therefore.6 Limitations Causeconsequence analyses possess the following limitations:3. 365 . the scale of partial successes and failures is discernible. (5) Probabilities of unfavorable system operating consequences can be determined for a number of discrete. but must be foreseen by the analyst.5M 0 P0 P 1P 2 P 3 BUILDING DAMAGE ≈ $1. The initiating challenge is not disclosed by the analysis.
) and is best applied in phase B. Greenberg.11.8). 3. However. 366 . “Guidelines for Hazard Evaluation Procedures. it should be noted that models can be directly created in the failure domain.P.R. then it is performed after the identification of failure propagation paths by ETA but before FTA’s are begun. Determining the severity on consequences may be subjective and difficult for the analyst to defend.1 Description Directed graph (digraph) matrix analysis.7 Bibliography Battelle Columbus Division.2 Application This technique. If this technique is used as part of a PRA. according to reference 3. the analyst determines combinations of systems or combinations of subsystems within a single system for thorough assessment. then converted into the failure domain.” JBF Associates. 1980. The establishment of probabilities is often difficult and controversial.S Nuclear Power Systems Reliability and Risk Assessment. Butterworths. 3.J. 1983. J. Tennessee. The third phase consists of separating the digraph models into independent models.(3) (4) (5) Operating pathways must be foreseen by the analysts.15 This technique is applied to evaluate the failure propagation paths involving several systems and their support systems.11. First. part. then determining the singleton and doubleton minimal cutsets of each failure propagation path.11 Directed Graphic (Digraph) Matrix Analysis 3. 3. “Risk Assessment and Risk Management for the Chemical Process Industry. 3. The second phase consists of constructing a digraph model in the success domain. Knoxville.R. 3. Finally.: “Loss Prevention in the Process Industries. is a technique using matrix representation of symbolic logic models to analyze functional system interactions.: “On the Adaptation of CauseConsequence Analysis to U. Burdic.15. 1991. G. without first creating the model in the success domain.B. London.” 2 vols. 3. System Reliability and Risk Assessment. or within a single system involving several system elements (subsystem.” 1985. as described in reference 3.15. and Cramer... Logic models are first generated in the success domain.10. etc. H.. This phase is parallel to determining failure propagation paths using an ETA (sec. can be used independently or as an element of a PRA (sec. then converting this model to a digraph model in the failure domain for each failure propagation path. component. J.15).” Van Nostrand Reinhold. and Fussell. Inc. Lees. F. This technique consists of four phases. the fourth phase consists of an assessment of the minimal cut sets relative to probability of occurrence.
d.15 for performing a digraph matrix analysis. A comparison between the digraph and fault tree symbols is presented in figure 337. a. identify every credible initiator to an undesirable event and prepare an event tree that illustrates each specific failure propagation path. c. Connect upstream elements to a downstream element with an OR gate if the upstream element relies on the successful operation of only one of two or more downstream elements. safety assessment reports (such as PHA’s. 3. Create a success domain digraph model for each success path. An outline of the steps involved in producing the digraphs is presented below. Connect upstream elements to a downstream element with an AND gate if the upstream element relies on the successful operation of all the downstream components. Form a failure domain model by taking the model generated in step 2a and interchange all AND gates with OR gates and all OR gates with AND gates. topdown approach to construct a toplevel digraph. b. This simplifying assumption leaves only failure propagation paths that are combinations of systems that must fail for a serious threat to be posed. Study checklists of potential initiating challenges. Form an adjacency matrix that represents the digraph. 3. Continue expanding the elements of new digraphs until the desired resolution level of the analysis is reached. The matrix is constructed by the process illustrated in figure 338.3 Procedures Presented below is a summary of the detailed procedures found in reference 3. then expand each element into its own digraph. such as design specifications and packages. For a complete analysis. The symbols for AND and OR gates for a digraph are different than those used for a fault tree. b. c. From these checklists develop a list of initiators that are applicable to the systems being studied. and prior safety or reliability studies. Prepare a list of failure propagation paths from step 1c. Acquire pertinent information concerning the collection of systems to be assessed.8) to identify failure propagation paths. (1) Identify the associated group of systems (or associated system elements of a single system) to be thoroughly evaluated.2).3. however they represent the same logic as the fault tree symbols. Use a backward. 367 . Assume unity probability for all systems required to work in the failure propagation path. sec. a. Develop event trees for each initiating challenge to the system. (2) Construct a digraph model for each possible failure propagation path. Use event trees (sec. This failure domain model represents a path for failure propagation.11.
15) in which risk is assessed with the probability of the cut sets occurring and the severity of the consequence of the failure propagation path. The output of this code will show all elements connected by a path and illustrate which elements can be reached from a specific element. Assess the singleton and doubleton minimal cut sets. Comparison between digraph and fault tree logic gates. therefore all possible paths between pairs of nodes in the network. (4) 368 . e. Next.15.AND Gate A A Digraph C B B C OR Gate C C Fault tree A Represented logic B A B Event C will occur only if both event A and event B occur. 3. This code is described in detail in reference 3. Determine minimal singleton and doubleton cut sets from the cut sets determined in step 2d. This is accomplished by processing the adjacency matrix with the reachability code. Figure 337. Then determine singleton and doubleton minimal cut sets of the smaller independent digraphs. This assessment can be conducted in a manner similar to that for a conventional PRA (sec. Event C will occur only if event A or event B occurs. Next link all connected elements in the adjacency matrix. (3) Subdivide the digraph into independent digraphs if the success domain digraph model becomes too large to determine singleton and doubleton cut sets for the computer platform being used. d. use this information to determine singleton and doubleton cut sets.
element (M. For example. first generating a success domain model may prove to be beneficial. The system consists of two redundant power supplies to power a motor that drives a pump. M) is equal to PS–2 (nonzero or 1 value). The success domain model of this system is presented in figure 339(a). For example. there is no path from node i to j.15. 369 .j) = 1.P) = 1. In cases with more complex systems. adapted from reference 3. which means there is a no straight path between the main power supply and the auxiliary power supply. then there is a second component that must fail along with component i to cause component j to fail.4 Example An example digraph matrix analysis. The adjacency matrix and adjacency elements are presented in figures 339(c) and (d). presented in figure 339(b). The failure domain model.j) is ≠ 0 or 1. Note that this model represents the success path for successful operation of the pump.Type Digraph Adjacency Matrix A 0 0 B 1 Direct connection A Element A Element B B0 Element A AND gate Element C Element B Element A A A B0 C0 A A 0 0 0 0 0 0 B 0 A 0 B 0 1 0 C B C 1 OR gate Element C Element B B0 C0 Figure 338. for a simple system is illustrated in figure 339. respectively. PS–2) = 0. For example. was generated by replacing the OR gate in the success domain model with an AND gate.11. The adjacency matrix illustrates whether there is a direct path from node i to node j.j) = 0. Construction of digraph adjacency matrix. If element (i. element (PS–1. which means there is a straight (uninterrupted) and unconditional path between the motor and pump. Inspection of the two models suggests that for simple systems the failure domain model can easily be generated without first generating the success model. there is a path from node i to node j. adjacency element (PS–1. 3. If the adjacency element (i. If matrix element (i.
the reachability matrix yielded the new information that if both power supplies failed. by connecting linked pairs from the adjacency matrix. The reachability matrix and reachability elements are presented in figure 339(e) and (f). Simply stated.e. Processing the adjacency matrix into the reachability matrix yields the paths between all pairs of nodes. failure of both the main and auxiliary power supplies will cause the pump not to operate. Obviously. Therefore the reachability matrix illustrates the complete pathways (through linked node pairs) of the graphical model elements illustrated by the adjacency matrix. for complex systems which are modeled with many nodes and logic gates. Since the technique identifies singleton and doubleton minimal cut sets without first determining all minimal cut sets. this approach allows more rigorous subdividing of the independent subgraphs. in this example. The methodology to generate the reachability matrix from the adjacency matrix is presented in reference 3. the reachability matrix illustrates the pairs of nodes that a path exists between.e. the pump will not operate. the single failure of the motor will cause the pump not to operate. then P will not operate (even though neither PS–1 or PS–2 are directly adjacent to P).j) and either i or j is a value of 1. that the reachability elements include all the adjacent elements and the new information that if both PS–1 and PS–2 fail.. to cause M to fail to operate (i.This symbol represent the second component that must fail. The summary matrix presented in figure 339(g) illustrates which components can lead to failure of the pump. given the failure of PS–1. (2) 370 .15 (1) The analysis allows the analyst to examine each failure propagation path through several systems and their support systems in one single model.. The only singleton in this system is the motor.e. failure of both the main and auxiliary power supplies will cause the motor not to operate). If an “*” is entered as a matrix element (i. then the other corresponding component i or j is a singleton. considerable computer resources can be saved over other methods such as the FTA.15. i.. The only doubleton of this system is the pair of redundant power supplies. 3. P. Unlike the FTA with failure propagation paths divided in accordance to arbitrarily defined systems. this technique allows determination of singletons and doubletons which otherwise would not be as readily identified. Note. in this example the singletons (single point failures) and doubletons (double point failures) could have easily been identified without performing a digraph matrix analysis. i. respectively. If a “*” is entered as a matrix element (i. However.j) that corresponds to component i and component j.5 Advantages The digraph matrix analysis provides the following advantages: 3. The reachability elements are derived from the reachability matrix in the same manner that adjacency elements are derived from the adjacency matrix.11. Therefore. then component i and component j form a doubleton.
PS2 PS2. P. PS1 Main Power Supply. (f) Reachability element. M. PS2 Motor. P. M Auxiliary Power Supply. M Pump. M. Example digraph matrix analysis—Continued 371 .Main Power Supply. PS1 M. PS1 Motor. PS2 (a) Success domain model. P Auxiliary Power Supply. P (b) Failure domain model. PS1 PS1 PS2 M P 0 0 0 0 PS2 0 0 0 0 M PS2 PS1 0 0 P 0 0 1 0 PS1. PS2 PS2. P. Figure 339. PS1 (Adjacent) PS2. PS1 PS1 PS2 M P 0 0 0 0 PS2 0 0 0 0 M PS2 PS1 0 0 P PS2 PS1 1 0 (d) Adjacency elements. M. 1 (Adjacent) (e) Reachability matrix. Pump. PS1 M. M. PS2 (Adjacent) PS1. P. 1 (c) Adjacency matrix. PS1.
Figure 339. in 1982.6 Limitations Digraph matrix analyses possess the following limitations. “Digraph Analysis Assessment Report. and Avni.1 Description The combinatorial failure probability analysis using subjective information is described in reference 3. This technique provides the analyst a procedure to propagate probability data derived from the subjective probability scales defined in MIL–STD–882C.7 Bibliography Grumman Space Station Division. Kandel.3. PS2 (g) Summary matrix.15 (1) (2) Trained analysts and computer codes to perform this technique may be limited. E.12 Combinatorial Failure Probability Analysis Using Subjective Information 3.: “Engineering Risk and Hazard Assessment.” Reston Virginia. Boca Raton. 3. 3.. Example digraph matrix analysis—Continued. CRC Press Inc.2 372 . October 1991. This technique was developed by the System Effectiveness and Safety Technical Committee (SESTC) of the American Institute of Aeronautics and Astronomics (AIAA).. complete treatment may require more computer resources than FTA’s.11.11.” vol.12. 3. A. For particular types of logic models.1 1 PS1 PS2 M P *  PS1 *  PS2 *  M *  P Singletons: M Doubletons: PS1. Florida.3. 2.16 and presented below.
and causeconsequence analysis (sec. dimensionless “probability values” have been assigned to the probability increments (frequent.8).12.3. Descriptive words and definitions for the level of the scale are also given in this table. 318 and 319).2 Application This technique is typically performed in phase C and is applicable when no quantitative failure probability data are available and may be used in conjunction with other analyses such as an RBD (sec. as described in reference 3.00000 8×10–3 to 8×10–2 8×10–4to 8×10–3 8×10–5 to 8×10–4 0. remote. ETA (sec. 3. 3. (2) Estimate subjective failure probabilities of contributor events or conditions using the scale defined in MIL–STD–882C. Table 39.10).3 Procedures The procedures. 3. AIAA/SESTC Threshold Levels 8×10–2 to 1. STA (sec. Will occur several times in life of an item. 3. *Arbitrarily selected. dimensionless numbers. Table provided courtesy of Sverdrup Technology. Tullahoma. and improbable) defined in MIL–STD– 882C.3.00000 to 8×10–5 Probability Level* 3×10–1 3×10–2 3×10–3 3×10–4 3×10–4 Level A B C D E MIL–STD–882C Descriptive Word Frequent Probable Occasional Remote Improbable Definition Likely to occur frequently.2 The subjective scale for these arbitrary values is presented in table 39. probable. 3.5). So unlikely if can be assumed occurrence may not be experienced.6)..12. Unlikely but possible to occur in life of an item. for a combinatorial failure probability analysis using subjective information are presented below. Propagate these values in the same manner as quantitative data is combined in classical numerical methods (such as presented in figs.2 Select and consistently apply the same probability exposure interval (operating duration or number of events) for every initiator probability estimate used in the analysis.3. 3. Inc. Likely to occur sometime in life on an item.16. occasional. Convert the final probability number resulting from propagation (step 3) back into the subjective scale defined in MIL–STD–882C. Correlate the subjective estimate (step 2) with the arbitrary.2 (3) (4) 373 . FTA (sec. Combinatorial failure probability analysis subjective scale.3. dimensionless values (step 1). (1) Arbitrary. Tennessee.7).
Tennessee.2 • • • • What is the probability that any one of the six bearings will suffer burnout during the coming decade? The system schematic and fault tree are presented in figure 340(a) and (b).12. 3. Note both the arbitrary subjective probability value and letter representing the relevant probability level from table 39 are presented for each fault tree initiator. *This example was provided courtesy of Sverdrup Technology. Replacement of a bearing costs $18.3..000 and requires 3 wk of down time. these systems will stop operation of the rotating machine early enough to prevent bearing damage. 3.6 Limitations This technique should only be used when actual quantitative failure rate data is not available.3. unless used in a comparative fashion. The use of actual quantitative data is preferred over this method. there are sensing/alarm/shutdown systems for: • lube pressure failure • cooling water loss of flow.”) Failure records for the individual system components are not available. If they function properly. Problem/Background: • A large rotating machine has six mainshaft bearings. Inc.3. In addition. This tool should only be used for comparative analysis only.5 Advantages This technique allows the analyst to perform a probabilistic assessment based on the exercise of subjective engineering judgment when no quantitative probability estimates are available.12. Data and results. Each bearing is served by: • pressurized lubrication oil • a watercooled jacket • a temperature sensing/alarm/shutdown system. but probabilities can be estimated using the subjective scale of MIL–STD–882C.4 Example The following example* uses this subjective combinatorial technique in a fault tree problem.16 374 . Tullahoma. (System sensitivity makes the necessary allowance for machine “rollout” or “coasting. may be poorly received.12. respectively.
000 Replacement Costs • 3Week Interruption of Use (a) System schematic. Bearing Temperature Sensing/Alarm/Shutdown Failure UNRESOLVED UTILITY SERVICE FAILURE 2x106 SHUTDOWN FAILURE 3x10 2 B 10year failure probability estimates are entered at the lowest levels of the tree and propagated upward.PROTECTIVE FEATURES T dT /dt • Sensing • Alarm • Shutdown H2O Flow Lube Pressure UTILITY SUBSYSTEMS Bearing Burnout Loss Penalty: • $18. UNRESOLVED LUBE FAILURE 9x10 7 Lube Pressure Sensing/Alarm/Shutdown Failure UNRESOLVED COOLANT FAILURE 9x107 Coolant Loss Sensing/Alarm/Shutdown Failure LUBE PRESSURE FAILURE C 3x10 3 3x104 SHUTDOWN FAILURE D COOLING WATER FAILURE C 3x10 3 SHUTDOWN FAILURE D 3x104 (b) System fault tree. BEARING BURNOUT 6x108 E Bearing burnout is “ Improbable” for any one of the six bearings over the 10year period. Example combinatorial failure probability analysis. 375 . Figure 340.
e. liquid flow. Do the sensors currently exist or can they be developed? Can they be obtained in time to satisfy schedule requirements? (2) (3) (4) (5) (6) (7) 376 . pressure.13 Failure Mode Information Propagation Modeling 3. acceleration. The technique provides insight into both the types of information that should be measured to safeguard the system.17.2 Application This technique effectively directs resource deployment to optimally safeguard a system against potential failures by identifying measurement requirements. This information may be transmitted during the initial outset of a variety of failure modes.13. A minimal success set is a sensor group that encompasses all failure modes.13. sensor type. to perform failure mode information propagation modeling are presented below. friction.g.4). subassemblies. air flow. spring. thermal heat transfer. Identify and record the failure modes for each component and assign a letter to each failure mode for each component. as described in reference 3. 3.3 Procedures The procedures. (1) Divide the system into its principle functional components and assign a number to each component. Identify the physical links (energy flow and shared stress) between the components of the system. assemblies. Feasibility. rolling element.). These requirements are defined in terms of measured parameter. the resolution of this analysis is dependent upon the level (i. These links include such items as electrical power. thermal. Classify the failure mode information constituents by their signal characteristics (e.. etc. Identify and record the flow of failure mode information at each physical link that is available externally to each component and transmitted to one or more other components. and sensor location. cost. 3. gas flow. This technique is best applied in phase C but may also be applied in phase D. or piece parts) to which the system elements are resolved.17 and presented below. The following questions should be asked: a. Identify the minimal success sets of the sensor network.13. This technique involves separating a system into its basic functional components and examines the benefit of measuring precedent failure information that may be transmitted between components of a system. Like the FMEA (sec.3. etc. and location within the system at which sensors might be appropriately positioned.1 Description Failure mode information propagation modeling is a qualitative analysis method described in reference 3. and effectiveness. 3.. subsystems. Assess the various minimal success sets in terms of feasibility.
13. For example. frame. A schematic of the system is presented in figure 341(a).) Assume a constant aerodynamic fan load. Solution: (1) Perform steps 1–5 identified in section 3. the failure modes for the fan include shaft or rotor binding. and bearing. is linked to the fan belt by friction. through which power is delivered to the fan. a. the failure information available includes electric motor bearing vibrations (1– B). (Consider motor bearings as integral parts of the motor. For example. These steps are explained below and illustrated in figure 341(b). Tennessee. The electric motor.13. fan belt slipping and breaking (2–A/B). Divide the system into its principle functional components and assign a number to each component. and shorted winding.3. bearing vibration. c. Determine sensor network minimal success sets for the system. Step 5.3. Is the cost of installing. Problem: Consider a ventilating fan powered by an electric motor through a belt drive. open winding. the electric motor bearing vibration (1–B) and fan bearing vibration (5–B) can be monitored by an accelerometer at test point 4/1 (between frame. Identify the physical links (energy flow and shared stress) between the components of the system. These are the electrical motor. fan belt. Identify and record the failure modes for each component and assign a letter to each failure mode. and bearing binding (5–A). Cost. *This example was provided courtesy of Sverdrup Technology. A common frame structure supports both the motor and a bearing. maintaining. component 1 and electric motor. for the mechanical link between the electric motor and frame. Step 4. fan. and operating the sensor network less than the cost of the failure that the system is being safeguarded against? c.. Step 3. Catalog the flow of failure mode information at each physical link that is available externally to each component and transmitted to one or more other components. component 4).16 377 . b. and is mechanically and thermally linked to the frame. Step 1.b. e. Inc. Classify the failure mode information constituents by their signal characteristics. for example. Are there other preventive maintenance activities more effective than installing a sensor network? Will the sensing network forewarn before the start of system failures or does it just announce system crashes? Will the sensors impede normal system operation? Will they degrade system performance? Will they pose any new hazards to the system? Will the sensor network operate dependably? Will the sensors have adequate sensor redundancy? 3. Step 2. For example. d. Effectiveness. has electrical power input.4 Example The following example* uses failure mode information propagation modeling to a sensor network success set for a system. Tullahoma.
3. The propagation of a failure through the system is not addressed. The variables of these transfer functions are represented by probability density functions. a point estimate for failure of the component relative to the failure mode under consideration can be determined.5 Advantages Information propagation modeling provides the following advantages:3. The risks of the failure modes are not quantified in terms of criticality and severity.8 and 3.13.(2) From the information displayed in figure 341(b).17 (1) Allows the analyst to identify measurement requirements.18. as described in references 3. 3.18 Probabilistic Design Analysis 378 . This traditional approach is vulnerable if significant experience and historical data are not available for components similar to that which is being considered. Given that the probability distributions for both the load and capability functions are independent. if implemented.14. and capability (strength) the component has to withstand failure in that mode.14 3. 3.6 Limitations Information propagation modeling possesses the following limitations:3.1 Description A PDA. is a methodology to assess relative component reliability for given failure modes.8 3.2 Application A PDA can be used to analyze the reliability of a component during phase C of a program. Determine the minimum success sets of measurement sensors. Externally induced and common cause faults are not identified or addressed. The matrix and minimum success sets for this system are presented in figure 341(c).13. can help safeguard a system by providing warnings at the onset of a failure mode that threatens the system. The component is characterized by a pair of transfer functions that represent the load (stress.17 (1) (2) (3) (4) This technique is only applicable if the system is operating in a nearnormal range. the interference area of these two probability distributions is indicative of failure. that. construct a matrix of failure mode versus sensor type (with each test point identified). (2) 3. or burden) that the component is placed under by a given failure mode. 3.14. Under these conditions. and for the instant of time immediately prior to the initiation of a failure mode. The PDA approach offers an alternative to the more traditional approach of using safety factors and margins to ensure component reliability. These sets are sensor groups that encompass all failure modes. Compliments an FMEA.
Elements: • Electric Motor • Fan Belt • Bearing • Fan • Frame (a) System schematic. Tachometer 1A/C/D 2A/B 3A 5A 0/1 1A/C/D 2A/B 3A 5A Belt Slip Monitor 2A/B Tachometer 1A/C/D 2A/B 3A 5A 1A/C/D 2A/B 3A 2/3 5A Belt Slip Monitor 2A/B Electric Motor A. Figure 341. Slipping B. Example failure mode information propagation model—Continued 379 . Shorted Winding 1/2 1A/C/D 2A/B 3A 5A Fan Belt A. Shaft/Rotor Binding B. Vibration Mechanical Thermal Spring Liquid Flow 4 Accelerometer 3A 5B 5 •••••••••• Gas Flow (b) Model. Open Winding D. Breaking Fan A. Binding B. Blade Damage 1A/C/D 2A/B 3A 5A 3/00 Power Monitor 1A/C/D 2A/B 3A 5A 1 1B 2A/B 5A 2 Accelerometer 1B 5B Accelerometer 3A 5B 3/5 3A 5A/B 3 Flow Monitor 1A/C/D 2A/B 3A 5A 4/1 1A/C/D Heat Flux Monitor 1A/C/D Electrical Power Friction Air Flow 4/5 Rolling Element Frame 3A 5A/B Bearing A. Bearing Vibration C.
Confirm the selection of critical design parameters. Example failure mode information propagation model—Continued. 3. Identify the random variables that effect the variation in the load to be imposed on the component for the given failure mode.18. (1) (2) (3) (4) (5) (6) Specify the system design requirements. Establish relationships between the critical parameters and organizational.8 and 3. Identify the failure modes of the system by using a method such as a FMEA (sec.4). programmatic. Load Transfer Function: L = fL (X1. X2. 380 . . Incorporate these random variables into a transfer function that represents this load (stress. Ascertain the reliability associated with each critical failure mode with the following probabilistic analysis method: a. adapted from reference 3.3 Procedures The procedures. X3.14.. Figure 341.. 3.Te Po st int Minimal Success Sets* 0/1 1/2 2/3 3/00 3/5 4/5 4/1 on ito Ta r ch om ete r Be lt S lip Mo nit Ta or ch om ete r Be lt S lip Mo nit Flo or wM on ito r Ac cel ero me ter Ac cel ero me ter Ac cel ero me ter He Mo at F nit lux or Power Monitor at 0/1 or Tachometer at 1/2 or Tachometer at 2/3 Se nso r Failure Mode 1A 1B 1C 1D 2A 2B 3A 4 5A 5B √ Po we rM √ √ √ √ √ or Flow Monitor at 3/00 √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ Not Applicable √ √ √ √ and Accelerometer at 4/1 *Sensor groups that envelope all failure modes √ √ √ √ √ √ √ (c) Minimal success sets. or burden). for performing a PDA in the context of a total design reliability program for a system are presented below. Identify variables and parameters that are related to the design. and established failure criteria.. These requirements should be stated in clear and concise terms that are measurable and verifiable.Xn).
Expressions for PF between combinations of exponential. .Ym).8. Repeat step 8 for each critical component for the system. maintainability. gamma. log normal. then modify the design to increase reliability. Perform trade studies (sec. is dependent upon the type of load and capability distributions. or as a continuously variable form of the density function.. this interference is indicative the failure mode will occur. Y2. Consider each variable of the transfer function as a probability density function (illustrated in figure 342). including load and capability in this assessment. log normal. or other distribution. gamma. Reliability is the probability that the failure mode will not occur. Repeat the process until the design reliability goals or requirements are met. c. For independent load and capability functions. Determine the relative reliability of the system. Capability Transfer Function: C = gC(Y1. e. Y3. Also illustrated in this figure is the interference of the load and capability density functions. However.1) to reassess and optimize the design for performance. The expression for PF. Expressions for PF for various distributions are found in most advanced statistics textbooks and handbooks. (8) (9) (10) (11) 381 . Determine probability distributions of the load (stress. Note: The area under an entire probability density function curve is equal to a probability of one. Repeat the above steps to optimize system reliability. In figure 343. 2. Weibull. or burden) and capability (strength) of the failure mode. therefore a range between two values of the independent random variable of a density function curve is equal to a probability less than or equal to one. etc. environmental issues. d. Identify the random variables that affect the variation in the capability of the component to withstand the load imposed for the given failure mode. Calculate the reliability (R) for the failure mode from the load and capability distributions. Probability density functions of both load and capability continuous random variables for a given failure mode are presented in figure 343.. The expression for reliability is: R = 1 – PF . The density function can be represented as either a discrete variable distribution using empirical test data. cost. and Weibull distributions are found in reference 3. Incorporate these random variables into a transfer function that represent this capability (strength).. both density functions are normal distributions with different means and variances. generally one or both of these density functions may be an exponential. (7) Assess the reliability for each critical failure mode. Gather data to perform the load and capability calculations.b.
X2... .Xn) L C Capability Transfer Function C = g C(Y1.Ym) Figure 342. Y2 . .8 382 . Load and capability transfer functions.3...f1(X1) g1(Y1) X1 f 2(X2) g2(Y2) Y1 X2 Y2 fn(X n) gm(Ym) Xn Ym fL(L) gC(C) Load Transfer Function L = fL(X1...
8 Therefore PDA’s can be used to determine valuable areas of the design and aid in determining the resource allocation during the test and evaluation phase. f L(L) Load f L(L) Capability fc (C) Probability Density Function L Overlap Indicative of Failure C L.f C(C).8 The PDA approach offers a more accurate and truly quantitative alternative method to the more traditional approach of using safety factors and margins to ensure component reliability. Interference between load and capability density functions.3.14. 383 .8 3.3.18 Historical population data used must be very close to the asplanned design population to be viable. This technique mandates that the analyst address and quantify the uncertainty of design variables and understand its impact on system reliability of the design.3.3.8 Determining the density functions of the random variables in the load and capability transfer functions may be difficult.14.4 Advantages A PDA provides the following advantages: (1) Allows the analyst a practical method of quantitatively and statistically analyzing the relative reliability of a system during the design phase.18 The technique provides a more precise method for determining failure probabilities to support FTA’s than does use of subjective methods.5 Limitations A PDA possesses the following limitations: (1) (2) (3) The analyst must have experience in probability and statistical methods to apply this technique.C Figure 343.8 3. (2) (3) (4) 3.3. Extrapolation between populations can render the technique nonviable.3.
8). (2) defines the scope of the system.6. elements of the system vulnerable to hazards are identified. the system is defined. an assessment of risk is performed in terms of probability and severity. (6) and if the risk are unacceptable determines countermeasures to mitigate the risk. are typically performed during phase 1. Severity is established. b. 3. 3.2). 3. During phase 2. According to reference 3.2 Application A PRA is performed to identify consequence of failure in terms of potential injury to people. Define the system to be assessed. 3. Then.6).15) to assess the risk associated with the various failure modes. during phase 3. a PRA generally consists of three phases. 3. During phase 1. identify the elements (targets) of the systems that are susceptible to hazards.15 Probabilistic Risk Assessment 3. (1) Phase 1 (activities performed during the preliminary design stage). 384 . sec. 3. adopted from reference 3. Therefore. 3. FTA (sec. or quantitative in nature. PHA’s (sec. this technique should be used as one element among other elements of a PRA (sec.1 Description A PRA is a general term given to methodologies that assess risk. or loss of mission requirements. and by comparison to other societal risks. Perform a PHA (sec. damage to equipment or facilities. ETA (sec. these methods can be either subjective (as by use of the risk assessment matrix. offer guidance in performing a probabilistic risk assessment:. The PRA is typically performed in phase C. (5) assesses the risk for each hazard and target combination in terms of probability and severity.6.(4) This technique identifies the relative probabilities that various failure modes will occur. the analyst: (1) identifies targets. a consequence analysis is performed. and from an overall perspective identify potential hazards.15. hazards are identified. FMECA (sec. 3. and (7) and repeats the assessment with the countermeasures incorporated. (4) identifies hazards. 3. but does not address the severity of the failure modes. 3. a. Although PRA methods are customarily thought of as being quantitative. In performing a PHA. 3. the failure propagation paths and probabilities are established.4) and/or causeconsequence analysis (sec. Finally.3 Procedures The following procedures.15.1).10) are performed. (3) recognizes the acceptable risk limits.2).15. and the overall scope of types of hazards to be assessed is defined.
11) may be performed after the ETA is complete and before FTA’s have begun. Depending on the size and complexity of the system being assessed. 3. Assess risk of all failure modes in terms of severity and probability.8). Determine initiators and propagate probability of failure with methods such as FTA (sec.4) may be performed.15 e.5 Limitations A PRA possesses the following limitations: (1) (2) (3) Probabilistic risk assessment requires skilled analysts. a. Probability of failure modes can also be determined with the probabilistic analysis method presented in section 3.3.15. Identify failure propagation paths with techniques such as an ETA (sec. and (2) determines the alternate logic paths from the initiating event.6). a. Calibrate the risk of the system being examined by comparing it to other known societal risks.14. the analyst (1) identifies an initiating challenge to the system. allows operating decisions to be made. significant manhour and/or computer resources may be needed to complete.4 Advantages Assessing risk avoids unknowingly accepting intolerable and senseless risk.15. 3. 385 . c. In performing an ETA. If the analyst is untrained in the various tools required. Establish the severity of the failure modes. A causeconsequence analysis (sec.1 3. (3) Phase 3 (perform a consequence analysis). 3. c. A digraphmatrix analysis (sec.3. and improves resource distribution for control of loss resources. b.10) may be performed to establish both failure propagation path and probabilities of causes and consequences. An FMECA (sec. Sufficient information and data may not be available to perform a thorough assessment. 3. d. 3. Examine all failure modes and criticality ranking of each system element. 3.(2) Phase 2 (activities initiated after accomplishing hardware and configuration selections). the tool could be misapplied or the results misinterpreted. b.
Fault Tree/Reliability Block Diagram/Event Tree. Lecture presentation.2 “System Safety Program Requirements. SSDC–11. R.: “Transformations. and Kumamoto.L.. P. 3.9 Clemens. P.11 Briscoe. P. November 1992. December 1992. Sverdrup Technology. vol. NY. 3. October 1992..18 “Solid Propulsion Reliability Guidebook. 2.R. and Space (SAE). 3. R. and Koren.. Sea.: “Failure Modes and Effects Analysis.: “A New Approach to the Analysis of Reliability Block Diagrams.: “Energy Flow/Barrier Analysis. Sverdrup Technology. Inc. June 1990.. Inc.17 Clemens. Riley.J. Inc.: “Failure Information Propagation Modeling. and Guttman. 3. May 1993. I. 3. Lecture presentation. June 1993.” Fourth edition. 3.15 Alesso. Sverdrup Technology.: “Initial Guidance on DigraphMatrix Analysis for System Interaction Studies.L. 3.. 3. Los Altos.5 Mohr.1 Clemens.: “Event Tree Analysis.: “Probabilistic Risk Assessment. Lecture presentation.: “Fault Tree Analysis.” Sixth edition. H. NY. Inc.” Third edition.R.. and Lamberson: “Reliability in Engineering Design. Inc.4 Clemens. 3.P. Inc.” MIL–STD–882C. Sacks.” Lecture presentation.14 Clemens.” Fourth edition.” Second edition. Sverdrup Technology.8 Kampur. K. Inc. Lecture presentation. 386 .13 Clemens. Lecture presentation.” Second edition.” NUREG/CR–1278. Sverdrup Technology.F..” Second edition..L. James M.L. Sverdrup Technology. 1991. Lecture presentation. Inc.L.C. 3. P.: “Working with the Risk Assessment Matrix.. Air..” Proceedings from Annual Reliability and Maintainability Symposium.. P. P. SAIC. Lecture presentation. 3. June 1993.E. 1990. Lecture Presentation.: “Combinatorial Failure Probability Analysis Using MIL–STD–882B.: “Preliminary Hazard Analysis..: “CauseConsequence Analysis. Sverdrup Technology. July 1993.. and Smith.7 Gough.: “Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications.J.” John Wiley & Sons.” Fourth edition. W. January 1993. Sverdrup Technology. 3.REFERENCES 3. 3. Glen J.” The Institute of Electrical and Electronic Engineers. Sverdrup Technology.S. 3. Inc. Lecture presentation. 1977. August 1991.. H.D.” Lawrence Livermore National Laboratory. September 1982. A.. 3.: “Risk Management Guide... October 1989.3 Mohr.L. draft.” Third edition. C. NM. March 1983. Inc. H.10 Swain.. P. J. P. 3.” System Safety Development Center. Sverdrup Technology.6 Henley. DOE 7645/11.12 Clemens. 3.16 Clemens. E. Inc. June 1992.L.L.” The Phillips Laboratory and the Engineering Society for Advancing Mobility Land.
An alternate approach to approximate sensitivity is to assume a straightline relationship between two points in the sample space of the relationship. This method is often preferred for relationships with parameters that are interrelated.. or material strength).1.g. Find the coefficients of influence4.1. This can facilitate optimizing a system. Typical examples of the use of sensitivity analysis are manufacturing formulation and processes (e. and to solve the relationship for two conditions represented by two values of the parameters in question. Solve the equations for the coefficients of influence to find the sensitivity at given conditions.5.1. This section is included to give the systems engineer an understanding of the standard methods of dimensioning and tolerancing. or adjusting a system for wear or changing conditions.1 Description In sensitivity analysis. A summary of the advantages and limitations of each tool or methodology discussed in this section is presented in table 41. reducing variability. 4.2 Application Sensitivity analysis typically should be performed in phase C or D.1 Sensitivity (Parametric) Analysis 4.4. ANSI–Y–14. Sensitivity analysis can show which parameters affect a system the most or least. erosion rate. is discussed.3 Procedures The procedure for obtaining the sensitivity of a relationship by analytical methods is as follows: (1) (2) (3) Generate an equation for the relationship under consideration. bond strength. 4. when factors such as age affect performance. In addition. sensitivity functions (or coefficients of influence) are generated by taking partial derivatives with respect to each parameter that affects the outcome of a relationship. DESIGNRELATED ANALYTICAL TOOLS Two designrelated analytical tools (sensitivity analysis and tolerance stackup analysis) that can be useful to systems engineering are discussed in this section. Sensitivity analysis is especially useful when environmental conditions can change.1 by taking the partial derivatives for each parameter under consideration. burn rate. This analysis can be used for nearly any type of relationship. such as throat area and exit pressure in the thrust equation. 41 . or when manufacturing tolerances affect performance. Geometric Dimensioning and Tolerancing. 4.
machining and casting. their inclusion in tolerance stackup analysis is superfluous. Thus. it becomes increasingly unlikely that all dimensions will be simultaneously worstcase.g..g. For example. Consideration of significant digits may be helpful. For example. A moderate amount of training and practice is required to effectively use standard dimensioning and tolerancing. e. a functional part that would comply with true position tolerances may not comply with rectangular tolerances. e. a 0. 42 . Standard dimensioning and tolerancing 4.030 tolerance may have a smallest unit of measurement greater than a 0. some aspects of dimensioning and tolerancing per ANSI–Y–14.0005 tolerance. Limitations It is often not easy to isolate a variable to obtain a second derivative. and function.1 Advantages The effect of each parameter can be assessed to determine which parameters have the greatest effect on the outcome of a process and which parameters can yield the most benefit from adjustment. Statistical tolerance analysis usually assumes a normal distribution of dimensions in the tolerance zone.. If the approximation approach above is taken. Statistical analysis can show that. even if exceeding a requirement is possible. but changes the nozzle expansion ratio and exit pressure. This is facilitated by following the dimensioning and tolerancing system of ANSI–Y–14.3 Worstcase tolerance analysis is conservative in that when many tolerances combine.5 are better suited for production. (2) It may be superfluous to combine tolerances from different manufacturing processes. The pressure ratio is typically found by iteration or by tables. Designrelated analytical tools and methodologies.2 Dimensioning and tolerancing per ANSI–Y–14. in that: (1) If some tolerances are much smaller than others. care must be used to ensure a small enough range for parameter values to achieve the desired accuracy. it may be extremely unlikely. In addition. Tool or Methodology Sensitivity (parametric) analysis Section 4. which may be unrealistic. Worstcase tolerance analysis can simply determine the envelope of possible form.5 is fairly standard.Table 41. Dimensioning strategy can minimize the cumulative tolerance stackup. In addition. changing a throat diameter not only changes motor pressure.5. Tolerance stackup analysis 4. care must be exercised when combining tolerances. true positioning allows for a circular tolerance zone. whereas putting tolerances to rectangular coordinates allows a square tolerance zone. fit. when obtaining the sensitivity of thrust to throat diameter.
1).2 rb = 0.5 in/s C* = 5100 in/s g = 386.2) to obtain the slope at a particular value of A * .1) where Pc is the chamber pressure. The first step is to calculate the chamber pressure. The sensitivity of thrust to throat area is to be investigated for a hypothetical motor with the following characteristics: As = 300 in2 A* = 1.9 in2 .4 Example In the following hypothetical example. To find the sensitivity of pressure to motor throat area. r b is the propellant burn rate.9 in2 .4. The next step is to calculate Mach number (M) iteratively from equation (4. The equation for this analysis is the pressure equation. C * is the propellant gas characteristic exhaust velocity. C* Pc = – rb g A* 1 As 2 A* ( ) (4.1.1) with respect to A* . The equation for pressure is: C* Pc = rb g As A* (4.06 lbm/in3 γ = 1.3): Ae 2 1 2 * = 2 1+ A M +1 − 1 2 −1 .1 in2 Ae = 10 in2 = 0.40 in/s2 . As is the propellant burn surface area. M 2 +1 (4. the sensitivity of pressure with respect to throat area is being determined. 2. Another example of the approximation method is the substitution of selected values into the thrust equation (4.2) where ∂ designates a partial derivative. and A * is the throat area. Numbers can be substituted into equation (4. take the partial derivative of equation (4. is the propellant density. using the first value of A* which is 1.3) 43 .6). Pc is taken over a narrow range where rb is approximately constant. g is gravity. It is intuitively obvious that the relationship between the partial derivative and A* is both negative and inversely proportional to A*2. substituting into equation (4. The sensitivity is found by substituting values for the variability into the partial derivative equation.
4.93 CF 1. For example.6) The above calculations should be performed again. 1 +1 −1 2 2 2 2 −1 Pe CF = 1− P c − 1 + 1 (4.50 1.1 in 2 .75 Pe 1.62 174.1 in2 Conclusion: in2 The thrust (T ) is inversely proportional to the throat area (A * ). The values obtained from both calculations are shown in table 42. 4. Table 42.4).5) The final step is to calculate thrust (T) from equation (4. 44 .4) The next step is to calculate the thrust coefficient (CF) from equation (4. T = CF A* Pc . Pe = Pc 1 1+ − 1 2 −1 M 2 (4.9 A* = 2.52 56. and which parameters can yield the most benefit for adjustment. Pc 62. The pressure ratio is typically found by iteration or by tables.82 2.5 Advantages The effect of each parameter can be assessed to determine which parameters have the greatest effect on the outcome of a process.57 M 2. If the approximation approach above is taken.1.6).47 T 177. Sensitivity analysis calculations.The third step is to calculate nozzle exit plane pressure (Pe) from equation (4.1. (4. using A* = 2.60 A* = 1. when obtaining the sensitivity of thrust to throat diameter.5). care must be used to ensure a small enough range for the parameter values to achieve the desired accuracy. but changes the nozzle expansion ratio and exit pressure.6 Limitations It is often not easy to isolate a variable to obtain a second derivative. changing a throat diameter not only changes motor pressure.87 1.
4. unless standardized methods are employed at all stages of a project life from design to manufacture.2.3 Following is a list of feature controls used to specify how a tolerance is to be applied to a design feature. a tolerance of ± 0.5: straightness flatness circularity cylindricity profile of a line profile of a surface BXXX datum identifying letter basic dimension (XXX) angularity perpendicularity parallelism position concentricity circular runout total runout reference dimension A basic dimension is contained in a box (unless otherwise specified on the drawing).5 is an internationally recognized method of stating dimensions and tolerances. and function intended by the design engineer (or systems engineer). Basic dimensions set up a dimensional pattern.4. Standard dimensioning and tolerancing per ANSI–Y–14.2. This information is understood by draftsmen.4. from ANSI–Y–14. Basic dimensions are the controlling dimensions on a drawing.2. Some of the methods of specifying dimensions and tolerances are discussed here. Standard dimensioning and tolerancing allows the design engineer to indicate how tolerances are to be applied.g. The locations of the features in the pattern (e.. and have no tolerances associated with them. A tolerance is shown for each significant digit used in the basic dimensions on the drawing. fit.2 Standard Dimensioning and Tolerancing 4. Often the title block of a drawing will indicate standard tolerances peculiar to that drawing that will apply to all basic dimensions shown without a specified tolerance. and machinists to assure the form.1 may apply to all basic dimensions with one significant digit.1 Description Dimensioning and tolerancing on drawings is complicated enough to yield confusion. For example.3 Procedures This section explains how dimensions and tolerances are specified on design drawings. bolt holes or threads) are toleranced using trueposition tolerances. such as a bolt pattern.2 Application Standard dimensioning and tolerancing is typically applied in phase C but the technique could also be applied in phase D. 45 . 4. manufacturing engineers.
true position).005.00 0. 46 .020 tolerance zone perpendicular to the throat axis. The third (and any subsequent) section specifies which data are used (which feature or dimension the tolerances concern).005 R AB Figure 41.4 Example Following is a hypothetical fixed nozzle assembly used to show the purpose of dimension and tolerance methods: B 4.030 R 0.015 of the throat axis. 4. and the exit plane is to be within a 0.” “L.. The boss at the end of the nozzle is controlled by a total runout tolerance. The nozzle exit is referenced to datum A. thus datum A is the axis of the throat. (The “R” inside the circle indicates that the position tolerance is applied “regardless of feature size. The true position of the exit is to be within ±0. The true position tolerance is not affected by the feature size of the throat diameter. The first symbol in the second section is the type of measurement (a diametrical tolerance is shown in fig.020 0. In the example below.” thus the tolerance can be relaxed by an amount commensurate with the difference that the size of the feature is less than the maximum allowable size.” An “M” inside the circle would denote that the position tolerance applies to “maximum material condition.020 R AA A 1.Reference dimensions are the result of basic dimensions.) The exit plane also defines datum B. and the axis of the threads is to be within ±0. an inner and coincident outer diameter are specified. The second symbol in the second section (a circle with the letter “M.000 ±.25 X 12 UNF 0. Note that for the profile type tolerance controls (e..010 AB 3. The number is the size of the tolerance. A rectangular box is used as a feature control box. Example of dimensioning and tolerancing.030 of the throat axis (datum A). An “L” inside the circle would denote “least material condition” where the tolerance applies to the smallest feature size allowable. The surface is to be within a 0.g.2. datum A is defined by the throat of the nozzle. the thickness is a reference dimension. 41).010 tolerance zone perpendicular to the axis made by the throat and exit (datums A and B). the inner and outer diameters are of primary importance. The threads of the nozzle are to be concentric to the throat and exit axis within 0. In this example.” or “R”) specifies the relation of the tolerance with the size of the feature. In this situation. the thickness is of secondary importance. The symbol in the first section of the box is for the type of tolerance (e.g.000 ±0.003 1.
4.5 is fairly standard. This means that the total “width” of the acceptable deviation is defined by the tolerance. thus the axis of the nozzle exit must fall inside a circle of 0. The nozzle exit is toleranced to keep the thrust vector in line (within a certain amount) with the throat axis. it is a reference dimension.6 Limitations A moderate amount of training and practice is required to effectively use standard dimensioning and tolerancing.3 Tolerance Stackup Analysis 4. fit. The length of the nozzle is controlled by a basic dimension. or function problems. The thread is controlled by concentricity to the same axis to keep the thrust axis in line with the motor axis.3. This is facilitated by following the dimensioning and tolerancing system of ANSI–Y–14.3 Procedures Three typical methods for tolerance stackup analysis are: (1) Worstcase tolerance stackup analysis. 4. 4. 4. The boss surface tolerance is to facilitate a consistent seal with the motor. the form.010 tolerance.3. fit. Thus. If this dimension were not specified. fit. This technique is used to determine the possibility or probability of having form. the number call out is ±. fit.5 are better suited for production. and the required perpendicularity to the throat axis is greater than the true position tolerance.3.2. Thus a tolerance zone of 0. Note that the tolerances in this example control the thrust vector.2.030 radius around the throat axis. some aspects of dimensioning and tolerancing per ANSI–Y–14. For example. 47 . the number defines a tolerance zone.2 Application Tolerance stackup analysis is typically performed in phase C or D. 4. The exit is truepositioned to the basic dimension from the nozzle throat. The nozzle boss is controlled by runout to the axis defined by the throat and exit plane. For position tolerances. or function problems with a design. or function of the component would not be affected.1 Description Tolerance stackup analysis determines if a form. Tolerance stackup analysis is typically performed by either assuming worstcase allowable dimensions. true positioning allows for a circular tolerance zone. or to determine a tolerance or dimension necessary to avoid form. In addition.runout or perpendicularity). a functional part that would comply with true position tolerances may not comply with rectangular tolerances. It can be seen that the thickness of the boss is not a controlling dimension.020 is analogous to a ±0. or function problem exists when manufacturing tolerances combine in a finished part or assembly. whereas putting tolerances to rectangular coordinates allows a square tolerance zone. or by using statistical analysis of tolerances. Dimensioning strategy can minimize the cumulative tolerance stackup. used to determine size or position if all applicable dimensions occur at the worstcase extremes of the tolerance zones simultaneously.5 Advantages Dimensioning and tolerancing per ANSI–Y–14.5.
±0.3. 48 .4 Design using simulation methods.4.050)/0.5 4. For a squeeze of 0.(2) (3) Statistical analysis of tolerances.035.005/3 = 0.00167 and by summation of squares. The nominal squeeze is 0.035.050 inches.5 = 0. 42).0033)2 + (0.00167)2).010 Figure 43.035–0.4. The probability of the squeeze being less than 0.005.005 ±0.4 Example In the following hypothetical Oring joint assembly (fig. the tolerances of each component are shown in figure 43.0.010/3 = 0. The standard deviation is assumed to be one third of the tolerance on the parts (this means all parts will fall within 3 standard deviations of the nominal dimension) and is therefore: Component standard deviation = 0. system standard deviation = (2(0. Figure 42.0033 Oring standard deviation = 0. Find the maximum tolerance stackup possible to obtain the minimum squeeze.005 = –3. where a computer is used to do a Monte Carlo analysis of the possible combinations of tolerances.035 is obtained by finding the distance from the mean (in terms of standard deviations) that this condition represents. Oring joint components. Oring joint. used where the expected standard deviations of tolerances are combined to determine the probability of a final tolerance. the distance (in standard deviations) from the mean (z) is z = (0. and the probability that the squeeze will be less than 0.010 ±0.
4987) = 0. in that when many tolerances combine.Using a table for the normal distribution function.3 4. fit.030 tolerance may have a smallest unit of measurement greater than a 0. fit. a 0. in that: (1) If some tolerances are much smaller than others. This value is interpreted as 0. even if exceeding a requirement is possible. For example.0013. e. It can be extremely cumbersome to perform a statistical analysis of all the possible variations on some assemblies.4987.6 Limitations Worstcase tolerance analysis is conservative. the surfaces compressing the Oring may not be flat or normal to the direction of squeeze.5 Advantages Worstcase tolerance analysis can simply determine the envelope of possible form..4 The results can be used to determine probabilities that certain overall tolerances will exceed a critical value. Statistical tolerance analysis usually assumes a normal distribution of dimensions in the tolerance zone. care must be exercised when combining tolerances.0005 tolerance. Since this is a onesided question (no interest in the squeeze being 0.. A typical example of software is the “Variation Simulation Analysis”4. 4. machining and casting.g. which may be unrealistic. 49 .: “Managing Variation by Design Using Simulation Methods.065). Statistical analysis can show that. Consideration of significant digits may be helpful. or which tolerances are most important to form.035 or less. the area under the curve beyond z = 3 is (0. it may be extremely unlikely.7 Bibliography Craig. Inc. or function. A close look at the example above will show that more sources of variation are possible than those considered.3. so software exists to perform the statistical analysis. e. it becomes increasingly unlikely that all dimensions will be worstcase simultaneously.3.g. Also. 4. 4. (2) 4. M. and function. their inclusion in tolerance stackup analysis is superfluous. It may be superfluous to combine tolerances from different manufacturing processes. position tolerances are often determined at maximum material condition. the area under the half curve for z = ±3 is 0.3.5–0.5 that uses Monte Carlo methods to simulate the possible ways that the tolerances can stack up.” Applied Computer Solutions. thus position can vary more when not at maximum material condition.13 percent probability that the squeeze on the Oring will be 0. In addition.
345 East 47th Street. and Peterson. PA. Hill. McGrawHill. 1982.. MA.1 4.3 4. 300 Maple Park Blvd. Inc. United Engineering Center. November 1970. Jr. AddisonWesley Publishing Company.: “Mechanical Engineering Design. MI 48081.. NY 10017.R. Reading.” Edited by J. Stroudsburg.E.” ANSI–Y–14.. 410 . New York. The American Society of Mechanical Engineers.5M. “Variation Simulation Analysis Software.” Applied Computer Solutions Inc. Dowden.. Clair Shores. P.” Third edition.: “Mechanics and Dynamics of Propulsion.B. Getting Started Manual.REFERENCES 4. J. C.G. Shigley. Cruz. St.2 4.” Third edition.4 4.5 “System Sensitivity Analysis. Hutchinson and Rose. “Dimensioning and Tolerancing.
1 Description Scatter diagrams. is used typically when there is a need to know the relative importance of data or variables. This toolbox is to provide knowledge of the existence of these techniques. also called XY graphs. identify relationships. The scatter diagram. If there is a correlation between the two variables. section 5. section 5. and references for their appropriate application. 5.5. even when the connection between two variables is unexpected. or conditions that occur most frequently. monitors the performance of a process with frequent outputs. The control chart. is a bar chart that shows a dispersion of data over a specified range. Control charts are useful in trend analysis. This chart displays changes over time.2. The more that this cluster resembles a straight line. it can be assumed if the data from one are changed. The analysis can be used to monitor performance. The scatter diagram technique is best applied in phase E. section 8.7. A summary of the advantages and limitations of each tool or methodology discussed in this section is presented in table 51. The direction and compactness of the cluster of points gives a clue as to the strength of the relationship between the variables.5. One of the most common data displays is the timeline chart.1 5.4. section 7. This type of chart is commonly used in presentations to make data easier to interpret. and statistical process control. causes. 5. A histogram.2 51 .6. positive or negative. A Pareto chart.2 Application The graphic display of the scatter diagram can help one determine possible causes of problems. GRAPHICAL DATA INTERPRETATION TOOLS There are numerous excellent texts on the appropriate use of graphical data interpretation tools. No interpretation of the data should be attempted. This chart is discussed in section 5. While this section lists and briefly discusses some of the available tools. This chart will also identify the problems. makes it possible to determine if any relationship exists between two variables. The bar chart compares quantities of data to help identify distribution patterns. section 5.1. Sorting data that share a common characteristic into different groups is often accomplished with a stratification chart.3.5. section 5.1 Scatter Diagram 5.14. section 5.1. The scatter diagram displays one variable on the horizontal (X) axis and the other variable on the vertical (Y) axis. This chart is discussed in section 5. but correlations can be inferred. the stronger the correlation between the variables. plot raw data and allow the analyst to determine if any relationship exists between two variables.1. One way to analyze data is by graphical interpretation. and reveal the most important variables in a set of data. the neophyte reader is advised to read and utilize standard handbook references when using these techniques in problem solving to avoid misuse and error. then this will effect the data from the other.
(2) The chart can prevent tampering with processes that (2) The underlying causes are not determined.2 Bar chart 5. (4) Control charts can be used without extensive knowledge of statistics. data. (2) A histogram helps establish standards for a process. (2) Defect rates can be plotted on time lines in order to identify trends. thus possibly giving the appearance of a (2) The graph can help determine a possible cause(s) of correlation that is better or worse than reality. Some preplanning needs to be done before choosing categories.Table 51.7 (1) A histogram helps identify changes in a process as A histogram is not a good tool for computing process the data changes.6 (1) The pareto chart helps to identify the few areas of concern that are most important. Control chart 5. A poor pareto chart will result if the causes chosen to study are wrong. (3) The chart monitors the effects of process changes that are aimed at improvement. (2) It makes graphic comparisons of quantity easy to see. (2) The correlation does not prove a causeandeffect relationship. Stratification chart 5. (1) The timeline chart shows a “moving picture” of fluctuations over time.1 Advantages Limitations (1) The general relationship between two variables can (1) The choice of scale for the graph can distort the be determined at a glance. are under statistical control. problems by looking at correlations. Histograms 5. (1) The bar chart tells its story at a glance. 52 .3 Timeline chart 5. (2) The chart is useful in analyzing defect data. control. Graphical data interpretation tools and methodologies. Tool or Methodology Scatter diagram Section 5. A bar chart is limited in the number of data categories that can be displayed at one time. (2) All potentially important stratification variables cannot be determined without planning. capability.4 The timeline chart shows the direction of change but it gives no indication as to the reason for the change. The approach not only produces a priority ordering of the problems but also identifies an improvement strategy. (1) The control chart helps one understand the (1) The control chart tells only if the process is in capabilities of the process. Pareto chart 5.5 (1) The correct stratification variables for resolving a problem are generally not known prior to data collection.
(4) (5) 5. example shown in figure 51.1.1. thus possibly giving the appearance of a correlation that is better or worse than reality. The graph can help determine a possible cause of problems by looking at correlations. 5.6 Limitations (1) (2) The choice of scale for the graph can distort the data. a scatter diagram is prepared in the following manner: (1) (2) (3) Collect the two selected variables of each occurrence. A possible relationship can be determined by visual inspection of the graph. the one that you can have an effect on.3 Procedures As described in reference 5.4 Example As adapted from reference 5. Draw the horizontal and vertical scales with equal length.1. would show if there were any relationship between the test scores and the production levels. an aptitude test was given to 10 employees and the scores were then compared to the production levels of these employees over a certain time period. is assigned to the vertical (Y) axis. The scatter diagram. The independent variable is assigned to the horizontal (X) axis. The dependent variable.5.5 Advantages (1) (2) The general relationship between two variables can be determined at a glance. Plot each data point. Set the scale intervals and label. 5.1. Employee 1 2 3 4 5 6 7 8 9 10 Test Score 27 13 8 37 32 10 17 22 6 7 Production Level 120 80 60 150 135 70 95 105 50 55 This plot shows that the higher test scores result in higher production levels. The correlation does not prove a causeandeffect relationship.3.2. 53 .
percent defective (P).2. 5.2. The control limits show the natural change of a process. A stable process changes randomly. The upper control limits (UCL) and lower control limits (LCL) should not be confused with specification limits. Points that are outside of the control limits reveal that something has occurred that requires special attention because the points are outside of the builtin 54 . and defects per fixed unit (C).2 5. control charts are used to show the variation of several variables including average ( X ) and range (R) as well as the number of defects (PN).160 140 Production Level 120 100 80 60 40 20 0 0 5 • •• • • • • • • • 10 15 20 25 30 35 40 Test Score Figure 51. An unstable process does not change randomly.1 Description A control chart monitors the performance of a process with frequent outputs. 5. and groups of points from a stable process tend to fall within predictable bounds.2. defects per variable unit (U). Individual points of the process are unpredictable. such that points within the limits generally indicate normal and expected change. and when changes occur they are generally out of the range of normal operations. The control chart is based on four concepts: (1) (2) (3) (4) All processes change with time. The chart shows a pictorial representation of an ongoing process and determines whether or not the process is performing within acceptable parameters. Scatter diagram example.2 Application The control chart technique is best performed in phase E. As described in reference 5.2 Control Chart 5.
One point that is outside of the control limits does not mean the process is out of control but it should be explained. a control chart (fig.2 0. Plot the data on the control chart to evaluate performance and identify the points outside of the control limits.2.182 0.333 0.4 Example 0.55 0.6 0.111 0. 7.25 LCL=0.091 0.091 0. 52) is constructed in the following manner: (1) (2) (3) (4) (5) Determine the control limits to show the expected change of the process. Control charts are used in performing statistical process control (SPC) (sec.167 0. 5. The control chart can to be used continuously to determine whether the process remains within established control limits. 8.5 0.25 0.5 0.076 0. Gather data.182 0. the chart can be monitored for points that may fall outside of the limits and require causes to be identified.25 0.5 0.2 0. and improve the mean. reduce the normal variation.286 0.083 0 0 0 0 0 0 0 0 0.1 0.1 0. 55 .15 0. 5.076 5 10 15 Event 20 25 30 Figure 52.systematic cause of change in the process.125 0.2.5 UCL = 0. Determine why points are outside of the control limits. As new points are added.3 0.4 0.2 0.333 0. Find ways to identify causes of problem points.2.14) and trend analysis (sec. Control chart example.3 Procedures As described in reference 5.1 0 0 0.).
5. The control chart can prevent tampering with processes that are under statistical control. Give the bar chart a legend to identify different colors or patterns. The control chart monitors the effects of process changes that are aimed at improvement. 5. 7.2. percentage. Bar charts can show double and triple bars to compare different time periods or different populations.6 Limitations (1) (2) The control chart tells only if the process is in control.3 Bar Chart 5. Differences and similarities between and among selected categories are emphasized by the heights of the columns.5. 56 .2 Application Bar charts are one of the most common types of data display and this technique is typically performed in phase E.3. 5. or frequency of events.3.1 Description Bar charts show a comparison of quantities of data to help identify quantity changes. raw data are entered on a checklist (sec. Label the quantities on the vertical scale at the left. The quantities of data are depicted by the lengths of the bars that represent cost.5.2.5 Advantages (1) (2) (3) (4) The control chart helps the analyst understand the capabilities of the process.8).3. The control chart does not indicate the underlying cause unless data on outside processes are included in the analysis. 53) is constructed in the following manner: (1) (2) (3) (4) (5) If necessary. a bar chart (fig. Make sure the scale is broad enough to include the highest and lowest value in each category.2 5.3 Procedures As described in reference 5. Draw the bar according to the quantity of each category.2. Control charts can be used without extensive knowledge of statistics. List the categories across the horizontal scale at the bottom. The bars may be horizontal or vertical.
3.3.3. 5. Bar chart example. Limitations A bar chart is somewhat limited in the number of data categories that can be displayed at one 57 . 5. The bar chart makes graphic comparisons of quantity easy to see.5.4 Example Sale of Household Appliances 1980 versus 1990 (in millions) 10 9 8 7 6 5 4 3 2 1 0 1980 1990 (Nominal Categories) Figure 53.5 Advantages (1) (2) The bar chart tells its story at a glance.6 time.
etc. 58 . Establish time intervals (usually hours. Example A study was made comparing the average number of errors that were made per operator at different times of the day over a certain time period (fig.4.2 Application The timeline chart is a special case of XY plots where the independent variable is some time value. the quantities plotted for each successive interval.g. etc. weeks.4. 5.4.) for the horizontal axis. a timeline chart (fig.. The intervals should be evenly spaced and labeled.4.8). The chart graphically displays changes over a period of time. The chart connects data points with line segments. 54) is prepared in the following manner: (1) (2) (3) (4) (5) (6) 5.” “days.3 Procedures As described in reference 5.2 This technique is best performed in phase E. 20. 54). with line segments.1 Description The timeline chart is among the most common types of data displays. 30.5 Advantages (1) (2) The timeline shows a “moving picture” of fluctuations over time. Title the chart to define the time period for which the data are displayed. add horizontal and vertical grids. The vertical scale is a quantity while the horizontal scale is divided into time intervals such as “hours. Connect.2. 10. If the points are difficult to read.) and label the axis.4 TimeLine Chart 5. Establish the quantities for the vertical axis and make them evenly spaced (e.4. 5. days. 7.” and “weeks.5. Defect rates can be plotted on time lines in order to identify trends.4 Enter the raw data on a checklist (sec. The line segments connecting the points on the chart give a clear picture of changes over time.”5. 5.
The graph may show that one category is producing a higher defect rate than others.1 Description The term “stratification. What has been found is where the problem is occurring the most.5. The data can be depicted in graphic form for easy visual interpretation. time. 5. lot.4 59 05:00:00 PM 12 Noon .2 Application The stratification chart is best applied in phase E. For example. units. Timeline chart example. 5. shift. 5. This does not mean the “cause” of a problem has been found. You can stratify by vendor.6 5 Quantity 4 3 2 1 0 09:00:00 AM 10:00:00 AM 11:00:00 AM 01:00:00 PM 02:00:00 PM 03:00:00 PM l l l l l l l l l No.5 Stratification Chart 5. and compute a percent defective for each category (stratification variable). The cause has yet to be determined.” derived from “stratum. of Errors l 04:00:00 PM Time Intervals Figure 54. etc. Should the data not include a significant problem.5. a process is incurring a 10percent defect rate with a particular product. and machine. select other stratification variables and collect more data. or other types of strata can often lead to suggesting an improvement strategy. Some common stratification variables are shift. Comparisons of different groups.” is used in data analysis. machine.6 Limitations The timeline chart shows the direction of change but it gives no indication as to the reason for the change. Stratification is done by sorting data into different groups that share a common characteristic. operator. operator.4.5.
History of Discrepancy Reports for a Solid Rocket Motor 20 18 16 14 12 10 JOINT 8 6 4 NOZZLE 2 0 CASE IGNITER INSULATION LEGEND PROPELLANT Month Figure 55. Gather data and record the potentially important stratification variables. 510 . Analyze the data on the chosen stratification variables and compare to each other. If no conclusions are found. 5. choose different stratification variables. Determine the strategy to improve the problem.6). and histograms (sec.5.7). Separate the possible problem areas into special and common cause problems.3 Procedures As described in reference 5. 5. Stratification (histogram) chart example. 55) is performed in the following manner: (1) (2) (3) (4) (5) (6) (7) Choose the stratification variables. Pareto chart (sec. such as bar chart (sec.4.3).5. 5. the stratification process (fig. Graph the data using one of a number of different tools.
4 Example Shown in figure 55 is a histogram of discrepancy reports for a solid rocket motor (SRM).2 5. a Pareto chart (fig.6 Limitations (1) (2) The correct stratification variables for resolving a problem are generally not known prior to data collection. The Pareto chart is based on the “Pareto principle” which states that a few of the causes often account for most of the effects.5 Advantages The approach not only produces a priority ordering of the problems but also identifies areas for improvement. causes. The Pareto chart also helps to identify which problems.3 Procedures As described in reference 5. This technique is typically performed in phase E.5. This chart helps to highlight the few data or variables that may be vital. 5. Summarize the numbers of observations and calculate the percentages of each cause. Set the right vertical scale from zero to 100 percent. 511 .6 Pareto Chart 5. stratified by components.” and “why” of a suspected problem cause. causes.2).2 Application The Pareto chart can be used to examine the “how. The chart is an illustration of the data as of a specific time period.5. 56) is created in the following manner: (1) (2) (3) (4) (5) Identify the most likely causes of a problem (take from the cause/effect diagram (sec.5. All potentially important stratification variables cannot be determined without planning.” “when. 5.5 Pareto charts are used in performing problem trend analyses (sec.2. 5.5. 5.6. Make the left vertical scale the same height as the right scale and set it from zero to the number of observations. The data are arranged in descending order with the most important to the left.8). if necessary.” “what. 8. a Pareto chart is often used.6.1 Description When there is a need to know the relative importance of data or variables (problems.2)).5. use a checklist (sec. or conditions are the most important or most frequent so they can be addressed first. or conditions).5.” “where.6. 7. 7. Gather the data on causes.
corresponding to their sum. Calculate and add together the percentages of cause one and two.e. The second point. The third point is found by adding the percentage of cause three to the total of one and two. The total of all columns added together should be 100 percent. This observation is even more obvious when the heights of the bars are examined. is plotted across from the right scale directly over the second column. areas to the left of the most radical slope are the most probable problem areas. 512 . (9) The chart in figure 56 reveals the slope is more radical over the first two bars (power supply and machine calibration) and this means that the majority of the problems occur in these categories. The first point is plotted at the upper center of the first column.. (6) (7) (8) The columns are drawn using the left scale. and the last point is at the 100percent point. and plot. The plotted points are then joined with line segments. i. Pareto chart example.100 100% 80 80% 60 60% 40 40% 20 20% 0 Power Supply Machine Connection Calibration Electrical Component Feed Transformer Wrong Connection Operator Training H O Filter 2 0% Figure 56.
Fewer items fall on either side of the center. Wadsworth.” Ford Motor Company.1 5. many items tend to fall toward the center of the data distribution. D.3 Procedures As described in reference 5. Since group intervals are equal in size.5. Find the range of the data by subtracting the smallest data point from the largest. Scale the groups of data on the horizontal axis.6 Limitations A poor Pareto chart will result if the causes chosen to study are wrong. and Godfrey: “Modern Methods for Quality Control and Improvement. 5.2 Application When data are plotted on histograms.. 5.: “Defect Prevention. a histogram (fig.6.5. 1989.6.7. 5.1 Description Histograms are bar charts that show a dispersion of data over a specified range. and Montgomery.4 The histogram is best applied in phase E. The bars are proportional in height to the frequency of the group represented. S.” John Wiley. Livonia. The chart is useful in analyzing defect data. 1986. New York. Hines. The number of data bars in the graph should be limited to between 6 and 12.7.: “Probability and Statistics in Engineering and Management Science. 513 . MI. 1986.7 Bibliography Cane. the bars are of equal width. W.7 Histograms 5.W. 57) is constructed in the following manner: (1) (2) (3) (4) Gather the data to be plotted and count the total number of data points. 5.5.2. Use of Simple Statistical Tools. The width of each bar is determined by dividing the range of data by the selected number of bars.5 Advantages (1) (2) The Pareto chart helps to identify the few areas of concern that are most important.” John Wiley. New York.6. Some preplanning needs to be done before choosing categories.7. V.C. This spread of data makes presentations easier to interpret.E.
7.6 Limitations A histogram is not a good tool for computing process capability.4 Example The chart in figure 57 displays a typical histogram. Each bar.(5) (6) (7) Scale the frequency of occurrence or the numbers on the vertical scale.5 Advantages (1) (2) A histogram helps identify changes in a process as the data changes. is the same width.7.7. Plot the frequency of occurrence of the numbers in ascending order. 5. 5. including all data points. 5. Draw the height of each bar to show the number or frequency of the group interval using the scale on the vertical axis. 8 7 6 5 4 3 2 1 0 010 1020 2030 3040 4050 Time To Complete Tasks (Hours) Figure 57. A histogram helps establish standards for a process. 514 . Histogram example.
MI.2 5. Business One Irwin. 1992. 1991. Implementing the Best Ideas of the Masters.3 5.D.” DOD 5000.51–G (Draft).: “Tools of Total Quality. How to Implement a Competitive Quality Program.5 Brocka. A Guide for Implementation.” HD62.REFERENCES 5.” Ford Motor Company. V. 1989.: “Quality in America.: “Quality Management. Homewood. “Total Quality Management. 515 .E.15. IL 60430.S. Livonia. Cane. Lyonnet. February 15. V. IL 60430. Homewood. Use of Simple Statistical Tools.1 5.: “Defect Prevention.” Chapman & Hall. 1989. B.4 5.H86. P. Hunt. and Brocka.” Business One Irwin. M. An Introduction to Statistical Process Control.
the tdistribution becomes “lower and flatter” than the normal distribution. This technique is typically applied in phase D but may also be performed in phase C or E.2 Application “Studentt” analyses. 6. Determine if two sample means are equivalent to each other within a given probability of error. the tdistribution is equivalent to the standard normal distribution. In this section.1 Description As described in reference 6.” which is based on the sample mean and standard deviation. The tdistribution was described in 1908 by W. are used when sample sizes are low for the following functions: (1) (2) Determine if a sample mean is equivalent to a population mean within a given probability of error. or if the solution to a function for the probability distribution of points were available. factorial analysis. When only sample statistics are available (as is usually the case in engineering applications). the novice statistician is cautioned to read and utilize standard. analysis of variance (ANOVA). as described in reference 6. confidence analysis. While this section lists and briefly discusses some of the available tools. The tdistribution is similar to the normal distribution in that with an infinite sample size.6. In many of these analyses. there is a finite probability that they are “close” to the population statistics. the following typical statistical processes are discussed: “studentt” (t test) analysis. usually the probability is chosen and t is sought. to the tdistribution for the same sample size and a desired significance (probability of error). and response surface methodology. 61 . This toolbox does provide a suitable knowledge of the existence of these tools and references for their appropriate application. STATISTICAL TOOLS AND METHODOLOGIES There are numerous.” thus the name “studentt” analysis.1.S. However. excellent and highly detailed texts on the appropriate use of statistical techniques. the “studentt” compares the sample statistic “t.1.2. Sample statistics are made from actual measurements of a sample with a finite number of specimens.1. a tdistribution table is usually used to find t . Use solely of this text might well result in misuse and error. 6. correlation analysis. At sample sizes lower than infinity. 6. The output of the tdistribution chart is the probability (α) that t exceeds a certain t on the ordinate of the tdistribution chart. a comparison of sample statistics and population statistics will be made.1 “Studentt” Analysis 6. regression analysis. it is assumed that population statistics would be obtained if an infinite number of specimens could be measured.1 A summary of the advantages and limitations of each tool or methodology discussed in this section is presented in table 61. Gosset under the pen name “student. handbook references when using these techniques in problem solving. Here.
Sources of variation can be found. the actual line or computer. when the relationship is not obvious by generated will only approximate the actual relationship. A sample statistic must be known or assumed. when the relationship is not obvious by generated will only approximate the actual relationship. such as the population standard deviation.7 62 . and large numbers of variables can be solved.3 6. Response surface methodology 6. The fractional factorial analysis does not solve for all effects and higher order effects separately. Sources of variation can be found. A mathematical relationship can be determined. The analysis is quite simple.5 This analysis can give a realistic probability of whether or not a process may yield a value which is above or below a requirement.1 6.g.6 A mathematical relationship can be determined. interactions between variables can be isolated. random variation isolated. The processes in factorial analysis are more timeconsuming than the analysis of variance. integer data). by hand If the data are discrete (e. Limitations The parent distribution must be reasonably close to a normal distribution. inspection. or any chosen source of variability isolated. the actual line or computer. Regression analysis 6.g.2 Advantages The procedure is relatively simple to apply. A straightline relationship is assumed. by hand If the data are discrete (e. inspection. integer data). Correlation analysis Factorial analysis 6. Also.4 Confidence/reliability determination and analysis 6. before an analysis can be performed. or any chosen source of variability isolated.. random variation isolated..Table 61. Tool or Methodology “Studentt” analysis ANOVA Section 6. Statistical tools and methodologies. A full factorial analysis does not solve for exponential or polynomial effects. The processes are timeconsuming and often approximate.
Determine the t value using equation (6. If Ho had been rejected. or <34 percent. The null hypothesis will be a strain capability equal to 34 percent. if the propellant batch is representative of the nominal propellant. The nominal strain capability is 34 percent. >.10 level of significance unless the probability of a type II error is determined.1.10 significance. a twotailed analysis will be done.. the null hypothesis is disproved. Determine with a 0. Develop null and alternate hypotheses for the problem being investigated. i.1. The sample mean is 33 and the sample standard deviation is 2. The alternate hypothesis is that the sample mean is on the particular side of the population mean.1). If it is desired to prove that the sample mean is not on either side of the population mean.1.4 Example Pull tests of a propellant sample yielded the following strains before failure: 29. it could be stated that there was only one chance in ten that the null hypothesis was rejected when it should not have been. If it is desired to prove that the sample mean is on one particular side of the population mean. This is referred to as a type I error. 31. t = 2. t = 0.1) Compare t with tα for the desired significance and degreesoffreedom (DOF) (n–1). Thus.1539. if t is greater than t( /2) (or t is less than – t( /2)). If t is greater than t . but the two alternate hypotheses would be that the sample mean is above or below the population mean. For a twotailed analysis. and 36 percent.05 significance).e. This latter situation would use a “twotailed” analysis. it could be stated that the null hypothesis could not be rejected at the 0.1). it cannot be assumed with the chosen confidence that the sample mean is equivalent to the target mean.6. 6.915. 34. the null hypothesis is disproved.6. the null hypothesis is that the sample and population mean are equal. t = sample mean – target mean sample (5) /( n) 1/2 (3) (4) (6.3 Procedures The use of a ttest for determining if a sample mean is equal to a chosen population mean will be shown here. Substituting into equation (6. Since the mean of the propellant batch could be =. From the tdistribution table for 4 DOF. 63 . the null hypothesis would be the same. (1) (2) Determine the target mean and significance level desired. If Ho were not rejected.1 6. 35.134. Determine the mean and standard deviation of the sample.5 Advantages The procedure is relatively simple to apply. α/2 will be used (0. The determination of the probability of a type II error is complicated and many texts consider it beyond their scope.
2)) is compared to F . E. Determining the significance of each factor. A factor F (equation (6. Handbook 91.” U.2. NOTS 948.” NAVORD Report 3369. Belmont.5) to compare sample statistics.1.6 Limitations The distribution of the parent population must be reasonably close to a normal distribution. F. 6.S.: “Statistics Manual.1. based on the number of samples (k) and the sample size (n). a value that is related to the total DOF. 6.2. and Maxfield.2. Mendenhall. betweensample variance mean ofwithinsample variance F= (6.3 Determining if two or more processes are producing products that are consistent with each other.2 Analysis of Variance 6.1. Procedures As described in reference 6. W.. 5. Eliminate one source of variation to determine the effect of the others. Wadsworth Publishing Company.A. Department of Commerce.W.” Fourth edition. Davis. Determine which two or more processes are different if a difference in processes is detected. 6. M.L. National Bureau of Standards. Some of the uses for analysis of variance are: (1) (2) (3) (4) 6.1 Description ANOVA is a technique used in design of experiments (sec.6. to determine if the variation of the mean and variance between two or more populations are attributable to sources other than random variation. CA 94002.7 Bibliography Crow. 1975. If only one source of variation is being investigated. a one way classification is used.: “Introduction to Probability and Statistics. “Experimental Statistics. the following methods can be used to determine if the withinsample variation is greater than the sampletosample variation.2 Application The ANOVA technique is typically performed in phase D but may also be performed in phase C or E. to determine if two or more samples have different sample statistics.1 6.2) 64 .
and the DOF of the sample size is n–1. and one data point from each sample representing the other source of variation is put into each block (see example below). 65 . Rather than determining sample statistics for each sample.4) MS(bs2) SS(bs2)/ b − 1 = MSE SSE / K(a − 1)(b − 1) (6. the following approximation equations can be used: MS(bs1) SS(bs1)/ a − 1 = MSE SSE / K(a − 1)(b − 1) F(bs1) = and F(bs2) = where SSE = SST – SS(bs1) – SS(bs2). If two sources are being investigated. If F exceeds F . F= SS(bs)/ (k − 1) . approximation formulas that use sums and averages of squares. F is found from an F distribution table. SST and SS(bs) can be found using the formulas SST = ∑ (yij)2 – C. The total DOF is k*(n–1). Data can be arranged in blocks representing one source of variation. then a difference exists between the samples that is not only due to random variation. MS = Mean square SS(bs1) = ∑(Ti )2/b – C. can be used. T = total of all data points. and where a = the number of samples of one source of variation and b = the number of samples of the other source of variation.3) where SS(bs) is the sum of squares (betweensample) and SSE is the sum of squares error. C = T2/(k*n). and Ti = total for each sample.The DOF of the number of samples is k–1. SSE / k(n − 1) (6. a twoway classification is used. (6. SS(bs) = ∑ (Ti )2/n – C.5) SST = ∑(yij)2 – C. The SSE is determined from the sum of squares total (SST) and SS(bs) by the formula SSE = SST – SS(bs). If two sources of variation are being investigated. MSE = Mean square error SS(bs2) = ∑(Tj )2/a – C. and C = T2/(k*n) where y ij = each data point.
SS(bs1) = (642 + 682 + 642)/2 – 6402. The latin square method eliminates three sources.67. Six batches of propellant are mixed and tested with the following average results: Polymer 1 1 2 2 3 3 Curative 1 2 1 2 1 2 Percent Strain 30 34 32 36 31 33 The following table is arranged with parameter A set up in columns and parameter B set up in rows: Curative Lot 1 30 32 31 93 Curative Lot 2 34 36 33 103 Total for Polymer 64 68 64 196 Polymer lot 1 Polymer lot 2 Polymer lot 3 Total for curative here C = (196)2/6 = 6402.88.67/1 = 16. SS(bs2) = (932 + 1032)/3 – 6402.67 = 24.33/2 = 2.67 = 23.2. and the GrecoLatin method eliminates four sources.67/0.33/((3–1)(21)) = 0. These methods must use n2 observations. MS(bs2) = 16.67.33.0. SST = 302 + 342 + 322 + 362 + 312 + 332 – 6402. Analysis of covariance is a similar technique used when conditions (such as environmental) change. and F(sb2) = 16. 66 .67 = 16. The use of three lots of polymer (parameter A) and two lots of curative (parameter B) will be investigated. F(sb1) = 2.67/0.67 = 6426 – 6402.Other methods exist to isolate more sources of variability simultaneously. 6. MS(bs1) = 5.67.67.67.67 = 4.33. The effect of this change is accounted for by using regression.67 = 6408 – 6402. This involves partitioning a total sum of products rather than squares.4 Example In the following hypothetical example. the effect of two parameters on the variability of strain capability of a solid propellant will be investigated. MSE = 1.67 = 5.
strain capability is affected by the curative lot. If r is close to 0.3.6. as found in reference 6. 6.3 Correlation Analysis 6. 6.05 significance is greater than 5.3. random variation isolated. (3) Determine the significance of the r value by using the following equation: z= (n − 3) (1+ r) 1n 2 (1− r) (6.7) 67 .2 Application Correlation analysis can be used to determine if a relationship exists between two independent sets of variables. The value of r will be between –1 and 1.6 Limitations The processes are timeconsuming and often approximate. 6.2. Since F(sb1) for a 0.2.14). 6. for determining if two sets of data are linearly related is as follows: (1) (2) Determine the mean of each set of data.1 Description Correlation analysis measures the strength of a linear relationship between two sets of data. This technique is best performed in phase D but may also be performed in phase C or E.99.5 Advantages Sources of variation can be found.6) where x and y are the means of the first and second set of data respectively. Since F(sb1) is less than F for a 0.3. and b = 2 is the number of sources of variation of parameter B.3 Procedures The procedures.Note that a = 3 is the number of sources of variation of parameter A (polymer). if r is close to 1 (or –1) then a high degree of correlation is implied. Determine the r value of the two sets of data using the following equation: r= Σ ( xi − x ) ( Σ(x i − x )(yi − y ) 2 1/2 ) Σ(( yi − y ) ) 2 1/2 (6.3 6. polymer has no effect on strain capability. then no correlation is implied. or any chosen source of variability isolated.05 significance (F = 5.3.
but only at two levels for each variable.14 0. 68 . and 2.1 Description There are three types of factorial analysis described in this section—factorial analysis.6 dx –1. Thus.1024 0.. two to four).2 Application Factorial analysis is used for applications similar to those for which ANOVA is used.1156 0.8 1.6 Limitations A straightline relationship is assumed. the number would be 64.12 –0. 6. 7.0 1.5.0448 –0. full factorial analysis. 6.3.256 0. For six variables.0768 –0. z is –3. the r value is 0. 2. and squares of the deviations from the means are shown in the following. For example.0016 0.25 0. respectively .3 2. Using this value for n = 5. and fractional factorial analysis.02 0.3528 –0.1.1 6.2. The deviations.292 Using equation (6.4 Factorial Analysis 6.1296 0.0 7.16 –0.3808 –0. except that factorial analysis deals with levels of variables. however.5 Advantages This analysis is simple to apply.3.04 –0.6. and this is without replication. Factorial analysis is similar to ANOVA in that the analysis is based on sums of squares.8.5 7.3.5.548 dy2 0.4 6. 6. fractional factorial analysis is often economically necessary.96.2 6. 6. 6.4.1 2. and 7.6.9604 2. 6.g.36 dx * dy –0. 1.1 This technique is typically performed in phase C but may also be performed in phase D or E .98 dy 0.938. if five variables are being investigated.5 summations y 2. The mean of the two sets are 6.856 dx2 1. thus there is less than a 0. products. Fractional factorial analysis is used when so many variables are being investigated that experimenting with them is unfeasible.4. factorial analysis further subdivides the treatment of sums of squares into components and can show interaction effects between parameters. 25 or 32 experiments would have to be performed.34 0.6.01 percent chance of these two data sets not being related.0196 0. x 5.2304 0.992. 2.52 and 1.3. and 1.4 Example The following hypothetical sets of measurements were taken: 5.6).32 –0. Factorial analysis is used with a small number of variables (e. Full factorial analysis is performed for more variables.48 0.0004 0.4.0008 –0.(4) Look up the z value in a standard normal distribution table to determine the probability of having a correlation.
c.C1 A1.B1. 2.B0. Add a row for the error sum of squares and error mean square. and to account for experimental variability. B1. Here n will be the number of factors rather than the sample size (which is effectively two). With factorial analysis. determined as in ANOVA. The procedure for performing a full factorial analysis will be discussed here. Column 3 is constructed the same way from column 2.C0 total A0. A0. Factorial analysis factors and magnitudes. 2:1 means the first element of column 2.6.C1 M1 M1 M1 M1 M1 M1 M1 M1 M2 M2 M2 M2 M2 M2 M2 M2 M3 M3 M3 M3 M3 M3 M3 M3 total A0.B0.3 Procedures As described in reference 6. B0. factorial analysis is performed the same as analysis of variance except that an analysis of variance is performed for each variable against each other variable.C1 total A0. (1) Arrange the factors and magnitudes in a table such as the following: Table 62. With factorial analysis. Add the third and fourth totals in the totals column to get the second element in column 1.C1 total A1. Column 2 is constructed the same way from column 1 as column 1 was constructed from the totals column. B1. r replications will be performed. The notation in column n (3) and the sum of squares column is shortened.B1. assuming straight line relationships. b. 2n factorial experiments will be performed.B1. Column 3 is the effect totals as in analysis of variation.C1 etc. Obtain the fifth through eighth elements in column 1 the same way except that the totals are subtracted (first value subtracted from the second). B1.C0 total A0.C0 A0. Continue in a like manner to get the third and fourth elements in column 1. B0. where the first column represents the experimental conditions and M1.C0 A1. B0. B1. and M3 represent the resulting magnitudes after the experiment for replication 1. and 3.C0 A1.C1 A1.C1 A0. Add the first two totals in the totals column to get the first element in column 1. Add n (3) columns in the place of the middle columns and three columns to the right side of the table (table 63).1. B0.C1 total A1. The following is the procedure for using the factorial analysis where n = 3.B1.C0 total A1. certain computational shortcuts can be applied when only two levels of each variable are used.4.B0.B0.C0 total A1. M2. The last column is the total of all replications of experiments for each experimental condition. 69 . Apply the method of Yates to this table as follows: a.C0 A0. (2) (3) Obtain a table of effects totals by removing the middle columns in the above table.
C0 t2 A0. if a five variable investigation (32 experiments) is split into 1/4. B0. mix speed. B0. Exp. The DOF will be 1 for each effect. B1. B1.Table 63.C0 t4 A0.4 Example The following are the results of a hypothetical experiment to determine if mix time. B1. 6. DOF. (6) (7) A fractional factorial analysis is performed the same way as the full factorial analysis except the analysis is split into fractions of (1/2)p. but will be n–1 for the error row.C0 t3 A1.4.C1 t8 summation 1 t1 + t2 t3 + t4 t5 + t6 t7 + t8 t2 – t1 t4 – t3 t6 – t5 t8 – t7 2 (t1 + t2) + (t3 + t4) (t5 + t6) + (t7 + t8) (t2 – t1) + (t4 – t3) (t6 – t5) + (t8 – t7) (t3 + t4) – (t1 + t2) (t7 + t8) – (t5 + t6) (t4 – t3) – (t2 – t1) (t8 – t7) – (t6 – t5) 3 2:1 + 2:2 2:3 + 2:4 2:5 + 2:6 2:7 + 2:8 2:2 – 2:1 2:4 – 2:3 2:6 – 2:5 2:8 – 2:7 Sum of Squares 3:1/(r2n) 3:1/(r2n) 3:1/(r2n) 3:1/(r2n) 3:1/(r2n) 3:1/(r2n) 3:1/(r2n) 3:1/(r2n) SSE Mean of Squares SS1/DOF SS2/DOF SS3/DOF SS4/DOF SS5/DOF SS6/DOF SS7/DOF SS8/DOF SME F MS1/SME MS2/SME MS3/SME MS4/SME MS5/SME MS6/SME MS7/SME MS8/SME 1 2 3 4 5 6 7 8 To find: 2:1 + 2:2 = (t1 + t2) + (t3 + t4) + (t5 + t6) + (t7 + t8) 2:3 + 2:4 = (t2 – t1) + (t4 – t3) + (t6 – t5) + (t8 – t7) 2:2 – 2:1 = (t5 + t6) + (t7 + t8) – (t1 + t2) + (t3 + t4) 2:4 – 2:3 = (t6 – t5) + (t8 – t7) – (t2 – t1) + (t4 – t3) (4) (5) The sum of squares column is generated by dividing the square of each adjacent element in column 3 by r * 2n. and mix vacuum affects the burn rate of a propellant. Thus. Totals from Condition Above A0. The mean of squares column is generated by dividing each adjacent element in the sum of squares column by its respective DOF.C0 t1 A1.C1 t5 A1. Obtain each F by dividing each mean square by the error mean square. B0.C1 t6 A0.2 atm 610 . B1. Two levels of each parameter were tested as follows: Effect A B C Parameter mix time mix speed vacuum Low (0) 2 hr 1 rps no vacuum High (1) 3 hr 2 rps 0. If any F exceeds F . that effect is significant. Factorial analysis example. the number of experiments will be 2n–p (eight) experiments.C1 t7 A1. B0. Compare each F to F for n–1.
51 0. one for the sum of squares.52 Rep 3 0.000017 0.55 0.02 0.06 Sum of Squares 6.10) (6.52 0.g.52 0.000417 0.00667 0 0.9) 611 .47 0.003615 0.8) (6.02 –0.55 1.06 0.46 0.84 6.55 1.02 0.50 0.55 1.8616 0.04 0.000017 0.48 0.48 1.0803 0. 1 rps was assigned as low.138 The correction term (C) is as follows: (Sum of totals )2 C = .00667 0 0.01 0 –0.02 3 12.04 –0.50 0.54 Total 1.0745 0. The high and low levels are designated as 0 and 1. Condition A0 B0 C0 A1 B0 C0 A0 B1 C0 A1 B1 C0 A0 B0 C1 A1 B0 C1 A0 B1 C1 A1 B1 C1 Rep 1 0. The sum of squares treatment (SSTr) is as follows: SSTr = [(Sum of each individual total squared)/Number of effects] – C.51 0.58 1.Each effect was assigned a high and low level (e. respectively.56 The table is repeated with the replication columns deleted and replaced with the application of three columns for the Method of Yates.77 0 0.10 0.02 –0.46 1.48 1. Condition A0 B0 C0 A1 B0 C0 A0 B1 C0 A1 B1 C0 A0 B0 C1 A1 B0 C1 A0 B1 C1 A1 B1 C1 Replicates error Total 1.50 Rep 2 0.95 3. Exp.000224 DOF 1 1 1 1 1 1 1 1 2 7 F 0. Three additional columns are added.47 0.04 0.04 0 –0.00723 0.08 –0.56 SSR SSE 1 2.47 1.54 0. 2 rps was assigned as high).2977 1.43 1.50 0.14 –0. (Number of effects ) (Number of totals ) The SST is as follows: SST = Sum of each individual replication squared – C.03 0.52 0.52 0.49 0.000067 0. one for the mean square.48 0.24 –0.00015 0.00157 Mean of Squares 6.46 0.000417 0.51 0.00015 0.02 SMR SME 2 5.10 3. and one for the F value for each effect.55 1. (6.0745 29.000017 0. Each experimental condition was repeated three times with the following results: Exp.000067 0.0803 0.89 2.47 0.46 1.000017 0.58 1.47 1.669 16..54 0.43 1.
4. The sum of mean replicate (SMR) is as follows: SMR = SSR/DOF.The sum of squares replication (SSR) is as follows: SSR = [(Sum of vertical replication total squared)/Number of rows] – C. (The third batch of propellant may have been different for another reason such as contamination. or reliability measurements such as mean time between failures. means. with a chosen probability of being within that interval.59. (For example. 6.05 confidence is 5. The sum of mean error (SME) is as follows: SME = SSE/DOF.5 Confidence/Reliability Determination and Analysis 6.5. means. Also. A full factorial analysis does not solve for exponential or polynomial effects. therefore effect C (vacuum) and replication have a significant effect on the burn rate.5. then the interaction of mix time and mix speed would have a significant interaction). (6.2 Application Confidence analysis is used to determine the interval of values that a data point could take. and larger numbers of variables can be solved for. Confidence analysis can be used with individual points. standard deviations. 6. The sum of squares error (SSE) is as follows: SSE = SST – SSTr – SSR.13) (6.4.12) (6. 6. The fractional factorial analysis does not solve for all effects and higher order effects separately. if the fourth line had an F greater than F .1 Description Confidence analysis compares sample values. then no interactions have a significant effect on burn rate. 6.6.) Note that since no values of F are greater than F for any conditions where two or more effects are 1. interactions between variables can be isolated. with a chosen significance.1 This technique is typically performed in phase C or E. or standard deviations with population standard deviations to obtain a confidence interval.6 Limitations The processes in factor analysis are more timeconsuming than the analysis of variance. 612 .11) F for a 0.14) (6. random variation isolated.5 Advantages Sources of variation can be found. regression lines. or any chosen source of variability isolated.
The confidence interval for the population standard deviation. given the sample standard deviation. sp 1 ± z( /2) / ss / (2* n)1/2 (6. the sample standard deviation can be used instead of the population standard deviation. For large n. Obtain the z(α/2) value by looking up the z value for α/2 in a normal distribution table. For linear regression. from past experience (or by adjusting the sample standard deviation).16) is used. Determine.6.16) where ss is the sample standard deviation. will be discussed here. The values for either end of the confidence interval is given by the equation: Int = ms ± z(α/2) * sp/n1/2 where Int is the low or high confidence interval value. (1) (2) (3) (4) Choose a confidence (α) level and obtain the α/2 term by dividing the confidence level by 2.3 Procedures As described in reference 6. i 613 . sp is the population standard deviation. the population standard deviation. is determined in the same way as above.15) Int = (6. except equation (6.18) where Sxx = n * Σxi2 − (Σxi ) 2 . the confidence for the equation of the line is: Int = (a + bx o ) ± t /2 * se *(1/ n + n(xo − ms )2 / Sxx )1/2 and for the y value: Int = (a + bx o ) ± t /2 * se *(1 + 1/ n + n(x o − ms )2 / Sxx )1/2 where se = 1/ (n − 2)Σ(yi − (a 2 2 2 = Sxx *Syy −(Sxy ) + bxi)) n(n−2)Sxx (6. the procedures for determining the confidence interval for the population mean.17) (6. and n is the sample size.5.1. m s is the sample mean. and Sxy = n * Σxi yi − ( Σx i ) * ( ΣYi ) .Syy = n * Σy2 − (Σyi )2 . given a sample mean.
182+3(0. Entering the above values into equation (6.6.1 So.15).5. polynomial.96 * 0.6 Regression Analysis 6.1 3. A goodness of fit test is often performed to see how well the generated relationship fits the data.95)/2 = 0.5.6 Limitations A sample statistic must be known or assumed. 6.042 to 1.1 Description Regression analysis is a form of curve fitting to find a mathematical relationship for a group of data. the confidence interval is 1.0 minimum value.207)+0.025.4 Example Determine the confidence interval for insulation erosion at a station in the RSRM aft dome to determine if the associated compliance safety factor (CSF) may actually fall below the 1.112 ± 1.5 Advantages This analysis can give a realistic probability of whether or not a process may yield a value which is above or below a requirement. with a 95 percent confidence. There are typically two types of regression: regression and multiple regression. The safety factor is then calculated using the maximum erosion value and is: CSF = CSF = Min Ins t Erosion+3sp+0.6. The equation for the method of least squares is obtained by setting the derivative equal to zero of the equation for the sum of the vertical distance from each y value to the mean y value.207 in (defined as known s p) n α/2 is (1–0. such as the population standard deviation.766 . before an analysis can be performed. in this instance the confidence interval is used to calculate a safety value that can be compared to a performance requirement.5.112 in 0.6.36 = 1. 614 . The sample data for 18 flights (36 motors) is: Erosion mean Standard deviation 36 1.96.182 for erosion.207/(36)1/2 = from 1. Typical types of relationships which are assumed for regression include linear (straight line). and exponential. 1.3 The method of least squares is probably the most frequently used method of regression. therefore the z(α/2) term is 1. 6. 6.
b= (4) Σ(xi − x )(yi − y ) Σ(xi − x )2 (6. is as follows: (1) (2) (3) Determine the mean of the xi values ( x ) and yi values ( y ). (6.6.1.19) Determine the y intercept by subtracting the product of the slope and the mean x value from the mean y value (equation (6.20)). Determine the slope of the trend line by dividing the summation of the multiple of the deviations by the summation of the square of the x deviations (equation (6. Regression analysis is best applied in phase D but may also be applied in phase E.20) (6.21) The intercept and slope are used in equation (6. The least squares method is a commonly used method of regression. If the slope (b) is negative. 6. as described in reference 6.7. and will be discussed here (assuming a straightline relationship). Determine the deviation of each xi and yi value. the use of the least squares method for finding the equation of a line of the form y = a + bx. To determine if the magnitude of a measurement is increasing or decreasing with time or event. then a decreasing trend may be indicated.3 Procedures As described in reference 6. The explanatory power can be determined by R 2 as follows: (1) (2) Determine y values for each x value using the line generated above. The R 2 indicates the percent variation in the dependent variable that can be explained by the independent variable.3.19)). Determine the deviation of each generated y value from the mean y.6. is typically used for three purposes: (1) (2) To find the mathematical relationship represented by a group of data points. a = y – (b) x . There are several methods of regression.19) for a line representing the straightline relationship.6.2 Application Regression. 615 . Multiple regression will be discussed in section 6.
(6.4). R2 = 0. the slope (b) is 0.0004 0. Using these data.533.4).09 8.0196 0.25 6.7.59 4.5).55 –0. the mean x value is 5.(3) Obtain the R2 value by dividing the sum of the square of the generated y deviations by the sum of the square of the actual y deviations (equation (6.5). (7.05 3.6). yg = generated points for each x. assume the set of ordered pairs (1.0817 summation where dx = xi – x .0121 0. (5.15 –1.0303(x) + 4.25 2.6.49 0.05 –3.25 12.09 1.0064 0.69 0.1 (dx)(dy) 3. and products that go into the equations above: x 1 2 3 4 5 6 7 8 9 10 55 y 4 5 6 3 5 5 4 6 4 5 47 (dx)2 20.84 dyg2 0.81 4. Figure 61 shows the points and the generated line for this data.0101.75 4.5.62 4. (2.35 2.25 0. squares. 10 9 8 7 6 5 4 3 2 1 0 0 1 2 3 4 5 6 generated line 7 8 9 10 Figure 61.3). The equation for the line is y = 0. R2 = Σ(gen yi − y )2 Σ(yi − y )2 (6.09 0.22) A good relationship is indicated by an R2 value close to 1.69 2.25 12.5 (dy)2 0.0303. (9. and (10.5).45 1. dy = yi – y . 616 .0064 0.15 –1.25 20.0025 0.0196 0. No significant relationship is indicated for this example. the mean y value is 4.78 4.68 4.25 6.25 82.56 4.533.65 4.0121 0.5).09 0. and dyg = yg – y .25 0. (4. Line generated with least squares method.6).25 2. 6. The following table shows summations. (3.25 2.89 0.4).0001 0. and the y intercept (a) is 4.49 0.21)).0025 0. (8.4 Example As adapted from reference 6.15 0.25 –2.71 4.50 yg 4.3.49 1.
or it can be a more complex surface. ∑(x1 * y) = b0 * ∑x1 + b1 * ∑x12 + b2 * ∑(x1 * x2). (6.5 Advantages A mathematical relationship can be determined. Factorial experimentation is discussed in section 6.7.g. the equations for two independent variables are: ∑y = nb0 + b1 * ∑x 1 + b2 * ∑x2. the least squares method of multiple regression.6.6 Limitations If the data are discrete. This equation is minimized. using polynomial relationships. + b nxn))2 to zero. + b nxn.. 6.4.6. and b2. 6. This methodology is best performed in phase D or E..7.1 Description Response surface methodology is a method for surface fitting.. The surface can be a plane. when the relationship is not obvious by inspection.6.3 Procedures As described in reference 6. These equations are solved simultaneously for b0 .2 Application Response surface analysis is typically used for the following purposes: (1) (2) To find the mathematical relationship represented by a group of data points. integer data.7 Response Surface Methodology 6.6..23) 617 .7. b1. will be shown here.3. 6. much like regression is a method for curve fitting. assuming a straightline relationship. The basic form of the equation for a plane surface is y = a + b1x1 + b2x2 + b3x3 +. After setting the derivative of the equation for the sum of the vertical distances or ∑ (yi – (a + b1x1 + b2x2 + b3x3 +.1 6. There are two typically used methods for response surface analysis—multiple regression and factorial experimentation. ∑(x2 * y) = b0 * ∑x2 + b1 * ∑(x1 * x2) + b2 * ∑x22. by hand or computer. using two independent variables and straightline relationships. e.. To optimize independent variables for maximum or minimum results. the actual line generated will only approximate the actual relationship.
b1 = –0. x22. x1y.23).520 x2y 360 352 347 1.23).000 48.25.400 19.700 41. x1 100 120 140 100 120 140 100 120 140 1.400 19.000 14. 080 57 37. and 140° for 1. This practice makes solving for the coefficients much easier with very little cost in accuracy. 6.840 x1y 36.080 + b2 57 377.151 = 9b0 + b1 1.080 132.600 10.020 377. propellant was aged at 100°. a set of numbers is coded.7520 132.840 19.000 x22 1 1 1 36 36 36 144 144 144 543 x1x2 100 120 140 600 720 840 1. Solving the simultaneous equations (6.000 42.116 19.188 4.Often.300 35.600 10.7.080 57 1.000 + b2 6.840 543 b0 = 9 1. Mean modulus of elasticity measurements are given for three propellantaging temperatures and times. as adapted from reference 6.000 6.151 x12 10.6117.800 42.845 The equations for finding the constants are as follows: From equation (6. and b2 = –0. and x2y and the bottom row of summations are derived from the first two columns. Therefore the equation for modulus of elasticity for the sample propellant is y = 383.845 = b0 57 + b1 6.151 1.840 19.25 * x1 – 0. and 12 mo.840 543 b1 and b2 are calculated in the same manner.240 48.200 1. 3. 6.080 + b1 132.98.080 x2 1 1 1 6 6 6 12 12 12 57 y 360 352 347 358 350 345 347 349 343 3. 618 .4 Example In the following hypothetical example. if the numbers are equally spaced.284 4.840 57 6.600 132. For example.6117 * x2.548 2.000 14. x1x2. The columns for x12. 120°.98 – 0.400 19.440 1.840 + b2 543 3. the numbers are substituted by assuming a smaller whole number for each original number.000 14.880 48.100 2.3. the constants are b0 = 383.520 = b0 1.580 35.680 6. 000 6.845 6.070 4.
when the relationship is not obvious by inspection. by hand or computer.6 Limitations If the data are discrete (e. 6.7.7. integer data). the actual line generated will only approximate the actual relationship.g.6. 619 ..5 Advantages A mathematical relationship can be determined.
620 . W.C. Inc.W. J. D. “Trend Analysis Techniques..” John Wiley and Sons.: “Probability and Statistics for Engineers. Inc.2 6.” Second edition. 1977. I.E.” October 1990.. Hines. and Freund.REFERENCES 6. NJ 07632. and Montgomery.1 6.3 Miller.: “Probability and Statistics in Engineering and Management Science. 1990.5A. Englewood Cliffs. NASA–STD–8070. Prentice Hall.
3. TQM is applied to continuously improve performance at all levels of operation. Another tool that might apply to the group consensus technique is the force field analysis. Finally. This approach is discussed in section 7. discussed in section 7. design of experiments (DOE).6. Quality loss function. respectively.9. and this tool is discussed in section 7.10. TQM is an ongoing effort that demands commitment and discipline. using all available human and capital resources. flowchart analysis and work flow analysis (WFA). the product can be benchmarked against the competition in the areas of how well the product stacks up against the competition as far as handling the identified problems. 71 . These techniques are discussed in sections 7. Design of experiments varies all possible combinations of factors and levels in an attempt to obtain the optimum settings for a desired output. the WFA. SPC. This technique is discussed in section 7. A summary of the advantages and limitations of each tool or methodology discussed in this section is presented in table 71. quality. and how well the product stacks up against the competition as far as meeting the appropriate engineering standards. discussed in section 7. market share. and nominal group technique (NGT). A methodology for improving quality by looking at the production process is the evolutionary operation technique. is a method of determining “loss to society” when a product is not at the mean but is still within specification limits. Delphi. The flowchart analysis. priorities are given to the possible solutions as they relate to the identified problems. The cause and effect technique relates identified problems to their causes. A tool to assess an operation against other operations is the benchmarking technique which is discussed in section 7. 7. This methodology counts the positive and negative forces. schedule and growth. TOTAL QUALITY MANAGEMENT TOOLS This section describes several TQM tools available to the system engineer analyst. is a process improvement tool that helps identify problems quickly and accurately.5. that effect the results of a proposed solution or change in process.2. discussed in section 7. and it is discussed in section 7. and evolutionary operation (EVOP). Concurrent engineering is more of an approach to quality management than a technique and it is an interaction of disciplines during the design but before production.11. This tool is discussed in section 7. and 7. as well as their magnitudes. A methodology that is applied early in a design process is the quality function deployment (QFD) technique which is discussed in section 7.7. By using a chart known as the house of quality. in all areas of an organization. Three such tools are brainstorming.14. SPC. The force field analysis is discussed in section 7.12.8. discussed in section 7. Improvement is addressed toward such areas as cost. pictorially represents the steps of a process thus making it easier to eliminate nonvalued steps of the process. Three tools that attempt to improve the quality program are the cost of quality. Also.13.16. A methodology for collecting data quickly and easily in a simplified manner is the checklist technique. This technique is used to solve problems before the production phase begins and thus assists in the design of competitive products. These tools are quality loss function.7.4.1. Group consensus techniques are often applied to solve problems. examines the work process for possible improvements in performance and the quality of work life. This technique is discussed in section 7. The final four tools that are discussed in this section are applied to improve a process. The cost of quality tracks a quality program and attempts to identify ways to improve the program.15.
(2) Cost of quality 7. (2) The DOE technique is often performed without a “verification experiment” in which the predicted “optimized” parameters are tested for performance (in agreement with the predictions). 72 . the results generally do not include parameter interactions. While time and money are saved overall within the effort. The technique optimizes product and process design. (2) Activities and processes that need improvement can be prioritized. Preknowledge of interaction significance is required to support appropriate DOE technique selection. is required at the front end of a program to perform the coordinated planning. Helps identify and maintain awareness of industry best practices.3 (1) Advantages Helps meet customer requirements. (2) Determining industry “best practices” is difficult and often subjective enough to be biased by the reviewing company’s “wants” rather than the reviewing company’s customer’s wants. TQM tools and methodologies—Continued Tool or Methodology Benchmarking Section 7. In. Reduces costs in the designtodevelopment life cycle. Helps determine true measures of productivity. the institutional knowledge of the organization becomes very difficult to capture or employ in the design decisions. stabilizes production processes. (3) Parameters must be interpolated from within the tested data set rather than extrapolated beyond it. (1) The performance of the analysis is time consuming and. Useful in analyzing statistical process control (SPC) problems.1 (1) (2) (3) (4) (5) Cause and effect diagrams 7. Shortens and makes more efficient the designtodevelopment life cycle by employing the interactions of functional disciplines by a crossfunctional team.5 (1) Helps to reveal and explain the more significant costs. Limitations (1) Must be continuous in order to keep up with the latest industry changes. Helps to attain and maintain a competitive position. (3) Helps to reveal and explain the hidden costs of a product or service. Enables quality analysis groups to thoroughly examine all possible causes or categories.Table 71.4 Design of experiments 7. SPC detects problems but can poses no solutions. (2) Significant additional time. (1) The technique itself can be expensive. and desensitizes production variables. Helps establish goals and priorities. this technique is subject to misuse in this regard. it is often difficult to “frontload” large tasks. Arriving at a group consensus is timeconsuming. reduces costs.2 (1) (2) Concurrent engineering 7. (3) If design is pursued by projectized teams. (1) The degree of success of this technique depends upon the degree of cooperation between the multifunctional team members. thus making its goals of saving/eliminating costs unachievable. and associated funding. addition a mistake is often made by taking the “best” experiment’s parameters as an optimized set rather than an interpolated set. (2) Measurement for measurement's sake is an easy paradigm to fall into.
EVOP is slow. one.9 Nominal group technique Force field analysis Quality function deployment 7. a short time. (3) Inputs from experts unavailable for a single meeting are included. The technique takes advantage of the ideas of a group to (1) The technique only proposes a solution but does not determine arrive at a quick consensus. TQM tools and methodologies—Continued Tool or Methodology Evolutionary operation Section 7. (1) Useful in eliminating personality clashes. Useful in determining which proposed solution. (3) Engineering changes. and startup costs are reduced.7 Checklists Delphi technique 7. so it can be run continuously. (1) Helps organizations design more competitive. design cycle. (8) Easy to learn. among The technique is time consuming in arriving at a consensus on the many. values (weights) of the forces.Table 71. (6) Prevents problems from “falling through the crack.6 Advantages Limitations (1) The cost is very low. dominate the discussion. (2) The technique is limited by the ability of the group to achieve consensus.8 7. (5) The technique is proactive. not reactive. (2) Helps ensure quality products and processes by detecting and solving problems early. Very effective in producing many new ideas/solutions in (1) Assembling the group participants is difficult/timeconsuming. be collected. (1) The tool is quick and easy to use. and is highly subjective.” (7) The technique is costeffective. products easier and (2) The technique is not easy to perform. (4) Voice of the customer is heard. (1) Assembling the group participants is difficult/timeconsuming. (2) This technique increases a plant’s capacity and thus profits will increase. Time must be taken to assemble a group to decide what data should (2) Checklists help to minimize errors and confusion.12 73 .10 7. higherquality. will meet the least resistance. (2) Divergence in weighting factors is common.11 7. (2) Useful when powerful personalities are likely to (2) Assembling the group participants is difficult/timeconsuming. quicker. (1) Arriving at a group consensus is time consuming. Brainstorming 7. so progress is slow. (3) The tool is simple and relatively straightforward. lowercost.
(1) Allows the examination of and understanding of relationships in a process. Statistical process control 7. 74 . (2) Provides a stepbystep picture that creates a common understanding about how the elements of the process fit together. The technique increases productivity and improves working conditions.13 Advantages (1) Evaluates loss at earliest stage of product/process development. (2) The technique improves process performance.Table 71.15 The development process is time consuming. (3) SPC helps identify problems quickly and accurately. (3) Comparing a flowchart to actual process activities highlights areas where policies are unclear or are being violated. (1) This technique determines the cause of variation based on a statistical analysis of the problem. TQM tools and methodologies—Continued. Limitations (1) It may be difficult to convince manufacturers to apply the technique. SPC detects problems but poses no solutions. (2) It is often difficult to characterize the loss function.16 (1) The technique requires cooperation between employees and management to be successful. (2) Useful results obtained quickly and at low cost. (2) The observed operation may not be fully representative of a “typical” process that would occur without scrutiny. Tool or Methodology Quality loss function Section 7.14 Flowchart analysis 7. Work flow analysis 7.
as necessary. Analyze the data. Assess the results of all the changes. Choose the particular characteristics of the operation or product to benchmark. Benchmarking helps improve a process by recognizing priorities and goals.2 Application The benchmarking technique is typically performed in phase E but may also be performed in phase A or B.1. or even noncompetitors that may be recognized as being superior. These strengths and weaknesses can then be compared to internal guidelines to evaluate the organization’s conformance to those guidelines. The technique must be continuously applied in order to be effective because practices constantly change (continuous improvement) affecting strategy.. Once the strengths and weaknesses of competing products are known. Collect data on the processes or products that are being benchmarked. By accomplishing this analysis. then the operation may become inefficient by not keeping up with the latest industry best practices.1 Description Benchmarking. is a technique used to assess how an organization. an organization can also incorporate the strengths of their competitors that exist in certain areas. and implement the plan. If the benchmarking process is performed once and forgotten. (3) (4) (5) (6) (7) 75 .1. or benchmark against noncompetitors that are considered industry leaders. Repeat the benchmarking technique.7.1 Benchmarking 7.1. 7. the company can attempt to differentiate their capabilities in the marketplace. Benchmarking can be applied to identify the strengths for products that directly compete with the organization’s specific product under consideration.3.e. benchmark against competitors. 7. in order to stay uptodate with the applicable operation. This technique can be applied when it is desirable to know the strengths and weaknesses of an organization’s own operation.1. competitors. i. the basic elements of benchmarking include the following: (1) (2) Decide which process(es) or product(s) to benchmark.3 Procedures As adapted from reference 7. Determine the criteria to benchmark. The manufacturers of those competing products are probably using the same benchmarking technique to evaluate the competitors for their product. is performing against internal guidelines. or process. prepare an action plan. benchmark internally against established guidelines. as described in reference 7.
shows an example of comparative benchmarking between one company’s process and five competitors on a scale of 1 (worse) to 10 (better). 7.1.3. Benchmarking helps establish goals and priorities.4 Example The following illustration. This illustration reveals that this company needs to look closely at the operations of competitors 2 and 3 and consider implementing into their process any strengths that are discovered. The reviewing company may well bias their results based on company “wants” rather than customer “wants. Benchmarking helps identify and maintain awareness of industry’s best practices. Better Organization Process 10 9 8 7 6 5 4 3 2 1 Competitor 2 Competitor 3 Competitor 1 Competitor 5 Competitor 4 Worse Figure 71. Benchmarking helps to attain and maintain a competitive position.7. adapted from reference 7.5 Advantages (1) (2) (3) (4) (5) 7. Limitations (1) (2) The benchmarking process must be continuous in order to keep up with the latest industry changes.1. Determining industry “best practices” is often difficult and subjective. Benchmarking helps determine true measures of productivity.1.6 Benchmarking helps meet customer requirements. This company should also look at those competitors rated lower on the scale and identify their weaknesses and ensure that those weaknesses do not exist in their operation. Comparative benchmarking.” 76 .
Any event past. as adapted from reference 7. A desired future outcome and its related factors. an expansion of ideas can then be examined.3. (sec. present. The technique is also useful in planning activities and brainstorming. These discussions bring out many possible viewpoints on the subject. Possible causes are listed. and discussions are begun.2. The group then prioritizes the causes by multivoting. then display the categories on the diagram. thinking is stimulated. commonly referred to as “fish. Each member of the group lists the causes in order of significance. Cause and effect diagrams are developed in a form.3. the effect can be redefined. These causes are then studied and the causes that affect these causes are identified. The group then assigns priorities to the causes and action plans are developed. 7. Once all causes are identified. The diagram is basically a controlled way of gathering and using suggestions through group consensus. The effects and causes can be expressed in words or data.3 Procedures A cause and effect diagram.7. 7. Once all participants reach a similar level of understanding about an issue. The bones of the fish show the organized causes. as described in reference 7. 7. thoughts are organized. (2) (3) (4) 77 . The cause and effect diagram is useful in examining processes such as SPC. The problem is stated in terms acceptable to the group. This effect represents the “problem” that is being investigated. This will continue until no new causes are thought of by the group. cause and effect diagrams are used to examine many different topics which include the following: (1) (2) (3) The relationships between a known problem and the factors that might affect it. list all categories. if necessary. is developed in the following manner: (1) Define the effect as clearly as is possible and place it at the head of the fish.2 Application As adapted from reference 7. graphically represents the relationships between a problem (effect) and its possible causes.2. When a cause and effect diagram is constructed.” where the effect is found in a box to the right which is the head of the fish.2 Cause and Effect Diagrams (Also Known as Fishbone Diagrams or Ishakawa Diagrams) 7. As data are collected. The group brainstorms the causes and lists them in no particular order.2. SPC problems.1 Description The cause and effect diagram. The development process is started in a group session led by a trained facilitator. or future and its causal factors. Votes are counted and a final list is written.14) problems. The cause and effect diagram technique is best applied in phase E but may also be applied in phase A or B.3.
e. Design rework cause and effect diagram.5 Advantages (1) (2) The cause and effect diagram enables quality analysis groups to thoroughly examine all possible causes or categories. When complete. (6) (7) 7. SPC detects a problem but can pose no solution. a team has put their thoughts in writing and arrived at a consensus. The brainstorming effort for this problem is covered in section 7. 72). The next highest prioritized causes will be listed on the diagram as the medium bones. CHANGES SCHEDULE Poor Rushed Outdated Under Tracking Equipment Staffed Late Start Excessive Poor Meetings Notification Not Clear Cost Rules No Second Shift Requires Try It Now. As categories and causes are included on the diagram.7.2.(5) The highest prioritized causes are listed on the diagram as the big bones. The group fills in the probable root causes through “brainstorming” ideas (sec. Backtracking Change Later GRAPHICS Doesn't Know System Responsibility Not Defined DESIGN REWORK Lacking Skills Working Outside Discipline SKILL Conflict ByPassed Discontented Out of Sequence Loanee Not Clear INTERFACES SPECS Figure 72. Teams are formed to research and report on preventive measures. 78 .4 Examples Example 1: Assume the problem is design rework (fig. the group prioritizes the causes using multivoting. thinking may be stimulated and new causes may be identified. The cause and effect diagram is useful in analyzing SPC problems. Finally. the least prioritized causes will be listed on the diagram as the small bones.7).2. Example 2: Figure 73 illustrates the resulting cause and effect diagram after the brainstorming session on identifying problems in receiving telephone messages.. 7. In conclusion. Votes are counted and a final list is written. proactive) measures. 7. This is a technique where each person lists the causes in order of significance.4. Teams are then formed to research and report on preventive (i.
HUMAN ERROR Message light not turned on Employee forgets to sign out Forget to undo call forward Call recipient does not deliver message message misplaced distribution Poor contrast HARDWARE ENVIRONMENT Employee does not see light Peak Activity Too small Not enough phones Number of calls Not enough trunk lines Lack of equipment to take long detailed/technical messages Wrong message taken . Cause and effect diagram on receiving telephone messages.incomplete message rude caller Employee fails to look at light Untimely delivery of message Criticality of message not identified (no guidelines) Inability to take long detailed message Info not available to call recipient long detailed messages Recipient doesn't know how to obtain info employee whereabouts No guidelines for message takers call pickup Phone System Options No guideline for phone system setup call transfer call coverage Procedures No standard guidelines for message takers distractions Lack of interactiveautomated directions to caller Messages are notdelivered in a timely manner Inadequate message delivery system Employee Unaware ofmessage Message Taker responsibilities No feedback of message delivered No identified point of contact No method to reachemployee notaccessible while offi METHOD TRAINING Figure 73. 79 .
7. Use such techniques as DOE.3. all elements of the product life cycle from conception through disposal. it generally will save time and money. computeraided design.2. 7. as described in reference 7.6 Limitations The development of the cause and effect diagram can be timeconsuming in order to arrive at a group consensus.” The Association for Overseas Technical Scholarships. focuses on both the product and the process simultaneously.3.3: (1) (2) (3) Establish multifunction teams which include members from design. Select and use design parameters that will help identify and reduce variability in the production process. quality. etc.7. This technique is typically performed in phase C but may also be performed in phase B. This way.7. 710 . QFD. and value analysis to extend the traditional design approach. H. safety.7 Bibliography Kume.3.12) and DOE (sec. 1985.7. from the outset. Traditionally. Through this technique. marketing. 7.3.1 The approach attempts to link and integrate.: “Statistical Methods for Quality Improvement.2 Application Because the concurrent engineering approach is used to address the product and process simultaneously early in the design phase. One method of achieving this approach is by forming multifunction teams consisting of engineers from several departments. manufacturing. group technology. as adapted from reference 7. the team will establish design goals as well as perform tradeoff analyses using such tools as QFD (sec. support.1 Description Concurrent engineering is the interaction of technical disciplines during the design phase to produce a robust design prior to production. each department will follow the complete process simultaneously rather than one department examining the design and then passing it on to the next department and so on.2.7.3 Concurrent Engineering 7. Concurrent engineering. This process is more of an engineering approach to quality management than a technique.3 Procedures The basic elements involved in applying concurrent engineering include the following.4 The concurrent engineering approach has been known for many years although its use is just receiving widespread application in the United States.5). 7. and producibility do not review an element until after the design has been completed.5 7. 7. robust design. quality.
QFD (Section 7.4 Example Figure 74 illustrates an example of how concurrent engineering is applied. By using multifunctional teams.11) LIFE Quality Function Deployment. Development Maintainability Reliability Safety MULTIFUNCTIONAL TEAMS Sample Techniques Cause and Effect Diagram (Section 7. NGT (Section 7.3.9) BALANCED Nominal Group Technique. Concurrent engineering example. thus making the design process more efficient in terms of both cost and schedule. DOE (Section 7.5) Brainstorming (Section 7. all phases of a product’s life cycle are simultaneously examined.7.12) CYCLE Statistical Process Control. 711 .7) OUTPUT Delphi Technique (Section 7.14) Verification INPUT Customer Needs Logistics Manufacturing Training Deployment Operations Support Disposal Figure 74.2) Design of Experiments. SPC (Section 7.10) PRODUCT Force Field Analysis (Section 7.
4.6 Limitations (1) (2) The degree of success of this technique depends upon the degree of cooperation between the multifunctional team members. the cost of quality technique tracks the expense and benefit of a quality program. If design is pursued by projectized teams. is required at the front end of a program to perform the coordinated planning. 7. This technique can identify the unwanted cost of not doing the job right the first time as well as the cost of improving the job. The approach can also be applied to reduce costs in the designtodevelopment life cycle.3. This technique can identify the most significant costs and thus make it possible to prioritize the activities and/or processes that may need improvement.3.7. Costs will not be reduced by merely tracking the cost of quality but the technique may point out areas where a greater return on investment could be made. Significant additional time. and associated funding.4 Cost of Quality (3) 7. (2) 7. 7. it is often difficult to “frontload” large tasks. a.5 Advantages (1) The concurrent engineering approach can be used to shorten and make more efficient the designtodevelopment life cycle by employing the interactions of functional disciplines by a crossfunctional team. Internal failure (IF) costs External failure (EF) costs Appraisal (A) costs Prevention (P) costs 712 . Cost of quality includes all of the costs associated with maintaining an acceptable quality program.3 Procedures The cost of quality technique is applied in the following manner: (1) Collect cost data for the following categories. This technique is applied to understand the hidden costs of a product or service and to reduce or eliminate these costs. While time and money are saved overall within the effort. as well as the costs incurred as a result of failure to reach the acceptable quality level. c.4.2 Application The cost of quality technique is best applied in phase E. 7.3.1 Description As described in reference 7. b. d. This technique allows the analyst to identify costs that are often hidden.4. the institutional knowledge of the organization becomes very difficult to capture or employ in the design decisions.
inspections). There are indications that. Concurrent engineering (sec. Failures are indirectly proportional to the appraisals/preventions. Programs normally consist of three phases: (1) (2) (3) Initiation. the suggestion system and/or savings shown in process improvement measures are considered prevention. audits. to optimize costbenefit relationships. Prevention costs run 2 percent or less of sales as a national average. Once data are collected and analyzed. As the program progresses. prevention costs (proactive) should increase. EF $ A. P TIME Figure 75. In some companies. Drawings per month. manpower (reactive) should be decreased. Prevention is the key. Development.. Examples are: (1) (2) (3) Manhours per drawing. Cost of quality programs requires a crossfunctional.e. Be careful not to create a system and become so enamored with the system that the objective of savings is obscured. 713 .(2) Data are trended pe