December 1994
B.E. Goldberg, K. Everhart, R. Stevens,
N. Babbitt III, P. Clemens, and L. Stout
System Engineering “Toolbox” for
DesignOriented Engineers
National Aeronautics and Space Administration
Marshall Space Flight Center • MSFC, Alabama 35812
NASA Reference Publication 1358
December 1994
B.E. Goldberg
Marshall Space Flight Center • MSFC, Alabama
K. Everhart, R. Stevens, N. Babbitt III,
P. Clemens, and L. Stout
Sverdrup Technology, Inc.
System Engineering “Toolbox” for
DesignOriented Engineers
ii
ACKNOWLEDGMENTS
The authors are very grateful for the help received from the following persons in producing this
document. Becky Mohr contributed information and illustrations concerning preliminary hazard
analyses and failure modes and effects analyses. Bryan Bachman provided a thorough review of drafts of
the entire document. Larry Thomson prepared a figure in the system safety and reliability tools section.
Jimmy Howell verified all numerical calculations in the examples. The following persons reviewed the
indicated sections of this document and offered suggestions that greatly enhanced the discussions of the
tools and methodologies presented:
Bill Cooley DesignRelated Analytical Tools
Melissa Van Dyke Trend Analysis Tools
Karl Knight System Safety and Reliability Tools
Charles Martin Statistical Tools and Methodologies
Graphical Data Interpretation Tools
Ben Shackelford Case Study
iii
Figures provided courtesy of Sverdrup Technology, Inc., Tullahoma, Tennessee.
Figure 31 Figure 321
Figure 32 Figure 322
Figure 33 Figure 323
Figure 34 Figure 324
Figure 35 Figure 325
Figure 36 Figure 326
Figure 37 Figure 327
Figure 38 Figure 328
Figure 39 Figure 329
Figure 310 Figure 330
Figure 311 Figure 331
Figure 312 Figure 332
Figure 315 Figure 333
Figure 316 Figure 334
Figure 317 Figure 335
Figure 318 Figure 336
Figure 319 Figure 340
Figure 320 Figure 341
iv
TABLE OF CONTENTS
Page
1. INTRODUCTION .................................................................................................................. 11
1.1 Purpose ......................................................................................................................... 11
1.2 Scope ......................................................................................................................... 11
1.3 Relationship With Program or Project Phases.............................................................. 12
References .................................................................................................................... 18
2. CONCEPT DEVELOPMENT TOOLS .................................................................................. 21
2.1 Trade Studies ................................................................................................................ 21
2.1.1 Description ...................................................................................................... 21
2.1.2 Application...................................................................................................... 23
2.1.3 Procedures....................................................................................................... 23
2.1.4 Example .......................................................................................................... 26
2.1.5 Advantages...................................................................................................... 28
2.1.6 Limitations ...................................................................................................... 29
2.1.7 Bibliography.................................................................................................... 29
2.2 CostVersusBenefit Studies......................................................................................... 29
2.2.1 Description ...................................................................................................... 29
2.2.2 Application...................................................................................................... 29
2.2.3 Procedures....................................................................................................... 210
2.2.4 Example .......................................................................................................... 211
2.2.5 Advantages...................................................................................................... 213
2.2.6 Limitations ...................................................................................................... 213
2.2.7 Bibliography.................................................................................................... 213
References ...................................................................................................... 214
3. SYSTEM SAFETY AND RELIABILITY TOOLS ............................................................... 31
3.1 Risk Assessment Matrix ............................................................................................... 32
3.1.1 Description ...................................................................................................... 32
3.1.2 Application...................................................................................................... 35
3.1.3 Procedures....................................................................................................... 36
3.1.4 Example .......................................................................................................... 310
3.1.5 Advantages...................................................................................................... 312
3.1.6 Limitations ...................................................................................................... 312
3.1.7 Bibliography.................................................................................................... 312
v
TABLE OF CONTENTS (Continued)
Page
3.2 Preliminary Hazard Analysis........................................................................................ 312
3.2.1 Description ...................................................................................................... 312
3.2.2 Application...................................................................................................... 313
3.2.3 Procedures....................................................................................................... 313
3.2.4 Example .......................................................................................................... 316
3.2.5 Advantages...................................................................................................... 317
3.2.6 Limitations ...................................................................................................... 318
3.2.7 Bibliography.................................................................................................... 318
3.3 Energy Flow/Barrier Analysis .......................................................................................... 318
3.3.1 Description .......................................................................................................... 318
3.3.2 Application.......................................................................................................... 319
3.3.3 Procedures........................................................................................................... 319
3.3.4 Example .............................................................................................................. 319
3.3.5 Advantages.......................................................................................................... 321
3.3.6 Limitations .......................................................................................................... 321
3.3.7 Bibliography........................................................................................................ 321
3.4 Failure Modes and Effects (and Criticality) Analysis ...................................................... 321
3.4.1 Description .......................................................................................................... 321
3.4.2 Application.......................................................................................................... 322
3.4.3 Procedures........................................................................................................... 322
3.4.4 Example .............................................................................................................. 326
3.4.5 Advantages.......................................................................................................... 328
3.4.6 Limitations .......................................................................................................... 329
3.4.7 Bibliography........................................................................................................ 329
3.5 Reliability Block Diagram................................................................................................ 330
3.5.1 Description .......................................................................................................... 330
3.5.2 Application.......................................................................................................... 332
3.5.3 Procedures........................................................................................................... 332
3.5.4 Example .............................................................................................................. 333
3.5.5 Advantages.......................................................................................................... 334
3.5.6 Limitations .......................................................................................................... 334
3.5.7 Bibliography........................................................................................................ 335
3.6 Fault Tree Analysis .......................................................................................................... 335
3.6.1 Description .......................................................................................................... 335
3.6.2 Application.......................................................................................................... 335
3.6.3 Procedures........................................................................................................... 336
3.6.3.1 Fault Tree Generation .......................................................................... 337
3.6.3.2 Probability Determination.................................................................... 337
3.6.3.3 Identifying and Assessing Cut Sets...................................................... 341
3.6.3.4 Identifying Path Sets ............................................................................ 343
vi
TABLE OF CONTENTS (Continued)
Page
3.6.4 Examples............................................................................................................. 344
3.6.4.1 Fault Tree Construction and Probability Propagation.......................... 344
3.6.4.2 Cut Sets ................................................................................................ 345
3.6.4.3 Path Sets............................................................................................... 346
3.6.5 Advantages.......................................................................................................... 346
3.6.6 Limitations .......................................................................................................... 347
3.6.7 Bibliography........................................................................................................ 347
3.7 Success Tree Analysis ...................................................................................................... 348
3.7.1 Description .......................................................................................................... 348
3.7.2 Application.......................................................................................................... 348
3.7.3 Procedures........................................................................................................... 349
3.7.4 Example .............................................................................................................. 350
3.7.5 Advantages.......................................................................................................... 351
3.7.6 Limitations .......................................................................................................... 351
3.7.7 Bibliography........................................................................................................ 351
3.8 Event Tree Analysis.......................................................................................................... 351
3.8.1 Description .......................................................................................................... 351
3.8.2 Application.......................................................................................................... 352
3.8.3 Procedures........................................................................................................... 353
3.8.4 Example .............................................................................................................. 354
3.8.5 Advantages.......................................................................................................... 354
3.8.6 Limitations .......................................................................................................... 354
3.8.7 Bibliography........................................................................................................ 356
3.9 Fault Tree, Reliability Block Diagram, and Event Tree Transformations ....................... 356
3.9.1 Description .......................................................................................................... 356
3.9.2 Application.......................................................................................................... 356
3.9.3 Procedures........................................................................................................... 356
3.9.3.1 Fault Tree to RBD Transformation...................................................... 356
3.9.3.2 RBD and Fault TreetoEvent Tree Transformation............................ 356
3.9.3.3 RBD to Fault Tree Transformation...................................................... 358
3.9.3.4 Event Tree to RBD and Fault Tree Transformation ............................ 358
3.9.4 Example .............................................................................................................. 358
3.9.5 Advantages.......................................................................................................... 361
3.9.6 Limitations .......................................................................................................... 361
3.9.7 Bibliography........................................................................................................ 361
3.10 CauseConsequence Analysis........................................................................................... 361
3.10.1 Description .......................................................................................................... 361
3.10.2 Application.......................................................................................................... 362
3.10.3 Procedures........................................................................................................... 362
3.10.4 Example .............................................................................................................. 364
3.10.5 Advantages.......................................................................................................... 364
3.10.6 Limitations .......................................................................................................... 365
3.10.7 Bibliography........................................................................................................ 366
vii
TABLE OF CONTENTS (Continued)
Page
3.11 Directed Graphic (Digraph) Matrix Analysis ................................................................... 366
3.11.1 Description .......................................................................................................... 366
3.11.2 Application.......................................................................................................... 366
3.11.3 Procedures........................................................................................................... 367
3.11.4 Example .............................................................................................................. 369
3.11.5 Advantages.......................................................................................................... 370
3.11.6 Limitations .......................................................................................................... 372
3.11.7 Bibliography........................................................................................................ 372
3.12 Combinatorial Failure Probability Analysis Using Subjective Information..................... 372
3.12.1 Description .......................................................................................................... 372
3.12.2 Application.......................................................................................................... 373
3.12.3 Procedures........................................................................................................... 373
3.12.4 Example .............................................................................................................. 374
3.12.5 Advantages.......................................................................................................... 374
3.12.6 Limitations .......................................................................................................... 374
3.13 Failure Mode Information Propagation Modeling............................................................ 376
3.13.1 Description .......................................................................................................... 376
3.13.2 Application.......................................................................................................... 376
3.13.3 Procedures........................................................................................................... 376
3.13.4 Example .............................................................................................................. 377
3.13.5 Advantages.......................................................................................................... 378
3.13.6 Limitations .......................................................................................................... 378
3.14 Probabilistic Design Analysis........................................................................................... 378
3.14.1 Description .......................................................................................................... 378
3.14.2 Application.......................................................................................................... 378
3.14.3 Procedures........................................................................................................... 380
3.14.4 Advantages.......................................................................................................... 383
3.14.5 Limitations .......................................................................................................... 383
3.15 Probabilistic Risk Assessment.......................................................................................... 384
3.15.1 Description .......................................................................................................... 384
3.15.2 Application.......................................................................................................... 384
3.15.3 Procedures........................................................................................................... 384
3.15.4 Advantages.......................................................................................................... 385
3.15.5 Limitations .......................................................................................................... 385
References .......................................................................................................... 386
viii
TABLE OF CONTENTS (Continued)
Page
4. DESIGNRELATED ANALYTICAL TOOLS.......................................................................... 41
4.1 Sensitivity (Parametric) Analysis ..................................................................................... 41
4.1.1 Description .......................................................................................................... 41
4.1.2 Application.......................................................................................................... 41
4.1.3 Procedures........................................................................................................... 41
4.1.4 Example .............................................................................................................. 43
4.1.5 Advantages.......................................................................................................... 44
4.1.6 Limitations .......................................................................................................... 44
4.2 Standard Dimensioning and Tolerancing ......................................................................... 45
4.2.1 Description .......................................................................................................... 45
4.2.2 Application.......................................................................................................... 45
4.2.3 Procedures........................................................................................................... 45
4.2.4 Example .............................................................................................................. 46
4.2.5 Advantages.......................................................................................................... 47
4.2.6 Limitations .......................................................................................................... 47
4.3 Tolerance Stackup Analysis ............................................................................................. 47
4.3.1 Description .......................................................................................................... 47
4.3.2 Application.......................................................................................................... 47
4.3.3 Procedures........................................................................................................... 47
4.3.4 Example .............................................................................................................. 48
4.3.5 Advantages.......................................................................................................... 49
4.3.6 Limitations .......................................................................................................... 49
4.3.7 Bibliography........................................................................................................ 49
References .......................................................................................................... 410
5. GRAPHICAL DATA INTERPRETATION TOOLS................................................................. 51
5.1 Scatter Diagram................................................................................................................ 51
5.1.1 Description .......................................................................................................... 51
5.1.2 Application.......................................................................................................... 51
5.1.3 Procedures........................................................................................................... 53
5.1.4 Example .............................................................................................................. 53
5.1.5 Advantages.......................................................................................................... 53
5.1.6 Limitations .......................................................................................................... 53
5.2 Control Chart .................................................................................................................... 54
5.2.1 Description .......................................................................................................... 54
5.2.2 Application.......................................................................................................... 54
5.2.3 Procedures........................................................................................................... 55
5.2.4 Example .............................................................................................................. 55
5.2.5 Advantages.......................................................................................................... 56
5.2.6 Limitations .......................................................................................................... 56
ix
TABLE OF CONTENTS (Continued)
Page
5.3 Bar Chart........................................................................................................................... 56
5.3.1 Description .......................................................................................................... 56
5.3.2 Application.......................................................................................................... 56
5.3.3 Procedures........................................................................................................... 56
5.3.4 Example .............................................................................................................. 57
5.3.5 Advantages.......................................................................................................... 57
5.3.6 Limitations .......................................................................................................... 57
5.4 TimeLine Chart .............................................................................................................. 58
5.4.1 Description .......................................................................................................... 58
5.4.2 Application.......................................................................................................... 58
5.4.3 Procedures........................................................................................................... 58
5.4.4 Example .............................................................................................................. 58
5.4.5 Advantages.......................................................................................................... 58
5.4.6 Limitations .......................................................................................................... 59
5.5 Stratification Chart............................................................................................................ 59
5.5.1 Description .......................................................................................................... 59
5.5.2 Application.......................................................................................................... 59
5.5.3 Procedures........................................................................................................... 510
5.5.4 Example .............................................................................................................. 511
5.5.5 Advantages.......................................................................................................... 511
5.5.6 Limitations .......................................................................................................... 511
5.6 Pareto Chart ...................................................................................................................... 511
5.6.1 Description .......................................................................................................... 511
5.6.2 Application.......................................................................................................... 511
5.6.3 Procedures........................................................................................................... 511
5.6.4 Example .............................................................................................................. 512
5.6.5 Advantages.......................................................................................................... 513
5.6.6 Limitations .......................................................................................................... 513
5.6.7 Bibliography........................................................................................................ 513
5.7 Histograms........................................................................................................................ 513
5.7.1 Description .......................................................................................................... 513
5.7.2 Application.......................................................................................................... 513
5.7.3 Procedures........................................................................................................... 513
5.7.4 Example .............................................................................................................. 514
5.7.5 Advantages.......................................................................................................... 514
5.7.6 Limitations .......................................................................................................... 514
References........................................................................................................... 515
x
TABLE OF CONTENTS (Continued)
Page
6. STATISTICAL TOOLS AND METHODOLOGIES................................................................. 61
6.1 “Studentt” Analysis ......................................................................................................... 61
6.1.1 Description .......................................................................................................... 61
6.1.2 Application.......................................................................................................... 61
6.1.3 Procedures........................................................................................................... 63
6.1.4 Example .............................................................................................................. 63
6.1.5 Advantages.......................................................................................................... 63
6.1.6 Limitations .......................................................................................................... 64
6.1.7 Bibliography........................................................................................................ 64
6.2 Analysis of Variance......................................................................................................... 64
6.2.1 Description .......................................................................................................... 64
6.2.2 Application.......................................................................................................... 64
6.2.3 Procedures........................................................................................................... 64
6.2.4 Example .............................................................................................................. 66
6.2.5 Advantages.......................................................................................................... 67
6.2.6 Limitations .......................................................................................................... 67
6.3 Correlation Analysis ......................................................................................................... 67
6.3.1 Description .......................................................................................................... 67
6.3.2 Application.......................................................................................................... 67
6.3.3 Procedures........................................................................................................... 67
6.3.4 Example .............................................................................................................. 68
6.3.5 Advantages.......................................................................................................... 68
6.3.6 Limitations .......................................................................................................... 68
6.4 Factorial Analysis ............................................................................................................. 68
6.4.1 Description .......................................................................................................... 68
6.4.2 Application.......................................................................................................... 68
6.4.3 Procedures........................................................................................................... 69
6.4.4 Example .............................................................................................................. 610
6.4.5 Advantages.......................................................................................................... 612
6.4.6 Limitations .......................................................................................................... 612
6.5 Confidence/Reliability Determination and Analysis ........................................................ 612
6.5.1 Description .......................................................................................................... 612
6.5.2 Application.......................................................................................................... 612
6.5.3 Procedures........................................................................................................... 613
6.5.4 Example .............................................................................................................. 614
6.5.5 Advantages.......................................................................................................... 614
6.5.6 Limitations .......................................................................................................... 614
xi
TABLE OF CONTENTS (Continued)
Page
6.6 Regression Analysis.......................................................................................................... 614
6.6.1 Description .......................................................................................................... 614
6.6.2 Application.......................................................................................................... 615
6.6.3 Procedures........................................................................................................... 615
6.6.4 Example .............................................................................................................. 616
6.6.5 Advantages.......................................................................................................... 617
6.6.6 Limitations .......................................................................................................... 617
6.7 Response Surface Methodology ....................................................................................... 617
6.7.1 Description .......................................................................................................... 617
6.7.2 Application.......................................................................................................... 617
6.7.3 Procedures........................................................................................................... 617
6.7.4 Example .............................................................................................................. 618
6.7.5 Advantages.......................................................................................................... 619
6.7.6 Limitations .......................................................................................................... 619
References .......................................................................................................... 620
7. TOTAL QUALITY MANAGEMENT (TQM) TOOLS ............................................................. 71
7.1 Benchmarking................................................................................................................... 75
7.1.1 Description .......................................................................................................... 75
7.1.2 Application.......................................................................................................... 75
7.1.3 Procedures........................................................................................................... 75
7.1.4 Example .............................................................................................................. 76
7.1.5 Advantages.......................................................................................................... 76
7.1.6 Limitations .......................................................................................................... 76
7.2 Cause and Effect Diagrams (Also Known as Fishbone Diagrams
or Ishakawa Diagrams) ....................................................................................... 77
7.2.1 Description .......................................................................................................... 77
7.2.2 Application.......................................................................................................... 77
7.2.3 Procedures........................................................................................................... 77
7.2.4 Examples............................................................................................................. 78
7.2.5 Advantages.......................................................................................................... 78
7.2.6 Limitations .......................................................................................................... 710
7.2.7 Bibliography........................................................................................................ 710
7.3 Concurrent Engineering.................................................................................................... 710
7.3.1 Description .......................................................................................................... 710
7.3.2 Application.......................................................................................................... 710
7.3.3 Procedures........................................................................................................... 710
7.3.4 Example .............................................................................................................. 711
7.3.5 Advantages.......................................................................................................... 712
7.3.6 Limitations .......................................................................................................... 712
xii
TABLE OF CONTENTS (Continued)
Page
7.4 Cost of Quality.................................................................................................................. 712
7.4.1 Description .......................................................................................................... 712
7.4.2 Application.......................................................................................................... 712
7.4.3 Procedures........................................................................................................... 712
7.4.4 Example .............................................................................................................. 714
7.4.5 Advantages.......................................................................................................... 714
7.4.6 Limitations .......................................................................................................... 715
7.5 Design of Experiments ..................................................................................................... 715
7.5.1 Description .......................................................................................................... 715
7.5.2 Application.......................................................................................................... 715
7.5.3 Procedures........................................................................................................... 715
7.5.4 Example .............................................................................................................. 716
7.5.5 Advantages.......................................................................................................... 718
7.5.6 Limitations .......................................................................................................... 718
7.5.7 Bibliography........................................................................................................ 719
7.6 Evolutionary Operation .................................................................................................... 719
7.6.1 Description .......................................................................................................... 719
7.6.2 Application.......................................................................................................... 719
7.6.3 Procedures........................................................................................................... 719
7.6.4 Example .............................................................................................................. 720
7.6.5 Advantages.......................................................................................................... 723
7.6.6 Limitations .......................................................................................................... 723
7.7 Brainstorming ................................................................................................................... 723
7.7.1 Description .......................................................................................................... 723
7.7.2 Application.......................................................................................................... 723
7.7.3 Procedures........................................................................................................... 724
7.7.4 Example .............................................................................................................. 724
7.7.5 Advantages.......................................................................................................... 725
7.7.6 Limitations .......................................................................................................... 725
7.8 Checklists.......................................................................................................................... 726
7.8.1 Description .......................................................................................................... 726
7.8.2 Application.......................................................................................................... 726
7.8.3 Procedures........................................................................................................... 726
7.8.4 Example .............................................................................................................. 726
7.8.5 Advantages.......................................................................................................... 726
7.8.6 Limitations .......................................................................................................... 726
xiii
TABLE OF CONTENTS (Continued)
Page
7.9 Delphi Technique.............................................................................................................. 727
7.9.1 Description .......................................................................................................... 727
7.9.2 Application.......................................................................................................... 727
7.9.3 Procedures........................................................................................................... 727
7.9.4 Example .............................................................................................................. 728
7.9.5 Advantages.......................................................................................................... 729
7.9.6 Limitations .......................................................................................................... 729
7.10 Nominal Group Technique ............................................................................................... 730
7.10.1 Description .......................................................................................................... 730
7.10.2 Application.......................................................................................................... 730
7.10.3 Procedures........................................................................................................... 730
7.10.4 Example .............................................................................................................. 730
7.10.5 Advantages.......................................................................................................... 732
7.10.6 Limitations .......................................................................................................... 732
7.11 Force Field Analysis ......................................................................................................... 732
7.11.1 Description .......................................................................................................... 732
7.11.2 Application.......................................................................................................... 732
7.11.3 Procedures........................................................................................................... 734
7.11.4 Example .............................................................................................................. 734
7.11.5 Advantages.......................................................................................................... 735
7.11.6 Limitations .......................................................................................................... 735
7.12 Quality Function Deployment .......................................................................................... 735
7.12.1 Description .......................................................................................................... 735
7.12.2 Application.......................................................................................................... 736
7.12.3 Procedures........................................................................................................... 736
7.12.4 Example .............................................................................................................. 737
7.12.5 Advantages.......................................................................................................... 740
7.12.6 Limitations .......................................................................................................... 740
7.12.7 Bibliography........................................................................................................ 740
7.13 Quality Loss Function....................................................................................................... 741
7.13.1 Description .......................................................................................................... 741
7.13.2 Application.......................................................................................................... 742
7.13.3 Procedures........................................................................................................... 742
7.13.4 Example .............................................................................................................. 743
7.13.5 Advantages.......................................................................................................... 744
7.13.6 Limitations .......................................................................................................... 744
xiv
TABLE OF CONTENTS (Continued)
Page
7.14 Statistical Process Control ................................................................................................ 744
7.14.1 Description .......................................................................................................... 744
7.14.2 Application.......................................................................................................... 744
7.14.3 Procedures........................................................................................................... 744
7.14.4 Example .............................................................................................................. 746
7.14.5 Advantages.......................................................................................................... 748
7.14.6 Limitations .......................................................................................................... 749
7.14.7 Bibliography........................................................................................................ 749
7.15 Flowchart Analysis ........................................................................................................... 749
7.15.1 Description .......................................................................................................... 749
7.15.2 Application.......................................................................................................... 752
7.15.3 Procedures........................................................................................................... 752
7.15.4 Example .............................................................................................................. 752
7.15.5 Advantages.......................................................................................................... 752
7.15.6 Limitations .......................................................................................................... 752
7.16 Work Flow Analysis ......................................................................................................... 753
7.16.1 Description .......................................................................................................... 753
7.16.2 Application.......................................................................................................... 754
7.16.3 Procedures........................................................................................................... 754
7.16.4 Example .............................................................................................................. 754
7.16.5 Advantages.......................................................................................................... 755
7.16.6 Limitations .......................................................................................................... 755
References .......................................................................................................... 756
8. TREND ANALYSIS TOOLS..................................................................................................... 81
8.1 Performance Trend Analysis ............................................................................................ 84
8.1.1 Description .......................................................................................................... 84
8.1.2 Application.......................................................................................................... 84
8.1.3 Procedures........................................................................................................... 85
8.1.4 Example .............................................................................................................. 87
8.1.5 Advantages.......................................................................................................... 88
8.1.6 Limitations .......................................................................................................... 88
8.1.7 Bibliography........................................................................................................ 88
8.2 Problem Trend Analysis ................................................................................................... 88
8.2.1 Description .......................................................................................................... 88
8.2.2 Application.......................................................................................................... 89
8.2.3 Procedures........................................................................................................... 810
8.2.4 Example .............................................................................................................. 811
8.2.5 Advantages.......................................................................................................... 815
8.2.6 Limitations .......................................................................................................... 815
8.2.7 Bibliography........................................................................................................ 815
xv
TABLE OF CONTENTS (Concluded)
Page
8.3 Programmatic Trend Analysis .......................................................................................... 815
8.3.1 Description .......................................................................................................... 815
8.3.2 Application.......................................................................................................... 816
8.3.3 Procedures........................................................................................................... 816
8.3.4 Example .............................................................................................................. 818
8.3.5 Advantages.......................................................................................................... 818
8.3.6 Limitations .......................................................................................................... 818
8.3.7 Bibliography........................................................................................................ 818
8.4 Supportability Trend Analysis.......................................................................................... 819
8.4.1 Description .......................................................................................................... 819
8.4.2 Application.......................................................................................................... 820
8.4.3 Procedures........................................................................................................... 821
8.4.4 Example .............................................................................................................. 822
8.4.5 Advantages.......................................................................................................... 822
8.4.6 Limitations .......................................................................................................... 822
8.4.7 Bibliography........................................................................................................ 823
8.5 Reliability Trend Analysis................................................................................................ 824
8.5.1 Description .......................................................................................................... 824
8.5.2 Application.......................................................................................................... 824
8.5.3 Procedures........................................................................................................... 825
8.5.4 Example .............................................................................................................. 825
8.5.5 Advantages.......................................................................................................... 825
8.5.6 Limitations .......................................................................................................... 826
8.5.7 Bibliography........................................................................................................ 826
References........................................................................................................... 827
Appendix A ...................................................................................................................................... A1
Appendix B ...................................................................................................................................... B1
Appendix C ...................................................................................................................................... C1
Appendix D ...................................................................................................................................... D1
Appendix E ...................................................................................................................................... E1
Appendix F ....................................................................................................................................... F1
xvi
LIST OF ILLUSTRATIONS
Figure Title Page
21. Example utility functions ................................................................................... 27
22. Example weighted factor trade study summary table ........................................ 28
31. Risk plane ........................................................................................................... 35
32. Isorisk contour usage ........................................................................................ 36
33. Risk plane to risk matrix transformation ............................................................ 37
34. Helpful hints in creating a risk assessment matrix ............................................. 38
35. Typical risk assessment matrix ........................................................................... 310
36. Severity and probability interpretations ............................................................. 311
37. PHA process flowchart ...................................................................................... 314
38. Typical PHA ...................................................................................................... 316
39. Example of system breakdown and numerical coding ....................................... 323
310. FMECA process flowchart ................................................................................. 324
311. Typical FMECA worksheet ............................................................................... 326
312. Example of an FMECA ...................................................................................... 327
313. Typical complex RBD ........................................................................................ 331
314. Example RBD ..................................................................................................... 333
315. Fault tree construction process............................................................................ 339
316. Log average method of probability estimation ................................................... 339
317. Relationship between reliability and failure probability propagation................. 340
318. Failure probability propagation through OR and AND gates ............................. 340
319. Exact solution of OR gate failure probability propagation ................................ 341
320. Example fault tree .............................................................................................. 344
321. Example of determining cut sets ........................................................................ 345
322. Example of determining path sets ...................................................................... 346
323. Success tree construction process ...................................................................... 349
xvii
LIST OF ILLUSTRATIONS
Figure Title Page
324. Example success tree ......................................................................................... 350
325. Event tree (generic case) .................................................................................... 352
326. Event tree (Bernoulli model) .............................................................................. 353
327. Example ETA ..................................................................................................... 355
328. Fault tree to RBD transformation. ...................................................................... 357
329. Deriving cut and path sets from an RBD ........................................................... 357
330. RBD to event tree transformation ...................................................................... 358
331. RBD to fault tree transformation ....................................................................... 359
332. Event tree to fault tree transformation ............................................................... 359
333. Equivalent logic RBD and fault tree .................................................................. 360
334. Relationship between cause and consequence ................................................... 362
335. Causeconsequence analysis format ................................................................... 364
336. Example causeconsequence analysis ................................................................ 365
337. Comparison between digraph and fault tree logic gates .................................... 368
338. Construction of digraph adjacency matrix ......................................................... 369
339. Example digraph matrix analysis ....................................................................... 371
340. Example combinatorial failure probability analysis .......................................... 375
341. Example failure mode information propagation model ...................................... 379
342. Load and capability transfer functions ............................................................... 382
343. Interference between load and capability density functions .............................. 383
41. Example of dimensioning and tolerancing ......................................................... 46
42. Oring joint ......................................................................................................... 48
43. Oring joint components ..................................................................................... 48
51. Scatter diagram example..................................................................................... 54
52. Control chart example......................................................................................... 55
xviii
LIST OF ILLUSTRATIONS
Figure Title Page
53. Bar chart example ............................................................................................... 57
54. Timeline chart example ..................................................................................... 59
55. Stratification (histogram) chart example ............................................................ 510
56. Pareto chart example........................................................................................... 512
57. Histogram example ............................................................................................. 514
61. Line generated with least squares method .......................................................... 616
71. Comparative benchmarking ................................................................................ 76
72. Design rework cause and effect diagram............................................................ 78
73. Cause and effect diagram on receiving telephone messages .............................. 79
74. Concurrent engineering example ........................................................................ 711
75. Standard cost of quality curve............................................................................. 713
76. Factor/level effects graph.................................................................................... 718
77. EVOP example.................................................................................................... 720
78. Sample of a partial igniter subsystem fault tree.................................................. 728
79. Fault tree sample with estimates assigned .......................................................... 729
710. Force field analysis example............................................................................... 734
711. House of quality .................................................................................................. 735
712. QFD example on automobile industry ................................................................ 739
713. Traditional view to meeting specification........................................................... 741
714. Quality loss function for NIB.............................................................................. 742
715. Quality loss function example............................................................................. 743
716. Control chart showing mean deviation for each part .......................................... 747
717. Range chart showing mean range for each part .................................................. 747
718. Pareto chart showing mean deviation for each hole guide.................................. 748
719. Control chart showing mean deviation for hole guide 1..................................... 748
xix
LIST OF ILLUSTRATIONS
Figure Title Page
720. Example of topdown flowchart ......................................................................... 749
721. Example of detailed flowchart ........................................................................... 750
722. Common flowchart symbols .............................................................................. 751
723. Work flow diagram example .............................................................................. 753
724. WFA example .................................................................................................... 755
81. Performance trend analysis example .................................................................. 87
82. Problem trend analysis example ........................................................................ 812
83. Programmatic trend analysis example ............................................................... 819
84. Supportability trend analysis example ............................................................... 823
85 Reliability trend analysis example ..................................................................... 825
xx
LIST OF TABLES
Table Title Page
11. System engineering “toolbox” function matrix ................................................... 13
12. System engineering “toolbox” project phase matrix ............................................ 15
21. Concept development tools and methodologies ................................................... 22
22. Typical weighted trade study summary table ....................................................... 25
23. Example selection criteria for costversusbenefit analyses ................................ 210
31. Symbolic logic techniques ................................................................................... 32
32. System safety and reliability tools and methodologies ........................................ 33
33. Examples of strategies to manage harmful energy flow ...................................... 320
34 . Simple RBD construction ..................................................................................... 330
35. FTA procedures .................................................................................................... 336
36. Fault tree construction symbols ........................................................................... 338
37. Probability propagation expressions for logic gates ............................................ 342
38. Causeconsequence tree construction symbols .................................................... 363
39. Combinatorial failure probability analysis subjective scale ................................ 373
41. Designrelated analytical tools and methodologies ............................................. 42
42. Sensitivity analysis calculations ........................................................................... 44
51. Graphical data interpretation tools and methodologies ........................................ 52
61. Statistical tools and methodologies ...................................................................... 62
62. Factorial analysis factors and magnitudes ........................................................... 69
63. Factorial analysis example ................................................................................... 610
71. TQM tools and methodologies ............................................................................. 72
72. Month’s cost of quality ......................................................................................... 714
73. 2
3
factorial design data.......................................................................................... 716
74. Trial, effects, and results ....................................................................................... 716
xxi
LIST OF TABLES (Continued)
Table Title Page
75. Calculation of effects ........................................................................................... 717
76. EVOP cycle No. 1 data ........................................................................................ 720
77. EVOP cycle No. 2 data ........................................................................................ 721
78. Comparison of EVOP cycle No. 1 and cycle No. 2 data ..................................... 722
79. Motor postflight checklist ..................................................................................... 727
710. Replacement technology concerns ....................................................................... 731
711. Concerns with assigned weighting factors ........................................................... 733
712. QFD matrix sample calculations .......................................................................... 737
713. Nominal hole size deviations and drill guide positions ....................................... 746
81. Trend analysis tools and methodologies .............................................................. 83
xxii
ACRONYMS
AHP Analytical hierarchy process
AHPA Analytical hierarchy process approach
AIAA American Institute of Aeronautics and Astronomics
ANOVA Analysis of variance
B/C Benefittocost
CIL Critical items list
CIM Change in mean
CSF Compliance safety factor
DAS Data acquisition system
DOE Design of experiments
DOF Degreeoffreedom
DR Discrepancy report
EF External failure
ETA Event tree analysis
EVOP Evolutionary operation
FMEA Failure modes and effects analysis
FMECA Failure modes, effects, and criticality analysis
FTA Fault tree analysis
IF Internal failure
L(y) Loss function (quality)
LCL Lower control limits
LDL Lower decision line
LIB Larger is better
xxiii
ACRONYMS (Continued)
LSL Lower specification limit
MTBF Mean time between failures
MTBR Mean time between repairs
MTTR Mean time to repair
NASA National Aeronautics and Space Administration
NGT Nominal group technique
NIB Nominal is best
PDA Probabilistic design analysis
PHA Preliminary hazard analysis
PRA Probabilistic risk assessment
PRACA Problem reporting and corrective action
OSHA Occupational Safety and Health Administration
QFD Quality function deployment
RBD Reliability block diagram
RSM Response surface methodology
SE Standard error
SESTC System Effectiveness and Safety Technical Committee
SIB Smaller is better
SME Sum of mean error
SMQ Safety and mission quality
SMR Sum of mean replicate
SPC Statistical process control
SRM Solid rocket motor
xxiv
ACRONYMS (Continued)
SSE Sum of squares error
SSR Sum of squares replication
SST Total sum of squares
STA Success tree analysis
TQM Total quality management
UCL Upper control limit
UCLR Upper control limit range
UDL Upper decision line
USL Upper specification limit
WFA Work flow analysis
xxv
xxvi
REFERENCE PUBLICATION
SYSTEM ENGINEERING “TOOLBOX” FOR DESIGNORIENTED ENGINEERS
1. INTRODUCTION
1.1 Purpose
Many references are available on systems engineering from the project management perspective.
Too often, these references are of only limited utility from the designer’s standpoint. A practicing,
designoriented systems engineer has difficulty finding any ready reference as to what tools and
methodologies are available.
The purpose of this system engineering toolbox is to provide tools and methodologies available
to the designoriented systems engineer. A tool, as used herein, is defined as a set of procedures to
accomplish a specific function. A methodology is defined as a collection of tools, rules, and postulates to
accomplish a purpose. A thorough literature search was performed to identify the prevalent tools and
methodologies. For each concept addressed in the toolbox, the following information is provided: (1)
description, (2) application, (3) procedures, (4) example, if practical, (5) advantages, (6) limitations, and
(7) bibliography and/or references.
This toolbox is intended solely as guidance for potential tools and methodologies, rather than
direction or instruction for specific technique selection or utilization. It is left to the user to determine
which technique(s), at which level of detail are applicable, and what might be the expected “value
added” for their purposes. Caution should be exercised in the use of these tools and methodologies. Use
of the techniques for the sake of “using techniques” is rarely resourceeffective. In addition, while
techniques have been categorized for recommended areas of use, this is not intended to be restrictive.
Readers are encouraged to question, comment (app. A) and, in general, use this reference as one source
among many. The reader is also cautioned to validate results from a given tool to ensure accuracy and
applicability to the problem at hand.
1.2 Scope
The tools and methodologies available to the designoriented systems engineer can be
categorized in various ways depending upon the application. Concept development tools, section 2, are
useful when selecting the preferred option of several alternatives. Among these alternatives are such
things as cost, complexity, weight, safety, manufacturability, or perhaps determining the ratio of
expected future benefits to the expected future costs.
System safety and reliability tools, section 3, address the following areas of concern: (1) identify
and assess hazards, (2) identify failure modes and show their consequences or effects, and (3) symbolic
logic modeling tools used to understand the failure mechanisms of the system. These tools are also used
to determine the probability of failure occurring or the reliability that a component will operate success
fully, either in comparative or absolute terms, as applicable.
Designrelated analytical tools, section 4, are applied to show (1) which parameters affect a sys
tem the most or least, (2) a method for specifying dimensions and tolerances, and (3) the determination
of the possibility or probability of having form, fit, or function problems with a design, or to determine a
tolerance or dimension necessary to avoid these problems.
11
When there is a desire to monitor performance, identify relationships, or reveal the most
important variables in a set of data, graphical data interpretation tools are typically applied. These tools
are discussed in section 5. Statistical tools and methodologies, section 6, compare sample statistics and
population statistics. Variations are identified and mathematical relationships are determined. Many
excellent texts are available on statistical methods, as are software packages. For this reason, this
document touches only lightly on this area.
Total quality management (TQM) tools, section 7, are applied to continuously improve perfor
mance at all levels of operation, in all areas of an organization, using all available human and capital
resources. Finally, quantitative tools that are used to identify potentially hazardous conditions based on
past empirical data are trend analysis tools, section 8. The ultimate objective for these tools is to assess
the current status, and to forecast future events.
To assist in further defining optimal areas in which each technique may be useful, table 11
provides a functional matrix which categorizes the functionality of each tool or methodology into (1)
data analysis, (2) problem identification, (3) decision making, (4) modeling, (5) prevention, (6) creative,
and (7) graphical. These functionality categories are found in reference 1.1.
Extensive research was performed in order to identify all prevalent tools and methodologies
available to the designoriented systems engineer. Nevertheless, important tools or methodologies may
have been overlooked. If a tool or methodology should be considered for this toolbox, appendix A is
provided for the reader to complete and return to the individual indicated on the form.
To further illustrate how selected tools and methodologies in this toolbox are applied, and misap
plied, appendix B provides a case study illustrating the trials and tribulations of an engineer applying his
recently acquired knowledge of the techniques to a given work assignment.
Appendix C provides a glossary of terms applicable to the tools and methodologies in this toolbox.
1.3 Relationship With Program or Project Phases
Each tool or methodology may be performed in a minimum of one of the following phases, as
described in reference 1.2, of a project design cycle.
(1) Phase A (conceptual trade studies)—a quantitative and/or qualitative comparison of
candidate concepts against key evaluation criteria to determine the best alternative.
(2) Phase B (concept definition)—the establishment of system design requirements as well as
conceptually designing a mission, conducting feasibility studies and design tradeoff
studies.
(3) Phase C (design and development)—the initiation of product development and the
establishment of system specifications.
(4) Phase D (fabrication, integration, test, and evaluation)—system verification.
(5) Phase E (operations)—the deployment of the product and performance validation.
Table 12 provides a project phase matrix for all of the tools and methodologies identified in this
toolbox. An entry of (1) for the phase means the technique is primarily performed in that phase and an
entry of (2) means the technique is secondarily performed in that phase. Though the entries in this matrix
are a result of research by the authors, other phases should be considered by the user for a particular tool
or methodology.
12
Table 11. System engineering “toolbox” function matrix—Continued
Section Tool or Methodology Data
Analysis
Problem
Identification
Decision
Making
Modeling Prevention Creative Graphical
Concept development tools
2.1 Trade studies √ √
2.2 Costversusbenefit studies √ √
System safety and reliability tools
3.1 Risk assessment matrix √ √
3.2 Preliminary hazard analysis, √ √ √ √
3.3 Energy flow/barrier analysis √ √ √ √
3.4 Failure modes and effects analysis √ √ √ √
3.5 Reliability block diagram √ √ √ √
3.6 Fault tree analysis √ √ √ √ √ √
3.7 Success tree analysis √ √ √ √ √
3.8 Event tree analysis √ √ √ √ √
3.9 Fault tree/reliability block diagram/event tree
transformations
√ √ √ √ √
3.10 Causeconsequence analysis √ √ √ √ √ √
3.11 Directed graph (digraph) matrix analysis √ √ √ √ √
3.12 Combinatorial failure probability analysis using
subjective information
√ √ √
3.13 Failure mode information propagation modeling √ √ √ √
3.14 Probabilistic design analysis √ √
3.15 Probabilistic risk assessment √ √ √ √ √
Designrelated analytical tools
4.1 Sensitivity (parametric) analysis √ √
4.2 Standard dimensioning and tolerancing √
4.3 Tolerance stackup analysis √ √ √
Graphical data interpretation tools
5.1 Scatter diagram √
5.2 Control chart √ √ √ √
5.3 Bar chart √
5.4 Timeline chart √
5.5 Stratification chart √
5.6 Pareto chart √ √ √ √
5.7 Histograms √
Note: Functionality categories found in reference 1.1.
13
Table 11. System engineering “toolbox” function matrix—Continued.
Section Tool or Methodology Data
Analysis
Problem
Identification
Decision
Making
Modeling Prevention Creative Graphical
Statistical tools and methodologies
6.1 “Studentt” analysis √ √ √
6.2 Analysis of variance √ √ √
6.3 Correlation analysis √ √ √
6.4 Factorial arrays √ √ √
6.5 Confidence/reliability determination and analysis √ √ √
6.6 Regression analysis √ √ √ √
6.7 Response surface methodology √ √ √ √
TQM tools
7.1 Benchmarking √ √
7.2 Cause and effect diagrams √ √
7.3 Concurrent engineering √ √ √
7.4 Cost of quality √ √
7.5 Design of experiments √ √
7.6 Evolutionary operation √ √ √
7.7 Brainstorming √ √ √
7.8 Checklists √ √ √
7.9 Delphi technique √ √
7.10 Nominal group technique √ √ √
7.11 Force field analysis √ √ √
7.12 Quality function deployment √ √ √
7.13 Quality loss function √ √
7.14 Statistical process control √ √ √ √
7.15 Flowchart analysis √ √
7.16 Work flow analysis √ √
Trend analysis tools
8.1 Performance trend analysis √ √ √ √
8.2 Problem trend analysis √ √ √ √
8.3 Programmatic trend analysis √ √ √ √
8.4 Supportability trend analysis √ √ √ √
8.5 Reliability trend analysis √ √ √ √
Note: Functionality categories found in reference 1.1.
14
Table 12. System engineering “toolbox” project phase matrix—Continued
Code: 1—Primary
2—Secondary
Phase A Phase B Phase C Phase D Phase E
Section Tool or Methodology Conceptual
Trade Studies
Concept
Definition
Design and
Development
Fabrication,
Integration,
Test, and
Evaluation
Operations
Concept development tools
2.1 Trade studies 1 2
2.2 Cost versus benefit studies 1 2 2
System safety and reliability tools
3.1 Risk assessment matrix 2 1
3.2 Preliminary hazard analysis, 2 1
3.3 Energy flow/barrier analysis 2 1 2
3.4 Failure modes and effects analysis 1
3.5 Reliability block diagram 1
3.6 Fault tree analysis 1 2
3.7 Success tree analysis 1 2
3.8 Event tree analysis 1 2 1
3.9 Fault tree/reliability block diagram/event tree
transformations
2 1
3.10 Causeconsequence analysis 1 2 1
3.11 Directed graph (digraph) matrix analysis 1
3.12 Combinatorial failure probability analysis using
subjective information
1
3.13 Failure mode information propagation modeling 1 2
3.14 Probabilistic design analysis 1
3.15 Probabilistic risk assessment 1
Note: Phases discussed in reference 1.2.
15
Table 12. System engineering “toolbox” project phase matrix—Continued
Code: 1—Primary
2—Secondary
Phase A Phase B Phase C Phase D Phase E
Section Tool or Methodology Conceptual
Trade Studies
Concept
Definition
Design and
Development
Fabrication,
Integration,
Test, and
Evaluation
Operations
Designrelated analytical tools
4.1 Sensitivity (parameteric) analysis 1 1
4.2 Standard dimensioning and tolerancing 1 2
4.3 Tolerance stackup analysis 1 1
Graphical data interpretation tools
5.1 Scatter diagram 1
5.2 Control chart 1
5.3 Bar chart 1
5.4 Timeline chart 1
5.5 Stratification chart 1
5.6 Pareto chart 1
5.7 Histograms 1
Statistical tools and methodologies
6.1 “Studentt” analysis 2 1 2
6.2 Analysis of variance 2 1 2
6.3 Correlation analysis 2 1 2
6.4 Factorial arrays 1 2 2
6.5 Confidence/reliability determination and analysis 1 1 1
6.6 Regression analysis 1 2
6.7 Response surface methodology 1 1
Note: Phases discussed in reference 1.2.
16
Table 12. System engineering “toolbox” project phase matrix—Continued.
Code: 1—Primary
2—Secondary
Phase A Phase B Phase C Phase D Phase E
Section Tool or Methodology Conceptual
Trade Studies
Concept
Definition
Design and
Development
Fabrication,
Integration,
Test, and
Evaluation
Operations
TQM tools
7.1 Benchmarking 2 2 1
7.2 Cause and effect diagrams 2 2 1
7.3 Concurrent engineering 2 1
7.4 Cost of quality 1
7.5 Design of experiment 1 2
7.6 Evolutionary operation 2 1
7.7 Brainstorming 1 2
7.8 Checklists 2 2 1
7.9 Delphi technique 1 1 2
7.10 Nominal group technique 1 1 2
7.11 Force field analysis 1 1
7.12 Quality function deployment 2 2 1
7.13 Quality loss function 2 1
7.14 Statistical process control 1
7.15 Flowchart analysis 1 2
7.16 Work flow analysis 1
Trend analysis tools
8.1 Performance trend analysis 2 1
8.2 Problem trend analysis 2 1
8.3 Programmatic trend analysis 1
8.4 Supportability trend analysis 1
8.5 Reliability trend analysis 2 1
Note: Phases discussed in reference 1.2.
17
REFERENCES
1.1 Brocka, B., and Brocka, M.S.: “Quality Management, Implementing the Best Ideas of the
Masters.” Business One Irwin, Homewood, Illinois 60430.
1.2 “System Engineering Process (Short Course Lecture Notebook).” Center for Systems Management
(CSM), Santa Rosa, California, September 1991.
18
2. CONCEPT DEVELOPMENT TOOLS
Trade studies and costversusbenefit studies are presented in this section. These tools are used to
select the preferred option of several alternatives. Trade studies, section 2.1, are quantitative and/or
qualitative comparison techniques to choose an alternative when considering such items as cost,
complexity, weight, safety, manufacturability, etc. Costversusbenefit studies, section 2.2, provide a
method to assess alternatives by determining the ratio of expected future benefits to expected future
costs.
A summary of the advantages and limitations of each tool or methodology discussed in this
section is presented in table 21.
2.1 TRADE STUDIES
2.1.1 Description
In general, trade (or tradeoff) studies provide a mechanism for systematic depiction of both sys
tem requirements and system design options for achieving those requirements. Once tabulated, a
comparison of relevant data (cost, complexity, weight, safety, manufacturability, etc.) is then performed
to rank those candidate design options in order of desirability.
These studies are categorized as either a weighted factor trade study or an analytical hierarchy
trade study, with the latter being a special version of the former. These techniques are described in
reference 2.1. A trade tree can be generated with either of the above two options. A trade tree is simply a
pictorial representation of how highlevel alternatives (or issues) in the decision process are logically
resolved into decreasingly lower level alternatives (or issues). A trade tree may be presented without
results or simply as a representation of options.
A weighted factor trade study is usually performed when each of the options under consideration
is very well defined and there is good definition of the program requirements as well. All factors
(program requirements) that are determined to be important, are delineated with an associated weighting
factor. The options are then assessed with respect to each of the factors and an equation is developed that
weighs this assessment. The decision is then based upon the numerical results of the analysis.
The analytical hierarchy process (AHP) is a variation of the weighted factors analysis and is the
most complex of the trade studies presented here. This approach allows for delineation of the facts and
rationale that go into the subjective assessment of each of the options. Further, pseudoquantitative equa
tions may be developed (as in probabilistic assessment equations for failure causes in fault tree analyses)
to increase confidence in analysis results. The AHP provides a multicriteria analysis methodology that
employs a pairwise comparison process to compare options to factors in a relative manner. This is used
when subjective verbal expressions (equal, moderate, strong, very strong, etc.) are easier to develop than
numerical (3 versus 3.2, etc.) assessments. Pseudoquantitative numbers are then ascribed to the words
and a score developed for each of the options.
A key to any trade study is the initial selection and prioritization of specific desirable attributes.
This is often very difficult and the prioritization delineation may change during the early phases of the
program. It is very important, and often overlooked, that when the prioritization changes, a cursory look
at the significant, completed trades should be performed to determine any impacts to their conclusions.
21
Table 21. Concept development tools and methodologies.
Tool or Methodology Section Advantages Limitations
Trade studies 2.1 (1) Different kinds and/or levels of study allow flexi
bility in the depth of the review, i.e., resources
expended can be commensurate with the benefits of
the task.
(2) Adaptive to prioritization based upon programmatic
(cost, schedule) considerations as well as technical
(weight, reliability, etc.) ones.
(3) Identification of disadvantages of specific design
option may lead to the definition of effective
countermeasures if combined with other techniques.
(1) Very dependent upon the expertise of the analyst
and the amount of available accurate quantitative
data.
(2) Improper generation of selection criteria, weight
factors, and utility functions can prejudice the
assessment and lead to incorrect results.
(3) The number of alternatives which can be considered
is limited by the expenditure of resources required to
perform the analysis.
(4) Options evaluated are not determined as a result of
the study but must be decided upon prior to the
assessment by the operator.
(5) Weighting factors and advantages/disadvantages are
very subjective (although objective data may be
added which significantly complicates and enlarges
the study) and this subjectivism is very near to the
study conclusions.
Costversusbenefit studies 2.2 (1) By performing a costversusbenefit analysis, the
analyst can assess the cost effectiveness of several
alternatives over the entire life cycle of the proposed
system under consideration.
(2) Provides documentation of the parameters evaluated
and the prioritized options considered.
(1) The analysis is flawed if system requirements are
incomplete or inadequate. If the system operating
environment is not understood or accurately
characterized, the total costs can be underestimated.
If the system requirements are too general or vague,
the effectiveness of benefits can not be addressed in
specific, measurable terms.
(2) The analysis is only as good as the list of alter
natives considered. An incomplete list of alter
natives will lead to an incomplete analysis.
(3) The analysis is flawed if incomplete or inaccurate
cost estimates are used.
(4) The analyst must be able to quantify the value of
benefits, which are often intangible or insubstantial
and difficult to characterize in terms of monetary
value.
22
2.1.2 Application
These studies should typically be performed in phase A of NASA projects. However, trade
studies can also be performed in phase B, or whenever a method is needed to select alternatives, such as
selecting test methods, evaluating design change proposals, or performing makeorbuy decisions. A
trade study analysis allows a systematic approach to evaluation of design options with respect to
programmatic considerations or other, nonreliability related considerations (weight, maintainability,
manufacturability). These studies may also be used to help the designer delineate which system require
ments are most important (used in conjunction with the Pareto chart analysis, sec. 5.6).
2.1.3 Procedures
The procedures for performing a weighted trade study are presented below. By performing step
6, an AHP weighted trade study will be performed. These procedures are described in detail and were
adapted from reference 2.1.
(1) Define the mission objectives and requirements for the system under consideration. These
objectives and requirements should be clear, accurate, and specific. These requirements will
provide the scope of the assessment and the basis for the selection criteria. Prioritize the
objectives/requirements if possible; this will aid in the weight factors for the selection criteria.
(2) Identify credible alternative candidates for the system under consideration. These
alternatives can be imposed or obtained in brainstorming sessions (sec. 7.7). The list of
alternatives selected during brainstorming sessions may be reduced by eliminating
alternatives which do not appear capable of meeting requirements. The list may be reduced
further by eliminating alternatives with low probability of successful implementation or
those which are expected to exceed cost constraints. The remaining alternatives should be
described in sufficient detail that the relative merits between them can be ascertained.
(3) Develop a trade tree (optional). A trade tree is developed to graphically illustrate the
alternatives and how highlevel alternatives in the decision process are logically resolved
into decreasingly lower level alternatives. For large trade studies with many alternatives
and criteria attributes, create a trade tree to group alternatives with unique criteria
attributes. A large trade study may be resolved into several smaller trade studies with fewer
required total comparison evaluations. This will lead to fewer resources to conduct the
assessment without degradation of the results.
(4) Develop and specify the selection criteria to be used in the analysis. The selection criteria
are benchmarks to assess the effectiveness and applicability characteristics of the
alternatives to be considered. Ideally, the selection criteria should have the following
characteristics:
a. Be expressed in general terms that mean the same thing to every evaluator.
b. Be practical to measure or predict within acceptable uncertainty and cost limits.
c. Provide a distinction between alternatives without prejudice.
d. Correlate directly to the established requirements and high priority issues. (A
numbering system, showing the specific correlation, is often useful here.)
e. Be separate and independent from each of the other selection criterion in all aspects of
the assessment.
23
(5) Establish weights for the selection criteria. These weights should reflect the importance of
each criterion relative to its importance to the overall selection decision. The weights
should be given numerical values to accommodate objective comparisons between
unrelated criteria. The numerical values of the weight factors should sum to 100. The
weights should be predetermined by the person (or group) with the ultimate decision
authority, but not necessarily shared with the analysts to ensure that alternatives are
assessed against each criterion objectively. Each criterion may be resolved into several
levels of components to establish its weight. The degree to which the individual criterion is
resolved into components is dependent on how effective the criterion components can be
evaluated, and represents the resolution limit of the assessment.
Consult with the end user of the system (the internal or external customer) to verify that the
selection criteria and weights are compatible with his needs.
(6) Perform an analytical hierarchy process as described in reference 2.2 to establish weights
for the selection criteria (optional). This technique is beneficial for very complex trade
studies when operational data are not available and a subjective analysis is to be performed.
The following steps define this process:
a. Establish a scale of the relative level of significance to the system objectives between
two given criteria attributes. Establish three to five definitions to subjectively define
this scale of relative level of significance. Generate clarifications for each definition so
that qualified managers and engineers can subjectively use the definitions. If five
definitions are used, assign the numerical values 1, 3, 5, 7, and 9 to these definitions in
order of increasing diversity between the given two attributes. Reserve the numerical
values of 2, 4, 6, and 8 as values to be assigned when interpolating between two of the
definitions. If attribute n has a numerical value of relative level of significance of “j”
relative to attribute m, then attribute m has a numerical value of relative level of
significance of “1/j” relative to attribute n.
b. Survey a group of qualified managers and engineers (or customers) to establish a
consensus on the relative relationships between each attribute and the rest of the
attributes.
c. Create a normalized matrix (all the attributes versus all the attributes) with these
relationships. Note that all elements of the diagonal of this matrix equal 1.
d. Determine the relative weights for each criterion component by performing an
eigenvector analysis.
e. Determine the weight for all attributes by calculating the product of each individual
attribute weighing factor and its weights of associated category headings.
(7) Generate utility functions (optional). This technique is used to establish a consistent scale
for dissimilar criteria. A relationship is established between a measure of effectiveness for
each selection criterion and a common scale (for example, 0 to 10). The relationship may
be a continuous function (not necessarily a straight line) or discrete values. For attributes
other than technical, such as cost, schedule, risk, etc., a subjective verbal scale may be used
(i.e., high, medium, low).
(8) Assess each alternative relative to the selection criteria. First estimate the performance of
every alternative for a given criterion in terms of the measure of effectiveness used in gen
erating the utility functions. For the ideal situation, the analyst may use test data, vendor pro
vided data, similarity comparison, modeling, engineering experience, parametric
24
analysis, or other costeffective and reliable methods to generate the performance estimates.
In reality, this is often very difficult to perform objectively. It is worthwhile, however, even
when somewhat subjective (i.e., heavy use of engineering experience). If quantification of
qualitative ranking is required, use caution in drawing conclusions. Assume that a difference
in the conclusion of less than onehalf the quantified number of a onestep difference is an
equivalent answer. This corresponds to a confidence band for the evaluation.
Next, determine the score for each alternative relative to a given criterion by correlating the
estimate of performance for all the criteria to the mutual scale using the utility functions
generated in step 7. Next, multiply the scores for all alternatives by the weight factor for the
criterion (determined in steps 5 or 6) to determine the weighted score for all alternatives for
that criterion. Repeat this procedure for all criteria attributes.
(9) Tabulate the results. Generate a matrix of criteria versus alternatives to summarize the
results from the preceding steps. A typical table is illustrated in table 22 and was
generalized from an example presented in reference 2.1.
Table 22. Typical weighted trade study summary table.
2.1
Criteria Alternates, x
i
(x
1
through x
n
)
Criterion,
y
j
Weights
w
j
= 100 Alternate x
1
Alternate x
2
Alternate x
3
Through
Alternate x
n–1
Alternate x
n
Score
(0–10)
Weighted
Score
Score
(0–10)
Weighted
Score
Score
(0–10)
Weighted
Score
Score
(0–10)
Weighted
Score
y
1
w
1
s
11
w
1
s
11
s
21
w
1
s
21
s
n1
w
1
s
n1
y
2
w
2
s
12
w
2
s
12
s
22
w
2
s
22
s
n2
w
2
s
n2
y
3
w
3
s
13
w
2
s
13
s
23
w
2
s
23
s
n3
w
3
s
n3
y
4
to y
m–1
y
m
w
m
s
1m
w
m
s
1m
s
2m
w
2
s
2m
s
nm
w
m
s
nm
Total ∑(w
j
s
ij
) ∑(w
j
s
ij
)
j
∑(w
j
s
ij
)
j
(10) Perform a sensitivity analysis to evaluate the merit of the results relative to making an alter
nate selection. Examine the results of the weighted trade study to see if any total weighted
scores of any alternatives are closer in numerical value than is warranted in making a deci
sion due to the confidence levels of the performance estimates that had been used to estab
lished the scores. If this is the case, then gather more data to increase the confidence level of
the performance estimates, repeat the assessment, and regenerate the summary table for the
weighted trade study. If, after the analysis is repeated, the alternative numerical total score is
too close to make a decision, reconsider the selection criterion and weighting factors.
(11) Select the superior alternative. Select the alternative with the highest value of total
weighted scores.
25
2.1.4 Example
Problem:
Four alternatives for a new automobile design are being considered. The selection decision will
be based on comparing the four alternatives to the following criteria attributes and their associated
weight factors:
Item Criteria Attribute Weight Factor
1 Average fuel economy 20
2 Acceleration (0 to 60 mph) 15
3 Braking (70 to 0 mph) 15
4 Road handling 15
5 Implement new technology
risk
10
6 Cost 25
Total 100
Utility functions have been generated for each criteria attribute and are presented in figure 21.
The estimates for each alternative relative to each criteria attribute are listed below:
Item Criteria attribute
Measure of
Effectiveness
Alt.
A
Alt.
B
Alt.
C
Alt.
D
1 Average fuel economy miles per gallon 16 19 23 18
2 Acceleration (0 to 60 mph) seconds 7 9 10 12
3 Braking (70 to 0 mph) feet 180 177 190 197
4 Road handling
(300 ft dia. skidpad)
g 0.86 0.88 0.83 0.78
5 Implementing new
technology risks
 Low Avg. High Very
low
6 Cost Dollars, × 1,000 21 20 24 22
From the information given above, formulate a weighted factor trade study summary table, and
select the superior alternative.
26
10
5
0
Average Fuel Economy, mpg
25
Acceleration (0 to 60 mph), seconds
Implementing New Technology Risks
Road Handling, g
Cost, dollars × 1,000
15
10
5
0
10
5
0
10
5
0
10
5
0
5 15
0.7 0.9 15 25
Very
High
Braking (70 to 0 mph), feet
10
5
0
175 200
Very
Low
Avg.
Score
Score
Score
Score
Score Score
Figure 21. Example utility functions.
27
Solution:
Presented in figure 22 is the completed weighted factor trade study summary. Scores were
determined from effectiveness measures for all alternatives relative to all criteria attributes and the utility
functions. Based on the results of the trade study, alternative B is the preferred option.
Criteria Alternates, x
i
Attribute
Item
Weights
w
j
=100 Alternate A Alternate B Alternate C Alternate D
Score
(0–10)
Weighted
Score
Score
(0–10)
Weighted
Score
Score
(0–10)
Weighted
Score
Score
(0–10)
Weighted
Score
1 20 1 20 4 80 8 160 3 60
2 15 9 135 8 120 7.5 112.5 5 75
3 15 9.8 147 9.9 148.5 8.5 127.5 5 75
4 15 4.5 67.5 7 105 2.5 37.5 1.5 22.5
5 10 8 80 6 60 4 40 10 100
6 25 4 100 5 125 1 25 3 75
Total 549.5 638.5 502.5 407.5
Figure 22. Example weighted factor trade study summary table.
2.1.5 Advantages
The following advantages can be realized from performing trade studies:
(1) Different kinds and/or levels of study allow flexibility in the depth of the review, i.e.,
resources expended can be commensurate with the benefits of the task.
(2) This technique is adaptive to prioritization based upon programmatic considerations (cost,
schedule) as well as technical ones (weight, reliability, etc.).
(3) Identification of disadvantages of a specific design option may lead to the definition of
effective countermeasures if combined with other techniques.
(4) The method provides a clearly documented analysis in which the (a) prioritized objectives
and requirements, (b) considered alternatives, and (c) selection methodology are recorded.
28
2.1.6 Limitations
The following limitations are associated with performing trade studies:
(1) These techniques are very dependent upon the expertise of the analyst and the amount of
available accurate quantitative data.
2.1
(2) Improper generation of selection criteria, weight factors, and utility functions can prejudice
the assessment and lead to incorrect results.
2.1
(3) The number of alternatives which can be considered is limited by the expenditure of
resources to perform the analysis.
2.1
(4) Options evaluated are not determined as a result of the study but must be decided upon
prior to the assessment by the person (or group) with decision authority.
(5) Weighting factors and advantages/disadvantages are very subjective (although objective
data may be added in the analytical hierarchy process approach (AHPA), this significantly
complicates and enlarges the study) and this subjectivism significantly influences the study
conclusions.
2.1.7 Bibliography
Blanchard, B.S., and Fabreycky, W.J.: “System Engineering and Analysis.” Second edition, Englewood
Cliffs, Prentice Hall, New Jersey, pp. 67–72, 1990.
Cross, N.: “Engineering Design Methods.” John Wiley & Sons, pp. 101–121, 1989.
Saate, T. L.: “Analytical Hierarchy Process.” McGrawHill, 1980.
2.2 COSTVERSUSBENEFIT STUDIES
2.2.1 Description
Costversusbenefit studies are also known as benefitcost analyses,
2.32.4
benefitcost ratio
analyses,
2.5
and costbenefit analyses.
2.6
Costversusbenefit studies, as described in reference 2.5, pro
vide a method to assess alternates by determining the ratio of expected future benefits to expected future
costs. Both the expected future benefits and costs are expressed in terms of present value. The
alternatives are ranked in decreasing order with the preferred option being the alternative with the
highest benefittocost (B/C) ratio, while falling within overall cost restraints.
2.2.2 Application
Benefitcost analyses apply to the selection of projects
2.32.4
or machines or systems
2.52.6
based on
their relative B/C ratios. Costversusbenefit studies, as discussed in this section, will apply to the
selection of system or system element alternatives based on their relative B/C ratios. These studies
should typically be performed in phase A, however, they could also be performed in phases B or C.
These studies can be used when two or more alternatives are being considered with fixed cost
constraints, fixed desired results or benefits, or when both costs and desired results vary.
29
2.2.3 Procedures
The following procedures to perform costversusbenefit studies were adapted from references
2.3, 2.5, and 2.6.
(1) Define the requirements for the system or system element under consideration. These
requirements should be measurable and verifiable. Translate general and vague
requirements into specific, quantitative requirements in which system effectiveness can be
measured and assessed.
2.6
Prioritize these requirements, if possible.
(2) Define a list of credible, mutually exclusive alternatives; that is, if one alternative is selected,
the others are not to be implemented.
2.3
Each alternative should be characterized to a level of
completeness such that all substantial costs and benefits can be identified.
2.6
Note that the
alternatives require an implicit determination of technical and schedule viability.
(3) Develop and specify the selection criteria to be used in the analysis. The example selection
criteria presented in table 23 were adapted from reference 2.5.
Table 23. Example selection criteria for costversusbenefit analyses.
Condition or Circumstance Selection Criteria
Budget C is fixed Maximum B/C ratio.
Desired result B is fixed. Maximum B/C ratio.
Two alternatives are being considered with
neither budget C or desired result B fixed.
Calculate the ∆Bto∆C ratio between the
alternatives. Choose the lower cost
alternative, unless the ∆Bto∆C ratio is
≥1. Then choose the higher cost
alternative.
More than two alternatives are being
considered with neither budget C or desired
result B fixed.
Select alternative per benefitcost ratio
incremental analysis (sec. 2.2.3,
step 11).
(4) Identify the cost or savings for each alternative. The cost should include such items as initial
investment, and ongoing operating and maintenance expenses (including depreciation) for the
life of the system. The savings should include such items as residual or salvage values, etc.
2.3
(5) Identify the benefit or detriments for each alternative. The benefits might include such
items as increased performance, reduced operating times, compressed schedules, increased
reliability, increased safety, value added due to increase productivity, etc. The detriments
might include such items as loss of production time, increased schedules, increased
equipment operating costs, environmental impacts, reduced property value, etc.
2.3
The cost
risk and technical maturity for each alternative may be included as a multiplying factor (f)
for this analysis. Since it is subjective, use of only three factors—0.5, 0.75, or 1—is
probably as fine a distinction as is warranted.
(6) Specify the time interval (expected operating life of the system) to which the analysis is to
apply.
210
(7) Develop cost and savings estimates and the benefits and detriments estimates for each
alternative.
2.3
The estimates for each alternative should be for the same time interval
specified in step 6. Every attempt should be made to base cost and savings estimates on
actual historical cost data.
(8) Identify the interest rate that will be assumed for the analysis.
2.3
Convert all costs, savings,
benefits, and detriments estimates to present worth values.
2.5
(9) Determine the total cost for each alternative by algebraically summing all costs as positive
values and all savings as negative values.
(10) Determine the total benefit value for each alternative by algebraically summing all benefits
as positive values and all detriments as negative values.
(11) Calculate the B/C ratio for each alternative by dividing the total benefit (B) by the total cost
(C).
For cases with fixed cost restraints or fixed desired results or benefits, perform step 12.
2.5
(12) Rank the alternatives relative to their respective costtobenefit ratios, and select the
superior alternative based on selection criteria established in step 3.
For cases with cost restraints and desired results or benefits that vary, perform steps 13
through 17.
2.5
(13) If there exists any alternatives with a B/C ≥1, then do not give further consideration to
alternatives with a B/C <1.
(14) Order the remaining alternatives in sequence of increasing total C.
(15) Determine the incremental B/C ratio ∆B/C for each consecutive pair of alternatives with
increasing total cost.
∆B/C
i
= ∆B
i
/∆ C
i
, where ∆B
i
= B
i +1
– B
i
and ∆C
i
= C
i+1
– C
i
for each i
th
pair of (n – 1) pairs of n alternatives where alternative i = 1,2,...,n listed in order
of C.
(16) Next, examine each distinct increment of increased cost investment. If the ∆B/C
i
is <1, then
the increment is not beneficial. If the ∆B/C
i
is >1, then the increment is beneficial.
(17) The preferred alternative is the last alternate listed in order of increasing cost whose
incremental ∆B/C
i
is >1. Therefore, the preferred alternative may not necessarily have the
greatest B/C ratio.
2.2.4 Example
Problem:
Five data acquisition systems (DAS) are under consideration to acquire data for solid rocket
motor tests in a test stand over a 10yr time interval. Each system has a different total cost and the
capabilities of each system are different in terms of maximum number of channels, maximum sample
211
rates, required maintenance, data accuracy, turnaround time between tests, and mean time between
system failures. The present value of the estimated total cost and total value of combined benefits of the
system are presented below. The present values of cost and benefits were determined over a 10yr
expected system life, with an assumed annual interest rate of 10 percent. Perform a costversusbenefit
analysis to determine the best alternative.
System A B C D E
Total cost (dollars) 500k 300k 750k 800k 400k
Total benefits (dollars) 750k 400k 900k 750k 600k
B/C 1.50 1.33 1.20 0.93 1.50
Solution:
Step 1. Delete options with a B/C ratio <1. Since the B/C for system D is <1, this option will no
longer be considered.
Step 2. List the remaining options in order of increasing total cost.
System B E A C
Total cost (dollars) 300k 400k 500k 750k
Total benefits (dollars) 400k 600k 750k 900k
Step 3. Determine the incremental B/C ratio ∆B/C for each consecutive pair of alternatives with
increasing total cost.
Increment E–B A–E C–A
∆ Total cost (dollars) 100k 100k 250k
∆ Total benefits (dollars) 200k 150k 150k
∆ B/C 2.0 1.5 0.6
Step 4. Identify the preferred alternative as the last alternate listed in order of increasing cost
whose incremental ∆B/C
i
is >1.
By inspection, the last incremental ∆B/C with a value >1 is A–E. Therefore, the preferred
alternative is DAS A.
212
2.2.5 Advantages
The following advantages are realized by performing costversusbenefit analyses:
(1) The analyst can assess the cost effectiveness of several alternatives over the entire life cycle
of the proposed system under consideration.
(2) The method provides a clearly documented analysis in which the prioritized
objectives/requirements, the alternatives considered, and the selection methodology are
recorded.
2.2.6 Limitations
Costversusbenefit analyses possess the following limitations:
(1) The analysis is flawed if system requirements are incomplete or inadequate. If the system
operating environment is not understood or accurately characterized, the total costs can be
underestimated. If the system requirements are too general or vague, benefits cannot be
addressed in specific, measurable terms of effectiveness.
2.5
(2) The analysis is only as good as the list of alternatives considered. An incomplete list of
alternatives will lead to an incomplete analysis.
2.6
(3) The analysis is flawed if incomplete or inaccurate cost estimates are used.
2.6
(4) The analyst must be able to quantify the value of benefits, which are often intangible or
insubstantial and difficult to characterize in terms of monetary value.
2.3
(5) The analysis does not take into account technical complexity or maturity of an alternative,
except as a cost uncertainty factor. Further, system reliability and safety issues are not
treated except by the selection of the alternative. As cost is generally only one of many
factors, this tool is generally insufficient for selection of large, new design efforts, but more
appropriate to productionlevel design solutions.
2.2.7 Bibliography
Thuesen, G.J., and Fabrycky, W.J.: “Engineering Economy.” Seventh edition, Englewood Cliffs,
Prentice Hall, New Jersey, 1989.
213
REFERENCES
2.1 “System Engineering Management Guide.” Defense Systems Management College, January 1990.
2.2 Saaty, T.L.: “Priority Setting in Complex Problems.” IEEE Transactions, pp. 140–155, August
1983.
2.3 Jelen, F.C., and Black, J.H.: “Cost and Optimization Engineering.” Second edition, New York,
McGrawHill Book Company, 1983.
2.4 Beattie, C.J., and Reader, R.D.: “Quantitative Management in R & D.” London, Chaplin and Hall
Ltd., 1971.
2.5 Newnan, D.G.: “Engineering Economic Analysis.” Second edition, Engineering Press, Inc., San
Jose, California, 1983.
2.6 Chestnut, H.: “System Engineering Methods.” John Wiley & Sons Inc., New York, 1967.
214
3. SYSTEM SAFETY AND RELIABILITY TOOLS
This section describes several system safety and reliability tools available to the system engineer
analyst. The risk assessment matrix is discussed in section 3.1. This device supports a standard
methodology to subjectively evaluate hazards as to their risks. It is used in conjunction with hazard
analyses, such as the preliminary hazard analysis (PHA) technique discussed in section 3.2. The PHA
can be used to identify hazards and to guide development of countermeasures to mitigate the risk posed
by these hazards. The energy/flow barrier analysis discussed in section 3.3 is also a technique to identify
hazards and to evaluate their corresponding countermeasures.
Once hazards are identified, they can be further explored if failure modes of the elements of the
system are known. The failure modes and effects analysis (FMEA), discussed in section 3.4, can be used
to identify failure modes and their consequences or effects. Also discussed in section 3.4 is the failure
modes, effects, and criticality analysis (FMECA). The FMECA is similar to the FMEA but also
addresses the criticality, or risk, associated with each failure mode.
Several symbolic logic methods are presented in this section. These methods construct
conceptual models of failure or success mechanisms within a system. These tools are also used to
determine either the probability of failures occurring or the probability that a system or component will
operate successfully. The probability of a successful operation is the reliability. If the probability of
failure (P
F
) is examined, then the model is generated in the failure domain and if the probability of
success (P
S
) is examined, then the model is generated in the success domain. For convenience, the
analyst can model either in the failure or success domain (or both domains), then convert the final
probabilities to the desired domain using the following expression: P
F
+ P
S
= 1.
These models are developed using forward (bottomup) or backwards (topdown) logic. When
using forward logic the analyst builds the model by repeatedly asking, “What happens when a given fail
ure occurs?” The analyst views the system from a “bottomup” perspective. This means he starts by
looking at the lowest level elements in the system and their functions. Classically, the FMEA, for exam
ple, is a bottomup technique. When using backwards logic to build a model, the analyst repeatedly asks,
“What will cause a given failure to occur?” The analyst views the system from a “topdown”
perspective. This means he starts by looking at a high level system failure and proceeds down into the
system to trace failure paths. The symbolic logic techniques discussed in this section and their
characteristics are presented in table 31.
Each of the symbolic logic techniques has its own unique advantages and disadvantages.
Sometimes it is beneficial to construct a model using one technique, then transform that model into the
domain of another technique to exploit the advantages of both techniques. Fault trees are generated in
the failure domain, reliability diagrams are generated in the success domain, and event trees are
generated both in the success and failure domains. Methods are presented in section 3.9 to transform any
one of the above models into the other two by translating equivalent logic from the success to failure or
failure to success domains.
Probabilities are propagated through the logic models to determine the probability that a system
will fail or the probability the system will operate successfully, i.e., the reliability. Probability data may
be derived from available empirical data or found in handbooks. If quantitative data are not available,
then subjective probability estimates may be used as described in section 3.12. Caution must be
exercised when quoting reliability numbers. Use of confidence bands is important. Often the value is in a
comparison of numbers that allows effective resource allocation, rather than “exact” determination of
31
Table 31. Symbolic logic techniques.
Technique Section Success
Domain
Failure
Domain
Forward
(BottomUp)
Backwards
(TopDown)
Reliability block
diagram
3.5 √ √
Fault tree analysis 3.6 √ √
Success tree analysis 3.7 √ √
Event tree analysis 3.8 √ √ √
Causeconsequence
analysis
3.10 √ √ √ √
Directed graph matrix
analysis
3.11 √ √ √
expected reliability levels. Probabilistic design analysis (PDA) is discuss in section 3.14. This technique
uses advanced statistical methods to determine P
F
modes.
Failure mode information propagation modeling is discussed in section 3.13. This technique
allows the analyst to determine what information is needed, and how and where the information should
be measured in a system to detect the onset of a failure mode that could damage the system.
Finally, probabilistic risk assessment (PRA) is discussed in section 3.15. This is a general
methodology that shows how most of the techniques mentioned above can be used in conjunction to
assess risk with severity and probability.
A summary of the major advantages and limitations of each tool or methodology discussed in
this section is presented in table 32.
3.1 Risk Assessment Matrix
3.1.1 Description
The risk assessment matrix, as described in reference 3.1, is a tool to conduct subjective risk
assessments for use in hazard analysis. The definition of risk and the principle of the isorisk contour are
the basis for this technique.
Risk for a given hazard can be expressed in terms of an expectation of loss, the combined
severity and probability of loss, or the longterm rate of loss. Risk is the product of severity and
probability (loss events per unit time or activity). Note: the probability component of risk must be
attached to an exposure time interval.
The severity and probability dimensions of risk define a risk plane. As shown in figure 31, iso
risk contours depict constant risk within the plane.
32
Table 32. System safety and reliability tools and methodologies—Continued
Tool or Methodology Section Advantages Limitations
Risk assessment matrix 3.1 Provides standard tool to subjectively assess risk. Only used to assess risk of hazards, does not identify
hazards.
Preliminary hazard analysis 3.2 Identifies and provides inventory of hazards and
countermeasures.
Does not address coexisting system failure modes.
Energy flow/barrier analysis 3.3 Identify hazards associated with energy sources and
determines if barriers are adequate countermeasures.
(1) Does not address coexisting system failure modes.
(2) Fails to identify certain classes of hazards, e.g.,
asphyxia in oxygendeficient confined spaces.
Failure modes and effects (and
criticality) analysis
3.4 Thorough methods of identifying single point failures
and their consequences. A criticality analysis provides a
risk assessment of these failure modes.
Can be extremely labor intense. Does not address
coexisting system failure modes.
Reliability block diagram 3.5 A symbolic logic model that is relatively easy for the
analyst to construct. System reliability can be derived,
given component reliability.
Component reliability estimates may not be readily
available; total calculated reliability may be
unrealistically high.
Fault tree analysis 3.6 (1) Enables assessment of probabilities of coexisting
faults or failures.
(2) May identify unnecessary design elements.
(1) Addresses only one undesirable event or condition
that must be foreseen by the analyst.
(2) Comprehensive trees may be very large and
cumbersome.
Success tree analysis 3.7 Assesses probability of favorable outcome of system
operation.
(1) Addresses only one desirable event or condition that
must be foreseen by the analyst.
(2) Comprehensive trees may be very large and
cumbersome.
Event tree analysis 3.8 (1) Enables assessment of probabilities of coexisting
faults or failures.
(2) Functions simultaneously in failure and success
domain.
(3) End events need not be anticipated. Accident
sequences through a system can be identified.
(1) Addresses only one initiating challenge that must
be foreseen by the analyst.
(2) Discrete levels of success and failure are not
distinguishable.
Fault tree, reliability, block
diagram, and event tree
transformations
3.9 Allows the analyst to overcome weakness of one
technique by transforming a model of a system into an
equivalent logic model in another analysis technique.
This technique offers no additional information and is
only as good as the input model.
33
Table 32. System safety and reliability tools and methodologies—Continued.
Tool or Methodology Section Advantages Limitations
Causeconsequence analysis 3.10 (1) Enables assessment of probabilities of coexisting
faults or failures.
(2) End events need not be anticipated.
(3) Discrete levels of success and failure are
distinguishable.
(1) Addresses only one initiating challenge that must
be foreseen by the analyst.
(2) May be very subjective as to consequence severity.
Directed graph (digraph) matrix
analysis
3.11 (1) Allows the analyst to examine the fault propagation
through several primary and support systems.
(2) Minimal cut sets, singlepoint failure, and double
point failures can be determined with less computer
computation than with FTA.
(1) Trained analyst and computer codes and resources to
perform this technique may be limited.
(2) Only identifies single point (singleton) and dual
points (doubleton) of failure.
Combinatorial failure
probability analysis using
subjective information
3.12 Allows analyst to perform qualitative probabilistic risk
assessment based on the exercise of subjective
engineering judgment when no quantitative data is
available.
Use of actual quantitative data is preferred to this
method. Should only be used when actual quantitative
failure data is unavailable.
Failure mode information
propagation modeling
3.13 Measurement requirements can be determined that if
implemented can help safeguard a system in operation
by providing warning at the onset of a threatening
failure mode.
(1) This technique is only applicable if the system is
operating in a near normal range and for the instant
of time just before initiation of a failure.
(2) Data and results, unless used in a comparative
fashion, may be poorly received.
Probabilistic design analysis 3.14 (1) Allows the analyst a practical method of
quantitatively and statistically estimating the
reliability of a system during the design phase.
(2) Provides alternative to the traditional method of
imposing safety factors and margins to ensure
system reliability. That method might be flawed if
significant experience and historical data of similar
components are not available.
(1) Analyst must have significant experience in
probability and statistical methods to apply this
technique.
(2) Historical population data used must very close to
asplanned design population to be viable.
Extrapolation between populations can render
technique nonviable.
Probabilistic risk assessment 3.15 Provides methodology to assess overall system risks;
avoids accepting unknown, intolerable, and senseless
risk.
Performing the techniques of this methodology requires
skilled analysts. Techniques can be misapplied and
results misinterpreted.
34
PROBABILITY
S
E
V
E
R
I
T
Y L
I
K
E
L
Y
0
NEVER
CATACLYSMIC
I
n
c
r
e
a
s
i
n
g
R
i
s
k
Isorisk
Contours
R = P x S = K
1
R = K
2
> K
1
R = K
3
> K
2
RISK
is
CONSTANT
along any
ISORISK
CONTOUR.
PROBABILITY
is a function of
EXPOSURE
INTERVAL.
SEVERITY
and
PROBABILITY,
the
two variables
that
constitute risk,
define a
RISK PLANE.
Figure 31. Risk plane.
The concept of the isorisk contour is useful to provide guides, convention, and acceptance limits
for risk assessments (fig. 32).
Risk should be evaluated for worst credible case, not worst conceivable case, conditions. Failure
to assume credible (even if conceivable is substituted) may result in an optimistic analysis; it will result
in a nonviable analysis.
3.1.2 Application
The risk assessment matrix is typically performed in phase C but may also be performed in phase
A. This technique is used as a predetermined guide or criteria to evaluate identified hazards as to their
risks. These risks are expressed in terms of severity and probability. Use of this tool allows an organiza
tion to institute and standardize the approach to perform hazard analyses. The PHA, defined in section
3.2, is such an analysis.
35
PROBABILITY
S
E
V
E
R
I
T
Y
0
0
RISK ASSESSMENT
GUIDES: If Risk for a
given Hazard can be
assessed at any severity
level, an isorisk contour
gives its probability at all
severity levels. (Most,
but not all hazards
behave this way. Be wary
of exceptions — usually
highenergy cases.)
RISK ASSESSMENT
CONVENTION: If
possible, assess Risk for
the Worst Credible
Severity of outcome. (It’ll
fall at the top end of its
own isorisk contour.)
ACCEPTABLE
(de minimis)
PROVISIONALLY
ACCEPTABLE
NOT
ACCEPTABLE
1
2
ACCEPTANCE: Risk
Tolerance Boundaries
follow isorisk contours.
3
Figure 32. Isorisk contour usage.
3.1.3 Procedures
Procedures, as described in reference 3.1, for developing a risk assessment matrix are presented
below:
(1) Categorize and scale the subjective probability levels for all targets, such as frequent,
probable, occasional, remote, improbable, and impossible (adapted from MIL–STD–
882C).
3.2
Note: A target is defined as the “what” which is at risk. One typical breakout of
targets is personnel, equipment, downtime, product loss, and environmental effects.
(2) Categorize and scale the subjective severity levels for each target, such as catastrophic,
critical, marginal, and negligible.
(3) Create a matrix of consequence severity versus the probability of the mishap. Approximate
the continuous, isorisk contour functions in the risk plane with matrix cells (fig. 33).
These matrix cells fix the limits of risk tolerance zones. Note that not the analyst but man
agement establishes and approves the risk tolerance boundaries.
(4) The following hints will be of help when creating the matrix:
a. Increase adjacent probability steps by orders of magnitude. The lowest step,
“impossible,” is an exception (fig. 34(a)).
b. Avoid creating too many matrix cells. Since the assessment is subjective, too many
steps add confusion with no additional resolution (fig. 34(b)).
36
“Zoning” the Risk Plane into
judgmentally tractable cells
produces a Matrix.
Matrix cells approximate the
continuous, isorisk contour
functions in the Risk Plane.
Steps in the Matrix define Risk
Tolerance Boundaries.
PROBABILITY
S
E
V
E
R
I
T
Y
PROBABILITY
S
E
V
E
R
I
T
Y
A B C D E F
I
II
III
IV
Figure 33. Risk plane to risk matrix transformation.
c. Avoid discontinuities in establishing the risk zones, i.e., make sure every onestep path
does not pass through more than one zone (fig. 34(c)).
d. Establish only a few risk zones. There should only be as many zones, as there are
desired categories of resolution to risk issues, i.e., (1) unacceptable, (2) accepted by
waiver, and (3) routinely accepted (fig. 34(d)).
(5) Calibrate the risk matrix by selecting a cell and attaching a practical hazard scenario to it.
The scenario should be familiar to potential analysts or characterize a tolerable perceivable
threat. Assign its risk to the highest level severity cell just inside the acceptable risk zone.
This calibration point should be used as a benchmark to aid in evaluating other, less
familiar risks.
37
PROBABILITY
S
E
V
E
R
I
T
Y
1
2
3
A B C D E F
I
II
III
IV
Factors of 10 separate
adjacent Probability Steps.
D = 10 E
C = 10 D
B = 10 C
A = 10 B
…but F = 0 (“Impossible”)
(a) Useful conventions.
PROBABILITY
S
E
V
E
R
I
T
Y
1
2
3
A B C D E F
I
II
III
IV
PREFERRED
A 24cell Matrix can be resolved
into 9 levels of “priority,” or even
more. But what are the rational
functions for the many levels?
Three zones will usually suffice.
A Hazard’s Risk is either…
• (3) Routinely Accepted
• (2) Accepted by Waiver, or
• (1) Avoided.
PROBABILITY
S
E
V
E
R
I
T
Y
1
A B C D E F
I
II
III
IV
4
2 3 6
7
8
8
6
9 7
2 3
4
5 4 4
5
5
FLAWED
(b) Do not create too many cells.
Figure 34. Helpful hints in creating a risk assessment matrix—Continued
38
PROBABILITY
S
E
V
E
R
I
T
Y
1
2
3
A B C D E F
I
II
III
IV
Can a countermeasure make the
“leap” from Zone (1) to Zone (3)
in a single step?
PREFERRED
Make every onestep path
from a high Risk Zone (1)
to a lower Risk Zone (3)
pass through the
intermediate Zones (2).
PROBABILITY
S
E
V
E
R
I
T
Y
1
2
3
A B C D E F
I
II
III
IV
FLAWED
?
?
(c) Avoid discontinuities.
PROBABILITY
S
E
V
E
R
I
T
Y
1
2
3
A B C D E F
I
II
III
IV
PROBABILITY
S
E
V
E
R
I
T
Y
A B C D E F G K L H I J
I
II
III
IV
V
VI
VII
1
2
3
Subjective judgment can’t
readily resolve more than six
discrete probability steps.
Added steps become
confused/meaningless.
Keep it SIMPLE!
4 6 = 24 cells
is better than
7 12 = 84 cells PREFERRED
FLAWED
(d) Do not create too many zones.
Figure 34. Helpful hints in creating a risk assessment matrix—Continued.
39
3.1.4 Example
A typical risk assessment matrix, adapted from MIL–STD–882C,
3.2
is presented in figure 35. Example
interpretations of the severity and probability steps for this matrix are presented in figure 36.
I
CATASTROPHIC
III
MARGINAL
IV
NEGLIGIBLE
Severity
of
Consequences
II
CRITICAL
Probability of Mishap**
A
FREQUENT
C
OCCASIONAL
F
IMPOSSIBLE
E
IMPROBABLE
D
REMOTE
B
PROBABLE
3
2
1
*Adapted from MILSTD882C **Life Cycle = 25 yrs.
Risk Code/ Actions
NOTE: Personnel must not be exposed to hazards in Risk Zones 1 and 2.
3
Operation permissible.
2
Operation requires written, timelimited waiver, endorsed
by management.
1 Imperative to suppress risk to lower level.
Figure 35. Typical risk assessment matrix.
310
Provide
stepwise
scaling of
SEVERITY
levels for
each
TARGET.
Decide on
TARGETS.
Provide
stepwise
scaling of
PROBABILITY
levels for
all
TARGETS.
PROBABILITY
is a function of
EXPOSURE
INTERVAL.
Longterm (5 yrs or
greater) environ
mental damage or
requiring >$1M to
correct and/or in
penalties
Mediumterm (15
yrs) environmental
damage or requir
ing $250K$1M to
correct and/or in
penalties
Shortterm (<1 yr)
environmental dam
age or requiring
$1K$250K to cor
rect and/or in pen
alties
Minor environment
al damage, readily
repaired and/or
requiring <$1K to
correct and/or in
penalties
DESCRIPTIVE
WORD
DEFINITION
Probability of Mishap**
LEVEL
DOWN
TIME
>4 months
2 weeks
to
4 months
1 day
to
2 weeks
<1 day
EQUIPMENT
LOSS ($)**
>1M
250K
to
1M
1K
to
250K
<1K
PERSONNEL
ILLNESS/
INJURY
Death
Severe injury or
severe
occupational
illness
Minor
injury or
minor
occupation
al illness
No injury or
illness
CATEGORY/
DESCRIPTIVE
WORD
I
CATASTROPHIC
II
CRITICAL
III
MARGINAL
IV
NEGLIGIBLE
Severity of Consequences
A
B
C
D
E
F
Likely to occur
repeatedly in system
life cycle
Physically impossible
to occur
IMPOSSIBLE
IMPROBABLE
REMOTE
OCCASIONAL
PROBABLE
FREQUENT
PRODUCT
LOSS
Values as
for loss
Equipment
Loss
ENVIRONMENTAL
EFFECT
*Adapted from MILSTD882C **Life Cycle = 25 yrs.
Probability of
occurrence cannot be
distinguished from
zero
Likely to occur several
times in system life
cycle
Likely to occur
sometime in system
life cycle
Not likely to occur
repeatedly in system
life cycle, but possible
Figure 36. Severity and probability interpretations.
311
3.1.5 Advantages
The risk assessment matrix provides the following advantages:
3.1
(1) The risk matrix provides a useful guide for prudent engineering.
(2) The risk matrix provides a standard tool of treating the relationship between severity and
probability in assessing risk for a given hazard.
(3) Assessing risk subjectively avoids unknowingly accepting intolerable and senseless risk,
allows operating decisions to be made, and improves resource distribution for mitigation of
loss resources.
3.1.6 Limitations
The risk assessment matrix possesses the following limitations:
3.1
(1) The risk assessment matrix can only be used if hazards are already identified. This tool
does not assist the analyst in identifying hazards.
(2) This method is subjective without data and is a comparative analysis only.
3.1.7 Bibliography
Code of Federal Regulations, Medical devices, “PreMarket Notification.” sec. 807.90,
vol. 21.
Code of Federal Regulations, “Process Safety Management of Highly Hazardous Chemicals.” sec.
1910.119 (e), vol. 29.
Department of Defense Instruction, No. 5000.36. “System Safety Engineering & Management.”
NASA NHB 1700.1, vol. 3,“System Safety.”
NUREG/GR0005. “RiskBased Inspection – Development of Guidelines.”
3.2 Preliminary Hazard Analysis
3.2.1 Description
A PHA, as described in reference 3.3, produces a line item tabular inventory of nontrivial system
hazards, and an assessment of their remaining risk after countermeasures have been imposed. This
inventory includes qualitative, not quantitative, assessments of risks. Also, often included is a tabular
listing of countermeasures with a qualitative delineation of their predicted effectiveness. A PHA is an
early or initial system safety study of system hazards.
312
3.2.2 Application
PHA’s are best applied in phase C but may also be applied in phase B. This tool is applied to cover
wholesystem and interface hazards for all mission phases. A PHA may be carried out, however, at any
point in the life cycle of a system. This tool allows early definition of the countermeasure type and
incorporation of design countermeasures as appropriate.
3.2.3 Procedures
A flowchart describing the process to perform a PHA is presented in figure 37. Procedures for
performing PHA’s, as described in reference 3.3, are presented below:
(1) Identify resources of value, such as personnel, facilities, equipment, productivity, mission
or test objectives, environment, etc. to be protected. These resources are targets.
(2) Identify and observe the levels of acceptable risk that have been predetermined and
approved by management. These limits may be the risk matrix boundaries defined in a risk
assessment matrix (sec. 3.1).
(3) Define the extent of the system to be assessed. Define the physical boundaries and
operating phases (such as shakedown, activation, standard operation, emergency shutdown,
maintenance, deactivation, etc.). State other assumptions, such as if the assessment is
based on an asbuilt or asdesigned system, or whether current installed countermeasures
will be considered.
(4) Detect and confirm hazards to the system. Identify the targets threatened by each hazard. A
hazard is defined as an activity or circumstance posing “a potential of loss or harm” to a
target and is a condition required for an “undesired loss event.”
3.3
Hazards should be
distinguished from consequences and considered in terms of a source (hazard), mechanism
(process), and outcome (consequence). A team approach to identifying hazards, such as
brainstorming (sec. 7.7), is recommended over a single analyst. If schedule and resource
restraints are considerations, then a proficient engineer with knowledge of the system
should identify the hazards, but that assessment should be reviewed by a peer. A list of
proven methods
*
for finding hazards is presented below:
a. Use intuitive “engineering sense.”
b. Examine and inspect similar facilities or systems and interview workers assigned to
those facilities or systems.
c. Examine system specifications and expectations.
d. Review codes, regulations, and consensus standards.
e. Interview current or intended system users or operators.
f. Consult checklists (app. D).
g. Review system safety studies from other similar systems.
*
Provided courtesy of Sverdrup Technology, Inc., Tullahoma, Tennessee.
3.3
313
Figure 37. PHA Process flowchart
314
h. Review historical documents—mishap files, nearmiss reports, OSHArecordable injury
rates, National Safety Council data, manufacturer’s reliability analyses, etc.
i. Consider “external influences” like local weather, environment, or personnel
tendencies.
j. Consider all mission phases.
k. Consider “common causes.” A common cause is a circumstance or environmental
condition that, if it exists, will induce two or more fault/failure conditions within a
system.
l. Brainstorm (sec. 7.7)—mentally develop credible problems and play “whatif” games.
m. Consider all energy sources. What is necessary to keep them under control; what
happens if they get out of control?
(5) Assess worstcredible case (not the worstconceivable case) severity and probability for
each hazard and target combination. Keep the following considerations in mind during the
evaluation:
a. Remember that severity for a specific hazard varies as a function of targets and
operational phases.
b. A probability interval must be established before probability can be determined. This
interval can be in terms of time, or number of cycles or operations.
c. The assessment will underestimate the true risk if a shortterm probability interval is
used unless the risk acceptance criterion is adjusted accordingly. Probability intervals
expressed in hours, days, weeks, or months are too brief to be practical. The interval
should depict the estimated facility, equipment, or each human operator working life
span. An interval of 25 to 30 yr is typically used and represents a practical value.
d. The probability for a specific hazard varies as a function of exposure time, target,
population, and operational phase.
e. Since probability is determined in a subjective manner, draw on the experience of
several experts as opposed to a single analyst.
(6) Assess risk for each hazard using a risk assessment matrix (sec. 3.1). The matrix should be
consistent with the established probability interval and force or fleet size for this
assessment.
(7) Categorize each identified risk as acceptable or unacceptable, or develop countermeasures
for the risk, if unacceptable.
(8) Select countermeasures in the following descending priority order to optimize
effectiveness: (1) “design change,” (2) “engineering safety systems,” (3) “safety devices,”
(4) “warning devices,” and (5) “procedures and training.”
3.3
315
Note that this delineation, while in decreasing order of effectiveness, is also typically in
decreasing order of cost and schedule impact (i.e., design changes have the highest
potential for cost and schedule impact). A trade study (sec. 2.1) might be performed to
determine a countermeasure of adequate effectiveness and minimized program impact.
(9) Reevaluate the risk with the new countermeasure installed.
(10) If countermeasures are developed, determine if they introduce new hazards or intolerably
diminish system performance. If added hazards or degraded performance are unacceptable,
determine new countermeasures and reevaluate the risk.
3.2.4 Example
An example of a completed PHA worksheet
3.3
for a pressurized chemical intermediate transfer system is
presented in figure 38. (A blank form is included in appendix E.)
Brief Descriptive Title (Portion of System/Subsystem/Operational Phases covered by this analysis):
Probability Interval: 25 years
H
a
z
a
r
d
T
a
r
g
e
t
*
S
e
v
e
r
i
t
y
P
r
o
b
a
b
i
l
i
t
y
R
i
s
k
C
o
d
e
Description of Countermeasures
S
e
v
e
r
i
t
y
P
r
o
b
a
b
i
l
i
t
y
R
i
s
k
C
o
d
e
Approved by/Date:
Preliminary Hazard Analysis
Identify countermeasures by appropriate code letter(s):
D = Design Alteration E = Engineered Safety Feature
S = Safety Device W = Warning Device
P = Procedures/Training
Analysis: Initial
Revision Addition
System Number: __________
Date:
Hazard No. / Description
Risk
Before
Risk
After
Prepared by/Date: *Target Codes: P—Personnel
T—Downtime
E—Equipment
R—Product V—Environment
Pressurized UnFo Containment and Replenishment Reservoir and Piping / Startup, Routine Operation, Standard Stop, Emergency Shutdown
25 Feb. 1993
X
E
P
T
I
II
III
D
C
C
2
2
3
I
II
III
E
D
D
3
3
3
SrdA.a.042 — Flange Seal A29 leakage, releasing
pressurized UnFo
chemical intermediate from
containment system, producing toxic vapors and
attacking nearby equipment.
Surround flange with sealed annular stainless steel
catchment housing, with gravity runoff conduit led to
DetectoBox™ containing detector/alarm device and
chemical neutralizer (S/W). Inspect flange seal at 2month
intervals, and regasket during annual plant maintenance
shutdown (P). Provide personal protective equipment
(Schedule 4) and training for response/cleanup crew (S/P).
Show hazard alphanumeric
designator. Describe hazard
source mechanism, worst
credible outcome.
Assess worstcredible Severity, and
Probability for that outcome. Show
Risk (from assessment matrix) for
hazard “asis” — i.e., with no
added countermeasures.
SrdA (Chem/Int)
Identify target(s).
Describe newly proposed countermeasures to
reduce Probability/Severity.
NOTE: THESE COUNTERMEASURES MUST
BE IN PLACE PRIOR TO OPERATION.
Reassess Probability/Severity, and
show Risk (from assessment matrix)
for hazard, presuming new
countermeasures to be in place. If Risk
is not acceptable, new countermeasures
must be developed.
3
3
Figure 38. Typical PHA.
316
Note that the worksheet from this example contains the following information:
a. Brief description of the portion of the system, subsystem, or operation covered in the
analysis.
b. Declaration of the probability interval.
c. System number.
d. Date of analysis.
e. Hazard (description and identification number).
f. Hazard targets (check boxes for personnel, equipment, downtime, product
environment).
g. Risk assessment before countermeasures are considered; including severity level,
probability level, and risk priority code (zone from risk matrix, fig. 35).
h. Description of countermeasure (with codes for various types).
i. Risk assessment after countermeasures are considered, including severity level,
probability level, and risk priority code.
j. Signature blocks for the analyst and reviewers/approvers.
The PHA worksheet used in the example is typical. However, an organization may create
their own worksheet customized for their operation. For example, different target types may be listed. In
any case, great care should be given in designing the form to encourage effective usage. Although
helpful, a PHA is not a structured approach that assists the analyst in identifying hazards or threats.
3.2.5 Advantages
A PHA provides the following advantages:
3.3
(1) Identifies and provides a log of primary system hazards and their corresponding risks.
(2) Provides a logically based evaluation of a system’s weak points early enough to allow
design mitigation of risk rather than a procedural or inspection level approach.
(3) Provides information to management to make decisions to allocate resources and prioritize
activities to bring risk within acceptable limits.
(4) Provides a relatively quick review and delineation of the most significant risks associated
with a specific system.
317
3.2.6 Limitations
A PHA possesses the following limitations:
3.3
(1) A PHA fails to assess risks of combined hazards or coexisting system failure modes.
Therefore a false conclusion may be made that overall system risk is acceptable simply
because each hazard element risk identified, when viewed singularly, is acceptable.
(2) If inappropriate or insufficient targets or operational phases are chosen, the assessment will
be flawed. While on the other hand, if too many targets or operational phases are chosen,
the effort will become too large and costly to implement.
3.2.7 Bibliography
Air Force Systems Command Design Handbook DH 16, “System Safety.” December 1982.
Army Regulation 389516, “System Safety Engineering and Management.” May 1990.
Browning, R.L.: “The Loss Rate Concept in Safety Engineering.” Marcel Dekker, Inc., 1980.
Hammer, W.: “Handbook of System and Product Safety.” PrenticeHall, Inc., 1972.
Henley, E.J., and Kumamoto, H.: “Probabilistic Risk Assessment.” The Institute of Electrical and
Electronic Engineers, Inc., New York, 1991.
Malasky, S.W.: “System Safety: Technology and Application.” Garland STPM Press, 1982.
Raheja, D.G. “Assurance Technology and Application  Principles and Practices.” McGrawHill, 1991.
Roland, H.E., and Moriaty, B.: “System Safety Engineering and Management.” John Wiley & Sons,
Inc., 1983.
3.3 Energy Flow/Barrier Analysis
3.3.1 Description
The energy flow/barrier analysis, as described in reference 3.4, is a system safety analysis tool,
used to identify hazards and determine the effectiveness of countermeasures employed or suggested to
mitigate the risk induced by these hazards. This tool is also known as energy trace/barrier analysis. The
energy flow/barrier method is a useful supplement to the PHA discussed in section 3.2.
Energy sources such as electrical, mechanical, chemical, radiation, etc., are identified.
Opportunities for undesired energy flow between the sources and targets are assessed. Barriers are
countermeasures against hazards caused by flows from these energy sources to targets. Examples of
barriers include barricades, blast walls, fences, lead shields, gloves, safety glasses, procedures, etc.
318
3.3.2 Application
An energy flow/barrier analysis can be beneficially applied whenever assessments are needed to
assure an identified target is being safeguarded against a potential energy source that can impose harm.
This assessment can be applied during phase C but may also be applied in phase E or phase B. This
analysis can also be applied in failure investigations.
3.3.3 Procedures
Procedures to perform an energy flow/barrier analysis, as described in reference 3.4, are
presented below:
(1) Examine the system and identify all energy sources.
(2) Examine each potential energy flow path in the system. Consider the following for each
energy flow path:
a. What are the potential targets, such as personnel, facilities, equipment, productivity,
mission or test objectives, environment, etc.? Remember every energy source could
have multiple flow paths and targets.
b. Is the energy flow unwanted or detrimental to a target?
c. Are existing barriers sufficient countermeasures to mitigate the risk to the targets?
(3) Consider the following strategies extracted from reference 3.4 to control harmful energy
flow:
a. Eliminate energy concentrations.
b. Limit quantity and/or level of energy.
c. Prevent the release of energy.
d. Modify the rate of release of energy.
e. Separate energy from target in time and/or space.
f. Isolate by imposing a barrier.
g. Modify target contact surface or basic structure.
h. Strengthen potential target.
i. Control improper energy input.
3.3.4 Example
Examples of strategies to manage harmful energy flows are presented in table 33.
319
Table 33. Examples* of strategies to manage harmful energy flow.
Strategy Examples
Eliminate energy concentrations · Control/limit floor loading
· Disconnect/remove energy source from system
· Remove combustibles from welding site
· Change to nonflammable solvent
Limit quantity and/or level of energy · Store heavy loads on ground floor
· Lower dam height
· Reduce system design voltage/operating pressure
· Use small(er) electrical capacitors/pressure accumulators
· Reduce/ control vehicle speed
· Monitor/limit radiation exposure
· Substitute less energetic chemicals
Prevent release of energy · Heavywall pipe or vessels
· Interlocks
· Tagout – lockouts
· Doublewalled tankers
· Wheel chocks
Modify rate of release of energy · Flow restrictors in discharge lines
· Resistors in discharge circuits
· Fuses/circuit interrupters
Separate energy from target in time
and/or space
· Evacuate explosive test areas
· Impose explosives quantitydistance rules
· Install traffic signals
· Use yellow nopassing lines on highways
· Control hazardous operations remotely
Isolate by imposing a barrier · Guard rails
· Toe boards
· Hard hats
· Face shields
· Machine tool guards
· Dikes
· Grounded appliance frames/housing
· Safety goggles
Modify target contact surface or basic
structure
· Cushioned dashboard
· Fluted stacks
· Padded rocket motor test cell interior
· Whipple plate meteorite shielding
· Breakaway highway sign supports
· Foamed runways
Strengthen potential target · Select superior material
· Substitute forged part for cast part
· “Harden” control room bunker
· Crossbrace transmission line tower
Control improper energy input · Use coded, keyed electrical connectors
· Use matchthreaded piping connectors
· Use back flow preventors
*Examples provided courtesy of Sverdrup Technology Inc., Tullahoma, Tennessee.
320
3.3.5 Advantages
The energy flow/barrier analysis provides a systematic thought process to identify hazards
associated with energy sources and determines if current or planned barriers are adequate
countermeasures to protect exposed targets.
3.4
3.3.6 Limitations
The energy flow/barrier analysis possesses the following limitations.
3.4
(1) Even after a thorough analysis, all hazards might not be discovered. Like the PHA (sec.
3.2), an energy flow/barrier analysis fails to assess risks of combined hazards or coexisting
system failure modes.
(2) This tool also fails to identify certain classes of hazards, e.g., asphyxia in oxygendeficient
confined spaces.
(3) Due to design and performance requirements, it is not always obvious that energy may be
reduced or redirected. A reexamination of energy as heat, potential versus kinetic
mechanical energy, electrical, chemical, etc. may aid this thought process.
3.3.7 Bibliography
Department of Energy, DOD 76–451: SSDC–29, “Barrier Analysis.”
Haddon, W., Jr., M.D.: “Energy Damage and the Ten Countermeasure Strategies.” Human Factors
Journal, August 1973.
Johnson, W.G.: “MORT Safety Assurance Systems.” Marcel Dekker, Inc., 1980.
3.4 Failure Modes and Effects (and Criticality) Analysis
3.4.1 Description
An FMEA, as described in reference 3.5, is a forward logic (bottomup), tabular technique that
explores the ways or modes in which each system element can fail and assesses the consequences of
each of these failures. In its practical application, its use is often guided by topdown “screening” (as
described in sec. 3.4.3) to establish the limit of analytical resolution. An FMECA also addresses the
criticality or risk of individual failures. Countermeasures can be defined, for each failure mode, and
consequent reductions in risk can be evaluated. FMEA and FMECA are useful tools for cost and benefit
studies (sec. 2.2), to implement effective risk mitigation and countermeasure, and as precursors to a fault
tree analysis (FTA) (sec. 3.5).
321
3.4.2 Application
An FMEA can be used to call attention to system vulnerability to failures of individual compo
nents. Singlepoint failures can be identified. This tool can be used to provide reassurance that the cause,
effect, and associated risk (FMECA) of component failures have been appropriately addressed. These
tools are applicable within systems or at the systemsubsystem interfaces and can be applied at the
system, subsystem, component, or part levels.
These failure mode analyses are typically performed during phase C. During this phase, these
analyses can be done with or shortly after the PHA (sec. 3.2). The vulnerable points identified in the
analyses can aid management in making decisions to allocate resources in order to reduce vulnerability.
3.4.3 Procedures
Procedures for preparing and performing FMECA’s, as described in reference 3.5, are presented
below. Procedures for preparing an FMEA are the same with steps 8 through 12 omitted.
Steps prior to performing the FMEA or FMECA:
(1) Define the scope and boundaries of the system to be assessed. Gather pertinent information
relating to the system, such as requirement specifications, descriptions, drawings, compo
nents and parts lists, etc. Establish the mission phases to be considered in the analysis.
(2) Partition and categorize the system into advantageous and reasonable elements to be
analyzed. These system elements include subsystems, assemblies, subassemblies,
components, and piece parts.
(3) Develop a numerical coding system that corresponds to the system breakdown (fig. 39).
Steps in performing the FMEA or FMECA (see flowchart presented in fig. 310):
(4) Identify resources of value, such as personnel, facilities, equipment, productivity, mission
or test objectives, environment, etc. to be protected. These resources are targets.
(5) Identify and observe the levels of acceptable risk that have been predetermined and
approved by management. These limits may be the risk matrix boundaries defined in a risk
assessment matrix (sec. 3.2).
(6) By answering the following questions posed in reference 3.5, the scope and resources
required to perform a classical FMEA can be reduced, without loss of benefit:
a. Will a system failure render an unacceptable or unwanted loss?
If the answer is no, the analysis is complete. Document the results. (This has the addi
tional benefit of providing visibility of nonvalue added systems, or it may serve to
correct incomplete criteria being used for the FMEA.) If the answer is yes, ask the
following question for each subsystem identified in step 2 above.
322
Subassembly 1
Typical Coding System: Subsystem No.  Assembly No.  Subassembly No.  Component No.  Part No.
For example, code number for part 2 above is 0301030102
System
Subsystem 2 Subsystem 3 Subsystem 4
Assembly 1 Assembly 2 Assembly 3
Subassembly 2 Subassembly 3
Component 1 Component 2 Component 3
Part 1 Part 2
Part 3
Subsystem 1
Figure 39. Example of system breakdown and numerical coding.
3.5
b. Will a subsystem failure render an unacceptable or unwanted loss?
If the answer for each subsystem is no, the analysis is complete. Document the results.
If the answer is yes for any subsystem, ask the following question for each assembly of
those subsystems identified in step 2 above.
c. Will an assembly failure render an unacceptable or unwanted loss?
If the answer for each assembly is no, the analysis is complete. Document the results. If
the answer is yes for any assembly, ask the following question for each component of
those assemblies identified in step 2 above.
d. Will a subassembly failure render an unacceptable or unwanted loss?
If the answer for each subassembly is no, the analysis is complete. Document the
results. If the answer is yes for any subassembly, ask the following question for each
component of those subassemblies identified in step 2 above:
323
Figure 310. FMECA process flowchart.
324
e. Will a component failure render an unacceptable or unwanted loss?
If the answer for each component is no, the analysis is complete. Document the results.
If the answer is yes for any component, ask the following question for each part of
those components as identified in step 2 above.
f. Will a part failure render an unacceptable or unwanted loss?
(7) For each element (system, subsystem, assembly, subassembly, component, or part) for
which failure would render an unacceptable or unwanted loss, ask and answer the following
questions:
a. What are the failure modes (ways to fail) for this element?
b. What are the effects (or consequences) of each failure mode on each target?
(8) Assess worstcredible case (not the worstconceivable case) severity and probability for
each failure mode, effect, and target combination.
(9) Assess risk of each failure mode using a risk assessment matrix (sec. 3.1). The matrix
should be consistent with the established probability interval and force or fleet size for this
assessment.
(10) Categorize each identified risk as acceptable or unacceptable.
(11) If the risk is unacceptable, then develop countermeasures to mitigate the risk.
(12) Then reevaluate the risk with the new countermeasure installed.
(13) If countermeasures are developed, determine if they introduce new hazards or intolerable or
diminished system performance. If added hazards or degraded performance are unaccept
able, develop new countermeasures and reevaluate the risk.
(14) Document your completed analysis on an FMEA or FMECA worksheet. The contents and
formats of these worksheets vary among organizations. Countermeasures may or may not
be listed. Typically, the information found in an FMECA worksheet, according to reference
3.5, is presented in figure 311. A worksheet for an FMEA would be similar with the risk
assessment information removed. A sample FMEA worksheet is attached in appendix F.
325
FMEA NO: ________________________
PROJECT NO.: ______________________
SUBSYSTEM NO.: __________________
SYSTEM NO.: ______________________
PROB. INTERVAL: ___________________
FAILURE MODES, EFFECTS,
AND CRITICALITY ANALYSIS
WORKSHEET
SHEET ___ OF ____
DATE _________________________
PREPARED BY: __________________
REVIEWED BY: __________________
APPROVED BY:__________________
TARGET CODE: P – PERSONNEL / E – EQUIPMENT / T – DOWNTIME / R – PRODUCTS / D –DATA / V –ENVIRONMENT
Item/ Failure Failure Failure
T
a
r
Risk Assessment
Action Required/
Id. No. Functional
Ident.
Mode Cause Event g
e
t
S
e
v
P
r
o
b
R C
i o
s d
k e
Comments
Figure 311. Typical FMECA worksheet.
3.5
3.4.4 Example
An example FMECA
3.5
is illustrated in figure 312. The system being assessed is an automated
mountain climbing rig. A schematic of the system is presented in figure 312(a). Figure 312(b)
illustrates the breakdown and coding of the system into subsystem, assembly, and subassembly
elements. An FMECA worksheet for the control subsystem is presented in figure 312(c).
326
(a) System.
Subsystem Assembly Subassembly
Hoist (A) Motor (A01) Windings (A01a)
Inboard bearing (A01b)
Outboard bearing (A01c)
Rotor (A01d)
Stator (A01e)
Frame (A01f)
Mounting plate (A01g)
Wiring terminals (A01h)
Drum (A02)
External power source (B)
Cage (C) Frame (C01)
Lifting Lug (C02)
Cabling (D) Cable (D01)
Hook (D02)
Pulleys (D03)
Controls (E) Switch (E01) START (E01a)
FULL UP LIMIT (E01b)
Wiring (E01c)
(b) System breakdown and coding.
Figure 312. Example of an FMECA—Continued
327
FMEA NO: ________________________
PROJECT NO.:
______________________
SUBSYSTEM NO.: Controls
SYSTEM NO.: Mountain Climbing
Rig_____
PROB. INTERVAL: 30 years
___________
FAILURE MODES, EFFECTS,
AND CRITICALITY ANALYSIS
WORKSHEET
SHEET ___ OF ____
DATE _________________________
PREPARED BY: __________________
REVIEWED BY: __________________
APPROVED BY:__________________
TARGET CODE: P – PERSONNEL / E – EQUIPMENT / T – DOWNTIME / R – PRODUCTS / D – DATA / V – ENVIRONMENT
Item/ Failure Failure Failure
T
a
r
Risk Assessment Action Required/
Id. No. Functional
Ident.
Mode Cause Event g
e
t
S
e
v
P
r
o
b
R C
i o
s d
k e
Comments
E–01–a Start switch Switch fails
closed.
Mechanical
failure or
corrosion.
Cage will
not move.
P
E
T
IV
IV
IV
C
C
C
3
3
3
E–01–b Full up switch Switch fails
open.
Mechanical
failure or
corrosion.
Cage does
not stop.
P II A 1
E–02 Wiring Cut, dis
connected.
Varmint
invasion.
No
response a
switch.
Start switch
fails open.
Stop switch
fails
closed.
Cage stays
in safe
position.
P
E
T
IV
IV
IV
D
D
D
3
3
3
(c) Worksheet.
Figure 312. Example of an FMECA—Continued.
3.4.5 Advantages
Performing FMEA’s and FMECA’s provide the following advantages:
3.5
(1) Provides a mechanism to be exhaustively thorough in identifying potential singlepoint
failures and their consequences. An FMECA provides risk assessments of these failures.
(2) Results can be used to optimize reliability, optimize designs, incorporate “fail safe”
features into the system design, obtain satisfactory operation using equipment of “low
reliability,” and guide in component and manufacturer selection.
(3) Provide further analysis at the piecepart level for highrisk hazards identified in a PHA.
328
(4) Identify hazards caused by failures to be added to the PHA that may have been previously
overlooked in the PHA.
(5) Provides a mechanism for more thorough analysis than an FTA, since every failure mode of
each component of the system is assessed.
3.6
3.4.6 Limitations
The following limitations are imposed when performing FMEA’s and FMECA’s.
(1) Costly in manhour resources, especially when performed at the partscount level within
large, complex systems.
(2) Probabilities or the consequences of system failures induced by coexisting, multiple
element faults or failures within the system are not addressed or evaluated.
(3) Although systematic, and guidelines/check sheets are available for assistance, no check
methodology exists to evaluate the degree of completeness of the analysis.
(4) This analysis is heavily dependent upon the ability and expertise of the analyst for finding
all necessary modes.
(5) Human error and hostile environments frequently are overlooked.
(6) Failure probability data are often difficult to obtain for an FMECA.
(7) If too much emphasis is placed on identifying and eliminating singlepoint failures, then
focus on more severe system threats may be overlooked.
(8) An FMECA can be a very thorough analysis suitable for prioritizing resources to higher
risk areas if it can be performed early enough in the design phase. However, the level of
design maturity required for an FMECA is not generally achieved until late in the design
phase, often too late to guide this prioritization.
3.4.7 Bibliography
Layton, D.: “System Safety – Including DOD Standards.” Weber Systems Inc., Chester, OH, 1989.
Lees, F.P.: “Loss Prevention in the Process Industries.” 2 vols. Butterworths, London, 1980.
MIL–STD–1629A, “Procedures for Performing a Failure Modes, Effects, and Criticality Analysis.”
November 1980.
Raheja, D.G.: “Assurance Technologies – Principles and Practices.” McGrawHill, Inc., 1991.
Roberts, N.H., Vesely, W.E., Haasl, D.F., and Goldberg, F.F.: “Fault Tree Handbook.” NUREG–0492.
U.S. Government Printing Office, Washington, DC.
Roland, H.E., and Moriarty, B.: “System Safety Engineering and Management. ” John Wiley & Sons,
1982.
329
3.5 Reliability Block Diagram
3.5.1 Description
A reliability block diagram (RBD) is a backwards (topdown) symbolic logic model generated in
the success domain. The descriptions of RDB’s contained herein was obtained from references 3.7 and
3.8. Each RBD has an input and an output and flows left to right from the input to the output. Blocks
may depict the events or system element functions within a system. However, these blocks typically
depict system element functions only. A system element can be a subsystem, subassembly, component,
or part.
Simple RBD’s are constructed of series, parallel, or combinations of series and parallel elements
(table 34). Each block represents an event or system element function. These blocks are connected in
series if all elements must operate successfully for the system to operate successfully. These blocks are
connected in parallel if only one element needs to operate successfully for the system to operate success
fully. A diagram may contain a combination of series and parallel branches. The system operates if an
uninterrupted path exists between the input and output.
Table 34 . Simple RBD construction.
Type branch Block diagram representation System reliability #
Series
A B
R
S
= R
A *
R
B
Parallel
A
B
R
S
= 1 – (1–R
A
)(1–R
B
)
Seriesparallel
A
B
C
D
R
S
= (1 – (1–R
A)
(1–R
B
))
* (1 – (1–R
C)
(1–R
D
))
Parallelseries
A
C
B
D
R
S
= 1 – (1 – (R
A *
R
B
))
*
(1 – (R
C *
R
D
))
#
Assumes all components function independently of each other.
330
RBD’s illustrate system reliability. Reliability is the probability of successful operation during a
defined time interval. Each element of a block diagram is assumed to function (operate successfully) or
fail independently of each other element. The relationships between element reliability and system relia
bility for series and parallel systems are presented below, and their derivations are found in reference
3.8.
Series Systems:
R
S
= ∏
i
n
R
i
= R
1*
R
2
*
R
3 *
. . . .
Rn.
Parallel Systems:
R
S
= 1–∏
i
n
(1–R
i
) = [1–(1–R
1
)
*
(1–R
2
) (1–R
3
)
*
. . . .
(1–Rn )]
where
R
S
= system reliability,
R
i
= system element reliability, and
n = number of system elements (which are assumed to function independently).
Not all systems can be modeled with simple RBD’s. Some complex systems cannot be modeled
with true series and parallel branches. These systems must be modeled with a complex RBD. Such an
RBD is presented in figure 313. Notice in this example, if element E fails, then paths B, E, G and B, E,
H are not success paths, thus this is not a true series or parallel arrangement.
A
C
D
F
B E
G
H
Figure 313. Typical complex RBD.
331
3.5.2 Application
An RBD allows evaluation of various potential design configurations.
3.8
Required subsystem and
element reliability levels to achieve the desired system reliability can be determined. Typically, these
functions are performed during phase C. An RBD may also be used to identify elements and logic as a
precursor to performing an FTA (sec. 3.6).
3.5.3 Procedures
The procedures (adapted from reference 3.8) to generate a simple RBD are presented below:
(1) Divide a system into its elements. A functional diagram of the system is helpful.
(2) Construct a block diagram using the convention illustrated in table 34.
(3) Calculate system reliability band, R
SL
(low) to R
SH
(high), from each individual element’s
reliability band, R
iL
(low) to R
iH
(high), in the following manner:
a. For series systems with n elements that are to function independently,
R
SL
= ∏
i
n
(R
iL
) = R
1L *
R
2L *
R
3L *
. . . .
R
nL
R
SH
= ∏
i
n
(R
iH
) = R
1H *
R
2H *
R
3H *
. . . .
R
nH.
b. For parallel systems with n elements that are to function independently,
R
SL
= 1–∏
i
n
(1–R
pL
) = [1–(1–R
1L
)
*
(1–R
2L
) (1–R
3L
)
*
. . . .
(1–R
nL
)]
R
SH
= 1–∏
i
n
(1–R
pH
) = [1–(1–R
1H
)
*
(1–R
2H
) (1–R
3H
)
*
. . . .
(1–R
nH
)].
c. For seriesparallel systems, first determine the reliability for each parallel branch using
the equations in step 3b. Then treat each parallel branch as an element in a series branch
and determine the system reliability by using the equations in step 3a.
d. For parallelseries systems, first determine the reliability for each series branch using
the equations in step 3a. Then treat each series branch as an element in a parallel branch
and determine the system reliability by using the equations in step 3b.
332
e. For systems that are composed of the four above arrangements, determine the reliability
for the simplest branches. Then, treat these as branches within the remaining block dia
gram, and determine the reliability for the new simplest branches. Continue this process
until one of the above four basic arrangements remains, then determine the system
reliability.
3.5.4 Example
A system has two subsystems designated 1 and 2. Subsystem 2 is designed to be a backup for
subsystem 1. Subsystem 1 has three components and at least one of the three must function successfully
for the subsystem to operate. Subsystem 2 has three components that all need to function successfully
for the subsystem to operate. The estimated reliability band for each individual component over the
system’s estimated 10yr life interval is presented below:
Subsystem Component Reliability Bands
Low High
1 A 0.70 0.72
1 B 0.80 0.84
1 C 0.60 0.62
2 D 0.98 0.99
2 E 0.96 0.97
2 F 0.98 0.99
An RBD for the system is presented in figure 314. Note that the components for subsystem 1 are
in a parallel branch with the components of subsystem 2. Also, note that the components for subsystem 1
form a series branch and the components for subsystem 2 form a parallel branch.
D E F
A
B
0.70  0.72
0.80  0.84
0.980.99 0.960.97 0.980.99
C
0.60  0.62
Figure 314. Example RBD.
333
C alculations for subsystem and system reliabilities are presented below:
Subsystem 1: R
1L
= 1 – (1–0.70)(1–0.80)(1–0.60) = 0.976 (low band value)
R
1H
= 1 – (1–0.72)(1–0.84)(1–0.62) = 0.983 (high band value)
Subsystem 2: R
2L
= (0.98)(0.96)(0.98) = 0.922 (low band value)
R
2H
= (0.99)(0.97)(0.99) = 0.951 (high band value)
System: R
SL
= 1 – (1–0.976)(1–0.922) = 0.998 (low band value)
R
SH
= 1 – (10.983)(1–0.951) = 0.999 (high band value)
Therefore, the reliability band for the system is 0.998 to 0.999.
3.5.5 Advantages
An RBD provides the following advantages:
(1) Allows early assessment of design concepts when design changes can be readily and
economically incorporated.
3.8
(2) Tends to be easier for an analyst to visualize than other logic models, such as a fault tree.
3.7
(3) Blocks representing elements in an RBD can be arranged in a manner that represent how
these elements function in the system.
3.7
(4) Since RBD’s are easy to visualize, they can be generated prior to performing an FTA and
transformed into a fault tree by the method discussed in section 3.9.
3.5.6 Limitations
An RBD possesses the following limitations:
(1) Systems must be broken down into elements where reliability estimates can be obtained.
Such a breakdown for a large system can be a significant effort.
3.8
(2) System element reliability estimates might not be readily available for all elements. Some
reliability estimates may be very subjective, difficult to validate, and not be accepted by
others in the decision making process. If the element reliability values have different
confidence bands, this can lead to significant problems.
(3) Not all systems can be modeled with combinations of series, parallel, seriesparallel, or
parallelseries branches. These complex systems can be modeled with a complex RBD.
However, determining system reliability for such a system is more difficult than for a
simple RBD.
3.73.8
334
3.5.7 Bibliography
Pages, A. and Godran, M.: “System Preliminary Evaluation & Prediction in Engineering.” Springer
Verb, 1986.
3.6 Fault Tree Analysis
3.6.1 Description
A fault tree analysis (FTA), as described in reference 3.9, is a topdown symbolic logic model
generated in the failure domain. This model traces the failure pathways from a predetermined,
undesirable condition or event, called the TOP event, of a system to the failures or faults (fault tree
initiators) that could act as causal agents. Previous identification of the undesirable event also includes a
recognition of its severity. An FTA can be carried out either quantitatively or subjectively.
The FTA includes generating a fault tree (symbolic logic model), entering failure probabilities
for each fault tree initiator, propagating failure probabilities to determining the TOP event failure
probability, and determining cut sets and path sets. A cut set is any group of initiators that will, if they
all occur, cause the TOP event to occur. A minimal cut is a least group of initiators that will, if they all
occur, cause the TOP event to occur. A path set is a group of fault tree initiators that, if none of them
occurs, will guarantee the TOP event cannot occur.
The probability of failure for a given event is defined as the number of failures per number of
attempts. This can be expressed as:
P
F
= F/(S+F) , where F = number of failures and S = number of successes.
Since reliability for a given event is defined as the number of successes per number of attempts,
then the relationship between the probability of failure and reliability can be expressed as follows:
R = S/(S+F),
therefore
R + P
F
= S/(S+F) + F/(S+F) = 1
and
P
F
= 1–R.
3.6.2 Application
FTA’s are particularly useful for high energy systems (i.e., potentially high severity events), to
ensure that an ensemble of countermeasures adequately suppresses the probability of mishaps. An FTA
is a powerful diagnostic tool for analysis of complex systems and is used as an aid for design
improvement.
335
This type of analysis is sometimes useful in mishap investigations to determine cause or to rank
potential causes. Action items resulting from the investigation may be numerically coded to the fault tree
elements they address, and resources prioritized by the perceived highest probability elements.
FTA’s are applicable both to hardware and nonhardware systems and allow probabilistic assess
ment of system risk as well as prioritization of the effort based upon root cause evaluation. The
subjective nature of risk assessment is relegated to the lowest level (root causes of effects) in this study
rather than at the top level. Sensitivity studies can be performed allowing assessment of the sensitivity of
the TOP event to basic initiator probabilities.
FTA’s are typically performed in phase C but may also be performed in phase D. FTA’s can be
used to identify cut sets and initiators with relatively high failure probabilities. Therefore, deployment of
resources to mitigate risk of highrisk TOP events can be optimized.
3.6.3 Procedures
The procedures, as described in reference 3.9, for performing an FTA are presented below. These
procedures are divided into the four phases: (1) fault tree generation, (2) probability determination,
(3) identifying and assessing cut sets, and (4) identifying path sets. The analyst does not have to perform
all four phases, but can progress through the phases until the specific analysis objectives are met. The
benefits for each of the four phases are summarized in table 35.
Table 35. FTA procedures.
Section Procedures Benefits
3.6.3.1 Fault tree generation All basic events (initiators), intermediate events, and the TOP
event are identified. A symbolic logic model illustrating fault
propagation to the TOP event is produced.
3.6.3.2 Probability determination Probabilities are identified for each initiator and propagated to
intermediate events and the TOP event.
3.6.3.3 Identifying and assessing cut sets All cut sets and minimal cuts sets are determined. A cut set is
any group of initiators that will, if they all occur, cause the
TOP event to occur. A minimal cut set is a least group of
initiators that, if they all occur, will cause the TOP event to
occur. Analysis of a cut set can help evaluate the probability
of the TOP event, identify qualitative common cause
vulnerability, and assess quantitative common cause
probability. Cut sets also enable analyzing structural,
quantitative, and item significance of the tree.
3.6.3.4 Identifying path sets All path sets are determined. A path set is a group of fault tree
initiators that, if none of them occurs, will guarantee the TOP
event cannot occur.
336
3.6.3.1 Fault Tree Generation
Fault trees are constructed with various event and gate logic symbols, defined in table 36.
Although many event and gate symbols exist, most fault trees can be constructed with the following four
symbols: (1) TOP or Intermediate event, (2) inclusive OR gate, (3) AND gate, and (4) basic event. The
procedures, as described in reference 3.9, to construct a fault tree are illustrated in figure 315.
A frequent error in fault tree construction is neglecting to identify common causes. A common
cause is a condition, event, or phenomenon that will simultaneously induce two or more elements of the
fault tree to occur. A method for detecting common causes is described in section 3.6.3.3, step 8.
Sections 3.6.3.2 through 3.6.4.3 are included for completeness and to provide insight as to the
mathematics that takes place in the commercially available fault tree programs. All large trees are
typically analyzed using these programs; for small trees hand analysis may be practical.
3.6.3.2 Probability Determination
If a fault tree is to be used as a quantitative tool, the probability of failure must be determined for
each basic event or initiator. Sources for these failure probabilities may be found from manufacturer’s
data, industry consensus standards, MILstandards, historical evidence (of the same or similar systems),
simulation or testing, Delphi estimates, and the log average method. A source for human error probabili
ties is found in reference 3.10. The Delphi technique (sec. 7.9) derives estimates from the consensus of
experts. The log average method is useful when the failure probability cannot be estimated but credible
upper and lower boundaries can be estimated. This technique is described in reference 3.11 and is
illustrated in figure 316.
Failure probabilities can also be determined from a PDA as discussed in section 3.14.3, step 6.
Probabilities must be used with caution to avoid the loss of credibility of the analysis. In many
cases it is best to stay with comparative probabilities rather than the “absolute” values. Normalizing data
to a standard, explicitly declared meaningless value is a useful technique here. Also, confidence or error
bands, on each cited probability number, are required to determine the significance of any quantitatively
driven conclusion.
Once probabilities are estimated for all basic events or initiators, they are propagated through
logic gates to the intermediate events and finally the TOP event. The probability of failure of
independent inputs through an AND gate is the intersection of their respective individual probabilities.
The probability of failure of independent events through an OR (inclusive) gate is the union of their
respective individual probabilities. Propagation of confidence and error bands is performed simply by
propagation of minimum and maximum values within the tree.
The relationship between reliability and failure probability propagation of two and three inputs
through OR (inclusive) and AND gates is illustrated in figure 317. Propagation of failure probabilities
for two independent inputs through an AND and OR (inclusive) is conceptually illustrated in figure 3
18. As shown in figures 317, the propagation solution through an OR gate is simplified by the rare
event approximation assumption. The exact solution for OR gate propagation is presented in figure 319.
However, the use of this exact solution is seldom warranted.
The propagation equations for the logic gates, including the gates infrequently used, are
presented in table 37.
337
Table 36. Fault tree construction symbols.
Symbol Name Description
Event (TOP or
intermediate)
*
TOP Event – This is the conceivable, undesired event to which
failure paths of lower level events lead.
Intermediate Event – This event describes a system condition
produced by preceding events.
Inclusive OR gate
*
An output occurs if one or more inputs exist. Any single input is
necessary and sufficient to cause the output event to single occur.
Exclusive OR gate An output occurs if one, but only one input exists. Any single
input is necessary and sufficient to cause the output event to
occur.
M
Mutually exclusive
OR gate
An output occurs if one or more inputs exist. However, all other
inputs are then precluded. Any single input is necessary and
sufficient to cause the output event to occur.
AND gate
*
An output occurs if all inputs exist. All inputs are necessary and
sufficient to cause the output event to occur.
Priority AND gate An output occurs if all inputs exist and occur in a predetermined
sequence. All inputs are necessary and sufficient to cause the
output event to occur.
INHIBIT gate An output occurs if a single input event occurs in presence of an
enabling condition.
Basic event
*
An initiating fault or failure that is not developed further. These
events determine the resolution limit of the analysis. They are
also called leaves or initiators.
External event An event that under normal conditions is expected to occur.
Undeveloped event An event not further developed due to a lack of need, resources,
or information.
Conditioning Event These symbols are used to affix conditions, restraints, or
restrictions to other events.
*
Most fault trees can be constructed with these four logic symbols.
338
6. Repeat /
continue…
4. Identify secondlevel
contributors.
5. Link secondlevel
contributors to TOP by
logic gates.
1. Identify undesirable TOP
event.
3. Link contributors to TOP
by logic gates.
2. Identify firstlevel
contributors.
Basic Event…(“Leaf,”
“Initiator,” or “Basic”)
indicates limit of analytical
resolution.
Figure 315. Fault tree construction process.
• Estimate upper and lower credible bounds of probability for the phenomenon in question.
• Average the logarithms of the upper and lower bounds.
• The antilogarithm of the average of the logarithms of the upper and lower bounds is less
than the upper bound and greater than the lower bound by the same factor. Thus, it is
geometrically midway between the limits of estimation.
0.1
0.0316
+
0.01 0.02 0.03 0.04 0.05 0.07
PL
Lower
Probability
Bound
10
–2
PU
Upper
Probability
Bound
10
–1
Log P
L
+ Log P
U
2
(–2) + (–1)
2
Log Average = Antilog = Antilog = 10
–1.5
= 0.0316228
Note that, for the example shown, the arithmetic average would be…
i.e., 5.5 times the lower bound and 0.55 times the upper bound.
0.01 + 0.1
2
= 0.055
Figure 316. Log average method of probability estimation.
339
P
F
= 1 – R
T
P
F
= 1 – (R
A
R
B
)
P
F
= 1 – [(1 – P
A
)(1 – P
B
)]
R + P
F
≡ 1
OR Gate AND Gate
Either of two, independent,
element failures produces
system failure.
Both of two, independent
elements must fail to produce
system failure.
R
T
= R
A
+ R
B
– R
A
R
B
R
T
= R
A
R
B
P
F
= 1 – R
T
P
F
= 1 – (R
A
+ R
B
– R
A
R
B
)
P
F
= 1 – [(1 – P
A
) + (1 – P
B
) – (1 – P
A
)(1 – P
B
)]
P
F
= P
A
P
B
P
F
= P
A
+ P
B
– P
A
P
B
“Rare
Event
Approximation”
…for P
A,B
≤ 0.2
P
F
≅ P
A
+ P
B
with error ≤ 11%
P
F
= P
A
P
B
P
C Omit for
approximation
For 3 Inputs
For 2 Inputs
P
F
= P
A
+ P
B
+ P
C
– P
A
P
B
– P
A
P
C
– P
B
P
C
+ P
A
P
B
P
C
[Union / ] [Intersection / ]
Figure 317. Relationship between reliability and failure probability propagation.
Intersection /
P
T
= P
1
+ P
2
– P
1
P
2
P
T
= P
1
P
2
AND Gate…
TOP
P
1
P
2
P
T
= P
1
P
2
1 2
OR Gate…
TOP
P
1
P
2
P
T
P
1
+ P
2
1 2
1 & 2
are
INDEPENDENT
events
Usually negligible…
P
T
= P
e
P
T
P
e
Union /
Figure 318. Failure probability propagation through OR and AND gates.
340
The ip operator ( ) is the
cofunction of pi ( ∏ ). It
provides an exact solution
for propagating probabilities
through the OR gate. Its use
is rarely justifiable.
P
T
= P
e
= 1 – P (1 – P
e
)
P
T
= 1 – [(1 – P
1
) (1 – P
2
) (1 – P3 ) (1 – Pn
)] …
TOP
P
1
P
2
1 2
3
P
3
P
T
= P
e
TOP
P
1
P
2
1 2
3
P
3
P
T
= ?
TOP
1 2
3
P
T
= (1 – P
e
)
P
1
= (1 – P
1
)
P
2
= (1 – P
2
)
P
3
= (1 – P
3
)
Failure
Failure Success
Figure 319. Exact solution of OR gate failure probability propagation.
3.6.3.3 Identifying and Assessing Cut Sets
A cut set is any group of initiators that will produce the TOP event, if all the initiators in the
group occur. A minimal cut set is the smallest number (in terms of elements, not probability) of initiators
that will produce the TOP event, if all the initiators in the group occur. One method of determining and
analyzing cut sets is presented below. These procedures for determining cut sets are described in
reference 3.9 and are based on the MOCUS computer algorithm attributed to J.B. Fussell. Analysis of a
cut set can help evaluate the probability of the TOP event, identify common cause vulnerability, and
assess common cause probability. Cut sets also enable analyzing structural, quantitative, and item
significance of the tree.
Determining Cut Sets:
(1) Consider only the basic events or initiators (discarding intermediate events and the TOP
event).
(2) Assign a unique letter to each gate and a unique number to each initiator, starting from the
top of the tree.
(3) From the top of the tree downwards, create a matrix using the letters and numbers. The
letter for the gate directly beneath the TOP event will be the first entry in the matrix.
Proceed through the matrix construction by (1) substituting the letters for each AND gate
with letters for the gates and numbers of the initiators that input into that gate (arrange
these letters and numbers horizontally in the matrix rows), and (2) substituting the letters
for each OR gate with letters for the gates and numbers of the initiators that input into that
gate (arrange these letters and numbers vertically in the matrix columns).
341
Table 37. Probability propagation expressions for logic gates.
Symbol Name Venn Diagram Propagation Expressions
Inclusive OR
gate
‡
P1 P2
P
T
= P
1
+ P
2
– (P
1
*
P
2
)
P
T
= P
1
+ P
2
#
Exclusive OR
gate
P2 P1
P
T
= P
1
+ P
2
–2 (P
1
*
P
2
)
P
T
= P
1
+ P
2
#
M
Mutually
exclusive OR
gate
P1 P2
P
T
= P
1
+ P
2
AND gate
‡
and
(priority AND
gate)
P1 P2
P
T
= P
1 *
P
2
‡
Most fault trees can be constructed with these two logic gates.
#
Simplified expression for rare event approximation assumption.
(4) When all the gate’s letters have been replaced, a final matrix is produced with only
numbers of initiators. Each row of this matrix represents a Booleanindicated cut set.
(5) Visually inspect the final matrix and eliminate any row that contains all elements of a lesser
row. Next, through visual inspection, eliminate redundant elements within rows and rows
that repeat other rows. The remaining rows define the minimal cut sets of the fault tree.
Assessing Cut Sets:
(6) Since a cut set is any group of initiators that will produce the TOP event, if all the initiators
in the group occur, the cut set probability, P
K
(the probability that the cut set will induce the
TOP event) is mathematically the same as the propagation through an AND gate, expressed
as:
P
K
= P
1 *
P
2 *
P
3*
P
4
...
*
P
n
.
342
(7) Determine common cause vulnerability by uniquely assigning letter subscripts for common
causes to each numbered initiator (such as m for moisture, h for human operator, q for heat,
v for vibration, etc.). Note that some initiators may have more than one subscript, while
others will have none. Check to see if any minimal cut sets have elements with identical
subscripts. If that is the case, then the TOP event is vulnerable to the common cause the
subscript represents. This indicates that the probability number, calculated as above, may
be significantly in error, since the same event (the socalled common cause) could act to
precipitate each event, i.e., they no longer represent statistically independent events.
(8) Analyze the probability of each common cause at its individual probability level of both
occurring, and inducing all terms within the affected cut set.
(9) Assess the structural significance of the cut sets to provide qualitative ranking of contribu
tions to system failure. Assuming all other things are equal then:
a. A cut set with many elements indicates low vulnerability.
b. A cut set with few elements indicates high vulnerability.
c. Numerous cut sets indicates high vulnerability.
d. A cut set with a single initiator, called a singleton, indicates a potential singlepoint
failure.
(10) Assess the quantitative Importance, I
K
,
of each cut set, K. That is, determine the numerical
probability that this cut set induced the TOP event, assuming it has occurred.
I
K
= P
K
/P
T
where
P
K
= the probability that the cut set will occur (see step 6 above), and
P
T
= the probability of the TOP event occurring.
(11) Assess the quantitative importance, I
e,
of each individual initiator, e. That is, determine the
numerical probability that initiator e contributed to the TOP event, if it has occurred:
I
e
= I
K
e
e
N
e
∑
where
N
e
= number of minimal cut sets containing initiator e, and
I
K
e
= importance of the minimal cut sets containing initiator e.
3.6.3.4 Identifying Path Sets
A path set is a group of fault tree initiators that, if none of them occurs, ensures the TOP event
cannot occur. Path sets can be used to transform a fault tree into a reliability diagram (sec. 3.9). The
procedures to determine path sets are as follows:
(1) Exchange all AND gates for OR gates and all OR gates for AND gates on the fault tree.
343
(2) Construct a matrix in the same manner as for cut sets (sec. 3.6.3.3, steps 1–5).
Each row of the final matrix defines a path set of the original fault tree.
3.6.4 Examples
3.6.4.1 Fault Tree Construction and Probability Propagation
An example of a fault tree with probabilities propagated to the TOP event is presented in
figure 320. In this example the TOP event is the “artificial wakeup fails.” The system being examined
consists of alarm clocks used to awaken someone. In this example for brevity, only a nominal
probability value for each fault tree initiator is propagated through the fault tree to the TOP event.
However, for a thorough analysis, both low and high probability values that define a probability band for
each initiator could be propagated through the fault tree to determine a probability band for the TOP
event.
Faults/Operation……8x10
3
Rate, Faults/Year………2/1
Assume 260 Operations/Year
KEY:
ARTIFICIAL
WAKEUP
FAILS
3.34x10
4
ALARM
CLOCKS
FAIL
3.34x10
4
MAIN,
PLUGIN
CLOCK FAILS
1.82x10
2
POWER
OUTAGE
1x10
2
FAULTY
INNARDS
3x10
4
ELECTRICAL
FAULT
3x10
4
MECHANICAL
FAULT
8x10
8
HOUR
HAND
FALLS
OFF
4x10
4
HOUR
HAND
JAMS
WORKS
2x10
4
FORGET TO
SET
8x10
3
BACKUP
(WINDUP)
CLOCK FAILS
1.83x10
2
FAULTY
MECHANISM
4x10
4
FORGET TO
SET
8x10
3
FORGET TO
WIND
1x10
2
NOCTURNAL
DEAFNESS
Negligible
Approx. 0.1/yr
3/1 2/1 1/10 2/1
1/15
1/10
3/1
1/20
Figure 320. Example fault tree.
344
3.6.4.2 Cut Sets
An example of how to determine Booleanindicated minimal cut sets for a fault tree is presented
in figure 321.
PROCEDURE:
• Assign letters to
gates. (TOP gate
is “A.”) Do not
repeat letters.
• Assign numbers
to basic initiators.
If a basic appears
more than once,
represent it by the
same number at
each appearance.
TOP
C
D B
A
1
2 3
4 2
• Construct a matrix, starting with the TOP “A” gate…
2 2 3
1 2
1 4
2 4 3
1 2
2 3
1 4
A B D
C D
1 D
2 D 3
1 D
2 D 3
1 4
1 2
TOP event gate
is A, the
initial matrix
entry.
A is an AND
gate. B & D,
its inputs, re
place it hori
zontally.
B is an OR
gate. 1 & C,
its inputs, re
place it verti
cally. Each
requires a new
row.
C is an AND
gate. 2 & 3,
its inputs,
replace it
horizontally.
D (top row), is
an OR gate.
2 & 4, its in
puts, replace
it vertically.
Each requires a
new row.
D (2nd row), is
an OR gate.
Replace as
before.
These BooleanIndicated
Cut Sets…
…reduce to
these Minimal
Cut Sets.
Minimal Cut Set
rows are least
groups of
initiators which
will induce TOP.
Figure 321. Example of determining cut sets.
345
3.6.4.3 Path Sets
An example of how to determine path sets for a fault tree is presented in figure 322.
1 2
1 3
1 4
3 4 5 6
1 3
1 4
1 5
1 6
2 3 4
This
Fault
Tree
has…
…these Minimal
Cut Sets
…and these Path
Sets.
Path Sets are
least groups of
initiators which, if
they cannot
occur, guarantee
against TOP
occurring.
TOP
1
2
3
3
4
6
G
F
E
D
C B
A
5
4 1
“Barring” terms (n) denotes consideration
of their success properties.
Figure 322. Example of determining path sets.
3.6.5 Advantages
An FTA provides the following advantages:
3.9
(1) Enables assessment of probabilities of combined faults/failures within a complex system.
(2) Singlepoint and common cause failures can be identified and assessed.
(3) System vulnerability and lowpayoff countermeasures are identified, thereby guiding
deployment of resources for improved control of risk.
(4) This tool can be used to reconfigure a system to reduce vulnerability.
(5) Path sets can be used in trade studies to compare reduced failure probabilities with
increases in cost to implement countermeasures.
346
3.6.6 Limitations
A FTA possesses the following limitations:
(1) Address only one undesirable condition or event that must be foreseen by the analyst. Thus,
several or many FTA’s may be needed for a particular system.
(2) Fault trees used for probabilistic assessment of large systems may not fit or run on
conventional PCbased software.
(3) The generation of an accurate probabilistic assessment may require significant time and
resources. Caution must be taken not to “over work” determining probabilities or
evaluating the system, i.e., limit the size of the tree.
(4) A fault tree is not accurate unless all significant contributors of faults or failures are
anticipated.
(5) Events or conditions under the same logic gate must be independent of each other.
(6) A fault tree is flawed if common causes have not been identified.
(7) Events or conditions at any level of the tree must be independent and immediate
contributors to the next level event or condition.
(8) The failure rate of each initiator must be constant and predictable. Specific
(noncomparative) estimates of failure probabilities are typically difficult to find, to achieve
agreement on, and to successfully use to drive conclusions. Comparative analyses are
typically as valuable with better receptions from the program and design teams.
3.6.7 Bibliography
Crosetti, P.A.: “Reliability and Faulttree Analysis Guide.” Department of Energy No. DOE
7645/22, 1982.
Dillon, B.S., and Singh, C.: “Engineering Reliability – New Techniques and Applications.” John Wiley
& Sons, 1981.
Fussell, J.B., and Burdick, G.R.: “Nuclear Systems Reliability Engineering and Risk Assessment.”
Society for Industrial and Applied Mathematics, 1977.
Hammer, W: “Handbook of System and Product Safety.” Prentice Hall, 1972.
Henley, E.J., and Kumamoto, H.: “Probabilistic Risk Assessment.” The Institute of Electrical and
Electronic Engineers, Inc., New York, 1991.
Malsaky, S.W.: “System Safety: Technology and Application.” Garland Press, 1983.
Roberts, N.H., Vesely, W.E., Haasl, D.F., and Goldberg, F.F.: “Fault Tree Handbook.” NUREG–0492,
U.S. Government Printing Office, Washington, DC.
Roland, H.E., and Moriarty, B.: “System Safety Engineering and Management.” John Wiley & Sons,
1982.
347
William S.G., Riley, J., and Koren, J.M.: “A New Approach to the Analysis of Reliability Block
Diagrams.” Proceedings from Annual Reliability and Maintainability Symposium, SAIC, Los
Altos, New Mexico, 1990.
Wynholds, W., Potterfield, R., and Bass, L.: “Fault Tree Graphics – Application to System Safety.”
Proceedings of the Second International System Safety Conference, 1975.
3.7 Success Tree Analysis
3.7.1 Description
A success tree analysis (STA) is a backwards (topdown) symbolic logic model generated in the
success domain. This model traces the success pathways from a predetermined, desirable condition or
event (TOP event) of a system to the successes (success tree initiators) that could act as causal agents.
An STA is the compliment of an FTA (sec. 3.6) which is generated in the failure domain with failure
pathways from undesirable events.
The STA includes generating a success tree (symbolic logic model), determining success proba
bilities for each tree initiator, propagating each initiator probability to determining the TOP event
probability and determining cut sets and path sets. In the success domain, a cut set is any group of
initiators that will, if they all occur, prevent the TOP event from occurring. A minimal cut set is a least
group of initiators that will, if they all occur, prevent the TOP event from occurring. A path set is a
group of success tree initiators that, if all of them occur, will guarantee the TOP event occurs.
The probability of success for a given event is defined as the number of successes per number of
attempts. This can be expressed as:
P
s
= S/(S+F) , where S = number of successes and F = number of failures.
Since reliability for a given event is also defined as the number of successes per number of
attempts, then
R = P
S
.
3.7.2 Application
Particularly useful for high energy systems (i.e., potentially high severity events), to ensure that
an ensemble of countermeasures adequately leads to a successful top event. This technique is a powerful
diagnostic tool for analysis of complex systems and is used as an aid for design improvement and is
applicable both to hardware and nonhardware systems. This technique also allows probabilistic assess
ment of causal benefits as well as prioritization of effort based upon root cause evaluation. The
subjective nature of the probability assessment is relegated to the lowest level (root causes of effects) in
this study rather than at top level. Sensitivity studies can be performed allowing assessment of the
sensitivity of study results to subjective numbers.
348
The STA is typically applied in phase C but may also be applied in phase D. A success tree can
be used to verify the logic of a fault tree. Since a success tree is the logic compliment of a fault tree, if a
success tree is generated from a fault tree, the logic of the success tree needs to be valid if the logic of a
fault tree is to be valid.
3.7.3 Procedures
Success trees, like fault trees, are constructed with various event and gate logic symbols. These
symbols are defined in table 36. Although many event and gate symbols exist, most success trees can be
constructed with the following four symbols: (1) TOP or intermediate event, (2) inclusive OR gate, (3)
AND gate, and (4) basic event. The procedures, as described in reference 3.9, to construct a fault tree
also apply to success tree generation and are illustrated in figure 323. The commercial computer
programs are similar, as are the cautions for use of probability values.
6. Repeat / continue…
4. Identify secondlevel
contributors.
5. Link secondlevel
contributors to TOP by
logic gates.
1. Identify desirable TOP
event.
3. Link contributors to TOP
by logic gates.
2. Identify firstlevel
contributors.
Basic Event…(“Leaf,”
“Initiator,” or “Basic”)
indicates limit of analytical
resolution.
Figure 323. Success tree construction process.
A success tree can be constructed from a fault tree. Transform a success tree from a fault tree by
simply changing all AND gates to OR gates and OR gates to AND gates, and restating each initiator,
intermediate event, and top event as a success opposed to a failure.
Determine the probability of success (P
s)
for each basic event or initiator. Sources for these suc
cess probabilities may be found from manufacturer’s data, industry consensus standards, MIL standards,
historical evidence (of similar systems), simulation or testing. Delphi estimates, and the log average
method. The Delphi technique (sec. 7.9) derives estimates from the consensus of experts. Remember that
the probability of success equals reliability (R) and may be determined from (P
F
) as shown in the
following equation:
P
S
= 1 – P
F
.
Once probabilities are estimated for all basic events or initiators, propagate these probabilities
through logic gates to the intermediate events and finally the TOP event. Use the expressions presented
in table 37 to propagate probabilities through logic gates.
349
Generate cut sets and path sets in the same manner as for fault trees, as presented in sections
3.7.3.3 and 3.7.3.4, respectively.
3.7.4 Example
The compliment success tree for the fault tree presented in section 3.6.4 is presented in figure 3
24.
Artificial Wakeup
Succeeds
0.9996
Possess
Keen Hearing
1.000
Alarm Clock Works
Properly
0.9996
Main, Plugin
Clock Works
0.9806
Functioning Clock
Components
0.9997
Mechanical
Component
Success
Hour
Hand Does
Not Jam
Works
0.9998
Hour
Hand Stays
On
0.9996
Electrical
Components
Perform
Properly
0.9997
Uninterrupted
Power
0.9885
Remembered
to Set Alarm
0.9923
Backup Clock
(Windup) Performs
Properly
0.9805
Unflawed
Mechanism
0.9996
Remembered
to Set Backup
Alarm
0.9923
to Wind Clock
0.9885
1.000
Remembered
Figure 324. Example success tree.
350
3.7.5 Advantages
An STA provides the following advantages:
3.9
(1) Assesses probability of favorable outcome of system operation.
(2) Compliments the FTA by providing a method to verify the logic of the fault tree.
3.7.6 Limitations
An STA possesses the following limitations:
3.9
(1) Address only one desirable condition or event that must be foreseen by the analyst. Thus,
several or many STA’s may be needed for a particular system.
(2) Success trees used for probabilistic assessment of large systems may not fit/run on
conventional PCbased software.
(3) The generation of an accurate probabilistic assessment may require significant time and
resources. Caution must be taken not to overdo the number generation portion.
(4) A success tree is not accurate unless all significant contributors to system successes are
anticipated.
(5) Events or conditions under the same logic gate must be independent of each other.
(6) Events or conditions at any level of the tree must be independent and immediate
contributors to the next level event or condition.
(7) The probability of success (reliability) of each initiator must be constant and predictable.
3.7.7 Bibliography
Henley, E.J., and Kumamoto, H.: “Probabilistic Risk Assessment.” The Institute of Electrical and
Electronic Engineers, Inc., New York, 1991.
3.8 Event Tree Analysis
3.8.1 Description
An event tree analysis (ETA), as described in references 3.6 and 3.12, is a forward (bottomup)
symbolic logic modeling technique generated in both the success and failure domain. This technique
explores system responses to an initiating “challenge” and enables assessment of the probability of an
unfavorable or favorable outcome. The system challenge may be a failure or fault, an undesirable event,
or a normal system operating command.
351
A generic event tree portrays all plausible system operating alternate paths from the initiating
event. A generic event tree is illustrated in figure 325. A Bernoulli model event tree uses binary
branching to illustrate that the system either succeeds or fails at each system logic branching node. A
Bernoulli model event tree is illustrated in figure 326. A decision tree is a specialized event tree with
unity probability for the system outcome.
Portray all credible system operating permutations.
.
SUCCESS
FAILURE
INITIATION
OPERATION/
OUTCOME
D
E
C
I
S
I
O
N
/
A
C
T
I
O
N
D
/
A
O/O
A B C
N
1
2
3
n
OPERATION/
OUTCOME
D
E
C
I
S
I
O
N
/
A
C
T
I
O
N
D
E
C
I
S
I
O
N
/
A
C
T
I
O
N
FAILURE
FAILURE
FAILURE
SUCCESS
SUCCESS
SUCCESS
OPERATION/
OUTCOME
Trace each path to eventual success or failure.
Figure 325. Event tree (generic case).
3.8.2 Application
The ETA is particularly useful in analyzing commandstart or commandstop protective devices,
emergency response systems, and engineered safety features. The technique is useful in evaluating
operating procedures, management decision options, and other nonhardware systems. The ETA is also
useful in evaluating effect and benefit of subtiered or redundant design countermeasures for design
trades and assessment.
An ETA may be used in conjunction with an FTA to provide a technique sensitivity assessment.
However, success or failure probabilities used must be used with caution to avoid the loss of credibility
of the analysis. In many cases it is best to stay with comparative probabilities rather than the “absolute”
values. Normalizing data to a standard, explicitly declared meaningless value is a useful technique here.
Also, confidence or error bands, on each cited probability number, are required to determine the signifi
cance of any quantitatively driven conclusion.
An ETA may also be performed to compliment an FMEA. This technique is typically performed
in phase C or E but may also be performed in phase D.
352
Reduce tree to simplified representation of
system behavior. Use binary branching.
Lead unrecoverable failures and
undefeatable successes directly to final
outcomes.
A fault tree or other analysis
may be necessary to determine
probability of the initiating event
or condition. (Unity probability
may be assumed.)
INITIATION
SUCCESS
FAILURE
FAILURE
FAILURE
FAILURE
FAILURE
FAILURE
FAILURE
SUCCESS
SUCCESS
SUCCESS
SUCCESS
SUCCESS
Figure 326. Event tree (Bernoulli model).
3.8.3 Procedures
The procedures, as described in reference 3.12, for performing an ETA are presented below.
(1) Identify the initiating challenge to the system being examined.
(2) Determine the paths (alternate logic sequences) by answering the question, “What happens
when the system is challenged by the initiation event?” By convention, trace successful
paths upwards and failure paths downwards.
a. For the general event tree, trace all plausible system operating permutations to a success
or failure termination.
b. For the Bernoulli model event tree, use binary branching to show the system pathways.
Simplify the tree by pruning unnecessary alternate branches of nonrecoverable failures
or undefeatable successes.
(3) Determine the probability of the initiating event by applying a fault tree (sec. 3.6) or other
analysis. For a decision tree, assume the probability of the initiating event is one.
(4) Determine the probability of each potential path by multiplying the individual probabilities
of events making up the path.
(5) Determine the probability of the system success by adding the probabilities for all paths
terminating in success.
(6) Determine the probability of the system failure by adding the probabilities for all paths
terminating in failure.
353
3.8.4 Example
An example of an ETA is presented in figure 327. The example includes the system and
scenario being assessed and the resulting event tree. Note that in this example the probability of the
challenging initiator is assumed to be one and the tree has been pruned to its simplest form by using
engineering logic. For example, since failure of the float switch is a nonrecoverable failure, its path leads
directly to a final failure outcome with no alternate paths. In a similar manner since successful operation
of the pump is an undefeatable success, its path also leads to a final success outcome with no alternate
paths.
3.8.5 Advantages
An ETA provides the following advantages:
(1) Enables the assessment of multiple, coexisting system faults and failures.
(2) Functions simultaneously in the failure of success domain.
(3) End events need not be anticipated.
(4) Potential singlepoint failures, areas of system vulnerability, and lowpayoff countermea
sures are identified and assessed, thereby guiding deployment of resources for improved
control of risk and optimized utilization of limited resources.
(5) Failure propagation paths of a system can be identified and traced. This can be a “quick and
dirty” comparative technique and provides very clear visibility of ineffective countermea
sures.
3.8.6 Limitations
An ETA possesses the following limitations:
(1) Address only one initiating challenge. Thus, multiple ETA’s may be needed for a particular
system.
(2) The initiating challenge is not disclosed by the analysis, but must be foreseen by the
analyst.
(3) Operating pathways must be foreseen by the analyst.
(4) Although multiple pathways to system failure may be disclosed, the levels of loss
associated with particular pathways may not be distinguishable without additional analyses.
(5) Specific, noncomparative success or failure probability estimates are typically difficult to
find, difficult to achieve agreement on, and to successfully use to drive conclusions.
Comparative analyses are typically as valuable, with better reception from the program and
design teams.
354
KLAXON PUMP
K
P
S
B
BACKGROUND/PROBLEM — A subgrade compartment containing
important control equipment is protected against flooding by the system
shown. Rising flood waters close float switch S, powering pump P from
an uninterruptible power supply. A klaxon K is also sounded, alerting
operators to perform manual bailing, B, should the pump fail. Either
pumping or bailing will dewater the compartment effectively. Assume
flooding has commenced, and analyze responses available to the
dewatering system…
• Develop an event tree representing system responses.
• Develop a reliability block diagram for the system.
• Develop a fault tree for the TOP event Failure to Dewater.
SIMPLIFYING ASSUMPTIONS:
• Power is available full time.
• Treat only the four system components S , P , K, and B.
• Consider operator error as included within the bailing function, B .
EVENT TREE…
[P
B
P
P
– P
B
P
P
P
S
–
P
B
P
K
P
P
+
P
B
P
K
P
P
P
S
]
[1 – P
S
– P
P
+ P
P
P
S
]
[P
P
– P
P
P
S
–
P
K
P
P
+
P
K
P
P
P
S
]
[P
K
P
P
– P
K
P
P
P
S
]
[P
P
– P
P
P
S
]
Water Rises
(1.0)
Float
Switch
Succeeds
(1 – P
S
)
Float
Switch
Fails
(P
S
)
Pump Succeeds
(1 – P
P
)
Pump Fails
(P
P
)
Klaxon Succeeds
(1 – P
K
)
Klaxon Fails
(P
K
)
Bailing Succeeds
(1 – P
B
)
Bailing
Fails
(P
B
)
S
U
C
C
E
S
S
F
A
I
L
U
R
E
[P
S
]
P
SUCCESS
= 1 – P
S
– P
K
P
P
+ P
K
P
P
P
S
– P
B
P
P
+ P
B
P
P
P
S
+ P
B
P
K
P
P
– P
B
P
K
P
P
P
S
P
FAILURE
= P
S
+ P
K
P
P
– P
K
P
P
P
S
+ P
B
P
P
– P
B
P
P
P
S
– P
B
P
K
P
P
+ P
B
P
K
P
P
P
S
P
SUCCESS
+ P
FAILURE
= 1
[P
P
– P
P
P
S
– P
K
P
P
+ P
K
P
P
P
S
–
P
B
P
P
+ P
B
P
P
P
S
+ P
B
P
K
P
P
–
P
B
P
K
P
P
P
S
]
Figure 327. Example ETA.
355
3.8.7 Bibliography
Battelle Columbus Division, “Guidelines for Hazard Evaluation Procedures.” 1985.
Henley, E.J., and Kumamoto, H., “Reliability Engineering and Risk Assessment.” New York, 1981.
Lees, F.P. “Loss Prevention in the Process Industries.” 2 vols., Butterworths, London, 1980.
3.9 Fault Tree, Reliability Block Diagram, and Event Tree Transformations
3.9.1 Description
Fault trees (sec. 3.6), RBD’s (sec. 3.5), and event trees (sec. 3.7) are all symbolic logic models.
Fault trees are generated in the failure domain, reliability diagrams are generated in the success domain,
and event trees are generated in the success and failure domains. These techniques, described in
reference 3.13 and presented below, transform any one of the above models into the other two by
translating equivalent logic from the success to failure or failure to success domain.
3.9.2 Application
These techniques are applicable by the analyst that wishes to exploit the benefits of the fault tree,
RBD, and event tree. Fault trees offer the analyst comprehensive qualitative or quantitative analysis.
RBD’s offer the analyst a simplistic method to represent system logic. Event trees allow the analyst to
assess a system in both the success and failure domains. This technique is typically performed in phase
C but may also be performed in phase B.
3.9.3 Procedures
The procedures for transforming a fault tree, RBD, or event tree to either of the other two logic
models are presented in the following sections.
3.13
3.9.3.1 Fault Tree to RBD Transformation
An RBD represents system component functions that, if these functions prevail, produces
success in place of a TOP fault event. A fault tree can be transformed into a reliability diagram as
illustrated in figure 328.
3.9.3.2 RBD and Fault TreetoEvent Tree Transformation
An event tree represents path sets in the success branches of the tree and all the cut sets in the
failure branches of the tree. Therefore, if the path sets and cut sets of a system are known for a certain
challenge to a system (TOP event of a fault tree), then an event tree can be constructed.
Cut sets and path sets may be obtained from a reliability diagram as shown in figure 329.
For large complex fault trees, cut sets and path sets are obtainable using the MOCUS algorithm
described in sections 3.6.3.3 and 3.6.3.4, respectively.
356
!
1 2 3
4 & 5 & 6
Nastiness
4 5 6
The OR gate in this position
indicates a series string of
component functions.
1 3
4
5
6
2
!
This AND gate indicates a
paralleled set of component
functions in the series string.
Figure 328. Fault tree to RBD transformation.
Path
Sets
1 / 2 / 3 / 4
1 / 2 / 3 / 5
1 / 2 / 3 / 6
Minimal
Cut Sets
1
2
3
4 / 5 / 6
!
6
5
4
1 2 3
Figure 329. Deriving cut and path sets from an RBD.
357
To transform an RBD into an event tree, proceed as shown in figure 330. To transform a fault
tree into an event tree, first transform the fault tree into an RBD (sec. 3.9.3.1).
1 3
4
5
6
2
!
1
2
3
4
5
6
Success
Failure
All of these parallel
elements must fail to
produce system failure.
Failure of any one of
these series elements
makes system failure
irreversible.
Figure 330. RBD to event tree transformation.
3.9.3.3 RBD to Fault Tree Transformation
A fault tree represents system functions which, if they fail, produce TOP event fault in place of
the success to which the reliability block path lead. The series nodes of an RBD denote an OR gate
beneath the TOP event of a fault tree. The parallel paths in an RBD denote the AND gate for redundant
component functions in a fault tree. Therefore, a reliability diagram can be transformed into a fault tree
as shown in figure 331.
3.9.3.4 Event Tree to RBD and Fault Tree Transformation
An event tree represents path sets in the success branches of the tree and all the cut sets in the
failure branches of the tree. To transform an event tree into an RBD, reverse the process illustrated in
figure 330. Once the RBD is formed, a fault tree can be formed as illustrated in figure 331. Also, an
event tree can be transformed into a fault tree by inspection as shown in figure 332.
3.9.4 Example
An RBD and fault tree are transformed from the example event tree presented in figure 327, and
presented in figure 333(a) and (b), respectively. All three of the models represent equivalent logic of the
system.
358
These series nodes
indicate an OR gate
beneath TOP.
1
2
3
4
5
6
7
!
!
1
2 3 4 5
6 & 7
Evil
6 7
2 & 3
Grief
4 & 5
Woe
These parallel paths indicate
AND gates for redundant
component functions.
Figure 331. RBD to fault tree transformation.
SUCCESS
FAILURE A
1
FAILURE B
1
FAILURE B
2
FAILURE C
FAILURE B
3
FAILURE A
2
FAILURE D
SUCCESS
SUCCESS
SUCCESS
SUCCESS
SUCCESS
SUCCESS
SUCCESS
FAILURE
18
19
20
21
30
1
i
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
22
23
24
25
26
27
28
29
FAILURE
A12
FAILURE
A
1
FAILURE
A
2
16 7* 1* i 26 12 5* 2 3* i
*Note that not all events represented here are failures.
Figure 332. Event tree to fault tree transformation.
359
FLOAT
SWITCH
S
PUMP
P
KLAXON
K
BAILING
B
CUT
SETS
S
P / K
P / B
PATH
SETS
S / P
S / K / B
See Figure 327.
(a) RBD.
EXACT SOLUTION:
P
TOP
= P
S
+ P
P
P
K
– P
P
P
K
P
S
+ P
B
P
P
–
P
B
P
P
P
S
– P
B
P
K
P
P
+ P
B
P
K
P
P
P
S
RARE EVENT APPROXIMATION:
P
TOP
= P
S
+ P
P
P
K
+ P
P
P
B
CUT
SETS
S
P / K
P / B
PATH
SETS
S / P
S / K / B
COMMAND
FAILURE
RESPONSE
FAILURE
FAILURE
TO
DEWATER
S
FLOAT
SWITCH
FAILS
OPEN
WATER
REMOVAL
FAILS
P
PUMP
FAILS
MANUAL
REMOVAL
FAILS
K
KLAXON
FAILS
B
BAILING
FAILS
See Figure 327.
(b) Fault tree.
Figure 333. Equivalent logic RBD and fault tree.
360
3.9.5 Advantages
These techniques allow the analyst to overcome weaknesses of one analysis technique by trans
forming a system model into an equivalent logic model as another analysis technique. For example, a
complex system that may be hard to model as a fault tree might be easily modeled with an RBD. Then,
the RBD can be transformed into a fault tree, and extensive quantitative or pseudoquantitative analysis
can be performed.
3.9.6 Limitations
These techniques possess the following limitations:
(1) No new information concerning the system is obtained and the models are only as good as
the models being transformed.
(2) The cut sets and path sets required to perform these transformations for large complex
systems may require many manhours or extensive computer resources to determine.
3.9.7 Bibliography
Gough, W.S., Riley, J., and Koren, J.M.: “A New Approach to the Analysis of Reliability Block
Diagrams.” Proceedings from Annual Reliability and Maintainability Symposium, SAIC, Los
Altos, New Mexico, 1990.
3.10 CauseConsequence Analysis
3.10.1 Description
A causeconsequence analysis is a symbolic logic technique described in references 3.6 and 3.14,
and presented below. This technique explores system responses to an initiating “challenge” and enables
assessment of the probabilities of unfavorable outcomes at each of a number of mutually exclusive loss
levels. The analyst starts with an initiating event and performs a forward (bottomup) analysis using an
event tree (sec. 3.8). This technique provides data similar to that available with an event tree; however, it
affords two advantages over the event tree—time sequencing of events is better portrayed, and discrete,
staged levels of outcome are analyzed.
The cause portion of this technique is a system challenge that may represent either a desired or
undesired event or condition. The cause may be a fault tree TOP event and is normally, but not always,
quantified as to probability. The consequence portion of this technique yields a display of potential out
comes representing incremental levels of success or failure. Each increment has an associated level of
assumed or calculated probability, based on variations of response available within the system.
A conceptual illustration of how a cause is assessed to understand its consequences is presented
in figure 334. Note that the cause has an associated probability, and each consequence has an associated
severity and probability.
361
P
0
S
1
, P
C1
S
3
, P
C3
S
n
, P
Cn
S
2
, P
C2
CONSEQUENCE 3
CONSEQUENCE 2
CONSEQUENCE 1
CONSEQUENCE n
ANALYSIS
S
n
= Severity of
the n
th
Consequence
P
Cn
= Probability of
the n
th
Consequence
occurring
CAUSE
OVERPRESSURE
RELIEF FAILS
Y N
P
0
, probability of
the Cause, may be
determined by Fault
Tree Analysis.
Figure 334. Relationship between cause and consequence.
3.10.2 Application
This technique is typically applied in phase C or E but may also be applied in phase D. The
causeconsequence analysis is particularly useful in analyzing commandstart/commandstop protective
devices, emergency response systems, and engineered safety features. Causeconsequence analyses are
useful in evaluating operating procedures, management decision options, and other nonhardware
systems. Also, it will evaluate the effect/benefit of subtiered/redundant design countermeasures for
design trades and assessment. This technique may be used in conjunction with an FTA to provide a
technique sensitivity assessment. This technique may also be used to compliment an FMEA.
3.10.3 Procedures
The procedures, as described in references 3.6 and 3.14, for performing a causeconsequence
analysis are presented below.
(1) Identify the initiating event that challenges the system.
(2) Determine the probability, P
0,
that this event will occur. This probability may be
determined from an FTA (sec. 3.6.3.2) or assumed.
(3) Next, trace the possible consequences to the system from the initiating event. At various
levels the path may branch with two possible outcomes. Construct the consequence
diagram by asking the following questions:
3.6
a. What circumstances allow this event to proceed to subsequent events?
b. What other events may occur under different system operating circumstances?
362
c. What other system elements does this event influence?
d. What subsequent event could possibly result as an outcome of this event?
(4) Use the symbols presented in table 38 to construct the consequence diagram.
Table 38. Causeconsequence tree construction symbols.
3.14
AND
Gate
OR
Gate
Coexistence of all inputs opens gate and produces
an output.
Gate opens to produce output when any input
exists.
Branching
Operator
Event
Y N
Output is “Yes” if condition is met and “No” if it
is not met. Branching operator statement may be
written in either the fault or the success domain.
The outputs are mutually exclusive, therefore
P
Y
+P
N
= 1.
Basic
Event
An independent initiating event, representing the
lower resolution limit of the analysis.
Consequence
Descriptor
End event/condition to which analysis leads, with
the severity level stated.
Symbol Name Description
(5) The format of the consequence tree is presented in figure 335. Note that all paths lead into
branching operators or consequence descriptors. The branching operator always has one
input and two output paths (yes and no). The consequence descriptor has one input, no
outputs, and is a termination point in the diagram.
(6) For each branching operator, establish the probability, P
i
, that the event can happen.
Therefore, P
i
and (1–P
i
) are the probabilities for the yes and no paths from the branch
operator, respectively. This step is often difficult and subjective due to a scarcity of data.
Probability bands are often useful to provide an understanding of the analyst's confidence
in the delineated probabilities.
(7) Determine the probability of each consequence descriptor, P
ci
, by multiplying event
probabilities along the path that terminates at that consequence descriptor.
(8) Finally, determine the severity of each consequence descriptor, S
i
.
363
Note that, because the analysis is exhaustive…
(P
0
P
1
) + P
0
(1 – P
1
) (1 – P
2
) + P
0
(1 – P
1
) P
2
= P
0
CONSEQUENCE
DESCRIPTOR 1
CONSEQUENCE
DESCRIPTOR 2
CONSEQUENCE
DESCRIPTOR 3
BRANCHING
OPERATOR
Y N
BRANCHING
OPERATOR
Y N
P
0
P
1
P
0
P
1
INITIATING
CHALLENGE
P
0
(1 – P
1
)
P
2
P
0
(1 – P
1
) P
2
P
0
(1 – P
1
) (1 – P
2
)
Fault trees or other analyses
may be used to establish
probabilities for the Initiating
Challenge and for Branching
Operator Y/N outcomes.
Figure 335. Causeconsequence analysis format.
3.10.4 Example
*
Problem:
A copying machine uses an electrically heated drum to fix dry ink to copy paper. The drum
heater is thermostatically controlled. The drum is also equipped with an automatic overheat safety cutoff
to prevent damage to the copier. The probability of failure is finite for both the drum thermostat and the
overheat cutoff. Combustibles are often present in the copying room near the machine. Uncontrolled
drum temperature can rise high enough to ignite them. The room is equipped with an automatic sprinkler
system initiated by a heat detector. Employees frequent the room and can initiate an emergency response
alarm in the event of fire. After a delay, a fire brigade responds to extinguish the blaze.
The causeconsequence analysis for the above problem is presented in figure 336.
3.10.5 Advantages
Causeconsequence analyses provide the following advantages:
3.14
(1) The analysis is not limited to a “worstcredible case” consequence for a given failure.
Therefore, a less conservative, more realistic assessment is possible.
(2) Enable assessment of multiple, coexisting system faults and failures.
(3) End events need not be anticipated.
(4) The time order of events is examined.
*
This example was provided courtesy of Sverdrup Technology, Inc., Tullahoma, Tennessee.
3.1.
364
FIRE
BRIGADE
RESPONSE
FAILS
EMPLOYEE
DETECTION/
RESPONSE
FAILS
BUILDING DAMAGE
≈ $1.5M
WATER/FIRE/SMOKE
DAMAGE ≈ $50,000
EMERGENCY
RESPONSE
FAILS
Y N
NEARBY
COMBUSTIBLES
IGNITE
Y N
DRUM
OVERHEATS
DRUM
THERMOSTAT
FAILS
CLOSED
OVERHEAT
CUTOFF
FAILS
COMBUSTIBLES
PRESENT
NEARBY
IGNITION
TEMPERATURE
REACHED
COPIER DAMAGE
≈ $250
HEAT DETECTOR/
AUTO SPRINKLER
FAIL
Y N
P
0
P
1
P
0
(1 – P
1
)
P
2
MANUFACTURER'S
TEST DATA P
0
P
1
(1 – P
2
)
P
3
BUILDING LOSS
≈ $6.5M
P
0
P
1
P
2
P
3
P
0
P
1
P
2
(1 – P
3
)
Note that, because the analysis is exhaustive…
P
0
P
1
P
2
P
3
+ P
0
P
1
P
2
(1 – P
3
) + P
0
P
1
(1 – P
2
) + P
0
(1 – P
1
) = P
0
Figure 336. Example causeconsequence analysis.
(5) Probabilities of unfavorable system operating consequences can be determined for a
number of discrete, mutually exclusive levels of loss outcome. Therefore, the scale of
partial successes and failures is discernible.
(6) Potential singlepoint failures or successes, areas of system vulnerability, and lowpayoff
countermeasures are identified and assessed, thereby guiding deployment of resources for
improved control of risk and optimized utilization of limited resources.
3.10.6 Limitations
Causeconsequence analyses possess the following limitations:
3.14
(1) Address only one initiating challenge. Thus, multiple analyses may be needed for a
particular system.
(2) The initiating challenge is not disclosed by the analysis, but must be foreseen by the
analyst.
365
(3) Operating pathways must be foreseen by the analysts.
(4) The establishment of probabilities is often difficult and controversial.
(5) Determining the severity on consequences may be subjective and difficult for the analyst to
defend.
3.10.7 Bibliography
Battelle Columbus Division, “Guidelines for Hazard Evaluation Procedures.” 1985.
Burdic, G.R., and Fussell, J.B.: “On the Adaptation of CauseConsequence Analysis to U.S Nuclear
Power Systems Reliability and Risk Assessment, System Reliability and Risk Assessment.” JBF
Associates, Inc., Knoxville, Tennessee, 1983.
Greenberg, H.R., and Cramer, J.J. “Risk Assessment and Risk Management for the Chemical Process
Industry.” Van Nostrand Reinhold, 1991.
Lees, F.P.: “Loss Prevention in the Process Industries.” 2 vols, Butterworths, London, 1980.
3.11 Directed Graphic (Digraph) Matrix Analysis
3.11.1 Description
Directed graph (digraph) matrix analysis, as described in reference 3.15, is a technique using
matrix representation of symbolic logic models to analyze functional system interactions. Logic models
are first generated in the success domain, then converted into the failure domain. However, it should be
noted that models can be directly created in the failure domain, without first creating the model in the
success domain.
This technique consists of four phases. First, the analyst determines combinations of systems or
combinations of subsystems within a single system for thorough assessment. This phase is parallel to
determining failure propagation paths using an ETA (sec. 3.8). The second phase consists of
constructing a digraph model in the success domain, then converting this model to a digraph model in
the failure domain for each failure propagation path. The third phase consists of separating the digraph
models into independent models, then determining the singleton and doubleton minimal cutsets of each
failure propagation path. Finally, the fourth phase consists of an assessment of the minimal cut sets
relative to probability of occurrence.
3.11.2 Application
This technique, according to reference 3.15, can be used independently or as an element of a
PRA (sec. 3.15). If this technique is used as part of a PRA, then it is performed after the identification of
failure propagation paths by ETA but before FTA’s are begun.
3.15
This technique is applied to evaluate
the failure propagation paths involving several systems and their support systems, or within a single
system involving several system elements (subsystem, component, part, etc.) and is best applied in phase
B.
366
3.11.3 Procedures
Presented below is a summary of the detailed procedures found in reference 3.15 for performing
a digraph matrix analysis.
(1) Identify the associated group of systems (or associated system elements of a single system)
to be thoroughly evaluated. Use event trees (sec. 3.8) to identify failure propagation paths.
For a complete analysis, identify every credible initiator to an undesirable event and
prepare an event tree that illustrates each specific failure propagation path.
a. Acquire pertinent information concerning the collection of systems to be assessed, such
as design specifications and packages, safety assessment reports (such as PHA’s, sec.
3.2), and prior safety or reliability studies.
b. Study checklists of potential initiating challenges. From these checklists develop a list
of initiators that are applicable to the systems being studied.
c. Develop event trees for each initiating challenge to the system.
d. Prepare a list of failure propagation paths from step 1c. Assume unity probability for all
systems required to work in the failure propagation path. This simplifying assumption
leaves only failure propagation paths that are combinations of systems that must fail for
a serious threat to be posed.
(2) Construct a digraph model for each possible failure propagation path. Use a backward, top
down approach to construct a toplevel digraph, then expand each element into its own
digraph. Continue expanding the elements of new digraphs until the desired resolution level
of the analysis is reached. An outline of the steps involved in producing the digraphs is
presented below.
a. Create a success domain digraph model for each success path. Connect upstream
elements to a downstream element with an AND gate if the upstream element relies on
the successful operation of all the downstream components. Connect upstream elements
to a downstream element with an OR gate if the upstream element relies on the
successful operation of only one of two or more downstream elements. The symbols for
AND and OR gates for a digraph are different than those used for a fault tree, however
they represent the same logic as the fault tree symbols. A comparison between the
digraph and fault tree symbols is presented in figure 337.
b. Form a failure domain model by taking the model generated in step 2a and interchange
all AND gates with OR gates and all OR gates with AND gates. This failure domain
model represents a path for failure propagation.
c. Form an adjacency matrix that represents the digraph. The matrix is constructed by the
process illustrated in figure 338.
367
AND Gate OR Gate
Digraph
A
B
C
A
B
C
Fault tree
A B
C
A B
C
Represented logic Event C will occur only if
both event A and event B
occur.
Event C will occur only if
event A or event B occurs.
Figure 337. Comparison between digraph and fault tree logic gates.
d. Next link all connected elements in the adjacency matrix. This is accomplished by
processing the adjacency matrix with the reachability code. This code is described in
detail in reference 3.15. The output of this code will show all elements connected by a
path and illustrate which elements can be reached from a specific element, therefore all
possible paths between pairs of nodes in the network. Next, use this information to
determine singleton and doubleton cut sets.
e. Determine minimal singleton and doubleton cut sets from the cut sets determined in
step 2d.
(3) Subdivide the digraph into independent digraphs if the success domain digraph model
becomes too large to determine singleton and doubleton cut sets for the computer platform
being used. Then determine singleton and doubleton minimal cut sets of the smaller
independent digraphs.
(4) Assess the singleton and doubleton minimal cut sets. This assessment can be conducted in a
manner similar to that for a conventional PRA (sec. 3.15) in which risk is assessed with the
probability of the cut sets occurring and the severity of the consequence of the failure
propagation path.
368
Type Digraph Adjacency Matrix
Direct
connection
Element A Element B
A B
A 0 1
B0 0
AND gate
Element A
Element C
Element B
A B C
A 0 0 B
B0 0 A
C0 0 0
OR gate
Element C
Element A
Element B
A B C
A 0 0 1
B0 0 1
C0 0 0
Figure 338. Construction of digraph adjacency matrix.
3.11.4 Example
An example digraph matrix analysis, adapted from reference 3.15, for a simple system is
illustrated in figure 339. The system consists of two redundant power supplies to power a motor that
drives a pump. The success domain model of this system is presented in figure 339(a). Note that this
model represents the success path for successful operation of the pump. The failure domain model,
presented in figure 339(b), was generated by replacing the OR gate in the success domain model with
an AND gate. Inspection of the two models suggests that for simple systems the failure domain model
can easily be generated without first generating the success model. In cases with more complex systems,
first generating a success domain model may prove to be beneficial.
The adjacency matrix and adjacency elements are presented in figures 339(c) and (d),
respectively. The adjacency matrix illustrates whether there is a direct path from node i to node j. If
matrix element (i,j) = 1, there is a path from node i to node j. For example, element (M,P) = 1, which
means there is a straight (uninterrupted) and unconditional path between the motor and pump. If element
(i,j) = 0, there is no path from node i to j. For example, element (PS–1, PS–2) = 0, which means there is
a no straight path between the main power supply and the auxiliary power supply. If the adjacency
element (i,j) is ≠ 0 or 1, then there is a second component that must fail along with component i to cause
component j to fail. For example, adjacency element (PS–1, M) is equal to PS–2 (nonzero or 1 value).
369
This symbol represent the second component that must fail, given the failure of PS–1, to cause M to fail
to operate (i.e., failure of both the main and auxiliary power supplies will cause the motor not to
operate).
The reachability matrix and reachability elements are presented in figure 339(e) and (f), respec
tively. The methodology to generate the reachability matrix from the adjacency matrix is presented in
reference 3.15. Simply stated, the reachability matrix illustrates the pairs of nodes that a path exists
between, by connecting linked pairs from the adjacency matrix. Therefore the reachability matrix
illustrates the complete pathways (through linked node pairs) of the graphical model elements illustrated
by the adjacency matrix. Processing the adjacency matrix into the reachability matrix yields the paths
between all pairs of nodes. The reachability elements are derived from the reachability matrix in the
same manner that adjacency elements are derived from the adjacency matrix. Note, in this example, that
the reachability elements include all the adjacent elements and the new information that if both PS–1 and
PS–2 fail, then P will not operate (even though neither PS–1 or PS–2 are directly adjacent to P).
Therefore, the reachability matrix yielded the new information that if both power supplies failed, the
pump will not operate.
The summary matrix presented in figure 339(g) illustrates which components can lead to failure
of the pump, P. If an “*” is entered as a matrix element (i,j) and either i or j is a value of 1, then the other
corresponding component i or j is a singleton. The only singleton in this system is the motor, i.e., the
single failure of the motor will cause the pump not to operate. If a “*” is entered as a matrix element (i,j)
that corresponds to component i and component j, then component i and component j form a doubleton.
The only doubleton of this system is the pair of redundant power supplies, i.e., failure of both the main
and auxiliary power supplies will cause the pump not to operate.
Obviously, in this example the singletons (single point failures) and doubletons (double point
failures) could have easily been identified without performing a digraph matrix analysis. However, for
complex systems which are modeled with many nodes and logic gates, this technique allows
determination of singletons and doubletons which otherwise would not be as readily identified.
3.11.5 Advantages
The digraph matrix analysis provides the following advantages:
3.15
(1) The analysis allows the analyst to examine each failure propagation path through several
systems and their support systems in one single model. Unlike the FTA with failure propa
gation paths divided in accordance to arbitrarily defined systems, this approach allows
more rigorous subdividing of the independent subgraphs.
(2) Since the technique identifies singleton and doubleton minimal cut sets without first deter
mining all minimal cut sets, considerable computer resources can be saved over other
methods such as the FTA.
370
Motor, M
Main Power
Supply,
PS1
Auxiliary
Power
Supply,
PS2
Pump, P
Main Power
Supply,
PS1
Auxiliary
Power
Supply,
PS2
Motor, M
Pump, P
(a) Success domain model. (b) Failure domain model.
PS1 PS2 M P
PS1 0 0 PS2 0
PS2 0 0 PS1 0
M 0 0 0 1
P 0 0 0 0
PS1, M, PS2
PS2, M, PS1
M, P, 1
(c) Adjacency matrix. (d) Adjacency elements.
PS1 PS2 M P
PS1 0 0 PS2 PS2
PS2 0 0 PS1 PS1
M 0 0 0 1
P 0 0 0 0
PS1, M, PS2 (Adjacent)
PS1, P, PS2
PS2, M, PS1 (Adjacent)
PS2, P, PS1
M, P, 1 (Adjacent)
(e) Reachability matrix. (f) Reachability element.
Figure 339. Example digraph matrix analysis—Continued
371
1 PS1 PS2 M P
PS1  *   
PS2   *  
M *    
1    * 
P     
Singletons: M
Doubletons: PS1, PS2
(g) Summary matrix.
Figure 339. Example digraph matrix analysis—Continued.
3.11.6 Limitations
Digraph matrix analyses possess the following limitations.
3.15
(1) Trained analysts and computer codes to perform this technique may be limited.
(2) For particular types of logic models, complete treatment may require more computer
resources than FTA’s.
3.11.7 Bibliography
Grumman Space Station Division, “Digraph Analysis Assessment Report.” Reston Virginia, October
1991.
Kandel, A., and Avni, E.: “Engineering Risk and Hazard Assessment.” vol. 2, CRC Press Inc., Boca
Raton, Florida.
3.12 Combinatorial Failure Probability Analysis Using Subjective Information
3.12.1 Description
The combinatorial failure probability analysis using subjective information is described in refer
ence 3.16 and presented below. This technique was developed by the System Effectiveness and Safety
Technical Committee (SESTC) of the American Institute of Aeronautics and Astronomics (AIAA), in
1982. This technique provides the analyst a procedure to propagate probability data derived from the
subjective probability scales defined in MIL–STD–882C.
3.2
372
3.12.2 Application
This technique is typically performed in phase C and is applicable when no quantitative failure
probability data are available and may be used in conjunction with other analyses such as an RBD (sec.
3.5), FTA (sec. 3.6), STA (sec. 3.7), ETA (sec. 3.8), and causeconsequence analysis (sec. 3.10).
3.12.3 Procedures
The procedures, as described in reference 3.16, for a combinatorial failure probability analysis
using subjective information are presented below.
(1) Arbitrary, dimensionless “probability values” have been assigned to the probability incre
ments (frequent, probable, occasional, remote, and improbable) defined in MIL–STD–
882C.
3.2
The subjective scale for these arbitrary values is presented in table 39. Descriptive
words and definitions for the level of the scale are also given in this table.
Table 39. Combinatorial failure probability analysis subjective scale.
AIAA/SESTC MIL–STD–882C
Threshold
Levels
Probability
Level
*
Level
Descriptive Word
Definition
8×10
–2
to
1.00000
3×10
–1
A Frequent Likely to occur frequently.
8×10
–3
to
8×10
–2
3×10
–2
B Probable Will occur several times in life of an item.
8×10
–4
to
8×10
–3
3×10
–3
C Occasional Likely to occur sometime in life on an item.
8×10
–5
to
8×10
–4
3×10
–4
D Remote Unlikely but possible to occur in life of an
item.
0.00000 to
8×10
–5
3×10
–4
E Improbable So unlikely if can be assumed occurrence may
not be experienced.
*
Arbitrarily selected, dimensionless numbers.
Table provided courtesy of Sverdrup Technology, Inc., Tullahoma, Tennessee.
(2) Estimate subjective failure probabilities of contributor events or conditions using the scale
defined in MIL–STD–882C.
3.2
Select and consistently apply the same probability exposure
interval (operating duration or number of events) for every initiator probability estimate
used in the analysis.
(3) Correlate the subjective estimate (step 2) with the arbitrary, dimensionless values (step 1).
Propagate these values in the same manner as quantitative data is combined in classical
numerical methods (such as presented in figs. 318 and 319).
(4) Convert the final probability number resulting from propagation (step 3) back into the sub
jective scale defined in MIL–STD–882C.
3.2
373
3.12.4 Example
The following example
*
uses this subjective combinatorial technique in a fault tree problem.
Problem/Background:
• A large rotating machine has six mainshaft bearings. Replacement of a bearing costs $18,000
and requires 3 wk of down time.
• Each bearing is served by:
• pressurized lubrication oil
• a watercooled jacket
• a temperature sensing/alarm/shutdown system.
• In addition, there are sensing/alarm/shutdown systems for:
• lube pressure failure
• cooling water loss of flow.
• If they function properly, these systems will stop operation of the rotating machine early
enough to prevent bearing damage. (System sensitivity makes the necessary allowance for
machine “rollout” or “coasting.”)
• Failure records for the individual system components are not available, but probabilities can
be estimated using the subjective scale of MIL–STD–882C.
3.2
What is the probability that any one of the six bearings will suffer burnout during the coming
decade?
The system schematic and fault tree are presented in figure 340(a) and (b), respectively. Note
both the arbitrary subjective probability value and letter representing the relevant probability level from
table 39 are presented for each fault tree initiator.
3.12.5 Advantages
This technique allows the analyst to perform a probabilistic assessment based on the exercise of
subjective engineering judgment when no quantitative probability estimates are available.
3.12.6 Limitations
This technique should only be used when actual quantitative failure rate data is not available. The
use of actual quantitative data is preferred over this method. This tool should only be used for
comparative analysis only. Data and results, unless used in a comparative fashion, may be poorly
received.
*
This example was provided courtesy of Sverdrup Technology, Inc., Tullahoma, Tennessee.
3.16
374
Bearing Burnout Loss Penalty:
• $18,000 Replacement Costs
• 3Week Interruption of Use
UTILITY SUBSYSTEMS
PROTECTIVE FEATURES
• Sensing
• Alarm
• Shutdown
dT
/
dt
T
H
2
O
Flow
Lube
Pressure
(a) System schematic.
6x10
8
3x10
3 3x10 4
3x10
3
3x10
4
3x10
2
9x10
7
9x10
7
2x10
6
C D
B
E
D C
BEARING
BURNOUT
UNRESOLVED
UTILITY SERVICE
FAILURE
SHUTDOWN
FAILURE
UNRESOLVED
LUBE
FAILURE
UNRESOLVED
COOLANT
FAILURE
LUBE
PRESSURE
FAILURE
COOLING
WATER
FAILURE
SHUTDOWN
FAILURE
SHUTDOWN
FAILURE
Bearing burnout is “ Improbable” f or any one
of t he six bearings over t he 10year period.
10year f ailure probabilit y est imat es
are ent ered at t he lowest levels of t he
t ree and propagat ed upward.
Bearing Temperat ure
Sensing/ Alarm/ Shut down
Failure
Coolant Loss
Sensing/ Alarm/ Shut down
Failure
Lube Pressure
Sensing/ Alarm/ Shut down
Failure
(b) System fault tree.
Figure 340. Example combinatorial failure probability analysis.
375
3.13 Failure Mode Information Propagation Modeling
3.13.1 Description
Failure mode information propagation modeling is a qualitative analysis method described in
reference 3.17 and presented below. This technique involves separating a system into its basic functional
components and examines the benefit of measuring precedent failure information that may be
transmitted between components of a system. This information may be transmitted during the initial
outset of a variety of failure modes. The technique provides insight into both the types of information
that should be measured to safeguard the system, and location within the system at which sensors might
be appropriately positioned.
3.13.2 Application
This technique effectively directs resource deployment to optimally safeguard a system against
potential failures by identifying measurement requirements. These requirements are defined in terms of
measured parameter, sensor type, and sensor location. This technique is best applied in phase C but may
also be applied in phase D.
3.13.3 Procedures
The procedures, as described in reference 3.17, to perform failure mode information propagation
modeling are presented below.
(1) Divide the system into its principle functional components and assign a number to each
component. Like the FMEA (sec. 3.4), the resolution of this analysis is dependent upon the
level (i.e., subsystems, assemblies, subassemblies, or piece parts) to which the system
elements are resolved.
(2) Identify the physical links (energy flow and shared stress) between the components of the
system. These links include such items as electrical power, air flow, liquid flow, gas flow,
thermal heat transfer, friction, spring, rolling element, etc.
(3) Identify and record the failure modes for each component and assign a letter to each failure
mode for each component.
(4) Identify and record the flow of failure mode information at each physical link that is
available externally to each component and transmitted to one or more other components.
(5) Classify the failure mode information constituents by their signal characteristics (e.g.,
thermal, pressure, acceleration, etc.).
(6) Identify the minimal success sets of the sensor network. A minimal success set is a sensor
group that encompasses all failure modes.
(7) Assess the various minimal success sets in terms of feasibility, cost, and effectiveness. The
following questions should be asked:
a. Feasibility. Do the sensors currently exist or can they be developed? Can they be
obtained in time to satisfy schedule requirements?
376
b. Cost. Is the cost of installing, maintaining, and operating the sensor network less than
the cost of the failure that the system is being safeguarded against?
c. Effectiveness. Are there other preventive maintenance activities more effective than
installing a sensor network? Will the sensing network forewarn before the start of
system failures or does it just announce system crashes? Will the sensors impede
normal system operation? Will they degrade system performance? Will they pose any
new hazards to the system? Will the sensor network operate dependably? Will the
sensors have adequate sensor redundancy?
3.13.4 Example
The following example
*
uses failure mode information propagation modeling to a sensor
network success set for a system.
Problem:
Consider a ventilating fan powered by an electric motor through a belt drive. A common frame
structure supports both the motor and a bearing, through which power is delivered to the fan. (Consider
motor bearings as integral parts of the motor.) Assume a constant aerodynamic fan load. A schematic of
the system is presented in figure 341(a). Determine sensor network minimal success sets for the system.
Solution:
(1) Perform steps 1–5 identified in section 3.13.3. These steps are explained below and illus
trated in figure 341(b).
a. Step 1. Divide the system into its principle functional components and assign a number
to each component. These are the electrical motor, fan belt, fan, frame, and bearing.
b. Step 2. Identify the physical links (energy flow and shared stress) between the compo
nents of the system. The electric motor, for example, has electrical power input, is
linked to the fan belt by friction, and is mechanically and thermally linked to the frame.
c. Step 3. Identify and record the failure modes for each component and assign a letter to
each failure mode. For example, the failure modes for the fan include shaft or rotor
binding, bearing vibration, open winding, and shorted winding.
d. Step 4. Catalog the flow of failure mode information at each physical link that is
available externally to each component and transmitted to one or more other
components. For example, for the mechanical link between the electric motor and
frame, the failure information available includes electric motor bearing vibrations (1–
B), fan belt slipping and breaking (2–A/B), and bearing binding (5–A).
e. Step 5. Classify the failure mode information constituents by their signal
characteristics. For example, the electric motor bearing vibration (1–B) and fan bearing
vibration (5–B) can be monitored by an accelerometer at test point 4/1 (between frame,
component 1 and electric motor, component 4).
*
This example was provided courtesy of Sverdrup Technology, Inc., Tullahoma, Tennessee.
3.16
377
(2) From the information displayed in figure 341(b), construct a matrix of failure mode versus
sensor type (with each test point identified). Determine the minimum success sets of mea
surement sensors. These sets are sensor groups that encompass all failure modes. The
matrix and minimum success sets for this system are presented in figure 341(c).
3.13.5 Advantages
Information propagation modeling provides the following advantages:
3.17
(1) Allows the analyst to identify measurement requirements, that, if implemented, can help
safeguard a system by providing warnings at the onset of a failure mode that threatens the
system.
(2) Compliments an FMEA.
3.13.6 Limitations
Information propagation modeling possesses the following limitations:
3.17
(1) This technique is only applicable if the system is operating in a nearnormal range, and for
the instant of time immediately prior to the initiation of a failure mode.
(2) Externally induced and common cause faults are not identified or addressed.
(3) The risks of the failure modes are not quantified in terms of criticality and severity.
(4) The propagation of a failure through the system is not addressed.
3.14 Probabilistic Design Analysis
3.14.1 Description
A PDA, as described in references 3.8 and 3.18, is a methodology to assess relative component
reliability for given failure modes. The component is characterized by a pair of transfer functions that
represent the load (stress, or burden) that the component is placed under by a given failure mode, and
capability (strength) the component has to withstand failure in that mode. The variables of these transfer
functions are represented by probability density functions. Given that the probability distributions for
both the load and capability functions are independent, the interference area of these two probability
distributions is indicative of failure. Under these conditions, a point estimate for failure of the
component relative to the failure mode under consideration can be determined.
3.14.2 Application
A PDA can be used to analyze the reliability of a component during phase C of a program. The
PDA approach offers an alternative to the more traditional approach of using safety factors and margins
to ensure component reliability. This traditional approach is vulnerable if significant experience and
historical data are not available for components similar to that which is being considered.
3.8 3.18
378
Elements:
• Electric Motor
• Fan Belt
• Bearing
• Fan
• Frame
(a) System schematic.
Electric
Motor Fan Belt Fan
Frame Bearing
1 2 3
4 5
A. Binding
B. Vibration
A. Blade Damage
A. Slipping
B. Breaking
A. Shaft/Rotor Binding
B. Bearing Vibration
C. Open Winding
D. Shorted Winding
1A/C/D
2A/B
3A
5A
1A/C/D
2A/B
3A
5A
1A/C/D
2A/B
3A
5A
1A/C/D
2A/B
3A
5A
1B
2A/B
5A
1A/C/D
3A
5A/B
3A
5A/B
0/1 1/2
4/1
4/5
3/5
2/3
3/00
Accelerometer
3A
5B
Flow Monitor
1A/C/D
2A/B
3A
5A Accelerometer
1B
5B
Tachometer
1A/C/D
2A/B
3A
5A
Belt Slip Monitor
2A/B
Tachometer
1A/C/D
2A/B
3A
5A
Belt Slip
Monitor
2A/B
Power Monitor
1A/C/D
2A/B
3A
5A
Heat Flux Monitor
1A/C/D
Accelerometer
3A
5B
• • • • • • • • • •
Electrical Power
Friction
Air Flow
Rolling Element
Mechanical
Thermal
Spring
Gas Flow
Liquid Flow
(b)
Model.
Figure 341. Example failure mode information propagation model—Continued
379
Minimal Success Sets*
Power Monitor at 0/1
or
Tachometer at 1/2
or
Tachometer at 2/3
or
Flow Monitor at 3/00
and
Accelerometer at 4/1
*Sensor groups that envelope all
failure modes
1A
1B
1C
1D
2A
2B
3A
4
5A
5B
Failure
Mode
3/00 3/5 4/5 4/1 2/3 1/2 0/1
P
o
w
e
r
M
o
n
i
t
o
r
T
a
c
h
o
m
e
t
e
r
B
e
l
t
S
l
i
p
M
o
n
i
t
o
r
T
a
c
h
o
m
e
t
e
r
B
e
l
t
S
l
i
p
M
o
n
i
t
o
r
F
l
o
w
M
o
n
i
t
o
r
A
c
c
e
l
e
r
o
m
e
t
e
r
A
c
c
e
l
e
r
o
m
e
t
e
r
A
c
c
e
l
e
r
o
m
e
t
e
r
H
e
a
t
F
l
u
x
M
o
n
i
t
o
r
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
S
e
n
s
o
r
T
e
s
t
P
o
i
n
t
Not Applicable
(c) Minimal success sets.
Figure 341. Example failure mode information propagation model—Continued.
3.14.3 Procedures
The procedures, adapted from reference 3.8 and 3.18, for performing a PDA in the context of a
total design reliability program for a system are presented below.
(1) Specify the system design requirements. These requirements should be stated in clear and
concise terms that are measurable and verifiable.
(2) Identify variables and parameters that are related to the design.
(3) Identify the failure modes of the system by using a method such as a FMEA (sec. 3.4).
(4) Confirm the selection of critical design parameters.
(5) Establish relationships between the critical parameters and organizational, programmatic,
and established failure criteria.
(6) Ascertain the reliability associated with each critical failure mode with the following proba
bilistic analysis method:
a. Identify the random variables that effect the variation in the load to be imposed on the
component for the given failure mode. Incorporate these random variables into a
transfer function that represents this load (stress, or burden).
Load Transfer Function: L = f
L
(X
1
, X
2
, X
3
, ....X
n
).
380
b. Identify the random variables that affect the variation in the capability of the component
to withstand the load imposed for the given failure mode. Incorporate these random
variables into a transfer function that represent this capability (strength).
Capability Transfer Function: C = g
C
(Y
1
, Y
2
, Y
3
, ....Y
m
).
c. Gather data to perform the load and capability calculations.
d. Determine probability distributions of the load (stress, or burden) and capability
(strength) of the failure mode. Consider each variable of the transfer function as a prob
ability density function (illustrated in figure 342). The density function can be repre
sented as either a discrete variable distribution using empirical test data, or as a continu
ously variable form of the density function.
Note: The area under an entire probability density function curve is equal to a
probability of one, therefore a range between two values of the independent random
variable of a density function curve is equal to a probability less than or equal to one.
Probability density functions of both load and capability continuous random variables
for a given failure mode are presented in figure 343. Also illustrated in this figure is
the interference of the load and capability density functions. For independent load and
capability functions, this interference is indicative the failure mode will occur. In figure
343, both density functions are normal distributions with different means and
variances. However, generally one or both of these density functions may be an
exponential, log normal, gamma, Weibull, or other distribution.
e. Calculate the reliability (R) for the failure mode from the load and capability
distributions. Reliability is the probability that the failure mode will not occur. The
expression for reliability is:
R = 1 – P
F
.
The expression for P
F
, is dependent upon the type of load and capability distributions.
Expressions for P
F
for various distributions are found in most advanced statistics text
books and handbooks. Expressions for P
F
between combinations of exponential, log
normal, gamma, and Weibull distributions are found in reference 3.8.
(7) Assess the reliability for each critical failure mode, including load and capability in this
assessment, then modify the design to increase reliability. Repeat the process until the
design reliability goals or requirements are met.
(8) Perform trade studies (sec. 2.1) to reassess and optimize the design for performance, cost,
environmental issues, maintainability, etc.
(9) Repeat step 8 for each critical component for the system.
(10) Determine the relative reliability of the system.
(11) Repeat the above steps to optimize system reliability.
381
f1(X1)
X1
f2(X2)
X2
fn(Xn)
Xn
g1(Y1)
Y1
g2(Y2)
Y2
gm(Ym)
Ym
fL(L)
L
Load Transfer Function
L = fL(X1, X2, ....Xn)
gC(C)
C
Capability Transfer Function
C = g C(Y1, Y2, ....Ym)
Figure 342. Load and capability transfer functions.
3.8
382
Probability
Density
Function
Load
f (L)
L
Capability
fc (C)
f (L)
L
f (C), C
L C L , C
Overlap Indicative of Failure
Figure 343. Interference between load and capability density functions.
3.8
3.14.4 Advantages
A PDA provides the following advantages:
(1) Allows the analyst a practical method of quantitatively and statistically analyzing the
relative reliability of a system during the design phase.
3.8
Therefore PDA’s can be used to
determine valuable areas of the design and aid in determining the resource allocation during
the test and evaluation phase.
(2) This technique mandates that the analyst address and quantify the uncertainty of design
variables and understand its impact on system reliability of the design.
3.8
(3) The PDA approach offers a more accurate and truly quantitative alternative method to the
more traditional approach of using safety factors and margins to ensure component
reliability.
3.8 3.18
(4) The technique provides a more precise method for determining failure probabilities to
support FTA’s than does use of subjective methods.
3.14.5 Limitations
A PDA possesses the following limitations:
(1) The analyst must have experience in probability and statistical methods to apply this
technique.
3.8
(2) Determining the density functions of the random variables in the load and capability
transfer functions may be difficult.
3.18
(3) Historical population data used must be very close to the asplanned design population to
be viable. Extrapolation between populations can render the technique nonviable.
383
(4) This technique identifies the relative probabilities that various failure modes will occur, but
does not address the severity of the failure modes. Therefore, this technique should be used
as one element among other elements of a PRA (sec. 3.15) to assess the risk associated with
the various failure modes.
3.15 Probabilistic Risk Assessment
3.15.1 Description
A PRA is a general term given to methodologies that assess risk. Although PRA methods are
customarily thought of as being quantitative, these methods can be either subjective (as by use of the risk
assessment matrix, sec. 3.1), or quantitative in nature.
According to reference 3.6, a PRA generally consists of three phases. During phase 1, the system
is defined, hazards are identified, elements of the system vulnerable to hazards are identified, and the
overall scope of types of hazards to be assessed is defined. PHA’s (sec. 3.2), are typically performed
during phase 1.
During phase 2, the failure propagation paths and probabilities are established. ETA (sec. 3.8),
FTA (sec. 3.6), FMECA (sec. 3.4) and/or causeconsequence analysis (sec. 3.10) are performed.
Finally, during phase 3, a consequence analysis is performed. Severity is established. Then, an
assessment of risk is performed in terms of probability and severity, and by comparison to other societal
risks.
3.15.2 Application
A PRA is performed to identify consequence of failure in terms of potential injury to people,
damage to equipment or facilities, or loss of mission requirements. The PRA is typically performed in
phase C.
3.15.3 Procedures
The following procedures, adopted from reference 3.6, offer guidance in performing a
probabilistic risk assessment:.
(1) Phase 1 (activities performed during the preliminary design stage).
a. Define the system to be assessed, identify the elements (targets) of the systems that are
susceptible to hazards, and from an overall perspective identify potential hazards.
b. Perform a PHA (sec. 3.2). In performing a PHA, the analyst: (1) identifies targets, (2)
defines the scope of the system, (3) recognizes the acceptable risk limits, (4) identifies
hazards, (5) assesses the risk for each hazard and target combination in terms of proba
bility and severity, (6) and if the risk are unacceptable determines countermeasures to
mitigate the risk, and (7) and repeats the assessment with the countermeasures
incorporated.
384
(2) Phase 2 (activities initiated after accomplishing hardware and configuration selections).
a. Identify failure propagation paths with techniques such as an ETA (sec. 3.8). In
performing an ETA, the analyst (1) identifies an initiating challenge to the system, and
(2) determines the alternate logic paths from the initiating event.
b. Determine initiators and propagate probability of failure with methods such as FTA
(sec. 3.6). Probability of failure modes can also be determined with the probabilistic
analysis method presented in section 3.14.
c. A causeconsequence analysis (sec. 3.10) may be performed to establish both failure
propagation path and probabilities of causes and consequences.
d. A digraphmatrix analysis (sec. 3.11) may be performed after the ETA is complete and
before FTA’s have begun.
3.15
e. An FMECA (sec. 3.4) may be performed. Examine all failure modes and criticality
ranking of each system element.
(3) Phase 3 (perform a consequence analysis).
a. Establish the severity of the failure modes.
b. Assess risk of all failure modes in terms of severity and probability.
c. Calibrate the risk of the system being examined by comparing it to other known societal
risks.
3.15.4 Advantages
Assessing risk avoids unknowingly accepting intolerable and senseless risk, allows operating
decisions to be made, and improves resource distribution for control of loss resources.
3.1
3.15.5 Limitations
A PRA possesses the following limitations:
(1) Probabilistic risk assessment requires skilled analysts. If the analyst is untrained in the
various tools required, the tool could be misapplied or the results misinterpreted.
(2) Depending on the size and complexity of the system being assessed, significant manhour
and/or computer resources may be needed to complete.
(3) Sufficient information and data may not be available to perform a thorough assessment.
385
REFERENCES
3.1 Clemens, P.L.: “Working with the Risk Assessment Matrix.” Second edition, Lecture
Presentation, Sverdrup Technology, Inc., June 1993.
3.2 “System Safety Program Requirements.” MIL–STD–882C, January 1993.
3.3 Mohr, R.R.: “Preliminary Hazard Analysis.” Fourth edition, Lecture presentation, Sverdrup
Technology, Inc., July 1993.
3.4 Clemens, P.L.: “Energy Flow/Barrier Analysis.” Third edition, Lecture presentation, Sverdrup
Technology, Inc., June 1993.
3.5 Mohr, R.R.: “Failure Modes and Effects Analysis.” Sixth edition, Lecture presentation, Sverdrup
Technology, Inc., October 1992.
3.6 Henley, E.J., and Kumamoto, H.: “Probabilistic Risk Assessment.” The Institute of Electrical and
Electronic Engineers, Inc., NY, 1991.
3.7 Gough, W.S., Riley, J., and Koren, James M.: “A New Approach to the Analysis of Reliability
Block Diagrams.” Proceedings from Annual Reliability and Maintainability Symposium, SAIC,
Los Altos, NM, 1990.
3.8 Kampur, K.C., and Lamberson: “Reliability in Engineering Design.” John Wiley & Sons, NY,
1977.
3.9 Clemens, P.L.: “Fault Tree Analysis.” Fourth edition, Lecture presentation, Sverdrup Technology,
Inc., May 1993.
3.10 Swain, A.D., and Guttman, H.E.: “Handbook of Human Reliability Analysis with Emphasis on
Nuclear Power Plant Applications.” NUREG/CR–1278.
3.11 Briscoe, Glen J.: “Risk Management Guide.” System Safety Development Center, SSDC–11,
DOE 7645/11, September 1982.
3.12 Clemens, P.L.: “Event Tree Analysis.” Second edition, Lecture presentation, Sverdrup
Technology, Inc., June 1990.
3.13 Clemens, P.L.: “Transformations, Fault Tree/Reliability Block Diagram/Event Tree.” Lecture
presentation, Sverdrup Technology, Inc., November 1992.
3.14 Clemens, P.L.: “CauseConsequence Analysis.” Third edition, Lecture presentation, Sverdrup
Technology, Inc., December 1992.
3.15 Alesso, H.P., Sacks, I.J., and Smith, C.F.: “Initial Guidance on DigraphMatrix Analysis for
System Interaction Studies.” Lawrence Livermore National Laboratory, March 1983.
3.16 Clemens, P.L.: “Combinatorial Failure Probability Analysis Using MIL–STD–882B,” Fourth
edition, Lecture presentation, Sverdrup Technology, Inc., August 1991.
3.17 Clemens, P.L.: “Failure Information Propagation Modeling.” Second edition, Lecture
presentation, Sverdrup Technology, Inc., October 1989.
3.18 “Solid Propulsion Reliability Guidebook.” The Phillips Laboratory and the Engineering Society
for Advancing Mobility Land, Sea, Air, and Space (SAE), vol. 2, draft, June 1992.
386
4. DESIGNRELATED ANALYTICAL TOOLS
Two designrelated analytical tools (sensitivity analysis and tolerance stackup analysis) that can
be useful to systems engineering are discussed in this section. In addition, Geometric Dimensioning and
Tolerancing, ANSI–Y–14.5, is discussed. This section is included to give the systems engineer an under
standing of the standard methods of dimensioning and tolerancing.
A summary of the advantages and limitations of each tool or methodology discussed in this
section is presented in table 41.
4.1 Sensitivity (Parametric) Analysis
4.1.1 Description
In sensitivity analysis, sensitivity functions (or coefficients of influence) are generated by taking
partial derivatives with respect to each parameter that affects the outcome of a relationship.
4.1.2 Application
Sensitivity analysis typically should be performed in phase C or D. This analysis can be used for
nearly any type of relationship. Sensitivity analysis is especially useful when environmental conditions
can change, when factors such as age affect performance, or when manufacturing tolerances affect
performance. Sensitivity analysis can show which parameters affect a system the most or least. This can
facilitate optimizing a system, reducing variability, or adjusting a system for wear or changing
conditions. Typical examples of the use of sensitivity analysis are manufacturing formulation and
processes (e.g., bond strength, burn rate, erosion rate, or material strength).
4.1.3 Procedures
The procedure for obtaining the sensitivity of a relationship by analytical methods is as follows:
(1) Generate an equation for the relationship under consideration.
(2) Find the coefficients of influence
4.1
by taking the partial derivatives for each parameter
under consideration.
(3) Solve the equations for the coefficients of influence to find the sensitivity at given
conditions.
An alternate approach to approximate sensitivity is to assume a straightline relationship between
two points in the sample space of the relationship, and to solve the relationship for two conditions repre
sented by two values of the parameters in question. This method is often preferred for relationships with
parameters that are interrelated, such as throat area and exit pressure in the thrust equation.
41
42
Table 41. Designrelated analytical tools and methodologies.
Tool or Methodology Section Advantages Limitations
Sensitivity (parametric) analysis 4.1 The effect of each parameter can be assessed to determine
which parameters have the greatest effect on the
outcome of a process and which parameters can yield the
most benefit from adjustment.
It is often not easy to isolate a variable to obtain a
second derivative. For example, when obtaining the
sensitivity of thrust to throat diameter, changing a
throat diameter not only changes motor pressure, but
changes the nozzle expansion ratio and exit pressure.
The pressure ratio is typically found by iteration or by
tables. If the approximation approach above is taken,
care must be used to ensure a small enough range for
parameter values to achieve the desired accuracy.
Standard dimensioning and
tolerancing
4.2 Dimensioning and tolerancing per ANSI–Y–14.5 is
fairly standard. In addition, some aspects of dimen
sioning and tolerancing per ANSI–Y–14.5 are better
suited for production. For example, true positioning
allows for a circular tolerance zone, whereas putting
tolerances to rectangular coordinates allows a square
tolerance zone. Thus, a functional part that would
comply with true position tolerances may not comply
with rectangular tolerances. Dimensioning strategy can
minimize the cumulative tolerance stackup. This is
facilitated by following the dimensioning and
tolerancing system of ANSI–Y–14.5.
A moderate amount of training and practice is required to
effectively use standard dimensioning and tolerancing.
Tolerance stackup analysis 4.3 Worstcase tolerance analysis can simply determine the
envelope of possible form, fit, and function. Statistical
analysis can show that, even if exceeding a requirement
is possible, it may be extremely unlikely.
Worstcase tolerance analysis is conservative in that
when many tolerances combine, it becomes increasingly
unlikely that all dimensions will be simultaneously
worstcase. Statistical tolerance analysis usually
assumes a normal distribution of dimensions in the
tolerance zone, which may be unrealistic. In addition,
care must be exercised when combining tolerances, in
that:
(1) If some tolerances are much smaller than others,
their inclusion in tolerance stackup analysis is
superfluous. Consideration of significant digits
may be helpful, e.g., a 0.030 tolerance may have
a smallest unit of measurement greater than a
0.0005 tolerance.
(2) It may be superfluous to combine tolerances from
different manufacturing processes, e.g.,
machining and casting.
4.1.4 Example
In the following hypothetical example, the sensitivity of pressure with respect to throat area is
being determined. The equation for this analysis is the pressure equation. The equation for pressure is:
P
c
= r
b
C
*
g
A
s
A
*
(4.1)
where P
c
is the chamber pressure, r
b
is the propellant burn rate, C
*
is the propellant gas characteristic
exhaust velocity, is the propellant density, g is gravity, A
s
is the propellant burn surface area, and A
*
is
the throat area. To find the sensitivity of pressure to motor throat area, take the partial derivative of
equation (4.1) with respect to A
*
. P
c
is taken over a narrow range where r
b
is approximately constant.
P
c
A
*
= – r
b
C
*
g
A
s
¸
1
]
1
1
A
*
( )
2
(4.2)
where ∂ designates a partial derivative. The sensitivity is found by substituting values for the variability
into the partial derivative equation. Numbers can be substituted into equation (4.2) to obtain the slope at
a particular value of A
*
. It is intuitively obvious that the relationship between the partial derivative and
A
*
is both negative and inversely proportional to A
*2
.
Another example of the approximation method is the substitution of selected values into the
thrust equation (4.6). The sensitivity of thrust to throat area is to be investigated for a hypothetical motor
with the following characteristics:
A
s
= 300 in
2
A
*
= 1.9 in
2
,
2.1 in
2
A
e
= 10 in
2
= 0.06 lbm/in
3
γ = 1.2
r
b
= 0.5 in/s
C
*
= 5100 in/s
g = 386.40 in/s
2
.
The first step is to calculate the chamber pressure, substituting into equation (4.1), using the first
value of A
*
which is 1.9 in
2
. The next step is to calculate Mach number (M) iteratively from equation (4.3):
A
e
A
*
¸
¸
_
,
2
·
1
M
2
2
+1
1+
−1
2
M
2 ¸
¸
_
,
¸
1
]
1
+1
−1
.
(4.3)
43
The third step is to calculate nozzle exit plane pressure (P
e
) from equation (4.4).
P
e
P
c
·
1
1+
−1
2
M
2
¸
1
]
1
−1
(4.4)
The next step is to calculate the thrust coefficient (C
F)
from equation (4.5).
C
F
·
2
2
−1
2
+1
¸
¸
_
,
+1
−1
¸
1
]
1
1
1
1−
P
e
P
c
¸
¸
_
,
−1
¸
1
]
1
1
1
¹
'
¹
¹
¹
¹
;
¹
¹
¹
1
2
(4.5)
The final step is to calculate thrust (T) from equation (4.6).
T = C
F
A
*
P
c
. (4.6)
The above calculations should be performed again, using A
*
= 2.1 in
2
. The values obtained from
both calculations are shown in table 42.
Table 42. Sensitivity analysis calculations.
P
c
M P
e
C
F
T
A
*
= 1.9 in
2
62.52 2.82 1.87 1.50 177.62
A
*
= 2.1 in
2
56.57 2.75 1.93 1.47 174.60
Conclusion:
The thrust (T) is inversely proportional to the throat area (A
*
).
4.1.5 Advantages
The effect of each parameter can be assessed to determine which parameters have the greatest
effect on the outcome of a process, and which parameters can yield the most benefit for adjustment.
4.1.6 Limitations
It is often not easy to isolate a variable to obtain a second derivative. For example, when
obtaining the sensitivity of thrust to throat diameter, changing a throat diameter not only changes motor
pressure, but changes the nozzle expansion ratio and exit pressure. The pressure ratio is typically found
by iteration or by tables. If the approximation approach above is taken, care must be used to ensure a
small enough range for the parameter values to achieve the desired accuracy.
44
4.2 Standard Dimensioning and Tolerancing
4.2.1 Description
Dimensioning and tolerancing on drawings is complicated enough to yield confusion, unless
standardized methods are employed at all stages of a project life from design to manufacture. Standard
dimensioning and tolerancing per ANSI–Y–14.5 is an internationally recognized method of stating
dimensions and tolerances.
4.2.2 Application
Standard dimensioning and tolerancing is typically applied in phase C but the technique could
also be applied in phase D. Standard dimensioning and tolerancing allows the design engineer to
indicate how tolerances are to be applied. This information is understood by draftsmen, manufacturing
engineers, and machinists to assure the form, fit, and function intended by the design engineer (or
systems engineer). Some of the methods of specifying dimensions and tolerances are discussed here.
4.2.3 Procedures
This section explains how dimensions and tolerances are specified on design drawings.
4.3
Following is a list of feature controls used to specify how a tolerance is to be applied to a design feature,
from ANSI–Y–14.5:
st r aight ness
f l at ness
ci r cul ar i t y
cy l i ndr i ci t y
pr of i l e of a l i ne
pr of ile of a sur f ace
angul ar i t y
per pendi cul ar i t y
par al l el i sm
posi t i on
concent r i ci t y
ci r cul ar r unout
t ot al runout
 B
dat um ident if ying
l et t er
XXX
basic dimension (XXX) ref erence dimension
A basic dimension is contained in a box (unless otherwise specified on the drawing). Basic
dimensions are the controlling dimensions on a drawing, and have no tolerances associated with them.
Basic dimensions set up a dimensional pattern, such as a bolt pattern. The locations of the features in the
pattern (e.g., bolt holes or threads) are toleranced using trueposition tolerances. Often the title block of a
drawing will indicate standard tolerances peculiar to that drawing that will apply to all basic dimensions
shown without a specified tolerance. A tolerance is shown for each significant digit used in the basic
dimensions on the drawing. For example, a tolerance of ± 0.1 may apply to all basic dimensions with
one significant digit.
45
Reference dimensions are the result of basic dimensions. In the example below, an inner and
coincident outer diameter are specified; the thickness is a reference dimension. In this situation, the inner
and outer diameters are of primary importance; the thickness is of secondary importance.
A rectangular box is used as a feature control box. The symbol in the first section of the box is
for the type of tolerance (e.g., true position). The first symbol in the second section is the type of
measurement (a diametrical tolerance is shown in fig. 41). The number is the size of the tolerance. The
second symbol in the second section (a circle with the letter “M,” “L,” or “R”) specifies the relation of
the tolerance with the size of the feature. The third (and any subsequent) section specifies which data are
used (which feature or dimension the tolerances concern).
4.2.4 Example
Following is a hypothetical fixed nozzle assembly used to show the purpose of dimension and
tolerance methods:
3.000 ±.020
1.000 ±0.003
4.00
 A 
0.030 A R
0.020 A R
 B
0.010 A B
1.25 X 12 UNF
0.005 A B R
Figure 41. Example of dimensioning and tolerancing.
In this example, datum A is defined by the throat of the nozzle, thus datum A is the axis of the
throat. The nozzle exit is referenced to datum A. The true position of the exit is to be within ±0.030 of the
throat axis (datum A), and the exit plane is to be within a 0.020 tolerance zone perpendicular to the throat
axis. The true position tolerance is not affected by the feature size of the throat diameter. (The “R” inside
the circle indicates that the position tolerance is applied “regardless of feature size.” An “M” inside the
circle would denote that the position tolerance applies to “maximum material condition;” thus the
tolerance can be relaxed by an amount commensurate with the difference that the size of the feature is
less than the maximum allowable size. An “L” inside the circle would denote “least material condition”
where the tolerance applies to the smallest feature size allowable.) The exit plane also defines datum B.
The boss at the end of the nozzle is controlled by a total runout tolerance. The surface is to be
within a 0.010 tolerance zone perpendicular to the axis made by the throat and exit (datums A and B).
The threads of the nozzle are to be concentric to the throat and exit axis within 0.005, and the axis of the
threads is to be within ±0.015 of the throat axis. Note that for the profile type tolerance controls (e.g.,
46
runout or perpendicularity), the number defines a tolerance zone. This means that the total “width” of the
acceptable deviation is defined by the tolerance. Thus a tolerance zone of 0.020 is analogous to a ±0.010
tolerance. For position tolerances, the number call out is ±, thus the axis of the nozzle exit must fall
inside a circle of 0.030 radius around the throat axis.
Note that the tolerances in this example control the thrust vector. The length of the nozzle is con
trolled by a basic dimension. The exit is truepositioned to the basic dimension from the nozzle throat,
and the required perpendicularity to the throat axis is greater than the true position tolerance. The nozzle
exit is toleranced to keep the thrust vector in line (within a certain amount) with the throat axis. The
nozzle boss is controlled by runout to the axis defined by the throat and exit plane. The boss surface
tolerance is to facilitate a consistent seal with the motor. The thread is controlled by concentricity to the
same axis to keep the thrust axis in line with the motor axis. It can be seen that the thickness of the boss
is not a controlling dimension; it is a reference dimension. If this dimension were not specified, the form,
fit, or function of the component would not be affected.
4.2.5 Advantages
Dimensioning and tolerancing per ANSI–Y–14.5 is fairly standard. In addition, some aspects of
dimensioning and tolerancing per ANSI–Y–14.5 are better suited for production. For example, true posi
tioning allows for a circular tolerance zone, whereas putting tolerances to rectangular coordinates allows
a square tolerance zone. Thus, a functional part that would comply with true position tolerances may not
comply with rectangular tolerances. Dimensioning strategy can minimize the cumulative tolerance
stackup. This is facilitated by following the dimensioning and tolerancing system of ANSI–Y–14.5.
4.2.6 Limitations
A moderate amount of training and practice is required to effectively use standard dimensioning
and tolerancing.
4.3 Tolerance Stackup Analysis
4.3.1 Description
Tolerance stackup analysis determines if a form, fit, or function problem exists when
manufacturing tolerances combine in a finished part or assembly. Tolerance stackup analysis is typically
performed by either assuming worstcase allowable dimensions, or by using statistical analysis of
tolerances.
4.3.2 Application
Tolerance stackup analysis is typically performed in phase C or D. This technique is used to
determine the possibility or probability of having form, fit, or function problems with a design, or to
determine a tolerance or dimension necessary to avoid form, fit, or function problems.
4.3.3 Procedures
Three typical methods for tolerance stackup analysis are:
(1) Worstcase tolerance stackup analysis, used to determine size or position if all applicable
dimensions occur at the worstcase extremes of the tolerance zones simultaneously.
47
(2) Statistical analysis of tolerances, used where the expected standard deviations of tolerances
are combined to determine the probability of a final tolerance.
4.4
(3) Design using simulation methods, where a computer is used to do a Monte Carlo analysis
of the possible combinations of tolerances.
4.5
4.3.4 Example
In the following hypothetical Oring joint assembly (fig. 42), the tolerances of each component
are shown in figure 43. Find the maximum tolerance stackup possible to obtain the minimum squeeze,
and the probability that the squeeze will be less than 0.035. The nominal squeeze is 0.050 inches.
Figure 42. Oring joint.
±0.010
±0.010
±0.005
Figure 43. Oring joint components.
The probability of the squeeze being less than 0.035 is obtained by finding the distance from the
mean (in terms of standard deviations) that this condition represents. The standard deviation is assumed
to be one third of the tolerance on the parts (this means all parts will fall within 3 standard deviations of
the nominal dimension) and is therefore:
Component standard deviation = 0.010/3 = 0.0033
Oring standard deviation = 0.005/3 = 0.00167
and by summation of squares,
system standard deviation = (2(0.0033)
2
+ (0.00167)
2
)
.5
= 0.005.
For a squeeze of 0.035, the distance (in standard deviations) from the mean (z) is
z = (0.035–0.050)/0.005 = –3.0.
48
Using a table for the normal distribution function, the area under the half curve for z = ±3 is
0.4987. Since this is a onesided question (no interest in the squeeze being 0.065), the area under the
curve beyond z = 3 is (0.5–0.4987) = 0.0013. This value is interpreted as 0.13 percent probability that
the squeeze on the Oring will be 0.035 or less.
A close look at the example above will show that more sources of variation are possible than
those considered. For example, the surfaces compressing the Oring may not be flat or normal to the
direction of squeeze. Also, position tolerances are often determined at maximum material condition, thus
position can vary more when not at maximum material condition. It can be extremely cumbersome to
perform a statistical analysis of all the possible variations on some assemblies, so software exists to
perform the statistical analysis. A typical example of software is the “Variation Simulation Analysis”
4.5
that uses Monte Carlo methods to simulate the possible ways that the tolerances can stack up.
4.3 4.4
The
results can be used to determine probabilities that certain overall tolerances will exceed a critical value,
or which tolerances are most important to form, fit, or function.
4.3.5 Advantages
Worstcase tolerance analysis can simply determine the envelope of possible form, fit, and func
tion. Statistical analysis can show that, even if exceeding a requirement is possible, it may be extremely
unlikely.
4.3.6 Limitations
Worstcase tolerance analysis is conservative, in that when many tolerances combine, it becomes
increasingly unlikely that all dimensions will be worstcase simultaneously. Statistical tolerance analysis
usually assumes a normal distribution of dimensions in the tolerance zone, which may be unrealistic. In
addition, care must be exercised when combining tolerances, in that:
(1) If some tolerances are much smaller than others, their inclusion in tolerance stackup
analysis is superfluous. Consideration of significant digits may be helpful, e.g., a 0.030
tolerance may have a smallest unit of measurement greater than a 0.0005 tolerance.
(2) It may be superfluous to combine tolerances from different manufacturing processes, e.g.,
machining and casting.
4.3.7 Bibliography
Craig, M.: “Managing Variation by Design Using Simulation Methods.” Applied Computer Solutions,
Inc.
49
REFERENCES
4.1 “System Sensitivity Analysis.” Edited by J.B. Cruz, Jr., Dowden, Hutchinson and Rose, Inc.
Stroudsburg, PA.
4.2 Hill, P.G., and Peterson, C.R.: “Mechanics and Dynamics of Propulsion.” Third edition,
AddisonWesley Publishing Company, Reading, MA, November 1970.
4.3 “Dimensioning and Tolerancing.” ANSI–Y–14.5M, The American Society of Mechanical
Engineers, United Engineering Center, 345 East 47th Street, New York, NY 10017, 1982.
4.4 Shigley, J.E.: “Mechanical Engineering Design.” Third edition, McGrawHill.
4.5 “Variation Simulation Analysis Software, Getting Started Manual.” Applied Computer Solutions
Inc., 300 Maple Park Blvd., St. Clair Shores, MI 48081.
410
5. GRAPHICAL DATA INTERPRETATION TOOLS
There are numerous excellent texts on the appropriate use of graphical data interpretation tools.
While this section lists and briefly discusses some of the available tools, the neophyte reader is advised
to read and utilize standard handbook references when using these techniques in problem solving to
avoid misuse and error. This toolbox is to provide knowledge of the existence of these techniques, and
references for their appropriate application.
One way to analyze data is by graphical interpretation. The analysis can be used to monitor
performance, identify relationships, and reveal the most important variables in a set of data. The scatter
diagram, section 5.1, makes it possible to determine if any relationship exists between two variables.
The control chart, section 5.2, monitors the performance of a process with frequent outputs. Control
charts are useful in trend analysis, section 8, and statistical process control, section 7.14. The bar chart
compares quantities of data to help identify distribution patterns. This chart is discussed in section 5.3.
One of the most common data displays is the timeline chart, section 5.4. This chart displays
changes over time. Sorting data that share a common characteristic into different groups is often
accomplished with a stratification chart. This chart is discussed in section 5.5. A Pareto chart, section
5.6, is used typically when there is a need to know the relative importance of data or variables. This
chart will also identify the problems, causes, or conditions that occur most frequently. A histogram,
section 5.7, is a bar chart that shows a dispersion of data over a specified range. This type of chart is
commonly used in presentations to make data easier to interpret.
A summary of the advantages and limitations of each tool or methodology discussed in this
section is presented in table 51.
5.1 Scatter Diagram
5.1.1 Description
Scatter diagrams, also called XY graphs, plot raw data and allow the analyst to determine if any
relationship exists between two variables. No interpretation of the data should be attempted, but
correlations can be inferred.
5.1
5.1.2 Application
The graphic display of the scatter diagram can help one determine possible causes of problems,
even when the connection between two variables is unexpected. The direction and compactness of the
cluster of points gives a clue as to the strength of the relationship between the variables. The more that
this cluster resembles a straight line, the stronger the correlation between the variables. The scatter
diagram technique is best applied in phase E.
The scatter diagram displays one variable on the horizontal (X) axis and the other variable on the
vertical (Y) axis. If there is a correlation between the two variables, positive or negative, it can be
assumed if the data from one are changed, then this will effect the data from the other.
5.2
51
52
Table 51. Graphical data interpretation tools and methodologies.
Tool or Methodology Section Advantages Limitations
Scatter diagram 5.1 (1) The general relationship between two variables can
be determined at a glance.
(2) The graph can help determine a possible cause(s) of
problems by looking at correlations.
(1) The choice of scale for the graph can distort the
data, thus possibly giving the appearance of a
correlation that is better or worse than reality.
(2) The correlation does not prove a causeandeffect
relationship.
Control chart 5.2 (1) The control chart helps one understand the
capabilities of the process.
(2) The chart can prevent tampering with processes that
are under statistical control.
(3) The chart monitors the effects of process changes
that are aimed at improvement.
(4) Control charts can be used without extensive
knowledge of statistics.
(1) The control chart tells only if the process is in
control.
(2) The underlying causes are not determined.
Bar chart 5.3 (1) The bar chart tells its story at a glance.
(2) It makes graphic comparisons of quantity easy to
see.
A bar chart is limited in the number of data categories
that can be displayed at one time.
Timeline chart 5.4 (1) The timeline chart shows a “moving picture” of
fluctuations over time.
(2) Defect rates can be plotted on time lines in order to
identify trends.
The timeline chart shows the direction of change but it
gives no indication as to the reason for the change.
Stratification chart 5.5 The approach not only produces a priority ordering of
the problems but also identifies an improvement
strategy.
(1) The correct stratification variables for resolving a
problem are generally not known prior to data
collection.
(2) All potentially important stratification variables
cannot be determined without planning.
Pareto chart 5.6 (1) The pareto chart helps to identify the few areas of
concern that are most important.
(2) The chart is useful in analyzing defect data.
A poor pareto chart will result if the causes chosen to
study are wrong. Some preplanning needs to be done
before choosing categories.
Histograms 5.7 (1) A histogram helps identify changes in a process as
the data changes.
(2) A histogram helps establish standards for a process.
A histogram is not a good tool for computing process
capability.
5.1.3 Procedures
As described in reference 5.2, a scatter diagram is prepared in the following manner:
(1) Collect the two selected variables of each occurrence.
(2) Draw the horizontal and vertical scales with equal length.
(3) The dependent variable, the one that you can have an effect on, is assigned to the vertical
(Y) axis. The independent variable is assigned to the horizontal (X) axis. Set the scale
intervals and label.
(4) Plot each data point.
(5) A possible relationship can be determined by visual inspection of the graph.
5.1.4 Example
As adapted from reference 5.3, an aptitude test was given to 10 employees and the scores were
then compared to the production levels of these employees over a certain time period. The scatter
diagram, example shown in figure 51, would show if there were any relationship between the test scores
and the production levels.
Employee Test Score Production Level
1 27 120
2 13 80
3 8 60
4 37 150
5 32 135
6 10 70
7 17 95
8 22 105
9 6 50
10 7 55
This plot shows that the higher test scores result in higher production levels.
5.1.5 Advantages
(1) The general relationship between two variables can be determined at a glance.
(2) The graph can help determine a possible cause of problems by looking at correlations.
5.1.6 Limitations
(1) The choice of scale for the graph can distort the data, thus possibly giving the appearance
of a correlation that is better or worse than reality.
(2) The correlation does not prove a causeandeffect relationship.
53
•
•
•
•
•
•
•
•
•
•
0
20
40
60
80
100
120
140
160
0 5 10 15 20 25 30 35 40
P
r
o
d
u
c
t
i
o
n
L
e
v
e
l
Test Score
Figure 51. Scatter diagram example.
5.2 Control Chart
5.2.1 Description
A control chart monitors the performance of a process with frequent outputs. The chart shows a
pictorial representation of an ongoing process and determines whether or not the process is performing
within acceptable parameters. The control chart is based on four concepts:
(1) All processes change with time.
(2) Individual points of the process are unpredictable.
(3) A stable process changes randomly, and groups of points from a stable process tend to fall
within predictable bounds.
(4) An unstable process does not change randomly, and when changes occur they are generally
out of the range of normal operations.
5.2
5.2.2 Application
The control chart technique is best performed in phase E. As described in reference 5.2, control
charts are used to show the variation of several variables including average ( X ) and range (R) as well as
the number of defects (PN), percent defective (P), defects per variable unit (U), and defects per fixed
unit (C).
The upper control limits (UCL) and lower control limits (LCL) should not be confused with
specification limits. The control limits show the natural change of a process, such that points within the
limits generally indicate normal and expected change. Points that are outside of the control limits reveal
that something has occurred that requires special attention because the points are outside of the builtin
54
systematic cause of change in the process. One point that is outside of the control limits does not mean
the process is out of control but it should be explained.
The control chart can to be used continuously to determine whether the process remains within
established control limits. As new points are added, the chart can be monitored for points that may fall
outside of the limits and require causes to be identified.
Control charts are used in performing statistical process control (SPC) (sec. 7.14) and trend
analysis (sec. 8.).
5.2.3 Procedures
As described in reference 5.2, a control chart (fig. 52) is constructed in the following manner:
(1) Determine the control limits to show the expected change of the process.
(2) Gather data.
(3) Plot the data on the control chart to evaluate performance and identify the points outside of
the control limits.
(4) Determine why points are outside of the control limits.
(5) Find ways to identify causes of problem points, reduce the normal variation, and improve
the mean.
5.2.4 Example
0 5 10 15 20 25 30
0
0.1
0.2
0.3
0.4
0.5
0.6
Event
0
0.1
0.25
0.076
0.083
0.125
0.25
0.333
0.286
0.333
0.5
0.182
0.076
0.1
0.091
0.5
0.2
0.25
0.5
0.2
0 0
0.091
0.167
0 0 0
0.111
0
0.182
0
UCL = 0.55
LCL=0.15
Figure 52. Control chart example.
55
5.2.5 Advantages
(1) The control chart helps the analyst understand the capabilities of the process.
(2) The control chart can prevent tampering with processes that are under statistical control.
(3) The control chart monitors the effects of process changes that are aimed at improvement.
(4) Control charts can be used without extensive knowledge of statistics.
5.2.6 Limitations
(1) The control chart tells only if the process is in control.
(2) The control chart does not indicate the underlying cause unless data on outside processes
are included in the analysis.
5.3 Bar Chart
5.3.1 Description
Bar charts show a comparison of quantities of data to help identify quantity changes. The
quantities of data are depicted by the lengths of the bars that represent cost, percentage, or frequency of
events. The bars may be horizontal or vertical.
5.2
5.3.2 Application
Bar charts are one of the most common types of data display and this technique is typically
performed in phase E. Differences and similarities between and among selected categories are
emphasized by the heights of the columns. Bar charts can show double and triple bars to compare
different time periods or different populations.
5.3.3 Procedures
As described in reference 5.2, a bar chart (fig. 53) is constructed in the following manner:
(1) If necessary, raw data are entered on a checklist (sec. 7.8).
(2) List the categories across the horizontal scale at the bottom.
(3) Label the quantities on the vertical scale at the left. Make sure the scale is broad enough to
include the highest and lowest value in each category.
(4) Draw the bar according to the quantity of each category.
(5) Give the bar chart a legend to identify different colors or patterns.
56
5.3.4 Example
Sale of Household Appliances
1980 versus 1990
(in millions)
0
1
2
3
4
5
6
7
8
9
10
(Nominal Categories)
1980
1990
Figure 53. Bar chart example.
5.3.5 Advantages
(1) The bar chart tells its story at a glance.
(2) The bar chart makes graphic comparisons of quantity easy to see.
5.3.6 Limitations
A bar chart is somewhat limited in the number of data categories that can be displayed at one
time.
57
5.4 TimeLine Chart
5.4.1 Description
The timeline chart is among the most common types of data displays. The chart graphically
displays changes over a period of time.
5.4.2 Application
The timeline chart is a special case of XY plots where the independent variable is some time
value. The chart connects data points with line segments. The line segments connecting the points on the
chart give a clear picture of changes over time. The vertical scale is a quantity while the horizontal scale
is divided into time intervals such as “hours,” “days,” and “weeks.”
5.2
This technique is best performed
in phase E.
5.4.3 Procedures
As described in reference 5.2, a timeline chart (fig. 54) is prepared in the following manner:
(1) Enter the raw data on a checklist (sec. 7.8).
(2) Establish time intervals (usually hours, days, weeks, etc.) for the horizontal axis. The
intervals should be evenly spaced and labeled.
(3) Establish the quantities for the vertical axis and make them evenly spaced (e.g., 10, 20, 30,
etc.) and label the axis.
(4) Connect, with line segments, the quantities plotted for each successive interval.
(5) If the points are difficult to read, add horizontal and vertical grids.
(6) Title the chart to define the time period for which the data are displayed.
5.4.4 Example
A study was made comparing the average number of errors that were made per operator at
different times of the day over a certain time period (fig. 54).
5.4.5 Advantages
(1) The timeline shows a “moving picture” of fluctuations over time.
(2) Defect rates can be plotted on time lines in order to identify trends.
58
l
l
l
l
l
l
l
l l
0
9
:
0
0
:
0
0
A
M
1
0
:
0
0
:
0
0
A
M
1
1
:
0
0
:
0
0
A
M
1
2
N
o
o
n
0
1
:
0
0
:
0
0
P
M
0
2
:
0
0
:
0
0
P
M
0
3
:
0
0
:
0
0
P
M
0
4
:
0
0
:
0
0
P
M
0
5
:
0
0
:
0
0
P
M
0
1
2
3
4
5
6
Q
u
a
n
t
i
t
y
Time Intervals
l
No. of Errors
Figure 54. Timeline chart example.
5.4.6 Limitations
The timeline chart shows the direction of change but it gives no indication as to the reason for the
change.
5.5 Stratification Chart
5.5.1 Description
The term “stratification,” derived from “stratum,” is used in data analysis. Stratification is done by
sorting data into different groups that share a common characteristic. Some common stratification
variables are shift, operator, and machine.
5.5.2 Application
The stratification chart is best applied in phase E. Comparisons of different groups, units, or other types
of strata can often lead to suggesting an improvement strategy. For example, a process is incurring a 10
percent defect rate with a particular product. You can stratify by vendor, lot, operator, shift, time,
machine, etc. and compute a percent defective for each category (stratification variable).
The data can be depicted in graphic form for easy visual interpretation. Should the data not include a
significant problem, select other stratification variables and collect more data. The graph may show that
one category is producing a higher defect rate than others. This does not mean the “cause” of a problem
has been found. What has been found is where the problem is occurring the most. The cause has yet to
be determined.
5.4
59
5.5.3 Procedures
As described in reference 5.4, the stratification process (fig. 55) is performed in the following
manner:
(1) Choose the stratification variables.
(2) Gather data and record the potentially important stratification variables.
(3) Graph the data using one of a number of different tools, such as bar chart (sec. 5.3), Pareto
chart (sec. 5.6), and histograms (sec. 5.7).
(4) Analyze the data on the chosen stratification variables and compare to each other.
(5) Separate the possible problem areas into special and common cause problems.
(6) If no conclusions are found, choose different stratification variables.
(7) Determine the strategy to improve the problem.
History of Discrepancy Reports for a Solid Rocket Motor
0
2
4
6
8
10
12
14
16
18
20
Month
NOZZLE
CASE
JOINT
INSULATION
IGNITER
PROPELLANT
LEGEND
Figure 55. Stratification (histogram) chart example.
510
5.5.4 Example
Shown in figure 55 is a histogram of discrepancy reports for a solid rocket motor (SRM),
stratified by components.
5.5.5 Advantages
The approach not only produces a priority ordering of the problems but also identifies areas for
improvement.
5.5.6 Limitations
(1) The correct stratification variables for resolving a problem are generally not known prior to
data collection.
(2) All potentially important stratification variables cannot be determined without planning.
5.6 Pareto Chart
5.6.1 Description
When there is a need to know the relative importance of data or variables (problems, causes, or
conditions), a Pareto chart is often used. This chart helps to highlight the few data or variables that may
be vital. The Pareto chart also helps to identify which problems, causes, or conditions are the most
important or most frequent so they can be addressed first.
5.2
5.6.2 Application
The Pareto chart can be used to examine the “how,” “what,” “when,” “where,” and “why”
of a suspected problem cause. This technique is typically performed in phase E. The chart is an
illustration of the data as of a specific time period. The data are arranged in descending order with the
most important to the left. The Pareto chart is based on the “Pareto principle” which states that a few of
the causes often account for most of the effects.
5.5
Pareto charts are used in performing problem trend
analyses (sec. 8.2).
5.6.3 Procedures
As described in reference 5.2, a Pareto chart (fig. 56) is created in the following manner:
(1) Identify the most likely causes of a problem (take from the cause/effect diagram (sec. 7.2)).
(2) Gather the data on causes; if necessary, use a checklist (sec. 7.8).
(3) Summarize the numbers of observations and calculate the percentages of each cause.
(4) Set the right vertical scale from zero to 100 percent.
(5) Make the left vertical scale the same height as the right scale and set it from zero to the
number of observations.
511
Power
Supply
Machine
Calibration
Connection Electrical
Component
Feed
Transformer
Wrong
Connection
Operator
Training
100
80
60
40
20
0
20%
40%
60%
80%
100%
0%
H
2
O Filter
Figure 56. Pareto chart example.
(6) The columns are drawn using the left scale.
(7) The first point is plotted at the upper center of the first column.
(8) Calculate and add together the percentages of cause one and two. The second point,
corresponding to their sum, is plotted across from the right scale directly over the second
column. The third point is found by adding the percentage of cause three to the total of one
and two, and plot. The total of all columns added together should be 100 percent, and the
last point is at the 100percent point.
(9) The plotted points are then joined with line segments.
The chart in figure 56 reveals the slope is more radical over the first two bars (power supply and
machine calibration) and this means that the majority of the problems occur in these categories, i.e.,
areas to the left of the most radical slope are the most probable problem areas. This observation is even
more obvious when the heights of the bars are examined.
512
5.6.5 Advantages
(1) The Pareto chart helps to identify the few areas of concern that are most important.
(2) The chart is useful in analyzing defect data.
5.6.6 Limitations
A poor Pareto chart will result if the causes chosen to study are wrong. Some preplanning needs
to be done before choosing categories.
5.6.7 Bibliography
Cane, V.E.: “Defect Prevention, Use of Simple Statistical Tools.” Ford Motor Company, Livonia, MI,
1989.
Hines, W.W., and Montgomery, D.C.: “Probability and Statistics in Engineering and Management
Science.” John Wiley, New York, 1986.
Wadsworth, S. and Godfrey: “Modern Methods for Quality Control and Improvement.” John Wiley,
New York, 1986.
5.7 Histograms
5.7.1 Description
Histograms are bar charts that show a dispersion of data over a specified range. This spread of
data makes presentations easier to interpret.
5.1
5.7.2 Application
When data are plotted on histograms, many items tend to fall toward the center of the data
distribution. Fewer items fall on either side of the center. The bars are proportional in height to the
frequency of the group represented. Since group intervals are equal in size, the bars are of equal width.
5.4
The histogram is best applied in phase E.
5.7.3 Procedures
As described in reference 5.2, a histogram (fig. 57) is constructed in the following manner:
(1) Gather the data to be plotted and count the total number of data points.
(2) Find the range of the data by subtracting the smallest data point from the largest.
(3) The number of data bars in the graph should be limited to between 6 and 12. The width of
each bar is determined by dividing the range of data by the selected number of bars.
(4) Scale the groups of data on the horizontal axis.
513
(5) Scale the frequency of occurrence or the numbers on the vertical scale.
(6) Plot the frequency of occurrence of the numbers in ascending order.
(7) Draw the height of each bar to show the number or frequency of the group interval using
the scale on the vertical axis. Each bar, including all data points, is the same width.
5.7.4 Example
The chart in figure 57 displays a typical histogram.
0
1
2
3
4
5
6
7
8
Time To Complete Tasks (Hours)
0
1
2
3
4
5
6
7
8
010 1020 2030 3040 4050
Figure 57. Histogram example.
5.7.5 Advantages
(1) A histogram helps identify changes in a process as the data changes.
(2) A histogram helps establish standards for a process.
5.7.6 Limitations
A histogram is not a good tool for computing process capability.
514
REFERENCES
5.1 Brocka, B. and Brocka, M.S.: “Quality Management, Implementing the Best Ideas of the
Masters.” Business One Irwin, Homewood, IL 60430.
5.2 Hunt, V.D.: “Quality in America, How to Implement a Competitive Quality Program.”
HD62.15.H86, Business One Irwin, Homewood, IL 60430, 1992.
5.3 Lyonnet, P.: “Tools of Total Quality, An Introduction to Statistical Process Control.” Chapman
& Hall, 1991.
5.4 Cane, V.E.: “Defect Prevention, Use of Simple Statistical Tools.” Ford Motor Company,
Livonia, MI, 1989.
5.5 “Total Quality Management, A Guide for Implementation.” DOD 5000.51–G (Draft), February
15, 1989.
515
6. STATISTICAL TOOLS AND METHODOLOGIES
There are numerous, excellent and highly detailed texts on the appropriate use of statistical
techniques. While this section lists and briefly discusses some of the available tools, the novice
statistician is cautioned to read and utilize standard, handbook references when using these techniques in
problem solving. Use solely of this text might well result in misuse and error. This toolbox does provide
a suitable knowledge of the existence of these tools and references for their appropriate application.
In this section, the following typical statistical processes are discussed: “studentt” (t test)
analysis, analysis of variance (ANOVA), correlation analysis, factorial analysis, confidence analysis,
regression analysis, and response surface methodology.
In many of these analyses, a comparison of sample statistics and population statistics will be
made. Here, it is assumed that population statistics would be obtained if an infinite number of specimens
could be measured, or if the solution to a function for the probability distribution of points were
available. Sample statistics are made from actual measurements of a sample with a finite number of
specimens. When only sample statistics are available (as is usually the case in engineering applications),
there is a finite probability that they are “close” to the population statistics.
6.1
A summary of the advantages and limitations of each tool or methodology discussed in this
section is presented in table 61.
6.1 “Studentt” Analysis
6.1.1 Description
As described in reference 6.1, the “studentt” compares the sample statistic “t,” which is based on
the sample mean and standard deviation, to the tdistribution for the same sample size and a desired
significance (probability of error). The tdistribution is similar to the normal distribution in that with an
infinite sample size, the tdistribution is equivalent to the standard normal distribution. At sample sizes
lower than infinity, the tdistribution becomes “lower and flatter” than the normal distribution. The
output of the tdistribution chart is the probability (α) that t exceeds a certain t
on the ordinate of the t
distribution chart. However, usually the probability is chosen and t
is sought; a tdistribution table is
usually used to find t
.
The tdistribution was described in 1908 by W.S. Gosset under the pen name “student,” thus the
name “studentt” analysis.
6.1.2 Application
“Studentt” analyses, as described in reference 6.2, are used when sample sizes are low for the following
functions:
(1) Determine if a sample mean is equivalent to a population mean within a given probability
of error.
(2) Determine if two sample means are equivalent to each other within a given probability of
error.
This technique is typically applied in phase D but may also be performed in phase C or E.
61
62
Table 61. Statistical tools and methodologies.
Tool or Methodology Section Advantages Limitations
“Studentt” analysis 6.1 The procedure is relatively simple to apply. The parent distribution must be reasonably close to a
normal distribution.
ANOVA 6.2 Sources of variation can be found, random variation
isolated, or any chosen source of variability isolated.
The processes are timeconsuming and often
approximate.
Correlation analysis 6.3 The analysis is quite simple. A straightline relationship is assumed.
Factorial analysis 6.4 Sources of variation can be found, random variation
isolated, or any chosen source of variability isolated.
Also, interactions between variables can be isolated, and
large numbers of variables can be solved.
The processes in factorial analysis are more time
consuming than the analysis of variance. A full factorial
analysis does not solve for exponential or polynomial
effects. The fractional factorial analysis does not solve
for all effects and higher order effects separately.
Confidence/reliability
determination and analysis
6.5 This analysis can give a realistic probability of whether
or not a process may yield a value which is above or
below a requirement.
A sample statistic must be known or assumed, such as
the population standard deviation, before an analysis can
be performed.
Regression analysis 6.6 A mathematical relationship can be determined, by hand
or computer, when the relationship is not obvious by
inspection.
If the data are discrete (e.g., integer data), the actual line
generated will only approximate the actual relationship.
Response surface methodology 6.7 A mathematical relationship can be determined, by hand
or computer, when the relationship is not obvious by
inspection.
If the data are discrete (e.g., integer data), the actual line
generated will only approximate the actual relationship.
6.1.3 Procedures
The use of a ttest for determining if a sample mean is equal to a chosen population mean will be
shown here.
(1) Determine the target mean and significance level desired.
(2) Develop null and alternate hypotheses for the problem being investigated. If it is desired to
prove that the sample mean is on one particular side of the population mean, the null
hypothesis is that the sample and population mean are equal. The alternate hypothesis is
that the sample mean is on the particular side of the population mean. If it is desired to
prove that the sample mean is not on either side of the population mean, the null hypothesis
would be the same, but the two alternate hypotheses would be that the sample mean is
above or below the population mean. This latter situation would use a “twotailed”
analysis.
(3) Determine the mean and standard deviation of the sample.
(4) Determine the t value using equation (6.1).
t · sample mean – target mean
sample /( n)
1/2
(6.1)
(5) Compare t with t
α
for the desired significance and degreesoffreedom (DOF) (n–1).
If t is greater than t , the null hypothesis is disproved, i.e., it cannot be assumed with the chosen
confidence that the sample mean is equivalent to the target mean. For a twotailed analysis, if t is greater
than t
( /2)
(or t is less than – t
( /2)
), the null hypothesis is disproved.
6.1
6.1.4 Example
Pull tests of a propellant sample yielded the following strains before failure: 29, 31, 35, 34,
and 36 percent. The nominal strain capability is 34 percent. Determine with a 0.10 significance, if the
propellant batch is representative of the nominal propellant. Since the mean of the propellant batch could
be =, >, or <34 percent, a twotailed analysis will be done. Thus, α/2 will be used (0.05 significance).
The null hypothesis will be a strain capability equal to 34 percent.
The sample mean is 33 and the sample standard deviation is 2.915. Substituting into equation (6.1),
t = 0.1539. From the tdistribution table for 4 DOF, t
= 2.134.
If H
o
had been rejected, it could be stated that there was only one chance in ten that the null
hypothesis was rejected when it should not have been. This is referred to as a type I error.
If H
o
were not rejected, it could be stated that the null hypothesis could not be rejected at the
0.10 level of significance unless the probability of a type II error is determined. The determination of the
probability of a type II error is complicated and many texts consider it beyond their scope.
6.1.5 Advantages
The procedure is relatively simple to apply.
63
6.1.6 Limitations
The distribution of the parent population must be reasonably close to a normal distribution.
6.1.7 Bibliography
Crow, E.L., Davis, F.A. and Maxfield, M.W.: “Statistics Manual.” NAVORD Report 3369, NOTS 948.
Handbook 91, “Experimental Statistics.” U.S. Department of Commerce, National Bureau of Standards.
Mendenhall, W.: “Introduction to Probability and Statistics.” Fourth edition, Wadsworth Publishing
Company, Belmont, CA 94002, 1975.
6.2 Analysis of Variance
6.2.1 Description
ANOVA is a technique used in design of experiments (sec. 5.5) to compare sample statistics, to
determine if the variation of the mean and variance between two or more populations are attributable to
sources other than random variation.
6.1
6.2.2 Application
The ANOVA technique is typically performed in phase D but may also be performed in
phase C or E.
Some of the uses for analysis of variance are:
(1) Determining if two or more processes are producing products that are consistent with each
other.
(2) Determine which two or more processes are different if a difference in processes is
detected.
(3) Eliminate one source of variation to determine the effect of the others.
(4) Determining the significance of each factor.
6.2.3 Procedures
As described in reference 6.1, to determine if two or more samples have different sample
statistics, the following methods can be used to determine if the withinsample variation is greater than
the sampletosample variation. If only one source of variation is being investigated, a one way
classification is used. A factor F (equation (6.2)) is compared to F , a value that is related to the total
DOF, based on the number of samples (k) and the sample size (n).
F·
betweensample variance
mean ofwithinsample variance
(6.2)
64
The DOF of the number of samples is k–1, and the DOF of the sample size is n–1. The total DOF
is k*(n–1). If F exceeds F ,
then a difference exists between the samples that is not only due to random
variation. F is found from an F distribution table.
Rather than determining sample statistics for each sample, approximation formulas that use sums
and averages of squares, can be used.
F ·
SS(bs)/ (k −1)
SSE / k(n −1)
, (6.3)
where SS(bs) is the sum of squares (betweensample) and SSE is the sum of squares error. The SSE is
determined from the sum of squares total (SST) and SS(bs) by the formula
SSE = SST – SS(bs).
SST and SS(bs) can be found using the formulas
SST = ∑ (y
ij
)
2
– C, SS(bs) = ∑ (T
i
)
2
/n – C,
and
C = T
2
/(k*n)
where y
ij
= each data point, T = total of all data points, and T
i
= total for each sample.
If two sources of variation are being investigated, a twoway classification is used. Data can be
arranged in blocks representing one source of variation, and one data point from each sample
representing the other source of variation is put into each block (see example below). If two sources are
being investigated, the following approximation equations can be used:
F(bs1) ·
MS(bs1)
MSE
·
SS(bs1)/ a −1
SSE / K(a −1)(b −1)
(6.4)
and
F(bs2) ·
MS(bs2)
MSE
·
SS(bs2)/ b −1
SSE / K(a −1)(b −1)
(6.5)
where
SSE = SST – SS(bs1) – SS(bs2); SST = ∑(y
ij
)
2
– C;
MS = Mean square MSE = Mean square error
SS(bs1) = ∑(T
i
)
2
/b – C; SS(bs2) = ∑(T
j
)
2
/a – C; and
C = T
2
/(k*n),
where a = the number of samples of one source of variation and b = the number of samples of the other
source of variation.
65
Other methods exist to isolate more sources of variability simultaneously. The latin square
method eliminates three sources, and the GrecoLatin method eliminates four sources. These methods
must use n
2
observations.
Analysis of covariance is a similar technique used when conditions (such as environmental)
change. The effect of this change is accounted for by using regression. This involves partitioning a total
sum of products rather than squares.
6.2.4 Example
In the following hypothetical example, the effect of two parameters on the variability of strain
capability of a solid propellant will be investigated. The use of three lots of polymer (parameter A) and
two lots of curative (parameter B) will be investigated. Six batches of propellant are mixed and tested
with the following average results:
Polymer Curative Percent Strain
1 1 30
1 2 34
2 1 32
2 2 36
3 1 31
3 2 33
The following table is arranged with parameter A set up in columns and parameter B set up in rows:
Curative Lot 1 Curative Lot 2 Total for Polymer
Polymer lot 1 30 34 64
Polymer lot 2 32 36 68
Polymer lot 3 31 33 64
Total for curative 93 103 196
here
C = (196)
2
/6 = 6402.67,
SST = 30
2
+ 34
2
+ 32
2
+ 36
2
+ 31
2
+ 33
2
– 6402.67 = 6426 – 6402.67 = 23.33,
SS(bs1) = (64
2
+ 68
2
+ 64
2
)/2 – 6402.67 = 6408 – 6402.67 = 5.33,
SS(bs2) = (93
2
+ 103
2
)/3 – 6402.67 = 16.67,
MS(bs1) = 5.33/2 = 2.67,
MS(bs2) = 16.67/1 = 16.67,
MSE = 1.33/((3–1)(21)) = 0.67,
F(sb1) = 2.67/0.67 = 4.0, and
F(sb2) = 16.67/0.67 = 24.88.
66
Note that a = 3 is the number of sources of variation of parameter A (polymer), and b = 2 is the
number of sources of variation of parameter B. Since F(sb1) is less than F for a 0.05 significance
(F = 5.14), polymer has no effect on strain capability. Since F(sb1) for a 0.05 significance is greater
than 5.99, strain capability is affected by the curative lot.
6.2.5 Advantages
Sources of variation can be found, random variation isolated, or any chosen source of variability
isolated.
6.2.6 Limitations
The processes are timeconsuming and often approximate.
6.3 Correlation Analysis
6.3.1 Description
Correlation analysis measures the strength of a linear relationship between two sets of data.
6.3
6.3.2 Application
Correlation analysis can be used to determine if a relationship exists between two independent sets
of variables. This technique is best performed in phase D but may also be performed in phase C or E.
6.3.3 Procedures
The procedures, as found in reference 6.3, for determining if two sets of data are linearly related
is as follows:
(1) Determine the mean of each set of data.
(2) Determine the r value of the two sets of data using the following equation:
r ·
Σ(x
i
− x )(y
i
− y )
Σ x
i
− x ( )
2
( )
1/2
Σ y
i
− y ( )
2
( )
1/2
(6.6)
where x and y are the means of the first and second set of data respectively. The value of r will
be between –1 and 1. If r is close to 0, then no correlation is implied; if r is close to 1 (or –1) then
a high degree of correlation is implied.
(3) Determine the significance of the r value by using the following equation:
z ·
(n − 3)
2
1n
(1+ r)
(1− r)
(6.7)
67
(4) Look up the z value in a standard normal distribution table to determine the probability of
having a correlation.
6.3.4 Example
The following hypothetical sets of measurements were taken: 5.4, 6.2, 6.5, 7, and 7.5; and 2.3,
2.1, 2, 1.8, and 1.6. The mean of the two sets are 6.52 and 1.96, respectively . The deviations, products,
and squares of the deviations from the means are shown in the following.
x y dx dy dx * dy dx
2
dy
2
5.4 2.3 –1.12 0.34 –0.3808 1.25 0.1156
6.2 2.1 –0.32 0.14 –0.0448 0.1024 0.0196
6.5 2.0 –0.02 0.04 –0.0008 0.0004 0.0016
7.0 1.8 0.48 –0.16 –0.0768 0.2304 0.256
7.5 1.6 0.98 –0.36 –0.3528 0.9604 0.1296
summations –0.856 2.548 0.292
Using equation (6.6), the r value is 0.992. Using this value for n = 5, z is –3.938, thus there is
less than a 0.01 percent chance of these two data sets not being related.
6.3.5 Advantages
This analysis is simple to apply.
6.3.6 Limitations
A straightline relationship is assumed.
6.4 Factorial Analysis
6.4.1 Description
There are three types of factorial analysis described in this section—factorial analysis, full
factorial analysis, and fractional factorial analysis. Factorial analysis is similar to ANOVA in that the
analysis is based on sums of squares, however, factorial analysis further subdivides the treatment of
sums of squares into components and can show interaction effects between parameters.
6.1
6.4.2 Application
Factorial analysis is used for applications similar to those for which ANOVA is used, except that
factorial analysis deals with levels of variables. Factorial analysis is used with a small number of
variables (e.g., two to four). Full factorial analysis is performed for more variables, but only at two
levels for each variable. Fractional factorial analysis is used when so many variables are being
investigated that experimenting with them is unfeasible. For example, if five variables are being
investigated, 2
5
or 32 experiments would have to be performed. For six variables, the number would be
64, and this is without replication. Thus, fractional factorial analysis is often economically necessary.
6.1
This technique is typically performed in phase C but may also be performed in phase D or E .
68
6.4.3 Procedures
As described in reference 6.1, factorial analysis is performed the same as analysis of variance
except that an analysis of variance is performed for each variable against each other variable.
The procedure for performing a full factorial analysis will be discussed here. With factorial
analysis, 2
n
factorial experiments will be performed, and to account for experimental variability, r
replications will be performed. Here n will be the number of factors rather than the sample size (which is
effectively two). With factorial analysis, certain computational shortcuts can be applied when only two
levels of each variable are used, assuming straight line relationships. The following is the procedure for
using the factorial analysis where n = 3.
(1) Arrange the factors and magnitudes in a table such as the following:
Table 62. Factorial analysis factors and magnitudes.
A0,B0,C0 M1 M2 M3 total A0, B0,C0
A1,B0,C0 M1 M2 M3 total A1, B0,C0
A0,B1,C0 M1 M2 M3 total A0, B1,C0
A1,B1,C0 M1 M2 M3 total A1, B1,C0
A0,B0,C1 M1 M2 M3 total A0, B0,C1
A1,B0,C1 M1 M2 M3 total A1, B0,C1
A0,B1,C1 M1 M2 M3 total A0, B1,C1
A1,B1,C1 M1 M2 M3 total A1, B1,C1
etc. where the first column represents the experimental conditions and M1, M2, and M3 represent the
resulting magnitudes after the experiment for replication 1, 2, and 3. The last column is the total of all
replications of experiments for each experimental condition.
(2) Obtain a table of effects totals by removing the middle columns in the above table.
(3) Apply the method of Yates to this table as follows:
a. Add n (3) columns in the place of the middle columns and three columns to the right
side of the table (table 63).
b. Add the first two totals in the totals column to get the first element in column 1. Add
the third and fourth totals in the totals column to get the second element in column 1.
Continue in a like manner to get the third and fourth elements in column 1. Obtain the
fifth through eighth elements in column 1 the same way except that the totals are
subtracted (first value subtracted from the second). Column 2 is constructed the same
way from column 1 as column 1 was constructed from the totals column. Column 3 is
constructed the same way from column 2. Column 3 is the effect totals as in analysis of
variation. The notation in column n (3) and the sum of squares column is shortened;
2:1 means the first element of column 2.
c. Add a row for the error sum of squares and error mean square, determined as in
ANOVA.
69
Table 63. Factorial analysis example.
Exp.
Condition
Totals from
Above
1 2 3 Sum of
Squares
Mean of
Squares
F
1 A0, B0,C0 t1 t1 + t2 (t1 + t2) + (t3 + t4) 2:1 + 2:2
3:1/(r2
n
)
SS1/DOF MS1/SME
2 A1, B0,C0 t2 t3 + t4 (t5 + t6) + (t7 + t8) 2:3 + 2:4
3:1/(r2
n
)
SS2/DOF MS2/SME
3 A0, B1,C0 t3 t5 + t6 (t2 – t1) + (t4 – t3) 2:5 + 2:6
3:1/(r2
n
)
SS3/DOF MS3/SME
4 A1, B1,C0 t4 t7 + t8 (t6 – t5) + (t8 – t7) 2:7 + 2:8
3:1/(r2
n
)
SS4/DOF MS4/SME
5 A0, B0,C1 t5 t2 – t1 (t3 + t4) – (t1 + t2) 2:2 – 2:1
3:1/(r2
n
)
SS5/DOF MS5/SME
6 A1, B0,C1 t6 t4 – t3 (t7 + t8) – (t5 + t6) 2:4 – 2:3
3:1/(r2
n
)
SS6/DOF MS6/SME
7 A0, B1,C1 t7 t6 – t5 (t4 – t3) – (t2 – t1) 2:6 – 2:5
3:1/(r2
n
)
SS7/DOF MS7/SME
8 A1, B1,C1 t8 t8 – t7 (t8 – t7) – (t6 – t5) 2:8 – 2:7
3:1/(r2
n
)
SS8/DOF MS8/SME
summation SSE SME
To find:
2:1 + 2:2 = (t1 + t2) + (t3 + t4) + (t5 + t6) + (t7 + t8)
2:3 + 2:4 = (t2 – t1) + (t4 – t3) + (t6 – t5) + (t8 – t7)
2:2 – 2:1 = (t5 + t6) + (t7 + t8) – (t1 + t2) + (t3 + t4)
2:4 – 2:3 = (t6 – t5) + (t8 – t7) – (t2 – t1) + (t4 – t3)
(4) The sum of squares column is generated by dividing the square of each adjacent element in
column 3 by r * 2
n
.
(5) The mean of squares column is generated by dividing each adjacent element in the sum of
squares column by its respective DOF. The DOF will be 1 for each effect, but will be n–1
for the error row.
(6) Obtain each F by dividing each mean square by the error mean square.
(7) Compare each F to F for n–1, DOF. If any F exceeds F , that effect is significant.
A fractional factorial analysis is performed the same way as the full factorial analysis except the
analysis is split into fractions of (1/2)
p
. Thus, if a five variable investigation (32 experiments) is split
into 1/4, the number of experiments will be 2
n–p
(eight) experiments.
6.4.4 Example
The following are the results of a hypothetical experiment to determine if mix time, mix speed, and
mix vacuum affects the burn rate of a propellant. Two levels of each parameter were tested as follows:
Effect Parameter Low (0) High (1)
A mix time 2 hr 3 hr
B mix speed 1 rps 2 rps
C vacuum no vacuum 0.2 atm
610
Each effect was assigned a high and low level (e.g., 1 rps was assigned as low, 2 rps was
assigned as high). The high and low levels are designated as 0 and 1, respectively. Each experimental
condition was repeated three times with the following results:
Exp. Condition Rep 1 Rep 2 Rep 3 Total
A0 B0 C0 0.47 0.47 0.52 1.46
A1 B0 C0 0.46 0.46 0.51 1.43
A0 B1 C0 0.47 0.48 0.52 1.47
A1 B1 C0 0.48 0.50 0.50 1.48
A0 B0 C1 0.51 0.50 0.54 1.55
A1 B0 C1 0.49 0.52 0.54 1.55
A0 B1 C1 0.52 0.51 0.55 1.58
A1 B1 C1 0.50 0.52 0.54 1.56
The table is repeated with the replication columns deleted and replaced with the application of three
columns for the Method of Yates. Three additional columns are added, one for the sum of squares, one
for the mean square, and one for the F value for each effect.
Exp.
Condition
Total 1 2 3 Sum of
Squares
Mean of
Squares
DOF F
A0 B0 C0 1.46 2.89 5.84 12.08 6.0803 6.0803 1
A1 B0 C0 1.43 2.95 6.24 –0.04 0.000067 0.000067 1 0.2977
A0 B1 C0 1.47 3.10 –0.02 0.10 0.000417 0.000417 1 1.8616
A1 B1 C0 1.48 3.14 –0.02 0.02 0.000017 0.000017 1 0.0745
A0 B0 C1 1.55 –0.03 0.06 0.04 0.00667 0.00667 1 29.77
A1 B0 C1 1.55 0.01 0.04 0 0 0 1 0
A0 B1 C1 1.58 0 0.04 –0.02 0.000017 0.000017 1 0.0745
A1 B1 C1 1.56 –0.02 –0.02 –0.06 0.00015 0.00015 1 0.669
Replicates SSR SMR 0.00723 0.003615 2 16.138
error SSE SME 0.00157 0.000224 7
The correction term (C) is as follows:
C =
Sum of totals ( )
2
Number of effects ( ) Number of totals ( )
. (6.8)
The SST is as follows:
SST = Sum of each individual replication squared – C. (6.9)
The sum of squares treatment (SSTr) is as follows:
SSTr = [(Sum of each individual total squared)/Number of effects] – C. (6.10)
611
The sum of squares replication (SSR) is as follows:
SSR = [(Sum of vertical replication total squared)/Number of rows] – C. (6.11)
The sum of squares error (SSE) is as follows:
SSE = SST – SSTr – SSR. (6.12)
The sum of mean replicate (SMR) is as follows:
SMR = SSR/DOF. (6.13)
The sum of mean error (SME) is as follows:
SME = SSE/DOF. (6.14)
F
for a 0.05 confidence is 5.59, therefore effect C (vacuum) and replication have a significant
effect on the burn rate. (The third batch of propellant may have been different for another reason such as
contamination.) Note that since no values of F are greater than F for any conditions where two or more
effects are 1, then no interactions have a significant effect on burn rate. (For example, if the fourth line
had an F greater than F , then the interaction of mix time and mix speed would have a significant
interaction).
6.4.5 Advantages
Sources of variation can be found, random variation isolated, or any chosen source of variability
isolated. Also, interactions between variables can be isolated, and larger numbers of variables can be
solved for.
6.4.6 Limitations
The processes in factor analysis are more timeconsuming than the analysis of variance. A full
factorial analysis does not solve for exponential or polynomial effects. The fractional factorial analysis
does not solve for all effects and higher order effects separately.
6.5 Confidence/Reliability Determination and Analysis
6.5.1 Description
Confidence analysis compares sample values, means, or standard deviations with population
standard deviations to obtain a confidence interval, with a chosen significance.
6.5.2 Application
Confidence analysis is used to determine the interval of values that a data point could take, with a
chosen probability of being within that interval. Confidence analysis can be used with individual points,
means, standard deviations, regression lines, or reliability measurements such as mean time between
failures.
6.1
This technique is typically performed in phase C or E.
612
6.5.3 Procedures
As described in reference 6.1, the procedures for determining the confidence interval for the population
mean, given a sample mean, will be discussed here.
(1) Choose a confidence (α) level and obtain the α/2 term by dividing the confidence level by
2.
(2) Determine, from past experience (or by adjusting the sample standard deviation), the
population standard deviation.
(3) Obtain the z
(α/2)
value by looking up the z value for α/2 in a normal distribution table.
(4) The values for either end of the confidence interval is given by the equation:
Int = m
s
± z
(α/2)
* s
p
/n
1/2
(6.15)
where Int is the low or high confidence interval value, m
s
is the sample mean, s
p
is the population
standard deviation, and n is the sample size. For large n, the sample standard deviation can be used
instead of the population standard deviation.
The confidence interval for the population standard deviation, given the sample standard deviation, is
determined in the same way as above, except equation (6.16) is used.
Int ·
s
p
1 t z
( /2)
/ s
s
/ (2* n)
1/2
(6.16)
where s
s
is the sample standard deviation. For linear regression, the confidence for the equation of the
line is:
Int · (a + bx
o
) t t
/2
* s
e
*(1/ n + n(x
o
− m
s
)
2
/ S
xx
)
1/2
(6.17)
and for the y value:
Int · (a + bx
o
) t t
/2
* s
e
*(1+ 1/ n + n(x
o
− m
s
)
2
/ S
xx
)
1/2
(6.18)
where
se
2
·1/ (n − 2)Σ(yi − (a + bxi))
2
·
S
xx
*S
yy
−(S
xy
)
2
n(n−2)S
xx
where
S
xx
· n * Σx
i
2
− (Σx
i
)
2
, S
yy
· n * Σy
i
2
− (Σy
i
)
2
, and S
xy
· n * Σx
i
y
i
− Σx
i
( ) *( ΣY
i
) .
613
6.5.4 Example
Determine the confidence interval for insulation erosion at a station in the RSRM aft dome to
determine if the associated compliance safety factor (CSF) may actually fall below the 1.0 minimum
value, with a 95 percent confidence. The sample data for 18 flights (36 motors) is:
Erosion mean 1.112 in
Standard deviation 0.207 in (defined as known s
p
)
n 36
α/2 is (1–0.95)/2 = 0.025, therefore the z(
α/2
) term is 1.96. Entering the above values into equation
(6.15), the confidence interval is 1.112 ± 1.96 * 0.207/(36)
1/2
= from 1.042 to 1.182 for erosion.
The safety factor is then calculated using the maximum erosion value and is:
CSF = Min Ins t
Erosion+3s
p
+0.1
CSF = 3.36 = 1.766 .
1.182+3(0.207)+0.1
So, in this instance the confidence interval is used to calculate a safety value that can be compared to a
performance requirement.
6.5.5 Advantages
This analysis can give a realistic probability of whether or not a process may yield a value which
is above or below a requirement.
6.5.6 Limitations
A sample statistic must be known or assumed, such as the population standard deviation, before
an analysis can be performed.
6.6 Regression Analysis
6.6.1 Description
Regression analysis is a form of curve fitting to find a mathematical relationship for a group of
data. There are typically two types of regression: regression and multiple regression. Typical types of
relationships which are assumed for regression include linear (straight line), polynomial, and
exponential. A goodness of fit test is often performed to see how well the generated relationship fits the
data.
6.3
The method of least squares is probably the most frequently used method of regression. The
equation for the method of least squares is obtained by setting the derivative equal to zero of the
equation for the sum of the vertical distance from each y value to the mean y value.
614
6.6.2 Application
Regression, as described in reference 6.1, is typically used for three purposes:
(1) To find the mathematical relationship represented by a group of data points.
(2) To determine if the magnitude of a measurement is increasing or decreasing with time or
event.
Regression analysis is best applied in phase D but may also be applied in phase E. There are
several methods of regression. Multiple regression will be discussed in section 6.7. The least squares
method is a commonly used method of regression, and will be discussed here (assuming a straightline
relationship). The R
2
indicates the percent variation in the dependent variable that can be explained by
the independent variable.
6.6.3 Procedures
As described in reference 6.3, the use of the least squares method for finding the equation of a
line of the form
y = a + bx, (6.19)
is as follows:
(1) Determine the mean of the x
i
values ( x ) and y
i
values ( y ).
(2) Determine the deviation of each x
i
and y
i
value.
(3) Determine the slope of the trend line by dividing the summation of the multiple of the
deviations by the summation of the square of the x deviations (equation (6.19)).
b ·
Σ(x
i
− x )(y
i
− y )
Σ(x
i
− x )
2
(6.20)
(4) Determine the y intercept by subtracting the product of the slope and the mean x value from
the mean y value (equation (6.20)).
a = y – (b) x . (6.21)
The intercept and slope are used in equation (6.19) for a line representing the straightline
relationship. If the slope (b) is negative, then a decreasing trend may be indicated.
The explanatory power can be determined by R
2
as follows:
(1) Determine y values for each x value using the line generated above.
(2) Determine the deviation of each generated y value from the mean y.
615
(3) Obtain the R
2
value by dividing the sum of the square of the generated y deviations by the
sum of the square of the actual y deviations (equation (6.21)).
R
2
·
Σ(gen y
i
− y)
2
Σ(y
i
− y)
2
(6.22)
A good relationship is indicated by an R
2
value close to 1.
6.6.4 Example
As adapted from reference 6.3, assume the set of ordered pairs (1,4), (2,5), (3,6), (4,3), (5,5),
(6,5), (7,4), (8,6), (9,4), and (10,5). The following table shows summations, squares, and products that
go into the equations above:
x y (dx)
2
(dy)
2
(dx)(dy) y
g
dy
g
2
1 4 20.25 0.49 3.15 4.56 0.0196
2 5 12.25 0.09 –1.05 4.59 0.0121
3 6 6.25 1.69 –3.25 4.62 0.0064
4 3 2.25 2.89 2.55 4.65 0.0025
5 5 0.25 0.09 –0.15 4.68 0.0004
6 5 0.25 0.09 0.15 4.71 0.0001
7 4 2.25 0.49 –1.05 4.75 0.0025
8 6 6.25 1.69 3.25 4.78 0.0064
9 4 12.25 0.49 –2.45 4.81 0.0121
10 5 20.25 0.09 1.35 4.84 0.0196
summation 55 47 82.5 8.1 2.50 0.0817
where dx = xi – x , dy = yi – y , yg = generated points for each x, and dyg = yg – y . Using these data, the
mean x value is 5.5, the mean y value is 4.7, the slope (b) is 0.0303, and the y intercept (a) is 4.533. The
equation for the line is y = 0.0303(x) + 4.533. No significant relationship is indicated for this example,
R2 = 0.0101. Figure 61 shows the points and the generated line for this data.
10
9
8
7
6
5
4
3
2
1
0
1 3 4 5 6 7 8 9 10 0 2
generat ed line
Figure 61. Line generated with least squares method.
616
6.6.5 Advantages
A mathematical relationship can be determined, by hand or computer, when the relationship is
not obvious by inspection.
6.6.6 Limitations
If the data are discrete, e.g., integer data, the actual line generated will only approximate the
actual relationship.
6.7 Response Surface Methodology
6.7.1 Description
Response surface methodology is a method for surface fitting, much like regression is a method
for curve fitting. The surface can be a plane, using two independent variables and straightline
relationships, or it can be a more complex surface, using polynomial relationships. There are two
typically used methods for response surface analysis—multiple regression and factorial experimentation.
Factorial experimentation is discussed in section 6.4.
6.1
6.7.2 Application
Response surface analysis is typically used for the following purposes:
(1) To find the mathematical relationship represented by a group of data points.
(2) To optimize independent variables for maximum or minimum results.
This methodology is best performed in phase D or E.
6.7.3 Procedures
As described in reference 6.3, the least squares method of multiple regression, assuming a
straightline relationship, will be shown here. The basic form of the equation for a plane surface is y = a
+ b
1
x
1
+ b
2
x
2
+ b
3
x
3
+... + b
n
x
n
. This equation is minimized. After setting the derivative of the equation
for the sum of the vertical distances or ∑ (yi – (a + b
1
x
1
+ b
2
x
2
+ b
3
x
3
+... + b
n
x
n
))
2
to zero, the
equations for two independent variables are:
∑y = nb
0
+ b
1
* ∑x
1
+ b
2
* ∑x
2
,
∑(x
1
* y) = b
0
* ∑x
1
+ b
1
* ∑x
1
2
+ b
2
* ∑(x
1
* x
2
),
∑(x
2
* y) = b
0
* ∑x
2
+ b
1
* ∑(x
1
* x
2
) + b
2
* ∑x
2
2
. (6.23)
These equations are solved simultaneously for b
0
, b
1
, and b
2
.
617
Often, if the numbers are equally spaced, a set of numbers is coded. For example, the numbers
are substituted by assuming a smaller whole number for each original number. This practice makes
solving for the coefficients much easier with very little cost in accuracy.
6.7.4 Example
In the following hypothetical example, as adapted from reference 6.3, propellant was aged at
100°, 120°, and 140° for 1, 6, and 12 mo. Mean modulus of elasticity measurements are given for three
propellantaging temperatures and times. The columns for x
1
2
, x
2
2
, x
1
x
2
, x
1
y, and x
2
y and the bottom row
of summations are derived from the first two columns.
x
1
x
2
y x
1
2
x
2
2
x
1
x
2
x
1
y x
2
y
100 1 360 10,000 1 100 36,000 360
120 1 352 14,400 1 120 42,240 352
140 1 347 19,600 1 140 48,580 347
100 6 358 10,000 36 600 35,800 1,548
120 6 350 14,400 36 720 42,000 2,100
140 6 345 19,600 36 840 48,300 2,070
100 12 347 10,000 144 1,200 35,700 4,284
120 12 349 14,400 144 1,440 41,880 4,188
140 12 343 19,600 144 1,680 48,020 4,116
1,080 57 3,151 132,000 543 6,840 377,520 19,845
The equations for finding the constants are as follows:
From equation (6.23),
3,151 = 9b
0
+ b
1
1,080 + b
2
57
377,520 = b
0
1,080 + b
1
132,000 + b
2
6,840
19,845 = b
0
57 + b
1
6,840 + b
2
543
b
0
·
3,151 1, 080 57
37,7520 132, 000 6,840
19,845 6,840 543
9 1,080 57
1,080 132,000 6,840
57 6,840 543
b
1
and b
2
are calculated in the same manner. Solving the simultaneous equations (6.23), the constants are
b
0
= 383.98, b
1
= –0.25, and b
2
= –0.6117. Therefore the equation for modulus of elasticity for the
sample propellant is
y = 383.98 – 0.25 * x
1
– 0.6117 *
x
2
.
618
6.7.5 Advantages
A mathematical relationship can be determined, by hand or computer, when the relationship is not
obvious by inspection.
6.7.6 Limitations
If the data are discrete (e.g., integer data), the actual line generated will only approximate the actual
relationship.
619
REFERENCES
6.1 Miller, I. and Freund, J.E.: “Probability and Statistics for Engineers.” Second edition, Prentice
Hall, Inc., Englewood Cliffs, NJ 07632, 1977.
6.2 Hines, W.W. and Montgomery, D.C.: “Probability and Statistics in Engineering and Management
Science.” John Wiley and Sons, Inc., 1990.
6.3 NASA–STD–8070.5A, “Trend Analysis Techniques.” October 1990.
620
7. TOTAL QUALITY MANAGEMENT TOOLS
This section describes several TQM tools available to the system engineer analyst. TQM is
applied to continuously improve performance at all levels of operation, in all areas of an organization,
using all available human and capital resources. Improvement is addressed toward such areas as cost,
quality, market share, schedule and growth. TQM is an ongoing effort that demands commitment and
discipline.
A tool to assess an operation against other operations is the benchmarking technique which is
discussed in section 7.1. The cause and effect technique relates identified problems to their causes, and
this tool is discussed in section 7.2.
Concurrent engineering is more of an approach to quality management than a technique and it
is an interaction of disciplines during the design but before production. This approach is
discussed in section 7.3.
Three tools that attempt to improve the quality program are the cost of quality, design of
experiments (DOE), and evolutionary operation (EVOP). The cost of quality tracks a quality program
and attempts to identify ways to improve the program. This technique is discussed in section 7.4. Design
of experiments varies all possible combinations of factors and levels in an attempt to obtain the optimum
settings for a desired output. This technique is discussed in section 7.5. A methodology for improving
quality by looking at the production process is the evolutionary operation technique, and it is discussed
in section 7.6.
Group consensus techniques are often applied to solve problems. Three such tools are
brainstorming, Delphi, and nominal group technique (NGT). These techniques are discussed in sections
7.7, 7.9, and 7.10, respectively.
A methodology for collecting data quickly and easily in a simplified manner is the checklist
technique. This tool is discussed in section 7.8.
Another tool that might apply to the group consensus technique is the force field analysis. This
methodology counts the positive and negative forces, as well as their magnitudes, that effect the results
of a proposed solution or change in process. The force field analysis is discussed in section 7.11.
A methodology that is applied early in a design process is the quality function deployment
(QFD) technique which is discussed in section 7.12. This technique is used to solve problems before the
production phase begins and thus assists in the design of competitive products. By using a chart known
as the house of quality, priorities are given to the possible solutions as they relate to the identified
problems. Also, the product can be benchmarked against the competition in the areas of how well the
product stacks up against the competition as far as handling the identified problems, and how well the
product stacks up against the competition as far as meeting the appropriate engineering standards.
The final four tools that are discussed in this section are applied to improve a process. These
tools are quality loss function, SPC, flowchart analysis and work flow analysis (WFA). Quality loss
function, discussed in section 7.13, is a method of determining “loss to society” when a product is not at
the mean but is still within specification limits. SPC, discussed in section 7.14, is a process improvement
tool that helps identify problems quickly and accurately. The flowchart analysis, discussed in section
7.15, pictorially represents the steps of a process thus making it easier to eliminate nonvalued steps of
the process. Finally, the WFA, discussed in section 7.16, examines the work process for possible
improvements in performance and the quality of work life.
A summary of the advantages and limitations of each tool or methodology discussed in this
section is presented in table 71.
71
Table 71. TQM tools and methodologies—Continued
Tool or Methodology Section Advantages Limitations
Benchmarking 7.1 (1) Helps meet customer requirements.
(2) Helps establish goals and priorities.
(3) Helps determine true measures of productivity.
(4) Helps to attain and maintain a competitive
position.
(5) Helps identify and maintain awareness of industry
best practices.
(1) Must be continuous in order to keep up with the latest industry
changes.
(2) Determining industry “best practices” is difficult and often
subjective enough to be biased by the reviewing company’s
“wants” rather than the reviewing company’s customer’s wants.
Cause and effect diagrams 7.2 (1) Enables quality analysis groups to thoroughly
examine all possible causes or categories.
(2) Useful in analyzing statistical process control
(SPC) problems; SPC detects problems but can
poses no solutions.
Arriving at a group consensus is timeconsuming.
Concurrent engineering 7.3 (1) Shortens and makes more efficient the designto
development life cycle by employing the
interactions of functional disciplines by a cross
functional team.
(2) Reduces costs in the designtodevelopment life
cycle.
(1) The degree of success of this technique depends upon the degree
of cooperation between the multifunctional team members.
(2) Significant additional time, and associated funding, is required at
the front end of a program to perform the coordinated planning.
While time and money are saved overall within the effort, it is
often difficult to “frontload” large tasks.
(3) If design is pursued by projectized teams, the institutional
knowledge of the organization becomes very difficult to capture
or employ in the design decisions.
Cost of quality 7.4 (1) Helps to reveal and explain the more significant
costs.
(2) Activities and processes that need improvement can
be prioritized.
(3) Helps to reveal and explain the hidden costs of a
product or service.
(1) The technique itself can be expensive, thus making its goals of
saving/eliminating costs unachievable.
(2) Measurement for measurement's sake is an easy paradigm to fall
into; this technique is subject to misuse in this regard.
Design of experiments 7.5 The technique optimizes product and process design,
reduces costs, stabilizes production processes, and
desensitizes production variables.
(1) The performance of the analysis is time consuming and, the
results generally do not include parameter interactions.
Preknowledge of interaction significance is required to support
appropriate DOE technique selection.
(2) The DOE technique is often performed without a “verification
experiment” in which the predicted “optimized” parameters are
tested for performance (in agreement with the predictions). In,
addition a mistake is often made by taking the “best”
experiment’s parameters as an optimized set rather than an
interpolated set.
(3) Parameters must be interpolated from within the tested data set
rather than extrapolated beyond it.
72
Table 71. TQM tools and methodologies—Continued
Tool or Methodology Section Advantages Limitations
Evolutionary operation 7.6 (1) The cost is very low, so it can be run continuously.
(2) This technique increases a plant’s capacity and thus
profits will increase.
(3) The tool is simple and relatively straightforward.
EVOP is slow, so progress is slow.
Brainstorming 7.7 The technique takes advantage of the ideas of a group to
arrive at a quick consensus.
(1) The technique only proposes a solution but does not determine
one.
(2) The technique is limited by the ability of the group to achieve
consensus.
Checklists 7.8 (1) The tool is quick and easy to use.
(2) Checklists help to minimize errors and confusion.
Time must be taken to assemble a group to decide what data should
be collected.
Delphi technique 7.9 (1) Useful in eliminating personality clashes.
(2) Useful when powerful personalities are likely to
dominate the discussion.
(3) Inputs from experts unavailable for a single
meeting are included.
(1) Arriving at a group consensus is time consuming.
(2) Assembling the group participants is difficult/timeconsuming.
Nominal group technique 7.10 Very effective in producing many new ideas/solutions in
a short time.
(1) Assembling the group participants is difficult/timeconsuming.
(2) Divergence in weighting factors is common.
Force field analysis 7.11 Useful in determining which proposed solution, among
many, will meet the least resistance.
The technique is time consuming in arriving at a consensus on the
values (weights) of the forces, and is highly subjective.
Quality function deployment 7.12 (1) Helps organizations design more competitive,
higherquality, lowercost, products easier and
quicker.
(2) Helps ensure quality products and processes by
detecting and solving problems early.
(3) Engineering changes, design cycle, and startup
costs are reduced.
(4) Voice of the customer is heard.
(5) The technique is proactive, not reactive.
(6) Prevents problems from “falling through the crack.”
(7) The technique is costeffective.
(8) Easy to learn.
(1) Assembling the group participants is difficult/timeconsuming.
(2) The technique is not easy to perform.
73
Table 71. TQM tools and methodologies—Continued.
Tool or Methodology Section Advantages Limitations
Quality loss function 7.13 (1) Evaluates loss at earliest stage of product/process
development.
(2) Useful results obtained quickly and at low cost.
(1) It may be difficult to convince manufacturers to apply the
technique.
(2) It is often difficult to characterize the loss function.
Statistical process control 7.14 (1) This technique determines the cause of variation
based on a statistical analysis of the problem.
(2) The technique improves process performance.
(3) SPC helps identify problems quickly and
accurately.
SPC detects problems but poses no solutions.
Flowchart analysis 7.15 (1) Allows the examination of and understanding of
relationships in a process.
(2) Provides a stepbystep picture that creates a
common understanding about how the elements of
the process fit together.
(3) Comparing a flowchart to actual process activities
highlights areas where policies are unclear or are
being violated.
The development process is time consuming.
Work flow analysis 7.16 The technique increases productivity and improves
working conditions.
(1) The technique requires cooperation between employees and
management to be successful.
(2) The observed operation may not be fully representative of a
“typical” process that would occur without scrutiny.
74
7.1 Benchmarking
7.1.1 Description
Benchmarking, as described in reference 7.1, is a technique used to assess how an organization,
or process, is performing against internal guidelines, competitors, or even noncompetitors that may be
recognized as being superior. Benchmarking helps improve a process by recognizing priorities and
goals. The technique must be continuously applied in order to be effective because practices constantly
change (continuous improvement) affecting strategy. If the benchmarking process is performed once and
forgotten, then the operation may become inefficient by not keeping up with the latest industry best
practices.
7.1.2 Application
The benchmarking technique is typically performed in phase E but may also be performed in
phase A or B. This technique can be applied when it is desirable to know the strengths and weaknesses
of an organization’s own operation. These strengths and weaknesses can then be compared to internal
guidelines to evaluate the organization’s conformance to those guidelines.
Benchmarking can be applied to identify the strengths for products that directly compete with the
organization’s specific product under consideration. The manufacturers of those competing products are
probably using the same benchmarking technique to evaluate the competitors for their product. Once the
strengths and weaknesses of competing products are known, the company can attempt to differentiate
their capabilities in the marketplace.
By accomplishing this analysis, an organization can also incorporate the strengths of their
competitors that exist in certain areas.
7.1.3 Procedures
As adapted from reference 7.3, the basic elements of benchmarking include the following:
(1) Decide which process(es) or product(s) to benchmark.
(2) Determine the criteria to benchmark, i.e., benchmark internally against established
guidelines, benchmark against competitors, or benchmark against noncompetitors that are
considered industry leaders.
(3) Choose the particular characteristics of the operation or product to benchmark.
(4) Collect data on the processes or products that are being benchmarked.
(5) Analyze the data, prepare an action plan, and implement the plan.
(6) Assess the results of all the changes.
(7) Repeat the benchmarking technique, as necessary, in order to stay uptodate with the
applicable operation.
75
7.1.4 Example
The following illustration, adapted from reference 7.3, shows an example of comparative bench
marking between one company’s process and five competitors on a scale of 1 (worse) to 10 (better).
Better
Worse
10
9
8
7
6
5
4
3
2
1
Process
Competitor 2
Competitor 3
Competitor 1
Competitor 5
Competitor 4
Organization
Figure 71. Comparative benchmarking.
This illustration reveals that this company needs to look closely at the operations of competitors
2 and 3 and consider implementing into their process any strengths that are discovered. This company
should also look at those competitors rated lower on the scale and identify their weaknesses and ensure
that those weaknesses do not exist in their operation.
7.1.5 Advantages
(1) Benchmarking helps meet customer requirements.
(2) Benchmarking helps establish goals and priorities.
(3) Benchmarking helps determine true measures of productivity.
(4) Benchmarking helps to attain and maintain a competitive position.
(5) Benchmarking helps identify and maintain awareness of industry’s best practices.
7.1.6 Limitations
(1) The benchmarking process must be continuous in order to keep up with the latest industry
changes.
(2) Determining industry “best practices” is often difficult and subjective. The reviewing
company may well bias their results based on company “wants” rather than customer
“wants.”
76
7.2 Cause and Effect Diagrams (Also Known as Fishbone Diagrams
or Ishakawa Diagrams)
7.2.1 Description
The cause and effect diagram, as described in reference 7.3, graphically represents the
relationships between a problem (effect) and its possible causes. The development process is started in a
group session led by a trained facilitator. The problem is stated in terms acceptable to the group.
Possible causes are listed. The group then assigns priorities to the causes and action plans are developed.
When a cause and effect diagram is constructed, thinking is stimulated, thoughts are organized,
and discussions are begun. These discussions bring out many possible viewpoints on the subject. Once
all participants reach a similar level of understanding about an issue, an expansion of ideas can then be
examined.
Cause and effect diagrams are developed in a form, commonly referred to as “fish,” where the
effect is found in a box to the right which is the head of the fish. The bones of the fish show the
organized causes. The effects and causes can be expressed in words or data.
7.2.2 Application
As adapted from reference 7.3, cause and effect diagrams are used to examine many different
topics which include the following:
(1) The relationships between a known problem and the factors that might affect it.
(2) A desired future outcome and its related factors.
(3) Any event past, present, or future and its causal factors.
The cause and effect diagram is useful in examining processes such as SPC, SPC problems, (sec.
7.14) problems. The cause and effect diagram technique is best applied in phase E but may also be
applied in phase A or B. The technique is also useful in planning activities and brainstorming. The
diagram is basically a controlled way of gathering and using suggestions through group consensus.
7.2.3 Procedures
A cause and effect diagram, as adapted from reference 7.3, is developed in the following manner:
(1) Define the effect as clearly as is possible and place it at the head of the fish. This effect
represents the “problem” that is being investigated. As data are collected, the effect can be
redefined, if necessary.
(2) The group brainstorms the causes and lists them in no particular order. These causes are
then studied and the causes that affect these causes are identified. This will continue until
no new causes are thought of by the group.
(3) Once all causes are identified, list all categories, then display the categories on the diagram.
(4) The group then prioritizes the causes by multivoting. Each member of the group lists the
causes in order of significance. Votes are counted and a final list is written.
77
(5) The highest prioritized causes are listed on the diagram as the big bones. The next highest
prioritized causes will be listed on the diagram as the medium bones. Finally, the least
prioritized causes will be listed on the diagram as the small bones.
(6) As categories and causes are included on the diagram, thinking may be stimulated and new
causes may be identified.
(7) Teams are then formed to research and report on preventive (i.e., proactive) measures.
7.2.4 Examples
Example 1:
Assume the problem is design rework (fig. 72). The group fills in the probable root causes
through “brainstorming” ideas (sec. 7.7). When complete, the group prioritizes the causes using
multivoting. This is a technique where each person lists the causes in order of significance. Votes are
counted and a final list is written. Teams are formed to research and report on preventive measures. In
conclusion, a team has put their thoughts in writing and arrived at a consensus.
CHANGES SCHEDULE
No Second Shift
Under
Staffed
Lacking
Skills
Working
Outside
Discipline
INTERFACES
SKILL
SPECS
Discontented
Loanee
Doesn't Know
System
ByPassed
Outdated
Equipment
GRAPHICS
Poor
Notification
Try It Now,
Change Later
Tracking
Poor
Not Clear
Requires
Backtracking
Responsibility
Not Defined
Out of Sequence
Rushed
Late Start
Excessive
Meetings
Conflict
Not Clear
Cost Rules
REWORK
DESIGN
Figure 72. Design rework cause and effect diagram.
Example 2:
Figure 73 illustrates the resulting cause and effect diagram after the brainstorming session on
identifying problems in receiving telephone messages. The brainstorming effort for this problem is
covered in section 7.7.4.
7.2.5 Advantages
(1) The cause and effect diagram enables quality analysis groups to thoroughly examine all
possible causes or categories.
(2) The cause and effect diagram is useful in analyzing SPC problems. SPC detects a problem
but can pose no solution.
78
79
ENVIRONMENT
TRAINING METHOD
HUMAN ERROR HARDWARE
Message light not turned on
Criticality of message not identified (no guidelines)
Procedures
Messages are
notdelivered in a
timely manner
Message Taker responsibilities
Phone System Options
Inability to take long detailed message
No standard guidelines
for message takers
long detailed messages
Inadequate message delivery system
Employee
Unaware
ofmessage
No feedback of message
delivered
call pickup
No guideline for phone
system setup
Info not available to call recipient
Recipient doesn't know how
to obtain info employee
whereabouts
No guidelines for message takers
call transfer
call coverage
Employee forgets to sign out
Forget to undo call forward
Call recipient does not deliver message
distribution message misplaced
Lack of equipment to take long
detailed/technical messages
Not enough phones
Not enough trunk
lines
Lack of
interactiveautomated
directions to caller
Employee does not see light
Too small Poor contrast
Wrong message taken  incomplete message
rude caller
distractions
Employee fails to look at light
No identified
point of contact
No method to
reachemployee
notaccessible while off
i
Untimely delivery of message
Peak Activity
Number of calls
Figure 73. Cause and effect diagram on receiving telephone messages.
7.2.6 Limitations
The development of the cause and effect diagram can be timeconsuming in order to arrive at a
group consensus.
7.2.7 Bibliography
Kume, H.: “Statistical Methods for Quality Improvement.” The Association for Overseas Technical
Scholarships, 1985.
7.3 Concurrent Engineering
7.3.1 Description
Concurrent engineering is the interaction of technical disciplines during the design phase to
produce a robust design prior to production. This process is more of an engineering approach to quality
management than a technique.
7.1
The approach attempts to link and integrate, from the outset, all
elements of the product life cycle from conception through disposal.
Traditionally, quality, and producibility do not review an element until after the design has been
completed. Concurrent engineering, as described in reference 7.3, focuses on both the product and the
process simultaneously. One method of achieving this approach is by forming multifunction teams
consisting of engineers from several departments. This way, each department will follow the complete
process simultaneously rather than one department examining the design and then passing it on to the
next department and so on.
7.4
The concurrent engineering approach has been known for many years although its use is just
receiving widespread application in the United States.
7.5
7.3.2 Application
Because the concurrent engineering approach is used to address the product and process simulta
neously early in the design phase, it generally will save time and money. Through this technique, the
team will establish design goals as well as perform tradeoff analyses using such tools as QFD (sec.
7.12) and DOE (sec. 7.5). This technique is typically performed in phase C but may also be performed in
phase B.
7.3.3 Procedures
The basic elements involved in applying concurrent engineering include the following, as
adapted from reference 7.3:
(1) Establish multifunction teams which include members from design, quality, safety,
marketing, manufacturing, support, etc.
(2) Select and use design parameters that will help identify and reduce variability in the
production process.
(3) Use such techniques as DOE, QFD, computeraided design, robust design, group
technology, and value analysis to extend the traditional design approach.
710
7.3.4 Example
Figure 74 illustrates an example of how concurrent engineering is applied. By using
multifunctional teams, all phases of a product’s life cycle are simultaneously examined, thus making the
design process more efficient in terms of both cost and schedule.
INPUT
Cust omer
Needs
Development
Maint ainabilit y
Rel i abi l i t y
Saf et y
Verif icat ion
Logist ics
Manuf act uring
Training
Deployment
Operat ions
Support
Disposal
MULTIFUNCTIONAL
TEAMS
Sample Techniques
Cause and Ef f ect Diagram
( Sect ion 7.2)
Design of Experiment s, DOE
( Sect ion 7.5)
Brainst orming
( Sect ion 7.7)
Delphi Technique ( Sect ion 7.9)
Nominal Group Technique, NGT ( Sect ion 7.10)
Force Field Analysis ( Sect ion 7.11)
Qualit y Funct ion Deployment , QFD( Sect ion 7.12)
St at ist ical Process Cont rol, SPC ( Sect ion 7.14)
OUTPUT
BALANCED
PRODUCT
LIFE
CYCLE
Figure 74. Concurrent engineering example.
711
7.3.5 Advantages
(1) The concurrent engineering approach can be used to shorten and make more efficient the
designtodevelopment life cycle by employing the interactions of functional disciplines by
a crossfunctional team.
(2) The approach can also be applied to reduce costs in the designtodevelopment life cycle.
7.3.6 Limitations
(1) The degree of success of this technique depends upon the degree of cooperation between
the multifunctional team members.
(2) Significant additional time, and associated funding, is required at the front end of a
program to perform the coordinated planning. While time and money are saved overall
within the effort, it is often difficult to “frontload” large tasks.
(3) If design is pursued by projectized teams, the institutional knowledge of the organization
becomes very difficult to capture or employ in the design decisions.
7.4 Cost of Quality
7.4.1 Description
As described in reference 7.3, the cost of quality technique tracks the expense and benefit of a
quality program. This technique can identify the unwanted cost of not doing the job right the first time as
well as the cost of improving the job.
Cost of quality includes all of the costs associated with maintaining an acceptable quality program,
as well as the costs incurred as a result of failure to reach the acceptable quality level. This technique
allows the analyst to identify costs that are often hidden. Costs will not be reduced by merely tracking the
cost of quality but the technique may point out areas where a greater return on investment could be made.
7.4.2 Application
The cost of quality technique is best applied in phase E. This technique is applied to understand
the hidden costs of a product or service and to reduce or eliminate these costs. This technique can
identify the most significant costs and thus make it possible to prioritize the activities and/or processes
that may need improvement.
7.4.3 Procedures
The cost of quality technique is applied in the following manner:
(1) Collect cost data for the following categories;
a. Internal failure (IF) costs
b. External failure (EF) costs
c. Appraisal (A) costs
d. Prevention (P) costs
712
(2) Data are trended periodically on the standard cost of quality curve shown in figure 75:
TIME
$
A, P
IF, EF
Figure 75. Standard cost of quality curve.
As appraisal (reactive) and prevention efforts increase, failures decrease. A significant prevention effort
resulting in decreased failure warrants a decrease in appraisal (i.e., audits, inspections).
Prevention is the key. Concurrent engineering (sec. 7.3) helps achieve prevention. In some companies,
the suggestion system and/or savings shown in process improvement measures are considered
prevention.
Cost of quality programs requires a crossfunctional, interdepartment team to agree on what constitutes a
cost. Programs normally consist of three phases:
(1) Initiation.
(2) Development.
(3) Solidified gains.
Failures are indirectly proportional to the appraisals/preventions. As failures decrease, manpower
(reactive) should be decreased. Prevention costs run 2 percent or less of sales as a national average.
There are indications that, to optimize costbenefit relationships, it should be 10 percent. As the program
progresses, prevention costs (proactive) should increase.
Collection of data can be on a ROM basis and need not involve finances. Be careful not to create a
system and become so enamored with the system that the objective of savings is obscured.
Once data are collected and analyzed, they should be compared to a base. Examples are:
(1) Manhours per drawing.
(2) Direct cost per hour.
(3) Drawings per month.
713
7.4.4 Example
An example of a cost of quality data summary for a month is shown in table 72.
Table 72. Month’s cost of quality.
Cost Subj ect P A IF EF
32,000 Dr awi ng Er r or s X
2,000 Tr ai ni ng X
78,000
Erroneous
X
Inf or mat i on
18,000 War r ant y Claims X
10,000 Inspect ion/ Audit s
X
140,000 2,000 10,000 110,000 18,000
($)
The percentage breakdown is:
Prevention = 2,000/140,000 = 1.43 percent
Appraisal = 10,000/140,000 = 7.14 percent
Internal failure = 110,000/140,000 = 78.57 percent
External failure = 18,000/140,000 = 12.86 percent
100 percent
The total failure cost is $128,000 with only $2,000 spent on prevention. This example is 98.57
percent reactive and only 1.43 percent proactive.
7.4.5 Advantages
The following advantages were adapted from reference 7.6:
(1) The cost of quality technique helps to reveal and explain the more significant costs.
(2) Because of increased demands for time, energy, and money, it is helpful to develop a
quality technique whereby activities and processes that need improvement can be
prioritized. The cost of quality technique will accomplish this.
(3) The technique helps to reveal and explain the hidden costs of a product or service.
714
7.4.6 Limitations
(1) If not done as part of an overall plan, the cost of quality technique can be expensive, thus
making the goals of saving/eliminating costs unachievable.
(2) Measurement for measurement’s sake is an easy paradigm to fall into. This technique is
subject to misuse in this regard.
7.5 Design of Experiments
7.5.1 Description
The DOE technique is a control method of selecting factors, and levels of factors, in a predeter
mined way and varying possible combinations of these factors and levels. Quantitative results are
analyzed to show interactions and optimum settings of factors/levels to produce a desired output.
This technique may make the designtoproduction transition more efficient by optimizing the
product and process design, reducing costs, stabilizing production processes, and desensitizing
production variables.
7.3
7.5.2 Application
The design of experiments technique is typically performed in phase C but may also be
performed in phase D. This technique is used to achieve a robust design as an alternative to
experimenting in the production mode after the design has been completed. As described in reference
7.3, the following are among the applications for the DOE analysis:
(1) Compare two machines or methodologies.
(2) Examine the relative effects of various process variables.
(3) Determine the optimum values for process variables.
(4) Investigate errors in measurement systems.
(5) Determine design tolerances.
7.5.3 Procedures
As described in reference 7.3, the DOE technique is implemented as follows:
(1) Determine all of the pertinent variables whether they be product or process parameters,
material or components from suppliers, or environmental or measuring equipment factors.
(2) Separate the important variables which typically number no more than four.
(3) Reduce the variation on the important variables (including the control of interaction effects)
through redesign, close tolerance design, supplier process improvement, etc.
(4) Increase the tolerances on the less important variables to reduce costs.
715
7.5.4 Example
Data (yield in pounds) were recorded in table 73. For example, when A was at the low (A
1
)
level (10 °F), B was at the high (B
2
) level (60 psi), and C was at the low (C
1
) level (30 GPM), yield was
2.1 lbs.
Table 73. 2
3
factorial design data.
A
1
A
2
B
1
B
2
B
1
B
2
C
1
(1)
8.0
(7)
2.1
(6)
8.4
(4)
2.8
C
2
(5)
9.9
(3)
3.2
(2)
8.8
(8)
3.0
Numbers in parenthesis are standard cell designators. Normally four readings are averaged (e.g., 8.0 at
A
1
, B
1
, and C
1
, is an average of four data).
The orthogonal array is shown in table 74 along with the result of table 73. This array is used as
a “run recipe” in the actual conduct of the experiment. For example, all factors (A, B, C) are set at their
low level during trial 1.
Table 74. Trial, effects, and results.
Trial Main Effects SecondOrder Effects ThirdOrder
Effects
Results
A B C AB AC BC ABC
1 – – – + + + – 8.0
2 + – – – – + + 8.4
3 – + – – + – + 2.1
4 + + – + – – – 2.8
5 – – + + – – + 9.9
6 + – + – + – – 8.8
7 – + + – – + – 3.2
8 + + + + + + + 3.0
An example of the average of first order or main effects is shown using A
1
data and cells 1, 3, 5,
7; thus:
A
1 effects
=
8.0 + 3.2 + 9.9 + 2.1
4
· 5.80.
An example of a second order interaction (e.g., AB) is calculated by averaging data in the cells
where A and B are at like (L) levels and unlike (U) levels. They are:
AB
L
= cells 1,5, 4,8 ·
8.0 + 9.9 + 2.8 + 3.0
4
· 5.93.
716
AB
U
· cells 7,3,6,2 · 2.1+ 3.2 + 8.4 + 8.8
4
· 5.63.
An example of the third order interaction (i.e., ABC) is calculated using cell data where the
sum of the ABC subscripts are odd (O), then even (E). They are:
In cell #1, the factor levels are: A’s level is 1, B’s level is 1, and C’s level is 1. Therefore,
1+1+1 = 3, which is an odd number. The four cells having odd sums of levels are 1, 2, 3, 4.
In cell #5, the factor levels are: A’s level is 1, B’s level is 1, and C’s level is 2. Therefore,
1+1+2 = 4, which is an even number. The four cells having even sums of levels are 5, 6, 7, 8.
The calculations for all factors/levels are shown in table 75.
Table 75. Calculation of effects.
Summation Cells Computation Effect
A
1
1, 3, 5, 7 (8.0+3.2+9.9+2.1)/4 5.80
A
2
2, 4, 6, 8 (8.8+2.8+8.4+3.0)/4 5.75
B
1
1, 2, 5, 6 (8.0+8.8+9.9+8.4)/4 8.78
B
2
3, 4, 7, 8 (3.2+2.8+2.1+3.0)/4 2.78
C
1
1, 4, 6, 7 (8.0+2.8+8.4+2.1)/4 5.33
C
2
2, 3, 5, 8 (8.8+3.2+9.9+3.0)/4 6.23
AB
L
1, 4, 5, 8 (8.0+9.9+2.0+3.0)/4 5.725
AB
U
2, 3, 6, 7 (8.8+3.2+8.4+2.1)/4 5.63
AC
L
1, 2, 7, 8 (8.0+8.8+2.1+3.0)/4 5.48
AC
U
3, 4, 5, 6 (3.2+2.8+9.9+8.4)/4 6.08
BC
L
1, 3, 6, 8 (8.0+3.2+8.4+3.0)/4 5.65
BC
U
2, 4, 5, 7 (8.8+2.8+9.9+2.1)/4 5.90
ABC
O
1, 2, 3, 4 (8.0+8.8+3.2+2.8)/4 5.70
ABC
E
5, 6, 7, 8 (9.9+8.4+2.1+3.0)/4 5.85
Steps:
(1) Find C
Avg
:
This is the overall average of all data in all cells or,
C
Avg
=
8+9.9+2.1+3.2+8.4+8.8+2.8+3
8
· 5.78.
(2) Find an estimate of σ
c
;
Estimated σ
c
= (C
Avg)
1/2
/(4)
1/2
= (5.78)
1/2
/2 = 1.202.
717
(3) Ott
7.7
uses upper decision lines (UDL) and lower decision lines (LDL) instead of 3σ control
limits. The reason is that a decision of significant effects must be made when the plotted
data are beyond these lines. Ott also has a table called “exact factors for oneway analysis
of means, H
α
twosided.” H
.05
is found in the table. Then calculate the 95 percent UDL and
LDL, where α = .05, as follows:
UDL = C
Avg
+H
.05
(Estimated
c
) = 5.78+(1.39×1.188) = 7.43
LDL = C
Avg
–H
.05
(Estimated
c
) = 5.78–(1.39×1.188) = 4.13.
(4) The data from table 75, C
Avg
, UDL, and LDL are graphed in figure 76.
Yield
(pounds)
Effect
1
2
3
4
5
6
7
8
9
10
A
1
A
2
B
1
B
2
C
1
C
2
A
L
B A B A C
L U
A C B C
L
B C
U o
A B C ABC
E U
UDL=7.43
LDL=4.13
5.80
5.75
8.78
2.78
5.33
6.23
5.72
5.63
5.48
6.08
5.65
5.90
5.70
5.85
Figure 76. Factor/level effects graph.
Conclusion:
The main effect of B is very significant. Going from the high to the low level decreased yield 5
lbs. Raise B from 20 to 40 psi and run another experiment.
7.5.5 Advantages
This technique makes the designtoproduction transition more efficient by optimizing the product and
process design, reducing costs, stabilizing production processes, and desensitizing production
variables.
7.3
7.5.6 Limitations
(1) The performance of the analysis is timeconsuming and, if less than full factorial arrays are
employed, the results will not include all parametric interactions. Preknowledge of
interaction significance is required to support appropriate DOE technique selection.
718
(2) The DOE technique is often performed without a “verification experiment,” in which the
predicted “optimized” parameters are tested for performance (in agreement with the
predictions). In addition, a mistake is often made by taking the “best” experiment’s
parameters as an optimized set rather than an interpolated set.
(3) In order to perform the analysis, parameters must be interpolated from within the tested
data set rather than extrapolated beyond it.
7.5.7 Bibliography
Bhole, K.R.: “World Class Quality.” American Management Association, 1991.
7.6 Evolutionary Operation
7.6.1 Description
The EVOP technique is based on the idea that the production process reveals information on how
to improve the quality of a process. The technique has a minimal disruption to a process and creates
variation to produce data for analysis. Optimum control factor settings are identified for desired results.
Small, planned changes in the operating conditions are made and the results are analyzed. When
a direction for improvement is identified, process modifications can be made. The changes can continue
to be made until the rate of finding improvements decreases and then the changes can be applied to
different operating variables to identify more directions for improvement.
7.8
7.6.2 Application
The EVOP technique is best performed in phase E but may also be performed in phase D. This
technique is applied to reveal ways to improve a process. An experiment may use two or more control
factors (i.e., psi and degrees F are set) that produce a response (yield) known as response surface
methodology (RSM) (sec. 6.7). The question that may be asked is, “What are the degrees F and psi
settings that will produce maximum yield (pounds per batch)?”
Evolutionary operation works well with the SPC technique (sec. 7.14) in that SPC will monitor a
process and EVOP will reveal ways to improve the process.
7.6.3 Procedures
The EVOP technique is applied in the following manner:
(1) Choose two or three variables that are likely to affect quality.
(2) Make small changes to these variables according to a predetermined plan.
(3) Analyze the results and identify directions for improvement.
(4) Repeat until optimal conditions are found.
(5) The technique can then be applied to different variables.
719
7.6.4 Example
100
200
300
400
(Factor B)
F
20 30 40 50
60
PSI
(Factor A)
1 (B) 3 (D)
(Yield)
4 (E) 2 (C)
0 (A)
8 6
5 7
Cycle
#2
Cycle
#1
0
Figure 77. EVOP example.
Cycle No 1:
Per figure 77 above, select a reference point “0” (center of the box). The aim is to choose the psi
and degrees F that yield maximum output (body of the graph). Output (yield) can be volume, length, etc.
Corner No. 2 was maximum. Cycle No. 2 uses that corner as the reference point for the second box
(cycle). Actually, this is a simple 2
2
factorial experiment where the low and high levels of two factors,
i.e., degrees F and PSI were selected. Data for this example are shown in table 76.
Table 76. EVOP cycle No. 1 data.
RUN TIME ( A) TEMPERATURE ( B) POSITION YIELD
1
2
3
4

+

+


+
+
1
3
4
2
0
20
30
40
50
10
Legend:
" " = Low Level " +" = High Level
720
Main effects are calculated for A and B and second order interaction AB as follows:
A
EFFECT
(∑ High Levels–∑ Low Levels)/2
= [(30+50)–(20+40)]/2 = 10
B
EFFECT
= (∑ High Levels–∑ Low Levels)/2
= [(40+50)–(20+30)]/2 = 20
AB
INTERACTION
= (yield when A and B have like signs – yield when A
and B have unlike signs)/2
= [(20+50)–(30+40)]/2 = 0.
The change in mean (CIM) and 2 standard error (S.E.) cannot be calculated until two cycles are
complete. The S.E. is really a variation and encompasses 95percent confidence within the normal curve.
The 95percent is symmetrical with a 5percent level of significance, or a left and right tail of 2
1
/2
percent each. The CIM tells when a minimum or maximum occurs by comparing the results of the four
box corners to the reference point.
Cycle No. 2:
Corner No. 2 produced a maximum yield (i.e., 50) and becomes the new reference point. New
data were recorded as shown in table 77.
Table 77. EVOP cycle No. 2 data.
RUN TIME ( A) TEMPERATURE ( B) POSITION YIELD
1
2
3
4

+

+


+
+
5
7
8
6
0
26
32
38
48
18
Now, compare cycles (table 78).
721
Table 78. Comparison of EVOP cycle No. 1 and cycle No. 2 data.
CORNER SUBJECT
YIELD AT POSITION
0 6 7 8
A
B
C
D
E
F
Sum From Cycle No.1
Average From Cycle No.1
New Yield Dat a
B  C
New Sum = B + C
New Average = E/ n
10 20 30 40 50
10 20 30 40 50
18 26 32 38 48
 8  6  2 2 2
28 46 62 78 98
14 23 31 39 49
5
The new averages are used to calculate results. The levels of factors are determined by
examining the cycle No. 2 box of figure 77. For example, when A is at the high level, use corners 6 and
7. When A is high and B is low, use corner 7, etc.
A
EFFECT
= [(31+39)–(23+39)]/2 =4
B
EFFECT
= [(31+49)–(23+39)]/2 = 9
AB
INTERACTION
= [(23+31)–(39+49)]/2 = –17.
The CIM is calculated by multiplying the reference point data by 4 (now representative of four
corners) and letting the product be a sample, i.e., n = 1. The product is subtracted from the sum of the
four corners and divided by 5 (i.e., four corners are n = 4 + the reference point of n = 1):
23+31+39+49 = 142
4×14 = 56
86/5 = 17.2
The standard deviation and 2 S.E. when n = 2 are calculated using standard factors developed by
Box and Hunter.
7.9
They are K = 0.3, L = 1.41, and M = 1.26.
For the sample standard deviation:
s = K (corner “d” range)
s = 0.3 (–8 to +2) = 3
for 2 S.E. For new averages/effects:
L(s) = 1.41×3 = 4.23.
722
For CIM
M(s) = 1.26×3 = 3.78.
Results:
Psi limits are 4 t 4.23 = –0.23, 8.23
Temperature limits are 9 t 4.23 = 4.77, 13.23.
Conclusion:
Since the AB interaction = –17, there is a significant impact on the maximum yield. The psi can
be negative, positive, or nil. The temperature is borderline, but it should increase yield if it is decreased.
Select corner No. 7 and run a third cycle.
7.6.5 Advantages
The following advantages are adapted from reference 7.8:
(1) The cost of running EVOP is very low so it can be run continuously.
(2) EVOP will increase a plant’s capacity and thus profits will also increase.
(3) EVOP is simple and relatively straightforward.
7.6.6 Limitations
As described in reference 7.8, EVOP is slow, so progress is slow. If quick improvements are
needed, then this technique is inappropriate.
7.7 Brainstorming
7.7.1 Description
Brainstorming, as described in reference 7.3, is a group process wherein individuals quickly
generate ideas on a particular problem, free from criticism. The emphasis is on the quantity of ideas, not
the quality. In the end, the goal is to arrive at a proposed solution by group consensus. All members of
the group are equals and each is free to express ideas openly. The technique is an excellent way of
bringing out the creative thinking from a group.
7.7.2 Application
Brainstorming, as described in reference 7.1, is often used in business for such things as arriving at
compromises during union negotiations, coming up with advertising slogans, identifying root causes of a
problem, and finding solutions to a customer service problem.
If done properly, bashful yet creative people can be coaxed to propose good ideas. For some important
brainstorming sessions, a facilitator is necessary. The facilitator should be knowledgeable in the
brainstorming process and help as much as possible in the generation of ideas but should have no stake
in the outcome of the brainstorming session. This technique is typically performed in phase A but may
also be performed in phase C.
723
There are three phases of brainstorming, as adapted from reference 7.3:
(1) Generation phase—group members generate a list of ideas.
(2) Clarification phase—the group reviews the list of ideas to make sure all members
understand each one, discussions occur.
(3) Evaluation phase—the group eliminates duplication, irrelevancies, or issues that are off
limits.
7.7.3 Procedures
As described in reference 7.3, conduct a brainstorming session as follows:
(1) Clearly state the purpose of the brainstorming session.
(2) Group members can take turns expressing ideas, or a spontaneous discussion can occur.
(3) Discuss one topic at a time.
(4) Do not criticize ideas.
(5) Expand on ideas from others.
(6) Make the entire list of ideas available for all group members to review.
(7) After discussions and eliminations, arrive at a final proposed solution by group consensus.
7.7.4 Example
A group was assembled to brainstorm the causes for telephone messages not being received in a
timely manner. Each group member was given an opportunity to express ideas on the subject. A
spontaneous discussion developed, with some group members expanding on the ideas of others. The
following is a list of possible causes for the telephone message problem as a result of the brainstorming
session:
(1) Employee not at desk
(2) Secretary not available
(3) Volume of calls inhouse
(4) Too many incoming calls to receptionist
(5) Employee misses message
(6) Employee doesn’t see light or message
(7) Incomplete message taking
(8) Message mishandled
(9) Nonstandard message delivery system
(10) Employee offsite
(11) Criticality of message not identified
724
(12) Phone grouping not identified
(13) Whereabouts of employee unknown by call recipient
(14) Not utilizing available resources
(15) Caller leaves no message
(16) Message light not turned on
(17) Inadequate phone system
(18) No feedback of message delivered
(19) Lack of procedures
(20) No identified points of contact
(21) No answering machines
(22) Complicated phone system
(23) Forgetting to undo callforwarding
(24) People do not know how to use phone options
(25) Secretary does not deliver messages
(26) Secretary not in loop
(27) Cannot find known message in loop
(28) Wrong message taken
(29) Untimely delivery of message
(30) No guidelines for message taking
(31) Not enough phones
(32) Not enough trunk lines
(33) Volume of calls
(34) Congestion at receptionist’s desk
(35) Discontinuity at receptionist’s desk
(36) No beepers.
Following the brainstorming session for the causes of the problem, a cause and effect diagram
was developed as shown in section 7.2.4, example 2. Once this was completed and more discussions
were held, a proposed solution to the problem was presented.
7.7.5 Advantages
The technique takes advantage of the ideas of a group to arrive at a quick consensus.
7.7.6 Limitations
(1) Brainstorming only proposes a solution but does not determine one.
(2) The technique is limited by the ability of the group to achieve consensus.
725
7.8 Checklists
7.8.1 Description
A checklist, as described in reference 7.3, provides a list of checkoff items that enable data to be
collected quickly and easily in a simplified manner. The data are entered on a clear, orderly form. Proper
use of the checklist helps to minimize errors and confusion.
7.8.2 Application
Checklists should be laid out in advance or data may be omitted. If done right, the checklist will
be easy to complete and will allow for quick entry of data. One common method of data entry on a
checklist is hash marking.
Checklists are often used to collect data on such things as numbers of defective items, defect
locations, and defect causes. This technique is best applied in phase E but may also be applied in phase
A or B.
7.8.3 Procedures
As adapted from reference 7.3, a checklist is created in the following manner:
(1) A group should decide ahead of time what data should be collected.
(2) Make a draft of the checklist and ask the individuals who will fill out the form for input—
revise as necessary.
(3) Implement the checklist.
(4) As data are collected, review the results and, again, revise the checklist, as necessary, to
optimize use of the form.
7.8.4 Example
Table 79 illustrates a sample of the results of postflight hardware inspections for an imaginary
SRM. The listed defects occurred on the corresponding motor where checked.
7.8.5 Advantages
(1) The checklist is quick and easy to use.
(2) Checklists help to minimize errors and confusion.
7.8.6 Limitations
Time must be taken to assemble a group to decide what data should be collected.
726
Table 79. Motor postflight checklist.
Outer Igniter Joint
Discoloration
Aft Edge GEI
Insulation Chips
Water Under
Moisture Seal
Polysulfide
Porosity
Wet Soot on
Rubber
Edge
Insulation Exposure
Inhibitor Erosion
Defect
Description
Motor Number
01 02 03 04 05 06 07 08 09 10
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
7.9 Delphi Technique
7.9.1 Description
The Delphi technique, as described in reference 7.1, is an iterative process that results in a consensus by
a group of experts. The subject is presented to the experts. Without discussing the subject among
themselves, the experts send their comments to a facilitator. The facilitator reviews the comments and
eliminates those not applicable to the subject. Then, the comments are redistributed to the experts for
further review. This iteration is repeated until a consensus is reached.
7.9.2 Application
The Delphi technique is best performed in phase A or B but may also be performed in phase E. This
technique is a useful tool for finding a solution when personality differences exist between members of
involved technical areas. A group of experts can examine the problem and, through consensus, the
effects of the differences can be minimized. Another application for this technique is to allow all parties
to have equal input when one personality may otherwise overpower another in a discussion.
7.9.3 Procedures
As adapted from reference 7.1, the Delphi technique is applied in the following manner:
(1) Define the subject upon which the experts are to comment.
(2) Assemble a monitor group to determine task objectives, develop questionnaires, tabulate
results, etc.
(3) Choose the experts, making sure they have no vested interest in the outcome.
(4) Distribute the objectives, questionnaires, etc. to the experts for their initial set of opinions.
727
(5) The monitor team consolidates the opinions and redistributes the comments to the experts,
making sure that the comments remain anonymous.
(6) Repeat steps 4 and 5 until a group consensus is reached.
7.9.4 Example
The following example was adapted from reference 7.10:
A fault tree was generated for an SRM igniter, documenting all conceivable failure modes
associated with the subsystems. A sample of part of the fault tree is shown in figure 78. The fault tree
was then distributed to technical experts in the solid rocket industry. The expertise represented SRM
experience in design, structures, and processing. These experts were asked to assign subjective
estimations of failure probabilities of each mode and cause.
Igniter case
external insulation
fails to protect the
closure and
retaining ring 
structural, erosion
Inferior /
nonspec
insulation
Improper
material thickness
or geometry
Material
aging
degradation
Formulation
error
Improper
molding
Damage
to material or
threads due to
processing,
handling.
Poor
bonding of
material at adapter
Figure 78. Sample of a partial igniter subsystem fault tree.
The relative probabilities were based on a rating system which utilized a tailored version of
MIL–STD–882C
7.11
(sec. 3.12). The experts used letters to correspond to the descriptive words as
follows:
Level Descriptive Words Probability
A Infrequent 0.1
B Remote 0.01
C Improbable 0.001
D Very improbable 0.0001
E Almost nil 0.00001
728
Figure 79 shows an example of how the technical experts assigned estimations to each failure
level.
C E C
D
D D B B
Igniter case
external insulation
fails to protect the
closure and
retaining ring 
structural, erosion
Inferior /
nonspec
insulation
Improper
material thickness
or geometry
Material
aging
degradation
Formulation
error
Improper
molding
Damage
to material or
threads due to
processing,
handling.
Poor
bonding of
material at adapter
Figure 79. Fault tree sample with estimates assigned.
The team that generated the fault tree then took all the responses and assigned each failure level a failure
probability based on the letters assigned by the experts. An average was derived for each failure level
and applied to the fault tree. This labeled fault tree was distributed to the technical experts.
This process was repeated until a consensus on the assigned failure probabilities was arrived at by all of
the technical experts.
7.9.5 Advantages
(1) This technique can be useful in eliminating personality clashes.
(2) This technique can be useful when powerful personalities are likely to dominate the
discussion.
(3) Inputs from experts unavailable for a single meeting are included.
7.9.6 Limitations
(1) Arriving at a group consensus is timeconsuming.
(2) Assembling the group participants is difficult/timeconsuming.
729
7.10 Nominal Group Technique
7.10.1 Description
The NGT, as described in reference 7.1, is another tool used to reach a group consensus. When
priorities or rankings must be established, this decisionmaking process can be used. NGT is similar to
brainstorming (sec. 7.7) and the Delphi technique (sec. 7.9), but it is a structured approach that is
oriented toward more specialized problems. The group should be small (i.e., only 10 to 15 people), and
every member of the group is required to participate. This technique is often categorized as a silent
brainstorming session with a decision analysis process.
7.10.2 Application
The nominal group technique is an effective tool for producing many ideas and/or solutions in a
short time. The technique can be used for many of the same applications as brainstorming and the Delphi
technique. The NGT is best applied in phase A or B but may also be applied in phase E. Company
internal technical problems can be solved, personality clashes can be overcome, and NGT can be used to
develop new ideas to satisfy a particular problem.
7.3
7.10.3 Procedures
The NGT, as adapted from reference 7.1, is applied in the following manner:
(1) Generate the idea for discussion—a facilitator presents the problem and instructions to the
team.
(2) The team quietly generates ideas for 5 to 15 min—no discussion is allowed and no one
leaves until everyone is finished.
(3) The facilitator gathers the ideas roundrobin and posts them in no particular order on a flip
chart.
(4) The ideas are then discussed by the group; no arguments, just clarifications. Duplications
are eliminated.
(5) Each member of the group silently sets priorities on the ideas.
(6) The group votes to establish the priority or rank of each idea.
(7) The votes are tabulated and an action plan is developed.
7.10.4 Example
The following example was adapted from reference 7.12:
The overall objective of this task was to define an appropriate methodology for effective
prioritization of technology efforts required to develop replacement technologies (chemicals) mandated
by imposed and forecast legislation.
The methodology used was a semiquantitative approach derived from QFD techniques (sec.
7.12). This methodology aimed to weight the full environmental, cost, safety, reliability, and
programmatic implications of replacement technology development to an appropriate identification of
viable candidates and programmatic alternatives.
730
A list of concerns that needed to be addressed was developed as follows in table 710.
Table 710. Replacement technology concerns.
Chemical Concerns Environmental Concerns
(1) Number of sources (1) Clean air monitoring
(2) Limits of resources (2) Pollution prevention
(3) Availability (3) Toxic emissions
(4) Stability (4) Emissions control
(5) Drying ability (5) Ozone depletion potential
(6) Base material compatibility (6) Chemical storage availability
(7) Toxicity (7) Resource/ingredient recovery and
(8) Flash Point recycling
(9) Ease of maintenance (8) Hazardous waste management
(10) Historical data base
(11) Desirable reactivity Cost Concerns
(12) Undesirable reactivity (1) Manpower dollars
(13) Lottolot variability (2) Operations dollars
(14) Age sensitivity (3) Facilities dollars
(15) Shelf life (4) Materials dollars
(16) Bondline thickness (5) Chemical dollars
(6) Other hardware dollars
Process Concerns (7) Contracts dollars
(1) Contaminants removed (8) Change of specifications dollars
(2) Process steps (9) Specification verification dollars
(3) Parts processed at one time (10) Change of drawings dollars
(4) Required surface preparation (11) Development of procedure dollars
(5) Bondline thickness (12) Emissions control equipment dollars
(6) Process interaction (13) Emissions control testing dollars
(7) Bondline strength required
(8) Operator sensitivity Scheduling Concerns
(9) Lottolot variability Federal, State,and Local
(10) General cleaning ability (1) Research
(11) Surface requirements (2) Trade studies
(12) Possibility of stress corrosion cracking (3) Modification in planning
(13) Useful life of process part (4) Specification documentation
(14) Damage caused by process (5) Requirements documentation
(6) Drawing/design changes
Regulatory Concerns (7) Production time
(1) OSHA requirements (8) Testing
(2) State environmental laws (9) Vendor selection and certification
(3) Local environmental laws
(4) Federal environmental requirements Present Program Schedule
(5) Future federal regulations (1) Research
(2) Trade studies
Safety Concerns (3) Modification in planning
(1) Worker exposure limits (4) Specification documentation
(2) Spill response plans (5) Requirements documentation
(3) Fire response plans (6) Drawing/design changes
(4) Explosion response plans (7) Production time
(8) Testing
(9) Vendor selection and certification
731
A necessary step for developing a QFD matrix was to assign weighting factors to all of the
concerns. A group of 10–15 people knowledgeable in the subject of chemical replacement was
assembled to weight the concerns as follows:
(1) Each individual of the group, without discussion, generated ideas about the importance of
each concern.
(2) The facilitator collected the lists of ideas and posted them in no particular order.
(3) The ideas were discussed to clear up any misunderstandings.
(4) The group then voted on establishing the weighting factors on each concern.
Table 711 shows the list of assigned weighting factors based on a scale of 1 (least critical) to 20
(most critical).
7.10.5 Advantages
NGT is very effective in producing many new ideas/solutions in a short time.
7.10.6 Limitations
(1) Assembling the group participants is difficult/timeconsuming.
(2) Limiting discussion often limits full understanding of others ideas, with consequent
divergence of weighting factors as a likely result.
7.11 Force Field Analysis
7.11.1 Description
The force field analysis, as described in reference 7.1, is a technique that counts both the number
and magnitude of positive and negative forces that effect the results of a proposed solution or change in
process. The analysis of these positive and negative forces generally occurs after performing a
brainstorming session (sec. 7.7) or a cause and effect diagramming session (sec. 7.2).
This technique categorizes the identified forces as either positive or negative, and assigns a value
(weight) to each force. All positives and negatives are added and the more positive the total, the more
likely the proposed solution is the correct one. The more negative the total, the more likely the proposed
solution is not correct. A strategy is then developed to lessen the negative forces and enhance the
positive forces.
7.11.2 Application
The force field analysis is best applied in phase D or E. This analysis is often applied in
determining which proposed solution, among many, will meet the least resistance. The number of forces
should not be too high (i.e., < 20) or other more sophisticated approaches should be considered.
732
Table 711. Concerns with assigned weighting factors.
Chemical Concerns Environmental Concerns
(1) Number of sources 7 (1) Clean air monitoring 12
(2) Limits of resources 7 (2) Pollution prevention 12
(3) Availability 14 (3) Toxic emissions 15
(4) Stability 15 (4) Emissions control 12
(5) Drying ability 14 (5) Ozone depletor potential 15
(6) Base material compatibility 17 (6) Chemical storage availability 10
(7) Toxicity 13 (7) Resource/ingredient recovery 10
(8) Flash point 13 and recycling
(9) Ease of maintenance 8 (8) Hazardous waste management 12
(10) Historical data base 9
(11) Desirable reactivity 13 Cost Concerns
(12) Undesirable reactivity 13 (1) Manpower dollars 17
(13) Lottolot variability 11 (2) Facilities dollars 15
(15) Shelf life 9 (4) Materials dollars 14
(16) Bondline thickness 7 (5) Chemical dollars 16
(6) Other hardware dollars 14
Process Concerns (7) Contracts dollars 12
(1) Contaminants removed 15 (8) Change of specifications dollars 13
(2) Process steps 9 (9) Specification verification dollars 13
(3) Parts processed at one time 7 (10) Change of drawings dollars 11
(4) Required surface preparation 12 (11) Development of procedure dollars 12
(5) Bondline thickness 7 (12) Emissions control equipment dollars 15
(6) Process interaction 9 (13) Emissions control testing dollars 12
(7) Bondline strength required 9
(8) Operator sensitivity 12 Scheduling Concerns
(9) Lottolot variability 11 Federal, State, and Local
(10) General cleaning ability 13 (1) Research 9
(11) Surface requirements 14 (2) Trade studies 8
(12) Possible stress corr. crack. 14 (3) Modification in planning 9
(13) Useful life of process part 14 (4) Specification documentation 10
(14) Damage caused by process 13 (5) Requirements documentation 11
(6) Drawing/design changes 8
Regulatory Concerns (7) Production time 11
(1) OSHA requirements 13 (8) Testing 14
(2) State environmental laws 14 (9) Vendor selection & certification 12
(3) Local environmental laws 14
(4) Federal env. requirements 15 Present Program Schedule
(5) Future federal regulations 14 (1) Research 10
(2) Trade studies 11
Safety Concerns (3) Modification in planning 10
(1) Worker exposure limits 12 (4) Specification documentation 11
(2) Spill response plans 13 (5) Requirements documentation 11
(3) Fire response plans 14 (6) Drawing/design changes 10
(4) Explosion response plans 16 (7) Production time 11
(8) Testing 12
(9) Vendor selection & certification 11
733
Application of the force field analysis requires a proposed solution and inputs to the process.
These inputs might come from using group consensus techniques like those discussed in earlier sections.
Also, assigning the value (weight) to each force might also require group consensus techniques.
7.11.3 Procedures
The force field analysis, as adapted from reference 7.1, is performed in the following manner:
(1) Identify the proposed solution or change in process.
(2) Determine the forces, positive and negative, that might effect the implementation of this
proposed solution.
(3) Separate the forces into positive and negative lists and assign a value (weight) to each
force. Arriving at these values may be achieved by use of a group consensus technique like
the Delphi technique (sec. 7.9).
(4) Establish a strategy to lessen the negative forces and enhance the positive forces.
7.11.4 Example
Management met to discuss the possibility of approving a suggestion to allow employees to work
flextime. The group identified the positive and negative forces that will affect this decision as follows:
Positive forces Negative forces
Employees welcome change Employee accessible to customer
8 8
Increased production Employees present to receive messages
9 6
Coordinate hours to improve personal life Management aware of employee's
schedule
9 4
Total : 26 Total : 18
Figure 710. Force field analysis example.
The positive forces clearly outweighed the negative forces. Management developed a strategy to
lessen the magnitudes of the negative forces listed and thus enabled the proposal of flextime to be
approved.
734
7.11.5 Advantages
The force field analysis is useful in determining which proposed solution, among many, will
meet the least resistance.
7.11.6 Limitations
This technique is timeconsuming in arriving at a consensus on the values (weights) of the forces,
and is highly subjective.
7.12 Quality Function Deployment
7.12.1 Description
QFD, as described in reference 7.12, is a conceptual map that provides the means for cross
functional planning and communications. This technique is a method of turning the customer’s voice
into engineering language. A matrix is developed known as the “house of quality” and the main
elements of the matrix are the WHATs (customer concerns) and the HOWs (quantifiable solutions to the
concerns). The reason for the name “house” is because the matrix is shaped like a house and elements
are separated into rooms, as illustrated in figure 711.
MATRIX
RELATIONSHIP
HOWs
CORRELATION
MATRIX
WHATs
BENCH
MARKS
ENGINEERING
PARAMETERS
Figure 711. House of quality.
The other rooms of the house are defined as follows:
(1) Relationship matrix—This is the main body of the matrix, and it is the relationship between
each WHAT and HOW. These relationships are denoted by symbols or numbers which
correspond to weak, medium, and strong relationships.
Example : 1 = Weak
3 = Medium
9 = Strong.
735
(2) Correlation matrix—This is often called the “roof” of the house. The roof relates each of
the HOWs to each other and is also denoted by symbols or numbers which correspond to
strongpositive, mediumpositive, strongnegative and mediumnegative.
Example: ++ = Strongpositive
+ = Mediumpositive
– – = Strongnegative
– = Mediumnegative.
These data become important when the consideration of tradeoff factors is necessary.
(3) Benchmarks—This room is used to assess how well the product stacks up against the
competition.
(4) Engineering parameters—This room is used to assess how well the product stacks up to
applicable target values.
Inputs to the QFD matrix will require group sessions which will involve brainstorming (sec. 7.7), cause
and effect analysis (sec. 7.2) and other techniques that might help to gather information about customer
requirements
7.1
7.12.2 Application
The QFD technique is typically performed in phase C but may also be performed in phase A or B.
This technique may be used by every function in the producing organization and in every stage of product
development. The main focus is to implement change during design rather than during production.
Not only does the QFD matrix allow assessment of the product against the competition and other
benchmarks, it also enables a prioritization of the HOWs, i.e., the results of the QFD analysis can give
overall ratings for each quantifiable solution to the stated concerns. These ratings indicate which solutions
are most important and need to be considered first. The most important reason for the QFD analysis is to
identify the problem areas and the quantifiable solutions to these problems early in the design phase so
these issues will not have to be faced during production, which could lead to delays and higher costs.
7.12.3 Procedures
As adapted from reference 7.13, a QFD analysis is performed as follows:
(1) List and prioritize the WHATs that concern the customer. These items are generally very
vague and require further definition. This list will be placed in rows at the left side of the
house. Each item is weighted for importance to the customer.
(2) List the HOWs that address the WHATs. This list of quantifiable solutions to the WHATs
will be placed in columns and because the WHATs are so vague, one or more HOWs can
relate to each WHAT.
(3) Correlate the WHATs and HOWs. This correlation is entered into the main body of the
matrix (relationship matrix). These relationships are weighted as noted in section 7.12.1.
(4) List the benchmarks and perform an assessment. The assessment can be performed on both
the HOWs and the WHATs. Areas for improvement can easily be noted here by comparing
how well this product stacks up against the competition.
736
(5) Correlate the HOWs to each other as noted in section 7.12.1, step 2.
(6) Calculate the scores of the relationships. The score for each HOW as related to each
WHAT is determined by multiplying the weighting factor for each WHAT by the
corresponding value in the relationship matrix. The overall ratings for the values in table 7
12 are calculated as follows:
Table 712. QFD matrix sample calculations.
Concern 1
Concern 2
Concern 3
Overall Rating
W
H
A
T
S
H O S W
Solution 1 Solution 2 Solution 3
3 9 1
1
9
3
9 1
9
153 237 163
10
15
12
Weighting Factors
Solution 1 would have an overall rating of (10×3)+(15×1)+(12×9) = 30+15+108 = 153.
Solution 2 would have an overall rating of (10×9)+(15×9)+(12×1)= 90+135+12 = 237.
Solution 3 would have an overall rating of (10×1)+(15×3)+(12×9)= 10+45+108 = 163.
This example reveals that solution 2 is the most important HOW in achieving the collective
WHATs.
7.12.4 Example
A planning team for an automobile company performed a task of trying to anticipate problem
areas in a design so they can be improved upon or eliminated early. Six customer concerns (WHATs) for
an automobile were studied:
(1) Good performance
(2) Quiet
(3) Safe
(4) Good gas mileage
(5) Affordable
(6) Roomy.
737
Next, all possible solutions to these concerns (HOWs) were identified and they are:
(1) 0–60 (s)
(2) Fuel economy (mpg)
(3) Horsepower
(4) Weight (klbs)
(5) Emissions (ppm)
(6) Noise level (dB)
(7) Energy absorption rate (mph)
(8) Purchase price (k$)
(9) Maintenance cost ($)
(10) Head room (in)
(11) Elbow room (in)
(12) Leg room (in).
This automobile company was benchmarked (sec. 7.1) against three competitors as to how well
each company stacks up to meeting each WHAT. The benchmark rating scale used was from 1 (low) to
5 (high).
Engineering parameters were identified for each HOW. The first parameter for each was the
desired parameter for this company to target. The next row delineated the current company practice for
each parameter. A final entry for these parameters, was the percent difference between the company’s
present level and the desired target.
The roof was included which identified the relationships between the HOWs. The rating scale
used was as follows:
9 = Strong positive
3 = Medium positive
–3 = Medium negative
–9 = Strong negative.
Finally, weighting factors were given to each customer concern. That is, on a scale of 1 (low) to
10 (high), each concern was rated for importance. All of the data were coordinated and a QFD matrix
was developed as shown in figure 712.
738
Figure 712. QFD example on auto industry.
739
Conclusions:
(1) Looking at the overall ratings showed that the two most important solutions in achieving
the collective concerns were the horsepower rating followed by the time taken to get from 0
to 60 mph.
(2) The benchmarking of this company to the three main competitors revealed that, overall,
this company rated as well or better than the competitors. The matrix showed that this
company could stand to improve on achieving a quiet ride, getting better gas mileage, and
making the automobiles roomier.
7.12.5 Advantages
(1) The QFD technique helps organizations design more competitive, higherquality, and
lowercost products easier and quicker, and is aimed primarily at the development of new
products.
(2) This technique helps ensure quality products and processes by detecting and solving
problems early.
(3) Engineering changes are reduced.
(4) The design cycle is reduced.
(5) Startup costs are reduced.
(6) The voice of the customer is heard.
(7) The technique is proactive instead of reactive.
(8) The technique prevents problems from “falling through the crack.”
(9) The technique is economical.
(10) The QFD technique is easy to learn.
7.12.6 Limitations
(1) Assembling the group participants is difficult/timeconsuming.
(2) Even though the analysis is easy to learn, it is not easy to perform.
7.12.7 Bibliography
Hauser, J.R. and Clausing, D.: “The House of Quality.” Harvard Business Review, No. 88307, vol. 66 ,
No. 3, May–June 1988.
Sullivan, L.P.: “Quality Function Deployment.” Quality Process, June 1986.
740
7.13 Quality Loss Function
7.13.1 Description
The quality loss function technique is a Taguchi method of determining the “loss to society”
when a product is not at the true value (i.e., mean) although it still lies within specification limits.
As described in reference 7.14, in order to develop a function to quantify the loss incurred by
failure to achieve the desired quality, the following characteristics must be considered:
(1) Larger is better (LIB)—the target is infinity.
(2) Nominal is best (NIB)—a characteristic with a specific target value.
(3) Smaller is better (SIB)—the ultimate target is zero.
Traditionally, manufacturers have considered a product “perfect” if it lies between the lower and
upper specification limits as illustrated in figure 713.
LOSS LOSS
PERFECT
LSL N
USL
Figure 713. Traditional view to meeting specification.
where
LSL = Lower specification limit
N = Nominal
USL = Upper specification limit.
The problem with this approach is that when “tolerance stackup” (sec. 4.3) is considered,
difficulties arise. If two mating parts are being manufactured, they may fall at opposite ends of their
specific tolerance and they may not assemble properly.
741
7.13.2 Application
The quality loss function technique is typically performed in phase E but may also be performed in
phase D. This technique is used to improve a process, thus it can be used for productivity improvement
measurements. For each quality characteristic there is a function which defines the relationship between
economic loss (dollars) and the deviation of the quality characteristic from the nominal value.
7.14
The application of the quality loss function L(y) also reveals indications of customer
dissatisfaction. The further the characteristic lies from the nominal value, the more problems may arise
and thus more customer complaints. These complaints, in turn, will lead to a financial loss.
Of course, just because a characteristic meets the target value, it does not mean that the quality of
the product is adequate. The specification limits may be out of line.
7.13.3 Procedures
As described in reference 7.14, the L(y) around the target value n is given by:
L(y) = k (y–n)
2
(7.1)
where
L(y) = loss in dollars per unit product when the quality characteristic is equal to y.
y = the value of the quality characteristic, i.e., length, width, concentration,
surface finish, flatness, etc.
n = target value of y.
k = a proportionality constant.
By applying equation (7.1) and examining figure 714, it can be seen that L(y) is a minimum at
y = n and L(y) increases as y deviates from n.
L(y)
($)
Ao
∆o
n
y
Figure 714. Quality loss function for NIB.
742
where
Ao = consumer’s loss
and
∆o = tolerance.
To apply the quality loss function equation, proceed as follows:
(1) As given in equation (7.2):
L(y) = k (y – n)
2
(2) To calculate a dollar loss at some value (y), first calculate k.
k =
Ao
∆o
. (7.2)
(3) Calculate L(y).
7.13.4 Example
Determine the dollars lost at some value (y) per figure 715.
L(y)
($)
Ao
∆o
y
500 =
80 100
y = 85
120
Figure 715. Quality loss function example.
L(y) = k (y – n)
2
k =
Ao
∆o
·
500
(20)
2
·
500
400
= 1.25
L(y) = 1.25 (85 – 100)
2
= 1.25 (–15)
2
= 1.25 (225) = $281.25.
743
7.13.5 Advantages
(1) The quality loss function technique is an excellent tool for evaluating loss at the earliest
stage of the product/process development.
(2) Useful results can be obtained quickly and at low cost.
7.13.6 Limitations
(1) With many manufacturers following the guidelines that their product is adequate if certain
measurements are within the specification limits, it is difficult to convince them to apply
this technique.
(2) It is often very difficult to calculate the quality loss function for a given process. The
parameter y and the relationship to any A
o
are generally obscure.
7.14 Statistical Process Control
7.14.1 Description
SPC is a method of using statistics applied to the results of a process to control the process.
Historical data of the performance of the process (or operation of hardware) are statistically analyzed to
predict future performance or to determine if a process is “in control.” A process is defined as “in
control” if there are only random sources of variation present in the process and the associated data. In
these cases, the data can correctly be investigated with the standard methods of statistical analysis. If the
data are not “in control,” there is some special cause of variation present in the process, and this is
reflected in the data from that process. In these cases, this section on SPC assumes that the data
variability is still reasonably distributed around the mean, and these procedures are applicable. If these
procedures lead to a result of special cause variation at nearly every data point, these procedures cannot
be correctly applied.
7.14.2 Application
The SPC technique is best performed in phase E. This technique is used to determine if special
causes of variation are present in a process, or if all variation is random. In other words, SPC is used to
ensure that a product is being produced consistently, or is about to become inconsistent. Thus, SPC can
be used to isolate problems in a process before defective hardware is delivered. This technique can be
used for measurement type data (real numbers) or attribute data. There are two types of attribute data—
binomial data and poisson data. Binomial data have a given number of outcomes, e.g., three of four parts
on an assembly can be defective. Poisson data have an unlimited number of possible outcomes, e.g., a
yard of material may have 1, 10, or 100 flaws.
7.14.3 Procedures
The basic steps for conducting SPC are:
(1) Decide how to group the data. Subgroups should be chosen to show the performance of the
part or process of interest. For example, if a machine is producing several parts at a time,
the parts produced at one time will be a logical subgroup.
(2) Construct a control chart and range chart (see below).
744
(3) Determine and apply control limits to the data.
(4) Determine if any control limits are violated. If any control limits are violated, a special
cause is indicated. In addition to the specific control limits, the analyst must examine the
data plot for other visual indications of special causes in the data. Any particular pattern,
for example, would indicate a special cause is present. The use of engineering judgment is
critical to extracting the maximum amount of data from the SPC plots.
(5) Determine the special cause. This may require Pareto analysis or engineering judgment
using past experience.
(6) Implement a fix for the special cause of variation.
(7) Plot the data to ensure that the fix has been effective.
Control charts (sec. 5.2) are made as follows:
(1) A plot is made of the data, in temporal order of generation, on a scatter plot.
(2) If the data are subgrouped, the mean values of the subgroups are plotted.
(3) A range chart is made where the range is plotted for each subgroup. If the subgroup size is
one, a moving range chart is made. The moving range for an abscissa (“x” value) is the
absolute value of the difference of the ordinates for the abscissas and the previous abscissa.
(4) Determine control limits as discussed below.
(5) Apply appropriate rules to detect a lack of control (see below).
There are typically three control limits based on the population standard deviation of the process
(sec. 6). If negative values of data are possible, there are six control limits. They are the mean of data
plus or minus one, two, and three standard deviations. If one datum exceeds the mean plus three standard
deviations, a rule 1 violation exists. If two of three data points exceed the mean plus two standard
deviations, a rule 2 violation exists. If four of five consecutive data points exceed the mean plus one
standard deviation, a rule 3 violation exists. If eight consecutive points exceed the mean, a rule 4
violation exists. If negative values of data are possible, these rules apply if the values are below the
control limit.
For real number data, the population standard deviation is determined from the average of the
data by the equation:
s = R
m
/d
2
(7.3)
where s is the population standard deviation, R
m
is the mean of the subgroup ranges, and d
2
is a factor
for converting the mean range to the population standard deviation. The constant d
2
can be found in
reference 7.15. If the data are not subgrouped, the average moving range is used. The moving range is
the difference between a data point and the preceding point.
For binomial data, the population standard deviation is given by the equation
s = p
m
(1– p
m
)/ n
m
( )
0.5
(7.4)
where p
m
is the mean fraction defective, and n
m
is the number in each sample.
745
For Poisson data the population standard deviation is given by the equation
s = (C)
0.5
(7.5)
where C is the average number of nonconformities per subgroup.
The discussion in this section has thus far been centered on a violation of a control limit
indicating a special cause of variation being present. The special cause itself may be a shift in the entire
data pattern defined as a mean shift or population shift. In these cases, the limits should be modified or
recalculated to be appropriate for the subsequent data points. A mean shift is generally attributable to an
obvious special cause such as a change in process, material, operator, cutting head, or specification. Data
points immediately preceding and following a mean shift should not be grouped together for any other
analyses.
7.14.4 Example
A hypothetical drill jig is set up to drill five holes in a component. The five holes are of the same
size and have the same positional tolerance. Provide a control chart showing the performance of the drill
jig with the data below, and determine the source of any deviation from nominal hole position. Table 7
13 below shows the deviation from nominal hole size and position made by each drill guide for each
part.
Table 713. Nominal hole size deviations and drill guide positions.
Part # Tempora
l
Process
Order
Hole 1 Hole 2 Hole 3 Hole 4 Hole 5 Range Mean
2 1 2 1 2 3 1 2 1.8
1 2 1 2 3 4 3 3 2.6
4 3 3 3 1 2 2 2 2.2
5 4 2 2 2 3 1 3 2.0
3 5 4 2 3 2 2 2 2.6
6 6 2 1 2 3 1 2 1.8
7 7 6 3 1 2 3 5 3.0
10 8 7 2 2 1 3 6 3.0
8 9 9 3 2 2 2 7 3.6
9 10 10 2 1 3 4 9 4.0
range 9 2 2 2 2 5
mean 4.6 2.1 1.9 2.5 2.2 4.1 2.66
The mean and range for each part and each hole is shown in the table 713. Each part will be
considered to be a subgroup. If the variation between holes is of primary interest, it could be better to
treat each hole as a subgroup. However, the performance of the entire jig is of primary interest in this
example, so each part will be treated as a subgroup. The first control chart (fig. 716) shows the
performance of the jig with the mean plotted against the timephased process order. The UCL is shown.
The UCL is calculated using equation (7.6) to obtain the population standard deviation, multiplying it by
3 and adding it to the mean of the mean deviation. Notice that the mean measurement is increasing for
the last few parts, but no control limits are exceeded.
746
1 2 3 4 5 6 7 8 9 10
Mean
Deviation
6
5
4
3
2
1
UCL
Temporal Process Order
Figure 716. Control chart showing mean deviation for each part.
The second chart (fig. 717) is a range chart that shows the mean range for each part plotted
against part number (note that it remains in temporal order). Part number 9 exceeded the UCL range
(UCLR). UCLR is given by the equation:
UCLR = R
m
[1 + 3(d
3
/d
2
)] (7.6)
where R
m
is the mean range and d
3
is a factor for converting the mean range to the standard deviation of
the range. The constant d
3
can be found in reference 7.15. This shows that the withingroup variation is
increasing more that the grouptogroup variation.
2 3 4 5 6 7 8 9 10
10
9
8
7
6
5
4
3
2
1
Deviat ion
Range
UCLR
1
Figure 717. Range chart showing mean range for each part.
747
The third chart (fig. 718) shows a Pareto chart (sec. 5.6) where the mean deviation is plotted
against hole number. By examination, it can be seen that drill guide position 1 is producing holes with a
mean measurement that is higher than the other drill guide positions.
1 2 3 4 5
5
4
3
2
1
Mean
Deviat ion
Hole Number
Figure 718. Pareto chart showing mean deviation for each hole guide.
The fourth chart, figure 719, shows the deviation produced by hole guide 1 plotted against part
number. By examination, it can be seen that the deviation is increasing starting with part 7.
1 2 3 4 5 6 7 8 9 10
10
9
8
7
6
5
4
3
2
1
Part Number
Deviat ion
Figure 719. Control chart showing mean deviation for hole guide 1.
7.14.5 Advantages
(1) SPC is an excellent technique for determining the cause of variation based on a statistical
analysis of the problem.
(2) The technique improves process performance.
(3) SPC helps identify problems quickly and accurately.
748
7.14.6 Limitations
SPC detects problems but poses no solutions.
7.14.7 Bibliography
Miller, I., and Freund, J.E.: “Probability and Statistics for Engineers.” Second edition, Prentice Hall Inc.,
Englewood Cliffs, NJ 07632.
Nolan, T.W., and Provost, L.P.: “Understanding Variation.” Quality Progress, May 1990.
Weisbrodand, S., and McFarland, C.S., “A New Approach to Statistical Process Control in a Test
Environment: The Empirical Delta Control Chart.” IEEE Melbourne, FL, 1991.
7.15 Flowchart Analysis
7.15.1 Description
A flowchart, as described in reference 7.3, is a pictorial representation of the steps in a process
where each step is represented by a block. The review of a flowchart allows the elimination of nonvalue
added steps. When prepared by a group, the chart represents a consensus. The flowchart analysis is a
useful tool for determining how a process works. By studying how process steps relate to each other,
potential sources of problems can often be identified.
Many different types of flowcharts are useful in the continuous improvement process. Flowcharts
often used are the topdown flowchart, the detailed flowchart, and the work flow diagram. The topdown
flowchart, figure 720, presents only the major and most fundamental steps in a process. This chart
makes it easy to visualize the process in a single, simple flow diagram. Key actions associated with each
major activity are listed below their respective flow diagram steps. A topdown flowchart can be
constructed fairly quickly and easily. This type of flowchart is generally developed before attempting to
produce the detailed flowcharts for a process. By limiting the topdown flowchart to key actions, the
probability of becoming bogged down in the detail is reduced.
TQM
Exposure
Apply
TQM
Show
Purpose
TQM
Develop
Training
Procedures
Meet
Customer
Needs
Subordinates
• Develop
Implementation
Plan
First
Step
• Take
• Be
Committed
• Be
Consistent
• Examine
Your Use
• Develop
User's Guide
• Sell Idea
of
Implementation
• On the job
Training
• Provide
Training
Classes
• Use
Available
Resources
• Listen
to
Customer
• Understand
Customer
Needs
• Establish
Routine
Communication
with
Customer
• Become
Familiar
with TQM
• Familiarize
with TQM
of
Figure 720. Example of topdown flowchart.
749
The detailed flowchart, figure 721, gives specific information about process flow. At the most
detailed level, every decision point, feedback loop, and process step is represented. Detailed flowcharts
should only be used when the level of detail provided by the topdown or other simple flowcharts is
insufficient to support the analysis of the process.
Calculation
Are the bolts
strong enough?
Can they be
bigger?
Is there a
stronger material?
Can more bolts
be used?
End
Yes
Increase
bolt size
Yes
Select
stronger
material
Yes
Add more
bolts
Yes
No
No
No
No
End with
no satisfactory
answer
Do Bolt
Strength
Figure 721. Example of detailed flowchart.
The work flow diagram (section 7.15.4) is a graphic representation of how work flows through a facility.
This diagram is useful for analyzing flow processes, illustrating flow efficiency, and planning process
flow improvement. Figure 722 illustrates the most common flowchart symbols.
750
Activity Symbol
Decision Symbol
Terminal Symbol
Flow Line
Document Symbol
Data Base Symbol
On Page Connector
Off Page Connector
Brick Wall
Inspiration
Black Hole
Dead End
Magic Happens Here
DEAD
END
"POOF"
 Action that is taking place.
 Yes/No Decision.
 Beginning or end of
process.
 Shows direction of process flow.
 Indicates a document
source.
 Indicates a database
source.
 Indicates point elsewhere
on a large page where
process continues.
 Indicates point on
another page where
process continues.
 Shows obstacle beyond your control.
 Indicates a possible solution.
 Indicates a problem that
consumes all resources.
 Shows particular path of a process
has no acceptable solution.
 Indicates that, with a
breakthrough, we can
continue the process.
Figure 722. Common flowchart symbols.
751
7.15.2 Application
A flowchart is best applied in phase B but may also be applied in phase E. This chart is used to
provide a picture of the process prior to writing a procedure. Flowcharts should be created, then
procedures written to follow the flowchart. The chart should be included as an appendix in the
procedure. Flowcharts can be applied to anything from material flow to the steps it takes to service or
sell a product.
7.15.3 Procedures
A flowchart, as described in reference 7.1, is prepared in the following manner:
(1) A development team creates a diagram that defines the scope of the task to be undertaken.
Also identified are the major inputs and outputs.
(2) Create a data flow diagram. Start with executive level data that are involved in the process,
followed by department data and finally branch data.
(3) Using the data, create an initial model. The team should walk through the process and look
for any details that need to be clarified, added, or deleted.
(4) Make a data dictionary. This ensures that everyone involved in the project has a consistent
understanding of the terms and steps used.
(5) Add the process symbols.
(6) Revise, as necessary.
7.15.4 Example
The following example, figure 723, illustrates a work flow diagram for encountering problems
with a copy machine.
7.15.5 Advantages
The following advantages are adapted from reference 7.16:
(1) Flowcharts allow examination and understanding of relationships in a process.
(2) Flowcharts provide a stepbystep picture that creates a common understanding about how
the elements of a process fit together.
(3) Comparing a flowchart to actual process activities highlights areas where policies are
unclear or are being violated.
7.15.6 Limitations
The flowchart development process can be timeconsuming.
752
Start
Error
Light
Flashes
Clear Paper
Path
Still
Flashing
Press Help
Button
Follow
Directions
Call
Repairman
End
No
Yes
Yes
No
Yes
No Still
Flashing
Figure 723. Work flow diagram example.
7.16 Work Flow Analysis
7.16.1 Description
A WFA, as described in reference 7.1, examines the work process for possible improvements in
performance and quality of work life. This technique is really a special case of flowcharting (sec. 7.15).
The goal is to overcome the excuses for not changing work habits on the part of the employee as well as
management. Such excuses are, “It has always been done this way,” and “It’s not my responsibility.”
753
7.16.2 Application
A WFA is best applied in phase E. The analysis is performed in an employee/management
partnership, where the goal for each party is to improve productivity as well as the quality of work life.
The technique will work if executed by a partnership of management and employees.
7.16.3 Procedures
As adapted from reference 7.1, a WFA is performed in the following manner:
(1) Collect data concerning the operation being analyzed. This can be done by observing the
operation or asking questions, but not by reading an operations plan that would tell how the
operation is supposed to be done.
(2) Flowchart the process (sec. 7.15).
(3) Research and collect ideas on how to improve the operation from any sources available.
(4) Define the desired performance versus the actual performance.
(5) Identify the gaps in performance and propose changes to eliminate these gaps.
(6) Analyze these changes by using a multifunctional team.
(7) Once the changes are agreed upon, prototype them on a small basis in a certain area or shift.
(8) Once the bugs are ironed out and the changes are operating smoothly, implement them on a
largescale basis.
(9) Flowchart the new operation and revise the operating procedure documentation to reflect
the changes.
7.16.4 Example
An analysis team was assembled to analyze the food preparation process at a local fast food
restaurant in an attempt to find areas where the operation could be run more efficiently. The steps of the
analysis are as follows:
(1) The first step involved observing the operation and then flowcharting the process as shown
in figure 724 below.
(2) Members of the team then observed other restaurants to find ways of improving the process.
(3) Once the research was completed, the desired performance was identified and compared to
the actual process.
(4) The team, which involved management, employees, and outside consultants, then
developed a new plan for the process.
(5) This new process was first tried out during slow business hours to ensure the new process
ran smoothly.
(6) Once everyone agreed that the new process was more efficient, then it was implemented.
754
Start
No
Yes
Yes
No
Fry
Burger
Is meat
cooked enough?
Put on
Bun
Are
toppings
conveniently
located?
Reorganize
Serve
Figure 724. WFA example.
7.16.5 Advantages
The technique may increase productivity and improve working conditions.
7.16.6 Limitations
(1) The technique requires cooperation between employees and management to be most
successful.
(2) The observed operation may not be fully representative of a “typical” process that would
occur without scrutiny.
755
REFERENCES
7.1 Brocka, B., and M. S.: “Quality Management, Implementing the Best Ideas of the Masters.”
Business One Irwin, Homewood, IL 60430, 1992.
7.2 Camp, R.C.: “Benchmarking, The Search for Industry Best Practices that Lead to Superior
Performance.” Quality Press, 1989.
7.3 Hunt, V.D.: “Quality in America, How to Implement a Competitive Quality Program.” Business
One Irwin, Homewood, IL 60430, 1992.
7.4 Lake, J.G.: “Concurrent/Systems Engineering.” Defense Systems Management College, 1992.
7.5 “Total Quality Management – Executive Briefing.” American Supplier Institute, Dearborn, MI,
1990.
7.6 Choppin, J.: “Quality Through People, A Blueprint for Proactive Total Quality Management.”
IFS Publications, United Kingdom, 1991.
7.7 Ott, E.R., and Schilling E.G.: “Process Quality Control.” McGrawHill Book Company, 1990.
7.8 Juran, J.M., Gryna, F.M., Jr., and Bingham, R.S., Jr.: “Quality Control Handbook.” McGrawHill
Book Company, 1974.
7.9 Box, G.E.P., Hunter, W.G., and Hunter, J.S.: “Statistics For Engineers.” John Wiley and Sons,
Inc., 1978.
7.10 “Advanced Solid Rocket Motor Ignition Subsystem Assessment.” Larry Thomson, Kurt
Everhart, Curtis Ballard, Report No. 31400493259, August 1993.
7.11 “System Safety Program Requirements.” MIL–STD–882C, January 1993.
7.12 “A Comprehensive Tool for Planning and Development.” M.A. Schubert, Reliability Technology
Consultant, Inland Division of General Motors Corporation, Dayton, OH, 1989.
7.13 “Prioritization Methodology for Chemical Replacement.” W. Cruit, S. Schutzenhofer, B.
Goldberg, and K. Everhart, May 1993.
7.14 “Introduction to Quality Engineering – Four Day Workshop, Implementation Manual.”
American Supplier Institute, Dearborn, MI, 1989.
7.15 NASA–STD–8070.5A, “Trend Analysis Techniques.” October 1990.
7.16 Harrington, H.J.: “Business Process Improvement.” McGrawHill Book Company, TS5156
H338, 1991.
756
8. TREND ANALYSIS TOOLS
Trend analysis, as described in reference 8.1, is a quantitative tool used to identify potentially
hazardous conditions and cost savings based on past empirical data. Trend analysis evaluates variations
of data to find trends, with the ultimate objective of assessing current status and forecasting future
events. Trend analysis can be reactive or proactive. Data examined from past events can uncover a cause
of a problem or inefficiency in a product or operation. Also, realtime data can be tracked to detect
adverse trends that could indicate an incipient failure or can be used to reduce discrepancies in a product
or operation.
Program level trending exchanges data between organizations and correlates trends from the
various organizations to find relationships and allows integration of the trend analysis effort with any
planned TQM effort (sec. 7), such as SPC (sec. 7.14). It also allows upper level management to forecast
problems such as shortages, schedule delays, or failures. Finally, in starting a program level trending
effort early in the program, data collection will be more efficient and costeffective.
The use of trend analysis has several benefits. Among them are:
(1) Predicting system or process failure or violation of a process limit criterion.
(2) Indicating that a unit can remain in service longer than anticipated or projecting the service
life of a unit.
(3) Eliminating the need for some hardware inspections.
(4) Increase costeffectiveness by reducing variability in a process.
There are different levels of trend analysis parameter criticality based on the degree of the benefit
derived from the results of the trend analysis for that parameter. Some parameters have a direct effect on
system safety while others will have an impact on cost or timeliness of a process. Criticality levels have
an impact on the amount of trending to be performed, the level to which it is to be reported, the data that
are to be stored, and the time over which the trending is to be performed. Examples of criteria for levels
of requirements are:
(1) Parameters impacting personnel safety.
(2) Parameters impacting successful system performance.
(3) Parameters which could cause failure of a component that would not result in system
failure.
(4) Parameters impacting schedule of the system.
(5) Parameters impacting delivery schedule of components.
(6) Parameters impacting cost of manufacturing.
Trending can be used at levels from program management to component and system production
and vendors. Upper level management would conduct trending on program level issues, and individual
organizations would conduct trending on issues pertinent to that organization at a component/material,
subsystem, or system level.
81
Examples of trending activities are:
(1) Componentreceiving organizations can conduct trending on such things as would indicate
the quality of incoming components, materials, and problems of receiving them in proper
condition.
(2) Manufacturing can conduct trending on component and system requirements, and
production problems.
(3) Test, launch, and refurbishment organizations can conduct trending on performance, time
to conduct operations, and problems encountered.
Some trending results will be reported to upper level management, engineering, and the
customer, while other results would be for local use by the individual organizations.
Five trending analysis techniques will be discussed in this section. Performance trend analysis,
discussed in section 8.1, detects a degrading parameter prior to a potential failure as well as predicting
future parameter values.
Problem trend analysis, discussed in section 8.2, provides an early indicator of significant issues
in other types of trend analysis. Other applications of this analysis are to “examine the frequency of
problem occurrence, monitor the progress of problem resolution, uncover recurring problems, and assess
the effectiveness of recurrence control.”
8.2
A technique that provides visibility to determine the current/projected health of the human
support element is programmatic trend analysis. This analysis is discussed in section 8.3. A technique
that monitors the current health of support systems and forecasts support problems to enable resolution
with minimum adverse effect is supportability trend analysis. This analysis is discussed in section 8.4.
Finally, reliability trend analysis is discussed in section 8.5. This technique is similar to
performance trend analysis and problem trend analysis. Reliability trend analysis measures reliability
degradation or improvement and enables the prediction of a failure so action can be taken to avert the
failure.
There can be a high level of overlap for some of these types of trend analysis, depending on
individual definitions of performance, reliability, and problems. Since many tools are useful for all types
of trending and the trend analysis customer typically looks for known parameters, this overlap is not a
problem. Performance, problem, and reliability trend analyses are more directly applicable to the needs
of a system engineer, than programmatic or supportability trend analyses. However, the former two
types of trend analysis are presented here, since results from these analyses may impact the system for
which the system engineer is responsible.
A summary of the advantages and limitations of each tool or methodology discussed in this
section is presented in table 81.
82
Table 81. Trend analysis tools and methodologies.
Tool or Methodology Section Advantages Limitations
Performance trend analysis 8.1 (1) Detects a degrading parameter prior to a potential
failure.
(2) Predicts future parameter values or estimates the
longterm range of values of influential variables.
(3) The service life of systems or system elements can
be predicted.
(1) Parameter sensors may need to be installed to
obtain trending data; this can be costly.
(2) The operating state, output, or load, about/
through which a system/subsystem/component
fluctuates, often cannot be controlled to achieve
consistent trend data. (Data must be statistically
stable.)
(3) The slope and stability of the data approaching/
departing the recorded data point are not known
without using a data buffer.
(4) Data are not always easily quantifiable, limiting the
usefulness of the technique.
Problem trend analysis 8.2 (1) Provides an early indicator of significant issues in
other types of trend analysis.
(2) Examines the frequency of problem occurrence,
monitors the progress of problem resolution,
uncovers recurring problems and assesses the
effectiveness of recurrence control.
Candidate items should be chosen carefully because the
analysis can be costly if performed for all potential
problem areas.
Programmatic trend analysis 8.3 This technique monitors programmatic posture and
provides visibility to determine current/projected health
of the human support element.
The data collection process can be extensive because of a
potentially large and varied number of sources.
Supportability trend analysis 8.4 This technique monitors the current health of support
systems and forecasts support problems to enable
resolution with minimum adverse effect.
Determining the extent of analysis and identifying the
appropriate parameter variations that must be measured
can be difficult.
Reliability trend analysis 8.5 This technique measures reliability degradation or
improvement and enables the prediction of failures so
action can be taken to avert failure.
Candidate items must be chosen carefully because the
analysis can be costly if performed for all potential
parameters.
83
8.1 Performance Trend Analysis
8.1.1 Description
Performance trend analysis, as described in references 8.1 and 8.2, is a parametric assessment of
hardware and software operations to evaluate their status or to anticipate anomalies or possible
problems. This assessment not only includes operational performance, such as ballistics of an SRM but
also assesses hardware performance, such as insulation and inhibitor systems, the motor case, or the
nozzle system. For example, postflight measurements of insulation indicate the performance of the
insulation during motor operation. The independent variable in performance trend analysis can be time
or sequence. Some performance data, for example, that relating to safety, may be recorded and trended
on a realtime basis.
As an example, for an SRM, typical operational performance parameters to be trended could be
peak pressure, total impulse, ignition delay, thrust risetime characteristics, and propellant structural or
ballistic properties. Typical hardware performance parameters to be trended could include insulation
anomalies, structural factor of safety (calculated from asbuilt drawings), and seal performance (as
measured, i.e., from leak checks).
As described in reference 8.2, data sources for performance trend analysis might include new,
refurbished, and repaired component and subassembly acceptance inspection, checkout, and test data for
development and verification and production hardware including, but not limited to:
(1) Alignment data.
(2) Contamination data.
(3) Dimensional data.
(4) Nondestructive test data, e.g., magnetic particle, radiography, penetrant, and ultrasonic
data.
(5) Proof test data, e.g., leak check and hydroproof data.
(6) Functional or performance data, e.g., quantitative and qualitative data.
8.1.2 Application
Performance trend analysis is best applied in phase E but may also be applied in phase D. This
analysis can be used to identify certain parameters that will indicate that a system or system element
(i.e., subsystem, assembly, subassembly, component and piecepart) is degrading and will potentially
fail. These parameters can include, but are not limited to, the following:
8.2
(1) Direct measures of degradation, such as wear, erosion, pitting, and delamination.
(2) Measures of conditions that might introduce degradation, such as pressure anomalies,
temperature anomalies, vibration, friction, leakage, and contamination.
(3) Measures that indicate a shift in performance,such a changes in material properties,
calibrations, and electrical resistance.
Attendance to maintenance can help to detect degrading parameters which could lead to failure
or delay resulting from an exceedance of criteria.
84
8.1.3 Procedures
The procedures to apply performance trend analysis, adapted from references 8.1 and 8.2, are
presented below:
(1) Identify the elements of the system. Assess those hardware or software system elements to
identify items that could cause critical or costly failures. Each element of the system should
be considered, i.e., each subsystem, assembly, subassembly, component and piecepart. List
these system elements as candidates for performance trend analysis.
(2) From the list, select which items will be analyzed. Concerns (in terms of risk, safety, cost,
availability, or schedule) and expected benefits should be the basis for setting priorities
when considering which items to select for performance trend analysis.
(3) Determine the parameters that characterize the performance of the selected system
elements. Select parameters that will indicate performance deterioration of the given
system element in a timely manner for corrective actions to be approved by management
and implemented. Review the following to identify possible candidate parameters for
performance trending:
a. FMEA (sec. 3.4)/critical items list (FMEA/CIL).
b. Drawings and specifications.
c. Previous problem reports.
d. Equipment acceptance data.
e. Original equipment manufacturer’s data.
f. Operations manual.
(4) Establish the criticality of each selected parameter. The parameter criticality should be
based on the FMEA/CIL or other criteria that have been preapproved by management. The
criticality of the parameter will indicate the magnitude of the impact if an adverse trend is
detected and to what level of management that adverse trend is reported.
(5) Determine if the selected parameters can be quantified with obtainable data. A parameter
may be quantified with direct measured data (such as temperature, pressure, force, strain,
acceleration, heat flux, etc.) or by calculation involving two or more direct measurements
(such as specific impulse for rocket engines or compressor and turbine efficiencies for jet
engines). If data are not available, establish a system to acquire the data or drop the item
from trend analysis.
The availability of the data—the more available the data are, and assuming statistical
stability, the greater the likelihood of successful trending. Ten to twenty data points for a
parameter are desirable as a minimum.
(6) Develop acceptance levels for the parameters. These levels or limits become the basis for
determining if a parameter is in control or corrective actions are required. First, determine the
boundaries that define the required range for normal operation. These boundaries should be
identified for each parameter from a review of vendersupplied data, test or operational data,
or specifications or requirement documents. Next, determine action limits that fall within
these boundaries in which corrective action will be initiated if the action limits are exceeded.
85
Care should be taken in choosing the action limits so that (1) variation in normal acceptable
operation will not cause the action limits to be exceeded (causing unnecessary expenditure
of resources), and (2) corrective actions can be implemented promptly, once the action limit
is exceeded but before the boundaries for desired normal operation are exceeded. These
action limits should be taken from historical data that represent the same distribution for the
parameter as that in which future measurements will be recorded and tracked.
(7) Analyze the selected parameters for trends. Various statistical and graphical techniques for
performing trend analysis can be found in reference 8.3. Use graphical tools to transform
raw, measured, or calculated data into usable information. The graphical tools can include
scatter plots (sec. 5.1) and control charts (sec. 5.2). Use statistical tools, such as regression
analysis (sec. 6.6), to determine the trend line through a given set of performance data.
Determine how well the trend line fits the data by using techniques such as R
2
or Chi
Square measure of fit tests. These tests are described in detail in reference 8.3 and statistical
textbooks and handbooks. Use the trend line to detect if there is a trend that is approaching
or has exceeded the action limits determined in step 6.
(8) Resolve adverse trends. If an adverse trend is detected, determine the cause of the adverse
trend. Perform correlation analyses (sec. 6.3) to determine what other parameters (factors)
are contributing to the adverse trend. Once the cause of the adverse trend is identified,
propose a remedy to correct the problem before the boundaries for desired normal operation
are exceeded. Implement (management approval may be required) the remedy, then trend
future performance and assess the effectiveness of the remedy.
(9) Report the results. To maximize the benefits of the trend analysis effort, the results should
be documented and distributed to the appropriate levels of management and functional
organizations to ensure corrective actions are implemented in a timely manner once an
adverse trend is detected. Typically, these reports should contain the following items
(adapted from reference 8.2):
a. System element (from step 1).
b. Parameter identification (from step 3).
c. Criticality (from step 4).
d. Data source (from step 5).
e. Failure mode as described in the FMEA.
f. Baseline changes, if applicable.
g. Indication of excluded data, trends, their direction and disposition (adverse or
acceptable).
h. Corrective action used and its effectiveness, if applicable.
i. Need for additional data, if applicable.
j. Recommendations, as necessary.
k. Applicability to other types of trending.
l. Need for additional correlation analysis, if applicable.
86
8.1.4 Example
In a machine shop, the service life of saw blades was studied. The objectives of the study were to
determine the expected life of the blades and develop a methodology to determine when special causes
were effecting machine performance. Performance trend analysis was performed to address both these
questions. Blades are replaced when their performance degrades from 10 to 3 cuts per hour. First,
performance data were collected for 30 blades to statistically establish the expected service life and the
band for expected normal performance.
The daily average cuts per hour for each blade of the 30 blades were measured and recorded until
the 3 cutsperhour limit was reached. A linear regression analysis of these data was performed to
determine the relationship between the cuts per hour and work days. The variation of the 30 blades was
examined for each day of operation. This analysis revealed that the variation grew linearly with time. A
band was established from ± 3 standard deviations from the regression line for each day of operation. The
expected service life range for a given blade was expressed as the time range defined by the regression ± 3
standard deviation band of the regression intercepted the three cutsperhour replacement limit.
The lower (–3 standard deviation) limit of the band was defined as the action limit to ensure the
machine is operating properly. The daily average cuts per hour is tracked for a blade in operation. When
the action limit is exceed, the machine is examined to determine if there is a special cause that is
reducing the blade service life.
The expected band for normal operation and expected service life are illustrated on the performance
trend analysis plot presented in figure 81. The performance of a given blade that has just reached the
end of its service has been tracked on this chart. Note that the action limit is the lower limit of the
expected normal operation band.
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
0
2
4
6
8
10
12
0 5 10 15 20 25
Blade Service Life, Work Days
Blade Replacement Limit
± 3 Std. Dev. Band
(Historical Data)
Expected Service Life
Action Limit
Figure 81. Performance trend analysis example.
87
8.1.5 Advantages
(1) Performance trend analysis can be used to detect a degrading parameter prior to a potential
failure.
(2) This technique can predict future parameter values or estimate the longterm range of
values of influential variables.
(3) The service life of systems or system elements can be predicted.
8.1.6 Limitations
(1) Parameter sensors may need to be installed to obtain trending data—this can be costly.
(2) The operating state, output, or load, about/through which a system/subsystem/component
fluctuates, often cannot be controlled to achieve consistent trend data.
(3) The slope and stability of the data approaching/departing the recorded data point are not
known without using a data buffer.
(4) Data are not always easily quantifiable, limiting the usefulness of this technique.
8.1.7 Bibliography
“ASRM Trend Analysis Requirements Document.” Sverdrup Technology, Inc., Report No. 31400491
115, December 1991.
NASA Technical Memorandum 85840, “The Planning and Control of NASA Programs and Resources.”
NMI 1103.39, “Role and Responsibilities – Associate Administrator for Safety and Mission Quality
(SMQ).”
NMI 8070.3, “Problem Reporting, Corrective Action, and Trend Analysis Requirements.”
NMI 8621.1, “Mishap Reporting and Investigating.”
NHB 5300.4 (1A1), “Reliability Program Requirements for Aeronautical and Space System
Contractors.”
NHB 8070.TBD, “Significant Problem Reporting System.”
Special Study Z001U61, “Marshall Operations Reliability Trend Analysis Standard.” Sept. 16, 1991.
8.2 Problem Trend Analysis
8.2.1 Description
Problem trend analysis, as described in references 8.1 and 8.2, identifies repetitive problems and
assesses how often given problems occur. Also, problem trend analysis provides a mechanism to track
88
progress of problem resolution. Finally, problem trend analysis evaluates organizational proficiency in
preventing repetitive problems. Underlying causes can be uncovered when several problems are
compared. Problem trend analysis is often an early indicator of significant issues in other types of trend
analysis.
There are three basic objectives in problem trend analysis:
(1) Isolate problems to specific causes and examine the frequency of occurrence of these
causes. Problem trending is often initiated on a system level but finished on a component
(or lower) level.
(2) Track problems to determine if occurrence is increasing or decreasing, or if some problems
are affecting other parameters.
(3) Determine if baseline changes or corrective actions increase or decrease the frequency of
problem occurrence.
Data sources for problem trend analysis may include, but need not be limited to:
(1) Failure or problem reporting and corrective action systems such as Problem Reporting and
Corrective Action (PRACA).
8.2
(2) Discrepancy reports (DR’s).
(3) Problems identified by the other four types of trend analysis.
8.2.2 Application
As described in reference 8.2, problem trend analysis is used to identify recurring problems and
assesses the progress in resolving these problems and eliminating the recurrence of the problems. This
analysis is best applied in phase E but may also be applied in phase D. The main interest in this analysis
is locating where the key problems are occurring and the frequency of occurrence. Graphical techniques
such as the Pareto analysis (sec. 5.6) are useful in focusing attention and determining where other
analyses such as performance trend analysis (sec. 8.1) can be beneficial.
Problem trend analysis provides a historical overview of problems in an easytounderstand
graphical format. This overview assists in decisionmaking relative to design effectiveness, process, or
procedural changes over time. Problem trend analysis can be the first step in the initiation of corrective
action to improve system performance.
Basic criteria (from reference 8.2) for the selection of candidate items include:
(1) Problem frequency (establish from historical problem report databases).
(2) Criticality (usually determined from FMEA’s).
(3) Engineering judgment (by cognizant personnel familiar with both the hardware and
requirements).
(4) Unique program or project requirements (these requirements indicate more severe
consequences than normally associated with a given type of problem).
89
8.2.3 Procedures
Procedures (adapted from reference 8.2) to perform problem trend analysis are as follows :
(1) Gather pertinent data. Examine the applicable historical data base(s) and acquire the
appropriate data. These data bases contain information concerning problem reporting. The
data bases are usually maintained by the organization responsible for design and
manufacture of a system element or the operational organization that uses the system.
Typically, searches are made for rejection rates from acceptance testing, operation
problems, and configuration nonconformance. These searches should be performed for a
given time frame. The data bases should be searched for events, operating cycles, hardware
identification codes (i.e., system, subsystem, assembly, subassembly, component or piece
part number), failure mode codes from the FMEA, or key words for given hardware
failures or failure modes.
(2) Identify frequency of problems for the system element under consideration. The system
element may be the subsystem, assembly, subassembly, component or piecepart.
Determine the number of problems (without distinction of failure mode) associated with the
system element during given time periods (i.e., days, weeks, months, years, etc.). Next,
normalize these unrefined frequency data to the number of operations, cycles, missions, or
elements produced during the given time periods. Construct a bar chart (sec. 5.3) for both
the unrefined and normalized data. The unrefined data are plotted as a function of
occurrences versus time, while the normalized data are plotted as a function of occurrence
rates versus time.
(3) Identify primary causes of the problems. For each system element under consideration,
determine the categories of failure modes or causes that induced the problems identified in
step 2. Careful review of the problem reports should be performed to ensure that
inconsistent wording of problem reports by different authors does not mask the true value
of each failure mode or cause. Next, determine the number of occurrences for each failure
mode or cause. Construct a Pareto chart (sec. 5.6) of the number of occurrences versus
failure modes or causes and identify areas of concern. From the Pareto chart, identify the
failure modes or cause of consequence that require further assessment.
(4) Determine if a trend over time exists for each of the identified failure modes or cause of
consequence. Normalize the failure mode or cause as the problems were normalized in step
2 (i.e., normalized by the number of operations, cycles, missions, or elements produced
during the given time periods). Construct a bar chart (sec. 5.3) for each failure mode or
cause. These bar charts should present the total and normalized number of occurrences
versus time. Procedure, process, configuration or design changes and the time of their
implementation should be noted on these charts.
Once the bar chart is generated, fit the normalized failure mode or cause occurrences with
either a linear, exponential, power, logarithmic, or positive parabolic trend line. Determine
the goodness of fit for each trend line model to the data with such statistical methods as the
R
2
test. Refer to reference 8.3 or statistical textbooks or handbooks for details in fitting the
data with trend lines or testing for goodness of fit.
(5) Report the results. Prepare a summary assessment of the problem trend analysis, including:
a. System element (from step 2).
b. Data source, i.e., the historical problem report data base (from step 1).
810
c. Failure modes trended and total number of problem reports assessed.
d. Criticality (from FMEA) of the failure mode(s) and date of last occurrence.
e. Baseline procedure, process, configuration or design changes, if applicable.
f. Chief failure mode or cause of consequence.
g. Indication of trends, their direction and disposition (adverse or acceptable).
h. Corrective action used and its effectiveness, if applicable.
i. Need for additional data, if applicable.
j. Recommendations, as necessary
k. Applicability to other types of trending.
l. Need for additional correlation analysis, if applicable.
8.2.4 Example
The monthly rejection rate of wickets exceeded a company’s goal of 5 units per 1,000 units
produced (0.5 percent) during a 3mo period last year. A problem trend analysis effort was conducted to
understand the reason for the increased rejection rate and to formulate a plan to prevent future excessive
rejection rates. The manufacturing reports for a 1yr production of wickets were reviewed. The results
were summarized by month and are presented in figure 82(a). Also, the monthly production and
rejection rates are shown in figure 82(a).
The cause of each rejection was also identified from the manufacturing problem reports and was
categorized as being due to human error, inadequate properties of raw materials, production machine
malfunctions, or other miscellaneous causes. These results are presented for each month in figure 82(b).
The number of rejections and the rejection rates were plotted on a bar chart and the results are
presented in figure 82(c). The rejection rates were normalized to units produced monthly. As seen on
this chart, the rejection rate exceeded the company goal of 0.5 percent during August, September, and
October; therefore, this time period became the focus of the analysis.
Note from this figure that the normalized rejection rate data, not the absolute number of
rejections, indicate the time period of concern.
A Pareto chart (shown in figure 82(d)) was produced for the entire year to establish the
significance of each cause for rejection. This chart revealed that human error was the most significant
cause for rejection over the entire year period. However, a Pareto chart generated for the 3mo period of
concern, revealed that inadequate material properties was the most significant cause for unit rejection.
Comparing the two Pareto charts shows that inadequate material properties was a much more significant
problem during the 3mo period, and that human error was over the entire year. This chart for the 3mo
time period is presented in figure 82(e).
The number of rejections and the rejection rates due to inadequate properties of raw materials
were plotted on a bar chart and the results are presented in figure 82(f). The rejection rates were
normalized to units produced monthly. As seen on this chart, the increase in the rejection rate due to
inadequate material properties was the driving factor in exceeding of the maximum rejection goal.
811
Month Units Produced Units Rejected Rejection Rate
(Units/1,000 Produced)
Jan. 5,100 12 2.35
Feb. 4,600 21 4.28
Mar. 4,900 16 3.26
Apr. 2,900 12 4.14
May 3,150 13 4.12
Jun. 3,050 10 3.27
Jul. 3,000 12 4.00
Aug. 1,700 14 10.35
Sep. 1,400 14 9.65
Oct. 1,750 15 8.57
Nov. 3,100 9 2.90
Dec. 4,950 21 4.24
(a) History of unit rejections.
Causes for Unit Rejection
Month Human
Error
Inadequate
Material
Properties
Machine
Malfunction
Other Units
Rejected
Jan. 6 2 3 1 12
Feb. 10 4 5 2 21
Mar. 8 3 4 1 16
Apr. 6 3 3 0 12
May 6 2 4 1 13
Jun. 5 2 3 0 10
Jul. 6 2 2 2 12
Aug. 3 10 1 0 14
Sep. 3 9 1 1 14
Oct. 4 9 2 0 15
Nov. 5 2 2 0 9
Dec. 10 5 5 1 21
Total 72 53 35 9 169
(b) History of unit reject by cause.
Figure 82. Problem trend analysis example—Continued
812
J
A
N
F
E
B
M
A
R
A
P
R
M
A
Y
J
U
N
J
U
L
A
U
G
S
E
P
O
C
T
N
O
V
D
E
C
0
5
10
15
20
25
0
5
10
15
20
25
N
u
m
b
e
r
o
f
U
n
i
t
s
R
e
j
e
c
t
e
d
R
e
j
e
c
t
i
o
n
R
a
t
e
,
U
n
i
t
s
/
1
,
0
0
0
P
r
o
d
u
c
e
d
Time, Months
Units Rejected
Rejection Rate
(c) Total unit rejection and rejection rate versus time.
Human Error Inadequate
Material
Properties
Machine
Malfunctions
Other
0
20
40
60
80
100
0
20
40
60
80
100
N
u
m
b
e
r
o
f
U
n
i
t
R
e
j
e
c
t
i
o
n
s
P
e
r
c
e
n
t
Causes of Unit Rejections
(d) Pareto chart of causes for entire period.
Figure 82. Problem trend analysis example—Continued
813
Inadequate
Material
Properties
Human Error Machine
Malfunctions
Other
0
5
10
15
20
25
30
35
40
0
20
40
60
80
100
N
u
m
b
e
r
o
f
U
n
i
t
R
e
j
e
c
t
i
o
n
s
P
e
r
c
e
n
t
Causes of Unit Rejection
(for August, September, and October only)
(e) Pareto chart of causes for period of concern.
J
A
N
F
E
B
M
A
R
A
P
R
M
A
Y
J
U
N
J
U
L
A
U
G
S
E
P
O
C
T
N
O
V
D
E
C
0
2
4
6
8
10
12
0
2
4
6
8
10
12
N
u
m
b
e
r
o
f
U
n
i
t
s
R
e
j
e
c
t
e
d
R
e
j
e
c
t
i
o
n
R
a
t
e
,
U
n
i
t
s
/
1
,
0
0
0
P
r
o
d
u
c
e
d
Time, Months
Units Rejected
Rejection Rate
(f) Unit rejection and rejection rate (due to inadequate material properties) versus time.
Figure 82. Problem trend analysis example—Continued.
814
Further analysis showed that a statistically significant larger portion of the units rejected for
material properties came from one lot of materials used during the August to October period. This lot
met acceptance test criteria, however it was by a narrow margin. To avoid further high rejection rates,
the specifications for the raw material were tightened as were the corresponding acceptance tests.
8.2.5 Advantages
(1) Problem trend analysis can provide an early indicator of significant issues in other types of
trend analysis.
(2) This technique examines the frequency of problem occurrence, monitors the progress of
problem resolution, uncovers recurring problems, and assesses the effectiveness of
recurrence control.
8.2.6 Limitations
Candidate items should be chosen carefully because the analysis can be costly or noninformative
if performed for all potential problem areas.
8.2.7 Bibliography
“ASRM Trend Analysis Requirements Document.” Sverdrup Technology, Inc., Report No. 31400491
115, December 1991.
NASA Technical Memorandum 85840, “The Planning and Control of NASA Programs and Resources.”
NHB 5300.4 (1A1). “Reliability Program Requirements for Aeronautical and Space System
Contractors.”
NHB 8070.TBD, “Significant Problem Reporting System.”
NMI 1103.39, “Role and Responsibilities  Associate Administrator for Safety and Mission Quality
(SMQ).”
NMI 8070.3, “Problem Reporting, Corrective Action, and Trend Analysis Requirements.”
NMI 8621.1, “Mishap Reporting and Investigating.”
Special Study Z001U61, “Marshall Operations Reliability Trend Analysis Standard.” Sept. 16, 1991.
8.3 Programmatic Trend Analysis
8.3.1 Description
Programmatic trend analysis, as described in references 8.1 and 8.2, is concerned with
organizational or programmatic issues that may impact safety or system success. These issues include
general program health, schedule issues, overtime or sick time usage, production bottlenecks, accidents
or equipment damage, supply of critical skills (critical resource scheduling), cost of upkeep versus
redesign or reprocurement, noncompliances, and cost of replacement versus cost of repair.
815
8.3.2 Application
Programmatic trend analysis is best applied in phase E. The objective of programmatic trend
analysis is to provide management a status on programmatic issues or early warning of programmatic
problems. For example, warning of inappropriate demands on manpower, impending delays, mismatches
between demand and available expertise, alerting management on areas needing attention (e.g., damage,
injury or accident frequency), supporting program/project improvement changes, support management in
monitoring project management performance indicators over time to indicate endproduct safety and
reliability.
Common candidates for programmatic trend analysis include the following:
(1) “Manpower strength by speciality, experience, qualification, certification, and grade.”
8.2
(2) “Schedule changes/slippages or overages.”
8.2
(3) Accident or sick time frequency.
(4) “Overtime usage versus approved policy.”
8.2
(5) Labor problems.
(6) “Requirement changes, including waivers and deviations.”
8.2
(7) “System nonconformances and problems due to human error.”
8.2
(8) “Rework expenditures.”
8.2
(9) Time/cost considerations for redesign.
Concerns (in terms of risk, safety, cost, availability, or schedule) and expected benefits should be
the basis for setting priorities when considering using programmatic trend analysis.
8.2
Some programmatic trend data will be obtained from other parameters; however, some
parameters will be unique to programmatic trends. Trending parameters and supporting data to be
recorded and trended that would have a programmatic impact must be selected.
8.3.3 Procedures
As described in reference 8.2, apply the following steps to perform the programmatic trend
analysis:
(1) Determine the programmatic parameters to be assessed. Determine which programmatic
parameters will be trended. Common parameters that are trended are presented in section
8.3.2. However, the selection of parameters should be determined by the unique needs of
the organization or program involved. Maintain a list of parameters for which
programmatic data are to be supplied.
(2) Acquire and compile data for the selected parameters. Data sources (adapted from reference
8.2) for programmatic parameters include, but are not be limited to:
816
a. Planned versus actual cost reports (so that number and magnitudes of cost overruns
and underruns can be determined).
b. Planned versus actual schedule charts (so that the number and magnitude of schedule
delays and accelerations can be determined).
c. Quality assurance reports (documenting the number of noncompliances).
d. Development and verification status reports (documenting the success or failure in
verifying system requirements or specifications).
e. Inventory control records (documenting the number of times work was delayed due to
unavailable material).
f. Facility, equipment, and hardware problem and corrective action reports.
g. Acceptance records (documenting number of units produced that were accepted or not
accepted by the customer).
h. Shipping and receiving logs (including planned versus actual shipping and receiving
dates).
i. Work authorization and control documents.
j. Planned versus actual staff level reports.
k. Safety, mishap, or incident reports.
(3) Ensure the validity of the data. Care should be taken to ensure the data analyzed are
accurate and are an appropriate measure for the programmatic parameter being trended.
(4) Develop the required analytical techniques and controls (e.g., Pareto charts (sec. 5.6) and
histograms (sec. 5.7)). Action limits should be establish in which corrective action will be
initiated if the action limits are exceeded. Action limits can be set to ensure parameters stay
within the operating and administrative policies and procedures, work standards, and goals
of the organization.
(5) Determine the structure for project data collection, maintenance, and reporting. Identify the
organizations and personnel responsible for collecting, maintaining, assessing, and
reporting the data.
(6) Make data available to program management.
(7) Analyze the data for trends. Use control charts (sec. 5.2) to display the historical trends of
validated data for the programmatic parameters being measured, along with the realistic
action limits established.
(8) Resolve adverse trends. When an adverse trend has been identified, conduct an analysis of
that trend. Preparing a cause and effect diagram (sec. 7.2) may be useful in identifying the
root cause of the adverse trend. Once the cause of the adverse trend is identified, propose a
remedy to correct the problem before the boundaries for desired normal operation are
exceeded. Implement the remedy (management approval may be required), then trend
future performance for the programmatic parameter and assess the effectiveness of the
remedy.
817
(9) Report the results. The reports should be published at intervals that will allow management
to take prompt action to correct problems before they become unmanageable. The reports
should contain sufficient details so that management can accurately assess the risk
associated with an averse trend. Suggested reporting formats for common programmatic
parameters can be found in reference 8.2.
8.3.4 Example
At the start of a new program, candidate parameters were identified for programmatic trend
analysis. The list was reviewed by both the project team and management, and trending parameters were
selected. Arrangements were made for data to be collected and assessed for each parameter. Action
limits were determined from company policies and procedures and program requirements
documentation.
The following example illustrates how programmatic trend analysis was applied for a specific
programmatic parameter—overtime usage. Review of the company policy revealed that the average
overtime rate for a project with more than the equivalent of 100 fulltime workers should not exceed 10
percent per month. This particular program average staffing level was 125. An action limit of 8 percent
per month maximum overtime rate was established. If this action limit is approached or exceeded,
management should be notified and corrective action taken.
The actual overtime rate, expressed in percentage versus month worked, is presented for 1991 in
figure 83. As seen in this figure, the overtime rate exceeded the action limit in May. Management was
notified and overtime usage was reviewed. The cause for the increased rate was due to new negotiated
work to be performed. However, the scheduled completion date for the project had remained fixed.
Overtime projections revealed that the overtime rate would range from 10 to 13 percent for the
remainder of the calendar year.
Work was identified that could be subcontracted. This work was approximately 6 percent of the
total project. Management agreed to subcontract the work starting in midJune. Tracking the overtime
usage rate past the time the corrective action was implemented revealed that the fix of the programmatic
problem was effective (as shown in fig. 8.3).
8.3.5 Advantages
The programmatic trend analysis technique monitors programmatic posture and provides visibility to
determine the current/projected health of the human support element.
8.3.6 Limitations
The data collection process can be extensive because of a potentially large and varied number of
sources.
8.3.7 Bibliography
“ASRM Trend Analysis Requirements Document.” Sverdrup Technology, Inc., Report No. 31400491
115, December 1991.
NASA Technical Memorandum 85840, “The Planning and Control of NASA Programs and Resources.”
818
0
2
4
6
8
10
12
1991
Company Policy Limit
Action Limit
Surplus work
subcontracted.
Months
Figure 83. Programmatic trend analysis example.
NHB 5300.4 (1A–1), “Reliability Program Requirements for Aeronautical and Space System
Contractors.”
NHB 8070.TBD, “Significant Problem Reporting System.”
NMI 1103.39, “Role and Responsibilities – Associate Administrator for Safety and Mission Quality
(SMQ).”
NMI 8070.3, “Problem Reporting, Corrective Action, and Trend Analysis Requirements.”
NMI 8621.1, “Mishap Reporting and Investigating.”
Special Study Z001U61, “Marshall Operations Reliability Trend Analysis Standard.” Sept. 16, 1991.
8.4 Supportability Trend Analysis
8.4.1 Description
Supportability trend analysis, as described in references 8.1 and 8.2, is performed to evaluate the
proficiency of an organization at controlling the logistics factors supporting a program. Logistic concerns
likely to be trended are supplies of spare parts, replaceability, frequency of cannibalization, late deliveries,
819
shortages, maintenance, etc. Typically, data used for supportability trend analysis are not in a form that is
readily usable. Processing certain data is laborintensive or may not be feasible due to contractual
considerations. Sometimes indirect or related parameters may be used to indicate supportability.
8.4.2 Application
The supportability trend analysis technique is best applied in phase E. This analysis assesses the
effectiveness of logistics factors (extracted from reference 8.2) such as the following:
(1) Maintenance.
(2) Supply support.
(3) Facilities management and maintenance.
(4) Support personnel and training.
(5) Packaging, handling, storage, and transportation.
(6) Technical data support.
(7) Automated data processing hardware/software support.
(8) Logistics engineering support.
Supportability trend analysis monitors the current status of the support systems and forecasts the
future status in order to resolve problems with minimum adverse effect. The current support systems are
analyzed in order to estimate the future requirements of the systems. Also, support elements that can be
improved are identified and the effects on the supportability of other program factors are determined.
Another application of supportability trend analysis is to optimize system availability over
operating life. This is done by identifying the support elements that can be improved. Also, the effects
of system reliability and maintainability on supportability are measured, and areas for improvement are
identified.
Candidates used to evaluate system reliability/maintainability/availability support characteristics
include the following:
8.2
(1) Meantimebetweenfailures (MTBF)
(2) Meantimetorepair (MTTR)
(3) Meantimebetweenrepairs (MTBR).
Concerns (in terms of risk, safety, cost, availability, or schedule) and expected benefits should be
the basis for setting priorities when considering using supportability trend analysis.
8.2
Supportability trending parameters should be selected that indicate the effectiveness of the
support elements and the maintainability design factors. Operations and support systems should be
analyzed, if support degrades, to identify items that could lead to a system failure, schedule delay, or
cost increase.
820
8.4.3 Procedures
The procedures (adapted from reference 8.2) to perform supportability trend analysis are as
follows:
(1) Assess the overall operation. Identify parameters that could indicate impending system
failure, cost impacts, and schedule slippages if support functions deteriorate.
(2) Select parameters to be trended. Determine which parameters (identified in step 1) can best
be used to evaluate whether support functions are varying at a sufficient rate to require
management attention. Special consideration should be given to parameters that predict
system safety or success.
(3) Determine if quantitative data are available and adequately represent these parameters.
Supportability parameters may be derived directly from measurements or from calculations
involving two or more measurements. If measurement data are not available, develop a
system to measure the data or eliminate the parameter from the list to be trended.
(4) Establish acceptance limits for the selected parameters. These levels or limits become the
basis for determining if a parameter is in control or corrective action is required. First,
determine the acceptance levels and minimum baselines that define the required level of
support for normal operation. Acceptance limits and minimum support baselines should be
taken directly from program or project support requirements documentation. These
boundaries can also be determined from review of operation, maintenance, and logistics
manuals, and design requirements and specifications documents.
Next, determine action limits that fall within these boundaries, for which corrective action
will be initiated if the action limits are exceeded. Care should be taken in choosing the
action limits so that (1) variation in normal acceptable operation will not cause the action
limits to be exceeded (causing unnecessary expenditure of resources), and (2) corrective
actions can be implemented promptly, once the action limit is exceeded, but before the
boundaries for required support for normal operation are exceeded.
(5) Gather, measure, or calculate the data to be used to trend the selected supportability
parameters. Data sources (extracted from reference 8.2) for supportability trend analysis
may include, but need not be limited to:
a. Equipment problem reports.
b. Work authorization documents.
c. Contractual acceptance records.
d. Shipping and receiving reports.
e. Payment records for maintenance.
f. Transportation records.
g. Inventory records.
h. Issues and turnin records.
i. Training course attendance records.
821
j. Technical documentation error reporting.
k. Consumable replenishment records.
(6) Analyze the selected parameters for trends. Various statistical and graphical techniques for
performing supportability trend analysis can be found in reference 8.3. Use graphical tools
to transform raw, measured, or calculated data into usable information. These graphical
tools can include scatter plots (sec. 5.1), bar charts (sec. 5.3), and control charts (sec. 5.2).
Use statistical tools, such as regression analysis (sec. 6.6), to determine the trend line
through a given set of performance data. Determine how well the trend line fits the data by
using techniques such as R
2
or ChiSquare measure of fit tests. These tests are described in
detail in reference 8.3 and statistical textbooks and handbooks. Use the trend line to detect
if there is a trend that is approaching or has exceeded the action limits established in step 4.
(7) Resolve adverse trends. When an adverse trend has been identified, conduct an analysis for
that trend. A cause and effect diagram (sec. 7.2) may be useful in identifying the root cause
of the adverse trend. Once the cause of the adverse trend is identified, propose a remedy to
correct the problem before the boundaries for required support of normal operation are
exceeded. Implement the remedy (management approval may be required), then continue to
trend the supportability parameter and assess the effectiveness of the remedy.
(8) Report the results. The reports should be published at intervals that will allow management
to take prompt action to correct support problems before they become unmanageable. The
reports should contain sufficient details so that management can accurately assess the risk
to normal operation due to an adverse trend. Suggested reporting formats for common
supportability parameters can be found in reference 8.2.
8.4.4 Example
The following example illustrates supportability trend analysis for inventory control of a specific
spare part. Review of the project support requirements document revealed that at least eight spare parts
were always required. To ensure the inventory never reached this level, an action limit of 10 spare parts
was established. The inventory level for the parts for 11 months in 1988 and 1989 is presented in figure
84. As seen in this figure, the inventory level reached the action level in August 1988. Assessment of
the cause for the low inventory level revealed that usage of the spare parts did not increase, however,
more parts received from the vendor were being rejected in acceptance tests. The corrective action was
to change vendors for the parts. This occurred in September 1988. Tracking the inventory level past the
time the corrective action was implemented revealed that the fix to the support problem was effective.
8.4.5 Advantages
Supportability trend analysis monitors the current health of support systems and forecasts
support problems to enable resolution with minimum adverse effect.
8.4.6 Limitations
Determining the extent of analysis and identifying the appropriate parameter variations that must
be measured can be difficult.
822
M
A
R
A
P
R
M
A
Y
J
U
N
J
U
L
A
U
G
S
E
P
O
C
T
N
O
V
D
E
C
J
A
N
F
E
B
0
5
10
15
20
S
p
a
r
e
P
a
r
t
s
1988  1989
Action Limit
Minimum Required
Inventory Limit
New vendor acquired.
Months
Figure 84. Supportability trend analysis example.
8.4.7 Bibliography
“ASRM Trend Analysis Requirements Document.” Sverdrup Technology, Inc., Report No. 31400491
115, December 1991.
NASA Technical Memorandum 85840, “The Planning and Control of NASA Programs and Resources.”
NHB 5300.4 (1A–1), “Reliability Program Requirements for Aeronautical and Space System
Contractors.”
NHB 8070.TBD, “Significant Problem Reporting System.”
NMI 1103.39, “Role and Responsibilities – Associate Administrator for Safety and Mission Quality
(SMQ).”
NMI 8070.3, “Problem Reporting, Corrective Action, and Trend Analysis Requirements.”
NMI 8621.1, “Mishap Reporting and Investigating.”
Special Study Z001U61, “Marshall Operations Reliability Trend Analysis Standard.” Sept. 16, 1991.
823
8.5 Reliability Trend Analysis
8.5.1 Description
Reliability trend analysis, as described in reference 8.1, performs a parametric assessment of
factors affecting system reliability. The objectives of reliability trend analysis are to measure reliability
degradation or improvement, to predict an outofline failure, to verify design certification limits, to
determine life limits, and to evaluate inspection intervals. Although some parameters will be unique to
reliability, many parameters pertaining to reliability trending also pertain to performance or problem
trending.
Data sources for reliability trend analysis might include new, refurbished, and repaired
component and subassembly acceptance inspection, checkout, and test data for development and
verification and production hardware including, but not limited to:
(1) Alignment data.
(2) Contamination data.
(3) Dimensional data.
(4) Nondestructive test data, e.g., magnetic particle, radiography, penetrant, and ultrasonic
data.
(5) Proof test data, e.g., leak check and hydroproof data.
(6) Functional or performance data, e.g., quantitative and qualitative data.
8.5.2 Application
Reliability trend analysis is best applied in phase E but may also be applied in phase D.
Reliability trending parameters should be selected to indicate changes in the reliability of a system and
explain their causes. These parameters could also be performance or problem trending parameters or
strictly reliability parameters. The criteria for selecting parameters should consider criticality, problem
frequency, engineering judgment, etc. as deemed necessary. Trending parameters should be selected, as
applicable, for each system, subsystem, or component by:
(1) For each parameter, reviewing the FMEA/CIL, contract end item specification, limitedlife
items lists, previous problem reports, original equipment manufacturer’s data, equipment
acceptance data, operations manuals, etc. to determine if it is necessary or beneficial to
perform reliability trending.
(2) Determining the product life indicators necessary to determine the health of the system,
subsystem, or component, e.g., MTBF.
(3) Determining the failure modes pertinent to the system, subsystem, or component.
(4) Determining if time/cycle and failure data are available. Typically, at least 10 failures are
necessary to perform a reliability trend analysis; however, an action limit can be set to
indicate a “failure” data point. At least half of the time/cycle intervals should have an
average of at least one “failure” per time period. (For example, if six time intervals of 2 yr
are chosen, at least three intervals should have at least two failures.) Design/process change
data should be available.
824
(5) If necessary data are not available (e.g., failure and time/cycle data), consider the addition
of data sensors, obtaining alternate data, changing the parameter, or using engineering
judgment for the trend analysis.
(6) Determining if the parameter concerns reusability/repairability or a onetime failure.
8.5.3 Procedures
The only differences between performance and reliability trend analysis are the parameters
trended. Therefore, the procedures to perform reliability trend analysis are same as presented in section
8.1.3 for performance trend analysis.
8.5.4 Example
This example is a plot of reliability trending where the CSF is plotted for the first 20 flight
motors. The lines for the mean ± 3 standard deviations are based on the motors up to number 50 and
give an indication that the later motors have a higher CSF than the first motors.
Also plotted is the minimum CSF allowable by specification (1.5) that shows that this station is
well above the requirement. Most stations do not lie this far above the minimum CSF value.
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
2 4 6 8 10 12 14 16 18 20
0
5
10
15
20
Flight Sequence Number
Upper 3 Limit
Lower 3 Limit
Minimum Specification Limit
Mean
σ
σ
Figure 85. Reliability trend analysis example.
8.5.5 Advantages
Reliability trend analysis measures reliability degradation or improvement and enables the
prediction of possible failures so action can be taken to avert failure.
825
8.5.6 Limitations
Candidate items must be chosen carefully because the analysis can be costly if performed for all
potential problem areas.
8.5.7 Bibliography
“ASRM Trend Analysis Requirements Document.” Sverdrup Technology, Inc., Report No. 31400491
115, December 1991.
NASA Technical Memorandum 85840, “The Planning and Control of NASA Programs and Resources.”
NHB 5300.4 (1A–1), “Reliability Program Requirements for Aeronautical and Space System
Contractors.”
NHB 8070.TBD, “Significant Problem Reporting System.”
NMI 1103.39, “Role and Responsibilities – Associate Administrator for Safety and Mission Quality
(SMQ).”
NMI 8070.3, “Problem Reporting, Corrective Action, and Trend Analysis Requirements.”
NMI 8621.1, “Mishap Reporting and Investigating.”
Special Study Z001U61, “Marshall Operations Reliability Trend Analysis Standard.” Sept. 16, 1991.
826
REFERENCES
8.1 Babbitt, N.E., III: “Trend Analysis for Large Solid Rocket Motors.” AIAA 923357, Sverdrup
Technology, Inc., Huntsville, AL, 1992.
8.2 NHB 8070.5A, “NASA Trend Analysis and Reporting.” Draft, December 1991.
8.3 NASA–STD–8070.5A, “Trend Analysis Techniques.” NASA System Assessment and Trend
Analysis Division, Office of the Associate Administrator for Safety and Mission Quality, October
1990.
827
APPENDIX A
TOOL OR METHODOLOGY SUGGESTION FORM
A1
A2
Toolbox or Methodology Suggestion Form
A3
APPENDIX B
CASE STUDY:
TRIALS AND TRIBULATIONS OF USING SYSTEM ENGINEERING TOOLS
B1
B2
CASE STUDY:
TRIALS AND TRIBULATIONS
OF USING SYSTEM ENGINEERING TECHNIQUES
The Assignment
Charlie Smith came in to work early Monday morning. And why not? He was excited! He’d just
completed a course in System Engineering where they’d shown him all these “techniques” to make his
job easier and less subjective. He’d known about some of the techniques. But he wished he’d had the
course about 10 years ago—back when he was just starting as a systems engineer. Well, no matter...
Today his boss was going to give him a new assignment, and he’d show all of them his newlyfound
proficiency with the toolbox. This should be easy...
His boss, Mr. Jones came in about 9. It had been hard on Charlie, waiting that long, but he had
used the extra time to read his mail, do a little filing, return his phone messages, and write a report.
“Hmmm,” he thought, “maybe I came in a little too early...”
Aw well, his boss, Jones, had finally made it. Without exchanging pleasantries, Jones gave him a
package—“It’s your new assignment, but I don’t see how you’ll do it. The boss wants everything
measurable this time—wants to see how things were decided. Good luck—let me know how it goes.”
With that, Jones left and Smith tore open the package. “A Handson Science Museum display suitable
for grades K–12, for the Museum’s Chemistry Section.” Since Smith was a designer of aerospace
hardware, he really wasn’t sure about this one. What was he supposed to do? What were the ground
rules? Why get this assignment now, just when he was prepared to use all his training to really produce
real hardware, not a vague illdefined thing like this? Smith decided to talk to his boss—this day wasn’t
getting any better.
Jones’ secretary let him know that Jones was gone for the day. He went back to his desk and
found a message from the System Engineering class instructor, Ms. Doe. Puzzled, he called her back,
but he was so preoccupied with his new task that he started right in talking about it.
“Can you imagine, I thought I’d get to use those techniques to build something. Guess I’ll have
to finish this task first though. I think I’ll just pick something and press on. I don’t think the tools apply
here, you know? It’s not really defined enough and I don’t really think the data even could exist, much
less that I could get it. I mean, with a problem like this, there really aren’t any 'data’ to look at anyway!”
Charlie was getting himself kind of worked up.
Ms. Doe (Jane to her friends) almost laughed when she replied, “Buck (Smith’s boss—no one
knew his real name) asked me to call you because he thought you might react this way. Now remember
what we talked about in class. Every problem seems this way at first. The techniques really do add
value, as long as you think about why you’re using them. Tell you what, why don’t you look through the
phase A group, think about it, and we can talk it through tomorrow?”
B3
Phase A—If at First You Don’t Succeed...
After calming down a bit, Smith agreed that this was worth a try, but he really didn’t think it
would work. He hung up, asked his secretary to hold his calls (he liked doing that) and brought out his
matrix. “Let’s see... trade studies, costversusbenefit studies, risk assessment matrix...” No wait, that
risk assessment thing was a secondary for phase A. He carefully crossed it off his list and continued
“benchmarking, cause and effect, checklists, and quality function deployment,” all no good, they were
secondaries. That left brainstorming, Delphi technique, and nominal group technique. Well, that made
five techniques for him to use. Too bad about quality function deployment—he really liked that one, but
he’d follow the priorities Ms. Doe had set—after all, she ought to know.
Smith wanted to be systematic, so he placed the five techniques down in alphabetical order on a
piece of paper:
Brainstorming
Costversusbenefit studies
Delphi technique
Nominal group technique
Trade studies.
He’d start with brainstorming first. Jones was about to ask his secretary to call together his group
when he started feeling a little silly about asking for their help. After all, he wasn’t sure himself what he
wanted and didn’t want to look stupid to his employees. “If only this assignment had been better...,” he
thought. Anyway, (he just wanted to get this done!) he began to brainstorm by himself.
About an hour later he decided it wasn’t going very well. He had been to museums like the one
described in his project, and he was a chemical engineer by trade—but so far he just had a page listing of
potential chemistryrelated topics:
(A) Types of Matter:
Solid – materials
Liquid
Gas
Plasma
(B) Types of materials:
Metal
Ceramic
Polymer
Glass
(C) Building blocks for materials
Largest building blocks for materials
Molecules
Macromolecules, molecules
Smaller building blocks for materials
Atoms
Electrons
Neutrons
Protons
Subatomic particles
Quarks, etc.
B4
(D) Chemistry
(E) Designing Materials.
He didn’t know what kind of exhibit this might make, and brainstorming wasn’t going well. He
remembered from the class that brainstorming was best performed in a group, but he thought again about
looking foolish. Well, maybe he’d try the next technique—let’s see, that was costversusbenefit studies.
There was clearly no way to do costversusbenefit until he figured out what exhibit he was
going to build. He remembered from the class that that technique required data, and he didn’t have any.
He decided not to waste any more time on costversusbenefit—he’d be well prepared to argue with Jane
in the morning.
The next two methods, Delphi technique and nominal group technique, fell to similar fates. He
wasn’t really familiar with them. She must have taught them when he was out, checking in with his
office. That was OK because trade studies, the final technique, was one he liked and had used before.
Smith began by asking himself what the desirable features of a handson museum science exhibit for
chemistry were. He prepared a listing.
Features:
(1) Should be fun as well as educational.
(2) Should accommodate crowds—not just one person (or if one person, should have pretty
good throughput).
(3) Should be sturdy.
(4) Should have to do with chemistry (he’d almost forgotten that one!).
(5) Should fit in a space of...? (he’d have to find out).
(6) Must be ready by...? (he’d have to find out).
(7) Must cost no more than...? (he’d have to find out).
(8) Should be interesting to ages 5 through 18.
(9) Should have minimal consumables (he knew, from experience, that consumables could
destroy a budget).
After writing down these features, Smith realized that features 1 and 8 were kind of motherhood
statements, so he took another cut at figuring what he meant by these:
(1) Should be fun as well as educational.
What makes things fun for kids?
a. It should involve activity, not passive participation (nothing that the kids just watch).
The younger children might have more activity (crawling, climbing, jumping, running
etc.) than the older ones.
B5
b. It might involve winning or high scores or a competition of some sort.
c. It might involve their making something—visible accomplishment was usually fun.
d. It could involve testing of their physical or cognitive powers, but should have a
black/white answer.
e. It should not be perceived as childish—must be something an adult would do—don’t
insult the kids!!!
f. Moving parts were good—he might want to stay away from solely computerized
stuff.
g. High tech was good, maybe having to do with exploration or environmental themes—
tie it to something they knew something about.
h. If he wanted to get them to do it again and again, it should be something where they
might measure improvement, or at least get a different result with a different effort or
problem.
Smith sat back, well pleased with this listing. He realized that feature 8 was pretty well covered
by his current list and decided not to work on it separately. He wanted a little refresher on the trade study
methodology before he went on, so he pulled out his toolbox. Let’s see... page 23 (section 2.1.3) said to
“Define the mission objective and requirements for the system under consideration.” All right, he’d done
that, now what? “Identify credible alternative candidates for the system under consideration”—that’s
what the toolbox said, but how could he do that when he didn’t know what he was building? This
toolbox thing wasn’t as helpful as he thought. Smith packed up for the day and headed home—tomorrow
he was going to have a serious talk with Jane. She clearly hadn’t taught this stuff right and anyway, why
was Buck calling her about his new assignment, and why couldn’t it have been a better one, and... Oh
well, he’d save all that for tomorrow.
Phase A—...Try, Try Again
It was a bleak, rainy Tuesday morning. Smith’s brooding sleep had been interrupted often by the
sharp concussions of thunderstorms. He was going to be ready for Jane Doe! He arrived at work and
pulled together his files. His secretary had managed to get some additional information on the science
exhibit—the space allowable was approximately 3,000 ft
2
, and his timeframe was approximately 18 mo
until, it, the museum opened. She had left a note saying that there was still no hard data on his budget
but it would likely be on the order of $400,000. Well, that was something anyway. He checked his
calender and found that Jane Doe would be there in about 15 min. He used the time to prepare himself.
Jane arrived on time, wet, and most infuriating of all, cheerful. “So how did it go yesterday?”
Smith began, in a controlled but bitter tone: “Poorly. The tools didn’t work very well—there isn’t
enough data. I went systematically by your procedure—which by the way eliminated some things I
thought would be very useful—and I don’t think I’ve made any real progress. Another thing...”
B6
Jane interrupted him here, with that nowveryannoying laugh, “Slow down, slow down, let’s
take it one thing at a time. I’ve got all morning, and I think we can make this work. If not, I’ll talk to
Buck about it. Deal?”
Smith couldn’t say “no” to that. He figured with just one morning’s effort he’d be able to show
Jane that this wasn’t going to work—then it would be someone else’s problem. “Deal.”
They sat down at Smith’s desk. He asked his secretary to hold all calls. (He really liked doing
that.)
Smith showed Jane the results of his technique downselection—the alphabetized listing he’d
taken from the course toolbox. Jane began, “Charlie, you have to remember, the matrix on page 17 is
just a guide. You didn’t have to use all of those techniques that were marked priority 1, or ignore the
priority 2’s and the unmarked items for that matter. But, since that’s how you started, how did it go?”
Jane wasn’t wasting any time.
Smith told her of his difficulties in brainstorming and his concern for calling a team together
before he had anything planned out. She acknowledged that this was one of the shortcomings of the
brainstorming technique, and she understood—but didn’t seem to agree with—his reluctance to pull a
team together. She didn’t want to talk about costversusbenefit—she agreed that it wasn’t yet
appropriate and congratulated Smith on not trying to forcefit the technique to an inappropriate
application. This was not what Smith had expected. They skipped quickly over the next two techniques.
She explained they were sort of variations on brainstorming anyway, and got right to his trade study.
Smith was quite confident of his knowledge of this technique and felt secure that he could show Jane
once and for all that this project was just inappropriate for his newly mastered skills.
Jane read his nine features without comment. She then looked at his breakout for feature 1 and
frowned a bit. Smith didn’t want to lose his opportunity (she was turning out to be pretty hard to corner).
And he didn’t like that frown one bit. As soon as she looked up he let her have it. “You see, I followed
the guideline for trade studies—and by the way, I’ve been using them for many years—and couldn’t get
past the second step. How do I know what specifics to trade when I don’t have any specifics? And how
can I develop any specifics without data? I just don’t see how this thing is supposed to work!”
Jane’s response surprised Charlie. “These techniques are only to be used where they can help,
and you’re the only one who can decide where that is. They don’t replace data. In fact, many of them
may highlight where data are required, or just how much you don’t know. But, with your specific
problem, I have a few questions. I would have thought things like safety and access for handicapped
would be highpriority features. Also, what about education—you’ve analyzed some things that clarify
fun but what are the specific educational aspects that you’d like to focus on? I think a focus on that
might help a lot.”
Charlie knew that throughout the class she’d discussed using the toolbox as a guideline, and that
it wasn’t necessary to use each technique. He just hadn’t trusted his own knowledge of the toolbox well
enough to select against the toolbox guidance—cookbooks and gobys were a lot easier. OK, he’d give
her that one. That bit about safety and handicapped access—those were good and he added them to his
listing as features 10 and 11, respectively. As for the educational aspects, that was a great observation.
Together they began to make up a listing. It didn’t go very well at first, so they called in Dalton from the
Applied Chemistry department. After about an hour, they had a listing for the top 15 educational areas
that they wished to focus on:
B7
Educational Areas:
(1) Demonstrate units of mass, length, volume, temperature etc.
(2) Demonstrate intrinsic (color, viscosity, melting point, hardness, density...) versus extrinsic
properties (size, shape temperature...). Note intrinsic properties are also known as physical
properties.
(3) Demonstrate chemical properties (the tendency of the substance to change, through
interactions with other substances or singly).
(4) Demonstrate chemical change (new substance is formed) vs. physical change—include
exothermic and endothermic changes.
(5) Demonstrate elements, compounds, mixtures, and solutions.
(6) Demonstrate the states of matter: solid, liquid, gas, plasma.
(7) Demonstrate the laws of conservation of mass and energy.
(8) Provide a feel for Avogadro’s number.
(9) Demonstrate crystalline nature of many solids.
(10) Demonstrate the nature of polymer chains.
(11) Demonstrate the nature of metals and semiconductor materials.
(12) Demonstrate the principles of catalysis.
(13) Demonstrate the principles of combustion.
(14) Demonstrate the special nature of organic chemistry.
(15) Demonstrate the standard and quantum theory for the atom.
Smith knew from experience that the next thing to do was to combine these with the features
listing and see which areas were likely to make exhibits and which might combine, etc. But this sounded
like a combination of brainstorming and trade studies and checklist all twisted together. He asked Jane if
that was all right. She explained that there was no problem, as long as what they did was documented
and reasonably systematic. Charlie felt more like he had while in class—he was starting to get the hang
of this, again. They decided to brainstorm potential exhibits for each of the 15 specific educational areas,
and then use the features as a checklist to see if they were satisfied.
Charlie rewrote the features, renumbering and eliminating items, as appropriate:
(1) Should accommodate crowds—not just one person (or, if one person, should have pretty
good throughput).
(2) Should be sturdy.
B8
(3) Must be ready in 18 mo.
(4) Should be interesting to ages 5 through 18.
(5) Should have minimal consumables (he knew, from experience, that consumables could
destroy a budget).
(6) It should involve activity, not passive participation (nothing that the kids just watched). The
younger children might have more activity (crawling, climbing, jumping, running etc.) than
the older ones.
(7) It might involve winning or high scores or a competition of some sort.
(8) It might involve their making something—visible accomplishment was usually fun.
(9) It could involve testing of their physical or cognitive powers but should have a black/white
answer.
(10) It should not be perceived as childish—must be something an adult would do—don’t insult
the kids!!!
(11) Moving parts were good—he might want to stay away from solely computerized stuff.
(12) High tech was good, maybe having to do with exploration or environmental themes—tie it
to something they knew something about.
(13) If he wanted to get them to do it again and again, it should be something where they might
measure improvement, or at least get a different result with a different effort or problem.
(14) Must be safe.
(15) Should be handicappedaccessible.
He then rewrote the educational goals, indexing them by lettering them to avoid confusion with
the numbered features list:
a. Demonstrate units of mass, length, volume, temperature, etc.
b. Demonstrate intrinsic (color, viscosity, melting point, hardness, density...) versus extrinsic
properties (size, shape, temperature...). Note intrinsic properties are also known as physical
properties.
c. Demonstrate chemical properties (the tendency of the substance to change, through
interactions with other substances or singly).
d. Demonstrate chemical change (new substance is formed) versus physical change—include
exothermic and endothermic changes.
e. Demonstrate elements, compounds, mixtures, and solutions.
B9
f. Demonstrate the states of matter: solid, liquid, gas, plasma.
g. Demonstrate the laws of conservation of mass and energy.
h. Provide a feel for Avogadro’s number.
i. Demonstrate crystalline nature of many solids.
j. Demonstrate the nature of polymer chains.
k. Demonstrate the nature of metals and semiconductor materials.
l Demonstrate the principles of catalysis.
m. Demonstrate the principles of combustion.
n. Demonstrate the special nature of organic chemistry.
o. Demonstrate the standard and quantum theory for the atom.
Phase B—Starting to Have a Ball
Applied Chemist Dalton suggested that they divide the educational goals among several people.
Charlie agreed, and decided to work the problem with product development teams. He quickly formed
several teams and parcelled out the work. That took some explanations! He’d selected Dalton and Jane
for members of his team, along with design artist Mike Angelo, and a marketing executive who worked
their company’s precollege outreach efforts, Hewitt Wissard. Their task was to develop exhibits for
items h, j and k. Jane facilitated the brainstorming session, and by lunchtime they had several concepts
developed for each of the educational areas. Charlie copied the concept suggestions down from the
yellow stickies they’d used in brainstorming:
Provide a feel for Avogadro’s number (item h)
(1) Build a “ball pit” where the number of balls was some percentage of Avogadro’s number
and smaller kids could play. (Hewitt had seen something like this at a pizza place and his
daughter liked it a lot.)
(2) Have a sugar bed filled with grains of sugar that were some percentage of Avogadro’s
number. This could also be used for experiments (of some sort) and for microscopy when
discussing the crystal educational area. Maybe used for eating, too.
(3) Develop some kind of strengthtest thing where kids could compete to get close to
Avogadro’s Number on a scale or something. (Jane really wasn’t a scientist, but in
brainstorming, everyone’s input could be important).
B10
Demonstrate the nature of polymer chains (item j)
(1) Have microscopes set up to look at polymer crystals.
(2) Have a sort of maze set up that was partially amorphous and partially crystalline, like some
polymers are. Let the kids walk through it.
Demonstrate the nature of metals and semiconductor materials (item k)
(1) Have a large blast furnace that the kids could use to heattreat metals, and then measure the
resultant properties using an Instron tester. Also have water, oil, and salt quenching baths.
(2) Set up something where they could provide various amounts of dopant to semiconductor
crystals, and then measure the resistance etc.
(3) Have a display showing the crystal structure and how semiconductors and metals work
(electrically).
(4) Have polishing wheels set up with microscopes so they could polish specimens and look at
grain structure and stuff.
They were far from done, but it was a good start. When Jane asked Charlie if he still wanted her
to talk to Buck, he was surprised. He’d forgotten the deal during the long morning’s work. “No thanks, I
admit we’re making pretty good progress. I guess we might even start some phase B work this
afternoon, huh?”
“Sure,” replied Jane, “but you’re on your own for a while. I’ve got appointments for the next
couple of days. I think you’ll do fine, and if you run into problems, I’ll be back Thursday afternoon.
OK?”
“Not quite,” said Charlie with his own laugh, “I’ll see you Friday morning for a team meeting.
OK?”
“OK,” laughed Jane, and they all went out to lunch.
Friday came, and Charlie was impressed with the work they had accomplished. After several
revisions, they had centered on working on the Avogadro’s number thing, the one with the “ball pit.”
The decision had come after long discussion, and an aborted attempt to perform a weighted average
trade study to help quantify the results and the decisionmaking process. When Jane came in, Charlie
(and the rest of the group) was eager to find out what they had done wrong in using the trade study
methodology—although Charlie wasn’t interested in arguing this time. He was kind of looking forward
to working with Jane again. They showed Jane the brief attempt at a trade study that they had
formulated:
B11
Item Criteria Attribute Weight Factor
1 Avogadro’s number (H) 30
2 Fun (4,6,7,8, 30
9,10,11,12)
3 Cost (5) 25
4 Safe (14) 10
5 Variable outcome (13) 5
Charlie explained that the letter and numbers in parenthesis referred back to the specific feature
and the educational goal delineations they had previously produced. He was pleased with this, as he
thought it quite sensible to have tied the criteria attributes to the required features/goals that they had
agreed upon. Jane agreed that their approach did represent a very logical progression, but Charlie saw
her half smile again. By now he knew that meant they had made an error, and she had spotted it.
“Go on,” was all Jane said.
They let Jane know that they had spent a good deal of time discussing the weighting criteria; it
represented a group consensus. They then showed her the estimates for each alternative relative to each
criteria attribute. They had used the TQM techniques of ranking each of the qualitative items as 1, 5, or 9
to allow separation of results. These rankings, too, were consensus:
Item Criteria Attribute Weight Measure of Alt 1 Alt 2 Alt 3
Factor Effectiveness
1 Avogadro’s number 30 Obvious 9 9 9
2 Fun 30 See features 9 9 9
3 Cost 25 ROM estimate 9 9 5
4 Safe 10 Standard stuff 9 9
9
5 Variable outcome 05 Obvious 9 9 9
They hadn’t bothered to calculate the scores or weighted scores. It was obvious that it would come out a
wash.
Jane was wearing a large smile now. She said to Charlie, “I think you know where the problem
was, but I don’t think you recognize the value of what you did! Let’s start with the problem. Tell us
why you think it didn’t work.”
B12
Charlie was again caught off guard—he wished she’d stop doing that—but he answered her
question. “I don’t think our designs were well formulated enough to be able to evaluate and score. I
don’t think we did a very good job on defining quantitative, objective criteria attributes. But for this type
of effort, I’m not sure how to do any better. So how can you use a trade study then, when you’re at this
stage of a design? Why was it one of the recommended tools?”
Jane’s eyes glittered with her smile as she began, “You’re right about the criteria attributes. One
way might have been to simply count the features (an example: How many of the features defining fun
were incorporated under each alternate?) and quantify how many the alternate would have met. But that
wouldn’t have solved your fundamental problem. You’re right, the designs really weren’t mature enough
for this type of study to give a clear selection. To evaluate ‘Safe,’ a PHA would really be required,
which means you have to have at least a preliminary design. To evaluate ‘Fun’ and ‘Cost,’ the same
level of maturity is also necessary. But, what I don’t think you’ve realized is by performing this study,
you were able to identify that. At this stage of design maturity, no concepts were inappropriate. The fact
that nothing washed out gave you a valuable answer, and let you choose based on ‘gut feel’—what’s
often called ‘engineering’ or ‘programmatic judgement.’ Further, you elected to quantify your
qualitative feel for the alternate by using the 1,5,9 technique. I think you guys did just great! You
wouldn’t have tried to be specific about why you had selected one idea to pursue if you hadn’t had these
techniques—you knew intuitively that there weren’t enough data to use appropriate criteria. These
techniques won’t let you do things that can’t otherwise be accomplished. They’re just an aid. And I think
you did great. When it wasn’t helping, you tried something else. Which one did you wind up selecting,
anyway?”
“Alternate 1 the ball pit,” replied Charlie. “Now I thought we might flowchart the effort
required for phase B to figure out where we’re going with this. You know—the planyourwork and then
workyourplan kind of stuff.”
After some long discussions over the blank page that they were trying to use to do a flowchart,
Jane suggested that a good way to get started might be to flowchart what they had already done.
Although it seemingly added nothing to their completed tasks, she noted it was often easier to add to
something, and even easier to edit or change something, than it was to create it in the first place. Starting
the flowchart with the efforts they had already accomplished would give them a base to add upon, rather
than the nowbeginningtobeannoying blank page. They agreed and by the end of the day the group
had produced a reasonable flowchart (see figure B1.) Much of the day had been spent on trying to
define which tools would be used. This time they only used the toolbox as a guide and kept asking,
“Will this tool help me? Do I need it...?” Their flowchart represented their choices—to their surprise it
also provided some insights to the design process.
Many of the phase A decisions were management/programmatictype decisions that held
significant consequences for the remainder of the effort. It was also true that most of the data required to
support credible costversusbenefit or riskversusbenefit trade studies did not exist at this stage. Charlie
began to hold an even greater appreciation for the toolbox—not so much for the major type decisions,
but for the potential to reveal the more subtle ramifications of decisions that might otherwise go
unnoted. He spoke his thoughts to the group, noting that these subtleties were particularly critical at the
beginning of a project. He received the typical reaction by a group to someone who speaks the
obvious—they laughed.
B13
Prepare Requirement's Checklist:
Features and Goals
Form PDT and Divide Work
Brainstorm Concepts
Downselect Single Concept
Receive Assignment
Costvs.Benefit Studies
Delphi Technique
Nominal Group Techniques
Trade Studies
Select Tools for Phase A
Is the
technique
primarily
for Phase A?
Don't Use Technique
No
Yes
Attempt to use
the selected
technique
Selection Process that Was Used
Brainstorming
Toolbox Recommendations
Technique Familiarity (Personnal)
Judgement (schedule, criticality, resources etc...)
Yes
No
Review Potential tools
from Handbook for applicability
Will use of
the selected
technique
add value?
Attempt to use
the selected
technique
Don't Use Technique
Selection Process that Should Have Been Used
Prepare Features Prepare Goals
Brainstorming Brainstorming
Trade Study
B
B
Delineate Consolidated Checklist for Goals
and Features
A A C
Brainstorming
Brainstorming
WeightedAverage Trade Study
Phase A Flowchart: Note most decisions are
Programmatic / Management decisions with
a paucity of data and farreaching consequences.
Yes
No
Is the
Feature
quantifiable?
Additional Step that Should
Have Been Added
C
B
A
Yes
No
Are the
features well
defined and mutu
ally exclu
sive
Selection Process that Was Used
to determine Features / Goals
A
B
D D
E
(a) Phase A.
Figure B1. Flowchart—Continued
B14
E
Perform Preliminary Hazard Analysis
Flowchart the Proposed Process
Review Potential tools
from Toolbox for applicability
Toolbox Recommendations
Technique Familiarity (Personnal)
Judgement (schedule, criticality, resources etc...)
Yes
No
Will use of
the selected
technique
add value?
Include selected
technique in
flowchart
Don't Use Technique
Perform Proposed Studies on Selected Design, and Iterate Design
Perform Probabilistic Risk Assessment
Perform Reliability Block Diagram
Perform Digraph Analysis
Setup Strawman Design for Selected Concept
Use marble size balls
Pit is 20 x 20 x 3 foot deep
Use radium tracers for diffusion modeling
Use Radio wrist bands for children
Special pit area used for close packing demo
F
F
F
F
G
Yes
No
Yes
No
F
Does
technique
indicate a design
problem?
Have
all analyses
been completed?
G
G
H
Phase B Flowchart: Note from post "flowchart creation" step on, most
analyses and decisions are technical and disciplinespecialist oriented.
(b) Phase B.
Figure B1. Flowchart—Continued.
B15
Jane wasn’t laughing though. She noted that there was a change from the fundamental desirable
attributes of a phase A program manager to those of a phase B manager. Decisions at the phase A level
really required overall system experience and a capability for accepting available data, determining the
appropriate level for additional data, and decision selection from the minimum, adequate (often very
sparse) pool which that represented. Phase B efforts required a much more detaildriven selection
process which employed the talents and capabilities of discipline specialists—management did well as a
cheerleader here but might often leave the technical and some of the programmatic optimization
decisions to the concurrent engineering team working the problem. Phases C and D were similar in
nature to
phase A.
Charlie began to feel a little better about things. He also noted that brainstorming was a
technique that was seemingly suitable almost everywhere. He and Jane decided to use it to select a
restaurant for dinner, after the rest of the group had gone home.
Charlie had gotten home very late Friday night, he was still sleeping when his boss’s secretary,
Fawn Hunter, called Saturday morning to let him know that first thing Monday morning Buck wanted to
see what kind of progress they had made and discuss the project. He thanked Fawn for the information
and got up to do some more work on the flowchart, and to try to start some of the other tools. It was
looking like it was going to be a long weekend. About an hour later Jane called, volunteering to help.
Charlie was the one smiling now—the prospects for the weekend were looking up. He liked the phase A
and B flowcharts and added some of the symbols from the toolbox, just to key in on the difficulties for
those particular areas. He also added the “should have done” boxes in addition to the “as performed”
flows, and changed the phase B flowchart accordingly.
Charlie ran the calculations for the number of marbles in the ball pit:
Calculation of marble volume:
Volume of a sphere = (4/3) π r
3
r
marble
= 0.5 in
r
3
marble
= 0.125 in
3
V
marble
= 0.52 in
3
.
Calculation of ball pit size:
Assume 20 ft×20 ft×3 ft
V
ballpit
= 1,200 ft
3
×1,728 in/1 ft
3
= 2.07×10
6
in
3
.
The maximum packing density for spheres of a single diameter is 74 percent.
Calculation of number of marbles to “fill ball pit:”
No.
marbles
= (0.74) 2.07×10
6
in
3
/0.52 in
3
= 2.95×10
6
.
B16
Although that was a huge amount of marbles (Charlie started wondering about the feasibility and
cost of that many marbles), it didn’t begin to approach Avogadro’s number. Charlie was still at a loss for
how to relate the two, and the diffusion part was still evading him. But now that they had this much of a
start, he and Jane decided to perform the PHA. Once again it was time for brainstorming.
They filled up the first page with a list of 11 hazards (see figure B2), first listing all of them,
then identifying the targets for each of them, then working the severity and probability and risk factor. In
this way they were able to concentrate on the ideas first, without getting caught up in the specific
assessment issues or countermeasures. They used the toolbox risk assessment matrix (on page 310) that
had been adapted from MIL–STD–882C. Jane suggested that they only work countermeasures for those
items with a risk priority code of 1 or 2. There wasn’t any need to improve on those with a 3. Charlie
was quite surprised to find that the marbles represented such a significant risk. They settled on a design
change for that item.
After filling in the countermeasures and the new risk Priority Codes, they were left only with two
items of code level 2. Charlie didn’t know how to work these and neither did Jane. Jane did mention
though that they might just be a function of the integrated exhibit area (IEA)—disease transmission in
children’s museums was a common occurrence and wherever children jumped, someone was liable to
get jumped on. They decided to go eat a late lunch, pizza, and watch one of these ball pits in action.
After returning from lunch, Charlie did calculations for larger balls. He had gotten the idea of filling the
balls with sand or salt to better compare with Avogadro’s number. This also might be useful for partially
addressing the crystal educational goal. He and Jane worked the new calculations for the larger balls,
and for a salt filler.
Calculation of large ball volume:
volume of a sphere = (4/3) π r
3
r
ball
= 3.5 in
r
3
ball
= 4.29×10
1
in
3
V
ball
= 1.8×10
2
in
3
.
The maximum packing density for spheres of a single diameter is 74 percent.
Calculation of number of balls to “fill the ball pit:”
No.
balls
= (0.74) 2.07×10
6
in
3
/1.8×10
2
in
3
= 8.52×10
3
.
Volume of a grain of salt:
volume of a cube = abc
assume (a) is approximately equal to (b), which is approximately equal to (c)
assume a = 0.01 in.
B17
B18
Brief Descriptive Title (Portion of System/Subsystem/Operational Phases covered by this analysis):
Probability Interval: 25 years
H
a
z
a
r
d
T
a
r
g
e
t
*
S
e
v
e
r
i
t
y
P
r
o
b
a
b
i
l
i
t
y
R
i
s
k
C
o
d
e
Description of Countermeasures
S
e
v
e
r
i
t
y
P
r
o
b
a
b
i
l
i
t
y
R
i
s
k
C
o
d
e
Approved by/Date:
Preliminary Hazard Analysis
Identify countermeasures by appropriate code letter(s):
D = Design Alteration E = Engineered Safety Feature
S = Safety Device W = Warning Device
P = Procedures/Training
Analysis: Initial
Revision Addition System Number: __________
Date:
Hazard No. / Description
Risk
Before
Risk
After
Prepared by/Date: *Target Codes: P—Personnel
T—Downtime
E—Equipment
R—Product V—Environment
February 1994
X
2. Fall in pit. P III A 2
E
T
P  Have posted rules and trained monitors.
D  Use soft balls and padded surfaces.
P III B 2 III E 3
1. Cut hand on sharp edges.
D Place bumpers on all potential sharp edges.
P I D 2 I E 3 3. Suffocate under balls.
D  Use larger balls.
E  Limit use to children 3 feet tall or taller.
P I C 1
I F 3 4. Suffocate on marbles. D  Use larger balls.
P II E 3 5. Cancer risk from radium.
P III D 3 6. Health risk from radio waves.
P III A 2 III A
7. Children landing on each other.
P  Have posted rules and trained monitors. 2
P  Limit age of children in the pit to 7 years old or less.
P III D 3
8. Balls (marbles) breaking and leaving sharp
objects in pit.
9. Big children landing on small children or pit walls. P III B 2
IV C
T
P
III A 2
III A
10. Risk of disease communication. P  No very effective countermeasure .
2
P III A 2
III C 3
11. Balls becoming projectiles in and out of pit. P  Have posted rules and trained monitors.
Ball Pit for Science HandsOn Museum Display
T
C II 3
3
IV
IV
A
A
3
3
D  Use padded surfaces.
III D 3
IV B 3
IV C
C
3
3
IV
III C 3
B18
V
salt grain
= (0.01in)
3
= 1.0×10
–6
in
3
.
Calculation of the number of grains of salt to “fill ball pit:”
No.
salt
grains
= 2.07×10
6
in
3
/1.0×10
–6
in
3
= 2.07×10
12
.
The assumption was made that a ball has zero wall thickness and the salt grains will “magically
stack” in a sperical container.
Calculation of the number of grains of salt per ball:
No.
grains/ball
= (V
ball
/V
salt
) = (1.8×10
2
in
3
/1.0×10
–6
in
3
)
= 1.8×10
8
grains/ball
Calculation of number of grainfilled balls required to house Avogadro’s number of grains:
No.
balls
= Av= (1.8×10
8
grains/ball
) (x) = 6.02×10
23
= 3.34×10
15
.
The maximum packing density for spheres of a single diameter is 74 percent.
Calculation of required ball pit volume to contain 3.34×10
15
balls:
V
ballpit
= (V
ball
) (No.
balls
) = (1.8×10
2
in
3
) ( 3.34×10
15
)/0.74
= 8.12×10
17
in
3
= 3,195 mi
3
Calculation of cube side required to make a cube of volume = 2,364 miles
3
:
Side = (3,195 miles
3
)
1/3
= 17.97 mi.
There. They had made some major progress, and Charlie was beginning to visualize this exhibit. He
knew they were ready to talk with Buck on Monday. He did want to find out about using the PHA tool
for programmatic risk evaluation, and he had begun doodling with some small fault trees and was
impressed by what he could do with them. He had already pretty much decided not to do the PRA
assessment and....
Epilogue...Two (and a Half) for the Show
Opening day of the museum was a media event. There were speeches and interviews and plenty
of good words for all. Mr. and Mrs. Smith stayed in the background letting others soak up the limelight.
They were pleased and proud of what they had done, and excited that their soontobeborn child would
get to visit the museum often. Those lifetime passes for their family turned out to be a great wedding
gift! Charlie was putting together a short report on the lessons learned during those first few months of
the project—Jane was going to use it as a case study during her next class on the toolbox. He had left it
at home for Jane to read, she smiled again as she recalled the listing:
B19
(1) The toolbox is just a guide. Use techniques that have value specific to the requirements, not
simply because they are available or recommended.
(2) Don’t be afraid to use techniques that you’re unfamiliar with—but get expert help when
required! Anything can be misused.
(3) Expect to make mistakes enroute to success. Learn to recognize and correct them.
(4) Using the techniques does not mitigate the need for facts and data—rather it better defines
the need (garbage in—garbage out).
(5) Brainstorming is almost universally useful.
(6) When she smiles, my wife is always right.
B20
APPENDIX C
GLOSSARY OF TERMS
C1
C2
GLOSSARY OF TERMS
Te r m De f i n i t i o n
Analysis An examination of the elements of a system; separation of a whole
into its component parts. (Reference Section 4.1)
AND Gate A logic gate for which an output occurs if all inputs coexist. All
inputs are necessary and sufficient to cause the output to occur.
(Reference Section 3.5)
Backwards Logic The mental process in which an analyst models a system by
repeatedly asking the question, "What will cause a given failure to
occur?" Also called topdown logic. (Reference Section 3.0)
Barrier A countermeasure against hazards caused by a flow from an
energy source to a target. (Reference Section 3.3).
Basic Event An initiating fault or failure in a fault tree that is not developed
further. Also called an initiator or leaf. These events determine
the resolution limit for a fault tree analysis.
Cause The event or condition responsible for an action or result.
(Reference Section 3.10)
Common Cause A source of variation that is always present; part of the random
variation inherent in the process itself.
Consequence Something that follows from an action or condition; the relation of
a result to its cause. (Reference Section 3.10)
Control Limits Limits (also called action limits) set between the mean or nominal
values of a parameter and specification limits. If a control limit is
exceeded, corrective actions may need to be implemented before
the specification limit is exceeded. (Reference Section 5.2)
Countermeasure An action taken or a feature adopted to reduce the probability
and/or severity of risk for a hazard. (Reference Sections 3.2 and
3.4))
Creative Function The means of seeing new ways to perform work by breaking
through barriers that often stifle thinking. Some techniques that
are considered creative tools are evolutionary operation (Section
7.6), brainstorming (Section 7.7), and nominal group technique
(Section 7.10). (Reference Table 11)
C3
GLOSSARY OF TERMS
Te r m De f i n i t i o n
Critical Items List
(CIL)
A FMEAderived list (published as FMEA/CIL) containing system
items that have a criticality of 1 or 2, and items that are criticality
1R or 2R and fail redundancy screens. (Reference Section 3.4)
Criticality In reference to a parameter, criticality is the level of importance the
parameter has to the operation of the system. (Reference Section
3.4)
Customer The internal or external person or organization that is the user of a
product being produced or service being rendered. The immediate
customer is the user of the product or service in the next step of the
process.
Cut Set Any group of fault tree initiators which, if all occur, will cause the
TOP event to occur. (Reference Section 3.6)
Data Analysis
Function
The means of analyzing a process by using a data display. Some
techniques that are considered data analysis tools are checklists
(Section 7.8), control charts (Section 5.2), and force field analysis
(Section 7.11). (Reference Table 11)
Decision Making
Function
After analyzing all available data, a decision is made on how to
optimize the subject process. Some techniques that are considered
decision making tools are benchmarking (Section 7.1), nominal
group technique (Section 7.10), and force field analysis (Section
7.11). (Reference Table 11)
Degrees of Freedom The number of independent unknowns in the total estimate of a
factorial effect or a residual. (Reference Section 6.2)
Facilitator A person trained in group dynamics and problemsolving
structures who assumes the responsibility for ensuring a full
exchange of information between team members. (Reference
Section 7.2)
Factor A parameter or variable that affects product/process performance.
(Reference Section 6.2)
Fail Safe Proper function is impaired or lost but no further threat of harm
occurs. (Reference Section 3.4)
Failure A fault owing to breakage, wear out, compromised structural
integrity, etc. (Reference Section 3.4)
C4
GLOSSARY OF TERMS
Te r m De f i n i t i o n
Failure Domain In analysis work, failure domain refers to an analysis that seeks
the probability of a system not operating correctly. (Reference
Section 3.8)
Failure Mode The manner in which a failure occurs, i.e. the manner in which it
malfunctions. (Reference Section 3.4)
Failure Propagation
Path
The sequence of events that leads to an undesirable event or loss.
Also called an accident sequence.
Fault Inability to function in a desired manner, or operation in an
undesired manner, regardless of cause. (Reference Section 3.6)
Forward Logic The mental process in which an analyst models a system by
repeatedly asking the question, "What happens when a given
failure occurs?" Also called bottomup logic. (Reference Section
3.0)
Graphical Function The means of analyzing the data of a process by applying graphs
and/or charts. Some of the techniques that are considered
graphical tools are cause and effect diagram (Section 7.2), control
charts (Section 5.2), and quality function deployment (Section
7.12). (Reference Table 11)
Hazard An activity or condition which poses a threat of loss or harm; a
condition requisite to a mishap. (Reference Section 3.2)
Intermediate Event An event that describes a system condition produced by preceding
event and contributing to later events.
Mean The term used to describe a sample population average.
(Reference Section 6.1)
Mean Square
Deviation (MSD)
A measure of variability around the mean or target value.
Mishap An undesired loss event. (Reference Section 8.3)
Modeling Function The means of analyzing and modeling a process against standards
and/or other processes. Some of the techniques that are
considered modeling tools are benchmarking (Section 7.1), quality
function deployment (Section 7.12), and work flow analysis
(Section 7.16). (Reference Table 11)
C5
GLOSSARY OF TERMS
Te r m De f i n i t i o n
OR Gate A logic gate in which an output occurs if one or more inputs exist.
Any single input is necessary and sufficient to cause the output to
occur. (Reference Section 3.5)
Parameter The term applied to population or sample characteristics such as
the mean and standard deviation. (Reference Section 5.2)
Path Set A group of fault tree initiators which, if none of them occurs, will
guarantee that the TOP event cannot occur. (Reference Section
3.6)
Population The universe of data under investigation from which a sample will
be taken. (Reference Section 6.1)
Preliminary Coming before and usually forming a necessary prelude to
something. As in a preliminary hazard analysis, the analysis can
be performed in the design or preoperation phase, or it can be the
first analysis performed for a mature system. (Reference Section
3.2)
Prevention Function The means of analyzing data to be able to recognize potential
problems and prevent the process from heading in an adverse
direction. Some of the techniques that are considered preventive
tools are control charts (Section 5.2), Pareto analysis (Section
5.6), and design of experiments (Section 7.5). (Reference Table
11)
Probability The liklihood an event will occur within a defined time interval.
(Reference Section 3.14)
Problem
Identification
Function
The means of identifying potential problems from a data display as
a result of an analysis of the process. Some techniques that are
considered problem identification tools are control charts (Section
5.2), brainstorming (Section 7.7), and quality function
deployment (Section 7.12). (Reference Table 11)
Process A series of events progressively moving forward over time to
produce products or services for a customer. (Reference Section
7.1)
Project Phase A The conceptual trade studies phase of a project. Quantitative
and/or qualitative comparison of candidate concepts against key
evaluation criteria are performed to determine the best alternative.
(Reference Section 1.3)
C6
GLOSSARY OF TERMS
Te r m De f i n i t i o n
Project Phase B The concept definition phase of a project. The system mission and
design requirements are established and design feasibility studies
and design trade studies are performed during this phase.
(Reference Section 1.3)
Project Phase C The design and development phase of a project. System
development is initiated and specifications are established during
this phase. (Reference Section 1.3)
Project Phase D The fabrication integration, test, and evaluation phase of a project.
The system is manufactured and requirements verified during this
phase. (Reference Section 1.3)
Project Phase E The operations phase of a project. The system is deployed and
system performance is validated during this phase. (Reference
Section 1.3)
Qualitative Data that are not numerical in nature. (Reference Section 2.1)
Quantitative Data that are numerical in nature or can be described numerically.
(Reference Section 2.1)
Range A measure of the variation in a set of data. It is calculated by
subtracting the lowest value in the data set from the highest value
in that same set. (Reference Section 5.2)
Raw Data Data as measured or as taken directly from instruments or sensors.
(Reference Section 8.4)
Reliability The probability of successful operation of a system over a defined
time interval. (Reference Section 3.3)
Risk For a given hazard, risk is the longterm rate of loss; the product
of loss severity and loss probability. (Reference Section 3.1)
Sample One or more individual events or measurements selected from the
output of a process for purposes of identifying characteristics and
performance of the whole. (Reference Section 6.1)
Severity The degree of the consequence of a potential loss for a hazard.
(Reference Section 3.1)
Special Cause A source of variation that is intermittent, unpredictable, unstable;
sometimes called an assignment cause. It is signalled by a point
beyond the control limits. (Reference Section 8.1)
C7
GLOSSARY OF TERMS
Te r m De f i n i t i o n
Standard Deviation A measure of variability used in common statistical tests. The
square root of the variance. (Reference Section 6.1)
Subassembly A composite of components. (Reference Section 3.4)
Success Domain In analysis work, success domain refers to an analysis that seeks
the probability of a system operating correctly. (Reference Section
3.8)
System A composite of subsystems whose functions are integrated to
achieve a mission (includes materials, tools, personnel, facilities,
software, and equipment).
System Element A constituent of a system that may be a subsystem assembly,
component, or piecepart.
Target An object having worth that is threatened by a hazard. The object
may be personnel, equipment, downtime, product, data,
environment, etc. (Reference Section 3.1)
Threat A potential for loss. A hazard. (Reference Section 3.1)
TOP Event The conceivable, undesired event to which failure paths of lower
level events lead. (Reference Section 3.6)
Trends The patterns in a run chart or control chart that feature the
continued rise or fall of a series of points. Like runs, attention
should be paid to such patterns when they exceed a predetermined
number (statistically based). (Reference Section 8.0)
Upper Control Limit
Range
The upper control limit for the moving range chart for a set of data.
(Reference Section 7.14)
Variation The inevitable difference among individual outputs of a process.
The sources of variation can be grouped into two major classes:
Common Causes and Special Causes. (Reference Section 6.2)
Weighting Factor A method of rating the relative importance of a concern or selection
criterion as related to comparable concerns or selected criteria.
(Reference Sections 2.1 and 7.12)
C8
APPENDIX D
HAZARDS CHECKLIST
D1
D2
HAZARDS CHECKLIST
Notes:
1. Neither this nor any other hazards checklist should be considered complete. This list should be enlarged as
experience dictates. This list contains intentional redundant entries.
2. This checklist was extracted from "Preliminary Hazard Analysis (Lecture Presentation)", R.R. Mohr, Sverdrup
Technology, Inc., June 1993 (Fourth Edition).
D3
Electrical
_____ Shock
_____ Burns
_____ Overheating
_____ Ignition of Combustibles
_____ Inadvertent Activation
_____ Power Outage
_____ Distribution Backfeed
_____ Unsafe Failure to Operate
_____ Explosion/Electrical (Electrostatic)
_____ Explosion/Electrical (Arc)
Mechanical
_____ Sharp Edges/Points
_____ Rotating Equipment
_____ Reciprocating Equipment
_____ Pinch Points
_____ Lifting Weights
_____ Stability/Topping Potential
_____ Ejected Parts/Fragments
_____ Crushing Surfaces
Pneumatic/Hydraulic Pressure
_____ Overpressurization
_____ Pipe/Vessel/Duct Rupture
_____ Implosion
_____ Mislocated Relief Device
_____ Dynamic Pressure Loading
_____ Relief Pressure Improperly Set
_____ Backflow
_____ Crossflow
_____ Hydraulic Ram
_____ Inadvertent Release
_____ Miscalibrated Relief Device
_____ Blown Objects
_____ Pipe/Hose Whip
_____ Blast
Acceleration/Deceleration/Gravity
_____ Inadvertent Motion
_____ Loose Object Translation
_____ Impacts
_____ Falling Objects
_____ Fragments/Missiles
_____ Sloshing Liquids
_____ Slip/Trip
_____ Falls
Temperature Extremes
_____ Heat Source/Sink
_____ Hot/Cold Surface Burns
_____ Pressure Evaluation
_____ Confined Gas/Liquid
_____ Elevated Flammability
_____ Elevated Volatility
_____ Elevated Reactivity
_____ Freezing
_____ Humidity/Moisture
_____ Reduced Reliability
_____ Altered Structural Properties
(e.g., Embrittlement)
Radiation (Ionizing)
_____ Alpha
_____ Beta
_____ Neutron
_____ Gamma
_____ XRay
Radiation (NonIonizing)
_____ Laser
_____ Infrared
_____ Microwave
_____ Ultraviolet
HAZARDS CHECKLIST
Notes:
1. Neither this nor any other hazards checklist should be considered complete. This list should be enlarged as
experience dictates. This list contains intentional redundant entries.
2. This checklist was extracted from "Preliminary Hazard Analysis (Lecture Presentation)", R.R. Mohr, Sverdrup
Technology, Inc., June 1993 (Fourth Edition).
D4
Fire/Flammability  Presence of:
_____ Fuel
_____ Ignition Source
_____ Oxidizer
_____ Propellant
Explosives (Initiators)
_____ Heat
_____ Friction
_____ Impact/Shock
_____ Vibration
_____ Electrostatic Discharge
_____ Chemical Contamination
_____ Lightning
_____ Welding (Stray Current/Sparks)
Explosives (Effects)
_____ Mass Fire
_____ Blast Overpressure
_____ Thrown Fragments
_____ Seismic Ground Wave
_____ Meteorological Reinforcement
Explosives (Sensitizes)
_____ Heat/Cold
_____ Vibration
_____ Impact/Shock
_____ Low Humidity
_____ Chemical Contamination
Explosives (Conditions)
_____ Explosive Propellant Present
_____ Explosive Gas Present
_____ Explosive Liquid Present
_____ Explosive Vapor Present
_____ Explosive Dust Present
Leaks/Spills (Material Conditions)
_____ Liquid/Cryogens
_____ Gases/Vapors
_____ Dusts  Irritating
_____ Radiation Sources
_____ Flammable
_____ Toxic
_____ Reactive
_____ Corrosive
_____ Slippery
_____ Odorous
_____ Pathogenic
_____ Asphyxiating
_____ Flooding
_____ Run Off
_____ Vapor Propagation
Chemical/Water Contamination
_____ SystemCross Connection
_____ Leaks/Spills
_____ Vessel/Pipe/Conduit Rupture
_____ Backflow/Siphon Effect
Physiological (See Ergonomic)
_____ Temperature Extremes
_____ Nuisance Dusts/Odors
_____ Baropressure Extremes
_____ Fatigue
_____ Lifted Weights
_____ Noise
_____ Vibration (Raynaud's Syndrome)
_____ Mutagens
_____ Asphyxiants
_____ Allergens
_____ Pathogens
_____ Radiation (See Radiation)
_____ Cryogens
_____ Carcinogens
_____ Teratogens
_____ Toxins
_____ Irritants
HAZARDS CHECKLIST
Notes:
1. Neither this nor any other hazards checklist should be considered complete. This list should be enlarged as
experience dictates. This list contains intentional redundant entries.
2. This checklist was extracted from "Preliminary Hazard Analysis (Lecture Presentation)", R.R. Mohr, Sverdrup
Technology, Inc., June 1993 (Fourth Edition).
D5
Human Factors (See Ergonomic)
_____ Operator Error
_____ Inadvertent Operation
_____ Failure to Operate
_____ Operation Early/Late
_____ Operation Out of Sequence
_____ Right Operation/Wrong Control
_____ Operated Too Long
_____ Operate Too Briefly
Ergonomic (See Human Factors)
_____ Fatigue
_____ Inaccessibility
_____ Nonexistent/Inadequate "Kill"
Switches
_____ Glare
_____ Inadequate Control/Readout
Differentiation
_____ Inappropriate Control/Readout
Location
_____ Faulty/Inadequate
Control/Readout Labeling
_____ Faulty Work Station Design
_____ Inadequate/Improper Illumination
Control Systems
_____ Power Outage
_____ Interferences (EMI/ESI)
_____ Moisture
_____ Sneak Circuit
_____ Sneak Software
_____ Lightning Strike
_____ Grounding Failure
_____ Inadvertent Activation
Unannunciated Utility Outages
_____ Electricity
_____ Steam
_____ Heating/Cooling
_____ Ventilation
_____ Air Conditioning
_____ Compressed Air/Gas
_____ Lubrication Drains/Slumps
_____ Fuel
_____ Exhaust
Common Causes
_____ Utility Outages
_____ Moisture/Humidity
_____ Temperature Extremes
_____ Seismic Disturbance/Impact
_____ Vibration
_____ Flooding
_____ Dust/Dirt
_____ Faulty Calibration
_____ Fire
_____ SingleOperator Coupling
_____ Location
_____ Radiation
_____ WearOut
_____ Maintenance Error
_____ Vermin/Varmints/Mud Daubers
Contingencies (Emergency Responses by
System/Operators to "Unusual" Events):
_____ "Hard" Shutdowns/Failures
_____ Freezing
_____ Fire
_____ Windstorm
_____ Hailstorm
_____ Utility Outrages
_____ Flooding
_____ Earthquake
_____ Snow/Ice Load
HAZARDS CHECKLIST
Notes:
1. Neither this nor any other hazards checklist should be considered complete. This list should be enlarged as
experience dictates. This list contains intentional redundant entries.
2. This checklist was extracted from "Preliminary Hazard Analysis (Lecture Presentation)", R.R. Mohr, Sverdrup
Technology, Inc., June 1993 (Fourth Edition).
D6
Mission Phasing
_____ Transport
_____ Delivery
_____ Installation
_____ Calibration
_____ Checkout
_____ Shake Down
_____ Activation
_____ Standard Start
_____ Emergency Start
_____ Normal Operation
_____ Load Change
_____ Coupling/Uncoupling
_____ Stressed Operation
_____ Standard Shutdown
_____ Shutdown Emergency
_____ Diagnosis/Trouble Shooting
_____ Maintenance
APPENDIX E
EXAMPLE PRELIMINARY HAZARD ANALYSIS WORKSHEET
E1
E2
Example Preliminary Hazard Analysis Worksheet*
*This worksheet was extracted from “Preliminary Hazard Analysis (Lecture Presentation),” R.R. Mohr, Sverdrup Technology, Inc., June 1993.
E3
APPENDIX F
EXAMPLE FAILURE MODES AND EFFECTS ANALYSIS WORKSHEET
F1
F2
Example Failure Modes And Effects Analysis Worksheet*
*This worksheet was extracted from “Failure Modes and Effects Analysis (Lecture Presentation),” R.R. Mohr, Sverdrup Technology, Inc., July 1993.
F3
NASA Reference Publication 1358
System Engineering “Toolbox” for DesignOriented Engineers
B.E. Goldberg Marshall Space Flight Center • MSFC, Alabama K. Everhart, R. Stevens, N. Babbitt III, P. Clemens, and L. Stout Sverdrup Technology, Inc.
National Aeronautics and Space Administration Marshall Space Flight Center • MSFC, Alabama 35812
December 1994
ii
ACKNOWLEDGMENTS The authors are very grateful for the help received from the following persons in producing this document. Becky Mohr contributed information and illustrations concerning preliminary hazard analyses and failure modes and effects analyses. Bryan Bachman provided a thorough review of drafts of the entire document. Larry Thomson prepared a figure in the system safety and reliability tools section. Jimmy Howell verified all numerical calculations in the examples. The following persons reviewed the indicated sections of this document and offered suggestions that greatly enhanced the discussions of the tools and methodologies presented: Bill Cooley Melissa Van Dyke Karl Knight Charles Martin Ben Shackelford DesignRelated Analytical Tools Trend Analysis Tools System Safety and Reliability Tools Statistical Tools and Methodologies Graphical Data Interpretation Tools Case Study
iii
Figures provided courtesy of Sverdrup Technology, Inc., Tullahoma, Tennessee. Figure 31 Figure 32 Figure 33 Figure 34 Figure 35 Figure 36 Figure 37 Figure 38 Figure 39 Figure 310 Figure 311 Figure 312 Figure 315 Figure 316 Figure 317 Figure 318 Figure 319 Figure 320 Figure 321 Figure 322 Figure 323 Figure 324 Figure 325 Figure 326 Figure 327 Figure 328 Figure 329 Figure 330 Figure 331 Figure 332 Figure 333 Figure 334 Figure 335 Figure 336 Figure 340 Figure 341
iv
TABLE OF CONTENTS Page 1. INTRODUCTION .................................................................................................................. 1.1 1.2 1.3 Purpose ......................................................................................................................... Scope ......................................................................................................................... Relationship With Program or Project Phases.............................................................. References .................................................................................................................... 11 11 11 12 18 21 21 21 23 23 26 28 29 29 29 29 29 210 211 213 213 213 214 31 32 32 35 36 310 312 312 312
2. CONCEPT DEVELOPMENT TOOLS .................................................................................. 2.1 Trade Studies ................................................................................................................ 2.1.1 Description ...................................................................................................... 2.1.2 Application...................................................................................................... 2.1.3 Procedures....................................................................................................... 2.1.4 Example .......................................................................................................... 2.1.5 Advantages...................................................................................................... 2.1.6 Limitations ...................................................................................................... 2.1.7 Bibliography.................................................................................................... CostVersusBenefit Studies......................................................................................... 2.2.1 Description ...................................................................................................... 2.2.2 Application...................................................................................................... 2.2.3 Procedures....................................................................................................... 2.2.4 Example .......................................................................................................... 2.2.5 Advantages...................................................................................................... 2.2.6 Limitations ...................................................................................................... 2.2.7 Bibliography.................................................................................................... References ......................................................................................................
2.2
3. SYSTEM SAFETY AND RELIABILITY TOOLS ............................................................... 3.1 Risk Assessment Matrix ............................................................................................... 3.1.1 Description ...................................................................................................... 3.1.2 Application...................................................................................................... 3.1.3 Procedures....................................................................................................... 3.1.4 Example .......................................................................................................... 3.1.5 Advantages...................................................................................................... 3.1.6 Limitations ...................................................................................................... 3.1.7 Bibliography....................................................................................................
v
.........2 Application...............................................................6 Limitations ...............................................................................................2...........................2 Application.................3..........4.......4 Example ........................................ 3............................................................ Energy Flow/Barrier Analysis ....................................................................................................3....... 3. 3............................................2........1 Fault Tree Generation ...............5 3...... 3..................... 3.............................................2..............4..........................................................................................3 Identifying and Assessing Cut Sets..................................................................................................... 3...............1 Description .......................................................................................................... 3......................................5 Advantages............................4 Identifying Path Sets ........................................5..................3..................................................................................... 3.....................................2 Application................................5 Advantages..............................................5................................................................................................................2 Application.........................................................2 Application.......................7 Bibliography............................................ 3..................................................................................................3 Procedures...............................6...........................................................2 Preliminary Hazard Analysis.........................3 3...4.3 Procedures..3 Procedures.......................... 3.................................................5 Advantages.......4...............................................7 Bibliography..................6. 3................................................................ 3.................................... Failure Modes and Effects (and Criticality) Analysis .....................5.........................................................................3.......1 Description .............. Reliability Block Diagram...........................4 Example ...........................5.6........................... 3..........................3 Procedures....6 Limitations ..... 3.................................................................. 3....................................................................................................... 312 312 313 313 316 317 318 318 318 318 319 319 319 321 321 321 321 321 322 322 326 328 329 329 330 330 332 332 333 334 334 335 335 335 335 336 337 337 341 343 3.. 3...........6 Limitations .3 Procedures...................................................................................................................................... 3....................................................7 Bibliography...................................................................... 3.6.................TABLE OF CONTENTS (Continued) Page 3.......................... 3.......................................3..3................. 3...............................................4 3................................5..1 Description ............................ 3................. 3................................ 3......................... Fault Tree Analysis ..................................................................2....4................................... 3.............. 3..................................................... 3......................................5 Advantages........ 3........................ 3.....2..............................................................................6..................................................................................................................3....................................7 Bibliography................6. 3.....................2....... 3...... 3.....................................................................3.................2 Probability Determination.............4....................1 Description ............................6 vi .......................... 3................4 Example ................................2...5..........................................................5..................................................................3...................................................................................... 3..................4 Example ......... 3..4...........6............................3...................................1 Description ............................................... 3...........6 Limitations .................................3...................
............1 Fault Tree to RBD Transformation...........................9..........................................6.... 3................................................................ Fault Tree...................8 3..........................10.......... 3.............................4 Event Tree to RBD and Fault Tree Transformation ................... 3.....1 Description .....7.............. Limitations ..............9..................................3 Procedures.............................................................................................4...........2 Application.......................................3 Procedures..........3 Procedures.....10 CauseConsequence Analysis..............................................................7 Success Tree Analysis .......................................7 Bibliography...................................................................................6.4 Examples......................................................................6 Limitations ........9..............................10............. 3..................................................... 3.....6.1 Fault Tree Construction and Probability Propagation........ 3..................................................................................................... and Event Tree Transformations ................ 3......................................10............................................................................................................................................................................................................................................ Reliability Block Diagram..TABLE OF CONTENTS (Continued) Page 3..............9.......................... 344 344 345 346 346 347 347 348 348 348 349 350 351 351 351 351 351 352 353 354 354 354 356 356 356 356 356 356 356 358 358 358 361 361 361 361 361 362 362 364 364 365 366 3.........3.......10................................. Advantages.................... 3.............6.................................................................................................................1 Description ..................................................................2 RBD and Fault TreetoEvent Tree Transformation...................................4 Example .. 3................ 3..........................................................................................................................................................9.....................................................................................................................................................2 Application.................8.......................9................ 3.............................................................. 3....... 3........7... 3.8....10........................6..............................5 Advantages..................7 Bibliography... 3....4.......................................................................... 3........................... 3................................................................8........... 3........7 Bibliography....................2 Application.............................................................. 3...................9...............................................7...............................................................................3....................... 3.....................1 Description ......................2 Application.......6................ 3..............................4 Example .............6 Limitations ...........7......................................................................................................................9...............................7 3.................................. 3....3 RBD to Fault Tree Transformation...................5 Advantages..............5 Advantages.....................10....................6 Limitations .....................6 3.......... 3................... Event Tree Analysis...........8.................... 3....................................................4 Example ............5 3. 3............................ 3..............................4.................................................................................. 3..............7.........9... 3........ 3............................ vii ........................7...............................8..................................................... Bibliography.......... 3.......... 3.......................................................................................................... 3..................3..............5 Advantages...................................................................................................... 3.........6 Limitations ................................................................................9.................................. 3.......3 Path Sets.................................8...3......1 Description .......6............................2 Cut Sets ................................................................7 Bibliography.......................................................................... 3..9 3.......................... 3.4 Example ...................7...................................................................8............10.....3 Procedures.........................................9..........................
............................................... 3.....................................................12...................................................................... 3........................................................................7 Bibliography.....................13.............................................4 Example .................................... 3....................................3 Procedures.... 3..........................................................14................................ 3.....................13...................................................................................................................15.......6 Limitations ...3 Procedures....15.......................................11...................2 Application.....13 Failure Mode Information Propagation Modeling............................................................... 3..........12.......................................................................................................................................................... 3......................................................... 3............................................................ 3...15.......................................1 Description . 3. 3.14......11....................... 3..14...4 Example ................................................................................................................................................................... 3................................................................13.......................................................... 3.............................................. 3..............5 Advantages.......................................................11.......12.............1 Description ..............11 Directed Graphic (Digraph) Matrix Analysis ............................2 Application.....................................12.............................................. 3........ 3....13........2 Application................................................................ 3....... 3....1 Description .........12.....................................3 Procedures...5 Advantages.15 Probabilistic Risk Assessment...........................4 Advantages..................................................................................11....... 3....................................................... 3...........................6 Limitations .....11.................................................................................................................................................5 Advantages. 366 366 366 367 369 370 372 372 372 372 373 373 374 374 374 376 376 376 376 377 378 378 378 378 378 380 383 383 384 384 384 384 385 385 386 viii ..........14....................................2 Application.................................................... 3..........................3 Procedures........... References .........................................................TABLE OF CONTENTS (Continued) Page 3..................................................13....................................................... 3............................15............................................................................................................................................. 3.............................................. 3....................................................14....................................................................... 3..............2 Application................. 3..................................................................12.........................11............................... 3............................14 Probabilistic Design Analysis.........1 Description .....................12 Combinatorial Failure Probability Analysis Using Subjective Information....................................... 3.............................. 3................3 Procedures..............11.............................................................................. 3.....................................................................................................6 Limitations ................................................................ 3..........4 Advantages..................13...................5 Limitations ............................ 3......15...........................................................................5 Limitations ................................4 Example .......................................1 Description ........................
..2........................................................ 5................4 Example .............................................1 Description ................................5 Advantages....................................2 Application......2 Application..................................................................................2........................................................................................... Control Chart ...... 4.......................................................................................................4 Example ............................ 4........................................... 4...........................2............................3..2....................................................................................................................... 5.................................6 Limitations .......................... 4..................2...........................................................................................3..... 4.......................................................3............ 4.........................................................................................................................3......................................................3 Procedures.2....4 Example ......... 4...........................................................................................................6 Limitations ...................1.... 5......................................................................................................................................3 Procedures..................1..................................... 5................................1 Sensitivity (Parametric) Analysis .................................. 4..1...................1 Description ........................................................................................... 5........3...........................6 Limitations .......................1 Description ................................................................................ 4.................. 5.................................7 Bibliography...................................................................................................................2 4........................................................................... 5...........................1........................................ 41 41 41 41 41 43 44 44 45 45 45 45 46 47 47 47 47 47 47 48 49 49 49 410 51 51 51 51 53 53 53 53 54 54 54 55 55 56 56 4........................... Standard Dimensioning and Tolerancing .................1.................................................... 5......3.........................3 5........1............................. 4..................................... 5........2.......................5 Advantages............ 4..........1 Description ....2 Application.......................3 Procedures................................. DESIGNRELATED ANALYTICAL TOOLS....................................................................................2 Application..............................2 ix ................ 4.............................................5 Advantages.............1..... 5...............................1 Description .........3 Procedures..................................... Tolerance Stackup Analysis ................................................................... 5...................... 5............ 4...............TABLE OF CONTENTS (Continued) Page 4...........................................................................1.................2..... 5.......................................................................................................... 4...1 Scatter Diagram ................. 4.............................................. References .... 4.......... 4...........2...........2....................................... GRAPHICAL DATA INTERPRETATION TOOLS.............................................................................................................................................................................................................................................................3 Procedures.5 Advantages...........................................................................................................1.......................1..................................................6 Limitations ........................4 Example .......................................5 Advantages..... 5.......1................................................................................................................3.........1.......................6 Limitations ....................................................................2 Application.......2............................................. 4...2................. 4.....4 Example ............................................................ 4...........................................................................................................
.......... 5............ 5... 5..........................................................3...................................................... 5......................... Stratification Chart..............................................................................5................................................................7 x ....................................................3................................4 Example .......5 Advantages................. Histograms..1 Description ...................... 5...................................................4............. 5.....................................................................................................................................6 Limitations ..... 5........................................... References................................ 5..........6..........................................................3 Procedures.... Pareto Chart .........6...........1 Description ...............5 Advantages.......7............................................ 5.........................7..............................................................5 Advantages...........1 Description ....................................................................3............................... 5.........................2 Application........................7...............6 5..................... 5........TABLE OF CONTENTS (Continued) Page 5...............................................................................................................................................................................6.............................................................................4 Example ........3 Bar Chart..........5...........6.......................................................2 Application...........5...........................................1 Description ...........................................................5 Advantages...... 5............ 5.......................5 5..........................3 Procedures............................. 5..... 5....................................................................................................... 5................................................................................6 Limitations ................................................................... 5..........2 Application.........................................................................................................................6..............................5................................................................................. 5.......................4 Example ...................................................................................................1 Description ...............5.........................................6 Limitations ..........................................6..................................................................... 5............. 5.......................................... 5........................................................6...............................................7.................6 Limitations ...................... 56 56 56 56 57 57 57 58 58 58 58 58 58 59 59 59 59 510 511 511 511 511 511 511 511 512 513 513 513 513 513 513 513 514 514 514 515 5........................ 5..................................................................................................................................................................................................................2 Application..3 Procedures........5 Advantages.........................................................4 Example .. 5.......................4......7 Bibliography......... TimeLine Chart ...................................................................................................3....................................................................... 5......................3.3 Procedures.................... 5.......................................................................4 5....6 Limitations ............ 5.............. 5................................................................................................................4 Example ............................ 5...................... 5.....................................3 Procedures.........3................................................2 Application.....................................4............................................................................................................................... 5...7............................4...............................................................5..................................................... 5..................4..................................................................................7........................4............................
.................................... 6..........................................................................2...............................................1 Description ................................................ 6.............................................................................................................................................................................................................................................................7 Bibliography...........3........................... 6.................................................... 6.3...................1......................2 Application.................................................................................................................................................................3 6.......................................................................... 6..........3............................ 6....................5 Advantages....................... 6................ 6................................................................................................................ 6............................................................ 6...3........................................1 Description ............... 6......................................................5............................................ 61 61 61 61 63 63 63 64 64 64 64 64 64 66 67 67 67 67 67 67 68 68 68 68 68 68 69 610 612 612 612 612 612 613 614 614 614 6............... 6........5..............................................4 Example .................1.................................................6 Limitations ................................................................................................................................................. STATISTICAL TOOLS AND METHODOLOGIES.....1 “Studentt” Analysis ............................ 6..... 6...........................2 6................................3........................1 Description ............1 Description ........... 6..................................................................................... 6................5.............................................4.........2......................................................5 Advantages............................................. 6....... 6..............................................................................6 Limitations .............2........................2..........4.........1....................5 xi .......3 Procedures.................6 Limitations ......... 6............................................................................................................................3 Procedures...1..3 Procedures.................5.................... 6................. 6................................... 6..........1 Description ......... 6....................3.............4 Example .....5 Advantages....... 6....................1.....................................4...............1............ 6..............................3 Procedures.....................................................................................3 Procedures................... 6................... Factorial Analysis ......................................5.............1........ 6....................................................................4.5................................5 Advantages........................................2...................................................................................................4 6.................5 Advantages........................................................................4 Example ..................................................................................................................... 6...........................................................................................................2 Application..........................4 Example ..... 6..................................6 Limitations ................................. 6..................................................... 6.......... 6........................................................................2 Application..........................2 Application...... Correlation Analysis ........................................................6 Limitations ............................................................................................4.......................................................... Confidence/Reliability Determination and Analysis ..............................................4 Example ........................................................2 Application...............4.............2...................................TABLE OF CONTENTS (Continued) Page 6...... Analysis of Variance............................
..............3...............................................2 Application....................................5 Advantages................................4 Example ..............................................................7.......................................................... 7...2 Application.2 7.....................2..................6................... 7.................................................... 7........................................... 7.......................................................................4 Examples.6......................................................................6 Limitations ......................... 7............ 6..4 Example ................................................................................................................................................................................1........................................................1 Benchmarking.......7 Bibliography.................................................................................................................1............1 Description ............... Response Surface Methodology ...............................................................1 Description ...............................3.... 6...............................................2 Application....................2..1.................................................................3 Procedures.................................................................................3 Procedures.................................................. 614 614 615 615 616 617 617 617 617 617 617 618 619 619 620 71 75 75 75 75 76 76 76 77 77 77 77 78 78 710 710 710 710 710 710 711 712 712 6.... 7..........................................................................................................................3 Procedures.................................2........ 7...2.................................................................................... 6..........................3........................................... 7.................... 7.................................TABLE OF CONTENTS (Continued) Page 6....................................................... 6................................3....................... References ........ 7...................7....5 Advantages.............7....6 Limitations ................................................................................5 Advantages..........6..................................................... 6.................. 6...............................3 xii ... 7.................................. 7......................................................1 Description ..................................6 Regression Analysis.........7............................................. 7............................................................2 Application.......................................................................................6................3....2...................... 6................. 6...3 Procedures.................................................. 7...........5 Advantages.................1 Description ........ 6......................... 7.6 Limitations ........................................................................................... 7..........................................................1 Description ......................................................................................................................7...3............................................ 7...........................5 Advantages.........................................................4 Example ....1............6 Limitations ...............6.. 7............................................................................................6 Limitations .......... 6........... Cause and Effect Diagrams (Also Known as Fishbone Diagrams or Ishakawa Diagrams) ...3 Procedures......................................................................................................................................................... TOTAL QUALITY MANAGEMENT (TQM) TOOLS ......................................................................................... 7.......................................................................2 Application........ 7...............................................2...........................................................................................................2....................................7.6............................................................................1................................................. 6....... Concurrent Engineering......................................................................... 7........... 6............................................................................................1................7 7.......................................................4 Example ..............................................
.....................................................................6 Limitations ................ 7................. 7......... 7.................................................................................................................. 7............7 7..............................6......................................................................................................................................................................................................................................................................................................................................................................8................................. 7............................................... Checklists....................7.............................................................. 7..........................4 Cost of Quality........................................................................... 7................2 Application...........................................................7.................7...............5.........8...6 Limitations ................................ 7.......................4.....................................................................................7............................... 7............................. 7.....................8................. 7...................................................................4 Example .......................................3 Procedures........3 Procedures......................................................... 7....................5 7...................... 7.......5.......................................................................................................................2 Application..............................................................................6...4 Example .6................ 7.........................5........................................4........5 Advantages..................... 7......... 7......4 Example .......................................................... 7......................................5................................................5................................ Design of Experiments .........................3 Procedures......................................................................................................................................................................................................................................................................................5................................................. Brainstorming .............5 Advantages................... 7...........5 Advantages......................................................... 7... 7.....................4......3 Procedures..........................1 Description ...................... 7......8 xiii ........................................................1 Description .2 Application.........................................................7...................................................................................8.........4 Example ..........................................................................7............................................................................3 Procedures................. 7...................5... 7.......... 7.............................. 7.................................................................................................................................6................6 Limitations .............................................................6 Limitations .......................................................8........................................................................1 Description .......6 7.................6....................8....................................................................................................1 Description ...5 Advantages.............5 Advantages........................................2 Application.....................2 Application....... 7....................4 Example ........TABLE OF CONTENTS (Continued) Page 7.............. 7.............................................. 7. 7.4..................1 Description ..........................................6....................................................... 7..............................................4.....7 Bibliography...........4.. 7.....................6 Limitations ....... Evolutionary Operation ........................................................................................................................ 712 712 712 712 714 714 715 715 715 715 715 716 718 718 719 719 719 719 719 720 723 723 723 723 723 724 724 725 725 726 726 726 726 726 726 726 7.............
...................... 7....................................................... 7........................................................................................................................ 7..................................12 Quality Function Deployment .........13.....................1 Description ....4 Example ...................................................................... 7............................................................................13... xiv .... 7...................10...................................9...........4 Example .........................................................................................2 Application.................13 Quality Loss Function.......................................9......................9......12.....................................................9..................................................................... 7.......13...........11 Force Field Analysis ..........................................9......... 7......... 7........5 Advantages........................................ 7.....................................................2 Application........................... 7......................................................... 7............. 7.............4 Example ............................................................ 7...........5 Advantages.............6 Limitations .....................13... 7......................11..........................................................11........................................... 7...........................................10... 7..............12.......1 Description ...4 Example .................................................................................................................................................................... 7................12.............................................. 7............... 727 727 727 727 728 729 729 730 730 730 730 730 732 732 732 732 732 734 734 735 735 735 735 736 736 737 740 740 740 741 741 742 742 743 744 744 7....................... 7.......1 Description .11................ 7............................................................. 7......12........................................................6 Limitations ................................................................11.....................................................TABLE OF CONTENTS (Continued) Page 7.............................................................................................................................10.............6 Limitations ..1 Description ..................13.........................6 Limitations ........ 7...... 7......11... 7.....................................................................................................10.......12.... 7............................................................................3 Procedures..................................................................................................2 Application..................................................... 7..............................3 Procedures.............................................................................................................................................12.................10........................................................ 7......................................................7 Bibliography............................ 7.................13...................12.........1 Description ...........9 Delphi Technique.....................................................................................................................2 Application................ 7............................................................... 7.......................................11..................................................5 Advantages...................................... 7..............................10 Nominal Group Technique ................................................................5 Advantages.................................. 7.......................3 Procedures............................................10......................................................................................................................5 Advantages.........................................2 Application........................................................3 Procedures..............................................................3 Procedures........ 7............................. 7.................................9....................4 Example ..................6 Limitations ..........................................................................................................................................................
................................................................................................ TREND ANALYSIS TOOLS................................................................. 8.....................2 Application.............2....................................................... 7................................. 7............... 8....................... 7.....1 Description ..................................................................................................15..16....................................................16 Work Flow Analysis ......2 xv ..............................6 Limitations ............................................................................3 Procedures................................................................................................................... 8....... 8............... 7................................................................................... 8..............................16....................................................... 8.........................16...................................................................................... 8......................2..................................... 7.......................................................................... 7........................................................................................................1..................15..................6 Limitations ..........15...............................................1................................................................. 7.................3 Procedures...................................... 7.............................................................................. 8.........................................14..........................2..................................................................................14 Statistical Process Control .............3 Procedures..................1............................15..................7 Bibliography......................14................2...............5 Advantages....................................................... 8.......... 744 744 744 744 746 748 749 749 749 749 752 752 752 752 752 753 753 754 754 754 755 755 756 81 84 84 84 85 87 88 88 88 88 88 89 810 811 815 815 815 8.................................2...................................................4 Example ..... 7............................................3 Procedures............................3 Procedures.........................................5 Advantages.........................................................2 Application...................... 7................. 7............ 7.4 Example ...............1 Performance Trend Analysis ...............4 Example ......................................................................................................................2........................1 Description ..........................................................1 Description ...................1............6 Limitations ........2 Application.......................................................................................................................1... 7......5 Advantages...... 8...........................................14.....................................................................TABLE OF CONTENTS (Continued) Page 7.................................................. 7..................................................... 8.............................................................................................................. 7.................... 8............................ Problem Trend Analysis ..............................5 Advantages..........................................14........... 7...............................................................................................7 Bibliography..................................................14.............6 Limitations ..........16.6 Limitations ........15 Flowchart Analysis ........................................14................................................................................................. References ............................................................... 8......................5 Advantages................................1 Description ...4 Example .........4 Example ............................................................................................... 7........................7 Bibliography...................................... 7............... 8...............................1 Description ............ 7.........................................2 Application....................................... 7...............2...15........................ 8................15.................. 7...................................................................................................................................................................................................16..........16.........................................1................................................1.................................. 8..2 Application...........14.............................................................................................
............. 8........................4.................................. 8.............................. 8.............. 8.........................................................3...................................................................... 8.......................................................................... 8...................................................2 Application.......4 Example ............... Supportability Trend Analysis....................................2 Application.............................................5........5....................................................... Appendix F ..................... 8...........................3....1 Description ........ 8....................................................................7 Bibliography................ 8..4..............5.................................................................................................................................................6 Limitations ................................................................................................................................................ 8.......................................................................................4 Example ...................................................................................................................................................................................5 Appendix A ..........................................3.........................5.............................5......................... Appendix B ........................................................4 Example .........6 Limitations .....................3....5 Advantages.............................. Appendix D .....................................................................................2 Application....................................................................................................................................................6 Limitations ..................................3 Procedures.......5................................................4........................................................... 815 815 816 816 818 818 818 818 819 819 820 821 822 822 822 823 824 824 824 825 825 825 826 826 827 A1 B1 C1 D1 E1 F1 8......................................................................................................................... Reliability Trend Analysis...........3......................................................................4.............................TABLE OF CONTENTS (Concluded) Page 8..................................................................................4.................................................................. 8......................... 8................. 8.......... 8................................................... 8.....................................................3 Procedures..7 Bibliography..................................................................................................................3.............................1 Description ...................................7 Bibliography..................................................................................... 8..............5............. 8.............................................3 Procedures.......... References................5 Advantages...............................................................................................4................. Appendix C ...........3 Programmatic Trend Analysis .. 8..........................................................................1 Description ....................... 8.............. xvi .....................................4 8..................................................4...................................................................................... Appendix E ...................3.............................................................5 Advantages.............................................................................................................................................................. 8..................... 8.................................
......... PHA process flowchart ....................................... 36................................ Log average method of probability estimation ........ 317................................ Typical FMECA worksheet .......................................................................................... 311. Typical complex RBD .............. Fault tree construction process...................... Example of determining cut sets ................... Example of determining path sets ............................................................................................................................... 322............ 38.................................................................................. 310...................................................... 323.................................. 31........................................................................................ Typical risk assessment matrix ........... Success tree construction process ................... 33........... 315..................................................... 34................................................ Risk plane ..................................... Example of system breakdown and numerical coding ... Exact solution of OR gate failure probability propagation ...................... 32................. FMECA process flowchart ..... 318........................ Relationship between reliability and failure probability propagation............................. 320............................................................................................ 22....... Isorisk contour usage ......... 37..............................................................LIST OF ILLUSTRATIONS Figure 21................................................ 316................................................. Example of an FMECA ................................................................................................................. Example weighted factor trade study summary table .......... Risk plane to risk matrix transformation . 314.................................. 35................ 319................................................................................................................................ Failure probability propagation through OR and AND gates ..................................... 312.... 313............. Title Example utility functions ............................. Helpful hints in creating a risk assessment matrix ........... 321.......... Page 27 28 35 36 37 38 310 311 314 316 323 324 326 327 331 333 339 339 340 340 341 344 345 346 349 xvii . Severity and probability interpretations ...................................... 39.................................. Typical PHA ...................... Example fault tree .............. Example RBD ..................................
.............. 339..........................................................................................................................................................................................LIST OF ILLUSTRATIONS Figure 324............................ 329................... Construction of digraph adjacency matrix .................................................................... 325.. Example ETA ................................................................................. Causeconsequence analysis format ............................................................................ 332............................ 43............................ 336.. 52................................................................. Equivalent logic RBD and fault tree ........................................................................................... 337............... 340......... Fault tree to RBD transformation....... 335...... Example causeconsequence analysis ...................... 333............................................. Scatter diagram example........................................... 341.................................................. 334........................................................ Control chart example............................ 330....... xviii Page 350 352 353 355 357 357 358 359 359 360 362 364 365 368 369 371 375 379 382 383 46 48 48 54 55 ................................ Example combinatorial failure probability analysis ...... 41... 338.................................. Title Example success tree .......... Load and capability transfer functions ......... Oring joint components ........ 328................ Example of dimensioning and tolerancing .................. 327.................... RBD to event tree transformation ......................................................................................... 326........................... 343................................. ... 331................... 42...................................................... Event tree (Bernoulli model) ....... Event tree to fault tree transformation ......................................... 51... Event tree (generic case) ........ Comparison between digraph and fault tree logic gates ..... Interference between load and capability density functions ......... Oring joint ................................... Example failure mode information propagation model ....................... Example digraph matrix analysis ............. RBD to fault tree transformation .................................... Relationship between cause and consequence .. 342........................................ Deriving cut and path sets from an RBD ...........................................................................................................................................
. Timeline chart example ............................................................................................... Quality loss function for NIB....................................... Cause and effect diagram on receiving telephone messages .................. 713........................ Control chart showing mean deviation for each part ................ Range chart showing mean range for each part ........................... Fault tree sample with estimates assigned .......... 57................................................. Concurrent engineering example .................. 72................................................................................. Sample of a partial igniter subsystem fault tree..................................................... 78................... 74..................................................................................................................................................................................................................... Standard cost of quality curve........................................................ Histogram example ........................................................ QFD example on automobile industry ..... 56... 75. Control chart showing mean deviation for hole guide 1.............................................................................. 71........................................... Force field analysis example............................... Quality loss function example...... 55.................................................................................................................. 718.....LIST OF ILLUSTRATIONS Figure 53........... Line generated with least squares method ........................................ 73....... House of quality .................. Factor/level effects graph............................. Comparative benchmarking ........................................................ 54....................................................... 711............................... Title Bar chart example .......................................... 716.................... 79...................... 712............................. Design rework cause and effect diagram ...................................................... 77.... EVOP example........ Pareto chart showing mean deviation for each hole guide.. Stratification (histogram) chart example ............. 719...................... 61......................... 714.................. 710. Page 57 59 510 512 514 616 76 78 79 711 713 718 720 728 729 734 735 739 741 742 743 747 747 748 748 xix ............................ 715............... 717. Traditional view to meeting specification.................. Pareto chart example.................... 76.......................................................................................
.... Common flowchart symbols ............................................................ Supportability trend analysis example ..................... Performance trend analysis example ........................................... Programmatic trend analysis example .... 723.............. 721......... 82...... 81.......................................................... 85 Title Example of topdown flowchart .................................................................................... Reliability trend analysis example ..................... 84...................................................... 722........................................................................ Example of detailed flowchart ............................................................................................. Problem trend analysis example ............ Page 749 750 751 753 755 87 812 819 823 825 xx ........................... 724......................................................................... Work flow diagram example .....LIST OF ILLUSTRATIONS Figure 720....................... 83.......................... WFA example ......................................
................ 38.............. 22.................... Statistical tools and methodologies ................................... Graphical data interpretation tools and methodologies ......................................... Title System engineering “toolbox” function matrix .......................... 35..... 31........................... 51...................................... Sensitivity analysis calculations .......................... 32.............................. FTA procedures .............................. Examples of strategies to manage harmful energy flow .. 42............................................................................ TQM tools and methodologies ... Factorial analysis example ......................................... 21.................................... Page 13 15 22 25 210 32 33 320 330 336 338 342 363 373 42 44 52 62 69 610 72 714 716 716 xxi ..................... and results .................................................................................................................. System engineering “toolbox” project phase matrix ........................... 71. Trial............ Causeconsequence tree construction symbols ........................................................ 39....................................... Symbolic logic techniques ..... Probability propagation expressions for logic gates .................................. 61............ 73......................... Month’s cost of quality ............................. 41.................................. Example selection criteria for costversusbenefit analyses ........................................................... System safety and reliability tools and methodologies .... 36.................................................................... 74................................... 63.......................................LIST OF TABLES Table 11... 12....................................... effects................ 62....... Fault tree construction symbols ................................................ 72.......................................... 37...... 33.................. Simple RBD construction .......................... Typical weighted trade study summary table ................. 34 .............. Designrelated analytical tools and methodologies ............................. Concept development tools and methodologies ......... Factorial analysis factors and magnitudes ... 23 factorial design data............................ 23....................................................................................... Combinatorial failure probability analysis subjective scale ....................................................
..................... 2 data ................................................. Motor postflight checklist ................................................ 2 data ..................... EVOP cycle No................. QFD matrix sample calculations .............. Concerns with assigned weighting factors ..................... Trend analysis tools and methodologies .............................................. 76... 1 and cycle No........... Page 717 720 721 722 727 731 733 737 746 83 xxii ....................................... Nominal hole size deviations and drill guide positions ........ 78........ 713......................... 81.. 710..........................................................................LIST OF TABLES (Continued) Table 75.... Comparison of EVOP cycle No............ Title Calculation of effects ............................. EVOP cycle No................ Replacement technology concerns .............................................................................................. 712........................................................................... 77. 711................................... 1 data ................................. 79...
effects. and criticality analysis Fault tree analysis Internal failure Loss function (quality) Lower control limits Lower decision line Larger is better xxiii .ACRONYMS AHP AHPA AIAA ANOVA B/C CIL CIM CSF DAS DOE DOF DR EF ETA EVOP FMEA FMECA FTA IF L(y) LCL LDL LIB Analytical hierarchy process Analytical hierarchy process approach American Institute of Aeronautics and Astronomics Analysis of variance Benefittocost Critical items list Change in mean Compliance safety factor Data acquisition system Design of experiments Degreeoffreedom Discrepancy report External failure Event tree analysis Evolutionary operation Failure modes and effects analysis Failure modes.
ACRONYMS (Continued) LSL MTBF MTBR MTTR NASA NGT NIB PDA PHA PRA PRACA OSHA QFD RBD RSM SE SESTC SIB SME SMQ SMR SPC SRM Lower specification limit Mean time between failures Mean time between repairs Mean time to repair National Aeronautics and Space Administration Nominal group technique Nominal is best Probabilistic design analysis Preliminary hazard analysis Probabilistic risk assessment Problem reporting and corrective action Occupational Safety and Health Administration Quality function deployment Reliability block diagram Response surface methodology Standard error System Effectiveness and Safety Technical Committee Smaller is better Sum of mean error Safety and mission quality Sum of mean replicate Statistical process control Solid rocket motor xxiv .
ACRONYMS (Continued) SSE SSR SST STA TQM UCL UCLR UDL USL WFA Sum of squares error Sum of squares replication Total sum of squares Success tree analysis Total quality management Upper control limit Upper control limit range Upper decision line Upper specification limit Work flow analysis xxv .
xxvi .
These tools are also used to determine the probability of failure occurring or the reliability that a component will operate successfully. or function problems with a design. in general. the following information is provided: (1) description. and postulates to accomplish a purpose. A methodology is defined as a collection of tools. comment (app. address the following areas of concern: (1) identify and assess hazards. either in comparative or absolute terms. section 2. rather than direction or instruction for specific technique selection or utilization. safety. Caution should be exercised in the use of these tools and methodologies. this is not intended to be restrictive. For each concept addressed in the toolbox. A tool. fit. rules.2 Scope The tools and methodologies available to the designoriented systems engineer can be categorized in various ways depending upon the application. Concept development tools. are useful when selecting the preferred option of several alternatives. manufacturability. In addition. at which level of detail are applicable. INTRODUCTION 1. and (7) bibliography and/or references. Among these alternatives are such things as cost. A) and. A thorough literature search was performed to identify the prevalent tools and methodologies. 1. Use of the techniques for the sake of “using techniques” is rarely resourceeffective. or perhaps determining the ratio of expected future benefits to the expected future costs. and (3) the determination of the possibility or probability of having form. (2) a method for specifying dimensions and tolerances. section 4. and what might be the expected “value added” for their purposes. 11 . as used herein. or to determine a tolerance or dimension necessary to avoid these problems. The reader is also cautioned to validate results from a given tool to ensure accuracy and applicability to the problem at hand. while techniques have been categorized for recommended areas of use. as applicable. Designrelated analytical tools. section 3. if practical. Too often. (3) procedures. Readers are encouraged to question. are applied to show (1) which parameters affect a system the most or least. use this reference as one source among many. weight. (2) identify failure modes and show their consequences or effects.REFERENCE PUBLICATION SYSTEM ENGINEERING “TOOLBOX” FOR DESIGNORIENTED ENGINEERS 1. (2) application. (4) example. is defined as a set of procedures to accomplish a specific function. It is left to the user to determine which technique(s). This toolbox is intended solely as guidance for potential tools and methodologies. (5) advantages. (6) limitations. designoriented systems engineer has difficulty finding any ready reference as to what tools and methodologies are available. The purpose of this system engineering toolbox is to provide tools and methodologies available to the designoriented systems engineer. System safety and reliability tools. these references are of only limited utility from the designer’s standpoint. and (3) symbolic logic modeling tools used to understand the failure mechanisms of the system.1 Purpose Many references are available on systems engineering from the project management perspective. A practicing. complexity.
An entry of (1) for the phase means the technique is primarily performed in that phase and an entry of (2) means the technique is secondarily performed in that phase. Phase B (concept definition)—the establishment of system design requirements as well as conceptually designing a mission. are applied to continuously improve performance at all levels of operation. conducting feasibility studies and design tradeoff studies. of a project design cycle. and to forecast future events. Variations are identified and mathematical relationships are determined. (1) (2) Phase A (conceptual trade studies)—a quantitative and/or qualitative comparison of candidate concepts against key evaluation criteria to determine the best alternative. (6) creative. To assist in further defining optimal areas in which each technique may be useful.1. (3) decision making.2. and (7) graphical. other phases should be considered by the user for a particular tool or methodology. section 7. Though the entries in this matrix are a result of research by the authors. Finally. Phase C (design and development)—the initiation of product development and the establishment of system specifications. These tools are discussed in section 5. For this reason. Extensive research was performed in order to identify all prevalent tools and methodologies available to the designoriented systems engineer. These functionality categories are found in reference 1. Many excellent texts are available on statistical methods. as described in reference 1. appendix B provides a case study illustrating the trials and tribulations of an engineer applying his recently acquired knowledge of the techniques to a given work assignment. and evaluation)—system verification. test. section 6. Appendix C provides a glossary of terms applicable to the tools and methodologies in this toolbox. (5) prevention. Phase E (operations)—the deployment of the product and performance validation. important tools or methodologies may have been overlooked. To further illustrate how selected tools and methodologies in this toolbox are applied. (3) (4) (5) Table 12 provides a project phase matrix for all of the tools and methodologies identified in this toolbox. 12 . and misapplied. quantitative tools that are used to identify potentially hazardous conditions based on past empirical data are trend analysis tools. identify relationships. in all areas of an organization. Statistical tools and methodologies. or reveal the most important variables in a set of data.When there is a desire to monitor performance. (2) problem identification. (4) modeling. graphical data interpretation tools are typically applied. Phase D (fabrication. appendix A is provided for the reader to complete and return to the individual indicated on the form. integration.3 Relationship With Program or Project Phases Each tool or methodology may be performed in a minimum of one of the following phases. section 8. table 11 provides a functional matrix which categorizes the functionality of each tool or methodology into (1) data analysis. If a tool or methodology should be considered for this toolbox. compare sample statistics and population statistics. Total quality management (TQM) tools. The ultimate objective for these tools is to assess the current status. as are software packages. 1. Nevertheless. this document touches only lightly on this area. using all available human and capital resources.
5 5.Table 11.1.2 3.13 3.11 3.3 5.3 3.2 4.5 3.7 3.10 3.3 5.1 5.15 4.1 4.1 3.2 5.12 3.6 3.6 5. System engineering “toolbox” function matrix—Continued Section 2.8 3.1 2. Energy flow/barrier analysis Failure modes and effects analysis Reliability block diagram Fault tree analysis Success tree analysis Event tree analysis Fault tree/reliability block diagram/event tree transformations Causeconsequence analysis Directed graph (digraph) matrix analysis Combinatorial failure probability analysis using subjective information Failure mode information propagation modeling Probabilistic design analysis Probabilistic risk assessment Designrelated analytical tools Sensitivity (parametric) analysis Standard dimensioning and tolerancing Tolerance stackup analysis Graphical data interpretation tools Scatter diagram Control chart Bar chart Timeline chart Stratification chart Pareto chart Histograms Note: Functionality categories found in reference 1.7 Tool or Methodology Concept development tools Trade studies Costversusbenefit studies System safety and reliability tools Risk assessment matrix Preliminary hazard analysis.2 3.4 3.14 3.4 5. Data Analysis √ √ √ √ √ √ √ √ √ √ √ √ Problem Identification Decision Making √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ Modeling Prevention Creative Graphical √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ 13 .9 3.
7 7.5 7.3 6. Section Tool or Methodology Statistical tools and methodologies “Studentt” analysis Analysis of variance Correlation analysis Factorial arrays Confidence/reliability determination and analysis Regression analysis Response surface methodology TQM tools Benchmarking Cause and effect diagrams Concurrent engineering Cost of quality Design of experiments Evolutionary operation Brainstorming Checklists Delphi technique Nominal group technique Force field analysis Quality function deployment Quality loss function Statistical process control Flowchart analysis Work flow analysis Trend analysis tools Performance trend analysis Problem trend analysis Programmatic trend analysis Supportability trend analysis Reliability trend analysis Data Analysis √ √ √ √ √ √ √ Problem Identification √ √ √ √ √ √ √ Decision Making √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ Modeling Prevention Creative Graphical 6.16 √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ 8.8 7. System engineering “toolbox” function matrix—Continued.1 8.1 6.3 8.13 7.5 √ √ √ √ √ √ √ √ √ √ Note: Functionality categories found in reference 1. 14 .3 7.6 7.11 7.9 7.12 7.6 6.2 6.4 8.2 7.10 7.4 7.7 √ √ 7.1 7.2 8.14 7.15 7.1.4 6.5 6.Table 11.
2 1 1 2 2 2 3.15 2 2 2 2 1 1 1 1 1 1 1 1 1 1 2 2 2 2 1 2 1 1 1 1 1 1 2 Note: Phases discussed in reference 1.3 3.12 3.2 3. Energy flow/barrier analysis Failure modes and effects analysis Reliability block diagram Fault tree analysis Success tree analysis Event tree analysis Fault tree/reliability block diagram/event tree transformations Causeconsequence analysis Directed graph (digraph) matrix analysis Combinatorial failure probability analysis using subjective information Failure mode information propagation modeling Probabilistic design analysis Probabilistic risk assessment Phase A Conceptual Trade Studies Phase B Concept Definition Phase C Design and Development Phase D Fabrication.4 3.6 3. Integration. System engineering “toolbox” project phase matrix—Continued Code: 1—Primary 2—Secondary Section Tool or Methodology Concept development tools Trade studies Cost versus benefit studies System safety and reliability tools Risk assessment matrix Preliminary hazard analysis.11 3.9 3.7 3.8 3.1 3.13 3.5 3.1 2.14 3. and Evaluation Phase E Operations 2.Table 12. Test. 15 .2.10 3.
Integration.2 5.1 4.4 5.3 6.1 6.6 6. and Evaluation 1 2 1 Phase E Operations 4.2 6.Table 12.5 6.7 2 2 2 1 1 1 1 1 2 1 1 1 2 2 2 2 1 2 1 Note: Phases discussed in reference 1.5 5.7 1 1 1 1 1 1 1 6.2 4.2.3 5.6 5.3 1 1 1 5.4 6. System engineering “toolbox” project phase matrix—Continued Code: 1—Primary 2—Secondary Section Tool or Methodology Designrelated analytical tools Sensitivity (parameteric) analysis Standard dimensioning and tolerancing Tolerance stackup analysis Graphical data interpretation tools Scatter diagram Control chart Bar chart Timeline chart Stratification chart Pareto chart Histograms Statistical tools and methodologies “Studentt” analysis Analysis of variance Correlation analysis Factorial arrays Confidence/reliability determination and analysis Regression analysis Response surface methodology Phase A Conceptual Trade Studies Phase B Concept Definition Phase C Design and Development Phase D Fabrication. Test.1 5. 16 .
2 8.3 7. Test.10 7.16 2 2 2 2 2 1 1 1 1 1 2 2 1 1 2 2 1 1 1 2 1 1 2 1 1 2 2 2 1 1 1 2 1 2 1 8.9 7.Table 12.15 7.12 7.13 7.2 7. 17 . System engineering “toolbox” project phase matrix—Continued.2.1 7. and Evaluation Phase E Operations 7.4 7.11 7.8 7.5 7.14 7. Code: 1—Primary 2—Secondary Section Tool or Methodology TQM tools Benchmarking Cause and effect diagrams Concurrent engineering Cost of quality Design of experiment Evolutionary operation Brainstorming Checklists Delphi technique Nominal group technique Force field analysis Quality function deployment Quality loss function Statistical process control Flowchart analysis Work flow analysis Trend analysis tools Performance trend analysis Problem trend analysis Programmatic trend analysis Supportability trend analysis Reliability trend analysis Phase A Conceptual Trade Studies Phase B Concept Definition Phase C Design and Development Phase D Fabrication.1 8.5 2 2 2 1 1 1 1 1 Note: Phases discussed in reference 1.7 7.4 8.6 7. Integration.3 8.
California. Homewood.REFERENCES 1. “System Engineering Process (Short Course Lecture Notebook).S. Illinois 60430. September 1991. Santa Rosa. B. and Brocka. Implementing the Best Ideas of the Masters. 18 .1 1.: “Quality Management. M.” Center for Systems Management (CSM).2 Brocka.” Business One Irwin..
pseudoquantitative equations may be developed (as in probabilistic assessment equations for failure causes in fault tree analyses) to increase confidence in analysis results. complexity. etc. manufacturability.1. section 2. It is very important.1. CONCEPT DEVELOPMENT TOOLS Trade studies and costversusbenefit studies are presented in this section. etc. A trade tree may be presented without results or simply as a representation of options. moderate. provide a method to assess alternatives by determining the ratio of expected future benefits to expected future costs. etc. A trade tree is simply a pictorial representation of how highlevel alternatives (or issues) in the decision process are logically resolved into decreasingly lower level alternatives (or issues). These techniques are described in reference 2. etc.2. section 2. strong. manufacturability. safety. a comparison of relevant data (cost.) is then performed to rank those candidate design options in order of desirability. Costversusbenefit studies. are delineated with an associated weighting factor. The analytical hierarchy process (AHP) is a variation of the weighted factors analysis and is the most complex of the trade studies presented here. These tools are used to select the preferred option of several alternatives. This is used when subjective verbal expressions (equal.) are easier to develop than numerical (3 versus 3. The AHP provides a multicriteria analysis methodology that employs a pairwise comparison process to compare options to factors in a relative manner.2.1. The decision is then based upon the numerical results of the analysis. This approach allows for delineation of the facts and rationale that go into the subjective assessment of each of the options. These studies are categorized as either a weighted factor trade study or an analytical hierarchy trade study. very strong. This is often very difficult and the prioritization delineation may change during the early phases of the program. The options are then assessed with respect to each of the factors and an equation is developed that weighs this assessment.1 Description In general. All factors (program requirements) that are determined to be important. weight. that when the prioritization changes. complexity. and often overlooked. A key to any trade study is the initial selection and prioritization of specific desirable attributes. A weighted factor trade study is usually performed when each of the options under consideration is very well defined and there is good definition of the program requirements as well. are quantitative and/or qualitative comparison techniques to choose an alternative when considering such items as cost. Trade studies.) assessments. trade (or tradeoff) studies provide a mechanism for systematic depiction of both system requirements and system design options for achieving those requirements. 2. A trade tree can be generated with either of the above two options.2. a cursory look at the significant. Once tabulated. Further. Pseudoquantitative numbers are then ascribed to the words and a score developed for each of the options.1 TRADE STUDIES 2. safety. A summary of the advantages and limitations of each tool or methodology discussed in this section is presented in table 21. weight. completed trades should be performed to determine any impacts to their conclusions. with the latter being a special version of the former. 21 .
and utility functions can prejudice the assessment and lead to incorrect results. (2) The analysis is only as good as the list of alternatives considered. (4) Options evaluated are not determined as a result of the study but must be decided upon prior to the assessment by the operator.) ones. (2) Adaptive to prioritization based upon programmatic (cost.Table 21. resources expended can be commensurate with the benefits of the task. If the system operating alternatives over the entire life cycle of the proposed environment is not understood or accurately system under consideration. (1) By performing a costversusbenefit analysis. (2) Provides documentation of the parameters evaluated If the system requirements are too general or vague. Concept development tools and methodologies. schedule) considerations as well as technical (weight. Costversusbenefit studies 2. measurable terms. Tool or Methodology Trade studies Section 2. reliability.2 22 . (2) Improper generation of selection criteria. etc. (3) The number of alternatives which can be considered is limited by the expenditure of resources required to perform the analysis. weight factors. (3) The analysis is flawed if incomplete or inaccurate cost estimates are used. the (1) The analysis is flawed if system requirements are analyst can assess the cost effectiveness of several incomplete or inadequate. (4) The analyst must be able to quantify the value of benefits. the effectiveness of benefits can not be addressed in specific.e.. (5) Weighting factors and advantages/disadvantages are very subjective (although objective data may be added which significantly complicates and enlarges the study) and this subjectivism is very near to the study conclusions. characterized.1 Advantages (1) Different kinds and/or levels of study allow flexibility in the depth of the review. Limitations (1) Very dependent upon the expertise of the analyst and the amount of available accurate quantitative data. (3) Identification of disadvantages of specific design option may lead to the definition of effective countermeasures if combined with other techniques. i. which are often intangible or insubstantial and difficult to characterize in terms of monetary value. and the prioritized options considered. An incomplete list of alternatives will lead to an incomplete analysis. the total costs can be underestimated.
2 Application These studies should typically be performed in phase A of NASA projects. This will lead to fewer resources to conduct the assessment without degradation of the results. and specific. such as selecting test methods. 7. Develop a trade tree (optional). The list of alternatives selected during brainstorming sessions may be reduced by eliminating alternatives which do not appear capable of meeting requirements. By performing step 6. c. the selection criteria should have the following characteristics: a. Develop and specify the selection criteria to be used in the analysis.1. b.) e. For large trade studies with many alternatives and criteria attributes. is often useful here. an AHP weighted trade study will be performed. These alternatives can be imposed or obtained in brainstorming sessions (sec. Prioritize the objectives/requirements if possible. (A numbering system. nonreliability related considerations (weight. or whenever a method is needed to select alternatives. Be expressed in general terms that mean the same thing to every evaluator. These requirements will provide the scope of the assessment and the basis for the selection criteria. sec. However. The selection criteria are benchmarks to assess the effectiveness and applicability characteristics of the alternatives to be considered.1. Identify credible alternative candidates for the system under consideration. Correlate directly to the established requirements and high priority issues.3 Procedures The procedures for performing a weighted trade study are presented below. The remaining alternatives should be described in sufficient detail that the relative merits between them can be ascertained. or performing makeorbuy decisions. create a trade tree to group alternatives with unique criteria attributes. A large trade study may be resolved into several smaller trade studies with fewer required total comparison evaluations. accurate. trade studies can also be performed in phase B.7). 5.1. These studies may also be used to help the designer delineate which system requirements are most important (used in conjunction with the Pareto chart analysis. 23 (2) (3) (4) . A trade tree is developed to graphically illustrate the alternatives and how highlevel alternatives in the decision process are logically resolved into decreasingly lower level alternatives. evaluating design change proposals. d. Ideally. 2.6). (1) Define the mission objectives and requirements for the system under consideration.2. These procedures are described in detail and were adapted from reference 2. The list may be reduced further by eliminating alternatives with low probability of successful implementation or those which are expected to exceed cost constraints. showing the specific correlation. maintainability. A trade study analysis allows a systematic approach to evaluation of design options with respect to programmatic considerations or other. Provide a distinction between alternatives without prejudice. These objectives and requirements should be clear. Be practical to measure or predict within acceptable uncertainty and cost limits. this will aid in the weight factors for the selection criteria. Be separate and independent from each of the other selection criterion in all aspects of the assessment. manufacturability).
such as cost. and 8 as values to be assigned when interpolating between two of the definitions. 4. low). and 9 to these definitions in order of increasing diversity between the given two attributes. The numerical values of the weight factors should sum to 100. If five definitions are used. 0 to 10). etc. Create a normalized matrix (all the attributes versus all the attributes) with these relationships. schedule. medium. (7) Generate utility functions (optional). The degree to which the individual criterion is resolved into components is dependent on how effective the criterion components can be evaluated. Establish a scale of the relative level of significance to the system objectives between two given criteria attributes. The following steps define this process: a. If attribute n has a numerical value of relative level of significance of “j” relative to attribute m. risk. the analyst may use test data. Determine the relative weights for each criterion component by performing an eigenvector analysis. 7. Assess each alternative relative to the selection criteria. For the ideal situation. assign the numerical values 1. Survey a group of qualified managers and engineers (or customers) to establish a consensus on the relative relationships between each attribute and the rest of the attributes.. Consult with the end user of the system (the internal or external customer) to verify that the selection criteria and weights are compatible with his needs. parametric 24 (8) .e. and represents the resolution limit of the assessment. then attribute m has a numerical value of relative level of significance of “1/j” relative to attribute n. Note that all elements of the diagonal of this matrix equal 1. The relationship may be a continuous function (not necessarily a straight line) or discrete values.(5) Establish weights for the selection criteria. c. similarity comparison. Determine the weight for all attributes by calculating the product of each individual attribute weighing factor and its weights of associated category headings. Establish three to five definitions to subjectively define this scale of relative level of significance. First estimate the performance of every alternative for a given criterion in terms of the measure of effectiveness used in generating the utility functions. high. a subjective verbal scale may be used (i. but not necessarily shared with the analysts to ensure that alternatives are assessed against each criterion objectively. modeling. b. For attributes other than technical. (6) Perform an analytical hierarchy process as described in reference 2. These weights should reflect the importance of each criterion relative to its importance to the overall selection decision. 3. Generate clarifications for each definition so that qualified managers and engineers can subjectively use the definitions. A relationship is established between a measure of effectiveness for each selection criterion and a common scale (for example. The weights should be given numerical values to accommodate objective comparisons between unrelated criteria. This technique is beneficial for very complex trade studies when operational data are not available and a subjective analysis is to be performed. Each criterion may be resolved into several levels of components to establish its weight. 5. d. e. This technique is used to establish a consistent scale for dissimilar criteria. The weights should be predetermined by the person (or group) with the ultimate decision authority. vendor provided data.2 to establish weights for the selection criteria (optional). 6. Reserve the numerical values of 2.. engineering experience.
(9) Tabulate the results.2. repeat the assessment. Table 22. If quantification of qualitative ranking is required. use caution in drawing conclusions. yj Weights w j = 100 Alternate x1 Alternates. reconsider the selection criterion and weighting factors.analysis. In reality. A typical table is illustrated in table 22 and was generalized from an example presented in reference 2. the alternative numerical total score is too close to make a decision. Assume that a difference in the conclusion of less than onehalf the quantified number of a onestep difference is an equivalent answer. Next.1 Criteria Criterion. determine the score for each alternative relative to a given criterion by correlating the estimate of performance for all the criteria to the mutual scale using the utility functions generated in step 7. x i (x1 through xn) Alternate x2 Alternate x3 Through Alternate xn–1 Score (0–10) Weighted Score Alternate xn Score (0–10) Weighted Score Score (0–10) Weighted Score Score (0–10) Weighted Score y1 y2 y3 y4 to ym–1 ym Total w1 w2 w3 s11 s12 s13 w 1 s11 w 2 s12 w 2 s13 s21 s22 s23 w 1 s21 w 2 s22 w 2 s23 sn1 sn2 sn3 w 1 sn1 w 2 sn2 w 3 sn3 wm s1m w ms1m ∑(wj sij) s2m w 2 s2m ∑(wj sij)j snm w msnm ∑(wj sij)j (10) Perform a sensitivity analysis to evaluate the merit of the results relative to making an alternate selection. Next.e. Generate a matrix of criteria versus alternatives to summarize the results from the preceding steps. Typical weighted trade study summary table. and regenerate the summary table for the weighted trade study. after the analysis is repeated.1. Select the superior alternative. then gather more data to increase the confidence level of the performance estimates. multiply the scores for all alternatives by the weight factor for the criterion (determined in steps 5 or 6) to determine the weighted score for all alternatives for that criterion. even when somewhat subjective (i. It is worthwhile. or other costeffective and reliable methods to generate the performance estimates. this is often very difficult to perform objectively.. If. (11) 25 . If this is the case. however. This corresponds to a confidence band for the evaluation. Select the alternative with the highest value of total weighted scores. Examine the results of the weighted trade study to see if any total weighted scores of any alternatives are closer in numerical value than is warranted in making a decision due to the confidence levels of the performance estimates that had been used to established the scores. heavy use of engineering experience). Repeat this procedure for all criteria attributes.
D 18 12 197 0. C 23 10 190 0. A 16 7 180 0.88 Avg.2. The selection decision will be based on comparing the four alternatives to the following criteria attributes and their associated weight factors: Item 1 2 3 4 5 6 Criteria Attribute Average fuel economy Acceleration (0 to 60 mph) Braking (70 to 0 mph) Road handling Implement new technology risk Cost Total Weight Factor 20 15 15 15 10 25 100 Utility functions have been generated for each criteria attribute and are presented in figure 21.000 Alt. 26 . The estimates for each alternative relative to each criteria attribute are listed below: Measure of Effectiveness miles per gallon seconds feet g Dollars.86 Low 21 Alt. B 19 9 177 0. skidpad) Implementing new technology risks Cost From the information given above. 20 Alt.4 Example Problem: Four alternatives for a new automobile design are being considered. × 1.1. and select the superior alternative.78 Very low 22 Item 1 2 3 4 5 6 Criteria attribute Average fuel economy Acceleration (0 to 60 mph) Braking (70 to 0 mph) Road handling (300 ft dia.83 High 24 Alt. formulate a weighted factor trade study summary table.
feet Score 10 5 0 0. seconds Score 10 0 175 200 Braking (70 to 0 mph). mpg Very Very Avg.000 Figure 21. High Low Implementing New Technology Risks 0 Score 10 5 Score 10 5 0 5 15 Acceleration (0 to 60 mph). dollars × 1.Score 10 5 Score 10 5 0 15 25 Average Fuel Economy. Example utility functions. 27 .9 Road Handling. g 5 0 15 25 Cost.7 0.
5 105 60 125 638.5 10 3 60 75 75 22.5 3 5 5 1.5 127.5 37. schedule) as well as technical ones (weight.5 8 4 20 135 147 67.5 Advantages The following advantages can be realized from performing trade studies: (1) (2) (3) (4) Different kinds and/or levels of study allow flexibility in the depth of the review.8 4.5 100 75 407. 28 . 2. Scores were determined from effectiveness measures for all alternatives relative to all criteria attributes and the utility functions.e.5 80 100 549. This technique is adaptive to prioritization based upon programmatic considerations (cost.5 40 25 502. i. and (c) selection methodology are recorded. alternative B is the preferred option.1.5 4 8 9.5 8.Solution: Presented in figure 22 is the completed weighted factor trade study summary. Example weighted factor trade study summary table..5 8 7.5 Figure 22. (b) considered alternatives.).5 2. Identification of disadvantages of a specific design option may lead to the definition of effective countermeasures if combined with other techniques.5 4 1 160 112. reliability. Criteria Attribute Item Weights wj =100 Alternate A Score (0–10) Weighted Score Alternates. Based on the results of the trade study.9 7 6 5 80 120 148. etc. x i Alternate B Score (0–10) Weighted Score Alternate C Score (0–10) Weighted Score Alternate D Score (0–10) Weighted Score 1 2 3 4 5 6 Total 20 15 15 15 10 25 1 9 9. The method provides a clearly documented analysis in which the (a) prioritized objectives and requirements. resources expended can be commensurate with the benefits of the task.
These studies should typically be performed in phase A. New Jersey. pp.6 Costversusbenefit studies. These studies can be used when two or more alternatives are being considered with fixed cost constraints.5.” John Wiley & Sons. 2. 1980. B.32. they could also be performed in phases B or C. Both the expected future benefits and costs are expressed in terms of present value. fixed desired results or benefits. provide a method to assess alternates by determining the ratio of expected future benefits to expected future costs. N. Cross. Costversusbenefit studies. W. 2.1. pp. as described in reference 2. weight factors. L. this significantly complicates and enlarges the study) and this subjectivism significantly influences the study conclusions. Saate.4 or machines or systems2. 101–121. Englewood Cliffs.1 The number of alternatives which can be considered is limited by the expenditure of resources to perform the analysis. 67–72.2 Application Benefitcost analyses apply to the selection of projects 2. 1989.” McGrawHill.52. 29 .S.2. 2..2. Weighting factors and advantages/disadvantages are very subjective (although objective data may be added in the analytical hierarchy process approach (AHPA). The alternatives are ranked in decreasing order with the preferred option being the alternative with the highest benefittocost (B/C) ratio.: “Analytical Hierarchy Process.2.6 Limitations The following limitations are associated with performing trade studies: (1) (2) (3) (4) (5) These techniques are very dependent upon the expertise of the analyst and the amount of available accurate quantitative data.32. T.2 COSTVERSUSBENEFIT STUDIES 2. 1990.2.: “Engineering Design Methods. and Fabreycky. 2. as discussed in this section.J.4 benefitcost ratio analyses.1 Options evaluated are not determined as a result of the study but must be decided upon prior to the assessment by the person (or group) with decision authority.7 Bibliography Blanchard. will apply to the selection of system or system element alternatives based on their relative B/C ratios. Prentice Hall. or when both costs and desired results vary.1.2.2.1 Description Costversusbenefit studies are also known as benefitcost analyses. however.1 Improper generation of selection criteria.” Second edition.: “System Engineering and Analysis.5 and costbenefit analyses. and utility functions can prejudice the assessment and lead to incorrect results.6 based on their relative B/C ratios. 2. while falling within overall cost restraints. 2.
3. Calculate the ∆Bto∆C ratio between the alternatives. Since it is subjective. if possible. The cost should include such items as initial investment. 210 (5) (6) . The detriments might include such items as loss of production time. step 11). Maximum B/C ratio.2.6 Note that the alternatives require an implicit determination of technical and schedule viability. (4) Identify the cost or savings for each alternative.3 Identify the benefit or detriments for each alternative. The benefits might include such items as increased performance. (1) Define the requirements for the system or system element under consideration.3.2. etc. mutually exclusive alternatives. Table 23.3 Each alternative should be characterized to a level of completeness such that all substantial costs and benefits can be identified.2. environmental impacts. result B fixed.75.6.2. 2. etc. if one alternative is selected.3 The cost risk and technical maturity for each alternative may be included as a multiplying factor (f) for this analysis. Example selection criteria for costversusbenefit analyses.5. that is. The savings should include such items as residual or salvage values. Two alternatives are being considered with neither budget C or desired result B fixed. Define a list of credible. Then choose the higher cost alternative. the others are not to be implemented. Choose the lower cost alternative.5. and ongoing operating and maintenance expenses (including depreciation) for the life of the system. Translate general and vague requirements into specific. compressed schedules.5. These requirements should be measurable and verifiable. Condition or Circumstance Budget C is fixed Desired result B is fixed. value added due to increase productivity.2. Selection Criteria Maximum B/C ratio. Specify the time interval (expected operating life of the system) to which the analysis is to apply. increased schedules. etc. increased safety. reduced operating times. unless the ∆Bto∆C ratio is ≥1. reduced property value. quantitative requirements in which system effectiveness can be measured and assessed.3 Procedures The following procedures to perform costversusbenefit studies were adapted from references 2. or 1—is probably as fine a distinction as is warranted. 2. Develop and specify the selection criteria to be used in the analysis. The example selection criteria presented in table 23 were adapted from reference 2.2.2. increased equipment operating costs. and 2. increased reliability. (2) (3) Select alternative per benefitcost ratio More than two alternatives are being considered with neither budget C or desired incremental analysis (sec. use of only three factors—0.6 Prioritize these requirements. 0. 2.
savings. perform step 12.5 (13) (14) (15) If there exists any alternatives with a B/C ≥1.n listed in order of C. the preferred alternative may not necessarily have the greatest B/C ratio. benefits.2. 2.(7) Develop cost and savings estimates and the benefits and detriments estimates for each alternative. Therefore. Identify the interest rate that will be assumed for the analysis. If the ∆B/Ci is <1. Calculate the B/C ratio for each alternative by dividing the total benefit (B) by the total cost (C). The preferred alternative is the last alternate listed in order of increasing cost whose incremental ∆B/Ci is >1. Each system has a different total cost and the capabilities of each system are different in terms of maximum number of channels. perform steps 13 through 17. ∆B/Ci = ∆Bi /∆ Ci . where ∆Bi = Bi +1 – Bi and ∆Ci = Ci+1 – Ci for each ith pair of (n – 1) pairs of n alternatives where alternative i = 1.2. Determine the incremental B/C ratio ∆B/C for each consecutive pair of alternatives with increasing total cost. and detriments estimates to present worth values. Determine the total benefit value for each alternative by algebraically summing all benefits as positive values and all detriments as negative values. examine each distinct increment of increased cost investment. Order the remaining alternatives in sequence of increasing total C... If the ∆B/Ci is >1.2. then do not give further consideration to alternatives with a B/C <1. then the increment is beneficial.4 Example Problem: Five data acquisition systems (DAS) are under consideration to acquire data for solid rocket motor tests in a test stand over a 10yr time interval.2. 2. maximum sample 211 . Every attempt should be made to base cost and savings estimates on actual historical cost data. then the increment is not beneficial.3 The estimates for each alternative should be for the same time interval specified in step 6. and select the superior alternative based on selection criteria established in step 3. For cases with cost restraints and desired results or benefits that vary.2..3 Convert all costs..5 Determine the total cost for each alternative by algebraically summing all costs as positive values and all savings as negative values. (16) (17) Next.5 (8) (9) (10) (11) (12) Rank the alternatives relative to their respective costtobenefit ratios. For cases with fixed cost restraints or fixed desired results or benefits.2.
required maintenance. Since the B/C for system D is <1.50 B 300k 400k 1.6 Step 4.0 A–E 100k 150k 1.33 C 750k 900k 1. By inspection. Identify the preferred alternative as the last alternate listed in order of increasing cost whose incremental ∆B/Ci is >1. Determine the incremental B/C ratio ∆B/C for each consecutive pair of alternatives with increasing total cost. turnaround time between tests.20 D 800k 750k 0. The present values of cost and benefits were determined over a 10yr expected system life. Increment ∆ Total cost (dollars) ∆ Total benefits (dollars) ∆ B/C E–B 100k 200k 2. The present value of the estimated total cost and total value of combined benefits of the system are presented below. List the remaining options in order of increasing total cost. System Total cost (dollars) Total benefits (dollars) B 300k 400k E 400k 600k A 500k 750k C 750k 900k A 500k 750k 1. Therefore.rates. the last incremental ∆B/C with a value >1 is A–E. the preferred alternative is DAS A.5 C–A 250k 150k 0. this option will no longer be considered. Delete options with a B/C ratio <1.93 E 400k 600k 1. and mean time between system failures.50 Step 3. System Total cost (dollars) Total benefits (dollars) B/C Solution: Step 1. with an assumed annual interest rate of 10 percent. 212 . Step 2. data accuracy. Perform a costversusbenefit analysis to determine the best alternative.
New Jersey.7 Bibliography Thuesen.3 The analysis does not take into account technical complexity or maturity of an alternative.2.6 The analysis is flawed if incomplete or inaccurate cost estimates are used.2.” Seventh edition.2.: “Engineering Economy.6 The analyst must be able to quantify the value of benefits.6 Limitations Costversusbenefit analyses possess the following limitations: (1) The analysis is flawed if system requirements are incomplete or inadequate. new design efforts. Further.. Prentice Hall. benefits cannot be addressed in specific.2. G.J. 1989. An incomplete list of alternatives will lead to an incomplete analysis. 2. measurable terms of effectiveness. W. except as a cost uncertainty factor. As cost is generally only one of many factors.2. Englewood Cliffs. If the system requirements are too general or vague. system reliability and safety issues are not treated except by the selection of the alternative. and Fabrycky. but more appropriate to productionlevel design solutions. If the system operating environment is not understood or accurately characterized.2.2. which are often intangible or insubstantial and difficult to characterize in terms of monetary value. The method provides a clearly documented analysis in which the prioritized objectives/requirements. (2) (3) (4) (5) 2.2. this tool is generally insufficient for selection of large. 213 .5 Advantages The following advantages are realized by performing costversusbenefit analyses: (1) (2) The analyst can assess the cost effectiveness of several alternatives over the entire life cycle of the proposed system under consideration. the total costs can be underestimated.5 The analysis is only as good as the list of alternatives considered. and the selection methodology are recorded.J. the alternatives considered.
Inc. San Jose. J.4 2. R.H. H...” IEEE Transactions...1 2.6 “System Engineering Management Guide. Chaplin and Hall Ltd. August 1983. California.” Defense Systems Management College.J. 140–155.” Second edition. Saaty.: “Quantitative Management in R & D. F. pp. Jelen. Beattie.D.5 2.3 2. 214 . 1983. D.REFERENCES 2. Engineering Press.: “Cost and Optimization Engineering.. and Black. and Reader. New York.” John Wiley & Sons Inc.: “Engineering Economic Analysis. January 1990.G. New York. McGrawHill Book Company.: “System Engineering Methods.C. 1971. C.: “Priority Setting in Complex Problems. 1967. Newnan.2 2. 1983.” Second edition. Chestnut.L.” London. T.
Each of the symbolic logic techniques has its own unique advantages and disadvantages. If quantitative data are not available. the reliability.4. The energy/flow barrier analysis discussed in section 3. SYSTEM SAFETY AND RELIABILITY TOOLS This section describes several system safety and reliability tools available to the system engineer analyst. These models are developed using forward (bottomup) or backwards (topdown) logic. The failure modes and effects analysis (FMEA). can be used to identify failure modes and their consequences or effects. such as the preliminary hazard analysis (PHA) technique discussed in section 3. Often the value is in a comparison of numbers that allows effective resource allocation. then subjective probability estimates may be used as described in section 3. “What will cause a given failure to occur?” The analyst views the system from a “topdown” perspective. If the probability of failure (P F) is examined.1. The probability of a successful operation is the reliability. Caution must be exercised when quoting reliability numbers.. then the model is generated in the failure domain and if the probability of success (PS ) is examined. Sometimes it is beneficial to construct a model using one technique. the analyst can model either in the failure or success domain (or both domains). Several symbolic logic methods are presented in this section. they can be further explored if failure modes of the elements of the system are known. or risk.3 is also a technique to identify hazards and to evaluate their corresponding countermeasures.e. Classically. discussed in section 3. Probabilities are propagated through the logic models to determine the probability that a system will fail or the probability the system will operate successfully.12. for example. When using forward logic the analyst builds the model by repeatedly asking. Also discussed in section 3. associated with each failure mode. then transform that model into the domain of another technique to exploit the advantages of both techniques. the analyst repeatedly asks. then convert the final probabilities to the desired domain using the following expression: PF + PS = 1.9 to transform any one of the above models into the other two by translating equivalent logic from the success to failure or failure to success domains.4 is the failure modes.3. and criticality analysis (FMECA). rather than “exact” determination of 31 . These methods construct conceptual models of failure or success mechanisms within a system. The risk assessment matrix is discussed in section 3. For convenience. reliability diagrams are generated in the success domain. Fault trees are generated in the failure domain. Methods are presented in section 3. Use of confidence bands is important. It is used in conjunction with hazard analyses. When using backwards logic to build a model. and event trees are generated both in the success and failure domains. This means he starts by looking at the lowest level elements in the system and their functions. These tools are also used to determine either the probability of failures occurring or the probability that a system or component will operate successfully. The PHA can be used to identify hazards and to guide development of countermeasures to mitigate the risk posed by these hazards. i. effects. the FMEA. This means he starts by looking at a high level system failure and proceeds down into the system to trace failure paths.2. is a bottomup technique. Probability data may be derived from available empirical data or found in handbooks. Once hazards are identified. The FMECA is similar to the FMEA but also addresses the criticality. This device supports a standard methodology to subjectively evaluate hazards as to their risks. The symbolic logic techniques discussed in this section and their characteristics are presented in table 31. then the model is generated in the success domain. “What happens when a given failure occurs?” The analyst views the system from a “bottomup” perspective.
13. This technique uses advanced statistical methods to determine P F modes.14.7 3. Technique Reliability block diagram Fault tree analysis Success tree analysis Event tree analysis Causeconsequence analysis Directed graph matrix analysis Section 3. 32 . Risk for a given hazard can be expressed in terms of an expectation of loss. probabilistic risk assessment (PRA) is discussed in section 3. and how and where the information should be measured in a system to detect the onset of a failure mode that could damage the system. The severity and probability dimensions of risk define a risk plane. isorisk contours depict constant risk within the plane. Symbolic logic techniques. Risk is the product of severity and probability (loss events per unit time or activity).1 Description The risk assessment matrix.11 Success Domain √ √ √ √ √ Failure Domain √ √ √ √ √ √ Forward (BottomUp) Backwards (TopDown) √ √ √ √ √ expected reliability levels.1. is a tool to conduct subjective risk assessments for use in hazard analysis. Note: the probability component of risk must be attached to an exposure time interval. This technique allows the analyst to determine what information is needed.1. as described in reference 3. This is a general methodology that shows how most of the techniques mentioned above can be used in conjunction to assess risk with severity and probability.1 Risk Assessment Matrix 3.15. Failure mode information propagation modeling is discussed in section 3. Probabilistic design analysis (PDA) is discuss in section 3. Finally.8 3. 3. A summary of the major advantages and limitations of each tool or methodology discussed in this section is presented in table 32.10 3. As shown in figure 31.5 3.Table 31. or the longterm rate of loss.6 3. the combined severity and probability of loss. The definition of risk and the principle of the isorisk contour are the basis for this technique.
reliability.1 Advantages Provides standard tool to subjectively assess risk. (2) Fails to identify certain classes of hazards. Failure modes and effects (and criticality) analysis 3. (1) Addresses only one desirable event or condition that must be foreseen by the analyst.7 Assesses probability of favorable outcome of system operation. e.g.Table 32. Event tree analysis 3.5 Fault tree analysis 3. Component reliability estimates may not be readily available. Preliminary hazard analysis 3. (2) Functions simultaneously in failure and success domain. asphyxia in oxygendeficient confined spaces.6 Success tree analysis 3.2 Identifies and provides inventory of hazards and countermeasures. Energy flow/barrier analysis 3. (1) Addresses only one undesirable event or condition that must be foreseen by the analyst. System safety and reliability tools and methodologies—Continued Tool or Methodology Risk assessment matrix Section 3. This technique offers no additional information and is only as good as the input model. given component reliability. Does not address and their consequences. (2) Comprehensive trees may be very large and cumbersome. Does not address coexisting system failure modes.8 (1) Enables assessment of probabilities of coexisting faults or failures. (1) Enables assessment of probabilities of coexisting faults or failures. total calculated reliability may be unrealistically high. and event tree transformations 3. block diagram. Identify hazards associated with energy sources and determines if barriers are adequate countermeasures. Fault tree.3 (1) Does not address coexisting system failure modes. Limitations Only used to assess risk of hazards. Reliability block diagram 3.. (3) End events need not be anticipated. (1) Addresses only one initiating challenge that must be foreseen by the analyst. (2) Discrete levels of success and failure are not distinguishable. (2) Comprehensive trees may be very large and cumbersome. System reliability can be derived. risk assessment of these failure modes. (2) May identify unnecessary design elements.9 Allows the analyst to overcome weakness of one technique by transforming a model of a system into an equivalent logic model in another analysis technique. does not identify hazards. A symbolic logic model that is relatively easy for the analyst to construct. Accident sequences through a system can be identified.4 Thorough methods of identifying single point failures Can be extremely labor intense. A criticality analysis provides a coexisting system failure modes. 33 .
Probabilistic design analysis 3.11 (1) Allows the analyst to examine the fault propagation (1) Trained analyst and computer codes and resources to through several primary and support systems.14 (1) Allows the analyst a practical method of quantitatively and statistically estimating the reliability of a system during the design phase. 34 . and double point failures can be determined with less computer computation than with FTA.Table 32. (3) Discrete levels of success and failure are distinguishable. (2) Provides alternative to the traditional method of imposing safety factors and margins to ensure system reliability. Extrapolation between populations can render technique nonviable. (2) End events need not be anticipated. and senseless risk.10 Advantages (1) Enables assessment of probabilities of coexisting faults or failures. (2) Only identifies single point (singleton) and dual points (doubleton) of failure. (2) Historical population data used must very close to asplanned design population to be viable. Failure mode information propagation modeling 3. Combinatorial failure probability analysis using subjective information 3. Probabilistic risk assessment 3. Measurement requirements can be determined that if implemented can help safeguard a system in operation by providing warning at the onset of a threatening failure mode. Techniques can be misapplied and results misinterpreted. Tool or Methodology Causeconsequence analysis Section 3. Performing the techniques of this methodology requires skilled analysts.13 (1) This technique is only applicable if the system is operating in a near normal range and for the instant of time just before initiation of a failure. avoids accepting unknown. System safety and reliability tools and methodologies—Continued. intolerable. Limitations (1) Addresses only one initiating challenge that must be foreseen by the analyst. (2) Minimal cut sets. (1) Analyst must have significant experience in probability and statistical methods to apply this technique. unless used in a comparative fashion.12 Allows analyst to perform qualitative probabilistic risk assessment based on the exercise of subjective engineering judgment when no quantitative data is available. Should only be used when actual quantitative failure data is unavailable. Directed graph (digraph) matrix analysis 3. (2) Data and results. (2) May be very subjective as to consequence severity. That method might be flawed if significant experience and historical data of similar components are not available. perform this technique may be limited. Use of actual quantitative data is preferred to this method.15 Provides methodology to assess overall system risks. singlepoint failure. may be poorly received.
The PHA. convention. is such an analysis. the two variables that constitute risk. Risk should be evaluated for worst credible case. and acceptance limits for risk assessments (fig. CATACLYSMIC R=K >K 3 In cr e R asi is n k g 2 R=K >K 2 1 R=PxS=K 1 LIKELY Isorisk Contours RISK is CONSTANT along any ISORISK CONTOUR. Figure 31. 35 .2. 3.2 Application The risk assessment matrix is typically performed in phase C but may also be performed in phase A. define a RISK PLANE. defined in section 3. These risks are expressed in terms of severity and probability. not worst conceivable case.SEVERITY SEVERITY and PROBABILITY. 32). This technique is used as a predetermined guide or criteria to evaluate identified hazards as to their risks. it will result in a nonviable analysis. The concept of the isorisk contour is useful to provide guides. 0 NEVER PROBABILITY is a function of EXPOSURE PROBABILITY INTERVAL.1. Failure to assume credible (even if conceivable is substituted) may result in an optimistic analysis. conditions. Use of this tool allows an organization to institute and standardize the approach to perform hazard analyses. Risk plane.
remote. Since the assessment is subjective. critical. 1 RISK ASSESSMENT GUIDES: If Risk for a given Hazard can be assessed at any severity level. Categorize and scale the subjective severity levels for each target. These matrix cells fix the limits of risk tolerance zones. “impossible. The lowest step. improbable.3 below: Procedures Procedures. product loss. 34(a)). 33). Isorisk contour usage. too many steps add confusion with no additional resolution (fig. Increase adjacent probability steps by orders of magnitude. and impossible (adapted from MIL–STD– 882C). Avoid creating too many matrix cells. as described in reference 3.1. 34(b)). occasional.) 3 ACCEPTANCE: Risk Tolerance Boundaries follow isorisk contours. such as catastrophic. b. and environmental effects.3. Approximate the continuous. Note that not the analyst but management establishes and approves the risk tolerance boundaries.2 RISK ASSESSMENT CONVENTION: If possible. (2) (3) (4) 36 . equipment. (It’ll fall at the top end of its own isorisk contour. downtime. 3.1. such as frequent. and negligible. probable.Credible Severity of outcome. Create a matrix of consequence severity versus the probability of the mishap. for developing a risk assessment matrix are presented (1) Categorize and scale the subjective probability levels for all targets. (Most.” is an exception (fig. isorisk contour functions in the risk plane with matrix cells (fig. marginal. but not all hazards behave this way. Be wary of exceptions — usually highenergy cases.2 Note: A target is defined as the “what” which is at risk. assess Risk for the Worst. an isorisk contour gives its probability at all severity levels. The following hints will be of help when creating the matrix: a.) SEVERITY NOT ACCEPTABLE PROVISIONALLY ACCEPTABLE ACCEPTABLE (de minimis) 0 0 PROBABILITY Figure 32. One typical breakout of targets is personnel.
(1) unacceptable.e. The scenario should be familiar to potential analysts or characterize a tolerable perceivable threat. and (3) routinely accepted (fig. make sure every onestep path does not pass through more than one zone (fig.. d. Steps in the Matrix define Risk Tolerance Boundaries. isorisk contour functions in the Risk Plane. 34(c)). Assign its risk to the highest level severity cell just inside the acceptable risk zone. Risk plane to risk matrix transformation. 34(d)).S E V E R I T Y “Zoning” the Risk Plane into judgmentally tractable cells produces a Matrix.. 37 . PROBABILITY S I E V E II R I T Y III IV F E D C B A Matrix cells approximate the continuous. i. (2) accepted by waiver. (5) Calibrate the risk matrix by selecting a cell and attaching a practical hazard scenario to it. PROBABILITY Figure 33. as there are desired categories of resolution to risk issues. There should only be as many zones.e. less familiar risks. Establish only a few risk zones. This calibration point should be used as a benchmark to aid in evaluating other. i. Avoid discontinuities in establishing the risk zones. c.
or • (1) Avoided. F I S E V E II R I T Y III 6 E 5 D 4 C 3 B 2 A 1 7 5 4 3 2 A 24cell Matrix can be resolved into 9 levels of “priority. Figure 34.” or even more.Factors of 10 separate adjacent Probability Steps. 2 3 PREFERRED PROBABILITY (b) Do not create too many cells. Helpful hints in creating a risk assessment matrix—Continued 38 . D = 10 E C = 10 D B = 10 C A = 10 B …but F = 0 (“Impossible”) F I S E V II E R I T III Y IV E D C B A 1 2 3 PROBABILITY (a) Useful conventions. A Hazard’s Risk is either… • (3) Routinely Accepted • (2) Accepted by Waiver. But what are the rational functions for the many levels? 8 6 5 4 4 IV 9 FLAWED 8 7 F I S E V II E R I T III Y IV E D C B A 1 PROBABILITY Three zones will usually suffice.
Helpful hints in creating a risk assessment matrix—Continued. 39 . Figure 34. 3 FLAWED F PROBABILITY I S E V II E R I T III Y IV E D C B A 1 Keep it SIMPLE! 4 6 = 24 cells is better than 7 12 = 84 cells 2 3 PREFERRED PROBABILITY (d) Do not create too many zones. L I S E V E R I T Y II III IV V VI VII K J I H G F E D C B 1 A 2 Subjective judgment can’t readily resolve more than six discrete probability steps. Added steps become confused/meaningless.F I S E V II E R I T III Y IV E D C B A 1 ? ? 3 2 Can a countermeasure make the “leap” from Zone (1) to Zone (3) in a single step? FLAWED F PROBABILITY I S E V II E R I T III Y IV E D C B A 1 Make every onestep path from a high Risk Zone (1) to a lower Risk Zone (3) pass through the intermediate Zones (2). 2 3 PREFERRED PROBABILITY (c) Avoid discontinuities.
Typical risk assessment matrix.1.3.3. Example interpretations of the severity and probability steps for this matrix are presented in figure 36. 2 3 Operation permissible.4 Example A typical risk assessment matrix. timelimited waiver. Figure 35. 310 . Severity of F Consequences IMPOSSIBLE I CATASTROPHIC II CRITICAL III MARGINAL IV NEGLIGIBLE Probability of Mishap** E IMPROBABLE D REMOTE C OCCASIONAL B PROBABLE A FREQUENT 1 2 3 Risk Code/ 1 Actions Imperative to suppress risk to lower level. adapted from MIL–STD–882C. endorsed by management. : *Adapted from MILSTD882C **Life Cycle = 25 yrs. Operation requires written. NOTE Personnel must not be exposed to hazards in Risk Zones 1 and 2.2 is presented in figure 35.
Figure 36.Severity of Consequences CATEGORY/ DESCRIPTIVE WORD I CATASTROPHIC Death >1M PERSONNEL ILLNESS/ INJURY EQUIPMENT LOSS ($)** DOWN TIME PRODUCT LOSS ENVIRONMENTAL EFFECT LEVEL Probability of Mishap** DESCRIPTIVE WORD DEFINITION >4 months Longterm (5 yrs or greater) environmental damage or requiring >$1M to correct and/or in penalties A FREQUENT Likely to occur repeatedly in system life cycle Provide stepwise scaling of SEVERITY levels for each TARGET. readily repaired and/or requiring <$1K to correct and/or in penalties PROBABILITY is a function of EXPOSURE INTERVAL. 311 . III MARGINAL Minor injury or minor occupation al illness 1K to 250K 1 day to 2 weeks environmental damage or requiring $1K$250K to correct and/or in penalties D REMOTE Not likely to occur repeatedly in system life cycle. II CRITICAL Severe injury or severe occupational illness 250K to 1M 2 weeks to 4 months Values as for loss Equipment Loss Mediumterm (15 yrs) environmental damage or requiring $250K$1M to correct and/or in penalties Shortterm (<1 yr) B PROBABLE Likely to occur several times in system life cycle C OCCASIONAL Likely to occur sometime in system life cycle Provide stepwise scaling of PROBABILITY levels for all TARGETS. F IMPOSSIBLE Physically impossible to occur Decide on TARGETS. Severity and probability interpretations. *Adapted from MILSTD882C **Life Cycle = 25 yrs. but possible IV NEGLIGIBLE Minor environmentNo injury or illness <1K E IMPROBABLE Probability of occurrence cannot be distinguished from zero <1 day al damage.
and an assessment of their remaining risk after countermeasures have been imposed.1 (1) The risk matrix provides a useful guide for prudent engineering. assessments of risks.2 Preliminary Hazard Analysis 3.” NUREG/GR0005. “PreMarket Notification. This method is subjective without data and is a comparative analysis only. not quantitative. Department of Defense Instruction.“System Safety.1 Description A PHA. 312 . Assessing risk subjectively avoids unknowingly accepting intolerable and senseless risk.6 Limitations The risk assessment matrix possesses the following limitations:3. No. (2) (3) The risk matrix provides a standard tool of treating the relationship between severity and probability in assessing risk for a given hazard. often included is a tabular listing of countermeasures with a qualitative delineation of their predicted effectiveness.5 Advantages The risk assessment matrix provides the following advantages: 3. produces a line item tabular inventory of nontrivial system hazards. 5000. Medical devices.1. A PHA is an early or initial system safety study of system hazards.” sec.2.119 (e).1 (1) (2) The risk assessment matrix can only be used if hazards are already identified. 3.1.3. vol.90. vol.” 3.” NASA NHB 1700.36. and improves resource distribution for mitigation of loss resources. “RiskBased Inspection – Development of Guidelines. 1910. allows operating decisions to be made.7 Bibliography Code of Federal Regulations. “Process Safety Management of Highly Hazardous Chemicals. 807. Also. 3.” sec. Code of Federal Regulations.1.1. 3. vol. as described in reference 3. This inventory includes qualitative. This tool does not assist the analyst in identifying hazards.3. 21. 29. “System Safety Engineering & Management.
This tool is applied to cover wholesystem and interface hazards for all mission phases.3 313 . Tullahoma. or whether current installed countermeasures will be considered.3. then a proficient engineer with knowledge of the system should identify the hazards. facilities. emergency shutdown. Examine system specifications and expectations. activation. 3. and consensus standards. at any point in the life cycle of a system. *Provided (3) (4) courtesy of Sverdrup Technology. Identify and observe the levels of acceptable risk that have been predetermined and approved by management.3. as described in reference 3. e. Identify the targets threatened by each hazard. and outcome (consequence).2 Application PHA’s are best applied in phase C but may also be applied in phase B. Review system safety studies from other similar systems. regulations. to be protected. Use intuitive “engineering sense. etc. but that assessment should be reviewed by a peer. d. maintenance. c.1). A team approach to identifying hazards.2. mechanism (process).. 3. productivity.). Review codes. f. A list of proven methods* for finding hazards is presented below: a. Define the extent of the system to be assessed. etc. These resources are targets.3 Procedures A flowchart describing the process to perform a PHA is presented in figure 37. are presented below: (1) (2) Identify resources of value. deactivation. equipment. g. Inc. Define the physical boundaries and operating phases (such as shakedown.” 3.3. Consult checklists (app. A hazard is defined as an activity or circumstance posing “a potential of loss or harm” to a target and is a condition required for an “undesired loss event. 7. Examine and inspect similar facilities or systems and interview workers assigned to those facilities or systems. D).2. environment. however.7). such as personnel. standard operation. Detect and confirm hazards to the system. Interview current or intended system users or operators. Procedures for performing PHA’s. mission or test objectives. If schedule and resource restraints are considerations. This tool allows early definition of the countermeasure type and incorporation of design countermeasures as appropriate. such as if the assessment is based on an asbuilt or asdesigned system. is recommended over a single analyst. such as brainstorming (sec. These limits may be the risk matrix boundaries defined in a risk assessment matrix (sec. A PHA may be carried out. Tennessee.3 Hazards should be distinguished from consequences and considered in terms of a source (hazard). State other assumptions.” b.
PHA Process flowchart 314 .Figure 37.
Consider all energy sources. Review historical documents—mishap files.7)—mentally develop credible problems and play “whatif” games. Remember that severity for a specific hazard varies as a function of targets and operational phases. 3. Consider all mission phases. environment. (6) Assess risk for each hazard using a risk assessment matrix (sec. Select countermeasures in the following descending priority order to optimize effectiveness: (1) “design change. 7. will induce two or more fault/failure conditions within a system. draw on the experience of several experts as opposed to a single analyst.1). what happens if they get out of control? (5) Assess worstcredible case (not the worstconceivable case) severity and probability for each hazard and target combination. Since probability is determined in a subjective manner. This interval can be in terms of time. if it exists. An interval of 25 to 30 yr is typically used and represents a practical value. if unacceptable. e. target. Keep the following considerations in mind during the evaluation: a. equipment. What is necessary to keep them under control. manufacturer’s reliability analyses.h. c. The matrix should be consistent with the established probability interval and force or fleet size for this assessment. Brainstorm (sec. d. or personnel tendencies.” (4) “warning devices. k. National Safety Council data. j.” (3) “safety devices. A probability interval must be established before probability can be determined. or months are too brief to be practical. or each human operator working life span.” (2) “engineering safety systems. or number of cycles or operations. Categorize each identified risk as acceptable or unacceptable.” A common cause is a circumstance or environmental condition that. population. Consider “common causes. days. and operational phase. The assessment will underestimate the true risk if a shortterm probability interval is used unless the risk acceptance criterion is adjusted accordingly. The interval should depict the estimated facility. b.3 (7) (8) 315 . nearmiss reports.”3. weeks. i. m. or develop countermeasures for the risk. The probability for a specific hazard varies as a function of exposure time. Probability intervals expressed in hours. etc. OSHArecordable injury rates.” and (5) “procedures and training. Consider “external influences” like local weather. l.
design changes have the highest potential for cost and schedule impact). Prepared by/Date: *Target Codes: P—Personnel T—Downtime R—Product Figure 38. Inspect flange seal at 2month intervals. Emergency Shutdown 3 Analysis: SrdA (Chem/Int) System Number: __________ Revision X Initial Addition Hazard Target* Probability Interval: 25 years Date: 25 Feb. Routine Operation.2. determine new countermeasures and reevaluate the risk. 316 . Show Risk (from assessment matrix) for hazard “asis” — i.. If countermeasures are developed.4 Example An example of a completed PHA worksheet3. Describe newly proposed countermeasures to reduce Probability/Severity. 2. Reassess Probability/Severity. I E 3 II D 3 III D 3 Show hazard alphanumeric designator. (9) (10) Reevaluate the risk with the new countermeasure installed. and regasket during annual plant maintenance shutdown (P). (A blank form is included in appendix E. producing toxic vapors and attacking nearby equipment. while in decreasing order of effectiveness.) Preliminary Hazard Analysis Brief Descriptive Title (Portion of System/Subsystem/Operational Phases covered by this analysis): Pressurized UnFo Containment and Replenishment Reservoir and Piping / Startup. A trade study (sec.042 — Flange Seal A29 leakage. presuming new countermeasures to be in place. 1993 Risk Before Probability Risk Code Severity Description of Countermeasures Severity Identify countermeasures by appropriate code letter(s): D = Design Alteration E = Engineered Safety Feature S = Safety Device W = Warning Device P = Procedures/Training Risk After Probability Risk Code Hazard No.Note that this delineation. and show Risk (from assessment matrix) for hazard.a. new countermeasures must be developed. E—Equipment V—Environment Approved by/Date: Assess worstcredible Severity.e. P E I II D 2 C 2 T III C 3 Surround flange with sealed annular stainless steel catchment housing. is also typically in decreasing order of cost and schedule impact (i. NOTE: THESE COUNTERMEASURES MUST BE IN PLACE PRIOR TO OPERATION. Typical PHA. 3. determine if they introduce new hazards or intolerably diminish system performance. Standard Stop. Identify target(s).3 for a pressurized chemical intermediate transfer system is presented in figure 38.e. Describe hazard source mechanism. If added hazards or degraded performance are unacceptable. with gravity runoff conduit led to DetectoBox™ containing detector/alarm device and chemical neutralizer (S/W). releasing pressurized UnFo 3 chemical intermediate from containment system. If Risk is not acceptable. / Description SrdA. with no added countermeasures. worstcredible outcome.1) might be performed to determine a countermeasure of adequate effectiveness and minimized program impact.. Provide personal protective equipment (Schedule 4) and training for response/cleanup crew (S/P). and Probability for that outcome.
Signature blocks for the analyst and reviewers/approvers. 317 . and risk priority code. In any case. an organization may create their own worksheet customized for their operation. j. probability level. great care should be given in designing the form to encourage effective usage. probability level. product environment). downtime.3 (1) (2) (3) (4) Identifies and provides a log of primary system hazards and their corresponding risks.2. Provides information to management to make decisions to allocate resources and prioritize activities to bring risk within acceptable limits. or operation covered in the analysis. b. Risk assessment after countermeasures are considered. i. Provides a logically based evaluation of a system’s weak points early enough to allow design mitigation of risk rather than a procedural or inspection level approach. Brief description of the portion of the system. equipment. c. 35).5 Advantages A PHA provides the following advantages:3. 3. e. fig. different target types may be listed. and risk priority code (zone from risk matrix. Provides a relatively quick review and delineation of the most significant risks associated with a specific system. including severity level. System number. f. Hazard targets (check boxes for personnel. Hazard (description and identification number).Note that the worksheet from this example contains the following information: a. h. Although helpful. Declaration of the probability interval. subsystem. Description of countermeasure (with codes for various types). The PHA worksheet used in the example is typical. d. including severity level. g. Risk assessment before countermeasures are considered. For example. a PHA is not a structured approach that assists the analyst in identifying hazards or threats. However. Date of analysis.
1983. D. Hammer. and Moriaty.Principles and Practices.” John Wiley & Sons. as described in reference 3.: “Handbook of System and Product Safety. blast walls.3. Henley. etc. Raheja. 1980. etc.3 (1) A PHA fails to assess risks of combined hazards or coexisting system failure modes. B. mechanical.6 Limitations A PHA possesses the following limitations:3. New York. “Assurance Technology and Application . “System Safety. Examples of barriers include barricades. the assessment will be flawed.2. If inappropriate or insufficient targets or operational phases are chosen. E. fences.. Roland. the effort will become too large and costly to implement.: “Probabilistic Risk Assessment.G. chemical. Inc.J. Inc. While on the other hand.. Opportunities for undesired energy flow between the sources and targets are assessed. procedures.: “The Loss Rate Concept in Safety Engineering.. Barriers are countermeasures against hazards caused by flows from these energy sources to targets. The energy flow/barrier method is a useful supplement to the PHA discussed in section 3. Energy sources such as electrical. is a system safety analysis tool. are identified. 3. lead shields.3.. (2) 3.L.4.. 1991..1 Description The energy flow/barrier analysis.” PrenticeHall. 318 . 1991.2. Malasky. when viewed singularly.” McGrawHill.: “System Safety Engineering and Management.E. H. safety glasses.” December 1982.: “System Safety: Technology and Application. used to identify hazards and determine the effectiveness of countermeasures employed or suggested to mitigate the risk induced by these hazards.2. Inc. R. This tool is also known as energy trace/barrier analysis.” May 1990.W. Browning. if too many targets or operational phases are chosen.. is acceptable. Therefore a false conclusion may be made that overall system risk is acceptable simply because each hazard element risk identified.” Garland STPM Press. H. “System Safety Engineering and Management. 1982.” The Institute of Electrical and Electronic Engineers. radiation. 1972. Inc.3 Energy Flow/Barrier Analysis 3. W. Army Regulation 389516. gloves. S.7 Bibliography Air Force Systems Command Design Handbook DH 16.” Marcel Dekker. and Kumamoto.
? Remember every energy source could have multiple flow paths and targets. h. f. e. b. This analysis can also be applied in failure investigations. Are existing barriers sufficient countermeasures to mitigate the risk to the targets? (3) Consider the following strategies extracted from reference 3. Modify the rate of release of energy. g. Limit quantity and/or level of energy. Prevent the release of energy. Separate energy from target in time and/or space.3.4 Example Examples of strategies to manage harmful energy flows are presented in table 33. Is the energy flow unwanted or detrimental to a target? c. productivity. as described in reference 3. This assessment can be applied during phase C but may also be applied in phase E or phase B. Strengthen potential target. facilities.3. Control improper energy input. 3. such as personnel. d. b. mission or test objectives. Eliminate energy concentrations.2 Application An energy flow/barrier analysis can be beneficially applied whenever assessments are needed to assure an identified target is being safeguarded against a potential energy source that can impose harm. What are the potential targets.4. i. Isolate by imposing a barrier. 319 . are presented below: (1) (2) Examine the system and identify all energy sources. environment. etc. c. equipment.4 to control harmful energy flow: a.3.3. Modify target contact surface or basic structure. 3.3 Procedures Procedures to perform an energy flow/barrier analysis. Consider the following for each energy flow path: a. Examine each potential energy flow path in the system.
Tullahoma. Tennessee.. Strategy Eliminate energy concentrations Examples · Control/limit floor loading · Disconnect/remove energy source from system · Remove combustibles from welding site · Change to nonflammable solvent · Store heavy loads on ground floor · Lower dam height · Reduce system design voltage/operating pressure · Use small(er) electrical capacitors/pressure accumulators · Reduce/ control vehicle speed · Monitor/limit radiation exposure · Substitute less energetic chemicals · Heavywall pipe or vessels · Interlocks · Tagout – lockouts · Doublewalled tankers · Wheel chocks · Flow restrictors in discharge lines · Resistors in discharge circuits · Fuses/circuit interrupters · Evacuate explosive test areas · Impose explosives quantitydistance rules · Install traffic signals · Use yellow nopassing lines on highways · Control hazardous operations remotely · Guard rails · Toe boards · Hard hats · Face shields · Machine tool guards · Dikes · Grounded appliance frames/housing · Safety goggles · Cushioned dashboard · Fluted stacks · Padded rocket motor test cell interior · Whipple plate meteorite shielding · Breakaway highway sign supports · Foamed runways · Select superior material · Substitute forged part for cast part · “Harden” control room bunker · Crossbrace transmission line tower · Use coded. 320 . keyed electrical connectors · Use matchthreaded piping connectors · Use back flow preventors Limit quantity and/or level of energy Prevent release of energy Modify rate of release of energy Separate energy from target in time and/or space Isolate by imposing a barrier Modify target contact surface or basic structure Strengthen potential target Control improper energy input *Examples provided courtesy of Sverdrup Technology Inc.Table 33. Examples* of strategies to manage harmful energy flow.
3. etc. DOD 76–451: SSDC–29.: “Energy Damage and the Ten Countermeasure Strategies.1 Description An FMEA.G.2). 1980. its use is often guided by topdown “screening” (as described in sec.5 Advantages The energy flow/barrier analysis provides a systematic thought process to identify hazards associated with energy sources and determines if current or planned barriers are adequate countermeasures to protect exposed targets. 3. Due to design and performance requirements.3.” Haddon.3. may aid this thought process.: “MORT Safety Assurance Systems.4.4 Failure Modes and Effects (and Criticality) Analysis 3.4 (1) Even after a thorough analysis. an energy flow/barrier analysis fails to assess risks of combined hazards or coexisting system failure modes..D. Johnson.. This tool also fails to identify certain classes of hazards. and consequent reductions in risk can be evaluated. W. and as precursors to a fault tree analysis (FTA) (sec. 3. all hazards might not be discovered.7 Bibliography Department of Energy. it is not always obvious that energy may be reduced or redirected. tabular technique that explores the ways or modes in which each system element can fail and assesses the consequences of each of these failures. is a forward logic (bottomup).4 3.3.2). FMEA and FMECA are useful tools for cost and benefit studies (sec. to implement effective risk mitigation and countermeasure. 3.” Marcel Dekker. 2. for each failure mode. potential versus kinetic mechanical energy. Countermeasures can be defined.” Human Factors Journal.5. An FMECA also addresses the criticality or risk of individual failures.. M. electrical. (2) (3) 3. In its practical application.5). W. chemical. August 1973. “Barrier Analysis. 321 . asphyxia in oxygendeficient confined spaces. A reexamination of energy as heat.3) to establish the limit of analytical resolution.4. 3. as described in reference 3.. 3. Inc.6 Limitations The energy flow/barrier analysis possesses the following limitations. e.3.g. Like the PHA (sec. Jr.
subsystem.2).2 Application An FMEA can be used to call attention to system vulnerability to failures of individual components. Steps prior to performing the FMEA or FMECA: (1) Define the scope and boundaries of the system to be assessed. Document the results. These failure mode analyses are typically performed during phase C. or part levels. these analyses can be done with or shortly after the PHA (sec. components and parts lists. to be protected. Develop a numerical coding system that corresponds to the system breakdown (fig. facilities. as described in reference 3. assemblies. Gather pertinent information relating to the system.5.3. productivity. During this phase.) If the answer is yes. components. 39). component. equipment. and associated risk (FMECA) of component failures have been appropriately addressed. Identify and observe the levels of acceptable risk that have been predetermined and approved by management. drawings. These limits may be the risk matrix boundaries defined in a risk assessment matrix (sec. By answering the following questions posed in reference 3. Procedures for preparing an FMEA are the same with steps 8 through 12 omitted. Establish the mission phases to be considered in the analysis. (2) (3) Steps in performing the FMEA or FMECA (see flowchart presented in fig. or it may serve to correct incomplete criteria being used for the FMEA. Partition and categorize the system into advantageous and reasonable elements to be analyzed. and piece parts.4. These resources are targets. This tool can be used to provide reassurance that the cause. such as requirement specifications. the scope and resources required to perform a classical FMEA can be reduced.5. descriptions. 310): (4) (5) Identify resources of value. 3. (This has the additional benefit of providing visibility of nonvalue added systems. without loss of benefit: a. (6) 322 . 3. mission or test objectives. environment.4. etc. subassemblies.3 Procedures Procedures for preparing and performing FMECA’s.2). etc. such as personnel. the analysis is complete. effect. 3. These system elements include subsystems. are presented below. Will a system failure render an unacceptable or unwanted loss? If the answer is no. ask the following question for each subsystem identified in step 2 above. These tools are applicable within systems or at the systemsubsystem interfaces and can be applied at the system. Singlepoint failures can be identified. The vulnerable points identified in the analyses can aid management in making decisions to allocate resources in order to reduce vulnerability.
If the answer is yes for any subassembly. Will an assembly failure render an unacceptable or unwanted loss? If the answer for each assembly is no.Part No. ask the following question for each assembly of those subsystems identified in step 2 above. Document the results.Assembly No. For example. .5 b. ask the following question for each component of those subassemblies identified in step 2 above: 323 . the analysis is complete.Component No. Will a subassembly failure render an unacceptable or unwanted loss? If the answer for each subassembly is no. . Will a subsystem failure render an unacceptable or unwanted loss? If the answer for each subsystem is no. If the answer is yes for any subsystem. . Document the results.System Subsystem 1 Subsystem 2 Subsystem 3 Subsystem 4 Assembly 1 Assembly 2 Assembly 3 Subassembly 1 Subassembly 2 Subassembly 3 Component 1 Component 2 Component 3 Part 1 Part 2 Part 3 Typical Coding System: Subsystem No. ask the following question for each component of those assemblies identified in step 2 above. 3. Document the results.Subassembly No. Example of system breakdown and numerical coding. d. the analysis is complete. c. code number for part 2 above is 0301030102 Figure 39. the analysis is complete. . If the answer is yes for any assembly.
Figure 310. 324 . FMECA process flowchart.
effect. subassembly. What are the effects (or consequences) of each failure mode on each target? (8) (9) Assess worstcredible case (not the worstconceivable case) severity and probability for each failure mode. according to reference 3. Will a component failure render an unacceptable or unwanted loss? If the answer for each component is no. Typically. Then reevaluate the risk with the new countermeasure installed. The contents and formats of these worksheets vary among organizations. is presented in figure 311. Countermeasures may or may not be listed. If countermeasures are developed. component. ask and answer the following questions: a. the information found in an FMECA worksheet. subsystem. Will a part failure render an unacceptable or unwanted loss? (7) For each element (system.5. then develop countermeasures to mitigate the risk. A worksheet for an FMEA would be similar with the risk assessment information removed. determine if they introduce new hazards or intolerable or diminished system performance. Document your completed analysis on an FMEA or FMECA worksheet. If the answer is yes for any component.1).e. If the risk is unacceptable. If added hazards or degraded performance are unacceptable. A sample FMEA worksheet is attached in appendix F. Categorize each identified risk as acceptable or unacceptable. and target combination. What are the failure modes (ways to fail) for this element? b. Document the results. f. or part) for which failure would render an unacceptable or unwanted loss. (10) (11) (12) (13) (14) 325 . ask the following question for each part of those components as identified in step 2 above. Assess risk of each failure mode using a risk assessment matrix (sec. 3. develop new countermeasures and reevaluate the risk. assembly. the analysis is complete. The matrix should be consistent with the established probability interval and force or fleet size for this assessment.
and subassembly elements.5 is illustrated in figure 312. 326 . An FMECA worksheet for the control subsystem is presented in figure 312(c).3.5 3.: __________________ SYSTEM NO. No.FMEA NO: ________________________ PROJECT NO. A schematic of the system is presented in figure 312(a). Figure 312(b) illustrates the breakdown and coding of the system into subsystem. The system being assessed is an automated mountain climbing rig. Item/ Functional Ident. EFFECTS. AND CRITICALITY ANALYSIS WORKSHEET SHEET ___ OF ____ DATE _________________________ PREPARED BY: __________________ REVIEWED BY: __________________ APPROVED BY:__________________ TARGET CODE: P – PERSONNEL / E – EQUIPMENT / T – DOWNTIME / R – PRODUCTS / D –DATA / V –ENVIRONMENT Id. assembly.4 Example An example FMECA3.4.: ______________________ PROB. Failure Mode Failure Cause Failure Event T a r g e t Risk Assessment S e v P r o b RC i o s d k e Action Required/ Comments Figure 311. INTERVAL: ___________________ FAILURE MODES. Typical FMECA worksheet.: ______________________ SUBSYSTEM NO.
Figure 312. Subsystem Hoist (A) Assembly Motor (A01) Subassembly Windings (A01a) Inboard bearing (A01b) Outboard bearing (A01c) Rotor (A01d) Stator (A01e) Frame (A01f) Mounting plate (A01g) Wiring terminals (A01h) Drum (A02) External power source (B) Cage (C) Frame (C01) Lifting Lug (C02) Cable (D01) Hook (D02) Pulleys (D03) Switch (E01) START (E01a) Cabling (D) Controls (E) FULL UP LIMIT (E01b) Wiring (E01c) (b) System breakdown and coding.(a) System. Example of an FMECA—Continued 327 .
(3) 328 .: ______________________ SUBSYSTEM NO. Cage does not stop. INTERVAL: 30 years ___________ FAILURE MODES. EFFECTS. (c) Worksheet. Cage stays in safe position. No response a switch. Varmint invasion. 3. incorporate “fail safe” features into the system design. Mechanical failure or corrosion. No. Start switch Full up switch Wiring Failure Mode Failure Cause Failure Event T a r g e t P E T P P E T Risk Assessment S e v IV IV IV II IV IV IV P r o b C C C A D D D RC i o s d k e 3 3 3 1 3 3 3 Action Required/ Comments E–01–a E–01–b E–02 Switch fails closed.: Mountain Climbing Rig_____ PROB. Item/ Functional Ident. An FMECA provides risk assessments of these failures. Results can be used to optimize reliability. Example of an FMECA—Continued. disconnected.5 Advantages Performing FMEA’s and FMECA’s provide the following advantages: 3. Figure 312. Mechanical failure or corrosion. Stop switch fails closed. Start switch fails open. obtain satisfactory operation using equipment of “low reliability.5 (1) (2) Provides a mechanism to be exhaustively thorough in identifying potential singlepoint failures and their consequences.4. Switch fails open.” and guide in component and manufacturer selection. Cut.: Controls SYSTEM NO. Provide further analysis at the piecepart level for highrisk hazards identified in a PHA. AND CRITICALITY ANALYSIS WORKSHEET SHEET ___ OF ____ DATE _________________________ PREPARED BY: __________________ REVIEWED BY: __________________ APPROVED BY:__________________ TARGET CODE: P – PERSONNEL / E – EQUIPMENT / T – DOWNTIME / R – PRODUCTS / D – DATA / V – ENVIRONMENT Id. optimize designs.FMEA NO: ________________________ PROJECT NO. Cage will not move.
However. 329 . D. MIL–STD–1629A. especially when performed at the partscount level within large.: “Loss Prevention in the Process Industries. London.S. and Goldberg. Raheja.4. and Criticality Analysis.6 3. H.” November 1980. DC. Inc. Chester.: “Fault Tree Handbook. This analysis is heavily dependent upon the ability and expertise of the analyst for finding all necessary modes. Although systematic. ” John Wiley & Sons.” 2 vols.: “Assurance Technologies – Principles and Practices. Effects. 1980. Government Printing Office. F. Washington. (1) (2) (3) (4) (5) (6) (7) (8) Costly in manhour resources. and guidelines/check sheets are available for assistance.” Weber Systems Inc. “Procedures for Performing a Failure Modes.4. F. If too much emphasis is placed on identifying and eliminating singlepoint failures. Probabilities or the consequences of system failures induced by coexisting. U. B. An FMECA can be a very thorough analysis suitable for prioritizing resources to higher risk areas if it can be performed early enough in the design phase.G. often too late to guide this prioritization. W.F. 1989. N. Roberts.: “System Safety Engineering and Management. D. Failure probability data are often difficult to obtain for an FMECA. Provides a mechanism for more thorough analysis than an FTA.3.: “System Safety – Including DOD Standards.” McGrawHill. no check methodology exists to evaluate the degree of completeness of the analysis. Butterworths. Roland.6 Limitations The following limitations are imposed when performing FMEA’s and FMECA’s..(4) (5) Identify hazards caused by failures to be added to the PHA that may have been previously overlooked in the PHA. Human error and hostile environments frequently are overlooked.H. complex systems. then focus on more severe system threats may be overlooked. OH. the level of design maturity required for an FMECA is not generally achieved until late in the design phase.7 Bibliography Layton. Vesely.P.. 1991. 3. Haasl....F.” NUREG–0492.E. Lees.E. 1982. since every failure mode of each component of the system is assessed. multipleelement faults or failures within the system are not addressed or evaluated.. D. and Moriarty.
8. Blocks may depict the events or system element functions within a system. parallel. subassembly.3. component. Table 34 . Type branch Series Block diagram representation System reliability # RS = RA * RB A B Parallel A RS = 1 – (1–RA)(1–RB) B Seriesparallel A C RS = (1 – (1–RA)(1–RB)) * (1 – (1–RC)(1–RD)) B Parallelseries A B D D RS = 1 – (1 – (RA * RB)) * (1 – (RC * RD)) C #Assumes all components function independently of each other. 330 . A diagram may contain a combination of series and parallel branches. Simple RBD’s are constructed of series. or part.5.1 Description A reliability block diagram (RBD) is a backwards (topdown) symbolic logic model generated in the success domain. These blocks are connected in series if all elements must operate successfully for the system to operate successfully. A system element can be a subsystem. Each block represents an event or system element function.7 and 3. The system operates if an uninterrupted path exists between the input and output.5 Reliability Block Diagram 3. these blocks typically depict system element functions only. These blocks are connected in parallel if only one element needs to operate successfully for the system to operate successfully. Simple RBD construction. The descriptions of RDB’s contained herein was obtained from references 3. Each RBD has an input and an output and flows left to right from the input to the output. or combinations of series and parallel elements (table 34). However.
H are not success paths. A D G B E C F H Figure 313. These systems must be modeled with a complex RBD. and n = number of system elements (which are assumed to function independently). Each element of a block diagram is assumed to function (operate successfully) or fail independently of each other element.8. Notice in this example. Reliability is the probability of successful operation during a defined time interval. and their derivations are found in reference 3. (1–Rn )] i n where RS = system reliability. thus this is not a true series or parallel arrangement. . if element E fails. Ri = system element reliability. then paths B. . Such an RBD is presented in figure 313. G and B. . . i n Parallel Systems: RS = 1– ∏ (1–Ri ) = [1–(1–R1 )*(1–R2 ) (1–R3 )* . 331 . The relationships between element reliability and system reliability for series and parallel systems are presented below.RBD’s illustrate system reliability. Typical complex RBD. . E. Series Systems: RS = ∏ Ri = R1* R2 * R3 * . E. Some complex systems cannot be modeled with true series and parallel branches. Not all systems can be modeled with simple RBD’s. Rn. .
Then treat each series branch as an element in a parallel branch and determine the system reliability by using the equations in step 3b.8) to generate a simple RBD are presented below: (1) (2) (3) Divide a system into its elements. .3.2 Application An RBD allows evaluation of various potential design configurations. RiL (low) to RiH (high).6). (1–RnH)]. An RBD may also be used to identify elements and logic as a precursor to performing an FTA (sec. from each individual element’s reliability band. . in the following manner: a. 3. RSL = 1– ∏ (1–RpL ) = [1–(1–R1L )*(1–R2L ) (1–R3L )* . . For parallel systems with n elements that are to function independently.8 Required subsystem and element reliability levels to achieve the desired system reliability can be determined.3. b. 3. . .5. d. . (1–RnL )] i n i n RSH = 1– ∏ (1–RpH) = [1–(1–R1H )*(1–R2H) (1–R3H )* . first determine the reliability for each parallel branch using the equations in step 3b.5. 332 . RSL = ∏ (RiL) = R1L *R2L *R3L * . For seriesparallel systems. RSL (low) to RSH (high). . . Typically. c. first determine the reliability for each series branch using the equations in step 3a. For parallelseries systems. these functions are performed during phase C. Then treat each parallel branch as an element in a series branch and determine the system reliability by using the equations in step 3a. Construct a block diagram using the convention illustrated in table 34. . .3 Procedures The procedures (adapted from reference 3. RnH. A functional diagram of the system is helpful. For series systems with n elements that are to function independently. . . RnL i n i n RSH = ∏ (RiH) = R1H *R2H *R3H * . Calculate system reliability band.
98 0. Subsystem 1 has three components and at least one of the three must function successfully for the subsystem to operate. Continue this process until one of the above four basic arrangements remains. Example RBD.980.80 0.99 0. 3. Note that the components for subsystem 1 are in a parallel branch with the components of subsystem 2. Then.0.72 A 0.980. then determine the system reliability.84 0.60 0.98 0.e.0.84 B 0.62 0.99 0. 0.97 0.960.96 0.99 An RBD for the system is presented in figure 314.80 . and determine the reliability for the new simplest branches.5.62 C 0. For systems that are composed of the four above arrangements.70 . determine the reliability for the simplest branches.72 0.4 Example A system has two subsystems designated 1 and 2. Subsystem 2 has three components that all need to function successfully for the subsystem to operate.70 0. Also.60 . 333 .97 0. Subsystem 2 is designed to be a backup for subsystem 1.0. note that the components for subsystem 1 form a series branch and the components for subsystem 2 form a parallel branch. The estimated reliability band for each individual component over the system’s estimated 10yr life interval is presented below: Subsystem 1 1 1 2 2 2 Component A B C D E F Reliability Bands Low High 0.99 E F D Figure 314. treat these as branches within the remaining block diagram.
seriesparallel. If the element reliability values have different confidence bands.5.983 Subsystem 2: R2L = (0.976 R1H = 1 – (1–0.6 Limitations An RBD possesses the following limitations: (1) (2) Systems must be broken down into elements where reliability estimates can be obtained. (low band value) (high band value) (low band value) (high band value) (low band value) (high band value) 3.84)(1–0. However.C alculations for subsystem and system reliabilities are presented below: Subsystem 1: R1L = 1 – (1–0.999 Therefore.998 to 0.96)(0.5 Advantages An RBD provides the following advantages: (1) (2) (3) (4) Allows early assessment of design concepts when design changes can be readily and economically incorporated.9. difficult to validate.3.3.98)(0.60) = 0. the reliability band for the system is 0. Not all systems can be modeled with combinations of series. These complex systems can be modeled with a complex RBD. 3.983)(1–0.922 R2H = (0.7 Blocks representing elements in an RBD can be arranged in a manner that represent how these elements function in the system. parallel.951) = 0.998 RSH = 1 – (10.62) = 0.99) = 0.8 System element reliability estimates might not be readily available for all elements.3.73.3. Such a breakdown for a large system can be a significant effort. determining system reliability for such a system is more difficult than for a simple RBD.97)(0. and not be accepted by others in the decision making process.999.80)(1–0.3.976)(1–0. or parallelseries branches.8 (3) 334 .7 Since RBD’s are easy to visualize.99)(0.5.951 System: RSL = 1 – (1–0.70)(1–0. Some reliability estimates may be very subjective.98) = 0.72)(1–0. they can be generated prior to performing an FTA and transformed into a fault tree by the method discussed in section 3. this can lead to significant problems.8 Tends to be easier for an analyst to visualize than other logic models. such as a fault tree.922) = 0.
This can be expressed as: PF = F/(S+F) . The FTA includes generating a fault tree (symbolic logic model). cause the TOP event to occur. therefore R + PF = S/(S+F) + F/(S+F) = 1 and PF = 1–R. A minimal cut is a least group of initiators that will. potentially high severity events). A. Since reliability for a given event is defined as the number of successes per number of attempts. 3. 3. propagating failure probabilities to determining the TOP event failure probability.3. A path set is a group of fault tree initiators that. to ensure that an ensemble of countermeasures adequately suppresses the probability of mishaps.” SpringerVerb. where F = number of failures and S = number of successes. and determining cut sets and path sets. as described in reference 3..: “System Preliminary Evaluation & Prediction in Engineering.2 Application FTA’s are particularly useful for high energy systems (i. This model traces the failure pathways from a predetermined. M. called the TOP event.5. is a topdown symbolic logic model generated in the failure domain. and Godran.e. An FTA can be carried out either quantitatively or subjectively.9. if none of them occurs. if they all occur. An FTA is a powerful diagnostic tool for analysis of complex systems and is used as an aid for design improvement. of a system to the failures or faults (fault tree initiators) that could act as causal agents.6. then the relationship between the probability of failure and reliability can be expressed as follows: R = S/(S+F).6. entering failure probabilities for each fault tree initiator.1 Description A fault tree analysis (FTA).7 Bibliography Pages. Previous identification of the undesirable event also includes a recognition of its severity. 335 . A cut set is any group of initiators that will. 1986. undesirable condition or event.6 Fault Tree Analysis 3. will guarantee the TOP event cannot occur. cause the TOP event to occur. if they all occur. The probability of failure for a given event is defined as the number of failures per number of attempts.
All path sets are determined.6. The benefits for each of the four phases are summarized in table 35. as described in reference 3. deployment of resources to mitigate risk of highrisk TOP events can be optimized. FTA’s are applicable both to hardware and nonhardware systems and allow probabilistic assessment of system risk as well as prioritization of the effort based upon root cause evaluation. A minimal cut set is a least group of initiators that. (3) identifying and assessing cut sets. 3. (2) probability determination. and resources prioritized by the perceived highest probability elements. FTA’s are typically performed in phase C but may also be performed in phase D.3. and (4) identifying path sets. cause the TOP event to occur. identify qualitative common cause vulnerability. if none of them occurs. All cut sets and minimal cuts sets are determined. The analyst does not have to perform all four phases. FTA’s can be used to identify cut sets and initiators with relatively high failure probabilities.3 Probability determination Identifying and assessing cut sets 3. A path set is a group of fault tree initiators that. The subjective nature of risk assessment is relegated to the lowest level (root causes of effects) in this study rather than at the top level. Action items resulting from the investigation may be numerically coded to the fault tree elements they address. FTA procedures. Sensitivity studies can be performed allowing assessment of the sensitivity of the TOP event to basic initiator probabilities. if they all occur. for performing an FTA are presented below. and the TOP event are identified. Table 35. and assess quantitative common cause probability. intermediate events. These procedures are divided into the four phases: (1) fault tree generation.6. if they all occur.2 3.3. will cause the TOP event to occur.6. A cut set is any group of initiators that will. A symbolic logic model illustrating fault propagation to the TOP event is produced. 3. quantitative.3.1 Procedures Fault tree generation Benefits All basic events (initiators). Section 3. Cut sets also enable analyzing structural. will guarantee the TOP event cannot occur.This type of analysis is sometimes useful in mishap investigations to determine cause or to rank potential causes. Probabilities are identified for each initiator and propagated to intermediate events and the TOP event.4 Identifying path sets 336 .3 Procedures The procedures. and item significance of the tree.3. Analysis of a cut set can help evaluate the probability of the TOP event.6. Therefore. but can progress through the phases until the specific analysis objectives are met.6.9.
as described in reference 3. The exact solution for OR gate propagation is presented in figure 319.9) derives estimates from the consensus of experts. and (4) basic event. Sources for these failure probabilities may be found from manufacturer’s data. In many cases it is best to stay with comparative probabilities rather than the “absolute” values. on each cited probability number. The probability of failure of independent events through an OR (inclusive) gate is the union of their respective individual probabilities.2 Probability Determination If a fault tree is to be used as a quantitative tool.3. However. 7. the probability of failure must be determined for each basic event or initiator.4. defined in table 36. confidence or error bands.9. Normalizing data to a standard.6. most fault trees can be constructed with the following four symbols: (1) TOP or Intermediate event. are presented in table 37.2 through 3.3. including the gates infrequently used. The procedures.6. The log average method is useful when the failure probability cannot be estimated but credible upper and lower boundaries can be estimated. Propagation of failure probabilities for two independent inputs through an AND and OR (inclusive) is conceptually illustrated in figure 318. Sections 3. Delphi estimates. explicitly declared meaningless value is a useful technique here. As shown in figures 317. step 6. Although many event and gate symbols exist. (3) AND gate. 3. Once probabilities are estimated for all basic events or initiators.6. Also. are required to determine the significance of any quantitatively driven conclusion. and the log average method. A method for detecting common causes is described in section 3. simulation or testing. step 8. Propagation of confidence and error bands is performed simply by propagation of minimum and maximum values within the tree.3. 337 .1 Fault Tree Generation Fault trees are constructed with various event and gate logic symbols. Failure probabilities can also be determined from a PDA as discussed in section 3. they are propagated through logic gates to the intermediate events and finally the TOP event. This technique is described in reference 3. The Delphi technique (sec. event. for small trees hand analysis may be practical. MILstandards. historical evidence (of the same or similar systems).10. (2) inclusive OR gate.14. the use of this exact solution is seldom warranted. Probabilities must be used with caution to avoid the loss of credibility of the analysis. The probability of failure of independent inputs through an AND gate is the intersection of their respective individual probabilities. The relationship between reliability and failure probability propagation of two and three inputs through OR (inclusive) and AND gates is illustrated in figure 317. or phenomenon that will simultaneously induce two or more elements of the fault tree to occur.3. A common cause is a condition.3 are included for completeness and to provide insight as to the mathematics that takes place in the commercially available fault tree programs. A frequent error in fault tree construction is neglecting to identify common causes. The propagation equations for the logic gates.3. A source for human error probabilities is found in reference 3. to construct a fault tree are illustrated in figure 315.11 and is illustrated in figure 316.3.3. All large trees are typically analyzed using these programs.6. industry consensus standards.6. the propagation solution through an OR gate is simplified by the rare event approximation assumption.
338 . Fault tree construction symbols. Basic event* External event An initiating fault or failure that is not developed further. Symbol Name Event (TOP or intermediate)* Inclusive OR gate* Description TOP Event – This is the conceivable. Mutually exclusive OR gate M AND gate* An output occurs if one or more inputs exist. or restrictions to other events. However. An output occurs if one or more inputs exist. Any single input is necessary and sufficient to cause the output event to occur. An event that under normal conditions is expected to occur. These events determine the resolution limit of the analysis. Any single input is necessary and sufficient to cause the output event to single occur. or information. All inputs are necessary and sufficient to cause the output event to occur. but only one input exists.Table 36. They are also called leaves or initiators. Exclusive OR gate An output occurs if one. Undeveloped event An event not further developed due to a lack of need. * Most fault trees can be constructed with these four logic symbols. An output occurs if all inputs exist. restraints. Intermediate Event – This event describes a system condition produced by preceding events. undesired event to which failure paths of lower level events lead. INHIBIT gate An output occurs if a single input event occurs in presence of an enabling condition. Priority AND gate An output occurs if all inputs exist and occur in a predetermined sequence. All inputs are necessary and sufficient to cause the output event to occur. Any single input is necessary and sufficient to cause the output event to occur. all other inputs are then precluded. Conditioning Event These symbols are used to affix conditions. resources.
2.055 Figure 316. Identify undesirable TOP event. 6. 5. Fault tree construction process.5 times the lower bound and 0.07 0. Identify secondlevel contributors. • The antilogarithm of the average of the logarithms of the upper and lower bounds is less than the upper bound and greater than the lower bound by the same factor.” “Initiator. Link secondlevel contributors to TOP by logic gates. 0.0316228 –1. 3.01 + 0. = 0. • Average the logarithms of the upper and lower bounds.” or “Basic”) indicates limit of analytical resolution. Basic Event…(“Leaf. it is geometrically midway between the limits of estimation. Repeat / continue… Figure 315.55 times the upper bound.5 PU Upper Probability Bound 10–1 Note that. 339 .e. • Estimate upper and lower credible bounds of probability for the phenomenon in question.01 0. 5. for the example shown.03 0. the arithmetic average would be… 0. Link contributors to TOP by logic gates.1 2 i. Log average method of probability estimation.1. Identify firstlevel contributors.04 0..1 0. Thus.05 0.02 0.0316 + PL Lower Log PL+ Log P U = Antilog(–2) + (–1) Probability Log Average = Antilog 2 2 Bound –2 10 = 10 = 0. 4.
independent element failures produces elements must fail to produce system failure.For 2 Inputs OR Gate AND Gate Either of two. system failure. Relationship between reliability and failure probability propagation. independent.2 PF ≅ PA + PB with error ≤ 11% PF = P APB [Intersection / “Rare Event Approximation” PF = P A + P B + PC – P APB – PA PC – P BPC + P A PBP C For 3 Inputs Omit for approximation PF = P APB PC Figure 317. Both of two.B ≤ 0. R + PF ≡ 1 R T = RA RB R T = R A + R B – RAR B PF = 1 – R T P F = 1 – (R ARB) P F = 1 – [(1 – P A )(1 – P B)] PF = 1 – R T P F = 1 – (R A + R B – R AR B ) P F = 1 – [(1 – P A) + (1 – P B) – (1 – P [Union / ] A )(1 – P B)] ] PF = PA + PB – P A PB …for P A. AND Gate… TOP PT = Pe PT = P 1P 2 Intersection / OR Gate… TOP PT Union / Pe PT P1 + P 2 1 P1 2 P2 1&2 are INDEPENDENT events 1 P1 2 P2 P T = P 1 P2 P T = P 1 + P 2 – P 1 P2 Usually negligible… Figure 318. Failure probability propagation through OR and AND gates. 340 .
B. not probability) of initiators that will produce the TOP event. Proceed through the matrix construction by (1) substituting the letters for each AND gate with letters for the gates and numbers of the initiators that input into that gate (arrange these letters and numbers horizontally in the matrix rows). 341 . One method of determining and analyzing cut sets is presented below. and assess common cause probability. if all the initiators in the group occur. Analysis of a cut set can help evaluate the probability of the TOP event. These procedures for determining cut sets are described in reference 3. create a matrix using the letters and numbers. identify common cause vulnerability. Cut sets also enable analyzing structural. From the top of the tree downwards. A minimal cut set is the smallest number (in terms of elements. quantitative.Failure TOP PT = ? Success TOP PT = Failure TOP PT = Pe (1 – P e ) 1 P1 2 P2 3 P3 1 2 3 1 P1 2 P2 3 P3 P1 = (1 – P 1 ) P3 = (1 – P 3 ) P2 = (1 – P 2 ) = P e = 1 – P (1 – P e ) The ip operator ( cofunction of pi ( provides an exact solution ) is the ∏ ). 3.3 Identifying and Assessing Cut Sets A cut set is any group of initiators that will produce the TOP event. and (2) substituting the letters for each OR gate with letters for the gates and numbers of the initiators that input into that gate (arrange these letters and numbers vertically in the matrix columns). It P P T for propagating probabilities through the OR gate. Determining Cut Sets: (1) (2) (3) Consider only the basic events or initiators (discarding intermediate events and the TOP event). starting from the top of the tree. Its use is rarely justifiable. if all the initiators in the group occur. T = 1 – [(1 – P 1) (1 – P 2 ) (1 – P 3 ) … (1 – Pn )] Figure 319.9 and are based on the MOCUS computer algorithm attributed to J.6. The letter for the gate directly beneath the TOP event will be the first entry in the matrix.3. Assign a unique letter to each gate and a unique number to each initiator. and item significance of the tree. Exact solution of OR gate failure probability propagation. Fussell.
Symbol Name Inclusive OR gate‡ P1 P2 Venn Diagram Propagation Expressions PT = P1 + P2 – (P1 * P2) PT = P1 + P2# Exclusive OR gate P1 P2 PT = P1 + P2 –2 (P1 * P2) PT = P1 + P2# M Mutually exclusive OR gate P1 P2 PT = P1 + P2 AND gate ‡ and (priority AND gate) P1 P2 PT = P1 * P2 ‡Most fault trees can be constructed with these two logic gates.. PK (the probability that the cut set will induce the TOP event) is mathematically the same as the propagation through an AND gate. The remaining rows define the minimal cut sets of the fault tree. through visual inspection. expressed as: PK = P1 * P2 * P3* P4. 342 . Next. #Simplified expression for rare event approximation assumption. Visually inspect the final matrix and eliminate any row that contains all elements of a lesser row. if all the initiators in the group occur. Assessing Cut Sets: (6) Since a cut set is any group of initiators that will produce the TOP event. (4) (5) When all the gate’s letters have been replaced.* Pn. the cut set probability. a final matrix is produced with only numbers of initiators. eliminate redundant elements within rows and rows that repeat other rows. Probability propagation expressions for logic gates.Table 37. Each row of this matrix represents a Booleanindicated cut set..
Analyze the probability of each common cause at its individual probability level of both occurring. e. called a singleton. b. calculated as above. then the TOP event is vulnerable to the common cause the subscript represents.4 Identifying Path Sets A path set is a group of fault tree initiators that. A cut set with many elements indicates low vulnerability. (8) (9) (10) Assess the quantitative Importance.9). This indicates that the probability number. A cut set with a single initiator.6. of each cut set. Ie. h for human operator. may be significantly in error. That is. etc. 3. and inducing all terms within the affected cut set. and PT = the probability of the TOP event occurring. Check to see if any minimal cut sets have elements with identical subscripts.(7) Determine common cause vulnerability by uniquely assigning letter subscripts for common causes to each numbered initiator (such as m for moisture. A cut set with few elements indicates high vulnerability. Note that some initiators may have more than one subscript. v for vibration. 343 . K.. ensures the TOP event cannot occur. (11) Assess the quantitative importance. while others will have none. d. determine the numerical probability that this cut set induced the TOP event. if none of them occurs. The procedures to determine path sets are as follows: (1) Exchange all AND gates for OR gates and all OR gates for AND gates on the fault tree. and IKe = importance of the minimal cut sets containing initiator e. if it has occurred: Ie = where ∑ IK e e Ne Ne = number of minimal cut sets containing initiator e. Path sets can be used to transform a fault tree into a reliability diagram (sec. of each individual initiator. since the same event (the socalled common cause) could act to precipitate each event. c. Numerous cut sets indicates high vulnerability. That is. 3. I K. IK = PK/PT where PK = the probability that the cut set will occur (see step 6 above). assuming it has occurred. If that is the case.e. indicates a potential singlepoint failure.). Assess the structural significance of the cut sets to provide qualitative ranking of contributions to system failure. q for heat. determine the numerical probability that initiator e contributed to the TOP event.3. Assuming all other things are equal then: a. they no longer represent statistically independent events. i.
344 .1/yr ALARM CLOCKS FAIL 3. PLUGIN CLOCK FAILS 1. both low and high probability values that define a probability band for each initiator could be propagated through the fault tree to determine a probability band for the TOP event. In this example the TOP event is the “artificial wakeup fails.34x10 4 Negligible NOCTURNAL DEAFNESS MAIN. only a nominal probability value for each fault tree initiator is propagated through the fault tree to the TOP event. Example fault tree. In this example for brevity.82x102 BACKUP (WINDUP) CLOCK FAILS 1.” The system being examined consists of alarm clocks used to awaken someone.6. ARTIFICIAL WAKEUP FAILS 3. 0.6.6.1 Examples Fault Tree Construction and Probability Propagation An example of a fault tree with probabilities propagated to the TOP event is presented in figure 320. Faults/Year………2/1 Assume 260 Operations/Year 4x10 4 1/10 2x104 1/20 Figure 320.4. for a thorough analysis.83x10 2 POWER OUTAGE FAULTY INNARDS 3x10 4 3/1 MECHANICAL FAULT 8x10 8 FORGET TO SET FAULTY MECHANISM FORGET TO SET FORGET TO WIND 1x102 8x10 3 2/1 4x104 1/10 8x103 2/1 1x102 3/1 ELECTRICAL FAULT 3x104 1/15 HOUR HAND FALLS OFF HOUR HAND JAMS WORKS KEY: Faults/Operation……8x10 3 Rate.34x10 4 Approx.4 3. However. 3.(2) Construct a matrix in the same manner as for cut sets (sec. 3. Each row of the final matrix defines a path set of the original fault tree.3. steps 1–5).3.
represent it by the same number at each appearance. 2 & 4. is an OR gate. replace it vertically.”) Do not repeat letters.3. 1 D 2 D 3 C is an AND gate.4. replace it horizontally. starting with the TOP “A” gate… A B D 1 D C D B is an OR gate. 1 2 2 D 3 1 4 D (top row).2 Cut Sets An example of how to determine Booleanindicated minimal cut sets for a fault tree is presented in figure 321. 1 2 2 3 1 4 Minimal Cut Set rows are least groups of initiators which will induce TOP. If a basic appears more than once. A is an AND gate. Figure 321. the initial matrix entry. These BooleanIndicated 1 2 Cut Sets… 2 2 3 1 4 …reduce to 2 4 3 these Minimal Cut Sets. its inputs. TOP event gate is A. its inputs. 2 & 3. • Assign numbers to basic initiators. D (2nd row). Each requires a new row. its inputs. 345 . B & D . is an OR gate. Example of determining cut sets. A B D 1 C 2 4 2 3 • Construct a matrix. replace it horizontally.6. its inputs. (TOP gate is “A. replace it vertically. TOP PROCEDURE: • Assign letters to gates. Each requires a new row. Replace as before. 1 & C.
4.6.9 (1) (2) (3) (4) (5) Enables assessment of probabilities of combined faults/failures within a complex system. TOP A B C 1 D F 6 This Fault Tree has… 2 E 3 5 G Path Sets are least groups of initiators which.3. if they cannot occur. 6 1 1 1 1 2 3 4 5 6 3 4 “Barring” terms (n) denotes consideration of their success properties.5 Advantages An FTA provides the following advantages:3.3 Path Sets An example of how to determine path sets for a fault tree is presented in figure 322. 1 1 1 3 2 3 4 4 5 3 4 4 1 …these Minimal Cut Sets …and these Path Sets.6. 3. This tool can be used to reconfigure a system to reduce vulnerability. Path sets can be used in trade studies to compare reduced failure probabilities with increases in cost to implement countermeasures. System vulnerability and lowpayoff countermeasures are identified. guarantee against TOP occurring. Example of determining path sets. Figure 322. 346 . Singlepoint and common cause failures can be identified and assessed. thereby guiding deployment of resources for improved control of risk.
U.. Haasl. A fault tree is not accurate unless all significant contributors of faults or failures are anticipated.. DOE 7645/22.: “System Safety: Technology and Application.E. Fault trees used for probabilistic assessment of large systems may not fit or run on conventional PCbased software.3. Government Printing Office. several or many FTA’s may be needed for a particular system.S. Specific (noncomparative) estimates of failure probabilities are typically difficult to find.. Vesely.6 Limitations A FTA possesses the following limitations: (1) (2) (3) Address only one undesirable condition or event that must be foreseen by the analyst.A.” Garland Press.H. G. B.. Malsaky. to achieve agreement on. J. F.. E.: “Reliability and Faulttree Analysis Guide. 347 .: “Fault Tree Handbook.. The generation of an accurate probabilistic assessment may require significant time and resources.S. Roberts. and Singh. and Kumamoto. Caution must be taken not to “over work” determining probabilities or evaluating the system.R.” NUREG–0492.” John Wiley & Sons. Henley. B.: “Nuclear Systems Reliability Engineering and Risk Assessment. Roland. 1972.6.B. The failure rate of each initiator must be constant and predictable.. 1981. Fussell. Washington. A fault tree is flawed if common causes have not been identified.: “Probabilistic Risk Assessment. Dillon. and Moriarty. P.F.: “System Safety Engineering and Management.W. (4) (5) (6) (7) (8) 3. Inc. limit the size of the tree.: “Engineering Reliability – New Techniques and Applications. H..” The Institute of Electrical and Electronic Engineers. W: “Handbook of System and Product Safety.F.. New York. 1983. and Goldberg. Events or conditions under the same logic gate must be independent of each other. and to successfully use to drive conclusions. DC.6.” Department of Energy No. H. C. W. S. and Burdick. N.e. 1977. Hammer.” Prentice Hall.J. 1982. Comparative analyses are typically as valuable with better receptions from the program and design teams. 1982. Events or conditions at any level of the tree must be independent and immediate contributors to the next level event or condition.7 Bibliography Crosetti. i. Thus.” John Wiley & Sons.” Society for Industrial and Applied Mathematics. 1991.E. D.
. This technique is a powerful diagnostic tool for analysis of complex systems and is used as an aid for design improvement and is applicable both to hardware and nonhardware systems. will guarantee the TOP event occurs. where S = number of successes and F = number of failures. determining success probabilities for each tree initiator.6) which is generated in the failure domain with failure pathways from undesirable events. This can be expressed as: Ps = S/(S+F) .2 Application Particularly useful for high energy systems (i. prevent the TOP event from occurring. Wynholds. propagating each initiator probability to determining the TOP event probability and determining cut sets and path sets.” Proceedings from Annual Reliability and Maintainability Symposium.1 Description A success tree analysis (STA) is a backwards (topdown) symbolic logic model generated in the success domain. R. and Koren. A minimal cut set is a least group of initiators that will.G. J. prevent the TOP event from occurring. 1990. Los Altos. 3. then R = PS. desirable condition or event (TOP event) of a system to the successes (success tree initiators) that could act as causal agents. Potterfield.7.” Proceedings of the Second International System Safety Conference. The probability of success for a given event is defined as the number of successes per number of attempts. W. and Bass.e.. 3. This model traces the success pathways from a predetermined.M.: “A New Approach to the Analysis of Reliability Block Diagrams.. New Mexico. a cut set is any group of initiators that will. 1975. The STA includes generating a success tree (symbolic logic model). J. potentially high severity events). The subjective nature of the probability assessment is relegated to the lowest level (root causes of effects) in this study rather than at top level. L.7 Success Tree Analysis 3. SAIC. if they all occur. to ensure that an ensemble of countermeasures adequately leads to a successful top event. if all of them occur. A path set is a group of success tree initiators that.7. 348 . In the success domain. An STA is the compliment of an FTA (sec. if they all occur.. This technique also allows probabilistic assessment of causal benefits as well as prioritization of effort based upon root cause evaluation. Riley.: “Fault Tree Graphics – Application to System Safety.. Sensitivity studies can be performed allowing assessment of the sensitivity of study results to subjective numbers. 3.William S. Since reliability for a given event is also defined as the number of successes per number of attempts.
The commercial computer programs are similar. These symbols are defined in table 36. A success tree can be constructed from a fault tree. Determine the probability of success (Ps) for each basic event or initiator. to construct a fault tree also apply to success tree generation and are illustrated in figure 323. 7.7.” “Initiator. historical evidence (of similar systems). and the log average method. 2. as are the cautions for use of probability values. Identify firstlevel contributors.” or “Basic”) indicates limit of analytical resolution.3 Procedures Success trees. Although many event and gate symbols exist. simulation or testing. 1. 4. 3. Use the expressions presented in table 37 to propagate probabilities through logic gates. and (4) basic event. and restating each initiator. 349 . Sources for these success probabilities may be found from manufacturer’s data. Figure 323. 3.The STA is typically applied in phase C but may also be applied in phase D. (2) inclusive OR gate. if a success tree is generated from a fault tree.9. Repeat / continue… 6. and top event as a success opposed to a failure. MIL standards. (3) AND gate. Delphi estimates. most success trees can be constructed with the following four symbols: (1) TOP or intermediate event. Basic Event…(“Leaf. Since a success tree is the logic compliment of a fault tree. Transform a success tree from a fault tree by simply changing all AND gates to OR gates and OR gates to AND gates. the logic of the success tree needs to be valid if the logic of a fault tree is to be valid. The procedures. propagate these probabilities through logic gates to the intermediate events and finally the TOP event. Identify desirable TOP event. as described in reference 3. The Delphi technique (sec. Link contributors to TOP by logic gates. 5. industry consensus standards. Link secondlevel contributors to TOP by logic gates. Remember that the probability of success equals reliability (R) and may be determined from (PF) as shown in the following equation: PS = 1 – PF . A success tree can be used to verify the logic of a fault tree. like fault trees. are constructed with various event and gate logic symbols. Once probabilities are estimated for all basic events or initiators. Success tree construction process. intermediate event.9) derives estimates from the consensus of experts. Identify secondlevel contributors.
Example success tree.7.9996 Main.9998 0.9997 Hour Hand Does Not Jam Works Hour Hand Stays On 0.9996 0. Plugin Clock Works Backup Clock (Windup) Performs Properly 0. Example The compliment success tree for the fault tree presented in section 3.9923 0.9997 0. 350 .3 and 3.3.4 is presented in figure 3 Artificial Wakeup Succeeds 0.4 24. 3. as presented in sections 3.9996 Alarm Clock Works Possess Keen Hearing Properly 1.7.9885 Mechanical Component Success Electrical Components Perform Properly 1.9923 0.9805 Functioning Clock Components Uninterrupted Power Remembered to Set Alarm Unflawed Mechanism Remembered to Set Backup Alarm Remembered to Wind Clock 0.3.9885 0.000 0.000 0.Generate cut sets and path sets in the same manner as for fault trees.6. respectively.9806 0.7.4.9996 Figure 324.
3. Thus.7 Bibliography Henley. E.: “Probabilistic Risk Assessment.. Events or conditions under the same logic gate must be independent of each other. 351 . A success tree is not accurate unless all significant contributors to system successes are anticipated. This technique explores system responses to an initiating “challenge” and enables assessment of the probability of an unfavorable or favorable outcome.8 Event Tree Analysis 3..7.8. Events or conditions at any level of the tree must be independent and immediate contributors to the next level event or condition.9 (1) (2) Assesses probability of favorable outcome of system operation. The system challenge may be a failure or fault. as described in references 3.7. is a forward (bottomup) symbolic logic modeling technique generated in both the success and failure domain. Compliments the FTA by providing a method to verify the logic of the fault tree.9 (1) (2) (3) (4) (5) (6) (7) Address only one desirable condition or event that must be foreseen by the analyst. 3.5 Advantages An STA provides the following advantages:3.1 Description An event tree analysis (ETA). Inc. The generation of an accurate probabilistic assessment may require significant time and resources. or a normal system operating command. 1991. and Kumamoto.” The Institute of Electrical and Electronic Engineers. New York.7.6 Limitations An STA possesses the following limitations:3. 3. H.6 and 3. several or many STA’s may be needed for a particular system.J. 3.12. an undesirable event. The probability of success (reliability) of each initiator must be constant and predictable. Success trees used for probabilistic assessment of large systems may not fit/run on conventional PCbased software. Caution must be taken not to overdo the number generation portion.
Trace each path to eventual success or failure. N D/A SUCCESS n FAILURE SUCCESS FAILURE O/O OPERATION/ OUTCOME OPERATION/ OUTCOME 1 INITIATION 3 DECISION/ ACTION C DECISION/ ACTION A DECISION/ ACTION B SUCCESS FAILURE SUCCESS FAILURE OPERATION/ OUTCOME 2 Figure 325. An ETA may also be performed to compliment an FMEA. and engineered safety features. are required to determine the significance of any quantitatively driven conclusion.2 Application The ETA is particularly useful in analyzing commandstart or commandstop protective devices. Portray all credible system operating permutations. A Bernoulli model event tree uses binary branching to illustrate that the system either succeeds or fails at each system logic branching node. and other nonhardware systems. explicitly declared meaningless value is a useful technique here. . success or failure probabilities used must be used with caution to avoid the loss of credibility of the analysis.A generic event tree portrays all plausible system operating alternate paths from the initiating event. The technique is useful in evaluating operating procedures. This technique is typically performed in phase C or E but may also be performed in phase D. Event tree (generic case). A generic event tree is illustrated in figure 325. An ETA may be used in conjunction with an FTA to provide a technique sensitivity assessment. The ETA is also useful in evaluating effect and benefit of subtiered or redundant design countermeasures for design trades and assessment. A Bernoulli model event tree is illustrated in figure 326. However. on each cited probability number. Also. emergency response systems. In many cases it is best to stay with comparative probabilities rather than the “absolute” values. A decision tree is a specialized event tree with unity probability for the system outcome. 3. Normalizing data to a standard. management decision options. confidence or error bands.8. 352 .
3. Determine the probability of each potential path by multiplying the individual probabilities of events making up the path. trace all plausible system operating permutations to a success or failure termination.Reduce tree to simplified representation of system behavior. b. 3.) SUCCESS FAILURE SUCCESS FAILURE FAILURE Figure 326. (3) (4) (5) (6) Determine the probability of the initiating event by applying a fault tree (sec. For a decision tree. “What happens when the system is challenged by the initiation event?” By convention. Use binary branching. Determine the probability of the system success by adding the probabilities for all paths terminating in success. For the general event tree. Determine the paths (alternate logic sequences) by answering the question.3 Procedures The procedures. a. 353 .6) or other analysis. (Unity probability may be assumed. Event tree (Bernoulli model). Lead unrecoverable failures and undefeatable successes directly to final outcomes. use binary branching to show the system pathways. SUCCESS FAILURE FAILURE SUCCESS FAILURE SUCCESS INITIATION FAILURE SUCCESS A fault tree or other analysis may be necessary to determine probability of the initiating event or condition. (1) (2) Identify the initiating challenge to the system being examined. trace successful paths upwards and failure paths downwards. Determine the probability of the system failure by adding the probabilities for all paths terminating in failure. Simplify the tree by pruning unnecessary alternate branches of nonrecoverable failures or undefeatable successes. as described in reference 3. assume the probability of the initiating event is one. for performing an ETA are presented below.8.12. For the Bernoulli model event tree.
Functions simultaneously in the failure of success domain. 354 . Failure propagation paths of a system can be identified and traced. Comparative analyses are typically as valuable. difficult to achieve agreement on. and to successfully use to drive conclusions. and lowpayoff countermeasures are identified and assessed.8.4 Example An example of an ETA is presented in figure 327. This can be a “quick and dirty” comparative technique and provides very clear visibility of ineffective countermeasures. thereby guiding deployment of resources for improved control of risk and optimized utilization of limited resources. Note that in this example the probability of the challenging initiator is assumed to be one and the tree has been pruned to its simplest form by using engineering logic. For example. Although multiple pathways to system failure may be disclosed. multiple ETA’s may be needed for a particular system. Thus. the levels of loss associated with particular pathways may not be distinguishable without additional analyses.3. its path leads directly to a final failure outcome with no alternate paths. The example includes the system and scenario being assessed and the resulting event tree. End events need not be anticipated. coexisting system faults and failures. Potential singlepoint failures. noncomparative success or failure probability estimates are typically difficult to find. Specific. with better reception from the program and design teams.6 Limitations An ETA possesses the following limitations: (1) (2) (3) (4) (5) Address only one initiating challenge.8. The initiating challenge is not disclosed by the analysis. areas of system vulnerability. its path also leads to a final success outcome with no alternate paths. 3.8.5 Advantages An ETA provides the following advantages: (1) (2) (3) (4) Enables the assessment of multiple. but must be foreseen by the analyst. Operating pathways must be foreseen by the analyst. since failure of the float switch is a nonrecoverable failure. In a similar manner since successful operation of the pump is an undefeatable success. (5) 3.
P. FAILURE Bailing Fails (PB) SUCCESS 355 . SIMPLIFYING ASSUMPTIONS: • Power is available full time. • Consider operator error as included within the bailing function. Example ETA.0) Pump Fails (P P) [PP – P P PS ] PKP P + PKP PP S] [PB PP – P BP PP S – Klaxon Fails (PK ) PB PK PP + PB PK PP PS ] Float Switch Fails (P S) [PKP P – P K PP PS ] [PS] PSUCCESS = 1 – P S – P K PP + P K PP PS – P BP P + P B PP PS + P B PK PP – P BP KP PP S PFAILURE = P S + P KP P – P K PP PS + P BP P – P BP PP S – P B PK PP + P BP KP PP S P SUCCESS + P FAILURE = 1 Figure 327.PUMP KLAXON P S K B BACKGROUND/PROBLEM — A subgrade compartment containing important control equipment is protected against flooding by the system shown. Either pumping or bailing will dewater the compartment effectively. • Treat only the four system components S. Assume flooding has commenced. powering pump P from an uninterruptible power supply. Pump Succeeds (1 – P P) [1 – P S – P P + P PP S] Float Switch Succeeds (1 – P S) EVENT TREE… Klaxon Succeeds (1 – PK) [PP – P P PS – Bailing Succeeds (1 – P B ) [PP – P PP S – P K PP + P K PP PS – PB PP + P B PP PS + P BP KP P – PB PK PP PS ] Water Rises (1. K. and B. Rising flood waters close float switch S . alerting operators to perform manual bailing. • Develop a fault tree for the TOP event Failure to Dewater. and analyze responses available to the dewatering system… • Develop an event tree representing system responses. B . • Develop a reliability block diagram for the system. A klaxon K is also sounded. should the pump fail. B.
3.” New York.” 2 vols. Lees. “Loss Prevention in the Process Industries. RBD’s (sec.9 Fault Tree.1 Fault Tree to RBD Transformation An RBD represents system component functions that. 3.4.9. Reliability Block Diagram. Fault trees offer the analyst comprehensive qualitative or quantitative analysis. 1980. Event trees allow the analyst to assess a system in both the success and failure domains. reliability diagrams are generated in the success domain. A fault tree can be transformed into a reliability diagram as illustrated in figure 328.3.13 and presented below. Fault trees are generated in the failure domain. RBD. then an event tree can be constructed. and Kumamoto.9. 3.9.2 RBD and Fault TreetoEvent Tree Transformation An event tree represents path sets in the success branches of the tree and all the cut sets in the failure branches of the tree. if these functions prevail. and Event Tree Transformations 3. and event trees are generated in the success and failure domains. London..8. respectively. produces success in place of a TOP fault event. 3. For large complex fault trees.” 1985.J. 3. These techniques. F. RBD’s offer the analyst a simplistic method to represent system logic. 3. 1981.9. “Reliability Engineering and Risk Assessment.P. 3. 356 .6. described in reference 3.3.1 Description Fault trees (sec. if the path sets and cut sets of a system are known for a certain challenge to a system (TOP event of a fault tree). cut sets and path sets are obtainable using the MOCUS algorithm described in sections 3. E.6. and event tree. Therefore.9.7) are all symbolic logic models. transform any one of the above models into the other two by translating equivalent logic from the success to failure or failure to success domain. 3.7 Bibliography Battelle Columbus Division. and event trees (sec.5). Cut sets and path sets may be obtained from a reliability diagram as shown in figure 329. H. or event tree to either of the other two logic models are presented in the following sections. 3.6). Henley.3 and 3.. This technique is typically performed in phase C but may also be performed in phase B. “Guidelines for Hazard Evaluation Procedures.3 Procedures The procedures for transforming a fault tree.3.2 Application These techniques are applicable by the analyst that wishes to exploit the benefits of the fault tree.3.. Butterworths. RBD.13 3.
4 1 Minimal Cut Sets 1 2 3 4/5/6 2 3 5 6 ! Path Sets 1/2/3/4 1/2/3/5 1/2/3/6 Figure 329. Deriving cut and path sets from an RBD. Fault tree to RBD transformation. 357 .! The OR gate in this position indicates a series string of component functions. 1 2 3 4&5&6 Nastiness 4 5 6 1 2 3 4 5 6 ! Figure 328. This AND gate indicates a paralleled set of component functions in the series string.
produce TOP event fault in place of the success to which the reliability block path lead.9. The series nodes of an RBD denote an OR gate beneath the TOP event of a fault tree. reverse the process illustrated in figure 330. RBD to event tree transformation. a reliability diagram can be transformed into a fault tree as shown in figure 331.To transform an RBD into an event tree.3. 3. Failure Figure 330. Once the RBD is formed. first transform the fault tree into an RBD (sec. 4 1 2 3 5 6 Failure of any one of these series elements makes system failure irreversible. respectively. 2 6 1 Success 4 3 5 ! All of these parallel elements must fail to produce system failure. a fault tree can be formed as illustrated in figure 331.9. The parallel paths in an RBD denote the AND gate for redundant component functions in a fault tree. To transform an event tree into an RBD. 358 .4 Event Tree to RBD and Fault Tree Transformation An event tree represents path sets in the success branches of the tree and all the cut sets in the failure branches of the tree. and presented in figure 333(a) and (b).9. All three of the models represent equivalent logic of the system.1).3 RBD to Fault Tree Transformation A fault tree represents system functions which. if they fail. an event tree can be transformed into a fault tree by inspection as shown in figure 332. proceed as shown in figure 330. 3.3. 3. To transform a fault tree into an event tree.3. 3. Also.4 Example An RBD and fault tree are transformed from the example event tree presented in figure 327. Therefore.9.
4 5 6 ! 7 These parallel paths indicate AND gates for redundant component functions.2 1 3 These series nodes indicate an OR gate beneath TOP. Event tree to fault tree transformation. 359 . RBD to fault tree transformation. Figure 332. 15 7 16 3 17 8 18 1 19 9 20 4 21 10 22 i 23 11 24 5 25 12 26 2 27 13 28 6 29 14 30 SUCCESS FAILURE A 1 SUCCESS FAILURE B 1 SUCCESS FAILURE B 2 SUCCESS FAILURE C SUCCESS FAILURE B 3 SUCCESS FAILURE A 2 SUCCESS FAILURE D FAILURE FAILURE A2 FAILURE A 12 SUCCESS FAILURE A 1 16 7* 3* 1* i 26 12 5* 2 i *Note that not all events represented here are failures. ! 1 2&3 Grief 4&5 Woe 6&7 Evil 2 3 4 5 6 7 Figure 331.
Figure 333. Equivalent logic RBD and fault tree. 360 . S P/K P/B (a) RBD.PUMP P FLOAT SWITCH S KLAXON K BAILING B CUT SETS PATH SETS S/P S/K/B See Figure 327. (b) Fault tree. COMMAND FAILURE FAILURE TO DEWATER RESPONSE FAILURE EXACT SOLUTION: PTOP = PS + PP PK – PP PK PS + PBP P – PBPPP S – P BP KPP + PBP KPP PS RARE EVENT APPROXIMATION: PTOP = PS + PP PK + P PP B CUT SETS PATH SETS S/P S/K/B S P/K P/B S FLOAT SWITCH FAILS OPEN WATER REMOVAL FAILS P PUMP FAILS MANUAL REMOVAL FAILS K KLAXON FAILS B BAILING FAILS See Figure 327.
. based on variations of response available within the system. Each increment has an associated level of assumed or calculated probability. 3. J. and presented below. SAIC.6 Limitations These techniques possess the following limitations: (1) (2) No new information concerning the system is obtained and the models are only as good as the models being transformed. The analyst starts with an initiating event and performs a forward (bottomup) analysis using an event tree (sec. and extensive quantitative or pseudoquantitative analysis can be performed. New Mexico. and discrete. and each consequence has an associated severity and probability. For example. a complex system that may be hard to model as a fault tree might be easily modeled with an RBD.7 Bibliography Gough.S.: “A New Approach to the Analysis of Reliability Block Diagrams.6 and 3. 361 . the RBD can be transformed into a fault tree.9. Los Altos. quantified as to probability. W. however. Note that the cause has an associated probability. The cause may be a fault tree TOP event and is normally.1 Description A causeconsequence analysis is a symbolic logic technique described in references 3.8). This technique provides data similar to that available with an event tree. but not always. 3. it affords two advantages over the event tree—time sequencing of events is better portrayed. The consequence portion of this technique yields a display of potential outcomes representing incremental levels of success or failure.9. A conceptual illustration of how a cause is assessed to understand its consequences is presented in figure 334.M.10 CauseConsequence Analysis 3. staged levels of outcome are analyzed. and Koren. 3..9. 1990. Then. 3. The cut sets and path sets required to perform these transformations for large complex systems may require many manhours or extensive computer resources to determine.3. The cause portion of this technique is a system challenge that may represent either a desired or undesired event or condition.14. J.10.” Proceedings from Annual Reliability and Maintainability Symposium. Riley. This technique explores system responses to an initiating “challenge” and enables assessment of the probabilities of unfavorable outcomes at each of a number of mutually exclusive loss levels.5 Advantages These techniques allow the analyst to overcome weaknesses of one analysis technique by transforming a system model into an equivalent logic model as another analysis technique.
This technique may also be used to compliment an FMEA. and other nonhardware systems. What circumstances allow this event to proceed to subsequent events? b. 3. This technique may be used in conjunction with an FTA to provide a technique sensitivity assessment. that this event will occur.3. 3.10. and engineered safety features.2 Application This technique is typically applied in phase C or E but may also be applied in phase D. as described in references 3. Causeconsequence analyses are useful in evaluating operating procedures.6. CAUSE P0 Figure 334. probability of the Cause. Also. P0. P C3 CONSEQUENCE n S n.3 Procedures The procedures. trace the possible consequences to the system from the initiating event.6 a. Determine the probability. for performing a causeconsequence analysis are presented below.6 and 3.10. Relationship between cause and consequence. it will evaluate the effect/benefit of subtiered/redundant design countermeasures for design trades and assessment. The causeconsequence analysis is particularly useful in analyzing commandstart/commandstop protective devices.2) or assumed. management decision options. (1) (2) (3) Identify the initiating event that challenges the system.CONSEQUENCE 1 S 1. P C1 CONSEQUENCE 2 CONSEQUENCE 3 S3 . What other events may occur under different system operating circumstances? 362 . may be determined by Fault Tree Analysis. emergency response systems.14. Construct the consequence diagram by asking the following questions:3. Next. At various levels the path may branch with two possible outcomes. This probability may be determined from an FTA (sec. 3. P Cn S 2. P C2 ANALYSIS Y N OVERPRESSURE RELIEF FAILS P Cn = Probability of the n th Consequence occurring S n = Severity of the n th Consequence P 0.
Y Event N Branching Operator Output is “Yes” if condition is met and “No” if it is not met. Basic Event An independent initiating event. determine the severity of each consequence descriptor. therefore PY+PN = 1. Probability bands are often useful to provide an understanding of the analyst's confidence in the delineated probabilities. The outputs are mutually exclusive. Note that all paths lead into branching operators or consequence descriptors.14 Symbol Name OR Gate Description Gate opens to produce output when any input exists. Pi .c. representing the lower resolution limit of the analysis. What subsequent event could possibly result as an outcome of this event? (4) Use the symbols presented in table 38 to construct the consequence diagram. Branching operator statement may be written in either the fault or the success domain. What other system elements does this event influence? d. P i and (1–Pi ) are the probabilities for the yes and no paths from the branch operator. Si. by multiplying event probabilities along the path that terminates at that consequence descriptor. Therefore. no outputs. End event/condition to which analysis leads. and is a termination point in the diagram.3. This step is often difficult and subjective due to a scarcity of data. P ci. Table 38. The consequence descriptor has one input. that the event can happen. The branching operator always has one input and two output paths (yes and no). establish the probability. Consequence Descriptor (5) The format of the consequence tree is presented in figure 335. Determine the probability of each consequence descriptor. For each branching operator. Causeconsequence tree construction symbols. Finally. (6) (7) (8) 363 . AND Gate Coexistence of all inputs opens gate and produces an output. respectively. with the severity level stated.
Causeconsequence analysis format. After a delay. 3. Tullahoma. End events need not be anticipated..14 (1) (2) (3) (4) *This The analysis is not limited to a “worstcredible case” consequence for a given failure. because the analysis is exhaustive… (P P ) + P (1 – P ) (1 – P ) + P (1 – P ) P = P P 0 1 0 1 2 0 1 2 0 0 Fault trees or other analyses may be used to establish probabilities for the Initiating Challenge and for Branching Operator Y/N outcomes. example was provided courtesy of Sverdrup Technology. Enable assessment of multiple. Inc. The probability of failure is finite for both the drum thermostat and the overheat cutoff.1. 3. Employees frequent the room and can initiate an emergency response alarm in the event of fire. Combustibles are often present in the copying room near the machine. 364 . The drum is also equipped with an automatic overheat safety cutoff to prevent damage to the copier. Tennessee. more realistic assessment is possible. The room is equipped with an automatic sprinkler system initiated by a heat detector. The drum heater is thermostatically controlled. coexisting system faults and failures. a fire brigade responds to extinguish the blaze. 3.10. Uncontrolled drum temperature can rise high enough to ignite them.4 Example* Problem: A copying machine uses an electrically heated drum to fix dry ink to copy paper. The time order of events is examined. a less conservative. Figure 335. The causeconsequence analysis for the above problem is presented in figure 336.5 Advantages Causeconsequence analyses provide the following advantages:3. Therefore.CONSEQUENCE DESCRIPTOR 1 CONSEQUENCE DESCRIPTOR 2 CONSEQUENCE DESCRIPTOR 3 1 2 P P 0 1 P (1 – P ) (1 – P ) 0 P (1 – P ) P 0 1 2 Y N BRANCHING OPERATOR P 2 P 1 Y N BRANCHING OPERATOR P0 (1 – P 1 ) INITIATING CHALLENGE Note that.10.
(6) 3. Therefore. thereby guiding deployment of resources for improved control of risk and optimized utilization of limited resources. mutually exclusive levels of loss outcome.10.5M P 0P 1P 2 (1 – P 3 ) P3 Y N EMERGENCY RESPONSE FAILS WATER/FIRE/SMOKE DAMAGE ≈ $50. The initiating challenge is not disclosed by the analysis.Note that.000 MANUFACTURER'S TEST DATA EMPLOYEE DETECTION/ RESPONSE FAILS FIRE RESPONSE BRIGADE FAILS P2 Y N HEAT DETECTOR/ AUTO SPRINKLER FAIL P 0P 1 (1 – P 2) COPIER DAMAGE ≈ $250 Y N P1 NEARBY COMBUSTIBLES IGNITE DRUM OVERHEATS P 0 (1 – P 1) COMBUSTIBLES PRESENT NEARBY IGNITION TEMPERATURE REACHED P0 DRUM THERMOSTAT FAILS CLOSED OVERHEAT CUTOFF FAILS Figure 336. 365 . multiple analyses may be needed for a particular system. Potential singlepoint failures or successes. the scale of partial successes and failures is discernible. (5) Probabilities of unfavorable system operating consequences can be determined for a number of discrete. because the analysis is exhaustive… P0 P1 P2 P3 + P 0P 1P 2 (1 – P 3) + P 0P 1 (1 – P 2) + P 0 (1 – P 1) = P BUILDING LOSS ≈ $6. but must be foreseen by the analyst. Example causeconsequence analysis.14 (1) (2) Address only one initiating challenge. Thus.6 Limitations Causeconsequence analyses possess the following limitations:3. and lowpayoff countermeasures are identified and assessed. areas of system vulnerability.5M 0 P0 P 1P 2 P 3 BUILDING DAMAGE ≈ $1.
according to reference 3.(3) (4) (5) Operating pathways must be foreseen by the analysts.. The third phase consists of separating the digraph models into independent models. J. the analyst determines combinations of systems or combinations of subsystems within a single system for thorough assessment. 366 . part. However. 1991. J. London. the fourth phase consists of an assessment of the minimal cut sets relative to probability of occurrence. First. If this technique is used as part of a PRA.” 2 vols.J.: “On the Adaptation of CauseConsequence Analysis to U.10. without first creating the model in the success domain. “Risk Assessment and Risk Management for the Chemical Process Industry. Butterworths. 1980.15).) and is best applied in phase B. 3. Lees. 3.S Nuclear Power Systems Reliability and Risk Assessment.2 Application This technique. G. component.R. can be used independently or as an element of a PRA (sec. Knoxville. it should be noted that models can be directly created in the failure domain. then it is performed after the identification of failure propagation paths by ETA but before FTA’s are begun.15 This technique is applied to evaluate the failure propagation paths involving several systems and their support systems.P. 3. is a technique using matrix representation of symbolic logic models to analyze functional system interactions.15. The establishment of probabilities is often difficult and controversial. then determining the singleton and doubleton minimal cutsets of each failure propagation path. System Reliability and Risk Assessment.. Burdic. H. F.11. 3. 3. and Fussell. Finally. Inc.8).” 1985.1 Description Directed graph (digraph) matrix analysis.R. and Cramer. Logic models are first generated in the success domain. Determining the severity on consequences may be subjective and difficult for the analyst to defend.11. This technique consists of four phases.. Greenberg.11 Directed Graphic (Digraph) Matrix Analysis 3. The second phase consists of constructing a digraph model in the success domain. Tennessee.” JBF Associates. 1983. then converted into the failure domain.: “Loss Prevention in the Process Industries. “Guidelines for Hazard Evaluation Procedures. as described in reference 3.B.7 Bibliography Battelle Columbus Division. etc. or within a single system involving several system elements (subsystem.” Van Nostrand Reinhold. then converting this model to a digraph model in the failure domain for each failure propagation path. This phase is parallel to determining failure propagation paths using an ETA (sec.15. 3.
Assume unity probability for all systems required to work in the failure propagation path.2). Form a failure domain model by taking the model generated in step 2a and interchange all AND gates with OR gates and all OR gates with AND gates. 3. A comparison between the digraph and fault tree symbols is presented in figure 337. Use event trees (sec. From these checklists develop a list of initiators that are applicable to the systems being studied. b. d. b. a. such as design specifications and packages. Connect upstream elements to a downstream element with an AND gate if the upstream element relies on the successful operation of all the downstream components. The symbols for AND and OR gates for a digraph are different than those used for a fault tree. Create a success domain digraph model for each success path. This failure domain model represents a path for failure propagation. (1) Identify the associated group of systems (or associated system elements of a single system) to be thoroughly evaluated.3. c. Form an adjacency matrix that represents the digraph. This simplifying assumption leaves only failure propagation paths that are combinations of systems that must fail for a serious threat to be posed. c. 3.15 for performing a digraph matrix analysis. and prior safety or reliability studies. topdown approach to construct a toplevel digraph.3 Procedures Presented below is a summary of the detailed procedures found in reference 3. identify every credible initiator to an undesirable event and prepare an event tree that illustrates each specific failure propagation path.8) to identify failure propagation paths.11. An outline of the steps involved in producing the digraphs is presented below. Continue expanding the elements of new digraphs until the desired resolution level of the analysis is reached. then expand each element into its own digraph. 367 . however they represent the same logic as the fault tree symbols. Acquire pertinent information concerning the collection of systems to be assessed. For a complete analysis. Develop event trees for each initiating challenge to the system. The matrix is constructed by the process illustrated in figure 338. Use a backward. a. Study checklists of potential initiating challenges. sec. (2) Construct a digraph model for each possible failure propagation path. safety assessment reports (such as PHA’s. Connect upstream elements to a downstream element with an OR gate if the upstream element relies on the successful operation of only one of two or more downstream elements. Prepare a list of failure propagation paths from step 1c.
AND Gate A A Digraph C B B C OR Gate C C Fault tree A Represented logic B A B Event C will occur only if both event A and event B occur. Next. Figure 337. 3. This code is described in detail in reference 3. Then determine singleton and doubleton minimal cut sets of the smaller independent digraphs. d. Comparison between digraph and fault tree logic gates. therefore all possible paths between pairs of nodes in the network. Determine minimal singleton and doubleton cut sets from the cut sets determined in step 2d. (4) 368 . e. (3) Subdivide the digraph into independent digraphs if the success domain digraph model becomes too large to determine singleton and doubleton cut sets for the computer platform being used. Assess the singleton and doubleton minimal cut sets.15) in which risk is assessed with the probability of the cut sets occurring and the severity of the consequence of the failure propagation path. Next link all connected elements in the adjacency matrix. The output of this code will show all elements connected by a path and illustrate which elements can be reached from a specific element. use this information to determine singleton and doubleton cut sets.15. Event C will occur only if event A or event B occurs. This is accomplished by processing the adjacency matrix with the reachability code. This assessment can be conducted in a manner similar to that for a conventional PRA (sec.
M) is equal to PS–2 (nonzero or 1 value).4 Example An example digraph matrix analysis. there is a path from node i to node j. The adjacency matrix and adjacency elements are presented in figures 339(c) and (d). which means there is a no straight path between the main power supply and the auxiliary power supply. If the adjacency element (i. In cases with more complex systems.P) = 1. 369 . there is no path from node i to j. then there is a second component that must fail along with component i to cause component j to fail.j) is ≠ 0 or 1. was generated by replacing the OR gate in the success domain model with an AND gate. The failure domain model. 3. element (PS–1.11. for a simple system is illustrated in figure 339. For example. presented in figure 339(b). Note that this model represents the success path for successful operation of the pump. Inspection of the two models suggests that for simple systems the failure domain model can easily be generated without first generating the success model. For example. If matrix element (i. which means there is a straight (uninterrupted) and unconditional path between the motor and pump. respectively. adjacency element (PS–1. The system consists of two redundant power supplies to power a motor that drives a pump.Type Digraph Adjacency Matrix A 0 0 B 1 Direct connection A Element A Element B B0 Element A AND gate Element C Element B Element A A A B0 C0 A A 0 0 0 0 0 0 B 0 A 0 B 0 1 0 C B C 1 OR gate Element C Element B B0 C0 Figure 338.j) = 1.j) = 0. PS–2) = 0. The success domain model of this system is presented in figure 339(a). element (M.15. Construction of digraph adjacency matrix. If element (i. For example. first generating a success domain model may prove to be beneficial. adapted from reference 3. The adjacency matrix illustrates whether there is a direct path from node i to node j.
Processing the adjacency matrix into the reachability matrix yields the paths between all pairs of nodes.11. by connecting linked pairs from the adjacency matrix. then the other corresponding component i or j is a singleton. for complex systems which are modeled with many nodes and logic gates. Since the technique identifies singleton and doubleton minimal cut sets without first determining all minimal cut sets. then P will not operate (even though neither PS–1 or PS–2 are directly adjacent to P).. failure of both the main and auxiliary power supplies will cause the motor not to operate).This symbol represent the second component that must fail. i. The reachability elements are derived from the reachability matrix in the same manner that adjacency elements are derived from the adjacency matrix. If an “*” is entered as a matrix element (i.15 (1) The analysis allows the analyst to examine each failure propagation path through several systems and their support systems in one single model. i. considerable computer resources can be saved over other methods such as the FTA.e. The summary matrix presented in figure 339(g) illustrates which components can lead to failure of the pump.e. The only singleton in this system is the motor. Therefore the reachability matrix illustrates the complete pathways (through linked node pairs) of the graphical model elements illustrated by the adjacency matrix. (2) 370 . the reachability matrix yielded the new information that if both power supplies failed. the pump will not operate.. the reachability matrix illustrates the pairs of nodes that a path exists between. respectively. The methodology to generate the reachability matrix from the adjacency matrix is presented in reference 3. in this example the singletons (single point failures) and doubletons (double point failures) could have easily been identified without performing a digraph matrix analysis. then component i and component j form a doubleton.e. 3. that the reachability elements include all the adjacent elements and the new information that if both PS–1 and PS–2 fail.j) and either i or j is a value of 1. Therefore. in this example. Simply stated. Note. P. However. If a “*” is entered as a matrix element (i.5 Advantages The digraph matrix analysis provides the following advantages: 3. the single failure of the motor will cause the pump not to operate. failure of both the main and auxiliary power supplies will cause the pump not to operate. The only doubleton of this system is the pair of redundant power supplies.. Obviously.15. this technique allows determination of singletons and doubletons which otherwise would not be as readily identified. The reachability matrix and reachability elements are presented in figure 339(e) and (f). given the failure of PS–1. Unlike the FTA with failure propagation paths divided in accordance to arbitrarily defined systems. this approach allows more rigorous subdividing of the independent subgraphs. to cause M to fail to operate (i.j) that corresponds to component i and component j.
Figure 339. P. M Pump. 1 (Adjacent) (e) Reachability matrix. PS1 Main Power Supply. P. PS2 Motor. M. PS2 PS2. PS1 M. M. PS2 (Adjacent) PS1. PS1 PS1 PS2 M P 0 0 0 0 PS2 0 0 0 0 M PS2 PS1 0 0 P 0 0 1 0 PS1. Pump. P Auxiliary Power Supply. PS1 Motor. P. 1 (c) Adjacency matrix. PS1 (Adjacent) PS2. M. PS2 PS2. M. P (b) Failure domain model.Main Power Supply. M Auxiliary Power Supply. Example digraph matrix analysis—Continued 371 . (f) Reachability element. PS2 (a) Success domain model. PS1 PS1 PS2 M P 0 0 0 0 PS2 0 0 0 0 M PS2 PS1 0 0 P PS2 PS1 1 0 (d) Adjacency elements. P. PS1 M. PS1.
1 1 PS1 PS2 M P *  PS1 *  PS2 *  M *  P Singletons: M Doubletons: PS1.11. A.16 and presented below.3.11..6 Limitations Digraph matrix analyses possess the following limitations. 3. 3. For particular types of logic models. E. and Avni. Florida. CRC Press Inc. Boca Raton. “Digraph Analysis Assessment Report. complete treatment may require more computer resources than FTA’s.: “Engineering Risk and Hazard Assessment. 3. 2. Example digraph matrix analysis—Continued. in 1982. October 1991.7 Bibliography Grumman Space Station Division. Figure 339. This technique provides the analyst a procedure to propagate probability data derived from the subjective probability scales defined in MIL–STD–882C. PS2 (g) Summary matrix..1 Description The combinatorial failure probability analysis using subjective information is described in reference 3.3.12.2 372 .” Reston Virginia. Kandel.12 Combinatorial Failure Probability Analysis Using Subjective Information 3.” vol. This technique was developed by the System Effectiveness and Safety Technical Committee (SESTC) of the American Institute of Aeronautics and Astronomics (AIAA).15 (1) (2) Trained analysts and computer codes to perform this technique may be limited.
dimensionless numbers. AIAA/SESTC Threshold Levels 8×10–2 to 1. STA (sec. Likely to occur sometime in life on an item. 3.2 The subjective scale for these arbitrary values is presented in table 39.3. 3.3 Procedures The procedures. Tullahoma. Tennessee.2 (3) (4) 373 . Table 39.3. (1) Arbitrary.16.6). Unlikely but possible to occur in life of an item. Descriptive words and definitions for the level of the scale are also given in this table. 3. Propagate these values in the same manner as quantitative data is combined in classical numerical methods (such as presented in figs.10). as described in reference 3. Inc. Convert the final probability number resulting from propagation (step 3) back into the subjective scale defined in MIL–STD–882C. for a combinatorial failure probability analysis using subjective information are presented below. ETA (sec. 3.5).3. FTA (sec. Correlate the subjective estimate (step 2) with the arbitrary. probable.12.7). *Arbitrarily selected. and causeconsequence analysis (sec.. occasional. 3.00000 to 8×10–5 Probability Level* 3×10–1 3×10–2 3×10–3 3×10–4 3×10–4 Level A B C D E MIL–STD–882C Descriptive Word Frequent Probable Occasional Remote Improbable Definition Likely to occur frequently. So unlikely if can be assumed occurrence may not be experienced. Will occur several times in life of an item. 318 and 319). 3. dimensionless values (step 1).8).3. Table provided courtesy of Sverdrup Technology. and improbable) defined in MIL–STD– 882C. remote. dimensionless “probability values” have been assigned to the probability increments (frequent.2 Application This technique is typically performed in phase C and is applicable when no quantitative failure probability data are available and may be used in conjunction with other analyses such as an RBD (sec.12. Combinatorial failure probability analysis subjective scale.2 Select and consistently apply the same probability exposure interval (operating duration or number of events) for every initiator probability estimate used in the analysis.00000 8×10–3 to 8×10–2 8×10–4to 8×10–3 8×10–5 to 8×10–4 0. (2) Estimate subjective failure probabilities of contributor events or conditions using the scale defined in MIL–STD–882C.
may be poorly received. • Each bearing is served by: • pressurized lubrication oil • a watercooled jacket • a temperature sensing/alarm/shutdown system. Tullahoma.3.4 Example The following example* uses this subjective combinatorial technique in a fault tree problem. Data and results.12. these systems will stop operation of the rotating machine early enough to prevent bearing damage. Inc. This tool should only be used for comparative analysis only. unless used in a comparative fashion.12.6 Limitations This technique should only be used when actual quantitative failure rate data is not available.000 and requires 3 wk of down time. (System sensitivity makes the necessary allowance for machine “rollout” or “coasting.3.5 Advantages This technique allows the analyst to perform a probabilistic assessment based on the exercise of subjective engineering judgment when no quantitative probability estimates are available. The use of actual quantitative data is preferred over this method.2 What is the probability that any one of the six bearings will suffer burnout during the coming decade? The system schematic and fault tree are presented in figure 340(a) and (b). but probabilities can be estimated using the subjective scale of MIL–STD–882C. • In addition. Tennessee. *This example was provided courtesy of Sverdrup Technology. there are sensing/alarm/shutdown systems for: • lube pressure failure • cooling water loss of flow. • If they function properly. Note both the arbitrary subjective probability value and letter representing the relevant probability level from table 39 are presented for each fault tree initiator. Problem/Background: • A large rotating machine has six mainshaft bearings.16 374 . Replacement of a bearing costs $18.3.12.”) • Failure records for the individual system components are not available. respectively. 3. 3..
Bearing Temperature Sensing/Alarm/Shutdown Failure UNRESOLVED UTILITY SERVICE FAILURE 2x106 SHUTDOWN FAILURE 3x10 2 B 10year failure probability estimates are entered at the lowest levels of the tree and propagated upward. Example combinatorial failure probability analysis. Figure 340.000 Replacement Costs • 3Week Interruption of Use (a) System schematic. BEARING BURNOUT 6x108 E Bearing burnout is “ Improbable” for any one of the six bearings over the 10year period. 375 . UNRESOLVED LUBE FAILURE 9x10 7 Lube Pressure Sensing/Alarm/Shutdown Failure UNRESOLVED COOLANT FAILURE 9x107 Coolant Loss Sensing/Alarm/Shutdown Failure LUBE PRESSURE FAILURE C 3x10 3 3x104 SHUTDOWN FAILURE D COOLING WATER FAILURE C 3x10 3 SHUTDOWN FAILURE D 3x104 (b) System fault tree.PROTECTIVE FEATURES T dT /dt • Sensing • Alarm • Shutdown H2O Flow Lube Pressure UTILITY SUBSYSTEMS Bearing Burnout Loss Penalty: • $18.
Assess the various minimal success sets in terms of feasibility.1 Description Failure mode information propagation modeling is a qualitative analysis method described in reference 3. liquid flow. air flow.. and sensor location.e. (1) Divide the system into its principle functional components and assign a number to each component. etc. as described in reference 3. friction. the resolution of this analysis is dependent upon the level (i. This technique involves separating a system into its basic functional components and examines the benefit of measuring precedent failure information that may be transmitted between components of a system. 3.17 and presented below. A minimal success set is a sensor group that encompasses all failure modes.g. Classify the failure mode information constituents by their signal characteristics (e. subassemblies. etc.). or piece parts) to which the system elements are resolved. Identify and record the flow of failure mode information at each physical link that is available externally to each component and transmitted to one or more other components.4).17.3 Procedures The procedures. assemblies. acceleration. subsystems. 3.3.. spring. The following questions should be asked: a. Like the FMEA (sec. This information may be transmitted during the initial outset of a variety of failure modes. The technique provides insight into both the types of information that should be measured to safeguard the system.13. 3. thermal heat transfer. Identify the minimal success sets of the sensor network.13. cost. pressure.13 Failure Mode Information Propagation Modeling 3. sensor type. to perform failure mode information propagation modeling are presented below. gas flow. These links include such items as electrical power.13. thermal.2 Application This technique effectively directs resource deployment to optimally safeguard a system against potential failures by identifying measurement requirements. This technique is best applied in phase C but may also be applied in phase D. rolling element. Identify the physical links (energy flow and shared stress) between the components of the system. and effectiveness. Identify and record the failure modes for each component and assign a letter to each failure mode for each component. Feasibility. Do the sensors currently exist or can they be developed? Can they be obtained in time to satisfy schedule requirements? (2) (3) (4) (5) (6) (7) 376 . These requirements are defined in terms of measured parameter. and location within the system at which sensors might be appropriately positioned.
Are there other preventive maintenance activities more effective than installing a sensor network? Will the sensing network forewarn before the start of system failures or does it just announce system crashes? Will the sensors impede normal system operation? Will they degrade system performance? Will they pose any new hazards to the system? Will the sensor network operate dependably? Will the sensors have adequate sensor redundancy? 3. Tullahoma. Identify the physical links (energy flow and shared stress) between the components of the system. through which power is delivered to the fan. component 1 and electric motor. open winding. Problem: Consider a ventilating fan powered by an electric motor through a belt drive.13.4 Example The following example* uses failure mode information propagation modeling to a sensor network success set for a system. For example. and bearing binding (5–A). Tennessee.b. bearing vibration. the failure modes for the fan include shaft or rotor binding. (Consider motor bearings as integral parts of the motor.3. Identify and record the failure modes for each component and assign a letter to each failure mode. a. for example. *This example was provided courtesy of Sverdrup Technology. Effectiveness. and operating the sensor network less than the cost of the failure that the system is being safeguarded against? c. e. d. Cost.) Assume a constant aerodynamic fan load. Step 2. Solution: (1) Perform steps 1–5 identified in section 3.13. A schematic of the system is presented in figure 341(a). component 4). for the mechanical link between the electric motor and frame.16 377 . is linked to the fan belt by friction. The electric motor. Classify the failure mode information constituents by their signal characteristics. A common frame structure supports both the motor and a bearing. Step 4. For example. frame. These are the electrical motor. fan. Divide the system into its principle functional components and assign a number to each component. maintaining. fan belt. and shorted winding. the electric motor bearing vibration (1–B) and fan bearing vibration (5–B) can be monitored by an accelerometer at test point 4/1 (between frame. has electrical power input. Step 5. fan belt slipping and breaking (2–A/B). Step 3.. b. and is mechanically and thermally linked to the frame. the failure information available includes electric motor bearing vibrations (1– B). and bearing. Inc. c. Catalog the flow of failure mode information at each physical link that is available externally to each component and transmitted to one or more other components. Is the cost of installing.3. For example. Step 1. These steps are explained below and illustrated in figure 341(b). Determine sensor network minimal success sets for the system.
The variables of these transfer functions are represented by probability density functions. 3.14.2 Application A PDA can be used to analyze the reliability of a component during phase C of a program.14. This traditional approach is vulnerable if significant experience and historical data are not available for components similar to that which is being considered. Under these conditions. is a methodology to assess relative component reliability for given failure modes. The propagation of a failure through the system is not addressed. The component is characterized by a pair of transfer functions that represent the load (stress. 3. a point estimate for failure of the component relative to the failure mode under consideration can be determined. construct a matrix of failure mode versus sensor type (with each test point identified).1 Description A PDA.13.14 3. (2) 3. Given that the probability distributions for both the load and capability functions are independent.8 3. Compliments an FMEA. if implemented. 3.17 (1) Allows the analyst to identify measurement requirements. These sets are sensor groups that encompass all failure modes. that.6 Limitations Information propagation modeling possesses the following limitations:3. as described in references 3.13.8 and 3.(2) From the information displayed in figure 341(b). the interference area of these two probability distributions is indicative of failure. Externally induced and common cause faults are not identified or addressed. and capability (strength) the component has to withstand failure in that mode. 3.5 Advantages Information propagation modeling provides the following advantages:3.18.18 Probabilistic Design Analysis 378 . can help safeguard a system by providing warnings at the onset of a failure mode that threatens the system.17 (1) (2) (3) (4) This technique is only applicable if the system is operating in a nearnormal range. The PDA approach offers an alternative to the more traditional approach of using safety factors and margins to ensure component reliability. or burden) that the component is placed under by a given failure mode. Determine the minimum success sets of measurement sensors. The risks of the failure modes are not quantified in terms of criticality and severity. and for the instant of time immediately prior to the initiation of a failure mode. The matrix and minimum success sets for this system are presented in figure 341(c).
Example failure mode information propagation model—Continued 379 . Slipping B. Vibration Mechanical Thermal Spring Liquid Flow 4 Accelerometer 3A 5B 5 •••••••••• Gas Flow (b) Model. Breaking Fan A. Blade Damage 1A/C/D 2A/B 3A 5A 3/00 Power Monitor 1A/C/D 2A/B 3A 5A 1 1B 2A/B 5A 2 Accelerometer 1B 5B Accelerometer 3A 5B 3/5 3A 5A/B 3 Flow Monitor 1A/C/D 2A/B 3A 5A 4/1 1A/C/D Heat Flux Monitor 1A/C/D Electrical Power Friction Air Flow 4/5 Rolling Element Frame 3A 5A/B Bearing A.Elements: • Electric Motor • Fan Belt • Bearing • Fan • Frame (a) System schematic. Open Winding D. Binding B. Shorted Winding 1/2 1A/C/D 2A/B 3A 5A Fan Belt A. Bearing Vibration C. Figure 341. Shaft/Rotor Binding B. Tachometer 1A/C/D 2A/B 3A 5A 0/1 1A/C/D 2A/B 3A 5A Belt Slip Monitor 2A/B Tachometer 1A/C/D 2A/B 3A 5A 1A/C/D 2A/B 3A 2/3 5A Belt Slip Monitor 2A/B Electric Motor A.
4). Establish relationships between the critical parameters and organizational. Identify the random variables that effect the variation in the load to be imposed on the component for the given failure mode. or burden). Confirm the selection of critical design parameters. Figure 341.Te Po st int Minimal Success Sets* 0/1 1/2 2/3 3/00 3/5 4/5 4/1 on ito Ta r ch om ete r Be lt S lip Mo nit Ta or ch om ete r Be lt S lip Mo nit Flo or wM on ito r Ac cel ero me ter Ac cel ero me ter Ac cel ero me ter He Mo at F nit lux or Power Monitor at 0/1 or Tachometer at 1/2 or Tachometer at 2/3 Se nso r Failure Mode 1A 1B 1C 1D 2A 2B 3A 4 5A 5B √ Po we rM √ √ √ √ √ or Flow Monitor at 3/00 √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ Not Applicable √ √ √ √ and Accelerometer at 4/1 *Sensor groups that envelope all failure modes √ √ √ √ √ √ √ (c) Minimal success sets. Example failure mode information propagation model—Continued. for performing a PDA in the context of a total design reliability program for a system are presented below. These requirements should be stated in clear and concise terms that are measurable and verifiable.. adapted from reference 3. Identify variables and parameters that are related to the design. 3. programmatic.8 and 3.3 Procedures The procedures.14. 380 . X3. Incorporate these random variables into a transfer function that represents this load (stress. and established failure criteria. X2. 3. (1) (2) (3) (4) (5) (6) Specify the system design requirements... Load Transfer Function: L = fL (X1.18. .Xn). Ascertain the reliability associated with each critical failure mode with the following probabilistic analysis method: a. Identify the failure modes of the system by using a method such as a FMEA (sec.
or burden) and capability (strength) of the failure mode. including load and capability in this assessment. maintainability. Perform trade studies (sec. d. The expression for reliability is: R = 1 – PF . Gather data to perform the load and capability calculations. The density function can be represented as either a discrete variable distribution using empirical test data. Incorporate these random variables into a transfer function that represent this capability (strength). e. . this interference is indicative the failure mode will occur. therefore a range between two values of the independent random variable of a density function curve is equal to a probability less than or equal to one. gamma. etc. Weibull. Y3. generally one or both of these density functions may be an exponential. Repeat step 8 for each critical component for the system. Repeat the process until the design reliability goals or requirements are met. For independent load and capability functions. Expressions for PF for various distributions are found in most advanced statistics textbooks and handbooks. 2.. (8) (9) (10) (11) 381 . In figure 343. environmental issues. Capability Transfer Function: C = gC(Y1. Reliability is the probability that the failure mode will not occur. (7) Assess the reliability for each critical failure mode.b. is dependent upon the type of load and capability distributions. or as a continuously variable form of the density function. Expressions for PF between combinations of exponential. c. Note: The area under an entire probability density function curve is equal to a probability of one. Probability density functions of both load and capability continuous random variables for a given failure mode are presented in figure 343. Calculate the reliability (R) for the failure mode from the load and capability distributions. Consider each variable of the transfer function as a probability density function (illustrated in figure 342). and Weibull distributions are found in reference 3. log normal. log normal. The expression for PF.. Also illustrated in this figure is the interference of the load and capability density functions. Repeat the above steps to optimize system reliability. gamma. Determine probability distributions of the load (stress. Y2. Determine the relative reliability of the system. then modify the design to increase reliability. However. cost.8. or other distribution. Identify the random variables that affect the variation in the capability of the component to withstand the load imposed for the given failure mode.. both density functions are normal distributions with different means and variances.Ym).1) to reassess and optimize the design for performance.
. X2..3.8 382 .. Y2 .Xn) L C Capability Transfer Function C = g C(Y1..f1(X1) g1(Y1) X1 f 2(X2) g2(Y2) Y1 X2 Y2 fn(X n) gm(Ym) Xn Ym fL(L) gC(C) Load Transfer Function L = fL(X1.Ym) Figure 342... . Load and capability transfer functions. .
3.4 Advantages A PDA provides the following advantages: (1) Allows the analyst a practical method of quantitatively and statistically analyzing the relative reliability of a system during the design phase.8 Determining the density functions of the random variables in the load and capability transfer functions may be difficult.C Figure 343. f L(L) Load f L(L) Capability fc (C) Probability Density Function L Overlap Indicative of Failure C L.18 Historical population data used must be very close to the asplanned design population to be viable.8 3.18 The technique provides a more precise method for determining failure probabilities to support FTA’s than does use of subjective methods.8 3.3.3.14.3. 383 .8 Therefore PDA’s can be used to determine valuable areas of the design and aid in determining the resource allocation during the test and evaluation phase.3. (2) (3) (4) 3.f C(C). Interference between load and capability density functions.3. Extrapolation between populations can render the technique nonviable.5 Limitations A PDA possesses the following limitations: (1) (2) (3) The analyst must have experience in probability and statistical methods to apply this technique.8 The PDA approach offers a more accurate and truly quantitative alternative method to the more traditional approach of using safety factors and margins to ensure component reliability.14. This technique mandates that the analyst address and quantify the uncertainty of design variables and understand its impact on system reliability of the design.
sec.2). 3. During phase 2. and the overall scope of types of hazards to be assessed is defined. 3. a consequence analysis is performed. In performing a PHA. b. are typically performed during phase 1. Then. Perform a PHA (sec. 3.10) are performed. 384 . adopted from reference 3.15 Probabilistic Risk Assessment 3.8).4) and/or causeconsequence analysis (sec. 3. the failure propagation paths and probabilities are established. ETA (sec.15. and (7) and repeats the assessment with the countermeasures incorporated. 3. 3. 3. the analyst: (1) identifies targets. 3.15. or quantitative in nature. and by comparison to other societal risks.6. During phase 1. and from an overall perspective identify potential hazards. (2) defines the scope of the system.2 Application A PRA is performed to identify consequence of failure in terms of potential injury to people. hazards are identified. Severity is established. these methods can be either subjective (as by use of the risk assessment matrix. offer guidance in performing a probabilistic risk assessment:. 3. FMECA (sec. (3) recognizes the acceptable risk limits. FTA (sec. Therefore. but does not address the severity of the failure modes. this technique should be used as one element among other elements of a PRA (sec. (5) assesses the risk for each hazard and target combination in terms of probability and severity.1 Description A PRA is a general term given to methodologies that assess risk.1). or loss of mission requirements. a PRA generally consists of three phases. The PRA is typically performed in phase C. a. 3. (4) identifies hazards.15) to assess the risk associated with the various failure modes. According to reference 3. elements of the system vulnerable to hazards are identified. Although PRA methods are customarily thought of as being quantitative. an assessment of risk is performed in terms of probability and severity. Finally. 3.2).15.(4) This technique identifies the relative probabilities that various failure modes will occur. during phase 3. (1) Phase 1 (activities performed during the preliminary design stage). PHA’s (sec. Define the system to be assessed. the system is defined. damage to equipment or facilities. identify the elements (targets) of the systems that are susceptible to hazards.6).3 Procedures The following procedures.6. (6) and if the risk are unacceptable determines countermeasures to mitigate the risk.
Examine all failure modes and criticality ranking of each system element. the tool could be misapplied or the results misinterpreted. If the analyst is untrained in the various tools required. Assess risk of all failure modes in terms of severity and probability. d. In performing an ETA. the analyst (1) identifies an initiating challenge to the system. 3. (3) Phase 3 (perform a consequence analysis).4) may be performed. b. b. 3. Calibrate the risk of the system being examined by comparing it to other known societal risks. and improves resource distribution for control of loss resources. a. 3. 3. Determine initiators and propagate probability of failure with methods such as FTA (sec.11) may be performed after the ETA is complete and before FTA’s have begun. 385 .5 Limitations A PRA possesses the following limitations: (1) (2) (3) Probabilistic risk assessment requires skilled analysts. c. 3.15.14.15 e. A causeconsequence analysis (sec. An FMECA (sec.4 Advantages Assessing risk avoids unknowingly accepting intolerable and senseless risk.10) may be performed to establish both failure propagation path and probabilities of causes and consequences. 3. a. Establish the severity of the failure modes. Identify failure propagation paths with techniques such as an ETA (sec. allows operating decisions to be made.6). c.8). A digraphmatrix analysis (sec. Probability of failure modes can also be determined with the probabilistic analysis method presented in section 3.3.15.(2) Phase 2 (activities initiated after accomplishing hardware and configuration selections).3. Depending on the size and complexity of the system being assessed. and (2) determines the alternate logic paths from the initiating event. Sufficient information and data may not be available to perform a thorough assessment.1 3. significant manhour and/or computer resources may be needed to complete.
3.: “Failure Modes and Effects Analysis.3 Mohr. January 1993. 3. June 1992.” Lecture presentation. September 1982. Lecture presentation. Inc.: “Combinatorial Failure Probability Analysis Using MIL–STD–882B. Inc. October 1989.7 Gough. P. Sea. June 1990. P.F. Inc. 3.: “Energy Flow/Barrier Analysis.: “Risk Management Guide..” Third edition.: “Fault Tree Analysis. Lecture presentation. SSDC–11.” The Phillips Laboratory and the Engineering Society for Advancing Mobility Land.: “Failure Information Propagation Modeling. and Smith. H. Lecture presentation. 3.D. October 1992.L.” Fourth edition. 1990.12 Clemens. Inc. Glen J.. E. H.L. J... R. I. SAIC. Fault Tree/Reliability Block Diagram/Event Tree. 3. Sverdrup Technology.2 “System Safety Program Requirements.. and Lamberson: “Reliability in Engineering Design.” Proceedings from Annual Reliability and Maintainability Symposium.” Second edition. A. K.18 “Solid Propulsion Reliability Guidebook. June 1993.: “Event Tree Analysis. NY. C.: “Transformations.9 Clemens. Sverdrup Technology.16 Clemens. vol. Lecture Presentation.1 Clemens. DOE 7645/11..L. Sverdrup Technology.” Fourth edition. 1977. 1991.5 Mohr.. P. Sverdrup Technology. and Kumamoto. Sverdrup Technology.4 Clemens. James M.L. 3. Sverdrup Technology. Inc.J.R..” System Safety Development Center.: “Probabilistic Risk Assessment.E.: “CauseConsequence Analysis. Inc. Inc.” Second edition.10 Swain. P.13 Clemens.C.L. P. 3. Sverdrup Technology.. W.. P.REFERENCES 3.” NUREG/CR–1278.11 Briscoe. and Space (SAE).. March 1983.8 Kampur. 3. 3..17 Clemens. NM. Inc. Sacks.. 3.. P. 3. Lecture presentation.. Sverdrup Technology. Lecture presentation.. Lecture presentation.6 Henley. Inc. August 1991. 3.R.15 Alesso. Inc.L. NY.” Sixth edition. Lecture presentation. R. draft.14 Clemens.: “Preliminary Hazard Analysis.: “Initial Guidance on DigraphMatrix Analysis for System Interaction Studies. June 1993. July 1993. P. Air. 386 . November 1992.P. 3. 2. 3.J.L.” Fourth edition.” Second edition..: “Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications. 3. H. Inc. December 1992. 3. May 1993.L.. Sverdrup Technology. Los Altos. and Koren.S. 3.: “A New Approach to the Analysis of Reliability Block Diagrams.” John Wiley & Sons.” MIL–STD–882C. Sverdrup Technology. and Guttman.: “Working with the Risk Assessment Matrix.” The Institute of Electrical and Electronic Engineers.” Lawrence Livermore National Laboratory. Riley.” Third edition. Lecture presentation.
Sensitivity analysis can show which parameters affect a system the most or least. A summary of the advantages and limitations of each tool or methodology discussed in this section is presented in table 41.3 Procedures The procedure for obtaining the sensitivity of a relationship by analytical methods is as follows: (1) (2) (3) Generate an equation for the relationship under consideration. burn rate. when factors such as age affect performance. This can facilitate optimizing a system.2 Application Sensitivity analysis typically should be performed in phase C or D.1. 4. In addition. or adjusting a system for wear or changing conditions. Typical examples of the use of sensitivity analysis are manufacturing formulation and processes (e. Geometric Dimensioning and Tolerancing. is discussed.1 by taking the partial derivatives for each parameter under consideration.1.1 Sensitivity (Parametric) Analysis 4. such as throat area and exit pressure in the thrust equation. or material strength). This method is often preferred for relationships with parameters that are interrelated. An alternate approach to approximate sensitivity is to assume a straightline relationship between two points in the sample space of the relationship. bond strength. 41 . Find the coefficients of influence4.5.1 Description In sensitivity analysis. Sensitivity analysis is especially useful when environmental conditions can change. or when manufacturing tolerances affect performance.1.g.. reducing variability.4. 4. sensitivity functions (or coefficients of influence) are generated by taking partial derivatives with respect to each parameter that affects the outcome of a relationship. DESIGNRELATED ANALYTICAL TOOLS Two designrelated analytical tools (sensitivity analysis and tolerance stackup analysis) that can be useful to systems engineering are discussed in this section. This section is included to give the systems engineer an understanding of the standard methods of dimensioning and tolerancing. Solve the equations for the coefficients of influence to find the sensitivity at given conditions. 4. and to solve the relationship for two conditions represented by two values of the parameters in question. This analysis can be used for nearly any type of relationship. ANSI–Y–14. erosion rate.
Designrelated analytical tools and methodologies. This is facilitated by following the dimensioning and tolerancing system of ANSI–Y–14.Table 41.5 are better suited for production. The pressure ratio is typically found by iteration or by tables.g. some aspects of dimensioning and tolerancing per ANSI–Y–14. changing a throat diameter not only changes motor pressure. but changes the nozzle expansion ratio and exit pressure. care must be used to ensure a small enough range for parameter values to achieve the desired accuracy. In addition. Dimensioning strategy can minimize the cumulative tolerance stackup.030 tolerance may have a smallest unit of measurement greater than a 0. their inclusion in tolerance stackup analysis is superfluous.2 Dimensioning and tolerancing per ANSI–Y–14.g. and function. 42 . in that: (1) If some tolerances are much smaller than others. If the approximation approach above is taken. it becomes increasingly unlikely that all dimensions will be simultaneously worstcase. In addition. when obtaining the sensitivity of thrust to throat diameter. a 0. true positioning allows for a circular tolerance zone. Thus. A moderate amount of training and practice is required to effectively use standard dimensioning and tolerancing. (2) It may be superfluous to combine tolerances from different manufacturing processes. machining and casting. which may be unrealistic.. e. whereas putting tolerances to rectangular coordinates allows a square tolerance zone.5 is fairly standard. Tool or Methodology Sensitivity (parametric) analysis Section 4. fit.3 Worstcase tolerance analysis is conservative in that when many tolerances combine. Standard dimensioning and tolerancing 4. Tolerance stackup analysis 4. a functional part that would comply with true position tolerances may not comply with rectangular tolerances. For example.0005 tolerance. Statistical tolerance analysis usually assumes a normal distribution of dimensions in the tolerance zone. e. care must be exercised when combining tolerances. it may be extremely unlikely.5. Limitations It is often not easy to isolate a variable to obtain a second derivative.1 Advantages The effect of each parameter can be assessed to determine which parameters have the greatest effect on the outcome of a process and which parameters can yield the most benefit from adjustment. Statistical analysis can show that. For example. Consideration of significant digits may be helpful. even if exceeding a requirement is possible.. Worstcase tolerance analysis can simply determine the envelope of possible form.
C * is the propellant gas characteristic exhaust velocity. The sensitivity is found by substituting values for the variability into the partial derivative equation.6).2) where ∂ designates a partial derivative.40 in/s2 . 2.1) where Pc is the chamber pressure.4. C* Pc = – rb g A* 1 As 2 A* ( ) (4.2 rb = 0. using the first value of A* which is 1.5 in/s C* = 5100 in/s g = 386.3) 43 . The equation for pressure is: C* Pc = rb g As A* (4. substituting into equation (4. As is the propellant burn surface area. M 2 +1 (4. The first step is to calculate the chamber pressure.1) with respect to A* . To find the sensitivity of pressure to motor throat area.4 Example In the following hypothetical example. is the propellant density.3): Ae 2 1 2 * = 2 1+ A M +1 − 1 2 −1 . It is intuitively obvious that the relationship between the partial derivative and A* is both negative and inversely proportional to A*2.1). g is gravity. the sensitivity of pressure with respect to throat area is being determined. The equation for this analysis is the pressure equation. The next step is to calculate Mach number (M) iteratively from equation (4.9 in2 .1.9 in2 .2) to obtain the slope at a particular value of A * . Pc is taken over a narrow range where rb is approximately constant. take the partial derivative of equation (4. Numbers can be substituted into equation (4.06 lbm/in3 γ = 1. and A * is the throat area.1 in2 Ae = 10 in2 = 0. Another example of the approximation method is the substitution of selected values into the thrust equation (4. The sensitivity of thrust to throat area is to be investigated for a hypothetical motor with the following characteristics: As = 300 in2 A* = 1. r b is the propellant burn rate.
care must be used to ensure a small enough range for the parameter values to achieve the desired accuracy. 4. Table 42.52 56. If the approximation approach above is taken.5). but changes the nozzle expansion ratio and exit pressure.82 2.1. using A* = 2.60 A* = 1. 4.5) The final step is to calculate thrust (T) from equation (4. and which parameters can yield the most benefit for adjustment. 1 +1 −1 2 2 2 2 −1 Pe CF = 1− P c − 1 + 1 (4. The values obtained from both calculations are shown in table 42. Pe = Pc 1 1+ − 1 2 −1 M 2 (4.47 T 177.93 CF 1.6). The pressure ratio is typically found by iteration or by tables. when obtaining the sensitivity of thrust to throat diameter.4) The next step is to calculate the thrust coefficient (CF) from equation (4.87 1. (4. T = CF A* Pc .6 Limitations It is often not easy to isolate a variable to obtain a second derivative.1 in 2 .4).1 in2 Conclusion: in2 The thrust (T ) is inversely proportional to the throat area (A * ).5 Advantages The effect of each parameter can be assessed to determine which parameters have the greatest effect on the outcome of a process. Sensitivity analysis calculations.62 174. 44 .57 M 2. For example.75 Pe 1.6) The above calculations should be performed again. Pc 62.1. changing a throat diameter not only changes motor pressure.9 A* = 2.50 1.The third step is to calculate nozzle exit plane pressure (Pe) from equation (4.
2. from ANSI–Y–14. 4. A tolerance is shown for each significant digit used in the basic dimensions on the drawing.5 is an internationally recognized method of stating dimensions and tolerances. and machinists to assure the form. 4.5: straightness flatness circularity cylindricity profile of a line profile of a surface BXXX datum identifying letter basic dimension (XXX) angularity perpendicularity parallelism position concentricity circular runout total runout reference dimension A basic dimension is contained in a box (unless otherwise specified on the drawing). and function intended by the design engineer (or systems engineer). and have no tolerances associated with them..2 Application Standard dimensioning and tolerancing is typically applied in phase C but the technique could also be applied in phase D.2 Standard Dimensioning and Tolerancing 4. Often the title block of a drawing will indicate standard tolerances peculiar to that drawing that will apply to all basic dimensions shown without a specified tolerance.4. Standard dimensioning and tolerancing per ANSI–Y–14.2. such as a bolt pattern. manufacturing engineers.3 Following is a list of feature controls used to specify how a tolerance is to be applied to a design feature. Basic dimensions set up a dimensional pattern.2. fit. Basic dimensions are the controlling dimensions on a drawing. For example.1 may apply to all basic dimensions with one significant digit. unless standardized methods are employed at all stages of a project life from design to manufacture. bolt holes or threads) are toleranced using trueposition tolerances. This information is understood by draftsmen. Standard dimensioning and tolerancing allows the design engineer to indicate how tolerances are to be applied. Some of the methods of specifying dimensions and tolerances are discussed here. a tolerance of ± 0.1 Description Dimensioning and tolerancing on drawings is complicated enough to yield confusion. The locations of the features in the pattern (e.g.3 Procedures This section explains how dimensions and tolerances are specified on design drawings.4. 45 .
005.g. In this situation.020 tolerance zone perpendicular to the throat axis. The surface is to be within a 0. The second symbol in the second section (a circle with the letter “M.g. the inner and outer diameters are of primary importance. A rectangular box is used as a feature control box. The symbol in the first section of the box is for the type of tolerance (e. The true position of the exit is to be within ±0.030 of the throat axis (datum A).015 of the throat axis.” “L. 46 . The true position tolerance is not affected by the feature size of the throat diameter.” An “M” inside the circle would denote that the position tolerance applies to “maximum material condition.030 R 0.010 tolerance zone perpendicular to the axis made by the throat and exit (datums A and B).010 AB 3. thus datum A is the axis of the throat.) The exit plane also defines datum B.003 1. an inner and coincident outer diameter are specified. the thickness is a reference dimension. and the axis of the threads is to be within ±0. (The “R” inside the circle indicates that the position tolerance is applied “regardless of feature size.000 ±0.. The number is the size of the tolerance.. An “L” inside the circle would denote “least material condition” where the tolerance applies to the smallest feature size allowable.00 0. The boss at the end of the nozzle is controlled by a total runout tolerance. 41).” thus the tolerance can be relaxed by an amount commensurate with the difference that the size of the feature is less than the maximum allowable size.2.4 Example Following is a hypothetical fixed nozzle assembly used to show the purpose of dimension and tolerance methods: B 4. Note that for the profile type tolerance controls (e.020 R AA A 1. The first symbol in the second section is the type of measurement (a diametrical tolerance is shown in fig.020 0. The threads of the nozzle are to be concentric to the throat and exit axis within 0. datum A is defined by the throat of the nozzle. and the exit plane is to be within a 0. 4.” or “R”) specifies the relation of the tolerance with the size of the feature. The nozzle exit is referenced to datum A.Reference dimensions are the result of basic dimensions. the thickness is of secondary importance. In the example below. Example of dimensioning and tolerancing. The third (and any subsequent) section specifies which data are used (which feature or dimension the tolerances concern).25 X 12 UNF 0.005 R AB Figure 41. In this example. true position).000 ±.
030 radius around the throat axis. The boss surface tolerance is to facilitate a consistent seal with the motor. It can be seen that the thickness of the boss is not a controlling dimension. 47 . or function of the component would not be affected. or to determine a tolerance or dimension necessary to avoid form.5 Advantages Dimensioning and tolerancing per ANSI–Y–14. and the required perpendicularity to the throat axis is greater than the true position tolerance. used to determine size or position if all applicable dimensions occur at the worstcase extremes of the tolerance zones simultaneously.2. This is facilitated by following the dimensioning and tolerancing system of ANSI–Y–14. Thus a tolerance zone of 0. This means that the total “width” of the acceptable deviation is defined by the tolerance. whereas putting tolerances to rectangular coordinates allows a square tolerance zone. The nozzle boss is controlled by runout to the axis defined by the throat and exit plane. a functional part that would comply with true position tolerances may not comply with rectangular tolerances. Dimensioning strategy can minimize the cumulative tolerance stackup.3 Procedures Three typical methods for tolerance stackup analysis are: (1) Worstcase tolerance stackup analysis.5. it is a reference dimension.6 Limitations A moderate amount of training and practice is required to effectively use standard dimensioning and tolerancing.5 are better suited for production. Tolerance stackup analysis is typically performed by either assuming worstcase allowable dimensions. the number defines a tolerance zone.1 Description Tolerance stackup analysis determines if a form. For example. For position tolerances.runout or perpendicularity). or function problems. In addition. thus the axis of the nozzle exit must fall inside a circle of 0. fit.3. some aspects of dimensioning and tolerancing per ANSI–Y–14.2 Application Tolerance stackup analysis is typically performed in phase C or D. Note that the tolerances in this example control the thrust vector. 4.3. The exit is truepositioned to the basic dimension from the nozzle throat. 4. true positioning allows for a circular tolerance zone. 4. This technique is used to determine the possibility or probability of having form.2. the number call out is ±. Thus. The length of the nozzle is controlled by a basic dimension. fit. 4.3. 4. fit.010 tolerance. or function problem exists when manufacturing tolerances combine in a finished part or assembly. or by using statistical analysis of tolerances. The thread is controlled by concentricity to the same axis to keep the thrust axis in line with the motor axis. The nozzle exit is toleranced to keep the thrust vector in line (within a certain amount) with the throat axis. If this dimension were not specified.020 is analogous to a ±0. or function problems with a design.5 is fairly standard. fit.3 Tolerance Stackup Analysis 4. the form.
3.035 is obtained by finding the distance from the mean (in terms of standard deviations) that this condition represents. The standard deviation is assumed to be one third of the tolerance on the parts (this means all parts will fall within 3 standard deviations of the nominal dimension) and is therefore: Component standard deviation = 0. the tolerances of each component are shown in figure 43. where a computer is used to do a Monte Carlo analysis of the possible combinations of tolerances.00167 and by summation of squares.035.035. and the probability that the squeeze will be less than 0. The probability of the squeeze being less than 0. ±0.005.5 4. Oring joint components. Find the maximum tolerance stackup possible to obtain the minimum squeeze.5 = 0.010 Figure 43.4. For a squeeze of 0. system standard deviation = (2(0. the distance (in standard deviations) from the mean (z) is z = (0. used where the expected standard deviations of tolerances are combined to determine the probability of a final tolerance.050 inches.050)/0. Oring joint.0033 Oring standard deviation = 0.005/3 = 0. 48 .4.010 ±0.(2) (3) Statistical analysis of tolerances.0. Figure 42.010/3 = 0.00167)2).4 Example In the following hypothetical Oring joint assembly (fig.4 Design using simulation methods. 42).0033)2 + (0.005 ±0.035–0. The nominal squeeze is 0.005 = –3.
their inclusion in tolerance stackup analysis is superfluous.4987. M. the area under the curve beyond z = 3 is (0. 4. fit.. fit. care must be exercised when combining tolerances.0005 tolerance. so software exists to perform the statistical analysis. it becomes increasingly unlikely that all dimensions will be worstcase simultaneously.13 percent probability that the squeeze on the Oring will be 0.Using a table for the normal distribution function. Since this is a onesided question (no interest in the squeeze being 0. e. e.g. It may be superfluous to combine tolerances from different manufacturing processes. or function.5 Advantages Worstcase tolerance analysis can simply determine the envelope of possible form. Inc. In addition. thus position can vary more when not at maximum material condition. even if exceeding a requirement is possible. It can be extremely cumbersome to perform a statistical analysis of all the possible variations on some assemblies. A typical example of software is the “Variation Simulation Analysis”4. in that when many tolerances combine. position tolerances are often determined at maximum material condition. which may be unrealistic.0013.” Applied Computer Solutions.3.3 4. the area under the half curve for z = ±3 is 0. a 0. it may be extremely unlikely.6 Limitations Worstcase tolerance analysis is conservative. 4. or which tolerances are most important to form. machining and casting.3.4 The results can be used to determine probabilities that certain overall tolerances will exceed a critical value.g.065). (2) 4.4987) = 0. For example.3. and function.5 that uses Monte Carlo methods to simulate the possible ways that the tolerances can stack up. A close look at the example above will show that more sources of variation are possible than those considered. Statistical tolerance analysis usually assumes a normal distribution of dimensions in the tolerance zone.7 Bibliography Craig.035 or less.030 tolerance may have a smallest unit of measurement greater than a 0. Statistical analysis can show that. the surfaces compressing the Oring may not be flat or normal to the direction of squeeze. 49 . Also..5–0. This value is interpreted as 0. Consideration of significant digits may be helpful. 4. in that: (1) If some tolerances are much smaller than others.: “Managing Variation by Design Using Simulation Methods.
“Dimensioning and Tolerancing.” ANSI–Y–14.. Cruz. 300 Maple Park Blvd. Hutchinson and Rose. PA. Inc..” Edited by J.” Applied Computer Solutions Inc.” Third edition.2 4. St. New York.” Third edition.E. AddisonWesley Publishing Company. United Engineering Center.. The American Society of Mechanical Engineers. “Variation Simulation Analysis Software. 410 . Dowden.REFERENCES 4.B. 1982. Stroudsburg. C. McGrawHill. November 1970.4 4. P.5 “System Sensitivity Analysis. Reading.3 4. Shigley.: “Mechanical Engineering Design. Clair Shores. NY 10017. J. MA. Jr. Getting Started Manual. and Peterson. Hill.. MI 48081.5M. 345 East 47th Street.1 4.: “Mechanics and Dynamics of Propulsion.G.R.
The scatter diagram displays one variable on the horizontal (X) axis and the other variable on the vertical (Y) axis. A summary of the advantages and limitations of each tool or methodology discussed in this section is presented in table 51.5. is a bar chart that shows a dispersion of data over a specified range. GRAPHICAL DATA INTERPRETATION TOOLS There are numerous excellent texts on the appropriate use of graphical data interpretation tools. The more that this cluster resembles a straight line. A Pareto chart. This chart is discussed in section 5.1 5. is used typically when there is a need to know the relative importance of data or variables. the neophyte reader is advised to read and utilize standard handbook references when using these techniques in problem solving to avoid misuse and error. plot raw data and allow the analyst to determine if any relationship exists between two variables. causes. One of the most common data displays is the timeline chart. Control charts are useful in trend analysis. section 5.14. This chart is discussed in section 5. it can be assumed if the data from one are changed.1. even when the connection between two variables is unexpected. The analysis can be used to monitor performance. One way to analyze data is by graphical interpretation.2 51 . This type of chart is commonly used in presentations to make data easier to interpret. This chart displays changes over time. identify relationships.6. monitors the performance of a process with frequent outputs. section 5. section 8. the stronger the correlation between the variables. The control chart.1.4.5. section 5. and references for their appropriate application. 5. makes it possible to determine if any relationship exists between two variables. The direction and compactness of the cluster of points gives a clue as to the strength of the relationship between the variables. Sorting data that share a common characteristic into different groups is often accomplished with a stratification chart. also called XY graphs. The scatter diagram technique is best applied in phase E. No interpretation of the data should be attempted.3. A histogram. This chart will also identify the problems.1 Scatter Diagram 5.2.2 Application The graphic display of the scatter diagram can help one determine possible causes of problems. The scatter diagram. section 7. section 5.1 Description Scatter diagrams.7. If there is a correlation between the two variables. and statistical process control. This toolbox is to provide knowledge of the existence of these techniques. positive or negative.5. then this will effect the data from the other. but correlations can be inferred.1. section 5. While this section lists and briefly discusses some of the available tools. The bar chart compares quantities of data to help identify distribution patterns. and reveal the most important variables in a set of data. or conditions that occur most frequently. 5.
Some preplanning needs to be done before choosing categories.4 The timeline chart shows the direction of change but it gives no indication as to the reason for the change.3 Timeline chart 5. (2) It makes graphic comparisons of quantity easy to see. A bar chart is limited in the number of data categories that can be displayed at one time. (1) The timeline chart shows a “moving picture” of fluctuations over time. (2) A histogram helps establish standards for a process. Histograms 5.7 (1) A histogram helps identify changes in a process as A histogram is not a good tool for computing process the data changes. Control chart 5.1 Advantages Limitations (1) The general relationship between two variables can (1) The choice of scale for the graph can distort the be determined at a glance. (2) The chart is useful in analyzing defect data. Tool or Methodology Scatter diagram Section 5.5 (1) The correct stratification variables for resolving a problem are generally not known prior to data collection. (2) Defect rates can be plotted on time lines in order to identify trends.Table 51. data. (1) The control chart helps one understand the (1) The control chart tells only if the process is in capabilities of the process. (2) The chart can prevent tampering with processes that (2) The underlying causes are not determined. A poor pareto chart will result if the causes chosen to study are wrong. (2) The correlation does not prove a causeandeffect relationship. capability. Pareto chart 5.6 (1) The pareto chart helps to identify the few areas of concern that are most important. (3) The chart monitors the effects of process changes that are aimed at improvement. control. The approach not only produces a priority ordering of the problems but also identifies an improvement strategy. Graphical data interpretation tools and methodologies. 52 . are under statistical control. (4) Control charts can be used without extensive knowledge of statistics. Stratification chart 5. thus possibly giving the appearance of a (2) The graph can help determine a possible cause(s) of correlation that is better or worse than reality.2 Bar chart 5. (1) The bar chart tells its story at a glance. (2) All potentially important stratification variables cannot be determined without planning. problems by looking at correlations.
3.1. a scatter diagram is prepared in the following manner: (1) (2) (3) Collect the two selected variables of each occurrence. The correlation does not prove a causeandeffect relationship. A possible relationship can be determined by visual inspection of the graph. The dependent variable.5 Advantages (1) (2) The general relationship between two variables can be determined at a glance. Set the scale intervals and label. the one that you can have an effect on. The graph can help determine a possible cause of problems by looking at correlations. example shown in figure 51.6 Limitations (1) (2) The choice of scale for the graph can distort the data.4 Example As adapted from reference 5.2. 5.1. (4) (5) 5. would show if there were any relationship between the test scores and the production levels. The independent variable is assigned to the horizontal (X) axis.1. 5. an aptitude test was given to 10 employees and the scores were then compared to the production levels of these employees over a certain time period.1. Employee 1 2 3 4 5 6 7 8 9 10 Test Score 27 13 8 37 32 10 17 22 6 7 Production Level 120 80 60 150 135 70 95 105 50 55 This plot shows that the higher test scores result in higher production levels.3 Procedures As described in reference 5. Plot each data point.5. 53 . The scatter diagram. is assigned to the vertical (Y) axis. thus possibly giving the appearance of a correlation that is better or worse than reality. Draw the horizontal and vertical scales with equal length.
1 Description A control chart monitors the performance of a process with frequent outputs. 5. and groups of points from a stable process tend to fall within predictable bounds.2 5. and when changes occur they are generally out of the range of normal operations. The chart shows a pictorial representation of an ongoing process and determines whether or not the process is performing within acceptable parameters.160 140 Production Level 120 100 80 60 40 20 0 0 5 • •• • • • • • • • 10 15 20 25 30 35 40 Test Score Figure 51.2 Application The control chart technique is best performed in phase E.2. percent defective (P). control charts are used to show the variation of several variables including average ( X ) and range (R) as well as the number of defects (PN). The upper control limits (UCL) and lower control limits (LCL) should not be confused with specification limits.2. As described in reference 5.2 Control Chart 5. A stable process changes randomly. 5. Scatter diagram example. defects per variable unit (U). An unstable process does not change randomly. and defects per fixed unit (C). Points that are outside of the control limits reveal that something has occurred that requires special attention because the points are outside of the builtin 54 . The control chart is based on four concepts: (1) (2) (3) (4) All processes change with time. The control limits show the natural change of a process.2. such that points within the limits generally indicate normal and expected change. Individual points of the process are unpredictable.
As new points are added. a control chart (fig.125 0.1 0 0 0.14) and trend analysis (sec.6 0.167 0.15 0. 5. Plot the data on the control chart to evaluate performance and identify the points outside of the control limits.25 LCL=0. 5.25 0.182 0. The control chart can to be used continuously to determine whether the process remains within established control limits.25 0.333 0.083 0 0 0 0 0 0 0 0 0.2 0. 7.4 0.3 Procedures As described in reference 5.systematic cause of change in the process. Control charts are used in performing statistical process control (SPC) (sec.3 0.076 0.5 0.111 0.5 0. Gather data.1 0.091 0. Determine why points are outside of the control limits.2.076 5 10 15 Event 20 25 30 Figure 52.5 0.286 0.333 0.). Find ways to identify causes of problem points.1 0. Control chart example.2. One point that is outside of the control limits does not mean the process is out of control but it should be explained. 55 .5 UCL = 0. reduce the normal variation.2.091 0.4 Example 0.2 0. and improve the mean.2 0. 8.55 0. 52) is constructed in the following manner: (1) (2) (3) (4) (5) Determine the control limits to show the expected change of the process. the chart can be monitored for points that may fall outside of the limits and require causes to be identified.182 0.
The quantities of data are depicted by the lengths of the bars that represent cost. 53) is constructed in the following manner: (1) (2) (3) (4) (5) If necessary. The control chart can prevent tampering with processes that are under statistical control.5.5 Advantages (1) (2) (3) (4) The control chart helps the analyst understand the capabilities of the process. The control chart does not indicate the underlying cause unless data on outside processes are included in the analysis. raw data are entered on a checklist (sec.6 Limitations (1) (2) The control chart tells only if the process is in control. percentage.3.8). List the categories across the horizontal scale at the bottom.2.3 Bar Chart 5.3. Give the bar chart a legend to identify different colors or patterns.1 Description Bar charts show a comparison of quantities of data to help identify quantity changes. 5. Label the quantities on the vertical scale at the left. 7.3.2.3 Procedures As described in reference 5.2. The control chart monitors the effects of process changes that are aimed at improvement. The bars may be horizontal or vertical.5. Differences and similarities between and among selected categories are emphasized by the heights of the columns. Control charts can be used without extensive knowledge of statistics. or frequency of events. 5. Make sure the scale is broad enough to include the highest and lowest value in each category. Bar charts can show double and triple bars to compare different time periods or different populations.2 5. a bar chart (fig.2 Application Bar charts are one of the most common types of data display and this technique is typically performed in phase E. 5. Draw the bar according to the quantity of each category. 56 .
4 Example Sale of Household Appliances 1980 versus 1990 (in millions) 10 9 8 7 6 5 4 3 2 1 0 1980 1990 (Nominal Categories) Figure 53.5. 5. The bar chart makes graphic comparisons of quantity easy to see.3.3.5 Advantages (1) (2) The bar chart tells its story at a glance.6 time.3. Bar chart example. Limitations A bar chart is somewhat limited in the number of data categories that can be displayed at one 57 . 5.
add horizontal and vertical grids. 20.2 This technique is best performed in phase E.1 Description The timeline chart is among the most common types of data displays. Connect.) and label the axis.8). with line segments. The line segments connecting the points on the chart give a clear picture of changes over time. the quantities plotted for each successive interval. 7.2.” and “weeks.4. Defect rates can be plotted on time lines in order to identify trends. If the points are difficult to read.4.4 TimeLine Chart 5.) for the horizontal axis.3 Procedures As described in reference 5. The chart connects data points with line segments. 54) is prepared in the following manner: (1) (2) (3) (4) (5) (6) 5.g. etc.” “days.5. Example A study was made comparing the average number of errors that were made per operator at different times of the day over a certain time period (fig. 5. 58 . 5.”5. a timeline chart (fig.4.. 54).4 Enter the raw data on a checklist (sec. weeks. The vertical scale is a quantity while the horizontal scale is divided into time intervals such as “hours. days. The chart graphically displays changes over a period of time. Title the chart to define the time period for which the data are displayed. The intervals should be evenly spaced and labeled.4. Establish time intervals (usually hours. 10. 30.2 Application The timeline chart is a special case of XY plots where the independent variable is some time value. Establish the quantities for the vertical axis and make them evenly spaced (e. 5.4.5 Advantages (1) (2) The timeline shows a “moving picture” of fluctuations over time. etc.
5. a process is incurring a 10percent defect rate with a particular product. lot. machine. Timeline chart example. etc. time. operator.5.5. operator. This does not mean the “cause” of a problem has been found.5. What has been found is where the problem is occurring the most.” is used in data analysis.1 Description The term “stratification. Stratification is done by sorting data into different groups that share a common characteristic. units.5 Stratification Chart 5. shift.4. Some common stratification variables are shift. You can stratify by vendor. of Errors l 04:00:00 PM Time Intervals Figure 54. 5. or other types of strata can often lead to suggesting an improvement strategy. 5.2 Application The stratification chart is best applied in phase E. select other stratification variables and collect more data. and compute a percent defective for each category (stratification variable).4 59 05:00:00 PM 12 Noon . Should the data not include a significant problem. The graph may show that one category is producing a higher defect rate than others.” derived from “stratum.6 5 Quantity 4 3 2 1 0 09:00:00 AM 10:00:00 AM 11:00:00 AM 01:00:00 PM 02:00:00 PM 03:00:00 PM l l l l l l l l l No. The data can be depicted in graphic form for easy visual interpretation. Comparisons of different groups.6 Limitations The timeline chart shows the direction of change but it gives no indication as to the reason for the change. The cause has yet to be determined. and machine. For example.
5. and histograms (sec. Determine the strategy to improve the problem.5. 5. the stratification process (fig. Analyze the data on the chosen stratification variables and compare to each other.6).3 Procedures As described in reference 5. 5. Graph the data using one of a number of different tools. Separate the possible problem areas into special and common cause problems. If no conclusions are found.4. such as bar chart (sec. 55) is performed in the following manner: (1) (2) (3) (4) (5) (6) (7) Choose the stratification variables. Pareto chart (sec. 5. Gather data and record the potentially important stratification variables. History of Discrepancy Reports for a Solid Rocket Motor 20 18 16 14 12 10 JOINT 8 6 4 NOZZLE 2 0 CASE IGNITER INSULATION LEGEND PROPELLANT Month Figure 55. choose different stratification variables.3). Stratification (histogram) chart example.7). 510 .
2).5.5.5 Advantages The approach not only produces a priority ordering of the problems but also identifies areas for improvement.” and “why” of a suspected problem cause.1 Description When there is a need to know the relative importance of data or variables (problems. or conditions are the most important or most frequent so they can be addressed first. causes. a Pareto chart (fig. use a checklist (sec. 7.6. 5.5. Gather the data on causes.2 Application The Pareto chart can be used to examine the “how. The Pareto chart also helps to identify which problems.5 Pareto charts are used in performing problem trend analyses (sec. 5. The chart is an illustration of the data as of a specific time period. This technique is typically performed in phase E.” “what. or conditions). stratified by components. causes.4 Example Shown in figure 55 is a histogram of discrepancy reports for a solid rocket motor (SRM). 511 . 56) is created in the following manner: (1) (2) (3) (4) (5) Identify the most likely causes of a problem (take from the cause/effect diagram (sec.5.3 Procedures As described in reference 5. 7. 8.2 5. 5.5. Summarize the numbers of observations and calculate the percentages of each cause.6.8).6. This chart helps to highlight the few data or variables that may be vital. 5.2)).” “where.5. if necessary. The Pareto chart is based on the “Pareto principle” which states that a few of the causes often account for most of the effects. The data are arranged in descending order with the most important to the left.6 Limitations (1) (2) The correct stratification variables for resolving a problem are generally not known prior to data collection.” “when. a Pareto chart is often used. All potentially important stratification variables cannot be determined without planning.2. Make the left vertical scale the same height as the right scale and set it from zero to the number of observations. Set the right vertical scale from zero to 100 percent.6 Pareto Chart 5.
100 100% 80 80% 60 60% 40 40% 20 20% 0 Power Supply Machine Connection Calibration Electrical Component Feed Transformer Wrong Connection Operator Training H O Filter 2 0% Figure 56. The plotted points are then joined with line segments. areas to the left of the most radical slope are the most probable problem areas. Calculate and add together the percentages of cause one and two. Pareto chart example.e. 512 . The third point is found by adding the percentage of cause three to the total of one and two. (6) (7) (8) The columns are drawn using the left scale. is plotted across from the right scale directly over the second column. corresponding to their sum. i.. The second point. and the last point is at the 100percent point. The first point is plotted at the upper center of the first column. and plot. The total of all columns added together should be 100 percent. This observation is even more obvious when the heights of the bars are examined. (9) The chart in figure 56 reveals the slope is more radical over the first two bars (power supply and machine calibration) and this means that the majority of the problems occur in these categories.
This spread of data makes presentations easier to interpret.5 Advantages (1) (2) The Pareto chart helps to identify the few areas of concern that are most important.” Ford Motor Company.6. The chart is useful in analyzing defect data..1 5.” John Wiley. Use of Simple Statistical Tools.7. 5. Wadsworth.2. the bars are of equal width. 57) is constructed in the following manner: (1) (2) (3) (4) Gather the data to be plotted and count the total number of data points. 1989. W. D.4 The histogram is best applied in phase E. Scale the groups of data on the horizontal axis.C. The width of each bar is determined by dividing the range of data by the selected number of bars. Find the range of the data by subtracting the smallest data point from the largest.6.7. and Godfrey: “Modern Methods for Quality Control and Improvement.6.: “Defect Prevention.E. MI.5.5. 1986. New York.7 Histograms 5. 5. Fewer items fall on either side of the center. 5. Since group intervals are equal in size. The bars are proportional in height to the frequency of the group represented. S.3 Procedures As described in reference 5. Some preplanning needs to be done before choosing categories. a histogram (fig. and Montgomery.6 Limitations A poor Pareto chart will result if the causes chosen to study are wrong.W. 5. V.1 Description Histograms are bar charts that show a dispersion of data over a specified range. The number of data bars in the graph should be limited to between 6 and 12.7. New York.5. Hines.: “Probability and Statistics in Engineering and Management Science.2 Application When data are plotted on histograms. 1986. 513 .” John Wiley.7 Bibliography Cane. many items tend to fall toward the center of the data distribution. Livonia.
is the same width. 5. Draw the height of each bar to show the number or frequency of the group interval using the scale on the vertical axis. Each bar. Plot the frequency of occurrence of the numbers in ascending order. 5.5 Advantages (1) (2) A histogram helps identify changes in a process as the data changes.7. 514 . 8 7 6 5 4 3 2 1 0 010 1020 2030 3040 4050 Time To Complete Tasks (Hours) Figure 57.7.7. A histogram helps establish standards for a process.(5) (6) (7) Scale the frequency of occurrence or the numbers on the vertical scale. 5.6 Limitations A histogram is not a good tool for computing process capability. Histogram example. including all data points.4 Example The chart in figure 57 displays a typical histogram.
: “Tools of Total Quality. and Brocka. Homewood. Implementing the Best Ideas of the Masters.: “Quality Management. Livonia. V. V.15. IL 60430. “Total Quality Management. 515 .” Ford Motor Company. Use of Simple Statistical Tools. 1992.” Chapman & Hall.51–G (Draft).1 5. IL 60430. B. February 15.REFERENCES 5. Hunt. Lyonnet.2 5.3 5.H86. How to Implement a Competitive Quality Program. 1991. A Guide for Implementation. MI.S. Business One Irwin.5 Brocka. 1989.” HD62. Homewood.: “Quality in America. Cane.” Business One Irwin.E.4 5.D. An Introduction to Statistical Process Control. M.: “Defect Prevention. P. 1989.” DOD 5000.
there is a finite probability that they are “close” to the population statistics. and response surface methodology.1 Description As described in reference 6. 61 .” which is based on the sample mean and standard deviation. 6.1 “Studentt” Analysis 6. The tdistribution is similar to the normal distribution in that with an infinite sample size. The output of the tdistribution chart is the probability (α) that t exceeds a certain t on the ordinate of the tdistribution chart. When only sample statistics are available (as is usually the case in engineering applications). usually the probability is chosen and t is sought. the “studentt” compares the sample statistic “t. regression analysis. confidence analysis. Determine if two sample means are equivalent to each other within a given probability of error. the novice statistician is cautioned to read and utilize standard. or if the solution to a function for the probability distribution of points were available. as described in reference 6. a comparison of sample statistics and population statistics will be made. to the tdistribution for the same sample size and a desired significance (probability of error).1. handbook references when using these techniques in problem solving.” thus the name “studentt” analysis.1 A summary of the advantages and limitations of each tool or methodology discussed in this section is presented in table 61. However. STATISTICAL TOOLS AND METHODOLOGIES There are numerous. are used when sample sizes are low for the following functions: (1) (2) Determine if a sample mean is equivalent to a population mean within a given probability of error. In many of these analyses.1. At sample sizes lower than infinity. The tdistribution was described in 1908 by W.S.2 Application “Studentt” analyses. Gosset under the pen name “student. Sample statistics are made from actual measurements of a sample with a finite number of specimens.1.2. 6. This technique is typically applied in phase D but may also be performed in phase C or E. excellent and highly detailed texts on the appropriate use of statistical techniques. the tdistribution is equivalent to the standard normal distribution. 6. analysis of variance (ANOVA). Use solely of this text might well result in misuse and error. In this section. Here. the following typical statistical processes are discussed: “studentt” (t test) analysis.6. it is assumed that population statistics would be obtained if an infinite number of specimens could be measured. While this section lists and briefly discusses some of the available tools. the tdistribution becomes “lower and flatter” than the normal distribution. correlation analysis. a tdistribution table is usually used to find t . factorial analysis. This toolbox does provide a suitable knowledge of the existence of these tools and references for their appropriate application.
Limitations The parent distribution must be reasonably close to a normal distribution. when the relationship is not obvious by generated will only approximate the actual relationship. interactions between variables can be isolated. Sources of variation can be found.4 Confidence/reliability determination and analysis 6.3 6. Regression analysis 6. A full factorial analysis does not solve for exponential or polynomial effects. inspection. such as the population standard deviation. Also. before an analysis can be performed.1 6. The processes are timeconsuming and often approximate. integer data). when the relationship is not obvious by generated will only approximate the actual relationship. Tool or Methodology “Studentt” analysis ANOVA Section 6. and large numbers of variables can be solved. A sample statistic must be known or assumed.g. A mathematical relationship can be determined.2 Advantages The procedure is relatively simple to apply. by hand If the data are discrete (e. or any chosen source of variability isolated. Response surface methodology 6. the actual line or computer. Sources of variation can be found.6 A mathematical relationship can be determined. random variation isolated.. or any chosen source of variability isolated. The fractional factorial analysis does not solve for all effects and higher order effects separately. integer data). random variation isolated. Statistical tools and methodologies.. The processes in factorial analysis are more timeconsuming than the analysis of variance.g. Correlation analysis Factorial analysis 6. by hand If the data are discrete (e. inspection. the actual line or computer.7 62 . A straightline relationship is assumed.5 This analysis can give a realistic probability of whether or not a process may yield a value which is above or below a requirement. The analysis is quite simple.Table 61.
05 significance).134. the null hypothesis is that the sample and population mean are equal.6. This is referred to as a type I error. Determine with a 0. the null hypothesis is disproved. The determination of the probability of a type II error is complicated and many texts consider it beyond their scope.1). a twotailed analysis will be done.1 6.1) Compare t with tα for the desired significance and degreesoffreedom (DOF) (n–1). t = 2. The null hypothesis will be a strain capability equal to 34 percent. If t is greater than t .3 Procedures The use of a ttest for determining if a sample mean is equal to a chosen population mean will be shown here. The sample mean is 33 and the sample standard deviation is 2.1. 31.915. The nominal strain capability is 34 percent. it cannot be assumed with the chosen confidence that the sample mean is equivalent to the target mean. Thus. it could be stated that there was only one chance in ten that the null hypothesis was rejected when it should not have been. it could be stated that the null hypothesis could not be rejected at the 0. Substituting into equation (6. but the two alternate hypotheses would be that the sample mean is above or below the population mean. if t is greater than t( /2) (or t is less than – t( /2)).10 significance. The alternate hypothesis is that the sample mean is on the particular side of the population mean. the null hypothesis is disproved.10 level of significance unless the probability of a type II error is determined. 34. 6. Determine the mean and standard deviation of the sample.. (1) (2) Determine the target mean and significance level desired. For a twotailed analysis. If it is desired to prove that the sample mean is on one particular side of the population mean. 35. If it is desired to prove that the sample mean is not on either side of the population mean. Determine the t value using equation (6. α/2 will be used (0.5 Advantages The procedure is relatively simple to apply.1. t = 0. and 36 percent. This latter situation would use a “twotailed” analysis.1539. If Ho were not rejected. Since the mean of the propellant batch could be =.4 Example Pull tests of a propellant sample yielded the following strains before failure: 29. Develop null and alternate hypotheses for the problem being investigated. i. or <34 percent. t = sample mean – target mean sample (5) /( n) 1/2 (3) (4) (6.1. the null hypothesis would be the same.1). From the tdistribution table for 4 DOF.e. If Ho had been rejected. >. 63 .6. if the propellant batch is representative of the nominal propellant.
a value that is related to the total DOF.: “Statistics Manual.2. 6. M. Mendenhall. E. 1975. “Experimental Statistics.S. National Bureau of Standards.A. F. Wadsworth Publishing Company. Davis.6. Handbook 91.2 Analysis of Variance 6. 5. based on the number of samples (k) and the sample size (n). Determining the significance of each factor.5) to compare sample statistics.L. to determine if the variation of the mean and variance between two or more populations are attributable to sources other than random variation.3 Determining if two or more processes are producing products that are consistent with each other. Belmont.” Fourth edition. W.2)) is compared to F . a one way classification is used. Some of the uses for analysis of variance are: (1) (2) (3) (4) 6.2 Application The ANOVA technique is typically performed in phase D but may also be performed in phase C or E. Procedures As described in reference 6.2. to determine if two or more samples have different sample statistics.2. the following methods can be used to determine if the withinsample variation is greater than the sampletosample variation. Eliminate one source of variation to determine the effect of the others.1.1.1 Description ANOVA is a technique used in design of experiments (sec.” U.: “Introduction to Probability and Statistics. 6. Department of Commerce. A factor F (equation (6. betweensample variance mean ofwithinsample variance F= (6. CA 94002.1 6. 6.1.6 Limitations The distribution of the parent population must be reasonably close to a normal distribution.7 Bibliography Crow. If only one source of variation is being investigated.W.. and Maxfield.2) 64 . Determine which two or more processes are different if a difference in processes is detected.” NAVORD Report 3369. NOTS 948.
MS = Mean square SS(bs1) = ∑(Ti )2/b – C. Rather than determining sample statistics for each sample. and the DOF of the sample size is n–1.The DOF of the number of samples is k–1. and where a = the number of samples of one source of variation and b = the number of samples of the other source of variation. If two sources of variation are being investigated. and Ti = total for each sample.4) MS(bs2) SS(bs2)/ b − 1 = MSE SSE / K(a − 1)(b − 1) (6. The SSE is determined from the sum of squares total (SST) and SS(bs) by the formula SSE = SST – SS(bs). a twoway classification is used. The total DOF is k*(n–1). T = total of all data points. the following approximation equations can be used: MS(bs1) SS(bs1)/ a − 1 = MSE SSE / K(a − 1)(b − 1) F(bs1) = and F(bs2) = where SSE = SST – SS(bs1) – SS(bs2). F is found from an F distribution table. Data can be arranged in blocks representing one source of variation. (6. can be used. SS(bs) = ∑ (Ti )2/n – C. F= SS(bs)/ (k − 1) . C = T2/(k*n). MSE = Mean square error SS(bs2) = ∑(Tj )2/a – C. If F exceeds F . and C = T2/(k*n) where y ij = each data point. and one data point from each sample representing the other source of variation is put into each block (see example below). then a difference exists between the samples that is not only due to random variation.3) where SS(bs) is the sum of squares (betweensample) and SSE is the sum of squares error. SST and SS(bs) can be found using the formulas SST = ∑ (yij)2 – C.5) SST = ∑(yij)2 – C. approximation formulas that use sums and averages of squares. 65 . If two sources are being investigated. SSE / k(n − 1) (6.
0. The latin square method eliminates three sources. 6. SS(bs2) = (932 + 1032)/3 – 6402.67 = 5. These methods must use n2 observations. MS(bs1) = 5.33.Other methods exist to isolate more sources of variability simultaneously.67. The use of three lots of polymer (parameter A) and two lots of curative (parameter B) will be investigated. SST = 302 + 342 + 322 + 362 + 312 + 332 – 6402.2. the effect of two parameters on the variability of strain capability of a solid propellant will be investigated.67 = 23. F(sb1) = 2. MS(bs2) = 16.67 = 24. SS(bs1) = (642 + 682 + 642)/2 – 6402.67 = 6408 – 6402.67/0. and F(sb2) = 16.67. Six batches of propellant are mixed and tested with the following average results: Polymer 1 1 2 2 3 3 Curative 1 2 1 2 1 2 Percent Strain 30 34 32 36 31 33 The following table is arranged with parameter A set up in columns and parameter B set up in rows: Curative Lot 1 30 32 31 93 Curative Lot 2 34 36 33 103 Total for Polymer 64 68 64 196 Polymer lot 1 Polymer lot 2 Polymer lot 3 Total for curative here C = (196)2/6 = 6402. This involves partitioning a total sum of products rather than squares.88.67/0.67. 66 .67 = 6426 – 6402.33/2 = 2.4 Example In the following hypothetical example. MSE = 1. and the GrecoLatin method eliminates four sources.67.33/((3–1)(21)) = 0.67.67/1 = 16.67 = 4. Analysis of covariance is a similar technique used when conditions (such as environmental) change.33.67 = 16. The effect of this change is accounted for by using regression.
2 Application Correlation analysis can be used to determine if a relationship exists between two independent sets of variables.14). 6. 6. 6.05 significance is greater than 5. if r is close to 1 (or –1) then a high degree of correlation is implied.6. If r is close to 0.2. and b = 2 is the number of sources of variation of parameter B. as found in reference 6. then no correlation is implied. strain capability is affected by the curative lot. for determining if two sets of data are linearly related is as follows: (1) (2) Determine the mean of each set of data.3 6. Since F(sb1) is less than F for a 0.05 significance (F = 5.2. This technique is best performed in phase D but may also be performed in phase C or E. The value of r will be between –1 and 1. 6.6 Limitations The processes are timeconsuming and often approximate. polymer has no effect on strain capability.3.1 Description Correlation analysis measures the strength of a linear relationship between two sets of data.3. Determine the r value of the two sets of data using the following equation: r= Σ ( xi − x ) ( Σ(x i − x )(yi − y ) 2 1/2 ) Σ(( yi − y ) ) 2 1/2 (6.3.7) 67 .Note that a = 3 is the number of sources of variation of parameter A (polymer).6) where x and y are the means of the first and second set of data respectively. Since F(sb1) for a 0.5 Advantages Sources of variation can be found.3 Correlation Analysis 6. random variation isolated. or any chosen source of variability isolated.3 Procedures The procedures. (3) Determine the significance of the r value by using the following equation: z= (n − 3) (1+ r) 1n 2 (1− r) (6.3.99.
but only at two levels for each variable.34 0.0004 0.(4) Look up the z value in a standard normal distribution table to determine the probability of having a correlation.01 percent chance of these two data sets not being related.1 2.6. Fractional factorial analysis is used when so many variables are being investigated that experimenting with them is unfeasible.0008 –0.0448 –0. factorial analysis further subdivides the treatment of sums of squares into components and can show interaction effects between parameters. 6.0 1.52 and 1.1. the number would be 64.992.548 dy2 0.4.04 –0.8.5.16 –0. The mean of the two sets are 6. x 5.4.3. respectively . 6.1296 0. Using this value for n = 5.0196 0.5.4 6. Factorial analysis is similar to ANOVA in that the analysis is based on sums of squares. and 2.3528 –0.02 0.938.2304 0.1024 0. 6. the r value is 0. Thus.1156 0. if five variables are being investigated. 2.98 dy 0. fractional factorial analysis is often economically necessary.14 0. except that factorial analysis deals with levels of variables.6. and squares of the deviations from the means are shown in the following. Factorial analysis is used with a small number of variables (e. and fractional factorial analysis. and 1. 6.1 6.25 0.3. For example. and 7. 25 or 32 experiments would have to be performed.1 Description There are three types of factorial analysis described in this section—factorial analysis. 68 .4 Factorial Analysis 6. For six variables.856 dx2 1.5 Advantages This analysis is simple to apply.3 2.256 0.48 0. 1. full factorial analysis.96.6). 7. Full factorial analysis is performed for more variables.9604 2.8 1. 6.36 dx * dy –0. 2.2 Application Factorial analysis is used for applications similar to those for which ANOVA is used.6.3.3808 –0.6 Limitations A straightline relationship is assumed.0 7..g. and this is without replication.3.2 6.6 dx –1.5 7.292 Using equation (6.12 –0. however. two to four).0016 0. z is –3.0768 –0.32 –0.2.5 summations y 2. 6. products.4 Example The following hypothetical sets of measurements were taken: 5.1 This technique is typically performed in phase C but may also be performed in phase D or E . thus there is less than a 0. The deviations.4.
The notation in column n (3) and the sum of squares column is shortened. B1.C1 total A1.C0 total A1.1. B0.4. 2:1 means the first element of column 2. 2. assuming straight line relationships. B1. Column 3 is the effect totals as in analysis of variation.C0 A1. (2) (3) Obtain a table of effects totals by removing the middle columns in the above table. Column 2 is constructed the same way from column 1 as column 1 was constructed from the totals column. and M3 represent the resulting magnitudes after the experiment for replication 1. B0.B0.C0 A0.C0 A0.C1 M1 M1 M1 M1 M1 M1 M1 M1 M2 M2 M2 M2 M2 M2 M2 M2 M3 M3 M3 M3 M3 M3 M3 M3 total A0.B0.C1 etc.B1. With factorial analysis.3 Procedures As described in reference 6.C0 total A0. Continue in a like manner to get the third and fourth elements in column 1.B0.B1. and 3. 69 .B0. r replications will be performed.C1 total A1. where the first column represents the experimental conditions and M1. factorial analysis is performed the same as analysis of variance except that an analysis of variance is performed for each variable against each other variable.C1 total A0. Add n (3) columns in the place of the middle columns and three columns to the right side of the table (table 63). A0. B0. B1.C1 A0. Add a row for the error sum of squares and error mean square. c.C0 A1.C0 total A1. The following is the procedure for using the factorial analysis where n = 3. The last column is the total of all replications of experiments for each experimental condition. Factorial analysis factors and magnitudes. Column 3 is constructed the same way from column 2. and to account for experimental variability. With factorial analysis. B0. 2n factorial experiments will be performed. (1) Arrange the factors and magnitudes in a table such as the following: Table 62. B1. b.C1 A1. Apply the method of Yates to this table as follows: a.C1 A1. M2.6.B1.C0 total A0. Here n will be the number of factors rather than the sample size (which is effectively two). Obtain the fifth through eighth elements in column 1 the same way except that the totals are subtracted (first value subtracted from the second). Add the first two totals in the totals column to get the first element in column 1.B1. Add the third and fourth totals in the totals column to get the second element in column 1. determined as in ANOVA. The procedure for performing a full factorial analysis will be discussed here. certain computational shortcuts can be applied when only two levels of each variable are used.
C0 t4 A0. Thus. B1.C0 t1 A1.C1 t7 A1. Factorial analysis example. B1.C1 t6 A0.4 Example The following are the results of a hypothetical experiment to determine if mix time.C1 t8 summation 1 t1 + t2 t3 + t4 t5 + t6 t7 + t8 t2 – t1 t4 – t3 t6 – t5 t8 – t7 2 (t1 + t2) + (t3 + t4) (t5 + t6) + (t7 + t8) (t2 – t1) + (t4 – t3) (t6 – t5) + (t8 – t7) (t3 + t4) – (t1 + t2) (t7 + t8) – (t5 + t6) (t4 – t3) – (t2 – t1) (t8 – t7) – (t6 – t5) 3 2:1 + 2:2 2:3 + 2:4 2:5 + 2:6 2:7 + 2:8 2:2 – 2:1 2:4 – 2:3 2:6 – 2:5 2:8 – 2:7 Sum of Squares 3:1/(r2n) 3:1/(r2n) 3:1/(r2n) 3:1/(r2n) 3:1/(r2n) 3:1/(r2n) 3:1/(r2n) 3:1/(r2n) SSE Mean of Squares SS1/DOF SS2/DOF SS3/DOF SS4/DOF SS5/DOF SS6/DOF SS7/DOF SS8/DOF SME F MS1/SME MS2/SME MS3/SME MS4/SME MS5/SME MS6/SME MS7/SME MS8/SME 1 2 3 4 5 6 7 8 To find: 2:1 + 2:2 = (t1 + t2) + (t3 + t4) + (t5 + t6) + (t7 + t8) 2:3 + 2:4 = (t2 – t1) + (t4 – t3) + (t6 – t5) + (t8 – t7) 2:2 – 2:1 = (t5 + t6) + (t7 + t8) – (t1 + t2) + (t3 + t4) 2:4 – 2:3 = (t6 – t5) + (t8 – t7) – (t2 – t1) + (t4 – t3) (4) (5) The sum of squares column is generated by dividing the square of each adjacent element in column 3 by r * 2n. The mean of squares column is generated by dividing each adjacent element in the sum of squares column by its respective DOF. Two levels of each parameter were tested as follows: Effect A B C Parameter mix time mix speed vacuum Low (0) 2 hr 1 rps no vacuum High (1) 3 hr 2 rps 0. B0. B0.C0 t3 A1. B1. (6) (7) A fractional factorial analysis is performed the same way as the full factorial analysis except the analysis is split into fractions of (1/2)p. Totals from Condition Above A0. 6. B0. that effect is significant. mix speed. and mix vacuum affects the burn rate of a propellant. If any F exceeds F . Compare each F to F for n–1. DOF.C0 t2 A0.4. Obtain each F by dividing each mean square by the error mean square. Exp. B0.Table 63. but will be n–1 for the error row. The DOF will be 1 for each effect. B1. the number of experiments will be 2n–p (eight) experiments.C1 t5 A1.2 atm 610 . if a five variable investigation (32 experiments) is split into 1/4.
04 0.Each effect was assigned a high and low level (e.02 0. The high and low levels are designated as 0 and 1. respectively.77 0 0.58 1.50 Rep 2 0.55 1.43 1.00015 0.00667 0 0. 1 rps was assigned as low.000017 0.000017 0.51 0.50 0.000067 0.04 –0.0745 0.48 0.46 0.01 0 –0.47 0.138 The correction term (C) is as follows: (Sum of totals )2 C = .54 0.46 1. 2 rps was assigned as high). Three additional columns are added.46 0.003615 0. Condition A0 B0 C0 A1 B0 C0 A0 B1 C0 A1 B1 C0 A0 B0 C1 A1 B0 C1 A0 B1 C1 A1 B1 C1 Rep 1 0.000417 0.08 –0.48 0.0803 0.06 0. one for the mean square.02 –0.54 0.52 0.000017 0.0745 29.24 –0.46 1.47 0.9) 611 .000017 0. one for the sum of squares.000417 0.55 1. Each experimental condition was repeated three times with the following results: Exp.47 1.52 0.10 0.48 1.00015 0.50 0.000067 0.51 0. The sum of squares treatment (SSTr) is as follows: SSTr = [(Sum of each individual total squared)/Number of effects] – C.89 2.49 0.56 SSR SSE 1 2. (6.51 0.04 0.55 1.02 0.10 3.8) (6.84 6.47 0.95 3. and one for the F value for each effect. Exp.47 1.54 Total 1. Condition A0 B0 C0 A1 B0 C0 A0 B1 C0 A1 B1 C0 A0 B0 C1 A1 B0 C1 A0 B1 C1 A1 B1 C1 Replicates error Total 1.52 0. (Number of effects ) (Number of totals ) The SST is as follows: SST = Sum of each individual replication squared – C.00723 0.06 Sum of Squares 6.02 –0.8616 0.000224 DOF 1 1 1 1 1 1 1 1 2 7 F 0.669 16.03 0.56 The table is repeated with the replication columns deleted and replaced with the application of three columns for the Method of Yates.48 1.55 0.55 1.43 1.50 0.02 3 12.10) (6.02 SMR SME 2 5.04 0 –0.2977 1.58 1.00667 0 0.52 Rep 3 0..00157 Mean of Squares 6.52 0.14 –0.0803 0.g.
and larger numbers of variables can be solved for. The sum of mean replicate (SMR) is as follows: SMR = SSR/DOF.) Note that since no values of F are greater than F for any conditions where two or more effects are 1. A full factorial analysis does not solve for exponential or polynomial effects. with a chosen significance. 6. Confidence analysis can be used with individual points. or any chosen source of variability isolated. then no interactions have a significant effect on burn rate.12) (6. means. Also. 6.2 Application Confidence analysis is used to determine the interval of values that a data point could take. The fractional factorial analysis does not solve for all effects and higher order effects separately. then the interaction of mix time and mix speed would have a significant interaction).5 Advantages Sources of variation can be found.11) F for a 0. interactions between variables can be isolated. or reliability measurements such as mean time between failures. 6.6 Limitations The processes in factor analysis are more timeconsuming than the analysis of variance.6.1 Description Confidence analysis compares sample values.1 This technique is typically performed in phase C or E. 6. regression lines. means.59. with a chosen probability of being within that interval. 612 .5 Confidence/Reliability Determination and Analysis 6.05 confidence is 5. The sum of mean error (SME) is as follows: SME = SSE/DOF. therefore effect C (vacuum) and replication have a significant effect on the burn rate.The sum of squares replication (SSR) is as follows: SSR = [(Sum of vertical replication total squared)/Number of rows] – C. if the fourth line had an F greater than F . The sum of squares error (SSE) is as follows: SSE = SST – SSTr – SSR.4.5. standard deviations.4. (For example. (6.14) (6.5. (The third batch of propellant may have been different for another reason such as contamination.13) (6. random variation isolated. or standard deviations with population standard deviations to obtain a confidence interval.
18) where Sxx = n * Σxi2 − (Σxi ) 2 . the procedures for determining the confidence interval for the population mean.17) (6. the sample standard deviation can be used instead of the population standard deviation.16) is used. the confidence for the equation of the line is: Int = (a + bx o ) ± t /2 * se *(1/ n + n(xo − ms )2 / Sxx )1/2 and for the y value: Int = (a + bx o ) ± t /2 * se *(1 + 1/ n + n(x o − ms )2 / Sxx )1/2 where se 2 2 2 = Sxx *Syy −(Sxy ) = 1/ (n − 2)Σ(yi − (a + bxi)) n(n−2)Sxx (6. The values for either end of the confidence interval is given by the equation: Int = ms ± z(α/2) * sp/n1/2 where Int is the low or high confidence interval value. given a sample mean. For large n. i 613 . sp is the population standard deviation. and Sxy = n * Σxi yi − ( Σx i ) * ( ΣYi ) .Syy = n * Σy2 − (Σyi )2 . sp 1 ± z( /2) / ss / (2* n)1/2 (6. from past experience (or by adjusting the sample standard deviation). given the sample standard deviation. will be discussed here. (1) (2) (3) (4) Choose a confidence (α) level and obtain the α/2 term by dividing the confidence level by 2. Obtain the z(α/2) value by looking up the z value for α/2 in a normal distribution table. the population standard deviation.6.1. and n is the sample size. is determined in the same way as above. For linear regression.16) where ss is the sample standard deviation.15) Int = (6. m s is the sample mean.3 Procedures As described in reference 6. Determine.5. except equation (6. The confidence interval for the population standard deviation.
1.025.207)+0.1 So.1 3.766 .36 = 1.207/(36)1/2 = from 1. before an analysis can be performed. 6.6 Regression Analysis 6.6.1 Description Regression analysis is a form of curve fitting to find a mathematical relationship for a group of data.042 to 1. The safety factor is then calculated using the maximum erosion value and is: CSF = CSF = Min Ins t Erosion+3sp+0. polynomial. A goodness of fit test is often performed to see how well the generated relationship fits the data. The sample data for 18 flights (36 motors) is: Erosion mean Standard deviation 36 1. 6. with a 95 percent confidence.6 Limitations A sample statistic must be known or assumed.4 Example Determine the confidence interval for insulation erosion at a station in the RSRM aft dome to determine if the associated compliance safety factor (CSF) may actually fall below the 1. 614 .5. and exponential.207 in (defined as known s p) n α/2 is (1–0.0 minimum value.5. 6.112 ± 1.96 * 0. in this instance the confidence interval is used to calculate a safety value that can be compared to a performance requirement.182 for erosion. the confidence interval is 1.15).182+3(0. Entering the above values into equation (6. There are typically two types of regression: regression and multiple regression.5.96.112 in 0.5 Advantages This analysis can give a realistic probability of whether or not a process may yield a value which is above or below a requirement. therefore the z(α/2) term is 1.3 The method of least squares is probably the most frequently used method of regression.6. The equation for the method of least squares is obtained by setting the derivative equal to zero of the equation for the sum of the vertical distance from each y value to the mean y value. Typical types of relationships which are assumed for regression include linear (straight line).95)/2 = 0.6. such as the population standard deviation.
20) (6. as described in reference 6. Determine the deviation of each xi and yi value. the use of the least squares method for finding the equation of a line of the form y = a + bx. 615 . If the slope (b) is negative.1.2 Application Regression.6. is typically used for three purposes: (1) (2) To find the mathematical relationship represented by a group of data points.3.7. and will be discussed here (assuming a straightline relationship). There are several methods of regression. The least squares method is a commonly used method of regression. Determine the slope of the trend line by dividing the summation of the multiple of the deviations by the summation of the square of the x deviations (equation (6.19)). Determine the deviation of each generated y value from the mean y. (6. b= (4) Σ(xi − x )(yi − y ) Σ(xi − x )2 (6. then a decreasing trend may be indicated.19) Determine the y intercept by subtracting the product of the slope and the mean x value from the mean y value (equation (6. is as follows: (1) (2) (3) Determine the mean of the xi values ( x ) and yi values ( y ). The R 2 indicates the percent variation in the dependent variable that can be explained by the independent variable. 6.6.20)). a = y – (b) x .21) The intercept and slope are used in equation (6.6. Regression analysis is best applied in phase D but may also be applied in phase E. The explanatory power can be determined by R 2 as follows: (1) (2) Determine y values for each x value using the line generated above. To determine if the magnitude of a measurement is increasing or decreasing with time or event.3 Procedures As described in reference 6.19) for a line representing the straightline relationship. Multiple regression will be discussed in section 6.
09 8.05 –3. (7. squares.1 (dx)(dy) 3.6).4).3.65 4.25 –2.0196 0.50 yg 4.25 12.0121 0.05 3.25 2.84 dyg2 0. No significant relationship is indicated for this example.3).56 4.0101.15 0.25 20.5). The equation for the line is y = 0.25 82.533.45 1.25 12. (2.0196 0.09 0.68 4.4).59 4.533.0303.49 0.25 0. and (10. (5.5). the mean y value is 4.62 4. (4.5).25 6.15 –1.69 0.5).6. Using these data.0303(x) + 4.09 1.78 4. 6.35 2. dy = yi – y .55 –0. (3.0817 summation where dx = xi – x .25 0.6).0025 0.71 4.4).25 6. R2 = 0. R2 = Σ(gen yi − y )2 Σ(yi − y )2 (6.49 0. Line generated with least squares method. and dyg = yg – y .0004 0. the slope (b) is 0.81 4.09 0.21)). The following table shows summations. assume the set of ordered pairs (1.49 1.5 (dy)2 0.69 2. 10 9 8 7 6 5 4 3 2 1 0 0 1 2 3 4 5 6 generated line 7 8 9 10 Figure 61.4 Example As adapted from reference 6.25 2.(3) Obtain the R2 value by dividing the sum of the square of the generated y deviations by the sum of the square of the actual y deviations (equation (6. and the y intercept (a) is 4.0064 0. (9.0025 0. yg = generated points for each x. (6. 616 .25 2.0121 0.0064 0.89 0. Figure 61 shows the points and the generated line for this data.5.0001 0. (8.15 –1.22) A good relationship is indicated by an R2 value close to 1. the mean x value is 5.7.75 4. and products that go into the equations above: x 1 2 3 4 5 6 7 8 9 10 55 y 4 5 6 3 5 5 4 6 4 5 47 (dx)2 20.
. and b2. The surface can be a plane. To optimize independent variables for maximum or minimum results. the equations for two independent variables are: ∑y = nb0 + b1 * ∑x 1 + b2 * ∑x2. integer data.. There are two typically used methods for response surface analysis—multiple regression and factorial experimentation. ∑(x2 * y) = b0 * ∑x2 + b1 * ∑(x1 * x2) + b2 * ∑x22.7.7.6.2 Application Response surface analysis is typically used for the following purposes: (1) (2) To find the mathematical relationship represented by a group of data points.. 6. ∑(x1 * y) = b0 * ∑x1 + b1 * ∑x12 + b2 * ∑(x1 * x2).6. + b nxn.g.. This methodology is best performed in phase D or E. or it can be a more complex surface.3.5 Advantages A mathematical relationship can be determined.7. by hand or computer. (6. b1.7 Response Surface Methodology 6.. e.6 Limitations If the data are discrete. After setting the derivative of the equation for the sum of the vertical distances or ∑ (yi – (a + b1x1 + b2x2 + b3x3 +.4.23) 617 .1 6. using polynomial relationships. Factorial experimentation is discussed in section 6. much like regression is a method for curve fitting. will be shown here. using two independent variables and straightline relationships. + b nxn))2 to zero. assuming a straightline relationship. The basic form of the equation for a plane surface is y = a + b1x1 + b2x2 + b3x3 +. These equations are solved simultaneously for b0 . 6. 6.3 Procedures As described in reference 6.1 Description Response surface methodology is a method for surface fitting. when the relationship is not obvious by inspection.6. the actual line generated will only approximate the actual relationship. the least squares method of multiple regression. This equation is minimized.6.
520 = b0 1. the numbers are substituted by assuming a smaller whole number for each original number.020 377.151 = 9b0 + b1 1.116 19. Solving the simultaneous equations (6. 3.840 19.080 57 1.400 19. and 140° for 1. Therefore the equation for modulus of elasticity for the sample propellant is y = 383. 618 .880 48.400 19. propellant was aged at 100°. a set of numbers is coded.000 14.23).200 1.151 x12 10.700 41.800 42.680 6.6117 * x2. 120°.240 48.070 4. x1x2.845 The equations for finding the constants are as follows: From equation (6.400 19.000 14.840 x1y 36.600 10. and 12 mo. the constants are b0 = 383.840 + b2 543 3. 6.520 x2y 360 352 347 1.7.580 35.3. The columns for x12.840 19.188 4.000 + b2 6. 080 57 37.284 4.000 42.080 132.7520 132. x1 100 120 140 100 120 140 100 120 140 1.000 48.600 132.Often.548 2.25 * x1 – 0. 6.4 Example In the following hypothetical example. b1 = –0. and b2 = –0. Mean modulus of elasticity measurements are given for three propellantaging temperatures and times.000 14.845 = b0 57 + b1 6.840 543 b0 = 9 1. x22.151 1. This practice makes solving for the coefficients much easier with very little cost in accuracy. as adapted from reference 6.300 35.845 6.600 10. if the numbers are equally spaced.25.6117.98 – 0.080 + b1 132. and x2y and the bottom row of summations are derived from the first two columns. For example.23).840 543 b1 and b2 are calculated in the same manner.080 x2 1 1 1 6 6 6 12 12 12 57 y 360 352 347 358 350 345 347 349 343 3. x1y.98.440 1. 000 6.000 x22 1 1 1 36 36 36 144 144 144 543 x1x2 100 120 140 600 720 840 1.840 57 6.000 6.100 2.080 + b2 57 377.
7. by hand or computer.6. the actual line generated will only approximate the actual relationship.6 Limitations If the data are discrete (e. when the relationship is not obvious by inspection.7. 6. 619 .g. integer data)..5 Advantages A mathematical relationship can be determined.
W.C.5A.3 Miller. Prentice Hall.E. NASA–STD–8070. J. Inc. D. 620 . 1990.REFERENCES 6.. “Trend Analysis Techniques.” October 1990.” John Wiley and Sons. NJ 07632. Hines.: “Probability and Statistics for Engineers.W.2 6..” Second edition.1 6. 1977. I. and Montgomery. Inc.: “Probability and Statistics in Engineering and Management Science. Englewood Cliffs. and Freund.
The cause and effect technique relates identified problems to their causes. A tool to assess an operation against other operations is the benchmarking technique which is discussed in section 7. These tools are quality loss function. Group consensus techniques are often applied to solve problems. and 7. A methodology for improving quality by looking at the production process is the evolutionary operation technique. Design of experiments varies all possible combinations of factors and levels in an attempt to obtain the optimum settings for a desired output. These techniques are discussed in sections 7. as well as their magnitudes. in all areas of an organization. and it is discussed in section 7. flowchart analysis and work flow analysis (WFA). This technique is used to solve problems before the production phase begins and thus assists in the design of competitive products. discussed in section 7. Finally. 71 . quality. design of experiments (DOE).10.4. A methodology for collecting data quickly and easily in a simplified manner is the checklist technique.2. SPC. and how well the product stacks up against the competition as far as meeting the appropriate engineering standards. priorities are given to the possible solutions as they relate to the identified problems. 7. schedule and growth. SPC. This tool is discussed in section 7. pictorially represents the steps of a process thus making it easier to eliminate nonvalued steps of the process. and evolutionary operation (EVOP). Improvement is addressed toward such areas as cost. The force field analysis is discussed in section 7. Quality loss function. Three tools that attempt to improve the quality program are the cost of quality.5. TOTAL QUALITY MANAGEMENT TOOLS This section describes several TQM tools available to the system engineer analyst. is a process improvement tool that helps identify problems quickly and accurately. the WFA.3. TQM is an ongoing effort that demands commitment and discipline. discussed in section 7.7.15. This technique is discussed in section 7.6. TQM is applied to continuously improve performance at all levels of operation. Three such tools are brainstorming.9. discussed in section 7. A summary of the advantages and limitations of each tool or methodology discussed in this section is presented in table 71. and this tool is discussed in section 7. market share.7. the product can be benchmarked against the competition in the areas of how well the product stacks up against the competition as far as handling the identified problems. By using a chart known as the house of quality. This technique is discussed in section 7. is a method of determining “loss to society” when a product is not at the mean but is still within specification limits. This approach is discussed in section 7. Delphi.11. examines the work process for possible improvements in performance and the quality of work life. A methodology that is applied early in a design process is the quality function deployment (QFD) technique which is discussed in section 7. Concurrent engineering is more of an approach to quality management than a technique and it is an interaction of disciplines during the design but before production. discussed in section 7. The cost of quality tracks a quality program and attempts to identify ways to improve the program.1. Another tool that might apply to the group consensus technique is the force field analysis. and nominal group technique (NGT). respectively. using all available human and capital resources. Also. The final four tools that are discussed in this section are applied to improve a process. The flowchart analysis.16.12.8. that effect the results of a proposed solution or change in process. This methodology counts the positive and negative forces.13.14.
3 (1) Advantages Helps meet customer requirements. (2) Measurement for measurement's sake is an easy paradigm to fall into. (2) Significant additional time. 72 . Helps establish goals and priorities. Reduces costs in the designtodevelopment life cycle. (3) Parameters must be interpolated from within the tested data set rather than extrapolated beyond it. it is often difficult to “frontload” large tasks. SPC detects problems but can poses no solutions.1 (1) (2) (3) (4) (5) Cause and effect diagrams 7. (1) The technique itself can be expensive.4 Design of experiments 7. Limitations (1) Must be continuous in order to keep up with the latest industry changes. (2) Cost of quality 7. the institutional knowledge of the organization becomes very difficult to capture or employ in the design decisions.5 (1) Helps to reveal and explain the more significant costs. Preknowledge of interaction significance is required to support appropriate DOE technique selection. (2) The DOE technique is often performed without a “verification experiment” in which the predicted “optimized” parameters are tested for performance (in agreement with the predictions). Useful in analyzing statistical process control (SPC) problems. this technique is subject to misuse in this regard. (3) If design is pursued by projectized teams. Helps determine true measures of productivity. reduces costs. is required at the front end of a program to perform the coordinated planning. (2) Determining industry “best practices” is difficult and often subjective enough to be biased by the reviewing company’s “wants” rather than the reviewing company’s customer’s wants. While time and money are saved overall within the effort. and desensitizes production variables. (1) The degree of success of this technique depends upon the degree of cooperation between the multifunctional team members. Enables quality analysis groups to thoroughly examine all possible causes or categories. and associated funding.Table 71. The technique optimizes product and process design. addition a mistake is often made by taking the “best” experiment’s parameters as an optimized set rather than an interpolated set.2 (1) (2) Concurrent engineering 7. (2) Activities and processes that need improvement can be prioritized. Shortens and makes more efficient the designtodevelopment life cycle by employing the interactions of functional disciplines by a crossfunctional team. In. stabilizes production processes. TQM tools and methodologies—Continued Tool or Methodology Benchmarking Section 7. Helps to attain and maintain a competitive position. Arriving at a group consensus is timeconsuming. the results generally do not include parameter interactions. thus making its goals of saving/eliminating costs unachievable. (1) The performance of the analysis is time consuming and. Helps identify and maintain awareness of industry best practices. (3) Helps to reveal and explain the hidden costs of a product or service.
be collected. (2) Helps ensure quality products and processes by detecting and solving problems early. (5) The technique is proactive. and startup costs are reduced.8 7. not reactive. EVOP is slow. (3) Engineering changes. (1) Arriving at a group consensus is time consuming. TQM tools and methodologies—Continued Tool or Methodology Evolutionary operation Section 7. a short time.6 Advantages Limitations (1) The cost is very low. (3) The tool is simple and relatively straightforward. dominate the discussion. and is highly subjective. Very effective in producing many new ideas/solutions in (1) Assembling the group participants is difficult/timeconsuming. so it can be run continuously. will meet the least resistance. (6) Prevents problems from “falling through the crack. (1) The tool is quick and easy to use. (1) Useful in eliminating personality clashes. lowercost. (8) Easy to learn.11 7. (1) Assembling the group participants is difficult/timeconsuming.9 Nominal group technique Force field analysis Quality function deployment 7. (2) Useful when powerful personalities are likely to (2) Assembling the group participants is difficult/timeconsuming. (2) This technique increases a plant’s capacity and thus profits will increase. among The technique is time consuming in arriving at a consensus on the many. products easier and (2) The technique is not easy to perform. (1) Helps organizations design more competitive.10 7. Time must be taken to assemble a group to decide what data should (2) Checklists help to minimize errors and confusion. higherquality. values (weights) of the forces. The technique takes advantage of the ideas of a group to (1) The technique only proposes a solution but does not determine arrive at a quick consensus.Table 71. one.” (7) The technique is costeffective.12 73 . (2) Divergence in weighting factors is common. so progress is slow.7 Checklists Delphi technique 7. Useful in determining which proposed solution. Brainstorming 7. quicker. (3) Inputs from experts unavailable for a single meeting are included. design cycle. (4) Voice of the customer is heard. (2) The technique is limited by the ability of the group to achieve consensus.
13 Advantages (1) Evaluates loss at earliest stage of product/process development. Work flow analysis 7. The technique increases productivity and improves working conditions. (3) SPC helps identify problems quickly and accurately. (2) The observed operation may not be fully representative of a “typical” process that would occur without scrutiny.16 (1) The technique requires cooperation between employees and management to be successful. (2) Useful results obtained quickly and at low cost. (1) This technique determines the cause of variation based on a statistical analysis of the problem. (1) Allows the examination of and understanding of relationships in a process.Table 71. (2) Provides a stepbystep picture that creates a common understanding about how the elements of the process fit together. Limitations (1) It may be difficult to convince manufacturers to apply the technique. (2) It is often difficult to characterize the loss function.14 Flowchart analysis 7. TQM tools and methodologies—Continued. Statistical process control 7. 74 .15 The development process is time consuming. SPC detects problems but poses no solutions. (3) Comparing a flowchart to actual process activities highlights areas where policies are unclear or are being violated. (2) The technique improves process performance. Tool or Methodology Quality loss function Section 7.
or benchmark against noncompetitors that are considered industry leaders. i. Determine the criteria to benchmark. Once the strengths and weaknesses of competing products are known. 7. If the benchmarking process is performed once and forgotten. Repeat the benchmarking technique.e. (3) (4) (5) (6) (7) 75 . Collect data on the processes or products that are being benchmarked.1 Benchmarking 7. Assess the results of all the changes.3 Procedures As adapted from reference 7.3.1. or process. The manufacturers of those competing products are probably using the same benchmarking technique to evaluate the competitors for their product. as necessary. is a technique used to assess how an organization. This technique can be applied when it is desirable to know the strengths and weaknesses of an organization’s own operation. The technique must be continuously applied in order to be effective because practices constantly change (continuous improvement) affecting strategy. Choose the particular characteristics of the operation or product to benchmark. as described in reference 7. Benchmarking helps improve a process by recognizing priorities and goals. the company can attempt to differentiate their capabilities in the marketplace. Benchmarking can be applied to identify the strengths for products that directly compete with the organization’s specific product under consideration. the basic elements of benchmarking include the following: (1) (2) Decide which process(es) or product(s) to benchmark. then the operation may become inefficient by not keeping up with the latest industry best practices. competitors. an organization can also incorporate the strengths of their competitors that exist in certain areas.2 Application The benchmarking technique is typically performed in phase E but may also be performed in phase A or B.1. prepare an action plan. Analyze the data. or even noncompetitors that may be recognized as being superior. benchmark internally against established guidelines.1. in order to stay uptodate with the applicable operation.1 Description Benchmarking.7. 7. By accomplishing this analysis..1. and implement the plan. benchmark against competitors. These strengths and weaknesses can then be compared to internal guidelines to evaluate the organization’s conformance to those guidelines. is performing against internal guidelines.
Determining industry “best practices” is often difficult and subjective. Comparative benchmarking. Benchmarking helps identify and maintain awareness of industry’s best practices.4 Example The following illustration.6 Benchmarking helps meet customer requirements. The reviewing company may well bias their results based on company “wants” rather than customer “wants.1.3. This company should also look at those competitors rated lower on the scale and identify their weaknesses and ensure that those weaknesses do not exist in their operation. Benchmarking helps to attain and maintain a competitive position.7. Benchmarking helps determine true measures of productivity. Limitations (1) (2) The benchmarking process must be continuous in order to keep up with the latest industry changes.1.1. adapted from reference 7.5 Advantages (1) (2) (3) (4) (5) 7. Better Organization Process 10 9 8 7 6 5 4 3 2 1 Competitor 2 Competitor 3 Competitor 1 Competitor 5 Competitor 4 Worse Figure 71. This illustration reveals that this company needs to look closely at the operations of competitors 2 and 3 and consider implementing into their process any strengths that are discovered. Benchmarking helps establish goals and priorities. shows an example of comparative benchmarking between one company’s process and five competitors on a scale of 1 (worse) to 10 (better). 7.” 76 .
Each member of the group lists the causes in order of significance. SPC problems.14) problems. then display the categories on the diagram.2. 7. Votes are counted and a final list is written.2 Cause and Effect Diagrams (Also Known as Fishbone Diagrams or Ishakawa Diagrams) 7. The bones of the fish show the organized causes. Once all participants reach a similar level of understanding about an issue. Any event past. or future and its causal factors. The group then assigns priorities to the causes and action plans are developed. as described in reference 7. an expansion of ideas can then be examined. Once all causes are identified.” where the effect is found in a box to the right which is the head of the fish. 7. This effect represents the “problem” that is being investigated.7. thinking is stimulated.3 Procedures A cause and effect diagram.3. The cause and effect diagram technique is best applied in phase E but may also be applied in phase A or B.1 Description The cause and effect diagram. The effects and causes can be expressed in words or data. These causes are then studied and the causes that affect these causes are identified. thoughts are organized. A desired future outcome and its related factors. The technique is also useful in planning activities and brainstorming. The group then prioritizes the causes by multivoting. cause and effect diagrams are used to examine many different topics which include the following: (1) (2) (3) The relationships between a known problem and the factors that might affect it. Possible causes are listed. is developed in the following manner: (1) Define the effect as clearly as is possible and place it at the head of the fish. if necessary. list all categories. graphically represents the relationships between a problem (effect) and its possible causes. present. The diagram is basically a controlled way of gathering and using suggestions through group consensus.3. 7. commonly referred to as “fish. and discussions are begun.2. The problem is stated in terms acceptable to the group. (sec. The group brainstorms the causes and lists them in no particular order. The development process is started in a group session led by a trained facilitator. As data are collected. as adapted from reference 7. the effect can be redefined. (2) (3) (4) 77 .3. This will continue until no new causes are thought of by the group. The cause and effect diagram is useful in examining processes such as SPC.2 Application As adapted from reference 7. These discussions bring out many possible viewpoints on the subject. Cause and effect diagrams are developed in a form. When a cause and effect diagram is constructed.2.
The brainstorming effort for this problem is covered in section 7. As categories and causes are included on the diagram.e.4.2. CHANGES SCHEDULE Poor Rushed Outdated Under Tracking Equipment Staffed Late Start Excessive Poor Meetings Notification Not Clear Cost Rules No Second Shift Requires Try It Now. SPC detects a problem but can pose no solution. The cause and effect diagram is useful in analyzing SPC problems.4 Examples Example 1: Assume the problem is design rework (fig. Finally.7). thinking may be stimulated and new causes may be identified. Teams are then formed to research and report on preventive (i. 78 .5 Advantages (1) (2) The cause and effect diagram enables quality analysis groups to thoroughly examine all possible causes or categories. This is a technique where each person lists the causes in order of significance.2. 72). (6) (7) 7. When complete. the least prioritized causes will be listed on the diagram as the small bones. 7. Backtracking Change Later GRAPHICS Doesn't Know System Responsibility Not Defined DESIGN REWORK Lacking Skills Working Outside Discipline SKILL Conflict ByPassed Discontented Out of Sequence Loanee Not Clear INTERFACES SPECS Figure 72. In conclusion.. The group fills in the probable root causes through “brainstorming” ideas (sec. Example 2: Figure 73 illustrates the resulting cause and effect diagram after the brainstorming session on identifying problems in receiving telephone messages. a team has put their thoughts in writing and arrived at a consensus. proactive) measures.7. the group prioritizes the causes using multivoting. 7. The next highest prioritized causes will be listed on the diagram as the medium bones. Design rework cause and effect diagram. Teams are formed to research and report on preventive measures.(5) The highest prioritized causes are listed on the diagram as the big bones. Votes are counted and a final list is written.
HUMAN ERROR Message light not turned on Employee forgets to sign out Forget to undo call forward Call recipient does not deliver message message misplaced distribution Poor contrast HARDWARE ENVIRONMENT Employee does not see light Peak Activity Too small Not enough phones Number of calls Not enough trunk lines Lack of equipment to take long detailed/technical messages Wrong message taken . 79 .incomplete message rude caller Employee fails to look at light Untimely delivery of message Criticality of message not identified (no guidelines) Inability to take long detailed message Info not available to call recipient long detailed messages Recipient doesn't know how to obtain info employee whereabouts No guidelines for message takers call pickup Phone System Options No guideline for phone system setup call transfer call coverage Procedures No standard guidelines for message takers distractions Lack of interactiveautomated directions to caller Messages are notdelivered in a timely manner Inadequate message delivery system Employee Unaware ofmessage Message Taker responsibilities No feedback of message delivered No identified point of contact No method to reachemployee notaccessible while offi METHOD TRAINING Figure 73. Cause and effect diagram on receiving telephone messages.
7.1 Description Concurrent engineering is the interaction of technical disciplines during the design phase to produce a robust design prior to production.3 Procedures The basic elements involved in applying concurrent engineering include the following. Traditionally.5).7 Bibliography Kume. safety. 7. 7.7. it generally will save time and money.1 The approach attempts to link and integrate. quality.” The Association for Overseas Technical Scholarships.3. 7.3. 7. all elements of the product life cycle from conception through disposal. as described in reference 7.7. One method of achieving this approach is by forming multifunction teams consisting of engineers from several departments. and value analysis to extend the traditional design approach.3: (1) (2) (3) Establish multifunction teams which include members from design.12) and DOE (sec.7. robust design. This process is more of an engineering approach to quality management than a technique. 7. each department will follow the complete process simultaneously rather than one department examining the design and then passing it on to the next department and so on. H.2 Application Because the concurrent engineering approach is used to address the product and process simultaneously early in the design phase. from the outset. computeraided design. the team will establish design goals as well as perform tradeoff analyses using such tools as QFD (sec. Use such techniques as DOE. marketing. 710 . Select and use design parameters that will help identify and reduce variability in the production process.5 7. Concurrent engineering. focuses on both the product and the process simultaneously.3. This technique is typically performed in phase C but may also be performed in phase B. QFD. manufacturing.4 The concurrent engineering approach has been known for many years although its use is just receiving widespread application in the United States. etc. and producibility do not review an element until after the design has been completed. quality. This way.3.2. support.2.3 Concurrent Engineering 7.: “Statistical Methods for Quality Improvement. as adapted from reference 7. Through this technique. 1985. group technology.6 Limitations The development of the cause and effect diagram can be timeconsuming in order to arrive at a group consensus.
QFD (Section 7.14) Verification INPUT Customer Needs Logistics Manufacturing Training Deployment Operations Support Disposal Figure 74. DOE (Section 7. NGT (Section 7.4 Example Figure 74 illustrates an example of how concurrent engineering is applied. Development Maintainability Reliability Safety MULTIFUNCTIONAL TEAMS Sample Techniques Cause and Effect Diagram (Section 7. SPC (Section 7.5) Brainstorming (Section 7.3.7.10) PRODUCT Force Field Analysis (Section 7.12) CYCLE Statistical Process Control. all phases of a product’s life cycle are simultaneously examined.9) BALANCED Nominal Group Technique.2) Design of Experiments.11) LIFE Quality Function Deployment. Concurrent engineering example. By using multifunctional teams. 711 . thus making the design process more efficient in terms of both cost and schedule.7) OUTPUT Delphi Technique (Section 7.
This technique can identify the