Course Notes - Estadistica Industrial II

Industrial Statistics II
An Introduction to Statistical Methods for

Reliability Data
Preliminaries
• Presentation focus: concepts, ideas, examples, models, data

analysis, interpretation.
W. Q. Meeker, L. A. Escobar, and F. G. Pascual • Structure of the Course Notes.

Iowa State University, Louisiana State University, and Wash-
• Other written material: Meeker, Escobar, and Pascual draft
ington State University.
textbook (to be published by Wiley in 2019).
Copyright 2018 W. Q. Meeker, L. A. Escobar, and F. G. Pas- • Feel free to ask questions.
cual.
• Please provide feedback and in particular provide correc-
Based on the authors’ book Statistical Methods for Relia- tions and suggestions.
bility Data, Second Edition, John Wiley & Sons Inc., to be
published in 2019.
June 11, 2018

17h 44min 0-1 0-2
Other Arrangements for this Course
Tentative Content • The course grade will be based on three group report
projects plus an individual closed book final exam.
• Part 1: Introduction to reliability data and non-parametric We need to divide the class in eight groups of five stu-
estimation of reliability. Chapters 1, 2, and 3. dents each. I need to have the list of the eight groups by
Wednesday June 13. You are responsible for making up
the groups.
• Part 2: Log-location-scale models and probability plotting.
Chapters 4 and 6.
• The final exam will be administered on July 10 in class.
• Part 3: Parametric likelihood fitting concepts and applica-

• I will host office hours on Tuesday and Thursday each week,
tions. Chapters 7 and 8.
12:15pm - 1:15pm. I will announce later the place for the
office hours.
• Part 4: Analysis of data using explanatory variables or aux-
iliary information. Chapters 15 and 16. • If needed you might contact my by email at
luescobarr@unal.edu.co. I, however, will entertain only
simple questions through the email.
0-3 0-4
Chapter 1
Reliability Concepts and an Introduction to Reliability Chapter 1

Data Reliability Concepts and Reliability Data
Topics discussed in this chapter are:

• Basic reliability concepts.
• Examples of studies that resulted in different kinds of failure-

W. Q. Meeker, L. A. Escobar, and F. G. Pascual time data and a general strategy and issues in failure-time
data analysis.
Copyright 2018 W. Q. Meeker, L. A. Escobar, and F. G. Pas-

cual. • Failure-time data based on inspections.
Based on the authors’ book Statistical Methods for Relia-

bility Data, Second Edition, John Wiley & Sons Inc., to be • Failure-time data with explanatory variables, degradation
published in 2019. reliability data, and recurrent-event reliability data.
June 7, 2018
8h 14min 1-1 1-2
Quality and Reliability
Pressures on manufacturers to produce high quality, low cost

products. Pressures due to:
Chapter 1
• Rapid advances in technology.
Segment 1
• Development of highly sophisticated products.
Basic Reliability Concepts
• Intense global competition.
• Increasing customer expectations.
1-3 1-4
Reasons for Collecting Reliability Data
• Assess characteristics of materials.

Definitions of Reliability
• Predict product reliability in design stage.

• Technical: Reliability is the probability that a system, ve-
hicle, machine, device, and so on, will perform its intended
• Assess the effect of a proposed design change.
function under encountered operating conditions, for a
specified period of time.
• Compare two or more different materials, designs, or man-
ufacturers.
• Succinct:
Quality over time (Condra 1993). • Assess product reliability in field.
Failure avoidance.
• Check the veracity of an advertising claim.
• Predict product warranty costs.

1-5 1-6
Types of Reliability Data
• Failure-time data (Chapters 2–18), also known as
Life data.
Chapter 1 Survival data.
Segment 2 Event-time data.
Introduction to Failure-Time Data • Degradation data.
Destructive degradation data (Chapter 19).

Repeated measures degradation data (Chapter 20).
• Recurrent event data (Chapter 21).
1-7 1-8
Ball Bearing Data

Ball Bearing Failure Data Event Plot
Data from fatigue endurance tests for deep–groove ball bear- (Lawless 2003)
ings from four major bearing companies (Lieblein and Ze-
len 1956 and Lawless 1982).
Lieblein and Zelen Ball Bearing Failure Data
The data: Millions of revolutions to failure for each of Row

1
n = 23 bearings before fatigue failure. 2
3
4
5
6
Main objectives of the study: Determine best values of 7
8
9
the parameters in equation relating fatigue to load. 10
11
12
13
14
Motivation: 15
16
17
18
19
• Fatigue affects service life of ball bearings. 20
21
22
23
• Disagreement in manufacturing industries on the appro-

0 50 100 150
priate model parameter values to use to characterize the
Megacycles
failure-time distribution.
1-9 1 - 10
Ball Bearing Failure Data

(Lieblein and Zelen 1956. Data from Lawless 2003) Distinguishing Features of Failure-Time Data
and Reliability Data Analysis
• Data are typically censored (bounds on observations).

8
• Positive responses (e.g., time or cycles to failure) need to

be modeled. Commonly used distributions include the expo-
6
nential, lognormal, and Weibull distributions. The normal

Frequency
distribution is not common.

4
• Model parameters not of primary interest (instead, failure

rates, quantiles, and failure or survival probabilities).
2
• Extrapolation often required (e.g., want proportion failing

0
0 50 100 150 200 at three years for a product in the field only six months).
Millions of Cycles
1 - 11 1 - 12
Issues in Failure-Time Data Analysis
IC Data
• Causes of failure and degradation leading to failure.

Integrated circuit failure times in hours. When the test ended
at 1,370 hours, 4,128 units were still running (Meeker 1987)
• Environmental effects on reliability.
0.10 0.10 0.15 0.60 0.80 0.80
1.20 2.5 3.0 4.0 4.0 6.0
• Definition of time scale. 10.0 10.0 12.5 20. 20. 43.
43. 48. 48. 54. 74. 84.
94. 168. 263. 593.
• Definitions of time origin and time of failure.
1 - 13 1 - 14
Event Plot of the Integrated Circuit Life Test Data

Where 28 Out of 4156 Units Failed in the 1370 Hour
Test at 80◦C and 80% RH (Meeker 1987)
IC Data
n = 4,128 IC’s tested for 1,370 hours at 80◦C and 80% Integrated Circuit Failure Data After 1370 Hours
relative humidity, there were 28 failures (Meeker 1987). Count
Questions: Row
1 2
2
3
4 2
5
• Proportion of units that will fail in 105 hours (proportion of 6
7
defective units)? 8 2
9
10 2
11
12 2
13 2
• Hazard function 14 2
15
16
17
For the product population? 18
19
20
21
For the defective subpopulation? 22 4128
0 200 400 600 800 1000 1200 1400

Hours
1 - 15 1 - 16
Failure Pattern in the Shock Absorber Data

Shock Absorber Failure Data (O’Connor 1985)
First reported in O’Connor (1985).
Shock Absorber Data (Two Failure Modes)

• Failure times, in number of kilometers of use, of vehicle
shock absorbers. Row
1 1
2
3
4
5 2
6
7
8
• Two failure modes, denoted by Mode 1 and Mode 2. 9
10
11
12
13 1
14
15 2
16
17
18
Unit
• One might be interested in the distribution of time to fail- 19

20
1
1
21
22
ure for 23
24
25
26
27 2
28
Mode 1 (e.g., after Mode 2 has been eliminated). 29
30
31 2
32 1
33
34 1
35
Mode 2 (e.g., after Mode 2 has been eliminated). 36
37
38
1
A shock absorber with both Mode 1 and Mode 2 active. 0 5000 10000 15000 20000 25000 30000
Kilometers
1 - 17 1 - 18
Strategy for
Data Analysis, Modeling, and Inference Computer Software
• Model-free graphical data analysis. Capabilities for reliability data analysis can be found in:
• Model fitting (parametric, including possible use of prior • R/RSplida

information). • JMP
• Inference: estimation or prediction (including statistical in- • MINITAB
tervals to reflect statistical uncertainty).
• SAS
• Graphical display of model-fitting results.
• WinSMITH
• Graphical and analytical diagnostics and assessment of as-
sumptions. • Weibull++/ALTA
• Sensitivity analysis. With JMP one can accomplish most of the data analyses
needed for this course.
• Conclusions.
1 - 19 1 - 20
Heat Exchanger Tube Crack Data
Data from heat exchangers from power plants.
• 100 tubes in each exchanger.
• Each tube inspected at the end of each year for cracks.

Chapter 1 Cracked tubes are plugged, reducing efficiency.
Segment 3 • Data from 3 plants with same design. Each plant entered
into service at different dates.
Failure-time Data Based on Inspections
Cracked tubes at the end of each year,
Exchanger Year 1 Year 2 Year 3

1 1 2 2
2 2 3 no inspection
3 1 no inspection no inspection
1 - 21 1 - 22
Heat Exchanger Tube Crack Inspection Data Heat Exchanger Tube Crack Inspection Data
in Calendar Time in Operating Time
100 new 1 failure 2 failures 2 failures 100 new 1 failure 2 failures 2 failures
tubes tubes
Plant 1 95 Plant 1 95
.. .. .. .. .. ..
. . . . . .
100 new 100 new
tubes 2 failures 3 failures 2 failures 3 failures
tubes
Plant 2 95 Plant 2 95
.. .. .. ..
. . . .
100 new
1 failure 100 new
tubes tubes 1 failure
Plant 3 99 Plant 3
99
.. ..
. .
1981 1982 1983 Year 1 Year 2 Year 3
1 - 23 1 - 24
Turbine Wheel Data (Nelson 1982)
Current-Status Data
Each of n = 432 wheels was inspected once to determine if

Current-status data arise when
it had started to crack or not. Wheels had different ages at
inspection (Nelson 1982).
• A population (or sample from a population) of units is in-
spected within a short period of time to learn the extent of
In this case, some important objectives are:
a recently discovered unanticipated serious failure mode.
Excessive wear found in an aircraft jackscrew triggered • Need to schedule regular inspections.
inspection of similar aircraft.
A crack detected in an aircraft engine rotating compo- • Estimate the distribution of time to crack.
nent triggered inspection of all such components in ser-
vice.
• Is the reliability of the wheels getting worse as the wheels age?
An increasing hazard function would require replacement

• Rockets in a weapons stockpile either operated correctly or
of the wheels by some age when the risk of cracking is
not when fired.
too high.
1 - 25 1 - 26
Event Plot
Turbine Wheel Inspection Data (Nelson 1982)
Turbine Wheel Data
Summary at Time of Study
Turbine Wheel Crack Initiation Data
Count
100-hours of # Cracked # Not Cracked Row
Exposure Left Censored Right Censored 1 39
2 ? | 4
3 49
4 0 39 4 ? | 2
5 31
10 4 49 6 ? | 7
14 2 31 7
8 ? |
66
5
18 7 66 9 25
10 ? | 9
22 5 25 11 30
12 ? | 9
26 9 30 13 33
14 ? | 6
30 9 33 15 7
34 6 7 16
17
? | 22
12
38 22 12 18
19
? | 21
19
42 21 19 20 ? | 21
21 15
46 21 15
0 10 20 30 40 50
Hundreds of Hours
1 - 27 1 - 28
Scatter Plot of Low-Cycle Fatigue Life Versus

Pseudo-Stress for Specimens of a Nickel-Base
Superalloy (Nelson 1990)
250
Chapter 1
200
Segment 4
Failure-Time Data with Explanatory Variables 150

Kilocycles
Degradation Reliability Data

Recurrent-event Reliability Data 100
50
80 90 100 110 120 130 140 150

Pseudo−stress (ksi)
1 - 29 1 - 30
Scatter Plot of Printed Circuit Board Accelerated Life
Test Data (Meeker & LuValle 1995)
Degradation Data
• Provides information on progression toward failure.

48/70 censored
5000
11/68 censored
x
x x
x • Becoming more common in certain areas of component re-
xxx
x x liability where few or no failures expected in life tests.
x
x xx
Hours to Failure
x
x
x
x xxx 0/70 censored
500
x xx
x xxx xx
xxx xxx 0/70 censored • Important connections with physical mechanisms of failure
xx xx
x
xx xxx
x xx xx and failure-time reliability models.
xx xxx
xx
xx x
xxx
50
xxx
xxx x
Destructive degradation data (Chapter 19).
xx
x x
x
Repeated measures degradation data (Chapter 20).
10
40 50 60 70 80 90
Relative Humidity
1 - 31 1 - 32
Fatigue Crack Size as a Function of Number of Cycles Change in Resistance Over Time of Carbon-Film
(Bogdanoff & Kozin 1985) Resistors (Shiomi & Yanagisawa 1979)
1.8 3
10.0
9
7
Critical crack size 1 12
1.6 5.0 173 Degrees C
Crack Length (inches)
Percent Increase
14
1.4
18
133 Degrees C
21
1.0
1.2
83 Degrees C
0.5
1.0 Censoring time ->
0.0 0.02 0.04 0.06 0.08 0.10 0.12 0 2000 4000 6000 8000 10000
Millions of Cycles Hours
1 - 33 1 - 34
Recurrent-Event Data
Valve Seat Replacements
Repairable Systems and Nonrepairable Units
• Failure-time or degradation data on components or non-

repairable units.
422
420
Laboratory tests on materials or components. 418
416
414
412
Data on components or replaceable subsystems from sys- 410
409
System ID
408
tem tests or monitoring. 406
404
402
Time to first failure of a system. 400
398
396
394
392
390
• Repairable Systems result in recurrent-event data that 331
329
327
describe the failure trends and patterns of a system (Chap- 251
ter 21).
0 200 400 600
Engine Age in Days
1 - 35 1 - 36
Chapter 2
Chapter 2
Models, Censoring, and Likelihood for
Failure-Time Data Models, Censoring, and Likelihood for
Failure-Time Data

• Describe models for continuous failure-time processes.
• Describe some reliability metrics.

W. Q. Meeker, L. A. Escobar, and F. G. Pascual • Describe models that we will use for the discrete data from
Iowa State University, Louisiana State University, and Wash- these continuous failure-time processes.
• Describe common censoring mechanisms that restrict our
Copyright 2018 W. Q. Meeker, L. A. Escobar, and F. G. Pas- ability to observe all of the failure times that might occur
cual.
in a reliability study.
• Explain the principles of likelihood, how it is related to the
published in 2019. probability of the observed data, and how likelihood ideas
can be used to make inferences from reliability data.
June 7, 2018
8h 14min 2-1 2-2
Typical Failure-time cdf, pdf, hf, and sf

F (t) = 1 − exp(−t1.7); f (t) = 1.7 × t0.7 × exp(−t1.7)
S(t) = exp(−t1.7); h(t) = 1.7 × t0.7
Cumulative Distribution Function Probability Density Function
1 0.8
0.6
Chapter 2 F (t ) 0.5 f (t ) 0.4
0.2
Segment 1 0
0.0 0.5 1.0 1.5 2.0 2.5 0.0 0.5 1.0 1.5 2.0 2.5
t t
Failure-time Models and Metrics
Survival Function Hazard Function
1
2.00
1.00
0.50
S (t ) 0.5 h (t )
0.20
0.10
0.05
0
0.0 0.5 1.0 1.5 2.0 2.5 0.0 0.5 1.0 1.5 2.0 2.5
t t
2-3 2-4
Models for Continuous Failure-Time Processes

Hazard Function
T is a nonnegative, continuous random variable describing The hazard function is defined by

the failure-time process. The distribution of T can be char- Pr(t < T ≤ t + Δt | T > t)
acterized by any of the following functions: h(t) = lim
Δt→0 Δt
• The cumulative distribution function (cdf):
f (t)
F (t) = Pr(T ≤ t). = .
1 − F (t)
Example, F (t) = 1 − exp(−t1.7).
• The probability density function (pdf): f (t) = dF (t)/dt.

Notes:
Example, f (t) = 1.7 × t0.7 × exp(−t1.7).

• F (t) = 1 − exp[− 0t h(x)dx].
• Survival function (or reliability function):
∞ • h(t) describes propensity of failure in the next small interval
S(t) = Pr(T > t) = 1 − F (t) = f (x)dx. of time given survival to time t
t
Example, S(t) = exp(−t1.7). h(t) × Δt ≈ Pr(t < T ≤ t + Δt | T > t).
• The hazard function: h(t) = f (t)/[1 − F (t)].
• Some reliability engineers think of modeling in terms of h(t).
Example, h(t) = 1.7 × t0.7.
2-5 2-6
Bathtub Curve Hazard Function
Cumulative Hazard and Average Hazard
• Cumulative hazard function:

t
H(t) = h(x) dx.
0

Failures caused by Note that, F (t) = 1 − exp[−H(t)] = 1 − exp − 0t h(x) dx .
h (t ) Infant mortality external forces Wearout failures
• The average hazard rate in the interval (t1, t2]:

t2
t1 h(u)du H(t2) − H(t1)
AHR(t1, t2) = = .
t2 − t1 t2 − t1
2-7 2-8
Plots Showing that Distribution Quantiles

the Quantile Function is the Inverse of the cdf
• The p quantile of F is the smallest time tp such that
Probability Density Function Cumulative Distribution Function Pr(T ≤ tp) = F (tp) ≥ p, where 0 < p < 1.
0.8 1.0 t0.20 is the time by which 20% of the population will fail.
For, F (t) = 1 − exp(−t1.7), p = F (tp) gives tp = [− log(1 −
0.8 p)]1/1.7 and t0.2 = [− log(1 − 0.2)]1/1.7 = 0.414.
0.6
• When F (t) is strictly increasing there is a unique value tp
0.6 that satisfies F (tp) = p, and we write
f (t ) 0.4 F (t )
tp = F −1(p).
0.4
0.2 • When F (t) is constant over some intervals, there can be

0.2
more than one solution t to the equation F (t) ≥ p. Taking
Shaded Area = 0.2
tp equal to the smallest t value satisfying F (t) ≥ p is the
0.0 0.0 standard convention.
0.0 0.5 1.0 1.5 2.0 2.5 0.0 0.5 1.0 1.5 2.0 2.5
t tt • tp is also known as B100p (e.g., t0.10 is also known as B10).
2-9 2 - 10
Distribution of Lifetime Conditional on Survival to t0
• Consider the conditional (left-truncated) distribution

F (t) − F (t0)
Chapter 2 Pr(T ≤ t|T > t0) = , t ≥ t0
1 − F (t0)
Segment 2 with corresponding pdf
f (t)
Distribution of Remaining Life , t ≥ t0 .
1 − F (t0)
• This distribution is useful to describe the age at which a

unit will fail, conditional on survival to age t0.
2 - 11 2 - 12
Distribution of Remaining Life
Distribution of Remaining Life
• Certain applications require consideration of the distribution
of remaining life:
Prediction of future field failures for a population of units

that have been in service.
Assessment expected remaining life of particular units.
Assessment of used-asset value.
g(u | t 0)
• Consider a unit with failure time T that has survived until

t0. The distribution G(u|t0) of remaining life U = T − t0 is f (t )
the probability that the unit will fail within the next u time
units, given that T > t0. That is:
G(u|t0) = Pr(U ≤ u|T > t0) = Pr(T − t0 ≤ u|T > t0)
F (u + t0) − F (t0)
= Pr(T ≤ u + t0|T > t0) = , u ≥ 0.
1 − F (t0) 0 t0 t
and the corresponding pdf is u
0
f (u + t0)
g(u|t0) = , u ≥ 0.
1 − F (t0)
2 - 13 2 - 14
Example: Distribution of Remaining Life of an

Automobile Transmission
• Suppose the lifetime (in thousands of miles) of an automo-

bile transmission has a cdf

F (t) = 1 − exp −(t/140)2 , t ≥ 0.
• The cdf of the remaining life of an automobile transmission

that has been in service for t = 95 thousand miles is Chapter 2
F (u + 95) − F (95)
G(u|95) = Pr(U ≤ u|95) = Segment 3
1 − F (95)

95 2 u + 95 2
= 1 − exp − , u ≥ 0. A Nonparametric Model for Failure-Time Data
140 140
• The probability that the automobile transmission will survive

the next u = 45 thousand miles is

95 2 45 + 95 2
Pr(U ≥ 45|95) = 1 − Pr(U ≤ u|95) = exp −
140 140

95 2
= exp − 1 = 0.583
140
2 - 15 2 - 16
Graphical Interpretation of the π’s
Partitioning of Time into Non-Overlapping Intervals

1.0
F (t 4)
π4
F (t 3)
0.8
π1 π2 π3 ... πm−1 πm πm+1 π3
... ... F (t 2)
0.6
t0 = 0 t1 t2 ... t m−1 tm t m+1 = ∞

π2
0.4
F (t 1)
Times need not be equally spaced. 0.2
π1
0.0
0.0 0.5 1.0 1.5 2.0
t0 t1 t2 t3 t4
2 - 17 2 - 18
Models for Discrete Data
from a Continuous Time Processes
Models for Discrete Data

All data are discrete! Partition (0, ∞) into m + 1 intervals
from a Continuous Time Processes–Continued
depending on inspection times and roundoff as follows:
(t0, t1], (t1, t2], . . . , (tm−1, tm], (tm, tm+1) Following from the previous result,
where t0 = 0 and tm+1 = ∞. The last interval is of infinite m+1

S(ti−1) = Pr(T > ti−1) = πj
length.
j=i
πi = piS(ti−1)
Define,
i

πi = Pr(ti−1 < T ≤ ti) = F (ti) − F (ti−1) S(ti) = 1 − pj , i = 1, . . . , m + 1
j=1
F (ti) − F (ti−1)
pi = Pr(ti−1 < T ≤ ti | T > ti−1) = Either π = (π1, . . . , πm+1) or p = (p1, . . . , pm) can be used as
1 − F (ti−1)
“nonparametric parameters.”
Because the πi values are multinomial probabilities, πi ≥ 0

and m+1j=1 πj = 1. Also, pm+1 = 1 but the only restriction

on p1, . . . , pm is 0 ≤ pi ≤ 1
2 - 19 2 - 20
Probabilities for the Multinomial Failure Time Model

Computed from F (t) = 1 − exp(−t1.7)
ti F (ti) S(ti) πi pi 1 − pi Chapter 2

0.0 0.000 1.000
0.5 0.265 0.735 0.265 0.265 0.735 Segment 4
1.0 0.632 0.368 0.367 0.500 0.500
1.5 0.864 0.136 0.231 0.629 0.371 Censoring and Likelihood for Failure-Time Data
2.0 0.961 0.0388 0.0976 0.715 0.285
∞ 1.000 0.000 0.0388 1.000 0.000
1.000
2 - 21 2 - 22
Examples of Censoring Mechanisms
Censoring restricts our ability to observe T . Some sources of

censoring are: Likelihood (Probability of the Data)
as a Unifying Concept
• Fixed time to end a life test (lower bound on T for unfailed
units).
• Likelihood provides a general and versatile method of esti-
• Inspections times (upper and lower bounds on T ). mation.
• Staggered entry of units into service leads to multiple cen- • Model/Parameters combinations with relatively large like-
soring. lihood are plausible.
• Multiple failure modes (also known as competing risks) and • Allows for censored, interval, and truncated data.
other random censoring mechanisms resulting in multiple
right censoring): • Theory is simple in regular models.
Independent (simple). • Theory more complicated in non-regular models

(but concepts are similar).
Non independent (complicated).
• Simple modeling and analysis requires non-informative cen-

soring assumption.
2 - 23 2 - 24
Likelihood (Probability of the Data)
Contributions for Different Kinds of Censoring

Pr(Data) = n i=1 Pr(datai ) = Pr(data1 ) × · · · × Pr(datan )
Determining the Likelihood (Probability of the Data)
The form of the likelihood will depend on:

0.8 Left censoring
• Question/focus of study. Interval censoring
0.6
• Assumed model.
f (t ) 0.4
• Measurement system (form of available data).
0.2 Right censoring
• Identifiability/parameterization.
0.0
0.0 0.5 1.0 1.5 2.0 2.5
t
2 - 25 2 - 26
Likelihood Contributions for Different Kinds of

Censoring with F (t) = 1 − exp(−t1.7) Likelihood for Life Table Data
• For a life table the data are: the number of failures (di),
• Interval-censored observations: right censored (ri), and left censored (i) units on each of
t
i the nonoverlapping interval (ti−1, ti], i = 1, . . . , m+1, t0 = 0.
Li = f (t) dt = F (ti) − F (ti−1).
ti−1
• The likelihood (probability of the data) for a single obser-
If a unit is still operating at t = 1.0 but has failed at t = 1.5
vation, datai, in (ti−1, ti] is
inspection, Li = F (1.5) − F (1.0) = 0.231.
Li(π ; datai) = F (ti; π ) − F (ti−1; π ).
• Left-censored observations:
t
Li =
i
f (t) dt = F (ti) − F (0) = F (ti). • Assuming that the censoring is at ti (note that F (t) depends
0 on either π or p):
If a failure is found at the first inspection time t = 0.5,
Type of Characteristic Number Likelihood of
Li = F (0.5) = 0.265. Censoring of Cases Responses Li(π ; datai)
• Right-censored observations: Left at ti T ≤ ti i [F (ti)]i
∞

Li = f (t) dt = F (∞) − F (ti) = 1 − F (ti). Interval ti−1 < T ≤ ti di F (ti) − F (ti−1) di
ti
If a unit has not failed by the last inspection at t = 2, Right at ti T > ti ri [1 − F (ti)]ri
Li = 1 − F (2) = 0.0388.
2 - 27 2 - 28
Likelihood: Probability of the Failure-time Data Likelihood for Arbitrary Censored Failure-Time Data
• The total likelihood, or joint probability of the DATA, for • In general, observation i consists of an interval (tL i , ti ], i =
n independent observations is (note that F (t) depends on 1, . . . , n (tL
i < ti ) that contains the time event T for individ-
either π or p): ual i.
n
The intervals (tL
i , ti ] may overlap and their union may not
L(π ; DATA) = C Li(π ; datai)
cover the entire time line (0, ∞). In general tLi = ti−1 .
i=1
m+1
• Assuming that the censoring is at ti
=C [F (ti)]i F (ti) − F (ti−1) di [1 − F (ti)]ri
i=1 Type of Characteristic Likelihood of a single

where n = m+1 Censoring Response Li(π ; datai)
j=1 dj + rj + j and C is a constant depend-
ing on the sampling inspection scheme but not on π (so we Left at ti T ≤ ti F (ti)
can take C = 1). Interval i < T ≤ ti
tL F (ti) − F (tL
i)
• Want to find π so that L(π ; DATA) is large. Right at ti T > ti 1 − F (ti)
2 - 29 2 - 30
Likelihood for General Failure-Time Data
• The total likelihood for the DATA with n independent ob-

servations is
n

L(π ; DATA) = Li(π ; datai).
i=1
• Some of the observations may have multiple occurrences

(e.g., identical observations). Let (tL
j , tj ], j = 1, . . . , k be
the distinct intervals in the DATA and let wj be the number
of (frequency or weight for) observations in (tL j , tj ]. Then
k
w
j
L(π ; DATA) = Lj (π ; dataj ) .
j=1
• In this case the nonparametric parameters in π correspond

to probabilities of a partition of (0, ∞) determined by the
data.
2 - 31
Chapter 3 Chapter 3
Nonparametric Estimation
Nonparametric Estimation
of a Failure-Time Distribution
• Use of the binomial distribution to estimate F (t) from in-
terval and singly right censored failure-time data, without
assumptions about the form of F (t). This is called non-
parametric estimation.
• Methods for computing confidence intervals for F (t) with

W. Q. Meeker, L. A. Escobar, and F. G. Pascual singly right censored failure-time data.
ington State University. • Nonparametric estimation with multiply-censored and interval-
censored failure-time data
cual. • The Kaplan-Meier nonparametric estimator for multiply-
censored failure-time data and exact failure times.
bility Data, Second Edition, John Wiley & Sons Inc., to be • Nonparametric simultaneous confidence bands for F (t)
published in 2019.
• Nonparametric estimation of F (ti) with current-status data
June 7, 2018 or arbitrary censoring
8h 14min 3-1 3-2
Data for Plant 1 of the

Heat Exchanger Tube Crack Data
Cracked tubes
Chapter 3
100 tubes at start Year 1 Year 2 Year 3 Uncracked tubes
Segment 1 Plant 1 1 2 2 95
Nonparametric Estimation with Singly-Censored Unconditional

π3 π4
Failure Probability π1 π2
Failure-Time Data
Likelihood: L(π ) = C × [π1]1 × [π2]2 × [π3]2 × [π4]95
4

πi = 1.
i=1
3-3 3-4
A Nonparametric Estimator of F (ti) Based on Nonparametric Estimate for Plant 1

Binomial Methods for Interval Singly-Censored Data from the Heat Exchanger Tube Crack Data
Consider the nonparametric estimator of F (ti) for data sit-
uations illustrated by the Heat Exchanger Tube Crack from
Plant 1:
• The data are:

0.20
n : sample size
di : # of failures (deaths) in interval i
0.15
for ti, i = 1, 2, 3.
Proportion Failing
• Application of simple binomial methods gives

i 0.10
# of failures up to time ti j=1 dj
F (ti) = =
n n

0.05
F (t ) 1 − F (t )
i i
seF = .
n
0.0
• For Plant 1 (n = 100, d1 = 1, d2 = 2, d3 = 2),
0.0 0.5 1.0 1.5 2.0 2.5 3.0
F (1) = 1/100, F (2) = 3/100, F (3) = 5/100. Years
3-5 3-6
Comments on
the Nonparametric Estimate of F (ti) from
Interval-Censored and Singly-Right-Censored
Failure-Time Data
Chapter 3
• F (t) is defined only at the upper ends of the intervals
Segment 2
(ti−1, ti].
Nonparametric Confidence Intervals
for a Failure-Time Distribution
• F (ti) is the ML estimator of F (ti).
based on Singly-Censored Failure-Time Data
• The increase in F at each value of ti is
F (ti) − F (ti−1) = di/n.
3-7 3-8
Features of Confidence Intervals
• The level of confidence expresses one’s confidence (not

Confidence Intervals
probability) that a specific interval contains the quantity of
interest.
A point estimate can be misleading. It is important to
• The actual coverage probability is the probability that the
quantify statistical uncertainty in point estimates.
procedure will result in an interval containing the quantity
• Confidence intervals are used to quantify statistical uncer- of interest.
tainty in point estimates due to sampling error arising from
• A confidence interval is approximate if the specified level
limited data.
of confidence is not equal to the actual coverage probability.
• Confidence intervals do not quantify deviations arising from
• With censored data most confidence interval procedures are
incorrectly specified model assumptions.
approximate.
• Some confidence intervals procedures are conservative (cov-

erage probability is larger than the confidence level).
3-9 3 - 10
Pointwise Wald-Approximate (Normal Approximate) Pointwise Jeffreys

Confidence Interval for F (ti) Confidence Interval for F (ti)
• For a specified value of ti, an approximate 100(1 − α)% • A 100(1 − α)% approximate confidence interval for F (ti) is
confidence interval for F (ti) is
F (ti) = qbeta(α/2; nF + 0.5, n − nF + 0.5)
[F (ti), F̃ (ti)] = F (ti) ∓ z(1−α/2)seF . ⎧ ⎫−1

where z(1−α/2) is the 1−α/2 quantile of the standard normal
⎪
⎨ (n − nF + 0.5)F(1−α/2;2n−2nF+1,2nF+1) ⎪
⎬
= 1+
⎪
⎩ nF + 0.5 ⎪
⎭
distribution and seF = F (ti) 1 − F (ti) /n is an estimate of
the standard error of F (ti). F̃ (ti) = qbeta(1 − α/2; nF + 0.5, n − nF + 0.5)
⎧ ⎫−1
⎪
⎨ ⎪
⎬
n − nF + 0.5
= 1+
• This confidence interval is based on ⎪
⎩ (nF + 0.5)F(1−α/2;2nF+1,2n−2nF+1) ⎪
⎭
F (ti) − F (ti)
ZF = ∼
˙ NORM(0, 1). where F = F (ti) and F(1−α/2;ν1,ν2) is the 100(1−α/2) quan-
seF
tile of the F distribution with (ν1, ν2) degrees of freedom.
• This method is computationally simple but is not recom- • This method is based on a Bayesian interval using a Jef-
mended because of its poor coverage probability properties freys prior distribution and has coverage probability prop-
(see Section 6.2.6 of Meeker, Hahn, and Escobar, 2017). erties that are much better than the Wald method (see
Instead, use the Jeffreys or the Conservative method. Section 6.2.6 of Meeker, Hahn, and Escobar, 2017).
3 - 11 3 - 12
Pointwise Conservative Plant 1 Heat Exchanger Tube Crack Nonparametric
Confidence Interval for F (ti) Estimate with Conservative Pointwise 95% Confidence
Intervals Based on Binomial Theory
• A 100(1 − α)% conservative confidence interval for F (ti)

based on binomial sampling (see Chapter 6 of Meeker, Hahn,
and Escobar, 2017) is
0.20
F (ti) = qbeta(α/2; nF , n − nF + 1)
⎧ ⎫−1
⎪
⎨(n − nF + 1)F(1−α/2;2n−2nF+2,2nF) ⎪
⎬
= 1+
0.15
⎪
⎩ nF ⎪
⎭
Proportion Failing
F̃ (ti) = qbeta(1 − α/2; nF + 1, n − nF )
⎧ ⎫−1 0.10
⎪
⎨ ⎪
⎬
n − nF
= 1+
⎪
⎩ (nF + 1)F(1−α/2;2nF+2,2n−2nF) ⎪
⎭
0.05
where F = F (ti) and F(1−α/2;ν1,ν2) is the 100(1−α/2) quan-
tile of the F distribution with (ν1, ν2) degrees of freedom.
0.0
• This confidence interval is conservative in that the actual 0.0 0.5 1.0 1.5 2.0 2.5 3.0
coverage probability is greater than or equal to 1 − α. Years
3 - 13 3 - 14
Summary of the Nonparametric Estimate of F (ti) for

Plant 1 from the Heat Exchanger Tube Crack Data
Year ti di F (ti )
se F
Pointwise Confidence Interval Integrated Circuit (IC) Failure Times in Hours
F (ti ) F̃ (ti ) Data from Meeker (1987)

(0 − 1] 1 1 0.01 0.00995
95% Confidence Intervals for F (1)
Wald [−0.0095, 0.0295]
Jeffreys [0.0011, 0.0458]
0.10 0.10 0.15 0.60 0.80 0.80
Conservative [0.0003, 0.0545] 1.20 2.50 3.00 4.00 4.00 6.00
10.00 10.00 12.50 20.00 20.00 43.00
(1 − 2] 2 2 0.03 0.01706 43.00 48.00 48.00 54.00 74.00 84.00
95% Confidence Intervals for F (2) 94.00 168.00 263.00 593.00
Wald [−0.0034, 0.0634]
Jeffreys [0.0085, 0.0779]
Conservative [0.0062, 0.0852] When the test ended at 1370 hours, there were 28
observed failures and 4128 unfailed units.
(2 − 3] 3 2 0.05 0.02179
95% Confidence Intervals for F (3) Note: Ties in the data. Reason?
Wald [0.0073, 0.0927]
Jeffreys [0.0193, 0.1061]
Conservative [0.0164, 0.1128]
3 - 15 3 - 16
Event Plot
Nonparametric Estimator of F (t) Integrated Circuit Life Test Data
Based on Binomial Methods for Exact Failures
and Singly Right-Censored Data
When the number of inspections increases the width of the

Integrated Circuit Failure Data After 1370 Hours
time intervals (ti−1, ti] approaches zero and the failure times
Count
are exact. Row
1 2
• For the integrated circuit life test data, we have: n = 2

3
4 2
4156 with 28 exact failures in 1370 hours. 5
6
7
For any particular te, 0 < te ≤ 1370, simple binomial meth- 8
9
2
10 2
ods give 11
12 2
# of failures up to time te 13 2
F (te) = 14 2
n 15
16
F (te ) 1 − F (te ) 17
18
seF = . 19
20
n 21
22 4128
• Confidence interval methods for F (te) are the same as the

methods described for interval data. 0 200 400 600 800 1000 1200 1400
Hours
3 - 17 3 - 18
Nonparametric Estimate with Pointwise
Wald-Approximate 95% Confidence Intervals
Based on Zlogit(F) for the IC Data
Comments on the Nonparametric Estimate of F (t)
• F (t) is defined for all t in the interval (0, tc] where tc is the
0.012 single censoring time.
0.010 • F (t) is the ML estimator of F (t).
• The estimate F (t) is a step-up function with a step of size

Proportion Failing
0.008
1/n at each exact failure time (unless there are ties).
0.006
Sometimes the step size is an integer multiple of 1/n be-
0.004 cause there are ties in the failure times.
0.002
• When there is no censoring, F (t) is the well known empiri-
cal cdf.
0.0
0 200 400 600 800 1000 1200 1400

Hours
3 - 19 3 - 20
Pooling of the Heat Exchanger Tube Crack Data
Plant 1
100 99 97
1 2 2 95
Plant 2
100 98
2 3 95
Chapter 3 100
Plant 3 1 99
Segment 3
Nonparametric Estimation with Multiply-Censored and

99 95 Uncracked tubes
Interval-Censored Failure-Time Data
All Plants
300 197 97
4 5 2 95
Failure Probability π1 π2 π3 π4
4 5 2 95 95 99
Likelihood: L( π
__) = C [π 1] [π 2] [ π 3] [π4] [π 3+ π 4] [π 2+ π 3+ π 4]
3 - 21 3 - 22
A Nonparametric Estimator of F (ti) Based on Interval

Data and Multiple Right Censoring
The pooled data heat exchanger tube crack data are multiply
censored and the simple binomial method to estimate F (ti)
cannot be used. Summary of the Nonparametric Estimate of F (ti) for
the Pooled Heat Exchanger Tube Crack Data
Consider the more general nonparametric estimator of F (ti)
based on the probability model introduced in Chapter 2: Year ti ni di ri pi 1 − pi )
S(t F (ti)
i
F (ti) = 1 − S(t
)
i (0 − 1] 1 300 4 99 4/300 296/300 0.9867 0.0133
i
dj (1 − 2] 2 197 5 95 5/197 192/197 0.9616 0.0384
where )=
S(t 1 − pj with pj =
i
j=1 nj (2 − 3] 3 97 2 95 2/97 95/97 0.9418 0.0582
n : size of the risk set size at time 0

di : # of failures (deaths) in interval i
i−1
i−1

ni = n − dj − rj , the risk set just after ti−1
j=0 j=0
ri : # of right censored observations at ti
3 - 23 3 - 24
Nonparametric Estimate
for the Heat Exchanger Tube Crack Data
Approximate Variance of F (ti)
i
• Recall, F (ti) = 1 − S(t
) and S(t
i
)=
i j=1 1 − p
j .
0.20
• Then Var F (ti) = Var S(t
) .
i
) is
• A Taylor series first-order approximation of S(t
0.15 i
Proportion Failing
i
∂S
) ≈ S(t ) +
S(t qj − qj
i i
0.10 j=1 ∂qj qj
where qj = 1 − pj .
0.05 • Then it follows that

i
pj
) ≈ S 2(t )
Var S(t .
i i
0.0 j=1 nj (1 − pj )
0.0 0.5 1.0 1.5 2.0 2.5 3.0

Years
3 - 25 3 - 26
Pointwise Wald-Approximate Confidence

Interval for F (ti) Based on the Logit Transformation
Estimating the Standard Error of F (ti)
• Better confidence intervals can be obtained by using the
• Using the variance formula, one gets logit transformation (logit(p) = log[p/(1 − p)]) and basing
the confidence intervals on
i
pj
! F (t ) = Var
Var i
! S(t
) =S
i
2(t )
i logit[F (ti)] − logit[F (ti)]
n j (1 − pj ) Zlogit(F) = ∼
˙ NORM(0, 1).
j=1 selogit(F)
which is known as Greenwood’s formula.
• A pointwise Wald-Approximate 100(1 − α)% confidence in-
• An estimate of the standard error of F (ti) is
terval for logit[F (ti)] is
" #
i pj
! ) = logit(F ) ∓ z
seF = Var F (ti) = S(ti) . logit(F ), logit(F (1−α/2)selogit(F )
i n
j=1 j (1 − pj )
= logit(F ) ∓ z(1−α/2)seF /[F (1 − F )]
because selogit(F) = seF /[F (1 − F )].
3 - 27 3 - 28
Pointwise Wald-Approximate Confidence

Pointwise Wald-Approximate Confidence Intervals for
Interval for F (ti)-Based on the Logit Transformation
the Heat Exchanger Tube Crack Data
• The confidence interval for F (ti) is obtained from the inter-

• Computation of standard errors
val for logit(F ) and using the inverse logit transformation
i
pj
1 ! F (t ) = S
2(t )
logit−1(v) = Var i i
1 + exp(−v) j=1 n j (1 − pj )

• Then 0.0133
" # ! F (t ) = (0.9867)2
Var = 0.0000438
1
[F (ti), F̃ (ti)] = logit−1 logit(F ) ∓ z(1−α/2)selogit(F) 300(0.9867)
√
1 seF(t ) = 0.0000438 = 0.00662
= " # 1
1 + exp −logit(F ) ∓ z(1−α/2)selogit(F)

0.0133 0.0254
F F ! F (t ) = (0.9616)2
Var 2 + = 0.0001639
= , 300(0.9867) 197(0.9746)
F + (1 − F ) × w F + (1 − F )/w
√
where w = exp{z(1−α/2)seF /[F (1 − F )]}. seF(t ) = 0.0001639 = 0.0128
2
• The endpoints F (ti) and F̃ (ti) will always lie between 0 and 1.

3 - 29 3 - 30
Pointwise Wald-Approximate Confidence Intervals for Summary of the Nonparametric Pointwise Confidence
the Heat Exchanger Tube Crack Data Intervals for F (ti) Based on the Heat Exchanger Tube
Crack Data
Computation of approximate 95% confidence intervals:
Year ti F (ti )
se F
Pointwise Confidence Intervals
√
• For F (1) with F (t1) = 0.0133, se
=
F (t1 )
0.0000438 = 0.00662 (0 − 1] 1 0.0133 0.00662
Based on: ZF = [F (t1) − F (t1)]/se
∼ ˙ NORM(0, 1). 95% Confidence Intervals for F (1)
F
Based on Zlogit(F) ∼
˙ NORM(0, 1) [0.0050, 0.0350]
[F (t1), F̃ (t1)] = 0.0133 ∓ 1.96(0.00662) = [0.0003, 0.0263]. Based on ZF ∼˙ NORM(0, 1) [0.0003, 0.0263]

Based on: Zlogit(F) = [logit[F (t1)]−logit[F (t1)]/se
) ∼˙ NORM(0, 1).
logit(F (1 − 2] 2 .0384 .0128
" #
0.0133 0.0133 95% Confidence Intervals for F (2)
[F (t1), F̃ (t1)] = ,
0.0133 + (1 − 0.0133) × w 0.0133 + (1 − 0.0133)/w Based on Zlogit(F) ∼
˙ NORM(0, 1) [0.0198, 0.0730]
= [0.0050, 0.0350]. Based on ZF ∼˙ NORM(0, 1) [0.0133, 0.0635]
where w = exp{1.96(0.00662)/[0.0133(1 − 0.0133)]} = 2.687816.
√ (2 − 3] 3 .0582 .0187
• For F (2) with F (t2) = 0.0384, se
=
F (t )
0.0001639 = 0.0128
2
95% Confidence Intervals for F (3)
Based on: ZF, [F (t2), F̃ (t2)] = [0.0133, 0.0635]. Based on Zlogit(F) ∼
˙ NORM(0, 1) [0.0307, 0.1076]
Based on ZF ∼˙ NORM(0, 1) [0.0216, 0.0949]
Based on: Zlogit(F) , [F (t2), F̃ (t2)] = [0.0198, 0.0730].

3 - 31 3 - 32
Nonparametric Estimate with Pointwise

Wald-Approximate
95% Confidence Intervals Based on Zlogit(F) for the
Heat Exchanger Tube Crack Failure-Time Distribution
0.20
Chapter 3
Segment 4
0.15
Proportion Failing
The Kaplan-Meier Nonparametric Estimator for

Multiply-Censored Failure-Time Data
0.10
and Exact Failure Times
0.05
0.0
0.0 0.5 1.0 1.5 2.0 2.5 3.0

Years
3 - 33 3 - 34
Event Plot
Shock Absorber Field-Failure Data
Shock Absorber Failure Data Failure Mode Information Ignored
First reported in O’Connor (1985). ShockAbsorber Data (Both Failure Modes)
Row
1
2
• Failure times, in number of kilometers of use, of vehicle 3
4
5
6
shock absorbers. 7
8
9
10
11
• Two failure modes, denoted by M1 and M2.
12
13
14
15
16
17
• One might be interested in the distribution of time to fail- 18
19
20
21
22
ure for mode M1, mode M2, or in the overall failure-time 23
24
25
distribution of the part. 26
27
28
29
30
31
Here we do not differentiate between modes M1 and M2. 32
33
34
35
We will estimate the distribution of time to failure by either 36
37
38
mode M1 or M2.
0 5000 10000 15000 20000 25000 30000
Kilometers
3 - 35 3 - 36
Nonparametric Estimation of F (t) with Exact Failures Kaplan-Meier Estimates for the Shock Absorber Data
Using the Kaplan-Meier Estimator up to 12,220 km
In the limit, as the number of inspections increases and the

width of the inspection intervals approaches zero, we get Conditional Unconditional
the product-limit or Kaplan-Meier estimator: tj (km) nj dj rj pj 1 − pj
S(t F (tj )
j)
• Failures are concentrated in a small number of intervals of 6,700 38 1 0 1/38 37/38 0.97368 0.02632
infinitesimal length. 6,950 37 0 1
7,820 36 0 1
• F (t) will be constant over all intervals that have no failures. 8,790 35 0 1
9,120 34 1 0 1/34 33/34 0.94505 0.05495
• F (t) is a step function with jumps at each reported failure 9,660 33 0 1
time. 9,820 32 0 1
11,310 31 0 1
Confidence intervals are computed in a manner similar to 11,690 30 0 1
that used with interval censoring. 11,850 29 0 1
11,880 28 0 1
Note: The binomial estimator for exact failures and singly 12,140 27 0 1
right censored data is a special case of the Kaplan-Meier 12,200 26 1 0 1/26 25/26 0.90870 0.09130
estimator. ... ... ... ... ... ... ... ...
3 - 37 3 - 38
Kaplan-Meier Estimate with Pointwise 95%

Confidence Intervals Based on Zlogit(F)
for the Shock Absorber Data
1.0
Chapter 3
0.8
Segment 5
Proportion Failing
0.6
Nonparametric Simultaneous Confidence Bands
for F (t)
0.4
0.2
0.0
0 5000 10000 15000 20000 25000

Kilometers
3 - 39 3 - 40
Kaplan-Meier Estimate with Pointwise 95%

Confidence Intervals Based on Zlogit(F)
Need for Nonparametric Simultaneous

Confidence Bands for F (t)
1.0
• Pointwise confidence intervals for F (t) are useful for

0.8
quantifying the statistical uncertainty in F (t) at one par-
Proportion Failing
ticular value of t.
0.6
• Simultaneous confidence bands for F (t) are necessary 0.4

to quantify the the statistical uncertainty over a range of
values of t. 0.2
0.0
0 5000 10000 15000 20000 25000

Kilometers
3 - 41 3 - 42
Kaplan-Meier Estimate with Simultaneous 95% Nonparametric Simultaneous Confidence Bands
Confidence Intervals Based on Zlogit(F) for F (t)
Simultaneous approximate 100(1 − α)% confidence bands for
F (t) can be obtained from

F (t), F̃ (t) = F (t) ∓ e(a,b,1−α/2)seF (t) for all t ∈ [tL(a), tU (b)]

1.0 where tL(a) and tU (b)] are complicated functions of the cen-
soring pattern in the data.
0.8
Proportion Failing
Comments:
0.6
• These particular simultaneous confidence bands are known
as “equal precision” or “EP” bands.
0.4
• The approximate factors e(a,b,1−α/2) can be computed from
0.2 a large-sample approximation given in Nair (1984).
• e(a,b,1−α/2) is the same for all values of t.

0.0
• The factors e(a,b,1−α/2) are greater than the corresponding
0 5000 10000 15000 20000 25000
z(1−α/2).
Kilometers
3 - 43 3 - 44
Better Nonparametric Simultaneous Confidence

Factors e(a,b,1−α/2) for Computing the EP
Bands for F (t)
Nonparametric Simultaneous
Approximate Confidence Bands
• The approximate 100(1−α)% simultaneous confidence bands
Limits Confidence Level
F (t), F̃ (t) = F (t) ∓ e(a,b,1−α/2)se
(t) for all t ∈ [tL(a), tU (b)]
a b 0.80 0.90 0.95 0.99 F
are based on the approximate distribution of

0.005 0.999 2.92 3.17 3.41 3.88
⎡ ⎤
0.01 0.999 2.90 3.15 3.39 3.87
0.05 0.999 2.84 3.10 3.34 3.82 max ⎣
F (t) − F (t) ⎦
ZmaxF = .
0.001 0.995 2.92 3.17 3.41 3.88 t ∈ [tL(a), tU (b)] se
0.005 0.995 2.86 3.12 3.36 3.85 F (t)
0.01 0.995 2.84 3.10 3.34 3.83
0.05 0.995 2.76 3.03 3.28 3.77 • It is generally better to compute the simultaneous confi-
0.001 0.99 2.90 3.15 3.39 3.87 dence bands based on the logit transformation of F . This
0.005 0.99 2.84 3.10 3.34 3.83 gives
0.01 0.99 2.81 3.07 3.31 3.81
0.05 0.99 2.73 3.00 3.25 3.75 F (t) F (t)
[F (t), F̃ (t)] = ,
0.001 0.95 2.84 3.10 3.34 3.82 F (t) + [1 − F (t)] × w F (t) + [1 − F (t)]/w
0.005 0.95 2.76 3.03 3.28 3.77
0.01 0.95 2.73 3.00 3.25 3.75 /[F (1 − F )]}.
where w = exp{e(a,b,1−α/2)se F
0.05 0.95 2.62 2.91 3.16 3.68
0.001 0.9 2.80 3.07 3.31 3.80 These are based on the approximate distribution of
0.005 0.9 2.72 3.00 3.25 3.75
0.01 0.9 2.68 2.96 3.21 3.72 max logit[F (t)] − logit[F (t)]
Zmax logit(F) = .
0.05 0.9 2.56 2.85 3.11 3.64 t ∈ [tL (a), tU (b)]
se logit[F (t)]
3 - 45 3 - 46
Nonparametric Estimate with Simultaneous 95% Nonparametric Estimate with Pointwise Wald 95%
Confidence Bands Based on Zmax logit(F) Confidence Intervals Based on Zlogit(F)
for the Heat Exchanger Tube Crack Data for the Heat Exchanger Tube Crack Data
0.20 0.20
0.15 0.15
Proportion Failing
Proportion Failing
0.10 0.10
0.05 0.05
0.0 0.0
0.0 0.5 1.0 1.5 2.0 2.5 3.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0
Years Years
3 - 47 3 - 48
Nonparametric Estimation of F (ti)
with Arbitrary Censoring
• The methods described so far work only for some kinds of

censoring patterns (multiple right censoring, interval cen-
soring with intervals that do not overlap, and some other
very special censoring patterns.)
Chapter 3
Segment 6 • The nonparametric maximum likelihood generalizations pro-

vided by the Peto/Turnbull estimator can be used for
Nonparametric Estimation of F (ti)
with Current-Status Data or Arbitrary Censoring Current-status data (e.g., both left- and right-censored,
overlapping).
Interval censoring with overlapping intervals.
Arbitrary censoring—combinations of the above possibly

with exact failures
Truncated data.
3 - 49 3 - 50
Turbine Wheel Inspection Data Summary

Event Plot
Turbine Wheel Current-Status Data 100-hours of # Cracked # Not Cracked Proportion Cracked
Exposure Left Censored Right Censored Crude Estimate of
ti F (t)
4 0 39 0/39 = 0.000
Turbine Wheel Crack Initiation Data 10 4 49 4/53 = 0.075
Count 14 2 31 2/33 = 0.060
Row 18 7 66 7/73 = 0.096
1 39 22 5 25 5/30 = 0.167
2 ? | 4
3 49
26 9 30 9/39 = 0.231
4 ? | 2 30 9 33 9/42 = 0.214
5 31 34 6 7 6/13 = 0.462
6 ? | 7
7 66 38 22 12 22/34 = 0.647
8 ? | 5 42 21 19 21/40 = 0.525
9 25
10 ? | 9 46 21 15 21/36 = 0.583
11 30
12 ? | 9 Data from Nelson (1982), page 409.
13 33
14 ? | 6
15 7
16 ? | 22
17 12 • The analysts did not know the initiation time for any of the
18 ? | 21
19 19 wheels.
20 ? | 21
21 15
• All they knew about each wheel was its exposure time and
whether a crack had initiated or not. For convenience and
0 10 20 30 40 50
data compression, units were grouped by amount of expo-
Hundreds of Hours
sure time.
3 - 51 3 - 52
Plot of Crude Estimates of the Proportions Failing

Versus Hours of Exposure for the Turbine Wheel
Current-Status Data
Basic Parameters Used in Computing the

Nonparametric ML Estimate of F (t) for the Turbine
Wheel Data
0.8
0 39 4 49 2 31 21 15
0.6
Proportion Failing
π1 π 2 π 3 π 4 π 5 π6 π 7 π 8 π 9 π10 π11 π12

0.4
0 4 10 18 26 34 42 46
0.2
Hundreds of Hours
0.0
0 10 20 30 40 50
Hundreds of Hours
3 - 53 3 - 54
Nonparametric ML Estimate with Pointwise
Nonparametric Estimation of F (t)
Approximate 95% Confidence Intervals for F (ti) Based
with Current-Status Data
on Zlogit(F) for the Turbine Wheel Data
• Basic idea: Write the likelihood (probability of the data)

and maximize to obtain p or π
from which one can compute
F (ti) (Peto 1973). 0.8
• Illustration: The likelihood for the turbine wheel current-

status data is 0.6
Proportion Failing
L(π ) = L(π ; DATA) = C × [π1]0 × [π2 + · · · + π12]39×
×[π1 + π2]4 × [π3 + · · · + π12]49 0.4
×[π1 + · · · + π3]2 × [π4 + · · · + π12]31
...
0.2
×[π1 + · · · + π11]21 × [π12]15

where π12 = 1 − 11 i=1 πi . The values of π1 , . . . , π11 that

maximize L(π ) gives π , the ML estimator of π . Then 0.0

F (ti) = ij=1 π̂j , i = 1, . . . , m. 0 10 20 30 40 50

Hundreds of Hours
3 - 55 3 - 56
Plot of Crude Estimates of the Proportions Failing

Versus Hours of Exposure
for the Turbine Wheel Data
0.8
0.6
Proportion Failing
0.4
0.2
0.0
0 10 20 30 40 50
Hundreds of Hours
3 - 57
Chapter 4 Chapter 4
Location-Scale and Log-Location-Scale Parametric
Location-Scale and Log-Location-Scale Parametric Distributions
Distributions
• Motivation, definition, metrics, and importance of location-
scale and log-location-scale parametric distributions.
• Description and properties of the exponential, normal, log-

normal, smallest extreme value, and Weibull distributions.
W. Q. Meeker, L. A. Escobar, and F. G. Pascual
Iowa State University, Louisiana State University, and Wash- • Description and properties of other location-scale and log-
ington State University. location-scale parametric distributions including the largest
extreme value, Fréchet, logistic, and loglogistic distribu-
Copyright 2018 W. Q. Meeker, L. A. Escobar, and F. G. Pas- tions.
cual.
Based on the authors’ book Statistical Methods for Relia- • Description and properties of the generalized gamma distri-
bility Data, Second Edition, John Wiley & Sons Inc., to be bution.
published in 2019.
• Description and properties of threshold log-location-scale
June 7, 2018 distributions.
8h 15min 4-1 4-2
Motivation for Parametric Models
• Complements nonparametric models.
• Parametric models can be described concisely with just a

Chapter 4
few parameters, instead of having to report an entire curve.
Segment 1
• It is possible to use a parametric model to extrapolate (in
Location-Scale and Log-Location-Scale Parametric
time) to the lower or upper tail of a distribution.
Distributions: Motivation, Definition, Metrics, and
Importance
• Parametric models provide smooth estimates of failure-time
distributions.
In practice it is often useful to compare various parametric

estimates with nonparametric estimates for a given data set.
4-3 4-4
Location-Scale Distributions Log-Location-Scale Distributions
The distribution of the random variable Y belongs to the The distribution of the random variable T belongs to the log-
location-scale family of distributions if the cdf of Y can be location-scale family of distributions if the cdf of T can be
expressed as expressed as
( )
y−μ log(t) − μ
F (y; μ, σ) = Pr(Y ≤ y) = Φ , −∞ < y < ∞ F (t; μ, σ) = Pr(T ≤ t) = Φ , 0<t<∞
σ σ
where −∞ < μ < ∞ is a location parameter and σ > 0 is a where 0 < exp(μ) < ∞ is a scale parameter and σ > 0 is a
scale parameter. shape parameter.
Φ is the cdf of Y when μ = 0 and σ = 1 and Φ does not Φ is the cdf of log(T ) when μ = 0 and σ = 1 and Φ does not
depend on any unknown parameters. depend on any unknown parameters.
Note: Z = (Y − μ)/σ is a standardized random variable and Note: Z = (log(T ) − μ)/σ is a standardized random variable
Z ∼ Φ(z). Thus the distribution of Z does not depend on and Z ∼ Φ(z). Thus the distribution of Z does not depend
any unknown parameters. on any unknown parameters.
4-5 4-6
Metrics/Functions of the Parameters Metrics/Functions of the Parameters-Continued
• Cumulative distribution function (cdf) of T • The mean time to failure, MTTF, of T (also known as the
expectation of T )
F (t; θ ) = Pr(T ≤ t), t > 0. ∞ ∞
E(T ) = tf (t; θ ) dt = [1 − F (t; θ )] dt.
0 0

• The p quantile of T is the smallest value tp such that If 0∞ tf (t; θ ) dt = ∞, we say that the mean of T does not
exist.
F (tp; θ ) ≥ p.
• The variance (or the second central moment) of T and the
standard deviation
• Hazard function of T ∞
Var(T ) = [t − E(T )]2f (t; θ ) dt
f (t; θ ) *0
h(t; θ ) = , t > 0.
1 − F (t; θ ) SD(T ) = Var(T ).
4-7 4-8
Importance of (Log)-Location-Scale Distributions
• The most commonly used statistical distributions are mem-

bers of these classes of distributions: exponential, normal,
Weibull, lognormal, Fréchet, logistic, loglogistic, smallest Chapter 4
extreme value and largest extreme value distributions.
Segment 2
• Methods of inference, statistical theory, and computer soft- Exponential, Normal, Lognormal, Smallest Extreme
ware generated for the general family can be applied to this Value, and Weibull Distributions
large, important class of models.
• Theory for (log)-location-scale distributions is relatively sim-

ple.
4-9 4 - 10
Examples of Exponential Distributions

Exponential Distribution
For T ∼ EXP(θ, γ),


t−γ 1 2.0
F (t; θ, γ) = 1 − exp −
θ 1.5
1 t−γ
f (t; θ, γ) = exp − F (t ) 0.5 f (t ) 1.0
θ θ
f (t; θ, γ) 1 0.5
h(t; θ, γ) = = , t > γ,
1 − F (t; θ, γ) θ 0 0.0
0.0 0.5 1.0 1.5 2.0 2.5 3.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0
t t
where θ > 0 is a scale parameter and γ is both a location and
Hazard Function
a threshold parameter. When γ = 0 one gets the well-known 2.5
θ γ
one-parameter exponential distribution. 2.0
0.5 0
1.5 1.0 0
h (t )
Quantiles: tp = γ − θ log(1 − p). 1.0
2.0 0
Moments: For integer m > 0, E[(T − γ)m] = m! θm. Then 0.5
0.0
E(T ) = γ + θ, Var(T ) = θ2. 0.0 0.5 1.0 1.5 2.0 2.5 3.0
t
4 - 11 4 - 12
Motivation for the Exponential Distribution
Normal (Gaussian) Distribution
• Simplest distribution used in the analysis of reliability data.

For Y ∼ NORM(μ, σ)
• Has the important characteristic that its hazard function is
y−μ
constant (does not depend on time t). F (y; μ, σ) = Φnorm
σ
1 y−μ
• A popular distribution for some kinds of electronic compo- f (y; μ, σ) = φnorm , −∞ < y < ∞.
σ σ
nents (e.g., capacitors with a chemically-stable dielectric or √
where φnorm(z) = (1/ 2π) exp(−z 2/2) and
robust, high-quality integrated circuits). z
Φnorm(z) = −∞ φnorm(w)dw are pdf and cdf for the standard
• This distribution would not be appropriate for a population normal (μ = 0, σ = 1). −∞ < μ < ∞ is a location parameter
of electronic components having failure-causing quality-defects and σ > 0 is a scale parameter.
(infant mortality).
Quantiles: yp = μ + σΦ−1 −1
norm (p) where Φnorm (p) is the p
• Might be useful to describe failure times for components
quantile for the standard normal distribution.
that exhibit physical wearout only after expected techno-
Moments: For integer m > 0, E[(Y − μ)m] = 0 if m is odd,
logical life of the system in which the component would be
and E[(Y − μ)m] = (m)!σ m/[2m/2 (m/2)!] if m is even. Thus
installed (e.g., some components of a personal computer
have an increasing hazard function after four or five years E(Y ) = μ, Var(Y ) = σ 2.
of operation, but most such computers are retired by then).
4 - 13 4 - 14
Examples of Normal Distributions

Lognormal Distribution
Cumulative Distribution Function Probability Density Function If T ∼ LNORM(μ, σ) then log(T ) ∼ NORM(μ, σ) with
1
1.2 log(t) − μ
1.0 F (t; μ, σ) = Φnorm
σ
F (t ) 0.5 f (t )
0.8

0.6 1 log(t) − μ
0.4 f (t; μ, σ) = φnorm , t > 0.
0.2 σt σ
0 0.0
φnorm and Φnorm are the pdf and cdf for the standard normal
3 4 5 6 7 3 4 5 6 7
t t distribution. exp(μ) is a scale parameter; σ > 0 is a shape
Hazard Function
parameter.
20 σ μ
0.3 5 Quantiles: tp = exp μ + σΦ−1 −1
norm (p) , where Φnorm (p) is the
15
0.5 5 p quantile for the standard normal distribution.

h (t ) 10 0.8 5
Moments: For integer m > 0, E(T m) = exp mμ + m2σ 2/2 .
5

0 E(T ) = exp μ + σ 2/2 , Var(T ) = exp 2μ + σ 2 exp(σ 2) − 1 .
3 4 5 6 7
t
4 - 15 4 - 16
Examples of Lognormal Distributions Motivation for the Lognormal Distribution
Cumulative Distribution Function Probability Density Function • The lognormal distribution is a common model for fail-
1 5 ure times.
4
3 • It can be justified for a random variable that arises from the

F (t ) 0.5 f (t )
2 product of a number of identically distributed independent
1 positive random quantities.
0 0
0.0 0.5 1.0 1.5 2.0 2.5 0.0 0.5 1.0 1.5 2.0 2.5 • It has been suggested as an appropriate model for failure
t t
time caused by a degradation process with combinations of
Hazard Function random rates that combine multiplicatively.
5
σ μ
4 • Commonly used to describe time to fracture from fatigue
0.4 0.1
3 0.7 −0.5 crack growth in metals.
h (t ) 1.0 −2
2
1.8 −5 • Useful in modeling failure time of a population electronic
1
0
components with a decreasing hazard function (due to a
0.0 0.5 1.0 1.5 2.0 2.5 small proportion of defects in the population).
t
4 - 17 4 - 18
Examples of Smallest Extreme Value Distributions
Smallest Extreme Value Distribution
For Y ∼ SEV(μ, σ), Cumulative Distribution Function Probability Density Function

y−μ 1
F (y; μ, σ) = Φsev 0.3
σ
1 y−μ
f (y; μ, σ) = φsev F (t ) 0.5 f (t ) 0.2
σ σ
1 y−μ 0.1
h(y; μ, σ) = exp , −∞ < y < ∞.
σ σ 0 0.0
Φsev(z) = 1 − exp[− exp(z)], φsev(z) = exp[z − exp(z)] are cdf 5 10

t
15 20 5 10
t
15 20
and pdf for the standard SEV (μ = 0, σ = 1) distribution.

Hazard Function
−∞ < μ < ∞ is a location parameter and σ > 0 is a scale
10
parameter. σ μ
8
1 15
6 2 15
Quantiles: yp = μ + Φ−1sev (p)σ = μ + log[− log(1 − p)]σ.
h (t )
4
3 15
Mean and Variance: E(Y ) = μ − σγ, Var(Y ) = σ 2π 2/6, 2
where γ ≈ 0.5772, π ≈ 3.1416. 0

5 10 15 20
Note: the hazard function is unbounded and increasing. t
4 - 19 4 - 20
Weibull Distribution
Examples of Weibull Distributions
T ∼ WEIB(η, β)
⎡ ( ) ⎤
β
t ⎦
F (t; η, β) = Pr(T ≤ t) = 1 − exp⎣− Cumulative Distribution Function Probability Density Function
η 1 2.0
( )β−1 ⎡ ( ) ⎤
β
β t t ⎦ 1.5
f (t; η, β) = exp⎣−
η η η F (t ) 0.5 f (t ) 1.0
( )β−1
β t 0.5
h(t; η, β) =, t>0
η η 0 0.0
0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0
where η > 0 is a scale parameter and β > 0 is a shape pa- t t
rameter.
Hazard Function
Quantiles: tp = η[− log(1 − p)]1/β . 2.0

β η
Moments: For integer m > 0, E(T m) = η mΓ(1+m/β). Then 1.5 0.5 1
( ) ( ) ( ) 1.0 1
1 2 1 h (t ) 1.0 1.5 1
E(T ) = ηΓ 1 + , Var(T ) = η 2 Γ 1 + − Γ2 1 + 3.0 1
β β β 0.5
∞ 0.0
where Γ(κ) = wκ−1 exp(−w)dw is the gamma function. 0.0 0.5 1.0
t
1.5 2.0
0
Note: When β = 1 then T ∼ EXP(η).
4 - 21 4 - 22
Motivation for the Weibull Distribution

Alternative Weibull Parameterization
Note: If T ∼ WEIB(η, β) then Y = log(T ) ∼ SEV(μ, σ). • The theory of extreme values shows that the Weibull dis-
Then the Weibull cdf can be written as tribution can be used to model the minimum of a large

log(t) − μ number of independent positive random variables from cer-
F (t; μ, σ) = Pr(T ≤ t) = Φsev
σ tain classes of distributions.

1 log(t) − μ
f (t; μ, σ) = φsev Failure of the weakest link in a chain with many links
σt σ
with failure mechanisms (e.g., creep or fatigue) in each
where σ = 1/β is a scale parameter for log(T ) (shape param-
link acting approximately independent.
eter for T ), μ = log(η) is a location parameter for log(T ),
and Failure of a system with a large number of components in
series and with approximately independent failure mech-
φsev(z) = exp[z − exp(z)] anisms in each component.
Φsev(z) = 1 − exp[− exp(z)].
• A common justification for its use is empirical: the Weibull
Quantiles:
distribution can be used to model failure-time data with a

tp = exp μ + σΦ−1
sev (p) , t>0 decreasing or an increasing hazard function.
where Φ−1
sev (p) is the p quantile for the standard SEV (i.e., • When compared with the lognormal distribution, Weibull in-
μ = 0, σ = 1) distribution. ferences are conservative when extrapolating into either tail.
4 - 23 4 - 24
Largest Extreme Value Distribution
When Y ∼ LEV(μ, σ),

y−μ
F (y; μ, σ) = Φlev
Chapter 4 σ
1 y−μ
f (y; μ, σ) = φlev
Segment 3 σ σ
exp − y−μ
σ
h(y; μ, σ) = + ,, −∞ < y < ∞.
Other (Log)-Location-Scale Distributions: Largest σ exp exp − y−μ
σ −1
Extreme Value, Fréchet, LOGISTIC, and Loglogistic
where Φlev(z) = exp[− exp(−z)] and φlev(z) = exp[−z−exp(−z)]
are the cdf and pdf for the standard LEV (μ = 0, σ = 1) dis-
tribution.
−∞ < μ < ∞ is a location parameter and σ > 0 is a scale
parameter.
4 - 25 4 - 26
Examples of Largest Extreme Value Distributions
Largest Extreme Value Distribution - Continued

0.15 Quantiles: yp = μ − σ log[− log(p)].

Mean and Variance: E(Y ) = μ + σγ, Var(Y ) = σ 2π 2/6,
F (t ) 0.5 f (t )0.10
where γ ≈ 0.5772, π ≈ 3.1416.
0.05
0 0.00
0 5 10 15 20 25 0 5 10 15 20 25
Notes:
t t
• The hazard is increasing but is bounded in the sense that

Hazard Function
0.5
σ μ limy→∞ h(y; μ, σ) = 1/σ.
0.4
2 9
0.3 3 9
h (t ) 5 9
• If Y ∼ LEV(μ, σ) then −Y ∼ SEV(−μ, σ).
0.2
0.1
0.0
0 5 10 15 20 25
t
4 - 27 4 - 28
Fréchet Distribution Examples of Fréchet Distributions
T ∼ FREC(η, β)
ηβ
F (t; η, β) = Pr(T ≤ t) = exp − , 1 1.2
t 1.0
β
β η β η 0.8
f (t; η, β) = exp − F (t ) 0.5 f (t ) 0.6
t t t
0.4
β η β 0.2
h(t) = t t , t>0

0 0.0
η β 0 1 2 3 4 0 1 2 3 4
exp −1 t t
t
Hazard Function
where η > 0 is a scale parameter and β > 0 is a shape pa-
rameter. Quantiles: tp = η[− log(p)]−1/β . 1.5
β η
1 1
2 1
h (t ) 1.0 3 1
• If T ∼ FREC(η, β), then 1/T ∼ WEIB(1/η, β) 0.5
• The Fréchet distribution hazard function has a shape similar 0.0

0 1 2 3 4
to the that of the lognormal distribution. t
4 - 29 4 - 30
Fréchet Distribution- Continued
Moments: For integer m > 0, the mth Fréchet moment Alternative Fréchet Parameterization
E(T m) exists if and only if β > m. In particular,
⎧ ( ) If T ∼ FREC(η, β) then Y = log(T ) ∼ LEV(μ, σ). Then
⎪
⎪
⎨η m Γ 1 − m for β > m
E(T m) = β log(t) − μ
⎪
⎪ F (t; μ, σ) = Φlev
⎩∞ otherwise σ

where Γ(κ) is the gamma function. Then 1 log(t) − μ
f (t; μ, σ) = φlev , t>0
σt σ
E(T ) = ηΓ(1 − 1/β), β>1
where σ = 1/β, μ = log(η), and
Var(T ) = η 2 Γ(1 − 2/β) − Γ2(1 − 1/β) , β > 2.
Φlev(z) = exp[− exp(−z)]
φlev(z) = exp[−z − exp(−z)].
• The Fréchet distribution can be used to model the max-
imum of a large number of independent identically dis- Quantiles:
tributed (iid) positive random variables.
tp = exp μ + σΦ−1
lev (p)
• When extrapolating into either tail of a failure-time distri-
where Φ−1lev (p) is the p quantile for the standard LEV (i.e.,
bution, the Fréchet distribution provides more optimistic
μ = 0, σ = 1).
(anti-conservative) estimates relative to the lognormal or
the Weibull distribution.
4 - 31 4 - 32
Logistic Distribution Examples of Logistic Distributions
For Y ∼ LOGIS(μ, σ),

y−μ 1 0.25
F (y; μ, σ) = Φlogis
σ 0.20
1 y−μ
f (y; μ, σ) = φlogis F (t ) 0.5
0.15
f (t )
σ σ 0.10
1 y−μ
h(y; μ, σ) = Φlogis , −∞ < y < ∞. 0.05
σ σ
0 0.00
where −∞ < μ < ∞ is a location parameter and σ > 0 is a 0 5 10 15 20 0 5 10 15 20
t t
scale parameter.
Hazard Function
1.0
φlogis and Φlogis are pdf and cdf for the standard logistic σ μ
0.8
distribution defined by 1 10
0.6 2 10
exp(z) h (t ) 5 10
0.4
φlogis(z) =
[1 + exp(z)]2 0.2
exp(z) 0.0
Φlogis(z) = . 0 5 10 15 20
1 + exp(z) t
4 - 33 4 - 34
Logistic Distribution-Continued
Loglogistic Distribution
Quantiles: yp = μ + σΦ−1
logis (p) = μ + σ log[p/(1 − p)], where If Y ∼ LOGIS(μ, σ) then T = exp(Y ) ∼ LOGLOGIS(μ, σ)
Φ−1
logis (p) = log[p/(1 − p)] is the p quantile for the standard with
logistic distribution.
log(t) − μ
F (t; μ, σ) = Φlogis
σ

E[(Y − μ) ]
Moments: For integer m > 0, m = 0 if m is odd,

∞ (1/i)m if m 1 log(t) − μ
and E[(Y − μ) ] = 2σ (m!) 1 − (1/2)
m m m−1
i=1 f (t; μ, σ) = φlogis
σt σ
is even. Thus
1 log(t) − μ
σ 2π 2 h(t; μ, σ) = Φlogis , t > 0.
E(Y ) = μ and Var(Y ) = . σt σ
3
exp(μ) is a scale parameter; σ > 0 is a shape parameter.
Note: the hazard function is always increasing but bounded
Φlogis and φlogis are cdf and pdf for a LOGIS(0, 1).
by 1/σ, as shown in the plots.
4 - 35 4 - 36
Examples of Loglogistic Distributions
Loglogistic Distribution-Continued
1 3.0
2.5 Quantiles: tp = exp μ + σΦ−1 σ
logis(p) = exp(μ)[p/(1 − p)] .
2.0
Moments: For integer m > 0,
F (t ) 0.5 f (t ) 1.5
1.0
E(T m) = exp(mμ) Γ(1 + mσ) Γ(1 − mσ).
0.5
0 0.0 The m moment is not finite when mσ ≥ 1.
0.0 0.5 1.0 1.5 0.0 0.5 1.0 1.5
t t
For σ < 1,
Hazard Function
4
σ μ E(T ) = exp(μ) Γ(1 + σ) Γ(1 − σ),
3 0.1 0.1
0.3 −1 and for σ < 1/2,
h (t ) 2
1.7 −2
1
Var(T ) = exp(2μ) Γ(1 + 2σ) Γ(1 − 2σ) − Γ2(1 + σ) Γ2(1 − σ) .
0
0.0 0.5 1.0 1.5
t
4 - 37 4 - 38
Generalized Gamma Distribution
T ∼ GENG(μ, σ, λ)
⎧
⎪
⎪
⎪ Φlg λω + log(λ−2); λ−2 if λ>0
⎪
⎨
F (t; μ, σ, λ) = Φnorm(ω) if λ=0
⎪
⎪
⎪
⎪
⎩ 1 − Φlg λω + log(λ−2); λ−2 if λ<0
⎧
|λ|
⎨ σt φlg λω + log(λ−2); λ−2 if λ
⎪ = 0
Chapter 4
f (t; μ, σ, λ) =
⎪
⎩ 1
Segment 4 σt φnorm (ω) if λ = 0
where t > 0, ω = [log(t) − μ]/σ and
The Generalized Gamma Distribution Φlg(z; κ) = ΓI[exp(z); κ],
1
φlg(z; κ) = exp[κz − exp(z)].
Γ(κ)
−∞ < μ < ∞, exp(μ) is a scale parameter and −∞ < λ < ∞
and σ > 0 are shape parameters and ΓI(v; κ) is the incomplete
gamma function defined by

v xκ−1 exp(−x) dx
ΓI(v; κ) = 0 , v > 0.
Γ(κ)
4 - 39 4 - 40
Generalized Gamma Distribution Moments
• Moments: For integer m and λ = 0

⎧ - .mσ/λ −1
⎪
⎨ exp(mμ) λ2 Γ λ (mσ+λ−1 ) Generalized Gamma Distribution Quantiles
E(T m) = −2
Γ(λ )
if mλσ + 1 > 0
⎪
⎩ ∞ if mλσ + 1 ≤ 0.
The GENG quantile function is
When λ = 0, E(T m) is the same as that for the lognormal
distribution. tp = exp[μ + σ ω(p; λ)],
where ω(p; λ) is the p quantile of [log(T ) − μ]/σ given by
• Thus when the mean and the variance are finite and λ = 0, ⎧
⎪ 1
⎪
⎪
⎪
⎪
log λ2Γ−1 (p; λ−2) if λ > 0
θ Γ λ−1(σ + λ−1) ⎪
⎪
⎪
λ I
E(T ) = ⎨
−1
Γ(λ−2) ω(p; λ) = Φnorm(p) if λ = 0
⎪
⎪
⎡ ⎤ ⎪
⎪
Γ λ−1(2σ + λ−1) Γ2 λ−1(σ + λ−1) ⎪
⎪ 1
⎪
Var(T ) = θ2⎣ − ⎦. ⎪
⎩ log λ2Γ−1 −2
I (1 − p; λ ) if λ < 0.
Γ(λ−2) Γ2(λ−2) λ
• When λ = 0, E(T ) and Var(T ) are the same as that for the
lognormal distribution.
4 - 41 4 - 42
Generalized Gamma Distribution Special Cases
• If λ = 1, T ∼ WEIB(μ, σ).
• If λ = 0, T ∼ LNORM(μ, σ). Chapter 4
Segment 4
• If λ = −1, T ∼ FREC(μ, σ).
Threshold Log-Location-Scale Distributions
• When λ = σ, T is has a Gamma distribution with a scale
parameter λ2 exp(μ) and a shape parameter λ−2.
• When λ = σ = 1, T ∼ EXP(θ), where θ = exp(μ).
4 - 43 4 - 44
pdfs for Threshold (Three-Parameter) Lognormal Distributions with a Threshold Parameter

Distributions for μ = 0 and σ = 0.5 with γ = 1,2,3.
• So far we have discussed nonnegative random variables with

cdfs that begin increasing at t = 0.
0.8
• One can generalize these and similar distributions by adding
a threshold parameter, γ, to shift the beginning of the
distribution away from 0.
0.6
• Distributions with a threshold are particularly useful for fit-
ting skewed distributions that are shifted far to the right of 0.
f (t ) 0.4
• The cdf for location-scale log-based threshold distributions is

log(t − γ) − μ
γ=3 F (t; μ, σ, γ) = Φ
σ
0.2 ⎧ ⎡( )1/σ ⎤⎫
⎨ t−γ ⎬
γ=2 = Φ log⎣ ⎦ , t>γ
⎩ exp(μ) ⎭
γ=1
0.0 where −∞ < γ < ∞, −∞ < μ < ∞, σ > 0, and Φ(z) is a
1 2 3 4 5 6 7
t completely specified cdf (i.e., no unknown parameters).
4 - 45 4 - 46
Properties of Distributions with a Threshold

Examples of Distributions with a Threshold Parameter
• When the distribution of T has a threshold, γ, then the
• Three-parameter lognormal distribution distribution of W = T −γ has a distribution with 0 threshold.

log(t − γ) − μ • The properties of the distribution of T are closely related
F (t; μ, σ, γ) = Φnorm , t > γ.
σ to the properties of the distribution of W .
• In general, E(T ) = γ + E(W ) and tp = γ + wp, where wp is

• Three-parameter Weibull distribution the p quantile of the distribution of W .
⎡ ( )β ⎤
t−γ • Changing γ simply shifts the distribution on the time axis,
F (t; η, β, γ) = 1 − exp⎣− ⎦
η there is no effect on the distribution’s spread or shape. Thus

log(t − γ) − μ Var(T ) = Var(W ).
= Φsev ,t>γ
σ
• There are, however, some very specific issues in the esti-
where σ = 1/β and μ = log(η). mation of γ because the points at which the cdf is positive
depends on γ.
4 - 47 4 - 48
Chapter 5
System Reliability Concepts and Methods
System Reliability Concepts and Methods
Chapter 5 Objectives

• Explain important system reliability concepts like system
structure, redundancy, nonrepairable, and repairable sys-
tems, maintainability and availability.
W. Q. Meeker, L. A. Escobar, and F. G. Pascual • Describe basic concepts of system reliability modeling.
ington State University. • Give expressions for the distribution of system failure time
as a function of individual component failure time distribu-
Copyright 2018 W. Q. Meeker, L. A. Escobar, and F. G. Pas- tions.
cual.
• Illustrate the analysis of failure-time data with two failure
Based on the authors’ book Statistical Methods for Relia- modes.
published in 2019. • Provide examples of the use of component test data to
estimate system reliability.
June 7, 2018
8h 15min 5-1 5-2
Definitions
• System: a collection of components needed to realize a

Chapter 5
given task.
Segment 1
• System structure: a logic diagram illustrating the function
Introduction to System Reliability and Series Systems
of the components within the system and how they relate
to system operational state (usually operational or not
operational).
5-3 5-4
Time Dependency of System Reliability
System Structures and System Failure Probability For the time to first failure of a new system with s compo-
nents (all components starting a time 0)
System failure probability FT (t; θ ) is the probability that • The cdf for component i is Fi = Fi(t; θ i). The correspond-
a system fails before t (time to first failure for a repairable ing survival probability is Si = Si(t; θ i) = 1 − Fi(t; θ i). The
system). θ is may have some elements in common. Here θ denotes
the unique elements in (θ 1, . . . , θ s).
The failure probability of the system depends on:
• The cdf for the system is denoted by FT = FT (t; θ ). This cdf
• Time in operation (or other measure of use) denoted by t is determined by the Fi’s and the system structure. Then
• System structure. FT (t; θ ) = g[F1(t; θ 1), . . . , Fs(t; θ s)]
• Reliability of system components, including interconnec- or one of the simpler forms
tions, interfaces, and human operators.
FT (θ ) = g[F1(θ 1), . . . , Fs(θ s)]
• Environmental conditions. FT = g(F1, . . . , Fs).
To simplify the presentation, time-(and parameter)-dependency
will usually be suppressed in this chapter.
5-5 5-6
Examples of Series Systems
A Series System with Two Components

• Chain
• Multi-cell battery
1 2 • Inexpensive personal computer
• Cell phone
5-7 5-8
Series System cdf Reliability of a System with s Identical Independent

Components in Series
A series structure with s components works iff all the com-

ponents work. Then
• For two independent components 1.0

s=1
FT (t) = Pr(T ≤ t) = 1 − Pr(T > t)

0.8
= 1 − Pr(T1 > t ∩ T2 > t) s=10
Series System Reliability
= 1 − Pr(T1 > t) Pr(T2 > t)

0.6
= 1 − (1 − F1)(1 − F2) s=25
• For s independent components 0.4

s=50
s

FT (t) = 1 − (1 − Fi)
0.2
i=1 s=100
s=200
s=400
• For s iid components (so F = Fi, i = 1, . . . , s) 0.0
FT (t) = 1 − (1 − F )s. 0.980 0.985 0.990 0.995 1.000

Individual Component Reliability
5-9 5 - 10
Effect of Positive Dependency in a Series System

Effect of Positive Dependency in a with Two Identical Components
Two-Component Series System Having Lognormal Failure Times
• For a series system with two components and dependent • The distributions of log failure times for the individual com-
failure times ponents is bivariate normal with the same (arbitrary) mean
FT (t) = Pr(T ≤ t) = 1 − Pr(T > t) = 1 − Pr(T1 > t ∩ T2 > t). and standard deviation for both components and correla-
tion ρ.
In this case, the evaluation has to be done with respect to
the bivariate distribution of T1 and T2. • The reliability 1 − FT (t) of the system can be expressed as
a function of the individual reliability components 1 − F (t)
• If the correlation between the two components is positive, and ρ.
then the assumption of independence is conservative in the
sense that the actual FT (t) is smaller than that predicted • When ρ = 1 (so the two components are perfectly depen-
by the independent-component model. dent and will fail at exactly the same time), the system
reliability 1 − FT (t) is the same as the reliability for a single
• These results extend to the s components in series, the component.
system FT (t) would have to be computed with respect to
the underlying s-variate distribution. Such computations • When ρ = 0 (so the two components are independent),
are, in general, difficult. 1 − FT (t) corresponds to the system reliability for an s = 2
series system with independent components.
5 - 11 5 - 12
Reliability of a Series System
with 2 Identical Dependent Components
Two−Component Series−System Reliability 1.00
Chapter 5
0.95
Segment 2
0.90 Redundancy and Parallel Systems
0.85
ρ=1
0.9
0.7
0.80 0.4 0
0.80 0.85 0.90 0.95 1.00

5 - 13 5 - 14
A Parallel System with Two Components Examples of Systems with Components in Parallel
• Automobile headlights
1
• RAID array computer disk systems
• Stairwells with emergency lighting
2 • Multiple light banks in an office
5 - 15 5 - 16
Parallel System cdf Reliability of a System with s Identical Independent

Components in Parallel
A parallel structure with s components works if at least one

of the components works. Then
• For two independent components 1.00

s=5
FT (t) = Pr(T ≤ t)
= Pr(T1 ≤ t ∩ T2 ≤ t) 0.95
Parallel System Reliability
= Pr(T1 ≤ t) Pr(T2 ≤ t) s=4
= F1 F2
0.90
• For s independent components

s=3
s

FT (t) = Fi 0.85
i=1 s=2
s=1
• For s iid components (Fi = F, i = 1, . . . , s) 0.80
FT (t) = F s. 0.5 0.6 0.7 0.8 0.9 1.0

5 - 17 5 - 18
Effect of Positive Dependency in a Parallel System
Effect of Positive Dependency in a with Two Identical Components
Two-Component Parallel System Having Lognormal Failure Times
• The distributions of log failure times for the individual com-

• For a parallel system with two components and dependent
ponents is bivariate normal with the same (arbitrary) mean
failure times and standard deviation for both components and correla-
tion ρ.
FT (t) = Pr(T ≤ t) = Pr(T1 ≤ t ∩ T2 ≤ t).
In this case, the evaluation has to be done with respect to • The reliability 1 − FT (t) of the system can be expressed as
a function of the individual reliability components 1 − F (t)
the bivariate distribution of T1 and T2.
and ρ.
• If the dependency between the two components is positive, • When ρ = 1 (so the two components are perfectly dependent
then the assumption of independence is anti-conservative in and will fail at exactly the same time), the system reliability
the sense that the actual FT (t) is larger than that predicted 1−FT (t) is the same as the reliability for a single component.
by the independent-component model.
• When ρ = 0 (so the two components are independent),
• These results extend to the s components in parallel, the 1 − FT (t) corresponds to the system reliability for a s = 2
independent parallel system.
system FT (t) would have to be computed with respect to
the underlying s-variate distribution. Such computations • The advantages of redundancy can be seriously degraded
are, in general, difficult. when the failure times of the individual components have
positive dependence.
5 - 19 5 - 20
Reliability of a Parallel System with

Two Identical Dependent Components
Having Lognormal Failure Times
1.00
Two−Component Parallel−System Reliability
Chapter 5
0.95
Segment 3
0.90 Series-Parallel Systems
0.85
ρ=0 0.4 0.7 0.9 1
0.80
0.5 0.6 0.7 0.8 0.9 1.0
5 - 21 5 - 22
Systems Using a Combination of Series and Parallel

Structures
Series and parallel structures are the basis for building mod- A Series-Parallel System Structure
els for more complicated systems which use redundancy to with Component-Level Redundancy
increase reliability.
Some examples are:

11 12
• Series-parallel with component-level redundancy.
• Series-parallel with system-level redundancy. 21 22
• Series system with parallel redundancy in critical compo-

nents.
5 - 23 5 - 24
Systems with Component-Level Redundancy
A k × r component-level redundant structure has k series

Examples Series-Parallel System Structure structures each one made of r units in parallel.
with Component-Level Parallel Redundancy
• For 2 × 2 series-parallel with independent components
FT (t) = 1 − Pr(T > t)

• Spare lasers in each repeater in an under-sea fiber-optic
data transmission system = 1 − Pr(parallel subsystem 1 OK ∩ parallel subsystem 2
= 1 − (1 − F11F21)(1 − F12F22)
where Fij , i = 1, 2 are the cdfs for the parallel subsystem j.
• Automobile with two headlights. Separate front and rear
hydraulic brake systems • For a k × r series-parallel with independent components
⎛ ⎞
k
r

FT (t) = 1 − ⎝1 − Fij ⎠
• Human body (hands, eyes, lungs, kidneys) j=1 i=1
• When all of the components are iid
FT (t) = 1 − (1 − F r )k
5 - 25 5 - 26
Examples Series-Parallel System Structure

with System-Level Parallel Redundancy
A Series-Parallel System Structure
with System-Level Redundancy
• Dual central processors for a system-critical communica-
tions switching system
11 12
• Multiple computers, working in parallel on the space shuttle
• Multiple trans-Atlantic transmission cables

21 22
• Fiber bundle or stranded wire
5 - 27 5 - 28
Series-Parallel Structure
with System-Level Redundancy
A r × k series-parallel system-level redundancy structure has

r parallel sets each of k units in series.
• For 2 × 2 structure with independent components
FT (t) = Pr(T ≤ t) Chapter 5

= Pr(series subsystem 1 failed ∩ series subsystem 2 faile
= [1 − (1 − F11)(1 − F12)][1 − (1 − F21)(1 − F22)] Segment 4
where Fij , j = 1, 2 are the cdfs for the series system i. More Complicated System Structures
• For a r × k structure with independent components
⎡ ⎤
r
k

FT (t) = ⎣1 − 1 − Fij ⎦
i=1 j=1
• For a r × k parallel-series structure with iid components

r
FT (t) = 1 − (1 − F )k
5 - 29 5 - 30
Bridge System Structure cdf
A Bridge System Structure • Let Ai be the event that the i unit is working.
• Conditioning on the event A3 and using the law of total

probability gives
1 2
FT (t) = Pr(T ≤ t ∩ A3) + Pr(T ≤ t ∩ Ac3)
3 = Pr(T ≤ t|A3) Pr(A3) + Pr(T ≤ t|Ac3) Pr(Ac3)
= Pr[(Ac1 ∩ Ac4) ∪ (Ac2 ∩ Ac5)|A3] Pr(A3)

4 5 + Pr[(Ac1 ∪ Ac2) ∩ (Ac4 ∪ Ac5)|Ac3] Pr(Ac3)
= [F1F4 + F2F5 − F1F2F4F5](1 − F3)

+ [F1 + F2 − F1F2][F4 + F5 − F4F5]F3
5 - 31 5 - 32
k out of s System Structures
A 2 out of 3 System Structure A k out of s remains operable as long as at least k of the

system’s s are operable. Examples include
• A satellite battery system in which system will continue to

operate as long as 6 of 10 batteries to operate correctly.
1 2
• Solid-state disks which continue to provide error-free service
1 3 by having redundant memory locations.
2 3 • A web-hosting service uses a system with ten servers and

the system operates satisfactorily if at least seven of those
servers are operating.
5 - 33 5 - 34
k out of s System Structures
• For k out of s independent components

⎧ ⎡ ⎤⎫
s
⎪
⎨ s ⎪
⎬
⎣ δ
2 out of 3 System Structure cdf FT (t) = Fi i (1 − Fi)(1−δi)⎦
⎪ ⎪
j=s−k+1⎩δ∈A i=1j
⎭
For a 2 out of 3 independent components where δ = (δ1, . . . , δs) with δi = 1 indicating failure of unit
i by time t and δi = 0 otherwise and Aj is the set of all δ
FT (t) = Pr(T ≤ t) such that δ δ = j.
= Pr(exactly two fail) + Pr(exactly three fail)
• FT (t) can also be viewed as the distribution of the sum of
= F1F2(1 − F3) + F1F3(1 − F2) + F2F3(1 − F1) + F1F2F3
s independent non-identically distributed Bernoulli random
= F1F2 + F1F3 + F2F3 − 2F1F2F3
variables, and is known as the Poisson-binomial distribution.
• For identically distributed components (F = Fi, i = 1, . . . , n)

s
s
FT (t) = F j (1 − F )s−j .
j=s−k+1
j
5 - 35 5 - 36
Other System Structures
Standby or passive redundancy: a redundant unit is ac-

tivated only when another unit fails and the redundant unit
is need to keep the system working.
Chapter 5
There are many variations of this:
Segment 5
• Cold standby. Component is not turned on until it is needed.
Multistate and Markov System Reliability Models
• Partially loaded redundancy.
Need to consider the reliability of the switching mechanism

that activates the standby units.
5 - 37 5 - 38
Multistate Systems A Three-State Non-Repairable System
• Multi-state reliability models are useful for certain applica-

tions.
• Multi-state reliability models allow additional flexibility to

decribe dependency between system components.
λ1 λ2
• When transition times in a multi-state system are described
by an exponential distribution, simple results for common
reliability metrics like the the cdf and quantiles of the failure-
time distribution are available. 1 2 3
• A simple example of a three-state system is a two-component

parallel system and the state would indicate the number of
operating components.
5 - 39 5 - 40
A Three-State Repairable System
Repairable Systems
• Multi-state reliability models are also useful for modeling

repairable systems.
• There are additional metrics for repairable systems including λ1 λ2

system availability amd mean time between failures (MTBF).
• When transition times in a multi-state system are described

by an exponential distribution, simple close-form results for 1 2 3
common repairable systems reliability metrics (like avail- μ1
ablity and MTBF) are available.
μ2
5 - 41 5 - 42
Markov and Other More General Models
Markov models allow the modeling of repairable and non-

repairable systems, allowing for dependence among compo-
nents and common-cause failures.
• Markov models are, however, only suitable for relatively

small systems.
• The Markov models are also limited by the life and repair
distributions that can be employed.
• Non-Markovian generalizations are possible, but lead to com-

putational difficulties. Analysis of non-Markovian models is
generally done with simulation.
5 - 43
Chapter 6
Probability Plotting Probability Plotting


• The purposes of probability plots.
• The basic concepts of probability plotting.

W. Q. Meeker, L. A. Escobar, and F. G. Pascual • How to linearize a cdf by using special plotting scales.
ington State University. • How to plot a nonparametric estimate F to judge the ade-
quacy of a particular parametric distribution.
cual. • Using probability plots to obtain graphical estimates of reli-
ability characteristics like failure probabilities and quantiles.
bility Data, Second Edition, John Wiley & Sons Inc., to be • Methods of separating useful information from noise when
published in 2019. interpreting a probability plot.
June 7, 2018
8h 15min 6-1 6-2
Purposes of Probability Plots
Probability plots are used to:
• Assess the adequacy of a particular distributional model.
• To detect multiple failure modes or mixture of different

Chapter 6 populations.
Segment 1 • Displaying the results of a parametric maximum likelihood

fit along with the data.
Purposes of Probability Plots and Linearizing a cdf
• Obtain, by drawing a smooth curve through the points, a

semiparametric estimate of failure probabilities and distri-
butional quantiles.
• Obtain graphical estimates of model parameters (e.g., by

fitting a straight line through the points on a probability
plot).
6-3 6-4
Linearizing the Exponential cdf

Probability Plotting Scales: Linearizing a cdf

cdf: p = F (t; θ, γ) = 1 − exp − (t−γ)
θ , t ≥ γ.
Main Idea: For a given cdf, F (t), one can linearize the
{ t versus F (t) } plot by: Quantiles : tp = γ − θ log(1 − p).
Conclusion:
The { tp versus − log(1 − p) } plot is a straight line (the cdf
• Finding transformations of F (t) and t such that the rela-
line).
tionship between the transformed variables is linear.
• The transformed axes are relabeled in terms of the original We plot tp on the horizontal axis and p on the vertical axis.
probability and time variables. γ is the intercept on the time axis and 1/θ is equal to the
slope of the cdf line.
The resulting probability axis is generally nonlinear and is
called the probability scale. The data axis is usually a linear
Note:
axis or a log axis.
Changing θ changes the slope of the line and changing γ
changes the position of the line.
6-5 6-6
Plot with Exponential Distribution Probability Scales
Showing Exponential cdfs as Straight Lines for Linearizing the Normal Distribution cdf
Combinations of Parameters θ = 50, 200 and γ = 0, 200
tp = γ − θ log(1 − p)
cdf: p = F (y; μ, σ) = Φnorm y−μ
σ , −∞ < y < ∞.
Quantiles : yp = μ + σΦ−1
norm (p).
Φ−1
.99
norm (p) is the p quantile of the standard normal distribution.
4
.98
θ, γ = 50, 0
Conclusion:
Standard exponential quantile

200, 0 { yp versus Φ−1
norm (p) } will plot as a straight line.
.95 3
Proportion Failing
50, 200
.9 μ is the point at the time axis where the cdf intersects the
2
Φ−1(p) = 0 line (i.e., p = 0.5). The slope of the cdf line on
200, 200
.8
the graph is 1/σ.
.7
1
.6
Note:
.4
.2 Any normal distribution cdf plots as a straight line with a
.01 0
positive slope. Also, any straight line with positive slope
0 200 400 600 800 1000
corresponds to a normal cdf.
Time
6-7 6-8
Plot with Normal Distribution Probability Scales Linearizing the Lognormal Distribution cdf
Showing Normal distribution cdfs as Straight Lines for
Combinations of Parameters μ = 40, 80 and σ = 5, 10
yp = μ + σΦ−1
norm (p)
cdf: p = F (t; μ, σ) = Φnorm log(t)−μ
σ , t > 0.

Quantiles : tp = exp μ + σΦ−1
norm (p) .
.99
.98 2 Then log(tp) = μ + σΦ−1 norm (p)
.95 Conclusion:
.9 { log(tp) versus Φ−1
norm (p) } will plot as a straight line.
1
.8
Standard quantile
Proportion Failing
.7 The median exp(μ) can be read from the time axis at the
.6
.5 0
point where the cdf intersects the horizontal line Φ−1
norm (p) =
.4 0 (which corresponds to p = 0.50. The slope of the cdf
.3
line on the graph is 1/σ (but in the computations use base e
.2
80, 10
-1 logarithms for the times rather than the base 10 logarithms
.1
shown on the figures).
.05
μ, σ = 40, 10
40, 5 -2
.02 80, 5
Note:
.01
Any lognormal distribution cdf plots as a straight line with
0 20 40 60 80 100
a positive slope. Also, any straight line with positive slope
Time
corresponds to a lognormal distribution.
6-9 6 - 10
Plot with Lognormal Distribution Probability Scales

Showing Lognormal Distribution cdfs as Straight Lines
for Combinations of exp(μ) = 50, 500 and σ = 1, 2
log(tp) = μ + σΦ−1
norm (p) Linearizing the Weibull Distribution cdf
0.0 0.5 1.0 1.5 2.0 2.5 3.0
.99
.98 2 cdf: p = F (t; μ, σ) = Φsev log(t)−μ
σ , t > 0.
.95
.9 Quantiles : tp = exp μ + σΦ−1
sev (p) = η[− log(1 − p)]
1/β ,
1
where Φ−1
sev (p) = log[− log(1 − p)], η = exp(μ), β = 1/σ.
.8
Standard quantile
Proportion Failing
.7
.6
.5 0
.4
This leads to
.3
500, 2 1
.2 log(tp) = μ + σ log[− log(1 − p)] = log(η) + log[− log(1 − p)]
-1 β
exp(μ), σ = 50, 2
.1
Conclusion:
.05
50, 1 500, 1
-2
{ log(tp) versus log[− log(1 − p)] } will plot as a straight line.
.02
.01
1 2 5 10 20 50 100 200 500 1000
Time
6 - 11 6 - 12
Plot with Weibull Distribution Probability Scales
Showing Weibull cdfs as Straight Lines for
Linearizing the Weibull Distribution cdf-Continued Combinations of η = 50, 500 and β = 0.5, 1
log(tp) = log(η) + β1 log[− log(1 − p)]
Comments:
0.0 0.5 1.0 1.5 2.0 2.5 3.0
• η = exp(μ) can be read from the time axis at the point

.98
where the cdf intersects the horizontal log[− log(1 − p)] = 0 50, 1
1
.9
line, which corresponds to p ≈ 0.632.
.7
0
• The slope of the cdf line on the graph is β = 1/σ (but in .5
Standard quantile
500, 0.5
Proportion Failing
the computations use base e logarithms for the times rather η, β = 50, 0.5 -1
.3
than the base 10 logarithms used for the figures). .2
• Any Weibull distribution cdf plots as a straight line with

-2
.1
a positive slope. And any straight line with positive slope

.05 -3
corresponds to a Weibull distribution cdf.
.03
• Exponential distribution cdfs plot as straight lines with slopes

.02 500, 1 -4
equal to 1. .01
1 2 5 10 20 50 100 200 500 1000
Time
6 - 13 6 - 14
Choosing Plotting Positions to

Plot the Nonparametric Estimate of F
Chapter 6 • The discontinuity and randomness of F (t) make it diffi-

cult to choose a definition for pairs of points (t, F ) to plot.
Segment 2
• General Idea: Plot an estimate of F at some specified set
Choice of Plotting Positions of points in time and define plotting positions consisting
of a corresponding estimate of F at these points in time.
• With times reported as exact, it is has been traditional to

plot { ti versus F (ti) } at the observed failure times.
6 - 15 6 - 16
Nonparametric Estimate of F (t) for the Shock

Absorbers. Simultaneous Approximate 95%
Criteria for Choosing Plotting Positions
Criteria for choosing plotting positions should depend on the

application or purpose for constructing the probability plot.
1.0
Some applications that suggest criteria:

0.8
Proportion Failing
• Checking distributional assumptions. 0.6
0.4
• Display of maximum likelihood results with data.
0.2
• Estimation of parameters.
0.0
0 5000 10000 15000 20000 25000

Kilometers
6 - 17 6 - 18
Weibull Probability Plot of the Shock Absorber Data.
Plotting Positions: Continuous Inspection Data
Also Shown are Simultaneous Approximate 95%
and Multiple Censoring
F (t) is a step function until the last reported failure time,

but the step increases may be different than 1/n.
Log10 Thousands of Cycles
3.8 3.9 4.0 4.1 4.2 4.3 4.4 4.5
Plotting Positions: { t(i) versus pi} with .98
1+ , .9
pi = F t(i) + Δ + F t(i) − Δ . .7
0
2 .5
.3
.2
• Justification: This is consistent with the definition for sin-
Standard quantile
Proportion Failing
−2
.1
gle censoring. .05
.03
.02 −4
.01
• When the model fits well, the ML line approximately goes
.005
through the points. .003
−6
.001
• Need to adjust these plotting positions when there are ties. 6000 8000 10000 12000 14000 16000 20000 24000 28000
Kilometers
6 - 19 6 - 20
Lognormal Probability Plot of the Shock Absorber

Data. Also Shown are Simultaneous Approximate 95% Plotting Positions: Continuous Inspection Data and
Confidence Bands for F (t) Single Censoring
Let t(1), t(2), . . . be the ordered failure times with no ties.

Log10 Thousands of Cycles When there is not ties, F (t) is a step function increasing by
3.8 3.9 4.0 4.1 4.2 4.3 4.4 4.5
an amount 1/n until the last reported failure time.
.95
.9
1 + ,
.8
Plotting Positions: ti versus i−0.5
n .
.7
.6
.5 0 • Justification:
Standard quantile
1+ ,
Proportion Failing
i − 0.5
.4
.3
= F t + Δ + F t(i) − Δ
.2
n 2 (i)
−1
i − 0.5
.1
E t(i) ≈ F −1 .
.05 n
−2
.02 where Δ is positive and small.
.01
.005
.002 • A simple modification is required if there are ties in the

−3
.001
reported failure times.
6000 8000 10000 12000 14000 16000 20000 24000 28000
Kilometers
6 - 21 6 - 22
Plot of Nonparametric Estimate of F (t) for the Alloy

T7987 Fatigue Life and Simultaneous Approximate
95% Confidence Bands for F (t)
1
Chapter 6
Segment 3 0.8
The Alloy T7987 and

Proportion Failing
0.6
Heat-Exchanger Tube Crack Examples

0.4
0.2
50 100 150 200 250 300 350
Thousand Cycles
6 - 23 6 - 24
Weibull Probability Plot for the Alloy T7987 Fatigue Lognormal Probability Plot for the Alloy T7987
Life and Simultaneous Approximate 95% Confidence Fatigue Life and Simultaneous Approximate 95%
Bands for F (t) Confidence Bands for F (t)
Log10 Thousands of Cycles Log10 Thousands of Cycles
1.9 2.0 2.1 2.2 2.3 2.4 2.5 1.9 2.0 2.1 2.2 2.3 2.4 2.5
.99 .99
.98
.98 2
.9
.95
.7
0 .9
.5
1
.8
.3
.7
.2
Standard quantile
Standard quantile
.6
Proportion Failing
Proportion Failing
−2 .5 0
.1
.4
.05 .3
.03 .2
−1
.02 −4 .1
.01 .05
.005 .02 −2
.003 .01
−6 .005
.002
−3
.001 .001
80 100 120 140 160 180 200 220 240 280 320 360 80 100 120 140 160 180 200 220 240 280 320 360
Thousand Cycles Thousand Cycles
6 - 25 6 - 26
Plotting Positions: Interval Censored Inspection Data Weibull Distribution Probability Plot of the
Heat-Exchanger Tube Crack Data and Simultaneous
Approximate 95% Confidence Bands for F (t)
• Let (t0, t1], . . . , (tm−1, tm] be the inspection times.
• The upper endpoints of the inspection intervals ti, i =

1, 2, . . ., are convenient plotting times.
.2
−
.15
• Plotting Positions: { ti versus pi } with .1

−
−
pi = F (ti) .05
Proportion Failing
−
• When there are no censored observations beyond tm,
−
F (tm) = 1 and this point cannot be plotted on probability .01
paper.
• Justification: with no losses, from standard binomial the-

ory, .001
E[F (ti)] = F (ti).

1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0
Years
6 - 27 6 - 28
Lognormal Distribution Probability Plot of the

Heat-Exchanger Tube Crack Data and Simultaneous
Approximate 95% Confidence Bands for F (t)
.2
− Chapter 6
.15
−
.1
Segment 4
−
.05
Using Simulation to Calibrate
Proportion Failing
Interpretation of Probability Plots

−
−
.01
.001
1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0
Years
6 - 29 6 - 30
Random Normal Variates Plotted on Normal Random Exponential Variates Plotted on Normal
Probability Plots with Sample Sizes of n=10, 20, and Probability Plots with Sample Sizes of n=10, 20, and
40. Five Replications of Each Probability Plot 40. Five Replications of Each Probability Plot
Sample Size= 10 Sample Size= 10

0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
• • • • • • • • • •
0.90 • 0.90 • 0.90 • 0.90 • 0.90 • 0.90 • 0.90 • 0.90 • 0.90 • 0.90 •
0.70 • 0.70 • 0.70 • 0.70 • 0.70 • 0.70 • 0.70 • 0.70 • 0.70 • 0.70 •
• • •• • • •• ••
• •• •
•• ••
• •• ••
0.30 • • 0.30 • • 0.30 •• 0.30 •• 0.30 • 0.30 •• 0.30 • 0.30 • 0.30 •• 0.30 ••
• • • • • • • • • •
• • • • • • • • • •
0.05 • 0.05 • 0.05 • 0.05 • 0.05 • 0.05 • 0.05 • 0.05 • 0.05 • 0.05 •
0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
-0.5 0.5 1.5 -2 -1 0 1 2 -1 0 1 2 -1.5 -0.5 0.5 -0.5 0.5 0 1 2 3 4 5 0.0 1.0 2.0 0 1 2 3 0.0 1.0 2.0 3.0 0.0 0.4 0.8 1.2

0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99
• • • • • • • • • •
0.90 • 0.90 • 0.90 • 0.90 • 0.90 • 0.90 • 0.90 • 0.90 • 0.90 • 0.90 •
• • •• • • • • • • •
•• •• ••
•
•• ••
• • • ••
•
• •
• •• ••
•
0.70 • •• 0.70 •• 0.70 •• 0.70 •• 0.70 •• 0.70 ••
• 0.70 •• 0.70 •• 0.70 •• 0.70 • •
••• ••• •• • • •• • •• •• • •• • •• • ••• •••
• •• • • • • •• ••• ••• ••• ••• •• • •••• •• •
0.30 •• 0.30 • •• 0.30 •• 0.30 •• 0.30 •• 0.30 •• 0.30 •• 0.30 •• 0.30 •• 0.30 ••
•• •• •• •• •• •• •• •• •• ••
0.05 • 0.05 • 0.05 • 0.05 • 0.05 • 0.05 • 0.05 • 0.05 • 0.05 • 0.05 •
• • • • • • • • • •
0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
-2 -1 0 1 -2 -1 0 1 -2 -1 0 1 2 -2 -1 0 1 2 -1.0 0.0 1.0 0.0 1.0 0.0 1.0 2.0 0.0 1.0 2.0 0 1 2 3 4 0.5 1.5 2.5

0.99 • 0.99 • 0.99 • 0.99 • 0.99 • 0.99 • 0.99 • 0.99 • 0.99 • 0.99 •
•• • • • • • • •• • • •• • • • • • •
0.90 •• 0.90 •• 0.90 ••• 0.90 •• 0.90 •• 0.90 ••• 0.90 •• 0.90 • •• 0.90 •• 0.90 • •
••• •• •• •• • •• •• •• •• • • • •• •••
•
0.70 ••• 0.70 •• •• 0.70 ••• • 0.70 • •• ••
•• 0.70 • •• 0.70 ••• 0.70 •••• 0.70 ••• • 0.70 ••• 0.70 •••
•••• •• • ••• ••• ••• ••• ••• • • •• • ••
• •• ••••• •••• •••• ••• ••• •••
•
••• •••• ••••
••
• ••• •• ••• ••• •• ••• •••• •• ••
0.30
•••
0.30
• ••• 0.30
•••
0.30
•••• 0.30
••
0.30
••
0.30
••
0.30 0.30
••
0.30
••
••• • •• • •• •• • •• •• •••• ••• ••
••• ••• •••
0.05 •• 0.05 • • 0.05 •• 0.05 • • 0.05 • 0.05 • 0.05 •• 0.05 0.05 •• 0.05 •
0.01 • 0.01 • 0.01 • 0.01 • 0.01 • 0.01 • 0.01 • 0.01 • 0.01 • 0.01 •
-2 -1 0 1 2 -2 -1 0 1 -2 -1 0 1 2 -2 -1 0 1 2 -1 0 1 2 0 1 2 3 4 5 0.0 1.0 2.0 0 1 2 3 4 5 0.0 1.0 2.0 0 1 2 3 4 5 6
6 - 31 6 - 32
Notes on the Application of Probability Plotting
• Try different assumed distributions and compare the results.

Chapter 6
• Assess linearity, allowing for more variability in the tails.
Segment 5
• To help calibrate, use
Bleed System Example
Simultaneous nonparametric confidence bands.
Segmenting Data to Explain Variability
Simulation or bootstrap. Transmitter Vacuum Tube Example
Probability Plots with Enhancements
• A sharp bend or change in slope in a probability plot gen-
erally indicates the appearance of a different failure mode
(different than the early failures).
6 - 33 6 - 34
Bleed System (All Bases)

Event Plot
Jet Engine Bleed System Failures
• Data from the Weibull Handbook (1984). Bleed Failure Data

Count
Row
1
2 39
52
3
4 46
• Field data from 2256 systems in the field; staggered entry– 5
6
7
8
31
48
102
9
10
11 158
multiple censoring. 12
13
14
2
312
101
15
16 2
101
17
18 2
19
20 100
9
21
22
23
24 100
23
25
• Unexpected failures. 26
27
28
29
30
2
56
27
55
31
32 20
33
34 56
35
36 22
37
38 55
39
40 22
• What is going on??
41
42 56
11
43
44
45
46 53
11
47
48 55
20
49
50 55
51
52 8
55
53
54 4
55
55
56 2
152
• The Weibull probability plot suggest changes in the failure 57
58
59
60
3
152
3
distribution after 600 hours.

0 500 1000 1500 2000
Hours
6 - 35 6 - 36
Bleed System
Bleed System (All Bases)
Separate Weibull Probability Plots
Weibull Probability Plot
for Base D and Other Bases
Bleed Failure Data

with Nonparametric Simultaneous 95% Confidence Bands Bleed Failure Data
Weibull Probability Plot With Individual Weibull Distribution ML Estimates
.05
.03 .1 DBase
.02 OtherBase
.05
.01
.03
.005 .02
.003
Fraction Failing
.01
Fraction Failing
.001
.005
.0005 .003
.0002
.0001 .001
.00005 .0005
.0003
.00002
.0002
.00001
.0001
20 50 100 200 500 1000 2000 5000
20 50 100 200 500 1000 2000 5000
Hours
Hours
6 - 37 6 - 38
Bleed System
Failure Data Analysis-Conclusions Transmitter Vacuum Tube Data
(Davis 1952)
• A shift in the slope of a probability plot often indicates a

different failure mode. • Life data for a certain kind of transmitter vacuum tube used
in the output stage of high-power transmitters.
• Look for explanatory variables to help better understand
data sources. • The data are read-out (interval censored) data.
• Separate analyses of the Base D data and the data from Days
the other bases indicated different failure distributions.
Interval Endpoint Number
• The large slope (β ≈ 5) for Base D indicated strong wearout. Lower Upper Failing
• The relatively small slope for the other bases (β ≈ 0.85) 0 25 109
suggested infant mortality or accidental failures. 25 50 42
50 75 17
• The problem at base D was caused by salt air. A change in 75 100 7
maintenance procedures there solved the main part of the 100 ∞ 13
reliability problem with the bleed systems.
6 - 39 6 - 40
Weibull Probability Plot of the V7 Transmitter Tube

V7 Transmitter Tube Failure Data
Failure Data with Simultaneous Approximate 95%
Event Plot
Transmitting Tube Time-to-Failure Data β
Count Log10 Days

0.0
Row 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0
F(t)
.98
0.5 −
1 ? | 109
−
1.0
1.0
.9
2 | ? | 42 −
−
1.5 0.5
Standard quantile
−
3 | ? | 17
.7 − −
2.0
4 | ? | 7 0.0
2.5 .5
5 13
− −0.5
3.0
.3 −1.0
0 20 40 60 80 100 3.5
20 30 40 50 60 70 80 90 100
Days
Days
6 - 41 6 - 42
Chapter 7 Chapter 7
Parametric Likelihood Fitting Concepts:
Parametric Likelihood Fitting Concepts:
Exponential Distribution Exponential Distribution
Objectives

• How to compute a likelihood for a parametric model using
interval-censored and right-censored data.
W. Q. Meeker, L. A. Escobar, and F. G. Pascual • The use of likelihood and Wald methods of computing con-
Iowa State University, Louisiana State University, and Wash- fidence intervals for model parameters and other quantities
ington State University. of interest.
Copyright 2018 W. Q. Meeker, L. A. Escobar, and F. G. Pas- • The appropriate use of the density approximation for obser-
cual. vations reported as exact failures.
Based on the authors’ book Statistical Methods for Relia- • The effect that sample size has on confidence interval width
bility Data, Second Edition, John Wiley & Sons Inc., to be and the likelihood shape.
published in 2019.
• How to make exponential distribution inferences with zero-
June 7, 2018 failures.
8h 15min 7-1 7-2
Example: Times Between α-Particle Emissions of

Americium-241 (Berkson 1966)
Berkson (1966) investigates the randomness of α-particle

emissions of Americium-241, which has a half-life of about
458 years.
Chapter 7
Segment 1 Data: Interarrival times (units: 1/5000 seconds).
• n =10,220 observations.
Likelihood for Interval-Censored Data
Times Between α-Particle Arrivals • Data binned into intervals from 0 to 4000 time
units. Interval sizes ranging from 25 to 100 units. Ad-
ditional interval for observed times exceeding 4,000 time
units.
• Smaller samples analyzed here to illustrate sample size ef-

fect. We start the analysis with n =200.
7-3 7-4
Histogram of the n = 200 Sample of α-Particle

Data for α-Particle Emissions of Americium-241 Interarrival Time Data
Interarrival Times
Time Frequency of Occurrence
Interval Endpoint All Times Random Samples of Times

40
lower upper n = 10220 n = 200 n = 20

tj−1 tj dj dj dj
30
0 100 1609 41 3
Frequency
100 300 2424 44 7

20
300 500 1770 24 4

500 700 1306 32 1
700 1000 1213 29 3
10
1000 2000 1528 21 2

2000 4000 354 9 0
4000 ∞ 16 0 0
0
10220 200 20 0 1000 2000 3000 4000

1/5000 Seconds
7-5 7-6
Exponential Probability Plot of the n = 200 Sample of Parametric Likelihood
α-Particle Interarrival Time Data. The Plot also Probability of the Data
Shows Approximate 95% Simultaneous Nonparametric
Confidence Bands.
• Using the model Pr(T ≤ t) = F (t; θ ) for continuous T ,
the likelihood (probability) for a single observation in the
interval (ti−1, ti] is
.99
Li(θ ; datai) = Pr(ti−1 < T ≤ ti) = F (ti; θ ) − F (ti−1; θ ).
−
.98 Can be generalized to allow for explanatory variables, mul-

tiple sources of variability, and other model features.
.95
• The total likelihood is the joint probability of the data. As-

Probability
−
.9 suming n independent observations
−
n

.8 − L(θ ) = L(θ ; DATA) = C Li(θ ; datai).
−
.7 i=1
−
.6 −
−
.5
.4
.3 − −
−
• As explained in Chapter 2, we take C = 1.
.2
.1 −
• We will find values of θ to make L(θ ) large.

0 500 1000 1500 2000
1/5000 Seconds
7-7 7-8
for the n = 200 α-Particle Interarrival

R(θ) = L(θ)/L(θ)
Exponential Distribution and Likelihood
Time Data. Vertical Lines Give an Approximate 95%
for Interval Data
Likelihood-Based Confidence Interval for θ
Data: α-particle emissions of americium-241
1.2
• The exponential distribution cdf is Estimate of the mean using all n=10220 times →

t
F (t; θ) = 1 − exp −
, t > 0. 1.0
θ
Relative Likelihood R (θ)
where θ = E(T ) is the mean time between arrivals. 0.8 0.50
Confidence Level
0.60
• The interval-data likelihood for the α-particle data is
0.6 0.70
200
200

L(θ) = Li(θ) = F (ti; θ) − F (ti−1; θ) 0.80
0.4 n=200 →
i=1 i=1
8
d 8 "

tj−1

tj
#d
j 0.90
= F (tj ; θ) − F (tj−1; θ) = exp − − exp − 0.2
j=1 j=1 θ θ 0.95
where dj is the number of interarrival times in interval j 0.0

0.99
(i.e., the number of times between tj−1 and tj ). 200 300 400 500 600 700 900
Mean Time Between Arrivals θ
7-9 7 - 10
Exponential Probability Plot for the n = 200 Sample of

α-Particle Interarrival Time Data. The Plot Also
Shows Parametric Exponential ML Estimate and 95%
Confidence Intervals for F (t).
.99
.98
Chapter 7
Segment 2
.95
Likelihood as a Tool for Modeling and Inference

Probability
.9
and Methods for Confidence Intervals
.8
.7
.6
.5
.4
.3
.2 θ^ = 572.3
.1
0 500 1000 1500 2000
1/5000 Seconds
7 - 11 7 - 12
Likelihood as a Tool for Modeling and Inference Likelihood as a Tool for Modeling and Inference
(Continued)
What can we do with the (log) likelihood?
n
• Regions of high likelihood are credible; regions of low likeli-
L(θ ) = log[L(θ )] = Li(θ ).
hood are not credible (suggests confidence regions for pa-
i=1
rameters).
• Study the surface.

• If the length of θ is > 1 or 2 and interest centers on subset
of θ (need to get rid of nuisance parameters), look at pro-
• Maximize with respect to θ (ML point estimates). files (suggests confidence regions/intervals for parameter
subsets).
• Look at curvature at maximum (gives estimate of Fisher
information and asymptotic variance). • Calibrate confidence regions/intervals with χ2 or simulation
(or parametric bootstrap).
• Observe effect of perturbations in data and model on like-
lihood (sensitivity, influence analysis). • Use reparameterization to study functions of θ .
7 - 13 7 - 14
Large-Sample Approximate Theory for Likelihood for the n = 200 α-Particle Interarrival
R(θ) = L(θ)/L(θ)
Ratios for a Scalar Parameter Time Data. Vertical Lines Give an Approximate 95%
Likelihood-Based Confidence Interval for θ
• Relative likelihood for θ is
L(θ)
R(θ) =
.
L(θ)
1.2
Estimate of the mean using all n=10220 times →
• If evaluated at the true θ, then, asymptotically, −2 log[R(θ)]
follows, a chi-square distribution with 1 degree of freedom. 1.0
• An approximate 100(1 − α)% likelihood-based confidence

0.8 0.50
region for θ is the set of all values of θ such that
Confidence Level
0.60
−2 log[R(θ)] < χ2
(1−α;1) 0.6 0.70
or, equivalently, the set defined by
0.4 n=200 →
0.80
R(θ) > exp −χ2
(1−α;1)/2 .
0.90
• A one-sided approximate 100(1 − α)% lower or upper confi- 0.2
0.95
dence bound is obtained by replacing 1 − α with 1 − 2α and
0.99
using the appropriate endpoint. 0.0
200 300 400 500 600 700 900
• General theory in the Appendix.
7 - 15 7 - 16
Wald Confidence Intervals for θ

Wald Confidence Intervals for θ (continued)
• A 100(1 − α)% Wald (or normal-approximation) confidence

• A 100(1 − α)% Wald (or normal-approximation) confidence
interval for θ is
interval for θ is
[θ, θ̃] = θ ± z(1−α/2)seθ.
[θ ,
θ̃] = [θ/w, θ × w]
−1
where seθ = −d2L(θ)/dθ 2 .
is evaluated at θ This follows after transform-
where w = exp[z(1−α/2)seθ/θ].
ing (by exponentiation) the confidence interval
• Based on
θ − θ = log(θ)
[log(θ), log(θ)] ±z
(1−α/2)selog(θ)

Zθ = ∼
˙ NORM(0, 1)
seθ which is based on
− log(θ)
log(θ)
• From the definition of NORM(0, 1) quantiles Zlog(θ) ∼
= ˙ NORM(0, 1)
selog(θ)

Pr z(α/2) < Zθ ≤ z(1−α/2) ≈ 1 − α
• Because log(θ) is unrestricted in sign, Z
implies that is usually
log(θ)
closer to an NORM(0, 1) distribution than is Zθ.
Pr θ − z(1−α/2)seθ < θ ≤ θ + z(1−α/2)seθ ≈ 1 − α.
7 - 17 7 - 18
Comparison of Confidence Intervals for the α-Particle
Confidence Intervals for Functions of θ
Data
All Times Sample of Times
• For one-parameter distributions, confidence intervals for n =10,220 n = 200 n=20
θ can be translated directly into confidence intervals for
monotone functions of θ. Mean Time Between Arrivals θ
ML Estimate θ 596 572 440

Standard Error se 6.1 41.7 101
θ
• The arrival rate λ = 1/θ is a decreasing function of θ. 95% Confidence Intervals
for θ Based on
[λ, λ̃] = [1/θ̃, 1/θ ] Likelihood [585, 608] [498, 662] [289, 713]
Zlog(
θ)
∼
˙ NORM(0, 1) [585, 608] [496, 660] [281, 690]
= [1/662, 1/498] = [.00151, .00201]. Z ∼˙ NORM(0, 1) [585, 608] [491, 654] [242, 638]
θ
Arrival Rate λ × 105

ML Estimate λ × 105 168 175 227
• F (t; θ) is a decreasing function of θ
5
Standard Error se 1.7 13 52
λ×10
[F (te), F̃ (te)] = [F (te; θ̃), F (te; θ)] 95% Confidence Intervals
for λ × 10 Based on
5
[F (1000), F̃ (1000)] = [1 − exp(−1000/662), 1 − exp(−1000/49
Likelihood [164, 171] [151, 201] [140, 346]
= [0.779, 0.866]. Zlog(
λ)
∼˙ NORM(0, 1) [164, 171] [152, 202] [145, 356]
Z
λ
∼˙ NORM(0, 1) [164, 171] [149, 200] [125, 329]
7 - 19 7 - 20
Density Approximation for Exact Observations
• If ti−1 = ti − Δi, Δi > 0, and the correct likelihood
F (ti; θ ) − F (ti−1; θ ) = F (ti; θ ) − F (ti − Δi; θ )

can be approximated with the density f (t) as
t
Chapter 7 i
[F (ti; θ ) − F (ti − Δi; θ )] = f (t)d t ≈ f (ti; θ )Δi
(ti −Δi )
Segment 3 then the density approximation for exact observations
Density Approximation for “Exact” Failures Li(θ ; datai) = f (ti; θ )

may be appropriate.
• For most common models, the density approximation is ad-

equate for small Δi.
• There are, however, situations where the approximation

breaks down as Δi → 0 (Chapter 10).
7 - 21 7 - 22
ML Estimates for the Exponential Distribution Mean Confidence Interval for the Mean Life of a New
Based on the Density Approximation Insulating Material
• With r exact failures and n − r right-censored observations

the ML estimate of θ is • A life test for a new insulating material used 25 specimens
n
which were tested simultaneously at a high voltage of 30 kV.
TTT ti
θ = = i=1
r r • The test was run until 15 of the specimens failed.

TTT = n i=1 ti , total time in test, is the sum of the failure

times plus the censoring time of the units that are censored. • The 15 failure times (hours) were recorded as:
• Using the observed curvature in the likelihood: 1.15, 3.16, 10.38, 10.75, 12.53, 16.74, 22.54, 25.01, 33.02,
3 33.93, 36.17, 39.06, 44.56, 46.65, 55.93
−1
d2L(θ) θ2 θ

seθ = − 2
= =√ . Then TTT = 1.15 + · · · + 55.93 + 10 × 55.93 = 950.88 hours.
dθ r r
θ
• The ML estimate of θ and a 95% confidence interval are:
• If the data are complete or failure censored, 2TTT /θ ∼ χ2
2r . θ = 950.88/15 = 63.392 hours
Then an exact 100(1 − α)% confidence interval for θ is ⎡ ⎤
" #
⎡ ⎤ 2(950.88) 2(950.88) 1901.76 1901.76
2(TTT ) 2(TTT ) ⎦ θ , θ̃ = ⎣ 2 , ⎦= ,
[θ , θ̃] = ⎣ 2 , . χ(0.975;30) χ2
(0.025;30)
46.98 16.79
χ(1−α/2;2r) χ2
(α/2;2r) = [40.48, 113.26].
7 - 23 7 - 24
Example. α-Particle Pseudo Data Constructed
with Constant Proportion within Each Bin
Interarrival Times
Time Frequency of Occurrence
Interval Endpoint Samples of Times
Chapter 7
lower upper n=20000 n=2000 n=200 n=20
Segment 4 tj−1 tj dj
0 100 3000 300 30 3
Effect of Sample Size on Confidence Interval Width 100 300 5000 500 50 5
and the Likelihood Shape 300 500 3000 300 30 3
500 700 3000 300 30 3
700 1000 2000 200 20 2
1000 2000 3000 300 30 3
2000 4000 1000 100 10 1
4000 ∞ 0000 000 0 0
20000 2000 200 20
7 - 25 7 - 26
for the n = 20, 200, and 2000 Pseudo

R(θ) = L(θ)/L(θ) Relative Likelihood for Simulated
Data. Vertical Lines Give Corresponding Approximate Exponential (θ = 5) Samples of Size n = 3
95% Likelihood-Based Confidence Intervals
1.2 1.0
← Estimate using all
n = 20,000 times
1.0
0.8 0.50
0.60
Confidence Level
0.8 0.50
Confidence Level
0.6 0.70
0.60
0.6 0.70
0.80
n=20 → 0.4
0.80
0.4
← n=2,000 0.90
0.90 0.2
n=200 →
0.2 0.95
0.95
0.99
0.99 0.0
0.0
0 10 20 30 40 50
300 400 500 600 700 900 1100 1300 1600 θ
7 - 27 7 - 28
Relative Likelihood for Simulated

Exponential (θ = 5) Samples of Size n = 1000
Effect of Sample Size on the Likelihood
• In large samples, the curvature at the maximum of the like-

1.0 lihood will be large, resulting in narrow confidence intervals.
0.8 0.50
• In large samples, the log-likelihood can be approximated
0.60
well by a quadratic function.
Confidence Level
0.6 0.70
0.80 • In small samples, the likelihood for the exponential mean

0.4
can be skewed to the right.
0.90
0.2
0.95
• In small samples there can be much variability of the width
0.99
0.0 of confidence intervals.
4.6 4.8 5.0 5.2 5.4 5.6
θ
7 - 29 7 - 30
Exponential Distribution Inferences with Zero Failures
• An ML estimate for the exponential distribution mean θ

cannot be computed unless the available data contains one
or more failures.
• For a sample of n units with running times t1, . . . , tn and an

assumed exponential distribution, a conservative 100(1 −
Chapter 7 α)% lower confidence bound for θ is
Segment 5 2(TTT ) 2(TTT ) TTT

θ= = = .
χ2
(1−α;2)
−2 log(α) − log(α)
Exponential Distribution Inferences with Zero-Failures
• The lower bound θ can be translated into an lower confi-

dence bound for functions like tp for specified p or a upper
confidence bound for F (te) for a specified te.
• This bound is based on the fact that under the exponen-

tial failure-time distribution, with immediate replacement of
failed units, the number of failures observed in a life test
with a fixed total time on test has a Poisson distribution.
7 - 31 7 - 32
Analysis of the Diesel Generator Fan Data Relative Likelihood for the Diesel Generator Fan Data
After 200 Hours of Service After 200 Hours of Service and Zero Failures
• Suppose that all fans were removed from service after 200
hours of operation at which time all of the 70 fans were still
running (zero failures). 1.0
0.60
• Then TTT = 70 × 200=14,000 hours. 0.70

0.8
One Sided Confidence Level

• The likelihood and relative likelihood functions (they are 0.80
Relative Likelihood
identical) are monotone increasing in θ but bounded above 0.6

by 1.
0.90
• The ML estimate does not exist, but it is possible to find 0.4
a lower confidence bound on θ.

0.95
0.2
• The likelihood method could be used, but because it de-
pends on a large-sample approximation (and there are zero 0.99
failures), the procedure might not be trustworthy. 0.0

0 10000 20000 30000 40000 50000
θ
• Fortunately, the conservative method is available.
7 - 33 7 - 34
Conservative Confidence Bound Based on the

Diesel Generator Fan Data
After 200 Hours of Service
• Again, with TTT =14,000 hours, a conservative 95% lower

confidence bound on θ is
2(TTT ) 28000
θ= = = 4674.
χ2
(0.95;2)
5.991
• A conservative 95% upper confidence bound on F (10000; θ)

is F (10000) = F (10000; θ ) = 1 − exp(−10000/4674) =

0.882.
• Using the entire data set, θ = 28,701 and a likelihood-

based approximate 95% lower confidence bound is θ =

18,485 hours.
• Again, using the entire data set, the upper confidence bound
on F (10000; θ) is F (10000) = F (10000; θ ) = 1−exp(−10000/18485

0.4178. This shows how little information is available from
a short test with few or zero failures.
7 - 35
Chapter 8 Chapter 8
Maximum Likelihood
Maximum Likelihood for Log-Location-Scale Distributions
for Log-Location-Scale Distributions Objectives

• How to express the likelihood for location-distributions like
the normal distribution and log-location-scale distributions
like the Lognormal and Weibull distributions.
W. Q. Meeker, L. A. Escobar, and F. G. Pascual • How to construct and interpret likelihood confidence in-
Iowa State University, Louisiana State University, and Wash- tervals for parameters and for functions of parameters of
ington State University. log-location-scale distributions.
cual. • The construction of Wald (normal-approximation) confi-
dence intervals for parameters and functions of parameters
Based on the authors’ book Statistical Methods for Relia- of log-location-scale distributions.
published in 2019.
• Inference for log-location-scale distribution parameters and
June 7, 2018 functions of parameters with a given shape parameter.
8h 15min 8-1 8-2
Weibull Probability Plot of the Shock Absorber Data
Log10 Thousands of Cycles
3.8 3.9 4.0 4.1 4.2 4.3 4.4 4.5
.98
.9
.7
0
Chapter 8 .5
.3
Segment 1 .2
Standard quantile
Proportion Failing
−2
.1
Likelihood for Log-Location-Scale Distributions .05
.03
.02 −4
.01
.005
.003
−6
.001
6000 8000 10000 12000 14000 16000 20000 24000 28000
Kilometers
8-3 8-4
Weibull Distribution Likelihood Lognormal Distribution Likelihood

for Right Censored Data for Right Censored Data
• The Weibull distribution model is • The lognormal distribution model is
Pr(T ≤ t) = F (t; μ, σ) = Φsev{[log(t) − μ]/σ}. Pr(T ≤ t) = F (t; μ, σ) = Φnorm{[log(t) − μ]/σ}.
• The likelihood has the form • The likelihood has the form

n
n
L(μ, σ) = Li (μ, σ; datai ) L(μ, σ) = Li (μ, σ; datai )
i=1 i=1
n
δ 1−δi
n
δ 1−δi
= [f (ti ; μ, σ)] i [1 − F (ti ; μ, σ)] = [f (ti ; μ, σ)] i [1 − F (ti ; μ, σ)]
i=1 i=1
n " #δi " #1−δi n " #δi " #1−δi
1 log(ti ) − μ log(ti ) − μ 1 log(ti ) − μ log(ti ) − μ
= φsev × 1 − Φsev = φnorm × 1 − Φnorm
i=1
σti σ σ i=1
σti σ σ
4 4
1 if ti is an exact observation 1 if ti is an exact observation
δi = δi =
0 if ti is a right censored observation 0 if ti is a right censored observation
φsev(z) is the standardized smallest extreme value density. φnorm(z) is the standardized normal density.
8-5 8-6
Weibull Relative Likelihood Weibull Relative Likelihood
for the Shock Absorber Data for the Shock Absorber Data
ML Estimates: μ = 10.23 and σ = .3164 ML Estimates: μ = 10.23 and σ = .3164
log(σ
R(μ, log(σ)) = L(μ, log(σ))/L(μ, )) σ
R(μ, σ) = L(μ, σ)/L(μ, )
0.7
0.6
0 0.2 0.4 0.6 0.8 1
Relative Likelihood
0.5 0.1
0.01
0.2
0.4
0.4
0.7
0.9
σ
0.3
10
.8
10
.6
10
.4
10 -1.6
μ .2
-1.2
-1.4
10 -1 0.2
-0.8
-0.6 log(σ)
0.001 0.001
9.8 10.0 10.2 10.4 10.6 10.8
μ
8-7 8-8
Weibull Probability Plot of Shock Absorber Failure Lognormal Probability Plots of Shock Absorber Data
Times (Both Failure Modes) with Maximum with ML Estimates and Wald 95% Pointwise
Likelihood Estimates and Wald 95% Pointwise Confidence Intervals for F (t). The Curved Line is the
Confidence Intervals for F (t) Weibull ML Estimate.
.9 0.95
0.9
.7
0.8
.5
0.7
.3 0.6
0.5
.2
0.4
0.3
.1
Proportion Failing
Proportion Failing
0.2
.05 0.1
.03 0.05
.02 0.02
0.01
.01 0.005
0.002
.005 0.001
0.0005
.003 ^ = 27719
η Lognormal Distribution ML Fit
Weibull Distribution ML Fit
0.0001
0.00005 Lognormal Distribution 95% Pointwise Confidence Intervals
^
β = 3.16
.001 0.00001
6000 8000 10000 12000 14000 16000 20000 24000 28000 5000 10000 15000 20000 25000 30000 35000
Kilometers Kilometers
8-9 8 - 10
Large-Sample Approximate Theory for Likelihood

Ratios for Parameter Vector
Chapter 8 • Relative likelihood for (μ, σ) is

L(μ, σ)
Segment 2 R(μ, σ) = .
σ
L(μ, )
Likelihood-Based Confidence Intervals for

Log-Location-Scale Distribution Parameters μ and σ • If evaluated at the true (μ, σ), then, asymptotically, −2 log[R(μ, σ)]
follows, a chi-square distribution with 2 degrees of freedom.
• General theory in the Appendix.
8 - 11 8 - 12
Weibull Relative Likelihood Weibull Likelihood-Based Joint Confidence Regions for
for the Shock Absorber Data μ and σ for the Shock Absorber Data
ML Estimates: μ = 10.23 and σ = .3164 ML Estimates: μ = 10.23 and σ
= .3164
σ
R(μ, σ) = L(μ, σ)/L(μ, ) R(μ, σ) > exp −χ2
(1−α;2)
/2 = α gives a 100(1 − α)% region
0.7 0.7
0.6 0.6
95
0.5 0.1 0.5 90
0.01 99
0.2 70 80
0.4 60
0.4 0.4 50
0.7
25
0.9
σ σ
0.3 0.3
0.2 0.2
0.001 0.001
9.8 10.0 10.2 10.4 10.6 10.8 9.8 10.0 10.2 10.4 10.6 10.8
μ μ
8 - 13 8 - 14
Weibull Profile Likelihood R(μ) (exp(μ) = η =≈ t0.63) Weibull Profile Likelihood R(σ) (σ = 1/β)
for the Shock Absorber
Data for the Shock Absorber
Data
R(μ) = max
σ
L(μ,σ)
,σ
L(μ )
R(σ) = max
μ
L(μ,σ)
,σ
L(μ )
1.0 1.0
0.8 0.50 0.8 0.50

Confidence Level
Confidence Level
Profile Likelihood
Profile Likelihood
0.60 0.60
0.6 0.70 0.6 0.70
0.80 0.80
0.4 0.4
0.90 0.90
0.2 0.2
0.95 0.95
0.99 0.99
0.0 0.0
9.8 10.0 10.2 10.4 10.6 10.8 0.2 0.3 0.4 0.5 0.6
μ σ
8 - 15 8 - 16
Large-Sample Approximate Theory of Likelihood

Large-Sample Approximate Theory for Likelihood Ratios – Continued
Ratios for Parameter Vector Subset
Need: Inferences on subset θ 1, from the partition θ = (θ 1, θ 2). • An approximate 100(1 − α)% likelihood-based confidence
region for θ 1 is the set of all values of θ 1 such that
• k1 = length(θ 1). −2 log[R(θ 1)] < χ2

(1−α;k1)
or, equivalently, the set defined by
• When (θ 1, θ 2) = (μ, σ), profile likelihood for θ 1 = μ is

L(μ, σ) R(θ 1) > exp −χ2
(1−α;k1) /2 .
R(μ) = max
σ .
σ
L(μ, )
• Transformation of θ 1 will not affect the confidence state-

• If evaluated at the true θ 1 = μ, then, asymptotically, −2 log[R(μ)]
ment.
follows, a chi-square distribution with k1 = 1 degrees of
freedom.
• General theory in the Appendix. • Can improve the asymptotic approximation with simulation
(only small effect except in very small samples).
8 - 17 8 - 18
Confidence Regions and Intervals for
Functions of μ and σ
• The likelihood approach can be applied to functions of pa-

rameters. For monotone functions of a single parameter
(e.g., β = 1/σ), the interval translates directly.
Chapter 8
Segment 3 • Otherwise, define the function of interest as one of the

parameters, replacing one of the original parameters giving
Likelihood-Based Confidence Intervals one-to-one reparameterization g (μ, σ) = [g1(μ, σ), g2(μ, σ)].
for Functions of μ and σ
• Then use a profile likelihood, as with the original parame-

ters.
• Simple to implement if the function and its inverse are easy

to compute.
8 - 19 8 - 20
Contour Plot of Weibull Relative Likelihood R(t0.1, σ)

(Parameterized with t0.1 and σ)
Reparameterization to Make tp a Parameter R(t0.1, σ) = L(t0.1, σ)/L(t0.1, σ

)
• We want to re-express the likelihood so that

tp = exp[μ + σΦ−1(p)] replaces μ in the likelihood.
0.01
0.6
• This can be done by substituting μ = log(tp) − σΦ−1(p) for

μ in the (log)-likelihood expression, giving an expression for 0.5
L(tp, σ).
σ
0.4
• A similar reparameterization is possible for writing the like- 0.3 0.9

0.7 0.4 0.2 0.1
lihood as a function of F (te) and σ for a given te.

0.2
6000 10000 14000 18000
t0.1
8 - 21 8 - 22
Weibull Profile Likelihood R(t0.1)

Weibull Profile Likelihood R[F (10000)]
for the Shock Absorber
" Data
#
⎧
for the Shock Absorber Data ⎫
R(t0.1) = max L(t0.1,σ)
⎨ ⎬
σ )
L(t0.1,σ max L[F
(10000),σ]
R[F (10000)] = σ
⎩ L F (10000),σ
⎭
1.0
1.0
0.8 0.50
0.8 0.50
Confidence Level
Profile Likelihood
0.60
Confidence Level
Profile Likelihood
0.60
0.6 0.70 0.6 0.70
0.80 0.80
0.4 0.4
0.90 0.90
0.2 0.2
0.95 0.95
0.99
0.99 0.0
0.0
0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14
6000 8000 10000 12000 14000 16000 18000 20000
F(10000)
t0.1
8 - 23 8 - 24
Large-Sample Approximation Theory of ML Estimation
denote the ML estimator of θ .

Let θ
• If evaluated at the true value of θ , then asymptotically,

has a MVN(θ , Σ ) and thus the Wald
(large samples) θ
Chapter 8
θ
statistic
−1
Segment 4 − θ ) Σ − θ)
(θ (θ
θ
Wald Approximate Confidence Intervals for has a chi-square distribution with k degrees of freedom,
Log-Location-Scale Distribution Parameters μ and σ where k is the length of θ .
and Functions of μ and σ
• Here, Σ = Iθ−1 is the large-sample approximate covariance
θ
matrix where the Fisher information matrix for θ is

∂ 2L(θ )
Iθ = E − .
∂ θ∂ θ
• For (log)-location-scale distributions, θ = (μ, σ).
8 - 25 8 - 26
Large-Sample Approximation Theory Large-Sample Approximation Theory

for Wald’s Statistic for Wald’s Statistic – Continued
• Alternative asymptotic theory is based on the large-sample • An Wald approximate 100(1 − α)% confidence region for θ
distribution of quadratic forms (Wald’s statistic). is the set of all values of θ in the ellipsoid
−1
− θ ) Σ̂
(θ − θ ) ≤ χ2
(θ
θ (1−α;k).
• Let Σ̂ be a consistent estimator of Σ , the asymptotic
θ θ
. For example,
covariance matrix of θ
• This is sometimes known as the normal-approximation con-
−1
∂ 2L(θ ) fidence region.
Σ̂ = −
θ ∂ θ∂ θ
.
where the derivatives are evaluated at θ • Can specialize to functions or subsets of θ .
• Asymptotically, the Wald statistic • Wald confidence intervals are not transformation invariant.
−1
− θ ) Σ̂ − θ) Thus there are multiple ways to compute a Wald interval.
w(θ ) = (θ (θ
θ
when evaluated at the true θ , follows a chi-square distri-
bution with k degrees of freedom, where k is the length • Can try to find a transformation that results in a log-
of θ . likelihood with approximate quadratic shape.
8 - 27 8 - 28
Wald Confidence Intervals

for μ and σ
Wald Confidence Intervals
for a Function g1 = g1(μ, σ)
• Estimated variance matrix for the shock absorber data

! μ)
Var( ! μ,
Cov( σ) .01208 .00399
Σ̂μ,σ = ! μ, ! σ = • ML estimate g1 = g1(μ,
σ ).
Cov( σ
) Var( ) .00399 .00535
• Assuming Zg1 = (g1 − g1)/seg1 ∼
˙ NORM(0, 1), an approxi-
• Assuming that Zμ = (μ−μ)/
seμ ∼
˙ NORM(0, 1) distribution, mate 100(1 − α)% confidence interval for g1 is
an approximate 100(1 − α)% confidence interval for μ is
[g 1, g̃1] = g1 ± z(1−α/2)seg1 ,
[μ, ± z(1−α/2)s
μ̃] = μ eμ
where
*
! μ). * 2 2 1
where seμ = Var( ∂g1 ∂g1 ∂g1 ∂g1 !
2
g =
se ! g1) =
Var( ! μ
Var( ) + ! σ
Var( ) + 2 , σ
Cov(μ )
1
∂μ ∂σ ∂μ ∂σ
)−log(σ)]/s
• Assuming that Zlog(σ) = [log(σ elog(σ) ∼
˙ NORM(0, 1)
an approximate 100(1 − α)% confidence interval for σ is • Partial derivatives evaluated at μ,
σ.
[σ , /w,
σ̃] = [σ × w]
σ • General theory in the appendix.
*

where w = exp z(1−α/2)seσ /σ
and s
eσ = ! σ
Var( ).
8 - 29 8 - 30
Wald Confidence Interval for F (te; μ, σ)–Continued
Note: Wald confidence intervals depend on the parameter-

Wald Confidence Interval for F (te; μ, σ) ization used to derive the intervals.
For example, an approximate 100(1−α)% confidence interval

Objective: Obtain a point estimate and a confidence interval for F (te; μ, σ) can be obtained using:
for Pr(T ≤ te) = F (te; μ, σ) at a given point te.
• The asymptotic normality of ZF = (F − F )/seF

= (μ,
• The ML estimates θ σ ) and Σ̂ are available.
θ [F , F̃ ] = F (te) ± z(1−α/2)seF .

• The ML estimate for F (te; μ, σ) is
F = F (te; μ,
σ ) = Φ(ze ) • The asymptotic normality of ze = [log(te) − μ]/
σ
where ze = [log(te) − μ]/

σ . [ze, z5e] = ze ± z(1−α/2)seze .
5
Then
• There are many ways to obtain a Wald confidence interval
for F (te; μ, σ). [F (te), F (te)] = [Φ(ze), Φ(z5e)].
5
• Expressions for seF and seze are obtained by using the delta
method.
8 - 31 8 - 32
Wald Confidence Interval for F (te; μ, σ)—Continued

Example: Bearing-A Life Test Results
Comments:
• Continuous-run test for a newly designed bearing.
• The confidence interval procedure based on the asymptotic

normality of ZF has poor statistical properties because ZF • Sample of 12 units put on test; one early removal; 3 failures.
convergence slowly toward normality.
• Test terminated at 1100 thousand cycles.
• The confidence interval procedure based on ze has bet-
ter statistical properties because ze converges to normality
• What is the failure-time distribution of the bearing?
faster than ZF .
8 - 33 8 - 34
Bearing-A Weibull Probability Plot and ML Estimate Bearing-A Weibull Probability Plot and ML Estimate
.9 .9
.7 .7
.5 .5
.3 .3
Proportion Failing
Proportion Failing
.2 .2
.1 .1
.05 .05
.03 .03
.02 .02
^ = 3417
η ^ = 3417
η
.01 .01
^ ^
β = 0.984 β = 0.984
.005 .005
200 300 400 500 600 700 900 1100 200 300 400 500 600 700 900 1100
Thousands of Cycles Thousands of Cycles
8 - 35 8 - 36
Bearing-A Life Test Example Conclusions
• Sometimes the ML estimate does not go through the points

on a Weibull (or other) probability plot. Chapter 8
Segment 5
• When the ML estimate does not go through the points, it
is an indication that the Weibull distribution does not agree Weibull Distribution Inference
with the data. with Few Failures
• It is important to find out the reason that the line does not
fit.
8 - 37 8 - 38
Weibull Inference with Few Failures Bearing-Cage Fracture Field Data
• Suppose that β is given. Knowledge of the failure mecha- • n = 1703 units had been introduced into service over time;
nism will often provide information about β. oldest unit had 2220 hours of operation.
• Simplifies problem. Only one parameter with r failures and • 6 units had failed observed.
t1, . . . , tn failures and censor times
⎛
⎞1/β 3
β
nt η̂ 1 • Design life specification was B10=t0.1 = 8000 hours.
η̂ = ⎝ i=1 i ⎠ , seη̂ = .
r β r
• ML estimate is t0.1= 3.903 thousand hours. Does this in-

• Provides much more precision, especially with small r. %be- dicate a problem?
ginitemize
• Requires sensitivity analysis because β is unknown. • How many replacement parts will be needed?
8 - 39 8 - 40
Weibull Probability Plots Bearing-Cage Fracture Data

Bearing-Cage Fracture Data Event Plot with Weibull ML Estimates and Sets of 95% Pointwise
Confidence Intervals for F (t) with β Estimated.
Bearing Cage Failure Data

Count
Row
1 288 .9
2 148
3 .7
4 124 .5
5
6 111 .3
7 .2
8 106
9 99 .1
10 110
.05
Proportion Failing
11 114
12 119 .03
13 127 .02
14
15 .01
16 123 .005
17 93
18 47 .003
19 41
20 27
21
.001
22 11 .0005
23 6
24 .0003
25 2 .0002
^ = 11792
η
.0001
.00005 ^
β = 2.035
0 500 1000 1500 2000 .00003
Hours 100 200 500 1000 2000 5000 10000
Hours
8 - 41 8 - 42
Weibull Probability Plots Bearing-Cage Fracture Data Weibull Probability Plots Bearing-Cage Fracture Data
with Weibull ML Estimates and Sets of 95% Pointwise with Weibull ML Estimates and Sets of 95% Pointwise
Confidence Intervals for F (t) with Given β = 1.5. Confidence Intervals for F (t) with Given β = 2.
.9 .9
.7 .7
.5 .5
.3 .3
.2 .2
.1 .1
.05 .05
Proportion Failing
Proportion Failing
.03 .03
.02 .02
.01 .01
.005 .005
.003 .003
.001 .001
.0005 .0005
.0003 .0003
.0002 .0002
^ = 28987
η ^ = 12320
η
.0001 .0001
.00005 β = 1.5 .00005 β=2
.00003 .00003
100 200 500 1000 2000 5000 10000 100 200 500 1000 2000 5000 10000
Hours Hours
8 - 43 8 - 44
Weibull Probability Plots Bearing-Cage Fracture Data Lognormal and Weibull Comparison
with Weibull ML Estimates and Sets of 95% Pointwise Bearing-Cage Fracture Field Data
Confidence Intervals for F (t) with Given β = 3. Lognormal Probability Paper
.9 .8
.7 .7
.5 .6
.3 .5
.2 .4
.1 .3
.05 .2
Proportion Failing
.03
Proportion Failing
.02 .1
.01 .05
.005 .02
.003
.01
.005
.001
.002
.0005 .001
.0003 .0005
.0002
^ = 5408
η Lognormal Distribution ML Fit
.0001 .0001 Weibull Distribution ML Fit
.00005 Lognormal Distribution 95% Pointwise Confidence Intervals
.00005 β=3
.00003 .00001
100 200 500 1000 2000 5000 10000 100 200 500 1000 2000 5000 10000
Hours Hours
8 - 45 8 - 46
Relative Weibull Likelihood Relative Weibull Likelihood

with One Failure at 1 and One Survivor at 2 with One Failure at 1.9 and One Survivor at 2
1.0
14
12 0.8
0.1
10
0.1
0.6
sigma
sigma
6 0.4
0.2
0.2
4
0.3
0.3 0.2
2 0.5 0.5
0.9 0.9
0 0.0
-2 2 6 10 14 0.6 1.0 1.4 1.8
mu mu
8 - 47 8 - 48
Weibull Distribution with Given β
and Zero Failures
• ML estimate for the Weibull scale parameter η cannot be

computed unless the available data contains one or more
failures.
Chapter 8
• For a sample of n units with running times t1, . . . , tn and no
Segment 6 failures, a conservative 100(1−α)% lower confidence bound
for η is
Log-Location-Scale Distribution Inference
⎛
⎞1
with Zero Failures β
2 n ti β
η= ⎝ i=1 ⎠ .
χ2
(1−α;2)
• The lower bound η can be translated into an lower confi-

dence bound for functions like tp for specified p or a upper
confidence bound for F (te) for a specified te.
8 - 49 8 - 50
Component A Safe Data Weibull Model 95% Upper Confidence Bounds on

F (t) for Component-A with Different Fixed Values for
the Weibull Shape Parameter
• A metal component in a ship’s propulsion system fails from
fatigue-caused fracture.
0
• Because of persistent reliability problems, the component F(t)
was redesigned to have a longer service life. 1

.5
• Previous experience suggests that the Weibull shape param-

.3
.2
eter is near β = 2, and almost certainly between 1.5 and 2
2.5. .1
3
• Many copies of a newly designed component were put into
.05
.03
service during the past year and no failures have been re- .02
4
ported. β = 1.5
.01
5 .005 β = 2.0
Hours: 500 1000 1500 2000 2500 3000 3500 4000 .003
β = 2.5
Number of Units: 10 12 8 9 7 9 6 3 6
Staggered entry data, with no reported failures. .001
7 .0005
• Can the replacement time be safely increased from 2000 500 1000 1500 2000 2500 3000 3500 4500
hours to 4000 hours? 8 Hours
8 - 51 8 - 52
Regularity Conditions
• Each technical result (e.g., asymptotic distribution of an

estimator) has its own set of conditions on the model (see
Lehmann 1983, Rao 1973).
• Frequent reference to Regularity Conditions which give rise

to simple results.
Chapter 8
• For special cases the regularity conditions are easy to state
Segment 7 and check. For example, for some location-scale distribu-
tions the needed conditions are:
Regularity Conditions and Other Topics
z 2φ2(z)
lim =0
Φ(z)
z→−∞
2 2
z φ (z)
lim = 0.
z→+∞ 1 − Φ(z)
• In non-regular models, asymptotic behavior is more com-

plicated (e.g., behavior depends on θ ), but there are still
useful asymptotic results.
8 - 53 8 - 54
Regularity Conditions – Continued Other Topics Related to Parametric Likelihood
Covered in the Book
Some typical regularity conditions include:

• Bayesian methods (Chapter 9).
• Support does not depend on unknown parameters.

• Truncated data (Chapter 10).
• Number of parameters does not grow too fast with n.

• Threshold parameters (Chapter 10).
• Continuous derivatives of log likelihood (w.r.t. θ ).

• Other distributions (e.g., generalized gamma) (Chapter 10).
• Bounded derivatives of likelihood. • Prediction (Chapter 14).
• Can exchange the order of differentiation of log likelihood • Multiple failure modes (Chapter 15).
w.r.t. θ and integration w.r.t. data.
• Regression anaysis and accelerated testing (Chapters 16-

• Identifiability. 18).
8 - 55 8 - 56
Chapter 15
Analysis of Data with More than One Failure Mode Analysis of Data with More than One Failure Mode

• Examples, model, data and likelihood for describing multiple-
failure-mode data.
W. Q. Meeker, L. A. Escobar, and F. G. Pascual

Iowa State University, Louisiana State University, and Wash- • Maximum likelihood estimation, illustrated by estimating
ington State University. the MTTF of Device-G.

cual. • Estimating the Shock Absorber failure-time distribution and
when it is important to use failure-mode information.
published in 2019. • Other topics related to multiple failure modes.
June 7, 2018
8h 16min 15 - 1 15 - 2
Examples of Products With Two or More Causes of

Failure
(or Multiple Failure Modes)
Many units, systems, subsystems, or components have more

than one cause of failure. For example:
Chapter 15
• Laptop computer failure modes include keyboard, screen,
Segment 1 processor, main memory, solid-state drive, and power sup-
ply.
Examples
• A shock absorber may fail from a broken spring or a leak in
Model
the casing.
Data and Likelihood
• A semiconductor device can fail at a junction or at a lead.
• A hard drive can fail because a manufacturing defect (infant

mortality) or because of mechanical wearout.
• For an automobile tire, tread can wearout or the tire may

suffer a puncture.
15 - 3 15 - 4
Basic Ideas Behind the Analysis of Data with Multiple Multiple Failure Modes Data and
Failure Modes the Series-System Model
• There is a joint distribution of failure times for all failure • Let Tj be the lifetime from failure mode j, j = 1, . . . J.
modes.
• The failure-time of the system is
• Usually, only the first failure is observed. T = min{T1, . . . TJ }
• When the failure modes are independent and have a con-

• If the failure times for the individual failure modes are inde- tinuous distribution, the cdf for the failure-time is
pendent, then analysis is relatively simple. J

F (t) = 1 − [1 − Fj (t)].
j=1
• When only the first failure is observed and no parametric
where Fj (t) is the marginal cdf for mode j and
distribution assumption is made, there is no information
fj (t) = dFj (t)/d t is the marginal pdf for mode j.
in the data about the nature of the dependency.
• The observed data are (ti, κi), i = 1, . . . , n where ti is a given
time for observation i and
• When only the first failure is observed and a parametric 4
distribution is assumed, there is some information in the j if the cause of failure i was mode j
κi =
data about the nature of the dependency (but not much). 0 if the unit i was censored
15 - 5 15 - 6
Analysis of Data with Multiple Failure Modes
Assuming Independence
• When failure times for the different failure modes are in-
dependent, it is possible to analyze the individual failure
modes separately.
Chapter 15
• Create a separate data set, with n cases, for each failure
mode. Segment 2
• For failure mode j, the likelihood is Estimating the MTTF of Device-G

n
δ 1−δ
i i
L(μj , σj ) = fj (ti; μj , σj ) 1 − Fj (ti; μj , σj )
i=1
where
4
1 if κ = j
δi =
0 otherwise
15 - 7 15 - 8
Device-G Data
Device-G Background
Thousands Failure Thousands Failure Thousands Failure

• Design life of 300 thousand cycles.
of Cycles Mode of Cycles Mode of Cycles Mode
• Units were failing in the field more rapidly than had been 275 W 106 S 88 S
expected. 13 S 300 – 247 S
147 W 300 – 28 S
• Needs: information on how to improve reliability and an 23 S 212 W 143 S
estimate of device MTTF. 181 W 300 – 300 –
30 S 300 – 23 S
• Mode W failures, caused by normal product wear, began to 65 S 300 – 300 –
appear after 100 thousand cycles of use. 10 S 2 S 80 S
300 – 261 S 245 W
• Mode S failures were caused by failures on an electronic 173 S 293 W 266 W
component due to electrical surge. These failures predom-
W indicates a wearout failure, S indicates an electrical
inated early in life.
surge failure, and – indicates a unit still operating after
300 thousand cycles.
15 - 9 15 - 10
Weibull Analyses of Device-G Data Estimating Time

Event Plot of Device-G Data
to Failure Ignoring the Cause of Failure
Row
.98
1 W
2 S
3 W
.9
4 S
5 W
6 S
7 S .7
8 S
9
10 S .5
11 S
12
13
Proportion Failing
.3
14 W
15
16 .2
17
18 S
19 S
20 W .1
21 S
22 S
23 S
24 S .05
25
26 S
.03
27
28 S ^ = 242.6
η
29 W .02
30 W
^
β = 0.9268
.01
0 50 100 150 200 250 300
1 2 5 10 20 50 100 200 500
Kilocycles
Kilocycles
15 - 11 15 - 12
Weibull Analyses of Device-G Data
Individual Failure Modes
Series-System Model for Device G
• Let TS be the lifetime from the electrical surge failure mode .98 Surge
Wearout
(S) and TW be the lifetime from the wearout failure mode (W). .9
• The failure-time of the product is .7
.5
T = min(TS, TW)
Proportion Failing
.3
.2
• When the failure modes are independent, the cdf for the
failure-time is .1
F (t) = Pr(T ≤ t) = 1 − Pr(T > t) = 1 − Pr(TS > t ∩ TW > t) .05
= 1 − Pr(TS > t) Pr(TW > t) .03
.02
= 1 − [1 − FS(t)][1 − FW(t)].
.01
1 2 5 10 20 50 100 200 500
Kilocycles
15 - 13 15 - 14
Weibull Analyses of Device-G Data Weibull Analyses of Device-G Data Estimating Time
Individual Failure Modes to Failure Using Series System Model
2 8
.98 Surge .98 Surge

Wearout Wearout
.9 .9 Combined
.7 .7
.5 .5
Proportion Failing
Proportion Failing
.3 .3
.2 .2
.1 .1
.05 .05
.03 .03
.02 .02
.01 .01
1 2 5 10 20 50 100 200 500 1 2 5 10 20 50 100 200 500
Kilocycles Kilocycles
15 - 15 15 - 16
Device-G Field-Tracking Data

Weibull Distribution Models for
ML Weibull Distribution Estimation Results for the
the Device-G Data
Electric Surge (S) and Wearout (W) Failure Modes
• Failure times for each of the two failure modes modeled

with a separate Weibull distribution. That is, 95% Approximate
( ) ML Standard Confidence Interval
log(t) − μi
Fi(t) = Φsev , i = S, W. Mode Parameter Estimate Error Lower Upper
σi
S μS 6.11 0.427 5.27 6.95
• The series model for two independent failure modes acting σS 1.49 0.35 0.94 2.36
together is
W μW 5.83 0.11 5.62 6.04
F (t) = 1 − [1 − FS(t)] × [1 − FW(t)]. σW .23 .08 0.12 0.44
• The model ignoring the cause of failure information is S & W μSW 5.49 0.23 5.04 5.94
( )
log(t) − μSW σSW 1.08 .21 0.74 1.57
F (t) = Φsev .
σSW For Mode S alone, LS = −101.36 for Mode W alone, LW =
−47.16, and for both modes together, LSW = −142.62.
15 - 17 15 - 18
Some Comments on the
Weibull Analyses of Device-G Data
• The Weibull distribution provides a good fit to both the S

failure mode and the W failure mode data.
• Weibull analysis ignoring the cause of failure information Chapter 15

shows evidence of a change in the slope of the plotted
points, indicating a gradual shift from one failure mode to Segment 3
another.
Estimating the Shock Absorber Failure-Time
• The Weibull cdf estimate obtained from ignoring the cause Distribution
of failure and the series-system cdf estimate for the two fail- and When it is Important to Use Failure-Mode
ure modes acting together diverge rapidly after 200 thou- Information
sand cycles.
=
• Estimates of the mean time to failure computed from MTTF
∞ (t)]dt were 251.3 and 196.0 thousands of cycles,
0 [1 − F T
for the models ignoring and using the failure mode informa-
tion, respectively.
15 - 19 15 - 20
Shock Absorber Failure Data

Event Plot of Shock Absorber Data
First reported in O’Connor (1985).
Shock Absorber Data (Two Failure Modes)
• Failure times, in number of kilometers of use, of vehicle

Row
1 1
shock absorbers. 2
3
4
5 2
6
7
8
9
10
• Two failure modes, denoted by Mode 1 and Mode 2. 11
12
13 1
14
15 2
16
17
18
Unit
19 1
20 1
• Need to estimate the failure-time distribution for the shock 21
22
23
24
absorbers. 25
26
27 2
28
29
30
31 2
32 1
33
• One might be interested in the failure-time distribution for 34 1
35
36 1
37
38
Mode 1 (e.g., after Mode 2 has been eliminated).

0 5000 10000 15000 20000 25000 30000
Kilometers
Mode 2 (e.g., after Mode 1 has been eliminated).
15 - 21 15 - 22
Weibull Analyses of Shock Absorber Data Estimating Weibull Analyses of Shock Absorber Data
Time to Failure Ignoring the Cause of Failure Individual Failure Modes
.9 .9 Mode1
Mode2
.7 .7
.5 .5
.3 .3
.2 .2
.1 .1
Proportion Failing
Proportion Failing
.05 .05
.03 .03
.02 .02
.01 .01
.005 .005
.003 ^ = 27719
η .003
^
β = 3.16
.001 .001
6000 8000 10000 12000 14000 16000 20000 24000 28000 6000 8000 10000 12000 14000 16000 20000 24000 28000
15 - 23 15 - 24
Weibull Analyses of Shock Absorber Data Weibull Analyses of Shock Absorber Data Estimating
Individual Failure Modes Time to Failure Using Series System Model
.9 Mode1 .9 Mode1
Mode2 Mode2
.7 .7 Combined
.5 .5
.3 .3
.2 .2
.1 .1
Proportion Failing
Proportion Failing
.05 .05
.03 .03
.02 .02
.01 .01
.005 .005
.003 .003
.001 .001
6000 8000 10000 12000 14000 16000 20000 24000 28000 6000 8000 10000 12000 14000 16000 20000 24000 28000
15 - 25 15 - 26
Weibull Analyses of Shock Absorber Data Estimating Weibull Analyses of Shock Absorber Data Estimating
Time to Failure Ignoring the Cause of Failure Time to Failure Using Series System Model
.9 .9 Mode1
Mode2
.7 .7 Combined
.5 .5
.3 .3
.2 .2
.1 .1
Proportion Failing
Proportion Failing
.05 .05
.03 .03
.02 .02
.01 .01
.005 .005
.003 ^ = 27719
η .003
^
β = 3.16
.001 .001
6000 8000 10000 12000 14000 16000 20000 24000 28000 6000 8000 10000 12000 14000 16000 20000 24000 28000
15 - 27 15 - 28
When is it Important to Take Account of

Failure-Mode Information?
• If there is more than one failure mode, it is important to

separate and analyze failure modes separately when:
Shape parameters are importantly different (e.g., when

there is both infant mortality and wearout).
Chapter 15
There is need to assess the impact of eliminating a failure
mode. Segment 4
There is a need to predict causes of future failures to as- Other Topics Related to Multiple Failure Modes
sure that a sufficient number of appropriate replacement
parts are available.
• If interest is on the failure time distribution of the entire

system and that distribution can be adequately described
by a simple distribution, failure-mode information can be
ignored.
15 - 29 15 - 30
Estimation When Failure Mode is Identified for Only
Some Failures Effect of Dependency Among Failure Modes
When failure modes are not identified or are only partially • The common assumption of independent failure modes is
identified for some units, it is still possible to estimate the sometimes unrealistic.
individual Fi(t) distributions by using maximum likelihood.
When there is dependence, still one can use the relationship
F (t) = Pr(T ≤ t) = 1 − Pr(T1 > t ∩ T2 > t)

• Known as masking of failure modes.
but the evaluation has to be done with respect to the bi-
variate joint distribution of T1 and T2.
• More difficult because the analysis is not separable.
• Usually, when there is dependence, the dependence is posi-
tive. Then long (short) failure times of one mode tend to
• Parameter estimates for the distribution for one mode will go with long (short) failure times of another.
be correlated with those of the other modes.
• If the failure modes are positively dependent, then attempt-
ing to predict the effect of eliminating one of the failure
• In practice, one is likely to analyze the data as if there were modes, using the independent failure mode model, can give
only a single mode. Potentially misleading if extrapolating incorrect and overly optimistic predictions.
outside the range of the data.
15 - 31 15 - 32
Maximum Likelihood Estimation of System Reliability

Modeling Dependency Among Failure Modes
The system cdf and other related functions can be esti-
mated using ML estimates for the components.
• Dependent multiple failure modes can be modeled by spec-
ifying marginal distributions for the different failures modes
along with a “copula” function to describe the dependence. be the ML estimate of θ , and Σ̂ the ML estimate of
• Let θ
θ
Σ obtained from the component data. Then using same
θ
methods as in previous chapters
• There is no information in the data to help specify the form
) = g[F (θ
F̂T = FT (θ
1 ), . . . , Fs (θ s )]
of the copula.

! F̂ ) = ∂FT ∂FT
• A combination of external information and sensitivity anal- Var( T Σ̂
∂θ θ ∂θ
ysis can be used to help specify a copula. .
where the derivatives are evaluated at θ
• Bayesian methods would be useful to describe uncertainty

• Confidence interval methods can be computed using these
in copula parameters.
results (several alternative methods are available) or using
the likelihood method.
15 - 33 15 - 34
Chapter 16 Chapter 16
Failure-Time Regression Analysis
Failure-Time Regression Analysis Objectives
• Describe applications of failure-time regression.
• Describe graphical methods for displaying censored regres-

sion data.
W. Q. Meeker, L. A. Escobar, and F. G. Pascual • Describe simple regression models to relate life to explana-
Iowa State University, Louisiana State University, and Wash- tory variables.
Copyright 2018 W. Q. Meeker, L. A. Escobar, and F. G. Pas- • Illustrate the use of likelihood methods for censored regres-
cual. sion data.

bility Data, Second Edition, John Wiley & Sons Inc., to be • Explain the importance of model diagnostics.
published in 2019.
• Describe and illustrate extensions to nonstandard multiple
June 7, 2018 regression models.
8h 16min 16 - 1 16 - 2
Computer Program Execution Time

Versus System Load
Chapter 16 • Time to complete a computationally intensive task.
Segment 1
• Information from the Unix uptime command.
Introduction to Failure-Time Regression
• Predictions needed for scheduling subsequent steps in a

multi-step computational process.
16 - 3 16 - 4
Scatter Plot of Computer Program Execution Time Explanatory Variables for Failure Times
Versus System Load
Useful explanatory variables explain/predict why some units
fail quickly and some units survive a long time.
800 • Continuous variables like stress, temperature, voltage, and

700 • pressure.
600
• Discrete variables like number of hardening treatments or
500
Seconds
number of simultaneous users of a system.

400
300 •
• • Categorical variables like manufacturer, design, and loca-
• ••
200 tion.
•
100 ••• ••• •• • •
0 Regression model relates failure time distribution to explana-

tory variables x = (x1, . . . , xk ):
0 1 2 3 4 5 6
System Load Pr(T ≤ t) = F (t) = F (t; x).
16 - 5 16 - 6
Lognormal Distribution Simple Regression Model with
Constant Shape Parameter σ
• Material in this chapter is an extension of statistical re-
gression analysis with normal distributed data and
• The lognormal simple regression model is
mean = β0 + β1x1 + · · · + βk xk
log(t) − μ
Pr(T ≤ t) = F (t; μ, σ) = F (t; β0, β1, σ) = Φnorm
where the xi are explanatory variables. σ
where μ = μ(x) = β0 + β1x and σ does not depend on x.
• The ideas presented here are more general:
• The failure-time log quantile function
Data not necessarily from a normal distribution.
log[tp(x)] = μ(x) + Φ−1
norm (p) σ
Data may be censored.
is linear in x.
Nonstandard regression models that relate life to ex- Notice that
planatory variables. tp(x)
= exp(β1x)
tp(0)
• Presentation motivated by practical problems in reliabil-
implies that changes in x only scale time.
ity analysis.
16 - 7 16 - 8
Computer Program Execution Time Versus System

Load Log-Linear Lognormal Regression Model
log[tp(x)] = μ(x)
+ Φ−1
norm (p)σ
Likelihood for Lognormal Distribution Simple
Regression Model with Right Censored Data
The likelihood for n independent observations has the form

1000 90%
n
L(β0, β1 , σ) = Li (β0, β1, σ; datai )
• 50% i=1
n 6 " #7δi 6 " #71−δi
1 log(ti ) − μi log(ti ) − μi
500 10% = φnorm 1 − Φnorm
i=1
σti σ σ
• where datai = (xi, ti, δi), μi = β0 + β1xi,
•
Seconds
• 4
200 • 1 exact observation
• δi =
0 right censored observation
•
• •
• • • φnorm(z) is the standardized normal pdf and Φnorm(z) is the
100 • •
corresponding normal cdf.
••
50 The parameters are θ = (β0, β1, σ).
0 1 2 3 4 5 6 7
System Load
16 - 9 16 - 10
Standard Errors and Confidence Intervals

for Parameters
Estimated Parameter Variance-Covariance Matrix
• Lognormal ML estimates for the computer time experiment

Local (observed information) estimate = (β , β , σ
⎡ ⎤ were θ 0 1 ) = (4.49, 0.290, 0.312) and an estimate
! β ) ! β , β ) Cov(
! β , σ of the variance-covariance matrix for θ is
⎢
Var( 0 Cov( 0 1 0 ) ⎥
⎢ ! !
Σ̂ = ⎣ Cov(β1, β0) Var(β1) ! ) ⎥ ⎡ ⎤
θ
Cov(β1, σ ⎦ 0.012 −0.0037 0
! ! !
Cov(σ , β0) Cov(σ , β1) Var(σ ) ⎢ ⎥
Σ̂ = ⎣ −0.0037 0.0021 0 ⎦.
θ
⎡ ⎤−1 0 0 0.0029
− ∂ L(β0,β 1 ,σ) − ∂ L(β0 ,β1 ,σ) − ∂ L(β0 ,β1 ,σ)
2 2 2
⎢ ∂β02 ∂β0∂β1 ∂β0 ∂σ ⎥
⎢ ⎥
⎢ ∂ 2L(β0,β1,σ) ∂ 2 L(β ,β ,σ) ∂ 2 L(β ,β ,σ) ⎥
= ⎢ − ∂β ∂β − 0 1 − ∂β 0∂σ1 ⎥ • Wald confidence interval for the computer execution time
⎢ 1 0 ∂β12 1 ⎥
⎣ ⎦
− ∂ L(β 0 ,β1 ,σ) − ∂ L(β0 ,β1 ,σ) − ∂ L(β0 ,β1 ,σ)
2 2 2 regression slope is
∂σ∂β0 ∂σ∂β1 ∂σ 2
.
[β1, β˜1] = β1 ± z(0.975)seβ = 0.290 ± 1.96(0.046) = [.20, 0.38]
Partial derivatives are evaluated at β0, β1, σ 5 1
√
where seβ = 0.0021 = 0.046.
1
16 - 11 16 - 12
Standard Errors and Confidence Intervals for
Quantities at Specific Explanatory Variable Conditions
• Unknown values of μ and σ at each level of x.
= β0 + β1x, σ does not depend on x, and

• μ
Chapter 16
! μ)
Var( ! μ,
Cov( σ)
Σ̂μ,σ = ! μ, ! σ
Cov( σ
) Var( ) Segment 2
! μ)
is obtained from Var( = Var(! β )+2xCov(
! β , β )+x2Var(
! β )
0 1 0 1
! ! ! Nonconstant Variance in Failure-Time Regression
and Cov(μ, σ ) = Cov(β0, σ
) + xCov(β1, σ
).
• Use the above results with the methods from Chapter 8 to

compute Wald confidence intervals for F (t), h(t), and tp.
• Could also use likelihood or simulation-based confidence in-

tervals.
16 - 13 16 - 14
Nickel-Base Super-Alloy Fatigue Data
Nickel-Base Super-Alloy Fatigue Data

26 Observations in Total, 4 Censored
(Nelson 1984, 1990) 250
Originally described and analyzed by Nelson (1984 and 1990).

200
• Thousands of cycles to failure as a function of pseudo- 150

Kilocycles
stress in ksi.
100
• 26 units tested; 4 units did not fail.
50
Objective: Explore models that might be used to describe

the relationship between life length and the amount of pseudo- 0
stress applied to the tested specimens. 80 90 100 110 120 130 140 150
Pseudo−stress (ksi)
16 - 15 16 - 16
Log-Quadratic Weibull Regression Model with

Constant (β = 1/σ) Fit to the Super-Alloy Data
log[tp(x)] = μ(x)
+ Φ−1
sev (p)σ , x = log(pseudo-stress)
= β0 + β1x + β2x2
μ
Weibull Distribution Quadratic Regression Model

with Constant Shape Parameter β = 1/σ
500
This is a lifetime model with the following characteristics: •

•••
• •
Thousands of Cycles
100
• The Weibull quadratic regression model is 50 •

•
Pr[T ≤ t] = Φsev{[log(t) − μ]/σ} •••
• • •• • 90%
where μ = μ(x) = β0 + β1x + β2x2 and σ does not depend 10
••
•• 50%
on x. 5
10%
60 80 100 120 140 160
Pseudo-stress (ksi)
16 - 17 16 - 18
Weibull Distribution Quadratic Regression Model
Likelihood for Weibull Distribution Quadratic
with Nonconstant β = 1/σ
Regression Model with Right Censored Data
The likelihood for n independent observations is

• The Weibull quadratic regression model is

n
L(β0, β1 , β2 , σ) = Li (β0, β1, β2 , σ; datai )
Pr[T ≤ t] = Φsev{[log(t) − μ]/σ},
i=1
n 6 " #7δi 6 " #71−δi
1 log(ti ) − μi log(ti ) − μi [μ] [μ]
where μ = μ(x) = β0 + β1 x + β2 x2 and
[μ]
= φsev 1 − Φsev .
i=1
σti σ σ [σ] [σ]
log(σ) = log[σ(x)] = β0 + β1 x.
where μi = β0 + β1xi + β2x2
i,
4 • The failure-time log quantile function is
1 exact observation
δi =
0 right censored observation log[tp(x)] = μ(x) + Φ−1
sev (p) σ(x)
The parameters are θ = (β0, β1, β2, σ). which is not quadratic in x.
16 - 19 16 - 20
Log-Quadratic Weibull Regression Model with

Nonconstant β = 1/σ Fit to the Super-Alloy Data
log[tp(x)] = μ(x)
+ Φ−1 (x) x = log(pseudo-stress)
sev (p)σ
= β0 + β1x + β2x2, log(σ
) = β0 + β1 x
[σ] [σ]
μ Likelihood for Weibull Distribution Quadratic
Regression Model with Nonconstant β = 1/σ
and Right Censored Data
500
The likelihood for n independent observations has the form
• [μ] [μ] [μ] [σ]
L(β0 , β1 , β2 , β0 , β1 )
[σ]
•••
• • n

Thousands of Cycles
100
[μ] [μ] [μ] [σ] [σ]
= Li(β0 , β1 , β2 , β0 , β1 ; datai)
50 •
• i=1
n 4

:δ 4 :1−δ
1 log(ti) − μi i
log(ti) − μi i
••• = φsev 1 − Φsev .
• ••
• • 90% i=1 σi ti σi σi
10
••
•• 50% [μ] [μ] [μ] [σ] [σ]
5 10%
where μi = β0 + β1 xi + β2 x2 i and σi = exp β0 + β1 xi .
[μ] [μ] [μ] [σ] [σ]
Parameters are θ = (β0 , β1 , β2 , β0 , β1 ).
1
60 80 100 120 140 160
Pseudo-stress (ksi)
16 - 21 16 - 22
Extrapolation and Empirical Models
• Empirical models can be useful, providing a smooth curve

to describe a population or a process.
Chapter 16 • When using an empirical model, it is dangerous to extrap-

olate outside of the range of one’s data.
Segment 3
• There are different kinds of extrapolation
Empirical Models and Extrapolation and
In an explanatory variable like stress or temperature.
Checking Model Assumptions
To the upper tail of a distribution.
To the lower tail of a distribution.
• Need to get the right curve to extrapolate: look toward

physical or other process theory.
16 - 23 16 - 24
Definition of Residuals
Checking Model Assumptions

• For location-scale distributions like the normal, logistic, and
smallest extreme value,
• Graphical checks using generalizations of usual diagnostics
y − yi
(including residual analysis) i = i

σ
Residuals versus fitted values. where yi is an appropriately defined fitted value (e.g., yi =
i).
μ
Probability plot of residuals.
Other residual plots. • With models for positive random variables like Weibull, log-
normal, and loglogistic, standardized residuals are defined
as
• Most analytical tests can be suitably generalized, at least ( )1/σ

log(ti) − log(t̂i) ti
approximately, for censored data (especially using likelihood exp(i) = exp =

σ t̂i
ratio tests).
i) and when ti is a censored observation,
where t̂i = exp(μ
the corresponding residual is also censored.
16 - 25 16 - 26
Plot of Standardized Residuals Versus Fitted Values Probability Plot of the Standardized Residuals from
for the Log-Quadratic Weibull Regression Model Fit the Log-Quadratic Weibull Regression Model
to the Super Alloy Data on Log-Log Axes Fit to the Super Alloy Data
.98
•
5.000 .9 •
• •
2.000 • • ••
• • .7
••
1.000
••
• • • • ••
• • .5
••
0.500 • • •• ••
Standardized Residuals
• .3 ••
••
Probability
0.200
0.100 • .2
•
• •
0.050 • .1 •
0.020 •
.05
0.010
.03
0.005
.02 •
0.002
•
0.001 .01
5 10 20 50 100 200 500 0.001 0.005 0.020 0.050 0.200 0.500 2.000 5.000
Fitted Values Standardized Residuals

16 - 27 16 - 28
Empirical Regression Models

and Sensitivity Analysis
Objectives and Strategy
Empirical Regression Models
• Describe a class of regression models that can be used to and Sensitivity Analysis
describe the relationship between failure time and explana- Objectives and Strategy (continued)
tory variables. Use data and previous experience to choose
a base-line model: • Assess uncertainty
Distribution at individual conditions. Confidence intervals quantify statistical uncertainty.

Relationship between explanatory variables and distribu- Perturb and otherwise change the model and reanalyze
tions at individual conditions. (sensitivity analysis) to assess model uncertainty.
• Fit the chosen empirical regression models and use diag-

nostics (e.g., residual analysis) to check its fit.
16 - 29 16 - 30
Transformations of a
Positive Explanatory Variable
• In choosing an empirical model, it is often necessary to

transform the explanatory variable in order to achieve a
better fit to data.
Chapter 16
• For example, curvature in a plot of the data may suggest
Segment 4 that a model quadratic in x will provide a better fit than one
that is linear in x. In this case, the response ti is modeled
Transformations of a Positive Explanatory Variable using x∗i = x2i.
• A formal way of choosing an appropriate transformation is

to consider one from the Box-Cox family of transformations.
• A sensitivity analysis should be performed to assess the ef-

fect of different transformations on the analysis.
16 - 31 16 - 32
Examples of Monotone Increasing Box–Cox Transformation

Power Transformations of a
Positive Explanatory Variable
• The Box–Cox family of power transformations of a positive
explanatory variable is
λ Transformation ⎧
⎨ xi − 1
⎪
⎪
λ
−2 x∗i = −1/x2 λ = 0
i x∗i = λ
⎪
⎪
−1 x∗i = −1/xi ⎩ log(xi) λ = 0
√
−0.5 x∗i = −1/ xi where xi is the original, untransformed explanatory variable
for observation i and λ is the power transformation param-
x∗i = −1/xi
1/3
−0.333
eter.
x∗i = log(xi)
def
0
x∗i = xi
1/3
0.333 • The Box–Cox transformation has the following properties:
√
0.5 x∗i = xi
The transformed value x∗i is an increasing function of xi.
1 x∗i = xi
2 x∗i = x2 For fixed, xi, x∗i is a continuous function of λ through 0.
i
16 - 33 16 - 34
Laminate Panel Data Scatter Plot

Linear Axes
Estimation of an S-N Curve for a Laminate Panel
• 125 circular-holed notched specimens of a carbon eight-

25000
harness-satin/epoxy laminate panel were subjected to a cyclic
four-point out-of-plane bending.
20000
Thousands of Cycles
• Units tested at 270, 280, 300, 340, and 380 MPa. 15000
10000
• Some “runouts” at 270 and 280 MPa (8 and 2, respec-
tively).
5000
• Data are from Shimokawa and Hamaguchi (1987). 0

260 280 300 320 340 360 380
MPa
16 - 35 16 - 36
Laminate Panel Data
Laminate Panel Data Scatter Plot
Multiple Weibull Probability Plot
Log–Log Axes
Different Shape Parameters
.99 270MPa
.98
280MPa
50000 .9 300MPa
340MPa
380MPa
.7
20000
.5
10000
.3
5000
.2
Thousands of Cycles
Proportion Failing
2000 .1
1000 .05
500 .03
.02
200
.01
100
.005
50 .003
20
.001
260 280 300 320 340 360 380
101 102 103 104 105
MPa
Thousands of Cycles
16 - 37 16 - 38
Laminate Panel Data Laminate Panel Data

Multiple Lognormal Probability Plot Multiple Lognormal Probability Plot
Different Shape Parameters Common Shape Parameter
.99 270MPa .99 270MPa

280MPa 280MPa
.98 .98
300MPa 300MPa
340MPa 340MPa
.95 380MPa .95 380MPa
.9 .9
.8 .8
.7 .7
.6 .6
Proportion Failing
Proportion Failing
.5 .5
.4 .4
.3 .3
.2 .2
.1 .1
.05 .05
.02 .02
.01 .01
.005 .005
.002 .002
.001 .001
1 2 3 4 5
10 10 10 10 10 101 102 103 104 105
16 - 39 16 - 40

Multiple Lognormal Probability Plot Multiple Lognormal Probability Plot
Inverse Power Rule Model Box-Cox Power Law Model λ = −2
.99 270MPa .99 270MPa

280MPa 280MPa
.98 .98
300MPa 300MPa
340MPa 340MPa
.95 380MPa .95 380MPa
.9 .9
.8 .8
.7 .7
.6 .6
Proportion Failing
Proportion Failing
.5 .5
.4 .4
.3 .3
.2 .2
.1 .1
.05 .05
.02 .02
.01 .01
.005 .005
.002 .002
.001 .001
1 2 3 4 5
10 10 10 10 10 101 102 103 104 105
16 - 41 16 - 42
Model Plot Model Plot
Box-Cox Power Law Model λ = −2 Log Transformation
105 105
104 104
Thousands of Cycles
Thousands of Cycles
103 103
90%
102 102 90%
50%
50%
10%
10%
101 101
260 270 280 290 300 310 330 350 370 390 260 270 280 290 300 310 330 350 370 390
MPa on Box−Cox(−2) Scale MPa on Log Scale
16 - 43 16 - 44
Laminate Panel Data

Lognormal Model-Fitting Summary
Laminate Panel Box-Cox Sensitivity Analysis
Box-Cox Regression Model with Power −2
Laminate Panel Fatigue Data
Model −2LogLike AIC # Param with Lognormal Stress:log at 300
Power Transformation Sensitivity Analysis on Stress
1600
SepDist 1765 1785 10 ML estimate of the 0.1 quantile
0.1 Quantile of Kilocycles Distribution

Approximate 95% Pointwise confidence intervals
ConstSig 1777 1789 6
1400
RegrModel 1779 1785 3
Pooled 2130 2134 2
1200
Laminate Panel Data Lognormal Likelihood Ratio 1000

Tests
Box-Cox Regression Model with Power −2 800
Comparison LR Statistic dof p-value 600

-4 -3 -2 -1 0 1
SepDist vs ConstSig 12.31 4 0.015 Stress Box-Cox Transformation Power
ConstSig vs RegrModel 1.54 3 0.67
RegrModel vs Pooled 351.31 1 < 0.001
16 - 45 16 - 46
Laminate Panel Box-Cox Sensitivity Analysis Profile

Laminate Panel Fatigue Data
Approximate Profile Likelihood and 95% Confidence Interval
for Stress Box-Cox Transformation Power from the Lognormal Distribution
1.0
Approximate Profile Likelihood
Chapter 16
0.8 0.50
Confidence Level
0.60
0.6
Segment 5
0.70
0.4
0.80
0.90
with Two Explanatory Variables
0.2
0.95
0.99
0.0
-4 -3 -2 -1 0
Stress Box-Cox Transformation Power
16 - 47 16 - 48
Scatter Plot the Effect of Voltage and Temperature
on Glass Capacitor Life
Two or More Explanatory Variables: Glass Capacitor

Failure Data
1200
• Experiment designed to determine the effect of voltage and 170 Degrees C
180 Degrees C
temperature on capacitor life.
1000
800
• 2 × 4 factorial, 8 units at each combination.
Hours
600
• Test at each combination run until 4 of 8 units failed (Type

400
II censoring).
200
• Original data from Zelen (1959).

0
200 250 300 350

Volts
16 - 49 16 - 50
Lognormal Probability Plot Weibull Probability Plot

Glass Capacitor Life Test Results Glass Capacitor Life Test Results
Different Shape Parameters Different Shape Parameters
.6 170DegreesC;200Volts .7 170DegreesC;200Volts
170DegreesC;250Volts 170DegreesC;250Volts
.5 180DegreesC;200Volts 180DegreesC;200Volts
180DegreesC;250Volts .5 180DegreesC;250Volts
.4
.3
Proportion Failing
Proportion Failing
.3
.2
.2
.1
.1
.05 .05
100 200 500 1000 2000 100 200 500 1000 2000
Hours Hours
16 - 51 16 - 52

Glass Capacitor Life Test Results
Common Shape Parameter
Glass Capacitor Life Test
Two-Variable Regression Models
.7 170DegreesC;200Volts
170DegreesC;250Volts
• Additive model
log[tp(x)] = yp(x) = β0 + β1x1 + β2x2 + Φ−1(p)σ.

.5 180DegreesC;250Volts
.3
• The interaction model
Proportion Failing
log[tp(x)] = yp(x) = β0 + β1x1 + β2x2 + β3x1x2 + Φ−1(p)σ.

.2
• Comparing the two models gives

.1
−2 × (L1 − L2) = −2 × (−244.24 + 244.17) = 0.14
which is small relative to χ2
0.95,1 = 3.84.
.05
100 200 500 1000 2000
Hours
16 - 53 16 - 54
Weibull Probability Plot Weibull Probability Plot
Glass Capacitor Life Test Results Glass Capacitor Life Test Results
Arrhenius/Power Law Model Common Shape Parameter
.3 .3
Proportion Failing
Proportion Failing
.2 .2
.1 .1
.05 .05
100 200 500 1000 2000 100 200 500 1000 2000
Hours Hours
16 - 55 16 - 56
Estimates of Weibull t0.5 Plotted for each Combination

of the Glass Capacitor Test Conditions
Glass Capacitor Life Test Results Points are Regression-Model-Free Estimates
Weibull Distribution-Fitting Summary
Arrhenius and Power Rule
Model −2LogLike AIC # Param
SepDist 463.3 495.3 16 1400
ConstSig 476.3 494.3 9 1200
RegrModel 488.5 496.5 4 1000
Pooled 509.1 513.1 2 170 Degrees C
800
600
Hours
Glass Capacitor Life Test Results

Weibull Distribution-Fitting Summary
Arrhenius and Power Rule 400
180 Degrees C
Comparison LR Statistic dof p-value

SepDist vs ConstSig 12.96 7 0.073
ConstSig vs RegrModel 12.19 5 0.032 200
RegrModel vs Pooled 20.57 2 < 0.001
200 220 240 260 280 300 320 340 360
Volts
16 - 57 16 - 58
The End

Course Notes - Estadistica Industrial II

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Course Notes - Estadistica Industrial II

Uploaded by

Copyright:

Available Formats

Industrial Statistics II

An Introduction to Statistical Methods for

• Presentation focus: concepts, ideas, examples, models, data

W. Q. Meeker, L. A. Escobar, and F. G. Pascual • Structure of the Course Notes.

June 11, 2018

Other Arrangements for this Course

• Part 3: Parametric likelihood ﬁtting concepts and applica-

Reliability Concepts and an Introduction to Reliability Chapter 1

Topics discussed in this chapter are:

• Examples of studies that resulted in diﬀerent kinds of failure-

Copyright 2018 W. Q. Meeker, L. A. Escobar, and F. G. Pas-

Based on the authors’ book Statistical Methods for Relia-

Quality and Reliability

Pressures on manufacturers to produce high quality, low cost

• Intense global competition.

• Increasing customer expectations.

Reasons for Collecting Reliability Data

• Assess characteristics of materials.

• Predict product reliability in design stage.

Quality over time (Condra 1993). • Assess product reliability in ﬁeld.

• Predict product warranty costs.

• Failure-time data (Chapters 2–18), also known as

Segment 2 Event-time data.

Introduction to Failure-Time Data • Degradation data.

Destructive degradation data (Chapter 19).

• Recurrent event data (Chapter 21).

Ball Bearing Data

The data: Millions of revolutions to failure for each of Row

• Disagreement in manufacturing industries on the appro-

Ball Bearing Failure Data

• Data are typically censored (bounds on observations).

• Positive responses (e.g., time or cycles to failure) need to

nential, lognormal, and Weibull distributions. The normal

distribution is not common.

• Model parameters not of primary interest (instead, failure

• Extrapolation often required (e.g., want proportion failing

• Causes of failure and degradation leading to failure.

Event Plot of the Integrated Circuit Life Test Data

0 200 400 600 800 1000 1200 1400

Failure Pattern in the Shock Absorber Data

First reported in O’Connor (1985).

Shock Absorber Data (Two Failure Modes)

• One might be interested in the distribution of time to fail- 19

• Model ﬁtting (parametric, including possible use of prior • R/RSplida

Heat Exchanger Tube Crack Data

Data from heat exchangers from power plants.

• 100 tubes in each exchanger.

• Each tube inspected at the end of each year for cracks.

Exchanger Year 1 Year 2 Year 3

Each of n = 432 wheels was inspected once to determine if

An increasing hazard function would require replacement

Scatter Plot of Low-Cycle Fatigue Life Versus

Failure-Time Data with Explanatory Variables 150

Degradation Reliability Data

80 90 100 110 120 130 140 150

• Provides information on progression toward failure.

1.0 Censoring time ->

Millions of Cycles Hours

• Failure-time or degradation data on components or non-

Topics discussed in this chapter are:

• Describe some reliability metrics.

Typical Failure-time cdf, pdf, hf, and sf

Cumulative Distribution Function Probability Density Function

• The increase in F at each value of ti is

F (ti) − F (ti−1) = di/n.

0.010 • F (t) is the ML estimator of F (t).

• The estimate F (t) is a step-up function with a step of size

because selogit(F) = seF /[F (1 − F )].