You are on page 1of 10

\

TECHNOMETRICS

VOL.

7, No.4

NOVEMBER,

1965

MaximumLil~elihoo~ Estimation in the Weibull Distribution Based' On Complete and On Censored Samples
A.
CLIFFORD COHEN* The University of Georgia This paper is concerned with the two-parameter Weibull distribution which is' widely employed as a model in life testing. Maximum likelihood equations are derived for estimating the distribution parameters from (i) complete samples, (ii) singly censored samples and (iii) progressively (multiple) censored samples. Asymptotic variance-covariance matrices are given for each of these sample types. An illustrative example is included.

1.

INTRODUCTION

Because of its versatility in fitting time-to-failure distributions of a rather extensive variety of complex mechanisms, the Weibull distribution has in recent years assumed a position of importance in the field of reliability and life testing. Various problems associated with this distribution have been considered by numerous writers, among whom are Dubey [3], Esary and Proschan [4J, Jaech [5], Kao [6, 7, 8], Lehman [10], Leone, Rutenberg, and Topp [11], Lloyd and Lipow [12], Menon [13], Procassini and Romano [14], and Proschan [15]. A major deterrent to wider usage of the Weibull distribution has been the difficulty in estimating its parameters. Unfortunately, the calculations involved are not always simple. This paper is concerned with maximum likelihood estimation in both complete and censored samples from the two-parameter Weibull distribution with density function f(x) = ('Y e)x ,,(-1 exp ( - x"(/ e) ; / x >0, 'Y > 0,
0

> 0.

(1)

When'Y = 1, this becomes the density function of the well-known one-parameter exponential distribution. The particular form in which (1) is written was chosen for the purpose of simplifying derivations of the maximum likelihood estiReceived September 1964; revised April 1965 performed while author was on temporary duty with Headquarters Strategic Air Command, Offutt Air ForceBase during August, 1964. The views expressed herein are those of the author and do not necessarily reflect the views of the U. S. Air Force or the Department of Defense ..

* Research

The readers may be interested in noting the article entitled "Maximum-Likelihood Estimation of the Parameters of Gamma and Weibull Populations from Complete and from, Censored Samples" by H. Leon Harter and Albert H, Moore which, appears in this same issue. Each of these papers was carried out independently of the other. (Editor's comment.)

579

,

J'

j

""

samples.4. 2. (1) is equivalent to the two-parameter Weibull density function considered by Proschan [15]. () is estimated from the second equation of (3) as ()= r t xi n 1 In Xi - 1] - L xJ =- 1 "1 n L In n' 1 Xi . we have which may now be solved for the M. This can be accomplished with the aid of standard iterative procedures. the time is noted. estimators from the parameters being estimated. Finally at some pre-determined fixed time Xv I I - . n (5) The symbol C) is employed here to distinguish M. NO. VOL. Xn . . 1'. "12 . COMPLETE SAMPLES ~ •1 I 1..I I I I I I I I I I I I I I I I I I I I I I I I I I I I' 580 TECHNOMETRICS. . estimate 1. J On eliminating (Jbetween these two equations and simplifying. are presented in a form analogous to that employed for complete samples. I. The likelihood function of this sample is L(x1 . differentiating with respect to and equating to zero. • j I ~ I Consider a random sample consisting of n observations when (1) is the applicable density function. (4) '. With 1thus determined. though differing in form. Apparently censored sampleafrom the Weibull distribution have received little if any previous attention in the literature. . but in most instances a simple trial and error approach will suffice. "1 On taking logarithms of (2). 7. N specimens are placed under observation and as each failure occurs. . As written here. L. NOVEMBER 1965 mating equations. Asymptotic variances and covariances are given for the estimators obtained here from both complete and censored. Estimating equations for censored samples which arise naturally from life tests which are discontinued before all sample specimens fail. 3. r I 8) = IT ('Y/O)xJ-1 i=l n exp (-x~/O).J 1 Xi In Xi = 0. Estimating equations obtained here for complete samples are presented in a form which seems to be somewhat simpler to solve than equivalent equations given previously by other writers. SINGLY CENSORED SAMPLES In a typical life test. L.! r L x~/n.. Once two values "11 and 1'2 have been found within a sufficiently narrow interval such that "11 < l' «. we obtain the estimating equations and e in turn aInL a "1 ="1 n + L: In Xi n 1 - -(J 1~'Y L. linear interpolation will yield the required value. Menon [13] and others.. (j 02 -7-' Xi 'Y 0. (3) = a In L _ _ ~ + _!_ ~ I t se -.

In both of these cases the data collected consist of observations Xl . Xo or :lin • In particular L.J .. Xo in the former case.J 1 ~ In 1 Xi - 1 -e 'Y . and z. X2 . In both type I and type II censoring. when Xo is fixed and n is thus a random variable.assigned the value XT . a a 'Y ao In L =- + 'Y n n L. the time of termination XT = Xo . and on eliminating e between the two equations of (9). plus the information that (N ./o) 1 dtle [1 .. F(x) L" 'Yt'Y.e... and in type II censoring XT = Xn .n)x. In the form given above.ESTIMATION IN THE WEI BUll DISTRIBUTION 581 or after some pre-determined fixed number of sample specimens fail.* L.-' exp (-x. When n is fixed and the time of termination Xn is a random variable. z. •••.! t = 1 In Xi . L: . i.* Xi = where L:* signifies that the summation extends over the entire sample with the (N .exp (-Pie) 1 .)t-·. L ~ (N ~! n)! [IT ('Y/O)x. (8) On differentiating (8) with respect to 'Y and we obtain the estimating equations e in Xi 'Y turn and equating to zero. the test is terminated.. estimating equations (9) are fully analogous with equations (3) for complete samples. (6) where in type I censoring..* (10) xi = L: xi + (N . (7) Accordingly In L follows from (6) and (7) as In L = n In 'Y - n In e + ('Y - 1) L: In I n Xi - (II - e) L: xi 1 n [(N - n)leJx~ + const.* Xi 'Y In Xi = L xi n 1· n 1 In Xi + (N .F(x.n) survivors .n)x.n) specimens' survived beyond the time of termination. . (11) . the likelihood function may be written as . Consistent with st~ndard terminology as employed in [1].1 .. we have [ L:* L Xl xi In Xi - 'Y .-0 + rP L.exp ( - X 'Y I e) . XT . censoring is said to be of type II. in the latter..vcensoring is said to be of type I. a ln L .J In Xi = 0. (9) 0.!J n. In . The distribution function F(x) follows from (1) as .J "..

** ' 'Y f)2 L.-1 F(T. If we let N· designate the total sample size as in section 3. i (14) t where C is a constant. sample specimens selected randomly from the survivors at time T.!.J 1 ". it then follows from the second equation of (9) that o A = "L. ~ . NOVEMBER 1965 to be solved for '9 employing the same techniques suggested for use in solving equation (4) in the case of a complete sample. and n the number of specimens which fail and therefore provide completely determined life spans.** Xi 'Y In Xi = 0. observations censored at time 'I'. the resulting estimating equations follow as ' a "0 L = . a In L af) __ !!: + - (16) 0 _.J 1 ~ In Xi " - -0 L. 1 Ie (13) I In type I progressive censoring where the 'I'. .)r . (12) 4. > T'_l . 1 n n . PROGRESSIVELY CENSORED SAMPLES I • f • ~ In many life testing situations. . and that at the zth stage of censoring r. . and F(x) is the distribution function.".I II I I n + 2:r •. More specifically I . 2 . assigned the value Xi = To . With f(x) given by (1) and F(x) by (7). VOL 7. censoring may be progressive through several stages.J ".Progressive censoring in connection with the normal and the exponential distributions was considered in an earlier paper [2].. are fixed. the logarithm of the likelihood function becomes In L = n In 'Y . With '9 thus determined. the initial censoring results in withdrawal of only a portion of the survivors.y xjn. it follows that N = I i I .(1/0) (1/0) L riT. " k i-I . where L** signifies summation over the entire sample ~ith the r. the likelihood function may be written as L =C I III(Xi) II[1 -.n In 0 + (-y - 1) L In Xi 2: X. For sufficientlylarge samples. Suppose that censoring occurs progressively in k stages at times T i where T.582 TECHNOMETRiCS._ ".J Xi =0 .. f(x) is the density function. with some remaining on test and therefore continuing under observation until ultimate failure or until a subsequent stage of censoring is performed.4.*. + In C.+ In n 'Y 'Y L. are removed (censored) from further observation. NO. i = 1. k. 1 k (15) On differentiating (15) with respect to 'Y and 0 in turn and equating to zero.

J A . () 1 2·" .8 = - a 2 de. I' .6 ri In L -'a'Yae I -1 1. intermediate steps in the derivation of estimating equations . VARIANCES ~D COVARIANCES OF ESTIMATES The asymptotic variance-covariance matrix of (1. (18) to be ~olved for 1in the same manner as that suggested' for solving (4) in the case of a complete sample and for solving (11) in the case of a singly censored sample. (20) V(O) The elements of the information matrix on the left side of (20) follow by differentiating (3) for complete samples.8 - -"'2 n e + ~ LxI. it seems appropriate to approximate the expected values by their maximum likelihood estimates. '2 = 4' + -()A 2: xl (In Xi) .ae a'Y a 2 In L I t. xi Xi L** - I' .** x ~/n.a'Yao a In L 2 2 I' 1.8 = - ff ~ 1" t Xi In Xi . and (16) for progressively censored samples. ())]. e) is obtained by inverting the information matrix with elements that are negatives of 'expected values of the second order derivatives of logarithms of the likelihood functions. and the estimating equations given here are applicable for both sample types.!] .6 a 2 a()2 In L I1.!~ f-' = n In z. it follows from the second equation of (16) that ()= L. . a'Y In L I 1. are the times at which withdrawals are actually made . ·..ESTIMATION IN THE WEIBULL DISTRIBUTION 583 (17) In the above form.1" . We thereby 'obtain a 2 a I' In L 2 _ I .6 r Vel') COY (1._ 1. for type II progressively censored samples differ from corresponding steps in the case of type I samples. 5. ff) 1. we have as the approximate variance-covariance matrix ri In L (11'2 I . (9) for singly censored samples. With 1 thus determined. 1 1. The end result. It is necessary only to keep in mind that the times 'I'. Accordingly on eliminating e between the two equations of (16). In the present situation.8 A For Complete Samples n . (19) As noted in [2]. In fact equations (3) and (9) may be considered as special cases of equations (16).9 _:_loov (1. (21) _a a()2 In L I. Accordingly. is the same in both cases.. however. estimating equations (16) are fully analogous with equations (3) for complete samples and with equations (9) for singly censored samples. we have In [ 2:** xl.

2 Xi) . '9. (22) a~ = 'Y.8 = 'Y -:::2 1" + -OA L. it must be recognized that errors due to bias sometimes greatly exceed the errors induced by large estimate variances.B A = .n) survivors assigned the value XT .8 = aaoIn L I a 'Y 'Y. assigned the values Xi = T •. In small samples.B A A = r/ A 1 2: **. Therefore with the aid of a suitable graph or table of this function. they may be relied upon to provide reasonable approximations to estimate variances and covariances for moderate size samples.xl A In Xi . t = {fI'Yr[(k/y) + 1].a aIn2 L 'Y I' '9.J * 2 X '9( i In Xi ')2 . 7. Although the foregoing results are valid in a strict sense only for large samples. 'Y 2 2 Lin L..I For Singly Censored Samples A a In L .J . rem) = lex> o xm-1e-z dx. (23) 02 + rl 2: 2 ** X~.B A A =- . The kth non-central moment readily follows from (1) as J.J X '9 . of order statistics will ultimately provide unbiassed estimates that may be preferred over the estimates considered here. L** signifies summation over the entire sample with the r..- (25) Accordingly. NO.a'Yao _ a In 2 2 I '9.* Xi '9 In Xi . NOVEMBER 1965 I I I I I I I I . Perhaps some linear combination.L~ ft.a'Yao 2 a 2 ao In 2 Lin I'9. when the variance is divided by the square of the mean. and as in the case of estimating equations (16).. I I I I I I where as in the case of estimating equations (9).584 TECHNOMETRICS. A FIRST ApPROXIMATION TO THE SHAPE PARAMETER The coefficient of variation of the Weibull distribution is a function of the shape parameter 'Y alone. we obtain the following expression which is a function of'Y only .a aIn2.2 a In L . 2:* signifies summation over the entire sample with the (N . observations censored at times T. 02 1 L. This is an area . OA A For Progressively Censored Samples . VOL.8 A A n 'Y 1 . At least the maximum likelihood estimates are consistent-and we are thereby assured that the bias diminishes as the sample size becomes large.4. '(m> 0).(In .which requires further investigation with respect to the Weibull distribution. 6.8 'Y. = =- aaoIn L a 'Y 2 I 'Y.** + -OA L. (24) where I' signifies the gamma function. it is easy to obtain a good first approximation to the maximum likelihood estimate "9 by equating the first two moments of a'" complete sample to corresponding population (theoretical) moments.

0 4.!. our distribution is in fact the exponential distribution. as . then CV = 1 and from (29) .2 = jJ.f2.ESTIMATION IN THE WEIBULL DISTRIBUTION .:.0 . ()* = {x/r[(l/y*) + I]}'Y.1323 0. (27) Following is an ~ridged table of the coefficient of variation and of its square with 'Yas the argument. Kendall [9.2361 1. (28) The stars (*) serve to distinguish moment estimates from maximum likelihood estimates denoted C)· and from the parameters being estimated.{2.= jJ. suppose n = 50. jJ.6165 0. the moment estimate thus read will be of such accuracy that no further improvement through iteration is necessary. On taking square roots of (26). The value thusread should provide a satisfactory first approximation for use in iterating to the maximum likelihood estimate.3801 . and suppose further that 'Y = 1.e..19 5 1.. For many purposes.l Vex) r[(2/y) + 1] 2 r2[(1/y) r [(1/y) + 1] + 1] (26) ..0787 4..3589 2. the moment estimate 'Y*might be read from Figure 1 with an accuracy of at least one decimal and perhaps two.2732 0.~ . i. variance of the coefficient of variation (valid for large samples) is V(CV) . (CV)2/2n.8050 0. page 233] has shown that the approximate asymptotic sampling...0 3. 1 The Wetoull Coefficient of Variation as a Function of the Shape Parameter TABLE 'Y V(x)/JLP Coefficient of Variation 1/3 1/2 0.6480 0.5904 1 0. At some future date it might be advisable to prepare a mote extensive table at closer intervals of the argument as a further aid in obtaining moment estimates of'Y in practical applications. In such cases the moment estimate of ()follows from (24) with k = 1 and with jJ. With the sample quantity 82/X2 equated to V(x)/ jJ.585 -'2.8 1.3637 0.0 5/4 5/3 2. 0. For example. When'Y = 1.5227 0.2805 The graphs of Figure 1 were plotted using values from this table.2611 1 0. (29) This result can be used in conjuction with the CV graph of Figure 1 to provide an indication of the extent of ·the sampling variation in estimates of 'Y.i equated to x. where Vex) = jJ. we have for the coefficient of variation CV = vr[(2/y) r2[(1/y) r[(l/y) + 1] + 11- + 1].

this corresponds to an interval of from . 7.. As can be shown from the graph.8 to 1. results obtained here.c 0.25 for the estimate 7*.00 .4 0.83 to 1. 75 2. and there is approximately a 95% probability that the sample value of CV will be in the interval .e . i.0 . Data for this sample are given below.Accordingly O'~v = 0.0 I! ..l~~~~~~~~~~~~~~~~~~~~~~~~~ 0. .01.00 V(CV) ~ 0.25.50 1.is 50 and actually 7 = L . . This result tends to justify the natural reluctance which one might have toward abandoning the much simpler exponential distribution as a model for a life distribution in favor of the Weibull distribution on the sole evidence of a single small sample. NOVEMBER 1965 C<I :. VOL 7. AN ILLUSTRATIVE EXAMPLE " I I .0 30. A sample given by Menon [13}has been selected to illustrate the practical application of..5 0.75 1. NO..0 10.1.0 TECHNOMETRICS. when the sample size .5 0. 4.2 O.0 40. .3 0..4..2.25 1.0 fj 2.:C." 1.586 • 50.. .) 5.25 0.83 < 1'* < 1..

640 1.-0.550 9.098 0. l' K('Y) .506 7.628 1. L~o X~·6 In X.ESTIMATION IN THE WEIBULL DISTRIBUTION 587 x lnx x In x x lnx x In x 0. The moment estimate of (J was calculated using equation .(28).5000 1.likelihood estimate of the shape parameter.686 -0. L~o x~ = 3479..064 -6.0432.6) = 0. Reading from the charts of .5) .086.410 -1.43. 1'* = 0. K('Y) = --=l'--::-n-- ~ c: X.832 0. (CV)2 = 7.550 1.046 0.5 using estimating equation (4).649 .0432 0 0.684 -3. On writing equation (4) in the form ' n . To obtain 0.6501._ 1 l' - nL.001 0.532 0.824 -0.438 According to Menon.363. = 61.L.580.030 -2.664 0. Estimates 1. Since both of these values are negative. the maximum likelihood estimate we decide to try interpolating between :"r ·= .030 0.469 -0.506 . To obtain .018 and it follows that K(. L~o z.0. and (J = Ve _:_ 1.506.895.262 -0.185 0.345 0. It follows that x = 4:42225.032 1. and CV = 2.970 0. this sample is from a population in which 'Y = . and Liox~·5In Xi = 41.505 0.716 -1.363 ..033 3.500 .057 2.435 1.071 4. and L~o In Xi = ·-8.302.600 Thus we have '9 .216 -0.4700. We subsequently calculate 4 L~o x~·. we calculate K(A) = -0.170201. we interpolate linearly as shown below..: In X.649.5. we now calculate L~o X~·6 = 32. we have as the moment estimate and first approximation to the maximum . = 880445. • = 0 .758 57.806 0.t _~ 1 n In z.L~o x~·5=27.' Lxi 1 -.6501 X~·506 - .810.438 2. we calculate L~o from (5) 27.954 0.40 . along with the population values.23 ..007.261 and o = 27. -0. and l' = . . Figure 1.208 -0.661..754 -0.637. and estimates based on Menon's results.1540402.054 0. Lio x~·4In xi=27.506 . To obtain the required estimate. For comparison. These data are summarized as: n = 20.4=23.43 1.261/20 = 1. the maximum likelihood estimates are listed below with corresponding moment estimates.9U8and K(.i.57 1. Population Values 8 'Y Moment Estimates Menon's Estimates M. .

. j . K. I. Reliability and Quality Control. JR. J. KENDALL. . 389-407.A. Technometrics. M. 13. No. .0.F. ' 7.48. V(1) -=. 32-38.Y...94J-1 10. Estimation of the shape and scale parameters of the Weibull distribution. 9. Technometrics.. Assoc. LEHMAN. Inc. Computer methods.007 0. Statistical Laboratory.. R. Calculations for censored samples are essentially the same as for the complete sample illustrated here and are not likely to present any unusual computational difficulties.09 -22. and LIPOW. Cleveland. 58. Trans. Simplified estimators for the normal distribution when samples are singly censored or truncated. Relationship between system failure rate and component failure rates. CLIFFORD.. Order statistics and estimators for the Weibull distribution. 217-37. 6. 1963. Englewood Cliffs. Shapes. and STUARTA.. John H.254404. A graphical estimation of mixed Weibull parameters in life-testing of electron tubes. S. 1.123 . 12. J.123. Y. Computer methods for estimating Weibull parameters in reliability studies. D.. 11.. C. H. VOL. Hafner Publishing Co. " .A. 1964. New Jersey.M.8 = l 200. V(O) -=. 1960. NOVEMBER 1965 . Technometrics. H. Kao. 1958. KAO. 1~63. DAVID K. K. R-12.FRANK. MENON.. COHEN. 15.for estimating Weibull parameters in reliability studies. 1. and PROSCHAN. JJJ. 2. Technometrics.77 - [0. Statist. J.. Vol. 1958. V.007.. 1959. PROSCHAN. Transistor reliability estimates improved with A Weibull distribution function. IRE Transactions. JAECH. PROCASSINI. 4.588 TECHNOMETRICS. Technometrics. A.1959. We then evaluate the partials of (21). Amer. 166. 1. RUTENBERG. 3. COHEN. Case Institute of Technology. K. PGRQC-13 (July) 15-22. G. Estimation of Weibull distribution shape parameter when no more than two failures occur per lot. 1958.4.94 0.EUGENE. 5. and Cov (1. (abstract). 5.A.0. and ROUANO. 0) VV(1)' V(O) = 0. The correlation coefficient between estimates follows as .5. ESARY.! . 1026 (AFOSR Jteport No. P1. IEEE Transactions on Reliability. 15-22.. 9. 1963. REFERENCES 1. JAMES D. On some statistical inferences for Weibull laws. Prentice-Hall.FRANK. Motorola Engineering Bulletin. 175-82.2.j i 1 . 1963. 3.H. 0) . Methods. L.CLIFFORD.327-39. 8. 7. and Mathematics. LEONE.014. moments and estimators of the Weibull distribution. New York. we first obtain the additional summations LiD X~·5D6ln =42. NO. J. 16-18.. MYRON. and Tozr. SATYAD. 'Technometrics.014J -22. 1963. 415-422. MO. Cov (1. H. W. N. 1961. The Advanced Theory of Statistics. 375-83.. 13. 10. and the desired matrix follows as Accordingly. 183-89..014 0. Reliability: Management. Technometrics.5. 1962. 14. LLOYD. Case Statistical Laboratory Publication No. 1963. TN 60-389).0.. Progressively censored samples in life testing. 549. Theoretical explanation of observed decreasing failure rate. In order to evaluate the variance-covariance matrix (20). 6.611596 and Lio X~·506 (In xil Xi . DUBEY.