Performance Evaluation for Maximum Likelihood and Moment Parameter

Estimation Methods on Classical Two Weibull Distribution

F.Q. Yuan1, A. Barabadi1, J.M. Lu1, A.H.S. Garmabaki2

1
Department of Engineering and Safety, University of Tromsø - The Arctic University of Norway, Tromsø, Norway
2
Luleå University of Technology, Luleå, Sweden &
Department of Mathematics and Computer Science, Islamic Azad University, Nour Branch, Nour, Iran
(yuan.fuqing@uit.no)

Abstract - The two-parameter Weibull distribution is the II. MAXIMUM LIKELIHOOD METHOD
most widely used distribution in reliability engineering.
Parameter estimation is a key issue to apply the Weibull to In this paper, the probability density function (PDF) of
practical engineering. This paper aims to compare the two parameters Weibull density function is defined as
performance of the maximum parameter estimation method /
(MLE) and the moment parameter method. It firstly ; 0, 0, 0 (1)
investigates some mathematical properties such as solution The is the scale parameter and the is the shape
uniqueness, estimator’s equivariance and the confidence parameter.The MLE assumes the most likely parameter as
interval, that are important to practice. Later on their
the desired estimator. This principle is intuitive and can
performances have been evaluated by using simulation. In
the simulation, data sets ranged from extreme small to large
apply to the most real situation (but not always).
have been considered. Weibull distribution with increasing, Suppose the observed failure time are , , … . , , . . . .
constant and deceasing failure rate have been chosen in the The likelihood function is defined as
simulation to ensure the simulation’s results concrete. ∏ (2)
Which can be rewritten as
Keywords – Maximum Likelihood method, Moment /
method Parameter Estimation, Simulation, Weibull ∏ (3)
distribution. The is a very small number in most cases that causes
inconvenience of computation, as shown in Table 1,
I. INTRODUCTION where the likelihood function value is very small.
Table 1. L vs ln L
Weibull distribution is most important distribution in
reliability engineering[1]. The major characteristics of
this distribution is that it can represent an increasing, 10 0.10 4.81E-130 297.7654 4.81E-130
constant and decreasing failure rate, as shown in Fig 1. 20 0.64 8.18E-102 232.7616 8.18E-102
This advantage endows this distribution a great 30 1.19 9,15E-103 234.9524 9,15E-103
importance in reliability[2-4].
40 1.73 1.07E-110 253.214 1.07E-110
4
… … … … …
Time vs Beta 2
Time vs Beta 3
3 Time vs Beta5
Time vs 0.8
One trick takes the logarithm of to transform the into
Failure Rate

2
an easily tractable number. Finding the maximum of is
equivalent to find the minimum ln , as ln is a
1
monotonic function of .

0 2 4 6 8 10

Time
Fig 1. Failure Rate of Weibull

Parameter estimation is critical for the application of

Weibull to practical engineering. Maximum likelihood
method (MLE) is the method used in most reliability
softwares, for example Reliasoft, Weibull ++ and Matlab.
Moment method is also an important parameter
estimation method in statistics. This paper tries to
Fig 2. Minimum value of ln
compare their performance by investigating some
practical mathematical properties and by simulation.
The negative logarithm of (3) is

978-1-4673-8066-9/15/$31.00 ©2015 IEEE 802

Authorized licensed use limited to: UNIVERSIDAD POLITECNICA SALESIANA. Downloaded on September 30,2020 at 20:34:55 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the 2015 IEEE IEEM

ln ln ln / 1 ∑ ln ∑ Γ 1 (10)
(4)
For continuous function as (4), the simple method to find Since there are two parameters, the first two orders can be
the minimum is to take first derivative with respect to used to estimate parameters. In this paper, the method
each parameter, i.e. used to estimate the parameters are as
̅
(11)
∑ ln
Similar to the maximum likelihood estimate, numerical
∑ ln ∑ ln (5) method must be used to find the solution. Our
experiments shows the unique solution always exists so
Equal them to zero and simplify the expression. The (5) is the uniqueness of the solution can be guaranteed.
then rewritten as
The confidence interval of estimators can be derived from
/ moment also. According to large number theory, for a
∑ / ; series of i.i.d. data , if | | ∞, estimators of
∑ parameters
∑ ln 0; (6)
∑ , ,… , , , …, . (12)
Let and , , then
The depends on the . The can be obtained from
√ → 0, ′ (13)
Equation (6). The equation has not the closed-form
We omitted the proof of (13) in this paper. Variance of
expression. One has to resort numerical method, e.g. the
numerical Newton-Raphson method to find the can be approximated from (13). The remaining problem
now is that the explicit expression of
approximate . This is a weakness of MLE for Weibull
Distribution.Generally, MLE obtained from first , , …, is not available.The for (13)
derivatives like (6) does not guaranty the values obtained can be obtained Jacobian matrix. Let ,
is theminimal. Section VI will address this problem
specially. According to the Jacobian,
Confidence Interval Derivation / … / … /
.. … …
/ … / … /
The second derivative of (5) is , , …,
(14)
/ … / … /
ln .. … …
1 / … / /
…
ln
ln The first derivative of each implicit function can be
obtained from (10).The is the covariance of , . For
ln example, the . Using the
∑ ln / / (7) parameter value estimated from (11), a numerical matrix
of ′ can be obtained. In the matrix , the
diagonal corresponds the variance for each parameter as
The fisher information matrix [5] is
, (15)

, (8) As √ → 0, , at significance level 1 ,

the confidence interval for parameter is
The since the function ln is
, (16)
continuous. The estimators converge to Normal
distribution as
→ However, for 0, the (16) could cover negative
0, (9)
value. One tricky is to transform the into ln . The
where is the inverse matrix of . The confidence corresponding confidence interval can be obtained from
interval is obtained from (9).
, / , (17)
III. MOMENT METHOD
where exp . The derivation of (17)
Moment method is another important parameter
estimation method after maximum likelihood estimate. refers to [6]. The confidence interval from maximum
The moment of Weibull distribution is as likelihood estimation is also from (17), just by replacing

803
Authorized licensed use limited to: UNIVERSIDAD POLITECNICA SALESIANA. Downloaded on September 30,2020 at 20:34:55 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the 2015 IEEE IEEM

the to , in (9). 25

IV. UNIQUENESS OF ESTIMATOR 20

As mentioned in section II, the solution of MLE from its 15

1/Beta
first derivative may not exist, or it may not be unique. As 10

there is no close form to the solution of (6), it has to use

the numerical method to search the solution. However, 5

solution from numerical method depends largely on the 0

initial search point and the search algorithm. When there 0 2 4

Beta
6 8 10

are several solutions to the problem, the estimators will Fig 3. Cross of RHS and LHS of (18)
vary. Furthermore, the solution may not exist. The
estimator from numerical method may not have any The plot of RHS and LHS of (18) must cross to ensure the
sense. One has to ensure the solution exist and it is unique solution to (18) exists. Let us write it in terms of
before applying numerical method. mathematics, sufficient additional conditions for the
existence of solution to (18) are
In state of art, some authors claim that the MLE is not
unique. Some have proved it is unique, however their i. lim → lim →
proof is not complete and not mathematically
strict[7].This section proves the MLE exists and it is ii. lim → lim →
unique. Let us rewrite the Formula (6) as
∑
∑ ln It is straightforward to findlim → ∞ and
(18)
∑ lim 0 . It is obvious that the
→
Let write RHS of (18) as
lim → lim → . The condition i holds. For
∑
∑ ln (19) condition ii,
∑
The first derivative of lim → 0.
∑ ∑ ∑
(20)
∑ Let max , ,,… ,…. . The
Let ln and , the numerator of RHS ∑ ln ∑ ln
(20) is rewritten as
∑
∑ ∑ ∑ (21)
It is obvious
According to Cauchy-Schwarz inequality, lim 1;
0 (22) →
The equality holds when there exist constants and lim 0;
(not both of them are 0) for all the 1 →
(23) So
i.e. ∑ ln

Lim ln ln ln
→
ln (24) ∑
Simplify the (23) as ln /
Since / 1, the ∑ ln / 0
|ln | (25)
lim → 0 (26)
As for ,
The condition ii therefore holds. For two monotonic
The (24) holds only when and are both 0. This
curve crosses, the cross point is only one. It thus
contradicts to the condition of equality of (23). Therefore,
concludes that the MLE estimator exists and it is unique.
the (23) does not hold. The 0and 0. So
V. EQUIVARIANCE OF ESTIMATORS
0
Life data in engineering is common to change in it scale.
For example, the time to falure is perhaps in hours, or in
The in (19) is strictly monotonic. days, in weeks. The shape parameer that characterizes the
reliability should not change with the scale. The estimator
This monotonicity of RHS of (18) is not sufficient to for shape parameter should therefore unique regardless of
guaranty the existence of the solution. The plot of LHS of the scale. In statistics, it is named equivariance.
(18) is as the blue line

804
Authorized licensed use limited to: UNIVERSIDAD POLITECNICA SALESIANA. Downloaded on September 30,2020 at 20:34:55 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the 2015 IEEE IEEM

Without losing the generality, we write atime to failure in (Maximum Likelihood Method). It is LCI in
the new scale as Table 2-4.
; 0 (27)
The is positive constant. The formula to estimate the The simulation results are shown in Table from 2 to 4.As
shape parameter in the new scale is the confidence interval is set for 95%. The error tolerated
∑ should be around 5%. However, for the extreme data size,
∑ ln (28) the error exceeds 5% significantly, around 15% as shown
∑
Substitute the (27) to (28). The constant is canceled and in Table 2. It is around 5% for middle data size as N=80
the (28) is equivalent to (18), i.e. they have the same as shown in Table 2. This means the confidence interval
solution. does not work for extreme small data size, only works for
The scale parameter is estimated from middle and large data sample size.
/
∑ / (29) Error of point estimation decreases, when the data size
As expected, the scale parameter scales with the scale of becomes bigger, as shown in the Fig 4. The error of CI
the life data. also decreases when data size bigger. However, the
The moment method use the following formula to CItend to be stable when the data size reaches around 100
estimate as shown in Fig 5. The CI length shows significant
difference between the two methods. The length from
∑
(30) likelihood method always outperforms moment method,
∑
especially when the data size is big as shown in Fig 6.
Substitute the to (30), it is readily to find
∑ ∑
. The solution to is thus equal. The scale Conclusively, for point estimation, for all the data size
∑ ∑
and Weibull models, the two methods has similar
parameter is estimated from
performance. For CI, the likelihood method outperforms
Γ 1 (31) the moment method, especially for the big data size.
Substitute the to (31), it is readily to find 0,25

, it is the same as (29). Both the likelihood

method and the moment hold the equivariance.
0,20

Likelihood Method
0,15 Moment Method

VI. SIMULATION STUDY

MSE %

0,10

The simulation is carried out on the Matlab platform. 0,05

Random variables are generated by Wblrnd function. The 0,00

random data is drew from Weibull distribution (

1, 2 ), ( 10, 0.8 ), ( 100, 1 ), each
0 2000 4000 6000 8000 10000

Sample Size
represents the shape parameter equal to 1, greater than 1 Fig 4. MSE of against sample size
and less than 1. The sample size N is from the small
sample size to large sample size (N=5, N=25,
18

N=50,N=80,N=160, N=500, N=1000, 16

Percentage of CI within Interval

Moment

N=5000,N=10,000). As the reliability data ranges mostly 14

Likelihood

from 20 to 200, the chosen Nsaredenser from this range 12

than that from other. 10

For each sample size, total 10,000 time simulations are 6

run, in order to ensure the sufficiency of evaluated
performance. For the point estimation, the performance is 4
0 2000 4000 6000 8000 10000 12000

defined how close the estimators to the true parameters. Sample Size

Fig 5. Percentage of estimator within CI

This paper chooses the MSE to define the closeness
100

∑
Percentage of length of M longer than L

95
(32) 90

85

Other criteria are

80

 The percentage of the confident interval derived

75

70

from (17) contains the true parameters. It is the 65

CI in the tables 2-4. 60

 The percentage of the length of CI of M

0 2000 4000 6000 8000 10000 12000

Sample Size

(Moment method) is longer than that of L Fig 6. Percentage of CI Length of Moment longer than Likelihood

805
Authorized licensed use limited to: UNIVERSIDAD POLITECNICA SALESIANA. Downloaded on September 30,2020 at 20:34:55 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the 2015 IEEE IEEM

(10,0.8) N=1000 N=5000 N=10000

VII. CONCLUSION M L M L M L
MSE % 0.05 0.04 0.02 0.02 0.01 0.01
MSE % 0.04 0.02 0.02 0.01 0.01 0.01
Both maximum likelihood method and moment method LCI % 100 100 100
have unique solution and their estimators are equivariant. LCI % 100 100 100
From the simulation results, the performance on the point
estimation shows very close for all the range of sample Table 4. Simulated results for ( 100, 1)
size between these two methods. Moment method is (100,1) N=5 N=25 N=50
M L M L M L
surprisingly as efficient as likelihood in terms of point
MSE % 0.49 0.48 0.22 0.21 0.15 0.15
estimator. However, for the confidence interval, for MSE % 0.91 0.92 0.21 0.19 0.14 0.12
middle and large sample size, the likelihood method LCI % 47 53 61
outperforms moment method significantly. Likelihood has LCI % 80 99 100
narrower confidence interval. For extreme small data size,
likelihood’s outperformance over moment is just slightly. (100,1) N=80 N=160 N=500
Generally, the likelihood shows advantage for middle and M L M L M L
large data size. Conclusively, for life data analysis, it MSE % 0.12 0.11 0.09 0.08 0.05 0.05
suggest using maximum likelihood parameter method for MSE 0.11 0.09 0.08 0.06 0.04 0.04
two-parameter Weibull distribution. %
LCI % 67 78 93
APPENDIX LCI % 100 100 100
Table 2. Simulated results for ( 1, 2)
(1,2) N=5 N=25 N=50 (100,1) N=1000 N=5000 N=10000
M L M L M L M L M L M L
MSE % 0.23 0.23 0.10 0.10 0.1 0.1 MSE % 0.03 0.03 0.02 0.02 0.01 0.01
MSE 0.89 0.92 0.18 0.19 0.1 0.1
MSE % 0.03 0.02 0.01 0.01 0.01 0.01
CI (%) 16.4 17.0 6.1 6.1 5.7 5.7
CI (%) 13.6 15.0 6.1 6.6 5.6 5.3 LCI % 100 100 100
LCI % 66 65 64 LCI % 99 100 100
LCI % 44 55 62
REFERENCES
(1,2) N=80 N=160 N=500
M L M L M L [1] H. Rinne. (2009). The Weibull distribution a
MSE % 0.06 0.06 0.04 0.04 0.02 0.02
handbook. Available:
MSE % 0.09 0.09 0.06 0.06 0.04 0.04
CI (%) 4.8 4.9 5.3 5.1 5.1 5.0
http://www.crcnetbase.com/isbn/978-1-4200-
CI (%) 5.1 5.5 5.1 5.3 5.3 5.4 8743-7
LCI % 64 63 68 [2] E. E. Elmahdy, "A new approach for Weibull
LCI % 67 76 92 modeling for reliability life data analysis,"
Applied Mathematics and Computation, vol. 250,
(1,2) N=1000 N=5000 N=10000 pp. 708-720, Jan 1 2015.
M L M L M L
[3] S. Y. Grodzensky, "Reliability models based on
MSE % 0.02 0.02 0.01 0.01 0.005 0.005
MSE % 0.03 0.02 0.01 0.01 0.008 0.008 modified Weibull distributions," Measurement
CI (%) 5.0 5.1 4.9 5.0 4.9 4.9 Techniques, vol. 56, pp. 768-774, Oct 2013.
CI (%) 5.0 5.3 5.2 5.1 5.1 5.4 [4] J. McCool, Using the Weibull distribution :
LCI % 71 87 95 reliability, modeling, and inference. Hoboken,
LCI % 97 100 100 N.J.: John Wiley & Sons, 2012.
[5] M. G. Kendall, A. Stuart, and J. K. Ord, The
Table 3. Simulated results for ( 10, 0.8)
advanced theory of statistics, 4th ed. London:
(10,0.8) N=5 N=25 N=50
M L M L M L Charles Griffin, 1977.
MSE % 0.67 0.63 0.28 0.27 0.20 0.19 [6] W. Q. Meeker and L. A. Escobar, Statistical
MSE % 0.93 0.90 0.24 0.19 0.16 0.16 methods for reliability data. New York ;
LCI % 42 64 77 Chichester: Wiley, 1998.
LCI % 89 100 100 [7] N. Balakrishnan and M. Kateri, "On the
maximum likelihood estimation of parameters of
(10,0.8) N=80 N=160 N=500
M L M L M L
Weibull distribution based on complete and
MSE % 0.16 0.15 0.11 0.10 0.06 0.06 censored data," Statistics & Probability Letters,
MSE % 0.13 0.09 0.09 0.06 0.05 0.04 vol. 78, pp. 2971-2975, Dec 1 2008.
LCI % 86 96 99
LCI % 100 100 100

806
Authorized licensed use limited to: UNIVERSIDAD POLITECNICA SALESIANA. Downloaded on September 30,2020 at 20:34:55 UTC from IEEE Xplore. Restrictions apply.