Professional Documents
Culture Documents
net/publication/316580210
CITATIONS READS
2 453
1 author:
Hassan Abu-Hassan
Birzeit University
9 PUBLICATIONS 41 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Hassan Abu-Hassan on 29 April 2017.
by
Hassan Abuhassan
A Dissertation
Submitted in Partial Fulfillment of the Requirements for the
Doctor of Philosophy Degree
Department of Mathematics
in the Graduate School
Southern Illinois University Carbondale
October, 2007
DISSERTATION APPROVAL
Hassan Abuhassan
Doctor of Philosophy
Approved by:
Bhaskar Bhattacharya
Edward Neuman
Salah-Eldin Mohammed
Todd Headrick
Graduate School
Southern Illinois University Carbondale
October 18, 2007
AN ABSTRACT OF THE DISSERTATION OF
There are several useful asymmetric location-scale families that are one pa-
rameter exponential families when the location parameter is known. In this case
inference is simple and the maximum likelihood estimator (MLE) and uniformly
minimum variance unbiased estimator (UMVUE) are important point estimators.
The Burr, largest extreme value, Pareto, power, Rayleigh, smallest extreme value,
mal distribution, eight new competitors for these distributions are obtained.
half normal distribution HN (µ, σ 2 ) and gave confidence intervals for the parame-
ters, we give a better confidence interval for µ.
We also studied the Pareto, Rayleigh, and Weibull distributions and give
confidence intervals for the parameters. In the case of the Pareto distribution
P areto(σ, λ), the obtained confidence intervals for σ and λ seem to be new, while
in the case of the Weibull distribution W eibull(φ, λ) our contribution was that we
used robust estimators for φ and λ which are used in the iteration procedures to
find the MLEs.
ii
DEDICATION
iii
ACKNOWLEDGMENTS
I would like to thank Dr. David Olive for his invaluable assistance and insights
leading to the writing of this paper. My sincere thanks also goes to Dr. Bhaskar
Bhattacharya, Dr. Edward Neuman, Dr. Salah-Eldin Mohammed, and Dr. Todd
who contributed to my work in so many ways. Special thanks to Diane Fritcher for
your endless assistance, and Linda Gibson for your patient help on my thesis-writing.
At last, I would like to thank all my friends and family, particularly my father,
my wife, and my kids for their patience and support, my sisters and their families
for their help and hospitality.
iv
TABLE OF CONTENTS
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
Dedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
v
2.16 The HTEV Distribution . . . . . . . . . . . . . . . . . . . . . . . . 43
Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Vita . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
vi
LIST OF TABLES
3.1 Actual Coverage Levels for Nominal 90% Confidence Interval for σ 2 when
µ is unknown for sample sizes ranging from 5 to 500 . . . . . . . . . . . 53
3.2 Actual Coverage Levels for Nominal 90% Confidence Interval for σ 2 when
3.5 Actual Coverage Levels for Nominal 95% Confidence Interval for σ 2 when
µ is known for sample sizes ranging from 5 to 500 . . . . . . . . . . . . 55
3.6 Actual Coverage Levels for Nominal 95% Confidence Interval for µ for
3.9 Actual Coverage Levels for Nominal 95% Confidence Interval for λ2 when
θ is known for sample sizes ranging from 5 to 500 . . . . . . . . . . . . 57
3.10 Actual Coverage Levels for Nominal 95% Confidence Interval for λ2 for
vii
3.13 Actual Coverage Levels for Nominal 95% Confidence Interval for λ when
3.14 Actual Coverage Levels for Nominal 95% Confidence Interval for θ for
sample sizes ranging from 5 to 500 . . . . . . . . . . . . . . . . . . . . 61
3.15 Actual Coverage Levels for Nominal 95% Confidence Interval for λ when
3.18 Actual Coverage Levels for Nominal 95% Confidence Interval for λ for
sample sizes ranging from 5 to 500 . . . . . . . . . . . . . . . . . . . . 65
3.19 Actual Coverage Levels for Nominal 95% Confidence Interval for φ and
3.22 Actual Coverage Levels for Nominal 95% Confidence Interval for φ and
λ for sample sizes ranging from 25 to 500 . . . . . . . . . . . . . . . . . 68
3.23 Actual Coverage Levels for Nominal 95% Confidence Interval for φ and
viii
3.26 Actual Coverage Levels for Nominal 95% Confidence Interval for φ and
3.27 Actual Coverage Levels for Nominal 95% Confidence Interval for µ and
σ for sample sizes ranging from 25 to 500 . . . . . . . . . . . . . . . . . 71
3.28 Actual Coverage Levels for Nominal 95% Confidence Interval for µ and
3.31 Actual Coverage Levels for Nominal 95% Confidence Interval for µ and
σ for sample sizes ranging from 25 to 500 . . . . . . . . . . . . . . . . . 73
3.32 Actual Coverage Levels for Nominal 95% Confidence Interval for µ and
3.35 Sample means and standard deviations of the MLEs for µ and σ using
sample sizes ranging from 25 to 500 . . . . . . . . . . . . . . . . . . . . 75
3.36 Sample means and standard deviations of the MLEs for µ and σ using
ix
LIST OF FIGURES
x
CHAPTER 1
1.1 INTRODUCTION
There are several useful asymmetric location-scale families that are one pa-
rameter exponential families when the location parameter is known. In this case
inference is simple and the maximum likelihood estimator (MLE) and uniformly
the exponential distribution. By applying the same transformation to the half nor-
mal distribution, eight new competitors for these distributions are obtained.
for the original distributions and the invariance principle. Pewsey [15] studied the
half normal distribution HN (µ, σ 2 ) and gave confidence intervals for the parame-
confidence intervals for the parameters. In the case of the Pareto distribution
P areto(σ, λ) which is studied extensively by Arnold (1983) [1], the obtained con-
fidence intervals for σ and λ seem to be new, while in the case of the Weibull
distribution W eibull(φ, λ) our contribution was that we used robust estimators for
φ and λ which are used in the iteration procedures to find the MLEs.
1
Definition 2. The population median absolute deviation is MAD(Y) =
MED(|Y − MED(Y)|).
Xk
f (x|θ) = h(x)c(θ) exp[ wi (θ)ti (x)] (1.1)
i=1
smallest integer where the above equation holds. If k = 1 and the exponential family
is regular then it is called one parameter regular exponential family denoted by a
1P–REF.
Suppose that Y = t(W ) and W = t−1(Y ) where W has a pdf with parameters
θ, the transformation t does not depend on any unknown parameters, and the pdf
of Y is
dt−1 (y)
fY (y) = fW (t−1 (y))| |.
dy
[4]. If W1 , W2, . . . , Wn are iid with pdf fW (w), assume that the MLE of θ̂ is θ W (w)
where the wi are the observed values of Wi and w = (w1, w2 , . . . , wn )T .
If Y1 , Y2 , . . . , Yn are iid and the yi are the observed values of Yi , then the
likelihood is
! n
Yn
dt−1 (yi ) Y Yn
LY (θ) = | | fW (t−1 (yi )|θ) = c fW (t−1 (yi )|θ)
i=1
dy i=1 i=1
X
n X
n
log(LY (θ)) = d + log[fW (t−1 (yi )|θ)] = d + log[fW (wi |θ)] = d + log[LW (θ)]
i=1 i=1
2
where wi = t−1(yi ). Hence maximizing the log(LY (θ)) is equivalent to maximizing
θ̂ Y (y) = θ̂W (w) = θ̂ W (t−1 (y1), t−1 (y2), . . . , t−1(yn )). (1.2)
This result is useful since if the MLE based on the Wi has simple inference,
then the MLE based on the Yi will also have simple inference. For example, If
W1 , W2, . . . , Wn are iid ∼ EXP (θ = log(σ), λ) and Y1 , Y2, . . . , Yn are iid Pareto
(σ = eθ , λ), then Y = eW = t(W ) and W = log(Y ) = t−1(Y ). The MLE of (θ, λ)
based on the Wi is (θ̂, λ̂) = (W(1), W − W(1)). Hence by (1.2) and invariance, the
1X
n
λ̂ = W − W(1) = log(Yi ) − log(Y(1) ).
n i=1
duces the new transformed distributions and gives a review of the literature. Chapter
2 studies inference in both the exponential distribution and the half-normal distri-
which is better than the confidence interval given by Pewsey [15]. The Burr, largest
extreme value, Pareto, power, Rayleigh, smallest extreme value, truncated extreme
value, and Weibull distributions are obtained by transforming the exponential dis-
tribution. By applying the same transformation to the half normal distribution, new
competitors for these distributions are obtained. We studied each of these sixteen
transformed distributions in this chapter by allocating one section for each distri-
bution. In each section we tried to give the pdf of the distribution, and its graph
for selected values of it’s parameter(s). The MLEs and confidence intervals for the
3
parameter(s) were given for several distributions.
establish the actual coverage of the confidence intervals presented in chapter two.
Sample sizes used in the simulations ranges from 5 to 500 and the number of runs
ranges from 100 to 5000. The results of the simulation studies are found to give
parameter exponential distribution and X ∼ G(1, λ) where G stands for the Gamma
distribution. Inference for this distribution is discussed in Johnson and Kotz (1970,
4
CHAPTER 2
STATISTICAL DISTRIBUTIONS
This is a location–scale family. If X ∼ EXP (λ), then X ∼ EXP (0, λ) has a one
parameter exponential distribution and X ∼ G(1, λ) where G stands for the Gamma
distribution. Inference for this distribution is discussed in Johnson and Kotz (1970,
p. 219) and Mann, Schafer, and Singpurwalla (1974, p. 176).
0.4
0.2
0.0
0 2 4 6 8 10
5
Let Y1 , ..., Yn be iid EXP (θ, λ) random variables. Let Y(1) = min(Y1 , ..., Yn).
Let Dn = nλ̂. For n > 1, an exact 100(1 − α)% confidence interval (CI) for θ is
Proof:
6
Pn Pn
2 i=1 (Yi −θ) 2 i=1 (Yi −θ)
= P( χ22n,1−α/2
<λ< χ22n,α/2
).
√ √ √ √
Using χ2n,α/ n ≈ 2zα + n, it can be shown that n CI length converges
to λ(z1−α/2 − zα/2) for CIs (2.2) and (2.3) (in probability). It can be shown that n
length CI (2.1) converges to −λ log(α) [12]. Proof:
√ √ √
χ22n,α/2/ 2n ≈ 2zα/2 + 2n...............(A)
√ √ √ √ √ √
⇒ χ22n,α/2/ n = 2χ22n,α/2/ 2n ≈ 2[ 2zα/2 + 2n]
√ √
= 2zα/2 + 2 n = 2(zα/2 + n)...............(B)
Now
Pn Pn
√ √ 2 i=1 (Yi − θ) 2 i=1 (Yi − θ)
n CI length = n( − )
χ22n,α/2 χ22n,1−α/2
Pn P
2 2 ni=1 (Yi − θ)
i=1 (Yi − θ)
=( 2
√− 2 √ ) and by (B) above, this equals:
χ2n,α/2/ n χ2n,1−α/2/ n
P P
2 ni=1 (Yi − θ) 2 ni=1 (Yi − θ)
( √ − √ )
2(zα/2 + n) 2(z1−α/2 + n)
Pn
(Yi − θ)[z1−α/2 − zα/2] 1/n
= i=1 √ √
(zα/2 + n)(z1−α/2 + n) 1/n
Pn
1 i=1 (Yi −θ)[z1−α/2 −zα/2 ] D
= n an
−→ λ[z1−α/2 − zα/2].
since
D
an −→ 1.
Also
α−1/n−1 − 1 0
lim n[α−1/n−1 − 1] = lim =
1/n 0
so by L’Hopital Rule the limit equals
α−1/n−1 log(α)(−1/n2 )
lim
−1/n2
7
Hence, n length CI (2.1) converges to
2 −(y − µ)2
f (y) = √ exp ( )
2π σ 2σ 2
where σ > 0 and y ≥ µ and µ is real. This is a location-scale family. Let Φ(y)
y −µ
F (y) = 2Φ( )−1
σ
0.2
0.0
0 1 2 3 4 5
8
This is an asymmetric location–scale family that has the same distribution as
µ + σ|Z| where Z ∼ N (0, 1). Note that Z 2 ∼ χ21. Hence the formula for the rth
moment of the χ21 random variable can be used to find the moments of Y [12].
MED(Y ) = µ + 0.6745σ.
MAD(Y ) = 0.3990916σ.
Notice that
2 −1 2
f (y) = √ I(y > µ) exp ( 2 )(y − µ)
2π σ 2σ
X
Tn = (Yi − µ)2 ∼ G(n/2, 2σ 2 ).
n 1 X
log(L(σ 2)) = d − log(σ 2) − 2 (yi − µ)2 .
2 2σ
Hence
d 2 −n 1 X set
2
log(L(σ )) = 2
+ 2 2
(yi − µ)2 = 0,
d(σ ) 2(σ ) 2(σ )
P
or (yi − µ)2 = nσ 2 or
1X
σ̂ 2 = (Yi − µ)2 .
n
Notice that
d2
log(L(σ 2)) =
d(σ 2 )2
P
n (yi − µ)2 n nσ̂ 2 2 −n
− = − = 2 < 0.
2
2(σ ) 2 2
(σ ) 3
σ 2 =σ̂ 2
2
2(σ̂ ) 2
(σˆ2)3 2 2σ̂
9
Thus σ̂ 2 is the UMVUE and MLE of σ 2 if µ is known, while µ̂ = Y(1) is the ML
interval for σ 2 is
Tn Tn
( 2
, ) (2.4)
χn,1− α χ2n, α
2 2
Pn 2
Pn
where Tn = i=1 (Yi − µ) . If µ is unknown, let Dn = i=1 (Yi − Y(1))2 . Then a
Dn Dn
( 2
, 2
) (2.5)
χn−1,1− α χn−1, α
2 2
Pewsey [15] states that the limiting distribution of the MLE µ̂ = Y(1) is
1 1
D
(µ̂ − µ)/ σΦ−1 2 + 2n
−→ EXP (1)
where Φ(·) denotes the distribution function of the standard normal distribution.
10
Proof:
⇒ P (Y(1) + σΦ−1 ( 12 + 1
2n
) log( α2 ) < µ < Y(1) + σΦ−1 ( 12 + 1
2n
) log(1 − α2 )) ≈ 1 − α.
1 1 13
(µ̂ + σ̂ log(α)Φ−1 ( + )(1 + 2 ), µ̂). (2.7)
2 2n n
If σ is known then a large-sample confidence interval for µ with the same nominal
confidence level is obtained by substituting σ for σ̂ in (2.7). Pewsey (2002, p. 1048)
said that
D
Dn −→ χ2n−1 .
This can’t happen as since the righthand limit depends on n, hence we introduce
Then
D
Dn − Tn −→ −σ 2χ22.
Proof.
Pn
Dn = − µ + µ − Y(1) )2
i=1 (Yi
Pn Pn Pn
= i=1 (Yi − µ)2 + 2 i=1 (Yi − µ)(µ − Y(1) ) + i=1 (µ − Y(1))2
11
Pn
= Tn + n(µ − Y(1) )2 − 2(Y(1) − µ) i=1 (Yi − µ).
Pn
Dn Tn 1 1 n(Y(1) −µ) i=1 (Yi −µ)
then σ2
= σ2
+ n σ2
[n(Y(1) − µ)]2 − 2[ σ
][ nσ
]
⇒
Pn
Dn − T n 1 n(Y(1) − µ) 2 n(Y(1) − µ) i=1 (Yi − µ)
= [ ] − 2[ ][ ] (2.8)
σ2 n σ σ nσ
Pewsey (2002, p. 1048) showed that
Y(1) −µ D
−→ EXP (1).
[σΦ−1 ( 12 + 2n
1
)]
and since
Φ−1 ( 12 + 2n
1
) D
√π −→ 1,
2
/n
Y(1) −µ Φ−1 ( 12 + 2n
1
) D
√π −→ EXP (1).
[σΦ−1 ( 12 + 2n1 )] 2
/n
Hence
n(Y(1) −µ) D
√π −→ EXP (1).
2
σ
Pn q
i=1 (Yi −µ) D 2
Z= nσ
−→ E(Z) = π
Yi −µ
where Zi = σ
∼ HN (0, 1).
Since
n(Y(1) −µ) D pπ
σ
−→ 2
EXP (1),
1 n(Y(1) −µ) 2 D
n
[ σ ] −→ 0.
12
Hence
pπ q
Dn −Tn D
σ2
−→ 0 − 2 2 EXP (1) π2 = −2EXP(1).
Or
D
Dn − Tn −→ −σ 2χ22. QED
Pn−p
Let Vn = σ 2χ22 and Tn−p = i=1 (Yi − µ)2 . Then
X
n
Dn = Tn−p + (Yi − µ)2 − Vn
i=n−p+1
where
Vn D
σ2
−→ χ22.
Hence Pn
Dn i=n−p+1 (Yi − µ)2 Vn
=1+ −
Tn−p Tn−p Tn−p
Hence
Dn D
Tn−p
−→ 1.
Since
n−p
Tn−p X Yi − µ) 2
= ( ) ∼ χ2n−p ,
σ2 i=1
σ
Dn
≈ χ2n−p
σ2
where p is a nonnegative integer. Pewsey (2002) used p = 1.
is:
s s
Tn Tn
( , ) (2.9)
χ2n,1− α χ2n, α
2 2
13
and if µ is unknown, then a large sample 100(1 − α)% confidence interval for σ is
s s
Dn Dn
( , ). (2.10)
χ2n−1,1− α χ2n−1, α
2 2
√ √ √ √
Using χ2n,α/ n ≈ 2Zα + n, it can be shown that nCI length converges
√
in probability to 2σ 2[Z1−α/2 − Zα/2] for CIs (2.4) and (2.5). Also it can be shown
p
that nCI length converges to −σ log(α) π/2 for CI (2.7).
Proof:
√ √ Tn Tn
nCI length = n( 2 − 2 )
χn,α/2 χn,1−α/2
Pn 2
Pn 2
i=1 (Yi − µ) i=1 (Yi − µ)
=( √ − √ )
χ2n,α/2/ n χ2n,1−α/2/ n
Pn 2
Pn
(Y i − µ) (Yi − µ)2 1/n
= ( √i=1 √ − √ i=1 √ )
2zα/2 + n 2z1−α/2 + n 1/n
√ Pn √ 2
1 2 i=1 (Yi −µ)2 [z1−α/2 −zα/2 ] D
= n an
−→ 2σ [z1−α/2 − zα/2]
since
D
an −→ 1.
1 1
since a Taylor series approximation of Φ−1 2
+ 2n
is (π/2)1/2/n.
14
2.3 THE BURR DISTRIBUTION
φ
If Y ∼ Burr(φ, λ), then the cdf of Y is F (y) = 1 − exp( − log(1+y
λ
)
) = 1 − (1 +
1
y φ )(− λ ) for y > 0 and the pdf of Y is
1 φy φ−1
f (y) = λ (1+y φ )1+ λ1
,y > 0, φ > 0, λ > 0.
Hence W ∼ EXP(λ).
1
Let Y = (eW − 1) φ . Then
1
FY (y) = P (Y ≤ y) = P ((eW − 1) φ ≤ y) = P (eW − 1 ≤ y φ ) = P (eW ≤ 1 + y φ) =
φ
P (W ≤ log(1 + y φ)) = 1 − exp( − log(1+y
λ
)
).
Hence Y ∼ Burr(φ, λ).
0.3
0.2
0.1
0.0
0 2 4 6 8 10
15
2.4 THE HBURR DISTRIBUTION
0.3
0.2
0.1
0.0
0 1 2 3 4 5
1
Let W = log(1 + Y φ ), then Y = r(W ) = (eW − 1) φ
1
2 w −1) φ ]φ−1 1
⇒ gW (w) = fY (r(w))| dr(w)
dw
|= √2
2πλ
w
exp(− 2λ 2)
φ[(e
1 | φ1 (ew − 1) φ −1 ew |
[(ew −1) φ ]φ +1
−1 1
2 φ(e w −1)(ew −1) φ w −1) φ ew
= √2
2πλ
w
exp(− 2λ 2) (ew −1)+1
| φ1 (e ew −1
|
16
1
2 φ(ew −1)
= √2
2πλ
w
exp(− 2λ 2) 1 | φ1 (ew − 1) φ −1 ew |
(ew −1) φ ew
w 2
= √ 2 e− 2λ2 , w ≥ 0.
2πλ
⇒ W ∼ HN (0, λ).
Y
n Qn Xn
2 n φn i=1 yiφ−1 −(log(yiφ + 1))2
L(λ) = f (yi ) = ( √ ) Qn φ
exp( 2
),
i=1
2πλ (y
i=1 i + 1) i=1
2λ
√ Xn
(log(yiφ + 1))2
log L = n log(2) − n log( 2π) − n log λ − 2
+ n log φ
i=1
2λ
X
n
+ ((φ − 1) log yi − log(yiφ + 1)).
i=1
Hence
Pn φ
d log L −n 2 i=1 (log(yi + 1))2
= + := 0
dλ λ 2λ3
or
s
Pn φ 2
Pn φ
2 i=1 (log(yi + 1)) i=1 (log(yi + 1))2
λ = , orλ̂ = .
n n
Pn φ 2
d2 log L n 3 i=1 (log(yi +1)) n
Notice that dλ2
|λ̂ = λ̂2
− λ̂4
= λ̂2
[1 − 3] < 0. Hence λ̂ is the MLE of
λ if φ is known.
17
2.5 THE LARGEST EXTREME VALUE DISTRIBUTION
1 y−θ y−θ
f (y) = exp(−( )) exp[− exp(−( ))]
σ σ σ
where y and θ are real and σ > 0. This distribution is a location scale family. The
cdf of Y is
y−θ
F (y) = exp[− exp(−( ))].
σ
0.1
0.0
−2 −1 0 1 2 3 4 5
Figure 2.5. Plot of the pdf of the Largest Extreme Value Distribution
σ ≥ 0, −∞ < y < ∞
18
P (− (Y σ−θ) ≤ log w) = P (Y − θ ≥ −σ log w) = P (Y ≥ θ − σ log w) = 1 −
⇒ W ∼ EXP (1).
2 1 (y − θ) 1 −(y − θ) 2
f (y) = √ exp(− ) exp(− [exp( )] ), y ∈ R, θ ∈ R, λ > 0,
2π λ λ 2 λ
0.2
0.1
0.0
−2 0 2 4 6
exp(− Y λ−θ )
−2(y−θ)
− exp( )
⇒ fY (y) = gW (s(y))| ds(y)
dy
|= √2
2π
exp( 2
λ
)| exp(− y−θ
λ
) −1
λ
|
19
= √2 1
2π λ
exp(− (y−θ)
λ
) exp(− 12 [exp( −(y−θ)
λ
)]2), y ∈ R, θ ∈ R, λ > 0,
= √2
2πλ
exp(− (−λ log(w)+θ−θ)
λ
) exp(− 12 [exp(− (−λ log(w)+θ−θ)
λ
]2).| −λ
w
|
2
= √2 exp(log(w)) exp[− 12 w2 ]( wλ ) = √ 2 w exp(− w ) wλ
2πλ 2πλ 2
w 2
= √2 e− 2 , w ≥ 0.
2π
⇒ W ∼ HN (0, 1).
where y ≥ σ, σ > 0, and λ > 0. The cdf of Y is F (y) = 1 − (σ/y)1/λ for y > σ.
Hence
1 −w 1 −w 1 −w (w−log σ)
−1
fW (w) = −σ λ e λ
λ
= λ1 σ λ e λ = λ1 elog σ λ e λ = λ1 e− λ , w ≥ log σ
1 λ 1
1 1 σ
⇒ f (y) = λ1 σ λ y − λ −1 = λ
1 ,y ≥ σ
y 1+ λ
⇒ Y ∼ P AR(σ, λ).
20
Pareto Distribution PAR(sigma=1, lambda=1)
1.0
0.8
0.6
f(x)
0.4
0.2
1 2 3 4 5
Let θ = log(σ). The MLE (θ̂, λ̂) = (W(1), W − W(1)), and by invari-
P
ance, the MLE (σ̂, λ̂) = (eθ̂ , W − W(1)) = (eW(1) , n1 ni=1 log Yi − log Y(1)) =
P P
(Y(1), n1 ni=1 (log Yi − log Y(1))) = (Y(1), n1 ni=1 log(Yi /Y(1))).
If σ is known,
Pn
i=1 log(Yi /σ)
λ̂ =
n
is the UMVUE and MLE of λ.
Inference is simple. If θ = log(σ) so σ = eθ , then a 100 (1 − α)% CI for θ is
21
A 100(1 − α)% CI for λ is
2Dn 2Dn
( , ). (2.12)
χ22(n−1),1−α/2 χ22(n−1),α/2
Hence
d log L Pn Pn
dλ
= − nλ − n log
λ2
σ
− i=1 log yi (− λ12 ) = − nλ − n log
λ2
σ
+ 1
λ2 i=1 log yi := 0
P P n y
log( i )
⇒ λ1 [ ni=1 log yi − n log σ] = n ⇒ λ̂ = i=1 n σ̂ .
Notice that
2 X
n
d2 log L n 2n log σ
| = + − log yi
dλ2 λ̂ λ̂2 λ̂3 λ̂3 i=1
P P
n 2( ni=1 log σ)n3 2n3 ni=1 log yi
= (1) + Pn − Pn
λ̂2 ( i=1 log yσi )3 ( i=1 log yσi )3
n n3 X
n X
n
= 2 + Pn yi 3 [2 log σ − 2 log yi ]
λ ( i=1 log σ
) i=1 i=1
n 2n 3 X
n
yi n n3
= 2 + Pn (− log ) = 2 − 2 Pn
λ ( i=1 log yσi )3 i=1
σ λ ( i=1 log yσi )2
n2 2n3
= Pn yi 2 − Pn < 0.
( i=1 log σ ) ( i=1 log yσi )2
Hence λ̂ is the MLE of λ given σ.
11 log(y) − log(σ)
f (y) = exp[−( )]I(y(1) ≥ σ),
yλ λ
22
Hence the likelihood
1 X log(yi ) − log(σ)
n
L(λ, σ) = c exp[− ( )]I(y(1) ≥ σ)
λn i=1
λ
Xn
log(yi ) − log(σ)
log L(λ, σ) = [d − n log(λ) − ( )]I(y(1) ≥ σ)
i=1
λ
Xn
wi − θ)
log L(λ, σ) = [d − n log(λ) − ( )]I(w(1) ≥ θ),
i=1
λ
which has the same form as the log likelihood of the EXP(θ, λ) distribution.
Hence (λ̂, θ̂) = (W − W(1), W(1)), and by invariance, the MLE
the usual way since log L is unbounded on the random variable Y, log L must be
maximized subject to the constraint:
σ̂ ≤ minYi .
σ̂ = minYi = Y(1)
so, Pn
i=1 log( YY(1)
i
)
λ̂ = .
n
23
2.8 THE HPARETO DISTRIBUTION
0.8
0.6
f(x)
0.4
0.2
1 2 3 4 5
Proof:
From the graph , the mode occurs at Y(1) = θ. Also from the formula of the pdf of
2
Y we want to maximize √ 2
2πλy
exp( −(log(y)−log(θ))
2λ2
) and that happens at Y(1).
df (y)
In addition, dy
has no zeros on (θ, ∞). Proof:
(log y−log θ)2
log(f (y)) = c − log y − 2λ2
So d
dy
log f (y) = 0 − 1
y
− 1
λ2
(log y − log θ) 1y = y1 [−1 − 1
λ2
(log( yθ ))] := 0
or
2
1= −1
λ2
log( yθ ), or y = θe−λ which is not in support of Y .
24
Let Y = eW , then W = s(Y ) = log(Y )
⇒
ds(y) 2 −(log(y) − µ)2 1
fY (y) = gW (s(y))| |=√ exp( )| |
dy 2πσ 2σ 2 y
(w ≥ µ ⇒ log(y) ≥ µ ⇒ y ≥ eµ ≥ 0 ⇒ |y| = y)
2 −(log(y) − µ)2
= √ exp( )
y 2πσ 2σ 2
log(y) > µ, so y > eµ = θ.
2
Let θ = eµ , λ = σ,then f (y) = √ 2
2πλy
exp( −(log(y)−log(θ))
2λ2
), y > θ,
⇒ Y ∼ HP AR(θ, λ).
w 2
⇒ g(w) = fY (r(w))| dr(w)
dw
|= √ 2
2πλew
exp[ −(log(e 2λ)−log(θ))
2 ]ew
2
= √2
2πλ
exp( −(w−µ)
2λ2
),where µ = log(θ).
2
So g(w) = √2
2πσ
exp(− (w−µ)
2σ 2
), w ≥ µ, λ = σ
⇒ W ∼ HN (µ, σ).
P P −log θ)2
= c − n log λ − ni=1 log yi − ni=1 (log yi2λ 2 .
Pn
In order to maximize log L, we need to minimize i=1 (log yi − log θ)2 subject to the
constraint y(1) ≥ θ. This occurs when θ = y(1). Hence MLE θ̂ = y(1).
25
1 X
n
−n
= + 3 (log yi − log y(1))2 := 0
λ λ i=1
or
Pn
2 i=1 (log yi − log y(1))2
λ = ,
n
or r Pn
i=1 (log yi − log y(1))2
λ̂ = .
n
Notice that
3 X
n
d2 log L n n
2
|λ̂ = − (log yi − log y(1))2 = [1 − 3] < 0
dλ λ̂2 λ̂4 i=1 λ̂2
unknown is
nλ̂2 nλ̂2
( , )
χ2n−1,1− α χ2n−1, α
2 2
or
Pn Pn
i=1 (log yi − log y(1))2 i=1 (log yi − log y(1))
2
( , ). (2.13)
χ2n−1,1− α χ2n−1, α
2 2
Taking square roots of the endpoints gives a large sample 100(1 − α)% CI for λ.
A CI for µ = log(θ) is given by (2.7), that is, a large sample 100(1 − α)% CI for
log(θ) is
1 1 13
(log θ̂ + λ̂ log(α)Φ−1 ( + )(1 + 2 ), log θ̂)
2 2n n
26
so exponentiate the endpoints of (2.7) for a CI for θ :
1 1 13
(θ̂ exp(λ̂ log(α)Φ−1 ( + )(1 + 2 )), θ̂)
2 2n n
or
1 1 13
(Y(1) exp(λ̂ log(α)Φ−1 ( + )(1 + 2 )), Y(1)). (2.15)
2 2n n
2
1
0
27
⇒ W ∼ EXP (λ).
Hence
−n X −1
n
d log L
= + 2
log yi := 0
dλ λ i=1
λ
or
−1 X
n
n
= 2 log yi ,
λ λ i=1
or
Pn
− i=1 log yi
λ̂ = .
n
Notice that
Pn Pn
n 2 X
n
d2 log L n log yi n log yi n
2
|λ̂ = + 2 log yi = +2 i=1 = −2n i=1 = [1−2] < 0.
dλ 2 3
λ̂ λ̂ i=1 λ̂ 2 λ̂ 3 λ̂ 2 nλ̂ 3 λ̂2
Hence λ̂ is the MLE of λ.
28
2.10 THE HPOWER DISTRIBUTION
If Y ∼ HP OW (λ), then
2
f (y) = √2 1
2π λy
exp( −(log(y))
2λ2
)I[0 ≤ y ≤ 1], λ > 0.
10
8
6
f(x)
4
2
0
q λ2
2 2 2e 2
This distribution is unimodal with mode at y = e−λ and f (e−λ ) = π λ
.
Proof:
(log y)2
log(f (y)) = c − log y − 2λ2
.
d
So dy
log f (y) = 0 + − 1y − 1
λ2
(log y) 1y = y1 [−1 − 1
λ2
(log(y)] := 0
or
−1 2
1= λ2
log(y), or y = e−λ ,
or
2 2
d
dy
f (y) = √ 2
2πλy
exp( −(log(y))
2λ2
−2
)( 2λ 1
2 log(y) y ) + exp(
−(log(y))
2λ2
−2
)( √2πλy 2)
2
= √ −2
2πλy 2
exp( −(log(y))
2λ2
)[ λ12 log(y) + 1] := 0, but the first two terms can’t be zero,
29
1 2
hence λ2
log(y) + 1 = 0 or −λ2 = log(y) or y = e−λ . Notice that
d2 f (y) −2 −(log(y))2 1
| −λ2 = √ exp( )( 2 )
dy 2 y=e 2πλy 2 2λ2 λ y
1 d −2 −(log(y))2
+( log(y) + 1)[ ( √ exp( ))] < 0
λ2 dy 2πλy 2 2λ2
since the first term is less than zero and the second term is zero because ( λ12 log(y)+1)
2 1 2 2
at y = e−λ is equal to λ2
log(e−λ ) + 1 = −1 + 1 = 0. Hence y = e−λ is a local
2
maximum for f (y). So the mode is at y = e−λ .
⇒ Y ∼ HP OW (λ).
−w ))2
⇒ g(w) = fY (r(w))| dr(w)
dw
|= √2 1
2πλ e−w
exp[ −(log(e
2λ2
]I[0 ≤ e−w ≤ 1]|e−w |
2
= √2 exp( −w ) I[0 ≤ w < ∞]
2πλ 2λ2
⇒ W ∼ HN (0, λ).
30
or
Pn 2
n i=1 [log yi ]
= ,
λ λ3
or r Pn
2
i=1 [log yi ]
λ̂ = .
n
Notice that
Pn
d2 log L n i=1 3[log yi]2 n
2
|λ̂ = − = [1 − 3] < 0.
dλ λ̂2 λ̂4 λ̂2
Hence λ̂ is the MLE of λ.
31
2.11 THE RAYLEIGH DISTRIBUTION
0.3
0.2
0.1
0.0
0 1 2 3 4 5
√
Let W = (Y − µ)2 , then Y = r(W ) = µ + W , and
√ √
FW (w) = gW (r(w))| dr(w)
dw
|= (µ+ w)−µ
σ2
exp(− 12 ( µ+ σ
w−µ 2
) )| 2√1w |
1 −w 2
= 2σ 2
exp( 2σ 2 ) ⇒ W ∼ EXP (2σ ).
√
If W ∼ EXP (2σ 2 ), let Y = W + µ. Then
√ √
FY (y) = P (Y ≤ y) = P (( W + µ) ≤ y) = P ( W ≤ y − µ) = P (W ≤ (y − µ)2 ) =
32
2
1 − exp( −(y−µ)
2σ 2
), y ≥ µ, σ > 0
⇒ Y ∼ R(µ, σ).
Let Y1 , Y2 , . . . , Yn be iid R(µ, σ), then the likelihood
Q Q Pn
L(σ) = ni=1 f (yi ) = ni=1 ( yiσ−µ 2 ) exp
1 yi −µ 2
i=1 − 2 ( σ ) ,
Hence
d log L X σ 2 (µ − yi )2σ 1 X yi − µ µ − yi
n n
= 4
− 2( )( 2 )
dσ i=1
yi−µ σ 2 i=1
σ σ
X
n
−2 X
n
(yi − µ)2
= + := 0
i=1
σ i=1
σ3
or
Pn
2n i=1 (yi − µ)2
= ,
σ σ3
or
Pn
i=1 (yi − µ)2
σb2 = ,
2n
or
r Pn
i=1 (yi − µ)2
b=
σ .
2n
Notice that
d2 log L
nX 2 X 3(yi − µ)2
n
2n 3(2n) 2n
| σ̂ = − = 2− = 2 [1 − 3] < 0.
σb4
dσ 2 σ̂ 2 σ̂ 4 σ̂ σ̂
i=1 i=1
33
The confidence interval for σ 2 when µ is known will be derived next.
Hence
1 X
n
d log L 2n
=− + 3 (yi − µ)2 := 0
dσ σ σ i=1
gives
Pn
2 i=1 (yi − µ̂)2
σ̂ = . (2.21)
2n
Also
d log L Xn
1 X
n
−1
=− (yi − µ) + 2 (yi − µ) := 0
dµ i=1
σ i=1
34
gives
X Pn
1 X
n n
−1 yi nµ
(yi − µ) = 2 (yi − µ) = i=1 − 2
i=1
σ i=1 σ2 σ
or
Pn X
n
σ2 yi
µ= [ i=1
− (yi − µ)−1 ].
n σ2 i=1
One way to find the MLE is by iteration using Newton’s method, where start-
ing values can be found using the method of moments. Newton’s method is used to
solve g(θ) = 0 for θ, where the solution is called θ̂, and uses
where
∂ ∂
g (θ)
∂θ1 1
... g (θ)
∂θp 1
.. ..
Dg (θ) =
. . .
∂ ∂
g (θ) . . .
∂θ1 p
g (θ)
∂θp p
If the MLE is the solution of the likelihood equations, then use g(θ) =
D = Dg (θ ). Then
∂ ∂2 X ∂2 n
Dij = gi (θ) = log(L(θ)) = log(f (xk |θ)),
∂θj ∂θi ∂θj ∂θi∂θj
k=1
and
1 X ∂2
n
1 D ∂2
Dij = log(f (Xk |θ)) → E log(f (X|θ)) .
n n ∂θi∂θj ∂θi∂θj
k=1
Newton’s method converges if the initial estimator is sufficiently close, but
√
may diverge otherwise. Hence n consistent initial estimators are recommended.
Newton’s method is also popular because if the partial derivative and integration
35
For example, the regularity conditions hold for a kP-REF. Then a 100 (1 − α)%
q
θ̂i ± z1−α/2 −D−1
ii
where
h i−1
D −1 = D ˆ .
g (θ)
This result follows because
q q
−Dii ≈ [I −1 (θ̂)]ii/n.
−1
Next, apply the above results to the Rayleigh (µ, σ) distribution (although no
check has been made on whether the regularity conditions hold for the Rayleigh
distribution which is not a 2P-REF).
Y
yi − µ 1 X 2
L(µ, σ) = exp − 2 (yi − µ) .
σ2 2σ
Notice that for fixed σ, L(Y(1) , σ) = 0. Hence the MLE µ̂ < Y(1) . Now the log
likelihood
X
n
1 X (yi − µ)2
log(L(µ, σ)) = log(yi − µ) − 2n log(σ) − .
i=1
2 σ2
Hence g1 (µ, σ) =
∂ X 1 n
1 X set
n
log(L(µ, σ)) = − + 2 (yi − µ) = 0,
∂µ i=1
yi − µ σ i=1
and g2 (µ, σ) =
1 X
n
∂ −2n set
log(L(µ, σ)) = + 3 (yi − µ)2 = 0,
∂σ σ σ i=1
36
p
To obtain initial estimators, let σ̂M = S 2 /0.429204 and µ̂M = Y −
2 n−1 2
1.253314σ̂M . These would be the method of moments estimators if SM = n
s
was used instead of the sample variance S 2. Then use µ0 = min(µ̂M , Y(1) − µ̂M ) and
pP
σ0 = (Yi − µ0 )2/(2n). Now θ = (µ, σ)T and
∂ ∂
g (θ)
∂µ 1
g (θ)
∂σ 1
D ≡ Dg (θ) = =
∂ ∂
g (θ)
∂µ 2
g (θ)
∂σ 2
Pn 1 n
Pn
− i=1 (yi −µ)2 − σ2 − σ23 i=1 (yi − µ)
.
2
Pn 2n 3
Pn 2
− σ3 i=1 (yi − µ) σ2
− σ4 i=1 (yi − µ)
So
Pn Pn −1
1 n
− i=1 (yi −µk )2 − σk2
− σ23 i=1 (yi − µk )
k
θk+1 = θ k −
g(θ k )
Pn Pn
− σ23 i=1 (yi − µk ) 2n
σk2
− 3
σk4 i=1 (yi − µk ) 2
k
where
Pn Pn
! ! − 1
− 1
i=1 (yi − µk )
i=1 (yi −µk ) σk2
g1 (θk ) ∂
log L(θ k )
g(θ k ) = = ∂µ
∂
=
.
g2 (θk ) ∂σ
log L(θ k ) −2n 1
Pn
σk
+ σk3 i=1 (yi − µk ) 2
This formula was iterated for 100 steps resulting in θ 101 = (µ101 , σ101)T . Then
37
2.12 THE HRAYLEIGH DISTRIBUTION
4
If Y ∼ HRAY (θ, λ), then f (y) = √ 4 (y
2πλ
− θ) exp( −(y−θ)
2λ2
) for θ > 0, λ >
0, y ≥ θ.
1.0
0.8
0.6
f(x)
0.4
0.2
0.0
⇒
ds(y) 2 −(y − θ)4
f (y) = gW (s(y))| |= √ exp( )|2(y − θ)|
dy 2πλ 2λ2
4 −(y − θ)4
=√ (y − θ) exp( )
2πλ 2λ2
for θ > 0, λ > 0, y ≥ θ,
√
Let W = (Y − θ)2 , then Y = r(W ) = W + θ
√ √
−( w+θ−θ)4
⇒ g(w) = f (r(w))| dr(w)
dw
| = √ 4 ( w + θ − θ) exp[
2πλ 2λ2
]| 2√1w |
4
√ 2
= √2πλ w exp( −w ) √1
2λ2 2 w
38
2
= √2 exp( −w ), w ≥ 0
2πλ 2λ2
⇒ W ∼ HN (0, λ).
If Y has a smallest extreme value distribution, Y ∼ SEV (θ, σ), then the pdf
of Y is
1 y−θ y−θ
f (y) = exp( ) exp[− exp( )]
σ σ σ
where y and θ are real and σ > 0. This distribution is a location scale family.
The cdf of Y is
y−θ
F (y) = 1 − exp[− exp( )].
σ
0.1
0.0
−4 −2 0 2
Figure 2.13. Plot of the pdf of the Smallest Extreme Value Distribution
⇒ W ∼ EXP (1).
39
If W ∼ EXP (1), let Y = σ log W + θ. Then
y−θ
FY (y) = P (Y ≤ y) = P (σ log W + θ ≤ y) = P (log W ≤ σ
) = P (W ≤ exp( y−θ
σ
)) =
2 y−θ 1 y−θ 2
f (y) = √ exp( ) exp(− [exp( )] ), y ∈ R, θ ∈ R, λ > 0
2πλ λ 2 λ
0.2
0.1
0.0
−4 −2 0 2
40
Let W = exp( Y λ−θ ),then Y = λ log(W ) + θ ⇒ g(w) = f (r(w))| dr(w)
dw
|
= √2
2πλ
exp[ λ log(w)+θ−θ
λ
] exp[− 12 (exp( λ log(w)+θ−θ
λ
))2]| wλ |
2 2
= √2 w exp( −w ) w1 = √2 exp( −w ), w ≥ 0
2π 2 2π 2
⇒ W ∼ HN (0, 1).
0.4
0.2
0.0
Figure 2.15. Plot of the pdf of the Truncated Extreme Value Distribution
= 1 − e−w/λ
⇒ W ∼ EXP (λ).
41
If W ∼ EXP (λ), let Y = log(1 + w). Then
FY (y) = P (Y ≤ y) = P (log(1 + W ) ≤ y) = P (1 + W ≤ ey ) = P (W ≤ eY − 1)
y
= 1 − exp(− e λ−1 ), y > 0
⇒ Y ∼ T EV (λ).
Hence
−n X eyi − 1
n
d log L
= + 2
:= 0
dλ λ i=1
λ
or
Pn y
i=1 (ei − 1)
λ̂ = .
n
Notice that
d2 log L n X 2(ey − 1)
n
n 2n n
i
| λ̂ = − = − λ̂ = [1 − 2] < 0.
λb3
dλ 2
λ̂2 i=1 λ̂3 λ̂2 λ̂2
42
2.16 THE HTEV DISTRIBUTION
y 2
If Y ∼ HT EV (λ), then f (y) = √ 2 ey
2πλ
exp( −(e2λ−1)
2 )I(y ≥ 0), λ > 0.
This distribution is unimodal and the mode at ym is found by solving the following
equation:
Proof:
2 (ey −1)2
log(f (y)) = log( √2πλ )+y− 2λ2
d e y
So dy
log f (y) = 0 + 1 − (2(ey − 1)( 2λ 2 ) := 0
or
2ey (ey −1)
1= 2λ2
⇒ λ2 = ey (ey − 1).
0.4
0.2
0.0
43
⇒ Y ∼ HT EV (λ).
⇒ W ∼ HN (0, λ).
Hence
Pn
−n X −2(eyi − 1)2
n yi
d log L −n i=1 (e − 1)2
= − 3
= + := 0
dλ λ i=1
2λ λ λ3
or
Pn yi
2 i=1 (e − 1)2
λ = ,
n
or r Pn
yi − 1)2
i=1 (e
λ̂ = .
n
Notice that
Pn
d2 log L n i=1 3(eyi − 1)2 n
| λ̂ = − = [1 − 3] < 0.
dλ2 λ̂2 λ̂4 λ̂2
Hence λ̂ is the MLE of λ.
44
Z ∼ N(0, 1), then
q Pn
yi
i=1 (e −1)
2
2
Pn yi 2
Pn 2
Since λ̂ = n
, we have n λ̂ = i=1 (e − 1) = i=1 wi
Pn
= λ2 i=1 x2i
nλ̂2
⇒ λ2
∼ χ2n .
nλ̂2 nλ̂2
( , )
χ2n,1− α χ2n, α
2 2
or
Pn Yi
Pn
i=1 (e − 1)2 Yi
i=1 (e − 1)2
( , ). (2.26)
χ2n,1− α χ2n, α
2 2
0.4
0.2
0.0
0 1 2 3 4 5
45
Let W = Y φ , then
1 1
FW (w) = P (W ≤ w) = P (Y φ ≤ w) = P (Y ≤ w φ ) = FY (w φ )
1
= 1 − exp(−(w φ )φ /λ) = 1 − exp( −w
λ
)
⇒ W ∼ EXP (λ).
1
If W ∼ EXP (λ), let Y = W φ . Then
1 −y φ
FY (y) = P (Y ≤ y) = P (W φ ≤ y) = P (W ≤ y φ ) = FW (y φ ) = 1 − e λ
−y φ
⇒ f (y) = φλ y φ−1 exp λ
⇒ Y ∼ W (φ, λ).
X 1 , ..., X n are iid k × 1 random vectors with E(X) = µ and Cov(X) = Σ, then
√ D
n(X − µ) → Nk (0, Σ)
then
√ D
n(g(T n ) − g(θ)) → Nd (0, Dg (θ) ΣDTg (θ) )
θ ∈ <k .
46
If Y1 , ..., Yn are iid Weibull (φ, λ), then the MLE (φ̂, λ̂) must be found before
X
n
1X φ
log(L(φ, λ)) = n log(φ) − n log(λ) + (φ − 1) log(yi ) − yi .
i=1
λ
Hence
P φ
∂ −n y set
log(L(φ, λ)) = + 2 i = 0,
∂λ λ λ
P
or yiφ = nλ, or
P
yiφ̂
λ̂ = .
n
Notice that
n X 1X φ
n
∂ set
log(L(φ, λ)) = + log(yi ) − yi log(yi ) = 0,
∂φ φ i=1 λ
so
X
n
1X φ
n + φ[ log(yi ) − yi log(yi )] = 0,
i=1
λ
or
n
φ̂ = P Pn .
1
λ̂
yiφ̂ log(yi) − i=1 log(yi )
and
n
φ̂k = P Pn .
1 φ̂
λ̂k
yi k−1 log(yi ) − i=1 log(yi )
47
Since W = log(Y ) ∼ SEV (θ = log(λ1/φ), σ = 1/φ), Olive [14] gave the following
and
Then φ̂0 = 1/σ̂R and λ̂0 = exp(θ̂R/σ̂R ). The iteration might be run until both
|φ̂k − φ̂k−1 | < 10−6 and |λ̂k − λ̂k−1 | < 10−6 . Then take (φ̂, λ̂) = (φ̂k , λ̂k ). If µ = λ1/φ
so µφ = λ, and Y ∼ W eibull(φ, µ) then the Weibull pdf
φ−1 " #
φ
φ y y
f (y) = exp − .
µ µ µ
Let (µ̂, φ̂) be the MLE of (µ, φ). According to Bain (1978, p. 215) [2],
! ! ! ! 2
! !
√ µ̂ µ D 0 1.109 µφ2 0.257µ
n − →N , .
φ̂ φ 0 0.257µ 0.608φ2
So
∂ φ ∂ φ
∂ ∂
g (θ)
∂θ1 1
g (θ)
∂θ2 1 ∂µ
µ ∂φ
µ φµφ−1 µφ log(µ)
Dg (θ) = = = .
∂ ∂ ∂ ∂
g (θ)
∂θ1 2
g (θ)
∂θ2 2 ∂µ
φ ∂φ
φ 0 1
where
Σ = I −1 (η) = [I(g(θ))]−1 = D g (θ) I −1 (θ)DTg (θ)
48
2
φµφ−1 µφ log µ 1.109 µφ2 .257µ φµφ−1 0
=
0 1 .257µ .608φ2 µφ log µ 1
φ+1
φµφ−1 µφ log µ 1.109 µ φ + .257µφ+1 log µ .257µ
=
0 1 .257φµφ + .608φ2 µφ log µ .608φ2
1.109µ2φ + .514φµ2φ log µ + .608φ2 µ2φ log2 µ .257φµφ + .608φ2 µφ log µ
=
.257φµφ + .608φ2 µφ log µ .608φ2
2
1.109λ2 + .514φλ2 logφ λ + .608φ2 λ2 logφ2 λ .257φλ + .608φλ log λ
=
.257φλ + .608φλ log λ .608φ2
1.109λ2 (1 + .4635 log λ + .5482(log λ)2 ) .257φλ + .608φλ log λ
= .
.257φλ + .608φλ log λ .608φ2
Similarly,
√ D
n(λ̂ − λ) −→ N (0, 1.109λ2 (1 + .4635 log λ + .5482(log λ)2 )),
49
In simulations, for small n the number of iterations for the MLE to converge could
be in the thousands, and the coverage of the large sample CIs is poor for n < 50.
0.4
0.2
0.0
⇒ Y ∼ HW (φ, λ).
1
Let W = Y φ , then Y = r(W ) = W φ
50
2
= √2 exp( −w ) I[w ≥ 0]
2πλ 2λ2
⇒ W ∼ HN (0, λ).
Qn Qn Pn −yi
2φ
L(λ) = i=1
2
f (yi ) = ( √2πλ ) n φn i=1 yiφ−1 exp( i=1 2λ2 ),
Hence
Pn
−n X −4λyi2φ
n 2φ
d log L −n i=1 yi
= − 4
= + := 0
dλ λ i=1
4λ λ λ3
or s
Pn 2φ Pn
2 i=1 yi i=1 yi2φ
λ = , orλ̂ = .
n n
Notice that
Pn 2φ
d2 log L n i=1 3yi n
2
|λ̂ = − = [1 − 3] < 0.
dλ λ̂2 λ̂4 λ̂2
51
CHAPTER 3
The following tables show the results obtained from a simulation study de-
signed to establish the actual coverage of confidence intervals for µ and σ 2 with
nominal 90% and 95% confidence levels respectively and sample sizes ranging from
5 to 500. Two types of confidence intervals for µ are used; the Pewsey interval is
given by
α 1 1 α 1 1
(Y(1) + σ̂ log( )Φ−1 + , Y(1) + σ̂ log(1 − )Φ−1 + ) (3.1)
2 2 2n 2 2 2n
and our new modified confidence interval
−1 1 1 13
(Y(1) + σ̂ log(α)Φ + 1 + 2 , Y(1)). (3.2)
2 2n n
The confidence interval used for σ 2 when µ is known is given by
Tn Tn
( 2
, ) (3.3)
χn,1− α χ2n, α
2 2
Pn 2
where Tn = i=1 (Yi − µ) .
and when µ is unknown is given by
Dn Dn
( 2
, 2
) (3.4)
χn−1,1− α χn−1, α
2 2
Pn
where Dn = i=1 (Yi − Y(1) )2 Each quoted coverage percentage was obtained from
Pewsey interval and the modified Pewsey interval for µ we notice that the modified
Pewsey interval has higher coverage for small sample sizes (n < 10) and similar
coverage to that of Pewsey interval for other sample sizes, also, we note that it has
52
Table 3.1. Actual Coverage Levels for Nominal 90% Confidence
Interval for σ 2 when µ is unknown for sample sizes ranging from
5 to 500
53
Table 3.3. Actual Coverage Levels for Nominal 90% Confidence
Interval for µ for sample sizes ranging from 5 to 500 - Modified
and Pewsey intervals
modified Pewsey
n d = (n-1) Coverage Slen Coverage Slen
5 4 0.9158 3.40 0.8410 2.86
10 9 0.8948 2.88 0.8694 3.26
25 24 0.8930 2.80 0.9000 3.51
50 49 0.8920 2.83 0.8898 3.60
100 99 0.9000 2.85 0.9026 3.64
500 499 0.9020 2.88 0.9052 3.68
∞ ∞ .9 2.89 .9 3.69
54
Table 3.5. Actual Coverage Levels for Nominal 95% Confidence
Interval for σ 2 when µ is known for sample sizes ranging from 5
to 500
modified Pewsey
n d = (n-1) Coverage Slen Coverage Slen
5 4 0.9474 4.44 0.9004 3.57
10 9 0.9404 3.75 0.9320 4.03
25 24 0.9438 3.64 0.9438 4.36
50 49 0.9478 3.69 0.9480 4.48
100 99 0.9482 3.71 0.9496 4.53
500 499 0.9456 3.74 0.9474 4.58
∞ ∞ .95 3.76 .95 4.59
55
3.2 HPARETO DISTRIBUTION
The following tables show the results obtained from a simulation study de-
signed to establish the actual coverage of confidence intervals for θ and λ2 with
nominal 95% confidence level and sample sizes ranging from 5 to 500. Each quoted
coverage percentage was obtained from 5000 pseudo-random samples of size n from
the (standard) half-normal distribution. The standard error of any entry is thus, at
most, 0.004.
56
Table 3.8. Actual Coverage Levels for Nominal 95% Confidence
Interval for λ2 when θ is unknown for sample sizes ranging from
5 to 500
57
3.3 HPOWER DISTRIBUTION
The following tables show the results obtained from a simulation study de-
signed to establish the actual coverage of confidence intervals for λ2 with nominal
95% confidence level and sample sizes ranging from 5 to 500. Each quoted coverage
percentage was obtained from 5000 pseudo-random samples of size n from the (stan-
dard) half-normal distribution. The standard error of any entry is thus, at most,
0.004. The confidence interval formula used is (2.18).
58
3.4 HTEV DISTRIBUTION
The following tables show the results obtained from a simulation study de-
signed to establish the actual coverage of confidence intervals for λ2 with nominal
95% confidence level and sample sizes ranging from 5 to 500. Each quoted coverage
percentage was obtained from 5000 pseudo-random samples of size n from the (stan-
dard) half-normal distribution. The standard error of any entry is thus, at most,
0.004. The confidence interval formula used is (2.26).
59
3.5 TWO PARAMETER EXPONENTIAL DISTRIBUTION
The following tables show the results obtained from a simulation study de-
signed to establish the actual coverage of confidence intervals for θ and λ2 with
nominal 95% confidence level and sample sizes ranging from 5 to 500. Each quoted
coverage percentage was obtained from 5000 pseudo-random samples of size n from
the exponential distribution. The standard error of any entry is thus, at most, 0.004.
The confidence interval formula used is (2.2).
60
Table 3.13. Actual Coverage Levels for Nominal 95% Confidence
Interval for λ when θ is known for sample sizes ranging from 5
to 500
n d = 2n Coverage Slen
5 4 0.9536 5.74
10 9 0.9480 4.72
25 24 0.9518 4.24
50 49 0.9464 4.07
100 99 0.9482 4.00
500 499 0.9496 3.93
∞ ∞ .95 3.92
61
3.6 THE PARETO DISTRIBUTION
The following tables show the results obtained from a simulation study de-
signed to establish the actual coverage of confidence intervals for σ and λ with
nominal 95% confidence level and sample sizes ranging from 5 to 500. Each quoted
coverage percentage was obtained from 5000 pseudo-random samples of size n from
the pareto distribution. The standard error of any entry is thus, at most, 0.004.
The confidence interval formula used is (2.12).
62
Table 3.16. Actual Coverage Levels for Nominal 95% Confidence
Interval for σ for sample sizes ranging from 5 to 500
63
3.7 THE POWER DISTRIBUTION
The following tables show the results obtained from a simulation study de-
signed to establish the actual coverage of confidence intervals for λ with nominal
95% confidence level and sample sizes ranging from 5 to 500. Each quoted coverage
percentage was obtained from 5000 pseudo-random samples of size n from the Power
n d = 2n Coverage Slen
5 4 0.9464 5.80
10 9 0.9540 4.75
25 24 0.9506 4.23
50 49 0.9500 4.06
100 99 0.9458 4.00
500 499 0.9442 3.93
∞ ∞ .95 3.92
64
3.8 THE TRUNCATED EXTREME VALUE DISTRIBUTION
The following tables show the results obtained from a simulation study de-
signed to establish the actual coverage of confidence intervals for λ with nominal
95% confidence level and sample sizes ranging from 5 to 500. Each quoted coverage
percentage was obtained from 5000 pseudo-random samples of size n from the trun-
cated extreme value distribution. The standard error of any entry is thus, at most,
0.004. The confidence interval formula used is (2.25).
n d = 2n Coverage Slen
5 4 0.9454 5.83
10 9 0.9474 4.73
25 24 0.9526 4.22
50 49 0.9574 4.08
100 99 0.9552 4.00
500 499 0.9496 3.93
∞ ∞ .95 3.92
65
3.9 THE WEIBULL DISTRIBUTION
The following tables show the results obtained from a simulation study de-
signed to establish the actual coverage of confidence intervals for φ and λ with
nominal 95% confidence level and sample sizes ranging from 25 to 500. Each quoted
coverage percentage was obtained from 100 pseudo-random samples of size n from
the Weibull distribution. The standard error of any entry is thus, at most, 0.043.
The confidence interval formulas used are (2.27) and (2.28).
φ=1 λ=1
n Coverage Slen Coverage Slen
25 .95 3.29 .94 4.35
50 .91 3.12 .94 4.23
100 .94 3.05 .92 4.18
500 .96 3.07 .94 4.15
∞ .95 3.06 .95 4.13
66
Table 3.20. Actual Coverage Levels for Nominal 95% Confidence
Interval for φ and λ for sample sizes ranging from 25 to 500
φ=1 λ=5
n Coverage Slen Coverage Slen
25 .95 3.23 .93 48.56
50 .92 3.16 .94 43.44
100 .93 3.07 .90 38.99
500 .94 3.06 .95 37.65
∞ .95 3.06 .95 36.73
φ=1 λ = 10
n Coverage Slen Coverage Slen
25 .95 3.22 .97 133.83
50 .97 3.16 .97 115.44
100 .95 3.12 .96 107.08
500 .94 3.07 .96 94.45
∞ .95 3.06 .95 92.07
67
Table 3.22. Actual Coverage Levels for Nominal 95% Confidence
Interval for φ and λ for sample sizes ranging from 25 to 500
φ=1 λ = 20
n Coverage Slen Coverage Slen
25 .94 3.05 .91 332.56
50 .96 3.12 .92 273.23
100 .95 3.14 .95 270.37
500 .97 3.06 .96 231.96
∞ .95 3.06 .95 223.20
φ = 20 λ=1
n Coverage Slen Coverage Slen
25 .98 63.63 .95 4.14
50 .90 62.93 .91 4.23
100 .94 62.20 .93 4.22
500 .96 61.51 .98 4.16
∞ .95 61.13 .95 4.13
68
Table 3.24. Actual Coverage Levels for Nominal 95% Confidence
Interval for φ and λ for sample sizes ranging from 25 to 500
φ = 20 λ=5
n Coverage Slen Coverage Slen
25 .99 65.17 .97 47.18
50 .97 62.56 .94 43.34
100 .93 62.69 .94 41.11
500 .98 61.53 1.00 37.80
∞ .95 61.13 .95 36.73
φ = 20 λ = 10
n Coverage Slen Coverage Slen
25 .94 61.78 .92 135.83
50 .94 61.75 .91 106.16
100 .97 61.34 .97 97.01
500 .96 61.15 .95 94.72
∞ .95 61.13 .95 92.07
69
Table 3.26. Actual Coverage Levels for Nominal 95% Confidence
Interval for φ and λ for sample sizes ranging from 25 to 500
φ = 20 λ = 20
n Coverage Slen Coverage Slen
50 .92 61.61 .91 261.31
100 .94 61.56 .95 248.74
500 .94 61.51 .95 237.89
∞ .95 61.13 .95 223.20
70
3.10 THE RAYLEIGH DISTRIBUTION
The following tables show the results obtained from a simulation study de-
signed to establish the actual coverage of confidence intervals for µ and σ with
nominal 95% confidence level and sample sizes ranging from 25 to 500. Each quoted
coverage percentage was obtained from 100 pseudo-random samples of size n from
the Rayleigh distribution. The standard error of any entry is thus, at most, 0.043.
The confidence interval formulas used are (2.22) and (2.23).
µ=1 σ=1
n Coverage Slen Coverage Slen
25 .91 2.84 .94 2.81
50 .98 2.35 .96 2.47
100 .93 2.22 .93 3.04
500 .92 2.32 .94 2.24
µ=2 σ=5
n Coverage Slen Coverage Slen
25 .92 13.58 .92 11.82
50 .93 12.06 .94 12.24
100 .92 12.06 .96 12.42
500 .94 9.99 .94 12.97
71
Table 3.29. Actual Coverage Levels for Nominal 95% Confidence
Interval for µ and σ for sample sizes ranging from 25 to 500
µ=2 σ = 10
n Coverage Slen Coverage Slen
25 .91 29.29 .94 26.71
50 .93 22.78 .94 24.74
100 .92 22.31 .96 23.16
500 .93 23.31 .95 23.63
µ=5 σ=2
n Coverage Slen Coverage Slen
25 .86 4.97 .99 24.82
50 .91 4.65 .95 4.46
100 .92 4.54 .92 5.25
500 .92 4.22 .96 4.65
72
Table 3.31. Actual Coverage Levels for Nominal 95% Confidence
Interval for µ and σ for sample sizes ranging from 25 to 500
µ = 10 σ=2
n Coverage Slen Coverage Slen
25 .92 5.43 .96 5.22
50 .92 4.30 .93 4.91
100 .95 4.89 .95 4.79
500 .92 4.21 .97 4.43
µ = 20 σ = 20
n Coverage Slen Coverage Slen
25 .92 57.34 .93 49.13
50 .87 44.79 .94 50.50
100 .93 42.22 .96 46.10
500 .93 37.55 .96 40.07
73
The following tables show the results obtained from a simulation study de-
signed to find the sample means and standard deviations of the MLEs for µ and σ
using sample sizes ranging from 25 to 500. Each quoted mean and standard devi-
ation was obtained from 100 pseudo-random samples of size n from the Rayleigh
distribution. The MLEs were found by iteration using Newton’s method, and the
µ=1 σ=1
n µ̂ SD(µ̂) σ̂ SD(σ̂)
25 1.08 0.14 0.95 0.14
50 1.03 0.09 0.98 0.09
100 1.02 0.07 1.02 0.07
500 1.01 0.03 1.02 0.03
µ=2 σ=5
n µ̂ SD(µ̂) σ̂ SD(σ̂)
25 2.34 0.69 4.69 0.60
50 2.23 0.43 4.76 0.44
100 2.14 0.31 4.90 0.32
500 2.02 0.11 5.00 0.15
74
Table 3.35. Sample means and standard deviations of the MLEs
for µ and σ using sample sizes ranging from 25 to 500
µ=2 σ = 10
n µ̂ SD(µ̂) σ̂ SD(σ̂)
25 3.01 1.49 9.27 1.36
¯ ¯
50 2.39 0.82 9.72 0.89
100 2.19 0.57 9.78 0.59
500 2.05 0.27 10.00 0.27
µ=5 σ=2
n µ̂ SD(µ̂) σ̂ SD(σ̂)
25 5.19 0.25 1.83 0.24
50 5.03 0.17 1.96 0.16
100 5.04 0.12 1.97 0.13
500 5.02 0.05 1.99 0.05
75
Table 3.37. Sample means and standard deviations of the MLEs
for µ and σ using sample sizes ranging from 25 to 500
µ = 10 σ=2
n µ̂ SD(µ̂) σ̂ SD(σ̂)
25 10.10 0.27 1.94 0.27
50 10.07 0.16 1.93 0.18
100 10.03 0.13 1.99 0.12
500 10.02 0.05 1.99 0.05
µ = 20 σ = 20
n µ̂ SD(µ̂) σ̂ SD(σ̂)
25 21.57 2.93 19.13 2.51
50 20.93 1.82 19.27 1.82
100 20.44 1.07 19.68 1.18
500 20.13 0.43 19.94 0.46
76
REFERENCES
Grove, CA.
[5] Cohen, A. C. (1988), Parameter Estimation in Reliability and Life Span Models,
Marcel Dekker, NY.
[6] Evans, M. (2000), Statistical Distributions, John Wiley and Sons, NY.
[7] Johnson, N.L., and Kotz, S. (1970), Continuous Univariate Distributions, vol.
1, Houghton Mifflin Company, Boston, MA.
[9] Lehmann, E. L. (1983), Theory of Point Estimation, John Wiley and Sons, NY.
[10] Mann, N.R., Schafer, R. E., and Singpurwalla, N.D. (1974), Methods for Sta-
tistical Analysis of Reliability and Life Data, John Wiley and Sons, NY.
[11] Meeker, W. Q., Escobar, L. A. (1998), Statistical Methods for Reliability Data,
(http://www.math.siu.edu/olive/infbook.htm).
[13] Olive, D.J. (2007), “A Simple Limit Theorem for Exponential Families,” avail-
[14] Olive, D.J. (2007), “Robust Estimators for Transformed Location Scale Fami-
77
lies,” available from (http://www.math.siu.edu/olive).
[15] Pewsey, A. (2002), “Large-Sample Inference for the General Half-Normal Dis-
78
APPENDIX
R CODE FOR THE SIMULATIONS, the Statistical package is
{ scov <- 0
ccov <- 0
mcov <- 0
d <- 2 * (n - p)
d2 <- 2 * n
for(i in 1:nruns) {
y <- theta + lambda * rexp(n)
79
#get CI for lambda
num <- 2 * dn
slow[i] <- num/qchisq(lcut, df = d)
if(slow[i] < lambda && sup[i] > lambda) scov <- scov + 1
#get CI for lambda when theta is known
80
#simulates Pewsey HN 100(1-alpha)% CI for sigma^2 and one for mu,
hnsim<-
function(n = 10, nruns = 5000, mu = 0, sigma = 1, alpha = 0.05, p = 1)
scov <- 0
lcov <- 0
lcov2 <- 0
ccov <- 0
d <- n - p
for(i in 1:nruns) {
81
miny <- min(y)
82
slen <- sqrt(n) * mean(sup - slow)
83
#simulates HPareto 100(1-alpha)% CI for sigma^2 and one for mu,
"hparsim1"<-
scov <- 0
lcov <- 0
ccov <- 0
for(i in 1:nruns) {
minw<- min(w)
miny <- min(y)
84
#get CI for sigma^2
#get CI for mu
85
#Simulates HPower 100(1-alpha)% CI for sigma^2,
{ cov <- 0
low <- 1:nruns
up <- low
sigsq <-sigma^2
for(i in 1:nruns){
y<- exp(-w)
miny<-min(y)
wn<- sum((log(y))^2)
86
#Simulates HTEV 100(1-alpha)% CI for sigma^2,
{
cov <- 0
up <- low
ucut <- alpha/2
for(i in 1:nruns){
wn<- sum((exp(y)-1)^2)
87
#simulates Pareto 100(1-alpha)% CI for lambda and CI for theta,
= 1)
{ scov <- 0
ccov <- 0
mcov <- 0
slow <- 1:nruns
d2 <- 2 * n
lval <- log(alpha/2)/n
for(i in 1:nruns) {
88
num <- 2 * dn
89
#Simulates POW 100(1-alpha)% CI for lambda^2,
{ theta = 0
cov <- 0
up <- low
ucut <- alpha/2
for(i in 1:nruns){
wn<- 2 * sum(log(1/y))
90
#simulates MLEs and CIs for mu and sigma in the Rayleigh distribution
{ countm <- 0
counts <- 0
count2s <- 0
mnew <- 0
snew <- 0
muold <- 0
sigmaold <- 0
muvold <- 0
sigmavold <- 0
scov <- 0
mlow <- 0
mup <- 0
slow <- 0
sup <- 0
for(i in 1:runs) {
91
y <- sqrt(w) + mu
for(j in 1:iter) {
D <- -2*n/sigmaold^2*sum((y-muold)^-2)+3/sigmaold^4*sum((y-muold)^-2)
*sum((y-muold)^2)- 2*n^2/sigmaold^4+3*n/sigmaold^6*sum((y-muold)^2)
-4/sigmaold^6*(sum(y-muold))^2
a1 <- -2*n/sigmaold^2*sum((y-muold)^-1)+2*n/sigmaold^4*sum(y-muold)
+3*sum((y-muold)^2)* sum((y-muold)^-1)/sigmaold^4-3*sum((y-muold)^2)
*sum(y-muold)/sigmaold^6- 4*n*sum(y-muold)/sigmaold^4+2*sum(y-muold)
*sum((y-muold)^2)/sigmaold^6
a2 <- - 2/sigmaold^3*sum(y-muold)*sum((y-muold)^-1)+2/sigmaold^5
*(sum(y-muold))^2+ 2*n^2/sigmaold^3-n*sum((y-muold)^2)/sigmaold^5
+2*n/sigmaold*sum((y-muold)^-2)- 1/sigmaold^3*sum((y-muold)^-2)
*sum((y-muold)^2)
snew <- sigmaold - (a2/D)
92
if(snew > 10*sigma){snew <- sigmaold}
}
muo[i] <- muvold
#get CI for mu
mlow[i] <- munew[i] - 1.96 * sqrt(Sig11)
}
mcov <- mcov/runs
93
slens <- sqrt(n) * mean(sup - slow)
94
#Simulates TEV 100(1-alpha) CI for lambda^2,
{ theta = 0
cov <- 0
up <- low
ucut <- alpha/2
for(i in 1:nruns){
wn<- 2 * sum(exp(y)-1)
95
# Simulates Weibull 100(1-alpha) CI for Phi and lambda
"weibsim"<-
lnew <- 0
pnew <- 0
lamold <- 0
lamvold <- 0
phiold <- 0
phivold <- 0
pcov <- 0
lcov <- 0
pcov2 <- 0
lcov2 <- 0
plow <- 1:runs
AssSDphi <- 0
AssSDlam <- 0
for(i in 1:runs)
96
{
for(j in 1:iter)
{
}
phio[i] <- phivold
97
plow[i] <- phinew[i] - 1.96 * .7797 * phinew[i]/sqrt(n)
if(plow[i] < phi && pup[i] > phi) {pcov <- pcov + 1}
#get CI for lambda
*(1+.4635*log(lamnew[i])+.5824*(log(lamnew[i]))^2))/sqrt(n)
lup [i] <- lamnew[i] + 1.96 * sqrt(1.109*lamnew[i]^2
*(1+.4635*log(lamnew[i])+.5824*(log(lamnew[i]))^2))/sqrt(n)
if(llow[i] < lam && lup[i] > lam) {lcov <- lcov + 1}
*(log(lam))^2)/n)
vec[4] <- sqrt(var(lamnew))
98
lamSDfromI=AsSDlam, phiconv = pconv, lamconv = lconv)
99
VITA
Graduate School
Southern Illinois University
100