Likelihood Ratio Test

LIKELIHOOD RATIO TEST
1|P a g e
TABLE OF CONTENTS
Likelihood ratio test (LRT) 3
Review 3
Assumptions 4
Definition …………………………………………………………………………………………………………………………………………………………....4
Purpose of Likelihood Ratio Test………………………………………………………………………………………………………………………....4
Likelihood Ratio Test for Simple Hypothesis..……………………………………………………………………………………………………..6
Generalized Likelihood ratio test 5

Periodicity distribution of the test statistic Premise 6
Decision Criteria 6
Critical Region .…………………………………………………………………………………………………………………………………………….…..6
Demerit…..………………………………………………………………………………………………………………………………………………………..7
GLRT for Simple Null Hypothesis……..………………………………………………………………………………………………………………..7
Large Sample Likelihood Ratio Test…………………………………………………………………………………………………………………..8
Example and Numerical……………………………………………………………………………….…………………………………………………….8
Example……………………………………………………………………………………………………………………………………………………………8
Example#1 ……………………………………………………………………………………………………………………………………………………..11
Example#2..…………………………………………………………………………………………………………………………………………………….14
Example#3 ……………………………………………………………………………………………………………………………………………………..20
References………………………………………………………………………………………………………………………………………………………..23
2|P a g e
LIKELIHOOD RATIO TEST (LRT)
REVIEW
It is the statistical investigation of the goodness of fit among two styles. In this check, a
genuinely additional perplexing model is being contrasted with a considerably less confounded
form that permits you to take conveyance of or reject the confine on the parameter. The
opportunity proportion check is legitimate assuming we will analyze progressively settled styles.
Thusly, a more prominent complex rendition need to contrasts from a substantially less look at
form with the guide of the expansion of one or additional parameters.
The (generalized) likelihood-ratio check (every so often called the probability ratio chi
rectangular take a look at) was proposed through J. Neyman and E.S. Pearson in 1928. They also
proved (1933) that of all degree- α tests for checking out one easy hypothesis in opposition to
another, the chance-ratio test is the most effective among all of the level-α checks. This is
likewise referred to as the generalized likelihood ratio test. LRT is one in all oldest classical
LIKELIHOOD
RATIO TEST
3|P a g e
approaches of hypothesis checking out. Likelihood ratio exams (LRTs) are as broadly used as
maximum probability estimation.
ASSUMPTIONS
To derive the asymptotic properties of the statistic LR, we anticipate that both of the restricted
and unrestricted estimators are asymptotically every day. Components of g are continuously
differentiable on Θ with appreciation to all of the components of θ.
DEFINITION
A LIKELIHOOD RATIO (LR) TEST IS A HYPOTHESIS TEST IN WHICH TWO
DIFFERENT MAXIMUM LIKELIHOOD ESTIMATES OF A PARAMETER ARE
COMPARED TO DECIDE WHETHER OR NOT TO REJECT A PARAMETER
RESTRICTION. ACCORDING TO STATISTICS, THE LIKELIHOOD RATIO TEST
ASSESSES THE GOODNESS-OF-FIT OF TWO COMPETING STATISTICAL
MODELS BASED ON THE RATIO OF THEIR LIKELIHOODS, NAMELY, ONE
FOUND BY MAXIMIZING THE ENTIRE PARAMETER SPACE AND ANOTHER
FOUND AFTER IMPOSING A PARTICULAR RESTRICTION.
PURPOSE OF THE LIKELIHOOD RATIO TEST

For example, let’s link diagnostics with a likelihood ratio test here the Likelihood ratios (LRs) are
used to assess two things:
i. the possible utility of a particular diagnostic test

ii. the likelihood that a patient has a disease or condition.
Likelihood ratios are essentially the ratio of either the possibility that the test outcome is
correct or the probability that the test result is incorrect.
4|P a g e
LIKELIHOOD RATIO TEST FOR SIMPLE HYPOTHESIS

Let X 1 , X 2 , X 3 ,… , X n be a random sample from a distribution with parameter θ . Suppose that the
observed values are X 1 =x1 , X 2=x 2 , … , X n=x n. The two simple hypothesis are
H 0 :θ=θ 0
H 1 :θ=θ1
we define
L ( x1, x2, … , xn ; θ0 )
λ ( x 1 , x2 , … , xn )=
L ( x 1 , x 2 , … , x n ; θ 1)
To execute a likelihood ratio test (LRT), we choose a constant c .
λ<c ⟹ Reject H 0
λ ≥ c ⟹ Accept H 0
The value of c can be chosen based on the desired α .
GENERALIZED LIKELIHOOD RATIO TEST:

Generalized likelihood ratio tests are used when the hypotheses are not straightforward. They are
not optimal in general, but they are usually not optimal in situations with no optimal test, and
they usually work reasonably well. The generalized likelihood ratio test is a general procedure
for compound test problems. The basic idea is to compare the best model in class H 1 with the
best in class H 0 , which is formalized as:
H 0 :θ ϵ Θ0
Against
Θ
H1: θ ϵ
Θ1
It can be used when H 0is composite.

¿
For a random sample x 1 , … . , x nwith the density f ( x ; θ ) ,θ ϵ θ ,as we seek a test
H 0 :θ ϵ Θ¿
H 1 :θ ϵ Θ1
5|P a g e
Let L(θ ; x 1 , … , x n) be the likelihood function for a sample x 1 , … , x n having joint density
f ( x 1 , … , x n ; θ )where θ ϵ θ¿. The generalized likelihood ratio, denoted by λ∨λ ndefined to be as
¿ L(θ ; x , … , x )
θ ϵ Θ ¿0 1 n
λ=λ n=λ ( x 1 , … , x n )= ¿
L(θ ; x 1 , … , x n )
θ ϵ Θ1
Can be expressed as
¿ L(θ ; x ,… , x )
¿ 1 n
θ ϵ Θ0
Λ=λ=λn =λ ( x 1 , … , x n )=
¿ L(θ ; x , … , x )
1 n
θ ϵ Θ1
Remember that λ is the function of x 1 , … , x n, namely λ ( x 1 , … , x n ). When the observations are

replaced by their corresponding random variables x 1 , … , x n ,then we write Λ forλ; that is
Λ=λ ( x 1 ,… , x n ). Λ is the function of random variables and itself a random variable. Λ is a
statistic since it does not depend upon unknown parameters.
PERIODICITY DISTRIBUTION OF THE TEST STATISTIC

PREMISE
If the null hypothesis is genuine and some technical situations are completely filled, then the
probability ratio check statistic converges to a chi-square distribution with okay degrees of
freedom.
Some notes that follow
 λ Necessarily satisfies 0 ≤ λ ≤1 ; λ ≥ 0.Since we have the nonnegative portions inside the

ratio and λ ≤ 1 then the supremum taken inside the denominator is over the most extensive
set of parameter values than within the numerator.
 The denominator gains an extra fee with admire to the numerator.
 The parameter θ might be a vector-valued.
 The denominator is the likelihood characteristic predicted at most chance estimator.
 In a generalized probability ratio, pattern can be a random sample. As sample x 1 … x n be a
pattern from density f ( x ; θ ) .
The values of Λ statistic used to formulate a check of the above cited speculation that if
6|P a g e
DECISION CRITERIA:
H 0is to be rejected if and only if λ ≤ λ 0.
Where λ 0is some fixed constant satisfying 0 ≤ λ ≤1 .
CRITICAL REGION:
The critical region of GLRT is defined as
R={ y : λ ( y ) ≤ a }
Where a is a constant value chosen to give significance level α , that is such that
P { λ ( y ) ≤ a|H 0 }=α
The level−α GLRT will reject H 0 when Λ ≤ c . c is chosen so that P H 0 [ A ≤ c ] equals (or
approximately equals) to α .
It makes good sense that our test statistic tends to be smaller if our null hypothesis H 0is false.
Since the denominator of λ tends to be larger than the numerator.
DEMERIT:
i. Sometimes, it becomes challenging to find the ¿ L(θ ; x 1 , … , x n)
ii. It can be challenging to find the distribution of Λ which is required to test the
power of the test.
GLRT FOR SIMPLE NULL HYPOTHESIS:

Let f (θ) be a parametric model, and let θ0 be a particular parameter value. As
iid
x1 , x2 , … , xn f (θ)
∼
H 0 :θ=θ 0
H 1 :θ ≠ θ0
The generalized likelihood ratio test statistic is
l(θ0 )
Λ=
max θ l (θ)
7|P a g e
n
Where ( θ ) = ∏ f (θ) is the likelihood function and the numerator is the value of the likelihood
(i=1)
function at θ0 . Numerator is called the restricted maximum likelihood estimate of θ under H 0,

where l(θ) is the likelihood function.
The denominator is the maximum value of the likelihood function from all the possible values of
^ where θ^ is the MLE.
θ in the parameter space. The denominator is equivalent to l( θ)
Note that where l (θ ) ≥ lik ( θ0 )because θ0 is the parameter space, so Λ ≤ 1.
We expect the l θ^ to be closer to lik θ0 under H 0 rather than under H 1.
Rejecting H 0 for small values of Λ is equivalent to reject H 0for large values of
LARGE SAMPLE LIKELIHOOD RATIO TEST

In large samples the test statistic
G=[ −2 log Λ=2 log θ−2

^ logθ 0 ]
Here, G is the difference between the -2log likelihoods. If we are doing MLE numerically, then
test statistic G is the difference between two minimum values.
Then we will use −2 log Λ as the test statistic for the GLRT. From the above observation that
Λ ≤ 1, we have −2 log Λ ≥ 0. If the dimensions of the parameter space is k , then the GLRT at
level α rejects H 0when
2
−2 log Λ ≥ χ ( α )
Hence χ 2 ( α ) is the upper α point of chi square distribution with k degrees of freedom.
For example, if the null hypothesis is that there are 5 elements of θ equals to zero then 4 is the
degrees of freedom.
EXAMPLE AND NUMEICAL

#Example
Let x 1 , x 2 , … , x n be a random sample f ( x ; θ )=θ e−θx Ι ( 0 ,θ )(x ) , WhereΘ= {θ ;θ> 0 } .Test

H ∘ :θ ≤ θ∘versus H 1 θ >θ∘.test the hypotheses by using the likelihood ratio.
8|P a g e
Solution:
HYPOTHESIS:
H ∘ :θ ≤ θ∘
H 1 :θ >θ∘.
LEVEL OF SIGNIFICANCE:
P (λ (Y) ≤ a | H 0) = α.
The level-α GLRT will reject H 0when λ ≤ λ ο. C is chosen so that P H 0 [ λ ≤ λ ο] equals (or
approximately equals) to α.
TEST STATISTIC OR LIKELIHOOD RATIO FUNCTION:

^
Λ=λ=l(θ° )/ l (θ)
COMPUTATIONS:
Then the pdf of exponential distribution will be given as: (A)
f ( x ; θ )=θ e−θx
The likelihood function will be as:

n
L ( x ; θ ) =∏ f (x ; θ)
i
n
L ( x ; θ ) =∏ θ e
−θxi
Applying product sum:
L ( x ; θ ) =θn e−θΣxi (B)
Log likelihood function will be as follows:
l (θ )=log ( θ n e−θΣxi )
l (θ )=nlog ( θ )−θΣxi (1)
Now take partial derivative w.r.t ‘θ' from equation 1:
9|P a g e
∂l ( θ ) n
= −Σxi ( 2 )
∂ (θ) θ
Put it equal to zero
∂l ( θ )
=0
∂ (θ)
We have,
n
−Σxi=0
θ^
n
= Σxi
θ^
n ^
=θ
Σxi
Or
^ 1
θ=
x
Again we will take the second partial derivative from equation2
∂2 l ( θ ) −n
2
= 2 <0
∂ (θ) θ
1
Hence is the mle of θ .
x
Now solving for numerator from equation A&B
l ( θ° )=θ° n e−θ Σxi

°
Then solve for denominator using equation B
^ 1n
l ( θ )= exp −
x [() ]
1
x
( Σxi )
Putting numerator and denominator into equation B we have
10 | P a g e
λ=θ° en −θ ° Σxi 1n
/ exp −
x
1
x [() ]
( Σxi )
λ=θ° e
n −θ ° Σxi −n
/ x exp [ ]
−Σxi
x
λ=θ° exp [ θ° n x ]/ x ∗exp −n

n −n
[ ] x
x
n
θ°
λ= −n
exp [ −θ° n x +n ]
x
θ °n
λ= exp (−n ) [ θ° x−1 ]
x−n
λ=¿
Let y=θ° x
λ =¿
DECISION RULE:
1 θ °n 1
λ ( x 1 , x 1 , … , xn )={1 ≤θ ο dont reject H 0 −n exp [−θ° n x +n ] > θο reject H 0
x x x
If 0< λο <1 ,then a generalized likelihood ratio test is given by the following: reject H ο if λ ≤ λ οor
1
Reject H ο if >θο and ¿
x
Then we can write this non-randomized test (LRT) by using the critical function
CRITICAL FUNCTION:
ψ ( x 1 , x 2 ,… , xn ) ={0 ¿
If and only if y ≤ k , where k is a constant satisfying 0< k <1.
We see that the generalized likelihood ratio test reduces to the following:
Reject H ο if and only ifθ° x <k , where 0<k < 1.
11 | P a g e
Example 1:
Derive the likelihood ratio test for H0: μ ≤ μ 0versus H1: μ¿μ0, when the population is
normal and population variance σ2 is known.
Solution:
HYPOTHESES:
H ° :μ≤μ0 where σ 2 is known
H 1 : μ> μ 0
LEVEL OF SIGNIFICANCE
P (λ (Y) ≤ a| H 0) = α.
The level-α GLRT will reject H 0when λ ≤ λ ο. c is chosen so that P H 0 [ λ ≤ λ ο] equals (or
TEST STATISTIC:
L (ϖ )
Λ=λ= ∨ Λ=L ( μ ° )/ L ( μ^ ) (A)
L (Ω)
ϖ ={ μ : μ=μ ο } and Ω={ μ :−∞< μ<+ ∞ }
COMPUTATION
The pdf of normal distribution will be as follows:
1
f ( x ; μ , σ )=
2
exp ¿
√ 2 π σ2
First derive the likelihood function of distribution
n
L ( x ; μ , σ ) =∏ f ( xi; μ , σ )
2 2
n
1
L ( x ; μ , σ )=∏
2
exp ¿ ¿
i √2 π σ 2
L ( x ; μ , σ 2 ) =¿
12 | P a g e
Log likelihood function will be as
−n n 1
l ( μ , σ )=
2 2
log σ − log (2 π )− 2 Σ ¿
2 2 2σ
−n n 1
l ( μ , σ )=
2 2
log σ − log (2 π )− ¿
2 2 2
Now the first partial derivative w.r.t μ
∂l ( μ , σ 2 ) 1
=0−0− 2 Σ 2 ¿
∂μ 2σ
∂l ( μ , σ 2 ) 1
= 2 Σ ¿= Σ xi−nμ/σ 2 (2)
∂μ σ
∂l ( μ , σ 2 )
=0
∂μ
Σ xi−n μ^
We have, => =0 => Σ xi−n ^μ=0=¿ Σxi=n μ^
σ2
^μ= Σ xi /n => ^μ= x
This is the candidate of MLE so we will go for second order condition from equation 1
∂2 l ( μ , σ 2 ) 1 −n
( 0−n ) = 2 < 0 hence X is the MLE of μ
2 = 2
∂ μ σ σ
Now solving for the numerator of equation A using 1

n
1
L ( μο )=∏ exp ¿ ¿
i √2π σ 2
Then solving for the denominator from equation A using 1

n
1
L ( μ^ )=∏ exp ¿ ¿
i √2π σ 2
Now we will replace the value that maximize the L ( Ω )∨L ( μ^ )
L ( μ^ )=¿
13 | P a g e
By putting values we have
L( ϖ ) =
λ= L ( μ° )/ L ( μ^ )= ¿/¿
^)
L( Ω
λ = L ( μ° )/ L ( μ^ ) =exp ¿* exp ¿
L ( μ° )/ L ( μ^ )=exp ¿
Consider Σ ¿
¿= ( xi−x ¿ ¿2 +¿( xi−x ¿ ¿2 ¿
Appling sum on the above equation
λ = Σ ( xi−x ¿ ¿2 + Σ ¿ ( xi−x ¿ ¿2 ¿
Sum of square deviation from its mean is zero
λ=n ¿
So equation 5 reduced into
λ =exp ¿ =exp ¿
DECISION RULE:
λ ( x )={1 x ≤ μ ο exp ¿
Note that by GLRT we will reject null the hypothesis if λ ( x ) <c where is some constant lie
between0< c<1
Then will use the critical function
ψ ( x 1 , x 2 ,… , xn ) ={0 exp ¿
Where λ οis a constant that lies between0< λο <1
Example 2:
Derive the likelihood ratio test by testing the hypothesis for H 0: μ¿ μ0versus H1: μ≠ μ0
, when x 1 , x 2 , … , x n be a random sample drawn from anormal distribution with mean
μand variance σ2is unknown.
Solution:
14 | P a g e
HYPOTHESIS:
H 0 :μ¿ μ0; σ2> 0
H 1 : μ ≠ μ0 ; σ2< 0
P (λ(Y) ≤ a | H 0) = α.
The level-α GLRT will reject H 0when λ ≤ λ ο. c is chosen so that P H 0 [ λ ≤ λ ο] equals (or
TEST STATISTIC:
L (ϖ )
Λ=λ= ∨ Λ=L ( μ ° )/ L ( μ^ )(A)
L (Ω)
ϖ ={ μ : μ=μ ο } and Ω={ μ :−∞< μ<+ ∞ }
COMPUTATION
The pdf of normal distribution will be as follows:
1
f ( x ; μ , σ )=
2
exp ¿
√ 2 π σ2
First derive the likelihood function of distribution
n
L ( x ; μ , σ ) =∏ f ( xi; μ , σ 2 )
2
n
1
L ( x ; μ , σ )=∏
2
exp ¿ ¿
i √2π σ2
L ( x ; μ , σ ) =¿
2
Log likelihood function will be as
−n n 1
l ( μ , σ )=
2 2
log σ − log (2 π )− 2 Σ ¿
2 2 2σ
15 | P a g e
−n n 1
l ( μ , σ 2 )= log σ 2− log (2 π )− ¿
2 2 2
Now take the first partial derivative w.r.t μ
∂l ( μ , σ )
2
1
=0−0− 2 Σ 2 ¿
∂μ 2σ
∂l ( μ , σ ) 1
2
= 2 Σ ¿= Σ xi−nμ/σ 2 (4)
∂μ σ
∂l ( μ , σ 2 )
=0
∂μ
Σ xi−n μ^
We have, => =0 => Σ xi−n ^μ=0=¿ Σxi=n μ^
σ2
^μ= Σ xi /n => ^μ= x
This is the candidate of MLE so we will go for second order condition from the equation
∂ l (μ,σ )
2 2
1
= ( 0−n )
∂2 μ σ2
n
¿− 2 <0 hence X is the MLE of μ
σ
Now taking the first partial derivative w.r.t σ 2
∂l ( μ , σ 2 ) −n 1
∂σ
2
=
2 σ 2 ( ) 1
( 1 )−0− Σ¿
2
−n
= + Σ ¿¿
2 σ2
−n
= 2+ Σ ¿ ¿
2σ
−1
= (n- Σ ¿ ¿)
2 σ2
16 | P a g e
∂l ( μ , σ )
2
2
=0
∂σ
−1
We have, => 2 (n- Σ ¿ ¿) =0
2σ
n- Σ ¿ ¿ =0
n =Σ ¿ ¿
σ^ =Σ¿ ¿
2
σ^2=Σ¿ ¿
This is the candidate of MLE, so we will go for second order condition from equation
∂2 l ( μ , σ 2 )
2 2 =−Σ ¿ ¿
∂σ
¿−Σ¿ ¿
n S2 n
¿− +
S
2
2¿¿
n
¿
¿¿
n
¿− <0
¿¿
hence Σ ¿ ¿ is the MLE of σ 2
Now solving for numerator of equation A using 1

n
1
L ( μ , σ 2 )=∏ exp ¿ ¿
i √2π σ2
¿¿
Then solving for denominator from equation A using 1

n
1
L ( ^μ , σ^ ) =∏
2
exp ¿¿
i
√ 2 π σ^
2
Now we will replace the value that maximize the L ( Ω )∨L ( μ^ )
17 | P a g e
L ( ^μ , σ^ ) =¿
2
By putting values we have
L( ϖ )
^ ) = L μ , σ / L ( ^μ , σ )
λ= ( 2) ^2
L( Ω
λ =¿/¿
λ = L ( μ , σ 2 )/ L ( ^μ , σ^2 )=¿ ¿
λ = L ( μ , σ 2 )/ L ( ^μ , σ^2 )=( Σ ¿ ¿¿ (a)
Consider Σ ¿
Σ ¿= Σ ( xi−x ¿ ¿2 +n ¿( xi−x ¿ ¿2 ¿
As we know that sum of square deviation from its mean is zero so,
Σ ¿= Σ ( xi−x ¿ ¿2 +n ¿
So equation (a) reduced into
λ =¿
=(1+n ¿ ¿ ¿
DECISION RULE:
Note that by GLRT we will reject null hypothesis if λ ( x ) <c where is some constant lie between
0< c<1 OR
As 0< Λ <1because Θo ⊂Θ , thus when Λ< c we would reject H ° ,where c < 1 is a constant.
So,
(1+ n ¿ ¿ ¿
(1+n ¿ ¿
n¿¿
n¿¿
n¿¿
18 | P a g e
n ¿ ¿)
¿ ¿)
Taking square root on both sides
√¿ ¿¿
|x−μ ο|
S
n
√
≥ (n−1)( c ¿ ¿
−n
2
−1)¿
√
t ≥ (n−1)(c ¿ ¿
−n
2
−1) ¿
=t α2 ,n−1 for the test with size α
Then will use critical function
√ (n−1)(c ¿ ¿
−n
2
t
−1) ¿ = α2 ,n−1
Taking square on both sides, we get
−n 2
(n−1)( c ¿ ¿ −1)¿= t α ,n−1
2 2
2
−n tα
c −1=
, n−1
2 2
n−1
−n t 2α
c = +1
, n−1
2 2
n−1
2
tα −2
c= ( 2
, n−1
+1¿ ¿ n
n−1
so we reject H ο if |t |≥t α2 ,n−1
Example 3:
Suppose that an engineer wishes to compare the number of the complaints per week
that is filled by union stewards for two different shifts at a manufacturing plant.
19 | P a g e
There are 100 independent observations on the number of the complaints gave mean
x=20 for the 1 shift and y=22 for 2 shift. Assume that the number of complaints per
week on the ith shift has a poisson distribution with mean θi for i=1,2. Use likelihood
ratio test method to test the hypothesis H o :θ1=θ 2 verse H 1 :θ 1 ≠ θ2 with significance
level ¿ 0.01 .
SOLUTION:
HYPOTHESIS:
H ° :θ=θO
H 1 :θ ≠ θO
P (λ(Y) ≤ a| H 0) = α=0.01.
The level-α GLRT will reject H 0when θ ≤ θ ο. c is chosen so that P H 0 [θ ≤ θ ο] equals (or
TEST STATISTIC:
L (ϖ)
Λ=λ= ∨ Λ=L ( θ° )/ L ( θ^ ) (1)
L (Ω)
COMPUTATION
The pdf of Poisson Distribution will be as follows:
e λ λx eλ λ y
f ( x : λ)= And f ( y : λ )=
x! y!
So the likelihood function for joint probability function of x i ' sand y i ' s is given as
n −θ 1 x n −θ2 y
e θ1 e θ2
L ( θ1 ,θ 2) =∏ ∏
i=1 x! i=1 y!
θ1 ∑ e θ2 ∑
−n θ1 xi −n θ2 yi
e
= n n
∏ x! ∏ y!
i=1 i=1
1 −n θ ∑ x −n θ ∑ y
= e θ1 e θ 21 i 2 i
20 | P a g e
n n
Where k=∏ x ! ∏ y !
i=1 i=1
For the numerator of equation (1), θ1=θ 2=θ whereθ is unknown so the likelihood function of a
single parameterθ is
1
L ( θ° ) = e−nθ θ∑ x e−nθ θ∑ y i i
1 −2 nθ ∑ x + ∑ y
= e θ i i
For the maximizing function of L (θ ), we can find

n n
∑ xi ∑y 1
i=1
+ i=1 = (x + y )
n n 2
^
θ=
2
Is the observed value of θ^ where x=20 and y=22 . by putting in it we get,
^ 1 (20+22) =21
θ=
2
Fro For the denominator of equation (1), θ1 ≠θ 2 and also θ1 >0 , θ2 >0 . So the MLE of L ( θ1 ,θ 2) is
maximized when θ^ 1= xand θ^ 2= y . So the denominator becomes
1 −n θ^
L ( θ^ ) = e θ^ 1∑ e θ^ 2∑
x −n ^θ
1 y i 2 i
1
−n x ∑ x −n y ∑ y
= e x e y i i
1
−n (x+ y) n x n y
= e x y
k
So equation (1) becomes
1 −2 n θ^ ^ ∑ x +∑ y
e θ i i
^ k
Λ=L ( θ ° )/ L ( θ ) =
1 −n (x+ y) n x n y
e x y
k
^
( θ)
n x+n y
=
( x ¿) ( y ¿)n y ¿ ¿
nx
21 | P a g e
^
By putting the values of x=20 , y=22 , n=100 and θ=21, the observed value of Λ becomes
(21)100(20 +22)
Λ=
(20)100(20 )(22)100(22)
And by applying theorem -2log Λ , we get
-2log Λ=−2(4200 log 20−2000 log 20−2200 log 22)
=9.53
In this r=2 and r o =1 , as the approximate distribution of -2log Λ is a χ 2 with degree of freedom
v= r-r o =1.
DECISION RULE:
we reject H 0 if| χ 2|≥ χ 21 (1−α )
so,
9.53 ≥ 6.635
CONCLUSION:
So we reject null hypothesis conclude that the number of complaints filed by the union
stewards do differ at the significance level of α =0.01
REFERENCES
https://sites.warnercnr.colostate.edu/gwhite/wp-content/uploads/sites/73/2017/04/LikelihoodRatioTests.pdf
https://www.statlect.com/fundamentals-of-statistics/likelihood-ratio-test
http://fisher.stats.uwo.ca/faculty/kulperger/SS3858/Handouts/ExponentialLikelihoodRatio.pdf
http://www.utstat.toronto.edu/burkett/sta442f15/STA442_7_LRTs.pdf
https://pdfs.semanticscholar.org/a7e8/2b5ddfd6100cf9d3ab4c1e79cfb0843f1755.pdf
22 | P a g e
http://people.missouristate.edu/songfengzheng/Teaching/MTH541/Lecture%20notes/LRT.pdf
https://www.math.drexel.edu/~tolya/test%20for%20mean%20of%20a%20normal%20distribution.pdf
https://www.youtube.com/watch?v=VE6hjm5IUIo
https://www.youtube.com/watch?v=gUoM_6B-TVY
https://www.youtube.com/watch?v=xRVzo3J-IBw
https://www.youtube.com/watch?v=fVXA9CK-Nb0
https://www.youtube.com/watch?v=9RvLy554NnI
https://www.youtube.com/watch?v=yzO80fa0_Y4
http://www.stat.uchicago.edu/~yibi/teaching/stat226/lectures/C01.pdf
https://web.stanford.edu/class/archive/stats/stats200/stats200.1172/Lecture22.pdf
http://people.missouristate.edu/songfengzheng/Teaching/MTH541/Lecture%20notes/LRT.pdf
https://people.stat.sc.edu/habing/courses/703/GLRTExample.pdf
https://webspace.maths.qmul.ac.uk/b.bogacka/MS_Lectures_23and24.pdf
https://cpb-us-w2.wpmucdn.com/voices.uchicago.edu/dist/9/1193/files/2016/01/05a-CompositeHTests.pdf
http://people.musc.edu/~bandyopd/bmtry711.11/lecture_02.pdf
23 | P a g e

Likelihood Ratio Test

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Likelihood Ratio Test

Uploaded by

Copyright:

Available Formats

LIKELIHOOD RATIO TEST

Likelihood ratio test (LRT) 3

Likelihood Ratio Test for Simple Hypothesis..……………………………………………………………………………………………………..6

Generalized Likelihood ratio test 5

GLRT for Simple Null Hypothesis……..………………………………………………………………………………………………………………..7

Large Sample Likelihood Ratio Test…………………………………………………………………………………………………………………..8

Example and Numerical……………………………………………………………………………….…………………………………………………….8

LIKELIHOOD RATIO TEST (LRT)

PURPOSE OF THE LIKELIHOOD RATIO TEST

i. the possible utility of a particular diagnostic test

LIKELIHOOD RATIO TEST FOR SIMPLE HYPOTHESIS

To execute a likelihood ratio test (LRT), we choose a constant c .

The value of c can be chosen based on the desired α .

GENERALIZED LIKELIHOOD RATIO TEST:

It can be used when H 0is composite.

Remember that λ is the function of x 1 , … , x n, namely λ ( x 1 , … , x n ). When the observations are

PERIODICITY DISTRIBUTION OF THE TEST STATISTIC

Some notes that follow

 λ Necessarily satisfies 0 ≤ λ ≤1 ; λ ≥ 0.Since we have the nonnegative portions inside the

Where λ 0is some fixed constant satisfying 0 ≤ λ ≤1 .

GLRT FOR SIMPLE NULL HYPOTHESIS:

The generalized likelihood ratio test statistic is

function at θ0 . Numerator is called the restricted maximum likelihood estimate of θ under H 0,

Note that where l (θ ) ≥ lik ( θ0 )because θ0 is the parameter space, so Λ ≤ 1.

We expect the l θ^ to be closer to lik θ0 under H 0 rather than under H 1.

Rejecting H 0 for small values of Λ is equivalent to reject H 0for large values of

LARGE SAMPLE LIKELIHOOD RATIO TEST

G=[ −2 log Λ=2 log θ−2

EXAMPLE AND NUMEICAL

Let x 1 , x 2 , … , x n be a random sample f ( x ; θ )=θ e−θx Ι ( 0 ,θ )(x ) , WhereΘ= {θ ;θ> 0 } .Test

TEST STATISTIC OR LIKELIHOOD RATIO FUNCTION:

The likelihood function will be as:

Applying product sum:

L ( x ; θ ) =θn e−θΣxi (B)

Log likelihood function will be as follows:

l (θ )=nlog ( θ )−θΣxi (1)

Now take partial derivative w.r.t ‘θ' from equation 1:

Put it equal to zero

Again we will take the second partial derivative from equation2

Now solving for numerator from equation A&B

l ( θ° )=θ° n e−θ Σxi

Then solve for denominator using equation B

Putting numerator and denominator into equation B we have

λ=θ° exp [ θ° n x ]/ x ∗exp −n

If and only if y ≤ k , where k is a constant satisfying 0< k <1.

Reject H ο if and only ifθ° x <k , where 0<k < 1.

ϖ ={ μ : μ=μ ο } and Ω={ μ :−∞< μ<+ ∞ }

Log likelihood function will be as

Now the first partial derivative w.r.t μ

Put it equal to zero

^μ= Σ xi /n => ^μ= x

Now solving for the numerator of equation A using 1

Then solving for the denominator from equation A using 1

By putting values we have

¿= ( xi−x ¿ ¿2 +¿( xi−x ¿ ¿2 ¿

Appling sum on the above equation

Sum of square deviation from its mean is zero

So equation 5 reduced into

Then will use the critical function

Where λ οis a constant that lies between0< λο <1

ϖ ={ μ : μ=μ ο } and Ω={ μ :−∞< μ<+ ∞ }

Log likelihood function will be as

Now take the first partial derivative w.r.t μ