Loglinear Models 4

Estimation of Parameters in Loglinear Models

Consider a 2x2 table for categorical response variables X and Y that is based on Poisson
sampling ( n is not fixed). Suppose we wish to fit the independence model
logm ij = μ + λ Xi + λ Yj
In this situation the n ij are independent Poisson random variables with means m ij. Let
n = n 1, n 2 , n 3 , n 4  ′
and
m = m 1, m 2 , m 3 , m 4  ′
where n 1 = n 11, n 2 = n 12 , n 3 = n 21 , n 4 = n 22 and
m = m 11, m 2 = m 12 , m 3 = m 21 , m 4 = m 22 . Since
I J

∑ λ Xi =∑ λ Yj = 0
i=1 j=1
we can write the abaove model in matrix form as follows:

logm 11  logm 1  1 1 1
μ
logm 12  logm 2  1 1 −1
logm = = = λ Xi = Xβ
logm 21  logm 3  1 −1 1
λ Yj
logm 22  logm 4  1 −1 −1
where
β1 μ
β = β 2 = λ Xi
β3 λ Yj
Thus all loglinear models can bae written in the form logm = Xβ where β is a model
parameter column vector containing t parameters.
ni
n
e =m i m i i
For Poisson sampling, the likelihood of the data is Π ni!
so that the kernel is Π e =m i m ni i .
i i
The log likelihood of the kernel is
Lm =∑ n i logm i  −∑ m i =∑ n i ∑ x ij β j  −∑ exp∑ x ij β j 
i i i j i j

Since m i = exp ∑ x ij β j 
j
∂Lm
=∑ n i x ij −∑ m i x ij , j = 1, ..., t
∂β j
i i

The likelihood equations equate these derivataives to zero and have form X ′ n = X ′ m.
The matrix of second partial derivatives of the log likelihood has elements

1
 ∂ 2 Lm
∂β j ∂β k
=− ∑ x ij ∂m i
∂β k
=− ∑ x ij x ik m i
i i

We estimate the information matrix, (the negative of the above matrix) by

X′DiagmX

As n → ∞, the maximum likelihood estimator β of β has an asymptotic normal distribution
with mean β and covariance matrix equal to the inverse
 of the information matrix. Thus, for
Poisson sampling, the estimated covariance matrix of β is
 −1
Covβ = X′DiagmX
Similar results hold when we use maximum likelihood estimation after conditioning on
n.When the n i are independent Poisson random variables (i = 1, ..., N) the conditional
distribution of m given n is multinomial with parameters π i = m i / ∑ m i . It is not necessary to
develop separate inferential theory for multinomial loglinear models,as shown by the
following (see Birch 1963).

First, express the Poisson loglinear model as

logm i = μ + x i β τ
where 1, x i  is the i th row of X and (μ, β ′r  ′ is the model parameter vector. The Poisson log
likelihood is
Lμ, β τ  = ∑ n i logm i  − ∑ m i
= ∑ n i μ + x i β τ  − ∑ expμ + x i β τ 

= nμ + ∑ n i x i β τ − τ
where
τ= ∑ m i = ∑ expμ + x i β τ 

Since
logτ = μ + log∑ expx i β τ 
this log likelihood has the form
Lτ, β τ  = ∑ n i x i β τ − n log∑ expx i β τ  + n logτ − τ
Now
π i = m i / ∑ m i = expμ + x i β r / ∑ expμ + x i β τ 
and exp μ cancels in the numerator and the denominator.

2
Thus the first term in Lτ, β τ  is
∑ n i x i β τ − n log∑ expx i β τ = ∑ n logπ i 
which is the multinomial log likelihood conditional on n.

Unconditionally, n = ∑ n i has a Poisson distribution with mean ∑ m i = τ so the second

term of Lτ, β τ  given by n logτ − τ is the Poisson log likelihood for n.

Since β τ enters only in the first term, the maximum likelihood estimator β τ and its
covariance matrix for the Poisson likelihood Lμ, β τ  are identical to those for multinomial
likelihood. Note, however, that the Poisson loglinear model requires one more parameter
than the multinomial loglinear model.

A similar argument applies when there are several independent multinomial samples rather
than a single one. Each log likelihood term is a sum of components from diferent samples,
but the Poisson log likelihood again decomposes into two parts- one part is a Poisson log
likelihood for the independent sample sizes, and the other part is the sum of the
independent multinomial log likelihoods.