1

HW #5

1. Consider a mixed logistic model as follows:

(i) Given the random e↵ects u1 , . . . , um , binary responses yij , i = 1, . . . , m, j = 1, . . . , ni are
(conditionally) independent such that logit{P(yij = 1|u)} = x0ij + ui .
(ii) u1 , . . . , um are independent ⇠ N (0, 2 ).
a. Show that for any measurable g(·) such that E{|g(ui )|} < 1, we have E{g(ui )|y} =
E(g(ui )|yi }, where yi = (yij )1jni .
b. Consider sampling from f (ui |yi ) using the M-H algorithm, given the current parameters.
Let the jumping distribution be f (ui ), the marginal pdf of ui . Show that

exp{yij (x0ij + ui )
f (yij |ui ) = .
1 + exp(x0ij + ui )

c. Show that the acceptance probability is ↵ = ^ 1, where

8 9
<Y ni
1 + exp(x0ij + ui ) = ⇤
=
: 1 + exp(x 0 + u ⇤ ) ; exp {yi· (ui ui )} ,
j=1 ij i

where ui is the current ui and u⇤i is the candidate draw from the jumping distribution.

2. Consider estimating the normal mean from observations y1 , . . . , yn , which are independent
from the N (✓, 1) distribution. Let the prior distribution for ✓ be N (µ, 1), where µ is a given
number.
a. Derive the posterior mean ✓˜ = E{✓|y (K) }, where y = (yi )1in and y (K) = (y, . . . , y),
where the y’s are independent copies repeated K times.
b. What is the MLE of ✓, ✓? ˆ
c. How di↵erent is the MLE from the posterior mean? Does the di↵erence vanish as K
increases? Does the choice of µ matter when K goes to infinity?
 Let g be measurable ,
El glue) ) so

a) Yij only related to Yi and hi independent

conditionally independent
are
U:
are .

on

Elohim 's ) El gladly ya yn) = IE lgluillyi ) .

Actual proof
last page
. . . .

. .

the

ftp.g-nij.Btdiflyiglei
Bl Pig) log
b)
Yij #
Milos Pis t (t ki ) ly fi p
?
Did
- -

'
"
-

) -
Pig if .
, -

p, ;)

exp f big log Iii ,

t
tog 'll Pil }
-

't tail } Pij

exp { :( ng
=
9 t I -

"'

fog ftp..ge
!i÷ nii 't
.

=p ,
=

log =
too

fhteeis
, .

→ pig
.

( a teii ) -
-
Elise as Pii =

Elis l

l -

Pig . p - =

flit
/ f erie It

→ fly
.gr/uiS.expd9iilnij9steiI41texpfnijtBtUi9
 f- ( 4. lui ) =

¥! expdbislno.gr/BtUiI41texpEnoj'Btai }
*

.it?YMfie:jyis;: :.tf:?!g.:.i expdsiilnis

*

Hid
,

x.
f- ( bilked I
texpdu.gr 'B tails
*

qexpdbijlxi ktllil-Yijltigktnif.si/texpEnoj'Btaiy } I 1-
exp { hog 'Btai* }

gift texpsnijitai !
}
*
⇐
"
"
P f9ij ( Ui -

ki A }
I
texpfq.jp + g a
,

a) I Yn n
NO ' ) O -

NIM D ,

ni in
. . .

. . ,

'"
ne
Y -
( Yi Yn ) Y =
( Y ,Yn Y ,Un . ! . . . .
Yn )
:
.
. .
. .
.
,
. . . .
.
. .

.
.

,
. .
.
.
,

( Yi Ya In )
Ya 92 Bn
=
-
, . . . . . . -
- -

,
, , , ,
,
m me -
k k k
 Yi -
( Yi Yi )
Kritik exp f-%
.
-

}
,
.

( Sia ol
'
5- Gilo )
,

I
-

f- kilos @lt5khe.xp f Ya ktti ol

}
- -
'
-

"
'
%
f- ( Y' lo )
exp f Yak ! }
''

HTT
'

-2 Iyo ol
-

> -

@itYkexpf.Yglo-ieJ2lgf oIymJ.y
f- (o ) =

fly lo) f- (o)

"'

fly lol Hol

@Itjknkexpf.4ak5ilyi.oYlgxCaitIkexpf.t.sk
""
-
-

MY
×
expf-tafk-2.la .
-

ol 't lo -

AM }.

exp f- ya f. k£65 20%1-07

*

]}
' '
to aorta
-

4
f Yg Ik -2in Yi aok-2iyitkno.to goatee 't
'

}
'

exp
-

let
simplify
k
-2in Yi '
-

aok-2iyitkm.IE
'
-

sortie
.

= OH at kn ) -

201k -291.tn/tk-2iYi2tM
'

aofkfyu.tn/tkIiyit9
' "

O -

'

.
I t kn t tkn

It kn

f lo Ffi
Him }
"
-

Holy " ) x
exp

k£9: th =§ variance EE t

Posterior mean
is
I fkn
A fkn
 § .
o :( KEY ta ) as B . - E' ( keg +
u
)
.

""

kexpf
-

b) Holy ) "'
-
Htt ) -

Yak -21%-0124
'

blob ) -2 ! ( )
""

Kzn log
'
29 Yi O
-

zk
= - -

of Eg H
=
-
.
2 . -

I .
-2 ! Yi -
o
'
-
O

kf-2.is ,
-

nof -

-
o as 8--271=5
MLK is g-

a) 8=5 -
( Yin ta
) 8=5

( t.at/..so--glo
.

)
B - E.

J -
O = NKT '
Ft 82M -
j

g- In # E' a) -
*tF'a

817 :* ) -
t t Fink
"

K -80 J -

O to
K sa
µ
-

- se 0
No , sthechrice M does not matter since Itnk
of
HI
to

Let g be measurable ,
El glue) ) so

a) Note
that is Big are
conditionally independent .

Ii ) re : only related to Yi

iii ) and Ri are

independent

fu !
""
Ekdahl 's )
pfmfgnglui ) dude
-
.

I 1

.
=

I
In fan I
glad this )
fly )
da dy

f,µnfµm glad fluincooilmid ) die .

. . .

dandy
f- ( y )

fam ! ! glue ) flags ) da ;

=

dis
fly )

↳ f! glue ) flu Ym )
=

:
.
In .
. .
.

da . da . . .

dyn
m

$44 ,
. .
.

,
Dm)

f! f! glee) this 's :)

=

duo dy ,
.

f- ( Bo )

=
f- ( ghosts ) .