EC484 CLASSES: WEEK 5

KAMILA NOWAKOWICZ

Question 4, Problem Set 2

In this question we are asked to prove an “if and only if” statement, which means we have to
prove both directions.
Notice that the assumptions given in the question are the same as those in Theorem 13:
For y = Z + u and Qn = Z 0 Z, if E(ui ) = 0, E(ui uj ) = 2
1(i = j), then
ˆn 2nd
! () (Qn ) ! 1 as n ! 1

Where (Qn ) denotes the minimum eigenvalue of Qn . In the proof we will use the fact that the
minimum eigenvalue can be expressed as:

(A) = inf x0 Ax.

kxk=1

Solution. In this question we have:

Qn = Z 0 Z
n
X
= zi zi0
i=1
!
n
X 1 ⇣ ⌘
= 1 i
i=1
i
n
!
X 1 i
=
i=1
i i2
Pn !
n i=1 i
= Pn Pn
i=1 i i=1 i2
and by Theorem 13, we need to show
1
(Qn ) ! 1 if and only if  < 0 or > 0.
2
(1) Necessity:

1
(Qn ) ! 1 )  < 0 or > 0.
2
These solutions are adapted from solutions by Chen Qiu, which were based on Prof Hidalgo’s notes and
solutions for EC484. Their aim is to fill some gaps between notes and exercises. Prof Hidalgo’s notes should
always be the final reference. If you spot any mistakes in this file please contact me: K.Nowakowicz@lse.ac.uk.
1
 This is equivalent to its contrapositive:

1
< or = 0 ) (Qn ) 9 1.
2

!
n n
(i). If = 0: Qn = . The determinant of this matrix is
n n

det(Qn ) = n2 n2 = 0

and the determinant is a product of eigenvalues, so there is an eigenvalue equal

to 0. Hence the smallest eigenvalue can’t go to infinity1.
(ii). If < 1
2
, we can use the fact that (Qn )  x0 Qn x, with x = ( 01 ), or Remark 6
(which states that the smallest eigenvalue is less than or equal to any diagonal
element of Qn ) to show that (Qn ) is bounded above by the diagonal entry
Pn 2
i=1 i , which is smaller than infinity. For < 21 we have 2 < 1, and
P1 ↵
i=1 i < 1 for any ↵ < 1. Hence:

n
X
(Qn )  i2 < 1.
i=1

(2) Sufficiency:

1
 < 0 or > 0 ) (Qn ) ! 1
2
Pn Pn
For 1
2
 < 0 or > 0 the entries i=1 i2 and i=1 i increase to infinity
with n. We are interested in knowing how fast they increase relative to each other.
We can approximate their rate of growth using a trick involving Riemann Integrals.
Recall that we define the Riemann integral as the limit of a sum of areas of rectangles
(approximating the area under the graph) as the width of the rectangles goes to zero.

1We could also find the eigenvalues by solving the equation:

!
n n
= (n )2 n2 = 2n + 2
= ( 2n)
n n

which gives 1 = 0 and 2 = 2n.

Or we could notice that 1
1 is the eigenvector corresponding to the eigenvalue 0.
2
 Firstly, we want to rewrite the sum in a form of a Riemann sum, for rectangles with
width 1
n
approximating the function f (x) = x :
X n Xn ✓ ◆
+1 i 1
i =n
i=1 i=1
n n
Z 1
+1
⇠n i di
0
 +1 1
+1 i
=n
+1 0
✓ ◆
+1 1
=n 0
+1
+1
n
=
+1
Note that the integral is a limit of the sum as n ! 1. We use bounds 0 and 1 because
i
n
! 0 for i = 1 and i
n
= 1 for i = n.

x x

x 1 2 3 4 5 x
1 2 3 4 5
n n n n n ··· 1 n n n n n ··· 1
For 1
2
< < 0. For > 0.
Similarly:
n
X Xn ✓ ◆2
2 2 +1 i 1
i =n
i=1 i=1
n n
Z 1
⇠ n2 +1 i2 di
0
 1
2 +1 i2 +1
=n
2 +1 0
✓ ◆
1
= n2 +1 0
2 +1
n2 +1
=
2 +1
However, note that the procedure fails when = 12 . The above expression is correct for
1
2
< < 0 and > 0. We need to consider = 1
2
separately. For = 1
2
we want to
3
 Pn
find the limit of i=1 i .
1
We use a similar trick, but instead of letting the width of the
rectangle be 1
n
we set it to 1:
n
X Z n
1 1
⇠ di
i=1
i 1 i
= [log(i)]n1
= log(n) log(1)
= log(n).

Now that we have found the rates of convergence of all elements of the matrix we can
show that the smallest eigenvalues go to infinity. We consider two cases:
(i). When 1
2
< < 0, or > 0, we can approximate
Pn !
n i
Pn Pni=1 2
i=1 i i=1 i

by
+1
!
n
n +1
n +1 n2 +1
.
+1 2 +1

We want to show (Qn ) ! 1. Since

(Qn ) = inf x0 Qn x
kxk=1

it suffices to show x0 Qn x ! 1, 8 kxk = 1. For any x = (a, b) such that a2 +b2 = 1:

! !
⇣ ⌘0 n n +1
a n +1 2 +1
+1 2 2 n
a b n +1 n2 +1
= a n + 2ab + b
+1 2 +1
b +1 2 +1
✓ 2
◆
2 n 2 n
= n a + 2ab +b
+1 2 +1
✓ ◆2 !
n b2 n 2 2
= |{z}
n a+b +
+1 (2 + 1)( + 1)2
!1
| {z }
>0

The term on the right is a sum of two terms which are weakly positive. The
second term can only be zero if b = 0, but then a = 1 (the vector (a, b) has
length 1) so the first term is strictly positive. When the first term is zero we
have b = +1
n
a 6= 0 (can’t have both a and b equal to 0), so the second term
is strictly positive. Hence the multiplier of n is always strictly positive, so the
whole expression always goes to infinity and we always have (Qn ) ! 1.
(ii). When = 1
2
, Qn is approximated by
!
n 2n1/2
.
2n1/2 log n
4
 We can show that (Qn ) ! 1 by showing that tr (Qn 1 ) ! 0 and Qn is positive
semidefinite. Then we know that all eigenvalues are non-negative and their sum
goes to zero in the limit, so all eigenvalues of Qn 1 must go to zero.
!
1/2
1 log(n) 2n
Qn 1 ⇠
n log(n) 4n 2n1/2 n
so that
log(n) + n 1
tr Qn 1 ⇠ ⇠ !0
n log(n) 4n log(n)
as n ! 1. To see that Qn is positive semidefinite, note that Qn = Z 0 Z. Then
for any x 2 Rn :
n
X
0 0 0 0 0
x Qn x = x Z Zx = (Zx) (Zx) = y y = yi2 0
i=1

where Zx = y = (y1 , y2 , . . . , yn )0 .
This completes the proof.

Question 5, Problem Set 2

So far we have used the following methods to show convergence in probability:

(1) by definition: P r {|Xn X| > } ! 0;
(2) by convergence in 2nd mean;
(3) by convergence in 1st mean.
In this question we can use a new method: convergence in distribution. Note that in gen-
eral convergence in distribution is weaker than convergence in probability. However, they are
equivalent when the limiting variable is a constant, which is the case in this example2.

Solution. I will solve the question under the additional assumption that variables xi are inde-
pendent. All existing sets of solutions I have seen assume independence and I don’t see how to
solve the question without adding that assumption. In fact, without independence the required
result may not be true.3
(a) We want to show that Yn = min(x1 , . . . , xn ), where xi have pdf:
8
<0 if x < ✓
f (x, ✓) =
:e✓ x if x ✓

2Since the estimators take the form min(.) or max(.) finding their expectations is not straightforward, so the
rth mean method is not a good choice. However, you can show these result from definition, and the proofs are
very similar to what we show here.
3Note that the identical marginal distributions given in the question are not enough to guarantee indepen-

dence: the variables could vary from perfectly correlated (x1 = x2 = · · · = xn ) to independent. To conclude
independence from distribution we would need to know the joint distribution of all xs. When they are perfectly
correlated the result is clearly violated.
5
converges in distribution to the constant ✓. The distribution of the limiting variable is
FY (y) = 1(y ✓), which has a discontinuity at y = ✓. By definition of convergence in
distribution we want to show:

lim FYn (y) = FY (y)

n!1

for all continuity points of FY . That is:

(i). for all y > ✓: limn!1 FYn (y) = 1,
(ii). for all y < ✓: limn!1 FYn (y) = 0.
We start by finding the expression for FYn .

FYn (y) = P r (Yn  y)

= P r (min (x1 , . . . , xn )  y)
=1 P r (min (x1 , . . . , xn ) > y)
=1 P r (x1 > y, x2 > y, . . . , xn > y)
n
Y
=1 P r (xi > y)
i=1

=1 (P r (x1 > y))n

=1 (1 Fx1 (y))n .

We now need an expression for the CDF of x1 . If y < ✓, Fx1 (y) = 0. If y > ✓:

Fx1 (y) = P r (x1  y)

Z y
= e✓ x dx
✓
Z y
✓
=e e x dx
✓
✓
⇥ ⇤y
=e e x ✓
= e✓ e y
+e ✓

=1 e✓ y .

So when y < ✓:

FYn (y) = 1 (1 Fx1 (y))n

=1 (1 0)n
=1 1
=0!0
6
 as n ! 1, as required. For y > ✓:

FYn (y) = 1 (1 Fx1 (y))n

n
=1 1 1 e✓ y

y n
=1 e✓
=1 en(✓ y)
!1

as n ! 1 (since ✓ y < 0 for y > ✓, so the exponent ! 1 and e 1

= 0). Hence
d p
Yn ! ✓, so also Yn ! ✓.
(b) This part is very similar, but now Yn = max(x1 , . . . , xn ) for xi ⇠ U [0, ✓]. As above, we
want to show convergence in distribution to ✓:
(i). for all y > ✓: limn!1 FYn (y) = 1,
(ii). for all y < ✓: limn!1 FYn (y) = 0.
We have:

FYn (y) = P r(Yn  y)

= P r (max (x1 , . . . , xn )  y)
= P r (x1  y, . . . , xn  y)
n
Y
= P (xi  y)
i=1

= (P (x1  y))n
= (Fx1 (y))n

Since x is uniformly distributed on [0, ✓]:

8
>
> 0 if y  0,
>
<
Fx1 (y) = P r (x1  y) = y✓ if 0 < y  ✓,
>
>
>
:1 if y > ✓.
Then if y > ✓:

FYn (y) = (Fx1 (y))n = 1n = 1 ! 1 as n ! 1

if 0 < y < ✓: ⇣ y ⌘n
FYn (y) = (Fx1 (y))n = ! 0 as n ! 1
✓
(note that in this case y
✓
< 1) and if y < 0:

FYn (y) = (Fx1 (y))n = 0n = 0 ! 0 as n ! 1.

d p
Hence Yn ! ✓, so Yn ! ✓ too.