VE564 Homework 2

Try your best! Honor Code Enforced!

Due on June 26, Electronic Version
Problem 1: Consider the measurement model with two samples
xrns “ Asrns ` wrns n “ 0, 1
where s is known and w has zero mean and covariance matrix C.
‚ Find the BLUE of A and corresponding minimum variance.
‚ What happens if s is an eigenvector of C?
‚ Let C “ diagp1, 1q and C̃ “ diagp1, αq. Suppose in the above BLUE estimator, we wrongly use C̃ instead of
C, please compare the variances of the resulting estimators for the three cases α Ñ 0, α “ 1 and α Ñ 8.

Problem 2: Please find the MLEs for the following two models:
‚ We have N i.i.d samples from N p0, 1{θq with θ ą 0.
‚ We have N i.i.d samples from N pA, σ 2 q and want to estimate α “ A2 {σ 2 .

Problem 3: We try to derive the order-recursive update using the geometrical interpretation of LS. Starting from
θ̂ k and ŝk “ Hk θ̂ k . If we add hk`1 , the LS estimate becomes
ŝk`1 “ ŝk ` αh1k`1
where h1k`1 is the component of hk`1 orthogonal to the subspace spanned by th1 , h2 , ¨ ¨ ¨ , hk u. Note that αh1k`1
is the orthogonal projection of x onto h1k`1 .
Please find h1k`1 , α and then θ̂ k`1 .

Problem 4: Use the Levenberg-Marquardt algorithm to fit an exponential function of the form
y “ f px; θq “ θ1 eθ2 x
for the data pxi , yi q: p0, 5.2q, p1, 4.5q, p2, 2.7q, p3, 2.5q, p4, 2.1q, p5, 1.9q. After obtaining θ̂1 , θ̂2 , please plot the fitting
curve along with the data points.

Problem 5: Consider the model

xrns “ A ` Bn ` wrns ´M ďnďM
where wrns „ N p0, σ 2 q. If we have the prior knowledge that
„  ˆ„  „ 2 ˙
A A0 σA ρ
„N , 2
B B0 ρ σB
Please find the MMSE of A, B and the corresponding MSE. What happens if ρ “ 0?

Problem 6: An inner product in a real space xx, yy must satisfy the following properties
‚ xx, xy ě 0 and xx, xy “ 0 if and only if x “ 0
‚ xx, yy “ xy, xy
 ‚ xc1 x1 ` c2 x2 , yy “ c1 xx1 , yy ` c2 xx2 , yy @c1 , c2 P R
Please verify that Epxyq is indeed a valid inner product of zero-mean random variables. For non zero-mean random
variables, is covpx, yq a valid inner product?