EECS 16B Designing Information Devices and Systems II

Spring 2018 J. Roychowdhury and M. Maharbiz Homework 9

This homework is due on Thursday, April 19, 2018, at 11:59AM (NOON).
Self-grades are due on Monday, April 23, 2018, at 11:59AM (NOON).

1. Interpolation
Samples from the sinusoid f (x) = sin(0.2πx) are shown in Figure 1. Draw the results of interpolation using
each of the following three methods:

Figure 1: Samples of f (x).

.

(a) Zero order hold interpolation.

Solution:

Figure 2: Zero order hold.

.

EECS 16B, Spring 2018, Homework 9 1

 (b) Linear interpolation.
Solution:

Figure 3: Linear interpolation.

.

(c) Sinc interpolation.

Solution:

Figure 4: Sinc interpolation.

.

2. Lagrange Interpolation by Polynomials

Given n distinct points and the corresponding evaluations/sampling of a function f (x), (xi , f (xi )) for 0 ≤
i ≤ n − 1, the Lagrange interpolating polynomial is the polynomial of the least degree that passes through
all the given points.
Given n distinct points and the corresponding evaluations, (xi , f (xi )) for 0 ≤ i ≤ n − 1, the Lagrange poly-
nomial is
i=n−1
Pn (x) = ∑ f (xi )Li (x),
i=0
where
j=n−1
(x − x j ) (x − x0 ) (x − xi−1 ) (x − xi+1 ) (x − xn−1 )
Li (x) = ∏ = ··· ···
j=0, j6=i (xi − x j ) (xi − x0 ) (xi − xi−1 ) (xi − xi+1 ) (xi − xn−1 )

EECS 16B, Spring 2018, Homework 9 2

.
Here is an example: for two data points, (x0 , f (x0 )) = (0, 4), (x1 , f (x1 )) = (-1, -3), we have

x − x1 x − (−1)
L0 (x) = = = x+1
x0 − x1 0 − (−1)
and
x − x0 x − (0)
L1 (x) = = = −x
x1 − x0 (−1) − (0)
. Then
P2 (x) = f (x0 )L0 (x) + f (x1 )L1 (x) = 4(x + 1) + (−3)(−x) = 7x + 4
We can sketch those equations on the 2D plane as follows:

(a) Given three data points, (2, 3), (0, −1), and (−1, −6), find a polynomial f (x) = ax2 + bx + c fitting
the three points. Do this by solving a system of linear equations for the unknowns a, b, c. Is this
polynomial unique?
Solution:
Plug in the three data points into f (x) = ax2 +bx+c, we get 4a+2b+c = 3, c = −1 and a−b+c = −6.
Solving these equations for the coefficients a, b and c, we get f (x) = −x2 + 4x − 1.
This polynomial is the unique degree 2 polynomial defined by these three distinct points.
(b) Like the monomial basis {1, x, x2 , x3 , . . .}, the set {Li (x)} is a new basis for the subspace of degree n
or lower polynomials. Pn (x) is the sum of the scaled basis polynomials. Find the Li (x) corresponding
to the three sample points in (a). Show your work.
Solution:
Assume x0 = 2, x1 = 0 and x2 = −1, each corresponding Li (x) is:

EECS 16B, Spring 2018, Homework 9 3

 (x − x1 ) (x − x2 ) (x − 0) (x − (−1)) x2 + x
L0 (x) = = =
(x0 − x1 ) (x0 − x2 ) (2 − 0) (2 − (−1)) 6

(x − x0 ) (x − x2 ) (x − 2) (x − (−1)) x2 − x − 2
L1 (x) = = =
(x1 − x0 ) (x1 − x2 ) (0 − 2) (0 − (−1)) −2

(x − x0 ) (x − x1 ) (x − 2) (x − 0) x2 − 2x
L2 (x) = = =
(x2 − x0 ) (x2 − x1 ) (−1 − (2)) (−1 − 0) 3
(c) Find the Lagrange polynomial Pn (x) for the three points in (a). Compare the result to the answer in (a).
Are they different from each other? Why or why not?
Solution:

x2 + x x2 − x − 2 x2 − 2x
Pn (x) = f (x0 )L0 (x) + f (x1 )L1 (x) + f (x2 )L2 (x) = 3 × + (−1) × + (−6) ×
6 −2 3
Therefore, Pn (x) = −x2 + 4x − 1.
The answer is the same as the one we computed in (a). For these three points, we cannot have a
polynomial of less than 2 degree fitting all of them. Lagrange interpolation must construct the least
degree (n = 2) polynomial passing the three points, which must be unique.
(d) Sketch Pn (x) and each f (xi )Li (x) on the 2D plane.
Solution:

EECS 16B, Spring 2018, Homework 9 4

 (e) Show that the Lagrange interpolating polynomial must pass through all given points. In other words,
show that Pn (xi ) = f (xi ) for all xi . Do this in general, not just for the example above.
Solution:
j=n−1
(x − x j )
For each xi , the corresponding Li (x) is Li (x) = ∏ . Plugging in xi , we get
j=0, j6=i (xi − x j )

j=n−1
(xi − x j )
Li (xi ) = ∏ = 1.
j=0; j6=i (xi − x j )

j=n−1
(xi − x j )
For any other Lm (x), where m 6= i, plug in xi , Lm (xi ) = ∏ . Because m 6= i, Lm (xi ) must
j=0, j6=m (xm − x j )
(xi −xi )
have this term (xm −xi ) = 0. Hence all Lm (xi ) must be zero.
Therefore, Pn (xi ) = ∑k=n−1
k=0 f (xk )Lk (xi ) = f (xi ) · 1 = f (xi ).

3. The Vector Space of Polynomials

A polynomial of degree at most n on a single variable can be written as

p(x) = p0 + p1 x + p2 x2 + · · · pn xn ,

where we assume that the coefficients p0 , p1 , . . . , pn are real. Let Pn be the vector space of all polynomials
of degree at most n.

(a) Consider the representation of p ∈ Pn as the vector of its coefficients in Rn+1 .

h iT
~p = p0 p1 · · · pn

Show that the set Bn = {1, x, x2 , . . . , xn } forms a basis of Pn , by showing the following.
• Every element of Pn can be expressed as a linear combination of elements in Bn .
• No element in Bn can be expressed as a linear combination of the other elements of Bn .
(Hint: Use the aspect of the fundamental theorem of algebra which says that a nonzero polynomial
of degree n has at most n roots and use a proof by contradiction.)
Solution:
We can write every polynomial of degree n on x as p(x) = p0 + p1 x + · · · pn xn , which is a linear
combination of the elements in Bn .
Let xi ∈ Bn , for 0 ≤ i ≤ n. We show that xi cannot be written as a linear combination of the other
elements of Bn . Suppose that there exist real constants c j for j 6= i, such that xi = ∑ j6=i c j x j . This gives
us an equation
∑ c j x j − xi = 0 .
0≤ j≤n, j6=i

The left hand side is a nonzero polynomial of degree at most n, which we know has at most n roots
by the relevant aspect of the fundamental theorem of algebra. Thus the equality cannot hold for all x,
which is a contradiction.

EECS 16B, Spring 2018, Homework 9 5

 (b) Suppose that the coefficients p0 , . . . , pn of p are unknown. To determine the coefficients, we evaluate
p on n + 1 points, x0 , . . . , xn . Suppose that p(xi ) = yi for 0 ≤ i ≤ n. Write the matrix V in terms of the
xi , such that    
p0 y0
 p1  y1 
   
V ..  =  ..  .
  
  .
.
pn yn

Solution:
We can write each equation p(xi ) = yi as
 
p0
i p1 
h 
p0 + p1 xi + p2 xi2 + · · · + pn xin = 1 xi xi2

· · · xin 
 ..  = yi .

.
pn
If we arrange these n + 1 equations into a matrix, we get
    
1 x0 x02 · · · x0n p0 y0
2 n
1 x1 x1 · · · x1   p1  y1 
    
  ..  =  ..  .
. . .. .. ..     
. .
 . . . . .  .   . 
2
1 xn xn · · · xn n pn yn
h i
The matrix V above with rows 1 xi xi2 · · · xin is also known as a Vandermonde matrix.
(c) For the case where n = 2, compute the determinant of V and show that it is equal to
det(V ) = ∏ (x j − xi ) .
0≤i< j≤n

Under what conditions is this matrix invertible?

Solution:
If n = 2, we can write V as  
1 x0 x02
V = 1 x1 x12 
 
1 x2 x22
We can compute the determinant of V by using row reduction. This gives us the matrix
 
1 x0 x02
0 x1 − x0 x12 − x02
 

0 0 x22 − x1 x2 − x0 x2 + x0 x1
Since this is upper-triangular, we can compute the determinant by taking the product of the diagonal
entries.
det(V ) = x1 x22 − x12 x2 − x0 x22 + x02 x2 + x0 x12 − x02 x1 .
We can check that this is equal to
(x2 − x1 )(x2 − x0 )(x1 − x0 ) = x1 x22 − x12 x2 − x0 x22 + x02 x2 + x0 x12 − x02 x1
Note that det(V ) is nonzero, or V is invertible, if and only if all x0 , x1 , x2 are distinct. If this is the case,
then there is a unique solution for p0 , p1 , p2 .

EECS 16B, Spring 2018, Homework 9 6

 (d) We can define an inner product on Pn by setting
Z 1
hp, qi = p(x)q(x) dx .
−1

Show that this satisfies the following properties of a real inner product. (We would need a complex
conjugate if we wanted a complex inner product).
• hp, pi ≥ 0, with equality if and only if p = 0.
• For all a ∈ R, hap, qi = ahp, qi.
• hp, qi = hq, pi.
Solution:
1
p2 (x) dx ≥ 0 since p2 (x) ≥ 0 for all x. The integral is 0 if and only if p = 0 for all x,
R
• hp, pi = −1
or p is the zero polynomial.
R1 R1
• hap, qi = −1 (ap(x))(q(x)) dx = a −1 p(x)q(x) dx = ahp, qi.
R1 R1
• hp, qi = −1 p(x)q(x) dx = −1 q(x)p(x) dx = hq, pi.
(e) OPTIONAL: An alternative inner-product could be placed upon real polynomials if we simply repre-
sented them by a sequence of their evaluations at 0, 1, . . . , n and adopted the standard Euclidean inner
product on sequences of real numbers. Can you give an example of an orthonormal basis with this
alternative inner product?
Solution:
For p, q ∈ Pn , this alternative inner product is given by
n
hp, qi = ∑ p(i)q(i) .
i=0

An example of an orthonormal basis with this inner product would be the Lagrange polynomials Li (x)
constructed on the points 0, 1, . . . , n. Since Li (i) = 1 and Li ( j) = 0 for j 6= i, 0 ≤ i, j ≤ n, thus for i 6= j
we clearly have hLi , L j i = 0 and we can also check that hLi , Li i = 1.

4. Sinc Functions
The sinc function is defined as (
sin(πx)
πx , x 6= 0
sinc(x) = .
1, x=0

(a) Verify that

sin(πx)
lim = 1.
x→0 πx
Hint. Look into L’Hospital’s Rule.
Solution:

d
sin(πx) dx sin(πx)
lim = lim d
x→0 πx x→0
dx πx
π cos(πx)
= lim
x→0 π
= 1 = sinc(0)

EECS 16B, Spring 2018, Homework 9 7

 (b) Verify that
1
Z π
cos(ωx)dω = sinc(x)
π 0

Solution:

1 sin (ωx) π
 
1
Z π
cos(ωx)dω =
π 0 π x ω=0
sin (πx)
=
πx

(c) If we think of the above integral as a sum of oscillatory functions of different frequencies, the sinc
function has continuous frequency components in the range [0, π]. What is the range of frequencies of
the function  
x
f (x) = sinc ?
T
Hint: Try to use substitution and the above integral.
Solution:
We have, 
1
Z π
ωx
cos dω
π 0 T
ω
Let λ = T. Then,
π
1
Z
T
π cos (λ x) dλ
T 0
π
Since we’re summing
 π  over a continuum of frequencies from 0 to T, this tells us that the range of
frequencies is in 0, T .
(d) Continuing the idea of integration as the continuous version of sums, we define the continuous inner
product between two real functions as
Z ∞
hp, qi = p(x)q(x)dx.
−∞

Given that the length squared of a vector ~v of length n is

n
k~vk2 = h~v,~vi = ∑ vi vi ,
i=1

what is the analogous length squared of a function, f , using the inner product definition above? This
is a one-line answer.
Solution:
Z ∞
2
kfk = hf, fi = | f (x)|2 dx
−∞

We only consider interpolating functions where k f k < ∞, so the inner product is well defined. It turns out
that the set of shifted sinc functions are orthonormal with the above inner product! Let

φk (x) = sinc (x − k) , where k is an integer.

EECS 16B, Spring 2018, Homework 9 8

 Then, (
1, if m = n
kφk k = 1 and hφm , φn i =
0, otherwise

5. LTI Inputs
We have an LTI system whose exact characteristics we do not know. However, we know that it has a finite
impulse response that is not longer than 5 samples. We also observed two sequences, x1 and x2 , pass through
the system and observed the system’s responses y1 and y2 .

x1 0 2 2 0 −1 0 0 0 0 0
y1 0 4 10 8 −2 −3 1 1 −1 0

x2 −2 0 1 0 0 0 0 0 0 0
y2 −4 −6 0 5 −1 −1 1 0 0 0

(a) Given the above sequences, what would be the output for the input?

x3 0 0 −2 0 1 0 0 0 0 0

Solution:
This is a shifted version of x2 . Since the system is time-invariant, the output is:

y3 0 0 −4 −6 0 5 −1 −1 1 0

(b) What is the output of the system for the input x1 − x2 ?

Solution:
Since the system is linear, the output is y1 − y2 :

EECS 16B, Spring 2018, Homework 9 9

 y1 − y2 4 10 10 3 −1 −2 0 1 −1 0

(c) Given the above information, how could you find the impulse response of this system? What is the
impulse response?
Solution:
Since 21 (x1 + x3 ) is an impulse at the second sample, 1
2 (y1 + y3 ) is the system’s impulse response
starting from the second sample.

y1 + y3 0 4 6 2 −2 2 0 0 0 0

so
n n<0 0 1 2 3 4 n>4
h[n] 0 2 3 1 −1 1 0
(d) What is the output of this system for the following input:

x4 1 1 1 0 0 0 0 0 0 0

Solution:

y4 2 5 6 3 1 0 1 0 0 0

6. LTI Low Pass Filters

Given a sequence of discrete samples with high frequency noise, we can de-noise our signal with a discrete
low-pass filter. Two examples are given below:

y[n] = 0.5y[n − 1] + x[n] (1)

y[n] = 0.25x[n] + 0.25x[n − 1] + 0.25x[n − 2] + 0.25x[n − 3] (2)

(a) Write the impulse responses h[n] for (1) and (2). Are they IIR or FIR?
Solution:
For (1), (
0.5n , n ≥ 0
h[n] =
0, n<0
This has an infinite impulse response (IIR).
For (2), (
0.25, n = 0, 1, 2, 3
h[n] =
0, otherwise
This has a finite impulse response (FIR).
(b) Are either of these filters causal? Are either of these filters stable?
Solution:
Yes, y[n] depends only on inputs from x[n] and before.
Yes, both filters are stable because the impulse responses are absolutely summable.

EECS 16B, Spring 2018, Homework 9 10

 (c) Find the output y for each filter given the input sequence x[n] = 2 cos(πn) + n from n = 0 to n = 7.
Assume that y[n] = 0 for n < 0.
Solution:
For (1),
y = [2, 0, 4, 3, 7.5, 6.75, 11.375, 10.6875]
For (2),
y = [0.5, 0.25, 1.25, 1.5, 2.5, 3.5, 4.5, 5.5]

7. (OPTIONAL) Polynomial Interpolation and Regression

Variometers indicate the rate of climb or descent (vertical velocity) in an aerial vehicle. I have a small
variometer sensor that I would like to use on a quadrotor for altitude control. In order to evaluate the
sensor’s performance, I launch it on a small model rocket and record its measurements. The recorded values
are:

Time (s) Velocity ( ms )

0 3.75
0.5 −1.5
1.5 −8.5

(a) Perform polynomial interpolation on these points. Show your work (you can use a computer to calcu-
late the matrix inverse).
Solution:
Since there are three points, we can find a unique second-order polynomial that will pass through all
of the points:     
1 x1 x12 a0 y1
2
1 x2 x2  a1  = y2 
    
1 x3 x32 a2 y3
Plugging in the values from the table, we have:
    
1 0 0 a0 3.75
1 0.5 0.25 a1  = −1.5
    
1 1.5 2.25 a2 −8.5

We can solve for the coefficients ai by inverting the Vandermonde matrix:

    
15
a0 1 0 0
   8  4 
a1  = − 3 3 − 13   − 32 
4
a2 3 −2 23 − 17
2
 
15
 4 
= − 35
3
7
3

The final polynomial is

7 35 15
v(t) = t 2 − t +
3 3 4

EECS 16B, Spring 2018, Homework 9 11

 (b) The rocket is in freefall, so I expect that it will approximately follow a parabolic trajectory (its flight
will not be exactly parabolic due to wind resistance). The velocity profile of a parabolic trajectory is
linear. Perform polynomial regression using a first-order polynomial.
Solution:
The final polynomial is v(t) = −8t + 15
4.

Contributors:

• John Maidens.

• Siddharth Iyer.

• Justin Yim.

EECS 16B, Spring 2018, Homework 9 12