ELEC505/ELEC405 Handout #3

Linear System Theory Monday, October 26, 2020

Homework Set #1
Due: Friday, November 6, 2020.

1. Sampling and Matrix Notation

h iT
Let’s represent a finite duration discrete time sequence with x ∈ <n , i.e., x = x1 x2 . . . xn .
Represent the following linear operations in the form y = Ax:

(a) Let y ∈ <2n−1 be the output of the upsampling system defined by

(
x k+1 k is odd
yk = 2 (1)
0 k is even
h iT
for k = 1, . . . , 2n − 1 and y = y1 y2 . . . y2n−1 .
n
(b) Assume n is even and y ∈ < . We define yk = x2k for k = 1, . . . , n2 .
2

n x2k +x2k−1
(c) Assume n is even and y ∈ < 2 . We define yk = 2
for k = 1, . . . , n2 .

2. Vector Space of Polynomials

Let Pn−1 represent the vectors space of polynomials with degree less than or equal to
n − 1. Therefore, each element in this vector space can be written as

p(x) = c0 + c1 x + c2 x2 + . . . + cn−1 xn−1 . (2)

As we discussed in class, we can form a one to one correspondence between Pn−1 and
h iT
<n . Let a = c0 c1 . . . cn−1 be the vector in <n corresponding to p(x) above,
dp(x)
and let b be the vector corresponding to dx
. Show that the relation between b and
a can be written as

b = Ha (3)

by explicitly writing H.

3. Affine functions
A function f : <n → <m is called affine if and only if for any x, y ∈ <n and any
α, β ∈ < satisfying α + β = 1, we have

f (αx + βy) = αf (x) + βf (y) (4)

(Note that, if we didn’t restrict α + β = 1, this would be the definition of a linear

function)

(a) Suppose that A ∈ <m×n and b ∈ <m . Show that f (x) = Ax + b is affine.

1
 (b) Now the converse: Show that any affine function f can be represented as f (x) =
Ax + b for some A ∈ <m×n and b ∈ <m . (Hint: Show that g(x) = f (x) − f (0)
is linear).

Therefore, an affine function can be considered as a linear function shifted by an offset.

4. Yet Another Proof of Cauchy-Schwarz...

√
(a) Suppose a ≥ 0, c ≥ 0 and for all λ ∈ R, a + 2bλ + cλ2 ≥ 0. Show that |b| ≤ ac.
n T
(b) Given u, w ∈ R , explain why (u + λw) (u + λw) ≥ 0 for all λ ∈ R.
(c) Apply a to the quadratic resulting from the expansion of the expression in (b), to
get the Cauchy-Schwarz inequality:
√ √
|vT w| ≤ vT v wT w

5. Inequality Proof
For a, b ∈ <n , show that ka + bk ≥ kak − kbk, where k · k is the Euclidian norm.

6. First steps in multivariable quadratic functions

Please write the multivariable quadratic function

f (x1 , x2 , x3 ) = 2x21 + 3x1 x3 − 2x3 x2 − x22 + 4x23 (5)

in the form f (x) = xT Ax where x ∈ <3 . Either prove that A is unique and provide the
corresponding A, or show that it is non-unique by providing two alternative matrices
for A.

7. On Gradients....
Let f : <n → < be a differentiable multivariate function. Its gradient (at point x) is
defined as the vector

 ∂f 
 ∂x 1
 ∂f


 ∂x2
∇f (x) =  . .

(6)
 .. 
 
∂f
∂xn

where the partial derivatives are evaluated at x.

Find the explicit gradients of the following functions (using matrix notation only)

(a) f (x) = aT x + b for some a ∈ <n and b ∈ <.

(b) f (x) = xT Ax for some A ∈ <n×n .

2
8. What is a homework without a computational part?....
Given a differentiable function f (x), where x ∈ <n ,

a(x) = f (x0 ) + ∇f (x0 )T (x − x0 ) (7)

is an affine approximation of f (x) around x0 .

Note that this affine function defines an hyperplane in <n+1 in the standard form
" #T " # " #!
∇f (x0 ) x x0
− = 0. (8)
−1 a(x) f (x0 )

Now consider the function

f (x) = xT Ax (9)

defined over <2 , where

" #
0.01 0.001
A= . (10)
0.001 0.01

Please plot the surfaces corresponding to this function and its affine approximation at
h iT
the point x0 = 25 0 as show in Figure below:
Please submit your Matlab/Python/Julia code and your plot (Hint: Yes, you can
shamelessly reuse your drawhyperplane3D function from your previous homework.) In
case of Python/Julia notebooks, please submit a print out of your executed notebook.

3
Figure 1: Affine approximation of a quadratic function.