EE547: Fall 2018

Lecture 5: Normed Spaces

Lecturer: L.J. Ratliff

Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications.
They may be distributed outside this class only with the permission of the Instructor.

5.1 Normed Linear Spaces

Remark 1.1 (Why? ) We need to be able to measure distances between things and hence, we need to
formally introduce a metric onto the linear spaces we consider. This is going to be important for classical
stability notions.

Let the field F be R or C.

Definition 1.1 (Normed Linear Space.) A linear space (V, F ) is said to be a normed linear space if
there exists a map k · k : V → R+ satisfying
1. the ‘triangle inequality’ holds:

kv1 + v2 k ≤ kv1 k + kv2 k, ∀ v1 , v2 ∈ V

2. length under scalar multiplication satisfies

kαvk = |α|kvk, ∀ v ∈ V, α ∈ F

3. the only element with zero length is the zero vector:

kvk = 0 ⇐⇒ v = 0

Remark 1.2 (What do we do with norms? ) A typical engineering use of norms is in deciding
whether or not an iterative technique converges or not—this will be very important in our analysis of
stability.

Definition 1.2 (Convergence.) Given a normed linear space (V, F, k · k) and a sequence of vectors
{vi }∞ ∞
i=1 ⊂ V , we say the sequence (vi )i=1 converges to the vector v ∈ V iff the sequence of non-negative
real numbers kvi − vk tends to zero as i → ∞.

We denote convergence by vi → v or limi→∞ vi = v.

5.1.1 Equivalence of Norms

Two norms k · ka and k · kb on (V, F ) are said to be equivalent if ∃ m` , mu ∈ R+ such that for all v ∈ V

m` kvka ≤ kvkb ≤ mu kvka

It is crucial to note that the same m` and mu must work for all v ∈ V .

5-1
5-2 Lecture 5: Normed Spaces

Fact 1.1 Consider (V, F ) and two equivalent norms k · ka and k · kb . Then, convergence is equivalent—i.e.
i→∞ i→∞
vi −→ v in k · ka ⇐⇒ vi −→ v in k · kb

Remark 1.3 (What is the point of this? ) As far as convergence is concerned, k · ka and k · kb lead
to the same answer, yet in practice the analysis may be more easy in one than the other.

5.1.2 Continuity and Induced Norms

Continuous maps have the desirable property that a small perturbation in x results in a small perturbation
in the operator at that point. They are paramount in the study of robustness and the effect of perturbations
of data.

Definition 1.3 (Continuity.) Let F = R (or C) and consider two normed linear spaces (U, G, k · kU )
and (V, F, k · kV ). Let f be a map (or operator) s.t. f : U → V .
a. (Local continuity). We say f is continuous at u ∈ U iff for every ε > 0, there exists δ > 0 (possibly
depending on ε at u) such that considering points u0 ∈ U

ku0 − ukU < δ =⇒ kf (u0 ) − f (u)kV < ε

b. (Global continuity). We say f is continuous on U iff it is continuous at every u ∈ U .

Definition 1.4 (Induced Operator Norm.) The induced (operator) norm of f is defined to be
 
kf (u)kV
kf k = sup
u6=0 kukU

Let A : U → V be a (continuous) linear operator. Let U and V be endowed with the norms k · kU and k · kV ,
resp. Then, the induced norm of A is defined by

kAukV
kAki = sup
u6=0 kukU

The induced norm kAk is the maximum gain of the map A over all directions; moreover, kAk depends on
the choice of the norms k · kU and k · kV in the domain and co-domain, resp.

Theorem 1.1 (Facts about induced norms.) Let (U, F, k · kU ), (V, F, k · kV ), and (W, F, k · kW ) be
normed linear spaces and let A : V → W , Ã : V → W and B : U → V be linear maps. Note that ∀α ∈ F ,
αA : V → W , A + Ã : V → W and AB : U → W are also linear maps. Then we have that
1. kAvkW ≤ kAkkvkV
2. kαAk = |α|kAk, ∀ α ∈ F
3. kA + Ãk ≤ kAk + kÃk
4. kAk = 0 ⇐⇒ A = 0 (zero map)
5. kABk ≤ kAkkBk

Matrix norms are induced norms or operator norms. Consider A ∈ F m×n , A : F n → F m . The induced
matrix norm is defined by
 
kAxk
kAk = sup {kAxk : x ∈ F n , kxk = 1} = sup : x ∈ F n, x =
6 0
kxk
Lecture 5: Normed Spaces 5-3

With this definition, we can show the following

• 1–norm (column sums)
m
X
kAki,1 = max |aij |
1≤j≤n
i=1

• 2–norm (largest singular value—we will discuss singular values later)

p
kAki,2 = λmax (A∗ A) = σmax (A)

• Forbenius norm
 1/2
m X
X n
kAki,F =  |aij |2 
i=1 j=1

Note: kAk2 ≤ kAkF

• ∞–norm (row sums)
n
X
kAki,∞ = max |aij |
1≤i≤m
j=1

5.2 Inner Product Space (Hilbert Spaces)

Let the field F be R or C and consider the linear space (H, F ). The function

h·, ·i : H × H → F [(x, y) 7→ hx, yi]

is called inner product iff

1. hx, y + zi = hx, yi + hx, zi, ∀x, y, z ∈ H
2. hx, αyi = αhx, yi, ∀α ∈ F, ∀x, y ∈ H
3. kxk2 = hx, xi > 0 ⇐⇒ x ∈ H s.t. x 6= 0
4. hx, yi = hy, xi
where the overbar denotes the complex conjugate of hy, xi. Moreover, in (c) k · k is called the norm induced
by the inner product.

Definition 2.1 (Hilbert Space.) A space equipped with an inner product is called a Hilbert space and
denoted by (H, F, h·, ·i).

Example 2.1 The following are examples:

1. (F n , F, h·, ·i) is a Hilbert space under the inner product
n
X
hx, yi = x̄i yi = x∗ y
i=1

2. L2 ([t0 , t1 ], F n ) (space of square, integrable F n -valued functions on [t0 , t1 ]) defined by the inner product
by
t1
Z
hf, gi = f (t)∗ g(t) dt
t0
5-4 Lecture 5: Normed Spaces

Theorem 2.1 (Schwawrz’s Inequality.) Let (H, F, h·, ·i) be an inner product space.

|hx, yi| ≤ kxkkyk ∀ x, y ∈ H

This inequality implies the triangle inequality for the inner product norm kxk2 = hx, xi.

5.2.1 Orthogonality

Definition 2.2 (Orthogonality.) Two vectors x, y ∈ H are said to be orthogonal iff hx, yi = 0.

If two vectors are orthogonal, they satisfy

kx + yk2 = kxk2 + kyk2 (Pythogoras’ Theorem)

Definition 2.3 (Orthogonal Complement.) If M is a subspace of a Hilbert space, the subset

M ⊥ = {y ∈ H| hx, yi = 0, ∀x ∈ M }

is called the orthogonal complement of M .

Fact 2.1 M ∩ M ⊥ = {0}.

5.2.2 Adjoints

Let F be R or C and let (U, F, h·, ·iU ) and (V, F, h·, ·iV ) be Hilbert spaces. Let A : U → V be continuous
and linear.

Definition 2.4 (Adjoint.) The adjoint of A, denoted A∗ , is the map A∗ : V → U such that

hv, AuiV = hA∗ v, uiU

Example 2.2 Let f (·), g(·) ∈ C([t0 , t1 ], Rn ) and define A : C([t0 , t1 ], Rn ) → R by

A : f (·) 7→ hg(·), f (·)i

Find the adjoint map of A.

Lecture 5: Normed Spaces 5-5

5.2.3 Self-Adjoint Maps

Let (H, F, h·, ·iH ) be a Hilbert space and let A : H → H be a continuous linear map with adjoint A∗ : H → H.

Definition 2.5 (Self-Adjoint.) The map A is self-adjoint iff A = A∗ , or equivalently

hx, AyiH = hAx, yiH , ∀x, y ∈ H

Definition 2.6 (Hermitian matrices.) Let H = F n and let A be represented by a matrix A :

n
F →F ,A∈Fn n×n
. Then A is self-adjoint iff the matrix A is Hermitian (i.e. A = A∗ , viz. aij = āij ,
∀i, j ∈ {1, . . . , n}.

Fact 2.2 Let A : H → H be a self-adjoint, continuous linear map. Then

a. all eigenvalues of A are real
b. if λi and λk are distinct eigenvalues of A, then any eigenvector vi associated with λi is orthogonal to
any eigenvector vk associated with λk .

5.2.4 Properties of the Adjoint

Let A be a linear continuous map from U to V (both Hilbert spaces). Then, A∗ is linear and continuous
with induced norm kA∗ k = kAk. Moreover, A∗∗ = A.

Theorem 2.2 (Finite Rank Operator Lemma (FROL).) Let F be R or C. Let (H, F, h·, ·iH ) be a
Hilbert space and consider F m as the Hilbert space (F m , F, h·, ·iF m ). Let A : H → F m be a continuous
5-6 Lecture 5: Normed Spaces

R(A∗ ) R(A)

A∗
0H 0m

N (A) N (A∗ )

H Fm

Figure 5.1: The orthogonal decomposition of the domain and the co-domain of a finite rank operator A :
H → F m and its associated bijections.

linear map with adjoint A∗ : F m → H. Then

A∗ : F m → H, AA∗ : F m → F m , A∗ A : H → H

are continuous linear maps with AA∗ and A∗ A self-adjoint. Furthermore,

⊥ ⊥
a. H = R(A∗ ) ⊕ N (A), F m = R(A) ⊕ N (A∗ )
b. The restriction A|R(A∗ ) is a bijection of R(A∗ ) onto R(A) and

N (AA∗ ) = N (A∗ ), R(AA∗ ) = R(A)

c. The restriction A∗ |R(A) is a bijection of R(A) onto R(A∗ ) and

N (A∗ A) = N (A), R(A∗ A) = R(A∗ )