Professional Documents
Culture Documents
Introduction To Functional Analysis PDF
Introduction To Functional Analysis PDF
and Applications
This textbook includes two parts; the first part presents theory corresponding to
the course whereas the second part presents related applications. The material
is intended for engineering students, not for mathematics students. The author
ignores some material which is considered to be too complicated for engineer-
ing students. Especially in the second part, the author attempts to show the
relationship between theory and engineering applications rather than to show
rigorous proofs for the aforementioned engineering applications. The material
included in this textbook is referenced from documents including books, papers,
etc, listed in bibliography.
Finally, the author would like to thank Assoc. Prof. Vutipong Areekul and
Dr. Phunsak Thiennviboon for providing digital signal processing notes and
adaptive filtering notes, respectively, and Assoc. Prof. Yotsanan Meemark for
advice.
i
ii PREFACE
Contents
Preface i
1 Mathematical Fundamentals 1
1.1 Supremum, Maximum, Infemum and Minimum . . . . . . . . . . 1
1.2 Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Countable and Uncountable Sets . . . . . . . . . . . . . . . . . . 3
1.4 Neighborhood, Points and Sets . . . . . . . . . . . . . . . . . . . 4
1.5 Sequences and Limits . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Vector Spaces 7
iii
iv CONTENTS
5.3 Functionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.4 Hilbert-Adjoint Operators . . . . . . . . . . . . . . . . . . . . . . 53
5.5 Self-Adjoint Operators and Formally Self-Adjoint Operators . . . 57
5.6 Positive Operators . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.7 Compact Operators . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.8 Contraction Mapping . . . . . . . . . . . . . . . . . . . . . . . . . 62
5.9 Minimization of Functionals . . . . . . . . . . . . . . . . . . . . . 64
5.10 Sturm - Liouville Operator . . . . . . . . . . . . . . . . . . . . . 65
5.11 Hilbert-Schmidt Operator . . . . . . . . . . . . . . . . . . . . . . 67
7 Examples of Applications 77
7.1 Finite Element Method . . . . . . . . . . . . . . . . . . . . . . . 77
7.2 Method of Moments . . . . . . . . . . . . . . . . . . . . . . . . . 78
A Matrices 79
B Integrals 81
B.1 Riemann Integral . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
B.2 Riemann-Stieltjes Integral . . . . . . . . . . . . . . . . . . . . . . 81
B.3 Lebesgue Integral . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
D Complex Analysis 87
E Gateaux Differential 91
Chapter 1
Mathematical
Fundamentals
1. α is an upper bound of S.
α “ sup S
α “ max S
1. α is a lower bound of S.
α “ inf S
1
2 CHAPTER 1. MATHEMATICAL FUNDAMENTALS
α “ min S
Solution
It is trivial that an ď 1 for all n P N. In addition, we can show that, for any
c ă 1, there exists n such that an ą c. Therefore, suptan u “ 1.
nPN
Moreover, since
ˆ ˙ ˆ ˙
1 1 1 1
an`1 ´ an “ 1´ ´ 1´ “ ´ ą 0,
n`1 n n n`1
1
Example 2. Let an “ 1 ` , n P N. Find inf tan u and mintan u
n nPN nPN
Solution
It is trivial that an ě 1 for all n P N. In addition, we can show that, for any
c ą 1, there exists n such that an ă c. Therefore, inf tan u “ 1.
nPN
Moreover, since
ˆ ˙ ˆ ˙
1 1 1 1
an`1 ´ an “ 1` ´ 1` “ ´ ă 0,
n`1 n n`1 n
1.2 Mappings
Definition 5 (Functions or Mappings). Consider sets X and Y . Suppose that
for each x P X, there is an associated element, y P Y , which is denoted by f pxq.
Then, f is called a function or mapping from X into Y , we write f : X Ñ Y .
X is called the domain of f whereas ty | y “ f pxq for some x P Xu is called the
range of f .
Definition 6 (1-to-1 Functions or Mappings). Consider a function f : X Ñ Y .
Suppose that, for all x1 , x2 P X, f px1 q “ f px2 q, then x1 “ x2 . Therefore, f is
called a 1 ´ 1 function or mapping.
1.3. COUNTABLE AND UNCOUNTABLE SETS 3
where J “ t1, 2, 3, . . .u
It is obvious that f is a bijective mapping. Therefore, S is countable. In
particular, S is infinitely countable.
Nr pxq “ ty P R | |x ´ y| ă r, r ą 0u
S “ S Y S1
2 The proof for the uniqueness of the limit of a convergent sequence in a metric space pX, dq
is shown in example ??
6 CHAPTER 1. MATHEMATICAL FUNDAMENTALS
Chapter 2
Vector Spaces
5. For all α P F there exists 0 P F such that α`0 “ α and, for all α P F, there
exists 1 P F such that 1.α “ α
Solution Since the set of all real numbers R satisfies the following properties
5. For all α P R there exists 0 P R such that α`0 “ α and, for all α P R, there
exists 1 P R such that 1.α “ α
7
8 CHAPTER 2. VECTOR SPACES
Solution
Since the set of all complex numbers C satisfies the following properties
5. For all α P C there exists 0 P C such that α`0 “ α and, for all α P C, there
exists 1 P C such that 1.α “ α
Solution
Since the set of all integers I does not satisfy the following property (For
all α P I provided that α ‰ 0 there exists γ P F such that α.γ “ 1), it follows
that the set of all integers I is not a field.
1. x ` y P V for x and y P V
2. x ` y “ y ` x for x and y P V
3. px ` yq ` z “ x ` py ` zq for x, y and z P V
9. 1x “ x for x P V
Solution
Since R satisfies the following properties
1. x ` y P R for x and y P R
2. x ` y “ y ` x for x and y P R
3. px ` yq ` z “ x ` py ` zq for x, y and z P R
9. 1x “ x for x P R
Solution
Since C satisfies the following properties
1. x ` y P C for x and y P C
2. x ` y “ y ` x for x and y P C
3. px ` yq ` z “ x ` py ` zq for x, y and z P C
9. 1x “ x for x P C
Solution
Since Rn satisfies the following properties
1. x ` y P Rn for x and y P Rn
10 CHAPTER 2. VECTOR SPACES
2. x ` y “ y ` x for x and y P Rn
3. px ` yq ` z “ x ` py ` zq for x, y and z P Rn
4. For all x P Rn there exists 0 P Rn such that x ` 0 “ x
5. For all x P Rn , there exists ´x P Rn such that x ` p´xq “ 0
6. αpβxq “ pαβqx for α and β P R and x P Rn
7. pα ` βqx “ αx ` βx for α and β P R and x P Rn
8. αpx ` yq “ αx ` αy for α P R and x, y P Rn
9. 1x “ x for x P Rn
it follows that Rn is a vector space over R.
Example 10. Show that, for n P N, Cn is a vector space over C.
Solution
Since Cn satisfies the following properties
1. x ` y P Cn for x and y P Cn
2. x ` y “ y ` x for x and y P Cn
3. px ` yq ` z “ x ` py ` zq for x, y and z P Cn
4. For all x P Cn there exists 0 P Cn such that x ` 0 “ x
5. For all x P Cn , there exists ´x P Cn such that x ` p´xq “ 0
6. αpβxq “ pαβqx for α and β P C and x P Cn
7. pα ` βqx “ αx ` βx for α and β P C and x P Cn
8. αpx ` yq “ αx ` αy for α P C and x, y P Cn
9. 1x “ x for x P Cn
it follows that Cn is a vector space over C.
Example 11. Show that a set of all polynomial functions in the interval ra, bs
is a vector space over R
Solution
Let P ra, bs be a set of all polynomial functions in the interval ra, bs. Since R
satisfies the following properties
1. f ` g P P ra, bs for f and g P P ra, bs
2. f ` g “ g ` f for f and g P P ra, bs
3. pf ` gq ` h “ f ` pg ` hq for f, g and h P P ra, bs
11
α1 “ α2 “ . . . “ αn “ 0.
dimpVq “ cardpSq
v “ x ` y, x P X and y P Y
Y is an algebraic complement of X in V
Definition 35 (Convex Sets). Let S be a subset of a vector space V. S is called
a convex set iff αx1 ` p1 ´ αqx2 for any α P r0, 1s and x1 , x2 P S
14 CHAPTER 2. VECTOR SPACES
Chapter 3
15
16 CHAPTER 3. NORMED SPACES AND BANACH SPACES
|}x1 } ´ }y 1 }| ď }x1 ´ y 1 }
Definition 37 (Equivalent Norms). }.}1 and }.}2 are said to be equivalent iff
for any sequence pxn q and x in a vector space V,
}x ´ xn }1 Ñ 0 as n Ñ 8 iff }x ´ xn }2 Ñ 0 as n Ñ 8
Theorem 4. Let }.}1 and }.}2 be norms in a vector space V. }.}1 and }.}2 are
equivalent iff there exist α ą 0 and β ą 0 such that
Proof
First of all, suppose that there exist α ą 0 and β ą 0 such that α}x}1 ď
}x}2 ď β}x}1 . Clearly, it follows that
}x ´ xn }1 Ñ 0 as n Ñ 8 iff }x ´ xn }2 Ñ 0 as n Ñ 8.
}x ´ xn }1 Ñ 0 as n Ñ 8 iff }x ´ xn }2 Ñ 0 as n Ñ 8.
then, there exist α ą 0 and β ą 0 such that α}x}1 ď }x}2 ď β}x}1 by contra-
diction.
Suppose that there exists no α ą 0 such that
α}x}1 ď }x}2
for all x P V
Therefore, for all n, there exists xn P V such that
1
}xn }1 ą }xn }2
n
if we define
1 xn
yn “ ? ,
n }xn }2
then
1 }xn }2 1
}yn }2 “ ? “ ? Ñ 0.
n }xn }2 n
and thus ?
}yn }1 ě n}y}2 ě n.
It follows that there exists α satisfying the required property. The existence of
β can be proved in a similar way.
Theorem 5. For a finite-dimensional vector space, all norms are equivalent.
Proof
See the proof in [11].
3.1. NORM AND NORMED SPACES 17
n
ÿ
• }x}1 “ |xi |
i“1
}Ax}1
• }A}1 “ max
}x}1 “1 }x}1
}Ax}8
• }A}8 “ max
}x}8 “1 }x}8
}Ax}p
• }A}p “ max ,p ě 1
}x}p “1 }x}p
}An } ď }A}n
Proof
Let xp‰ 0q P Rm . It follows that
}An x} ď }A}}An´1 x}
Consequently,
}An x}
ď }A}n
}x}
That is,
}An x}
}An } “ sup ď }A}n
x‰0 }x}
18 CHAPTER 3. NORMED SPACES AND BANACH SPACES
where an ě 0.
Consequently, if }f }8 exists, }f }8 can be evaluated in the following.
}f }8 “ sup |f pxq|
N
ÿ
“| an xn |
n“0
N
ÿ
ď| |an ||xn |
n“0
“8
Consequently, }f }8 does not exist. That is, the set of all polynomial functions
is not a normed vector space over R equipped with a norm. }.}8 .
Definition 39 (Bounded Sequences). A sequence txn u8 n“1 in a normed space
V is said to be bounded iff there exists M such that }xn } ă M for n “ 1, 2, 3, . . ..
Definition 40 (Cauchy Sequences). Let txn u8 n“1 be a sequence in a vector space
V equipped with norm }.}. Therefore, txn u8 n“1 is a Cauchy sequence iff, for all
ą 0. there exist N ą 0 such that, for all m ą N and n ą N , }xm ´ xn } ă
Definition 41 (Convergent and Divergent Sequences). Let txn u8 n“1 be a se-
quence in a vector space V equipped with norm }.}. Therefore, txn u8 n“1 is a
convergent sequence iff, for all ą 0, there exist N ą 0 such that, for all
n ą N , there exists a fixed x P V such that }x ´ xn } ă . Moreover, x is called
a limit of sequence txn u8n“1 . In addition, a sequence which is not convergent is
said to be a divergent sequence.
The limit mentioned in definition 41 can be proved to be unique in the
following theorem.
Theorem 6. If x is a limit of convergent sequence txn u8
n“1 then x is unique.
Proof. Suppose that x and x1 are both limits of convergent sequence txn u8
n“1 .
1
By definition, for all “ ą 0, there exists an integer N ą 0 such that
2
}xn ´ x} ă 1 and }xn ´ x1 } ă 1
3.1. NORM AND NORMED SPACES 19
then txn u8
n“1 is called “(Schauder) basis”.
Proof
Suppose that txn u8n“1 is a convergent sequence. Therefore, by definition,
there exists N ą 0 such that
}x ´ xn } ă
for all ą 0
Since
}xn } ´ }x} ď }x ´ xn },
we obtain,for all ą 0, there exists N ą 0 such that, for all n ą N ,
for all ą 0
That is, txn u8
n“1 is bounded.
20 CHAPTER 3. NORMED SPACES AND BANACH SPACES
Particularly, for p “ 1,
Definition 49 (l1 space). l1 is defined by
8
ÿ
tx “ pxi q8
i“1 | |xi | ă 8u
i“1
and for p “ 2
Definition 50 (l2 space). l2 is defined by
8
ÿ
tx “ pxi q8
i“1 | |xi |2 ă 8u
i“1
It should be noted that the essential supremum of f is the smallest value that
bound f almost everywhere.
Proof
Since a closed subspace of a Banach space is a closed set. Therefore, by
definition, every convergent sequence in a closed subspace has a limit point in
itself. That is, a closed subspace of a Banach space is complete. Hence, it is a
Banach space.
3.3 Exercise
1. Show that lp , p ě 1 is a Banach space.
2. Show that l8 is a Banach space.
25
26 CHAPTER 4. INNER PRODUCT SPACES AND HILBERT SPACES
Proof
Since txn u8 8
n“1 and tyn un“1 be convergent sequences in H such that xn Ñ x
as n Ñ 8 and yn Ñ y as n Ñ 8, by triangular inequality, we obtain
Consequently,
Proof Note that x is any vector in a vector space V equipped with an inner
product x¨, ¨y and }x} is defined by the following equation
1
}x} “ xx, xy 2
2. By the definition of the inner product, we obtain xx, xy ě 0. That is, }x} ě
0
3. Let x and y be any vectors in the vector space V. Therefore,
}x ` y}2 “ xx ` y, x ` yy
“ xx, x ` yy ` xy, x ` yy
“ xx ` y, xy ` xx ` y, yy
“ xx, xy ` xy, xy ` xx, yy ` xy, yy
“ xx, xy ` xx, yy ` xx, yy ` xy, yy
Since xx, xy “ xx, xy, xy, yy “ xy, yy and xx, yy “ xx, yy,
}x ` y}2 “ xx, xy ` xx, yy ` xx, yy ` xy, yy
Since 2< pxx, yyq “ xx, yy ` xx, yy, }x}2 “ xx, xy and }y}2 “ xy, yy,
}x ` y}2 “ }x}2 ` 2< pxx, yyq ` }y}2
Since < pxx, yyq ď |xx, yy|,
}x ` y}2 “ }x}2 ` 2< pxx, yyq ` }y}2
ď }x}2 ` 2 |xx, yy| ` }y}2
By Cauchy-Schwarz Inequality C.1, we obtain
}x ` y}2 ď }x}2 ` 2}x}}y} ` }y}2
2
“ p}x} ` }y}q
That is,
}x ` y} ď }x} ` }y}
1
4. Let x be a vector in the vector space V. Therefore, }x} “ xx, xy 2 “
0 iff x “ 0.
28 CHAPTER 4. INNER PRODUCT SPACES AND HILBERT SPACES
8
ÿ
1 1
and induced norm defined by }x}2 “ xx, xy 2 “ p |xi |2 q 2 . It follows that
i“1
8
ÿ 1
}x}2 “ p |xi |2 q 2
i“1
Definition
“ ‰ 59 (Space of Square Lebesgue Integrable Functions defined“ ‰ on
a, b ). Spaces of square Lebesgue integrable functions are defined on a, b as
" ˇż b *
“ ‰
L2 p a, b q “ f ˇˇ |f pxq|2 dx ă 8
ˇ
a
where
¨ ˛ 21
ÿ
}f }H k “ ˝ }Dα f }2L2 pRq ‚
|α|ďk
Example 15. Let V “ Cr0, 1s be a vector space over R equipped with the inner
product x., .y defined by
ż1
xf, gy “ f pxqgpxq dx.
0
It follows that ż1
}fn ´ fm }2 “ pfn pxq ´ fm pxqq2 dx Ñ 0
0
as m, n Ñ 8
That is, tfn u is a Cauchy sequence.
30 CHAPTER 4. INNER PRODUCT SPACES AND HILBERT SPACES
# “ ‰
0, x P 0, 21
f“ ` ‰
1, x P 21 , 1
and
# “ ‰
0, x P 0, 21
f“ “ ‰
1, x P 12 , 1
where α is the field (either R or C) associated with the inner product space
V and tu1 , u2 , . . . , uN u is an orthonormal basis of the finite-dimensional inner
product space V. Therefore, vn and vm can be written as
N
ÿ N
ÿ
vn “ αi,n ui and vm “ αi,m ui , respectively.
i“1 i“1
4.2. COMPLETION AND HILBERT SPACES 31
Consequently,
ÿN ÿN
}vn ´ vm } “
αi,n ui ´ αi,m ui
i“1 i“1
ÿN
“
pαi,n ui ´ αi qui,m
i“1
g
fN
fÿ
“ e |αi,n ´ αi,m |2 }ui }2
i“1
g
fN
fÿ
“ e |α i,n ´ αi,m |2
i“1
and
|αi,n ´ αi,m | ď }vn ´ vm } ď
That is, the sequence tai,n u8n“1 is also a Cauchy sequence. By using the fact
the field F (either R or C) is complete, that is, for some αi for all 1 “ ą 0,
N
there exists an integer N ą 0 such that
|αi,n ´ αi | ă 1 “
N
N
ÿ
By letting v “ αi ui , we obtain
i“1
ÿN N
ÿ
}vn ´ v} “
αi,n ui ´ αi ui
i“1 i“1
N
ÿ
ď
pαi,n ui ´ αi qui,m
i“1
N
ÿ
“ |αi,n ui ´ αi |
ui,m
i“1
N
ÿ
“ |αi,n ui ´ αi |
i“1
ă
where “ N 1 “ N
N
That is, for all 1 “ ą 0, there exists an integer N ą 0 such that
N
}vn ´ v} ă
32 CHAPTER 4. INNER PRODUCT SPACES AND HILBERT SPACES
Consequently, for all n ă N . That is, the Cauchy sequence tvn u8 n“1 is also a
convergent sequence. That is, the finite-dimensional inner product space V is
complete. This proves the fact that every finite dimensional inner product space
is a Hilbert space.
Solution
n ÿ
ÿ n
xA, By “ aij bij
i“1 j“1
ÿn ÿ n
“ aji bji
j“1 i“1
ÿn ÿ n
“ aji bji
i“1 j“1
(4.1)
By using the fact that A and B are symmetric and antisymmetric, respectively.
That is,
aji “ aij and bji “ ´bij
Consequently,
n ÿ
ÿ n n ÿ
ÿ n
aij bij “ ´ aij bij
i“1 j“1 i“1 j“1
and, then,
n ÿ
ÿ n
xA, By “ aij bij “ 0
i“1 j“1
Moreover,
xαx ` βy, zy “ αxx, yy ` βxy, zy “ 0
for α, β P F
That is, S K is a subspace of H
Next, prove that S K is closed. Let txn u be a Cauchy sequence such that
xn Ñ x for some x P H. By using theorem 11 (Continuity of Inner peoduct), it
follows that
xxn , yy Ñ xx, yy
for any y P S
Moreover, since
xxn , yy “ 0,
it follows that
xx, yy “ 0
K
That is, S is closed.
Lemma 5. Let S ‰ H be a subspace of a Hilbert space H. Therefore,
S X S K “ t0u
Proof
Let x P S X S K . That is, x P S and x P S K . It follows that, by definition of
orthogonal complement,
xx, xy “ 0
and then
x“0
Therefore, S X S K “ t0u
Lemma 6. Let S ‰ H be a subset of a Hilbert space H. spantSu is dense in H
iff S K “ t0u
Proof
We first prove that if spantSu is dense in H, then S K “ t0u. Let x P S K .
Since spantSu is dense in H, that is, spantSu “ H, x P spantSu and there exists
a sequence txn u8
n“1 P spantSu such that lim xn “ x. In addition, since x P S
K
nÑ8
and S K KspantSu, we obtain
xx, xn y “ 0
34 CHAPTER 4. INNER PRODUCT SPACES AND HILBERT SPACES
Consequently,
xx, xy “ }x}2 “ 0
That is, x “ 0. Therefore, S K “ t0u.
Conversely, we prove that if S K “ t0u, then spantSu is dense in H (or
spantSu “ H). Suppose that S K “ t0u. Let xKspantSu. It implies that xKS.
Consequently, x P S K and thus x “ 0 and thus spantSuK “ t0u. It follows that
K
spantSu “ t0u. Since spantSu is a closed subspace of H, we obtain
à K
H “ spantSu spantSu
K
Since spantSu “ t0u, we obtain
à
H “ spantSu t0u
Proof
At first, we need to prove the existence of best approximation. Let tvn u8
n“1
be a sequence in S such that
dn Ñ d (4.2)
or
d “ lim dn
nÑ8
where dn “ }v ´ vn } and d “ }v ´ v0 }
By setting δn “ vn ´ v, we obtain
dn “ }δn }
and
vn ` vm
Since S is convex, P S. Moreover, we have vn ´ vm “ δn ´ δm . Also,
2
by parallelogram law, we obtain
}vn ´ vm }2 “ }δn ´ δm }2
` ˘
“ ´}δn ` δm }2 ` 2 }δn }2 ` }δm }2
ď ´p2dq2 ` 2pd2n ` d2m q
}v ´ v0 } ě }v ´ vn } ` }vn ´ v0 } “ dn ` }vn ´ v0 } Ñ d
it implies }v ´ v0 } “ d.
Next, we need to prove the uniqueness of the best approximation. Suppose
that v0 , v01 P S satisfy
}v ´ v0 } “ d and }v ´ v01 } “ d
36 CHAPTER 4. INNER PRODUCT SPACES AND HILBERT SPACES
By parallelogram law,
v0 ` v01
Since S is convex, P S. It follows that
2
1
v ´ v0 ` v0
ě d
2
That is,
}v0 ´ v01 } ď 0
However, by the definition of norms, we have
}v0 ´ v01 } ě 0
That is,
}v0 ´ v01 } “ 0 or v0 “ v01
Theorem 19 (Direct Sum). Let M be a closed subspace of a Hilbert space H.
Therefore, à K
H“M M
That is, for every x P H, x is uniquely represented by
x“y`z
where y P M and z P M K
Proof
It is trivial that, for x P M , x “ x ` 0. Suppose that x R H. Further suppose
that x can be both represented by
x “ y1 ` z1
and
x“y`z
where y, y 1 P M and z, z 1 P M K By recognizing that M is a closed subspace of
H, it follows that M is a closed convex subset of H. Since M is a closed convex
subset of H, by using theorem 18, we can choose y such that
}x ´ y} “ inf
1
}x ´ y 1 }
y PM
xy ´ y 1 , z ´ z 1 y “ 0
4.4. BEST APPROXIMATION AND ORTHOGONAL PROJECTION 37
xy ´ y 1 , y ´ y 1 y “ 0
Consequently,
y “ y1 and z “ z 1
That is, x can be uniquely represented by
x“y`z
where y P M and z P M K
Definition 67 (Orthogonal Projection). Let M be a closed subspace of a Hilbert
space H. The orthogonal projection of H onto M is the function PM : H Ñ H
such that for all x P H such that
x ´ PM pxqKM
α1 PM px1 q ` α2 PM px2 q P M.
and
pα1 x1 `α2 x2 q´pα1 PM px1 q`α2 PM px2 qq “ α1 px1 ´PM px1 qq`α1 px1 ´PM px1 qq
and
α1 px1 ´ PM px1 qq ` α1 px1 ´ PM px1 qq P M K
That is,
PM px1 ` x2 q “ PM px1 q ` PM px2 q
Consequently,
2
PM pxq ´ PM pPM pxqq “ PM pxq ´ PM pxq “ y ´ PM pyqKM
38 CHAPTER 4. INNER PRODUCT SPACES AND HILBERT SPACES
and then
y ´ PM pyq P M K
2
Moreover, since y “ PM pxq P M and PM pxq “ PM pyq P M , PM pxq ´
2
PM pxq P M . That is,
2
PM pxq ´ PM pxq P M X M K
2
By lemma 5, PM pxq ´ PM pxq “ 0. Therefore, for all x P H,
2
PM pxq “ PM pxq
That is,
2
PM “ PM
Theorem 21. Let M be a closed subspace of a Hilbert space H and ten u be an
orthonormal basis of M . Therefore, the projection of x onto S is given by
ÿ
PM pxq “ xx, en yen
n
Proof
We can separate the proof into two cases:
1. M is of finite dimension
For x P H, ÿ ÿ
x “ xx, en yen ` px ´ xx, en yen q
n n
ÿ ÿ
It is obvious that xx, en yen P S and x ´ xx, en yen P M K
À K n n
Since M M , there uniquely exist y P M and y 1 P M K . That is,
x “ y ` y1
Consequently, ÿ
y“ xx, en yen
n
2. M is of infinite dimension
For x P H, ÿ ÿ
x“ xx, en yen ` px ´ xx, en yen q
nďN nďN
Since M is closed, as N Ñ 8,
ÿ
xx, en yen P M
nďN
À
Since H “ M M K , there uniquely exist y P M and y 1 P M K . That is,
x “ y ` y1
Consequently, ÿ
y“ xx, en yen
n
4.4. BEST APPROXIMATION AND ORTHOGONAL PROJECTION 39
8
ÿ
x“ xx, xn yxn
n“1
N
ÿ N
ÿ
}x ´ xx, φn yφn }2 “ }x}2 ´ |xx, φn y|2 (4.3)
n“1 n“1
and
8
ÿ
|xx, φn y|2 ď }x}2 for any x P H
n“1
Proof
N
ÿ
0 ď }x ´ xx, φn yφn }2
n“1
N
ÿ
“ }x}2 ´ 2Re pxx, φn yxφn , xyq ` |xx, φn y|2
n“1
N
ÿ
“ }x}2 ´ |xx, φn y|2
n“1
N
ÿ
|xx, φn y|2 ď }x}2
n“1
By taking N Ñ 8, we obtain
8
ÿ
|xx, φn y|2 ď }x}2
n“1
40 CHAPTER 4. INNER PRODUCT SPACES AND HILBERT SPACES
Proof
For x P H, by using equation 4.3, we obtain
N
ÿ N
ÿ
}x ´ xx, φn yφn }2 “ }x}2 ´ |xx, φn y|2
n“1 n“1
and, hence,
N
ÿ
|xx, φn y|2 Ñ }x}2 as N Ñ8
n“1
It follows that
8
ÿ
}x}2 “ |xx, φn y|2 for any x P H
n“1
Conversely, if
8
ÿ
}x}2 “ |xx, φn y|2 for any x P H
n“1
4.6. GRAM-SCHMIDT ORTHONONORMALIZATION OR PROCESS 41
then
N
ÿ
}x}2 ´ |xx, φn y|2 Ñ 0 as N Ñ8
n“1
2
ÿN
ÿN
Since
xx, φn yφn
“ |xx, φn y|2 , we have
n“1
n“1
2
ÿN
2
xx, φn yφn
Ñ 0 as N Ñ8
}x} ´
n“1
and thus
2
ÿN
x ´ xx, xn yxn
Ñ 0 as N Ñ8
n“1
4.7 Exercise
1. Show that l2 is a Hilbert space.
2. Show that L2 pRq is a Hilbert space.
42 CHAPTER 4. INNER PRODUCT SPACES AND HILBERT SPACES
Chapter 5
5.1 Operators
Definition 70 (Operators). An operator is a mapping of a vector space1 into
a vector space2 .
Definition 71 (Domain and Range). Let T : V Ñ W be an operator. The
domain of T denoted by DpT q is defined as
DpT q “ tx | T pxq is defined u
That is,
DpT q “ V
The range of T denoted by RpT q is defined as
RpT q “ tT pxq | T pxq is defined u
It should be noted that RpT q can possibly be W or a subset of W
Definition 72 (Linear Operators). Let T : V Ñ W be an operator (mapping
from a vector space V into a vector space W). T is said to be a linear operator
iff
1. T pu ` vq “ T puq ` T pvq for u, v P V
2. T pαuq “ αT puq for α P F and u P V
Definition 73 (Isomorphism). Let T : V Ñ W be a bijective linear opera-
tor where V and W are inner product spaces over the same field. T is called
isomorphism iff
xT x, T yyW “ xx, yyV for all x, y P V and T x, T y P W
where x., .yV and x., .yW are inner products over V and W, respectively.
1 in particular, a normed space or inner product space
2 in particular, a normed space or inner product space
43
44 CHAPTER 5. OPERATORS AND FUNCTIONALS
Example 17. Let T : V Ñ V1 where V onto V1 are R2 over the same field R.
T is defined as
ˆ„ ˙ „ „
x cos θ ´ sin θ x
T “
y sin θ cos θ y
and inner product x., .y is defined as
B„ „ F
x1 x
, 2 “ x1 x2 ` y1 y2 .
y1 y2
Solution
ˆ„ ˙ „ „
x1 cos θ ´ sin θ
x1
T “
y1 sin θ cos θ
y1
„
x cos θ ´ y1 sin θ
“ 1
x1 sin θ ` y1 cos θ
and
ˆ„ ˙ „ „
x2 cos θ ´ sin θ x2
T “
y2 sin θ cos θ y2
„
x2 cos θ ´ y2 sin θ
“
x2 sin θ ` y2 cos θ
Consequently,
B ˆ„ ˙ ˆ„ ˙F B„ „ „ „ F
x1 x2 cos θ ´ sin θ x1 cos θ ´ sin θ x2
T ,T “ ,
y1 y2 sin θ cos θ y1 sin θ cos θ y2
B„ „ F
x1 cos θ ´ y1 sin θ x cos θ ´ y2 sin θ
“ , 2
x1 sin θ ` y1 cos θ x2 sin θ ` y2 cos θ
“ x1 y1 pcos2 θ ` sin2 θq ` x2 y2 pcos2 θ ` sin2 θq
In addition,
B„ „ F
x1 x
, 2 “ x1 y1 ` x2 y2
y1 y2
Therefore,
B„ „ F B ˆ„ ˙ ˆ„ ˙F
x1 x x1 x2
, 2 “ T ,T
y1 y2 y1 y2
5.1. OPERATORS 45
}T pxq}
Example 18. Let T be a bounded linear operator. Prove that }T } “ sup
xPDpT q }x}
x‰0
satisfies the definition of norms.
Solution We can show that }T } satisfies the definition of norms as follows:
1. }αT } “ |α|}T } for all α P F and x P V By using the fact that
|αT pxq| “ |α||T pxq|
Consequently,
|αT pxq| |T pxq|
“ |α| ,x ‰ 0
|x| |x|
2. (Non-negativity) }T } ě 0 for all T P V
3. (Triangular Inequality) }x ` y} ď }x} ` }y} for all x, y P V
4. (Non-degeneracy) }x} “ 0 iff x “ 0
Definition 76 (Continuous Operator). Let T : V Ñ W where V and W are
normed spaces equipped with }.}V and }.}W , respectively. Then, T is continuous
at x “ x0 iff, for all ą 0, there exists δ ą 0 such that
}x ´ x0 }V ă δ Ñ }T pxq ´ T px0 q}W ă
Moreover, if f is continuous at every x0 P V, then f is said to be continuous
3 The proof is left in example 18
46 CHAPTER 5. OPERATORS AND FUNCTIONALS
for all x, y P V
Theorem 26. A bounded linear operator is continuous.
Proof
Let T : V Ñ W where V and W are normed spaces equipped with }.}V and
}.}W , respectively be a linear operator. It follows that
Proof
Let a sequence txn u8 n“1 converge to a fixed x. That is, x “ nÑ8
lim xn . There-
fore, for all ą 0, there exists δ ą 0 such that
0pxq “ 0
for all x P V
5.2. KERNELS, NULL SPACES AND RANGE SPACES 47
kerpT q “ tv P V | T pvq “ 0W u
Solution
T pu ` vq “ T puq ` T pvq “ 0 ` 0 “ 0
Therefore, u ` v P kerpT q.
T pαuq “ αT puq “ α0 “ 0
Therefore, αu P kerpT q
N pT q “ tx | T pxq “ 0u
4 For a linear operator T , the null space of T is actually the kernel of T . It follows that
N pT q is a subspace of V
48 CHAPTER 5. OPERATORS AND FUNCTIONALS
RpT q “ tT pxq | x P Vu
Consequently,
for α1 , α2 P F
Since α1 x1 ` α2 x2 P V, α1 y1 ` α2 y2 “ T pα1 x1 ` α2 x2 q P RpT q. That is, RpT q
is a subspace of W.
Theorem 29. Let T : H Ñ H be a bounded linear operator. It follows that
Proof
Let x P RpT q. It follows that there exists y P H such that
x “ T pyq
For all z P N pT ‹ q,
That is,
xx, zy “ 0
Consequently, RpT q Ă pN pT ‹ qqK . Since, by theorem 15, N pT ‹ q is closed,
RpT q Ă pN pT ‹ qqK
Therefore,
xy, T ‹ pxqy “ 0
for all y P H
It follows that
T ‹ pxq “ 0
5.3. FUNCTIONALS 49
and
pRpT qqK Ă N pT ‹ q
Consequently, by taking orthogonal complement,
N pT ‹ qK “ RpT q
and thus
N pT q “ RpT ‹ qK
5.3 Functionals
Definition 84 (Functionals). A functional is an operator whose range lies in
R or C
3. spαx, yq “ αspx, yq
spx, yq spx, yq
}s} “ sup “ sup
x‰0 }x}}y} }x}“1 }x}}y}
y‰0 }y}“1
Proof
First of all, prove that there exists a z P H such that f pxq “ xx, zy. It is
trivial for a bounded linear functional f pxq “ 0 for all x P H that the functional
f can be of the form
f pxq “ xx, zy
where z “ 0
For a bounded linear functional f such that f pxq ‰ 0 for some x P H, RpT q Ă H
contains a z0 ‰ 0. Let
y “ f pxqz0 ´ f pz0 qx
We obtain
xx, z0 y f pz0 q
f pxq “ f pz0 q “ xx, z0 y
xz0 , z0 y xz0 , z0 y
f pz0 q
By setting z “ z0 , we obtain
xz0 , z0 y
f pxq “ xx, zy
Secondly, prove that z is unique. Suppose that there exist z and z 1 such that
It implies that
xx, z ´ z 1 y “ 0
It follows that, by setting x “ z ´ z 1 , we obtain
xz ´ z 1 , z ´ z 1 y “ 0
and then
z “ z1
That is, by contradiction, z is unique.
Thirdly, }f } “ }z} where }f } “ sup |f pxq|. It is trivial for f “ 0. Suppose
}x}“1
that f ‰ 0. We have
f pxq “ xx, zy
By setting x “ z,
|xz, zy| “ }z}2 “ |f pzq|
5.3. FUNCTIONALS 51
and then
}f } ď }z}
That is,
}f } “ }z}
Theorem 31 (Riez Representation Theorem for Sesquilinear Forms). Let s :
V ˆ W Ñ F be a bounded sesquilinear form. Then s is of the form
spx, yq “ xSx, yy
}S} “ }s}
Proof
For a fixed x, we can define a functional f as
f pyq “ spx, yq
f pyq “ xy, zy
That is,
spx, yq “ xy, zy
or
spx, yq “ xz, yy
where z P W is unique but z depends on x.
Suppose that z “ Sx. We have
spx, yq “ xSx, yy
52 CHAPTER 5. OPERATORS AND FUNCTIONALS
|spx, yq|
|spx, y| “ sup
x‰0 }x}}y}
y‰0
|xSx, yy|
“ sup
x‰0 }x}}y}
y‰0
|xSx, Sxy|
ě sup
x‰0 }x}}Sx}
}Sx}
“ sup
x‰0 }x}
“ }S}
|spx, yq|
}s} “ sup
x‰0 }x}}y}
y‰0
|xSx, yy|
“ sup
x‰0 }x}}y}
y‰0
||xSx, Sxy|
ě sup
x‰0 }x}}Sx}
Sx‰0
|}Sx}
“ sup
x‰0 }x}
“ sup }Sx}
}x}‰1
“ }S}
5.4. HILBERT-ADJOINT OPERATORS 53
|xSx, yy|
“ sup
x‰0 }x}}y}
y‰0
}Sx}}y}
ď sup
x‰0 }x}}y}
y‰0
}Sx}
“ sup
x‰0 }x}
“ sup }Sx}
}x}“1
It follows that
}s} “ }S}
Finally, we prove that S is unique. Suppose that there exists a bounded linear
operator T : V Ñ W such that
spx, yq “ xSx, yy “ xT x, yy
for all x P V and y P W
It follows that
xSx ´ T x, yy “ 0
and then
Sx “ T x
for all x P V
It implies that S “ T and then S is unique.
By Cauchy-Schwarz inequality,
|spx, yq| “ |xx, T yy| ď }x}}T y}
By using the fact that
}T } ď }T }}y},
it implies that
|spx, yq| ď }T }}x}}y}
and then
}s} ď }T }
Moreover, by replacing x with T y
x y
}s} “ sup |sp , q|
x‰0 }x} }y}
y‰0
1
“ sup |xx, T yy|
x‰0 }x}}y}
y‰0
1
ě sup |xT y, T yy|
T y‰0 }T y}}y}
y‰0
1
“ sup }T y}
y‰0 }y}
“ sup }T y}
}y}“1
“ }T }
5 sometimes called ”Adjoint operators”
5.4. HILBERT-ADJOINT OPERATORS 55
It follows that
}s} “ }T }
It follows directly from definition 88 that
2. pL‹ q‹ “ L
3. pαLq‹ “ αL‹
Proof
1.
That is,
pL1 ` L2 q‹ “ L‹1 ` L‹2
2.
xLx, yy “ xx, L‹ yy
“ xL‹ y, xy
“ xy, pL‹ q‹ xy
“ xpL‹ q‹ x, yy
It follows that
xLx, yy ´ xpL‹ q‹ x, yy “ 0
and
xpL‹ q‹ x ´ Lx, yy ´ x, yy “ 0
It implies that
That is,
pL‹ q‹ “ L
56 CHAPTER 5. OPERATORS AND FUNCTIONALS
3.
xpαLqx, yy “ αxLx, yy
“ αxx, L‹ yy
“ αxL‹ y, xy
“ αxL‹ y, xy
“ xαL‹ y, xy
“ xx, αL‹ yy
(5.4)
Moreover, we have
xpαLqx, yy “ xx, pαLq‹ yy (5.5)
It implies that
xx, αL‹ yy “ xx, pαLq‹ yy
and then
xx, pαLq‹ y ´ αL‹ yy “ 0 (5.6)
It follows that
pαLq‹ y ´ αL‹ y “ 0
or
pαLq‹ “ αL‹
4.
xpL1 L2 qx, yy “ xpL1 pL2 xq, yy
“ xL2 x, L‹1 yy
“ xx, L‹2 L‹1 yy
(5.7)
Moreover, we have
xpL1 L2 qx, yy “ xx, pL1 L2 q‹ yy
It implies that
xx, pL1 L2 q‹ y ´ L‹2 L‹1 yy “ 0
(5.8)
It follows that
pL1 L2 q‹ y “ L‹2 L‹1 y
or
pL1 L2 q‹ “ L‹2 L‹1
5.5. SELF-ADJOINT OPERATORS AND FORMALLY SELF-ADJOINT OPERATORS57
That is,
“ ‰b
xLu, vy “ vpxqu1 pxq ´ upxqv 1 pxq a ` xu, Lvy
b
It follows that rvpxqu1 pxq ´ upxqv 1 pxqsa “ 0 for all u P L iff
That is, L is formally self-adjoint but not self-adjoint since DpLq ‰ DpL‹ q
dupxq
Lupxq “ .
dx
Consider a boundary value problem
Lupxq “ 0
Solution
Consider that
B F
du
xLu, vy “ ,v
dx
żb
dupxq
“ vpxq dx
a dx
ż x“b
“ vpxq dupxq
x“a
ż x“b
b
“ rvpxqupxqsa ´ upxqdvpxq
x“a
That is,
b
xLu, vy “ rvpxqupxqsa ´ xu, Lvy (5.11)
for all u P DpLq. That is, L is neither self-adjoint nor formally self-adjoint.
5.6. POSITIVE OPERATORS 59
xT1 x, xy “ xT2 x, xy
xpT1 ´ T2 qx, xy ě 0
and thus
xT1 x, xy ě xT2 x, xy
Therefore, we write T1 ě T2
1. T ‹ T is positive.
2. T T ‹ is positive.
Proof
Let x P H. It follows that
1. xT ‹ T x, xy “ xT x, T xy “ }T x}2 ě 0
That is, T ‹ T is positive.
2. xT T ‹ x, xy “ xT ‹ x, T ‹ xy “ }T ‹ x}2 ě 0
That is, T T ‹ is positive.
Proof
Let S “ Bp0, q “ tx | }x} ă u, ą 0. Since Bp0, q is bounded and T
is compact, there exists a sequence tT pxn qu8
n“1 which contains a convergence
subsequence. Suppose that T pSq is unbounded. Then, there exists a strictly
monotone and unbounded sequence tT pxn qu8 8
n“1 . It implies that tT pxn qun“1
does not contain a convergence subsequence. By contradiction, for all ą 0
T pSq is bounded. That is, T is bounded. It follows that, by theorem 26, T is
continuous.
Theorem 36. Let tTn u8 n“1 be a convergent sequence of compact operators on
a Banach Space V. If Tn Ñ T as n Ñ 8, T is also a compact operator.
Proof
Let txn u8n“1 be a bounded sequence in V. Since T1 is a compact operator
on a Banach Space V, it implies that there exists a subsequence tx1,n u8 n“1 of
txn u8
n“1 such that tT px qu
1 1,n n“1
8
is a convergent subsequence. Moreover, since
tTn u8
n“1 is a convergent sequence of compact operators on a Banach Space V,
it implies that there exists a subsequence txk,n u8
n“1 , k “ 2, 3, . . . of tx u 8
k´1,n n“1
such that tTk pxk,n qu8
n“1 is a convergent subsequence. If we let
xan “ xn,n ,
such that
}T pxan q ´ Tk pxan q} ď
3
for all n ą k 1 .
It implies that, for all ą 0, there exist k such that
}T pxan q ´ T pxam q} ď }T pxan q ´ Tk pxan q} ` }Tk pxan q ´ Tk pxam q} ` }Tk pxam q ´ T pxam q}
ă ` `
3 3 3
“
for all n, m ą k
That is, tT pxan qu8
n“1 is also convergent and thus T is also compact.
Proof
1. Since dimpVq is finite, for a bounded sequence txn u8
n“1 , xn can be written
in terms of e1 , e2 , . . . , eN as
N
ÿ
xn “ αi,n ei
i“1
Consequently, by linearity,
N
ÿ N
ÿ
}T pxq ´ T pxan q} “ } αi T pei q ´ αi,an T pei q}
i“1 i“1
N
ÿ
“} pαi ´ αi,an qT pei q}
i“1
N
ÿ
ď| pαi ´ αi,an q}T pei q}
i“1
N
ÿ
ď |αi ´ αi,an |}T pei q}
i“1
for all n ą N 1
That is, for every bounded sequence txn u8 8
n“1 , tT pxn qun“1 has a convergent
subsequence. Therefore, T is compact.
62 CHAPTER 5. OPERATORS AND FUNCTIONALS
By triangular inequality,
N
ÿ N
ÿ N
ÿ
}T pxn q} “ } αi,n T pei q} ď |αi,n |}T pei q} ď |αi,n |}T }}ei }
i“1 i“1 i“1
Since
N
ÿ
|αi,n |}ei } “ }αi,n ei } ď } αi,n ei } ď }xn }
i“1
8
and txn u8
n“1 is bounded, it implies that tαi,n un“1 is bounded. By Bolzano-
Weierstrass theorem, tαi,n un“1 has a convergent subsequence tαi,an u8
8
n“1 .
Consequently, for all 1 ą 0, there exists N 1 such that
|αi ´ αi,an | ă 1
for all n ą N 1 .
It implies that, for all “ 1 }T }maxt}ei }u ą 0, there exists N 1 such that
N
ÿ N
ÿ N
ÿ
}T pxq´T pxan q} “ } αi T pei q´ αi,an T pei q} ď |αi ´αi,an |}T }}ei } ă
i“1 i“1 i“1
for all n ą N 1 .
That is, tT xn u8 n“1 has a convergent subsequence. That is, T is compact.
for all x, y P V
5.8. CONTRACTION MAPPING 63
xn`1 “ T xn , n ě 0
Consequently,
xn “ T n px0 q
where T n is the nth iteration of T .
Consequently, for n ě m ě 1,
}xn ´ xm }V “ }T n px0 q ´ T m px0 q}V
By using }T pxq ´ T pyq}V ď c}x ´ y}V , we obtain
}T n px0 q ´ T m px0 q}V ď cm }T n´m px0 q ´ x0 }V
That is,
}xn ´ xm }V ď cm }T n´m px0 q ´ x0 }V
By using triangular inequality and }T pxq ´ T pyq}V ď c}x ´ y}V , it follows that
n´m´1
ÿ
}T n´m px0 q ´ x0 }V ď }T i`1 px0 q ´ T i px0 q}V
i“0
n´m´1
ÿ
ď ci }T px0 q ´ x0 }V
i“0
n´m´1
ÿ
ď ci }x1 ´ x0 }V
i“0
Consequently,
}xn ´ xm }V ď cm }T n´m px0 q ´ x0 }V
n´m´1
ÿ
ď cm ci }x1 ´ x0 }V
i“0
8
ÿ
ď cm ci }x1 ´ x0 }V
i“0
(5.12)
By definition 94, 0 ď c ă 1, we obtain
8
ÿ 1
ci “
i“0
1´c
64 CHAPTER 5. OPERATORS AND FUNCTIONALS
Thus,
cm
}xn ´ xm }V ď }x1 ´ x0 }V
1´c
It follows that txn u8
n“1 is a Cauchy sequence. Since V is a Banach space,
txn u8
n“1 is a convergent sequence. That is, there exists x P V such that
lim xn “ x
nÑ8
Since T is continuous,
T pxq “ T p lim xn q “ lim T pxn q “ lim T n px0 q
nÑ8 nÑ8 nÑ8
By using the fact that L is a linear and self-adjoint operator on a real Hilbert
space,
` ˘
F pφq ´ F pφ0 q “ xLφ0 , φ0 y ` 2txLφ0 , νy ` t2 xLν, νy ´ 2xf, φ0 ` tνy ´ pxLφ0 , φ0 y ´ 2xf, φ0 yq
“ 2txLφ0 , νy ´ 2txf, νy ` t2 xLν, νy
“ 2txLφ0 ´ f, νy ` t2 xLν, νy
Consequently,
F pφq ´ F pφ0 q
“ 2xLφ0 ´ f, νy ` txLν, νy
t
As t Ñ 0, Gateaux differential dF pφ0 , νq can be expressed as follows
F pφq ´ F pφ0 q
dF pφ0 , νq “ lim
tÑ0 t
“ 2xLφ0 ´ f, νy
for all ν P H
Since F has a local minimum at φ “ φ0 , F pφ0 , νq “ 0, for all ν P H. That
is, Lφ0 ´ f “ 0. This proves that φ0 is a solution to Lφ “ f .
Theorem 40. Suppose that L is a linear, self-adjoint and positive operator on
a complex Hilbert space H . The function φ that minimizes the functional
1 1 1
F pφq “ xLφ, φy ´ xφ, f y ´ xf, φy
2 2 2
is the solution to
Lφ “ f
Solution
Consider
„
1 d d
L“´ ppxq ` qpxq (5.16)
wpxq dx dx
ppxq d2 p1 pxq d
L“´ ´ ` qpxq
wpxq dx2 wpxq dx
ppxq
a0 pxq “ ´ (5.17)
wpxq
p1 pxq
a1 pxq “ ´ (5.18)
wpxq
a1 pxq p1 pxq
“
a0 pxq ppxq
Consequently,
ż t“x żx
dppxq a1 pxq dpptq a1 ptq
“ and “ dt
ppxq a0 pxq t“0 pptq 0 a0 ptq
That is, ˆż x ˙
a1 ptq
ppxq “ pp0qexp dt
0 a0 ptq
By choosing pp0q “ 1, we get
ˆż x ˙
a1 px1 q 1
ppxq “ exp dx
0 a0 px1 q
p1 pxq
wpxq “ ´
a1 pxq
qpxq “ a2
5.11. HILBERT-SCHMIDT OPERATOR 67
d2 d
In conclusions, L “ a0 pxq 2
` a1 pxq ` a2 pxq can be written in the form of
dx dx „
1 d d
Sturm - Liouville operator L “ ´ ppxq ` qpxq by choosing
wpxq dx dx
ˆż x ˙
a1 px1 q 1
ppxq “ exp dx
0 a0 px1 q
p1 pxq
wpxq “ ´
a1 pxq
qpxq “ a2
Proof By definition,
„ „
1 d du 1 d dv
Lu “ ´ ppxq ` qpxqu and Lv “ ´ ppxq ` qpxqv
wpxq dx dx wpxq dx dx
Consequently,
ˆ „ ˙ ˆ „ ˙
1 d dv 1 d du
uLv ´ vLu “ u ´ ppxq ` qpxqv ´ v ´ ppxq ` qpxqu
wpxq dx dx wpxq dx dx
ˆ „ ˙ ˆ „ ˙
1 d dv 1 d du
“u ´ ppxq ´v ´ ppxq
wpxq dx dx wpxq dx dx
„ ˆ „ ˙ ˆ „ ˙
1 d du d dv
“ v ppxq ´u ppxq
wpxq dx dx dx dx
„ ˆ ˙
1 d dv du
“ p u ´v
wpxq dx dx dx
2. K is a bounded operator.
Consequently,
ż ż ż
}Ku}2LDˆD ď p |kpx, yq|2 dyqp |upyq|2 dyqdx ă 8
D D D
That is,
}Ku}2L2 pDq ď }k}2L2 pDˆDq }u}L2 pDq ă 8
It follows that K is a bounded operator.
7 It should be noted that k P L2 pD ˆ Dq
Chapter 6
Eigenvalues, Eigenvectors,
Eigenfunctions
Ax “ λx, x ‰ 0
Av “ λv, v ‰ 0.
Consequently,
Av ´ µv “ λv ´ µv, v ‰ 0
or
pA ´ µIqv “ pλ ´ µqv, v ‰ 0
Therefore, λ ´ µ is an eigenvalue of A ´ µI associated with an eigenvector v.
Theorem 43. Let A and n be a square matrix and a positive integer, respec-
tively. λ is an eigenvalue of A associated with an eigenvector v of A. Therefore,
λn is an eigenvalue associated with an eigenvector v of An .
69
70 CHAPTER 6. EIGENVALUES, EIGENVECTORS, EIGENFUNCTIONS
Av “ λv, v ‰ 0.
Consequently,
A2 v “ Apλqv
“ λAv
“ λpλvq
“ λ2 v
By iteration, we obtain
An v “ λn v
Proof
N
ÿ
Suppose that ppAq “ an An .
n“0
Since
An v “ λ n v
In particular,
A0 v “ Iv “ λ0 v,
N
ÿ N
ÿ
an An v “ an λn v
n“0 n“0
That is,
ppAqv “ ppλqv
Proof
To determine if v1 and v2 are independent, consider
c1 v1 ` c2 v2 “ 0 (6.1)
Since
Av1 “ λ1 v1 and Av2 “ λ2 v2 ,
it follows that
Apc1 v1 ` c2 v2 q “ C1 Av1 ` c2 Av2 “ 0
and
c1 λ1 v1 ` c2 λ2 v2 “ 0
6.1. EIGENVALUES AND EIGENVECTORS 71
A “ U ΣV T
where“ ‰ “ ‰
U “ u1 u2 . . . un and V “ v1 v2 . . . vn are orthogonal matrices.
Σ “ diagpσ1 , σ2 , . . . , σminpm,nq q where σ1 ě σ2 ě . . . ě σminpm,nq ě 0 is a
diagonal matrix.
That is,
“ ‰ “ ‰T
A “ u1 u2 ... um diagpσ1 , σ2 , . . . , σminpm,nq q v1 v2 ... vn
1. m ď n
“ ‰“ ‰T
A “ σ1 u1 σ2 u2 ... σm um v1 v2 ... vn
» T fi
v1
T ffi
“ —v2 ffi
‰—
“ σ1 u1 σ2 u2 ... σm um — . ffi
– .. fl
vnT
m
ÿ
“ σi ui viT
i“1
2. m “ n
“ ‰“ ‰T
A “ σ1 u1 σ2 u2 ... σm um v1 v2 ... vn
» T fi
v1
T ffi
“ —v2 ffi
‰—
“ σ1 u1 σ2 u2 ... σm um — . ffi
– .. fl
vnT
m
ÿ
“ σi ui viT
i“1
3. m ą n
“ ‰“ ‰T
A “ σ1 u1 σ2 u2 ... σm um v1 v2 ... vn
» T fi
v1
T ffi
“ —v2 ffi
‰—
“ σ1 u1 σ2 u2 ... σm um — . ffi
– .. fl
vnT
n
ÿ
“ σi ui viT
i“1
6.2. EIGENVALUES AND EIGENFUNCTIONS 73
That is,
minpm,nq
ÿ
A“ σi ui viT
i“1
Example 24. Let A be a matrix with m rows and n columns. Prove that
g
fminpm,nq
f ÿ
}A}F “ e σi2
i“1
Solution
1. A is a real-valued matrix
minpm,nq
ÿ
A“ σi ui viT
i“1
2. A is a complex-valued matrix
α1 φ1 ` α2 φ2 “ 0 (6.4)
Since
Lφ1 “ λ1 φ1 and Lφ2 “ λ2 φ2
we obtain
α1 λ1 φ1 ` α2 λ2 φ2 “ 0
α1 λ1 φ1 ` α2 λ1 φ2 “ 0 (6.5)
and
α1 λ2 φ1 ` α2 λ2 φ2 “ 0, (6.6)
α1 pλ1 ´ λ2 qφ1 “ 0
α2 pλ2 ´ λ1 qφ2 “ 0
α1 “ α2 “ 0
Lemma 9. Let A P Rnˆn . For all ą 0, there exists an induced norm }.}1
such that
}A} ď ρpAq `
Proof
See the proof in [72]
1 }.} depends on
Chapter 7
Examples of Applications
B2 u
“ f, x P p0, 1q
Bx2
with boundary condition up0q “ up1q “ 0 We can construct weak formulation
in the following equation.
ż1 ż1
B2 u
v dx “ f v dx
0 Bx2 0
Consequently,
ˇ1 ż 1 ż1
ˇ
u1 v ˇˇ ´ u1 v 1 dx “ f v dx
0 0 0
Divide the interval r0, 1s into n subintervals; each of which is rxi´1 , xi s for
B2 u
i “ 1, 2, . . . , n. We now want to approximate the solution to “ f, x P
Bx2
p0, 1q with boundary condition up0q “ up1q “ 0 by a function uh P Vh “
spantφ1 , φ2 , φ3 , . . . , φn u
where $
x ´ xi
’
’ , x P rxi´1 , xi s
& h
’
φi pxq “ x i`1 ´ x
, x P rxi , xi`1 s
’
’
’ h
%0, otherwise
Suppose that
n
ÿ
uh “ αi φi pxq.
i“1
77
78 CHAPTER 7. EXAMPLES OF APPLICATIONS
n
ż1 ÿ ż1
´ p αi φi pxqq1 φ1j dx “ f φj dx for j “ 1, 2, 3, . . . , n
0 i“1 0
By rearranging the above equation, we can rewrite the above equation in matrix
form as follows:
» ż1 ż1 fi »ż 1 fi
1 1 1 1
— ´ φ φ
1 1 dx . . . ´ φ φ
1 n dx »
ffi α1
fi
— f φ 1 dx ffi
— 0 0 ffi — 0 ffi
.. .. .
.. . .
ffi – .. fl “ — ..
— ffi — ffi — ffi
—
— . . ffi —
ffi
ffi
– ż1 ż1 fl αn –ż 1 fl
1 1 1 1
´ φn φ1 dx . . . ´ φn φn dx f φn dx
0 0 0
N
ÿ
αn xφn , wm y “ xf, wm y for m “ 1, 2, 3, . . . , N
n“1
Matrices
v T Av ě 0 for all v ‰ 0,
1 It should be noted that a diagonal matrix could not be a square matrix.
79
80 APPENDIX A. MATRICES
v T Av ą 0 for all v ‰ 0,
v T Av ď 0 for all v ‰ 0,
v T Av ď 0 for all v ‰ 0,
Integrals
81
82 APPENDIX B. INTEGRALS
f “ f ` ´ f ´.
Definition 112 (Null Set). Let Ω Ď R. The set Ω is called a null set iff its
characteristic function is a null function.
Definition 113 (Equality Almost Everywhere). Let functions f and g be de-
fined on R. If tx P R | f pxq ‰ gpxqu is a null set, f and g are said to be equal
almost everywhere (f “ g a.e.).
As a result, we can show that
ż
1 The norm }.} is defined in terms of Lebesgue integral by }f } “ |f | dx
R
84 APPENDIX B. INTEGRALS
ż
Theorem 52. |f ´ g| “ 0 iff f “ g a.e.
2 non-increasing or non-decreasing
Appendix C
Holder, Cauchy-Schwarz
and Minkowski Inequalities
85
86APPENDIX C. HOLDER, CAUCHY-SCHWARZ AND MINKOWSKI INEQUALITIES
Appendix D
Complex Analysis
f pz ` ∆zq ´ f pzq
f 1 pzq “ lim
∆zÑ0 ∆z
Definition 118 (Derivatives of Order n).
dF
“ f pzq
dz
for an open set D
87
88 APPENDIX D. COMPLEX ANALYSIS
z “ zptq, a ď t ď b
ż żb
f pzq dz “ f pzptqqz 1 ptq dt
C a
and then
¿
n 1 B f psq
f pzq “ ds
2πı Bz s ´ z
C
¿
n! f psq
“ ds
2πı ps ´ zqn`1
C
That is,
f n pzq
¿
1 f psq
“ ds
n! 2πı ps ´ zqn`1
C
8
ÿ
Lemma 14. Let f pzq “ cn pz ´ z0 qn and on C “ tz | |z ´ z0 | “ R, R ą 0u
n“´8
which is in the counterclockwise direction. Therefore,
¿
f pzq dz “ 2πıRes f pzq
z“z0
C
90 APPENDIX D. COMPLEX ANALYSIS
Gateaux Differential
f px ` hq ´ f pxq “ df px ` ηh, hq
91
92 APPENDIX E. GATEAUX DIFFERENTIAL
Bibliography
[3] R. Durrett, Probability: Theory and Examples, 4th ed., Cambridge Univer-
sity Press, 2010.
[4] M. Vetterli and J. Kovacevic, Wavelets and Subband Coding, 2th ed., Pren-
tice Hall, 2007.
[7] R. L. Allen, Signal Analysis Time, Frequency, Scale and Structure, 1st ed.,
IEEE Press Wiley-Interscience, 2004.
[12] Hal Daume III, “From Zero to Reproducing Kernel Hilbert Spaces in
Twelve Pages or Less,” 2004.
93
94 BIBLIOGRAPHY
[26] E. Sulli, Lecture Notes on Finite Element Methods for Partial Differential
Equations, Mathematical Instistute, University of Oxford, 2012.
[27] Y. Zhu and A. C. Cangellaris, Multigrid Finite Element Methods for Elec-
tromagnetic Field Modeling, 1st ed., IEEE Press, 2006.
[28] G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed., The Johns
Hopkins University Press Baltimore and London, 1996.
[51] I. N. Sneddon, The Use of Integral Transforms, 1st ed., McGraw - Hill,
1972.
[52] J. M. Erdman, “Companion to Real Analysis,” Lecture Note at Portland
State University, 2012.
[68] https://en.wikipedia.org
[69] J. K. Hunter,“Applied Analysis,” Lecture Notes at University of California
at Davis.
[70] math.unice.fr/„frapetti/CorsoF/cours2.pdf
[71] math.unice.fr/„frapetti/CorsoF/cours3.pdf
[72] http://www.math.drexel.edu/„foucart/TeachingFiles/F12/M504Lect6.pdf
[73] people.reed.edu/„jerry/311/lec08.pdf
[74] M. Beck, G. Merchesi, D. Pixton and L. Sabalka,“A First Course in Com-
plex Analysis,” Version 1.52, 2016.
[75] C. Berg, “Complex Analysis,” Department of Mathematical Sciences, Uni-
versitetsparken 5, 2012.
[76] E. M. Stein and R. Sharkarchi, “Complex Analysis,” Princeton Lectures in
Analysis, Princeton University Press, 2007.
[77] C. Bennewitz, “Complex Analysis,” Lecture Notes at Lund University 2006.
[78] G. Cain, “Complex Analysis,” Lecture Notes at Georgia Institute of Tech-
nology, 1999.