Jim Lambers

MAT 415/515
Fall Semester 2013-14
Lecture 3 Notes

These notes correspond to Section 5.2 in the text.

Gram-Schmidt Orthogonalization
We have seen that it can be very convenient to have an orthonormal basis for a given vector
space, in order to compute expansions of arbitrary vectors within that space. Therefore, given a
non-orthonormal basis, it is desirable to have a process for obtaining an orthonormal basis from it.
Fortunately, we have such a process, known as Gram-Schmidt orthogonalization. Suppose that
we have a linearly independent, but not orthonormal, set of functions {χ1 , χ2 , . . .} that span a given
vector space V . To construct an orthonormal set {ϕ1 , ϕ2 , . . .} from this set, we proceed as follows.
First, to obtain ϕ1 , we simply normalize χ1 :
χ1
ϕ1 = .
kχ1 k

Next, to obtain ϕ2 , we need to ensure that it is orthogonal to ϕ1 , and then normalize it.
As an intermediate step, we seek a function ψ2 of the form

ψ2 = χ2 + c12 ϕ1

such that hϕ1 |ψ2 i = 0. Then, we can set ϕ2 = ψ2 /kψ2 k. Taking the scalar product of both sides of
the above equation with ϕ1 , we obtain

0 = hϕ1 |ψ2 i = hϕ1 |χ2 i + c12 hϕ1 |ϕ1 i.

Because the ϕj are orthonormal, it follows that

c12 = −hϕ1 |χ2 i.

We conclude that ϕ2 can be obtained as follows:

ψ2 = χ2 − hϕ1 |χ2 iϕ1

ψ2
ϕ2 = .
kψ2 k

We now have a set of two functions that is orthonormal.

Now, to obtain ϕ3 , we must ensure that it is orthogonal to ϕ1 and ϕ2 , and then normalized.
To that end, we seek a function ψ3 of the form

ψ3 = χ3 + c13 ϕ1 + c23 ϕ2

such that hϕ1 |ψ3 i = hϕ2 |ψ3 i = 0. Then, we can set ϕ3 = ψ3 /kψ3 k. Taking the scalar product of
both sides of the above equation with ϕ1 , and then separately, ϕ2 , we obtain

0 = hϕ1 |ψ3 i = hϕ1 |χ3 i + c13 hϕ1 |ϕ1 i + c23 hϕ1 |ϕ2 i
0 = hϕ2 |ψ3 i = hϕ2 |χ3 i + c13 hϕ2 |ϕ1 i + c23 hϕ2 |ϕ2 i.

1
Because the ϕj are orthonormal, it follows that

c13 = −hϕ1 |χ3 i, c23 = −hϕ2 |χ3 i.

We conclude that ϕ3 can be obtained as follows:

ψ3 = χ3 − hϕ1 |χ3 iϕ1 − hϕ2 |χ3 iϕ2

ψ3
ϕ3 = .
kψ3 k

We now have a set of three functions that are orthonormal.

Continuing this process, we see that we can obtain each function ϕj as follows:
j−1
X
ψj = χj − hϕk |χj iϕk
k=0
ψj
ϕj = .
kψj k

This yields a set of functions {ϕ1 , ϕ2 , . . .} that is an orthonormal basis of the space spanned by
{χ1 , χ2 , . . .}, with respect to the scalar product that is used.
Example We wish to obtain a set of orthonormal polynomials with respect to the scalar product
Z 1
hf |gi = f ∗ (s)g(s) ds.
−1

This will be accomplished by applying Gram-Schmidt orthogonalization to the set {1, x, x2 , x3 , . . .}.
Setting χj (x) = xj for j = 0, 1, 2, . . . , our orthogonal set {ϕj }, j = 0, 1, 2, . . . ,, is obtained as follows:

ψ0 (x) = χ0 (x)
= 1,
ψ0 (x)
ϕ0 (x) =
kψ0 k
1
=
h1|1i1/2
1
= hR i1/2
1
−1 1 ds
1
= √ ,
2
ψ1 (x) = χ1 (x) − hϕ0 |χ1 iϕ0 (x)
  
1  1
= x − √ x √
2 2
Z 1
1
= x− s ds
2 −1
1
= x− 0
2
= x,

2
 ψ1 (x)
ϕ1 (x) =
kψ1 k
x
=
hx|xi1/2
x
= hR i1/2
1 2
−1 s ds
r
3
= x,
2
ψ2 (x) = χ2 (x) − hϕ0 |χ2 iϕ0 (x) − hϕ1 |χ2 iϕ1 (x)
   *r  + r
1 1 3  2 3
x2 − √  x2 √ −

= x x x
2 2 2  2
1 1 2
Z Z 1
2 3
= x − s ds − x s3 ds
2 −1 2 −1
12 3
= x2 − − 0
23 2
1
= x2 − ,
3
ψ2 (x)
ϕ2 (x) =
kψ2 k
x2 − 13
= 1/2
x2 − 1  x2 − 1


3 3
x2 − 13
= hR i1/2
1 1 2


−1 x − 3 ds
r  
5 3 2 1
= x − .
2 2 2

Continuing this process, we obtain

r  
7 5 3 3
ϕ3 (x) = x − x .
2 2 2
In general, r
2n + 1
ϕn (x) = Pn (x),
2
where Pn (x) is the Legendre polynomial of nth degree. We will learn more about these orthogonal
(but not orthonormal) polynomials later in this course. 2
While Gram-Schmidt orthogonalization can be applied to the monomial bais {1, x, x2 , x3 , . . .}
to obtain an orthonormal sequence of polynomials, it can be quite cumbersome, as can be seen from
the preceding example. However, a modification of this procedure can yield a much more efficient
approach.
Suppose that we have already generated a sequence of n orthonormal polynomials ϕ0 , ϕ1 , ϕ2 , . . . , ϕn−1
with respect to some scalar product
Z b
hf |gi = f ∗ (x)g(x)w(x) dx,
a

3
where ϕj is of degree j for j = 0, 1, 2, . . . , n − 1. Then, to obtain ϕn , which of degree n, we
orthogonalize xϕn−1 (x), which is of degree n, against ϕ0 , ϕ1 , . . . , ϕn−1 using the same approach as
in Gram-Schmidt orthogonalization. That is, we compute
n−1
X
ψn (x) = xϕn−1 (x) − hϕj |xϕn−1 iϕj (x),
j=0
ψn (x)
ϕn (x) = .
kψn k
Now, consider the scalar product hϕj |xϕn−1 i. Using the properties of the scalar product, we
have
hϕj |xϕn−1 i = hxϕj |ϕn−1 i.
However, because ϕ0 , ϕ1 , . . . , ϕn−1 are orthonormal, hp|ϕn−1 i = 0 if p(x) is any polynomial of
degree less than n − 1. Because xϕj (x) is of degree j + 1, it follows that hϕj |xϕn−1 i = 0 whenever
j + 1 < n − 1, or j < n − 2. Therefore, our orthogonalization procedure simplifies to

ψn (x) = xϕn−1 (x) − hϕn−2 |xϕn−1 iϕn−2 (x) − hϕn−1 |xϕn−1 iϕn−1 (x),
ψn (x)
ϕn (x) = .
kψn k
That is, any family of orthogonal polynomials satisfies a three-term recurrence relation, in which
each polynomial depends on the previous two. Table lists several families of orthogonal polynomials
that can be generated from such a recurrence relation; we will see some of these families later in
the course.
Polynomials Scalar Product
R1
Legendre Pn (x)Pm (x) dx = 2δmn /(2n + 1)
R−1
1 ∗ ∗ (x) dx = δ
Shifted Legendre P (x)Pm mn /(2n + 1)
R01 n
Chebyshev, first kind Tn (x)Tm (x)(1 − x )−1/2 dx = δmn π/(2 − δn0 )
2
R−1
1 ∗ ∗ (x)[x(1 − x)]−1/2 dx = δ
Shifted Chebyshev, first kind T (x)Tm mn π/(2 − δn0 )
R01 n 2 )1/2 dx = δ
Chebyshev, second kind R−1 U n (x)U m (x)(1 − x mn π/2
∞ −x
Leguerre R0∞ Lnk (x)Lm (x)e dx = δmn
Associated Laguerre L (x)L k (x)e−x dx = δ
mn (n + k)!/n!
R0∞ n m
−x 2 √
Hermite −∞ Hn (x)Hm (x)e dx = 2n δmn πn!

As can be seen in the following example, Gram-Schmidt orthogonalization can be applied to

vectors in any inner product space, such as vectors in Rn .
Example Given the vectors in R3 ,
     
1 1 0
|a1 i =  1  , |a2 i =  2  , |a3 i =  1  ,
−2 −3 1

we will use Gram-Schmidt orthogonalization to obtain an orthonormal set of vectors, {|b1 i, |b2 i, |b3 i}.
We have
|a1 i
|b1 i =
ha1 |a1 i1/2

4
 1
= |a1 i
61/2
 
1
1
= √  1 ,
6 −2
|b02 i = |a2 i − hb1 |a2 i|b1 i
   
1 1
9 1
=  2 − √ √  1 
−3 6 6 −2
 
−1/2
=  1/2  ,
0
|b02 i
|b2 i =
hb2 |b02 i1/2
0

1
= |b02 i
1 1/2


2
 
−1
1 
= √ 1 ,
2 0
|b03 i = |a3 i − hb1 |a3 i|b1 i − hb2 |a3 i|b2 i
     
0   1 −1
1 1 1 1
=  1  − −√ √  1 − √ √  1 
1 6 6 −2 2 2 0
     
0 1 −1
1 1
=  1 +  1 −  1 
6 2
1 −2 0
 
1
2 
= 1 ,
3
1
|b03 i
|b3 i =
hb03 |b03 i1/2
1
= |b03 i
4 1/2


3


1
1
= √  1 .
3 1

We conclude that our orthonormal set of vectors is

     
1 −1 1
1 1 1
|b1 i = √  1  , |b2 i = √  1  , |b3 i = √  1  .
6 −2 2 0 3 1
2