Chapter 8 Gram-Schmidt Orthogonalization

(September 8, 2010)

_______________________________________________________________________
Jørgen Pedersen Gram (June 27, 1850 – April 29, 1916) was a Danish actuary and
mathematician who was born in Nustrup, Duchy of Schleswig, Denmark and died in
Copenhagen, Denmark. Important papers of his include On series expansions determined
by the methods of least squares, and Investigations of the number of primes less than a
given number. The process that bears his name, the Gram–Schmidt process, was first
published in the former paper, in 1883.

http://en.wikipedia.org/wiki/J%C3%B8rgen_Pedersen_Gram
________________________________________________________________________

Erhard Schmidt (January 13, 1876 – December 6, 1959) was a German

mathematician whose work significantly influenced the direction of mathematics in the
twentieth century. He was born in Tartu, Governorate of Livonia (now Estonia). His
advisor was David Hilbert and he was awarded his doctorate from Georg-August
University of Göttingen in 1905. His doctoral dissertation was entitled Entwickelung
willkürlicher Funktionen nach Systemen vorgeschriebener and was a work on integral
equations. Together with David Hilbert he made important contributions to functional
analysis.
http://en.wikipedia.org/wiki/File:Erhard_Schmidt.jpg
________________________________________________________________________

8.1 Gram-Schmidt Procedure I

Gram-Schmidt orthogonalization is a method that takes a non-orthogonal set of
linearly independent function and literally constructs an orthogonal set over an arbitrary
interval and with respect to an arbitrary weighting function. Here for convenience, all
functions are assumed to be real.

un(x) linearly independent non-orthogonal un-normalized functions

Here we use the following notations.

un ( x)  x n (n = 0, 1, 2, 3, …..).

 n (x) linearly independent orthogonal un-normalized functions

 n (x) linearly independent orthogonal normalized functions

with

  ( x )
a
i j ( x ) w ( x ) dx   i , j .

Starting with n = 0, let

  0 ( x )  u0 ( x ) ,

 0 ( x)
0 ( x)  b
.


2
0 w( x)dx
a

For n = 1,

 1 ( x)  u1 ( x)  a100 ( x) .

We demand that  1 ( x) be orthogonal to 0 ( x).

b b b

 1 ( x)0 ( x)w( x)dx  0   u1 ( x)0 ( x)w( x)dx  a10  0 ( x) w( x)dx

2

a a a

or

b
a10    u1 ( x) 0 ( x) w( x)dx .
a

Normalizing, we define

 1 ( x)
1 ( x)  b
.


2
1 w( x)dx
a

Generalizing, we have

 i ( x)
i ( x )  b
,


2
i w( x)dx
a

where

 i ( x )  u i ( x )  ai 0 0 ( x )  ai11 ( x )  ai 2 2 ( x )  ...  ai , i 1 i 1 ( x ) .

The coefficient (aij) are given by

 b
aij    ui ( x) j ( x) w( x)dx .
a

since

 ( x) ( x)w( x)dx  0

a
i j

b
  {ui ( x)  ai 00 ( x)  ai11 ( x)  ai 2 2 ( x)  ...  ai ,i 1i 1 ( x)} j ( x) w( x)dx
a

b b
0   ui ( x) j ( x) w( x)dx  aij  [ j ( x)]2 w( x)dx .
a a

8.2 Example: Hermite polynomials

The Hermite polynomial Hn(x) can be derived from the orthogonalization,

un ( x)  x n (n = 0,1,2,3, ),

with the weight function

2
w( x)  e  x .

Starting with n = 0, let

u0 ( x) 1
0 ( x)  
  1/ 4
e
x2
dx


For n = 1, let

 1 ( x)  u1 ( x)  a100 ( x) ,

 
0   0 ( x)e  x  1 ( x)dx   0 ( x)e  x [u1 ( x)  a100 ( x)]dx
2 2

 

or


0   0 ( x)e  x u1 ( x)dx  a10
2


or

 
1
a10    0 ( x)e x2
u1 ( x)dx    xe
 x2
dx  0

 1/ 4


Then we have

 1 ( x)  u1 ( x) .

u1 ( x) 2x
1 ( x)   .
  1/ 4

x e
2
2 x
dx


For n = 2, let

 2 ( x )  u 2 ( x )  a 20 0 ( x )  a 21 1 ( x ) ,

 
0   0 ( x)e  2 ( x)   0 ( x)e  x [u2 ( x)  a200 ( x)]dx ,
x2 2

 

or

 
1  1
a20    u2 ( x)e  x 0 ( x)dx    x 2e  x
2 2
dx   ,
 
 1/ 4
2  1/ 4

 
2x
a21    u2 ( x)e x2
1 ( x)dx    x 2e  x
2
dx  0 ,
 
 1/ 4

 1 1 1
 2 ( x)  x 2   x2  .
2  
1/ 4 1/ 4
2

Normalizing,
  2 ( x)
2 ( x)  


2

2 ( x)e  x  2 ( x)dx


1
x2 
 2

1 x2
 ( x  x  4 )e dx
4

or

1
x2  2x2  1
2 ( x)  2  .
1 2 1 / 4
(  1 / 2 )1 / 2
2

For n = 3, let

 3 ( x)  u3 ( x)  a300 ( x)  a311 ( x)  a322 ( x) ,

 
0   0 ( x)e  x  3 ( x)   0 ( x)e  x [u3 ( x)  a300 ( x)]dx ,
2 2

 

or

 
1
a30    u3 ( x)e  x2
0 ( x)dx    x 3e  x
2
dx  0 ,
 
 1/ 4

 
2 3
a31    u3 ( x)e  x 1 ( x)dx    x 4 e  x
2 2
dx   2  1/ 4 ,
 
 1/ 4
4

1
  x2 
a32    u3 ( x)e  x  2 ( x)dx    x 3e  x
2 2
2 dx  0 ,
1
  (  1 / 2 )1 / 2
2

3 1 / 4 2 x 3
 3 ( x)  x  2 3
 x3  x .
4  1/ 4
2

Normalizing,
  3 ( x)
 3 ( x) 


 3 ( x)e  3 ( x)dx
2
x



3
x3  x
2  1 / 4
  (2 x 3  3 x)

3 2 x2 3
 (x 
3
x) e dx

2

We find that  n (x) is proportional to the Hermite polynomial. The Hermite polynomials
are given by

n Hn(x)

0 1
1 2x
2 2  4 x2
3 12 x  8 x3
4 12  48 x2  16 x4
5 120 x  160 x3  32 x5
6 120  720 x2  480 x4  64 x6
7 1680 x  3360 x3  1344 x5  128 x7
8 1680  13 440 x2  13 440 x4  3584 x6  256 x8
9 30 240 x  80 640 x3  48 384 x5  9216 x7  512 x9
10 30 240  302 400 x2  403 200 x4  161 280 x6  23 040 x8  1024 x10

8.3 Independence of un(x) = xn; Wronskian

We show the independence of {un(x) = xn} using the Wrtonskian determinant.

((Mathematica))
Wronskian

Gk_ : Tablexn , n, 0, k 

W k_, m_ : DGk , x, m 

WR k_ : Table W k , m ,  m, 0, k 

G6

1, x, x2 , x3 , x4 , x5 , x6 

WR 6

1, x, x2 , x3 , x4 , x5 , x6 , 0, 1, 2 x, 3 x2 , 4 x3 , 5 x4 , 6 x5 ,
0, 0, 2, 6 x, 12 x2 , 20 x3 , 30 x4 , 0, 0, 0, 6, 24 x, 60 x2 , 120 x3 ,
0, 0, 0, 0, 24, 120 x, 360 x2 , 0, 0, 0, 0, 0, 120, 720 x, 0, 0, 0, 0, 0, 0, 720

WR 6  MatrixForm

1 x x2 x3 x4 x5 x6
0 1 2x 3x 2 4x 3 5x 4 6 x5
0 2
0 2 6 x 12 x 20 x 3 30 x4
0 0 0 6 24 x 60 x 120 x3
2

0 0 0 0 24 120 x 360 x2
0 0 0 0 0 120 720 x
0 0 0 0 0 0 720

TableDet WR i, i, 1, 8

1, 2, 12, 288, 34 560, 24 883 200, 125 411 328 000, 5 056 584 744 960 000 

8.4 Gram-Schmidt orthogonalization procedure II

We consider a family of functions

u1(x), u2(x), u3(x), ……., un(x),….

where {un{x}} is independent function and

b
(ui , u j )  aij   ui ( x) w( x)u j ( x)dx
a

Suppose that the relation is always valid for the range a  x  b .

a1u1 ( x)  a2u2 ( x)  ...  anun ( x)  0

Then we have, a1 = a2 = …= an = 0. We now construct a linear combination of the

functions
 a11 a12 a1n
a21 a22 a2 n
 n ( x) 
an 1,1 an 1, 2 an 1, n
u1 ( x) u2 ( x) un ( x )

This function is always orthogonal to ui(x). Thus  n (x) is orthogonal to

1 ( x),  2 ( x),...,  n1 ( x) .

We introduce a Gram determinant, An. A0 = 1 and

a11 a12 a1n

a21 a22 a2 n
An 
an 1,1 an 1, 2 an 1, n
an ,1 an , 2 an , n

a11 a12 a1n

a21 a22 a2 n
( n ,  n )   (un ,  n ) An 1
an 1,1 an 1, 2 an 1, n
0 0 (un ,  n )

a11 a12 a1n a11 a12 a1n

a21 a22 a2 n a21 a22 a2 n
(un ,  n )    An
an 1,1 an 1, 2 an 1, n an 1,1 an 1, 2 an 1, n
(un , u1 ) (un , u2 ) (un , un ) an ,1 an , 2 an , n

Then we have

( n ,  n )  An An 1

or

Then we have the final form of the orthogonal (normalized) functions,

 n
n  (n = 1, 2, …)
An 1 An

where A0 = 1.

8.5 Legendre polynomials

Gram-Schmidt orthogonalization procedure for the Legendre polynomials. We use
the method discussed in 8.4.

w(x) = 1 and un(x) = xn, for -1≤x≤1.

Clear"Global`"
((Mathematica))

weight  1; ui_, x_ : x i ;

ai_, j_ : Integrateweight ui, x uj, x, x,  1, 1,
GenerateConditions  False

gram n_ : DetTableai, j, i, 0, n , j, 0, n ;

gram  1  1;

n_, x_ :
Det Append Tableai, j, i, 0, n  1, j, 0, n , x ^ Range0, n  
Simplify;

n , x 
n_, x_ :
gram n  1 gram n 
;

Tablen, n, x, LegendrePn, x , n, 0, 5  Simplify  TableForm

1
0 1
2

1 3 x x
2

2 1 5   1  3 x2  1  1  3 x2 
2 2 2

3 1 7 x   3  5 x2  1 x  3  5 x2 
2 2 2

3 330 x2 35 x4  1 3  30 x2
4  35 x4 
8 2 8

5 1 11 x 15  70 x2  63 x4  1 x 15  70 x2  63 x4 
8 2 8
 fnx

Legendre
n=4 2

1 n=1
n=0
n=2
x
- 1.0 - 0.5 0.5 1.0

-1 n=3

n=5 -2

Fig. Normalized Legendre polynomials. n = 0, 1, 2, 3, 4, and 5.

8.6 Hermite polynomials

Gram-Schmidt orthogonalization procedure for the Hermite polynomials.

w(x) = exp(-x2) and un(x) = xn, for -∞≤x≤∞.

((Mathematica))
 Clear"Global`"

weight  x ; ui_, x_ : x i ;

2

ai_, j_ : Integrateweight ui, x uj, x, x, , ,

GenerateConditions  False

gram n_ : DetTableai, j, i, 0, n , j, 0, n ;

gram  1  1;

n_, x_ :
Det Append Tableai, j, i, 0, n  1, j, 0, n , x ^ Range0, n  
Simplify;

n , x 
n_, x_ :
gram n  1 gram n 
;

Tablen, n, x, HermiteHn, x , n, 0, 5  Simplify  TableForm

 14
1
0 1

 14
2 x
1 2x

2  14
12 x2
2  2  4 x2
x 32 x2 
4 x  3  2 x2 
3  14
3

4 312 x 14x4 4 3  12 x2  4 x4 
2 4
6 
x 1520 x2 4 x4 
2

8 x 15  20 x2  4 x4 
2 15  14
5

fn x

n=4 Hermite n=2

n=1

n=0
x
-3 -2 -1 1 2 3

n=3

n=5 -5

Fig. Normalized Hermite polynomials. n = 0, 1, 2, 3, 4, and 5.

8.7 Laguerre polynomials
Gram-Schmidt orthogonalization procedure for the Laguerre polynomials.

w(x) = exp(-x) and un(x) = xn, for 0≤x≤∞.

((Mathematica))

Clear"Global`"

weight  x ; ui_, x_ : x i ;

ai_, j_ : Integrateweight ui, x uj, x, x, 0, ,
GenerateConditions  False

gram n_ : DetTableai, j, i, 0, n , j, 0, n ;

gram  1  1;

n_, x_ :
Det Append Tableai, j, i, 0, n  1, j, 0, n , x ^ Range0, n  
Simplify;

n , x 
n_, x_ :
gram n  1 gram n 
;

Tablen, n, x, LaguerreLn, x , n, 0, 5  Simplify  TableForm

0 1 1

2 1 2  4 x  x2  2  4 x  x2 
1 1  x 1x
1

3 1  6  18 x  9 x2  x3  6  18 x  9 x2  x3 
2 2
1
6 6
3 4 3 4
4 1  4 x  3 x2  2 x  x 1  4 x  3 x2  2 x  x
3 24 3 24
5 x3 4 5 5 x3 5 x4 5
2
5 1  5 x  5 x   5x  x 15x5x  2   x
3 24 120 3 24 120

8.8 Mathematica program for the general case

Here we make a Mathematica program called "gramschmidt" for the Gram-Schmidt
orthogonalization procedure of more general cases.
General case for Gram-Schmidt procedure

Clear"Global`"

gramschmidtweight_, var_, interval_, n_ :

Modulea, gram, , ,
ai_, j_ : Integrateweight varij , Joinvar , interval,
GenerateConditions  False;
gram  m_ : DetTableai, j, i, 0, m , j, 0, m ; gram  1  1;
 m_ :
Det Append Tableai, j, i, 0, m  1, j, 0, m , var ^ Range0, m  
 m 
Simplify;  m_ : ; Tablek, k , k, 0, n 
gram  m  1 gram  m 

Legendre polynomials

gramschmidt1, x,  1, 1, 3

0, , 1, x, 2,  1  3 x2 , 3, x  3  5 x2 

1 3 1 5 1 7
2 2 2 2 2 2

Shifted Legendre polynomials

gramschmidt1, x, 0, 1, 3

0, 1, 1, 2 3   x,

1

2, 5 1  6 x  6 x2 , 3, 7  1  12 x  30 x2  20 x3 

2

Hermite polynomials

gramschmidtExp x2 , x, , , 3

x   3  2 x2 
0, , 1, , 2, , 3, 
 1  2 x2
14 14 2 14 3 14
1 2 x

Laguerre polynomials

gramschmidtExp x, x, 0, , 3

0, 1, 1,  1  x, 2, 2  4 x  x2 , 3,  6  18 x  9 x2  x3 

1 1
2 6

Associate Laguerre polynomials

gramschmidtx Exp x, x, 0, , 3

0, 1, 1, , 2, , 3,  24  36 x  12 x2  x3 

2  x 6  6 x  x2 1
2 2 3 12

Chebyshev I polynomials

gramschmidt , x,  1, 1, 3
1

1  x2

0, , 1, x, 2,  1  2 x2 , 3, x  3  4 x2 

1 2 2 2
   

Shifted Chebyshev I polynomials

gramschmidt , x, 0, 1, 3

1

x 1  x

0, , 1, 2   x,

1 2 1
  2

2, 1  8 x  8 x2 , 3,  1  18 x  48 x2  32 x3 

2 2
 

Chebyshev II polynomials

gramschmidt 1  x2 , x,  1, 1, 3

0, , 1, 2 x, 2,  1  4 x2 , 3, 4 x  1  2 x2 

2 2 2 2
   