Notes: MA1102: Series and Matrices

M.T. Nair (IIT Madras)

January - May 2018

April 13, 2018

Contents
1 Sequences 3
1.1 Definition and examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Convergence and divergence . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4 Monotone convergence theorem . . . . . . . . . . . . . . . . . . . . . . . . . 11

2 Series 12
2.1 Some typical examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2 Some tests for convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2.1 Integral test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2.2 Comparison test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2.3 Ratio tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2.4 Root test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3 Absolutely convergent series and alternating series . . . . . . . . . . . . . . . 19
2.4 Rearrangements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3 Improper Integrals 24
3.1 Various types of improper integrals . . . . . . . . . . . . . . . . . . . . . . . 24
3.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.3 Gamma and Beta functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4 Power series 29
4.1 Abel’s theorem and radius of convergence . . . . . . . . . . . . . . . . . . . . 29
4.2 Termwise differentiation and termwise integration . . . . . . . . . . . . . . . 31
4.3 Evaluation at convergent endpoints . . . . . . . . . . . . . . . . . . . . . . . 33
4.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.5 Power series centered at a point . . . . . . . . . . . . . . . . . . . . . . . . . 35

5 Taylor’s series and Taylor’s formulas 37

5.1 Taylor’s theorem - Lagrange form . . . . . . . . . . . . . . . . . . . . . . . . 38
5.2 Taylor’s theorem - Cauchy form . . . . . . . . . . . . . . . . . . . . . . . . . 40

1
6 Fourier Series 42
6.1 Trigonometric series and Trigonometric polynomials . . . . . . . . . . . . . . 42
6.2 Fourier series of 2π-periodic functions . . . . . . . . . . . . . . . . . . . . . . 43
6.3 Fourier series for even and odd functions . . . . . . . . . . . . . . . . . . . . 46
6.4 Sine and cosine series expansions . . . . . . . . . . . . . . . . . . . . . . . . 49
6.5 Fourier Series of 2`-Periodic Functions . . . . . . . . . . . . . . . . . . . . . 52
6.6 Fourier Series on Arbitrary Intervals . . . . . . . . . . . . . . . . . . . . . . 54
6.7 Additional Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

7 Matrices 57
7.1 Various types of matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
7.2 Echelon and row reduced echelon form (RREF) . . . . . . . . . . . . . . . . 59
7.3 Inner product and norms of vectors . . . . . . . . . . . . . . . . . . . . . . . 61
7.4 Linear dependence and independence . . . . . . . . . . . . . . . . . . . . . . 63
7.5 RREF, Linear independence and rank . . . . . . . . . . . . . . . . . . . . . 68
7.6 Determinants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
7.6.1 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
7.6.2 Solution of equations by Cramer’s rule . . . . . . . . . . . . . . . . . 71
7.7 Inverse and solution of equations using RREF . . . . . . . . . . . . . . . . . 72
7.8 Eigenvalues and eigenvectors . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
7.8.1 Eigenvalues of hermitian, normal and unitary matrices . . . . . . . . 78
7.9 Eigenvalue representations - diagonalization theorems . . . . . . . . . . . . . 79

2
1 Sequences
1.1 Definition and examples
List of numbers a1 , a2 , . . . is called a sequence. Such a sequence is denoted one of the following
forms:
(a1 , a2 , . . .), {a1 , a2 , . . .}, (an ), {an }.
More precisely:
Definition 1. A sequence of real numbers is a function f : N → R, and it is usually
represented by it range {an := f (n) : n ∈ N}, which is also represneted by the way we have
already mentioned above. C
Example 2. The following are examples of sequences:

(i) (0.1, 0.11, 0.111, . . .).

(ii) ( 21 , 13 , 14 , . . . , n1 , . . .).

(iii) ( 12 , 23 , 34 , . . . , n+1
n
, . . .).

(iv) (1, 2, 1, 2, . . .).

2
(v) ( 12 , 52 , 10
3
, . . . , n n+1 , . . .).

1.2 Convergence and divergence

In Example 5 (1), we have
an = 0.111 · · · 1
1 1 1 1
= + 2+ 3 + ··· + n
10 10 10 10
1 
1 1 − 10n
 1  1 
= 1 = 1− n .
10 1 − 10 9 10
⇒
1 1 1
− an = n
< n.
9 9 × 10 10
1
Thus, first n decimal places of 9 and an are the same. Taking n large enough, the quantity
| 91 − an | can be made as small as we want. For instance, given any positive number ε,
1  1 1 1
n
 − an  < ε if < ε if 10 > if n > log10
 
9 10n ε ε
Definition 3. A sequence (an ) of real numbers is said to converge to a ∈ R if for every
ε > 0, there is a positive integer N such that
|an − a| < ε for all n ≥ N.
A sequence which does not converge is called a divergent sequence. C

3
 If (an ) converges to a, then we write

an → a as n → ∞ or lim an = a,
n→∞

or sometimes an → a when the variable n is understood from the context.

Definition 4. A sequence (an ) of real numbers is said to diverge to ∞ if for every M > 0,
there there is a positive integer N such that

an > M for all n ≥ N.

The sequence (an ) is said to diverge to −∞ if for every M > 0, there there is a positive
integer N such that
an < −M for all n ≥ N.
C

If (an ) diverge to ∞, then we write

an → ∞ as n → ∞,

and if (an ) diverge to −∞, then we write

an → −∞ as n → ∞.

Note that

• an → a ⇐⇒ ∀ ε > 0, ∃ N ∈ N such that an ∈ (a − ε, a + ε) for all n ≥ N .

• an → a ⇐⇒ for every open interval I containing x, ∃ N ∈ N such that an ∈ I for all

n ≥ N.

• (an ) diverges iff for every a ∈ R, there exists an ε > 0, such that an 6∈ (a − ε, a + ε) for
infinitely many n’s.

Example 5. The sequence:

(i) (0.1, 0.11, 0.111, . . .) converges to 1/9.

(ii) ( 21 , 13 , 14 , . . . , n1 , . . .) converges to 0.

(iii) ( 12 , 23 , 34 , . . . , n+1
n
, . . .) converges to 1.
n
(iv) ( 1+n2 ) converges to 0.

(v) (1, 2, 1, 3, . . .) diverges, and diverges to ∞.

2
(vi) ( 12 , 52 , 10
3
, . . . , n n+1 , . . .) diverges.

4
Proofs. (i) We have already seen that (0.1, 0.11, 0.111, . . .) converges to 1/9.
(ii) Let ε > 0 be given. Then
1  1 h1i
 − 0 = < ε ∀ n ≥ + 1.
 
n n ε
(iii) Let ε > 0 be given. Then
 n  1 h1i
− 1 = < ε ∀n ≥ .
 
n+1 n+1 ε


(iv) Let ε > 0 be given. Then

 n  n n 1 h1i
− 0 = < = < ε ∀ n ≥ + 1.
 
1 + n2 1 + n2 n2 n ε
 

For showing that the sequences in (v) and (vi) diverge, it is necessary to show that they
do not converge to any number a ∈ R.
How can we show this? Can we verify with each and every number a ∈ R? There are
infinitely many such number!
So, we may adopt a particular logical method, called deductio ad absurdum or proof by
contradiction: Suppose there is some a ∈ R such that an → a. Proceeding logically, if we
arrive at a contradiction, then we can conclude that our assumption that there is some a ∈ R
such that an → a is wrong.In other words, there is NO a ∈ R such that an → a.
1 if n odd,
(v) In this case an = Suppose an → a for some a ∈ R. Then taking
2 if n even.
ε = 1/2, we should be able to find N ∈ N such that |an − a| < 1/2 for all n ≥ N . This is
impossible: Note that

|an − a| < 1/2 ⇐⇒ an − 1/2 < a < an + 1/2

so that 1/2 < a < 3/2 & 3/2 < a < 5/2, which is impossible.
Also, note that for any a ∈ R, if we take 0 < ε < 1/2, we see that either 1 6∈ (a − ε, a + ε)
or (2 6∈ (a − ε, a + ε). Thus an 6∈ (a − ε, a + ε) for infinitely many n’s.
(vi) Note that
n2 + 1 1
= n + > n ∀ n ∈ N.
n n
Therefore, for any a ∈ R and for any ε > 0,

n2 + 1
6∈ (a − ε, a + ε) for all large enough n ∈ N.
n
2
In fact, the sequence ( n n+1 ) diverges to ∞, because, for any M > 0,

n2 + 1 1
=n+ >n>M ∀ n > M.
n n

5
1.3 Properties
You must have heard about the Fibonacci sequence1 :

1, 1, 2, 3 5, 8, 13, . . . .

More generally, if a1 = 1 and a2 = 1, then an given by the recurrence relation

an+2 = an+1 + an .

Of course, this sequence (an ) diverges to ∞. However, the sequence (an+1 /an ) converges,
and the limit is known as golden ratio.
Note that, if rn = an+1 /an , then
an+2 an+1 + an an 1
rn+1 = = =1+ =1+ .
an+1 an+1 an+1 rn
Suppose rn → r. Then we have
1 1
rn+1 → r and rn+1 → 1 + →1+
rn r

so that r = 1 + 1r , and hence

√
1+ 5
r= , the golden ratio.
2
In the above discussion we used a few facts, which we have not proved:

(1) (rn ) converges.

(2) rn → r and r 6= 0 imply 1/rn → 1/r and 1 + 1/rn → 1 + 1/r.

√
1+ 5
Note that r = 2
satisfies r = 1 + 1r . Thus,
 1  1 1 1 r − rn
rn+1 − r = 1 + − 1+ = − =
rn r rn r rn r
so that
|r − rn | |r − rn |
|rn+1 − r| = ≤ .
rn r r
From this, it can be shown that
√
|r − rn | |r − rn | 1 1 1  5 − 1
|rn+1 − r| = ≤ ≤ n |r − r1 | = n |r − r1 | = n .
rn r r r r r 2
You can see that (we shall prove it soon!), since r ≥ 3/2 so that 1/rn ≤ (2/3)n → 0. Hence,
we can deduce that |rn+1 − r| → 0. Again, we used the property that
1
Also known as Hemachandra sequence after the 12th century Jain scholar Hemachandra (1089-1173),
who used this sequence about 50 years before Fibonacci.

6
 • |an | ≤ |bn | for all n ∈ N and bn → 0 imply an → 0.

This is a simple exercise. Thus, (1) is proved.

For the property (2), we prove the following.

Theorem 6. Suppose an → a and bn → b. Then we have the following.

1. an + bn → a + b.

2. an bn → ab.
an
3. If bn 6= 0 for all n ∈ N and b 6= 0, then bn
→ ab .

Remark 7. Note that, there can exist sequences (an ) and (bn ) such that an → a and bn → b
with bn 6= 0 for all n ∈ N, but (an /bn ) need not converge. For example, take an = 1 for all
n ∈ N and bn = 1/n. In this case, an /bn → ∞. C

Before proving the above, let us observe some simpler facts:

Proposition 8. Let (an ) be a sequence of real numbers. The the following results hold.

(1) Suppose (an ) converges. Then there exists M > 0 such that |an | ≤ M for all n ∈ N.

(2) Suppose an → a and a 6= 0. Then there exists α > 0 and N ∈ N such that |an ≥ α for
all n ≥ N .

Proof. Let an → a, and let ε > 0 be given. Then there exists N ∈ N such that |an − a| < ε
for all n ≥ N . (1) Note that

|an | = |(an − a) + a| ≤ |an − a| + |a| < ε + 1 ∀ n ≥ N.

Therefore,
|an | ≤ M := max{ε + 1, |a1 |, . . . , |aN |} ∀ n ∈ N.
(2) Suppose a 6= 0. We know that

|an | = |(an − a) + a| ≤ |a| − |an − a| ≥ |a| − ε ∀ n ≥ N.

Taking ε < |a|/2, we obtain |an | ≤ |a|/2 for all n ≥ N .

A definition is in order:

Definition 9. A sequence (an ) is said to be a bounded sequence if there exists M > 0

such that |an | ≤ M for all n ∈ N. C

• Proposition 8 (1) says that every convergent sequence is bounded.

7
Proof of Theorem 6. (1) Let ε > 0 be given. Note that

(an + bn ) − (a + b)| = |(an − a) + (bn − b)| ≤ |an − a| + |bn − b| ∀ n ∈ N.

Let N1 , N2 ∈ N be such that

|an − a| < ε/2 ∀ n ≥ N1 and |bn − b| < ε/2 ∀ n ≥ N2 .

Hence,
|an − a| + |bn − b| < ε/2 + ε/2 ∀ n ≥ N := max{N1 , N2 }.
(2) Note that

|an bn − ab| = |an (bn − b) + (an − a)b| ≤ |an | |bn − b| + |an − a| |b|

Since (an ) converges, there exists M > 0 such that |an | ≤ M for all n ∈ N. Hence,

|an bn − ab| ≤ M |bn − b| + |an − a| |b|.

Let ε > 0 be given, and let N1 , N2 ∈ N be such that

ε ε
|an − a| < ∀ n ≥ N1 and |an − a| < ∀ n ≥ N2 .
2(|b| + 1) 2M
Then we have
ε ε
|an bn − ab| ≤ M |bn − b| + |an − a| |b| < + = ε ∀ n ≥ N := max{N1 , N2 }.
2 2
Thus, an bn → ab.
(3) Note that

 n a  |an b − bn a| |(an − a)b − a(bn − b)| |an − a| |b| + |a| |bn − b|

a 
 − = = ≤
bn b |bn b| |bn b| |bn | |b|
Since bn → b and b 6= 0, there exists α > 0 and N0 ∈ N such that |bn | ≥ α for all n ≥ N0 .
Hence,
 n a  |an − a| |b| + |a| |bn − b| |an − a| |b| + |a| |bn − b|
a 
− ≤ ≤
bn b |bn | |b| α |b|


Let N1 , N2 ∈ N be such that

|an − a| |b| ε |a| |bn − b| ε
< ∀ n ≥ N1 and < ∀ n ≥ N2 .
α |b| 2 α |b| 2
Then
 n a  |an − a| |b| + |a| |bn − b|
a 
 − ≤ < ε ∀n ≥ N := max{N1 , N2 }.
bn b α |b|
an
Thus, bn
→ ab .
Theorem 10. (Sandwich theorem) Suppose an → ` and bn → `. If an ≤ cn ≤ bn for all
n ∈ N, then cn → `.

8
Proof. Let ε > 0 be given. Then there exists N ∈ N such that an , bn ∈ (` − ε, ` + ε) for all
n ≥ N . Since an ≤ cn ≤ bn , we also have cn ∈ (` − ε, ` + ε) for all n ≥ N .
We know that every convergent sequence is bounded. But, a bounded sequence need not
be convergent. A simple example to this effect is:

an = (−1)n , n ∈ N.
However, we shall see that every bounded sequence (an ) which is either monotonically
increasing, that is, an ≤ an+1 for every n ∈ N, or monotonically decreasing, that is, an ≥ an+1
for every n ∈ N, is convergent. This is making use of an important property of the set of
real numbers, called, least upper bound property.
For stating this property we require a few defintions:

Definition 11. Let S ⊆ R.

(1) S is said to be bounded above if there exists M > 0 such that s ≤ M for all s ∈ S.

(2) A number M as in (1) is called an upper bound of S.

(3) An upper bound M0 of S is called a least upper bound (lub) if M0 is least among
all upper bounds of S.

(4) S is said to be bounded below if there exists M 0 > 0 such that s ≥ M 0 for all s ∈ S.

(5) A number M 0 as in (1) is called a lower bound of S.

(6) A lower bound M1 of S is called a greatest lower bound (glb) if M1 is greatest

among all lower bounds of S.

We observe the following:

• Suppose S is bounded above. Then M0 is the lub of S iff for every ε > 0, there exists
s ∈ S such that M0 − ε < s ≤ M0 .

• Suppose S is bounded below. Then M1 is the glb of S iff for every ε > 0, there exists
s ∈ S such that M1 ≤ s < M1 + ε.

A natural question is:

If S is bounded above (resp. bounded below), does it have a least upper bound
(resp. greatest lower bound)?

Least Upper Bound (lub) Property: Every set S ⊆ R which is bounded above has a
least upper bound.
Observe that:

• S is bounded above iff −S := {−s : s ∈ S} is bounded below.

9
Hence, from the Least Upper Bound Property, it follows that:
If S ⊆ R is bounded below, then it has a greatest lower bound.
It can be seen easily that:
• Least upper bound of every set which is bounded above is is unique.
• Greatest lower bound of every set which is bounded below is unique.
Notation:
• Let S be a set which bounded above. Then the least upper bound of S is called the
supremum of S, and it is denoted by sup(S).
• Let S be a set which bounded below. Then the greatest lower bound of S is called the
infimum of S, and it is denoted by inf(S).
It is to be mentioned that:
The supremum (resp. infimum) of a set need not be an element of the set. If it
is in the set, then it is called the maximum (resp. minimum) of the set.
Example 12. (i) The set S = {x ∈ R : 0 ≤ x < 1} is bounded above and 1 is the lub of
S.
(ii) The set S = { n+1
n
: n ∈ N} is bounded above and 2 is the lub of S.
(iii) The set S = N is not bounded above.
2
(iv) The set S = { n n+1 : n ∈ N} is not bounded above.
Example 13. (1) The set S = {x ∈ R : 0 ≤ x < 1} is bounded below and 0 is the glb of
S.
(2) The set S = { n+1
n
: n ∈ N} is bounded below and 1 is the lub of S.
(3) The set S = N is bounded below and 1 is the glb of N.
2
(4) The set S = { n n+1 : n ∈ N} is bounded below and 2 is the glb of S.
(5) The set S = {1, −1, 1, −1, . . .} is bounded above and bounded below, and 1 and −1,
respectively, are the lub and glb of S.
(6) The set S = {1, −1, 2, −2, . . .} is neither bounded above nor bounded below.
Let S ⊆ R.
• If (an ) is a sequence in R and if the set S := {an : n ∈ N} is bounded above, then we
say that (an ) is bounded above, and its supremum is denoted by supn an .
• If (an ) is a sequence in R and if the set S := {an : n ∈ N} is bounded below, then we
say that (an ) is bounded below, and the its infimum is denoted by inf n an .
Definition 14. A sequence (an ) is said to be monotonically increasing if an ≤ an+1 for
all n ∈ N, and it is called monotonically decreasing if an ≥ an+1 for all n ∈ N. C

10
1.4 Monotone convergence theorem
Theorem 15. (Monotone convergence theorem-1) If (an ) is monotonically incresing
and bounded above, then it converges to its supremum.

Proof. Suppose (an ) is monotonically incresing and bounded above and let α := supn (an ).
Then for every ε > 0, there exists N such that

M − ε < aN ≤ M.

Since an ≤ an+1 ≤ α for all n ∈ N, we also have

M − ε < an ≤ α ∀ n ≥ N.

Thus, an → α.
The following theorem is a consequence of the above (How?).

Theorem 16. (Monotone convergence theorem-2) If (an ) is monotonically decreasing

and bounded below, then it converges to its infimum.

Theorem 17. If 0 < a < 1, then an → 0.

Proof. Let a = 1/(1 + b). Then

1 1
an = n
≤ .
(1 + b) nb
1
Since nb
→ 0, it follows that an → 0.

Theorem 18. If c > 1, then cn → ∞.

Proof. Let a = 1/c. Then 0 < a < 1 so that an → 0. Hence, cn → ∞.

Example 19. Let 0 < a < 1 and sn := 1 + a + · · · + an . Then (sn ) converges and its limit
is 1/(1 − a):
1 − an+1 1 − an+1
sn := 1 + a + · · · + an = → .
1−a 1−a
Example 20. n1/n → 1 as n → ∞, and for any a > 0, a1/n → 1.
Let n1/n = 1 + an . Then
n(n − 1) 2
n = (1 + an )n ≥ an .
2!
Hence
2
0 ≤ an ≤ → 0.
(n − 1)
Thus, n1/n = 1 + an → 1. We may use similar trick to show that a1/n → 1.

11
2 Series
Given a sequence (an ) of real numbers, we can forma new sequence (sn ) with
n
X
s n = a1 + · · · + an = ak .
k=1

Definition 21. Given a sequence (an ), an expression of the form ∞

P
k=1 aP
k is called a series,
∞
and sn is called the n-th partial sum of the series.
P∞We say that the series k=1 ak converges
to s ∈P R if sn → s, and in that case we write k=1 ak = s, and s is called the sum of the
series ∞ k=1 ak .
If (sn ) diverges, then we say that the series ∞
P
k=1 ak diverges. C

Here is an important property.

Theorem 22. If ∞
P
n=1 an converges, then an → 0.

Proof. Suppose sn → `. Then an+1 = sn+1 − sn → ` − ` = 0.

2.1 Some typical examples

1 1
Example 23. (1) The series + 2 + · · · converges to 1/9.
10 10
(2) For a ∈ R, consider the series ∞ n
P
n=1 a .
2 n
If a = 1, then sn = 1 + a + a + · · · + a = n + 1 → ∞. Then Suppose a 6= 1. Then

1 − an+1 1 an+1
1 + a + a2 + · · · + an = = − .
1−a 1−a 1−a
If |a| < 1, then |an+1 | = |a|n+1 → 0, and hence, in this case,

1 an+1 1
sn = − → .
1−a 1−a 1−a
P∞
If |a| ≥ 1, then |an+1 | = |a|n+1 ≥ 1 for all n ∈ N, and hence, an 6→ 0, so that n=1 an
diverges. Thus, we have proved that

1 + a + a2 + · · · converges iff |a| < 1 .

1 2 n
(3) The series + + ··· + + · · · diverges to ∞.
2 3 n+1
n
Note that for any n ∈ N, n+1
≥ 1/2. Hence, sn ≥ n/2 for all n ∈ N. Hence, (sn ) diverges
to ∞.

12
 1 1 1 1
(4) The series + + + · · · + + · · · diverges to ∞
1 2 3 n

Note that
1 1 1 1
s2n := + + + ··· + n
1 2 3 2
can be written as
1 1 1 1 1 1 1 1 1  1 1
+ + + + + ··· + + + + ··· + + n−1 + ··· + n .
1 2 3 4 5 8 9 10 16 2 +1 2
Note that
1 1
+ ≥ 1,
1 2
1 1 2
+ ≥ ,
3 4 22
1 1 1 23
+ + ··· + ≥ 4,
9 10 16 2
 1 1 2n−1
+ ··· + n ≥ ,
2n−1 + 1 2 2n
so that
2 22 23 2n−1 n−1 n+1
s2n ≥ 1 + 2
+ 3
+ 4
+ · · · + n
=1+ = .
2 2 2 2 2 2
Thus,
n+1
s2n ≥ ∀ n ∈ N.
2
This shows that (sn ) is unbounded, and hence it is divergent. In fact, it diverges to ∞.

1 1 1 1
(5) The series + 2 + 2 + · · · + 2 + · · · converges.
1 2 3 n

Clearly, the sequence (sn ) of its partial sums is monotonically incresing. Hence, it is
enough to show that it is bounded above. Note that
1 1 2 1
2
+ 2 ≤ 2 = ,
2 3 2 2
1 1 1 1 4 1
2
+ 2 + 2 + 2 ≤ 2 = 2,
4 5 6 7 4 2
n−1
1 1 1 2 1
n−1 2
+ n−1 2
+ · · · + n−1 2
≤ n−1 2
= n−1 .
(2 ) (2 + 1) (2 − 1) (2 ) 2
Hence sn ≤ 2 for all n ∈ N. Thus, (sn ) is monotonically increasing and bounded above, and
hence, it converges.

1 1 1 1
(6) For every p ≤ 1, the series + p + p + · · · + p + · · · diverges to ∞.
1 2 3 n

13
 (p) (p) (1) (1)
Let sn be its n-th partial sum. Since sn ≥ sn for all n ∈ N and (sn ) is unbounded,
(p)
(sn ) is also unbounded, and hence diverges to ∞.

1 1 1 1
(7) For every p ≥ 2, the series + p + p + · · · + p + · · · converges.
1 2 3 n
(p) (p) (2) (1)
Let sn be its n-th partial sum. Note that sn ≤ sn for all n ∈ N and (sn ) is bounded.
(p) (p)
Since (sn ) is monotonically increasing, it follows that (sn ) monotonically increasing and
bounded, and hence it converges.

1 1 1 1
(8) For every p > 1, the series + p + p + · · · + p + · · · converges.
1 2 3 n

Note that
Z n+1 Z n+1 Z n+1
1 1 dx 1 1
= dx ≤ ≤ dx = p
(n + 1)p n (n + 1)p n xp n n p n
⇒
k k Z n+1 k
X 1 X dx X 1
≤ ≤ .
n=1
(n + 1)p n=1 n xp n=1
np
But,
k Z n+1 Z k+1  −p+1 x=k+1  
X dx dx x 1 1 1
= = = 1− ≤ .
n=1 n xp 1 x p −p + 1 x=1 p−1 (k + 1) p−1 p−1

Hence,
k+1 k k+1 Z n+1
X 1 X 1 X dx 1
p
= p
≤ p
≤ .
n=2
n n=1
(n + 1) n=1 n x p − 1
Thus, the sequence of partial sums,which is already monotonically increasing, is bounded as
well. Hence, the given series converges. Thus, we have
∞
X 1
For p ∈ R, the series converges ⇐⇒ p > 1.
n=1
np

1 1 1 1
(9) The series 1+ + + + ··· + + · · · converges.
1! 2! 3! n!

Since
1 1
= n ∀ n ≥ 2.
n! 2
Hence,
1 1 1 1 1 1 1 1
+ + + ··· +
sn = 1 + ≤ 1 + + + 2 + · · · + n ≤ 3.
1! 2! 3! n! 1 2 2 2
Since (sn ) is monotonically increasing and bounded above, it converges.

14
 ∞
X 1
• The sum of the series is denoted by e.
n=0
n!
 1 n
• It can be shown that lim 1+ exists and it is equal to e (See the book2 ).
n→∞ n

2.2 Some tests for convergence

2.2.1 Integral test
The following theorem can be proved using the techniques adopted in Example 23 (8).

Theorem 24. (Integral test) If f : [1, ∞) → [0, ∞) is monotonically decreasing, then

∞
X Z n 
f (n) converges ⇐⇒ f (x)dx converges.
n=1 1

2.2.2 Comparison test

Theorem 25. (Comparison test) Suppose (an ) and (bn ) are sequences of non-negative
terms. Suppose there exists k ∈ N such that an ≤ bn for every n ≥ k.

1. If ∞
P P∞
n=1 bn converges, then n=1 an converges.

2. If ∞
P P∞
n=1 an diverges, then n=1 bn diverges.

Proof. Hint: Use monotone convergence theorem, by observing that (i) the sequence of
partial sums is monotonically increasing and (ii) every convergent sequence is bounded.

Theorem 26. (Limit comparison test) Suppose (an ) and (bn ) are sequences of positive
an
terms. Suppose lim = `.
n→∞ bn

1. If 0 < ` < ∞, then ∞

P P∞
n=1 an converges ⇐⇒ n=1 bn converges.

2. If ` = 0, then ∞
P P∞
n=1 bn converges ⇒ n=1 an converges.

3. If ` = ∞, then ∞
P P∞
n=1 an converges ⇒ n=1 bn converges.

Proof. Let ε > 0. Then there exists N ∈ N such that

an
`−ε< < ` + ε ∀ n ≥ N.
bn
(1) Suppose α > 0 and 0 < ε < α/2. Then

` an 3`
< < ∀ n ≥ N.
2 bn 2
2
See Example 1.25, page 30 in Calculus of One Variable by M.T. Nair

15
Hence
` 3`
bn < an < bn ∀ n ≥ N.
2 2
Therefore, the result follows from Theorem 25.
(2) Suppose ` = 0. Then
an
0< < ε ∀n ≥ N
bn
so that
0 < an < εbn ∀ n ≥ N.
Therefore, the result follows from Theorem 25.
∞
X 2n + n 2n +n
Example 27. Consider the series . Clearly, 3n −n
> 0 for all n ∈ N. Also,
n=1
3n −n

2n + n  2 n  1 + n/2n 
= .
3n − n 3 1 − n/3n
n 1+n/2n
Note that (verify) 1+n/2
1−n/3n
→ 1 as n → ∞. Hence, there exists k ∈ N such that 1−n/3n
< 3
2
for all n ≥ k. Therefore,

2n + n  2 n  1 + n/2n  3  2 n
= ≤ ∀ n ≥ k.
3n − n 3 1 − n/3n 2 3
P∞  2  n P∞ 2n +n
Since n=1 3
converges, it follows that n=1 3n −n also converges.

2.2.3 Ratio tests

Theorem 28. Suppose (an ) is a sequence of positive terms.
an+1
(i) Suppose
P∞ there exists k ∈ N and 0 < c < 1 such that an
≤ c for all n ≥ k. Then
n=1 an converges.
an+1 P∞
(ii) Suppose there exists k ∈ N such that an
≥ 1 for all n ≥ k. Then n=1 an diverges.

n+1−k
P∞ n
Proof.
P∞ (i) a n+1 ≤ c a k for all n ≥ k. Since n=k c converges, by comparison test,
n=k an also converges.
(ii) an+1 ≥ ak > 0 for all n ≥ k. Hence, an 6→ 0. Therefore, ∞
P
n=k an diverges.

Theorem 29. (d’Alembert’s atio test) Suppose (an ) is a sequence of positive terms, a and
an+1 n+1

lim = `, where either ` is a non-negative real number or ` = ∞, which means
n→∞ an an
diverges to ∞.
P∞
(i) If ` < 1, then n=1 an converges.
P∞
(ii) If ` > 1, then n=1 an diverges.

16
  
an+1
If an
diverges to ∞, then the conclusion in (ii) holds.

Proof. Suppose 0 ≤ ` < ∞. Let ε > 0 and let N ∈ N be such that ` − ε < an+1 an
< ` + ε for
all n ≥ N .
(i) Tale ε > 0 such that ` + ε < 1 and apply Theorem 28 (1).
(ii) Tale ε > 0 such that ` − ε ≥ 1 and apply Theorem 28 (2).
an+1
P∞an ≥ 1 for all n ≥ N . Hence, an+1 ≥ aN > 0
If ` = ∞, then there exists N ∈ N such that
for all n ≥ N so that an 6→ 0; consequently, n=1 an diverges to ∞.

2.2.4 Root test

Theorem 30. (Cauchy’s root test) Suppose (an ) is a sequence of positive terms,  a and
1/n n+1
lim an = `, where either ` is a non-negative real number or ` = ∞, which means
n→∞ an
diverges to ∞.
P∞
(i) If ` < 1, then n=1 an converges.
P∞
(ii) If ` > 1, then n=1 an diverges.

Proof. Use similar techniques as in the proof of d’Alembert’s ratio test.

Exercise 31. If ` =P1, then d’Alembert’s ratio test and Cauchy’s ratio test fail. One may
∞ 1 P∞ 1
consider the series n=1 n and n=1 n . C
∞
X xn
Example 32. For any x ∈ R, the series converges. This follows from ratio test, since
n=0
n!

xn+1 /(n + 1)! x

n
= → 0.
x /n! n+1
∞
X xn
The limit of the above series is denoted by exp(x) or by the notation ex .
n=1
n!

It can be shown that

• exp(x + y) = exp(x) exp(y);

• The function f (x) := exp(x) is strictly monotonically increasing, that is, x < y implies
exp(x) < exp(y).

• The function f (x) := exp(x) is continuous on R and it is one-one and onto [0, ∞).
an+1
Theorem 33. Let (an ) be a sequence of positive real numbers such that lim = `. If
n→∞ an
0 < ` < ∞, then lim a1/n
n = `.
n→∞

17
Proof. Let ε > 0 be given and let k ∈ N be such that
an+1
`−ε< < ` + ε ∀ n ≥ k,
an
that is,
(` − ε)an < an+1 < (` + ε)an ∀ n ≥ k.
Taking ε small enough such that ` − ε > 0, we obtain

(` − ε)j ak < ak+j < (` + ε)j ak ∀ j ∈ N.

Writing k + j = n, we have

(` − ε)n−k ak < an < (` + ε)n−k ak ∀ n > k,

that is,
ak n n ak
(` − ε) < an < (` + ε) ∀ n > k,
(` − ε)k (` − ε)k
so that h ak i1/n 1/n
h a
k
i1/n
(` − ε) < an < (` + ε) ∀ n > k,
(` − ε)k (` − ε)k
h a i1/n
k
Recall that for any x > 0, lim x1/n = 1. Hence, lim = 1. Let k1 ∈ N be
n→∞ n→∞ (` − ε)k
k1 ≥ k and
h a i1/n
k
1−ε< < 1 + ε ∀ n ≥ k1 .
(` − ε)k
Thus,
(1 − ε)(` − ε) < a1/n
n < (` + ε)(1 + ε) ∀ n ≥ k1 ,
that is,
` − ε(` + 1 + ε) < a1/n
n < ` + ε(` + 1 + ε) ∀ n ≥ k1 .
Hence, lim a1/n
n exists and it is equal to `.
n→∞

It can happen that root test can be applied whereas ratio test many not be applied.

Example 34. Consider the series

1 1 1 1 1 1 1 1 1 1
+ + + + + + · · · + + + + + ··· .
22 2 24 23 26 25 2n+1 2n 2n+3 2n+2
Hence, in this case,
a2n a2n+1 1
= 2 and = ,
a2n−1 a2n 2
1 1 1
whereas a1/n
n ∈ { 1+ 1 , } so that lim an1/2 = .
2 n 2 n→∞ 2
Thus, ratio test cannot be applied, whereas root test shows that the series converges.

18
2.3 Absolutely convergent series and alternating series
P∞
Theorem 35. Let (an ) be a sequence of real numbers such that n+1 |an | converges. Then
P∞
n+1 an converges.

For its proof we shall make use of the following result, whose proof we omit:
Theorem 36. (Cauchy’s criterion of convergence of sequences) Let (an ) be a sequence
of real numbers. Then (an ) converges iff for every ε > 0, there exists N ∈ N such that
|an − am | < ε ∀ n, m ≥ N.
Definition 37. A sequence (an ) of real numbers is called a Cauchy sequence if for every
ε > 0, there exists N ∈ N such that
|an − am | < ε ∀ n, m ≥ N. C
Thus, Theorem 36 says that

(an ) converges ⇐⇒ (an ) is a Cauchy sequence.

It is to be observed that:

Every convergent sequence is a Cauchy sequence.

Indeed, if an → a, then for any n, m ∈ N,

|an − am | ≤ |an − a| + |a − am |.
For any give ε > 0, let N ∈ N be such that |an − a| < ε/2 for all n ≥ N . Then from the
above,
ε ε
|an − am | ≤ |an − a| + |a − am | < + = ε ∀ n, m ≥ N.
2 2
Definition 38. A series n+1 an is said to be absolutely convergent if ∞
P∞ P
n+1 |an | con-
verges. C
Thus, Theorem 35 says that:

Every absolutely convergent series is convergent.

P∞ P∞
Proof of Theorem 35. Let sn and s0n be the n-th partial sums of n=1 an and n=1 |an |,
respectively. Then for n > m, we have
|sn − sm | ≤ s0n − s0m . (∗)
P∞
Since n=1 |an | assumed to be convergent, (s0n ) is a Cauchy sequence, and by the relation
(∗), (sn ) is also a Cauchy sequence. Hence, by Theorem 36, (sn ) converges.

19
 ∞
X xn
Example 39. For any x ∈ R, the series is absolutely convergent, and hence converges
n=1
n!
as well.

Definition 40. The function exp : R → R defined by

∞
X xn
exp(x) = , x ∈ R,
n=1
n!

is called the exponential function. C

It can be shown that3

exp(x + y) = exp(x) exp(y) ∀ x, y ∈ R.

Note that exp(x) > 0 for x > 0, and exp(0) = 1. Hence,

exp(x) exp(−x) = exp(0) = 1

so that
1
exp(x) 6= 0 and exp(−x) = ∀ x ∈ R.
exp(x)
In particular, exp(x) > 0 for every x ∈ R. It can also be proved that4

the function x 7→ exp(x) is continuous, strictly increasing, exp(x) → ∞ as x → ∞

and exp(x) → 0 as x → −∞.

Hence, for every y > 0, there exists a unique x ∈ R such that

exp(x) = y.

This x is called the logarithm of y, denoted by log(y).

Definition 41. A series of the form ∞ n+1

P
n=1 (−1) an , where an > 0 for all n ∈ N, is called
an alternating series. C
∞
X (−1)n
By Theorem 35, for any p > 1, the series is absolutely convergent, and hence
np
n=1
converges as well. Does the above series converge for 0 < p ≤ 1? The answer is in affirmative,
due to the following theorem.

Theorem 42. (Leibnitz test for convergence of alternating series) P∞If (an ) is a strictly
n+1
decreasing sequence of positive real numbers such that an → 0, then n=1 (−1) an con-
verges.
3
See page 110, Theorem 2.29, in Calculus of One variable, Anne Books, Pvt. Ltd, 2014, by M.T. Nair.
4
See page 110, Theorem 2.29, in Calculus of One variable, Anne Books, Pvt. Ltd, 2014, by M.T. Nair.

20
Proof. Let (an ) be a strictly decreasing sequence
P∞ of positive real numbers such that an → 0.
n
Let sn be the n-th partial sum of the series n=1 (−1) an . Then for every n ∈ N,
s2n = a1 − a2 + a3 − a4 + · · · + a2n−1 − a2n
= (a1 − a2 ) + (a3 − a4 ) + · · · + (a2n−1 − a2n )
= a1 − (a2 − a3 ) − (a4 − a5 ) · · · − (a2n−2 − a2n−1 ) − a2n .
From the above it follows that (s2n is a monotonically increasing and bounded above by a1 .
Hence, it converges, say a2n → s. Then we have
s2n+1 = s2n + a2n+1 → s + 0 = s.
Now, let ε > 0 be given and let N1 , N2 ∈ N be such that
|s2n − s| < ε ∀n ≥ N1 and |s2n+1 − s| < ε ∀ n ≥ N2 .

∞
X (−1)n
By the above theorem, the series is convergent. In fact:
n=1
n
∞
X (−1)n
For any p > 0, converges.
n=1
np
Remark 43. Does the series
1 1 1 1 1 1 1
1+ − − + + − − + ···
2 3 4 5 6 7 8
∞
X
converge? Note that the above series can be written as (an + bn ), where
n=1

(−1)n+1 (−1)n+1
an = , bn = .
2n − 1 2n
∞ ∞ ∞ h
X (−1)n+1 X (−1)n+1 X (−1)n+1 (−1)n+1 i
Since and converge, it follows that + also
n=1
2n − 1 n=1
2n n=1
2n − 1 2n
converges. C

2.4 Rearrangements
Suppose, for a given sequence (an ) of real numbers, the series ∞
P
n=1 an converges. If P
(bn ) is a
sequence obtained from (an ) by rearranging its terms, then is it true that the series ∞ n=1 bn
is also convergent?
In other words, ifP ∞
P
n=1 an converges and if σ : N → N is a bijective function, then is it
true that the series ∞ n=1 aσ(n) is also convergent?
The answer is in affirmative if the terms of the series are all positive (Exercise). In fact,
in this case, the sums of all the rearranged series are the same.
From the above discussion, it follows that:

21
 ∞
X ∞
X
If an is absolutely convergent, then an = s and all the
n=1 n=1
rearranged series are convergent.
However, if the series is conditionally convergent, then different rearrangements of the series
can have different sums.
Example 44. Consider the series
1 1 1 1 1 1 1
1 − + − + − + − + ··· . (1)
2 3 4 5 6 7 8
We know that this series converge. Now consider the series
1 1 1 1 1 1 1 1
1 − − + − − + ··· + − − + ··· . (2)
2 4 3 6 8 2n − 1 4n − 2 4n
If sn and s0n are the n-th partial sums of the series in (1) and (2), respectively, and if s is
the sum of the series in (1), then it can be shown that5
s2n
s03n = ∀ n ∈ N,
2
so that
s2n s
s03n = → ,
2 2
0 0 1 s
s3n+1 = s3n + → ,
2n + 1 2
1 1 s
s03n+2 = s03n + − → .
2n + 1 4n − 2 2
0
Therefore (why?) sn → s/2. Thus, the series in (1) and (2) have different sums.

Question: If ∞
P P∞
n=1 an is convergent, then is it true that every series obtained from n=1 an
by rearranging the terms also convergent?
Not necessarily. Look at the following example.
Example 45. Consider the convergent alternating series
1 1 1 1 1 1 1
1 − + − + − + − + ··· .
2 3 4 5 6 7 8
Consider the following rearranged series:
 1 1 1 1 1 1 1
1+ − + + + + −
3 2 5 7 9 11 4
1 1 1 1 1 1 1
+ + + + + + −
13 15 17 19 21 23 6
..
.
 1 1 1
+ + ··· + −
2(n − 1)n − 1 2n(n + 1) − 1 2n
+
..
.

5
See page 59, Example 153 in Calculus of One variable, Anne Books, Pvt. Ltd, 2014, by M.T. Nair.

22
If sn is the n-th partial sum of the above series, then we see that
 1 1
sn(n+2) = 1 + −
3 2
1 1 1 1 1
+ + + + −
5 7 9 11 4
1 1 1 1 1 1 1
+ + + + + + −
13 15 17 19 21 23 6
..
.
 1 1 1
+ + ··· + − .
2(n − 1)n − 1 2n(n + 1) − 1 2n

Note that 2n(n + 1) − 1 = [2(n − 1)n − 1] + 4n so that there are 2n number of terms in the
1 1
last bracket and, since 2n(n+1)−1 ≥ 2n(n+1) we have
 1 1  2n 1
+ ··· + ≥ = .
2(n − 1)n − 1 2n(n + 1) − 1 2n(n + 1) n+1

Thus,
1 1 1  1 1 1
sn(n+2) ≥ + ··· +
+ − + + ··· +
2 3 n+1 2 4 2n
1 1 1  11 1 1
= + + ··· + − + + ··· +
2 3 n+1 2 1 2 n
1 1  1 1
= + ··· + + −
2 n n+1 2

Hence, sn(n+2) → ∞ as n → ∞, so that the rearranged series diverges to ∞.

P∞
Question:
P∞ If n=1 an is absolutely convergent, then is it true that all the series obtained
from n=1 an with terms rearranged converge to the same sum?

23
3 Improper Integrals
3.1 Various types of improper integrals
Rb
Recall that definite integral a f (x)dx is usually defined for a bounded function f : [a, b] → R,
which is either continuous or piecewise continuous. We would like to see if we can extent
the usual type of integral to the cases to have integrals of the forms
Z ∞ Z b Z ∞ Z b
f (x)dx, f (x)dx, f (x)dx f (x)dx
a −∞ −∞ a

even when f is not necessarily bounded in an interval containing the end-points. It is natural
to define them as follows:
Rt
1. Suppose f is defined and continuous on [a, ∞). If lim a f (x)dx exists, then we define
Z ∞ t→∞

the improper integral f (x)dx as

a
Z ∞ Z t
f (x)dx = lim f (x)dx.
a t→∞ a

Rb
2. Suppose f is defined and continuous on (−∞, b]. If lim t
f (x)dx exists, then we
t→−∞
Rb
define the improper integral −∞ f (x)dx as
Z b Z b
f (x)dx = lim f (x)dx.
−∞ t→−∞ t

Rt
3. Suppose f is defined and continuous on [a, b). If lim a
f (x)dx exists, then we define
t→b
Rb
the improper integral a f (x)dx as
Z b Z t
f (x)dx = lim f (x)dx.
a t→b a

Rb
4. Suppose f is defined and continuous on (a, b]. If lim t
f (x)dx exists, then we define
t→a
Rb
the improper integral a f (x)dx as
Z b Z b
f (x)dx = lim f (x)dx.
a t→a t

Rc
5. If f is defined and continuous on [a, c) and (c, b], and if the improper integrals a
f (x)dx
Rb Rb
and c f (x)dx exist, then then we define the improper integral a f (x)dx as
Z b Z c Z b
f (x)dx = f (x)dx + f (x)dx.
a a c

24
 R∞ Rc
6. If f is continuous on R and if the improper integrals f (x)dx and c f (x)dx exist
−∞
Z ∞
for some c ∈ R, then then we define the improper integral f (x)dx as
−∞
Z ∞ Z c Z ∞
f (x)dx = f (x)dx + f (x)dx.
−∞ −∞ c

If an improper integral exist, then we also say that the improper integral converges.
Here are a few results on improper integrals over an interval J which is of finite or infinite
length.
Z Z
• If 0 ≤ f (x) ≤ g(x) for all x ∈ J, and if g(x)dx exists, then f (x)dx also exists,
Z Z J J

and f (x)dx ≤ f (x)dx.

J J
Z Z
• if |f (x)|dx exists, then f (x)dx also exists.
J J

3.2 Examples
Example 46. We show that for ever p ∈ R,

Z ∞ Z ∞
dx dx 1
converges ⇐⇒ p > 1 and in that case = .
1 xp 1 xp p−1

Clearly, if p ≤ 0, then q := −p ≥ 0 and

Z t Z t  q+1 t
dx q x 1
p
= x dx = = (tq+1 − 1) → ∞ as t → ∞.
1 x 1 q + 1 1 q + 1
Now, assume that p > 0. First consider the case of p = 1. In this case,
Z t
dx
= [log(x)]t1 = log(t) → ∞ as t → ∞.
1 x

Next, let p > 0 with p 6= 1. Then

Z t  −p+1 t
dx x 1
p
= = (t−p+1 − 1)
1 x −p + 1 1 −p + 1
Note that
1 1
0<p<1 ⇒ (t−p+1 − 1) = (t1−p − 1) → ∞ as t → ∞,
−p + 1 1−p
1 1 1
p>1 ⇒ (t−p+1 − 1) = (1 − t1−p ) → as t → ∞.
−p + 1 p−1 p−1
Z ∞ Z ∞
dx dx 1
Hence, p
converges ⇐⇒ p > 1 and in that case p
= .
1 x 1 x p−1

25
Example 47. We show that for ever p ∈ R,

Z 1 Z 1
dx dx 1
converges ⇐⇒ p < 1 and in that case = .
0 xp 0 x p 1−p

Clearly, if p ≤ 0, then q := −p ≥ 0 and

Z 1 Z 1  q+1 1
dx q x 1
p
= x dx = = .
0 x 0 q+1 0 q+1

Now, assume that p > 0. First consider the case of p = 1. In this case,
Z 1
dx
= [log(x)]1t = − log(t) = log(1/t) → ∞ as t → 0.
t x
Next, let p > 0 with p 6= 1. Then
Z 1  −p+1 1
dx x 1
p
= = (1 − t−p+1 )
t x −p + 1 t −p + 1

Note that
1 1 1
0<p<1 ⇒ (1 − t−p+1 ) = (1 − t1−p ) → as t → 0,
−p + 1 1−p 1−p
1 1
p>1 ⇒ (1 − t−p+1 ) = (t1−p − 1) → ∞ as t → 0.
−p + 1 p−1
Z 1 Z 1
dx dx 1
Hence, p
converges ⇐⇒ p < 1 and in that case p
= .
0 x 0 x 1−p
Example 48. For p ∈ R and for any a > 0,

Z ∞
dx
converges ⇐⇒ p > 1.
a xp

Note that, Z t Z t Z a
dx dx dx
t>a>1 ⇒ = − ,
a xp 1 xp 1 xp
Z t Z 1 Z t
dx dx dx
t>1>a ⇒ = + .
a xp a xp 1 xp
Hence, Z t Z t
dx dx
lim exists ⇐⇒ lim exists.
t→∞ a xp t→∞ 1 xp
Example 49. For p ∈ R and for any b > 0,

26
 Z b
dx
converges ⇐⇒ p < 1.
0 xp

Note that, Z b Z 1 Z b
dx dx dx
b>1>t>0 ⇒ = + ,
t xp t xp 1 xp
Z b Z 1 Z 1
dx dx dx
1>b>t>0 ⇒ = − .
t xp t xp b xp
Hence, Z b Z 1
dx dx
lim exists ⇐⇒ lim exists.
t→0 t xp t→0 t xp
Example 50. We have

Z ∞ Z ∞
sin x cos x
dx and dx converge for p > 1
1 xp 1 xp

This follows from the fact that

Z ∞  Z ∞ Z ∞ Z ∞
 sin x  dx  cos x  dx
  dx ≤ ,  p  dx ≤
 
 xp  xp x xp
1 1 1 1
Z ∞
dx
and converges for p > 1.
1 xp
In fact,

Z ∞ Z ∞
sin x cos x
dx and dx converge for all p > 0
1 xp 1 xp

This can be seen by using integration by parts for the proper integrals and taking limits
(Exercise).
For more examples, see Section 4.2 in the book by Nair6

3.3 Gamma and Beta functions

The gamma function Γ(x) for x > 0 is defined as
Z ∞
Γ(x) = tx−1 e−t dt.
0
6
Calculus of One variable, Anne Books, Pvt. Ltd, 2014.

27
Since x > 0 and e−t ≤ 1, we have
tx−1 e−t ≤ tx−1
R1
so that 0
tx−1 e−t dt converges. Also, since

tx−1 e−t
→ 0 as t → ∞,
1/t2
Z ∞ Z ∞
dt
and 2
converges, tx−1 e−t dt also converges.
1 t 1
Hence, Γ(x) is well-defined for x > 0. It can be seen that

Γ(1) = 1, Γ(x + 1) = xΓ(x) for x ≥ 1

so that
Γ(n + 1) = n! ∀ n ∈ N.
The beta function β(x, y) for x > 0, y > 0 is defined as
Z 1
β(x, y) = tx−1 (1 − t)y−1 dt.
0

Note that  1 1−y 1

(1 − t)y−1 = ≤ 21−y ⇐⇒ t ≤ .
1−t 2
Hence,
Z 1/2 Z 1/2 Z 1/2
x−1 y−1 x−1 1−y
t (1 − t) dt ≤ t 2 dt ≤ 2tx−1 dt.
0 0 0
R 1/2 R 1/2
Since 0
2tx−1 dt converges, 0
tx−1 (1 − t)y−1 dt also converges. Also,
Z 1 Z 1/2
x−1 y−1
t (1 − t) dt = sy−1 (1 − s)x−1 ds.
1/2 0

R1
Thus, 1/2
tx−1 (1 − t)y−1 dt converges, and hence β(x, y) is well-defined for x, y > 0.

Γ(x)Γ(y)
• β(x, y) = .
Γ(x + y)

28
4 Power series
Power series is a generalization of polynomials. Recall that, by a polynomial we mean an
expression of the form
a0 + a1 x + · · · + an x n ,
where a0 , a1 , a2 , . . . , an are real numbers, and x is a real variable. Given real numbers
a0 , a1 , a2 , . . . , an , a polynomial can also be thought of as a function f : R → R defined
by defined by
f (x) = a0 + a1 x + · · · + an xn , x ∈ R.
Instead of finite number of numbers a0 , a1 , a2 , . . . , an , if we are give a sequence (a0 , a1 , a2 , . . . , )
of real numbers, then we may consider a series

a0 + a1 x + · · · + an x n + · · ·

or more compactly as
∞
X
an x n .
n=0
P∞
Such a series is called a power P
series. We say that the power series n=0 an xn converges
at at point c if the the series ∞ n
n=0 an c converges.

4.1 Abel’s theorem and radius of convergence

Note that partial sums of a power series are polynomials.
A natural question is:
For what values of x the above series converges?
In this regard, we have the following important result.
Theorem 51. (Abel’s theorem) If the series ∞ n
P
n=0 an x converges at a point x = c, then
it converges absolutely for all x with |x| < |c|.
Proof. Suppose ∞ n
P
n=0 an c converges. Let x be such that |x| < |c|. Then
  x n   |x| n
|an xn | = an cn  ≤ |an cn | ∀ n ∈ N.
 
c |c|
As ∞ n n n
P
n=0 an c converges, |an c | → 0. In particular, (|an c |) is bounded, that is, there exists
M > 0 such that |an cn | ≤ M for all n ∈ N. Thus,
 |x| n
n
|an x | ≤ M ∀ n ∈ N.
|c|

Note that |x| < 1. Hence, by comparison test, ∞ n

P
|c| n=0 |an x | converges.

Corollary 52. If the series ∞ n

P
n=0 an x converges at a point x = d, then it converges abso-
lutely for all x with |x| > |d|.

29
Proof. Follows from Theorem 66 (How?).

Remark 53. Note that for the proof of Abels’s theorem (Theorem 66) we used only the
boundedness of the sequence (an cn ). Thus, we actually have, in place of Theorem 66, the
following:

If (an cn ) is bounded for some c 6= 0, then the series ∞ n

P
n=0 an x converges abso-
lutely for all x with |x| < |c|.

From this it follows that ∞ n2

P
n=0 x converges for |x| < 1. C

Definition 54. Given a power series ∞ n

P
n=0 an x ,

∞
X
R := sup{|x| : an xn converges at x}
n=0
P∞
is called the radius of convergence of n=0 an xn , and
∞
X
D := sup{x ∈ R : an xn converges at x}
n=0
P∞
is called the domain of convergence of n=0 an x n . C

Note that

• R ≥ 0;

• if ∞ n
P
n=0 an x diverges at a point, then R < ∞;

• if ∞ n
P
n=0 an x converges at every point in R, then R = ∞;

• {x ∈ R : |x| < R} ⊆ D and if R < ∞, then {x ∈ R : |x| > R} ⊆ Dc .

Note that the domain of convergence of a power series is an interval. Therefore, it is also
called the interval of convergence of the power series.

Example 55. (i) The ∞ n

P
n=0 x converges at every x with |x| < 1, and diverges at every x
with |x| ≥ 1.
P∞ xn
• The radius of convergence of n=0 n is 1.
P∞ xn
• The domain of convergence of n=0 n is (−1, 1).

(ii) The ∞ xn
P
n=0 n converges at every x with |x| < 1 and also at x = −1, and diverges at
every x with |x| > 1 and also at x = 1.
P∞ xn
• The radius of convergence of n=0 n is 1.
P∞ xn
• The domain of convergence of n=0 n is [−1, 1).

30
 P∞ xn
(iii) The n=0 n2 converges (absolutely) at every x with |x| ≤ 1 and diverges at every x
with |x| > 1.
P∞ xn
• The radius of convergence of n=0 n2 is 1.
P∞ xn
• The domain of convergence of n=0 n2 is [−1, 1].

P∞ xn
(iv) The converges (absolutely) at every x ∈ R.
n=0 n!

• The radius of convergence of ∞ xn

P
n=0 n! is ∞.

• The domain of convergence of ∞ xn

P
n=0 n! is R := (−∞, ∞).

How to determine the radius of convergence?

Theorem 56. Suppose ` := lim |an |1/n exists or |an |1/n → ` := ∞. Then the radius of
n→∞ 
P∞ ∞, ` = 0,
convergence of n=0 an xn is R = 1/`, with the convention that R =
0, ` = ∞.
Proof. Follows from Cauchy’s root test.
a 
 n+1 
Theorem 57. Suppose there exists k ∈ N such that an 6= for all n ≥ k, ` := lim 
an

n→∞
 a 
 n+1  ∞
 → ` := ∞. Then the radius of convergence of n=0 an xn is R = 1/`,
P
exists or 
an 
∞, ` = 0,
with the convention that R =
0, ` = ∞.
Proof. Follows from d’Alembert’s ratio test.
The procedure in theP above two theorems can be used for determining the radius of
convergence of the series ∞ n
n=0 an (x − x0 ) as well.

4.2 Termwise differentiation and termwise integration

Recall that every polynomial can be differentiated as many times as we want. Moreover, for
a polynomial, f (x) = a0 + a1 x + · · · + an xn , we have

(1) f (2) (0)

a0 = f (0), a1 = f (0), a2 =
2
, and in general,
f (k) (0)
ak = , k = 0, 1, 2, . . . , n.
k!
This shows that the coefficients of a polynomial are uniquely determined by the values of
the polynomial and its derivatives at 0.
Do we have similar result for power series? In this regard we have the following theorem7 .
7
Calculus of One variable, Anne Books, Pvt. Ltd, 2014.

31
 P∞
Theorem 58. Let R be the radius of convergence of the power series n=0 an xn . Then the
function f defined by
X∞
f (x) = an xn , −R < x < R
n=0

is differentiable in the interval (−R, R), and

∞
X
0
f (x) = nan xn−1 , −R < x < R
n=1
P∞
and radius of convergence of n=1 nan xn−1 also equal to R.
Using the above theorem, repeatedly, we have the following.
Corollary 59. Let R be the radius of convergence of the power series ∞ n
P
n=0 an x . Then for
every k ∈ N, the k-th derivative of f , namely f (k) , exists at every x ∈ (−R, R) and
∞
(k)
X n!
f (x) = an xn−k , −R < x < R.
n=k
(n − k)!

f (k) (0)
In particular, ak = , k = 0, 1, 2, . . . .
k!

If the power series ∞ n

P
n=0 an x convergesP in an open interval I
containing 0, then the function f (x) := ∞ n
n=0 an x is infinitely
f (k) (0)
differentiable in I and ak = , k = 0, 1, 2, . . . .
k!

From the above corollary we can deduce the following.

P∞ n
P∞
Corollary 60. (Uniqueness) Suppose a
n=0 Pn x and bn xn have radii of conver-
n=0P
gence R1 and R2 , respectively, and suppose that n=0 an x = ∞
∞ n n
n=0 bn x in some interval
containing 0. Then R1 = R2 and an = bn for every n.
Not only that a power series can be differentiated term by term, it can also be integrated
term by term.
Theorem 61. Let R be the radius of convergence of the power series ∞ n
P
n=0 an x . Let

∞
X
f (x) = an x n , −R < x < R.
n=0

Then for every a, b ∈ (−R, R),

Z b ∞ Z b ∞
X X an n+1
f (x)dx = an xn dx = [b − an+1 ], −R < x < R.
a n=0 a n=0
n + 1

32
4.3 Evaluation at convergent endpoints
P∞
Let R be the radius of convergence of n=0 an xn and let
∞
X
f (x) = an x n , −R < x < R.
n=0

Suppose the series ∞

P n
P∞ n
n=0 an x converges at x = R, that is, n=0 an R converges. What can
we say about its value? Here is a theorem regarding this8 .

Theorem 62. (Abel) Let R be the radius of convergence of ∞ n

P
n=0 an x and let

∞
X
f (x) = an x n , −R < x < R.
n=0
P∞
If n=0 an xn converges at x = R, then f is left continuous at x = R and
∞
X
lim f (x) = an R n .
x→R−
n=0
P∞
Similarly, if n=0 an xn converges at x = −R, then then f is right continuous at x = −R
and ∞
X
lim f (x) = an (−1)n Rn .
x→−R+
n=0

4.4 Examples
∞
X
Example 63. Consider the power series (−1)n xn for |x| < 1. Note that
n=0

∞
1 X
= (−1)n xn , |x| < 1.
1 + x n=0

Integrating term by term:

t ∞
tn+1
Z
dx X
log(1 + t) = = (−1)n , |x| < 1.
0 1 + x n=0 n+1

Since the above series converges at the right endpoint 1, by Theorem 69,
∞
X (−1)n
log(2) = .
n=0
n+1
8
G. PedrikA First Course in Analysis, Springer, 2009, Page 247

33
Example 64. Consider the series expansion
∞
1 X
= (−1)n x2n , |x| < 1.
1 + x2 n=0

Integrating term by term:

∞
−1
X x2n+1
tan x= (−1)n , |x| < 1.
n=0
2n + 1
Since the above series converges at the right endpoint 1, by Theorem 69, we obtain9
∞
π X (−1)n
= .
4 n=0
2n + 1
Remark 65. For any α ∈ R and x ∈ R, we have
(1 − α)(1 + α + · · · + αk ) = 1 − αk+1 ∀ k ∈ N.
Therefore, if α 6= 1, then
k
1 X αn+1
= αn + .
1 − α n=0 1−α
Taking α = −x with x 6= −1,
k
1 X xk+1
= (−1)n xn + (−1)k+1 .
1 + x n=0 1+x
Integrating
k x
xn+1 tk+1
X Z
n
log(1 + x) = (−1) + (−1)k+1 dt.
n=0
n + 1 0 1+t
Note that for 0 < x ≤ 1,
x
 Z x
tk+1  xk+2
 Z

(−1)k+1 1
dt ≤ tk+1 dt ≤ ≤ .

0 1+t 0 k+2 k+2
Hence,  
k n+1 
 X x 1
log(1 + x) − (−1)n ≤ .
 

n=0
n + 1 k + 2
This is true for all x with 0 ≤ x ≤ 1. In particular, we have
 
k
 X 1  1
log 2 − (−1)n ≤ .
 

n=0
n + 1  k+2
Letting k → ∞, we have
∞
X 1
log 2 = (−1)n .
n=0
n+1
Thus, we prove the conclusion in Example 63 without using Theorem 69. Analogously, we
obtain the conclusion in Example 64 without using Theorem 69. C
9
Value for the Madhava series

34
4.5 Power series centered at a point
P∞
So far we have been only concerned with power series of the form n=0 an x n .
For a given x0 ∈ R and a real numbers a0 , a1 , a2 , . . ., the series
∞
X
an (x − x0 )n
n=0

is called a power series centered at x0 . The analogue of Abel’s theorem is the following:
Theorem 66. (Abel’s theorem) If the series ∞ n
P
n=0 an (x−x0 ) converges at a point x = c,
then it converges absolutely for all x with |x − x0 | < |c − x0 |.
The radiusPof convergence and domain of convergence of this series can be defined by
writing it as ∞ a
n=0 n y n
with y = x − x0 . Thus,
∞
X
R := sup{|x − x0 | : an xn converges at x}
n=0
P∞
is the radius of convergence of n=0 an (x − x0 )n , and
∞
X
D := sup{x ∈ R : an (x − x0 )n converges at x}
n=0

is the domain of convergence of n=0 an xn . Note that ∞

P∞ P n
n=0 an (x − x0 ) converges for
every x with |x − x0 | P
< R. The interval of convergence, in this case, is (x0 − R, x0 + R). The
general power series ∞ n
n=0 an (x − x0 ) can be viewed as the earlier one in terms of a new
variable y := x − x0 , i.e.,
X∞
an y n with y = x − x0 .
n=0

Thus,
∞
X ∞
X
n
an y converges at y = c ⇐⇒ an (x − x0 )n converges at x = x0 + c,
n=0 n=0

|y| < r ⇐⇒ |x − x0 | < r.

In this general case we have the following:
P∞
Theorem 67. Let R be the radius of convergence of the power series n=0 an (x − x0 )n .
Then the function f defined by
∞
X
f (x) = an (x − x0 )n , −R < x < R
n=0

is infinitely differentiable in the interval (−R, R), and for each k ∈ N,

∞
(k)
X n!
f (x) = an xn−k , −R < x < R
n=k
(n − k)!

35
 P∞ n!
and radius of convergence of n=k (n−k)! an (x − x0 )n−k is also equal to R. In particular,

f (k) (x0 )
ak = , k = 0, 1, 2, . . . .
k!
P∞
Theorem 68. Let R be the radius of convergence of the power series n=0 an (x − x0 )n . Let
∞
X
f (x) = an (x − x0 )n , −R < x < R.
n=0

Then for every a, b ∈ (−R, R),

Z b ∞
X Z b
f (x)dx = an (x − x0 )n dx, −R < x < R.
a n=0 a

P∞
Theorem 69. (Abel) Let R be the radius of convergence of n=0 an (x − x0 )n and let
∞
X
f (x) = an (x − x0 )n , −R < x < R.
n=0
P∞
If n=0 an xn converges at x = R, then
∞
X
lim f (x) = an (R − x0 )n .
x→R−
n=0
P∞
Similarly, if n=0 an xn converges at x = −R, then
∞
X
lim f (x) = an (−1)n (R + x0 )n .
x→−R+
n=0

36
5 Taylor’s series and Taylor’s formulas
∞
X f (n) (x0 )
Question: Does the series (x − x0 )n converge to f (x) for x ∈ I?
n=0
n!
We have seen that if a function f can be represented as a power series ∞
P
n=0 converges
f (n) (x0 )
in an interval (x0 − r, x0 + r), then an = n! for every n ∈ N. Thus, in this case we do
have ∞
X f (n) (x0 )
f (x) = (x − x0 )n for x ∈ (x0 − r, x0 + r).
n=0
n!

Definition 70. Suppose f is infinitely differentiable at in an interval containing x0 . Then

f (n) (x0 )
the series ∞ (x − x0 )n is called the Taylor’s series of f centered at x0 , and we
P
n=0 n!
write ∞
X f (n) (x0 )
f (x) ∼ (x − x0 )n .
n=0
n!
C

• If f is a polynomial, then there is a k ∈ N such that f (n) (x0 ) = 0 for all n > k, and
k ∞
X f (n) (x0 ) X f (n) (x0 )
f (x) = (x − x0 )n = (x − x0 )n .
n=0
n! n=0
n!

Example 71. We know that

∞
1 X
= xn , |x| < 1,
1−x n=0
∞
1 X
= (−1)n xn , |x| < 1,
1+x n=0
∞
1 X
2
= (−1)n x2n , |x| < 1.
1+x n=0

Therefore,
dn  1  dn  1 
= n!, = (−1)n n!,
dxn 1 − x x=0 dxn 1 + x x=0
dn  1 

0, n odd,
= n/2
n 2
dx 1 + x x=0 (−1) (n)!, n even
Example 72. (1) Let f (x) = sin x. Then we have

f (2n) (x) = (−1)n sin x, f (2n+1) (x) = (−1)n cos x, n = 0, 1, 2, . . . .

Thus the Taylor’s series of f (x) := sin x is

∞ 
(−1)n sin x0 (−1)n cos x0
X 
2n 2n+1
sin x ∼ (x − x0 ) + (x − x0 ) .
n=0
(2n)! (2n + 1)!

37
In particular, taking x0 = 0,
∞
X (−1)n 2n+1
sin x ∼ x .
n=0
(2n + 1)!

Similarly,
∞
X (−1)n
cos x ∼ x2n .
n=0
(2n)!

5.1 Taylor’s theorem - Lagrange form

Question: Under what condition can we say that

∞
X f (n) (x0 )
f (x) = (x − x0 )n
n=0
n!

for x in an interval containing x0 ?

In this context, first we prove the following theorem.
Theorem 73. (Taylor’s theorem - Lagrange form) Suppose f is n + 1 times differen-
tiable in an open interval (a, b) and f (n+1) is continuous in [a, b]. Let x0 ∈ (a, b). Then for
any x ∈ [a, b], there exists cx between x0 and x such that
n
X f (k) (x0 ) f (n+1) (cx )
f (x) = (x − x0 )k + (x − x0 )n+1 .
k=0
k! (n + 1)!

Proof. For y ∈ [a, b], let

n
X f (k) (x0 )
Pn (y) = (y − x0 )k
k=0
k!
and
g(y) = f (y) − Pn (y) − ϕ(x)(y − x0 )n+1 ,
where
f (x) − Pn (x)
ϕ(x) = .
(x − x0 )n+1
Note that

g(x0 ) = f (x0 ) − Pn (x0 ) = 0,

g(x) = f (x) − Pn (x) − ϕ(x)(x − x0 )n+1 = 0.

Also, since
Pn(k) (x0 ) = f (k) (x0 ), k = 0, 1, 2, . . . , n + 1,
we have
g (k) (x0 ) = 0, k = 0, 1, 2, . . . , n + 1.
Now, we apply Roll’s theorem repeatedly:

38
 g(x0 ) = 0 = g(x)⇒∃ x1 between x0 and x such that g 0 (x1 ) = 0.
g 0 (x0 ) = 0 = g 0 (x1 )⇒∃ x2 between x0 and x1 such that g (2) (x2 ) = 0.
g (2) (x0 ) = 0 = g (2) (x2 )⇒∃ x3 between x0 and x2 such that g (3) (x3 ) = 0.
Continuing this, we obtain x1 , x2 , . . . xn such that g (k) (xk ) = 0 for k = 1, 2, . . . , n.
Finally, g (n) (x0 ) = 0 = g (n) (xn )⇒∃ xn+1 between x0 and xn such that g (n+1) (xn+1 ) =
0.

Let c := xn+1 . Since g (n+1) (y) = f (n+1) (y) − ϕ(x)(n + 1)!, we obtain

f (n+1) (c) − ϕ(x)(n + 1)! = 0

so that
f (x) − Pn (x) f (n+1) (c)
= ϕ(x) = .
(x − x0 )n+1 (n + 1)!
f (n+1) (c)
That is, f (x) = Pn (x) + (x − x0 )n+1 .
(n + 1)!
Corollary 74. Suppose f is infinitely differentiable in an open interval (a, b). If there exists
M > 0 such that |f (n+1) (x)| ≤ M for all x ∈ (a, b), then for any x, x0 ∈ (a, b),
∞
X f (n) (x0 )
f (x) = (x − x0 )n .
n=0
n!

n
X (−1)k 2k+1
Example 75. Taking f (x) = sin x, we have P2n+1 (x) = x and |f n (x)| ≤ 1,
k=0
(2k + 1)!
so that ∞
X (−1)n 2n+1
sin x = x .
n=0
(2n + 1)!
n
X (−1)k
Similarly, taking f (x) = cos x, we have P2n+1 (x) = x2k and |f n (x)| ≤ 1, so that
k=0
(2k)!

∞
X (−1)n
cos x = x2n .
n=0
(2n)!
Z x
dx
Example 76. Recall that f (x) = log(1 + x) := for x ≥ 0, so that
0 1+t

1 −1 2(−1)2 2 × 3(−1)3
f 0 (x) = , f 00 (x) = , f (3) (x) = , f (4) (x) = ,
1+x (1 + x)2 (1 + x)3 (1 + x)4

and more generally,

n!(−1)n
f (n+1) (x) = .
(1 + x)n+1

39
In particular,
f (n+1) (0) (−1)n
= .
(n + 1)! n+1
Note that, for each fixed x ≥ 0,

(n+1)
 1 n!(−1)n  1 n!
|f (x)| =  = → ∞ as n → ∞.

(1 + x) (1 + x)n (1 + x) (1 + x)n
Thus, Corollary 74 cannot be applied. However, we know that
∞
X xn+1
log(1 + x) = (−1)n , x ≥ 0,
n=0
n+1

that is, log(1 + x) has the Taylor series expansion for x ≥ 0.

5.2 Taylor’s theorem - Cauchy form

Theorem 77. (Taylor’s theorem - Cauchy form) Suppose f is n + 1 times differentiable
in an open interval (a, b) and f (n+1) is continuous in [a, b]. Let x0 ∈ (a, b). Then for any
x ∈ [a, b],
n
f (k) (x0 ) 1 x (n+1)
X Z
k
f (x) = (x − x0 ) + f (t)(x − t)n dt.
k=0
k! n! x0

Proof. By fundamental theorem of calculus,

Z x
f (x) = f (x0 ) + f 0 (t)dt.
x0

Thus, the theorem is true if n = 0. If n = 1, then by integration by parts, taking first

function as f 0 (t) and the second function as (x − t)0 ,
Z x Z x
0 0
f (t)dt = [−f (t)(x − t)]x0 +x
f 00 (t)(x − t)dt
x0
Z x x0
= f 0 (x0 )(x − x0 ) + f 00 (t)(x − t)dt.
x0

Now, assume that the theorem is true for some n = m ∈ N, that is,
m Z x
X f (k) (x0 ) k 1
f (x) = (x − x0 ) + f (m+1) (t)(x − t)m dt,
k=0
k! m! x0

and the assumptions of the theorem are satisfied for n = m + 1. Then, by integration by
parts,
Z x x Z x
(x − t)m+1 (x − t)m+1

(m+1) m (m+1)
f (t)(x − t) dt = −f (t) + f (m+2) (t) dt
x0 m+1 x0 x0 m+1
Z x
(m+1) (x − x0 )m+1 (x − t)m+1
= f (x0 ) + f (m+2) (t) dt.
m+1 x0 m+1

40
Thus,
m+1 x
f (k) (x0 )
Z
X 1
f (x) = (x − x0 )k + f (m+2) (t)(x − t)m+1 dt.
k=0
k! (m + 1)! x0

Hence, by induction, we obtain the required formula.

,

Corollary 78. Suppose f is n + 1 times differentiable in an open interval (a, b) and f (n+1)
is continuous in [a, b]. Let x0 ∈ (a, b). If Mn > 0 is such that

|f (n+1) (t)| ≤ Mn ∀ t ∈ [a, b],

then for every x ∈ [a, b],

 
n
 X f (k) (x0 )  (x − t)n+1
f (x) − (x − x0 )k  ≤ Mn .
 
 k!
k=0
 (n + 1)!

P∞ f (k) (x0 )
In particular, if (Mn ) is bounded, then k=0 k!
(x − x0 )k converges and
∞
X f (k) (x0 )
f (x) = (x − x0 )k .
k=0
k!

41
6 Fourier Series
The material for this section is taken from the book Calculus of One Variable10

6.1 Trigonometric series and Trigonometric polynomials

Definition 79. Let (an ) and (bn ) be sequences of real numbers. Then a series of the form
∞
X
c0 + (an cos nx + bn sin nx)
n=1

is called a trigonometric series. C

The trigonometric series c0 + ∞
P
n=1 (an cos nx + bn sin nx) is also written as

c0 + a1 cos x + b1 sin x + a2 cos 2x + b2 sin 2x + · · · .

As in the case of power series, if there exists a k ∈ N ∪ {0} such that an = 0 and bn = 0 for
all n ≥ k, then the resulting series can be represented as
k
X
c0 + (an cos nx + bn sin nx) .
n=1

Definition 80. A function of the form

Xk
(an cos nx + bn sin nx) .
n=0

where an , bn ∈ R are called a trigonometric polynomials. C

We observe that trigonometric polynomials are 2π-periodic on R, i.e., if f (x) is a trigono-

metric polynomial, then
f (x + 2π) = f (x) ∀ x ∈ R.
From this, we can infer that, if the trigonometric series
X∞
c0 + (an cos nx + bn sin nx)
n=1

converges at a point x ∈ R, then it has to converge at x+2π as well; and hence at x+2nπ for
all integers n. This shows that we can restrict the discussion of convergence
P∞ of a trigonometric
series to an interval of length 2π. In particular, if the series c0 + n=1 (an cos nx + bn sin nx)
converges on an interval I, then we have a 2π-periodic function f defined by
∞
X
f (x) = c0 + (an cos nx + bn sin nx)
n=1

for x in the set {x + 2nπ : x ∈ I, n ∈ Z}. Hence, we cannot expect to have a trigonometric
series expansion for a function f : R → R if it is not a 2π-periodic function.
Definition 81. A function f : R → R is said to have period T for some T > 0 if f (x+T ) =
f (x) for all x ∈ R. A function f : R → R with period T is called a T -periodic function. C
10
M.T. Nair, Calculus of One Variable, Ane Publishers, 2014.

42
6.2 Fourier series of 2π-periodic functions
We know that a convergent trigonometric series is 2π-periodic. What about the converse.
That is, suppose that f is a 2π-periodic function. Is it possible to represent f as a trigono-
metric series?
Suppose, for a moment, that we can write
∞
X
f (x) = c0 + (an cos nx + bn sin nx)
n=1

for all x ∈ R. Then what should be the coefficients c0 , an , bn ? To answer this question, let
us further assume that
f is integrable on [−π, π] and the series can be integrated term by term.
For instance if the above series is uniformly convergent11 to f in [−π, π], then term by term
integration is possible. By Weierstrass test12 , we have the following result:
If ∞
P P∞
n=0 (|an | + |bn |) converges, then c0 + n=1 (an cos nx + bn sin nx) is a domi-
nated series on R and hence it is uniformly convergent.
For n, m ∈ N ∩ {0}, we observe the following orthogonality relations:

Z π  0, if n 6= m
cos nx cos mxdx = π, if n = m 6= 0,
−π 
2π, if n = m = 0,
Z π 
0, if n 6= m
sin nx sin mxdx =
π, if n = m,
Z −π
π
cos nx sin mxdx = 0.
−π

Thus, under the assumption that f is integrable on [−π, π] and the series can be integrated
term by term, we obtain Z π
1
c0 = f (x)dx,
2π −π
1 π 1 π
Z Z
an = f (x) cos nxdx, bn = f (x) sin nxdx.
π −π π −π
Definition 82. The Fourier series of a 2π-periodic function f is the trigonometric series
∞
a0 X
+ (an cos nx + bn sin nx) ,
2 n=1

1 π 1 π
Z Z
where an = f (x) cos nxdx and bn = f (x) sin nxdx and this fact is written as
π −π π −π
∞
a0 X
f (x) ∼ + (an cos nx + bn sin nx) .
2 n=1

The numbers an and bn are called the Fourier coefficients of f . C

11
See M.T. Nair, Calculus of One Variable, Ane Books, 2014.
12
See M.T. Nair, Calculus of One Variable, Ane Books, 2014.

43
 If f is a trigonometric polynomial, then its Fourier series is itself.

The following two theorems show that there is a large class of functions which can be
represented by their Fourier series. Interested readers may look for their proofs in books on
Fourier series; for example Bhatia13 .

Theorem 83. Suppose f is a monotonic function on [−π, π]. Then the Fourier series of f
converges, and the limit function f˜(x) is given by

f (x) if f is continuous at x,
f˜(x) = 1
2
[f (x−) + f (x+)] if f is not continuous at x.

Theorem 84. (Dirichlet’s theorem) Suppose f : R → R is a 2π-periodic function which

is piecewie continuous, piecewise differentiable and f 0 is piece wise continuous on [−π, π].
Then the Fourier series of f converges, and the limit function f˜(x) is given by

f (x) if f is continuous at x,
f˜(x) = 1
2
[f (x−) + f (x+)] if f is not continuous at x.

In the above theorem we used the terms, piecewie continuous and piecewise differentiable.
They are in the following sense:

Definition 85. Let f : [a, b] → R is said to be

1. piecewie continuous if there exist a finite number of points x0 , x1 , . . . , xn in [a, b]

with a = x0 < x1 < · · · < xn = b such that f is continuous in each open interval
(xi−1 , xi ) and the limits

f (xi−1 +) := lim f (xi−1 + h) and f (xi −) := lim f (xi − h) exist;

h→0+ h→0+

2. piecewie differentiable if there exist a finite number of points x0 , x1 , . . . , xn in [a, b]

with a = x0 < x1 < · · · < xn = b such that f is differentiale in each open interval
(xi−1 , xi ) and the limits

f (xi−1 + h) − f (xi−1 +) f (xi −) − f (xi − h)

f 0 (xi−1 +) := lim and f 0 (xi −) := lim exist.
h→0+ h h→0+ h

C

0, −1 ≤ x ≤ 0,
Example 86. Let f (x) = In this case, take we take the points
1 − x, 0 < x ≤ 1.
x0 = −1, x1 = 0, x2 = 1. We see that f is differentiable in the open intervals (−1, 0), (0, 1),
and
f (−1+) = 0, f (0−) = 0, f (0+) = 1, f (1−) = 0,
f 0 (−1+) = 0, f 0 (0−) = 0, f 0 (0+) = −1, f (1−) = −1.
13
R. Bhatia, Forrier Series, Trim Series, –

44
Indeed,
f (−1 + h) − f (−1+) 0−0
f 0 (−1+) = lim = lim = 0,
h→0+ h h→0+ h
f (0−) − f (0 − h) 0−0
f 0 (0−) = lim = lim = 0,
h→0+ h h→0+ h
f (0 + h) − f (0+) 1−h−1
f 0 (0+) = lim = lim = −1,
h→0+ h h→0+ h
f (1−) − f (1 − h) 0 − [1 − (1 − h)]
f 0 (1−) = lim = lim = −1.
h→0+ h h→0+ h
Remark 87. It is known that there are continuous functions f defined on [−π, π] whose
Fourier series does not converge pointwise, even on a dense subset of [−π, π]. Its proof relies
on concepts from advanced mathematics. C
Recall that, in the case of a power series, the partial sums are polynomials and the
limit function is infinitely differentiable. In the case of a Fourier series, the partial sums
are infinitely differentiable, but, the limit function, if exists, need not be even continuous at
certain points. This fact is best illustrated by the following example.

0, −π ≤ x ≤ 0,
Example 88. Let f (x) = Note that this function satisfies the condi-
1, 0 < x ≤ π.
tions in Dirichlet’s theorem (Theorem 84). Hence, its Fourier series converges to f (x) for
every x 6= 0, and at the point 0, the series converges to 1/2. Note that
1 π
Z 
1, n = 0,
an = cos nxdx =
π 0 0, n 6= 0,
and for n ∈ N,
π
1 1 − (−1)n
   
1 1 − cos nπ
Z
1
bn = sin nxdx = = .
π 0 π n π n
Thus,
2


 , n odd,
bn = πn

0, n even.


Thus, Fourier series of f is

∞
1 2 X sin(2n + 1)x
+ ,
2 π n=0 (2n + 1)
and by Theorem 84,
∞
1 2 X sin(2n + 1)x
f (x) = + , x 6= 0
2 π n=0 (2n + 1)
In particular, for x = π/2,
∞ ∞
1 2 X sin[(2n + 1)π/2] 1 2 X (−1)n
1= + = +
2 π n=0 (2n + 1) 2 π n=0 (2n + 1)

45
which leads to the Madhava–Nilakantha series
∞
π X (−1)n
= . ♦
4 n=0
(2n + 1)

6.3 Fourier series for even and odd functions

Recall that a function f : R → R is called an

(i) even function, if f (−x) = f (x) for all x ∈ R,

(ii) odd function, if f (−x) = −f (x) for all x ∈ R.

Consider the cases (i) and (ii) separately:

Case(i): Suppose f is an even function. In this case, f (x) cos nx is an even function and
f (x) sin nx is an odd function. Hence bn = 0 for all n ∈ N. Thus:

If f is an even function, then the Fourier series of f is

∞
2 π
Z
a0 X
+ an cos nx with an := f (x) cos nxdx.
2 n=1
π 0

Note that at the points x = 0 and x = π, the above series takes the forms
∞ ∞
a0 X a0 X
+ an and + (−1)n an ,
2 n=0
2 n=1

respectively.
Case(ii): Suppose f is an odd function. In this case, f (x) sin nx is an even function. Hence
an = 0 for all n ∈ N ∪ {0}. Thus:

If f is an odd function, then the Fourier series of f is

∞
2 π
X Z
bn sin nx with bn := f (x) sin nxdx.
n=1
π 0

At the point x = π/2, the series takes the form

∞
X
(−1)n b2n+1 .
n=0

Let us illustrate the above observations by some examples.

Example 89. Consider the function f defined by

f (x) = |x|, x ∈ [−π, π].

46
Note that f is an even function. Hence, bn = 0 for n = 1, 2, . . ., and the Fourier series is
∞
a0 X
+ an cos nx, x ∈ [−π, π]
2 n=1

with Z π
2
a0 = x dx = π
π 0
and for n = 1, 2, . . .,
π Z π
2 π
Z  
2 sin nx sin nx
an = x cos nxdx = x − dx
π 0 π n 0 0 n
2 (−1)n − 1
 
2 h cos nx iπ
= =
π n2 0 π n2

Thus,
−4
a2n = 0, a2n+1 = , n = 1, 2, . . . .
π(2n + 1)2
By Theorem 84,
∞
π 4 X cos(2n + 1)x
|x| = − , x ∈ [−π, π].
2 π n=0 (2n + 1)2
Taking x = 0, we obtain
∞
π2 X 1
= 2
. ♦
8 n=0
(2n + 1)

Example 90. Consider the function f defined by

f (x) = x, x ∈ [−π, π].

Note that this f is an odd function. Hence, an = 0 for n = 0, 1, 2, . . ., and the Fourier series
is ∞
X
bn sin nx, x ∈ [−π, π]
n=1

with
2 π
Z  Z π 
2 h cos nx iπ cos nx
bn = x sin nx dx = −x + dx
π 0 π n 0 0 n
2 n cos nπ o (−1)n+1 2
= −π = .
π n n
By Theorem 83 or Theorem 84, we have
∞
X (−1)n+1
x=2 sin nx.
n=1
n

47
In particular, with x = π/2 we have
∞ ∞
π X (−1)n+1 nπ X (−1)n
= sin = ,
4 n=1
n 2 n=0
2n + 1

the Madhava–Nilakantha series.

Example 91. Consider the function f with


−1, −π ≤ x < 0,
f (x) =
1, 0 ≤ x ≤ π.

Note that f is an odd function. Hence, an = 0 for n = 0, 1, 2, . . ., and the Fourier series is
∞
X
bn sin nx,
n=1

with Z π
2 2 2
bn = sin nx dx = (1 − cos nπ) = [1 − (−1)n ].
π 0 π π
Thus ∞
4 X sin(2n + 1)x
f (x) ∼
π n=0 2n + 1
and by Theorem 84,
∞
4 X sin(2n + 1)x
f (x) = , x 6= 0.
π n=0 2n + 1
Taking x = π/2, we have again the Madhava–Nilakantha series
∞
π X (−1)n
= . ♦
4 n=0
2n + 1

Example 92. Consider the function f defined by

f (x) = x2 , x ∈ [−π, π].

Note that f is an even function. Hence, bn = 0 for n = 1, 2, . . ., and the Fourier series is
∞ Z π
a0 X 2
+ an cos nx, x ∈ [−π, π], an = x2 cos nx dx.
2 n=1
π 0

It can be see that a0 = 2π 2 /3, and an = (−1)n 4/n2 . Thus

∞
2 π2 X (−1)n cos nx
x ∼ +4 , x ∈ [−π, π],
3 n=1
n2

48
and by Theorem 84,
∞
2 π2 X (−1)n cos nx
x = +4 , x ∈ [−π, π].
3 n=1
n2

Taking x = 0 and x = π, we have

∞ ∞
π 2 X (−1)n+1 π2 X 1
= , =
12 n=1 n2 6 n=1
n2

respectively.

6.4 Sine and cosine series expansions

We have seen that Fourier series of an odd function on [−π, π] involves only sine functions,
whereas Fourier series of an even function on [−π, π] involves only cosine functions. The
above observation points to the following:
Suppose a function f is defined on [0, π]. Then we may extend f to all of [−π, π] by
defining it arbitrarily on [−π, 0). However, if we extended function f˜ is odd or even, then we
get the corresponding series as a sine series expansion or cosine series expansion, respectively.

Odd extension of f lead to sine series expansion and even extension of f lead to
cosine series expansion of f on the interval [0, π].

The odd extension and even extension of f , denoted by fo and fe , respectively, are defined
below: 
f (x) if 0 ≤ x < π,
fo (x) = ,
−f (−x) if − π ≤ x < 0,

f (x) if 0 ≤ x < π,
fe (x) =
f (−x) if − π ≤ x < 0.
Clearly,
fo (−x) = −fo (x), fe (−x) = fe (x)
for all x ∈ [−π, π], so that fo is an odd function and fe is an even function on [−π, π].
Therefore,
∞
X
f (x) = fo (x) ∼ bn sin nx, x ∈ [0, π]
n=1

and ∞
a0 X
f (x) = fe (x) ∼ + an cos nx, x ∈ [0, π]
2 n=1

with Z π Z π
2 2
an = f (x) cos nx dx, bn = f (x) sin nx dx.
π 0 π 0

49
Definition 93. The functions fo and fe are called, respectively, the odd extension and
even extension of f , and the two series
∞
X
f (x) ∼ bn sin nx, x ∈ [0, π],
n=1

and ∞
a0 X
f (x) ∼ + an cos nx, x ∈ [0, π],
2 n=1

with Z π Z π
2 2
an = f (x) cos nx dx, bn = f (x) sin nx dx
π 0 π 0
are called, respectively, the sine series expansion and cosine series expansion of f on
[0, π]. C

Odd extension of an odd function is the function itself.

Even extension of an even function is the function itself.

Sine expansion of an odd function is the Fourier expansion

of the function itself.

Cosine expansion of an even function is the Fourier expansion

of the function itself.

Example 94. Consider the function f (x) = x2 , x ∈ [0, π]. Then

 2
x, if 0 ≤ x < π,
fo (x) = 2 ,
−x , if − π ≤ x < 0,

fe (x) = x2 , x ∈ [−π, π].

Hence, even expansion of f on [0, π] is
∞
2π2 X (−1)n cos nx
x ∼ +4 , x ∈ [0, π].
3 n=1
n2

However, its odd expansion is

∞
X
f (x) ∼ bn sin nx, x ∈ [0, π],
n=1

50
with
2 π 2
Z
bn = x sin nx dx
π 0
 Z π 
2 h 2 cos nx iπ cos nx
= −x + 2x dx .
π n 0 0 n
Note that h cos nx iπ cos nπ (−1)n+1
−x2 = −π 2 = π2 ,
n 0 n n
Z π  π Z π
cos nx sin nx sin nx
2x dx = 2x − 2 dx
0 n n 0 0 n
(−1)n − 1
h cos nx iπ  
= 2 =2 .
n2 0 n2
Thus,
n+1
(−1)n − 1
  
2 2 (−1)
bn = π +2
π n n2
(−1)n+1 4 (−1)n − 1
 
= 2π + .
n π n2
By Theorem 84, we have
∞
2 π2 X (−1)n cos nx
x = +4 2
, x ∈ [0, π], x ∈ [0, π],
3 n=1
n

and ∞
X
2
x bn sin nx, x ∈ [0, π], x ∈ [0, π],
n=1
n+1
h i
(−1)n −1
with bn = 2π (−1)n + 4
π n2
.

Example 95. Consider the function f (x) = x, x ∈ [0, π]. Note that

fo (x) = x, x ∈ [−π, π]

and 
x, if 0 ≤ x < π,
fe (x) =
−x, if − π ≤ x < 0.
Thus,
fe (x) = |x|, x ∈ [−π, π].
From Examples 90 and 89, and Theorem 84, we obtain
∞
X (−1)n+1
x=2 sin nx, x ∈ [0, π]
n=1
n

51
and ∞
π 4 X cos(2n + 1)x
x= − , x ∈ [0, π]. ♦
2 π n=0 (2n + 1)2

Example 96. Let us consider sine and cosine expansions of the function

0, if 0 ≤ x < π/2,
f (x) =
1, if π/2 ≤ x < π.

Then the sine series of f is given by

∞
X
f (x) ∼ bn sin nx, x ∈ [0, π],
n=1

where π  
2 cos nπ/2 − cos nπ
Z
2 2 h cos nx iπ
bn = sin nx dx = − = .
π π/2 π n π/2 π n
2
Note that b2n−1 = and
(2n − 1)π
2

2 − nπ if n odd,
b2n = [(−1)n − 1] =
2nπ 0 if n even.

Thus, for x ∈ [0, π], we have

π sin x sin 2x sin 3x sin 5x sin(4n − 3)x
f (x) ∼ − + + + ··· +
2 1 1 3 5 4n − 3
sin(4n − 2)x sin(4n − 1)x sin(4n + 1)x
− + + + ··· .
4n − 2 4n − 1 4n + 1

6.5 Fourier Series of 2`-Periodic Functions

Suppose f is a T -periodic function. We may write T = 2`. Then we may consider the change
of variable t = πx/` so that the function

f (x) := f (`t/π),

as a function of t is 2π-periodic. Hence, its Fourier series is

∞
a0 X
+ (an cos nt + bn sin nt)
2 n=1

where  
Z π Z `
1 `t 1 nπx
an = f cos ntdt = f (x) cos dx,
π −π π ` −` `
1 π `
Z   Z
`t 1 nπx
bn = f sin ntdt = f (x) sin dx.
π −π π ` −` `

52
 The Fourier series of a 2`-periodic function f is:
∞
a0 X h  nπx   nπx i
+ an cos + bn sin with
2 n=1
` `
1 ` 1 `
Z Z
nπx nπx
an = f (x) cos dx, bn f (x) sin dx.
` −` ` ` −` `
In particular,
2 `
Z
nπx
1. f is even implies bn = 0 for all n and an = f (x) cos dx,
` 0 `
R`
2. f is odd implies an = 0 for all n and bn = 2` 0 f (x) sin nπx
`
dx.

Example 97. Consider the function

f (x) = 1 − |x|, −1 ≤ x ≤ 1.

Here, ` = 1, so that
Z 1 Z 1
an = (1 − |x|) cos nπx dx = 2 (1 − |x|) cos nπx dx
−1 0

and Z 1
bn = (1 − |x|) sin nπx dx = 0.
−1

Now, 1
Z 1 
sin nπx
cos nπx dx = = 0,
0 n 0

Z 1  1 Z 1
sin nπx sin nπx
x cos nπx dx = x − dx
0 n 0 0 n
h cos nπx i1
=
n 0
n
(−1) − 1
= .
n
Hence, Z 1 
2 0, neven,
an = 2 (1 − |x|) cos nπx dx = [1 − (−1)n ] =
0 n 4/n, n odd.
Thus,
∞
X 4
f (x) ∼ cos nπx. ♦
n=0
2n + 1

53
6.6 Fourier Series on Arbitrary Intervals
Suppose a function f is defined in an interval [a, b]. We can obtain Fourier expansion of it
on [a, b] as follows:
Method 1: Consider the a new variable y such that
a + b b − a  2π  a + b
x= + y, i.e., y = x− .
2 2π b−a 2
Note that as x varies over [a, b], y varies over [−π, π]. Given f : [a, b] → R, or a (b−a)-period
function f , we may define
a + b b − a 
g(y) := f (x) = f + y .
2 2π
Since g is a 2π-periodic function, we have its Fourier series as
∞
a0 X  
g(y) ∼ + an cos ny + bn sin ny ,
2 n=1

that is ∞
a0 X  
f (x) ∼ + an cos ny + bn sin ny ,
2 n=1
where
Z π Z b  
1 2 2nπ  a + b
an = g(y) cos ny dy = f (x) cos x− dx,
π −π b−a a b−a 2
1 π
Z Z b  
2 2nπ  a + b
bn = g(y) sin ny dy = f (x) sin x− dx.
π −π b−a a b−a 2

The Fourier series of a (b − a)-periodic function f is:

∞
a0 X  
f (x) ∼ + an cos ny + bn sin ny , with
2 n=1
Z b  
2 2nπ  a + b
an = f (x) cos x− dx,
b − a Za b−a 2 
b 
2  2nπ  a + b
bn = f (x) sin x− dx.
b−a a b−a 2
• If [a, b] = [0, π], then we have
∞
a0 X  h  π i h  π i 
f (x) ∼ + an cos 2n x − + bn sin 2n x − , x ∈ [0, π]
2 n=1
2 2
where
2 π
Z h  π i
an = f (x) cos 2n x − dy,
π 0 2
Z π
2 h  π i
bn = f (x) sin 2n x − dy.
π 0 2

54
Method 2: Consider the a new variable y such that
b − a π(x − a)
x=a+ y, i.e., y = .
π b−a
Note that as x varies over [a, b], y varies over [0, π]. Given f : [a, b] → R, or a (b−a)-period
function f , we may define function g as
b − a 

g(y) := f (x) = f a + y , y ∈ [0, π].
π
Taking odd and even extension of g we have
∞
X
g(y) ∼ bn sin ny, y ∈ [0, π],
n=1

that is,
∞
X nπ(x − a)
f (x) ∼ bn sin ,
n=1
b−a
with
π b
nπ(x − a)
Z Z
2 2
bn = g(y) sin ny dy = f (x) sin dx,
π 0 b−a a b−a
and ∞
a0 X
g(y) ∼ + an cos ny, y ∈ [0, π],
2 n=1

that is,
∞
a0 X nπ(x − a)
f (x) ∼ + an cos ,
2 n=1
b − a
with
π b
nπ(x − a)
Z Z
2 2
an = g(y) cos ny dy = f (x) cos dx,
π 0 b−a a b−a

A sine series of a function f defined on [a, b] is:

∞ Z b
X nπ(x − a) 2 nπ(x − a)
f (x) ∼ bn sin with bn = f (x) sin dx,
n=1
b − a b − a a b − a
and a cosine series of a function f defined on [a, b] is:
∞ Z b
a0 X nπ(x − a) 2 nπ(x − a)
f (x) ∼ + an cos with an = f (x) cos dx,
2 n=1
b − a b − a a b − a

55
6.7 Additional Exercises
1. Find the Fourier series of the 2π- period function f such that:
1, −π ≤ x < π2

(a) f (x) = 2
0, π2 < x < 3π 2
.
−π
≤ x < π2

x, 2
(b) f (x) =
π − x, 2 < x < 3π
π
2
.
1 + 2x

π
, −π ≤ x ≤ 0
(c) f (x) = 2x
1 − π , 0 ≤ x ≤ π.
x2
(d) f (x) = 4
, −π ≤ x ≤ π.
2. Using the Fourier series in Exercise 1, find the sum of the following series:
1 1 1 1 1 1
(a) 1 − + − + . . ., (b) 1 + + + + . . ..
3 5 7 4 9 16
1 1 1 1 1 1
(c) 1 − + − + . . ., (d) 1 + 2 + 2 + 2 + . . ..
4 9 16 3 5 7
sin x, 0 ≤ x ≤ π4

3. If f (x) = , then show that
cos x, π4 ≤ x < π2
 
8 π sin x sin 3x sin 10x
f (x) ∼ cos + + + ... .
π 4 1.3 5.7 9.11

4. Show that for 0 < x < 1,

 
28 sin xπ sin 3πx sin 5πx
x−x = 2 + + + ... .
π 13 33 53

5. Show that for 0 < x < π,

sin 3x sin 5x π
sin x + + + ... = .
3 5 4
6. Show that for −π < x < π,
1 2 2 2
x sin x = 1 − cos x − cos 2x + cos 3x − cos 4x + . . . ,
2 1.3 2.4 3.5
and find the sum of the series
1 1 1 1
− + − + ....
1.3 3.5 5.7 7.9
7. Show that for 0 ≤ x ≤ π,
π2
 
cos 2x cos 4x cos 6x
x(π − x) = − + + + ... ,
6 12 22 32
 
8 sin x sin 3x sin 5x
x(π − x) = + + + ... .
π 13 33 53

56
 8. Assuming that the Fourier series of f converges uniformly on [−π, π), show that
∞
1 π a20 X 2
Z
2
[f (x)] dx = + (an + b2n ).
π −π 2 n=1

9. Using Exercises 7 and 8 show that

∞ ∞
X 1 π4 X (−1)n−1 π2
(a) = , (b) =
n=1
n4 90 n=1
n2 12
∞ ∞
X 1 π6 X (−1)n−1 π3
(c) = (d) =
n=1
n6 945 n=1
(2n − 1)3 32

10. Write down the Fourier series of f (x) = x for x ∈ [1, 2) so that it converges to 1/2 at
x = 1.

7 Matrices
7.1 Various types of matrices
In the following F denotes the set of all real numbers R or the set of all complex numbers C.
In school one must have come across simultaneous equations of the form
a11 x1 + a12 x2 + ··· + a1n xn = b1
a21 x1 + a22 x2 + ··· + a2n xn = b2
(1)
··· + ··· + ··· + ··· = ···
am1 x1 + am2 x2 + ··· + amn xn = bm
Note that the above equations involve the array of numbers
   
a11 a12 · · · a1n b1
 a21 a22 · · · a2n   b2 
  and  
 ··· ··· ··· ···   ··· 
am1 am2 · · · amn bm
Thus, for convenience, we may represent the above simultaneous equations in the form
    
a11 a12 · · · a1n x1 b1
 a21 a22 · · · a2n   x2   b2
    

 ··· ··· ··· = . (2)
···  ···   ··· 
am1 am2 · · · amn xn bm
Definition 98. By an m × n matrix A (read as an m by n matrix A), we mean an array
of numbers aij for i = 1, . . . , m; j = 1, . . . , n, written in m rows and n columns as follows:
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
A=  ··· ··· ··· ··· 


am1 am2 · · · amn

Such a matrix shall be written, in short, as (aij )m× or [aij ]m× or (aij ) or [aij ]. C

57
 Thus, (2) can be written as
Ax = b, (3)
where      
a11 a12 · · · a1n x1 b1
 a21 a22 · · · a2n   x2   b2 
A=
 ···
, x=
 ··· ,
 b=
 ··· .

··· ··· ··· 
am1 am2 · · · amn xn bm
Note that the equations in (1) can be solved easily if
aij = 0 ∀ i > j.
Such matrices are called upper triangular matrices. Similarly, the equations in (1) can be
solved easily if
aij = 0 ∀ i < j.
Such matrices are called lower triangular matrices.
• The set of all m × n matrices is denoted by Fm×n .
Definition 99. Given a matrix A = [aij ] ∈ Fm×n , the matrix AT := [bij ] ∈ Fn×m with
bij := aji is called the transpose of A, and the matrix A∗ := [cij ] ∈ Fn×m with cij := āji is
called the adjoint of A. C
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
Thus, if A =  · · · · · · · · · · · · , then


am1 am2 · · · amn

   
a11 a21 · · · am1 ā11 ā21 · · · ām1
 a12 a22 · · · am2 
 , A∗ =  ā12 ā22 · · · ām2 
 
AT =   ··· ··· ··· ···   ··· ··· ··· ··· 
a1n a2n · · · amn ā1n ā2n · · · āmn
Definition 100. A square matrix A ∈ Fn×n is called
1. hermitian or self adjoint if A∗ = A;
2. normal if A∗ A = AA∗ ;
3. unitary if A∗ A = I = AA∗ ;
A matrix A ∈ Rn×n is called
1. symmetric if it is hermitian, i.e., if AT = A;
2. orthogonal if it is unitary, i.e., if AT A = I = AAT .
C
• If A is orthogonal, and if A1 , . . . , An are the columns of A, then A1 , . . . , An are orthog-
onal, that is,
ATi Aj = δij .

58
7.2 Echelon and row reduced echelon form (RREF)
While solving a system of equations
Ax = b, (1)
where A ∈ Fm×n and b ∈ Fm , it is better to convert the above system in an equivalent form

Ãx = b̃ (2)

so that the latter can be solved easily. By saying (1) and (2) are equivalent we means that
a vector x ∈ Fn is a solution of (1) iff it is a solution of (2).
For example if à is a generalized upper triangular form, then solution x is obtained by
back-substitution. There are other simpler forms as well.
A matrix A := [aij ] is said to be in echelon form if

1. nonzero rows are above all zero rows,

2. all column entries below the first nonzero entry in a row are all zeros.

Example 101. The following matrices are in echelon forms:

   
1 0 3 0 1 0 3 0 1
2 1 2 0 1 1 2 0 2
0 3 0 1 , 0 0 1 1 3
   

0 0 0 0 0 0 0 0 0

A still simpler form:

A matrix A := [aij ] is said to be in row reduced echelon form (RREF) if

1. the first nonzero entry in each row, i.e., the pivot, is 1,

2. the pivot in a row is strictly right of the pivots in the rows above it,

3. all other entries in a column where a pivot appears are zeros.

Example 102. The following matrices are in RREFs:

   
1 0 3 0 1 0 3 0 1
0 1 2 0 0 1 2 0 2
0 0 0 1 , 0 0 0
   
1 3
0 0 0 0 0 0 0 0 0

Example 103. The following matrices are NOT in RREFs:

     
1 0 3 0 1 1 3 0 1 0 1 2 0 2
0 1 2 0 0 1 2 0 2  0 0 0 1 3
0 0 0 1 , 0 , 
   
0 0 1 3 0 0 0 0 0
0 0 0 1 0 0 0 0 0 1 0 0 0 0

The following elementary operations can be used to transform a matrix into a RREF.

59
 1. Interchanging of any two rows.

2. Multiplication of any row by a nonzero number.

3. Adding to a two a nonzero multiple of any other row.

Example 104.
     
1 2 3 0 1 2 3 0 1 2 3 0
2 1 2 0 : R2 − 2 × R1 → 0 −3 −4 0 : R3 − R1 → 0 −3 −4 0
1 2 0 1 1 2 0 1 0 0 −3 1
   
1 2 3 0 1 0 1/3 0
: (−3)−1 R2 → 0 1 4/3 0 : R1 − 2 × R2 → 0 1 4/3 0
0 0 −3 1 0 0 −3 1
   
1 0 1/3 0 1 0 1/3 0
: (−3)−1 R3 → 0 1 4/3 0  : R2 − (4/3)R3 → 0 1 0 4/9 
0 0 1 −1/3 0 0 1 −1/3
 
1 0 0 1/9
: R1 − (1/3)R3 → 0 1 0 4/9 

0 0 1 −1/3
Let us use the above RREF to solve a system of equations:
Example 105. Consider
x1 + 2x2 + 3x3 = 0
2x1 + x2 + 2x3 = 0
x1 + 2x2 + 0x3 = 1
The system correspond to the RREF is:
x1 + 0 + 0 = 1/9
0 + x2 + 0 = 4/9
0 + 0 + x3 = −1/3
Hence
x1 = 1/9, x2 = 4/9, x3 = −1/3.
Check by substitution.
Theorem 106. Given a matrix A, if à is the matrix obtained after an elementary operation
on A, then à = P A for some invertible matrix P .
In particular, given a system of equations Ax = b, if [Ã, b̃] is the RREF of [A, b], then
for x ∈ Fn , Ax = b iff Ãx = b̃.
Theorem 107. The RREF of a matrix is unique.
Recall that we defined a square matrix A ∈ Fn×n to be invertible if there exists B ∈ Fn×n
such that AB = I = BA. The next theorem shows that for A ∈ Fn×n to be invertible it is
sufficient that there exists B ∈ Fn×n such that either AB = I or BA = I.

60
Theorem 108. Suppose A ∈ Fn×n . If there exists B ∈ Fn×n such that AB = I, then
BA = I and B is the inverse of A.

Proof. Suppose AB = I. Let B̃ := P B be the RREF of B, where P is the product of

invertible elementary matrices. Since B is a square matrix and AB = I, the columns of B
are linearly independent14 . Hence P B = I, and hence B = P −1 , so that B is invertible with
B −1 = P and BP = I. Therefore, ABP = A, that is, P = A and

BA = BP = I.

Theorem 109. Suppose A ∈ Fn×n . Suppose there exists B ∈ Fn×n such that BA = I. Then

1. Ax = 0 has only zero solution.

2. Ax = b has a unique solution for every b ∈ Fn×1 .

Proof. (1) Let x = [α1 · · · αn ]T ∈ F n×1 be such that Ax = 0. Then BAx = 0. Hence
x = BAx = 0.
(2) BA = I implies AB = I implies, for any b ∈ Fn×1 , ABb = b so that for x = Bb,
Ax = b. Also, Ax = b implies BAx = Bb implies x = Bb.

7.3 Inner product and norms of vectors

In the following, if we are not distinguishing Fn×1 and F1×n , then we denote it by Fn .
For x, y ∈ Fn , we define
n
X
hx, yi := xi ȳi .
i=1
n
Note that for x, y ∈ F ,
y ∗ x if x, y ∈ Fn×1 ,

hx, yi =
xy ∗ if x, y ∈ F1×n .
It can be verified that

1. hx, xi ≥ 0 for all x ∈ Fn , and hx, xi = 0 ⇐⇒ x = 0;

2. hx + u, yi = hx, yi + hu, yi for all x, y, u ∈ Fn ;

3. hx, yi = hy, xi for all x, y ∈ Fn ;

4. hαx, yi = αhx, yi for all x, y ∈ Fn and α ∈ F;

Definition 110. 1. The map (x, y) 7→ hx, yi is call an inner product on Fn .

2. Vectors x, y ∈ Fn are said to be orthogonal (to each other) if hx, yi = 0, and this fact
is also written as x ⊥ y.
14
If u1 , . . . , un are columns of B, and if α1 , . . . , αn are scalars such that α1 u1 + . . . + αn nun = 0, then
α1 Bu1 + . . . + αn nBun = 0. But, [Bu1 · · · Bun ] is the identity matrix. Hence, α1 = 0, α2 = 0, . . . , αn = 0

61
 3. A set S of vectors in Fn is said to be an orthogonal set if for any two distinct x, y ∈ S,
hx, yi = 0.

4. A set S of vectors in Fn is said to be an orthonormal set if it is orthogonal and

hx, xi = 1 for every x ∈ S.
C

If S ⊆ Fn and x ∈ Fn , then we write

x⊥S if x ⊥ y ∀ y ∈ S.

We shall denote
kxk := hx, xi1/2 , x ∈ Fn ,
and call it as the norm of x.
The following two theorems are immediate from the definition of the norm.

Theorem 111. Let x and y be vectors in Fn .

(i) (Pythagoras theorem) kx + yk2 + kx − yk2 = 2(kxk2 + kyk2 ).

(ii) (Polarization identity) ) If F = R, then

1
hx, yi = {kx + yk2 − kx − yk2 }
4
and if F = C, then
1
hx, yi = {kx + yk2 − kx − iyk2 + ikx + iyk2 − ikx − iyk2 }
4

Theorem 112. (Pythagoras theorem) If x and y are orthogonal vectors in Fn , then

kx + yk2 = kxk2 + yk2 .

Theorem 113. (Schwarz inequality) If x and y are vectors in Fn , then

|hx, yi| ≤ kxk kyk.

In particular,
n
X n
X n
1/2  X 1/2
2
|xi yi | ≤ |xi | |yi |2 .
i=1 i=1 i=1

Proof. Let x ∈ Fn and u ∈ Fn be such that kuk = 1. Then taking v = hx, uiu, we obtain

hx − v, ui = hx − hx, uiu, ui = 0.

⇒
hx − v, vi = 0.

62
Hence, by Pythagoras theorem,

kxk2 = k(x − v) + vk2 = kx − vk2 + kvk2 ≥ kvk2 .

That is,
|hx, ui| ≤ kxk
for all x ∈ Fn and for all u ∈ Fn with kuk = 1. If y 6= 0, then taking u = y/kyk, we have

|hx, yi| ≤ kxk kyk.

This inequality is true for y = 0 as well.

Theorem 114. (Triangle inequality) If x and y are vectors in Fn , then

kx + yk ≤ xk + kyk.

Proof. Let x and y are vectors in Fn . Then

kx + yk2 = hx + y, x + yi
= hx, xi + hx, yi + hy, xi + hy, yi
= kxk2 + 2Rehx, yi + kyk2
≤ kxk2 + 2kxk kykkyk2
= (kxk + kyk)2 .

7.4 Linear dependence and independence

Let A ∈ Fm×n and b ∈ Fm×1 . Suppose u1 , . . . , un are the columns of A. We observe that the
linear system
Ax = b
has a solution iff there exists α1 , . . . , αn ∈ F such that

α1 u1 + · · · + αn un = b.

That is, iff b is a linear combination of u1 , . . . , un .

Definition 115. A vector x ∈ Fn is said to be a linear combination of vectors v1 , . . . , vk

in Fn if there are α1 , . . . , αn ∈ F such that

x = α1 v1 + · · · + αn vn .

Definition 116. The set of all linear combinations of vectors v1 , . . . , vk in Fn is called the
span of v1 , . . . , vk , and in that case, this set is denoted by span{v1 , . . . , vk }. C

63
Definition 117. Given S ⊆ Fn , by span(S) we mean the set of all (finite) linear combinations
of vectors from S. C

• Given S ⊆ Fn and x ∈ Fn , x ∈ span(S) iff there exist v1 , . . . , vk in S such that

x ∈ span{v1 , . . . , vk }.

• If u1 , . . . , un are the columns of A, then Ax = b has a solution iff

b ∈ span{u1 , . . . , un }.

Definition 118. Vectors v1 , . . . , vk in Fn are said to be linearly dependent if at least one

of them is a linear combination of the remaining, that is, if there exists j ∈ {1, . . . , n} such
that
uj ∈ span{ui : i 6= j}.
C

• Vectors v1 , . . . , vk in Fn are linearly dependent iff there are α1 , . . . , αn ∈ F, with atleast

one of them nonzero, such that

α1 u1 + · · · + αn uk = 0.

Definition 119. A set S of vectors is said to be linearly dependent if there exists u ∈ S

such that
u ∈ span(S \ {u}),
that is u is a linear combination some finite number of vectors from S \ {u}. C

Definition 120. Vectors v1 , . . . , vk in Fn are said to be linearly independent if they are

not linearly dependent. C

• Vectors v1 , . . . , vk in Fn are linearly independent iff for any α1 , . . . , αn ∈ F,

α1 u1 + · · · + αn uk = 0 ⇒ α1 = 0, . . . , αk = 0.

Definition 121. A set S of vectors is said to be linearly independent if it is not linearly

dependent. C

• Every superset of a linearly dependent set is linearly dependent.

• Every subset of a linearly independent set is linearly independent.

• Any set of vectors containing 0 is linearly dependent.

   
1 1
Example 122. 1. Vectors u1 := and u2 := are linearly independent and
0 −1
their span is F2×1 .

64
     
1 1 3
2. Vectors u1 :=  0 , u2 :=  −1  and u3 :=  −2  are linearly dependent in F3×1 .
1 1 3
Theorem 123. Let S ⊆ Fn and x ∈ span(S). If x ⊥ y for all y ∈ S, then x = 0.
Proof. Let x ∈ span(S). Then there exists u1 , . . . , uk ∈ S and α1 , . . . , αk in F such that
x = α1 u1 + · · · + αk uk . Suppose x ⊥ y for all y ∈ S. Then
hx, xi = α1 hu1 , xi + · · · + αk huk , xi = 0.
Hence x = 0.
Theorem 124. Let u1 , . . . , uk be nonszero vectors in Fn which are orthogonal. Then they
are linearly independent.
Proof. Let α1 , . . . , αk in F be such that α1 u1 + · · · + αk uk = 0, that is ki=1 αi ui = 0. Then
P
for each j ∈ {1, . . . , k}, we have ’
k
DX E k
X
0= αi ui , uj = αi hui , uj i = αj huj , uj i.
i=1 i=1

Since uj 6= 0, αj = 0 for j = 1, . . . , k. Thus, u1 , . . . , uk are linearly independent.

Theorem 125. (Gram-Schimdt orthogonalization) Let u1 , . . . , uk be linearly indepen-
dent vectors in Fn . Let v1 = u1 and for j = 2, . . . , k, let
j−1
X huj , vi i
vj = uj − vi .
i=1
kvi k2

Then v1 , . . . , vk are (nonzero) orthogonal vectors and

span{v1 , . . . , vj } = span{u1 , . . . , uj } for j = 1, . . . , k.
hu2 ,v1 i
Proof. Note that v2 = u2 − kv1 k2 1
v so that

hu2 , v1 i
hv2 , v1 i = hu2 , v1 i − hv1 , v1 i = 0.
kv1 k2
Also, since u1 , u2 are linearly independent, v2 6= 0. After, obtaining orthogonal vectors
v1 , . . . , vj−1 such that span{v1 , . . . , v` } = span{u1 , . . . , v` } for ` = 1, . . . , j − 1, let
j−1
X huj , vi i
vj = uj − vi .
i=1
kvi k2

Then, for ` = 1, . . . , j − 1,
j−1
X huj , vi i huj , v` i
hvj , v` i = huj , v` i − hvi , v` i = huj , v` i − hv` , v` i = 0
i=1
kvi k2 kv` k2

and span{v1 , . . . , vj } = span{u1 , . . . , uj }.

65
      
1 1 0
Example 126. Let u1 :=  0 , u2 :=  −1  and u3 :=  1 . Note that
1 1 1
  
α1 + α2 0
α1 u1 + α2 u2 + α3 u3 = 0 ⇐⇒  −α2 + α3 = 0 
α1 + α2 + α3 0
⇒ α1 = 0, α2 = 0, α3 = 0. Thus, u1 , u2 , u3 are linearly independent. Now, v1 = u1 ,
hu2 , v1 i
v2 = u2 − v1 ,
kv1 k2
hu3 , v1 i hu3 , v2 i
v3 = u3 − 2
v1 − v2 .
kv1 k kv2 k2
Note that
kv1 k2 = ku1 k2 = 2, hu2 , v1 i = hu2 , u1 i = 2
so that  
0
hu2 , v1 i
v2 = u2 − v1 = u2 − v1 = u2 − u1 =  −1  ,
kv1 k2
0
and
kv2 k2 = 1, hu3 , v1 i = hu3 , u1 i = 1, hu3 , v2 i = −1,
so that       
0 1 0 −1/2
1 1
v3 = u3 − v1 + v2 =  1  −  0  +  −1  =  0  .
2 2
1 1 0 1/2
Thus we obtain the orthogonal vectors:
     
1 0 −1/2
u1 :=  0  , v2 :=  −1  , v3 :=  0  .
1 0 1/2
Definition 127. Let S be a set of vectors in Fn .
1. S is said to be an orthogonal set if x ⊥ y for every distinct x, y ∈ S.
2. S is said to be an orthonormal set if S is orthogonal and kxk = 1 for every x ∈ S.
C
Theorem 128. (Gram-Schimdt orthonormalization) Let u1 , . . . , uk be linearly inde-
pendent vectors in Fn . Let v1 = u1 and w1 = v1 /
v1 , and for j = 2, . . . , k, let
j−1
X vj
vj = uj − huj , wi iwi , wj := .
i=1
kvj k
Then w1 , . . . , wk are orthonormal vectors and
span{w1 , . . . , wj } = span{u1 , . . . , uj } for j = 1, . . . , k.

66
Remark 129. Gram-Schimdt orthogonalization and orthogonalization can be done even
when the given vectors u1 , . . . , uk are not linearly independent: If vj = 0 at some stage, then
remove uj and proceed. By this, we obtain a orthogonal (respectively, orthonormal) vectors
v1 , vn2 , . . . , vn` such that
span{v1 , vn2 , . . . , vn` } = span{u1 , . . . , uj } for j = 1, . . . , k.
C
Now, suppose u1 , . . . , un are orthonormal column vectors in Fn . Consider the matrix A
with columns u1 , . . . , uk , that is,
A = [u1 u2 · · · un ]
Then  
u∗1
 u∗2 
A∗ = 
 
.. 
 . 
u∗n
It can be seen that
 
u∗1 u∗1 u1 u∗1 u2 · · · u∗1 un
 

∗
 u∗2   u∗2 u1 u∗2 u2 · · · u∗2 un 
A A=  [u1 u2 · · · un ] =  =I
 
..  ··· ··· ··· ··· 
 . 
u∗n uk u1 u∗k u2
∗
· · · u∗n un

and  
u∗1
∗
 u∗2 
AA = [u1 u2 · · · uk ]   = u1 u∗1 + u2 u∗2 + · · · uk u∗k .
 
..
 . 
u∗k
Note that
 
ur1
 
ur1 ūr1 ur1 ūr2 · · · ur1 ūrn
 ur2   ur2 ūr1 ur2 ūr2 · · · ur2 ūrn 
ur u∗r =  [ūr1 ūr2 · · · ūrn ] =  .
 
..  ··· ··· ··· ··· 
 . 
urn urn ūr1 urn ūr2 · · · urn ūrn

Hence,

AA∗ = u1 u∗1 + u2 u∗2 + · · · un u∗n

Question: Can you verify that the above sum is the identity matrix?
Since A∗ A = I, by Theorem 108,
AA∗ = I.
In particular, proved the following theorem.
Theorem 130. If columns of a matrix A ∈ Fn×n are orthonormal, then A is a unitary
matrix.

67
7.5 RREF, Linear independence and rank
Let A ∈ Fm×n , and let à be the RREF of A.
Definition 131. The number of pivots in à is called the rank of A, and it is denoted by
rank(A). C

• Pivotal columns of à are linearly independent, and non-pivotal columns of à are linear
combinations of pivotal columns of Ã;

• Pivotal rows of Ã, which are the nonzero rows of P A, are linearly independent.

Suppose à is:  
1 0 1 0
0 1 2 0
0 0 0 1
In this case there
 are
 three pivotal columns, and hence the rank(A) =  3. Note that the non
1 1 0
pivotal column 2 is the linear combinations of the pivotal columns 0 and 1.
    
0 0 0
It can be seen that given any vectors v1 , . . . , vk in Fn×1 and α1 , . . . , αk in F, and an
invertible matrix B ∈ Fn×n ,

α1 v1 + . . . αk vk = 0 ⇐⇒ α1 Bv1 + . . . αk Bvk = 0.

Hence, v1 , . . . , vk are linearly independent ⇐⇒ v1 , . . . , vk are linearly independent.

We know that the RREF of a matrix A ∈ Fm×n can be written as P A, where ∈ Fm×m is
an invertible matrix. Hence, if A = [u1 u2 · · · un ], then P A = [P u1 P u2 · · · P un ]. Therefore,
the following quantities are equal:
• Number of pivotal columns in RREF of A,

• Number of nonzero rows RREF of A,

• Maximum number linearly independent columns of A,

• Maximum number linearly independent rows of A

 
1 0 1 0
Note that, in RREF 0 1
 2 0, number of pivotal columns is same as number of
0 0 0 1
nonzero rows.
Definition 132. Maximum number linearly independent columns of A is called the column
rank of A and the maximum number linearly independent rows of A is called the row rank
of A. C
Thus,
column rank of A = row rank of A = rank of A.

68
It also follows that
• rank(A) = rank(AT ) = rankA∗ ).
Theorem 133. Any n + 1 vectors in Fn×1 are linearly dependent.
Proof. Let u1 , . . . , un , un+1 are in Fn×1 . Consider the matrix A = [u1 . . . un un+1 ]. Then
there are non-pivotal columns for A, and every non-pivotal column is linear combination of
some of the pivotal columns. Hence, columns of A are linearly dependent.
Theorem 134. If u1 , . . . , un in Fn×1 are linearly independent, then Fn×1 = span{u1 , . . . , un }.
Proof. Suppose u1 , . . . , un in Fn×1 are linearly independent. Assume for a moment that
Fn×1 6= span{u1 , . . . , un }. Then there exists b ∈ Fn×1 such that u1 , . . . , un , b are linearly
independent. Let A = [u1 . . . , un b]. Then the RREF of A has n + 1 pivotal columns, which
is impossible.
Recall, that after Gram-Schmidt orthogonalization the following question was asked:
If u1 , . . . , un are orthonormal in Fn×1 , then is it true that
u1 u∗1 + · · · + un u∗n = I?

We knew that its answer is in affirmative. Now, we give another proof to it.
First observe that if u1 , . . . , un are orthonormal in Fn×1 , then
u1 u∗1 + · · · + un u∗n = I ⇐⇒ (u1 u∗1 (x) + · · · + (un u∗n )(x) = x
⇐⇒ u1 (u∗1 x) + · · · + un (u∗n x) = x
⇐⇒ hx, u1 iu1 + · · · + hx, un iun = x

So, it is enough to prove the following.

Theorem 135. If u1 , . . . , un are orthonormal in Fn×1 , then
x = hx, u1 iu1 + · · · + hx, un iun ∀ x ∈ Fn×1 .
Proof. Suppose u1 , . . . , un are orthonormal in Fn×1 . Then they are linearly independent in
Fn×1 . Let x ∈ Fn×1 and let A = [u1 u2 · · · un ] ∈ Fn×n . Since columns of A are linearly
independent, there is a unique vector y = [α1 α2 . . . αn ]T ∈ Fn×1 such that Ay = x. Thus,
we gave
α1 u1 + · · · + αn un = x.
Since u1 , . . . , un are orthonormal, for each j,
hα1 u1 + · · · + αn un , uj i = hx, uj i.
But, hα1 u1 + · · · + αn un , uj i = αj . Thus,
x = α1 u1 + · · · + αn un = hx, u1 iu1 + · · · + hx, un iun .
This completes the proof.

69
7.6 Determinants
 
a b
Recall from school that that the determinant of a 2 × 2 matrix is defined as
c d
 
a b 
 c d := ad − bc.
 

 
a11 a12 a13
Using this definition, you also know that the determinant of a 3 × 3 matrix a21 a22 a23 
a31 a32 a33
is defined as
 
a11 a12 a13       
  a22 a23  a21 a23  a21 a22 
a32 a33  − a12 a31 a33  + a13 a31 a32  .
a21 a22 a23  := a11      
 
a31 a32 a33 

Now we give a general definition of the determinant for any A ∈ Fn×n . First let us introduce
a few notations:
• Aij is the matrix obtained from A by deleting the i-th row and j-th column of A.
• For x ∈ Fn×n , Aj (x) is the matrix obtained from A by replacing its j-th by x.
Note that if A = [u1 u2 · · · un ], then A = Aj (uj ).
Definition 136. The determinant of A is defined iteratively by
n
X
det(A) = (−1)1+j a1j det(A1j ).
j=1

7.6.1 Properties
Some properties of determinant are list here.
n
X
1. For any i ∈ {1, . . . , n}, det(A) = (−1)i+j aij det(Aij ).
j=1

n
X
2. For any j ∈ {1, . . . , n}, det(A) = (−1)i+j aij det(Aij ).
i=1

3. If any of the columns or rows of A contains only zeros, then det(A) = 0.

This follows from (1)-(2).
4. For x, y ∈ Fn×1 , α ∈ F,

det(Aj (x + y)) = det(Aj (x)) + det(Aj (y)), det(Aj (αx)) = αdet(Aj (x)).

This follows from (2).

70
 5. If B is the matrix obtained from A by interchanging any tow rows or any two columns,
then det(B) = −det(A).

6. If any any two columns (or any two rows) of A are the same, then det(A) = 0:
Follows from (5).

7. If a scalar multiple of a column is added to another column, then the determinant is

unchanged:
Let A = [u1 u2 · · · un ] and let B be the matrix obtained from A by adding α times uk
to uj is Aj (uj + αuk ). Then B = Aj (uj + αuk ) so that from (4),

det(B) = det(Aj (uj + αuk ) = det(Aj (uj )) + αdet(Aj (uk )).

8. For any A, B ∈ Fn×n , det(AB) = det(A)det(B).

From the above, we have: If A is invertible, then det(A) 6= 0.

9. For any A ∈ Fn×n ,

A adj(A) = det(A)I,
where adj(A), called the adjugate of A, is the transpose of the matrix whose ij-th
entry is (−1)i+j det(Aij ).
From the above, we have: If det(A) 6= 0, then A is invertible. (8) and (9) imply:
1
10. For A ∈ Fn×n , A invertible iff det(A) 6= 0, and in that case A−1 = adj(A).
det(A)
It can be easily seen that:

• If A ∈ Fn×n is upper triangular or lower triangular, then determinant of A is the prod-

uct of the diagonal entries of A. In particular, for a diagonal matrix, the determinant
is the product of the diagonal entries.
   
1 1 1 −1 1 1
Example 137. Let A = 0 −1 1. Then M := ((−1)i+j det(Aij )) = −1 0 1 .
 1 0 1 2 −1 −1
−1 −1 2
Hence, adj(A) =  1 0 −1 .
1 1 −1

7.6.2 Solution of equations by Cramer’s rule

Theorem 138. (Cramer’s rule) If A ∈ Fn×n is an invertible matrix, b ∈ Fn×1 , and if
x = [x1 , · · · xn ]T is such that Ax = b, then

det(Aj (b))
xj = , j = 1, . . . , n.
det(A)

71
Proof. Let A = [u1 u2 · · · un ]. Then

Ax = b ⇐⇒ x1 u1 + x2 u2 + · · · + xn un = b ⇐⇒ (x1 u1 − b) + x2 u2 + · · · + xn un = 0.

Hence, the vector x1 u1 − b is a linear combination of u2 , . . . , un . Therefore

det[x1 u1 − b, u2 ··· un ] = 0,

that is,
x1 det[u1 u2 ··· un ] − det[b u2 ··· un ] = 0,
that is,
x1 det(A) − det[A1 (b)] = 0,
that is,
det[A1 (b)]
x1 = .
det(A)
det[Aj (b)]
Similarly, we obtain xj = det(A)
for j = 2, . . . , n.

Example 139. Consider the system Ax = b, where

   
1 0 0 1
A= 2  1 1 and b = 0 .

1 1 2 1

In this case, det(A) = 1, and hence, the solution [x1 , x2 , x3 ]T is given by

     
1 0 0 1 1 0 1 0 1
    
x1 = 0 1 1 = 1, x2 = 2 0 1 = −4, x3 = 2 1 0 = 2.
1 1 2 1 1 2 1 1 1

Clearly,     
1 0 0 1 1
2 1 1 −4 = 0
1 1 2 2 1

7.7 Inverse and solution of equations using RREF

Finding inverse and solving equations using determinants is not a viable method. It is
computationally expensive. Easier method is using RREF. We may observe that for an
invertible matrix A, its RREF Ã is the identity matrix. Hence, considering the augmented
matrix [A|I], if à = EA, then the RREF of the augmented matrix [Ã|E]. If à = EA = I,
then it follows that E = A−1 . Thus, In this case we have

[A|I] → (RREF ) → [Ã|E] = [I|A−1 ].

If à 6= I, then A is not invertible.

72
    
1 1 1 1 1 1 1 0 0
Example 140. Let 0 −1 1. Then the augmented matrix [A|I] is 0 −1 1 0 1 1.
1 0 1 1 0 1 0 0 1
Row operations on it lead to:
   
1 1 1 1 0 0 1 1 1 1 0 0
0 −1 1 0 1 1 → 0 1 −1 0 −1 1
1 0 1 0 0 1 0 −1 0 −1 0 1
   
1 0 2 1 1 0 1 0 0 −1 −1 2
→ 0 1 0 1 0 −1 → 0 1 0 1 0 −1 .
0 0 1 1 1 −1 0 0 1 1 1 −1
Thus,  
−1 −1 2
A−1 = 1 0 −1 .
1 1 −1
   
1 1 3 1 1 3 1 0 0
Example 141. Let 0 1 1. Then the augmented matrix [A|I] is 0 1 1 0 1 0.
1 1 3 1 1 3 0 0 1
Row operations on it lead to:
   
1 1 3 1 0 0 1 0 2 1 −1 0
0 1 1 0 1 0 → 0 1 1 0 1 0
1 0 0 −1 0 1 0 0 0 −1 0 1

This shows that A is not invertible.

Similarly, for solving the system Ax = b with A ∈ Fm×n and b ∈ Fm , we may apply
elementary operations:
Ax = b ' EAx = Eb.
Hence, if m = n and EA = I, then E = A−1 and x = Eb = A−1 b. In this case,

[A|b] ' [EA|Eb] ⇐⇒ [I|Eb] = [I|A−1 b].

Otherwise, a solution need not exist or there can be infinitely many solutions.

• In case there is a non-zero entry in Eb below the last pivot in EA, then there is no
solution for the system.

• In case there is a zero row in (EA|Eb) and there is no non-zero entry in Eb below the
last pivot in EA, then there are infinitely many solutions.

Example 142. Consider the system

    
1 1 1 x1 1
0 −1 1 x2  = 1 .
1 0 1 x3 1

73
Then the augmented matrix:
       
1 1 1 1 1 1 1 1 1 0 2 2 1 0 0 0
0 −1 1 1 → 0 −1 1 1 → 0 1 −1 −1 → 0 1 0 0
1 0 1 1 0 −1 0 0 0 0 1 1 0 0 1 1
Hence x = [0 0 1]T .
Let A = [u1 u2 · · · un ] ∈ Fm×n . Let us observe the following facts:
• Ax = b has a solution iff b is a linear combination of the columns of A. Thus,
• Ax = b has a solution iff rank(A) = rank[A|b].
• Suppose Ax = b has a solution, that is, rank(A) = rank[A|b]. Then Ax = b has a
unique solution iff 0 is the only solution of Ax = 0.
• 0 is the only solution of Ax = 0 iff α1 u1 + · · · αn un = 0 implies α1 = 0, . . . , αn = 0 iff
columns of A are linearly independent. Hence,
• Ax = b has a unique solution iff rank(A) = rank[A|b] = n.
• If rank(A) = rank[A|b] = r < n, then Ax = b has infinitely many solution, which are
obtained by assigning values for n − r free variable.
Example 143. Consider the system
    
1 1 0 x1 1
0 −1 1 x2  = 1 .
1 0 1 x3 1
 
1 1 0 1
Let us find the RREF of the augmented matrix [A|b]: 0 −1 1
 1.
1 0 1 1
       
1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 2
0 −1 1 1 → 0 −1 1 1 → 0 1 −1 −1 → 0 1 −1 −1
1 0 1 1 0 −1 1 0 0 −1 1 0 0 0 0 −1
This shows that ranl(A) 6= rank[A|b], and hence the system has no solution.
Example 144. Consider the system
    
1 1 0 x1 2
0 −1 1 x2  = −1 .
1 0 1 x3 1
 
1 1 0 2
Let us find the RREF of the augmented matrix [A|b]: 0 −1
 1 −1.
1 0 1 1
       
1 1 0 2 1 1 0 1 1 1 0 2 1 0 1 1
0 −1 1 −1 → 0 −1 1 −1 → 0 1 −1 1 → 0 1 −1 1
1 0 1 1 0 −1 1 −1 0 −1 1 −1 0 0 0 0

74
This shows that ranl(A) = ranl[A|b], and hence the system has a solution. From the above
we get:
x1 + x3 = 1, x2 − x3 = 1 so that x1 = 1 − x3 , x2 = 1 + x3
Hence the set of solutions is: S := {(2 − α, 1 + α, α) : α ∈ F}.

Example 145. Consider the system

    
1 1 1 x1 0
0 −1 1 x2  = 1 .
1 0 1 x3 1
 
1 1 1 0
Let us find the RREF of the augmented matrix [A|b]: 0 −1 1 1.
1 0 1 1
     
1 1 1 0 1 1 1 0 1 1 1 0
0 −1 1 1 → 0 −1 1 1 → 0 1 −1 −1 .
1 0 1 1 0 −1 0 1 0 −1 0 1
     
1 1 1 0 1 0 2 1 1 0 0 1
→ 0 1 −1 −1  →  0 1 −1 −1  →  0 1 0 −1
0 0 −1 0 0 0 1 0 0 0 1 0
This shows that ranl(A) = rank[A|b] = n. Hence, the system has a unique solution. We
obtain:
x1 = 1, x2 = −1, x3 = 0.

75
7.8 Eigenvalues and eigenvectors
Let A ∈ Fn×n .

Definition 146. A scalar λ ∈ F is called an eigenvalue of A if there is a nonzero vector

v ∈ Fn× such that Av = λv, and in that case v is called an eigenvector of A corresponding
to the eigenvalue λ. . C

Note that: For λ ∈ F,

λ is an eigenvalue of A ⇐⇒ ∃ v 6= 0 such that (A − λI)v = 0

⇐⇒ columns of A − λI are linearly dependent
⇐⇒ det(A − λI) = 0.

Definition 147. The polynomial q(λ) := det(A − λI) is called the characteristic poly-
nomial of A, and the zeros of q are called the characteristic values or characteristic
roots of A. C

Thus,

• λ ∈ F is an eigenvalue of A ⇐⇒ λ is a characteristic value of A.

If F = R, then a matrix A ∈ Rn×n need not have an eigenvalue.

 
0 1
For example, consider the matrix A := . Then, A as a real matrix does not have
−1 0
any eigenvalue, as  
−λ 1
det(A − λI) = det = λ2 + 1
−1 −λ
does not have any real zeroes.
However, A as a complex matrix has two distinct eigenvalues, namely, λ1 = i and λ2 = −i.
The eigenvectors are obtained by solving the equation
    
−λ 1 x1 0
=
−1 −λ x2 0

for λ = i and λ = −i. Thus:

    
−λ 1 x1 0
=
−1 −λ x2 0
⇐⇒

−λx1 + x2 = 0 & − x1 − λx2 = 0 ⇐⇒ x1 = −λx2 = −λ2 x1 ⇐⇒ (1 + λ2 )x1 = 0

   
2 1 1
Since 1 + λ = 0, x1 6= 0. Thus for x1 = 1 implies x2 = λx1 = λ. Hence, and
i −i
are eigenvectors corresponding to the eigenvalues i and −i respectively.

76
Theorem 148. (Fundamental theorem of algebra) Every polynomial of degree n with
complex coefficients has atleast one zero, and atmost n zeros.
It can be shown that:
• If λ := α + iβ with α, β ∈ R and β 6= 0 is a complex zero of a polynomial with real
coefficients, then its complex conjugate, λ̄ := α−iβ is also zero of the same polynomial.
Thus,
• Every complex matrix has at least one eigenvalue.
In this section, we shall consider all matrices as complex matrices.
The following results can be verified easily:
• For an upper triangular or lower triangular matrix, the diagonal entries are the eigen-
values. In particular, for a diagonal matrix, the diagonal entries are the eigenvalues.
• Eigenvalues of a matrix and its transpose are the same.
• λ is an eigenvalue of A iff λ̄ is an eigenvalue of A∗ .
• If λ := α + iβ with α, β ∈ R and β 6= 0 is an eigenvalue of a matrix A with real entries,
then λ̄ := α − iβ is also an eigenvalue of A.
• Similar matrices have the same eigenvalues. (Recall that A is similar to B iff there
exists an invertible matrix P such that B = P −1 AP .
• det(A) is the product of characteristic values and trace(A) is the sum of the charac-
teristic values.
Theorem 149. (Caley-Hamilton theorem) If p(λ) = det(A − λI), the characteristic
polynomial, then then p(A) = 0.
Proof. Recall that
p(λ)I = det(A − λI)I = (A − λI)adj(A − λI),
where adj(A − λI) is a matrix of the form B0 + λB1 + · · · + λn−1 Bn−1 . Thus,
p(λ)I = (A − λI)(B0 + λB1 + · · · + λn−1 Bn−1 )
and hence p(A) = 0.
Recall that p(λ) = det(A − λI) is a polynomial of degree n in the variable λ, which is of
the form
p(λ) = (−1)n (λn + a1 λn−1 + · · · + an−1 λ + an ).
By Caley-Hamilton theorem, p(A) = 0 so that
An + a1 An−1 + · · · + an−1 Al + an I = 0.
Hence,
An = −(a1 An−1 + · · · + an−1 Al + an I),
that is An is a linear combination of I, A, A2 , . . . , An−1 .

77
Definition 150. Let λ be an eigenvalue of A. The multiplicity of λ as the zero of the
characteristic polynomial of A is called the algebraic multiplicity of the eigenvalue λ. C
 
1 0 0
Example 151. Let A :=  1 1 0  . The characteristic polynomial for this is:
1 0 2
 
1−λ 0 0
det(A − λ) := det  1 1−λ 0  = (1 − λ)2 (2 − λ).
1 0 2−λ
Hence λ = 1 is an eigenvalue of algebraic multiplicity 2, and λ = 2 is an eigenvalue of
algebraic multiplicity 1.

7.8.1 Eigenvalues of hermitian, normal and unitary matrices

Recall that for A ∈ Fn×n and λ ∈ F,
det(A∗ − λ̄I) = det[(A − λI)∗ ] = det(A − λI)
so that λ is an eigenvalue of A iff λ̄ is an eigenvalue of A∗ .
Suppose A is hermitian matrix, i.e., A∗ = A and let λ ∈ F be an eigenvalue of A with a
corresponding eigenvector v ∈ Fn×1 . Then
hAv, vi = hλv, vi = λhv, vi,
hAv, vi = hv, A∗ vi = hv, Avi = hv, λvi = λ̄hv, vi.
Thus, λhv, vi = λ̄hv, vi. Since hv, vi =
6 0, we have λ̄ = λ. Thus,
• Eigenvalues of hermitian matrix are real.
Suppose A is a normal matrix, i.e., A∗ A = AA∗ . For λ ∈ F and v ∈ Fn×1 , we have
k(A − λI)vk2 = h(A − λI)v, (A − λI)vi = h(A∗ − λ̄I)(A − λI)v, vi.
Since A is normal, (A∗ − λ̄I)(A − λI) = (A − λI)(A∗ − λ̄I). Therefore,
k(A − λI)vk2 = h(A∗ − λ̄I)(A − λI)v, vi = h(A − λI)(A∗ − λ̄I)v, vi = k(A∗ − λ̄I)vk2 .
Hence,
• If A is a normal matrix, i.e., A∗ A = AA∗ , then for every λ ∈ F and v ∈ Fn×1 ,
kAv − λvk = kA∗ v − λ̄vk.
In particular, for a normal matrix A,
Av = λv ⇐⇒ A∗ v = λ̄v.
Also, if A is a normal matrix, λ, µ ∈ F and u, v ∈ Fn×1 , Au = λu, Av = µv implies
∗
λhu, vi = hλu, vi = hAu, vi = hu, A vi = hu, µ̄vi = µ̄hu, vi.
Hence,
• If A is a normal matrix, and if λ, µ ∈ F and u, v ∈ Fn×1 such that Au = λu and
6 µ, then u ⊥ v. In particular,
Av = µv, and if λ =
• eigenvectors associated with distinct eigenvalues of a normal matrix are orthogonal.

78
7.9 Eigenvalue representations - diagonalization theorems
Theorem 152. If A ∈ Fn×n is a normal matrix having n distinct eigenvalues, then there
exists a unitary matrix U ∈ Fn×n such that U −1 AU is a diagonal matrix with diagonal entries
as the eigenvalues of A.

Proof. Suppose A has n distinct eigenvalues, say λ1 , . . . , λn . Let u1 , . . . , un be orthonormal

eigenvectors of A corresponding the the eigenvalues λ1 , . . . , λn , respectively. Take U =
[u1 u2 · · · un ]. Then diag(λ1 , . . . , λn ) is the required diagonal matrix.
Suppose A is a unitary matrix. Let λ ∈ F be an eigenvalue of A with a corresponding
eigenvector v ∈ Fn×1 . Then

v = A∗ Av = A∗ (λv) = λA∗ v = λλ̄v = |λ|2 v.

Thus (1 − |λ|2 )v = 0. Since v 6= 0, |λ| = 1. Thus,

• If λ is an eigenvalue of a unitary matrix, then |λ| = 1.

Theorem 153. Let A ∈ Fn×n . Then eigenvectors associated with distinct eigenvalues of A
are linearly independent.

Corollary 154. If A ∈ Fn×n has n distinct eigenvalues, then there exists an invertible matrix
P ∈ Fn×n such that P −1 AP is a diagonal matrix with its diagonal entries as the eigenvalues
of A.

Proof. Suppose A has n distinct eigenvalues, say λ1 , . . . , λn . Let u1 , . . . , un be eigenvectors

of A corresponding the the eigenvalues λ1 , . . . , λn , respectively. Take P = [u1 u2 · · · un ].
Then diag(λ1 , . . . , λn ) is the required diagonal matrix.
In the above corollary, we assumed that A has n distinct eigenvalues, and then used
eigenvectors corresponding to these eigenvalues to construct the matrix P . So, essentially
we have proved the follwing theorem.

Theorem 155. Suppose A ∈ Fn×n is such that there are n linearly independent vectors
v1 , . . . , vn which are eigenvectors of A corresponding to eigenvalues λ1 , . . . , λn . (Note that
there may be repetition in the list λ1 , . . . , λn .) Let P = [u1 u2 · · · un ]. Then P is invertible
and
P −1 AP = diag(λ1 , . . . , λn ).

Note that, for a matrix A ∈ Fn×n , if there exists an invertible P such that P −1 AP is a
diagonal matrix, say P −1 AP = diag(λ1 , . . . , λn ) =: D, then, writing P = [u1 u2 · · · un ], we
have
AP = P D
so that Auj = λj uj for j = 1, . . . , n, that is, u1 , u2 · · · un are eigenvectors of A which are
linearly independent.

Definition 156. A matrix A ∈ Fn×n is said to be diagonalizable if there exists an invertible

P such that P −1 AP is a diagonal matrix. C

79
 Thus we have already proved the following theorem.
Theorem 157. A matrix A ∈ Fn×n is diagonalizable iff there are linearly independent vectors
u1 , u2 , · · · , un which are eigenvectors of A.
Theorem 158. (Diagonalization theorem or Spectral theorem)
(i) If A ∈ Cn×n is a normal matrix, then there exists a unitary matrix U ∈ Fn×n such that
U −1 AU is a diagonal matrix with diagonal entries as the eigenvalues of A.
(ii) If A ∈ Rn×n is a (real) symmetric matrix, then there exists a unitary matrix U ∈ Fn×n
such that U −1 AU is a diagonal matrix with diagonal entries as the eigenvalues of A.
 
1 −1 −1
Example 159. Let A :=  −1 1 −1  . The characteristic polynomial for this
−1 −1 1
 
1−λ −1 −1
det(A − λ) := det  −1 1 − λ −1  = (−1)(λ + 1)(λ − 2)2 .
−1 −1 1 − λ
Hence λ = 2 is an eigenvalue of algebraic multiplicity 2, and λ = −1 is an eigenvalue of
algebraic multiplicity 1. Let us check whether A has 3 linearly independent eigenvectors:
Let [α, β, γ]T be an eigenvector of A corresponding to the eigenvalue 2. Then

α − β − γ = 2α (1)
−α + β − γ = 2β (2)
−α − β + γ = 2γ. (3)

(1) + (2) ⇒ −2γ = 2(α + β); (2) + (3) ⇒ −2α = 2(β + γ);
(1) + (3) ⇒ −2β = 2(α + γ). Thus,

−γ = (α + β).

Thus, u1 = [1, 0, −1]T is an eigenvector corresponding to the eigenvalue λ = 2. Now, we see

that u2 = [1, −2, 1]T is also an eigenvector corresponding to the eigenvalue λ = 2. Note that
hu1 , u2 i = 0. Thus, v1 = √12 [1, 0, −1]T and v2 = √16 [1, −2, 1]T are orthonormal eigenvectors
corresponding to the eigenvalue λ = 2.
Let [a, b, c]T be an eigenvector of A corresponding to the eigenvalue −1. Then

a − b − c = −a (1)
−a + b − c = −b (2)
−a − b + c = −c. (3)

That is, 2a − b − c = 0, −a + 2b − c = 0, −a − b + 2c = 0.

(1) − (2) ⇒ 3a − 3b = 0 ⇒ a = b; ⇒ c = a.

80
Thus, v3 = √1 [1, 1, 1]T is an eigenvector corresponding to the eigenvalue λ = −1. Hence,
3
 1 1 1

√ √ √
2 6 3
U = [v1 v2 v3 ] = 
 0 − √26 √1
3


−1
√ √1 √1
2 6 3

is a unitary matrix such that U −1 AU is the diagonal matrix diag(2, 2, −1).

Note that after obtaining two or more linearly independent eigenvectors corresponding to
a single eigenvalue, they may be orthogonalized to obtain orthonormal eigenvectors. In the
above case, u = [1, 0, −1]T is an eigenvector corresponding to the eigenvalue λ = 2. Also,
v = [0, 1, −1]T is another eigenvector corresponding to the eigenvalue λ = 2. Let ũ = u and
1 T 1 T 1
ṽ := v − hv, uiu = [0, 1, −1] − [1, 0, −1] = [−1, 2, −1]T
kuk2 2 2
Thus, we obtain the orthonormal eigenvectors √12 [1, 0, −1]T and √16 [1, −2, 1]T corresponding
to the eigenvalue λ = 2.
 
1 −1 1
Example 160. Let A :=  −1 1 −1  . Let λ be an eigenvalue of A with a correspond-
1 −1 1
T
ing eigenvector x = [α, β, γ] . Then
α − β + γ = λα (1)
−α + β − γ = λβ (2)
α − β + γ = λγ. (3)

• Since l.h.s of (1) and (3) are the same, we obtain λ(α − γ) = 0.
• Adding (1) and (2) we get 0 = λ(α + β).
Thus, if λ 6= 0, then α = γ = −β, and in that case λ = 3. Hence,
• [1, −1, 1]T is an eigenvector of A corresponding to the eigenvalue λ = 3.
If λ = 0, then we get α = β − γ. Thus,
• [1, 2, 1]T and [1, 0, −1]T are the orthonormal eigenvectors corresponding to the eigen-
value λ = 0. Thus,
• √1 [1, −1, 1]T , √1 [1, 2, 1]T and √1 [1, 0, −1]T are the orthonormal eigenvectors of A.
3 6 2

• Taking U with these eigenvectors as columns we obtain the diagonal matrix AU = U D,

where D is the diagonal matrix diag(3, 0, 0).
That is,
 −1    √1 
√1 √1 √1 √1 √1
 
3 6 2 1 −1 1 3 6 2 3 0 0

 − √13 √2
6
0 
  −1 1 −1   − √13
 √2
6
0 
 =  0 0 0 .
√1 √1 − √12 1 −1 1 √1 √1 − √12 0 0 0
3 6 3 6

81