CHAPTER 1.

INTRODUCTION
Differentiation (I)

• Product rule
(fg )0 = f 0 g + g 0 f .
so
(fgh)0 = f 0 (gh) + (gh)0 f = f 0 gh + g 0 hf + h0 gf .
Example
d
(x − x1 )(x − x2 )(x − x3 ) = (x − x2 )(x − x3 ) + (x − x1 )(x − x3 )
dx
+(x − x1 )(x − x2 ).
Differentiation (II)

n
Y
(x − xi ) = (x − x1 )(x − x2 ) . . . (x − xn ).
i=1

Then
n
d Y
(x − xi ) = (x − x2 )(x − x3 ) . . . (x − xn ) +
dx
i=1
(x − x1 )(x − x3 ) . . . (x − xn ) + . . . +
(x − x1 )(x − x2 ) . . . (x − xi−1 )(x − xi+1 ) . . . (x − xn ) + . . .
+(x − x1 )(x − x2 ) . . . (x − xn−1 ).
O notation (I)

Let h be a quantity. Then we say that a function p(h) is O(h) if

there is a constant C independent of h such that:

|p| ≤ C |h|.

Examples: 2h = O(h).

Let n be a positive integer. Then

2 1 n+1 1
= O( ), and = O( )
n n n2 n
because n + 1 1 1 2
 2 = + 2 ≤ .
 
n n n n
O notation (II)

If |h| < 1 then |h2 | < |h|, |h3 | < |h2 |, ....Then

|ah + bh2 | ≤ |ah| + |bh2 | ≤ (|a| + |b|)|h|,

so ah + bh2 is O(h).
Taylor expansion (I)

f 00 (x0 ) f (n) (x0 )

f (x) = f (x0 )+f 0 (x0 )(x−x0 )+ (x−x0 )2 +. . .+ (x−x0 )n +. . . .
2! n!
The finite form
f 00 (x0 )
f (x) = f (x0 ) + f 0 (x0 )(x − x0 ) + (x − x0 )2 + . . . +
2!
f (n−1) (x0 ) f (n) (ξ)
(x − x0 )n−1 + (x − x0 )n ,
(n − 1)! n!

where ξ is between x and x0 : i.e. if x < x0 then x ≤ ξ ≤ x0 ; if

x0 < x then x0 ≤ ξ ≤ x.
Taylor expansion (II)

Taylor expansion converges if |x − x0 | < a for a fixed a; a is the

radius of convergence. Sometimes a can be infinity.
Example:
x2 x3
ex = 1 + x + + + ....
2! 3!
This holds for all x; here x0 = 0 and a = ∞.

x2 x3
ln(1 + x) = x − + − ...;
2 3
this holds for |x| < 1 i.e. a = 1.
There are tests to determine the radius of convergence, but this is
beyond the scope of this course.
Taylor expansion (III)

When |x − x0 | < 1, then

f (x) = f (x0 ) + O(x − x0 )

= f (x0 ) + f 0 (x0 )(x − x0 ) + O((x − x0 )2 )
f 00 (x0 )
= f (x0 ) + f 0 (x0 )(x − x0 ) + (x − x0 )2 + O((x − x0 )3 ).
2
We therefore can take

f (x0 ), f (x0 )+f 0 (x0 )(x−x0 ), f (x0 )+f 0 (x0 )(x−x0 )+(f 00 (x0 )/2)(x−x0 )2 , . . .

as approximations for f (x).

Integration

Definition of integration:

Divide an interval [a, b] into n parts by

a = x0 < x1 < . . . < xn−1 < xn = b.
Pick up a point ξi in (xi−1 , xi ).
Then
Z b Xn
f (x)dx = lim f (ξi )(xi − xi−1 ).
a max |xi −xi−1 |→0
i=1
About numerical analysis (I)

• Example:
Z 1 1
x x
e dx = e  = e − 1 = 1.718281828.
0 0

But for Z 1
2
e x dx,
0
we cannot find it exactly. We will need to find approximations.

Numerical analysis uses mathematical analysis to approximate

quantities which cannot be, or are difficult to be, computed exactly.
About numerical analysis (II)

Coming back to the example: from the definition of integrals, if we

divide [0, 1] into 1 interval (itself) i.e. x0 = 0, x1 = 1. Pick up
ξ1 = 1/2 then
Z 1
2
e x ≈ e 1/4 = 1.284025.
0
If we divide [0, 1] into 2 intervals, i.e. x0 = 0, x1 = 1/2 and
x2 = 1. Pick up ξ1 = 1/4 and ξ2 = 3/4, then
Z 1
2 1
e x ≈ (e 1/16 + e 9/16 ) = 1.40977.
0 2

Dividing [0, 1] into three intervals [0, 1/3], [1/3, 2/3] and [2/3, 1],
and pick up ξ1 = 1/6, ξ2 = 1/2 and ξ3 = 5/6, then
Z 1
2
e x ≈ 1.43826
0
About numerical analysis (III)
The more points we take, the better. To illustrate this, consider
another simple example
Z 1
x 2 dx = 1/3 = 0.333333.
0

Dividing [0, 1] into one interval, and take ξ1 = 1/2, then

Z 1
1
x 2 ≈ ( )2 = 0.25.
0 2
Dividing [0, 1] into two intervals as above, then
Z 1
1 1 2 3 
x 2 dx ≈ ( ) + ( )2 = 0.3125.
0 2 4 4
Dividing [0, 1] into three intervals as above, then
Z 1
x 2 dx ≈ 0.324074.
0
About numerical analysis (IV)

We do not need to choose xi so that the length of (xi−1 , xi ), i.e.

|xi − xi−1 |, is exactly the same for all i.

Later on, we will see that it is better that these lengths are not
equal, and we have special ways to choose ξi .

In this course, we will learn how to approximate roots of an

equation, approximate the values of a function with limited
information, approximate derivatives and integrals, etc.
Error (I)

If p̄ is an approximation for p, then the:

Absolute error is defined as |p − p̄|.

Relative error is defined as |p − p̄|/|p|.

R1
Example: for the integral 0 x 2 dx = 1/3. For the approximated
value 0.3125, the error is

|1/3 − 0.3125| = 0.020833,

and the relative error is

|1/3 − 0.3125|/(1/3) = 0.0625.

Error (II)

In practice, we do not know the exact quantity p (otherwise there

is no need to find an approximation). We can only estimate the
error, though this task may be difficult.

For example, from Taylor expansion:

f (x + h) = f (x) + f 0 (x)h + O(h2 ),

if we only know f (x) and f 0 (x) and nothing else, we can say that
f (x) + f 0 (x)h approximates f (x + h) with the error

e = f (x + h) − [f (x) + f 0 (x)h],

which is bounded by Ch2 when h → 0, for a constant C .

Why relative error

How can we say which approximation is better?

If p = 100, if p̄1 and p̄2 are approximations with errors 1 and 2

respectively, then obviously p̄1 is better.

If q = 1 and if q̄ is an approximation with error 0.1, then how can

we compare p̄1 and q̄?

We need to compare the error to the size of the exact quantity,

that is the relative error.
For p̄1 , the relative error is 1/100 = 0.01, i.e. the error is 1%. For
q̄, the relative error is 0.1/1 = 0.1, i.e. the error is 10%. So the
approximation p̄1 for p is better than q̄ for q.
Computer rounding off errors

We can only work with a finite number of digits.

There are two methods to take a finite number of digits: chopping
and rounding off.

Consider π = 3.14159265.... We want to take 5 digits.

For chopping, we take 3.1415.

For rounding off, we take 3.1416 as the next digit is 9, which is
large.

In any case, chopping and rounding off always introduce an error,

which we call (for both cases) rounding off errors.
Floating point

• Floating point numbers of k digits:

±0.d1 d2 . . . dk × 10n , 1 ≤ d1 ≤ 9, 0 ≤ di ≤ 9 for i > 1.

For example: 3.1415 = 0.31415 × 10, 3.1416 = 0.31416 × 10.

Given a number y = 0.d1 d2 . . . dk dk+1 . . . × 10n .

Floating point number: for chopping we just take the first k digits:

0.d1 d2 . . . dk × 10n .

For rounding off: if dk+1 < 5 then we do exactly as chopping, if

dk+1 ≥ 5, we add 1 into dk .
Loss of accuracy due to rounding off

• Solve ax 2 + bx + c = 0, a = 1, b = 62.1, c = 1.
The exact roots are
√ √
−b + b 2 − 4ac −b − b 2 − 4ac
x1 = , x2 = ,
2a 2a
i.e. x1 = −0.01610723, x2 = −62.08390.
√ √
• Taking 4 digits: b 2 − 4ac = 3852 = 62.06.
so
−62.1 + 62.06
x1 = = −0.0200.
2
We have lost some accuracy.
√ √
If we take 6 digits, i.e. b 2 − 4ac = 3852.41 = 62.0678, then
x1 = (−62.1 + 62.0678)/2 = −0.016100.
Reformulate to avoid loss of accuracy due to rounding off

Sometimes, we can reformulate the problem to get better accuracy.

We can rewrite x1 √
as (by multiplying both the denominator and
numerator by b + b 2 − 4ac)

−2c
x1 = √ .
b + b 2 − 4ac
√ √
Then if we only take 4 digits: b 2 − 4ac = 3852 = 62.06, then
−2.000
x1 = = −0.01611.
62.10 + 62.06
Algorithm

• An algorithm is a procedure describing steps to be performed.

• Example: computing the sum x1 + x2 + . . . + xN .

• Algorithm:

- Take the first number x1 and add x2

- Note the sum
- Add x3 to the sum
- Note the new sum
- Continue until xN .
Pseudocode

• We describe this algorithm by a pseudocode

Step 1: INPUT N, x1 , x2 ,. . . , xN .
Step 2: Set SUM = 0.
Step 3: For i = 1, 2, . . . , N
Set SUM = SUM + xi .
Step 4: OUTPUT SUM.

We can implement this in a computer language such as Fortran,

C++, Matlab, ...
Rate of convergence (I)

• Let {βn } be a sequence that converges to 0, and {αn } be a

sequence that converges to α. If there is a constant K such that

|αn − α| ≤ K |βn |,

then we say that αn converges to α with the rate of convergence

O(βn ).

• Normally, βn is of the form

1
βn = .
np
Rate of convergence (II)

Consider
n2 + n + 1
αn = .
n2
Then αn → 1 as n → ∞.
n+1 1 1 2
|αn − 1| = 2
= + 2 ≤ ,
n n n n
so |αn − 1| = O(1/n) – the rate of convergence is O(1/n).

Consider
n3 + n + 1
α̂n = .
n3
Then
n+1 1 1 2
|α̂n − 1| = 3
= 2 + 3 ≤ 2.
n n n n
The rate of convergence is O(1/n2 ).
Rate of convergence (III)
For functions: Suppose that limh→0 G (h) = 0 and
limh→0 F (h) = L. If there is a positive constant K such that

|F (h) − L| ≤ K |G (h)|,

then we say that F (h) converges to L with the rate O(G (h)).

For example: From Taylor expansion:

f 00 (0) 2 f 000 (ξ) 3

f (h) = f (0) + f 0 (0)h + h + h .
2 6
Assume that |f 000 (ξ)| ≤ K for all ξ. Then

f 00 (0) 2 K
|f (h) − [f (0) + f 0 (0)h + h ]| ≤ |h3 |.
2 6
so f (h) − f 0 (0)h − (f 00 (0)/2)h2 converges to f (0) with the rate
O(h3 ).