Introduction to Numerical Methods

and Matlab Programming for Engineers
Todd Young and Martin J. Mohlenkamp
Department of Mathematics
Ohio University
Athens, OH 45701
young@math.ohiou.edu
August 27, 2011
ii
Copyright c _ 2008, 2009, 2011 Todd Young and Martin J. Mohlenkamp.
Original edition 2004, by Todd Young. Permission is granted to copy, distribute
and/or modify this document under the terms of the GNU Free Documenta-
tion License, Version 1.3 or any later version published by the Free Software
Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-
Cover Texts. A copy of the license is included in the section entitled “GNU
Free Documentation License”.
Preface
These notes were developed by the first author in the process of teaching a course on applied
numerical methods for Civil Engineering majors during 2002-2004 and was modified to
include Mechanical Engineering in 2005. The materials have been periodically updated
since then and underwent a major revision by the second author in 2006-2007.
The main goals of these lectures are to introduce concepts of numerical methods and intro-
duce Matlab in an Engineering framework. By this we do not mean that every problem
is a “real life” engineering application, but more that the engineering way of thinking is
emphasized throughout the discussion.
The philosophy of this book was formed over the course of many years. My father was a
Civil Engineer and surveyor, and he introduced me to engineering ideas from an early age.
At the University of Kentucky I took most of the basic Engineering courses while getting
a Bachelor’s degree in Mathematics. Immediately afterward I completed a M.S. degree
in Engineering Mechanics at Kentucky. While working on my Ph.D. in Mathematics at
Georgia Tech I taught all of the introductory math courses for engineers. During my
education, I observed that incorporation of computation in coursework had been extremely
unfocused and poor. For instance during my college career I had to learn 8 different
programming and markup languages on 4 different platforms plus numerous other software
applications. There was almost no technical help provided in the courses and I wasted
innumerable hours figuring out software on my own. A typical, but useless, inclusion of
software has been (and still is in most calculus books) to set up a difficult ‘applied’ problem
and then add the line “write a program to solve” or “use a computer algebra system to
solve”.
At Ohio University we have tried to take a much more disciplined and focused approach.
The Russ College of Engineering and Technology decided that Matlab should be the
primary computational software for undergraduates. At about the same time members of
the Mathematics Department proposed an 1804 project to bring Matlab into the calculus
sequence and provide access to the program at nearly all computers on campus, including
in the dorm rooms. The stated goal of this project was to make Matlab the universal
language for computation on campus. That project was approved and implemented in the
2001-2002 academic year.
In these lecture notes, instruction on using Matlab is dispersed through the material on
numerical methods. In these lectures details about how to use Matlab are detailed (but
not verbose) and explicit. To teach programming, students are usually given examples of
working programs and are asked to make modifications.
iii
iv PREFACE
The lectures are designed to be used in a computer classroom, but could be used in a
lecture format with students doing computer exercises afterward. The lectures are divided
into four Parts with a summary provided at the end of each Part. Throughout the text
Matlab commands are preceded by the symbol >, which is the prompt in the command
window. Programs are surrounded by a box.
Todd Young, August 27, 2011
Contents
Preface iii
I Matlab and Solving Equations 1
Lecture 1. Vectors, Functions, and Plots in Matlab 2
Lecture 2. Matlab Programs 5
Lecture 3. Newton’s Method and Loops 8
Lecture 4. Controlling Error and Conditional Statements 11
Lecture 5. The Bisection Method and Locating Roots 14
Lecture 6. Secant Methods* 17
Lecture 7. Symbolic Computations 20
Review of Part I 23
II Linear Algebra 27
Lecture 8. Matrices and Matrix Operations in Matlab 28
Lecture 9. Introduction to Linear Systems 32
Lecture 10. Some Facts About Linear Systems 36
Lecture 11. Accuracy, Condition Numbers and Pivoting 39
Lecture 12. LU Decomposition 43
Lecture 13. Nonlinear Systems - Newton’s Method 46
Lecture 14. Eigenvalues and Eigenvectors 50
Lecture 15. An Application of Eigenvectors: Vibrational Modes 53
Lecture 16. Numerical Methods for Eigenvalues 56
Lecture 17. The QR Method* 60
v
vi CONTENTS
Lecture 18. Iterative solution of linear systems* 62
Review of Part II 63
III Functions and Data 67
Lecture 19. Polynomial and Spline Interpolation 68
Lecture 20. Least Squares Fitting: Noisy Data 72
Lecture 21. Integration: Left, Right and Trapezoid Rules 75
Lecture 22. Integration: Midpoint and Simpson’s Rules 79
Lecture 23. Plotting Functions of Two Variables 83
Lecture 24. Double Integrals for Rectangles 86
Lecture 25. Double Integrals for Non-rectangles 90
Lecture 26. Gaussian Quadrature* 93
Lecture 27. Numerical Differentiation 94
Lecture 28. The Main Sources of Error 97
Review of Part III 100
IV Differential Equations 105
Lecture 29. Reduction of Higher Order Equations to Systems 106
Lecture 30. Euler Methods 109
Lecture 31. Higher Order Methods 113
Lecture 32. Multi-step Methods* 116
Lecture 33. ODE Boundary Value Problems and Finite Differences 117
Lecture 34. Finite Difference Method – Nonlinear ODE 120
Lecture 35. Parabolic PDEs - Explicit Method 122
Lecture 36. Solution Instability for the Explicit Method 126
Lecture 37. Implicit Methods 129
Lecture 38. Insulated Boundary Conditions 132
Lecture 39. Finite Difference Method for Elliptic PDEs 136
CONTENTS vii
Lecture 40. Convection-Diffusion Equations 139
Lecture 41. Finite Elements 140
Lecture 42. Determining Internal Node Values 144
Review of Part IV 147
V Appendices 149
Lecture A. Sample Exams 150
Lecture B. Glossary of Matlab Commands 156
GNU Free Documentation License 162
1. APPLICABILITY AND DEFINITIONS . . . . . . . . . . . . . . . . . . . . . 162
2. VERBATIM COPYING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
3. COPYING IN QUANTITY . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
4. MODIFICATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
5. COMBINING DOCUMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
6. COLLECTIONS OF DOCUMENTS . . . . . . . . . . . . . . . . . . . . . . . 168
7. AGGREGATION WITH INDEPENDENT WORKS . . . . . . . . . . . . . . 168
8. TRANSLATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
9. TERMINATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
10. FUTURE REVISIONS OF THIS LICENSE . . . . . . . . . . . . . . . . . . 170
11. RELICENSING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
ADDENDUM: How to use this License for your documents . . . . . . . . . . . . 171
viii CONTENTS
Part I
Matlab and Solving Equations
c _Copyright, Todd Young and Martin Mohlenkamp, Mathematics Department, Ohio University, 2007
Lecture 1
Vectors, Functions, and Plots in Matlab
In this book > will indicate commands to be entered in the command window. You do not actually
type the command prompt > .
Entering vectors
In Matlab, the basic objects are matrices, i.e. arrays of numbers. Vectors can be thought of as
special matrices. A row vector is recorded as a 1 n matrix and a column vector is recorded as
a m 1 matrix. To enter a row vector in Matlab, type the following at the prompt ( > ) in the
command window:
> v = [0 1 2 3]
and press enter. Matlab will print out the row vector. To enter a column vector type:
> u = [9; 10; 11; 12; 13]
You can access an entry in a vector with
> u(2)
and change the value of that entry with
> u(2)=47
You can extract a slice out of a vector with
> u(2:4)
You can change a row vector into a column vector, and vice versa easily in Matlab using:
> w = v’
(This is called transposing the vector and we call ’ the transpose operator.) There are also useful
shortcuts to make vectors such as:
> x = -1:.1:1
and
> y = linspace(0,1,11)
Plotting Data
Consider the following table, obtained from experiments on the viscosity of a liquid.
1
We can enter
T (C

) 5 20 30 50 55
µ 0.08 0.015 0.009 0.006 0.0055
this data into Matlab with the following commands entered in the command window:
> x = [ 5 20 30 50 55 ]
1
Adapted from Ayyup & McCuen 1996, p.174.
2
3
> y = [ 0.08 0.015 0.009 0.006 0.0055]
Entering the name of the variable retrieves its current values. For instance:
> x
> y
We can plot data in the form of vectors using the plot command:
> plot(x,y)
This will produce a graph with the data points connected by lines. If you would prefer that the
data points be represented by symbols you can do so. For instance:
> plot(x,y,’*’)
> plot(x,y,’o’)
> plot(x,y,’.’)
Data as a Representation of a Function
A major theme in this course is that often we are interested in a certain function y = f(x), but
the only information we have about this function is a discrete set of data ¦(x
i
, y
i
)¦. Plotting the
data, as we did above, can be thought of envisioning the function using just the data. We will find
later that we can also do other things with the function, like differentiating and integrating, just
using the available data. Numerical methods, the topic of this course, means doing mathematics by
computer. Since a computer can only store a finite amount of information, we will almost always
be working with a finite, discrete set of values of the function (data), rather than a formula for the
function.
Built-in Functions
If we wish to deal with formulas for functions, Matlab contains a number of built-in functions,
including all the usual functions, such as sin( ), exp( ), etc.. The meaning of most of these is
clear. The dependent variable (input) always goes in parentheses in Matlab. For instance:
> sin(pi)
should return the value of sin π, which is of course 0 and
> exp(0)
will return e
0
which is 1. More importantly, the built-in functions can operate not only on single
numbers but on vectors. For example:
> x = linspace(0,2*pi,40)
> y = sin(x)
> plot(x,y)
will return a plot of sin x on the interval [0, 2π]
Some of the built-in functions in Matlab include: cos( ), tan( ), sinh( ), cosh( ), log( )
(natural logarithm), log10( ) (log base 10), asin( ) (inverse sine), acos( ), atan( ). To find
out more about a function, use the help command; try
> help plot
User-Defined Inline Functions
If we wish to deal with a function that is a combination of the built-in functions, Matlab has a
couple of ways for the user to define functions. One that we will use a lot is the inline function, which
4 LECTURE 1. VECTORS, FUNCTIONS, AND PLOTS IN MATLAB
is a way to define a function in the command window. The following is a typical inline function:
> f = inline(’2*x.^2 - 3*x + 1’,’x’)
This produces the function f(x) = 2x
2
−3x + 1. To obtain a single value of this function enter:
> f(2.23572)
Just as for built-in functions, the function f as we defined it can operate not only on single numbers
but on vectors. Try the following:
> x = -2:.2:2
> y = f(x)
This is an example of vectorization, i.e. putting several numbers into a vector and treating the
vector all at once, rather than one component at a time, and is one of the strengths of Mat-
lab. The reason f(x) works when x is a vector is because we represented x
2
by x.^2. The .
turns the exponent operator ^ into entry-wise exponentiation, so that [-2 -1.8 -1.6].^2 means
[(−2)
2
, (−1.8)
2
, (−1.6)
2
] and yields [4 3.24 2.56]. In contrast, [-2 -1.8 -1.6]^2 means the ma-
trix product [−2, −1.8, −1.6][−2, −1.8, −1.6] and yields only an error. The . is needed in .^, .*,
and ./. It is not needed when you * or / by a scalar or for +.
The results can be plotted using the plot command, just as for data:
> plot(x,y)
Notice that before plotting the function, we in effect converted it into data. Plotting on any machine
always requires this step.
Exercises
1.1 Find a table of data in an engineering textbook. Input it as vectors and plot it. Use the insert
icon to label the axes and add a title to your graph. Turn in the graph. Indicate what the
data is and where it came from.
1.2 Make an inline function g(x) = x + cos(x
5
). Plot it using vectors x = -5:.1:5; and
y = g(x);. What is wrong with this graph? Find a way to make it look more like the
graph of g(x) should. Turn in both plots.
Lecture 2
Matlab Programs
In Matlab, programs may be written and saved in files with a suffix .m called M-files. There are
two types of M-file programs: functions and scripts.
Function Programs
Begin by clicking on the new document icon in the top left of the Matlab window (it looks like an
empty sheet of paper).
In the document window type the following:
function y = myfunc(x)
y = 2*x.^2 - 3*x + 1;
Save this file as: myfunc.m in your working directory. This file can now be used in the command
window just like any predefined Matlab function; in the command window enter:
> x = -2:.1:2; . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Produces a vector of x values.
> y = myfunc(x); . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Produces a vector of y values.
> plot(x,y)
Note that the fact we used x and y in both the function program and in the command window was
just a coincidence. In fact, it is the name of the file myfunc.m that actually mattered, not what
anything in it was called. We could just as well have made the function
function nonsense = yourfunc(inputvector)
nonsense = 2*inputvector.^2 - 3*inputvector + 1;
Look back at the program. All function programs are like this one, the essential elements are:
• Begin with the word function.
• There are inputs and outputs.
• The outputs, name of the function and the inputs must appear in the first line.
• The body of the program must assign values to the outputs.
Functions can have multiple inputs and/or multiple outputs. Next let’s create a function that has 1
input and 3 output variables. Open a new document and type:
function [x2 x3 x4] = mypowers(x)
x2 = x.^2;
x3 = x.^3;
x4 = x.^4;
5
6 LECTURE 2. MATLAB PROGRAMS
Save this file as mypowers.m. In the command window, we can use the results of the program to
make graphs:
> x = -1:.1:1
> [x2 x3 x4] = mypowers(x);
> plot(x,x,’black’,x,x2,’blue’,x,x3,’green’,x,x4,’red’)
Script Programs
Matlab uses a second type of program that differs from a function program in several ways, namely:
• There are no inputs and outputs.
• A script program may use and change variables in the current workspace (the variables used
by the command window.)
Below is a script program that accomplishes the same thing as the function program plus the
commands in the previous section:
x2 = x.^2;
x3 = x.^3;
x4 = x.^4;
plot(x,x,’black’,x,x2,’blue’,x,x3,’green’,x,x4,’red’)
Type this program into a new document and save it as mygraphs.m. In the command window enter:
> x = -1:.1:1;
> mygraphs
Note that the program used the variable x in its calculations, even though x was defined in the
command window, not in the program.
Many people use script programs for routine calculations that would require typing more than one
command in the command window. They do this because correcting mistakes is easier in a program
than in the command window.
Program Comments
For programs that have more than a couple of lines it is important to include comments. Comments
allow other people to know what your program does and they also remind yourself what your
program does if you set it aside and come back to it later. It is best to include comments not only
at the top of a program, but also with each section. In Matlab anything that comes in a line after
a % is a comment.
For a function program, the comments should at least give the purpose, inputs, and outputs. A
properly commented version of the function with which we started this section is:
function y = myfunc(x)
% Computes the function 2x^2 -3x +1
% Input: x -- a number or vector; for a vector the computation is elementwise
% Output: y -- a number or vector of the same size as x
y = 2*x.^2 - 3*x + 1;
7
For a script program it is often helpful to include the name of the program at the beginning. For
example:
% mygraphs
% plots the graphs of x, x^2, x^3, and x^4
% on the interval [-1,1]
% fix the domain and evaluation points
x = -1:.1:1;
% calculate powers
% x1 is just x
x2 = x.^2;
x3 = x.^3;
x4 = x.^4;
% plot each of the graphs
plot(x,x,’black’,x,x2,’blue’,x,x3,’green’,x,x4,’red’)
The Matlab command help prints the first block of comments from a file. If we save the above as
mygraphs.m and then do
> help mygraphs
it will print into the command window:
mygraphs
plots the graphs of x, x^2, x^3, and x^4
on the interval [-1,1]
Exercises
2.1 Write a function program for the function x
2
e
−x
2
, using entry-wise operations (such as .*
and .^). To get e
x
use exp(x). Include adequate comments in the program. Plot the function
on [−5, 5]. Turn in printouts of the program and the graph.
2.2 Write a script program that graphs the functions sin x, sin 2x, sin 3x, sin 4x, sin 5x and sin 6x
on the interval [0, 2π] on one plot. (π is pi in Matlab.) Include comments in the program.
Turn in the program and the graph.
Lecture 3
Newton’s Method and Loops
Solving equations numerically
For the next few lectures we will focus on the problem of solving an equation:
f(x) = 0. (3.1)
As you learned in calculus, the final step in many optimization problems is to solve an equation of
this form where f is the derivative of a function, F, that you want to maximize or minimize. In real
engineering problems the function you wish to optimize can come from a large variety of sources,
including formulas, solutions of differential equations, experiments, or simulations.
Newton iterations
We will denote an actual solution of equation (3.1) by x

. There are three methods which you may
have discussed in Calculus: the bisection method, the secant method and Newton’s method. All
three depend on beginning close (in some sense) to an actual solution x

.
Recall Newton’s method. You should know that the basis for Newton’s method is approximation of
a function by it linearization at a point, i.e.
f(x) ≈ f(x
0
) +f

(x
0
)(x −x
0
). (3.2)
Since we wish to find x so that f(x) = 0, set the left hand side (f(x)) of this approximation equal
to 0 and solve for x to obtain:
x ≈ x
0

f(x
0
)
f

(x
0
)
. (3.3)
We begin the method with the initial guess x
0
, which we hope is fairly close to x

. Then we define
a sequence of points ¦x
0
, x
1
, x
2
, x
3
, . . .¦ from the formula:
x
i+1
= x
i

f(x
i
)
f

(x
i
)
, (3.4)
which comes from (3.3). If f(x) is reasonably well-behaved near x

and x
0
is close enough to x

,
then it is a fact that the sequence will converge to x

and will do it very quickly.
The loop: for ... end
In order to do Newton’s method, we need to repeat the calculation in (3.4) a number of times. This
is accomplished in a program using a loop, which means a section of a program which is repeated.
8
9
The simplest way to accomplish this is to count the number of times through. In Matlab, a
for ... end statement makes a loop as in the following simple function program:
function S = mysum(n)
% gives the sum of the first n integers
S = 0; % start at zero
% The loop:
for i = 1:n % do n times
S = S + i; % add the current integer
end % end of the loop
Call this function in the command window as:
> mysum(100)
The result will be the sum of the first 100 integers. All for ... end loops have the same format, it
begins with for, followed by an index (i) and a range of numbers (1:n). Then come the commands
that are to be repeated. Last comes the end command.
Loops are one of the main ways that computers are made to do calculations that humans cannot.
Any calculation that involves a repeated process is easily done by a loop.
Now let’s do a program that does n steps (iterations) of Newton’s method. We will need to input
the function, its derivative, the initial guess, and the number of steps. The output will be the final
value of x, i.e. x
n
. If we are only interested in the final approximation, not the intermediate steps,
which is usually the case in the real world, then we can use a single variable x in the program and
change it at each step:
function x = mynewton(f,f1,x0,n)
% Solves f(x) = 0 by doing n steps of Newton’s method starting at x0.
% Inputs: f -- the function, input as an inline
% f1 -- it’s derivative, input as an inline
% x0 -- starting guess, a number
% n -- the number of steps to do
% Output: x -- the approximate solution
format long % prints more digits
format compact % makes the output more compact
x = x0; % set x equal to the initial guess x0
for i = 1:n % Do n times
x = x - f(x)/f1(x) % Newton’s formula, prints x too
end
In the command window define an inline function: f(x) = x
3
−5 i.e.
> f = inline(’x^3 - 5’)
and define f1 to be its derivative, i.e.
> f1 = inline(’3*x^2’).
Then run mynewton on this function. By trial and error, what is the lowest value of n for which the
program converges (stops changing). By simple algebra, the true root of this function is
3

5. How
close is the program’s answer to the true value?
Convergence
Newton’s method converges rapidly when f

(x

) is nonzero and finite, and x
0
is close enough to x

that the linear approximation (3.2) is valid. Let us take a look at what can go wrong.
10 LECTURE 3. NEWTON’S METHOD AND LOOPS
For f(x) = x
1/3
we have x

= 0 but f

(x

) = ∞. If you try
> f = inline(’x^(1/3)’)
> f1 = inline(’(1/3)*x^(-2/3)’)
> x = mynewton(f,f1,0.1,10)
then x explodes.
For f(x) = x
2
we have x

= 0 but f

(x

) = 0. If you try
> f = inline(’x^2’)
> f1 = inline(’2*x’)
> x = mynewton(f,f1,1,10)
then x does converge to 0, but not that rapidly.
If x
0
is not close enough to x

that the linear approximation (3.2) is valid, then the iteration (3.4)
gives some x
1
that may or may not be any better than x
0
. If we keep iterating, then either
• x
n
will eventually get close to x

and the method will then converge (rapidly), or
• the iterations will not approach x

.
Exercises
3.1 Suppose a ball with a coefficient of elasticity of .9 is dropped from a height of 2 meters onto
a hard surface. Write a script program to calculate the distance traveled by the ball after n
bounces. Include lots of comments. By trial and error approximate how large n must be so
that total distance stops changing. Turn in the program and a brief summary of the results.
3.2 Enter: format long. Use mynewton on the function f(x) = x
5
−7, with x
0
= 2. By trial and
error, what is the lowest value of n for which the program converges (stops changing). How
close is the answer to the true value? Plug the program’s answer into f(x); is the value zero?
3.3 For f(x) = x
3
−4, perform 3 iterations of Newton’s method with starting point x
0
= 2. (On
paper, but use a calculator.) Calculate the solution (x

= 4
1/3
) on a calculator and find the
errors and percentage errors of x
0
, x
1
, x
2
and x
3
. Put the results in a table.
Lecture 4
Controlling Error and Conditional
Statements
Measuring error
If we are trying to find a numerical solution of an equation f(x) = 0, then there are a few different
ways we can measure the error of our approximation. The most direct way to measure the error
would be as:
¦Error at step n¦ = e
n
= x
n
−x

where x
n
is the n-th approximation and x

is the true value. However, we usually do not know the
value of x

, or we wouldn’t be trying to approximate it. This makes it impossible to know the error
directly, and so we must be more clever.
For Newton’s method we have the following principle: At each step the number of significant
digits roughly doubles. While this is an important statement about the error (since it means
Newton’s method converges really quickly), it is somewhat hard to use in a program.
Rather than measure how close x
n
is to x

, in this and many other situations it is much more
practical to measure how close the equation is to being satisfied, in other words, how close f(x
n
)
is to 0. We will use the quantity r
n
= f(x
n
) − 0, called the residual, in many different situations.
Most of the time we only care about the size of r
n
, so we look at [r
n
[ = [f(x
n
)[.
The if ... end statement
If we have a certain tolerance for [r
n
[ = [f(x
n
)[, then we can incorporate that into our Newton
method program using an if ... end statement:
11
12 LECTURE 4. CONTROLLING ERROR AND CONDITIONAL STATEMENTS
function x = mynewton(f,f1,x0,n,tol)
% Solves f(x) = 0 by doing n steps of Newton’s method starting at x0.
% Inputs: f -- the function, input as an inline
% f1 -- it’s derivative, input as an inline
% x0 -- starting guess, a number
% tol -- desired tolerance, prints a warning if |f(x)|>tol
% Output: x -- the approximate solution
x = x0; % set x equal to the initial guess x0
for i = 1:n % Do n times
x = x - f(x)/f1(x); % Newton’s formula
end
r = abs(f(x));
if r > tol
warning(’The desired accuracy was not attained’)
end
In this program if checks if abs(y) > tol is true or not. If it is true then it does everything
between there and end. If not true, then it skips ahead to end.
In the command window define the function
> f = inline(’x^3-5’,’x’)
and its derivative
> f1 = inline(’3*x^2’,’x’).
Then use the program with n = 5, tol = .01, and x
0
= 2. Next, change tol to 10
−10
and repeat.
The loop: while ... end
While the previous program will tell us if it worked or not, we still have to input n, the number of
steps to take. Even for a well-behaved problem, if we make n too small then the tolerance will not
be attained and we will have to go back and increase it, or, if we make n too big, then the program
will take more steps than necessary.
One way to control the number of steps taken is to iterate until the residual [r
n
[ = [f(x)[ = [y[ is
small enough. In Matlab this is easily accomplished with a while ... end loop.
function x = mynewtontol(f,f1,x0,tol)
% Solves f(x) = 0 by doing Newton’s method starting at x0.
% Inputs: f -- the function, input as an inline
% f1 -- it’s derivative, input as an inline
% x0 -- starting guess, a number
% tol -- desired tolerance, goes until |f(x)|<tol
% Output: x -- the approximate solution
x = x0; % set x equal to the initial guess x0
y = f(x);
while abs(y) > tol % Do until the tolerence is reached.
x = x - y/f1(x); % Newton’s formula
y = f(x);
end
The statement while ... end is a loop, similar to for ... end, but instead of going through the
loop a fixed number of times it keeps going as long as the statement abs(y) > tol is true.
13
One obvious drawback of the program is that abs(y) might never get smaller than tol. If this
happens, the program would continue to run over and over until we stop it. Try this by setting the
tolerance to a really small number:
> tol = 10^(-100)
then run the program again for f(x) = x
3
−5.
You can use Ctrl-c to stop the program when it’s stuck.
Exercises
4.1 In Calculus we learn that a geometric series has an exact sum:

i=0
r
i
=
1
1 −r
provided that [r[ < 1. For instance, if r = .5 then the sum is exactly 2. Below is a script
program that lacks one line as written. Put in the missing command and then use the program
to verify the result above.
How many steps does it take? How close is the answer to 2? Change r = .5 to r=.999. How
many steps does it take now? How accurate is the answer?
format long
r = .5;
Snew = 0; % start sum at 0
Sold = -1; % set Sold to trick while the first time
i = 0; % count iterations
while Snew > Sold % is the sum still changing?
Sold = Snew; % save previous value to compare to
Snew = Snew + r^i;
i=i+1;
Snew % prints the final value.
i % prints the # of iterations.
4.2 Modify your program from exercise 3.1 to compute the total distance traveled by the ball
while its bounces are at least 1 centimeter high. Use a while loop to decide when to stop
summing; do not use a for loop or trial and error. Turn in your modified program and a brief
summary of the results.
Lecture 5
The Bisection Method and Locating
Roots
Bisecting and the if ... else ... end statement
Recall the bisection method. Suppose that c = f(a) < 0 and d = f(b) > 0. If f is continuous,
then obviously it must be zero at some x

between a and b. The bisection method then consists of
looking half way between a and b for the zero of f, i.e. let x = (a + b)/2 and evaluate y = f(x).
Unless this is zero, then from the signs of c, d and y we can decide which new interval to subdivide.
In particular, if c and y have the same sign, then [x, b] should be the new interval, but if c and y
have different signs, then [a, x] should be the new interval. (See Figure 5.1.)
Deciding to do different things in different situations in a program is called flow control. The
most common way to do this is the if ... else ... end statement which is an extension of the
if ... end statement we have used already.
Bounding the Error
One good thing about the bisection method, that we don’t have with Newton’s method, is that we
always know that the actual solution x

is inside the current interval [a, b], since f(a) and f(b) have
different signs. This allows us to be sure about what the maximum error can be. Precisely, the
error is always less than half of the length of the current interval [a, b], i.e.
¦Absolute Error¦ = [x −x

[ < (b −a)/2,
where x is the center point between the current a and b.
x
0
x
1
x
2
a
0
b
0
a
1
b
1
a
2
b
2
u
u
u
u
Figure 5.1: The bisection method.
14
15
The following function program (also available on the webpage) does n iterations of the bisection
method and returns not only the final value, but also the maximum possible error:
function [x e] = mybisect(f,a,b,n)
% function [x e] = mybisect(f,a,b,n)
% Does n iterations of the bisection method for a function f
% Inputs: f -- an inline function
% a,b -- left and right edges of the interval
% n -- the number of bisections to do.
% Outputs: x -- the estimated solution of f(x) = 0
% e -- an upper bound on the error
format long
c = f(a); d = f(b);
if c*d > 0.0
error(’Function has same sign at both endpoints.’)
end
disp(’ x y’)
for i = 1:n
x = (a + b)/2;
y = f(x);
disp([ x y])
if y == 0.0 % solved the equation exactly
e = 0;
break % jumps out of the for loop
end
if c*y < 0
b=x;
else
a=x;
end
end
e = (b-a)/2;
Another important aspect of bisection is that it always works. We saw that Newton’s method can
fail to converge to x

if x
0
is not close enough to x

. In contrast, the current interval [a, b] in
bisection will always get decreased by a factor of 2 at each step and so it will always eventually
shrink down as small as you want it.
Locating a root
The bisection method and Newton’s method are both used to obtain closer and closer approximations
of a solution, but both require starting places. The bisection method requires two points a and b
that have a root between them, and Newton’s method requires one point x
0
which is reasonably
close to a root. How do you come up with these starting points? It depends. If you are solving an
equation once, then the best thing to do first is to just graph it. From an accurate graph you can
see approximately where the graph crosses zero.
There are other situations where you are not just solving an equation once, but have to solve the same
equation many times, but with different coefficients. This happens often when you are developing
software for a specific application. In this situation the first thing you want to take advantage of is
the natural domain of the problem, i.e. on what interval is a solution physically reasonable. If that
16 LECTURE 5. THE BISECTION METHOD AND LOCATING ROOTS
is known, then it is easy to get close to the root by simply checking the sign of the function at a
fixed number of points inside the interval. Whenever the sign changes from one point to the next,
there is a root between those points. The following program will look for the roots of a function f
on a specified interval [a
0
, b
0
].
function [a,b] = myrootfind(f,a0,b0)
% function [a,b] = myrootfind(f,a0,b0)
% Looks for subintervals where the function changes sign
% Inputs: f -- an inline function
% a0 -- the left edge of the domain
% b0 -- the right edge of the domain
% Outputs: a -- an array, giving the left edges of subintervals
% on which f changes sign
% b -- an array, giving the right edges of the subintervals
n = 1001; % number of test points to use
a = []; % start empty array
b = [];
x = linspace(a0,b0,n);
y = f(x);
for i = 1:(n-1)
if y(i)*y(i+1) < 0 % The sign changed, record it
a = [a x(i)];
b = [b x(i+1)];
end
end
if a == []
warning(’no roots were found’)
end
The final situation is writing a program that will look for roots with no given information. This is
a difficult problem and one that is not often encountered in engineering applications.
Once a root has been located on an interval [a, b], these a and b can serve as the beginning points
for the bisection and secant methods (see the next section). For Newton’s method one would want
to choose x
0
between a and b. One obvious choice would be to let x
0
be the bisector of a and b,
i.e. x
0
= (a +b)/2. An even better choice would be to use the secant method to choose x
0
.
Exercises
5.1 Modify mybisect to solve until the error is bounded by a given tolerance. Use a while loop to
do this. How should error be measured? Run your program on the function f(x) = 2x
3
+3x−1
with starting interval [0, 1] and a tolerance of 10
−8
. How many steps does the program use
to achieve this tolerance? (You can count the step by adding 1 to a counting variable i in
the loop of the program.) Turn in your program and a brief summary of the results.
5.2 Perform 3 iterations of the bisection method on the function f(x) = x
3
− 4, with starting
interval [1, 3]. (On paper, but use a calculator.) Calculate the errors and percentage errors of
x
0
, x
1
, and x
2
. Compare the errors with those in exercise 3.3.
Lecture 6
Secant Methods*
In this lecture we introduce two additional methods to find numerical solutions of the equation
f(x) = 0. Both of these methods are based on approximating the function by secant lines just as
Newton’s method was based on approximating the function by tangent lines.
The Secant Method
The secant method requires two initial points x
0
and x
1
which are both reasonably close to the
solution x

. Preferably the signs of y
0
= f(x
0
) and y
1
= f(x
1
) should be different. Once x
0
and x
1
are determined the method proceeds by the following formula:
x
i+1
= x
i

x
i
−x
i−1
y
i
−y
i−1
y
i
(6.1)
Example: Suppose f(x) = x
4
− 5 for which the true solution is x

=
4

5. Plotting this function
reveals that the solution is at about 1.25. If we let x
0
= 1 and x
1
= 2 then we know that the root
is in between x
0
and x
1
. Next we have that y
0
= f(1) = −4 and y
1
= f(2) = 11. We may then
calculate x
2
from the formula (6.1):
x
2
= 2 −
2 −1
11 −(−4)
11 =
19
15
≈ 1.2666....
Pluggin x
2
= 19/15 into f(x) we obtain y
2
= f(19/15) ≈ −2.425758.... In the next step we would
use x
1
= 2 and x
2
= 19/15 in the formula (6.1) to find x
3
and so on.
Below is a program for the secant method. Notice that it requires two input guesses x
0
and x
1
, but
it does not require the derivative to be input.
figure yet to be drawn, alas
Figure 6.1: The secant method.
17
18 LECTURE 6. SECANT METHODS*
function x = mysecant(f,x0,x1,n)
format long % prints more digits
format compact % makes the output more compact
% Solves f(x) = 0 by doing n steps of the secant method starting with x0 and x1.
% Inputs: f -- the function, input as an inline function
% x0 -- starting guess, a number
% x1 -- second starting geuss
% n -- the number of steps to do
% Output: x -- the approximate solution
y0 = f(x0);
y1 = f(x1);
for i = 1:n % Do n times
x = x1 - (x1-x0)*y1/(y1-y0) % secant formula.
y=f(x) % y value at the new approximate solution.
% Move numbers to get ready for the next step
x0=x1;
y0=y1;
x1=x;
y1=y;
end
The Regula Falsi Method
The Regula Falsi method is somewhat a combination of the secant method and bisection method.
The idea is to use secant lines to approximate f(x), but choose how to update using the sign of
f(x
n
).
Just as in the bisection method, we begin with a and b for which f(a) and f(b) have different signs.
Then let:
x = b −
b −a
f(b) −f(a)
f(b).
Next check the sign of f(x). If it is the same as the sign of f(a) then x becomes the new a. Otherwise
let b = x.
Convergence
If we can begin with a good choice x
0
, then Newton’s method will converge to x

rapidly. The secant
method is a little slower than Newton’s method and the Regula Falsi method is slightly slower than
that. Both however are still much faster than the bisection method.
If we do not have a good starting point or interval, then the secant method, just like Newton’s
method can fail altogether. The Regula Falsi method, just like the bisection method always works
because it keeps the solution inside a definite interval.
Simulations and Experiments
Although Newton’s method converges faster than any other method, there are contexts when it
is not convenient, or even impossible. One obvious situation is when it is difficult to calculate a
19
formula for f

(x) even though one knows the formula for f(x). This is often the case when f(x) is not
defined explicitly, but implicitly. There are other situations, which are very common in engineering
and science, where even a formula for f(x) is not known. This happens when f(x) is the result of
experiment or simulation rather than a formula. In such situations, the secant method is usually
the best choice.
Exercises
6.1 Use the program mysecant on f(x) = x
3
− 4. Compare the results with those of Exercises
3.3 and 5.2. Then modify the program to accept a tolerance using a while loop. Test the
program and turn in the program as well as the work above.
6.2 Write a program myregfalsi based on mybisect. Use it on f(x) = x
3
− 4. Compare the
results with those of Exercises 3.3 and 5.2.
Lecture 7
Symbolic Computations
The focus of this course is on numerical computations, i.e. calculations, usually approximations,
with floating point numbers. However, Matlab can also do symbolic computations which means
exact calculations using symbols as in Algebra or Calculus. You should have done some symbolic
Matlab computations in your Calculus courses and in this chapter we review what you should
already know.
Defining functions and basic operations
Before doing any symbolic computation, one must declare the variables used to be symbolic:
> syms x y
A function is defined by simply typing the formula:
> f = cos(x) + 3*x^2
Note that coefficients must be multiplied using *. To find specific values, you must use the command
subs:
> subs(f,pi)
This command stands for substitute, it substitutes π for x in the formula for f.
If we define another function:
> g = exp(-y^2)
then we can compose the functions:
> h = compose(g,f)
i.e. h(x) = g(f(x)). Since f and g are functions of different variables, their product must be a
function of two variables:
> k = f*g
> subs(k,[x,y],[0,1])
We can do simple calculus operations, like differentiation:
> f1 = diff(f)
indefinite integrals (antiderivatives):
> F = int(f)
and definite integrals:
> int(f,0,2*pi)
To change a symbolic answer into a numerical answer, use the double command which stands for
double precision, (not times 2):
> double(ans)
Note that some antiderivatives cannot be found in terms of elementary functions, for some of these
it can be expressed in terms of special functions:
> G = int(g)
and for others Matlab does the best it can:
20
21
−6 −4 −2 0 2 4 6
−1
−0.5
0
0.5
1
x
cos(x
5
)
Figure 7.1: Graph of cos(x
5
) produced by the ezplot command. It is wrong because cos u
should oscillate smoothly between −1 and 1. The problem with the plot is that cos(x
5
)
oscillates extremely rapidly, and the plot did not consider enough points.
> int(h)
For definite integrals that cannot be evaluated exactly, Matlab does nothing and prints a warning:
> int(h,0,1)
We will see later that even functions that don’t have an antiderivative can be integrated numerically.
You can change the last answer to a numerical answer using:
> double(ans)
Plotting a symbolic function can be done as follows:
> ezplot(f)
or the domain can be specified:
> ezplot(g,-10,10)
> ezplot(g,-2,2)
To plot a symbolic function of two variables use:
> ezsurf(k)
It is important to keep in mind that even though we have defined our variables to be symbolic
variables, plotting can only plot a finite set of points. For intance:
> ezplot(cos(x^5))
will produce a plot that is clearly wrong, because it does not plot enough points.
Other useful symbolic operations
Matlab allows you to do simple algebra. For instance:
> poly = (x - 3)^5
> polyex = expand(poly)
> polysi = simple(polyex)
22 LECTURE 7. SYMBOLIC COMPUTATIONS
To find the symbolic solutions of an equation, f(x) = 0, use:
> solve(f)
> solve(g)
> solve(polyex)
Another useful property of symbolic functions is that you can substitute numerical vectors for the
variables:
> X = 2:0.1:4;
> Y = subs(polyex,X);
> plot(X,Y)
Exercises
7.1 Starting from mynewton write a function program mysymnewton that takes as its input a
symbolic function f and the ordinary variables x
0
and n. Let the program take the symbolic
derivative f

, and then use subs to proceed with Newton’s method. Test it on f(x) = x
3
−4
starting with x
0
= 2. Turn in the program and a brief summary of the results.
Review of Part I
Methods and Formulas
Solving equations numerically:
f(x) = 0 — an equation we wish to solve.
x

— a true solution.
x
0
— starting approximation.
x
n
— approximation after n steps.
e
n
= x
n
−x

— error of n-th step.
r
n
= y
n
= f(x
n
) — residual at step n. Often [r
n
[ is sufficient.
Newton’s method:
x
i+1
= x
i

f(x
i
)
f

(x
i
)
Bisection method:
f(a) and f(b) must have different signs.
x = (a +b)/2
Choose a = x or b = x, depending on signs.
x

is always inside [a, b].
e < (b −a)/2, current maximum error.
Secant method*:
x
i+1
= x
i

x
i
−x
i−1
y
i
−y
i−1
y
i
Regula Falsi*:
x = b −
b −a
f(b) −f(a)
f(b)
Choose a = x or b = x, depending on signs.
Convergence:
Bisection is very slow.
Newton is very fast.
Secant methods are intermediate in speed.
Bisection and Regula Falsi never fail to converge.
Newton and Secant can fail if x
0
is not close to x

.
Locating roots:
Use knowledge of the problem to begin with a reasonable domain.
Systematically search for sign changes of f(x).
Choose x
0
between sign changes using bisection or secant.
Usage:
For Newton’s method one must have formulas for f(x) and f

(x).
23
24 REVIEW OF PART I
Secant methods are better for experiments and simulations.
Matlab
Commands:
> v = [0 1 2 3] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Make a row vector.
> u = [0; 1; 2; 3] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Make a column vector.
> w = v’ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Transpose: row vector ↔ column vector
> x = linspace(0,1,11) . . . . . . . . . . . . . . . . . . . . . . . . . . . Make an evenly spaced vector of length 11.
> x = -1:.1:1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Make an evenly spaced vector, with increments 0.1.
> y = x.^2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Square all entries.
> plot(x,y) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . plot y vs. x.
> f = inline(’2*x.^2 - 3*x + 1’,’x’) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Make a function.
> y = f(x) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A function can act on a vector.
> plot(x,y,’*’,’red’) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A plot with options.
> Control-c . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Stops a computation.
Program structures:
for ... end
Example:
for i=1:20
S = S + i;
end
if ... end
Example:
if y == 0
disp(’An exact solution has been found’)
end
while ... end
Example:
while i <= 20
S = S + i;
i = i + 1;
end
if ... else ... end
Example:
if c*y>0
a = x;
else
b = x;
end
Function Programs:
- Begin with the word function.
- There are inputs and outputs.
- The outputs, name of the function and the inputs must appear in the first line.
i.e. function x = mynewton(f,x0,n) - The body of the program must assign values to the
outputs.
25
- internal variables are not visible outside the function.
Script Programs:
- There are no inputs and outputs.
- A script program may use and change variables in the current workspace.
Symbolic:
> syms x y
> f = 2*x^2 - sqrt(3*x)
> subs(f,sym(pi))
> double(ans)
> g = log(abs(y)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Matlab uses log for natural logarithm.
> h(x) = compose(g,f)
> k(x,y) = f*g
> ezplot(f)
> ezplot(g,-10,10)
> ezsurf(k)
> f1 = diff(f,’x’)
> F = int(f,’x’) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . indefinite integral (antiderivative)
> int(f,0,2*pi) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . definite integral
> poly = x*(x - 3)*(x-2)*(x-1)*(x+1)
> polyex = expand(poly)
> polysi = simple(polyex)
> solve(f)
> solve(g)
> solve(polyex)
26 REVIEW OF PART I
Part II
Linear Algebra
c _Copyright, Todd Young and Martin Mohlenkamp, Mathematics Department, Ohio University, 2007
Lecture 8
Matrices and Matrix Operations in
Matlab
Matrix operations
Recall how to multiply a matrix A times a vector v:
Av =
_
1 2
3 4
__
−1
2
_
=
_
1 (−1) + 2 2
3 (−1) + 4 2
_
=
_
3
5
_
.
This is a special case of matrix multiplication. To multiply two matrices, A and B you proceed as
follows:
AB =
_
1 2
3 4
__
−1 −2
2 1
_
=
_
−1 + 4 −2 + 2
−3 + 8 −6 + 4
_
=
_
3 0
5 −2
_
.
Here both A and B are 2 2 matrices. Matrices can be multiplied together in this way provided
that the number of columns of A match the number of rows of B. We always list the size of a matrix
by rows, then columns, so a 3 5 matrix would have 3 rows and 5 columns. So, if A is mn and
B is p q, then we can multiply AB if and only if n = p. A column vector can be thought of as a
p 1 matrix and a row vector as a 1 q matrix. Unless otherwise specified we will assume a vector
v to be a column vector and so Av makes sense as long as the number of columns of A matches the
number of entries in v.
Printing matrices on the screen takes up a lot of space, so you may want to use
> format compact
Enter a matrix into Matlab with the following syntax:
> A = [ 1 3 -2 5 ; -1 -1 5 4 ; 0 1 -9 0]
Also enter a vector u:
> u = [ 1 2 3 4]’
To multiply a matrix times a vector Au use *:
> A*u
Since A is 3 by 4 and u is 4 by 1 this multiplication is valid and the result is a 3 by 1 vector.
Now enter another matrix B using:
> B = [3 2 1; 7 6 5; 4 3 2]
You can multiply B times A:
> B*A
but A times B is not defined and
> A*B
will result in an error message.
You can multiply a matrix by a scalar:
> 2*A
28
29
Adding matrices A+A will give the same result:
> A + A
You can even add a number to a matrix:
> A + 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . This should add 3 to every entry of A.
Component-wise operations
Just as for vectors, adding a ’.’ before ‘*’, ‘/’, or ‘^’ produces entry-wise multiplication, division
and exponentiation. If you enter:
> B*B
the result will be BB, i.e. matrix multiplication of B times itself. But, if you enter:
> B.*B
the entries of the resulting matrix will contain the squares of the same entries of B. Similarly if you
want B multiplied by itself 3 times then enter:
> B^3
but, if you want to cube all the entries of B then enter:
> B.^3
Note that B*B and B^3 only make sense if B is square, but B.*B and B.^3 make sense for any size
matrix.
The identity matrix and the inverse of a matrix
The n n identity matrix is a square matrix with ones on the diagonal and zeros everywhere else.
It is called the identity because it plays the same role that 1 plays in multiplication, i.e.
AI = A, IA = A, Iv = v
for any matrix A or vector v where the sizes match. An identity matrix in Matlab is produced by
the command:
> I = eye(3)
A square matrix A can have an inverse which is denoted by A
−1
. The definition of the inverse is
that:
AA
−1
= I and A
−1
A = I.
In theory an inverse is very important, because if you have an equation:
Ax = b
where A and b are known and x is unknown (as we will see, such problems are very common and
important) then the theoretical solution is:
x = A
−1
b.
We will see later that this is not a practical way to solve an equation, and A
−1
is only important
for the purpose of derivations.
In Matlab we can calculate a matrix’s inverse very conveniently:
> C = randn(5,5)
> inv(C)
However, not all square matrices have inverses:
> D = ones(5,5)
> inv(D)
30 LECTURE 8. MATRICES AND MATRIX OPERATIONS IN MATLAB
The “Norm” of a matrix
For a vector, the “norm” means the same thing as the length. Another way to think of it is how far
the vector is from being the zero vector. We want to measure a matrix in much the same way and
the norm is such a quantity. The usual definition of the norm of a matrix is the following:
Definition 1 Suppose A is a mn matrix. The norm of A is
[A[ ≡ max
|v|=1
[Av[.
The maximum in the definition is taken over all vectors with length 1 (unit vectors), so the definition
means the largest factor that the matrix stretches (or shrinks) a unit vector. This definition seems
cumbersome at first, but it turns out to be the best one. For example, with this definition we have
the following inequality for any vector v:
[Av[ ≤ [A[[v[.
In Matlab the norm of a matrix is obtained by the command:
> norm(A)
For instance the norm of an identity matrix is 1:
> norm(eye(100))
and the norm of a zero matrix is 0:
> norm(zeros(50,50))
For a matrix the norm defined above and calculated by Matlab is not the square root of the sum
of the square of its entries. That quantity is called the Froebenius norm, which is also sometimes
useful, but we will not need it.
Some other useful commands
Try out the following:
> C = rand(5,5) . . . . . . . . . . . . . . . . . . . . . . . . . . . . random matrix with uniform distribution in [0, 1].
> size(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . gives the dimensions (mn) of C.
> det(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the determinant of the matrix.
> max(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the maximum of each column.
> min(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the minimum in each column.
> sum(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . sums each column.
> mean(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the average of each column.
> diag(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . just the diagonal elements.
> C’ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . tranpose the matrix.
In addition to ones, eye, zeros, rand and randn, Matlab has several other commands that auto-
matically produce special matrices:
> hilb(6)
> pascal(5)
31
Exercises
8.1 Enter the matrix M by
> M = [1,3,-1,6;2,4,0,-1;0,-2,3,-1;-1,2,-5,1]
and also the matrix
N =
_
¸
¸
_
−1 −3 3
2 −1 6
1 4 −1
2 −1 2
_
¸
¸
_
.
Multiply M and N using M * N. Can the order of multiplication be switched? Why or why
not? Try it to see how Matlab reacts.
8.2 By hand, calculate Av, AB, and BA for:
A =
_
_
2 4 −1
−2 1 9
−1 −1 0
_
_
, B =
_
_
0 −1 −1
1 0 2
−1 −2 0
_
_
, v =
_
_
3
1
−1
_
_
.
Check the results using Matlab. Think about how fast computers are. Turn in your hand
work.
8.3 (a) Write a Matlab function program myinvcheck that makes a n n random matrix
(normally distributed, A = randn(n,n)), calculates its inverse (B = inv(A)), multiplies
the two back together, calculates the error equal to the norm of the difference of the
result from the n n identity matrix (eye(n)), and returns error.
(b) Write a Matlab script program that calls myinvcheck for n = 10, 20, 40, . . . , 2
i
10,
records the results of each trial, and plots the error versus n using a log plot. (See
help loglog.)
What happens to error as n gets big? Turn in a printout of the programs, the plot, and a
very brief report on the results of your experiments.
Lecture 9
Introduction to Linear Systems
How linear systems occur
Linear systems of equations naturally occur in many places in engineering, such as structural anal-
ysis, dynamics and electric circuits. Computers have made it possible to quickly and accurately
solve larger and larger systems of equations. Not only has this allowed engineers to handle more
and more complex problems where linear systems naturally occur, but has also prompted engineers
to use linear systems to solve problems where they do not naturally occur such as thermodynamics,
internal stress-strain analysis, fluids and chemical processes. It has become standard practice in
many areas to analyze a problem by transforming it into a linear systems of equations and then
solving those equation by computer. In this way, computers have made linear systems of equations
the most frequently used tool in modern engineering.
In Figure 9.1 we show a truss with equilateral triangles. In Statics you may use the “method of
joints” to write equations for each node of the truss
1
. This set of equations is an example of a linear
system. Making the approximation

3/2 ≈ .8660, the equations for this truss are:
.5 T
1
+T
2
= R
1
= f
1
.866 T
1
= −R
2
= −.433 f
1
−.5 f
2
−.5 T
1
+.5 T
3
+T
4
= −f
1
.866 T
1
+.866 T
3
= 0
−T
2
−.5 T
3
+.5 T
5
+T
6
= 0
.866 T
3
+.866 T
5
= f
2
−T
4
−.5 T
5
+.5 T
7
= 0,
(9.1)
where T
i
represents the tension in the i-th member of the truss.
You could solve this system by hand with a little time and patience; systematically eliminating
variables and substituting. Obviously, it would be a lot better to put the equations on a computer
and let the computer solve it. In the next few lectures we will learn how to use a computer
effectively to solve linear systems. The first key to dealing with linear systems is to realize that they
are equivalent to matrices, which contain numbers, not variables.
As we discuss various aspects of matrices, we wish to keep in mind that the matrices that come up
in engineering systems are really large. It is not unusual in real engineering to use matrices whose
dimensions are in the thousands! It is frequently the case that a method that is fine for a 2 2 or
3 3 matrix is totally inappropriate for a 2000 2000 matrix. We thus want to emphasize methods
that work for large matrices.
1
See http://en.wikipedia.org/wiki/Truss or http://en.wikibooks.org/wiki/Statics for reference.
32
33
u r u
r r
A
B
C
D
E
1
2
3
4
5
6
7
E
f
1
c
f
2
T
R
2
T
R
3
'
R
1
Figure 9.1: An equilateral truss. Joints or nodes are labeled alphabetically, A, B, . . . and
Members (edges) are labeled numerically: 1, 2, . . . . The forces f
1
and f
2
are applied loads
and R
1
, R
2
and R
3
are reaction forces applied by the supports.
Linear systems are equivalent to matrix equations
The system of linear equations,
x
1
−2x
2
+ 3x
3
= 4
2x
1
−5x
2
+ 12x
3
= 15
2x
2
−10x
3
= −10,
is equivalent to the matrix equation,
_
_
1 −2 3
2 −5 12
0 2 −10
_
_
_
_
x
1
x
2
x
3
_
_
=
_
_
4
15
−10
_
_
,
which is equivalent to the augmented matrix,
_
_
1 −2 3 4
2 −5 12 15
0 2 −10 −10
_
_
.
The advantage of the augmented matrix, is that it contains only numbers, not variables. The reason
this is better is because computers are much better in dealing with numbers than variables. To solve
this system, the main steps are called Gaussian elimination and back substitution.
The augmented matrix for the equilateral truss equations (9.1) is given by:
_
_
_
_
_
_
_
_
_
_
.5 1 0 0 0 0 0 f
1
.866 0 0 0 0 0 0 −.433f
1
−.5 f
2
−.5 0 .5 1 0 0 0 −f
1
.866 0 .866 0 0 0 0 0
0 −1 −.5 0 .5 1 0 0
0 0 .866 0 .866 0 0 f
2
0 0 0 −1 −.5 0 .5 0
_
_
_
_
_
_
_
_
_
_
. (9.2)
Notice that a lot of the entries are 0. Matrices like this, called sparse, are common in applications
and there are methods specifically designed to efficiently handle sparse matrices.
34 LECTURE 9. INTRODUCTION TO LINEAR SYSTEMS
Triangular matrices and back substitution
Consider a linear system whose augmented matrix happens to be:
_
_
1 −2 3 4
0 −1 6 7
0 0 2 4
_
_
. (9.3)
Recall that each row represents an equation and each column a variable. The last row represents
the equation 2x
3
= 4. The equation is easily solved, i.e. x
3
= 2. The second row represents the
equation −x
2
+ 6x
3
= 7, but since we know x
3
= 2, this simplifies to: −x
2
+ 12 = 7. This is easily
solved, giving x
2
= 5. Finally, since we know x
2
and x
3
, the first row simplifies to: x
1
−10 +6 = 4.
Thus we have x
1
= 8 and so we know the whole solution vector: x = ¸8, 5, 2¸. The process we just
did is called back substitution, which is both efficient and easily programmed. The property that
made it possible to solve the system so easily is that A in this case is upper triangular. In the next
section we show an efficient way to transform an augmented matrix into an upper triangular matrix.
Gaussian Elimination
Consider the matrix:
A =
_
_
1 −2 3 4
2 −5 12 15
0 2 −10 −10
_
_
.
The first step of Gaussian elimination is to get rid of the 2 in the (2,1) position by subtracting 2
times the first row from the second row, i.e. (new 2nd = old 2nd - (2) 1st). We can do this because
it is essentially the same as adding equations, which is a valid algebraic operation. This leads to:
_
_
1 −2 3 4
0 −1 6 7
0 2 −10 −10
_
_
.
There is already a zero in the lower left corner, so we don’t need to eliminate anything there. To
eliminate the third row, second column, we need to subtract −2 times the second row from the third
row, (new 3rd = old 3rd - (-2) 2nd):
_
_
1 −2 3 4
0 −1 6 7
0 0 2 4
_
_
.
This is now just exactly the matrix in equation (9.3), which we can now solve by back substitution.
Matlab’s matrix solve command
In Matlab the standard way to solve a system Ax = b is by the command:
> x = A\b
This command carries out Gaussian elimination and back substitution. We can do the above com-
putations as follows:
> A = [1 -2 3 ; 2 -5 12 ; 0 2 -10]
> b = [4 15 -10]’
> x = A\b
35
Next, use the Matlab commands above to solve Ax = b when the augmented matrix for the system
is:
_
_
1 2 3 4
5 6 7 8
9 10 11 12
_
_
,
by entering:
> x1 = A\b Check the result by entering:
> A*x1 - b
You will see that the resulting answer satisfies the equation exactly. Next try solving using the
inverse of A:
> x2 = inv(A)*b
This answer can be seen to be inaccurate by checking:
> A*x2 - b
Thus we see one of the reasons why the inverse is never used for actual computations, only for
theory.
Exercises
9.1 Set f
1
= 1000N and f
2
= 5000N in the equations (9.1) for the equailateral truss. Input the
coefficient matrix A and the right hand side vector b in (9.2) into Matlab. Solve the system
using the command \ to find the tension in each member of the truss. Save the matrix A
as A_equil_truss and keep it for later use. (Enter save A_equil_truss A.) Print out and
turn in A, b and solution x.
9.2 Write each system of equations as an augmented matrix, then find the solutions using Gaussian
elimination and back substitution. Check your solutions using Matlab.
(a)
x
1
+x
2
= 2
4x
1
+ 5x
2
= 10
(b)
x
1
+ 2x
2
+ 3x
3
= −1
4x
1
+ 7x
2
+ 14x
3
= 3
x
1
+ 4x
2
+ 4x
3
= 1
Lecture 10
Some Facts About Linear Systems
Some inconvenient truths
In the last lecture we learned how to solve a linear system using Matlab. Input the following:
> A = ones(4,4)
> b = randn(4,1)
> x = A\b
As you will find, there is no solution to the equation Ax = b. This unfortunate circumstance is
mostly the fault of the matrix, A, which is too simple, its columns (and rows) are all the same. Now
try:
> b = ones(4,1)
> x = [ 1 0 0 0]’
> A*x
So the system Ax = b does have a solution. Still unfortunately, that is not the only solution. Try:
> x = [ 0 1 0 0]’
> A*x
We see that this x is also a solution. Next try: > x = [ -4 5 2.27 -2.27]’
> A*x
This x is a solution! It is not hard to see that there are endless possibilities for solutions of this
equation.
Basic theory
The most basic theoretical fact about linear systems is:
Theorem 1 A linear system Ax = b may have 0, 1, or infinitely many solutions.
Obviously, in most engineering applications we would want to have exactly one solution. The
following two theorems show that having one and only one solution is a property of A.
Theorem 2 Suppose A is a square (n n) matrix. The following are all equivalent:
1. The equation Ax = b has exactly one solution for any b.
2. det(A) ,= 0.
3. A has an inverse.
4. The only solution of Ax = 0 is x = 0.
5. The columns of A are linearly independent (as vectors).
6. The rows of A are linearly independent.
If A has these properties then it is called non-singular.
36
37
On the other hand, a matrix that does not have these properties is called singular.
Theorem 3 Suppose A is a square matrix. The following are all equivalent:
1. The equation Ax = b has 0 or ∞ many solutions depending on b.
2. det(A) = 0.
3. A does not have an inverse.
4. The equation Ax = 0 has solutions other than x = 0.
5. The columns of A are linearly dependent as vectors.
6. The rows of A are linearly dependent.
To see how the two theorems work, define two matrices (type in A1 then scroll up and modify to
make A2) :
A1 =
_
_
1 2 3
4 5 6
7 8 9
_
_
, A2 =
_
_
1 2 3
4 5 6
7 8 8
_
_
,
and two vectors:
b1 =
_
_
0
3
6
_
_
, b2 =
_
_
1
3
6
_
_
.
First calculate the determinants of the matrices:
> det(A1)
> det(A2)
Then attempt to find the inverses:
> inv(A1)
> inv(A2)
Which matrix is singular and which is non-singular? Finally, attempt to solve all the possible
equations Ax = b:
> x = A1\b1
> x = A1\b2
> x = A2\b1
> x = A2\b2
As you can see, equations involving the non-singular matrix have one and only one solution, but
equation involving a singular matrix are more complicated.
The residual vector
Recall that the residual for an approximate solution x of an equation f(x) = 0 is defined as r = f(x).
It is a measure of how close the equation is to being satisfied. For a linear system of equations we
define the residual of an approximate solution, x by
r = Ax −b. (10.1)
Notice that r is a vector. Its size (norm) is an indication of how close we have come to solving
Ax = b.
38 LECTURE 10. SOME FACTS ABOUT LINEAR SYSTEMS
Exercises
10.1 By hand, find all the solutions (if any) of the following linear system using the augmented
matrix and Gaussian elimination:
x
1
+ 2x
2
+ 3x
3
= 4,
4x
1
+ 5x
2
+ 6x
3
= 10, and
7x
1
+ 8x
2
+ 9x
3
= 14 .
10.2 (a) Write a Matlab function program mysolvecheck with input a number n that makes
a random n n matrix A and a random vector b, solves the linear system Ax = b,
calculates the norm of the residual r = Ax−b, and outputs that number as the error e.
(b) Write a Matlab script program that calls mysolvecheck 5 times each for n = 5, 10,
50, 100, 500, 1000, . . ., records and averages the results, and makes a log-log plot of the
average e vs. n.
Turn in the plot and the two programs.
Lecture 11
Accuracy, Condition Numbers and
Pivoting
In this lecture we will discuss two separate issues of accuracy in solving linear systems. The first,
pivoting, is a method that ensures that Gaussian elimination proceeds as accurately as possible.
The second, condition number, is a measure of how bad a matrix is. We will see that if a matrix
has a bad condition number, the solutions are unstable with respect to small changes in data.
The effect of rounding
All computers store numbers as finite strings of binary floating point digits. This limits numbers to
a fixed number of significant digits and implies that after even the most basic calculations, rounding
must happen.
Consider the following exaggerated example. Suppose that our computer can only store 2 significant
digits and it is asked to do Gaussian elimination on:
_
.001 1 3
1 2 5
_
.
Doing the elimination exactly would produce:
_
.001 1 3
0 −998 −2995
_
,
but rounding to 2 digits, our computer would store this as:
_
.001 1 3
0 −1000 −3000
_
.
Backsolving this reduced system gives:
x
1
= 0, x
2
= 3.
This seems fine until you realize that backsolving the unrounded system gives:
x
1
= −1, x
2
= 3.001.
Row Pivoting
A way to fix the problem is to use pivoting, which means to switch rows of the matrix. Since
switching rows of the augmented matrix just corresponds to switching the order of the equations,
39
40 LECTURE 11. ACCURACY, CONDITION NUMBERS AND PIVOTING
no harm is done:
_
1 2 5
.001 1 3
_
.
Exact elimination would produce:
_
1 2 5
0 .998 2.995
_
.
Storing this result with only 2 significant digits gives:
_
1 2 5
0 1 3
_
.
Now backsolving produces:
x
1
= −1, x
2
= 3,
which is exactly right.
The reason this worked is because 1 is bigger than 0.001. To pivot we switch rows so that the largest
entry in a column is the one used to eliminate the others. In bigger matrices, after each column is
completed, compare the diagonal element of the next column with all the entries below it. Switch
it (and the entire row) with the one with greatest absolute value. For example in the following
matrix, the first column is finished and before doing the second column, pivoting should occur since
[ −2[ > [1[:
_
_
1 −2 3 4
0 1 6 7
0 −2 −10 −10
_
_
.
Pivoting the 2nd and 3rd rows would produce:
_
_
1 −2 3 4
0 −2 −10 −10
0 1 6 7
_
_
.
Condition number
In some systems, problems occur even without rounding. Consider the following augmented matri-
ces:
_
1 1/2 3/2
1/2 1/3 1
_
and
_
1 1/2 3/2
1/2 1/3 5/6
_
.
Here we have the same A, but two different input vectors:
b
1
= (3/2, 1)

and b
2
= (3/2, 5/6)

which are pretty close to one another. You would expect then that the solutions x
1
and x
2
would
also be close. Notice that this matrix does not need pivoting. Eliminating exactly we get:
_
1 1/2 3/2
0 1/12 1/4
_
and
_
1 1/2 3/2
0 1/12 1/12
_
.
Now solving we find:
x
1
= (0, 3)

and x
2
= (1, 1)

which are not close at all despite the fact that we did the calculations exactly. This poses a new
problem: some matrices are very sensitive to small changes in input data. The extent of this
41
sensitivity is measured by the condition number. The definition of condition number is: consider
all small changes δA and δb in A and b and the resulting change, δx, in the solution x. Then:
cond(A) ≡ max
_
_
[δx[/[x[
|δA|
|A|
+
|δb|
|b|
_
_
= max
_
Relative error of output
Relative error of inputs
_
.
Put another way, changes in the input data get multiplied by the condition number to produce
changes in the outputs. Thus a high condition number is bad. It implies that small errors in the
input can cause large errors in the output.
In Matlab enter:
> H = hilb(2)
which should result in the matrix above. Matlab produces the condition number of a matrix with
the command:
> cond(H)
Thus for this matrix small errors in the input can get magnified by 19 in the output. Next try the
matrix:
> A = [ 1.2969 0.8648 ; .2161 .1441]
> cond(A)
For this matrix small errors in the input can get magnified by 2.5 10
8
in the output! (We will see
this happen in the exercise.) This is obviously not very good for engineering where all measurements,
constants and inputs are approximate.
Is there a solution to the problem of bad condition numbers? Usually, bad condition numbers in
engineering contexts result from poor design. So, the engineering solution to bad conditioning is
redesign.
Finally, find the determinant of the matrix A above:
> det(A)
which will be small. If det(A) = 0 then the matrix is singular, which is bad because it implies there
will not be a unique solution. The case here, det(A) ≈ 0, is also bad, because it means the matrix is
almost singular. Although det(A) ≈ 0 generally indicates that the condition number will be large,
they are actually independent things. To see this, find the determinant and condition number of
the matrix [1e-10,0;0,1e-10] and the matrix [1e+10,0;0,1e-10].
Exercises
11.1 Find the determinant and inverse of:
A =
_
1.2969 .8648
.2161 .1441
_
.
Let B be the matrix obtained from A by rounding off to three decimal places (1.2969 →1.297).
Find the determinant and inverse of B. How do A
−1
and B
−1
differ? Explain how this
happened.
Set b1 = [1.2969; 0.2161] and do x = A\b1 . Repeat the process but with a vector b2
obtained from b1 by rounding off to three decimal places. Explain exactly what happened.
Why was the first answer so simple? Why do the two answers differ by so much?
42 LECTURE 11. ACCURACY, CONDITION NUMBERS AND PIVOTING
11.2 Try
> B = [sin(sym(1)) sin(sym(2)); sin(sym(3)) sin(sym(4))]
> c = [1; 2]
> x = B\c
> pretty(x)
Next input the matrix:
Cs =
_
1 2
2 4
_
symbolically as above. Create a numerical version via Cn = double(Cs) and define the
two vectors d1 = [4; 8] and d2 = [1; 1]. Solve the systems Cs*x = d1, Cn*x = d1,
Cs*x = d2, and Cn*x = d2. Explain the results. Does the symbolic or non-symbolic way give
more information?
Lecture 12
LU Decomposition
In many applications where linear systems appear, one needs to solve Ax = b for many different
vectors b. For instance, a structure must be tested under several different loads, not just one. As in
the example of a truss (9.2), the loading in such a problem is usually represented by the vector b.
Gaussian elimination with pivoting is the most efficient and accurate way to solve a linear system.
Most of the work in this method is spent on the matrix A itself. If we need to solve several different
systems with the same A, and A is big, then we would like to avoid repeating the steps of Gaussian
elimination on A for every different b. This can be accomplished by the LU decomposition, which
in effect records the steps of Gaussian elimination.
LU decomposition
The main idea of the LU decomposition is to record the steps used in Gaussian elimination on A in
the places where the zero is produced. Consider the matrix:
A =
_
_
1 −2 3
2 −5 12
0 2 −10
_
_
.
The first step of Gaussian elimination is to subtract 2 times the first row from the second row. In
order to record what we have done, we will put the multiplier, 2, into the place it was used to make
a zero, i.e. the second row, first column. In order to make it clear that it is a record of the step and
not an element of A, we will put it in parentheses. This leads to:
_
_
1 −2 3
(2) −1 6
0 2 −10
_
_
.
There is already a zero in the lower left corner, so we don’t need to eliminate anything there. We
record this fact with a (0). To eliminate the third row, second column, we need to subtract −2
times the second row from the third row. Recording the −2 in the spot it was used we have:
_
_
1 −2 3
(2) −1 6
(0) (−2) 2
_
_
.
Let U be the upper triangular matrix produced, and let L be the lower triangular matrix with the
records and ones on the diagonal, i.e.:
L =
_
_
1 0 0
2 1 0
0 −2 1
_
_
, U =
_
_
1 −2 3
0 −1 6
0 0 2
_
_
,
43
44 LECTURE 12. LU DECOMPOSITION
then we have the following mysterious coincidence:
LU =
_
_
1 0 0
2 1 0
0 −2 1
_
_
_
_
1 −2 3
0 −1 6
0 0 2
_
_
=
_
_
1 −2 3
2 −5 12
0 2 −10
_
_
= A.
Thus we see that A is actually the product of L and U. Here L is lower triangular and U is
upper triangular. When a matrix can be written as a product of simpler matrices, we call that a
decomposition of A and this one we call the LU decomposition.
Using LU to solve equations
If we also include pivoting, then an LU decomposition for A consists of three matrices P, L and U
such that:
PA = LU. (12.1)
The pivot matrix P is the identity matrix, with the same rows switched as the rows of A are switched
in the pivoting. For instance:
P =
_
_
1 0 0
0 0 1
0 1 0
_
_
,
would be the pivot matrix if the second and third rows of A are switched by pivoting. Matlab will
produce an LU decomposition with pivoting for a matrix A with the following command:
> [L U P] = lu(A)
where P is the pivot matrix. To use this information to solve Ax = b we first pivot both sides by
multiplying by the pivot matrix:
PAx = Pb ≡ d.
Substituting LU for PA we get:
LUx = d.
Then we need only to solve two back substitution problems:
Ly = d
and
Ux = y.
In Matlab this would work as follows:
> A = rand(5,5)
> [L U P] = lu(A)
> b = rand(5,1)
> d = P*b
> y = L\d
> x = U\y
> rnorm = norm(A*x - b) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Check the result.
We can then solve for any other b without redoing the LU step. Repeat the sequence for a new
right hand side: c = randn(5,1); you can start at the third line. While this may not seem like a
big savings, it would be if A were a large matrix from an actual application.
45
Exercises
12.1 Solve the systems below by hand using the LU decomposition. Pivot if appropriate. In each
of the two problems, check by hand that LU = PA and Ax = b.
(a) A =
_
2 4
.5 4
_
, b =
_
0
−3
_
(b) A =
_
1 4
3 5
_
, b =
_
3
2
_
12.2 Finish the following Matlab function program:
function [x1, e1, x2, e2] = mysolve(A,b)
% Solves linear systems using the LU decomposition with pivoting
% and also with the built-in solve function A\b.
% Inputs: A -- the matrix
% b -- the right-hand vector
% Outputs: x1 -- the solution using the LU method
% e1 -- the norm of the residual using the LU method
% x2 -- the solution using the built-in method
% e2 -- the norm of the residual using the built-in method
format long
Test the program on both random matrices (randn(n,n)) and Hilbert matrices (hilb(n))
with n really big and document the results.
Lecture 13
Nonlinear Systems - Newton’s Method
An Example
The LORAN (LOng RAnge Navigation) system calculates the position of a boat at sea using signals
from fixed transmitters. From the time differences of the incoming signals, the boat obtains dif-
ferences of distances to the transmitters. This leads to two equations each representing hyperbolas
defined by the differences of distance of two points (foci). An example of such equations from [2]
are:
x
2
186
2

y
2
300
2
−186
2
= 1 and
(y −500)
2
279
2

(x −300)
2
500
2
−279
2
= 1 .
(13.1)
Solving two quadratic equations with two unknowns, would require solving a 4 degree polynomial
equation. We could do this by hand, but for a navigational system to work well, it must do the
calculations automatically and numerically. We note that the Global Positioning System (GPS)
works on similar principles and must do similar computations.
Vector Notation
In general, we can usually find solutions to a system of equations when the number of unknowns
matches the number of equations. Thus, we wish to find solutions to systems that have the form:
f
1
(x
1
, x
2
, x
3
, . . . , x
n
) = 0
f
2
(x
1
, x
2
, x
3
, . . . , x
n
) = 0
f
3
(x
1
, x
2
, x
3
, . . . , x
n
) = 0
.
.
.
f
n
(x
1
, x
2
, x
3
, . . . , x
n
) = 0.
(13.2)
For convenience we can think of (x
1
, x
2
, x
3
, . . . , x
n
) as a vector x and (f
1
, f
2
, . . . , f
n
) as a vector-
valued function f . With this notation, we can write the system of equations (13.2) simply as:
f (x) = 0,
i.e. we wish to find a vector that makes the vector function equal to the zero vector.
As in Newton’s method for one variable, we need to start with an initial guess x
0
. In theory, the more
variables one has, the harder it is to find a good initial guess. In practice, this must be overcome by
46
47
using physically reasonable assumptions about the possible values of a solution, i.e. take advantage
of engineering knowledge of the problem. Once x
0
is chosen, let
∆x = x
1
−x
0
.
Linear Approximation for Vector Functions
In the single variable case, Newton’s method was derived by considering the linear approximation
of the function f at the initial guess x
0
. From Calculus, the following is the linear approximation
of f at x
0
, for vectors and vector-valued functions:
f (x) ≈ f (x
0
) +Df (x
0
)(x −x
0
).
Here Df (x
0
) is an n n matrix whose entries are the various partial derivative of the components
of f . Specifically:
Df (x
0
) =
_
_
_
_
_
_
_
_
_
∂f1
∂x1
(x
0
)
∂f1
∂x2
(x
0
)
∂f1
∂x3
(x
0
) . . .
∂f1
∂xn
(x
0
)
∂f2
∂x1
(x
0
)
∂f2
∂x2
(x
0
)
∂f2
∂x3
(x
0
) . . .
∂f2
∂xn
(x
0
)
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
∂fn
∂x1
(x
0
)
∂fn
∂x2
(x
0
)
∂fn
∂x3
(x
0
) . . .
∂fn
∂xn
(x
0
)
_
_
_
_
_
_
_
_
_
. (13.3)
Newton’s Method
We wish to find x that makes f equal to the zero vectors, so let’s choose x
1
so that
f (x
0
) +Df (x
0
)(x
1
−x
0
) = 0.
Since Df (x
0
) is a square matrix, we can solve this equation by
x
1
= x
0
−(Df (x
0
))
−1
f (x
0
),
provided that the inverse exists. The formula is the vector equivalent of the Newton’s method for-
mula we learned before. However, in practice we never use the inverse of a matrix for computations,
so we cannot use this formula directly. Rather, we can do the following. First solve the equation
Df (x
0
)∆x = −f (x
0
). (13.4)
Since Df (x
0
) is a known matrix and −f (x
0
) is a known vector, this equation is just a system of
linear equations, which can be solved efficiently and accurately. Once we have the solution vector
∆x, we can obtain our improved estimate x
1
by:
x
1
= x
0
+ ∆x.
For subsequent steps, we have the following process:
• Solve Df (x
i
)∆x = −f (x
i
) for ∆x.
• Let x
i+1
= x
i
+ ∆x
48 LECTURE 13. NONLINEAR SYSTEMS - NEWTON’S METHOD
−4 −3 −2 −1 0 1 2 3 4
−4
−3
−2
−1
0
1
2
3
4
x
y
x
3
+y=1,
y
3
−x=−1
Figure 13.1: Graphs of the equations x
3
+ y = 1 and y
3
− x = −1. There is one and only
one intersection; at (x, y) = (1, 0).
An Experiment
We will solve the following set of equations:
x
3
+y = 1
y
3
−x = −1.
(13.5)
You can easily check that (x, y) = (1, 0) is a solution of this system. By graphing both of the
equations you can also see that (1, 0) is the only solution (Figure 13.1).
We can put these equations into vector form by letting x
1
= x, x
2
= y and
f
1
(x
1
, x
2
) = x
3
1
+x
2
−1
f
2
(x
1
, x
2
) = x
3
2
−x
1
+ 1.
(13.6)
or
f (x) =
_
x
3
1
+x
2
−1
x
3
2
−x
1
+ 1
_
. (13.7)
Now that we have the equation in the vector form, write the following script program:
format long
f = inline(’[ x(1)^3+x(2)-1 ; x(2)^3-x(1)+1 ]’);
x = [.5;.5]
x = fsolve(f,x)
Save this program as myfsolve.m and run it. You will see that the internal Matlab solving
command fsolve approximates the solution, but only to about 7 decimal places. While that would
be close enough for most applications, one would expect that we could do better on such a simple
problem.
49
Next we will implement Newton’s method for this problem. Modify your myfsolve program to:
% mymultnewton
format long
n=8 % set some number of iterations, may need adjusting
f = inline(’[x(1)^3+x(2)-1 ; x(2)^3-x(1)+1]’); % the vector function
Df = inline(’[3*x(1)^2, 1 ; -1, 3*x(2)^2]’); % the matrix of partial derivatives
x = [.5;.5] % starting guess
for i = 1:n
Dx = -Df(x)\f(x); % solve for increment
x = x + Dx % add on to get new guess
f(x) % see if f(x) is really zero
end
Save and run this program (as mymultnewton) and you will see that it finds the root exactly (to
machine precision) in only 6 iterations. Why is this simple program able to do better than Matlab’s
built-in program?
Exercises
13.1 (a) Put the LORAN equations (13.1) into the function form (13.2).
(b) Construct the matrix of partial derivatives Df in (13.3).
(c) Adapt the mymultnewton program to find a solution for these equations. By trying
different starting vectors, find at least three different solutions. (There are actually four
solutions.) Think of at least one way that the navigational system could determine
which solution is correct.
Lecture 14
Eigenvalues and Eigenvectors
Suppose that A is a square (n n) matrix. We say that a nonzero vector v is an eigenvector (ev)
and a number λ is its eigenvalue (ew) if
Av = λv. (14.1)
Geometrically this means that Av is in the same direction as v, since multiplying a vector by a
number changes its length, but not its direction.
Matlab has a built-in routine for finding eigenvalues and eigenvectors:
> A = pascal(4)
> [v e] = eig(A)
The results are a matrix v that contains eigenvectors as columns and a diagonal matrix e that
contains eigenvalues on the diagonal. We can check this by:
> v1 = v(:,1)
> A*v1
> e(1,1)*v1
Finding Eigenvalues for 2 2 and 3 3
If A is 22 or 33 then we can find its eigenvalues and eigenvectors by hand. Notice that Equation
(14.1) can be rewritten as:
Av −λv = 0. (14.2)
It would be nice to factor out the v from the right-hand side of this equation, but we can’t because
A is a matrix and λ is a number. However, since Iv = v, we can do the following:
Av −λv = Av −λIv
= (A−λI)v
= 0
(14.3)
If v is nonzero, then by Theorem 3 in Lecture 10 the matrix (A − λI) must be singular. By the
same theorem, we must have:
det(A−λI) = 0. (14.4)
This is called the characteristic equation.
50
51
For a 2 2 matrix, A−λI is calculated as in the following example:
A−λI =
_
1 4
3 5
_
−λ
_
1 0
0 1
_
=
_
1 4
3 5
_

_
λ 0
0 λ
_
=
_
1 −λ 4
3 5 −λ
_
.
(14.5)
The determinant of A−λI is then
det(A−λI) = (1 −λ)(5 −λ) −4 3
= −7 −6λ +λ
2
.
(14.6)
The characteristic equation det(A−λI) = 0 is simply a quadratic equation:
λ
2
−6λ −7 = 0. (14.7)
The roots of this equation are λ
1
= 7 and λ
2
= −1. These are the ew’s of the matrix A. Now to
find the corresponding ev’s we return to the equation (A−λI)v = 0. For λ
1
= 7, the equation for
the ev (A−λI)v = 0 is equivalent to the augmented matrix
_
−6 4 0
3 −2 0
_
. (14.8)
Notice that the first and second rows of this matrix are multiples of one another. Thus Gaussian
elimination would produce all zeros on the bottom row. Thus this equation has infinitely many
solutions, i.e. infinitely many ev’s. Since only the direction of the ev matters, this is okay, we only
need to find one of the ev’s. Since the second row of the augmented matrix represents the equation
3x −2y = 0,
we can let
v
1
=
_
2
3
_
.
This comes from noticing that (x, y) = (2, 3) is a solution of 3x −2y = 0.
For λ
2
= −1, (A−λI)v = 0 is equivalent to the augmented matrix:
_
2 4 0
3 6 0
_
.
Once again the first and second rows of this matrix are multiples of one another. For simplicity we
can let
v
2
=
_
−2
1
_
.
One can always check an ev and ew by multiplying:
Av
1
=
_
1 4
3 5
__
2
3
_
=
_
14
21
_
= 7
_
2
3
_
= 7v
1
and
Av
2
=
_
1 4
3 5
__
−2
1
_
=
_
2
−1
_
= −1
_
−2
1
_
= −1v
2
.
For a 3 3 matrix we could complete the same process. The det(A − λI) = 0 would be a cubic
polynomial and we would expect to usually get 3 roots, which are the ew’s.
52 LECTURE 14. EIGENVALUES AND EIGENVECTORS
Larger Matrices
For a n n matrix with n ≥ 4 this process is too long and cumbersome to complete by hand.
Further, this process is not well suited even to implementation on a computer program since it
involves determinants and solving a n-degree polynomial. For n ≥ 4 we need more ingenious
methods. These methods rely on the geometric meaning of ev’s and ew’s rather than solving
algebraic equations. We will overview these methods in Lecture 16.
Complex Eigenvalues
It turns out that the eigenvalues of some matrices are complex numbers, even when the matrix only
contains real numbers. When this happens the complex ew’s must occur in conjugate pairs, i.e.:
λ
1,2
= α ±iβ.
The corresponding ev’s must also come in conjugate pairs:
w = u ±iv.
In applications, the imaginary part of the ew, β, often is related to the frequency of an oscillation.
This is because of Euler’s formula
e
α+iβ
= e
α
(cos β +i sin β).
Certain kinds of matrices that arise in applications can only have real ew’s and ev’s. The most
common such type of matrix is the symmetric matrix. A matrix is symmetric if it is equal to its
own transpose, i.e. it is symmetric across the diagonal. For example,
_
1 3
3 −5
_
is symmetric and so we know beforehand that its ew’s will be real, not complex.
Exercises
14.1 Find the eigenvalues and eigenvectors of the following matrix by hand:
A =
_
2 1
1 2
_
.
14.2 Find the eigenvalues and eigenvectors of the following matrix by hand:
B =
_
1 −2
2 1
_
.
Can you guess the ew’s of the matrix
C =
_
a −b
b a
_
?
Lecture 15
An Application of Eigenvectors:
Vibrational Modes
One application of ew’s and ev’s is in the analysis of vibration problems. A simple nontrivial
vibration problem is the motion of two objects with equal masses m attached to each other and
fixed outer walls by equal springs with spring constants k, as shown in Figure 15.1.
Let x
1
denote the displacement of the first mass and x
2
the displacement of the second, and note
the displacement of the walls is zero. Each mass experiences forces from the adjacent springs
proportional to the stretch or compression of the spring. Ignoring any friction, Newton’s law of
motion ma = F, leads to:
m¨ x
1
= −k(x
1
−0) +k(x
2
−x
1
) = −2kx
1
+kx
2
m¨ x
2
= −k(x
2
−x
1
) +k(0 −x
2
) = kx
1
−2kx
2
. (15.1)
Dividing both sides by m we can write these equations in matrix form:
¨ x = −Ax, (15.2)
where
A =
k
m
B =
k
m
_
2 −1
−1 2
_
. (15.3)
For this type of equation, the general solution is:
x(t) = c
1
v
1
sin
_
_
k
m
λ
1
t +φ
1
_
+c
2
v
2
sin
_
_
k
m
λ
2
t +φ
2
_
(15.4)
where λ
1
and λ
2
are ew’s of B with corresponding ev’s v
1
and v
2
. We can interpret the ew’s as
the squares of the frequencies of oscillation. We can find the ew’s and ev’s of B using Matlab:
> B = [2 -1 ; -1 2]
> [v e] = eig(B)
This should produce a matrix v whose columns are the ev’s of B and a diagonal matrix e whose
entries are the ew’s of B. In the first eigenvector, v
1
, the two entries are equal. This represents the
mode of oscillation where the two masses move in sync with each other. The second ev, v
2
, has the
u u
E
x
1
E
x
2
Figure 15.1: Two equal masses attached to each other and fixed walls by equal springs.
53
54LECTURE 15. AN APPLICATION OF EIGENVECTORS: VIBRATIONAL MODES
u u
u u
u u
u u
u u
u u
u u
u u
u u
u u
Figure 15.2: Two vibrational modes of a simple oscillating system. In the left mode the
weights move together and in the right mode they move opposite. Note that the two modes
actually move at different speeds.
same entries but opposite signs. This represents the mode where the two masses oscillate in anti-
synchronization. Notice that the frequency for anti-sync motion is

3 times that of synchronous
motion.
Which of the two modes is the most dangerous for a structure or machine? It is the one with the
lowest frequency because that mode can have the largest displacement. Sometimes this mode is
called the fundamental mode.
To get the frequencies for the matrix A = k/mB, notice that if v
i
is one of the ev’s for B then:
Av
i
=
k
m
Bv
i
=
k
m
λ
i
v
i
.
Thus we can conclude that A has the same ev’s as B, but the ew’s are multiplied by the factor
k/m. Thus the two frequencies are:
_
k
m
and
_
3k
m
.
We can do the same for three equal masses. The corresponding matrix B would be:
B =
_
_
2 −1 0
−1 2 −1
0 −1 2
_
_
.
Find the ev’s and ew’s as above. There are three different modes. Interpret them from the ev’s.
55
Exercises
15.1 Find the frequencies and modes for 4 equal masses with equal springs. Interpret the modes.
15.2 Find the frequencies and modes for non-identical masses with equal springs in the following
cases. How does this effect the modes?
(a) Two masses with m
1
= 1 and m
2
= 2.
(b) Three masses with m
1
= 1, m
2
= 2 and m
3
= 3.
Lecture 16
Numerical Methods for Eigenvalues
As mentioned above, the ew’s and ev’s of an n n matrix where n ≥ 4 must be found numerically
instead of by hand. The numerical methods that are used in practice depend on the geometric
meaning of ew’s and ev’s which is equation (14.1). The essence of all these methods is captured in
the Power method, which we now introduce.
The Power Method
In the command window of Matlab enter the following:
> A = hilb(5)
> x = ones(5,1)
> x = A*x
> el = max(x)
> x = x/el
Compare the new value of x with the original. Repeat the last three lines (you can use the scroll up
button). Compare the newest value of x with the previous one and the original. Notice that there is
less change between the second two. Repeat the last three commands over and over until the values
stop changing. You have completed what is known as the Power Method. Now try the command:
> [v e] = eig(A)
The last entry in e should be the final el we computed. The last column in v is x/norm(x). Below
we explain why our commands gave this eigenvalue and eigenvector.
For illustration consider a 2 2 matrix whose ew’s are 1/3 and 2 and whose corresponding ev’s are
v
1
and v
2
. Let x
0
be any vector which is a combination of v
1
and v
2
, e.g.,
x
0
= v
1
+v
2
.
Now let x
1
be A times x
0
. It follows from (14.1) that
x
1
= Av
1
+Av
2
=
1
3
v
1
+ 2v
2
.
(16.1)
Thus the v
1
part is shrunk while the v
2
is stretched. If we repeat this process k times then:
x
k
= Ax
k−1
= A
k
x
0
=
_
1
3
_
k
v
1
+ 2
k
v
2
.
(16.2)
56
57
Clearly, x
k
grows in the direction of v
2
and shrinks in the direction of v
1
. This is the principle of
the Power Method, vectors multiplied by A are stretched most in the direction of the ev whose ew
has the largest absolute value.
The ew with the largest absolute value is called the dominant ew. In many applications this
quantity will necessarily be positive for physical reasons. When this is the case, the Matlab code
above will work since max(v) will be the element with the largest absolute value. In applications
where the dominant ew may be negative, the program must use flow control to determine the correct
number.
Summarizing the Power Method:
• Repeatedly multiply x by A and divide by the element with the largest absolute value.
• The element of largest absolute value converges to largest absolute ew.
• The vector converges to the corresponding ev.
Note that this logic only works when the eigenvalue largest in magnitude is real. If the matrix and
starting vector are real then the power method can never give a result with an imaginary part.
Eigenvalues with imaginary part mean the matrix has a rotational component, so the eigenvector
would not settle down either.
Symmetric, Positive-Definite Matrices
As noted in the previous paragraph, the power method can fail if A has complex eigenvalues. One
class of matrices that appear often in applications and for which the eigenvalues are always real are
called the symmetric matrices. A matrix is symmetric if:
A

= A,
i.e. A is symmetric with respect to reflections about its diagonal. Next we consider an even more
specialized class for which the eigenvalues are not only real, but positive. A symmetric matrix is
called positive definite if for all vectors v ,= 0 the following holds:
Av v > 0.
In summary:
• If A is symmetric then its eigenvalues are real.
• If A is symmetric positive definite, then its eigenvalues are positive numbers.
The Inverse Power Method
In the application of vibration analysis, the mode (ev) with the lowest frequency (ew) is the most
dangerous for the machine or structure. The Power Method gives us instead the largest ew, which
is the least important frequency. In this section we introduce a method, the Inverse Power Method
which produces exactly what is needed.
58 LECTURE 16. NUMERICAL METHODS FOR EIGENVALUES
The following facts are at the heart of the Inverse Power Method:
• If λ is an ew of A then 1/λ is an ew for A
−1
.
• The ev’s for A and A
−1
are the same.
Thus if we apply the Power Method to A
−1
we will obtain the largest absolute ew of A
−1
, which is
exactly the reciprocal of the smallest absolute ew of A. We will also obtain the corresponding ev,
which is an ev for both A
−1
and A. Recall that in the application of vibration mode analysis, the
smallest ew and its ev correspond exactly to the frequency and mode that we are most interested
in, i.e. the one that can do the most damage.
Here as always, we do not really want to calculate the inverse of A directly if we can help it.
Fortunately, multiplying x
i
by A
−1
to get x
i+1
is equivalent to solving the system:
Ax
i+1
= x
i
,
which can be done efficiently and accurately. Since iterating this process involves solving a linear
system with the same A but many different right hand sides, it is a perfect time to use the LU
decomposition to save computations. The following function program does n steps of the Inverse
Power Method.
function [v e] = myipm(A,n)
% performs n steps of the inverse power method on A
% Inputs:
% A -- a square matrix
% n -- the number of iterations to do
% Outputs:
% e -- the smallest absolute eigenvalue of A and
% v -- the corresponding eigenvector.
% LU decomposition of A with pivoting
[L U P] = lu(A);
% make an initial vector with ones
m = size(A,1);
v = ones(m,1);
% main loop of Inverse Power Method
for i = 1:n
pv = P*v;
y = L\pv;
v = U\y;
M = max(v);
m = min(v);
% use the biggest in absolute value
if abs(M) >= abs(m)
el = M;
else
el = m;
end
v = v/el;
end
e = 1/el;
59
Exercises
16.1 For each of the following matrices, perform one interation of the power method by hand
starting with a vector of all ones. State the resulting approximations of the eigenvalue and
eigenvector.
A =
_
1 2
3 4
_
(16.3)
B =
_
_
−2 1 0
1 −2 1
0 1 −2
_
_
(16.4)
16.2 (a) Write a Matlab function program mypm that inputs a matrix and a tolerance, applies
the power method until the change in the vector is less than the tolerance, and outputs
the number of steps used and the estimated eigenvalue and eigenvector.
(b) Test your program on the matrices A and B in the previous exercise.
Lecture 17
The QR Method*
The Power Method and Inverse Power Method each give us only one ew–ev pair. While both of
these methods can be modified to give more ew’s and ev’s, there is a better method for obtaining
all the ew’s called the QR method. This is the basis of all modern ew software, including Matlab,
so we summarize it briefly here.
The QR method uses the fact that any square matrix has a QR decomposition. That is, for any A
there are matrices Q and R such the A = QR where Q has the property
Q
−1
= Q

and R is upper triangular. A matrix Q with the property that its transpose equals its inverse is
called an orthogonal matrix, because its column vectors are mutually orthogonal.
The QR method consists of iterating following steps:
- Transform A into a tridiagonal matrix H.
- decompose H in QR.
- multiply Q and R together in reverse order to form a new H.
The diagonal of H will converge to the eigenvalues.
We details of what makes this method converge are beyond the scope of the this book. However,
we note the following theory behind it for those with more familiarity with linear algrebra. First
the Hessian matrix H is obtained from A by a series of similarity transformation, thus it has the
same ew’s as A. Secondly, if we denote by H
0
, H
1
, H
2
, . . ., the sequence of matrices produced by
the iteration, then
H
i+1
= R
i
Q
i
= Q
−1
i
Q
i
R
i
Q
i
= Q

i
H
i
Q
i
.
Thus each H
i+1
is a related to H
i
by an (orthogonal) simularity transformation and so they have
the same ew’s as A.
There is a built-in QR decomposition in Matlab which is called with the command: [Q R] = qr(A).
Thus the following program implements QR method until it converges:
60
61
function E = myqrmethod(A)
[m n] = size(A);
if m ~= n
warning(’The input matrix is not square.’)
return
end
H = hess(A);
E = diag(H);
change = 1;
steps = 0;
while change > 0
Eold = E;
[Q R] = qr(H);
H = R*Q;
E = diag(H);
change = norm(E - Eold);
steps = steps +1;
end
steps
As you can see the main steps of the program are very simple. The really hard calculations are
contained in the built-in command qr(A).
Run this program and compare the results with Matlab’s built in command:
> format long
> format compact
> A = hilb(5)
> Eqr = myqrmtheod(A)
> Eml =eig(A)
Exercises
17.1 Modify myqrmethod to stop after a maximum number of iterations. Use the modified program,
with the maximum iterations set to 1000, on the matrix A = hilb(n) with n equal to 10, 50,
and 200. Use the norm to compare the results to the ew’s obtained from Matlab’s built-in
program eig. Turn in a printout of your program and a brief report on the experiment.
Lecture 18
Iterative solution of linear systems*
Newton refinement
Conjugate gradient method
62
Review of Part II
Methods and Formulas
Basic Matrix Theory:
Identity matrix: AI = A, IA = A, and Iv = v
Inverse matrix: AA
−1
= I and A
−1
A = I
Norm of a matrix: [A[ ≡ max
|v|=1
[Av[
A matrix may be singular or nonsingular. See Lecture 10.
Solving Process:
Gaussian Elimination produces LU decomposition
Row Pivoting
Back Substitution
Condition number:
cond(A) ≡ max
_
_
[δx[/[x[
|δA|
|A|
+
|δb|
|b|
_
_
= max
_
Relative error of output
Relative error of inputs
_
.
A big condition number is bad; in engineering it usually results from poor design.
LU factorization:
PA = LU.
Solving steps:
Multiply by P: d = Pb
Forwardsolve: Ly = d
Backsolve: Ux = y
Eigenvalues and eigenvectors:
A nonzero vector v is an eigenvector (ev) and a number λ is its eigenvalue (ew) if
Av = λv.
Characteristic equation: det(A−λI) = 0
Equation of the eigenvector: (A−λI)v = 0
Complex ew’s:
Occur in conjugate pairs: λ
1,2
= α ±iβ
and ev’s must also come in conjugate pairs: w = u ±iv.
Vibrational modes:
Eigenvalues are frequencies squared. Eigenvectors are modes.
63
64 REVIEW OF PART II
Power Method:
- Repeatedly multiply x by A and divide by the element with the largest absolute value.
- The element of largest absolute value converges to largest absolute ew.
- The vector converges to the corresponding ev.
- Convergence assured for a real symmetric matrix, but not for an arbitrary matrix, which may not
have real eigenvalues at all.
Inverse Power Method:
- Apply power method to A
−1
.
- Use solving rather than the inverse.
- If λ is an ew of A then 1/λ is an ew for A
−1
.
- The ev’s for A and A
−1
are the same.
Symmetric and Positive definite:
- Symmetric: A = A

.
- If A is symmetric its ew’s are real.
- Positive definite: Ax x > 0.
- If A is positive definite, then its ew’s are positive.
QR method:
- Transform A into H the Hessian form of A.
- Decompose H in QR.
- Multiply Q and R together in reverse order to form a new H.
- Repeat
- The diagonal of H will converge to the ew’s of A.
Matlab
Matrix arithmetic:
> A = [ 1 3 -2 5 ; -1 -1 5 4 ; 0 1 -9 0] . . . . . . . . . . . . . . . . . Manually enter a matrix.
> u = [ 1 2 3 4]’
> A*u
> B = [3 2 1; 7 6 5; 4 3 2]
> B*A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . multiply B times A.
> 2*A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . multiply a matrix by a scalar.
> A + A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . add matrices.
> A + 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . add 3 to every entry of a matrix.
> B.*B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . component-wise multiplication.
> B.^3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . component-wise exponentiation.
Special matrices:
> I = eye(3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . identity matrix
> D = ones(5,5)
> O = zeros(10,10)
> C = rand(5,5) . . . . . . . . . . . . . . . . . . . . . . . . . . . . random matrix with uniform distribution in [0, 1].
> C = randn(5,5) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . random matrix with normal distribution.
> hilb(6)
> pascal(5)
General matrix commands:
> size(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . gives the dimensions (mn) of A.
> norm(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . gives the norm of the matrix.
65
> det(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the determinant of the matrix.
> max(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the maximum of each row.
> min(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the minimum in each row.
> sum(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . sums each row.
> mean(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the average of each row.
> diag(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . just the diagonal elements.
> inv(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . inverse of the matrix.
> C’ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . transpose of the matrix.
Matrix decompositions:
> [L U P] = lu(C)
> [Q R] = qr(C)
> [U S V] = svd(C) . . . . . . . . . . singular value decomposition (important, but we did not use it).
> H = hess(C) transform into a Hessian tri-diagonal matrix. A matrix that has the same eigen-
values as A. > [U T] = schur(C) . . . . . . . . . . . . . . . . . . . . . . . . . . Schur Decomposition A = U

TU .
> R = chol(C*C’) Cholesky decomposition of a symmetric, positive definite matrix, A = R

R.
66 REVIEW OF PART II
Part III
Functions and Data
c _Copyright, Todd Young and Martin Mohlenkamp, Mathematics Department, Ohio University, 2007
Lecture 19
Polynomial and Spline Interpolation
A Chemical Reaction
In a chemical reaction the concentration level y of the product at time t was measured every half
hour. The following results were found:
t 0 .5 1.0 1.5 2.0
y 0 .19 .26 .29 .31
We can input this data into Matlab as:
> t1 = 0:.5:2
> y1 = [ 0 .19 .26 .29 .31 ]
and plot the data with:
> plot(t1,y1)
Matlab automatically connects the data with line segments. This is the simplest form of interpo-
lation, meaning fitting a graph (of a function) between data points. What if we want a smoother
graph? Try:
> plot(t1,y1,’*’)
which will produce just asterisks at the data points. Next click on Tools, then click on the
Basic Fitting option. This should produce a small window with several fitting options. Begin
clicking them one at a time, clicking them off before clicking the next. Which ones produce a good-
looking fit? You should note that the spline, the shape-preserving interpolant and the 4th degree
polynomial produce very good curves. The others do not. We will discuss polynomial interpolation
and spline interpolation in this lecture.
Polynomial Fitting
The most general degree n polynomial is:
p
n
(x) = a
n
x
n
+a
n−1
x
n−1
+. . . +a
1
x +a
0
.
If we have exactly n + 1 data points, that is enough to exactly determine the n + 1 coefficients of
p
n
(x) (as long as the data does not have repeated x values). If there are more data points, that
would give us an overdetermined system (more equations than unknowns) and if there is less data
the system would be undetermined.
Conversely, if we have n data points, then an n−1 degree polynomial has exactly enough coefficients
to fit the data. This is the case in the example above; there are 5 data points so there is exactly
one 4th degree polynomial that fits the data.
When we tried to use a 5th or higher degree polynomial Matlab returned a warning that the
polynomial is not unique since “degree >= number of data points”.
68
69
When we tried a lower degree polynomial, we did not get a warning, but we did notice that the
resulting curve does not hit all the data points. If there was an overdetermined system how did
Matlab come up with a pretty good approximation (for cubic)? The answer is the Least Squares
Method, which we will study later.
Predicting the future?
Suppose we want to use the data to extrapolate into the future. Set the plot to the 4th degree
polynomial. Then click the Edit button and select the Axes Properties option. A box should
appear that allows you to adjust the domain of the x axes. Change the upper limit of the x-axis
from 2 to 4. Based on the 4th degree polynomial, what will the chemical reaction do in the future?
Is this reasonable?
Next change from 4th degree polynomial to spline interpolant. According to the spline, what will
the chemical reaction do in the future? Try the shape-preserving interpolant also.
From our (limited) knowledge of chemical reactions, what should be the behavior as time goes
on? It should reach a limiting value (chemical equilibrium). Could we use the data to predict this
equilibrium value? Yes, we could and it is done all the time in many different contexts, but to do
so we need to know that there is an equilibrium to predict. This requires that we understand the
chemistry of the problem. Thus we have the following principle: To extrapolate beyond the data,
one must have some knowledge of the process.
More data
Generally one would think that more data is better. Input the following data vectors:
> t2 = [ 0 .1 .4 .5 .6 1.0 1.4 1.5 1.6 1.9 2.0]
> y2 = [ 0 .06 .17 .19 .21 .26 .29 .29 .30 .31 .31]
There are 11 data points, so a 10-th degree polynomial will fit the data. However, this does not give
a good graph. Thus: A polynomial fit is better for small data sets.
Finally, we note that each data point (x
i
, y
i
) gives an equation:
p
n
(x
i
) = a
0
+a
1
x
i
+a
2
x
2
i
+ +a
n
x
n
i
= y
i
.
The unknowns are the coefficients a
0
, . . . , a
n
. This equation is linear in these unknowns, so deter-
mining a polynomial fit requires solving a linear system of equations. This makes polynomial fits
quick to calculate since solving linear systems is what computers do best.
A challenging data set
Input the following data set:
> x = -4:1:5
> y = [ 0 0 0 1 1 1 0 0 0 0]
and plot it:
> plot(x,y,’*’)
There are 10 data points, so there is a unique 9 degree polynomial that fits the data. Under Tools
and Basic Fitting select the 9th degree polynomial fit. How does it look? De-select the 9th degree
polynomial and select the spline interpolant. This should produce a much more satisfactory graph
and the shape-preserving spline should be even better. In this section we learn what a spline is.
70 LECTURE 19. POLYNOMIAL AND SPLINE INTERPOLATION
The idea of a spline
The general idea of a spline is this: on each interval between data points, represent the graph with a
simple function. The simplest spline is something very familiar to you; it is obtained by connecting
the data with lines. Since linear is the most simple function of all, linear interpolation is the simplest
form of spline. The next simplest function is quadratic. If we put a quadratic function on each
interval then we should be able to make the graph a lot smoother. If we were really careful then
we should be able to make the curve smooth at the data points themselves by matching up the
derivatives. This can be done and the result is called a quadratic spline. Using cubic functions
or 4th degree functions should be smoother still. So, where should we stop? There is an almost
universal consensus that cubic is the optimal degree for splines and so we focus the rest of the lecture
on cubic splines.
Cubic spline
Again, the basic idea of the cubic spline is that we represent the function by a different cubic function
on each interval between data points. That is, if there are n data points, then the spline S(x) is the
function:
S(x) =
_
_
_
C
1
(x), x
0
≤ x ≤ x
1
C
i
(x), x
i−1
≤ x ≤ x
i
C
n
(x), x
n−1
≤ x ≤ x
n
(19.1)
where each C
i
is a cubic function. The most general cubic function has the form:
C
i
(x) = a
i
+b
i
x +c
i
x
2
+d
i
x
3
.
To determine the spline we must determine the coefficients, a
i
, b
i
, c
i
, and d
i
for each i. Since there
are n intervals, there are 4n coefficients to determine. First we require that the spline be exact at
the data:
C
i
(x
i−1
) = y
i−1
and C
i
(x
i
) = y
i
, (19.2)
at every data point. In other words,
a
i
+b
i
x
i−1
+c
i
x
2
i−1
+d
i
x
3
i−1
= y
i−1
and a
i
+b
i
x
i
+c
i
x
2
i
+d
i
x
3
i
= y
i
.
Notice that there are 2n of these conditions. Then to make S(x) as smooth as possible we require:
C

i
(x
i
) = C

i+1
(x
i
)
C
′′
i
(x
i
) = C
′′
i+1
(x
i
),
(19.3)
at all the internal points, i.e. x
1
, x
2
, x
3
, . . . , x
n−1
. In terms of the points these conditions can be
written as:
b
i
+ 2c
i
x
i
+ 3d
i
x
2
i
= b
i+1
+ 2c
i+1
x
i
+ 3d
i+1
x
2
i
2c
i
+ 6d
i
x
i
= 2c
i+1
+ 6d
i+1
x
i
.
(19.4)
There are 2(n − 1) of these conditions. Since each C
i
is cubic, there are a total of 4n coefficients
in the formula for S(x). So far we have 4n −2 equations, so we are 2 equations short of being able
to determine all the coefficients. At this point we have to make a choice. The usual choice is to
require:
C
′′
1
(x
0
) = C
′′
n
(x
n
) = 0. (19.5)
These are called natural or simple boundary conditions. The other common option is called clamped
boundary conditions:
C

1
(x
0
) = C

n
(x
n
) = 0. (19.6)
71
The terminology used here is obviously parallel to that used for beams. That is not the only parallel
between beams and cubic splines. It is an interesting fact that a cubic spline is exactly the shape
of a (linear) beam restrained to match the data by simple supports.
Note that the equations above are all linear equations with respect to the unknowns (coefficients).
This feature makes splines easy to calculate since solving linear systems is what computers do best.
Exercises
19.1 You are given the following data:
> t = [ 0 .1 .499 .5 .6 1.0 1.4 1.5 1.899 1.9 2.0]
> y = [ 0 .06 .17 .19 .21 .26 .29 .29 .30 .31 .31]
(a) Plot the data, then try a polynomial fit of the correct degree to interpolate this number
of data points: What do you observe. Give an explanation of this error, in particular
why is the term badly conditioned used?
(b) Plot the data along with a spline interpolant. How does this compare with the plot
above? What is a way to make the plot better?
Lecture 20
Least Squares Fitting: Noisy Data
Very often data has a significant amount of noise. The least squares approximation is intentionally
well-suited to represent noisy data. The next illustration shows the effects noise can have and how
least squares is used.
Traffic flow model
Suppose you are interested in the time it takes to travel on a certain section of highway for the sake
of planning. According to theory, assuming up to a moderate amount of traffic, the time should be
approximately:
T(x) = ax +b
where b is the travel time when there is no other traffic, and x is the current number of cars on the
road (in hundreds). To determine the coefficients a and b you could run several experiments which
consist of driving the highway at different times of day and also estimating the number of cars on
the road using a counter. Of course both of these measurements will contain noise, i.e. random
fluctuations.
We could simulate such data in Matlab as follows:
> x = 1:.1:6;
> T = .1*x + 1;
> Tn = T + .1*randn(size(x));
> plot(x,Tn,’.’)
The data should look like it lies on a line, but with noise. Click on the Tools button and choose
Basic fitting. Then choose a linear fit. The resulting line should go through the data in
what looks like a very reasonable way. Click on show equations. Compare the equation with
T(x) = .1x +1. The coefficients should be pretty close considering the amount of noise in the plot.
Next, try to fit the data with a spline. The result should be ugly. We can see from this example
that splines are not suited to noisy data.
How does Matlab obtain a very nice line to approximate noisy data? The answer is a very standard
numerical/statistical method known as least squares.
Linear least squares
Consider in the previous example that we wish to fit a line to a lot of data that does not exactly
lie on a line. For the equation of the line we have only two free coefficients, but we have many data
points. We can not possibly make the line go through every data point, we can only wish for it to
come reasonably close to as many data points as possible. Thus, our line must have an error with
72
73
respect to each data point. If ℓ(x) is our line and ¦(x
i
, y
i
)¦ are the data, then
e
i
= y
i
−ℓ(x
i
)
is the error of ℓ with respect to each (x
i
, y
i
). To make ℓ(x) reasonable, we wish to simultaneously
minimize all the errors: ¦e
1
, e
2
, . . . , e
n
¦. There are many possible ways one could go about this, but
the standard one is to try to minimize the sum of the squares of the errors. That is, we denote by
c the sum of the squares:
c =
n

i=1
(y
i
−ℓ(x
i
))
2
=
n

i=1
(y
i
−ax
i
−b)
2
. (20.1)
In the above expression x
i
and y
i
are given, but we are free to choose a and b, so we can think of
c as a function of a and b, i.e. c(a, b). In calculus, when one wishes to find a minimum value of a
function of two variables, we set the partial derivatives equal to zero:
∂c
∂a
= −2
n

i=1
(y
i
−ax
i
−b) x
i
= 0
∂c
∂b
= −2
n

i=1
(y
i
−ax
i
−b) = 0.
(20.2)
We can simplify these equations to obtain:
_
n

i=1
x
2
i
_
a +
_
n

i=1
x
i
_
b =
n

i=1
x
i
y
i
_
n

i=1
x
i
_
a +nb =
n

i=1
y
i
.
(20.3)
Thus, the whole problem reduces to a 2 by 2 linear system to find the coefficients a and b. The
entries in the matrix are determined from simple formulas using the data. The process is quick and
easily automated, which is one reason it is very standard.
We could use the same process to obtain a quadratic or higher polynomial fit to data. If we try to
fit an n degree polynomial, the software has to solve an n n linear system, which is easily done.
This is what Matlab’s basic fitting tool uses to obtain an n degree polynomial fit whenever the
number of data points is more than n + 1.
Drag coefficients
Drag due to air resistance is proportional to the square of the velocity, i.e. d = kv
2
. In a wind
tunnel experiment the velocity v can be varied by setting the speed of the fan and the drag can be
measured directly (it is the force on the object). In this and every experiment some random noise
will occur. The following sequence of commands replicates the data one might receive from a wind
tunnel:
> v = 0:1:60;
> d = .1234*v.^2;
> dn = d + .4*v.*randn(size(v));
> plot(v,dn,’*’)
The plot should look like a quadratic, but with some noise. Using the tools menu, add a quadratic
fit and enable the “show equations” option. What is the coefficient of x
2
? How close is it to 0.1234?
74 LECTURE 20. LEAST SQUARES FITTING: NOISY DATA
Note that whenever you select a polynomial in Matlab with a degree less than n−1 Matlab will
produce a least squares fit.
You will notice that the quadratic fit includes both a constant and linear term. We know from
the physical situation that these should not be there; they are remnants of noise and the fitting
process. Since we know the curve should be kv
2
, we can do better by employing that knowledge.
For instance, we know that the graph of d versus v
2
should be a straight line. We can produce this
easily:
> vs = v.^2;
> plot(vs,dn,’*’)
By changing the independent variable from v to v
2
we produced a plot that looks like a line with
noise. Add a linear fit. What is the linear coefficient? This should be closer to 0.1234 than using a
quadratic fit.
The second fit still has a constant term, which we know should not be there. If there was no
noise, then at every data point we would have k = d/v
2
. We can express this as a linear system
vs’*k = dn’, which is badly overdetermined since there are more equations than unknowns. Since
there is noise, each point will give a different estimate for k. In other words, the overdetermined
linear system is also inconsistent. When Matlab encounters such systems, it automatically gives a
least squares solution of the matrix problem, i.e. one that minimizes the sum of the squared errors,
which is exactly what we want. To get the least squares estimate for k, do
> k = vs’\dn’
This will produce a number close to .1234.
Note that this is an application where we have physical knowledge. In this situation extrapolation
would be meaningful. For instance we could use the plot to find the predicted drag at 80 mph.
Exercises
20.1 Find two tables of data in an engineering textbook or engineering website. Plot each and
decide if the data is best represented by a polynomial interpolant, spline interpolant, or least
squares fit polynomial. Turn in the best plot of each. Indicate the source and meaning of the
data.
20.2 The following table contains information from a chemistry experiment in which the concen-
tration of a product was measured at one minute intervals.
T 0 1 2 3 4 5 6 7
C 3.033 3.306 3.672 3.929 4.123 4.282 4.399 4.527
Plot this data. Suppose it is known that this chemical reaction should follow the law:
c = a −b exp(−0.2t). Following the example in the notes about the drag coefficients, change
one of the variables so that the law is a linear function. Then plot the new variables and use
the linear fit option to estimate a and b. What will be the eventual concentration? Finally,
plot the graph of a −b exp(−0.2t) on the interval [0,10], along with the data (as *).
Lecture 21
Integration: Left, Right and Trapezoid
Rules
The Left and Right endpoint rules
In this section, we wish to approximate a definite integral:
_
b
a
f(x) dx
where f(x) is a continuous function. In calculus we learned that integrals are (signed) areas and can
be approximated by sums of smaller areas, such as the areas of rectangles. We begin by choosing
points ¦x
i
¦ that subdivide [a, b]:
a = x
0
< x
1
< . . . < x
n−1
< x
n
= b.
The subintervals [x
i−1
, x
i
] determine the width ∆x
i
of each of the approximating rectangles. For
the height, we learned that we can choose any height of the function f(x

i
) where x

i
∈ [x
i−1
, x
i
].
The resulting approximation is:
_
b
a
f(x) dx ≈
n

i=1
f(x

i
)∆x
i
.
To use this to approximate integrals with actual numbers, we need to have a specific x

i
in each
interval. The two simplest (and worst) ways to choose x

i
are as the left-hand point or the right-hand
point of each interval. This gives concrete approximations which we denote by L
n
and R
n
given by
L
n
=
n

i=1
f(x
i−1
)∆x
i
and
R
n
=
n

i=1
f(x
i
)∆x
i
.
function L = myleftsum(x,y)
% produces the left sum from data input.
% Inputs: x -- vector of the x coordinates of the partition
% y -- vector of the corresponding y coordinates
% Output: returns the approximate integral
n = max(size(x)); % safe for column or row vectors
L = 0;
for i = 1:n-1
L = L + y(i)*(x(i+1) - x(i));
end
75
76 LECTURE 21. INTEGRATION: LEFT, RIGHT AND TRAPEZOID RULES
E
T
¨
¨
¨
¨
¨
¨
¨
¨
¨
¨
¨
E
T
¨
¨
¨
¨
¨
¨
¨
¨
¨
¨
¨
Figure 21.1: The left and right sums, L
n
and R
n
.
Often we can take ¦x
i
¦ to be evenly spaced, with each interval having the same width:
h =
b −a
n
,
where n is the number of subintervals. If this is the case, then L
n
and R
n
simplify to:
L
n
=
b −a
n
n−1

i=0
f(x
i
) and (21.1)
R
n
=
b −a
n
n

i=1
f(x
i
). (21.2)
The foolishness of choosing left or right endpoints is illustrated in Figure 21.1. As you can see, for
a very simple function like f(x) = 1 +.5x, each rectangle of L
n
is too short, while each rectangle of
R
n
is too tall. This will hold for any increasing function. For decreasing functions L
n
will always
be too large while R
n
will always be too small.
The Trapezoid rule
Knowing that the errors of L
n
and R
n
are of opposite sign, a very reasonable way to get a better
approximation is to take an average of the two. We will call the new approximation T
n
:
T
n
=
L
n
+R
n
2
.
This method also has a straight-forward geometric interpretation. On each subrectangle we are
using:
A
i
=
f(x
i−1
) +f(x
i
)
2
∆x
i
,
which is exactly the area of the trapezoid with sides f(x
i−1
) and f(x
i
). We thus call the method
the trapezoid method. See Figure 21.2.
We can rewrite T
n
as
T
n
=
n

i=1
f(x
i−1
) +f(x
i
)
2
∆x
i
.
77
E
T
r
r
r
r
r

r
r
r
r
r
Figure 21.2: The trapezoid rule, T
n
.
In the evenly spaced case, we can write this as
T
n
=
b −a
2n
_
f(x
0
) + 2f(x
1
) +. . . + 2f(x
n−1
) +f(x
n
)
_
. (21.3)
Caution: The convention used here is to begin numbering the points at 0, i.e. x
0
= a; this allows n
to be the number of subintervals and the index of the last point x
n
. However, Matlab’s indexing
convention begins at 1. Thus, when programming in Matlab, the first entry in x will be x
0
,
i.e. x(1)= x
0
and x(n+1)= x
n
.
If we are given data about the function, rather than a formula for the function, often the data are
not evenly spaced. The following function program could then be used.
function T = mytrap(x,y)
% Calculates the Trapezoid rule approximation of the integral from input data
% Inputs: x -- vector of the x coordinates of the partitian
% y -- vector of the corresponding y coordinates
% Output: returns the approximate integral
n = max(size(x)); % safe for column or row vectors
T = 0;
for i = 1:n-1
T = T + .5*(y(i)+y(i+1))*(x(i+1) - x(i));
end
Using the Trapezoid rule for areas in the plane
In multi-variable calculus you were supposed to learn that you can calculate the area of a region R
in the plane by calculating the line integral:
A = −
_
C
y dx, (21.4)
where C is the counter-clockwise curve around the boundary of the region. We can represent such a
curve by consecutive points on it, i.e. ¯ x = (x
0
, x
1
, x
2
, . . . , x
n−1
, x
n
), and ¯ y = (y
0
, y
1
, y
2
, . . . , y
n−1
, y
n
).
78 LECTURE 21. INTEGRATION: LEFT, RIGHT AND TRAPEZOID RULES
Since we are assuming the curve ends where it begins, we require (x
n
, y
n
) = (x
0
, y
0
). Applying the
trapezoid method to the integral (21.4) gives
A = −
n

i=1
y
i−1
+y
i
2
(x
i
−x
i−1
).
This formula then is the basis for calculating areas when coordinates of boundary points are known,
but not necessarily formulas for the boundaries such as in a land survey.
In the following script, we can use this method to approximate the area of a unit circle using n
points on the circle:
% Calculates pi using a trapezoid approximation of the unit circle.
format long
n = 10;
t = linspace(0,2*pi,n+1);
x = cos(t);
y = sin(t);
plot(x,y)
A = 0
for i = 1:n
A = A - (y(i)+y(i+1))*(x(i+1)-x(i))/2;
end
A
Exercises
21.1 For the integral
_
2
1

xdx calculate L
4
, R
4
, and T
4
with even spacing (by hand, but use
a calculator) using formulas (21.1), (21.2) and (21.3). Find the percentage error of these
approximations, using the exact value.
21.2 Write a function program myints whose inputs are f, a, b and n and whose outputs are L, R
and T, the left, right and trapezoid integral approximations for f on [a, b] with n subintervals.
To make it efficient, use x = linspace(a,b,n+1) to make the x values and y = f(x) to make
the y values, add the 2nd to nth y entries once, and then add on the missing terms to obtain
the left, right and trapezoid approximations.
Change to format long and apply your program to the integral
_
2
1

xdx. Compare with the
results of the previous exercise. Also find L
100
, R
100
and T
100
and the relative errors of these
approximations.
Turn in the program and a brief summary of the results.
Lecture 22
Integration: Midpoint and Simpson’s
Rules
Midpoint rule
If we use the endpoints of the subintervals to approximate the integral, we run the risk that the values
at the endpoints do not accurately represent the average value of the function on the subinterval. A
point which is much more likely to be close to the average would be the midpoint of each subinterval.
Using the midpoint in the sum is called the midpoint rule. On the i-th interval [x
i−1
, x
i
] we will call
the midpoint ¯ x
i
, i.e.
¯ x
i
=
x
i−1
+x
i
2
.
If ∆x
i
= x
i
−x
i−1
is the length of each interval, then using midpoints to approximate the integral
would give the formula:
M
n
=
n

i=1
f(¯ x
i
)∆x
i
.
For even spacing, ∆x = (b −a)/n, and the formula is:
M
n
=
b −a
n
n

i=1
f(¯ x
i
) =
b −a
n
(ˆ y
1
+ ˆ y
2
+. . . + ˆ y
n
), (22.1)
where we define ˆ y
i
= f(¯ x
i
).
While the midpoint method is obviously better than L
n
or R
n
, it is not obvious that it is actually
better than the trapezoid method T
n
, but it is.
Simpson’s rule
Consider Figure 22.1. If f is not linear on a subinterval, then it can be seen that the errors for
the midpoint and trapezoid rules behave in a very predictable way, they have opposite sign. For
example, if the function is concave up then T
n
will be too high, while M
n
will be too low. Thus it
makes sense that a better estimate would be to average T
n
and M
n
. However, in this case we can
do better than a simple average. The error will be minimized if we use a weighted average. To find
the proper weight we take advantage of the fact that for a quadratic function the errors EM
n
and
ET
n
are exactly related by:
[EM
n
[ =
1
2
[ET
n
[ .
79
80 LECTURE 22. INTEGRATION: MIDPOINT AND SIMPSON’S RULES
E
T
u
¨
¨
¨
¨
¨
¨
¨
¨
¨
¨
¨
¨
¨
¨
¨
¨
¨
¨
¨
Figure 22.1: Comparing the trapezoid and midpoint method on a single subinterval. The
function is concave up, in which case T
n
is too high, while M
n
is too low.
Thus we take the following weighted average of the two, which is called Simpson’s rule:
S
2n
=
2
3
M
n
+
1
3
T
n
.
If we use this weighting on a quadratic function the two errors will exactly cancel.
Notice that we write the subscript as 2n. That is because we usually think of 2n subintervals in the
approximation; the n subintervals of the trapezoid are further subdivided by the midpoints. We can
then number all the points using integers. If we number them this way we notice that the number
of subintervals must be an even number.
The formula for Simpson’s rule if the subintervals are evenly spaced is the following (with n intervals,
where n is even):
S
n
=
b −a
3n
(f(x
0
) + 4f(x
1
) + 2f(x
2
) + 4f(x
3
) +. . . + 2f(x
n−2
) + 4f(x
n−1
) +f(x
n
)) .
Note that if we are presented with data ¦x
i
, y
i
¦ where the x
i
points are evenly spaced with x
i+1
−x
i
=
∆x, it is easy to apply Simpson’s method:
S
n
=
∆x
3
(y
0
+ 4y
1
+ 2y
2
+ 4y
3
+. . . + 2y
n−2
+ 4y
n−1
+y
n
) . (22.2)
Notice the pattern of the coefficients. The following program will produce these coefficients for n
intervals, if n is an even number. Try it for n = 6, 7, 100.
81
function w = mysimpweights(n)
% computes the weights for Simpson’s rule
% Input: n -- the number of intervals, must be even
% Output: a vector with the weights, length n+1
if rem(n,2) ~= 0
error(’n must be even’)
end
w = ones(n+1,1);
for i = 2:n
if rem(i,2)==0
w(i)=4;
else
w(i)=2;
end
end
Simpson’s rule is incredibly accurate. We will consider just how accurate in the next section. The
one drawback is that the points used must either be evenly spaced, or at least the odd number
points must lie exactly at the midpoint between the even numbered points. In applications where
you can choose the spacing, this is not a problem. In applications such as experimental or simulated
data, you might not have control over the spacing and then you cannot use Simpson’s rule.
Error bounds
The trapezoid, midpoint, and Simpson’s rules are all approximations. As with any approximation,
before you can safely use it, you must know how good (or bad) the approximation might be. For
these methods there are formulas that give upper bounds on the error. In other words, the worst
case errors for the methods. These bounds are given by the following statements:
• Suppose f
′′
is continuous on [a,b]. Let K
2
= max
x∈[a,b]
[f
′′
(x)[. Then the errors ET
n
and
EM
n
of the Trapezoid and Midpoint rules, respectively, applied to
_
b
a
f dx satisfy:
[ET
n
[ ≤
K
2
(b −a)
3
12n
2
=
K
2
(b −a)
12
∆x
2
and
[EM
n
[ ≤
K
2
(b −a)
3
24n
2
=
K
2
(b −a)
24
∆x
2
.
• Suppose f
(4)
is continuous on [a,b]. Let K
4
= max
x∈[a,b]
[f
(4)
(x)[. Then the error ES
n
of
Simpson’s rule applied to
_
b
a
f dx satisfies:
[ES
n
[ ≤
K
4
(b −a)
5
180n
4
=
K
4
(b −a)
180
∆x
4
.
In practice K
2
and K
4
are themselves approximated from the values of f at the evaluation points.
The most important thing in these error bounds is the last term. For the trapezoid and midpoint
method, the error depends on ∆x
2
whereas the error for Simpson’s rule depends on ∆x
4
. If ∆x is
just moderately small, then there is a huge advantage with Simpson’s method.
When an error depends on a power of a parameter, such as above, we sometimes use the order
notation. In the above error bounds we say that the trapezoid and midpoint rules have errors of
order O(∆x
2
), whereas Simpson’s rule has error of order O(∆x
4
).
82 LECTURE 22. INTEGRATION: MIDPOINT AND SIMPSON’S RULES
In Matlab there is a built-in command for definite integrals: quad(f,a,b) where the f is an inline
function and a and b are the endpoints. Here quad stands for quadrature, which is a term for
numerical integration. The command uses “adaptive Simpson quadrature”, a form of Simpson’s
rule that checks its own accuracy and adjusts the grid size where needed. Here is an example of its
usage:
> f = inline(’x.^(1/3).*sin(x.^3)’)
> I = quad(f,0,1)
Exercises
22.1 Using formulas (22.1) and (22.2), for the integral
_
2
1

xdx calculate M
4
and S
4
(by hand, but
use a calculator). Find the percentage error of these approximations, using the exact value.
Compare with exercise 21.1.
22.2 Write a function program mymidpoint that calculates the midpoint rule approximation for
_
f on the interval [a, b] with n subintervals. The inputs should be f, a, b and n. Use your
program on the integral
_
2
1

xdx to obtain M
4
and M
100
. Compare these with the previous
exercise and the true value of the integral.
22.3 Write a function program mysimpson that calculates the Simpson’s rule approximation for
_
f on the interval [a, b] with n subintervals. It should call the program mysimpweights to
produce the coefficients. Use your program on the integral
_
2
1

xdx to obtain S
4
and S
100
.
Compare these with the previous exercise and the true value.
Lecture 23
Plotting Functions of Two Variables
Functions on Rectangular Grids
Suppose you wish to plot a function f(x, y) on the rectangle a ≤ x ≤ b and c ≤ y ≤ d. The graph
of a function of two variables is of course a three dimensional object. Visualizing the graph is often
very useful.
For example, suppose you have a formula:
f(x, y) = xsin(xy)
and you are interested in the function on the region 0 ≤ x ≤ 5, π ≤ y ≤ 2π. A way to plot this
function in Matlab would be the following sequence of commands:
> f = inline(’x.*sin(x.*y)’,’x’,’y’)
> [X,Y] = meshgrid(0:.1:5,pi:.01*pi:2*pi);
> Z = f(X,Y)
> mesh(X,Y,Z)
This will produce a 3-D plot that you can rotate by clicking on the rotate icon and then dragging
with the mouse.
Instead of the command mesh, you could use the command:
> surf(X,Y,Z)
The key command in this sequence is [X Y] = meshgrid(a:h:b,c:k:d), which produces matrices
of x and y values in X and Y. Enter:
> size(X)
> size(Y)
> size(Z)
to see that each of these variables is a 101 51 matrix. To see the first few entries of X enter:
> X(1:6,1:6)
and to see the first few values of Y type:
> Y(1:6,1:6)
You should observe that the x values in X begin at 0 on the left column and increase from left to
right. The y values on the other have start at π at the top and increase from top to bottom. Note
that this arrangement is flipped from the usual arrangment in the x-y plane.
In the command [X Y] = meshgrid(a:h:b,c:k:d), h is the increment in the x direction and k is
the increment in the y direction. Often we will calculate:
h =
b −a
m
and k =
d −c
n
,
where m is the number of intervals in the x direction and n is the number of intervals in the y
direction. To obtain a good plot it is best if m and n can be set between 10 and 100.
83
84 LECTURE 23. PLOTTING FUNCTIONS OF TWO VARIABLES
For another example of how meshgrid works, try the following and look at the output in X and
Y.
> [X,Y] = meshgrid(0:.5:4,1:.2:2);
Scattered Data and Triangulation
Often we are interested in objects whose bases are not rectangular. For instance, data does not
usually come arranged in a nice rectangular grid; rather, measurements are taken where convenient.
In Matlab we can produce triangles for a region by recording the coordinates of the vertices and
recording which vertices belong to each triangle. The following script program produces such a set
of triangles:
% mytriangles
% Program to produce a triangulation.
% V contains vertices, which are (x,y) pairs
V = [ 1/2 1/2 ; 1 1 ; 3/2 1/2 ; .5 1.5 ; 0 0
1 0 ; 2 0 ; 2 1 ; 1.5 1.5 ; 1 2
0 2 ; 0 1]
% x, y are row vectors containing coordinates of vertices
x = V(:,1)’;
y = V(:,2)’;
% Assign the triangles
T = delaunay(x,y)
You can plot the triangles using the following command:
> trimesh(T,x,y)
You can also prescribe values (heights) at each vertex directly (say from a survey):
> z1 = [ 2 3 2.5 2 1 1 .5 1.5 1.6 1.7 .9 .5 ];
or using a function:
> f = inline(’abs(sin(x.*y)).^(3/2)’,’x’,’y’);
> z2 = f(x,y);
The resulting profiles can be plotted:
> trimesh(T,x,y,z1)
> trisurf(T,x,y,z2)
Each row of the matrix T corresponds to a triangle, so T(i,:) gives triangle number i. The three
corners of triangle number i are at indices T(i,1), T(i,2), and T(i,3). So for example to get the
y-coordinate of the second point of triangle number 5, enter:
> y(T(5,2))
To see other examples of regions defined by triangle get mywedge.m and mywasher.m from the class
web site and run them. Each of these programs defines vectors x and y of x and y values of vertices
and a matrix T. As before T is a list of sets of three integers. Each triple of integers indicates which
vertices to connect in a triangle.
To plot a function, say f(x, y) = x
2
−y
2
on the washer figure try:
> mywasher
> z = x.^2 - y.^2
> trisurf(T,x,y,z)
Note again that this plot can be rotated using the icon and mouse.
85
Exercises
23.1 Plot the function f(x, y) = xe
−x
2
−y
2
on the rectangle −3 ≤ x ≤ 3, −2 ≤ y ≤ 2 using
meshgrid. Make an appropriate choice of h and k and if necessary a rotation to produce a
good plot. Turn in your plot and the calculation of k and h.
23.2 Make up an interesting or useful convex irregular region and choose well-distributed points in
it and on the boundary as a basis for a triangulation. Make sure to include a good number of
both interior and boundary points. (Graph paper would be handy.) Record the coordinates of
these points in the matrix V in a copy of the program mytriangles.m. Plot the triangulation.
Also create values at the vertices using a function and plot the resulting profile.
Lecture 24
Double Integrals for Rectangles
The center point method
Suppose that we need to find the integral of a function, f(x, y), on a rectangle:
R = ¦(x, y) : a ≤ x ≤ b, c ≤ y ≤ d¦.
In calculus you learned to do this by an iterated integral:
I =
__
R
f dA =
_
b
a
_
d
c
f(x, y) dydx =
_
d
c
_
b
a
f(x, y) dxdy.
You also should have learned that the integral is the limit of the Riemann sums of the function as
the size of the subrectangles goes to zero. This means that the Riemann sums are approximations
of the integral, which is precisely what we need for numerical methods.
For a rectangle R, we begin by subdividing into smaller subrectangles ¦R
ij
¦, in a systematic way. We
will divide [a, b] into m subintervals and [c, d] into n subintervals. Then R
ij
will be the “intersection”
of the i-th subinterval in [a, b] with the j-th subinterval of [c, d]. In this way the entire rectangle is
subdivided into mn subrectangles, numbered as in Figure 24.1.
A Riemann sum using this subdivision would have the form:
S =
m,n

i,j=1,1
f(x

ij
)A
ij
=
n

j=1
_
m

i=1
f(x

ij
)A
ij
_
,
a
b
c
d
R
11
R
21
R
31
R
41
R
12
R
13
R
14
R
15
Figure 24.1: Subdivision of the rectangle R = [a, b] [c, d] into subrectangles R
ij
.
86
87
where A
ij
= ∆x
i
∆y
j
is the area of R
ij
, and x

ij
is a point in R
ij
. The theory of integrals tells us
that if f is continuous, then this sum will converge to the same number, no matter how we choose
x

ij
. For instance, we could choose x

ij
to be the point in the lower left corner of R
ij
and the sum
would still converge as the size of the subrectangles goes to zero. However, in practice we wish to
choose x

ij
in such a way to make S as accurate as possible even when the subrectangles are not very
small. The obvious choice for the best point in R
ij
would be the center point. The center point is
most likely of all points to have a value of f close to the average value of f. If we denote the center
points by c
ij
, then the sum becomes
C
mn
=
m,n

i,j=1,1
f(c
ij
)A
ij
.
Here
c
ij
=
_
x
i−1
+x
i
2
,
y
i−1
+y
i
2
_
.
Note that if the subdivision is evenly spaced then ∆x ≡ (b − a)/m and ∆y ≡ (d − c)/n, and so in
that case
C
mn
=
(b −a)(d −c)
mn
m,n

i,j=1,1
f(c
ij
).
The four corners method
Another good idea would be to take the value of f not only at one point, but as the average of the
values at several points. An obvious choice would be to evaluate f at all four corners of each R
ij
then average those. If we note that the lower left corner is (x
i
, y
j
), the upper left is (x
i
, y
j+1
), the
lower right is (x
i+1
, y
i
) and the upper right is (x
i+1
, y
i+1
), then the corresponding sum will be
F
mn
=
m,n

i,j=1,1
1
4
(f(x
i
, y
j
) +f(x
i
, y
j+1
) +f(x
i+1
, y
i
) +f(x
i+1
, y
j+1
)) A
ij
,
which we will call the four-corners method. If the subrectangles are evenly spaced, then we can
simplify this expression. Notice that f(x
i
, y
j
) gets counted multiple times depending on where
(x
i
, y
j
) is located. For instance if (x
i
, y
j
) is in the interior of R then it is the corner of 4 subrectangles.
Thus the sum becomes
F
mn
=
A
4
_
_

corners
f(x
i
, y
j
) + 2

edges
f(x
i
, y
j
) + 4

interior
f(x
i
, y
j
)
_
_
,
where A = ∆x∆y is the area of the subrectangles. We can think of this as a weighted average of
the values of f at the grid points (x
i
, y
j
). The weights used are represented in the matrix
W =
_
_
_
_
_
_
_
_
_
1 2 2 2 2 2 1
2 4 4 4 4 4 2
2 4 4 4 4 4 2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2 4 4 4 4 4 2
1 2 2 2 2 2 1
_
_
_
_
_
_
_
_
_
. (24.1)
We could implement the four-corner method by forming a matrix (f
ij
) of f values at the grid points,
then doing entry-wise multiplication of the matrix with the weight matrix. Then the integral would
88 LECTURE 24. DOUBLE INTEGRALS FOR RECTANGLES
be obtained by summing all the entries of the resulting matrix and multiplying that by A/4. The
formula would be
F
mn
=
(b −a)(d −c)
4mn
m+1,n+1

i,j=1,1
W
ij
f(x
i
, y
j
).
Notice that the four-corners method coincides with applying the trapezoid rule in each direction.
Thus it is in fact a double trapezoid rule.
The double Simpson method
The next improvement one might make would be to take an average of the center point sum C
mn
and the four corners sum F
mn
. However, a more standard way to obtain a more accurate method
is the Simpson double integral. It is obtained by applying Simpson’s rule for single integrals to the
iterated double integral. The resulting method requires that both m and n be even numbers and
the grid be evenly spaced. If this is the case we sum up the values f(x
i
, y
j
) with weights represented
in the matrix
W =
_
_
_
_
_
_
_
_
_
_
_
_
_
1 4 2 4 2 4 1
4 16 8 16 8 16 4
2 8 4 8 4 8 2
4 16 8 16 8 16 4
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2 8 4 8 4 8 2
4 16 8 16 8 16 4
1 4 2 4 2 4 1
_
_
_
_
_
_
_
_
_
_
_
_
_
. (24.2)
The sum of the weighted values is multiplied by A/9 and the formula is
S
mn
=
(b −a)(d −c)
9mn
m+1,n+1

i,j=1,1
W
ij
f(x
i
, y
j
).
Matlab has a built in command for double integrals on rectangles: dblquad(f,a,b,c,d). Here is
an example:
> f = inline(’sin(x.*y)./sqrt(x+y)’,’x’,’y’)
> I = dblquad(f,0.5,1,0.5,2)
Below is a Matlab function which will produce the matrix of weights needed for Simpson’s rule for
double integrals. It uses the function mysimpweights from Lecture 22.
function W = mydblsimpweights(m,n)
% Produces the m by n matrix of weights for Simpson’s rule
% for double integrals
% Inputs: m -- number of intervals in the row direction.
% must be even.
% n -- number of intervals in the column direction.
% must be even.
% Output: W -- a (m+1)x(n+1) matrix of the weights
if rem(m,2)~=0 | rem(n,2)~=0
error(’m and n must be even’)
end
u = mysimpweights(m);
v = mysimpweights(n);
W = u*v’;
89
Exercises
24.1 Download the program mylowerleft from the web site. Modify it to make a program
mycenter that does the center point method. Implement the change by changing the “mesh-
grid” to put the grid points at the centers of the subrectangles. Look at the mesh plot pro-
duced to make sure the program is putting the grid where you want it. Use both programs
to approximate the integral
_
2
0
_
5
1

xy dy dx,
using (m, n) = (10, 18). Evaluate this integral exactly to make comparisons.
24.2 Write a program mydblsimp that computes the Simpson’s rule approximation. Let it call the
program mydblsimpweights to make the weight matrix (24.2). Check the accuracy of the
program on the integral in the previous problem.
24.3 Using mysimpweights and mydblsimpweights as models make programs mytrapweights and
mydbltrapweights that will produce the weights for the trapezoid rule and the weight matrix
for the four corners (double trapezoid) method (24.1). Turn in the programs and the resulting
weights for a 5 6 grid.
Lecture 25
Double Integrals for Non-rectangles
In the previous lecture we considered only integrals over rectangular regions. In practice, regions of
interest are rarely rectangles and so in this lecture we consider two strategies for evaluating integrals
over other regions.
Redefining the function
One strategy is to redefine the function so that it is zero outside the region of interest, then integrate
over a rectangle that includes the region.
For example, suppose we need to approximate the value of
I =
__
T
sin
3
(xy) dxdy
where T is the triangle with corners at (0, 0), (1, 0) and (0, 2). Then we could let R be the rectangle
[0, 1] [0, 2] which contains the triangle T. Notice that the hypotenuse of the triangle has the
equation 2x + y = 2. Then make f(x) = sin
3
(xy) if 2x + y ≤ 2 and f(x) = 0 if 2x + y > 2. In
Matlab we can make this function with the command:
> f = inline(’sin(x.*y).^3.*(2*x + y <= 2)’)
In this command <= is a logical command. The term in parentheses is then a logical statement and
is given the value 1 if the statement is true and 0 if it is false. We can then integrate the modified
f on [0, 1] [0, 2] using the command:
> I = dblquad(f,0,1,0,2)
As another example, suppose we need to integrate x
2
exp(xy) inside the circle of radius 2 centered
at (1, 2). The equation for this circle is (x −1)
2
+(y −2)
2
= 4. Note that the inside of the circle is
(x −1)
2
+(y −2)
2
≤ 4 and that the circle is contained in the rectangle [−1, 3] [0, 4]. Thus we can
create the right function and integrate it by:
> f = inline(’x.^2.*exp(x.*y).*((x-1).^2 + (y-2).^2 <= 4)’)
> I = dblquad(f,-1,3,0,4)
90
91
Integration Based on Triangles
The second approach to integrating over non-rectangular regions, is based on subdividing the region
into triangles. Such a subdivision is called a triangulation. On regions where the boundary consists
of line segments, this can be done exactly. Even on regions where the boundary contains curves, this
can be done approximately. This is a very important idea for several reasons, the most important
of which is that the finite elements method is based on it. Another reason this is important is that
often the values of f are not given by a formula, but from data. For example, suppose you are
surveying on a construction site and you want to know how much fill will be needed to bring the
level up to the plan. You would proceed by taking elevations at numerous points across the site.
However, if the site is irregularly shaped or if there are obstacles on the site, then you cannot make
these measurements on an exact rectangular grid. In this case, you can use triangles by connecting
your points with triangles. Many software packages will even choose the triangles for you (Matlab
will do it using the command delaunay).
The basic idea of integrals based on triangles is exactly the same as that for rectangles; the integral
is approximated by a sum where each term is a value times an area
I ≈
n

i=1
f(x

i
)A
i
,
where n is the number of triangles, A
i
is the area of the triangle and x

a point in the triangle.
However, rather than considering the value of f at just one point people often consider an average
of values at several points. The most convenient of these is of course the corner points. We can
represent this sum by
T
n
=
n

i=1
¯
f
i
A
i
,
where
¯
f is the average of f at the corners.
If the triangle has vertices (x
1
, y
1
), (x
2
, y
2
) and (x
3
, y
3
), the formula for area is
A =
1
2
¸
¸
¸
¸
¸
¸
det
_
_
x
1
x
2
x
3
y
1
y
2
y
3
1 1 1
_
_
¸
¸
¸
¸
¸
¸
. (25.1)
A function mythreecorners to compute using the three corners method is given below.
Another idea would be to use the center point (centroid) of each triangle. If the triangle has vertices
(x
1
, y
1
), (x
2
, y
2
) and (x
3
, y
3
), then the centroid is given by the simple formulas
¯ x =
x
1
+x
2
+x
3
3
and ¯ y =
y
1
+y
2
+y
3
3
. (25.2)
92 LECTURE 25. DOUBLE INTEGRALS FOR NON-RECTANGLES
function I = mythreecorners(f,V,T)
% Integrates a function based on a triangulation, using three corners
% Inputs: f -- the function to integrate, as an inline
% V -- the vertices. Each row has the x and y coordinates of a vertex
% T -- the triangulation. Each row gives the indices of three corners
% Output: the approximate integral
x = V(:,1); % extract x and y coordinates of all nodes
y = V(:,2);
I=0;
p = size(T,1);
for i = 1:p
x1 = x(T(i,1)); % find coordinates and area
x2 = x(T(i,2));
x3 = x(T(i,3));
y1 = y(T(i,1));
y2 = y(T(i,2));
y3 = y(T(i,3));
A = .5*abs(det([x1, x2, x3; y1, y2, y3; 1, 1, 1]));
z1 = f(x1,y1); % find values and average
z2 = f(x2,y2);
z3 = f(x3,y3);
zavg = (z1 + z2 + z3)/3;
I = I + zavg*A; % accumulate integral
end
Exercises
25.1 Download the programs mywedge.m and mywasher.m from the web site. Plot f(x, y) =
x+y
x
2
+y
2
on the region produced by mywasher.m and f(x, y) = sin(x) +

y on the region produced by
mywedge.m.
25.2 Modify the program mythreecorners.m to a new program mycenters.m that does the cen-
terpoint method for triangles. Run the program on the region produced by mywasher.m with
the function f(x, y) =
x+y
x
2
+y
2
and on the region produced by mywedge.m with the function
f(x, y) = sin(x) +

y.
Lecture 26
Gaussian Quadrature*
Exercises
26.1
26.2
93
Lecture 27
Numerical Differentiation
Approximating derivatives from data
Suppose that a variable y depends on another variable x, i.e. y = f(x), but we only know the values
of f at a finite set of points, e.g., as data from an experiment or a simulation:
(x
1
, y
1
), (x
2
, y
2
), . . . , (x
n
, y
n
).
Suppose then that we need information about the derivative of f(x). One obvious idea would be to
approximate f

(x
i
) by the Forward Difference:
f

(x
i
) = y

i

y
i+1
−y
i
x
i+1
−x
i
.
This formula follows directly from the definition of the derivative in calculus. An alternative would
be to use a Backward Difference:
f

(x
i
) ≈
y
i
−y
i−1
x
i
−x
i−1
.
Since the errors for the forward difference and backward difference tend to have opposite signs, it
would seem likely that averaging the two methods would give a better result than either alone. If the
points are evenly spaced, i.e. x
i+1
−x
i
= x
i
−x
i−1
= h, then averaging the forward and backward
differences leads to a symmetric expression called the Central Difference:
f

(x
i
) = y

i

y
i+1
−y
i−1
2h
.
Errors of approximation
We can use Taylor polynomials to derive the accuracy of the forward, backward and central difference
formulas. For example the usual form of the Taylor polynomial with remainder (sometimes called
Taylor’s Theorem) is:
f(x +h) = f(x) +hf

(x) +
h
2
2
f
′′
(c),
where c is some (unknown) number between x and x +h. Letting x = x
i
, x +h = x
i+1
and solving
for f

(x
i
) leads to:
f

(x
i
) =
f(x
i+1
) −f(x
i
)
h

h
2
f
′′
(c).
Notice that the quotient in this equation is exactly the forward difference formula. Thus the error of
the forward difference is −(h/2)f
′′
(c) which means it is O(h). Replacing h in the above calculation
94
95
−0.2 0 0.2 0.4 0.6 0.8 1 1.2
−0.2
0
0.2
0.4
0.6
0.8
1
x
y
backward difference
forward difference
central difference
(x
i
,y
i
)
(x
i −1
,y
i −1
)
(x
i+1
,y
i+1
)
Figure 27.1: The three difference approximations of y

i
.
by −h gives the error for the backward difference formula; it is also O(h). For the central difference,
the error can be found from the third degree Taylor polynomial with remainder:
f(x
i+1
) = f(x
i
+h) = f(x
i
) +hf

(x
i
) +
h
2
2
f
′′
(x
i
) +
h
3
3!
f
′′′
(c
1
) and
f(x
i−1
) = f(x
i
−h) = f(x
i
) −hf

(x
i
) +
h
2
2
f
′′
(x
i
) −
h
3
3!
f
′′′
(c
2
)
where x
i
≤ c
1
≤ x
i+1
and x
i−1
≤ c
2
≤ x
i
. Subtracting these two equations and solving for f

(x
i
)
leads to:
f

(x
i
) =
f(x
i+1
) −f(x
i−1
)
2h

h
2
3!
f
′′′
(c
1
) +f
′′′
(c
2
)
2
.
This shows that the error for the central difference formula is O(h
2
). Thus, central differences are
significantly better and so: It is best to use central differences whenever possible.
There are also central difference formulas for higher order derivatives. These all have error of order
O(h
2
):
f
′′
(x
i
) = y
′′
i

y
i+1
−2y
i
+y
i−1
h
2
,
f
′′′
(x
i
) = y
′′′
i

1
2h
3
[y
i+2
−2y
i+1
+ 2y
i−1
−y
i−2
] , and
f
(4)
(x
i
) = y
(4)
i

1
h
4
[y
i+2
−4y
i+1
+ 6y
i
−4y
i−1
+y
i−2
] .
Partial Derivatives
Suppose u = u(x, y) is a function of two variables that we only know at grid points (x
i
, y
j
). We will
use the notation:
u
i,j
= u(x
i
, y
j
)
96 LECTURE 27. NUMERICAL DIFFERENTIATION
frequently throughout the rest of the lectures. We can suppose that the grid points are evenly
spaced, with an increment of h in the x direction and k in the y direction. The central difference
formulas for the partial derivatives would be:
u
x
(x
i
, y
j
) ≈
1
2h
(u
i+1,j
−u
i−1,j
) and
u
y
(x
i
, y
j
) ≈
1
2k
(u
i,j+1
−u
i,j−1
) .
The second partial derivatives are:
u
xx
(x
i
, y
j
) ≈
1
h
2
(u
i+1,j
−2u
i,j
+u
i−1,j
) and
u
yy
(x
i
, y
j
) ≈
1
k
2
(u
i,j+1
−2u
i,j
+u
i,j−1
) ,
and the mixed partial derivative is:
u
xy
(x
i
, y
j
) ≈
1
4hk
(u
i+1,j+1
−u
i+1,j−1
−u
i−1,j+1
+u
i−1,j−1
) .
Caution: Notice that we have indexed u
ij
so that as a matrix each row represents the values of
u at a certain x
i
and each column contains values at y
j
. The arrangement in the matrix does not
coincide with the usual orientation of the xy-plane.
Let’s consider an example. Let the values of u at (x
i
, y
j
) be recorded in the matrix:
(u
ij
) =
_
_
_
_
5.1 6.5 7.5 8.1 8.4
5.5 6.8 7.8 8.3 8.9
5.5 6.9 9.0 8.4 9.1
5.4 9.6 9.1 8.6 9.4
_
_
_
_
(27.1)
Assume the indices begin at 1, i is the index for rows and j the index for columns. Suppose that
h = .5 and k = .2. Then u
y
(x
2
, y
4
) would be approximated by the central difference:
u
y
(x
2
, y
4
) ≈
u
2,5
−u
2,3
2k

8.9 −7.8
2 0.2
= 2.75.
The partial derivative u
xy
(x
2
, y
4
) is approximated by:
u
xy
(x
2
, y
4
) ≈
u
3,5
−u
3,3
−u
1,5
+u
1,3
4hk

9.1 −9.0 −8.4 + 7.5
4 .5 .2
= −2.
Exercises
27.1 Suppose you are given the data in the following table.
t 0 .5 1.0 1.5 2.0
y 0 .19 .26 .29 .31
a. Give the forward, backward and central difference approximations of f

(1).
b. Give the central difference approximations for f
′′
(1), f
′′′
(1) and f
(4)
(1).
27.2 Suppose values of u(x, y) at points (x
i
, y
j
) are given in the matrix (27.1). Suppose that h = .1
and k = .5. Approximate the following derivatives by central differences:
a. u
x
(x
2
, y
4
)
b. u
xx
(x
3
, y
2
)
c. u
yy
(x
3
, y
2
)
d. u
xy
(x
2
, y
3
)
Lecture 28
The Main Sources of Error
Truncation Error
Truncation error is defined as the error caused directly by an approximation method. For instance,
all numerical integration methods are approximations and so there is error, even if the calculations
are performed exactly. Numerical differentiation also has a truncation error, as will the differential
equations methods we will study in Part IV, which are based on numerical differentiation formulas.
There are two ways to minimize truncation error: (1) use a higher order method, and (2) use a
finer grid so that points are closer together. Unless the grid is very small, truncation errors are
usually much larger than roundoff errors. The obvious tradeoff is that a smaller grid requires more
calculations, which in turn produces more roundoff errors and requires more running time.
Roundoff Error
Roundoff error always occurs when a finite number of digits are recorded after an operation. For-
tunately, this error is extremely small. The standard measure of how small is called machine
epsilon. It is defined as the smallest number that can be added to 1 to produce another number
on the machine, i.e. if a smaller number is added the result will be rounded down to 1. In IEEE
standard double precision (used by Matlab and most serious software), machine epsilon is 2
−52
or about 2.2 10
−16
. A different, but equivalent, way of thinking about this is that the machine
records 52 floating binary digits or about 15 floating decimal digits. Thus there are never more than
15 significant digits in any calculation. This of course is more than adequate for any application.
However, there are ways in which this very small error can cause problems.
To see an unexpected occurence of round-off try the following commands:
> (2^52+1) - 2^52
> (2^53+1) - 2^53
Loss of Precision (also called Loss of Significance)
One way in which small roundoff errors can become more significant is when significant digits cancel.
For instance, if you were to calculate:
1234567(1 −.9999987654321)
then the result should be
1.52415678859930.
97
98 LECTURE 28. THE MAIN SOURCES OF ERROR
But, if you input:
> 1234567 * ( 1 - .9999987654321)
you will get:
> 1.52415678862573
In the correct calculation there are 15 significant digits, but the Matlab calculation has only 11
significant digits.
Although this seems like a silly example, this type of loss of precision can happen by accident if you
are not careful. For example in f

(x) ≈ (f(x +h) −f(x))/h you will lose precision when h gets too
small. Try:
> format long
> f=inline(’x^2’,’x’)
> for i = 1:30
> h=10^(-i)
> df=(f(1+h)-f(1))/h
> relerr=(2-df)/2
> end
At first the relative error decreases since truncation error is reduced. Then loss of precision takes
over and the relative error increases to 1.
Bad Conditioning
We encountered bad conditioning in Part II, when we talked about solving linear systems. Bad
conditioning means that the problem is unstable in the sense that small input errors can produce
large output errors. This can be a problem in a couple of ways. First, the measurements used to get
the inputs cannot be completely accurate. Second, the computations along the way have roundoff
errors. Errors in the computations near the beginning especially can be magnified by a factor close
to the condition number of the matrix. Thus what was a very small problem with roundoff can
become a very big problem.
It turns out that matrix equations are not the only place where condition numbers occur. In any
problem one can define the condition number as the maximum ratio of the relative errors in the
output versus input, i.e.
condition # of a problem = max
_
Relative error of output
Relative error of inputs
_
.
An easy example is solving a simple equation:
f(x) = 0.
Suppose that f

is close to zero at the solution x

. Then a very small change in f (caused perhaps
by an inaccurate measurement of some of the coefficients in f) can cause a large change in x

. It
can be shown that the condition number of this problem is 1/f

(x

).
99
Exercises
28.1 Identify the (main) source of error in each case and propose a way to reduce this error if
possible.
(a) If we do (

3)
2
we should get 3, but if we check
> mythree=(sqrt(3))^2
> mythree-3
we find the error is ans=-4.4409e-16.
(b) Since it is a quarter of a circle of radius 2, we should have
_
2
0

4 −x
2
dx =
1
4
π2
2
= π.
We try to use mytrap from Lecture 21 and do
> x=0:.2:2;
> y=sqrt(4-x.^2);
> mypi=mytrap(x,y)
> mypi-pi
and find the error is ans=-0.0371.
28.2
1
The function f(x) = (x − 2)
9
could be evaluated as written, or first expanded as f(x) =
x
9
−18x
8
+ and then evaluated. To find the expanded version, type
> syms x
> expand((x-2)^9)
> clear
To evaluate it the first way, type
> f1=inline(’(x-2).^9’,’x’)
> x=1.92:.001:2.08;
> y1=f1(x);
> plot(x,y1,’blue’)
To do it the second way, convert the expansion above to an inline function f2 and then type
> y2=f2(x);
> hold on
> plot(x,y2,’red’)
Carefully study the resulting graphs. Should the graphs be the same? Which is more correct?
Matlab does calculations using approximately 16 decimal places. What is the largest error in
the graph, and how big is it relative to 10
−16
? Which source of error is causing this problem?
1
From Numerical Linear Algebra by L. Trefethen and D. Baum, SIAM, 1997.
Review of Part III
Methods and Formulas
Polynomial Interpolation:
An exact fit to the data.
For n data points it is a n −1 degree polynomial.
Only good for very few, accurate data points.
The coefficients are found by solving a linear system of equations.
Spline Interpolation:
Fit a simple function between each pair of points.
Joining points by line segments is the most simple spline.
Cubic is by far the most common and important.
Cubic matches derivatives and second derivatives at data points.
Simply supported and clamped ends are available.
Good for more, but accurate points.
The coefficients are found by solving a linear system of equations.
Least Squares:
Makes a “close fit” of a simple function to all the data.
Minimizes the sum of the squares of the errors.
Good for noisy data.
The coefficients are found by solving a linear system of equations.
Interpolation vs. Extrapolation: Polynomials, Splines and Least Squares are generally used for
Interpolation, fitting between the data. Extrapolation, i.e. making fits beyond the data, is much
more tricky. To make predictions beyond the data, you must have knowledge of the underlying
process, i.e. what the function should be.
Numerical Integration:
Left Endpoint:
L
n
=
n

i=1
f(x
i−1
)∆x
i
Right Endpoint:
R
n
=
n

i=1
f(x
i
)∆x
i
.
Trapezoid Rule:
T
n
=
n

i=1
f(x
i−1
) +f(x
i
)
2
∆x
i
.
Midpoint Rule:
M
n
=
n

i=1
f(¯ x
i
)∆x
i
where ¯ x
i
=
x
i−1
+x
i
2
.
100
101
Numerical Integration Rules with Even Spacing:
For even spacing: ∆x =
b−a
n
where n is the number of subintervals, then:
L
n
= ∆x
n−1

i=0
y
i
=
b −a
n
n−1

i=0
f(x
i
),
R
n
= ∆x
n

i=1
y
i
=
b −a
n
n

i=1
f(x
i
),
T
n
= ∆x
_
y
0
+ 2y
1
+. . . + 2y
n−1
+y
n
_
=
b −a
2n
_
f(x
0
) + 2f(x
1
) +. . . + 2f(x
n−1
) +f(x
n
)
_
,
M
n
= ∆x
n

i=1
¯ y
i
=
b −a
n
n

i=1
f(¯ x
i
),
Simpson’s rule:
S
n
= ∆x
_
y
0
+ 4y
1
+ 2y
2
+ 4y
3
+. . . + 2y
n−2
+ 4y
n−1
+y
n
_
=
b −a
3n
(f(x
0
) + 4f(x
1
) + 2f(x
2
) + 4f(x
3
) +. . . + 2f(x
n−2
) + 4f(x
n−1
) +f(x
n
)) .
Area of a region:
A = −
_
C
y dx,
where C is the counter-clockwise curve around the boundary of the region. We can represent such a
curve, by consecutive points on it, i.e. ¯ x = (x
0
, x
1
, x
2
, . . . , x
n−1
, x
n
), and ¯ y = (y
0
, y
1
, y
2
, . . . , y
n−1
, y
n
)
with (x
n
, y
n
) = (x
0
, y
0
). Applying the trapezoid method to the integral (21.4):
A = −
n

i=1
y
i−1
+y
i
2
(x
i
−x
i−1
)
Accuracy of integration rules:
Right and Left endpoint are O(∆x)
Trapezoid and Midpoint are O(∆x
2
)
Simpson is O(∆x
4
)
Double Integrals on Rectangles:
Centerpoint:
C
mn
=
m,n

i,j=1,1
f(c
ij
)A
ij
.
where
c
ij
=
_
x
i−1
+x
i
2
,
y
i−1
+y
i
2
_
.
Centerpoint – Evenly spaced:
C
mn
= ∆x∆y
m,n

i,j=1,1
z
ij
=
(b −a)(d −c)
mn
m,n

i,j=1,1
f(c
ij
).
Four corners:
F
mn
=
m,n

i,j=1,1
1
4
(f(x
i
, y
j
) +f(x
i
, y
j+1
) +f(x
i+1
, y
i
) +f(x
i+1
, y
j+1
)) A
ij
,
102 REVIEW OF PART III
Four Corners – Evenly spaced:
F
mn
=
A
4
_
_

corners
f(x
i
, y
j
) + 2

edges
f(x
i
, y
j
) + 4

interior
f(x
i
, y
j
)
_
_
=
(b −a)(d −c)
4mn
m,n

i,j=1,1
W
ij
f(x
i
, y
j
).
where
W =
_
_
_
_
_
_
_
_
_
1 2 2 2 2 2 1
2 4 4 4 4 4 2
2 4 4 4 4 4 2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2 4 4 4 4 4 2
1 2 2 2 2 2 1
_
_
_
_
_
_
_
_
_
.
Double Simpson:
S
mn
=
(b −a)(d −c)
9mn
m,n

i,j=1,1
W
ij
f(x
i
, y
j
).
where
W =
_
_
_
_
_
_
_
_
_
_
_
_
_
1 4 2 4 2 4 1
4 16 8 16 8 16 4
2 8 4 8 4 8 2
4 16 8 16 8 16 4
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2 8 4 8 4 8 2
4 16 8 16 8 16 4
1 4 2 4 2 4 1
_
_
_
_
_
_
_
_
_
_
_
_
_
.
Integration based on triangles:
– Triangulation: Dividing a region up into triangles.
– Triangles are suitable for odd-shaped regions.
– A triangulation is better if the triangles are nearly equilateral.
Three corners:
T
n
=
n

i=1
¯
f
i
A
i
where
¯
f is the average of f at the corners of the i-th triangle.
Area of a triangle with corners (x
1
, y
1
), (x
2
, y
2
), (x
3
, y
3
):
A =
1
2
¸
¸
¸
¸
¸
¸
det
_
_
x
1
x
2
x
3
y
1
y
2
y
3
1 1 1
_
_
¸
¸
¸
¸
¸
¸
.
Centerpoint:
C =
n

i=1
f(¯ x
i
, ¯ y
i
)A
i
, with
¯ x =
x
1
+x
2
+x
3
3
and ¯ y =
y
1
+y
2
+y
3
3
.
103
Forward Difference:
f

(x
i
) = y

i

y
i+1
−y
i
x
i+1
−x
i
.
Backward Difference:
f

(x
i
) ≈
y
i
−y
i−1
x
i
−x
i−1
.
Central Difference:
f

(x
i
) = y

i

y
i+1
−y
i−1
2h
.
Higher order central differences:
f
′′
(x
i
) = y
′′
i

y
i+1
−2y
i
+y
i−1
h
2
,
f
′′′
(x
i
) = y
′′′
i

1
2h
3
[y
i+2
−2y
i+1
+ 2y
i−1
−y
i−2
] ,
f
(4)
(x
i
) = y
(4)
i

1
h
4
[y
i+2
−4y
i+1
+ 6y
i
−4y
i−1
+y
i−2
] .
Partial Derivatives: Denote u
i,j
= u(x
i
, y
j
).
u
x
(x
i
, y
j
) ≈
1
2h
(u
i+1,j
−u
i−1,j
) ,
u
y
(x
i
, y
j
) ≈
1
2k
(u
i,j+1
−u
i,j−1
) ,
u
xx
(x
i
, y
j
) ≈
1
h
2
(u
i+1,j
−2u
i,j
+u
i−1,j
) ,
u
yy
(x
i
, y
j
) ≈
1
k
2
(u
i,j+1
−2u
i,j
+u
i,j−1
) ,
u
xy
(x
i
, y
j
) ≈
1
4hk
(u
i+1,j+1
−u
i+1,j−1
−u
i−1,j+1
+u
i−1,j−1
) .
Sources of error:
Truncation – the method is an approximation.
Roundoff – double precision arithmetic uses ≈ 15 significant digits.
Loss of precision – an amplification of roundoff error due to cancellations.
Bad conditioning – the problem is sensitive to input errors.
Error can build up after multiple calculations.
Matlab
Data Interpolation:
Use the plot command plot(x,y,’*’) to plot the data.
Use the Basic Fitting tool to make an interpolation or spline.
If you choose an n−1 degree polynomial with n data points the result will be the exact polynomial
interpolation.
If you select a polynomial degree less than n−1, then Matlab will produce a least squares approx-
imation.
Integration
> quadl(f,a,b) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Numerical integral of f(x) on [a, b].
104 REVIEW OF PART III
> dblquad(f,a,b,c,d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Integral of f(x, y) on [a, b] [c, d].
Example:
> f = inline(’sin(x.*y)/sqrt(x+y)’,’x’,’y’)
> I = dblquad(f,0,1,0,2)
Matlab uses an advanced form of Simpson’s method.
Integration over non-rectangles:
Redefine the function to be zero outside the region. For example:
> f = inline(’sin(x.*y).^3.*(2*x + y <= 2)’)
> I = dblquad(f,0,1,0,2)
Integrates f(x, y) = sin
3
(xy) on the triangle with corners (0, 0), (0, 2), and (1, 0).
Triangles:
Matlab stores triangulations as a matrix of vertices V and triangles T.
> T = delaunay(V) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Produces triangles from vertices.
> trimesh(T,x,y)
> trimesh(T,x,y,z)
> trisurf(T,x,y)
> trisurf(T,x,y,z)
Part IV
Differential Equations
c _Copyright, Todd Young and Martin Mohlenkamp, Mathematics Department, Ohio University, 2007
Lecture 29
Reduction of Higher Order Equations to
Systems
The motion of a pendulum
Consider the motion of an ideal pendulum that consists of a mass m attached to an arm of length ℓ.
If we ignore friction, then Newton’s laws of motion tell us:
m
¨
θ = −
mg

sin θ,
where θ is the angle of displacement.
t
e
e
e
e
e
e
e
ew

θ
Figure 29.1: A pendulum.
If we also incorporate moving friction and sinusoidal forcing then the equation takes the form:
m
¨
θ +γ
˙
θ +
mg

sin θ = Asin Ωt.
Here γ is the coefficient of friction and A and Ω are the amplitude and frequency of the forcing.
Usually, this equation would be rewritten by dividing through by m to produce:
¨
θ +c
˙
θ +ω sin θ = a sin Ωt, (29.1)
where c = γ/m. ω = g/ℓ and a = A/m.
This is a second order ODE because the second derivative with respect to time t is the highest
derivative. It is nonlinear because it has the term sin θ and which is a nonlinear function of the
dependent variable θ. A solution of the equation would be a function θ(t). To get a specific solution
we need side conditions. Because it is second order, 2 conditions are needed, and the usual conditions
are initial conditions:
θ(0) = θ
0
and
˙
θ(0) = v
0
. (29.2)
106
107
Converting a general higher order equation
All of the standard methods for solving ordinary differential equations are intended for first order
equations. For this reason, it is inconvenient to solve higher order equations numerically. However,
most higher-order differential equations that occur in applications can be converted to a system of
first order equations and that is what is usually done in practice.
Suppose that an n-th order equation can be solved for the n-th derivative, i.e. it can be written in
the form:
x
(n)
= f
_
t, x, ˙ x, ¨ x, . . . ,
d
n−1
x
dt
n−1
_
. (29.3)
Then it can be converted to a first-order system by this standard change of variables:
y
1
= x
y
2
= ˙ x
.
.
.
y
n
= x
(n−1)
=
d
n−1
x
dt
n−1
.
(29.4)
The resulting first-order system is the following:
˙ y
1
= ˙ x = y
2
˙ y
2
= ¨ x = y
3
.
.
.
˙ y
n
= x
(n)
= f(t, y
1
, y
2
, . . . , y
n
).
(29.5)
In vector form this is simply ˙ y = f (t, y) with f
i
(t, y) = y
i+1
for i < n and f
n
(t, y) = f(t, y
1
, y
2
, . . . , y
n
).
For the example of the pendulum (29.1) the change of variables has the form
y
1
= θ
y
2
=
˙
θ,
(29.6)
and the resulting equations are
˙ y
1
= y
2
˙ y
2
= −cy
2
−ω sin(y
1
) +a sin(Ωt).
(29.7)
In vector form this is
˙ y =
_
y
2
−cy
2
−ω sin(y
1
) +a sin(Ωt)
_
.
The initial conditions are converted to
y(0) =
_
y
1
(0)
y
2
(0)
_
=
_
θ
0
v
0
_
. (29.8)
As stated above, the main reason we wish to change a higher order equation into a system of
equations is that this form is convenient for solving the equation numerically. Most general software
for solving ODEs (including Matlab) requires that the ODE be input in the form of a first-order
system. In addition, there is a conceptual reason to make the change. In a system described by a
108 LECTURE 29. REDUCTION OF HIGHER ORDER EQUATIONS TO SYSTEMS
higher order equation, knowing the position is not enough to know what the system is doing. In
the case of a second order equation, such as the pendulum, one must know both the angle and the
angular velocity to know what the pendulum is really doing. We call the pair (θ,
˙
θ) the state of the
system. Generally in applications the vector y is the state of the system described by the differential
equation.
Using Matlab to solve a system of ODE’s
In Matlab there are several commands that can be used to solve an initial value problem for
a system of differential equations. Each of these correspond to different solving methods. The
standard one to use is ode45, which uses the algorithm “Runge-Kutta 4 5”. We will learn about
this algorithm later.
To implement ode45 for a system, we have to input the vector function f that defines the system.
For the pendulum system (29.7), we will assume that all the constants are 1, except c = .1, then we
can input the right hand side as:
> f = inline(’[y(2);-.1*y(2)-sin(y(1))+sin(t)]’,’t’,’y’)
The command ode45 is then used as follows:
> [T Y] = ode45(f,[0 20],[1;-1.5]);
Here [0 20] is the time span you want to consider and [1;-1.5] is the initial value of the vector
y. The output T contains times and Y contains values of the vector y at those times. Try:
> size(T)
> T(1:10)
> size(Y)
> Y(1:10,:)
Since the first coordinate of the vector is the position (angle), we are mainly interested in its values:
> theta = Y(:,1);
> plot(T,theta)
In the next two sections we will learn enough about numerical methods for initial value problems
to understand roughly how Matlab produces this approximate solution.
Exercises
29.1 Transform the given ODE into a first order system:
...
x + ¨ x
2
−3 ˙ x
3
+ cos
2
x = e
−t
sin(3t).
Suppose the initial conditions for the ODE are: x(1) = 1, ˙ x(1) = 2, and ¨ x(1) = 0. Find
a numerical solution of this IVP using ode45 and plot the first coordinate (x). Try time
intervals [1 2] and [1 2.1] and explain what you observe. (Remember to use entry-wise
operations, .* and .^, in the definition of the vector function.)
Lecture 30
Euler Methods
Numerical Solution of an IVP
Suppose we wish to numerically solve the initial value problem
˙ y = f (t, y), y(a) = y
0
, (30.1)
on an interval of time [a, b].
By a numerical solution, we must mean an approximation of the solution at a finite number of
points, i.e.
(t
0
, y
0
), (t
1
, y
1
), (t
2
, y
2
), . . . , (t
n
, y
n
),
where t
0
= a and t
n
= b. The first of these points is exactly the initial value. If we take n steps as
above, and the steps are evenly spaced, then the time change in each step is
h =
b −a
n
, (30.2)
and the times t
i
are given simply by t
i
= a + ih. This leaves the most important part of finding a
numerical solution: determining y
1
, y
2
, . . . , y
n
in a way that is as consistent as possible with (30.1).
To do this, first write the differential equation in the indexed notation
˙ y
i
≈ f (t
i
, y
i
), (30.3)
and will then replace the derivative ˙ y by a difference. There are many ways we might carry this out
and in the next section we study the simplest.
The Euler Method
The most straight forward approach is to replace ˙ y
i
in (30.3) by its forward difference approximation.
This gives
y
i+1
−y
i
h
= f (t
i
, y
i
).
Rearranging this gives us a way to obtain y
i+1
from y
i
known as Euler’s method:
y
i+1
= y
i
+hf (t
i
, y
i
). (30.4)
With this formula, we can start from (t
0
, y
0
) and compute all the subsequent approximations (t
i
, y
i
).
This is very easy to implement, as you can see from the following program (which can be downloaded
from the class web site):
109
110 LECTURE 30. EULER METHODS
function [T , Y] = myeuler(f,tspan,y0,n)
% function [T , Y] = myeuler(f,tspan,y0,n)
% Solves dy/dt = f(t,y) with initial condition y(a) = y0
% on the interval [a,b] using n steps of Euler s method.
% Inputs: f -- a function f(t,y) that returns a column vector of the same
% length as y
% tspan -- a vector [a,b] with the start and end times
% y0 -- a column vector of the initial values, y(a) = y0
% n -- number of steps to use
% Outputs: T -- a n+1 column vector containing the times
% Y -- a (n+1) by d matrix where d is the length of y
% Y(t,i) gives the ith component of y at time t
% parse starting and ending points
a = tspan(1);
b = tspan(2);
h = (b-a)/n; % step size
t = a; y = y0; % t and y are the current variables
T = a; Y = y0’; % T and Y will record all steps
for i = 1:n
y = y + h*f(t,y);
t = a + i*h;
T = [T; t];
Y = [Y; y’];
end
To use this program we need a function, such as the vector function for the pendulum:
> f = inline(’[y(2);-.1*y(2)-sin(y(1)) + sin(t)]’,’t’,’y’)
Then type:
> [T Y] = myeuler(f,[0 20],[1;-1.5],5);
Here [0 20] is the time span you want to consider, [1;-1.5] is the initial value of the vector y and
5 is the number of steps. The output T contains times and Y contains values of the vector as the
times. Try:
> size(T)
> size(Y)
Since the first coordinate of the vector is the angle, we only plot its values:
> theta = Y(:,1);
> plot(T,theta)
In this plot it is clear that n = 5 is not adequate to represent the function. Type:
> hold on
then redo the above with 5 replaced by 10. Next try 20, 40, 80, and 200. As you can see the graph
becomes increasingly better as n increases. We can compare these calculations with Matlab’s
built-in function with the commands:
> [T Y]= ode45(f,[0 20],[1;-1.5]);
> theta = Y(:,1);
> plot(T,theta,’r’)
111
The problem with the Euler method
You can think of the Euler method as finding a linear approximate solution to the initial value
problem on each time interval. An obvious shortcoming of the method is that it makes the approx-
imation based on information at the beginning of the time interval only. This problem is illustrated
well by the following IVP:
¨ x +x = 0, x(0) = 1, ˙ x(0) = 0 . (30.5)
You can easily check that the exact solution of this IVP is
x(t) = cos(t).
If we make the standard change of variables
y
1
= x, y
2
= ˙ x,
then we get
˙ y
1
= y
2
, ˙ y
2
= −y
1
.
Then the solution should be y
1
(t) = cos(t), y
2
(t) = sin(t). If we then plot the solution in the (y
1
, y
2
)
plane, we should get exactly a unit circle. We can solve this IVP with Euler’s method:
> g = inline(’[y(2);-y(1)]’,’t’,’y’)
> [T Y] = myeuler(g,[0 4*pi],[1;0],20)
> y1 = Y(:,1);
> y2 = Y(:,2);
> plot(y1,y2)
As you can see the approximate solution goes far from the true solution. Even if you increase the
number of steps, the Euler solution will eventually drift outward away from the circle because it
does not take into account the curvature of the solution.
The Modified Euler Method
An idea which is similar to the idea behind the trapezoid method would be to consider f at both
the beginning and end of the time step and take the average of the two. Doing this produces the
Modified (or Improved) Euler method represented by the following equations:
k
1
= hf (t
i
, y
i
)
k
2
= hf (t
i
+h, y
i
+k
1
)
y
i+1
= y
i
+
1
2
(k
1
+k
2
) .
(30.6)
Here k
1
captures the information at the beginning of the time step (same as Euler), while k
2
is the
information at the end of the time step.
The following program implements the Modified method. It may be downloaded from the class web
site.
112 LECTURE 30. EULER METHODS
function [T , Y] = mymodeuler(f,tspan,y0,n)
% Solves dy/dt = f(t,y) with initial condition y(a) = y0
% on the interval [a,b] using n steps of the modified Euler’s method.
% Inputs: f -- a function f(t,y) that returns a column vector of the same
% length as y
% tspan -- a vector [a,b] with the start and end times
% y0 -- a column vector of the initial values, y(a) = y0
% n -- number of steps to use
% Outputs: T -- a n+1 column vector contianing the times
% Y -- a (n+1) by d matrix where d is the length of y
% Y(t,i) gives the ith component of y at time t
% parse starting and ending points
a = tspan(1);
b = tspan(2);
h = (b-a)/n; % step size
t = a; y = y0; % t and y are the current variables
T = a; Y = y0’; % T and Y will record all steps
for i = 1:n
k1 = h*f(t,y);
k2 = h*f(t+h,y+k1);
y = y + .5*(k1+k2);
t = a + i*h;
T = [T; t];
Y = [Y; y’];
end
We can test this program on the IVP above:
> [T Ym] = mymodeuler(g,[0 4*pi],[1;0],20)
> ym1 = Ym(:,1);
> ym2 = Ym(:,2);
> plot(ym1,ym2)
Exercises
30.1 Download the files myeuler.m and mymodeuler.m from the class web site.
1. Type the following commands:
> hold on
> f = inline(’sin(t)*cos(x)’,’t’,’x’)
> [T Y]=myeuler(f,[0,12],.1,10);
> plot(T,Y)
Position the plot window so that it can always be seen and type:
> [T Y]=myeuler(f,[0,12],.1,20);
> plot(T,Y)
Continue to increase the last number in the above until the graph stops changing (as
far as you can see). Record this number and print the final graph. Type hold off and
kill the plot window.
2. Follow the same procedure using mymodeuler.m .
3. Describe what you observed. In particular compare how fast the two methods converge
as n is increased (h is decreased).
Lecture 31
Higher Order Methods
The order of a method
For numerical solutions of an initial value problem there are two ways to measure the error. The
first is the error of each step. This is called the Local Truncation Error or LTE. The other is the
total error for the whole interval [a, b]. We call this the Global Truncation Error or GTE.
For the Euler method the LTE is of order O(h
2
), i.e. the error is comparable to h
2
. We can show
this directly using Taylor’s Theorem:
y(t +h) = y(t) +h ˙ y(t) +
h
2
2
¨ y(c)
for some c between t and t + h. In this equation we can replace ˙ y(t) by f(t, y(t)), which makes
the first two terms of the right hand side be exactly the Euler method. The error is then
h
2
2
¨ y(c)
or O(h
2
). It would be slightly more difficult to show that the LTE of the modified Euler method is
O(h
3
), an improvement of one power of h.
We can roughly get the GTE from the LTE by considering the number of steps times the LTE. For
any method, if [a, b] is the interval and h is the step size, then n = (b −a)/h is the number of steps.
Thus for any method, the GTE is one power lower in h than the LTE. Thus the GTE for Euler is
O(h) and for modified Euler it is O(h
2
).
By the order of a method, we mean the power of h in the GTE. Thus the Euler method is a 1st
order method and modified Euler is a 2nd order method.
Fourth Order Runge-Kutta
The most famous of all IVP methods is the classic Runge-Kutta method of order 4:
k
1
= hf (t
i
, y)
k
2
= hf (t
i
+h/2, y
i
+k
1
/2)
k
3
= hf (t
i
+h/2, y
i
+k
2
/2)
k
4
= hf (t
i
+h, y
i
+k
3
)
y
i+1
= y
i
+
1
6
(k
1
+ 2k
2
+ 2k
3
+k
4
) .
(31.1)
Notice that this method uses values of f (t, y) at 4 different points. In general a method needs n
values of f to achieve order n. The constants used in this method and other methods are obtained
from Taylor’s Theorem. They are precisely the values needed to make all error terms cancel up to
h
n+1
f
(n+1)
(c)/(n + 1)!.
113
114 LECTURE 31. HIGHER ORDER METHODS
Variable Step Size and RK45
If the order of a method is n, then the GTE is comparable to h
n
, which means it is approximately
Ch
n
, where C is some constant. However, for different differential equations, the values of C may
be very different. Thus it is not easy beforehand to tell how small h should be to get the error
within a given tolerence. For instance, if the true solution oscillates very rapidly, we will obviously
need a smaller step size than for a solution that is nearly constant.
How can a program then choose h small enough to produce the required accuracy? We also do not
wish to make h much smaller than necessary, since that would increase the number of steps. To
accomplish this a program tries an h and tests to see if that h is small enough. If not it tries again
with a smaller h. If it is too small, it accepts that step, but on the next step it tries a larger h. This
process is called variable step size.
Deciding if a single step is accurate enough could be accomplished in several ways, but the most
common are called embedded methods. The Runge-Kutta 45 method, which is used in ode45,
is an embedded method. In the RK45, the function f is evaluated at 5 different points. These are
used to make a 5th order estimate y
i+1
. At the same time, 4 of the 5 values are used to also get a
4th order estimate. If the 4th order and 5th order estimates are close, then we can conclude that
they are accurate. If there is a large discrepency, then we can conclude that they are not accurate
and a smaller h should be used.
To see variable step size in action, we will define and solve two different ODEs and solve them on
the same interval:
> f1 = inline(’[-y(2);y(1)]’,’t’,’y’)
> f2 = inline(’[-5*y(2);5*y(1)]’,’t’,’y’)
> [T1 Y1] = ode45(f1,[0 20],[1;0]);
> [T2 Y2] = ode45(f2,[0 20],[1;0]);
> y1 = Y1(:,1);
> y2 = Y2(:,1);
> plot(T1,y1,’bx-’)
> hold on
> plot(T2,y2,’ro-’)
> size(T1)
> size(T2)
Why order matters
Many people would conclude on first encounter that the advantage of a higher order method would
be that you can get a more accurate answer than for a lower order method. In reality, this is not
quite how things work. In engineering problems, the accuracy needed is usually a given and it is
usually not extremely high. Thus getting more and more accurate solutions is not very useful. So
where is the advantage? Consider the following example.
Suppose that you need to solve an IVP with an error of less than 10
−4
. If you use the Euler
method, which has GTE of order O(h), then you would need h ≈ 10
−4
. So you would need about
n ≈ (b −a) 10
4
steps to find the solution.
Suppose you use the second order, modified Euler method. In that case the GTE is O(h
2
), so you
would need to use h
2
≈ 10
−4
, or h ≈ 10
−2
. This would require about n ≈ (b − a) 10
2
steps.
115
That is a hundred times fewer steps than you would need to get the same accuracy with the Euler
method.
If you use the RK4 method, then h
4
needs to be approximately 10
−4
, and so h ≈ 10
−1
. This means
you need only about n ≈ (b −a) 10 steps to solve the problem, i.e. a thousand times fewer steps
than for the Euler method.
Thus the real advantage of higher order methods is that they can run a lot faster at the same
accuracy. This can be especially important in applications where one is trying to make real-time
adjustments based on the calculations. Such is often the case in robots and other applications with
dynamic controls.
Exercises
31.1 There is a Runge-Kutta 2 method, which is also known as the midpoint method. It is sum-
marized by the following equations:
k
1
= hf (t
i
, y
i
)
k
2
= hf (t
i
+h/2, y
i
+k
1
/2)
y
i+1
= y
i
+k
2
.
(31.2)
Modify the program mymodeuler into a program myRK2 that does the RK2 method. Compare
myRK2 and mymodeuler on the following IVP with time span [0, 4π]:
¨ x +x = 0, x(0) = 1, ˙ x(0) = 0.
Using format long compare the results of the two programs for n = 10, 100, 1000. In partic-
ular, compare the errors of y(4π), which should be (1, 0).
Turn in the modified program and a summary of the results.
Lecture 32
Multi-step Methods*
Exercises
32.1
116
Lecture 33
ODE Boundary Value Problems and
Finite Differences
Steady State Heat and Diffusion
If we consider the movement of heat in a long thin object (like a metal bar), it is known that the
temperature, u(x, t), at a location x and time t satisfies the partial differential equation
u
t
−u
xx
= g(x, t), (33.1)
where g(x, t) is the effect of any external heat source. The same equation also describes the diffusion
of a chemical in a one-dimensional environment. For example the environment might be a canal,
and then g(x, t) would represent how a chemical is introduced.
Sometimes we are interested only in the steady state of the system, supposing g(x, t) = g(x) and
u(x, t) = u(x). In this case
u
xx
= −g(x).
This is a linear second-order ordinary differential equation. We could find its solution exactly if g(x)
is not too complicated. If the environment or object we consider has length L, then typically one
would have conditions on each end of the object, such as u(0) = 0, u(L) = 0. Thus instead of an
initial value problem, we have a boundary value problem or BVP.
Beam With Tension
Consider a simply supported beam with modulus of elasticity E, moment af inertia I, a uniform
load w, and end tension T (see Figure 33.1). If y(x) denotes the deflection at each point x in the
beam, then y(x) satisfies the differential equation:
y
′′
(1 + (y

)
2
)
3/2

T
EI
y =
wx(L −x)
2EI
, (33.2)
with boundary conditions: y(0) = y(L) = 0. This equation is nonlinear and there is no hope to
solve it exactly. If the deflection is small then (y

)
2
is negligible compared to 1 and the equation
approximately simplifies to:
y
′′

T
EI
y =
wx(L −x)
2EI
. (33.3)
This is a linear equation and we can find the exact solution. We can rewrite the equation as
y
′′
−αy = βx(L −x), (33.4)
117
118LECTURE 33. ODE BOUNDARYVALUE PROBLEMS AND FINITE DIFFERENCES
t t ' E
T T
c c c c c
w
Figure 33.1: A simply supported beam with a uniform load w and end tension T.
where
α =
T
EI
and β =
w
2EI
, (33.5)
and then the exact solution is
y(x) =

α
2
e

αL
e

αL
+ 1
e


αx
+

α
2
1
e

αL
+ 1
e

αx
+
β
α
x
2

βL
α
x +

α
2
. (33.6)
Finite Difference Method – Linear ODE
A finite difference equation is an equation obtained from a differential equation by replacing the
variables by their discrete versions and derivatives by difference formulas.
First we will consider equation (33.3). Suppose that the beam is a W12x22 structural steel I-beam.
Then L = 120 in., E = 29 10
6
lb./in.
2
and I = 121in.
4
. Suppose that the beam is carrying a
uniform load of 100, 000 lb. so that w = 120, 000/120 = 10, 000 and a tension of T = 10, 000 lb.. W
e calculate from (33.5) α = 2.850 10
−6
and β = 1.425 10
−6
. Thus we have the following BVP:
y
′′
= 2.850 10
−6
y + 1.425 10
−6
x(120 −x), y(0) = y(120) = 0. (33.7)
First subdivide the interval [0, 120] into four equal subintervals. The nodes of this subdivion are
x
0
= 0, x
1
= 30, x
2
= 60, . . . , x
4
= 120. We will then let y
0
, y
1
, . . . , y
4
denote the deflections at the
nodes. From the boundary conditions we have immediately:
y
0
= y
4
= 0.
To determine the deflections at the interior points we will rely on the differential equation. Recall
the central difference formula
y
′′
(x
i
) ≈
y
i+1
−2y
i
+y
i−1
h
2
.
In this case we have h = (b −a)/n = (120 −0)/4 = 30. Replacing all the variables in the equation
(33.4) by their discrete versions we get
y
i+1
−2y
i
+y
i−1
= h
2
αy
i
+h
2
βx
i
(L −x
i
).
Substituting in for α, β and h we obtain:
y
i+1
−2y
i
+y
i−1
= 900 2.850 10
−6
y
i
+ 900 1.425 10
−6
x
i
(120 −x
i
)
= 2.565 10
−3
y
i
+ 1.282 10
−3
x
i
(120 −x
i
).
119
This equation makes sense for i = 1, 2, 3. At x
1
= 30, the equation becomes:
y
2
−2y
1
+y
0
= 2.565 10
−3
y
1
+ 1.282 10
−3
30(90)
⇔y
2
−2.002565y
1
= 3.463.
(33.8)
Note that this equation is linear in the unknowns y
1
and y
2
. At x
2
= 60 we have:
y
3
−2y
2
+y
1
= .002565y
2
+ 1.282 10
−3
60
2
⇔y
3
−2.002565y
2
+y
1
= 4.615.
(33.9)
At x
3
= 90 we have (since y
4
= 0)
−2.002565y
3
+y
2
= 3.463. (33.10)
Thus (y
1
, y
2
, y
3
) is the solution of the linear system:
_
_
−2.002565 1 0 3.463
1 −2.002565 1 4.615
0 1 −2.002565 3.463
_
_
.
We can easily find the solution of this system in Matlab:
> A = [ -2.002565 1 0 ; 1 -2.002565 1 ; 0 1 -2.002565]
> b = [ 3.463 4.615 3.463 ]’
> y = A\b
To graph the solution, we need define the x values and add on the values at the endpoints:
> x = 0:30:120
> y = [0 ; y ; 0]
> plot(x,y,’d’)
Adding a spline will result in an excellent graph.
The exact solution of this BVP is given in (33.6). That equation, with the parameter values for the
W12x22 I-beam as in the example, is in the program myexactbeam.m on the web site. We can plot
the true solution on the same graph:
> hold on
> myexactbeam
Thus our numerical solution is extremely good considering how few subintervals we used and how
very large the deflection is.
An amusing exercise is to set T = 0 in the program myexactbeam.m; the program fails because the
exact solution is no longer valid. Also try T = .1 for which you will observe loss of precision. On
the other hand the finite difference method still works when we set T = 0.
Exercises
33.1 Derive the finite difference equations for the BVP (33.7) on the same domain ([0, 120]), but
with eight subintervals and solve (using Matlab) as in the example. Plot your result, together
with the exact solution (33.6) from the program myexactbeam.m.
33.2 By replacing y
′′
and y

with central differences, derive the finite difference equation for the
BVP:
y
′′
+y

−y = x with y(0) = y(1) = 0
with 4 subintervals. Solve them and plot the solution using Matlab.
Lecture 34
Finite Difference Method – Nonlinear
ODE
Heat conduction with radiation
If we again consider the heat in a metal bar of length L, but this time consider the effect of radiation
as well as conduction, then the steady state equation has the form
u
xx
−d(u
4
−u
4
b
) = −g(x), (34.1)
where u
b
is the temperature of the background, d incorporates a coefficient of radiation and g(x) is
the heat source.
If we again replace the continuous problem by its discrete approximation then we get
u
i+1
−2u
i
+u
i−1
h
2
−d(u
4
i
−u
4
b
) = −g
i
= −g(x
i
). (34.2)
This equation is nonlinear in the unknowns, thus we no longer have a system of linear equations
to solve, but a system of nonlinear equations. One way to solve these equations would be by the
multivariable Newton method. Instead, we introduce another interative method.
Relaxation Method for Nonlinear Finite Differences
We can rewrite equation (34.2) as
u
i+1
−2u
i
+u
i−1
= h
2
d(u
4
i
−u
4
b
) −h
2
g
i
.
From this we can solve for u
i
in terms of the other quantities:
2u
i
= u
i+1
+u
i−1
−h
2
d(u
4
i
−u
4
b
) +h
2
g
i
.
Next we add u
i
to both sides of the equation to obtain
3u
i
= u
i+1
+u
i
+u
i−1
−h
2
d(u
4
i
−u
4
b
) +h
2
g
i
,
and then divide by 3 to get
u
i
=
1
3
(u
i+1
+u
i
+u
i−1
) −
h
2
3
_
d(u
4
i
−u
4
b
) +g
i
_
.
Now for the main idea. We will begin with an initial guess for the value of u
i
for each i, which we
can represent as a vector u
0
. Then we will use the above equation to get better estimates, u
1
, u
2
,
. . . , and hope that they converge to the correct answer.
120
121
If we let
u
j
= (u
j
0
, u
j
1
, u
j
2
, . . . , u
j
n−1
, u
j
n
)
denote the jth approximation, then we can obtain that j + 1st estimate from the formula
u
j+1
i
=
1
3
_
u
j
i+1
+u
j
i
+u
j
i−1
_

h
2
3
_
d((u
j
i
)
4
−u
4
b
) +g
i
_
.
Notice that g
i
and u
b
do not change. In the resulting equation, we have u
i
at each successive step
depending on its previous value and the equation itself.
Implementing the Relaxation Method
In the following program we solve the finite difference equations (34.2) with the boundary conditions
u(0) = 0 and u(L) = 0. We let L = 4, n = 4, d = 1, and g(x) = sin(πx/4).
Notice that the vector u always contains the current estimate of the values of u.
% mynonlinheat (lacks comments)
L = 4; n = 4;
h = L/n; hh = h^2/3;
u0 = 0; uL = 0; ub = .5; ub4 = ub^4;
x = 0:h:L; g = sin(pi*x/4); u = zeros(1,n+1);
steps = 4;
u(1)=u0; u(n+1)=uL;
for j = 1:steps
u(2:n) = (u(3:n+1)+u(2:n)+u(1:n-1))/3 + hh*(-u(2:n).^4+ub4+g(2:n));
end
plot(x,u)
If you run this program with the given n and steps the result will not seem reasonable.
We can plot the initial guess by adding the command plot(x,u) right before the for loop. We can
also plot successive iterations by moving the last plot(x,u) before the end. Now we can experiment
and see if the iteration is converging. Try various values of steps and n to produce a good plot.
You will notice that this method converges quite slowly. In particular, as we increase n, we need to
increase steps like n
2
, i.e. if n is large then steps needs to be really large.
Exercises
34.1 (a) Modify the script program mynonlinheat to plot the initial guess and all intermediate
approximations. Add complete comments to the program. Print the program and a plot
using large enough n and steps to see convergence.
(b) Modify your improved mynonlinheat to mynonlinheattwo that has the boundary con-
ditions
u(0) = 5 and u(L) = 10.
Fix the comments to reflect the new boundary conditions. Print the program and a plot
using large enough n and steps to see convergence.
Lecture 35
Parabolic PDEs - Explicit Method
Heat Flow and Diffusion
In the previous sections we studied PDE that represent steady-state heat problem. There was no
time variable in the equation. In this section we begin to study how to solve equations that involve
time, i.e. we calculate temperature profiles that are changing.
The conduction of heat and diffusion of a chemical happen to be modeled by the same differential
equation. The reason for this is that they both involve similar processes. Heat conduction occurs
when hot, fast moving molecules bump into slower molecules and transfer some of their energy. In a
solid this involves moles of molecules all moving in different, nearly random ways, but the net effect
is that the energy eventually spreads itself out over a larger region. The diffusion of a chemical in a
gas or liquid simliarly involves large numbers of molecules moving in different, nearly random ways.
These molecules eventually spread out over a larger region.
In three dimensions, the equation that governs both of these processes is the heat/diffusion equation
u
t
= c∆u,
where c is the coefficient of conduction or diffusion, and ∆u(x, y, z) = u
xx
+u
yy
+u
zz
. The symbol ∆
in this context is called the Laplacian. If there is also a heat/chemical source, then it is incorporated
a function g(x, y, z, t) in the equation as
u
t
= c∆u +g.
In some problems the z dimension is irrelevent, either because the object in question is very thin,
or u does not change in the z direction. In this case the equation is
u
t
= c∆u = c(u
xx
+u
yy
).
Finally, in some cases only the x direction matters. In this case the equation is just
u
t
= cu
xx
, (35.1)
or
u
t
= cu
xx
+g(x, t). (35.2)
In this lecture we will learn a straight-forward technique for solving (35.1) and (35.2). It is very
similar to the finite difference method we used for nonlinear boundary value problems.
It is worth mentioning a related equation
u
t
= c∆(u
γ
) for γ > 1 ,
122
123
which is called the porus-media equation. This equation models diffusion in a solid, but porus,
material, such as sandstone or an earthen structure. We will not solve this equation numerically,
but the methods introduced here would work. Many equations that involve 1 time derivative and 2
spatial derivatives are parabolic and the methods introduced here will work for most of them.
Explicit Method Finite Differences
The one dimensional heat/diffusion equation u
t
= cu
xx
, has two independent variables, t and x,
and so we have to discretize both. Since we are considering 0 ≤ x ≤ L, we subdivide [0, L] into m
equal subintervals, i.e. let
h = L/m
and
(x
0
, x
1
, x
2
, . . . , x
m−1
, x
m
) = (0, h, 2h, . . . , L −h, L).
Similarly, if we are interested in solving the equation on an interval of time [0, T], let
k = T/n
and
(t
0
, t
1
, t
2
, . . . , t
n−1
, t
n
) = (0, k, 2k, . . . , T −k, T).
We will then denote the approximate solution at the grid points by
u
ij
≈ u(x
i
, t
j
).
The equation u
t
= cu
xx
can then be replaced by the difference equations
u
i,j+1
−u
i,j
k
=
c
h
2
(u
i−1,j
−2u
i,j
+u
i+1,j
). (35.3)
Here we have used the forward difference for u
t
and the central difference for u
xx
. This equation
can be solved for u
i,j+1
to produce
u
i,j+1
= ru
i−1,j
+ (1 −2r)u
i,j
+ru
i+1,j
(35.4)
for 1 ≤ i ≤ m−1, 0 ≤ j ≤ n −1, where
r =
ck
h
2
. (35.5)
The formula (35.4) allows us to calculate all the values of u at step j +1 using the values at step j.
Notice that u
i,j+1
depends on u
i,j
, u
i−1,j
and u
i+1,j
. That is u at grid point i depends on its
previous value and the values of its two nearest neighbors at the previous step (see Figure 35.1).
Initial Condition
To solve the partial differential equation (35.1) or (35.2) we need an initial condition. This rep-
resents the state of the system when we begin, i.e. the initial temperature distribution or initial
concentration profile. This is represented by
u(x, 0) = f(x).
To implement this in a program we let
u
i,0
= f(x
i
).
124 LECTURE 35. PARABOLIC PDES - EXPLICIT METHOD
x x x x x
x | | | x
x x
u
i,j+1
x x
x x x x x
T
t
j
t
j+1
x
i−1
x
i
x
i+1
Figure 35.1: The value at grid point i depends on its previous values and the previous
values of its nearest neighbors.
Boundary Conditions
To solve the partial differential equation (35.1) or (35.2) we also need boundary conditions. Just
as in the previous section we will have to specify something about the ends of the domain, i.e. at
x = 0 and x = L. One possibility is fixed boundary conditions, which we can implement just as we
did for the ODE boundary value problem.
A second possibility is called variable boundary conditions. This is represented by time-
dependent functions,
u(0, t) = g
1
(t) and u(L, t) = g
2
(t).
In a heat problem, g
1
and g
2
would represent heating or cooling applied to the ends. These are
easily implemented in a program by letting u
0,j
= g
1
(t
j
) and u
m,j
= g
2
(t
j
).
Implementation
The following program (also available on the web page) implements the explicit method. It in-
corporates variable boundary conditions at both ends. To run it you must define functions f, g
1
and g
2
. Notice that the main loop has only one line. The values of u are kept as a matrix. It is
often convenient to define a matrix of the right dimension containing all zeros, and then fill in the
calculated values as the program runs.
Run this program using L = 2, T = 20, f(x) = .5x, g
1
(t) = 0, and g
2
(t) = cos(t). Note that g
1
(t)
must be input as g1 = inline(’0*t’).
125
function [t x u] = myheat(f,g1,g2,L,T,m,n,c)
% function [t x u] = myheat(f,g1,g2,L,T,m,n,c)
% solve u_t = c u_xx for 0<=x<=L, 0<=t<=T
% BC: u(0, t) = g1(t); u(L,t) = g2(t)
% IC: u(x, 0) = f(x)
% Inputs:
% f -- inline function for IC
% g1,g2 -- inline functions for BC
% L -- length of rod
% T -- length of time interval
% m -- number of subintervals for x
% n -- number of subintervals for t
% c -- rate constant in equation
% Outputs:
% t -- vector of time points
% x -- vector of x points
% u -- matrix of the solution, u(i,j)~=u(x(i),t(j))
h = L/m; k = T/n;
r = c*k/h^2; rr = 1 - 2*r;
x = linspace(0,L,m+1);
t = linspace(0,T,n+1);
%Set up the matrix for u:
u = zeros(m+1,n+1);
% assign initial conditions
u(:,1) = f(x);
% assign boundary conditions
u(1,:) = g1(t); u(m+1,:) = g2(t);
% find solution at remaining time steps
for j = 1:n
u(2:m,j+1) = r*u(1:m-1,j) + rr*u(2:m,j) + r*u(3:m+1,j);
end
% plot the results
mesh(x,t,u’)
Exercises
35.1 Run the program myheat.m with L = 2π, T = 20, c = .5, g
1
(t) = sin(t) and f(x) = −sin(x/4).
Set m = 20 and experiment with n. Get a plot when the program is stable and one when it
isn’t. Turn in the plots.
Lecture 36
Solution Instability for the Explicit
Method
As we saw in experiments using myheat.m, the solution can become unbounded unless the time
steps are small. In this lecture we consider why.
Writing the Difference Equations in Matrix Form
If we use the boundary conditions u(0) = u(L) = 0 then the explicit method of the previous section
has the form:
u
i,j+1
= ru
i−1,j
+ (1 −2r)u
i,j
+ru
i+1,j
, 1 ≤ i ≤ m−1, 0 ≤ j ≤ n −1, (36.1)
where u
0,j
= 0 and u
m,j
= 0. This is equivalent to the matrix equation
u
j+1
= Au
j
, (36.2)
where u
j
is the column vector (u
1,j
, u
2,j
, . . . , u
m,j
)

representing the state at the jth time step and
A is the matrix
A =
_
_
_
_
_
_
_
_
_
1 −2r r 0 0
r 1 −2r r
.
.
.
.
.
.
0
.
.
.
.
.
.
.
.
. 0
.
.
.
.
.
. r 1 −2r r
0 0 r 1 −2r
_
_
_
_
_
_
_
_
_
. (36.3)
Unfortunately, this matrix can have a property which is very bad in this context. Namely, it can
cause exponential growth of error unless r is small. To see how this happens, suppose that U
j
is
the vector of correct values of u at time step t
j
and E
j
is the error of the approximation u
j
, then
u
j
= U
j
+E
j
.
From (36.2), the approximation at the next time step will be
u
j+1
= AU
j
+AE
j
,
and if we continue for k steps,
u
j+k
= A
k
U
j
+A
k
E
j
.
The problem with this is the term A
k
E
j
. This term is exactly what we would do in the power
method for finding the eigenvalue of A with the largest absolute value. If the matrix A has ew’s
with absolute value greater than 1, then this term will grow exponentially. Figure 36.1 shows the
largest absolute value of an ew of A as a function of the parameter r for various sizes of the matrix
A. As you can see, for r > 1/2 the largest absolute ew grows rapidly for any m and quickly becomes
greater than 1.
126
127
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
0.5
1
1.5
2
2.5
3
r
EW
m=2
m=3
Figure 36.1: Maximum absolute eigenvalue EW as a function of r for the matrix A from
the explicit method for the heat equation calculated for matrices A of sizes m = 2 . . . 10.
Whenever EW > 1 the method is unstable, i.e. errors grow exponentially with each step.
When using the explicit method r < 1/2 is a safe choice.
Consequences
Recall that r = ck/h
2
. Since this must be less than 1/2, we have
k <
h
2
2c
.
The first consequence is obvious: k must be relatively small. The second is that h cannot be too
small. Since h
2
appears in the formula, making h small would force k to be extremely small!
A third consequence is that we have a converse of this analysis. Suppose r < .5. Then all the
eigenvalues will be less than one. Recall that the error terms satisfy
u
j+k
= A
k
U
j
+A
k
E
j
.
If all the eigenvalues of A are less than 1 in absolute value then A
k
E
j
grows smaller and smaller as
k increases. This is really good. Rather than building up, the effect of any error diminishes as time
passes! From this we arrive at the following principle: If the explicit numerical solution for a
parabolic equation does not blow up, then errors from previous steps fade away!
Finally, we note that if we have other boundary conditions then instead of equation (36.2) we have
u
j+1
= Au
j
+rb
j
, (36.4)
where the first and last entries of b
j
contain the boundary conditions and all the other entries are
zero. In this case the errors behave just as before, if r > 1/2 then the errors grow and if r < 1/2
the errors fade away.
128 LECTURE 36. SOLUTION INSTABILITY FOR THE EXPLICIT METHOD
We can write a function program myexppmatrix that produces the matrix A in (36.3), for given
inputs m and r. Without using loops we can use the diag command to set up the matrix:
function A = myexpmatrix(m,r)
% produces the matrix for the explicit method for a parabolic equation
% Inputs: m -- the size of the matrix
% r -- the main parameter, ck/h^2
% Output: A -- an m by m matrix
u = (1-2*r)*ones(m,1);
v = r*ones(m-1,1);
A = diag(u) + diag(v,1) + diag(v,-1);
Test this using m = 6 and r = .4, .6. Check the eigenvalues and eigenvectors of the resulting
matirices:
> A = myexpmatrix(6,.6)
> [v e] = eig(A)
What is the “mode” represented by the eigenvector with the largest absolute eigenvalue? How is
that reflected in the unstable solutions?
Exercises
36.1 Let L = π, T = 20, f(x) = .1 sin(x), g
1
(t) = 0, g
2
(t) = 0, c = .5, and m = 20, as used in
the program myheat.m. What the value of n corresponding to r = 1/2? Try different n in
myheat.m to find precisely when the method works and when it fails. Is r = 1/2 the boundary
between failure and success? Hand in a plot of the last success and the first failure. Include
the values of n and r in each.
36.2 Write a script program that produces the graph in Figure 36.1 for m = 4. Your program
should: define r values from 0 to 1, for each r create the matrix A by calling myexppmatrix,
calculate the eigenvalues of A, find the max of the absolute values, and plot these numbers
versus r.
Lecture 37
Implicit Methods
The Implicit Difference Equations
By approximating u
xx
and u
t
at t
j+1
rather than t
j
, and using a backwards difference for u
t
, the
equation u
t
= cu
xx
is approximated by
u
i,j+1
−u
i,j
k
=
c
h
2
(u
i−1,j+1
−2u
i,j+1
+u
i+1,j+1
). (37.1)
Note that all the terms have index j + 1 except one and isolating this term leads to
u
i,j
= −ru
i−1,j+1
+ (1 + 2r)u
i,j+1
−ru
i+1,j+1
for 1 ≤ i ≤ m−1, (37.2)
where r = ck/h
2
as before.
Now we have u
j
given in terms of u
j+1
. This would seem like a problem, until you consider that
the relationship is linear. Using matrix notation, we have
u
j
= Bu
j+1
−rb
j+1
,
where b
j+1
represents the boundary condition. Thus to find u
j+1
, we need only solve the linear
system
Bu
j+1
= u
j
+rb
j+1
, (37.3)
where u
j
and b
j+1
are given and
B =
_
_
_
_
_
_
_
1 + 2r −r
−r 1 + 2r −r
.
.
.
.
.
.
.
.
.
−r 1 + 2r −r
−r 1 + 2r
_
_
_
_
_
_
_
. (37.4)
Using this scheme is called the implicit method since u
j+1
is defined implicitly.
Since we are solving (37.1), the most important quantity is the maximum absolute eigenvalue of
B
−1
, which is 1 divided by the smallest ew of B. Figure 37.1 shows the maximum absolute ew’s of
B
−1
as a function of r for various size matrices. Notice that this absolute maximum is always less
than 1. Thus errors are always diminished over time and so this method is always stable. For the
same reason it is also always as accurate as the individual steps.
Both this implicit method and the explicit method in the previous lecture make O(h
2
) error in
approximating u
xx
and O(k) error in approximating u
t
, so they have total error O(h
2
+ k). Thus
although the stability condition allows the implicit method to use arbitrarily large k, to maintain
accuracy we still need k ∼ h
2
.
129
130 LECTURE 37. IMPLICIT METHODS
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
0.4
0.5
0.6
0.7
0.8
0.9
1
r
EW
m = 2
m = 3
Figure 37.1: Maximum absolute eigenvalue EW as a function of r for the matrix B
−1
from
the implicit method for the heat equation calculated for matrices B of sizes m = 2 . . . 10.
Whenever EW < 1 the method is stable, i.e. it is always stable.
Crank-Nicholson Method
Now that we have two different methods for solving parabolic equation, it is natural to ask, “can
we improve by taking an average of the two methods?” The answer is yes.
We implement a weighted average of the two methods by considering an average of the approxima-
tions of u
xx
at j and j + 1. This leads to the equations
u
i,j+1
−u
i,j
k
=
λc
h
2
(u
i−1,j+1
−2u
i,j+1
+u
i+1,j+1
) +
(1 −λ)c
h
2
(u
i−1,j
−2u
i,j
+u
i+1,j
). (37.5)
The implicit method contained in these equations is called the Crank-Nicholson method. Gath-
ering terms yields the equations
−rλu
i−1,j+1
+ (1 + 2rλ)u
i,j+1
−rλu
i+1,j+1
= r(1 −λ)u
i−1,j
+ (1 −2r(1 −λ))u
i,j
+r(1 −λ)u
i+1,j
.
In matrix notation this is
B
λ
u
j+1
= A
λ
u
j
+rb
j+1
,
where
A
λ
=
_
_
_
_
_
_
_
1 −2(1 −λ)r (1 −λ)r
(1 −λ)r 1 −2(1 −λ)r (1 −λ)r
.
.
.
.
.
.
.
.
.
(1 −λ)r 1 −2(1 −λ)r (1 −λ)r
(1 −λ)r 1 −2(1 −λ)r
_
_
_
_
_
_
_
(37.6)
131
and
B
λ
=
_
_
_
_
_
_
_
1 + 2rλ −rλ
−rλ 1 + 2rλ −rλ
.
.
.
.
.
.
.
.
.
−rλ 1 + 2rλ −rλ
−rλ 1 + 2rλ
_
_
_
_
_
_
_
. (37.7)
In this equation u
j
and b
j+1
are known, Au
j
can be calculated directly, and then the equation is
solved for u
j+1
.
If we choose λ = 1/2, then we are in effect doing a central difference for u
t
, which has error O(k
2
).
Our total error is then O(h
2
+ k
2
). With a bit of work, we can show that the method is always
stable, and so we can use k ∼ h without a problem.
To get optimal accuracy with a weighted average, it is always necessary to use the right weights.
For the Crank-Nicholson method with a given r, we need to choose
λ =
r −1/6
2r
.
This choice will make the method have truncation error of order O(h
4
+ k
2
), which is really good
considering that the implicit and explicit methods each have truncation errors of order O(h
2
+ k).
Surprisingly, we can do even better if we also require
r =

5
10
≈ 0.22361,
and, consequently,
λ =
3 −

5
6
≈ 0.12732.
With these choices, the method has truncation error of order O(h
6
), which is absolutely amazing.
To appreciate the implications, suppose that we need to solve a problem with 4 significant digits. If
we use the explicit or implicit method alone then we will need h
2
≈ k ≈ 10
−4
. If L = 1 and T ≈ 1,
then we need m ≈ 100 and n ≈ 10, 000. Thus we would have a total of 1, 000, 000 grid points,
almost all on the interior. This is a lot.
Next suppose we solve the same problem using the optimal Crank-Nicholson method. We would
need h
6
≈ 10
−4
which would require us to take m ≈ 4.64, so we would take m = 5 and have h = 1/5.
For k we need k = (

5/10)h
2
/c. If c = 1, this gives k =

5/250 ≈ 0.0089442 so we would need
n ≈ 112 to get T ≈ 1. This gives us a total of 560 interior grid points, or, a factor of 1785 fewer
than the explicit or implicit method alone.
Exercises
37.1 Modify the program myexppmatrix from exercise 36.2 into a function program myimpmatrix
that produces the matrix B in (37.4) for given inputs m and r. Modify your script from
exercise 36.2 to use B
−1
for m = 4. It should produce a graph similar to that in Figure 37.1
for m = 4. Turn in the programs and the plot.
Lecture 38
Insulated Boundary Conditions
Insulation
In many of the previous sections we have considered fixed boundary conditions, i.e. u(0) = a,
u(L) = b. We implemented these simply by assigning u
j
0
= a and u
j
n
= b for all j.
We also considered variable boundary conditions, such as u(0, t) = g
1
(t). For example, we might
have u(0, t) = sin(t) which could represents periodic heating and cooling of the end at x = 0.
A third important type of boundary condition is called the insulated boundary condition. It is so
named because it mimics an insulator at the boundary. Physically, the effect of insulation is that
no heat flows across the boundary. This means that the temperature gradient is zero, which implies
that should require the mathematical boundary condition u

(L) = 0.
To use it in a program, we must replace u

(L) = 0 by a discrete version. Recall that in our discrete
equations we usually have L = x
n
. Recall from the section on numerical derivatives, that there are
three different ways to replace a derivative by a difference equation, left, right and central differences.
The three of them at x
n
would be:
u

(x
n
) ≈
u
n
−u
n−1
h

u
n+1
−u
n
h

u
n+1
−u
n−1
2h
.
If x
n
is the last node of our grid, then it is clear that we cannot use the right or central difference,
but are stuck with the first of these. Setting that expression to zero implies:
u
n
= u
n−1
.
This restriction can be easily implemented in a program simply by putting a statement u(n+1)=u(n)
inside the loop that updates values of the profile. However, since this method replaces u

(L) = 0 by
an expression that is only accurate to first order, it is not very accurate and is usually avoided.
Instead we want to use the most accurate version, the central difference. For that we should have
u

(L) = u

(x
n
) =
u
n+1
−u
n−1
2h
= 0.
or simply
u
n+1
= u
n−1
.
However, u
n+1
would represent u(x
n+1
) and x
n+1
would be L + h, which is outside the domain.
This however is not an obstacle in a program. We can simply extend the grid to one more node,
x
n+1
, and let u
n+1
always equal u
n−1
by copying u
n−1
into u
n+1
whenever u
n−1
changes. The
point x
n+1
is “fictional”, but a computer does not know the difference between fiction and reality!
This idea is carried out in the calculations of the next section and illustrated in Figure 38.1.
132
133
T
u
u u u e
u
u
u
e E
copy
x
n−2
x
n−1
x
n
x
n+1
Figure 38.1: Illustation of an insulated boundary condition using a fictional point x
n+1
with u
n+1
= u
n−1
.
A way to think of an insulated boundary that makes sense of the point L+h is to think of two bars
joined end to end, where you let the second bar be mirror image of the first bar. If you do this,
then no heat will flow across the joint, which is exactly the same effect as insulating.
Another practical way to implement an insulated boundary is to let the grid points straddle the
boundary. For example suppose we want to impose insulated boundary at the left end of a bar, i.e.
u

(0) = 0, then you could let the first two grid points be at x
0
= −h/2 and x
1
= h/2. Then you
can let
u
0
= u
1
.
This will again force the central difference at x = 0 to be 0.
Implementation in a linear equation by elimination
Consider the BVP:
u
xx
= −1, u(0) = 5, u

(1) = 0. (38.1)
This represents the steady state temperature of a bar with a uniformly applied heat source, with
one end held at a fixed temperature and the other end insulated.
If we use 4 equally spaced intervals, then
m = 4, L = 1, ⇒ h =
L
m
=
1
4
,
and
x
0
= 0, x
1
= .25, x
2
= .5, x
3
= .75, x
4
= 1, and x
5
= 1.25.
The point x
5
= 1.25 is outside the region and thus fictional. The boundary condition at x
0
= 0 is
implemented as
u
0
= 5.
For the insulated condition, we will require:
u
5
= u
3
.
This makes the central difference for u

(x
4
) be zero. We can write the differential equation as a
difference equation:
u
i−1
−2u
i
+u
i+1
2h
= −1
or
u
i−1
−2u
i
+u
i+1
= −0.5, i = 1, 2, 3, 4.
134 LECTURE 38. INSULATED BOUNDARY CONDITIONS
For i = 1, recalling that u
0
= 5 we have:
5 −2u
1
+u
2
= −.5 or −2u
1
+u
2
= −5.5.
For i = 2 and i = 3 we have:
u
1
−2u
2
+u
3
= −.5 and u
2
−2u
3
+u
4
= −.5.
For i = 4 we have:
u
3
−2u
4
+u
5
= −.5.
Note that we now have 5 unknowns in our problem: u
1
, . . . , u
5
. However, from the boundary
condition u
5
= u
3
and so we can eliminate u
5
from our equations and write:
2u
3
−2u
4
= −.5.
Summarizing, we can put the unknown quantities in a vector u = (u
1
, u
2
, u
3
, u
4
)

and write the
equations as a matrix equation Au = b where:
A
=
_
_
_
_
−2 1 0 0
1 −2 1 0
0 1 −2 1
0 0 2 −2
_
_
_
_
and b = (−5.5, −.5, −.5. −.5)

. Solve this system and plot the results:
> u = A\b
> u = [5 ; u]
> x = 0:.25:1
> plot(x,u,’d’)
The exact solution of this BVP is:
U(x) = 5 +x −.5x
2
.
Use hold on and plot this function on the same graph to compare. You should see that our
approximate results are not bad!
Insulated boundary conditions in time-dependent problems
To implement the insulated boundary condition in an explicit difference equation with time, we
need to copy values from inside the region to fictional points just outside the region. See Figure 38.2
for an illustration.
An example
The steady state temperature u(r) (given in polar coordinates) of a disk subjected to a radially
symmetric heat load g(r) and cooled by conduction to the rim of the disk and radiation to its
environment is determined by the boundary value problem:

2
u
∂r
2
+
1
r
∂u
∂r
= d(u
4
−u
4
b
) −g(r) with u(R) = u
R
and u

(0) = 0. (38.2)
Here u
b
is the (fixed) background temperature and u
R
is the (fixed) temperature at the rim of the
disk.
135
x x x x h
x x x x h
x x x x h
x x x x h
T T T
T T T
T T T

d
d
d s
d
d
d s
d
d
d s
d
d
d s
d
d
d s
d
d
d s
d
d
d s
d
d
d s
d
d
d s
‚
‚
‚
‚
t
j
t
j+1
x
m−1
x
m
x
m+1
Figure 38.2: Illustration of information flow for the explicit method with an insulated
boundary condition. The open circles are the fictional points and the curved arrows repre-
sent copying the value.
The class web site has a program myheatdisk.m that implements these equations for parameter
values R = 5, d = .1, u
R
= u
b
= 10 and g(r) = (r − 5)
2
. Notice that the equations have a
singularity (discontinuity) at r = 0. How does the program avoid this problem? How does the
program implement u
R
= 10 and u

(0) = 0? Run the program.
Exercises
38.1 Redo the calculations for the BVP (38.1) except do not include the fictional point x
5
. Instead,
let x
4
be the last point and impose the insulated boundary by requiring u
4
= u
3
. (Your system
of equations should be 3 3.) Compare this solution with the true solution and the better
approximation in the lecture. Illustrate this comparison on a single plot.
38.2 Modify the program myheat.m to have an insulated boundary at x = L (rather than u(L, t) =
g
2
(t)). You will need to change the domain to: x = 0:h:L+h and change the dimensions of
all the other objects to fit this domain. Run the program with L = 2π, T = 20, c = .5,
g
1
(t) = sin(t) and f(x) = −sin(x/4). Set m = 20 and experiment with n. Get a plot when
the program is stable. Turn in your program and plots.
38.3 Starting from (38.2), derive the finite difference equations for this example. Then solve for
u
i
and get the relaxation update formula used in program myheatdisk.m. In the program
change the radiation coefficient d to .01. How does this affect the final temperature at r = 0?
How does it affect the rate of convergence of the method? Turn in your derivation, the plots
for both d = 0.1 and d = 0.01, and answers to the questions.
Lecture 39
Finite Difference Method for Elliptic
PDEs
Examples of Elliptic PDEs
Elliptic PDE’s are equations with second derivatives in space and no time derivative. The most
important examples are Laplace’s equation
∆u = u
xx
+u
yy
+u
zz
= 0
and the Poisson equation
∆u = f(x, y, z).
These equations are used in a large variety of physical situations such as: steady state heat problems,
steady state chemical distributions, electrostatic potentials, elastic deformation and steady state
fluid flows.
For the sake of clarity we will only consider the two dimensional problem. A good model problem
in this dimension is the elastic deflection of a membrane. Suppose that a membrane such as a sheet
of rubber is stretched across a rectangular frame. If some of the edges of the frame are bent, or if
forces are applied to the sheet then it will deflect by an amount u(x, y) at each point (x, y). This u
will satify the boundary value problem:
u
xx
+u
yy
= f(x, y) for (x, y) in R,
u(x, y) = g(x, y) for (x, y) on ∂R,
(39.1)
where R is the rectangle, ∂R is the edge of the rectangle, f(x, y) is the force density (pressure)
applied at each point and g(x, y) is the deflection at the edge.
The Finite Difference Equations
Suppose the rectangle is described by
R = ¦a ≤ x ≤ b, c ≤ y ≤ d¦.
We will divide R in subrectangles. If we have m subdivisions in the x direction and n subdivisions
in the y direction, then the step size in the x and y directions respectively are
h =
b −a
m
and k =
d −c
n
.
136
137
We obtain the finite difference equations for (39.1) by replacing u
xx
and u
yy
by their central differ-
ences to obtain
u
i+1,j
−2u
ij
+u
i−1,j
h
2
+
u
i,j+1
−2u
ij
+u
i,j−1
k
2
= f(x
i
, y
j
) = f
ij
(39.2)
for 1 ≤ i ≤ m−1 and 1 ≤ j ≤ n −1. The boundary conditions are introduced by:
u
0,j
= g(a, y
j
), u
m,j
= g(b, y
j
), u
i,0
= g(x
i
, c), and u
i,n
= g(x
i
, d). (39.3)
Direct Solution of the Equations
Notice that since the edge values are prescribed, there are (m− 1) (n − 1) grid points where we
need to determine the solution. Note also that there are exactly (m − 1) (n − 1) equations in
(39.2). Finally, notice that the equations are all linear. Thus we could solve the equations exactly
using matrix methods. To do this we would first need to express the u
ij
’s as a vector, rather then
a matrix. To do this there is a standard procedure: let u be the column vector we get by placing
one column after another from the columns of (u
ij
). Thus we would list u
:,1
first then u
:,2
, etc..
Next we would need to write the matrix A that contains the coefficients of the equations (39.2) and
incorporate the boundary conditions in a vector b. Then we could solve an equation of the form
Au = b. (39.4)
Setting up and solving this equation is called the direct method.
An advantage of the direct method is that solving (39.4) can be done relatively quickly and accu-
rately. The drawback of the direct method is that one must set up u, A and b, which is confusing.
Further, the matrix A has dimensions (m−1)(n −1) (m−1)(n −1), which can be rather large.
Although A is large, many of its elements are zero. Such a matrix is called sparse and there are
special methods intended for efficiently working with sparse matrices.
Iterative Solution
A usually prefered alternative to the direct method described above is to solve the finite difference
equations iteratively. To do this, first solve (39.2) for u
ij
, which yields
u
ij
=
1
2(h
2
+k
2
)
_
k
2
(u
i+1,j
+u
i−1,j
) +h
2
(u
i,j+1
+u
i,j−1
) −h
2
k
2
f
ij
_
. (39.5)
This method is another example of a relaxation method. Using this formula, along with (39.3),
we can update u
ij
from its neighbors, just as we did in the relaxation method for the nonlinear
boundary value problem. If this method converges, then the result is an approximate solution.
The iterative solution is implemented in the program mypoisson.m on the class web site. Download
and read it. You will notice that maxit is set to 0. Thus the program will not do any iteration, but
will plot the initial guess. The initial guess in this case consists of the proper boundary values at
the edges, and zero everywhere in the interior. To see the solution evolve, gradually increase maxit.
138 LECTURE 39. FINITE DIFFERENCE METHOD FOR ELLIPTIC PDES
Exercises
39.1 Modify the program mypoisson.m (from the class web page) in the following ways:
• Change x = b to be an insulated boundary, i.e. u
x
(b, y) = 0.
• Change the force f(x, y) to a negative constant −p.
Experiment with various values of p and maxit. Obtain plots for small and large p. Tell
how many iterations are needed for convergence in each. For large p plot also a couple of
intermediate steps.
Lecture 40
Convection-Diffusion Equations
Exercises
40.1
139
Lecture 41
Finite Elements
Triangulating a Region
A disadvantage of finite difference methods is that they require a very regular grid, and thus a very
regular region, either rectangular or a regular part of a rectangle. Finite elements is a method that
works for any shape region because it is not built of a grid, but on triangulation of the region,
i.e. cutting the region up into triangles as we did in a previous lecture.
The following figure shows a triangularization of a region.
−2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5
−2.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
x
y
Figure 41.1: An annular region with a triangulation. Notice that the nodes and triangles
are very evenly spaced.
This figure was produced by the script program mywasher.m. Notice that the nodes are evenly
distributed. This is good for the finite element process where we will use it.
Open the program mywasher.m. This program defines a triangulation by defining the vertices in a
matrix V in which each row contains the x and y coordinates of a vertex. Notice that we list the
interior nodes first, then the boundary nodes.
Triangles are defined in the matrix T. Each row of T has three integer numbers indicating the indices
of the nodes that form a triangle. For instance the first row is 43 42 25, so T
1
is the triangle
140
141
t
h
h
h
h
h
h
h
h ¤
¤
¤
¤
¤
¤
¤
¤
t
f
f
f
f
f
f
f
f ¢
¢
¢
¢
¢
¢
¢
¢
t
e
e
e
e
e
e
e
e ¡
¡
¡
¡
¡
¡
¡
¡
t
e
e
e
e
e
e
e
e ¡
¡
¡
¡
¡
¡
¡
¡
t
f
f
f
f
f
f
f
f ¢
¢
¢
¢
¢
¢
¢
¢
t
Figure 41.2: The finite elements for the triangulation of a one dimensional object.
with vertices v
43
, v
42
and v
25
. The matrix T in this case was produced by the Matlab command
delaunay. The command produced more triangles than desire and the unwanted ones were deleted.
We a three dimensional plot of the region and triangles is produced by having the last line be
trimesh(T,x,y,z).
What is a finite element?
The finite element method is a mathematically complicated process. However, a finite element is
actually a very simple object. To each node v
j
we associate a function Φ
j
that has the properties
Φ
j
(v
j
) = 1 and Φ
j
(v
i
) = 0 for i ,= j. Between nodes, Φ
j
is a linear function. This function is the
finite element. (There are fancier types of finite elements that we will not discuss.)
If we consider one dimension, then a triangulation is just a subdivision into subintervals. In Fig-
ure 41.2 we show an uneven subdivision of an interval and the finite elements corresponding to each
node.
In two dimensions, Φ
j
is composed of triangular piece of planes. Thus Φ
j
is a function whose graph
is a pyramid with its peak over node v
j
.
What is a finite element solution?
A finite element (approximate) solution is a linear combination of the elements:
U(x) =
n

j=1
C
j
Φ
j
(x). (41.1)
Thus finding a finite element solution amounts to finding the best values for the constants ¦C
j
¦
n
j=1
.
In the program mywasher.m, the vector z contains the node values C
j
. These values give the height
at each node in the graph. For instance if we set all equal to 0 except one equal to 1, then the
function is a finite element. Do this for one boundary node, then for one interior node.
Notice that a sum of linear functions is a linear function. Thus the solution using linear elements is
a piecewise linear function. Also notice that if we denote the j-th vertex by v
j
, then
U(v
j
) = C
j
. (41.2)
Thus we see that the constants C
j
are just the values at the nodes.
142 LECTURE 41. FINITE ELEMENTS
Take the one-dimensional case. Since we know that the solution is linear on each subinterval,
knowing the values at the endpoints of the subintervals, i.e. the nodes, gives us complete knowledge
of the solution. Figure 41.3 could be a finite element solution since it is piecewise linear.

¤
¤
¤
¤
¤
¤
¤
t
d
d
d
t

t t
e
e
e
e
e
e
t
Figure 41.3: A possible finite element solution for a one dimensional object. Values are
assigned at each node and a linear interpolant is used in between.
In the two-dimensional case, the solution is linear on each triangle, and so again, if we know the
values ¦C
j
¦
n
j=1
at the nodes then we know everything.
Experiment with finite elements
By changing the values in c in the program we can produce different three dimensional shapes based
on the triangles. The point then of a finite element solution is to find the values at the nodes that
best approximate the true solution. This task can be subdivided into two parts: (1) assigning the
values at the boundary nodes and (2) assigning the values at the interior nodes.
Values at boundary nodes
Once a triangulation and a set of finite elements is set up, the next step is to incorporate the
boundary conditions of the problem. Suppose that we have fixed boundary conditions, i.e. of the
form
u(x) = g(x), for x ∈ ∂D,
where D is the object (domain) and ∂D is its boundary. Then the boundary condition directly
determines the values on the boundary nodes.
In particular, suppose that v

is a boundary node. Since Φ

(v

) = 1 and all the other elements are
zero at node v

, then to make
U(v

) =
n

j=1
C
j
Φ
j
(v

) = g(v

),
we must choose
C

= g(v

).
Thus the constants C
j
for the boundary nodes are set at exactly the value of the bound-
ary condition at the nodes.
143
Thus, if there are m interior nodes, then
C
j
= g(v
j
), for all m+ 1 ≤ j ≤ n.
In the program mywasher the first 32 vertices correspond to interior nodes and the last 32 correspond
to boundary nodes. By setting the last 32 values of z, we achieve the boundary conditions. We
could do this by adding the following commands to the program:
z(33:64) = .5;
or more elaborately we might use functions:
z(33:48) = x(33:48).^2 - .5*y(33:48).^2;
z(49:64) = .2*cos(y(49:64));
Exercises
41.1 Generate an interesting or useful 2-d object and a well-distributed triangulation of it.
• Plot the region.
• Plot one interior finite element.
• Plot one boundary finite element.
• Assign values to the boundary using a function (or functions) and plot the region with
the boundary values.
Turn in your code and the four plots.
Lecture 42
Determining Internal Node Values
As seen in the previous section, a finite element solution of a boundary value problem boils down to
finding the best values of the constants ¦C
j
¦
n
j=1
, which are the values of the solution at the nodes.
The interior nodes values are determined by variational principles. Variational principles usually
amount to minimizing internal energy. It is a physical principle that systems seek to be in a
state of minimal energy and this principle is used to find the internal node values.
Variational Principles
For the differential equations that describe many physical systems, the internal energy of the system
is an integral. For instance, for the steady state heat equation
u
xx
+u
yy
= g(x, y) (42.1)
the internal energy is the integral
I[u] =
__
R
u
2
x
+u
2
y
+ 2g(x, y)u(x, y) dA, (42.2)
where R is the region on which we are working. It can be shown that u(x, u) is a solution of (42.1)
if and only if it is minimizer of I[u] in (42.2).
The finite element solution
Recall that a finite element solution is a linear combination of finite element functions:
U(x, y) =
n

j=1
C
j
Φ
j
(x, y),
where n is the number of nodes. To obtain the values at the internal nodes, we will plug U(x, y)
into the energy integral and minimize. That is, we find the minimum of
I[U]
for all choices of ¦C
j
¦
m
j=1
, where m is the number of internal nodes. In this as with any other
minimization problem, the way to find a possible minimum is to differentiate the quantity with
respect to the variables and set the results to zero. In this case the free variables are ¦C
j
¦
m
j=1
. Thus
to find the minimizer we should try to solve
∂I[U]
∂C
j
= 0 for 1 ≤ j ≤ m. (42.3)
144
145
We call this set of equations the internal node equations. At this point we should ask whether
the internal node equations can be solved, and if so, is the solution actually a minimizer (and not
a maximizer). The following two facts answer these questions. These facts make the finite element
method practical:
• For most applications the internal node equations are linear.
• For most applications the internal node equations give a minimizer.
We can demonstrate the first fact using an example.
Application to the steady state heat equation
If we plug the candidate finite element solution U(x, y) into the energy integral for the heat equation
(42.2), we obtain
I[U] =
__
R
U
x
(x, y)
2
+U
y
(x, y)
2
+ 2g(x, y)U(x, y) dA. (42.4)
Differentiating with respect to C
j
we obtain the internal node equations
0 =
__
R
2U
x
∂U
x
∂C
j
+ 2U
y
∂U
y
∂C
j
+ 2g(x, y)
∂U
∂C
j
dA for 1 ≤ j ≤ m. (42.5)
Now we have several simplifications. First note that since
U(x, y) =
n

j=1
C
j
Φ
j
(x, y),
we have
∂U
∂C
j
= Φ
j
(x, y) .
Also note that
U
x
(x, y) =
n

j=1
C
j

∂x
Φ
j
(x, y),
and so
∂U
x
∂C
j
= (Φ
j
)
x
.
Similarly,
∂Uy
∂Cj
= (Φ
j
)
y
. The integral (42.5) then becomes
0 = 2
__
U
x

j
)
x
+U
y

j
)
y
+g(x, y)Φ
j
(x, y) dA for 1 ≤ j ≤ m.
Next we use the fact that the region R is subdivided into triangles ¦T
i
¦
p
i=1
and the functions in
question have different definitions on each triangle. The integral then is a sum of the integrals:
0 = 2
p

i=1
__
Ti
U
x

j
)
x
+U
y

j
)
y
+g(x, y)Φ
j
(x, y) dA for 1 ≤ j ≤ m.
Now note that the function Φ
j
(x, y) is linear on triangle T
i
and so
Φ
ij
(x, y) = Φ
j
[
Ti
(x, y) = a
ij
+b
ij
x +c
ij
y.
146 LECTURE 42. DETERMINING INTERNAL NODE VALUES
This gives us the following simplification:

ij
)
x
(x, y) = b
ij
and (Φ
ij
)
y
(x, y) = c
ij
.
Also, U
x
and U
y
restricted to T
i
have the form:
U
x
=
n

k=1
C
k
b
ik
and U
y
=
n

k=1
C
k
c
ik
.
The internal node equations then reduce to
0 =
p

i=1
__
Ti
_
n

k=1
C
k
b
ik
_
b
ij
+
_
n

k=1
C
k
c
ik
_
c
ij
+g(x, y)Φ
ij
(x, y) dA for 1 ≤ j ≤ m.
Now notice that (

n
k=1
C
k
b
ik
) b
ij
is just a constant on T
i
, and, thus, we have
__
Ti
_
n

k=1
C
k
b
ik
_
b
ij
+
_
n

k=1
C
k
c
ik
_
c
ij
=
__
n

k=1
C
k
b
ik
_
b
ij
+
_
n

k=1
C
k
c
ik
_
c
ij
_
A
i
,
where A
i
is just the area of T
i
. Finally, we apply the Three Corners rule to make an approximation
to the integral
__
Ti
g(x, y)Φ
ij
(x, y) dA.
Since Φ
ij
(x
k
, y
k
) = 0 if k ,= j and even Φ
ij
(x
j
, y
j
) = 0 if T
i
does not have a corner at (x
j
, y
j
), we
get the approximation
Φ
ij
(x
j
, y
j
)g(x
j
, y
j
)A
i
/3.
If T
i
does have a corner at (x
j
, y
j
) then Φ
ij
(x
j
, y
j
) = 1.
Summarizing, the internal node equations are:
0 =
p

i=1
__
n

k=1
C
k
b
ik
_
b
ij
+
_
n

k=1
C
k
c
ik
_
c
ij
+
1
3
g(x
j
, y
j

ij
(x
j
, y
j
)
_
A
i
for 1 ≤ j ≤ m.
While not pretty, these equations are in fact linear in the unknowns ¦C
j
¦.
Experiment
Download the program myfiniteelem.m from the class web site. This program produces a finite
element solution for the steady state heat equation without source term:
u
xx
+u
yy
= 0 .
To use it, you first need to set up the region and boundary values by running a script such as
mywasher or mywedge. Try different settings for the boundary values z. You will see that the
program works no matter what you choose.
Exercises
42.1 Study for the final!
Review of Part IV
Methods and Formulas
Initial Value Problems
Reduction to First order system:
For an n-th order equation that can be solved for the n-th derivative:
x
(n)
= f
_
t, x, ˙ x, ¨ x, . . . ,
d
n−1
x
dt
n−1
_
. (42.6)
Then use the standard change of variables:
y
1
= x
y
2
= ˙ x
.
.
.
y
n
= x
(n−1)
=
d
n−1
x
dt
n−1
.
(42.7)
Differentiating results in a first-order system:
˙ y
1
= ˙ x = y
2
˙ y
2
= ¨ x = y
3
.
.
.
˙ y
n
= x
(n)
= f(t, y
1
, y
2
, . . . , y
n
).
(42.8)
Euler’s method:
y
i+1
= y
i
+hf(t
i
, y
i
).
Modified (or Improved) Euler method:
k
1
= hf(t
i
, y
i
)
k
2
= hf(t
i
+h, y
i
+k
1
)
y
i+1
= y
i
+
1
2
(k
1
+k
2
)
Boundary Value Probems
Finite Differences:
Replace the Differential Equation by Difference Equations on a grid.
Review the lecture on Numerical Differentiation.
147
148 REVIEW OF PART IV
Explicit Method Finite Differences for Parabolic PDE (heat):
u
t

u
i,j+1
−u
ij
k
u
xx

u
i−1,j
−2u
ij
+u
i+1,j
h
2
(42.9)
leads to:
u
i,j+1
= ru
i−1,j
+ (1 −2r)u
i,j
+ru
i+1,j
,
where h = L/m, k = T/n, and r = ck/h
2
. The stability condition is r < 1/2.
Implicit Method Finite Differences for Parabolic PDE (heat):
u
t

u
i,j+1
−u
ij
k
u
xx

u
i−1,j+1
−2u
i,j+1
+u
i+1,j+1
h
2
(42.10)
leads to:
u
i,j
= −ru
i−1,j+1
+ (1 + 2r)u
i,j+1
−ru
i+1,j+1
,
which is always stable and has truncation error O(h
2
+k).
Crank-Nicholson Method Finite Differences for Parabolic PDE (heat):
−ru
i−1,j+1
+ 2(1 +r)u
i,j+1
−ru
i+1,j+1
= ru
i−1,j
+ 2(1 −r)u
i,j
+ru
i+1,j
,
which is always stable and has truncation error O(h
2
+k
2
) or better if r and λ are chosen optimally.
Finite Difference Method for Elliptic PDEs:
u
xx
+u
yy
= f(x, y) →
u
i+1,j
−2u
ij
+u
i−1,j
h
2
+
u
i,j+1
−2u
ij
+u
i,j−1
k
2
= f(x
i
, y
j
) = f
ij
,
Finite Elements:
Based on triangles instead of rectangles.
Can be used for irregularly shaped objects.
An element: Pyramid shaped function at a node.
A finite element solution is a linear combination of finite element functions:
U(x, y) =
n

j=1
C
j
Φ
j
(x, y),
where n is the number of nodes, and where U is an approximation of the true solution.
C
j
is the value of the solution at node j.
C
j
at the boundary nodes are given by boundary conditions.
C
j
at interior nodes are determined by variation principles.
The last step in determining C
j
’s is solving a linear system of equations.
Matlab
> [T Y ] = ode45(f,tspan,y0)
Initial value problem solver that uses the Runge-Kutta 45 method.
f must be a vector valued function, y0 is the initial vector.
RK 45 uses variable step size to control speed vs. accuracy.
Part V
Appendices
c _Copyright, Todd Young and Martin Mohlenkamp, Mathematics Department, Ohio University, 2007
Appendix A
Sample Exams
MATH344 Midterm Exam – Spring 04
1. For f(x) = x
2
− 5, do 2 iterations of Newton’s method, starting with x
0
= 2.0. What is the
relative error of x
2
? About how many more steps would be needed to make the error less
than 10
−16
?
2. Write a Matlab program to do n steps of the bisection method for a function f with starting
interval [a, b]. Let f, a, b and n be the inputs and the final x the output.
3. Given a data set represented by vectors x and y, describe how you would use Matlab to get
a Least Squares Approximation, Polynomial Interpolation and Spline Interpolation?
4. Given that the LU decomposition of A =
_
3 3
1 2
_
is LU =
_
1 0
1/3 1
_ _
3 3
0 1
_
, solve
Ax = b where b = (1, 2)

.
5. Suppose A
−1
=
_
−1 2
1 −1
_
. Using v
0
= (1, 1)

as the starting vector do 2 iterations of the
Inverse Power Method for A. What do the results mean?
6. Let A =
_
1 .5
2 1
_
. Find the LU factorization with pivoting.
7. What is the condition number of a matrix? How do you find it with Matlab? What are the
implications of the condition number when solving a linear system? What is the engineering
solution to a problem with a bad condition number?
8. Write a Matlab program to do n iterations of the Power Method. Let the matrix A and n
be inputs and let [e v] (the eigenvalue and eigenvector) be the outputs.
9. Write a Matlab program to that solves a linear system Ax = b using LU decomposition. Let
A, b and tol be the inputs and x the output. If the error (residual) is not less than tol, then
display a warning.
10. List your 10 least favorite Matlab commands.
X. What problem did Dr. Young totally botch in class?
150
151
MATH 344 – Midterm Exam – Winter 05
1. For f(x) = x
2
−5, do 2 iterations of the bisection method, starting with [a, b, ] = [2, 3]. What
is the relative error? About how many more steps would be needed to make the error less
than 10
−6
?
2. Write a Matlab program to do n steps of Newton’s method for a function f with starting
interval [a, b]. Let f, f

, x
0
and n be the inputs and the final x the output.
3. Suppose f(x) has been defined as an inline function. Give Matlab commands to plot it on
the interval [0, 10].
4. Write a function program which will find the roots of a function f on an interval [a, b].
5. Suppose A =
_
−1 2
1 −1
_
. Using v
0
= (1, 1)

as the starting vector do 2 iterations of the
Power Method for A. What do the results mean?
6. Let A =
_
−1 5
2 2
_
. Find the LU factorization with pivoting.
7. What is the condition number of a matrix? How do you find it with Matlab? What are the
implications of the condition number when solving a linear system? What is the engineering
solution to a problem with a bad condition number?
8. Write a Matlab program to do n iterations of the QR Method. Let the matrix A and n be
inputs and let e be the output.
9. Write a Matlab program to that solves a linear system Ax = b using LU decomposition.
Let A, b and tol be the inputs and x the output. If the error (residual) is not less than tol,
then display a warning.
10. Give the Matlab commands, or sequences of commands for solving a linear system Ax = b
in as many ways as you know. Which of these are the worst and best?
X. When using Newton’s method, how does one measure its effectiveness?
152 APPENDIX A. SAMPLE EXAMS
MATH 344 – Midterm Exam – Spring 2006
1. What 10 commands in Matlab are the least useful?
2. For f(x) = x
3
−6, do 2 iterations of Newton’s method, starting with x
0
= 2.
3. What are the main differences in the uses of: Polynomial Interpolation, Splines and Least
Squares Fitting?
4. Find the LU decomposition of A using pivoting if needed:
A =
_
3 −2
6 1
_
5. Write a Matlab function program that calculates the sum of the squares of the first n
integers.
6. What is the command in Matlab to produce the eigenvalues and eigenvectors of a matrix.
Which method does it use? What will be the form of the output?
7. What is the condition number of a matrix? How do you find it with Matlab? What are the
implications of the condition number when solving a linear system?
8. Find the eigenvalues and eigenvectors of the matrix:
A =
_
3 2
2 3
_
9. Write a Matlab function program that takes an input n, produces a random n n matrix
A and random vector
¯
b, solves A¯ x =
¯
b (using the built in command) and outputs the residual
(number).
10. Write a Matlab script program that will use Newton’s method to find a root of the system
of functions f
1
(x, y) = x
3
− y
2
+ 1 and f
2
(x, y) = y
3
+ x − 1 starting from the initial guess
(0, 0).
X. In this class, every problem leads to ... .
153
MATH344 Final Exam – Spring 2004
1. Estimate the integral
_
π
0
sin xdx using L
4
, R
4
and T
4
. Calculate the exact value and the
percentage errors of each of the approximations.
2. Write a Matlab program to do the midpoint method for integration. Let the inputs be the
function f, the endpoints a, b and the number of subintervals n.
3. Write a Matlab program to do n steps of the Euler method for a differential equation
˙ x = f (x, t), with x(a) = x
0
. Let the first line be:
function [t, x] = myeuler(f,x0,a,b,n).
4. What is the condition number of a matrix? How do you find it with MatLab? What are the
implications of the condition number when solving a linear system? What is the engineering
solution to a problem with a bad condition number?.
5. Write the equation θ” + aθ

+ b sin θ = c sin t as a linear system of first order equations and
set up the Euler method for the system.
6. Describe RK45. What is the command for it in Matlab?
7. When and why does the explicit finite difference method for the heat/diffusion equation
become unstable?
8. Derive the implicit finite difference equations for solving the heat/diffusion equation u
t
= cu
xx
.
9. Set up the finite difference equations for the BVP: u
xx
+ u
yy
= f(x, y), on the rectangle
0 ≤ x ≤ a and 0 ≤ y ≤ b, with u = 0 on all the boundaries. Explain how the difference
equations could be solved as a linear system.
10. Write a Matlab program to do Newton’s method with starting point x
0
that does n steps
or stops when a tolerance is reached. Include f

in the inputs. Let the first line be:
function x = mynewton(f,f1,x0,n,tol).
11. What is the geometric meaning of eigenvalue and eigenvector. What is the Matlab command
to find them, with correct syntax. Describe an application of ew’s and ev’s.
12. Give the Matlab command(s) for as many different ways as you know to solve a system of
linear equations. Rank them.
13. Discuss uses of Polynomial Interpolation, Splines and Least Squares Interpolations.
14. What is a finite element and a finite element solution?
15. Very generally, how are the boundary and interior values of the finite element solution ob-
tained.
X. When you have two different methods of approximating something, how can you get an even
better approximation?
154 APPENDIX A. SAMPLE EXAMS
MATH344 – Final Exam – Winter 2005
1. Estimate the integral
_
16
0

xdx using L
4
, R
4
, T
4
and S
4
. Calculate the exact value and the
percentage errors of each of the approximations.
2. Write a Matlab function program to do the trapezoid method for integration. Let the inputs
be the function f, the endpoints a, b and the number of subintervals n.
3. Write a Matlab program to do n steps of the modified Euler method for a differential
equation ˙ x = f (x, t), with x(a) = x
0
. Let the first line be:
function [t, x] = myeuler(f,x0,a,b,n).
4. Write a Matlab script program that calculates

i=0
.5
i
i
2
sin(i) by adding terms until the sum
stops changing. How do you know that it would stop?
5. Write the IVP: θ” + aθ

+ b sin θ = c sin t, θ(0) = π/4, θ

(0) = 0 as a system of first order
equations. Give all the Matlab commands needed to solve this IVP.
6. Describe RK45. What is the command for it in Matlab?
7. Explain why order matters in engineering problems.
8. Derive the explicit finite difference equations for solving the heat/diffusion equation u
t
= cu
xx
,
with boundary conditions, u(0, t) = a, u(L, t) = b, and u(x, 0) = f(x).
9. Set up the finite difference equations for the BVP: u
rr
+
1
r
u
r
= f(r), on the interal 0 ≤ r ≤ R,
with u(R) = 0 and u
r
(0) = 0. Explain how to avoid the problem at r = 0.
10. Write a Matlab program to do Newton’s method with starting point x
0
that does n steps
or stops when a tolerance is reached. Include f

in the inputs. Let the first line be:
function x = mynewton(f,f1,x0,n,tol).
11. Write a Matlab program to do n iterations of the inverse power method. Let the matrix A
and n be the inputs and [e v] be the output. What is the meaning of e and v?
12. Describe methods for approximating double integrals.
13. What are: Polynomial Interpolations, Splines and Least Squares Interpolations. How do you
get them from Matlab?
14. Discuss: triangulation, finite element and finite element solution.
15. How are the boundary and interior values of the finite element solution obtained.
X. When I can’t get my Matlab program to work, I ... .
155
MATH344 – Final Exam – Spring 2006
1. Approximate the integral
_
π
0
sin xdx using M
4
and S
4
. Which do you expect to be more
accurate?
2. Write a Matlab function program to do the Trapezoid Rule for integration of data. Let the
inputs be vectors x and y, where it is assumed that y is a function of x and x is not necessarily
evenly spaced.
3. Describe and give formulas for 2 methods to approximate double integrals based on triangles.
4. Explain how to incorporate an insulated boundary in a finite difference method.
5. Set up the finite difference equations for the BVP: u
rr
+
1
r
u
r
= f(r), on the interval 0 ≤ r ≤ R,
with u(R) = 4 and u
r
(0) = 0. Explain how to avoid the problem at r = 0.
6. Do 2 steps of Newton’s method to solve the equation f(x) = x
2
−5 = 0, with starting point
x
0
= 2. Find the percentage errors of x
0
, x
1
and x
2
.
7. Write a Matlab script program that will use Newton’s method to find a root of the system
of functions f
1
(x, y) = x
3
+ y − 1 and f
2
(x, y) = y
3
− x + 1 starting from the initial guess
(.5, .5).
8. Write a Matlab function program to do n steps of the Modified Euler method for a differential
equation ˙ x = f (t, x), on the time interval [a, b], with x(a) = x
0
.
9. Write the IVP: θ
′′
+.5θ

+sin θ = sin 2t, θ(0) = 1, θ

(0) = 0 as a system of first order equations.
Give all the Matlab commands needed to solve this IVP on the interval 0 ≤ t ≤ 10.
10. What is variable step size? How is it implemented RK45?
11. What are main differences between the Finite Difference Method and Finite Elements Method?
12. If U(x) =
n

j=1
C
j
Φ
j
(¯ x) is a finite element solution, what is the meaning of C
j
? What is the
final step of finding a finite element solution?
13. Write a Matlab program to do n iterations of the power method. Let the matrix A and n
be the inputs and [e v] be the output. What is the meaning of e and v?
14. Let A =
_
1 2
2 1
_
. Find the LU factorization with pivoting.
15. What is a polynomial interpolation? How do you get one in Matlab?
X. When you have two different methods to make an approximation, how can you get a poten-
tially more accurate method?
Appendix B
Glossary of Matlab Commands
Mathematical Operations
+ Addition. Type help plus for information.
- Subtraction. Type help minus for information.
* Scalar or matrix multiplication. Type help mtimes for information.
/ Scalar or right matrix division. Type help slash for information.
For matrices, the command A/B is equivalent to A*inv(B).
^ Scalar or matrix powers. Type help mpower for information.
.* Element by element multiplication. Type help times for information.
.^ Element by element exponentiation. Type help power for information.
./ Element by element division.
Built-in Mathematical Constants
eps Machine epsilon, i.e. approximately the computer’s floating point roundoff error.
i

−1.
Inf ∞.
NaN Not a number. Indicates an invalid operation such as 0/0.
pi π = 3.14159 . . ..
Built-in Mathematical Functions
abs(x) Absolute value[x[.
acos(x) Inverse cosine arccos x.
asin(x) Inverse sine arcsin x.
156
157
atan(x) Inverse tangent arctan x.
cos(x) Cosine cos x.
cosh(x) Hyperbolic cosine cosh x.
cot(x) Cotangent cot x.
exp(x) Exponential function e
x
= exp x.
log(x) Natural logarithm ln x = log
e
x.
sec(x) Secant sec x.
sin(x) Sine sin x.
sinh(x) Hyperbolic sine sinh x.
sqrt(x) Square root

x.
tan(x) Tangent tan x.
tanh(x) Hyperbolic tangent tanh x.
max Computes maximum of the rows of a matrix.
mean Computes the average of the rows of a matrix.
min Computes the minimum of the rows of a matrix.
Built-in Numerical Mathematical Operations
fzero Tries to find a zero of the specified function near a starting point or on a
specified interval.
inline Define a function in the command window.
ode113 Numerical multiple step ODE solver.
ode45 Runga-Kutta 45 numerical ODE solver.
quad Numerical integration using an adaptive Simpson’s rule.
dblquad Double integration.
triplequad Triple integration.
Built-in Symbolic Mathematical Operations
collect Collects powers of the specified variable is a given symbolic expression.
compose Composition of symbolic functions.
diff Symbolic differentiation.
double Displays double-precision representation of a symbolic expression.
dsolve Symbolic ODE solver.
expand Expands an algebraic expression.
factor Factor a polynomial.
int Symbolic integration; either definite or indefinite.
158 APPENDIX B. GLOSSARY OF MATLAB COMMANDS
limit Finds two-sided limit, if it exists.
pretty Displays a symbolic expression in a nice format.
simple Simplifies a symbolic expression.
subs Substitutes for parts a a symbolic expression.
sym or syms Create symbolic variables.
symsum Performs a symbolic summation, possibly with infinitely many entries.
taylor Gives a Taylor polynomial approximation of a given order at a specified point.
Graphics Commands
contour Plots level curves of a function of two variables.
contourf Filled contour plot.
ezcontour Easy contour plot.
loglog Creates a log-log plot.
mesh Draws a mesh surface.
meshgrid Creates arrays that can be used as inputs in graphics commands
such as contour, mesh, quiver, and surf.
ezmesh Easy mesh surface plot.
plot Plots data vectors.
ezplot Easy plot for symbolic functions.
plot3 Plots curves in 3-D.
polar Plots in polar coordinates.
quiver Plots a vector field.
semilogy Semilog plot, with logarithmic scale along the vertical direction.
surf Solid surface plot.
trimesh Plot based on a triangulation trisurf Surface plot based on a triangulation
Special Matlab Commands
: Range operator, used for defining vectors and in loops. Type help colon
for information.
; Suppresses output. Also separates rows of a matrix.
= Assigns the variable on the left hand side the value of the right hand side.
ans The value of the most recent unassigned.
cd Change directory.
clear Clears all values and definitions of variables and functions. You may also use to
159
clear only specified variables.
diary Writes a transcript of a Matlab session to a file.
dir Lists the contents in the current working directory. Same as ls.
help
inline Define an inline function.
format Specifies output format, e.g. > format long.
load Load variables from a file.
save Saves workspace variables to a file.
Matlab Programming
== Is equal?
~= Is not equal?
< Less than?
> Greater than?
<= Less than or equal?
break Breaks out of a for or while loop.
end Terminates an if, for or while statement.
else Alternative in an if statement.
error Displays and error message and ends execution of a program.
for Repeats a block of commands a specified number of times.
function First word in a function program.
if Checks a condition before executing a block of statements.
return Terminates execution of a program.
warning Displays a warning message.
while Repeats a block of commands as long as a condition is true.
Commands for Matrices and Linear Algebra
Matrix arithmetic:
> A = [ 1 3 -2 5 ; -1 -1 5 4 ; 0 1 -9 0] . . . . . . . . . Manually enter a matrix.
> u = [ 1 2 3 4]’
> A*u
> B = [3 2 1; 7 6 5; 4 3 2]
> B*A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . multiply B times A.
160 APPENDIX B. GLOSSARY OF MATLAB COMMANDS
> 2*A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . multiply a matrix by a scalar.
> A + A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . add matrices.
> A + 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . add a number to every entry of a matrix.
> B.*B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . component-wise multiplication.
> B.^3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . component-wise exponentiation.
Special matrices:
> I = eye(3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . identity matrix
> D = ones(5,5)
> O = zeros(10,10)
> C = rand(5,5) . . . . . . . . . . . . . . . . . . . . random matrix with uniform distribution in [0, 1].
> C = randn(5,5) . . . . . . . . . . . . . . . . . . . . . . . . . . . random matrix with normal distribution.
> hilb(6)
> pascal(5)
General matrix commands:
> size(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . gives the dimensions (mn) of A.
> norm(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . gives the norm of the matrix.
> det(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the determinant of the matrix.
> max(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the maximum of each row.
> min(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the minimum in each row.
> sum(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . sums each row.
> mean(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the average of each row.
> diag(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . just the diagonal elements.
> inv(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . inverse of the matrix.
Matrix decompositions:
> [L U P] = lu(C)
> [Q R] = qr(C)
> [U S V] = svm(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . singular value decomposition.
Bibliography
[1] Ayyup and McCuen, 1996.
[2] E. Johnston, J. Mathews, Calculus, Addison-Wesley, 2001
161
GNU Free Documentation License
Version 1.3, 3 November 2008
Copyright c _ 2000, 2001, 2002, 2007, 2008 Free Software Foundation, Inc.
<http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies of this license document,
but changing it is not allowed.
Preamble
The purpose of this License is to make a manual, textbook, or other functional and useful
document “free” in the sense of freedom: to assure everyone the effective freedom to copy
and redistribute it, with or without modifying it, either commercially or noncommercially.
Secondarily, this License preserves for the author and publisher a way to get credit for their
work, while not being considered responsible for modifications made by others.
This License is a kind of “copyleft”, which means that derivative works of the document
must themselves be free in the same sense. It complements the GNU General Public License,
which is a copyleft license designed for free software.
We have designed this License in order to use it for manuals for free software, because free
software needs free documentation: a free program should come with manuals providing the
same freedoms that the software does. But this License is not limited to software manuals;
it can be used for any textual work, regardless of subject matter or whether it is published
as a printed book. We recommend this License principally for works whose purpose is
instruction or reference.
1. APPLICABILITY AND DEFINITIONS
This License applies to any manual or other work, in any medium, that contains a notice
placed by the copyright holder saying it can be distributed under the terms of this License.
162
GNU Free Documentation License 163
Such a notice grants a world-wide, royalty-free license, unlimited in duration, to use that
work under the conditions stated herein. The “Document”, below, refers to any such
manual or work. Any member of the public is a licensee, and is addressed as “you”. You
accept the license if you copy, modify or distribute the work in a way requiring permission
under copyright law.
A “Modified Version” of the Document means any work containing the Document or a
portion of it, either copied verbatim, or with modifications and/or translated into another
language.
A “Secondary Section” is a named appendix or a front-matter section of the Document
that deals exclusively with the relationship of the publishers or authors of the Document
to the Document’s overall subject (or to related matters) and contains nothing that could
fall directly within that overall subject. (Thus, if the Document is in part a textbook of
mathematics, a Secondary Section may not explain any mathematics.) The relationship
could be a matter of historical connection with the subject or with related matters, or of
legal, commercial, philosophical, ethical or political position regarding them.
The “Invariant Sections” are certain Secondary Sections whose titles are designated, as
being those of Invariant Sections, in the notice that says that the Document is released
under this License. If a section does not fit the above definition of Secondary then it is not
allowed to be designated as Invariant. The Document may contain zero Invariant Sections.
If the Document does not identify any Invariant Sections then there are none.
The “Cover Texts” are certain short passages of text that are listed, as Front-Cover Texts
or Back-Cover Texts, in the notice that says that the Document is released under this
License. A Front-Cover Text may be at most 5 words, and a Back-Cover Text may be at
most 25 words.
A “Transparent” copy of the Document means a machine-readable copy, represented in
a format whose specification is available to the general public, that is suitable for revising
the document straightforwardly with generic text editors or (for images composed of pixels)
generic paint programs or (for drawings) some widely available drawing editor, and that
is suitable for input to text formatters or for automatic translation to a variety of formats
suitable for input to text formatters. A copy made in an otherwise Transparent file format
whose markup, or absence of markup, has been arranged to thwart or discourage subsequent
modification by readers is not Transparent. An image format is not Transparent if used for
any substantial amount of text. A copy that is not “Transparent” is called “Opaque”.
Examples of suitable formats for Transparent copies include plain ASCII without markup,
Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD,
164 GNU Free Documentation License
and standard-conforming simple HTML, PostScript or PDF designed for human modifica-
tion. Examples of transparent image formats include PNG, XCF and JPG. Opaque formats
include proprietary formats that can be read and edited only by proprietary word proces-
sors, SGML or XML for which the DTD and/or processing tools are not generally available,
and the machine-generated HTML, PostScript or PDF produced by some word processors
for output purposes only.
The “Title Page” means, for a printed book, the title page itself, plus such following pages
as are needed to hold, legibly, the material this License requires to appear in the title page.
For works in formats which do not have any title page as such, “Title Page” means the
text near the most prominent appearance of the work’s title, preceding the beginning of
the body of the text.
The “publisher” means any person or entity that distributes copies of the Document to
the public.
A section “Entitled XYZ” means a named subunit of the Document whose title ei-
ther is precisely XYZ or contains XYZ in parentheses following text that translates XYZ
in another language. (Here XYZ stands for a specific section name mentioned below,
such as “Acknowledgements”, “Dedications”, “Endorsements”, or “History”.) To
“Preserve the Title” of such a section when you modify the Document means that it
remains a section “Entitled XYZ” according to this definition.
The Document may include Warranty Disclaimers next to the notice which states that
this License applies to the Document. These Warranty Disclaimers are considered to be
included by reference in this License, but only as regards disclaiming warranties: any other
implication that these Warranty Disclaimers may have is void and has no effect on the
meaning of this License.
2. VERBATIM COPYING
You may copy and distribute the Document in any medium, either commercially or noncom-
mercially, provided that this License, the copyright notices, and the license notice saying
this License applies to the Document are reproduced in all copies, and that you add no
other conditions whatsoever to those of this License. You may not use technical measures
to obstruct or control the reading or further copying of the copies you make or distribute.
However, you may accept compensation in exchange for copies. If you distribute a large
enough number of copies you must also follow the conditions in section 3.
GNU Free Documentation License 165
You may also lend copies, under the same conditions stated above, and you may publicly
display copies.
3. COPYING IN QUANTITY
If you publish printed copies (or copies in media that commonly have printed covers) of the
Document, numbering more than 100, and the Document’s license notice requires Cover
Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover
Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both
covers must also clearly and legibly identify you as the publisher of these copies. The front
cover must present the full title with all words of the title equally prominent and visible.
You may add other material on the covers in addition. Copying with changes limited to
the covers, as long as they preserve the title of the Document and satisfy these conditions,
can be treated as verbatim copying in other respects.
If the required texts for either cover are too voluminous to fit legibly, you should put the
first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto
adjacent pages.
If you publish or distribute Opaque copies of the Document numbering more than 100, you
must either include a machine-readable Transparent copy along with each Opaque copy,
or state in or with each Opaque copy a computer-network location from which the general
network-using public has access to download using public-standard network protocols a
complete Transparent copy of the Document, free of added material. If you use the latter
option, you must take reasonably prudent steps, when you begin distribution of Opaque
copies in quantity, to ensure that this Transparent copy will remain thus accessible at the
stated location until at least one year after the last time you distribute an Opaque copy
(directly or through your agents or retailers) of that edition to the public.
It is requested, but not required, that you contact the authors of the Document well before
redistributing any large number of copies, to give them a chance to provide you with an
updated version of the Document.
4. MODIFICATIONS
You may copy and distribute a Modified Version of the Document under the conditions
of sections 2 and 3 above, provided that you release the Modified Version under precisely
this License, with the Modified Version filling the role of the Document, thus licensing
166 GNU Free Documentation License
distribution and modification of the Modified Version to whoever possesses a copy of it. In
addition, you must do these things in the Modified Version:
A. Use in the Title Page (and on the covers, if any) a title distinct from that of the
Document, and from those of previous versions (which should, if there were any, be
listed in the History section of the Document). You may use the same title as a
previous version if the original publisher of that version gives permission.
B. List on the Title Page, as authors, one or more persons or entities responsible for
authorship of the modifications in the Modified Version, together with at least five
of the principal authors of the Document (all of its principal authors, if it has fewer
than five), unless they release you from this requirement.
C. State on the Title page the name of the publisher of the Modified Version, as the
publisher.
D. Preserve all the copyright notices of the Document.
E. Add an appropriate copyright notice for your modifications adjacent to the other
copyright notices.
F. Include, immediately after the copyright notices, a license notice giving the public
permission to use the Modified Version under the terms of this License, in the form
shown in the Addendum below.
G. Preserve in that license notice the full lists of Invariant Sections and required Cover
Texts given in the Document’s license notice.
H. Include an unaltered copy of this License.
I. Preserve the section Entitled “History”, Preserve its Title, and add to it an item
stating at least the title, year, new authors, and publisher of the Modified Version as
given on the Title Page. If there is no section Entitled “History” in the Document,
create one stating the title, year, authors, and publisher of the Document as given
on its Title Page, then add an item describing the Modified Version as stated in the
previous sentence.
J. Preserve the network location, if any, given in the Document for public access to
a Transparent copy of the Document, and likewise the network locations given in
the Document for previous versions it was based on. These may be placed in the
“History” section. You may omit a network location for a work that was published at
least four years before the Document itself, or if the original publisher of the version
it refers to gives permission.
GNU Free Documentation License 167
K. For any section Entitled “Acknowledgements” or “Dedications”, Preserve the Title
of the section, and preserve in the section all the substance and tone of each of the
contributor acknowledgements and/or dedications given therein.
L. Preserve all the Invariant Sections of the Document, unaltered in their text and in
their titles. Section numbers or the equivalent are not considered part of the section
titles.
M. Delete any section Entitled “Endorsements”. Such a section may not be included in
the Modified Version.
N. Do not retitle any existing section to be Entitled “Endorsements” or to conflict in
title with any Invariant Section.
O. Preserve any Warranty Disclaimers.
If the Modified Version includes new front-matter sections or appendices that qualify as
Secondary Sections and contain no material copied from the Document, you may at your
option designate some or all of these sections as invariant. To do this, add their titles to
the list of Invariant Sections in the Modified Version’s license notice. These titles must be
distinct from any other section titles.
You may add a section Entitled “Endorsements”, provided it contains nothing but endorse-
ments of your Modified Version by various parties—for example, statements of peer review
or that the text has been approved by an organization as the authoritative definition of a
standard.
You may add a passage of up to five words as a Front-Cover Text, and a passage of up
to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified
Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added
by (or through arrangements made by) any one entity. If the Document already includes
a cover text for the same cover, previously added by you or by arrangement made by the
same entity you are acting on behalf of, you may not add another; but you may replace the
old one, on explicit permission from the previous publisher that added the old one.
The author(s) and publisher(s) of the Document do not by this License give permission to
use their names for publicity for or to assert or imply endorsement of any Modified Version.
5. COMBINING DOCUMENTS
You may combine the Document with other documents released under this License, under
the terms defined in section 4 above for modified versions, provided that you include in the
168 GNU Free Documentation License
combination all of the Invariant Sections of all of the original documents, unmodified, and
list them all as Invariant Sections of your combined work in its license notice, and that you
preserve all their Warranty Disclaimers.
The combined work need only contain one copy of this License, and multiple identical
Invariant Sections may be replaced with a single copy. If there are multiple Invariant
Sections with the same name but different contents, make the title of each such section
unique by adding at the end of it, in parentheses, the name of the original author or
publisher of that section if known, or else a unique number. Make the same adjustment to
the section titles in the list of Invariant Sections in the license notice of the combined work.
In the combination, you must combine any sections Entitled “History” in the various original
documents, forming one section Entitled “History”; likewise combine any sections Entitled
“Acknowledgements”, and any sections Entitled “Dedications”. You must delete all sections
Entitled “Endorsements”.
6. COLLECTIONS OF DOCUMENTS
You may make a collection consisting of the Document and other documents released under
this License, and replace the individual copies of this License in the various documents with
a single copy that is included in the collection, provided that you follow the rules of this
License for verbatim copying of each of the documents in all other respects.
You may extract a single document from such a collection, and distribute it individually
under this License, provided you insert a copy of this License into the extracted document,
and follow this License in all other respects regarding verbatim copying of that document.
7. AGGREGATION WITH INDEPENDENT WORKS
A compilation of the Document or its derivatives with other separate and independent
documents or works, in or on a volume of a storage or distribution medium, is called an
“aggregate” if the copyright resulting from the compilation is not used to limit the legal
rights of the compilation’s users beyond what the individual works permit. When the
Document is included in an aggregate, this License does not apply to the other works in
the aggregate which are not themselves derivative works of the Document.
If the Cover Text requirement of section 3 is applicable to these copies of the Document,
then if the Document is less than one half of the entire aggregate, the Document’s Cover
Texts may be placed on covers that bracket the Document within the aggregate, or the
GNU Free Documentation License 169
electronic equivalent of covers if the Document is in electronic form. Otherwise they must
appear on printed covers that bracket the whole aggregate.
8. TRANSLATION
Translation is considered a kind of modification, so you may distribute translations of the
Document under the terms of section 4. Replacing Invariant Sections with translations
requires special permission from their copyright holders, but you may include translations
of some or all Invariant Sections in addition to the original versions of these Invariant
Sections. You may include a translation of this License, and all the license notices in
the Document, and any Warranty Disclaimers, provided that you also include the original
English version of this License and the original versions of those notices and disclaimers.
In case of a disagreement between the translation and the original version of this License
or a notice or disclaimer, the original version will prevail.
If a section in the Document is Entitled “Acknowledgements”, “Dedications”, or “History”,
the requirement (section 4) to Preserve its Title (section 1) will typically require changing
the actual title.
9. TERMINATION
You may not copy, modify, sublicense, or distribute the Document except as expressly pro-
vided under this License. Any attempt otherwise to copy, modify, sublicense, or distribute
it is void, and will automatically terminate your rights under this License.
However, if you cease all violation of this License, then your license from a particular copy-
right holder is reinstated (a) provisionally, unless and until the copyright holder explicitly
and finally terminates your license, and (b) permanently, if the copyright holder fails to
notify you of the violation by some reasonable means prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is reinstated permanently if the
copyright holder notifies you of the violation by some reasonable means, this is the first
time you have received notice of violation of this License (for any work) from that copyright
holder, and you cure the violation prior to 30 days after your receipt of the notice.
Termination of your rights under this section does not terminate the licenses of parties
who have received copies or rights from you under this License. If your rights have been
terminated and not permanently reinstated, receipt of a copy of some or all of the same
material does not give you any rights to use it.
170 GNU Free Documentation License
10. FUTURE REVISIONS OF THIS LICENSE
The Free Software Foundation may publish new, revised versions of the GNU Free Doc-
umentation License from time to time. Such new versions will be similar in spirit to
the present version, but may differ in detail to address new problems or concerns. See
http://www.gnu.org/copyleft/.
Each version of the License is given a distinguishing version number. If the Document
specifies that a particular numbered version of this License “or any later version” applies
to it, you have the option of following the terms and conditions either of that specified
version or of any later version that has been published (not as a draft) by the Free Software
Foundation. If the Document does not specify a version number of this License, you may
choose any version ever published (not as a draft) by the Free Software Foundation. If the
Document specifies that a proxy can decide which future versions of this License can be
used, that proxy’s public statement of acceptance of a version permanently authorizes you
to choose that version for the Document.
11. RELICENSING
“Massive Multiauthor Collaboration Site” (or “MMC Site”) means any World Wide Web
server that publishes copyrightable works and also provides prominent facilities for anybody
to edit those works. A public wiki that anybody can edit is an example of such a server.
A “Massive Multiauthor Collaboration” (or “MMC”) contained in the site means any set
of copyrightable works thus published on the MMC site.
“CC-BY-SA” means the Creative Commons Attribution-Share Alike 3.0 license published
by Creative Commons Corporation, a not-for-profit corporation with a principal place of
business in San Francisco, California, as well as future copyleft versions of that license
published by that same organization.
“Incorporate” means to publish or republish a Document, in whole or in part, as part of
another Document.
An MMC is “eligible for relicensing” if it is licensed under this License, and if all works that
were first published under this License somewhere other than this MMC, and subsequently
incorporated in whole or in part into the MMC, (1) had no cover texts or invariant sections,
and (2) were thus incorporated prior to November 1, 2008.
The operator of an MMC Site may republish an MMC contained in the site under CC-BY-
SA on the same site at any time before August 1, 2009, provided the MMC is eligible for
GNU Free Documentation License 171
relicensing.
ADDENDUM: How to use this License for your documents
To use this License in a document you have written, include a copy of the License in the
document and put the following copyright and license notices just after the title page:
Copyright c _ YEAR YOUR NAME. Permission is granted to copy, distribute
and/or modify this document under the terms of the GNU Free Documenta-
tion License, Version 1.3 or any later version published by the Free Software
Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-
Cover Texts. A copy of the license is included in the section entitled “GNU
Free Documentation License”.
If you have Invariant Sections, Front-Cover Texts and Back-Cover Texts, replace the “with
. . . Texts.” line with this:
with the Invariant Sections being LIST THEIR TITLES, with the Front-Cover
Texts being LIST, and with the Back-Cover Texts being LIST.
If you have Invariant Sections without Cover Texts, or some other combination of the three,
merge those two alternatives to suit the situation.
If your document contains nontrivial examples of program code, we recommend releasing
these examples in parallel under your choice of free software license, such as the GNU
General Public License, to permit their use in free software.

ii

Copyright c 2008, 2009, 2011 Todd Young and Martin J. Mohlenkamp. Original edition 2004, by Todd Young. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no BackCover Texts. A copy of the license is included in the section entitled “GNU Free Documentation License”.

Preface
These notes were developed by the first author in the process of teaching a course on applied numerical methods for Civil Engineering majors during 2002-2004 and was modified to include Mechanical Engineering in 2005. The materials have been periodically updated since then and underwent a major revision by the second author in 2006-2007. The main goals of these lectures are to introduce concepts of numerical methods and introduce Matlab in an Engineering framework. By this we do not mean that every problem is a “real life” engineering application, but more that the engineering way of thinking is emphasized throughout the discussion. The philosophy of this book was formed over the course of many years. My father was a Civil Engineer and surveyor, and he introduced me to engineering ideas from an early age. At the University of Kentucky I took most of the basic Engineering courses while getting a Bachelor’s degree in Mathematics. Immediately afterward I completed a M.S. degree in Engineering Mechanics at Kentucky. While working on my Ph.D. in Mathematics at Georgia Tech I taught all of the introductory math courses for engineers. During my education, I observed that incorporation of computation in coursework had been extremely unfocused and poor. For instance during my college career I had to learn 8 different programming and markup languages on 4 different platforms plus numerous other software applications. There was almost no technical help provided in the courses and I wasted innumerable hours figuring out software on my own. A typical, but useless, inclusion of software has been (and still is in most calculus books) to set up a difficult ‘applied’ problem and then add the line “write a program to solve” or “use a computer algebra system to solve”. At Ohio University we have tried to take a much more disciplined and focused approach. The Russ College of Engineering and Technology decided that Matlab should be the primary computational software for undergraduates. At about the same time members of the Mathematics Department proposed an 1804 project to bring Matlab into the calculus sequence and provide access to the program at nearly all computers on campus, including in the dorm rooms. The stated goal of this project was to make Matlab the universal language for computation on campus. That project was approved and implemented in the 2001-2002 academic year. In these lecture notes, instruction on using Matlab is dispersed through the material on numerical methods. In these lectures details about how to use Matlab are detailed (but not verbose) and explicit. To teach programming, students are usually given examples of working programs and are asked to make modifications.

iii

Throughout the text Matlab commands are preceded by the symbol >. but could be used in a lecture format with students doing computer exercises afterward. 2011 .iv PREFACE The lectures are designed to be used in a computer classroom. Todd Young. Programs are surrounded by a box. which is the prompt in the command window. The lectures are divided into four Parts with a summary provided at the end of each Part. August 27.

An Application of Eigenvectors: Vibrational Modes Lecture 16. Matrices and Matrix Operations in Matlab Lecture 9. The QR Method* . Eigenvalues and Eigenvectors Lecture 15. LU Decomposition Lecture 13. The Bisection Method and Locating Roots Lecture 6. Condition Numbers and Pivoting Lecture 12. Some Facts About Linear Systems Lecture 11. Controlling Error and Conditional Statements Lecture 5. Nonlinear Systems .Contents Preface iii I Matlab and Solving Equations 1 2 5 8 11 14 17 20 23 Lecture 1. Secant Methods* Lecture 7. Vectors. Matlab Programs Lecture 3. Numerical Methods for Eigenvalues Lecture 17. Newton’s Method and Loops Lecture 4. and Plots in Matlab Lecture 2. Functions.Newton’s Method Lecture 14. Symbolic Computations Review of Part I II Linear Algebra 27 28 32 36 39 43 46 50 53 56 60 v Lecture 8. Introduction to Linear Systems Lecture 10. Accuracy.

Polynomial and Spline Interpolation Lecture 20. Insulated Boundary Conditions Lecture 39. Euler Methods Lecture 31. The Main Sources of Error Review of Part III IV Differential Equations 105 106 109 113 116 117 120 122 126 129 132 136 Lecture 29.vi Lecture 18. Double Integrals for Non-rectangles Lecture 26. Solution Instability for the Explicit Method Lecture 37. Higher Order Methods Lecture 32. Multi-step Methods* Lecture 33. Least Squares Fitting: Noisy Data Lecture 21. Right and Trapezoid Rules Lecture 22. Parabolic PDEs . Reduction of Higher Order Equations to Systems Lecture 30. Gaussian Quadrature* Lecture 27. Numerical Differentiation Lecture 28. Finite Difference Method – Nonlinear ODE Lecture 35. Iterative solution of linear systems* Review of Part II CONTENTS 62 63 III Functions and Data 67 68 72 75 79 83 86 90 93 94 97 100 Lecture 19. Integration: Left. Double Integrals for Rectangles Lecture 25. ODE Boundary Value Problems and Finite Differences Lecture 34. Finite Difference Method for Elliptic PDEs . Implicit Methods Lecture 38. Integration: Midpoint and Simpson’s Rules Lecture 23. Plotting Functions of Two Variables Lecture 24.Explicit Method Lecture 36.

. . . . . 170 11. . . . . . . . . . . . . . . . . . 169 10. . . . Convection-Diffusion Equations Lecture 41. . . . . . COPYING IN QUANTITY . . . . . . . RELICENSING . . . . . . . . . . . . . 171 . . . . Glossary of Matlab Commands GNU Free Documentation License 1. . . . . . . . . . . . . . . . . COMBINING DOCUMENTS . 170 ADDENDUM: How to use this License for your documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 3. . . . . . . . . . Determining Internal Node Values Review of Part IV vii 139 140 144 147 V Appendices 149 150 156 162 Lecture A. .CONTENTS Lecture 40. 168 8. . . . . TERMINATION . . . . . . . . . . . . . . . . . . . . . . MODIFICATIONS . . . . . . . . . . . . 169 9. . FUTURE REVISIONS OF THIS LICENSE . . . . . . . . . . . . . . . . . AGGREGATION WITH INDEPENDENT WORKS . . . . . . . . . . . . Sample Exams Lecture B. . . . . . 167 6. . 165 5. . . . . . . . . . . . . . . . . . . . . . . . . . . . Finite Elements Lecture 42. . . . . . . . . . . . . . . . . . . . . . . . . . . . TRANSLATION . 168 7. . . . . . . . . . . . . . APPLICABILITY AND DEFINITIONS . . . . . . . . . . . . . . . . . 165 4. . . . COLLECTIONS OF DOCUMENTS . . . . . 162 2. . . . . . . VERBATIM COPYING . . . . . . . . . . . .

viii CONTENTS .

2007 . Mathematics Department. Todd Young and Martin Mohlenkamp. Ohio University.Part I Matlab and Solving Equations c Copyright.

006 55 0.1 We can enter T (C◦ ) µ 5 0. 10. 12. To enter a column vector type: > u = [9. 13] You can access an entry in a vector with > u(2) and change the value of that entry with > u(2)=47 You can extract a slice out of a vector with > u(2:4) You can change a row vector into a column vector.009 50 0. Vectors can be thought of as special matrices. Entering vectors In Matlab.e.015 30 0.1:1 and > y = linspace(0. A row vector is recorded as a 1 × n matrix and a column vector is recorded as a m × 1 matrix. p.08 20 0. 2 . obtained from experiments on the viscosity of a liquid. the basic objects are matrices. and Plots in Matlab In this book > will indicate commands to be entered in the command window.) There are also useful shortcuts to make vectors such as: > x = -1:.174. arrays of numbers. i.11) Plotting Data Consider the following table. and vice versa easily in Matlab using: > w = v’ (This is called transposing the vector and we call ’ the transpose operator. 11. Matlab will print out the row vector. Functions.1.0055 this data into Matlab with the following commands entered in the command window: > x = [ 5 20 30 50 55 ] 1 Adapted from Ayyup & McCuen 1996. type the following at the prompt ( > ) in the command window: > v = [0 1 2 3] and press enter. You do not actually type the command prompt > .Lecture 1 Vectors. To enter a row vector in Matlab.

just using the available data. rather than a formula for the function.y) will return a plot of sin x on the interval [0. To find out more about a function. yi )}.y. try > help plot User-Defined Inline Functions If we wish to deal with a function that is a combination of the built-in functions.y. For example: > x = linspace(0.y) This will produce a graph with the data points connected by lines.40) > y = sin(x) > plot(x.2*pi. We will find later that we can also do other things with the function.08 0. we will almost always be working with a finite. log( ) (natural logarithm). discrete set of values of the function (data). The dependent variable (input) always goes in parentheses in Matlab. If you would prefer that the data points be represented by symbols you can do so. tan( ).015 0. the topic of this course. can be thought of envisioning the function using just the data. Built-in Functions If we wish to deal with formulas for functions.’) Data as a Representation of a Function A major theme in this course is that often we are interested in a certain function y = f (x). acos( ).. which . the built-in functions can operate not only on single numbers but on vectors.’. which is of course 0 and > exp(0) will return e0 which is 1.0055] Entering the name of the variable retrieves its current values.006 0. For instance: > plot(x. exp( ). cosh( ). but the only information we have about this function is a discrete set of data {(xi . Since a computer can only store a finite amount of information. including all the usual functions.’o’) > plot(x. Plotting the data. atan( ). use the help command. asin( ) (inverse sine). such as sin( ). Matlab has a couple of ways for the user to define functions.009 0. as we did above.’*’) > plot(x. log10( ) (log base 10). like differentiating and integrating. One that we will use a lot is the inline function. 2π] Some of the built-in functions in Matlab include: cos( ). More importantly. etc.y. means doing mathematics by computer. sinh( ). The meaning of most of these is clear. Matlab contains a number of built-in functions. For instance: > sin(pi) should return the value of sin π.3 > y = [ 0. Numerical methods. For instance: > x > y We can plot data in the form of vectors using the plot command: > plot(x.

^. turns the exponent operator ^ into entry-wise exponentiation. rather than one component at a time.6] and yields only an error.8. is needed in .2 Make an inline function g(x) = x + cos(x5 ).. Try the following: > x = -2:. FUNCTIONS. Plotting on any machine always requires this step.^2 means [(−2)2 ./. −1. .1 Find a table of data in an engineering textbook. Exercises 1. To obtain a single value of this function enter: > f(2. It is not needed when you * or / by a scalar or for +. Input it as vectors and plot it.6][−2. (−1.^2. The .8 -1. The .8 -1. and y = g(x). . The results can be plotted using the plot command. What is wrong with this graph? Find a way to make it look more like the graph of g(x) should.23572) Just as for built-in functions. The following is a typical inline function: > f = inline(’2*x. the function f as we defined it can operate not only on single numbers but on vectors. Turn in both plots.y) Notice that before plotting the function.*.e. Turn in the graph. −1. Use the insert icon to label the axes and add a title to your graph.^2 .1:5. Plot it using vectors x = -5:.6].8. −1. VECTORS.56]. just as for data: > plot(x. −1. The reason f (x) works when x is a vector is because we represented x2 by x.8)2 . we in effect converted it into data. (−1. AND PLOTS IN MATLAB is a way to define a function in the command window.2:2 > y = f(x) This is an example of vectorization. Indicate what the data is and where it came from.3*x + 1’. [-2 -1.6)2 ] and yields [4 3. putting several numbers into a vector and treating the vector all at once. In contrast. and .4 LECTURE 1.24 2. so that [-2 -1.’x’) This produces the function f (x) = 2x2 − 3x + 1. 1. and is one of the strengths of Matlab. i.6]^2 means the matrix product [−2.

. programs may be written and saved in files with a suffix . . . . • The outputs. name of the function and the inputs must appear in the first line.^3. . . .3*x + 1. . . . . . . . . .^4. . . . . . . . In fact. . Save this file as: myfunc. . .m called M-files. Produces a vector of y values. the essential elements are: • Begin with the word function. . . . . This file can now be used in the command window just like any predefined Matlab function. In the document window type the following: function y = myfunc(x) y = 2*x. . Produces a vector of x values. There are two types of M-file programs: functions and scripts. x4 = x. . . . . in the command window enter: > x = -2:. . . . . . . Look back at the program. x3 = x. . . > plot(x. . • There are inputs and outputs. . = mypowers(x) 5 . . . . . . . Open a new document and type: function [x2 x3 x4] x2 = x. • The body of the program must assign values to the outputs. . not what anything in it was called. . All function programs are like this one. . . .^2 . > y = myfunc(x). .m that actually mattered.Lecture 2 Matlab Programs In Matlab. . Functions can have multiple inputs and/or multiple outputs. . . . . .y) Note that the fact we used x and y in both the function program and in the command window was just a coincidence.3*inputvector + 1. . . We could just as well have made the function function nonsense = yourfunc(inputvector) nonsense = 2*inputvector. . . . . . . . . . . it is the name of the file myfunc. .m in your working directory. . . . . .^2 . . . . Function Programs Begin by clicking on the new document icon in the top left of the Matlab window (it looks like an empty sheet of paper). . . . . . . . . .^2. .1:2. . . Next let’s create a function that has 1 input and 3 output variables. . .

’blue’.1:1 > [x2 x3 x4] = mypowers(x). x3 = x. • A script program may use and change variables in the current workspace (the variables used by the command window. we can use the results of the program to make graphs: > x = -1:.x. for a vector the computation is elementwise % Output: y -. . plot(x.m.x4.x. MATLAB PROGRAMS Save this file as mypowers.a number or vector of the same size as x y = 2*x. the comments should at least give the purpose. > plot(x.^2. namely: • There are no inputs and outputs. Program Comments For programs that have more than a couple of lines it is important to include comments.x.a number or vector.3*x + 1. inputs. It is best to include comments not only at the top of a program.x. not in the program.x3.x2. Many people use script programs for routine calculations that would require typing more than one command in the command window.’red’) Type this program into a new document and save it as mygraphs.’black’.x4.) Below is a script program that accomplishes the same thing as the function program plus the commands in the previous section: x2 = x.x3. Comments allow other people to know what your program does and they also remind yourself what your program does if you set it aside and come back to it later.’green’. but also with each section.’blue’.’black’.x. A properly commented version of the function with which we started this section is: function y = myfunc(x) % Computes the function 2x^2 -3x +1 % Input: x -. In the command window. They do this because correcting mistakes is easier in a program than in the command window. even though x was defined in the command window. In Matlab anything that comes in a line after a % is a comment.’green’. In the command window enter: > x = -1:.m.1:1.’red’) Script Programs Matlab uses a second type of program that differs from a function program in several ways.^4.x.^3. and outputs. > mygraphs Note that the program used the variable x in its calculations. x4 = x.x.x.^2 . For a function program.x2.6 LECTURE 2.

* and .’blue’.x. x3 = x. x^3.x.’red’) The Matlab command help prints the first block of comments from a file. Turn in printouts of the program and the graph. x4 = x. For example: % mygraphs % plots the graphs of x. and x^4 % on the interval [-1.1] Exercises 2. (π is pi in Matlab.1 Write a function program for the function x2 e−x .1:1.m and then do > help mygraphs it will print into the command window: mygraphs plots the graphs of x. Plot the function on [−5. 2.7 For a script program it is often helpful to include the name of the program at the beginning.x3.x2. and x^4 on the interval [-1. 2π] on one plot.^2.) Include comments in the program.’green’. Turn in the program and the graph.^4. sin 2x. Include adequate comments in the program.2 Write a script program that graphs the functions sin x. 2 . To get ex use exp(x). x^2. If we save the above as mygraphs. sin 3x.x.’black’. % plot each of the graphs plot(x. % calculate powers % x1 is just x x2 = x.^3. sin 4x.x4. x^2.^). sin 5x and sin 6x on the interval [0. x^3. using entry-wise operations (such as .1] % fix the domain and evaluation points x = -1:. 5].x.

. (3. end In order to do Newton’s method.4) a number of times. the secant method and Newton’s method. x2 . You should know that the basis for Newton’s method is approximation of a function by it linearization at a point. . then it is a fact that the sequence will converge to x∗ and will do it very quickly.1) As you learned in calculus. experiments. 8 . In real engineering problems the function you wish to optimize can come from a large variety of sources. we need to repeat the calculation in (3. The loop: for .. . i. All three depend on beginning close (in some sense) to an actual solution x∗ . This is accomplished in a program using a loop. Then we define a sequence of points {x0 . (3. or simulations. including formulas.} from the formula: xi+1 = xi − f (xi ) . set the left hand side (f (x)) of this approximation equal to 0 and solve for x to obtain: f (x0 ) . the final step in many optimization problems is to solve an equation of this form where f is the derivative of a function.1) by x∗ .3) x ≈ x0 − ′ f (x0 ) We begin the method with the initial guess x0 . There are three methods which you may have discussed in Calculus: the bisection method. solutions of differential equations. x1 . Newton iterations We will denote an actual solution of equation (3.Lecture 3 Newton’s Method and Loops Solving equations numerically For the next few lectures we will focus on the problem of solving an equation: f (x) = 0. f ′ (xi ) (3.4) which comes from (3. which means a section of a program which is repeated..e.2) Since we wish to find x so that f (x) = 0. (3. f (x) ≈ f (x0 ) + f ′ (x0 )(x − x0 ). Recall Newton’s method. If f (x) is reasonably well-behaved near x∗ and x0 is close enough to x∗ . F . which we hope is fairly close to x∗ .3). x3 . that you want to maximize or minimize.

which is usually the case in the real world. a number % n -.the function.starting guess. input as an inline % f1 -.x0. not the intermediate steps. > f1 = inline(’3*x^2’). We will need to input the function. Let us take a look at what can go wrong.n) % Solves f(x) = 0 by doing n steps of Newton’s method starting at x0. i. end loops have the same format. Any calculation that involves a repeated process is easily done by a loop.e. All for . % add the current integer end % end of the loop Call this function in the command window as: > mysum(100) The result will be the sum of the first 100 integers.. % Inputs: f -. end statement makes a loop as in the following simple function program: function S = mysum(n) % gives the sum of the first n integers S = 0. % set x equal to the initial guess x0 for i = 1:n % Do n times x = x . By simple algebra. Now let’s do a program that does n steps (iterations) of Newton’s method. its derivative. Last comes the end command. By trial and error.f1.e.e. xn .it’s derivative. input as an inline % x0 -. Loops are one of the main ways that computers are made to do calculations that humans cannot. and x0 is close enough to x∗ that the linear approximation (3. followed by an index (i) and a range of numbers (1:n). If we are only interested in the final approximation.5’) and define f 1 to be its derivative. The output will be the final value of x. . what is the lowest value of n for which the √ program converges (stops changing). prints x too end In the command window define an inline function: f (x) = x3 − 5 i. the initial guess. the true root of this function is 3 5. In Matlab. a for . i..the number of steps to do % Output: x -..2) is valid. > f = inline(’x^3 . Then run mynewton on this function. it begins with for.. Then come the commands that are to be repeated. How close is the program’s answer to the true value? Convergence Newton’s method converges rapidly when f ′ (x∗ ) is nonzero and finite. and the number of steps.f(x)/f1(x) % Newton’s formula. % start at zero % The loop: for i = 1:n % do n times S = S + i.9 The simplest way to accomplish this is to count the number of times through. then we can use a single variable x in the program and change it at each step: function x = mynewton(f.the approximate solution format long % prints more digits format compact % makes the output more compact x = x0.

Turn in the program and a brief summary of the results.4) gives some x1 that may or may not be any better than x0 . with x0 = 2. If you try > f = inline(’x^2’) > f1 = inline(’2*x’) > x = mynewton(f.0. what is the lowest value of n for which the program converges (stops changing).2) is valid. If x0 is not close enough to x∗ that the linear approximation (3.f1. Put the results in a table. x2 and x3 .10) then x does converge to 0. but use a calculator.1.10 LECTURE 3. By trial and error approximate how large n must be so that total distance stops changing. then the iteration (3. By trial and error. Use mynewton on the function f (x) = x5 − 7. How close is the answer to the true value? Plug the program’s answer into f (x). but not that rapidly. NEWTON’S METHOD AND LOOPS For f (x) = x1/3 we have x∗ = 0 but f ′ (x∗ ) = ∞. then either • xn will eventually get close to x∗ and the method will then converge (rapidly). perform 3 iterations of Newton’s method with starting point x0 = 2. is the value zero? 3.) Calculate the solution (x∗ = 41/3 ) on a calculator and find the errors and percentage errors of x0 .1 Suppose a ball with a coefficient of elasticity of . .3 For f (x) = x3 − 4. If you try > f = inline(’x^(1/3)’) > f1 = inline(’(1/3)*x^(-2/3)’) > x = mynewton(f. Write a script program to calculate the distance traveled by the ball after n bounces.f1. Include lots of comments. x1 . or • the iterations will not approach x∗ . If we keep iterating. Exercises 3.1.2 Enter: format long. (On paper. For f (x) = x2 we have x∗ = 0 but f ′ (x∗ ) = 0. 3.9 is dropped from a height of 2 meters onto a hard surface.10) then x explodes.

This makes it impossible to know the error directly. The if . how close f (xn ) is to 0.. we usually do not know the value of x∗ . For Newton’s method we have the following principle: At each step the number of significant digits roughly doubles. so we look at |rn | = |f (xn )|. We will use the quantity rn = f (xn ) − 0. in other words. end statement: 11 . called the residual. in this and many other situations it is much more practical to measure how close the equation is to being satisfied.. in many different situations.Lecture 4 Controlling Error and Conditional Statements Measuring error If we are trying to find a numerical solution of an equation f (x) = 0. Most of the time we only care about the size of rn . then we can incorporate that into our Newton method program using an if . Rather than measure how close xn is to x∗ . and so we must be more clever. it is somewhat hard to use in a program. then there are a few different ways we can measure the error of our approximation. or we wouldn’t be trying to approximate it. While this is an important statement about the error (since it means Newton’s method converges really quickly).. end statement If we have a certain tolerance for |rn | = |f (xn )|. The most direct way to measure the error would be as: {Error at step n} = en = xn − x∗ where xn is the n-th approximation and x∗ is the true value.. However.

the approximate solution x = x0.’x’).x0. while abs(y) > tol % Do until the tolerence is reached.. input as an inline % x0 -. end loop. the number of steps to take. input as an inline % f1 -. x = x . end While the previous program will tell us if it worked or not.it’s derivative. but instead of going through the loop a fixed number of times it keeps going as long as the statement abs(y) > tol is true. % Inputs: f -. we still have to input n. tol = . input as an inline % x0 -. end The statement while . function x = mynewtontol(f. % Inputs: f -.starting guess. In Matlab this is easily accomplished with a while . prints a warning if |f(x)|>tol % Output: x -. goes until |f(x)|<tol % Output: x -. then the program will take more steps than necessary.12 LECTURE 4.f1. Then use the program with n = 5. % set x equal to the initial guess x0 y = f(x). % set x equal to the initial guess x0 for i = 1:n % Do n times x = x .it’s derivative. a number % tol -. a number % tol -.. If not true.the approximate solution x = x0. Next.tol) % Solves f(x) = 0 by doing n steps of Newton’s method starting at x0.f(x)/f1(x)..starting guess..y/f1(x). change tol to 10−10 and repeat.x0.tol) % Solves f(x) = 0 by doing Newton’s method starting at x0.f1. One way to control the number of steps taken is to iterate until the residual |rn | = |f (x)| = |y| is small enough.desired tolerance. The loop: while . similar to for . if r > tol warning(’The desired accuracy was not attained’) end In this program if checks if abs(y) > tol is true or not. ..’x’) and its derivative > f1 = inline(’3*x^2’. input as an inline % f1 -. Even for a well-behaved problem.. end.n. if we make n too big.desired tolerance..the function. end is a loop. if we make n too small then the tolerance will not be attained and we will have to go back and increase it. In the command window define the function > f = inline(’x^3-5’.the function. % Newton’s formula y = f(x).. CONTROLLING ERROR AND CONDITIONAL STATEMENTS function x = mynewton(f. If it is true then it does everything between there and end. and x0 = 2. or. then it skips ahead to end.01. % Newton’s formula end r = abs(f(x)).

do not use a for loop or trial and error. the program would continue to run over and over until we stop it. Exercises 4.1 to compute the total distance traveled by the ball while its bounces are at least 1 centimeter high. Put in the missing command and then use the program to verify the result above. How many steps does it take? How close is the answer to 2? Change r = .1 In Calculus we learn that a geometric series has an exact sum: ∞ i=0 ri = 1 1−r provided that |r| < 1. 4. Below is a script program that lacks one line as written. Try this by setting the tolerance to a really small number: > tol = 10^(-100) then run the program again for f (x) = x3 − 5.13 One obvious drawback of the program is that abs(y) might never get smaller than tol. Snew = 0.5 to r=. Snew % i % start sum at 0 set Sold to trick while the first time count iterations is the sum still changing? save previous value to compare to prints the final value. % while Snew > Sold % Sold = Snew. How many steps does it take now? How accurate is the answer? format long r = .999. For instance. . % Snew = Snew + r^i. % Sold = -1. prints the # of iterations. If this happens. Turn in your modified program and a brief summary of the results.5 then the sum is exactly 2. i=i+1. if r = . % i = 0.2 Modify your program from exercise 3. Use a while loop to decide when to stop summing. You can use Ctrl-c to stop the program when it’s stuck.5.

else .e. the error is always less than half of the length of the current interval [a.Lecture 5 The Bisection Method and Locating Roots Bisecting and the if . where x is the center point between the current a and b.. if c and y have the same sign.. d and y we can decide which new interval to subdivide. else . This allows us to be sure about what the maximum error can be. If f is continuous. {Absolute Error} = |x − x∗ | < (b − a)/2. 14 .. end statement we have used already.. end statement which is an extension of the if . let x = (a + b)/2 and evaluate y = f (x). i.. In particular. that we don’t have with Newton’s method. i. (See Figure 5. then from the signs of c. since f (a) and f (b) have different signs. is that we always know that the actual solution x∗ is inside the current interval [a. b] should be the new interval. x] should be the new interval.. then [a.. end statement Recall the bisection method.e. Suppose that c = f (a) < 0 and d = f (b) > 0.. The most common way to do this is the if .) Deciding to do different things in different situations in a program is called flow control. Precisely. then [x. Unless this is zero. then obviously it must be zero at some x∗ between a and b.1: The bisection method. Bounding the Error One good thing about the bisection method.1. The bisection method then consists of looking half way between a and b for the zero of f .. b]. u u a0 a1 u x1 u x2 x0 b1 b2 b0 a2 Figure 5. but if c and y have different signs. b]..

b. Locating a root The bisection method and Newton’s method are both used to obtain closer and closer approximations of a solution. but have to solve the same equation many times.a. d = f(b). From an accurate graph you can see approximately where the graph crosses zero. How do you come up with these starting points? It depends. We saw that Newton’s method can fail to converge to x∗ if x0 is not close enough to x∗ .15 The following function program (also available on the webpage) does n iterations of the bisection method and returns not only the final value. the current interval [a. There are other situations where you are not just solving an equation once.the estimated solution of f(x) = 0 % e -. Another important aspect of bisection is that it always works. else a=x. b] in bisection will always get decreased by a factor of 2 at each step and so it will always eventually shrink down as small as you want it.e.b.0 error(’Function has same sign at both endpoints.the number of bisections to do. disp([ x y]) if y == 0. but with different coefficients. This happens often when you are developing software for a specific application. y = f(x). If that .’) end disp(’ x y’) for i = 1:n x = (a + b)/2. but also the maximum possible error: function [x e] = mybisect(f. i.0 % solved the equation exactly e = 0. If you are solving an equation once.b -.left and right edges of the interval % n -. and Newton’s method requires one point x0 which is reasonably close to a root. on what interval is a solution physically reasonable.n) % function [x e] = mybisect(f. but both require starting places.an inline function % a. % Outputs: x -.n) % Does n iterations of the bisection method for a function f % Inputs: f -. The bisection method requires two points a and b that have a root between them. then the best thing to do first is to just graph it.a. end end e = (b-a)/2. In this situation the first thing you want to take advantage of is the natural domain of the problem. if c*d > 0. In contrast. break % jumps out of the for loop end if c*y < 0 b=x.an upper bound on the error format long c = f(a).

the right edge of the domain % Outputs: a -. Exercises 5. For Newton’s method one would want to choose x0 between a and b.) Calculate the errors and percentage errors of x0 . for i = 1:(n-1) if y(i)*y(i+1) < 0 % The sign changed. x0 = (a + b)/2. % number of test points to use a = []. y = f(x).the left edge of the domain % b0 -. these a and b can serve as the beginning points for the bisection and secant methods (see the next section). b].16 LECTURE 5. (On paper. and x2 . THE BISECTION METHOD AND LOCATING ROOTS is known.a0. 5. 1] and a tolerance of 10−8 .) Turn in your program and a brief summary of the results. % start empty array b = []. How many steps does the program use to achieve this tolerance? (You can count the step by adding 1 to a counting variable i in the loop of the program. The following program will look for the roots of a function f on a specified interval [a0 .1 Modify mybisect to solve until the error is bounded by a given tolerance.b] = myrootfind(f. with starting interval [1. An even better choice would be to use the secant method to choose x0 .b0) % function [a. then it is easy to get close to the root by simply checking the sign of the function at a fixed number of points inside the interval.b0) % Looks for subintervals where the function changes sign % Inputs: f -. function [a. giving the left edges of subintervals % on which f changes sign % b -. Once a root has been located on an interval [a. record it a = [a x(i)]. Compare the errors with those in exercise 3. This is a difficult problem and one that is not often encountered in engineering applications. b = [b x(i+1)].3. x = linspace(a0. How should error be measured? Run your program on the function f (x) = 2x3 +3x−1 with starting interval [0.b] = myrootfind(f. giving the right edges of the subintervals n = 1001. . but use a calculator. Use a while loop to do this. there is a root between those points.b0. end end if a == [] warning(’no roots were found’) end The final situation is writing a program that will look for roots with no given information. One obvious choice would be to let x0 be the bisector of a and b. i. Whenever the sign changes from one point to the next. 3].an inline function % a0 -.n).an array.2 Perform 3 iterations of the bisection method on the function f (x) = x3 − 4.e. x1 .a0.an array. b0 ].

. In the next step we would use x1 = 2 and x2 = 19/15 in the formula (6...1) to find x3 and so on. Notice that it requires two input guesses x0 and x1 . We may then calculate x2 from the formula (6.25. The Secant Method The secant method requires two initial points x0 and x1 which are both reasonably close to the solution x∗ .. alas Figure 6..425758.Lecture 6 Secant Methods* In this lecture we introduce two additional methods to find numerical solutions of the equation f (x) = 0.1): 2−1 19 11 = ≈ 1.2666. Below is a program for the secant method. 11 − (−4) 15 x2 = 2 − Pluggin x2 = 19/15 into f (x) we obtain y2 = f (19/15) ≈ −2. Once x0 and x1 are determined the method proceeds by the following formula: xi − xi−1 yi yi − yi−1 xi+1 = xi − (6. If we let x0 = 1 and x1 = 2 then we know that the root is in between x0 and x1 .1) √ Example: Suppose f (x) = x4 − 5 for which the true solution is x∗ = 4 5. Preferably the signs of y0 = f (x0 ) and y1 = f (x1 ) should be different. Plotting this function reveals that the solution is at about 1.. but it does not require the derivative to be input. figure yet to be drawn.1: The secant method. Next we have that y0 = f (1) = −4 and y1 = f (2) = 11. 17 . Both of these methods are based on approximating the function by secant lines just as Newton’s method was based on approximating the function by tangent lines.

but choose how to update using the sign of f (xn ).starting guess.the approximate solution y0 = f(x0). end The Regula Falsi Method The Regula Falsi method is somewhat a combination of the secant method and bisection method.x1. Then let: b−a f (b). for i = 1:n % Do n times x = x1 . The idea is to use secant lines to approximate f (x). Both however are still much faster than the bisection method. y1=y.the function. SECANT METHODS* function x = mysecant(f.18 LECTURE 6. The secant method is a little slower than Newton’s method and the Regula Falsi method is slightly slower than that. input as an inline function % x0 -. a number % x1 -. x1=x. y1 = f(x1). y0=y1.the number of steps to do % Output: x -. Otherwise let b = x. we begin with a and b for which f (a) and f (b) have different signs.x0. there are contexts when it is not convenient. The Regula Falsi method. Convergence If we can begin with a good choice x0 .(x1-x0)*y1/(y1-y0) % secant formula.n) format long % prints more digits format compact % makes the output more compact % Solves f(x) = 0 by doing n steps of the secant method starting with x0 and x1. % Move numbers to get ready for the next step x0=x1. One obvious situation is when it is difficult to calculate a . then Newton’s method will converge to x∗ rapidly. If we do not have a good starting point or interval. % Inputs: f -. Simulations and Experiments Although Newton’s method converges faster than any other method. x=b− f (b) − f (a) Next check the sign of f (x). y=f(x) % y value at the new approximate solution. or even impossible. just like Newton’s method can fail altogether. just like the bisection method always works because it keeps the solution inside a definite interval.second starting geuss % n -. If it is the same as the sign of f (a) then x becomes the new a. Just as in the bisection method. then the secant method.

19
formula for f ′ (x) even though one knows the formula for f (x). This is often the case when f (x) is not defined explicitly, but implicitly. There are other situations, which are very common in engineering and science, where even a formula for f (x) is not known. This happens when f (x) is the result of experiment or simulation rather than a formula. In such situations, the secant method is usually the best choice.

Exercises
6.1 Use the program mysecant on f (x) = x3 − 4. Compare the results with those of Exercises 3.3 and 5.2. Then modify the program to accept a tolerance using a while loop. Test the program and turn in the program as well as the work above. 6.2 Write a program myregfalsi based on mybisect. Use it on f (x) = x3 − 4. Compare the results with those of Exercises 3.3 and 5.2.

Lecture 7 Symbolic Computations
The focus of this course is on numerical computations, i.e. calculations, usually approximations, with floating point numbers. However, Matlab can also do symbolic computations which means exact calculations using symbols as in Algebra or Calculus. You should have done some symbolic Matlab computations in your Calculus courses and in this chapter we review what you should already know.

Defining functions and basic operations
Before doing any symbolic computation, one must declare the variables used to be symbolic: > syms x y A function is defined by simply typing the formula: > f = cos(x) + 3*x^2 Note that coefficients must be multiplied using *. To find specific values, you must use the command subs: > subs(f,pi) This command stands for substitute, it substitutes π for x in the formula for f . If we define another function: > g = exp(-y^2) then we can compose the functions: > h = compose(g,f) i.e. h(x) = g(f (x)). Since f and g are functions of different variables, their product must be a function of two variables: > k = f*g > subs(k,[x,y],[0,1]) We can do simple calculus operations, like differentiation: > f1 = diff(f) indefinite integrals (antiderivatives): > F = int(f) and definite integrals: > int(f,0,2*pi) To change a symbolic answer into a numerical answer, use the double command which stands for double precision, (not times 2): > double(ans) Note that some antiderivatives cannot be found in terms of elementary functions, for some of these it can be expressed in terms of special functions: > G = int(g) and for others Matlab does the best it can:

20

21
cos(x5)

1

0.5

0

−0.5

−1 −6 −4 −2 0 x 2 4 6

Figure 7.1: Graph of cos(x5 ) produced by the ezplot command. It is wrong because cos u should oscillate smoothly between −1 and 1. The problem with the plot is that cos(x5 ) oscillates extremely rapidly, and the plot did not consider enough points.
> int(h) For definite integrals that cannot be evaluated exactly, Matlab does nothing and prints a warning: > int(h,0,1) We will see later that even functions that don’t have an antiderivative can be integrated numerically. You can change the last answer to a numerical answer using: > double(ans) Plotting a symbolic function can be done as follows: > ezplot(f) or the domain can be specified: > ezplot(g,-10,10) > ezplot(g,-2,2) To plot a symbolic function of two variables use: > ezsurf(k) It is important to keep in mind that even though we have defined our variables to be symbolic variables, plotting can only plot a finite set of points. For intance: > ezplot(cos(x^5)) will produce a plot that is clearly wrong, because it does not plot enough points.

Other useful symbolic operations
Matlab allows you to do simple algebra. For instance: > poly = (x - 3)^5 > polyex = expand(poly) > polysi = simple(polyex)

Test it on f (x) = x3 − 4 starting with x0 = 2.Y) Exercises 7.1:4.22 To > > > LECTURE 7. f (x) = 0. Turn in the program and a brief summary of the results. Let the program take the symbolic derivative f ′ .X). use: solve(f) solve(g) solve(polyex) Another useful property of symbolic functions is that you can substitute numerical vectors for the variables: > X = 2:0. SYMBOLIC COMPUTATIONS find the symbolic solutions of an equation. > plot(X.1 Starting from mynewton write a function program mysymnewton that takes as its input a symbolic function f and the ordinary variables x0 and n. and then use subs to proceed with Newton’s method. > Y = subs(polyex. .

Newton and Secant can fail if x0 is not close to x∗ . 23 . x∗ is always inside [a. Choose x0 between sign changes using bisection or secant. b−a f (b) f (b) − f (a) xi − xi−1 yi yi − yi−1 f (xi ) f ′ (xi ) Convergence: Bisection is very slow. Systematically search for sign changes of f (x). x = (a + b)/2 Choose a = x or b = x. Usage: For Newton’s method one must have formulas for f (x) and f ′ (x). e < (b − a)/2.Review of Part I Methods and Formulas Solving equations numerically: f (x) = 0 — an equation we wish to solve. Bisection and Regula Falsi never fail to converge. en = xn − x∗ — error of n-th step. Newton’s method: xi+1 = xi − Bisection method: f (a) and f (b) must have different signs. Locating roots: Use knowledge of the problem to begin with a reasonable domain. b]. Newton is very fast. Secant method*: xi+1 = xi − Regula Falsi*: x=b− Choose a = x or b = x. x0 — starting approximation. depending on signs. rn = yn = f (xn ) — residual at step n. Often |rn | is sufficient. x∗ — a true solution. current maximum error. Secant methods are intermediate in speed. xn — approximation after n steps. depending on signs.

. . . . . . . . . .n) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .^2 . . . . . . . . . . . . . . . . . .1. i. . . . . . . > w = v’ . . . Make an evenly spaced vector. . . . . . . . Make an evenly spaced vector of length 11. . . . . .1. function x = mynewton(f.11) . . . . name of the function and the inputs must appear in the first line. . . . .The body of the program must assign values to the outputs. Make a row vector. . . . . . if . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24 Secant methods are better for experiments and simulations. . . . . . . . . . . . . . . . . . . . . Stops a computation. . . . . . . . . . . . . . . . . else . . . . . . . . . . . . . end Example: if c*y>0 a = x. . . . . . . . .. . . .The outputs. else b = x.There are inputs and outputs. . . > x = -1:. Example: while i S = i = end end <= 20 S + i. . . . . . . . . . . . . . . . . . . . .. . . . x. . . . . . . . . . .3*x + 1’. . . . . . . . . . . . 2. . . . > u = [0. . . . . . . . . . . . . .. . . . .. . . Program structures: for . . end Example: if y == 0 disp(’An exact solution has been found’) end while . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > y = f(x) . . . . . . A function can act on a vector. . . . > f = inline(’2*x. . . . . . .’x’) . . . . . . . . . . . . .. . . . . . . . .y. . . . . . . . . .. . . . . . . . . . . . . . . . end if . . . . . . . . . . . . . .y) . . . . end Function Programs: . . . . . . . end Example: for i=1:20 S = S + i. . . Make a column vector. . . . . . . . . . . . Square all entries. . . .. . .’red’) . . plot y vs. . . . . . . . > plot(x.A plot with options. . . . . 1. . . . > Control-c . . . . . .x0. .e. . . .. . . . . . . . i + 1. with increments 0. . . > plot(x. . . . . . . . . . . . . .’*’. . . . . .. . . . . . . . . .Begin with the word function. . . . . . . . . Transpose: row vector ↔ column vector > x = linspace(0. . . Make a function. . . . . . . . . . . . . . . . . . > y = x. . . . . . . . . . . . . . . . . . . . . . . REVIEW OF PART I Matlab Commands: > v = [0 1 2 3] . . 3] .^2 . . . . . . . . . . . . .1:1 . . .. . . . . . . . . . . .

. . . . . . . indefinite integral (antiderivative) > int(f. . . . . . . .-10. . . . . .There are no inputs and outputs. . . . . . . .A script program may use and change variables in the current workspace. . . > h(x) = compose(g.2*pi) . . . . . . .25 .sym(pi)) > double(ans) > g = log(abs(y)) . .sqrt(3*x) > subs(f. . . . . . . . . . . Symbolic: > syms x y > f = 2*x^2 . . . . . .f) > k(x. . . . .10) > ezsurf(k) > f1 = diff(f. . . . . . . . . Matlab uses log for natural logarithm. .internal variables are not visible outside the function. . . . . .’x’) > F = int(f. . . . . . . definite integral > poly = x*(x . . . . . . . .3)*(x-2)*(x-1)*(x+1) > polyex = expand(poly) > polysi = simple(polyex) > solve(f) > solve(g) > solve(polyex) . . . . . . . . . . . Script Programs: . . . . . . . . . . . . . . . . . . . . . .’x’) . . . . . . . . . . . . . .0. . .y) = f*g > ezplot(f) > ezplot(g. . . . . . . . . . . .

26 REVIEW OF PART I .

Part II Linear Algebra c Copyright. Ohio University. Mathematics Department. 2007 . Todd Young and Martin Mohlenkamp.

Printing matrices on the screen takes up a lot of space. = = AB = 5 −2 −3 + 8 −6 + 4 2 1 3 4 28 . A column vector can be thought of as a p × 1 matrix and a row vector as a 1 × q matrix. So. so you may want to use > format compact Enter a matrix into Matlab with the following syntax: > A = [ 1 3 -2 5 . then we can multiply AB if and only if n = p. 0 1 -9 0] Also enter a vector u: > u = [ 1 2 3 4]’ To multiply a matrix times a vector Au use *: > A*u Since A is 3 by 4 and u is 4 by 1 this multiplication is valid and the result is a 3 by 1 vector. Matrices can be multiplied together in this way provided that the number of columns of A match the number of rows of B. then columns. Now enter another matrix B using: > B = [3 2 1. Unless otherwise specified we will assume a vector v to be a column vector and so Av makes sense as long as the number of columns of A matches the number of entries in v. so a 3 × 5 matrix would have 3 rows and 5 columns. We always list the size of a matrix by rows. To multiply two matrices. if A is m × n and B is p × q. 7 6 5.Lecture 8 Matrices and Matrix Operations in Matlab Matrix operations Recall how to multiply a matrix A times a vector v: Av = 1 2 3 4 −1 2 = 1 · (−1) + 2 · 2 3 · (−1) + 4 · 2 = 3 5 . Here both A and B are 2 × 2 matrices. You can multiply a matrix by a scalar: > 2*A This is a special case of matrix multiplication. -1 -1 5 4 . 4 3 2] You can multiply B times A: > B*A but A times B is not defined and > A*B will result in an error message. A and B you proceed as follows: 3 0 −1 + 4 −2 + 2 −1 −2 1 2 .

. .e. . . It is called the identity because it plays the same role that 1 plays in multiplication. . . . . and A−1 is only important for the purpose of derivations. . . . But. .^3 make sense for any size matrix. Similarly if you want B multiplied by itself 3 times then enter: > B^3 but. . .*B and B. . Iv = v for any matrix A or vector v where the sizes match. adding a ’. . . . i. The identity matrix and the inverse of a matrix The n × n identity matrix is a square matrix with ones on the diagonal and zeros everywhere else. . . .*B the entries of the resulting matrix will contain the squares of the same entries of B. An identity matrix in Matlab is produced by the command: > I = eye(3) A square matrix A can have an inverse which is denoted by A−1 . We will see later that this is not a practical way to solve an equation. .e. AI = A. . i. . . . . . ‘/’. . . .5) > inv(C) However. such problems are very common and important) then the theoretical solution is: x = A−1 b. . . . If you enter: > B*B the result will be BB. . . IA = A. . . . or ‘^’ produces entry-wise multiplication. Component-wise operations Just as for vectors. . but B. matrix multiplication of B times itself. . .^3 Note that B*B and B^3 only make sense if B is square. In theory an inverse is very important. division and exponentiation.’ before ‘*’. .29 Adding matrices A + A will give the same result: > A + A You can even add a number to a matrix: > A + 3 . . not all square matrices have inverses: > D = ones(5. . In Matlab we can calculate a matrix’s inverse very conveniently: > C = randn(5. The definition of the inverse is that: AA−1 = I and A−1 A = I. . . because if you have an equation: Ax = b where A and b are known and x is unknown (as we will see. if you enter: > B.5) > inv(D) . if you want to cube all the entries of B then enter: > B. . This should add 3 to every entry of A.

. . . . . . . the determinant of the matrix. . . . . . .the minimum in each column. . . . . . . . . . . . MATRICES AND MATRIX OPERATIONS IN MATLAB The “Norm” of a matrix For a vector. . . . . . . . . . . . . . . . . . . . . Another way to think of it is how far the vector is from being the zero vector. . > size(C) . . . . . . . . . . . . . . . . > min(C) . . . . . . . . . . . . . . . . . zeros. . . . . . . . . . . . . . . . .50)) For a matrix the norm defined above and calculated by Matlab is not the square root of the sum of the square of its entries. . . . . . . . . . . . . . . . . . . . . The norm of A is |A| ≡ max |Av|. . . . . . . . . . . . . . . |v|=1 The maximum in the definition is taken over all vectors with length 1 (unit vectors). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > diag(C) . . . . . . . . . . . . . . . . . . . > max(C) . . . . . . . . . . Some other useful commands Try out the following: > C = rand(5. . . . . . . . . . . . . . . . . . . . 1]. . . . . . . . . . > det(C) . . . . . . . . . . In addition to ones. . . . In Matlab the norm of a matrix is obtained by the command: > norm(A) For instance the norm of an identity matrix is 1: > norm(eye(100)) and the norm of a zero matrix is 0: > norm(zeros(50. . . . . . . . . . sums each column. . . . . . . . . but it turns out to be the best one. . . . . . . . . . . . the “norm” means the same thing as the length. . . . . . . . . . . . . . . . . > C’ . . . . . . . random matrix with uniform distribution in [0. . . . . . . . . . . . . . . . . . . . . . . with this definition we have the following inequality for any vector v: |Av| ≤ |A||v|. . . For example. eye. . > sum(C) . . . . . . . . . . . . . . . . . but we will not need it. . . . . . . That quantity is called the Froebenius norm. . . . . so the definition means the largest factor that the matrix stretches (or shrinks) a unit vector. . . We want to measure a matrix in much the same way and the norm is such a quantity. . gives the dimensions (m × n) of C. . . the average of each column. which is also sometimes useful. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Matlab has several other commands that automatically produce special matrices: > hilb(6) > pascal(5) . . . . . . . . . . . . . . . . . . This definition seems cumbersome at first. . . . . . . . . . . . . . just the diagonal elements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the maximum of each column. . . . . . . .30 LECTURE 8. > mean(C) .tranpose the matrix. . . The usual definition of the norm of a matrix is the following: Definition 1 Suppose A is a m × n matrix. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . rand and randn. . . . . . .5) . . . . . . . .

) What happens to error as n gets big? Turn in a printout of the programs. N=   1 4 −1  2 −1 2 8.2 By hand. (a) Write a Matlab function program myinvcheck that makes a n × n random matrix (normally distributed. A = randn(n.3 (b) Write a Matlab script program that calls myinvcheck for n = 10.4. and a very brief report on the results of your experiments. Can the order of multiplication be switched? Why or why not? Try it to see how Matlab reacts. 2i 10. .0. −2 0  3 v =  1 . .-2. (See help loglog.1 Enter the matrix M by > M = [1.-5. .2. Check the results using Matlab. B =  1 −1 0 Multiply M and N using M * N.0.  −1 −1 0 2 .2. and plots the error versus n using a log plot. calculates the error equal to the norm of the difference of the result from the n × n identity matrix (eye(n)). . 40.n)).-1.3. records the results of each trial. calculates its inverse (B = inv(A)). the plot. 20.3. .-1.31 Exercises 8. multiplies the two back together. AB.-1. calculate Av. and returns error.-1. Think about how fast computers are. Turn in your hand work.1] and also the matrix   −1 −3 3  2 −1 6  .6.  2 4 A =  −2 1 −1 −1 and BA for:   0 −1 9 . −1  8.

Lecture 9 Introduction to Linear Systems
How linear systems occur
Linear systems of equations naturally occur in many places in engineering, such as structural analysis, dynamics and electric circuits. Computers have made it possible to quickly and accurately solve larger and larger systems of equations. Not only has this allowed engineers to handle more and more complex problems where linear systems naturally occur, but has also prompted engineers to use linear systems to solve problems where they do not naturally occur such as thermodynamics, internal stress-strain analysis, fluids and chemical processes. It has become standard practice in many areas to analyze a problem by transforming it into a linear systems of equations and then solving those equation by computer. In this way, computers have made linear systems of equations the most frequently used tool in modern engineering. In Figure 9.1 we show a truss with equilateral triangles. In Statics you may use the “method of joints” to write equations for each node of the truss1 . This set of equations is an example of a linear √ system. Making the approximation 3/2 ≈ .8660, the equations for this truss are: .5 T1 + T2 = R1 = f1 .866 T1 = −R2 = −.433 f1 − .5 f2 (9.1)

−.5 T1 + .5 T3 + T4 = −f1 .866 T1 + .866 T3 = 0 −T2 − .5 T3 + .5 T5 + T6 = 0 .866 T3 + .866 T5 = f2 −T4 − .5 T5 + .5 T7 = 0,

where Ti represents the tension in the i-th member of the truss. You could solve this system by hand with a little time and patience; systematically eliminating variables and substituting. Obviously, it would be a lot better to put the equations on a computer and let the computer solve it. In the next few lectures we will learn how to use a computer effectively to solve linear systems. The first key to dealing with linear systems is to realize that they are equivalent to matrices, which contain numbers, not variables. As we discuss various aspects of matrices, we wish to keep in mind that the matrices that come up in engineering systems are really large. It is not unusual in real engineering to use matrices whose dimensions are in the thousands! It is frequently the case that a method that is fine for a 2 × 2 or 3 × 3 matrix is totally inappropriate for a 2000 × 2000 matrix. We thus want to emphasize methods that work for large matrices.
1

See http://en.wikipedia.org/wiki/Truss or http://en.wikibooks.org/wiki/Statics for reference.

32

33 f1 E
r

B 1 R1 '
uA

4 3 5 C
r c

r

D 7 6 E u

2

T

T

R2

f2

R3

Figure 9.1: An equilateral truss. Joints or nodes are labeled alphabetically, A, B, . . . and Members (edges) are labeled numerically: 1, 2, . . . . The forces f1 and f2 are applied loads and R1 , R2 and R3 are reaction forces applied by the supports.

Linear systems are equivalent to matrix equations
The system of linear equations, x1 − 2x2 + 3x3 = 4 2x1 − 5x2 + 12x3 = 15 2x2 − 10x3 = −10, is equivalent to the matrix equation,      4 1 −2 3 x1  2 −5 12   x2  =  15  , −10 0 2 −10 x3  4 15  . −10

The advantage of the augmented matrix, is that it contains only numbers, not variables. The reason this is better is because computers are much better in dealing with numbers than variables. To solve this system, the main steps are called Gaussian elimination and back substitution. The augmented matrix for the equilateral truss equations (9.1) is given by:  .5 1 0 0 0 0 0 f1  .866 0 0 0 0 0 0 −.433f1 − .5 f2   −.5 0 .5 1 0 0 0 −f1   .866 0 .866 0 0 0 0 0   0 −1 −.5 0 .5 1 0 0   0 0 .866 0 .866 0 0 f2 0 0 0 −1 −.5 0 .5 0

which is equivalent to the augmented matrix,  1 −2 3  2 −5 12 0 2 −10

Notice that a lot of the entries are 0. Matrices like this, called sparse, are common in applications and there are methods specifically designed to efficiently handle sparse matrices.

    .    

(9.2)

34

LECTURE 9. INTRODUCTION TO LINEAR SYSTEMS

Triangular matrices and back substitution
Consider a linear system whose augmented matrix happens to be:   1 −2 3 4  0 −1 6 7  . 0 0 2 4

(9.3)

Recall that each row represents an equation and each column a variable. The last row represents the equation 2x3 = 4. The equation is easily solved, i.e. x3 = 2. The second row represents the equation −x2 + 6x3 = 7, but since we know x3 = 2, this simplifies to: −x2 + 12 = 7. This is easily solved, giving x2 = 5. Finally, since we know x2 and x3 , the first row simplifies to: x1 − 10 + 6 = 4. Thus we have x1 = 8 and so we know the whole solution vector: x = 8, 5, 2 . The process we just did is called back substitution, which is both efficient and easily programmed. The property that made it possible to solve the system so easily is that A in this case is upper triangular. In the next section we show an efficient way to transform an augmented matrix into an upper triangular matrix.

Gaussian Elimination
Consider the matrix: 1 −2 A =  2 −5 0 2   4 3 12 15  . −10 −10

There is already a zero in the lower left corner, so we don’t need to eliminate anything there. To eliminate the third row, second column, we need to subtract −2 times the second row from the third row, (new 3rd = old 3rd - (-2) 2nd):   1 −2 3 4  0 −1 6 7  . 0 0 2 4 This is now just exactly the matrix in equation (9.3), which we can now solve by back substitution.

The first step of Gaussian elimination is to get rid of the 2 in the (2,1) position by subtracting 2 times the first row from the second row, i.e. (new 2nd = old 2nd - (2) 1st). We can do this because it is essentially the same as adding equations, which is a valid algebraic operation. This leads to:   1 −2 3 4  0 −1 6 7 . 0 2 −10 −10

Matlab’s matrix solve command
In Matlab the standard way to solve a system Ax = b is by the command: > x = A\b This command carries out Gaussian elimination and back substitution. We can do the above computations as follows: > A = [1 -2 3 ; 2 -5 12 ; 0 2 -10] > b = [4 15 -10]’ > x = A\b

only for theory.1 Set f1 = 1000N and f2 = 5000N in the equations (9. b and solution x. (Enter save A_equil_truss A.2) into Matlab. Check your solutions using Matlab. 7 9 10 11 12 by entering: > x1 = A\b Check the result by entering: > A*x1 . use the Matlab commands above to solve Ax = b when the augmented matrix for the system is:   4 1 2 3  5 6 8 . 9.35 Next.2 Write each system of equations as an augmented matrix. Save the matrix A as A_equil_truss and keep it for later use. (a) x1 + x2 = 2 4x1 + 5x2 = 10 (b) x1 + 2x2 + 3x3 = −1 4x1 + 7x2 + 14x3 = 3 x1 + 4x2 + 4x3 = 1 .1) for the equailateral truss. Solve the system using the command \ to find the tension in each member of the truss.) Print out and turn in A. Exercises 9.b Thus we see one of the reasons why the inverse is never used for actual computations. then find the solutions using Gaussian elimination and back substitution. Next try solving using the inverse of A: > x2 = inv(A)*b This answer can be seen to be inaccurate by checking: > A*x2 . Input the coefficient matrix A and the right hand side vector b in (9.b You will see that the resulting answer satisfies the equation exactly.

which is too simple.Lecture 10 Some Facts About Linear Systems Some inconvenient truths In the last lecture we learned how to solve a linear system using Matlab. Now try: > b = ones(4. Basic theory The most basic theoretical fact about linear systems is: Theorem 1 A linear system Ax = b may have 0. The columns of A are linearly independent (as vectors).4) > b = randn(4. Input the following: > A = ones(4.27 -2. Obviously. If A has these properties then it is called non-singular. The following are all equivalent: 1. Next try: > x = [ -4 5 2. The only solution of Ax = 0 is x = 0. A has an inverse.1) > x = A\b As you will find. Theorem 2 Suppose A is a square (n × n) matrix.27]’ > A*x This x is a solution! It is not hard to see that there are endless possibilities for solutions of this equation. Still unfortunately. or infinitely many solutions. in most engineering applications we would want to have exactly one solution. 36 . that is not the only solution. 6. 1. its columns (and rows) are all the same. The equation Ax = b has exactly one solution for any b. 2. The rows of A are linearly independent. A.1) > x = [ 1 0 0 0]’ > A*x So the system Ax = b does have a solution. there is no solution to the equation Ax = b. This unfortunate circumstance is mostly the fault of the matrix. 4. det(A) = 0. The following two theorems show that having one and only one solution is a property of A. Try: > x = [ 0 1 0 0]’ > A*x We see that this x is also a solution. 3. 5.

6. To see how the two theorems work. make A2) :  1 A1 =  4 7 and two vectors: define two matrices (type in A1 then scroll up and modify to 2 5 8  3 6 . attempt to solve all the possible equations Ax = b: > x = A1\b1 > x = A1\b2 > x = A2\b1 > x = A2\b2 As you can see. Its size (norm) is an indication of how close we have come to solving Ax = b.37 On the other hand. 8  0 b1 =  3  . 3. 5. 6  First calculate the determinants of the matrices: > det(A1) > det(A2) Then attempt to find the inverses: > inv(A1) > inv(A2) Which matrix is singular and which is non-singular? Finally. 4. . 2. The equation Ax = 0 has solutions other than x = 0. For a linear system of equations we define the residual of an approximate solution. The rows of A are linearly dependent. 6    3 6 . (10. a matrix that does not have these properties is called singular. The columns of A are linearly dependent as vectors. det(A) = 0. The residual vector Recall that the residual for an approximate solution x of an equation f (x) = 0 is defined as r = f (x). 9 1 2 A2 =  4 5 7 8  1 b2 =  3  .1) Notice that r is a vector. A does not have an inverse. The equation Ax = b has 0 or ∞ many solutions depending on b. x by r = Ax − b. The following are all equivalent: 1. but equation involving a singular matrix are more complicated. Theorem 3 Suppose A is a square matrix. It is a measure of how close the equation is to being satisfied. equations involving the non-singular matrix have one and only one solution.

n. 10. 500. 7x1 + 8x2 + 9x3 = 14 . and (b) Write a Matlab script program that calls mysolvecheck 5 times each for n = 5. . solves the linear system Ax = b.. 10. and makes a log-log plot of the average e vs. Turn in the plot and the two programs. SOME FACTS ABOUT LINEAR SYSTEMS Exercises 10.2 (a) Write a Matlab function program mysolvecheck with input a number n that makes a random n × n matrix A and a random vector b. . find all the solutions (if any) of the following linear system using the augmented matrix and Gaussian elimination: x1 + 2x2 + 3x3 = 4.38 LECTURE 10. 100. . records and averages the results. 50. and outputs that number as the error e. 4x1 + 5x2 + 6x3 = 10.1 By hand. calculates the norm of the residual r = Ax − b. . 1000.

Consider the following exaggerated example. is a measure of how bad a matrix is.Lecture 11 Accuracy. This seems fine until you realize that backsolving the unrounded system gives: x1 = −1. . condition number.001. Row Pivoting A way to fix the problem is to use pivoting. Suppose that our computer can only store 2 significant digits and it is asked to do Gaussian elimination on: .001 0 1 −998 1 −1000 3 −2995 3 −3000 x2 = 3. The first. but rounding to 2 digits. our computer would store this as: . x2 = 3. pivoting. Condition Numbers and Pivoting In this lecture we will discuss two separate issues of accuracy in solving linear systems.001 1 Doing the elimination exactly would produce: . 1 3 2 5 . is a method that ensures that Gaussian elimination proceeds as accurately as possible. the solutions are unstable with respect to small changes in data. This limits numbers to a fixed number of significant digits and implies that after even the most basic calculations. The effect of rounding All computers store numbers as finite strings of binary floating point digits.001 0 Backsolving this reduced system gives: x1 = 0. Since switching rows of the augmented matrix just corresponds to switching the order of the equations. The second. . We will see that if a matrix has a bad condition number. 39 . which means to switch rows of the matrix. rounding must happen.

after each column is completed. Consider the following augmented matrices: 1 1/2 3/2 1 1/2 3/2 and . Condition number In some systems. Switch it (and the entire row) with the one with greatest absolute value. 1)′ which are not close at all despite the fact that we did the calculations exactly. To pivot we switch rows so that the largest entry in a column is the one used to eliminate the others. 0 −2 −10 −10 Pivoting the 2nd and 3rd rows would produce:  1 −2 3  0 −2 −10 0 1 6  4 −10  . .001. but two different input vectors: b1 = (3/2.998 2 1 5 3 5 2. compare the diagonal element of the next column with all the entries below it. ACCURACY. This poses a new problem: some matrices are very sensitive to small changes in input data.40 LECTURE 11. the first column is finished and before doing the second column. In bigger matrices. For example in the following matrix. pivoting should occur since | − 2| > |1|:   1 −2 3 4  0 1 6 7 . Exact elimination would produce: .001 1 2 0 . CONDITION NUMBERS AND PIVOTING no harm is done: 1 . The reason this worked is because 1 is bigger than 0. 7 x2 = 3. The extent of this 3/2 1/4 and 1 1/2 0 1/12 3/2 1/12 . 5/6)′ which are pretty close to one another. 5 3 . 3)′ and x2 = (1.995 . Eliminating exactly we get: 1 1/2 0 1/12 Now solving we find: x1 = (0. problems occur even without rounding. 1/2 1/3 1 1/2 1/3 5/6 Here we have the same A. Storing this result with only 2 significant digits gives: 1 2 0 1 Now backsolving produces: x1 = −1. Notice that this matrix does not need pivoting. which is exactly right. 1)′ and b2 = (3/2. You would expect then that the solutions x1 and x2 would also be close.

2969. How do A−1 and B −1 differ? Explain how this happened.0. they are actually independent things.) This is obviously not very good for engineering where all measurements. Why was the first answer so simple? Why do the two answers differ by so much? . In Matlab enter: > H = hilb(2) which should result in the matrix above. Is there a solution to the problem of bad condition numbers? Usually. Next try the matrix: > A = [ 1. Then:   Relative error of output |δx|/|x| .41 sensitivity is measured by the condition number.2161 . bad condition numbers in engineering contexts result from poor design. the engineering solution to bad conditioning is redesign. If det(A) = 0 then the matrix is singular. find the determinant and condition number of the matrix [1e-10.2969 → 1. Set b1 = [1. Explain exactly what happened.0. Let B be the matrix obtained from A by rounding off to three decimal places (1.1e-10] and the matrix [1e+10. changes in the input data get multiplied by the condition number to produce changes in the outputs. in the solution x. Repeat the process but with a vector b2 obtained from b1 by rounding off to three decimal places.1441 .2161 . So.2161] and do x = A\b1 . det(A) ≈ 0.0. because it means the matrix is almost singular. The definition of condition number is: consider all small changes δA and δb in A and b and the resulting change. δx.1441] > cond(A) For this matrix small errors in the input can get magnified by 2.1e-10]. Finally. is also bad. Exercises 11.8648 . which is bad because it implies there will not be a unique solution.0. Although det(A) ≈ 0 generally indicates that the condition number will be large.297). To see this. 0. find the determinant of the matrix A above: > det(A) which will be small. cond(A) ≡ max  |δA| |δb|  = max Relative error of inputs + |A| |b| Put another way. constants and inputs are approximate. Matlab produces the condition number of a matrix with the command: > cond(H) Thus for this matrix small errors in the input can get magnified by 19 in the output. Thus a high condition number is bad.5 × 108 in the output! (We will see this happen in the exercise. The case here.1 Find the determinant and inverse of: A= 1. It implies that small errors in the input can cause large errors in the output.2969 . .8648 .2969 0. Find the determinant and inverse of B.

Create a numerical version via Cn = double(Cs) and define the two vectors d1 = [4. ACCURACY. Explain the results. 2] > x = B\c > pretty(x) Next input the matrix: Cs = 1 2 2 4 symbolically as above. 8] and d2 = [1. sin(sym(3)) sin(sym(4))] > c = [1. Does the symbolic or non-symbolic way give more information? . Cs*x = d2. CONDITION NUMBERS AND PIVOTING 11.2 Try > B = [sin(sym(1)) sin(sym(2)). Cn*x = d1.42 LECTURE 11. 1]. Solve the systems Cs*x = d1. and Cn*x = d2.

L =  2 1 0 . For instance. To eliminate the third row. We record this fact with a (0). first column. 0 0 2 0 −2 1 The first step of Gaussian elimination is to subtract 2 times the first row from the second row. a structure must be tested under several different loads. and A is big. As in the example of a truss (9. This leads to:   1 −2 3  (2) −1 6 .:     1 −2 3 1 0 0 U =  0 −1 6  . the loading in such a problem is usually represented by the vector b. 2. we need to subtract −2 times the second row from the third row. i. 0 2 −10 There is already a zero in the lower left corner.e. which in effect records the steps of Gaussian elimination. Gaussian elimination with pivoting is the most efficient and accurate way to solve a linear system. In order to make it clear that it is a record of the step and not an element of A. Most of the work in this method is spent on the matrix A itself. one needs to solve Ax = b for many different vectors b.2). Consider the matrix:   1 −2 3 A =  2 −5 12  .e. the second row.Lecture 12 LU Decomposition In many applications where linear systems appear. and let L be the lower triangular matrix with the records and ones on the diagonal. not just one. we will put the multiplier. LU decomposition The main idea of the LU decomposition is to record the steps used in Gaussian elimination on A in the places where the zero is produced. into the place it was used to make a zero. second column. then we would like to avoid repeating the steps of Gaussian elimination on A for every different b. so we don’t need to eliminate anything there. In order to record what we have done. i. If we need to solve several different systems with the same A. Recording the −2 in the spot it was used we have:   1 −2 3  (2) −1 6  . 0 2 −10 43 . we will put it in parentheses. (0) (−2) 2 Let U be the upper triangular matrix produced. This can be accomplished by the LU decomposition.

LU DECOMPOSITION then we have the following mysterious coincidence:     1 −2 1 0 0 1 −2 3 LU =  2 1 0   0 −1 6  =  2 −5 0 2 0 −2 1 0 0 2 Thus we see that A is actually the product of L and U . . . L and U such that: P A = LU. . (12. . . . Substituting LU for P A we get: LU x = d. In Matlab this would work as follows: > A = rand(5. . . . .1). When a matrix can be written as a product of simpler matrices. . .44 LECTURE 12. . . 0 1 0 would be the pivot matrix if the second and third rows of A are switched by pivoting. .1) The pivot matrix P is the identity matrix. . . . . . . . Here L is lower triangular and U is upper triangular. While this may not seem like a big savings. . . Matlab will produce an LU decomposition with pivoting for a matrix A with the following command: > [L U P] = lu(A) where P is the pivot matrix.1) > d = P*b > y = L\d > x = U\y > rnorm = norm(A*x . To use this information to solve Ax = b we first pivot both sides by multiplying by the pivot matrix: P Ax = P b ≡ d. . then an LU decomposition for A consists of three matrices P . . Check the result. .  3 12  = A. Then we need only to solve two back substitution problems: Ly = d and U x = y. . . . you can start at the third line. . Repeat the sequence for a new right hand side: c = randn(5. . For instance:   1 0 0 P =  0 0 1 . .b) . . . . . . . . . with the same rows switched as the rows of A are switched in the pivoting. . . . We can then solve for any other b without redoing the LU step. . we call that a decomposition of A and this one we call the LU decomposition. it would be if A were a large matrix from an actual application. . . . . −10 Using LU to solve equations If we also include pivoting.5) > [L U P] = lu(A) > b = rand(5.

5 4 4 .45 Exercises 12. .1 Solve the systems below by hand using the LU decomposition. x2.n)) and Hilbert matrices (hilb(n)) with n really big and document the results. In each of the two problems.the norm of the residual using the LU method % x2 -. b= b= 0 −3 1 4 3 5 3 2 12.the solution using the LU method % e1 -.the solution using the built-in method % e2 -. % Inputs: A -.the right-hand vector % Outputs: x1 -.the norm of the residual using the built-in method format long Test the program on both random matrices (randn(n. . (a) A = (b) A = 2 .b) % Solves linear systems using the LU decomposition with pivoting % and also with the built-in solve function A\b. check by hand that LU = P A and Ax = b. e1.2 Finish the following Matlab function program: function [x1. Pivot if appropriate.the matrix % b -. e2] = mysolve(A.

fn (x1 . . x2 . it must do the calculations automatically and numerically.Lecture 13 Nonlinear Systems . we can write the system of equations (13. we need to start with an initial guess x0 . . we can usually find solutions to a system of equations when the number of unknowns matches the number of equations. the boat obtains differences of distances to the transmitters. . this must be overcome by 46 .2) For convenience we can think of (x1 . . the harder it is to find a good initial guess. xn ) as a vector x and (f1 .Newton’s Method An Example The LORAN (LOng RAnge Navigation) system calculates the position of a boat at sea using signals from fixed transmitters. x3 . . . . . Thus. xn ) = 0 f3 (x1 . Vector Notation In general. f2 . . xn ) = 0. . In practice. . We could do this by hand. An example of such equations from [2] are: y2 x2 − = 1 and 1862 3002 − 1862 (x − 300)2 (y − 500)2 − = 1. we wish to find solutions to systems that have the form: f1 (x1 . . . xn ) = 0 f2 (x1 .2) simply as: f (x) = 0. . . fn ) as a vectorvalued function f . . . In theory. x3 . . . 2 279 5002 − 2792 (13.e. . As in Newton’s method for one variable. i. x3 . x2 . but for a navigational system to work well. From the time differences of the incoming signals. x2 . we wish to find a vector that makes the vector function equal to the zero vector. x2 . . x3 . . would require solving a 4 degree polynomial equation. This leads to two equations each representing hyperbolas defined by the differences of distance of two points (foci). . x3 . With this notation. We note that the Global Positioning System (GPS) works on similar principles and must do similar computations. the more variables one has. . . (13. x2 . . xn ) = 0 .1) Solving two quadratic equations with two unknowns.

(13. . For subsequent steps. However. in practice we never use the inverse of a matrix for computations. we can obtain our improved estimate x1 by: x1 = x0 + ∆x. • Let xi+1 = xi + ∆x ..47 using physically reasonable assumptions about the possible values of a solution. i.  . which can be solved efficiently and accurately.. ∂fn ∂xn (x0 ) . we can do the following. . . Once x0 is chosen.  ∂fn ∂fn ∂fn ∂x1 (x0 ) ∂x2 (x0 ) ∂x3 (x0 ) partial derivative of the components . From Calculus. this equation is just a system of linear equations. we have the following process: • Solve Df (xi )∆x = −f (xi ) for ∆x.. so let’s choose x1 so that f (x0 ) + Df (x0 )(x1 − x0 ) = 0. Newton’s method was derived by considering the linear approximation of the function f at the initial guess x0 . . the following is the linear approximation of f at x0 .    (13.4) Since Df (x0 ) is a known matrix and −f (x0 ) is a known vector. we can solve this equation by x1 = x0 − (Df (x0 ))−1 f (x0 ).     . let ∆x = x1 − x0 . Since Df (x0 ) is a square matrix. . .3) Newton’s Method We wish to find x that makes f equal to the zero vectors. The formula is the vector equivalent of the Newton’s method formula we learned before. . . .  . provided that the inverse exists... for vectors and vector-valued functions: f (x) ≈ f (x0 ) + Df (x0 )(x − x0 ). so we cannot use this formula directly. Once we have the solution vector ∆x. Specifically:  ∂f ∂f ∂f 1 (x0 ) ∂x1 (x0 ) ∂x1 (x0 ) 2 3  ∂x1  ∂f2 ∂f2 ∂f2   ∂x1 (x0 ) ∂x2 (x0 ) ∂x3 (x0 ) Df (x0 ) =  . Linear Approximation for Vector Functions In the single variable case.e. . First solve the equation Df (x0 )∆x = −f (x0 ). take advantage of engineering knowledge of the problem. Here Df (x0 ) is an n × n matrix whose entries are the various of f . Rather. ∂f1 ∂xn (x0 ) ∂f2 ∂xn (x0 )  . ...

7) Now that we have the equation in the vector form. .5. y) = (1. x2 ) = x3 − x1 + 1. but only to about 7 decimal places. x2 ) = x3 + x2 − 1 1 or f (x) = f2 (x1 . An Experiment We will solve the following set of equations: x3 + y = 1 y 3 − x = −1. at (x. While that would be close enough for most applications. x2 = y and f1 (x1 . x = [.NEWTON’S METHOD x3+y=1. write the following script program: format long f = inline(’[ x(1)^3+x(2)-1 . y) = (1. 0) is the only solution (Figure 13. By graphing both of the equations you can also see that (1.m and run it.x) Save this program as myfsolve.1). (13. one would expect that we could do better on such a simple problem.6) (13. 0). We can put these equations into vector form by letting x1 = x. NONLINEAR SYSTEMS ..48 LECTURE 13. 2 x3 + x2 − 1 1 x3 − x1 + 1 2 . 0) is a solution of this system. You will see that the internal Matlab solving command fsolve approximates the solution.1: Graphs of the equations x3 + y = 1 and y 3 − x = −1.5) You can easily check that (x. There is one and only one intersection. x(2)^3-x(1)+1 ]’). (13.5] x = fsolve(f. 3 y −x=−1 4 3 2 1 0 y −1 −2 −3 −4 −4 −3 −2 −1 0 1 2 3 4 x Figure 13.

. % solve for increment x = x + Dx % add on to get new guess f(x) % see if f(x) is really zero end Save and run this program (as mymultnewton) and you will see that it finds the root exactly (to machine precision) in only 6 iterations. % the vector function Df = inline(’[3*x(1)^2. % the matrix of partial derivatives x = [. may need adjusting f = inline(’[x(1)^3+x(2)-1 .2). 3*x(2)^2]’). 1 .49 Next we will implement Newton’s method for this problem. -1. By trying different starting vectors.5.. find at least three different solutions. (b) Construct the matrix of partial derivatives Df in (13. Why is this simple program able to do better than Matlab’s built-in program? Exercises 13.) Think of at least one way that the navigational system could determine which solution is correct. (There are actually four solutions.3).1) into the function form (13.1 (a) Put the LORAN equations (13.5] % starting guess for i = 1:n Dx = -Df(x)\f(x). x(2)^3-x(1)+1]’). (c) Adapt the mymultnewton program to find a solution for these equations. Modify your myfsolve program to: % mymultnewton format long n=8 % set some number of iterations.

we must have: det(A − λI) = 0.1) Geometrically this means that Av is in the same direction as v.2) It would be nice to factor out the v from the right-hand side of this equation. then by Theorem 3 in Lecture 10 the matrix (A − λI) must be singular. We can check this by: > v1 = v(:. since Iv = v.1) > A*v1 > e(1. Notice that Equation (14. This is called the characteristic equation. (14. However. but we can’t because A is a matrix and λ is a number.1)*v1 Finding Eigenvalues for 2 × 2 and 3 × 3 If A is 2 × 2 or 3 × 3 then we can find its eigenvalues and eigenvectors by hand. Matlab has a built-in routine for finding eigenvalues and eigenvectors: > A = pascal(4) > [v e] = eig(A) The results are a matrix v that contains eigenvectors as columns and a diagonal matrix e that contains eigenvalues on the diagonal. (14. we can do the following: Av − λv = Av − λIv = (A − λI)v =0 (14. By the same theorem. since multiplying a vector by a number changes its length.4) 50 . (14.Lecture 14 Eigenvalues and Eigenvectors Suppose that A is a square (n × n) matrix. We say that a nonzero vector v is an eigenvector (ev) and a number λ is its eigenvalue (ew) if Av = λv. but not its direction.3) If v is nonzero.1) can be rewritten as: Av − λv = 0.

Thus this equation has infinitely many solutions. Thus Gaussian elimination would produce all zeros on the bottom row. y) = (2. This comes from noticing that (x. A − λI is calculated as in the following example: A − λI = = = The determinant of A − λI is then det(A − λI) = (1 − λ)(5 − λ) − 4 · 3 = −7 − 6λ + λ2 . The characteristic equation det(A − λI) = 0 is simply a quadratic equation: λ2 − 6λ − 7 = 0. For λ2 = −1. (14. = −1 For a 3 × 3 matrix we could complete the same process. Now to find the corresponding ev’s we return to the equation (A − λI)v = 0. which are the ew’s.e. i. These are the ew’s of the matrix A. For simplicity we can let −2 v2 = . The det(A − λI) = 0 would be a cubic polynomial and we would expect to usually get 3 roots. we can let v1 = 2 3 . this is okay.6) 1 4 3 5 1 4 3 5 1−λ 3 −λ − 4 5−λ λ 0 1 0 0 1 0 λ . For λ1 = 7.51 For a 2 × 2 matrix. we only need to find one of the ev’s. infinitely many ev’s. 1 One can always check an ev and ew by multiplying: Av1 = Av2 = 1 3 1 3 4 5 4 5 2 3 −2 1 = = 14 21 2 −1 =7 2 3 = 7v1 −2 1 and = −1v2 . Since only the direction of the ev matters. (14.8) (14. (14. Since the second row of the augmented matrix represents the equation 3x − 2y = 0. .5) Notice that the first and second rows of this matrix are multiples of one another. 3) is a solution of 3x − 2y = 0. the equation for the ev (A − λI)v = 0 is equivalent to the augmented matrix −6 3 4 0 −2 0 . Once again the first and second rows of this matrix are multiples of one another.7) The roots of this equation are λ1 = 7 and λ2 = −1. (A − λI)v = 0 is equivalent to the augmented matrix: 2 3 4 0 6 0 .

Exercises 14. β. EIGENVALUES AND EIGENVECTORS Larger Matrices For a n × n matrix with n ≥ 4 this process is too long and cumbersome to complete by hand. often is related to the frequency of an oscillation. These methods rely on the geometric meaning of ev’s and ew’s rather than solving algebraic equations.52 LECTURE 14. A matrix is symmetric if it is equal to its own transpose. even when the matrix only contains real numbers. The most common such type of matrix is the symmetric matrix. We will overview these methods in Lecture 16. i. 1 3 3 −5 is symmetric and so we know beforehand that its ew’s will be real. For example. not complex. this process is not well suited even to implementation on a computer program since it involves determinants and solving a n-degree polynomial. Complex Eigenvalues It turns out that the eigenvalues of some matrices are complex numbers. This is because of Euler’s formula eα+iβ = eα (cos β + i sin β). Further.e. the imaginary part of the ew. For n ≥ 4 we need more ingenious methods. it is symmetric across the diagonal.e.2 = α ± iβ. . In applications.2 Find the eigenvalues and eigenvectors of the following matrix by hand: B= Can you guess the ew’s of the matrix C= a b −b a ? 1 2 −2 1 . The corresponding ev’s must also come in conjugate pairs: w = u ± iv. Certain kinds of matrices that arise in applications can only have real ew’s and ev’s. i. When this happens the complex ew’s must occur in conjugate pairs. 14.1 Find the eigenvalues and eigenvectors of the following matrix by hand: A= 2 1 1 2 .: λ1.

and note the displacement of the walls is zero. the general solution is: x(t) = c1 v1 sin k λ1 t + φ 1 m + c2 v2 sin k λ2 t + φ 2 m (15. 53 . A simple nontrivial vibration problem is the motion of two objects with equal masses m attached to each other and fixed outer walls by equal springs with spring constants k.1) Dividing both sides by m we can write these equations in matrix form: ¨ x = −Ax. (15.1.2) (15.1: Two equal masses attached to each other and fixed walls by equal springs. Each mass experiences forces from the adjacent springs proportional to the stretch or compression of the spring.3) For this type of equation. as shown in Figure 15.Lecture 15 An Application of Eigenvectors: Vibrational Modes One application of ew’s and ev’s is in the analysis of vibration problems. We can find the ew’s and ev’s of B using Matlab: > B = [2 -1 . the two entries are equal. Let x1 denote the displacement of the first mass and x2 the displacement of the second. v1 . v2 . Ignoring any friction. In the first eigenvector. This represents the mode of oscillation where the two masses move in sync with each other. -1 2] > [v e] = eig(B) This should produce a matrix v whose columns are the ev’s of B and a diagonal matrix e whose entries are the ew’s of B. We can interpret the ew’s as the squares of the frequencies of oscillation. = kx1 − 2kx2 (15.4) where λ1 and λ2 are ew’s of B with corresponding ev’s v1 and v2 . Newton’s law of motion ma = F . has the u E x1 u E x2 Figure 15. where A= k k B= m m 2 −1 −1 2 . leads to: m¨1 x m¨2 x = −k(x1 − 0) = +k(x2 − x1 ) −k(x2 − x1 ) +k(0 − x2 ) = −2kx1 + kx2 . The second ev.

The  2 B =  −1 0 Find the ev’s and ew’s as above.54LECTURE 15. There are three different modes. Thus the two frequencies are: k m and 3k . corresponding matrix B would be:  −1 0 2 −1  . Notice that the frequency for anti-sync motion is 3 times that of synchronous motion. m We can do the same for three equal masses. This represents the mode where the two masses oscillate in anti√ synchronization. Which of the two modes is the most dangerous for a structure or machine? It is the one with the lowest frequency because that mode can have the largest displacement. m m Thus we can conclude that A has the same ev’s as B. To get the frequencies for the matrix A = k/mB. Note that the two modes actually move at different speeds. Sometimes this mode is called the fundamental mode. same entries but opposite signs.2: Two vibrational modes of a simple oscillating system. but the ew’s are multiplied by the factor k/m. Interpret them from the ev’s. −1 2 . In the left mode the weights move together and in the right mode they move opposite. notice that if vi is one of the ev’s for B then: Avi = k k Bvi = λi vi . AN APPLICATION OF EIGENVECTORS: VIBRATIONAL MODES u u u u u u u u u u u u u u u u u u u u Figure 15.

(b) Three masses with m1 = 1. . m2 = 2 and m3 = 3. Interpret the modes.55 Exercises 15.1 Find the frequencies and modes for 4 equal masses with equal springs. How does this effect the modes? (a) Two masses with m1 = 1 and m2 = 2.2 Find the frequencies and modes for non-identical masses with equal springs in the following cases. 15.

1) that x1 = Av1 + Av2 1 = v1 + 2v2 . which we now introduce.. e. The Power Method In the command window of Matlab enter the following: > A = hilb(5) > x = ones(5. Notice that there is less change between the second two.g. Repeat the last three commands over and over until the values stop changing.2) v1 + 2 v2 . The last column in v is x/norm(x). k 56 . 3 Thus the v1 part is shrunk while the v2 is stretched.1) (16. Let x0 be any vector which is a combination of v1 and v2 . x0 = v1 + v2 .1) > x = A*x > el = max(x) > x = x/el Compare the new value of x with the original.Lecture 16 Numerical Methods for Eigenvalues As mentioned above. If we repeat this process k times then: xk = Axk−1 = Ak x0 = 1 3 k (16. Below we explain why our commands gave this eigenvalue and eigenvector. The numerical methods that are used in practice depend on the geometric meaning of ew’s and ev’s which is equation (14. Now let x1 be A times x0 . Now try the command: > [v e] = eig(A) The last entry in e should be the final el we computed. You have completed what is known as the Power Method. Compare the newest value of x with the previous one and the original. The essence of all these methods is captured in the Power method. It follows from (14.1). For illustration consider a 2 × 2 matrix whose ew’s are 1/3 and 2 and whose corresponding ev’s are v1 and v2 . Repeat the last three lines (you can use the scroll up button). the ew’s and ev’s of an n × n matrix where n ≥ 4 must be found numerically instead of by hand.

. Summarizing the Power Method: • Repeatedly multiply x by A and divide by the element with the largest absolute value. the mode (ev) with the lowest frequency (ew) is the most dangerous for the machine or structure. i. the program must use flow control to determine the correct number.57 Clearly. In this section we introduce a method. the Matlab code above will work since max(v) will be the element with the largest absolute value. Next we consider an even more specialized class for which the eigenvalues are not only real. xk grows in the direction of v2 and shrinks in the direction of v1 . One class of matrices that appear often in applications and for which the eigenvalues are always real are called the symmetric matrices. This is the principle of the Power Method. The Power Method gives us instead the largest ew. the Inverse Power Method which produces exactly what is needed. If the matrix and starting vector are real then the power method can never give a result with an imaginary part. Positive-Definite Matrices As noted in the previous paragraph. A is symmetric with respect to reflections about its diagonal. Symmetric. • The vector converges to the corresponding ev.e. which is the least important frequency. Eigenvalues with imaginary part mean the matrix has a rotational component. but positive. A matrix is symmetric if: A′ = A. The Inverse Power Method In the application of vibration analysis. When this is the case. In summary: • If A is symmetric then its eigenvalues are real. the power method can fail if A has complex eigenvalues. In many applications this quantity will necessarily be positive for physical reasons. • The element of largest absolute value converges to largest absolute ew. • If A is symmetric positive definite. vectors multiplied by A are stretched most in the direction of the ev whose ew has the largest absolute value. In applications where the dominant ew may be negative. A symmetric matrix is called positive definite if for all vectors v = 0 the following holds: Av · v > 0. so the eigenvector would not settle down either. The ew with the largest absolute value is called the dominant ew. Note that this logic only works when the eigenvalue largest in magnitude is real. then its eigenvalues are positive numbers.

% LU decomposition of A with pivoting [L U P] = lu(A). we do not really want to calculate the inverse of A directly if we can help it. • The ev’s for A and A−1 are the same.the smallest absolute eigenvalue of A and % v -. end e = 1/el. Here as always. the smallest ew and its ev correspond exactly to the frequency and mode that we are most interested in. v = U\y. which is an ev for both A−1 and A.the corresponding eigenvector. Since iterating this process involves solving a linear system with the same A but many different right hand sides. m = min(v). Thus if we apply the Power Method to A−1 we will obtain the largest absolute ew of A−1 . % main loop of Inverse Power Method for i = 1:n pv = P*v. it is a perfect time to use the LU decomposition to save computations. M = max(v). % use the biggest in absolute value if abs(M) >= abs(m) el = M.a square matrix % n -. NUMERICAL METHODS FOR EIGENVALUES The following facts are at the heart of the Inverse Power Method: • If λ is an ew of A then 1/λ is an ew for A−1 .1).1). . i. the one that can do the most damage. which can be done efficiently and accurately. Fortunately. The following function program does n steps of the Inverse Power Method. % make an initial vector with ones m = size(A. which is exactly the reciprocal of the smallest absolute ew of A.58 LECTURE 16. Recall that in the application of vibration mode analysis.n) % performs n steps of the inverse power method on A % Inputs: % A -. We will also obtain the corresponding ev. function [v e] = myipm(A. end v = v/el.the number of iterations to do % Outputs: % e -. y = L\pv.e. multiplying xi by A−1 to get xi+1 is equivalent to solving the system: Axi+1 = xi . else el = m. v = ones(m.

applies the power method until the change in the vector is less than the tolerance.3) (16. (b) Test your program on the matrices A and B in the previous exercise.4)  −2 1 0 B =  1 −2 1  0 1 −2 16. A=  1 2 3 4 (16. .2 (a) Write a Matlab function program mypm that inputs a matrix and a tolerance.59 Exercises 16. perform one interation of the power method by hand starting with a vector of all ones. State the resulting approximations of the eigenvalue and eigenvector.1 For each of the following matrices. and outputs the number of steps used and the estimated eigenvalue and eigenvector.

H1 . However. because its column vectors are mutually orthogonal. A matrix Q with the property that its transpose equals its inverse is called an orthogonal matrix. . There is a built-in QR decomposition in Matlab which is called with the command: [Q R] = qr(A). H2 . i i Thus each Hi+1 is a related to Hi by an (orthogonal) simularity transformation and so they have the same ew’s as A. First the Hessian matrix H is obtained from A by a series of similarity transformation. . there is a better method for obtaining all the ew’s called the QR method. we note the following theory behind it for those with more familiarity with linear algrebra. thus it has the same ew’s as A. This is the basis of all modern ew software. The QR method uses the fact that any square matrix has a QR decomposition. if we denote by H0 . The diagonal of H will converge to the eigenvalues.Transform A into a tridiagonal matrix H. the sequence of matrices produced by the iteration. . The QR method consists of iterating following steps: . That is. While both of these methods can be modified to give more ew’s and ev’s.multiply Q and R together in reverse order to form a new H.decompose H in QR. Secondly.Lecture 17 The QR Method* The Power Method and Inverse Power Method each give us only one ew–ev pair.. so we summarize it briefly here. . including Matlab. We details of what makes this method converge are beyond the scope of the this book. . Thus the following program implements QR method until it converges: 60 . for any A there are matrices Q and R such the A = QR where Q has the property Q−1 = Q′ and R is upper triangular. then Hi+1 = Ri Qi = Q−1 Qi Ri Qi = Q′ Hi Qi .

while change > 0 Eold = E. if m ~= n warning(’The input matrix is not square. change = norm(E . with the maximum iterations set to 1000.61 function E = myqrmethod(A) [m n] = size(A). . Run this program and compare the results with Matlab’s built in command: > format long > format compact > A = hilb(5) > Eqr = myqrmtheod(A) > Eml =eig(A) Exercises 17. Turn in a printout of your program and a brief report on the experiment. steps = 0.1 Modify myqrmethod to stop after a maximum number of iterations. Use the modified program. H = R*Q. The really hard calculations are contained in the built-in command qr(A). change = 1. Use the norm to compare the results to the ew’s obtained from Matlab’s built-in program eig. on the matrix A = hilb(n) with n equal to 10. E = diag(H).’) return end H = hess(A). steps = steps +1. [Q R] = qr(H). and 200. end steps As you can see the main steps of the program are very simple. 50.Eold). E = diag(H).

Lecture 18 Iterative solution of linear systems* Newton refinement Conjugate gradient method 62 .

and Iv = v Inverse matrix: AA−1 = I and A−1 A = I Norm of a matrix: |A| ≡ max|v|=1 |Av| A matrix may be singular or nonsingular. Vibrational modes: Eigenvalues are frequencies squared. LU factorization: P A = LU. Characteristic equation: det(A − λI) = 0 Equation of the eigenvector: (A − λI)v = 0 Complex ew’s: Occur in conjugate pairs: λ1. See Lecture 10. 63 . A big condition number is bad. IA = A.Review of Part II Methods and Formulas Basic Matrix Theory: Identity matrix: AI = A. in engineering it usually results from poor design. Solving Process: Gaussian Elimination produces LU decomposition Row Pivoting Back Substitution Condition number: |δx|/|x| cond(A) ≡ max  |δA| |δb|  = max |A| + |b|   Relative error of output Relative error of inputs . Eigenvectors are modes. Solving steps: Multiply by P: d = P b Forwardsolve: Ly = d Backsolve: U x = y Eigenvalues and eigenvectors: A nonzero vector v is an eigenvector (ev) and a number λ is its eigenvalue (ew) if Av = λv.2 = α ± iβ and ev’s must also come in conjugate pairs: w = u ± iv.

Manually enter a matrix. . . . component-wise exponentiation. . . . . identity matrix > D = ones(5. . . . . . . . .The ev’s for A and A−1 are the same. . . . . . . . . . . . . . . . . . . . . random matrix with uniform distribution in [0. . . . . . . . . .Multiply Q and R together in reverse order to form a new H. . 7 6 5. . . . 4 3 2] > B*A . . . . . . . . . . . . . . . . . . . add matrices. . . . . . .5) . . . > B. . . . . . . . multiply a matrix by a scalar. . . . . . . . . . . . . . . . . . . > B.The element of largest absolute value converges to largest absolute ew. . . . . . . . . . . . . . . . . .5) > O = zeros(10. . . . . . . . . . 1]. . . . . . . . . . .Apply power method to A−1 . . . . . . . . .5) . . . . . . . . . . . . . . . . . . .Convergence assured for a real symmetric matrix. . . . . . . .*B . . . . . . . . gives the dimensions (m × n) of A. .Repeatedly multiply x by A and divide by the element with the largest absolute value. . . . . . . . . . QR method: . . . . . . . . . . > C = randn(5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > hilb(6) > pascal(5) General matrix commands: > size(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .If λ is an ew of A then 1/λ is an ew for A−1 .Repeat . . . . but not for an arbitrary matrix. . . . . . . . . . . . . multiply B times A. . . . . . . . . . . 0 1 -9 0] . . . . . . . . . .If A is positive definite. . . . . . . . . Symmetric and Positive definite: . . . . . . . . . . . . . . . . . . . . .10) > C = rand(5. . . . . . . . . . . . . . . . . . . .Symmetric: A = A′ . . > 2*A . . . . . . . . . . . . . . .Decompose H in QR. . . . . . . . . . . . . . . . . . then its ew’s are positive. gives the norm of the matrix. component-wise multiplication. . . . . . . Special matrices: > I = eye(3) . . . . . . . .Use solving rather than the inverse. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > A + 3 . . . . . . . . . . . . . . . . random matrix with normal distribution. . . . . . . . . . . which may not have real eigenvalues at all. . . . . . . . . . . . . . . . . . . .If A is symmetric its ew’s are real. . . . . . add 3 to every entry of a matrix. . . . . .64 REVIEW OF PART II Power Method: . . . . . .The diagonal of H will converge to the ew’s of A. . . .Positive definite: Ax · x > 0. . . . . . . . . . . > A + A . . . .The vector converges to the corresponding ev. . . . . . . . . . . .^3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Inverse Power Method: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Matlab Matrix arithmetic: > A = [ 1 3 -2 5 . . . . . . -1 -1 5 4 . > u = [ 1 2 3 4]’ > A*u > B = [3 2 1. . . . . . . . . . . . . . . . . . . . . . . . . . .Transform A into H the Hessian form of A. . . . . . . . . . . . . . . . . . . . > norm(C) . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . the maximum of each row. . . . . . . . . . . . . . . . . . . . . . . . . . . A = R′ R. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > H = hess(C) transform into a Hessian tri-diagonal matrix. . . . . inverse of the matrix. . . . . . . . . inv(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . Matrix decompositions: > [L U P] = lu(C) > [Q R] = qr(C) > [U S V] = svd(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . diag(C) . . . . . . . > [U T] = schur(C) . . . . . . . . . . . . . . . . . . . . . . . . max(C) . . . . > R = chol(C*C’) Cholesky decomposition of a symmetric. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . but we did not use it). . . . . . . . . . . . . . . . the minimum in each row. . . . . . positive definite matrix. . . . . . . . . . . . . . . . . . . . . . . . . . . . Schur Decomposition A = U ′ T U . . . singular value decomposition (important. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .65 > > > > > > > > det(C) . . . . . . . . . . . . . . . . . . . . the determinant of the matrix. . . . . . . . . . . . . . . . . . . . . . . . just the diagonal elements. . . . . . . . . . . . . . . . . . . . . . . . . mean(C) . . . the average of each row. . . . . . . . . . . . . . . . . . . . C’ . . . . . . . . transpose of the matrix. . . . . . . . . . . . min(C) . . . . . . . . . . . . . . . . sum(C) . . . . . . . . sums each row. . . . . . A matrix that has the same eigenvalues as A. . . . . .

66 REVIEW OF PART II .

Ohio University. Todd Young and Martin Mohlenkamp. Mathematics Department.Part III Functions and Data c Copyright. 2007 .

This should produce a small window with several fitting options. The others do not.31 ] and plot the data with: > plot(t1.5 1.31 We can input this data into Matlab as: > t1 = 0:.0 1.19 . Conversely.26 . clicking them off before clicking the next. then an n−1 degree polynomial has exactly enough coefficients to fit the data. This is the simplest form of interpolation. Next click on Tools.29 . meaning fitting a graph (of a function) between data points. if we have n data points. Which ones produce a goodlooking fit? You should note that the spline. that would give us an overdetermined system (more equations than unknowns) and if there is less data the system would be undetermined. What if we want a smoother graph? Try: > plot(t1.’*’) which will produce just asterisks at the data points. 68 . the shape-preserving interpolant and the 4th degree polynomial produce very good curves.26 . When we tried to use a 5th or higher degree polynomial Matlab returned a warning that the polynomial is not unique since “degree >= number of data points”.19 . If there are more data points. The following results were found: t 0 . then click on the Basic Fitting option. Begin clicking them one at a time. that is enough to exactly determine the n + 1 coefficients of pn (x) (as long as the data does not have repeated x values). . If we have exactly n + 1 data points. We will discuss polynomial interpolation and spline interpolation in this lecture.5:2 > y1 = [ 0 . Polynomial Fitting The most general degree n polynomial is: pn (x) = an xn + an−1 xn−1 + . + a1 x + a0 .0 y 0 .Lecture 19 Polynomial and Spline Interpolation A Chemical Reaction In a chemical reaction the concentration level y of the product at time t was measured every half hour.y1.y1) Matlab automatically connects the data with line segments.29 . This is the case in the example above.5 2. there are 5 data points so there is exactly one 4th degree polynomial that fits the data. .

this does not give a good graph. . . one must have some knowledge of the process. i i The unknowns are the coefficients a0 . In this section we learn what a spline is. How does it look? De-select the 9th degree polynomial and select the spline interpolant.21 .06 . we could and it is done all the time in many different contexts.29 . Under Tools and Basic Fitting select the 9th degree polynomial fit.19 . we did not get a warning. Based on the 4th degree polynomial. Set the plot to the 4th degree polynomial. yi ) gives an equation: pn (xi ) = a0 + a1 xi + a2 x2 + · · · + an xn = yi . . More data Generally one would think that more data is better. Thus: A polynomial fit is better for small data sets. Change the upper limit of the x-axis from 2 to 4. This makes polynomial fits quick to calculate since solving linear systems is what computers do best.1 . what should be the behavior as time goes on? It should reach a limiting value (chemical equilibrium). which we will study later.31 . what will the chemical reaction do in the future? Is this reasonable? Next change from 4th degree polynomial to spline interpolant. an . This requires that we understand the chemistry of the problem.4 . but to do so we need to know that there is an equilibrium to predict. Finally. so there is a unique 9 degree polynomial that fits the data.6 1.4 1.5 1.6 1. so determining a polynomial fit requires solving a linear system of equations. Input the following data vectors: > t2 = [ 0 .0] > y2 = [ 0 . Thus we have the following principle: To extrapolate beyond the data.9 2. This equation is linear in these unknowns.y. From our (limited) knowledge of chemical reactions. However. what will the chemical reaction do in the future? Try the shape-preserving interpolant also.69 When we tried a lower degree polynomial. .30 . we note that each data point (xi .’*’) There are 10 data points.26 .0 1.31] There are 11 data points. A challenging data set Input the following data set: > x = -4:1:5 > y = [ 0 0 0 1 1 1 0 0 0 0] and plot it: > plot(x. Could we use the data to predict this equilibrium value? Yes. .5 . Predicting the future? Suppose we want to use the data to extrapolate into the future.17 .29 . but we did notice that the resulting curve does not hit all the data points. According to the spline. Then click the Edit button and select the Axes Properties option. This should produce a much more satisfactory graph and the shape-preserving spline should be even better. A box should appear that allows you to adjust the domain of the x axes. If there was an overdetermined system how did Matlab come up with a pretty good approximation (for cubic)? The answer is the Least Squares Method. so a 10-th degree polynomial will fit the data.

Then to make S(x) as smooth as possible we require: ′ ′ Ci (xi ) = Ci+1 (xi ) ′′ ′′ Ci (xi ) = Ci+1 (xi ). so we are 2 equations short of being able to determine all the coefficients. The most general cubic function has the form: Notice that there are 2n of these conditions. where should we stop? There is an almost universal consensus that cubic is the optimal degree for splines and so we focus the rest of the lecture on cubic splines. xn−1 .2) at every data point.e. i i where each Ci is a cubic function. The next simplest function is quadratic. xi−1 ≤ x ≤ xi (19. bi .70 LECTURE 19. xn−1 ≤ x ≤ xn Ci (x) = ai + bi x + ci x2 + di x3 . (19.4) There are 2(n − 1) of these conditions. First we require that the spline be exact at the data: Ci (xi−1 ) = yi−1 and Ci (xi ) = yi . it is obtained by connecting the data with lines. Since each Ci is cubic. . This can be done and the result is called a quadratic spline. If we put a quadratic function on each interval then we should be able to make the graph a lot smoother. . ai . So far we have 4n − 2 equations. The other common option is called clamped boundary conditions: ′ ′ (19.5) These are called natural or simple boundary conditions. the basic idea of the cubic spline is that we represent the function by a different cubic function on each interval between data points. To determine the spline we must determine the coefficients. ai + bi xi−1 + ci x2 + di x3 = yi−1 i−1 i−1 and a i + bi x i + c i x 2 + d i x 3 = y i . That is. represent the graph with a simple function.3) at all the internal points. In terms of the points these conditions can be written as: bi + 2ci xi + 3di x2 = bi+1 + 2ci+1 xi + 3di+1 x2 i i 2ci + 6di xi = 2ci+1 + 6di+1 xi . if there are n data points. Since linear is the most simple function of all. x3 . If we were really careful then we should be able to make the curve smooth at the data points themselves by matching up the derivatives. So. (19. . At this point we have to make a choice. linear interpolation is the simplest form of spline. x1 . . then the spline S(x) is the function:  x0 ≤ x ≤ x1  C1 (x). (19. . there are a total of 4n coefficients in the formula for S(x). (19.6) C1 (x0 ) = Cn (xn ) = 0. The simplest spline is something very familiar to you. The usual choice is to require: ′′ ′′ C1 (x0 ) = Cn (xn ) = 0.1) S(x) =  Cn (x). x2 . Using cubic functions or 4th degree functions should be smoother still. Since there are n intervals. POLYNOMIAL AND SPLINE INTERPOLATION The idea of a spline The general idea of a spline is this: on each interval between data points. Cubic spline Again. In other words. and di for each i. ci . i. Ci (x). there are 4n coefficients to determine.

6 1. It is an interesting fact that a cubic spline is exactly the shape of a (linear) beam restrained to match the data by simple supports.5 1.0 1.30 .9 2. That is not the only parallel between beams and cubic splines. How does this compare with the plot above? What is a way to make the plot better? . Give an explanation of this error.1 .71 The terminology used here is obviously parallel to that used for beams.31] (a) Plot the data. then try a polynomial fit of the correct degree to interpolate this number of data points: What do you observe.17 . Exercises 19.29 .899 1.21 .1 You are given the following data: > t = [ 0 . Note that the equations above are all linear equations with respect to the unknowns (coefficients).06 .31 . This feature makes splines easy to calculate since solving linear systems is what computers do best.19 . in particular why is the term badly conditioned used? (b) Plot the data along with a spline interpolant.26 .29 .4 1.0] > y = [ 0 .5 .499 .

Lecture 20 Least Squares Fitting: Noisy Data Very often data has a significant amount of noise. assuming up to a moderate amount of traffic. The resulting line should go through the data in what looks like a very reasonable way. To determine the coefficients a and b you could run several experiments which consist of driving the highway at different times of day and also estimating the number of cars on the road using a counter. > T = .1*randn(size(x)). We can not possibly make the line go through every data point. We could simulate such data in Matlab as follows: > x = 1:. According to theory. our line must have an error with 72 . and x is the current number of cars on the road (in hundreds). Of course both of these measurements will contain noise. How does Matlab obtain a very nice line to approximate noisy data? The answer is a very standard numerical/statistical method known as least squares. Compare the equation with T (x) = . For the equation of the line we have only two free coefficients. > Tn = T + . Then choose a linear fit. The result should be ugly.1*x + 1. Click on show equations. i. the time should be approximately: T (x) = ax + b where b is the travel time when there is no other traffic. Thus. Traffic flow model Suppose you are interested in the time it takes to travel on a certain section of highway for the sake of planning.1:6.’) The data should look like it lies on a line.e. but with noise. Click on the Tools button and choose Basic fitting. Linear least squares Consider in the previous example that we wish to fit a line to a lot of data that does not exactly lie on a line. we can only wish for it to come reasonably close to as many data points as possible. > plot(x. try to fit the data with a spline.’. Next.1x + 1. random fluctuations. The next illustration shows the effects noise can have and how least squares is used. We can see from this example that splines are not suited to noisy data. but we have many data points. The least squares approximation is intentionally well-suited to represent noisy data.Tn. The coefficients should be pretty close considering the amount of noise in the plot.

To make ℓ(x) reasonable.73 respect to each data point. There are many possible ways one could go about this. .e. i. The entries in the matrix are determined from simple formulas using the data. The following sequence of commands replicates the data one might receive from a wind tunnel: > v = 0:1:60. This is what Matlab’s basic fitting tool uses to obtain an n degree polynomial fit whenever the number of data points is more than n + 1. d = kv 2 . b). 2 (20.dn. so we can think of E as a function of a and b. e2 . yi ). What is the coefficient of x2 ? How close is it to 0. Using the tools menu. en }.1234*v. > dn = d + . which is easily done. If we try to fit an n degree polynomial. when one wishes to find a minimum value of a function of two variables. but the standard one is to try to minimize the sum of the squares of the errors. i=1 a + nb = i=1 Thus. The process is quick and easily automated. add a quadratic fit and enable the “show equations” option. We could use the same process to obtain a quadratic or higher polynomial fit to data. E(a. = −2 ∂b i=1 We can simplify these equations to obtain: n n n n n (20. which is one reason it is very standard.2) x2 i i=1 a+ i=1 n xi xi b= i=1 n x i yi (20. the software has to solve an n × n linear system. That is.^2.*randn(size(v)). . we wish to simultaneously minimize all the errors: {e1 . > plot(v. .1234? . yi )} are the data. Drag coefficients Drag due to air resistance is proportional to the square of the velocity. In a wind tunnel experiment the velocity v can be varied by setting the speed of the fan and the drag can be measured directly (it is the force on the object).3) yi .e. but with some noise. . we set the partial derivatives equal to zero: ∂E (yi − axi − b) xi = 0 = −2 ∂a i=1 ∂E (yi − axi − b) = 0. In calculus.’*’) The plot should look like a quadratic.4*v. If ℓ(x) is our line and {(xi . then ei = yi − ℓ(xi ) is the error of ℓ with respect to each (xi . the whole problem reduces to a 2 by 2 linear system to find the coefficients a and b. > d = .1) In the above expression xi and yi are given. In this and every experiment some random noise will occur. but we are free to choose a and b. we denote by E the sum of the squares: n n E= i=1 (yi − ℓ(xi )) = 2 i=1 (yi − axi − b) . i.

LEAST SQUARES FITTING: NOISY DATA Note that whenever you select a polynomial in Matlab with a degree less than n − 1 Matlab will produce a least squares fit. do > k = vs’\dn’ This will produce a number close to . spline interpolant.306 3. Then plot the new variables and use the linear fit option to estimate a and b. 0 1 2 3 4 5 6 7 T C 3. If there was no noise. What will be the eventual concentration? Finally.’*’) By changing the independent variable from v to v 2 we produced a plot that looks like a line with noise. In this situation extrapolation would be meaningful. . We know from the physical situation that these should not be there. Exercises 20. We can produce this easily: > vs = v.1234.2t). it automatically gives a least squares solution of the matrix problem. Plot each and decide if the data is best represented by a polynomial interpolant. the overdetermined linear system is also inconsistent. or least squares fit polynomial. each point will give a different estimate for k.123 4.282 4. Suppose it is known that this chemical reaction should follow the law: c = a − b exp(−0. plot the graph of a − b exp(−0. change one of the variables so that the law is a linear function.399 4. Note that this is an application where we have physical knowledge. one that minimizes the sum of the squared errors.672 3. which is badly overdetermined since there are more equations than unknowns. > plot(vs. they are remnants of noise and the fitting process. which we know should not be there. We can express this as a linear system vs’*k = dn’.1234 than using a quadratic fit. To get the least squares estimate for k. Since we know the curve should be kv 2 .527 Plot this data.e. 20. Add a linear fit. You will notice that the quadratic fit includes both a constant and linear term. along with the data (as *).10].2t) on the interval [0.dn. then at every data point we would have k = d/v 2 . For instance we could use the plot to find the predicted drag at 80 mph.033 3. When Matlab encounters such systems. i. Following the example in the notes about the drag coefficients. What is the linear coefficient? This should be closer to 0. Since there is noise. which is exactly what we want. Indicate the source and meaning of the data. we can do better by employing that knowledge.74 LECTURE 20. Turn in the best plot of each.929 4. For instance. we know that the graph of d versus v 2 should be a straight line. In other words. The second fit still has a constant term.^2.2 The following table contains information from a chemistry experiment in which the concentration of a product was measured at one minute intervals.1 Find two tables of data in an engineering textbook or engineering website.

The subintervals [xi−1 . In calculus we learned that integrals are (signed) areas and can be approximated by sums of smaller areas. xi ]. i i The resulting approximation is: b a n f (x) dx ≈ f (x∗ )∆xi . i i=1 To use this to approximate integrals with actual numbers. % Inputs: x -.y) % produces the left sum from data input. for i = 1:n-1 L = L + y(i)*(x(i+1) .vector of the x coordinates of the partition % y -. i=1 and Rn = function L = myleftsum(x. we learned that we can choose any height of the function f (x∗ ) where x∗ ∈ [xi−1 . < xn−1 < xn = b.Lecture 21 Integration: Left. we wish to approximate a definite integral: b f (x) dx a where f (x) is a continuous function.vector of the corresponding y coordinates % Output: returns the approximate integral n = max(size(x)). The two simplest (and worst) ways to choose x∗ are as the left-hand point or the right-hand i point of each interval. b]: a = x0 < x1 < . such as the areas of rectangles. This gives concrete approximations which we denote by Ln and Rn given by n Ln = i=1 n f (xi−1 )∆xi f (xi )∆xi . . end 75 . xi ] determine the width ∆xi of each of the approximating rectangles. . For the height.x(i)). Right and Trapezoid Rules The Left and Right endpoint rules In this section. we need to have a specific x∗ in each i interval. We begin by choosing points {xi } that subdivide [a. % safe for column or row vectors L = 0.

with each interval having the same width: h= b−a . If this is the case.1) (21. This will hold for any increasing function.1: The left and right sums.5x. for a very simple function like f (x) = 1 + . 2 This method also has a straight-forward geometric interpretation. INTEGRATION: LEFT. We thus call the method the trapezoid method. Ln and Rn . The Trapezoid rule Knowing that the errors of Ln and Rn are of opposite sign. 2 . n where n is the number of subintervals. each rectangle of Ln is too short. while each rectangle of Rn is too tall.2) f (xi ).2.1. a very reasonable way to get a better approximation is to take an average of the two.76 LECTURE 21. See Figure 21. As you can see. On each subrectangle we are using: f (xi−1 ) + f (xi ) ∆xi . i=1 The foolishness of choosing left or right endpoints is illustrated in Figure 21. then Ln and Rn simplify to: Ln = Rn = b−a n b−a n n−1 f (xi ) i=0 n and (21. We will call the new approximation Tn : Tn = Ln + R n . Often we can take {xi } to be evenly spaced. RIGHT AND TRAPEZOID RULES T ¨¨ ¨¨ ¨¨ ¨¨ ¨ ¨¨ T ¨¨ ¨¨ ¨¨ ¨¨ ¨ ¨¨ E E Figure 21. Ai = 2 which is exactly the area of the trapezoid with sides f (xi−1 ) and f (xi ). We can rewrite Tn as Tn = i=1 n f (xi−1 ) + f (xi ) ∆xi . For decreasing functions Ln will always be too large while Rn will always be too small.

xn−1 . rather than a formula for the function. and y = (y0 .77 T   r  rr r  r rr rr  r         E Figure 21.x(i)). x1 .e. . .e. the first entry in x will be x0 . y2 . yn−1 . . end Using the Trapezoid rule for areas in the plane In multi-variable calculus you were supposed to learn that you can calculate the area of a region R in the plane by calculating the line integral: A=− y dx.4) where C is the counter-clockwise curve around the boundary of the region. . x(1)= x0 and x(n+1)= xn . Tn . xn ). yn ). . The following function program could then be used. . Thus.e. However. % safe for column or row vectors T = 0. If we are given data about the function. i. for i = 1:n-1 T = T + .vector of the corresponding y coordinates % Output: returns the approximate integral n = max(size(x)).y) % Calculates the Trapezoid rule approximation of the integral from input data % Inputs: x -. x = (x0 . x2 . y1 . i. In the evenly spaced case. when programming in Matlab. 2n (21.5*(y(i)+y(i+1))*(x(i+1) . function T = mytrap(x. . we can write this as Tn = b−a f (x0 ) + 2f (x1 ) + .3) Caution: The convention used here is to begin numbering the points at 0. ¯ ¯ . + 2f (xn−1 ) + f (xn ) . We can represent such a curve by consecutive points on it. C (21. often the data are not evenly spaced. Matlab’s indexing convention begins at 1. this allows n to be the number of subintervals and the index of the last point xn . i. .vector of the x coordinates of the partitian % y -. x0 = a. .2: The trapezoid rule. .

4) gives n A=− i=1 yi−1 + yi (xi − xi−1 ). end A Exercises 2√ 21.78 LECTURE 21. y = sin(t). (21. format long n = 10. t = linspace(0. x = cos(t). Also find L100 . plot(x.2*pi. using the exact value. and then add on the missing terms to obtain the left. we can use this method to approximate the area of a unit circle using n points on the circle: % Calculates pi using a trapezoid approximation of the unit circle. right and trapezoid integral approximations for f on [a. and T4 with even spacing (by hand. R100 and T100 and the relative errors of these approximations. Compare with the results of the previous exercise.1). . a. R and T .b.y) A = 0 for i = 1:n A = A . use x = linspace(a. R4 . RIGHT AND TRAPEZOID RULES Since we are assuming the curve ends where it begins. right and trapezoid approximations.n+1). but use a calculator) using formulas (21. Turn in the program and a brief summary of the results. Find the percentage error of these approximations.1 For the integral 1 x dx calculate L4 . 2√ Change to format long and apply your program to the integral 1 x dx.3). add the 2nd to nth y entries once. y0 ).2) and (21.2 Write a function program myints whose inputs are f . INTEGRATION: LEFT.n+1) to make the x values and y = f(x) to make the y values. In the following script. we require (xn . yn ) = (x0 . Applying the trapezoid method to the integral (21.(y(i)+y(i+1))*(x(i+1)-x(i))/2. To make it efficient. the left. b and n and whose outputs are L. 21. but not necessarily formulas for the boundaries such as in a land survey. 2 This formula then is the basis for calculating areas when coordinates of boundary points are known. b] with n subintervals.

Thus it makes sense that a better estimate would be to average Tn and Mn . 2 79 . then using midpoints to approximate the integral would give the formula: n Mn = i=1 f (¯i )∆xi . 2 If ∆xi = xi − xi−1 is the length of each interval. x For even spacing. and the formula is: Mn = where we define yi = f (¯i ). while Mn will be too low. If f is not linear on a subinterval. . it is not obvious that it is actually better than the trapezoid method Tn . . To find the proper weight we take advantage of the fact that for a quadratic function the errors EMn and ETn are exactly related by: 1 |EMn | = |ETn | . but it is. ¯ xi−1 + xi xi = ¯ . in this case we can do better than a simple average. b−a n n f (¯i ) = x i=1 b−a (ˆ1 + y2 + . The error will be minimized if we use a weighted average. they have opposite sign. if the function is concave up then Tn will be too high. i.1) Simpson’s rule Consider Figure 22.1.e. then it can be seen that the errors for the midpoint and trapezoid rules behave in a very predictable way. ˆ x While the midpoint method is obviously better than Ln or Rn . A point which is much more likely to be close to the average would be the midpoint of each subinterval. xi ] we will call the midpoint xi . Using the midpoint in the sum is called the midpoint rule. + yn ). However. On the i-th interval [xi−1 .Lecture 22 Integration: Midpoint and Simpson’s Rules Midpoint rule If we use the endpoints of the subintervals to approximate the integral. we run the risk that the values at the endpoints do not accurately represent the average value of the function on the subinterval. ∆x = (b − a)/n. For example. y ˆ ˆ n (22.

3 (22. . . 7. + 2f (xn−2 ) + 4f (xn−1 ) + f (xn )) .80 LECTURE 22. . where n is even): Sn = b−a (f (x0 ) + 4f (x1 ) + 2f (x2 ) + 4f (x3 ) + . Thus we take the following weighted average of the two. . . We can then number all the points using integers. INTEGRATION: MIDPOINT AND SIMPSON’S RULES T ¨¨ ¨¨ ¨ ¨ ¨¨ ¨¨ ¨¨ ¨ ¨¨ ¨¨ u ¨¨ E Figure 22. 3n Note that if we are presented with data {xi . if n is an even number. The function is concave up. while Mn is too low. Try it for n = 6. + 2yn−2 + 4yn−1 + yn ) . 3 3 If we use this weighting on a quadratic function the two errors will exactly cancel. which is called Simpson’s rule: S2n = 1 2 Mn + T n . Notice that we write the subscript as 2n. yi } where the xi points are evenly spaced with xi+1 −xi = ∆x. That is because we usually think of 2n subintervals in the approximation. If we number them this way we notice that the number of subintervals must be an even number. in which case Tn is too high.2) Notice the pattern of the coefficients. the n subintervals of the trapezoid are further subdivided by the midpoints. 100. The formula for Simpson’s rule if the subintervals are evenly spaced is the following (with n intervals. The following program will produce these coefficients for n intervals. it is easy to apply Simpson’s method: Sn = ∆x (y0 + 4y1 + 2y2 + 4y3 + .1: Comparing the trapezoid and midpoint method on a single subinterval.

1). else w(i)=2. midpoint.b]. If ∆x is just moderately small.b] |f (4) (x)|.the number of intervals. whereas Simpson’s rule has error of order O(∆x4 ). you might not have control over the spacing and then you cannot use Simpson’s rule. length n+1 if rem(n. or at least the odd number points must lie exactly at the midpoint between the even numbered points. Then the error ESn of Simpson’s rule applied to b a f dx satisfies: |ESn | ≤ K4 (b − a)5 K4 (b − a) = ∆x4 . must be even % Output: a vector with the weights. For these methods there are formulas that give upper bounds on the error. before you can safely use it. applied to a f dx satisfy: K2 (b − a)3 K2 (b − a) = ∆x2 and 2 12n 12 K2 (b − a)3 K2 (b − a) |EMn | ≤ = ∆x2 . In applications such as experimental or simulated data. In the above error bounds we say that the trapezoid and midpoint rules have errors of order O(∆x2 ). 24n2 24 |ETn | ≤ • Suppose f (4) is continuous on [a. you must know how good (or bad) the approximation might be. In applications where you can choose the spacing. The most important thing in these error bounds is the last term. the error depends on ∆x2 whereas the error for Simpson’s rule depends on ∆x4 . The one drawback is that the points used must either be evenly spaced. Then the errors ETn and b EMn of the Trapezoid and Midpoint rules. We will consider just how accurate in the next section. As with any approximation. end end Simpson’s rule is incredibly accurate. Let K4 = maxx∈[a. For the trapezoid and midpoint method. These bounds are given by the following statements: • Suppose f ′′ is continuous on [a. Error bounds The trapezoid. 180n4 180 In practice K2 and K4 are themselves approximated from the values of f at the evaluation points.2) ~= 0 error(’n must be even’) end w = ones(n+1. respectively.b] |f ′′ (x)|. then there is a huge advantage with Simpson’s method. When an error depends on a power of a parameter. such as above. the worst case errors for the methods. and Simpson’s rules are all approximations. for i = 2:n if rem(i.2)==0 w(i)=4. In other words. . this is not a problem.81 function w = mysimpweights(n) % computes the weights for Simpson’s rule % Input: n -. Let K2 = maxx∈[a. we sometimes use the order notation.b].

3 Write a function program mysimpson that calculates the Simpson’s rule approximation for f on the interval [a.1. The inputs should be f .1) Exercises 2√ 22. Compare with exercise 21. Use your program on the integral 1 x dx to obtain S4 and S100 . Here is an example of its usage: > f = inline(’x. using the exact value. . It should call the program mysimpweights to 2√ produce the coefficients. 22.0.^(1/3). Compare these with the previous exercise and the true value of the integral.1 Using formulas (22. a form of Simpson’s rule that checks its own accuracy and adjusts the grid size where needed. b and n.^3)’) > I = quad(f.a.b) where the f is an inline function and a and b are the endpoints.2). Use your 2√ program on the integral 1 x dx to obtain M4 and M100 . for the integral 1 x dx calculate M4 and S4 (by hand. INTEGRATION: MIDPOINT AND SIMPSON’S RULES In Matlab there is a built-in command for definite integrals: quad(f. a. 22. b] with n subintervals. Here quad stands for quadrature.82 LECTURE 22.1) and (22.*sin(x. which is a term for numerical integration. The command uses “adaptive Simpson quadrature”. Find the percentage error of these approximations. but use a calculator). b] with n subintervals. Compare these with the previous exercise and the true value.2 Write a function program mymidpoint that calculates the midpoint rule approximation for f on the interval [a.

y) on the rectangle a ≤ x ≤ b and c ≤ y ≤ d.Y. which produces matrices of x and y values in X and Y. suppose you have a formula: f (x.1:6) and to see the first few values of Y type: > Y(1:6. you could use the command: > surf(X.’y’) > [X. Instead of the command mesh.*sin(x.Z) This will produce a 3-D plot that you can rotate by clicking on the rotate icon and then dragging with the mouse.c:k:d). y) = x sin(xy) and you are interested in the function on the region 0 ≤ x ≤ 5.Lecture 23 Plotting Functions of Two Variables Functions on Rectangular Grids Suppose you wish to plot a function f (x. The graph of a function of two variables is of course a three dimensional object. The y values on the other have start at π at the top and increase from top to bottom. > Z = f(X.01*pi:2*pi).’x’.1:5. Enter: > size(X) > size(Y) > size(Z) to see that each of these variables is a 101 × 51 matrix.pi:.Y. m n where m is the number of intervals in the x direction and n is the number of intervals in the y direction.1:6) You should observe that the x values in X begin at 0 on the left column and increase from left to right. h is the increment in the x direction and k is the increment in the y direction. To see the first few entries of X enter: > X(1:6. π ≤ y ≤ 2π. To obtain a good plot it is best if m and n can be set between 10 and 100. For example.Z) The key command in this sequence is [X Y] = meshgrid(a:h:b.*y)’. A way to plot this function in Matlab would be the following sequence of commands: > f = inline(’x.c:k:d). Note that this arrangement is flipped from the usual arrangment in the x-y plane. Visualizing the graph is often very useful. h= 83 . In the command [X Y] = meshgrid(a:h:b.Y] = meshgrid(0:.Y) > mesh(X. Often we will calculate: d−c b−a and k = .

% Assign the triangles T = delaunay(x. try the following and look at the output in X and Y.y) pairs V = [ 1/2 1/2 .^(3/2)’. rather.y) You can plot the triangles using the following command: > trimesh(T.*y)). . measurements are taken where convenient.2:2). 1 2 0 2 .9 . Each of these programs defines vectors x and y of x and y values of vertices and a matrix T. T(i.2)’.y. so T(i. say f (x. The three corners of triangle number i are at indices T(i.^2 > trisurf(T.5 1. Each triple of integers indicates which vertices to connect in a triangle. For instance. % V contains vertices. data does not usually come arranged in a nice rectangular grid.5:4.y.m from the class web site and run them.Y] = meshgrid(0:.y) You can also prescribe values (heights) at each vertex directly (say from a survey): > z1 = [ 2 3 2.y. y are row vectors containing coordinates of vertices x = V(:. 2 1 .z1) > trisurf(T. > [X.’y’). 2 0 . y = V(:. As before T is a list of sets of three integers.z2) Each row of the matrix T corresponds to a triangle.3). To plot a function.1:. PLOTTING FUNCTIONS OF TWO VARIABLES For another example of how meshgrid works.5 1.1).x. The following script program produces such a set of triangles: % mytriangles % Program to produce a triangulation.y).84 LECTURE 23.6 1. y) = x2 − y 2 on the washer figure try: > mywasher > z = x.:) gives triangle number i. Scattered Data and Triangulation Often we are interested in objects whose bases are not rectangular. which are (x.1)’. 1 1 .x.x.^2 .5 .y. 3/2 1/2 . and T(i. 1. 0 1] % x. .5 1. enter: > y(T(5.2)) To see other examples of regions defined by triangle get mywedge. So for example to get the y-coordinate of the second point of triangle number 5.7 . or using a function: > f = inline(’abs(sin(x.m and mywasher.2).x. In Matlab we can produce triangles for a region by recording the coordinates of the vertices and recording which vertices belong to each triangle.5 ].5 1.z) Note again that this plot can be rotated using the icon and mouse.5 2 1 1 . > z2 = f(x.5 . 0 0 1 0 . The resulting profiles can be plotted: > trimesh(T.’x’.

Make an appropriate choice of h and k and if necessary a rotation to produce a good plot.85 Exercises 23. Plot the triangulation.1 Plot the function f (x. Also create values at the vertices using a function and plot the resulting profile. Make sure to include a good number of both interior and boundary points. 23. y) = xe−x −y on the rectangle −3 ≤ x ≤ 3.) Record the coordinates of these points in the matrix V in a copy of the program mytriangles. 2 2 .m.2 Make up an interesting or useful convex irregular region and choose well-distributed points in it and on the boundary as a basis for a triangulation. Turn in your plot and the calculation of k and h. −2 ≤ y ≤ 2 using meshgrid. (Graph paper would be handy.

This means that the Riemann sums are approximations of the integral. numbered as in Figure 24. Then Rij will be the “intersection” of the i-th subinterval in [a. y) dydx = c a f (x.1.j=1. we begin by subdividing into smaller subrectangles {Rij }. b] × [c. You also should have learned that the integral is the limit of the Riemann sums of the function as the size of the subrectangles goes to zero. d]. y) dxdy. in a systematic way.Lecture 24 Double Integrals for Rectangles The center point method Suppose that we need to find the integral of a function. 86 . In this way the entire rectangle is subdivided into mn subrectangles. which is precisely what we need for numerical methods.1: Subdivision of the rectangle R = [a. on a rectangle: R = {(x. For a rectangle R. c ≤ y ≤ d}. b] into m subintervals and [c. d] into subrectangles Rij . y). b] with the j-th subinterval of [c. f (x. d] into n subintervals. A Riemann sum using this subdivision would have the form: m. In calculus you learned to do this by an iterated integral: b d d b I= R f dA = a c f (x. y) : a ≤ x ≤ b.1 f (x∗ )Aij = ij j=1 i=1 f (x∗ )Aij ij .n n m S= i. We will divide [a. d R15 R14 R13 R12 R11 R21 R31 R41 b c a Figure 24.

yi+1 ). yj ) + 2 f (xi . the upper left is (xi . . .j=1. . Notice that f (xi . . yi ) + f (xi+1 . Then the integral would which we will call the four-corners method. .  2 4 4 4 ··· 4 4 2  1 2 2 2 ··· 2 2 1 We could implement the four-corner method by forming a matrix (fij ) of f values at the grid points. For instance if (xi . and x∗ is a point in Rij . We can think of this as a weighted average of the values of f at the grid points (xi . but as the average of the values at several points. ··· . then the corresponding sum will be Fmn = 1 (f (xi . in practice we wish to choose x∗ in such a way to make S as accurate as possible even when the subrectangles are not very ij small. yj ). The center point is most likely of all points to have a value of f close to the average value of f . yj+1 ) + f (xi+1 . . yj ) gets counted multiple times depending on where (xi .   . Cmn = mn i. yi ) and the upper right is (xi+1 .1 m. yj+1 ). . . yj ) is located. . yj+1 )) Aij . yj ) + 4 Fmn =  4 corners edges interior . .1) W = . and so in that case m. However. . the lower right is (xi+1 . If the subrectangles are evenly spaced. yj ) + f (xi .n Cmn = i.  .1 f (cij )Aij . Note that if the subdivision is evenly spaced then ∆x ≡ (b − a)/m and ∆y ≡ (d − c)/n. . . then the sum becomes m. Here cij = xi−1 + xi yi−1 + yi . then this sum will converge to the same number. The weights used are represented in the matrix   1 2 2 2 ··· 2 2 1  2 4 4 4 ··· 4 4 2     2 4 4 4 ··· 4 4 2    (24. An obvious choice would be to evaluate f at all four corners of each Rij then average those. yj ). .n where A = ∆x ∆y is the area of the subrectangles.1 The four corners method Another good idea would be to take the value of f not only at one point. then doing entry-wise multiplication of the matrix with the weight matrix. If we note that the lower left corner is (xi . . If we denote the center points by cij .n (b − a)(d − c) f (cij ). . yj ) . . The theory of integrals tells us ij that if f is continuous. no matter how we choose x∗ . 4 i.j=1. For instance. Thus the sum becomes   A f (xi . 2 2 .  . yj ) is in the interior of R then it is the corner of 4 subrectangles.87 where Aij = ∆xi ∆yj is the area of Rij . we could choose x∗ to be the point in the lower left corner of Rij and the sum ij ij would still converge as the size of the subrectangles goes to zero. then we can simplify this expression. f (xi .j=1. The obvious choice for the best point in Rij would be the center point.

. % must be even. It uses the function mysimpweights from Lecture 22. W = u*v’. .   . .  2 8 4 8 ··· 4 8 2     4 16 8 16 · · · 8 16 4  1 4 2 4 ··· 2 4 1 The sum of the weighted values is multiplied by A/9 and the formula is Smn (b − a)(d − c) = 9mn m+1.c. yj ). .2)~=0 | rem(n.n) % Produces the m by n matrix of weights for Simpson’s rule % for double integrals % Inputs: m -. It is obtained by applying Simpson’s rule for single integrals to the iterated double integral.’y’) > I = dblquad(f.1. . . Here is an example: > f = inline(’sin(x. . If this is the case we sum up the values f (xi ./sqrt(x+y)’.j=1. The double Simpson method The next improvement one might make would be to take an average of the center point sum Cmn and the four corners sum Fmn . yj ). yj ) with weights represented in the matrix   1 4 2 4 ··· 2 4 1  4 16 8 16 · · · 8 16 4     2 8 4 8 ··· 4 8 2     4 16 8 16 · · · 8 16 4    W = . .0. . . (24.2)~=0 error(’m and n must be even’) end u = mysimpweights(m). . . v = mysimpweights(n). Fmn = 4mn i. % n -.d). The formula would be m+1. . i.*y).b. . % Output: W -. The resulting method requires that both m and n be even numbers and the grid be evenly spaced.5.  . ··· . % must be even. Thus it is in fact a double trapezoid rule.  . .number of intervals in the row direction.a. function W = mydblsimpweights(m. a more standard way to obtain a more accurate method is the Simpson double integral. .5.j=1.2) .’x’.2) Below is a Matlab function which will produce the matrix of weights needed for Simpson’s rule for double integrals. However.1 Matlab has a built in command for double integrals on rectangles: dblquad(f.88 LECTURE 24.number of intervals in the column direction.n+1 (b − a)(d − c) Wij f (xi . . DOUBLE INTEGRALS FOR RECTANGLES be obtained by summing all the entries of the resulting matrix and multiplying that by A/4.n+1 Wij f (xi .0.1 Notice that the four-corners method coincides with applying the trapezoid rule in each direction.a (m+1)x(n+1) matrix of the weights if rem(m.

Use both programs to approximate the integral 5 2 √ xy dy dx. Look at the mesh plot produced to make sure the program is putting the grid where you want it.89 Exercises 24. 18). 24. Turn in the programs and the resulting weights for a 5 × 6 grid.3 Using mysimpweights and mydblsimpweights as models make programs mytrapweights and mydbltrapweights that will produce the weights for the trapezoid rule and the weight matrix for the four corners (double trapezoid) method (24.2 Write a program mydblsimp that computes the Simpson’s rule approximation. Implement the change by changing the “meshgrid” to put the grid points at the centers of the subrectangles.1 Download the program mylowerleft from the web site. 24. Check the accuracy of the program on the integral in the previous problem.1). Evaluate this integral exactly to make comparisons. . 0 1 using (m. n) = (10. Modify it to make a program mycenter that does the center point method. Let it call the program mydblsimpweights to make the weight matrix (24.2).

Lecture 25 Double Integrals for Non-rectangles In the previous lecture we considered only integrals over rectangular regions. Note that the inside of the circle is (x − 1)2 + (y − 2)2 ≤ 4 and that the circle is contained in the rectangle [−1.0. Then make f (x) = sin3 (xy) if 2x + y ≤ 2 and f (x) = 0 if 2x + y > 2.1. 1] × [0. suppose we need to approximate the value of I= T sin3 (xy) dx dy where T is the triangle with corners at (0. 3] × [0.^2 + (y-2). We can then integrate the modified f on [0.0.3. Notice that the hypotenuse of the triangle has the equation 2x + y = 2. Redefining the function One strategy is to redefine the function so that it is zero outside the region of interest.^2.*(2*x + y <= 2)’) In this command <= is a logical command.0. Then we could let R be the rectangle [0.*exp(x.-1. 4].2) As another example. Thus we can create the right function and integrate it by: > f = inline(’x. 2). regions of interest are rarely rectangles and so in this lecture we consider two strategies for evaluating integrals over other regions. 1] × [0. 2). 2] which contains the triangle T .^2 <= 4)’) > I = dblquad(f. 2] using the command: > I = dblquad(f.*y). (1. The equation for this circle is (x − 1)2 + (y − 2)2 = 4. The term in parentheses is then a logical statement and is given the value 1 if the statement is true and 0 if it is false. For example.*((x-1). In Matlab we can make this function with the command: > f = inline(’sin(x.4) 90 . In practice. then integrate over a rectangle that includes the region.*y). 0) and (0.^3. 0). suppose we need to integrate x2 exp(xy) inside the circle of radius 2 centered at (1.

3 x= ¯ and y= ¯ (25.2) . the most important of which is that the finite elements method is based on it. 1 (25. rather than considering the value of f at just one point people often consider an average of values at several points. suppose you are surveying on a construction site and you want to know how much fill will be needed to bring the level up to the plan. (x2 . The most convenient of these is of course the corner points. y3 ). We can represent this sum by n Tn = i=1 ¯ fi Ai . (x2 . Another reason this is important is that often the values of f are not given by a formula. Such a subdivision is called a triangulation. Another idea would be to use the center point (centroid) of each triangle. i i=1 where n is the number of triangles. For example. On regions where the boundary consists of line segments. ¯ where f is the average of f at the corners. this can be done approximately. then the centroid is given by the simple formulas x1 + x2 + x3 3 y1 + y2 + y3 . However. In this case. Many software packages will even choose the triangles for you (Matlab will do it using the command delaunay).1) A function mythreecorners to compute using the three corners method is given below. However. is based on subdividing the region into triangles. the integral is approximated by a sum where each term is a value times an area n I≈ f (x∗ )Ai . Ai is the area of the triangle and x∗ a point in the triangle. you can use triangles by connecting your points with triangles. if the site is irregularly shaped or if there are obstacles on the site. This is a very important idea for several reasons. y1 ). this can be done exactly. y3 ). If the triangle has vertices (x1 . y2 ) and (x3 . The basic idea of integrals based on triangles is exactly the same as that for rectangles. y2 ) and (x3 . You would proceed by taking elevations at numerous points across the site. y1 ).91 Integration Based on Triangles The second approach to integrating over non-rectangular regions. then you cannot make these measurements on an exact rectangular grid. the formula for area is x1 1 A= det  y1 2 1  x2 y2 1  x3 y3  . but from data. If the triangle has vertices (x1 . Even on regions where the boundary contains curves.

V.2 Modify the program mythreecorners.1)). % extract x and y coordinates of all nodes y = V(:.2)). Each row gives the indices of three corners % Output: the approximate integral x = V(:. for i = 1:p x1 = x(T(i. y2 = y(T(i. z1 = f(x1. Each row has the x and y coordinates of a vertex % T -.1). Run the program on the region produced by mywasher.2). I = I + zavg*A. zavg = (z1 + z2 + z3)/3. I=0.3)).y2). % find values and average z2 = f(x2. as an inline % V -.the function to integrate. DOUBLE INTEGRALS FOR NON-RECTANGLES function I = mythreecorners(f. p = size(T. y) = xx+y2 and on the region produced by mywedge. using three corners % Inputs: f -.2)). Plot f (x. y2.92 LECTURE 25.m from the web site. x2.1 Download the programs mywedge. y1.m to a new program mycenters. x3 = x(T(i.m and f (x.y3). y) = xx+y2 2 +y √ on the region produced by mywasher.y1).m with the function f (x.m and mywasher. y1 = y(T(i. 25. % find coordinates and area x2 = x(T(i.m with the function 2 +y √ f (x. y) = sin(x) + y. y) = sin(x) + y on the region produced by mywedge. x3. A = . % accumulate integral end Exercises 25. y3.1)).the vertices.the triangulation.T) % Integrates a function based on a triangulation.1).5*abs(det([x1. y3 = y(T(i.m that does the centerpoint method for triangles.m. 1. z3 = f(x3. 1. . 1])).3)).

2 93 .1 26.Lecture 26 Gaussian Quadrature* Exercises 26.

e. . For example the usual form of the Taylor polynomial with remainder (sometimes called Taylor’s Theorem) is: h2 f (x + h) = f (x) + hf ′ (x) + f ′′ (c). One obvious idea would be to approximate f ′ (xi ) by the Forward Difference: ′ f ′ (xi ) = yi ≈ yi+1 − yi . xi+1 − xi This formula follows directly from the definition of the derivative in calculus. e. . . x + h = xi+1 and solving for f ′ (xi ) leads to: f (xi+1 ) − f (xi ) h ′′ f ′ (xi ) = − f (c). but we only know the values of f at a finite set of points. y2 ). it would seem likely that averaging the two methods would give a better result than either alone. (xn .g. 2h Errors of approximation We can use Taylor polynomials to derive the accuracy of the forward. Suppose then that we need information about the derivative of f (x).Lecture 27 Numerical Differentiation Approximating derivatives from data Suppose that a variable y depends on another variable x. backward and central difference formulas. i. Thus the error of the forward difference is −(h/2)f ′′ (c) which means it is O(h). xi+1 − xi = xi − xi−1 = h.. An alternative would be to use a Backward Difference: yi − yi−1 f ′ (xi ) ≈ . xi − xi−1 Since the errors for the forward difference and backward difference tend to have opposite signs. .e. Letting x = xi . 2 where c is some (unknown) number between x and x + h. as data from an experiment or a simulation: (x1 . (x2 . h 2 Notice that the quotient in this equation is exactly the forward difference formula. then averaging the forward and backward differences leads to a symmetric expression called the Central Difference: ′ f ′ (xi ) = yi ≈ yi+1 − yi−1 . If the points are evenly spaced. y = f (x). i. y1 ). Replacing h in the above calculation 94 . yn ).

2 (x .2 0 ′ Figure 27. Thus.4 x 0.2 0.8 1 1.2 i −1 i −1 .y ) 0. and 2h 1 (4) (4) f (xi ) = yi ≈ 4 [yi+2 − 4yi+1 + 6yi − 4yi−1 + yi−2 ] . central differences are significantly better and so: It is best to use central differences whenever possible. Subtracting these two equations and solving for f ′ (xi ) leads to: f (xi+1 ) − f (xi−1 ) h2 f ′′′ (c1 ) + f ′′′ (c2 ) − .y ) i i 0 (x −0.95 (x .y 1 backward difference forward difference central difference i+1 i+1 ) 0. For the central difference.6 0. by −h gives the error for the backward difference formula.8 0. h2 1 ′′′ f ′′′ (xi ) = yi ≈ 3 [yi+2 − 2yi+1 + 2yi−1 − yi−2 ] .1: The three difference approximations of yi .2 −0.j = u(xi . y) is a function of two variables that we only know at grid points (xi . yj ) . it is also O(h). f ′ (xi ) = 2h 3! 2 This shows that the error for the central difference formula is O(h2 ). There are also central difference formulas for higher order derivatives. We will use the notation: ui.4 0. yj ). These all have error of order O(h2 ): yi+1 − 2yi + yi−1 . the error can be found from the third degree Taylor polynomial with remainder: h2 ′′ f (xi ) + 2 h2 f (xi−1 ) = f (xi − h) = f (xi ) − hf ′ (xi ) + f ′′ (xi ) − 2 f (xi+1 ) = f (xi + h) = f (xi ) + hf ′ (xi ) + h3 ′′′ f (c1 ) 3! h3 ′′′ f (c2 ) 3! and where xi ≤ c1 ≤ xi+1 and xi−1 ≤ c2 ≤ xi .6 y 0. h ′′ f ′′ (xi ) = yi ≈ Partial Derivatives Suppose u = u(x.

Give the central difference approximations for f ′′ (1). y4 ) ≈ u3.3 9. yj ) be recorded in the matrix:   5.26 .3 8. Let’s consider an example.0 − 8. yj ) are given in the matrix (27.31 a. Suppose that h = .8 8. t 0 . yj ) ≈ 2k The second partial derivatives are: ux (xi . NUMERICAL DIFFERENTIATION frequently throughout the rest of the lectures.j ) and 2h 1 (ui.j ) and h2 1 uyy (xi .j−1 ) .4  5. The central difference formulas for the partial derivatives would be: 1 (ui+1. 4hk 4 · .1 Suppose you are given the data in the following table. y4 ) would be approximated by the central difference: Exercises 27.2 Suppose values of u(x.4 9. backward and central difference approximations of f ′ (1).5 1.1 6. 4hk Caution: Notice that we have indexed uij so that as a matrix each row represents the values of u at a certain xi and each column contains values at yj .3 − u1.j+1 − ui.j + ui−1. with an increment of h in the x direction and k in the y direction.j−1 − ui−1.2. 27.9 − 7. k and the mixed partial derivative is: uxx (xi .0 1. yj ) ≈ 2 (ui. uy (xi . Then uy (x2 .5.1 and k = .96 LECTURE 27.1 8.8 ≈ = 2.j − ui−1.4 u2.j − 2ui. y) at points (xi . yj ) ≈ 1 (ui+1.j+1 − 2ui.j+1 − ui+1.1) Assume the indices begin at 1.6 9.j+1 + ui−1.5 − u3.8 7. i is the index for rows and j the index for columns. uxx (x3 .75. f ′′′ (1) and f (4) (1).0 8. uxy (x2 .3 8.9   (uij ) =   5. Give the forward. y4 ) is approximated by: uy (x2 .5 − u2.1 − 9.1).9 9. y3 ) .5 6.5 2. y2 ) c.j + ui. We can suppose that the grid points are evenly spaced.2 (27.5 8.0 y 0 .j−1 ) .1  5. Suppose that h = .5 7. y2 ) d.19 .5 · .5 + u1.4 9. uyy (x3 . Let the values of u at (xi .1 8. yj ) ≈ 1 (ui+1.5 6. 2k 2 · 0. y4 ) b.29 .5 and k = .2 The partial derivative uxy (x2 .j−1 ) . yj ) ≈ uxy (xi . b.4 + 7. The arrangement in the matrix does not coincide with the usual orientation of the xy-plane. y4 ) ≈ uxy (x2 . Approximate the following derivatives by central differences: a.6 9. ux (x2 .5 ≈ = −2.

However. 97 . In IEEE standard double precision (used by Matlab and most serious software). all numerical integration methods are approximations and so there is error. which are based on numerical differentiation formulas. there are ways in which this very small error can cause problems. A different. way of thinking about this is that the machine records 52 floating binary digits or about 15 floating decimal digits. i. and (2) use a finer grid so that points are closer together.2^52 > (2^53+1) . This of course is more than adequate for any application. To see an unexpected occurence of round-off try the following commands: > (2^52+1) .2 × 10−16 . even if the calculations are performed exactly.e. Fortunately. The standard measure of how small is called machine epsilon.Lecture 28 The Main Sources of Error Truncation Error Truncation error is defined as the error caused directly by an approximation method. Thus there are never more than 15 significant digits in any calculation. For instance. machine epsilon is 2−52 or about 2. Numerical differentiation also has a truncation error. For instance. as will the differential equations methods we will study in Part IV. The obvious tradeoff is that a smaller grid requires more calculations. truncation errors are usually much larger than roundoff errors. It is defined as the smallest number that can be added to 1 to produce another number on the machine. Unless the grid is very small. this error is extremely small. which in turn produces more roundoff errors and requires more running time.52415678859930.9999987654321) then the result should be 1. There are two ways to minimize truncation error: (1) use a higher order method. Roundoff Error Roundoff error always occurs when a finite number of digits are recorded after an operation. if a smaller number is added the result will be rounded down to 1. but equivalent.2^53 Loss of Precision (also called Loss of Significance) One way in which small roundoff errors can become more significant is when significant digits cancel. if you were to calculate: 1234567(1 − .

Thus what was a very small problem with roundoff can become a very big problem. the measurements used to get the inputs cannot be completely accurate. First. Errors in the computations near the beginning especially can be magnified by a factor close to the condition number of the matrix. For example in f ′ (x) ≈ (f (x + h) − f (x))/h you will lose precision when h gets too small. Suppose that f ′ is close to zero at the solution x∗ .’x’) for i = 1:30 h=10^(-i) df=(f(1+h)-f(1))/h relerr=(2-df)/2 end At first the relative error decreases since truncation error is reduced.9999987654321) you will get: > 1.98 LECTURE 28. In any problem one can define the condition number as the maximum ratio of the relative errors in the output versus input. but the Matlab calculation has only 11 significant digits. when we talked about solving linear systems. i.52415678862573 In the correct calculation there are 15 significant digits. Then loss of precision takes over and the relative error increases to 1. this type of loss of precision can happen by accident if you are not careful. Relative error of output Relative error of inputs . Bad Conditioning We encountered bad conditioning in Part II.. the computations along the way have roundoff errors. Then a very small change in f (caused perhaps by an inaccurate measurement of some of the coefficients in f ) can cause a large change in x∗ . Second. This can be a problem in a couple of ways. Bad conditioning means that the problem is unstable in the sense that small input errors can produce large output errors. It can be shown that the condition number of this problem is 1/f ′ (x∗ ). condition # of a problem = max An easy example is solving a simple equation: f (x) = 0. Try: > > > > > > > format long f=inline(’x^2’. . if you input: > 1234567 * ( 1 . It turns out that matrix equations are not the only place where condition numbers occur.e. THE MAIN SOURCES OF ERROR But. Although this seems like a silly example.

What is the largest error in the graph. convert the expansion above to an inline function f2 and then type > y2=f2(x).99 Exercises 28. > mypi=mytrap(x.^9’.y2. To find the expanded version. Baum. or first expanded as f (x) = x − 18x8 + · · · and then evaluated. SIAM. 2√ (b) Since it is a quarter of a circle of radius 2. √ (a) If we do ( 3)2 we should get 3.^2).92:.’x’) > x=1. > plot(x.1 Identify the (main) source of error in each case and propose a way to reduce this error if possible.’red’) 9 Carefully study the resulting graphs. 28. 4 We try to use mytrap from Lecture 21 and do > x=0:. > y1=f1(x). we should have 0 4 − x2 dx = 1 π22 = π.y) > mypi-pi and find the error is ans=-0.001:2. 1997.2 1 The function f (x) = (x − 2)9 could be evaluated as written. .4409e-16. Should the graphs be the same? Which is more correct? Matlab does calculations using approximately 16 decimal places. > hold on > plot(x.0371. and how big is it relative to 10−16 ? Which source of error is causing this problem? 1 From Numerical Linear Algebra by L. type > syms x > expand((x-2)^9) > clear To evaluate it the first way. type > f1=inline(’(x-2).’blue’) To do it the second way.y1. > y=sqrt(4-x. Trefethen and D.2:2.08. but if we check > mythree=(sqrt(3))^2 > mythree-3 we find the error is ans=-4.

Extrapolation. Good for more. is much more tricky. Joining points by line segments is the most simple spline. fitting between the data.Review of Part III Methods and Formulas Polynomial Interpolation: An exact fit to the data.e. Extrapolation: Polynomials. To make predictions beyond the data. Minimizes the sum of the squares of the errors. Spline Interpolation: Fit a simple function between each pair of points. i=1 Trapezoid Rule: Tn = n i=1 f (xi−1 ) + f (xi ) ∆xi . For n data points it is a n − 1 degree polynomial. Cubic is by far the most common and important. but accurate points. The coefficients are found by solving a linear system of equations. The coefficients are found by solving a linear system of equations. Simply supported and clamped ends are available. 2 Midpoint Rule: Mn = n f (¯i )∆xi x i=1 where xi = ¯ 100 .e. Numerical Integration: Left Endpoint: Ln = i=1 n f (xi−1 )∆xi Right Endpoint: Rn = n f (xi )∆xi . Only good for very few. what the function should be. making fits beyond the data. Splines and Least Squares are generally used for Interpolation. accurate data points. The coefficients are found by solving a linear system of equations. you must have knowledge of the underlying process. Interpolation vs. Good for noisy data. Cubic matches derivatives and second derivatives at data points. 2 xi−1 + xi . i. i. Least Squares: Makes a “close fit” of a simple function to all the data.

4 i. .101 Numerical Integration Rules with Even Spacing: For even spacing: ∆x = b−a where n is the number of subintervals. . + 2yn−2 + 4yn−1 + yn b−a (f (x0 ) + 4f (x1 ) + 2f (x2 ) + 4f (x3 ) + .1 m. yi ) + f (xi+1 . + 2f (xn−2 ) + 4f (xn−1 ) + f (xn )) . yj+1 )) Aij . mn i. . xn ). . y0 ). yj ) + f (xi . = 3n Area of a region: A=− y dx.j=1. where cij = Centerpoint – Evenly spaced: m. . + 2yn−1 + yn = n b−a f (x0 ) + 2f (x1 ) + . yn ) = (x0 . i=0 n Rn = ∆x i=1 yi = f (xi ). . x i=1 Simpson’s rule: Sn = ∆x y0 + 4y1 + 2y2 + 4y3 + . i. .4): n A=− Accuracy of integration rules: Right and Left endpoint are O(∆x) Trapezoid and Midpoint are O(∆x2 ) Simpson is O(∆x4 ) Double Integrals on Rectangles: Centerpoint: i=1 yi−1 + yi (xi − xi−1 ) 2 m.n . x = (x0 . We can represent such a curve. + 2f (xn−1 ) + f (xn ) .1 zij = (b − a)(d − c) f (cij ). by consecutive points on it. yj+1 ) + f (xi+1 . y1 . . . x1 . then: n n−1 Ln = ∆x i=0 n yi = b−a n b−a n n−1 f (xi ). xn−1 . 2n yi = ¯ b−a n n Mn = ∆x i=1 f (¯i ). . yn−1 .n Four corners: Fmn = 1 (f (xi . .j=1. . . . .j=1. 2 2 . i=1 Tn = ∆x y0 + 2y1 + .e.1 f (cij )Aij . Cmn = ∆x∆y i. . x2 . yn ) ¯ ¯ with (xn . C where C is the counter-clockwise curve around the boundary of the region. Applying the trapezoid method to the integral (21. y2 .1 m. and y = (y0 .j=1.n Cmn = i.n xi−1 + xi yi−1 + yi .

. 8 16 4 1 4 2 4 . (x2 . 2 8 4 16 1 4 2 8 4 8 . y3 ):   x1 x2 x3 1 det  y1 y2 y3  . . 3 x= ¯ x1 + x2 + x3 3 . Area of a triangle with corners (x1 .j=1. – Triangles are suitable for odd-shaped regions. . . yj ) + 2 Fmn = 4 corners m. yj ). yi )Ai . . yj ). yj ) + 4 edges interior = where (b − a)(d − c) Wij f (xi .1 1 4 4 16 2 8 4 16 . 4mn i. .n REVIEW OF PART III f (xi . . – A triangulation is better if the triangles are nearly equilateral. A= 2 1 1 1 Centerpoint: C= i=1 n f (¯i .   2  1 1 2 2 . . . 4 8 2 4 16 8 16 . . . . . 4 4 2 2     . . .   2   4  1 Integration based on triangles: – Triangulation: Dividing a region up into triangles. 4 4 2 2 ··· ··· ··· ··· ··· ··· m. x ¯ and y= ¯ with y1 + y2 + y3 .j=1. Three corners: n Tn = i=1 ¯ fi Ai ¯ where f is the average of f at the corners of the i-th triangle. . y2 ). y1 ). . 9mn i. . 2 4 1 2 2 2 4 4 4 4 . yj )  Double Simpson:     W =    Smn = 2 2 4 4 4 4 . .n f (xi . . .  (b − a)(d − c) Wij f (xi . . 8 16 4 ··· ··· ··· ··· ··· ··· ··· ··· 2 8 4 8 .  where       W =             . . (x3 . 4 8 2 4 16 8 16 . . .1  1 2 2 4 2 4 . . . .102 Four Corners – Evenly spaced:  A f (xi . . . . .

2h 1 (4) f (4) (xi ) = yi ≈ 4 [yi+2 − 4yi+1 + 6yi − 4yi−1 + yi−2 ] .j+1 − ui. Matlab Data Interpolation: Use the plot command plot(x. If you select a polynomial degree less than n − 1. . . . h2 1 ′′′ f ′′′ (xi ) = yi ≈ 3 [yi+2 − 2yi+1 + 2yi−1 − yi−2 ] . . .j−1 − ui−1. xi − xi−1 yi+1 − yi−1 . ux (xi . . .j+1 − ui+1. Bad conditioning – the problem is sensitive to input errors. Numerical integral of f (x) on [a. h ′′ f ′′ (xi ) = yi ≈ Partial Derivatives: Denote ui. . . Integration > quadl(f. . . Loss of precision – an amplification of roundoff error due to cancellations.j − ui−1. . . . .j ) .j−1 ) . then Matlab will produce a least squares approximation. Use the Basic Fitting tool to make an interpolation or spline.j−1 ) . . . . .j ) .103 Forward Difference: ′ f ′ (xi ) = yi ≈ Backward Difference: f ′ (xi ) ≈ Central Difference: Higher order central differences: yi+1 − yi . 2h ′ f ′ (xi ) = yi ≈ yi+1 − 2yi + yi−1 . . yj ) ≈ uxy (xi . .b) . .j−1 ) . 4hk Sources of error: Truncation – the method is an approximation. . . yj ) ≈ uxx (xi . . 2h 1 (ui. . yj ) ≈ 1 (ui+1. . yj ). k2 1 (ui+1. . . yj ) ≈ uy (xi . Roundoff – double precision arithmetic uses ≈ 15 significant digits. h2 1 (ui. 2k 1 (ui+1. .y.j = u(xi . .j+1 + ui−1. If you choose an n − 1 degree polynomial with n data points the result will be the exact polynomial interpolation. Error can build up after multiple calculations. xi+1 − xi yi − yi−1 . .j + ui−1. b]. . . yj ) ≈ uyy (xi . . . .’*’) to plot the data. . . .a. . .j + ui.j − 2ui. . .j+1 − 2ui.

. . . .b. . 0). .1. . . . Integral of f (x. .d) . .0.*(2*x + y <= 2)’) > I = dblquad(f. . . . . . . Triangles: Matlab stores triangulations as a matrix of vertices V and triangles T. . . For example: > f = inline(’sin(x.’x’. Produces triangles from vertices. 2). . . . (0. . . . . .’y’) > I = dblquad(f.0. . and (1. . . . y) = sin3 (xy) on the triangle with corners (0. . . Integration over non-rectangles: Redefine the function to be zero outside the region. . . . . . . .y) > trimesh(T. . . d]. . b] × [c. .1.104 REVIEW OF PART III > dblquad(f. . . .y. .z) .y. .x. . . > T = delaunay(V) .c. Example: > f = inline(’sin(x. . . 0). . .x. . y) on [a. . .2) Integrates f (x.0.*y)/sqrt(x+y)’. . . . .x.*y). .^3.x. . > trimesh(T.z) > trisurf(T. . . . . . .2) Matlab uses an advanced form of Simpson’s method. .a. . . . . . .0.y) > trisurf(T. . .

2007 . Mathematics Department.Part IV Differential Equations c Copyright. Todd Young and Martin Mohlenkamp. Ohio University.

Usually.1: A pendulum. (29.Lecture 29 Reduction of Higher Order Equations to Systems The motion of a pendulum Consider the motion of an ideal pendulum that consists of a mass m attached to an arm of length ℓ.2) (29. then Newton’s laws of motion tell us: mg ¨ sin θ.1) 106 . t e e θ e ℓ e e e ew e Figure 29. mθ = − ℓ where θ is the angle of displacement. 2 conditions are needed. If we ignore friction. This is a second order ODE because the second derivative with respect to time t is the highest derivative. If we also incorporate moving friction and sinusoidal forcing then the equation takes the form: ¨ ˙ mg sin θ = A sin Ωt. mθ + γ θ + ℓ Here γ is the coefficient of friction and A and Ω are the amplitude and frequency of the forcing. Because it is second order. and the usual conditions are initial conditions: ˙ θ(0) = θ0 and θ(0) = v0 . A solution of the equation would be a function θ(t). where c = γ/m. this equation would be rewritten by dividing through by m to produce: ¨ ˙ θ + cθ + ω sin θ = a sin Ωt. ω = g/ℓ and a = A/m. It is nonlinear because it has the term sin θ and which is a nonlinear function of the dependent variable θ. To get a specific solution we need side conditions.

7) (29. ˙ ˙ In vector form this is simply y = f (t. n−1 . y) = yi+1 for i < n and fn (t.1) the change of variables has the form y1 = θ ˙ y2 = θ. y) with fi (t. x. y1 . yn = x(n) = f (t. . ˙ In vector form this is ˙ y= y2 −cy2 − ω sin(y1 ) + a sin(Ωt) y1 (0) y2 (0) θ0 v0 . yn ). . In addition. . . . y2 . . i. and the resulting equations are y1 = y2 ˙ y2 = −cy2 − ω sin(y1 ) + a sin(Ωt). y2 . it can be written in the form: dn−1 x (29.4) (29. . .8) As stated above. x.5) The initial conditions are converted to y(0) = = . . yn = x(n−1) = The resulting first-order system is the following: y1 = x = y2 ˙ ˙ y2 = x = y3 ˙ ¨ .e. there is a conceptual reason to make the change. Most general software for solving ODEs (including Matlab) requires that the ODE be input in the form of a first-order system. most higher-order differential equations that occur in applications can be converted to a system of first order equations and that is what is usually done in practice. ˙ ¨ dt Then it can be converted to a first-order system by this standard change of variables: y1 = x y2 = x ˙ .107 Converting a general higher order equation All of the standard methods for solving ordinary differential equations are intended for first order equations.3) x(n) = f t. (29. (29. . y1 . In a system described by a . y) = f (t. . . Suppose that an n-th order equation can be solved for the n-th derivative. However.6) dn−1 x . . dtn−1 (29. the main reason we wish to change a higher order equation into a system of equations is that this form is convenient for solving the equation numerically. . For the example of the pendulum (29. For this reason. . . it is inconvenient to solve higher order equations numerically. yn ). x.

The output T contains times and Y contains values of the vector y at those times. Each of these correspond to different solving methods.5]).[1. we are mainly interested in its values: > theta = Y(:.* and .) . such as the pendulum. . one must know both the angle and the ˙ angular velocity to know what the pendulum is really doing. REDUCTION OF HIGHER ORDER EQUATIONS TO SYSTEMS higher order equation.-1.1).^. Here [0 20] is the time span you want to consider and [1.108 LECTURE 29.1] and explain what you observe. Generally in applications the vector y is the state of the system described by the differential equation.1 Transform the given ODE into a first order system: . We call the pair (θ.. x(1) = 2. Find ˙ ¨ a numerical solution of this IVP using ode45 and plot the first coordinate (x).’y’) The command ode45 is then used as follows: > [T Y] = ode45(f.theta) In the next two sections we will learn enough about numerical methods for initial value problems to understand roughly how Matlab produces this approximate solution.. and x(1) = 0.:) Since the first coordinate of the vector is the position (angle). then we can input the right hand side as: > f = inline(’[y(2). Try time intervals [1 2] and [1 2. We will learn about this algorithm later. To implement ode45 for a system. Exercises 29.’t’. Using Matlab to solve a system of ODE’s In Matlab there are several commands that can be used to solve an initial value problem for a system of differential equations.1*y(2)-sin(y(1))+sin(t)]’. θ) the state of the system.-. we will assume that all the constants are 1. The standard one to use is ode45. except c = .5] is the initial value of the vector y.7).-1.[0 20]. which uses the algorithm “Runge-Kutta 4 5”. in the definition of the vector function. (Remember to use entry-wise operations. ¨ ˙ Suppose the initial conditions for the ODE are: x(1) = 1. x + x2 − 3x3 + cos2 x = e−t sin(3t). For the pendulum system (29. knowing the position is not enough to know what the system is doing. In the case of a second order equation. we have to input the vector function f that defines the system. Try: > size(T) > T(1:10) > size(Y) > Y(1:10. > plot(T.1.

By a numerical solution. . y2 ).3) ˙ and will then replace the derivative y by a difference. y2 . The Euler Method ˙ The most straight forward approach is to replace yi in (30. The first of these points is exactly the initial value. (t0 . yi ). first write the differential equation in the indexed notation ˙ yi ≈ f (ti . y0 ). y1 ). and the steps are evenly spaced. (30. b]. yn ). i. To do this.Lecture 30 Euler Methods Numerical Solution of an IVP Suppose we wish to numerically solve the initial value problem ˙ y = f (t.1) and the times ti are given simply by ti = a + ih. yi ). . . y). yn in a way that is as consistent as possible with (30. . This gives yi+1 − yi = f (ti . This is very easy to implement. . (30. . If we take n steps as above. (t2 . (t1 . . where t0 = a and tn = b. (tn . h Rearranging this gives us a way to obtain yi+1 from yi known as Euler’s method: yi+1 = yi + hf (ti .4) With this formula. There are many ways we might carry this out and in the next section we study the simplest. we must mean an approximation of the solution at a finite number of points. (30.2) y(a) = y0 . yi ). yi ). on an interval of time [a. .1). we can start from (t0 . as you can see from the following program (which can be downloaded from the class web site): 109 . This leaves the most important part of finding a numerical solution: determining y1 . n (30. y0 ) and compute all the subsequent approximations (ti .e.3) by its forward difference approximation. then the time change in each step is h= b−a .

We can compare these calculations with Matlab’s built-in function with the commands: > [T Y]= ode45(f. > theta = Y(:.1). y = y0. Y = [Y.1*y(2)-sin(y(1)) + sin(t)]’.y) with initial condition y(a) = y0 % on the interval [a.a function f(t.5]. Next try 20. % T and Y will record all steps for i = 1:n y = y + h*f(t. Try: > size(T) > size(Y) Since the first coordinate of the vector is the angle.b] using n steps of Euler s method. 40. Type: > hold on then redo the above with 5 replaced by 10.theta) In this plot it is clear that n = 5 is not adequate to represent the function.’t’. % Inputs: f -. h = (b-a)/n. Y] = myeuler(f.y).a (n+1) by d matrix where d is the length of y % Y(t. we only plot its values: > theta = Y(:. As you can see the graph becomes increasingly better as n increases.y0.b] with the start and end times % y0 -. % t and y are the current variables T = a. Y = y0’. such as the vector function for the pendulum: > f = inline(’[y(2).1). [1. y(a) = y0 % n -. 80.tspan. Here [0 20] is the time span you want to consider. > plot(T.a column vector of the initial values.-1.[0 20]. t = a + i*h. % step size t = a.5). The output T contains times and Y contains values of the vector as the times.[1.i) gives the ith component of y at time t % parse starting and ending points a = tspan(1).n) % Solves dy/dt = f(t. and 200.’r’) . t].110 LECTURE 30. EULER METHODS function [T .-1. y’].y) that returns a column vector of the same % length as y % tspan -.-1.theta.a n+1 column vector containing the times % Y -.y0.[0 20]. > plot(T. end To use this program we need a function. Y] = myeuler(f.n) % function [T .number of steps to use % Outputs: T -.a vector [a.[1.’y’) Then type: > [T Y] = myeuler(f.5]).5] is the initial value of the vector y and 5 is the number of steps. b = tspan(2). T = [T.-.tspan.

111

The problem with the Euler method
You can think of the Euler method as finding a linear approximate solution to the initial value problem on each time interval. An obvious shortcoming of the method is that it makes the approximation based on information at the beginning of the time interval only. This problem is illustrated well by the following IVP: x + x = 0, ¨ x(0) = 1, x(0) = 0 . ˙ (30.5)

You can easily check that the exact solution of this IVP is x(t) = cos(t). If we make the standard change of variables y1 = x, then we get y1 = y2 , ˙ y2 = −y1 . ˙ y2 = x, ˙

Then the solution should be y1 (t) = cos(t), y2 (t) = sin(t). If we then plot the solution in the (y1 , y2 ) plane, we should get exactly a unit circle. We can solve this IVP with Euler’s method: > g = inline(’[y(2);-y(1)]’,’t’,’y’) > [T Y] = myeuler(g,[0 4*pi],[1;0],20) > y1 = Y(:,1); > y2 = Y(:,2); > plot(y1,y2) As you can see the approximate solution goes far from the true solution. Even if you increase the number of steps, the Euler solution will eventually drift outward away from the circle because it does not take into account the curvature of the solution.

The Modified Euler Method
An idea which is similar to the idea behind the trapezoid method would be to consider f at both the beginning and end of the time step and take the average of the two. Doing this produces the Modified (or Improved) Euler method represented by the following equations: k1 = hf (ti , yi ) k2 = hf (ti + h, yi + k1 ) 1 yi+1 = yi + (k1 + k2 ) . 2 (30.6)

Here k1 captures the information at the beginning of the time step (same as Euler), while k2 is the information at the end of the time step. The following program implements the Modified method. It may be downloaded from the class web site.

112

LECTURE 30. EULER METHODS

function [T , Y] = mymodeuler(f,tspan,y0,n) % Solves dy/dt = f(t,y) with initial condition y(a) = y0 % on the interval [a,b] using n steps of the modified Euler’s method. % Inputs: f -- a function f(t,y) that returns a column vector of the same % length as y % tspan -- a vector [a,b] with the start and end times % y0 -- a column vector of the initial values, y(a) = y0 % n -- number of steps to use % Outputs: T -- a n+1 column vector contianing the times % Y -- a (n+1) by d matrix where d is the length of y % Y(t,i) gives the ith component of y at time t % parse starting and ending points a = tspan(1); b = tspan(2); h = (b-a)/n; % step size t = a; y = y0; % t and y are the current variables T = a; Y = y0’; % T and Y will record all steps for i = 1:n k1 = h*f(t,y); k2 = h*f(t+h,y+k1); y = y + .5*(k1+k2); t = a + i*h; T = [T; t]; Y = [Y; y’]; end We can test this program on the IVP above: > [T Ym] = mymodeuler(g,[0 4*pi],[1;0],20) > ym1 = Ym(:,1); > ym2 = Ym(:,2); > plot(ym1,ym2)

Exercises
30.1 Download the files myeuler.m and mymodeuler.m from the class web site. 1. Type the following commands: > hold on > f = inline(’sin(t)*cos(x)’,’t’,’x’) > [T Y]=myeuler(f,[0,12],.1,10); > plot(T,Y) Position the plot window so that it can always be seen and type: > [T Y]=myeuler(f,[0,12],.1,20); > plot(T,Y) Continue to increase the last number in the above until the graph stops changing (as far as you can see). Record this number and print the final graph. Type hold off and kill the plot window. 2. Follow the same procedure using mymodeuler.m . 3. Describe what you observed. In particular compare how fast the two methods converge as n is increased (h is decreased).

Lecture 31 Higher Order Methods
The order of a method
For numerical solutions of an initial value problem there are two ways to measure the error. The first is the error of each step. This is called the Local Truncation Error or LTE. The other is the total error for the whole interval [a, b]. We call this the Global Truncation Error or GTE. For the Euler method the LTE is of order O(h2 ), i.e. the error is comparable to h2 . We can show this directly using Taylor’s Theorem: ˙ y(t + h) = y(t) + hy(t) + h2 ¨ y(c) 2

˙ for some c between t and t + h. In this equation we can replace y(t) by f (t, y(t)), which makes 2 ¨ the first two terms of the right hand side be exactly the Euler method. The error is then h y(c) 2 2 or O(h ). It would be slightly more difficult to show that the LTE of the modified Euler method is O(h3 ), an improvement of one power of h. We can roughly get the GTE from the LTE by considering the number of steps times the LTE. For any method, if [a, b] is the interval and h is the step size, then n = (b − a)/h is the number of steps. Thus for any method, the GTE is one power lower in h than the LTE. Thus the GTE for Euler is O(h) and for modified Euler it is O(h2 ). By the order of a method, we mean the power of h in the GTE. Thus the Euler method is a 1st order method and modified Euler is a 2nd order method.

Fourth Order Runge-Kutta
The most famous of all IVP methods is the classic Runge-Kutta method of order 4: k1 = hf (ti , y) k2 = hf (ti + h/2, yi + k1 /2) k3 = hf (ti + h/2, yi + k2 /2) k4 = hf (ti + h, yi + k3 ) 1 yi+1 = yi + (k1 + 2k2 + 2k3 + k4 ) . 6 Notice that this method uses values of f (t, y) at 4 different points. In general a method needs n values of f to achieve order n. The constants used in this method and other methods are obtained from Taylor’s Theorem. They are precisely the values needed to make all error terms cancel up to hn+1 f (n+1) (c)/(n + 1)!. (31.1)

113

which means it is approximately Chn . To see variable step size in action. Deciding if a single step is accurate enough could be accomplished in several ways. since that would increase the number of steps.y(1)]’. So where is the advantage? Consider the following example.[1. > plot(T1.’y’) > f2 = inline(’[-5*y(2). if the true solution oscillates very rapidly. or h ≈ 10−2 .’ro-’) > size(T1) > size(T2) Why order matters Many people would conclude on first encounter that the advantage of a higher order method would be that you can get a more accurate answer than for a lower order method.1). To accomplish this a program tries an h and tests to see if that h is small enough. In the RK45.’bx-’) > hold on > plot(T2.0]). > [T2 Y2] = ode45(f2.5*y(1)]’. So you would need about n ≈ (b − a) × 104 steps to find the solution.’t’. the function f is evaluated at 5 different points.[1. This process is called variable step size. then the GTE is comparable to hn . .1). Suppose you use the second order. In that case the GTE is O(h2 ). > y1 = Y1(:. At the same time. then you would need h ≈ 10−4 . If there is a large discrepency.y2. These are used to make a 5th order estimate yi+1 .’y’) > [T1 Y1] = ode45(f1. is an embedded method. > y2 = Y2(:. for different differential equations. If not it tries again with a smaller h. However. the accuracy needed is usually a given and it is usually not extremely high. it accepts that step. then we can conclude that they are not accurate and a smaller h should be used. Thus it is not easy beforehand to tell how small h should be to get the error within a given tolerence. For instance.114 LECTURE 31. If you use the Euler method. Suppose that you need to solve an IVP with an error of less than 10−4 .[0 20]. this is not quite how things work. so you would need to use h2 ≈ 10−4 . In engineering problems. which has GTE of order O(h). the values of C may be very different. Thus getting more and more accurate solutions is not very useful.0]). we will define and solve two different ODEs and solve them on the same interval: > f1 = inline(’[-y(2). where C is some constant. HIGHER ORDER METHODS Variable Step Size and RK45 If the order of a method is n. This would require about n ≈ (b − a) × 102 steps.’t’. modified Euler method.[0 20]. If it is too small. If the 4th order and 5th order estimates are close.y1. but the most common are called embedded methods. How can a program then choose h small enough to produce the required accuracy? We also do not wish to make h much smaller than necessary. The Runge-Kutta 45 method. In reality. we will obviously need a smaller step size than for a solution that is nearly constant. but on the next step it tries a larger h. then we can conclude that they are accurate. which is used in ode45. 4 of the 5 values are used to also get a 4th order estimate.

Compare myRK2 and mymodeuler on the following IVP with time span [0. a thousand times fewer steps than for the Euler method. 4π]: x + x = 0. then h4 needs to be approximately 10−4 . x(0) = 0. In particular. (31. yi ) k2 = hf (ti + h/2. Thus the real advantage of higher order methods is that they can run a lot faster at the same accuracy. Exercises 31.2) Modify the program mymodeuler into a program myRK2 that does the RK2 method.115 That is a hundred times fewer steps than you would need to get the same accuracy with the Euler method. 0). i. It is summarized by the following equations: k1 = hf (ti . which should be (1. Such is often the case in robots and other applications with dynamic controls. which is also known as the midpoint method. Turn in the modified program and a summary of the results. compare the errors of y(4π). ¨ x(0) = 1. yi + k1 /2) yi+1 = yi + k2 . If you use the RK4 method.1 There is a Runge-Kutta 2 method. ˙ Using format long compare the results of the two programs for n = 10. This means you need only about n ≈ (b − a) × 10 steps to solve the problem.e. This can be especially important in applications where one is trying to make real-time adjustments based on the calculations. 1000. 100. and so h ≈ 10−1 . .

1 116 .Lecture 32 Multi-step Methods* Exercises 32.

such as u(0) = 0. t) = g(x) and u(x.1). u(L) = 0. EI 2EI (1 + (y ′ )2 )3/2 (33. For example the environment might be a canal.3) y ′′ − EI 2EI This is a linear equation and we can find the exact solution. moment af inertia I. and end tension T (see Figure 33. t) = u(x). it is known that the temperature. (33. If the environment or object we consider has length L. a uniform load w. (33. t) is the effect of any external heat source. and then g(x. at a location x and time t satisfies the partial differential equation ut − uxx = g(x. We could find its solution exactly if g(x) is not too complicated. If y(x) denotes the deflection at each point x in the beam. supposing g(x.4) 117 . In this case uxx = −g(x). t) would represent how a chemical is introduced. The same equation also describes the diffusion of a chemical in a one-dimensional environment. t). t). This is a linear second-order ordinary differential equation. Sometimes we are interested only in the steady state of the system. we have a boundary value problem or BVP.1) where g(x.2) with boundary conditions: y(0) = y(L) = 0. If the deflection is small then (y ′ )2 is negligible compared to 1 and the equation approximately simplifies to: wx(L − x) T y= . This equation is nonlinear and there is no hope to solve it exactly. u(x. Thus instead of an initial value problem. We can rewrite the equation as y ′′ − αy = βx(L − x). (33.Lecture 33 ODE Boundary Value Problems and Finite Differences Steady State Heat and Diffusion If we consider the movement of heat in a long thin object (like a metal bar). Beam With Tension Consider a simply supported beam with modulus of elasticity E. then typically one would have conditions on each end of the object. then y(x) satisfies the differential equation: T wx(L − x) y ′′ − y= .

From the boundary conditions we have immediately: y0 = y4 = 0. 000 lb... x4 = 120. . y1 . The nodes of this subdivion are x0 = 0. Suppose that the beam is carrying a uniform load of 100. Recall the central difference formula yi+1 − 2yi + yi−1 y ′′ (xi ) ≈ . To determine the deflections at the interior points we will rely on the differential equation.118LECTURE 33. . . .850 × 10−6 y + 1.282 × 10−3 xi (120 − xi ). h2 In this case we have h = (b − a)/n = (120 − 0)/4 = 30.5) α = 2. .4) by their discrete versions we get yi+1 − 2yi + yi−1 = h2 αyi + h2 βxi (L − xi ).425 × 10−6 x(120 − x)./in. Then L = 120 in. 2EI (33. We will then let y0 . W e calculate from (33. e e αx + x2 − + 2 √αL y(x) = 2 √αL α e α e α α α +1 +1 √ T EI and β= w . x2 = 60. Suppose that the beam is a W12x22 structural steel I-beam.7) First subdivide the interval [0.6) Finite Difference Method – Linear ODE A finite difference equation is an equation obtained from a differential equation by replacing the variables by their discrete versions and derivatives by difference formulas. (33. so that w = 120. 000 lb.1: A simply supported beam with a uniform load w and end tension T . 120] into four equal subintervals. 000 and a tension of T = 10. . Thus we have the following BVP: y ′′ = 2. Replacing all the variables in the equation (33. x1 = 30.850 × 10−6 yi + 900 × 1.565 × 10−3 yi + 1. 000/120 = 10. ODE BOUNDARY VALUE PROBLEMS AND FINITE DIFFERENCES w c c c c c t T E T' t Figure 33. β and h we obtain: yi+1 − 2yi + yi−1 = 900 × 2.425 × 10−6 . where α= and then the exact solution is √ 1 βL 2β β 2β e αL −√αx 2β x+ 2. y(0) = y(120) = 0. y4 denote the deflections at the nodes.5) (33.3).2 and I = 121in. First we will consider equation (33.425 × 10−6 xi (120 − xi ) = 2. . .4 . E = 29 × 106 lb. .850 × 10−6 and β = 1. Substituting in for α.

m.002565 1 0  1 −2.565 × 10−3 y1 + 1. 120]). Solve them and plot the solution using Matlab.002565] > b = [ 3. On the other hand the finite difference method still works when we set T = 0.002565 (33.002565 1 . Note that this equation is linear in the unknowns y1 and y2 .002565 1 0 .8) ⇔ y2 − 2.10) We can easily find the solution of this system in Matlab: > A = [ -2.463 4.463 The exact solution of this BVP is given in (33.6) from the program myexactbeam.m on the web site.282 × 10−3 × 602 (33.1 for which you will observe loss of precision.615  . we need define the x values and add on the values at the endpoints: > x = 0:30:120 > y = [0 .7) on the same domain ([0.119 This equation makes sense for i = 1. together with the exact solution (33. An amusing exercise is to set T = 0 in the program myexactbeam.002565y2 + y1 = 4.9) ⇔ y3 − 2. We can plot the true solution on the same graph: > hold on > myexactbeam Thus our numerical solution is extremely good considering how few subintervals we used and how very large the deflection is. y3 ) is the solution of the linear system:  −2.002565 1 0 1 −2. 3.  3. the equation becomes: y2 − 2y1 + y0 = 2. At x2 = 60 we have: y3 − 2y2 + y1 = . 0] > plot(x. 33.615 3. Plot your result.y. 3. At x3 = 90 we have (since y4 = 0) −2.’d’) Adding a spline will result in an excellent graph. At x1 = 30. 2. 0 1 -2. is in the program myexactbeam. y2 .463 4. but with eight subintervals and solve (using Matlab) as in the example.1 Derive the finite difference equations for the BVP (33.463 ]’ > y = A\b To graph the solution.2 By replacing y ′′ and y ′ with central differences. 1 -2.002565y1 = 3.m. with the parameter values for the W12x22 I-beam as in the example. y . Exercises 33.002565y3 + y2 = 3.282 × 10−3 × 30(90) (33.463. the program fails because the exact solution is no longer valid. Thus (y1 .463.615. derive the finite difference equation for the BVP: y ′′ + y ′ − y = x with y(0) = y(1) = 0 with 4 subintervals. Also try T = . That equation.002565y2 + 1. .6).

and hope that they converge to the correct answer. . i b 3 3 Now for the main idea. Relaxation Method for Nonlinear Finite Differences We can rewrite equation (34. but this time consider the effect of radiation as well as conduction. .1) where ub is the temperature of the background.Lecture 34 Finite Difference Method – Nonlinear ODE Heat conduction with radiation If we again consider the heat in a metal bar of length L. i b Next we add ui to both sides of the equation to obtain 3ui = ui+1 + ui + ui−1 − h2 d(u4 − u4 ) + h2 gi . . i b h2 (34. Instead.2) This equation is nonlinear in the unknowns. If we again replace the continuous problem by its discrete approximation then we get ui+1 − 2ui + ui−1 − d(u4 − u4 ) = −gi = −g(xi ). i b From this we can solve for ui in terms of the other quantities: 2ui = ui+1 + ui−1 − h2 d(u4 − u4 ) + h2 gi . One way to solve these equations would be by the multivariable Newton method. u1 . i b and then divide by 3 to get ui = 1 h2 (ui+1 + ui + ui−1 ) − d(u4 − u4 ) + gi . d incorporates a coefficient of radiation and g(x) is the heat source. u2 . then the steady state equation has the form uxx − d(u4 − u4 ) = −g(x). b (34.2) as ui+1 − 2ui + ui−1 = h2 d(u4 − u4 ) − h2 gi . which we can represent as a vector u0 . thus we no longer have a system of linear equations to solve. we introduce another interative method. Then we will use the above equation to get better estimates. 120 . . We will begin with an initial guess for the value of ui for each i. but a system of nonlinear equations.

5.121 If we let uj = (uj .u) If you run this program with the given n and steps the result will not seem reasonable. . hh = h^2/3. Add complete comments to the program. Now we can experiment and see if the iteration is converging. . n = 4. Try various values of steps and n to produce a good plot. we have ui at each successive step depending on its previous value and the equation itself. Fix the comments to reflect the new boundary conditions.^4+ub4+g(2:n)).1 (a) Modify the script program mynonlinheat to plot the initial guess and all intermediate approximations. uL = 0. u0 = 0. Implementing the Relaxation Method In the following program we solve the finite difference equations (34.n+1). u(n+1)=uL. u(1)=u0. steps = 4. We can plot the initial guess by adding the command plot(x.u) right before the for loop. .2) with the boundary conditions u(0) = 0 and u(L) = 0. and g(x) = sin(πx/4). Print the program and a plot using large enough n and steps to see convergence. n = 4. (b) Modify your improved mynonlinheat to mynonlinheattwo that has the boundary conditions u(0) = 5 and u(L) = 10. as we increase n. . end plot(x. x = 0:h:L. Print the program and a plot using large enough n and steps to see convergence. Exercises 34. b i i−1 − i 3 3 Notice that gi and ub do not change. We can also plot successive iterations by moving the last plot(x. In particular. uj . then we can obtain that j + 1st estimate from the formula uj+1 = i h2 1 j ui+1 + uj + uj d((uj )4 − u4 ) + gi . we need to increase steps like n2 .u) before the end. g = sin(pi*x/4). u = zeros(1. In the resulting equation. % mynonlinheat (lacks comments) L = 4. You will notice that this method converges quite slowly. . d = 1. uj . ub = . i. if n is large then steps needs to be really large. uj ) n 0 1 2 n−1 denote the jth approximation.e. Notice that the vector u always contains the current estimate of the values of u. uj . We let L = 4. for j = 1:steps u(2:n) = (u(3:n+1)+u(2:n)+u(1:n-1))/3 + hh*(-u(2:n). h = L/n. ub4 = ub^4.

but the net effect is that the energy eventually spreads itself out over a larger region. The diffusion of a chemical in a gas or liquid simliarly involves large numbers of molecules moving in different. It is very similar to the finite difference method we used for nonlinear boundary value problems. There was no time variable in the equation. The conduction of heat and diffusion of a chemical happen to be modeled by the same differential equation. t) in the equation as ut = c∆u + g. then it is incorporated a function g(x. 122 . If there is also a heat/chemical source. In some problems the z dimension is irrelevent. The symbol ∆ in this context is called the Laplacian. (35. In this case the equation is ut = c∆u = c(uxx + uyy ). y. or ut = cuxx + g(x. and ∆u(x. where c is the coefficient of conduction or diffusion. or u does not change in the z direction. nearly random ways. In three dimensions. Finally. z. It is worth mentioning a related equation ut = c∆(uγ ) for γ > 1. In a solid this involves moles of molecules all moving in different. the equation that governs both of these processes is the heat/diffusion equation ut = c∆u . i.2). These molecules eventually spread out over a larger region. In this case the equation is just ut = cuxx .2) (35. In this section we begin to study how to solve equations that involve time. we calculate temperature profiles that are changing.1) In this lecture we will learn a straight-forward technique for solving (35.Lecture 35 Parabolic PDEs .Explicit Method Heat Flow and Diffusion In the previous sections we studied PDE that represent steady-state heat problem. fast moving molecules bump into slower molecules and transfer some of their energy.1) and (35. either because the object in question is very thin. t).e. Heat conduction occurs when hot. z) = uxx +uyy +uzz . The reason for this is that they both involve similar processes. in some cases only the x direction matters. nearly random ways. y.

t2 .123 which is called the porus-media equation. material. . Here we have used the forward difference for ut and the central difference for uxx .j+1 − ui. k h (35. Initial Condition To solve the partial differential equation (35. . . L). T ].j+1 depends on ui. 0 ≤ j ≤ n − 1. tn ) = (0.1) or (35.j + ui+1. That is u at grid point i depends on its previous value and the values of its two nearest neighbors at the previous step (see Figure 35. T − k.2) we need an initial condition. we subdivide [0. 0) = f (x).j ). i. . (x0 . has two independent variables. let h = L/m and Similarly. t1 .e.j − 2ui.j + rui+1.e.1). the initial temperature distribution or initial concentration profile. This equation models diffusion in a solid. (35. L − h. .j for 1 ≤ i ≤ m − 1.j . if we are interested in solving the equation on an interval of time [0. Explicit Method Finite Differences The one dimensional heat/diffusion equation ut = cuxx . 2k. .4) allows us to calculate all the values of u at step j + 1 using the values at step j. . Since we are considering 0 ≤ x ≤ L.j .5) h2 The formula (35. We will not solve this equation numerically. This equation can be solved for ui. where r= (35. The equation ut = cuxx can then be replaced by the difference equations ui. . This represents the state of the system when we begin. . . . T ). k. let k = T /n and We will then denote the approximate solution at the grid points by uij ≈ u(xi . but the methods introduced here would work.0 = f (xi ).j and ui+1.j+1 = rui−1. x2 . This is represented by u(x. . .j c = 2 (ui−1. such as sandstone or an earthen structure. xm ) = (0. tj ). 2h. Many equations that involve 1 time derivative and 2 spatial derivatives are parabolic and the methods introduced here will work for most of them. To implement this in a program we let ui. but porus. and so we have to discretize both.4) ck . xm−1 . Notice that ui. tn−1 . . L] into m equal subintervals. i.3) (t0 . t and x. .j+1 to produce ui. . .j + (1 − 2r)ui. h. x1 . ui−1.

. g1 and g2 would represent heating or cooling applied to the ends.1: The value at grid point i depends on its previous values and the previous values of its nearest neighbors.j = g2 (tj ). at x = 0 and x = L. A second possibility is called variable boundary conditions. g1 (t) = 0.1) or (35.j+1 T x x tj x | | | x x x x x x xi−1 xi xi+1 Figure 35. Just as in the previous section we will have to specify something about the ends of the domain.e. Boundary Conditions To solve the partial differential equation (35. u(0. In a heat problem. T = 20.5x.j = g1 (tj ) and um. t) = g2 (t). Notice that the main loop has only one line. and then fill in the calculated values as the program runs. and g2 (t) = cos(t). Run this program using L = 2. t) = g1 (t) and u(L. g1 and g2 . i. f (x) = . which we can implement just as we did for the ODE boundary value problem. One possibility is fixed boundary conditions. Implementation The following program (also available on the web page) implements the explicit method. Note that g1 (t) must be input as g1 = inline(’0*t’). It is often convenient to define a matrix of the right dimension containing all zeros. This is represented by timedependent functions. To run it you must define functions f . PARABOLIC PDES . These are easily implemented in a program by letting u0. The values of u are kept as a matrix.2) we also need boundary conditions.EXPLICIT METHOD x x x x x tj+1 x x ui. It incorporates variable boundary conditions at both ends.124 LECTURE 35.

% find solution at remaining time steps for j = 1:n u(2:m.:) = g2(t).j) + r*u(3:m+1.1 Run the program myheat.T.t(j)) h r x t = = = = L/m.matrix of the solution. c = . Turn in the plots. u(L.c) % function [t x u] = myheat(f.2*r.m.n.vector of time points % x -.j). rr = 1 .n.m with L = 2π. 0<=t<=T % BC: u(0.j)~=u(x(i).n+1).:) = g1(t). linspace(0.length of time interval % m -.u’) Exercises 35. %Set up the matrix for u: u = zeros(m+1. c*k/h^2. 0) = f(x) % Inputs: % f -.T.j+1) = r*u(1:m-1. u(m+1.T.inline function for IC % g1.1) = f(x).number of subintervals for t % c -.rate constant in equation % Outputs: % t -. linspace(0.g1.m+1).vector of x points % u -.t. Set m = 20 and experiment with n.m. k = T/n.j) + rr*u(2:m. u(i. % assign initial conditions u(:.g2 -. g1 (t) = sin(t) and f (x) = − sin(x/4).L.t) = g2(t) % IC: u(x.number of subintervals for x % n -.length of rod % T -. % assign boundary conditions u(1.g1.5. t) = g1(t).L.g2. end % plot the results mesh(x. .L.g2. Get a plot when the program is stable and one when it isn’t.125 function [t x u] = myheat(f.inline functions for BC % L -.c) % solve u_t = c u_xx for 0<=x<=L. T = 20.n+1).

 . (36. 126 . ′ (36. If the matrix A has ew’s with absolute value greater than 1.Lecture 36 Solution Instability for the Explicit Method As we saw in experiments using myheat. To see how this happens.m. Figure 36. . and if we continue for k steps.   (36. From (36. the approximation at the next time step will be uj+1 = AUj + AEj .2). uj+k = Ak Uj + Ak Ej . . suppose that Uj is the vector of correct values of u at time step tj and Ej is the error of the approximation uj .j . r 1 − 2r r   .2) the state at the jth time step and  0  . .  . this matrix can have a property which is very bad in this context. the solution can become unbounded unless the time steps are small... This is equivalent to the matrix equation uj+1 = Auj .j+1 = rui−1.j ) representing A is the matrix  1 − 2r r 0 ···  . The problem with this is the term Ak Ej .1 shows the largest absolute value of an ew of A as a function of the parameter r for various sizes of the matrix A.1) where u0. for r > 1/2 the largest absolute ew grows rapidly for any m and quickly becomes greater than 1.j = 0. Writing the Difference Equations in Matrix Form If we use the boundary conditions u(0) = u(L) = 0 then the explicit method of the previous section has the form: ui.j = 0 and um.j .j + (1 − 2r)ui. r 1 − 2r . 0    r 0 ··· 0 r 1 − 2r Unfortunately.j .. .. . u2. 0   . . . .3) .j + rui+1. This term is exactly what we would do in the power method for finding the eigenvalue of A with the largest absolute value. A= . Namely. it can cause exponential growth of error unless r is small. then this term will grow exponentially. . . 1 ≤ i ≤ m − 1.. .  . 0 ≤ j ≤ n − 1. where uj is the column vector (u1. then u j = U j + Ej . um. In this lecture we consider why. As you can see.

Rather than building up.2 0.5 m=2 1 0.4 0. Recall that the error terms satisfy uj+k = Ak Uj + Ak Ej . . (36.3 0.4) where the first and last entries of bj contain the boundary conditions and all the other entries are zero. Since this must be less than 1/2. the effect of any error diminishes as time passes! From this we arrive at the following principle: If the explicit numerical solution for a parabolic equation does not blow up. Consequences Recall that r = ck/h2 .127 3 2.5 0. Suppose r < .5. Whenever EW > 1 the method is unstable. we have k< h2 . errors grow exponentially with each step.e.1: Maximum absolute eigenvalue EW as a function of r for the matrix A from the explicit method for the heat equation calculated for matrices A of sizes m = 2 . When using the explicit method r < 1/2 is a safe choice. we note that if we have other boundary conditions then instead of equation (36.9 1 r Figure 36. The second is that h cannot be too small.1 0. making h small would force k to be extremely small! A third consequence is that we have a converse of this analysis. If all the eigenvalues of A are less than 1 in absolute value then Ak Ej grows smaller and smaller as k increases. if r > 1/2 then the errors grow and if r < 1/2 the errors fade away.5 0 0 0. i.5 m=3 2 EW 1. Then all the eigenvalues will be less than one. .8 0. . 10. In this case the errors behave just as before. 2c The first consequence is obvious: k must be relatively small. then errors from previous steps fade away! Finally.2) we have uj+1 = Auj + rbj . Since h2 appears in the formula. This is really good.7 0.6 0.

for each r create the matrix A by calling myexppmatrix. What the value of n corresponding to r = 1/2? Try different n in myheat. calculate the eigenvalues of A. A = diag(u) + diag(v. . for given inputs m and r. ck/h^2 % Output: A -. g1 (t) = 0.1 Let L = π.1 for m = 4. Include the values of n and r in each. 36.128 LECTURE 36.6) > [v e] = eig(A) What is the “mode” represented by the eigenvector with the largest absolute eigenvalue? How is that reflected in the unstable solutions? Exercises 36. Test this using m = 6 and r = . and plot these numbers versus r.4.1).the main parameter.1 sin(x). as used in the program myheat.r) % produces the matrix for the explicit method for a parabolic equation % Inputs: m -. SOLUTION INSTABILITY FOR THE EXPLICIT METHOD We can write a function program myexppmatrix that produces the matrix A in (36. Is r = 1/2 the boundary between failure and success? Hand in a plot of the last success and the first failure. Check the eigenvalues and eigenvectors of the resulting matirices: > A = myexpmatrix(6.6.m. g2 (t) = 0.3). v = r*ones(m-1.1).an m by m matrix u = (1-2*r)*ones(m.1) + diag(v.the size of the matrix % r -.m to find precisely when the method works and when it fails. . c = .5. and m = 20.. find the max of the absolute values.-1).2 Write a script program that produces the graph in Figure 36. T = 20. Without using loops we can use the diag command to set up the matrix: function A = myexpmatrix(m. f (x) = . Your program should: define r values from 0 to 1.

1 shows the maximum absolute ew’s of B −1 as a function of r for various size matrices. 1 + 2r −r .j+1 + ui+1.j+1 − ui.4) 129 . Figure 37. where bj+1 represents the boundary condition. k h Note that all the terms have index j + 1 except one and isolating this term leads to ui. until you consider that the relationship is linear.j+1 − 2ui.Lecture 37 Implicit Methods The Implicit Difference Equations By approximating uxx and ut at tj+1 rather than tj . and using a backwards difference for ut . Since we are solving (37. the equation ut = cuxx is approximated by ui.3) where uj and bj+1 are given and  1 + 2r  −r   B=   for 1 ≤ i ≤ m − 1. Thus to find uj+1 .1) −r 1 + 2r . For the same reason it is also always as accurate as the individual steps. Now we have uj given in terms of uj+1 .j+1 + (1 + 2r)ui..j+1 where r = ck/h2 as before. Both this implicit method and the explicit method in the previous lecture make O(h2 ) error in approximating uxx and O(k) error in approximating ut . This would seem like a problem.   (37.j = −rui−1. −r . (37.j+1 )... Notice that this absolute maximum is always less than 1. the most important quantity is the maximum absolute eigenvalue of B −1 .1). (37. . Using matrix notation. −r . .2) (37. we have uj = Buj+1 − rbj+1 .j c = 2 (ui−1. which is 1 divided by the smallest ew of B.    . Thus errors are always diminished over time and so this method is always stable. so they have total error O(h2 + k). Thus although the stability condition allows the implicit method to use arbitrarily large k. to maintain accuracy we still need k ∼ h2 .j+1 − rui+1.  −r 1 + 2r Using this scheme is called the implicit method since uj+1 is defined implicitly. we need only solve the linear system Buj+1 = uj + rbj+1 .

(37.1: Maximum absolute eigenvalue EW as a function of r for the matrix B −1 from the implicit method for the heat equation calculated for matrices B of sizes m = 2 .e.j − 2ui.6 m=2 0.j+1 − 2ui.6 1.. where 1 − 2(1 − λ)r  (1 − λ)r   Aλ =     (1 − λ)r 1 − 2(1 − λ)r .  (1 − λ)r 1 − 2(1 − λ)r       (1 − λ)r . We implement a weighted average of the two methods by considering an average of the approximations of uxx at j and j + 1.j+1 − ui. it is natural to ask. In matrix notation this is Bλ uj+1 = Aλ uj + rbj+1 .4 0.5) The implicit method contained in these equations is called the Crank-Nicholson method.j+1 − rλui+1.7 EW 0. . (1 − λ)r .4 1.2 0.j .4 0 0.6 0. This leads to the equations ui.2 1.9 0.j + (1 − 2r(1 − λ))ui. 1 − 2(1 − λ)r (1 − λ)r .j+1 ) + k h h2 (37. . Whenever EW < 1 the method is stable.130 LECTURE 37. Crank-Nicholson Method Now that we have two different methods for solving parabolic equation.8 0.5 m=3 0.j+1 = r(1 − λ)ui−1.. Gathering terms yields the equations −rλui−1. . “can we improve by taking an average of the two methods?” The answer is yes.j ).j+1 + ui+1.j+1 + (1 + 2rλ)ui.8 1 1. IMPLICIT METHODS 1 0. .6) .j + r(1 − λ)ui+1..8 2 r Figure 37. i.j (1 − λ)c λc (ui−1.j + ui+1. 10. it is always stable. = 2 (ui−1.

we need to choose λ= r − 1/6 .1 Modify the program myexppmatrix from exercise 36. If L = 1 and T ≈ 1.7) This choice will make the method have truncation error of order O(h4 + k 2 ).. consequently. Turn in the programs and the plot. Thus we would have a total of 1. and so we can use k ∼ h without a problem.131 and In this equation uj and bj+1 are known. If c = 1. almost all on the interior. Surprisingly. √ 3− 5 λ= ≈ 0. 000. Our total error is then O(h2 + k 2 ). we can do even better if we also require √ 5 r= ≈ 0. If we choose λ = 1/2. This gives us a total of 560 interior grid points. then we need m ≈ 100 and n ≈ 10. 10 and. . Exercises 37.22361. 1 + 2rλ −rλ . Auj can be calculated directly. If we use the explicit or implicit method alone then we will need h2 ≈ k ≈ 10−4 . and then the equation is solved for uj+1 .2 to use B −1 for m = 4. this gives k = 5/250 ≈ 0. To get optimal accuracy with a weighted average.4) for given inputs m and r. . This is a lot. −rλ . .  −rλ 1 + 2rλ    . We would need h6 ≈ 10−4 which would require us to take m ≈ 4. we can show that the method is always stable.1 for m = 4. then we are in effect doing a central difference for ut ..   (37. a factor of 1785 fewer than the explicit or implicit method alone. 2r 1 + 2rλ  −rλ   Bλ =     −rλ 1 + 2rλ . or.12732.. 000 grid points. 6 With these choices. so √ would take m = 5 and have h = 1/5. it is always necessary to use the right weights. 000. For the Crank-Nicholson method with a given r. With a bit of work. the method has truncation error of order O(h6 ). Next suppose we solve the same problem using the optimal Crank-Nicholson method. which is absolutely amazing.0089442 so we would need n ≈ 112 to get T ≈ 1.2 into a function program myimpmatrix that produces the matrix B in (37. which is really good considering that the implicit and explicit methods each have truncation errors of order O(h2 + k). we √ For k we need k = ( 5/10)h2 /c. To appreciate the implications.64. It should produce a graph similar to that in Figure 37. suppose that we need to solve a problem with 4 significant digits. Modify your script from exercise 36. which has error O(k 2 ). −rλ .

such as u(0. un+1 would represent u(xn+1 ) and xn+1 would be L + h. For example. and let un+1 always equal un−1 by copying un−1 into un+1 whenever un−1 changes. u(0) = a. We can simply extend the grid to one more node. 2h 132 . t) = g1 (t). un+1 − un−1 = 0. but a computer does not know the difference between fiction and reality! This idea is carried out in the calculations of the next section and illustrated in Figure 38. xn+1 . To use it in a program. h h 2h If xn is the last node of our grid. u(L) = b. Setting that expression to zero implies: un = un−1 . the central difference. For that we should have u′ (L) = u′ (xn ) = or simply un+1 = un−1 . Recall that in our discrete equations we usually have L = xn . This however is not an obstacle in a program. we might have u(0.Lecture 38 Insulated Boundary Conditions Insulation In many of the previous sections we have considered fixed boundary conditions. i. We implemented these simply by assigning uj = a and uj = b for all j. Recall from the section on numerical derivatives.e. the effect of insulation is that no heat flows across the boundary. that there are three different ways to replace a derivative by a difference equation. Instead we want to use the most accurate version. However. It is so named because it mimics an insulator at the boundary. Physically. This means that the temperature gradient is zero. since this method replaces u′ (L) = 0 by an expression that is only accurate to first order. This restriction can be easily implemented in a program simply by putting a statement u(n+1)=u(n) inside the loop that updates values of the profile. which is outside the domain. right and central differences. we must replace u′ (L) = 0 by a discrete version. A third important type of boundary condition is called the insulated boundary condition. but are stuck with the first of these. left. then it is clear that we cannot use the right or central difference. The point xn+1 is “fictional”. t) = sin(t) which could represents periodic heating and cooling of the end at x = 0. The three of them at xn would be: u′ (xn ) ≈ un − un−1 un+1 − un un+1 − un−1 ≈ ≈ . which implies that should require the mathematical boundary condition u′ (L) = 0.1. it is not very accurate and is usually avoided. However. n 0 We also considered variable boundary conditions.

5.1: Illustation of an insulated boundary condition using a fictional point xn+1 with un+1 = un−1 . (38. This will again force the central difference at x = 0 to be 0. Then you can let u0 = u1 . For the insulated condition. u(0) = 5. L = 1. m 4 . The point x5 = 1. x1 = . u′ (0) = 0.1) This represents the steady state temperature of a bar with a uniformly applied heat source.5. 2. i = 1. We can write the differential equation as a difference equation: ui−1 − 2ui + ui+1 = −1 2h or ui−1 − 2ui + ui+1 = −0. 4. we will require: u5 = u3 . Implementation in a linear equation by elimination Consider the BVP: uxx = −1.e. u′ (1) = 0. then you could let the first two grid points be at x0 = −h/2 and x1 = h/2. For example suppose we want to impose insulated boundary at the left end of a bar. where you let the second bar be mirror image of the first bar. with one end held at a fixed temperature and the other end insulated. and x0 = 0.133 u T u u copy u E e xn−2 xn−1 xn u u u xn+1 e Figure 38. This makes the central difference for u′ (x4 ) be zero.25 is outside the region and thus fictional. Another practical way to implement an insulated boundary is to let the grid points straddle the boundary. If you do this. x3 = . If we use 4 equally spaced intervals.25. ⇒ h= L 1 = . A way to think of an insulated boundary that makes sense of the point L + h is to think of two bars joined end to end.75. then m = 4. x2 = . The boundary condition at x0 = 0 is implemented as u0 = 5. then no heat will flow across the joint. which is exactly the same effect as insulating. x4 = 1. i.25. 3. and x5 = 1.

5x2 . An example The steady state temperature u(r) (given in polar coordinates) of a disk subjected to a radially symmetric heat load g(r) and cooled by conduction to the rim of the disk and radiation to its environment is determined by the boundary value problem: ∂ 2 u 1 ∂u + = d(u4 − u4 ) − g(r) b ∂r2 r ∂r with u(R) = uR and u′ (0) = 0. (38. u5 . INSULATED BOUNDARY CONDITIONS For i = 1. u2 .5. u] > x = 0:. recalling that u0 = 5 we have: 5 − 2u1 + u2 = −. However. Solve this system and plot the results: > u = A\b > u = [5 .2) Here ub is the (fixed) background temperature and uR is the (fixed) temperature at the rim of the disk. − .5.5. . we need to copy values from inside the region to fictional points just outside the region.25:1 > plot(x.’d’) The exact solution of this BVP is: Use hold on and plot this function on the same graph to compare.5.2 for an illustration. Insulated boundary conditions in time-dependent problems To implement the insulated boundary condition in an explicit difference equation with time. See Figure 38.5.134 LECTURE 38. −. . . from the boundary condition u5 = u3 and so we can eliminate u5 from our equations and write: 2u3 − 2u4 = −. . or − 2u1 + u2 = −5. −.5.5)′ .5.5 For i = 2 and i = 3 we have: u1 − 2u2 + u3 = −. .5 For i = 4 we have: Note that we now have 5 unknowns in our problem: u1 . You should see that our approximate results are not bad! U (x) = 5 + x − . and u2 − 2u3 + u4 = −. u3 . u4 )′ and write the 0 1 −2 2  0 0   1  −2 u3 − 2u4 + u5 = −.u. Summarizing. and b = (−5. we can put the unknown quantities equations as a matrix equation Au = b where:  −2 1  1 −2 A=   0 1 0 0 in a vector u = (u1 .

2: Illustration of information flow for the explicit method with an insulated boundary condition. You will need to change the domain to: x = 0:h:L+h and change the dimensions of all the other objects to fit this domain. Instead. T = 20. Set m = 20 and experiment with n.5.2 Modify the program myheat. Notice that the equations have a singularity (discontinuity) at r = 0. How does this affect the final temperature at r = 0? How does it affect the rate of convergence of the method? Turn in your derivation. Exercises 38. derive the finite difference equations for this example. d = . Turn in your program and plots. 38.2). g1 (t) = sin(t) and f (x) = − sin(x/4).135 ‚ x                x x x h Ts d d d  Ts    Ts   d  d  d ‚ dx dx dh     x d d d  Ts    Ts   Ts d  d  d ‚ dx dx dh     x Ts d d d  Ts    Ts   d  d  d ‚ dh dx dx   x   tj+1   x tj   x   x xm−1 xm xm+1 Figure 38.1. Illustrate this comparison on a single plot. Get a plot when the program is stable. The open circles are the fictional points and the curved arrows represent copying the value. Run the program with L = 2π.m to have an insulated boundary at x = L (rather than u(L.1) except do not include the fictional point x5 . (Your system of equations should be 3 × 3. the plots for both d = 0. In the program change the radiation coefficient d to . uR = ub = 10 and g(r) = (r − 5)2 . and answers to the questions. Then solve for ui and get the relaxation update formula used in program myheatdisk.01. let x4 be the last point and impose the insulated boundary by requiring u4 = u3 .01.) Compare this solution with the true solution and the better approximation in the lecture. The class web site has a program myheatdisk.1 Redo the calculations for the BVP (38.m that implements these equations for parameter values R = 5.3 Starting from (38. . c = . t) = g2 (t)). 38. How does the program avoid this problem? How does the program implement uR = 10 and u′ (0) = 0? Run the program.1 and d = 0.m.

Lecture 39 Finite Difference Method for Elliptic PDEs Examples of Elliptic PDEs Elliptic PDE’s are equations with second derivatives in space and no time derivative. steady state chemical distributions. or if forces are applied to the sheet then it will deflect by an amount u(x. y) on ∂R. c ≤ y ≤ d}. y) at each point (x. Suppose that a membrane such as a sheet of rubber is stretched across a rectangular frame. For the sake of clarity we will only consider the two dimensional problem. The most important examples are Laplace’s equation ∆u = uxx + uyy + uzz = 0 and the Poisson equation ∆u = f (x. ∂R is the edge of the rectangle. y) for for (x. then the step size in the x and y directions respectively are h= b−a m and k= d−c .1) where R is the rectangle. y). y) is the deflection at the edge. electrostatic potentials. z). y) in R. (x. The Finite Difference Equations Suppose the rectangle is described by R = {a ≤ x ≤ b. f (x. If we have m subdivisions in the x direction and n subdivisions in the y direction. y) u(x. These equations are used in a large variety of physical situations such as: steady state heat problems. A good model problem in this dimension is the elastic deflection of a membrane. We will divide R in subrectangles. If some of the edges of the frame are bent. y) is the force density (pressure) applied at each point and g(x. y. This u will satify the boundary value problem: uxx + uyy = f (x. (39. elastic deformation and steady state fluid flows. y) = g(x. n 136 .

and zero everywhere in the interior. the matrix A has dimensions (m − 1)(n − 1) × (m − 1)(n − 1). Thus the program will not do any iteration. and ui.5) This method is another example of a relaxation method. there are (m − 1) × (n − 1) grid points where we need to determine the solution. A and b. many of its elements are zero. just as we did in the relaxation method for the nonlinear boundary value problem.j − 2uij + ui−1.j ) + h2 (ui.j = g(a. yj ). Thus we could solve the equations exactly using matrix methods.4) Iterative Solution A usually prefered alternative to the direct method described above is to solve the finite difference equations iteratively.1 first then u:. ui.j = g(b. The boundary conditions are introduced by: u0.3).m on the class web site. + k2 ) (39. rather then a matrix. To do this we would first need to express the uij ’s as a vector. Such a matrix is called sparse and there are special methods intended for efficiently working with sparse matrices. Then we could solve an equation of the form Au = b. notice that the equations are all linear.2 . first solve (39. Although A is large. yj ). (39.2) for uij .n = g(xi .2) Direct Solution of the Equations Notice that since the edge values are prescribed.0 = g(xi . Further. Download and read it. An advantage of the direct method is that solving (39. yj ) = fij 2 h k2 for 1 ≤ i ≤ m − 1 and 1 ≤ j ≤ n − 1. Next we would need to write the matrix A that contains the coefficients of the equations (39. Finally. To do this there is a standard procedure: let u be the column vector we get by placing one column after another from the columns of (uij ).j+1 − 2uij + ui. but will plot the initial guess. Setting up and solving this equation is called the direct method. along with (39. To do this. Thus we would list u:. d). You will notice that maxit is set to 0.4) can be done relatively quickly and accurately. etc..j−1 ) − h2 k 2 fij . which is confusing. If this method converges. The iterative solution is implemented in the program mypoisson.2). Note also that there are exactly (m − 1) × (n − 1) equations in (39.j+1 + ui.1) by replacing uxx and uyy by their central differences to obtain ui. Using this formula. we can update uij from its neighbors.j + ui−1. um. The drawback of the direct method is that one must set up u. c). gradually increase maxit.137 We obtain the finite difference equations for (39. The initial guess in this case consists of the proper boundary values at the edges. (39. which yields uij = 2(h2 1 k 2 (ui+1. which can be rather large.j + = f (xi . .j−1 ui+1. then the result is an approximate solution. To see the solution evolve.2) and incorporate the boundary conditions in a vector b.3) (39.

138 LECTURE 39. Obtain plots for small and large p. For large p plot also a couple of intermediate steps. ux (b.m (from the class web page) in the following ways: • Change x = b to be an insulated boundary.e. i. y) = 0. y) to a negative constant −p. Tell how many iterations are needed for convergence in each. . FINITE DIFFERENCE METHOD FOR ELLIPTIC PDES Exercises 39. Experiment with various values of p and maxit. • Change the force f (x.1 Modify the program mypoisson.

1 139 .Lecture 40 Convection-Diffusion Equations Exercises 40.

5 2 2. Finite elements is a method that works for any shape region because it is not built of a grid.m. cutting the region up into triangles as we did in a previous lecture.1: An annular region with a triangulation. but on triangulation of the region. Each row of T has three integer numbers indicating the indices of the nodes that form a triangle.5 −2 −1.5 Figure 41. The following figure shows a triangularization of a region.5 −1 −0. Open the program mywasher. either rectangular or a regular part of a rectangle.5 0 x 0.5 1 1. This program defines a triangulation by defining the vertices in a matrix V in which each row contains the x and y coordinates of a vertex.5 −2. so T1 is the triangle 140 . Notice that we list the interior nodes first.5 0 y −0.Lecture 41 Finite Elements Triangulating a Region A disadvantage of finite difference methods is that they require a very regular grid.m.5 2 1. For instance the first row is 43 42 25.5 −2 −2. then the boundary nodes. 2. This figure was produced by the script program mywasher. Notice that the nodes are evenly distributed.5 1 0. and thus a very regular region. Triangles are defined in the matrix T.e.5 −1 −1. i. Notice that the nodes and triangles are very evenly spaced. This is good for the finite element process where we will use it.

This function is the finite element.1) Thus finding a finite element solution amounts to finding the best values for the constants {Cj }n . then the function is a finite element. The matrix T in this case was produced by the Matlab command delaunay. We a three dimensional plot of the region and triangles is produced by having the last line be trimesh(T. (41. Thus we see that the constants Cj are just the values at the nodes. These values give the height at each node in the graph. then a triangulation is just a subdivision into subintervals. However. Φj is a linear function. (41.141 h ¤f ¢e ¡e ¡f ¢ ¡ e ¡ f ¢ h ¤ f ¢ e h¤ f ¢ e ¡ e ¡ f ¢ h¤ f¢ e¡ e¡ f¢ ¤h ¢f ¡e ¡e ¢f ¤h ¢ f ¡ e ¡ e ¢ f ¤ h ¢ f ¡ ¡ ¢ f e e ¤t ht ¢ ft ¡ et ¡ e¢ t ft Figure 41. Φj is composed of triangular piece of planes.) If we consider one dimension.z). a finite element is actually a very simple object. Also notice that if we denote the j-th vertex by vj . Between nodes. What is a finite element? The finite element method is a mathematically complicated process. The command produced more triangles than desire and the unwanted ones were deleted.2 we show an uneven subdivision of an interval and the finite elements corresponding to each node. (There are fancier types of finite elements that we will not discuss. In two dimensions. In Figure 41.x. the vector z contains the node values Cj .m. Thus the solution using linear elements is a piecewise linear function. v42 and v25 . then for one interior node.y. What is a finite element solution? A finite element (approximate) solution is a linear combination of the elements: n U (x) = j=1 Cj Φj (x). with vertices v43 .2: The finite elements for the triangulation of a one dimensional object.2) . Do this for one boundary node. For instance if we set all equal to 0 except one equal to 1. Notice that a sum of linear functions is a linear function. j=1 In the program mywasher. then U (vj ) = Cj . Thus Φj is a function whose graph is a pyramid with its peak over node vj . To each node vj we associate a function Φj that has the properties Φj (vj ) = 1 and Φj (vi ) = 0 for i = j.

j=1 Experiment with finite elements By changing the values in c in the program we can produce different three dimensional shapes based on the triangles. gives us complete knowledge of the solution. Thus the constants Cj for the boundary nodes are set at exactly the value of the boundary condition at the nodes.e. Since Φℓ (vℓ ) = 1 and all the other elements are zero at node vℓ . of the form u(x) = g(x). FINITE ELEMENTS Take the one-dimensional case. we must choose Cℓ = g(vℓ ). Values at boundary nodes Once a triangulation and a set of finite elements is set up. This task can be subdivided into two parts: (1) assigning the values at the boundary nodes and (2) assigning the values at the interior nodes. the next step is to incorporate the boundary conditions of the problem. . In particular.3: A possible finite element solution for a one dimensional object. then to make n U (vℓ ) = j=1 Cj Φj (vℓ ) = g(vℓ ). where D is the object (domain) and ∂D is its boundary.3 could be a finite element solution since it is piecewise linear. Figure 41. Since we know that the solution is linear on each subinterval.142 LECTURE 41. Then the boundary condition directly determines the values on the boundary nodes.     e e e ¤ ¤ t ¤ ¤ ¤d   ¤ d d  ¤ ¤ e e t t t t t Figure 41. suppose that vℓ is a boundary node. In the two-dimensional case.e. i. and so again. Values are assigned at each node and a linear interpolant is used in between. knowing the values at the endpoints of the subintervals. i. The point then of a finite element solution is to find the values at the nodes that best approximate the true solution. Suppose that we have fixed boundary conditions. for x ∈ ∂D. the nodes. the solution is linear on each triangle. if we know the values {Cj }n at the nodes then we know everything.

^2 .143 Thus. In the program mywasher the first 32 vertices correspond to interior nodes and the last 32 correspond to boundary nodes.5.^2. By setting the last 32 values of z. if there are m interior nodes.2*cos(y(49:64)).1 Generate an interesting or useful 2-d object and a well-distributed triangulation of it.5*y(33:48). we achieve the boundary conditions. Turn in your code and the four plots.. • Plot one boundary finite element. We could do this by adding the following commands to the program: z(33:64) = . . Exercises 41. • Plot the region. for all m + 1 ≤ j ≤ n. z(49:64) = . • Plot one interior finite element. then Cj = g(vj ). • Assign values to the boundary using a function (or functions) and plot the region with the boundary values. or more elaborately we might use functions: z(33:48) = x(33:48).

y) into the energy integral and minimize. (42. In this case the free variables are {Cj }m . j=1 The interior nodes values are determined by variational principles. u) is a solution of (42. where n is the number of nodes. The finite element solution Recall that a finite element solution is a linear combination of finite element functions: n U (x. To obtain the values at the internal nodes. for the steady state heat equation uxx + uyy = g(x. It can be shown that u(x. y). Variational Principles For the differential equations that describe many physical systems. y) the internal energy is the integral I[u] = R (42. In this as with any other j=1 minimization problem. x y (42.3) 144 . Thus j=1 to find the minimizer we should try to solve ∂I[U ] =0 ∂Cj for 1 ≤ j ≤ m. For instance. the way to find a possible minimum is to differentiate the quantity with respect to the variables and set the results to zero.1) u2 + u2 + 2g(x. we find the minimum of I[U ] for all choices of {Cj }m . It is a physical principle that systems seek to be in a state of minimal energy and this principle is used to find the internal node values. y) dA. y) = j=1 Cj Φj (x. a finite element solution of a boundary value problem boils down to finding the best values of the constants {Cj }n . y)u(x. the internal energy of the system is an integral. which are the values of the solution at the nodes.2). we will plug U (x.2) where R is the region on which we are working. Variational principles usually amount to minimizing internal energy.Lecture 42 Determining Internal Node Values As seen in the previous section.1) if and only if it is minimizer of I[u] in (42. That is. where m is the number of internal nodes.

First note that since n U (x. y). y) . y)Φj (x. y) dA.2).4) Differentiating with respect to Cj we obtain the internal node equations 0= R 2Ux ∂Uy ∂U ∂Ux + 2Uy + 2g(x. The integral (42. y). y) = aij + bij x + cij y. ∂Cj n Also note that Ux (x. .5) Now we have several simplifications. (42. y)2 + Uy (x. y)U (x. y) = Φj |Ti (x. (42. The following two facts answer these questions. y) into the energy integral for the heat equation (42. Application to the steady state heat equation If we plug the candidate finite element solution U (x. • For most applications the internal node equations give a minimizer. we obtain I[U ] = R Ux (x. y) = j=1 Cj Φj (x. y) dA for 1 ≤ j ≤ m. y)2 + 2g(x. We can demonstrate the first fact using an example. Next we use the fact that the region R is subdivided into triangles {Ti }p and the functions in i=1 question have different definitions on each triangle. The integral then is a sum of the integrals: p 0=2 i=1 Ti Ux (Φj )x + Uy (Φj )y + g(x. ∂Cj ∂Uy ∂Cj Similarly.5) then becomes 0=2 Ux (Φj )x + Uy (Φj )y + g(x. y) dA for 1 ≤ j ≤ m. Now note that the function Φj (x. These facts make the finite element method practical: • For most applications the internal node equations are linear. y)Φj (x. is the solution actually a minimizer (and not a maximizer). y) is linear on triangle Ti and so Φij (x. = (Φj )y . and if so. y) = Cj j=1 ∂ Φj (x. y) dA for ∂Cj ∂Cj ∂Cj 1 ≤ j ≤ m.145 We call this set of equations the internal node equations. At this point we should ask whether the internal node equations can be solved. we have ∂U = Φj (x. ∂x and so ∂Ux = (Φj )x .

y) dA . y) = cij . Ux and Uy restricted to Ti have the form: n n Ux = k=1 Ck bik and Uy = k=1 Ck cik . we get the approximation Φij (xj . we apply the Three Corners rule to make an approximation to the integral g(x. you first need to set up the region and boundary values by running a script such as mywasher or mywedge. yj ) then Φij (xj . The internal node equations then reduce to p n n 0= i=1 Ti k=1 n k=1 Ck bik bij + k=1 Ck cik cij + g(x. Experiment Download the program myfiniteelem. You will see that the program works no matter what you choose. we have n n n Ck bik Ti k=1 bij + k=1 Ck cik cij = k=1 Ck bik bij + k=1 Ck cik cij Ai . y)Φij (x. yj ) = 1. yj )Ai /3. where Ai is just the area of Ti . these equations are in fact linear in the unknowns {Cj }. Summarizing. yj ) = 0 if Ti does not have a corner at (xj . DETERMINING INTERNAL NODE VALUES This gives us the following simplification: (Φij )x (x. and. If Ti does have a corner at (xj . Finally. y)Φij (x. Now notice that ( n Ck bik ) bij is just a constant on Ti . yj ).1 Study for the final! . Exercises 42. yk ) = 0 if k = j and even Φij (xj . y) dA for 1 ≤ j ≤ m.146 LECTURE 42. Also. To use it. This program produces a finite element solution for the steady state heat equation without source term: uxx + uyy = 0 . While not pretty. y) = bij and (Φij )y (x. yj )g(xj . yj ) Ai 3 for 1 ≤ j ≤ m. thus.m from the class web site. Ti Since Φij (xk . the internal node equations are: p n n 0= i=1 k=1 Ck bik bij + k=1 Ck cik 1 cij + g(xj . yj )Φij (xj . Try different settings for the boundary values z.

Review of Part IV Methods and Formulas Initial Value Problems Reduction to First order system: For an n-th order equation that can be solved for the n-th derivative: x(n) = f Then use the standard change of variables: y1 = x y2 = x ˙ . . . yi + k1 ) 1 yi+1 = yi + (k1 + k2 ) 2 (42.6) (42. ˙ Euler’s method: yi+1 = yi + hf (ti . dtn−1 t. (42. y1 . yn ). . . . . x. yi ). . ˙ ¨ dn−1 x dtn−1 . .8) dn−1 x . yi ) k2 = hf (ti + h. Modified (or Improved) Euler method: k1 = hf (ti .7) Boundary Value Probems Finite Differences: Replace the Differential Equation by Difference Equations on a grid. . . y2 . 147 . x. . x. . yn = x(n−1) = Differentiating results in a first-order system: y1 = x = y2 ˙ ˙ y2 = x = y3 ˙ ¨ . Review the lecture on Numerical Differentiation. yn = x(n) = f (t.

j .j+1 − uij k uxx → REVIEW OF PART IV ui−1.j+1 + ui+1.j + rui+1. k = T /n. An element: Pyramid shaped function at a node.j h2 (42. which is always stable and has truncation error O(h2 + k 2 ) or better if r and λ are chosen optimally. Cj is the value of the solution at node j.j+1 − 2ui. Cj at interior nodes are determined by variation principles. and r = ck/h2 . h2 k2 Finite Elements: Based on triangles instead of rectangles. y) = j=1 Cj Φj (x.tspan. where n is the number of nodes. ui. RK 45 uses variable step size to control speed vs.j+1 + 2(1 + r)ui.j+1 − uij k uxx → ui−1.j = −rui−1. y) → ui+1.j+1 + (1 + 2r)ui. accuracy.10) ui. ui.j .j+1 = rui−1. Matlab > [T Y ] = ode45(f.y0) Initial value problem solver that uses the Runge-Kutta 45 method.9) where h = L/m. Crank-Nicholson Method Finite Differences for Parabolic PDE (heat): −rui−1.j−1 + = f (xi .j + rui+1.j − 2uij + ui−1. Implicit Method Finite Differences for Parabolic PDE (heat): ut → leads to: which is always stable and has truncation error O(h2 + k).j+1 − 2uij + ui. The stability condition is r < 1/2. The last step in determining Cj ’s is solving a linear system of equations. Cj at the boundary nodes are given by boundary conditions.j ui. A finite element solution is a linear combination of finite element functions: n U (x.148 Explicit Method Finite Differences for Parabolic PDE (heat): ut → leads to: ui.j + (1 − 2r)ui.j − 2uij + ui+1. f must be a vector valued function.j+1 h2 (42. y0 is the initial vector. y). and where U is an approximation of the true solution.j + 2(1 − r)ui.j+1 = rui−1. yj ) = fij .j+1 . Can be used for irregularly shaped objects.j+1 − rui+1.j+1 − rui+1. . Finite Difference Method for Elliptic PDEs: uxx + uyy = f (x.

Part V Appendices

c Copyright, Todd Young and Martin Mohlenkamp, Mathematics Department, Ohio University, 2007

Appendix A Sample Exams
MATH344 Midterm Exam – Spring 04
1. For f (x) = x2 − 5, do 2 iterations of Newton’s method, starting with x0 = 2.0. What is the relative error of x2 ? About how many more steps would be needed to make the error less than 10−16 ? 2. Write a Matlab program to do n steps of the bisection method for a function f with starting interval [a, b]. Let f , a, b and n be the inputs and the final x the output. 3. Given a data set represented by vectors x and y, describe how you would use Matlab to get a Least Squares Approximation, Polynomial Interpolation and Spline Interpolation? 4. Given that the LU decomposition of A = Ax = b where b = (1, 2) . 5. Suppose A−1 = 1 2 −1 2 . Using v0 = (1, 1)′ as the starting vector do 2 iterations of the 1 −1 Inverse Power Method for A. What do the results mean? .5 1 . Find the LU factorization with pivoting.

3 3 1 2

is LU =

1 1/3

0 1

3 3 0 1

, solve

6. Let A =

7. What is the condition number of a matrix? How do you find it with Matlab? What are the implications of the condition number when solving a linear system? What is the engineering solution to a problem with a bad condition number? 8. Write a Matlab program to do n iterations of the Power Method. Let the matrix A and n be inputs and let [e v] (the eigenvalue and eigenvector) be the outputs. 9. Write a Matlab program to that solves a linear system Ax = b using LU decomposition. Let A, b and tol be the inputs and x the output. If the error (residual) is not less than tol, then display a warning. 10. List your 10 least favorite Matlab commands. X. What problem did Dr. Young totally botch in class?

150

151

MATH 344 – Midterm Exam – Winter 05
1. For f (x) = x2 − 5, do 2 iterations of the bisection method, starting with [a, b, ] = [2, 3]. What is the relative error? About how many more steps would be needed to make the error less than 10−6 ? 2. Write a Matlab program to do n steps of Newton’s method for a function f with starting interval [a, b]. Let f , f ′ , x0 and n be the inputs and the final x the output. 3. Suppose f (x) has been defined as an inline function. Give Matlab commands to plot it on the interval [0, 10]. 4. Write a function program which will find the roots of a function f on an interval [a, b]. 5. Suppose A = −1 2 . Using v0 = (1, 1)′ as the starting vector do 2 iterations of the 1 −1 Power Method for A. What do the results mean? −1 5 2 2 . Find the LU factorization with pivoting.

6. Let A =

7. What is the condition number of a matrix? How do you find it with Matlab? What are the implications of the condition number when solving a linear system? What is the engineering solution to a problem with a bad condition number? 8. Write a Matlab program to do n iterations of the QR Method. Let the matrix A and n be inputs and let e be the output. 9. Write a Matlab program to that solves a linear system Ax = b using LU decomposition. Let A, b and tol be the inputs and x the output. If the error (residual) is not less than tol, then display a warning. 10. Give the Matlab commands, or sequences of commands for solving a linear system Ax = b in as many ways as you know. Which of these are the worst and best? X. When using Newton’s method, how does one measure its effectiveness?

starting with x0 = 2. 3. Write a Matlab function program that calculates the sum of the squares of the first n integers.. What is the command in Matlab to produce the eigenvalues and eigenvectors of a matrix.152 APPENDIX A. x b (number). 6. Find the eigenvalues and eigenvectors of the matrix: 3 2 A= 2 3 9. Find the LU decomposition of A using pivoting if needed: 3 −2 A= 6 1 5. y) = x3 − y 2 + 1 and f2 (x. do 2 iterations of Newton’s method. . 0). SAMPLE EXAMS MATH 344 – Midterm Exam – Spring 2006 1. Write a Matlab function program that takes an input n. 10. . Splines and Least Squares Fitting? 4. X. What 10 commands in Matlab are the least useful? 2.. produces a random n × n matrix A and random vector ¯ solves A¯ = ¯ (using the built in command) and outputs the residual b. every problem leads to . In this class. What is the condition number of a matrix? How do you find it with Matlab? What are the implications of the condition number when solving a linear system? 8. Which method does it use? What will be the form of the output? 7. y) = y 3 + x − 1 starting from the initial guess (0. For f (x) = x3 − 6. What are the main differences in the uses of: Polynomial Interpolation. Write a Matlab script program that will use Newton’s method to find a root of the system of functions f1 (x.

Write a Matlab program to do n steps of the Euler method for a differential equation ˙ x = f (x. Describe an application of ew’s and ev’s.153 MATH344 Final Exam – Spring 2004 1. Explain how the difference equations could be solved as a linear system. Very generally. the endpoints a.f1. with u = 0 on all the boundaries.n). 5. What is a finite element and a finite element solution? 15. 9. on the rectangle 0 ≤ x ≤ a and 0 ≤ y ≤ b. 10. What is the geometric meaning of eigenvalue and eigenvector. 4.tol). Let the first line be: function x = mynewton(f. with x(a) = x0 . x] = myeuler(f. X. 6. Give the Matlab command(s) for as many different ways as you know to solve a system of linear equations. Let the first line be: function [t. Include f ′ in the inputs. Rank them. how are the boundary and interior values of the finite element solution obtained. Let the inputs be the function f . When you have two different methods of approximating something. 3. how can you get an even better approximation? π . 2.a. Estimate the integral 0 sin x dx using L4 . What is the command for it in Matlab? 7. t). When and why does the explicit finite difference method for the heat/diffusion equation become unstable? 8.x0. 14. Set up the finite difference equations for the BVP: uxx + uyy = f (x.n.x0. Derive the implicit finite difference equations for solving the heat/diffusion equation ut = cuxx . 11. b and the number of subintervals n. with correct syntax. Write a Matlab program to do Newton’s method with starting point x0 that does n steps or stops when a tolerance is reached. Calculate the exact value and the percentage errors of each of the approximations. 12.b. Splines and Least Squares Interpolations. What is the condition number of a matrix? How do you find it with MatLab? What are the implications of the condition number when solving a linear system? What is the engineering solution to a problem with a bad condition number?. Write a Matlab program to do the midpoint method for integration. Describe RK45. Discuss uses of Polynomial Interpolation. Write the equation θ” + aθ′ + b sin θ = c sin t as a linear system of first order equations and set up the Euler method for the system. 13. What is the Matlab command to find them. y). R4 and T4 .

with boundary conditions.tol). 15. Discuss: triangulation. 13. t) = a. Explain how to avoid the problem at r = 0. How do you get them from Matlab? 14. Write a Matlab program to do Newton’s method with starting point x0 that does n steps or stops when a tolerance is reached. with x(a) = x0 . Calculate the exact value and the 1. Write a Matlab script program that calculates . Give all the Matlab commands needed to solve this IVP. Let the first line be: function [t. What are: Polynomial Interpolations. 4. What is the command for it in Matlab? 7. 9. r with u(R) = 0 and ur (0) = 0. θ(0) = π/4.n). . Write a Matlab program to do n steps of the modified Euler method for a differential ˙ equation x = f (x. Write a Matlab program to do n iterations of the inverse power method. R4 . SAMPLE EXAMS MATH344 – Final Exam – Winter 2005 16 √ x dx using L4 . Describe methods for approximating double integrals.a.. finite element and finite element solution. t). u(0. Set up the finite difference equations for the BVP: urr + 1 ur = f (r).154 APPENDIX A. When I can’t get my Matlab program to work. θ′ (0) = 0 as a system of first order equations. 10. Write the IVP: θ” + aθ′ + b sin θ = c sin t. 11. Let the inputs be the function f .5i sin(i) by adding terms until the sum i2 i=0 stops changing. the endpoints a. Explain why order matters in engineering problems.x0. What is the meaning of e and v? 12. 6. I . Let the matrix A and n be the inputs and [e v] be the output. Estimate the integral 0 percentage errors of each of the approximations.n. on the interal 0 ≤ r ≤ R. t) = b. 2. x] = myeuler(f. Splines and Least Squares Interpolations. T4 and S4 . Derive the explicit finite difference equations for solving the heat/diffusion equation ut = cuxx . u(L. Describe RK45. and u(x.f1. 8. X. Write a Matlab function program to do the trapezoid method for integration. How do you know that it would stop? ∞ 5. Let the first line be: function x = mynewton(f. How are the boundary and interior values of the finite element solution obtained. b and the number of subintervals n. 3. . 0) = f (x)..b.x0. Include f ′ in the inputs.

Write a Matlab function program to do the Trapezoid Rule for integration of data. If U (x) = j=1 Cj Φj (¯) is a finite element solution. Find the percentage errors of x0 . with x(a) = x0 . r with u(R) = 4 and ur (0) = 0. What is variable step size? How is it implemented RK45? 11. Write the IVP: θ′′ +. Write a Matlab function program to do n steps of the Modified Euler method for a differential ˙ equation x = f (t.5θ′ +sin θ = sin 2t. θ′ (0) = 0 as a system of first order equations. x1 and x2 . 9. Describe and give formulas for 2 methods to approximate double integrals based on triangles. on the time interval [a. with starting point x0 = 2. Do 2 steps of Newton’s method to solve the equation f (x) = x2 − 5 = 0. Approximate the integral accurate? π 0 sin x dx using M4 and S4 . 6. Explain how to incorporate an insulated boundary in a finite difference method. Let A = 1 2 2 1 . Find the LU factorization with pivoting. 3. Let the inputs be vectors x and y. What is the meaning of e and v? 14. Write a Matlab program to do n iterations of the power method.155 MATH344 – Final Exam – Spring 2006 1. x). how can you get a potentially more accurate method? . Explain how to avoid the problem at r = 0. 10. Give all the Matlab commands needed to solve this IVP on the interval 0 ≤ t ≤ 10. where it is assumed that y is a function of x and x is not necessarily evenly spaced. . on the interval 0 ≤ r ≤ R. 7. 5. 4. b]. When you have two different methods to make an approximation. θ(0) = 1.5). Set up the finite difference equations for the BVP: urr + 1 ur = f (r). what is the meaning of Cj ? What is the x final step of finding a finite element solution? 13. Write a Matlab script program that will use Newton’s method to find a root of the system of functions f1 (x.5. What is a polynomial interpolation? How do you get one in Matlab? X. What are main differences between the Finite Difference Method and Finite Elements Method? n 12. Which do you expect to be more 2. Let the matrix A and n be the inputs and [e v] be the output. 8. y) = x3 + y − 1 and f2 (x. y) = y 3 − x + 1 starting from the initial guess (. 15.

Type help power for information. ^ Scalar or matrix powers. π = 3. i. Type help plus for information. 156 .e. . Not a number. Scalar or matrix multiplication. Type help slash for information. Built-in Mathematical Functions abs(x) Absolute value|x|.14159 . . . Type help times for information. Type help mpower for information. Scalar or right matrix division. Type help mtimes for information.Appendix B Glossary of Matlab Commands Mathematical Operations Addition./ Element by element division. Subtraction.* Element by element multiplication.. . √ −1. asin(x) Inverse sine arcsin x. + * / Built-in Mathematical Constants eps i Inf NaN pi Machine epsilon. approximately the computer’s floating point roundoff error. the command A/B is equivalent to A*inv(B). For matrices. Type help minus for information. Indicates an invalid operation such as 0/0.^ Element by element exponentiation. . acos(x) Inverse cosine arccos x. ∞.

√ Square root x. Displays double-precision representation of a symbolic expression. Cotangent cot x. Hyperbolic cosine cosh x. Symbolic differentiation. Secant sec x. Tangent tan x. dblquad Double integration. Hyperbolic sine sinh x. either definite or indefinite. Cosine cos x. inline Define a function in the command window.157 atan(x) cos(x) cosh(x) cot(x) exp(x) log(x) sec(x) sin(x) sinh(x) sqrt(x) tan(x) tanh(x) max mean min Inverse tangent arctan x. Symbolic integration. Computes maximum of the rows of a matrix. Sine sin x. ode113 Numerical multiple step ODE solver. . Hyperbolic tangent tanh x. ode45 Runga-Kutta 45 numerical ODE solver. Computes the minimum of the rows of a matrix. quad Numerical integration using an adaptive Simpson’s rule. Natural logarithm ln x = loge x. Built-in Numerical Mathematical Operations fzero Tries to find a zero of the specified function near a starting point or on specified interval. triplequad Triple integration. Symbolic ODE solver. Expands an algebraic expression. Factor a polynomial. Computes the average of the rows of a matrix. Exponential function ex = exp x. Composition of symbolic functions. a Built-in Symbolic Mathematical Operations collect compose diff double dsolve expand factor int Collects powers of the specified variable is a given symbolic expression.

cd Change directory. Plots a vector field. Plots data vectors. Type help colon for information. ans The value of the most recent unassigned. taylor Gives a Taylor polynomial approximation of a given order at a specified point. clear Clears all values and definitions of variables and functions. sym or syms Create symbolic variables.158 APPENDIX B. Creates a log-log plot. Draws a mesh surface. Semilog plot. Plot based on a triangulation trisurf Surface plot based on a triangulation Special Matlab Commands : Range operator. and surf. Suppresses output. Also separates rows of a matrix. Graphics Commands contour contourf ezcontour loglog mesh meshgrid ezmesh plot ezplot plot3 polar quiver semilogy surf trimesh Plots level curves of a function of two variables. if it exists. Easy contour plot. Filled contour plot. Plots curves in 3-D. possibly with infinitely many entries. subs Substitutes for parts a a symbolic expression. used for defining vectors and in loops. with logarithmic scale along the vertical direction. mesh. Creates arrays that can be used as inputs in graphics commands such as contour. Easy mesh surface plot. symsum Performs a symbolic summation. Solid surface plot. . Plots in polar coordinates. You may also use to . pretty Displays a symbolic expression in a nice format. simple Simplifies a symbolic expression. = Assigns the variable on the left hand side the value of the right hand side. Easy plot for symbolic functions. quiver. GLOSSARY OF MATLAB COMMANDS limit Finds two-sided limit.

. . . . . . . . . load Load variables from a file. . . . . . Terminates an if. . . dir Lists the contents in the current working directory. . Commands for Matrices and Linear Algebra Matrix arithmetic: > A = [ 1 3 -2 5 . . . . Repeats a block of commands as long as a condition is true. . > format long. . Same as ls. . . Repeats a block of commands a specified number of times. Checks a condition before executing a block of statements. . . . diary Writes a transcript of a Matlab session to a file. Manually enter a matrix. .g. help inline Define an inline function. . Displays and error message and ends execution of a program. . . . . > u = [ 1 2 3 4]’ > A*u > B = [3 2 1. . . . . . . -1 -1 5 4 . Terminates execution of a program. . . . . . . . Alternative in an if statement. . . . 7 6 5. 0 1 -9 0] . . 4 3 2] > B*A . First word in a function program. . for or while statement. . Matlab Programming == ~= < > <= break end else error for function if return warning while Is equal? Is not equal? Less than? Greater than? Less than or equal? Breaks out of a for or while loop.159 clear only specified variables. . . . . . . . . . . save Saves workspace variables to a file. . multiply B times A. . format Specifies output format. . . . . Displays a warning message. e. .

. . > inv(C) . . . . . . . . . > min(C) . . > max(C) . . . . . . > diag(C) . . . . . . . . . . . . . . . .^3 . . . . . . .multiply a matrix by a scalar. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .add matrices. . . . . . . . . . . . . . . . . . . . . . . Special matrices: > I = eye(3) . . . . . . . . . . . . . . . . . . . . add a number to every entry of a matrix. .5) . . . . . . . . . . . . . . . . . . . . . . . . . . . just the diagonal elements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . random matrix with uniform distribution in [0. > sum(C) . . . . . . . . . . . the determinant of the matrix. . . . . . . . . . sums each row. . . . . . . . . . . . . . . .5) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . component-wise exponentiation. . > norm(C) . .the average of each row. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > det(C) . . . . . . . . . . .10) > C = rand(5. . . . . . . . . . . . . Matrix decompositions: > [L U P] = lu(C) > [Q R] = qr(C) > [U S V] = svm(C) . . . . . . . . . . component-wise multiplication. . . . . . . . . . . . . . . . . . . . . . . . . . B. . . . . . . . . . . . . . . . . . . . . . . . . . . . the minimum in each row. . . . . . . . . 1]. . . random matrix with normal distribution. .*B . . . . . . . . . . . . . . . . . . . . gives the dimensions (m × n) of A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > hilb(6) > pascal(5) General matrix commands: > size(C) . . . . . . . . . . . . > mean(C) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A + A . . . . A + 3 . . . . . . . . . . . . . . . . . . . . . . . . B. . . . . . identity matrix > D = ones(5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > C = randn(5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . gives the norm of the matrix. . . . . singular value decomposition. . . . . . . . . . . . . . . .160 > > > > > APPENDIX B. . . . . . . . . . . . . . inverse of the matrix. . GLOSSARY OF MATLAB COMMANDS 2*A . . .5) > O = zeros(10. . . . . . . . . . . . . . . . . . . . . the maximum of each row.

Bibliography [1] Ayyup and McCuen. [2] E. J. Addison-Wesley. 1996. Mathews. Johnston. Calculus. 2001 161 .

162 . which is a copyleft license designed for free software. Secondarily. 2007. regardless of subject matter or whether it is published as a printed book. Preamble The purpose of this License is to make a manual. while not being considered responsible for modifications made by others. APPLICABILITY AND DEFINITIONS This License applies to any manual or other work. 2008 Free Software Foundation. it can be used for any textual work. But this License is not limited to software manuals. We recommend this License principally for works whose purpose is instruction or reference. in any medium. 3 November 2008 Copyright c 2000. that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. which means that derivative works of the document must themselves be free in the same sense. 2001. Inc. <http://fsf. We have designed this License in order to use it for manuals for free software.GNU Free Documentation License Version 1. textbook. 1. This License is a kind of “copyleft”. It complements the GNU General Public License. or other functional and useful document “free” in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it. with or without modifying it. 2002. either commercially or noncommercially.3.org/> Everyone is permitted to copy and distribute verbatim copies of this license document. this License preserves for the author and publisher a way to get credit for their work. but changing it is not allowed.

refers to any such manual or work. Examples of suitable formats for Transparent copies include plain ASCII without markup. A Front-Cover Text may be at most 5 words. The “Document”. in the notice that says that the Document is released under this License. and is addressed as “you”. as Front-Cover Texts or Back-Cover Texts. Texinfo input format. If a section does not fit the above definition of Secondary then it is not allowed to be designated as Invariant. if the Document is in part a textbook of mathematics. The Document may contain zero Invariant Sections. below. modify or distribute the work in a way requiring permission under copyright law. or of legal. . If the Document does not identify any Invariant Sections then there are none. ethical or political position regarding them. (Thus. as being those of Invariant Sections. and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A “Modified Version” of the Document means any work containing the Document or a portion of it. or absence of markup. to use that work under the conditions stated herein. a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters. represented in a format whose specification is available to the general public. A copy made in an otherwise Transparent file format whose markup. You accept the license if you copy. philosophical. LaTeX input format. that is suitable for revising the document straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor. has been arranged to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not “Transparent” is called “Opaque”. Any member of the public is a licensee. or with modifications and/or translated into another language. The “Invariant Sections” are certain Secondary Sections whose titles are designated.GNU Free Documentation License 163 Such a notice grants a world-wide. royalty-free license. A “Secondary Section” is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document’s overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. commercial. unlimited in duration. and a Back-Cover Text may be at most 25 words. A “Transparent” copy of the Document means a machine-readable copy. in the notice that says that the Document is released under this License. SGML or XML using a publicly available DTD. The “Cover Texts” are certain short passages of text that are listed. either copied verbatim. An image format is not Transparent if used for any substantial amount of text.

such as “Acknowledgements”. PostScript or PDF designed for human modification. “Title Page” means the text near the most prominent appearance of the work’s title. VERBATIM COPYING You may copy and distribute the Document in any medium. PostScript or PDF produced by some word processors for output purposes only. but only as regards disclaiming warranties: any other implication that these Warranty Disclaimers may have is void and has no effect on the meaning of this License. XCF and JPG.164 GNU Free Documentation License and standard-conforming simple HTML. SGML or XML for which the DTD and/or processing tools are not generally available. 2. the copyright notices.) To “Preserve the Title” of such a section when you modify the Document means that it remains a section “Entitled XYZ” according to this definition. plus such following pages as are needed to hold. Examples of transparent image formats include PNG. “Endorsements”. However. Opaque formats include proprietary formats that can be read and edited only by proprietary word processors. legibly. The Document may include Warranty Disclaimers next to the notice which states that this License applies to the Document. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. the title page itself. (Here XYZ stands for a specific section name mentioned below. For works in formats which do not have any title page as such. you may accept compensation in exchange for copies. and that you add no other conditions whatsoever to those of this License. either commercially or noncommercially. the material this License requires to appear in the title page. and the license notice saying this License applies to the Document are reproduced in all copies. for a printed book. “Dedications”. A section “Entitled XYZ” means a named subunit of the Document whose title either is precisely XYZ or contains XYZ in parentheses following text that translates XYZ in another language. or “History”. These Warranty Disclaimers are considered to be included by reference in this License. provided that this License. The “publisher” means any person or entity that distributes copies of the Document to the public. and the machine-generated HTML. The “Title Page” means. . preceding the beginning of the body of the text. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may add other material on the covers in addition. under the same conditions stated above. you must either include a machine-readable Transparent copy along with each Opaque copy.GNU Free Documentation License 165 You may also lend copies. It is requested. and the Document’s license notice requires Cover Texts. and you may publicly display copies. and continue the rest onto adjacent pages. If you publish or distribute Opaque copies of the Document numbering more than 100. you must enclose the copies in covers that carry. 4. numbering more than 100. when you begin distribution of Opaque copies in quantity. clearly and legibly. COPYING IN QUANTITY If you publish printed copies (or copies in media that commonly have printed covers) of the Document. as long as they preserve the title of the Document and satisfy these conditions. If you use the latter option. thus licensing . provided that you release the Modified Version under precisely this License. free of added material. but not required. to give them a chance to provide you with an updated version of the Document. Both covers must also clearly and legibly identify you as the publisher of these copies. with the Modified Version filling the role of the Document. all these Cover Texts: Front-Cover Texts on the front cover. and Back-Cover Texts on the back cover. 3. you must take reasonably prudent steps. can be treated as verbatim copying in other respects. If the required texts for either cover are too voluminous to fit legibly. or state in or with each Opaque copy a computer-network location from which the general network-using public has access to download using public-standard network protocols a complete Transparent copy of the Document. Copying with changes limited to the covers. you should put the first ones listed (as many as fit reasonably) on the actual cover. MODIFICATIONS You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above. that you contact the authors of the Document well before redistributing any large number of copies. The front cover must present the full title with all words of the title equally prominent and visible. to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

as the publisher. one or more persons or entities responsible for authorship of the modifications in the Modified Version. be listed in the History section of the Document). given in the Document for public access to a Transparent copy of the Document. you must do these things in the Modified Version: A. and from those of previous versions (which should. authors. H. G. then add an item describing the Modified Version as stated in the previous sentence. as authors. together with at least five of the principal authors of the Document (all of its principal authors. I. E. unless they release you from this requirement. . Include. D. and publisher of the Modified Version as given on the Title Page. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document’s license notice. State on the Title page the name of the publisher of the Modified Version. These may be placed in the “History” section. Include an unaltered copy of this License. In addition. immediately after the copyright notices. Use in the Title Page (and on the covers. and publisher of the Document as given on its Title Page. B. year. create one stating the title. If there is no section Entitled “History” in the Document. You may use the same title as a previous version if the original publisher of that version gives permission. if there were any. Preserve all the copyright notices of the Document. List on the Title Page. and add to it an item stating at least the title. if it has fewer than five). in the form shown in the Addendum below. if any) a title distinct from that of the Document. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices. a license notice giving the public permission to use the Modified Version under the terms of this License.166 GNU Free Documentation License distribution and modification of the Modified Version to whoever possesses a copy of it. You may omit a network location for a work that was published at least four years before the Document itself. F. C. if any. Preserve its Title. or if the original publisher of the version it refers to gives permission. new authors. Preserve the network location. year. Preserve the section Entitled “History”. and likewise the network locations given in the Document for previous versions it was based on. J.

add their titles to the list of Invariant Sections in the Modified Version’s license notice. unaltered in their text and in their titles. statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard. N. Preserve all the Invariant Sections of the Document. For any section Entitled “Acknowledgements” or “Dedications”. M. The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version. but you may replace the old one. These titles must be distinct from any other section titles. You may add a passage of up to five words as a Front-Cover Text. Preserve any Warranty Disclaimers. L. and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein. provided that you include in the . If the Document already includes a cover text for the same cover.GNU Free Documentation License 167 K. to the end of the list of Cover Texts in the Modified Version. under the terms defined in section 4 above for modified versions. If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document. Section numbers or the equivalent are not considered part of the section titles. Such a section may not be included in the Modified Version. To do this. O. you may not add another. Delete any section Entitled “Endorsements”. previously added by you or by arrangement made by the same entity you are acting on behalf of. provided it contains nothing but endorsements of your Modified Version by various parties—for example. COMBINING DOCUMENTS You may combine the Document with other documents released under this License. on explicit permission from the previous publisher that added the old one. you may at your option designate some or all of these sections as invariant. Preserve the Title of the section. and a passage of up to 25 words as a Back-Cover Text. You may add a section Entitled “Endorsements”. Do not retitle any existing section to be Entitled “Endorsements” or to conflict in title with any Invariant Section. 5. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity.

then if the Document is less than one half of the entire aggregate. unmodified. If the Cover Text requirement of section 3 is applicable to these copies of the Document. If there are multiple Invariant Sections with the same name but different contents. In the combination. this License does not apply to the other works in the aggregate which are not themselves derivative works of the Document. and that you preserve all their Warranty Disclaimers. or else a unique number. the name of the original author or publisher of that section if known. make the title of each such section unique by adding at the end of it. COLLECTIONS OF DOCUMENTS You may make a collection consisting of the Document and other documents released under this License. and follow this License in all other respects regarding verbatim copying of that document. is called an “aggregate” if the copyright resulting from the compilation is not used to limit the legal rights of the compilation’s users beyond what the individual works permit. You may extract a single document from such a collection. the Document’s Cover Texts may be placed on covers that bracket the Document within the aggregate.168 GNU Free Documentation License combination all of the Invariant Sections of all of the original documents. When the Document is included in an aggregate. 6. and list them all as Invariant Sections of your combined work in its license notice. likewise combine any sections Entitled “Acknowledgements”. AGGREGATION WITH INDEPENDENT WORKS A compilation of the Document or its derivatives with other separate and independent documents or works. and distribute it individually under this License. The combined work need only contain one copy of this License. in or on a volume of a storage or distribution medium. you must combine any sections Entitled “History” in the various original documents. provided you insert a copy of this License into the extracted document. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work. forming one section Entitled “History”. or the . You must delete all sections Entitled “Endorsements”. and replace the individual copies of this License in the various documents with a single copy that is included in the collection. and any sections Entitled “Dedications”. and multiple identical Invariant Sections may be replaced with a single copy. in parentheses. provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects. 7.

and will automatically terminate your rights under this License.GNU Free Documentation License 169 electronic equivalent of covers if the Document is in electronic form. and you cure the violation prior to 30 days after your receipt of the notice. “Dedications”. if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation. However. If your rights have been terminated and not permanently reinstated. receipt of a copy of some or all of the same material does not give you any rights to use it. TERMINATION You may not copy. your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means. and any Warranty Disclaimers. then your license from a particular copyright holder is reinstated (a) provisionally. Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. or “History”. Replacing Invariant Sections with translations requires special permission from their copyright holders. In case of a disagreement between the translation and the original version of this License or a notice or disclaimer. You may include a translation of this License. TRANSLATION Translation is considered a kind of modification. 9. this is the first time you have received notice of violation of this License (for any work) from that copyright holder. sublicense. provided that you also include the original English version of this License and the original versions of those notices and disclaimers. the original version will prevail. Any attempt otherwise to copy. . the requirement (section 4) to Preserve its Title (section 1) will typically require changing the actual title. but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. If a section in the Document is Entitled “Acknowledgements”. or distribute it is void. and all the license notices in the Document. so you may distribute translations of the Document under the terms of section 4. and (b) permanently. modify. sublicense. if you cease all violation of this License. unless and until the copyright holder explicitly and finally terminates your license. or distribute the Document except as expressly provided under this License. modify. 8. Otherwise they must appear on printed covers that bracket the whole aggregate. Moreover.

“Incorporate” means to publish or republish a Document. (1) had no cover texts or invariant sections. in whole or in part. and subsequently incorporated in whole or in part into the MMC. “CC-BY-SA” means the Creative Commons Attribution-Share Alike 3. See http://www. 11. and if all works that were first published under this License somewhere other than this MMC. as part of another Document. as well as future copyleft versions of that license published by that same organization. An MMC is “eligible for relicensing” if it is licensed under this License. If the Document specifies that a particular numbered version of this License “or any later version” applies to it. provided the MMC is eligible for . Each version of the License is given a distinguishing version number. RELICENSING “Massive Multiauthor Collaboration Site” (or “MMC Site”) means any World Wide Web server that publishes copyrightable works and also provides prominent facilities for anybody to edit those works.0 license published by Creative Commons Corporation. California. a not-for-profit corporation with a principal place of business in San Francisco. The operator of an MMC Site may republish an MMC contained in the site under CC-BYSA on the same site at any time before August 1. 2009.170 GNU Free Documentation License 10. revised versions of the GNU Free Documentation License from time to time. FUTURE REVISIONS OF THIS LICENSE The Free Software Foundation may publish new.org/copyleft/. but may differ in detail to address new problems or concerns. you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. A public wiki that anybody can edit is an example of such a server. Such new versions will be similar in spirit to the present version. you may choose any version ever published (not as a draft) by the Free Software Foundation. If the Document specifies that a proxy can decide which future versions of this License can be used. A “Massive Multiauthor Collaboration” (or “MMC”) contained in the site means any set of copyrightable works thus published on the MMC site. that proxy’s public statement of acceptance of a version permanently authorizes you to choose that version for the Document. If the Document does not specify a version number of this License. and (2) were thus incorporated prior to November 1.gnu. 2008.

such as the GNU General Public License. and no BackCover Texts. and with the Back-Cover Texts being LIST. distribute and/or modify this document under the terms of the GNU Free Documentation License. A copy of the license is included in the section entitled “GNU Free Documentation License”. Version 1.GNU Free Documentation License relicensing. If you have Invariant Sections. If you have Invariant Sections without Cover Texts. If your document contains nontrivial examples of program code. Permission is granted to copy. no Front-Cover Texts. replace the “with . to permit their use in free software.” line with this: with the Invariant Sections being LIST THEIR TITLES. merge those two alternatives to suit the situation. with no Invariant Sections. Front-Cover Texts and Back-Cover Texts. 171 ADDENDUM: How to use this License for your documents To use this License in a document you have written. or some other combination of the three. . include a copy of the License in the document and put the following copyright and license notices just after the title page: Copyright c YEAR YOUR NAME. we recommend releasing these examples in parallel under your choice of free software license.3 or any later version published by the Free Software Foundation. . with the Front-Cover Texts being LIST. . Texts.

Master your semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master your semester with Scribd & The New York Times

Cancel anytime.