Numerical Methods For Civil Engineers (MATLAB)

Welcome to
Numerical Methods for Civil Engineers

ภาคการศึกษาที่ 3 ปการศึกษา 2548
Lecture 1 -
MATLAB
Introduction
Instructor: Mongkol JIRAVACHARADET
School of Civil Engineering
Institute of Engineering
Suranaree University of Technology
Textbook
Applied Numerical Methods
With MATLAB for Engineers
and Scientists
STEVEN C. CHAPRA
McGraw-Hill International Edition

References
Numerical Methods For Engineers with Personal Computer Applications,
(Third Edition) by Chapra, S.C. and R.P. Canale, McGraw-Hill, 1998
Numerical Methods with MATLAB : Implementations and Applications,

Gerald W. Recktenwald, Prentice-Hall, 2000
An Introduction to Numerical Methods : A MATLAB Approach,

Abdelwahab Kharab and Ronald B. Guenther, Chapman & Hall/CRC, 2002
Numerical Methods using MATLAB,

John H. Mathews and Kurtis D. Fink, Prentice-Hall, 2004
The Matlab 7 Handbook , Mathwork Inc.
KEEP THESE BOOKS! They are excellent career references

(at least for a while)
Topics Covered
• Introduction to Matlab • Interpolation
• Approximations and Errors • Numerical Integration
• Roots of Equations • Ordinary Differential Equations
• Linear Systems • Optimization
• Curve Fitting
Conduct of Course
Homework/Projects/Quizzes 30 %
Midterm Exam 30 %
Final Exam 40 %
Grading Policy
Final Score Grade
100 - 90 A
89 - 85 B+
84 - 80 B
79 - 75 C+
74 - 70 C
69 - 65 D+
64 - 60 D
59 - 0 F
WARNINGS !!!
1) Participation expected, check by quizzes
2) Study in groups but submit work on your own
3) No Copying of Matlab code
4) Submit Homework at the beginning of class
5) Late homework with penalty 30%
6) No make up quizzes or exams
MATLAB
The Language of Technical Computing
M ATLAB เปนโปรแกรมที่รวมการคํานวณทาง
คณิตศาสตร การแสดงผล และภาษาที่มีประสิทธิภาพ
เพื่อสรางสภาวะแวดลอมที่ยืดหยุนสําหรับการคํานวณ
ทางเทคนิค ดวยสถาปตยกรรมแบบเปด ทําใหเปนการงาย
ที่จะใช MATLAB ในการสํารวจขอมูล สรางอัลกอริธึม หรือ
สรางเครื่องมือคํานวณตามที่ผูใชตองการ
The latest version Matlab 7.3 R2006b
www.mathworks.com
MATLAB = MATrix LABoratory
- Math and computation
- Algorithm development
- Modeling, Simulation, and Prototyping
- Data analysis, exploration, and visualization
- Scientific and Engineering Graphics
- Many toolboxes for solving problems:
Control System Toolbox Statistics Toolbox
Signal Processing Toolbox Optimization Toolbox
System Identification Toolbox Partial Diff. Equation Toolbox
Neural Network Toolbox Symbolic Math Toolbox
MATLAB Educational Sites

http://www.mathworks.com/
http://www.eece.maine.edu/mm/matweb.html
http://www.math.utah.edu/lab/ms/matlab/matlab.html
http://www.math.mtu.edu/~msgocken/intro/intro.html
MATLAB 1
- Getting started
- Basic Arithmetic
- Built-in Functions
- Built-in Variables
- Vector & Matrix
The MATLAB System

The MATLAB system consists of five main parts:
Development Environment: set of tools and facilities that help you use MATLAB
functions and files. Many of these tools are graphical user interfaces. It includes the
MATLAB desktop and Command Window, a command history, an editor and
dedugger, and browsers for viewing help, the workspace, files, and the search path.
The MALAB Mathematical Function Library: a vast collection of computational

algorithms.
The MATLAB Language: This is a high-level matrix/array language with control

flow statements, functions, data structures, input/output, and objected-oriented
programming features.
Graphics: MATLAB has extensive facilities 2-D and 3-D data visualization,
animation, and presentation graphics.
The MATLAB Application Program Interface (API): allows you to write C and
Fortran programs that interact with MATLAB.
Getting Started
MATLAB Desktop
Getting Started
Command Window
Command prompt >> DEMO

Getting Started
Editor/Debugger
Basic Arithmetic
Calculator functions work as you'd expect:
>>(1+4)*3
ans =
15
+ and - are addition, / is division, * is multiplication, ^ is an exponent.
>> 5*5*5
>> 5^(-2.5)
>> 3*(23+14.7-4/6)/3.5
Last-line editing Up Arrow

>> 2 + 6 - 4 Upper & Lower Case
ans = >> a=2;
4 >> A=3;
The value in ans can be recalled: >> 2*a
>> ans/2 >> 2*A
ans = 2 Several commands in the same line:

>>x=2;y=6+x,x=y+7
Assign value to a variable:
>> a = 5 … For too long command:

>> Num_Apples = 10;
>> b = 6
>> Num_Orange = 25;
>> c = b/a
>> Num_Pears = 12;
>> Num_Fruit=Num_Apples+Num_Orange...
+ Num_Pears
Built-in Functions
>> sin(pi/4) >> pi sin(x), cos(x), tan(x),

ans = ans = sqrt(x), log(x), log10(x),
0.7071 3.1416 asin(x), acos(x), atan(x)
The output of each command can be suppress by using semicolon ;
>> x = 5;
>> y = sqrt(59);
>> z = log(y) + x^0.25
z =
3.5341
The commas allow more than one command
on a line:
>> a = 5; b = sin(a), c = sinh(a)

b =
-0.9589
c =
74.2099
MATLAB Variables is created whenever it appears

on the left-hand of “ = “
>> t = 5;
>> t = t + 2
t =
7
Any variable appearing on the right-hand side of

“ = “ must already be defined. Format
>> x = 2*z
>> pi
??? Undefined function or variable ‘z’
>> format long
>> who >> pi
>> whos >> format short
>> clear >> format bank
Use long variable names is better to remember >> format short e

and understandable for others >> format long e
>> format compact
>> radius = 5.2;
>> format loose
>> area = pi*radius^2;
>> format
Built-in Variables
Use by MATLAB, Should not be assigned to other values
Variable Meaning
ans value of an expression when not assigned to variable

eps floating-point precision
i, j unit imaginary numbers, i = j = −1
pi π = 3.14159265 . . .
realmax largest positive floating-point number
realmin smallest positive floating-point number
Inf ∞, a number larger than realmax, result of 1/0
NaN not a number (0/0)
>> x = 0; >> help log

>> 5/x On-line Help:
>> x/x >> lookfor cosine
Matrices and Magic Squares

In MATLAB, a matrix is a rectangular array of numbers.
scalars = 1-by-1 matrices
vectors = one row or column matrices
it is usually best to think of everything

as a matrix.
The matrices operations in MATLAB are

designed to be as natural as possible.
MATLAB allows you to work with entire

matrices quickly and easily.
Renaissance engraving Melencolia I by the German artist and amateur

mathematician Albrecht Dürer.
Entering Matrices
You can enter matrices into MATLAB in several different ways:

• Enter an explicit list of elements.
• Load matrices from external data files.
• Generate matrices using built-in functions.
• Create matrices with your own functions in M-files.
Start by entering matrix as a list of its elements.

You only have to follow a few basic conventions:
• Separate the elements of a row with blanks or commas.
• Use a semicolon, ; , to indicate the end of each row.
• Surround the entire list of elements with square brackets, [ ] .
To enter matrix, simply type in the Command Window

>> A = [16 3 2 13; 5 10 11 8; 9 6 7 12; 4 15 14 1]
Row and Column Vectors
Row vector >> A = [ 2 3 5 7 11]

Column vector >> A = [ 2; 3; 5; 7; 11]
Transposition >> At = A’
Arrays Operations
>> A = [ 3 5 7 9 11 ] >> A / B
>> A(3) >> A ./ B
>> length(A) >> A .^ 2
>> clear(A) >> odd = 1:2:11
>> B = [ 2 4 6 8 10 ] >> even = 2:2:12
>> A + B >> natural = 1:6
>> A - B >> angle = 0:pi/10:pi;
>> A * B >> sin(angle)
>> A .* B
Generate matrices using built-in functions
>> A = zeros(4) >> A = zeros(3,4)

>> A = ones(4)
>> A = eye(4)
>> A = magic(4)
Elementary Matrix Operations
>> S = A + B >> [m , n] = size(A)

>> D = A - B >> det(A)
>> A*B >> inv(A)
>> C = [ 10 11; 12 13; 14 15]; >> v = [1 2 3];
>> A*C >> A = diag(v)
>> A^2 >> B = diag([1 2 1 2])
>> L = log10(A) >> w = diag(B)
Subscripts
The element in row i and column j of A is denoted by A(i , j)
A = [16 3 2 13; 5 10 11 8; 9 6 7 12; 4 15 14 1]

MATLAB displays the matrix you just entered.
A =
16 3 2 13
5 10 11 8
9 6 7 12
4 15 14 1
For example, A(4,2) is the number in the fourth row and second column.
>> A(4,2)
ans =
15
Submatrix
>> A(1,:)
>> A(:,2)
>> A(3:4,1:2)
Juxtaposition
>> B = [9 8 2 5 ; 4 5 6 7 ; 2 1 3 4]
>> [A B]
>> size(ans)
>> [A ; B]
The Colon Operator
The colon, :, is one of the most important MATLAB operators.

>> 1:10
To obtain nonunit spacing, specify an increment.
For example,
and
>> 100:-7:50 >> 0:pi/4:pi
Linspace
linspace function creates row vectors with equally spaced elements.

>> u = linspace(0.0,0.25,5)
>> v = linspace(0,9,4)’
>> x = linspace(0,pi/6,6*pi);
>> s = sin(x); >> t = tan(x);
>> c = cos(x); >> [x’ s’ c’ t’]

Workspace Browser
The MATLAB workspace consists of the set of variables (named arrays)

built up during a MATLAB session and stored in memory.
You add variables to the workspace by using functions, running M-files, and
loading saved workspaces.
To view the workspace and information about each variable,
use the Workspace browser, or use the functions who and whos.
Example 2.1 Transportation route analysis
The following table gives data for the distance travel along five truck routes and the
corresponding time required to traveled each route. Use the data to compute the
average speed required to drive each route. Find the route that has the highest
average speed.
1 2 3 4 5
Distance (miles) 560 440 490 530 370
Time (hrs) 10.3 8.2 9.1 10.1 7.5
Solution:
>>d = [560, 440, 490, 530, 370]
>>t = [10.3, 8.2, 9.1, 10.1, 7.5]
>>speed = d./t
speed =
54.3689 53.6585 53.8462 52.4752 49.3333
>>[highest_speed, route] = max(speed)
highest_speed = route =
54.3689 1
Lecture 2 - MATLAB 2
- Save & Load

- Input & Output
- Plotting Graph
Mongkol JIRAVACHARADET
SURANAREE INSTITUTE OF ENGINEERING

UNIVERSITY OF TECHNOLOGY SCHOOL OF CIVIL ENGINEERING
Save/Load Data to/from External Files

The save and load Commands
>> clear
>> x = 0:5; y=5*x;
>> save xyfile
>> save(‘xyfile’,’x’,’y’)
>> XY = [x’ y’];
>> save xyvals.txt XY -ascii
Loading Matrices from mat Files

>> clear
>> x = linspace(0,2*pi); y = cos(x); z = sin(x);
>> save trigvar
>> clear
>> whos
>> load trigvar
>> whos
Loading Data from Plain Text Files

>> clear
>> whos
>> XY = load(‘xyvals.txt’)
>> x = XY(:,1)
>> y = XY(:,2)
Input/Output Commands
Input Output - Screen

- File
User MATLAB
Command Description
disp (A) Displays the contents, but not the name, of the
Array A.
disp (‘text’) Displays the text string enclosed within single quotes.
fprintf Control the screen’s output display format.
x = input(‘text’) Displays the text in quotes, waits for user input from
the keyboard, and stores the value in x.
x = input(‘text’,’s’) Displays the text in quotes, waits for user input from
the keyboard, and stores the input as a string in x.
INPUT AND OUTPUT
Prompting for User Input
>> x = input(‘Enter a value for x’);
By default, the input function returns a numerical value.
To obtain a string input, a second parameter, ‘s’ , must be provided.
>> yourname = input(‘Enter your name ‘,‘s’);
function s = inputAbuse
% inputAbuse Use input messages to compute sum of 3 variables
x = input(‘Enter the first variable to be added ‘);

y = input(‘Enter the second variable to be added ‘);
z = input(‘Enter the third variable to be added ‘);
s = x+y+z;
Text Output
The disp Function

>> disp(‘My favorite color is red’)
>> Speed = 63;

>> disp(‘The vehicle’s predicted speed is:’)
>> disp(Speed)
Since disp requires only one argument, the message and variable
must be combined into a single string.
>> yourName = input(‘enter your name ‘,’s’);

>> disp([‘Your name is ‘,yourName])
Display Formats with the fprintf Command
Syntax Description
fprintf(‘format’,A,...) Displays the elements of the array A, and any
addition array arguments, according to the format
specified in the string ‘format’
‘format’ structure %[-][number1.number2]C, where
number1 specifies the minimum field width,
number2 specifies the number of digits to the
right of the decimal point, and C contains control
codes and format codes. Items in brackets are
optional. [-] specifies left justified.
fprintf(format)
fprintf(format,variables)
fprintf(fid,format,variables)
>> fprintf(‘Warning: x is negative\n’)
fprintf (Continue…)
Format Codes Description
%e Scientific format with lowercase e.

%E Scientific format with uppercase E.
%f Decimal format.
%g %e or %f, whichever is shorter.
Control Codes Description
\n Start new line.

\r Beginning of new line.
\b Backspace.
\t Tab.
“ Apostrophe.
\\ Backslash.
fprintf(format,variables)
>> name = ‘Elvis’; age = str2num(datestr(now,10)-1935;
>> fprintf(‘%s is %d years old\n’,name,age);
Elvis is 65 years old
>> fprintf(‘The speed is: %3.1f\n’,Speed)

The speed is: 63.2
>>r = [2.25:20:42.25];
2.25 14.137
>>circum = 2*pi*r;
22.25 139.8
>>y = [r;circum];
42.25 265.46
>>fprintf(‘%5.2f %11.5g\n’,y)
Low-Level Input/Output Functions

fopen open file Permission:
fid = fopen(filename, permission); ‘r’ read
‘rt’ read plain text file
...
‘w’ write
fclose(fid)
‘wt’ write plain text file
‘a’ append
fscanf read data from file ‘r+’ read & write
‘w+’ truncate or create
x = fscanf(fid, format); for read & write
x = fscanf(fid, format, size); ‘a+’ read & append
‘W’ write without
automatic flushing
fscanf print data to file ‘A’ append without
automatic flushing
fprintf(fid, format, variables);
fprintf(fid,format,variables)
>> x = ...
>> fout = fopen(‘myfile.dat’,’wt’);
>> fprintf(fout,’ k x(k)\n’);
>> for k = 1:length(x)
>> fprintf(fout,’%4d %5.2f\n’,k,x(k));
>> end
>> fclose(fout)
Creating a Plot
The plot function has different forms, depending on the input arguments.
If y is a vector, plot(y) produces a piecewise linear graph of the elements of y
versus the index of the elements of y.
If you specify two vectors as arguments, plot(x,y) produces a graph of y versus
x.
For example, these statements use the colon operator to create a vector of x values
ranging from zero to 2π, compute the sine of these values, and plot the result.
>> x = 0:pi/100:2*pi;
>> y = sin(x);
>> plot(x,y)
Now label the axes and add a title. The characters \pi create the symbol π.
>> xlabel('x = 0:2\pi')

>> ylabel('Sine of x')
>> title('Plot of the Sine
>> Function','FontSize',12)
Multiple Data Sets in One Graph
Multiple x-y pair arguments create multiple graphs with a single call to plot.
MATLAB automatically cycles through a predefined (but user settable) list of colors to
allow discrimination among sets of data.
For example, these statements plot three related functions of x, each curve in a
separate distinguishing color.
>> y2 = sin(x-.25);
>> y3 = sin(x-.5);
>> plot(x,y,x,y2,x,y3)
The legend command provides an easy

way to identify the individual plots.
>> legend('sin(x)','sin(x-.25)','sin(x-.5)')
Specifying Line Styles and Colors
It is possible to specify color, line styles, and markers (such as plus signs or
circles) when you plot your data using the plot command.
>> plot(x,y,'color_style_marker')
color_style_marker is a string containing from one to four characters
(enclosed in single quotation marks) constructed from a color, a line style, and
a marker type:
• Color strings are 'c', 'm', 'y', 'r', 'g', 'b', 'w', and 'k'.
These correspond to cyan, magenta, yellow, red, green, blue, white, and black.
• Linestyle strings are '-' for solid, '--' for dashed, ':' for dotted, '-.' for dash-dot.
Omit the linestyle for no line.
• Marker types are '+', 'o', '*', and 'x' and the filled marker types are :
's' for square, 'd' for diamond, '^' for up triangle, 'v' for down triangle,
'>' for right triangle, '<' for left triangle, 'p' for pentagram, 'h' for hexagram,
and none for no marker.
>> x1 = 0:pi/100:2*pi;
>> x2 = 0:pi/10:2*pi;
>> plot(x1,sin(x1),'r:',x2,sin(x2),'r+')
Adding Plots to an Existing Graph
The hold command enables you to add plots to an existing graph. When you type
hold on
MATLAB does not replace the existing graph when you issue another plotting
command; it adds the new data to the current graph, rescaling the axes if
necessary.
For example, these statements first create a contour plot of the peaks function, then
superimpose a pseudocolor plot of the same function.
[x,y,z] = peaks;
contour(x,y,z,20,'k')
hold on
pcolor(x,y,z)
shading interp
hold off
The hold on command causes the pcolor
plot to be combined with the contour
plot in one figure.
Lecture 3 - MATLAB 3
Programming with MATLAB :
- Script m-files
- Function m-files
- Flow Control

Script Files
You can perform operations in MATLAB in two ways:
1. Enter commands directly in the Command Window.
2. Run a script M-file which contains all the commands – one at a time.
M-file is just a plain text file ended with extension “.m”
To open the built-in editor, select menu: File > New > M-file
A Script to Plot Functions
trigplot.m
t = linspace(0,2*pi);
y1 = sin(t);
y2 = cos(t);
y3 = y1.*y2;
plot(t,y1,’-’,t,y2,’.’,t,y3,’—’);
>> trigplot
Replace the plot statement with
plot(t,y1,’-’,t,y2,’:’,t,y3,’—’);
axis([0 2*pi -1.5 1.5])
legend(‘sin(t)’,’cos(t)’,’sin(t)*cos(t)’);
>> close all; trigplot

Use TEX notation to display “θ”
legend(‘sin(\theta)’,’cos(\theta)’,
’sin(\theta)*cos(\theta)’);
xlabel(‘\theta (radius)’,’FontName’,’Times’,
’FontSize’,14)
>> close all; trigplot
>> x = linspace(0,2*pi);
>> y1 = sin(x);
>> y2 = cos(x);
>> y3 = y1.*y2;
>> plot(x,y1,'-',x,y2,':',x,y3,'--');
>> axis([0 2*pi -1.5 1.5])
>>
legend('sin(\theta)','cos(\theta)','sin(\theta)*cos(\theta)')
>> xlabel('\theta (radius)','FontName','Times','FontSize',14)
>> title('Plot of simple trigonometric functions',...
'FontName','Times','FontSize',12)
Function m-Files
input parameters
Command Window
>>
function
output parameters
function [outputParameterList] = functionName(inputPramaterList)

output argument
function name
Input argument
pyt.m
function h = pyt(a, b)
% PYT hypotenuse of a right-angled triangle
h = sqrt(a.^2 + b.^2);
>> pyt(3,4)
average.m is a simple function that calculates the average of the

elements in a vector.
function y = average(x)
% AVERAGE Mean of vector elements.
% AVERAGE(X), where X is a vector, is the mean of vector elements.
y = sum(x)/length(x); % Actual computation
>> z = 1:99;
>> average(z)
ans =
50
The H1 Line is the first help text line.
>> lookfor average
AVERAGE Mean of vector elements.
Help Text : >> help average

AVERAGE Mean of vector elements.
AVERAGE(X), where X is a vector, is the mean of vector elements.
Creating P-Code Files
You can convert average.m into a pseudocode called P-code file.
>> pcode average
Text file: Pseudocode:

can be viewed can’t be viewed
by any editor by any editor
- Faster for large program
- Use to hide algorithm
FLOW CONTROL
MATLAB has several flow control constructs:
• if
• switch & case
• for
• while
• continue
• break
if Conditional Control
The if statement evaluates a logical expression and executes a group of statements
when the expression is true.
The optional elseif and else keywords provide for the execution of alternate groups
of statements.
An end keyword, which matches the if, terminates the last group of statements.
Condition:
if condition
Equal A == B
expression
Not equal A ~= B
elseif condition
Greater A>B
expression
Smaller A<B
else Greater or equal A >= B
expression Smaller or equal A <= B
end AND &
OR |
Example 1:
if a < 0
disp(‘a is negative’);
end
Example 2:
if a < 0, disp(‘a is negative’); end
Example 3:
if x >= y
c = x^2 - y;
elseif y/x > 2.0
c = log(y/x);
else
c = x + y;
end
switch & case
The switch statement executes groups of statements based on the value of a variable or
expression.
The keywords case and otherwise delineate the groups.
Only the first matching case is executed. There must always be an end to match the switch.
switch expression
case value1 switch sign(x)
block of statements case -1
disp(‘x is negative’);
case value2
case 0
block of statements disp(‘x is exactly zero’);
. case 1
. disp(‘x is positive’);
. otherwise
disp(‘sign test fail’);
otherwise
end
block of statements
end
for
The for loop repeats a group of statements a fixed,
predetermined number of times.
A matching end delineates the statements.
>> P = zeros(5, 5);
>> for k = 1:5

for l = 1:5
P(k, l) = pyt(k, l);
end
end
>> P
while
The while loop repeats a group of statements
an indefinite number of times under control of a logical condition.
A matching end delineates the statements.
>> x=1;
while condition >> while 1+x > 1
expressions x = x/2;
end
end >> x
LOOP
Break Loops break
return
break
Return Loops return
Here is a complete program, illustrating while, if, else, and end,

that uses interval bisection to find a zero of a polynomial.
a = 0; fa = -Inf;
b = 3; fb = Inf; fb
while b-a > eps*b
x = (a+b)/2;
fx = x^3-2*x-5;
if sign(fx) == sign(fa)
a = x; fa = fx; x
else a b
b = x; fb = fx;
end
end fa
x
The result is a root of the polynomial x3 - 2x - 5, namely

x =
2.09455148154233
Figure Windows
Graphing functions automatically open a new figure window if there are no

figure windows already on the screen. If a figure window exists, MATLAB uses
that window for graphics output. If there are multiple figure windows open,
MATLAB targets the one that is designated the “current figure” (the last figure used
or clicked in).
To make an existing figure window the current figure, you can click the mouse while
the pointer is in that window or you can type
figure(n)
where n is the number in the figure title bar. The results of subsequent graphics
commands are displayed in this window.
To open a new figure window and make it the current figure, type
figure
Multiple Plots in One Figure

The subplot command enables you to display multiple plots in the same window or
print them on the same piece of paper. Typing
subplot(m,n,p)
partitions the figure window into an m-by-n matrix of small subplots and selects
the pth subplot for the current plot.
The plots are numbered along first the top row of the figure window, then the second
row, and so on.
For example, these statements plot data in four different subregions of the figure
window.
>> t = 0:pi/10:2*pi;
>> [X,Y,Z] = cylinder(4*cos(t));
>> subplot(2,2,1); mesh(X)
>> subplot(2,2,2); mesh(Y)
>> subplot(2,2,3); mesh(Z)
>> subplot(2,2,4); mesh(X,Y,Z)
3-D Plot
First, create 1D vectors describing the grids in the x- and y-directions:
>> x = (0:2*pi/20:2*pi)';
>> y = (0:4*pi/40:4*pi)';
Next, ``spread'' these grids into two dimensions using meshgrid:
>> [X,Y] = meshgrid(x,y);
>> whos
Evaluate a function z = f(x,y) of two variables on the rectangular grid:
>> z = cos(X).*cos(2*Y);
Plotting commands:
>> mesh(x,y,z)
>> surf(x,y,z)
>> contour(x,y,z)
Lecture 4
Approximations and Errors
- Significant Figures - Truncation Errors
- Accuracy & Precision - Taylor Series
- Error Definitions - Order Notation
- Round-off Errors

Composite surfaces and

terrain modelling
Mixed approximation methods
Significant Figures 49 km/h 2 digits
Car speed = ? 48.8 km/h 3 digits
48.9 km/h 3 digits
48.86 km/h 4 digits
ZERO is NO significant: 184500, 0.1845, 0.01845, 0.00001845 are 4 digits
Use of Significant Figures in Numerical Methods
1) Numerical method yield approximation results. So we have

to specify how confident in our approximated results.
2) Specific quantities such as π , e , or 7 cannot be expressed

exactly by a limited number of digits.
π = 3.141592653589793238462643 . . .
But computers can contain only . . .
>> format short ; pi = 3.1416 (5 digits)

>> format long ; pi = 3.14159265358979 (15 digits)
Try also exp(1), sqrt(7)
* Omission of the remaining significant figures is called round-off error.

Accuracy and Precision
Increasing accuracy
Increasing precision
UNCERTAINTY
ACCURACY
BIAS
Computes value
close to true value
P R E C I S I O N Computed value close to each other
Error Definitions
True value = Approximation + Error
Numerical Error Et = true value - approximation
true value - approximation

Relative Error εt = 100%
true value
Approximation Error
Iterative approach for better and better approximations
current approximation - previous approximation

εa = 100%
current approximation
EXAMPLE 4.1 Error Estimates for Iterative Methods
x2 x3 xn
Approximate e0.5 by using the series e = 1+ x +
x
+ +L +
2! 3! n!
true value: >> format long ; exp(0.5)

>> ans =
1.648721 . . .
1st estimate: ex = 1
2nd estimate: ex = 1 + x e0.5 = 1 + 0.5 = 1.5
1.648721 − 1.5
true error: εt = × 100% = 9.02%
1.648721
1.5 − 1
approximate error: εa = × 100% = 33.3%
1.5
Terms Results εt (%) εa (%)
1 1 39.3
2 1.5 9.02 33.3
3 1.625 1.44 7.69
4 1.645833333 0.175 1.27
5 1.648437500 0.0172 0.158
6 1.648697917 0.00142 0.0158
After six terms are included, the approximate error falls below εa = 0.05%,
and the computation is terminated.
>> Term = 1:6
>> et = [39.3 9.02 1.44 0.175

0.0172 0.00142]
>> ea = [33.3 7.69 1.27 0.158

0.0158]
>> plot(Term,et,'o-‘,
Term(2:6),ea,'+-')
ROUND-OFF ERRORS
Precision : Computers retain only a fixed number of significant figures
All computations in MATLAB are done in double precision
FORMAT LONG E Floating point format with 15 digits. ROUND-OFF
True value π = 3.141592653589793238462643 . . .

MATLAB pi = 3.141592653589793e+000
Creation of erroneous digit

>> format long e % display all of the significant digits
>> 2.6 + 0.2
>> ans + 0.2
>> ans + 0.2
Computer Representation of Numbers
Computers manipulate binary numbers
Base 10 Conversion Base 2
1 1 = 20 0000 0001
2 2 = 21 0000 0010
4 4 = 22 0000 0100
8 8 = 23 0000 1000
9 8+1 = 23+20 0000 1001
10 8+2 = 23+21 0000 1010
27 16 + 8 + 2 + 1 = 24+23+23+21 0001 1011
>> dec2bin(27)
>> type dec2bin
Bits & Bytes
BIT = Binary digit (0 or 1)
=0 =1
BYTE = Group of 8 bits
= 0001 1011
= 27
MAX BYTE NUMBER = 1111 1111 = 27+26+25+24+23+22+21+20 = 255
Integer Representation
Computer uses 2 bytes = 16 bits & 1st bit for sign
1 0 0 0 0 0 1 0 1 1 0 1 0 0 1 1
Sign Number
Upper limit = 214 + 213 + . . . + 22 + 21 + 20

= 215 - 1 = 32,767
Zero = 0000 0000 0000 0000
- Zero = 1000 0000 0000 0000
Redundant
Integer range: [ -32768 to 32767 ]
Floating-point Numbers
Stored in binary equivalent of scientific notation

123.456 = 123.456 × 100 = 1.23456 × 102 = 123456 × 10-3
37.5 = 0.3750 × 102
-812.5 = - 0.8125 × 103
Signed exponent
0.005781 = 0.5781 × 10-2
Sign Mantissa
How computer stored floating-point number

Signed exponent Mantissa
Sign
MATLAB uses the IEEE floating-point standard
Single Precision 32-bit

Sign Signed exponent Mantissa
1 bit 8 bit 23 bit
Range: +1.18 × 10-38 to +3.40 × 1038
−3.40 × 1038 to −1.18 × 10-38
Double Precision 64-bit

Sign Signed exponent Mantissa
1 bit 11 bit 53 bit
Range: +2.23 × 10-308 to +1.80 × 10308
−1.80 × 10308 to −2.23 × 10-308
MATLAB Discrete Approximation
denormal
under under
overflow usable range usable range overflow
flow flow
-10+308 -10-308 0 10-308 10308

- realmax - realmin realmin realmax
>> format long e

>> 10*realmax
>> realmin/10 denormal (fewer significant digits)
>> realmin/1e16
Effect of Order of Operations
0.99 + 0.0044 + 0.0042 = 0.9986
3-Digits: (0.99 + 0.0044) + 0.0042 = 0.998
0.99 + (0.0044 + 0.0042) = 0.999
x approximate x* to t significant digits when

x − x*
< 5 ×10− t
x
0.998 − 0.9986
Example: = 6.012 × 10−4 < 5 × 10 −3
0.998
= 3 significant digits
TRUNCATION ERRORS
result from using approximation in place of an exact mathematical
procedure
Example: Derivative of velocity
dv ∆v v(ti +1 ) − v(ti )
≅ =
dt ∆t ti +1 − ti
dv
Velocity
Continuous
dt Discrete
∆v
∆t
Time
ti ti+1
Bungee Jumper Problem
rate of change of velocity with respect to time,
dv c
= g − d v2
dt m
where v = vertical velocity (m/s), t = time (s),
g = gravity acceleration ( ≅ 9.81 m/s2)
cd = drag coefficient (kg/m)
m = jumper’s mass (kg)
Analytical solution by solving differential equation,
gm  gcd  ex − e− x
v(t ) = tanh t  tanh( x) = x − x
cd  m  e +e
Example: Compute velocity of a free fall bungee jumper with a mass of 70 kg.
Use a drag coefficient of 0.25 kg/m.
9.81(70)  9.81(0.25) 
v (t ) = tanh  t  = 52.41 tanh(0.1872t )
0.25  70 
>> t=0:2:12;
>> v=52.41*tanh(0.1872*t);
>> [t’ v’] Analytical Solution for the bungee jumper problem
60
Terminal velocity
ans =
0 0 50
2.0000 18.7541
4.0000 33.2506 Velocity, m/s 40
6.0000 42.3829 30
8.0000 47.4160
10.0000 49.9874 20
12.0000 51.2501
10
>> plot(t,v) 0
0 2 4 6 8 10 12
Time, s
Numerical Solution to the differential equation
Rate of change of velocity can be

approximated by v(ti+1)
True slope
dv ∆v v (t i +1 ) − v (t i )
≅ = ∆v dv/dt
dt ∆t t i +1 − t i Approx. slope
∆v v (t i +1 ) − v (ti )
=
Substitute into dv/dt = g – (cd/m)v2 to give v(ti) ∆t t i +1 − t i
v (t i +1 ) − v (t i ) c
= g − d v (t i )2
t i +1 − t i m
ti ti+1 t
Rearrange equation to yield
∆t
 c 
v (t i +1 ) = v ( t i ) + g − d v (t i )2 (t i +1 − t i )
 m 
Euler’
Euler’s method:
dv i
v ( t i +1 ) = v (t i ) + ∆t New value = old value x step size
dt
Example 2: Numerical Solution to the Bungee Jumper Problem
Perform the same computation as previous example but use
the Euler’s method. Employ a step size ∆t = 2 sec.
 c 
v (t i +1 ) = v ( t i ) + g − d v (t i )2 (t i +1 − t i )
 m 
@ start ti = 0 s, ti+1 = 2 s, v(0) = 0 m/s:
 0.25 2 
v ( 2) = 0 + 9.81 − (0)  × 2 = 19.62 m/s
 70 
Next step ti = 2 s, ti+1 = 4 s, v(2) = 19.62 m/s:
 0.25  t(s) v(m/s)

v ( 4) = 19.62 + 9.81 − (19.62) 2  × 2 = 36.49 m/s 0 0
 70 
2 19.62
4 36.49
MATLAB: 6 46.60
>> 0 [Enter] 8 50.71
>> ans+(9.81-(0.25/70)*ans^2)*2 [Enter] 10 51.96
[Up Arrow] [Enter] many time . . . 12 52.30
>> hold on
>> v2 = [0 19.62 36.49 46.6 50.71 51.96 52.3]
>> plot(t,v2,'+-')
Analytical Solution for the bungee jumper problem

60
Terminal velocity
50
Approx. solution
Velocity, m/s
40
Analytical solution
30
20
10
0
0 2 4 6 8 10 12
Time, s
TAYLOR SERIES
Predict function value of one point in term of function value
and its derivatives at other points
f(x)
f(xi) Zero-order approx.
f(xi+1) ≅ f(xi)
First o
rder
Known Se f(xi+1) ≅ f(xi) + f’(xi)h
co
nd
Tr or Slope @ xi
ue de
r
f(xi+1) ≅ f(xi) + f’(xi)h + f’’(xi)h2/2!
f(xi+1): want to know
xi xi+1 x
h
N th-order Approximations
f ′′( x i ) 2 f ′′′( x i ) 3 f ( n ) ( xi ) n
f ( x i +1 ) ≅ f ( xi ) + f ′( x i )h + h + h +L+ h
2! 3! n!
Taylor series expansion:
f ′′( x i ) 2 f ′′′( x i ) 3 f ( n ) ( xi ) n
f ( x i +1 ) = f ( xi ) + f ′( x i )h + h + h +L+ h + Rn
2! 3! n!
Note that, an equal sign replaces the approximate sign.

A reminder term is included to account for all terms from n+1 to infinity:
f ( n +1) (ξ ) n +1
Remainder: Rn = h where xi < ξ < xi+1
(n + 1)!
Truncation error of order n + 1
f ( n +1) (ξ ) n +1
Rn = h = O(h n +1 ) = Truncation error of order hn+1
(n + 1)!
Where ξ is not known but lies somewhere between xi and xi+1
xi ξ xi+1
f(x) =Pn(x) + O(hn+1)
x2 xn
Example: e = 1+ x + +L + +L
x
2! n!
2
x
e x = 1 + x + + O( x3 ) → error of order x3
2!
x2 xn
e = 1 + x + + L + + O( x n +1 ) → error of order x n +1
x
2! n!
Taylor series of a function
Exponential: f ( x) = e x
f ′′( xi ) 2 f ( n ) ( xi ) n
from f ( xi +1 ) = f ( xi ) + f ′( xi ) h + h + ... + h
2! n!
Set: xi+1 = x and xi = 0 so h = xi+1 - xi = x
f ( x) = e x , f ′( x) = e x , ..., f ( n) ( x) = e x
So
f (0) = e0 = 1, f ′(0) = 1, ..., f ( n) ( x) = 1
x 2 x3 xn
f ( x) = e = 1 + x + + + ... +
x
2! 3! n!
Taylor Series Expansions for Some Common Functions
x3 x5 x 2 n −1
sin( x) = x − + − L − (−1) n
for all x
3! 5! (2n − 1)!
x2 x4 n x
2n
cos( x) = 1 − + − L + (−1) for all x
2! 4! (2n)!
x 2 x3 xn
e = 1+ x + + + L +
x
for all x
2! 3! n!
x 2 x3 x 4 n x
n
ln(1 + x) = x − + − − L − (−1) -1 ≤ x ≤ 1
2! 3! 4! n!
x3 x5 x2n
arctan( x) = x − + − L − (−1) n
-1 ≤ x ≤ 1
3! 5! (2n − 1)!
p ( p − 1) 2 p ( p − 1)( p − 2) 3
(1 + x) p = 1 + px + x + x + L for x < 1
2! 3!
Example: Taylor series approximation of function sin(x)

x3 x5
Plot the first n = 1 to 6 terms of the Taylor series of sin( x ) = x − + −L
6 120
1.5
sin(x)
n=1
1 n=2
n=3
0.5
-0.5
-1
-1.5
-6 -4 -2 0 2 4 6
>> x = -2*pi:pi/30:2*pi;
>> y = sin(x);plot(x,y)
>> axis([-2*pi 2*pi -1.5 1.5])
>> hold on
>> y1 = x; plot(x,y1)
>> y2 = x-x.^3/6; plot(x,y2)
>> y3 = x-x.^3/6+x.^5/120; plot(x,y3)
>> legend('sin(x)','n = 1','n = 2','n = 3')
Example: Approximation of function cos(x) by Taylor Series Expansion
Use Taylor series expansions with n = 0 to 6 to approximate f(x) = cos x at xi+1 = π /3

on the basis of the value of f(x) and its derivatives at xi = π /4.
Solution:
Step size: h = π /3 - π /4 = π /12 True value: cos(π /3) = 0.5
Zero-order approximation: f(π /3) ≅ cos(π /4) = 0.707106781
which represents a percent relative error of
0.5 − 0.707106781
εt = × 100% = 41.4%
0.5
First-order approximation: f’(x) = -sin(x)
f(π /3) ≅ cos(π /4) – sin(π /4)(π /12) = 0.521986659
which has εt = 4.40%
Second-order approximation: f’’(x) = -cos(x)

2
π  π   π  π  cos(π / 4)  π 
f   ≅ cos  − sin   −   = 0.497754491
3 4  4  12  2  12 
which has εt = 0.449%
The process can be continued and the results listed as in
Order n f(π/3) εt(%)
0 0.707106781 41.4
1 0.521986659 4.40
2 0.497754491 0.449
3 0.499869147 2.62 x 10-2
4 0.500007551 1.51 x 10-3
5 0.500000304 6.08 x 10-5
6 0.499999988 2.44 x 10-6
Example: Evaluating the Series for sin(x)
x3 x5 x 2 n −1
sin( x) = x − + − L − (−1) n
3! 5! (2n − 1)!
function ssum = sinser(x)

% sinser Evaluate the series of sin function
tol = 5e-9; % Tolerance on accumulated sum
nmax = 15; % Maximum number of terms
term = x; ssum = term; % Initialize series
fprintf(‘Series approximation to sin(%f)\n’,x);
fprintf(‘ k term ssum\n’);
fprintf(‘%3d %11.3e %12.8f\n’,1,term,ssum);
− x2
Recursion: Tk = Tk − 2 , k = {1, 3, 5,...}
k (k − 1)
for k=3:2:(2*nmax-1)
term = -term*x*x/(k*(k-1));
ssum = ssum + term;
fprintf(‘%3d %11.3e %12.8f\n’,k,term,ssum);
if abs(term/ssum)<tol, break; end
end
fprintf(‘\nTruncation error after %d terms is %g\n\n’,
(k+1)/2,abs(ssum-sin(x)));
>> sinser(pi/6);
Lecture 5
Roots of Equations
- Graphical Methods
- Fixed-
Fixed-Point Iteration
- Bisection Method
- Newton’s Method
- Secant Method
fx = - Roots of Polynomials

What Are Roots?

Quadratic formula: f(x) = ax2 + bx + c = 0
f(x)
− b ± b 2 − 4ac
x=
2a
x
0 x1 x2
Roots = Values of x that make f(x) = 0

GRAPHICAL METHODS
Plot the function and observe where it crosses the x axis.
Example: Throw a Turtle out of a Plane !
Drag = cv
Consider the forces acting on a turtle thrown out from a plane.
According to Newton’s law :
Net force acting on dv

= m 1
the turtle dt
where m = is the mass of the turtle (kg),
v = velocity of the turtle (m/s), Gravity = mg
t = time (s)
For a turtle in free fall, Net force = mg – cv

dv
m = mg − cv 2
where c is some coefficient of friction. dt
Let’s now find the solution. Divide Eq. (2) by m gives:
gm  − 
ct
dv c
=g− v v (t ) = 1 − e m 
dt m c  

if v = 0 at t = 0
gm/c
v(t)
t
Problem: From the velocity equation of the turtle, find the value of the drag
coefficient c such that a turtle of mass m = 5 can attain a prescribed velocity,
v = 10, at a set period of time, t = 9. Use g = 10.
In the context of the problem mentioned above, m, v, and t are constant

(parameters) of the problem.
The only variable that we can adjust is c.
To solve this problem, rewrite the velocity equation as
gm  −  50  
ct 9c
−
f (c ) =  1 − e m  − v (t ) f (c ) = 1 − e 5  − 10
c  
 c  

So our task of solving the problem reduces to merely finding the value of c,
such that f( c ) =0, for the given values of v, m and t.
Graphical method
วิธีที่งายที่สุดคือการพล็อตกราฟ f(x) ซึ่งรากของสมการ f(x) = 0 ก็คือจุดที่กราฟ

ตัดแกน x นั่นเอง สังเกตดวยตาเปลา
f(x)
x
Graphical method : NOT precise
ZOOM
ZOOM
Double root?
Bracketing Method
Coarse level search for roots over large interval

1) Subdividing large interval into smaller subinterval
2) Examine sign at the end of each subinterval
f (x) + + + + - - - + +
+++++++++ ++++
----------
x1 x2 x
Possible ways in an interval
Part (a) and (c) : f(xl) and f(xu) same sign
No roots or Even number of roots
Part (b) and (d) : f(xl) and f(xu) different signs
One roots or Odd number of roots
Exceptions:
(a) Multiple roots
(b) Discontinuous function
Bracketing Algorithm
Given: f (x), xmin, xmax, n
dx = (xmax-xmin)/n % Size of bracket interval
xa = xmin % Initialize left side of test bracket
i = 0 % Initialize counter
while i < n
i = i + 1
xb = xa + dx
if f (xa) and f (xb) have different sign
save [ xa , xb ] for further root finding
end
xa = xb
end
Detecting Sign Change : f (xa) & f (xb)
1st Solution: f (xa) × f (xb) < 0 ?
>> format long e
>> fa = 1e-120; fb = -2e-300;
>> fa * fb < realmin
ans = 0
2nd Solution: Use sign function
>> fa = 1e-120; fb = -2e-300;

>> sign(fa) ~= sign(fb) return true value = 1
ans = 1
Example: Graphical Methods
Plot the function and observe where it crosses x axis

f(x)
f ( x) =
667.38
x
(1 − e −0.147 x ) − 40 40
x f(x)
4 34.115 20
8 17.653
12 6.067
0.00 16
16 -2.269 0
4 8 12 20 x
20 -8.401
-10
Observe: x ≈ 15 → f (15) = -0.4133

Interpolation (False-position method)
Assume as linear function for short interval
6.607
12 16 2.269
6.607
Interpolation: x = 12 + (16 − 12) = 14.977 f (14.977) = -0.3691
6.607 − (−2.269)
f (15) = -0. 4133, f (14) = 1.582
1.582
2nd Interpolation: x = 14 + (15 − 14) = 14.793 f (14.793) = -0.013
1.582 − (−0.4133)
Root Finding Methods

Two major classes of methods:
Bracketing methods: start with guesses that bracket, or contain, the root
and then systematically reduce the width of the bracket
§ Graphical method
f(x)
f(x) = 0
§ False-position method
x=?
§ Bisection method
Open Methods: require only a single starting value of x without bracketing

the root.
§ Fixed-point iteration
§ Newton-Raphson method
§ Secant method
Fixed-Point Iteration
1) Rewrite original equation f(x) = 0 into another form x = g(x).
x2 + 3
Example: x − 2x + 3 = 0
2
⇒ x =
2
sin x = 0 ⇒ x = sin x + x
2) Select initial value x0
3) Predict new xi+1 as a function of old xi xi+1 = g (xi)
4) Use iteration xi+1 = g(xi) to find a value that reaches convergence.
Iterative until satisfy
xi +1 − xi
εa = ×100%
xi +1
Example: Finding root of f (x) = x - e-x = 0 → xi +1 = e − xi

1
i xi εa (%)
True value = 0.5671
0 0 0.8
1 1 100
Approximated Root
2 0.3679 171.8
0.6
3 0.6922 46.9
4 0.5005 38.3
5 0.6062 17.4 0.4
6 0.5454 11.2
7 0.5796 5.90 0.2
8 0.5601 3.48
9 0.5711 1.93 0
10 0.5649 1.11 0 1 2 3 4 5 6 7 8 9 10 11 12
Number of Iteration
>> 0
>> exp(-ans)
Two-Curve Graphical Method
1 1
0.8 f ( x) = e − x
0.6 f ( x) = e− x − x 0.8
0.4
f(x) = exp(-x) - x
0.2 Root 0.6

0
-0.2 0.4 f ( x) = x
-0.4
-0.6 0.2
-0.8
-1 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
x x
>> x = 0 : 0.1 : 1; >> x = 0 : 0.1 : 1;

>> plot(x, exp(-x)-x) >> plot(x, x, x, exp(-x))
Convergence
1
y2 = g(x) y1 = x
0.8
0.6922
0.6062
0.6
y 0.5005
0.4
0.3679
0.2
0
0 0.2 0.4 0.6 0.8 1
x
Divergence
1
0.8
y1 = x
y2 = g(x)
0.6
y
0.4
0.2
x0
0
0 0.2 0.4 0.6 0.8 1
x
Bisection Method
Repeatedly halve interval while bracketing root
f (x) +
+
xb
xa xm x
-
-
1) Choose interval [ xa , xb ] which has sign-change
2) Compute midpoint of interval xm = ( xa + xb ) / 2
3) Select subinterval which has sign-change and repeat 2)
Example: Use bisection method find the root of equation
(1 − e −0.147 x ) − 40
667.38 True value of the root:
f ( x) = 14.7802
x
Estimate root at midpoint: f(x)
40
12 + 16
xr = = 14
2
14.7802 − 14 Initial interval
εt = × 100% = 5.279% 20
14.7802 [ 12 , 16 ]
Select interval with sign-change: 16

+ - 0
4 8 12 20 x
f (14) = 1.582 → [ 14 , 16 ]
-10
15 − 14
Next estimation: xr = (14+16)/2 = 15 εa = ×100% = 6.667%
14
Termination Criteria and Error Estimates

Stop computation when εa < εs , Ex. εs = 0.5%
Iteration xa xb xr εa(%) εt(%)

1 12 16 14 5.279
2 14 16 15 6.667 1.487
3 14 15 14.5 3.448 1.896
4 14.5 15 14.75 1.695 0.204
5 14.75 15 14.875 0.840 0.641
6 14.75 14.875 14.8125 0.422 0.219
< εs = 0.5%
Bisection Algorithm
initialize: a = . . . , b = . . .
for k = 1, 2, . . . less susceptible to
xm = a + (b-a)/2 roundoff error than
if sign(f (xm)) = sign(f (a)) (a+b)/2
a = xm
else
b = xm
end
if converged, stop
end
Example: Apply Bisection to x - x1/3 - 2 = 0

bisect.m
function xm = bisect(a, b, n)
if nargin<3, n=15; end % Default number of iterations
fa = a - a^(1/3) - 2; % Initial value of f(a) and f(b)
fb = b - b^(1/3) - 2;
fprintf(‘ k a xmid b f(xmid)\n’);
for k = 1:n
xm = a + 0.5*(b-a); % Computing midpoint
fm = xm - xm^(1/3) - 2; % Function value at midpoint
fprintf(‘%3d %12.8f %12.8f %12.8f %12.3e\n’, k, a, xm, b, fm);
if sign(fm) == sign(fa) % Root lies in [xm, b] replace a
a = xm;
fa = fm;
else % Root lies in [xm, a] replace b
b = xm;
fb = fm;
end
end
Newton’s Method
First-order Taylor series: f ( xi +1 ) ≅ f ( xi ) + f ′( xi )( xi+1 − xi )
f ( xi )
Set f ( xi +1 ) = 0 → xi +1 = xi −
f ′( xi )
f(x)
slope = f’(x)
f(xi)
Root
xi+1 xi x
f ( xi )
xi − xi +1 =
f ′( xi )
Approximation Sequence of Newton’s Method
f(x)
f(x1)
x2
x3 x1 x
f(x2)
Failure of Newton’s Method
Case 1: Inflection point in vicinity of root
f(x)
x2 x1 x3
x

Case 2: Oscillate around local maximum or minimum
f(x)
x
Case 3: Jump away for several roots
f(x)

Case 4: Disaster from zero slope
f(x)
x
Example: Apply Newton’s Method to x - x1/3 - 2 = 0
newton.m
function x = newton(x0, n)
if nargin<2, n=5; end % Default number of iterations
x = x0; % Initial guess
fprintf(‘ k f(x) dfdx x(k+1)\n’);
for k = 1:n
f = x - x.^(1/3) - 2; % Function value
dfdx = 1 - (1/3)*x.^(-2/3); % Derivative value
x = x - f/dfdx;
fprintf(‘%3d %12.3e %12.3e %18.15f\n’, k-1, f, dfdx, x);
end
Secant Method
for function whose derivatives are difficult to evaluate
Derivative approximated by backward finite difference
f ( xi −1 ) − f ( xi ) xi +1 = xi −
f ( xi )
f ′ ( xi ) ≅
xi −1 − xi f ′( xi )
f ( xi ) ( xi −1 − xi )
f(xi)
xi +1 = xi −
f ( xi −1 ) − f ( xi )
f(xi-1)
x
xi-1 xi
Example: Use secant method to estimate root of e-x - x = 0
Initial estimate x-1 = 0 and x0 = 1.0
True root = 0.56714329. . .
First iteration:
x-1 = 0 f (x-1) = 1.00000
x0 = 1 f (x0) = -0.63212
−0.63212(0 − 1)
x1 = 1 − = 0.61270 ε t = 8.0%
1 − ( −0.63212)
Second iteration:
x0 = 1 f (x0) = -0.63212
x1 = 0.61270 f (x1) = -0.07081
−0.07081(1 − 0.61270)
x2 = 0.61270 − = 0.56384 ε t = 0.58%
−0.63212 − (−0.07081)
Third iteration:
x1 = 0.61270 f (x1) = -0.07081
x2 = 0.56384 f (x2) = 0.00518
0.00518(0.61270 − 0.56384)
x3 = 0.56384 − = 0.56717
−0.07081 − ( −0.00518)
ε t = 0.0048%
Roots of Polynomials
f n ( x ) = a0 + a1 x + a2 x 2 + L + an x n
where n = order of polynomial
a’s = constant coefficients
Roots of polynomials:
(1) nth-order equation has n real or complex roots
(2) If n is odd, at least one root is real.
(3) Complex roots exist with conjugate pairs (λ + µi and λ − µi)
Ordinary Differential Equation (ODE)
d2y dy
a2 2 + a1 + a0 y = F (t )
dt dt
d2y dy
General Solution: a2 2 + a1 + a0 y = 0
dt dt
Set y = ert → a2 r 2 ert + a1rert + a0 ert = 0
Canceling exponents → a2 r 2 + a1r + a0 = 0

Characteristic equation
Characteristic Equation a2 r 2 + a1r + a0 = 0
Roots of equation by quadratic formula
r1 − a1 ± a12 − 4a2 a0 EIGENVALUES

=
r2 2a2
(1) If a12 - 4a2a0 > 0, roots are real

y
Overdamped case
y = c1er1t + c2er2t
(2) If a12 - 4a2a0 = 0, single root is real

y
Critically damped case
y = ( c1 + c2t ) e rt
(3) If a12 - 4a2a0 < 0, two roots are complex conjugate

Underdamped case y
r1
= λ ± µi t
r2
y = c1eλt cos µ t + c2 eλt sin µ t
f1 ( x) = x 2 − 3 x + 2
r1 3 ± 32 − 4(2) 2 Distinct real roots
= =
r2 2 1
f 2 ( x) = x 2 − 10 x + 25
r1 10 ± 102 − 4(25) 5 Repeated real roots
= =
r2 2 5
f3 ( x) = x 2 − 17 x + 72.5
Complex roots
r1 17 ± 17 2 − 4(72.5) 8.5 + 0.5i
= =
r2 2 8.5 − 0.5i
Roots of Polynomials
3
2.5
1.5
y = f(x)
0.5
-0.5
distinct repeated complex
real roots real roots roots
-1
0 2 4 6 8 10
x
MATLAB’s roots Function
Compute eigenvalues of companion matrix
4th-order polynomial: c5 λ 4 + c4 λ 3 + c3λ 2 + c2λ + c1 = 0
 −c2 / c1 −c3 / c1 −c4 / c1 −c5 / c1 
 1 0 0 0 
Companion
A= 
Matrix  0 1 0 0 
 
 0 0 1 0 
Eigenvalue problem Aυ = λυ
f1 ( x) = x 2 − 3 x + 2 →  root([1 -3 2])
f 2 ( x) = x 2 − 10 x + 25 →  root([1 -10 25])
f3 ( x) = x 2 − 17 x + 72.5 →  root([1 -17 72.5])
Lecture 6
System of Linear Algebraic Equations
- Basic Concepts
- Gaussian Elimination
- Backward Substitution
- LU Decomposition

Linear Algebraic Equations

A system of n linear equations in n unknowns,
a11x1 + a12x2 + . . . a1nxn = b1

a21x1 + a22x2 + . . . a2nxn = b2
n equations
. .
. .
. .
an1x1 + an2x2 + . . . annxn = bn
n unknowns
Example: c 1 + 3c 2 − 5c 3 = 7
2c 1 − 8c 2 + 3c 3 = −2
5c 1 + 7c 2 − 4c 3 = 9
Matrix Form: Augment Form:
1 3 − 5  c 1   7  1 3 − 5 7 
2 − 8 3  c  = − 2   2 − 8 3 − 2
   2    
5 7 − 4  c 3   9  5 7 − 4 9 
Symbolic Notation A x = b
System of linear algebraic equations Ax=b

 a11 a12 L a1n   x1   b1  m = number of rows
a L a2n   x2  b 
 21 a22 =  2
 M M O M  M  M n = number of columns
    
am1 am2 L amn   xm  bm 
where aij and bi are constants, i = 1,2,…,m, j = 1,2,…,n
Solution : Ax = b MATLAB :
A-1A x = A-1b - Use matrix inversion,
x = A-1b >> x = inv(A)*b

- Use backslash operator,
>> x = A\b
Requirements for a Solution, Am×n
m rows and n columns
m=n
Solving Ax = b for n unknowns from n equations
m>n
Overdetermined systems
(e.g., least-squares problems)
m<n
Underdetermined systems
(e.g., optimization)
Matrix Rank
rank(A) = number of linearly independent columns in A
>> A = [3 -4 1;6 10 2;9 -7 3];
>> rank(A)
ans =
For matrix A with m rows and n columns, and Ax = b has a solution
- if rank(A) < n , → infinite number of solutions
- if rank(A) = n , → unique solution
For a system of n equations in n unknowns written in the form Ax = b,

the solution x exists and is unique for any b if and only if rank(A) = n.
Consistency Test
Element of vector x are coefficients in the linear equation
 a11   a12   a1n   b1 
a  a  a  b 
Ax = b ⇔ x1   + x2   + L + xn  2 n  =  2 
21 22
 M   M   M  M
       
 am1   am 2   amn  bm 
 a11 a12 a13 L a1n b1 

a a11 a11 L a11 b2 
 11
Augmented matrix: A% =  a11 a11 a11 L a11 b3  = [Ab]
 
M M M O M M
 a11 a11 a11 L a11 bm 
Consistent when A and A% have the same rank
Example: An Inconsistent System from Data Fitting
x 1 2 3
Experiment data: Practical Model: y = α x + β
y 2 1 0.5
Data → Model:
3
α + β = 2
2 2α + β = 1
1 3α + β = 0.5
0
Matrix Form:
 1 1  2
-1  2 1 α  1 
 
 β  =  
-2  3 1    0.5
0 1 2 3 4
Consistency test : >> A=[1 1; 2 1; 3 1]; b=[2; 1; 0.5];

>> rank(A)
>> rank([A b])
2nd Experiment: 1 1  2  1  1  2 
 α
         
 2 1   β  = 1  ⇒ ( −1) 2  + ( 3) 1  = 1 
3 1    0  3  1  0 
Consistency test: [α β ] = [− 1 3 ]
T T
>> A = [1 1; 2 1; 3 1]; b = [2; 1; 0];

>> rank(A)
>> rank([A b]) 3
2 Exact solution
1
-1
-2
0 1 2 3 4
Nonsingular Case
§ If the coefficient matrix A is nonsingular, then it is invertible and we can

solve Ax = b as follows:
Ax =b x = A-1 b
§ This solution is therefore unique. Also, if b = 0, it follows that the unique
-1
solution to Ax = 0 is x = A 0 = 0.
§ Thus if A is nonsingular, then the only solution to Ax = 0 is the trivial

solution x = 0.
Naive Gauss Elimination
The most basic systematic scheme for solving system of linear equations.
The procedure consisted of two steps:
1) Forward elimination of the linear equation matrix using row operation

to obtain an upper triangular matrix.
2) Backward substitution to solve for the unknowns
elimination backward
row operation substitution
Matrix Upper triangular Solution

Forward elimination
a11 a12 a13 b1  a11 a12 a13 b1 

a b2  ⇒  ′ ′ b2′ 
 21 a22 a23  a22 a23
a31 a32 a33 b3   ′′
a33 b3′′ 
Backward substitution
x3 = b3′′ / a33
′′
x2 = (b2′ − a23
′ x 3 ) / a22
′
x1 = (b1 − a13 x3 − a12 x 2 ) / a11
Substitutions
Backward substitution: Upper triangular matrix
bn
xn =
ann
n
1
xi = (bi − ∑ aij x j ), i = n − 1, n − 2, L, 2, 1
aii j = i +1
Forward substitution: Lower triangular matrix

b1
x1 =
a11
i −1
1
x i = (bi − ∑ aij x j ), i = 1, 2, 3, L, n
aii j =1
EXAMPLE : Naive Gauss Elimination
− 3 x1 + 2 x2 − x3 = −1 − 3 2 − 1 − 1 R1
6 x1 − 6 x2 + 7 x3 = −7 ⇒  6 − 6 7 − 7 R2
 
3 x1 − 4 x2 + 4 x3 = −6  3 − 4 4 − 6  R3
Forward elimination (Row operation):

Pivot element
− 3 2 − 1 − 1 Pivot row
 0 − 2 5 − 9
R2 + 2×R1, R3+R1: ⇒  
 0 − 2 3 − 7
− 3 2 − 1 − 1
R3 - R2: ⇒
 0 −2 5 − 9 
 
 0 0 −2 2
− 3 2 − 1 − 1 − 3 x1 + 2 x 2 − x 3 = −1
 0 −2 5 − 9 ⇒ − 2 x 2 + 5 x3 = −9

 0 0 −2 2 − 2 x3 = 2
Backward substitution:
1
− 3 x1 + 2 x2 − x3 = −1 ⇒ x1 = ( −1 − 2 x2 + x3 ) = 2
−3
1
− 2x 2 + 5 x3 = −9 ⇒ x2 = ( −9 − 5 x3 ) = 2
−2
2
− 2x3 = 2 ⇒ x3 = = −1
−2
MATLAB: >> A = [-3 2 -1;6 -6 7;3 -4 4]

>> b = [-1;-7;-6]
>> x=A\b
Gaussian Elimination with Pivoting
To prevent failure from zero diagonal elements
 2 4 −2 −2 − 4  R1
1 2 4 −3 5  R2
Augmented matrix: A% = [ A b ] = 
 −3 −3 8 −2 7  R3
 
 −1 1 6 −3 7  R4
First row operation: 2 4 −2 −2 − 4 

0 0 5 −2 7 
R2 – R1/2, 
R3 + 3R1/2, 0 3 5 −5 1 
R4 + R1/2  
0 3 5 −4 5 
next elimination fail
PIVOTING
Exchange row to avoid zero pivot element
2 4 −2 −2 − 4  2 4 −2 −2 − 4 
0 0 5 −2 7  0 3 5 −4 5 
 
0 3 5 −5 1  0 3 5 −5 1 
   
0 3 5 −4 5  0 0 5 −2 7 
R3=R3-R2
2 4 −2 −2 − 4  2 4 −2 −2 − 4 
0 3 5 −4 5  0 3 5 −4 5 
 
0 0 0 −1 −4  0 0 5 −2 7 
   
0 0 5 −2 7  0 0 0 −1 4 
Solving Systems with the Backslash Operator
MATLAB
Ax =b x = A-1 b x =A\b
A \ . . . means multiply on the left by the inverse of A
>> A = [2 4 -2 -2;1 2 4 -3;-3 -3 8 -2;-1 1 6 -3]

>> b = [-4;5;7;7]
>> x = A\b
x =
1.0000
2.0000
3.0000
4.0000
LU Decomposition
To solve several Ax = b systems with the same A but different b

LU decomposition avoids repeating steps of Gaussian elimination on A
Main idea is to record the steps used in Gaussian elimination.
Consider the matrix: 1 − 2 3 

A = 2 − 5 12 
0 2 − 10 
Gaussian elimination:
1 − 2 3 
(2)
0 − 1 6 
R2 – 2×R1  
0 2 − 10 
For recording
Gaussian elimination (con’t): Lower triangular matrix:
 1 − 2 3 1 0 0 
R3 – 0×R1 (2) − 1 6
  ⇒ L = 2 1 0 
R3 + 2×R1 (0 ) (−2) 2 0 − 2 1
Mysterious coincidence !!!
 1 0 0   1 − 2 3  1 − 2 3 
LU = 2 1 0  0 − 1 6  = 2 − 5 12  = A
    
0 − 2 1 0 0 2 0 2 − 10 
MATLAB: >> L = [1 0 0;2 1 0;0 -2 1]

>> U = [1 -2 3;0 -1 6;0 0 2]
U >> A = L*U
A =
A 1 -2 3
L 2 -5 12
0 2 -10
Using LU to solve equations
Ly = b : Forward substitution
Ax = b → LUx = b
Ux = y : Backward substitution
For Gauss elimination with pivoting:
P = identity matrix with same rows

PA = LU switched as A in pivoting.
For example PA : (switch R2 ↔ R3)
1 0 0 1 − 2 3  1 − 2 3 
0 0 1 2 − 5 12  = 0 2 − 10 
    
0 1 0  0 2 − 10  2 − 5 12 
Ly = b’
PAx = Pb ≡ b’ → LUx = b’
Ux = y
Gauss elimination as LU decomposition
Gauss elimination can be used to decompose A into L and U as illustrated

for a three-equation system,
a11 a12 a13   x1   b1  Forward a11 a12 a13 

a a23   x 2  = b2   0 a′ ′  = U
 21 a22 elimination
 22 a23 
a31 a32 a33   x 3  b3   0 0 a33 
′′
Store row operation @ zeros position:
 a11 a12 a13 

(f ) a′ a′
′23 
a21 a
a where f21 = , f31 = 31 , and f32 = 32
 21 22  a11 a11 ′
a22
(f31 ) (f32 ) a33′′ 
1 0 0 a11 a12 a13 

[A] → [L][U] L = f21 1 0 U =  0 a22
′ ′ 
a23 
f31 f32 1  0 0 a33 
′
′
EXAMPLE : LU Decomposition with Gauss Elimination
From previous example,

− 3 2 − 1 a11 a12 a13 
 
A =  6 − 6 7  = a21 a22 a23 
a21 6
First step: R2+2R1 f21 = = = −2,
a11 − 3
 3 − 4 4  a31 a32 a33 
a31 3
Second step: R3+R1 f31 = = = −1,
a11 − 3
− 3 2 − 1 a11 a12 a13 
 ′ a′23 
After the second step of forward elimination ⇒  0 − 2 5  =  0 a22 
′
a32 −2  0 − 2 3   0 a32 a33 
 ′ ′
Third step: R3-R2 f32 = = =1
′
a22 −2
− 3 2 − 1 a11 a12 a13 
 ′ 
After forward elimination, U =  0 − 2 5  =  0 a22
′ a23

 
 ′′ 
 0 0 − 2  0 0 a33 
1 0 0  1 0 0
 0 = − 2 1 0
Lower triangular matrix, L = f21 1
f31 f32 1  − 1 1 1
Check LU = A or not ?
Using LU to solve equations (con’t)
Ly = b
Ax = b → LUx = b
− 3 2 − 1  x1   − 1 Ux = y
 6 − 6 7   x  = − 7
   2  
 3 − 4 4   x2  − 6 
Ly = b
1 0 0  y1   b1   1 0 0  y 1   − 1
f 0  y  = b  − 2 1 0   y  =  − 7 
 21 1  2  2    2  
f31 f32 1  y 3  b3   − 1 1 1 y 3  − 6
y1 = b1 y1 = −1
y 2 = b2 − y 1 f21 y 2 = −7 − ( −1)(−2) = −9
y 3 = b3 − y 1 f31 − y 2 f32 y 3 = −6 − ( −1)( −1) − ( −9) × 1 = 2
Ux = y
a11 a12 a13   x1   y1  − 3 2 − 1  x1   − 1
 0 a′ ′  x2  = y 2 
a23  0 − 2 5   x  =  − 9
 22        2  
 0 0 ′′   x3   y 3 
a33  0 0 − 2  x3   2 
y3 2
x3 = x3 = = −1
′′
a33 −2
′
y 2 − x3 a23 − 9 − ( −1)(5)
x2 = x2 = =2
′
a22 −2
y 1 − x 2 a12 − x3 a13 − 1 − 2 × 2 − ( −1)( −1)
x1 = x1 = =2
a11 −3
MATLAB Function: lu
>> [L, U] = lu(A)
− 3 2 − 1  x1   − 1
 6 − 6 7   x  =  − 7
   2  
 3 − 4 4   x2  − 6
LU Decomposition:
>> A = [-3 2 -1;6 -6 7;3 -4 4];

>> b = [-1; -7; -6];
>> [L,U] = lu(A)
Solve equations:
>> y = L\b
>> x = U\y
Various LU decompositions
Crout’
Crout’s reduction (U has ones on the diagonal)
l11 0 0  1 u12 u13  a11 a12 a13 
l 0  × 0 1 u23  = a21 a22 a23 
 21 l 22
l 31 l 32 l 33  0 0 1  a31 a32 a33 
Doolittle’
Doolittle’s method (L has ones on the diagonal)
1 0 0  u11 u12 u13  a11 a12 a13 
l 0  ×  0 u 22 u23  = a21 a22 a23 
 21 1
l 31 l 32 1  0 0 u33  a31 a32 a33 
Cholesky’
Cholesky’s method (The diagonal terms are the same value for
the L and U matrices)
l11 0 0  u11 u12 u13  a11 a12 a13 
l 0  ×  0 u 22 u 23  = a21 a22 a23 
 21 l 22
l 31 l 32 l 33   0 0 u33  a31 a32 a33 
where, lii = uii

Cholesky Decompositions
One of the most popular approaches based on the fact that a symmetric
matrix can be decomposed as
- [A] is a symmetric matrix
[A] = [U]T [U] - Computational advantages:
- Half storage
- Half computation time
Decomposition by recurrence relations. For the i th row:
i −1
uii = aii − ∑ uki2
k =1
i −1
aij − ∑ uki ukj
uij = k =1
for j = i + 1, K, n
uii
EXAMPLE : Cholesky Decomposition
4 8 54  For the first row (i = 1): u11 = a11 = 4 = 2

A =  8 32 216  a 8
u12 = 12 = = 4
54 216 1,494  u11 2
a 54
For the second row (i = 2): u13 = 13 = = 27
u11 2
u22 = a22 − u122
= 32 − 42 = 4
u11 u12 u13 
a23 − u12u13 216 − 4 × 27
u23 = = = 27 U =  0 u 22 u 23 
u22 4
 0 0 u33 
For the third row (i = 3):
u33 = a33 − u13

2
− u23
2
= 1,494 − 27 2 − 27 2 = 6
2 4 27  4 8 27 
Thus, Cholesky yields U = 0 4 27  ⇒ U T U =  8 32 216  = A
  
0 0 6  27 216 1,494 
MATLAB Function: chol
>> U = chol(A)
4 8 54 
A =  8 32 216 
54 216 1,494 
Cholesky Decomposition:
>> A = [4 8 54;8 32 216;54 216 1494];

>> U = chol(A)
Test:
>> U’*U
Example : Member forces in truss

1000 N
[ΣM2 = 0] V3(2) – 1000(1.5) = 0 → V3 = 750 N 1
90o
0.866 m
[ΣFx = 0] H2 = 0 F1
F3
[ΣFy = 0] V2 = 250 N H2 2 30o 60o 3
@ node 1 :
F2
V2 V3
[ΣFx = 0] F1cos30o – F3cos60o = 0
1.5 m 0.5 m
[ΣFy = 0] F1sin30o + F3sin60o = 1000
@ node 3 :
@ node 2 :
[ΣFx = 0] – F2 + F3cos60o = 0
[ΣFx = 0] – F1cos30o + F2 = 0
[ΣFy = 0] F3 = 750/sin60o = 866 N
[ΣFy = 0] F1sin30o = 250
F2 = 866cos60o = 433 N
F1 = 250/sin30o = 500 N
F1sin30o + F3sin60o = 1000  sin 30° 0 sin 60°   F1  1,000 
− cos 30° 1 0  F2  =  0 
– F1cos30o + F2 = 0 
 0 − 1 cos 60° F3   0 
– F2 + F3cos60o = 0
>> A=[sin(pi/6) 0 sin(pi/3);-cos(pi/6) 1 0;0 -1 cos(pi/3)]
>> b=[1000;0;0]
>> F=A\b
F =
500.0000
433.0127
866.0254
Lecture 7 Curve Fitting
< Linear Regression

< Polynomial Regression
< Multiple Linear Regression
LINEAR REGRESSION
We want to find the curve that will fit the data.
Candidate lines for curve fit

y = α x +β
No exact solution but many approximated solutions

Error Between Model and Observation
y Observation: [ xi yi ]
Model: y = α x + β
Error: ei = yi – α xi – β
x
Criteria for a “Best” Fit
Find the BEST line which minimize the sum of error for all data
BEST line with error minimized ?
The problem is how to minimize the error.
We can use the error

defined as:
ei = y i − yˆ
Where
ŷ = α x i + β However, the errors can
cancel one another and
still be wrong.
ERROR Definition
To avoid ± signs cancellation, the

error may be defined as:
ei = y i − yˆ
But, the error minimization is going to have problems.
The solution is the minimization of the sum of squares.
S = ∑ (ei )
2
This will give a least square solution.
Least-Square Fit of a Straight Line
Minimize sum of the square of the errors

n n
S r = ∑ e = ∑ (y i − β − αx i )
2 2
i
i =1 i =1
Differentiate with respect to each coefficient:

∂S r
= −2 ∑( y i − β − αx i )
∂β
∂S r
= −2 ∑[( y i − β − αx i ) x i ]
∂α
Setting derivatives = 0 :
0 = ∑ y i − ∑ β − ∑ αx i
0 = ∑ y i x i − ∑ βx i − ∑ αx i2
From Σβ = n β , express equations as set of 2 unknowns ( β , α )
nβ + α ∑ x i = ∑ y i
β ∑ x i + α ∑ x i2 = ∑ y i x i
Solve equations simultaneously:
1
∑ xi yi − ∑ xi ∑ yi S xy
α= n α=
1 S xx
∑ xi2 − ( ∑ xi )
2
β = y −α x
where y and x are the mean of y and x
1
Define: S xy = Σxi yi − Σxi Σyi
n Approximated y for any x is
1
S xx = Σxi2 − ( Σxi ) ŷ = α x + β
2
n
1
S yy = Σyi2 − ( i)
Σ
2
y
n
Example: Fit a straight line to x and y values
xi yi xi2 xi yi yi2
n =7
1 0.5 1 0.5 0.25
2 2.5 4 5.0 6.25 28
3 2.0 9 6.0 4 x = =4
4 4.0 16 16.0 16 7
5 3.5 25 17.5 12.25
6 6.0 36 36.0 36 24
7 5.5 49 38.5 30.25 y = = 3 . 4286
7
Σ 28 24 140 119.5 105
(119.5) − (28)(24) / 7
α= = 0.8393 Least-square fit:
(140) − (28) 2 / 7
y = 0 . 8393 x + 0 . 0714
β = 3.4286 − 0.8393(4) = 0.0714
How good is our fit?
Sum of the square of the errors:
S r = ∑ e = ∑ ( yi − β − α xi ) = ( S xx S yy − S xy2 ) / S xx
n n
2 2
i
i =1 i =1
n
St = ∑ ( yi − y ) = S yy
2
Sum of the square around the mean:
i =1
Sr
Standard errors of the estimation: sy / x =
n−2
St
Standard deviation: sy =
n−2
Linear regression
sy > sy/x
sy sy/x
y
St − S r S xy2
Coefficient of determination r2 = =
St S xx S yy
r2 คืออัตราสวนการแปรเปลี่ยนคา y ที่เกิดจากการเปลี่ยนคา x
For perfect fit Sr = 0 and r = r2 = 1
Example: error analysis of the linear fit y = 3.4286

α = 0.8393
( yi − y ) ( yi - β - α xi)2
2
xi yi β = 0.0714
1 0.5 8.5765 0.1687
22.7143
2 2.5 0.8622 0.5626 sy =
3 2.0 2.0408 0.3473 7−2
4 4.0 0.3265 0.3265 = 2.131
5 3.5 0.0051 0.5896
2.9911
6 6.0 6.6122 0.7972 sy / x =
7 5.5 4.2908 0.1993 7−2
= 0.773
Σ 28 24 22.7143 2.9911
St Sr
Since sy/x < sy , linear regression has merit.
22.7143 − 2.9911
r= = 0.868 = 0.932
22.7143
Linear model explains 86.8% of original uncertainty.
OR Example: error analysis of the linear fit
xi yi x i2 xi yi y i2
S xx = 140 − 282 / 7 = 28
1 0.5 1 0.5 0.25
2 2.5 4 5.0 6.25 S yy = 105 − 242 / 7 = 22.7
3 2.0 9 6.0 4
4 4.0 16 16.0 16
5 3.5 25 17.5 12.25 S xy = 119.5 − 28 × 24 / 7 = 23.5
6 6.0 36 36.0 36
7 5.5 49 38.5 30.25 Sr = (28 × 22.7 − 23.52 ) / 28
Σ 28 24 140 119.5 105 = 2.977
Since sy/x < sy , linear regression has merit. 22.7

sy = = 2.131
7−2
23.52 2.977
r =
2
= 0.869 sy / x = = 0.772
28 × 22.7 7−2
Confidence Interval (CI)

A confidence interval is an interval in which a measurement or trial
falls corresponding to a given probability.
yˆ i ± ∆
yˆ i
y y
x xi x
For CI 95%, you can be 95% confident that the two curved
confidence bands enclose the true best-fit linear regression line,
leaving a 5% chance that the true line is outside those boundaries.
A 100 (1 - α) % confidence interval for yi is given by
Confidence interval 95% → α = 0.05
1 ( xi − x ) 2
yˆ i ± tα / 2 s y / x +
n S xx
Example: to estimate y when x is 3.4 using 95% confidence interval:
xi yi yˆ = α x + β = 0.8363(3.4) + 0.0714 = 2.9148
95% Confidence → α = 0.05 → tα/2 = t0.025(df = n-2 = 5) = 2.571

1 0.5
2 2.5
1 (3.4 − 4)2
3 2.0 Interval: 2.9148 ± (2.571) (0.772) +
4 4.0 7 28
5 3.5
6 6.0 2.9148 ± 0.7832
7 5.5
T-Distribution
Probability density function of
t 0.025 t 0.025 the t distribution:
t 0.005 t 0.005
(1 + x 2 /ν ) − (ν +1) / 2
f ( x) =
t
B(0.5, 0.5ν ) ν
95% where B is the beta function and
99% ν is a positive integer
shape parameter.
The formula for the beta function is

1
B(α , β ) = ∫ t α −1 (1 − t ) β −1 dt
0
The following is the plot of the t probability density function for 4 different
values of the shape parameter.
ν = df
Degree of freedom
In fact, the t distribution with ν equal to 1 is a Cauchy distribution.

The t distribution approaches a normal distribution as ν becomes large.
The approximation is quite good for values of ν > 30.
Critical Values of t
Confidence Interval
80% 90% 95% 98% 99% 99.8%
df 0.10 0.05 0.025 0.01 0.005 0.001
1 3.078 6.314 12.706 31.821 63.657 318.313

2 1.886 2.920 4.303 6.965 9.925 22.327
3 1.638 2.353 3.182 4.541 5.841 10.215
4 1.533 2.132 2.776 3.747 4.604 7.173
5 1.476 2.015 2.571 3.365 4.032 5.893
6 1.440 1.943 2.447 3.143 3.707 5.208

7 1.415 1.895 2.365 2.998 3.499 4.782
8 1.397 1.860 2.306 2.896 3.355 4.499
9 1.383 1.833 2.262 2.821 3.250 4.296
10 1.372 1.812 2.228 2.764 3.169 4.143
11 1.363 1.796 2.201 2.718 3.106 4.024

12 1.356 1.782 2.179 2.681 3.055 3.929
13 1.350 1.771 2.160 2.650 3.012 3.852
14 1.345 1.761 2.145 2.624 2.977 3.787
15 1.341 1.753 2.131 2.602 2.947 3.733
16 1.337 1.746 2.120 2.583 2.921 3.686

17 1.333 1.740 2.110 2.567 2.898 3.646
18 1.330 1.734 2.101 2.552 2.878 3.610
19 1.328 1.729 2.093 2.539 2.861 3.579
20 1.325 1.725 2.086 2.528 2.845 3.552
Confidence Interval
80% 90% 95% 98% 99% 99.8%
df 0.10 0.05 0.025 0.01 0.005 0.001
21 1.323 1.721 2.080 2.518 2.831 3.527

22 1.321 1.717 2.074 2.508 2.819 3.505
23 1.319 1.714 2.069 2.500 2.807 3.485
24 1.318 1.711 2.064 2.492 2.797 3.467
25 1.316 1.708 2.060 2.485 2.787 3.450
26 1.315 1.706 2.056 2.479 2.779 3.435

27 1.314 1.703 2.052 2.473 2.771 3.421
28 1.313 1.701 2.048 2.467 2.763 3.408
29 1.311 1.699 2.045 2.462 2.756 3.396
30 1.310 1.697 2.042 2.457 2.750 3.385
31 1.309 1.696 2.040 2.453 2.744 3.375

32 1.309 1.694 2.037 2.449 2.738 3.365
33 1.308 1.692 2.035 2.445 2.733 3.356
34 1.307 1.691 2.032 2.441 2.728 3.348
35 1.306 1.690 2.030 2.438 2.724 3.340
36 1.306 1.688 2.028 2.434 2.719 3.333

37 1.305 1.687 2.026 2.431 2.715 3.326
38 1.304 1.686 2.024 2.429 2.712 3.319
39 1.304 1.685 2.023 2.426 2.708 3.313
40 1.303 1.684 2.021 2.423 2.704 3.307
Confidence Interval
80% 90% 95% 98% 99% 99.8%
df 0.10 0.05 0.025 0.01 0.005 0.001
41 1.303 1.683 2.020 2.421 2.701 3.301

42 1.302 1.682 2.018 2.418 2.698 3.296
43 1.302 1.681 2.017 2.416 2.695 3.291
44 1.301 1.680 2.015 2.414 2.692 3.286
45 1.301 1.679 2.014 2.412 2.690 3.281
46 1.300 1.679 2.013 2.410 2.687 3.277

47 1.300 1.678 2.012 2.408 2.685 3.273
48 1.299 1.677 2.011 2.407 2.682 3.269
49 1.299 1.677 2.010 2.405 2.680 3.265
50 1.299 1.676 2.009 2.403 2.678 3.261
51 1.298 1.675 2.008 2.402 2.676 3.258

52 1.298 1.675 2.007 2.400 2.674 3.255
53 1.298 1.674 2.006 2.399 2.672 3.251
54 1.297 1.674 2.005 2.397 2.670 3.248
55 1.297 1.673 2.004 2.396 2.668 3.245
56 1.297 1.673 2.003 2.395 2.667 3.242

57 1.297 1.672 2.002 2.394 2.665 3.239
58 1.296 1.672 2.002 2.392 2.663 3.237
59 1.296 1.671 2.001 2.391 2.662 3.234
60 1.296 1.671 2.000 2.390 2.660 3.232
Confidence Interval
80% 90% 95% 98% 99% 99.8%
df 0.10 0.05 0.025 0.01 0.005 0.001
61 1.296 1.670 2.000 2.389 2.659 3.229

62 1.295 1.670 1.999 2.388 2.657 3.227
63 1.295 1.669 1.998 2.387 2.656 3.225
64 1.295 1.669 1.998 2.386 2.655 3.223
65 1.295 1.669 1.997 2.385 2.654 3.220
66 1.295 1.668 1.997 2.384 2.652 3.218

67 1.294 1.668 1.996 2.383 2.651 3.216
68 1.294 1.668 1.995 2.382 2.650 3.214
69 1.294 1.667 1.995 2.382 2.649 3.213
70 1.294 1.667 1.994 2.381 2.648 3.211
71 1.294 1.667 1.994 2.380 2.647 3.209

72 1.293 1.666 1.993 2.379 2.646 3.207
73 1.293 1.666 1.993 2.379 2.645 3.206
74 1.293 1.666 1.993 2.378 2.644 3.204
75 1.293 1.665 1.992 2.377 2.643 3.202
76 1.293 1.665 1.992 2.376 2.642 3.201

77 1.293 1.665 1.991 2.376 2.641 3.199
78 1.292 1.665 1.991 2.375 2.640 3.198
79 1.292 1.664 1.990 2.374 2.640 3.197
80 1.292 1.664 1.990 2.374 2.639 3.195
Confidence Interval
80% 90% 95% 98% 99% 99.8%
df 0.10 0.05 0.025 0.01 0.005 0.001
81 1.292 1.664 1.990 2.373 2.638 3.194

82 1.292 1.664 1.989 2.373 2.637 3.193
83 1.292 1.663 1.989 2.372 2.636 3.191
84 1.292 1.663 1.989 2.372 2.636 3.190
85 1.292 1.663 1.988 2.371 2.635 3.189
86 1.291 1.663 1.988 2.370 2.634 3.188

87 1.291 1.663 1.988 2.370 2.634 3.187
88 1.291 1.662 1.987 2.369 2.633 3.185
89 1.291 1.662 1.987 2.369 2.632 3.184
90 1.291 1.662 1.987 2.368 2.632 3.183
91 1.291 1.662 1.986 2.368 2.631 3.182

92 1.291 1.662 1.986 2.368 2.630 3.181
93 1.291 1.661 1.986 2.367 2.630 3.180
94 1.291 1.661 1.986 2.367 2.629 3.179
95 1.291 1.661 1.985 2.366 2.629 3.178
96 1.290 1.661 1.985 2.366 2.628 3.177

97 1.290 1.661 1.985 2.365 2.627 3.176
98 1.290 1.661 1.984 2.365 2.627 3.175
99 1.290 1.660 1.984 2.365 2.626 3.175
100 1.290 1.660 1.984 2.364 2.626 3.174
∞ 1.282 1.645 1.960 2.326 2.576 3.090

Polynomial Regression
Second-order polynomial:
y = a0 + a1x + a2 x2
Sum of the squares of the residuals:

S r = ∑( y i − a 0 − a 1 x i − a 2 x i2 ) 2
Take derivative with respect to each coefficients:

∂S r
= −2 ∑( y i − a 0 − a 1 x i − a 2 x i2 )
∂a 0
∂S r
= −2 ∑ x i ( y i − a 0 − a 1 x i − a 2 x i2 )
∂a 1
∂S r
= −2 ∑ x i2 ( y i − a 0 − a 1 x i − a 2 x i2 )
∂a 2
Normal equations:
( )
n a 0 + (∑ x i )a 1 + ∑ x i2 a 2 = ∑ y i
(∑ x i )a 0 + (∑ x i2 )a + (∑ x )a
1 i
3
2 = ∑ x iy i
(∑ x )a + (∑ x
i
2
0 i
3
)a + (∑ x )a
1 i
4
2 = ∑ x i2 y i
For mth-order polynomial: y = a0 + a1x + a2 x2 + . . . + amxm
We have to solve m+1 simultaneous linear equations.

MATLAB polyfit Function
For second-order polynomial, we can define
 x12 x1 1  y1 
 2  y   c1 
, Y =  2  , C =  c2 
x x2 1
A= 2
M M M M   
 2     c3 
 xm xm 1  ym 
and show that C = ( A ' A) −1 A ' Y or C=A -1Y
Fit norm Fit QR
>> C = polyfit(x, y, n)
>> [C, S] = polyfit(x, y, n)
x = independent variable C = coeff. of polynomial in

y = dependent variable descending power
n = degree of polynomial S = data structure for polyval
function
Example: Fit a second-order polynomial to the data.
( yi − y )
2
xi yi ( yi - a0 - a1xi - a2xi2)2
0 2.1 544.44 0.14332
1 7.7 314.47 1.00286
2 13.6 140.03 1.08158
3 27.2 3.12 0.80491
4 40.9 239.22 0.61951
5 61.1 1272.11 0.09439
Σ 15 152.6 2513.39 3.74657
From the given data: m=2 ∑ xi = 15 ∑ xi4 = 979

n=6 ∑ yi = 152.6 ∑ xi yi = 585.6
x = 2.5 ∑ xi2 = 55 ∑ xi2 yi = 585.6
y = 25.433 ∑ xi3 = 225

Simultaneous linear equations
6 15 25  a0   152.6 
15    
 55 225   a1  =  585.6 
55 225 979  a2  2488.8
Solving these equation gives a0 = 2.47857, a1 = 2.35929, and

a2 = 1.86071.
Least-squares quadratic equation:
y = 2.47857 + 2.35929x + 1.86071x2

Coefficient of determination:
2513.39 − 3.74657
r= = 0.99851 = 0.99925
2513.39
Solving by MATLAB polyfit Function
>> x = [0 1 2 3 4 5];
>> y = [2.1 7.7 13.6 27.2 40.9 61.1];
>> c = polyfit(x, y, 2)
>> [c, s] = polyfit(x, y, 2)
>> st = sum((y - mean(y)).^2)
>> sr = sum((y - polyval(c, x)).^2)
>> r = sqrt((st - sr) / st)
MATLAB polyval Function
Evaluate polynomial at the points defined by the input vector
>> y = polyval(c, x)
where x = Input vector
y = Value of polynomial evaluated at x
c = vector of coefficient in descending order
Y = c(1)*xn + c(2)*x(n-1) + ... + c(n)*x + c(n+1)
Example: y = 1.86071x2 + 2.35929x + 2.47857

>> c = [1.86071 2.35929 2.47857]
Polynomial Interpolation
70
60
50
40
y
30
20
10
0
0 1 2 3 4 5
x
>> y2 = polyval(c,x)
>> plot(x, y, ’o’, x, y2)
Error Bounds
By passing an optional second output parameter from polyfit as an input

to polyval.
>> [c,s] = polyfit(x,y,2)
>> [y2,delta] = polyval(c,x,s)
>> plot(x,y,'o',x,y2,'g-',x,y2+2*delta,'r:',x,y2-2*delta,'r:')
Interval of ±2∆ = 95% confidence interval
Linear Regression Example:
xi yi
1 0.5
2 2.5
3 2.0
4 4.0
5 3.5
6 6.0
7 5.5
Multiple Linear Regression
y = c0 + c1x1 + c2x2 + . . . + cpxp
Example case: two independent variables y = c0 + c1x1 + c2x2
Sum of squares of the residual: S r = ∑ ( yi − c0 − c1 x1i − c2 x2 i ) 2
Differentiate with respect to unknowns:

∂S r
= −2 ∑( yi − c0 − c1 x1i − c2 x2 i )
∂c0
∂S r
= −2 ∑ x1i ( yi − c0 − c1 x1i − c2 x2i )
∂c1
∂S r
= −2 ∑ x2i ( yi − c0 − c1 x1i − c2 x2i )
∂c2
Setting partial derivatives = 0 and expressing result in matrix form:
 n ∑ x1i ∑ x2i  c0   ∑ yi 

∑ x    
 1i ∑ x12i ∑ x1i x2i   c1  =  ∑ x1i yi 
 ∑ x2i ∑ x1i x2i ∑ x22i  c2  ∑ x2i yi 
Example:
x1 x2 y  6 16.5 14  c0   54 
16.5    
 76.25 48  c1  = 243.5
0 0 5
2 1 10  14 48 54  c2   100 
2.5 2 9
1 3 0
c0 = 5
4 6 3 c1 = 4
7 2 27
c2 = −3
Multivariate Fit in MATLAB
c0 + c1x11 + c2x12 + . . . + cpx1p = y1
c0 + c1x21 + c2x22 + . . . + cpx2p = y2
.
.
.
c0 + c1xm1 + c2xm2 + . . . + cpxmp = ym
Overdetermined system of equations: A c = y
 x11 x12 L x1 p 1  c0   y1 
x x22 L x2 p 1 c  y 
A= , c =   , and y =  2 
21 1
 M M O M M M M 
     
 xm1 xm 2 L xmp 1  p 
c  ym 
Fit norm >> c = (A’*A)\(A’*y)
Fit QR >> c = A\y
Example:
x1 x2 y
0 0 5
2 1 10
2.5 2 9
1 3 0
4 6 3
7 2 27
>> x1=[0 2 2.5 1 4 7]';

>> x2=[0 1 2 3 6 2]';
>> y=[5 10 9 0 3 27]';
>> A=[x1 x2 ones(size(x1))];
>> c=A\y
Lecture 7 Curve Fitting
< Linear Regression

< Polynomial Regression
< Multiple Linear Regression
LINEAR REGRESSION
We want to find the curve that will fit the data.
Candidate lines for curve fit

y = α x +β
No exact solution but many approximated solutions

Error Between Model and Observation
y Observation: [ xi yi ]
Model: y = α x + β
Error: ei = yi – α xi – β
x
Criteria for a “Best” Fit
Find the BEST line which minimize the sum of error for all data
BEST line with error minimized ?
The problem is how to minimize the error.
We can use the error

defined as:
ei = y i − yˆ
Where
ŷ = α x i + β However, the errors can
cancel one another and
still be wrong.
ERROR Definition
To avoid ± signs cancellation, the

error may be defined as:
ei = y i − yˆ
But, the error minimization is going to have problems.
The solution is the minimization of the sum of squares.
S = ∑ (ei )
2
This will give a least square solution.
Least-Square Fit of a Straight Line
Minimize sum of the square of the errors

n n
S r = ∑ e = ∑ (y i − β − αx i )
2 2
i
i =1 i =1
Differentiate with respect to each coefficient:

∂S r
= −2 ∑( y i − β − αx i )
∂β
∂S r
= −2 ∑[( y i − β − αx i ) x i ]
∂α
Setting derivatives = 0 :
0 = ∑ y i − ∑ β − ∑ αx i
0 = ∑ y i x i − ∑ βx i − ∑ αx i2
From Σβ = n β , express equations as set of 2 unknowns ( β , α )
nβ + α ∑ x i = ∑ y i
β ∑ x i + α ∑ x i2 = ∑ y i x i
Solve equations simultaneously:
1
∑ xi yi − ∑ xi ∑ yi S xy
α= n α=
1 S xx
∑ xi2 − ( ∑ xi )
2
β = y −α x
where y and x are the mean of y and x
1
Define: S xy = Σxi yi − Σxi Σyi
n Approximated y for any x is
1
S xx = Σxi2 − ( Σxi ) ŷ = α x + β
2
n
1
S yy = Σyi2 − ( i)
Σ
2
y
n
Example: Fit a straight line to x and y values
xi yi xi2 xi yi yi2
n =7
1 0.5 1 0.5 0.25
2 2.5 4 5.0 6.25 28
3 2.0 9 6.0 4 x = =4
4 4.0 16 16.0 16 7
5 3.5 25 17.5 12.25
6 6.0 36 36.0 36 24
7 5.5 49 38.5 30.25 y = = 3 . 4286
7
Σ 28 24 140 119.5 105
(119.5) − (28)(24) / 7
α= = 0.8393 Least-square fit:
(140) − (28) 2 / 7
y = 0 . 8393 x + 0 . 0714
β = 3.4286 − 0.8393(4) = 0.0714
How good is our fit?
Sum of the square of the errors:
S r = ∑ e = ∑ ( yi − β − α xi ) = ( S xx S yy − S xy2 ) / S xx
n n
2 2
i
i =1 i =1
n
St = ∑ ( yi − y ) = S yy
2
Sum of the square around the mean:
i =1
Sr
Standard errors of the estimation: sy / x =
n−2
St
Standard deviation: sy =
n−2
Linear regression
sy > sy/x
sy sy/x
y
St − S r S xy2
Coefficient of determination r2 = =
St S xx S yy
r2 คืออัตราสวนการแปรเปลี่ยนคา y ที่เกิดจากการเปลี่ยนคา x
For perfect fit Sr = 0 and r = r2 = 1
Example: error analysis of the linear fit y = 3.4286

α = 0.8393
( yi − y ) ( yi - β - α xi)2
2
xi yi β = 0.0714
1 0.5 8.5765 0.1687
22.7143
2 2.5 0.8622 0.5626 sy =
3 2.0 2.0408 0.3473 7−2
4 4.0 0.3265 0.3265 = 2.131
5 3.5 0.0051 0.5896
2.9911
6 6.0 6.6122 0.7972 sy / x =
7 5.5 4.2908 0.1993 7−2
= 0.773
Σ 28 24 22.7143 2.9911
St Sr
Since sy/x < sy , linear regression has merit.
22.7143 − 2.9911
r= = 0.868 = 0.932
22.7143
OR Example: error analysis of the linear fit
xi yi x i2 xi yi y i2
S xx = 140 − 282 / 7 = 28
1 0.5 1 0.5 0.25
2 2.5 4 5.0 6.25 S yy = 105 − 242 / 7 = 22.7
3 2.0 9 6.0 4
4 4.0 16 16.0 16
5 3.5 25 17.5 12.25 S xy = 119.5 − 28 × 24 / 7 = 23.5
6 6.0 36 36.0 36
7 5.5 49 38.5 30.25 Sr = (28 × 22.7 − 23.52 ) / 28
Σ 28 24 140 119.5 105 = 2.977
Since sy/x < sy , linear regression has merit. 22.7

sy = = 2.131
7−2
23.52 2.977
r =
2
= 0.869 sy / x = = 0.772
28 × 22.7 7−2
Confidence Interval (CI)

A confidence interval is an interval in which a measurement or trial
falls corresponding to a given probability.
yˆ i ± ∆
yˆ i
y y
x xi x
For CI 95%, you can be 95% confident that the two curved
confidence bands enclose the true best-fit linear regression line,
leaving a 5% chance that the true line is outside those boundaries.
A 100 (1 - α) % confidence interval for yi is given by
Confidence interval 95% → α = 0.05
1 ( xi − x ) 2
yˆ i ± tα / 2 s y / x +
n S xx
Example: to estimate y when x is 3.4 using 95% confidence interval:
xi yi yˆ = α x + β = 0.8363(3.4) + 0.0714 = 2.9148
95% Confidence → α = 0.05 → tα/2 = t0.025(df = n-2 = 5) = 2.571

1 0.5
2 2.5
1 (3.4 − 4)2
3 2.0 Interval: 2.9148 ± (2.571) (0.772) +
4 4.0 7 28
5 3.5
6 6.0 2.9148 ± 0.7832
7 5.5
T-Distribution
Probability density function of
t 0.025 t 0.025 the t distribution:
t 0.005 t 0.005
(1 + x 2 /ν ) − (ν +1) / 2
f ( x) =
t
B(0.5, 0.5ν ) ν
95% where B is the beta function and
99% ν is a positive integer
shape parameter.
The formula for the beta function is

1
B(α , β ) = ∫ t α −1 (1 − t ) β −1 dt
0
The following is the plot of the t probability density function for 4 different
values of the shape parameter.
ν = df
Degree of freedom
In fact, the t distribution with ν equal to 1 is a Cauchy distribution.

The t distribution approaches a normal distribution as ν becomes large.
The approximation is quite good for values of ν > 30.
Critical Values of t
Confidence Interval
80% 90% 95% 98% 99% 99.8%
df 0.10 0.05 0.025 0.01 0.005 0.001
1 3.078 6.314 12.706 31.821 63.657 318.313

2 1.886 2.920 4.303 6.965 9.925 22.327
3 1.638 2.353 3.182 4.541 5.841 10.215
4 1.533 2.132 2.776 3.747 4.604 7.173
5 1.476 2.015 2.571 3.365 4.032 5.893
6 1.440 1.943 2.447 3.143 3.707 5.208

7 1.415 1.895 2.365 2.998 3.499 4.782
8 1.397 1.860 2.306 2.896 3.355 4.499
9 1.383 1.833 2.262 2.821 3.250 4.296
10 1.372 1.812 2.228 2.764 3.169 4.143
11 1.363 1.796 2.201 2.718 3.106 4.024

12 1.356 1.782 2.179 2.681 3.055 3.929
13 1.350 1.771 2.160 2.650 3.012 3.852
14 1.345 1.761 2.145 2.624 2.977 3.787
15 1.341 1.753 2.131 2.602 2.947 3.733
16 1.337 1.746 2.120 2.583 2.921 3.686

17 1.333 1.740 2.110 2.567 2.898 3.646
18 1.330 1.734 2.101 2.552 2.878 3.610
19 1.328 1.729 2.093 2.539 2.861 3.579
20 1.325 1.725 2.086 2.528 2.845 3.552
Confidence Interval
80% 90% 95% 98% 99% 99.8%
df 0.10 0.05 0.025 0.01 0.005 0.001
21 1.323 1.721 2.080 2.518 2.831 3.527

22 1.321 1.717 2.074 2.508 2.819 3.505
23 1.319 1.714 2.069 2.500 2.807 3.485
24 1.318 1.711 2.064 2.492 2.797 3.467
25 1.316 1.708 2.060 2.485 2.787 3.450
26 1.315 1.706 2.056 2.479 2.779 3.435

27 1.314 1.703 2.052 2.473 2.771 3.421
28 1.313 1.701 2.048 2.467 2.763 3.408
29 1.311 1.699 2.045 2.462 2.756 3.396
30 1.310 1.697 2.042 2.457 2.750 3.385
31 1.309 1.696 2.040 2.453 2.744 3.375

32 1.309 1.694 2.037 2.449 2.738 3.365
33 1.308 1.692 2.035 2.445 2.733 3.356
34 1.307 1.691 2.032 2.441 2.728 3.348
35 1.306 1.690 2.030 2.438 2.724 3.340
36 1.306 1.688 2.028 2.434 2.719 3.333

37 1.305 1.687 2.026 2.431 2.715 3.326
38 1.304 1.686 2.024 2.429 2.712 3.319
39 1.304 1.685 2.023 2.426 2.708 3.313
40 1.303 1.684 2.021 2.423 2.704 3.307
Confidence Interval
80% 90% 95% 98% 99% 99.8%
df 0.10 0.05 0.025 0.01 0.005 0.001
41 1.303 1.683 2.020 2.421 2.701 3.301

42 1.302 1.682 2.018 2.418 2.698 3.296
43 1.302 1.681 2.017 2.416 2.695 3.291
44 1.301 1.680 2.015 2.414 2.692 3.286
45 1.301 1.679 2.014 2.412 2.690 3.281
46 1.300 1.679 2.013 2.410 2.687 3.277

47 1.300 1.678 2.012 2.408 2.685 3.273
48 1.299 1.677 2.011 2.407 2.682 3.269
49 1.299 1.677 2.010 2.405 2.680 3.265
50 1.299 1.676 2.009 2.403 2.678 3.261
51 1.298 1.675 2.008 2.402 2.676 3.258

52 1.298 1.675 2.007 2.400 2.674 3.255
53 1.298 1.674 2.006 2.399 2.672 3.251
54 1.297 1.674 2.005 2.397 2.670 3.248
55 1.297 1.673 2.004 2.396 2.668 3.245
56 1.297 1.673 2.003 2.395 2.667 3.242

57 1.297 1.672 2.002 2.394 2.665 3.239
58 1.296 1.672 2.002 2.392 2.663 3.237
59 1.296 1.671 2.001 2.391 2.662 3.234
60 1.296 1.671 2.000 2.390 2.660 3.232
Confidence Interval
80% 90% 95% 98% 99% 99.8%
df 0.10 0.05 0.025 0.01 0.005 0.001
61 1.296 1.670 2.000 2.389 2.659 3.229

62 1.295 1.670 1.999 2.388 2.657 3.227
63 1.295 1.669 1.998 2.387 2.656 3.225
64 1.295 1.669 1.998 2.386 2.655 3.223
65 1.295 1.669 1.997 2.385 2.654 3.220
66 1.295 1.668 1.997 2.384 2.652 3.218

67 1.294 1.668 1.996 2.383 2.651 3.216
68 1.294 1.668 1.995 2.382 2.650 3.214
69 1.294 1.667 1.995 2.382 2.649 3.213
70 1.294 1.667 1.994 2.381 2.648 3.211
71 1.294 1.667 1.994 2.380 2.647 3.209

72 1.293 1.666 1.993 2.379 2.646 3.207
73 1.293 1.666 1.993 2.379 2.645 3.206
74 1.293 1.666 1.993 2.378 2.644 3.204
75 1.293 1.665 1.992 2.377 2.643 3.202
76 1.293 1.665 1.992 2.376 2.642 3.201

77 1.293 1.665 1.991 2.376 2.641 3.199
78 1.292 1.665 1.991 2.375 2.640 3.198
79 1.292 1.664 1.990 2.374 2.640 3.197
80 1.292 1.664 1.990 2.374 2.639 3.195
Confidence Interval
80% 90% 95% 98% 99% 99.8%
df 0.10 0.05 0.025 0.01 0.005 0.001
81 1.292 1.664 1.990 2.373 2.638 3.194

82 1.292 1.664 1.989 2.373 2.637 3.193
83 1.292 1.663 1.989 2.372 2.636 3.191
84 1.292 1.663 1.989 2.372 2.636 3.190
85 1.292 1.663 1.988 2.371 2.635 3.189
86 1.291 1.663 1.988 2.370 2.634 3.188

87 1.291 1.663 1.988 2.370 2.634 3.187
88 1.291 1.662 1.987 2.369 2.633 3.185
89 1.291 1.662 1.987 2.369 2.632 3.184
90 1.291 1.662 1.987 2.368 2.632 3.183
91 1.291 1.662 1.986 2.368 2.631 3.182

92 1.291 1.662 1.986 2.368 2.630 3.181
93 1.291 1.661 1.986 2.367 2.630 3.180
94 1.291 1.661 1.986 2.367 2.629 3.179
95 1.291 1.661 1.985 2.366 2.629 3.178
96 1.290 1.661 1.985 2.366 2.628 3.177

97 1.290 1.661 1.985 2.365 2.627 3.176
98 1.290 1.661 1.984 2.365 2.627 3.175
99 1.290 1.660 1.984 2.365 2.626 3.175
100 1.290 1.660 1.984 2.364 2.626 3.174
∞ 1.282 1.645 1.960 2.326 2.576 3.090

Polynomial Regression
Second-order polynomial:
y = a0 + a1x + a2 x2
Sum of the squares of the residuals:

S r = ∑( y i − a 0 − a 1 x i − a 2 x i2 ) 2
Take derivative with respect to each coefficients:

∂S r
= −2 ∑( y i − a 0 − a 1 x i − a 2 x i2 )
∂a 0
∂S r
= −2 ∑ x i ( y i − a 0 − a 1 x i − a 2 x i2 )
∂a 1
∂S r
= −2 ∑ x i2 ( y i − a 0 − a 1 x i − a 2 x i2 )
∂a 2
Normal equations:
( )
n a 0 + (∑ x i )a 1 + ∑ x i2 a 2 = ∑ y i
(∑ x i )a 0 + (∑ x i2 )a + (∑ x )a
1 i
3
2 = ∑ x iy i
(∑ x )a + (∑ x
i
2
0 i
3
)a + (∑ x )a
1 i
4
2 = ∑ x i2 y i
For mth-order polynomial: y = a0 + a1x + a2 x2 + . . . + amxm
We have to solve m+1 simultaneous linear equations.

MATLAB polyfit Function
For second-order polynomial, we can define
 x12 x1 1  y1 
 2  y   c1 
, Y =  2  , C =  c2 
x x2 1
A= 2
M M M M   
 2     c3 
 xm xm 1  ym 
and show that C = ( A ' A) −1 A ' Y or C=A -1Y
Fit norm Fit QR
>> C = polyfit(x, y, n)
>> [C, S] = polyfit(x, y, n)
x = independent variable C = coeff. of polynomial in

y = dependent variable descending power
n = degree of polynomial S = data structure for polyval
function
Example: Fit a second-order polynomial to the data.
( yi − y )
2
xi yi ( yi - a0 - a1xi - a2xi2)2
0 2.1 544.44 0.14332
1 7.7 314.47 1.00286
2 13.6 140.03 1.08158
3 27.2 3.12 0.80491
4 40.9 239.22 0.61951
5 61.1 1272.11 0.09439
Σ 15 152.6 2513.39 3.74657
From the given data: m=2 ∑ xi = 15 ∑ xi4 = 979

n=6 ∑ yi = 152.6 ∑ xi yi = 585.6
x = 2.5 ∑ xi2 = 55 ∑ xi2 yi = 585.6
y = 25.433 ∑ xi3 = 225

Simultaneous linear equations
6 15 25  a0   152.6 
15    
 55 225   a1  =  585.6 
55 225 979  a2  2488.8
Solving these equation gives a0 = 2.47857, a1 = 2.35929, and

a2 = 1.86071.
Least-squares quadratic equation:
y = 2.47857 + 2.35929x + 1.86071x2

Coefficient of determination:
2513.39 − 3.74657
r= = 0.99851 = 0.99925
2513.39
Solving by MATLAB polyfit Function
>> x = [0 1 2 3 4 5];
>> y = [2.1 7.7 13.6 27.2 40.9 61.1];
>> c = polyfit(x, y, 2)
>> [c, s] = polyfit(x, y, 2)
>> st = sum((y - mean(y)).^2)
>> sr = sum((y - polyval(c, x)).^2)
>> r = sqrt((st - sr) / st)
MATLAB polyval Function
Evaluate polynomial at the points defined by the input vector
>> y = polyval(c, x)
where x = Input vector
y = Value of polynomial evaluated at x
c = vector of coefficient in descending order
Y = c(1)*xn + c(2)*x(n-1) + ... + c(n)*x + c(n+1)
Example: y = 1.86071x2 + 2.35929x + 2.47857

>> c = [1.86071 2.35929 2.47857]
70
60
50
40
y
30
20
10
0
0 1 2 3 4 5
x
>> y2 = polyval(c,x)
>> plot(x, y, ’o’, x, y2)
Error Bounds
By passing an optional second output parameter from polyfit as an input

to polyval.
Interval of ±2∆ = 95% confidence interval
Linear Regression Example:
xi yi
1 0.5
2 2.5
3 2.0
4 4.0
5 3.5
6 6.0
7 5.5
Multiple Linear Regression
y = c0 + c1x1 + c2x2 + . . . + cpxp
Example case: two independent variables y = c0 + c1x1 + c2x2
Sum of squares of the residual: S r = ∑ ( yi − c0 − c1 x1i − c2 x2 i ) 2
Differentiate with respect to unknowns:

∂S r
= −2 ∑( yi − c0 − c1 x1i − c2 x2 i )
∂c0
∂S r
= −2 ∑ x1i ( yi − c0 − c1 x1i − c2 x2i )
∂c1
∂S r
= −2 ∑ x2i ( yi − c0 − c1 x1i − c2 x2i )
∂c2
Setting partial derivatives = 0 and expressing result in matrix form:
 n ∑ x1i ∑ x2i  c0   ∑ yi 

∑ x    
 1i ∑ x12i ∑ x1i x2i   c1  =  ∑ x1i yi 
 ∑ x2i ∑ x1i x2i ∑ x22i  c2  ∑ x2i yi 
Example:
x1 x2 y  6 16.5 14  c0   54 
16.5    
 76.25 48  c1  = 243.5
0 0 5
2 1 10  14 48 54  c2   100 
2.5 2 9
1 3 0
c0 = 5
4 6 3 c1 = 4
7 2 27
c2 = −3
Multivariate Fit in MATLAB
c0 + c1x11 + c2x12 + . . . + cpx1p = y1
c0 + c1x21 + c2x22 + . . . + cpx2p = y2
.
.
.
c0 + c1xm1 + c2xm2 + . . . + cpxmp = ym
Overdetermined system of equations: A c = y
 x11 x12 L x1 p 1  c0   y1 
x x22 L x2 p 1 c  y 
A= , c =   , and y =  2 
21 1
 M M O M M M M 
     
 xm1 xm 2 L xmp 1  p 
c  ym 
Fit norm >> c = (A’*A)\(A’*y)
Fit QR >> c = A\y
Example:
x1 x2 y
0 0 5
2 1 10
2.5 2 9
1 3 0
4 6 3
7 2 27
>> x1=[0 2 2.5 1 4 7]';

>> x2=[0 1 2 3 6 2]';
>> y=[5 10 9 0 3 27]';
>> A=[x1 x2 ones(size(x1))];
>> c=A\y
Lecture 8 Interpolation
Linear Interpolation
Quadratic Interpolation
Piecewise Polynomial Interpolation
Visual Interpolation
60
40 80
20 100
km/h
0 120
Vehicle speed is approximately 49 km/h

Interpolation between data points
Consider a set of xy data collected during an experiment.

We use interpolation technique to estimate y at x where there’s no data.
y
known data
What is the
corresponding value
of y for this x ?
BASIC IDEAS
From the known data ( xi , yi ), interpolate yˆ = F ( xˆ ) for xˆ ≠ x i
x1 y1 y F(x)
M M
xi y i interpolate ŷ
xˆ → yˆ
x i +1 y i +1
M M
x
xn yn x̂
Determining coefficient a1, a2, . . . , an of basis function F(x)
F(x) = a1Φ1(x) + a2Φ2(x) + . . . + anΦn(x)
Polynomials are often used as the basis functions.
F(x) = a1 + a2 x + a3 x2 + . . . + an xn-1
Interpolation v.s. Curve Fitting
y known data
curve fit
interpolation
x
Curve fitting: fit function & data not exactly agree
Interpolation: function passes exactly through known data
Interpolation & Extrapolation

Interpolation approximate within the range of independent variable
of the given data set.
Extrapolation approximate outside the range of independent variable

of the given data set.
y
x1 x2 x
The most common way to estimate a data point between 2 points.

The function is estimated by a straight line drawn between them.
Interpolated Point
Linear, Quadratic, and Cubic Interpolations
2 points 3 points 4 points
Linear = 1st order Quadratic = 2nd order Cubic = 3rd order

f(x) = a0 + a1x f(x) = a0 + a1x + a2x2 f(x) = a0 + a1x+ a2x2 + a3x3
General formula for an (n – 1)th-order polynomial:
f(x) = a0 + a1x+ a2x2 +…+ anxn-1

Connect two data points with a straight line
y1
Using similar triangles:
F ( xˆ ) = yˆ yˆ − y 0 y − y0
= 1
xˆ − x0 x1 − x 0
y0
x0 x̂ x1
y1 − y 0
F ( xˆ ) = yˆ = y 0 + ( xˆ − x0 )
x1 − x0
Second-order polynomial interpolation using 3 data points
(x1, y1) Convenient form:

f2 ( x ) = b0 + b1( x − x0 ) + b2 ( x − x0 )( x − x1 )
(x2, y2) f2 ( x ) = a0 + a1x + a2 x 2

where a0 = b0 + b1x0 + b2 x 0 x1
(x0, y0) a1 = b1 + b2 x0 + b2 x1
a2 = b2
Substitute f2 ( x0 ) = y 0 → b0 = y 0
y1 − y 0
Substitute f2 ( x1 ) = y 1 → b1 =
x1 − x 0
y 3 − y 2 y 2 − y1
−
x3 − x 2 x 2 − x1
Substitute f2 ( x 2 ) = y 2 → b2 =
x3 − x1
EXAMPLE : Linear & Quadratic Interpolation ln(1) = 0
ln(2) = 0.6932
Estimate ln(2) by using linear & quadratic interpolation ln(4) = 1.3863
ln(6) = 1.7918
Linear interpolation from x1 = 1 to x2 = 6
1.7918 − 0
f1(2) = 0 + ( 2 − 1) = 0.3584 → ε t = 48 .3%
6 −1
Linear interpolation from x1 = 1 to x2 = 4
1.3863 − 0
f1(2) = 0 + ( 2 − 1) = 0.4621 → ε t = 33 .3%
4 −1
Quadratic interpolation from x1 = 1 to x2 = 4 and x3 = 6
1.3863 − 0
b0 = 0 , b1 = = 0.4621
4 −1
1.7918 − 1.3863
− 0.4621
b3 = 6 − 4 = −0.05187
6 −1
Substitute b0, b1 and b2 into equation
f2(x) = 0 + 0.4621(x - 1) - 0.05187(x - 1)(x - 4)
which can be evaluated at x = 2 for
f2(2) = 0 + 0.4621(2 - 1) - 0.05187(2 - 1)(2 - 4)
f2(2) = 0.5658 → εt = 18.4%
f2(x) = – 0.05187x2 + 0.7251x – 0.6696

2
MATLAB :
>> x = 1:0.01:7;
>> y = log(x);
>> y2= -0.05187*x.^2+...
0.7251*x-0.6696; 1
>> plot(x,y,x,y2)
1 2 3 4 5 6 7
Finding Pn-1(x) of degree n-1 that passes through n known data pairs
Linear = 1 : 2 pt.
Pn-1(x) = c1xn-1 + c2xn-2 + . . . + cn-1x + cn Quadratic = 2 : 3 pt.
Cubic = 3 : 4 pt.
Vandermonde Systems n pairs of (x, y)

n equations
n unknowns
Polynomial pass through each of data points
Example: Construct a quadratic interpolating function
y = c1x2 + c2x + c3
that pass through (x, y) support points (-2, -2), (-1, 1), and (2, -1)
Substitute known points into equation:

-2 = c1 (-2)2 + c2 (-2) + c3
1 = c1 (-1)2 + c2 (-1) + c3
-1 = c1 (2)2 + c2 (2) + c3
Rewritten in matrix form:
 4 −2 1  c1   −2 
1 −1 1  c2  =  1 

 2 2 1  c3   −1
 x12 x1 1  c1   y1 
 2 
Vandermonde Matrix
 x2 x2 1 c2  =  y2 
 x32 x3 1  c3   y3 

MATLAB:
>> x = [-2 -1 2]’;
>> A = [x.^2 x ones(size(x))];
or use the built-in vander function
>> A = vander([-2 -1 2]);
>> y = [-2 1 -1]’;
>> c = A\y
c =
-0.9167
0.2500
2.1667
MATLAB : polyfit and polyval Functions
Find y = c1x2 + c2x + c3 that pass through (x, y) support points

(-2, -2), (-1, 1), and (2, -1).
>> x = [-2 -1 2];

>> y = [-2 1 -1];
>> c = polyfit(x,y,2)
c =
-0.9167 0.2500 2.1667
We can then use the polyval function to perform an interpolation as in

Ex. To interpolate y at x = 1
>> polyval(c,1)
ans =
1.5000
Also, we can use the polyval function to plot the result as in
>> xhat = -3:0.1:3;

>> yhat = polyval(c,xhat);
>> plot(xhat,yhat,x,y,'o')
-2
-4
-6
-8
-3 -2 -1 0 1 2 3
Polynomials Wiggle
2nd-order
3rd-order
y
4th-order
5th-order
xi 1 2 3 4 5 6 7 8 9 10
yi 3.5 3.0 2.5 2.0 1.5 -2.4 -2.8 -3.2 -3.6 -4.0
MATLAB’s Command Lines to Demonstrate Polynomials Wiggle
>> x = [1 2 3 4 5 6 7 8 9 10];
>> y = [3.5 3.0 2.5 2.0 1.5 -2.4 -2.8 -3.2 -3.6 -4.0];
>> x0 = 1:0.1:10;
>> y2 = polyval(polyfit(x(4:6), y(4:6), 2), x0);
>> axis([0 10 -5 5])
>> plot(x, y, ‘o’)
>> hold on
>> plot(x0, y2)
>> plot(x0, y3)
>> plot(x0, y4)
>> plot(x0, y5)
Piecewise Polynomial Interpolation

Using a set of lower degree interpolants on subinterval of
the whole domain
= breakpoint or knot
y
x
Piecewise-linear interpolation
Piecewise-quadratic interpolation
Piecewise-cubic interpolation
f’(x) and f’’(x) continuous at breakpoint = cubic spline
MATLAB’s Built-in Interpolation Functions
Funnction Description
interp1 1-D interpolation with piecewise polynomials.
interp2 2-D interpolation with nearest neighbor, bilinear,

or bicubic interpolants.
interp3 3-D interpolation with nearest neighbor, bilinear,

or bicubic interpolants.
interpft 1-D interpolation of uniformly spaced data using
Fourier Series (FFT).
interpn n-D extension of methods used by interp3.
spline 1-D interpolation with cubic-splines using

not-a-knot or fixed-slope end conditions.
interp1 Built-in Function

1-D interpolation with one of the following 4 methods:
1. Nearest-neighbor uses piecewise-constant function.

Interpolant discontinue at midpoint between knots
2. Linear interpolation uses piecewise-linear polynomials.
3. Cubic interpolation uses piecewise-cubic polynomials.

Interpolant and f’ (x) are continuous.
4. Spline interpolation uses cubic splines. This option performs

the same interpolation as built-in function spline.
How to use interp1 ?
>> yhat = interp1(y, xhat)

>> yhat = interp1(x, y, xhat)
>> yhat = interp1(x, y, xhat, method)
where y = tabulated values to be interpolated.

x = independent values. If not given x=1:length(y).
xhat = values at which interpolant be evaluated.
method = ‘nearest’, ‘linear’, ‘cubic’ or ‘spline’
Example: Interpolation with piecewise-polynomials

>> x = [1 2 3 4 5 6 7 8 9 10];
>> y = [3.5 3.0 2.5 2.0 1.5 -2.4 -2.8 -3.2 -3.6 -4.0];
>> xhat = 1:0.1:10; % eval interpolant at xhat
>> yn = interp1(x, y, xhat, ‘nearest’);
>> plot(x, y, ‘o’, xhat, yn); pause;
>> yl = interp1(x, y, xhat, ‘linear’);
>> plot(x, y, ‘o’, xhat, yl); pause;
>> yc = interp1(x, y, xhat, ‘cubic’);
>> plot(x, y, ‘o’, xhat, yc); pause;
>> ys = interp1(x, y, xhat, ‘spline’);
or >> ys = spline(x, y, xhat);
>> plot(x, y, ‘o’, xhat, ys);
EXAMPLE : Linear interpolation by interp1 function
Estimate the value of y when x is equal to 3.5
x 0 1 2 3 4 5
y 15 10 9 6 2 0
>> x = 0:5; 15
>> y = [15 10 9 6 2 0];

>> interp1(x,y,3.5) 10
ans =
4 5
Plot Graph:
>> xhat = 0:0.2:5; 0

0 1 2 3 4 5
>> yhat = interp1(x,y,xhat);

>> plot(x,y,'bo-',xhat,yhat,'rx')
Lecture 9 Numerical Integration
- Basic Ideas
- Symbolic vs.
vs. Numerical Integration
- Trapezoid Rule
- Simpson’
Simpson’s Rule
- MATLAB quad and quad8
quad8 Functions
SURAN AR EE INSTITUTE OF ENGINEERING
BASIC IDEAS
b
I = ∫ f ( x ) dx = Area under curve f ( x )
a
Node
f (x)
Trapezoidal
region
x
a b
Approximated by piecewise-linear:
Area = Sum of trapezoidal regions
Symbolic Math Toolbox
Symbolic Math Toolboxes incorporate symbolic computation into the numeric
environment of MATLAB.
Symbolic Integration with MATLAB
>> I = int(f) % Indefinite integral

>> I = int(f, v) % Designating integration variable
>> I = int(f, a, b) % Definite integral on close interval
>> I = int(f, v, a, b)
where f = symbolic expression
Variables must be define as symbolic by sym or syms

>> x = sym(‘x’), y = sym(‘y’), z = sym(‘z’)
or >> syms x y z
Example : I = ∫ ( x 3 − c ) dx
a
>> syms x a b c
>> I = int(x^3-c,x,a,b)
I =
1/4*b^4-c*b-1/4*a^4+c*a
What is Integration?
The integral is equivalent to the area under the curve.
f(x)
I = ∫ f ( x ) dx
a I
x
a b
Examples of how integration is used to evaluate areas
Net force due to

wind blowing
against the building
Cross-sectional area of a river
Newton-Cotes Formulas
Replace a complicated function with a polynomial that is easy to integrate
b b
I = ∫ f ( x ) dx ≅ ∫ fn ( x ) dx
a a
where fn ( x ) = a0 + a1x + L + an x n −1 + an +1x n
f(x) f(x)
x x
a b a b
Approximate area by a straight line Approximate area by a parabola

Trapezoidal Rule
f(x) 1st degree polynomial = Linear line

f(b)
f (b ) − f (a )
f1( x ) = f (a) + ( x − a)
f1(x) b−a
f (a ) + f ( b )
b
f(a) I = ∫ f1( x ) dx = (b − a )
a
2
x
a b
Area of trapezoid = width x average height
width = b – a = h
f (a ) + f ( b )
average height =
2
Composite Trapezoidal Rule

Improve accuracy by dividing interval into subintervals
f(x) f5
f4
f3
f2
f1
x1 x2 x3 x4 x5
x5 x2 x3 x4 x5
∫ f ( x ) dx = ∫ f ( x ) dx + ∫ f ( x ) dx + ∫ f ( x ) dx
x1 x1 x2 x3
+ ∫ f ( x ) dx
x4
f(x)
There are n segment with equal width:
b–a
h =
n
x2 x3 xn
I = ∫ f ( x ) dx + ∫ f ( x ) dx + L + ∫ f ( x ) dx
x1 x2 x n −1
f ( x1 ) + f ( x 2 ) f ( x2 ) + f ( x 3 )
I =h +h +L
2 2
f ( x n−1 ) + f ( x n )
x1 x2 x3 x4 x5 x6 +h
2
x1 = a b–a xn = b
h=
n
f +f f +f f +f h  n −1

I = h 1 2 + h 2 3 + L + h n −1 n I = f1 + 2 ∑ fi + fn 
2 2 2 2  i =2 
f(x)
f4 f5
MEAN
f2 f3
f6
f1
x1 x2 x3 x4 x5 x6
 n −1

b–a
f
1

+ 2 ∑
i =2
f i + f n

Substitute h = I = (b − a )
n 2n
width average height
Example 9-1 Use Trapezoidal rule to numerically integrate
f ( x ) = 0.2 + 25 x − 200 x 2 + 675 x 3 − 900 x 4 + 400 x 5
from a = 0 to b = 0.8. Note that the exact value of the integral can be determined
analytically to be 1.640533.
or using MATLAB’s Symbolic Math Toolbox :

>> x = sym(‘x’)
>> I=int(0.2+25*x-200*x^2+675*x^3-900*x^4+400*x^5,x,0,0.8)
I = 3076/1875 = 1.6405
Single Trapeziod: f(0) = 0.2 and f(0.8) = 0.232 f(x)
f (a ) + f (b )
I = (b − a)
2
0.2 + 0.232
= (0.8 − 0) = 0.1728 Error
2
1.6405 − 0.1728
εt = × 100% = 89 .5% Integral estimate
1.6405 0 0.8
2 Trapeziods: n = 2 (h = 0.4) :
f(0) = 0.2 f(0.4) = 2.456 f(0.8) = 0.232
 n −1

f1 + 2 ∑ fi + fn 
I = (b − a)  i =2 
2n
0.2 + 2(2.456 ) + 0.232

I = (0.8 ) = 1.0688
4
1.6405 − 1.0688
εt = × 100 % = 34.9%
1.6405
MATLAB’s trapz function
Trapezoidal numerical integration
>> z = trapz(y) % Integral of y with unit spacing

>> z = trapz(x, y) % Integral of y with respect to x
90o
Example: ∫ sin x dx
0o
>> angle = 0:15:90;
>> x = (pi*angle/180);
>> y = sin(x);
>> z = trapz(x,y)
>> z =
0.9943
Example 9-2 Use MATLAB’s trapz function to numerically integrate

f ( x ) = 0.2 + 25 x − 200 x 2 + 675 x 3 − 900 x 4 + 400 x 5
For n = 2 : >> x = linspace(0,0.8,3);

>> fx = 0.2+25*x-200*x.^2+675*x.^3-900*x.^4+400*x.^5;
>> I = trapz(x,fx)
I =
1.0688
n I εt
2 1.0688 34.9
3 1.3639 16.5
4 1.4848 9.49
5 1.5399 6.13
6 1.5703 4.28
Simpson’s Rules
Using higher-order polynomials to connect the points
f(x) f(x)
x x
Simpson’s 1/3 rule Simpson’s 3/8 rule
Quadratic connecting 3 points Cubic connecting 4 points
Simpson’s 1/3 Rules

Quadratic connecting 3 points x2 x2
f(x)
f(x1) f2(x) I= ∫ f ( x)dx ≅ ∫ f 2 ( x)dx
f(x2) x0 x0
f(x0)
( x − x1 )( x − x2 )
f 2 ( x) = f ( x0 )
( x0 − x1 )( x0 − x2 )
( x − x0 )( x − x2 )
+ f ( x1 )
( x1 − x0 )( x1 − x2 )
( x − x0 )( x − x1 )
x0 x1 x2 + f ( x2 )
( x2 − x0 )( x2 − x1 )
h x −x
I≅ [ f ( x0 ) + 4 f ( x1 ) + f ( x2 )] , h = 2 0
3 2
Example 9-3 Use Simpson’s 1/3 rule to numerically integrate
f ( x ) = 0.2 + 25 x − 200 x 2 + 675 x 3 − 900 x 4 + 400 x 5
Solution: n = 2 (h = 0.4) :
f(0) = 0.2 f(0.4) = 2.456 f(0.8) = 0.232
0.4
I = (0.2 + 4( 2.456) + 0.232) = 1.3675
3
1.6405 − 1.3675
εt = × 100 % = 16.6%
1.6405
Composite Simpson’s 1/3 Rules

b−a
x2 x4 xn
I = ∫ f ( x ) dx + ∫ f ( x ) dx + L + ∫ f ( x ) dx where h = , n = even number

x0 x2 xn − 2
n
I=
h
[f0 + 4f1 + f2 ] + h [f2 + 4f3 + f4 ] + L + h [fn−2 + 4fn −1 + fn ]
3 3 3
1 4 1 1 4 1
1 4 1 1 4 1 1 4 1
4 2
4
1 2
4 1
2 4
4 2
a b
(b − a)  n −1 n −2 
I=
3n 
 0
f ( x ) + 4 ∑ i j =∑
f ( x ) + 2 f ( x j ) + f ( x n 
)
i =1,3,5 2, 4, 6 
Example 9-4 Use composite Simpson’s 1/3 rule with n = 4 to numerically integrate
f ( x ) = 0.2 + 25 x − 200 x 2 + 675 x 3 − 900 x 4 + 400 x 5
Solution: n = 4 (h = 0.2) :
f(0) = 0.2 f(0.2) = 1.288 f(0.4) = 2.456 f(0.6) = 3.464 f(0.8) = 0.232
(b − a )  n −1 n−2 
I=
3n 
 0
f ( x ) + 4 ∑ i j =∑
f ( x ) + 2 f ( x j ) + f ( x n 
)
i =1,3,5 2, 4,6 
0.8
I = ( 0.2 + 4(1.288 + 3.464) + 2( 2.456) + 0.232) = 1.624
3( 4)
1.6405 − 1.624
εt = × 100 % = 1.04%
1.6405
Simpson’s 3/8 Rules

Cubic connecting 4 points b b
I = ∫ f ( x)dx ≅ ∫ f 2 ( x)dx
a a
f(x)
3h
I≅ [ f ( x0 ) + 3 f ( x1 ) + 3 f ( x2 ) + f ( x3 )]
8
h = ( x3 − x0 ) / 3
x
x0 x1 x2 x3
a b
(b − a)
or… I = [f ( x0 ) + 3 f ( x1) + 3 f ( x2 ) + f ( x3 )]
8
Truncation Errors
Interval
Method Et width Et
1 3 (b − a )3
Trapezoid h f ′′(ξ ) h =b−a f ′′(ξ )
12 12
1 5 (4) b−a (b − a )5 (4)
Simpson’s 1/3 h f (ξ ) h= f (ξ )
90 2 2880
3 5 (4) b−a (b − a )5 (4)

Simpson’s 3/8 h f (ξ ) h= f (ξ )
80 3 6480
Simpson’s 3/8 rule is used when number of segments is odd.
Apply Simpson’s 1/3 and 3/8 Rules

To handle multiple application with odd number of intervals
f(x)
x
1/3 rule 3/8 rule
Example 9-5 Use Simpson’s 1/3 + 3/8 rule with n = 5 to numerically integrate
f ( x ) = 0.2 + 25 x − 200 x 2 + 675 x 3 − 900 x 4 + 400 x 5
f(x)
Solution: n = 5 (h = 0.16) :
f(0) = 0.2 f(0.16) = 1.297
f(0.32) = 1.743 f(0.48) = 3.186
f(0.64) = 3.182 f(0.8) = 0.232
For the first two segments use Simpson’s 1/3 :
0.16
I = (0.2 + 4(1.297) + 1.743) = 0.380
3
For the last three segments use Simpson’s 3/8 : x
0 0.16 0.32 0.48 0.64 0.8
0.48
I = (1.743 + 3(3.186 + 3.182) + 0.232)
8
1/3 rule 3/8 rule
= 1.265
1.6405 − 1.645
Total integral : I = 0.380 + 1.265 = 1.645 εt = × 100 % = 0.274 %
1.6405
MATLAB’s quad and quad8 Functions
quad : low order method, quad8: high order method

>> q = quad(’f',a,b)
Approximates the integral of f(x) from a to b within a relative
error of 1e-3 using an adaptive recursive Simpson's rule.
’f' is a string containing the name of the function.
>> quad(‘sin’, 0, pi/2)
Function f must return a vector of output values if given

a vector of input values.
0.8
Example: ∫
0
0.2 + 25x − 200 x 2 + 675x3 − 900 x 4 + 400 x5 dx
poly5.m
function p = poly5(x)
p=0.2+25*x-200*x.^2+675*x.^3-900*x.^4+400*x.^5;
>> quad(‘poly5’, 0, 0.8)
ans =
1.6405
Example: Computing the Length of a Curve
x(t ) = sin(2t ), y (t ) = cos(t ), z (t ) = t
where t ∈ [ 0, 3π ]
>> t = 0:0.1:3*pi;
>> plot3(sin(2*t), cos(t), t)
10
0
1
0.5 1
0 0.5
0
-0.5
- 0.5
-1 -1
3π
∫
Length of the curve:
4 cos(2t )2 + sin(t )2 + 1 dt
Norm of derivative
0
hcurve.m
function f = hcurve(t)
f = sqrt(4*cos(2*t).^2 + sin(t).^2 + 1);
>> len = quad(‘hcurve’,0,3*pi)
len =
17.2220
Example: Double Integration

y max x max
∫ ∫
y min x min
f ( x, y ) dx dy
For example: f (x, y) = y sin(x) + x cos(y)

integrnd.m
function out = integrnd(x,y)

out = y*sin(x) + x*cos(y);
>> xmin = pi; xmax = 2*pi;

>> ymin = 0; ymax = pi;
>> result = dblquad(‘integrnd’,xmin,xmax,ymin,ymax);
result =
-9.8698
Lecture 10 Ordinary Differential Equations
- Basic Ideas
- Euler’s Method
- Higher Order One-step Methods
- Predictor-Corrector Approach
- Runge-Kutta Methods
- Adaptive Stepsize Algorithms
Ordinary Differential Equations

Rate of change of one variable with respect to another.
dy
= f (t , y )
dt
where t = independent variable
y = dependent variable
Initial Condition y(t0) = y0 x

k
Example: Mass-spring system
m
c
d 2x dx
ma + cv + kx = m + c + kx = 0
dt dt
EULER’S METHOD
(t − t0 ) 2
From Taylor series: y (t ) = y (t0 ) + (t − t0 ) y ′(t0 ) + y ′′(t0 ) +L
2!
Retaining only first derivative: y (t ) = y0 + h f (t0 , y0 )
where h = (t - t0), y0 = y(t0), and f(t0, y0) = y’(t0)
y Predict
error
True
y1 = y0 + h f (t0 , y0 )
y2 = y1 + h f (t1 , y1 )
h
yi = yi −1 + h f (ti −1 , yi −1 )
t0 t1 t
New value = Old value + slope x step size
Example: Manual Calculation with Euler’s Method

dy
= t − 2 y, y (0) = 1
dt
Exact solution: y = ( 2t − 1 + 5e −2t )

1
4
Set h = 0.2
Euler
ti f (ti-1, yi-1) yi=yi-1+h f (ti-1, yi-1) Exact Error
0.0 NA initial cond. = 1.0 1.0000 0

0.2 0-2(1) = -2.0 1.0+0.2(-2.0) = 0.6 0.6879 -0.0879
0.4 0.2-2(0.6) = -1.0 0.6+0.2(-1.0) = 0.4 0.5117 -0.1117
0.6 0.4-2(0.4) = -0.4 0.4+0.2(-0.4) = 0.32 0.4265 -0.1065
Implementing Euler’s Method
odeEuler.m
function [t,y] = odeEuler(diffeq,tn,h,y0)
% Input: diffeq = (string) name of m-file that
% evaluate right hand side of ODE
% tn = stopping value of independent variable
% h = stepsize
% y0 = initial condition at t=0
t = (0:h:tn)’;
n = length(t);
y = y0*ones(n,1);
for i=2:n
y(i) = y(i-1)+h*feval(diffeq,t(i-1),y(i-1));
end
dy
Example: = t − 2 y, y (0) = 1, h = 0.2
dt
rhs1.m
function dydt = rhs1(t,y)
dydt = t-2*y;
>> [t,y] = odeEuler(‘rhs1’,0.6,0.2,1.0)

t =
0
0.2000
0.4000
0.6000
y =
1.0000
0.6000
0.4000
0.3200
Exact Solution :
>> y0 = (1/4)*(2*t-ones(size(t))+5*exp(-2*t))
y0 =
1.0000
0.6879
0.5117
0.4265
>> t0=0:0.01:0.6;
>> y0 = (1/4)*(2*t0-ones(size(t0))+5*exp(-2*t0));
>> plot(t0,y0,t,y)
Improvements of Euler’s Method

Heun’s Method: estimate slope from derivatives at the beginning
and at the end of the interval.
y Euler:
y0i = yi-1 + hk1 slope:
k2 = f(ti, y0i)
slope:
k1 = f(ti-1, yi-1)
h
t
ti-1 ti
k1 + k 2 f (t i −1, y i −1 ) + f (t i , y i0 )
Average slope = =
2 2
y
 k + k2 
y i = y i −1 + h  1 
 2 
Use slope:
(k1 + k2)/2
h
t
ti-1 ti
Heun’s Method
dy k1 = f (ti −1 , yi −1 )
= f (t , y ) y i0
dt
k2 = f (ti −1 + h, yi −1 + hk1 )
Euler
k +k
yi = yi −1 + h 1 2
2
Heun
True
yi-1
ti-1 ti
Predictor-Corrector Approach
Heun’s Method:
k1 = f (ti −1 , yi −1 )
Euler’s Method:
k2 = f (ti −1 + h, yi −1 + hk1 ) yi = yi −1 + h f (ti −1 , yi −1 )
k1 + k2
yi = yi −1 + h
2
Predictor: yi0 = yi −1 + h f (ti −1 , yi −1 )
f (ti −1 , yi −1 ) + f (ti , yi0 )

Corrector: yi = yi −1 + h
2
Iterating the Corrector of Heun’s Method

To improve the estimation
f (ti −1 , yi −1 ) + f (ti , y ) 0
yi yi −1 + h i
2
Example: Use Heun’s method to integrate
y ′ = 4e 0.8 x − 0.5 y , y (0) = 2

From x = 0 to 4, step size = 1
Analytical solution: y=
1.3
(
4 0.8 x −0.5 x
e −e ) + 2e −0.5 x
Predictor, yi0 = yi −1 + h f (ti −1 , yi −1 )
y10 = 2 + 1 (4e 0 − 0.5(2)) = 5
f (ti −1 , yi −1 ) + f (ti , yi0 )

Corrector, yi = yi −1 + h
2
(4e 0 − 0.5(2)) + (4e 0.8(1) − 0.5(5))
y1 = 2 + 1 = 6.7011
2
True value: y =
1.3
(
4 0.8(1) −0.5(1)
e −e ) + 2e −0.5(1) = 6.1946
6.1946 − 6.7011
Relative error, Et = ×100% = 8.18%
6.1946
(4e 0 − 0.5(2)) + (4e 0.8(1) − 0.5(6.7011))

2 nd
Corrector, y1 = 2 + 1
2
= 6.2758, Et = 1.31%
(4e 0 − 0.5(2)) + (4e 0.8(1) − 0.5(6.2758))

rd
3 Corrector, y1 = 2 + 1
2
= 6.3821, Et = 3.03%
Error may increases for large step sizes.
Iteration using up-arrow in MATLAB
>> y = 2+1*(4*exp(0)-0.5*2) % Predictor of y1

y = 5
>> y = 2+(1/2)*((4*exp(0)-0.5*2)+(4*exp(0.8*1)-0.5*y))
y = 6.7011 % 1st Corrector of y1
Press and Enter
y = 6.7011 % 2nd Corrector of y1

y = 6.2758 % 3rd Corrector of y1
y = 6.3821 % 4th Corrector of y1
y = 6.3555 % 5th Corrector of y1
y = 6.3609 % until no change
MATLAB’s Implementation
func1.m
function dydx = func1(x,y)
dydx = 4*exp(0.8*x)-0.5*y;
odeHeun.m
function [x,y] = odeHeun(diffeq,xn,h,y0,iter)
% Input: iter = number of corrector iterations
x = (0:h:xn)’;
n = length(x);
y = y0*ones(n,1);
for i=2:n
y(i) = y(i-1)+h*feval(diffeq,x(i-1),y(i-1));
for j=1:iter
y(i) = y(i-1)+h*(feval(diffeq,x(i-1),y(i-1)) ...
+ feval(diffeq,x(i),y(i)))/2;
end
end
For x = 0 to 4, step size = 1, y(0) = 2, Iteration = 15
>> [x,y] = odeHeun(‘func1’,4,1,2,15)

x =
0
1
2
3
4
y = y_true = Et =
2.0000 2.0000 0.00
6.3609 6.1946 2.68
15.3022 14.8439 3.09
34.7433 33.6772 3.17
77.7351 75.3390 3.18
Compare with Euler’s Method:
>> [x,y] = odeEuler(‘func1’,4,1,2)

x =
0
1
2
3
4
y = Et =
2.0000 0.00
5.0000 19.28
11.4022 23.19
25.5132 24.24
56.8493 24.54
Comparison of True Solution with Euler’s and Heun’s Method
>> xtrue = (0:0.1:4)’;

>> ytrue = (4/1.3)*(exp(0.8*xtrue)-exp(-0.5*xtrue))...
+ 2*exp(-0.5*xtrue);
>> [xeuler,yeuler] = odeEuler(‘func1’,4,1,2);
>> [xheun,yheun] = odeHeun(‘func1’,4,1,2,15);
>> plot(xtrue,ytrue,xeuler,yeuler,’o’,xheun,yheun,’+’)
80
True
Euler
60
Heun 15
40
y
20
0
0 1 2 3 4
x
RUNGE-KUTTA (RK) METHODS

General Formula using Weighted Average of slopes
m
yi = yi −1 + ∑ γ l kl
l =1
For Heun’s method γ1 = γ2 = 0.5
m
The weights must satisfy ∑γ
l =1
l =1
Second-Order RK (Ralston’s) Method:

1 2 
yi = yi −1 + h  k1 + k 2 
3 3 
k1 = f ( xi −1 , yi −1 )
 3 3 
k2 = f  xi −1 + h, yi −1 + k1h 
 4 4 
Third-Order Runge-Kutta Methods
h
yi = yi −1 + ( k1 + 4k2 + k3 )
6
where
k1 = f ( xi −1 , yi −1 )
 1 1 
k2 = f  xi −1 + h, yi −1 + k1h 
 2 2 
k3 = f ( xi −1 + h, yi −1 − k1h + 2k 2 h )
Fouth-Order Runge-Kutta Methods

The most popular RK methods
h
Classical 4th-order RK: yi = yi −1 + ( k1 + 2k2 + 2k3 + k4 )
6
where
k1 = f ( xi −1 , yi −1 )
 1 1 
k2 = f  xi −1 + h, yi −1 + k1h 
 2 2 
 1 1 
k3 = f  xi −1 + h, yi −1 + k2 h 
 2 2 
k4 = f ( xi −1 + h, yi −1 + k3 h )
Example: Use 4th-order RK method to integrate
y ′ = 4e 0.8 x − 0.5 y , y (0) = 2

From x = 0 to 4, step size = 1
Step 1: x0 = 0, y0 = 2
k1 = 4e0 − 0.5(2) = 3
1
k2 = 4e0.8(0.5) − 0.5(2 + × 3 × 1) = 4.2173
2
1
k3 = 4e0.8(0.5) − 0.5(2 + × 4.2173 ×1) = 3.9130
2
k4 = 4e0.8(1) − 0.5(2 + 3.9130 ×1) = 5.9457
1
y1 = 2 + (3 + 2 × 4.2173 + 2 × 3.9130 + 5.9457 )
6
= 6.2011
True value: y =
1.3
(
4 0.8(1) −0.5(1)
e −e ) + 2e −0.5(1) = 6.1946
6.1946 − 6.2010
Relative error, Et = ×100% = 0.1038%
6.1946
MATLAB’s Implementation
odeRK4.m
function [x,y] = odeRK4(diffeq,xn,h,y0)
x = (0:h:xn)’;
n = length(x);
y = y0*ones(n,1);
for i=2:n
k1 = feval(diffeq,x(i-1),y(i-1));
k2 = feval(diffeq,x(i-1)+h/2,y(i-1)+k1*h/2);
k3 = feval(diffeq,x(i-1)+h/2,y(i-1)+k2*h/2);
k4 = feval(diffeq,x(i-1)+h,y(i-1)+k3*h);
y(i) = y(i-1)+h/6*(k1+2*k2+2*k3+k4);
end
For x = 0 to 4, step size = 1, y(0) = 2, Iteration = 15
>> [x,y] = odeRK4(‘func1’,4,1,2)

x =
0
1
2
3
4
y = y_true = Et(%) =
2.0000 2.0000 0.00
6.2010 6.1946 0.1038
14.8625 14.8439 0.1250
33.7213 33.6772 0.1309
75.4392 75.3390 0.1328
Adaptive Stepsize Algorithms
MATLAB’s ode23 and ode45 Routines
ode23 use second- and third-order RK simultaneously
ode45 use fourth- and fifth-order RK simultaneously
Example: y ′ = 4e 0.8 x − 0.5 y , y (0) = 2 func1.m

For x = 0 to 4, y(0) = 2
>> x0 = 0; xn = 4; y0 = 2;
>> [x45,y45] = ode45(‘func1’,[x0,xn],y0);
>> xtrue = (0:0.1:4)’;
>> ytrue = (4/1.3)*(exp(0.8*xtrue)...
- exp(-0.5*xtrue))+ 2*exp(-0.5*xtrue);
>> plot(xtrue,ytrue,x45,y45)
Lecture 11 OPTIMIZATION
- Basic Ideas
- One-dimensional Unconstrained Optimization
- Golden-Section Search
- Quadratic Interpolation
- Newton’s Method
- Multidimensional Unconstrained Optimization
- Direct Methods
- Gradient Methods
Suranaree
UNIVERSITY OF TECHNOLOGY Mongkol JIRAVACHARADET
NAKORNRATCHSIMA School of Civil Engineering
BASIC IDEAS
f (x) f ′( x) = 0
Maximum
f ′′( x) < 0
f ( x) = 0
Root Root
0 x
Root
f ′( x) = 0
Minimum
f ′′( x) > 0
Examples of Optimization Problems in Engineering
- Design aircraft for minimum weight and maximum strength.

- Optimal trajectories of space vehicles.
- Design civil engineering structures for minimum cost.
- Predict structural behavior by minimizing potential energy.
- Material-cutting strategy for minimum cost.
- Design pump and heat transfer equipment for maximum efficiency.
- Shortest route of salesperson visiting various cities during one sales trip.
- Optimal pipeline network.
- Optimal planning and scheduling.
- Statistical analysis and models with minimum error.
- Minimize waiting and idling times.
One-dimensional Unconstrained Optimization
Global & Local Optima
f (x) Global
maximum
Local
maximum
0 x
Local
Global minimum
minimum
GOLDEN-SECTION SEARCH
- Similar to Bisection root finding method
- Define interval that contains single answer
- Need 3 points to detect maximum or minimum
- Pick 4th points then choose the first or last three points.
Intermediate
Point
Choice of Intermediate Points
Maximum
f (x)
0 x
xL xU
First L0
iteration L0 = L1 + L2
L1 L2
Second
iteration L1 L2
L2 =
L0 L1
L1 L2
From two condition, =
L1 + L2 L1
Set R = L2 / L1,
1
1+ R = or R2 + R −1 = 0
R
Solving for positive root,
−1 + 1 − 4(−1) 5 −1
R= = = 0.61803…
2 2
R = 0.61803 = Golden ratio since it allows optima to be
found efficiently
The Parthenon in Athens
0.61803 x
x
This golden ratio were considered aesthetically pleasing
by the Greeks.
Initial Step of the Golden-section Search
Eliminate
f (x) f (x1) Maximum
f (x2)
d
d
xL xU x
x2 x1
1) Guess initial bracket xL and xU
2) Choose two interior points x1 and x2 according to golden ratio,
5 −1
d= ( xU − xL )
2
x1 = xL + d , x2 = xU − d
3) If f (x1) > f (x2), eliminate [ xL , x2 ] and set x2 = xL for next round
Next Step to Complete the Algorithm
f (x) Maximum
xU x
xL x2 x1
Old x2 Old x1
4) Only new x1 need to be determined,
5 −1
x1 = xL + ( xU − xL )
2
Example: Golden-Section Search to find maximum
x2
f ( x) = 2sin x − , xL = 0 and xU = 4
10
Solution: (1) Create two interior points
5 −1
d= (4 − 0) = 2.472
2
x1 = 0 + 2.472 = 2.472
x2 = 4 − 2.472 = 1.528
(2) Evaluate function at interior points,
2.4722
f ( x1 ) = f (2.472) = 2sin(2.472) − = 0.63
10
f ( x2 ) = f (1.528) = 1.765
(3) Because f(x2) > f(x1), eliminate upper part,
New xU = x1 = 2.472 OLD
New x1 = x2 = 1.528
xL x2 x1 xU
(4) Compute new x2
5 −1 NEW
d= (2.472 − 0) = 1.528
2
x2 = 2.472 − 1.528 = 0.944 xL x2 x1 xU
(5) Evaluate function at x2

New f ( x1 ) = Old f ( x2 ) = 1.765
New f ( x2 ) = f (0.944) = 1.531
Because f(x1) > f(x2), eliminate lower part, . . .
Tabulated results:
i xL x2 f(x2) x1 f(x1) xU d
1 0 1.5279 1.7647 2.4721 0.6300 4.0000 2.4721

2 0 0.9443 1.5310 1.5279 1.7647 2.4721 1.5279
3 0.9443 1.5279 1.7647 1.8885 1.5432 2.4721 0.9443
4 0.9443 1.3050 1.7595 1.5279 1.7647 1.8885 0.5836
5 1.3050 1.5279 1.7647 1.6656 1.7136 1.8885 0.3607
6 1.3050 1.4427 1.7755 1.5279 1.7647 1.6656 0.2229
7 1.3050 1.3901 1.7742 1.4427 1.7755 1.5279 0.1378
8 1.3901 1.4427 1.7755 1.4752 1.7732 1.5279 0.0851
2nd polynomial provides good approximation near optimum
True maximum Quadratic
f (x) approximation
of maximum
x0 x1 x3 x2 x
f ( x0 )( x12 − x22 ) + f ( x1 )( x22 − x02 ) + f ( x2 )( x02 − x12 )

x3 =
2 f ( x0 )( x1 − x2 ) + 2 f ( x1 )( x2 − x0 ) + 2 f ( x2 )( x0 − x1 )
Example: Quadratic Interpolation to find maximum
x2
f ( x) = 2sin x − , x0 = 0, x1 =1 and x2 = 4
10
Solution: (1) Evaluate function values,
x0 = 0 f ( x0 ) = 0
x1 = 1 f ( x1 ) = 1.5829
x2 = 4 f ( x2 ) = −3.1136
0(12 − 42 ) + 1.5829(42 − 02 ) + ( −3.1136)(02 − 12 )

x3 = = 1.5055
2(0)(1 − 4) + 2(1.5829)(4 − 0) + 2( −3.1136)(0 − 1)
(2) Because f(1.5055) = 1.7691, employ golden-section search,
eliminate lower guess x0 = 0,
x0 = 1 f ( x0 ) = 1.5829
x1 = 1.5055 f ( x1 ) = 1.7691
x2 = 4 f ( x2 ) = −3.1136
1.5829(1.50552 − 42 ) + 1.7691(42 − 12 ) + ( −3.1136)(12 − 1.50552 )

x3 =
2(1.5829)(1.5055 − 4) + 2(1.7691)(4 − 1) + 2( −3.1136)(1 − 1.5055)
= 1.4903
After 5 iterations, x = 1.4276 gives maximum value of 1.7757

Newton’s Method
f ( xi )
f ( x) = 0 → xi +1 = xi −
f ′( xi )
Define new function, g ( x ) = f ′( x )
At optimal point x*, f ′( x*) = g ( x*) = 0
g ( xi ) f ′( xi )
g ( x) = 0 → xi +1 = xi − → xi +1 = xi −
g ′( xi ) f ′′( xi )
Example: Use Newton’s method to find maximum of
x2
f ( x) = 2sin x − , Initial guess: x0 = 2.5
10
Solution: (1) Evaluate 1st and 2nd derivatives of the function,
x
f ′( x) = 2 cos x −
5
1
f ′′( x) = −2sin x −
5
(2) Substitute into Newton’s formula,
2 cos xi − xi / 5
xi +1 = xi −
−2sin xi − 1/ 5
(3) Substituting initial guess,
2 cos 2.5 − 2.5 / 5
x1 = 2.5 − = 0.99508, f (0.99508) = 1.57859
−2sin 2.5 − 1/ 5
(4) Second iteration,

2 cos 0.995 − 0.995 / 5
x2 = 0.995 − = 1.46901, f (1.46901) = 1.77385
−2sin 0.995 − 1/ 5
i x f(x)
0 2.5 0.57194
1 0.99508 1.57859
2 1.46901 1.77385
3 1.42764 1.77573
4 1.42755 1.77573
After 4 iterations, result converges rapidly to the true value.

Multidimensional Unconstrained Optimization
Example: 2-D Topographic Map of 3-D Mountain
y Line of constant f
x
2D searches: Ascending a mountain (maximum) or
Descending into a valley (minimization)
DIRECT METHODS
Random Search
Univariate and Pattern Searches
GRADIENT METHODS
Gradients and Hessians

Steepest Ascent Method
Random Search
Repeatedly evaluates function at randomly selected points
Example: f ( x , y ) = y − x − 2 x 2 − 2 xy − y 2
Domain boundaries: x = -2 to 2 and y = 1 to 3

Write an M-file objfun1.m
function f = objfun1(x,y)
f = y-x-2*x^2-2*x*y-y^2;
>> rand % Uniformly distributed random numbers

% on the interval (0.0,1.0).
ans =
0.2311
Random search algorithm: rndsrch.m
function [maxx,maxy,maxf] = rndsrch(objfun,xmin,…

xmax,ymin,ymax)
iter = 100;
maxf = -inf;
for i = 1:iter
x = xmin + (xmax - xmin)*rand;
y = ymin + (ymax - ymin)*rand;
fn = feval(objfun,x,y);
if fn > maxf
maxf = fn;
maxx = x;
maxy = y;
end
end
>>[maxx,maxy,maxf]=rndsrch('objfun1',-2,2,1,3)
iter maxx maxy maxf

50 -0.9024 1.5913 1.2048
100 -0.8050 1.3092 1.2120
200 -1.1409 1.7708 1.2133
500 -1.0006 1.4983 1.2500
1000 -1.0253 1.5030 1.2489
TRUE -1.0000 1.5000 1.2500

Univariate and Pattern Searches
- More efficient and still no require derivative evaluation
- Change one variable at time while others constant
y
Pattern directions
4 3
1
2 x
From (1) move along x axis with y constant to max at (2)
Powell’s Method
Use pattern directions to find optimum efficiently
- Pt. 1 and 2 are obtained by 1-D
y
search from different starting pts.
- Line formed by 1 and 2 will be

directed toward maximum
2 Conjugate direction
1
x
GRADIENT METHODS
Use derivative to locate optima
Gradient : ∆y
∆f
∆h
∆x
∆f
= Slope along axis h
∆h
∂f ∂f
∇f = i + j = Steepest direction
∂x ∂y
The vector is called “del f “ = directional derivative of f (x , y)

Hessian
To determine whether maximum, minimum or saddle
2
∂ f ∂ f  ∂ f 
2 2 2
H = 2 − 
∂x ∂y  ∂x ∂y 
2
∂2 f
• If H > 0 and > 0 → local minimum
∂x 2
∂2 f
• If H > 0 and < 0 → local maximum
∂x 2
• If H < 0 → saddle point

Steepest Ascent Method
Climbing a hill with maximum slope
Repeat the process until

the maximum is reached
1 Follow steepest ascent path

until f (x, y) stop increasing
0 : Starting point (x0, y0)

x
Problems: 1) Determine the best direction of steepest ascent

2) Determine the best value along that search direction
Example: Steepest ascent method
f ( x, y ) = xy 2 , Starting point (2, 2)
>> contour(x,y,fxy)
4
x=0:0.1:4;
y=0:0.1:4; 3
fxy=zeros(length(y),length(x));
for i=1:length(y)
2
y
for j=1:length(x)
fxy(i,j)=x(j)*y(i)^2;
1
end
end
0
0 1 2 3 4
x
>> surf(x,y,fxy)
>> mesh(x,y,fxy)
Determine elevation: f (2, 2) = 2 × 22 = 8
Evaluate partial derivatives,

∂f
= y 2 = 22 = 4
∂x
∂f
= 2 xy = 2(2)(2) = 8
∂y
Determine the gradient, ∇ f = 4i + 8 j 63.40
8
−1
Direction, θ = tan   = 63.40 relative to x axis
4
Magnitude of ∇f is 42 + 82 = 8.944
We will gain 8.944 units of elevation rise for a unit distance

advanced along this steepest path.
63.4o
y 2
0
0 1 2 3 4
x
Relationship between steepest direction and x-y coordinate
Starting at x0, y0 any point in the

y ∇ f = 4i + 8 j
gradient direction can be computed by
10 ∂f
x = x0 + h
1
h= ∂x
∂f
y = y0 + h
∂y
2
0
Then, go along h axis to find maximum

h=
x
2 6 ∂f ∂f
Define g (h) = f ( x0 + h, y0 + h)
∂x ∂y
Set g ′(h*) = 0 to find h *
Example: Developing 1-D function along a gradient direction
f ( x, y ) = 2 xy + 2 x − x 2 − 2 y 2 , Starting point (−1,1)

Partial derivative at starting point:
∂f
= 2 y + 2 − 2 x = 2(1) + 2 − 2(−1) = 6
∂x
∂f
= 2 x − 4 y = 2(−1) − 4(1) = −6
∂y
Function along h axis:
∂f ∂f
f ( x0 + h, y0 + h) = f (−1 + 6h,1 − 6h)
∂x ∂y
= 2(−1 + 6h)(1 − 6h) + 2(−1 + 6h) − (−1 + 6h) 2 − 2(1 − 6h) 2
g (h) = −180h 2 + 72h − 7
Locate the maximum,
g ′(h*) = −360h * +72 = 0

h* = 0.2
Locate (x, y) coordinates according to h = 0.2,
x = −1 + 6(0.2) = 0.2
y = 1 − 6(0.2) = −0.2
Set new starting point to (0.2, -0.2) and repeat the process.
MATLAB Optimization Toolbox
fminunc : the unconstrained optimization function
f ( x, y ) = e x (4 x 2 + 2 y 2 + 4 xy + 2 y + 1), Starting point (−1, 1)
Write an M-file objfun2.m

function f = objfun2(x,y)
f = exp(x)*(4x^2+2*y^2+4*x*y+2*y+1);
>>x0 = [-1,1]; % Starting point

>>options = optimset(‘LargeScale’,’off’);
>>[x,fval,exitflag,output]=fminunc(‘objfun2’,x0,options);
x =
0.5000 -1.0000
Lecture 12 Constrained Optimization
- Linear Programming
- Nonlinear Constrained Optimization
- MATLAB’s fmincon Function
Suranaree
UNIVERSITY OF TECHNOLOGY Mongkol JIRAVACHARADET
NAKORNRATCHSIMA School of Civil Engineering
LINEAR PROGRAMMING
Standard Form
Objective Function: Maximize Z = c1x1+ c2x2+ . . . + cnxn

where cj = payoff of jth activity
xj = magnitude of jth activity
Z = total payoff due to all activities
Constraints: ai1x1 + ai2x2 + . . . + ainxn bi
where aij = amount of ith resource used by jth activity

bi = amount of ith resource available
Example: Setting Up the Linear Programming Problem
Producing product with limited resources: Gas processing plant
Product Resource
Resource Regular Premium Availability
Raw gas 7 m3/ton 11 m3/ton 77 m3/week

Production time 10 hr/ton 8 hr/ton 80 hr/week
Storage 9 ton 6 ton
Profit 150 / ton 175 / ton
Set x1 = amount of regular gas produced

x2 = amount of premium gas produced
Total profit = 150x1 + 175x2
Maximize Z = 150x1 + 175x2 Objective function
Total gas used = 7x1 + 11x2

cannot exceed 77 m3/week
7x1 + 11x2 77 Constraints
10x1 + 8x2 80
x1 9 x2 6
x1 , x2 0
Graphical Solution Maximize Z = 150x1 + 175x2
x2 7 x1 + 11x2 ≤ 77 (1)
10x1+8x2 = 80 10 x1 + 8 x2 ≤ 80 (2)
10x1+8x2 80 0 ≤ x1 ≤ 9 (3)
7
0 ≤ x2 ≤ 6 (4)
7x1+11x2 = 77
6
7x1+11x2 77
x1
0
9 11
Increasing Objective Function From Z = 150x1 + 175x2
For Z = 0, 150x1 + 175x2 = 0
x2
Unique solution
Z = 1400
x1
0
Z=0 Z = 600
Other Possible Outcomes
x2
Objective function parallel
to constraint.
Alternate Solution
0 x1
No Feasible Solution
x2
0 x1
Unbounded Problems
x2
0 x1
Simplex Method
Assumption: optimal solution will be an extreme point
x2
Extreme points: corner points
where two constraints meet
Z = 1400
Objective function
is LINEAR
x1
0 Z=0 Z = 600
Slack Variables
Reformulate constraint inequalities as equalities
slack variable
7 x1 + 11x2 ≤ 77 → 7 x1 + 11x2 + S1 = 77
Resource constraint How much slack of the resource

is available ?
IF . . .
S1 = + Surplus resource is not fully used.
S1 = - Exceeded the constraint.
S1 = 0 Exactly meet the constraint. All resource is used.

Fully Augmented Version
Maximize Z = 150x1 + 175x2

subject to
7 x1 + 11x2 ≤ 77 7 x1 + 11x2 + S1 = 77
10 x1 + 8 x2 ≤ 80 10 x1 + 8 x2 + S2 = 80
0 ≤ x1 ≤ 9 x1 + S3 =9
0 ≤ x2 ≤ 6 x2 + S4 =6
x1 , x2 , S1 , S 2 , S3 , S4 ≥ 0
Linear algebraic equations

solving 4 equations with 6 unknowns
Algebraic Solution
Underspecified : more unknowns (6) than equations (4)
x2 x1 = 9, S3 = 0
Extreme Point Zero Variables
10x1 + 8x2 = 80
S2 = 0 A x1, x2
x2 = 6, S4 = 0 B x2, S2
6 C S1, S2
E D
D S1, S4
C E x1, S4
7x1 + 11x2 = 77 @ Extreme point :
S1 = 0
A 4 equations with 4 unknowns
B
0 The problem can be solved.
x1
9
For point E : Setting x1 = S4 = 0 nonbasic variables
The standard form reduced to
0
7 x1 + 11x2 + S1 = 77 11x2 + S1 = 77
0
10 x1 + 8 x2 + S2 = 80 8 x2 + S2 = 80
0
x1 + S3 =9 S3 =9
0
x2 + S4 =6 x2 =6
which can be solved for
x1 x2 S1 S2 S3 S4
0 6 11 32 9 0
basic variables
Simplex Method Implementation S1 = 77
Start at point A : Setting x1 = x2 = 0 S2 = 80
S3 = 9
S4 = 6
Tableau: Z - 150x1 - 175x2 - 0S1 - 0S2 - 0S3 - 0S4 = 0
Basic Z x1 x2 S1 S2 S3 S4 Solution Intercept

Z 1 -150 -175 0 0 0 0 0
S1 0 7 11 1 0 0 0 77 11
S2 0 10 8 0 1 0 0 80 8
S3 0 1 0 0 0 1 0 9 9
S4 0 0 1 0 0 0 1 6 ∞
Graphical Depiction of Simplex Method
x2 Change x1 to basic variable (entering variable)

Move from A horizontally
S2
Check intercept:
6 S4
E D Intercept = Solution
S3 (for each S) x1 coeff.
C
S1 S2 give min. intercept = 8
A B F
0 x1
8 11
Therefore, change S2 to nonbasic(leaving) variable.

Move to point B : Setting x2 = S2 = 0 7 x1 + S1 = 77
10 x1 = 80
x1 + S3 = 9
S4 = 6
Pivot row S2: Dividing row by 10 and replacing S2 by x1
Basic Z x1 x2 S1 S2 S3 S4 Solution
Z 1 -150 -175 0 0 0 0 0
S1 0 7 11 1 0 0 0 77 x -150
x1 0 1 0.8 0 0.1 0 0 8
S3 0 1 0 0 0 1 0 9
S4 0 0 1 0 0 0 1 6
Z 1 0 -55 0 15 0 0 1200
Perform same operation on the remaining rows,
Basic Z x1 x2 S1 S2 S3 S4 Solution Intercept
Z 1 0 -55 0 15 0 0 1200
S1 0 0 5.4 1 -0.7 0 0 21 3.889
x1 0 1 0.8 0 0.1 0 0 8 10
S3 0 0 -0.8 0 -0.1 1 0 1 -1.25
S4 0 0 1 0 0 0 1 6 6
- From A to B, objective function has increased to 1200

- S1 give min. positive intercept = 3.889 is entering variable
- Move from B to C
Final Tableau:
Basic Z x1 x2 S1 S2 S3 S4 Solution
Z 1 0 0 10.19 7.87 0 0 1414
S1 0 0 1 0.19 -0.13 0 0 3.89
x1 0 1 0 -0.15 0.20 0 0 4.89
S3 0 0 0 0.15 -0.20 1 0 4.11
S4 0 0 0 -0.19 0.13 0 1 2.11
- No negative coeff. in objective func. row, No more increasing.
Final solution: x1 = 3.89, x2 = 4.89
Z = 1414
MATLAB’s fmincon Function
f ( x) = e x1 ( 4 x12 + 2 x22 + 4 x1 x2 + 2 x2 + 1)
subject to the constraints: x1 x2 − x1 − x2 ≤ −1.5
x1 x2 ≥ −10
rewrite the constraints in the form c(x) 0 :
x1 x2 − x1 − x2 + 1.5 ≤ 0
− x1 x2 − 10 ≤ 0
Write M-file confun.m for the constraints
function [c, ceq] = confun(x)

% nonlinear inequality constraints
c = [1.5 + x(1)*x(2) - x(1) - x(2);
-x(1)*x(2) - 10];
% nonlinear equality constraints
ceq = [];
> x0 = [-1,1];
> options = optimset(‘LargeScale’,’off’);
> [x, fval] = . . .
fmincon(‘objfun’,x0,[],[],[],[],[],[],’confun’,options)
x =
-9.5474 1.0474
fval =
0.0236

Numerical Methods For Civil Engineers (MATLAB)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Numerical Methods For Civil Engineers (MATLAB)

Uploaded by

Copyright:

Available Formats

Welcome to

Numerical Methods for Civil Engineers

School of Civil Engineering

Suranaree University of Technology

McGraw-Hill International Edition

Numerical Methods with MATLAB : Implementations and Applications,

An Introduction to Numerical Methods : A MATLAB Approach,

Numerical Methods using MATLAB,

The Matlab 7 Handbook , Mathwork Inc.

KEEP THESE BOOKS! They are excellent career references

• Approximations and Errors • Numerical Integration

• Roots of Equations • Ordinary Differential Equations

• Linear Systems • Optimization

2) Study in groups but submit work on your own

3) No Copying of Matlab code

4) Submit Homework at the beginning of class

5) Late homework with penalty 30%

6) No make up quizzes or exams

The latest version Matlab 7.3 R2006b

- Modeling, Simulation, and Prototyping

- Data analysis, exploration, and visualization

- Scientific and Engineering Graphics

- Many toolboxes for solving problems:

Control System Toolbox Statistics Toolbox

Signal Processing Toolbox Optimization Toolbox

System Identification Toolbox Partial Diff. Equation Toolbox

Neural Network Toolbox Symbolic Math Toolbox

MATLAB Educational Sites

- Vector & Matrix

The MATLAB System

The MALAB Mathematical Function Library: a vast collection of computational

The MATLAB Language: This is a high-level matrix/array language with control

Command prompt >> DEMO

Last-line editing Up Arrow

>> ans/2 >> 2*A

ans = 2 Several commands in the same line:

>> a = 5 … For too long command:

>> sin(pi/4) >> pi sin(x), cos(x), tan(x),

The output of each command can be suppress by using semicolon ;

>> a = 5; b = sin(a), c = sinh(a)

MATLAB Variables is created whenever it appears

Any variable appearing on the right-hand side of

Use long variable names is better to remember >> format short e

ans value of an expression when not assigned to variable

>> x = 0; >> help log

Matrices and Magic Squares

scalars = 1-by-1 matrices

vectors = one row or column matrices

it is usually best to think of everything

The matrices operations in MATLAB are

MATLAB allows you to work with entire

Renaissance engraving Melencolia I by the German artist and amateur

You can enter matrices into MATLAB in several different ways:

Start by entering matrix as a list of its elements.

To enter matrix, simply type in the Command Window

Row and Column Vectors

Row vector >> A = [ 2 3 5 7 11]

>> A = zeros(4) >> A = zeros(3,4)

Elementary Matrix Operations

>> S = A + B >> [m , n] = size(A)

The element in row i and column j of A is denoted by A(i , j)

A = [16 3 2 13; 5 10 11 8; 9 6 7 12; 4 15 14 1]

The Colon Operator