1

CISE301: Numerical Methods

Unit 1:
Introduction to Numerical methods and
Review of Taylor Series
Lectures 1-3:
Dr.Sayyid Anas Vaqar
Dr.Sayyid Anas Vaqar
OfficeHours
UTR8:009:00AMand11:0011:11:30AM ,
orbyappointment
Office221481
Tel8607876
Email:
usetheBlackboardemail
9/9/2013 2
2
GradingPolicy
StandardGradingpolicywillbeadapted
Attendance 5%(1%per
absence)
Quizzes 10%
HWs/CompWork 10%
MajorI 20%topics1,2,3
(Date: Monday24
th
ofJune2013
Time: 4:30PM 6:30PM)
MajorII 25%topics4,5,6
(Date: Saturday13
th
ofJuly2013
Time: 9:30PM 11:30PM)
Final 30%topics7,8,9
(DatesetbyRegistrar,Duration=2hours
Time: 7:00PM9:00PM)
Range Grade
[95,100]
A+
A
B+
B
C+
C
D+
D
[0,50 )
F
9/9/2013 3
RulesandRegulations
Nomakeup
quizzes/exams
DNgrade 4 unexcusedabsences
HomeworkAssignmentsareduetothe
beginningofthelectures.
Absenceisnotanexcusefornotsubmitting
theHomework.
Latesubmissionmaynotbeaccepted
9/9/2013 4
3
LearningObjectives
Definenumericalmethodsandlisttopicscoveredinthecourse
Compareanalyticsolutionsandnumericalsolutions;
Discusstheneedfornumericalmethods
Discusstheneedforanalyzingthequalityofnumericalsolutions;
ReadingAssignment:pages310oftextbook
AnalyticalSolutions
Analyticalmethodsgiveexactsolutions
Example:Analyticalmethodtoevaluatetheintegral
Numericalmethodsaremathematicalprocedurestocalculate
approximatesolution
(TrapezoidMethod)
3
1
3
0
3
1
3
1
0
3
1
0
2
= = =
}
x
dx x
( )
2
1
) 1 ( ) 0 (
2
0 1
2 2
1
0
2
= +

~
}
dx x
9/9/2013 6
4
NumericalMethods
NumericalMethods:
Algorithmsthatareusedtoobtainapproximate
solutions ofamathematicalproblem.
Whydoweneedthem?
1.Noanalyticalsolutionexists,
2.Ananalyticalsolutionisdifficulttoobtain
ornotpractical.
9/9/2013 7
Whatdoweneed
BasicNeedsintheNumericalMethods:
Practical:
canbecomputedinareasonableamountoftime.
Accurate:
Goodapproximatetothetruevalue
Informationabouttheapproximationerror
(Bounds,errororder,)isavailable
9/9/2013 8
5
OutlinesoftheCourse
1.TaylorTheorem,NumberRepresentation,Errors
2.SolutionofnonlinearEquations
3.SolutionoflinearEquations
4.LeastSquarescurvefitting
5.Interpolation
6.NumericalDifferentiation
7.NumericalIntegration
8.Solutionofordinarydifferentialequations
9.SolutionofPartialdifferentialequations
9/9/2013 9
SolutionofNonlinearEquations
Somesimpleequationscanbesolvedanalytically
Manyotherequationshavenoanalyticalsolution
9/9/2013 10
3 1
) 1 ( 2
) 3 )( 1 ( 4 4 4
: solution Analytic
0 3 4
2
2
= =

=
= + +
x and x
roots
x x
solution analytic No
0 5 2
2 9

=
= +
x
e x
x x
6
MethodsforsolvingNonlinearEquations
o BisectionMethod
o NewtonRaphsonMethod
o SecantMethod
9/9/2013 11
SolutionofSystemsofLinearEquations
9/9/2013 12
unknowns? 1000 in equations 1000
have we if do What to
1 2 3 , 2
5 2 3 , 3
as it solve can We
5 2
3
1 2
2 2 2 1
2 1
2 1
= = =
= + =
= +
= +
x x
x x x x
x x
x x
7
CramersRuleisnotpractical
9/9/2013 13
compute. to years 10 than more needs computer super A
needed. are tions multiplica 10 2.38 system, 30 by 30 a solve To
tions. multiplica 1)N! 1)(N (N need we unknowns N in equations N solve To
2
2 1
1 1
5 1
3 1
, 1
2 1
1 1
2 5
1 3
5 2
3
system the solve to used be can Rule s Cramer'
20
35
2 1
2 1
2 1

+
= = = =
)
`

= +
= +
x x
x x
x x
Cramers Rule
is not practical
for large
problems
MethodsforsolvingSystemsofLinear
Equations
o NaiveGaussianElimination
o GaussianEliminationwithScaledPartial
pivoting
o AlgorithmforTridiagonalEquations
9/9/2013 14
8
CurveFitting
Givenasetofdata
Selectacurvethatbestfitthedata.
Onechoiceisfindthecurvesothatthesumofthe
squareoftheerrorisminimized.
9/9/2013 15
x
i
0 1 2
y
i
0.5 10.3 15.3


Interpolation
Givenasetofdata
findapolynomialP(x)whosegraphpasses
throughalltabulatedpoints.
9/9/2013 16
x
i
0 1 2
y
i
0.5 10.3 15.3


table in the is ) (
i i i
x if x P y =
9
MethodsforCurveFitting
o LeastSquares
o LinearRegression
o NonlinearleastSquaresProblems
o Interpolation
o Newtonpolynomialinterpolation
o Lagrangeinterpolation
9/9/2013 17
Integration
Somefunctionscanbeintegratedanalytically
9/9/2013 18
?
solutions analytical no have functions many But
=
= = =
}
}

dx e
x xdx
a
x
0
3
1
2
3
1
2
4
2
1
2
9
2
1
10
MethodsforNumericalIntegration
o UpperandLowerSums
o TrapezoidMethod
o RombergMethod
o GaussQuadrature
9/9/2013 19
SolutionofOrdinaryDifferentialEquations
9/9/2013 20
only cases special
for available are solutions Analytical *
equations the satisfies that function a is
0 ) 0 ( ; 1 ) 0 (
0 ) ( 2 ) ( 3 ) (
equation al differenti the o solution t A
x(t)
x x
t x t x t x
= =
= + +

11
SolutionofPartialDifferential
Equations
PartialDifferentialEquationsaremoredifficulttosolve
thanordinarydifferentialequations
9/9/2013 21
) sin( ) 0 , ( , 0 ) , 1 ( ) , 0 (
0 2
2
2
2
2
x x u t u t u
t
u
x
u
t = = =
= +
c
c
+
c
c
Summary
NumericalMethods:
Algorithmsthatareusedto
obtainnumericalsolution
ofamathematicalproblem.
Weneedthemwhen
Noanalyticalsolutionexist
oritisdifficulttoobtain
analyticalsolution.
1. Introduction
2. SolutionofnonlinearEquations
3. SolutionoflinearEquations
4. LeastSquaresCurvefitting
5. Interpolation
6. NumericalIntegration
7. NumericalDifferentiation
8. Solutionofordinarydifferential
equations
9. SolutionofPartialdifferential
equations
9/9/2013 22
Topics Covered in the Course
12
NumberRepresentation
NormalizedFloatingPointRepresentation
SignificantDigits
AccuracyandPrecision
RoundingandChopping
Readingassignment:Chapter3
RepresentingRealNumbers
Youarefamiliarwiththedecimalsystem
DecimalSystemBase=10,Digits(0,1,9)
StandardRepresentations
2 1 0 1 2
10 5 10 4 10 2 10 1 10 3 45 . 312

+ + + + =
part part
fraction integral sign
5 4 . 2 1 3
9/9/2013 24
13
NormalizedFloatingPointRepresentation
NormalizedFloatingPointRepresentation
Nointegralpart,
Advantage Efficientinrepresentingverysmallorverylargenumbers
integer : , 0
exponent mantissa sign
10 . 0
1
4 3 2 1
n d
d d d d
n
=

9/9/2013 25
CalculatorExample
supposeyouwanttocompute
3.578*2.139
usingacalculatorwithtwodigitfractions
9/9/2013 26
3.57
*
2.13 7.60
=
7.653342
True answer
14
BinarySystem
BinarySystemBase=2,Digits{0,1}
1 0
1 1
= = b b
exponent mantissa sign
2 1 . 0
4 3 2
n
b b b
9/9/2013 27
10 10
3 2 1
2
) 625 . 0 ( ) 2 1 2 0 2 1 ( ) 101 . 0 ( = + + =

Fact
Numberthathavefiniteexpansioninonenumberingsystemmay
haveaninfiniteexpansioninanothernumberingsystem
Youcanneverrepresent0.1exactlyinanycomputer
9/9/2013 28
2 10
...) 01100 0001100110 . 0 ( ) 1 . 0 ( =
15
ConvertingDecimalFractionsto
Binary
A simple,stepbystepmethodforcomputingthebinaryexpansionontherighthandsideofthe
point.Wewillillustratethemethodbyconvertingthedecimalvalue.625toabinaryrepresentation..
Step1:Beginwiththedecimalfractionandmultiplyby2.Thewholenumberpartoftheresultisthe
firstbinarydigittotherightofthepoint.
Because.625x2= 1.25,thefirstbinarydigittotherightofthepointisa 1.
Sofar,wehave.625=.1???...(base2).
Step2:Nextwedisregardthewholenumberpartofthepreviousresult(the1inthiscase)and
multiplyby2onceagain.Thewholenumberpartofthisnewresultisthe second binarydigittothe
rightofthepoint.Wewillcontinuethisprocessuntilwegetazeroasourdecimalpartoruntilwe
recognizeaninfiniterepeatingpattern.Because.25x2= 0.50,thesecondbinarydigittotherightof
thepointisa 0.
Sofar,wehave.625=.10??...(base2).
Step3:Disregardingthewholenumberpartofthepreviousresult(thisresultwas.50sothereactually
isnowholenumberparttodisregardinthiscase),wemultiplyby2onceagain.Thewholenumber
partoftheresultisnowthenextbinarydigittotherightofthepoint.
Because.50x2= 1.00,thethirdbinarydigittotherightofthepointisa 1.
Sonowwehave.625=.101??...(base2).
Step4:Infact,wedonotneedaStep4.WearefinishedinStep3,becausewehad0asthefractional
partofourresultthere.
Hencetherepresentationof.625=.101(base2).
9/9/2013 29
ConvertingDecimalFractionsto
Binary
InfiniteBinaryFractions
Themethodwejustexploredcanbeusedtodemonstratehowsomedecimalfractionswillproduceinfinitebinaryfraction
expansions.Weillustratebyusingthatmethodtoseethatthebinaryrepresentationofthedecimalfraction1/10is,infact,
infinite.
Step1:Beginwiththedecimalfractionandmultiplyby2.Thewholenumberpartoftheresultisthefirstbinarydigittothe
rightofthepoint.Because.1x2= 0.2,thefirstbinarydigittotherightofthepointisa 0.Sofar,wehave.1(decimal)=.0???...
(base2).
Step2:Nextwedisregardthewholenumberpartofthepreviousresult(0inthiscase)andmultiplyby2onceagain.Thewhole
numberpartofthisnewresultisthe second binarydigittotherightofthepoint.Wewillcontinuethisprocessuntilwegeta
zeroasourdecimalpartoruntilwerecognizeaninfiniterepeatingpattern.Because.2x2= 0.4,thesecondbinarydigittothe
rightofthepointisalsoa 0.
Sofar,wehave.1(decimal)=.00??...(base2).
Step3:Disregardingthewholenumberpartofthepreviousresult(againa0),wemultiplyby2onceagain.Thewholenumber
partoftheresultisnowthenextbinarydigittotherightofthepoint.Because.4x2= 0.8,thethirdbinarydigittotherightof
thepointisalsoa 0.
Sonowwehave.1(decimal)=.000??...(base2).
Step4:Wemultiplyby2onceagain,disregardingthewholenumberpartofthepreviousresult(againa0inthiscase).Because
.8x2= 1.6,thefourthbinarydigittotherightofthepointisa 1.Sonowwehave.1(decimal)=.0001??...(base2).
Step5:Wemultiplyby2onceagain,disregardingthewholenumberpartofthepreviousresult(a1inthiscase).Because.6x2
= 1.2,thefifthbinarydigittotherightofthepointisa 1.Sonowwehave.1(decimal)=.00011??...(base2).
Step6:Wemultiplyby2onceagain,disregardingthewholenumberpartofthepreviousresult.Let'smakeanimportant
observationhere.Noticethatthisnextsteptobeperformed(multiply2.x2)is exactlythesameactionwehadinstep2.Weare
thenboundtorepeatsteps25,thenreturntoStep2againindefinitely.Inotherwords,wewillnevergeta0asthedecimal
fractionpartofourresult.Insteadwewilljustcyclethroughsteps25forever.Thismeanswewillobtainthesequenceofdigits
generatedinsteps25,namely0011,overandover.Hence,thefinalbinaryrepresentationwillbe.
.1(decimal)=.00011001100110011...(base2).
9/9/2013 30
16
7BitRepresentation
(sign:1bit,Mantissa3bits,exponent3bits)
9/9/2013 31
exponent mantissa sign
2 1 . 0
4 3 2
n
b b b
7BitRepresentation
(sign:1bit,Mantissa3bits,exponent3bits)
9/9/2013 32
17
Representation
9/9/2013 33
Hypothetical Machine (real computers use 23 bit mantissa)
Mantissa 2 bits exponent 2 bit sign 1 bit
Possible machine numbers
.25 .3125 .375 .4375 .5 .625 .75 .875
1 1.25 1.5 1.75
Remarks
Numbersthatcanbeexactlyrepresentedarecalled
machinenumbers
Differencebetweenmachinenumbersisnotuniform
sumofmachinenumbersisnotnecessarilyamachinenumber
0.25+.3125=0.5625
9/9/2013 34
Machine
number
Machine
number
NOT
Machine
number
18
SignificantDigits
9/9/2013 35
Significant digits are those digits that can be
used with confidence.
0 1 2 3 4
Length of green rectangle = 3.45
significant
LossofSignificance
Mathematicaloperationsmayleadtoreducingthe
numberofsignificantdigits
9/9/2013 36
0.123466 E+02 6 significant digits
0.123445 E+02 6 significant digits

0.000021E+02 2 significant digits

0. 210000E-02
Subtracting nearly equal numbers causes loss of significance
19
AccuracyandPrecision
9/9/2013 37
Accuracy is related to closeness to the true value
Precision is related to the closeness to other estimated
values
AccuracyandPrecision
9/9/2013 38
Accuracy is
related to
closeness to the
true value
Precision is
related to the
closeness to
other estimated
values
20
AccuracyandPrecision
9/9/2013 39
Better
Precision
Better
accuracy
Accuracy is
related to
closeness to the
true value
Precision is
related to the
closeness to
other estimated
values
RoundingandChopping
Rounding:Replacethenumberbythenearest
machinenumber
Chopping:Throwallextradigits
9/9/2013 40
0 1 2
True 1.1681
Rounding (1.2) Chopping (1.1)
21
ErrorDefinitions
TrueError
canbecomputedifthetruevalueisknown
100 *
value true
ion approximat value true
Error elative Percent R Absolute
ion approximat value true
Error True Absolute
t

=
=
c
t
E
9/9/2013 41
100 *
value true
ion approximat value true
Error Relative True
ion approximat value true
Error True
t

=
=
c
t
E
ErrorDefinitions
Estimatederror
Usedwhenthetruevalueisnotknown
100 *
estimate current
estimate prevoius estimate current
Error elative Percent R Absolute Estimated
estimate prevoius estimate current
Error Absolute Estimated

=
=
a
a
E
c
9/9/2013 42
100 *
estimate current
estimate prevoius estimate current
Error Relative Percent e Approximat
estimate prevoius estimate current
Error A

=
=
a
a
E
pproximate
c
22
Errorasstoppingcriterion
9/9/2013 43
Itisoftenusefultoemploytheabsolutevalueerroras
astoppingcriterion.
Weareinterestedinwhetherthepercentabsolute
errorislowerthanaprespecifiedpercenttolerance s
c
s a
c c <
Notation
Wesaytheestimateiscorrecttondecimaldigitsif
Wesaytheestimateiscorrecttondecimaldigits
rounded if
n
s 10
2
1
Error
n
s10 Error
9/9/2013 44
Wesaytheestimateiscorrecttondecimaldigitsif
Wesaytheestimateiscorrecttondecimaldigits
rounded if
n
s10 Error
n
s 10
2
1
Error
23
Notation
Wesaytheestimateiscorrecttondecimaldigitsif
Wesaytheestimateiscorrecttondecimaldigits
rounded if
n
s 10
2
1
Error
n
s10 Error
9/9/2013 45
Wesaytheestimateiscorrecttoatleastn
significantdigitsif:
Summary
9/9/2013 46
Number Representation
Number that have finite expansion in one numbering system may
have an infinite expansion in another numbering system.
Normalized Floating Point Representation
Efficient in representing very small or very large numbers
Difference between machine numbers is not uniform
Representation error depends on the number of bits used in the
mantissa.
24
Summary
AccuracyPrecision
RoundingChopping
ErrorDefinitions:
Absolutetrueerror
TruePercentrelativeerror
Estimatedabsoluteerror
Estimatedpercentrelativeerror
9/9/2013 47