Review Previous Lesson Before Class Starts

You are estimating the integral of a function using evenly

spaced points. As the grid spacing is refined, which integration
method in the following will be the fastest to converge to the
exact solution ?

A) Left Riemann integral

B) Right Riemann integral
C) Midpoint rule
D) Trapezoid rule
E) Simpson’s rule
What is the integral of f(x) from -1 to 2
as estimated by a right Riemann
integral with spacing Δx = 1?

A) -2
B) 1
C) 2
D) 4
E) 8
A) -12
B) -10
C) -6
D) 6
E) 12
E7 – Lecture Series
Lecture 23
• Monte-Carlo Integration
• Numerical Methods for Ordinary Differential
Equations (I)

Fall 2021
Instructor: Shaofan Li
Four-dimensional Sphere
Summary of Accuracy for Numerical Integration
Bonus Point
What is Monte Carlo Simulation (MCS) method ?

The Monte Carlo method is a numerical method

for statistical simulation which utilizes sequences
of random variables (numbers) to perform the simulation
1 0
 Summary: Central Limit Theorem (CLT)
If x1, x2, …, xn is an iid sample from a distribution with mean
m, and standard deviations s, then if n is large

has a normal distribution with mean

and variance
Summary: Monte Carlo Integration Method
 Sample Mean MC algorithm

1. Generate sequence of N of PRN : Ri

2. Compute Xi=a+Ri (b-a)

3. Compute f(Xi) , i=1,2,3,….,N

Note: if f(x) is not square

integrable ,then the MC
Estimate Î will still converge
to the true value, but the error
estimate becomes unreliable.
Use the reject and accept method or the hit and miss method
 Hit and miss algorithm

♣ Start from C=0

♣ If (X²+Y²<1) C=C+1

♣ # of dots inside circle=C

♣ total number of dots=N

𝜋 = 4 ∗ 𝐶/𝑁
 Take Home Message

Central limit theorem says that mean of the distribution of the sample means is equal to
the population mean irrespective of the distribution of the population when the sample size
is greater than 30.
To understand the Central Limit Theorem is to know where God is living.
------------ Anonymous
II. Ordinary Differential Equations
• Many practical problems in science and engineering involve rates (derivatives).

• Equations which are composed of an unknown function and its derivatives are called
differential equations (DEs)

Examples from: Numerical Methods for Engineers by S.C. Chapra and R.P. Canale
 Classification of Differential Equations
DEs can be divided into several types: Ordinary/Partial, Linear/Non-Linear…

Also we can classify them according to the highest order derivative

in the differential equation. (1st order, 2nd order etc.)

Partial Differential Equations (PDEs)

Ordinary Differential Equations (ODEs)
• Two or more independent variables
• A single independent variable • Involves partial derivatives
Ordinary Differential Equations (ODEs)
Example: Mass-spring-damper system The force diagram

Using the Newton’s Second Law…

𝑚𝑢ሷ + 𝑐 𝑢ሶ + 𝑘𝑢 = 𝑝(𝑡) ( 2nd order linear ODE)

where
𝑑2𝑢 𝑑𝑢
𝑡 = 𝑖𝑛𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡 𝑣𝑎𝑟𝑖𝑎𝑏𝑙𝑒 𝑢ሷ = 𝑢ሶ =
𝑑𝑡 2 𝑑𝑡
 Ordinary Differential Equations (ODEs)
Example: Simple Pendulum

Equilibrium of the moments of forces about O gives the ODE:

( 2nd order non-linear ODE)

where Non-linearity comes
𝑑2𝜃 from the sin(𝜃) term
𝑡 = 𝑖𝑛𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡 𝑣𝑎𝑟𝑖𝑎𝑏𝑙𝑒 𝜃ሷ =
𝑑𝑡 2

Figures from: Dynamics of Structures by A.K. Chopra

The Initial Conditions ---- The Initial-Value Problem (IVP)
A differential equation of order n has a solution containing n arbitrary constant,
which is called the general solution of the DE.

By letting these constants vary, we can obtain infinitely many solutions.

In practice, we are interested in finding a solution that satisfy some constraints, which is called
the particular solution of the DE. These constraints are called as boundary conditions.

Recall the example problem: 𝑚𝑢ሷ + 𝑐𝑢ሶ + 𝑘𝑢 = 𝑝(𝑡)

𝑢ሶ 𝑡 = 0 Two boundary conditions for

the second order ODE
𝑢(𝑡) = 0
(Initial velocity and displacement of the mass is zero)
Boundary Conditions--- Boundary-Value Problem (BVP)
𝒅𝒚
Consider the ODE: 𝒅𝒙 = 𝒄𝒐𝒔(𝒙) which has a general solution 𝒚 = 𝐬𝐢𝐧 𝒙 + 𝑪
where C is a constant, depending on the boundary condition.

Solutions curves, corresponding to for different C’s

Solutions to ODEs: Analytical Solution
In order to obtain a particular solution of a n’th order ODE, we need n boundary conditions!

Consider the free falling of an object under gravity, which is defined by the 2nd order ODE:

𝑑2 𝑥
2
=𝑔
𝑑𝑡
We can integrate this equation two times to obtain the general solution (with two constants):

𝑑𝑥 1 2
= 𝑔𝑡 + 𝐶1 𝑥 = 𝑔𝑡 + 𝐶1 𝑡 + 𝐶2
𝑑𝑡 2

Notice that 𝐶1 is the initial velocity and 𝐶2 is the initial position of the object:

1 2
𝑥 = 𝑔𝑡 + 𝑣0 𝑡 + 𝑥0
2
We need two boundary conditions (initial velocity and position) to obtain the particular solution!
Solutions to ODEs: Analytical Solution

Recall the pendulum example:

If we assume that rotations are small, we can use the small-angle approximation

Therefore the ODE becomes

𝑔
𝜃ሷ + 𝜃 = 0
𝐿
Given the boundary conditions
ሶ = 0) = 0
𝜃(𝑡 𝜃(𝑡 = 0) = 𝜃0

Particular solution can be obtained as:

𝑔
𝜃 𝑡 = 𝜃0 cos( 𝑡)
𝐿
Numerical Solutions to ODE: (I) Euler Methods

Previous examples are rare cases where we can obtain the analytical solution easily,
since the problem is simple enough or we can make assumptions to simplify the problem.

In general, finding the analytical solution of a differential equation is very difficult or

impossible. Therefore, we need numerical techniques to solve them.
 Euler’s Method

Consider the first order ODE defined on an interval with a given boundary (or initial) condition:
𝑑𝑦
= 𝑓(𝑡, 𝑦) 𝑡 ∈ [𝑡0 , 𝑡𝑛 ] 𝑦(𝑡 = 𝑡0 ) = 𝑦0
𝑑𝑡

We can divide the interval into equal lengths of h (step-size) to obtain the numerical grid:

[𝑡0 , 𝑡1 , 𝑡2 , … , 𝑡𝑛−1 , 𝑡𝑛 ] where 𝑡0 < 𝑡1 < 𝑡2 , … , 𝑡𝑓−1 < 𝑡𝑛 and 𝑡𝑖+1 = 𝑡𝑖 + ℎ 𝑓𝑜𝑟 𝑖 = 1,2, … , 𝑛

Starting with the initial value 𝑦0 , we can predict the next function value by making use of
the slope at 𝑡0 to extrapolate linearly over the step size h
𝑦1 = 𝑦0 + ℎ ∙ 𝑓 𝑡0 , 𝑦0
Euler’s Method

𝑦1 = 𝑦0 + ℎ ∙ 𝑓 𝑡0 , 𝑦0

𝑦1 = 𝑦0 + ℎ ∙ 𝑓 𝑡0 , 𝑦0

𝑦2 = 𝑦1 + ℎ ∙ 𝑓 𝑡1 , 𝑦1

𝑦3 = 𝑦2 + ℎ ∙ 𝑓 𝑡2 , 𝑦2

𝑦𝑛 = 𝑦𝑛−1 + ℎ ∙ 𝑓 𝑡𝑛−1 , 𝑦𝑛−1

 We can generalize this idea to obtain the Explicit (or forward) Euler Formula

𝑦𝑖+1 = 𝑦𝑖 + ℎ ∙ 𝑓 𝑡𝑖 , 𝑦𝑖

Explicit Euler Formula

𝑦

𝑦2 𝑦3 𝑦4
The algorithm is ‘explicit’ since it only requires information at
𝑦1
the previous time step 𝑡𝑖 to compute the state at 𝑡𝑖+1

𝑦0 𝑇𝑟𝑢𝑒 𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛, 𝑦(𝑡)

𝑡
Implicit Euler Method
𝑡 ∈ [𝑡0 , 𝑡𝑛 ] 𝑦(𝑡 = 𝑡0 ) = 𝑦0
𝑑𝑦
= 𝑓(𝑡, 𝑦)
𝑑𝑡

Another method can be derived using the 1st order Taylor expansion around 𝑡𝑖+1 and evaluating it at 𝑡𝑖

𝑦𝑖+1 = 𝑦𝑖 + ℎ ∙ 𝑓 𝑡𝑖+1 , 𝑦𝑖+1

Implicit Euler Formula

 DEM

Consider the initial value

problem:
𝑦 ′ = −2𝑥 3 + 12𝑥 2 − 20𝑥 + 8.5

𝑦 0 =1 𝑥 ∈ [0,4]

Apply Explicit Euler’s Method

to solve this ODE numerically
in the given domain
Explicit Euler’s Method
Consider the following ODE with a given initial condition: 𝑦 ′ = −2𝑥 3 + 12𝑥 2 − 20𝑥 + 8.5 𝑦 0 = 1
Apply Euler’s Method with different step-sizes:

𝐸𝑥𝑎𝑐𝑡 𝑆𝑜𝑙𝑢𝑡𝑖𝑜𝑛 ∶ 𝑦 𝑥 = −0.5𝑥 4 + 4𝑥 3 − 10𝑥 2 + 8.5𝑥 + 1

Euler’s method converges to the analytical solution as step size gets smaller.