Lecture 1.

Introduction to Stochastic Process

Julia Hinkel

J. Hinkel, 1. August, 2007 p.1/20

Outline
Basic concepts of probability theory; Markov process; Random walks; Master equation; FokkerPlank equation; From binomial distribution to Gaussian one;

J. Hinkel, 1. August, 2007 p.2/20

Ficks law; Einsteins (Smoluchowskis) derivation of diffusion equation. Langevin formalizm; Wiener process; From microscopic description of stochastic differential equations to mesoscopic one; Fluctuationdissipation theorem.

J. Hinkel, 1. August, 2007 p.3/20

Motivations
The study of complex systems coming from chemical, biological and social sciences; Investigations in this eld have been determined as interdisciplinary research. For example: Trafc ow; Granular matter; Ant colony behaviour; Transactions in nancial markets.

J. Hinkel, 1. August, 2007 p.4/20

These systems are interesting not only as objects of natural sciences but also from the physical points of view for fundamental understanding and detailed analysis of such exotic phenomena. What does it mean to understand into details??? The main goal of such investigations is to invent a theoretical model which can describe the general features of the phenomenon under consideration; To provide the detailed theoretical analysis and computer simulation of these models.

J. Hinkel, 1. August, 2007 p.5/20

How it is Possible to Describe a Process?

To describe the temporal evolution of the process x(t); Deterministic approach = x = F (x, t); Nondeterministic approach
x = F (x, t) + some nondeterministic term = + STOCHASTICITY or RANDOMNESS
(1)

What the difference between these two approaches? The way of the studying: To forecast the behaviour of the described object for the specic time (or OBSERVATION time) tobs .

J. Hinkel, 1. August, 2007 p.6/20

Deterministic process = one its realization and everything clear; Stochastic process = one realization, two, ..., ten, ..., hundred, may be more??? Ensemble of realization; Probability distributions.

J. Hinkel, 1. August, 2007 p.7/20

Markov Chain and Random Walk

Discrete time and space process with the Markov property: The conditional probability distribution of future states of the process, given the present state and all past states, depends only on the present state and not on any past states.
P (m, n) describes the probability of being at position m after n steps; p and q are the transition probabilities for a step m m + 1 and m m 1 respectively;

J. Hinkel, 1. August, 2007 p.8/20

The governing equation is

P (m, n + 1) = p P (m 1, n) + q P (m + 1, n)
(2)

The solution for the initial condition m = n = 0 is the binomial distribution. Master Equation: Continuous time and discrete space process; Transformation t = n t with the xed time step t;
t 0 = Master equation p q p+q P (m, t) = P (m 1, t) + P (m + 1, t) P (m, t) t t t t
(3)

Transition rates are w+ = p/t and w = q/t.

J. Hinkel, 1. August, 2007 p.9/20

DriftDiffusion Equation
Continuous space and time process; Transformation x = m x with the xed space step x;
x 0; P (m, t) p(x, t) dx;

FokkerPlanck equation
p(x, t) p(x, t) 2 p(x, t) = v +D t x x2
(4)

J. Hinkel, 1. August, 2007 p.10/20

Ficks Law
Diffusion Equation
2 p(x, t) p(x, t) =D t x2
(5)

Normalization condition:

+ p(x, t)

= 1;

continuity equation for certain position x: change of probability at a xed position in time equals to the spatial derivative of a probability ux:
p(x, t) jdif f (x, t) =0 t x
(6)

with diffusion ux: jdif f (x, t) = D p(x,t) . x

J. Hinkel, 1. August, 2007 p.11/20

Brownian Motion
The Brownian motion which observed originally by Robert Brown in 1827 has initiated the development of a new branch in physics; By modelling of the nature phenomena, it has been concluded, that the wellinvestigated deterministic approach based on the Newton theory could not give the correct description of such processes; It has been found that the systems under study do not have the deterministic behaviour. The detailed experimental investigations of the Brownian motion have shown that the observed motion is irregular and unpredictable

J. Hinkel, 1. August, 2007 p.12/20

First explanations of the Brownian motion have been given by Einstein and Smoluchowski and were provided in terms of probability density distribution; Langevin has used another approach based on Newtons law There are two forces having the inuence on the particle dynamics: deterministic force given by friction v ; stochastic force which uctuates rapidly (t).

J. Hinkel, 1. August, 2007 p.13/20

The onedimensional stochastic differential equation in terms of the Langevin notation has the form
dx = (t) dt
(7)

J. Hinkel, 1. August, 2007 p.14/20

The stochastic force (t) presented by Gaussian white noise; There is the problem that the Langevin equation is impossible to integrate; The integral of the random term is not differentiable; As result, another approach has to be used = stochastic integration; The stochastic integral is dened as a limit of the partial sums which include the increment of the Wiener process.

J. Hinkel, 1. August, 2007 p.15/20

Wiener Process
The Wiener process is the simplest continuous in time onedimensional stochastic process and its dynamics obeys the equation
dx(t) = dW (t)
(8)

together with the initial condition

x(t = 0) = 0 .
(9)

J. Hinkel, 1. August, 2007 p.16/20

The Wiener process W (t) is normally distributed with mean value:

W (t) = 0

and variance

(W (t))2 W (t)

= t.
(10)

All increments
W (t) = W (t + t) W (t)
(11)

are independent, therefore

W (t) = 0

and

(W (t))2 = t .

(12)

J. Hinkel, 1. August, 2007 p.17/20

Realization of Wiener Process

The increment W (t) (stochastic space step x = W ) can be calculated numerically from standard normally distributed random numbers Z N (0, 1) via W (t) = Z t . (13) Polar method: 1. Generate two uniform distributed random numbers U1 and U2 ; 2. Dene Vi = 2Ui 1; 3. Check the condition that W = V12 + V22 < 1; 4. If yes create Z = V1
2 log(W )/W ;

5. If no generate new random numbers and check this condition again;

J. Hinkel, 1. August, 2007 p.18/20

For the same realization the Box-Mller method proposes the necessary transformation after the generation U1 , U2 (0, 1) only by one step
Z= 2 ln U1 cos (2 U2 ) .
(14)

The difference between both described methods is insignicant and connected only with the simulation time.

J. Hinkel, 1. August, 2007 p.19/20

The uniform distribution at the interval [0, 1] 8 > 0 > < 1 > > : The mean value = 1/2, because of = Z
+

: x<0 : 0<x<1 : x>1

punif orm (x) =

9 > > = > > ;

(15)

x punif orm (x) dx =

x dx =

1 2

(16)

The variance 2 which equals to 2 = Z

+

(x )2 punif orm (x) dx = P12 ` 1 Ui 2 . i=1

1 0

1 x 2

dx =

1 . 12

(17)

The new variable Z =

J. Hinkel, 1. August, 2007 p.20/20