ImRODUCTION TO Lioctalik Processes

Stochases processes are random

quantities that
evelve intime. A soctostic process (Xilyes is a collection
of
(Xelter:= [Xx: tet3,
rv.
time t

X
represents of theprocess
the state at time to
The space
STATE I isthe set
of possible values of Xx
(XE)+et
A process can be in:

if
CONNOU8 Tire is uncountable
e.g. T=R,0
IS T is discrete
TIME
DISCRETE e.g. No
if I is uncountable
continuous space
ASCTE SPACE if I is countable

(Gambler's
-XMP Ruin)
A
gambler plays a
game. At each turn he wins

Is with prob. p and loses 15

with prob.
1-p.
He starts the game with x- and quits when
he reaches either OS or M = > x E.

xx gamblet's wealth attime t

=

(x+) isa 5.p. With TEN and2=90, 1, -, N3

M
X+ ------------
X -o
XXX58 possible trajectory/path
I 1. I I I it
s
SDISCT TIME, STATE
DISCRETE SPOC)
MARKOU CHAINS

The SOINT MSTRUTION of (X=) + isdescribed:

+

50,-,h3
P(Xoxo, Xx., -,
xn n)=

x (n-dimensional pinf)
In general we can decompose it:

P(Xox0,X1 x) P(Xox0)D(Xx)Xo-x0)
=
= = =

P(XxanX x) P(Xxo) P(Xx/Xr

:
=

x)P(Xx/Xxyloxo
= =
=

P(X, ...,Xn xn) P(Xoxr) PXx /X*s-s -,Xxo)

=
=

A discrete time (X),is a MARROW CHAIN if It

-x +- -,Xx) P(X+x (Xxx 1)

=
=

+
+

Mockov Property (MEMORYESS PrP*(

EXAMPGThe MC.
Gamber'swin process is a

E
P it j i 1
=
+

P(X = j(X x=i)

=

Pij
= =

7 -

P 1fj =i 3 -

for je [0,-,M} ⑧ otherwise

TIME 10MOGEN, T
the transition probabilitiesdo not depend an time
When IX1=N we can
arrange probabilities
the transition in
a NxN TRANSITION MTMX
(Pij(i,i eX. E.g. X:(1,2, N3 -,

P., Pi,z.--P,,N
p
Pr
=

:
PN, ----- PN, N

In the following Expat we mightbe interested in e.g:

weather Chain) How
many daysof
ruin on
everage
each year?
(Social Mobility) Do the fractionsoffamily in each
social chess stabilise as increases? To whice level?
I

Chambler's ruin. Whatisthe probability thatthe Gambler

reaches N before 0?
Cleve spread) What isthe probability thatthe harmful
dies out? How
gene A eventually many generations
will be needed for us to happen?
EXAMPL (weather chain)

be the weather condition t in Milan.

Let xt on
day
x x0 piny day and X,I means notrainy day.
=

means

itwill rain tomorrow with prob. a,

Suppose ifitrainstoday,
regarolless of previousdays. If it doesnot
rain
today it

will rain tomorrow with prob B.

TASK Draw the transition matrixP of (x) and itssixty space

DIAGRAM.

x 50,13. Transition matrix:

=

⑧ 3

02 1-x
P =

1 B
-

STATE SPACE MAGRAM:

S &i
A
X 0 A

i
ExMR (Social Mobility)
let (xx) be a family'ssocial to
classatgeneration

Here XS =

lower, medium, uppers coded as

1,2,35 respectively.
The transition probability matrix
of(Xx) is given:

I
0.2 8. 1

p.
8" I
Draw the state space
TASR
diagram.
0.2

686.
-=
-
0.3
-0.1
..... 10.2
③
C
0. A

example (Gamber'sRuin)

TASK Draw the state space

diagram of the
corresponding (XI).
1 -

P 1 P
-

...
- &

.
. -

Aussie P P
eXMR (Gene Spread)
A
gene has two variants(alleles), Aand B, one of
which is harmful. All animalsin a population have either
or B.
allele A [timplified WRIGHT FISHER
-

MODELI
· At each there
generation are N individualsin the population,
N
with fixed
⑧ The Nanimalsproduce a new generation of exactly N

offsprings and then die out.

· Each offspring hasallele Awith probability , where

i is the number individualswith

of allele A in the
N i
generation (and I
-

previous allele B with probability N

independently of the other offsprings
in the
Let us be the number of animalswith allele A
-- In
generation. (X) isa Marteor chain.
TASK Find the conditional Xxx,I
distribution of X+ i
probabilities
the MC transition
and write Pij.