You are on page 1of 60

Chng 1.

Qu trnh Markov

ng Hng Thng
Qu trnh ngu nhin v tnh ton ngu nhin.
NXB i hc quc gia H Ni 2007,
Tr 5 - 63.

T kho: Qu trnh ngu nhin, Qu trnh Markov, Xch Markov, Trng thi hu
han, Trng thi v hn m c.


Ti liu trong Th vin in t H Khoa hc T nhin c th s dng cho mc
ch hc tp v nghin cu c nhn. Nghim cm mi hnh thc sao chp, in n
phc v cc mc ch khc nu khng c s chp thun ca nh xut bn v
tc gi.





Chng 1
Qu trnh Markov
1.1 Xch Markov . . . . . . . . . . . . . . . . . . . . . 5
1.2 Phn loi trng thi xch Markov . . . . . . . . . 20
1.3 Qu trnh Markov . . . . . . . . . . . . . . . . . . 34
1.3.1 Trng hp khng gian trng thi hu hn . . . . 36
1.3.2 Trng hp khng gian trng thi v hn m c 42
1.3.3 Trng hp tng qut . . . . . . . . . . . . . . . . 54
1.4 Bi tp . . . . . . . . . . . . . . . . . . . . . . . . . 58
1.1 Xch Markov
Xt mt h no c quan st ti cc thi im ri rc 0, 1, 2, ... Gi s
cc quan st l X
0
, X
1
, ..., X
n
, ... Khi ta c mt dy cc i lng ngu
nhin (LNN) (X
n
) trong X
n
lg thi ca h ti thi im n. Gi thit
rng mi X
n
, n = 0, 1, ... l mt LNN ri rc. K hiu E l tp gi tr ca
cc (X
n
). Khi E l mt tp hu hn hay m c, cc phn t ca n
c k hiu l i, j, k... Ta gi E l khng gian trng thi ca dy.
6 Chng 1. Qu trnh Markov
nh ngha 1.1. Ta ni rng dy cc LNN (X
n
) l mt xch Markov nu
vi mi n
1
< ... < n
k
< n
k+1
v vi mi i
1
, i
2
, ...i
k+1
E
P{X
n
k+1
= i
k+1
|X
n
1
= i
1
.X
n
2
= i
2
..., X
n
k
= i
k
}
= P{X
n
k+1
= i
k+1
|X
n
k
= i
k
}.
Ta coi thi im n
k+1
l tng lai, n
k
l hin ti cn n
1
,...,n
k1
l qu
kh. Nh vy, xc sut c iu kin ca mt s kin B no trong tng
lai nu bit hin ti v qu kh ca h cng ging nh xc sut c iu kin
ca B nu ch bit trng thi hin ti ca h. chnh l tnh Markov ca
h. i khi tnh Markov ca h cn pht biu di dng: Nu bit trng thi
hin ti ca h th qu kh v tng lai c lp vi nhau.
Gi s P{X
m+n
= j|X
m
= i} l xc sut xch ti thi im m trng
thi i sau n bc, ti thi im m+n chuyn sang trng thi j. y l mt
con s ni chung ph thuc vo i, j, m, n. Nu i lng ny khng ph thuc
m ta ni xch l thun nht. Trong gio trnh ny ta ch xt xch Markov
thun nht.
K hiu
P
ij
= P{X
n+1
= j|X
n
= i}
P
ij
(n) = P{X
m+n
= j|X
m
= i}.
Ta gi (P
ij
, i, j E) l xc sut chuyn sau mt bc hay xc sut chuyn
cn (P
ij
(n), i, j E) l xc sut chuyn sau n bc. Ch rng

jE
P
ij
= 1

jE
P
ij
(n) = 1.
Phn b ca X
0
c gi l phn b ban u. Ta k hiu u
i
= P(X
0
= i).
nh l 1.1. Phn b ng thi ca (X
0
, X
1
, ..., X
n
) c hon ton xc
nh t phn b ban u v xc sut chuyn. C th ta c
P(X
0
= i
0
, X
1
= i
1
, ..., X
n
= i
n
) = u
i
0
P
i
0
i
1
...P
i
n1
in
.
1.1. Xch Markov 7
Tht vy theo cng thc nhn xc sut ta c
P(X
0
= i
0
, X
1
= i
1
, ..., X
n
= i
n
) =
= P(X
0
= i
0
)P(X
1
= i
1
|X
0
= i
0
) ...
P(X
k
= i
k
|X
0
= i
0
, ..., X
k1
= i
k1
) ...
P(X
n
= i
n
|X
0
= i
0
, ..., X
n1
= i
n1
).
S dng tnh Markov ta c
P(X
k
= i
k
|X
0
= i
0
, ..., X
k1
= i
k1
) = P(X
k
= i
k
|X
k1
= i
k1
)
= P
i
k1
i
k
.
Thnh th
P(X
0
= i
0
, X
1
= i
1
, ..., X
n
= i
n
) = u
i
0
P
i
0
i
1
...P
i
n1
in
nh l 1.2. (Phng trnh C - K (Chapman-Kolmogorov))
P
ij
(n +m) =

kE
P
ik
(n)P
kj
(m).
Chng minh. Theo cng thc xc sut y v tnh Markov ta c
P
ij
(n +m) = P(X
n+m
= j|X
0
= i)
=

kE
P(X
n
= k|X
0
= i)P(X
n+m
= j|X
n
= k, X
0
= i)
=

kE
P
ik
(n)P
kj
(m).
Trong trng hp E c d phn t, ta k hiu P = (P
ij
), P(n) = (P
ij
(n))
l cc ma trn vung cp dd. P c gi l ma trn xc sut chuyn, P(n)
c gi l ma trn xc sut chuyn sau n bc. Khi t phng trnh
Chapman-Kolmogorov tng ng vi
P(n +m) = P(n)P(m).
8 Chng 1. Qu trnh Markov
V P = P(1) nn bng quy np ta d thy
P(n) = P
n
.
Gi u
i
(n) = P(X
n
= i). K hiu vecto U(n) = (u
1
(n), ..., u
d
(n)) l vector
hng d - chiu m t phn b ca X
n
, U = U(0) = (u
1
, u
2
, ..., u
d
) l vector
hng d - chiu m t phn b ban u (phn b ca X
0
).
nh l 1.3. Ta c
U(m+n) = U(m)P
n
.
Ni ring
U(n) = UP
n
.
Chng minh. Tht vy, theo cng thc xc sut y ta c
u
j
(m+n) = P(X
n+m
= j) =
d

i=1
P(X
m
= i)P(X
n+m
= j|X
m
= i)
=
d

i=1
u
i
(m)P
ij
(n).
V d 1.1. Cho (
n
), n = 0, 1, 2, .. l dy cc LNN c lp, cng phn b.
Gi s P(
n
= i) = a
i
, i Z. t X
n
=
n

i=1

i
. Khi (X
n
) l mt xch
Markov vi khng gian trng thi Z.
P{X
n+1
= i
n+1
|X
0
= i
0
.X
1
= i
1
..., X
n
= i
n
}
= P{X
n
+
n+1
= i
n+1
|
0
= i
0
,
1
= i
1
i
0
, ...,
n
= i
n
i
n1
}
= P{
n+1
= i
n+1
i
n
} = P{X
n+1
= i
n+1
|X
n
= i
n
}.
Vy th (X
n
) l mt xch Markov vi khng gian trng thi Z. Xc sut
chuyn l P
ij
= a
ij
.
1.1. Xch Markov 9
V d 1.2. (M hnh Ehrenfest) Ta c hai bnh A, B v c d qu cu nh
s 1, 2, ...d. Ti thi im ban u c a qu cu trong A v da qu cu trong
B. Ti mi thi im n ta chn ngu nhin mt s trong tp {1, 2, ...d}. Khi
qu cu mang ch s c chn s c chuyn t bnh ang cha n sang
bnh kia. K hiu X
n
l s qu cu trong bnh A ti thi im n. Hin nhin
(X
n
) l xch Markov. Ta hy tnh xc sut chuyn P(X
n+1
= j|X
n
= i). V
A cha i qu cu nn vi xc sut i/d ta s chn c qu cu t A. Khi
qu cu ny c chuyn sang B. Vy P(X
n+1
= i 1|X
n
= i) = i/d. Tng
t vi xc sut 1 i/d s chn c qu cu ca B v qu cu ny s c
chuyn vo A. Vy P(X
n+1
= i + 1|X
n
= i) = 1 i/d. Thnh th
P
ij
=
_

_
i
d
nu j = i 1
d i
d
nu j = i + 1
0 nu j = i + 1, j = i 1.
(1.1)
M hnh ny c nh vt l ni ting Ehrenfest a ra nm 1907 nhm m
t s truyn nhit gia hai vt th.
V d 1.3. Ta nghin cu mt vn x hi no chng hn vn nghin
ht. Ta k hiu trng thi 0 l khng nghin v trng thi 1 l nghin. n
v thi gian l mt qu (3 thng). Thng k nhiu nm cho thy xc sut
mt ngi khng nghin sau mt qu vn khng nghin l 0,99 v xc
sut mt ngi nghin sau mt qu vn tip tc nghin l 0,88. Nh vy
trng thi ca mt ngi (nghin hay khng nghin) c m t bi mt xch
Markov vi hai trng thi E = {0, 1} vi ma trn xc sut chuyn nh sau
P =
_
0, 99 0, 01
0, 12 0, 88
_
Gi s lc u c 17% s ngi nghin. Nh vy phn b ban u l U(0) =
(0, 83, 0, 17). Sang qu hai, theo nh l 1.3 phn b s ngi nghin v khng
nghin s l
U(1) = U(0)P = (0.83, 0.17)
_
0, 99 0, 01
0, 12 0, 88
_
= (0, 845, 0, 155).
10 Chng 1. Qu trnh Markov
Sang qu ba na phn b s ngi khng nghin v nghin s l
U(2) = U(1)P = (0.845, 0.155)
_
0, 99 0, 01
0, 12 0, 88
_
= (0, 855, 0, 145)
tc l lc ny c 14,5% s ngi nghin.
V d 1.4. Gi s ta c d ca hng k hiu l 1, 2, ...d cng bn mt sn
phm no . Khch hng c th chn mua sn phm mt trong d ca hng
ny tu theo s thch ca h v trong tng thng h khng thay i ch mua
hng. Gi X
n
l ca hng m khch hng chn mua sn phm thng th
n. y l mt xch Markov c d trng thi, xc sut chuyn P
ij
c ngha l
xc sut khch hng, hin ti ang mua hng ti ca hng i sang thng
sau chuyn sang mua ca hng j. Xt d = 3 v ma trn xc sut chuyn
l
P =
_
_
_
0, 800 0, 100 0, 100
0, 070 0, 900 0, 030
0, 083 0, 067 0, 850
_
_
_
Gi s thng ging ca hng 1 chim 20% khch hng, ca hng 2 chim
50% khch hng v ca hng 3 chim 30% khch hng. Nh vy phn b ban
u l U(0) = (0, 2, 0, 5, 0, 3). Sang thng 2 phn b khch hng trong 3 ca
hng s l U(1) = U(0)P = (0, 22, 0, 49, 0, 29). Sang thng 3 phn b khch
hng trong 3 ca hng s l U(2) = U(1)P = (0, 234, 0, 483, 0, 283). Tip tc
qu trnh nh vy ta c th tnh thng 12 phn b khch hng trong 3 ca
hng s l U(11) = (0, 270, 0, 459, 0, 271) tc l trong thng 12 ca hng 1
chim 27% khch hng, ca hng 2 chim 45,9% khch hng v ca hng 3
chim 27,1% khch hng.
V d 1.5. Cho (X
n
) l mt xch Markov c 2 trng thi E = {0, 1} vi ma
trn xc sut chuyn l
P =
_
1 a a
b 1 b
_
1.1. Xch Markov 11
trong a, b > 0, 0 < a +b < 1. Ta hy tm biu thc ca ma trn xc sut
chuyn sau n bc P(n). Ta c
P
00
(n + 1) = P(X
n+1
= 0|X
0
= 0)
= P(X
n
= 0|X
0
= 0)P(X
n+1
= 0|X
n
= 0)+
+P(X
n
= 1|X
0
= 0)P(X
n+1
= 0|X
n
= 1)
= P
00
(n)(1 a) + (1 P
00
(n))b. (1.2)
Nu k hiu u
n
= P
00
(n), q = 1 a b t (1.2) ta c cng thc truy hi sau
u
n+1
= u
n
(1 a) + (1 u
n
)b = (1 a b)u
n
+b = qu
n
+b.
Ch rng u
0
= 1, t cng thc truy hi trn d thy
u
n
= q
n
+
b(1 q
n
)
1 q
= (1 a b)
n
+
b
a +b
(1 (1 a b)
n
)
hay
P
00
(n) =
b
a +b
+ (1 a b)
n
a
a +b
.
Suy ra
P
01
(n) = 1 P
00
(n) =
a
a +b
(1 a b)
n
a
a +b
.
Tng t ta c
P
10
(n + 1) = P(X
n+1
= 0|X
0
= 1)
= P(X
n
= 0|X
0
= 1)P(X
n+1
= 0|X
n
= 0)+
+P(X
n
= 1|X
0
= 1)P(X
n+1
= 0|X
n
= 1)
= P
10
(n)(1 a) + (1 P
10
(n))b. (1.3)
Nu k hiu v
n
= P
00
(n), q = 1 a b t (1.3) ta c cng thc truy hi sau
v
n+1
= v
n
(1 a) + (1 v
n
)b = (1 a b)v
n
+b = qv
n
+b.
Ch rng v
0
= 0, t cng thc trn ta thu c
v
n
=
b(1 q
n
)
(1 q)
=
b
a +b
(1 (1 a b)
n
).
12 Chng 1. Qu trnh Markov
Suy ra
P
10
(n) =
b
a +b
(1 a b)
n
b
a +b
P
11
(n) = 1 P
10
(n) =
a
a +b
+ (1 a b)
n
b
a +b
.
Vit di dng ma trn ta c
P(n) =
1
a +b
_
b a
b a
_
+
(1 a b)
n
a +b
_
a a
b b
_
.
nh ngha 1.2. Phn b ban u U = (u
i
), i E c gi l phn b dng
nu ta c U(n) = U vi mi n tc l u
i
(n) = u
i
i E, n. Khi dy
(X
n
) c cng phn b.
T nh l 3 ta suy ra U = (u
i
) l phn b dng nu v ch nu
1. u
i
0 v

iE
u
i
= 1
2. u
j
=

iE
u
i
P
ij
j E.
V d 1.6.
Cho (X
n
) l mt xch Markov c 3 trng thi E = {1, 2, 3} vi ma trn xc
sut chuyn l
P =
_
_
_
1/3 1/3 1/3
1/4 1/2 1/4
1/6 1/3 1/2
_
_
_
Hy tm tt c cc phn b dng.
t U = (x, y, z). Khi U l phn b dng khi v ch khi x, y, z l
nghim khng m ca h sau
_

_
x/3 +y/4 +z/6 = x
x/3 +y/2 +z/3 = y
x/3 +y/4 +z/2 = z
x +y +z = 1.
1.1. Xch Markov 13
T phng trnh th nht v th hai ca h kh z ta rt ra y = 5x/3. T
z = 3x/2. Th vo phng trnh (4) ta thu c x = 6/25, y = 10/25, z =
9/25.
V d 1.7.
Cho (X
n
) l mt xch Markov c 2 trng thi E = {0, 1} vi ma trn xc
sut chuyn l
P =
_
1 a a
b 1 b
_
trong a, b > 0, 0 < a +b < 1 (xem v d 1.5). Ta hy tm phn b dng.
t U = (x, y). Khi U l phn b dng khi v ch khi x, y l nghim
khng m ca h sau
_

_
(1 a)x +by = x
ax + (1 b)y = y
x +y = 1.
Phng trnh (1) v (2) ca h tng ng vi ax = by hay x =
by
a
. Th
vo phng trnh (3) ca h ta thu c
x =
b
a +b
, y =
a
a +b
.
Nh sau ny ta s thy phn b dng khng phi bao gi cng tn ti. Cu
hi t ra l: Vi iu kin no th tn ti phn b dng? Phn b dng nu
tn ti th c duy nht khng?
nh l 1.4. Gi s (X
n
) l xch Markov vi khng gian trng thi E =
{1, 2, ...} vi ma trn xc sut chuyn P = (P
ij
) v ma trn xc sut chuyn
sau n bc l P(n) = (P
ij
(n)). Gi s rng vi mi i, j E tn ti gii hn
lim
n
P
ij
(n) =
j
v gii hn ny khng ph thuc i. Khi
14 Chng 1. Qu trnh Markov
1.

jE

j
1 v
j
=

iE

i
P
ij
.
2. Hoc
j
= 0 vi mi j E, hoc

jE

j
= 1.
3. Nu

jE

j
= 1 th U = (
1
,
2
, ...) l phn b dng v phn b dng l
duy nht. Nu
j
= 0 vi mi j E th phn b dng khng tn ti.
Chng minh. 1. Theo b Fatou ta c

jE

j
=

jE
lim
n
P
ij
(n) liminf
n

jE
P
ij
(n) = 1.
S dng b Fatou v phng trnh C-K ta c

iE

i
P
ij
=

iE
lim
n
P
ki
(n)P
ij
liminf
n

iE
P
ki
(n)P
ij
= liminf
n
P
kj
(n + 1) =
j
.
t s
j
=
j

iE

i
P
ij
0 j E. Ta c

jE
s
j
=

jE

jE

iE

i
P
ij
=

jE

iE

jE

i
P
ij
=

jE

iE

jE
P
ij
=

jE

iE

i
= 0.
Vy s
j
= 0 j E hay
j
=

iE

i
P
ij
j E.
2. Ta c

j
=

iE

i
P
ij
=

iE
(

kE

k
P
ki
)P
ij
=

kE

k
(

iE
P
ki
P
ij
) =

kE

k
P
kj
(2).
1.1. Xch Markov 15
Bng quy np d thy vi mi n

j
=

kE

k
P
kj
(n).
V chui hi t u i vi n nn

j
= lim
n

kE

k
P
kj
(n) =

kE

k
lim
n
P
kj
(n) =
j

kE

k
Suy ra

j
_
1

kE

k
_
= 0 j E.
Vy nu

kE

k
< 1 th
j
= 0 j E.
3. Nu

kE

k
= 1 th t khng nh 1 ta suy ra = (
1
,
2
, ...) l phn
b dng. Ta chng minh y l phn b dng duy nht. Tht vy gi
s U = (u
i
) l phn b dng. Lp lun tng t nh trn ta c
u
j
=

kE
u
k
P
kj
(n).
V chui hi t u i vi n nn
u
j
=

kE
u
k
lim
n
P
kj
(n) =

kE
u
k

j
=
j
.
Do nu

jE

j
< 1 th phn b dng khng tn ti. Nu

jE

j
= 1 th
= (
1
,
2
, ...) l phn b dng duy nht.
nh ngha 1.3. Gi s (X
n
) l xch Markov vi khng gian trng thi
E = {1, 2, ...} vi ma trn xc sut chuyn P = (P
ij
) v ma trn xc sut
chuyn sau n bc l P(n) = P
ij
(n). Ta ni rng xch c phn b gii hn
nu vi mi i, j E tn ti gii hn
lim
n
P
ij
(n) =
j
.
Gii hn ny khng ph thuc i E v

jE

j
= 1. Ni cch khc, vecto gii
hn = (
1
,
2
, ...) lp thnh mt phn b xc sut trn E.
16 Chng 1. Qu trnh Markov
ngha ca phn b gii hn l nh sau: Gi u
i
(n) = P(X
n
= i). K
hiu vecto U(n) = (u
1
(n), u
2
(n), ...) l vector hng d-chiu m t phn b
ca X
n
. Ta c
P(X
n
= j) =

E
P(X
0
= i)P
ij
(n).
Do
lim
n
P{X
n
= j} =

E
P{X
0
= i} lim
n
P
ij
(n)
=

E
P{X
0
= i}
j
=
j
.
Vy phn b U(n) ca X
n
hi t ti phn b gii hn . Khi n kh ln ta
c P(X
n
= j)
j
.
Theo nh l 1.4 nu phn b gii hn tn ti th phn b dng cng
tn ti v duy nht. Hn na hai phn b ny trng nhau. Tuy nhin iu
ngc li khng ng tc l c nhng xch Markov c tn ti phn b dng
nhng khng tn ti phn b gii hn.
V d 1.8. Cho xch Markov (X
n
) c hai trng thi vi ma trn xc sut
chuyn l
P =
_
0 1
1 0
_
.
Ta c
P(2n) =
_
0 1
1 0
_
v
P(2n + 1) =
_
0 1
1 0
_
.
Do khng tn ti lim
n
P(n). Tuy nhin d dng kim tra c =
(1/2, 1/2) l phn b dng duy nht.
1.1. Xch Markov 17
Mt trong nhng bi ton quan trng trong nghin cu xch Markov l
tm nhng iu kin m bo s tn ti ca phn b gii hn v s tn
ti ca phn b dng. Di y l mt nh l nh vy.
nh l 1.5. Cho (X
n
) l xch Markov vi khng gian trng thi hu hn
E = {1, 2, ..., d} vi ma trn xc sut chuyn sau n bc l P(n) = (P
ij
(n)).
Khi c tn ti phn b gii hn = (
1
, ...,
d
) vi
j
> 0 j E khi v
ch khi xch l chnh quy theo ngha: Tn ti n
0
sao cho P
ij
(n
0
) > 0, i, j E.
Chng minh. Gi thit xch l chnh quy. Ta c nh j v t
m
j
(n) = min
iE
P
ij
(n)
M
j
(n) = max
iE
P
ij
(n).
Ta c
P
ij
(n + 1) =

k
P
ik
P
kj
(n)

k
P
ik
m
j
(n) = m
j
(n).
Suy ra m
j
(n + 1) m
j
(n). Vy dy (m
j
(n)), n = 1, 2, ... l dy tng v b
chn trn bi 1, do tn ti gii hn
lim
n
m
j
(n) = a
j
.
Lp lun tng t dy (M
j
(n)), n = 1, 2, ...) l dy gim b chn bi 0, do
tn ti gii hn
lim
n
M
j
(n) = A
j
.
Ta c m
j
(n) P
ij
(n) M
j
(n) do nh l c chng minh nu ta ch ra
a
j
= A
j
. K hiu r = min
i,j
P
ij
(n
0
) > 0. Ta c P
ik
(n
0
) r.1 P
jk
(n) nn
P
ik
(n
0
) rP
jk
(n) i, thnh th
P
ij
(n
0
+n) =

k
P
ik
(n
0
)P
kj
(n)
=

k
(P
ik
(n
0
) rP
jk
(n)) P
kj
(n) +r

k
P
jk
(n)P
kj
(n)
m
j
(n)

k
(P
ik
(n
0
) rP
jk
(n)) +rP
jj
(2n)
= m
j
(n)(1 r) +rP
jj
(2n).
18 Chng 1. Qu trnh Markov
V bt ng thc ny ng vi mi i nn ta c
m
j
(n
0
+n) m
j
(n)(1 r) +rP
jj
(2n).
Tng t ta c
M
j
(n
0
+n) M
j
(n)(1 r) +rP
jj
(2n).
Suy ra
M
j
(n
0
+n) m
j
(n
0
+n) (1 r) (M
j
(n) m
j
(n)) . (1.4)
Ta chng minh quy np rng vi mi k
M
j
(kn
0
+ 1) m
j
(kn
0
+ 1) (1 r)
k
(M
j
(1) m
j
(1)) . (1.5)
Tht vy vi k = 1 ng (Cho n = 1 (1.4)). Gi s ng vi k. Ta c
M
j
((k + 1)n
0
+ 1) m
j
((k + 1)n
0
+ 1)
= M
j
(kn
0
+ 1 +n
0
) m
j
(kn
0
+ 1 +n
0
)
(1 r) ((M
j
(kn
0
+ 1) m
j
(kn
0
+ 1))
(1 r)
k+1
(M
j
(1) m
j
(1)) .
Cho k trong (1.5) ta nhn c A
j
a
j
0. V A
j
a
j
0 nn ta kt
lun A
j
= a
j
.
o li gi s vi mi i, j E tn ti lim
n
P
ij
(n) =
j
> 0. Khi tn
ti n
0
(i, j) sao cho P
ij
(n) > 0 n > n
0
(i, j). t n
0
= max
i,j
n
0
(i, j) ta c
P
ij
(n) > 0 i, j En > n
0
V d 1.9. Mi ngi dn trong mt vng no c th trong ba tng lp:
giu, trung lu v ngho. Con ci ca h c th trong mt trong ba tng lp
ni trn vi cc xc sut khc nhau tu thuc vo vic h ang trong tng
lp no. Gi s bng thng k ngi ta xc nh c: Nu mt ngi giu
th vi xc sut 0,448 con h giu, vi xc sut 0,484 con h trung lu vi
xc sut 0,068 con h ngho. Tng t, vi mt ngi trung lu th xc sut
con h giu, trung lu hay ngho tng ng l 0,054. 0,699 v 0,247. Vi
1.1. Xch Markov 19
mt ngi ngho th xc sut con h giu, trung lu hay ngho tng ng
l 0,011, 0,503 v 0,486. Nh vy s thay i trng thi ca mt gia nh
trong x hi t th h ny qua th h khc c th m t bi mt xch Markov
ba trng thi : 1(giu), 2(trung lu), 3(ngho) vi xc sut chuyn nh sau
P =
_
_
_
0, 448 0, 484 0, 068
0, 054 0, 699 0, 247
0, 011 0, 503 0, 486
_
_
_
.
Xch Markov ny l chnh quy. Thnh th tn ti phn b gii hn =
(
1
,
2
,
3
). Phn b ny chnh l phn b dng duy nht v c tm bng
cch gii h phng trnh sau
(
1
,
2
,
3
)P = (
1
,
2
,
3
).
Gii ra ta tm c
1
= 0, 067;
2
= 0, 624;
3
= 0, 369. Nh vy qua nhiu
th h vng dn c ni trn s c 6,7% ngi giu, 62,4% trung lu v
36.9% ngi ngho.
V d 1.10. Xt xch Markov c d trng thi E = {1, 2, ..., d} v ma trn
xc sut chuyn ca n l chnh quy ng thi l mt ma trn kp ngha l

jE
P
ij
=

iE
P
ij
= 1.
Theo nh l trn, phn phi gii hn tn ti. Ta hy tm phn b gii hn
.Ta nhn xt rng phn b u = (1/d, 1/d, ..., 1/d) l phn b dng. Tht
vy t pi
j
= 1/d ta c

kE
pi
k
P
kj
= 1/d

kE
P
kj
= 1/d =
j
.
V phn b dng l duy nht v phn b gii hn chnh l phn b dng nn
ta kt lun rng phn b gii hn l phn b u = (1/d, 1/d, ..., 1/d) .
Chng hn ta tung con xc sc lin tip mt cch c lp. K hiu
n
l
s chm xut hin ln gieo th n, S
n
=

n
k=1

k
. S
n
l mt xch Markov
20 Chng 1. Qu trnh Markov
vi khng gian trng thi E = {1, 2, ...}. Gi X
n
l s d khi chia S
n
cho
7. Khi X
n
cng l mt l mt xch Markov vi khng gian trng thi
E = {0, 1, 2, ..., 6}. Ma trn xc sut chuyn ca X
n
l
P =
_
_
_
_
_
_
_
_
_
_
_
_
_
0 1/6 1/6 1/6 1/6 1/6 1/6
1/6 0 1/6 1/6 1/6 1/6 1/6
1/6 1/6 0 1/6 1/6 1/6 1/6
1/6 1/6 1/6 0 1/6 1/6 1/6
1/6 1/6 1/6 1/6 0 1/6 1/6
1/6 1/6 1/6 1/6 1/6 0 1/6
1/6 1/6 1/6 1/6 1/6 1/6 0
_
_
_
_
_
_
_
_
_
_
_
_
_
.
Xch Markov ny chnh quy v ma trn xc sut chuyn sau 2 bc
P(2) = P
2
=
_
_
_
_
_
_
_
_
_
_
_
_
_
1/6 5/36 5/36 5/36 5/36 5/36 5/36
5/36 1/6 5/36 5/36 5/36 5/36 5/36
5/36 5/36 1/6 5/36 5/36 5/36 5/36
5/36 5/36 5/36 1/6 5/36 5/36 5/36
5/36 5/36 5/36 5/36 1/6 5/36 5/36
5/36 5/36 5/36 5/36 5/36 1/6 5/36
5/36 5/36 5/36 5/36 5/36 5/6 1/6
_
_
_
_
_
_
_
_
_
_
_
_
_
c tt c cc phn t l s dng.
Vy phn b gii hn l
= (1/7, 1/7, 1/7, 1/7, 1/7, 1/7, 1/7, 1/7).
1.2 Phn loi trng thi xch Markov
gii quyt y hn bi ton v s tn ti ca phn b dng cng nh
bi ton v s tn ti ca phn b gii hn dn ta n vic phn loi cc
trng thi ca xch Markov nh sau:
1.2. Phn loi trng thi xch Markov 21
nh ngha 1.4. Ta ni rng trng thi i n c trng thi j v k hiu l
i j nu tn ti n o sao cho P
ij
(n) > 0. ( Ta quy c P
ii
(0) = 1, P
ij
(0) =
0 nu i = j)
Hai trng thi i v j c gi l lin lc c nu i j v j i. Trong
trng hp ta vit i j.
B 1.1 (Tnh cht bc cu). Nu i j, j k th i k.
Tht vy theo gi thit tn ti n, m sao cho
P
ij
(n) > 0, P
jk
(m) > 0.
Theo phng trnh Chapman - Kolmogorov ta c
P
ik
(n +m) =

jE
P
ij
(n)P
jk
(m) P
ij
(n)P
jk
(m) > 0.
T b , d kim tra rng quan h "lin lc c" l mt quan h tng
ng trn khng gian trng thi E. Theo quan h ny khng gian E c
phn hoch thnh cc lp ri nhau. Hai trng thi bt k cng thuc mt
lp th lin lc c vi nhau, hai trng thi khc lp khng th lin lc c
vi nhau.
nh ngha 1.5. Xch Markov c gi l ti gin nu hai trng thi bt k
l lin lc c. C ngha l theo cch phn lp trn th E khng th phn
hoch thnh cc lp con nh hn.
Nu xch khng ti gin th E c phn hoch thnh cc lp ri nhau
E = E
1
E
2
... E
k
. C th xem mi E
k
l khng gian trng thi ca xch
Markov ti gin. Nh vy vic nghin cu xch Markov c th quy v vic
nghin cu cc xch ti gin.
V d 1.11. Cho xch Markov vi 5 trng thi E = {1, 2, 3, 4, 5} vi ma trn
xc sut chuyn l
P =
_
P
1
0
0 P
2
_
22 Chng 1. Qu trnh Markov
trong
P
1
=
_
1/3 2/3
1/4 3/4
_
v
P
2
=
_
_
_
0 1 0
1/2 0 1/2
0 1 0
_
_
_
.
Khi
P(n) = P
n
=
_
P
n
1
0
0 P
n
2
_
.
Do E = E
1
E
2
vi E
1
= {1, 2}, E
2
= {3, 4, 5}.
V d 1.12. Cho xch Markov vi 4 trng thi E = {1, 2, 3, 4} vi ma trn
xc sut chuyn l
P
2
=
_
_
_
_
_
0 0 1/2 1/2
0 0 1/2 1/2
1/2 1/2 0 0
1/2 1/2 0 0
_
_
_
_
_
Xch ny l ti gin. Tht vy r rng 1 3, 1 4, 2 3, 2 4. Ta c
1 3, 3 2 suy ra 1 2. Li c 2 3, 3 1 suy ra 2 1. Vy 1 2 .
Tng t ta c 3 4. Vy hai trng thi bt k l lin lc c do y
l xch ti gin.
nh ngha 1.6. Chu k ca trng thi i k hiu l d(i) l c chung ln
nht ca tt c cc s nguyn dng n 1 m P
ii
(n) > 0. Nu P
ii
(n) = 0
vi mi n 1 th ta quy c t d(i) = 0.
nh l 1.6. Nu i j th d(i) = d(j). Vy cc trng thi cng mt lp c
cng mt chu k d v ta gi s d chung l chu k ca lp.
Chng minh. Do i j nn tn ti k, l sao cho P
ij
(k) > 0, P
ji
(l) > 0. Theo
phng trnh C-K ta c P
ii
(k + l) =

hE
P
ih
(k)P
hi
(l) P
ij
(k)P
ji
(l) > 0.
Vy d(i)|k +l. Gi s n 1 sao cho P
jj
(n) > 0. S dng phng trnh C-K
1.2. Phn loi trng thi xch Markov 23
nh trn ta c P
ii
(k +l +n) P
ij
(k)P
jj
(n)P
ji
(l) > 0. Vy d(i)|k +l +n
d(i)|n. Vy d(i)|d(j). Tng t d(j)|d(i). Thnh th d(i) = d(j).
Gi s d l chu k ca mt xch ti gin vi khng gian trng thi E.
Nu d = 1 ta ni rng xch khng c chu k. Nu d > 1 th c th chng
minh rng E c phn hoch thnh d tp conE = C
0
C
1
... C
d1
sao
cho sau mt bc h s chuyn t mt trng thi thuc C
k
sang mt trng
thi thuc C
k+1
(quy c C
d
= C
0
). V vy mi tp con c th ly lm khng
gian trng thi ca mt xch Markov mi. Xch ny ti gin v khng c chu
k. Tm li chng ta c th quy vic nghin cu xch Markov tng qut v
vic nghin cu xch ti gin, khng c chu k.
nh ngha 1.7. K hiu f
ii
(n) l xc sut h xut pht t i ln u tin
quay li i thi dim n. Ngha l
f
ii
(n) = P (X
n
= i, X
n1
= i, ..., X
1
= i|X
0
= i)
v k hiu
f

ii
=

n=1
f
ii
(n)
l xc sut h xut pht t i quay tr li i sau mt s hu hn bc. Nu
f

ii
= 1 ta ni i l trng thi hi quy (quay li). Nu tri li f

ii
< 1 ta ni i
l trng thi khng hi quy.
nh l c bn sau y cho ta mt tiu chun xc nh tnh hi quy
ca mt trng thi.
nh l 1.7. Trng thi i l hi quy khi v ch khi

n=1
P
ii
(n) = . (1.6)
Chng minh. Chng minh s dng hai b sau
24 Chng 1. Qu trnh Markov
B 1.2. Ta c
P
ii
(n) =
n

k=0
f
ii
(k)P
ii
(n k)
trong f
ii
(0) = 0.
Chng minh b 1.2. Vi mi 0 k n gi E
k
l bin c :" X
n
= i v
h ln u tin quay li i bc th k " ta c
P
ii
(n) = P(X
n
= i|X
0
= i) =
n

k=0
P(E
k
)P(X
n
= i|E
k
, X
0
= 1)
=
n

k=0
f
ii
(k)P(X
n
= i|X
k
= i) =
n

k=0
f
ii
(k)P
ii
(n k).
B 1.3. (Aben)
(i) Nu chui

k=0
a
k
= a hi t th
lim
s1

k=0
a
k
s
k
=

k=0
a
k
= a.
(ii) Nu a
k
0 v lim
s1

k=0
a
k
s
k
= a < th

k=0
a
k
= lim
s1

k=0
a
k
s
k
= a.
Ta bt u chng minh nh l. Xt cc chui lu tha sau
P
ii
(s) =

n=0
P
ii
(n)s
n
, , |s| < 1
F
ii
(s) =

n=0
f
ii
(n)s
n
, , |s| < 1
Ta c
F
ii
(s)P
ii
(s) =

n=0
c
k
s
k
, |s| < 1
1.2. Phn loi trng thi xch Markov 25
vi c
n
=

n
k=0
f
ii
(k)P
ii
(n k) = P
ii
(n) n 1 theo b 1. V c
0
=
0, P
ii
(0) = 1 nn ta suy ra F
ii
(s)P
ii
(s) = P
ii
(s) 1 hay
P
ii
(s) =
1
1 F
ii
(s)
. (1.7)
Gi s i hi quy tc l

n=0
f
ii
(n) = 1. Theo b Abel
lim
s1

F
ii
(s) = lim
s1

n=0
f
ii
(n)s
n
= 1.
T (1.7) suy ra
lim
s1

P
ii
(s) = lim
s1

n=0
P
ii
(n)s
n
= .
Vy li theo b Abel (ii) ta c

n=0
P
ii
(n) = .
o li gi s i khng hi quy tc l

n=0
f
ii
(n) < 1. S dng b Abel (i)
v h thc (1.7) ta rt ra lim
s1

P
ii
(s) < . Li p dng b Abel (ii) ta
thu c

n=0
P
ii
(n) < .
nh l 1.8. Nu i j v j hi quy th i hi quy.
Chng minh. Theo gi thit tn ti m, n sao cho P
ij
(n) > 0, P
ji
(m) > 0. Vi
mi s nguyn dng h t phng trnh C-P suy ra
P
ii
(n +h +m) P
ij
(n)P
jj
(h)P
ji
(m).
Vy

h=1
P
ii
(n +h +m) P
ij
(n)P
ji
(m)

h=1
P
jj
(h) = .
Thnh th i hi quy.
26 Chng 1. Qu trnh Markov
V d 1.13. Cho (r
n
) l dy cc LNN c lp c phn b xc sut nh sau
P(r
n
= 1) = p, P(r
n
= 1) = q, 0 < p < 1, p +q = 1.
(Dy ny c gi l dy Rademakher). Xt dy (X
n
) xc nh nh sau
X
0
= a, X
n+1
= X
n
+r
n+1
.
. Khi (X
n
) lp thnh xch Markov vi khng gian trng thi E = {0
1, 2}xc sut chuyn l P = (P
ij
)
P
ij
=
_

_
q nu j = i 1
p nu j = i + 1
0 nu j = i + 1, j = i 1.
Xch ny c gi l du ng ngu nhin 1 chiu m t s chuyn ng ngu
nhin ca mt ht trn ng thng: Sau mi n v thi gian ht dch sang
phi vi xc sut p v dch sang tri vi xc sut q. D thy y l mt xch
ti gin c chu k d = 2 v
P
ii
(2n) =
_
2n
n
_
p
n
q
n
. P
ii
(2n + 1) = 0.
S dng cng thc Stirling
n!

2ne
n
n
n
ta c
P
ii
(2n)
(4pq)
n

n
.
Nu du ng ngu nhin l i xng p = q = 1/2 th P
ii
(2n)
1

n
do

n
P
ii
(n) = .
1.2. Phn loi trng thi xch Markov 27
Vy theo nh l 1.7 mi trng thi u hi quy. Nu p = q th 4pq = a < 1
cho nn

n
P
ii
(n) < v chui

a
n

n
hi t khi a < 1.
Do theo nh l 1.7 mi trng thi l khng hi quy. V mt trc gic ta
thy nu p > q th c mt xc sut dng ht xut pht t trng thi i s
i sang bn phi mi mi ( sang bn tri mi mi nu p < q) khng quay li
im xut pht.
V d 1.14. Xt du ng ngu nhin ca mt ht trn li im nguyn trn
mt phng. Gi s xc sut ht dch ln trn, dch xung di mt n v
(theo phng thng ng), dch sang phi,sang tri mt n v (theo phng
nm ngang) u bng nhau v bng 1/4. C th thy rng P
00
(2n + 1) = 0
v
P
00
(2n) =

i,ji+j=n
(2n)!
i!i!j!j!
(1/4)
2n
= (1/4)
2n
_
2n
n
_
n

i=0
_
n
i
__
n
n i
_
= (1/4)
2n
_
2n
n
_
2
.
Cng thc Stirling cho ta
P
00
(2n)
1
n
.
Vy

n
P
00
(n) = .
Vy trng thi 0 l hi quy. V xch l ti gin (d thy) nn mi trng thi
u hi quy.
Ngi ta chng minh c mt iu th v l vi du ng ngu nhin
i xng trong khng gian ba chiu, mi trng thi u khng hi quy.
28 Chng 1. Qu trnh Markov
nh l 1.9. K hiu Q
ii
l xc sut h xut pht t i quay li i v s
ln, Q
ij
l xc sut h xut pht t i i qua j v s ln. Khi
(i) Nu i hi quy th Q
ii
= 1, nu i khng hi quy th Q
ii
= 0.
(ii) Nu i hi quy i j th Q
ij
= 1. Ni ring, vi xc sut 1 h xut pht
t i sau mt s hu hn bc s i qua j.
Chng minh.
(i) Gi s Q
(m)
ii
l xc sut h quay li i t nht m ln. S dng cng
thc xc sut y v tnh Markov ca h ta thu c phng trnh
Q
(m)
ii
=

k=1
f

ii
(k)Q
(m1)
ii
= f

ii
Q
(m1)
ii
T
Q
m
ii
= (f

ii
)
m1
Q
1
ii
= (f

ii
)
m
v r rng Q
1
ii
= f

ii
. V rng Q
ii
= lim
m
Q
(m)
ii
ta rt ra Q
ii
= 0 hay 1
tu theo f

ii
< 1 hay bng 1.
(ii) Gi f

ji
=

k=1
f
ji
(k) l xc sut h xut pht t j s ving thm i
sau mt s hu hn bc. S dng cng thc xc sut y v tnh
Markov ca h ta thu c phng trnh
Q
jj

k=1
f
ji
(k)Q
ij
+ 1 f

ji
= Q
ij
f

ji
+ 1 f

ji
. (1.8)
(1.9)
V j hi quy theo (1) Q
jj
= 1. Vy t (??)
1 Q
ij
f

ji
+ 1 f

ji
f

ji
Q
ij
f

ji
.
V j i nn f

ji
> 0 Suy ra Q
ij
= 1.
1.2. Phn loi trng thi xch Markov 29
V d 1.15. (S ph sn chc chn ca ngui chi c bc) Mt ngi vo
sng bc vi s tin trong ti l 1000 la v chi nh bc nh sau: Mi
vn chi anh ta tung mt ng tin cn i ng cht. Nu ng tin ra mt
nga anh ta c mt la, nu ra mt sp anh ta mt mt la. Gi X
n
l s tin anh ta c sau n vn chi. Khi X
0
= 1000 v X
0
, X
1
, ... l mt
du ng ngu nhin i xng vi trng thi ban u X
0
= 1000. Theo nh
l trn vi xc sut 1 X
n
s ving thm trng thi 0. Ni cch khc sm hay
mun anh ta s nhn ti.
nh l 1.10. Cho (X
n
) l xch ti gin khng hi quy. Khi vi mi i, j

n=1
P
ij
(n) < .
Ni ring
lim
n
P
ij
(n) = 0
v xch khng tn ti phn b dng.
Chng minh. Chng minh tng t nh b 2 ta c
P
ij
(n) =
n

k=1
f
ij
(k)P
jj
(n k). (1.10)
Vy

n=1
P
ij
(n) =

n=1
n

k=1
f
ij
(k)P
jj
(n k)
=

k=1
f
ij
(k)

n>k
P
jj
(n k)
=

k=1
f
ij
(k)

m=1
P
jj
(m)
= f

ij

m=1
P
jj
(m)

m=1
P
jj
(m) < .
30 Chng 1. Qu trnh Markov
nh l 1.11. Cho (X
n
) l xch ti gin hi quy khng c chu k . Khi
vi mi i, j
lim
n
P
ij
(n) =
1

j
=

k=1
kf
jj
(k).
Chng minh. Chng minh da c bn trn mt kt qu sau y ca gii tch
m ta s pht biu di dng mt b ( khng chng minh):
B 1.4. Cho (f
n
) l mt dy cc s khng m c tng bng 1 v c
chung ln nht ca tt c cc s j > 0 m f
j
> 0 bng 1. Cho (u
n
) l dy
xc nh truy hi theo cch sau
u
0
= 1, u
n
=
n

k=1
f
k
u
nk
.
Khi
lim
n
u
n
=
1

k=1
kf
k
.
C nh j. t u
n
= P
jj
(n), f
k
= f
jj
(k). B 4 cho php ta kt lun
lim
n
P
jj
(n) =
1

j
.
Tip theo vi quy c P
ij
(s) = 0 nu s < 0 ta vit li h thc (1.10) di
dng
P
ij
(n) =

k=1
f
ij
(k)P
jj
(n k).
Ch rng
lim
n
P
jj
(n k) =
1

j
1.2. Phn loi trng thi xch Markov 31
v

k=1
f
ij
(k) = f

ij
= Q
ij
= 1 ( theo nh l 1.7) p dng nh l hi t b
chn ta thu c
lim
n
P
ij
(n) =

k=1
f
ij
(k) lim
n
P
jj
(n k)
= f

ij
1

j
=
1

j
.
Nu i l trng thi hi quy th
i
chnh l thi gian trung bnh ( hay s
bc trung bnh) h ln u tin quay li i. Thi gian trung bnh ny c
th hu hn hay v hn. Ta c nh ngha sau:
nh ngha 1.8. Trng thi hi quy i c gi l trng thi hi quy dng
nu
i
< v c gi l trng thi hi quy khng nu
i
= .
V d 1.16. Gi s (X
n
) l du ng ngu nhin i xng trn ng thng.
Trong v d ta thy mi trng thi l hi quy. Ta chng minh mi trng
thi l hi quy khng. Tht vy ta c
P
ii
(2n) =
_
2n
n
_
(1/2)
n
(1/2)
n
, P
ii
(2n + 1) = 0.
Thnh th
P
ii
(s) =

n
P
ii
(n)s
n
=

m
_
2m
m
_
(1/2)
2m
s
2m
=

m
_
2m
m
_
(s
2
/4)
m
= ((1 s
2
)
1/2
.
T phng trnh (1.7) ta suy ra
F
ii
(s) = 1 P
ii
(s)
1
= 1 (1 s
2
)
1/2
. (1.11)
Theo b Abel

i
=

k
kf
ii
(k) = lim
s1

k
kf
ii
(k)s
k1
= lim
s1
F

ii
(s) = lim
s1
s(1 s
2
)
1/2
= .
32 Chng 1. Qu trnh Markov
nh l 1.12. Gi s i j. Nu i hi quy dng th j hi quy dng. Nu
i hi quy khng th j hi quy khng.
Chng minh. Ta ch cn chng minh nu tn ti trng thi j l hi quy
dng th mi trng thi i m i j cng hi quy dng. Tht vy tn ti
m, n sao cho P
ij
(n) > 0, P
ji
(m) > 0. Vi mi k ta c
P
ii
(n +k +m) P
ij
(n)P
jj
(k)P
ji
(m).
T v t nh l 1.11 ta c
1

i
= lim
k
P
ii
(n +k +m)
lim
k
P
ij
(n)P
jj
(k)P
ji
(m) = P
ij
(n)P
ji
(m)
1

j
> 0.
Vy
i
< tc i l hi quy dng. Theo nh l 1.4, = (
1
,
2
, ...) l phn
b gii hn duy nht ca xch.
T nh l 1.11 v 1.12 suy ra
nh l 1.13. Gi s (X
n
) l xch ti gin khng c chu k vi khng gian
trng thi m c E. Khi s xy ra mt trong ba kh nng sau y:
1) Mi trng thi l khng hi quy. Khi vi mi i, j
lim
n
P
ij
(n) = 0.
Xch khng c phn b dng.
2) Mi trng thi l hi quy khng. Khi vi mi i, j
lim
n
P
ij
(n) = 0
Xch khng c phn b dng.
1.2. Phn loi trng thi xch Markov 33
3) Mi trng thi l hi quy dng. Khi vi mi i, j
lim
n
P
ij
(n) =
j
> 0
v = (
1
,
2
, ...) l phn b gii hn (v cng l phn b dng) ca
xch.
nh l 1.14. Gi s (X
n
) l xch ti gin khng c chu k vi khng gian
trng thi hu hn E = {1, 2, ..., d}. Khi mi trng thi u hi quy dng
v xch c phn b gii hn = (
1
,
2
, ...,
d
). Phn b ny cng l phn b
dng duy nht ca xch.
Chng minh. Theo nh l 1.13 ta ch cn chng t kh nng 1) hoc 2)
khng xy ra. Tht vy nu tri li th vi mi i, j
lim
n
P
ij
(n) = 0.
Vy
1 = lim
n
d

j=1
P
ij
(n) =
d

j=1
lim
n
P
ij
(n) = 0.
Mu thun.
i vi xch ti gin (c th c chu k) ta c nh l sau y (khng
chng minh):
nh l 1.15. Gi s X
n
l xch ti gin vi khng gian trng thi E m
c. Khi
1. Vi mi i, j E
lim
n
n
1
n

k=1
P
ij
(k) =
1

j
.
Ni cch khc dy P
ij
(n) hi t theo trung bnh Cesaro ti
j
=
1

j
khng ph thuc i.
2. Dy = (
j
) tho mn
34 Chng 1. Qu trnh Markov

j=1

j
1

j
=

i=1

i
P
ij
.
Nh l mt h qu ca nh l trn ta c nh l sau (so snh vi 1.13):
nh l 1.16. Cho (X
n
) l xch Markov ti gin. Khi
1. Nu E hu hn c d phn t th (
1
, ...,
d
) l phn b dng duy nht.
2. Ch c cc kh nng sau:
Mi trng thi ca E l khng hi quy
Mi trng thi ca E l hi quy khng
Mi trng thi ca E l hi quy dng.
3. Nu E l v hn m c th xch c phn b dng khi v ch khi mi
trng tht ca E l hi quy dng. Trong trng hp ny phn b dng
l duy nht.
1.3 Qu trnh Markov
Xt h cc LNN ri rc (X
t
), t 0 vi tp ch s t l cc s thc khng
m t [0, ). K hiu E = X
t
() l tp gi tr ca X
t
. Khi E l mt
tp hu hn hay m c, cc phn t ca n c k hiu l i, j, k... .Ta
gi (X
t
) l mt qu trnh ngu nhin vi khng gian trng thi E .
nh ngha 1.9. Ta ni rng (X
t
) l mt qu trnh Markov nu vi mi
t
1
< ... < t
k
< t v vi mi i
1
, i
2
, ...i
n
, i E
P{X
t
= i|X
t
1
= i
1
, X
t
2
= i
2
..., X
t
k
= i
k
}
= P{X
t
= i|X
t
k
= i
k
}.
1.3. Qu trnh Markov 35
Nh vy, xc sut c iu kin ca mt s kin B no trong tng lai
nu bit hin ti v qu kh ca h cng ging nh xc sut c iu kin
ca B nu ch bit trng thi hin ti ca h. chnh l tnh Markov ca
h. i khi tnh Markov ca h cn pht biu di dng: Nu bit trng thi
hin ti X
t
ca h th qu kh X
u
, u < t v tong lai X
s
, s > t l c lp vi
nhau.
Gi s
P{X
t+s
= j|X
s
= i}
l xc sut xch ti thi im s trng thi i sau mt khong thi gian t
, ti thi im t +h chuyn sang trng thi j. y l mt con s ni chung
ph thuc vo i, j, t, s. Nu i lng ny khng ph thuc s ta ni xch l
thun nht. Trong gio trnh ny ta ch xt xch Markov thun nht.
K hiu
P
ij
(t) = P{X
t+s
= j|X
s
= i}.
Ta gi P
ij
(t) l xc sut chuyn ca h t trng thi i sang trng thi j sau
mt khong thi gian t . K hiu P(t) = (P
ij
(t), i, j E). P(t) l mt ma
trn hu hn hay v hn chiu. Ch rng
i)P
ij
(t) 0
ii)

jE
P
ij
(t) = 1.
Phn b ca X
0
c gi l phn b ban u. Ta k hiu u
i
= P(X
0
= i).
Chng minh tng t nh trng hp xch Markov ta c kt lun sau:
nh l 1.17. Phn b hu hn chiu ca qu trnh (X
t
) c hon ton
xc nh t phn b ban u v xc sut chuyn. C th vi t
1
< t
2
< ... < t
n
phn b ng thi ca (X
t
1
, ..., X
tn
) c tnh theo cng thc sau
P(X
t
1
= i
1
, ..., X
tn
= i
n
) =
=

iE
u
i
P
ii
1
(t
1
)P
i
1
i
2
(t
2
t
1
)...P
i
n1
in
(t
n
t
n1
).
36 Chng 1. Qu trnh Markov
nh l 1.18. (Phng trnh Chapman-Kolmogorov)
P
ij
(t +s) =

kE
P
ik
(t)P
kj
(s).
1.3.1 Trng hp khng gian trng thi hu hn
Gi s E = {1, 2, ..., d}. Khi t phng trnh C-K P(t), t > 0 l mt h
cc ma trn tho mn ng thc sau
P(t +s) = P(t)P(s).
Ni cch khc h (P(t), t > 0) lp thnh mt na nhm cc ma trn. T nay
v sau ta s lun gi thit thm rng
1. P
ij
(0) =
ij
2. lim
t0
P
ij
(t) =
ij
y
ij
l k hiu Kronecke

ij
=
_
_
_
1 nu i = j
0 nu i = j.
nh l 1.19. Hm ma trn P(t) l mt hm lin tc v tn ti
P

(0) = lim
h0+
P(t) I
h
.
Chng minh. Theo gi thit th P(0) = I v lim
t0
P(t) = I tc l P(t) lin
tc ti 0. Ta chng minh P(t) lin tc ti t > 0. Ta c do tnh cht na
nhm
lim
h0+
P(t +h) = lim
h0
P(t)P(h) = P(t)I = P(t).
Vy P(t) lin tc phi. Ta li c vi 0 < h < t th P(t) = P(h)P(t h) . Do
P(h) I khi h 0 nn tn ti P(h)
1
vi h nh. Vy
lim
h0+
P(t h) = lim
h0
P(t)P(h)
1
= P(t)I = P(t).
1.3. Qu trnh Markov 37
Vy P(t) cng lin tc tri do lin tc. (Thc ra c th chng minh c
rng P(t) lin tc u trn [0, ).
Tip theo s dng tnh cht na nhm ta c vi mi h > 0, n:
P(nh) I = (P(h) I) (I +P(h) +P(2h) +... +P((n 1)h)) . (1.12)
V P(t) lin tc nu h 0 sao cho nh t > 0 th
h
n1

k=0

_
t
0
P(u)du.
Tn ti t > 0 tch phn
_
t
0
P(u)du l ma trn khng suy bin. Ta li c
lim
n
P(nh) I = P(t) I. T (1.12) suy ra
lim
h0+
P(h) I
h
= (P(t) I)
__
t
0
P(u)du
_
1
K hiu
A = P

(0) = lim
h0+
P(h) I
h
.
A = (a
ij
) c gi l ma trn cc vi ca na nhm (P(t)). Ta c
lim
t0
P
ij
(t)
t
= a
ij
nu i = j
lim
t0
1 P
ii
(t)
t
= a
i
(1.13)
y a
i
= a
ii
. T (1.13) ta c
P
ij
(t) = a
ij
t +o(t)
1 P
ii
(t) = a
i
t +o(t).
Thnh th a
ij
c gi l cng chuyn t trng thi i sang trng thi j
v a
i
l cng thot khi trng thi i ca h.
38 Chng 1. Qu trnh Markov
T ng thc 0 =

j:j=i
P
ij
(t) + P
ii
1 chia hai v cho t v cho t 0 ta
thu c

j
a
ij
= 0
hay tng ng
a
i
=

j:j=i
a
ij
nh l 1.20. Cho qu trnh Markov vi na nhm P(t), t > 0 cc xc sut
chuyn. Gi A l ma trn cc vi ca na nhm. Khi ta c
P

(t) = P(t)A
P

ij
(t) =

k=j
P
ik
(t)a
kj
P
ij
(y)a
j
(1.14)
v
P

(t) = AP(t)
P

ij
(t) =

k=i
P
kj
a
ik
P
ij
(y)a
i
. (1.15)
Phng trnh ( (1.14)) c gi l phng trnh thun v phng trnh (1.15)
c gi l phng trnh ngc Kolmogorov.
Chng minh. Ta c do tnh cht na nhm
P(t +h) P(t)
h
=
P(t)(P(h) I)
h
=
(P(h) I)P(t)
h
.
Cho h 0 ta c iu cn chng minh.
Phng trnh thun v nghch l cc phng trnh vi phn vi iu kin
ban u P(0) = I c th gii c bng phng php quen thuc trong l
thuyt phng trnh vi phn. Ta c kt qu sau (khng chng minh):
1.3. Qu trnh Markov 39
nh l 1.21. Phong trnh (1.14) v phng trnh (1.15) c nghim l
P(t) = e
At
= I +

n=1
A
n
t
n
n!
.
Ngc li cho trc ma trn A = (a
ij
) cp d d tho mn a
ij
0 nu i = j
v

j
a
ij
= 0 . t
P(t) = e
At
.
Khi tn ti qu trnh Markov vi d trng thi nhn P(t) lm h ma trn
xc sut chuyn.
V d 1.17. (Qu trnh Markov hai trng thi) Xt qu trnh Markov vi
hai trng thi E = {0, 1}. (Chng hn ta xt s tin trin theo thi gian ca
mt h thng no trong trng thi 0 biu th trng thi tr tr cn trng
thi 1 biu th trng thi lm vic tch cc ca h thng). Gi s cng
chuyn t trng thi 0 sang trng thi 1 l v cng chuyn t trng
thi 1 sang trng thi 0 l . Ma trn cc vi l
A =
_


_
.
Ta i tm cng thc cho xc sut chuyn P
ij
(t) bng cch gii phng trnh
ngc. Phng trnh (1.15) cho ta
P

00
(t) = P
00
(t) +P
10
(t)
P

10
(t) = P
00
(t) P
10
(t).
Tr hai phng trnh trn v vi v ta c:
(P
00
(t) P
10
(t))

= ( +)(P
00
(t) P
10
(t))
P
00
(t) P
10
(t) = (P
00
(0) P
10
(0))e
(+)t
= e
(+)t
.
40 Chng 1. Qu trnh Markov
Vy
P

00
(t) = (P
00
(t) P
10
(t)) = e
(+)t
P
00
(t) = P
00
(0) +
_
t
0
P

00
(s)ds
= 1

+
(1 e
(+)t
)
=

+
+

+
e
(+)t
.
T
P
10
(t) = P
00
(t) e
(+)t
=

+


+
e
(+)t
.
Hon ton tng t t phng trnh (1.15) ta c
P

01
(t) = P
01
(t) +P
11
(t)
P

11
(t) = P
01
(t) P
11
(t).
ta tm c
P
01
(t) =

+


+
e
(+)t
P
11
(t) =

+
+

+
e
(+)t
.
By gi chng ta xt dng iu tim cn ca ma trn xc sut chuyn
P(t) khi t . Cho qu trnh Markov (X
t
) vi khng gian trng thi E
hu hn v ma trn xc sut chuyn P(t) = P
ij
(t). Ta ni rng qu trnh l
ti gin nu P
ij
(t) > 0 vi mi i, j E. (Ch rng ta khng c khi nim
"chu k ca mt trng thi" nh l i vi xch Markov).
nh l 1.22. Cho qu trnh Markov ti gin (X
t
)vi khng gian trng thi
E = {1, 2, ..., d} hu hn v ma trn xc sut chuyn P(t) = P
ij
(t). Khi
vi mi i, j E tn ti gii hn hu hn
lim
t
P
ij
(t) =
j
1.3. Qu trnh Markov 41
ch ph thuc j khng ph thuc i. Thm vo = (
1
,
2
, ...,
d
) l phn
b xc sut duy nht tho mn phng trnh
= P(t), t > 0.
Chng minh. Vi mi h > 0 c nh P(h) l mt ma trn xc sut chuyn
vi cc phn t u dong, vy theo nh l 1.5 ta c tn ti
lim
n
P(nh) = lim
n
P(h)
n
= (h)
trong (h) l ma trn vi cc dng nh nhau v bng (h) .Thm vo
(h) = (
1
(h),
2
(h), ...,
d
(h)) l phn b xc sut duy nht tho mn
phng trnh
(h) = (h)P(h), t > 0.
Mt khc ta bit rng P(t) lin tc u trn [0, ). Trong gii tch ta bit
rng nu mt hm P(t) lin tc u sao cho lim
n
P(nh) tn ti vi mi h > 0
th ko theo lim
t
P(t) tn ti. Vy ta kt lun tn ti
lim
t
P(t) =
vi = (h) vi mi h > 0. T suy ra kt lun ca nh l.
V d 1.18. Trong v d v qu trnh Markov hai trng thi t biu thc
ca P
ij
(t) ta c
lim
t
P
ij
(t) =
j
vi

0
=

+
,
1
=

+
.
Nu chn = (
0
,
1
) l phn b ban u ca qu trnh th
P(X
t
= 0) = P(X
0
= 0)P
00
(t) +P(X
0
= 1)P
10
(t)
=
0
P
00
(t) +
1
P
10
(t) =
0
=

+
.
Tng t
P(X
t
= 1) =
1
=

+
.
Nh vy phn b ca X
t
khng ph thuc vo t.
42 Chng 1. Qu trnh Markov
1.3.2 Trng hp khng gian trng thi v hn m
c
Trong trng hp khng gian trng thi E l v hn m c ta gp nhng
kh khn v ton hc khi mun m rng cc kt qu trong trng hp hu
hn. Ta c kt qu sau y (cng nhn khng chng minh):
nh l 1.23.
(1) Vi mi i = j gii hn
P

ij
(0) = lim
t0
P
ij
(t)
t
= a
ij
lun tn ti hu hn.
(2) Vi mi i gii hn
P

ii
(0) = lim
t0
P
ii
(t) 1
t
= a
ii
= a
i
tn ti nhng c th bng v cng.
i vi trng hp khng gian trng thi hu hn ta c

j
a
ij
= 0 hay

j=i
a
ij
= a
i
.
Trong trng hp E v hn ni chung ta ch c bt ng thc sau

j=i
a
ij
a
i
i E (1.16)
Tht vy ta c

j=i
P
ij
(t) = 1 P
ii
(t)
thnh th vi mi m
n

j=1,j=i
P
ij
(t) 1 P
ii
(t).
1.3. Qu trnh Markov 43
Chia hai v cho t v y t 0 ta thu c
m

j=1,j=i
a
ij
a
i
.
Cho m ta thu c (1.16).
T nay v sau ta ch xt cc qu trnh Markov tho mn iu kin

j=i
a
ij
= a
i
< . (1.17)
Ma trn v s chiu A = (a
ij
) cng c gi l ma trn cc vi ca qu trnh
ang xt.
nh l 1.24. Cho qu trnh Markov vi P(t) = (P
ij
(t)) l h cc ma trn
xc sut chuyn. Gi A l ma trn cc vi ca qu trnh. Khi ta c
P

(t) = P(t)A
P

ij
(t) =

k=j
P
ik
(t)a
kj
P
ij
(y)a
j
(1.18)
v
P

(t) = AP(t)
P

ij
(t) =

k=i
a
ik
P
kj
P
ij
(y)a
i
(1.19)
Phng trnh (1.18) c gi l phng trnh thun v phng trnh (1.19)
c gi l phng trnh ngc Kolmogorov.
Chng minh. Ta ch chng minh cho phng trnh ngc cn tha nhn s
ng n ca phng trnh thun v chng minh kh phc tp v ton hc.
Ta c:
P
ij
(s +t) P
ij
(t) =

k
P
ik
(s)P
kj
(t) P
ij
(t)
=

k=i
P
ik
(s)P
kj
(t) + (P
ii
(s) 1)P
ij
(t) (1.20)
44 Chng 1. Qu trnh Markov
Vi mi m c nh ta c
liminf
s0
s
1

k=i
P
ik
(s)P
kj
(t) liminf
s0
s
1
m

k=1,k=i
P
ik
(s)P
kj
(t)
=
m

k=1,k=i
a
ik
P
kj
(t).
Cho m ta c
liminf
s0
s
1

k=i
P
ik
(s)P
kj
(t)

k=i
a
ik
P
kj
(t). (1.21)
Tip theo vi m > i ta c

k=i
P
ik
(s)P
kj
(t)
m

k=1,k=i
P
ik
(s)P
kj
(t) +

m+1
P
ik
(s)
=
m

k=1,k=i
P
ik
(s)P
kj
(t) + 1 P
ii
(s)
m

k=1,k=i
P
ik
(s).
Chia hai v cho s v ly limsup ta thu c
limsup
s0
s
1

k=i
P
ik
(s)P
kj
(t)
m

k=1,k=i
a
ik
P
kj
(t) +a
i

k=1,k=i
a
ik
(s).
Cho m v ch n iu kin (1.17) ta c
limsup
s0
s
1

k=i
P
ik
(s)P
kj
(t)

k=i
a
ik
P
kj
(t).
So snh vi (1.21) ta nhn c
lim
s0
s
1

k=i
P
ik
(s)P
kj
(t) =

k=i
a
ik
P
kj
(t).
Trong 1.20 chia hai v cho s ri cho s 0 v p dng ng thc trn ta suy
ra (1.19).
1.3. Qu trnh Markov 45
By gi chng ta xt dng iu tim cn ca ma trn xc sut chuyn
P(t) khi t . Cho qu trnh Markov (X
t
) vi khng gian trng thi E v
hn m c v ma trn xc sut chuyn P(t) = P
ij
(t). Ta ni rng qu
trnh l ti gin nu P
ij
(t) > 0 vi mi i, j E. ( Ch rng ta khng c
khi nim "chu k ca mt trng thi" nh l i vi xch Markov).
nh l 1.25. Cho qu trnh Markov ti gin (X
t
)vi khng gian trng thi
E = {1, 2, ..., } m c v ma trn xc sut chuyn P(t) = P
ij
(t). Khi
vi mi i, j E tn ti gii hn hu hn
lim
t
P
ij
(t) =
j
ch ph thuc j khng ph thuc i. Thm vo gii hn = (
1
,
2
, ..., )
hoc l tt c bng khng

j
= 0 j E
hoc l tt c dng v lp thnh mt phn b xc sut. Phn b c gi
l phn b gii hn ca qu trnh

j
> 0 j E,

j
= 1.
Ta cng nhn v khng chng minh nh l ny.
nh l 1.26. Phn b gii hn = (
1
,
2
, ..., ) tho mn h phng trnh
tuyn tnh sau

j
a
j
=

k=j

k
a
kj
.
Chng minh. T phng trnh C-K
P
ij
(s +t) =

kE
P
ik
(s)P
kj
(t)
cho s ta thu c

j
=

k
P
kj
(t).
46 Chng 1. Qu trnh Markov
Suy ra

j
(1 P
jj
(t)) =

k=j

k
P
kj
(t).
Chia hai v cho t v cho t 0 ta c h thc phi chng minh.
V d 1.19. (Qu trnh sinh v cht.) Xt qu trnh Markov (X
t
) vi khng
gian trng thi E = {0, 1, 2, ...}. (X
t
) c gi l qu trnh sinh v cht nu
cc xc sut chuyn P
ij
(t) tho mn cc iu kin sau y
1. P
i.i+1
(t) =
i
t +o(t) i 0 khi t 0
2. P
i.i1
(t) =
i
t +o(t) i 1 khi t 0
3. P
ii
(t) = 1 (
i
+
i
)t +o(t), i 0 khi t 0
4. P
ij
(t) = o(t) vi |i j| > 1
5. P
ij
(0) =
ij
6.
i
= 0,
0
> 0,
i
,
i
> 0, i = 1, 2, ...
Qu trnh X
t
c s dng m t s pht trin ca mt qun th A no
. Mi c th ca qun th A ti mi thi im c th sinh ra mt c th
mi hoc b cht i. K hiu X
t
l s lng c th ca qun th ti thi im
t. Cc iu kin trn c ngha l nu ti thi im t qun th c i c th th
trong mt khong thi gian b (s, s + t) xc sut s lng qun th tng
thm mt c th l
i
t +o(t) v xc sut gim i mt c th l
i
t +o(t),
xc sut tng gim t nht hai c th l o(t). Cc tham s
i
, i = 0, 1, 2
c gi l cng sinh, cc tham s
i
, i = 1, 2, ... c gi l cng
cht.
Ma trn cc vi A = (a
ij
) c dng
a
i,i+1
=
i
a
i,i1
=
i
,
a
i
=
i
+
i
,
a
ij
= 0 nu |i j| > 1
1.3. Qu trnh Markov 47
tc l
A =
_
_
_
_
_
_
_
_

0

0
0 0 0...

1
(
1
+
1
)
1
0 0..
0
2
(
2
+
2
)
2
...
. . . . .
. . . .
_
_
_
_
_
_
_
_
Phng trnh thun (1.14) trong trng hp ny c dng
P

ij
(t) = P
i,j1
(t)
j1
+P
i,i+1
(t)
j+1
(
j
+
j
)P
ij
(t) (1.22)
P

i0
(t) =
0
P
i0
(t) +
1
P
i1
(t). (1.23)
Gi s qu trnh sinh v cht c phn b gii hn = (
i
). T nh l 1.26
ta c

0
=
1

1
(
k
+
k
)
k
=
k1

k1
+
k+1

k+1
t a
k
=
k

k

k1

k1
. T phng trnh trn suy ra a
k
= a
k+1
. V
a
1
=
1

0
= 0 nn suy ra a
k
= 0 k hay

k
=
k1

k1
k = 1, 2, ... (1.24)

k
=

k1

k1
(1.25)

k
=

k1

k2
...
0

k1
...
1

0
. (1.26)
V

k
= 1 nn t (1.24) suy ra
1 =
0
+
0

k=1
_

k1

k2
...
0

k1
...
1
_
.
Vy iu kin cn c phn b gii hn l chui

k=1
_

k1

k2
...
0

k1
...
1
_
(1.27)
hi t. Ngc li c th chng minh c iu kin chui (1.27) hi t cng
l iu kin qu trnh c phn b gii hn.
48 Chng 1. Qu trnh Markov
Ta xt mt s v d.
V d 1.20. (Mt m hnh n gin trong l thuyt xp hng) Gi s ca
hng dch v A ch c mt ngi phc v. Khch n xp hng i n lt
mnh c phc v v ca hng ch phc v tng khch mt. Khi ca hng
ang phc khch th cc khch mi c th n v xp hng ch. Khch c
phc v xong th lp tc ri khi ca hng. Gi s xc sut trong khong
thi gian (t, t + h) c mt khch mi vo hng l h + o(h) v cng gi s
rng nu ti thi im t khch ang c phc v th xc sut s phc v
hon tt trong khong thi gian (t, t + h) l h + o(h). Gi X
t
l s khch
c mt ti ca hng ti thi im t (tc l s khch ang xp hng ch phc
v cng vi khch ang c phc v ti thi im t). D thy y l mt
qu trnh sinh v cht vi cng sinh v cng cht u l hng s

i
= ,
i
= . Khi chui (1.27) tr thnh chui

k=1
(

)
k
.
Vy: Qu trnh c phn b gii hn (v y cng l phn b dng) khi v ch
khi < . Khi ta c

k
=
_
1

_
(

)
k
.
T s r =

gi l cng gii phng hng. Nu r < 1 th khi t kh ln, s


khch c mt ca hng X
t
l mt LNN c phn b hnh hc
P(X
t
= k) = (1 r)r
k
.
Suy ra s khch c mt trung bnh l EX
t
=
1
1r
.
Trong mt thi cc khc, ta gi s ca hng c rt nhiu nhn vin phc
v sao cho bt c ngi khch no n cng c phc v ngay. Gi X
t
l s
khch c mt ti ca hng ti thi im t (tc l s khch ang c phc
v ti thi im t). Ta c
P
i,i+1
(h) = P{X
t+h
= i + 1|X
t
= i}
= h +o(h),
1.3. Qu trnh Markov 49
P
i,i1
(h) = P(c ng mt khch trong i khch c phc v xong)
=
_
i
1
_
[h +o(h)][1 h +o(h)]
i1
= ih +o(h).
Vy X
t
l qu trnh sinh v cht vi
i
= ,
i
= i. Khi cng thc (1.24)
cho ta

k
=
1
k!
(

)
k

0
v chui (1.27) tr thnh chui

k=1
1
k!
(

)
k
= e
/
1.
T iu kin
1 =

k=0

k
=
0
e
/
rt ra

k
= e
r
r
k
k!
.
Nh vy khi t ln X
t
c phn b Poisson vi tham s r =

. Gi tr trung
bnh ca X
t
khi t ln l r, n chnh bng t s gia thi gian phc v
trung bnh 1/ vi khong thi gian trung bnh xut hin mt khch mi 1/.
Mt kt lun kh ph hp vi thc t!.
V d 1.21. (Qu trnh sinh thun tu) Nu qu trnh sinh v cht m khng
xy ra s cht th ta gi l qu trnh sinh thun tu. Nh vy i vi qu
trnh sinh thun tu ta c

j
= 0 j 1
v
P
ij
(t) = 0 nu j < i.
50 Chng 1. Qu trnh Markov
Phng trnh thun (1.22) tr thnh
P

ij
(t) =
j1
P
i,j1

j
P
ij
(t) (1.28)
P

ii
(t) =
i
P
ii
(t). (1.29)
V P
ii
(0) = 1 ta suy ra
P
ii
(t) = e

i
t
. (1.30)
gii phng trnh vi phn (1.28) ta c b d sau:
B 1.5. Phng trnh vi phn
f

(t) = f(t) +g(t) (1.31)


c nghim l
f(t) = f(0)e
t
+
_
t
0
e
(ts)
g(s)ds.
Tht vy nhn hai v ca (1.36) vi e
t
ta c
_
e
t
f(s)
_

= e
s
g(s).
Ly tch phn hai v t 0 ti t ta c
e
t
f(t) f(0) =
_
t
0
e
s
g(s)ds.
Nhn hai v vi e
t
ta c kt lun ca b .
V P
ij
(0) = 0 i = j nn t b v phng trnh (1.28) ta c
P
ij
(t) =
j1
_
t
0
e

j
(ts)
P
i,j1
(s)ds. (1.32)
Ta c th s dng (1.30) v (1.32) nh mt cng thc truy hi tm ln
lt P
ij
(t) vi j = i + 1, i + 2, .... Chng hn ta c
P
i,i+1
(t) =
i
_
t
0
e

i+1
(ts)
e

i
s
ds.
Suy ra
P
i,i+1
(t) =
_
_
_

i+1

i
_
(e

i
t
e

i+1
t
_
nu
i+1
=
i

i
te

i
t
nu
i+1
=
i
.
(1.33)
1.3. Qu trnh Markov 51
V d 1.22. (Qu trnh sinh tuyn tnh) Xt s tng trng c th ca mt
qun th no . Gi s rng mi c th ca qun th c lp vi nhau trong
khong thi gian (t, t +h) c xc sut sinh thm mt c th mi l h +o(h)
v xc sut khng sinh thm c th no trong khong thi gian (t, t +h) l
1 h +o(h). Go X
t
l s lng c th thi im t. X
t
l mt qu trnh
sinh thun tu v
P
ii
(h) = P{X
t+h
= i|X
t
= i}
= [1 h +o(h)]
i
= 1 ih +o(h),
P
i,i+1
(h) = P{X
t+h
= i + 1|X
t
= i}
=
_
i
1
_
[h +o(h)][1 h +o(h)]
i1
= ih +o(h).
Nh vy cng sinh l

i
= i.
T (1.33) ta c
P
i,i+1
(t) = ie
it
(1 e
t
).
tnh P
i,i+2
(t) ta t j = i + 2 trong (1.32) v thu c
P
i.i+2
(t) = (i + 1)i
_
t
0
e
(i+2)(ts)
e
is
(1 e
s
)ds
= (i + 1)ie
(i+2)t
_
t
0
e
2i
(1 e
s
)ds
= (i + 1)ie
(i+2)t
_
t
0
e
t
(e
s
1)ds
= (i + 1)ie
(i+2)t
(e
t
1)
2
2
_
i + 1
2
_
e
it
(1 e
t
)
2
.
52 Chng 1. Qu trnh Markov
Tip tc nh vy p dng cng thc truy hi (1.32) v bng quy np ta thu
c
P
i,i+k
(t) =
_
i +k 1
k
_
e
it
(1 e
t
)
k
.
Nh vy s gia tng dn s X
s+t
X
s
trong khong thi gian t c phn b
nh thc m vi cc tham s p = e
t
v r = i, i = X
s
. Thnh th
E[X
s+t
X
s
|X
s
= i] = ie
t
(1 e
t
).
Nu ti thi im ban u s = 0 qun th c s lng i c th th ti thi
im t s c th trung bnh s l
E[X
t
] = E[X
t
X
0
] +i = ie
t
.
Qu trnh sinh tuyn tnh ny i khi cn c gi l qu trnh Yule, do nh
ton sinh ngi Anh Yule a ra nm 1924.
V d 1.23. (Qu trnh Poisson.) Xt qu trnh sinh thun tu X
t
vi cng
sinh l hng s

i
= , i = 0, 1, ...
Khi cng thc (1.32) tr thnh
P
ij
(t) =
_
t
0
e
(ts)
P
i,j1
(s)ds. (1.34)
T cng thc (1.33) ta thu c
P
i,i+1
(t) = te
t
.
tnh P
i,i+2
(t) ta t j = i + 2 trong (1.34) v thu c
P
i,i+2
(t) =
_
t
0
e
(ts)
e
s
ds
=
2
e
t
_
t
0
sds =
()
2
2
e
t
.
1.3. Qu trnh Markov 53
Tip tc nh vy p dng cng thc truy hi (1.34) v bng quy np ta thu
c
P
i,i+k
(t) =
(t)
k
k!
e
t
.
Nh vy s gia tng dn s X
s+t
X
s
trong khong thi gian t c phn b
Poisson vi tham s t.
Mt cch tng qut ta s chng minh rng vi 0 < s < t th LNN
X
t
X
s
s c phn b Poisson vi tham s (t s). Tht vy ta c
P(X
t
X
s
= k) =

i=0
P(X
s
= i)P(X
t
= k +i|X
s
= i)
=

i=0
P(X
s
= i)P
i,i+k
(t s)
=

i=0
P(X
s
= i)
((t s))
k
k!
e
(ts)
=
((t s))
k
k!
e
(ts)

i=0
P(X
s
= i)
=
((t s)
k
)
k!
e
(ts)
.
Tip theo ta chng minh rng X
t
l qu trnh ngu nhin c gia s c lp.
Tht vy, vi 0 t
1
< t
2
< < t
n
ta c
P
_
X
t
2
X
t
1
= i
1
, ..., X
tn
X
t
n1
= i
n1
_
=

i=0
P(X
t
1
= i)P
0i
1
(t
2
t
1
)...P
0i
n1
(t
n
t
n1
)
= P
0i
1
(t
2
t
1
)...P
0i
n1
(t
n
t
n1
)

i=0
P(X
t
1
= i)
= P
0i
1
(t
2
t
1
)...P
0i
n1
(t
n
t
n1
)
= P(X
t
2
X
t
1
= i
1
)...P(X
tn
X
t
n1
= i
n1
).
Thnh th X
t
2
X
t
1
, ..., X
tn
X
t
n1
l cc LNN c lp.
54 Chng 1. Qu trnh Markov
Qu trnh X
t
, t 0 c gi l qu trnh Poisson vi cng > 0 nu
n tho mn cc iu kin sau
1. X
0
= 0.
2. Vi 0 s < t th LNN X
t
X
s
s c phn b Poisson vi tham s
(t s).
3. X
t
l qu trnh ngu nhin c gia s c lp.
Nh vy ta chng minh rng qu trnh sinh thun tu vi cng
sinh l hng s chnh l mt qu trnh Poisson vi cng > 0 . Qu
trnh Poisson c rt nhiu ng dng trong thc t. N dng m t s ln
xut hin ca mt s kin ngu nhin no trong khong thi gian t, chng
hn s ln gi n tng i, s khch hng n mt ca hng no , s ln
hng hc ca mt ng dy,...
1.3.3 Trng hp tng qut
Mt vi khi nim c bn v qu trnh ngu nhin. Xt qu trnh
Markov vi khng gian trng thi E bt k. Cho (E, A) l mt khng gian
o. Qu trnh ngu nhin X
t
c gi l qu trnh Markov nu
P(X
t+s
A|F
t
) = P(X
t+s
A|F
t
).
Ngha l : Nu ta bit trng thi ca h ti thi im hin ti t th mi thng
tin v hnh vi ca h trong qu kh khng c nh hng n s bin din
trong tng lai ca h. Ni cch khc: Qu kh v tng lai c lp vi nhau
khi bit hin ti. K hiu
P(s, x, t, A) = P(X
t
A|X
s
= x)
P(s, x, t, A) l xc sut h ti thi im s ang trng thi x sang thi
im t ri vo tp A. Ta gi P(s, x, t, A) l xc sut chuyn. H cc xc sut
chuyn c cc tnh cht sau:
1.3. Qu trnh Markov 55
nh l 1.27.
1. Vi mi s t, x E P(s, x, t, .) l mt o xc sut trn E
2. Vi mi s t, A A hm P(s, ., t, A) l mt hm o c trn E
3. ( Phng trnh C-K (Chapman- Kolmogorov) )
P(s, x, t, A) =
_
E
P(s, x, u, dy)P(u, y, t, A).
Qu trnh Markov X
t
c gi l thun nht nu xc sut chuyn
P(s, x, t, A) ch ph thuc vo hiu s t s ngha l
P(s +u, x, t +u, A) = P(s, x, t, A) u
Khi P(s, x, t, A) c dng
P(s, x, t, A) = P(t s, x, A)
P(t, x, A) = P(X
s+t
A|X
s
= x) l xc sut h ti thi im s
trng thi x sau mt khong thi gian t ( ti thi im t +s) ri vo A.
Phng trnh Chapman- Kolmogorov khi y tr thnh
P(t +s, x, A) =
_
E
P(t, x, dy)P(s, y, A).
Trong gio trnh ny ta ch xt qu trnh Markov thun nht.
Trong trng hp o P(t, x, .) c mt f(t, x, u) th phng trnh
C-K tng ng vi
f(t +s, x, z) =
_
E
f(t, x, y)f(s, y, z)dy.
Ngc li cho trc mt h cc hm P(t, x, A) tho mn cc iu kin nu
trong nh l (1.27). Vi mi o xc sut trn (E, A) ta xc nh h
56 Chng 1. Qu trnh Markov
cc phn b hu hn chiu (
t
1
,...,tn
) nh sau

t
1
,...,tn
(A
1
, ..., A
n
)
=
_
ynAn
...
_
y
1
A
1
_
xE
(dx)P(t
1
, x, dy
1
)P(t
2
t
1
, y
1
, dy
2
)...
P(t
n
t
n1
, y
n1
, dy
n
).
S dng cc tnh cht ca h xc sut chuyn ta chng minh c h cc
phn b xc sut ny tho mn iu kin tng thch Kolmogrov. Thnh
th tn ti mt qu trnh ngu nhin (X
t
) sao cho:
Phn b ca X
0
( phn b ban u) l .
Vi mi 0 t
1
< ... < t
n
phn b ng thi ca (X
t
1
, ..., X
tn
) l
t
1
,...,tn
.
Hn na c th chng minh rng X
t
l mt qu trnh Markov thun nht
nhn P(t, x, A) lm xc sut chuyn.
V d 1.24. (Chuyn ng Brown hay qu trnh Wiener.) Xt hm f(t, x, y)
cho bi cng thc
f(t, x, u) =
1

2t
e

(ux)
2
2t
v P(t, x, .) l o xc sut trn R nhn f(t, x, .) lm hm mt
P(t, x, A) =
_
A
f(t, x, du).
Khi ta c th chng minh rng h P(t, x, A) tho mn cc iu kin ca
nh l (1.27). Tht vy, gi
t
(u) l hm mt ca N(0, t) th nh ta bit

t+s
=
t

s
hay

t+s
(u) =
_

s
(v)
t
(u v)dv.
t u = x z, v = y z dv = dy suy ra

t+s
(x z) =
_

s
(y z)
t
(x y)dy
1.3. Qu trnh Markov 57
hay
f(t +s, x, z) =
_
f(t, x, y)f(s, y, z)dy.
Do tn ti qu trnh Markov vi h P(t, x, A) l h xc sut chuyn v
phn b ban u =
0
. Qu trnh Markov ny c gi l chuyn ng
Brown hay qu trnh Wiener v c k hiu l (W
t
).
Cc phn tch su sc hn na chng t (W
t
) c cc tnh cht sau
1. W
0
= 0 .
2. Vi mi 0 s < t th W
t
W
s
l LNN c phn b chun vi k vng
0 v phng sai l t s .
3. W(t) l mt qu trnh gia s c lp tc l vi mi 0 t
1
< t
2
<
... < t
n
cc LNN W
t
2
W
t
1
, W
t
3
W
t
2
, ..., W
tn
W
t
n1
l c lp.
4. W
t
c qu o lin tc.
By gi chng ta s trnh by phng php xy dng cc hm xc sut
chuyn. Nh bit mi o trn R ta cho ng vi phim hm T trn
C
b
(R) bi
T

=
_
R
(u)d(u).
Tng ng ny l mt-mt. Nu bit T vi mi th c th xc nh c
.Ta nh ngha
(t, x) =
_
R
(u)P(t, x, du). (1.35)
Nu vi mi ta tm c hm (t, x) tng ng th coi nh tm c
P(t, x, .). Ta c kt qu sau:
nh l 1.28. K hiu B(x, ) = [x , x + ], B
c
(x, ) = R \ B(x, ). Gi
58 Chng 1. Qu trnh Markov
s rng
lim
t0+
t
1
P(t.x, B
c
(x, )) = 0 x,
lim
t0+
t
1
_
B(x,)
(u x)P(t, x, du) = a(x)
lim
t0+
t
1
_
B(x,)
(u x)
2
P(t, x, du) = b(x)
Khi hm (t, x) xc nh bi (1.35) tho mn phng trnh vi phn sau

t
= a(x)

x
+
b(x)
2

x
2
(1.36)
vi iu kin ban u
lim
t0+
(t, x) = (x). (1.37)
Ngc li cho trc hai hm a(x), b(x). Khi vi mt s gi thit v hai
hm a(x), b(x) vi mi hm lin tc b chn phng trnh (1.36) vi iu
kin ban u (1.37) s c nghim duy nht (t, x) v do xc nh cho ta
o P(t, x, .). H o ny lp thnh mt h xc sut chuyn. Hn na
qu trnh Markov tng ng vi h xc sut chuyn ny c qu o lin tc.
1.4 Bi tp
1. Cho xch Markov X
n
, n = 0, 1, 2, ... vi khng gian trng thi E =
{0, 1, 2} v ma trn xc sut chuyn
P =
_
_
_
0.1 0, 2 0, 7
0.9 0, 0 0, 1
0, 1 0, 8 0.1
_
_
_
.
Bit phn b ban u l p
0
= 0, 3, p
1
= 0, 4, p
2
= 0, 3. Tnh
P(X
0
= 0, X
1
= 1, X
2
= 2).
1.4. Bi tp 59
2. Ngi ta truyn mt bc in gm cc tn hiu 0, 1 thng qua knh c
nhiu trm v mi trm nhn ng tn hiu vi xc sut a. K hiu X
0
l tn hiu truyn i v X
n
l tn hiu nhn c trm n. Bit rng
(X
n
) lp thnh xch Markov vi ma trn xc sut chuyn
P =
_
a 1 a
1 a a
_
.
Gi s tn hiu truyn i l tn hiu 0.
i) Tnh xc sut khng nhn sai tn hiu cho ti trm n = 2.
ii) Tnh xc sut nhn ng tn hiu ca trm n = 2.
ii) Tnh xc sut nhn ng tn hiu ca trm n = 5.
3. Cho xch Markov X
n
, n = 0, 1, 2, ... vi khng gian trng thi E = 0, 1, 2
v ma trn xc sut chuyn
P =
_
_
_
0.1 0, 2 0, 7
0.2 0, 2 0, 6
0, 6 0, 1 0.3
_
_
_
Tnh P
2
, P(X
3
= 1|X
1
= 0) v P(X
3
= 1|X
0
= 0).
4. Cho xch Markov X
n
, n = 0, 1, 2, ... vi khng gian trng thi E = 0, 1, 2
v ma trn xc sut chuyn
P =
_
_
_
0.0 0, 5 0, 5
0.5 0, 0 0, 5
0, 5 0, 5 0.0
_
_
_
Tnh P(X
2
= 1|X
0
= 0) v P(X
3
= 0|X
0
= 0).
5. Gi s X
n
cht lng ca chi tit ti cng on th n ca dy chuyn
sn xut vi X
n
= 0 c ngha l tt cn X
n
= 1 c ngha l xu. Bit
rng (X
n
)l xch Markov vi ma trn xc sut chuyn
P =
_
0.99 0.01
0.12 0, 88
_
60 Chng 1. Qu trnh Markov
Tnh P(X
4
= 1|X
1
= 1).
6. Cho xch Markov X
n
, n = 0, 1, 2, ... vi khng gian trng thi E = 1, 2, 3
v ma trn xc sut chuyn
P =
_
_
_
0.1 0, 5 0, 4
0.6 0, 2 0, 2
0, 3 0, 4 0.3
_
_
_
Phn b ban u l (0, 7; 0, 2; 0, 1).
i) Lp bng phn b xc sut ca X
2
.
ii) Tnh P(X
0
= 1, X
1
= 3, X
2
= 3, X
3
= 2).
iii) Tm phn b dng.
7. Cho xch Markov X
n
, n = 0, 1, 2, ... vi khng gian trng thi E =
{1, 2, 3} v ma trn xc sut chuyn
P =
_
_
_
3/7 3/7 1/7
1/11 2/11 8/11
1/11 4/11 6/11
_
_
_
Gi s P(X
0
= 1) = 1. t
Y
n
=
_
_
_
1 nu X
n
= 1
2 nu X
n
= 1
Chng minh rng (Y
n
) l mt xch Markov. Tm ma trn xc sut
chuyn.
8. Cho (r
n
) l dy Rademakher P(r
n
= 1) = p, P(r
n
= 1) = q = 1 p.
Trong cc dy LNN sau y dy no lp thnh xch Markov
(a) X
n
= r
n
r
n+1
(b) X
n
= r
1
r
2
...r
n
1.4. Bi tp 61
(c) X
n
= (r
n
, r
n+1
) trong
(1, 1) = 1, (1, 1) = 2, (1, 1) = 3, (1, 1) = 4
i vi dy lp thnh xch Markov hy tm ma trn xc sut chuyn.
9. Mi ngi dn ca th trn N c mt trong ba ngh A, B, C. Con ci
ca h ni tip ngh ca cha mnh vi xc sut tng ng cho cc ngh
A, B, C l 3/5; 2/3; 1/4. Nu khng theo ngh ca cha th chng chn
mt trong hai ngh cn li vi xc sut nh nhau. Gi s th h hin
ti 20% theo ngh A 30% theo ngh B v 50% theo ngh C. Hy tm
(a) Tm phn b ngh nghip th h tip theo.
(b) Tm phn b gii hn theo ngh nghip ca dn c th trn trong
tng lai xa xi.
10. Cho xch Markov (X
n
), n = 0, 1, 2, ... vi khng gian trng thi E =
{1, 2, 3} v ma trn xc sut chuyn
P =
_
_
_
0 0 1
1 0 0
1/2 1/2 0
_
_
_
(a) Chng minh rng xch ti gin.
(b) Tm chu k ca xch.
(c) Tm phn b dng.
11. Cho xch Markov (X
n
), n = 0, 1, 2, ... vi khng gian trng thi E =
{1, 2, 3, 4, 5} v ma trn xc sut chuyn
P =
_
_
_
_
_
_
_
_
0 1/3 2/3 0 0
0 0 0 1/4 3/4
0 0 0 1/4 3/4
1 0 0 0 0
1 0 0 0 0
_
_
_
_
_
_
_
_
62 Chng 1. Qu trnh Markov
(a) Chng minh rng xch ti gin.
(b) Tm chu k ca xch.
(c) Tm phn b dng.
12. Gi s ta theo di s xut hin ca bin c A theo thi gian. K hiu
X
t
l s ln xut hin A trong khong thi gian (0, t] v gi thit X
t
l qu trnh Poisson vi cng .
(a) Tm xc sut trong khong thi gian (0, s] bin c A xy ra m
ln nu bit rng trong khong thi gian (0, t] bin c A xy ra n
ln 0 < s < t, 0 < m < n.
(b) K hiu T
m
l thi im m bin c A xut hin ln th m. Tm
hm phn b xc sut ca T
m
.
(c) Tm hm mt ca T
m
.
(d) Tm P(T
1
s|X
t
= n) 0 < s < t.
13. Cho qu trnh cht thun tu X
t
vi khng gian trng thi E =
{0, 1, 2, ...}
(a) Vit phng trnh thun cho qu trnh X
t
.
(b) Tm P
ii
(t).
(c) Tm P
ij
(t) theo P
i,j+1
(t).
(d) Tnh P
i,i1
(t).
(e) Chng minh rng nu X
t
l qu trnh cht tuyn tnh tc l
i
= i
th vi i > j ta c
P
ij
(t) =
_
i
j
_
(e
t
)
j
(1 e
t
)
ij
.
14. Cho qu trnh sinh v cht X
t
vi
i
= i,
i
= i , l cc hng
s dng. C nh i. t
m
i
(t) = E
i
X(t) =

j=0
jP
ij
(t)
1.4. Bi tp 63
(a) Vit phng trnh thun cho qu trnh X
t
.
(b) S dng phng trnh thun chng minh rng m

i
(t) = (
)m
i
(t).
(c) Suy ra
m
i
(t) = ie
()t
.
15. Cho qu trnh sinh v cht X
t
vi
i
= i,
i
= i , l cc hng
s dng. C nh i. t
s
i
(t) = E
i
X
2
(t) =

j=0
j
2
P
ij
(t).
(a) S dng phng trnh thun chng minh rng
s

i
(t) = 2( )s
i
(t) + ( +)m
i
(t).
(b) Tm s
i
(t).
(c) Tm V arX
t
nu X
0
= i.
16. Chng minh rng h
f(t, x, y) =
t
(t
2
+ (x y)
2
)
lp thnh h mt chuyn tho mn phng trnh Chapman - Kol-
mogorov. Qu trnh Markov tng ng c tn l qu trnh Cauchy.