Department of Mathematics

MTL 106/MAL 250 (Probability and Stochastic Processes)

Tutorial Sheet No. 6
Answer for Selected Problems
1. Cov(X(s), X(t)) = E[X(s), X(t)] E[X(s)]E[X(t)]
Cov(X(s), X(t)) = 1 + st + s2 t2

E[X(t)] = 0,
2. Yes
3. Yes, since (a)

E(X(t)) =

1
2

for all t, (b) E(X 2 (t)) =

1
4

(c) Cov(X(t), X(s)) = 14 e2|ts|

5. State Space={0,1,2,3,4,5}

P =

0.3 0.4
0.3 0.4
0 0.3
0
0
0
0
0
0

0.3 0
0
0.3 0
0
0.4 0.3 0
0.3 0.4 0.3
0 0.3 0.4
0
0 0.3

0
0
0
0
0.3
0.7

6. S = {0, 1, 2, 3, . . .}

1p
q
0
0
0
0

p
1 (p + q)
q
0

0
p
1 (p + q)
q
q

7. S = {(0,
0), (0, 1), (1, 0), (1, 1)}
q0,0 1 q0,0
0
0
0
0
q0,1 1 q0,1

P =
q1,0 1 q1,0
0
0
0
0
q1,1 1 q1,1
2
+ q0,1 (1 q0,0 )
(c) q0,0
8. (a) 0.212

0
0
p
1 (p + q)
1 (p + q)

0 0
0 0
0 0
p 0
p

(b) 0.0016

9. S = (. . . , 2, 1, 0, 1, 2, . . .)

0
0
0
0

1
4

0
0
0

1
2
1
4

0
0

1
4
1
2
1
4

0
1
4
1
2
1
4

0
0
1
4
1
2

0
0
0
1
4

10. Yn = number of hosts having the message at the end of nth round with state space S = {1, 2, 3, 4, 5}

P =

0
0
0
0
0

1
16

9
16
8
64

6
16
38
64
81
256

0
0
0

0
0

0
0

18
64
175
256
1

{1, 2, 3, 4}-transient, {5}-absorbing (positive recurrent).

11.

1 v0
1 v1
1 v2
1 v3
1 v4
1 v5

0
0
0
0
0
0

v0
0
0
0
0
0

0
v1
0
0
0
0

0
0
v2
0
0
0

0
0
0
v3
0
0

{0, 2, 4, } Closed Communicating Class

i
When i=0 v2i = 0, i=0 n=0 v2n < - positive recurrent states,
otherwise null recurrent states

When i=0 v2i > 0 - transient states

{1, 3, 5, } transient states
12. a) {3} absorbing state, {0, 1, 2, 4} - transient states, {3} - closed communicating class
b) {2, 4} positive recurrent, closed communicating class
{0, 1, 3} transient states
{ n1
Cj1 pnj q j if j n
(n)
14. fj0 =
0
if j > n
15. S = {0, 1, 2, ....}
{0} absorbing state
{1, 2, 3, ...} - transient states.
16. (b) vi = pi (1 p)

i = 0, 1, 2, 3, . . .

17. ?