Cairo University

Graduate Computer Networks

Faculty of Engineering

2014 2015

Electronics & Comm. Dept.

Problem Set 1
1. A 1 is followed by a 1 with probability p and by a 0 with probability 1-p.
Likewise, a 0 is followed by a 0 with probability p and by a 1 with probability 1-p.
Find the steady-state probability of a 1 or a 0.
2. Whether it rains or not today depends on whether it rained or not during the last three
days.
a. Show that this can be modeled by a Markov chain; how many states are
needed?
b. Suppose that if it has rained for the past three days, it will rain today with
probability 0.8. If it did not rain on any of the past three days, it will rain today
with probability 0.2. In any other case, the weather will be the same as
yesterdays with probability 0.6. Find the state transition and steady-state
probabilities.
3. The following matrices are state-transition probability matrices of Markov chains,
determine whether each Markov Chain reaches Steady-State or not.

0
0 .5

0.5

0 .5
0
0 .5

0 .5
0.5
0

0
0

0 .5

0
0
0 .5
0

0
0
0
1

1
1
0

4. On day 0 a house has two new light bulbs in reserve. The probability that the house
will need a single new light bulb during day n is p, and the probability that it will not
need any is q = 1 p. Let Yn be the number of new light bulbs left in the house at the end
of day n. Show that Yn is a Markov chain and derive its state transition diagram, transition
probability matrix and the steady-state probabilities.