IE 621: Probability and Stochastic Processes 1

Quiz 2 Solution
1. (5 points) The students of a Training School are organized into four classes, labeled 1 (starting class), 2, 3, and
4. At the end of a week, a student may get promoted from one class to the next, or decide to leave the school.
A student in class 1 gets promoted to class 2 with probability 0.3, leaves the school with probability 0.1, or
continues in the same class at the beginning of the next week. A student in class 2 gets promoted to class 3 with
probability 0.2, leaves the school with probability 0.15, or continues in the same class at the beginning of the
next week. A student in class 3 gets promoted to class 4 with probability 0.05, leaves the school with probability
0.2, or continues in the same class at the beginning of the next week. Finally, a student in class 4 for the first
time either leaves the school successfully with probability 0.6 or continues an additional week in the same class.
At the end of the second week, the student always leaves the school: the probability of success at the end of the
second week is 0.8.
Model this as a discrete time homogeneous Markov chain. Please note that your model should encompass all
the information. Please mention very clearly the state space, transition matrix and the communication diagram.
Identify the communicating classes of this Markov chain.

Solution: Define the state space as S = {1, 2, 3, 4W 1, 4W 2, S, L}.

The Communication or Transition probability diagram is:

The Transition Probability Matrix is:

 
0.6 0.3 0 0 0 0 0.1
 0 0.65 0.2 0 0 0 0.15
 
0 0 0.75 0.05 0 0 0.2 
 
P = 0 0 0 0 0.4 0.6 0.1 

0 0 0 0 0 0.8 0.2 
 
0 0 0 0 0 1 0 
0 0 0 0 0 0 1

Aug - Nov, 2023 IEOR@IITB

IE 621: Quiz 2 2

The Communicating Classes are {1}, {2}, {3}, {4W 1}, {4W 2}, {S}, {L}.

2. (5 points) A small warehouse may contain up to three pallets. Every hour a truck comes to collect stored
pallets. Depending on the space available on it, the maximum number of pallets that the truck may collect is 0
with probability p0 = 0.1, 1 with probability p1 = 0.2, 2 with probability p2 = 0.4 and 3 otherwise. The truck
always collects as many pallets as possible compatibly with this constraint. The number of pallets shipped to the
warehouse during one hour is a random variable taking values 0 with probability q0 = 0.3, 1 with probability
q1 = 0.4 and 2 otherwise. Pallets arriving when the warehouse is full are rejected. Assume that pallet loading
and unloading times are negligible.
Model the system through a discrete-time homogeneous Markov chain. Please note that your model should
encompass all the information. Please mention very clearly the state space, transition matrix and the communi-
cation diagram. Identify the communicating classes of this Markov chain.

Solution: Let the state space be defined as S = {0, 1, 2, 3} denoting the number of pallets in the warehouse
after the truck has left. New pallets would be shipped to the warehouse during the hour before the next truck
collects pallets and leaves. Thus, the state transition probabilities can be computed as follows,
P0,0 = q0 + q1 (p1 + p2 + p3 ) + q2 (p2 + p3 ) = 0.3 + 0.4(0.9) + 0.3(0.7) = 0.87
P0,1 = q1 (p0 ) + q2 (p1 ) = 0.4(0.1) + 0.3(0.2) = 0.1
P0,2 = q2 (p0 ) = 0.3(0.1) = 0.03
P0,3 = 0
P1,0 = q0 (p1 + p2 + p3 ) + q1 (p2 + p3 ) + q2 (p3 ) = 0.3(0.9) + 0.4(0.7) + 0.3(0.3) = 0.64
P1,1 = q0 (p0 ) + q1 (p1 ) + q2 (p2 ) = 0.3(0.1) + 0.4(0.2) + 0.3(0.4) = 0.23
P1,2 = q1 (p0 ) + q2 (p1 ) = 0.4(0.1) + 0.3(0.2) = 0.1
P1,3 = q2 (p0 ) = 0.3(0.1) = 0.03
P2,0 = q0 (p2 + p3 ) + q1 (p3 ) + q2 (p3 ) = 0.3(0.7) + 0.4(0.3) + 0.3(0.3) = 0.42
P2,1 = q0 (p1 ) + q1 (p2 ) + q2 (p2 ) = 0.3(0.2) + 0.4(0.4) + 0.3(0.4) = 0.34
P2,2 = q0 (p0 ) + q1 (p1 ) + q2 (p1 ) = 0.3(0.1) + 0.4(0.2) + 0.3(0.2) = 0.17
P2,3 = q1 (p0 ) + q2 (p0 ) = 0.4(0.1) + 0.3(0.1) = 0.07
P3,0 = q0 (p3 ) + q1 (p3 ) + q2 (p3 ) = 0.3(0.3) + 0.4(0.3) + 0.3(0.3) = 0.3
P3,1 = q0 (p2 ) + q1 (p2 ) + q2 (p2 ) = 0.3(0.4) + 0.4(0.4) + 0.3(0.4) = 0.4
P3,2 = q0 (p1 ) + q2 (p1 ) + q2 (p1 ) = 0.3(0.2) + 0.4(0.2) + 0.3(0.2) = 0.2
P3,3 = q0 (p0 ) + q1 (p0 ) + q2 (p0 ) = 0.3(0.1) + 0.4(0.1) + 0.3(0.1) = 0.1

Aug - Nov, 2023 IEOR@IIT Bombay

IE 621: Quiz 2 3

There is only one communicating class which is S.

- BC-

Aug - Nov, 2023 IEOR@IIT Bombay