NCKU IIM Stochastic Process

Homework #2
This assignment has four problems and will be graded out of 100 points. Please show all work (or
hand in computer printouts if you use Excel/Maple/etc. to do calculations) in order to receive full
credit.

PROBLEM 1:
Consider the capacitated queueing model discussed in class, and assume that K = 6 , i.e.,
at most 6 people can be in the system at the same time. Suppose we start with an empty
system (i.e., X 0 = 0 ). Assuming that the number of people arriving during each period
has binomial distribution with parameters N = 4, p = 0.2, compute:
(a) The probability of the sample path (0, 1, 3, 4, 3, 2).
(b) P= ( X 2 j=
), j 1, 2,3, 4,5, 6.
(c) P= ( X 2 4,
= X 3 1) .
(d) =P( X 4 4= | X 2 1) .
(e) The expected number of customers in the system at time n for n=1, 3, 7 and 15. I
suggest you use Maple (or other software, like Matlab) for this problem, unless you
want to spend a lot of time multiplying matrices by hand!

PROBLEM 2:
Consider again the (uncapacitated) queueing model discussed in class. Now suppose that
at most a single customer arrives during a single period, but that the service time of a
customer is a random variable S with the geometric probability distribution, i.e.,
k ) α (1 α ) k −1 , k =
P( S ==− 1, 2,....
As before, let X n be the number of customers in the system at the end of period n.
Assuming that the probability that a customer arrives in a period is β (so the probability
that no customer arrives is 1-β ), specify the transition matrix for {X n } .
HINT: Notice that the geometric distribution has the property that

P( S = k , S ≥ k ) P( S = k) α (1 − α ) k −1
P(=
S k | S ≥ k=
) = = = α.
∑ j =k
∞
P( S ≥ k ) P( S ≥ k ) α (1 − α ) j −1

PROBLEM 3:
A production line produces one unit at a time. Each item produced is inspected until m consecutive
nondefective items are found. When this occurs, we enter a “sampling phase”, which means that
items are randomly inspected. More specifically, each item is inspected with probability 1/r. As
soon as a defective item is found, the sampling plan calls for reverting back to inspecting every
item until m consecutive nondefective items are again found — and the cycle repeats. Assume that
the probability that an item is defective is p.
1
 Let X n be defined as follows: X n = 0 if the nth item is found defective; otherwise, X n is
the number of previously consecutive inspected items found to be nondefective (with X n = m
if that number is bigger than m).
Is { X n } a Markov chain? Justify. If so, determine its state-space and compute the transition
probability matrix.

PROBLEM 4:
A simplified model for the spread of a disease goes this way: the total population size is N = 5, of
which some are diseased and some are healthy. During any single period of time, two people are
selected at random from the population and assumed to interact. The selection is such that an
encounter between any pair of individuals in the population is just as likely as any other pair. If one
of these persons is diseased and the other not, then with probability 0.1 the disease is transmitted to
the healthy person. Otherwise, no disease transmission takes place.

Describe a Markov Chain model for the system. That is, specify the stochastic process Xn, the time
index n, the state space, and the transition probability matrix.