MATH 5040(001) Fall 2019

Homework 4

Name:

Solve the following problems.

1. Consider a continuous time Markov chain with states {1, 2, 3, 4}. From 1 it jumps to any
of the other states with rate 1 each. From 2 it jumps to 3 at rate 2 and to 4 at rate 1.
From 3 it jumps to 1 or 4 at rate 1 and to 2 at rate 2. From 4 it can only jump to 3 and
it does that at rate 1.

(a) Draw the graph for this chain and write down the rate matrix (the infinitesimal
generator matrix).
(b) Find the equilibrium distribution.
(c) Suppose the chain starts in state 1. What is the expected amount of time it requires
to jump away from that state? Why?
(d) Assume the chain starts in state 2. What is the expected amount of time until the
chain is in state 1?
(e) Turn this Markov chain into a chain that uses one exponential clock and a transition
matrix of a discrete Markov chain. Compute the invariant measure of the discrete
Markov chain.
(f) Assume the discrete Markov chain starts in state 2. What is the expected amount of
steps it will take until it is in state 1?
2. Assume that Archie, Betty, and Veronica are shopping in Smith’s for Thanksgiving dinner.
It is so late in the day that they are the only customers in the store. There are two cashiers
at the store (numbered 1 and 2) and they serve customers at rate β and γ per minute,
respectively. Assume the trio approaches the registers in alphabetical order (i.e. Archie
goes to cashier 1, Betty to cashier 2, and Veronica waits for a cashier to become available).

(a) What is the expected time in minutes for Veronica to go to a register?

(b) What is the probability that Veronica goes to Archie’s register?
(c) What is the probability that Veronica waits more than 10 minutes until a register is
free?
(d) What is the expected total time for Veronica to exit the store?
(e) What is the probability that Veronica is the last to leave?
3. Consider a queue where new customers arrive at rate λ regardless of the length of the
queue, but if there are n customers at the queue then they are served at rate µn . µ0 = 0
since if there are no customers, there is no need to serve anyone.

(a) Find a condition on λ and µn so that the system works properly (i.e. so that the state
where there are no customers in line is reached infinitely often).
(b) An M/M/k queue is one where at most k of the current n customers are served
immediately. Write this queue as a birth and death chain and give a condition for
the system to work properly.
(c) Say you join the queue and notice that all k servers are busy with customers and that
there are m − 1 customers ahead of you that are not being served yet (so there are a
total of k + m customers in the queue, including you). What is the average amount
of time it will take for one of the servers to become free?
Hint: the time it takes a given server to become free is an exponential random variable
with rate µ. The time it takes one of the servers to become free is the minimum of
these times for all k servers.
(d) Once a server becomes free, the next customer who was waiting starts being served
immediately. Consequently, at that moment in time there will again be k servers
busy, but m − 2 customers ahead of you. The previous part of the problem then
computes the distribution of the time it will take for you to go from being mth in line
to being number m − 1 in line. Compute the average time it will take you from the
time you join the queue until you start being served.
4. Assume customers arrive at a single server bank with rate 4 and they are being serviced
with rate 6.

(a) Describe this M/M/1 queue as a birth and death chain. Explain what Xt is and how
to find the rate µn and λn .
(b) Is the chain transient, null recurrent, or positive recurrent?
(c) Find the invariant probability for the queue length and compute the expected queue
length when the chain has been running for a long time.