Introduction to Stochastic Processes Spring 2021 Assignment 10

May 10-12, 2021

1. Mr. W operates a small restaurant at Berkeley. Due to the pandemic, customers can only take out
the foods through walk-in service. Once a customer arrives, an order will be placed immediately and
the cooks of the restaurant will process orders in a first come first serve pattern. After an order is
finished by a cook, the customer will pick up the food and leave immediately. During the time the
food is prepared, a customer waits in the restaurant. For simplicity, we assume this restaurant opens
24 hours every day and customers arrive to the restaurant according to a Poisson process with
constant rate 20 per hour. The cooking time for each order is exponentially distributed with rate 10
per hour. Each cook in the restaurant can only process one order at one time. Each cook works
independently. The time it takes to place an order is considered to be negligible (e.g., through mobile
apps or kiosks) and is not counted in the model. Consider a continuous time stochastic process
{X(t) : t ≥ 0} where X(t) is the number of customers in the restaurant at time t.

a.) Suppose that there is only one cook in the restaurant. When an customer arrives at the

restaurant, she has a probability of immediately leaving the restaurant without placing an
order at all. This probability is n/(n + 1) if there are already n customers in the restaurant.
Find the invariant distribution of the number of customers in the restaurant.

b.) Suppose there are now 2 cooks in the restaurant. When an customer arrives at the restaurant,
she immediately leaves the restaurant without placing an order at all, if the restaurant already
has 5 customers. Otherwise the customer stays and places an order. In equilibrium, what is
the fraction of arriving customers that will leave immediately?

2. A police officer on University Avenue is monitoring the traffic. He finds the following information:
for each truck on the road, the probability that a car follows it is 3/5; for each car on the road, the
probability that a truck follows it is 1/5. Assuming all vehicles on the road are either truck or car,
and every vehicle is followed by exactly one vehicle. On average, what fraction of vehicles on the
road are trucks? What fraction of consecutive pairs of vehicles on the road are a car followed by a
truck?

3. Driver M has just purchased auto insurance. The insurance company has three different
categories into which it classifies drivers. The categories are A = low risk, B = medium risk,
C = high risk. The premiums charged per 6 month period are $300, $600, and $1200, respec-
tively. Suppose that Daniela decides to model her transitions between categories as a Markov chain
 Page 2 of 3
Introduction to Stochastic Processes

X = (Xn : n ≥ 0). A transition would only happen at the end of each 6-month cycle and remains
for 6 months before the next transition. Her estimated transition probability matrix is

A B C
 
A 1/2 1/4 1/4
Pjump = B  3/4 0 1/4 .
C 0 1/4 3/4

a.) Driver M wishes to compute the expected total cost of her policy over the next 18 months.
Compute this quantity. Assume she starts as an A driver in the first 6 months of the next 18
months.

b.) Driver M also wishes to compute an approximation to the total cost of her policy over the
next 10 years. Produce such an approximation and list the computing equations. (Hint: The
total cost can be approximated by the long run average cost multiplied by the number of time
periods. Theoretically, ni=1 r(Xi ) ≈ nπr when n is large.)
P

c.) Driver M believes that the insurance for class-C drivers is too high. If she ends up being in
this class, she has decided she will switch to another insurance company after one six-month
period in class C. What is the expected time periods to switch, if she starts as an A driver?

d.) Following part c.), what are the expected total premiums she can expect to pay out to the
company before switching, if she starts as an A driver?

4. Let (N (t) : t ≥ 0) be a Poisson arrival process with constant rate 1. Let Tn denote the time of
the n-th arrival. N (t) denotes the total number of arrivals that have occurred by time t. Compute
the following quantities.

a.) P (T4 > 5.5 | T3 = 5)

b.) E(N (15) − N (5) | T1 = 5)

c.) E(N (15) | N (3) = 5)

d.) E(N (15) − N (5) | T1 = 8)

e.) E(N (15) − N (5) | T2 = 8)

5. Consider the following transition graph for a discrete-time Markov chain (Xn : n ≥ 0).

a.) When x = 1/3 (such that the transition arrow from state 7 to state 2 would be gone), identify
the communicating classes. Which states are transient? Which states are recurrent?
 Page 3 of 3
Introduction to Stochastic Processes

b.) When x = −1/3 (such that some transition arrows on the graph would be gone), identify the
communicating classes. Which states are transient? Which states are recurrent?

c.) For what values of x is the Markov chain irreducible? Explain why. (Note that in the first
place such x should make the transition probabilities meaningful.)