Markov Analysis

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna
 Introduction

• Markov analysis is a technique that deals with the

probabilities of future occurrences by analyzing presently
known probabilities.

• The technique has numerous applications in business,

including market share analysis, bad debt prediction,
university enrollment predictions, and determining
whether a machine will break down in the future.

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna
 Introduction
• Markov analysis makes the assumption that the system starts
in an initial state or condition.
• For example, two competing manufacturers might have 40%
and 60% of the market sales, respectively, as initial states.
Perhaps in two months the market shares for the two
companies will change to 45% and 55% of the market,
respectively.
• Predicting these future states involves knowing the system's
likelihood or probability of changing from one state to
another.

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna
 Introduction

• For a particular problem, these probabilities can be

collected and placed in a matrix or table.

• This matrix of transition probabilities shows the

likelihood that the system will change from one time
period to the next. This is the Markov process, and it
enables us to predict future states or conditions.

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna
Assumptions of Markov Analysis
• There are a limited or finite number of possible states.

• The probability of change remains the same over time.

• Future state predictable from current state.

• Size of system remains the same.

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna
 States and State Probabilities
• States are used to identify all possible conditions of a
process or a system.
• For example, a machine can be in one of two states at any
point in time. It can be either functioning correctly or not
functioning correctly.
• We can call the proper operation of the machine the first
state, and we can call the incorrect functioning the second
state.
• Indeed, it is possible to identify specific states for many
processes or systems.

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna
 States and State Probabilities
• In Markov analysis we also assume that the states are
both collectively exhaustive and mutually exclusive.
• Collectively exhaustive means that we can list all of the
possible states of a system or process.
• Our discussion of Markov analysis assumes that there
is a finite number of states for any system.
• Mutually exclusive means that a system can be in only
one state at any point in time. A student can be in only
one of the three management specially areas and not in
two or more areas at the same time. It also means that
a person can only be a customer of one of the three
grocery stores at any point in time.

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna
 States and State Probabilities
• After the states have been identified, the next step is to
determine the probability that the system is in this state.
Such information is then placed into a vector of the
probabilities.
(i) = vector of State probabilities
= (1, 2, 3, … n)
Where
n = number of states
1, 2, 3, … n = probability of being in state 1, state 2,
….., state n.

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna
 States and State Probabilities
• In some cases, where we are only dealing with one item,
such as one machine, it is possible to know with complete
certainty what state this item is in.
• For example, if we are investigating only one machine,
we may know that at this point in time the machine is
functioning correctly. Then the vector of states can be
represented as follows:
(1)=(1,0)
Where
(1)= vector of states for the machine in period 1
1=1= probability of being in the first state
2=0= probability of being in the second state

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna
 States and State Probabilities
• The last one machine example shows that the
probability the machine is functioning correctly, state
1, is 1, and the probability that the machine is
functioning incorrectly, state 2, is 0 for the first period.

• In most cases, however, we are dealing with more than

one item.

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna
 Grocery Store Example
• Let's look at the vector of states for people in the small town
with the three grocery stores. There could be a total of
100,000 people that shop at the three grocery stores during
any given month. Forty thousand people may be shopping at
American Food Store, which will be called state 1. Thirty
thousand people may be shopping at Food Mart, which will
be called state2, and 30,000 people may be shopping at Atlas
Foods, which will be called state 3. The probability that a
person will be shopping at one of these three grocery stores
is as follows:
State 1 – American Food Store: 40,000/100,000=0.40=40%
State 2– Food Mart: 30,000/100,000=0.30=30%
State 3 – Atlas Foods: 30,000/100,000=0.30=30%

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna
 Grocery Store Example
• These probabilities can be placed in the vector of state
probabilities shown as follows:
(1)=(0.4, 0.3, 0.3)
Where
(1)= vector of state probabilities for the three grocery stores for
period 1
1= 0.4= probability that a person will shop at American Food,
state1

2= 0.3= probability that a person will shop at Food Mart, state2

3= 0.3= probability that a person will shop at Atlas Food, state3
To accompany Quantitative Analysis for
Management, 8e
by Render/Stair/Hanna
 Grocery Store Example
• You should also notice that the probabilities in the vector of
states for the three grocery stores represent the market
shares for these three stores for the first period. Thus,
American Food has 40% of the market, Food Mart has
30%, and Atlas Foods has 30% of the market in period 1.
when we are dealing with market shares, the market shares
can be used in place of probability values.

• When the initial states and state probabilities have been

determined, the next step is to find the matrix of transition
probabilities. This matrix is used along with the state
probabilities in predicting the future.
To accompany Quantitative Analysis for
Management, 8e
by Render/Stair/Hanna
Matrix of Transition Probabilities
• The concept that allows us to get from a current state,
such as market shares, to a future state is the matrix of
transitions probabilities.
• This is a matrix of conditional probabilities of being in
a future state given a current state. The following
definition is helpful:
Let Pij = conditional probability of being in state j in
the future given the current state of i.
• For example, P12 is the probability of being in state 2 in
the future given the event was in state 1 in the period
before.
To accompany Quantitative Analysis for
Management, 8e
by Render/Stair/Hanna
 Matrix of Transition
Probabilities
Let P = matrix of transition probabilities

 P11 P12 P13...........1n 

 P 21 P 22 P 23.........P 2n 
P 
 P31 P32 P33........P3n 
 
 Pm1............................Pmn 

• Individual Pij values are usually determined empirically.

For example, if we have observed over time that 10% of
the people currently shopping at store 1 (or state 1) will
be shopping at store 2 (state 2) next period, then we know
that P12 = 0.1 or 10%.

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna
 Transition Probabilities for the
Three Grocery Stores
• Let's say we can determine the matrix of transition
probabilities for the three grocery stores by using historical
data. The results of our analysis appear in the following
matrix:
.8 .1 .1
P  .1 .7 .2
.2 .2 .6

• Recall that American Food represents state 1, Food Mart is

state 2, and Atlas Foods is state 3. The meaning of these
probabilities can be expressed in terms of the various
states, as follows:
To accompany Quantitative Analysis for
Management, 8e
by Render/Stair/Hanna
 Transition Probabilities for the
Three Grocery Stores
• Row 1
0.8 = P11 = probability of being in state 1 after being in state 1 the preceding
period.
0.1 = P12 = probability of being in state 2 after being in state 1 the preceding
period.
0.1 = P13 = probability of being in state 3 after being in state 1 the preceding
period.

• Row 2
0.1 = P21 = probability of being in state 1 after being in state 2 the preceding
period.
0.7 = P22 = probability of being in state 2 after being in state 2 the preceding
period.
0.2 = P23 = probability of being in state 3 after being in state 2 the preceding
period.

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna
 Transition Probabilities for the
Three Grocery Stores
• Row 3
0.2 = P31 = probability of being in state 1 after being in state 3 the preceding
period.
0.2 = P32 = probability of being in state 2 after being in state 3 the preceding
period.
0.6 = P33 = probability of being in state 3 after being in state 3 the preceding
period.

• Note that the three probabilities in the top row sum to 1. The
probabilities for any row in a matrix of transition probabilities
will also sum to 1.
• After the state probabilities have been determined along with
the matrix of transition probabilities, it is possible to predict
future state probabilities.
To accompany Quantitative Analysis for
Management, 8e
by Render/Stair/Hanna
Predicting Future Markets Shares
• Given the vector of state probabilities and the matrix of
transition probabilities, it is not very difficult to determine
the state probabilities at a future date.
• With this type of analysis, we are able to compute the
probability that a person will be shopping at one of the
grocery stores in the future.
• Because this probability is equivalent to market share, it is
possible to determine future market shares for American
Food, Food Mart, and Atlas Foods.

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna
Predicting Future Markets Shares
• When the current period is 1, calculating the state
probabilities for the next period (period 2) can be
accomplished as follows:
(2)=  (1)P ………… (1)

• Furthermore, if we are in any period n, we can compute the

state probabilities for period n+1 as follows:
(n+1)=  (n)P ………… (2)

• Equation (1) can be used to answer the question of next

period's market shares for the grocery stores. The
computations are:

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna
 Predicting Future Markets Shares
(2)=  (1)P
.8 .1 .1
 (2)  .4 .3 .3 .1 .7 .2
.2 .2 .6
= [(0.4)(0.8) + (0.3)(0.1) + (0.3)(0.2), (0.4)(0.1) + (0.3)(0.7)+ (0.3)(0.2),
(0.4)(0.1) + (0.3)(0.2) + (0.3)(0.6)]

= (0.41, 0.31, 0.28)

As you can see, the market share for American Food and Food
Mart has increased while the market share for Atlas Foods
has decreased.

To accompany Quantitative Analysis for

Management, 8e
by Render/Stair/Hanna