# Markov Analysis

To accompany Quantitative Analysis for Management, 8e by Render/Stair/Hanna

Introduction
 Markov analysis is a technique that deals with the probabilities of future occurrences by analyzing presently known probabilities.  The technique has numerous applications in business, including market share analysis, bad debt prediction, university enrollment predictions, and determining whether a machine will break down in the future.

To accompany Quantitative Analysis for Management, 8e by Render/Stair/Hanna

Introduction  Markov analysis makes the assumption that the system starts in an initial state or condition. respectively. Perhaps in two months the market shares for the two companies will change to 45% and 55% of the market.  For example. two competing manufacturers might have 40% and 60% of the market sales. as initial states.  Predicting these future states involves knowing the system's likelihood or probability of changing from one state to another. 8e by Render/Stair/Hanna . respectively. To accompany Quantitative Analysis for Management.

and it enables us to predict future states or conditions.Introduction  For a particular problem. these probabilities can be collected and placed in a matrix or table.  This matrix of transition probabilities shows the likelihood that the system will change from one time period to the next. 8e by Render/Stair/Hanna . To accompany Quantitative Analysis for Management. This is the Markov process.

8e by Render/Stair/Hanna .Assumptions of Markov Analysis  There are a limited or finite number of possible states. To accompany Quantitative Analysis for Management.  The probability of change remains the same over time.  Size of system remains the same.  Future state predictable from current state.

and we can call the incorrect functioning the second state. To accompany Quantitative Analysis for Management.States and State Probabilities  States are used to identify all possible conditions of a process or a system. 8e by Render/Stair/Hanna . it is possible to identify specific states for many processes or systems. It can be either functioning correctly or not functioning correctly. a machine can be in one of two states at any point in time.  We can call the proper operation of the machine the first state.  Indeed.  For example.

States and State Probabilities  In Markov analysis we also assume that the states are both collectively exhaustive and mutually exclusive. To accompany Quantitative Analysis for Management. 8e by Render/Stair/Hanna .  Collectively exhaustive means that we can list all of the possible states of a system or process. A student can be in only one of the three management specially areas and not in two or more areas at the same time.  Our discussion of Markov analysis assumes that there is a finite number of states for any system. It also means that a person can only be a customer of one of the three grocery stores at any point in time.  Mutually exclusive means that a system can be in only one state at any point in time.

«. state 2. « Tn) Where n = number of states T1. T3. T2.States and State Probabilities  After the states have been identified. To accompany Quantitative Analysis for Management. Such information is then placed into a vector of the probabilities. T3. T2.. T(i) = vector of State probabilities = (T1.. « Tn = probability of being in state 1. state n. 8e by Render/Stair/Hanna . the next step is to determine the probability that the system is in this state.

such as one machine. it is possible to know with complete certainty what state this item is in. 8e by Render/Stair/Hanna . where we are only dealing with one item. if we are investigating only one machine.0) Where T(1)= vector of states for the machine in period 1 T1=1= probability of being in the first state T2=0= probability of being in the second state To accompany Quantitative Analysis for Management. we may know that at this point in time the machine is functioning correctly. Then the vector of states can be represented as follows: T(1)=(1.  For example.States and State Probabilities  In some cases.

8e by Render/Stair/Hanna . is 0 for the first period. To accompany Quantitative Analysis for Management.States and State Probabilities  The last one machine example shows that the probability the machine is functioning correctly. is 1. state 2.  In most cases. and the probability that the machine is functioning incorrectly. state 1. however. we are dealing with more than one item.

which will be called state 1. The probability that a person will be shopping at one of these three grocery stores is as follows: State 1 ± American Food Store: 40.30=30% To accompany Quantitative Analysis for Management. which will be called state 3.30=30% State 3 ± Atlas Foods: 30.000/100.000 people that shop at the three grocery stores during any given month. which will be called state2.000=0.Grocery Store Example  Let's look at the vector of states for people in the small town with the three grocery stores. 8e by Render/Stair/Hanna .000 people may be shopping at Atlas Foods. Thirty thousand people may be shopping at Food Mart. Forty thousand people may be shopping at American Food Store.000/100.000/100.000=0. and 30.000=0.40=40% State 2± Food Mart: 30. There could be a total of 100.

3.3= probability that a person will shop at Atlas Food.Grocery Store Example  These probabilities can be placed in the vector of state probabilities shown as follows: T(1)=(0. 0. 8e by Render/Stair/Hanna . state3 To accompany Quantitative Analysis for Management.3= probability that a person will shop at Food Mart.4. 0.3) Where T(1)= vector of state probabilities for the three grocery stores for period 1 T1= 0. state2 T3= 0. state1 T2= 0.4= probability that a person will shop at American Food.

Grocery Store Example  You should also notice that the probabilities in the vector of states for the three grocery stores represent the market shares for these three stores for the first period. and Atlas Foods has 30% of the market in period 1. To accompany Quantitative Analysis for Management. the market shares can be used in place of probability values. the next step is to find the matrix of transition probabilities. This matrix is used along with the state probabilities in predicting the future. 8e by Render/Stair/Hanna .  When the initial states and state probabilities have been determined. when we are dealing with market shares. American Food has 40% of the market. Food Mart has 30%. Thus.

The following definition is helpful: Let Pij = conditional probability of being in state j in the future given the current state of i. such as market shares.  For example.  This is a matrix of conditional probabilities of being in a future state given a current state. to a future state is the matrix of transitions probabilities.Matrix of Transition Probabilities  The concept that allows us to get from a current state. To accompany Quantitative Analysis for Management. P12 is the probability of being in state 2 in the future given the event was in state 1 in the period before. 8e by Render/Stair/Hanna .

. 8e by Render/Stair/Hanna ... For example.Pmn ½  Individual Pij values are usually determined empirically......1n » ¬ P 21 P 22 P 23...... if we have observed over time that 10% of the people currently shopping at store 1 (or state 1) will be shopping at store 2 (state 2) next period............ then we know that P12 = 0........1 or 10%...P3n ¼ ¼ ¬ ­ Pm1..Matrix of Transition Probabilities Let P = matrix of transition probabilities « P11 P12 P13..... To accompany Quantitative Analysis for Management.......P 2n ¼ ¼ P!¬ ¬ P31 P32 P33.........

as follows: To accompany Quantitative Analysis for Management.1 .2¼ ¬ ¼ ¬.2 . The meaning of these probabilities can be expressed in terms of the various states.1 .Transition Probabilities for the Three Grocery Stores  Let's say we can determine the matrix of transition probabilities for the three grocery stores by using historical data. and Atlas Foods is state 3.6¼ ­ ½  Recall that American Food represents state 1.8 .7 . 8e by Render/Stair/Hanna .1» ¬. Food Mart is state 2. The results of our analysis appear in the following matrix: P «.2 .

1 = P13 = probability of being in state 3 after being in state 1 the preceding period.  Row 2 0. 8e by Render/Stair/Hanna . 0.8 = P11 = probability of being in state 1 after being in state 1 the preceding period. To accompany Quantitative Analysis for Management. 0. 0.1 = P21 = probability of being in state 1 after being in state 2 the preceding period.7 = P22 = probability of being in state 2 after being in state 2 the preceding period.1 = P12 = probability of being in state 2 after being in state 1 the preceding period. 0.2 = P23 = probability of being in state 3 after being in state 2 the preceding period.Transition Probabilities for the Three Grocery Stores  Row 1 0.

0. it is possible to predict future state probabilities.  After the state probabilities have been determined along with the matrix of transition probabilities. 0.Transition Probabilities for the Three Grocery Stores  Row 3 0. To accompany Quantitative Analysis for Management.6 = P33 = probability of being in state 3 after being in state 3 the preceding period.  Note that the three probabilities in the top row sum to 1.2 = P31 = probability of being in state 1 after being in state 3 the preceding period. 8e by Render/Stair/Hanna .2 = P32 = probability of being in state 2 after being in state 3 the preceding period. The probabilities for any row in a matrix of transition probabilities will also sum to 1.

we are able to compute the probability that a person will be shopping at one of the grocery stores in the future. 8e by Render/Stair/Hanna .  Because this probability is equivalent to market share. and Atlas Foods.  With this type of analysis. To accompany Quantitative Analysis for Management.Predicting Future Markets Shares  Given the vector of state probabilities and the matrix of transition probabilities. Food Mart. it is not very difficult to determine the state probabilities at a future date. it is possible to determine future market shares for American Food.

if we are in any period n.Predicting Future Markets Shares  When the current period is 1. The computations are: To accompany Quantitative Analysis for Management. we can compute the state probabilities for period n+1 as follows: T(n+1)= T (n)P «««« (2)  Equation (1) can be used to answer the question of next period's market shares for the grocery stores. 8e by Render/Stair/Hanna . calculating the state probabilities for the next period (period 2) can be accomplished as follows: T(2)= T (1)P «««« (1)  Furthermore.

3)(0.3 .1) + (0. 0.3)(0.8) + (0. (0.4)(0.6¼ ­ ½ = [(0.2¼ .31.41.8 . To accompany Quantitative Analysis for Management.2) + (0.3)(0.1) + (0.4)(0. 0.3A¬. 8e by Render/Stair/Hanna .28) T(2)= T (1)P As you can see.3)(0.2).7 .6)] = (0.1» T (2) ! ? . the market share for American Food and Food Mart has increased while the market share for Atlas Foods has decreased.2 .2).3)(0.3)(0.1) + (0.4 ¬ ¼ ¬.2 .1 .Predicting Future Markets Shares «. (0.1 .7)+ (0.4)(0.