You are on page 1of 12

# REVISED

## M16_REND6289_10_IM_C16.QXD 5/15/08 10:54 AM Page 250

C H A P T E R

Markov Analysis

16

TEACHING SUGGESTIONS
Teaching Suggestion 16.1: Use of Matrix Algebra. Markov analysis requires the use of matrix algebra, primarily matrix multiplication. You may want to have students review basic concepts in matrix algebra before the material in the chapter is covered. If you plan to cover absorbing state analysis in detail, more advanced matrix algebra will be needed, including the identity matrix, matrix subtraction, and the inverse of a matrix. See Module 5. Teaching Suggestion 16.2: Matrix of Transition. Markov analysis requires a known and stable matrix of transition. Students should be told that Markov analysis is not valid if the matrix of transition does not remain the same. A small change in the matrix of transition can make a big difference in equilibrium calculations. Teaching Suggestion 16.3: Application of Markov Analysis. There are a number of applications of Markov analysis. The applications box in this chapter presents an example. Students can be asked to nd additional applications in quantitative analysis/ management science journals such as Interfaces. In addition, students can be asked to develop their own problems. For example, Markov analysis can be used to predict the percentage of students who will be in certain majors next year or in the long run. Teaching Suggestion 16.4: Sensitivity Analysis and Markov Analysis. Although sensitivity analysis is not a formal part of the material discussed in this chapter, it is an important and interesting topic. Students can be asked to determine how sensitive the results of Markov analysis are to changes in probability values. Teaching Suggestion 16.5: Equilibrium Conditions and the Beginning State or Condition. As mentioned in this chapter, equilibrium conditions do not depend on the initial state or condition. The only factor that needs to be considered is the matrix of transition. While this is true, the time or number of periods needed to approach equilibrium is a function of the beginning state. Students can be asked to determine what impact the initial state has on the number of periods it takes to reach equilibrium. Teaching Suggestion 16.6: Absorbing State Analysis and Matrix Algebra. Absorbing state analysis requires more complex matrix algebra, including the inverse of the (I B) matrix. If you plan to get into the mathematics of absorbing state analysis, you may have to

spend additional time covering more advanced matrix algebra. An alternative approach is to cover the assumptions and overall approach of the model and leave the computations to the computer.

ALTERNATIVE EXAMPLES
Alternative Example 16.1: Scuba Discovery (Store 1) currently splits the market for scuba classes with Bobs Dive Shop. Given the matrix of transition probabilities below, what will the market shares be next month (period)? .7 .3 P .4 .6 We start by noting that the initial state or p1 is (.5 .5) or equal shares of the market. We can determine the market shares for next month or p2 as follows: p2 p1P

## .7 .3 2 (.5 .5) .4 .6 p2 [(.5)(.7) (.5)(.4), (.5)(.3) (.5)(.6)]

p2 [.35 .20, .15 .30] [.55, .45] Thus, Scuba Discovery, or store 1, will have a market share of .55 next month. Alternative Example 16.2: Scuba Discovery would like to determine its equilibrium market share. (See Alternate Example 16-1.) Will the store eventually capture 60% of the market in the long run? To solve this problem, we set up the equilibrium equations and solve for p1 and p2. The results are below: p1 .7p1 .4p2 p2 .3p1 .6p2 (Equation 1) (Equation 2)

We also know that p1 p2 1 or p2 1 p1. We put this equation for p2 into Equation 1 and solve for p1 as follows: p1 .7p1 .4(1 p1) .7p1 .4 .4p1 p1 .7p1 .4p1 .4; .7p1 .4 p1 .4/.7 4/7 57.14%, and thus p2 3/7. Scuba Discovery will not reach 60% of the market in the long run. We can put these values back into Equation 2 as a check as follows: p2 .3p1 .6p2 (.3)(4/7) (.6)(3/7) p2 1.2/7 1.8/7 3/7.

250

REVISED
M16_REND6289_10_IM_C16.QXD 5/15/08 10:54 AM Page 251

CHAPTER 16

MARKOV ANALYSIS

251

## SOLUTIONS TO DISCUSSION QUESTIONS AND PROBLEMS

16-1. Markov analysis makes the assumption that the propensity to change over time stays the same. This tendency to change is embodied in the matrix of transition. Furthermore, it is assumed that there is a nite number of states or conditions. In addition, we are assuming that we can predict any future state from the preceding state and that the size and makeup of the system does not change. 16-2. The vector of state probabilities is a collection of probability values that the particular system will be in a given state. In most cases it is determined using historical data. For example, if the market shares for three stores are 30%, 50%, and 20%, the probability of being at one of these stores is 0.3, 0.5, and 0.2. The matrix of transition is a probability matrix that reveals the tendency of the system to change from one state to another over a given period of time. Like the vector of state probabilities, the matrix of transition is normally determined using empirical or historical data. 16-3. Future states can be determined by multiplying the current state probabilities times the matrix of transition. If we want to determine future market shares in August, we multiply the market shares in July times the matrix of transition. This process can be repeated to determine future states several months or years in the future. 16-4. An equilibrium condition is a condition in which the state probabilities do not change from one period to the next. We can compute the equilibrium conditions by setting the unknown equilibrium state probabilities equal to the unknown state probabilities times the matrix of transition. We also include the equation that sets the sum of the state probabilities equal to 1. With this set of equations it is possible to determine the equilibrium conditions. 16-5. An absorbing state is one in which once it is entered, it cannot be left. In the matrix of transition, an absorbing state is indicated by a probability value of 1. An example of an absorbing state is paying a bill. Once the bill for a particular item is paid in full, it is impossible to owe on that item in the future. 16-6. The fundamental matrix is equal to the inverse of the identity matrix minus the B matrix. The B matrix is determined by partitioning the matrix of transition. The fundamental matrix is then multiplied times the A matrix, which is another partition of the matrix of transition. 16-7.

## 0.9 0.1 P 0.3 0.7

Probability of starting is 90%.

b. If p(0) (1 0), p(1) (0.9, 0.1) c. If p(0) (0 1), p(1) (0.3, 0.7) Probability of starting is 30%. 16-9. a. If p(0) (1 0), p(1) (0.9, 0.1), p(5) (0.76944, 0.23056) Probability it will not start ve days from today is 23.056%. b. If p(0) (0 1), p(1) (0.3, 0.7), p(5) (0.69168, 0.30832) Probability it will not start ve days from today is 30.832%. c. In equilibrium p pP p1 0.9p1 0.3p2 p2 0.1p1 0.7p2 p1 p2 1 or, p1 0.75, p2 0.25 Therefore, long-run probability of starting is 75%. 16-10. If states 1, 2, and 3 represent Dress-Rite, Fashion, Inc., and Luxury living customers: 0.1 0.2 0.7 P 0.05 0.85 0.1 0.05 0.05 0.9 If p(0) (1/3 p(1)
1/3 1/3)

1/3(1 (4/15

1)
5/15 5/15, 6/15) 6/15):

1/3(0.8

1.2) 0.33

Market shares after one month are (4/15, p(2) (0.2233 p(3) (0.1952 0.3252

0.4467) 0.4796)

After three months, market shares will be 19.52%, 32.52%, and 47.96%. 16-11. Tree Diagram
Dress- Rite #1 1/3 0.7 0.1 0.2

#1 #2 #3

## 0.9 0.1 a. 0.7 0.2

0.8 0.1 b. 0.9 0.3

## 1.15 0.16 0.33 1.48

1.30 0.14 0.43 1.16
Fashion, Inc. #2 1/3 0.05 0.85 0.1

#1 #2 #3

## 0.7 0.2 c. 0.9 0.2

0.8 0.2 d. 0.7 0.1

## 1.53 0.34 0.34 1.19

1.30 0.37 0.19 1.48
Luxury Living #3 1/3 0.05 0.05 0.9

#1 #2 #3

## 0.01667 0.01667 0.3000

REVISED
M16_REND6289_10_IM_C16.QXD 5/15/08 10:54 AM Page 252

252

CHAPTER 16

MARKOV ANALYSIS

16-12. Let states 1, 2, 3 correspond to placing 49, 50, and 51 pounds into the bags. Then

## 0.5 0.3 0.2 P 0.1 0.7 0.2 0.1 0.4 0.5

a. 70% the second column (50 pounds) of (0 1 0) P b. 30% the second column of (1 0 0) P c. 40% the second column of (0 0 1) P 16-13. a. If p(0) (0 p(1) (0.1 p(2) (0.14 p(3) (0.156 p(4) (0.1624 p(5) (0.1650 b. If p(0) (1 p(1) (0.5 p(2) (0.3 p(3) (0.22 p(4) (0.188 p(5) (0.1752 c. If p(0) (0 p(1) (0.1 p(2) (0.14 p(3) (0.156 p(4) (0.1624 p(5) (0.1650 16-14.
0.6 0.0 P 0.1 0.05 0.1 0.1 0.2 0.1 0.1 0.8 0.0 0.05 0.1 0.8 0.2 0.7

The solution for this set of simultaneous equations is p1 V\cz; p2 <\cz; p3 ZC\cz; p4 N\cz. Hence the long-run market share of the four bookstores can be expected to be 12.90%, 25.81%, 41.94%, and 19.35%, respectively.
0.80 0.10 0.10 0.20 0.70 0.10 [111 75 54] [ 100 80 60 ] 16-16. .60 0.25 0.15 0. Thus, 111 cars will be at the Northside branch, 75 at the West End branch, and 54 at the Suburban branch.

## 0) 0.7 0.6 0.2), 0.26) 0.278) 0.2834) 0.2850)

0.566 0.5542 0.55 0) 0.3 0.44 0.502 0.5286 0.5398 1) 0.4 0.51 0.539 0.5461 0.5475 0.5) 0.2)

## 16-17. The matrix of transition probabilities is:

1-month 2-months overdue overdue 0 0 0.2 0 0 0 0 0 0.1 0

## Therefore probability of placing 50 pounds is 55%. 0

Paid Bad Debt Current 1-month overdue 2-months overdue

## Bad Debt 0 1 0 0 0.15

Current 0 0 0 0 0

## 0.997 0.003 0.985 0.015 FA 0.850 0.150

This means that 99.7% of the current bills will be paid, and 0.3% of these will be bad debts. Thus, 0.997(150,000) \$149,550 will be paid, and 0.003(150,000) \$450 will eventually become a bad debt. 16-18. The matrix of transition probabilities is
Horizon Horizon Local Cellular 0.8 0.3 Local Cellular 0.2 0.7

## Hence, probability of placing 50 pounds 54.75%.

Current market shares are equal, that is, 25% each. Hence, p(0) (0.25 0.25 0.25 0.25). Therefore, p(1) p(0) P (0.1875 0.2625 0.3 0.25). Hence, market shares for the next period will be 18.75%, 26.25%, 30%, and 25% for University, Bills, College, and Battles, respectively. 16-15. solve. In the long run, p pP. We drop the equation for p2 and 0.1p3 0.05p4 0.8p4 p3 p4

0.8 0.2 Market shares next year [100, 000 80, 000] 0.3 0.7

## p1 0.6p1 p4 0.1p1 0.1p2 1 p1

Horizon will have 104,000 customers, and Local Cellular will have 76,000. 16-19. Next year, Doorway will sell 210,000, Bell will sell 220,000, and Kumpaq will sell 170,000.
0.80 0.10 0.10 [200, 000 200, 000 200, 000] 0 0.05 0.05 0.90 0.20 0.10 0.70 [210, 000 220, 000 170, 000]

## p3 0.1p1 0.2p2 0.8p3 0.1p4 p2

REVISED
M16_REND6289_10_IM_C16.QXD 5/15/08 10:54 AM Page 253

CHAPTER 16

MARKOV ANALYSIS

253

16-20.

1 0 P 0.7 0.4

0 1 0 0.2

0 0 0.2 0.2

0 0 0.1 0.2

## M (2,000, 5,000) 0.161 1.29 F 0 . 322 1 .29

above, these probabilities have been represented by the variables, A, B, C, D, E, and F. To begin with, we know that the probabilities for any row in the matrix of transition must sum to one. We also know that the original market shares multiplied times the matrix of transition must be equal to the current market share, which was given in the problem. The current market share for the three printers is 0.38, 0.42, and 0.20. With this information, we can develop the appropriate equations with the appropriate unknowns to solve for the six probabilities that we need in the matrix of transition. Doing this is shown below. We know that 0.8 A B 1 C 0.7 D 1 E F 0.6 1 or B 0.2 A D 0.3 C F 0.4 E Putting these into the matrix of transition, we get

## 0.967 0.032 FA 0.741 0.258 MFA (5,645.16, 1,354.84)

Hence \$5,645 will eventually be paid and \$1,355 will end up as bad debt. 16-21.
1 0 P 0.6 0.3 0 1 0 0.3 0 0 0.1 0.2 0 0 0.3 0.2
1

## 0.9 0.3 F (1 B)1 0.8 0.2

1.21 0.45 0.30 1.36

## 0.2 A 0.8 A [0.3 0.5 0.2] C 0 . 7 0.3 C 0.4 E 0.6 E

[0.38 0.42 0.20] Since we know that the matrix of transition probabilities times a previous market share is equal to a future market share, we can solve the problem above and get three equations and three unknowns. These equations can be solved for A, C, and E. These values can then be substituted back into previous equations to determine values for B, D, and F. This will completely specify our matrix of transition probabilities. This is shown below. 1. (0.3)(0.8) (0.5)(C) (0.2)E 0.38 2. (0.3)(A) (0.5)(0.7) (0.2)(0.4 E) 0.42 3. (0.3)(0.2 A) (0.5)(0.3 C) (0.2)(0.6) 0.2 Solving the above, we get A 0.1 C 0.2 E 0.2 From this we can compute B, D, and F: B 0.2 A 0.1 D 0.3 C 0.1 F 0.4 E 0.2 Thus the matrix of transition is:
0.8 0.1 0.1 0.2 0.7 0.1 0.2 0.2 0.6

1.21 0.45 0.6 0.0 0.864 0.136 FA 0.30 1.36 0.3 0.3 0.591 0.409

If M (50 30)
0.864 0.136 MFA (50 30) (61 19) 0.591 0.409 Hence, 61 will pass and 19 fail the course.

16-22. The Hicourt Industries problem requires some careful thought and analysis. At rst glance, it appears that there is insufcient data to solve the problem. Important matrix of transition values are seemingly missing. For this particular problem, we will assume that Hicourt Industries will be state 1, the Printing House will be state 2, and Gandy Printers will be state 3. From the problem, we know that the current market shares are 0.3, 0.5, and 0.2 for these industries. We also know the probabilities that these various states will retain their customer base. This allows us to enter some of the probability values into the matrix of transition. Hicourt has an 80% chance of keeping its customers from one month to another. In other words, the rst row and rst column of the matrix of transition is 0.8. In a similar fashion, we know that the second row and second column for the matrix of transition for the Printing House is 0.7 and that the third row and third column of the matrix of transition for Gandy Printers is 0.6. These probabilities can be entered into the matrix of transition. We will also use variables A, B, C, D, E, and F to represent unknown probabilities for the matrix of transition. This is shown below

B 0.8 A [0.3 0.5 0.2] C 0.7 D F 0.6 E Before we can go any further, we must determine a value for the six unknown probabilities in the matrix of transition. As seen

Once we have determined all of the probability values of the matrix of transition, we can determine the equilibrium market shares. This was the original objective for the problem. The equilibrium probabilities are 50% for Hicourt, 30% for Printing House, and 20% for Gandy Printers. This tells us that Hicourt will not be able to run Printing House completely out of business. In the long run, Printing House will have 30% of the market, while Hicourt will have 50% of the market. Gandy Printers will keep 20% of the market.

REVISED
M16_REND6289_10_IM_C16.QXD 5/15/08 10:54 AM Page 254

254

CHAPTER 16

MARKOV ANALYSIS

16-23. For John to get the loan that he desires, he must keep at least 35% of the market share in the long run. Currently, John has 26 condominiums, representing 50% of the market (0.50 XN\bx). Cleanco has about 28.8% of the market (0.288 ZB\bx) and Beach Services has about 21.2% of the market (0.212 ZZ\bx). In order for us to determine equilibrium market shares, we must rst know the matrix of transition probabilities. Because of a careful survey conducted by John and Bayside laundry, we know that John will keep 18 of his original customers. He will lose 6 to Cleanco and 2 to Beach Services. From these data, we can determine the rst row of the matrix of transition probabilities by dividing 18, 6, and 2 by 26. The top row of the matrix of transition probabilities will thus be 0.692, 0.231, and 0.077. We can also determine the second row of the matrix of transition probabilities, which will represent the second state or Cleanco. We already know that Bayside will get one customer from Cleanco. Furthermore, we know that Cleanco will retain 80% of its current customers. This represents 12 customers out of 15. The remaining 2 customers out of the original 15, therefore, must have switched to Beach Services. By dividing the numbers 1, 12, and 2 by 15, we know the second row of the matrix of transition probabilities. Doing this will give us 0.067, 0.80, and 0.133 for the second row of the matrix of transition probabilities. Here, we run into a stumbling block. We know that Beach Services will give up 2 of its customers to Bayside. Since Beach Services has a total of 11 customers, this represents a probability of 0.182, which is the probability in the third row and rst column of the matrix of transition probabilities. Unfortunately, we do not know the last two probabilities, which are row 3 column 2, and row 3 column 3 of the matrix of transition probabilities. What we know for the matrix of transition probabilities is

For state 3, equil. prob. (share) 0.203 Equilibrium state probabilities (market shares) X 2, Y 7 For state 1, equil. prob. (share) 0.252 For state 2, equil. prob. (share) 0.508 For state 3, equil. prob. (share) 0.239 Equilibrium state probabilities (market shares) X 1, Y 8 For state 1, equil. prob. (share) 0.267 For state 2, equil. prob. (share) 0.441 For state 3, equil. prob. (share) 0.29 Equilibrium state probabilities (market shares) X 0, Y 9 For state 1, equil. prob. (share) 0.292 For state 2, equil. prob. (share) 0.337 For state 3, equil. prob. (share) 0.37 As you can see from the analysis, the highest market share that John will achieve would be approximately 29.2% of the market in the long run. This assumes that Beach Services will retain 9 of its current 11 customers and lose none to Cleanco (X 0 and Y 9). The worst circumstances for John would be a situation where Beach Services keeps 6 of its customers and loses 3 to Cleanco. Johns market share would be 24%. This represents a situation where X 3 and Y 6. As can be seen in this analysis, there are no circumstances under which John will retain 35% or more of the market in the long run. Therefore, it is unlikely he will be able to get the loan he desires at Bay Bank. 16-24. The vector of state probabilities is (0.40, 0.60) The matrix of transition probabilities is

## 0.692 0.231 0.077 0.067 0.8 0.133 Y 0.182 X

As you can see, all of the probabilities are known but the last two in the third row. These are represented by the variables X and Y. It was also given in the problem that Beach service would keep at least 50% of its current customers. Fifty percent of the current customers would be 6 customers. Since we know that 2 customers are going to Beach Services and at least 6 customers remain with Beach Services, we know that Beach Services could end up with 6, 7, 8, or 9 customers. The remaining 3, 2, 1, or 0 customers would go to Cleanco. By dividing these values by 11, we can convert these values into four possible probability values for X and Y. We can then compute an equilibrium market share for each of the possibilities. The four probabilities for X and Y are shown below.
X 3 2 1 0 Y 6 7 8 9 P (X ) 0.273 0.182 0.091 0 P (Y ) 0.545 0.636 0.727 0.818

## 0.8 0.2 0.1 0.9

16-25. p(2) (0.37, 0.63). 16-26. p1 3 ; p2 3 . Given no changes in the Markov assumptions, store 1 will eventually end up with one-third of the customers, while store 2 will end up with two-thirds of the customers. (A) 0.8p1 0.1p2 p1 Substituting (1 p2) for p1, we have (B) 0.8(1 p2) 0.1p2 1 p2 or 0.8 0.8p2 0.1p2 1 p2 Solving for p2 yields
1 2

2 2 3
Substituting ( 2 2 3 ) in (A) and solving for p1 gives

From the probability values above, we can determine the equilibrium market share for each possibility. The results are shown below. Equilibrium state probabilities (market shares) X 3, Y 6 For state 1, equil. prob. (share) 0.24 For state 2, equil. prob. (share) 0.555

1 1 3
16-27. This problem can be solved using QM for Windows. The results are presented below. As you can see from the ending probabilities, the market shares change for the next period. The market shares for University, Bills, College, and Battles stores are 27%, 25%, 26%, and 22%. The steady-state market shares do not change, as you would expect.

REVISED
M16_REND6289_10_IM_C16.QXD 5/15/08 10:54 AM Page 255

CHAPTER 16

MARKOV ANALYSIS

255

Data Initial State State State State 1 2 3 4 0.4 0.2 0.2 0.2 State 1 0.6 0 0.1 0.05 0.27 0.129 State 2 0.2 0.7 0.1 0.05 0.25 0.2581 State 3 0.1 0.2 0.8 0.1 0.26 0.4194 State 4 0.1 0.1 0 0.8 0.22 0.1936

## Ending probability Steady State probability

16-28. a. By using the Markov process, Sandy can determine market shares for each of the quick-oil-change operations for next period. The results are summarized below.
State 1 2 3 4 5 6 7 8 9 10 Value 0.36 0.14 0.13 0.16 0.07 0.04 0.02 0.02 0.02 0.02

c. The market shares for next period will change, but the equilibrium shares will remain the same. This is shown below:
State 1 2 3 4 5 6 7 8 9 10 Value 0.25 0.28 0.11 0.14 0.07 0.06 0.02 0.02 0.02 0.03

## b. The equilibrium market shares for this problem are as follows:

State 1 2 3 4 5 6 7 8 9 10 Probability 0.02439 0.05807 0.03934 0.15363 0.05226 0.41723 0.05779 0.08832 0.05228 0.05669 Value 0.02 0.06 0.04 0.15 0.05 0.42 0.06 0.09 0.05 0.06

STEADY STATE State 1 2 3 4 5 6 7 8 9 10 Probability 0.02439 0.05807 0.03934 0.15363 0.05226 0.41723 0.05779 0.08832 0.05228 0.05669 Value 0.02 0.06 0.04 0.15 0.05 0.42 0.06 0.09 0.05 0.06

REVISED
M16_REND6289_10_IM_C16.QXD 5/15/08 10:54 AM Page 256

256

CHAPTER 16

MARKOV ANALYSIS

d. The equilibrium shares for this situation are given below. As you can see, shop 1 will not have 99% of the market in the long run.
STEADY STATE State 1 2 3 4 5 6 7 8 9 10 Probability 0.50000 0.04762 0.01613 0.06494 0.02381 0.22112 0.02878 0.04333 0.02564 0.02864 Value 0.50 0.05 0.02 0.06 0.02 0.22 0.03 0.04 0.03 0.03

b. We make the same types of changes for Chris that we did for Sandy. The seventh row for the matrix of transition (representing shop 7) will have 0.99 in the rst column and zeros elsewhere except for shop 1, which will have a 0.01. The solution is given below. As with part (a), Chris has also increased his market share, but it is not 99%. He does not win his bet.

States 1 2 3 4 5 6 7 8 9 10

Probability 0.02439 0.02763 0.01872 0.07060 0.02182 0.13449 0.63934 0.03019 0.01342 0.01941

Value 0.02 0.03 0.02 0.07 0.02 0.13 0.64 0.03 0.01 0.02

16-29. a. This is a typical Markov problem. We assume the same data as presented in Problem 16-28 with the exception that the rst row for the matrix of transition (representing shop 1) will have 0.99 in the rst column and 0s elsewhere except for shop 7, which will have a 0.01. The solution is given below. While Sandy has increased her market share, it is not 99%. She does not win her bet.

States 1 2 3 4 5 6 7 8 9 10

Probability 0.50000 0.02381 0.01613 0.06494 0.02381 0.22112 0.04519 0.04546 0.03039 0.02916

Value 0.50 0.02 0.02 0.06 0.02 0.22 0.05 0.05 0.03 0.03

c. If both are correct, the market shares will be 50% each as seen in the following solution. Perhaps they can get together, each paying for his or her own meal. Neither Sandy or Chris will end up with 99% of the market.

States 1 2 3 4 5 6 7 8 9 10

Probability 0.50000 0.00000 0.00000 0.00000 0.00000 0.00000 0.50000 0.00000 0.00000 0.00000

Value 0.50 0.00 0.00 0.00 0.00 0.00 0.50 0.00 0.00 0.00

REVISED
M16_REND6289_10_IM_C16.QXD 5/15/08 10:54 AM Page 257

CHAPTER 16

MARKOV ANALYSIS

257

16-30. Changing the rst row will have an impact on the steadystate market shares. To see the impact, we can use QM for Windows to solve this problem. The results are presented below.
Data Initial State State State State State State State State State State 1 2 3 4 5 6 7 8 9 10 0.6 0.1 0.1 0.1 0.05 0.01 0.01 0.01 0.01 0.01 State 6 State State State State State State State State State State 1 2 3 4 5 6 7 8 9 10 0.03 0.1 0.1 0.01 0.01 0.91 0.1 0.1 0.1 0.1 State 1 0.0357 State 6 Steady State probability 0.4231 State 1 0.73 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 State 7 0.03 0.01 0.01 0.01 0.03 0.01 0.7 0.03 0.01 0.1 State 2 0.03 0.8 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 State 8 0.03 0.01 0.05 0.01 0.01 0.01 0.01 0.8 0.1 0.05 State 2 0.0510 State 7 0.0606 State 3 0.03 0.01 0.7 0.01 0.01 0.01 0.01 0.01 0.01 0.01 State 9 0.03 0.01 0.05 0.01 0.01 0.01 0.1 0.01 0.7 0 State 3 0.0346 State 8 0.0921 State 4 0.03 0.01 0.01 0.9 0.1 0.01 0.01 0.01 0.01 0.01 State 10 0.03 0.03 0.05 0.02 0.01 0.01 0.04 0.01 0.04 0.70 State 4 0.1391 State 9 0.0548 State 5 0.0510 State 10 0.0580 State 5 0.03 0.01 0.01 0.01 0.8 0.01 0.01 0.01 0.01 0.01

REVISED
M16_REND6289_10_IM_C16.QXD 5/15/08 10:54 AM Page 258

258

CHAPTER 16

MARKOV ANALYSIS

## SOLUTIONS TO INTERNET HOMEWORK PROBLEMS

16-31. Let states 1, 2, 3 represent fair, tolerable, and miserable trafc conditions, respectively:
0.7 0.2 0.1 P 0.2 0.75 0.05 0.3 0.1 0.6

In the long run there will be 333 minnows in Lake Jackson and 667 in Lake Bradford. 16-34. a. Original state (900 100) p(0) b. Let P probability of going from Lake Jackson to Lake Bradford. c.
1 P P P 0.05 0.95

## 1) 0.1 0.6) 0.395) 0.195

d. At equilibrium, p pP

p(2) (0.41

After two hours, trafc conditions will be fair with a probability of 41% and tolerable with a probability of 19.5%. 16-32. From Problem 16-31 we get the transition matrix P. In the long run, p pP: p1 0.7p1 0.2p20 0.3p3 p2 0.2p1 0.75p2 0.1p3 1 0.2p1 0.25p2 0.5p3 which reduces to 3p1 2p2 3p3 0 4p1 5p2 2p3 0 4p1 4p2 4p3 1 which has a solution of p1 Z>\vv, p2 Z<\vv, p3 M\vv. Therefore, in the long run, trafc conditions will be fair 43.18%, tolerable 40.91%, and miserable 15.9% of the time. 16-33. Let existence in Lake Jackson denote state 1 and existence in Lake Bradford denote state 2 for an individual tiger minnow. Then:
0.9 0.1 P 0.05 0.95

## 1 P P (900 100) (900 100) 0.05 0.95

which gives us 900 (900)(1 P) (100)(0.05) 900 900 900P 5 5 P 0.005556 900 Here is another way of solving the problem: 100 (900)(P) (100)(0.95) 100 900P 95 5 P 0.005556 900 e. Thus the probability of going from Lake Jackson to Lake Bradford must be 0.005556 to restore the original situation.

## SOLUTION TO RENTALL TRUCKS CASE

1. To determine what the market shares will be in one month, Markov analysis must be used. We will dene the following states for this case: State 1: State 2: State 3: Rentall Rentran National Total Therefore, n(0) (0.8 0.05 0.15) The next step is to determine the matrix of transition. If no changes are made, one has: Rentall Rentran National 80% 800 5% 50 115% 1,150 100% 1,000 people

(0) (900 100) In the long run, p pP 0.1p1 0.9p1 0.05p2 0.1p1 0.05p2 0.1p2 2p1 Also: p1 p2 1,000 p1 333, p2 667 or

## National 80/800 0.10 6/60 0.10 91/140 0.65

REVISED
M16_REND6289_10_IM_C16.QXD 5/15/08 10:54 AM Page 259

CHAPTER 16

MARKOV ANALYSIS

259

Thus

## 0.65 0.25 0.1 P 0.05 0.85 0.1 0.1 0.25 0.65

To determine the share of the market for next month, we multiply the market shares times the matrix of transition. n(1) n(0)P

0.65 0.25 0.1 (0.8 0.05 0.15) 0.05 0.85 0.1 5 0.65 0.1 0.25
(0.5375 0.2800 0.1825) Thus the new share of the market is about Rentall Rentran National 54% 28% 18% 100%

Customers Rentall Rentran National 800 60 140 Rentall 680/800 0.85 9/60 0.15 28/140 0.20 Rentran 100/800 0.125 45/60 0.750 35/140 0.250 National 20/800 0.025 6/60 0.100 77/140 0.550

## 0.85 0.125 0.025 P 0.15 0.75 0.1 0.2 0.25 0.55

n(1) n(0)P
0.85 0.125 0.025 (0.8 0.05 0.15) 0.15 0.75 0.10 0.2 0.25 0.55

0.175

0.107)

## 2. With no changes, in two months the market shares will be

0.65 0.25 0.1 n(2) (0.5375 0.2800 0.1825) 0.05 0.85 0.1 0.1 0.25 0.65
(0.3816 0.4180 0.2004) In three months, the market share will be

0.65 0.25 0.1 n(3) (0.3816 0.4180 0.2004) 0.05 0.85 0.1 0.1 0.25 0.65
(0.2890 0.5008 0.2102)

REVISED
M16_REND6289_10_IM_C16.QXD 5/15/08 10:54 AM Page 260

260

CHAPTER 16

MARKOV ANALYSIS

The approximate market shares after three months are listed below. Rentall Rentran National Total 29% 50% 121% 100%

With no changes,

0.65 0.25 0.1 (n1 n2 n3 ) (n1 n2 n3 ) 0.05 0.85 0.1 0.1 0.25 0.65 n1 0.65n1 0.05n2 0.1n3
n2 0.25n1 0.85n2 0.25n3 n3 0.1n1 0.1n2 0.65n3 Solving as above, the results are n1 0.153 n2 0.625 n3 0.222 1.000 The equilibrium market shares would be Rentall Rentran National Total 15% 63% 122% 100%

## If the changes are made, n(2) n(1)P

0.85 0.125 0.025 (0.718 0.175 0.107) 0.15 0.75 0.10 0.2 0.25 0.55
(0.658 n(3) n(2)P
0.85 0.125 0.025 (0.658 0.248 0.094) 0.15 0.75 0.10 0.2 0.25 0.55

0.248

0.094)

0.292

0.093)

## SOLUTIONS TO INTERNET CASES

University of TexasAustin
1. The transition probabilities of Figure 1 lead to the following Markov transition matrix, M:
To From S1 S2F S3 S4 S20 S5 S1 .25 0 .25 0 0 0 S2F .27 0 .13 0 0 0 S3 .48 0 .62 0 0 0 S4 0 .01 0 .26 .12 0 S20 0 .99 0 .74 .70 0 S5 0 0 0 0 .18 1.

## 3. At equilibrium, with these changes,

0.85 0.125 0.025 n3 ) 0.15 0.75 0.10 0.2 0.25 0.55 n1 0.85n1 0.15n2 0.2n3 (n1 n2 n3 ) (n1 n2

n2 0.125n1 0.75n2 0.25n3 n3 0.025n1 0.1n2 0.55n3 (1) 0.15n1 0.15n2 0.2n3 0 (2) 0.125n1 0.25n2 0.25n3 0 (3) 0.025n1 0.1n2 0.45n3 0 (4) n1 n2 n3 1 We have four equations and three unknowns. We drop equation 1 and solve equations 2, 3, and 4 for n1, n2, and n3. The results are n1 0.519 n2 0.370 n3 0.111 1.000 The equilibrium market shares are Rentall Rentran National Total 52% 37% 111% 100%

A student beginning the program corresponds to the vector v (1, 0, 0, 0, 0, 0). After six semesters the probability distribution for the student is: (.0850, .0586, .1965, .0589, .3668, .2341). The last state, S5, is graduation which is an absorbing state; 23.41 percent of the students graduate in or prior to the end of the sixth semester. 2. The balance assumption replaces the last row of the M matrix with S5 1 0 0 0 0 0. The resulting steady state probability vector is (.1683, .0731, .2126, .0668, .4060, .0731) so that 16.8 percent are in state S1, 7.3 percent are in state S2F, 21.3 percent are in state S3, 6.7 percent are in state S4, 40.6 percent are in state S20, and 7.3 percent are in state S5. The mean recurrence times for the graduation state S5 is 1/0.0731 13.7 semesters. 3. The Markov assumption states that the chance of going from one state to another depends only upon the state one is in and not on how the individual reached that state. Thus, a student who has

REVISED
M16_REND6289_10_IM_C16.QXD 5/15/08 10:54 AM Page 261

CHAPTER 16

MARKOV ANALYSIS

261

withdrawn after advancement (state S4) and has remained in that state for a number of semesters is just as likely to re-enroll in the program as a student who has just withdrawn, if the Markov assumption is true. In practice, one would have to obtain data to support this assumption. Along the same line, the graduation rate from state S20 is given as 0.18, irrespective of whether the candidate has been in state S20 one or many semesters. This may not be a reasonable assumption. The states of the system do not allow a student to exit the system other than by graduating. The creation of an additional state leave before graduation might be a mechanism to take this into account.

Let state 1 in operation Let state 2 out of operation From Markov analysis we dene the steady state probability as:
p (1 on day m given 1 on day 1) p(2) 1 p(1) p(2)

## For the AKZ centrifuge: To From 1 2 1 .96 .20 2 .04 .80

For the CM centrifuge: To From 1 2 1 .90 .85 2 .10 .15

## St. Pierre Salt Company

The case illustrates the use of Markov analysis. Assumptions for a Markov process are: a) The probability of going to each state depends only on the current state and not on the manner in which the current state was reached. b) Initial conditions take on less and less importance as the process operates. c) There is a discrete number of possible states and potential changes occur only once every time period.