# Business Analytics 1

Markov Model September 25

Andrey Andreyevich Markov

Learning Objectives

1. Give examples of systems that may lend themselves to be

**analyzed by a Markov model.
**

2. Explain the meaning of transition probabilities. 3. Describe the kinds of system behaviors that Markov analysis

pertains to.

4. Use a tree diagram to analyze system behavior. 5. Use matrix multiplication to analyze system behavior. 6. Use an algebraic method to solve for steady-state

probabilities.

Continue…

7. Analyze absorbing states, namely accounts receivable, using

a Markov model. 8. List the assumptions of a Markov model. 9. Use Excel to solve various problems pertaining to a Markov model.

**Characteristic of Markov System
**

1. It will operate or exist for a number of periods. 2. In each period, the system can assume one of a number of

states or conditions. 3. The states are both mutually exclusive and collectively exhaustive. 4. System changes between states from period to period can be described by transition probabilities, which remain constant. 5. The probability of the system being in a given state in a particular period depends only on its state in the preceding period and the transition probabilities. It is independent of all earlier periods.

Assumptions

Markov Analysis Assumptions

The probability that an item in the system either will change from one state (e.g., Airport A) to another or remain in its current state is a function of the transition probabilities only. The transition probabilities remain constant. The system is a closed one; there will be no arrivals to the system or exits from the system

Examples of Markov System

a car rental agency that has rental offices at each of a city’s airport. Customer are allowed to return car to either airport, regardless of which airport they rented from it. (It is assumed that this is a closed system: all cars must be returned to one of the airport.) If some cars will be used for one-way rentals to another city or if some cars are dropped off from other cities, they can be excluded from the system and treated separately. In addition , it is assumed that all the rentals will be for only one day, and that are at the end of the day, everycar will be returned to one of the two rental offices. Suppose the manager of the rental agency has made

A study of return behaviour and has found following information: 70% of the cars rented from Airport A tend to be returned to that airport, and 30% of the Airport A car tend to be returned to Airport B; 10% of the cars rented from Airport B are returned to Airport A and 90% to Airport B. The manager has following question to solve. Preparing transition matrix What proportion of cars will be returned to each airport in the short run? This information will help manager in scheduling help counter at each location?

What proportion of cars will be returned to each location over the long run? This information will be helpful in deciding which of the two locations should be chosen for construction of a facility for servicing and repair of cars.

Transition probability of Car Rental Example

System Behavior

Both the long-term behavior and the short-term behavior of a system are completely determined by the system’s transition probabilities. Short-term behavior is solely dependent on the system’s state in the current period and the transition probabilities. The long-run proportions are referred to as the steadystate proportions, or probabilities, of the system.

Expected Proportion of period 0f Rentals Returned to Airport A

**Methods of System Behavior Analysis
**

Tree Diagram

A visual portrayal of a system’s transitions composed of a series of branches, which represent the possible choices at each stage (period) and the conditional probabilities of each choice being selected.

Matrix Multiplication

Assumes that “current” state proportions are equal to the product of the proportions in the preceding period multiplied by the matrix of transition probabilities. Involves the multiplication of the “current” proportions, which is referred to as a probability vector, by the transition matrix.

Continue..

Algebraic Solution

The basis for an algebraic solution is a set of equations developed from the transition matrix. Because the states are mutually exclusive and collectively exhaustive, the sum of the state probabilities must be 1.00, and another equation can be developed from this requirement. The result is a set of equations that can be used to solve for the steady-state probabilities.

Tree diagram for Car Rental Example

Two Periods Tree Diagrams for car rental example

Period-by-Period Proportions for the Rental Example, and the Steady-State Proportions Based on Matrix Multiplications

Development of Algebraic Equation

Transition Probabilities for the Machine Maintenance Example

Decision Tree Representation of the Machine Maintenance Problem: Initial State = Operation

Decision Tree Representation of the Machine Maintenance Problem Initial State = Broken

Solver Pattern

Quiz–

Matrix Multiplication

To obtain expected state of proportion over the short time. It is based on state proportion for any period being equal to the product of the proportion in the preceding period multiplied by the matrix of transition probabilities. The proportion can be expressed through matrix form, this approach involves the multiplication of the “current proportions, which is referred to as probability vector, by transition matrix. The vector has one row and the same number of columns as the transition matrix. For instance, the intitial “ current” vector for a case with two possible states of nature, and starting from state A, would be (1,0). For the car rentals, this would mean all cars are at Airport A at the start.

Continue…

Starting in period 0 from state B would be represented by (0,1). For the car rentals, this would mean that all cars are at Airport B at the start. If there are three states of nature (e.g. C is added) period 0 would be expressed in the following way, depending on the starting state:

Starting From A B C ( ( ( Current Matric 1 0 0 0 1 0 0 0 1 ) ) )

Matrix multiplication requires that the elements of the row of the first (current) matrix be matched with (multiplied by) the elements of the first column of the transition matrix and summed to obtain the second state proportion, and so on. The number of column for the first matrix must be equal to the number of rows in the second matrix. The multiplication of two matrices, where the first matrix is of the order K*M and the second matrix is of the order M*N. Equation…

Transition Matrix Q2

Example 2 (a)

Construct two tree diagrams for the transition matrix , each showing two periods. Have one tree diagrams show a starting state of X and the other diagram begin with a starting state of Y. In both cases, use the tree diagram to determine the probability that the system will be in state X, state Y, or state Z given that each tree starts in period 0.

Tree Diagram for Example

Tree Diagram for Q3, Starting from Y (Initial State =Y)

Problem Set (b)

Use Matrix multiplication to determine the probability that the system described in problem 2 will be in each of the various possible states (X,Y and Z) for periods 0,1,2 and 3. Given that the initial state is X in period 0.

Problem 2 (c)

Determine the long run proportions (steady state probability) for each state. If there are 900 members in the system, with 400 in state X in period 0, 300 in state Y, and 200 in state Z. determine the expected number in each state in the long run.

**Cyclical, Transient, and Absorbing Systems
**

Cyclical system

A system that has a tendency to move from state to state in a definite pattern or cycle.

Transient system

A system in which there is at least one state—the transient state—where once a system leaves it, the system will never return to it.

Absorbing system

A system that gravitates to one or more states—once a member of a system enters an absorbing state, it becomes trapped and can never exit that state

An Example of a Cyclical System

An Example of System with a Transient State

An Example of a System with Absorbing States

Probability Transition Diagrams for the Transition Matrices