Markov Chain Analysis

Presented By:Aditi Misra

Markov Chains

A Markov Chain is a stochastic process . It has following Properties:3. Discrete state space, 4. Markovian property, 5. One step stationary transition proba

Stochastic Process
A stochastic process is defined by the family or set of random variables{Xt}, where t is a parameter(index) from a given set T.

Discrete State Space
A state space(S) is the sample space for all possible values of Xt. When a state space contain discrete values, it is called as discrete state space. If the discrete state space has a finite number of states, then a finite-state Markov chain has

Markovian Property
The probability of a state on the next trial depends only on the state in the current trial and not on the states prior to the present.

Transition probability
§ A transition probability is defined as the conditional probability that the process will be in a specific future state given its most recent state. § This probabilities are also called one-step transition probabilities, since they describe the system between t and t+1. § It may be presented in the tabular

Lets consider an example:-

Brand Switching Example:Suppose that there are two brands of detergents D1 and D2, selling in the market in the beginning of a year, when a third one, D3, is introduced into the market. The market is then observed continuously month-after-month for change in the brand loyalty. Let us say that the rate of brand switching has settled over time as follows:

Example continues……
Brand this month 60% next month 30% D1
10% D2 15% 5% 80% 50% 30% D3 D2

20% D1


Now, given these conditions about brand switching, assuming no further entry or exit, and given further that the market share for the brands on a certain date, say march 1, is 30%, 45% and 25% for brands D1, D2 and D3

Markov Analysis: Input & Output

Markov analysis provides for the following predictions: The probability of the system being in a given state at a given future time. The steady state probabilities.

 Transition


Transition probabilities for this problem:-

n It must satisfy the following 2 properties:ij 0<p<1 for all i,j J= 1


initial conditions:The initial conditions describe the situation the system presently is in. Here, the initial condition isThe market share for the brands is 30%, 45% and 25% for brands D1, D2 and D3 respectively. In a row matrix[0.30 0.45 0.25]


Probabilities:It is for calculating the probabilities for the system in specific states. The probability distribution of the system being in a certain state(i) in a certain period(k) may be expressed as a row matrix: Q(k) = [q1(k) q2(k) q3(k)……….. qn(k)]

For this example
Q(0)=[q (0) q (0) q (0)]=[0.30 D D D 1 2 0.45 0.25] 3 For calculating market share for the next month: Q(next)=Q(current)xP

To calculate the probability of a customer to buy D3 two months hence, given that his latest purchase has been D2
0 0 2 D2 to D3 D2 to .D2 D2 0 0 to.50 .30 0 D3 .3 D3 to D3 0
D2 to D1

To calculate the probability of a customer to buy a

D1 to D3 t=1 0 .10 t=2 Probability

0.20 x 0.10 = 0.02 0.50 x 0.30 = 0.15

0 .80

0.30 x 0.80 = 0.24 Total 0.41

Steady state probability:
A stablised system is said to be in a steady state or in equilibrium. Q(k) = Q(k-1)