State And Transition Probabilities

To predict the movement of the system from one state

to the next state, it is necessary to know the condition
or transitional probabilities of such a movement.
These probabilities can be represented as an element
of a square matrix (also known as a matrix of transition
probabilities) or by a transition diagram.
The matrix of transition probabilities enables us to
predict the future state (or condition) of any system
under study.

 
 Cont…
The states of the system and transition probabilities
can be represented by two types of diagrams.
Transition diagram
The transition probabilities can be represented by a
transition diagram as shown below, arrows from
each state indicate the possible transitions to a start
and their corresponding probabilities.
 Cont….

 Sum of a particular row is equal to one:

0.2+0.8=1 for the first row.
0.6+0.4=1 for the second row.
 Cont….
Probability tree diagram
These diagrams are used to illustrate only a
limited number of transitions of the Markov
chain. The number in the circle represents the
states at the beginning of the transition.
 Cont….
Let the probabilities of shifting from S1to S1it self
and S2 as well as from state S2 to S1 and S­2 itself
be represented as elements of the transition
matrix of the Markov chain, as follow:

P=

 
Here, p11+ p12=1 and p21 + p22=1
 
Multi-Period Transition
Probabilities
 One of the objectives to study Markov analysis is
to predict the future state of any system under
study.
 Given the set of state probabilities,
R0=[P11,P12…..,P1m] and the matrix of transition
probabilities, at time period n=0, we can
determine the state probabilities at the next time
period. For convenience, let R1 represent the
state probabilities at the time period(or state),
n=1.
 Cont.….
After one execution of the experiment, it can be written
in terms of row matrix as:
R1=R0*P
To compute the state probabilities at time
period(state)2,3,…..n, multiply the state of the system at
time,n=0 with transition matrix(p),that is:
R2=R1*P =R0*P2
.
.

.
Rn=Rn-1*pn=R0*Pn
 Cont….
The elements of the n-step transition
matrix=[Pijn]m*m are obtained by repeatedly
multiplying the transition matrix by itself.
In general, we have
Pn=Pn-1*P
Where each row i of Pn represent the state
probability distribution after n transition, given
that the process starts out in state i.
Procedure to Formulate Matrix of
Transition Probabilities
• The step of constructing a matrix of transition
probabilities may be summarized as follow:
 Step1:calculate retention probabilities.
 Step2:determine gains and losses probabilities.
 Step3:Develop matrix of transition probabilities: