SS 2020/2021 st.

year

3d TEST

1. In each step someone randomly selects precisely one number from sequence {1; 2; 3;
4}. The chosen number can be selected again in any of the subsequent steps. The
current state of the Markov chain is the maximum of all the numbers selected up to and
including the current step.
1. Write the initial distribution and the transition probability matrix for the Markov chain;
2. Draw the graph of the Markov chain;
3. Calculate the probability that after 2 steps the system will be in a state with a number
greater than 2.

2. A Continuous-Time Markov Chain Consisting of Two States. The process of

computer operation is considered. The flow of failures of a working computer is the
simplest Poisson process with parameter λ. If the computer fails, it is immediately
detected, and the maintenance personnel proceeds to eliminate the malfunction (repair).
The repair time is exponentially distributed with the parameter µ. At the initial moment
(t = 0), the computer is operational.

Find: 1) the probability that at time t the computer will work.

2) the probability that during the time (0, t) the computer will give at least one failure.
3) the final probabilities of the computer states.

HINT: write down the Kolmogorov differential equations for marginal probabilities.

3. For each graph of the given CTMC:

• Write the Kolmogorov system of the differential equations for marginal

probabilities with λ0 = λ1 = λ2 = λ4 = λ5 = 2 and λ3=1 .
• Write the intensity matrix.
• Find the final marginal probabilities.