Home Work 3:

Question # 1: Ehrenfest model:

Part a:
The transition matrix is given as:

Part b:
Starting from any state we can see that we can reach any other state. However, we can’t
do so in a single step for all cases therefore the Markov Chain is Ergodic but not Regular.

Part c:
Product sum of products of any of the numbers in a row and column does not exceed 1.
Therefore, Higher order of the matrix will not exceed 1 and will grow linearly instead of growing
exponentially.
Question # 2:
Part a:

Part b:
Transition matrix is given as:

Part c:
Reduced Echelon form of the transition matrix is given as:
Part D
The matrix F=(In−B)−1 is called the fundamental matrix for the absorbing Markov chain, where In is an
identity matrix of the same size as B. The i, j-th entry of this matrix tells us the average number of times
the process is in the non-absorbing state j before absorption if it started in the non-absorbing state

Question # 3: Professor’s Umbrella:

First we identify the chain with (2n+2) states

For it [0, n], let x(2i+1) describe the state that the professor is at home.

Let “i” be the umbrella at home, let n(i) be the umbrella at the work.

Let x(2i+2) denote the state that professor is at work.

The transition matrix can be described as:

 To find stationary distribution we need:

Since the professor gets wet when it rains, that means that the chain moves from x1 to x2.

So we can say that the probability of professor getting wet is:

Question # 4: Travelling Mathematician:

Part a:
Transition matrix for the Markov chain is given as:
The chain is Ergodic because it is possible to reach all states of the Markov chain from any state. It is
however not regular because it does have zeros in the transition matrix.

Part (b):
As time approaches infinity the probabilities for being in a state will be highest for state 2 followed by
equal probabilities for 3 and 4 and the lowest probability for state 1.

Question # 5: Given Transition matrix find out if the Markov Chain is

regular:
For stationary distribution, the equations are:

So by solving for these equations we have:

By solving we see that:

Hence, the stationary distribution is:

Question # 6: Fate of the spider:

If n = 6 and s = 12 while n = 0 means death the probability of the survival of the spider is
approximately 0.56, while probability of death is approximately 0.44.

If we run the simulation for 1000 iterations, we get 562 escapes and 438 deaths. The higher the
number of iterations the more accurate the value.

Code:

First we initialize the variables and import libraries:

step = 0.1

sim_no = {}

escape = 12

die = 0

pos = 6

deaths = 0

escapes = 0

no_of_walks = int(input("Enter number of simulations to run: "))

import random

import matplotlib.pylab as plt

Then we define a function that simulates a random walk based on the probabilities provided.

def fate(pos, deaths, escapes):

pos = 6

while pos != (0 or 12):

if random.random() < 0.51:

pos += 1

else:

pos -= 1

if pos == 0:

deaths += 1

return deaths, escapes

if pos == 12:

escapes += 1

return deaths, escapes

#print(deaths, escapes)

for i in range(no_of_walks):

deaths, escapes = fate(pos, deaths, escapes)

print("Deaths: ",deaths)

print("Escapes: ",escapes)

Please refer to the python file for the simulations.

Question # 7: Random Walks:
Part a:
Random walk with 0.5 probability:
Initializing the variables:

step = 0.1

time_step = 0.1

sim_no = {}

M = int(input("Enter how long to run each walk in seconds: "))

no_of_steps = M*10

no_of_walks = int(input("Enter number of simulations to run: "))

Then we perform random walk with 0.5 probability, using random function as:

for i in range(no_of_walks):

time_pos = {}

position = 0

time = 0

for j in range(no_of_steps):

time += time_step

if random.random() < 0.5:

position -= step

else:

position += step

time_pos[round(time, 1)] = round(position, 1)

sim_no[i] = time_pos

We plot the histogram which shows that most of the walks terminate around zero because probabilities
to move in either direction are equal.

lis_100 = []

for i in range(no_of_walks):
 lis_100.append(sim_no[i][10.0])

plt.hist(lis_100, bins =10)

plt.show()

Histogram is shown as:

Please refer to the python file for the simulations and graphics.

Part b:
Similarly, by changing probabilities we show that most of the walks will terminate between 3 and 4.
Similarly, the histogram also shows distortion towards higher values:

Please refer to the python file for the simulations and graphics.

Part c:
The difference between the two cases is that in first case the probabilities for moving left and right are
equal. And in the second case the value for moving to the right is higher which causes the last state to
gradually move to a higher number.