171 views

Uploaded by Chandrashekhar Choudhari

good set of questions on markov chain

- lecture-15-dynamic-programming.pdf
- Wagner Whitin
- Dynamic Programming[2003]
- Markov Chain
- Markov Chains
- Dynamic Programming
- Markov Chain
- Markov Chain Model
- markov_chains_4
- Integer Programming
- IP
- Dynamic Programming- Mirage Group
- MarkovChains
- continuous markov chain
- Dynamic Programming
- markov chain chapter 6
- GetPDFServlet
- Dynamic Programming
- Markov Chain
- Chapter 4 Network Planning 1

You are on page 1of 5

Following questions are for group project in IE 611. You can form a group of one, two or

three students for this Project.

Q1. Snakes and ladders game:

The game of snakes and ladders consists of a board having 100 squares labeled 1, 2, 3..100.

on board, there are ten snakes: (27,10), (55,16), (61,14), (69,50), (79,5), (81,44), (87,31),

(91,25), (95,49), and (97,59). There are also ten ladders: (6,23), (8,30), (13,47), (20,39),

(33,70), (37,75), (41,62), (57,83), (66,89), and (77,96). The player starts from square 1. In

his/her turn, he/she rolls a die. He/she is on square i and the number turned up is d, he/she

moves to square i+d. however, if square i+d =u is start of a snake or ladder (u,v), then he/she

has to move immediately to square v.

a) Formulate the game as a Markov chain. Write transition probability matrix.

b) How long does it typically take a single player to finish a game by reaching square 100?

c) For simplicity, assume that the board has only 20 squares labeled from 1 to 20.

i) Without any ladder or snake, find the probability that square i is occupied during a

game starting from square 1.

ii) Without any ladder or snake, what is expected number of tosses of a die a solo

player has to make before the game end?

iii) With ladder (3,10), what is expected number of tosses of a die a solo player has to

make before the game end?

[Adapted from: D.L. Minh. Applied probability models". Duxbury 2002.]

Q2. Buffer design of a communication channel:

Consider a communication system with a finite-capacity buffer and multiple transmission

channels. Messages arrive at the buffer according to a Poisson process with rate . The

messages are first stored in the buffer which has capacity for only N messages (excluding the

messages in transmission). Each message which finds, upon arrival, that the buffer is full is lost

and does not influence the system. At fixed clock times t=0,1,..messages are taken out from

the buffer and are synchronously transmitted. Each transmission channel can transmit only one

message at a time. The transmission is constant and equal equals one time unit for each

massage. There are c transmission channels and so at most c messages can synchronously be

transmitted. The transmission of a message can only start at a clock time so that a message

which finds, upon arrival, that a transmission line is idle has to wait until a subsequent clock

time

a) Model the system as a Markov chain.

b) How should we choose the buffer size so that the long-run fraction of messages lost does not

exceed a prespecified value?

[Adapted from: Henk C. Tijms. "Stochastic Models an algorithmic approch". John

wiley&sons.]

Q3) Consider an algorithm for generating a random spanning tree of an undirected, d-regular

connected graph G. The algorithm is as follows:

a) Choose a starting vertex uniformly at random

b) Follow the standard random walk over vertices

c) The first time a vertex is entered by the random walk, add the edge used to enter it to the

spanning tree (the starting vertex is \entered" at time 0 so receives no entering edge)

d) Stop when all vertices have been entered at least once

To analyze the algorithm, we need to examine a Markov chain that, on the surface, seems

unrelated. The Markov Chain is on rooted spanning trees of G, in other words, spanning trees

with a specific vertex selected as the root. Once a root r is selected, each vertex v r in the tree

has a unique parent: the first vertex on the (unique) path from v to r. Consider the following

random transition rule from tree T with root r:

1) Choose a random neighbor v of r to be the new root

2) Delete the parent edge of v from T

3) Add the edge (r; v) to the T (this is now r's parent edge)

Note that as the rooted-tree markov chain evolves, the root vertex is tracing out a standard

random walk on G.

(a) Prove that (once we add self loops) this Markov Chain has a unique stationary distribution.

(b) Prove that the stationary distribution is uniform over rooted spanning trees. Conclude that if

we ignore the root, then the tree of the stationary distribution is a (uniformly) random spanning

tree.

(c) Now we connect this Markov chain to the tree generation algorithm described initially.

Suppose that the markov chain is initiated in its stationary distribution infinitely far in the past,

and consider its state at time 0. Argue that to know the rooted tree at time 0, it is sufficient to

know, for each vertex v, the identity of the last vertex to replace v as root in the Markov chain

before time 0. In other words, that the state at time 0 is defined by the sequence of roots r......... r-1, r0 encountered before time 0, and that we only need a suffix of this sequence that

includes every vertex at least once.

Q4) In an urn experiment where P molecules are distributed in two containers A and B. At

each trial a molecule is chosen at random and moved from its container to another container.

Suppose there are exactly k molecules in the container A. Then the probability that the

k

Pk

molecule is chosen from urn A is P and that to from urn B is

P

A) Construct a markov chain and find its transition probability matrix

B) Let there initially be j molecule in the first container and let X(n) = 2k-a if at the nth step the

number of molecules in container A are k (so that X(n) is the difference of the number of

molecules in the 2 containers) let en = E(X(n)), prove that en+1 = [(a-2) en]/a, where en = (12/a)n(2j-a)

Q5) Neal's token instrument:

In single line operations on Indian Railways, some trains still run by the physical token system,

called the Neal's token instrument, as follows. For using the track between station A and B, let

trains going from A to B be called "down" trains and trains from B to A be called "up" trains.

Some number of metal balls called tokens (say K in number) are there which can be deposited

in an interlocked set of instruments at A and B. If a train arrives at A and there is a token

available in the instrument, it requests it. Till this token is physically deposited in the

instrument at B, no other train (up or down) can enter the A-B section. If trains requesting the

use of section A-B are either up or down trains with equal probability, what is the number of

tokens required so that trains do not have to wait for want of a token (track otherwise being

free)? You would have to make some assumptions about the arrival process, for example, you

can assume Poisson arrivals with the same rate (l) in each direction. In the Poisson case, you

may not be able to ensure that trains NEVER wait. See what you can say about this. Also try

some other sensible arrival distributions.

For photos of the Neal's token instrument on the Kalka Simla line, pl see

http://www.tribuneindia.com/2003/20031108/windows/main1.htm

Q6). Out of the d doors of my house, suppose that in the beginning k> 0 are unlocked and dk

are locked. Every day, I use exactly one door, and I am equally likely to pick any of the d

doors. At the end of the day, I leave the door I used that day locked.

(a) Show that the number of unlocked doors at the end of day n, Ln, evolves as the state in a

Markov process for n 1. Write down the transition probabilities pij.

(b) List transient and recurrent states.

(c) Is there an absorbing state? How does rij(n) behave as n ?

(d) Now, suppose that each day, if the door I pick in the morning is locked, I will leave it

unlocked at the end of the day, and if it is initially unlocked, I will leave it locked. Repeat parts

(a)-(c) for this strategy.

(e) My third strategy is to alternate between leaving the door I use locked one day and

unlocked the next day (regardless of the initial condition of the door.) In this case, does the

number of unlocked doors evolve as a Markov chain, why/why not?

Q7) Mr. Mean Variance has the only key which locks or unlocks the door to Building 59, the

Probability Building. He visits the door once each hour on the hour. When he arrives: If the

door is unlocked, he locks it with probability 0.3.If the door is locked, he unlocks it with

probability 0.8.

(a) After he has been on the job several months, is he more likely to lock the door or to unlock

it on a randomly selected visit?

(b) With the process in the steady state, Joe arrived at Building 59 two hours ahead of Harry.

What is the probability that each of them found the door in the same condition?

(c) Given the door was open at the time Mr. Variance was hired, determine the expected value

of the number of visits up to and including the one on which he unlocks the door himself for

the first time.

In the following, suppose that every time after he leaves the building, he comes back after T

hours, where T is equal to 1 with probability 1/2, and to 2 with probability 1/2.

Q8) Jogger problem

Each morning an individual leaves his house and goes for a run. He is equally likely to leave

either from the front or the back door. Upon leaving the house, he chooses a pair of running

shoes (or goes running barefoot if there are no shoes at the door from which he departed). On

his return he is equally likely to enter, and leave his running shoes, either by the front door or

the back door. If he owns a total of k pairs of running shoes, what proportion of the time does

he run barefoot?

[Adapted From: Sheldon M Ross: Introduction to Probability Models]

Q9) Umbrella problem

An individual has k umbrellas. Each morning when he leaves home, if it is raining, he takes

an umbrella with him to office. Similarly in the evening, if it is raining, he takes an umbrella

home with him. If it is not raining he does not carry an umbrella. Find the proportion of the

time that the man gets wet if on a given day it is equally likely to rain or not rain.

[Adapted from: Sheldon M Ross: Introduction to stochastic Models]

Q10) At the Probability Coffee house of IIT, there is only one cashier. Due to the limited

space, she allows only m customers to line before her at any time. If customer finds there is m

customer including the one being served by the cashier, he will leave the Coffee house

immediately. Every minute, exactly one of the following occurs:

a) One new customer arrives with probability p;

b) One existing customer leaves with probability kq, where k is the number of customers

in the House; or

c) No new customer arrives and no existing customer leaves with probability1-p-kq

if there is at least one customer in the House, and with probability 1-p otherwise.

Define appropriate states and draw the transition probability graph.

2. After the House has been open for a long time, you walk into the House.

Calculate how many customers you expect to see in line.

Q11) Suppose the gamblers ruin problem that the probability of winning a bet depends on the

gamblers present fortune. Specifically, suppose that i is the probability that the gambler wins

a bet when his or her fortune is i. given that the gamblers initial fortune is i, let P(i) denote the

probability that the gambler reaches N before 0.

1. Derive a formula that relates P(i) to P(i-1) and P(i+1)

2. Using the same approach as in the gambler ruin problem, solve the equation of

part (1) for P(i).

[Adapted from: Sheldon M. Ross Introduction to Probability models eighth edition,

Academic press,(263)]

Q12) At all times an urn contains N balls-some white balls and some black balls. At each

stage, a coin having probability p, 0<p<1, of landing heads is flipped. If head appears, then a

ball is chosen at random from the urn and is replaced with white ball; if tails appears, then a

ball is chosen from the urn and is replaced by a black ball. Let Xn denote the number of white

ball in the urn after the nth stage.

1. Is [Xn, ,n0] a Markov chain? If so explain why.

2. What are its classes? What are the periods? Are they transient or recurrent?

3. Compute the transition probabilities Pij.

4. Let N=2. Find the proportion of time in each state.

5. If p=1, what is the expected time until there are only white balls in the urn if

initially there are i white and N-i black?

[Adapted from: Sheldon M. Ross Introduction to Probability models eighth edition,

Academic press,(264)]

- lecture-15-dynamic-programming.pdfUploaded bykaviyamurugan
- Wagner WhitinUploaded bySaddam Hossain Joy
- Dynamic Programming[2003]Uploaded byJeff Anderson Collins
- Markov ChainUploaded byTJ Palanca
- Markov ChainsUploaded byLakshmi Burra
- Dynamic ProgrammingUploaded byVenkatesh Gopalaswamy
- Markov ChainUploaded bymustufahaider
- Markov Chain ModelUploaded byChinmay Patel
- markov_chains_4Uploaded byAravind Raja
- Integer ProgrammingUploaded bymanendra chopra
- IPUploaded byrajuraja
- Dynamic Programming- Mirage GroupUploaded byDoc Sandy
- MarkovChainsUploaded bydifx
- continuous markov chainUploaded bySimerjot Kaur
- Dynamic ProgrammingUploaded byHector Triana
- markov chain chapter 6Uploaded bydurga03
- GetPDFServletUploaded byDalibor Petkovic
- Dynamic ProgrammingUploaded byKyambadde Francisco
- Markov ChainUploaded bySwasti Singh
- Chapter 4 Network Planning 1Uploaded bythiru tmr
- Marcov chainUploaded byIsack Bulugu
- Dynamic ProgrammingUploaded byChaitanya Caprico
- CTMCnotesUploaded bydapias09
- Application of Markov Chain to the Assessment of Students Admission and Academic Performance in Ekiti State UniversityUploaded byIJSTR Research Publication
- TreesUploaded byNatasha Chandra
- Decision TheoryUploaded byNavlika Dutta
- UCLA Math 61 final reviewUploaded byfatfakernig
- 4703-07-Notes-MC.pdfUploaded byjpalacios806853
- LC21_GraphPrimUploaded byOgo Ikefuna
- IJCTT-V3I1P116Uploaded bysurendiran123

- Eagle Component NamingUploaded byChandrashekhar Choudhari
- Butterfly.pdfUploaded byneedernt
- x rayUploaded byChandrashekhar Choudhari
- Pros & ConsUploaded byChandrashekhar Choudhari
- Colourful Rhymes & Stories for KidsUploaded byAzura Ibrahim
- Andersen Fairy TalesUploaded byKhalinaNguyen
- Peer Tp Peer CommunicationUploaded byChandrashekhar Choudhari
- Cinderella.pdfUploaded byChiqui-Gra
- Markov ChainUploaded byChandrashekhar Choudhari
- B.PUploaded byChandrashekhar Choudhari
- The Moments of a Random VariableUploaded byChandrashekhar Choudhari
- Research Methodology Methods and Techniques by CR KothariUploaded byDaisy
- Proofs VarianceUploaded byClaudio Amaury C. Jiménez
- Matlab Ttutorial 1.pptUploaded byChandrashekhar Choudhari
- Mbp1007 Burns UsUploaded byChandrashekhar Choudhari
- Alfred Hitchcock-The Three Investigators 09-The Mystery of the Screaming ClockUploaded byChandrashekhar Choudhari
- En Duitsaku Lesson Plan PrimaryUploaded byChandrashekhar Choudhari
- Secrets of the Da Vinci CodeUploaded byChandrashekhar Choudhari
- Brochure_RACE(final).pdfUploaded byChandrashekhar Choudhari
- Existing Teaching Methods.pptUploaded byChandrashekhar Choudhari
- 01 IntroductionUploaded bycmc_pc2
- The Three Investigators = the Mystery of the Terror CastleUploaded byapi-3833682
- Temporary Out of OrderUploaded byChandrashekhar Choudhari
- Alfred Hitchcock-The Three Investigators 08-The Mystery of the Silver SpiderUploaded byChandrashekhar Choudhari

- Binaryno Using StackUploaded byMadhava Reddy Yeruva
- Bio – inspired Algorithms for Resource AllocationUploaded byBinu Ayyappan
- Lecture1.pptUploaded byhoo
- Express Learning Automata Theory and Formal Languages PDF Free DownloadUploaded byShanna
- Internet Chapter 8Uploaded byJames Lanham
- Java Data Structures HilfingerUploaded by789by13
- 6Uploaded byDoğukan Duman
- Spanning Tree AbstractUploaded bysrinubotsa
- VB6 LearningUploaded bytstsst
- FA Transition GraphsUploaded byadeelmurtaza5730
- CodesUploaded byAnkita Aggarwal
- CS2352 Principle of Compiler Desgin NotesUploaded byPuspha Vasanth R
- ERA2010 Conference ListUploaded byasif1253
- Novel multiple criteria decision making methods based on bipolar neutrosophic sets and bipolar neutrosophic graphsUploaded byMia Amalia
- CURS 2 - Tipuri de DateUploaded byAlexandra Stan
- Recursion TheoryUploaded byttnleeobh7382
- An emotionally biased ant colony algorithm for pathfinding in gamesUploaded bydaniel_cahyadi_1
- AI Practical FileUploaded byTridon
- Par Met is ManualUploaded byPrashanth Sheshappa
- AlgorithmUploaded byVijayalakshmi Vengattaramane
- 1209.3495v1Uploaded bynooblywojja2979
- acoUploaded byjoshika116
- Nuts and BoltsUploaded byVibhash Kumar
- DSA_First_DraftUploaded bymarcio1404
- GA faqUploaded byAmit Mishra
- H12-solution[www.alirezaweb.com].pdfUploaded byVahid Esmaeilzadeh
- Java New Notes for MSc ITUploaded byArun Kumar Garg
- MCA-06-PGDCA-06-MSC(IT)-06Uploaded bynaveengargns
- Data Compression Lempel-Ziv CodingUploaded bysfofoby
- 1 Introduction PEDROFINALUploaded bypaco449