Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Standard view
Full view
of .

1 Introduction
2 Examples of Markov chains
2.1 A mouse in a cage
2.2 Bank account
2.5 Actuarial chains
3 The Markov property
3.1 Deﬁnition of the Markov property
the Markov property
3.2 Transition probability and initial distribution
3.3 Time-homogeneous chains
4 Stationarity
4.1 Finding a stationary distribution
5 Topological structure
5.1 The graph of a chain
5.3 The relation “communicates with”
5.4 Period
6 Hitting times and ﬁrst-step analysis
7 Gambler’s ruin
8 Stopping times and the strong Markov property
9 Regenerative structure of a Markov chain
10 Recurrence and transience
10.1 First hitting time decomposition
10.2 Communicating classes and recurrence/transience
11 Positive recurrence
12 Law (=Theorem) of Large Numbers in Probability The-
13 Law of Large Numbers for Markov chains
13.1 Functions of excursions
13.2 Average reward
14 Construction of a stationary distribution
15 Positive recurrence is a class property
16 Uniqueness of the stationary distribution
17 Structure of a stationary distribution
18 Coupling and stability
18.1 Deﬁnition of stability
18.2 The fundamental stability theorem for Markov chains
18.3 Coupling
18.4 Proof of the fundamental stability theorem
18.5 Terminology
18.6 Periodic chains
19 Limiting theory for Markov chains*
19.1 Limiting probabilities for null recurrent states
19.2 The general case
20 Ergodic theorems
21 Finite chains
22 Time reversibility
23 New Markov chains from old ones*
23.1 Subsequences
23.2 Watching a Markov chain when it visits a set
23.3 Subordination
23.4 Deterministic function of a Markov chain
24 Applications
24.1 Branching processes: a population growth application
24.2 The ALOHA protocol: a computer communications application
24.4 The Wright-Fisher model: an example from biology
24.5 A storage or queueing application
25 Random walk
26 The simple symmetric random walk in dimension 1: path
26.1 Paths that start and end at speciﬁc points
27.1 Distribution after n steps
27.2 The ballot theorem
27.3 Some conditional probabilities
27.4 Some remarkable identities
28 The reﬂection principle for a simple random walk in di-
28.1 Distribution of hitting time and maximum
29 Urns and the ballot theorem
30 The asymmetric simple random walk in dimension 1
30.1 First hitting time analysis
31 Recurrence properties of simple random walks
31.1 Asymptotic distribution of the maximum
31.2 Return probabilities
31.3 Total number of visits to a state
32 Duality
33 Amount of time that a SSRW is positive*
0 of .
Results for:
P. 1
Markov Chain Random Walk

# Markov Chain Random Walk

Ratings: (0)|Views: 91 |Likes:

### Availability:

See more
See less

04/01/2014

pdf

text

original

Pages 4 to 85 are not shown in this preview.