Welcome to Scribd. Sign in or start your free trial to enjoy unlimited e-books, audiobooks & documents.Find out more
Standard view
Full view
of .
Look up keyword or section
Like this

Table Of Contents

1 Introduction
2 Examples of Markov chains
2.1 A mouse in a cage
2.2 Bank account
2.5 Actuarial chains
3 The Markov property
3.1 Definition of the Markov property
the Markov property
3.2 Transition probability and initial distribution
3.3 Time-homogeneous chains
4 Stationarity
4.1 Finding a stationary distribution
5 Topological structure
5.1 The graph of a chain
5.2 The relation “leads to”
5.3 The relation “communicates with”
5.4 Period
6 Hitting times and first-step analysis
7 Gambler’s ruin
8 Stopping times and the strong Markov property
9 Regenerative structure of a Markov chain
10 Recurrence and transience
10.1 First hitting time decomposition
10.2 Communicating classes and recurrence/transience
11 Positive recurrence
12 Law (=Theorem) of Large Numbers in Probability The-
13 Law of Large Numbers for Markov chains
13.1 Functions of excursions
13.2 Average reward
14 Construction of a stationary distribution
15 Positive recurrence is a class property
16 Uniqueness of the stationary distribution
17 Structure of a stationary distribution
18 Coupling and stability
18.1 Definition of stability
18.2 The fundamental stability theorem for Markov chains
18.3 Coupling
18.4 Proof of the fundamental stability theorem
18.5 Terminology
18.6 Periodic chains
19 Limiting theory for Markov chains*
19.1 Limiting probabilities for null recurrent states
19.2 The general case
20 Ergodic theorems
21 Finite chains
22 Time reversibility
23 New Markov chains from old ones*
23.1 Subsequences
23.2 Watching a Markov chain when it visits a set
23.3 Subordination
23.4 Deterministic function of a Markov chain
24 Applications
24.1 Branching processes: a population growth application
24.2 The ALOHA protocol: a computer communications application
24.3 PageRank (trademark of Google): A World Wide Web application
24.4 The Wright-Fisher model: an example from biology
24.5 A storage or queueing application
25 Random walk
26 The simple symmetric random walk in dimension 1: path
26.1 Paths that start and end at specific points
27.1 Distribution after n steps
27.2 The ballot theorem
27.3 Some conditional probabilities
27.4 Some remarkable identities
27.5 First return to zero
28 The reflection principle for a simple random walk in di-
28.1 Distribution of hitting time and maximum
29 Urns and the ballot theorem
30 The asymmetric simple random walk in dimension 1
30.1 First hitting time analysis
30.2 First return to the origin
30.3 The distribution of the first return to the origin
31 Recurrence properties of simple random walks
31.1 Asymptotic distribution of the maximum
31.2 Return probabilities
31.3 Total number of visits to a state
32 Duality
33 Amount of time that a SSRW is positive*
0 of .
Results for:
No results containing your search query
P. 1
Markov Chain Random Walk

Markov Chain Random Walk

Ratings: (0)|Views: 564|Likes:
Published by Fann Ming

More info:

Published by: Fann Ming on Feb 02, 2012
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less





You're Reading a Free Preview
Pages 4 to 85 are not shown in this preview.
You're Reading a Free Preview
Pages 90 to 104 are not shown in this preview.
You're Reading a Free Preview
Pages 108 to 128 are not shown in this preview.

Activity (6)

You've already reviewed this. Edit your review.
1 hundred reads
fwang1 liked this
Diego Del Arco liked this
Diego Del Arco liked this
Will Nicholson liked this
klavdaki3823 liked this

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->