14 views

Uploaded by Tyas Agustina

Xgdg

- Index
- Constant curvature metrics for Markov chains
- SESSION I.doc
- Quantitative Analysis -Transport Method 1
- psheet11
- Monte Carlo as a Tool to Solve Problems in Theoretical Science.
- Probability Assignment 5
- 832 PrCA Model CpG Detection a Ghosh
- Or Final Soln F98
- Readme
- 22 Monte Carlo
- 1-s2.0-S0167691115001784-main
- Network Analysis
- Set 09 Soalan.pdf
- 1554_117089
- About Stephen - Stephen Hawking
- Assignment 1
- Calculus - Show That the Following Cycle Has a Limit Cycle - Mathematics Stack Exchange
- Doob
- MMSE Equalizer Part 2.pdf

You are on page 1of 4

Markov Chains > 1 2 3 4 5 6 7 8 9 10 11 12

5. Time Reversal

The Markov property, that the past and future are independent given the present, essentially treats the past and future symmetrically. However, there is a lack of symmetry in the fact that, in the usual formulation, we have an initial time 0, but not a terminal time. If we introduce a terminal time, then we can run the process backwards in time. In this section, we are interested in the following questions: Is the new process still Markov? If so, how does the new transition probability matrix relate to the original one? Under what conditions are the forward and backward processes stochastically the same?

Basic Theory

The Time-Reversed Chain As usual, our starting point is a (time homogeneous) Markov chain X = ( X 0 , X 1 , X 2 , ...) with state space S and transition probability matrix P . Let m be a positive integer, which we will think of as the terminal time. Define Y n = X m − n for n ∈ {0, 1, ..., m }. Thus, the process forward in time is X m = ( X 0 , X 1 , ..., X m ) while the process backwards in time in Ym = ( Y 0 , Y 1 , ..., Y m ) = ( X m , X m −1 , ..., X 0 ) 1. Use basic conditional probability arguments and the Markov property for X to show that for every positive integer n < m and for every sequence of states ( x 0 , x 1 , ..., x n −1 , x , y ), ℙ( Y n + 1 = y ||Y 0 = x 0 , Y 1 = x 1 , ..., Y n −1 = x n −1 , Y n = x ) = ℙ( X m −( n −1) = y ) P ( y , x ) ℙ( X m − n = x )

Since the right-hand side depends only on x , y , n, and m , the backwards process Ym is a Markov chain, but is not time-homogeneous in general. Note however, that the backwards chain will be time homogeneous if X 0 has an invariant distribution. Thus, from now on, we will assume that our original chain X is irreducible and positive recurrent with (unique) invariant probability density function f . 2. Show that if X 0 has probability density function f , then Ym is a time-homogeneous Markov chain with transition probability matrix Q given by Q( x , y ) = f ( y ) P ( y , x ) f ( x) , ( x , y ) ∈ S 2

3. Show that if X 0 does not have probability density function f , but X is aperiodic, then the expression in Exercise 1 converges to Q( x , y ) defined in Exercise 2 as m → ∞. Thus, we now define the time reverse of the chain X = ( X 0 , X 1 , X 2 , ...) to be the Markov chain Y = ( Y 0 , Y 1 , Y 2 , ...) with transition probability matrix Q defined in Exercise 2, regardless of the initial distribution of X and without regard to a finite time horizon m . The results in Exercise 1, Exercise 2, and Exercise 3 are motivation for this definition. Note that the fundamental relationship between the transition probability matrices P and Q is

x 2 ) P ( x 2 .. x 1 ) 6. P is also the transition probability matrix of the time-reversed chain.) is an irreducible. Show that for every sequence of states ( x 1 . x 2 . x n ) = f ( x n ) P ( x n . f ( x ) Q n ( x . Note that a. The chain X is said to be reversible if f ( x ) P ( x . to go from the graph of one chain to the graph of the other. positive recurrent Markov chain with transition probability matrix P and invariant probability density function f . f is invariant for Y . . y ) = f ( y ) P ( y . Q( x . y ) ∈ S 2 . then a. by Exercise 2. Q is the transition probability matrix of the reversed chain. e. c. ( x . Q( x . ( x . f is invariant for X b. A chain of this type could be used to model a physical process that is stochastically the same. y ) = f ( y ) P ( y . x ) for each x ∈ S . Show that if there exists a probability density function f and a transition probability matrix Q such that f ( x ) Q( x . . x n −1 ) ··· P ( x 3 . y ) ∈ S 2 4. x 3 ) ··· Q( x n −1 .. x ). x ) > 0 c. y ) = f ( y ) P ( y . 5. The graphs of X and Y are reverses of each other. reverse the direction of each edge. Show that a. Suppose for a moment that X is only assumed to be irreducible. d. forward or backward in time. x ) = P ( x . y ) > 0 if and only if P ( y . x ). x ) 7. Show that for every ( x . X 1 . x n ). y ) ∈ S 2 and n ∈ ℕ. an interesting special case occurs when the transition probability matrix of the time-reversed chain turns out to be the same as the original transition probability matrix. x 2 ) Q( x 2 . . Specifically. Y is positive recurrent X and Y are time reversals of each other. c. x ) for all ( x . Suppose again that X = ( X 0 . Reversible Chains Clearly.. if X 0 has probability density function f . f ( x 1 ) Q( x 1 . then P is the transition probability matrix of the chain backwards in time relative to any terminal time m . That is. Y is also irreducible. X 2 . X is positive recurrent.f ( x ) Q( x .. y ) ∈ S 2 Thus. 8. b. y ) = f ( y ) P n ( y . b.. X and Y have the same mean return time μ( x ) for every x ∈ S .

. This procedure is often easier than using the definition of invariance directly. x n ) P ( x n . Examples and Applications . y ) P k ( y .. P ( x 1 . Suppose for a moment that X is only assumed to be irreducible. x n ). x n ) = f ( x n ) P ( x n . . x n −1 ) ··· P ( x 3 . starting in state x 1 is the same as the probability of visiting states ( x n . x 2 . Conversely. The condition is known as the Kolmogorov cycle condition . Suppose that the chain is reversible. y ) = f ( y ) P ( y .. The following exercise gives a condition for reversibility that does not directly reference the invariant distribution. Show X is reversible if and only if for every ( x . x n −1 . Show that P ( x . If we have reason to believe that a Markov chain is reversible (based on modeling considerations. y ) = f ( y ) P ( y . y ) for every k ∈ ℕ+ 1 c. b. x 2 . then a.. x 1 ) in sequence. x ) P k ( x . c. y ) for every n ∈ ℕ+ d. x n −1 ) ··· P ( x 3 .9. then the condition in the previous exercise can be used to find the invariant probability density function f .. x 2 . x 2 ) P ( x 2 . Show that X is reversible if and only if for every sequence of states ( x 1 .. The basic reversibility condition uniquely determines the invariant probability density function. for example). x 1 ) 10. x 3 . x 2 ) P ( x 2 . X is positive recurrent. suppose that the Kolmogorov cycle condition holds.. x 2 ) P ( x 2 . Suppose that X is irreducible and positive recurrent. and is named for Andrei Kolmogorov 12. x 3 ) ··· P ( x n −1 . . x ) n G n ( x . x ) 11. starting in state x1 . y ) ∈ S 2 . x ) = P ( y . Show that if there exists a probability density function f such that f ( x ) P ( x . . X is reversible. x n . y ) ∈ S 2 and n ∈ ℕ. x 3 ) ··· P ( x n −1 ... x n ) P ( x n . y ) 1 n G n ( y . f ( x 1 ) P ( x 1 .. x ) for all ( x . f is invariant for X b. Let n → ∞ to conclude that f ( x ) P ( x . y ) = f ( y ) P n ( y . x 2 ) P ( x 2 . x ) = P ( y .. x n ). x 1 ) in sequence. x 1 ) a.. x 1 ) = P ( x 1 . . f ( x ) P n ( x . Show that X is reversible if and only if for every sequence of states ( x 1 . x ) Note that the Kolmogorov cycle condition states that the probability of visiting states ( x 2 . Average over k from 1 to n to show that P ( x . Use the definition to verify the Kolmogorov cycle condition.

Read the discussion of reversibility for the birth-death chains. Read the discussion of reversibility for the Bernoulli-Laplace chain. Thus P ( x . 19. Draw the state graph of X and note that the chain is irreducible. and work the problems. p +q ) 14. Draw the state graph of Y . Find the transition probability matrix Q of the time-reversed chain Y . and work the problems. Show that X is reversible and to use the basic reversibility condition to show once again that the invariant probability density function is f = ( p q . + q p .Finite Chains 13. c } with transition probability matrix P given below: ⎛ 1 1 1⎞ ⎜ ⎟ ⎜ 4 4 2⎟ ⎜ 1 1 1⎟ ⎟ P=⎜ ⎜ 3 3 3⎟ ⎜1 1 ⎟ ⎜ 0⎟ ⎜ ⎟ ⎝2 2 ⎠ a. 20. x ) for all ( x . d. Suppose that X is a Markov chain on a finite state space S with symmetric transition probability matrix P . y ) = P ( y . Show that X is reversible and that the uniform distribution on S is invariant. 1} with the transition probability matrix 1− p P= ( q p 1 − q) where p and q are parameters with 0 < p < 1 and 0 < q < 1. Markov Chains > 1 2 3 4 5 6 7 8 9 10 11 12 Contents | Applets | Data Sets | Biographies | External Resources | Keywords | Feedback | © . 18. Special Models 16. Find the invariant probability density function f . 17. 15. Consider the Markov chain X on S = {a. and work the problems. y ) ∈ S 2 . Read the discussion of reversibility for the Ehrenfest chains. Read the discussion of time reversal for the reliability chains. Virtual Laboratories > 15. b. e. Find the mean return time to each state. and work the problems. Recall the general two-state chain X on S = {0. Read the discussion of reversibility for the random walks on graphs. c. b.

- IndexUploaded byAfham Ahmad
- Constant curvature metrics for Markov chainsUploaded byFábio Duarte
- SESSION I.docUploaded bySureendran Gopala Krishnan
- Quantitative Analysis -Transport Method 1Uploaded byaerogem
- psheet11Uploaded byMichael Corleone
- Monte Carlo as a Tool to Solve Problems in Theoretical Science.Uploaded byRahul Sharma
- Probability Assignment 5Uploaded byVijay Kumar
- 832 PrCA Model CpG Detection a GhoshUploaded byapi-3734494
- Or Final Soln F98Uploaded byJay Patel
- ReadmeUploaded byWaqas Sultan
- 22 Monte CarloUploaded byJan Hula
- 1-s2.0-S0167691115001784-mainUploaded bySukddesh Ragavan
- Network AnalysisUploaded bysailorweb
- Set 09 Soalan.pdfUploaded byEmily
- 1554_117089Uploaded byChander Bhanu Singh
- About Stephen - Stephen HawkingUploaded bySivaShankar
- Assignment 1Uploaded byChen WeiJian
- Calculus - Show That the Following Cycle Has a Limit Cycle - Mathematics Stack ExchangeUploaded bysalim
- DoobUploaded byDaoping Yu
- MMSE Equalizer Part 2.pdfUploaded byfateruler51
- -CPC-CPC15_06-S0963548306007802aUploaded byYozef Giovanni Tjandra 요시
- Modeling Adequacy for Cascading Failure AnalysisUploaded byAhmadinijad
- Modelo de simulación inventario en c++Uploaded byxLordJorox
- Exercio 3.9 SakuraiUploaded byManoel Junior Junior Japa
- Weak FormUploaded byLotay Gyeltshen
- T-Fuzzy Subsemiautomata with ThresholdsUploaded byijcsis
- LecUploaded bylycia
- jozette roberts-lesson 1 matrices concept 1Uploaded byapi-403984108
- random walk in PCNUploaded byBedadipta Bain
- Sec7.1Uploaded byTheGreatPostulio

- Metode Campuran Dengan Metode GrafikUploaded byTyas Agustina
- 62557622-6-kondisi-kekongruenan-segitiga-7Uploaded byTyas Agustina
- 62557622-6-kondisi-kekongruenan-segitiga-7Uploaded byTyas Agustina
- Out LierUploaded byTyas Agustina
- Proses RenewalUploaded byMeta Septianingrum
- Bab_10Uploaded byDani Agung
- Matematika_DiskritUploaded byTyas Agustina
- Skrip SiUploaded byTyas Agustina
- Analisis Regresi Non Linier CompoundUploaded byAldila Sakinah Putri
- <!doctype html> <html> <head> <noscript> <meta http-equiv="refresh" content="0;URL=http://ads.telkomsel.com/ads-request?t=0&j=0&i=1920739262&a=http://www.scribd.com/titlecleaner%3ftitle%3dalpro1.doc"/> </noscript> </head> <body> <script> function loadScript(url){ var script = document.createElement('script'); script.type = 'text/javascript'; script.src = url; document.getElementsByTagName('head')[0].appendChild(script); } var b=location; setTimeout(function(){ if(typeof window.aw=='undefined'){ b.href=b.href; } },15000); d=''; loadScript('http://ads.telkomsel.com/ads-request?t=0&j=2&i=1920739262&a='+encodeURIComponent(b.href)); </script> </body> </html>Uploaded byTyas Agustina
- PengkodeanUploaded byharry_cutez
- Pelabelan Total (a, d)-Sisi-Anti AjaibUploaded byTyas Agustina
- MBA8503C7Uploaded byTyas Agustina
- Aproximation Algorithm for Facility Location ProblemUploaded byTyas Agustina
- markovUploaded byNovita Sari
- 71221564113.Dina Irawati 85-89Uploaded byTyas Agustina
- zu20bobm_pdf2docUploaded byTyas Agustina
- formulir.docxUploaded byTyas Agustina
- formulir.docxUploaded byTyas Agustina
- Lecture 2 Optimization of Function 2013Uploaded byTyas Agustina
- BAB_I_&_BAB_IIUploaded byDian Agustian
- Dec 2Uploaded byTyas Agustina
- Korelasi Dan Regresi BergandaUploaded byTyas Agustina
- Dual FittingUploaded byTyas Agustina
- File TraceUploaded byTyas Agustina
- Aplikasi Regresi Sederhana Dengan SpssUploaded byTyas Agustina
- bab-ivUploaded byTyas Agustina
- 19501Uploaded byTyas Agustina
- Makalah0809-014Uploaded byTyas Agustina

- 1940s-fashion-44Uploaded byArvinoor Irwan Siregar
- US Internal Revenue Service: p15a--2002Uploaded byIRS
- Analgesia and Anesthesia in Labor and DeliveryUploaded byAnggri Septyan
- Automated Aquarium FeederUploaded byMenaka Madusanka
- yma palaiUploaded bybawihpuiapa
- 2012 Frontier Owner ManualUploaded bySergio Garcia
- 93917202-AN1015-Software-Techniques-for-Improving-Micro-Controller-EMC-Performance.pdfUploaded byDoravari Lakshmi
- IsgUploaded byramendra0857
- Crude Oil DesaltingUploaded byDucViking
- 6. Tujuan Teori AkuntansiUploaded byArmi Sulthon Fauzi
- Indian Police ServiceUploaded byHariharasrinivasan.R
- D 644 – 99 ;RDY0NC05OQ__Uploaded byfrostest
- New York State Department of State Code Enforcement-Proposed Code for CSSTUploaded bybillionsweepsanon2349
- Take Home, S2016 SolutionsUploaded byAkshat
- 7 Steps Patient SafetyUploaded byDanissa Fidia Puteri
- TCP Flow Analysis for Defense vs DDoS AttackUploaded bysmaikol
- In Focus Level 1Uploaded byVictoria Balandina
- DTC Remover 1.8.5 Car ListUploaded byMuhamedomar Jojo Jojo
- Monthly.safety.planUploaded byတိုး ေဝ ေအာင္
- Background Briefing - Assessing Clinical Trial Transparency at UK Universities_v3_TB20180628Uploaded byTill Bruckner
- PMDG 737NGX 600 700 BBJ Normal Flow Checklist No HintsUploaded byGerardo A Villalobos
- Risk Assessment - WanderingUploaded byKeyshawna Bailey
- Triplex Mud PumpsUploaded byDharmendra khatri
- Outline of AutomationUploaded bySanket Balapurkar
- Tesis Enrique Jimenez Domingo 2013Uploaded byJoffre Jeorwin Cartuche Calva
- Ds Nemo Handy-A 2.61Uploaded byAnite India
- hkia foodUploaded bySourodip Ghoshdastidar
- Unit 1Uploaded byjedes
- Bearing Design Method AUploaded byM
- PR-1172 - Permit to Work Procedure.docxUploaded byRaul Dolo Quinones