Stochastic Models I (for first-year doctoral students)

Attribution Non-Commercial (BY-NC)

17 views

Stochastic Models I (for first-year doctoral students)

Attribution Non-Commercial (BY-NC)

- Revised GRE Multiple Choice Questions
- Free Report on Virtual Football Trading.pdffree Report on Virtual Football
- Iecep Math 5
- Kalman and Bayesian Filters in Python
- Puzzle
- lk
- Betfair the Ultimate Guide eBook
- obp
- You Can Catch One of Any Pokemon Every 5 Minutes on Average
- 21st Annual Wealth Creation Study (2011-2016)
- Disc
- The Life of Game - Sports, Stats, And the Lessons They Teach Us
- lec4.5
- events involving not and or
- rubric (1)
- answers
- CAT Level Permutation and Combination, Probability Questions
- eli[1]
- Converging Factor Racing Formulas
- The First-Half Betting System - Part Two - Value Bets

You are on page 1of 5

Problem 4.1 Let Dn be the random demand of time period n. Clearly Dn is i.i.d. and independent of all Xk for k < n. Then we can represent Xn + 1 by Xn+1 = max{0, Xn 1[s,) (Xn ) + S 1[0,s) (Xn ) Dn+1 } which depends only on Xn since Dn+1 is independent of all history. Hence {Xn , n 1} is a Markov chain. It is easy to see assuming k = 0 for k < 0, S j if i < s, j > 0 if i < s, j = 0 k=S k Pij = ij if i s, j > 0 if i s, j = 0 k=i k The following three problems (4.2, 4.4, 4.5) needs a fact: P(A B |C ) = P(A|B C )P(B |C ) which requires a proof to use. Try to prove it by yourself.

Problem 4.2 Let S be the state space. First we show that P(Xnk +1 = j |Xn1 = i1 , , Xnk = ik ) = P(Xnk +1 = j |Xnk = ik ) by the following : Let A = {Xnk +1 = j }, B = {Xn1 = i1 , , Xnk = ik } and Bb , b I are elements of {(Xl , l nk , l = n1 , , l = nk ) : Xl S}. P(A|B ) =

bI

bI

= =

bI

bI

= P(A|Xnk = ik )

We consider the mathematical induction on l n m. For l = 1, we just showed. Now assume that the statement is true for all l l and consider l = l + 1: P(Xn = j |Xn1 = i1 , , Xnk = ik ) =

iS

P(Xn = j, Xn1 = i|Xn1 = i1 , , Xnk = ik ) P(Xn = j |Xn1 = i, Xn1 = i1 , , Xnk = ik )P(Xn1 = i|Xn1 = i1 , , Xnk = ik )

iS

= =

iS

P(Xn = j |Xn1 = i)P(Xn1 = i|Xnk = ik ) By l l cases P(Xn = j |Xn1 = i, Xnk = ik )P(Xn1 = i|Xnk = ik )

iS

= =

iS

= P(Xn = j |Xnk = ik ) which completes the proof for l = l + 1 case. Problem 4.3 Simply by Pigeon hole principle which saying that if n pigeons return to their m(< n) home (through hole), then at least one home contains more than one pigeon. Consider any path of states i0 = i, i1 , , in = j such that Pik ,ik+1 > 0. Call this a path from i to j . If j can be reached from i, then there must be a path from i to j . Let i0 . , in be such a path. If all of values i0 , , in are not distinct, then there must be a subpath from i to j having fewer elements (for instance, if i, 1, 2, 4, 1, 3, j is a path, then so is i, 1, 3, j ). Hence, if a path exists, there must be one with all distinct states. Problem 4.4 Let Y be the rst passage time to the state j starting the state i at time 0.

n Pij = P(Xn = j |X0 = i) n

=

k=0 n

k=0 n

= =

k=0 n

n k k Pjj fij k=0

Problem 4.5 (a) The probability that the chain, starting in state i, will be in state j at time n without ever having made a transition into state k . 2

(b) Let Y be the last time leaving the state i before rst reaching to the state j starting the state i at time 0.

n Pij = P(Xn = j |X0 = i) n

=

k=0 n

k=0 n

= =

k=0 n

k P(Xn = j, Y = k |Xk = i)Pii k=0 n

= =

k=0 n

=

k=0

nk k Pij/i Pii

Problem 4.7 (a) Here is an argument: Let x be the expected number of steps required to return to the initial state (the origin). Let y be the expected number of steps to move to the left 2 steps, which is the same as the expected number of steps required to move to the right 2 steps. Note that the expected number of steps required to go to the left 4 steps is clearly 2y , because you rst need to go to the left 2 steps, and from there you need to go to the left 2 steps again. Then, consider what happens in successive pairs of steps: Using symmetry, we get x = 2 + (0 (1/2) + y (1/2) = 2 + y/2 and y = 2 + (0 (1/4) + y (1/2) + (2 y ) (1/4) If we subtract y from both sides, this last equation yields 2=0. Hence there is no nite solution. The quantity y must be innite; a nite value cannot solve the equation. (b) Note that the expected number of returns in 2n steps is the sum of the probabilities of returning in 2k steps for k from 1 to n, each term of which is binomial. Thus, we have

n

E [N2n ] =

k=1

(2k )! (1/2)2k , k !k !

which can be shown to be equal to the given expression by mathematical induction. (c) We say that f (n) g (n) as n if f (n)/g (n) 1 as n . By Stirlings approximation, (2n + 1) so that E [Nn ] Problem 4.8 (a) Pij = 2n/ as n . (2n)! (1/2)2 n 2 n/ , n!n!

j k=i+1 k

j>i

(b) {Ti , i 1} is not a Markov chain - the distribution of Ti does depend on Ri . {(Ri+1 , Ti ), i 1} is a Markov chain. P(Ri+1 = j, Ti = n|Ri = l, Ti1 = m) = j

l l n1

k=l+1 k

k

k=0 n1 k=l+1

= j

k=0

j>l

(c) If Sn = j then the (n + 1)st record occurred at time j . However, knowledge of when these n + 1 records occurred does not yield any information about the set of values {X1 , , Xj }. Hence, the probability that the next record occurs at time k , k > j , is the probability that both max{X1 , , Xj } = max{X1 , , Xk1 } and that Xk = max{X1 , , Xk }. Therefore, we see that {Sn } is a Markov chain with Pjk = j 1 , k1 k k>j .

n Pij = E[number of visits to j |X0 = i] n=1

= E[number of visits to j |ever visit j, X0 = i]fij = (1 + E[number of visits to j |X0 = j ])fij fij = <. 1 fjj since 1 + number of visits to j |X0 = j is geometric with mean 4

1 1fjj .

= 1+

n=1

n Pjj .

Problem 4.12 If we add the irreducibility of P, it is easy to see that = unique) limiting probability.

1 n1

is a (and the

- Revised GRE Multiple Choice QuestionsUploaded byRejan Chitrakar
- Free Report on Virtual Football Trading.pdffree Report on Virtual FootballUploaded bykingdave73
- Iecep Math 5Uploaded byJulio Gabriel Aseron
- Kalman and Bayesian Filters in PythonUploaded bydmorenoc
- PuzzleUploaded byAnil Rai
- lkUploaded by53melmel
- Betfair the Ultimate Guide eBookUploaded byRicardo Oliveira
- obpUploaded bysharma05031989
- You Can Catch One of Any Pokemon Every 5 Minutes on AverageUploaded byJosh Pham
- 21st Annual Wealth Creation Study (2011-2016)Uploaded byKirmani
- DiscUploaded byasdmin
- The Life of Game - Sports, Stats, And the Lessons They Teach UsUploaded bypjs15
- lec4.5Uploaded bytamleduc1810dn
- events involving not and orUploaded byapi-275260919
- rubric (1)Uploaded byapi-289236415
- answersUploaded byapi-195130729
- CAT Level Permutation and Combination, Probability QuestionsUploaded byKshitija Sahani
- eli[1]Uploaded bytina
- Converging Factor Racing FormulasUploaded byhomomysticus
- The First-Half Betting System - Part Two - Value BetsUploaded byokwuosa Raphel
- intro robotics syllabusUploaded byapi-267316468
- 2_dicesUploaded bymusafir7777
- 2011 Fall 440 Midterm OneUploaded byEd Z
- Che410 Course Compact 20182019Uploaded bynelson
- Operations Incident TemplateUploaded byJohnPaulChristopherPablo
- Equations.pdfUploaded byIrene Ching
- 10th-science-ex-1-1-amir-shehzad.pdfUploaded byTlha Tlha
- Math Review for ACT College Entrance Exam.docxUploaded byMandalones Joebert
- dicrete distribution.docxUploaded byHarry Otagan Deogracias
- 12th math ncertUploaded byKumar Pranay

- midtermTwoF07solsUploaded bySongya Pan
- Midterm Two f 07Uploaded bySongya Pan
- midtermTwo6711F12solsUploaded bySongya Pan
- Midterm Two 6711 f 12Uploaded bySongya Pan
- midtermTwo6711F10solsUploaded bySongya Pan
- Midterm Two 6711 f 10Uploaded bySongya Pan
- midtermOne6711F12solsUploaded bySongya Pan
- Midterm One 6711 f 12Uploaded bySongya Pan
- midtermOne6711F10solsUploaded bySongya Pan
- Midterm One 6711 f 10Uploaded bytabrupt
- midtermOne6711F08solsUploaded bySongya Pan
- Midterm One 6711 f 08Uploaded bySongya Pan
- Lect 0910Uploaded bySongya Pan
- Lect 0905Uploaded bySongya Pan
- Lect 0903Uploaded bySongya Pan
- homewkLTUploaded bySongya Pan
- homewk13SolsUploaded bySongya Pan
- homewk13Uploaded bySongya Pan
- homewk12SolsUploaded bySongya Pan
- homewk12Uploaded bySongya Pan
- homewk11SolsUploaded bySongya Pan
- homewk11Uploaded bySongya Pan
- homewk10SolsUploaded bySongya Pan
- homewk10Uploaded bySongya Pan
- homewk9Uploaded bySongya Pan
- homewk8SolsUploaded bySongya Pan
- homewk8Uploaded bySongya Pan
- homewk7SolsUploaded bySongya Pan
- homewk7Uploaded bySongya Pan

- Basics of Electrical EnggUploaded byNandha Kumar Mohan
- ACI 318- NotesUploaded byKINGZOSER
- 1ª Lista de ExercíciosUploaded byJoziel Souza
- Molecules 18 2013Uploaded byMario Juan Simirgiotis
- Strut Design SheetUploaded byHafiz Kamarudin
- UNIT 1.docUploaded bypankaj12345katre
- 9702 p1 Electricield AllUploaded byAsha D'sa
- Optimization in Heat Transfer ProcessUploaded bynp484
- CHAP3 (for students 2016).pdfUploaded byMu Ji
- Introduction to VRAY Sketchup - NomeradonaUploaded byshifa musthafa
- Quantum KonicaUploaded bymamunngs
- oUploaded byJulius Rey Baniqued
- CHAPTER 4 B PERIODIC TABLE OF ELEMENTS.pptxUploaded bynor
- class 5 photosynthesis.pptUploaded byArpit Pradhan
- 1-s2.0-S0263876299718186-main.pdfUploaded byLeydi Patiño
- Consecutive Model for SoilUploaded bySushma Pulikanti
- Fuel EnergizerUploaded bySuri Suresh
- question on dimension.docxUploaded byMantu Das
- L2001A LAB MANUALUploaded byMuhammad Rais Fawwazi
- Hoek Brown HistoryUploaded bysetyawan_m_heery
- Groen Et Al (2003) Pore Size Distribution Review PaperUploaded byshivangi
- Chm121 Module 1Uploaded byDan Lewa
- Notes Chapter 4 Acid and BasesUploaded bychemdog82
- Mains Test 3Uploaded bySagarDalvi
- Lecture 1-3Uploaded byAdal Arasu
- unit 7 notes for webUploaded byapi-236826747
- Gyros CopUploaded byelfrich
- Dye Structure and ColourUploaded bySumera Sarwar
- presentation 19Uploaded byapi-300665835
- RusoUploaded byLeonardo Muñoz

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.