50 views

Uploaded by lex7396

UCSB PStat 160A Homework 6
Stochastic Process

- 1. Solution. a) the Following is the Program of a).
- Operation Research
- f-pqt
- img 20140710 0001
- Improving_the_accuracy_of_SINR_threshold.pdf
- Studio conflitto Palestina-Israele con hmm
- slide6m
- midterm.pdf
- Applications of Manpower With Various Stages in Business Using Stochastic Models
- Transformations of Graphs - Solutions
- Matlab Ref
- Hidden Markov Model in MATLAB - Stack Overflow
- PC an Introduction to functions
- 10.1.1.8
- social.pdf
- BOUNDING THE SMARANDACHE FUNCTION
- Summative in Math (AUGUST 9,2017)
- relationsfunctions
- A Contribution to the Stochastic Flow Shop Scheduling Problem
- Numerical analysis

You are on page 1of 2

D UE T HURSDAY, D EC 4, 2014

(1) Let (Xn ) be a random walk on {0, 1, . . . , M } with two reflecting barrier at 0 and at M > 1. That is,

the transition matrix of (Xn ) is

Pi,j

1 p

=

if 0 < i < M and j = i 1,

if i = M and j = M 1 or i = 0 and j = 1,

otherwise.

(b) Compute the invariant distribution for arbitrary M using the weighted graph method.

(c) Let T = inf {n 1 : Xn = 0} be the first return time to 0. Compute E [T | X0 = 0] and

E [T | X0 = 1].

(2) Let Z1 , Z2 , . . . be independent identically distributed, positive integer valued random variables.

These are the interarrival times of a bus service at a given bus stop. So, there is a bus at time 0,

the next bus arrives at Z1 , the next one at Z1 + Z2 and so on. Denote Un the time elapsed since the

last bus at time n and Vn the waiting time for the next bus at time n. The aim of this exercise is the

computation of the PMF Un + Vn for large n.

(a) What is the state space of the Markov chain Xn = (Un , Vn ), n 1?

(b) What are the transition probabilities of the Markov chain (Xn )n1 ?

(c) Is (Xn ) irreducible?

(d) What is the invariant distribution of (Xn ) in terms of the PMF of Z1 ?

(e) Compute

n

1X

P [Uk + Vk = j ] .

lim

n n

k=1

(f) If the law of Z1 is a so called non-lattice distribution, that is, the gratest common divisor of

{n : P [Z1 = n] > 0} is 1, then Xn is aperiodic. What is

lim P [Un + Vn = j ]

in this case?

(3) A particle performs a random walk on the non-negative integers {0, 1, . . .} as follows. At each step,

it moves right with probability p and left with probability q = 1 p, except that, if it is at 0 now,

it moves right with probability p and remains at 0 with probability q. This called a simple random

walk on D = {0, 1, 2, . . .} with a (not instanteneously) reflecting barrier at 0.

Let Xn be the particles position after the nth step. Then, (Xn )n0 is a Markov chain with state space

D.

(a) Write down the transition probabilities Pi,j = P [Xn+1 = j | Xn = i].

(b) Classify the states when p > q.

(c) Classify the states when p < q.

(d) Find the invariant distribution = (i )i0 when p < q.

(e) Let T = min {n 0 : Xn = 0}. Compute E [T | X0 = i] for i 1, when p < q.

Hint. What is the relationship between E [T | X0 = i] and E [T | X0 = 1]?

(4) Let (Xn )n0 be a Markov chain with state space D = {a, b, c, d, e} and transition matrix

0 0 0 0 1

1 0 0 0 0

1

1

P =

/2 /2 0 0 0

1/3 1/3 1/3 0 0

1/4 1/4 1/4 1/4 0

(a) Classify the states.

(b) Compute the invariant distribution = (a , b , c , d , e ).

(c) Let T = min {n 1 : Xn is b or c}. Compute

" T 1

#

X

E

f (Xn ) X0 = a .

n=0

(d) Define the reward function g by

2 if i = a and j = e,

g(i, j) = 3 if i = e and j = a,

0 otherwise.

Compute

n

1X

lim

g(Xk1 , Xk ).

n n

k=1

Each problem is worth 25 points. Write your name at the top right corner of each page you submit.

2

- 1. Solution. a) the Following is the Program of a).Uploaded byjjhong83
- Operation ResearchUploaded byRani Puspita Sari
- f-pqtUploaded bysweetsundari1710
- img 20140710 0001Uploaded byapi-258897284
- Improving_the_accuracy_of_SINR_threshold.pdfUploaded bySarsij Mishra
- Studio conflitto Palestina-Israele con hmmUploaded byFlavio58IT
- slide6mUploaded byIsnan Tow
- midterm.pdfUploaded byuniversedrill
- Applications of Manpower With Various Stages in Business Using Stochastic ModelsUploaded byDr.R.Arumugam
- Transformations of Graphs - SolutionsUploaded bywolfretonmaths
- Matlab RefUploaded byenchanting_dice
- Hidden Markov Model in MATLAB - Stack OverflowUploaded byNamrata Singh
- PC an Introduction to functionsUploaded byHector R.
- 10.1.1.8Uploaded byChaudhry Rehan
- social.pdfUploaded byMutemwa Emmanuel Tafira
- BOUNDING THE SMARANDACHE FUNCTIONUploaded byRyanElias
- Summative in Math (AUGUST 9,2017)Uploaded byFlorecita Cabañog
- relationsfunctionsUploaded byapi-361030628
- A Contribution to the Stochastic Flow Shop Scheduling ProblemUploaded bymoufid
- Numerical analysisUploaded byAnonymous edG2bq
- 1. ALJABAR MATRIKS.pdfUploaded bydika dika
- B.C Kuo Solutions 1Uploaded bysimply_tom
- Flow in Porous Media [Compatibility Mode]Uploaded byElena Proca
- Chapter 10 Sample ProblemsUploaded byMe, Myself and I
- Annotating Search Results From Web DatabasesUploaded byanoop185r
- Rank Numbers for Bent LaddersUploaded byPeter Richter
- Booth MultiplicationUploaded byVijay Dubey
- ijpraiUploaded byjesus1843
- Scarf 2002Uploaded byAnonymous gUySMcpSq
- Graph Theory 6Uploaded byZeeshan Ahmed

- pstat 171Uploaded bylex7396
- Syllabus for FM exam Febuary 2016Uploaded byAndrew Marrens
- Syllabus for FM exam Febuary 2016Uploaded byAndrew Marrens
- Syllabus for FM exam Febuary 2016Uploaded byAndrew Marrens
- godxUploaded byarbi19
- A1 Fundamentals ExamplesUploaded bylex7396
- Data Structures Using Java - Chapter 2Uploaded bylex7396
- FMUploaded byJoeKnittel
- FMUploaded byJoeKnittel
- Bond managementUploaded bysolman
- Bond managementUploaded bysolman
- MCDB 110 Lecture 15 2015Uploaded bylex7396
- pstat 120c 2.17.15Uploaded bylex7396
- Econ 171 Extra Credit.pdfUploaded bylex7396
- A.1. Practice Problems on Fundamentals of ProbabilityUploaded bylex7396
- Final Practice I UCSB Pstat 160AUploaded bylex7396
- UCSB Math34B HomeworkUploaded bylex7396
- Math104B Midterm Key 2010Uploaded bylex7396
- Math104B Midterm Key 2010Uploaded bylex7396
- MATH 360 (Mathematics of Investment)Uploaded byamy ashley sy
- PStat 160 A Homework Solutions (6)Uploaded bylex7396
- Pstat 160A Final Practice II SolutionsUploaded bylex7396

- IMG_0002_NEWUploaded byGalih Prabashinta P. Pramudita
- Harvard StyleUploaded byki_dvm
- Bioclimatism and Vernacular Architecture of North East India.pdfUploaded bysachinharry
- APP Assessment grid for ICTUploaded byAndrew Field
- Novel Nanodiamond Synthesis Company seeks Strategic PartnersUploaded byE. McClarence
- Chism's Five Components of a Teaching StatementUploaded bysweetienasexypa
- Impact of Dimensions of Mobile Banking on User SatisfactionUploaded byIwan Susanto
- Beginner's Guide to Radio CammUploaded byJorgeNitales
- zener drUploaded byArchana Sadanandan
- Strategic PlanUploaded by123shru
- Nchrp ReportUploaded byJay
- 04032016IARJES292.pdfUploaded byGOWTHAM
- MIL-PRF-31013Uploaded byGustavo Castro
- Impacto Sobre Tiendas y Supermercados BicaramangaUploaded byCesar Enrique Moran
- ms ess ss 04 08 14Uploaded byapi-371840203
- MUGI_FSE2002Uploaded byeuelvis1045
- safetynoticeno.2501s00Uploaded byFidel Arellano
- Brief History of MicroelectronicsUploaded byRishi Jha
- HP LaserJet 1000 Service ManualUploaded byrealtoxic
- 602939.pdfUploaded byzhuzai
- Cementing Products CatalogueUploaded byJayesh Sangani
- Mine Resourses and Reserve EvaluationUploaded byedson_dario_2
- Difference Between Metallic and NonUploaded byhabibullah rustam
- Pu Transcript FormUploaded byGurpreet Maan
- Ijso Stage - 02 AnalysisUploaded byRishit Sriwastava
- UNESCO higher ed.pdfUploaded byChester Ruiter
- reconstructionlessonplanspedUploaded byapi-208229156
- 170 Mr.rajesh Rev1 (1)Uploaded byThangaselviSubramanian
- Re GripperUploaded byopenid_9V8YnhpL
- 12. the Binomial Probability DistributionUploaded byRiaan Pretorius