Scribd Logo
Upload

Welcome to Scribd!

  • Stochastic Processes SM PDF

    Uploaded by

    Tomas Barra
    11 views520 pages

    Original Title

    Stochastic Processes SM.pdf

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document

    Copyright:

    © All Rights Reserved
    Download as PDF or read online from Scribd
    Flag for inappropriate content
    11 views520 pages

    Stochastic Processes SM PDF

    Uploaded by

    Tomas Barra

    Copyright:

    © All Rights Reserved
    Download as PDF or read online from Scribd
    Flag for inappropriate content
    of 520
    STOCHASTIC PROCESSES Second Edition Sheldon M. Ross University of California, Berkeley JOHN WILEY & SONS, INC. New York ¢ Chichester * Brisbane * Toronto * Singapore ACQUISITIONS EDITOR Brad Wiley II MARKETING MANAGER Debra Riegert SENIOR PRODUCTION EDITOR Tony VenGraitis MANUFACTURING MANAGER Dorothy Sinclair TEXT AND COVER DESIGN A Good Thing, Inc PRODUCTION COORDINATION Elm Street Publishing Services, Ine “This book was set in Times Roman by Bi-Comp, Inc and printed and bound by Courier/Stoughton The cover was printed by Phoenix Color Recognizing the importance of preserving what has been written, it is a policy of John Wiley & Sons, Inc to have books of enduring value published in the United States printed on acid-free paper, and we exert our best efforts to that end, ‘The paper in this book was manufactured by a mill whose forest management programs include sustained yield harvesting of its timberlands Sustained yield harvesting principles ensure that the number of trees cut each year does not exceed the amount of new growth Copyright © 1996, by John Wiley & Sons, Inc All rights reserved Published simultaneously in Canada Reproduction or translation of any part of this work beyond that permitted by Sections 107 and 108 of the 1976 United States Copyright ‘Act without the permission of the copyright owner is unlawful Requests for permission or further information should be addressed to the Permissions Department, John Wiley & Sons, Inc Library of Congress Cataloging-In-Publication Data: Ross, Sheldon M Stochastic processes/Sheldon M. Ross —2nd ed pcm Includes bibliographical references and index ISBN 0.471-12062.6 (cloth alk paper) 1 Stochastic processes I Title QA274 RES 1996 5192—de20 95-38012 cP Printed in the United States of America 1098765432 On March 30, 1980, a beautiful six-year-old girl died. This book is dedicated to the memory of Nichole Pornaras Preface to the First Edition This text is a nonmeasure theoretic introduction to stochastic processes, and as such assumes a knowledge of calculus and elementary probability. In it we attempt to present some of the theory of stochastic processes, to indicate its diverse range of applications, and also to give the student some probabilistic intuition and insight in thinking about problems We have attempted, wherever possible, to view processes from a probabilistic instead of an analytic point of view. This attempt, for instance, has led us to study most processes from a sample path point of view. I would like to thank Mark Brown, Cyrus Derman, Shun-Chen Niu, Michael Pinedo, and Zvi Schechner for their helpful comments SHELDON M. Ross vii Preface to the Second Edition The second edition of Stochastic Processes includes the following changes: (i) Additional material in Chapter 2 on compound Poisson random vari- ables, including an identity that can be used to efficiently compute moments, and which leads to an elegant recursive equation for the probability mass function of a nonnegative integer valued compound Poisson random variable; (ii) A separate chapter (Chapter 6) on martingales, including sections on the Azuma inequality; and (iii) A new chapter (Chapter 10) on Poisson approximations, including both the Stein-Chen method for bounding the error of these approximations and a method for improving the approximation itself. In addition, we have added numerous exercises and problems throughout the text. Additions to individual chapters follow: In Chapter 1, we have new examples on the probabilistic method, the multivanate normal distribution, random walks on graphs, and the complete match problem Also, we have new sections on probability inequalities (includ- ing Chernoff bounds) and on Bayes estimators (showing that they are almost never unbiased). A proof of the strong law of large numbers is given in the Appendix to this chapter. New examples on patterns and on memoryless optimal coin tossing strate- gies are given in Chapter 3. There is new material in Chapter 4 covering the mean time spent in transient states, as well as examples relating to the Gibb’s sampler, the Metropolis algorithm, and the mean cover time in star graphs. Chapter 5 includes an example on a two-sex population growth model. Chapter 6 has additional examples illustrating the use of the martingale stopping theorem, Chapter 7 includes new material on Spitzer’s identity and using it to compute mean delays in single-server queues with gamma-distnibuted interarrival and service times. Chapter 8 on Brownian motion has been moved to follow the chapter on martingales to allow us to utilize martingales to analyze Brownian motion. x PREFACE TO THE SECOND EDITION Chapter 9 on stochastic order relations now includes a section on associated random variables, as well as new examples utilizing coupling in coupon collect- ing and bin packing problems. ‘We would like to thank all those who were kind enough to write and send comments about the first edition, with particular thanks to He Sheng-wu, Stephen Herschkorn, Robert Kertz, James Matis, Erol Pekoz, Maria Rieders, and Tomasz Rolski for their many helpful comments. SHELDON M. Ross Contents CHAPTER 1. CHAPTER 23 24. oe PRELIMINARIES 1 Probability 1 . Random Variables 7 . Expected Value 9 Moment Generating, Characteristic Functions, and Laplace Transforms 15 . Conditional Expectation 20 1.5.1 Conditional Expectations and Bayes Estimators 33 . The Exponential Distribution, Lack of Memory, and Hazard Rate Functions 35 . Some Probability Inequalities 39 . Limit Theorems 41 ). Stochastic Processes 41 Problems 46 References 55 Appendix 56 . THE POISSON PROCESS 59 21 22. The Poisson Process 59 Interarrival and Waiting Time Distributions 64 Conditional Distribution of the Arrival Times 66 231. The M/GII Busy Period 73 Nonhomogeneous Poisson Process 78 Compound Poisson Random Variables and Processes 82 2.51. A Compound Poisson Identity 84 25.2. Compound Poisson Processes 87 xii 2.6 CHAPTER 3. 3.1 aa 33 34 35 36 ea: 38 CHAPTER 4, 41 42. 43 44 45 46. CONTENTS Conditional Poisson Processes 88 Problems 89 References 97 RENEWAL THEORY 98 Introduction and Preliminaries 98 Distribution of M(t) 99 Some Limit Theorems 101 331 Wald’s Equation — 104 332 Back to Renewal Theory 106 The Key Renewal Theorem and Applications 109 341 Alternating Renewal Processes 114 342 Limiting Mean Fxcess and Expansion of m() 119 343 Age-Dependent Branching Processes. 121 Delayed Renewal Processes 123 Renewal Reward Processes 132 361 A Queueing Application 138 Regenerative Processes 140 3.7.1 The Symmetric Random Walk and the Arc Sine Laws 142 Stationary Point Processes 149 Problems = 153 References 161 MARKOV CHAINS 163 Introduction and Examples 163 Chapman-Kolmogorov Equations and Classification of States 167 Limit Theorems 173 Transitions among Classes, the Gambler's Ruin Problem, and Mean Times in Transient States 185 Branching Processes. 191 Applications of Markov Chains 193 461 A Markov Chain Model of Algorithmic Efficiency 193 462 An Application to Runs—A Markov Chain with a Continuous State Space 195 463 List Ordering Rules—Optimality of the Transposition Rule 198 CONTENTS: Siii 4.7 Time-Reversible Markov Chains 203 48 Semi-Markov Processes 213 Problems 219 References 230 CHAPTER 5. CONTINUOUS-TIME MARKOV CHAINS 231 51 Introduction 231 52. Continuous-Time Markov Chains 231 5.3.-Birth and Death Processes 233 5.4 The Kolmogorov Differential Equations 239 5.4.1 Computing the Transition Probabilities 249 5.5. Limiting Probabilities 251 56. Time Reversibility 257 5.6.1 Tandem Queues 262 5.62 A Stochastic Population Model 263 5.7 Applications of the Reversed Chain to Queueing Theory 270 57.1. Network of Queues 21 $72. The Erlang Loss Formula 275 573 The M/G/I Shared Processor System 278 58 Uniformization 282 Problems 286 References 294 CHAPTER 6. MARTINGALES 295 Introduction 295 61 Martingales 295 62 Stopping Times 298 63. Azuma’s Inequality for Martingales 305 64, Submartingales, Supermartingales, and the Martingale Convergence Theorem 313 65. A Generalized Azuma Inequality 319 Problems 322 References 327 CHAPTER 7. RANDOMWALKS 328 Introduction 328 71. Duality in Random Walks 329 xiv 12 73 74 75 CHAPTER 8. 81 8.2. 83. 84 85 8.6 87. 88 CHAPTER 9. 91 CONTENTS Some Remarks Concerning Exchangeable Random Variables 338 Using Martingales to Analyze Random Walks 341 Applications to G/G/1 Queues and Ruin Problems 344 7.41 The G/G/ Queue — 344 742 A Ruin Problem 347 Blackwell's Theorem on the Line 349 Problems 352 References 355 BROWNIAN MOTION AND OTHER MARKOV PROCESSES 356 Introduction and Preliminaries 356 Hitting Times, Maximum Variable, and Arc Sine Laws 363 Variations on Brownian Motion 366 83.1 Brownian Motion Absorbed at a Value 366 8.3.2 Brownian Motion Reffected at the Origin 368 833 Geometric Brownian Motion 368 83.4 Integrated Brownian Motion 369 Brownian Motion with Drift 372 84.1 Using Martingales to Analyze Brownian Motion 381 Backward and Forward Diffusion Equations 383 Applications of the Kolmogorov Equations to Obtaining Limiting Distributions 385 8.61. Semi-Markov Processes 385, 862. The M/G/1 Queue 388 8.6.3. A Ruin Problem in Risk Theory 392 A Markov Shot Noise Process 393 Stationary Processes 396 Problems = 399 References 403 STOCHASTIC ORDER RELATIONS 404 Introduction 404 Stochastically Larger 404 CONTENTS 92. 93, 94, 95. 9.6 97, CHAPTER 10. 10.1 10.2 103, xv Coupling 409 9.21. Stochastic Monotonicity Properties of Birth and Death Processes 416 9.2.2. Exponential Convergence in Markov Chains 418 Hazard Rate Ordering and Applications to Counting Processes 420 Likelihood Ratio Ordering 428 Stochastically More Variable 433 Applications of Variability Orderings 437 9.6.1. Comparison of G/G/1 Queues 439 9.6.2. A Renewal Process Application 440 963. ABranching Process Application 443 Associated Random Variables 446 Problems 449 References 456. POISSON APPROXIMATIONS 457 Introduction 457 Brun’s Sieve 457 The Stein-Chen Method for Bounding the Error of the Poisson Approximation 462 Improving the Poisson Approximation 467 Problems 470 References 472 ANSWERS AND SOLUTIONS TO SELECTED PROBLEMS 473 INDEX 505 CHAPTER 1 Preliminaries 1.1 PropasiLity A basic notion in probability theory is random experiment an experiment whose outcome cannot be determined in advance. The set of all possible outcomes of an experiment is called the sample space of that experiment, and we denote it by S. ‘An event is a subset of a sample space, and is said to occur if the outcome of the experiment is an element of that subset. We shall suppose that for each event E of the sample space $ a number P(E) is defined and satisfies the following three axioms*: Axiom (1) 0-= P(E) <1. Axiom (2) P(S) = 1 Axiom (3) For any sequence of events Ey, E;,.. that are mutually exclusive, that is, events for which E,E, = when i j (where $s the null set), P (O 8) = z P(E). We refer to P(E) as the probability of the event E. Some simple consequences of axioms (1), (2), and (3) are. Ld. If EC F, then P(E) = P(F). 1.1.2. P(E‘) = 1 — P(E) where E* is the complement of E. 1.1.3. P(Uj E,) = Di P(E,) when the E, are mutually exclusive. 11.4. P(UF £) = Sf PUL). The inequality (1 1 4) is known as Boole’s inequality * Actually P(E) will only be defined for the so-called measurable events of $ But this restriction need not concern us 2 PRELIMINARIES, An important property of the probability function P is that it is continuous. To make this more precise, we need the concept of a limiting event, which we define as follows: A sequence of events {E,,n = 1} issaid tobe an increasing sequence if E, C Ensi, = 1 and is said to be decreasing if E, D Eni,n 2 1. If {E,, n 2 1} is an increasing sequence of events, then we define a new event, denoted by lity. E, by lim E,= UE, when E,C Ey, 221. Similarly if {E,, 1 = 1} is a decreasing sequence, then define lim... E, by limE,=()&, when £,D Ey, 221. We may now state the following: es PROPOSITION 1.1.1 If{E,, n = 1) is either an increasing or decreasing sequence of events, then (time.) Jim P(E,) = Proof Suppose, first, that {E,, n = 1} is an increasing sequence, and define events Fn = Lby AzE, y= (U ) = EE n>l That is, F, consists of those points in E, that are not in any of the earlier E,, i , denote a sequence of events If SPE)<, then Pian infinite number of the £, occur } = 0 Proof The event that an infinite number of the E, occur, called the lim sup E,, can be expressed as aad lim sup E, Aue This follows since if an infinite number of the E, occur, then U;-, E, occurs for each n and thus My. Use, E, occurs On the other hand, if Mai Ul, E, occurs, then Ur, E, occurs for each n, and thus for each n at least one of the £, occurs where i = n, and, hence, an infinite number of the E, occur PROBABILITY .s As Ul, E,,n 2 1, is a decreasing sequence of events, it follows from Proposition 111 that and the result is proven. Exampts 1.1(8) Let Xi, X2, .. be such that PIX, = 0} = Wnt =1- P{X,= 1, n=l If we let E, = {X, = 0}, then, as 2; P(E,) < ©, it follows from the Borel-Cantelli lemma that the probability that X, equals 0 for an infinite number of n is equal to 0. Hence, for all n sufficiently large, X, must equal 1, and so we may conclude that, with probabil- ity 1, lim X, For a converse to the Borel-Cantelli lemma, independence is required. es PROPOSITION 1.1.3 Converse to the Borel-Cantelli Lemma If E\, Ez, are independent events such that then P{an infinite number of the E, occur} 6 PRELIMINARIES Proof Now, TIPE) — (byindependence) = tl (- P(E) sJ]er (bythe inequality 1 ~ x =e) = exp (-3 Pte) =0. since >) P(E,) = @ forall n. Hence the result follows. rr Exampts 1.4(c) Let X;, X:,... be independent and such that P{X, = 0} = Wn =1- P{X,=1,, n=l ft we let E, = {X, = U}, then as Dy, P(E,) = © it follows from Proposition 1.1.3 that E, occurs infinitely often. Also, as 2-1 P(ES) = & it also follows that E; also occurs infinitely often. Hence, with probability 1, X, will equal 0 infinitely often and will also equal | infinitely often. Hence, with probability 1, X, will not approach a limiting value as n — RANDOM VARIABLES 7 1.2 Ranpom VaRIABLES Consider a random experiment having sample space $ A random variable X is a function that assigns a real value to each outcome in S. For any set of real numbers A, the probability that X will assume a value that is contained in the set A is equal to the probability that the outcome of the experiment is contained in X“'(A). That is, P{X € A} = P(X""(A)), where X"!(A) is the event consisting of all points s € S such that X(s) € A. The distribution function F of the random variable X is defined for any real number x by F(x) = P{X < x} = P{X € (—», x}. We shall denote 1 — F(x) by F(x), and so F(x) = P{X > x} A random variable X is said to be discrete if its set of possible values is countable For discrete random variables, F(x) = & PAX = yh fed A random variable is called continuous if there exists a function f(x), called the probability density function, such that Pex isin B= f(x) dx for every set B. Since F(x) = J", f(x) dx, it follows that fe) = LFW). The joint distribution function F of two random variables X and Y is de- fined by F(x, y) = PIX =x, ¥ Sy} The distribution functions of X and Y, Fy) = P(X =x} and Fy(y) = PY

    You might also like