Probability: Theory and Examples 5th

Edition Rick Durrett

Visit to download the full and correct content document:
https://textbookfull.com/product/probability-theory-and-examples-5th-edition-rick-durre
tt/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Probability theory and examples Fifth Edition Durrett

https://textbookfull.com/product/probability-theory-and-examples-
fifth-edition-durrett/

Rick Steves Snapshot Normandy 5th Edition Rick Steves

https://textbookfull.com/product/rick-steves-snapshot-
normandy-5th-edition-rick-steves/

Statistical independence in probability analysis number

theory Kac

https://textbookfull.com/product/statistical-independence-in-
probability-analysis-number-theory-kac/

Theory of stochastic objects probability stochastic

processes and inference 1st Edition Micheas

https://textbookfull.com/product/theory-of-stochastic-objects-
probability-stochastic-processes-and-inference-1st-edition-
micheas/
Theory of stochastic objects: probability, stochastic
processes, and inference First Edition Micheas

https://textbookfull.com/product/theory-of-stochastic-objects-
probability-stochastic-processes-and-inference-first-edition-
micheas/

A Basic Course in Probability Theory 2nd Edition Rabi

Bhattacharya

https://textbookfull.com/product/a-basic-course-in-probability-
theory-2nd-edition-rabi-bhattacharya/

Foundations of the Theory of Probability 2nd Edition

A.N. Kolmogorov

https://textbookfull.com/product/foundations-of-the-theory-of-
probability-2nd-edition-a-n-kolmogorov/

Rick Steves Italy 26th Edition Rick Steves

https://textbookfull.com/product/rick-steves-italy-26th-edition-
rick-steves/

Rick Steves Rome 22nd Edition Rick Steves

https://textbookfull.com/product/rick-steves-rome-22nd-edition-
rick-steves/
 Probability
This lively introduction to measure-theoretic probability theory covers laws of large
numbers, central limit theorems, random walks, martingales, Markov chains, ergodic
theorems, and Brownian motion. Concentrating on the results that are the most useful
for applications, this comprehensive treatment is a rigorous, measure theory–based
graduate text and reference. Operating under the philosophy that the best way to learn
probability is to see it in action, the book contains many extended examples that apply
the theory to concrete applications. Readers learn to recognize when a method works
and, more important, when it does not.
This fifth edition contains a new chapter on multidimensional Brownian motion
and its relationship to PDEs, an advanced topic that is finding new applications.
Setting the foundation for this expansion, Chapter 7 now features a proof of Itôs
formula. Key exercises that previously were simply proofs left to the reader have
been directly inserted into the text as lemmas. The new edition also reinstates discus-
sion about the central limit theorem for martingales and stationary sequences.

R I C K D U R R E T T is a James B. Duke professor in the mathematics department

of Duke University. He received his Ph.D. in Operations Research from Stanford
University in 1976. After nine years at UCLA and twenty-five at Cornell University,
he moved to Duke in 2010. He is the author of 8 books and more than 220 journal
articles on a wide variety of topics, and has supervised more than 45 Ph.D. students.
He is a member of National Academy of Science, American Academy of Arts and
Sciences, and a fellow of the Institute of Mathematical Statistics and of the American
Mathematical Society.
 C A M B R I D G E S E R I E S I N S TAT I S T I C A L A N D
P R O B A B I L I S T I C M AT H E M AT I C S

Editorial Board
Z. Ghahramani (Department of Engineering, University of Cambridge)
R. Gill (Mathematical Institute, Leiden University)
F. P. Kelly (Department of Pure Mathematics and Mathematical Statistics, University of Cambridge)
B. D. Ripley (Department of Statistics, University of Oxford)
S. Ross (Department of Industrial and Systems Engineering,
University of Southern California)
M. Stein (Department of Statistics, University of Chicago)

This series of high-quality upper-division textbooks and expository monographs covers all
aspects of stochastic applicable mathematics. The topics range from pure and applied statistics
to probability theory, operations research, optimization, and mathematical programming. The
books contain clear presentations of new developments in the field and also of the state of
the art in classical methods. While emphasizing rigorous treatment of theoretical methods, the
books also contain applications and discussions of new techniques made possible by advances
in computational practice.

A complete list of books in the series can be found at www.cambridge.org/statistics.

Recent titles include the following:

22. Saddlepoint Approximations with Applications, by Ronald W. Butler
23. Applied Asymptotics, by A. R. Brazzale, A. C. Davison and N. Reid
24. Random Networks for Communication, by Massimo Franceschetti and Ronald Meester
25. Design of Comparative Experiments, by R. A. Bailey
26. Symmetry Studies, by Marlos A. G. Viana
27. Model Selection and Model Averaging, by Gerda Claeskens and Nils Lid Hjort
28. Bayesian Nonparametrics, edited by Nils Lid Hjort et al.
29. From Finite Sample to Asymptotic Methods in Statistics, by Pranab K. Sen,
Julio M. Singer and Antonio C. Pedrosa de Lima
30. Brownian Motion, by Peter Mörters and Yuval Peres
31. Probability (Fourth Edition), by Rick Durrett
33. Stochastic Processes, by Richard F. Bass
34. Regression for Categorical Data, by Gerhard Tutz
35. Exercises in Probability (Second Edition), by Loı̈c Chaumont and Marc Yor
36. Statistical Principles for the Design of Experiments, by R. Mead, S. G. Gilmour and
A. Mead
37. Quantum Stochastics, by Mou-Hsiung Chang
38. Nonparametric Estimation under Shape Constraints, by Piet Groeneboom and
Geurt Jongbloed
39. Large Sample Covariance Matrices and High-Dimensional Data Analysis, by
Jianfeng Yao, Shurong Zheng and Zhidong Bai
40. Mathematical Foundations of Infinite-Dimensional Statistical Models, by Evarist Giné
and Richard Nickl
41. Confidence, Likelihood, Probability, by Tore Schweder and Nils Lid Hjort
42. Probability on Trees and Networks, by Russell Lyons and Yuval Peres
43. Random Graphs and Complex Networks (Volume 1), by Remco van der Hofstad
44. Fundamentals of Nonparametric Bayesian Inference, by Subhashis Ghosal and Aad van
der Vaart
45. Long-Range Dependence and Self-Similarity, by Vladas Pipiras and Murad S. Taqqu
46. Predictive Statistics, by Bertrand S. Clarke and Jennifer L. Clarke
47. High-Dimensional Probability, by Roman Vershynin
48. High-Dimensional Statistics, by Martin J. Wainwright
49. Probability: Theory and Examples (Fifth Edition), by Rick Durrett
 Probability
Theory and Examples

FIFTH EDITION

Rick Durrett
Duke University
 University Printing House, Cambridge CB2 8BS, United Kingdom

One Liberty Plaza, 20th Floor, New York, NY 10006, USA

477 Williamstown Road, Port Melbourne, VIC 3207, Australia

314–321, 3rd Floor, Plot 3, Splendor Forum, Jasola District Centre, New Delhi – 110025, India

79 Anson Road, #06–04/06, Singapore 079906

Cambridge University Press is part of the University of Cambridge.

It furthers the University’s mission by disseminating knowledge in the pursuit of

education, learning, and research at the highest international levels of excellence.

www.cambridge.org
Information on this title: www.cambridge.org/9781108473682
DOI: 10.1017/9781108591034

© Rick Durrett 2019

This publication is in copyright. Subject to statutory exception

and to the provisions of relevant collective licensing agreements,
no reproduction of any part may take place without the written
permission of Cambridge University Press.

First published 2019

Printed in the United Kingdom by TJ International Ltd., Padstow, Cornwall

A catalogue record for this publication is available from the British Library.

Library of Congress Cataloging-in-Publication Data

Names: Durrett, Richard, 1951– author.
Title: Probability : theory and examples / Rick Durrett (Duke University,
Durham, North Carolina).
Description: Fifth edition. | Cambridge ; New York, NY : Cambridge University
Press, 2019. | Series: Cambridge series in statistical and probabilistic
mathematics ; 49 | Includes bibliographical references and index.
Identifiers: LCCN 2018047195 | ISBN 9781108473682 (hardback : alk. paper)
Subjects: LCSH: Probabilities. | Probabilities–Textbooks.
Classification: LCC QA273 .D865 2019 | DDC 519.2–dc23
LC record available at https://lccn.loc.gov/2018047195

ISBN 978-1-108-47368-2 Hardback

Cambridge University Press has no responsibility for the persistence or accuracy

of URLs for external or third-party internet websites referred to in this publication
and does not guarantee that any content on such websites is, or will remain,
accurate or appropriate.
 Contents

Preface page xi

1 Measure Theory 1
1.1 Probability Spaces 1
1.2 Distributions 8
1.3 Random Variables 13
1.4 Integration 15
1.5 Properties of the Integral 21
1.6 Expected Value 25
1.6.1 Inequalities 25
1.6.2 Integration to the Limit 26
1.6.3 Computing Expected Values 28
1.7 Product Measures, Fubini’s Theorem 33
2 Laws of Large Numbers 37
2.1 Independence 37
2.1.1 Sufficient Conditions for Independence 39
2.1.2 Independence, Distribution, and Expectation 41
2.1.3 Sums of Independent Random Variables 43
2.1.4 Constructing Independent Random Variables 45
2.2 Weak Laws of Large Numbers 48
2.2.1 L2 Weak Laws 48
2.2.2 Triangular Arrays 51
2.2.3 Truncation 53
2.3 Borel-Cantelli Lemmas 58
2.4 Strong Law of Large Numbers 65
2.5 Convergence of Random Series* 69
2.5.1 Rates of Convergence 75
2.5.2 Infinite Mean 76
2.6 Renewal Theory* 78
2.7 Large Deviations* 90
3 Central Limit Theorems 98
3.1 The De Moivre-Laplace Theorem 98
3.2 Weak Convergence 100

vii
viii Contents

3.2.1 Examples 100

3.2.2 Theory 102
3.3 Characteristic Functions 108
3.3.1 Definition, Inversion Formula 108
3.3.2 Weak Convergence 114
3.3.3 Moments and Derivatives 116
3.3.4 Polya’s Criterion* 119
3.3.5 The Moment Problem* 121
3.4 Central Limit Theorems 125
3.4.1 i.i.d. Sequences 125
3.4.2 Triangular Arrays 128
3.4.3 Prime Divisors (Erdös-Kac)* 132
3.4.4 Rates of Convergence (Berry-Esseen)* 136
3.5 Local Limit Theorems* 140
3.6 Poisson Convergence 145
3.6.1 The Basic Limit Theorem 145
3.6.2 Two Examples with Dependence 149
3.7 Poisson Processes 151
3.7.1 Compound Poisson Processes 154
3.7.2 Thinning 155
3.7.3 Conditioning 157
3.8 Stable Laws* 159
3.9 Infinitely Divisible Distributions* 168
3.10 Limit Theorems in Rd 171
4 Martingales 178
4.1 Conditional Expectation 178
4.1.1 Examples 180
4.1.2 Properties 182
4.1.3 Regular Conditional Probabilities* 185
4.2 Martingales, Almost Sure Convergence 188
4.3 Examples 194
4.3.1 Bounded Increments 194
4.3.2 Polya’s Urn Scheme 196
4.3.3 Radon-Nikodym Derivatives 197
4.3.4 Branching Processes 200
4.4 Doob’s Inequality, Convergence in Lp , p > 1 203
4.5 Square Integrable Martingales* 208
4.6 Uniform Integrability, Convergence in L1 211
4.7 Backwards Martingales 216
4.8 Optional Stopping Theorems 221
4.8.1 Applications to Random Walks 223
4.9 Combinatorics of Simple Random Walk* 227
5 Markov Chains 232
5.1 Examples 232
5.2 Construction, Markov Properties 235
 Contents ix

5.3 Recurrence and Transience 243

5.4 Recurrence of Random Walks Stararred Section 248
5.5 Stationary Measures 259
5.6 Asymptotic Behavior 268
5.7 Periodicity, Tail σ -Field* 274
5.8 General State Space* 278
5.8.1 Recurrence and Transience 281
5.8.2 Stationary Measures 281
5.8.3 Convergence Theorem 282
5.8.4 GI/G/1 Queue 283
6 Ergodic Theorems 286
6.1 Definitions and Examples 286
6.2 Birkhoff’s Ergodic Theorem 289
6.3 Recurrence 293
6.4 A Subadditive Ergodic Theorem 296
6.5 Applications 300
7 Brownian Motion 305
7.1 Definition and Construction 305
7.2 Markov Property, Blumenthal’s 0-1 Law 311
7.3 Stopping Times, Strong Markov Property 316
7.4 Path Properties 320
7.4.1 Zeros of Brownian Motion 320
7.4.2 Hitting Times 321
7.5 Martingales 325
7.6 Itô’s Formula* 328
8 Applications to Random Walk 336
8.1 Donsker’s Theorem 336
8.2 CLTs for Martingales 342
8.3 CLTs for Stationary Sequences 347
8.3.1 Mixing Properties 351
8.4 Empirical Distributions, Brownian Bridge 354
8.5 Laws of the Iterated Logarithm 360
9 Multidimensional Brownian Motion 364
9.1 Martingales 364
9.2 Heat Equation 366
9.3 Inhomogeneous Heat Equation 368
9.4 Feynman-Kac Formula 370
9.5 Dirichlet Problem 373
9.5.1 Exit Distributions 377
9.6 Green’s Functions and Potential Kernels 379
9.7 Poisson’s Equation 382
9.7.1 Occupation Times 385
9.8 Schrödinger Equation 387
x Contents

Appendix A Measure Theory Details 394

A.1 Carathéodory’s Extension Theorem 394
A.2 Which Sets Are Measurable? 399
A.3 Kolmogorov’s Extension Theorem 402
A.4 Radon-Nikodym Theorem 403
A.5 Differentiating under the Integral 407
References 410
Index 415
 Preface

Some times the lights are shining on me. Other times I can barely see.
Lately it occurs to me what a long strange trip its been. (Grateful Dead)

In 1989, when the first edition of the book was completed, my sons David and Greg were
3 and 1, and the cover picture showed the Dow Jones at 2650. The last 29 years have
brought many changes, but the song remains the same: “The title of the book indicates
that as we develop the theory, we will focus our attention on examples. Hoping that the
book would be a useful reference for people who apply probability in their work, we have
tried to emphasize the results that are important for applications, and illustrated their use
with roughly 200 examples. Probability is not a spectator sport, so the book contains almost
450 exercises to challenge the reader and to deepen their understanding.”
The fifth edition has a number of changes:

• The exercises have been moved to the end of the section. The Examples, Theorems, and
Lemmas are now numbered in one sequence to make it easier to find things.
• There is a new chapter on multidimensional Brownian motion and its relationship to PDEs.
To make this possible, a proof of Itô’s formula has been added to Chapter 7.
• The lengthy Brownian motion chapter has been split into two, with the second one focus-
ing on Donsker’s theorem, etc. The material on the central limit theorem for martingales
and stationary sequences deleted from the fourth edition has been reinstated.
• The four sections of the random walk chapter have been relocated. Stopping times have
been moved to the martingale chapter; recurrence of random walks and the arcsine laws
to the Markov chain chapter; renewal theory has been moved to Chapter 2.
• Some of the exercises that were simply proofs left to the reader have been put into the text
as lemmas. There are a few new exercises.

Typos The fourth edition contains a list of the people who made corrections to the first three
editions. With apologies to those whose contributions I lost track of, this time I need to thank:
Richard Arratia, Benson Au, Swee Hong Chan, Conrado Costa, Nate Eldredge, Steve Evans,
Jason Farnon, Christina Goldschmidt, Eduardo Hota, Martin Hildebrand, Shlomo Leven-
thal, Jan Lieke, Kyle MacDonald, Ron Peled, Jonathan Peterson, Erfan Salavati, Byron
Schmuland, Timo Seppalainen, Antonio Carlos de Azevedo Sodre, Shouda Wang, and Ruth
Williams. I must confess that Christophe Leuridan pointed one out that I have not corrected.
Lemma 3.4.19 incorrectly asserts that the distributions in its statement have mean 0, but their
means do not exist. The conclusion remains valid, since they are differentiable at 0. A sixth

xi
xii Preface

edition is extremely unlikely, but you can e-mail me about typos and I will post them on my
website.
Family update As the fourth edition was being completed, David had recently graduated
from Ithaca College and Greg was in his last semester at MIT applying to graduate school
in computer science. Now, eight years later, Greg has graduated from Berkeley University,
and is an assistant professor in the Computer Science department at the University of Texas
in Austin. Greg works in the field of machine learning, specifically natural language pro-
cessing. No, I don’t know what that means, but it seems to pay well. David got his degree in
journalism. After an extensive job search process and some freelance work, David has settled
into a steady job working for a company that produces newsletters for athletic directors and
trainers.
In the summer of 2010, Susan and I moved to Durham. Since many people think that the
move was about the weather, I will mention that during our first summer it was 104 degrees
(and humid!) three days in a row. Yes, it almost never snows here, but when it does, three
inches of snow (typically mixed with ice) will shut down the whole town for four days. It
took some time for us to adjust to the Durham/Chapel Hill area, which has about 10 times
as many people as Ithaca and is criss-crossed by freeways, but we live in a nice quiet
neighborhood near the campus. Susan enjoys volunteering at the Sarah P. Duke gardens and
listening to their talks about the plants of North Carolina and future plans for the gardens.
As I write this, it is the last week before school starts.
 1

Measure Theory

In this chapter, we will recall some definitions and results from measure theory. Our purpose
here is to provide an introduction to readers who have not seen these concepts before and
to review that material for those who have. Harder proofs, especially those that do not
contribute much to one’s intuition, are hidden away in the Appendix. Readers with a solid
background in measure theory can skip Sections 1.4, 1.5, and 1.7, which were previously
part of the Appendix.

1.1 Probability Spaces

Here and throughout the book, terms being defined are set in boldface. We begin with the
most basic quantity. A probability space is a triple (, F ,P ), where  is a set of “out-
comes,” F is a set of “events,” and P : F → [0,1] is a function that assigns probabilities
to events. We assume that F is a σ -field (or σ -algebra), i.e., a (nonempty) collection of
subsets of  that satisfy
(i) if A ∈ F , then Ac ∈ F , and
(ii) if Ai ∈ F is a countable sequence of sets, then ∪i Ai ∈ F .
Here and in what follows, countable means finite or countably infinite. Since ∩i Ai =
(∪i Aci )c , it follows that a σ -field is closed under countable intersections. We omit the last
property from the definition to make it easier to check.
Without P , (, F ) is called a measurable space, i.e., it is a space on which we can put
a measure. A measure is a nonnegative countably additive set function; that is, a function
μ : F → R with
(i) μ(A) ≥ μ(∅) = 0 for all A ∈ F , and
(ii) if Ai ∈ F is a countable sequence of disjoint sets, then

μ(∪i Ai ) = μ(Ai )
i

If μ() = 1, we call μ a probability measure. In this book, probability measures are

usually denoted by P .
The next result gives some consequences of the definition of a measure that we will need
later. In all cases, we assume that the sets we mention are in F .

1
2 Measure Theory

Theorem 1.1.1 Let μ be a measure on (, F )

(i) monotonicity. If A ⊂ B, then μ(A) ≤ μ(B).
∞
(ii) subadditivity. If A ⊂ ∪∞
m=1 Am , then μ(A) ≤ m=1 μ(Am ).

(iii) continuity from below. If Ai ↑ A (i.e., A1 ⊂ A2 ⊂ . . . and ∪i Ai = A), then

μ(Ai ) ↑ μ(A).
(iv) continuity from above. If Ai ↓ A (i.e., A1 ⊃ A2 ⊃ . . . and ∩i Ai = A), with
μ(A1 ) < ∞, then μ(Ai ) ↓ μ(A).
Proof (i) Let B − A = B ∩ Ac be the difference of the two sets. Using + to denote disjoint
union, B = A + (B − A) so
μ(B) = μ(A) + μ(B − A) ≥ μ(A).
(ii) Let An = An ∩A, B1 = A1 and for n > 1, Bn = An −∪m=1
n−1 
Am . Since the Bn are disjoint
and have union A, we have using (ii) of the definition of measure, Bm ⊂ Am , and (i) of this
theorem
∞ ∞

μ(A) = μ(Bm ) ≤ μ(Am )
m=1 m=1

(iii) Let Bn = An −An−1 . Then the Bn are disjoint and have ∪∞

m=1 Bm = A, ∪m=1 Bm = An so
n

∞
 
n
μ(A) = μ(Bm ) = lim μ(Bm ) = lim μ(An )
n→∞ n→∞
m=1 m=1

(iv) A1 − An ↑ A1 − A so (iii) implies μ(A1 − An ) ↑ μ(A1 − A). Since A1 ⊃ A, we have

μ(A1 − A) = μ(A1 ) − μ(A) and it follows that μ(An ) ↓ μ(A).
The simplest setting, which should be familiar from undergraduate probability, is:
Example 1.1.2 (Discrete probability spaces) Let  = a countable set, i.e., finite or count-
ably infinite. Let F = the set of all subsets of . Let
 
P (A) = p(ω), where p(ω) ≥ 0 and p(ω) = 1
ω∈A ω∈

A little thought reveals that this is the most general probability measure on this space.
In many cases when  is a finite set, we have p(ω) = 1/||, where || = the number
of points in .
For a simple concrete example that requires this level of generality, consider the astragali,
dice used in ancient Egypt made from the ankle bones of sheep. This die could come to rest
on the top side of the bone for four points or on the bottom for three points. The side of the
bone was slightly rounded. The die could come to rest on a flat and narrow piece for six
points or somewhere on the rest of the side for one point. There is no reason to think that all
four outcomes are equally likely, so we need probabilities p1 , p3 , p4 , and p6 to describe P .
To prepare for our next definition, we need to note that it follows easily from the
definition: If Fi , i ∈ I are σ -fields, then ∩i∈I Fi is. Here I = ∅ is an arbitrary index set
 1.1 Probability Spaces 3

(i.e., possibly uncountable). From this it follows that if we are given a set  and a collection
A of subsets of , then there is a smallest σ -field containing A. We will call this the σ -field
generated by A and denote it by σ (A).
Let Rd be the set of vectors (x1, . . . xd ) of real numbers and Rd be the Borel sets, the
smallest σ -field containing the open sets. When d = 1, we drop the superscript.
Example 1.1.3 (Measures on the real line) Measures on (R, R) are defined by giving a
Stieltjes measure function with the following properties:
(i) F is nondecreasing.
(ii) F is right continuous, i.e., limy↓x F (y) = F (x).
Theorem 1.1.4 Associated with each Stieltjes measure function F there is a unique measure
μ on (R, R) with μ((a,b]) = F (b) − F (a)

μ((a,b]) = F (b) − F (a) (1.1.1)

When F (x) = x, the resulting measure is called Lebesgue measure.

The proof of Theorem 1.1.4 is a long and winding road, so we will content ourselves with
describing the main ideas involved in this section and hiding the remaining details in the
Appendix in Section A.1. The choice of “closed on the right” in (a,b] is dictated by the fact
that if bn ↓ b, then we have
∩n (a,bn ] = (a,b]

The next definition will explain the choice of “open on the left.”
A collection S of sets is said to be a semialgebra if (i) it is closed under intersection,
i.e., S, T ∈ S implies S ∩ T ∈ S , and (ii) if S ∈ S , then S c is a finite disjoint union of sets
in S . An important example of a semialgebra is
Example 1.1.5 Sd = the empty set plus all sets of the form

(a1,b1 ] × · · · × (ad ,bd ] ⊂ Rd where − ∞ ≤ ai < bi ≤ ∞

The definition in (1.1.1) gives the values of μ on the semialgebra S1 . To go from semial-
gebra to σ -algebra, we use an intermediate step. A collection A of subsets of  is called an
algebra (or field) if A,B ∈ A implies Ac and A ∪ B are in A. Since A ∩ B = (Ac ∪ B c )c ,
it follows that A ∩ B ∈ A. Obviously, a σ -algebra is an algebra. An example in which the
converse is false is:
Example 1.1.6 Let  = Z = the integers. A = the collection of A ⊂ Z so that A or Ac is
finite is an algebra.
Lemma 1.1.7 If S is a semialgebra, then S̄ = {finite disjoint unions of sets in S } is an
algebra, called the algebra generated by S .
Proof Suppose A = +i Si and B = +j Tj , where + denotes disjoint union and we assume
the index sets are finite. Then A ∩ B = +i,j Si ∩ Tj ∈ S̄ . As for complements, if A = +i Si
then Ac = ∩i Sic . The definition of S implies Sic ∈ S̄ . We have shown that S̄ is closed under
intersection, so it follows by induction that Ac ∈ S̄ .
4 Measure Theory

Example 1.1.8 Let  = R and S = S1 then S̄1 = the empty set plus all sets of the form

∪ki=1 (ai ,bi ] where − ∞ ≤ ai < bi ≤ ∞

Given a set function μ on S we can extend it to S̄ by

  n
μ +ni=1 Ai = μ(Ai )
i=1

By a measure on an algebra A, we mean a set function μ with

(i) μ(A) ≥ μ(∅) = 0 for all A ∈ A, and
(ii) if Ai ∈ A are disjoint and their union is in A, then
∞
  
μ ∪∞
i=1 A i = μ(Ai )
i=1

μ is said to be σ -finite if there is a sequence of sets An ∈ A so that μ(An ) < ∞ and

∪n An = . Letting A1 = A1 and for n ≥ 2,
 
An = ∪nm=1 Am or An = An ∩ ∩m=1 Am ∈ A
n−1 c

we can without loss of generality assume that An ↑  or the An are disjoint.

The next result helps us to extend a measure defined on a semialgebra S to the σ -algebra
it generates, σ (S )
Theorem 1.1.9 Let S be a semialgebra and let μ defined on S have μ(∅) = 0. Suppose
(i) if S ∈ S , is a finite disjoint union of sets
 Si ∈ S , then μ(S) = i μ(Si ), and (ii) if
Si ,S ∈ S with S = +i≥1 Si , then μ(S) ≤ i≥1 μ(Si ). Then μ has a unique extension μ̄
that is a measure on S̄ , the algebra generated by S . If μ̄ is sigma-finite, then there is a
unique extension ν that is a measure on σ (S ).
In (ii) above, and in what follows, i ≥ 1 indicates a countable union, while a plain subscript
i or j indicates a finite union. The proof of Theorems 1.1.9 is rather involved so it is given
in Section A.1. To check condition (ii) in the theorem the following is useful.
Lemma 1.1.10 Suppose only that (i) holds. 
(a) If A,Bi ∈ S̄ with A = +ni=1 Bi , then μ̄(A) = i μ̄(Bi ).

(b) If A,Bi ∈ S̄ with A ⊂ ∪ni=1 Bi , then μ̄(A) ≤ i μ̄(Bi ).
Proof Observe that it follows from the definition that if A = +i Bi is a finite disjoint union
of sets in S̄ and Bi = +j Si,j , then
 
μ̄(A) = μ(Si,j ) = μ̄(Bi )
i,j i

To prove (b), we begin with the case n = 1, B1 = B. B = A + (B ∩ Ac ) and B ∩ Ac ∈ S̄ , so

μ̄(A) ≤ μ̄(A) + μ̄(B ∩ Ac ) = μ̄(B)

 1.1 Probability Spaces 5

To handle n > 1 now, let Fk = B1c ∩ . . . ∩ Bk−1

c ∩ Bk and note

∪i Bi = F1 + · · · + Fn
A = A ∩ (∪i Bi ) = (A ∩ F1 ) + · · · + (A ∩ Fn )
so using (a), (b) with n = 1, and (a) again

n 
n
μ̄(A) = μ̄(A ∩ Fk ) ≤ μ̄(Fk ) = μ̄ (∪i Bi )
k=1 k=1

Proof of Theorem 1.1.4. Let S be the semialgebra of half-open intervals (a,b] with
−∞ ≤ a < b ≤ ∞. To define μ on S , we begin by observing that
F (∞) = lim F (x) and F (−∞) = lim F (x) exist
x↑∞ x↓−∞

and μ((a,b]) = F (b) − F (a) makes sense for all −∞ ≤ a < b ≤ ∞ since F (∞) > −∞
and F (−∞) < ∞.
If (a,b] = +ni=1 (ai ,bi ], then after relabeling the intervals we must have a1 = a, bn = b,
and ai = bi−1 for 2 ≤ i ≤ n, so condition (i) in Theorem 1.1.9 holds. To check (ii), suppose
first that −∞ < a < b < ∞, and (a,b] ⊂ ∪i≥1 (ai ,bi ] where (without loss of generality)
−∞ < ai < bi < ∞. Pick δ > 0 so that F (a + δ) < F (a) +  and pick ηi so that
F (bi + ηi ) < F (bi ) + 2−i
The open intervals (ai ,bi + ηi ) cover [a + δ,b], so there is a finite subcover (αj ,βj ),
1 ≤ j ≤ J . Since (a + δ,b] ⊂ ∪Jj=1 (αj ,βj ], (b) in Lemma 1.1.10 implies


J ∞

F (b) − F (a + δ) ≤ F (βj ) − F (αj ) ≤ (F (bi + ηi ) − F (ai ))
j =1 i=1

So, by the choice of δ and ηi ,

∞

F (b) − F (a) ≤ 2 + (F (bi ) − F (ai ))
i=1

and since  is arbitrary, we have proved the result in the case −∞ < a < b < ∞.
To remove the last restriction, observe that if (a,b] ⊂ ∪i (ai ,bi ] and (A,B] ⊂ (a,b] has
−∞ < A < B < ∞, then we have
∞

F (B) − F (A) ≤ (F (bi ) − F (ai ))
i=1

Since the last result holds for any finite (A,B] ⊂ (a,b], the desired result follows.
Measures on Rd
Our next goal is to prove a version of Theorem 1.1.4 for Rd . The first step is to introduce
the assumptions on the defining function F . By analogy with the case d = 1 it is natural to
assume:
6 Measure Theory

0 2/3 1

0 0 2/3

0 0 0

Figure 1.1 Picture of the counterexample.

(i) It is nondecreasing, i.e., if x ≤ y (meaning xi ≤ yi for all i), then F (x) ≤ F (y).
(ii) F is right continuous, i.e., limy↓x F (y) = F (x) (here y ↓ x means each yi ↓ xi ).
(iii) If xn ↓ −∞, i.e., each coordinate does, then F (xn ) ↓ 0. If xn ↑ −∞, i.e., each
coordinate does, then F (xn ) ↑ 1.
However, this time it is not enough. Consider the following F
⎧
⎪
⎪ 1 if x1,x2 ≥ 1
⎪
⎪
⎨2/3 if x ≥ 1 and 0 ≤ x < 1
1 2
F (x1,x2 ) =
⎪
⎪ 2/3 if x ≥ 1 and 0 ≤ x1 <1
⎪
⎪
2
⎩0 otherwise
See Figure 1.1 for a picture. A little thought shows that
μ((a1,b1 ] × (a2,b2 ]) = μ((−∞,b1 ] × (−∞,b2 ]) − μ((−∞,a1 ] × (−∞,b2 ])
− μ((−∞,b1 ] × (−∞,a2 ]) + μ((−∞,a1 ] × (−∞,a2 ])
= F (b1,b2 ) − F (a1,b2 ) − F (b1,a2 ) + F (a1,a2 )
Using this with a1 = a2 = 1 −  and b1 = b2 = 1 and letting  → 0 we see that
μ({1,1}) = 1 − 2/3 − 2/3 + 0 = −1/3
Similar reasoning shows that μ({1,0}) = μ({0,1}) = 2/3.
To formulate the third and final condition for F to define a measure, let
A = (a1,b1 ] × · · · × (ad ,bd ]
V = {a1,b1 } × · · · × {ad ,bd }
where −∞ < ai < bi < ∞. To emphasize that ∞’s are not allowed, we will call A a finite
rectangle. Then V = the vertices of the rectangle A. If v ∈ V , let

sgn (v) = (−1)# of a’s in v


AF = sgn (v)F (v)
v∈V
 1.1 Probability Spaces 7

We will let μ(A) = AF , so we must assume

(iv) AF ≥ 0 for all rectangles A.
Theorem 1.1.11 Suppose F : Rd → [0,1] satisfies (i)–(iv) given above. Then there is a
unique probability measure μ on (Rd , Rd ) so that μ(A) = A F for all finite rectangles.
d
Example 1.1.12 Suppose F (x) = i=1 Fi (x), where the Fi satisfy (i) and (ii) of
Theorem 1.1.4. In this case,
d
AF = (Fi (bi ) − Fi (ai ))
i=1

When Fi (x) = x for all i, the resulting measure is Lebesgue measure on Rd .

Proof We let μ(A) = A F for all finite rectangles and then use monotonicity to extend the
definition to Sd . To check (i) of Theorem 1.1.9, call A = +k Bk a regular subdivision of A if
there are sequences ai = αi,0 < αi,1 . . . < αi,ni = bi so that each rectangle Bk has the form
(α1,j1 −1,α1,j1 ] × · · · × (αd,jd −1,αd,jd ] where 1 ≤ ji ≤ ni

It is easy to see that for regular subdivisions λ(A) = k λ(Bk ). (First consider the case in
which all the endpoints are finite and then take limits to get the general case.) To extend this
result to a general finite subdivision A = +j Aj , subdivide further to get a regular one.
The proof of (ii) is almost identical to that in Theorem 1.1.4. To make things easier to
write and to bring out the analogies with Theorem 1.1.4, we let
(x,y) = (x1,y1 ) × · · · × (xd ,yd )
(x,y] = (x1,y1 ] × · · · × (xd ,yd ]
[x,y] = [x1,y1 ] × · · · × [xd ,yd ]
for x,y ∈ Rd . Suppose first that −∞ < a < b < ∞, where the inequalities mean that each
component is finite, and suppose (a,b] ⊂ ∪i≥1 (a i ,bi ], where (without loss of generality)
−∞ < a i < bi < ∞. Let 1̄ = (1, . . . ,1), pick δ > 0 so that
μ((a − δ 1̄,b]) > μ((a,b]) − 

Figure 1.2 Conversion of a subdivision to a regular one.

8 Measure Theory

and pick ηi so that

μ((a,bi + ηi 1̄]) < μ((a i ,bi ]) + 2−i
The open rectangles (a i ,bi + ηi 1̄) cover [a + δ 1̄,b], so there is a finite subcover (α j ,β j ),
1 ≤ j ≤ J . Since (a + δ 1̄,b] ⊂ ∪Jj=1 (α j ,β j ], (b) in Lemma 1.1.10 implies


J ∞

μ([a + δ 1̄,b]) ≤ μ((α ,β ]) ≤
j j
μ((a i ,bi + ηi 1̄])
j =1 i=1

So, by the choice of δ and ηi ,

∞

μ((a,b]) ≤ 2 + μ((a i ,bi ])
i=1
and since  is arbitrary, we have proved the result in the case −∞ < a < b < ∞. The proof
can now be completed exactly as before.

Exercises
1.1.1 Let  = R, F = all subsets so that A or Ac is countable, P (A) = 0 in the first case
and = 1 in the second. Show that (, F ,P ) is a probability space.
1.1.2 Recall the definition of Sd from Example 1.1.5. Show that σ (Sd ) = Rd , the Borel
subsets of Rd .
1.1.3 A σ -field F is said to be countably generated if there is a countable collection
C ⊂ F so that σ (C ) = F . Show that Rd is countably generated.
1.1.4 (i) Show that if F1 ⊂ F2 ⊂ . . . are σ -algebras, then ∪i Fi is an algebra. (ii) Give an
example to show that ∪i Fi need not be a σ -algebra.
1.1.5 A set A ⊂ {1,2, . . .} is said to have asymptotic density θ if
lim |A ∩ {1,2, . . . ,n}|/n = θ
n→∞
Let A be the collection of sets for which the asymptotic density exists. Is A a
σ -algebra? an algebra?

1.2 Distributions
Probability spaces become a little more interesting when we define random variables on
them. A real-valued function X defined on  is said to be a random variable if for every
Borel set B ⊂ R we have X −1 (B) = {ω : X(ω) ∈ B} ∈ F . When we need to emphasize the
σ -field, we will say that X is F -measurable or write X ∈ F . If  is a discrete probability
space (see Example 1.1.2), then any function X :  → R is a random variable. A second
trivial, but useful, type of example of a random variable is the indicator function of a set
A ∈ F: 
1 ω∈A
1A (ω) =
0 ω ∈ A
 1.2 Distributions 9

(,F,P ) (R,R) μ = P ◦ X −1

X

- A

X−1 (A)

Figure 1.3 Definition of the distribution of X.

The notation is supposed to remind you that this function is 1 on A. Analysts call this
object the characteristic function of A. In probability, that term is used for something quite
different. (See Section 3.3.)
If X is a random variable, then X induces a probability measure on R called its
distribution by setting μ(A) = P (X ∈ A) for Borel sets A. Using the notation introduced
previously, the right-hand side can be written as P (X −1 (A)). In words, we pull A ∈ R back
to X −1 (A) ∈ F and then take P of that set.
To check that μ is a probability measure we observe that if the Ai are disjoint, then using
the definition of μ; the fact that X lands in the union if and only if it lands in one of the Ai ;
the fact that if the sets Ai ∈ R are disjoint, then the events {X ∈ Ai } are disjoint; and the
definition of μ again; we have:
 
μ (∪i Ai ) = P (X ∈ ∪i Ai ) = P (∪i {X ∈ Ai }) = P (X ∈ Ai ) = μ(Ai )
i i
The distribution of a random variable X is usually described by giving its distribution
function, F (x) = P (X ≤ x).
Theorem 1.2.1 Any distribution function F has the following properties:
(i) F is nondecreasing.
(ii) limx→∞ F (x) = 1, limx→−∞ F (x) = 0.
(iii) F is right continuous, i.e., limy↓x F (y) = F (x).
(iv) If F (x−) = limy↑x F (y), then F (x−) = P (X < x).
(v) P (X = x) = F (x) − F (x−).
Proof To prove (i), note that if x ≤ y, then {X ≤ x} ⊂ {X ≤ y}, and then use (i) in
Theorem 1.1.1 to conclude that P (X ≤ x) ≤ P (X ≤ y).
To prove (ii), we observe that if x ↑ ∞, then {X ≤ x} ↑ , while if x ↓ −∞, then
{X ≤ x} ↓ ∅ and then use (iii) and (iv) of Theorem 1.1.1.
To prove (iii), we observe that if y ↓ x, then {X ≤ y} ↓ {X ≤ x}.
To prove (iv), we observe that if y ↑ x, then {X ≤ y} ↑ {X < x}.
For (v), note P (X = x) = P (X ≤ x) − P (X < x) and use (iii) and (iv).
The next result shows that we have found more than enough properties to characterize
distribution functions.
10 Measure Theory

y 


x
 


F −1 (x) F −1 (y)

Figure 1.4 Picture of the inverse defined in the proof of Theorem 1.2.2.

Theorem 1.2.2 If F satisfies (i), (ii), and (iii) in Theorem 1.2.1, then it is the distribution
function of some random variable.
Proof Let  = (0,1), F = the Borel sets, and P = Lebesgue measure. If ω ∈ (0,1), let
X(ω) = sup{y : F (y) < ω}
Once we show that
() {ω : X(ω) ≤ x} = {ω : ω ≤ F (x)}
the desired result follows immediately since P (ω : ω ≤ F (x)) = F (x). (Recall P is
Lebesgue measure.) To check (), we observe that if ω ≤ F (x), then X(ω) ≤ x, since
x∈ / {y : F (y) < ω}. On the other hand if ω > F (x), then since F is right continuous, there
is an  > 0 so that F (x + ) < ω and X(ω) ≥ x +  > x.
Even though F may not be 1-1 and onto we will call X the inverse of F and denote it
by F −1 . The scheme in the proof of Theorem 1.2.2 is useful in generating random variables
on a computer. Standard algorithms generate random variables U with a uniform distribu-
tion, then one applies the inverse of the distribution function defined in Theorem 1.2.2 to get
a random variable F −1 (U ) with distribution function F .
If X and Y induce the same distribution μ on (R, R), we say X and Y are equal in dis-
tribution. In view of Theorem 1.1.4, this holds if and only if X and Y have the same
distribution function, i.e., P (X ≤ x) = P (Y ≤ x) for all x. When X and Y have the
same distribution, we like to write
d
X=Y
but this is too tall to use in text, so for typographical reasons we will also use X =d Y .
When the distribution function F (x) = P (X ≤ x) has the form
 x
F (x) = f (y) dy (1.2.1)
−∞
we say that X has density function f . In remembering formulas, it is often useful to think
of f (x) as being P (X = x) although
 x+
P (X = x) = lim f (y) dy = 0
→0 x−

By popular demand, we have ceased our previous practice of writing P (X = x) for the
density function. Instead we will use things like the lovely and informative fX (x).
 1.2 Distributions 11

We can start with f and use (1.2.1) to define a distribution function F . In order
 to end up
with a distribution function, it is necessary and sufficient that f (x) ≥ 0 and f (x) dx = 1.
Three examples that will be important in what follows are:
Example 1.2.3 (Uniform distribution on (0,1)) f (x) = 1 for x ∈ (0,1) and 0 otherwise.
Distribution function:
⎧
⎪
⎨0 x ≤ 0
F (x) = x 0 ≤ x ≤ 1
⎪
⎩
1 x>1
Example 1.2.4 (Exponential distribution with rate λ) f (x) = λe−λx for x ≥ 0 and 0
otherwise. Distribution function:

0 x≤0
F (x) = −λx
1−e x≥0
Example 1.2.5 (Standard normal distribution)
f (x) = (2π )−1/2 exp(−x 2 /2)
In this case, there is no closed form expression for F (x), but we have the following bounds
that are useful for large x:
Theorem 1.2.6 For x > 0,
 ∞
(x −1 − x −3 ) exp(−x 2 /2) ≤ exp(−y 2 /2)dy ≤ x −1 exp(−x 2 /2)
x

Proof Changing variables y = x + z and using exp(−z2 /2) ≤ 1 gives

 ∞  ∞
exp(−y /2) dy ≤ exp(−x /2)
2 2
exp(−xz) dz = x −1 exp(−x 2 /2)
x 0
For the other direction, we observe
 ∞
(1 − 3y −4 ) exp(−y 2 /2) dy = (x −1 − x −3 ) exp(−x 2 /2)
x

A distribution function on R is said to be absolutely continuous if it has a density, and

singular if the corresponding measure is singular w.r.t. Lebesgue measure. See Section A.4
for more on these notions. An example of a singular distribution is:
Example 1.2.7 (Uniform distribution on the Cantor set) The Cantor set C is defined by
removing (1/3,2/3) from [0,1] and then removing the middle third of each interval that
remains. We define an associated distribution function by setting F (x) = 0 for x ≤ 0,
F (x) = 1 for x ≥ 1, F (x) = 1/2 for x ∈ [1/3,2/3], F (x) = 1/4 for x ∈ [1/9,2/9],
F (x) = 3/4 for x ∈ [7/9,8/9],... Then extend F to all of [0,1] using monotonicity. There
is no f for which (1.2.1) holds because such an f would be equal to 0 on a set of measure 1.
From the definition, it is immediate that the corresponding measure has μ(C c ) = 0.
A probability measure P (or its associated distribution function) is said to be discrete if
there is a countable set S with P (S c ) = 0. The simplest example of a discrete distribution is
12 Measure Theory
-
-
-
-
-
-
-
-
0 1

Figure 1.5 Cantor distribution function.

Example 1.2.8 (Point mass at 0) F (x) = 1 for x ≥ 0, F (x) = 0 for x < 0.

In Section 1.6, we will see the Bernoulli, Poisson, and geometric distributions. The next
example shows that the distribution function associated with a discrete probability measure
can be quite wild.
Example 1.2.9 (Dense
 discontinuities) Let q1 , q2 . . . be an enumeration of the rationals.
Let αi > 0 have ∞i=1 1 = 1 and let
α
∞

F (x) = αi 1[qi ,∞)
i=1
where 1[θ,∞) (x) = 1 if x ∈ [θ,∞) = 0 otherwise.

Exercises
1.2.1 Suppose X and Y are random variables on (, F ,P ) and let A ∈ F . Show that if
we let Z(ω) = X(ω) for ω ∈ A and Z(ω) = Y (ω) for ω ∈ Ac , then Z is a random
variable.
1.2.2 Let χ have the standard normal distribution. Use Theorem 1.2.6 to get upper and
lower bounds on P (χ ≥ 4).
1.2.3 Show that a distribution function has at most countably many discontinuities.
1.2.4 Show that if F (x) = P (X ≤ x) is continuous, then Y = F (X) has a uniform
distribution on (0,1), that is, if y ∈ [0,1], P (Y ≤ y) = y.
1.2.5 Suppose X has continuous density f , P (α ≤ X ≤ β) = 1 and g is a function
that is strictly increasing and differentiable on (α,β). Then g(X) has density
f (g −1 (y))/g  (g −1 (y)) for y ∈ (g(α),g(β)) and 0 otherwise. When g(x) = ax + b
with a > 0, g −1 (y) = (y − b)/a so the answer is (1/a)f ((y − b)/a).
1.2.6 Suppose X has a normal distribution. Use the previous exercise to compute the density
of exp(X). (The answer is called the lognormal distribution.)
1.2.7 (i) Suppose X has density function f . Compute the distribution function of X2 and
then differentiate to find its density function. (ii) Work out the answer when X has a
standard normal distribution to find the density of the chi-square distribution.
 1.3 Random Variables 13

1.3 Random Variables

In this section, we will develop some results that will help us later to prove that quantities
we define are random variables, i.e., they are measurable. Since most of what we have to
say is true for random elements of an arbitrary measurable space (S, S ) and the proofs are
the same (sometimes easier), we will develop our results in that generality. First we need a
definition. A function X :  → S is said to be a measurable map from (, F ) to (S, S ) if
X −1 (B) ≡ {ω : X(ω) ∈ B} ∈ F for all B ∈ S
If (S, S ) = (Rd , Rd ) and d > 1, then X is called a random vector. Of course, if d = 1, X
is called a random variable, or r.v. for short.
The next result is useful for proving that maps are measurable.
Theorem 1.3.1 If {ω : X(ω) ∈ A} ∈ F for all A ∈ A and A generates S (i.e., S is the
smallest σ -field that contains A), then X is measurable.
Proof Writing {X ∈ B} as shorthand for {ω : X(ω) ∈ B}, we have
{X ∈ ∪i Bi } = ∪i {X ∈ Bi }
{X ∈ B c } = {X ∈ B}c
So the class of sets B = {B : {X ∈ B} ∈ F } is a σ -field. Since B ⊃ A and A generates S ,
B ⊃ S.
It follows from the two equations displayed in the previous proof that if S is a σ -field, then
{{X ∈ B} : B ∈ S } is a σ -field. It is the smallest σ -field on  that makes X a measurable
map. It is called the σ -field generated by X and denoted σ (X). For future reference we
note that
σ (X) = {{X ∈ B} : B ∈ S } (1.3.1)
Example 1.3.2 If (S, S ) = (R, R), then possible choices of A in Theorem 1.3.1 are
{(−∞,x] : x ∈ R} or {(−∞,x) : x ∈ Q} where Q = the rationals.
Example 1.3.3 If (S, S ) = (Rd , Rd ), a useful choice of A is
{(a1,b1 ) × · · · × (ad ,bd ) : −∞ < ai < bi < ∞}
or occasionally the larger collection of open sets.
Theorem 1.3.4 If X : (, F ) → (S, S ) and f : (S, S ) → (T , T ) are measurable maps,
then f (X) is a measurable map from (, F ) to (T , T )
Proof Let B ∈ T . {ω : f (X(ω)) ∈ B} = {ω : X(ω) ∈ f −1 (B)} ∈ F , since by assumption
f −1 (B) ∈ S .
From Theorem 1.3.4, it follows immediately that if X is a random variable, then so is cX
for all c ∈ R, X 2 , sin(X), etc. The next result shows why we wanted to prove Theorem 1.3.4
for measurable maps.
Theorem 1.3.5 If X1, . . . Xn are random variables and f : (Rn, Rn ) → (R, R) is
measurable, then f (X1, . . . ,Xn ) is a random variable.
14 Measure Theory

Proof In view of Theorem 1.3.4, it suffices to show that (X1, . . . ,Xn ) is a random vector.
To do this, we observe that if A1, . . . ,An are Borel sets, then
{(X1, . . . ,Xn ) ∈ A1 × · · · × An } = ∩i {Xi ∈ Ai } ∈ F
Since sets of the form A1 × · · · × An generate Rn , the desired result follows from
Theorem 1.3.1.
Theorem 1.3.6 If X1, . . . ,Xn are random variables, then X1 + · · · + Xn is a random
variable.
Proof In view of Theorem 1.3.5 it suffices to show that f (x1, . . . ,xn ) = x1 + · · · + xn is
measurable. To do this, we use Example 1.3.2 and note that {x : x1 + · · · + xn < a} is an
open set and hence is in Rn .
Theorem 1.3.7 If X1,X2, . . . are random variables then so are
inf Xn sup Xn lim sup Xn lim inf Xn
n n n n

Proof Since the infimum of a sequence is < a if and only if some term is < a (if all terms
are ≥ a then the infimum is), we have
{inf Xn < a} = ∪n {Xn < a} ∈ F
n
A similar argument shows {supn Xn > a} = ∪n {Xn > a} ∈ F . For the last two, we observe
 
lim inf Xn = sup inf Xm
n→∞ n m≥n
 
lim sup Xn = inf sup Xm
n→∞ n m≥n

To complete the proof in the first case, note that Yn = infm≥n Xm is a random variable for
each n so supn Yn is as well.
From Theorem 1.3.7, we see that
o ≡ {ω : lim Xn exists } = {ω : lim sup Xn − lim inf Xn = 0}
n→∞ n→∞ n→∞

is a measurable set. (Here ≡ indicates that the first equality is a definition.) If P (o ) = 1, we
say that Xn converges almost surely, or a.s. for short. This type of convergence is called
almost everywhere in measure theory. To have a limit defined on the whole space, it is
convenient to let
X∞ = lim sup Xn
n→∞
but this random variable may take the value +∞ or −∞. To accommodate this and some
other headaches, we will generalize the definition of random variable.
A function whose domain is a set D ∈ F and whose range is R∗ ≡ [−∞,∞] is said to
be a random variable if for all B ∈ R∗ we have X −1 (B) = {ω : X(ω) ∈ B} ∈ F . Here
R∗ = the Borel subsets of R∗ with R∗ given the usual topology, i.e., the one generated by
intervals of the form [−∞,a), (a,b) and (b,∞] where a,b ∈ R. The reader should note that
the extended real line (R∗, R∗ ) is a measurable space, so all the results above generalize
immediately.
 1.4 Integration 15

Exercises
1.3.1 Show that if A generates S , then X −1 (A) ≡ {{X ∈ A} : A ∈ A} generates σ (X) =
{{X ∈ B} : B ∈ S }.
1.3.2 Prove Theorem 1.3.6 when n = 2 by checking {X1 + X2 < x} ∈ F .
1.3.3 Show that if f is continuous and Xn → X almost surely then f (Xn ) → f (X) almost
surely.
1.3.4 (i) Show that a continuous function from Rd → R is a measurable map from (Rd , Rd )
to (R, R). (ii) Show that Rd is the smallest σ -field that makes all the continuous
functions measurable.
1.3.5 A function f is said to be lower semicontinuous or l.s.c. if

lim inf f (y) ≥ f (x)

y→x

and upper semicontinuous (u.s.c.) if −f is l.s.c. Show that f is l.s.c. if and only if
{x : f (x) ≤ a} is closed for each a ∈ R and conclude that semicontinuous functions
are measurable.
1.3.6 Let f : Rd → R be an arbitrary function and let f δ (x) = sup{f (y) : |y − x| < δ}
and fδ (x) = inf{f (y) : |y − x| < δ} where |z| = (z12 + · · · + zd2 )1/2 . Show that f δ is
l.s.c. and fδ is u.s.c. Let f 0 = limδ↓0 f δ , f0 = limδ↓0 fδ , and conclude that the set of
points at which f is discontinuous = {f 0 = f0 } is measurable.
1.3.7 A function ϕ :  → R is said to be simple if

n
ϕ(ω) = cm 1Am (ω)
m=1

where the cm are real numbers and Am ∈ F . Show that the class of F measurable
functions is the smallest class containing the simple functions and closed under point-
wise limits.
1.3.8 Use the previous exercise to conclude that Y is measurable with respect to σ (X) if
and only if Y = f (X) where f : R → R is measurable.
1.3.9 To get a constructive proof of the last result, note that {ω : m2−n ≤ Y <
(m + 1)2−n } = {X ∈ Bm,n } for some Bm,n ∈ R and set fn (x) = m2−n for
x ∈ Bm,n and show that as n → ∞ fn (x) → f (x) and Y = f (X).

1.4 Integration
Let μ be a σ -finite measure on (, F ). We will be primarily interested in the special case
μ is a probability measure, but we will sometimes need to integrate with respect to infinite
measure and it is no harder to develop
 the results in general.
In this section we will define f dμ for a class of measurable functions. This is a four-
step procedure:
16 Measure Theory

1. Simple functions
2. Bounded functions
3. Nonnegative functions
4. General functions
This sequence of four steps is also useful in proving integration formulas. See, for example,
the proofs of Theorems 1.6.9 and 1.7.2.

Step 1. ϕ is said to be a simple function if ϕ(ω) = ni=1 ai 1Ai and Ai are disjoint sets with
μ(Ai ) < ∞. If ϕ is a simple function, we let
 n
ϕ dμ = ai μ(Ai )
i=1

The representation of ϕ is not unique since we have not supposed that the ai are distinct.
However, it is easy to see that the last definition does not contradict itself.
We will prove the next three conclusions four times, but before we can state them for the
first time, we need a definition. ϕ ≥ ψ μ-almost everywhere (or ϕ ≥ ψ μ-a.e.) means
μ({ω : ϕ(ω) < ψ(ω)}) = 0. When there is no doubt about what measure we are referring
to, we drop the μ.
Lemma 1.4.1 Let ϕ and  ψ be simple functions.
(i) If ϕ ≥ 0 a.e. then ϕ dμ ≥ 0.
 any a ∈ R,  aϕ dμ = a ϕ dμ.
(ii) For
(iii) ϕ + ψ dμ = ϕ dμ + ψ dμ.
Proof (i) and (ii) are immediate consequences of the definition. To prove (iii), suppose

m 
n
ϕ= ai 1Ai and ψ= bj 1Bj
i=1 j =1

To make the supports of the two functions the same, we let A0 = ∪i Bi − ∪i Ai , let
B0 = ∪i Ai − ∪i Bi , and let a0 = b0 = 0. Now

m 
n
ϕ+ψ = (ai + bj )1(Ai ∩Bj )
i=0 j =0

and the Ai ∩ Bj are pairwise disjoint, so

 m  n
(ϕ + ψ) dμ = (ai + bj )μ(Ai ∩ Bj )
i=0 j =0
m  n 
n 
m
= ai μ(Ai ∩ Bj ) + bj μ(Ai ∩ Bj )
i=0 j =0 j =0 i=0
m 
n  
= ai μ(Ai ) + bj μ(Bj ) = ϕ dμ + ψ dμ
i=0 j =0

In the next-to-last step, we used Ai = +j (Ai ∩Bj ) and Bj = +i (Ai ∩Bj ), where + denotes
a disjoint union.
Another random document with
no related content on Scribd:
 “Brother Frank, I must tell you, and when I do, you will not believe
me—It was not our father whom we both saw this morning.”
“It was no other whom I saw. What do you mean? Do you suppose
that I do not know my own father?”
“I tell you it was not, and could not be. I had an express from him
yesterday. He is two hundred miles from this, and cannot be in
Scotland sooner than three weeks hence.”
“You astonish me, Thomas. This is beyond human
comprehension.”
“It is true—that I avouch, and the certainty of it has sickened me at
heart. You must be aware that he came not home last night, and that
his horse and retinue have not arrived.”
“He was not at home, it is true, nor have his horse and retinue
arrived in Scotland. Still there is no denying that our father is here,
and that it was he who spoke to and admonished me.”
“I tell you it is impossible. A spirit has spoken to us in our father’s
likeness, for he is not, and cannot be, in Scotland at this time. My
faculties are altogether confounded by the event, not being able to
calculate on the qualities or condition of our monitor. An evil spirit it
certainly could not be, for all its admonitions pointed to good. I
sorely dread, Francis, that our father is no more: that there has been
another engagement, that he has lost his life, and that his soul has
been lingering around his family before taking its final leave of this
sphere. I believe that our father is dead; and for my part I am so sick
at heart, that my nerves are all unstrung. Pray, do you take horse and
post off for Salop, from whence his commission to me yesterday was
dated, and see what hath happened to our revered father.”
“I cannot, for my life, give credit to this, brother, or that it was any
other being but my father himself who rebuked me. Pray allow me to
tarry another day at least before I set out. Perhaps our father may
appear in the neighbourhood, and may be concealing himself for
some secret purpose. Did you tell him of our quarrel?”
“No. He never asked me concerning it, but charged me sharply
with my intent on the first word, and adjured me, by my regard for
his blessing, and my hope of heaven, to desist from my purpose.”
“Then he knew it all intuitively; for when I first went in view of the
spot appointed for our meeting, I perceived him walking sharply to
and fro, wrapped in his military cloak. He never so much as deigned
to look at me, till I came close to his side, and thinking it was
yourself, I fell to upbraiding him, and desired him to draw. He then
threw off his cloak, drew his sword, and, telling me he came in your
place, dared me to the encounter. But he knew all the grounds of our
quarrel minutely, and laid the blame on me. I own I am a little
puzzled to reconcile circumstances, but am convinced my father is
near at hand. I heard his words, and saw his eyes flashing anger and
indignation. Unfortunately, I did not touch him, which would have
put an end to all doubts; for he did not present the hand of
reconciliation to me, as I expected he would have done, on my
yielding implicitly to all his injunctions.”
The two brothers then parted, with protestations of mutual
forbearance in all time coming, and with an understanding, as that
was the morning of Saturday, that if their father, or some word of
him, did not reach home before the next evening, the Tutor of
Cassway was to take horse for the county of Salop early on Monday
morning.
Thomas, being thus once more left to himself, could do nothing
but toss and tumble in his bed, and reflect on the extraordinary
occurrence of that morning; and, after many troubled cogitations, it
at length occurred to his recollection what Mrs Jane Jerdan had said
to him:—“Do it, then. Do it with a vengeance!—But remember this,
that wherever ye set the place of combat, be it in hill or dale, deep
linn or moss hag, I shall have a thirdsman there to encourage you on.
I shall give you a meeting you little wot of.”
If he was confounded before, he was ten times more so at the
remembrance of these words of most ominous import.
At the time he totally disregarded them, taking them for mere
rhodomontade; but now the idea was to him terrible, that his father’s
spirit, like the prophet’s of old, should have been conjured up by
witchcraft; and then again he bethought himself that no witch would
have employed her power to prevent evil. In the end he knew not
what to think, and so, taking the hammer from its rest, he gave three
raps on the pipe drum (for there were no bells in the towers of those
days), and up came John Burgess, Thomas Beattie’s henchman,
huntsman, and groom of the chambers, one who had been attached
to the family for fifty years, and he says, in his slow west-border
tongue, “How’s thou now, callan’?—Is thou ony better-lins? There
has been tway stags seen in the Bloodhope-Linns this morning
already.”
“Ay, and there has been something else seen, John, that lies nearer
to my heart to-day.” John looked at his master with an inquisitive
eye and quivering lip, but said nothing. The latter went on: “I am
very unwell to-day, John, and cannot tell what is the matter with me.
I think I am bewitched.”
“It’s very like thou is, callan’. I pits nae doubt on’t at a’.”
“Is there anybody in this moor district whom you ever heard
blamed for the horrible crime of witchcraft?”
“Ay, that there is; mair than ane or tway. There’s our neighbour,
Lucky Jerdan, for instance, and her niece Nell,—the warst o’ the pair,
I doubt.” John said this with a sly stupid leer, for he had admitted the
old lady to an audience with his master the day before, and had eyed
him afterwards bending his course towards Drumfielding.
“John, I am not disposed to jest at this time; for I am disturbed in
mind, and very ill. Tell me, in reality, did you ever hear Mrs Jane
Jerdan accused of being a witch?”
“Why, look thee, master, I dare nae say she’s a witch; for Lucky has
mony good points in her character. But it’s weel kenned she has mair
power nor her ain, for she can stop a’ the plews in Eskdale wi’ a wave
o’ her hand, and can raise the dead out o’ their graves, just as a
matter of coorse.”
“That, John, is an extraordinary power indeed. But did you never
hear of her sending any living men to their graves? For as that is
rather the danger that hangs over me, I wish you would take a ride
over and desire Mrs Jane to come and see me. Tell her I am ill, and
request her to come and see me.”
“I shall do that, callan’. But are thou sure it is the auld witch I’m to
bring? For it strikes me the young ane maybe has done the deed; and
if sae, she is the fittest to effect the cure. But I shall bring the auld
ane.—Dinna flee intil a rage, for I shall bring the auld ane; though,
gude forgie me! it is unco like bringing the houdie.”
Away went John Burgess to Drumfielding; but Mrs Jane would not
move for all his entreaties. She sent back word to his master, to “rise
out o’ his bed, for he wad be waur if ony thing ailed him; and if he
had aught to say to auld Jane Jerdan, she would be ready to hear it at
hame, though he behoved to remember that it wasna ilka subject
under the sun that she could thole to be questioned anent.”
With this answer John was forced to return, and there being no
accounts of old Beattie having been seen in Scotland, the young men
remained all the Sabbath-day in the utmost consternation at the
apparition of their father they had seen, and the appalling rebuke
they had received from it. The most incredulous mind could scarce
doubt that they had had communion with a supernatural being; and
not being able to draw any other conclusion themselves, they became
persuaded that their father was dead; and accordingly, both prepared
for setting out early on Monday morning toward the county of Salop,
from whence they had last heard of him.
But just as they were ready to set out, when their spurs were
buckled on and their horses bridled, Andrew Johnston, their father’s
confidential servant, arrived from the place to which they were
bound. He had ridden night and day, never once stinting the light
gallop, as he said, and had changed his horse seven times. He
appeared as if his ideas were in a state of derangement and
confusion; and when he saw his young masters standing together,
and ready-mounted for a journey, he stared at them as if he scarcely
believed his own senses. They of course asked immediately about the
cause of his express; but his answers were equivocal, and he
appeared not to be able to assign any motive. They asked him
concerning their father, and if anything extraordinary had happened
to him. He would not say either that there had, or that there had not;
but inquired, in his turn, if nothing extraordinary had happened with
them at home. They looked to one another, and returned him no
answer; but at length the youngest said, “Why, Andrew, you profess
to have ridden express for the distance of two hundred miles; now
you surely must have some guess for what purpose you have done
this? Say, then, at once, what your message is: Is our father alive?”
“Ye—es; I think he is.”
“You think he is? Are you uncertain, then?”
“I am certain he is not dead,—at least, was not when I left him. But
—hum—certainly there has a change taken place. Hark ye, masters—
can a man be said to be in life when he is out of himself?”
 “Why, man, keep us not in this thrilling suspense. Is our father
well?”
“No—not quite well. I am sorry to say, honest gentlemen, that he is
not. But the truth is, my masters, now that I see you well and hearty,
and about to take a journey in company, I begin to suspect that I
have been posted all this way on a fool’s errand; and not another
syllable will I speak on the subject, till I have some refreshment, and
if you still insist on hearing a ridiculous story, you will hear it then.”
When the matter of the refreshment had been got over to Andrew’s
full satisfaction, he began as follows:—
“Why, faith, you see, my masters, it is not easy to say my errand to
you, for in fact I have none. Therefore, all that I can do is to tell you a
story—a most ridiculous one it is, as ever sent a poor fellow out on
the gallop for the matter of two hundred miles or so. On the morning
before last, right early, little Isaac, the page, comes to me, and he
says,—‘Johnston, thou must go and visit master. He’s bad.’”
“Bad!” says I, “Whatever way is he bad?”
“‘Why,’ says he, ‘he’s so far ill as he’s not well, and desires to see
you without one moment’s delay. He’s in fine taking, and that you’ll
find; but what for do I stand here? Lord, I never got such a fright.
Why, Johnston, does thou know that master hath lost himself?’
“‘How lost himself, rabbit?’ says I; ‘speak plain out, else I’ll have
thee lug-hauled, thou dwarf!’ for my blood rose at the imp, for
fooling at any mishap of my master’s. But my choler only made him
worse, for there is not a greater diel’s-buckie in all the Five Dales.
“‘Why, man, it is true that I said,’ quoth he, laughing; ‘the old gurly
squire hath lost himself; and it will be grand sport to see thee going
calling him at all the stane-crosses in the kingdom, in this here way.
—Ho, yes! and a two times ho, yes! and a three times ho, yes! Did
anybody no see the better half of my master, Laird of the twa
Cassway’s, Bloodhope, and Pentland, which was amissing overnight,
and is supposed to have gone a-woolgathering? If anybody hath seen
that better part of my master, whilk contains as much wit as a man
could drive on a hurlbarrow, let them restore it to me, Andrew
Johnston, piper, trumpeter, whacker, and wheedler, to the same
great and noble squire; and high shall be his reward. Ho, yes!’
 “‘The deuce restore thee to thy right mind!’ said I, knocking him
down, and leaving him sprawling in the kennel, and then hasted to
my master, whom I found feverish, restless, and raving, and yet with
an earnestness in his demeanour that stunned and terrified me. He
seized my hand in both his, which were burning like fire, and gave
me such a look of despair as I shall never forget. ‘Johnston, I am ill,’
said he, ‘grievously ill, and know not what is to become of me. Every
nerve in my body is in a burning heat, and my soul is as it were torn
to fritters with amazement. Johnston, as sure as you are in the body,
something most deplorable hath happened to them.’
“‘Yes, as sure as I am in the body, there has, master,’ says I. ‘But I’ll
have you bled and doctored in style, and you shall soon be as sound
as a roach,’ says I, ‘for a gentleman must not lose heart altogether for
a little fire-raising in his outworks, if it does not reach the citadel,’
says I to him. But he cut me short by shaking his head and flinging
my hand from him.
“‘A truce with your talking,’ says he. ‘That which hath befallen me
is as much above your comprehension as the sun is above the earth,
and never will be comprehended by mortal man; but I must inform
you of it, as I have no other means of gaining the intelligence I yearn
for, and which I am incapable of gaining personally. Johnston, there
never was a mortal man suffered what I have suffered since
midnight. I believe I have had doings with hell; for I have been
disembodied, and embodied again, and the intensity of my tortures
has been unparalleled.—I was at home this morning at daybreak.’
“‘At home at Cassway!’ says I. ‘I am sorry to hear you say so,
master, because you know, or should know, that the thing is
impossible, you being in the ancient town of Shrewsbury on the
king’s business.’
“‘I was at home in very deed, Andrew,’ returned he; ‘but whether in
the body or out of the body, I cannot tell—the Lord only knoweth.
But there I was in this guise, and with this heart and all its feelings
within me, where I saw scenes, heard words, and spoke others, which
I will here relate to you. I had finished my despatches last night by
midnight, and was sitting musing on the hard fate and improvidence
of my sovereign master, when, ere ever I was aware, a neighbour of
ours, Mrs Jane Jerdan, of Drumfielding, a mysterious character, with
whom I have had some strange doings in my time, came suddenly
into the chamber, and stood before me. I accosted her with doubt
and terror, asking what had brought her so far from home.’
“‘You are not so far from home as you imagine,’ said she; ‘and it is
fortunate for some that it is so. Your two sons have quarrelled about
the possession of niece Ellen, and though the eldest is blameless of
the quarrel, yet has he been forced into it, and they are engaged to
fight at daybreak at the Crook of Glendearg. There they will assuredly
fall by each other’s hands, if you interpose not; for there is no other
authority now on earth that can prevent this woful calamity.’
“‘Alas! how can I interfere,’ said I, ‘at a distance? It is already
within a few hours of the meeting, and before I get from among the
windings of the Severn, their swords will be bathed in each other’s
blood! I must trust to the interference of Heaven.’
“‘Is your name and influence, then, to perish for ever?’ said she. ‘Is
it so soon to follow your master’s, the great Maxwell of the Dales,
into utter oblivion? Why not rather rouse into requisition the
energies of the spirits that watch over human destinies? At least step
aside with me, that I may disclose the scene to your eyes. You know I
can do it; and you may then act according to your natural impulse.’
“Such was the import of the words she spoke to me, if not the very
words themselves. I understood them not at the time; nor do I yet.
But when she had done speaking, she took me by the hand, and
hurried me towards the door of the apartment, which she opened,
and the first step we took over the threshold, we stepped into a void
space and fell downward. I was going to call out, but felt my descent
so rapid, that my voice was stifled, and I could not so much as draw
my breath. I expected every moment to fall against something, and
be dashed to pieces; and I shut my eyes, clenched my teeth, and held
by the dame’s hand with a frenzied grasp, in expectation of the
catastrophe. But down we went—down and down, with a celerity
which tongue cannot describe, without light, breath, or any sort of
impediment. I now felt assured that we had both at once stepped
from off the earth, and were hurled into the immeasurable void. The
airs of darkness sung in my ears with a booming din as I rolled down
the steeps of everlasting night, an outcast from nature and all its
harmonies, and a journeyer into the depths of hell.
“I still held my companion’s hand, and felt the pressure of hers;
and so long did this our alarming descent continue, that I at length
caught myself breathing once more, but as quick as if I had been in
the height of a fever. I then tried every effort to speak, but they were
all unavailing; for I could not emit one sound, although my lips and
tongue fashioned the words. Think, then, of my astonishment, when
my companion sung out the following stanza with the greatest glee:—
‘Here we roll,
Body and soul,
Down to the deeps of the Paynim’s goal—
With speed and with spell,
With yo and with yell,
This is the way to the palace of hell—
Sing yo! ho!
Level and low,
Down to the Valley of Vision we go!’

“‘Ha, ha, ha! Tam Beattie,’ added she, ‘where is a’ your courage
now? Cannot ye lift up your voice and sing a stave wi’ your auld
crony? And cannot ye lift up your een, and see what region you are in
now?’
“I did force open my eyelids, and beheld light, and apparently
worlds, or huge lurid substances, gliding by me with speed beyond
that of the lightning of heaven. I certainly perceived light, though of a
dim, uncertain nature; but so precipitate was my descent, I could not
distinguish from whence it proceeded, or of what it consisted,
whether of the vapours of chaotic wastes, or the streamers of hell. So
I again shut my eyes closer than ever, and waited the event in terror
unutterable.
“We at length came upon something which interrupted our farther
progress. I had no feeling as we fell against it, but merely as if we
came in contact with some soft substance that impeded our descent;
and immediately afterwards I perceived that our motion had ceased.
“‘What a terrible tumble we hae gotten, Laird!’ said my
companion. ‘But ye are now in the place where you should be; and
deil speed the coward!’
“So saying, she quitted my hand, and I felt as if she were wrested
from me by a third object; but still I durst not open my eyes, being
convinced that I was lying in the depths of hell, or some hideous
place not to be dreamt of; so I lay still in despair, not even daring to
address a prayer to my Maker. At length I lifted my eyes slowly and
fearfully; but they had no power of distinguishing objects. All that I
perceived was a vision of something in nature, with which I had in
life been too well acquainted. It was a glimpse of green glens, long
withdrawing ridges, and one high hill, with a cairn on its summit. I
rubbed my eyes to divest them of the enchantment, but when I
opened them again, the illusion was still brighter and more
magnificent. Then springing to my feet, I perceived that I was lying
in a little fairy ring, not one hundred yards from the door of my own
hall!
“I was, as you may well conceive, dazzled with admiration; still I
felt that something was not right with me, and that I was struggling
with an enchantment; but recollecting the hideous story told me by
the beldame, of the deadly discord between my two sons, I hasted to
watch their motions, for the morning was yet but dawning. In a few
seconds after recovering my senses, I perceived my eldest son
Thomas leave his tower armed, and pass on towards the place of
appointment. I waylaid him, and remarked to him that he was very
early astir, and I feared on no good intent. He made no answer, but
stood like one in a stupor, and gazed at me. ‘I know your purpose,
son Thomas,’ said I; ‘so it is in vain for you to equivocate. You have
challenged your brother, and are going to meet him in deadly
combat; but as you value your father’s blessing, and would deprecate
his curse—as you value your hope of heaven, and would escape the
punishment of hell—abandon the hideous and cursed intent, and be
reconciled to your only brother.’
“On this, my dutiful son Thomas kneeled to me, and presented his
sword, disclaiming at the same time all intentions of taking away his
brother’s life, and all animosity for the vengeance sought against
himself, and thanked me in a flood of tears for my interference. I
then commanded him back to his couch, and taking his cloak and
sword, hasted away to the Crook of Glendearg, to wait the arrival of
his brother.”
Here Andrew Johnston’s narrative detailed the selfsame
circumstances recorded in a former part of this tale, as having passed
between the father and his younger son, so that it is needless to
recapitulate them; but beginning where that broke off, he added, in
the words of the old laird: “As soon as my son Francis had left me, in
order to be reconciled to his brother, I returned to the fairy knowe
and ring, where I first found myself seated at daybreak. I know not
why I went there, for though I considered with myself, I could
discover no motive that I had for doing so, but was led thither by a
sort of impulse which I could not resist, and from the same feeling
spread my son’s mantle on the spot, laid his sword beside it, and
stretched me down to sleep. I remember nothing farther with any
degree of accuracy, for I instantly fell into a chaos of suffering,
confusion, and racking dismay, from which I was only of late
released by awaking from a trance on the very seat, and in the same
guise in which I was the evening before. I am certain I was at home
in body or in spirit—saw my sons—spake these words to them, and
heard theirs in return. How I returned I know even less, if that is
possible, than how I went; for it seemed to me that the mysterious
force that presses us to this sphere, and supports us on it, was in my
case withdrawn or subverted, and that I merely fell from one part of
the earth’s surface and alighted on another. Now I am so ill that I
cannot move from this couch; therefore, Andrew, do you mount and
ride straight home. Spare no horseflesh, by night or by day, to bring
me word of my family, for I dread that some evil hath befallen them.
If you find them in life, give them many charges from me of brotherly
love and affection; if not—what can I say, but, in the words of the
patriarch, if I am bereaved of my children, I am bereaved.”
The two brothers, in utter amazement, went together to the green
ring on the top of the knoll above the castle of Cassway, and there
found the mantle lying spread, and the sword beside it. They then,
without letting Johnston into the awful secret, mounted straight, and
rode off with him to their father. They found him still in bed, and
very ill; and though rejoiced at seeing them, they soon lost hope of
his recovery, his spirits being broken and deranged in a wonderful
manner. Their conversations together were of the most solemn
nature, the visitation deigned to them having been above their
capacity. On the third or fourth day, their father was removed by
death from this terrestrial scene, and the minds of the young men
were so much impressed by the whole of the circumstances, that it
made a great alteration in their after life. Thomas, as solemnly
charged by his father, married Ellen Scott, and Francis was well
known afterwards as the celebrated Dr Beattie of Amherst. Ellen was
mother to twelve sons; and on the night that her seventh son was
born, her aunt Jerdan was lost, and never more heard of, either
living or dead.[9]
9. This will be viewed as a most romantic and unnatural story, as without
doubt it is; but I have the strongest reasons for believing that it is founded on a
literal fact, of which all the three were sensibly and positively convinced. It was
published in England in Dr Beattie’s lifetime, and by his acquiescence, and owing
to the respectable source from whence it came, it was never disputed in that day
that it had its origin in truth. It was again republished, with some miserable
alterations, in a London collection of 1770, by J. Smith, at No. 15, Paternoster Row,
and though I have seen none of these accounts, but relate the story wholly from
tradition, yet the assurance obtained from a friend of their existence, is a curious
corroborative circumstance, and proves that if the story was not true, the parties at
least believed it to be so.—Note by the Author.
 THE ELDER’S FUNERAL.

By Professor Wilson.

How beautiful to the eye and to the heart rise up, in a pastoral
region, the green silent hills from the dissolving snow-wreaths that
yet linger at their feet! A few warm sunny days, and a few breezy and
melting nights, have seemed to create the sweet season of spring out
of the winter’s bleakest desolation. We can scarcely believe that such
brightness of verdure could have been shrouded in the snow,
blending itself, as it now does, so vividly with the deep blue of
heaven. With the revival of nature our own souls feel restored.
Happiness becomes milder, meeker, and richer in pensive thought;
while sorrow catches a faint tinge of joy, and reposes itself on the
quietness of earth’s opening breast. Then is youth rejoicing—
manhood sedate—and old age resigned. The child shakes his golden
curls in his glee; he of riper life hails the coming year with temperate
exultation; and the eye that has been touched with dimness, in the
general spirit of delight, forgets or fears not the shadows of the grave.
On such a vernal day as this did we, who had visited the Elder on
his death-bed,[10] walk together to his house in the Hazel Glen, to
accompany his body to the place of burial. On the night he died, it
seemed to be the dead of winter. On the day he was buried, it seemed
to be the birth of spring. The old pastor and I were alone for awhile
as we pursued our path up the glen, by the banks of the little burn. It
had cleared itself off from the melted snow, and ran so pellucid a
race that every stone and pebble was visible in its yellow channel.
The willows, the alders, and the birches, the fairest and the earliest of
our native hill-trees, seemed almost tinged with a verdant light, as if
they were budding; and beneath them, here and there peeped out, as
in the pleasure of new existence, the primrose lonely, or in little
families and flocks. The bee had not yet ventured to leave his cell, yet
the flowers reminded one of his murmur. A few insects were dancing
in the air, and here and there some little moorland bird, touched at
the heart with the warm and sunny change, was piping his love-sweet
song among the braes. It was just such a day as a grave meditative
man, like him we were about to inter, would have chosen to walk
over his farm in religious contentment with his lot. That was the
thought that entered the pastor’s heart, as we paused to enjoy one
brighter gleam of the sun in a little meadow-field of peculiar beauty.
10. See ante, page 280.
“This is the last day of the week, and on that day often did the
Elder walk through this little happy kingdom of his own, with some
of his grandchildren beside and around him, and often his Bible in
his hand. It is, you feel, a solitary place,—all the vale is one seclusion
—and often have its quiet bounds been a place of undisturbed
meditation and prayer.”
We now came in sight of the cottage, and beyond it the
termination of the glen. There the high hills came sloping gently
down; and a little waterfall, in the distance, gave animation to a
scene of perfect repose. We were now joined by various small parties
coming to the funeral through openings among the hills; all sedate,
but none sad, and every greeting was that of kindness and peace. The
Elder had died full of years; and there was no need why any out of his
household should weep. A long life of piety had been beautifully
closed; and, therefore, we were all going to commit the body to the
earth, assured, as far as human beings may be so assured, that the
soul was in heaven. As the party increased on our approach to the
house, there was even cheerfulness among us. We spoke of the early
and bright promise of spring—of the sorrows and joys of other
families—of marriages and births—of the new schoolmaster—of to-
morrow’s Sabbath. There was no topic of which, on any common
occasion, it might have been fitting to speak, that did not now
perhaps occupy, for a few moments, some one or other of the group,
till we found ourselves ascending the greensward before the cottage,
and stood below the bare branches of the sycamores. Then we were
all silent, and, after a short pause, reverently entered into the house
of death.
 At the door the son received us with a calm, humble, and
untroubled face; and in his manner towards the old minister, there
was something that could not be misunderstood, expressing
penitence, gratitude, and resignation. We all sat down in the large
kitchen; and the son decently received each person at the door, and
showed him to his place. There were some old gray heads, more
becoming gray, and many bright in manhood and youth. But the
same solemn hush was over them all, and they sat all bound together
in one uniting and assimilating spirit of devotion and faith. Wine and
bread were to be sent round; but the son looked to the old minister,
who rose, lifted up his withered hand, and began a blessing and a
prayer.
There was so much composure and stillness in the old man’s
attitude, and something so affecting in his voice, tremulous and
broken, not in grief but age, that no sooner had he begun to pray,
than every heart and every breath at once were hushed. All stood
motionless, nor could one eye abstain from that placid and
patriarchal countenance, with its closed eyes, and long silvery hair.
There was nothing sad in his words, but they were all humble and
solemn, and at times even joyful in the kindling spirit of piety and
faith. He spoke of the dead man’s goodness as imperfect in the eyes
of his Great Judge, but such as, we were taught, might lead, through
intercession, to the kingdom of heaven. Might the blessing of God, he
prayed, which had so long rested on the head now coffined, not
forsake that of him who was now to be the father of this house. There
was more—more joy, we were told, in heaven, over one sinner that
repenteth, than over ninety and nine just persons which need no
repentance. Fervently, too, and tenderly, did the old man pray for
her, in her silent chamber, who had lost so kind a parent, and for all
the little children round her knees. Nor did he end his prayer without
some allusion to his own gray hairs, and to the approaching day on
which many then present would attend his burial.
Just as he ceased to speak, one solitary stifled sob was heard, and
all eyes turned kindly round to a little boy who was standing by the
side of the Elder’s son. Restored once more to his own father’s love,
his heart had been insensibly filled with peace since the old man’s
death. The returning tenderness of the living came in place of that of
the dead, and the child yearned towards his father now with a
stronger affection, relieved at last from all his fear. He had been
suffered to sit an hour each day beside the bed on which his
grandfather lay shrouded, and he had got reconciled to the cold but
silent and happy looks of death. His mother and his Bible told him to
obey God without repining in all things; and the child did so with
perfect simplicity. One sob had found its way at the close of that
pathetic prayer; but the tears that bathed his glistening cheeks were
far different from those that, on the day and night of his
grandfather’s decease, had burst from the agony of a breaking heart.
The old minister laid his hand silently upon his golden head; there
was a momentary murmur of kindness and pity over the room; the
child was pacified, and again all was repose and peace.
A sober voice said all was ready, and the son and the minister led
the way reverently out into the open air. The bier stood before the
door, and was lifted slowly up with its sable pall. Silently each
mourner took his place. The sun was shining pleasantly, and a gentle
breeze, passing through the sycamore, shook down the glittering
raindrops upon the funeral velvet. The small procession, with an
instinctive spirit, began to move along; and as I cast up my eyes to
take a farewell look of that beautiful dwelling, now finally left by him
who so long had blessed it, I saw at the half-open lattice of the little
bedroom window above, the pale weeping face of that stainless
matron, who was taking her last passionate farewell of the mortal
remains of her father, now slowly receding from her to the quiet field
of graves.
We proceeded along the edges of the hills, and along the meadow-
fields, crossed the old wooden bridge over the burn, now widening in
its course to the plain, and in an hour of pensive silence, or pleasant
talk, we found ourselves entering, in a closer body, the little gateway
of the churchyard. To the tolling of the bell we moved across the
green mounds, and arranged ourselves, according to the plan and
order which our feelings suggested, around the bier and its natural
supporters. There was no delay. In a few minutes the Elder was laid
among the mould of his forefathers, in their long-ago chosen spot of
rest. One by one the people dropped away, and none were left by the
new made grave but the son and his little boy, the pastor and myself.
As yet nothing was said, and in that pause I looked around me, over
the sweet burial-ground.
 Each tombstone and grave over which I had often walked in
boyhood arose in my memory, as I looked steadfastly upon their
long-forgotten inscriptions; and many had then been erected. The
whole character of the place was still simple and unostentatious, but
from the abodes of the dead I could see that there had been an
improvement in the condition of the living. There was a taste visible
in their decorations, not without much of native feeling, and
occasionally something even of native grace. If there was any other
inscription than the name and age of the poor inhabitants below, it
was, in general, some short text of Scripture; for it is most pleasant
and soothing to the pious mind, when bereaved of friends, to
commemorate them on earth by some touching expression taken
from that Book which reveals to them a life in heaven.
There is a sort of gradation, a scale of forgetfulness, in a country
churchyard, where the processes of nature are suffered to go on over
the green place of burial, that is extremely affecting in the
contemplation. The soul goes from the grave just covered up, to that
which seems scarcely joined together, on and on to those folded and
bound by the undisturbed verdure of many, many unremembered
years. It then glides at last into nooks and corners where the ground
seems perfectly calm and waveless, utter oblivion having smoothed
the earth over the long mouldered bones. Tombstones, on which the
inscriptions are hidden in green obliteration, or that are mouldering,
or falling to a side, are close to others which last week were brushed
by the chisel;—constant renovation and constant decay—vain
attempts to adhere to memory—and oblivion, now baffled and now
triumphant, smiling among all the memorials of human affection, as
they keep continually crumbling away into the world of
undistinguishable dust and ashes.
The churchyard, to the inhabitants of a rural parish, is the place to
which, as they grow older, all their thoughts and feelings turn. The
young take a look of it every Sabbath-day, not always perhaps a
careless look, but carry away from it, unconsciously, many salutary
impressions. What is more pleasant than the meeting of a rural
congregation in the churchyard before the minister appears? What is
there to shudder at in lying down, sooner or later, in such a peaceful
and sacred place, to be spoken of frequently on Sabbath among the
groups of which we used to be one, and our low burial-spot to be
visited, at such times, as long as there remains on earth any one to
whom our face was dear? To those who mix in the strife and dangers
of the world, the place is felt to be uncertain wherein they may finally
lie at rest. The soldier—the sailor—the traveller—can only see some
dim grave dug for him when he dies, in some place obscure,
nameless, and unfixed to the imagination. All he feels is, that his
burial will be—on earth—or in the sea. But the peaceful dwellers who
cultivate their paternal acres, or tilling at least the same small spot of
soil, shift only from a cottage on the hillside to one on the plain, still
within the bounds of one quiet parish; they look to lay their bones at
last in the burial-place of the kirk in which they were baptised, and
with them it almost literally is but a step from the cradle to the grave.
Such were the thoughts that calmly followed each other in my
reverie, as I stood beside the Elder’s grave, and the trodden grass was
again lifting up its blades from the pressure of many feet, now all, but
a few, departed. What a simple burial had it been! Dust was
consigned to dust—no more. Bare, naked, simple, and austere is in
Scotland the service of the grave. It is left to the soul itself to
consecrate, by its passion, the mould over which tears, but no words,
are poured. Surely there is a beauty in this; for the heart is left unto
its own sorrow—according as it is a friend—a brother—a parent—or a
child, that is covered up from our eyes. Yet call not other rites,
however different from this, less beautiful or pathetic. For willingly
does the soul connect its grief with any consecrated ritual of the
dead. Sound or silence—music—hymns—psalms—sable garments, or
raiment white as snow—all become holy symbols of the soul’s
affection; nor is it for any man to say which is the most natural,
which is the best, of the thousand shows and expressions, and
testimonies of sorrow, resignation, and love, by which mortal beings
would seek to express their souls when one of their brethren has
returned to his parent dust.
My mind was recalled from all these sad, yet not unpleasant
fancies, by a deep groan, and I beheld the Elder’s son fling himself
down upon the grave and kiss it passionately, imploring pardon from
God. “I distressed my father’s heart in his old age—I repented—and
received thy forgiveness even on thy death-bed! But how may I be
assured that God will forgive me for having so sinned against my old,
grayheaded father, when his limbs were weak and his eyesight dim!”
The old minister stood at the head of the grave without speaking a
word, with his solemn and pitiful eyes fixed upon the prostrate and
contrite man. His sin had been great, and tears that till now had, on
this day at least, been compressed within his heart by the presence of
so many of his friends, now poured down upon the sod as if they
would have found their way to the very body of his father. Neither of
us offered to lift him up, for we felt awed by the rueful passion of his
love, his remorse, and his penitence; and nature, we felt, ought to
have her way. “Fear not, my son,” at length said the old man, in a
gentle voice—“fear not, my son, but that you are already forgiven.
Dost thou not feel pardon within thy contrite spirit?” He rose up
from his knees with a faint smile, while the minister, with his white
head yet uncovered, held his hands over him as in benediction; and
that beautiful and loving child, who had been standing in a fit of
weeping terror at his father’s agony, now came up to him and kissed
his cheek—holding in his little hand a few faded primroses which he
had unconsciously gathered together as they lay on the turf of his
grandfather’s grave.
 MACDONALD, THE CATTLE-RIEVER.

Archibald Macdonald was perhaps the most perfect master of his

hazardous profession of any who ever practised it. Archibald was by
birth a gentleman, and proprietor of a small estate in Argyleshire,
which he however lost early in life. He soon distinguished himself as
a cattle-lifter on an extensive scale; and weak as the arm of the law
might then have been, he found it advisable to remove further from
its influence, and he shifted his residence from his native district of
Appin to the remote peninsula of Ardnamurchan, which was
admirably adapted to his purpose, from its geographical position. He
obtained a lease of an extensive farm, and he fitted up a large
cowhouse, though his whole visible live-stock consisted of one filly.
His neighbours could not help making remarks on this subject, but
he begged of them to have no anxiety on that head, assuring them
that his byre would be full ere Christmas; and he was as good as his
word. He had trained the filly to suit his purpose, and it was a
practice of his to tie other horses to her tail; she then directed her
course homeward by unfrequented routes, and always found her way
in safety.
His expeditions were generally carried on by sea, and he annoyed
the most distant of the Hebrides, both to the south and north. He
often changed the colour of his boats and sails, and adopted
whatever appeared best suited to his immediate purpose. In
consequence of this artifice, his depredations were frequently
ascribed to others, and sometimes to men of the first distinction in
that country, so dexterously did he imitate their birlings and their
insignia. He held his land from Campbell of Lochnell, into whose
favour he had insinuated himself by his knowledge and address.
When Lochnell resided at the castle of Mingary, Archibald was
often ordered to lie on a mattress in his bedroom, to entertain him at
night with the recitation of the poems of Ossian, and with tales.
Archibald contrived means to convert this circumstance to his
advantage. He ordered his men to be in readiness, and that night he
selected one of his longest poems. As he calculated, Lochnell fell
asleep before he had finished the recital; the robber slunk out and
soon joined his associates. He steered for the island of Mull, where
some of his men had been previously sent to execute his orders; he
carried off a whole fold of cattle, which he landed safely, and
returned to his mattress before Lochnell awoke. When he lay down
he purposely snored so loudly that the sleeping chief was disturbed,
and complained of the tremendous noise the fellow made, observing
that, fond as he was of poetry, he must deprive himself of it in future
on such conditions. To this Archibald had no objections; his
principal object was then accomplished, and taking up the tale where
he had stopped when his patron fell asleep, he finished it, and slept
soundly to an advanced hour.
The cattle were immediately missed, and suspicion fell on
Archibald; but he triumphantly referred to Lochnell for a proof of his
innocence, and this he obtained. That gentleman solemnly declared
that the robber had never been out of his room during that night, and
the charge was of course dropped.
A wealthy man who resided in the neighbourhood was noted for
his penurious habits, and he had incurred particular odium by
refusing a supply of meal to a poor widow in distress. This man had
sent a considerable quantity of grain to the mill, which, as usual, he
attended himself, and was conveying the meal home at night on
horseback. The horses were tied in a string, the halter of one fixed to
the tail of another; and the owner led the foremost by a long tether.
His road lay through a wood, and Archibald there watched his
approach. The night was dark, and the man walked slowly, humming
a song; the ground was soft, and the horses having no shoes (as is
still usual in that country), their tread made no noise. Archibald
ordered one of his men to loosen the tether from the head of the
front horse, and to hold it, himself occupying the place of the horse,
and walking on at the same pace. He thus got possession of the
whole. The miser soon arrived at his own door, and called for
assistance to deposit his winter store in safety, but, to his
astonishment, found he had but the halter!