Download textbook A Basic Course In Probability Theory 2Nd Edition Rabi Bhattacharya ebook all chapter pdf

A Basic Course in Probability Theory
2nd Edition Rabi Bhattacharya

Visit to download the full and correct content document:
https://textbookfull.com/product/a-basic-course-in-probability-theory-2nd-edition-rabi-b
hattacharya/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...
Rabi N Bhattacharya Selected Papers 1st Edition Manfred

Denker
https://textbookfull.com/product/rabi-n-bhattacharya-selected-
papers-1st-edition-manfred-denker/
Golosa A Basic Course in Russian Book One 6th Edition

Robin
https://textbookfull.com/product/golosa-a-basic-course-in-
russian-book-one-6th-edition-robin/
Foundations of the Theory of Probability 2nd Edition

A.N. Kolmogorov
https://textbookfull.com/product/foundations-of-the-theory-of-
probability-2nd-edition-a-n-kolmogorov/
Probability theory and statistical inference empirical

modeling with observational data 2nd Edition Spanos A
https://textbookfull.com/product/probability-theory-and-
statistical-inference-empirical-modeling-with-observational-
data-2nd-edition-spanos-a/
A Course in Finite Group Representation Theory Peter
Webb
https://textbookfull.com/product/a-course-in-finite-group-
representation-theory-peter-webb/
A Course in Functional Analysis and Measure Theory

Vladimir Kadets
https://textbookfull.com/product/a-course-in-functional-analysis-
and-measure-theory-vladimir-kadets/
Statistical independence in probability analysis number

theory Kac
https://textbookfull.com/product/statistical-independence-in-
probability-analysis-number-theory-kac/
A Second Course in Topos Quantum Theory 1st Edition

Cecilia Flori (Auth.)
https://textbookfull.com/product/a-second-course-in-topos-
quantum-theory-1st-edition-cecilia-flori-auth/
Stage Performance for Singers A Practical Course in 12

Basic Steps 1st Edition Martin Karnolsky (Author)
https://textbookfull.com/product/stage-performance-for-singers-a-
practical-course-in-12-basic-steps-1st-edition-martin-karnolsky-
author/
Universitext
Rabi Bhattacharya
Edward C. Waymire
A Basic Course
in Probability
Theory
Second Edition
Universitext
Universitext
Series editors
Sheldon Axler
San Francisco State University, San Francisco, CA, USA
Vincenzo Capasso
Università degli Studi di Milano, Milan, Italy
Carles Casacuberta
Universitat de Barcelona, Barcelona, Spain
Angus MacIntyre
Queen Mary University of London, London, UK
Kenneth Ribet
University of California, Berkeley, CA, USA
Claude Sabbah
École Polytechnique, CNRS, Université Paris-Saclay, Palaiseau, France
Endre Süli
University of Oxford, Oxford, UK
Wojbor A. Woyczyński
Case Western Reserve University, Cleveland, OH, USA
Universitext is a series of textbooks that presents material from a wide variety of

mathematical disciplines at master’s level and beyond. The books, often well
class-tested by their author, may have an informal, personal, even experimental
approach to their subject matter. Some of the most successful and established books
in the series have evolved through several editions, always following the evolution
of teaching curricula, into very polished texts.
Thus as research topics trickle down into graduate-level teaching, first textbooks
written for new, cutting-edge courses may make their way into Universitext.
More information about this series at http://www.springer.com/series/223

Rabi Bhattacharya Edward C. Waymire
•
A Basic Course in Probability

Theory
Second Edition
123
Rabi Bhattacharya Edward C. Waymire
Department of Mathematics Department of Mathematics
University of Arizona Oregon State Univeristy
Tucson, AZ Corvallis, OR
USA USA
ISSN 0172-5939 ISSN 2191-6675 (electronic)

Universitext
ISBN 978-3-319-47972-9 ISBN 978-3-319-47974-3 (eBook)
DOI 10.1007/978-3-319-47974-3
Library of Congress Control Number: 2016955325
Mathematics Subject Classification (2010): 60-xx, 60Jxx
1st edition: © Springer Science+Business Media, Inc. 2007

2nd edition: © Springer International Publishing AG 2016
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt from
the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, express or implied, with respect to the material contained herein or
for any errors or omissions that may have been made.
Printed on acid-free paper
This Springer imprint is published by Springer Nature

The registered company is Springer International Publishing AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface to Second Edition
This second edition continues to serve primarily as a text for a lively two-quarter or
one-semester course in probability theory for students from diverse disciplines,
including mathematics and statistics. Exercises have been added and reorganized
(i) for reinforcement in the use of techniques, and (ii) to complement some results.
Sufficient material has been added so that in its entirety the book may also be used
for a two-semester course in basic probability theory. The authors have reorganized
material to make Chapters III–XIII as self-contained as possible. This will aid
instructors of one semester (or two quarter) courses in picking and choosing some
material, while omitting some other. Material from a former chapter on Laplace
transforms has been redistributed to parts of the text where it is used.
The early introduction of conditional expectation and conditional probability
maintains the pedagogic innovation of the first edition. This enables the student to
quickly move to the fundamentally important notions of modern probability besides
independence, namely, martingale dependence and Markov dependence, where
new theory and examples have been added to the text. The former includes Doob’s
upcrossing inequality, the submartingale convergence theorem, and reverse
martingales, and the (reverse) martingale proof of the strong law of large numbers,
while retaining important earlier approaches such as those of Kolmogorov,
Etemadi, and of Marcinkiewicz–Zygmund.
A theorem of Polya is added to the chapter on weak convergence to show that
the convergence to the normal distribution function in the central limit theorem is
uniform.
The Cramér–Chernoff large deviation theory in Chapter V is sharpened by the
addition of a large deviation theorem of Bahadur and Ranga Rao using the Berry–
Esseen convergence rate in the central limit theorem. Also added in Chapter V is a
concentration of measure type inequality due to Hoeffding. The proof of the
aforementioned Berry–Esseen bound is deferred to Chapter VI on Fourier series
and Fourier transform. The Chung–Fuchs transience/recurrence criteria for ran-
dom walk based on Fourier analysis is a new addition to the text.
Special examples of Markov processes such as Brownian motion, and random
walks appear throughout the text to illustrate applications of (i) martingale theory
v
vi Preface to Second Edition
and stopping times in computations of certain important probabilities.

A culmination of the theory developed in the text occurs in Chapters XI and XII on
Brownian motion. This continues to rank among the primary goals attainable for a
course based on the text.
General Markov dependent sequences and their convergence to equilibrium is
the subject matter of the entirely new Chapter XIII. Illustrative examples are pro-
vided, including some of historical importance to the development of the kinetic
theory of matter in physics due to Boltzmann, Einstein, and Smoluchowski. The
treatment centers on describing a prototypical framework, namely Doeblin’s
theorem, for existence and convergence to a unique invariant probability for
Markov processes, together with illustrative examples for students with diverse
interests ranging from mathematics and statistics to contemporary mathematical
finance or biology. Examples include iterated random maps, the Ehrenfest model,
and products of random matrices. The Ornstein–Uhlenbeck process is shown to be
obtained as the unique solution to a stochastic differential equation, namely the
Langevin equation, using Picard iteration. This provides students with a glimpse
into the broad scope and utility of the probability that they have learned, while
motivating continued study of stochastic processes.
Complete references to authors of books cited in footnotes are provided in a
closing list of references. This also includes other textbook resources covering the
same topics and/or further applications.
The authors are grateful to William Faris, University of Arizona, and to Enrique
Thomann, Oregon State University, for providing comments and corrections to an
earlier draft based on their teaching of the course. Partial support from the National
Science Foundation under grants DMS 1406872 and DMS1408947, respectively, is
gratefully acknowledged by the authors.
Tucson, AZ, USA Rabi Bhattacharya

Corvallis, OR, USA Edward C. Waymire
September 2016
Preface to First Edition
In 1937, A.N. Kolmogorov introduced a measure-theoretic mathematical frame-

work for probability theory in response to David Hilbert’s Sixth Problem. This text
provides the basic elements of probability within this framework. It may be used for
a one-semester course in probability, or as a reference to prerequisite material in a
course on stochastic processes. Our pedagogical view is that the subsequent
applications to stochastic processes provide a continued opportunity to motivate
and reinforce these important mathematical foundations. The book is best suited for
students with some prior, or at least concurrent, exposure to measure theory and
analysis. But it also provides a fairly detailed overview, with proofs given in
appendices, of the measure theory and analysis used.
The selection of material presented in this text grew out of our effort to provide a
self-contained reference to foundational material that would facilitate a companion
treatise on stochastic processes that Theory and Applications of Stochastic
Processes we have been developing.1 While there are many excellent textbooks
available that provide the probability background for various continued studies of
stochastic processes, the present treatment was designed with this as an explicit
goal. This led to some unique features from the perspective of the ordering and
selection of material.
We begin with Chapter I on various measure-theoretic concepts and results
required for the proper mathematical formulation of a probability space, random
maps, distributions, and expected values. Standard results from measure theory are
motivated and explained with detailed proofs left to an appendix.
Chapter II is devoted to two of the most fundamental concepts in probability
theory: independence and conditional expectation (and/or conditional probability).
This continues to build upon, reinforce, and motivate basic ideas from real analysis
and measure theory that are regularly employed in probability theory, such as
Carathéodory constructions, the Radon–Nikodym theorem, and the Fubini–Tonelli
1
Bhattacharya, R. and E. Waymire (2007): Theory and Applications of Stochastic Processes,
Springer-Verlag, Graduate Texts in Mathematics.
vii
viii Preface to First Edition
theorem. A careful proof of the Markov property is given for discrete-parameter

random walks on ℝk to illustrate conditional probability calculations in some
generality.
Chapter III provides some basic elements of martingale theory that have evolved
to occupy a significant foundational role in probability theory. In particular,
optional stopping and maximal inequalities are cornerstone elements. This chapter
provides sufficient martingale background, for example, to take up a course in
stochastic differential equations developed in a chapter of our text on stochastic
processes. A more comprehensive treatment of martingale theory is deferred to
stochastic processes with further applications there as well.
The various laws of large numbers and elements of large deviation theory are
developed in Chapter IV. This includes the classical 0–1 laws of Kolmogorov and
Hewitt–Savage. Some emphasis is given to size-biasing in large deviation calcu-
lations which are of contemporary interest.
Chapter V analyzes in detail the topology of weak convergence of probabilities
defined on metric spaces, culminating in the notion of tightness and a proof of
Prohorov’s theorem.
The characteristic function is introduced in Chapter VI via a first principles
development of Fourier series and the Fourier transform. In addition to the oper-
ational calculus and inversion theorem, Herglotz’s theorem, Bochner’s theorem,
and the Cramér–Lévy continuity theorem are given. Probabilistic applications
include the Chung–Fuchs criterion for recurrence of random walks on ℝk, and the
classical central limit theorem for i.i.d. random vectors with finite second moments.
The law of rare events (i.e., Poisson approximation to binomial) is also included as
a simple illustration of the continuity theorem, although simple direct calculations
are also possible.
In Chapter VII, central limit theorems of Lindeberg and Lyapounov are derived.
Although there is some mention of stable and infinitely divisible laws, the full
treatment of infinite divisibility and Lévy–Khinchine representation is more prop-
erly deferred to a study of stochastic processes with independent increments.
The Laplace transform is developed in Chapter VIII with Karamata’s Tauberian
theorem as the main goal. This includes a heavy dose of exponential size-biasing
techniques to go from probabilistic considerations to general Radon measures. The
standard operational calculus for the Laplace transform is developed along the way.
Random series of independent summands are treated in Chapter IX. This
includes the mean square summability criterion and Kolmogorov’s three series
criteria based on Kolmogorov’s maximal inequality. An alternative proof to that
presented in Chapter IV for Kolmogorov’s strong law of large numbers is given,
together with the Marcinkiewicz and Zygmund extension, based on these criteria
and Kronecker’s lemma. The equivalence of a.s. convergence, convergence in
probability, and convergence in distribution for series of independent summands is
also included.
In Chapter X, Kolmogorov’s consistency conditions lead to the construction of
probability measures on the Cartesian product of infinitely many spaces. Applications
include a construction of Gaussian random fields and discrete-parameter Markov
Preface to First Edition ix
processes. The deficiency of Kolmogorov’s construction of a model for Brownian

motion is described, and the Lévy–Ciesielski “wavelet” construction is provided.
Basic properties of Brownian motion are taken up in Chapter XI. Included are
various rescalings and time inversion properties, together with the fine-scale
structure embodied in the law of the iterated logarithm for Brownian motion.
In Chapter XII many of the basic notions introduced in the text are tied together
via further considerations of Brownian motion. In particular, this chapter revisits
conditional probabilities in terms of the Markov and strong Markov properties for
Brownian motion, stopping times, and the optional stopping and/or sampling the-
orems for Brownian motion and related martingales, and leads to weak convergence
of rescaled random walks with finite second moments to Brownian motion, i.e.,
Donsker’s invariance principle or the functional central limit theorem, via the
Skorokhod embedding theorem.
The text is concluded with a historical overview, Chapter XIII, on Brownian
motion and its fundamental role in applications to physics, financial mathematics,
and partial differential equations, which inspired its creation.
Most of the material in this book has been used by us in graduate probability
courses taught at the University of Arizona, Indiana University, and Oregon State
University. The authors are grateful to Virginia Jones for superb word processing
skills that went into the preparation of this text. Also, two Oregon State University
graduate students, Jorge Ramirez and David Wing, did an outstanding job in
uncovering and reporting various bugs in earlier drafts of this text. Thanks go to the
editorial staff at Springer and anonymous referees for their insightful remarks.
March 2007 Rabi Bhattacharya

Edward C. Waymire
NOTE: Some of the first edition chapter numbers have changed in the second
edition. First edition Chapter VII was moved to Chapter IV, and the material in the
first edition Chapter VIII has been redistributed into other chapters. An entirely new
Chapter XIII was added to the second edition.
Contents
I Random Maps, Distribution, and Mathematical Expectation . . . . 1

Exercise Set I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
II Independence, Conditional Expectation . . . . . . . . . . . . . . . . . . . . . 25
Exercise Set II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
III Martingales and Stopping Times . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Exercise Set III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
IV Classical Central Limit Theorems. . . . . . . . . . . . . . . . . . . . . . . . . . 75
Exercise Set IV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
V Classical Zero–One Laws, Laws of Large Numbers
and Large Deviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Exercise Set V. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
VI Fourier Series, Fourier Transform, and Characteristic
Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Exercise Set VI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
VII Weak Convergence of Probability Measures on Metric
Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
Exercise Set VII . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
VIII Random Series of Independent Summands . . . . . . . . . . . . . . . . . . 159
Exercise Set VIII . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
IX Kolmogorov’s Extension Theorem and Brownian Motion . . . . . . 167
IX.1 A Wavelet Construction of Brownian Motion:
The Lévy–Ciesielski Construction . . . . . . . . . . . . . . . . . . . . . . 173
Exercise Set IX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
X Brownian Motion: The LIL and Some Fine-Scale Properties . . . . 179
Exercise Set X. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
xi
xii Contents
XI Strong Markov Property, Skorokhod Embedding,

and Donsker’s Invariance Principle . . . . . . . . . . . . . . . . . . . . . . . . 187
Exercise Set XI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
XII A Historical Note on Brownian Motion . . . . . . . . . . . . . . . . . . . . . 207
XIII Some Elements of the Theory of Markov Processes
and Their Convergence to Equilibrium . . . . . . . . . . . . . . . . . . . . . 211
Exercise Set XIII . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
Appendix A: Measure and Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
Appendix B: Topology and Function Spaces. . . . . . . . . . . . . . . . . . . . . . . 241
Appendix C: Hilbert Spaces and Applications in Measure Theory . . . . . 247
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
Symbol Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
Chapter I
Random Maps, Distribution,
and Mathematical Expectation
In the spirit of a refresher, we begin with an overview of the measure–theoretic

framework for probability. Readers for whom this is entirely new material may
wish to consult the appendices for statements and proofs of basic theorems from
analysis. A measure space is a triple (S, S, μ), where S is a nonempty set; S is a
collection of subsets of S, referred to as a σ-field, which includes ∅ and is closed
under complements and countable unions; and ∞μ : S → [0, ∞] satisfies (i) μ(∅) = 0,
(ii) (countable additivity) μ(∪∞ n=1 A n ) = n=1 μ(An ) if A1 , A2 , . . . is a sequence
of disjoint sets in S. Subsets of S belonging to S are called measurable sets. The
pair (S, S) is referred to as a measurable space, and the set function μ is called a
measure. Familiar examples from real analysis are Lebesgue measure μ on S = Rk ,
equipped with a σ-field S containing the class of all k-dimensional rectangles, say
R = (a1 , b1 ] × · · · × (ak , bk ], of “volume” measure μ(R) = kj=1 (b j − a j ); or
Dirac point mass measure μ = δx at x ∈ S defined by δx (B) = 1 if x ∈ B,
δx (B) = 0 if x ∈ B c , for B ∈ S. Such examples should suffice for the present,
but see Appendix A for constructions of these and related measures based on the
Carathéodory extension theorem. If μ(S) < ∞ then μ is referred to as a finite
measure. If one may write S = ∪∞ n=1 Sn , where each Sn ∈ S(n ≥ 1) and μ(Sn ) <
∞, ∀n, then μ is said to be a σ-finite measure.
A probability space is a triple (Ω, F, P), where Ω is a nonempty set, F is a
σ-field of subsets of Ω, and P is a finite measure on the measurable space (Ω, F) with
P(Ω) = 1. The measure P is referred to as a probability. Intuitively, Ω represents
the set of all possible “outcomes” of a random experiment, real or conceptual, for
some given coding of the results of the experiment. The set Ω is referred to as the
sample space and the elements ω ∈ Ω as sample points or possible outcomes. The
σ-field F comprises “events” A ⊂ Ω whose probability P(A) of occurrence is well
defined.
The finite total probability and countable additivity of a probability have many
important consequences, such as finite additivity, finite and countable subad-
ditivity, inclusion–exclusion, monotonicity, and the formulas for both relative
© Springer International Publishing AG 2016 1
R. Bhattacharya and E.C. Waymire, A Basic Course in Probability Theory,
Universitext, DOI 10.1007/978-3-319-47974-3_I
2 I Random Maps, Distribution, and Mathematical Expectation
complements and universal complements. Proofs of these properties are left to the
reader and included among the exercises.
Example 1 (Finite Sampling of a Fair Coin) Consider m repeated tosses of a fair
coin. Coding the individual outcomes as 1 or 0 (or, say, H, T), the possible outcomes
may be represented as sequences of binary digits of length m. Let Ω = {0, 1}m denote
the set of all such sequences and F = 2Ω , the power set of Ω. The condition that
the coin be fair may be defined by the requirement that P({ω}) is the same for each
sequence ω ∈ Ω. Since Ω has cardinality |Ω| = 2m , it follows from the finite
additivity and total probability requirements that
1 1
P({ω}) = = , ω ∈ Ω.
2 m |Ω|
Using finite additivity this completely and explicitly specifies the model (Ω, F, P)
with
|A|
P(A) = P({ω}) = , A ⊂ Ω.
ω∈A
|Ω|
The so-called continuity properties also follow from the definition as follows:
A sequence of events An , n ≥ 1, is said to be increasing (respectively, decreasing)
with respect to set inclusion if An ⊂ An+1 , ∀n ≥ 1 (respectively An ⊃ An+1 ∀n ≥ 1).
In the former case one defines limn An := ∪n An , while for decreasing measurable
events limn An := ∩n An . In either case the continuity of a probability, from below
or above, respectively, is the following consequence of countable additivity1 (Exer-
cise 1):
P(lim An ) = lim P(An ). (1.1)
n n
A bit more generally, if {An }∞

n=1 is a sequence of measurable events one defines
lim sup An := ∩n=1 ∪m≥n Am (1.2)

n
and
lim inf An := ∪∞
n=1 ∩m≥n Am . (1.3)
n
The event lim supn An denotes the collection of outcomes ω ∈ Ω that correspond
to the occurrences of An for infinitely many n; i.e., the events An occur infinitely
often This event is also commonly denoted by [An i.o.] := lim supn An . On the other
hand, lim inf n An is the set of outcomes ω that belong to An for all but finitely many
n. Note that [An i.o.]c is the event that the complementary event Acn occurs for all
but finitely many n and equals lim inf n Acn .
1 Withthe exception of properties for “complements” and “continuity from above,” these and the
aforementioned consequences can be checked to hold for any measure.
I Random Maps, Distribution, and Mathematical Expectation 3
Lemma 1 (Borel–Cantelli
I) Let (Ω, F, P) be a probability space and An ∈ F, n =
1, 2, . . . . If ∞
n=1 P(A n ) < ∞ then P(An i.o.) = 0.
Proof Apply (1.1) to the decreasing sequence of events
∪∞ ∞
m=1 Am ⊃ ∪m=2 Am ⊃ · · · ,
followed by subadditivity of the probability to get

∞

P(lim sup An ) = lim P(∪∞
m=n Am ) ≤ lim P(Am ) = 0.
n n→∞ n→∞
m=n

A partial converse (Borel–Cantelli II) will be given in the next chapter.
Example 2 Suppose that T1 , T2 , . . . is a sequence of positive random variables
defined on a probability space (Ω, F, P) such that for some constant λ > 0, P(Tn > t)
= e−λt , t ≥ 0, for n = 1, 2, . . . . Then P(Tn > n i.o.) = 0. In fact, P(Tn > θ log
n i.o.) = 0 for any value of θ > λ1 . This may also be expressed as P(Tn ≤ θ log n
eventually for all n) = 1 if θ > λ1 .
Example 3 (Infinite Sampling of a Fair Coin) The possible outcomes of nontermi-
nated repeated coin tosses can be coded as infinite binary sequences of 1’s and 0’s.
Thus the sample space is the infinite product space Ω = {0, 1}∞ . Observe that a
sequence ω ∈ Ω may be viewed as the digits in a binary expansion of a number x in
the unit interval. The binary expansion x = ∞ −n
n=1 ωn (x)2 , where ωn (x) ∈ {0, 1},
is not unique for binary rationals, e.g., 2 = .1000000 . . . = .011111 . . .. However
1
it may be made unique by requiring that infinitely many 0’s occur in the expansion.
Thus, up to a subset of probability zero, Ω and [0, 1) may be put in one-to-one
correspondence. Observe that for a given specification εn ∈ {0, 1}, n = 1, . . . , m,
of the first m tosses, the event A = {ω = (ω 1 , ω2 , . . . ) ∈ Ω : ωn = εn , n ≤ m} cor-
−n −n
responds to the subinterval [ m n=1 ε n 2 , m
n=1 εn 2 + 2−m ) of [0, 1) of length
−m
(Lebesgue measure) 2 . Again modeling the repeated tosses of a fair coin by
the requirement that for each fixed m, P(A) not depend on the specified values
εn ∈ {0, 1}, 1 ≤ n ≤ m, it follows from finite additivity and total probability one
that P(A) = 2−m = |A|, where |A| denotes the one-dimensional Lebesgue measure
of A. Based on these considerations, one may use Lebesgue measure on [0, 1) to
define a probability model for infinitely many tosses of a fair coin. As we will see
below, this is an essentially unique choice. For now, let us exploit the model with
an illustration of the Borel–Cantelli Lemma 1. Fix a nondecreasing sequence rn of
positive integers and let An = {x ∈ [0, 1) : ωk (x) = 1, k = n, n + 1, . . . , n +rn − 1}
denote the event that a run of 1’s occurs of length at least rn starting at the nth toss.
Note
∞ that this set is a union of length 2−rn . Thus, if rn increases so quickly that
−rn
n=1 2 < ∞ then the Borel–Cantelli Lemma 1 yields that P(An i.o.) = 0. For a
concrete illustration, let rn = [θ log2 n], for fixed θ > 0, with [·] denoting the integer
part. Then P(An i.o.) = 0 for θ > 1. Analysis of the case 0 < θ ≤ 1 requires more
detailed consideration of the fundamental notion of “statistical independence” of the

outcomes of the individual tosses implicit to this model. This concept is among the
most important in all of probability theory and will be precisely defined in the next
chapter.
Remark 1.1 A detailed construction of Lebesgue measure is given in Example 1

of Appendix A. The existence of Lebsegue measure on [0, 1) plays a fundamental
role in providing the probability space for repeated unending tosses of a fair coin in
the previous example. The existence of probability models corresponding to infinite
sequences of experiments is as fundamentally important to probability as existence
of Lebesgue measure is to analysis. A general existence theorem will be given in
Chapter IX that will cover the theory developed in the chapters leading up to it. For
now we generally take such existence theory for granted.
For a given collection C of subsets of Ω, the smallest σ-field that contains all of the
events in C is called the σ-field generated by C and is denoted by σ(C); if G is any σ-
field containing C then σ(C) ⊂ G. Note that, in general, if Fλ , λ ∈ Λ, is an arbitrary
collection of σ-fields of subsets of Ω, then λ∈Λ Fλ := {F ⊂ Ω : F ∈ Fλ ∀λ ∈ Λ}
is a σ-field. On the other hand λ∈Λ Fλ := {F ⊂ Ω : F ∈ F λ for some λ ∈ Λ}
is not generally a σ-field.
Define the join σ-field, denoted by λ∈Λ Fλ , to be the
σ-field generated by λ∈Λ Fλ .
It is not uncommon that F = σ(C) for a collection C closed under finite
intersections; such a collection C is called a π-system, e.g., Ω = (−∞, ∞),
C = {(a, b] : −∞ ≤ a ≤ b < ∞}, or infinite sequence space Ω = R∞ , and
C = {(a1 , b1 ] × · · · × (ak , bk ] × R∞ : −∞ ≤ ai ≤ bi < ∞, i = 1, . . . , k, k ≥ 1}.
A λ-system is a collection L of subsets of Ω such that (i) Ω ∈ L, (ii) If A ∈ L then
Ac ∈ L, (iii) If An ∈ L, An ∩ Am = ∅, n = m, n, m = 1, 2, . . . , then ∪n An ∈ L. A
σ-field is clearly also a λ-system. The following π-λ theorem provides a very useful
tool for checking measurability.
Theorem 1.1 (Dynkin’s π-λ Theorem) If L is a λ-system containing a π-system C,

then σ(C) ⊂ L.
Proof Let L(C) = ∩F, where the intersection is over all λ-systems F containing
C. We will prove the theorem by showing (i) L(C) is a π-system, and (ii) L(C)
is a λ-system. For then L(C) is a σ-field (see Exercise 15), and by its definition
σ(C) ⊂ L(C) ⊂ L. Now (ii) is simple to check. For clearly Ω ∈ F for all F,
and hence Ω ∈ L(C). If A ∈ L(C), then A ∈ F for all F, and since every F is a
λ-system, Ac ∈ F for every F. Thus Ac ∈ L(C). If An ∈ L(C), n ≥ 1, is a disjoint
sequence, then for each F, An ∈ F, for all n and A ≡ ∪n An ∈ F for all F. Since
this is true for every λ-system F, one has A ∈ L(C). It remains to prove (i). For
each set A, define the class L A := {B : A ∩ B ∈ L(C)}. It suffices to check that
L A ⊃ L(C) for all A ∈ L(C). First note that if A ∈ L(C), then L A is a λ-system,
by arguments along the line of (ii) above (Exercise 15). In particular, if A ∈ C, then
A ∩ B ∈ C for all B ∈ C, since C is closed under finite intersections. Thus L A ⊃ C.
This implies, in turn that L(C) ⊂ L A . This says that A ∩ B ∈ L(C) for all A ∈ C and
for all B ∈ L(C). Thus, if we fix B ∈ L(C), then L B ≡ {A : B ∩ A ∈ L(C)} ⊃ C.

Therefore L B ⊃ L(C). In other words, for every B ∈ L(C) and A ∈ L(C), one has
A ∩ B ∈ L(C).
In view of the additivity properties of a probability, the following is an immediate
and important corollary to the π-λ theorem.
Corollary 1.2 (Uniqueness) If P1 , P2 are two probability measures such that
P1 (C) = P2 (C) for all events C belonging to a π-system C, then P1 = P2 on
all of F = σ(C).
Proof Check that {A ∈ F : P1 (A) = P2 (A)} ⊃ C is a λ-system.
Remark 1.2 It is rather simple to construct examples of generating collections of
sets C and probability measures P1 , P2 such that P1 = P2 on C, but P1 = P2
on σ(C). For example take Ω = {1, 2, 3, 4}, C = {{1, 2, 3}, {2, 3, 4}}. Then
σ(C) = {{1, 2, 3}, {2, 3, 4}, {1}, {4}, {1, 4}, {2, 3}, Ω, ∅}. Let P1 ({1}) = P2 ({1}) =
P1 ({4}) = P2 ({4}) = 1/8, but P1 ({2}) = P2 ({3}) = 1/8, P1 ({3}) = P2 ({2}) = 5/8.
For a related application suppose that (S, ρ) is a metric space. The Borel σ-field
of S, denoted by B(S), is defined as the σ-field generated by the collection C = T of
open subsets of S, the collection T being referred to as the topology on S specified by
the metric ρ. More generally, one may specify a topology for a set S by a collection
T of subsets of S that includes both ∅ and S, and is closed under arbitrary unions
and finite intersections. Then (S, T ) is called a topological space and members of
T define the open subsets of S. The topology is said to be metrizable when it may
be specified by a metric ρ as above. In any case, one defines the Borel σ-field by
B(S) := σ(T ).
Definition 1.1 A class C ⊂ B(S) is said to be measure-determining if for any two
finite measures μ, ν such that μ(C) = ν(C) ∀C ∈ C, it follows that μ = ν on B(S).
One may directly apply the π-λ theorem, noting that S is both open and closed, to
see that the class T of all open sets is measure-determining, as is the class K of all
closed sets.
If (Si , Si ), i = 1, 2, is a pair of measurable spaces then a function f : S1 → S2
is said to be a measurable map if f −1 (B) := {x ∈ S1 : f (x) ∈ B} ∈ S1 for
all B ∈ S2 . In usual mathematical discourse the σ-fields required for this defini-
tion may not be explicitly mentioned and will need to be inferred from the context.
For example, if (S, S) is a measurable space, by a Borel-measurable function
f : S → R is meant measurability when R is given its Borel σ-field. A ran-
dom variable, or a random map, X is a measurable map on a probability space
(Ω, F, P) into a measurable space (S, S). Measurability of X means that each event2
[X ∈ B] := X −1 (B) belongs to F ∀ B ∈ S. The σ-field generated by X, denoted
σ(X ), is the smallest σ-field of subsets of Ω for which X : Ω → S is measurable. In
2 Throughout, this square-bracket notation will be used to denote events defined by inverse images.
particular, therefore, σ(X ) = {[X ∈ A] : A ∈ S} (Exercise 11). The term random

variable is most often used to denote a real-valued random variable, i.e., where
S = R, S = B(R). When S = Rk , S = B(Rk ), k > 1, one uses the term random
vector.
A common alternative to the use of a metric to define a metric space topology, is
to indirectly characterize the topology by specifying what it means for a sequence to
converge in the metric. That is, if T is a topology on S, then a sequence {xn }∞
n=1 in S
converges to x ∈ S with respect to the topology T if for arbitrary U ∈ T such that
x ∈ U , there is an N such that xn ∈ U for all n ≥ N . A topological space (S, T ),
or a topology T , is said to be metrizable if T coincides with the class of open sets
defined by a metric ρ on S. Alternatively, by specifying the meaning of convergence
in the metric, one has that closed sets, and therefore open sets via complements,
can also be defined. Using this notion, other commonly occurring measurable image
spaces may be described as follows: (i) S = R∞ —the space of all sequences of
reals with the (metrizable) topology of pointwise convergence, and S = B(R∞ ),
(ii) S = C[0, 1]—the space of all real-valued continuous functions on the interval
[0, 1] with the (metrizable) topology of uniform convergence, and S = B(C[0, 1]),
and (iii) S = C([0, ∞) : Rk )—the space of all continuous functions on [0, ∞) into
Rk , with the (metrizable) topology of uniform convergence on compact subsets of
[0, ∞), S = B(S) (see Exercise 10).
The relevant quantities for a random map X on a probability space (Ω, F, P) are
the probabilities with which X takes sets of values. In this regard, P determines the
most important aspect of X , namely, its distribution Q ≡ P ◦ X −1 defined on the
image space (S, S) by
Q(B) := P(X −1 (B)) ≡ P(X ∈ B), B ∈ S. (1.4)
The distribution is sometimes referred to as the induced measure of X under P.

For random vectors X = (X 1 , . . . , X k ) with values in Rk , it is often convenient to
restrict consideration to the (multivariate) distribution function defined by F(x) =
P(X ≤ x) ≡ P(X 1 ≤ x1 , . . . , X k ≤ xk ), x = (x1 , . . . , xk ) ∈ Rk ; see Exercise 16. A
familiar and important special case is that of an absolutely continuous distribution
function given by
xk x1
F(x) = ··· g(u)du, x ∈ Rk ,
−∞ −∞
for a nonnegative density function g with respect to Lebesgue measure on Rk ; here

we have used the convention of representing Lebesgue measure as du. In an abuse of
terminology, a random variable with an absolutely continuous distribution is often
referred to as a continuous random variable.
If a real-valued random variable X has the distribution function F, then P(X ∈
(a, b]) = P(a < X ≤ b) = F(b)− F(a). Moreover, P(X ∈ (a, b)) = P(a < X < b)
= F(b− ) − F(a). Since the collection C of all open intervals (a, b), −∞ < a ≤ b <
∞ is closed under finite intersections, and every open subset of R can be expressed
as a countable disjoint union of sets in C, the collection C is measure-determining (cf.
Exercise 19). One may similarly check that the (multivariate) distribution function of
a probability Q on the Borel sigma-field of Rk uniquely determines Q; cf. Exercise
19.
In general, let us also note that given any probability measure Q on a measurable
space (S, S) one can construct a probability space (Ω, F, P) and a random map X on
(Ω, F) with distribution Q. The simplest such construction is given by letting Ω = S,
F = S, P = Q, and X the identity map, X (ω) = ω, ω ∈ S. This is often called a
canonical construction, and (S, S, Q) with the identity map X is called a canonical
model. Note that any canonical model for X will generally be a noncanonical model
for a function of X . So it would not be prudent to restrict the theoretical development
to canonical models alone!
Before proceeding, it is of value to review the manner in which abstract Lebesgue
integration and, more specifically, mathematical expectation is defined. Throughout
1 A denotes the indicator
function of the set A, i.e., 1 A (x) = 1 if x ∈ A, and is zero
otherwise. If X = mj=1 a j 1 A j , A j ∈ F, Ai ∩ A j = ∅(i = j), is a discrete random
variable
m or, equivalently, a simple random variable, then EX ≡ Ω X d P :=
j=1 ja P(A j ). If X : Ω → [0, ∞) is a random variable, then EX , expected
is defined by the “simple function approximation” EX ≡ Ω X d P := sup{EY :
0 ≤ Y ≤ X, Y simple}. In particular, one may apply the standard simple function
approximations X = limn→∞ X n given by the nondecreasing sequence

n2 n
−1
j
X n := 1[ j2−n ≤X <( j+1)2−n ] + n1[X ≥n] , n = 1, 2, . . . , (1.5)
j=0
2n
to write
⎧ n ⎫
⎨n2−1
j ⎬
EX = lim EX n = lim −n −n
P( j2 ≤ X < ( j + 1)2 ) + n P(X ≥ n) .
n→∞ n→∞ ⎩ 2 n ⎭
j=0
(1.6)
Note that if EX < ∞, then n P(X > n) → 0 as n → ∞ (Exercise 30). Now, more
generally, if X is a real-valued random variable, then the expected value (or, mean,
first moment) of X is defined as

E(X ) ≡ X d P := EX + − EX − , (1.7)
Ω
provided at least one of E(X + ) and E(X − ) is finite, where X + = X 1[X ≥0] and X − =
−X 1[X ≤0] . If both EX + < ∞ and EX − < ∞, or equivalently, E|X | = EX + +
EX − < ∞, then X is said to be integrable with respect to the probability P. Note
that if X is bounded a.s., then applying (1.5) to X + and X − , one obtains a sequence
X n (n ≥ 1) of simple functions that converge uniformly to X , outside a P-null set

(Exercise 1.5(i)).
If X is a random variable with values in (S, S) and if h is a real-valued Borel-
measurable function on S, then using simple function approximations to h, one may
obtain the following basic change of variables formula

E(h(X )) ≡ h(X (ω))P(dω) = h(x)Q(d x), (1.8)
Ω S
where Q is the distribution of X , provided one of the two indicated integrals may be
shown to exist.
For arbitrary p ≥ 1, the order p-moment of a random variable X on (Ω, F, P)
having distribution Q is defined by

μ p := EX = p
X (ω)P(dω) =
p
x p Q(d x), (1.9)
Ω R
provided that X p is integrable, or nonnegative. Moments of lower order p than one,

including negative order ( p < 0) moments, may be defined similarly so long as X p
is real-valued random variable. Moments of absolute values |X | are referred to as
absolute moments of X . Let us record a useful formula for the moments of a random
variable derived from the Fubini–Tonelli theorem before proceeding. Namely,
Proposition 1.3 If X is a random variable on (Ω, F, P), then for any p > 0,
∞
E|X | p = p y p−1 P(|X | > y)dy. (1.10)
0
x
Proof For x ≥ 0, simply use x p = p 0 y p−1 dy in the formula
|X (ω)|
E|X | =
p
|X (ω)| P(dω) =
p
p y p−1
dy P(dω)
Ω Ω 0
and apply the Tonelli part (a) to reverse the order of integration. The assertion
follows.
Example 4 As a generalization of Example 2, suppose that X 1 , X 2 , . . . is a sequence

of positive random variables, each having distribution Q, with a finite moment of
order p > 0. Then an application of Borel–Cantelli I together with Proposition 1.3
1 p
shows that P(X n > n p i.o.) = 0 (Use Exercise 29 applied to X 1 .).
If X = (X 1 , X 2 , . . . , X k ) is a random vector whose components are integrable

real-valued random variables, then define E(X ) = (E(X 1 ), . . . , E(X k )). Similarly
for complex valued random variables X = U + i V , where U, V are integrable real-
valued random variables, one defines EX = EU + iEV . In particular, √ for complex
valued random variables EX exists if and only if E|X | = E U 2 + V 2 < ∞;
Exercise 36.
This definition of expectation as an integral in the sense of Lebesgue is precisely

the same as that used in real analysis to define S f (x)μ(d x) for a real-valued Borel-
measurable function f on an arbitrary measure space (S, S, μ); see Appendix A.
Almost sure convergence of a sequence of random maps X n , n ≥ 1, to X , each
defined on (Ω, F, P), is defined by X n (ω) → X (ω) as n → ∞ for all ω ∈ Ω up to
a subset of probability zero; i.e., convergence almost everywhere with respect to P.
One may exploit standard tools of real analysis (see Appendices A and C), such as
Lebesgue’s dominated convergence theorem, Lebesgue’s monotone convergence
theorem, Fatou’s lemma, Fubini–Tonelli theorem, Radon–Nykodym theorem,
for estimates and computations involving expected values.
The following lemma and proposition illustrate the often used exchange in the
order of integration.
Lemma 2 (Integration by parts) Let μ1 , μ2 be signed measures on R, which are

finite on finite intervals. Let
Fi (y) = μi (0, y], i = 1, 2, −∞ < y < ∞.
Then for any −∞ < a < b < ∞, one has

F1 (y)μ2 (dy) = F1 (b)F2 (b) − F1 (a)F2 (a) − F2 (y−)μ1 (dy).
(a,b] (a,b]
Proof Since a signed measure may be expressed as the difference of two measures,
without loss of generality it is sufficient to let both μ1 and μ2 be measures that are
finite on finite intervals. Then, using the Fubini–Tonelli theorem, one has

μ1 (du)μ2 (dv) = [F1 (v) − F1 (a)]μ2 (dv)
a<u≤v,a<v≤b a<v≤b

= −F1 (a)[F2 (b) − F2 (a)] + F1 (v)μ2 (dv).
a<v≤b
Also,

μ1 (du)μ2 (dv) = [F2 (b) − F2 (u−)]μ1 (du)
a<u≤v,a<v≤b a<u≤b

= F2 (b)[F1 (b) − F1 (a)] − F2 (u−)μ1 (du).
a<u≤b
Comparing these two iterations yields the asserted formula.
Remark 1.3 The “distribution functions” Fi , i = 1, 2, can be defined as Fi (y) =

μi ((c, y]), i = 1, 2, y ∈ R, for any real number c in place of zero, and the lemma
still holds. This formula has special utility when applied to a nondecreasing function,
or more generally a function of bounded variation, as an integrand.
The following is a useful version for expected values. A clever application is given
in Theorem 3.4.
Proposition 1.4 Let μ1 be an arbitrary measure on (0, ∞] which is finite on finite

intervals, and such that μ1 ({0}) = 0. Suppose that μ2 is a probability measure on
[0, ∞), and let Y be a random variable with distribution μ2 . Then, with Fi (y) =
μi ((0, y]), i = 1, 2, y ≥ 0, one has

EF1 (Y ) = P(Y ≥ y)μ1 (dy)
[0,∞)
Proof By the lemma, for any b > 0 one has

F1 (y)μ2 (dy) = F1 (b)F2 (b) − F1 (0)F2 (0) − F2 (y−)μ1 (dy)
(0,b] (0,b]

= F1 (b)F2 (b) − F2 (y−)μ1 (dy)
(0,b]

= [F2 (b) − F2 (y−)]μ1 (dy). (1.11)
(0,b]
The assertion follows by letting b ↑ ∞ on both sides, and using Lebesgue’s monotone
convergence theorem to obtain

F1 (y)μ2 (dy) = [1 − F2 (y−)]μ1 (dy).
(0,∞) (0,∞)
Definition 1.2 A sequence {X n }∞ n=1 of random variables on a probability space

(Ω, F, P) is said to converge in probability to a random variable X if for each
ε > 0, limn→∞ P(|X n − X | > ε) = 0. The convergence is said to be almost sure
(a.s.) if the event [X n → X ] ≡ {ω ∈ Ω : X n (ω) → X (ω)} has P-measure zero.
Convergence in probability is referred to as “convergence in measure” in analysis;

see Appendix A. Note that almost sure convergence always implies convergence in
probability, since for arbitrary ε > 0 one has 0 = P(∩∞ ∞
n=1 ∪m=n [|X m − X | >
∞
ε]) = limn→∞ P(∪m=n [|X m − X | > ε]) ≥ lim supn→∞ P(|X n − X | > ε); also see
Exercise 5. An equivalent formulation of convergence in probability can be cast in
terms of almost sure convergence as follows.
Proposition 1.5 A sequence of random variables {X n }∞ n=1 on (Ω, F, P) converges

in probability to a random variable X on (Ω, F, P) if and only if every subsequence
has an a.s. convergent subsequence to X .
Proof Suppose that X n → X in probability as n → ∞. Let {X n k }∞ k=1 be a subse-

quence, and for each m ≥ 1 recursively choose n k(0) = 1, n k(m) = min{n k > n k(m−1) :
P(|X n k − X | > 1/m) ≤ 2−m }. Then it follows from the Borel–Cantelli lemma (Part
I) that X n k(m) → X a.s. as m → ∞. For the converse suppose that X n does not
converge to X in probability. Then there exists ε > 0 and a sequence n 1 , n 2 , . . . such
that limk P(|X n k − X | > ε) = α > 0. Since a.s. convergence implies convergence
in probability (see Appendix A, Proposition 2.4), there cannot be an a.s. convergent
subsequence of {X n k }∞
k=1 .
The utility of Proposition 1.5 can be seen, for example, in demonstrating that if
a sequence of random variables X n , n ≥ 1, say, converges in probability to X , then
X n2 will converge in probability to X 2 by virture of continuity of the map x → x 2 ,
and considerations of almost sure convergence; see Exercise 6.
The notion of measure-determining classes of sets extends to classes of functions
as follows. Let μ, ν be arbitrary finite measures on the Borel σ-field of a metric space
S. A class Γ of real-valued bounded Borel-measurable functions on S is measure-
determining if S g dμ = S g dν ∀g ∈ Γ implies μ = ν.
Proposition 1.6 The class Cb (S) of real-valued bounded continuous functions on S

is measure-determining.
Proof To prove this, it is enough to show that for each (closed) F ∈ K there exists
a sequence of nonnegative functions { f n } ⊂ Cb (S) such that f n ↓ 1 F as n ↑
∞. Since F is closed, one may view x ∈ F in terms of the equivalent condition
that ρ(x, F) = 0, where ρ(x, F) := inf{ρ(x, y) : y ∈ F}. Let h n (r ) = 1 − nr
for 0 ≤ r ≤ 1/n, h n (r ) = 0 for r ≥ 1/n. Then take f n (x) = h n (ρ(x, F)). In
particular, 1 F (x) = limn f n (x), x ∈ S, and Lebesgue’s dominated convergence
theorem applies.
Note that the functions f n in the proof of Proposition 1.6 are uniformly continu-
ous, since | f n (x) − f n (y)| ≤ (nρ(x, y)) ∧ (2 supx | f n (x)|). It follows that the set
U Cb (S) of bounded uniformly continuous real-valued functions on S is measure-
determining. Measure-determining classes of functions are generally actually quite
extensive. For example, since the Borel σ-field on R can be generated by classes
of open intervals, closed intervals, half-lines etc., each of the corresponding class
of indicator functions 1(a,b) , −∞ ≤ a < b < ∞, 1[a,b] , −∞ < a < b < ∞,
1(−∞,x] , x ∈ R, is measure-determining (see Exercises 9, 16).
Consider the L p -space L p (Ω, F, P) of (real-valued) random variables X such
that E|X | p < ∞. When random variables that differ only on a P-null set are iden-
tified, then for p ≥ 1, it follows from Theorem 1.7(e) below that L p (Ω, F, P) is a
1
normed linear space with norm X p := ( Ω |X | p d P) p ) ≡ (E|X | p ) p . It is in this
sense that elements of L p (Ω, F, P) are, strictly speaking, represented by equiva-
lence classes of random variables that are equal almost surely. It may be shown that
with this norm (and distance X − Y p ), it is a complete metric space, and therefore
a Banach space (Exercise 35). In particular, L 2 (Ω, F, P) is a Hilbert space with
inner product (see Appendix C)

1
X, Y = EX Y ≡ X Y d P, ||X ||2 = X, X 2 . (1.12)
Ω
The L 2 (S, S, μ) spaces are the only Hilbert spaces that are required in this text,
where (S, S, μ) is a σ-finite measure space; see Appendix C for an exposition of
the essential structure of such spaces. Note that by taking S to be a countable set
with counting measure μ, this includes the l 2 sequence space. Unlike the case of a
measure space (Ω, F, μ) with an infinite measure μ, for finite measures it is always
true that
L r (Ω, F, P) ⊂ L s (Ω, F, P) if r > s ≥ 1, (1.13)
as can be checked using |x|s < |x|r for |x| > 1. The basic inequalities in the
following Theorem 1.7 are consequences of convexity at some level. So let us be
precise about this notion.
Definition 1.3 A function ϕ defined on an open interval J is said to be a convex
function if ϕ(ta + (1 − t)b) ≤ tϕ(a) + (1 − t)ϕ(b), for all a, b ∈ J , 0 ≤ t ≤ 1.
If the function ϕ is sufficiently smooth, one may use calculus to check convexity, see
Exercise 24. The following lemma is required to establish a geometrically obvious
“line of support property” of convex functions.
Lemma 3 (Line of Support) Suppose ϕ is convex on an interval J . (a) If J is open,

then (i) the left-hand and right-hand derivatives ϕ− and ϕ+ exist and are finite and
nondecreasing on J , and ϕ− ≤ ϕ+ . Also (ii) for each x0 ∈ J there is a constant
m = m(x0 ) such that ϕ(x) ≥ ϕ(x0 ) + m(x − x0 ), ∀x ∈ J . (b) If J has a left (or right)
endpoint and the right-hand (left-hand) derivative is finite, then the line of support
property holds at this endpoint x0 .
Proof (a) In the definition of convexity, one may take a < b, 0 < t < 1. Thus
convexity is equivalent to the following inequality with the identification a = x,
b = z, t = (z − y)/(z − x): For any x, y, z ∈ J with x < y < z,
ϕ(y) − ϕ(x) ϕ(z) − ϕ(y)

≤ . (1.14)
y−x z−y
More generally, use the definition of convexity to analyze monotonicity and bounds
on the Newton quotients (slopes of secant lines) from the right and left to see that
(1.14) implies ϕ(y)−ϕ(x)
y−x
≤ ϕ(z)−ϕ(x)
z−x
≤ ϕ(z)−ϕ(y)
z−y
(use the fact that c/d ≤ e/ f for
d, f > 0 implies c/d ≤ (c + e)/(d + f ) ≤ e/ f ). The first of these inequalities
shows that ϕ(y)−ϕ(x)
y−x
decreases as y decreases, so that the right-hand derivative ϕ+ (x)
exists and ϕ(y)−ϕ(x)
y−x
≥ ϕ+ (x). Letting z ↓ y in (1.14), one gets ϕ(y)−ϕ(x)
y−x
≤ ϕ+ (y)
for all y ∈ J . Hence ϕ+ is finite and nondecreasing on J . Now fix x0 ∈ J . By taking
x = x0 and y = x0 in turn in these two inequalities for ϕ+ , it follows that ϕ(y) −
ϕ(x0 ) ≥ ϕ+ (x0 )(y − x0 ) for all y ≥ x0 , and ϕ(x0 ) − ϕ(x) ≤ ϕ+ (x0 )(x0 − x) for all
x ≤ x0 . Thus the “line of support” property holds with m = ϕ+ (x0 ). (b) If J has
a left (right) endpoint x0 , and ϕ+ (x0 ) (ϕ− (x0 )) is finite, then the above argument
remains valid with m = ϕ+ (x0 ) (ϕ− (x0 )).
A similar proof applies to the left-hand derivative ϕ− (x) (Exercise 24). On letting
x ↑ y and z ↓ y in (1.14), one obtains ϕ− (y) ≤ ϕ+ (y) for all y. In particular, the
line of support property now follows for ϕ− (x0 ) ≤ m ≤ ϕ+ (x0 ).
Theorem 1.7 (Basic Inequalities) Let X, Y be random variables on (Ω, F, P).

(a) (Jensen’s Inequality) If ϕ is a convex function on the interval J and P(X ∈ J ) = 1,
then ϕ(EX ) ≤ E(ϕ(X )) provided that the indicated expectations exist. More-
over, if ϕ is strictly convex, then equality holds if and only if X is a.s. constant.
1 1
(b) (Lyapounov Inequality) If 0 < r < s then (E|X |r ) r ≤ (E|X |s ) s .
(c) (Hölder Inequality) Let p ≥ 1. If X ∈ L p , Y ∈ L q , 1p + q1 = 1, then X Y ∈ L 1
1 1
and E|X Y | ≤ (E|X | p ) p (E|Y |q ) q .
(d) (Cauchy–Schwarz
√ √ Inequality) If X, Y ∈ L 2 then X Y ∈ L 1 and one has |E(X Y )|
≤ EX EY .
2 2
(e) (Minkowski Triangle Inequality) Let p ≥ 1. If X, Y ∈ L p then X + Y p ≤

X p + Y p .
(f) (Markov and Chebyshev-type Inequalities) Let p ≥ 1. If X ∈ L p then P(|X | ≥
E(|X | p 1[|X |≥λ] )
λ) ≤ λp
≤ E|X
λp
|p
, λ > 0. More generally, if h is a nonnegative
increasing function on an interval containing the range of X , then P(X ≥ λ) ≤
E(h(X )1[X ≥λ] )/ h(λ).
Proof The proof of Jensen’s inequality hinges on the line of support property of
convex functions in Lemma 3 by taking x = X (ω), ω ∈ Ω, x0 = EX . The Lya-
s
pounov inequality follows from Jensen’s inequality by writing |X |s = (|X |r ) r , for
s
0 < r < s, since ϕ(x) = x r is convex on [0, ∞). For the Hölder inequality, let
p, q > 1 be conjugate exponents in the sense that 1p + q1 = 1. Using convex-
ity of the function exp(x) one sees that |ab| = exp(ln(|a| p )/ p + ln(|b|q )/q)) ≤
1
p
|a| p + q1 |b|q . Applying this to a = X|X|p , b = Y|Y|q and integrating, it fol-
1 1
lows that E|X Y | ≤ (E|X | p ) p (E|Y |q ) q . The Cauchy–Schwarz inequality is the
Hölder inequality with p = q = 2. For the proof of Minkowski’s inequality, first
use the inequality (1.27) to see that |X + Y | p is integrable from the integrabil-
ity of |X | p and |Y | p . Applying Hölder’s inequality to each term of the expansion
E(|X | + |Y |) p = E|X |(|X | + |Y |) p−1 + E|Y |(|X | + |Y |) p−1 , and solving the result-
ing inequality for E(|X | + |Y |) p (using conjugacy of exponents), it follows that
X + Y p ≤ X p + Y p . Finally, for the Markov and Chebyshev-type inequali-
|X | p 1{|X |≥λ} )
≤ |Xλ p| on Ω, taking expectations
p
ties simply observe that since 1{|X |≥λ} ≤ λ p
E(|X | p 1[|X |≥λ] )
yields P(|X | ≥ λ) ≤ λp
≤ E|X
λp
|p
, λ > 0. More generally, for increasing
h with h(λ) > 0, one has E(h(X )1[X ≥λ] ) ≥ h(λ)P(X ≥ λ).
Remark 1.4 One may note that the same proof may be used to check that correspond-
ing formulations of both the Hölder and the Minkowski inequality for functions on
arbitrary measure spaces can be verified with the same proof as above.
The Markov inequality refers to the case p = 1 in (f). Observe from the proofs
that (c–e) hold with the random variables X, Y replaced by measurable functions, in
fact complex valued, on an arbitrary (not necessarily finite) measure space (S, S, μ);
see Exercise 36.
Example 5 (Chebyshev Estimation of a Distribution Function) Suppose that X is a
nonnegative random variable with distribution function F, and having finite moments
μ p of order p < s for some s > 1. According to the Chebyshev inequality one has
μp
1 − F(x) = P(X > x) ≤ . (1.15)
xp
1 1
By Liapounov’s inequality one also has μ pp ≤ μ p+1 p+1

, p = 1, 2, . . . . Since p →
μp μ
log μ p is convex on [0, s] (Exercise 23), it follows that μ p−1 ≤ μp+1
p
. So
μp μ p+1 x p+1 μ p+1 μ p+1

≤ p+1 ⇐⇒ ≤ ⇐⇒ x ≤ . (1.16)
x p x xp μp μp
Thus upper bound estimates of 1 − F(x) may be obtained as follows:

⎧
⎪
⎪ 1 if x ≤ μ1 ,
⎪
⎪
⎨ μ2 if μ1 < x ≤ μ2
,
x2 μ1
F(x) := 1 − F(x) ≤ (1.17)
⎪
⎪ ···
⎪
⎪
⎩ μ pp if
μp
<x≤
μ p+1
,1 ≤ p ≤ s − 1.
x μ p−1 μp
From here one also has Chebyshev estimated lower bounds on F(x) in terms of the
moments μ p of X1 (Exercise 2).3
Suppose that (S, S, μ) is an arbitrary measure space and g : S → [0, ∞) a
Borel-measurable function, though not necessarily integrable. One may use g as a
density with respect to μ to define another measure ν on (S, S), i.e., with g as its
Radon–Nykodym derivative dν/dμ = g, also commonly denoted by dν = g dμ,
and meaning that ν(A) = A g dμ, A ∈ S; see Appendix C for a full treatment of
the Radon–Nikodym theorem.
Recall that a sequence of measurable functions {gn }∞
n=1 on S is said to converge
μ-a.e. to a measurable function g on S if and only if μ({x ∈ S : limn gn (x) =
g(x)}) = 0. The following simple theorem finds diverse uses in the framework of
probability theory.
Theorem 1.8 (Scheffé) Let (S, S, μ) be a measure space and suppose that ν, {νn }∞ n=1
are measures on (S, S) with respective nonnegative densities g, {gn }∞
n=1 with respect
to μ, such that
3 For an application see Bhattacharya, R.N., Kim, H., Majumdar, M.K. (2015): Sustainability in the
Stochastic Ramsey Model, J. Quant. Econ. 13, 169–184.


gn dμ = g dμ < ∞, ∀n = 1, 2, . . . .
S S
If gn → g as n → ∞, μ-a.e., then

sup | g dμ − gn dμ| ≤ |g − gn | dμ → 0, as n → ∞.
A∈S A A S
Proof The indicated bound on the supremum follows from the triangle inequality
for integrals. Since S (g − gn ) dμ = 0 for each n, S (g − gn )+ dμ = Ω (g − gn )− dμ.
In particular, since |g − gn | = (g − gn )+ + (g − gn )− ,

|g − gn | dμ = 2 (g − gn )+ dμ.
S S
But 0 ≤ (g − gn )+ ≤ g. Since g is μ-integrable, one obtains S (g − gn )

+
dμ → 0 as
n → ∞ from Lebesgue’s dominated convergence theorem.
Remark 1.5 Suppose gn , g are probability densities (with respect to a σ-finite mea-
sure μ) and gn → g in μ-measure. Then the conclusion of Theorem 1.8 holds.
For a measurable space (S, S), a useful metric (see Exercise 33) defined on the
space P(S) of probabilities on S = B(S) is furnished by the total variation distance
defined by
dv (μ, ν) := sup{|μ(A) − ν(A)| : A ∈ B(S)}, μ, ν ∈ P(S). (1.18)
Proposition 1.9 Suppose that (S, S) is a measurable space. Then

1
dv (μ, ν) = sup f dμ − f dν : f ∈ B(S), | f | ≤ 1 ,
2 S S
where B(S) denotes the space of bounded Borel-measurable functions on S. More-

over, (P(S), dv ) is a complete metric space.
Proof Let us first establish the formula for the total variation distance. By standard
simple function approximation it suffices to consider bounded simple functions in
k
the supremum. Fix arbitrary μ, ν ∈ P(S). Let f = i=1 ai 1 Ai ∈ B(S) with |ai | ≤
1, i = 1, . . . , k and disjoint sets Ai ∈ S, 1 ≤ i ≤ k. Let I + := {i ≤ k : μ(Ai ) ≥
ν(Ai )}. Let I − denote the complementary set of indices. Then by definition of the
integral of a simple function and splitting the sum over I ± one has upon twice using
the triangle inequality that

f dμ − f dν ≤ |ai |(μ(Ai ) − ν(Ai )) + |ai |(ν(Ai ) − μ(Ai ))

S S i∈I + i∈I −

≤ (μ(Ai ) − ν(Ai )) + (ν(Ai ) − μ(Ai ))
i∈I + i∈I −
= μ(∪i∈I + Ai ) − ν(∪i∈I + Ai ) + ν(∪i∈I − Ai ) − μ(∪i∈I − Ai )
≤ 2 sup{|μ(A) − ν(A)| : A ∈ S}. (1.19)
On the other hand, taking f = 1 A − 1 Ac , A ∈ S, one has

f dμ − f dν = |μ(A) − μ(Ac ) − ν(A) + ν(Ac )|

S S
= |μ(A) − ν(A) − 1 + μ(A) + 1 − ν(A)|
= 2|μ(A) − ν(A)|. (1.20)
Thus, taking the supremum over sets A ∈ S establishes the asserted formula for the
total variation distance. Next, to prove that the space P(S) of probabilities is complete
for this metric, let {μn }∞
n=1 be a Cauchy sequence in P(S). Since the closed interval
[0, 1] of real numbers is complete, one may define μ(A) := limn μn (A), A ∈ S.
Because this convergence is uniform over S, it is simple to check that μ ∈ P(S) and
μn → μ in the metric dv ; see Exercise 33.
For real-valued random variables X n , n ≥ 1, and X , having absolutely continuous

distribution functions with densities gn , g, say, with respect to Lebesgue measure, a
notion of convergence in distribution can be defined as Fn (x) ≡ P(X n ≤ x) →
F(x) ≡ P(X ≤ x) for all x ∈ R. It follows from Scheffé’s theorem that pointwise
convergence a.e. of the densities implies uniform convergence of the distribution
functions; Exercise 25. For contrast in the absence of a density see Exercise 26. A
treatment of convergence in distribution is the subject of Chapter V.
One may also note that Scheffé’s theorem provides conditions under which con-
vergence in measure implies L 1 (S, S, μ)-convergence of the densities gn to g, and
convergence in the total variation metric of the probabilities νn to ν.
We will conclude this chapter with some additional basic convergence theorems
for probability spaces. Namely, we consider L p -convergence ( p ≥ 1) of a sequence
of random variables X n , n ≥ 1 in L p (Ω, F, P), to X , i.e., E|X n − X | p → 0 as
n → ∞. We start with p = 1. Of course |EX n − EX | ≤ E|X n − X |, n ≥ 1, so that
L 1 -convergence implies convergence of the expected values.
For this purpose we require a definition.
Definition 1.4 A sequence {X n }∞
n=1 of random variables on a probability space
(Ω, F, P) is said to be uniformly integrable if
lim sup E{|X n |1[|X n |≥λ] } = 0.

λ→∞ n
Theorem 1.10 (L 1 -Convergence Criterion) Let {X n }∞n=1 be a sequence of random

variables on a probability space (Ω, F, P), X n ∈ L 1 (n ≥ 1). Then {X n }∞
n=1 con-
verges in L 1 to a random variable X if and only if (i) X n → X in probability as
n → ∞, and (ii) {X n }∞
n=1 is uniformly integrable.
Proof (Necessity) If X n → X in L 1 then convergence in probability (i) follows from

the Markov inequality. Also

|X n |d P ≤ |X n − X |d P + |X |d P
[|X n |≥λ] [|X n |≥λ] [|X n |≥λ]

≤ |X n − X |d P + |X |d P
Ω [|X |≥λ/2]

+ |X |d P. (1.21)
[|X |<λ/2,|X n −X |≥λ/2]
The first term of the last sum goes to zero as n → ∞ by hypothesis. For each λ > 0
the third term goes to zero by the dominated convergence theorem as n → ∞. The
second term goes to zero as λ → ∞ by the dominated convergence theorem too.
Thus, there are numbers n(ε) and λ(ε) such that for all λ ≥ λ(ε),

sup |X n |d P ≤ ε. (1.22)
n≥n(ε) [|X n |≥λ]
Since a finite sequence of integrable random variables {X n : 1 ≤ n ≤ n(ε)} is always

uniformly integrable, it follows that the full sequence {X n } is uniformly integrable.
(Sufficiency) Under the hypotheses (i), (ii), given ε > 0 one has for all n that

|X n |d P ≤ |X n |d P + λ ≤ ε + λ(ε) (1.23)
Ω [|X n |≥λ]
for sufficiently large λ(ε). In particular, { Ω |X n |d P}∞

n=1 is a bounded sequence. Thus
Ω |X |d P < ∞ since, using Fatou’s lemma, one has Ω |X |d P = Ω lim n |X n |d P
≤ lim inf n Ω |X n |d P < ∞. Now

|X n − X |d P = |X n − X |d P
[|X n −X |≥λ] [|X n −X |≥λ,|X n |≥λ/2]

+ |X n − X |d P
[|X n |<λ/2,|X n −X |≥λ]

≤ |X n |d P + |X |d P
[|X n |≥λ/2] [|X n −X |≥λ]

λ
+ ( + |X |)d P. (1.24)
[|X n |<λ/2,|X n −X |≥λ] 2
Another random document with
no related content on Scribd:
British property should be paid, and threatening, if further piracies
were committed, to send a force into the Rif to chastise his rebellious
subjects.
No attention was paid to this edict, for though the Rifians
acknowledge the Sultan of Morocco as ‘Kaliph[24] Allah,’ H.M. being
a direct descendant from the Prophet, and though they allow a
governor of Rif extraction to be appointed by him to reside amongst
them, they do not admit of his interference in the administration of
government or in any kind of legislation, unless it happens he is
voluntarily appealed to in cases of dispute.
The Rifians, however, pay annually a small tribute, which is
generally composed of mules and honey, the latter article being
much cultivated on the extensive tracts of heather in the Rif
mountains. This tribute is collected by the Governor and transmitted
to the Sultan.
After a lengthened correspondence with the Moorish Court,
negotiations were closed by the Sultan declaring he had no power of
control over the mountainous districts in the Rif, and therefore
declining to be held responsible for the depredations committed on
vessels approaching that coast. The British Government then
dispatched a squadron to Gibraltar under Admiral Sir Charles Napier,
with orders to embark a regiment at that garrison, and to proceed to
the Rif coast to chastise the lawless inhabitants.
On his arrival at the Spanish fort of Melilla, which is about fifty
miles to the westward of the Algerian frontier, Sir Charles called on
the Spanish Governor and requested him to invite the chiefs of the
neighbouring villages to come to Melilla to meet him.
On their arrival, the Admiral demanded compensation for the
losses sustained by the owner of the British vessels which had been
captured. The Rifians cunningly evaded discussion by replying that
they could not accede to demands which did not emanate from the
Sultan, whose orders they declared they would be prepared to obey.
Sir Charles accepted these vague assurances[25]; and with this

unsatisfactory result returned with the squadron to Gibraltar, and
addressed to me a communication, making known the language held
to him by the Rifians, and requesting that I would dispatch an
express courier to the Moorish Court to call upon the Sultan to give
the requisite orders to the Rifians who, he declared, were prepared
to obey, though he admitted he was ignorant of the names of the
chieftains with whom he had the parley.
In my reply to the Admiral I expressed my belief that the Rifians
had cunningly given these vague assurances to induce him to depart
with his ships from their coast, and that I apprehended the Sultan
would express his surprise that we should have been led to suppose
that the piratical and rebellious inhabitants of the Rif coast would pay
compensation or give other satisfaction, in pursuance of any orders
which H.S.M. might issue.
In this sense, as I had expected, the Sultan replied to my note;
holding out, however, a hope, which had been expressed in past
years, that he would seek at a more favourable moment to make the
Rif population, who had been from time immemorial in a semi-
independent state, more subservient to his control.
Some months after the squadron had returned to England, a
British vessel, becalmed off the village of Benibugaffer, was taken by
a Rifian piratical craft, and the English crew were made captives.
Tidings having reached Gibraltar of the capture of the British ship,
a gunboat was sent to Melilla to endeavour to obtain, through the
intervention of the Spanish authorities and an offer of a ransom, the
release of the British sailors, but this step was not attended with
success. Having heard that the Englishmen who had been captured
had been presented by the pirates to a Rif Marábet (or holy man)
named Alhádari, who resided on the coast, and as I had in past
years been in friendly communication with this person regarding
some Rifians who had proceeded in a British vessel to the East on a
pilgrimage to Mecca, and had been provided by me with letters of
recommendation to British Consular officers, I wrote him a friendly
letter, expressing the indignation I felt at the outrages which had
been committed by his piratical brethren on British vessels; that I had
been informed the authorities at Gibraltar had endeavoured, when
they heard British sailors were in the hands of the pirates, to pay a
ransom for their freedom, but had failed, as exorbitant demands had
been put forward; and that since I had learnt my countrymen were in
his hands, I felt satisfied they would be well treated, and that he
would facilitate at once their release and return to Gibraltar; that I
entertained too high an opinion of him to suppose he would not
consent to their release except on the payment of a ransom, and
therefore I would make no offer to purchase the liberty of my
countrymen, but renewed those assurances of friendship and
goodwill, of which I said I had already given proof in the past
treatment of his brethren.
Alhádari replied that the sailors were under his care, had been
well treated, and would be embarked in the first vessel which might
be sent to receive them.
This engagement was faithfully executed, and at my suggestion
the authorities at Gibraltar sent a suitable present to the worthy
Marábet. I wrote also to thank Alhádari, and to beg that he would use
his influence to put a stop to the disgraceful outrages committed in
past years by his brethren on the lives and property of British
subjects, and to say that I should probably take an opportunity of
seeking to have a parley with the chiefs, in the hope of coming to an
understanding with them to bring about a cessation of these
outrages; adding, that if my friendly intervention did not put a stop to
the piracy of his brethren, the British Government would be
compelled, in concert with the Sultan, to resort to hostile measures
on a large scale, and send forces by sea and land to chastise these
rebellious subjects of His Sherifian Majesty.
In the spring of 1856 H.M. frigate Miranda, Captain Hall, arrived at
Tangier with directions to convey me to the coast of Rif, and I
embarked on April 21, taking with me a Rifian friend, Hadj Abdallah
Lamarti, who was Sheikh of a village near Tangier called Suanni,
whose inhabitants are Rifians, or of Rif extraction.
Hadj Abdallah had left the Rif in consequence of a blood feud. He
was the chief of the boar-hunters at Tangier, and was looked up to
with respect, not only by the rural population in the neighbourhood of
that town, who are chiefly of Rif extraction, but also by the local
authorities, who frequently employed him in the settlement of
disputes with the refractory tribes in the mountainous districts of the
Tangier province.
We steamed along the rocky coast of Rif and touched at the
Spanish garrisons of Peñon and Alhucema. The former is a curious
little rock, separated from the mainland by a very narrow channel. A
colonel and a few soldiers garrisoned the fortress, which is
apparently of no possible use, though the authorities at that time
might have aided in checking piracy by stopping the passage of the
Rif galleys. The rock is so small that there was not a walk fifty yards
long on any part of it.
On the island of Alhucema, so called from the wild lavender that
grows there, we also landed. The Spanish authorities were civil, but
held out no hopes of being able to take steps to put a stop to piracy.
This island is also an insignificant possession, about half a mile
distant from the mainland. The inhabitants had occasional
communication with the Rifians, hoisting a flag of truce whenever a
boat was dispatched to the shore; but Spaniards were not at that
time allowed to make excursions on the mainland, nor were they
permitted to obtain provisions except a few fowls, eggs, and honey.
On our arrival at Melilla, the Governor, Colonel Buceta[26],

received us courteously. I made known to him that the British
Government had directed me to proceed to the coast of Rif, to
endeavour to come to an understanding with the chiefs with the view
of putting a stop to piracy on that coast, the Sultan of Morocco
having declared he had no power of control over his lawless
subjects, who had shown an utter disregard of the peremptory orders
which had been issued to restore British property captured by their
piratical galleys; that in order to carry out this object I was anxious to
have an interview with some of the chiefs, not only of the villages on
the coast where the owners of the piratical galleys dwelt, but more
especially with the chiefs of the neighbouring inland villages, as the
latter derived no immediate benefit from the plunder of shipping.
Colonel Buceta endeavoured to dissuade me from this purpose,
reminding me that Sir Charles Napier had failed in obtaining any
beneficial result from his parley with the Rifians who had an interview
with him in Melilla.
Perceiving from the Governor’s language that he entertained
those feelings of jealousy which prevail with Spaniards regarding the
intervention of any foreign Government in the affairs of Morocco, I let
him understand that, should no beneficial result be obtained by my
visit in putting a stop to the outrages committed on merchant vessels
approaching the Rif coast, it would become a serious matter for the
consideration of our Government whether steps should not be taken
to inflict a chastisement on the Rifians by landing a force, and in
conjunction with the Sultan’s troops which might be dispatched, at
our instigation, for that purpose, to destroy the hamlets and boats on
the coast. The question might also arise, perhaps, of erecting a
fortress in some sheltered spot where a gunboat could be placed to
guard the coast against pirates, which I observed the authorities at
Spanish fortresses had hitherto been unable to effect.
This language sufficed to decide Colonel Buceta to accede to my
wishes; but he informed me that, in consequence of late acts of
aggression on the part of the natives, all communication with the
garrison had been cut off, and that no Rifians were allowed to enter;
it was therefore out of the question that he could admit any chieftains
into Spanish territory. Neither did he think the latter would be
disposed to venture into the gates of the fortress.
I then proposed to be allowed to dispatch my Rifian friend Hadj
Abdallah Lamarti with an invitation to some of the neighbouring
chiefs, both on the seaboard and inland, to meet me on the neutral
ground.
Colonel Buceta assented, but he repeated that he could not admit
any Rifians into the garrison, nor send an escort to accompany me,
should I pass the gates to go into the Rif country, adding that he
thought I should be incurring a serious risk of being carried off a
prisoner by the Rifians, if in the parley I should happen to express
myself in language such as I had used to him regarding the outrages
committed by these lawless people.
His predecessor, he informed me, in consequence of the frequent
hostilities which had taken place between the natives and the
garrison, had proposed to have a meeting with some chieftains
within the garrison. This they declined, fearing, as they alleged,
some act of treachery; but it was finally agreed that they should meet
the Governor on the neutral ground; that he could bring an escort of
twenty-five armed men, and that the chiefs would also be
accompanied by an equal number of followers; that the Governor
and one chief, both unarmed, were to advance to a central spot that
was selected about 150 yards distant from where their followers
assembled, and that the Spanish Governor could also bring with him
an interpreter.
This arrangement was carried out, and a Rifian chief, a man of
gigantic stature and herculean frame, advanced to meet the Spanish
Governor.
The parley commenced in a friendly manner; propositions were
made by each party regarding the conditions upon which peaceful
relations were to be re-established; but without bringing about any
result.
The Spanish Governor, finding the demands put forward by the
chieftain to be of an unacceptable character, expressed himself
strongly on the subject. A warm dispute ensued, and on the
Governor using some offensive expression, the Rifian seized in his
brawny arms the Governor, who was a little man, and chucking him
over his shoulders like a sack of grain, called out to the Spanish
detachment of soldiers to blaze away, and at the same time to his
own men to fire if the Spanish soldiers fired or attempted to advance,
whilst the chieftain ran off with the Governor, who was like a shield
on his back, to his followers.
The officer in command of the Spanish detachment, fearing that
the Governor might be killed, did not venture to let his men fire or
advance, and the Governor was carried off prisoner to a village
about three miles off on the hills, and notice was then sent to the
fortress that he would not be released until a ransom of 3000 dollars
was sent.
The Rifians kept the Governor prisoner until a reference was
made to Madrid, and orders were sent for the ransom to be paid.
‘Now,’ said Colonel Buceta, ‘your fate if you trust yourself to these
treacherous people will probably be the same, and I shall be quite
unable to obtain your release.’
I thanked the Governor for the advice, but declared that I must
fulfil my mission and was prepared to run all risks, having been
accustomed for many years to deal with Rifians at Tangier.
Buceta then consented that I should be allowed to pass the gates
of the garrison and invite the chiefs of the neighbouring Rif villages
to a parley on the neutral ground.
Colonel Buceta, a distinguished officer well known for his great
courage and decision, was I believe, on the whole, pleased that I
held to my purpose, though he warned me again and again that I
was incurring a great risk, and that in no manner could he intervene,
if I and the English officer who might accompany me were taken
prisoners.
My messenger returned and informed me that the neighbouring
chiefs, both of the inland and of the piratical villages of Benibugaffer,
would meet me on the neutral ground as had been proposed to
them.
Accompanied by Capt. Hall, who commanded H.M.’s frigate
Miranda, my friend Hadj Abdallah, and a ‘kavass,’ we proceeded to
the rendezvous.
Five or six chiefs awaited our advent, attended by some hundred
followers, stalwart fellows, many of them more than six feet high.
The chiefs wore brown hooded dresses, not unlike the costume of
a Franciscan friar; but part of the shirt-sleeves and front were
embroidered with coloured silks. Handsome leather-belts girded their
loins. A few of the elders wore white woollen ‘haiks,’ like unto the
Roman toga or mantle without seam, such as our Saviour is said to
have worn.
Some of the wild fellows had doffed their outer garments, carrying
them on their shoulders as they are wont to do when going to battle.
Their inner costume was a white cotton tunic, coming down to the
knees, with long wide sleeves fastened behind the back by a cord.
Around their loins each wore a leathern girdle embroidered in
coloured silk, from which on the one side hung a dagger and a small
pouch for bullets; while on the other was suspended a larger
leathern pouch or bag prettily embroidered and having a deep fringe
of leather, in which powder is carried; containing also a pocket to
carry the palmetto fibre, curiously enough called ‘lif,’ used instead of
wads over powder and ball. Their heads were closely shaved, except
that on the right side hung a long lock of braided hair, carefully
combed and oiled. Several of them were fair men with brown or red
beards, descendants perhaps of those Goths who crossed over into
Africa.
The wild fellows reclined in groups on a bank, immediately behind
where the chiefs were standing to receive us. After mutual greetings
I addressed them in Arabic, which though not the common language,
for Berber is spoken in the Rif, yet is understood by the better
classes, who learn to read the Koran and to write in the ‘jama’ or
mosque school. The Berber is not a written language.
‘Oh, men! I come amongst you as a friend; an old friend of the
Mussulmans. I have been warned that Rifians are not to be trusted,
and that I and those who accompany me are in danger of treachery;
but I take no heed of such warnings, for Rifians are renowned for
bravery, and brave men never act in a dastardly manner. My best
friends at Tangier are Rifians, or those whose sires came from the
Rif, such as my friend here, Hadj Abdallah Lamarti. They are my
hunters, and I pass days and nights with them out hunting, and am
treated by them and look upon them as my brethren; so here I have
come to meet you, with the Captain of the frigate, unarmed, as you
see, and without even an escort of my countrymen from the ship-of-
war lying there, or from the Spanish garrison, for I felt sure I should
never require protection in the Rif against any man.’
‘You are welcome,’ exclaimed the chiefs. ‘The English have
always been our friends,’ and a murmur of approval ran through the
groups of armed men seated on the bank.
‘Yes!’ I continued, ‘the English have always been the friends of the
Sultan, the ‘Kaliph Allah,’ and of his people.
‘You are all Mussulmans, and as followers of the Prophet every
year a number of your brethren, who have the means, go to the
shrine of the Prophet at Mecca, as required by your religion. How do
they go? In English vessels from Tangier, as you know, and they are
therefore, when on board, under the English flag and protection.
They are well treated and their lives and property are safe. They
return to Tangier in the same manner, and many of them have come
to me to express their gratitude for the recommendations I have
given them to English officers in the East, and the kindness they
have received at their hands.
‘These facts, I think, are known to you; but let us now consider
what is the conduct of certain Rifians,—not all, I am happy to add,
but those who dwell on the coast and possess ‘karebs,’ for the
alleged purpose of trade with Tangier and Tetuan, and for fishing.
‘The inhabitants of these coast villages, especially of the
neighbouring village of Benibugaffer, when they espy a peaceful
merchant vessel becalmed off their coast, launch a ‘kareb’ with forty
or fifty armed men, and set out in pursuit. The crews of these
merchant vessels are unarmed, and generally consist of not more
than eight or nine men. When they observe a ‘kareb’ approaching
with a hostile appearance, they escape in their little boats to the
open sea, trusting to Providence to be picked up by some passing
vessel before bad weather sets in, which might cause their small
craft to founder. The merchant vessel is then towed to the beach,
where she is stranded, pillaged of cargo and rigging, and burnt.
‘I now appeal to all true Mussulmans whether such iniquitous acts
are not against the laws of God and of the Prophet. These pirates
are not waging war against enemies or infidels, they are mere sea
robbers, who set aside the laws of the Prophet to pillage the
peaceful ships of their friends the English, to whom they are
indebted for conveying their brethren in safety to worship at the Holy
‘Kaaba’ of their Prophet.
‘To these English whom they rob, and also murder if they attempt
to resist, they are indebted for much of the clothing they wear, for the
iron and steel of which their arms are made, and for other
commodities. I now appeal to those Rifians who dwell in inland
villages, and who take no part in and have no profit from these
lawless acts, and I ask whether they will continue to tolerate such
infractions of Allah’s laws? Can these men of Benibugaffer who have
been guilty of frequent acts of piracy, can they be Mussulmans? No,
they must be “kaffers” (rebels against God).’ As I said this, I heard
from the mound behind me, where the Benibugaffer people were
seated, the sound of the cocking of guns, and a murmur, ‘He calls us
kaffers.’ Looking round, I perceived guns levelled at my back.
One of the elder Chiefs rose and cried out, ‘Let the English Chief
speak! What he says is true! Those who rob and murder on the seas
innocent people are not Mussulmans, for they do not obey the law of
God.’
I continued: ‘Hear what your wise Chief says. I fancied I heard a
sound like the click of a gun being cocked. Some foolish boys must
be sitting amongst the assembly, for no brave Rifians, Benibugaffers
included, would ever commit a cowardly murder on an unarmed man
who has come amongst you trusting to the honour and friendship
between the Rifians and English from ancient times.
‘You have, I think, heard that the English Government has
frequently complained to the Sultan Mulai Abderahman, the Kaliph
Allah and Emir El Mumenin (Prince of Believers), of the commission
of these outrages, and has put forward a demand for reparation and
compensation for damages.
‘The Sultan, who is the friend of the powerful Queen of England,
my Sovereign, under whose sway there are fifty million of
Mussulmans whom she governs with justice and kindness, issued
his Sherifian commands to you Rifians to cease from these outrages;
but you paid no attention to the orders of the Kaliph of the Prophet.
‘The Queen then sent a squadron to chastise the pirates and
obtain redress; but the Admiral took pity on the villages, where
innocent women and children dwelt, and did not fire a gun or burn a
‘kareb,’ as he might have done. He had a parley with the
Benibugaffer people and other inhabitants of villages where boats
are kept.
‘They made false promises and pretended they would cease to
commit outrages, but, as was to be expected, they have broken faith,
and since that parley have been guilty of further acts of piracy. So
now I have come to see you and hear whether the Rifians in the
inland villages will continue to suffer these outrages to be committed
by those who dwell on the coast, which may expose all the honest
and innocent inhabitants of the Rif to the horrors of war.
‘I have begged that no steps should be taken by my countrymen,
lest the innocent should suffer, until I make this final attempt to come
to an understanding with you; but I have to warn you, as a true
friend, if another outrage be committed, my great and powerful
Sovereign, in conjunction with the Sultan, will send large forces by
sea and by land to carry fire and sword into your villages, and bring
the whole population under subjection. H.S.M. may then think fit to
compel the Rif tribes dwelling on the coast to migrate to the interior
of his realms, or, at any rate, they will no longer be allowed to
possess a single boat for trade, or even for fishing.
‘I now ask—Will you inland inhabitants tolerate the continuance of
piracy on the part of your brethren on the coast?—Will you brave
inhabitants of the coast continue to set Allah’s laws at defiance, and
thus expose your lives and property, and those of your inland
brethren, to destruction?’
The old Chief again spoke, and others stood up and joined him,
saying: ‘He is right. We shall not allow these robberies to be
committed on our friends the English; such outrages must cease,
and if continued, we shall be prepared to chastise the guilty.’
The Benibugaffer Chiefs said, ‘We approve.’
‘I know,’ I continued, ‘you Rifians do not sign treaties or like
documents; but the words of brave men are more worthy of trust
than treaties, which are too often broken. Give me your hands.’ I
held out mine. As the pledge of good faith I shook the hands of the
chiefs, including the Benibugaffer.
‘Remember,’ I said, ‘it is not English vessels, but all vessels
without exception must be respected on approaching your shores.’
‘We agree,’ they cried.
Upon which I exclaimed, ‘I have faith in your words. May God’s
mercy and blessing be on you all and grant you prosperity and
happiness! The Rifians and English shall remain true friends for ever.
I bid you farewell.’
‘Stay,’ said the chief of a neighbouring village, ‘come with us and
be our guest. We shall kill an ox to feast you and our brethren here,
and bid you welcome. You are a hunter; we shall show you sport,
and become better acquainted with each other. Upon our heads shall
be your life and those of your friends.’
Pointing to the frigate, I said: ‘That vessel has to return
immediately, and I have to report what has been done, in order to
stop all preparations for seeking through other means to obtain the
satisfaction you have so readily offered. I should have been
delighted to have gone with you and should have felt as safe as if
amongst my own countrymen. You are a brave race, incapable of
doing a wrong to a true friend. I shall never forget the manner in
which you have received me.
‘I bid you all farewell. I believe in your promises, even those made
by the Benibugaffer. Send messengers at once to the villages on the
coast and let them know the promises you have made, which they
also must be required to carry out strictly.’
The Chiefs and their followers tried all they could to persuade me
to accompany them but finally consented that I should depart, on
promising that I would some day revisit them.
Colonel Buceta was surprised to learn the result of my visit, but
said the Rifians would never keep faith, and that we should soon
hear of fresh acts of piracy. ‘In such case,’ I replied, ‘we shall have to
land a force and burn every hamlet and boat on the coast; but I have
every hope the Rifians will keep faith.’
They have kept faith, and since that parley near Melilla no
vessels, either British or of other nationality, have been captured or
molested by the Rifians[27]
It was amongst these wild and lawless Rifians that Mr. Hay found
the most thorough sportsmen, and also men capable of great
attachment and devotion. Always much interested in the history of
this race, in their customs and mode of life, he wrote an interesting
account of the tribes which inhabit the north of Morocco and of his
personal intercourse with them.
The Rif province extends along the Mediterranean coast to the

eastward from a site called Borj Ustrak, in the province of Tetuan, for
about a hundred and fifty miles to the stream marked in maps as
‘Fum Ajrud’ (mouth of Ajrud), the northern boundary between
Morocco and Algiers.
The Rif country to the southward, inland from the Mediterranean
coast, extends about thirty-five miles and on the westward is
bordered by the Tetuan province and the mountains of Khamás and
Ghamára.
The population of Rif amounts, as far as can be calculated, to
about 150,000 souls. The Rifians are a Berber race, and have never
been conquered by the various nations—Phœnicians, Romans,
Goths, and Arabs—who have invaded Mauritania: they have always
maintained their independence; but on the conquest of Morocco by
the Arabs, the Rifians accepted the Mohammedan faith, and
acknowledged the Sovereigns of Morocco as the Kaliphs of the
Prophet.
The country is mountainous, the soil in most parts poor, and
though the Rif is rich in iron, copper, and other minerals, there are no
roads or means of conveyance to the seaboard. There are large
forests of ‘el aris[28],’ which the Rifians convey in their ‘karebs’
(sailing boats) to Tetuan and Tangier. They have no saws, so when a
tree is felled it is cut away with a hatchet until a beam or plank is
shaped, generally about ten feet long by a foot wide. This timber has
a strong aromatic odour, and when not exposed to damp is more
durable than oak. It was used for the woodwork of the Alhambra at
Granada and other Moorish palaces in Spain, and though many of
the Arabesque ornaments in plaster or stucco have fallen into decay
and walls have crumbled, this woodwork remains sound.
The Rifians are an industrious race; but their barren hills do not
produce sufficient grain to provide food for the population. Large
numbers migrate every year to different parts of Morocco, especially
to the northern provinces, and are employed to cultivate orchards
and gardens round Tangier and Tetuan. The majority of the
inhabitants of the town and neighbouring districts of Tangier are of
Rif extraction.
In the Rif the natives do not submit to any authority except upon
religious or legal questions, such as marriage, inheritance, and title
deeds. The ‘f’ki,’ or chief priest in a village mosque, draws up, with
the aid of ‘tolba’ or public notaries, all legal documents regarding
marriage or property. In other matters the Rifian does not submit to
legislation; his gun, pistol, and dagger are his judge and jury—yet
crimes such as robbery, theft, or outrages on women are rarely
known, but murder from feud is rife throughout the country to a
frightful extent. No man’s life is secure, even though he be a distant
relative, such as the great-grandson, of some one who may have
taken a life thirty years before in a blood feud. The widow of a
murdered man will teach her son, as soon as he can carry a gun or
pistol, how to use those arms, and daily remind him that his father
must be avenged lest the son be looked upon as despicable.
The men always go armed even in their own villages. Cursing,
swearing, or abusive language, so common amongst the Moors, are
rarely heard in Rif; for the man who ventures to use an opprobrious
epithet knows that he incurs the risk of being stabbed or shot. A
Rifian never forgives or forgets an insult.
They are distinguished for their courage. During the war between
Spain and Morocco in 1859, they did not obey the appeal of the
Sultan for assistance; but the inhabitants of the district of Zarhon
near Fas, who are of Rif extraction, sent a contingent of 1,500 men
to Tetuan. They arrived a few days before the battle of ‘Agraz’—the
last which took place between the Moors and Spaniards before the
peace of 1860—and fought so determinedly that two-thirds of their
number fell during that battle.
Polygamy is extremely rare in Rif. Few men venture to take a
second wife lest offence be given thereby to the father or brother of
either of the women they have married. Even in Tangier, where there
is a population of over 9,000 Mohammedans, chiefly Rifians by
descent, I never heard of more than four or five Moors who had two
wives. When an exception occurs, it has generally been at the
request of the wife, who, having had no child, begs her husband to
marry some cousin or friend, selected perhaps by herself.
Immoral conduct on the part of married women or maidens is
unknown; for, should they be suspected of leading an irregular life by
father, husband, or other male relative, such disgrace is wiped out by
death.
Rifian women do not cover their faces. If a man sees a young
woman fetching water from a well or walking alone, he will avoid
meeting her, and even turn back rather than run the risk of being
seen by some relative of the female and be suspected of having
communicated with her by word or gesture. He will shun the woman
who may be alone, as a modest girl in Europe might try to avoid a
man whom she should happen to meet when walking in some lonely
spot.
Some years ago an old Rifian, one of my boar-hunters, who dwelt
at a village near Tangier, presented himself before me looking very
miserable and haggard. ‘I take refuge under the hem of your
garment,’ he exclaimed, ‘and deliver into your hands these title-
deeds of my hut and garden, also a document regarding a mare;
these are all my possessions. I am about to deliver myself up to the
Basha of Tangier, Kaid Abbas Emkashéd, and to ask that I be sent to
prison.’
On inquiring of the old hunter why he thought of taking such an
extraordinary step, and also what he expected me to do with his
papers and property, he replied, whilst trembling from head to foot,
with tears running down his rugged cheeks and his teeth chattering
as he spoke, ‘My youngest daughter, whom I loved so dearly’—here
he gasped for breath—‘is no more. I have buried her. She was put to
death with my consent.’ Poor Hadj Kassim then covered his face and
sobbed violently, paused to recover himself, and continued, ‘The
authorities have heard that my daughter, who was very beautiful, has
disappeared, and have given orders that some innocent persons
who are suspected should be arrested, as it is supposed she has
been carried off or murdered. I cannot remain a passive spectator
whilst innocent men suffer, feeling that the whole blame of the
disappearance of my child rests on me alone. My daughter was of a
joyous character, and, like a silly girl, thought only of amusement.
Both her mother and I had repeatedly punished her for going to
weddings or other festivities without our permission. She had been
warned that misconduct on her part, as a Rifian maiden, would never
be forgiven; but she took no heed. Some neighbours reported that
she had been seen going to Tangier to dance in the “mesriahs.” Her
shameless conduct became a source of great scandal in the village,
and as it was supposed that I countenanced her misconduct, I was
shunned by my friends. They no longer returned my salams, and
when I joined the elders, who are wont to assemble of an afternoon
on our village green, they turned their backs on me.
‘Life had become a burden, and my son, who was also taunted by
young men for having a sister of bad repute, came to me yesterday,
when he heard that she had again gone off to the town, and declared
that as Rifians we could not allow a daughter and sister who did not
obey her parents, and brought disgrace on her family, to live.
‘Though I loved dearly my foolish child,’ continued the old hunter,
‘I gave way to the passionate language of my son, and consented
that, should we discover she danced at the “mesriah,” she should
die.
‘We went to Tangier and concealed ourselves near the entrance
of a “mesriah” we were told she frequented. We saw her enter,
followed by some young Moors. A little before sunset she came out,
enveloped in her “haik,” and walked hurriedly towards our village.
She did not see us, and we followed her until we reached a path in
the brushwood not far from our village, and then we stopped her. My
son accused her of leading a disgraceful life, and then struck her
heavily with a bill-hook on the head. She fell, never to speak again.
We buried her in a secluded spot. My son killed her, but I am really
her murderer—I alone am responsible for her death; but my
wretched child could not have lived to be a curse and a disgrace.’
Then the poor Hadj trembled in his acute misery, and shook as if he
had the palsy.
‘I shall,’ he continued, ‘present myself to the Basha. I shall not say
I am the murderer, as the Basha is a Rifian, and will understand all
when I declare I wish no man to be arrested on account of the
disappearance of my child, and that I alone am responsible for
whatever may have happened to her.
‘Now,’ he added, ‘you know, according to Moorish law, no man
can be punished for murder unless he acknowledges his crime, and
that after twelve months’ imprisonment, should no witnesses appear,
the accused can claim to be liberated from prison. If I live, therefore,
I shall be released; but I care no longer for life, except it be to work

Download textbook A Basic Course In Probability Theory 2Nd Edition Rabi Bhattacharya ebook all chapter pdf

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Download textbook A Basic Course In Probability Theory 2Nd Edition Rabi Bhattacharya ebook all chapter pdf

Uploaded by

Copyright:

Available Formats

A Basic Course in Probability Theory

2nd Edition Rabi Bhattacharya

Rabi N Bhattacharya Selected Papers 1st Edition Manfred

Golosa A Basic Course in Russian Book One 6th Edition

Foundations of the Theory of Probability 2nd Edition

Probability theory and statistical inference empirical

A Course in Functional Analysis and Measure Theory

Statistical independence in probability analysis number

A Second Course in Topos Quantum Theory 1st Edition

Stage Performance for Singers A Practical Course in 12

Universitext is a series of textbooks that presents material from a wide variety of

More information about this series at http://www.springer.com/series/223

A Basic Course in Probability

ISSN 0172-5939 ISSN 2191-6675 (electronic)

Mathematics Subject Classiﬁcation (2010): 60-xx, 60Jxx

1st edition: © Springer Science+Business Media, Inc. 2007

Printed on acid-free paper

This Springer imprint is published by Springer Nature

and stopping times in computations of certain important probabilities.

Tucson, AZ, USA Rabi Bhattacharya

In 1937, A.N. Kolmogorov introduced a measure-theoretic mathematical frame-

theorem. A careful proof of the Markov property is given for discrete-parameter

processes. The deﬁciency of Kolmogorov’s construction of a model for Brownian

March 2007 Rabi Bhattacharya

I Random Maps, Distribution, and Mathematical Expectation . . . . 1

XI Strong Markov Property, Skorokhod Embedding,

In the spirit of a refresher, we begin with an overview of the measure–theoretic

A bit more generally, if {An }∞

lim sup An := ∩n=1 ∪m≥n Am (1.2)

followed by subadditivity of the probability to get

detailed consideration of the fundamental notion of “statistical independence” of the

Remark 1.1 A detailed construction of Lebesgue measure is given in Example 1

Theorem 1.1 (Dynkin’s π-λ Theorem) If L is a λ-system containing a π-system C,

for all B ∈ L(C). Thus, if we fix B ∈ L(C), then L B ≡ {A : B ∩ A ∈ L(C)} ⊃ C.

particular, therefore, σ(X ) = {[X ∈ A] : A ∈ S} (Exercise 11). The term random

Q(B) := P(X −1 (B)) ≡ P(X ∈ B), B ∈ S. (1.4)

The distribution is sometimes referred to as the induced measure of X under P.

for a nonnegative density function g with respect to Lebesgue measure on Rk ; here

X n (n ≥ 1) of simple functions that converge uniformly to X , outside a P-null set

provided that X p is integrable, or nonnegative. Moments of lower order p than one,

Example 4 As a generalization of Example 2, suppose that X 1 , X 2 , . . . is a sequence

If X = (X 1 , X 2 , . . . , X k ) is a random vector whose components are integrable

This definition of expectation as an integral in the sense of Lebesgue is precisely

Lemma 2 (Integration by parts) Let μ1 , μ2 be signed measures on R, which are

Fi (y) = μi (0, y], i = 1, 2, −∞ < y < ∞.

Then for any −∞ < a < b < ∞, one has

Comparing these two iterations yields the asserted formula.

Remark 1.3 The “distribution functions” Fi , i = 1, 2, can be defined as Fi (y) =

Proposition 1.4 Let μ1 be an arbitrary measure on (0, ∞] which is finite on finite

Proof By the lemma, for any b > 0 one has

Definition 1.2 A sequence {X n }∞ n=1 of random variables on a probability space

Convergence in probability is referred to as “convergence in measure” in analysis;

Proposition 1.5 A sequence of random variables {X n }∞ n=1 on (Ω, F, P) converges

Proof Suppose that X n → X in probability as n → ∞. Let {X n k }∞ k=1 be a subse-

Proposition 1.6 The class Cb (S) of real-valued bounded continuous functions on S

Lemma 3 (Line of Support) Suppose ϕ is convex on an interval J . (a) If J is open,

ϕ(y) − ϕ(x) ϕ(z) − ϕ(y)

Theorem 1.7 (Basic Inequalities) Let X, Y be random variables on (Ω, F, P).

(e) (Minkowski Triangle Inequality) Let p ≥ 1. If X, Y ∈ L p then X + Y  p ≤

By Liapounov’s inequality one also has μ pp ≤ μ p+1 p+1

μp μ p+1 x p+1 μ p+1 μ p+1

Thus upper bound estimates of 1 − F(x) may be obtained as follows:

Stochastic Ramsey Model, J. Quant. Econ. 13, 169–184.

But 0 ≤ (g − gn )+ ≤ g. Since g is μ-integrable, one obtains S (g − gn )

(e) (Minkowski Triangle Inequality) Let p ≥ 1. If X, Y ∈ L p then X + Y p ≤