This action might not be possible to undo. Are you sure you want to continue?

BooksAudiobooksComicsSheet Music### Categories

### Categories

Scribd Selects Books

Hand-picked favorites from

our editors

our editors

Scribd Selects Audiobooks

Hand-picked favorites from

our editors

our editors

Scribd Selects Comics

Hand-picked favorites from

our editors

our editors

Scribd Selects Sheet Music

Hand-picked favorites from

our editors

our editors

Top Books

What's trending, bestsellers,

award-winners & more

award-winners & more

Top Audiobooks

What's trending, bestsellers,

award-winners & more

award-winners & more

Top Comics

What's trending, bestsellers,

award-winners & more

award-winners & more

Top Sheet Music

What's trending, bestsellers,

award-winners & more

award-winners & more

P. 1

N13_1p|Views: 4|Likes: 0

Published by Francisco Alfaro Medina

Published by: Francisco Alfaro Medina on Apr 29, 2014

Copyright:Traditional Copyright: All rights reserved

See more

See less

https://www.scribd.com/doc/221014314/N13-1p

04/29/2014

text

original

Driver

**Math 280 (Probability Theory) Lecture Notes
**

February 23, 2007 File:prob.tex

Contents

Part Homework Problems: -1 Math 280B Homework Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -1.1 Homework 1. Due Monday, January 22, 2007 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -1.2 Homework 2. Due Monday, January 29, 2007 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -1.3 Homework #3 Due Monday, February 5, 2007 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -1.4 Homework #4 Solutions (Due Friday, February 16, 2007) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -1.5 Homework #5 Solutions (Due Friday, February 23, 2007) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -1.6 Homework #6 Solutions (Due Friday, March 2, 2007) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Math 280A Homework Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.1 Homework 1. Due Friday, September 29, 2006 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.2 Homework 2. Due Friday, October 6, 2006 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.3 Homework 3. Due Friday, October 13, 2006 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.4 Homework 4. Due Friday, October 20, 2006 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.5 Homework 5. Due Friday, October 27, 2006 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.6 Homework 6. Due Friday, November 3, 2006 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.7 Homework 7. Due Monday, November 13, 2006 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.7.1 Corrections and comments on Homework 7 (280A) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.8 Homework 8. Due Monday, November 27, 2006 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.9 Homework 9. Due Noon, on Wednesday, December 6, 2006 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 3 3 3 3 3 5 5 5 5 5 5 5 5 5 6 6

0

Part I Background Material 1 2 Limsups, Liminfs and Extended Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Basic Probabilistic Notions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4

Contents

Part II Formal Development 3 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Set Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Algebraic sub-structures of sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Finitely Additive Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Finitely Additive Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Examples of Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Simple Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Simple Independence and the Weak Law of Large Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Constructing Finitely Additive Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Countably Additive Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Distribution Function for Probability Measures on (R, BR ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Construction of Premeasures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Regularity and Uniqueness Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Construction of Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Completions of Measure Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6 A Baby Version of Kolmogorov’s Extension Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 19 21 21 25 25 26 28 30 32 35 35 35 37 38 41 42

4

5

6

Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 6.1 Measurable Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 6.2 Factoring Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 π – λ and Monotone Class Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.1 The Monotone Class Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Basic Properties of Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.1 An Example of Ranks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Borel-Cantelli Lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Kolmogorov and Hewitt-Savage Zero-One Laws . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Integration Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 A Quick Introduction to Lebesgue Integration Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Integrals of positive functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3 Integrals of Complex Valued Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4 Densities and Change of Variables Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5 Measurability on Complete Measure Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.6 Comparison of the Lebesgue and the Riemann Integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 51 53 54 58 59 63 67 67 70 74 78 81 81 83

7

8

Page: 4

job: prob

macro: svmonob.cls

date/time: 23-Feb-2007/15:20

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 Exercises . . . . . . . . .1 Iterated Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . 12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 Weak Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4. . . . . . . . . . . . . . . . . . .3 Fubini’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8 Necessity Proof of Kolmogorov’s Three Series Theorem . . . . . . . . . . . . . . . 12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12. . . . . . . . . . . . . .7. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11. . . . . . . . . . .2. . . . . . . 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Laws of Large Numbers Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 Strong Law of Large Numbers . . . . . . . 13. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5 Uniform Integrability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11. . . . . . . . .4 Relationships between diﬀerent Lp – spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11. . . . . . . . . . . . . 84 9 Functional Forms of the π – λ Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12. . . . . . . . . . . . . . . . . H¨ older’s and Minikowski’s Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Random Series Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 Examples . . . . . . . . . . . . . 85 91 91 91 93 95 96 102 104 106 107 107 111 113 113 115 115 119 119 10 Multiple and Iterated Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Main Results . . . .5 Maximal Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 More Spherical Coordinates . . . . . . . . . . . . . . . .4 Fubini’s Theorem and Completions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12. . . . . . . . . 13. . . . . . . . . . . . . . . . . . . . . . . . 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 Completeness of Lp – spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 Kolmogorov’s Convergence Criteria and the SSLN . . . 12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Modes of Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 126 128 128 129 130 132 133 136 138 140 143 143 144 147 Page: 5 job: prob macro: svmonob. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5 Lebesgue Measure on Rd and the Change of Variables Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11. . . . . . . . . . . . . . . . . . . . . . . . . .2 A WLLN Example . . 13 Weak Convergence Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Part III Convergence Results 12 Laws of Large Numbers . . . . . . . . . . . . . . . . . . . . . . . .3 Strong Law of Large Number Examples . . . . . . . .6 The Polar Decomposition of Lebesgue Measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 Jensen’s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Lp – spaces . . . . . . . . . . . . . . . . . . . . . . 12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12. . . . . . .2. . . . . . . .7 Appendix: Convex Functions . . . . . . . .1 Total Variation Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Summary: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Contents 5 8. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .cls date/time: 23-Feb-2007/15:20 . . . . . . . . . . . . . . . . .3 “Derived” Weak Convergence . 12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 Tonelli’s Theorem and Product Measure . . . . . . . . .4 More on the Weak Laws of Large Numbers . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .cls date/time: 23-Feb-2007/15:20 . 213 Page: 6 job: prob macro: svmonob. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 Regular Conditional Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Inﬁnitely Divisible and Stable Symmetric Distributions . . . . . . . . . . . . . . . . . 193 17 The Radon-Nikodym Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 Part IV Conditional Expectations and Martingales 16 Hilbert Space Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 15. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 152 156 157 161 161 164 165 169 171 173 176 177 14 Characteristic Functions (Fourier Transform) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Compactness and Tightness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 Examples . . . . . . . . . . . . . . . . .6 13. . . . . . . . . 199 18 Conditional Expectation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 15. . . . . . 18. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18. . . . Weak Convergence Examples . . . . . . . 14. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 Additional Properties of Conditional Expectations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weak Convergence in Metric Spaces . . . . . . . . . . . . 14. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 Appendix: A Multi-dimensional Weirstrass Approximation Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Weak Convergence of Random Sums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 204 207 208 209 References . . . . . . . . . . . . . .3 Continuity Theorem . . . . . . . . . . . . . .4 13. . . . . . . . . . . . . . .6 Appendix: Bochner’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5 13. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14. . . . . . . . . . . . . . . . 14. . . . . . . . . . . . . . . . . . . . . . . . .1 Examples . . . . . . . . . . . . . . . .5 Exercises . . . . . . . . . . . . . . .4 Appendix: Standard Borel Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8 Appendix: Some Calculus Estimates . . . 18. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14. . . . . . . . . . . . . . . . . . . . . . . . .1 Basic Properties of the Characteristic Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14. . . . . . . . . . . . . . . . .4 A Fourier Transform Inversion Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 Skorohod and the Convergence of Types Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 Contents 13. .1 Stable Laws . . . . . . . . 18. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Part Homework Problems: .

.

d. Then use the two series theorem.12.28.9. please explain what convention you are using when the denominator is 0. 38 (Hint: make use Proposition 7.9 a-e.13 (Assume Var (Nn ) > 0 for all n. 5*.3. ] = 0.4 Homework #4 Solutions (Due Friday.2. {N (t)}t≥0 . 8.2 Homework 2.20.) -1. January 29.36.5. 7. 2007) • Look at Resnick Chapter 10: 11 • Do the following Exercises from the Lecture Notes: 12. 9. 3. 2.5. 18. 34 (assume n σn > 0).3 Homework #3 Due Monday. 1. For 6. 7.. 196 : 6. 35 (hint: show P [ξn = 0 i. 8.1 Homework 1. • Resnick Chapter 8: Hand in 8. 8. 8. Due Monday.7 • Hand in from p.1) and take X0 = U0 and then deﬁne Xn inductively so that Xn+1 = Xn · Un+1 . -1.4.27 • Hand in from p. 234–246: 7. Also you may ﬁnd it easier to show S n → 0 in L rather than the weaker notion of in probability. February 5. use the assumptions to bound E [Xn ] in terms of E[Xn : Xn ≤ x]. 2007) • Resnick Chapter 9: Look at: 9.) -1. (ii) if 0 ≤ u < v then N (v ) − N (u) has the Poisson distribution with parameter λ(v − u).25.36 • Resnick Chapter 8: Hand in 8.} be i.1.1. • Also hand in Exercise 13. 18. 9. • Resnick Chapter 7: look at 7.o. March 2.4a-d. . For 7. .6 Homework #6 Solutions (Due Friday. 1. 18. January 22. 18. (For 28b.14. assume E[Xi Xj ] ≤ ρ(i − j ) for 2 n i ≥ j. February 16. Due Monday. 9.3 from these notes. 7. 6. 8** In part 2b. 8. let {Un : n = 0.42 Hints and comments. 2007 • Resnick Chapter 8: Look at: 8. observe that Xn = σn N (0.7.33 • Resnick Chapter 9: Hand in 9. Hint: use problem 8. † -1. 1) . 2. 8.31.12. d -1.). *A Poisson process. with parameter λ satisﬁes (by deﬁnition): (i) N has independent increments .2. 2007) • Resnick Chapter 9: Look at: 8 2 • Resnick Chapter 9: Hand in 11. 7. 28.33. so that N (s) and N (t) − N (s) are independent.1. and 14.5 Homework #5 Solutions (Due Friday.3.31 ﬁrst). For 7. random variables uniformly distributed on (0.34 *Ignore the part of the question referring to the moment generating function.) • Hand in Exercise 13.4 • Resnick Chapter 10: Hand in 2† .-1 Math 280B Homework Problems -1.i.31 and the convergence of types theorem.7. 14. 2007 • Resnick Chapter 7: Hand in 7. 8. 114 : 4.13. 9.22. 7.10 • Also hand in Exercise from these notes: 14. 7. February 23.16. 2007 • Hand in from p.6..36 (assume each Xn is integrable!).2 from these notes.17. **Hint: use Exercise 12..30* (Due 8.

.

155–166: 13. 6. 26 • Hand in lecture note exercises: 8. 110–116: 9. 15 • Also hand in the following exercise. November 3. 19 Look at lecture notes: exercise 4. 155–166: 7 Hand in lecture note exercise: 7. 64 of Resnick. 14. October 20. Due Friday. compute the expected number of diﬀerent coupons collected after buying n boxes of cereal.20.6 Homework 6. So the derivative makes sense to compute for s ∈ (−1. Suppose {fn }n=1 is a sequence of Random Variables on some measurable space. 7. 34 Hand in from p. 36 • p.2. Referring to the setup in Problem 7 on p. B is in the σ – algebra. . 85–90: 3.1 Corrections and comments on Homework 7 (280A) Problem 21 in Section 5.2 (280A-4. 21 • Hand in from p. 8. Exercise 0. Birkhauser. 8. 2006 • Look at from p. 20-27: Look at: 9. 155–166: 6. 0.1). 0. 2006 • p. October 6. 1] . 27. Due Friday. 0. 63-70: 16 Hand in lecture note exercises: 4. 2006 • Look at from p.1 – 4. 15. 13 and the following problem. all problems are from Resnick. Let B be the set of ω such that fn (ω ) is convergent as n → ∞. 11. 6.7 Homework 7. October 27.19. 0. 40.7. 18. 1) with no qualiﬁcations. A Probability Path. 21. 30. 5. 25 Hand in from p.4 Homework 4. Due Friday.3 Homework 3. 6. Due Friday.1. 28. 2006 • • • • Look at from p.19.2. October 13.1 (280A-2. 63-70: Look at: 18 • p. November 13. 17. 7. i. 2006 • p. Due Monday. 85–90: 4. 155–166: 11. Show the set B is measurable. When s = 1 you should interpret the derivative as the one sided derivative d P (1) − P (1 − h) |1 P (s) := lim h↓0 ds h . 1] .5 Hand in from p. 29 Look at from p.1 and 5. 20-27: Hand in: 5. Exercise 0. 18. 110–116: 1. 63-70: 5.1. Due Friday. 2006 • • • • • Look at from p. k=1 k Note that P (s) = k=0 pk s is well deﬁned and continuous (by DCT) for s ∈ [−1. d P (s) = ds ∞ ∞ 0. 12. 63-70: Hand in: 3. 9. 8. 110–116: 3. 5.3. ∞ kpk sk−1 for s ∈ [0. 8. 19 0.1). 0. Due Friday.5 Homework 5.2 Homework 2. 23. 1999. 2006 • Look at from p. 41 0. 5 • Hand in from p.10 of Resnick should read. 12.e.0 Math 280A Homework Problems Unless otherwise noted. 8. September 29. S.4 and read Section 5. 37 • Hand in from p. 11. 17.1 Homework 1. 16. 110–116: 3.

8 Homework 8. 30 • Hand in from p. 195–201: 15 (Hint: |a − b| = 2(a − b)+ − (a − b).and you will need to allow for this limit to be inﬁnite in case k=1 kpk = ∞. (The assertion to prove is false without this assumption. 155–166: 14.j. p). Also note that Cov(X. 22a-b Hand in from p. 155–166: 19. 1−s Hint for Exercise 8. 0.20: Start by observing that Sn −µ n 4 ∞ E dµ = E = 1 n4 1 n n 4 (Xk − µ) k=1 n E [(Xk − µ)(Xj − µ)(Xl − µ)(Xp − µ)] .s. 17. Due Noon. 34. j. l. 33 and 18 (Also assume EXn = 0)* Hand in lecture note exercises: 9. please add the missing assumption that the random variables should have mean zero. 2006 • • • • • Look at from p. 27.l. 0. 195–201: 1a. 14. 234–246: 1. 195–201: 19. 16. 2006 • Look at from p.d.b. Due Monday. 12. E [(Xk − µ)(Xj − µ)(Xl − µ)(Xp − µ)] = 0. 195–201: 3. * For Problem 18. 24 Hand in from p. on Wednesday.) With this assumption. 18 (Hint: see picture given in class.9 Homework 9. Y ) = 0 is equivalent to E[XY ] = EX · EY. December 6. d In computing ds |1 P (s) .).p=1 Then analyze for which groups of indices (k. Var(X ) = E[X 2 ]. you may wish to use the fact (draw a picture or give a calculus proof) that 1 − sk increases to k as s ↑ 1.1. 38 Look at from p. 13. 15 . 2 (Hint: it is just as easy to prove a. convergence). 4. ) • Hand in from p. November 27. k.

Part I Background Material .

.

Suppose {an }n=1 and {bn }n=1 are convergent sequences in R then: and bn = n−α with α > 0 shows the necessity for assuming right hand side of Eq.3 Suppose that {xn }n=1 ⊂ R This shows that the requirement that the right side of Eq. ∞ 4. it Notation 1. Similarly by considering the examples an = n 1 lim inf xn = lim inf {xk : k ≥ n} and n→∞ n→∞ (1. (1. Let a := limn→∞ an and b = limn→∞ bn . Let us also suppose that a > 0 (the case a < 0 is handled similarly) and let α := min a 2 . Proof of Eq. n then limn→∞ an ≤ limn→∞ bn . Before going to the proof consider the simple example where an = n and bn = −αn with α > 0. 2. n→∞ for all n ≥ N. are left to the reader.1). The convention being that sup Λ = ∞ if ∞ ∈ Λ or Λ is not bounded from above and inf Λ = −∞ if −∞ ∈ Λ or Λ is not bounded from below. an bn ≥ M α for all n ≥ N. Liminfs and Extended Limits ¯ := R∪ {±∞} . Then ∞ if α < 1 0 if α = 1 lim (an + bn ) = −∞ if α > 1 while n→∞ lim an + lim bn “ = ”∞ − ∞. and 2. ∞ ¯ is a sequence of numbers. The cases where b = −∞ or a = ±∞ are handled similarly. limn→∞ (can ) = c limn→∞ an .2) provided the right hand side is not of the for ±∞ · 0 of 0 · (±∞) . it follows that limn→∞ (an + bn ) = a + b. there exists N such that bn ≥ M and an ≥ a − 1 for all n ≥ N and this implies an + bn ≥ M + a − 1 for all n ≥ N. ∞ ∞ ¯. ±∞ + a = ±∞ for any a ∈ R.2. b ∈ R.2. 1.1 Limsups. there exists N ∈ N such that an ≥ α and bn ≥ M for all n ≥ N and for this choice of N. A sequence an ∈ R all M ∈ R there exists m ∈ N such that an ≥ M (an ≤ M ) for all n ≥ m. then for every ε > 0 there exists N ∈ N such that |a − an | ≤ ε and |b − bn | ≤ ε for all n ≥ N. We use the following conventions. In this case. ±∞ · a = ∓∞ if a ∈ R with a < 0. 1 . Proof. If c ∈ R. ±∞ · 0 = 0. (1. Then Notation 1.1) is not of form ∞−∞ is necessary in Lemma 1.a. We will also use the conventions that sup ∅ = −∞ and inf ∅ = +∞.e. Lemma 1. It will be left to the reader to prove the case where lim an and lim bn exist in R. {an bn }n=1 is convergent and n→∞ lim (an bn ) = lim an · lim bn n→∞ n→∞ (1. I will only consider the case where a = limn→∞ an = 0 and limn→∞ bn = ∞ here. If a. n→∞ n→∞ .. ±∞ · a = ±∞ if a ∈ R with a > 0. If an ≤ bn for1 a. (1.1) Therefore. ¯ .1 The extended real numbers is the set R is R with two new points called ∞ and −∞. Case 1. i.2). Case 2. for every M > 0. Since n is arbitrary. The proofs of items 1. Since α > 0 is ﬁxed and M is arbitrary it follows that limn→∞ (an bn ) = ∞ as desired. Since M is arbitrary it follows that an + bn → ∞ as n → ∞.a.a. let sup Λ and inf Λ denote the least upper bound and For any subset Λ ⊂ R greatest lower bound of Λ respectively. n iﬀ there exists N < ∞ such that an ≤ bn for all n ≥ N. ∞ 3. So an ≤ bn a. lim sup xn = lim sup{xk : k ≥ n}. Proof of Eq. n” as an abreviation for almost all n. ∞ + ∞ = ∞ and −∞ − ∞ = −∞ while ¯ is said to converge to ∞ (−∞) if for ∞ − ∞ is not deﬁned.3) (1.2) is not of the form ∞ · 0. If {an + bn }n=1 is convergent and n→∞ lim (an + bn ) = lim an + lim bn n→∞ n→∞ (1. Given any M < ∞. suppose b = ∞ in which case we must assume a > −∞.4) Here we use “a. |a + b − (an + bn )| = |a − an + b − bn | ≤ |a − an | + |b − bn | ≤ 2ε provided the right side is not of the form ∞ − ∞. (1.

10

**1 Limsups, Liminfs and Extended Limits
**

n→∞

We will also write lim for lim inf n→∞ and lim for lim sup . Remark 1.4. Notice that if ak := inf {xk : k ≥ n} and bk := sup{xk : k ≥ n}, then {ak } is an increasing sequence while {bk } is a decreasing sequence. ¯ and Therefore the limits in Eq. (1.3) and Eq. (1.4) always exist in R lim inf xn = sup inf {xk : k ≥ n} and

n→∞ n n

a − ε ≤ inf {ak : k ≥ N } ≤ sup{ak : k ≥ N } ≤ a + ε, i.e. a − ε ≤ ak ≤ a + ε for all k ≥ N. Hence by the deﬁnition of the limit, limk→∞ ak = a. If lim inf n→∞ an = ∞, then we know for all M ∈ (0, ∞) there is an integer N such that M ≤ inf {ak : k ≥ N } and hence limn→∞ an = ∞. The case where lim sup an = −∞ is handled simin→∞

**lim sup xn = inf sup{xk : k ≥ n}.
**

n→∞

**The following proposition contains some basic properties of liminfs and limsups.
**

∞ Proposition 1.5. Let {an }∞ n=1 and {bn }n=1 be two sequences of real numbers. Then

larly. ¯ exists. If A ∈ R, then for Conversely, suppose that limn→∞ an = A ∈ R every ε > 0 there exists N (ε) ∈ N such that |A − an | ≤ ε for all n ≥ N (ε), i.e. A − ε ≤ an ≤ A + ε for all n ≥ N (ε). From this we learn that A − ε ≤ lim inf an ≤ lim sup an ≤ A + ε.

n→∞ n→∞

**¯ iﬀ 1. lim inf n→∞ an ≤ lim sup an and limn→∞ an exists in R
**

n→∞

**¯. lim inf an = lim sup an ∈ R
**

n→∞ n→∞ ∞ 2. There is a subsequence {ank }∞ k=1 of {an }n=1 such that limk→∞ ank = ∞ lim sup an . Similarly, there is a subsequence {ank }∞ k=1 of {an }n=1 such that n→∞ limk→∞

**Since ε > 0 is arbitrary, it follows that A ≤ lim inf an ≤ lim sup an ≤ A,
**

n→∞ n→∞

**ank = lim inf n→∞ an . lim sup(an + bn ) ≤ lim sup an + lim sup bn
**

n→∞ n→∞ n→∞

3. (1.5)

**i.e. that A = lim inf n→∞ an = lim sup an . If A = ∞, then for all M > 0
**

n→∞

whenever the right side of this equation is not of the form ∞ − ∞. 4. If an ≥ 0 and bn ≥ 0 for all n ∈ N, then lim sup(an bn ) ≤ lim sup an · lim sup bn ,

n→∞ n→∞ n→∞

there exists N = N (M ) such that an ≥ M for all n ≥ N. This show that lim inf n→∞ an ≥ M and since M is arbitrary it follows that ∞ ≤ lim inf an ≤ lim sup an .

n→∞ n→∞

(1.6)

The proof for the case A = −∞ is analogous to the A = ∞ case. Proposition 1.6 (Tonelli’s theorem for sums). If {akn }k,n=1 is any sequence of non-negative numbers, then

∞ ∞ ∞ ∞ ∞

provided the right hand side of (1.6) is not of the form 0 · ∞ or ∞ · 0. Proof. Item 1. will be proved here leaving the remaining items as an exercise to the reader. Since inf {ak : k ≥ n} ≤ sup{ak : k ≥ n} ∀ n, lim inf an ≤ lim sup an .

n→∞ n→∞

akn =

k=1 n=1 n=1 k=1

akn .

**Here we allow for one and hence both sides to be inﬁnite. Proof. Let
**

K N N K

**Now suppose that lim inf n→∞ an = lim sup an = a ∈ R. Then for all ε > 0,
**

n→∞

there is an integer N such that

M := sup

k=1 n=1

akn : K, N ∈ N

= sup

n=1 k=1

akn : K, N ∈ N

Page: 10

job: prob

macro: svmonob.cls

date/time: 23-Feb-2007/15:20

and L :=

∞

∞

akn .

k=1 n=1

Since

∞ ∞ K ∞ K N

L=

k=1 n=1

akn = lim

N n=1

K →∞

akn = lim

k=1 n=1

K →∞ N →∞

lim

akn

k=1 n=1

and

K k=1

**akn ≤ M for all K and N, it follows that L ≤ M. Conversely,
**

K N K ∞ ∞ ∞

akn ≤

k=1 n=1 k=1 n=1

akn ≤

k=1 n=1

akn = L

and therefore taking the supremum of the left side of this inequality over K and N shows that M ≤ L. Thus we have shown

∞ ∞

akn = M.

k=1 n=1

By symmetry (or by a similar argument), we also have that M and hence the proof is complete.

∞ n=1

∞ k=1

akn =

**2 Basic Probabilistic Notions
**

Deﬁnition 2.1. A sample space Ω is a set which is to represents all possible outcomes of an “experiment.” for N throws, and

N Ω = DR

for an inﬁnite number of throws. 5. Suppose we release a perfume particle at location x ∈ R3 and follow its motion for all time, 0 ≤ t < ∞. In this case, we might take, Ω = ω ∈ C ([0, ∞) , R3 ) : ω (0) = x . Deﬁnition 2.3. An event is a subset of Ω. Example 2.4. Suppose that Ω = {0, 1} is the sample space for ﬂipping a coin an inﬁnite number of times. Here ωn = 1 represents the fact that a head was thrown on the nth – toss, while ωn = 0 represents a tail on the nth – toss. Example 2.2. 1. The sample space for ﬂipping a coin one time could be taken to be, Ω = {0, 1} . 2. The sample space for ﬂipping a coin N -times could be taken to be, Ω = N {0, 1} and for ﬂipping an inﬁnite number of times, Ω = {ω = (ω1 , ω2 , . . . ) : ωi ∈ {0, 1}} = {0, 1} . 3. If we have a roulette wheel with 40 entries, then we might take Ω = {00, 0, 1, 2, . . . , 36} for one spin, Ω = {00, 0, 1, 2, . . . , 36} for N spins, and Ω = {00, 0, 1, 2, . . . , 36}

N N N N

1. A = {ω ∈ Ω : ω3 = 1} represents the event that the third toss was a head. 2. A = ∪∞ i=1 {ω ∈ Ω : ωi = ωi+1 = 1} represents the event that (at least) two heads are tossed twice in a row at some time. 3. A = ∩∞ N =1 ∪n≥N {ω ∈ Ω : ωn = 1} is the event where there are inﬁnitely many heads tossed in the sequence. 4. A = ∪∞ N =1 ∩n≥N {ω ∈ Ω : ωn = 1} is the event where heads occurs from some time onwards, i.e. ω ∈ A iﬀ there exists, N = N (ω ) such that ωn = 1 for all n ≥ N. Ideally we would like to assign a probability, P (A) , to all events A ⊂ Ω. Given a physical experiment, we think of assigning this probability as follows. Run the experiment many times to get sample points, ω (n) ∈ Ω for each n ∈ N, then try to “deﬁne” P (A) by P (A) = lim

N →∞

1 # {1 ≤ k ≤ N : ω (k ) ∈ A} . N

(2.1)

**for an inﬁnite number of spins. 4. If we throw darts at a board of radius R, we may take Ω = DR := (x, y ) ∈ R2 : x2 + y 2 ≤ R for one throw,
**

N Ω = DR

That is we think of P (A) as being the long term relative frequency that the ∞ event A occurred for the sequence of experiments, {ω (k )}k=1 . Similarly supposed that A and B are two events and we wish to know how likely the event A is given that we now that B has occurred. Thus we would like to compute: P (A|B ) = lim

n→∞

# {k : 1 ≤ k ≤ n and ωk ∈ A ∩ B } , # {k : 1 ≤ k ≤ n and ωk ∈ B }

let An = {ω ∈ Ω : ωn = 1} = {heads at time n} BN := ∪n≥N An = {at least one heads at time N or later} and ∞ B = ∩∞ N =1 BN = {An i. P (BN ) = 1 for all N. Assuming there exists a probability. the resulting probabilities are the same as if we only ﬂipped the coin k times.5. If A and B are disjoint event. This may be rewritten as P (A|B ) = lim = 1 n # {k : 1 ≤ k ≤ n 1 n→∞ n # {k : 1 ≤ k ≤ Example 2. 3. N 2 2 That is if we ignore the ﬂips after time k. P (A ∩ B ) P (A|B ) := . . The previous example suggests that if we ﬂip a fair coin an inﬁnite N number of times. and P is continuous under taking decreasing limits. Then there is no probability function. P (∅) = 1 and P (Ω ) = 1.14 2 Basic Probabilistic Notions which represents the frequency that A occurs given that we know that B has occurred. ωk ) = σ }) = k and ωk ∈ A ∩ B } n and ωk ∈ B } 1 2k (2. 2N Observe that this probability has the following property.o.e. that the continuity condition on P is equivalent to the σ – additivity of P. so that now Ω = {0. we are supposing that macro: svmonob. . for all A ⊂ Ω. P (A) . For example the limit may not exist. that c P (BN )≤ we see that P (B ) = lim P (BN ) = 1.3) that is to say eiθ N is the set N rotated counter clockwise by angle θ. 1 → 0 as M → ∞. Let S = {z ∈ C : |z | = 1} be the unit circle. i. P : 2Ω → [0. then we should deﬁne P ({ω ∈ Ω : (ω1 . let us point out three key properties that P should have. P (B ) There are of course a number of problems with this deﬁnition of P in Eq. But ignoring these technicalities for the moment. P (B ) Deﬁnition 2. i. Page: 14 job: prob . the probability of the event B where an inﬁnite number of heads are tossed. (2. we would like to compute. . P (A) ∈ [0. Assuming this we are then forced to deﬁne P (A) = 1 # (A) . deﬁne the conditional probability of A given B by. P (B ) > 0. then P (A ∪ B ) = lim 1 # {1 ≤ k ≤ N : ω (k ) ∈ A ∪ B } N →∞ N 1 = lim [# {1 ≤ k ≤ N : ω (k ) ∈ A} + # {1 ≤ k ≤ N : ω (k ) ∈ B }] N →∞ N = P (A) + P (B ) . using BN ↓ B. To try to compute this. We are going to use the fact proved below in Lemma . For z ∈ S and N ⊂ S let zN := {zn ∈ S : n ∈ N }. If B is a non-null event. Proof. By assumption. . Since c c BN = ∩n≥N Ac n ⊂ ∩M ≥n≥N An = {ω ∈ Ω : ωN = · · · = ωM = 1} .cls date/time: 23-Feb-2007/15:20 Example 2. For example we have the following negative theorem.2) P (A ∩ B ) . . . P : 2S → [0. for example.1) including the fact that it is not mathematical nor necessarily well deﬁned. (2. Let us consider the tossing of a coin N times with a fair coin. for any k ≥ 1 and σ ∈ {0. If we assume that P is continuous under taking decreasing limits we may conclude. .7.e.6. In this case we would expect that every ω ∈ Ω is equally likely. 1] such that P (S ) = 1. 2M −N Therefore. Suppose that σ ∈ k {0. 2. 1} . 1} is a given sequence.2) holds. Additivity. Theorem 2. 1] for all A ⊂ Ω. 1} . (2.8 (No-Go Theorem). 1. . ωk ) = σ }) = 1 1 · 2N −k = k . P ({ω }) = 21 N . 1] such that Eq. The unfortunate fact is that we can not always assign a desired probability function.} = ∩N =1 ∪n≥N An . then P ({ω : (ω1 . A ∩ B = AB = ∅. P is invariant under rotations. N →∞ Without this continuity assumption we would not be able to compute P (B ) . i.e.

Let R := {z = ei2πt : t ∈ Q} = {z = ei2πt : t ∈ [0. That is to say S= (rN ) (2. w ∈ S are equivalent if z = rw for some r ∈ R.6) We have thus arrived at a contradiction. By Eqs. Then every point z ∈ S may be uniquely written as z = nr with n ∈ N and r ∈ R. since the right side of Eq. . (2. 1 = P (S ) = r ∈R P (rN ) = r ∈R P (N ). So we are going to only deﬁne P on particular subsets. (2. (2.6) is either equal to 0 or to ∞ depending on whether P (N ) = 0 or P (N ) > 0.5) r ∈R where α Aα is used to denote the union of pair-wise disjoint sets {Aα } . To avoid this problem. where z.5). 1) ∩ Q} (2. We will developed this below. we are going to have to relinquish the idea that P should necessarily be deﬁned on all of 2Ω . As above R acts on S by rotations and divides S up into equivalence classes. B ⊂ 2Ω .4) – a countable subgroup of S.4) and (2. Choose (using the axiom of choice) one representative point n from each of these equivalence classes and let N ⊂ S be the set of these representative points.P (zN ) = P (N ) for all z ∈ S and N ⊂ S.

.

Part II Formal Development .

.

e. 1} is completely determined by the set A := {x ∈ X : a(x) = 1} ⊂ X.. x ∈ An x ∈ An .2 More generally if {Xα : α ∈ A} is a collection of non-empty sets. let 2X denote the power set of X – the collection of all subsets of X including the empty set. we will write Y N in place of Y {1. lim sup An := {An i.} iﬀ ∀N ∈ N ∃ n ≥ N and this may be expressed as {An i.a..o. we will say that f ∈ Y N is a sequence ∞ with values in Y and often write fn for f (n) and express f as {fn }n=1 . f2 . 2.a. let XA = Xα and πα : XA → Xα be the canonical projection map deﬁned α∈A We also deﬁne the symmetric diﬀerence of A and B by A B := (B \ A) ∪ (A \ B ) . We will also use F to stand for either of the ﬁelds R or C. then an element of a ∈ 2X = {0. In this way elements in {0. If If Xα = X for some ﬁxed space X.} = ∪∞ N =1 ∩n≥N An . R the real numbers.} = ∩∞ N =1 ∪n≥N An . Notation 3.e. Recall that an element x ∈ XA is a “choice function.2. N }. and C the complex numbers.. . As usual if {Aα }α∈I is an indexed collection of subsets of X we deﬁne the union and the intersection of this collection by ∪α∈I Aα := {x ∈ X : ∃ α ∈ I x ∈ Aα } and ∩α∈I Aα := {x ∈ X : x ∈ Aα ∀ α ∈ I }. let Y X denote the collection of all functions f : X → Y. then we will write α∈A Xα as X A rather than XA .” i. ∀ n ≥ N. .4 We will also write α∈I Aα for ∪α∈I Aα in the case that {Aα }α∈I are pairwise disjoint. If X = {1. .o. 1} .o.1 Set Operations Let N denote the positive integers. The axiom of choice states that XA = ∅ provided that Xα = ∅ for each α ∈ A.} as An inﬁnitely often and {An a. Notation 3.} iﬀ ∃N ∈N which may be written as {An a. .N } and denote f ∈ Y N by f = (f1 .o. For example ∞ let {An }n=1 be a sequence of subsets from X and deﬁne k≥n k≥n inf An := ∩k≥n Ak . B ⊂ X let B \ A := {x ∈ B : x ∈ / A} = A ∩ B c .. Q the rational numbers. . Notation 3. x ∈ {An a.} := {x ∈ X : # {n : x ∈ An } = ∞} n→∞ and lim inf An := {An a.} as An almost always. i. If X = N.3 Preliminaries 3.) Then x ∈ {An i. . n→∞ (One should read {An i. Aα ∩ Aβ = ∅ if α = β. . 1} are in one to one correspondence with subsets of X.} := {x ∈ X : x ∈ An for all n suﬃciently large}. an assignment xα := x(α) ∈ Xα for each α ∈ A. Notice that ∪ is closely related to ∃ and ∩ is closely related to ∀. For A ∈ 2X let Ac := X \ A = {x ∈ X : x ∈ / A} and more generally if A.a. Similarly.3 Given a set X.1 Given two sets X and Y. N0 := N∪ {0} be the non-negative integers and Z = N0 ∪ (−N) – the positive and negative integers including 0. The reason for writing the power set of X as 2X is that if we think of 2 X meaning {0. by πα (x) = xα . X sup An := ∪k≥n Ak . fN ) where fn = f (n). .a. Notation 3.

3. lim sup An = {x ∈ X : n→∞ c ∞ n=1 1 if x ∈ A 0 if x ∈ /A 1An (x) = ∞} . If f : N →X and g : N →Y are surjective functions. For each m ∈ N let am : N →Am be a surjective function and then deﬁne f : N × N → ∪∞ m=1 Am by f (m. then A = ∪∞ m=1 Am is countable. j ) : i + j = k } one at a time. i : 2N → 2X by setting i (A) := {xn : n ∈ N} ⊂ X for all A ⊂ N. 5. A set X is said to be countable if is empty or there is an injective function f : X → N. . N 6.. .5. 1. ) . Let us begin by showing 2N = {0. h. . .9 If f : X → Y is a function and E ⊂ 2Y let f −1 E := f −1 (E ) := {f −1 (E )|E ∈ E}. 1) (1. . 1) (2. However this is impossible since we have already seed that 2N is uncountable. 1} is uncountable. 2). The function f is surjective and hence so is the composition. f (n)}) . 2) (3. 2. . The function f : N → Λ deﬁned this way is a bijection. where h : N → N × N is the bijection deﬁned above. In particular 2X is uncountable for any inﬁnite set X. Notation 3. Moreover. For sake of N contradiction suppose f : N → {0. since X is an inﬁnite set we may ﬁnd an injective map x : N → X and use this to set up an injection. 3. Lemma 3. . . h(4) = (3. We end this section with some notation which will be used frequently in the sequel. then the function (f × g ) ◦ h : N →X × Y is surjective where (f × g ) (m. 1} by an := 1 − fn (n). 2) (1. let f (x) = min g −1 ({x}) = min {n ∈ N : f (n) = x} . In this way we may assume Y0 is a two point set which may as well be Y0 = {0. Since Λ is inﬁnite the process continues indeﬁnitely. 4. h(2) = (2. This contradicts the assumption that f is surjective and shows 2N is uncountable. Now deﬁne a ∈ {0. f2 (n) .8 (Basic Properties of Countable Sets). let f∗ G := {A ∈ 2Y |f −1 (A) ∈ G}. In short. of f to the subset A. 3) . n) := (f (m). Lemma 3.20 3 Preliminaries Deﬁnition 3. f ◦ h : N → ∪∞ m=1 Am . . . 3) .} . . For example let h (1) = (1. Let us ﬁrst construct a bijection. Conversely if f : X → N is injective let x0 ∈ X be a ﬁxed point and deﬁne g : N → X by g (n) = f −1 (n) for n ∈ f (X ) and g (n) = x0 otherwise. 6. If f : X → N is an injective map then so is the restriction. Proof. . 2. 4. 1.7. and then “count” these elements by counting the sets {(i. If g : N → X is a surjective map. g (n)) for all (m. 5. 6.a. 5. the countable union of countable sets is still countable. . otherwise X is said to be uncountable. ∞ 2. 1) . If 2X were countable we could ﬁnd a surjective map f : 2X → N in which case f ◦ i : 2N → N would be surjective as well. 1lim inf n→∞ An = lim inf n→∞ 1An . . . 3) .} = {Ac n a. 2) (2. 1} . If A = ∅ then A is countable by deﬁnition so we may assume A = ∅. 4. Suppose for each m ∈ N that Am is a countable subset of a set X. inf 1Ak (x) = 1∩k≥n Ak = 1inf k≥n Ak . 3) and so on. 2). from N to N × N. . 1). f |A . n) := am (n). Given a set A ⊂ X. . . Then f : X → N is injective which combined with item Page: 20 job: prob . macro: svmonob. (3. For the general case. 1lim sup An = lim sup 1An . A non-empty set X is countable iﬀ there exists a surjective map. We have: 1. 1} is a surjection and write f (n) as N (f1 (n) . Let f (1) = min Λ and deﬁne f inductively by f (n + 1) = min (Λ \ {f (1).o. h (3) = (1. . .cls date/time: 23-Feb-2007/15:20 3. If X is an inﬁnite set and Y is a set with at least two elements. h(6) = (1. . (2. 1). let 1A (x) = be the characteristic function of A. With out loss of generality we may assume A1 = ∅ and by replacing Am by A1 if necessary we may also assume Am = ∅ for all m. 2. n supk≥n 1Ak (x) = 1∪k≥n Ak = 1supk≥n An . By construction fn (n) = an for all n and so a ∈ / f (N) . 1) (3. If G ⊂ 2X . . Any inﬁnite subset Λ ⊂ N is in one to one correspondence with N. since Y0X ⊂ Y X for any subset Y0 ⊂ Y. h(5) = (2. then Y X is uncountable. n) ∈ N × N.6. and n→∞ n→∞ 7. f3 (n) . lim inf n→∞ An = x ∈ X : n=1 1Ac (x) < ∞ . . if Y0X is uncountable then so is Y X . Deﬁnition 3. . To do this put the elements of N × N into an array of the form (1. g : N → X. {An i. (taking Λ = f (X )) shows X is countable. If X and Y are countable then X × Y is countable. If A ⊂ X is a subset of a countable set X then A is countable.

.1. {2.1. then B is a σ – algebra. Exercise 3. X. 2. Let E ⊂ 2X be a collection of sets. A collection of subsets.1. f −1 (∪i∈I Ai ) = ∪i∈I f −1 (Ai ). Proof.17.e. Give a simple example where A (E1 ) = A (E2 ) while E1 = E2 . ∅.12. Then there exists a unique smallest algebra A(E ) and σ – algebra σ (E ) which contains E . Find a counterexample which shows that f (C ∩ D) = f (C ) ∩ f (D) need not hold. Example 3.3. {2. Suppose X = {1. macro: svmonob. X.16.10. B is also closed under taking countable intersections. 3}} is an algebra while. 2} and deﬁne f (a) = f (b) = 1 and f (c) = 2. Example 3. B = {∅. and 2. if A1 . Deﬁnition 3. Similarly show. Suppose that Ei ⊂ 2X for i = 1. 2.cls date/time: 23-Feb-2007/15:20 . Show that A (E1 ) = A (E2 ) iﬀ E1 ⊂ A (E2 ) and E2 ⊂ A (E1 ) .4. X } is a σ – algebra called the trivial σ – ﬁeld. Exercise 3. σ (E1 ) = σ (E2 ) iﬀ E1 ⊂ σ (E2 ) and E2 ⊂ σ (E1 ) .3 Algebraic sub-structures of sets 21 Deﬁnition 3.15. A(E ) = σ (E ) = 2X . In view of conditions 1.2 Exercises Let f : X → Y be a function and {Ai }i∈I be an indexed family of subsets of Y.13. Exercise 3. A collection of subsets B of X is a σ – algebra (or sometimes called a σ – ﬁeld) if B is an algebra which also closed under countable ∞ unions. A collection of subsets A of a set X is a π – system or multiplicative system if A is closed under taking ﬁnite intersections. (Notice that since B is also closed under taking complements. see Figure 3.5. Let E be any collection of subsets of X. A is closed under ﬁnite intersections. X. verify the following assertions. i. then ∪∞ i=1 Ai ∈ B .6. .) Page: 21 job: prob Fig. show that B \ (∪i∈I Ai ) = ∩i∈I (B \ Ai ). . 3.. f −1 (∩i∈I Ai ) = ∩i∈I f −1 (Ai ). {1. Exercise 3. 1. {A : A is an algebra such that E ⊂ A} = {A ∩ E : E ∈ E} . b.14. (∩i∈I Ai ) = c ∪i∈I Ac i. then A (E ) = {∅.2. An ∈ A then A1 ∪· · ·∪An ∈ A. then A = {∅. Exercise 3. Deﬁnition 3. S := {∅. 2}. is equivalent to 3 . {1. c} and Y = {1. Simply take A(E ) := and σ (E ) := {M : M is a σ – algebra such that E ⊂ M}. 3. A ⊂ X. 2} = c c f ({a} ) = f ({a}) = {2} . 3}}. 3. A ∈ A implies that Ac ∈ A 3. 2. i. 3. {1} . 2}. Exercise 3. X ∈ A 2. {3}}. 2}} .3. B = 2X . 3} and E = {∅. 3}} is a not an algebra but is a π – system. Let X = {a.11. Let X = {1. A collection of subsets A of a set X is an algebra (Field) if 1. Then 3. iA : A → X be the inclusion map (iA (x) = x for all x ∈ A) and EA = 1 i− A (E ) Example 3.3 Algebraic sub-structures of sets Deﬁnition 3. Then ∅ = f ({a} ∩ {b}) = f ({a}) ∩ f ({b}) = {1} and {1. {1. On the other hand if E = {{1. Here are some examples of algebras. Proposition 3. A is closed under ﬁnite unions. . Suppose that B ⊂ Y. 3}. if {Ai }i=1 ⊂ B. 2. X.e.

. k Let us now deﬁne {Bi }i=1 to be an enumeration of {Ax }x∈X . To see this..jN =1 (B1j1 ∩ B2j2 ∩ · · · ∩ BN jN ) ∈ A wherein we have used the fact that B1j1 ∩ B2j2 ∩· · ·∩ BN jN is a ﬁnite intersection of sets from Ec . Hence Ax is the smallest set in A which contains x. It is now a straightforward exercise to show A = {∪i∈Λ Bi : Λ ⊂ {1. ∅} ∪ E c Then A(E ) := {ﬁnite unions of ﬁnite intersections of elements from Ec }. We proceed as in Example 3..22 (Countable/Co-countable σ – Field).. Page: 22 job: prob macro: svmonob. B is uncountable. We say that a family of sets F ⊂ 2X is a partition of X if distinct members of F are disjoint and if X is the union of the sets in F . let us ﬁrst consider the case where {x. Enumerate the elements of F as F = {Pn }N n=1 where N ∈ N or N = ∞. An is a partition of X.20.cls date/time: 23-Feb-2007/15:20 . A (E ) consists of those subsets. Similarly. For deﬁniteness. .20. 2. n}} where ∪i∈Λ Ai := ∅ when Λ = ∅. To check A is closed under complementation. .2) N Proof. I be an inﬁnite index set. wherein we have used A is ﬁnite to insure Ax ∈ A. . Suppose that X is a ﬁnite set and that A ⊂ 2X is an algebra. For each x ∈ X let Ax = ∩ {A ∈ B : x ∈ A} ∈ B . In this case we must have Ax ⊂ C and Ay ⊂ C and therefore Ax = Ay . A ⊂ R. Let A denote the right member of Eq.2). x ∈ / y. . Proposition 3. k }} . then Ax = Ay .n} ) = 2n .. and that σ – algebra. . . Thus any countable σ – algebra is necessarily ﬁnite. More generally we have the following exercise. Therefore. Let E c := {Ac : A ∈ E} and Ec := E ∪ {X. . Ax ∩ Ay = ∅. . Hence to ﬁnish that proof it suﬃces to show A is an algebra. Example 3. 1} → Aa = ∪{Pn : an = 1} ∈ B is bijective and therefore.22 3 Preliminaries Deﬁnition 3.. Let X be a set and E = {A1 .e. Proof. (3. Let X = R and E := {{x} : x ∈ R} . Now suppose either x or y is not in C. and E = {Ai }i∈I be a partition of X. Let X be a set. For each x ∈ X let Ax = ∩ {A ∈ A : x ∈ A} ∈ A.7. generated by E are given by A(E ) = {∪i∈Λ Ai : Λ ⊂ I with # (Λ) < ∞ or # (Λc ) < ∞} and σ (E ) = {∪i∈Λ Ai : Λ ⊂ I with Λ countable or Λc countable} respectively. This ﬁnishes the proof modulo the uniqueness assertion which is left as an exercise to the reader. 2.8. y } ⊂ C.2. . then the correspondence (3. wherein we have used B is a countable σ – algebra to insure Ax ∈ B. . Let X be a set and E ⊂ 2X . (3. such that A is countable or Ac is countable. Exercise 3.e. Proposition 3. . Let C = Ax ∩ Ay ∈ A. Let X be a set. In particular B is actually a ﬁnite set and # (B ) = 2n for some n ∈ N. Example 3. Then there exists a unique ﬁnite partition F of X such that F ⊂ B and every element B ∈ B is of the form B = ∪ {A ∈ F : A ⊂ B } . In this case A(E ) = σ (E ) = {∪i∈Λ Ai : Λ ⊂ {1. A (E ) . (3. such that A is ﬁnite or Ac is ﬁnite. Notice that # (A(E )) = #(2{1. say x ∈ / C. An } where A1 . σ (E ) .1) holds for all B ∈ B . . . Prove the algebra. i. From the deﬁnition of an algebra. .1) a ∈ {0. The proof of these assertions are routine except for possibly showing that A is closed under complementation. Example 3.21. A ⊂ R. Just as above either Ax ∩ Ay = ∅ or Ax = Ay and therefore F = {Ax : x ∈ X } ⊂ B is a (necessarily countable) partition of X for which Eq. Here we are using the convention that ∪i∈Λ Ai := ∅ when Λ = ∅. it is clear that E ⊂ A ⊂ A(E ). by Lemma 3. .. writing Bij = Ac ij ∈ Ec . I claim that if C = ∅. . Suppose that B ⊂ 2X is a σ – algebra and B is at most a countable set. Then σ (E ) consists of those subsets..19. we ﬁnd that N K K Zc = i=1 j =1 Bij = j1 . i. Then x ∈ Ax \ Ay ∈ A from which it follows that Ax = Ax \ Ay .23.18. let Z ∈ A be expressed as N K Z= i=1 j =1 Aij where Aij ∈ Ec . If N = ∞.

the set simple random variables. {[a.30. if A ∈ B such that A ⊂ (0.28. f −1 (σ (E )) ⊂ σ f −1 (E ) .6. Z= i=1 j =1 Aij For the reverse inclusion. Then E(0.3) with X = A and f = iA being the inclusion map implies 1 −1 (σ (E ))A = i− A (σ (E )) = σ (iA (E )) = σ (EA ). Show A is the collection of subsets of X which may be written as ﬁnite union of sets of the form F ∩ V where F is closed and V is open. is generated by any of the following collection of sets: 1. For example if A ∈ A.1] . Suppose f : X → Y is a function. ∞) : a ∈ Q} . then ∞ ∞ Zc = j1 =1.. Notation 3.9) are algebras (σ – algebras) provided F and B are algebras (σ – algebras). By Exercise 3. Also see Proposition 3. Exercise 3. Hint: make use of Exercise 3. 1].23. {(a. on R is the smallest σ -ﬁeld containing all of the open subsets of R. Deﬁnition 3.27. g ∈ S (A) and c ∈ C\ {0} . In particular. Thus the above description is not correct. Show f −1 F and f∗ B (see Notation 3. For every algebra A ⊂ 2Ω . then (∩n i=1 Vi ) ∩ ∩m F is simply a set of the form V ∩ F where V ⊂ X and F X. then A ∈ σ E(0. then 1A ∈ S (A) is a measurable simple function. then {f + cg = λ} = a.26. Lemma 3. b] : −∞ < a < b < ∞} and B = σ (E ) be the Borel σ – ﬁeld on R. F ⊂ 2Y and B ⊂ 2X . (3.8).j is a σ – algebra which contains E and thus σ (E ) ⊂ f∗ σ f −1 (E ) . i. ∞) : a ∈ Q} or 3. Lemma 3. A ⊂ 2Ω . since if ∞ ∞ f −1 Proof. S (A) . let S(A) denote the collection of simple random variables from Ω to C.jN =1. (Similar assertion hold with σ (·) being replaced by A (·) . If A ⊂ 2Ω is an algebra.1] .3) (3.. In general it is complicated to explicitly describe σ (E ). B = BR = B (R) . 2.j2 =1. (3.e.23 on page 39 of Folland for details. Exercise 3. A measurable simple function. Let us observe that 1Ω = 1 and 1∅ = 0 are in S (A) . Proof. Now if Vi ⊂o X and Fj X for each j.29 Given an algebra.b∈C:a+cb=λ where BA := {B ∩ A : B ∈ B} . Let τ be a topology on a set X and A = A(τ ) be the algebra generated by τ. Solution to Exercise (3. ∞) : a ∈ R} . f : Ω → Y is said to be simple if f (Ω ) ⊂ Y is a ﬁnite set. Suppose that f : X → Y is a function and E ⊂ 2Y and A ⊂ Y then σ f −1 (E ) = f −1 (σ (E )) and (σ (E ))A = σ (EA ).8.) Page: 23 job: prob ({f = a} ∩ {g = b}) ∈ A (3. Let E = {(a. If f.1] = {(a. f −1 (σ (E )) is a σ – algebra and since E ⊂ F . notice that f∗ σ f −1 (E ) := B ⊂ Y : f −1 (B ) ∈ σ f −1 (E ) with Aij ∈ Ec . It now follows that σ (f −1 (E )) ⊂ f −1 (σ (E )). In this case τc is the collection of sets which are either open or closed. (E ) ⊂ f −1 (σ (E )).4) Example 3. is called a simple random variable relative to A.3 Algebraic sub-structures of sets 23 Remark 3.cls date/time: 23-Feb-2007/15:20 . =1 Ac. b] : 0 ≤ a < b ≤ 1} and we have B(0. One might think that in general σ (E ) may be described as the countable unions of countable intersections of sets in E c . Exercise 3.. BR . Therefore j o j =1 the result is an immediate consequence of Proposition 3.21.1] = σ E(0. {(a.9.3. A function.5) macro: svmonob..24. forms an algebra.. Deﬁnition 3.10. see Proposition 1. f : Ω → C. we say that a simple function f : Ω → Y is measurable if {f = y } := f −1 ({y }) ∈ A for all y ∈ Y. which is now an uncountable union. However this is in general false.. Hence for every B ∈ σ (E ) we know that f −1 (B ) ∈ σ f −1 (E ) .25. Verify the σ – algebra. Applying Eq. The Borel σ – ﬁeld.10.

Proof. f ∈ S.32. A ∈ A (S (A)) ⇐⇒ 1A ∈ S (A) ⇐⇒ A ∈ A which shows that A (S (A)) = A. we see that g = 0 on ∪n i=1 {f = λi } while g = 1 on {f = α} . a simple function. (It is easily checked that A (S) is a sub-algebra of 2X . Show that the map A ∈ Algebras ⊂ 2X → S (A) ∈ {simple function algebras on X } is bijective and the map. With this preparation. Moreover. 1 = 1X ∈ S. Page: 24 job: prob macro: svmonob.) Lemma 3. if f ∈ S then 1{f =λ} ∈ S for all λ ∈ C. Exercise 3. g := i=1 (α − λi ) i=1 (f − λi 1) ∈ S.32. Therefore. 1. Similarly. then f = λ∈C λ1{f =λ} ∈ S. let A (S) := {A ⊂ X : 1A ∈ S} . Conversely.24 3 Preliminaries and {f · g = λ} = a. is a simple function. Show A (S) is an algebra of sets. then {f + cg = λ} = a. 3. If S is a simple function algebra. f : X → C is in S iﬀ 1{f =λ} ∈ S for all λ ∈ C. Continuing the notation introduced above: 1. Then {f = α} ∈ A (S) . f ∈ S (A (S)) ⇐⇒ {f = λ} ∈ A (S) ∀ λ ∈ C ⇐⇒ 1{f =λ} ∈ S ∀ λ ∈ C ⇐⇒ f ∈ S which shows S (A (S)) = S. Since 0 = 1∅ . Suppose that S is a simple function algebra.b∈F:a·b=λ ({f = a} ∩ {g = b}) ∈ A from which it follows that f + cg and f · g are back in S (A) . S → A (S) . by Lemma 3. Let {λi }i=0 be an enumeration of f (X ) with λ0 = α. is a subalgebra of the bounded complex functions on X such that 1 ∈ S and each function.b∈F:a+cb=λ ({f = a} ∩ {g = b}) ∈ A and {f · g = λ } = a. Solution to Exercise (3. A simple function algebra. If f : Ω → C is a simple function such that 1{f =λ} ∈ S for all λ ∈ C. If A ∈ A (S) .11).11. First oﬀ. Deﬁnition 3. Show S (A) is a simple function algebra. 2.31. So we have shown g = 1{f =α} ∈ S and therefore that {f = α} ∈ A. 2. S. If f.6) from which it follows that f + cg and f · g are back in S (A) . B ∈ A (S) then 1A∩B = 1A · 1B ∈ S and thus A ∩ B ∈ A (S) . Then n −1 n n 3.b∈C:a·b=λ ({f = a} ∩ {g = b}) ∈ A (3.cls date/time: 23-Feb-2007/15:20 . g ∈ S (A) and c ∈ F. we are now ready to complete the veriﬁcation. if A. is the inverse map. Finally. it follows that ∅ and X are in A (S) . then 1Ac = 1 − 1A ∈ S and so Ac ∈ A (S) . f ∈ S and α ∈ f (X ) .

Suppose that E ⊂ 2X is a collection of subsets of X and µ : E → [0. 4. (4. ˜j ’s are pair-wise disjoint and 3.3) µ(Ej ). (µ is ﬁnitely subbadditive) µ(∪n j =1 Ej ) ≤ 4. µ is sub-additive (ﬁnitely sub-additive) on E if n 1. Since A ∪ B = [A \ (A ∩ B )] [B \ (A ∩ B )] A ∩ B. ∞ 5. and µ is ﬁnitely additive on A. B ∈ A.3).1 Finitely Additive Measures Proposition 4. A ∈ A. ∞] . 1. is a premeasure iﬀ µ is sub-additve. 6. 2. 8. .2 (Basic properties of ﬁnitely additive measures). A ﬁnitely additive measure. B ∈ E with A ⊂ B. If E = A is an algebra. If E = A is an algebra. 3. E. We say that µ is a probability measure if µ (X ) = 1. (µ is monotone) µ (E ) ≤ µ(F ) if E ⊂ F. holds even when n = ∞. Adding µ (A ∩ B ) to both sides of this equation proves Eq. 4. . µ(E ) = i=1 n whenever E = i=1 Ei ∈ E with Ei ∈ E for i = 1. 3. µ(Ei ) (4.1) where A ∈ A and {Ai }i=1 ⊂ A are pairwise disjoint sets. µ. ∞] is a function. F ∈ A with n E ⊂ F and {Ej }j =1 ⊂ A. Since F is the disjoint union of E and (F \ E ) and F \ E = F ∩ E c ∈ A it follows that µ(F ) = µ(E ) + µ(F \ E ) ≥ µ(E ). and µ is σ – additive on A then µ is called a premeasure on A. 5.4 Finitely Additive Measures Deﬁnition 4. 7. µ is super-additive (ﬁnitely super-additive) on E if n µ(A) ≤ i=1 ∞ µ(Ai ) for A = i=1 Ai (4. µ is monotonic if µ (A) ≤ µ (B ) for all A. µ is σ – additive (or countable additive) on E if item 4. For A. µ is sub-additive on A iﬀ ∞ n j =1 (4.1. Then 1. (µ is countably superadditive) If A = i=1 Ai with Ai . A measure is a premeasure.4) µ(E ) ≥ i=1 n i=1 µ(Ei ) (4. µ (A ∪ B ) = µ (A ∪ B \ (A ∩ B )) + µ (A ∩ B ) = µ (A \ (A ∩ B )) + µ (B \ (A ∩ B )) + µ (A ∩ B ) . then ∞ ∞ whenever E = Ei ∈ E with n ∈ N∪ {∞} (n ∈ N). µ is additive or ﬁnitely additive on E if n µ i=1 Ai ≥ i=1 µ (Ai ) . Since Ej ⊂ Ej it follows from the monotonicity of µ that µ(E ) = µ(Ej ) ≤ µ(Ej ). A ⊂ 2X . then µ is said to be a ﬁnitely additive measure. then : . . the following strong additivity formula holds. Suppose µ is a ﬁnitely additive measure on an algebra. 2. where B is a σ – algebra. 6. n < ∞. µ (∅) = 0. . 2. Let Ej = Ej \ (E1 ∪ · · · ∪ Ej −1 ) so that the E n ˜ E = ∪j =1 Ej . 2. µ : B → [0.2) Proof. µ (∅) = 0. µ (A ∪ B ) + µ (A ∩ B ) = µ (A) + µ (B ) . µ(E ) ≤ i=1 n µ(Ei ) ∞ whenever E = i=1 Ei ∈ E with n ∈ N∪ {∞} (n ∈ N).

A ∈ A. ω ∈Ω P (A) = lim P N →∞ ∪N n=1 An = lim N →∞ P (An ) = n=1 n=1 P (An ) . 5 =⇒ 2. then P is countably sub-additive. then Ac n ↑ A and therefore. n→∞ n→∞ Ac n ↓ A and therefore we c Then P (A) := ω ∈A p (ω ) for all A ⊂ Ω c lim (1 − P (An )) = lim P (Ac n ) = P (A ) = 1 − P (A) . Let (Ω. Bi−1 ) ∈ A and B0 = ∅. then A \ An ↓ ∅ and therefore n→∞ µ(A) ≤ i=1 ∞ µ(Ai ) ≤ i=1 µ(Bi ). P (An ) ↓ 1.26 4 Finitely Additive Measures ∞ ∞ 4. We will call (Ω. P (A) = k=1 P (Ak ) = lim ∞ n→∞ P (Ak ) = lim P (∪n k=1 Ak ) = lim P (An ) . A probability measure on (Ω. ∞ n P (An ) ≤ n=1 P (An ) . An−1 ) ∈ B . Therefore. i=1 µ (Ai ) ≤ µ (A) . We will start by showing 1 ⇐⇒ 2 ⇐⇒ 3. This is a combination of items 5. Suppose that (Ω. 2 N ∞ 4. P (An ) ↑ P (A) . Remark 4.4) ∞ ∞ It is clear that 2 =⇒ 4 and that 3 =⇒ 5. then N ∪n=1 An ↑ A. let An := An \ (A1 ∪ .5.6. 5 =⇒ 3. 1] is a function such that p (ω ) = 1. Suppose that A = i=1 Ai with Ai . P is σ – additive on 2. Therefore. B . n→∞ 5. . For all An ∈ A such A. Suppose that P is a ﬁnitely additive probability measure on an algebra. c lim (1 − P (An )) = lim P (Ac n ) = P (A ) = 1 − P (A) . If {An }n=1 ⊂ A are disjoint and A := ∪∞ n=1 An ∈ A. then An \ A ↓ ∅. i. in the above proposition hold without the restriction that P (Ω ) = 1 and in fact P (Ω ) = ∞ may be allowed for this equivalence. 6. Suppose An ∈ A such that An ↑ A ∈ A. 1] such that any and hence all of the continuity properties in Proposition 4. An = ∪n k=1 Ak and A = ∪k=1 Ak . Example 4. P (An ) ↑ 1. deﬁnes a measure on 2Ω . Suppose that Ω is a ﬁnite set. there are a few special cases where we can describe explicitly what is going on. Suppose that An ∈ B and let A1 := A1 and for n ≥ 2.4. Therefore. P ) a probability space. If An ∈ A such that An ↑ A ∈ A. Lemma 4. macro: svmonob. If A = i=1 Bi with A ∈ A and Bi ∈ A. For all An ∈ A such 5. then A = i=1 Ai where Ai := Bi \ (B1 ∪ . k=1 n→∞ n→∞ =⇒ 1. If An ∈ A such that An ↑ A ∈ A. . n→∞ lim [P (An ) − P (A)] = lim P (An \ A) = 0.3 hold. To ﬁnish the proof we will show 5 =⇒ 2 and 5 =⇒ 3. and 6. A ⊂ 2Ω . . For all An ∈ A such 3. P ) is a probability space.7.2 Examples of Measures Most σ – algebras and σ -additive measures are somewhat diﬃcult to describe and deﬁne. and 2. (4. B . Proof. Letting n → ∞ in this equation shows µ is superadditive. P (An ) ↓ P (A) . P : B → [0. Let An := An \ An−1 ∞ ∞ with A0 := ∅.cls date/time: 23-Feb-2007/15:20 Page: 26 job: prob . n lim [P (A) − P (An )] = lim P (A \ An ) = 0.e. ↑ Ω. n→∞ n→∞ 3 =⇒ 2. . B ⊂ 2Ω is a σ – algebra. Therefore using the monotonicity of µ and Eq. If An ∈ A such that An ↓ A ∈ A. 1 =⇒ 2. B ) be a measurable space. then again have.3. Observe that the equivalence of items 1. n→∞ Proposition 4. Then {An }n=1 are disjoint. For all An ∈ A such 4. B := 2Ω . c 2 =⇒ 3. ↓ Ω. Then the following are equivalent: 1. and p : Ω → [0. If An ∈ A such that An ↓ A ∈ A. However. ↓ A ∈ A. B ) is a ﬁnitely additive probability measure. Then ∞ ∞ P (∪∞ n=1 An ) = P (∪n=1 An ) = n=1 ∞ Proof. that that that that An An An An ↑ A ∈ A. Deﬁnition 4. then i=1 Ai ⊂ A for all n n and so by the monotonicity and ﬁnite additivity of µ.

A ⊂ R which may be written as ﬁnite disjoint unions from S := {(a. a + 1/n] ↓ ∅ while P ((a. Example 4. A is an algebra. We will show below the following: µ(∪∞ i=1 Ai ) = µ(Ai ) = i=1 ∞ ∞ i=1 j =1 ∞ µj (Ai ) µj (∪∞ i=1 Ai ) j =1 = j =1 i=1 µj (Ai ) = µ(∪∞ i=1 Ai ) = wherein the third equality we used Theorem 1.2 Examples of Measures 27 Example 4. A consists of those sets. To every increasing function. Suppose that µ is a measure on X and λ > 0. for A ∈ B .10. ∞ where 1α⊂A is one if α ⊂ A and zero otherwise. Suppose that X is any set and x ∈ X is a point. Suppose that F ⊂ 2X is a countable or ﬁnite partition of X and B ⊂ 2X is the σ – algebra which consists of the collection of sets A ⊂ X such that A = ∪ {α ∈ F : α ⊂ A} .11. Page: 27 job: prob . Moreover.) To prove this ∞ we must show that µ is countably additive. if we are given any function λ : F → [0. i. Example 4. µ(A) = α∈F α⊂A λ(α) = α∈F λ(α)1α⊂A Then µ = δx is a measure on X called the Dirac delta measure at x. (Recall that BR = σ (A) . Example 4. macro: svmonob. then ∞ ∞ ∞ as desired. Thus we have shown that there is a one to one correspondence between measures µ on B and functions λ : F → [0. F : R → [0. P extends to a probability measure on BR iﬀ F is right continuous. then (a. ∞]. let δx (A) = 1 if x ∈ A 0 if x ∈ / A. if A = i=1 Ai and α ∈ F .8.4. Suppose that X is a set λ : X → [0. Conversely. Suppose that Ω = R. then λ · µ is also a ∞ measure on X.) 2.9. Suppose that {Ai }i=1 is a collection of pair-wise disjoint subsets of X. b] ∩ R : −∞ ≤ a ≤ b ≤ ∞} . P is σ – additive on A iﬀ F is right continuous.5) Any measure µ : B → [0. 4. Indeed. a + 1/n]) = F (a + 1/n) − F (a) ↓ F (a+) − F (a) > 0. 3. P = PF on A such that is a measure. then µ = j =1 µj . Then µ := x∈X 1.12. Hence P can not be σ – additive on A in this case. if {µj }j ∈J are all measures on X. ∞] is determined uniquely by its values on F . We may check that µ is a ∞ measure on B . For A ⊂ X. ∞] is a function. then α ⊂ A iﬀ α ⊂ Ai for ∞ one and hence exactly one Ai . (See Section 3.e. b] ∩ R) = F (b) − F (a) for all − ∞ ≤ a ≤ b ≤ ∞.6 and in the fourth we used that fact that µj is a measure.1 for the meaning of this sum.cls date/time: 23-Feb-2007/15:20 for all A ⊂ X. (4. Example 4. Therefore 1α⊂A = i=1 1α⊂Ai and hence ∞ µ(A) = α∈F ∞ λ(α)1α⊂A = α∈F λ(α) i=1 ∞ 1α⊂Ai µ(A) = j =1 µj (A) for all A ⊂ X = i=1 α∈F λ(α)1α⊂Ai = i=1 µ(Ai ) is a measure on X. explicitly µ(A) = x∈A λ(x) P ((a. ∞] we may deﬁne. 1] such that F (−∞) := lim F (x) = 0 and x→−∞ x→∞ λ(x)δx F (+∞) := lim F (x) = 1 there exists a ﬁnitely additive probability measure. Let us observe directly that if F (a+) := limx↓a F (x) = F (a) . The following example explains what is going on in a more typical case of interest to us in the sequel.

1) by 1 P (A) = lim # {1 ≤ k ≤ N : ω (k ) ∈ A} N →∞ N where ω (k ) ∈ Ω was the result of the k th “independent” experiment. g = b}) = a. g = b}) yP (f = y ) = lim 1 N →∞ N 1 N →∞ N N N →∞ 1 N y · # {1 ≤ k ≤ N : f (ω (k )) = y } y ∈C = z ∈C a+b=z = lim y· y ∈C N k=1 1f (ω(k))=y = lim 1 N →∞ N N f (ω (k )) · 1f (ω(k))=y k=1 y ∈C = a. Suppose that A ∈ A. g = b}) . g = b}) z ∈C = = z ∈C z a+b=z P ({f = a.b (a + b) P ({f = a. Writing {f = a.16. is deﬁned by EP (f ) = y ∈C 1. then E1A = 0 · P (Ac ) + 1 · P (A) = P (A) . (4. E is positive. Ef should represent the average of the values of f over many “independent” experiments.9) 3. (4.3 Simple Integration Proof. (2. If λ = 0. aP ({f = a. Suppose now that P is a ﬁnitely additive probability measure on an algebra A ⊂ 2X . E(f ) = EP (f ). If we use this interpretation back in Eq. then E(λf ) = y ∈C∪{∞} y P (λf = y ) = y ∈C∪{∞} y P (f = y/λ) yP (f = y ). If f ∈ S(A) and λ ∈ C. But f (ω (k )) .28 4 Finitely Additive Measures 4. then E(f + g ) = E(g ) + E(f ). The expectation operator. If f.cls date/time: 23-Feb-2007/15:20 . 2.6). (4. (4. i.10) Page: 28 job: prob = a aP (∪b {f = a. 3. g = b}) = Eg.13 (Simple Integral). a. E = EP . g = b}) Thus informally. (4. For all f ∈ S (A) . a≥0 macro: svmonob. Deﬁnition 4. |Ef | ≤ E |f | . g = b} for f −1 ({a}) ∩ g −1 ({b}). Let us recall that our intuitive notion of P (A) was given as in Eq.14. 4.7) The case λ = 0 is trivial. g ∈ S (A) . z ∈C∪{∞} Example 4.8) bP ({f = a. g = b}) (a + b) P ({f = a.b Equation (4. Proposition 4. then E(λf ) = λE(f ).6) = λz P (f = z ) = λE(f ). g = b}) aP ({f = a}) = Ef a = and similarly. For f ∈ S (A) the integral or expectation.e. then E(f + g ) = z ∈C Remark 4. If f ≥ 0 then E(f ) = aP (f = a) ≥ 0. we arrive at E(f ) = y ∈C z P (f + g = z ) z P (∪a+b=z {f = a.15. 2. satisﬁes: 1.9) is now a consequence of the last three displayed equations. E(f ) ≥ 0 if f is a non-negative measurable simple function. (4.b a = lim k=1 a b P ({f = a.

. |f |p εp 1|f |≥ε {f = λ} ∈ A as well. (4.17. (4. n=1 An Example 4. then N N N Taking expectations of this equation then gives Eq. n} . |Ef | = λ∈C from which it follows that λP (f = λ) ≤ λ∈C |λ| P (f = λ) = E |f | . This veriﬁes Eq. .13) are zero and so we will only consider the case where 1 ≤ m ≤ M. (4. εp |λ| 1{f =λ} |λ|≥ε p p (4.13) since 1∪M (ω ) = 1. Let ω ∈ Ω and by relabeling the sets {An } if necessary. . = 0≤n1 <n2 <···<nk ≤M (−1) k=0 k 0≤n1 <n2 <···<nk ≤M 1An1 ∩···∩Ank On the other hand. =1− k=0 (−1) (1) m is still a simple random variable. So by the Inclusion Exclusion Formula. (4. Suppose that f ∈ S(A). . n} → {1. If An ∈ A for n = 1. . M such that µ ∪M n=1 An < ∞. 2. We wish to compute the probability of the event. . 1∪M = 1A = n=1 An M (−1) k=1 k+1 1≤n1 <n2 <···<nk ≤M 1An1 ∩···∩Ank . let Ai := {ω ∈ Ω : ω (i) = i} and observe that B = ∪n i=1 Ai . m k Lemma 4. we have n n=1 n=1 n Page: 29 job: prob . Here is an alternate proof of Eq.19 (Inclusion Exclusion Formula). we may assume that ω ∈ A1 ∩ · · · ∩ Am and ω∈ / Am+1 ∪ · · · ∪ AM for some 0 ≤ m ≤ M. Every simple measurable function.12).3). and deﬁne P (A) := #( n! to be the uniform distribution (Haar measure) on Ω. . We will give a different and perhaps more illuminating proof here.11) = k=1 m (−1) k+1 = k=1 (−1) m k+1 m k k n−k is a simple random variable and {|f | ≥ ε} = Therefore. 2. Remark 4. . . First observe that |f | = λ∈C M M |λ| 1f =λ 1 − 1A = 1Ac = n=1 M 1 Ac = n n=1 k (1 − 1An ) 1An1 · · · 1Ank and therefore. B. Let Ω be the set of permutations (think of card A) shuﬄing). (4. To do this.13). may be written as N f = j =1 λj 1Aj for some λj ∈ C and some Aj ∈ C. then P ({|f | ≥ ε}) = E 1|f |≥ε ≤ E Observe that |f | = λ∈C p (−1) k=1 m k+1 1≤n1 <n2 <···<nk ≤M 1An1 ∩···∩Ank (ω ) 1An1 ∩···∩Ank (ω ) 1≤n1 <n2 <···<nk ≤m |f | p 1|f |≥ε ≤ ε−p E |f | .) With this notation we have M Ef = E j =1 λ j 1 Aj = j =1 λj E1Aj = j =1 λj P (Aj ) . c M c Since Ac = ∪M A = ∩ A . both sides of Eq. that a random permutation ﬁxes some index i.cls date/time: 23-Feb-2007/15:20 µ ∪M n=1 An = k=1 (−1) k+1 1≤n1 <n2 <···<nk ≤M µ (An1 ∩ · · · ∩ Ank ) . then M = 1 − (1 − 1) = 1.20. . (4.3 Simple Integration 29 4.13) Remark 4. This may be proved inductively from Eq. Remark 4.12) Proof. we have macro: svmonob. . and p > 0. . Let A := ∪M n=1 An . E |f | = E λ∈C = |λ| 1f =λ = λ∈C (−1) k=0 M |λ| E1f =λ = λ∈C |λ| P (f = λ) ≤ max |f | . 2.18 (Chebyshev’s Inequality). f : Ω → C. (4. ε > 0.21 (Coincidences). Moreover if f is represented this way. ω : {1.4. (When m = 0.

. . let X i = 1 Ai and observe that n n and P : 2Ω → [0.1 (Simple Independence 1. 2. k! 4. . We further suppose p : Ω → [0.cls date/time: 23-Feb-2007/15:20 . Ω = Λn . n and If p (ω ) = i=1 qi (ωi ) . ω (ik ) = ik }) = and # {1 ≤ i1 < i2 < i3 < · · · < ik ≤ n} = we ﬁnd P (B ) = k=1 n (−1) k=1 k+1 1 1 2 1 =1− + = k! 2 6 3 (n − k )! n! n .22.15) for any functions. Namely. let Λ be a ﬁnite set. fi : Λ → R that n n n denote the number of ﬁxed points of ω. (4. Suppose qi : Λ → [0. 2. and Xi : Ω → Λ be deﬁned by Xi (ω ) = ωi for ω ∈ Ω and i = 1. Show for any functions.4 Simple Independence and the Weak Law of Large Numbers For the next two problems.30 4 Finitely Additive Measures n P (B ) = k=1 (−1) k+1 1≤i1 <i2 <i3 <···<ik ≤n P (Ai1 ∩ · · · ∩ Aik ) . and so P (∃ a ﬁxed point) = while 3 4 2 = 6 3 Since P (Ai1 ∩ · · · ∩ Aik ) = P ({ω ∈ Ω : ω (i1 ) = i1 .632.). Hence we have n n n EP i=1 fi (Xi ) = i=1 EP [fi (Xi )] = i=1 EQi fi EN = i=1 EXi = i=1 P (Ai ) = i=1 (n − 1)! = 1. .21. k k+1 and EN = 1 (3 + 1 + 1 + 0 + 0 + 1) = 1. then there exists functions qi : Λ → [0. n P (B ) = − k=1 (−1) k 1 ∼ = − e−1 − 1 ∼ = 0. macro: svmonob. n! where Qi (γ ) = λ∈γ qi (λ) for all γ ⊂ Λ. n ∈ N. .e. .14) N (ω ) = i=1 Xi (ω ) = i=1 1ω(i)=i = # {i : ω (i) = i} . To this end. 1] is the probability measure deﬁned by P (A) := ω ∈A p (ω ) for all A ∈ 2Ω . i. 6 (−1) k+1 n (n − k )! = k n! n (−1) k=1 1 . Exercise 4. . Let us check the above formula when n = 6. 1] with n λ∈Λ qi (λ) = 1. . n. ω. k! Example 4. such that p (ω ) = i=1 qi (ωi ) . We now wish to compute the expected number of ﬁxed points of a random permutation.2 (Simple Independence 2. . . if n n EP i=1 fi (Xi ) = i=1 EP [fi (Xi )] (4. how many cards in the shuﬄed stack have not moved on average. . . 1] is a function such that p (ω ) = 1 ω ∈Ω For large n this gives. In this case we have 1 1 2 2 3 3 Page: 30 ω 2 3 1 3 1 2 3 2 3 1 2 1 N (ω ) 3 1 1 0 0 1 job: prob Exercise 4. Prove the converse of the previous exercise. fi : Λ → R.). 1] are funcn tions such that λ∈Λ qi (λ) = 1 for i = 1. Continue the notation in Example 4.

4.4 Simple Independence and the Weak Law of Large Numbers

31

Exercise 4.3 (A Weak Law of Large Numbers). Suppose that Λ ⊂ R n is a ﬁnite set, n ∈ N, Ω = Λn , p (ω ) = i=1 q (ωi ) where q : Λ → [0, 1] such that λ∈Λ q (λ) = 1, and let P : 2Ω → [0, 1] be the probability measure deﬁned as in Eq. (4.14). Further let Xi (ω ) = ωi for i = 1, 2, . . . , n, ξ := EXi , 2 σ 2 := E (Xi − ξ ) , and Sn = 1. Show, ξ =

λ∈Λ

**Proof. Let x ∈ [0, 1] , Λ = {0, 1} , q (0) = 1 − x, q (1) = x, Ω = Λn , and P Pn 1− n i=1 ωi Px ({ω }) = q (ω1 ) . . . q (ωn ) = x i=1 ωi · (1 − x) . As above, let Sn =
**

1 n

(X1 + · · · + Xn ) , where Xi (ω ) = ωi and observe that Px Sn = k n = n k n−k x (1 − x) . k

1 (X1 + · · · + Xn ) . n

λ q (λ) and

2

**Therefore, writing Ex for EPx , we have
**

2

σ =

λ∈Λ

(λ − ξ ) q (λ) =

λ∈Λ

λ q (λ) − ξ .

2

2

(4.16) Ex [f (Sn )] =

n

f

k=0

**2. Show, ESn = ξ. 3. Let δij = 1 if i = j and δij = 0 if i = j. Show E [(Xi − ξ ) (Xj − ξ )] = δij σ . 4. Using Sn − ξ may be expressed as,
**

1 n 2 n i=1 2

k n

n k n−k x (1 − x) = pn (x) . k

**Hence we ﬁnd |pn (x) − f (x)| = |Ex f (Sn ) − f (x)| = |Ex [f (Sn ) − f (x)]| ≤ Ex |f (Sn ) − f (x)| = Ex [|f (Sn ) − f (x)| : |Sn − x| ≥ ε] + Ex [|f (Sn ) − f (x)| : |Sn − x| < ε] ≤ 2M · Px (|Sn − x| ≥ ε) + δ (ε) where M := max |f (y )| and
**

y ∈[0,1]

(Xi − ξ ) , show (4.17)

E (Sn − ξ ) =

1 2 σ . n 1 2 σ . nε2

5. Conclude using Eq. (4.17) and Remark 4.18 that P (|Sn − ξ | ≥ ε) ≤ (4.18)

So for large n, Sn is concentrated near ξ = EXi with probability approaching 1 for n large. This is a version of the weak law of large numbers. Exercise 4.4 (Bernoulli Random Variables). Let Λ = {0, 1} , , X : Λ → R be deﬁned by X (0) = 0 and X (1) = 1, x ∈ [0, 1] , and deﬁne Q = xδ1 + (1 − x) δ0 , i.e. Q ({0}) = 1 − x and Q ({1}) = x. Verify, ξ (x) := EQ X = x and σ 2 (x) := EQ (X − x) = (1 − x) x ≤ 1/4. Theorem 4.23 (Weierstrass Approximation Theorem via Bernstein’s Polynomials.). Suppose that f ∈ C ([0, 1] , C) and

n 2

δ (ε) := sup {|f (y ) − f (x)| : x, y ∈ [0, 1] and |y − x| ≤ ε} is the modulus of continuity of f. Now by the above exercises, Px (|Sn − x| ≥ ε) ≤ and hence we may conclude that max |pn (x) − f (x)| ≤ M + δ (ε) 2nε2 1 4nε2 (see Figure 4.1)

x∈[0,1]

pn (x) :=

k=0

n f k

k n

xk (1 − x)

n−k

.

**and therefore, that lim sup max |pn (x) − f (x)| ≤ δ (ε) .
**

n→∞ x∈[0,1]

Then

n→∞ x∈[0,1]

lim

sup |f (x) − pn (x)| = 0.

This completes the proof, since by uniform continuity of f, δ (ε) ↓ 0 as ε ↓ 0.

**(See Theorem 14.42 for a multi-dimensional generalization of this theorem.)
**

Page: 31 job: prob macro: svmonob.cls date/time: 23-Feb-2007/15:20

32

4 Finitely Additive Measures

Proof. Let A denote the collection of sets which may be written as ﬁnite disjoint unions of sets from S . Clearly S ⊂ A ⊂ A(S ) so it suﬃces to show A is an algebra since A(S ) is the smallest algebra containing S . By the properties of S , we know that ∅, X ∈ A. Now suppose that Ai = F ∈Λi F ∈ A where, for i = 1, 2, . . . , n, Λi is a ﬁnite collection of disjoint sets from S . Then

n n

Ai =

i=1 i=1 F ∈Λi

F

=

(F1 ,,...,Fn )∈Λ1 ×···×Λn

(F1 ∩ F2 ∩ · · · ∩ Fn )

and this is a disjoint (you check) union of elements from S . Therefore A is closed under ﬁnite intersections. Similarly, if A = F ∈Λ F with Λ being a ﬁnite collection of disjoint sets from S , then Ac = F ∈Λ F c . Since by assumption F c ∈ A for F ∈ Λ ⊂ S and A is closed under ﬁnite intersections, it follows that Ac ∈ A.

Fig. 4.1. Plots of Px (Sn = k/n) versus k/n for n = 100 with x = 1/4 (black), x = 1/2 (red), and x = 5/6 (green).

¯ be as in Example Example 4.27. Let X = R and S := (a, b] ∩ R : a, b ∈ R 4.25. Then A(S ) may be described as being those sets which are ﬁnite disjoint unions of sets from S . Proposition 4.28 (Construction of Finitely Additive Measures). Suppose S ⊂ 2X is a semi-algebra (see Deﬁnition 4.24) and A = A(S ) is the algebra generated by S . Then every additive function µ : S → [0, ∞] such that µ (∅) = 0 extends uniquely to an additive measure (which we still denote by µ) on A. Proof. Since (by Proposition 4.26) every element A ∈ A is of the form A = i Ei for a ﬁnite collection of Ei ∈ S , it is clear that if µ extends to a measure then the extension is unique and must be given by µ(A) = µ(Ei ).

i

**4.5 Constructing Finitely Additive Measures
**

Deﬁnition 4.24. A set S ⊂ 2X is said to be an semialgebra or elementary class provided that • ∅∈S • S is closed under ﬁnite intersections • if E ∈ S , then E c is a ﬁnite disjoint union of sets from S . (In particular X = ∅c is a ﬁnite disjoint union of elements from S .) Example 4.25. Let X = R, then ¯ S := (a, b] ∩ R : a, b ∈ R = {(a, b] : a ∈ [−∞, ∞) and a < b < ∞} ∪ {∅, R} is a semi-ﬁeld

(4.19)

To prove existence, the main point is to show that µ(A) in Eq. (4.19) is well deﬁned; i.e. if we also have A = j Fj with Fj ∈ S , then we must show µ(Ei ) =

i j

µ(Fj ).

(4.20) µ(Ei ∩

Exercise 4.5. Let A ⊂ 2X and B ⊂ 2Y be semi-ﬁelds. Show the collection E := {A × B : A ∈ A and B ∈ B} is also a semi-ﬁeld. Proposition 4.26. Suppose S ⊂ 2X is a semi-ﬁeld, then A = A(S ) consists of sets which may be written as ﬁnite disjoint unions of sets from S .

i

**But Ei = j (Ei ∩ Fj ) and the additivity of µ on S implies µ(Ei ) = Fj ) and hence µ(Ei ) =
**

i j

j

µ(Ei ∩ Fj ) =

i,j

µ(Ei ∩ Fj ).

Similarly,

macro: svmonob.cls date/time: 23-Feb-2007/15:20

Page: 32

job: prob

µ(Fj ) =

j i,j

µ(Ei ∩ Fj )

which combined with the previous equation shows that Eq. (4.20) holds. It is now easy to verify that µ extended to A as in Eq. (4.19) is an additive measure on A. Proposition 4.29. Let X = R, S be a semi-algebra S = {(a, b] ∩ R : −∞ ≤ a ≤ b ≤ ∞}, (4.21)

and A = A(S ) be the algebra formed by taking ﬁnite disjoint unions of elements from S , see Proposition 4.26. To each ﬁnitely additive probability measures µ : ¯ → [0, 1] such that A → [0, ∞], there is a unique increasing function F : R F (−∞) = 0, F (∞) = 1 and ¯. µ((a, b] ∩ R) = F (b) − F (a) ∀ a ≤ b in R (4.22)

¯ → [0, 1] such that F (−∞) = 0, Conversely, given an increasing function F : R F (∞) = 1 there is a unique ﬁnitely additive measure µ = µF on A such that the relation in Eq. (4.22) holds. Proof. Given a ﬁnitely additive probability measure µ, let ¯. F (x) := µ ((−∞, x] ∩ R) for all x ∈ R Then F (∞) = 1, F (−∞) = 0 and for b > a, F (b) − F (a) = µ ((−∞, b] ∩ R) − µ ((−∞, a]) = µ ((a, b] ∩ R) . ¯ → [0, 1] as in the statement of the theorem is Conversely, suppose F : R given. Deﬁne µ on S using the formula in Eq. (4.22). The argument will be completed by showing µ is additive on S and hence, by Proposition 4.28, has a unique extension to a ﬁnitely additive measure on A. Suppose that

n

(a, b] =

i=1

(ai , bi ].

**By reordering (ai , bi ] if necessary, we may assume that a = a1 < b1 = a2 < b2 = a3 < · · · < bn−1 = an < bn = b. Therefore, by the telescoping series argument,
**

n n

µ((a, b] ∩ R) = F (b) − F (a) =

i=1

[F (bi ) − F (ai )] =

i=1

µ((ai , bi ] ∩ R).

Given a probability measure.14 that B ⊂ 2X is a σ – algebra on X if B is an algebra which is closed under countable unions and intersections.2. Recall from Deﬁnition 3. 0 for x < −1 p for −1 ≤ x < 1 . Suppose that S ⊂ 2X is a semi-algebra. ∞ Nn µ(Ej ) ≤ n=1 i=1 µ (En. x]) .i ∈ S . 5. r > 0 and p + q + r = 1. – 4.4. ∞ So suppose A = n=1 An with A ∈ A and each An ∈ A which we express as Nn i=1 A= k j =1 Ej with Ej ∈ S and An = ∞ En. The measure. Proof. F 4. the cumulative distribution function (CDF) of P is deﬁned as the function. 1].i with En. Because of Proposition 4. If F = FP : R → [0. and is right continuous. Example 5.5 (Uniform Distribution). then: 1.6. The function. In this case.19 below. Proof. there exists a unique probability measure.1 Distribution Function for Probability Measures on (R.5 Countably Additive Measures 5. The uniqueness assertion in the theorem is covered in Exercise 5. The existence portion of the Theorem follows from Proposition 5. is non-decreasing.7 and Theorem 5. PF . b]) = F (b) − F (a) for all − ∞ < a ≤ b < ∞. m is called the uniform distribution or Lebesgue measure on (0. 1]. 0 for x ≤ 0 F (x) := x for 0 ≤ x < 1 . Example 5. BR ) Deﬁnition 5. F 2. F (x) = p + q for 1 ≤ x < π 1 for π ≤ x < ∞ Lemma 5. Then µ is a premeasure on A iﬀ µ is sub-additive on S .2.1 below.3. q. (∞) := limx→∞ F (x) = 1. 1] is a distribution function for a probability measure. in Lemma 5.additive and hence sub-additive on S . Clearly if µ is a premeasure on A then µ is σ . P on BR . P. on BR such that PF ((a. to prove the converse it suﬃces to show that the sub-additivity of µ on S implies the sub-additivity of µ on A. m on BR which is concentrated on (0. Summing this equation on j and using the ﬁnite additivity of µ shows . on BR . 1 for 1 ≤ x < ∞ is the distribution function for a measure. 1] satisfying properties 1.1.i ∩ Ej which is a countable union and hence by assumption. F = FP : R → [0.2 Construction of Premeasures Proposition 5. To each function F : R → [0. F (−∞) := limx→−∞ F (x) = 0. Then ∞ Nn Theorem 5.i ∩ Ej ) . 1] given as F (x) := P ((−∞.3. Ej = A ∩ Ej = n=1 An ∩ Ej = n=1 i=1 En. Suppose that P = pδ−1 + qδ1 + rδπ with p. F 3. ∞] is a ﬁnitely additive measure. A = A(S ) and µ : A → [0.

n n o∞ o and so by what we have already proved. b] ⊂ J. b] is compact and I 1 ˜o J n there exists N < ∞ such that o ˜n To see this. (5. b]) ≤ n=1 µ (Jn ) + ε J= n=1 Jn . implies F (bn ) − F (a) = µ((a. Hence premeasures on A which are ﬁnite on bounded sets are in one to one correspondences with right continuous increasing functions which vanish at 0.7. Jn = (an . x] is ﬁnitely covered o o ˜n ˜n by . b]) = F (b) − F (a) ∀ − ∞ < a < b < ∞. Because of Proposition 5. As above. say by J .i ∩ Ej ) = n=1 i=1 µ (En. Then J = (a. let c := sup x ≤ b : [˜ a. suppose that b = ∞ so that ∞ µ(J ) ≤ n=1 µ(Jn ). b]) = F (b) − F (a) ≤ n=1 ∞ ∞ ˜n µ J (5.i ) = n=1 µ (An ) . we have shown limy↓b F (y ) = F (b). Hence by ﬁnite sub-additivity of µ. bn ] ∩ R. ﬁnitely Now let M → ∞ in this last inequality to ﬁnd that macro: svmonob.cls date/time: 23-Feb-2007/15:20 Page: 36 job: prob .1) with Jn = (an . ∞ µ (J ) = µ((a. ˜ ˜n J bn ] ⊃ J := (an . bn ] such that ∞ ∞ {bn }n=1 F (b) − F (˜ a) = µ(I ) ≤ n=1 ˜n ) ≤ µ(J n=1 ˜n ). ∞ n=1 ∞ IM := (a. We wish to show ∞ which veriﬁes Eq. b]. F (±∞) := limx→±∞ F (x) and µ = µF be the ﬁnitely additive measure on (R. Using this in Eq. µ(J Given ε > 0. we may use the right continuity of F to choose ˜ bn so that ˜n \ Jn ) = F (˜ µ(J bn ) − F (bn ) ≤ ε2−n ∀ n ∈ N. M ] = J ∩ IM = 1 n=1 Jn ∩ IM ¯ = [˜ ¯⊂ J ⊂ Since I a. To each right continuous increasing function F : R → R there exists a unique premeasure µ = µF on A such that µF ((a. ˜ bn > bn in which case I := (˜ a. ˜ bn ) ⊃ J n . The hard work is now done but we still have to check the cases where a = −∞ or b = ∞. If µ happens to be a premeasure on A. Proposition 5. A) described in Proposition 4. then. n=1 o o ˜m for some m and there exists x ∈ J ˜m such that [˜ then c ∈ J a.36 5 Countably Additive Measures k k ∞ Nn N N µ(A) = j =1 µ(Ej ) ≤ j =1 n=1 i=1 ∞ Nn k µ (En. o ˜n := (an . ∞) = n=1 Jn To do this choose numbers a ˜ > a. to ﬁnish the proof it suﬃces to show µ is sub-additive on S . c ] for all c ∈ J . Since was an arbitrary sequence such that bn ↓ b. The next proposition shows the converse is true as well. x] is ﬁnitely covered by J .2) the sub-additivity of µ on A. F is right continuous. For example. But this contradicts the deﬁnition of c. Proof. Now suppose that F : R → R be an increasing function.2) = n=1 µ (Jn ) + n=1 ˜n \ Jn ). bn ] with bn ↓ b as n → ∞. i. (5. If c < b.29. J = (a. J = n=1 i=1 j =1 µ (En. N ∞ which proves (using Proposition 4. b]) = F (b) − F (a).N ) F (M ) − F (a) = µ(IM ) ≤ n=1 µ(Jn ∩ IM ) ≤ n=1 µ(Jn ). First suppose that −∞ < a < b < ∞.1) since ε > 0 was arbitrary. We would then have that J n=1 n=1 n=1 o ˜m covers [a.2) shows ∞ µ(J ) = µ((a. letting An = (a. bn ]) ↓ µ((a.i ∩ Ej ) ∞ Nn ∞ ¯⊂ I⊂I n=1 ˜o ⊂ J n n=1 ˜n .e. (5. ∞ ∞ n o∞ o ˜n J n oN n omax(m.29.6. let F (±∞) := limx→±∞ F (x) and µ = µF be as in Proposition 4. µ(J Using the right continuity of F and letting a ˜ ↓ a in the above inequality.

N ∞ ∞ Let A := ∪∞ := ∪N i=1 Ai . Applying this result to B c shows there exists D ∈ Aσ such that B c ⊂ D and µ (B \ Dc ) = µ (D \ B c ) < ε. Then: 1. B := ∪i=1 Bi . Before continuing our development of the existence of measures.5. i. . Therefore. µ) is a σ – ﬁnite measure space. We also write Eσδ = (Eσ )δ and Eδσ = (Eδ )σ . A i=1 Ai ∈ Aδ . Now suppose that Bi ∈ B0 for i = 1. and C := ∪i=1 Ci ∈ N Aσ . . follows from items 1. is straight forward and item 2. The function µ is σ – ﬁnite on E if there exists En ∈ E such that µ(En ) < ∞ and X = ∪∞ n=1 En . Aδ is closed under taking countable intersections and ﬁnite unions. 3. By assumption there exists Ai ∈ Aδ and Ci ∈ Aσ such that Ai ⊂ Bi ⊂ Ci and µ (Ci \ Ai ) < 2−i ε. By construction Aσ is closed under countable unions. Lemma 5.3 Regularity and Uniqueness Results ∞ 37 µ((a. which shows that Aσ is also closed under ﬁnite intersections. Aσ is closed under taking countable unions and ﬁnite intersections. Moreover if ∞ A = ∪∞ i=1 Ai and B = ∪j =1 Bj with Ai . macro: svmonob. then A ∩ B = ∪∞ i. Suppose X is a set. Bj ∈ A. Given a collection of subsets. ∞ ∞ µ (C \ A) = µ (∪∞ i=1 [Ci \ A]) ≤ i=1 µ (Ci \ A) ≤ i=1 µ (Ci \ Ai ) < ε. Let B0 denote the collection of B ∈ B such that for every ε > 0 there here exists A ∈ Aδ and C ∈ Aσ such that A ⊂ B ⊂ C and µ (C \ A) < ε. Deﬁnition 5. and 3.10 continues to hold under the weaker assumption that µ : B → [0. let Eσ denote the collection of subsets of X which are ﬁnite or countable unions of sets from E .8. Suppose A ⊂ 2X is an algebra. B = σ (A) and µ : B → [0. ∞] is a function.9.cls date/time: 23-Feb-2007/15:20 . Then A ⊂ A ⊂ B ⊂ C and Page: 37 job: prob Since C \ AN ↓ C \ A. Proof.e. 2. Hence B0 is a sub-σ -algebra of B = σ (A) which contains A which shows B0 = B .12 (σ – Finite Regularity Result). 2. by Theorem 5. Proof. Item 3. it also follows that µ C \ AN < ε for suﬃciently large N and this shows B = ∪∞ i=1 Bi ∈ B0 . Suppose that A ⊂ 2X is an algebra. The other cases where a = −∞ and b ∈ R and a = −∞ and b = ∞ are handled similarly. The reader should check that if µ is a ﬁnitely additive measure on an algebra. Theorem 5.11. So if we let A := Dc ∈ Aδ . Many theorems in the sequel will require some control on the size of a measure µ. Now let C := ∪∞ n=1 [Xn ∩ Cn ] ∈ Aσ and observe that B ⊂ C and µ (C \ B ) = µ (∪∞ n=1 ([Xn ∩ Cn ] \ B )) ∞ ∞ ≤ n=1 µ ([Xn ∩ Cn ] \ B ) = n=1 µ (Xn ∩ [Cn \ B ]) < ε.10. of X. . E ⊂ B ⊂ 2X and µ : B → [0. Then for every ε > 0 and B ∈ B there exists A ∈ Aδ and C ∈ Aσ such that A ⊂ B ⊂ C and µ (C \ A) < ε.10 (Finite Regularity Result). µ (X ) < ∞. then A ⊂ B ⊂ C and µ (C \ A) = µ ([B \ A] ∪ [(C \ B ) \ A]) ≤ µ (B \ A) + µ (C \ B ) < 2ε and the result is proved. ∞) is a ﬁnite measure. ∞)) = F (∞) − F (a) ≤ n=1 µ(Jn ). It is now clear that A ⊂ B0 and that B0 is closed under complementation. for every B ∈ B there exists Cn ∈ Aσ such that B ⊂ Cn and µ (Xn ∩ [Cn \ B ]) = µn (Cn \ B ) < 2−n ε. The relevant notion for our purposes (and most purposes) is that of a σ – ﬁnite measure deﬁned next. 5. Proof.3 Regularity and Uniqueness Results Deﬁnition 5. {Ac : A ∈ Aσ } = Aδ and {Ac : A ∈ Aδ } = Aσ . If B is a σ – algebra and µ is a measure on B which is σ – ﬁnite on B we will say (X. and ε > 0 is given. etc.Since A ∈ B →µn (A) := µ (Xn ∩ A) is a ﬁnite measure on A ∈ B for each n. then µ is σ – ﬁnite on B iﬀ there exists Xn ∈ B such that Xn ↑ X and µ(Xn ) < ∞. This detour will also have the added beneﬁt of motivating Carathoedory’s existence proof to be given below. E . B .j =1 Ai ∩ Bj ∈ Aσ . Theorem 5. Let Xn ∈ A such that ∪∞ n=1 Xn = X and µ(Xn ) < ∞ for all n. ∞ ∞ C \ A = [∪∞ i=1 Ci ] \ A = ∪i=1 [Ci \ A] ⊂ ∪i=1 [Ci \ Ai ] . Corollary 5. B . ∞] is a measure which is σ – ﬁnite on A. we will pause to show that measures are often uniquely determined by their values on a generating sub-algebra. Similarly let Eδ denote the collection of subsets of X which are ﬁnite or countable intersections of sets from E .

n We may let n → ∞ in this inequality to ﬁnd. ∞] is a premeasure on A iﬀ µ (An ) ↑ µ(A) for all ∞ {An }n=1 ⊂ A such that An ↑ A ∈ A. Let us recall from Proposition 4.N ) ≤ n=1 µ (An ) ≤ n=1 µ (An ) .4 Construction of Measures Remark 5. 1. Let A. (5. B be sets in Aσ such that A ⊂ B and suppose {An }n=1 and ∞ {Bn }n=1 are sequences in A such that An ↑ A and Bn ↑ B as n → ∞. With this deﬁnition. Show µ = ν. let Vk := ∪x∈F B (x. then A ⊂ B ⊂ C and µ(C \ A) = µ(C \ B ) + µ(B \ A) ≤ µ(Cn \ B ) + µ(B \ An ) ≤ 2/n → 0 as n → ∞. then µ has a unique extension (still called µ) to a function on Aσ satisfying the following properties. the continuity of µ is clear and the monotonicity of µ follows from Eq.2. (5. we may assume that An ↑ and Cn ↓ as n increases. By Theorem 5. Since Bm ∩ An ↑ An as m → ∞.3). the continuity of µ on A implies. Let µ be a premeasure on an algebra A. Let A = ∪An ∈ Aδσ and C = ∩Cn ∈ Aσδ . 2. (Monotonicity) If A.i }i=1 ⊂ A such that An. (5. Proof. 3.i ↑ An as i → ∞. and µ (V \ F ) < ε.6) macro: svmonob.Additivity on Aσ ) The function µ is countably additive on Aσ . δ ) := {y ∈ Rn : |y − x| < δ } . such that K ⊂ B and µ (B \ K ) < ε.14. m→∞ m→∞ ∞ Exercise 5. Show B0 is a σ – algebra and use this along with the ﬁrst part of this exercise to conclude B = B 0 . we may choose An ∈ Aδ and Cn ∈ Aσ such that An ⊂ B ⊂ Cn and µ(Cn \ B ) ≤ 1/n and µ(B \ An ) ≤ 1/n. Then passing to the limit as n → ∞ in the identity. implies. Show: 1. Vk ↓ F as k → ∞. given B ∈ B . Then we have. µ (An ∪ Bn ) + µ (An ∩ Bn ) = µ (An ) + µ (Bn ) ∞ {An }n=1 5. n→∞ lim µ (An ) ≤ lim µ (Bm ) .10. Suppose A ⊂ 2X is an algebra and µ : B = σ (A) → [0.38 5 Countably Additive Measures Exercise 5. Generalize the previous assertion to the case where you only assume that µ and ν are σ – ﬁnite on A. Show. where B (x. b. there exists A ∈ Aδσ and C ∈ Aσδ such that A ⊂ B ⊂ C and µ (C \ A) = 0.3) 4. In particular. 3. (σ . m→∞ (5. µ (A) = lim µ (An ) n→∞ for any sequence ⊂ A such that An ↑ A.e. F ⊂ R and k ∈ N. Then for all B ∈ B. ∞ ∞ Let {An }n=1 be any sequence in Aσ and choose {An. then µ (A ∪ B ) + µ (A ∩ B ) = µ (A) + µ (B ) . B ∈ Aσ . (5.4) 5. then µ (An ) ↑ µ (A) as n → ∞. Corollary 5. V is open.cls date/time: 23-Feb-2007/15:20 . K ⊂ Rn .4 that a ﬁnitely additive measure µ : A → [0. Proposition 5. N By replacing AN by ∪N n=1 An and CN by ∩n=1 Cn . (5. B ∈ Aσ with A ⊂ B then µ (A) ≤ µ (B ) . ∞ ∞ Suppose that A. i. Let B = BRn = σ ({open subsets of Rn }) be the Borel σ – algebra on Rn and µ be a probability measure on B . Furthermore if µ (X ) < ∞.3 and Remark 4. then µ is a ∞ premeasure on A iﬀ µ(An ) ↓ 0 for all {An }n=1 ⊂ A such that An ↓ ∅.1. there exist a compact subset. then ∞ µ (∪∞ n=1 An ) ≤ n=1 µ (An ) . Suppose that µ and ν are ﬁnite measures such that µ = ν on A. Hint: follow closely the method used in the ﬁrst step of the proof of Theorem 5. Page: 38 job: prob proves Eq.15. Show for every ε > 0 and B ∈ B . 2. (Sub-Additivity on Aσ ) The function µ is sub-additive on Aσ . Therefore it is unambiguous to deﬁne µ (A) by. it follows that µ is ﬁnitely additive on Aσ . Suppose A ⊂ 2X is an algebra and µ and ν are two measures on B = σ (A) .N ≤ n=1 µ (An.10.13. (Continuity) If An ∈ A and An ↑ A ∈ Aσ .5) Using this equation when B = A. Further. (Strong Additivity) If A. a. Hint: given a closed subset.5). ∞] is a measure which is σ – ﬁnite on A. Hint: take K := F ∩ {x ∈ Rn : |x| ≤ n} for some suﬃciently large n. limn→∞ µ (An ) = limm→∞ µ (Bm ) whenever An ↑ A and Bn ↑ A. µ (An ) = lim µ (Bm ∩ An ) ≤ lim µ (Bm ) . B ∈ Aσ and {An }n=1 and {Bn }n=1 are sequences in A such that An ↑ A and Bn ↑ B as n → ∞. let B0 denote those sets B ∈ B such that for every ε > 0 there exists F ⊂ B ⊂ V such that F is closed. 1/k ) . B0 contains all closed subsets of B . if ∞ {An }n=1 ⊂ Aσ . N N ∞ µ ∪N n=1 An. Proof.

it follows that µ (X ) = µ (A) + µ (Ac ) . Ci ∈ Aσ .6) to conclude Eq. and 0 ≤ µ (C ) − µ ¯ (B ) < ε.7) Remark 5. and A ∈ Aδ ∩ Aσ . Ci ∈ Aσ . Then with A = A1 ∪ A2 . Suppose µ is a ﬁnite premeasure on an algebra A ⊂ 2X .3. ∞ and µ (Ci \ Ai ) < ε2−i for all i. Suppose µ is a ﬁnite premeasure on an algebra A ⊂ 2X . µ is additive when restricted to Aδ .4) holds. Suppose µ is a ﬁnite premeasure on an algebra. then µ (A) ≤ µ ¯ (B ) ≤ µ (C ) and in particular. Bn ∈ A such that An ↓ A and Bn ↓ B. It is clear that A ⊂ B and that B is closed under complementation. n→∞ n→∞ 2. µ (A) ≤ µ (C \ A) + µ (A) = µ (C ) from which it follows that µ (A) ≤ µ ¯ (B ) .3) with A → Ac ∈ Aσ and B → C ∈ Aσ shows µ (X ) + µ (C \ A) = µ (Ac ∪ C ) + µ (Ac ∩ C ) = µ (Ac ) + µ (C ) = µ (X ) − µ (A) + µ (C ) . if C ∈ Aσ with B ⊂ C . Since Ac n ↑ A ∈ Aσ . A ∈ Aδ and C ∈ Aσ are such that A ⊂ B ⊂ C and µ (C \ A) < ε. 2.16 to show. we have Aδ A ⊂ B ⊂ C ∈ Aσ . and µ has been extended to Aδ ∪ Aσ as described in Proposition 5. it follows from the sub-additivity of µ that with µ (C \ A) ≤ µ (C1 \ A1 ) + µ (C2 \ A2 ) < 2ε. µ (A ∪ B ) = lim µ (An ∪ Bn ) = lim [µ (An ) + µ (Bn ) − µ (An ∩ Bn )] n→∞ n→∞ = µ (A) + µ (B ) wherein the last equality we have used Proposition 4. Since the {Ai }i=1 are pairwise disjoint we may use Lemma 5. Proof. 3.15 it follows that µ (A) = µ (X ) − µ (Ac ) = µ (X ) − lim µ (Ac n) n→∞ = lim [µ (X ) − µ (Ac n )] = lim µ (An ) . We say that B ⊂ X is measurable if for all ε > 0 there exists A ∈ Aδ and C ∈ Aσ such that A ⊂ B ⊂ C and µ (C \ A) < ε. So applying the strong additivity of µ on Aσ in Eq. By assumption. 2. µ ¯ is the unique measure on B such that µ ¯|A = µ. by the ﬁnite additivity and monotonicity of µ on Aσ . to verify that B is a σ – algebra it suﬃces to show ∞ ∞ that B = n=1 Bn ∈ B whenever {Bn }n=1 is a disjoint sequence in B . then µ (C \ A) = µ (C ) − µ (A) . Page: 39 job: prob . A ⊂ 2X . Moreover. 3. Because B is an algebra.16. Ac ∈ Aσ and X = A ∪ Ac .8). Then B is a σ – algebra on X which contains A and µ ¯ is a σ – additive measure on B . We will denote the collection of measurable subsets of X by B = B (µ) .8) The previous two inequalities show µ is σ – additive on Aσ . ε > 0. ∞ If we further assume that {An }n=1 ⊂ Aσ is a disjoint sequence. macro: svmonob. we may let N → ∞ in Eq. 0≤µ ¯ (B ) − µ (A) < ε.N ↑ ∪n=1 An ∈ Aσ . From this observation we may extend µ to a function on Aδ ∪ Aσ by deﬁning µ (A) := µ (X ) − µ (Ac ) for all A ∈ Aδ . Now suppose that Bi ∈ B for i = 1. Suppose µ is a ﬁnite premeasure on an algebra. To prove B ∈ B .7) above. Indeed.cls date/time: 23-Feb-2007/15:20 (5. Since C \ A = (C1 \ A) ∪ (C2 \ A) ⊂ (C1 \ A1 ) ∪ (C2 \ A2 ) . by the deﬁnition of µ (A) and Proposition 5.15 and Eq. we have ∞ N 39 µ (An ) = lim n=1 N →∞ ∞ µ (An ) = lim µ ∪N n=1 An ≤ µ (∪n=1 An ) . then µ (A) = limn→∞ µ (An ) . then An ∪ Bn ↓ A ∪ B and therefore. c 1. 1. If A ∈ Aδ and An ∈ A such that An ↓ A. We may then choose Ai ⊂ Bi ⊂ Ci such that Ai ∈ Aδ . (5. If B ∈ B. We also deﬁne µ ¯ : B → [0. let ε > 0 be given and choose Ai ⊂ Bi ⊂ Ci such that Ai ∈ Aδ . B ∈ Aδ are disjoint sets and An . Hence we now know that B is an algebra. Proof. and µ (Ci \ Ai ) < ε for i = 1. then A ⊂ C and so by Lemma 5. Theorem 5. Since A. (5. (5.4 Construction of Measures ∞ Since A ∪N n=1 An. (5. B = B1 ∪ B2 and C = C1 ∪ C2 .16. (5. n=1 N →∞ Deﬁnition 5. Since ε > 0 was arbitrary. we have shown that B ∈ B . µ (X )] by µ ¯ (B ) = inf {µ (C ) : B ⊂ C ∈ Aσ } .17 (Measurable Sets). (5.5. (5. A ⊂ 2X . If A ∈ Aδ and C ∈ Aσ such that A ⊂ C.19 (Finite Premeasure Extension Theorem). The fact that µ ¯ (B ) ≤ µ (C ) follows directly from Eq.18. X = Ac ∪ C. Suppose A.9) Lemma 5. 2 and ε > 0 is given.

then one easily shows that mx is a measure on BR such that mx ((a. Example 5. ≤ε+ i=n+1 Thus µ ¯ (B ) ≤ µ ¯ (C ) = µ ¯ (B ) + µ ¯(C \ B ) ≤ µ ¯ (B ) + ε which. Lebesgue measure m is invariant under translations. there exists Cm ∈ Aσ such that Bm ⊂ Cm ⊂ Xm and µ ¯(Cm \ Bm ) = µ ¯m (Cm \ Bm ) < ε2−n .10) µn (A) := µn (A ∩ Xn ) for all A ∈ A. µ ¯ (Bi ) ≤ 2ε. ∞ Then C := ∪m=1 Cm ∈ Aσ and ∞ ∞ ∞ µ ¯(C \ B ) ≤ µ ¯ µ (Ci ) → ε as n → ∞. Given n ∈ N. µ ¯ := limn→∞ µ ¯n is a measure which extends µ. Then µ ¯ (B ) := inf {µ (C ) : B ⊂ C ∈ Aσ } ∀ B ∈ σ (A) (5. Page: 40 job: prob ∞ . Let mx (B ) := m(x + B ). since ε > 0 is arbitrary.19. For the converse.22. The proof will be completed by verifying that Eq. m=1 (Cm \ B ) ≤ m=1 µ ¯(Cm \ B ) ≤ m=1 µ ¯(Cm \ Bm ) < ε. we have macro: svmonob. suppose that m is translation invariant and m((0. µ ¯n . 1]) = 1 and Eq. C := ∪i=1 Ci ∈ Aσ and for n ∈ N let A := n n Then Aδ A ⊂ B ⊂ C ∈ Aσ . By Theorem 5. it follows that B ∈ B .cls date/time: 23-Feb-2007/15:20 µ ¯ (B ) − i=1 Since ε > 0 is arbitrary. n n ∞ C \ An = ∪∞ i=1 (Ci \ A ) ⊂ [∪i=1 (Ci \ Ai )] ∪ ∪i=n+1 Ci ∈ Aσ .11). (5. mx = m by the uniqueness assertion in Exercise 5. i. ∞ where λ ∈ R. This completes the proof that B is a σ . b]) = b − a for all a < b.19 each µn has an extension. for B ∈ BR and x ∈ R. Since the measure µ ¯n are increasing.13) µ ¯ (B ) − i=1 µ ¯ (Bi ) < 2ε + i=n+1 µ (Ci ) which upon letting n → ∞ gives. to a measure on σ (A) .10). m(x + B ) = m(B ).1.e. in this equation then shows ∞ µ (Ci ) ≤ µ (X ) + ε < ∞. we denote µF by m and call m Lebesgue measure on (R.11) ε2−i . 2−i < ε. Let {Xn }n=1 ⊂ A be chosen so that µ (Xn ) < ∞ for all n and Xn ↑ X as n → ∞ and let (5.18. m is the unique measure on BR such that m((0. (5. BR ) .12) Moreover. deﬁnes a measure on σ (A) and this measure is the unique extension of µ on A to a measure on σ (A) . n ∞ µ (C \ An ) ≤ i=1 µ (Ci \ Ai ) + i=n+1 ∞ µ (Ci ) Each µn is a premeasure (as is easily veriﬁed) on A and hence by Theorem 5.21. Suppose that µ is a σ – ﬁnite premeasure on an algebra A. The uniqueness of the extension µ ¯ is proved in Exercise 5. (5. If F (x) = x for all x ∈ R. 1]) = 1. using the sub-additivity of µ on Aσ and the estimate (5. i=1 Since ε > 0 is arbitrary. shows µ ¯ satisﬁes Eq. B ∈ BR and λB := {λx : x ∈ B }. Proof. Therefore. Proof. n i=1 ∞ = µ (∪n i=1 Ai ) + i=1 µ (Ci \ Ai ) ≤ µ (X ) + i=1 Passing to the limit. we have shown µ ¯ (B ) = i=1 µ ¯ (Bi ) .1. Moreover by repeated use of Remark 5. C \ A ∈ Aσ and Ai ∈ Aδ .20. n → ∞. (5.12) holds for B ∈ BR and x ∈ R. Therefore.11) holds. Moreover. i=1 ∞ n Let B = ∪∞ i=1 Bi . Bm = Xm ∩ B and ε > 0 be given. m has the scaling property m(λB ) = |λ| m(B ) (5. Let B ∈ σ (A) .40 5 Countably Additive Measures n n µ (Ci ) = i=1 i=1 (µ (Ai ) + µ (Ci \ Ai )) n n Theorem 5.algebra and that µ ¯ is a measure on B . we ﬁnd ∞ |µ ¯ (B ) − µ (An )| < ε + i=n+1 n n µ (Ci ) and n n µ ¯ (Bi ) − µ (An ) = i=1 i=1 [¯ µ (Bi ) − µ (Ai )] ≤ i=1 |µ ¯ (Bi ) − µ (Ai )| ≤ ε Combining these estimates shows n ∞ Theorem 5.

m ([λb. B . If Ai ∈ B and Ni ⊂ Fi ∈ B such that µ(Fi ) = 0 then ¯ since ∪Ai ∈ B and ∪Ni ⊂ ∪Fi and ∪(Ai ∪ Ni ) = (∪Ai ) ∪ (∪Ni ) ∈ B ¯ µ(∪Fi ) ≤ µ(Fi ) = 0. n l ]) = l/n for all l. bn ] ↓ (a. Completing a σ – algebra. f = g a. and (X.e. i. (A ∪ N )c = Ac ∩ N c = Ac ∩ (F \ N ∪ F c ) = [Ac ∩ (F \ N )] ∪ [Ac ∩ F c ] ¯ is closed under where [Ac ∩ (F \ N )] ∈ N and [Ac ∩ F c ] ∈ B . k−1 k . For example if f and g are two measurable functions on (X. ¯µ := {A ∪ N : A ∈ B and N ∈ N } and B=B µ ¯(A ∪ N ) := µ(A) for A ∈ B and N ∈ N . choose an . m is Lebesgue measure. we will use the terminology P a. n n Deﬁnition 5. m((0. Page: 41 job: prob . b ∈ Q with a < b. is called the completion of B relative to µ and µ space. m((0. Proposition 5. b]) = |λ| if λ < 0. ]) = n · m((0. m((a. ]). Then B ¯ is a well deﬁned measure on B ¯ is the ¯ which extends µ on B . µ) be a measure space.1. Clearly X. n→∞ n→∞ i. B .e. is called the completion of µ. n invariance of m.25 (Completion of a Measure). b]) = lim m((an . ∅ ∈ B n 1 = m((0.24.5. Suppose A ∪ N1 = B ∪ N2 with A. µ) is complete if every subset of a null set is in B . 1] = ∪n k=1 ( Therefore. bn ∈ Q such that bn ↓ b and an ↑ a. ¯ is a σ – algebra. µ). ] n n = k=1 1 1 m((0. Now let mλ (B ) := |λ| m(λB ). µ unique measure on B ¯) is complete measure ¯. −1 Fig. (to be read P almost everywhere) to mean E := {x ∈ X : P is false for x} is a null set.5 Completions of Measure Spaces 41 (0. 5. Set N = N µ := {N ⊂ X : ∃ F ∈ B such that N ⊂ F and µ(F ) = 0} . Then A ⊂ A ∪ N1 ⊂ A ∪ N1 ∪ F2 = B ∪ F2 which shows that µ(A) ≤ µ(B ) + µ(F2 ) = µ(B ). A set E ⊂ X is a null set if E ∈ B and µ(E ) = 0. λa)) = − |λ| −1 (λb − λa) = b − a that N ⊂ F and µ(F ) = 0. b]) = b − a for all a. B ¯.23. n ∈ N and therefore by the translation Similarly. b]) = λ−1 m ((λa. ] . Finally for a.e. A measure space (X. (5. If P is some “property” which is either true or false for each x ∈ X.1. µ see Fig. To prove Eq. then (an . The σ -algebra. for all F ⊂ X such that F ⊂ E ∈ B with µ(E ) = 0 implies that F ∈ B . n n That is to say 1 ]) = 1/n. b ∈ R such that a < b. Hence mλ = m.5 Completions of Measure Spaces Deﬁnition 5. ∈ N . B ¯. N2 . Thus B complements. Therefore.cls date/time: 23-Feb-2007/15:20 5. b] and thus m((a. bn ]) = lim (bn − an ) = b − a. 1]) = k=1 n m 1 k−1 + (0. 5.13) we may assume that λ = 0 −1 since this case is trivial to prove. B . means that µ(f = g ) = 0. µ ¯. Since N c = (F \ N ) ∪ F c . macro: svmonob. B ∈ B and N1 . Let (X. It is easily checked that mλ is again a measure on BR which satisﬁes mλ ((a. ] = ∪n k=1 n n k−1 1 + (0. B is a σ – algebra. Let A ∈ B and N ∈ N and choose F ∈ B such Proof. ¯.e. λb]) = λ−1 (λb − λa) = b − a if λ > 0 and mλ ((a.

. . But this is equivalent to showing λ ∈ Ak . This is a special case of Exercise 5. i. . Show A is an algebra. Then there exists a unique probability measure.19. we may ﬁnd Nn ∈ N and Bn ⊂ ΛNn such that An = Bn × Λ∞ . Since An ∈ A. . for all n ∈ N and (λ1 .28 (Existence of iid simple R. Moreover. . . . Continuing the notation above. . λn ) := P ({ω ∈ Ω : ω1 = λ1 . Lemma 5.a. λn ) ∈ Λn . Exercise 5.V. . . . . since ω (n) ∈ An ⊂ Ak for all k ≤ n. ωn = λn }) = q (λ1 ) .4). λ∈Λ p1 (λ) = 1 and 2.). Then there exists a unique probability measure P on σ (A) such that. From Theorem 5. Similarly. .4 (Consistency Conditions). . . . Using the observation just prior to this Lemma. . λ2 . for all n ∈ N and (λ1 . This certainly implies limn→∞ P (An ) = 0. . P on σ (A) such that Eq. 1] is a function such that λ∈Λ q (λ) = 1.42 5 Countably Additive Measures Similarly. . n and in particular P (An ) = 0 for a. we may ﬁnd a λ1 ∈ Λ and an inﬁnite subset.5 with pn (λ1 . . . . . . . for all N ∈ N we have (ω1 (n) . it follows that (ω1 (n) . ) ∈ Ω such that ω (n) ∈ An for all n. λn ) = λ∈Λ pn+1 (λ1 . . . ωN (n)) = (λ1 .3. λn ) := q (λ1 ) . . . pn (λ1 . ωn = λn }) . where A ⊂ Ω is a cylinder set iﬀ there exists n ∈ N and B ⊂ Λn such that A = B × Λ∞ := {ω ∈ Ω : (ω1 . λ) . . we learn from Eq. there exists ω (n) = (ω1 (n) . We are now going to complete the proof by showing that λ := (λ1 . Ω := Λ∞ := ΛN . λn . . let Λ be a ﬁnite set. . . Suppose {An }n=1 ⊂ A is a decreasing sequence of non-empty cylinder sets. (5. . Continuing this procedure inductively. (5. ) is in ∩∞ n=1 An . . Given a probability measure. By the construction above.e. . . q (λn ) . λNk ) = (ω1 (n) . by Lemma 5. . . Γ2 ⊂ Γ1 . we may assume ∞ that {Nn }n=1 is a strictly increasing sequence. λn ) ∈ Λn . let pn (λ1 . ω2 (n) . Page: 42 job: prob ∞ Exercise 5. . . then ∩∞ n=1 An = ∅. if An ∈ A and An ↓ ∅. (5. Since k ∈ N was arbitrary it follows that λ ∈ ∩∞ n=1 An . it suﬃces to show limn→∞ P (An ) = 0 whenever ∞ {An }n=1 ⊂ A with An ↓ ∅. However. . Example 5. . If pn is deﬁned as above. .cls date/time: 23-Feb-2007/15:20 . and ωj (n) = λj for all n ∈ Γj . Γ1 ⊂ N such that ω1 (n) = λ1 for all n ∈ Γ1. and let A denote the collection of cylinder subsets of Ω. . . ωNk (n)) ∈ Bk for all k ≤ n. . . Suppose now that q : Λ → [0. . Observe that we may also write A as A = B × Λ∞ where B = B × Λk ⊂ Λn+k for any k ≥ 0. P : A → [0. that it is countable additive. . . . q (λn ) . Theorem 5. .15) 5. . .27 (Kolmogorov’s Extension Theorem I. ωn ) ∈ B } . such that ω2 (n) = λ2 for all n ∈ Γ2 .6 A Baby Version of Kolmogorov’s Extension Theorem For this section. . . we must have that An = ∅ for a. . . . . Γj ⊂ N and points λj ∈ Λ such that Γ1 ⊃ Γ2 ⊃ Γ3 ⊃ . we have P ({ω ∈ Ω : ω1 = λ1 . Suppose for each n ∈ N we are given functions. . macro: svmonob. . .5 (Converse to 5. we show that µ(B ) ≤ µ(A) so that µ(A) = µ(B ) and hence µ ¯(A ∪ N ) := µ(A) is well deﬁned. .4 hold. It is left as an exercise to show µ ¯ is a measure. .26. λn ) ∈ Λn . λn ) ∈ Λn . . ωNk (n)) ∈ Bk . . λN ) for all n ∈ ΓN . Exercise 5.s). there exists (for all j ∈ N) inﬁnite subsets. (5.a. 1] and n ∈ N and (λ1 . 1] . every ﬁnitely additive probability measure. show: 1. . ω2 (n) . . . has a unique extension to a probability measure on σ (A) .26. By assumption. Proof. . there exists λ2 ∈ Λ and an inﬁnite set. . pn : Λn → [0. Taking N = Nk and n ∈ ΓNk with n ≥ k.14) Since Λ is a ﬁnite set. Proof. . P : σ (A) → [0. 1] such that the consistency conditions in Exercise 5.15) holds for all n ∈ N and (λ1 .14) that (λ1 . . n. .

i. f : X → Y is measurable iﬀ f −1 (E ) ⊂ M. Because of Corollary 6.5. f −1 (B ) ∈ M for all B ∈ F and therefore 1 −1 f |− (B ) ∈ MA for all B ∈ F . F ) are measurable spaces and further assume E ⊂ F generates F . F ) – measurable. F ) and a subset A ⊂ X. A function f : X → Y is measurable of more precisely. b]) ∈ M for all a < b. using Lemma 3.. M/F – measurable or (M. If f is measurable and A ⊂ X then f |A : A → Y is measurable. M) and (Y. Corollary 6. if f −1 (A) ∈ M for all A ∈ F .algebra on X then F = f∗ M ={A ∈ 2Y |f −1 (A) ∈ M} is the largest σ – algebra on Y such that f is (M. M) be a measure space and fn : X → R be a sequence of measurable functions on X. f −1 ((−∞.7 below. ai+1 ]).9 (Localizing Measurability).measurable . Then a map. F ) . 1. if f −1 (F ) ⊂ M. A (B ) = A ∩ f ai µ(f −1 (ai . Let (X. For this to make sense we will need to require f −1 ((a. Suppose there exist An ∈ M such that X = ∪∞ n=1 An and f |An is MAn measurable for all n.2. BR ) – measurable iﬀ A ∈ M. f is (M. 0<a1 <a2 <a3 <. 1− A (W ) is either −1 c ∅. ∞)) ∈ M for all a ∈ R.. Deﬁnition 6.e. then f is M – measurable. if M is a σ . Show that every monotone function f : R → R is (BR . f −1 (F ) = f −1 (σ (E )) = σ f −1 (E ) ⊂ M. then f −1 (E ) ⊂ f −1 (F ) ⊂ M. Exercise 6. then. f −1 ((a. M) and (Y. M) and (Y.7. 2. Similarly. Given measurable spaces (X. Suppose f : X → Y with Y being a ﬁnite set and F = 2 . M) and (Y. A or A for any W ⊂ R with 1A ({1}) = A. Example 6.7. Suppose that (X. ∞)) ∈ M for all a ∈ Q. Indeed. Then the following conditions on a function f : X → R are equivalent: 1. Exercise 6. BR ) – measurable. Let (X. Example 6. If f is M/F measurable. Exercise 6.e. Proof. Hint: See Exercise 3. Deﬁnition 6. BR ) – measurable. Conversely if f −1 (E ) ⊂ M. Show that {x : limn→∞ fn (x) exists in R} ∈ M.1 Measurable Functions Deﬁnition 6. If M is the σ – algebra generated by E ⊂ 2X . 3.6. Let (X. suppose (X.1.9. where X is a set and M is a σ – algebra on X. Remark 6.4 (Characteristic Functions). f −1 ((a. Let f : X → Y be a function. Then 1A is (M.3. Roughly speaking. M) be a measurable space and 1 A ⊂ X. Ω Y . F ) be measurable spaces and f : X → Y be a function. M) is a measurable space. X ∞ Proof. Then f is measurable iﬀ f −1 ({y }) ∈ M for all y ∈ Y. 2. M. A measurable space is a pair (X. a]) ∈ M for all a ∈ R. µ) is a measure space and f : X → R+ is a function. To motivate the notion of a measurable function.8. F = σ (E ) . i.measurable. Let (X. we are going to deﬁne f dµ as a certain limit of sums of the form.2. Prove Corollary 6. We say a function f : A → Y is measurable iﬀ f is MA /F – measurable. If f : X → Y is measurable.26. Proposition 6.6 Random Variables 6. F ) be measurable spaces. X. 1. F ) . Exercise 6. Suppose that (X. M). Proposition 6.4. the σ – algebra M := f −1 (F ) is the smallest σ – algebra on X such that f is (M.3. then M is the union of the σ – algebras generated by countable subsets F ⊂ E .1. this last condition is equivalent to the condition f −1 (BR ) ⊂ M. Given a σ – algebra F ⊂ 2 . 4.

b]) = F (b) − F (a) for all −∞ < a < b < ∞. The proof of the following exercise is routine and will be left to the reader. Since G : (0.1. F : R → [0. Then Y : (0. Thus we have shown {G ≤ x0 } ⊂ (0. 6. y ∈ G−1 ((−∞. (6. By the right continuity of F.3) by.44 6 Random Variables 2. i. it follows that F (x0 ) ≥ y. M. MAn ⊂ M and so the previous displayed equation shows f −1 (B ) ∈ M. As can be seen from this picture. 1) → R be deﬁned (see Figure 6. F (x0 )] ∩ (0. G (y ) ≤ x0 iﬀ y ≤ F (x0 ) and similalry. Let (X. 1) → R is Borel measurable and Y∗ m = µF where µF is the unique measure on (R. x0 ]) . 61 of Resnick for more details. Page: 44 job: prob (6. x]) = m G−1 ((−∞. µ) be a measure space. To give a formal proof of Eq. 1) : G (y ) ≤ x}) = m ((0. Theorem 6. (G∗ m) ((−∞. b]) = F (b) − F (a) for all −∞ < a < b < ∞.5.cls date/time: 23-Feb-2007/15:20 Proof. F : R → [0. F (x)] ∩ (0. Proposition 6. x0 ]) iﬀ G (y ) ≤ x0 . macro: svmonob. See section 2. 1) → R is Borel measurable and G∗ m = µF where µF is the unique measure on (R. As a consequence we have G∗ m = µF . 1) → R is a non-decreasing function. there exists xn ≥ x0 with xn ↓ x0 such that F (xn ) ≥ y. if y ≤ F (x0 ) then G (y ) = inf {x : F (x) ≥ y } ≤ x0 . Given a distribution function. G is measurable. 1] let G : (0.11. Fig.1). Indeed.e. ∞] by ν (A) := µ(f −1 (A)) for all A ∈ F . BR ) such that µF ((a. Theorem 6. 6. Y (x) := sup {y : F (y ) < x} . If B ∈ F . F (x0 )] ∩ R. 1) → R be deﬁned (see Figure 6. G (y ) ≤ x1 iﬀ y ≤ x1 . F ) . F (x0 )] ∩ (0. F ) be a measurable space and f : X → Y be a measurable map. x]) = m ({y ∈ (0.1) by. see Figure 6. x0 ]) = {y : G (y ) ≤ x0 } = (0. Observe that G (F (x0 )) = inf {x : F (x) ≥ F (x0 )} ≤ x0 and hence G (y ) ≤ x0 whenever y ≤ F (x0 ) . 1) ⊂ G−1 ((0. y ∈ {G ≤ x0 } . 1) . 1] let Y : (0. Given a distribution function.10.2. Then G : (0. Then ν is a measure on (Y. Fig.1) .2. (In the future we will denote ν by f∗ µ or µ ◦ f −1 and call f∗ µ the push-forward of µ by f or the law of f under µ. We also claim that. This shows that (0. then −1 −1 f −1 (B ) = ∪∞ (B ) ∩ An = ∪∞ n=1 f n=1 f |An (B ). BR ) such that µF ((a. for all x0 ∈ R. Deﬁne a function ν : F → [0. 1)) = F (x) . Since each An ∈ M.12 (Durret’s Version). that G−1 ((0.2 on p. A pictorial deﬁnition of G. G (y ) := inf {x : F (x) ≥ y } . (Y. Indeed. G (y ) = inf {x : F (x) ≥ y } ≤ x0 . For the converse.

then the composition fα ◦ g is (M. Suppose that (X. . Deﬁnition 6. . then F (y ) < x and hence. Let X and Y be two topological spaces and f : X → Y be a continuous function.13.16.cls date/time: 23-Feb-2007/15:20 Therefore. M) → (Z. πn ) = A × ΛN −n : A ⊂ Λn . i.13 (Composing Measurable Functions). . G ) are measurable spaces.2) and therefore If fα ◦ g is (M. F = 2Y . F (y0 )] ∩ (0. 1) and x ≤ F (y0 ) then (by deﬁnition of Y (x)) y0 ≥ Y (x) . i. macro: svmonob. Fα ) – measurable by Lemma 6.” if x ∈ (0. (⇐) Let −1 G = σ (fα : α ∈ A) = σ ∪α∈A fα (Fα ) . −1 σ (fα : α ∈ A) = σ (∪α fα (Fα )). G ) is measurable as well. F (y0 )] ∩ (0. (6. (⇒) If g is (M. Proof. We will now show {x ∈ (0. Lemma 6. Let X be a set and suppose there is a collection of measurable spaces {(Yα . y0 )) = m Y −1 −1 −1 g −1 ∪α∈A fα (Fα ) = ∪α∈A g −1 fα (Fα ) ⊂ M. Then g is (M. Then f is Borel measurable. .17. By assumption g −1 (G ) ⊂ F and f −1 (F ) ⊂ M so that (g ◦ f ) −1 (G ) = f −1 g −1 (G ) ⊂ f −1 (F ) ⊂ M.18. σ (fα : α ∈ A)) – measurable iﬀ fα ◦ g is (M. it follows that Y is measurable and (Y∗ m) ((−∞.3. we have F (y ) ≥ x and therefore. Suppose that Y is a ﬁnite set. . Fα ) : α ∈ A} and functions fα : X → Yα for all α ∈ A. Example 6. πi (y1. Deﬁnition 6. F ) → (Z. Since Y : (0. then x ≤ F (Y (x)) ≤ F (y0 ). A pictorial deﬁnition of Y (x) . and X = Y N for some N ∈ N. A function f : X → Y between two topological spaces is Borel measurable if f −1 (BY ) ⊂ BX . then −1 g −1 fα (Fα ) ⊂ M ∀ α ∈ A For the inclusion “⊂.15. 1) : Y (x) ≤ y0 } = (0. 1) . (6. yN ) = yi .14 and additionally let (Z.e. Y is measurable. y ↑Y (x) For y > Y (x) . (Y. F (Y (x) −) = lim F (y ) ≤ x. M) be a measurable space and g : Z → X be a function. Then. Fα )–measurable for all α ∈ A. F ) and g : (Y.1 Measurable Functions 45 Proof. σ (π1 . Let σ (fα : α ∈ A) denote the smallest σ – algebra on X such that each fα is measurable. Proposition 6. From the identity in Eq. . Fα ) – measurable for all α. M). Also observe. Fig. M) → (Y. If f : (X.6.14 (σ – Algebras Generated by Functions).2). 6. . Law (Y ) = µF as desired. 1) . G ) are measurable functions then g ◦ f : (X. 1) → R is a non-decreasing function. Page: 45 job: prob . 1) and Y (x) ≤ y0 . Let πi : Y N → Y be the projection maps. Proposition 6. F (y0 )] ∩ (0.e. if y < Y (x) . F ) and (Z. Proof. . Hence −1 g −1 (G ) = g −1 σ ∪α∈A fα (Fα ) −1 = σ (g −1 ∪α∈A fα (Fα ) ⊂ M (−∞. 1)) = F (y0 ) . x ∈ (0. which shows that g is (M. G ) – measurable. F (Y (x)) = F (Y (x) +) = lim F (y ) ≥ x y ↓Y (x) and so we have shown F (Y (x) −) ≤ x ≤ F (Y (x)) . Assuming the notation in Deﬁnition 6. Conversely if x ∈ (0. y0 ) = m ((0. σ (fα : α ∈ A)) – measurable. as the reader should check.

In particular. Since each πi is BRn /BR – measurable. it follows that σ (E ) ⊂ BRn . Hence we may conclude that BRn = σ (open sets) ⊂ σ (E0 ) ⊂ σ (E ) ⊂ BRn . . Therefore BRn may be described as the σ algebra generated by {A1 × · · · × An : Ai ∈ BR } . 2. a function f : X → C is (M. F is also measurable.16. i−1 (V \ {0}) is an open set and hence in BC . BC2 ) – measurable since π1 ◦ F = f and π2 ◦ F = g are (M.26 and BY = σ (τY ). i. BC ) – measurable. . . . . if (a. 1 z Therefore. . may be written as a (necessarily) countable union of elements from E0 . Therefore A± ◦ F = f ± g and M ◦ F = f · g. (X.46 6 Random Variables Proof.e. x macro: svmonob. Since E ⊂ BRn . πn ) . . BC ) – measurable iﬀ Re f and Im f are (M. Since every open set. E ⊂ σ (π1 . . For i = 1. g : X → C be (M. Proof. . z ) = w ± z and M (w. V ⊂ Rn . are also measurable. (6. . Theorem 6.V. Figure 6. BC ) – measurable function. Also F is (M. 2. we write a < b iﬀ ai < bi for i = 1. Then F (x) := is measurable. A± : C × C → C and M : C × C −→ C by F (x) = (f (x). Moreover. b) = (a1 . we have V ∈ σ (E0 ) ⊂ σ (E ) . .26. If (X. BC ) – measurable. Therefore i−1 (τC ) ⊂ BC and hence i−1 (BC ) = i−1 (σ (τC )) = σ (i−1 (τC )) ⊂ BC which shows that i is Borel measurable.20 and Proposition 6. πn ) and in particular. Corollary 6. . . . Let (X. b ∈ Qn n a < b} .19. b) : a. Proof.d simple R. then (a. (6. σ (E0 ) and hence σ (E ) contains all open subsets of Rn . b) is as in Eq. . . a]) is an inﬁnite half interval for all a and therefore i is measurable. . i−1 (V ∩ {0}) ∈ BC since i−1 (V ∩ {0}) is either the empty set or the one point set {0} .22.’s). b1 ) × · · · × (an . From the plot of i. . Proof. Let E0 := {(a. then f = (f1 . .25. A± (w.3). 2. If Ai ∈ BR for i = 1. z ) = wz.25. . let πi : Rn → R be deﬁned by πi (x) = xi . BR ) – measurable. simple random variables – see Example 5. n and let (a.22 below. We also have that BRn = σ (π1 . . Deﬁne i : C → C by i(z ) = For any open set V ⊂ C we have i−1 (V ) = i−1 (V \ {0}) ∪ i−1 (V ∩ {0}) Because i is continuous except at z = 0. BR ) – measurable for each i. . . it follows that σ (π1 . f2 .23 below. if z = 0 0 if z = 0. Example 6. . n. the reader may easily verify that 1 i−1 ((−∞.24 (Independent variables on product spaces).i. πn ) . b ∈ R . Corollary 6.21. Using Lemma 3. 2. .d.i. M) is a measurable space. bn ) . Remark 6.26. let us give another proof of the existence of i. Since F = i ◦ f is the composition of measurable functions. . Assertion 3. . then A1 × · · · × An = −1 ∩n i=1 πi (Ai ) ∈ σ (π1 . then BRn = σ (E ) . Then f ± g and f · g are also (M. . . As an example of this material. deﬁne i as above but now take z to real. Deﬁne F : X → C × C. . . .28 above. For the real case of Lemma 6. πn ) = BRn . . where. This is an application of Lemma 6. . A1 ×· · ·× An ∈ BRn whenever Ai ∈ BR for i = 1.3) Corollary 6. . . being the composition of measurable functions.cls date/time: 23-Feb-2007/15:20 1 f (x) (BY ) = f −1 (σ (τY )) = σ (f −1 (τY )) ⊂ σ (τX ) = BX . Page: 46 job: prob . . Assertion 2. f −1 Proof. Let α ∈ C. Assertion 1. . bi )) ∈ σ (π1 .20. . Lemma 6. . b) = −1 ∩n i=1 πi if α if f (x) = 0 f (x) = 0 ((ai . BC ) – measurable functions. g (x)).23 (Existence of i. This Theorem has been moved to Theorem 7. M) be a measurable space and f : X → C be a (M. Then each πi is continuous and therefore BRn /BR – measurable. . fn ) : X → Rn is (M. Then A± and M are continuous and hence (BC2 . for a. Lemma 6. . BC ) – measurable. πn ) and BRn = σ (E ) ⊂ σ (π1 . n. Let E denote the collection of open rectangle in Rn . . . BRn ) – measurable iﬀ fi : X → R is (M. This Corollary has been moved to Corollary 7. πn ) ⊂ BRn . M) be a measurable space and f. Moreover. n.

Let (X. Because A∆B ⊂ {±∞} and {∞} . (6.30 (Closure under sups. M) be a measurable space. f. inf j fj . Then f · g and f + g are measurable functions on X if both f and g are measurable. ∞] ∈ BR ∞ ¯ {∞} = ∩n=1 [n. then {x : g+ (x) ≤ a} = {x : fj (x) ≤ a ∀ j } = ∩j {x : fj (x) ≤ a} ∈ M so that g+ is measurable. Let us ﬁrst observe that ∞ c {−∞} = ∩∞ ¯. are all M/BR – measurable functions. ¯ = R∪ {±∞} . Deﬁne g+ (x) := sup j fj (x). Page: 47 job: prob lim inf fj = sup inf {fj : j ≥ n} n we are done by what we have already proved. M) is a topological space and measurable is replaced by continuous in the statement. n=1 [−∞. The proofs of the next two corollaries are left to the reader. Exercise 6.29. f −1 ((−∞. Then the following are equivalent 1. {−∞} ∈ BR ¯ and BR ⊂ BR ¯. This proves Eq.27 to handle item 4. Suppose that (X. Let BR and BR ¯ : A ∩ R ∈BR }. f −1 ({−∞}) ∈ M. ∞] : a ∈ R ¯ [a. f −1 ({∞}) ∈ M and f 0 : X → R deﬁned by f 0 (x) := 1R (f (x)) = is measurable. BR ¯ = {A ⊂ R In particular {∞} . then ¯ ). f −1 ((a. 3.6.28 noting that the equivalence of items 1.5 and 6. M) be a measurable space and f : X → R tion.1 Measurable Functions 47 ¯ be a funcCorollary 6. 0 · ∞ = 0 and deﬁne f · g : X → R and (f + g ) (x) = 0 if f (x) = ∞ and g (x) = −∞ or f (x) = −∞ and g (x) = ∞. Letting i : R → R −1 i−1 (BR ¯) = σ i =σ Thus we have shown ¯ [a. infs and limits).) Proof. Since lim sup fj = inf sup {fj : j ≥ n} and j →∞ j →∞ n BR = i−1 (BR ¯ ) = {A ∩ R : A ∈ BR ¯ }. lim sup fj and lim inf fj j →∞ j →∞ (6.5). f is (M. is a direct analogue of Corollary 6.cls date/time: 23-Feb-2007/15:20 . Then supj fj . Prove Corollary 6. Prove Corollary 6. a]) ∈ M for all a ∈ R.measurable.6. f (x) if f (x) ∈ R 0 if f (x) ∈ {±∞} ¯ be functions Corollary 6. macro: svmonob. ∞] ∩ R : a ∈ R =σ ¯ i−1 ([a. Similarly if g− (x) = inf j fj (x) then {x : g− (x) ≥ a} = ∩j {x : fj (x) ≥ a} ∈ M. Proposition 6. −n) = ∩n=1 [−n. Let (X. g : X → R ¯ and (f + g ) : X → R ¯ using the conventions. BR ¯ ) .27 (The Structure of BR ¯ be as above. (Note that this result is in generally false when (X. A ∈ BR ⇒ A ∩ R ∈BR and ¯ = ¯ is such that A ∩ R ∈BR there exists B ∈ BR 2. This implies: 1.5) ¯ be the inclusion map. ∞] : a ∈ R}) . ∞] ∈ BR ¯ and R = R\ {±∞} ∈ BR ¯. Exercise 6. – 3. Proposition 6. Use Proposition 6.29. 4.28.6. M) is a measurable space and fj : (X. ∞]) ∈ M for all a ∈ R. Proof. ∞]) : a ∈ R = σ ({[a.4) BR ¯ := σ ({[a. 2. ∞) : a ∈ R}) = BR .7. see Exercises 6.5. if A ⊂ R ¯ such that A ∩ R = B ∩ R. M) → R for j ∈ N is a sequence of M/BR – measurable functions. When talking We will often deal with functions f : X → R ¯ deﬁned about measurability in this context we will refer to the σ – algebra on R by (6. {−∞} ∈ BR ¯ we may conclude that A ∈ BR ¯ as well.

(6.4. It is clear by construction that ϕn (x) ≤ f (x) for all x and that 0 ≤ f (x) − ϕn (x) ≤ 2−n if x ∈ X2n . ﬁrst assume that f : X → R is a measurable function and choose ϕ± n to be simple functions such + − that ϕ± n ↑ f± as n → ∞ and deﬁne ϕn = ϕn − ϕn . 0} and Deﬁnition 6. Any such simple functions can be written as n ϕ= i=1 λi 1Ai with Ai ∈ M and λi ∈ F. Similarly (2n . 0) = − min (f (x). ∞] ⊂ R measurable and ϕ(X ) contains only ﬁnitely many elements. then Proposition 6. then there exists simple functions ϕn such that limn→∞ ϕn (x) = f (x) for all x and |ϕn | ↑ |f | as n → ∞. 2n+1 ] then ϕn (x) = 2n+1 < 2n+1 = ϕn+1 (x). Then + − − |ϕn | = ϕ+ n + ϕn ≤ ϕn+1 + ϕn+1 = |ϕn+1 | − + − and clearly |ϕn | = ϕ+ n + ϕn ↑ f+ + f− = |f | and ϕn = ϕn − ϕn → f+ − f− = f as n → ∞. ϕn (x) ↑ f (x) for all x ∈ X and ϕn ↑ f uniformly on the sets XM := {x ∈ X : f (x) ≤ M } with M < ∞. Then f is measurable iﬀ f± are measurable. see Figure 6. Therefore ϕn ≤ ϕn+1 for all n. Let ϕn = un + ivn . Conversely if f± are measurable then so is f = f+ − f− . ∞]. 2n+1 ] +1 2k 2k+1 2k+1 2k+2 −1 f ( 2n+1 .31. Let f : X → [0. n+1 ]. n+1 ] ∪ ( n+1 . λ2 . 2n ]) k 1 k k+1 (x) + n1{f >n2n } (x) 2n { 2n <f ≤ 2n } = k=0 then ϕn ≤ f for all n.6) Fig. Proof. Page: 48 macro: svmonob. ϕn (x) = 2n < 2n+1 = ϕn+1 (x) and for x ∈ f −1 ((2n . n2n −1 ϕn (x) := k=0 n2 −1 n k 1 −1 k k+1 (x) + n1f −1 ((n2n . ∞] = (2n . . if f : X → C is a measurable function. Since ( k k+1 2k 2k + 1 2k + 1 2k + 2 . take λ1 . If f is measurable. Note that this argument shows that any simple function may be written intrinsically as ϕ= y 1ϕ − 1 ( { y } ) . Now suppose that f : X → C is measurable. Moreover. Proof. Suppose (X.32. . |vn | ↑ |Im f | . ∞]. then 2 |ϕn | = u2 n + vn ↑ |Re f | + |Im f | = |f | 2 2 2 2 The next theorem shows that simple functions are “pointwise dense” in the space of measurable functions. We may now choose simple function un and vn such that |un | ↑ |Re f | . and so for x ∈ f −1 ((2n+1 . Hence we have shown that ϕn (x) ↑ f (x) for all x ∈ X and ϕn ↑ f uniformly on bounded sets. C or [0. Given a function f : X → R f− (x) := max (−f (x). n 2 2 2 2 2 2 job: prob and ϕn = un + ivn → Re f + i Im f = f as n → ∞. Indeed.7) y ∈F 2k+1 2k 2k then ϕn (x) = ϕn+1 (x) = 2n and if x ∈ if x ∈ f −1 ( 2n +1 . 0) . Deﬁnition 6. 2n+1 ]). . 2n+1 ] ∪ (2n+1 . Constructing simple functions approximating a function. 6. f : X → [0. . n ] = ( n+1 .30 implies f± are measurable. Let (X.4. Notice that f = f+ − f− . Theorem 6.33. ¯ is a Corollary 6. M) is a measurable space and f : X → R function.∞]) (x) 2n f (( 2n . ∞] be measurable and deﬁne. un → Re f and vn → Im f as n → ∞.48 6 Random Variables ¯ let f+ (x) := max {f (x). ϕn+1 (x) ≥ 2n = ϕn (x). ∞]). λn to be an enumeration of the range of ϕ and Ai = ϕ−1 ({λi }). (6.cls date/time: 23-Feb-2007/15:20 . A function ϕ : X → F ¯ ) is a simple function if ϕ is M – BF (F denotes either R. M) be a measurable space.34 (Approximation Theorem). For the second assertion.

Then the function F : X → Y deﬁned by [F (x)]α := fα (x) for each α ∈ A is (M. from X → R. H : X → R ¯ (F . Let X and A be sets. Fα ) and a function fα : X → Yα . where h := lim sup hn – a measurable function from Y to R n→∞ The following is an immediate corollary of Proposition 6.16 and Lemma 6. H. choose simple functions Hn converging to H. First suppose that H = 1A where A ∈ σ (F ) = F −1 (F ). Corollary 6.35. BR ¯ ) – measurable function h : Y → R such that H = h ◦ F.35. BR ¯) – ¯ such that H = h ◦ F. Then it follows that be simple functions on R H = lim Hn = lim sup Hn = lim sup hn ◦ F = h ◦ F n→∞ n→∞ n→∞ ¯. F ) – measurable with h := ai 1Bi – a simple function on R ¯ function. F ) – measurable ¯ is (M. Then to every (σ (F ). measurable function h from Y to R . More generally if H = ai 1Ai is a simple function. Proof. Let B ∈ F such that A = F −1 (B ) then 1A = 1F −1 (B ) = 1B ◦ F and hence the Lemma is valid in this case with h = 1B . F := ⊗α∈A Fα be the product σ – algebra on Y and M := σ (fα : α ∈ A) be the smallest σ – algebra on X such that each fα is measurable. Suppose that (Y. BR ¯ ) – measurable function. F ) is a measurable space and F : X → Y is a ¯ .36. For general (σ (F ). then there exists Bi ∈ F such that 1Ai = 1Bi ◦ F and hence H = h ◦ F ¯ .2 Factoring Random Variables Lemma 6. Let Y := α∈A Yα .6. Let hn ¯ such that Hn = hn ◦ F. there is a map. and suppose for α ∈ A we are give a measurable space (Yα . BR and a function H : X → R ¯ ) – measurable iﬀ there exists a (F .

.

D ∈ L. then P = Q on σ (P ) . If {An }n=1 ⊂ L are disjoint. then P ⊂ LC ⊂ L and hence again LC = L. Finally. L is a σ algebra. above. 2.3 (Alternate Axioms for a λ – System*). Taking complements of this result shows B \ A = Ac ∩ B ∈ L as well. then Bn := An \ An−1 ∈ L for all n. If An ∈ L with An ↑ A. since Ac = X \ A.e. b. For the rest of the proof. then σ (P ) ⊂ L. Clearly then postulates 1. As an immediate corollary. Suppose that P ⊂ 2Ω is a π – system.class containing P and hence LC = L. A ∈ LC . P and Q could be two measures such that P (Ω ) = Q (Ω ) < ∞. it is closed under intersections and therefore C is an algebra. B ∈ LC .4 (Dynkin’s π – λ Theorem). Since C ∈ L is arbitrary. then An ∩ C ∈ L and An ∩ C ↑ A ∩ C ∈ L. Proposition 7. Since C is also closed under increasing unions. So by Remark 7. if An ∈ LC with An ↑ A. C is a monotone class if it is closed under countable increasing unions and countable decreasing intersections. C is a λ–class if C satisﬁes the following properties: a) X ∈ C b) If A. we have shown. is a λ – system. B ∈ L such that A ∩ B = ∅.. Theorem 7. If C is a collection of subsets of Ω which is both a λ – class and a π – system then C is a σ – algebra. we have shown C ∩ D ∈ L for all C. then B \ A ∈ C . Hence it follows by postulate 3 that ∞ ∪∞ n=1 An = n=1 Bn ∈ L. then B c ∈ L and A ∩ B c = ∅ allows us to conclude that A ∪ B c ∈ L. A ∈ L implies Ac ∈ L. If P and Q are two probability1 measures on σ (P ) such that P = Q on P . – 3. If C is also a π – system. We may now conclude that if C ∈ L. Taking compliments of this result shows A ∪ B ∈ L as well.1 π – λ and Monotone Class Theorems Deﬁnition 7.5. i. Let C ⊂ 2X be a collection of sets. If L is a λ class which contains a contains a π – class. Therefore LC is closed under proper diﬀerences. then A ∈ C . the n=1 An ∈ L. Now suppose that L satisﬁes postulates 1. c. then A ⊂ B c and Ac ∩ B c = B c \ A ∈ L.) ∞ ∞ 3. if A ⊂ B with A. L is closed under ﬁnite disjoint unions. for any element C ∈ L. we may assume with out loss of generality that L is the smallest λ – class containing P – if not just replace L by the intersection of all λ – classes containing P .) Proof. C ∩ D ∈ L for all C ∈ P and D ∈ L.2. Since C ∈ P was arbitrary. we see that any λ .system is closed under complementation.2. To prove this claim. the collection of sets. Proof. that ∞ n=1 An ∈ L. 1 More generally.) c) If An ∈ C and An ↑ A. i. Since Bm ↑ n=1 An it follows from postulate c. LC := {D ∈ L : C ∩ D ∈ L} . B ∩ C ∈ L with A ∩ C ⊂ B \ C and (B \ A) ∩ C = [B ∩ C ] \ A = [B ∩ C ] \ [A ∩ C ] ∈ L. Suppose that L satisﬁes a. Therefore if A. (Closed under complementation. Suppose that L ⊂ 2Ω is a collection of subsets Ω.e. P . observe that: a. (Closed under countable increasing unions. C is a σ – algebra. Since σ (P ) is the smallest σ – algebra containing P it follows that σ (P ) ⊂ L. holds.1. we have the following uniqueness result. B ∈ C and A ⊂ B . Indeed. B ∈ L with A ⊂ B. then A ∩ C. m ∞ Bm := n=1 An ∈ L. So by induction. .7 Independence 7. Then L is a λ – class iﬀ λ satisﬁes the following postulates: 1. (Closed under proper diﬀerences. X ∈ L 2. postulate b. and 2. We start by proving the following assertion. Notice that ∅ ∈ L and by postulate 3.e. – c. Hence we have veriﬁed LC is still a λ – system. Suppose that A. X ∈ LC . Lemma 7.) Remark 7. (Closed under disjoint unions. C is a π – class if it is closed under ﬁnite intersections and 3. Then for C ∈ P we know that LC ⊂ L is a λ . L is a π – system. above. 1. hold. where by convention A0 = ∅. i.

Sn := X1 + · · · + Xn and Tn := Y1 + · · · + Yn . . . . . Suppose that f : Rn → Rk is a BRn /BRk – measurable function. . 1. then P (A) = limn→∞ P (An ) = limn→∞ Q (An ) = Q (A) which shows A ∈ L. B ) such that µ = ν on a π – system. F . (Ω. Solution to Exercise (7.1. Ω ∈ P ⊂ L. Prove the following assertions.6. Show lim sup Xn = lim sup Yn and similarly that lim inf n→∞ Xn = n→∞ n→∞ ∞ d ∞ d ∞ d ∞ ∞ ∞ ∞ ∞ d ∞ d ∞ ∞ ∞ d n n for all A ∈ B . then f (X1 . Xn ) ∈ B ) := P ({ω ∈ Ω : (X1 (ω ) . . .8. . Xn (ω )) ∈ B }) for all B ∈ BRn .1). . it follows that µ (A) = lim µ (A ∩ Ωn ) = lim µn (A) = lim νn (A) = lim ν (A ∩ Ωn ) = ν (A) n→∞ n→∞ n→∞ n→∞ Corollary 7. The measure. . P ) and (X. Xn ≤ xn ) for all Ai ∈ BR for all x = (x1 . . and if An ∈ L with An ↑ A. . Xn ) and (Y1 . with k = n and a judiciously chosen function. We write (X1 . . . . . . Let Ω := {a. . Then µn and νn are ﬁnite measure such µn (Ω ) = νn (Ω ) and µn = νn on P . A = {a. Yn ) if (X1 . The joint distribution. µ (B ) := P ((X1 . . . . . Therefore σ (P ) ⊂ L = σ (P ) and the proof is complete. if P ((X1 . Example 7. Q) respectively. . Indeed. Use your result in item 1. Let µn (A) := µ (A ∩ Ωn ) and νn (A) = ν (A ∩ Ωn ) for all A ∈ B. .5. Corollary 7. i. P . . . . Yn ) have the same distribution. . x1 ] × · · · × (−∞. Exercise 7. . . b. Let L := {A ∈ σ (P ) : P (A) = Q (A)} . . Indeed. Yn ) for all n ∈ N.10. . c} are in L but A ∩ B ∈ / L. Page: 52 job: prob n {Xi }i=1 lim inf n→∞ Yn .7. i) µ (Ωn ) = ν (Ωn ) < ∞ for all n and ii) Ωn ↑ Ω as n ↑ ∞. P. . Xn ) ∈ B ) = Q ((Y1 . Hint: Consider the measures. . .9. Suppose that is a sequence of random variables on a −1 probability space. x]) .cls date/time: 23-Feb-2007/15:20 .e. µ is uniquely determined from the knowledge of P ((X1 . . B . Xn ) on BRn is called the joint distribution of (X1 . . To be more explicit. Deﬁnition 7. xn ] ∈ BRn : xi ∈ R} for the second case. . . . More generally. . So by the continuity properties of µ and ν. Deﬁnition 7. Hint: with the aid of the set identity. µ ({x}) = 1 4 for all x ∈ Ω and ν ({a}) = ν ({d}) = 8 and ν ({b}) = ν ({c}) = 3/8. . P ) and (X. Let {Xi }i=1 and {Yi }i=1 be two sequences of random variables such that {Xi }i=1 = {Yi }i=1 . µn = νn on B . . . A probability measure. . . L := A ∈ 2Ω : P (A) = Q (A) is λ – system which is not an algebra. Xn ) = f (Y1 . d d 3. . . . Show µ = ν on B . Q) we write {Xi }i=1 = {Yi }i=1 iﬀ (X1 . . .5 with P being the π – systems deﬁned by P := {A1 × · · · × An ∈ BRn : Ai ∈ BR } for the ﬁrst case and P := {(−∞. Xn ) = (Y1 . F . . . . Xn ) = (Y1 . c. . In this example. . Xn ) ∈ A1 × · · · × An ) for all Ai ∈ BR or from the knowledge of P (X1 ≤ x1 . One easily shows L is a λ – class which contains P by assumption. Exercise 7. µ = P ◦ (X1 . Hint: apply item 1. then P (B \ A) = P (B ) − P (A) = Q (B ) − Q (A) = Q (B \ A) so that B \ A ∈ L. B . Proof. . . if {Xi }i=1 and {Yi }i=1 are two sequences of random variables on two probability spaces. P ) . f : Rn → Rn . . Suppose that µ and ν are two measure on a measure space. Suppose that {Xi }i=1 and {Yi }i=1 are two ﬁnite sequences of random variables on two probability spaces.2. macro: svmonob. Yn ) . Let {Sn }n=1 and {Tn }n=1 be deﬁned by. b} and B = {a. . 2. B .52 7 Independence Proof. (Ω. . Therefore by Proposition 7. Further assume B = σ (P ) and there exists Ωn ∈ P such that. Xn ) . . d} and let µ and ν be the probability measure 1 on 2Ω determined by. on (R. (Ω. Yn ) ∈ B ) for all B ∈ BRn . . if A. . F (x) := P ((−∞. BR ) is uniquely determined by its distribution function. µn (A) := µ (A ∩ Ωn ) and νn (A) = ν (A ∩ Ωn ) . xn ) ∈ Rn . to show {Sn }n=1 = {Tn }n=1 . (Ω. . Apply Proposition 7. . B ∈ L with A ⊂ B. . .

show P lim sup Xn ≥ x n→∞ Exercise 7.1 The Monotone Class Theorem This subsection may be safely skipped! Lemma 7. .7. Observe that An ∈ B and An ↑ A. then Bn and so is the symmetric diﬀerence of A and B. it follows that C ∩ B. if Bn ∈ C (C ) and Bn ↑ B. Show. that for every B ∈ B there exists A ∈ A such that that P (A B ) < ε.1. B ∈ C (C ). then A ∩ B. Then C = σ (A). C ∩ B c . using the π – λ theorem. 5. 4. Here A B := (A \ B ) ∪ (B \ A) = lim lim P (∪m k=n {Xk ≥ x}) . So C is closed under complements (since X ∈ A ⊂ C ) and ﬁnite intersections and increasing unions from which it easily follows that C is a σ – algebra. n→∞ m→∞ To use this identity you will also need to ﬁnd B ∈ BRm such that ∪m k=n {Xk ≥ x} = {(X1 . B ∩ C c ∈ C}. Again since C is the smallest monotone class containing A and C (B ) is a monotone class we conclude that C (B ) = C for all B ∈ C . and similarly. Let B ∈ C and notice that A ∈ C (B ) happens iﬀ B ∈ C (A). That is to say. C C C C ∩ Bn ↑ C ∩ B c C ∩ Bn ↓ C ∩ B c and Bn ∩ C c ↑ B ∩ C c . This observation and the fact that C (A) = C for all A ∈ A implies A ⊂ C (B ) ⊂ C for all B ∈ C . (A2 \ A1 ) \ (B2 \ B1 ) ⊂ (A2 \ B2 ) ∪ (B1 \ A1 ) so that (A2 \ A1 ) (B2 \ B1 ) ⊂ (B2 \ A2 ) ∪ (A1 \ B1 ) ∪ (A2 \ B2 ) ∪ (B1 \ A1 ) = (A1 B1 ) ∪ (A2 B2 ) . For C ∈ C let C (C ) = {B ∈ C : C ∩ B. if A. Let L be the collection of sets B for which the assertion of the theorem holds. Ac ∩ B ∈ C .11 (Monotone Class Theorem*). Indeed. then P (B An ) = P (B \ An )+P (An \ B ) → P (B \ A)+P (A \ B ) = P (A B) . Since C is the smallest monotone class containing A and C (A) is a monotone class containing A. c ↓ Bc then C (C ) is a monotone class. Show L is a λ – system which contains A.e. Proof. If A ∈ A ⊂ C . . 7. B ∈ C then A ∈ C = C (B ) and hence A ∩ B. i.cls date/time: 23-Feb-2007/15:20 Page: 53 job: prob . This shows that C (C ) is closed under increasing limits and a similar argument shows that C (C ) is closed under decreasing limits. Thus we have shown that C (C ) is a monotone class for all C ∈ C . we conclude that C (A) = C for any A ∈ A. . B := σ (A) . C ∩ B c .3. B ∩ C c ∈ C . Suppose that A ⊂ 2Ω is an algebra. B ∩ Ac ∈ A ⊂ C for all B ∈ A and hence it follows that A ⊂ C (A) ⊂ C . We also have c (B2 \ B1 ) \ (A2 \ A1 ) = B2 ∩ B1 ∩ (A2 \ A1 ) c c = B2 ∩ B1 ∩ (A2 ∩ Ac 1) c = B2 ∩ B1 ∩ (Ac 2 ∪ A1 ) c c = [B2 ∩ B1 ∩ Ac 2 ] ∪ [B2 ∩ B1 ∩ A1 ] ⊂ (B2 \ A2 ) ∪ (A1 \ B1 ) c Bi and Bi B ⊂ ∪i (Ai Bi ) . Suppose A ⊂ 2X is an algebra and C is the smallest monotone class containing A. and P is a probability measure on B . A ∩ B c . A ∩ B c . Xm ) ∈ B } . Since C is a monotone class. . Hints: 1. Also observe that if B = ∪Bi and A = ∪i Ai . then B \ A ⊂ ∪i (Bi \ Ai ) ⊂ ∪i Ai A \ B ⊂ ∪i (Ai \ Bi ) ⊂ ∪i Ai so that A 3.1 π – λ and Monotone Class Theorems 53 lim sup Xn ≥ x n→∞ = {Xn ≥ x i. macro: svmonob. 2. It may be useful to observe that 1A B = |1A − 1B | so that P (A B ) = E |1A − 1B | .} .o.

13. By applying the result we have just proved to the sequence. σ (Cj −1 ) are independent for each j = 1. . Cn . n. . We further say a ﬁnite sequence of collection of sets. .2 Basic Properties of Independence For this section we will suppose that (Ω. Q (A) = P (A ∩ A2 ∩ · · · ∩ An ) P (A ∩ A2 ∩ · · · ∩ An ) = P (A2 ) . .1) is equivalent to P (A) = Q (A) on σ (C1 ) . Then equation Eq. Then {σ (Ci )}i=1 are also independent. . . Cn . . σ (C1 ) are independent. . . Then {Ci }i=1 are independent. . Hence it follows for N large enough that P B AN < ε. if we let B := ∪n Bn and AN := ∪N n=1 An . But this is true by Proposition 7. Since L contains the π – system. Fix. Observe that if {Ci }i=1 . Similarly we show inductively that σ (Cj ) . A collection of subsets of B .16. Cn . we may always assume with out loss of generality that X ∈ Ci . Aj ∈ Cj for j = 2. . . . Example 7. . . . then {σ (Ct )}t∈T are independent as well.1) Since it is clear that this identity holds if P (Aj ) = 0 for some j = 2. . P (An ) P (A2 ∩ · · · ∩ An ) = P (A|A2 ∩ · · · ∩ An ) for all A ∈ σ (C1 ) . n. C1 .14.15. An ) ∈ C1 × · · · × Cn . C2 . . there exists Ai ∈ A such that P (Bi Ai ) = E |1Ai − 1Bi | < ε/2 and therefore. {Ci }i=1 . are independent iﬀ n n n P ∩n j =1 Aj = j =1 P (Aj ) for all (A1 . Also if Bn ↑ B with Bn ∈ L. Deﬁnition 7. n Moreover. . n Theorem 7. 2. An ) ∈ C2 × · · · × Cn were arbitrary we may now conclude that σ (C1 ) . .3). ∞ P ([∪n Bn ] [∪n An ]) ≤ n=1 P (Bn An ) < ε. σ (C1 ) .12. . then ∩Bi = A1 × A2 × · · · × An and we have for all Ai ∈ Ci and J ⊂ {1. More explicitly. . B = 2Ω . if we assume that X ∈ Ci for each i. Indeed. then P B AN = P B \ AN +P AN \ B → P (B \ A)+P (A \ B ) = P (B A) where A := ∪n An . Page: 54 job: prob n n P (∩Bi ) = ω ∈A1 ×A2 ×···×An i=1 qi (ωi ) = i=1 λ∈Ai qi (λ) macro: svmonob. {Ct }t∈T is said to be independent iﬀ {Ct }t∈Λ are independent for all ﬁnite subsets. 2. . Ω ∈ L since Ω ∈ A ⊂ L. C2 . σ (C1 ) shows that σ (C2 ) . . . are independent if P (∩j ∈J Aj ) = P (Aj ) j ∈J n whenever Λ is a ﬁnite subset of T and At ∈ Ct for all t ∈ Λ. Cn are independent. . As mentioned above. . If B1 ⊂ B2 with Bi ∈ L and ε > 0. Deﬁnition 7. C3 . . B . . . 1] are functions such that λ∈Λ qj (λ) = 1. . Suppose that Ω = Λn where Λ is a ﬁnite set. Cj +1 . Suppose that {Ci }i=1 is a ﬁnite sequence of independent π – n classes. . P ({ω }) = n j =1 qj (ωj ) where qj : Λ → [0. we may assume that P (Aj ) > 0 for j ≥ 2. are independent classes then so are {Ci ∪ {X }}i=1 . Clearly.5 using the fact that Q = P on the π – system. P ((B2 \ B1 ) (A2 \ A1 )) ≤ P ((A1 ≤ P ((A1 B1 ) ∪ (A2 B2 )) B1 )) + P ((A2 B2 )) < ε. If {Ct }t∈T is a collection of independent classes such that each Ct is a π – system. . n} . In this case we may deﬁne. . We say that A is independent of B is P (A|B ) = P (A) or equivalently that P (A ∩ B ) = P (A) P (B ) . . there exists An ∈ A such that P (Bn and therefore. . . . Corollary 7. n Let Ci := Λi−1 × A × Λn−i : A ⊂ Λ . . . P (An ) for all A ∈ σ (C1 ) . then {Ci }i=1 . Since (A2 . if Bi := Λi−1 × Ai × Λn−i . .54 7 Independence Solution to Exercise (7. P ) is a probability space. .cls date/time: 23-Feb-2007/15:20 . . 3. The desired result occurs at j = n. (7. A it suﬃces by the π – λ theorem to show L is a λ – system. (7. An ) < ε2−n Proof. . . n. we are requiring P (∩t∈Λ At ) = P (At ) t∈Λ 7. Moreover. Λ ⊂ T. . . We will begin by showing that P (A ∩ A2 ∩ · · · ∩ An ) = P (A) P (A2 ) . .

A collections of random variables. Λ ⊂ T. Hence it suﬃces to verify Eq. then ∩k P ∩k j =1 {Xj ∈ Vj } = P j =1 xj ∈Vj ∩Rj P (Xt ∈ At ) for all ﬁnite subsets. then {Xj = xj } ∈ σ (Xj ) for all xj ∈ R and therefore Eq. EP i=1 fi (Xi ) = i=1 EP [fi (Xi )] for all fi : Λ → R. random variables iﬀ n n n ∞ ∞ = i=1 P (Xi ∈ Ai ) P (X1 ∈ A1 .i..21.xk )∈ (x1 . Xn ∈ An ) = j =1 P (Xi ∈ Ai ) = j =1 P (X1 ∈ Ai ) = j =1 µ (Ai ) (7.2 Basic Properties of Independence 55 while P (Bi ) = n qi (ωi ) = ω ∈Λi−1 ×Ai ×Λn−i i=1 λ∈Ai qi (λ) . and all At ∈ BR for t ∈ Λ. . The equivalence of 1..15 with Ct := {{Xt ≤ a} : a ∈ R} and making use the observations that Ct is a π – system for all t and that σ (Ct ) = σ (Xt ) .cls date/time: 23-Feb-2007/15:20 . follows almost immediately form the deﬁnition of independence and the fact that σ (Xt ) = {{Xt ∈ A} : A ∈ BR } . is an application of Corollary 7.2) holds. Proof. Taking Ai ⊂ Λ and fi := 1Ai in the above identity shows that n n P (X1 ∈ A1 . Let X := {Xt : t ∈ T } be a collection of random variables..7.d. Example 7. we know that n n ∩k j =1 {Xj = xj } Vj ∩Rj (x1 . ranges are independent iﬀ k k {Xj }j =1 with countable Deﬁnition 7. 3. That is we should have P (Xn ∈ A) = P (Xk ∈ A) for all k.xk )∈ Qk j =1 P Vj ∩Rj k ∩k j =1 {Xj = xj } = k (x1 .. . Finally. and 2. Then the following are equivalent: 1.17. Indeed.i. P (∩t∈Λ {Xt ∈ At }) = t∈Λ P ∩k j =1 {Xj = xj } = j =1 P (Xj = xj ) (7.16 and further assume that n Λ ⊂ R and let Xi : Ω → Λ be deﬁned by. Alternatively. (7. B . (Ω. Then {Xi }i=1 are independent random variables. 2. P (∩t∈Λ {Xt ≤ xt }) = t∈Λ P (Xt ≤ xt ) {Xj = xj } Qk j =1 =P = for all ﬁnite subsets. . σ (Xi ) = Ci with Ci as in Example 7. n. n ∈ N and A ∈ BR .. . (7. Now if {Xj }j =1 are independent.3) as desired.1. (7.. implies 2. implies 3. A sequence of random variables. P ). 3. are i. Corollary 7. Page: 55 job: prob macro: svmonob.2) for all xj ∈ R.2) are zero unless xj is in the range of Xj for all j. {Xt : t ∈ T } are independent iﬀ {σ (Xt ) : t ∈ T } are independent. As sequences of random variables.d. . Theorem 7.19. from Exercise 4.xk )∈ Qk P (Xj = xj ) k j =1 j =1 Vj ∩Rj = j =1 xj ∈Vj ∩Rj P (Xj = xj ) = j =1 P (Xj ∈ Vj ) .. Λ ⊂ T. Clearly 2.18. Xi (ω ) = ωi .2) for those xj ∈ Ran(Xj ) =: Rj k for all j. . Observe that {Xn }n=1 are i. The collection X. . Xn ∈ An ) = EP i=1 n 1Ai (Xi ) = i=1 EP [1Ai (Xi )] Deﬁnition 7...20. (= independent and identically distributed) if they are independent and (Xn )∗ P = (Xk )∗ P for all k.2) and Vj ∈ BR . on a probability space. Observe that both sides of Eq. Conversely if Eq. Continue the notation of Example 7.16.. and all xt ∈ R for t ∈ Λ. (7. {Xn }n=1 . Proof. holds.. ..

.. .n} ∞ ik 1Ji1 . and so by induction. b]) . . . 1. . .. ..56 7 Independence where µ = (X1 )∗ P. P := T∗ m. . qik for all i1 .. .d simple R. qik . Xk (x) . Further let B := σ (Y1 . Ti1 ((0. 2... . . ik ∈ {0. ..1. The identity in Eq. Fig. . ik ∈ {0. qi1 as desired. ..’s). Then there exists a unique probability measure.ik ) = qik qik−1 . 1]. . Proof. 1] by Xk := i1 . 1. .. .. . . Suppose that {qi }i=0 is a n sequence of positive numbers such that i=0 qi = 1. 2. . qi1 . Here we suppose that p0 = 2/3 and p1 = 1/3 and then we construct Jl and Jl.ik ) = qik qik−1 .i2 . let j i=0 qi n and for any Ti ((a. . . Yk = ii }) = m ({Y1 ◦ T = i1 .V. b])) = qi (b − a) = qi m ((a. Given i1 . n. .. (7. . and for i=0 qi = 1. b]. let Yi : Ω → R be deﬁned by Yi (ω ) = ωi for all ω ∈ Ω.k for l. . i2 . . . Yk ◦ T = ii }) = m ({X1 = i1 . . k ∈ {0. .i. Xk = ii }) = qi1 . .i2 . . Xk : (0. Let {Xi }i=1 be as in Theorem 7. Xk = ii } = Ji1 . . (7. 7.. . Ω = Λ i ∈ N. P : B → [0. . n} on ((0. We now deﬁne... . n Page: 56 job: prob macro: svmonob. . . 1] such that P ({Y1 = i1 .2. qi > 0 with = ΛN . Observe that m (Ti ((a. qik . . . . Xk = ii }) = qi1 . 2 . . i2 .. For i = 0. 1. . 1} . . . Proof. n}. . . see Figure 7. . m) such that m ({X1 = i1 .1.1. .. . Y2 .. . Yn . . Yk = ii }) = qi1 . .3) is to hold for all n ∈ N and all Ai ∈ BR .22 (Existence of i. 1. Yk = ii }) = m T −1 ({Y1 = i1 . . .i2 . . n} .. . m (Ji1 . . . ) . .. . . . B . . a + σi (b − a)].22 and deﬁne T : (0. 2 . n} and all k ∈ N. Then we have P ({Y1 = i1 . . .ik and therefore. . . .ik := Tik Tik−1 (. . .ik . Xk = ii }) = m (Ji1 . ) . .. let σ−1 = 0 and σj := interval.ik ∈{0. Observe that T is measurable since Yi ◦ T = Xi is measurable for all i. 1] → R are measurable....cls date/time: 23-Feb-2007/15:20 ... . . . (a. ..22 shows the functions. b]) := (a + σi−1 (b − a) . Then there exists a sequence ∞ {Xk }k=1 of simple random variables taking values in Λ = {0. The reader should convince herself/himself that {X1 = i1 . . . .. Theorem 7. .i2 .23 (Independent variables on product spaces). . X2 (x) . 1])) and deﬁne {Xk }k=1 on (0.4) Corollary 7. Repeated applications of Corollary 6. . 1] → Ω by T (x) = (X1 (x) .. we have m ({X1 = i1 . . .i2 . Suppose n ∞ Λ = {0.i2 . 1. let Ji1 .

24. Therefore {BTs = σ (Cs )}s∈S is an independent family of σ – algebras.a. Suppose that {µn }n=1 are a sequence of probability measures on (R. then P ( U ≤ x ) = (0 ∨ x ) ∧ 1 . For each i. assume that {Xn = 0 a. ∞ {Zn }n=1 .25. Proof.j – σ {Xi. 1] . we claim that if x = n −j j =1 εj 2 with εj ∈ {0. K ⊂⊂ Ts } .8. It is now easily checked that {Cs }s∈S is an independent family of π – systems.27. U< 3 4 = 3 . If we let U := ∞ −n 2 X . Use Corollary 7. {Xn }n=1 such that P (Xn = λ) = q (λ) for all λ ∈ Λ.cls date/time: 23-Feb-2007/15:20 .24. = {X1 = 0. it follows that P (U < 0) = 0. Indeed. Since x → P (U < x) is left continuous we may now conclude that P (U < x) = x for all x ∈ (0.} . B . X2 = 0} ∞ Since x ≤ U < x + 2−(n+1) = ∩n j =1 {Xj = εj } ∩ {Xn+1 = 0} we see that P x ≤ U < x + 2−(n+1) = 2−(n+1) and hence P U < x + 2−(n+1) = x + 2−(n+1) which completes the induction argument. X2 = 0} . From these identities.i. With this simpliﬁcation. 1} . let Ui := j =1 2−i Xi. there exists a sequence of i. are independent σ i=1 Page: 57 job: prob macro: svmonob.) . see the proof of Lemma 3. Suppose that {Bt : t ∈ T } is an independent family of σ – ﬁelds. P 4 U< 1 2 = 1 . j ∈ N} . Proposition 7. Hence we may. Λ ⊂ R and a function q : Λ → [0.2 Basic Properties of Independence 57 Theorem 7. According to Proposition 7. By Theorem 7. such that P (Zn = 1) = P (Zn = 0) = 1 2 . by shrinking Ω if necessary. 2. Lemma 7. Proof. 1) . We may now show the existence of independent random variables with arbitrary distributions. Proof.5) P (U < x) = x.20 to shows that random variables constructed in Example 5. Given a ﬁnite subset. B .28 or Theorem 7.25. we have already veriﬁed (7. And further assume that T = s∈S Ts and let BTs = ∨t∈Ts Bs = σ (∪t∈Ts Bs ) . then (7.j }j =1 ∞ ∞ ∞ ∞ ∞ = {X1 = 0} ∪ {X1 = 1. (Ω. there exists a probability space. Suppose we have veriﬁed (7. P ) and a sequence ∞ −1 {Yn }n=1 independent random variables with Law (Yn ) := P ◦ Yn = µn for all n. Ui is uniformly distributed on [0. P (Xn = 0) = P (Xn = 1) = 1 2 .26. Let Cs = {∩α∈K Bα : Bα ∈ Bα . Then there exists a probability space. U has the uniform distribution n n=1 on [0. Hence we may conclude that P (U ≤ x) = (0 ∨ x) ∧ 1. {Xi.d. 1] such that λ∈Λ q (λ) = 1. Let us recall that P (Xn = 0 a. (Ω. σ {Xi. BR ) .i. 4 Theorem 7.26. and P 2 More generally.7.) = P (Xn = 1 a. These random variables may be put into a two dimensional array.22 ﬁt the bill. Then {BTs }s∈S is an independent family of σ ﬁelds. 1] . random variables.5) when n = n 1.a.j }j =1 – measurable random variable.j : i.} = ∅ = {Xn = 1 a. Suppose that {Xn }n=1 is a sequence of i. random variables with distribution.a. P ) and an ∞ independent sequence of random variables. X2 = 0} and = {X1 = 1. Proof. The proof is by induction on n.e.5) up to some n ∈ N and let x = j =1 εj 2−j and consider P U < x + 2−(n+1) = P (U < x) + P x ≤ U < x + 2−(n+1) =x+P x≤U <x+2 −(n+1) . i. Moreover by the grouping Lemma 7. 1) and since x → x is continuous we may also deduce that P (U ≤ x) = x for all x ∈ (0.a. we have 1 2 1 U< 4 1 3 ≤U < 2 4 U< and hence that U< 3 4 = U< 1 2 ∪ 1 3 ≤U < 2 4 = {X1 = 0} . P U< 1 4 = 1 .d.

The {Rn }n=1 is an independent sequence. (X2 . Yi := Gi (Ui ) has µi as ∞ its distribution. X3 . . . . we now know. F. . ∗. X5 . n Proof. By Problem 6 on p. As another example. X3 . . from lowest to highest. X4 . Then according to Theorem 6.11 below. n! For example if (X1 . . In this case we have. . . As observed before the statement of the theorem. εn )} = {Xσ1 < Xσ2 < · · · < Xσn } Rn := j =1 1Xj >Xn = # {j ≤ n : Xj > Xn } .i. . let {al }l=−∞ be a sequence such that. X3 . 2. . let Fi (x) := µ ((−∞. . εn ) . This may be proved directly with some work or will be an easy consequence of Fubini’s theorem to be considered later. n ∞ and the events. . with common continuous distribution function. R5 = 1. P (Xi = Xj ) ≤ l∈Z P (Xi ∈ (al . . n. we Page: 58 job: prob macro: svmonob. ) = (9. Since R5 = 1. . if we choose al = N P (Xi = Xj ) ≤ lim sup sup F N →∞ l∈Z l+1 N −F l N = 0. . x) : x ∈ R} ⊂ ∪l∈Z [(al . and therefore. of (X1 . then Xn < Xn−1 < · · · < X1 .d.e. . εn ) such that Xσ1 < Xσ2 < · · · < Xσn .28 (Renyi Theorem). (X1 . X5 ) . X5 ) . i. X5 ) . P (Rn = k ) = 1 for k = 1. . Xn ) and (Xσ1 . (here εi ∈ N with εi ≤ i) of (R1 . . it is easily seen that l . X1 .i. ∗.2. X3 . . and assume that ∞ F (x) := P (Xn ≤ x) is continuous. 7. X5 ) . see Example 10. . . al+1 ]] ∞ ∞ must have X5 in the last slot. we know that X3 is again the second from the right of the remaining slots. . An = {Xn is a record} = {Rn = 1} are independent as n varies and 1 P (An ) = P (Rn = 1) = .cls date/time: 23-Feb-2007/15:20 . x) : x ∈ R}) = 0. This can be determined by the values of Ri for i = 1. ) .58 7 Independence ∞ – algebras and hence {Ui }i=1 is a sequence of i. Since F is continuous and F (∞+) = 1 and F (∞−) = 0. . . X4 .e. . Let {Xn }n=1 be i. . . X5 ) and ﬁnally R1 = 1 gives. . 23. ∗. Since R4 = 2. n Since P ({Xσ1 < Xσ2 < · · · < Xσn }) is independent of σ we may now conclude that 1 P ({Xσ1 < Xσ2 < · · · < Xσn }) = n! for all σ. 3. . . . if Ri = i for i = 1. X3 . Finally. . X1 . X5 ) is (X2 . . . Rn ) there is a permutation. ∗. X4 . . X2 . ∗. (∗. X4 . X4 must be in the second from the far most right. we know out of the remaining slots.. . . −8. . liml→∞ al = ∞ and liml→−∞ al = −∞. ∗. X5 ) . we have F is uniformly continuous on R. random variables with the uniform distribution.j }j =1 – measurable and therefore the {Yi }i=1 are independent random variables. 2.11. . 110 of Resnick. to each realization (ε1 . Therefore. it now follows that up to a set of measure zero.e. Xσn ) have the same distribution for any permutation σ. . X4 . Xj ∈ (al . i. Since F is continuous. (∗. . . . 7. R3 = 2. Rn ) = (ε1 . Observe that rank order. . Ω= {Xσ1 < Xσ2 < · · · < Xσn } σ and therefore.i.e. X3 . Xn ) . . Rn ) = (ε1 .d. . al < al+1 for all l ∈ Z. .1 An Example of Ranks Let {Xn }n=1 be i. for any i = j. For the ∞ direct proof. Similarly. ∗. . Let Rn denote the “rank” of Xn in the list (X1 . Theorem 7. . Then {(x. n. we have R1 = 1. X2 . P ({(R1 . .d. From this it follows that {(R1 . l∈Z P ({Xσ1 < Xσ2 < · · · < Xσn }) . al+1 ]. al+1 ]) = l∈Z [F (al+1 ) − F (al )] 2 and therefore 1 = P (Ω ) = σ ≤ sup [F (al+1 ) − F (al )] l∈Z l∈Z [F (al+1 ) − F (al )] = sup [F (al+1 ) − F (al )] . (∗. . al+1 ] × (al . x]) for all x ∈ R and let Gi (y ) = inf {x : Fi (x) ≥ y } . . Moreover each Yi is σ {Xi. R2 = 2. . and R4 = 2. i. . σ = σ (ε1 . i. . 5 as follows. . X4 . 2. R2 = 2 implies (X2 . X4 . Since R3 = 2. . that P (Xi = Xj ) = µF ⊗ µF ({(x. εn )}) = P (Xσ1 < Xσ2 < · · · < Xσn ) = Since 1 . .

o.) = 1. . 2. . Suppose that {An }n=1 are measurable sets.. ϕ (x0 ) + ϕ (x0 ) (x − x0 ) ≤ ϕ (x) from which it follows that ∞ 1An < ∞ a. .s.o.3 Borel-Cantelli Lemmas Lemma 7. P lim Xn = 0 = 1. Example 7. A convex function. . n=1 (7. .6) then P ({An i.7. Proof. εn )}) = n! j =1 j j =1 n n n→∞ Figure 7.e.2 below serves as motivation for the following elementary lemma on convex functions. ..o. First Proof. n=1 and 2 which is equivalent to P ({An i. Rn ) = (ε1 . Suppose that {Xn } are Bernoulli random variables with P (Xn = 1) = pn and P (Xn = 0) = 1 − pn . ϕ ∈ P C 2 ((a. b) . 7.. . 1. .εn−1 ) P ({(R1 . . Suppose that ϕ ∈ P C 2 ((a. . {a = a0 < a1 < a2 < · · · < an−1 < an = b} . .29 (First Borel Cantelli-Lemma). (Warning: this proof require integration theory which is developed below. along with a cord and a tangent line. .aj ]∩(a.31 (Convex Functions). macro: svmonob.b) is C 2 for all j = 1. such that ϕ|[aj −1 . Notice that the tangent line is always below ϕ and the cord lies above ϕ between the points of intersection of the cord with the graph of ϕ. i.7) Second Proof. .) = 0 and hence P (Xn = 0 a. = (ε1 . b) .6) is equivalent to ∞ E n=1 1An < ∞ Lemma 7. Then ϕ satisﬁes.a. . (7.}) = P (∩∞ n=1 ∪k≥n Ak ) = lim P (∪k≥n Ak ) ≤ lim n→∞ n→∞ P (Ak ) = 0.}) = 0.30. b) → R) means the ϕ is C 1 and there are a ﬁnite number of points.) Equation (7. . for all x0 . .. = = P ({(R1 . . In particular. . . x ∈ (a. ϕ.3 Borel-Cantelli Lemmas 59 P ({Rn = εn }) = (ε1 .εn−1 ) we have shown that 1 1 P ({Rj = εj }) .2. k≥n Fig. . Rn ) = (ε1 . We have P ({An i..cls date/time: 23-Feb-2007/15:20 . n. 7. b) → R)2 with ϕ (x) ≥ 0 for almost all x ∈ (a. If pn < ∞ Page: 59 job: prob P C 2 denotes the space of piecewise C 2 – functions. εn )}) 1 1 1 = (n − 1)! · = n! n! n then P (Xn = 1 i..}) = 0. If ∞ ∞ P (An ) < ∞.o.

cls date/time: 23-Feb-2007/15:20 .) Show that if {an }n=1 ⊂ [0.60 7 Independence 2. f (x) = ϕ (x) − ϕ (x0 ) = x0 ϕ (y ) dy. n=1 ∞ (The limit exists since.8) Exercise 7. Let f (t) := ϕ (u) + t (ϕ (v ) − ϕ (u)) − ϕ (u + t (v − u)) . f (x) ≥ 0 for all x ∈ (a.) Proof. and so by the fundamental theorem of calculus. i. f Fig. 1] . Taking ϕ (x) := e−x . then N n=1 ∞ (1 − an ) ↓ as N ↑ . let ∞ N ∞ (1 − an ) := lim n=1 N →∞ (1 − an ) . Example 7. 7. 1 − x ≤ e−x for all x ∈ R and taking ϕ (x) = e−2x we learn that 1 − x ≥ e−2x for 0 ≤ x ≤ 1/2. 1. f (t) ≥ 0.e. we learn (see Figure 7. In particular. A graph of 1 − x and e−x showing that 1 − x ≤ e−x for all x. there exists. See Appendix 11.4). By the mean value theorem. 1) such that f˙ (t0 ) = 0 and then by the fundamental theorem of calculus it follows that t f˙ (t) = t0 ¨ (τ ) dt.32. and − ln x to name a few examples. ∞ (1 − an ) = 0 iﬀ n=1 n=1 an = ∞. For {an }n=1 ⊂ [0. t0 ∈ (0. (This lemma applies to the functions. Hence it follows that f (x) ≥ 0 for x > x0 and f (x) ≤ 0 for x < x0 and therefore. eλx for all λ ∈ R. ϕ (u + t (v − u)) ≤ ϕ (u) + t (ϕ (v ) − ϕ (u)) ∀ t ∈ [0. 1/2] . 1). for all u ≤ v with u. 7. v ∈ (a.e. 2. Solution to Exercise (7.4. b) . (7. b) . Let f (x) := ϕ (x) − [ϕ (x0 ) + ϕ (x0 ) (x − x0 )] . On one hand we have N N N (1 − an ) ≤ n=1 n=1 e−an = exp − n=1 an Page: 60 job: prob macro: svmonob. x α Fig. Then f (x0 ) = f (x0 ) = 0 while f (x) ≥ 0 a.3). A graph of 1 − x and e−2x showing that 1 − x ≥ e−2x for all x ∈ [0.3. ¨ (t) = − (v − u)2 ϕ (u + t (v − u)) ≤ 0 for almost Then f (0) = f (1) = 0 with f all t. |x| for α > 1.9) (7.7 below for much more on convex functions. f˙ (t) ≤ 0 for t > t0 and f˙ (t) ≥ 0 for t < t0 and hence f (t) ≥ f (1) = 0 for t ≥ t0 and f (t) ≥ f (0) = 0 for t ≤ t0 . 1] .4.

we ﬁnd from limm→∞ P (∩m≥k≥n Ac k ) = 0 and hence n→∞ m→∞ the above ∞ ≥ n=1 (1 − an ) · exp −2 n=m+1 an . Eq.7. Then P (limn→∞ Xn = 0) = 1 iﬀ pn < ∞.} ) = P ({Ac n a. (7. Combining this with the ﬁrst Borel Cantelli Lemma gives the (Borel) Zero-One law. Example 7. P (An i.11) while n=1 ∞ then P ({An i.s. By the second Borel-Cantelli Lemma.a}) = P (∪n=1 ∩k≥n Ak ) . ∞ 0 if n=1 P (An ) < ∞ .d.12) shows m (1 − an ) = n=1 n=1 m (1 − an ) · n=m+1 N (1 − an ) m N −2an P (∩m≥k≥n Ac k) ≤ m≥k≥n e−P (Ak ) = exp − k=n P (Ak ) . we have P (∩m≥k≥n Ac k) = m≥k≥n ∞ Hence if n=1 an = ∞ then n=1 (1 − an ) = 0.14) Then lim sup n→∞ Xn = 1 a. Suppose that {An }n=1 are independent sets.34 (Example 7.33 (Second Borel-Cantelli Lemma).}) = 1. suppose that n=1 an < ∞. n→∞ n→∞ m→∞ (1 − an ) ≤ exp − n=1 ∞ ∞ n=1 an .o. c c P (∪∞ n=1 ∩k≥n Ak ) = lim lim P (∩m≥k≥n Ak ) = lim 0 = 0 n→∞ as desired.30 continued). Making use of the independence of {Ak }k=1 and hence the independence of ∞ {Ac k }k=1 . With this notation we then have for N ≥ m that N m N P (Ac k) = m≥k≥n (1 − P (Ak )) .13) P (X1 > αcn ) < ∞. Sup∞ pose that {Xn }n=1 is a sequence of i. In this case an → 0 as n → ∞ and so there exists an m ∈ N such that an ∈ [0.12) Using the simple inequality in Eq.) = 1 iﬀ P (Xn = 1 i.i. n = 1 from which it follows that macro: svmonob.o. ∞ c m c ∞ Since ∩k≥n Ac k ↑ ∪n=1 ∩k≥n Ak as n → ∞ and ∩k=n Ak ↓ ∩n=1 ∪k≥n Ak as m → ∞.cls date/time: 23-Feb-2007/15:20 Page: 61 job: prob . We are going to prove Eq. If ∞ P (An ) = ∞. (7. ∞ So again letting N → ∞ shows.o. 1/2] for all n ≥ m.o. (7.8) along with Eq. Indeed. c (7.) = 0 iﬀ pn = P (Xn = 1) < ∞. cn (7.15) Proof. ∞ Conversely.10) P X1 > α−1 cn = ∞ (7.35 (Extremal behaviour of iid random variables). ∞ m ∞ (1 − an ) ≥ n=1 n=1 (1 − an ) · exp −2 n=m+1 an > 0. n=1 Proposition 7. (7. P (limn→∞ Xn = 0) = 1 iﬀ P (Xn = 0 a.10). inequality that ≥ n=1 m (1 − an ) · n=m+1 e = n=1 (1 − an ) · exp −2 n=m+1 an Using Eq. (7. Suppose that {Xn } are now independent Bernoulli random variables with P (Xn = 1) = pn and P (Xn = 0) = 1 − pn . Lemma 7.11) by showing.3 Borel-Cantelli Lemmas 61 which upon passing to the limit as N → ∞ gives ∞ ∞ c c c P (∪∞ n=1 ∩k≥n Ak ) = lim P (∩k≥n Ak ) = lim lim P (∩m≥k≥n Ak ) .13) implies P Xn > α−1 cn i. random variables and cn is an increasing sequence of positive real numbers such that for all α > 1 we have ∞ (7.a. ∞ c 0 = P ({An i. (7. n=1 (7.o.) = ∞ 1 if n=1 P (An ) = ∞ Proof.

by the ﬁrst Borel-Cantelli lemma. Thus we have shown that λn −λ λn e ≤ P (X1 ≥ n) ≤ . P Hence. xx = n. cn Xn ≤ αk cn k=n λk −λ λn −λ e = e k! n! λn −λ = e n! ∞ k=n ∞ n! k−n λ k! λn −λ n! λk ≤ e (k + n)! n! ∞ and hence working as above. it follows that lim sup n→∞ = 2 ln n = (n) − 2 (x) 3 ln n ( n ) − 2 2 ln n ln x En = 1 a.o. n! n! = 1. i. k! Similarly. In this case we have ∞ ∞ Taking α = αk = 1 + 1/k.36. P (Xn ≤ αcn a.35. (Observe that P (En ≤ 0) = 0) so that En > 0 a. (7. n) = 1. Now suppose that we wish to choose cn so that P (X1 ≥ cn ) ∼ 1/n.e.s. Suppose now that {Xn }n=1 are i. cn Example 7.s. Taking logarithms of this equation implies that ln n x= ln x and upon iteration we ﬁnd. lim sup n→∞ P (X1 ≥ n) = e−λ k=n λn −λ λk ≥ e k! n! and ∞ Xn ≤ α a. So by Proposition 7. P lim sup n→∞ k=0 k=0 λn 1 k λ = . we may assume that P (X1 ≥ x) ∼ λx λx ∼√ x! 2πxe−x xx Xn =1 cn =P lim sup n→∞ Xn Xn ≥ 1 ∩ lim sup ≤1 cn n→∞ cn Example 7.37.62 7 Independence lim sup n→∞ Xn ≥ α−1 a. √ x! ∼ 2πxe−x xx .i.) Then for cn > 0 and α > 0. x= = ln n ln ln n ln x P (En > αcn ) = n=1 n=1 e −αcn = n=1 e −cn α . P (X1 = k ) = = 1.a.. we ﬁnd P lim sup n→∞ Xn ≥1 cn =P ∩∞ k=1 lim sup n→∞ 1 Xn ≥ cn αk λk −λ e . then we have ∞ ∞ P (En > α ln n) = n=1 n=1 1 n α which is convergent iﬀ α > 1. That is to say. Hence if we choose cn = ln n so that e−cn = 1/n.cls 2 (n) − ln n 3 (n) + (x) . Page: 62 date/time: 23-Feb-2007/15:20 . k! n! lim sup n→∞ Xn ≤1 cn =P ∩∞ k=1 lim sup n→∞ = 1. λ. This suggests that we need to solve the equation. Let {En }n=1 be a sequence of independent random variables with exponential distributions determined by P (En > x) = e−(x∨0) or P (En ≤ x) = 1 − e−(x∨0) . Eq.s. n) = 0 or equivalently.d. distributed by the Poisson distribution with intensity. we have ∞ ∞ ∞ ∞ wherein we have used Stirling’s formula.s. ln n job: prob macro: svmonob. Thus in terms of convergence issues.14) implies P (Xn > αcn i.

7. for any k ∈ N. . n 2. and so by Proposition 7.cls date/time: 23-Feb-2007/15:20 . i. { ∞ n=1 ∞ ∞ We then have. Tn := σ (Xn+1 . where F ⊂ B is almost trivial. .39. B∞ := σ (X1 . lim Sn (Xk+1 + · · · + Xn ) = lim n →∞ bn bn limn→∞ Sn bn where δn (α) → 0 as n → ∞. lim Xn exists in R lim lim sup Xn = lim inf n→∞ Xn n→∞ ∈ T and similarly. 1 ln(n) nα(1+εn (α)) 2 ( n) = (n) 1 α (1+ δn (α)) ln (n) n 2 4. Then there exists c ∈ R a.times 63 7. B . ∞ ∞ Xk converges k=1 = k=n+1 Xk converges ∈ Tn where ln α − 3 (n) εn (α) := . . Example 7. 1} . Here are some example of tail events and tail measurable random variables: 1. . Xn ) . ∞ n→∞ P (X1 ≥ αcn ) < ∞ if α > 1 and n=1 ∞ from which it follows that = 0 ∈ Tk for all k. both lim sup Xn and lim inf n→∞ Xn are T – measurable as are lim sup S bn n→∞ n→∞ and lim inf n→∞ Hence we have Sn bn . A ∈ T . Let Sn := X1 + · · · + Xn and {bn }n=1 ⊂ (0. ) . Let Bn := σ (X1 .35 we may conclude that lim sup n→∞ Xn = 1 a. 1} for all A ∈ F . ln (n) / 2 (n) job: prob Page: 63 macro: svmonob. for α ∈ (0. Indeed. x ≤ ln (n) . We call T the tail σ – ﬁeld and events. 2 ( n) (n) and Let {Xn }n=1 be a sequence of random variables on a measurable space. . Since ln (λ/e) it follows that αcn ln(λ/e) ln n α = αcn ln (λ/e) = α ln (λ/e) = ln n 2 (n) . P (A) ∈ {0. Indeed. ¯ is a random variable which is F meaLemma 7. . it follows that 3 (x) ≤ hence that ln (n) ln (n) 3 (n) x= = 1+O .e. . ( n ) 2 ln(λ/e) α 2 (n) Sn ¯ exists in R bn = lim sup n→∞ Sn Sn = lim inf n →∞ bn bn ∈T and lim Sn exists in R bn Sn bn (λ/e) Therefore. X2 . αc n − αc n (αc ) 2παcn e 2παcn nα(1+εn (α)) n αc for all n ∈ N. Since.s. 2 (n) + O ( 3 (n)) 2 (n) 2 (n) Thus we are lead to take cn := (αcn ) αcn ln(n) . ∞) that = exp (αcn [ln α + ln cn ]) ln (n) [ln α + 2 (n) − 3 (n)] 2 (n) ln α − 3 (n) + 1 ln (n) = exp α 2 (n) = exp α = nα(1+εn (α)) Xn converges} ∈ T . ∞) such that bn ↑ ∞.4 Kolmogorov and Hewitt-Savage Zero-One Laws k . Let (Ω. A σ – ﬁeld. ) . From this observation. 2 (n) P (X1 ≥ αcn ) ∼ √ (λ/e) n λαcn 1 ∼ √ . F ⊂ B is almost trivial iﬀ P (F ) = {0. we may show. .s. ¯ = 3. Suppose that X : Ω → R ¯ such that X = c surable.38. limn→∞ = 0 ∈ T . B ) . (Ω. .4 Kolmogorov and Hewitt-Savage Zero-One Laws 3 where k = ln ◦ ln ◦ · · · ◦ ln. P ) be a probability space. are called tail events. P (X1 ≥ αcn ) = ∞ if α < 1 n=1 Deﬁnition 7. = −∞ < lim sup n→∞ Sn Sn = lim inf <∞ n→∞ bn bn ∈T. P (X1 ≥ αcn ) ∼ n α ln(λ/e) 2 (n) αcn =n . and T := ∩∞ n=1 Tn ⊂ B∞ .40. Xn+2 . .

Proposition 7. . 2 Proposition 7. we have −1 Tπ (B ) = {ω : ((Tπ (ω ))1 . −1 −1 (B ) . Since B = {(X1 . we ﬁnd tity. Keeping the assumptions in Proposition 7. Suppose that P is a ∞ probability measure on (Ω. F (x) := P (X ≤ x) ∈ {0.e. it follows that P (A) − [P (A) + O (ε)] 2 = |1A − 1B | and P (A B ) = E |1A − 1B | . . such that x = x2 is x = 0 or x = 1. In particular. and lim inf n→∞ Proof. P (X ≤ x) is either 0 or 1. ωπ2 . a bijective map on N). Since P (An ∆B ) = P ([An \ B ] ∪ [B \ An ]) = P (An \ B ) + P (B \ An ) → P (C \ B ) + P (B \ C ) < ε. B ) such that {Xn }n=1 are independent random variables. . Then T is almost trivial. .44 (Hewitt-Savage Zero-One Law). . Xn ) . Then lim sup Xn . for suﬃciently large n. 1} is right continuous. there exists An ∈ B0 such that An ↑ C ∈ (B0 )σ . . . Xn+n ) ∈ B } ∈ σ (Xn+1 . π (j + n) = j for j = 1. (B ) are independent with P (B ) = P Tπ it follows that B and Tπ 2 −1 Therefore P B ∩ Tπ B = P (B ) . . Since the function. Combining this observation with the iden−1 A . . ωn+n ) ∈ B } = {(Xn+1 . {X ≤ x} ∈ F and therefore. Proof.41 (Kolmogorov’s Zero-One Law). Then S is almost trivial. Let P be a probability ∞ measure on (Ω. Let B0 := ∪∞ n=1 σ (X1 . .a. . . 2. X2 .d. the result is proved. lim Sn n→∞ bn exists = 1 and in the either P latter case lim Sn exists n→∞ bn lim Sn = c a. In particular the tail events in Example 7. it suﬃces to ﬁnish the proof under the added condition that P (X ∈ R) = 1. In the proof below we will use the identities. ) . Since |P (A) − P (B )| ≤ P (A∆B ) < ε. lim inf n→∞ Xn .10. . X2 . But A ∈ B and hence A is independent of itself.43.i. Xn ) ∈ B } = {ω : (ω1 . ∞) such that bn ↑ ∞. . Since A ∈ Tn for all n and Tn is independent of Bn . Let A ∈ T ⊂ B∞ .s n→∞ bn = 0 or P ¯. . P (A) = P (A ∩ A) = P (A) P (A) .cls date/time: 23-Feb-2007/15:20 . . . . and π (j + 2n) = j + 2n for all j ∈ N. X2 .41 and let ∞ {bn }n=1 ⊂ (0. . For each x ∈ R. B ⊂ C. Hence. Since the only x ∈ R. Xn (ω ) = ωn for all ω ∈ Ω. . and P (C \ B ) < ε. π : N → N is ﬁnite if π (n) = n for a. . non-decreasing and F (−∞) = 0 and F (+∞) = 1. n. . Xn ) ⊂ B0 such that P (A∆B ) < ε. By the regularity Theorem 5. it follows that A is independent of B∞ = σ (∪Bn ) = ∨∞ n=1 Bn . there is a unique point c ∈ R where F (c) = 1 and F (c−) = 0. < ε. n. S ⊂ B. and A ∈ σ (X1 . . . . . . n→∞ n lim sup S bn . .64 7 Independence Proof. Corollary 7. ε > 0.42. we have P (X = c) = 1. if P (X = ∞) > 0 or P (X = −∞) > 0. we have P (A∆B ) < ε where A = An ∈ B0 . P (A) = P (A ∩ A) = P A ∩ Tπ n→∞ Sn bn are all constant almost surely. . then P (X = ∞) = 1 or P (X = −∞) = 1 respectively. . . Since {X = ∞} and {X = −∞} are in F . sequence. . . Let π : N → N be the permutation deﬁned by π (j ) = j + n. B ) such that {Xn }n=1 is an i. it follows that A is independent of ∪∞ n=1 Bn for all n. . We say a permutation (i. 1A B −1 −1 −1 = E [1A − 1B ] 1Tπ A + 1B 1Tπ A − 1Tπ B −1 −1 ≤ E |[1A − 1B ]| + E 1Tπ A − 1Tπ B −1 −1 = P (A∆B ) + P Tπ A∆Tπ B < 2ε. and B := σ (X1 . . P (A) ∈ {0. .e. . for some c ∈ R P (A) − P (B ) 2 −1 −1 = P A ∩ Tπ A − P B ∩ Tπ B −1 −1 ≤ E 1A∩Tπ A − 1B ∩Tπ B −1 −1 = E 1A 1Tπ A − 1B 1Tπ B −1 −1 = E 1A∩Tπ A − 1B ∩Tπ B Let us now suppose that Ω := R∞ = RN . . . ωn ) ∈ B } for some B ∈ BRn . . i. Now suppose that B ∈ S .e. 1} for all A ∈ T . Then B0 is an algebra and σ (B0 ) = B . . Deﬁnition 7. Since the latter set is a multiplicative set. macro: svmonob. . . ) . ωπn ) ∈ B } = {ω : (ωn+1 .38 have probability either 0 or 1. . is the collec−1 tion of sets. (Tπ (ω ))n ) ∈ B } = {ω : (ωπ1 . The permutation invariant σ – ﬁeld. A ∈ B such that Tπ (A) = A for all ﬁnite permutations π. At this point. Deﬁne Tπ : Ω → Ω by Tπ (ω ) = (ωπ1 . we may conclude that P (A) = P (A) for all A ∈ S . Page: 64 job: prob Since ε > 0 was arbitrary. . Xn+n ) . for any B ∈ B and ε > 0. . i.

lim sup Sn = ∞ and lim sup (−Sn ) = ∞ a. n→∞ n→∞ −1 Tπ lim sup Sn ≤ x n→∞ = lim sup Sn ◦ Tπ ≤ x n→∞ = lim sup Sn ≤ x n→∞ which shows that lim sup Sn is S – measurable.The results of Exercise 7. Indeed.44. −1 Tπ ({Sn ∈ B i. and −Xn = Xn . Therefore. n→∞ n→∞ 3. n→∞ n→∞ n→∞ d ∞ d ∞ ∞ ∞ d Thus we have c = lim sup (−Sn ) = − lim inf Sn ≥ − lim sup Sn = −c. Hence {Sn ∈ B i.s. n→∞ n→∞ .} is in the permutation invariant σ – ﬁeld.o. we must have either c ∈ {±∞} or X1 = 0 a.s.} .} 2.d. with c ∈ {±∞} .a.o. 1. c ≥ −c. As above. as well.s. Since the latter is not allowed. we know lim sup Sn = c a.45 (Some Random Walk 0 − 1 Law Results). we must c = ∞. From item 2.) ∈ {0. we know that and from what we have already proved. P (X1 ∈ A) = P (−X1 ∈ A) for all A ∈ BR . or lim sup Sn = −∞ a. if π is a ﬁnite permutation. if Sn = X1 + · · · + Xn .} = {Sn ∈ B i. i.s. for some c ∈ R n→∞ n→∞ n→∞ n→∞ n→∞ c = lim sup Sn+1 = lim sup (Sn + X1 ) = lim sup Sn + X1 = c + X1 . Hence in this symmetric case we have shown.s.i.e. If P (X1 = 0) > 0. n→∞ n→∞ or equivalently that lim sup Sn = ∞ and lim inf Sn = −∞ a. n→∞ n→∞ n→∞ Since the c = −∞ does not satisfy. The same goes for {Sn ∈ B a. Since.s.o.s.s. Indeed.}) = {Sn ◦ Tπ ∈ B i.2 then imply that lim sup Sn = lim sup (−Sn ) and in particular lim sup (−Sn ) = c a. 1} for all B ∈ BR .o. lim sup Sn = c ¯ .s. a. lim sup Sn = ∞ or lim sup Sn = −∞ a. it follows that {Xn }n=1 = {−Xn }n=1 . Continue the notation in Proposition 7.o.s. then lim sup Sn = ∞ a.Example 7. Now assume that P (X1 = 0) > 0 and X1 = −X1 . n→∞ d Since {Xn }n=1 and {−Xn }n=1 are i.. then P (Sn ∈ B i. a.

.

f : Ω → R ∞. . That is if fn ∈ L1 (µ) and there exists g ∈ L1 (µ) such that |fn | ≤ g and f := limn→∞ fn exists pointwise. ∞). Ω Ω n→∞ n→∞ Ω This integral has the following properties. λ∈[0.4 We write A f dµ := Ω 1A f dµ for all A ∈ B where f is a measurable function such that 1A f is either non-negative or integrable. ∞] measurable. R) and m be Lebesgue measure on R.3 (Extension to integrable functions). f dµ + λ Ω gdµ Notation 8.2. b].1 (Extension to positive functions). ∞] . then f dµ = lim fn dµ = lim fn dµ. . let mesh(π ) := max{|aj − aj −1 | : j = 1. 1. 2. Suppose −∞ < a < b < ∞. n} and fπ (x) := l=0 n−1 lim fn dµ = lim n→∞ fn dµ. integrable if it is measurable and We call a function.15 below. the integral of f with respect to µ is deﬁned by f (x) dµ (x) := sup {Eµ ϕ : ϕ is simple and ϕ ≤ f } .al+1 ] (x). This integral is linear in the sense that (f + λg ) dµ = Ω Ω Notation 8. if 0 ≤ fn ↑ f. b ∈ R a b a whenever f. . we will often write b f (x) dx or surable function and a < b with a. ∞] was a measurable simple function. |f | dµ < ¯ . Ω See the monotone convergence Theorem 8. B . f is a non-negative Borel mea¯ .34 ϕn ↑ f where n2 −1 n ϕn := k=0 k 1 k + n1{f >n2n } . we know from the approximation Theorem 6.6.b]∩R f dm. k+1 2n { 2n <f ≤ 2n } Then f (al ) 1(al . i. Given a partition.5 If m is Lebesgue measure on BR . The integral is continuous under increasing limits.3 above.∞] f dµ = lim Ω n→∞ ϕn dµ Ω n2n −1 k=0 = lim n→∞ k µ 2n k k+1 <f ≤ 2n 2n + nµ (f > n2n ) . . g ≥ 0 are measurable functions and λ ∈ [0. we will greatly extend the “simple” integral or expectation which was developed in Section 4.8 Integration Theory In this chapter.e. For a positive measurable function. We will denote the space of µ –integrable functions by L1 (µ) Ω 8.1 A Quick Introduction to Lebesgue Integration Theory Theorem 8. f ∈ C ([a.34). X Theorem 8. . µ) was measurable space and f : Ω → [0. Remark 8. Given f : Ω → [0. π = {a = a0 < a1 < · · · < an = b}. Moreover this extension is continuous under dominated convergence (see Theorem 8. Example 8. Therefore by the monotone convergence theorem. Recall there that if (Ω. then we let Eµ f := λµ (f = λ) . The integral extends to a linear function from L1 (µ) → R. f : Ω → [0. then f dµ = Ω Ω n→∞ f dm for (a.

R) ∩ C 1 ((a.cls date/time: 23-Feb-2007/15:20 .b] (x) for all x ∈ (a. For all λ > 0. Since gdm = M (b − a) < ∞.C. If G ∈ C ([a. b). we have by assumption that G (x) = F (x) for x ∈ (a. F − G = C for some constant C.b) ) then b The proof of these identities are similar. i. y and 1(a.51 below for a more general statement along these lines.1] 1 dm(x) xp 1 1/n ≤ 1 |h| x+h |f (y ) − f (x)| dm(y ) x x | f (y ) − f (x)| dm(y ) x+h if h > 0 if h < 0 1 x−p+1 dx = lim n→∞ 1 − p xp = ≤ sup {|f (y ) − f (x)| : y ∈ [x − |h| . b). b). Example 8. it easily follows that limk→∞ fπk (x) = f (x) and that |fπk (x)| ≤ g (x) := M 1(a. R b k→∞ b b We can use the above results to integrate some non-Riemann integrable functions: Example 8. Since F (x) := R 1(a.x) (y ) = 1(a. it follows from the dominated convergence Theorem 8. b). a R 1 dm(x) = lim N →∞ 1 + x2 N −N 1 dx 1 + x2 Proof.T. 1 + x2 This equation with Eq. For the converse direction.e. Let us also consider the functions x−p . to conclude. b]. (8. F ∈ C ([a. ∞ lim fπk dm = a a k→∞ lim fπk dm = a f dm. b). we may apply D.e.b] |f (x)| < ∞. 2. ∞ N N e−λx dm(x) = lim 0 N →∞ e−λx dm(x) = lim 0 N →∞ e−λx dx 0 = − lim and 1 dm(x) = lim N →∞ 1 + x2 = lim N →∞ N →∞ 1 −λx N e |0 = λ−1 λ N −N f (x)dm(x) = G(b) − G(a). R)∩L1 ((a.8. f ∈ C ((a. b].x) (y )f (y ) ≤ 1(a. R).34 that F is continuous on [a. the Lebesgue and the Riemann integral agree on continuous functions. where the latter integral is the Riemann integral. 1 dm(x) = π. Therefore by the mean value theorem. Therefore if {πk }k=1 is a sequence of partitions with limk→∞ mesh(πk ) = 0. Theorem 8.x) (y )f (y )dm(y ).1] (x) (0. limx→z 1(a. This shows F = f on (a. b]. by the continuity of f. See Theorem 8.1) a f (x)dm(x) = F (b) = F (b) − F (a) = (G(b) + C ) − (G(a) + C ) = G(b) − G(a). we know that b k→∞ b and the latter expression. m) and F (x) := a f (y )dm(y ). b) (i. Then 1. R) ∩ C 1 ((a. Simple manipulations show. 1 dm(x) = lim n→∞ xp = lim n→∞ 1 1−p 1 0 1 1 n 1 1( n .68 8 Integration Theory b n−1 n−1 fπ dm = a l=0 f (al ) m ((al .6 and the fundamental theorem of calculus for Riemann integrals (or Theorem 8. Hence b lim fπk dm = a a f (x) dx (8.7 (The Fundamental Theorem of Calculus). b). b].z) (y ) for m – a. goes to zero as h → 0 . R). Suppose −∞ < x a < b < ∞. 3. By the monotone convergence theorem.1) shows b b e−λx dm(x) = λ−1 and f (x) dx 0 R f dm = a a whenever f ∈ C ([a. R) is an anti-derivative of f on (a. F (x) = f (x) for all x ∈ (a. x+h 1 x [f (y ) − f (x)] dm(y ) if h > 0 F (x + h) − F (x) − f (x) = h |h| x [f (y ) − f (x)] dm(y ) if h < 0 x+h tan−1 (N ) − tan−1 (−N ) = π. b] . Using the (uniform) continuity of f on [a.b) (y ) |f (y )| is an L1 – function.7 below). al+1 ]) = l=0 ∞ f (al ) (al+1 − al ) is a Riemann sum. x + |h|]} if p < 1 ∞ if p > 1 Page: 68 job: prob macro: svmonob.e. b). b] where M := maxx∈[a. f = G |(a.

1. let fn (x) := 1 − n x ≥ 0 and by taking logarithms of Eq. fn (x) ↑ e−x as n → ∞ as claimed. −x n→∞ lim cos sin 0 e 0 −x dm(x) = 1 < ∞. 4. Thus we have shown. The following limit holds. we have 1− x n n θ n e−n sin(θ) dθ. 60. But for n > x.8.1 A Quick Introduction to Lebesgue Integration Theory 69 If p = 1 we ﬁnd 1 dm(x) = lim n→∞ xp 1 1 n (0. 1 p−1 if p > 1 Example 8. let us proceed to the formal development of the facts stated above. 1−y x n x n2 Exercise 8. n→∞ x x x n d d n ln 1 − = ln 1 − + ln fn (x) = dn dn n n 1− x x = ln 1 − + n x = h (x/n) n 1− n where.. it follows that h ≥ 0. π n→∞ x 1[0. For all a ∈ R compute.1] 1 dx = lim ln(x)|1 1/n = ∞. θ n fn (θ) := 1(0. let us consider the limit.28 on p. (1+ 0 n) n→∞ 1 1+nx2 lim 2 n dx n→∞ 0 (1+x ) ∞ n sin(x/n) lim x(1+x2 ) dx n→∞ 0 lim 1− 0 x n n ∞ dm(x) = lim = 0 n→∞ ∞ fn (x)dm(x) 0 ∞ n→∞ lim fn (x)dm(x) = 0 e−x dm(x) = 1. Show ∞ 1 1 dm (x) = xp ∞ if p ≤ 1 . lim cos sin 0 ln (1 − x) ≤ −x for x < 1. ∞). θ n e−n sin(θ) dθ = R 1{π} (θ) dm (θ) = m ({π }) = 0. To do this we must show that n → fn (x) is increasing in n for each x and for this it suﬃces to consider n > x.π] ∈ L1 (m) and from which it follows that 0 ≤ fn (x) ≤ e From Example 8.π] (θ) cos sin = en ln(1− n ) ≤ e−n( n ) = e−x x x e−n sin(θ) .π] (θ) 1{π} (θ) = 1{π} (θ) . Then limn→∞ fn (x) = e−x for all To verify this.8. ∞ The limit in the above example may also be computed using the monotone convergence theorem. Hence by the dominated conso that e vergence theorem. lim 2. is an integrable function on [0. for x < n. f (a) := lim n→∞ n(1 + n2 x2 )−1 dx. a Now that we have an overview of the Lebesgue integral. Then |fn | ≤ 1(0. (7. π n→∞ lim fn (θ) = 1(0.C.10 (Jordan’s Lemma).2 (Folland 2. macro: svmonob. x ∞ sin( n ) x n dx. for 0 ≤ y < 1.8). Therefore by the D. 3. Example 8.n] (x). we know ∞ −x for all x ≥ 0.).9. Let Therefore. h (y ) := ln(1 − y ) + Since h (0) = 0 and h (y ) = − 1 1 y + + >0 1 − y 1 − y (1 − y )2 y . n n→∞ lim 1− 0 n x n n dm(x) = 1. In this example.T. n n→∞ Exercise 8. Compute the following limits and justify your calculations: 1.cls date/time: 23-Feb-2007/15:20 Page: 69 job: prob .

Let X be a set and ρ : X → [0. ∞] : f is measurable}. X p (8.∞) z 1{ϕ=z} (x) = z ∈[0. = sup Eµ ϕ : ϕ is simple and λ−1 ϕ ≤ f = sup {Eµ [λψ ] : ψ is simple and ψ ≤ f } = sup {λEµ [ψ ] : ψ is simple and ψ ≤ f } =λ X Proof.∞) zµ({ϕ = z }) = X ϕdµ. ∞] be a function. Moreover. 3. then f dµ ≤ X (8. If X f dµ < ∞ then µ(f = ∞) = 0 (i.2) for all n.3) Lemma 8.3) is called Chebyshev’s Inequality for p = 1 and Markov’s inequality for p = 2. let f dµ := A X We say the f ∈ L+ is integrable if f (x) dµ (x) = A ≤ 1 f ε p and by monotonicity and the multiplicative property of the integral. of Proposition 4. g ∈ L (B ) .e. let µ = x∈X ρ(x)δx on B = 2X . (8. ∞) is a simple function. Let L+ = L+ (B ) = {f : X → [0. Page: 70 job: prob macro: svmonob. X 4.13. 1 3.∞) z x∈X ρ(x)1{ϕ=z} (x) f dµ. then λf dµ = λ X X + 1{f ≥ε} dµ ≤ 1 ε p 1{f ≥ε} f p dµ ≤ X 1 ε p f p dµ. If A ∈ B . Then: 1. ∞] is a function (which is necessarily measurable). Thus if µ (f = ∞) = 0. if λ ≥ 0.2) follows from the deﬁnition of the integral. = z ∈[0. X wherein λ X f dµ ≡ 0 if λ = 0. {f > 0} = ∪∞ n=1 {f > 1/n} with µ (f > 1/n) ≤ n X f dµ < ∞ then f dµ < ∞ for each n.e. Since 1{f ≥ε} ≤ 1{f ≥ε} 1 ε f ≤ ε f we have 1{f ≥ε} ≤ 1{f ≥ε} 1 f ε p f dµ := sup {Eµ ϕ : ϕ is simple and ϕ ≤ f } . then ϕn := n1{f =∞} is a simple function such that ϕn ≤ f for all n and hence nµ (f = ∞) = Eµ (ϕn ) ≤ f dµ X X f dµ f dµ = ∞.e. (8. 1.14 (Sums as Integrals). µ(A) = x∈A The inequality in Eq. µ(f ≥ ε) = X Remark 8. 4. Let f. Letting n → ∞ shows X f dµ = ∞.16. Lemma 8. i. Proof.70 8 Integration Theory 8. X ϕdµ = Eµ ϕ so that X is an extension of Eµ .2 Integrals of positive functions Deﬁnition 8.11. Since {ϕ is simple and ϕ ≤ f } ⊂ {ϕ is simple and ϕ ≤ g } . 1 µ(f ≥ ε) ≤ p ε 1 f 1{f ≥ε} dµ ≤ p ε X p f dµ. then f dµ = X X f ρ.∞) z 1{ϕ=z } and ϕρ = X x∈X ρ(x) z ∈[0. Suppose that ϕ : X → [0. Because of item 3. X gdµ. If µ (f = ∞) > 0.) and the set {f > 0} is σ – ﬁnite. f dµ < ∞. λf dµ = sup {Eµ ϕ : ϕ is simple and ϕ ≤ λf } X ρ(x). If f : X → [0. We may assume λ > 0 in which case.12. then ϕ = z ∈[0. 1A f dµ. if ϕ is a non-negative simple function. if 0 ≤ f ≤ g. even if 2. For all ε > 0 and p > 0. Eq.cls date/time: 23-Feb-2007/15:20 . Deﬁne f (x) dµ (x) = X X X 2. f < ∞ a.

cls n→∞ n→∞ ψn = date/time: 23-Feb-2007/15:20 . Notice that Xn ↑ X and fn ≥ α1Xn ϕ and so by deﬁnition of fn .Λ (x) := 1Λ (x)f N (x). Then (ϕn + ψn ) is simple as well and (ϕn + ψn ) ↑ (f1 + f2 ) so by the monotone convergence theorem.15 (Monotone Convergence Theorem). 1) was arbitrary we may further conclude. f (x)} and let ϕN. then ϕdµ = X X fn ≥ Eµ [α1Xn ϕ] = αEµ [1Xn ϕ] .Λ be the simple function given by ϕN. Page: 71 job: prob macro: svmonob.16.Λ ρ = X y lim µ({ϕ = y }) = Eµ [ϕ] y>0 n→∞ ϕN. Since f N ↑ f as N → ∞. If fn ∈ L+ is a sequence of functions then ∞ ∞ fn = Theorem 8.4) (f1 + f2 ) = lim = lim (ϕn + ψn ) = lim ϕn + lim ϕn + f1 + ψn f2 . Suppose fn ∈ L+ is a sequence of functions such that fn ↑ f (f is necessarily in L+ ) then fn ↑ f as n → ∞. ∞) is a simple function such that ϕ ≤ f. for all n ≤ m < ∞. (8.Λ (x) ≤ f (x). In particular.Λ dµ ≤ X f dµ. lim fn ≤ f. Because ϕN. fn < ∞ a.e. (8.5) to conclude limn→∞ fn ≥ αEµ [ϕ] and since α ∈ (0. let ϕ : X → [0. if ∞ n=1 fn < ∞ then Proof. Then using the continuity of µ under increasing unions.Eµ [ϕ] ≤ limn→∞ fn . fn ≤ from which if follows fm ≤ f fn is increasing in n and n→∞ by choosing non-negative simple function ϕn and ψn such that ϕn ↑ f1 and ψn ↑ f2 . (8. Set f N (x) = min {N. f dµ. This identity allows us to let n → ∞ in Eq. The latter inequality being true for all simple functions ϕ with ϕ ≤ f then implies that f ≤ lim n→∞ fn . Since fn ≤ fm ≤ f. f ρ= Λ X N ﬁnite sum y>0 y lim µ(Xn ∩ {ϕ = y }) n→∞ = ϕN. this implies fρ ≤ X X which combined with Eq.4) proves the theorem. α ∈ (0. n=1 n=1 ∞ n=1 fn . Taking the sup over ϕ in this last equation then shows that f dµ ≤ X X n→∞ n→∞ y 1{ϕ=y} f ρ. Since Λ is arbitrary. lim Eµ [1Xn ϕ] = lim 1Xn y>0 (8. Corollary 8. we may let N → ∞ in this last equation to concluded fρ ≤ Λ X f dµ. 1) and Xn := {fn ≥ αϕ} . n→∞ n→∞ For the opposite inequality. let Λ ⊂⊂ X be a ﬁnite set and N ∈ (0. ∞) be a simple function such that 0 ≤ ϕ ≤ f. ∞).5) ϕρ ≤ X f ρ.2 Integrals of positive functions 71 So if ϕ : X → [0.8. First oﬀ we show that (f1 + f2 ) = f1 + f2 Proof. = lim = n→∞ yµ(Xn ∩ {ϕ = y }) y>0 For the reverse inequality.

Also suppose that µn (A) is increasing in n for all A ∈ B. Suppose that {fn } is a sequence of non-negative measurable functions and f is a measurable function such that fn ↑ f oﬀ a null set.e.e. Suppose. then gN ↑ g and so again by monotone convergence theorem and the additivity just proved. Proposition 8.e. Therefore. Also if f. gk ≤ and therefore macro: svmonob. So by the monotone convergence theorem.e. f = 0 a.13). then f dµ ≤ gdµ. then f dµ = gdµ. N Proof. B := 2N . Since 1E c f ≤ 1E c g everywhere.18. the function N 1E c f dµ. Let E ⊂ X be a null set such that fn 1E c ↑ f 1E c as n → ∞.e. then by (Lemma 8.e.19. f dµ = Corollary 8. then fn ↑ f as n → ∞. fn Proof.e. Because g = 1E c g + 1E g and 1E g = 0 a.72 8 Integration Theory N ∞ Now to the general case. fN (·) := n=1 f (n) 1{n} is a simple function with fN ↑ f as N → ∞. In fact the deﬁnition f dµ makes sense for all functions f : X → [0.21 (Fatou’s Lemma).19. µ(f > 0) ≤ n=1 µ(f ≥ 1/n) = 0. = lim N →∞ gN = g =: n=1 fn . E := {x ∈ X : f (x) > g (x)}. Then by the monotone convergence theorem and Proposition 8. the linearity of the integral) that we really make use of the assumption that all of our functions are measurable.3. If f = 0 a. Then for f : N → [0. i. Since gk ≤ fn for all k ≤ n. ∞] not just measurable functions. gdµ = and similarly f dµ = 1E c gdµ + 1E gdµ = 1E c gdµ Remark 8. Suppose that f ≥ 0 is a measurable function. Conversely. ∞] are measures on B for n ∈ N. if f dµ = 0.17. we use the approximation Theorem 6. ∞] is a sequence of measurable functions then lim inf fn ≤ lim inf n→∞ n→∞ n≥k Exercise 8. Let gN := n=1 fn and g = 1 fn .34 which relies heavily on the measurability of the functions to be approximated. It is in the proof of this corollary (i. ∞] deﬁned by µ(A) := limn→∞ µn (A) is also a measure. X µ(f ≥ 1/n) ≤ n ∞ fn := lim n=1 N →∞ fn = lim n=1 N →∞ fn n=1 ∞ f dµ = 0 for all n. g ≥ 0 are measurable functions such that X f ≤ g a. Hint: use Example 8. and µ (A) = # (A) for A ⊂ Ω is the counting measure on B . fn = fn 1E c ↑ f 1E c = f as n → ∞. For the second assertion let E be the exceptional set where f > g. Lemma 8. ∞). Suppose that µn : B → [0..e.16. in the proof of Corollary 8. and ϕ ≤ f is a simple function then ϕ = 0 a. Prove that µ : B → [0. ∞ N N Proof. If fn : X → [0. This implies that µ(ϕ−1 ({y })) = 0 for all y > 0 and hence X ϕdµ = 0 and therefore f dµ = 0.e.20. In particular if f = g a. However. Example 8. By assumption E is a null set and 1E c f ≤ 1E c g everywhere. i. Deﬁne gk := inf fn so that gk ↑ lim inf n→∞ fn as k → ∞. 1E c f dµ ≤ 1E c gdµ = gdµ. Then f dµ = 0 iﬀ f = 0 a. f dµ = lim N N →∞ fN dµ = lim N N ∞ N →∞ f (n) µ ({n}) n=1 = lim N →∞ f (n) = n=1 n=1 f (n) . Ω = N.18 and the monotone convergence theorem.e.cls date/time: 23-Feb-2007/15:20 fn for all n ≥ k Page: 72 job: prob . Moreover the monotone convergence theorem holds in this generality with no change in the proof.

(Second Proof.8. µ) is a measure space and {An }n=1 ⊂ B is a collection of sets such that µ(Ai ∩ Aj ) = 0 for all i = j.6) {An i. 1 √ µ(An ) < ∞.) Let us ﬁrst observe that ∞ and the latter set has measure 0 being the countable union of sets of measure zero. Suppose that (X.a.cls date/time: 23-Feb-2007/15:20 Page: 73 job: prob .22 (The First Borell – Carntelli Lemma). (8.o.) Of course we may give a strictly measure theoretic proof of this fact: µ(An i.2 Integrals of positive functions 73 gk ≤ lim inf n→∞ fn for all k.e.o. and set ∞ it suﬃces to show ∞ 1An = 1∪∞ µ – a. Since µ (∪∞ n=1 An ) = X ∞ We may now use the monotone convergence theorem to let k → ∞ to ﬁnd lim inf fn = n→∞ k→∞ 1∪∞ dµ and n=1 An ∞ lim gk = MCT k→∞ lim gk ≤ lim inf n→∞ fn .6) and hence the corollary. B . In particular. B . µ(An ) < ∞ then ∞ ∞ ∞ ∞> n=1 ∞ µ(An ) = n=1 X 1An dµ = X n=1 1An dµ implies that n=1 1An (x) < ∞ for µ . µ (∪∞ n=1 An ) = n=1 µ(An ). = 5 if x = rn . ∞ we ﬁnd ∞ Corollary 8. that f < ∞ for almost every x ∈ [0. x. µ) be a measure space. n=1 An n=1 (8. 1 0 {An i. By Theorem 8. An ∈ B . m(f = ∞) = 0.o.}) = 0.o.24. Let (X. x: n=1 1An (x) = 1∪∞ (x) n=1 An = ∪i<j Ai ∩ Aj Proof.} = {x ∈ X : x ∈ An for inﬁnitely many n’s} = N =1 n≥N An .o.e. Example 8. µ(An ) = n=1 X n=1 1An dµ The following Lemma and the next Corollary are simple applications of Corollary 8. That is to say µ({An i. Proof. (First Proof. 1] and this implies that macro: svmonob. then ∞ f (x)dm(x) = [0. Now n=1 1An ≥ 1∪∞ and n=1 An some i = j.7.}) = 0.1] n=1 2−n [0. 1] and deﬁne ∞ 1 f (x) = 2−n | x − rn | n=1 with the convention that 1 |x − rn | Since. that is ∞ ∞ ∞ n=1 1An (x) = 1∪∞ (x) iﬀ x ∈ Ai ∩ Aj for n=1 An If ∞ n=1 µ(An ) < ∞ then µ({An i. Lemma 8. Let {rn }∞ n=1 be an enumeration of the points in Q ∩ [0. This proves Eq.23.) = lim µ N →∞ n≥N An µ(An ) 1 |x − rn | dx = ≤ lim and the last limit is zero since ∞ n=1 N →∞ n≥N rn 1 1 √ dx + dx x − r r −x n n rn 0 √ √ √ √ rn = 2 x − rn |1 1 − rn − rn rn − 2 rn − x|0 = 2 ≤ 4. i.} = Hence if ∞ n=1 x∈X: n=1 1An (x) = ∞ .e.1] 1 |x − rn | ∞ dx ≤ n=1 2−n 4 = 4 < ∞.16.

Since h = h+ − h− .e. let f dµ = f+ dµ − f− dµ A similar calculation works for a > 0 and the case a = 0 is trivial so we have shown that af = a f. job: prob macro: svmonob. we may assume that f. because |f | < ∞ and |g | < ∞ a. h2 ∈ f + g. then h1 = h2 a.cls date/time: 23-Feb-2007/15:20 . R) . If a < 0. R) because |af + bg | ≤ |a| |f | + |b| |g | ∈ L1 (µ. f dµ ∈ R f dµ ≤ gdµ for all f.e.25. Page: 74 |f | dµ < ∞ . implies 0 ≤ g − f a. Notice that if f. R) we may deﬁne (f + g ) dµ = hdµ X X Finally if f+ − f− = f ≤ g = g+ − g− then f+ + g− ≤ g+ + f− which implies that f+ + g− ≤ g+ + f− or equivalently that f= f+ − f− ≤ g+ − g− = g.74 8 Integration Theory ∞ 2−n n=1 1 |x − rn | < ∞ for a.19 that f dµ = gdµ. g ∈ L1 (µ. A measurable function f : X → C is integrable if |f | dµ < ∞. a measurable function f is integrable iﬀ L1 (µ. g ∈ L1 (µ.e. where h is any element of f + g. the fact that f ≤ g a. i. h+ + and hence h= h+ − h− = f+ + g+ − f− − g− = f+ g.26 (Abuse of notation) We will sometimes denote the integral f dµ by µ (f ) . Proposition 8. X h+ − h− = f+ − f− + g+ − g− or h+ + f− + g− = h− + f+ + g+ . is not of the form ∞ − ∞ or −∞ + ∞. We write L1 (µ. then f± = g± a. C) := f : X → C : f is measurable and X is linear and has the monotonicity property: L1 (µ. X Remark 8. Since f± ≤ |f | ≤ f+ + f− . R) and h1 . then (af )+ = −af− and (af )− = −af+ so that af = −a f− + a f+ = a( f+ − f− ) = a f. g ∈ L1 (µ. We use a similar convention for f − g. let X L1 (µ. g : X → R ¯ such that h(x) = f (x) + g (x) the collection of measurable functions h : X → R whenever f (x) + g (x) is well deﬁned. R) := |f | dµ < ∞. 1]. 8. With this notation we have µ (A) = µ (1A ) for all A ∈ B .28. x ∈ [0.e. Therefore. The map f ∈ L1 (µ. and Proposition 8.e.e. A measurable function f : X → R f 1{f ≥0} and f− = −f 1{f ≤0} are integrable. g ∈ L1 (µ. g ∈ Deﬁnition 8. b ∈ R. Notation 8.e. This result is somewhat surprising since the singularities of the summands form a dense subset of [0.e.e. ¯ are two measurable functions. Let f. g are real valued functions.19. and so it follows from Proposition 8.27. We have af + bg ∈ L1 (µ. R) and f = g a. Proof. f− + g− = h− + f+ + g+ ¯ : f is measurable and f :X→R If f. R) → X The monotonicity property is also a consequence of the linearity of the integral. let f + g denote Convention: If f. R) for the space of real valued integrable functions.3 Integrals of Complex Valued Functions ¯ is integrable if f+ := Deﬁnition 8. 1]. R) such that f ≤ g a. Analogously to the real case. In particular if f. R) and a. R) . For f ∈ L1 (µ.29. Now set h = f + g. Hence |f | dµ < ∞ . By modifying f and g on a null set.

f ∼ g iﬀ f = g a. More generally we may deﬁne Lp (µ) = Lp (X. Suppose that f ∈ L1 (µ.3 Integrals of Complex Valued Functions 75 denote √the complex valued integrable functions. |Im f |) ≤ |f | ≤ 2 max (|Re f | .e. C) deﬁne f dµ = Re f dµ + i Im f dµ. 1 Proof. ∞) as the set of measurable functions f such that |f | dµ < ∞ X p c) f = g a. By Chebyshev’s inequality. B .e. We make this into a normed space using the norm f −g L1 = |f − g | dµ L1 f dµ = X X Re e −iθ f dµ ≤ X Re e −iθ f dµ ≤ X |f | dµ. Let f. 1.cls date/time: 23-Feb-2007/15:20 Page: 75 . it must be that 8. let L1 (µ) be either L1 (µ. R) . Using the monotonicity in Proposition Deﬁnition 8. Similarly. (c) =⇒ (b) is clear and so is (b) =⇒ (a) since f− E E Proposition 8.31. max (|Re f | .33. C) . (8. On occasion this can be dangerous and this danger will be pointed out when necessary. in fact {|f | ≥ ∞ for all n. ∞] is a measurable function.30. Since R = e−iθ X g ≤ |f − g | = 0. Im e−iθ f dµ = 0. Proof. Because. (a) =⇒ (c) Notice that f= E g⇔ E E (f − g ) = 0 It is routine to show the integral is still linear on L1 (µ. Start by writing X f dµ = Reiθ with R ≥ 0.32.e. C) or L1 (µ. We may assume that R = X f dµ > 0 since otherwise there is nothing to prove.8.e and hence f − g = 0 a. g ) = f − g . B . g ∈ L (µ) . f dµ = X X e−iθ f dµ = X Re e−iθ f dµ + i X Im e−iθ f dµ. f = g a. 2.e. The following are equivalent a) E f = E g for all E ∈ B b) |f − g | = 0 X 1 n} 1 ↑ {f = 0} and µ(|f | ≥ 1 n) < Remark 8. then 1. If A ∈ B and f ∈ L1 (µ.e.e. Warning: in the future we will often not make much of a distinction between L1 (µ) and L1 (µ) .e.7) Similar µ(Re(f − g ) < 0) = 0 so that Re(f − g ) = 0 a.. µ(|f | ≥ 1 )≤n n |f | dµ < ∞ X |Im f | dµ < ∞. µ) be a measure space and L1 (µ) = L1 (X. The set {f = 0} is σ – ﬁnite. 1A f dµ. i. Im(f − g ) = 0 a. 2. In the remainder of this section. This implies that 1E = 0 a. C) (prove!). Proposition 8.13.e. f ∼ g iﬀ f = g a. then f dµ ≤ X X |f | dµ. |Im f |) . job: prob macro: svmonob. Taking E = {Re(f − g ) > 0} and using 1E Re(f − g ) ≥ 0. B . for all n. C) or f : X → [0. modulo the equivalence relation. µ) for p ∈ [1. and into a metric space using ρ1 (f. let f dµ := A X for all E ∈ B . Let (X. which happens iﬀ µ ({Re(f − g ) > 0}) = µ(E ) = 0. Lemma 8. |f | dµ < ∞ iﬀ |Re f | dµ + For f ∈ L (µ. µ) denote the set of L1 (µ) functions modulo the equivalence relation. we learn that 0 = Re E (f − g )dµ = 1E Re(f − g ) =⇒ 1E Re(f − g ) = 0 a.e.19.

gn → g a. Then f ∈ L1 (µ) and f dµ = lim X h→∞ Proof. . E [1A1 . so that f ∈ L1 (µ) . . ϕn ] = E [ϕ1 ] . Yn ] = E [Y1 ] . and therefore lim sup n→∞ X Now apply the dominated convergence theorem. Moreover. Then use the approximation Theorem 6. n Theorem 8. then n=1 fn is convergent a. Suppose fn . 1An ] = E [1A1 ] . By multi-linearity it follows that fn dµ. (8. Then j =1 Zj is also integrable and n n n is a norm and (Lp (µ). to conclude n n n n fn dµ ≤ X f dµ ≤ lim inf n→∞ X X fn dµ. Suppose that (Ω. E j =1 n Zj = j =1 EZj . X 1 (In most typical applications of this theorem gn = g ∈ L (µ) for all n.) Proof. By deﬁnition. Corollary 8. E [Yn ] whenever Yi is σ (Zi ) – measurable and either Yi ≥ 0 or Yi is bounded. P ) is a probability space and {Zj }j =1 n are independent integrable random variables.36. fn → f a.34 (Dominated Convergence Theorem).34 to handle the general case. for K > 0.. then n n gdµ ± X X f dµ ≤ X gdµ + lim inf n→∞ X fn dµ − lim sup X fn dµ n→∞ E j =1 Zj 1|Zj |≤K = j =1 E Zj 1|Zj |≤K . Let {fn }n=1 ⊂ L1 (µ) be a sequence ∞ ∞ n=1 fn L1 (µ) < ∞.30 by ﬁrst proving Eq. Then as we have seen in a homework problem. {Zj }j =1 are independent iﬀ {σ (Zj )}j =1 are independent. By approximation by simple functions and the monotone and dominated convergence theorem. and {|Zj |}j =1 respectively.35. . B .cls date/time: 23-Feb-2007/15:20 . .e. E [1An ] when Ai ∈ σ (Zi ) for each i. · Lp ) is a Banach space in this norm.4. This shows that lim n→∞ X fn dµ exists and is equal to The dominating functions used here are ∞ |Zj | . . . let Zi = Zi 1|Zi |≤K . Notice that |f | = limn→∞ |fn | ≤ limn→∞ |gn | ≤ g a.34 along with the dominated convergence Theorem 8.76 8 Integration Theory We will see in later that 1/p f Lp = |f | dµ p for f ∈ Lp (µ) Proposition 8.e. Give another proof of Proposition 8. |fn | ≤ gn ∈ L1 (µ) .e. E [Y1 .7) with f being a simple function in which case the triangle inequality for complex numbers will do the trick. E [ϕn ] whenever ϕi are bounded σ (Zi ) – measurable simple functions. and ∞ ∞ fn X n=1 dµ = n=1 X fn dµ. By Fatou’s Lemma. Taking Yi = |Zi | then implies that n n X n→∞ lim inf (gn ± fn ) dµ ≤ lim inf n→∞ X (gn ± fn ) dµ = lim = X n→∞ gn dµ + lim inf ± X n→∞ X fn dµ gdµ + lim inf ± n→∞ X fn dµ so that n j =1 E j =1 |Zj | = j =1 E |Zj | < ∞ Since lim inf n→∞ (−an ) = − lim sup an . we have shown. gn . f dµ. (g ± f )dµ = X E [ϕ1 . Page: 76 job: prob macro: svmonob. such that Exercise 8. . n + 1 – times. g ∈ L1 (µ) . E j =1 Zj = lim E K →∞ j =1 Zj 1|Zj |≤K = j =1 n j =1 K →∞ lim E Zj 1|Zj |≤K = j =1 n EZj . n→∞ K Zj is integrable. . . . . it suﬃces to prove the theorem in the case where f is real. . and X gn dµ → X gdµ. By considering the real and imaginary parts of f separately.e.

|an | |x| dm(x) n ≤ n=0 |an | |β | n+1 + |α| n+1 n+1 ∞ ≤ 2r n=0 |an | rn < ∞ f (t. 4. x) ∂f (t. |SN | ≤ n=1 |fn | ≤ n=1 |fn | ∈ L (µ) .38 (Diﬀerentiation Under the Integral). x) for all x ∈ X = t − t0 ∂t |an | |x| dm(x) ≤ n=0 n=0 ∞ 0 n |β | |an | |x| dm(x) + 0 n |α| and by Eq. f (t0 . x)dµ(x). Suppose R > 0 and is a ∞ sequence of complex numbers such that n=0 |an | rn < ∞ for all r ∈ (0. x) exists for all (t. x) = lim n(f (t + n−1 . Corollary 8. x)| + |f (t0 . (8. x). x)dµ(x) is a diﬀerentiable function on J and X d dt Page: 77 ˙ (t0 ) = for all sequences tn ∈ J \ {t0 } such that tn → t0 . x) − f (t0 . x) − f (t0 . 2. x)dµ(x). Therefore. x → f (t. x)dµ(x). There is a function g ∈ L1 (µ) such that ∂f ∂t (t. x) − f (t0 . x)| ≤ g (x) |t − t0 | + |f (t0 . Then β α ∞ ∞ β ∞ ∞ {an }n=0 G(t) − G(t0 ) = t − t0 By assumption.36 since ∞ β α ∞ t→t0 f (t.37 (Integration of Power Series). ·) ∈ L1 (µ) for all t ∈ J. x) is measurable for each t ∈ J. 1 So by the dominated convergence theorem.cls ∂f (t0 . X |f (t. The condition L1 (µ) . x)| ≤ |f (t.e. ·) ∈ L1 (µ) for all t ∈ J (i. ·) ∈ L1 (µ) for some t0 ∈ J. Then f (t. By considering the real and imaginary parts of f separately. t → f (t. x)| dµ(x) < ∞).8. Indeed this follows from Corollary 8. x) − f (t0 . t − t0 Therefore. Hence then ∞ n=1 ∞ n=1 fn is almost N ∞ fn L1 (µ) < ∞ is equivalent to n=1 |fn | ∈ N everywhere convergent and if SN := n=1 fn . then Example 8. ∂t job: prob date/time: 23-Feb-2007/15:20 . ∂f ∂t (t. ∞ fn X n=1 dµ = X N →∞ N lim SN dµ = lim N →∞ ∞ SN dµ X |f (t. |f (t. x) − f (t0 . we may apply the dominated convergence theorem to conclude lim G(tn ) − G(t0 ) = lim n→∞ tn − t0 f (tn . By the mean value theorem. ∞ Proof. x)| ≤ g (x) |t − t0 | for all t ∈ J and hence (8. |α|). x) − f (t0 . we may assume that f is real. ·) n→∞ ≤ g for each t ∈ J. t − t0 an xn n=0 dm(x) = n=0 an α xn dm(x) = n=0 an β n+1 − αn+1 n+1 for all −R < α < β < R. ∂t macro: svmonob.8) = lim N →∞ fn dµ = n=1 X n=1 X fn dµ.8). for x → ∂f ∂t (t. x) dµ(x). x) = lim dµ(x) n →∞ tn − t0 X ∂f = (t0 . Suppose that J ⊂ R is an open interval and f : J × X → C is a function such that 1. 3. x) − f (t. Let G(t) := X f (t. x) is a sequential limit of measurable functions and hence is measurable for all t ∈ J. x)| . x)dµ(x) = X X ∂f (t. x) ≤ g (x) for all t ∈ J and x ∈ X. G G(t)−G(t0 ) limt→t0 exists and t−t0 ˙ (t0 ) = G X f (t. lim X f (t. x)) n→∞ ∂t and therefore. x) dµ(x) tn − t0 X f (tn . This shows f (t. x) − f (t0 . Also notice that ∂f (t.3 Integrals of Complex Valued Functions 77 Proof. R). x)dµ(x) X ∂t where r = max(|β | .

5). Hint: ﬁrst prove the relationship for characteristic functions.40.39. Exercise 8. ∞] be a measurable function. It then holds for positive simple functions. Show ν : M → [0. Solution to Exercise (8. Corollary 8. i. Finally for general f ∈ L+ .5 by writing dν = ρdµ. ϕn . Show ν is a measure. By what we have just proved. M. E must be independent of t. Page: 78 job: prob (8.41 It is customary to informally describe ν deﬁned in Exercise 8.38 and induction gives n!λ−n−1 = = [0. On the other hand g (t) = t so that g ˙ (t) = 1. 0 g ˙ (t) = d dt ∞ 0 so that f ∈ L1 (µ) iﬀ |f | ρ ∈ L1 (µ).38 to g (t) := ∞ 1x≤t dm(x).∞) 2.cls date/time: 23-Feb-2007/15:20 . 1.10) for all measurable functions g : Y → [0. then for simple functions.) 2. Show gdν = (g ◦ f ) dµ Y X ∂ The last integral is zero since ∂t 1x≤t = 0 unless t = x in which case it is not deﬁned. Remark 8. for all f : X → C we have Γ (t) := |f | dν = X X |f | ρdµ (The reader should check that Γ (t) < ∞ for all t > 0.5.∞) X n→∞ lim ϕn ρdµ = X f ρdµ. Then using MCT twice we ﬁnd f dν = lim n→∞ Using this fact. by linearity. 3. (8.4 Densities and Change of Variables Theorems Exercise 8. ∂t The complex case easily follows from this identity. 1x≤t dm(x) = 0 ? ∞ ∂ 1x≤t dm(x). set ν (A) := A ρdµ.e.8 that λ−1 = [0. Show that a measurable function f : X → C is in L1 (ν ) iﬀ |f | ρ ∈ L1 (µ) and if f ∈ L1 (ν ) then Eq. 1. Let (X. (We will write ν = f∗ µ or ν = µ ◦ f −1 .9) holds when f = 1A by deﬁnition of ν. ∞] is a measure. Let f : X → [0. f ρdµ. (8. such that 0 ≤ ϕn ↑ f. For λ ≥ 2ε > 0 and n ∈ N there exists Cn (ε) < ∞ such that 0≤ − d dλ n e−λx = xn e−λx ≤ C (ε)e−εx .∞) x e dm(x). Consider what happens if we formally apply Corollary 8.9) still holds. ∞] by ν (A) := µ(f −1 (A)) for all A ∈ F . and then for general positive measurable functions. macro: svmonob. For A ∈ M. µ) be a measure space. f dν = X X f+ dν − X f− dν = X f+ ρdµ − X f− ρdµ = X [f+ ρ − f− ρ] dµ = X f ρdµ. Deﬁne a function ν : F → [0. (Y. Let (X. M.∞) − d dλ n λ−1 = [0.) We have just shown that Γ (n + 1) = n! for all n ∈ N.9) Let ε > 0. (The reader should decide which hypothesis of Corollary 8. [0. ϕn dν = lim X X n→∞ ϕn ρdµ = X That is n! = λ n [0. f.16. ∞] be a measurable function.6. If f ∈ L1 (µ) and f is real. show f dν = X X e−λx dm(x) for all λ > 0. Hint: see the hint from Exercise 8.38 may be generalized by allowing the hypothesis to hold for x ∈ X \ E where E ∈ B is a ﬁxed null set. Notation 8. choose simple functions. Clearly Eq.5. The fact that ν is a measure follows easily from Corollary 8. Corollary 8.∞) − d dλ n e−λx dm(x) x e n −λx n −λx dm(x). Recall that xt−1 e−x dx for t > 0. Recall from Example 8. ∞].78 8 Integration Theory Example 8.38 has been violated in this example. µ) be a measure space and ρ : X → [0. F ) be a measurable space and f : X → Y be a measurable map.) 8. (8.

See. 1] happens to be C 1 -function. (8. (8. The key computation is to observe that if A ∈ F and g = 1A . Statistics 12. The statements involving complex functions follows as in the solution to Exercise 8. Math. Robert D. then ν ((a. Ann. that b P (X ≥ x) ≤ and1 1 1 1 −x2 /2 √ e x 2π (8. To verify Eq. Birnbaum) 62. b] with a.b] b F dm = a F (x)dx = F (b) − F (a). ∞]. Suppose that X is a standard normal random variable. b]) = F (b) − F (a) = a F (x) dx = (a. x]) = F (x) for all x ∈ R. P ) .b] F dm. Then use the uniqueness assertions in Exercise 5. This result is also valid for all h ∈ L1 (m).7. E [f (X )] = R f (x) F (x) dm (x) . This identity now extends to nonnegative simple functions by linearity and then to all non-negative measurable functions by MCT. (Reviewer: Z. By Exercise 8.43. 364–366. (1941). f (x) dF (x) (8.7.4 Densities and Change of Variables Theorems 79 3.12) it suﬃces to observe. 1f −1 (A) (x) = 1 iﬀ x ∈ f −1 (A) which happens iﬀ f (x) ∈ A and hence 1f −1 (A) (x) = 1A (f (x)) = g (f (x)) for all x ∈ X. Show a measurable function g : Y → C is in L1 (ν ) iﬀ g ◦ f ∈ L1 (µ) and that Eq. The fact that ν is a measure is a direct check which will be left to the reader. If X is a random variable on a probability space. then dµF (x) = F (x) dm (x) and Eq. To prove Eq. then gdν = Y Y Exercise 8. and F (x) := P (X ≤ x) . we ﬁnd h ◦ F · F dm = R R where dF (x) is shorthand for dµF (x) and µF is the unique probability measure on (R. F (b)]) = F (b) − F (a) while µ((a. (8. b])) = m((F (a). B .14) (g ◦ f ) dµ whenever g is a characteristic function. by the inverse function theorem that F −1 is a C 1 – function. Hint: Start by showing dν = F dm on sets of the form A = (a.6 with g = h ◦ F and f = F −1 .11) It follows that both µ = ν = µF – where µF is the measure described in Proposition 5. (Ω. h ◦ F · F dm = R R Moreover.12) h ◦ F dν = R −1 h ◦ F d F∗ m = R (h ◦ F ) ◦ F −1 dm = R hdm. by the fundamental theorem of calculus. Then E [f (X )] = R which is valid for all Borel measurable functions h : R → [0.15) µF ((a. Let dµ = F dm and A = (a. Values of Mills’ ratio of area to bounding ordinate and of the normal probability integral for large values of the argument. b ∈ R and a < b.5.1 to conclude dν = F dm on all of BR . Remark 8. b]. Solution to Exercise (8. Use this result to prove the change of variable formula. Show dν = F dm.6). Therefore we have gdν = Y X hdm (8. b]) = (a. (8.e.6 with g = h ◦ F and f = F −1 . Moreover if F : R → [0. b]) = m(F ((a. (Notice that F is strictly increasing so that F −1 : R → R exists and moreover.13) then 1 P (X ∈ A) = √ 2π e−x A 2 /2 dx for all A ∈ BR . for all A ∈ BR . Gordon. Solution to Exercise (8. Lemma 8.) Let m be Lebesgue measure on BR and ν (A) = m(F (A)) = m( F −1 −1 −1 (A)) = F∗ m (A) 1A dν = ν (A) = µ f −1 (A) = X 1f −1 (A) dµ. Let F : R → R be a C 1 -function such that F (x) > 0 for all x ∈ R and limx→±∞ F (x) = ±∞.cls date/time: 23-Feb-2007/15:20 .0X Page: 79 job: prob macro: svmonob.11) may be written as (8. BR ) such that µF ((−∞.8. (8.42. W.14) apply Exercise 8. From this equation we may deduce that µF (A) = A F dm for all A ∈ BR .7). i.10) holds for all g ∈ L1 (ν ).

16) Hence if p = 2 − δ. To prove Eq. Page: 80 job: prob P (X ≥ x) 2 x→∞ 1 √1 e−x /2 x 2π = 1. we ﬁnd α2 − 1 x2 = x2(−1+δ) + 2xδ ≤ 3xδ so that 1≥ P (X ≥ x) 1 √1 −x2 /2 x 2π e δ 1 1 − e−3x /2 1 + x−(2−δ) P (X ≥ x) = x 2 1 √ e−y /2 dy ≤ 2π ∞ x 1 1 −y2 /2 −∞ 1 y −y2 /2 √ e dy = − √ e |x x 2π 2π x ≥ from which Eq. standard normal random variables. α or equivalently.cls date/time: 23-Feb-2007/15:20 . (8. let α > 1. xp Xn = 1 a. it follows that lim inf Since Eq. n=1 Hence an application of Proposition 7. (8.. 2 ln n macro: svmonob. Let {Xn }n=1 be i. ∞ P (Xn ≥ αcn ) = ∞ if α < 1 n=1 and ∞ P (Xn ≥ αcn ) < ∞ if α > 1. Additional information: Suppose that we now take α = 1 + x−p = Then α2 − 1 x2 = x−2p + 2x−p x2 = x2−2p + 2x2−p .) It then follows that P (Xn ≥ αcn ) ∼ and therefore 1 α 2 ln (n) e−α 2 ln(n) = 1 α 2 ln (n) 1 n−α2 Since α > 1 was arbitrary. We begin by observing that ∞ P (X ≥ x) = 1. c2 n /2 = ln (n/C ) or cn = 2 ln (n) − 2 ln (C ). suppose that we take cn so that e−cn /2 = 2 ∞ αx ∞ 1 −α2 c2 n /2 .15) follows.s. (8. From this equation it follows that lim inf P (X ≥ x) 2 x→∞ 1 √1 e−x /2 x 2π ≥ 1 . Example 8.15) implies that P (X ≥ x) lim sup 1 1 −x2 /2 = 1 x→∞ x √ e 2π we are done.35 shows lim sup √ n→∞ 1 + xp . Then P (Xn ≥ αcn ) ∼ Now. αx 2π for x suﬃciently large.44. then P (X ≥ x) = 2 2 1 1 √ e−y /2 dy ≥ √ e−y /2 dy 2π 2π x x αx 1 1 −y2 /2 αx 1 y −y2 /2 √ e dy = − √ e |x ≥ 2π αx 2π αx x 2 2 2 1 1 =√ e−x /2 − e−α x /2 .i. e αcn Hence P (X ≥ x) ≥ 1 √1 −x2 /2 x 2π e 2 αx 1 √ e−y /2 dy x 2π 1 √1 −x2 /2 x 2π e C n 1 e−x /2 − e−α ≥ α e−x2 /2 2 2 x2 /2 = 2 2 1 1 − e−(α −1)x /2 .d.16). α (We now take C = 1. (8.80 8 Integration Theory lim 2 x→∞ 1 √1 e−x /2 x 2π Proof.

f (x) = g (x) for µ ¯ – a. Gπ = f (a)1{a} + 1 Mj 1(tj−1 . Take g (0) = 0. Therefore Eq. 1. we see from Proposition 8. If fn : X → R are measurable and f : X → R is a function such that limn→∞ fn = f. 1. x.17) ¯ .e. then g is measurable. g (1) = 1. ¯ is the comLemma 8. 1. B ) – measurable simple function ϕ ˜n ≥ 0 such that En := {ϕn = ϕ ˜n } has zero µ ¯ – measure.19) gdµ.45 that f is (M ¯ . Suppose ﬁrst that such a function g exists so that µ ¯(E ) = 0. Now 1E c f is measurable so g will be measurable if we show 1E g is measurable.18) is easily veriﬁed for non-negative M – measurable simple functions. g is measurable. B ) – measurable such E = {x : f (x) = g (x)} ∈ M ¯ (E ) = 0. Then g = 1E c f + 1E g since f = g on E c . Suppose that (X. If g : X → R is a function such that f (x) = g (x) for µ – a. 2.tj ] and Page: 81 job: prob macro: svmonob. f ∈ L1 (¯ µ) iﬀ g ∈ L1 (µ) and in which case f dµ ¯= X X Because µ ¯ = µ on M. For this consider. Since µ ¯ (∪n En ) ≤ n µ ¯ (En ) . B) – g is also (M ¯ . µ) is a complete measure space2 and f : X → R is measurable. Since ¯ .. µ) is a measure space and M ¯ pletion of M relative to µ and µ ¯ is the extension of µ to M. Moreover for such a pair f and g. (8.34 it holds for all M – measurable functions g : X → [0. we may choose Bk ∈ M such that Bk ⊂ Ak and µ with Ak ∈ M ¯(Ak \ Bk ) = 0. then g = 0 a. . Conversely if f is (M 2 Recall this means that if N ⊂ X is a set such that N ⊂ A ∈ M and µ(A) = 0. b] is a ﬁnite subset π ⊂ [a. Since g := 1E f = limn→∞ 1E c fn .45. then f is measurable as well. . Since f = g . there exists F ∈ M such that ∪n En ⊂ F and µ(F ) = 0.6 Comparison of the Lebesgue and the Riemann Integral 81 8. B . b] containing {a.e. n}. g (2) = 2. Then a function ¯ .e. B = {{0}. (8.. {1. (8. .e.46. b}. Proposition 8. by considering f± we may measurable. f is also measurable. X f dµ ¯ = X gdµ ¯ so to prove Eq. 2. Because f = g on E c and µ(E ) = 0. f = g a. The above results are in general false if (X. Suppose that (X. (1E g )−1 (A) = E c ∪ (1E g )−1 (A \ {0}) if 0 ∈ A (1E g )−1 (A) if 0 ∈ /A (8. yet g is not measurable. Then by the monotone convergence theorem and the approximation Theorem 6. let −∞ < a < b < ∞ and f : [a. The rest of the assertions follow in the standard way by considering (Re g )± and (Im g )± . let X = {0. 8. ∞]. Mj = sup{f (x) : tj ≤ x ≤ tj −1 }.18) Since (1E g )−1 (B ) ⊂ E if 0 ∈ / B and µ(E ) = 0.e. M. B ) – measurable. x. Let E = {x : lim fn (x) = f (x)} by assumption E ∈ B and n→∞ µ(E ) = 0.17) shows that 1E g is measurable. B = B R ) – measurable iﬀ there exists a function g : X → R f : X → R is (M ¯ and µ that is (M. A partition of [a. This shows that g = 1F f is (M. 2}. µ . then N ∈ M as well. Writing ϕn = ak 1Ak ¯ . To each partition π = {a = t 0 < t 1 < · · · < t n = b } (8.e. b] → R be a bounded function.5 Measurability on Complete Measure Spaces In this subsection we will discuss a couple of measurability results concerning completions of measure spaces.e. B ) – measurable. (8. it follow by completeness of B that (1E g )−1 (B ) ∈ B if 0 ∈ / B. i. Proof. ϕ} and µ = δ0 .cls date/time: 23-Feb-2007/15:20 . .a. B ) – measurable simple function ϕn ≥ 0 such assume that f ≥ 0. b] let mesh(π ) := max{|tj − tj −1 | : j = 1. µ ¯ – a.8. B ) – measurable and that {f = g } ⊂ F has µ ¯ – measure zero. µ) is not complete. Let E = {x : f (x) = g (x)} which is assumed to be in B and µ(E ) = 0. X. Choose (M that ϕn ↑ f as n → ∞. gπ = f (a)1{a} + 1 mj 1(tj−1 .6 Comparison of the Lebesgue and the Riemann Integral For the rest of this chapter. It now follows that 1F · ϕ ˜n = 1F · ϕn ↑ g := 1F f as n → ∞. For example. of [a.tj ] . 2}. so by part 1. mj = inf {f (x) : tj ≤ x ≤ tj −1 } n n Proof. B .18) it suﬃces to prove gdµ ¯= X X gdµ. Letting ϕ ˜n := ak 1Bk we have produced a (M. Eq.

The proof of the following Lemma is left to the reader as Exercise 8. b] → R satisfy: 1. Since H = G and h = g except possibly on the countable set π. (8.cls date/time: 23-Feb-2007/15:20 Page: 82 . y ∈ [a. b] → R be a bounded function. It is clear that h(x) ≤ f (x) ≤ H (x) for all x and H (x) = h(x) iﬀ lim f (y ) y →x f is deﬁned to be the common value: b f (x)dx = a a f (x)dx = a f (x)dx. H and h are Borel measurable. ∞ 2. b]} and y →x y →x ε↓0 f= a [a. The function f is Riemann integrable iﬀ and which case the Riemann integral b b b a = b a (Note π is a countable set. b]}.24) Sπk f ↓ a f and sπk f ↑ a f as k → ∞. The upper and lower Riemann integrals are deﬁned respectively by b a f (x)dx = inf Sπ f and a π b f (x)dx = sup sπ f. f ∈R Proof. b].21) and b for all x ∈ / π. (8.b] k→∞ gπk = lim sπk f = [a. (8. y ∈ [a. b]} ≥ Gk (x) for all k large enough. A similar argument shows that h(x) = g (x) if x ∈ / π and hence Eq.b] k→∞ Gπk = lim Sπk f = [a. Moreover. h : [a. x + ε]. (8.22) Theorem 8. Lemma 8. Again letting k → ∞ implies sup f (y ) ≥ G(x) and therefore.) 3. 1. If {πk }k=1 is any increasing sequence of partitions such that mesh(πk ) ↓ 0 and G and g are deﬁned as in Eq. y ∈ [a. (You justify this statement. let H (x) = lim sup f (y ) := lim sup{f (y ) : |y − x| ≤ ε. Gk (x) ≥ H (x) ≥ f (x) ≥ h(x) ≥ gk (x) ∀ k and letting k → ∞ in this equation implies G(x) ≥ H (x) ≥ f (x) ≥ h(x) ≥ g (x) ∀ x ∈ / π. The functions G and g are limits of measurable functions and hence measurable. sup{f (y ) : |y − x| ≤ ε. since eventually Gk (x) is the supremum of f (y ) over some interval contained in [x − ε. b] and π ⊂ π then Gπ ≥ Gπ ≥ f ≥ gπ ≥ gπ and Sπ f ≥ Sπ f ≥ sπ f ≥ sπ f.50.20).b] k→∞ a f (x)dx.23) Sπ f = a Gπ dm and sπ f = a gπ dm.48. There exists an increasing sequence of partitions {πk }k=1 such that mesh(πk ) ↓ 0 and b b ∞ exists and is equal to f (x). If π and π are two partitions of [a.b] k→∞ a f (x)dx (8. π b f a Deﬁnition 8. Combining this equation with Eq. (8. 2.) Gdm = lim [a. b] and h(x) = H (x) iﬀ f is continuous at x.b] hdm (8.24) then implies H (x) = G(x) if x ∈ / π. Let Gk := Gπk ↓ G and gk := gπk ↑ g. For x ∈ / π. b gdm = lim [a.25) h(x) = lim inf f (y ) := lim inf {f (y ) : |y − x| ≤ ε. h(x) ≤ f (x) ≤ H (x) for all x ∈ [a. given ε > 0 and x ∈ / π. 3.b] Hdm and a f= [a. ε↓0 and the following statements are equivalent: job: prob macro: svmonob. That is H (x) = h(x) iﬀ f is continuous at x.23) is proved.18. Then b b Notation 8.82 8 Integration Theory Sπ f = Notice that Mj (tj − tj −1 ) and sπ f = b b mj (tj − tj −1 ). If we let G := lim Gπk and g := lim gπk k→∞ k→∞ (8. The functions H.49 For x ∈ [a. that |y −x|≤ε (8. Lemma 8.47.20) H (x) = lim sup f (y ) ≥ G(x) y →x then by the dominated convergence theorem. both H and h are also Borel measurable.51. Let f : [a. then G(x) = H (x) ≥ f (x) ≥ h(x) = g (x) ∀x∈ / π := ∪∞ k=1 πk .

(8.b] [a. ∞] are measures on M for n ∈ N.a. b] as described in Lemma 8. Suppose {an }n=−∞ ⊂ C is a summable sequence (i. Give examples of measurable functions {fn } on R such that fn decreases to 0 uniformly yet fn dm = ∞ for all n.21) and (8. Eq. Moreover if we let m ¯ denote the completion of m. the remaining assertions including Eq. then f (θ ) := n=−∞ an e θ ∈ R and π 1 f (θ)e−inθ dθ. Suppose that {fn }n=1 is a sequence of non-negative measurable functions such that fn → f pointwise and n→∞ ∞ f (x)dx for Then lim fn = f < ∞.13 on p. Also suppose that µn (A) is increasing in n for all A ∈ M.o. C ) for all A.e.9 (From problem 12 on p. Prove that µ : M → [0.12. The conclusion need not hold if limn→∞ f. Suppose that µn : M → [0. (8. Now suppose that Λ is some index set and for each λ ∈ Λ. In light of these results and Eq. B ) = µ(A∆B ) where A∆B = (A \ B ) ∪ (B \ A) . (8. the set E := {x ∈ [a. b]. x. Since E = {x : H (x) = h(x)}.8. M.26) Proof.” fn = Exercise 8. B ) = 0 iﬀ A ∼ B. Let (X.10. A) . Deﬁne A ∼ B iﬀ µ(A∆B ) = 0 and notice that ρ (A.}) ≤ lim inf µ (An ) n→∞ ∞ Hdm = [a. µ) be a ﬁnite measure space and for A. f = lim 8.15. 1] such that gn → 0 while gn dm = 1 for all n.13 (Folland 2. f is L/B – measurable where L is the Lebesgue σ – algebra and B is the Borel σ – algebra on [a. Let M/ ∼ denote M modulo the equivalence relation. B ) is gives a well deﬁned metric on M/ ∼ .. ∼. ∞] by µ(A) = λ∈Λ µλ (A) for each A ∈ M.52 In view of this theorem we will often write b f dm.). If f is Riemann integrable then f is Lebesgue measurable3 . µλ : M → [0.23). Let {πk }k=1 be an increasing sequence of partitions of [a. Show that µ is also a measure. Exercise 8.e.8. B ∈ M let ρ(A. an = 2π −π macro: svmonob.b] f dm ¯ = [a.7 Exercises Exercise 8.7 Exercises 83 1. then b 2. 52. B ) = ρ (B. ∞ ∞ inθ is a continuous function for n=−∞ |an | < ∞). Page: 83 job: prob . C ) ≤ ρ (A. H (x) = h(x) for m -a. this last condition is equivalent to E being a m – null set. Show that ρ ¯ ([A] .b] hdm and if µ (∪m≥n Am ) < ∞ for some n.).14.48 and let G and g be deﬁned as in Lemma 8. Exercise 8. Let µ be a measure on an algebra A ⊂ 2X . ∞] is a measure on M.25) is a consequence of Eqs. B.a.46. Also give an example of a sequence of measurable functions {gn } on [0. It is clear that ρ (A.e. 4. Since m(π ) = 0. µ) be a measure space and {An }n=1 ⊂ M. Notation 8. (8. C ∈ M. B ) + ρ (B. a b a Exercise 8. M. (8. ρ satisﬁes the triangle inequality: ρ (A.b] ∞ a f (x)dx = [a. f is Riemann integrable. Hint: “Fatou times two. f is Riemann integrable iﬀ Hdm = [a.11. x. Deﬁne µ : M → [0. n→∞ and because h ≤ f ≤ H this happens iﬀ h(x) = H (x) for m . 2. Exercise 8.e. (8. i. 3. then µ({An i. Let (X. 27 of Folland.cls date/time: 23-Feb-2007/15:20 ∞ f need not be Borel measurable. 3 E n→∞ fn E for all measurable sets E ∈ M. H = G a. show µ({An a.b] hdm.}) ≥ lim sup µ (An ) . ∞] deﬁned by µ(A) := limn→∞ µn (A) is also a measure. B ∈ A. Exercise 8.e. and let [A] := {B ∈ M : B ∼ A} . From Eq.50. Show “∼ ” is an equivalence relation.25). Similarly show µ ˜ ([A]) = µ (A) is a well deﬁned function on M/ ∼ and show µ ˜ : (M/ ∼) → R+ is ρ ¯ – continuous. 3.22). [B ]) := ρ (A. b] : f is discontinuous at x} is an m ¯ – null set. then µ(A) + µ(B ) = µ(A ∪ B ) + µ(A ∩ B ) for all A. Show: 1. Exercise 8.26) are now consequences of Lemma 8.

x] f (t) dm (t) is continuous in x. Also ﬁnd a ﬁnite measure. Page: 84 job: prob macro: svmonob. µ.31e on p. let (Ω. n n 2 n! n=0 n=0 4 (n!) ∞ ∞ Exercise 8. (The answer in 2.d.18. Exercise 8. and n σ2 ≤ 2 nε = . If E |Xn | = E |X1 | < ∞ let µ := EXn – be the mean of Xn .31b and 2. P ) be a probability ∞ n space.19 (A simple form of the Weak Law of Large Numbers).48. ε 4 n2 Exercise 8.x] f (t) dµ (t) is not continuous.1 Laws of Large Numbers Exercises For the rest of the problems of this section. on BR such that x → (−∞. 60. Folland 2. if E |Xn | 2 Conclude from the last estimate and the ﬁrst Borel Cantelli Lemma 8. 8.17. = E |X1 | 2 < ∞. 4 Suppose now that E |X1 | < ∞. In part (e). 2 Assume E |X1 | < ∞. s should be taken to be a.13b is wrong by a factor of −1 and the sum is on k = 1 to ∞. B . {Xn }n=1 be a sequence if i. n 2 Sn −µ n Sn −µ >ε n σ2 . You may also freely use the Taylor series expansion (1 − z )−1/2 = (2n − 1)!! n (2n)! n z = 2 z for |z | < 1. let γ := E |Xn − µ| 4 .cls date/time: 23-Feb-2007/15:20 Sn = µ. Prove Lemma 8.20 (A simple form of the Strong Law of Large Numbers). Show for all ε > 0 and n ∈ N that Sn −µ n 4 E = = 1 nγ + 3n(n − 1)σ 4 n4 1 n−1 γ + 3 1 − n−1 σ 4 n2 and use this along with Chebyshev’s inequality to show P Sn −µ >ε n ≤ n−1 γ + 3 1 − n−1 σ 4 .16. random variables. let 2 = E Xn − µ2 – be the standard deviation of Xn σ 2 := E (Xn − µ) and if E |Xn | 4 2 < ∞.i.7. For any function f ∈ L1 (m) .s. show x ∈ R → (−∞.22 that n limn→∞ S n = µ a. Show E E P for all ε > 0 and n ∈ N. and Sn := k=1 Xk . Exercise 8.84 8 Integration Theory Exercise 8.

. consisting of ﬁnite intersections of sets of the form. {fn }n=1 ⊂ H. M is a subset of H which is closed under pointwise multiplication).. i=1 from with it follows that Fn ∈ H for all n. then A ∈ L because 1An ∈ H and 1An ↑ 1A ∈ H.e. Then H is closed under uniform convergence. We say that H is closed under bounded convergence if. and if An ∈ L with An ↑ A. pl (x) such that pl → ϕn uniformly on [−M. there exists M < ∞ such that 0 ≤ fn (ω ) ≤ M for all ω ∈ Ω and n ∈ N. . Let gn := fn − δn + M with δn and M constants to be determined shortly. Let ϕn (x) = 0 ∨ [(nx) ∧ 1] (see Figure 9. as well. Let f ∞ := supω∈Ω |f (ω )| . We then have Ω ∈ L since 1Ω = 1 ∈ H. it follows that f = f + M − M ∈ H.1 below) so that ϕn (x) ↑ 1x>0 . {fn }n=1 ⊂ H. Since H is a vector space containing the constant functions. {f > a} with f ∈ M and a ∈ R. then f ∈ H. Since. Proof. . f (ω ) := limn→∞ fn (ω ) exists for all ω ∈ Ω. ∞ {fn }n=1 ⊂ H with supn∈N supω∈Ω |fn (ω )| < ∞ and fn → f. Suppose that H is a vector subspace of bounded real valued functions from Ω to R which is closed under monotone convergence. if A. f2 .Since pl is a polynomial k it is easily seen that i=1 pl ◦ (fi − ai ) ∈ H. . i=1. Similarly we say that H is closed under monotone conver∞ gence if. M ] . satisfying: 1. Let ε > 0 be given. 2. .. Therei=1 {fi >ai } fore L contains the π – system. i. . k + δn − δn+1 . By passing to a subsequence if necessary. ak ∈ R. then H contains all bounded σ (M) – measurable functions. we may assume f − fn ∞ ≤ ε2−(n+1) . k pl ◦ (fi − ai ) → Fn uniformly as l → ∞. Clearly if H is closed under bounded convergence then it is also closed under monotone convergence. then f ∈ H. then δn − δn+1 = ε2−n (1 − 1/2) = ε2−(n+1) in which case gn+1 − gn ≥ 0 for all n. B ∈ L with A ⊂ B then B \ A ∈ L since 1B \A = 1B − 1A ∈ H. So we have shown that H is closed under uniform convergence. 2. .23. . then f := limn→∞ fn ∈ H. . Let Ω be a set. P . Let us ﬁrst assume that {fn }n=1 ⊂ H such that fn converges uniformly to a bounded function. Therefore L is λ – system. let k Fn := i=1 ϕn (fi − ai ) and let M := sup sup |fi (ω ) − ai | .. for every sequence. fk ∈ M and a1 . If M is multiplicative system (i. Fn ↑ Taking δn := ε2−n . . Moreover.1 Let Ω be a set and H be a subset of the bounded real valued functions on H.k ω By the Weierstrass approximation Theorem 4. Suppose that H is a vector subspace of bounded functions from Ω to R which contains the constant functions and is closed under monotone convergence. We then have gn+1 − gn = fn+1 − fn + δn − δn+1 ≥ −ε2 −(n+1) ∞ Theorem 9. we will also have gn ≥ 0 for all n. Proposition 9. By choosing M suﬃciently large.e.9 Functional Forms of the π – λ Theorem Notation 9.2. Given f1 . gn ∈ H and since gn ↑ f + M. for ∞ every sequence.3 (Dynkin’s Multiplicative System Theorem). f : Ω → R. 1{fi >ai } = 1∩k i=1 {fi >ai } i=1 it follows that 1∩k ∈ H or equivalently that ∩k i=1 {fi > ai } ∈ L. we may ﬁnd polynomial functions. there exists M < ∞ such that |fn (ω )| ≤ M for all ω ∈ Ω and n ∈ N. Proof. Let L := {A ⊂ Ω : 1A ∈ H} . satisfying: 1. fn (ω ) is increasing in n for all ω ∈ Ω.

Then µ = ν on σ (M) . on R which contains Cc (R. Since any positive σ (M) – measurable function may be written as a increasing limit of simple functions. the reader will easily verify that B is closed under complementation. Suppose µ and ν are two probability measure on (Ω. let Hf := {g ∈ H : gf ∈ H} . Proof. 9. Finally. R)) = BR . recall for p ∈ [1. Using the fact that H is an algebra containing constants. since any bounded σ (M) – measurable functions may be written as the diﬀerence of two such non-negative simple functions. Suppose H is a real subspace of bounded functions such that 1 ∈ H and H is closed under bounded convergence.e. Indeed.1. Proof.6. Let M = {1}∪{1A : A ∈ P} . it follows that B is closed under increasing unions and hence that B is σ – algebra.2 is no longer needed. Corollary 9. The smallest subspace of real valued functions. and is closed under bounded convergence. Since H is a vector space. apply Theorem 9. The corollary now follows by an application of The result now follows by an application of Theorem 9. H is now an algebra of functions. This is of course a direct consequence of Theorem 9. for −∞ < a < b < ∞. R) (the space of continuous functions on R with compact support) is the collection of bounded Borel measurable function on R.) For f ∈ H. (9. (As usual such a space exists by taking the intersection of all such spaces. Suppose that H is a vector subspace of bounded functions from Ω to R which contains the constant functions and is closed under bounded convergence.e. it follows that H contains all non-negative bounded σ (M) – measurable functions.b] may be written as a bounded limit of continuous functions with compact support from which it follows that σ (Cc (R.7. contains M.5. Example 9. µ) is 1/p p the set of measurable functions f : Ω → R such that f Lp := |f | dµ < ∞.cls date/time: 23-Feb-2007/15:20 . of bounded measurable functions on Ω. Moreover. macro: svmonob.1) for all f in a multiplicative subset. For fun. i. 1(a. In particular if M = {1} ∪ {1A : A ∈ P} with P being a multiplicative class we learn that µ = ν on σ (M) = σ (P ) . then H contains all bounded σ (P ) – measurable functions. let us give another self-contained proof of this corollary which does not even refer to the π – λ theorem. Using the fact that H is closed under bounded convergence.4.1) holds. It is also easy to see that 1 is a bounded limit of functions in Cc (R. R) and hence 1 ∈ H.3. i. Moreover if f ∈ M. Proof. Having proved this it now follows for any f ∈ H that M ⊂ Hf and therefore f g ∈ H whenever f. Plots of ϕ1 . and contains Ω. L contains σ (P ) = σ (M) . It is easy to see that λf p = |λ| f p for all λ ∈ R and we will show below that f + g p ≤ f p + g p for all f.3. under the assumptions here.3 simpliﬁes in that Proposition 9.3 with H being the bounded measurable functions on Ω such that Eq. we will assume that H is the smallest subspace of bounded functions on Ω which contains the constant functions.3 with M := Cc (R. and Hf is closed under bounded convergence. Corollary 9. We will now show that B := {A ⊂ Ω : 1A ∈ H} is σ – algebra.86 9 Functional Forms of the π – λ Theorem algebra. g ∈ H. M. Corollary 9. By a homework problem. f g ∈ H for all f ∈ M and g ∈ H. then H contains all bounded σ (M) – measurable functions.e. H contains all B – measurable simple functions. Since every bounded B – measurable function may be written as a bounded limit of such simple functions. H = Hf . H. then M ⊂ Hf and so by the deﬁnition of H. ﬁnite intersections. For the rest of this chapter. g ∈ Lp (µ) . B ) such that f dµ = Ω Ω Fig. the proof of Theorem 9. If P ⊂ 2Ω is a multiplicative class such that 1A ∈ H for all A ∈ P .3. it follows that H contains all bounded σ (M) – measurable functions. ∞) that Lp (µ) = Lp (X. The proof is now completed by showing B contains σ (M) as was done in second paragraph of the proof of Theorem 9. ϕ2 and ϕ3 . · p satisﬁes the triangle inequality. B . If M is a subset of H which is closed under pointwise multiplication. The reader will now easily verify that Hf is a linear subspace of H. 1 ∈ Hf . it follows that H contains all bounded B – measurable functions. B is an Page: 86 job: prob f dν (9. In this proof.e. Then M ⊂ H is a multiplicative system and the proof is completed with an application of Theorem 9. In particular it follows that 1A ∈ H for all A ∈ σ (M) . i. R). i. As a consequence of the above paragraphs and the π – λ theorem.

11 (Separability of Lp – Spaces). Fix k ∈ N for the moment and let H denote those bounded B – ∞ measurable functions. f : Ω → R. M is dense in Lp (µ. It is left to reader to check D is dense in S(A. k The same line of reasoning used in Eq. i.2) now implies limk→∞ f − ϕk Lp (µ) = 0. To see this. the proof is then complete since S(A. To verify the latter assertion. from a set. p (Take the dominating function to be g = [2C |f |] where C is a bound on all of the |ψk | . there exist ϕn ∈ M such that limn→∞ f − ϕn Lp (µ) = 0. R) and ψk := 1[−k. More explicitly a bounded function. Page: 87 job: prob . Then take h (x.9 Functional Forms of the π – λ Theorem 87 Theorem 9. 1 ∈ H. Exercise 9. Notation 9.cls date/time: 23-Feb-2007/15:20 ≤ lim sup ψk (f − fn ) n→∞ n→∞ Lp (µ) Lp (µ) + lim sup ψk fn − ϕn =0 (9. p ∈ [1.k] . The theorem now follows from Theorem 9. M. Cc (R. for which there exists {ϕn }n=1 ⊂ M such that limn→∞ ψk f − ϕn Lp (µ) = 0. Theorem 9. M ⊂ H and H is closed under bounded convergence. R) and σ (M) = B . (9. Let M := S(A. There exists ψk ∈ M such that ψk → 1 boundedly. h : R2 → R such that E [h (X. there exists ϕk ∈ M such that 1 ψk 1{|f |≤k} f − ϕk Lp (µ) ≤ . R) . let M↑ (M↓ ) denote the the bounded monotone increasing (decreasing) limits of functions from M. Proof. limn→∞ ψk (f − fn ) Lp (µ) = 0. ∞). BR ) such that µ ([−M. i.e. Then. The dominated convergence theorem implies limk→∞ ψk 1{|f |≤k} f − f Lp (µ) = 0. Proof. If A ∈ A. µ) be a measure space and M be an algebra of bounded R – valued measurable functions such that 1. B . Once this is done.8 (Density Theorem). Use Corollary 9. Let f ∈ Lp (µ) be given. A ⊂ B ⊂ 2Ω is an algebra such that σ (A) = B and µ is σ – ﬁnite on A.10. X )] . Hint: Let H denote the bounded Borel measurable functions. suppose fn ∈ H and fn → f boundedly. by the dominated convergence theorem. now shows H contains all bounded measurable functions on Ω. f.10. Let (Ω. then Ωk ∩ A ∈ A and µ (Ωk ∩ A) < ∞ so that 1Ωk ∩A ∈ M. µ) is a dense subspace of Lp (µ) by Theorem 9. R) (the space of continuous functions on R with compact support) is dense in Lp (µ) for all 1 ≤ p < ∞. Then Lp (µ) is separable and D={ aj 1Aj : aj ∈ Q + iQ.1 (Take the dominating function to be g = p ∞ [2C |ψk |] where C is a constant bounding all of the {|fn |}n=1 .13 (Bounded Approximation Theorem). R) . Let (Ω. Therefore 1A = limk→∞ 1Ωk ∩A is σ (M) – measurable for every A ∈ A. Then S(A. ∞).8 with M = Cc (R. y ) = 1{x=y} .1. Show P (X = Y ) = 1. 1 It is at this point that the proof would break down if p = ∞. B . Then to every function f ∈ Lp (µ. Aj ∈ A with µ(Aj ) < ∞} is a countable dense subset.8 after observing ψk := 1Ωk ∈ M and ψk → 1 boundedly. µ) relative to the Lp (µ) – norm. σ (M) = B . g : R → R. Let p ∈ [1.4 to show H is the vector space of all bounded Borel measurable functions. Let µ be a measure on (R. µ) be a ﬁnite measure space and M be an algebra of bounded R – valued measurable functions such that: macro: svmonob. Suppose.12 Given a collection of bounded functions. By assumption there exists Ωk ∈ A such that µ(Ωk ) < ∞ and Ωk ↑ Ω as k → ∞. So we have shown that A ⊂ σ (M) ⊂ B and therefore B = σ (A) ⊂ σ (M) ⊂ B . Then. Example 9. to R. Ω. An application of Dynkin’s Multiplicative System Theorem 9. µ) denote the measurable simple functions. µ) is dense subspace of Lp (µ). (Ω. B . P ) be a probability space and X. 2. M ]) < ∞ for all M < ∞. Y : Ω → R be a pair of random variables such that E [f (X ) g (Y )] = E [f (X ) g (X )] for every pair of bounded measurable functions. apply Theorem 9.9. Y )] = E [h (X. f : Ω → R is in M↑ respectively M↓ iﬀ there exists fn ∈ M such that fn ↑ f respectively fn ↓ f. A ⊂ B is a countable algebra such that σ (A) = B and µ is σ – ﬁnite on A. Theorem 9. µ). M ⊂ Lp (µ. Suppose p ∈ [1.) Using this and what we have just proved.2) which implies f ∈ H.) We may now 1 choose ϕn ∈ M such that ϕn − ψk fn Lp (µ) ≤ n then lim sup ψk f − ϕn n→∞ Lp (µ) Theorem 9. Proof. A routine check shows H is a subspace of the bounded measurable R – valued functions on Ω. Let S(A. ∞). ϕ : Ω → R such {ϕ = y } ∈ A for all y ∈ R and µ ({ϕ = 0}) < ∞.e.3.

M is a “lattice” – if f. Let ∞ N Then for every bounded σ (M) measurable function. M↓ −h ≤ −g ≤ −f ∈ M↑ and µ (−f − (−h)) = µ (h − f ) < ε.i ∈ M such that fn. By observation (1). Moreover it is clear that gn ≤ max {fk : k ≤ n} = fn ≤ f and hence gn ↑ g := limn→∞ gn ≤ f. then λf ∈ M↑ (λf ∈ M↓ ) . So if ε > 0 is given. As similar argument shows M↓ ⊂ H. for this N. Then ∞ N δn = lim n=1 N →∞ δn = lim (gN +1 − g1 ) = g − g1 n=1 N →∞ and uN ≤ g − g1 ≤ v. For example.cls date/time: 23-Feb-2007/15:20 . macro: svmonob. |f | ∈ M for all f ∈ M. H is closed under scalar multiplication. we may conclude that g = (g − g1 ) + g1 ∈ H. we may further assume that un ≥ 0. there exists fn ∈ M ⊂ M↓ such that fn ↑ f.3. Clearly. M ⊂ H and in fact it is also easy to see that M↑ and M↓ are contained in H as well. we can ﬁnd. it suﬃces to show H is closed under monotone convergence. and ε > 0 is given. j ≤ n} ∈ M. if follows that g − g1 ∈ H. So suppose that gn ∈ H and gn ↑ g. By replacing un by un ∨ 0 ∈ M↓ (by observation 1. 4. j. and 3. we may ﬁnd fi ∈ M↓ and hi ∈ M↑ such that fi ≤ gi ≤ hi and µ (hi − fi ) < ε/2 for i = 1. ∞ it follows that for N ∈ N suﬃciently large that n=N +1 µ (vn ) < ε.) n=1 2. f := f1 + f2 ∈ M↓ . by deﬁnition. we have µ v − uN < 2ε and since ε > 0 is arbitrary. Then for λ ≥ 0. We will now show H is a vector sub-space of the bounded B = σ (M) – measurable functions. Moreover.i ↑ fn as i → ∞. If f ∈ M↑ then −f ∈ M↓ and visa versa. 2. g ∈ M then s. Since fij ≤ g for all i. s to of 1. g ∈ M↑ or f. there exists f ∈ M↓ and h ∈ M↑ such that f ≤ g ≤ h and µ (h − f ) < ε. gn := max {fij : i. M↓ un ≤ δn ≤ vn ∈ M↑ such that µ (vn − un ) ≤ 2−n ε for all n. and µ (h − f ) = µ (h1 − f1 ) + µ (h2 − f2 ) < ε. N ∞ N ∞ Now let H denote the collection of bounded measurable functions which satisfy the assertion of the theorem. 2. if f ∈ M↑ . let N uN := n=1 un ∈ M↓ (using observation 2). Indeed suppose that ε > 0 is given and f ∈ M↓ and h ∈ M↑ such that f ≤ g ≤ h and µ (h − f ) < ε. Indeed. if follows that λg ∈ H for λ ≥ 0. and 5. If gi ∈ H for i = 1. Page: 88 job: prob µ v − uN = n=1 µ (vn − un ) + n=N +1 ∞ µ (vn ) ≤ n=1 ε2−n + n=N +1 µ (vn ) ≤ε+ n=N +1 µ (vn ) . 3. If λ ≥ 0 and f ∈ M↑ (f ∈ M↓ ). and every niel ε > 0. So we have shown that gn ↑ f ∈ M↑ . Therefore. If g ∈ H then λg ∈ H for all λ ∈ R. since ∞ ∞ ∞ µ (vn ) ≤ n=1 n=1 ∞ µ δn + ε2−n = n=1 µ (δn ) + εµ (Ω ) = n=1 µ (g − g1 ) + εµ (Ω ) < ∞. which shows −g ∈ H as well. Because of Theorem 9. to complete this proof. Let us begin with a few simple observations. If fn ∈ M↑ and fn ↑ f where f : Ω → R is a bounded function. However. Since g1 ∈ H and H is a vector space. 2. it follows that g1 + g2 ∈ H.88 9 Functional Forms of the π – λ Theorem 1. σ (M) = B . g ∈ M↓ then f + g ∈ M↑ or f + g ∈ M↓ respectively. Since H is a vector space. it follows that f ∈ H.). then f ∈ M↑ . M↓ λf ≤ λg ≤ λh ∈ M↑ and µ (λh − λf ) = λµ (h − f ) < λε. Similarly. it follows that 0 ≤ δn := gn+1 − gn ∈ H for all n ∈ N. and for N ∈ N. and v := n=1 vn =↑ lim N →∞ vn ∈ M↑ (using observations 2. g : Ω → R. Since M↓ fn ≤ f ≤ f ∈ M↑ and µ (f − fn ) → 0 by the dominated convergence theorem. 2 If f. H is closed under addition. Since ε > 0 was arbitrary. where g : Ω → R is a bounded function. Since h = h1 + h2 ∈ M↑ . 1 f ∨ g = (f + g + |f − g |) ∈ M 2 1 f ∧ g = (f + g − |f − g |) ∈ M. it follows that fn = limj →∞ fnj ≤ g and consequently that f = limn→∞ fn ≤ g ≤ f. 1 ∈ M. in Proof. f ≤ g1 + g2 ≤ h. 5. by assumption there exists fn.

the functions Re f = 1 2 f + f and Im f = 2i f − f are in H or R R R M0 respectively.14 (Complex Multiplicative System Theorem). M0 is closed under complex conjugation and σ (M0 ) = σ (M) . M0 is a multiplicative system (as the reader should check) and therefore by Theorem 9. Let HR := {f ∈ H : f is real valued} and MR 0 := {f ∈ M0 : f is real valued} . Let M0 = spanC (M ∪ {1}) be the complex span of M. 1 ∈ H.Theorem 9.3. M0 ⊂ H. M0 is an algebra. If M ⊂ H is multiplicative system which is closed under conjugation. then f = Re f + i Im f ∈ H since Re f and Im f are in HR . and H is closed under bounded convergence. . Proof. Since H and M0 are complex linear spaces closed under complex conjugation. HR contains all bounded σ MR 0 – measurable real valued functions. Then HR is a real linear space of bounded real valued functions 1 which is closed R R under bounded convergence and MR 0 ⊂ H . and H = HR + iHR . Suppose H is a complex linear subspace of the bounded complex functions on Ω. H is closed under complex conjugation. As the reader should verify. for any f ∈ H or 1 ¯ ¯ f ∈ M0 . then H contains all bounded complex valued σ (M)-measurable functions. Therefore M0 = M0 + iM0 . σ M0 = σ (M0 ) = σ (M) . Moreover. Hence if f : Ω → C is a bounded σ (M) – measurable function.

.

y ) = f (x)g (y ). y )dν (y ) is M – B[0. the iterated integrals of f (when they make sense) are: and dµ(x) X Y x→ Y f (x.3) (10.1) – (10. y ) = f (x) and G(x. y ) = ν (B )µ(A). f (x. let f ⊗ g denote the function on X × Y given by f ⊗ g (x.5) hold. For the moment let us now further assume that µ(X ) < ∞ and ν (Y ) < ∞ and let H be the collection of all bounded (M ⊗ N . (10. Suppose (X.10 Multiple and Iterated Integrals 10. Then dµ(x)f (x.1 (Iterated Integrals) If (X. f (x. Moreover f (x. y ) := X Y f (x. M. µ) and (Y. (10. for each x ∈ X. BC ) – measurable. y ) = ν (B )µ(A) Theorem 10.3. ν ) are two measure spaces and f : X × Y → C is a M ⊗ N – measurable function. This shows that F is the composition of measurable functions and hence measurable. E . Similarly one shows that G is measurable.2 Suppose that f : X → C and g : Y → C are functions.∞] measurable. (10. f (x. (10. it follows by Corollary 9. y ) = Y dν (y ) X dµ(x)f (x. Notice that if f. Now F = f ◦ π1 where π1 : X × Y → X is the projection map.5) hold in this case as well. M.3) holds and we have dµ(x) X Y dν (y )f (x.2) hold.6) Similarly. To prove this let F (x.5 that H is the space .1) from which it follows that Eqs.5) and dν (y ) Y X Proof. y ) = 1A (x)1B (y ) and one sees that Eqs.2) (10. (10. (10.4) y→ dν (y )f (x. y ) = 1A×B (x. X (10. y ) = g (y ) so that f ⊗ g = F · G will be measurable provided that F and G are measurable. Using the fact that measurable functions are closed under pointwise limits and the dominated convergence theorem (the dominating function always being a constant).2 Tonelli’s Theorem and Product Measure Y dν (y ) X dµ(x)f (x.4) and (10. y ) := Y X f (x. x → f (x. y ). Suppose that E = A × B ∈ E := M × N and f = 1E . µ) and (Y. y )dµ(x) = µ(A)1B (y ) and X 10. N .1 Iterated Integrals Notation 10. then for each y ∈ Y. N . y )dν (y ) = Y Y Notation 10. y )dµ(x) dν (y ). BR ) – measurable function.1) and (10.∞] measurable. y ) is M – B[0.∞] measurable. ν ) are σ -ﬁnite measure spaces and f is a nonnegative (M ⊗ N . then f ⊗ g is (M ⊗ N .∞] measurable. so that Eq. y ) is N – B[0. y → f (x. (10. 1A (x)1B (y )dν (y ) = 1A (x)ν (B ). y )dν (y ) dµ(x) dµ(x) X Y dν (y )f (x. BR ) – measurable functions on X × Y such that Eqs. Since we have just veriﬁed that 1E ∈ H for all E in the π – class. g are measurable. one easily shows that H closed under bounded convergence. y )dµ(x) is N – B[0.

From the monotone convergence theorem.1) – (10.7. the theorem holds when f = 1E with E ∈ M ⊗ N . BR ¯ ) – measurable functions. ·)dµ(x) ∈ L+ (Y. Then there exists a unique measure π on M ⊗ N such that π (A × B ) = µ(A)ν (B ) for all A ∈ M and B ∈ N . Alternatively. Then use the change of variables theorem from undergraduate analysis. f (·. The validity of Eqs. Yn ∈ N such that Xn ↑ X. that 2 R v 1Yn dµ ↑ Y vdµ as n → ∞.6). Proof. ∞] is a (M ⊗ N . y ) dµ(x)f (x.8) (10. (10. n ∈ N. then f (·. Moreover this measure satisﬁes π (A × B ) = µ(A)ν (B ) for all A ∈ M and B ∈ N from Eq.4. µ) and (Y. In this example we are going to show. X (10.5) then follows by passing to the limits m → ∞ and then n → ∞ making use of the monotone convergence theorem in the following context. Y X f (x. let Xn ∈ M and Yn ∈ N be chosen so that µ(Xn ) < ∞. N ) and f dπ = X ×Y X dµ(x) Y dν (y )f (x. the theorem is also true for non-negative simple functions.92 10 Multiple and Iterated Integrals of all bounded (M ⊗ N . By Theorem 10. f : X × Y → [0.1) – (10. (10. vdµn = Y Y Proof. BR ) – measurable functions on X × Y. Example 10. ν ) are σ – ﬁnite measure spaces and π = µ ⊗ ν is the product measure on M ⊗ N . Moreover π is given by π (E ) = X I2 = R e−x 2 /2 dm (x) = R e−y 2 /2 R e−x 2 /2 dm (x) dm (y ) = R2 e−(x 2 +y 2 )/2 dm2 (x.34. Using the change of variables theorem described in Section 10. using Tonelli’s theorem. if f : X × Y → [0. (10.R)×(0. By what we have just proved Eqs. Suppose (X. choose Xn ∈ M. Notation 10. Then Eqs. M. Suppose (X.7) is a measure on M ⊗ N . µ) and (Y. one deduces the theorem for general f ∈ L+ (X × Y. that π deﬁned in Eq. N ). it suﬃces to observe. f (x. N ) for all x ∈ X. y ) where DR = (x. y )dν (y ) ∈ L+ (X. Repeated use of the monotone convergence theorem allows us to pass to the limit M → ∞ in these equations to deduce the theorem in the case µ and ν are ﬁnite measures. Using the linearity of all of the statements. I 2 = lim R→∞ for all E ∈ M ⊗ N and π is σ – ﬁnite. M ⊗ N ). Indeed. BR2 = BR ⊗ BR . ·) ∈ L+ (Y. Yn ↑ Y. M) for all y ∈ Y. BR ¯ ) – measurable function.5 and 5. M. y ) ∈ L+ (X. e−x 2 /2 dm (x) = Corollary 10.1 we ﬁnd e−(x DR 2 +y 2 )/2 dπ (x. y ) dµ(x) Y dν (y )1E (x. y ) : x2 + y 2 < R2 . ν (Yn ) < ∞. y ). µ(Xn ) < ∞ and ν (Yn ) < ∞ for all m. M). R macro: svmonob.7) where m2 = m ⊗ m is “Lebesgue measure” on R2 . For the σ – ﬁnite case. ν ) are σ – ﬁnite measure spaces. I := √ 2π. ∞].6 (Tonelli’s Theorem). M ⊗ N ).5 below. y ) = (0.1 Proposition 4. For all u ∈ L+ (X.3 and Corollary 10.26 with E = M × N . let fM = M ∧ f so that fM ↑ f as M → ∞. Xn ↑ X and Yn ↑ Y. To this end we observe.5) with µ replaced by µm and ν by νn for all (M ⊗ N . N .cls date/time: 23-Feb-2007/15:20 . M). N . (10. If f ∈ L+ (X × Y.1) – (10.9) = Y dν (y ) u1Xm dµ ↑ X udµ as m → ∞.5 The measure π is called the product measure of µ and ν and will be denoted by µ ⊗ ν.2π ) R e−r e−r 0 2 2 /2 rdrdθ 2 = 2π 1 /2 rdr = 2π 1 − e−R /2 . Then deﬁne µm (A) = µ(Xm ∩ A) and νn (B ) = ν (Yn ∩ B ) for all A ∈ M and B ∈ N or equivalently dµm = 1Xm dµ and dνn = 1Yn dν.4. you can easily show that the integral D f dm2 agrees with the R multiple integral in undergraduate analysis when f is continuous. Moreover.5) hold with f replaced by fM for all M ∈ N. The uniqueness assertion is a consequence of the combination of Exercises 4. y ) = Y dν (y ) X dµ(x)1E (x. udµm = X X Theorem 10. and for all and v ∈ L+ (Y. Notice that any measure π such that π (A × B ) = µ(A)ν (B ) for all A ∈ M and B ∈ N is necessarily σ – ﬁnite. Then using the monotone convergence theorem repeatedly along with the approximation Theorem 6. then Xn × Yn ∈ M ⊗ N . using the monotone convergence theorem. For the existence. y ) (10. Page: 92 job: prob e−(x DR 2 +y 2 )/2 dπ (x. Xn × Yn ↑ X × Y and π (Xn × Yn ) < ∞ for all n. (10.

then f (x. y ) is a positive M ⊗ N – measurable function. f (x. y ) − Y dν (y ) X dµ (x) f− (x. y ) − f− (x. which shows Y f (·. y )dv (y ) ∈ L1 (µ). = 1E c (x) f− (x.3 Fubini’s Theorem 93 From this we learn that I 2 = lim 2π 1 − e−R R→∞ 2 f (x.12) is a direct consequence of Tonelli’s Theorem 10. y )| dν (y ) dµ (x) = X Y X ×Y = Y 1E c (x) u (x. x → Y |f (x. The previous theorems have obvious generalizations to products of any ﬁnite number of σ – ﬁnite measure spaces. y ) X |f (x. y ) − Y dν (y ) |f (x.9) are still valid.e.8) still holds by a computation similar to Y that done in Eq. y ) dν (y ) Y = 2π Y = Y 1E c (x) [f+ (x.e. y ) dν (y ) = Y Y |f (x. (10.6.e.15) X ×Y If any one (and hence all) of these condition hold.10) – (10. macro: svmonob. Theorem 10. y )| dµ(x) dν (y ) < ∞. ν ) are σ – ﬁnite measure spaces.e. (10. being the diﬀerence of two measurable functions. f ∈ L1 (π ). Moreover Tonelli’s theorem implies |f (x. y. y ) dν (y ) dµ (x) ≤ X Y X Y |f (x.e. f : X → [0. y ) dν (y ) Y Then by Tonelli’s theorem. y )| dν (y ) = ∞ . Y f (·. (10. Integrating Eq. (10. X f (x.14) 10. µ) and (Y. Now suppose that f = u + iv is complex valued and again let E be as in Eq. ·) ∈ L1 (ν ) for µ-a. ∞]) is a non-integrable function we will still write X f dµ = ∞. f (x. y ) f dπ (10. y )dv (y ) ∈ L1 (µ). y ) − X dµ (x) Y dν (y ) 1E c (x) f− (x. by convention we will deﬁne X f dµ := 0.cls date/time: 23-Feb-2007/15:20 . y ) dν (y ) ∈ L1 (µ) and Eq. it follows from another application of Tonelli’s theorem that x → Y f (x. Similarly one shows f (·. Then the following three conditions are equivalent: |f | dπ < ∞. π = µ ⊗ ν is the product measure on M ⊗ N and f : X × Y → C is a M ⊗ N – measurable function. y ) dν (y ) − Y Y as desired.13) 1E c (x) f (x. y ) dν (y ) = /2 1E c (x) f (x. x.8 (Fubini’s Theorem). y ) = (1E c ⊗ 1Y · f± ) (x. Suppose (X.11) (10. (10. X |f | dµ = ∞. y ) dµ (x) 1E c (x) f− (x. However if f is a non-negative function (i. (10.3 Fubini’s Theorem The following convention will be in force for the rest of this section. y ) dν (y ) = Y 1E c (x) [u (x. y )| dν (y ) dµ(x) < ∞ and X Y = Y dν (y ) X dµ (x) 1E c (x) f+ (x. (10.10) (10. y )] dν (y ) 1E c (x) f+ (x. y ) dν (y ) + i |f | dπ < ∞ which implies that µ (E ) = 0. Proof. y )| dν (y ) dµ (x) < ∞. M. y )| dν (y ) is measurable and hence E ∈ M. y ) dν (y ) .8) holds. y ) dν (y ) dµ (x) X Y (10.10. i. i. For example the following theorem holds. y ) + iv (x.13). (10. Convention: If (X. The equivalence of Eqs.12) = X dµ (x) Y dν (y ) 1E c (x) f+ (x. M. By our convention. y )] dν (y ) 1E c (x) v (x.8) and (10. Y X = Y dν (y ) X dµ (x) f+ (x. Just as above we still have E ∈ M and µ (E ) = 0.15). X ×Y Noting that 1E c (x) f± (x.9) may be proved in the same way. Moreover f (x. y ) dν (y ) is M – measurable. then using the above convention we have Page: 93 job: prob which is measurable in x by what we have just proved. f (·.14) on x and using Tonelli’s theorem repeatedly implies. µ) is a measure space and f : X → C is a measurable but non-integrable function. Now suppose f ∈ L1 (π ) is a real valued function and let E := x∈X: Y = X ×Y f+ dπ − X ×Y f− dπ = X ×Y (f+ − f− ) dπ = which proves Eq. N . The assertions pertaining to Eq. (10. y ) ∈ L1 (µ) for ν -a. ·)dµ(x) ∈ L1 (ν ) and Eqs. (10. Let f± be the positive and negative parts of f.

. dµn (xn ) = (µ1 ⊗ · · · ⊗ µn ) (A) .4. xn ) dµ1 (x1 ) . . f ∈ L1 (π ) iﬀ dµσ(1) (xσ(1) ) . .16) Therefore. we have n = R dν (y ) R dµ (x) 1{x=y} = = R 0 dν (y ) = 0. M1 ⊗ · · · ⊗ Mn ) such that π (A1 × · · · × An ) = µ1 (A1 ) . f : (Rn . xn ) dµ1 (x1 ) . . B .17) to conclude that π (A) = P ((X1 . . . . . In this example we will show M M →∞ lim 0 sin x dx = π/2.94 10 Multiple and Iterated Integrals n Theorem 10. one can use the theorems already proved and induction on n. x (10. . . . 2. µ ({y }) = F (y ) − F (y −) = 0. on (X. . . see Exercise 10. .11 (No Ties). P ) and µk = P ◦ Xk is the distribution for Xk for −1 k = 1. . . B . . . Mi . . (taken in any order) (10. f : (Rn . . . . µi )}i=1 are σ – ﬁnite measure spaces and X := X1 × · · · × Xn . Alternatively. Suppose {(Xi . . Then it is easily checked that H is a vector space which contains the constant functions and is closed under bounded convergence. . . . π. Xn ) = 1A (x1 . . . . {Xk }k=1 are independent. y ) µ ({y }) dν (y ) R and 3.12. H contains all bounded σ (M) = BRn – measurable functions. . . . . BR ) . Proof. Then there exists a unique measure. let µ (A) := P (X ∈ A) and ν (A) = P (Y ∈ A) . Ef (X1 . H contains the multiplicative system. Suppose that X and Y are independent random variables on a probability space (Ω. for all bounded measurable functions. Xσ(n) dµσ(n) (xσ(n) ) f (x1 . Because F is continuous. . . σ. Xn ) . BR ) such that Eq. if f = 1A1 ×···×An where Ai ∈ BR . . xn ) dµ1 (x1 ) . Suppose that {Xk }k=1 are random variables on a prob−1 ability space (Ω. . . Xn ) = Rn n n P ((X1 . . . dµn (xn ) .17) holds. then P (X = Y ) = 0. . BRn ) → (R. This theorem can be proved by the same methods as in the two factor case.) If f : X → [0. Ef (X1 . Xn ) ∈ A1 × · · · × An ) = π (A1 × · · · × An ) = j =1 n µj (Aj ) = j =1 P (Xj ∈ Aj ) . and π := P ◦ (X1 . Xn ) = P ((X1 . . Proposition 10. Then the following are equivalent. dµn (xn ) . .18) To see this write Page: 94 job: prob macro: svmonob. P ) . . . (2 =⇒ 3) Let A ∈ BRn and f = 1A in Eq. π = µ1 ⊗ µ2 ⊗ · · · ⊗ µn . . . . n for some (and hence all) permutations. Xn ) ∈ A1 × · · · × An ) n n = j =1 P (Xj ∈ Aj ) = j =1 µj (Aj ) = (This measure and its completion will be denoted by µ1 ⊗ · · · ⊗ µn . . . 2. Example 10. Xσ(1) Xσ(n) dµσ(n) (xσ(n) ) |f (x1 .5 in this regard. . xn ) (10. .17) 1{x=y} d (µ ⊗ ν ) (x. . which is valid for all Aj ∈ BR . Moreover. . M := {1A1 ×···×An : Ai ∈ BR } and so by the multiplicative systems theorem. . . . . Rn where σ is any permutation of {1. This equation also holds for any f ∈ L1 (π ) and moreover. . .10. Xn ) is the joint distribution of (X1 . . . see Exercise 10. (10. xn )| < ∞ (3 =⇒ 1) This follows from the identity. . (10. . and hence P (X = Y ) = E 1{X =Y } = R2 f (x1 . µn (An ) for all Ai ∈ Mi . . .cls 1 x = ∞ −tx e dt 0 and use Fubini-Tonelli to conclude that date/time: 23-Feb-2007/15:20 . BRn ) → (R. n}. n. (1 =⇒ 2) Suppose that {Xk }k=1 are independent and let H denote the set of bounded measurable functions. 2. . Example 10. . . 1. . If F (x) := P (X ≤ x) is continuous. ∞] is a M1 ⊗ · · · ⊗ Mn – measurable function then f dπ = X Xσ(1) Rn f (x1 .9. . To prove this. Xn ) ∈ A) = E1A (X1 . . dµσ(1) (xσ(1) ) .

21) 1 − (cos M + (Λ + t) sin M ) e−M (Λ+t) 1 = π − arctan Λ − ε(M. t2 (1 + te The next example is a reﬁnement of this result.19) cos M + (Λ + t) sin M (Λ + t) + 1 ∞ 2 ≤ 1 + (Λ + t ) (Λ + t) + 1 2 ≤ C. and forΛ. M ∈ [0. → 1 + t2 2 0 = wherein we have used the dominated convergence theorem (for instance. and Fubini’s theorem. Making use of the identity ∞ 10. take 1 −t g (t) := 1+ + e−t )) to pass to the limit. (10. This estimate along with Eq. ·) and f y := f (·. If f : X × Y → C is a function let fx = f (x. (10.19) follows (using the dominated convergence theorem again) by letting M → ∞.20) where C = maxx≥0 1+x 1+x2 ∼ = 1. Λ)| ≤ 1 e−M Λ sin x −Λx e dx − π + arctan Λ ≤ C x 2 M = √1 2 2−2 0 e−M (Λ+t) dt = C e−M Λ . Example 10. We have ∞ 0 = 0 ∞ = 0 dt 2 (Λ + t) + 1 ∞ cos M + (Λ + t) sin M −M (Λ+t) 1 dt − e dt 2 2 (Λ + t) + 1 (Λ + t) + 1 0 (10. Λ) 2 where ε(M. Λ) = 0 ∞ cos M + (Λ + t) sin M (Λ + t) + 1 2 e−M (Λ+t) dt.18) is valid.10.13.2.18) follows by taking Λ → ∞ and Eq.21) proves Eq.cls date/time: 23-Feb-2007/15:20 . To verify these assertions. let e−tx dt = 1/x 0 xE := {y ∈ Y : (x.20) from which Eq. (10. x x x Note: you may skip the rest of this chapter! |sin x| = 0 cos ydy ≤ 0 |cos y | dy ≤ 0 1dy = |x| so sin x x ≤ 1 for all x = 0.4 Fubini’s Theorem and Completions M 0 95 sin x dx = x = M 0 ∞ 0 ∞ 0 0 ∞ M e−tx sin x dt dx 0 M sin x −Λx e dx = x = M ∞ dx sin x e−Λx 0 ∞ M 0 e−tx dt e−tx sin x dx dt dt 0 ∞ 0 dx sin x e−(Λ+t)x 1 1 − te−M t sin M − e−M t cos M dt 2 1 + t 0 ∞ π 1 dt = as M → ∞. Similarly if y ∈ Y is given let Ey := {x ∈ X : (x. y ) so that fx : Y → C and f y : X → C. In particular Eq. (10. M 0 |ε(M. (10.4 Fubini’s Theorem and Completions Notation 10. ﬁrst notice that by the fundamental theorem of calculus. y ) ∈ E }. y ) ∈ E }. Page: 95 job: prob macro: svmonob. ∞). M (10. Since sin x −Λx 1 e dx = π − arctan Λ for all Λ > 0 x 2 (10.14 Given E ⊂ X × Y and x ∈ X.

{hy = 0} ∈ N .e. x and µ(Ey ) = 0 for ν a.e. y )dλ(x. y ) : h(x. i. y ) dν (x)g (x. y ) = 0} ⊂ E and (µ ⊗ ν )(E ) = 0. g (x. A function f : Rd → R is Lebesgue measurable if f −1 (BR ) ⊂ Ld . y ) is in L1 (ν ) and f (x. L.e. y )dµ(x) = X X g (x. y respectively.5 Lebesgue Measure on Rd and the Change of Variables Theorem Notation 10. y that {hx = 0} ∈ M. y )dν (y ) = Y Y f dmd . y )dν (y ). y ) = 0} ⊂ Ey we learn that for µ a. µ ⊗ ν ). then there exists E ∈ M ⊗ N such that {(x. y )dλ(x. x → f (x. y )dν (y ) exists and equals 0 for µ a.e. y ) and (µ ⊗ ν )(E ) = 0.e. y. y ) Y and f dλ = X ×Y Y = dν X dν (y ) Y dµ f = X dµ Y dν f. y ) = g (x.e. x and similarly that X h(x.6. Similarly it is shown that dν (y ) Y X This shows that µ({x : ν (x E ) = 0}) = 0 and ν ({y : µ(Ey ) = 0}) = 0. x.e.e. Therefore 0= X ×Y 1 dµ(x)f (x. y → f (x. ν (x E ) = 0 for µ a. y and in case (b) fx ∈ L1 (ν ) and f y ∈ L1 (µ) for µ a.96 10 Multiple and Iterated Integrals Theorem 10.15. and µ({hy = 0}) = 0.. y ) X ×Y Proof. λ) be the completion of (X × Y.e.16 Let d times d d times m := m ⊗ · · · ⊗ m on BRd = BR ⊗ · · · ⊗ BR be the d – fold product of Lebesgue measure m on BR . ν ({hx = 0}) = 0 and a. y. For ν a. If E ∈ M ⊗ N is a µ ⊗ ν null set (i. x and f y is M – measurable for ν a. We will also use md to denote its completion and let Ld be the completion of BRd relative to md .e. λ− a. y ) = g (x. Since {hx = 0} = {y ∈ Y : h(x. y )dµ(x) exists and equals 0 for ν a. we may choose g ∈ L (M⊗N . y ).e. y ) for λ− a.e. For general f ∈ L (λ). This implies Y h(x. y ) + h(x. y ). y ) + h(x.6 f = g + h has the following properties: 1. From these assertions and Theorem 10. macro: svmonob.e. then = 0 = (µ ⊗ ν )(E ) = X f (x. y ) = g (x. N . 10. = g (x. M ⊗ N . For µ a. y ).e. f (x) dx = f dm = Rd Rd hdλ = Y X hdµ dν = X 1 Y hdν dµ. µ ⊗ ν ) such that f (x. y )dµ(x).e. Deﬁne h := f − g. Let (X × Y. Then h = 0. A subset A ∈ Ld is called a Lebesgue measurable set and md is called d – dimensional Lebesgue measure. If f is L – measurable and (a) f ≥ 0 or (b) f ∈ L1 (λ) then fx is N – measurable for µ a. Rd Hopefully the reader will understand the meaning from the context.e.e. y )| ≤ 1E (x. ν ) are complete σ – ﬁnite measure spaces. Moreover. X ×Y ν (x E )dµ(x) = X µ(Ey )dν (y ).e. Suppose (X. y ) = X dµ(x) Y dν (y )g (x. y ) = 0} ⊂ x E and {hy = 0} = {x ∈ X : h(x. Therefore |h(x. y. y )d(µ ⊗ ν )(x. If h is L measurable and h = 0 for λ – a. or just Lebesgue measure for short.18 I will often be sloppy in the sequel and write m for md and dx for dm(x) = dmd (x). Notation 10. Hence by what we have just proved and Theorem 10.e. Deﬁnition 10. x and ν a. it follows that dµ(x) X Y fx dν ∈ L1 (µ) and y→ X f y dµ ∈ L1 (ν ) dν (y )f (x. x and ν a. x→ Y 2. i. µ) and (Y. M.cls date/time: 23-Feb-2007/15:20 Page: 96 job: prob . y ) = X ×Y f (x. (x. (µ ⊗ ν )(E ) = 0). y ) is in L1 (µ) and f (x.17.

More explicitly.1.j – matrix entry of T (x) is given by T (x)ij = ∂i Tj (x) where T (x) = (T1 (x). Let Ω ⊂o Rd be an open set and T : Ω → T (Ω ) ⊂o Rd be a C 1 – diﬀeomorphism. then dy = | det T (x) |dx.e. Moreover md is the unique translation invariant measure on BRd such that md ((0. In this problem you are asked to show there is no reasonable notion of Lebesgue measure on an inﬁnite dimensional Hilbert space. the i . .β ]) (y ) dy T ([a. a < α < β < b such that [a. The proof of the second assertion is Exercise 10. 10. . As usual. then β β |T (x)| dx = α α T (x) dx = T (β ) − T (α) 1T ((α.2 see Figure 10. m(B0 (ε)) > 0 for all ε > 0. Page: 97 job: prob macro: svmonob. Let A = J1 × · · · × Jd with Ji ∈ BR and x ∈ Rd . Theorem 10.6. Show m(V ) = ∞ for all non-empty open subsets V ⊂ H. . Proof. .21.23) . b] . Theorem 10.20 (Change of Variables Theorem). Using this it is easily seen that m(x + A) = m(A) for all A ∈ Ld ..22) 1(α.b] [a.β ] (x) |T (x)| dx = α |T (x)| dx. 1]d ) = 1. ∂d T1 (x) . (10.19. . .b]) ∂1 Td (x) . . . . β ])) = while if T (x) < 0 on [a. if x ranges through Ω then y must range through T (Ω ) . ∞]. . Nevertheless. From this fact we see that the measure md (x + ·) and md (·) have the same null sets. . .20 is best remembered as the statement: if we make the change of variables y = T (x) .1. 2 = m (T ((α. suppose H is an inﬁnite dimensional Hilbert space and m is a countably additive measure on BH which is invariant under translations and satisﬁes. b] .10. then That is T : Ω → T (Ω ) ⊂o Rd is a continuously diﬀerentiable bijection and the inverse map T −1 : T (Ω ) → Ω is also continuously diﬀerentiable. m(xd + Jd ) = m(J1 ) . The case d = 1 was essentially done in Exercise 8. Remark 10. Td (x))tr and ∂i = ∂/∂xi .20. you must also change the limits of integration appropriately. Fig. i. If T (x) > 0 on [a. ∂d Td (x) i. Lebesgue measure md is translation invariant. The proof will be by induction on d. Proof. f : T (Ω ) → [0. . f (T (x)) | det T (x) |dx = Ω T (Ω ) β d where T (x) is the linear transformation on Rd deﬁned by T (x)v := dt |0 T (x + d tv ). viewing vectors in R as columns.1. b] is a compact subinterval of Ω. . . To be more precise.b] f (y ) dy. m(Jd ) = md (A) and hence md (x + A) = md (A) for all A ∈ BRd since it holds for A in a multiplicative system which generates BRd . . T (x) may be represented by the matrix ∂1 T1 (x) . Then x + A = (x1 + J1 ) × (x2 + J2 ) × · · · × (xd + Jd ) and therefore by translation invariance of m on BR we ﬁnd that md (x + A) = m(x1 + J1 ) .cls date/time: 23-Feb-2007/15:20 . T (x) = (10. The geometric setup of Theorem 10. Then for any Borel measurable function. for the sake of completeness let us give a proof here. . . Suppose d = 1.7.e. Exercise 10. .5 Lebesgue Measure on Rd and the Change of Variables Theorem 97 Theorem 10. Then | det T | = |T | and 1T ((α.β ]) (T (x)) |T (x)| dx = [a.

(10.bn )) · f ◦ T · |T |dm 1T ([αk . . . then by what we have already proved and the monotone convergence theorem 1(an . the case of T in Eq. .cls date/time: 23-Feb-2007/15:20 . wi−1 . T (x) . Expanding det T (wt ) along the ﬁrst row of the matrix T (wt ) whenever f is of the form f = 1T ((α. will always be taken to be given as in Eq. . Ωt be the (possibly empty) open subset of Rd−1 deﬁned by f (y ) dy T ([a. t.βk ]) · f ◦ T · |T | dm Ω = lim = lim k→∞ k→∞ T (Ω ) 1T ([αk . . . Summing this equality on n. (10.) N Recall that Ω = n=1 (an .β ]) (y ) dy. .25). · · · < N with N = ∞ possible. . . .2. Suppose T (x) has the form T (x) = (xi . . . For t ∈ R. the matrix associated to the diﬀerential. . . .24) Ωt := w ∈ Rd−1 : (w1 .2. . let us write vectors in Rd as row vectors rather than column vectors. (10.βk ]) · f dm = T (Ω ) 1T ((an .3 then implies that Eq. For deﬁniteness we will assume T is as in Eq.24) holds for every bounded measurable function f : T ([a.b]) (10. . wi+1 .bn ) · (f ◦ T ) · |T |dm = Ω Ω 1T ((an . bn ∈ R∪ {±∞} for n = 1. Fig. Td−1 (x) . Case 1. xi ) (10. T2 (x) . wd−1 ) ∈ Ω and Tt : Ωt → Rd−1 be deﬁned by Tt (w) = (T2 (wt ) . . b] . For notational compactness. T ([a.25) shows |det T (wt )| = |det Tt (w)| . The picture indicates the geometry associated with the map T and slicing the set Ω along planes where x3 = t.23). let it : Rd−1 → Rd be the inclusion map deﬁned by it (w) := wt := (w1 . In this picture d = i = 3 and Ω is an egg-shaped region with an egg-shaped hole. . .b] for some i ∈ {1.bn )) · f dm. (10. wd−1 ) . . b]) → R. [a. Td (x)) or T (x) = (T1 (x) . (10.22) holds. Nevertheless. we now suppose d > 1 and suppose the theorem is valid with d being replaced by d − 1. To carry out the induction step. . Now by the Fubini-Tonelli Theorem and the induction hypothesis. bn ) where an . . see Figure 10.b]) = m (T ((α. .98 10 Multiple and Iterated Integrals β β |T (x)| dx = − α α T (x) dx = T (α) − T (β ) 1T ((α. . . 10. Td (wt )) . 2. Hence if f : T (Ω ) → R + is a Borel measurable function and an < αk < βk < bn with αk ↓ an and βk ↑ bn . . . Page: 98 job: prob macro: svmonob. d} . An application of Dynkin’s multiplicative system Theorem 9. then shows Eq.β ]) with a < α < β < b.26) (10. wi+1 . . wi−1 . . (Observe that |T (x)| is continuous and hence bounded for x in the compact interval. t. .26) may be handled similarly. . β ])) = Combining the previous three equations shows f (T (x)) |T (x)| dx = [a.

z ) f (t. z ) dz dt = T (Ω ) Observe that S is a map of the form in Eq. Case 3.25). . there must be some i such that Mi = 0. . . zd . T2 (x) . (10. Tt (w)) | det Tt (w) |dw dt f (t.26). T (Ω ) = T (it (Ω )) = {(t. Then Ω = i=1 Wi and by repeated use of case 2. . . Let R : S (W ) → T (W ) ⊂o Rd to be the C 1 – diﬀeomorphism deﬁned by R (z ) := T ◦ S −1 (z ) for all z ∈ S (W ) . (10. (10.27) Observe that |det S (x0 )| = |Mi | = 0. two applications of the results in Case 1. shows. . Since d and Case 2. . Td (x))) for all x ∈ W. . Using the compactness of Kn and case 2. S (x) := (xi . Let {Wi }i=1 be an enumeration of ∪∞ n=1 Wn and set W1 = W1 and ∞ ˜ ˜ Wi := Wi \ (W1 ∪ · · · ∪ Wi−1 ) for all i ≥ 2.5 Lebesgue Measure on Rd and the Change of Variables Theorem 99 f ◦ T | det T |dm = Ω Rd 1Ω · f ◦ T | det T |dm 1Ω (wt ) (f ◦ T ) (wt ) | det T (wt ) |dwdt Rd and S : W → S (W ) is a C 1 – diﬀeomorphism. (10. if (z1 . ∞] is a Borel measurable function. macro: svmonob. R is a map of the form in Eq. z2 . ∞]. is proved. ∞] be a general non-negative Borel measurable function and let Kn := {x ∈ Ω : dist(x. . Fix an i such that Mi = 0 and let.10. . Let Mi be the 1-i minor of T (x0 ) . (Eq. (10. there is a ﬁnite open cover Wn of Kn such that W ⊂ Ω and Eq.) Let f : Ω → [0. T2 (x) . (General Case. z ) dz dt = f (y ) dy R Ωt (T1 (x) . . Td (x)) then R (z ) = T1 S −1 (z ) .) Suppose that T : Ω → Rd is a general map as in the statement of the theorem and x0 ∈ Ω is an arbitrary point. . . T (x) = R (S (x)) S (x) (by the chain rule) and (by the multiplicative property of the determinant) |det T (x)| = | det R (S (x)) | |det S (x)| ∀ x ∈ W.cls date/time: 23-Feb-2007/15:20 0 = det T (x0 ) = i=1 (−1) i+1 ∂i Tj (x0 ) · Mi . the determinant of T (x0 ) with the ﬁrst row and ith – column removed. there exist an open neighborhood W of x0 such that W ⊂o Ω and S (W ) ⊂o Rd Page: 99 job: prob . Hence by the inverse function Theorem. . . Td (x)) = T (x) = R (S (x)) = R ((xi . . . . i. .22) holds with Ω replaced by W for each ∞ ˜ W ∈ Wn . . Td (x)) . . f ◦ T · | det T |dm = W W wherein the last two equalities we have used Fubini-Tonelli along with the identity.. . z ) : z ∈ Tt (Ωt )} . (10. . f : T (W ) → [0. Because = = R (f ◦ T ) (wt ) | det T (wt ) |dw dt f (t. zd ) = S (x) = (xi .28) = R Ωt = R Rd−1 Tt (Ωt ) 1T (Ω ) (t.e. Ω c ) ≥ 1/n and |x| ≤ n} .22) is true locally. . for each n ∈ N. T2 (x) . t∈R t∈R (f ◦ R · | det R |) ◦ S · |det S | dm f ◦ R · | det R |dm = S (W ) R(S (W )) Case 2. We will now show there exists an open neighborhood W ⊂ Ω of x0 such that f ◦ T | det T |dm = W T (W ) = f dm f dm = T (W ) f dm holds for all Borel measurable function. So if f : T (W ) → [0. Then each Kn is a compact subset of Ω and Kn ↑ Ω as n → ∞. z2 .

e. i. θ)| drdθ = rdrdθ.29) is as follows. Then by the usual truncation argument. one often learns the change of variables formula as b T (b) This equation also shows that m ◦ T and m have the same null sets and hence the equality in Eq. For z ∈ T ([a. θ) = (r cos θ. (10.b)) f (y ) dy which is again implies Eq. An application of Dynkin’s multiplicative systems theorem now shows that Eq.22) with Ω = (a. 2π ) → R2 is deﬁned by x = T (r.100 10 Multiple and Iterated Integrals ∞ f ◦ T | det T |dm = Ω i=1 Ω ∞ 1W ˜ i · (f ◦ T ) · | det T |dm 1T (W ˜ i ) f ◦ T · | det T |dm i=1W ∞ i Example 10. (10. b)) = (T (b) . (10. b] . x1 = r cos θ and x2 = r sin θ for 0 < r < ∞ and 0 < θ < 2π. (10.29) is more general than Eq. r sin θ) .24 (Polar Coordinates).22). Suppose T : (0. m (T (A)) = |det T | m (A) for allA ∈ BRd .2. b) . 2π )) = := {(x.20 that 2π ∞ f (T (x)) (− |T (x)|) dx = − (a. The standard proof of Eq. (10. then m ◦ T = |det T | m.29) where f : [a.22. then Eq. Example 10. (10. Page: 100 job: prob f (x)dx = R2 0 dθ 0 dr r · f (r (cos θ.cls date/time: 23-Feb-2007/15:20 . b b b f (T (x)) T (x) dx = a a F (T (x)) T (x) dx = a T (b) d [F (T (x))] dx dx = F (T (x)) |b a = T (a) f (y ) dy. then m (T (A)) = Rd 1T (A) (y ) dy = Rd 1T (A) (T x) |det T (x)| dx = = Rd 1A (x) |det T (x)| dx n = i=1 T (Wi ) 1T (W ˜ i ) · f dm = i=1 T (Ω ) 1T (W ˜ i ) · f dm wherein the second equality we have made the change of variables. b) . If T > 0 on (a.31) for any Borel measurable function f : R2 → [0.20. ∞]. (10. md .22) since it does not require T to be injective.b) T ( b) f (y ) dy = − T ((a. b) then T ((a. y = T (x) . let z F (z ) := T (a) f (y ) dy.e. cos θ − r sin θ sin θ r cos θ Then by the chain rule and the fundamental theorem of calculus. it follows from the change of variables Theorem 10.30) = T (Ω ) f dm. Exercise 10. b) . ∞) × (0.23. macro: svmonob. b)) = (T (a) . θ) = and therefore dx = |det T (r. b) then T ((a.22) holds. T (b)) and Eq. Hence we have shown d (m ◦ T ) = |det T (·)| dm. (10. ∞) × (0. Show that f ∈ L1 T (Ω ) . 0) : x ≥ 0} has m2 – measure zero.29) is implies Eq. i. In particular if T ∈ GL(d. R) = GL(Rd ) – the space of d × d invertible matrices. b]) . md iﬀ |f ◦ T | | det T |dm < ∞ Ω f (T (x)) T (x) dx = a T (a) f (y ) dy (10. we are making the change of variable. In this case T (r.29) holds for all bounded measurable functions f on (a. Continuing the setup in Theorem 10. Remark 10. it also holds for all positive measurable functions on (a. b] → R is a continuous function and T is C 1 – function deﬁned in a neighborhood of [a. When d = 1. (10. Observing that R2 \ T ((0. T (a)) and Eq. if A ∈ BΩ . sin θ)) (10. (10. (10. On the other hand if T < 0 on (a. On the other hand Eq.29) is equivalent to T (a) and if f ∈ L1 T (Ω ) .30) is valid for any A ∈ Ld .

θ) . ϕ. 2) × (0. Suppose that f : Ω ⊂o C ∼ = R2 → C is an injective holomorphic function such that f (z ) = 0 for all z ∈ Ω.10. 10.3. ∞) × (0.25 (Holomorphic Change of Variables). w = u + iv = f (x + iy ) = U (x. Page: 101 job: prob macro: svmonob. see Figure 10.26. and therefore I = I+ + I− = 2 · (3/4) = 3/2. we learn Ux Vy − Uy Vx = Therefore 2 Ux Fig.2)×(0. see Figure 10. In this example we will evaluate the integral I := Ω x4 − y 4 dxdy where Ω = (x. y ) : 1 < x2 − y 2 < 2. Let D be the connected component in the ﬁrst quadrant so that Ω = −D ∪ D and T (±D) = (1. Recalling that U and V satisfy the Cauchy Riemann equations. y ) for all z = x + iy ∈ Ω. ϕ. cos ϕ) . The region Ω consists of the two curved rectangular regions shown. in which case dudv = det Notice that 1 ududv. The relation of x to (r.3 (Spherical Coordinates). (u.4.1) 1 u2 2 3 | ·1= 2 2 1 4 Fig. Hence if we make the change of variables. r sin ϕ sin θ. v ) := T (x. φ. show 2 dudv = |f (x + iy )| dxdy. y ) + iV (x. Exercise 10. Example 10. 0 < xy < 1 . 2 The function T is not injective on Ω but it is injective on each of its connected components. The change of variables theorem then implies x4 − y 4 = x2 − y 2 x2 + y 2 = u x2 + y 2 = I± := ±D 2x −2y y x dxdy = 2 x2 + y 2 dxdy x4 − y 4 dxdy = 1 2 ududv = (1. We may express f as f (x + iy ) = U (x. y ) + iV (x. sin ϕ sin θ.5 Lebesgue Measure on Rd and the Change of Variables Theorem 101 Example 10. By making the change of variables x = T (r.4.3. Ux = Vy and Uy = −Vx with f = Ux + iVx . + 2 Vx 2 = |f | . θ) = (r sin ϕ cos θ. Let T : (0. θ) in spherical coordinates. r cos ϕ) = r (sin ϕ cos θ. 1) . We are going to do this by making the change of variables. y ) then dudv = det Ux Uy Vx Vy dxdy = |Ux Vy − Uy Vx | dxdy. xy . 10. π ) × (0. 2π ) → R3 be deﬁned by T (r. y ) = x2 − y 2 .cls date/time: 23-Feb-2007/15:20 .

θ)) Deﬁnition 10. For E ∈ BS d−1 . 1 + ε] and ω ∈ E } (10. ∞) × S d−1 be deﬁned by Φ(x) := (|x| . ∞]. Since Φ and Φ−1 are continuous. BS d−1 and Φ : −1 Rd \ {0} → (0.cls date/time: 23-Feb-2007/15:20 . Therefore we expect the area of E should be given by σ (E ) = lim ε↓0 S d−1 = {x ∈ R : |x| := i=1 d x2 i = 1} be the unit sphere in Rd equipped with its Borel σ – algebra. a] and ω ∈ E } = Φ−1 ((0.5. ε The following theorem is motivated by Example 10.3. For E ∈ BS d−1 and a > 0. ω ) = rω. |x| x). let Ea := {rω : r ∈ (0. 1 + ε]E ) = m (E1+ε \ E1 ) = (1 + ε)d − 1 m(E1 ). Proof.24 and Exercise 10. Lemma 10. If E ⊂ S d−1 is a set and ε > 0 is a small number. ∞ then E1 = i=1 (Ei )1 and ∞ ∞ for any Borel measurable function.32) should be approximately given by m ((1. B )– measurable function then macro: svmonob. Let a > 0 and Id (a) := Rd e−a|x| dm(x). σ (E ) = d · m(E1 ) = 2 m ((Ei )1 ) = i=1 i=1 σ (Ei ).28. then the volume of (1. Motivating the deﬁnition of surface measure for a sphere. This shows that I2 (a) = π/a and the result now follows from Eq. On the other hand e−a|y| e−at md−1 (dy ) dt 2 2 So it suﬃces to compute: I2 (a) = R2 e−a|x| dm(x) = R2 \{0} 2 e−a(x1 +x2 ) dx1 dx2 . ∞ 2π I2 (a) = 0 dr r 0 M M →∞ 0 dθ e−ar = 2π 0 2 ∞ re−ar dr 2 2 = 2π lim re −ar 2 e−ar dr = 2π lim M →∞ −2a M = 0 2π = π/a. (10. let σ (E ) := d · m(E1 ).32).29 (Polar Coordinates). we ﬁnd. Then Id (a) = (π/a)d/2 . Id (a) = Rd−1 ×R d = Id−1 (a)I1 (a) = I1 (a). Theorem 10. see Figure 10. 2a Fig. Page: 102 job: prob (1 + ε)d − 1 m(E1 ) = d · m(E1 ).31).102 10 Multiple and Iterated Integrals π 2π ∞ f (x)dx = R3 0 dϕ 0 dθ 0 dr r2 sin ϕ · f (T (r. We call σ the surface measure on S d−1 . a] × E ) ∈ BRd .5 below. they are both Borel measurable. ∞) × S → R \ {0} . 10. is given by Φ (r. (10. By Tonelli’s theorem and induction. It is easy to check that σ is a measure. The inverse map. ∞] is a (BRd . then E1 = ∞ Φ−1 ((0. 1] × E ) ∈ BRd so that m(E1 ) is well deﬁned. If f : Rd → [0. 1 + ε] · E ) ∼ = σ (E )ε. 10. −1 d−1 d −1 Φ : (0. The intuition behind this deﬁnition is as follows. see Eq. f : R3 → [0. 1 + ε] · E = {rω : r ∈ (1. 2 2 Using polar coordinates. Indeed if E ∈ BS d−1 . ϕ.27. Moreover if E = i=1 Ei .6 The Polar Decomposition of Lebesgue Measure Let d 2 m ((1.

33) we must work out the measure Φ∗ m on B(0.6 proves Eq. is a π class (in fact it is an elementary class) such that σ (E ) = B(0. (10. Example 8. ∞ Proof.∞) . Γ (x + 1) = xΓ (x) is the consequence of the following integration by parts argument: ∞ Γ (x + 1) = 0 e−u ux+1 ∞ du = u ∞ ux − 0 d −u e du du =x 0 ux−1 e−u du = x Γ (x).38) Γ (1/2) = 2 Eq. 2 (10. (10.40).∞) ⊗ BS d−1 . (10.40) where V (r) = m (B (0. ρ(J ) = J Combing the the last two equations with Lemma 10. The surface area σ (S d−1 ) of the unit sphere S d−1 ⊂ Rd is (10. then Φ−1 (A) = {rω : r ∈ (a. Therefore by the basic scaling properties of m and the fundamental theorem of calculus.27 which states that Id (1) = π d/2 . b]) · σ (E ) = (ρ ⊗ σ ) ((a. Γ (1) = 1 and Γ (x + 1) = xΓ (x) for x > 0.6 The Polar Decomposition of Lebesgue Measure 103 f (x)dm(x) = Rd (0.33) Rd f dm = (0.37) may be written as (Φ∗ m) ((a. If A = (a.42) = bd m(E1 ) − ad m(E1 ) = d · m(E1 ) a rd−1 dr. b] and ω ∈ E } = bE1 \ aE1 wherein we have used Ea = aE1 in the last equality. Corollary 10.34) σ (S d−1 ) = where Γ is the gamma function given by ∞ f (|x|)dx = Rd 0 f (r)dV (r) 2π d/2 Γ (d/2) (10. (Φ∗ m) ((a. (10. 1)) = d−1 σ S d−1 rd . it follows from the π – λ Theorem and Eq. b] × E : 0 < a < b and E ∈ BS d−1 } . b] × E with 0 < a < b and E ∈ BS d−1 . Using Theorem 10.41) f ◦ Φ−1 ◦ Φ dm = d (Φ∗ m) (10.29 we ﬁnd ∞ and therefore to prove Eq.39) that Φ∗ m = ρ ⊗ σ. (10. b] × E ) = m (bE1 \ aE1 ) = m(bE1 ) − m(aE1 ) b Id (1) = 0 dr rd−1 e−r 2 dσ = σ (S d−1 ) S d−1 0 rd−1 e−r dr.35) Moreover.10. By Exercise 8. 2 ∞ −∞ e−r dr 2 (10.42). Since E = {(a. (10. date/time: 23-Feb-2007/15:20 . Γ (1/2) = √ π.e. (10. i.35).30.35) gives Page: 103 job: prob macro: svmonob.39) The relation. f dm = Rd Rd \{0} Γ (x) := f ◦ Φ−1 (0. Using this result in Eq.cls e−r dr = 0 √ = I1 (1) = π. b] × E ) = ρ((a.∞)×S d−1 ux−1 e−u du 0 (10. (10.∞) ⊗ BS d−1 deﬁned by Φ∗ m(A) := m Φ−1 (A) ∀ A ∈ B(0.∞)×S d−1 f (rω )rd−1 drdσ (ω ).∞) ⊗ BS d−1 . b] × E ) . we conclude that 1 π d/2 = Id (1) = σ (S d−1 )Γ (d/2) 2 which proves Eq. (10. ∞ r d−1 dr ∀ J ∈ B(0. Proof.36) We simplify this last integral by making the change of variables u = r2 so that −1/2 du. 2 0 2 u d−1 2 ∞ (10.37) Letting dρ(r) = rd−1 dr. The result is r = u1/2 and dr = 1 2u ∞ 0 rd−1 e−r dr = 2 1 e−u u−1/2 du 2 0 ∞ d 1 1 = u 2 −1 e−u du = Γ (d/2).6.8 implies Γ (1) = 1 and from Eq.∞)×S d−1 f ◦ Φ−1 d (ρ ⊗ σ ) In particular if f : R+ → R+ is measurable then ∞ which combined with Tonelli’s Theorem 10. r)) = rd m (B (0. (10.

r) = ∂Tn ∂Tn ∂θ ∂ϕ1 0 . ∆n (θ. . . (10. r sin ϕ1 ) r cos ϕ1 r sin ϕ1 cos θ = r sin ϕ1 sin θ =: T3 (θ. . . . . . r)| .7 More Spherical Coordinates In this section we will deﬁne spherical coordinates in all dimensions. .6. . θ) ∈ (0. ) . sin2 ϕ2 sin ϕ1 . r) := |det Tn (θ. x1 = r sin ϕn−2 .104 10 Multiple and Iterated Integrals 10. ϕn−2 . . r cos ϕ1 If f is a function on rS n−1 – the sphere of radius r centered at 0 inside of Rn . ϕ1 . .44) where ∆n (θ. We are going to compute ∆n inductively. ). . .0≤θ ≤2π f (Tn (θ. r sin ϕn−1 .0≤θ ≤2π dϕ1 . . ϕn−2 . . . . . r)dϕ1 . ϕ1 . ϕn−2 . . . . . ϕn−2 . . ϕ1 .ϕ1 . ϕn−1 . . . so that = 0 dr 0≤ϕi ≤π. ϕn−2 . . ∂Tn ∂Tn ∂Tn ∂ϕn−2 ∂ρ r cos ϕn−1 ∂ρ 0 −r sin ϕn−1 sin ϕn−1 cos ϕn−1 = r cos2 ϕn−1 + sin2 ϕn−1 ∆n (.6. r)) (10. x2 r sin θ For n = 3 we let x3 = r cos ϕ1 and then x1 x2 = T2 (θ. f (x)dm(x) Rn ∞ (10. ϕn−2 .. .45) x1 x2 = x3 T2 (θ. . . = r cos ϕn−1 xn xn+1 So for example. 0 . ϕ1 . r sin ϕ1 ). dϕn−2 dθ ∆n (θ. ϕn−2 . ϕn−2 . . r sin ϕn−1 ). and more generally. ϕ1 . ϕn−2 . . ϕn−2 . . ϕn−2 . . then f (x)dσ (x) = rn−1 rS n−1 S n−1 f (rω )dσ (ω ) We continue to work inductively this way to deﬁne x1 . ϕn−2 . θ. . sin ϕ2 sin ϕ1 sin θ x3 = r sin ϕn−2 . . . = Tn+1 (θ. Letting ρ := r sin ϕn−1 ∂Tn n and writing ∂T ∂ξ for ∂ξ (θ. dϕn−2 dθ (10. ϕn−2 . r) ×f (Tn (θ. ϕ1 . . . . 2π ) so that x1 r cos θ = = T2 (θ. r). ϕ1 . sin ϕ2 cos ϕ1 . Along the way we will develop an explicit method for computing surface integrals on spheres. . Setting up polar coordinates in two and three dimensions. ϕ1 . . . . .. .. . . . sin ϕ2 sin ϕ1 cos θ x2 = r sin ϕn−2 . ϕ1 . . . . . r. . x1 x2 x3 x4 = r sin ϕ2 sin ϕ1 cos θ = r sin ϕ2 sin ϕ1 sin θ = r sin ϕ2 cos ϕ1 = r cos ϕ2 = 0≤ϕi ≤π. . r). . . ρ) = r∆n (θ. ϕ1 . Page: 104 job: prob macro: svmonob. .cls date/time: 23-Feb-2007/15:20 . . r) = rn−1 sinn−2 ϕn−2 . .. ϕ1 . . 10. . ϕ1 . ϕn−1 . By the change of variables formula. xn−2 = r sin ϕn−2 sin ϕn−3 cos ϕn−4 xn−1 = r sin ϕn−2 cos ϕn−3 xn = r cos ϕn−2 . . .43) as can be seen from Figure 10. . As usual when n = 2 deﬁne spherical coordinates (r. ϕ1 . . . ∞) × [0. Tn (θ. ρ) we have ∆n+1 (θ. . ∆n is given by Fig.46) Proof. Proposition 10.31. r))∆n (θ. The Jacobian.

1 + (k − 1) [γk−2 − γk ] 2 2 (2π ) (2n − 1)!! (2π ) π = (2n − 1)!! (2n)!! (2n)!! n+1 and hence γk satisﬁes γ0 = π. . . 1 22 π 2 σ (S 3 ) = 2π · 2 · γ2 = 2π · 2 · π = .24.10. . γ1 = 2 and the recursion relation γk = Hence we may conclude γ0 = π. (10. . (2n)!! (2n + 1)!! (2n + 1)!! n+1 n+1 = 2δk. Equation (10.50) as n/2 2(2π) (n−1)!! for n even n n+1 σ (S ) = (2π) 2 for n odd. γ5 = 2. dϕn−2 dθ γk = σ (S n−2 )γn−2 k=1 = 2π where γk := π π 0 (10. r sin ϕn−1 ). k and more generally by induction that γ2k = π (2k − 1)!! (2k )!! and γ2k+1 = 2 . (10. . ϕn−2 . 2k + 1 2k + 2 (2k )!! (2k + 2)!! The recursion relation in Eq. we have by integration by parts that. (10. . γ4 = π. . ∆3 (θ.33) and (10.33). ϕ1 . ϕn−1 . . Eq.0≤θ ≤2π n−2 2k + 1 2k + 1 (2k − 1)!! (2k + 1)!! γ2k = π =π . (2k )!! (2k + 1)!! job: prob we may write σ (S 2n+1 ) = 2πn! which shows that Eqs. .49) = 2π implies σ S 1 = 2π. (n−1)!! macro: svmonob. ∆n (θ. sin2 ϕ2 sin ϕ1 which proves Eq. (10.45). . γ3 = 2.49). ϕ1 . . γ6 = π 2 3 42 53 642 k−1 γk−2 for k ≥ 2. . As a simple application. (10. r) = r∆n (θ.45). . r) = r∆3 (θ. .e. Indeed. 2 2!! 22 π 2 2 23 π 2 22 π 2 · γ3 = ·2 = σ (S 4 ) = 2!! 2!! 3 3!! 1 2 31 23 π 3 5 σ (S ) = 2π · 2 · π · 2 · π= . ϕ2 .46) implies σ (S n−1 ) = 0≤ϕi ≤π. π π (10. We may also write the formula in Eq. .47) implies. r) = rn−1 sinn−2 ϕn−2 . γ2(k+1)+1 = and 2k + 2 2k + 2 (2k )!! [2(k + 1)]!! γ2k+1 = 2 =2 2k + 3 2k + 3 (2k + 1)!! (2(k + 1) + 1)!! To arrive at this result we have expanded the determinant along the bottom row. . If k ≥ 1. . (10.44) and (10.48) sink ϕdϕ. (2 · 1) = 2n n! (2π ) (2n)!! 2 (2π ) 2 = .48) may be written as γ2(k+1) = σ (S n ) = σ S n−1 γn−1 which combined with σ S 1 (10. . ϕ1 .1 + (k − 1) 0 sin ϕ 1 − sin ϕ dϕ = 2δk. ϕn−2 . (10. γ1 = 2.cls date/time: 23-Feb-2007/15:20 n+1 Page: 105 . ∆n+1 (θ.47) Indeed. ϕn−2 . (10. 2 3 42 4!! 1 2 31 42 24 π 3 σ (S 6 ) = 2π · 2 · π · 2 · π· 2= 2 3 42 53 5!! and more generally that σ (S 2n ) = 2 (2π ) (2π ) and σ (S 2n+1 ) = (2n − 1)!! (2n)!! n n n+1 sinn−2 ϕn−2 .50) γk = 0 sink ϕdϕ = − 0 π sink−1 ϕ d cos ϕ = 2δk.1 + (k − 1) 0 k−2 sink−2 ϕ cos2 ϕdϕ which is veriﬁed inductively using Eq. r) = r already derived in Example 10. Staring with ∆2 (θ.46) now follows from Eqs. ϕ1 . σ (S 2 ) = 2π · γ1 = 2π · 2.50 are in agreement. σ (S 2n+1 ) = σ (S 2n )γ2n = and σ (S (n+1) ) = σ (S 2n+2 ) = σ (S 2n+1 )γ2n+1 = Using (2n)!! = 2n (2(n − 1)) . . ϕ1 .7 More Spherical Coordinates 105 i. ϕ1 . r sin ϕ2 ) = r3 sin2 ϕ2 sin ϕ1 . sin2 ϕ2 sin ϕ1 dϕ1 . Eq. . . (10. r sin ϕ1 ) = r2 sin ϕ1 ∆4 (θ. (10. . r) = r∆2 (θ. γ2 = 1 2 31 42 531 π.

Let f : X1 × X2 × X3 → [0. .7. Rotation invariance of surface measure on S n−1 . 1]d ) = 1.48 on p. 3 be σ – ﬁnite measure spaces. Prove the second assertion of Theorem 10. g (x)dx = 0 0 ∞ f (t)dt. Properties of the Γ – function. Exercise 10.56 on p.50 on p. 80. Show 0 sin x x ∞ sin x x dm(x) is not deﬁned as a Lebesgue integral. to get started deﬁne π (A) := X1 Exercise 10. .4. 2.19.) Exercise 10. Verify the identity. Folland Problem 2.e. BR ¯ ) – measurable function. Folland Problem 2.” 2.18. dm). a). ∞). Page: 106 job: prob macro: svmonob.) dµ (x1 ) .) Let X = [0. 1]. 69 pertaining to area under a curve.8. 80. a). We will write π := µ1 ⊗ µ2 ⊗ µ3 . Show F is ((M1 ⊗ M2 ) ⊗ M3 . 4.15. Folland Problem 2. Then π is the unique measure on M1 ⊗ M2 ⊗ M3 such that π (A1 × A2 × A3 ) = µ1 (A1 )µ2 (A2 )µ3 (A3 ) for all Ai ∈ Mi . x2 . 69. j =1 1D (x. Folland Problem 2. 77. So sin ∈ / L1 ([0.22. Exercise 10. Folland Problem 2. . show g ∈ L ((0. (M1 ⊗ M2 ) ⊗M3 ) → (X1 × X2 × X3 .64 on p.8 Exercises Exercise 10. Folland Problem 2.62 on p. a). .10. Show. x3 ). 77.17 that ωi ωj dσ (ω ) = S d−1 dµ3 (x3 ) X2 dµ2 (x2 ) X1 dµ1 (x1 )f (x1 . Exercise 10. Exercise 10. Exercise 10. M = B[0.6. g (x) = a f (t) 1 t dt for x ∈ (0.55 on p. (10. m be Lebesgue measure on [0.11. M1 ⊗M2 ⊗M3 ) is a “measure theoretic isomorphism. Mj .14.16. On the integrability of a b |x| |log |x|| for x near 0 and x near ∞ in Rn . Hint: Look at the proof of Theorem 5. (Counter example related to Fubini Theorem involving counting measures. 69. Let (Xj . x3 ) = (x1 .1] be the Borel σ – ﬁeld on X. Xn dµ (xn ) 1A (x1 . m) and Exercise 10. 0 Exercise 10.) Exercise 10.19. (Explicit integrations. Folland Problem 2. makes sense and is correct. dm) and x a a and then show Eq. d Hint: show S d−1 2 ωi dσ (ω ) is independent of i and therefore S d−1 2 ωi dσ (ω ) = 1 d d S d−1 2 ωj dσ (ω ) . 77. Folland Problem 2. x3 ). Exercise 10. π (A) = [(µ1 ⊗ µ2 ) ⊗ µ3 ] (F −1 (A)) for all A ∈ M1 ⊗ M2 ⊗ M3 .16) holds. Fractional integration. (Note the M × BR should be M ⊗ BR ¯ in this problem. (M1 ⊗ M2 ) ⊗ M3 ) – measurable.57 on p.) Also show the above identity holds for any one of the six possible orderings of the iterated integrals. ∞] be a (M1 ⊗ M2 ⊗ M3 . 1.12. f dπ = X1 ×X2 ×X3 X3 x x dm(x) = ∞. 3. . y )dν (y ) dm(x) = X X X X 1 δij σ S d−1 . Finally let D = {(x. Let π := F∗ [(µ1 ⊗ µ2 ) ⊗ µ3 ] .46 on p. ν (A) = #(A). 77.106 10 Multiple and Iterated Integrals 10. Suggestion. i. Prove Theorem 10.cls date/time: 23-Feb-2007/15:20 . (Part of Folland Problem 2. That is F : ((X1 × X2 ) × X3 . x2 . µj ) for j = 1.13.17. Exercise 10. x) ∈ X 2 : x ∈ X } be the diagonal in X 2 . Exercise 10. M1 ⊗ M2 ⊗ M3 ) – measurable and F −1 is (M1 ⊗ M2 ⊗ M3 .9. (Optional. 77.9. Exercise 10. 77. Show 1D (x. Folland Problem 2. Let f ∈ L1 ((0.61 on p. Let F : (X1 × X2 ) × X3 → X1 × X2 × X3 be deﬁned by F ((x1 . 1] and ν be counting measure.58 on p. using Problem 10. xn ) Exercise 10. Use the case of two factors as the model of your proof.60 on p. That is show md is the unique translation invariant measure on BRd such that md ((0. . y )dm(x) dν (y ) by explicitly computing both sides of this equation.5. x2 ). Folland Problem 2.

µ (|fn − fm | ≥ ε) ≤ p where f ∼ g iﬀ f = g a. fn → 0. 2.e.e. then it is Lp – Cauchy. µ (|f | ≥ ε) = µ (|f | ≥ εp ) ≤ and therefore if {fn } is Lp – Cauchy. If a sequence {fn }n=1 is Lp – convergent. we have fn − fm p µ = inf {a ≥ 0 : µ(|f | > a) = 0} (11. µ) = {f : Ω → C : f is measurable and f p < ∞}/ ∼ The case where p = 0 will be handled in Theorem 11. {fn } is Cauchy in Lp if limm. Lp (Ω.e. ∞).3). fn → f in Lp iﬀ f ∈ Lp and fn ∈ Lp for all n. {fn } is Cauchy in µ – measure (or L0 – Cauchy) if limm.e. When µ is a probability measure.e. let ∞ p := Ω |f | dµ p (11.7 below. n → ∞ εp showing {fn } is L0 – Cauchy. We have the following notions of convergence and Cauchy sequences. when p ∈ [1. Deﬁnition 11. Proof. Conversely. Notice that f − g p = 0 iﬀ f ∼ g and if f ∼ g then f p = g p . then 1 fn − fm p p → 0 as m. n → ∞. ∞] and fn → f in Lp . fn → f a. A similar argument holds for the Lp – convergent case. 3. µ(|f | > a) = 0 and therefore µ(|f | > M ) = limn→∞ µ(|f | > M + 1/n) = 0. i.11 Lp – spaces Let (Ω. if |f | ≤ M a.e. and limn→∞ fn − f p = 0. Suppose that f ∞ ≤ M. 2. For example. This leads to the identity: f ∞ 1 εp |f | dµ = Ω p 1 f εp p p = inf {a ≥ 0 : |f (ω )| ≤ a for µ – a. Let p ∈ [1.4 (Lp – convergence implies convergence in probability). fn ∞ 0 in L1 . Cauchy if there is a set E ∈ B such that µ(E ) = 0 and{1E c fn } is a pointwise Cauchy sequences. fn → f in µ – measure if limn→∞ µ(|fn − f | > ε) = 0 for all ε > 0.n→∞ µ(|fn − fm | > ε) = 0 for all ε > 0. {fn } is a. let f For 0 < p ≤ ∞. In general we will (by abuse of notation) use f to denote both the function f and the equivalence class containing f. Here is a sequence of functions where fn → 0 a. ω.1 Modes of Convergence Let {fn }n=1 ∪ {f } be a collection of complex valued measurable functions on Ω.3.1) Deﬁnition 11.n→∞ fn − fm p = 0. |f (ω )| ≤ M for µ a.2) ≤ fn − f p + f − fm p → 0 as m. Remark 11.e.1. fn → f as fn converging ∞ to f in probability. and a > M then µ(|f | > a) = 0 and hence f ∞ ≤ M. then for all a > M. 11.2. B . B . By Chebyshev’s inequality (8. we describe. m . 1. Lemma 11. if there is a set E ∈ B such that µ(E ) = 0 and limn→∞ 1E c fn = 1E c f.e. We µ will abbreviate this by saying fn → f in L0 or by fn → f. 1. If {fn } ⊂ Lp is Lp – convergent (Cauchy) then {fn } is also convergent (Cauchy) in measure. ω } .. µ) be a measure space and for 0 < p < ∞ and a measurable function f : Ω → C let 1/p f and when p = ∞. 3.

µ(E ) < k ε2 1 Moreover. then µ(Ek ) < ε2−k . yet fn measure. k=1 Ek .. fn fn → 0. we have |fn (ω ) − f (ω )| ≤ k for all n ≥ Nk and all k. (11. 0 = µ({|fn − f | > ε i.5 (Egoroﬀ ’s Theorem: almost sure convergence implies convergence in probability).e. Proof. then satisﬁes the estimate. or in Theorem 11. That is fn → f uniformly on E c . E := ∪∞ = ε.108 11 Lp – spaces Above is a sequence of functions where fn → 0 in L1 . Then for all ε > 0. Page: 108 job: prob macro: svmonob.o. Let fn → f a.cls date/time: 23-Feb-2007/15:20 . such that. Then for all ε > 0 there exists E = Eε ∈ µ B such that µ(E ) < ε and fn → f uniformly on E c . ≥ lim sup µ ({|fN − f | > ε}) N →∞ from which it follows that fn −→ f as n → ∞. To get the uniform convergence oﬀ a small exceptional set. fn → 0 but fn m 0 in L1 . 0 in L .. Suppose µ(Ω ) = 1 and fn → f a. n}) = lim µ N →∞ n≥N (11.3) {|fn − f | > ε} Here is a sequence of functions where fn → 0 a.e. −k The set. for ω ∈ / E. the equality in Eq.e. In particular fn −→ f as n → ∞.e. if Ek := n≥Nk µ |fn − f | > 1 k .. and Above is a sequence of functions where fn → 0 a.s.3) allows us to choose an ∞ increasing sequence {Nk }k=1 . m 1 0 a.

If fn → f then {fn }n=1 is Cauchy in measure.o. {fn }n=1 ∞ ∞ of {fn }n=1 has a further subsequence. 5. Therefore we have shown.e. Another application of Lemma 11. ε > 0 and m. f. Suppose {fn } is L0 (µ) – Cauchy and let εn > 0 such that (εn = 2−n would do) and set δn = subsequence of N such that µ({|gj +1 − gj | > εj }) ≤ εj . and a subsequence gj = fnj of {fn } such that limj →∞ gj := f exists a. Proof. i.6. Let m → ∞ in (11. there exists a measurable function. B .6 shows |f (ω ) − gj (ω )| ≤ δj for all j ≥ N. {an } is Cauchy. Theorem 11. µ(|f − g | > ε) ≤ µ(|f − fn | > ε/2) + µ(|g − fn | > ε/2) → 0 as n → ∞. {fn }n=1 . (11. Page: 109 job: prob macro: svmonob. |gj +1 (ω ) − gj (ω )| ≤ εj for all j ≥ N. Choose gj = fnj where {nj } is a µ({|gj +1 − gj | > εj }) ≤ j =1 j =1 εj < ∞. n → ∞. Let (Ω. Next we will show gN → f as N → ∞ where f and gN are as above. If f and g are measurable functions and fn → f and fn → g then f = g a. n → ∞. µ 4. it follows from the ﬁrst Borel-Cantelli lemma that 0 = µ (E ) = lim µ (FN ) .e. 1.n) → 0 as .4) {|fn − fm | > ε} ⊂ {|fn − f | > ε/2} ∪ {|fm − f | > ε/2} and hence µ (|fn − fm | > ε) ≤ µ (|fn − f | > ε/2)+µ (|fm − f | > ε/2) → 0 as m. Suppose fn → f. m.6.e. 1 n ∞ µ µ µ ∞ εn < ∞ n=1 εk . n ∈ N and ω ∈ Ω are such that |fn (ω ) − fm (ω )| > ε. Suppose an ∈ C and |an+1 − an | ≤ εn and ∞ n→∞ n=1 εn < ∞. Then 2. Let m > n then m−1 m−1 ∞ |am − an | = k=n (ak+1 − ak ) ≤ k=n |ak+1 − ak | ≤ k=n εk := δn . Suppose that f and g are measurable functions such that fn → g and µ fn → f as n → ∞ and ε > 0 is given.e. lim an = a ∈ C exists and |a − an | ≤ δn := k=n εk .1 Modes of Convergence ∞ µ 109 Lemma 11. Let FN := ∪j ≥N {|gj +1 − gj | > εj } and E := ∩∞ N =1 FN = {|gj +1 − gj | > εj i. In this case.e.a.cls date/time: 23-Feb-2007/15:20 . 1. then fn → f. which is almost surely convergent to f. Since {|f − g | > ε} = {|f − fn + fn − g | > ε} ⊂ {|f − fn | + |fn − g | > ε} ⊂ {|f − fn | > ε/2} ∪ {|g − fn | > ε/2} . If {fn }n=1 is Cauchy in measure. Proof.7. Let us now further assume that µ (Ω ) < ∞.11. {fn }n=1 converges to f in probability iﬀ every subsequence. 3.e. ∞ 3. If ≤ n=1 µ |f − g | > 1 n = 0. For ω ∈ E we may deﬁne f (ω ) ≡ 0. |gj +1 (ω ) − gj (ω )| ≤ εj for a. f = g a. f (ω ) := lim gj (ω ) exists. then c ω ∈ FN = ∩j ≥N {|gj +1 − gj | ≤ εj } . N →∞ j →∞ For ω ∈ / E. Since ∞ ∞ ∞ k=n ∞ So |am − an | ≤ δmin(m. µ ∞ 4.4) to ﬁnd |a − an | ≤ δn . If {fn }n=1 is Cauchy in measure and f is as in item 3. µ) be a measure space and {fn }n=1 be a sequence of measurable functions on Ω. j and so by Lemma 11. µ ∞ 2.} and observe that µ (FN ) ≤ δN < ∞. a sequence of func∞ ∞ tions. i. Then ε < |fn (ω ) − fm (ω )| ≤ |fn (ω ) − f (ω )| + |f (ω ) − fm (ω )| from which it follows that either |fn (ω ) − f (ω )| > ε/2 or |f (ω ) − fm (ω )| > ε/2. Hence µ(|f − g | > 0) = µ ∪∞ n=1 |f − g | > i.

µ) is a probability space. First notice that |f | ≤ g a. y ) = ϕ (x) .1 (Fatou’s Lemma). Any subsequence of {fnk } would have the same property and hence can not be almost surely convergent because of Theorem 11. Let (Ω. since {|fn − f | > ε} = {|f − gj + gj − fn | > ε} ⊂ {|f − gj | + |gj − fn | > ε} ⊂ {|f − gj | > ε/2} ∪ {|gj − fn | > ε/2}. and µ 4. fn → f in L1 . y ) = x + y. B . Taking complements of this equation shows {|f − gN | > δN } ⊂ ∪j ≥N {|f − gj | > δj } ⊂ FN . an application of the dominated convergence Theorem 8. {fn } ⊂ Lp (µ) µ p p and f ∈ Lp (µ) . p by Corollary 11. gN → f as N → ∞. = 0. p p p For the converse. and therefore. µ 2. Then 1. and hence f ∈ L1 since g ∈ L1 . Noting. then Ω f dµ ≤ lim inf n→∞ Ω fn dµ.. 3. Page: 110 1 µ µ gn → g as n → ∞. B . Fn ≤ Gn ∈ L1 . all follow from item 2. y ) = x · y respectively. To see that |f | ≤ g. Solution to Exercise (11.e.8 (Dominated Convergence Theorem). fn · gn −→ f · g. then there exists an ε > 0 and a subsequence. If fn ≥ 0 and fn → f in measure. {nk } such that inf k µ (|f − fnk | ≥ ε) > 0. By the triangle inequality. and Gn → G where G := 2p |f | ∈ L1 . gn ) −→ ψ (f. Exercise 11. Therefore. {gn } . Corollary 11. and 4. So it suﬃces to prove item 2.7 again. and ψ (x.5).9. f − fn p because {fn }n=1 was Cauchy in measure. Proof. Indeed. Therefore. |f − fn | = Fn → 0 = 0. To do this we will make repeated use of Theorem 11. letting j → ∞ in this inequality gives. ∞ Conversely if {fn }n=1 does not converge to f in probability. ∞). Then µ p Fn −→ 0. there is a ∞ ∞ further subsequence. k→∞ k→∞ If (for sake of contradiction) limn→∞ f − fn subsequence {fnk } of {fn } such that 1 = 0 there exists ε > 0 and a |f − fnk | ≥ ε for all k. B . we may assume (by passing to a further subsequences if necessary) that fnk → f and gnk → g almost everywhere. Then fn → f in Lp (µ) iﬀ fn −→ f and |fn | → |f | .e. Suppose {fn } . µ) be a measure space. let Fn := |f − fn | and Gn := 2p−1 [|f | + |fn | ] . use Theorem 11. {fn }n=1 is also Cauchy in probability. |f − fnk | ≤ g + gnk → 2g and (g + gnk ) → 2g. µ With this in hand. Moreover Chebyschev’s p p inequality implies fn −→ f if fn → f in Lp . ϕ (fn ) −→ ϕ (f ) . {fn }n=1 of {fn }n=1 which is convergent almost surely. Hence by item 3. ∞ 5. In particular job: prob macro: svmonob.7 to ﬁnd subsequences {fnk } and {gnk } of {fn } and {gn } respectively which are almost everywhere convergent.cls date/time: 23-Feb-2007/15:20 . µ) be a measure space.e. Suppose (Ω.e. Let (Ω. by taking ψ (x. µ µ µ Corollary 11. Item 1.. If {fn }n=1 is convergent and hence Cauchy in probability then any subse∞ quence. we have µ({|fn − f | > ε}) ≤ µ({|f − gj | > ε/2}) + µ(|gj − fn | > ε/2). Let (Ω. Exercise 11. µ 3.34 implies limk→∞ |f − fnk | = 0 which contradicts Eq. and Then f ∈ L1 and limn→∞ f − fn limn→∞ fn = f. ψ (x. µ(|f − gN | > δN ) ≤ µ(FN ) ≤ δN → 0 as N → ∞ and in particular.2. i.7. g ) . ∞ f p − fn p ≤ which shows µ |fn | → p |f | if fn → f in L . B . µ({|fn − f | > ε}) ≤ lim sup µ(|gj − fn | > ε/2) → 0 as n → ∞ j →∞ µ Proof. fn + gn −→ f + g. p ∈ [1. fn −→ f. µ) be a measure space.5) Using Theorem 11. fn −→ f and gn −→ g and ϕ : R → R and ψ : R2 → R are continuous functions.2). and g are in L1 and f ∈ L0 are functions such that |fn | ≤ gn a. (11. ψ (fn .5. gn −→ g. (11.110 11 Lp – spaces c FN ⊂ ∩j ≥N {ω ∈ Ω : |f (ω ) − gj (ω )| ≤ δj } . Then |f | = lim |fnk | ≤ lim gnk = g a.8. it is straightforward to show fn → f.

n and then Pn Pn s1 . p. q ∈ (1.8) follows from the inequality. Integrating Eq. (11. So we now assume that p. Hence.12 (H¨ older’s inequality). t. it now follows that k→∞ p p f dµ Ω ≤ Ω |f | dµ ≤ Ω |f | dµ. Also suppose that f ∈ L1 (µ). µ = i=1 p δi and i pi f (i) := ln si .10 (Jensen’s Inequality). (11. take pi = n and si = ai with ai > 0. Suppose that 1 ≤ p ≤ ∞ and q := p −1 . i. si > 0 for i = 1. √ n a1 . b) and let β ∈ R (β = ϕ ˙ (t) when ϕ ˙ (t) exists). (11. then ϕ ◦ f is integrable in the extended sense and ϕ ( f ) dµ = ∞ . or equivalently p + q −1 = 1. p + q = 1) then n st ≤ 1 p 1 q s + t .6) log(|f |)dµ ≤ log Ω Proof.10) is again easily veriﬁed.e. be such that ϕ(s) − ϕ(t) ≥ β (s − t) for all s ∈ (a. ∞) and f p · g q < ∞. then ϕ(f ) ≥ ϕ(t) + β (f − t) which shows that negative part of ϕ(f ) is integrable. we have applied Eq. µ) is a probability space. of N there is a subsequence. |f | dµ Ω (11. . . . Taking s = |f | / f p and t = |g |/ g q in Eq. (i. . 2. if ϕ(f ) is not integrable.10) p ϕ(f )dµ − ϕ(t) = Ω ϕ(f )dµ − ϕ( Ω f dµ). and ϕ : (a.) Then ϕ Ω 1 Indeed. we have the following inequalities exp Ω = g q q |f | a. Let t = Ω f dµ ∈ (a. . The cases p = 1 and q = ∞ or p = ∞ and q = 1 are easy to deal with and will be left to the reader. Ω Proof. H¨ older’s and Minikowski’s Inequalities 111 Given a subsequence. Eq. (11.e. the inequality in Eq.12) implies macro: svmonob. . implies that 0≤ Ω (When p = q = 1/2.7).e. Therefore. 2 0 ≤ (s − t) .12) f p g q p f p q g q with equality iﬀ |g/ g q | = |f | / f p p g q q |f | .) 1/n As another special case of Eq. i=1 (11. p (11.2 Jensen’s. b). µ is a positive measure and µ(Ω ) = 1.2 Jensen’s. ∞) .9) Theorem 11. g p < ∞. Page: 111 job: prob date/time: 23-Feb-2007/15:20 . Suppose that (Ω. then we get the arithmetic geometric mean inequality. . and yet a further subsequence {nk } of {nk } such that gnk → g a. pi (11. equality holds in Eq.7) 11. (11. Ω ϕ(f )dµ = ∞ in this case. . q ∈ (1. (11.) Then integrating the inequality.6) with Ω = {1. p |f g | 1 |f | 1 |g |q ≤ + (11.8) gives. . . (11. (11. i. (11. .11.s. B .38 below for the existence of such a β in the general case. − ln x for x > 0. and xp for x ≥ 0 and p ≥ 1 are all convex functions. . b) → R is a convex function. b). As a special case of Eq. So we will now assume that 0 < f q .8) f dµ ≤ Ω ϕ(f )dµ where if ϕ ◦ f ∈ / L1 (µ).2 when ϕ is C 1 and Theorem 11. As a special case of Eq.s. n 1 i=1 pi p lim ψ fnk .11) f dµ ≤ Ω ef dµ.10) iﬀ |f | q and |g | are linearly dependent as elements of L1 which happens iﬀ |g |q f p p Moreover. by the continuity of ψ. Example 11. if pi .e. n} . f : Ω → (a. {nk } of {nk } such that fnk → f a.6). an ≤ 1 n n ai . g ) a.7).11. gnk = ψ (f.31) and Figure 7. Assuming p ∈ (1. suppose that s. (11. {nk } . b) . Since ex for x ∈ R.cls p−1 (p−1) = |f | p/q / f p/q p . |g |q f p p = and for p ≥ 1. ϕ (x) ≥ 0 on (a. If f and g are measurable functions then p−1 fg 1 ≤ f p · g q.s. ∞) p 1 1 with q = p− 1 (i. ln si i p n ≤ i=1 1 ln sp i e i = pi n i=1 i sp i .e. which completes the proof. sn = e i=1 ln si = e i=1 1 pi = 1. (See Lemma 7. 2. p q (11. H¨ older’s and Minikowski’s Inequalities Theorem 11. ϕ(f ) − ϕ(t) ≥ β (f − t). If f q = 0 or ∞ or g p = 0 or ∞.

In light of Example 11. Alternatively we may give a proof along the lines of the proof of Theorem 11.13) Indeed. |f + g | ≤ (2 max (|f | . which implies1 f + g ∈ Lp since f +g p p p p p p p p ∞ ∞ a.15). We may assume f + g p . Integrating |f + g |p = |f + g ||f + g |p−1 ≤ (|f | + |g |)|f + g |p−1 and then applying Holder’s inequality with q = p/(p − 1) gives |f + g |p dµ ≤ Ω Ω are fi i=1 r ≤ i=1 fi pi . n and p1 .cls date/time: 23-Feb-2007/15:20 . . (11. Page: 112 job: prob macro: svmonob. f p and g all positive since otherwise the theorem is easily veriﬁed. . ∞) . sr n pi r ≤ i=1 i (sr i) pi /r p /r n =r i=1 i sp i . (11. . One may prove this theorem by induction based on H¨ older’s Theorem 11. Since Eq. ≤( f where |f + g |p−1 q q + g p ) |f + g | (11. (11. . the last 2p in the above inequality may be replaced by 2p−1 .13. .16) sr 1 .12 which is what we will do here.e. hence we may assume that fi pi > 0 for all i. (11. then i n n We now consider p ∈ (1.17) Combining Eqs. The proof is ﬁnished since it is easily checked p q q p that equality holds in Eq. Example 11. {1. |g | ) ≤ 2p (|f | + |g | ) . If 1 ≤ p ≤ ∞ and f.17) implies f +g p p Now replace si by |fi | / fi to ﬁnd 1 n i=1 n in the previous inequality and integrate the result n ≤ f p f +g p p/q p + g p f +g p/q p (11. pi = Ω (|f + g |p−1 )q dµ = Ω |f + g |p dµ = f + g p p. n} equipped with counting measure. (11. ak · 1 ≤ k=1 |ak | p k=1 1 q =n 1/q k=1 |ak | p ≤ 2p f p p + g p p < ∞. then n p n Theorem 11.14) Proof.15 (Minkowski’s Inequality). When p = ∞. . we have n n n 1/p n 1/q n 1/p ak = k=1 k=1 p p−1 .11) holds. ak k=1 ≤ np−1 k=1 |ak | . p Theorem 11.14 (Generalized H¨ older’s inequality). |g |)) = 2p max (|f | . so that |f + g | ≤ ≤ f ∞ + g ∞ . . . i=1 p i replace si by sr and p by p /r for each i in Eq. .12 above. Suppose that fi : Ω → C are measurable functions for i = 1. .10) when |f | = c |g | of |g | = c |f | for some constant c. . n and p ∈ [1.112 11 Lp – spaces fg f p 1 g ≤ q 1 1 + =1 p q with equality iﬀ Eq.e. Suppose that ak ∈ C for k = 1.e.7) to ﬁnd i i i n |f | |f + g |p−1 dµ + p |g | |f + g |p−1 dµ Ω p−1 q. . . ∞). .18) fi fi pi i=1 r ≤r i=1 1 1 i pi fi p pi n |fi | i dµ = Ω i=1 p r = 1. Eq. . 2. (11. pn and r are positive n 1 numbers such that i=1 p− = r−1 . (11. Furthermore. g ∈ Lp then f + g p ≤ f p + g p.14). and |g | ≤ g |f | + |g | ≤ f ∞ + g ∞ a.16) and (11. by H¨ older’s inequality applied using the measure space. we will n ri = 1.15) Proof. |f | ≤ f ∞ a.13. (11. when p = 1 we have f +g 1 where q = Taking the pth – power of this inequality then gives. By assumption. pi Solving this inequality for f + g 1 gives Eq. p (11. 2.14) is easily seen to hold if fi pi = 0 for some i. . . (11. and therefore f +g When p < ∞. . (11. = Ω |f + g |dµ ≤ Ω |f | dµ + Ω |g |dµ = f 1 + g 1.

L (µ) equipped with the Lp – norm.e.18 below. (2) If µ is counting measure. The Lp (µ) – norm controls two types of behaviors of f.n } and E := ∪Em. Then for all k ∈ N. Conversely. This shows that fn → f uniformly on E c .11.e. then locally near x0 it is harder for f to be in Lp (µ) as p increases. ∞ Let {fn }n=1 ⊂ Lp (µ) be a Cauchy sequence.e. there are two notable exceptions. (11. · p satisﬁes the triangle inequality. then ∞ (L∞ (Ω.4). the case p = ∞ being done in Theorem 11. µ(A) = #(A).n → 0 as m. For 1 ≤ p ≤ ∞. 11. Theorem 11. there exists Nk < ∞ such that µ |f − fn | > k Let E= ∪∞ k=1 ∪n≥Nk |f − fn | > k c −1 −1 −1 gj − f p p = k→∞ k→∞ lim inf |gj − gk |p dµ ≤ lim inf k→∞ p p |gj − gk |p dµ = lim inf gj − gk In particular. Letting f = limn→∞ 1E c fn .7 there exists a subsequence {gj } of {fn } such that gj → f a. Let · ∞ be as deﬁned in Eq. By Minkowski’s Theorem 11. Corollary 11. fn − f p p → 0 as j → ∞. On the other hand a function f ∈ Lp (µ) is allowed to decay at “inﬁnity” slower and slower as p increases. B . x∈E c Therefore. if there exists E ∈ B such that µ(E ) = 0 and fn → f uniformly on E c . {fn } is L0 -Cauchy (i. n → ∞.e. see Corollary 11. Proof. Take a ∈ [1.1)). if f blows up at a point x0 ∈ Ω.16.cls date/time: 23-Feb-2007/15:20 .2).5 for an important example of the use of this theorem. so there are no local singularities. i. Proof. However. f p = f ·1 p ≤ f q · 1 a = µ(Ω )1/a f q = µ(Ω )( p − q ) f 1 1 q. In this case Lp (µ) ⊂ Lq (µ) for all q ≥ p.n .n = {|fn − fm | > εm.17 (Completeness of L (µ)).15. we should not in general expect Lp (µ) ⊂ Lq (µ) or Lq (µ) ⊂ Lp (µ). = 0 for all n ≥ Nk . < ∞ so the f ∈ Lp and gj −→ f. The reader may easily check this ﬁnal formula is correct even when q = ∞ provided we interpret 1/p − 1/∞ to be 1/p.” So in particular. the inclusion map is bounded and in fact f p 1 −1 ≤ [µ(Ω )]( p q ) f Then µ(E ) = 0 and for x ∈ E . µ).15. Cauchy in measure) and by Theorem 11. i. then there is no behavior at inﬁnity to worry about and Lq (µ) ⊂ Lp (µ) for all q ≥ p as is shown in Corollary 11. f p ≤ gj − f proof is ﬁnished because. ∞] such that 1 1 1 pq = + . i.14. then all functions in Lp (µ) for any p can not blow up on a set of positive measure. it then follows that limn→∞ fn − f ∞ = 0. That is to say lim sup f − fn j →∞ ∞ ≤ ε for all ε > 0.3 Completeness of Lp – spaces Theorem 11. f − fn ∞ → 0 as n → ∞. then µ(E ) = 0 and sup |fm (x) − fn (x)| ≤ εm.4 Relationships between diﬀerent Lp – spaces 113 11. |f (x) − fn (x)| ≤ k for all n ≥ Nk . is a Banach space. By Fatou’s Lemma. n → ∞. The → 0 as j.16. µ (|f − fn | ≥ ε) = µ ({|f − fn | ≥ ε} ∩ E c ) = 0 for all n suﬃciently large.e. then for any ε > 0. Proof. As above the reader may easily check the remaining conditions that ensure · p is a norm. macro: svmonob. (11. Suppose εm. a = . Moreover. The density of simple functions follows from the approximation Theorem 6. By Chebyshev’s inequality (Lemma 11.23 below. namely the “behavior at inﬁnity” and the behavior of “local singularities. So we are left to prove the completeness of Lp (µ) for 1 ≤ p < ∞.18. Let Em. By Minkowski’s Theorem 11. So the last item to prove is the completeness of L∞ . With these insights in mind. n → ∞. · p (see Eq. Lp + gj p ≤ fn − gj p + gj − f p See Proposition 12. · ∞ ) is a Banach space.34. f := limn→∞ fn exists on E c and the limit is uniform on E c .n := fm − fn ∞ → 0 as m. The reader may easily check the remaining conditions that ensure · ∞ is a ∞ norm. · ∞ satisﬁes the triangle inequality. bounded simple functions are dense in L∞ . then Lq (µ) ⊂ Lp (µ). p a q q−p Then by Theorem 11. Suppose that {fn }n=1 ⊂ L∞ is a sequence such fn → f ∈ L∞ . A sequence {fn }n=1 ⊂ L∞ con∞ verges to f ∈ L iﬀ there exists E ∈ B such that µ(E ) = 0 and fn → f uniformly on E c . If µ(Ω ) < ∞ and 0 < p < q ≤ ∞. Page: 113 job: prob p p q .4 Relationships between diﬀerent Lp – spaces . (1) If µ(Ω ) < ∞.

1 n Hence it follows that lim a = lim = en 1 n ap i =1+p i=1 1 n n Then (Lp0 + Lp1 . an . an ) with ai > 0 for i = 1. . for p > 0. Proof. Indeed. .114 11 Lp – spaces The rest of this section may be skipped. ln ai + O p2 . . . So for p < 0. p1 1|f |≤M ap i i=1 ≤M p 2 f M pλ 1|f |≤M ≤ M p1 −pλ f A little algebra shows that λ may be computed in terms of p0 . = 1 1 n n i=1 1 ai q Conclusion. .cls date/time: 23-Feb-2007/15:20 . Let a := (a1 . and h p1 p1 ≤ M p0 1|f |>M ≤ M p0 −pλ f <∞ So if we now deﬁne a 0 := a1 . in fact f ≤ 2 f pλ for all f ∈ Lpλ . i maxi (1/ai ) 1/m p ap i i=1 . n and for p ∈ R \ {0} . √ We now claim that limp→0 a p = n a1 . λ ∈ (0. q = −p decreases. . 2. p → a we have 1 n n q a− i i=1 p is increasing in p for p > 0. Then by Corollary 11. 1 1 p M ≤ n n and therefore. Let M > 0. We will now show that limp→∞ a p = maxi ai =: M and limp→−∞ a p = mini ai =: m. .e. then the local singularities of f are contained in the set E := {|f | > M } and the behavior of f at “inﬁnity” is solely determined by f on E c . 1/an ) . e + h p1 :f =g+h . let a := 1 n n 1/p 1 Since n we have 1/p → 1 as p → ∞. . p→−∞ lim a p = lim q →∞ = 1 1 = = m = min ai . job: prob macro: svmonob. so where a 1 1 −1 that a q is decreasing and hence a is increasing. If we extend the deﬁnition of a p to p = ∞ and p = −∞ ¯ p → a ∈ (0. . an . By our earlier discussion we expect that g ∈ Lp0 and h ∈ Lp1 and this is the case since. ∞) is a by a ∞ = maxi ai and a −∞ = mini ai . pλ p1 − p0 M≤ a p ≤ M. ∞) is continuous and increasing in p. as p increases.20. . Example 11. pλ and p1 by λ= p0 p1 − pλ · . write ap i = 2 p ln ai = 1 + p ln ai + O p for p near zero. then R p continuous non-decreasing function of p. . it follows that limp→∞ a 1 1 a q p = M. p1 ) be deﬁned by 1 1−λ λ = + (11. 1 n Page: 114 1/p n = f 1|f |≤M ≤ M p1 p1 p1 = |f | p1 1|f |≤M = M p1 pλ pλ f M < ∞. Hence let g = f 1E and h = f 1E c so that f = g + h. Proposition 11. every function f ∈ Lpλ may be written as f = g + h with g ∈ Lp0 and h ∈ Lp1 .19) pλ p0 p1 with the interpretation that λ/p1 = 0 if p1 = ∞. Hence we have shown q that p → a p is increasing for p ∈ R \ {0} .2 Then Lpλ ⊂ Lp0 + Lp1 . 1) and pλ ∈ (p0 . . For 1 ≤ p0 < p1 ≤ ∞ and f ∈ Lp0 + Lp1 let f := inf g p0 −1/q 1/q = 1 a −1 q a p := 1 := (1/a1 . . For p = −q < 0. . i. an . i=1 p→0 p p→0 Pn 1 n n 1/p ap i i=1 ln ai = lim √ n √ n p→0 1 1+p n n 1/p ln ai + O p i=1 2 g p0 p0 = |f | p0 1|f |>M = M p0 f M pλ f M p0 1|f |>M pλ pλ i=1 = a1 . To prove this.19 (Power Inequalities). For p = −q < 0. . . . the map p ∈ R → a p ∈ (0.18. Therefore. . · ) is a Banach space and the inclusion map from Lpλ to Lp0 + Lp1 is bounded. Suppose that 0 < p0 < p1 ≤ ∞.

Let λ be determined as above. For 0 < p ≤ q ≤ ∞. then f has local singularities no worse than an Lp1 function and behavior at inﬁnity no worse than an Lp0 function.e. B . The estimate in Eq. · ) is a Banach space is left as Exercise 11. L0 – convergence implies almost everywhere convergence for some subsequence. and for f ∈ Lp0 ∩ Lp1 let f := f p0 p i. p1 ) as in Eq. Suppose now that µ is counting measure on Ω. λ ∈ (0.5 to the reader. (11. Then Lp (µ) ⊂ Lq (µ) for all 0 < p < q ≤ ∞ and f q ≤ f p . 3. p Since µ(|f | > ε) ≤ ε−p f p .6 to the reader.21 (Interpolation of L – norms). Corollary 11.20) f pλ ≤ f p0 f p1 . To show this space is complete. f pλ 11. +λ 1−pλ /p1 f pλ and then taking λ = 1 shows f ≤ 2 f pλ . p1 ) be deﬁned as in Eq. f = g a.5 Uniform Integrability This section will address the question as to what extra conditions are needed in order that an L0 – convergent sequence is Lp – convergent. Suppose that 0 < p < q = ∞. B ).22. To simplify matters a bit here.19). p1 ). Combining Proposition 11. B ∈ B let µ(f : A. apply Corollary 11. It is easily checked that · is a norm on Lp0 ∩ Lp1 .cls date/time: 23-Feb-2007/15:20 .7. 1. Hence there exist f ∈ Lp0 and g ∈ Lp1 such that limn→∞ f − fn p0 = 0 and limn→∞ g − fn pλ = 0. Suppose that 0 < p0 < p1 ≤ ∞.e. let µ(f : E ) := E = |f | |f | λ 1−λ pλ ≤ |f | λ a |f | 1−λ b = f λ p0 f 1−λ p1 . The proof that (Lp0 + Lp1 . suppose that {fn } ⊂ Lp0 ∩ Lp1 is a · – Cauchy sequence. λ ∈ (0. If p ≤ q. B ) := A∩B f dµ. Hence f ∈ Lpλ for any pλ between p0 and p1 . 2. (11. then Lp0 ∩ Lp1 ⊂ Lpλ and 1−λ λ (11. 1) and pλ ∈ (p0 . 6. then p ⊂ q and f q ≤ f p .1 Summary: Lp0 ∩ Lp1 ⊂ Lq ⊂ Lp0 + Lp1 for any q ∈ (p0 . (1 − λ)−1 f p0 + f p1 .19). Page: 115 job: prob When µ is a probability measure.20 and Corollary 11. Notation 11.24 For f ∈ L1 (µ) and E ∈ B . 5. 1) and pλ ∈ (p0 . This will lead us to the notion of uniform integrability. 4. in fact f pλ ≤ max λ−1 . µ) is a ﬁnite measure space for this section. Then (Lp0 ∩ Lp1 . Corollary 11.5 Uniform Integrability 115 Moreover this shows f ≤ M 1−pλ /p0 f Taking M = λ f pλ pλ /p0 pλ + M 1−pλ /p1 f pλ /p1 pλ . (11. · ) is a Banach space and the inclusion map of Lp0 ∩ Lp1 into Lpλ is bounded. a = p0 /λ and b = p1 /(1 − λ). and more generally if A. then Lq ⊂ Lp for all p ≤ q and Lq – convergence implies Lp – convergence.21) The heuristic explanation of this corollary is that if f ∈ Lp0 ∩ Lp1 . Proof.4. macro: svmonob. It now is clear that limn→∞ f − fn = 0. then f p ∞ p p p p then gives f ≤ λ 1−pλ /p0 = sup {|f (x)| : x ∈ Ω } ≤ x∈Ω |f (x)| = f . Remark 11. + f p1 . then by Theorem 11. it will be assumed that (Ω.4) fn → f and fn → g in measure and therefore by Theorem 11. B ] for µ(f : A. 11. Further assume 1 ≤ p0 < pλ < p1 ≤ ∞. If µ(Ω ) < ∞.21 gives L p0 f dµ. ∩L p1 ⊂L pλ ⊂L p0 +L p1 for 0 < p0 < p1 ≤ ∞.23. Then {fn } is both Lp0 and Lp1 – Cauchy.11. we will often write E [f : E ] for µ(f : E ) and E [f : A. Proof. (11.21 with p0 = p and p1 = ∞ to ﬁnd f q ≤ f p/q p f 1−p/q ∞ ≤ f p/q p f 1−p/q p = f p . By Chebyshev’s inequality (Lemma 11. Lp – convergence implies L0 – convergence. If µ(Ω ) < ∞ then almost everywhere convergence implies uniform convergence oﬀ certain sets of small measure and in particular we have L0 – convergence.14.21) is left as Exercise 11. f ∞ ≤ f p for all 0 < p < ∞.

Proof. Proof.27. Λ ⊂ L1 (µ) is said to be uniformly absolutely continuous if for all ε > 0 there exists δ > 0 such that sup µ (|f | : E ) < ε whenever µ (E ) < δ. then for f ∈ Λ f 1 This shows µ (|g | : E ) < ε provided that µ (E ) < ε (2 |ci |) −1 . then {fn + gn }n=1 is also uniformly integrable. for ﬁnite measure spaces without “atoms”. For example take Ω = {∗} and µ({∗}) = 1. = µ (|f | : |f | ≥ a) + µ (|f | : |f | < a) ≤ 1 + aµ (Ω ) . there would exist ε > 0 and sets En such that µ(En ) → 0 while µ(|g | : En ) ≥ ε for all n. n i=1 The condition in Eq. Then for f ∈ Λ. (11.30. Lemma 11. Λ = {g } is uniformly absolutely continuous. Moreover. we have µ (|f | ≥ a) ≤ f 1 /a ≤ K/a for all a > 0. E ) ≤ sup µ(|f | : |f | ≥ M ) + M µ(E ). For this choice of ε and δ.116 11 Lp – spaces Deﬁnition 11. A subset Λ ⊂ L1 (µ) is uniformly integrable iﬀ Λ ⊂ L1 (µ) is bounded is uniformly absolutely continuous. it follows that lima→∞ supf ∈Λ µ (|f | : |f | ≥ a) = 0 as desired.3 Indeed.29. ¨ Indeed. Suppose {fn }n=1 and {gn }n=1 are two uniformly integrable ∞ sequences. (⇐=) Let K := supf ∈Λ f 1 < ∞.29. Since |1En g | ≤ |g | ∈ L1 and for any δ > 0. f ∈Λ µ(|fn |) = =1 µ(|fn | : E ) ≤ k showing that µ(|fn |) ≤ k for all n. ∞ ∞ ∞ ∞ ∞ This is not necessarily the case if µ (Ω ) = ∞. µ(1En |g | > δ ) ≤ µ(En ) → 0 as n → ∞. Proof. Corollary 11. are bounded in L1 (µ) . Hence given ε > 0 and δ > 0 as in the deﬁnition of uniform absolute continuity. then k So given ε > 0 choose M so large that supf ∈Λ µ(|f | : |f | ≥ M ) < ε/2 and then ε take δ = 2M to verify that Λ is uniformly absolutely continuous. lima→∞ µ (|f | : |f | ≥ a) = 0 by the dominated convergence theorem. E ) + µ(|f | : |f | < M.26. Moreover.28 (This lemma may be skipped. and E ∈ B .23) Remark 11. choose a suﬃciently large so that supf ∈Λ µ (|f | : |f | ≥ a) ≤ 1. f ∈Λ (11. If the Lemma is false. Deﬁnition 11. if Ω = R and µ = m is ©∞ Lebesgue measure.23) holds with ε = 1. (11. Another application of Proposition 11. n Second Proof. the dominated convergence theorem of Corollary 11. Let fn (∗) = n. A collection of functions.n] n=1 are uniformly integrable but not bounded in L1 (m) . A collection of functions. Let ϕ = i=1 ci 1Bi be a simple function such that g − ϕ 1 < ε/2.cls date/time: 23-Feb-2007/15:20 . showing {fn + gn }n=1 uniformly absolutely continuous. However. (11. n Let us also note that if Λ = {f } with f ∈ L1 (µ) . for f ∈ Λ.8 implies limn→∞ µ(|g | : En ) = 0. It is not in general true that if {fn } ⊂ L1 (µ) is uniformly absolutely continuous implies supn fn 1 < ∞.22) ≤ i=1 |ci | µ (E ∩ Bi ) + g − ϕ 1 ≤ i=1 |ci | µ (E ) + ε/2. then Λ is uniformly integrable. Page: 116 job: prob macro: svmonob. a→∞ f ∈Λ µ (|g | : E ) ≤ µ (|ϕ| : E ) + µ (|g − ϕ| : E ) n n lim sup µ (|f | : |f | ≥ a) = 0. we may choose a = K/δ in which case sup µ (|f | : |f | ≥ a) < ε. First Proof. for every δ > 0 we may k ﬁnd a ﬁnite partition of Ω by sets {E } =1 with µ(E ) < δ.29 completes the proof. Then 3 Since ε > 0 was arbitrary.). This contradicts µ(|g | : En ) ≥ ε for all n and the proof is complete. the sequences of functions. for ε > 0 we may choose δ > 0 such that µ (|fn | : E ) < ε and µ (|gn | : E ) < ε whenever µ (E ) < δ. Indeed. Λ. fn := 1[−n. {fn }n=1 and {gn }n=1 are both bounded in L1 (µ) and are both uniformly absolutely continuous. Since for δ < 1 a set E ⊂ Ω such that µ(E ) < δ ∞ is in fact the empty set and hence {fn }n=1 is uniformly absolutely continuous. ( =⇒ ) We have already seen that uniformly integrable subsets. Λ ⊂ L1 (µ) is said to be uniformly integrable if.22) implies supf ∈Λ f 1 < ∞. Since fn + gn 1 ≤ ∞ fn 1 + gn 1 it follows that {fn + gn }n=1 is bounded in L1 (µ) as well. If Eq. For any g ∈ L1 (µ). By Proposition 11. we then have µ (|fn + gn | : E ) ≤ µ (|fn | + |gn | : E ) < 2ε whenever µ (E ) < δ. Proposition 11.25. µ(|f | : E ) = µ(|f | : |f | ≥ M.

1) let aε > 0 with limε↓0 aε = ∞ and let fε := aε 1[0.29) Λ0 is uniformly absolutely continuous and hence Λ0 is uniformly integrable. Since sup E [fε : fε ≥ M ] = sup [εaε · 1aε ≥M ] . and f : Ω → C be a measurable function. No matter how aε > 0 is chosen. |fn | → |f | in µ – measure.32. for large M we have εaε ≤ δ for ε small enough so that aε ≥ M. µ) be a ﬁnite ∞ measure space.1] and P = m be Lebesgue measure on B . Then f ∈ L1 (µ) and f − fn 1 → 0 as n → ∞ iﬀ fn → f in µ measure and Λ is uniformly integrable.25) where gN = |f | + |fn | ∈ L . as ε ↓ 0 but 0= Ω ε↓0 i.28 and Proposition 11. B . 196 of Resnick. From this we conclude that lim supε↓0 (εaε ) ≤ δ and since δ > 0 was arbitrary. we may pass to the limit in Eq. Since macro: svmonob. So from Theorem 11.31. Λ := {fn }n=1 be a sequence of functions in L1 (µ) .1] . to ﬁnd lim sup f − fn n→∞ 1 if {fε } is uniformly integrable and δ > 0 is given. (⇐=) If fn → f in µ measure and Λ = is uniformly integrable then we know M := supn fn 1 < ∞.3 (Problem 5 on p.9. to show Λ is uniformly integrable it suﬃces to shows Λ is uniformly absolutely continuous. This is a typical example of a bounded and pointwise convergent sequence in L1 which is not uniformly integrable. fε → 0 a. Then S n n=1 is uniformly integrable.31 (Vitali Convergence Theorem).). Then Efε = εaε and so supε>0 fε 1 =: K < ∞ iﬀ εaε ≤ K for all ε. It then follows from Eq. and for ε ∈ (0. Then f ∈ Lp (µ) and f − fn p → 0 as n → ∞ iﬀ fn → f in µ measure and p ∞ Λ := {|fn | }n=1 is uniformly integrable. Therefore. B . then by Chebyschev’s inequality it follows that fn → f in µ – measure.d random variables. Theorem 11. 1] . Suppose that ∞ n is a sequence of integrable and i.i.e. Let Ω = [0. 1) is bounded in L1 (P ) . Since convergent sequences are bounded.ε] . Proof. Example 11. n |fn | dµ ≤ M < ∞.cls date/time: 23-Feb-2007/15:20 p ∞ ∞ + µ(|f | : E ).28) such that µ (gN : E ) < ε if µ (E ) < δ.8). |f | dµ ≤ lim inf Ω n→∞ Ω ∞ ∞ {fn }n=1 ∞ {Xn }n=1 sup µ(|fn | : E ) ≤ sup µ(|fn | : E ) ∨ (εN + µ(|f | : E )) ≤ εN + µ (gN : E ) . Let (Ω. limε↓0 fε = 0 a.25) that sup µ(|fn | : E ) < ε/2 + ε/2 = ε when µ (E ) < δ. Let Ω = [0. By reversing these steps one sees the converse is also true. ε ε = µ (|f − fn | : |f − fn | ≥ a) + µ (|f − fn | : |f − fn | < a) ≤ ε (a) + Ω 1|f −fn |<a |f − fn | dµ (11. By Corollary 11. |f | ∈ L1 (µ) and |fn | → |f | in 1 L (µ) .24) where ε (a) := sup µ (|f − fm | : |f − fm | ≥ a) → 0 as a → ∞. f ∈ L1 (µ). .11. (11. and f : Ω → C be a measurable function. with the aid of the dominated convergence theorem (see Corollary 11.24). and by Theorem 11. p ∈ [1.34.33.e. limε↓0 εaε = 0 if {fε } is uniformly integrable. see Exercise 11. (11. Given ε > 0 ﬁx N large so that εN < ε/2 and then choose δ > 0 (by Lemma 11. One now easily checks that Λ0 := {f − fn }n=1 is bounded in L1 (µ) and (using Lemma 11. Then the collection of functions. Let (Ω. Now for E ∈ B and n ∈ N.31. f − fn 1 lim fε dP = lim ε↓0 Ω fε dP = 1. ( =⇒ ) If fn → f in L1 (µ) . n n≤N N n=1 1 (11.1.s. µ) be a ﬁnite measure space. µ(|fn | : E ) ≤ µ(|f − fn | : E ) + µ(|f | : E ) ≤ f − fn Let εN := supn>N f − fn Page: 117 1 1 Corollary 11. Hence and application of Fatou’s lemma. if {fε } is uniformly integrable we would have to have lim (εaε ) = lim Efε = E0 = 0. B = B[0. then εN ↓ 0 as N ↑ ∞ and job: prob . ∞). m Since 1|f −fn |<a |f − fn | ≤ a ∈ L1 (µ) and µ 1|f −fn |<a |f − fn | > ε ≤ µ (|f − fn | > ε) → 0 as n → ∞..5 Uniform Integrability 117 Exercise 11. Example 11. {fn }n=1 be a sequence of functions in Lp (µ) . ( ⇐= ) Suppose that fn → f in µ measure and Λ := {|fn | }n=1 p µ p is uniformly integrable. Alternatively. fε (x) := 2 ε (1 − x/ε) ∨ 0 for ε ∈ (0. Proof. 1] . P be Lebesgue measure on B = B[0. ε↓0 ε↓0 ≤ ε (a) → 0 as a → ∞. and µ p p p p hn := |f − fn | → 0.

ϕ (x) is increasing (so ϕ is convex) and ϕ (x) ≥ (n + 1) for x ≥ an . (11. ≤ n=0 (n + 1) µ |f | 1|f |≥an ≤ n=0 (n + 1) εan Page: 118 job: prob macro: svmonob.cls date/time: 23-Feb-2007/15:20 . n→∞ µ µ a→∞ f ∈Λ 2. implies f − fn p p p p p p p Proof. an+1 ] and therefore ϕ(x) ≤ (n + 1)x for x ≤ an+1 . and Λ ⊂ L (Ω ) is a collection of functions.an+1 ] (|f |) n=0 ∞ ∞ + fn where q := p/(p − 1). So for f ∈ Λ. ϕ(0) = 0. εa := supf ∈Λ µ |f | 1|f |≥a → 0 as a → ∞. Now deﬁne ϕ so that ϕ(0) = 0 and ∞ ϕ (x) = n=0 (n + 1) 1(an . Again fn → f in µ – measure by Lemma 11. By the mean value theorem. Conversely if Λ is uniformly integrable. ∞ i. (11. Let hn := ||fn |p − |f |p | ≤ |fn |p + |f |p =: gn ∈ L1 and g := 2|f |p ∈ L1 .4. limx→∞ ϕ(x)/x = ∞ and Eq.8. By assumption. The following Lemma gives a concrete necessary and suﬃcient conditions for verifying a sequence of functions is uniformly integrable. If there exists a non decreasing function ϕ : R+ → R+ such that limx→∞ ϕ(x)/x = ∞ and K := sup µ(ϕ(|f |)) < ∞ f ∈Λ 0 p p 1 4 where by convention a0 := 0. |fn |))p−1 ||f | − |fn || ≤ p(|f | + |fn |)p−1 ||f | − |fn || and therefore by H¨ older’s inequality. Therefore we may choose an ↑ ∞ such that ∞ (n + 1) εan < ∞ n=0 i. 2.8. |fn | → |f | in L (µ) . lim hn dµ = 0. Then for f ∈ Λ µ(|f | : |f | ≥ a) = µ |f | ϕ (|f |) : |f | ≥ a ϕ (|f |) x ϕ(x) → 0 as ≤ µ(ϕ (|f |) : |f | ≥ a)εa = Ω |f − fn | dµ = Ω p hn dµ → 0 as n → ∞.31 that Λ is uniformly integrable. Hence it follows from Theorem 11. hn → 0 and gn dµ → gdµ. ∞ then Λ is uniformly integrable. Therefore by the dominated convergence theorem in Corollary 11. the dominated convergence theorem in Corollary 11. Z ||f |p − |fn |p | dµ ≤ p Z (|f | + |fn |)p−1 ||f | − |fn || dµ ≤ p p Z µ (ϕ(|f |)) = (|f | + |fn |)p−1 |f − fn | dµ p/q p µ ϕ(|f |)1(an .26) By construction ϕ is continuous. and hence ≤ µ(ϕ (|f |))εa ≤ Kεa lim sup µ |f | 1|f |≥a ≤ lim Kεa = 0. Suppose that µ(Ω ) < ∞. 1.35.26) is valid.118 11 Lp – spaces hn := |f − fn | ≤ (|f | + |fn |) ≤ 2p−1 (|f | + |fn | ) =: gn ∈ L1 (µ) with gn → g := 2p−1 |f | in L1 (µ) .e. above and set εa := supx≥a a → ∞ by assumption.e.an+1 ] (|f |) n=0 ∞ ≤ p f − fn ≤ p( f p (|f | + |fn |)p−1 p/q p) q p = p |f | + |fn | f − fn ≤ p (n + 1) µ |f | 1(an . there exists a non-decreasing continuous function ϕ : R+ → R+ such that ϕ(0) = 0. 1. Let ϕ be as in item 1. a→∞ (=⇒) Suppose f ∈ Lp and fn → f in Lp . Then gn → g. In particular ϕ(x) ϕ(an ) + (n + 1)x ≥ ≥ n + 1 for x ≥ an x x from which we conclude limx→∞ ϕ(x)/x = ∞. Lemma 11. ϕ(x) = 0 x ϕ (y )dy = n=0 (n + 1) (x ∧ an+1 − x ∧ an ) .an+1 ] (x). ||f |p − |fn |p | ≤ p(max(|f | . This shows that R f − fn ||f |p − |fn |p | dµ → 0 as n → ∞. 4 Here is an alternative proof. We also have ϕ (x) ≤ (n + 1) on [0.

b ≥ 1 with a−1 + b−1 = 1 chosen appropriately. In particular. For a < α < β < b.27) (11. · ) is a Banach space. y ∈ (a. x) .6. Let f ∈ Lp ∩ L∞ for some p < ∞. ϕ (y ) ≥ ϕ (x) + t (y − x) for all x. we would gain no new information by this extension. f ∈ L∞ iﬀ limq→∞ f q < ∞. 6. y ) is increasing in each of its arguments. That is ϕ is Lipschitz continuous on [α. 5.6 Exercises Exercise 11. b) . (Part of Folland 6. show f ∞ = limq→∞ f q for all measurable functions f : X → C. x↑y 3.20).7 Appendix: Convex Functions Reference. y ) < ∞ and y ↓x (11. ϕ± are both increasing functions and further satisfy. Then (11. see Figure ?? below. b) → R is convex and for x. sa tb st ≤ + a b applied to the right side of Eq. y ) = F (y. y ) := . b) with x < y. (11. y ) > −∞.7 Appendix: Convex Functions 119 and hence sup µ (ϕ(|f |)) ≤ f ∈Λ ∞ (n + 1) εan < ∞.21 to show lim supq→∞ f q ≤ f ∞ and to show lim inf q→∞ f q ≥ f ∞ .37. 5 ϕ− (y ) := F (y −. Exercise 11.cls date/time: 23-Feb-2007/15:20 The same formula would deﬁne F (x.11. F (x. Page: 119 job: prob .21. y ) := lim F (x. The functions exp(x) and − log(x) are convex and |x| is convex iﬀ p ≥ 1 as follows from Lemma 7. macro: svmonob. y−x Then. with a. let K := max ϕ+ (α) . The following limits exist.30) (11. m1 ≤ m3 ≤ m2 . 186. A convex function with three cords.) Hint: Use the inequality. Hints: Use Corollary 11. β ] . 4. y ) for x = y.4. y ∈ (a. If we further assume µ(X ) < ∞.28) 11. y ∈ [α. Show f ∞ = limq→∞ f q . Exercise 11. For any t ∈ ϕ− (x) . Deﬁnition 11. ϕ+ (x) := F (x. ϕ+ (x) . However. 1] ϕ(xt ) ≤ tϕ(x1 ) + (1 − t)ϕ(x0 ) where xt = tx1 + (1 − t)x0 . b) → R is convex if for all a < x0 < x1 < b and t ∈ [0.36. see the appendix (page 500) of Revuz and Yor.1. − ∞ < ϕ− (x) ≤ ϕ+ (x) ≤ ϕ− (y ) < ∞ ∀ a < x < y < b. 7. since F (x. The functions. Example 11.31 for p > 1 and by inspection of p = 1. 11. A function ϕ : (a. ϕ− (β ) . Notice the slope relationships.29) p |ϕ (y ) − ϕ (x)| ≤ K |y − x| for all x.21) in Corollary 11. The set of discontinuity points for ϕ+ and for ϕ− are the same as the set of points of non-diﬀerentiability of ϕ. β ] .3 on p. Theorem 11. x+) := lim F (x. n=0 11.20 by showing (Lp + Lr . 2. The function ϕ+ is right continuous and ϕ− is left continuous. Fig. let5 ϕ (y ) − ϕ (x) F (x. 1. Moreover this set is at most countable.38. Prove Eq. Suppose that ϕ : (a. Complete the proof of Proposition 11. (11. let M < f ∞ and make use of Chebyshev’s inequality.5.

using the left continuity of ϕ− . Then t ≤ ϕ+ (x) = F (x. x+) ≤ F (x. they have at most countably many points of discontinuity. (11. y−x valid for all a < u < v < w < b. ϕ− is left continuous. Hence if ϕ− is continuous at y. x∈D Page: 120 job: prob macro: svmonob. Corollary 11. We may now let y ↓ c to conclude ϕ+ (c+) ≤ ϕ+ (c) . we have ϕ+ (x) = F (x. (11.31) shows F (x. then ϕ (y ) = sup [ϕ (x) + ϕ± (x) (y − x)] for all x. Since ϕ± are increasing functions. y ) = ϕ− (y ) ≤ ϕ− (β ) (11. y ) = or equivalently. then ϕ+ (y −) = ϕ− (y ) = ϕ (y ) = ϕ+ (y ) which shows ϕ+ is continuous at y. If ϕ : (a. ϕ (x) − ϕ (y ) x−y ϕ (y ) ≥ ϕ (x) − t (x − y ) = ϕ (x) + t (y − x) for y ≤ x. Therefore Eq.30) for all x. For a < c < x < y < b. −K ≤ ϕ+ (α) ≤ ϕ (y ) − ϕ (x) ≤ ϕ− (β ) ≤ K. again see Figure 11. y ∈ (a. 7. ϕ (v ) − ϕ (u) ϕ (w) − ϕ (u) ϕ (w) − ϕ (v ) ≤ ≤ . Similarly. it follows that ϕ− (c) = ϕ− (c−) . The ﬁrst (second) inequality in Eq. we have ϕ+ (α) ≤ ϕ+ (x) = F (x. x+) ≤ F (y −. for y < x. v−u w−u w−v (11.29). Using the increasing nature of F. y ) and letting x ↓ c (using the continuity of F ) we learn ϕ+ (c+) ≤ F (c. Let a < x < y < b. For a < α ≤ x < y ≤ β < b. ϕ (y ) − ϕ (x) y−x This last inequality implies. 6. y ) is increasing y (x). are monotone and hence exist as claimed. shows ϕ− (y ) = ϕ+ (y −) . Since ϕ (x1 ) − ϕ (x0 ) ϕ (x1 ) − ht ht − ϕ (x0 ) = = .120 11 Lp – spaces Proof. c) .e. b) → R is a convex function and D ⊂ (a. Hence we have proved Eq. 3. xt − x0 xt − x0 x1 − xt x1 − xt These inequalities may be written more compactly as. ϕ+ (x) . ϕ (x1 )) and the statement that ϕ is convex is then equivalent of ϕ (xt ) ≤ ht for all 0 ≤ t ≤ 1. i. x) = or equivalently. x+) ≤ F (x.1. y ) ≤ F (y −. Let t ∈ ϕ− (x) . then (xt . b) .cls date/time: 23-Feb-2007/15:20 . |ϕ (y ) − ϕ (x)| ≤ K (y − x) which is the desired Lipschitz bound. we have ϕ− (y ) ≥ F (x. it follows that ϕ+ (c) = ϕ+ (c+) and hence that ϕ+ is right continuous. Since ϕ+ (c) ≤ ϕ+ (c+) . ϕ (y ) ≥ ϕ (x) + t (y − x) for y ≥ x. x) ≥ F (y. y ) . 1.30) holds for y ≥ x.31) t ≥ ϕ− (x) = F (x−. Thus we have shown that set of discontinuity points of ϕ+ is the same as the set of points of non-diﬀerentiability of ϕ. x+) = ϕ+ (x) < ∞ and ϕ+ (x) = F (x. Conversely if ϕ is diﬀerentiable at y. xt − x0 x1 − x0 x1 − xt the convexity of ϕ is equivalent to ht − ϕ (x0 ) ϕ (x1 ) − ϕ (x0 ) ϕ (xt ) − ϕ (x0 ) ≤ = for all x0 ≤ xt ≤ x1 xt − x0 xt − x0 x1 − x0 and to ϕ (x1 ) − ht ϕ (x1 ) − ϕ (xt ) ϕ (x1 ) − ϕ (x0 ) = ≤ for all x0 ≤ xt ≤ x1 x1 − x0 x1 − xt x1 − xt and convexity also implies ϕ (xt ) − ϕ (x0 ) ht − ϕ (x0 ) ϕ (x1 ) − ht ϕ (x1 ) − ϕ (xt ) = = ≤ .39. ϕ− (y ) = ϕ− (y +) = ϕ+ (y ) and ϕ is diﬀerentiable at y. ϕ (x0 )) to (x1 . y ) = ϕ− (y ) as desired. Since ϕ− (c) ≥ ϕ− (c−) . and 2. If we let ht = tϕ(x1 ) + (1 − t)ϕ(x0 ). b) .32) and in particular. Letting x ↑ y in Eq. for a < x < y < c < b. x) ≤ F (x. 5. Similarly. b) is a dense set. x+) ≤ F (x. This then implies the limits in item 2. (11. That the discontinuity set of ϕ− is the same as the non-diﬀerentiability set of ϕ is proved similarly. y ) and letting y ↑ c (using the continuity of F ) we learn ϕ− (c−) ≥ F (x. (11. Now let x ↑ c to conclude ϕ− (c−) ≥ ϕ− (c) . y ∈ (a. 4. −∞ < ϕ− (x) = F (x−. ht ) is on the line segment joining (x0 .

ψ− (y ) ≥ ϕ (xn ) + ϕ− (xn ) (y − xn ) .30) above. b) is arbitrary we may take x = y to discover ψ− (y ) ≥ ϕ (y ) and hence ϕ (y ) = ψ− (y ) . (11. Let ψ± (y ) := supx∈D [ϕ (x) + ϕ± (x) (y − x)] . b) . shows ψ− (y ) ≥ ϕ (x) + ϕ− (x) (y − x) . Now suppose that x ∈ (a. we know that ϕ (y ) ≥ ψ± (y ) for all y ∈ (a. Since x ∈ (a. Then passing to the limit in the estimate. The proof that ϕ (y ) = ψ+ (y ) is similar. b) and xn ∈ Λ with xn ↑ x. According to Eq.Proof. .

.

Part III Convergence Results .

.

g (X )) ≥ 0. j =1 . P ) . Show Cov (f (X ) . For any constant k. i. The covariance. The variance of X. We leave most of this simple proof to the reader. X and Y. Let {Xn }n=1 be a sequence of uncorrelated square integrable random variables.3 (An L2 – Weak Law of Large Numbers). f. let us prove Var (X + k ) = Var (X ) . and we will set Sn := X1 + · · · + Xn for all n ∈ N. It also follows from Eq. B . Cov (X. {an } and µ ∈ R σn such that 1 an 1 a2 n then Sn an n ∞ µj → µ as n → ∞ and j =1 n 2 σj → 0 as n → ∞. µn = EXn and 2 = Var (Xn ) . Hence ESn = and Sn − E Hence. Cov (X. X ) = Cov (X. g : R → R are two increasing functions such that both f (X ) and g (X ) are square integrable. Y ) = 0 if either X or Y is constant.1) that Var (X ) ≤ E X 2 for all X ∈ L2 (P ) . X ) = Var (X ) . k ) = Cov (X + k. n j =1 Lemma 12. Y ) = E [(X − aX ) (Y − aY )] = E [XY ] − EX · EY where aX := EX and aY := EY.2) Theorem 12. g (Y ) are independent and hence uncorrelated for any choice of Borel measurable functions. g : R → R such that f (X ) and g (X ) are square integrable. If {Xk }k=1 are uncorrelated L2 (P ) – random variables. j =1 → µ in L2 (P ) and also in probability. Notice that if X and Y are independent random variables. X + k ) = Cov (X + k. Xj ) = 0 for all i = j.12 Laws of Large Numbers In this chapter {Xk }k=1 will be a sequence of random variables on a probability space. Var (X ) := Cov (X. X ) + Cov (k. 1 an µj n µj → µ j =1 2 an = 1 a2 n n 2 σj → 0. Y ) = 0. Var (X + k ) = Cov (X + k.1) We say that X and Y are uncorrelated if Cov (X. More generally we say {Xk }k=1 ⊂ L2 (P ) are uncorrelated iﬀ Cov (Xi . As an example of the type of argument involved. The covariance function. Suppose that X is a random variable and f. Hint: let Y be another random variable which has the same law as X and is independent of X.1 (A correlation inequality). If there exists an increasing positive sequence. Y ) of two square integrable random variables. X ) = E X 2 ∞ Exercise 12. Then consider E [(f (Y ) − f (X )) · (g (Y ) − g (X ))] . (12. E [XY ] = n EX · EY. is deﬁned by Cov (X.2. Var (X + k ) = n Var (X ) and Var (kX ) = k 2 Var (X ) . X ) + Cov (X + k. (12. then n Proof.e. We ﬁrst observe that ESn = E S n − j =1 n µj and n 2 µj = Var (Sn ) = n Var (Xj ) = j =1 j =1 2 σj . X ) = Cov (X. Var (Sn ) = k=1 Var (Xk ) . Deﬁnition 12. then f (X ) . (Ω. Y ) is bilinear in X and Y and Cov (X. n j =1 Proof.1. − (EX ) 2 (12.

Example 12. this end. Sn −µ n ≤ L2 (P ) Sn n − µn L2 (P ) → 0. + |µn − µ| → 0 as n → ∞. Therefore.Convergence of Random Sums). Supposing n > m.5 (L2 .5 under various assumptions. Proposition 12. n → ∞. 1 1 2 Var X1 1|X1 |≤n ≤ E X1 1|X1 |≤n n n ≤ E |X1 | 1|X1 |≤n and so again by the DCT. E Sn − µn n 2 = Var = = 1 n2 n = k=m+1 Var (Xk ) = Sn n = 1 Var (Sn ) n2 Var Xi 1|Xi |≤n i=1 Note well: since L2 (P ) convergence implies Lp (P ) – convergence for 0 ≤ p ≤ 2. we have 2 n 2 n We will now complete the proof by showing that. Proof. it suﬃces by the completeness of L (P ) (see Theorem 11.s. 2 Sn L (P ) → n where µk := EXk . Suppose that ∞ ∞ {Xk }k=1 ⊂ L2 (P ) are uncorrelated.17) to show Sn − Sm 2 → 0 as m. in fact.d. This completes the proof since. So it suﬃces to prove µ.126 12 Laws of Large Numbers Sn −µ an = L2 (P ) Sn − n j =1 µj an Sn − n j =1 + n j =1 µj an + −µ L2 (P ) n j =1 Theorem 12. Sn := i=1 n P 1 n Sn → ∞ µ = EX1 . L1 (P ) – random variables. simply apply Theorem 12. Moreover.3 with an = n. Then Sn n L2 (P ) ∞ we have {Sn = Sn } ⊂ ∪n i=1 {|Xi | > n} . If k=1 Var (Xk ) < ∞ then ∞ ≤ E [|X1 | : |X1 | > n] → 0. using Chebyschev’s inequality along with the dominated convergence theorem. k=1 − Sn P n → 0. To E Xi 1|Xi |≤n = E X1 1|X1 |≤n i=1 Sn − 2 Sm 2 =E k=m+1 n (Xk − µk ) n 2 σk → 0 as m. k=m+1 and observe that limn→∞ µn = µ by the DCT. Sn P n → (Xk − µk ) converges in L2 (P ) .e.4. n → ∞.i.6 (Khintchin’s WLLN).1 Main Results The proofs of most of the theorems in this section will be the subject of later parts of this chapter. Page: 126 job: prob Sn − µn n L2 (P ) In fact we have the stronger result. let µn := 1 1 ESn = n n n µ. If {Xn }n=1 are i. ≤ µj L2 (P ) µj an an Xi 1|Xi |≤n . Sn n S Sn − n >ε n n ≤ P (Sn = Sn ) → 0 as n → ∞. Suppose that {Xk }k=1 ⊂ L2 (P ) are uncorrelated identically distributed random variables. Letting Sn := k=1 (Xk − µk ) . where by L0 (P ) – convergence we mean convergence in probability. The remainder of this chapter is mostly devoted to proving a. Hence it follows that P i. 12. P (Sn = Sn ) ≤ i=1 P (|Xi | > n) = nP (|X1 | > n) To see this.cls date/time: 23-Feb-2007/15:20 . we have n → µ = EX1 as n → ∞.17 and Proposition 12. then Proof. convergence for the quantities in Theorem 11. Letting − µ → 0. These results will be described in the next section. macro: svmonob.

s. 2. One way to prove this is to appeal Proposition 12. Finally.s k=1 Proof. Proposition 12. The sum Xn is convergent a. then j =1 (Yj − EYj ) converges a. we may conclude that 12.s. 2. and a sequence an ↑ ∞ such that 1 n→∞ an lim then 1 n→∞ an lim n Var (Xn ) = n=1 n=1 Var Xn 1|Xn |≤c < ∞ Xk = X a.s k=1 and so by Kolmogorov’s convergence criteria. Since Sn ≥ Sn . Moreover.31 below. Suppose the three series converge for some c > 0.s. However.1 Main Results 127 Theorem 12. of random variables are tail equivalent if ∞ ∞ P (|Xn | > c) < ∞.s. = ∞ a.cls date/time: 23-Feb-2007/15:20 . Deﬁnition 12. iﬀ the sum generally we have P Xn is convergent Xn is convergent a. and E Xn 1|Xn |≤c converges. ∞ n=1 ∞ n=1 ∞ n=1 lim inf n→∞ 1 M Sn ≥ lim inf Sn = µM a.11 and 12. If {Xn } and {Xn } are tail equivalent. If there exists a random variable. if the three series above converge for some c > 0 then they converge for all values of c > 0. Suppose ∞ that {Yn }n=1 are independent square integrable random variables. The next theorem generalizes the previous theorem by giving necessary and suﬃcient conditions for a random series of independent random variables to converge.7 (Kolmogorov’s Strong Law of Large Numbers). the third series guarantees that n=1 EXn = n=1 E Xn 1|Xn |≤c is ∞ convergent. is almost surely convergent iﬀ there exists c > 0 such that 1. then n Sn → ∞ a. Suppose that ∞ ∞ {Xn }n=1 are independent random variables. (Xn − Xn ) converges a. n→∞ n n Sn n Since µM → ∞ as M → ∞. − 1 Remark 12.12 are standard convergence criteria for random series. Theorem 12.e. Two sequences. We will give this second proof below. {Xn } and {Xn } . Theorem 12. Proof. j =1 Xj .5 above and L´ evy’s Theorem 12.9. iﬀ Xn is integrable and in which case EXn = µ. ∞ ∞ Page: 127 job: prob macro: svmonob. 3. ∞ (Xn − EXn ) is almost surely convergent.s.7 is based on the study of random series.d. it follows that lim inf n→∞ n that limn→∞ S n = ∞ a. To prove this. ∞ Xn is convergent =1 Hence {Xn } and {Xn } are tail equivalent and so it suﬃces to show n=1 Xn is almost surely convergent. Var Xn 1|Xn |≤c < ∞. Xn (ω ) = Xn (ω ) for a.a n. As second method is to make use of Kolmogorov’s inequality. n n=1 Xk = X a. we know. ω. Proof.12. Then the random series. If E |X1 | = ∞ but EX1 < ∞. If ∞ ∞ j =1 Var (Yj ) < ∞. The proof of the reverse direction will be given in Section 12. Suppose ∞ that {Xn }n=1 are i.i. More P (Xn = Xn ) = n=1 n=1 P (|Xn | > c) < ∞. It follows from Theorem 1 M M M Sn → µM := EX1 a.s. X .. for a. and hence One proof of Theorem 12.11 (Kolmogorov’s Convergence Criteria). Proof of suﬃciency. then ∞ ∞ E n=1 1Xn =Xn = n=1 P (Xn = Xn ) < ∞. Then 1 there exists µ ∈ R such that n Sn → µ a.s. The proposition is an easy consequence of this observation.s. by the convergence of the second series we learn ∞ ∞ 3.s. If we let Xn := Xn 1|Xn |≤c . Suppose {Xn } and {Xn } are tail equivalent. n M M M for M > 0 let Xn := Xn ∧ M and Sn := i=1 Xi .12 (Kolmogorov’s Three Series Theorem). random variables and let Sn := X1 + · · · + Xn . therefore we may conclude n=1 Xn is convergent.7 that n Theorem 12.s.10.8.8 below. Then 1.

then the three series in Theorem 12. [σn δn + µn βn ] is convergent. A random variable. Suppose that ∞ {Yn }n=1 are independent square integrable random variables.3) is equivalent to writing E [f (Y )] = √ 1 2πσ 2 R d d ∞ α e− 2 x dx i. Observe that Eq. if j =1 Yj converges a. Y. i. i. Indeed.s. σ 2 . which would Bn and in either case we will have √1 2 π Bn contradict Eq. 2. f : R → R.3) If limn→∞ µn = 0 then there is a c > 0 such that either µn ≥ c i. ∞) i. (12. < ∞ and j j =1 j j =1 d µn = n=1 n=1 Yn − n=1 (Yn − µn ) Yj converges a. Since Yn = σn N + µn ..2.1 Random Series Examples Example 12. we have for any c > 0 that converges as well. 0) ⊂ 1 2 e− 2 x dx ≥ 1/2 i.4). Indeed. we will say Y is a standard normal random variable. (12. Also observe that Y = d N µ.2 Examples 12. αn → 1 as ∞ 2 n → ∞ and therefore we may conclude n=1 σn < ∞. ∞) ⊂ Bn and in the second (−∞.4) where Bn = (−∞. for some α < ∞ and so 1 √ 2π 1 2 1 e− 2 x dx ≥ √ 2π Bn We will abbreviate this by writing Y = N µ.s. The convergence of the second series for all c > 0 implies ∞ ∞ f (y ) e− 2σ2 (y−µ) dy ∞> 1 2 Var Yn 1|Yn |≤c = n=1 ∞ n=1 Var [σn N + µn ] 1|σn N +µn |≤c .4). Alternatively: we may also deduce the convergence of third series as well. f (y ) e− 2σ2 (y−µ) R ∞ 1 2 dy 1 =√ σ 2πσ 2 ∞ j =1 f (y ) e− 2σ2 (y−µ) dy. d 1.e. 1 2 (12. Hence we may concluded that limn→∞ µn = 0. Therefore we may conclude that limn→∞ µn = limn→∞ σn = 0. (12.. we ﬁnd 1 E [f (σN + µ)] = √ 2π 1 =√ 2π f (σx + µ) e− 2 x dx R 1 2 ∞> n=1 2 σn Var N 1|σn N +µn |≤c + µ2 n Var 1|σn N +µn |≤c ≥ n=1 2 σn αn . σ 2 is equivalent to Y = σN +µ. When µ = 0 and σ 2 = 1 we will simply write N for N (0.12 converge for all c > 0. n=1 Page: 128 job: prob macro: svmonob.e. by making the change of variable.15. y = σx + µ.14. then we may conclude that Bn contains a set of the form [α. − c + µn )∪ σn c − µn . σn e− 2σ2 (y−µ) dy for all B ∈ BR . Suppose that {Yn }n=1 are independent square integrable ran2 dom variables such that Yn = N µn . 1) and if Y = N.s.o. Then ∞ ∞ 2 σ µ converges. Deﬁnition 12. It now follows by Kol∞ mogorov’s convergence criteria that n=1 (Yn − µn ) is almost surely convergent and therefore ∞ ∞ ∞ Lemma 12.∞ . iﬀ Proof. R 1 2 where αn := Var N 1|σn N +µn |≤c . As the reader should check. for all c > 0 implies ∞ ∞ n=1 µn by the E [σn N + µn ] 1|σn N +µn |≤c n=1 ∞ is convergent.13.s.o. or µn ≤ −c i. To prove the converse directions we will make use of the Kolmogorov’s three series theo∞ rem. Namely.o. σn . Similarly if limn→∞ σn = 0.o. In the ﬁrst case in which case (0.13 (Kolmogorov’s Convergence Criteria Example). 1 2 which would again contradict Eq.128 12 Laws of Large Numbers ∞ 12. is normal with mean µ standard deviation σ 2 iﬀ P (Y ∈ B ) = √ 1 2πσ 2 B d ∞> n=1 P (|σn N + µn | > c) = 1 √ 2π n=1 ∞ e− 2 x dx Bn 1 2 (12.o.cls date/time: 23-Feb-2007/15:20 . ∞ for all bounded measurable functions. such that ∞ ∞ ∞ j =1 Var (Yj ) < ∞ and j =1 EYj converges a. The implication “ =⇒ ” is true without the assumption that the Yn are normal random variables as pointed out in Example 12. then j =1 Yj converges a.

2 Examples 129 where δn := E N · 1|σn N +µn |≤c and βn := E 1|σn N +µn |≤c . (12.5. With a little eﬀort one can show. Example 12. Let dom variable. This is a simple consequence of Kolmogorov’s convergence criteria. 2 Since e−k/σn ≤ Cσn for large n. Theorem 12.2. convergent for all t ∈ R.11.d. n=1 For the second series. ∞ and hence ∞ µn = n=1 n=1 µn βn − n=1 µn (βn − 1) must also be convergent. and the facts that E [an Nn sin ωn t] = 0 and 2 2 Var (an Nn sin ωn t) = a2 n sin ωn t ≤ an . ∞ 12. The process {Bt : 0 ≤ t ≤ 1} is Brownian Motion. {an }n=1 ⊂ R. Given this.. Page: 129 job: prob macro: svmonob. provided n=1 a2 n < ∞. 1 0 k=1.e.12 converge. n=1 n=1 E Xn 1|Xn |≤c ≤ n=1 E |Xn | 1|Xn |≤c ≤ n=1 2 Var (Yn ) < ∞. 23 ∞ n=1 |σn δn | ≤ C ∞ n=1 3 σn <∞ |µn (βn − 1)| ≤ C n=1 n=1 ∞ ∞ 2 |µn | σn <∞ Fact: Wiener in 1923 showed the series in Eq.i.12. We will apply Theorem 12.5) is in fact almost surely uniformly convergent. if we take ωn = (2n − 1) π 2 and an = that √ 2 2 π Nk Bt := sin k t π k 2 is a.cls date/time: 23-Feb-2007/15:20 . random variables with common distribution function. We need to then check the three series in the statement of Theorem 12. the P √ d 2 2 sin (kπt) dt πk dt = 1 weak law of large numbers implies Fn (x) → F (x) as n → ∞.d. For the ﬁrst series we have by the Markov inequality.11.5) has been determined by requiring. we will now use Theorem 12. standard normal ran∞ n=1 1 1 2 E |Xn | = 2 2 c c n=1 ∞ ∞ Var (Yn ) < ∞.i. 2π Var Xn 1|Xn |≤c ≤ n=1 n=1 E Xn 1|Xn |≤c 2 ≤ n=1 E 2 Xn = n=1 Var (Yn ) < ∞ Let {ωn }n=1 ⊂ R. x]) . t → Bt is almost surely continuous. P (Nn ∈ A) = A ∞ ∞ ∞ {Nn }n=1 P (|Xn | > c) ≤ be i.s.17. √ then it follows (12. then ∞ and for the third series (by Jensen’s or H¨ older’s inequality) ∞ ∞ ∞ an Nn sin ωn t converges a. n n 1 1 Fn (x) := 1X ≤x = δX ((−∞. For x ∈ R let Fn (x) be the empirical distribution function deﬁned by. the process. Moreover. it follows that ∞ so that n=1 µn βn is convergent.3.12 to give a proof of Theorem 12. ∞ ∞ 2 2 2 1 0 d sin dt kπ t 2 2 dt = = k2 π2 22 1 cos 0 kπ t 2 2 dt 1 k 2 π 2 2 kπ 1 t + sin kπt 22 kπ 4 4 = 0 k2 π2 .16 (Brownian Motion). and t ∈ R.12 with Xn := Yn − EYn . observe that ∞ ∞ ∞ 2 1 √ e−x /2 dx for all A ∈ BR . n j =1 j n j =1 j Since E1Xj ≤x = F (x) and 1Xj ≤x ∞ j =1 ∞ As a special case.2 A WLLN Example Let {Xn }n=1 be i. δn ∼ e−k/σn and 1 − βn ∼ e−k/σn for large n... As usual. 1) let F ← (p) := inf {x : F (x) ≥ p} and recall that F ← (p) ≤ x iﬀ F (x) ≥ p. As a simple application of Theorem 12. F (x) := P (Xn ≤ x) .12. for p ∈ (0. Example 12.s. √ The factor 2πk2 2 π (2n−1) . Let us notice that as seen by. 2 are Bernoulli random variables. i.

. let δε := p − F (F ← (p) − ε) > 0 and observe that ← ← {F ← (p) − Fn (p) ≥ ε} = {Fn (p) ≤ F ← (p) − ε} = {Fn (F ← (p) − ε) ≥ p} t↑∞ Page: 130 job: prob macro: svmonob. .19 (Renewal Theory). the p – quantile of the distribution F by observing {Xi }i=1 . The order statistic of ( n) ( n) ( n) X1 . the reader may easily check that ( n) ← (p) = X np . . ∞].e. . X 2 . (12. by the SSLN. . Xk ≤ x iﬀ # {j ≤ n : Xj ≤ x} ≥ k n iﬀ j =1 1Xj ≤x ≥ k. we set T0 = 0. .130 12 Laws of Large Numbers ← Fn (p) = inf {x : Fn (x) ≥ p} = inf x : n 1Xj ≤x ≥ np j =1 and hence. By convention. On these grounds we expect Nt ∼ t/µ and hence 1 1 Nt → a.i. Thus we can recover. Indeed. From the deﬁnition of Nt . (We assume the bulbs are replaced immediately on burning out. ← P (F ← (p) − Fn (p) ≥ ε) ← = P (Fn (F (p) − ε) − F (F ← (p) − ε) ≥ p − F (F ← (p) − ε)) = P (Fn (F ← (p) − ε) − F (F ← (p) − ε) ≥ δε ) → 0 as n → ∞. Let {Xi }i=1 be i. We will interpret Yi to be the amount of time the ith – bulb remains out after burning out before it is replaced by bulb number i + 1.s. Let us observe that Xk ( n) are all random variables for k ≤ n.d.cls date/time: 23-Feb-2007/15:20 . t] . Then ← ← ← {Fn (p) − F ← (p) > ε} = {Fn (p) ≤ ε + F ← (p)} = {Fn (p) ≤ ε + F ← (p)} ← = {Fn (ε + F (p)) ≥ p} c t TNt +1 TNt ≤ < .3 Strong Law of Large Number Examples Example 12. random variables with 0 < Xi < ∞ a. .d.s.6). Xn . Example 12. TNt ≤ t < TNt +1 and so ∞ ≤x = n 1Xj ≤x ≥ k j =1 ∈ B. if Ω0 := limn→∞ n Tn = µ then P (Ω0 ) = 1. Let {Xi }i=1 be i. Let ε > 0. 1) is a point where F (F ← (p) − ε) < p < F (F ← (p) + ε) for all ε > 0 ← (p) → F ← (p) as n → ∞.i. We are now going to show 1 EX1 lim Rt = . Also again assume that 0 < Xi < ∞ and 0 < Yi < ∞ a. we have.d. = inf {x : # {j ≤ n : Xj ≤ x} ≥ np} . X 2 . . . (12. . Ω1 := {Nt ↑ ∞ as t ↑ ∞} also has full measure and for ω ∈ Ω0 ∩ Ω1 we have µ = lim TNt (ω) (ω ) TNt (ω)+1 (ω ) Nt (ω ) + 1 t ≤ lim ≤ lim = µ. Xn denotes (X1 .i. Moreover. t EX1 + EY1 Similarly.s. Letting µ := EX1 ∈ (0. i. Let Rt be the amount of time that we have a working bulb in the time interval [0. Xn ) is the ﬁnite sequence. P arranged in increasing order with possible repetitions. where (X1 . t →∞ t →∞ Nt (ω ) Nt (ω ) Nt (ω ) + 1 Nt (ω ) ∞ ∞ so that ← {Fn (p) − F ← (p) > ε} = {Fn (F ← (p) + ε) < p} = {Fn (ε + F ← (p)) − F (ε + F ← (p)) < p − F (F ← (p) + ε)} . t→∞ Letting δε := F (F ← (p) + ε) − p > 0. ∞ ∞ with {Xi }i=1 being independent of the {Yi }i=1 . and {Yi }i=1 be i. Suppose that p ∈ (0. Fn Proposition 12. Nt Nt Nt Since Xi > 0 a. we have ETn = nµ – the expected time the nth – bulb burns out. . Xn ) ( n) Thus we have shown that X ( n) np → F ← (p) as n → ∞.) Further let Nt := sup {n ≥ 0 : Tn ≤ t} denote the number of bulbs which have burned out up to time n.18. with high then X np = Fn n th probability. as n → ∞.20 (Renewal Theory II).s. . Think of the Xi as the time that bulb number i burns and Tn := X1 + · · · + Xn is the time that the nth – bulb burns out. ( n) ( n) ( n) X1 .6) t µ 1 To prove Eq. Keeping the notation above. . . that ← P ({Fn (p) − F ← (p) > ε}) = P (Fn (ε + F ← (p)) − F (ε + F ← (p)) < −δε ) → 0. ( n) P Proof. Xk ( n) 12. . . if we let x = min {n ∈ Z : n ≥ x} . .

k − 1 and ω ∈ Ωk with P (Ωk ) = 1. .3 Strong Law of Large Number Examples 131 To prove this. by the strong law of large numbers.1 To do this we will use one more application of the strong law of large numbers applied to {1Xi <x } which allows us to conclude. by what we have just shown. Further let µn := n i=1 δXi be the empirical distribution with empirical distribution function.e.12. Our goal is to now show that this convergence is uniform. on F (x) ≤ lim inf Fn (x) ≤ lim sup Fn (x) ≤ F (x) for all x ∈ R. Page: 131 job: prob macro: svmonob. it follows by the strong law of large numbers the limn→∞ Fn (x) = F (x) a. x∈R Proof. so that 1 Nt = µ+ν t + o (t) a. Thus. let Λk := and let xi := k : i = 1. on Ω0 . . 2. 1 1 Rt = t t Nt 1 Xi = · t Nt i=1 Nt Nt n i=1 and x0 = −∞. we have F (r) = lim Fn (r) ≤ lim inf Fn (x) ≤ lim sup Fn (x) ≤ lim Fn (s) = F (s) . . i=1 lim sup |Fn (x) − F (x)| = 0 a. Xi → i=1 1 · µ a. Therefore. Since {1Xi ≤x }i=1 are i. if x ∈ R and r ≤ x ≤ s with r. . there is a set Ω0 ⊂ Ω such that P (Ω0 ) = 1 and on Ω0 . 1. Suppose that {Xn }n=1 are n 1 i. 2. 2. Then Rt = Nt 1 1 i=1 Xi . k − 1. (the null set depends on x).s. x]) = Then n→∞ x∈R ∞ 1 n n 1Xi ≤x . i. we now have t Nt → µ+ν a. Observe that it is possible that xi = xi+1 for some of the i. for n ≥ N (ω ) and i = 1. on Ω0 .s. xi+1 ) . for each x ∈ R. Moreover on Ω0 . k − 1 inf {x : F (x) ≥ i/k } for i = 1. Setting µ = EX1 and ν = EY1 . that n→∞ lim Fn (x−) = F (x−) a. Further let N (ω ) ∈ N be chosen so that |Fn (xi ) − F (xi )| < 1/k and |Fn (xi −) − F (xi −)| < 1/k . Now suppose i has been chosen so that xi < xi+1 and let x ∈ (xi . . µ+ν ∞ Theorem 12.s.s. . n→∞ n→∞ n→∞ n→∞ We may now let s ↓ x and r ↑ x to conclude.21 (Glivenko-Cantelli Theorem). We then have Fn (x) ≤ Fn (xi+1 −) ≤ F (xi+1 −) + 1/k ≤ F (x) + 2/k and Fn (x) ≥ Fn (xi ) ≥ F (xi ) − 1/k ≥ F (xi+1 −) − 2/k ≥ F (x) − 2/k. for each x ∈ R. If F is continouous then. Fn (r) → F (r) for all r ∈ Q. random variables and F (x) := P (Xi ≤ x) . . This can occur when F has jumps of size greater than 1/k.s. in this special case we have shown oﬀ a ﬁxed null set independent of x that limn→∞ Fn (x) = F (x) for all x ∈ R.s. i Given k ∈ N.i. .cls date/time: 23-Feb-2007/15:20 . . s ∈ Q. n→∞ n→∞ Hence it follows on Ω0 := ∩∞ k=1 Ωk (a set with P (Ω0 ) = 1) that n→∞ x∈R lim sup |Fn (x) − F (x)| = 0. now let Tn := (Xi + Yi ) be the time that the nth – bulb is replaced and Nt := sup {n ≥ 0 : Tn ≤ t} denote the number of bulbs which have burned out up to time n. From this it follows that |F (x) − Fn (x)| ≤ 2/k and we have shown for ω ∈ Ωk and n ≥ N (ω ) that sup |F (x) − Fn (x)| ≤ 2/k. Let us further set xk = ∞ 1 Observation. Fn (x) := µn ((−∞. .d. limn→∞ Fn (x) = F (x) . .d random variables with E1Xi ≤x = P (Xi ≤ x) = F (x) . .i.

. Let Sn := j =1 Xj and n ∞ {Xn }n=1 We now verify the hypothesis of Theorem 12. E Hence it suﬃces to show Sn −an n L (P ) 2 ≤ P (Sn = Sn ) ≤ k=1 Sn −an P → n P (|Xk | > n) → 0 as n → ∞. k=1 (12. A key ingredient in this proof and proofs of other versions of the law of large numbers is to introduce truncations of the {Xk } .22 (Shannon’s Theorem). r} ⊂ N. Let {Xi }i=1 be a sequence of i. that the random sample {X1 . . k=1 (12. . Xn } should occur is approximately e−nH with high probability. n H+ 1 1 ln πn > ε ∪ H + ln πn < −ε n n 1 1 ln πn > −H + ε ∪ ln πn < −H − ε n n Proof.4 More on the Weak Laws of Large Numbers Theorem 12. In this case we consider n Sn := k=1 Xk 1|Xk |≤n . By the dominated convergence theorem.d. If {Xn }n=1 are i.8) ln p (Xi ) → −E [ln p (X1 )] = − i=1 k=1 p (k ) ln p (k ) =: H (p) . .i. an 1 := n n n If Page: 132 job: prob E (Xk : |Xk | ≤ n) = E (X1 : |X1 | ≤ n) → µ.d. it follows that P e−n(H +ε) ≤ πn ≤ e−n(H −ε) → 1 as n → ∞. 1 1 − ln πn = − n n n r P (|Xk | > n) → 0 k=1 (12.i. .cls date/time: 23-Feb-2007/15:20 .23 in three situations. let πn (ω ) = p (X1 (ω )) . P H+ 1 ln πn > ε n H− = = 1 n ln πn > ε → 0 as n → ∞. 2.d. . {p (k )}k=1 . L2 (P ) – random variables. Since Sn − an P → 0. Since {ln p (Xi )}i=1 are i. Observe that ESn = an and therefore. . πn . In particular if ε > 0.i. . Corollary 12. p (Xn (ω )) be the probability of ∞ the realization. an := k=1 E (Xk : |Xk | ≤ n) = nE (X1 : |X1 | ≤ n) . → 0 as n → ∞. . then 1 n Sn ∞ P → µ = EX1 . random variables with values in {1. 0 as n → ∞ and for this it suﬃces to show.132 12 Laws of Large Numbers ∞ n Example 12.24. . . Further. . . Sn − an n 2 = ≤ 1 1 Var (Sn ) = 2 n2 n 1 n2 n n Var Xk 1|Xk |≤n k=1 2 E Xk 1|Xk |≤n → 0 as n → ∞. Thus the probability. P S − an Sn − an − n >ε n n =P Sn − Sn >ε n n = πn > en(−H +ε) c ∪ πn < en(−H −ε) c = πn ≤ en(−H +ε) ∩ πn ≥ en(−H −ε) = e−n(H +ε) ≤ πn ≤ e−n(H −ε) . Suppose that n is a sequence of independent random variables.7) and 1 n2 then n 2 E Xk : |Xk | ≤ n → 0.9) macro: svmonob.23 (Weak Law of Large Numbers). (X1 (ω ) . Xn (ω )) .. The number H is called the entropy r of the distribution. . k=1 12. = πn > en(−H +ε) ∪ πn < en(−H −ε) and H− 1 ln πn > ε n c Since {Sn = Sn } ⊂ ∪n k=1 {|Xk | > n} . Proof. Let p (k ) := P (Xi = k ) > 0 for 1 ≤ k ≤ r.

Then E |X | : |X | ≤ n = 2 0 2 M n With these observations we may now apply Theorem 12.7) is satisﬁed.5 Maximal Inequalities 133 Moreover.23 have been veriﬁed. Let {Xn } be a sequence of independent random variables with mean zero.i. then 1 2 lim E |X | : |X | ≤ n = 0. Proof. n ≤2 0 xP (|X | ≥ x) dx = 2 0 P (|Xk | > n) = nP (|X1 | > n) ≤ n k=1 1 2 E |X1 | → 0 as n → ∞.d. n2 Now given ε > 0. L (P ) – ran- 1 µ = EX1 . 12. Chebyschev’s inequality.12.i.28 (Kolmogorov’s Inequality). n→∞ n Since ε > 0 was arbitrary.26. Let X be a random variable such that τ (x) := xP (|X | ≥ x) → 0 as x → ∞. n Also 1 n2 n k=1 |X1 | 1 2 2 1|X1 |≤n E Xk : |Xk | ≤ n = E |X1 | : |X1 | ≤ n = E |X1 | n n P (|Xk | > n) = nP (|X1 | > n) = τ (n) → 0 as n → ∞. and by Chebyschev’s inequality.26 and the identity. 1 n2 n 2 E Xk : |Xk | ≤ n = k=1 1 2 E |X1 | : |X1 | ≤ n . then P 1 n Sn → ∞ {Xn }n=1 τ (x) dx + 2 M τ (x) dx ≤ 2KM + 2 (n − M ) ε are i.10) n→∞ n Note: If X ∈ L1 (P ) . To prove this we observe that 1 1 2 2 E X1 : |X1 | ≤ n ≤ E X1 → 0 as n → ∞ n n E |X | : |X | ≤ n = E 2 n 2 10≤x≤|X |≤n xdx = 2 n P (0 ≤ x ≤ |X | ≤ n) xdx τ (x) dx.25 (Khintchin’s WLLN). (12. and ∗ Sn = maxj ≤n |Sj | .8). k=1 and the latter expression goes to zero as n → ∞ by the dominated convergence theorem. since |X1 | |X1 | 1|X1 |≤n ≤ |X1 | ∈ L1 (P ) n 1| and limn→∞ |X1 | |X n 1|X1 |≤n = 0. Sn := X1 + · · · + Xn . the proof is complete.5 Maximal Inequalities Theorem 12. If dom variables. (12. and τ (x) := xP (|X1 | > x) → 0 as x → ∞. then by Chebyschev’s inequality and the dominated convergence theorem. (12. Then for any α > 0 we have ∗ P (SN ≥ α) ≤ 1 2 ∗ : |SN |≥α . (12. Eq. let M = M (ε) be chosen so that τ (x) ≤ ε for x ≥ M. Corollary 12. then the hypothesis of Theorem 12. E SN α2 Page: 133 job: prob macro: svmonob.27 (Feller’s WLLN). where K = sup {τ (x) : x ≥ 0} . follows from Lemma 12.23 are satisﬁed. and the dominated convergence theorem. If {Xn }n=1 are i. 1 n2 n 2 E Xk : |Xk | ≤ n = k=1 Proof. n Lemma 12. Eq.d. Again we have by Eq. Dividing this estimate by n and then letting n → ∞ shows 1 2 lim sup E |X | : |X | ≤ n ≤ 2ε.cls date/time: 23-Feb-2007/15:20 . τ (x) ≤ E [|X | : |X | ≥ x] → 0 as x → ∞. Hence again the hypothesis of Theorem 12. Since n ∞ Proof.9). Corollary 12. that n k=1 1 P (|Xk | > n) = nP (|X1 | > n) ≤ n E [|X1 | : |X1 | > n] → 0 as n → ∞.23 to complete the proof.

Xj ) . and Nn+1 /Nn → 1 as n → ∞. it follows that 0 = lim S∗ ≤ lim m p p m→∞ N m→∞ N m→∞ mp n(m) n(m)+1 ∗ ∗ SN SN n(m)+1 n(m)+1 lim = lim = 0 a. . ε2 N 2p ε N ε N E 2 SN : ∗ |SN | >α =E N 2 SN :J ≤N = j =1 E 2 2 SN Hence if we suppose that Nn = nα with α (2p − 1) > 1. Let Sn := X1 + · · · + Xn . Nn S∗ = ( ∗) = 0. then for any β ∈ (0. . . Since ∗ SN n(m) p Nn (m)+1 ∗ ∗ SN Sm n(m)+1 ≤ p ≤ p m Nn (m) α :J =j ≥α P [J = j ] = α P The equality. Observe that {J = j } = {|S1 | < α. Xn ) – measurable and hence 1J =j Sj and (Sn − Sj ) are independent. . where j ≤N cN (α) := max P (|SN − Sj | > α) . . 1 n n ≤ That is limm→∞ ∗ Sm mp Theorem 12. Letting Sn = k=1 Xk and p > 1/2. . . 2) (Sn − Sj ) is σ (Xj +1 .cls date/time: 23-Feb-2007/15:20 .s. j ≤N Yk − µ = O k=1 1 nβ . we have 1 Sn → 0 a. . Let {Xn } be a sequence of independent random variables and let α > 0. ∗ SN n(m) ∗ SN n(m) Corollary 12. . . np If {Yn } is a sequence of independent random variables EYn = µ and σ 2 = Var (Xn ) < ∞. Page: 134 job: prob macro: svmonob. > α) . 1/2) .36 below.29 (L2 – SSLN). we may choose n = n (m) such that ≥ j =1 E 2 Sj nα = Nn ≤ m < Nn+1 = (n + 1) . Let J = inf {j : |Sj | ≥ α} with the inﬁmum of the empty set being taken to be equal to ∞. Now N Proof.s. . we have for every ε > 0 that P ∗ SN ≥ε Np ∗ = P (SN ≥ εN p ) ≤ 1 C 1 2 E SN = 2 2p CN = 2 (2p−1) . Xj ) – measurable.s.s. and σ 2 = EXn < ∞. is a consequence of the observations: 1) 1J =j Sj is σ (X1 . |Sj −1 | < α. We will give another proof in Corollary 12. |Sj | ≥ α} ∈ σ (X1 . (The proof of this Corollary may be skipped. Then for all α > 0. . . for m ∈ N. (∗) . and so 3) E [Sj (SN − Sj ) : J = j ] = E [Sj 1J =j (SN − Sj )] = E [Sj 1J =j ] · E [SN − Sj ] = E [Sj 1J =j ] · 0 = 0. then we have :J =j ∞ P n=1 ∗ SN n p ≥ε Nn ∞ ≤ n=1 C ε2 nα(2p−1) <∞ = j =1 N E (Sj + SN − Sj ) : J = j 2 E Sj + (SN − Sj ) + 2Sj (SN − Sj ) : J = j j =1 N 2 and so by the ﬁrst Borel – Cantelli lemma we have P ∗ SN n p ≥ ε for n i. n To ﬁnish the proof. .30 (Skorohod’s Inequality).) From Theorem 12.28. P (|SN | > α) ≥ (1 − cN (α)) P max |Sj | > 2α .134 12 Laws of Large Numbers Proof. Let {Xn } be a sequence of independent rann 2 dom variables with mean zero. . . p p m→∞ N m→∞ N n(m) n(m)+1 = lim = 0 a. = E j =1 N 2 Sj + (SN − Sj ) : J = j N 2 j =1 2 ∗ (|SN | 2 Nn From this it follows that limn→∞ N p = 0 a.o.

n≥M ε 1−ε = j =1 ε . |SN − Sj | ≤ α) lim |Sn − Sm | = 0 a. it suﬃces to show. i. 1−ε Also observe that on {J = j } .n≥M m. N space and {ξi }i=1 be independent symmetric (i. let QM. then shows P (QM. it follows that δM → 0 as M → ∞.5 Maximal Inequalities 135 Proof. An application of Skorohod’s inequality. (12. |SN − Sj | ≤ α} we have |SN | > α. |SN | = |SN − Sj + Sj | ≥ |Sj | − |SN − Sj | > 2α − |SN − Sj | . Hence on the {J = j.cls date/time: 23-Feb-2007/15:20 .s. |Sj | > 2α} and therefore max |Sj | > 2α j ≤N N Proof. . SN < r). random ∞ ∞ variables then n=1 Xn converges in probability iﬀ n=1 Xn converges a. . SN < r ) (12. Since P (|SN | > α) ≥ j =1 P (J = j ) (1 − c) = (1 − c) P max |Sj | > 2α . by assumption. Let X be a separable Banach N d Under the assumption that P (|SN − Sj | > α) ≤ c for all j ≤ N. there exists M = M (ε) ∈ N such that maxM ≤j ≤N P (|SN − Sj | > ε) < ε for all N ≥ M. Hence ti follows from this identity and the independence of {Xn } that N we may further conclude. Let Sn := Xk .i. Page: 135 job: prob ∞ ∗ ∗ ∗ P (SN ≥ r) = P (SN ≥ r. P (QM ≥ 2ε) ≤ Since. Let Sk := i=1 ξi and Sk := supj ≤k Sj with the convention ∗ that S0 = 0. ∗ = P ( SN ≥ r) + P (SN ≥ r. SN ≥ r) + P (SN ≥ r. Since almost sure convergence implies convergence in probability.30 we have the following convergence result. Then ∗ P (SN ≥ r) ≤ 2P ( SN ≥ r) .d. j ≤N To this end.11) Proof.N := supM ≤n≤N |Sn − SM | .31 (L´ evy’s Theorem). Given M ∈ M. {J = j. ξi = −ξi ) random variables k ∗ with values in X. Proposition 12. (1 − maxM ≤j ≤N P (|SN − Sj | > ε)) 1−ε n k=1 Since QM.e. Sj −1 < r . |SN − Sj | ≤ α} ⊂ {|SN | > α} for all j ≤ N. if Sn is convergent in probability then Sn is almost surely convergent. j ≤N ∗ {SN ≥ r} = N j =1 ∗ Sj ≥ r. it P follows that limM →∞ δM =: δ exists and because δM → 0 we may concluded that δ = 0 a. we ﬁnd P (|SN − Sj | ≤ α) ≥ 1 − c and therefore.n→∞ ∞ P (|SN | > α) ≥ j =1 N P (J = j. Moreover. . and therefore {Sn }n=1 is almost surely Cauchy and hence almost surely convergent. Theorem 12. As an application of Theorem 12.s.N ≥ 2ε) ≤ ε P (|SN − SM | > ε) ≤ .e.32 (Reﬂection Principle). Suppose that {Xn }n=1 are i. let J = inf {j : |Sj | > 2α} with the inﬁmum of the empty set being taken to be equal to ∞. δM := sup |Sn − Sm | ≤ sup [|Sn − SM | + |SM − Sm |] = 2QM m.N ↑ QM as N → ∞.s. Observe that {J = j } = {|S1 | ≤ 2α. we may conclude {J = j } . since δM is decreasing in M. let QM := supn≥M |Sn − SM | and for M < N.12) where macro: svmonob. P (δM > 4ε) ≤ P and since ε > 0 is arbitrary. Given ε ∈ (0. . Our goal is to compute P max |Sj | > 2α . Thus we have shown m. |Sj −1 | ≤ 2α.12. = j =1 P (J = j ) P (|SN − Sj | ≤ α) . 1) .

for all M < N. SN > r δM := sup |Sj − Sk | ≤ sup [|Sj − SM | + |SM − Sk |] ≤ 2QM j. δM → 0 as M → ∞.11 which we restate here. P (QM ≥ α) ≤ Since 1 α2 ∞ If Sj ≥ r and 2Sj − SN < r.n→∞ ∞ P This estimate along with the estimate in Eq. ∗ Sj −1 < r. This shows. Let Sn := j =1 Xj where Xj := Yj − EYj . Theorem 12. lim |Sn − Sm | = 0 a.12) completes the proof of the theorem. ∗ = P ( Sj ≥ r. ∞) are sequences such that ak ↑ ∞ and k=1 x ak exists.28. By Proposition 12. If ∞ ∞ j =1 Var (Yj ) < ∞. Sj −1 < r. An application of L´ evy’s ∞ Theorem 12.s. k=1 Page: 136 job: prob macro: svmonob.34 (Kronecker’s Lemma). ∗ Sj −1 < r. j =M +1 SN < r ) ≤ j =1 ∗ P ( Sj ≥ r. Suppose ∞ that {Yn }n=1 are independent square integrable random variables. is L2 (P ) convergent and hence convergent in probability. 2S j − S N < r ⊂ Sj ≥ r.k≥M and therefore. and therefore {Sn }n=1 is almost surely Cauchy and hence almost surely convergent. j =M +1 2Sj − SN < r).s. Lemma 12. Proof. Sj −1 < r. it follows that P limM →∞ δM =: δ exists and because δM → 0 we may concluded that δ = 0 a. Thus we have shown m. then r > 2Sj − SN ≥ 2 Sj − SN ≥ 2r − SN and hence SN > r. the sum. SN < r ) = j =1 P ( Sj ≥ r. Then 1 n→∞ an lim n xk = 0. j =M +1 < r. (12. then j =1 (Yj − EYj ) converges a. we may further conclude. SN > r) ≤ P ( SN ≥ r). According to Kolmogorov’s inequality. Sj −1 < r.136 12 Laws of Large Numbers N ∗ P (SN ≥ r. (12. SN > r ) ∗ = P (SN ≥ r. P max |Sj − SM | ≥ α ≤ 1 1 2 E (SN − SM ) = 2 2 α α 1 α2 N N 2 E Xj j =M +1 ∞ Sj + k>j ξk < r ) M ≤j ≤N ∗ = P ( Sj ≥ r. j =1 (Yj − EYj ) .31 then shows j =1 (Yj − EYj ) is almost surely convergent. ∗ SN < r) ≤ P ( Sj ≥ r. Sj ≥ r. ∗ P ( Sj ≥ r. Sj −1 < r. ∗ Sj −1 Var (Xj ) . ∗ SN < r) = P ( Sj ≥ r. Since δM is decreasing in M.33 (Kolmogorov’s Convergence Criteria). SN < r). Sj −1 < r.s. Theorem 12. Sj −1 < r. First proof. 12. Suppose that {xk } ⊂ R and {ak } ⊂ ∞ k (0.e.13) gives N ∗ P (SN ≥ r. Sj −1 < r. Sj − k>j ξk < r ) = Var (Xj ) . with QM := supj ≥M |Sj − SM | . Var (Xj ) → 0 as M → ∞.5.13) By symmetry and independence we have ∗ P ( Sj ≥ r. (12. SN > r). n Second proof. Letting N → ∞ in this inequality shows. i.k≥M j.6 Kolmogorov’s Convergence Criteria and the SSLN We are now in a position to prove Theorem 12. P (δM ≥ 2α) ≤ 1 α2 ∞ Combining the estimate with Eq.cls date/time: 23-Feb-2007/15:20 .

For example. Since xk = ak (rk − rk+1 ) . or bn = n1/2 (ln n) Eq. and 1 lim n→∞ an Page: 137 m < ∞. Theorem 12. We are going to show n 1 lim x (s) ds = 0. Integrating this equation shows s s = sup |rk | · lim sup k≥m n→∞ 1 [an − am−1 ] = sup |rk | . we ﬁnd Sn 1 = an an = n (Xk − EXk ) = lim k=1 n→∞ Sn − ESn . Xk − EXk is convergent a.36 (L2 – SSLN). an k≥m This completes the proof since supk≥m |rk | → 0 as m → ∞. r (u) a (u) du. bn ak (rk − rk+1 ) = k=1 n 1 an n n+1 ak rk − k=1 k=2 ak−1 rk Corollary 12. aj Therefore an application of Kronecker’s Lemma implies 0 = lim 1 n→∞ bn n so that rk → 0 as k → ∞ by assumption. Let a (s) ∈ (0. we go to the discrete case. Letting Sn = k=1 Xk and µ := EXn . Let k s k=1 Var (Xk ) <∞ b2 k then Sn − ESn → 0 a.s. ≤ lim sup a (s) s→∞ u≥s0 u≥s0 With this as warm-up. Let {Xn } be a sequence of independent square integrable random variables and bn be a sequence such that bn ↑ ∞. we have 1 (Sn − nµ) → 0 a. (12. Let {Xn } be a sequence of independent rann 2 dom variables such that σ 2 = EXn < ∞. ∞) and x (s) ∈ R be continuous ∞ (s) functions such that a (s) ↑ ∞ as s → ∞ and 1 x a(s) ds exists.6 Kolmogorov’s Convergence Criteria and the SSLN 137 Proof. Before going to the proof. a (u) ≤ sup |rk | · lim sup k≥m n→∞ 1 an n (ak − ak−1 ) k=m Then by assumption.35.33. (summation by parts) k=2 Using the fact that ak − ak−1 ≥ 0 for all k ≥ 2.12. Corollary 12. we may conclude lim sup n→∞ Sn 1 ≤ lim sup an a n→∞ n = lim sup n→∞ n (ak − ak−1 ) |rk | k=2 n x (u) du and ∞ 1 an (ak − ak−1 ) |rk | k=m r (s) := s X (u) du = a (u) ∞ s x (u) du. we could take bn = n or 1/2+ε (ak − ak−1 ) |rk | = 0 k=2 bn = n for an p > 1/2.s. bk k=1 ∞ Sk := j =1 xj and rk := j =k xj .cls for any ε > 0.14) as macro: svmonob.s. n→∞ a (n) 1 Let X (s) := s 0 for any m ∈ N. We may rewrite job: prob date/time: 23-Feb-2007/15:20 . r (s) → 0 as s → 0 and X (s) = −a (s) r (s) .14) bn provided bn ↑ ∞ and p ∞ 1 n=1 b2 n 1 a1 r1 − an rn+1 + an (ak − ak−1 ) rk . If ∞ X (s) − X (s0 ) = − s0 a (u) r (u) du = −a (u) r (u) |s u=s0 + s0 Dividing this equation by a (s) and then letting s → ∞ gives 1 a (s0 ) r (s0 ) − a (s) r (s) |X (s)| = lim sup + r (u) a (u) du lim sup a (s) a (s) a (s) s0 s→∞ s→∞ s 1 ≤ lim sup −r (s) + |r (u)| a (u) du a (s) s0 s→∞ a (s) − a (s0 ) sup |r (u)| = sup |r (u)| → 0 as s0 → ∞. By Kolmogorov’s Convergence Criteria. (12. bn ∞ Proof. let us warm-up by proving the following continuous version of the lemma.

15) Page: 138 macro: svmonob. n n Proof. (n + 1) n Hence an application of Proposition 12. However. ∞ ∞ ∞ 1n≤y ≤ y ≤ n=1 n=1 1n≤y + 1 = n=0 1n≤y .7) under the assumption that E |X1 | < ∞. (12.d.35. y 1+2ε wherein we have made the change of variables..s. First observe that for all y ≥ 0 we have. y = ln x. ε n=0 job: prob ∞ lim 1 Sn = E [c] = c. Lemma 12.i. (12. Let us simply observe here that ∞ Proof.39.o. then E |X | = 0 p ∞ ∞ psp−1 P (|X | ≥ s) ds = 0 psp−1 P (|X | > s) ds. So to ﬁnish the proof it suﬃces to show 3.s. Proof.. If 1 n Sn ∞ 12. n ∞ ∞ 1 n1/2 (ln n) 1/2+ε 2 ∞ = 1 1+2ε n=2 n=2 n (ln n) by comparison with the integral ∞ 2 1 x ln 1+2ε ∞ x dx = ln 2 1 ey y 1+2ε ∞ ey dy = ln 2 1 dy < ∞.37 Under the hypothesis in Corollary 12.i.s. Moreover by ∞ 1 Exercise 11. Borel zero-one law shows ∞ the statement in Eq.16) and then take expectations gives the estimate in Eq. → c a.138 12 Laws of Large Numbers Sn − nµ = o (1) bn or equivalently.16) Taking y = |X | /ε in Eq. µ=E 1 Sn → E n n→∞ Taking expectations of this identity along with an application of Tonelli’s theorem completes the proof.40. n (12.38. 2.36. (12.15).o. n=1 P (|X1 | ≥ εn) < ∞. (12. By the fundamental theorem of calculus.. Proof. ∞ 2. and 3. Corollary 12.39. ∞ 3. Proposition 12. Sn+1 Sn 1 1 Xn+1 = − = ε n + Sn − n+1 n+1 n+1 n n+1 Sn 1 = εn + → 0 + 0 · c = 0. There exists ε > 0 such that n=1 P (|X1 | ≥ εn) < ∞. Proof. n| 4. n Sn n=1 is a uniformly integrable sequenced and therefore. = P (|Xn | ≥ nε i. random variables such that 1 1 n Sn → c ∈ R a. To this end n| we start by noting that limn→∞ |X n = 0 a. iﬀ 0=P |Xn | ≥ ε i. (12. then ∞ P (|X | ≥ nε) ≤ n=1 1 E |X | ≤ P (|X | ≥ nε) . easily follows from Lemma 12.17) Our next goal is to prove the Strong Law of Large numbers (in Theorem 12.cls date/time: 23-Feb-2007/15:20 .s. Suppose that {Xn }n=1 are i.3.) for all ε > 0. random variables. E |X1 | < ∞. |X | = 0 p |X | ∞ ∞ psp−1 ds = p 0 1s≤|X | · sp−1 ds = p 0 1s<|X | · sp−1 ds. then εn := Sn+1 n+1 − Sn n → 0 a. and therefore.17) is equivalent to n=1 P (|Xn | ≥ nε) < ∞ for all ε > 0.41. This corollary is a special case of Corollary 12. n→∞ lim Sn − nµ n1/2 (ln ln n) 1/2 = √ 2σ a. The equivalence of items 1.d. For all ε > 0.s.7 Strong Law of Large Numbers Lemma 12.40 shows Xn ∈ L1 (P ) . then the following are equivalent: 1. limn→∞ |X n = 0 a. Sn bn − µ = o (1) . Fact 12. since {|Xn | ≥ nε}n=1 are independent sets. is equivalent to 4. then Xn ∈ L (P ) and µ := EXn = c. If X is a random variable and ε > 0. Suppose that X : Ω → R is a random variable. Suppose that {Xn }n=1 are i.s.

12. But by Lemma 12. 2 n2 n x n=1 ∞ Similarly. For all x ≥ 0.s.7 below. Therefore by Kolmogorov’s convergence criteria.s.43. x ∞ 1 dt a t2 n≥x 1 1 2 2 2 = E |X | ϕ (|X |) E |X | : 1|X |≤n = E |X | 1 2 2 |X |≤n n n n=1 n=1 = 1/a.cls date/time: 23-Feb-2007/15:20 . x see Figure 12. Theorem 12.40. By Proposition 12. n Sn → µ a. 1 1 2 = = 2 ≤ for 0 < x ≤ 1. Therefore it suﬃces to show 1 limn→∞ n Sn = µ a.7 Strong Law of Large Numbers 139 Lemma 12.44 (Kolmogorov’s Strong Law of Large Numbers). for x > 1. where Sn := X1 + · · · + Xn . 1 ≤ 2 · min n2 1 . ≤ 2E |X | 2 ∞ ∞ Proof.7 which we restate here.42. For example. With this as preparation we are now in a position to prove Theorem 12.s.43. The implication. implies Xn ∈ L1 (P ) and EXn = µ has already been proved in Corollary 12. iﬀ Xn is integrable and in which case EXn = µ. k=1 Page: 139 job: prob macro: svmonob. Suppose that X : Ω → R is a random variable.41. Sup∞ pose that {Xn }n=1 are i. 2 n n=1 ∞ Xn − EXn is almost surely convergent. then 1 2 E |X | : 1|X |≤n ≤ 2E |X | . ϕ (x) := 1 1x≤n = n2 n=1 ∞ Proof. ∞ ∞ ∞ P (Xn = Xn ) = n=1 n=1 P (|Xn | > n) = n=1 P (|X1 | > n) ≤ E |X1 | < ∞. random variables and let Sn := X1 + · · · + Xn . 1 Then there exists µ ∈ R such that n Sn → µ a. So let us now assume Xn ∈ L1 (P ) and let µ := EXn .d. Let Xn := Xn 1|Xn |≤n . 1 Proof. and hence {Xn } and {Xn } are tail equivalent.i.42.1 . n n=1 Kronecker’s lemma then implies 1 n→∞ n lim n ∞ (Xk − EXk ) = 0 a. ∞ ∞ 1 1 ≤ 1 + dt = 1 + 1 = 2 2 2 n t 1 n=1 and so n≥x 1 ∧1 |X | ≤ 2E |X | . ∞ ∞ E |X | 1 2 n |Xn |≤n Var (Xn ) E |Xn | ≤ = 2 2 2 n n n n=1 n=1 n=1 ∞ ∞ 2 = n=1 E |X1 | 1|X1 |≤n n2 2 ≤ 2E |X1 | < ∞. This is a simple application of Lemma 12. 1 1 ≤ 2+ 2 n x ∞ x n≥x 1 1 1 1 dt = 2 + = 2 t x x x 1+ 1 x ≤ 2 . Lemma 12.s. The proof will be by comparison with the integral.

(12.s. Sn := macro: svmonob.cls n j =1 Yj . Proposition 12. Then limN →∞ a. An application of Kronecker’s Lemma then implies Page: 140 job: prob where as usual.12. Suppose that {Yn }n=1 are independent random variables such that there exists c < ∞ such that |Yn | ≤ c < ∞ a. We now assume that ∞ {Xn }n=1 are i. j j j2 and hence we conclude that limN →∞ ρN < ∞.i. and further assume ∞ ∞ 2 EYn = 0.45. So dividing Eq. We start with a couple of lemmas. random variables with a continuous distribution function and let Aj denote the event when Xj is a record. So in the spirit of the proof of the strong law of large numbers let us compute. We end this section with another example of using Kolmogorov’s convergence criteria in conjunction with Kronecker’s lemma.46.8 Necessity Proof of Kolmogorov’s Three Series Theorem This section is devoted to the necessity part of the proof of Kolmogorov’s Three Series Theorem 12. . date/time: 23-Feb-2007/15:20 . If n=1 Yn is almost surely convergent then n=1 EYn < ∞. .19) by ln N and letting N → ∞ gives the desired limit in Eq. E1Aj = Var 1Aj = E12 Aj − E1Aj Observing that n n 2 1 j ∞ 1 j n→∞ ln N = 1.140 12 Laws of Large Numbers So to ﬁnish the proof.19) = ρN + j =1 N j =1 1Aj µN ln N = 1 where N |ρN | = j =1 ln and j+1 1 − = j j N ln (1 + 1/j ) − j =1 1 ∼ j N j =1 1 j2 = 1 j−1 1 − 2 = .s. (12. X2 . . Xk−1 }} . Here we have used the dominated convergence theorem to see that an := E X1 1|X1 |≤n → µ as n → ∞. Lemma 12. E X1 1|X1 |≤n k=1 So to ﬁnish the proof it only remains to show lim N 1 j =1 j = lim E X1 1|X1 |≤n = µ. i.e.18). Proof. ∞ ∞ we are lead to try to normalize the sum j =1 1Aj by ln N. P (supn |Sn | ≤ λ) 2 (12. It is now easy (and standard) to check that n 1 limn→∞ n k=1 an = limn→∞ an = µ as well. y2 Therefore by Kolmogorov’s convergence criteria we may conclude ∞ 1 Aj − ln j 1 j ∞ = j =2 j =2 1 Aj 1Aj −E ln j ln j EYj2 ≤ j =1 (λ + c) for all λ > 0.s. ∞ Var j =2 1Aj ln j ∞ = j =2 1 j−1 ∼ ln2 j j 2 ∞ 2 1 1 dx = ln2 x x ∞ ln 2 1 dy < ∞. More precisely the following estimate holds. Aj := {Xj > max {X1 . Keeping the preceding notation and let µN := denote the number of records in the ﬁrst N observations.28 that {Aj }j =1 are independent and P (Aj ) = for all j. Recall from Renyi Theorem 7.d. . (12. Since 1Aj are Bernoulli random variables. it only remains to observe 1 lim n→∞ n n k=1 1 EXk = lim n→∞ n n→∞ n E Xn 1|Xn |≤n k=1 1 = lim n→∞ n n n→∞ lim N j =1 1 Aj − ln N 1 j = 0 a.18) To see this write N +1 ln (N + 1) = 1 N 1 dx = x N j j +1 j =1 1 dx x N j +1 j N = j =1 1 1 − x j 1 j dx + j =1 1 j (12. E1Aj = j =1 j =1 1 ∼ j N 1 N 1 dx = ln N x 12.20) is almost surely convergent.

i. Then for N ∈ N. Further let Yn (ω1 . Proof. ∞ n=1 = P (|Xn | > c) < ∞. P (supn |Sn | ≤ λ) job: prob 2 2 2 Thus by Kolmogorov’s convergence theorem. Proof. Hence it follows from Lemma 12.cls . τ = τλ := inf {n ≥ 1 : |Sn | > λ} . for all n. ω2 ) = n=1 n=1 Yn (ω1 ) − n=1 Yn (ω2 ) exists ≤ j =1 2 E (λ + c) : τ = j + E SN P [τ ≤ N ] 2 2 for P a.12. it follows that n=1 (Yn − EYn ) is ∞ ∞ convergent. and ≤ j =1 N 2 E (Sj −1 + Yj ) : τ = j + E SN j =1 P [τ = j ] ∞ ∞ ∞ Zn (ω1 . we may conclude that n=1 EYn is also convergent. τ = ∞ if {n ≥ 1 : |Sn | > λ} = ∅. then n=1 EYn converges as well. Let λ > 0 and τ be the ﬁrst time |Sn | > λ. Since Sn is convergent a. ω2 ) − Yn (ω1 . ω2 ) := Yn (ω2 ) and ∞ ∞ = = j =1 N E 2 Sj :τ =j + j =1 2 E (SN − Sj ) 2 P [τ = j ] N Zn (ω1 .46 that ∞ ∞ 2 EZn = n=1 ∞ n=1 ∞ 2 = (λ + c) + E SN P [τ ≤ N ] . ω2 ) = Yn (ω1 ) − Yn (ω2 ) . B := B0 ⊗ B0 . P (supn |Sn | ≤ λ) 2 Moreover. It follows from Eq.e. is almost surely convergent iﬀ for all c > 0 the following three series converge. ∞ then the random series. ∞> = Var (Zn ) = n=1 Var (Yn − Yn ) ∞ Putting this all together then gives.s. it follows that P (supn |Sn | < ∞) = 1 and therefore. (ω1 . P (supn |Sn | < λ) > 0 ad we learn that ∞ 2 EYj2 = lim E SN ≤ j =1 N →∞ (λ + c) < ∞.20) that if P (supn |Sn | < ∞) > 0.48. N 2 E SN :τ ≤N = j =1 N 2 E SN :τ =j = j =1 2 E Sj + 2Sj (SN − Sj ) + (SN − Sj ) : τ = j j =1 N N 2 N E |Sj + SN − Sj | : τ = j 2 Lemma 12.s. EZn = 0. λ↑∞ lim P sup |Sn | < λ n = 1. ω2 ) . convergent. and P := P0 ⊗ P0 . let τ be the “stopping time” deﬁned by. ∞ Proof. ω2 ) := Yn (ω1 . ∞ = (λ + c) + P [τ ≤ N ] · E form which it follows that 2 E SN ≤ 2 (λ + c) (λ + c) (λ + c) ≤ = 1 − P [τ ≤ N ] 1 − P [τ < ∞] P [τ = ∞] (λ + c) .e. 2 2 2 E SN = E SN : τ ≤ N + E SN :τ >N 2 ≤ E SN : τ ≤ N + λ2 P [τ > N ] . date/time: 23-Feb-2007/15:20 Page: 141 macro: svmonob. We are now ready to complete the proof of Theorem 12. Suppose that {Yn }n=1 are independent random variables such ∞ that there exists c < ∞ such that |Yn | ≤ c a. 2 2 E SN ≤ (λ + c) + E SN 2 ≤ (λ + c) + E SN 2 2 2 P [τ ≤ N ] + λ2 P [τ > N ] P [τ ≤ N ] + (λ + c) P [τ > N ] 2 SN 2 [Var (Yn ) + Var (Yn )] = 2 n=1 n=1 Var (Yn ) . P (supn |Sn | < ∞) .8 Necessity Proof of Kolmogorov’s Three Series Theorem 141 Remark 12. Our goal is to show if {Xn }n=1 are independent random variables.12. Hence for λ suﬃciently large. B0 . (12.s. If n=1 Yn converges ∞ in R a. n=1 Xn . As usual. P0 ) be the probability space that {Yn }n=1 is deﬁned on and let Ω := Ω0 × Ω0 . then ∞ ∞ 2 j =1 Yj = limn→∞ Sn j =1 EYj < ∞ and hence by Kolmogorov’s Theorem.s. and in particular. Since n=1 Yn is a. Let (Ω0 . Then |Zn | ≤ 2c a.. exists a. ω2 ) := Yn (ω1 ) and Yn (ω1 .47.s..s. 1.

and E Xn 1|Xn |≤c converges.48 (with Yn = Xn ∞ c EXn = n=1 n=1 ∞ ∞ E Xn 1|Xn |≤c converges. Since {Yn } is uniformly bounded and EYn = 0 for all n. So according to Lemma 12. an application of Lemma 12. c ).}) = 0. c c .o.s. n=1 Var Xn 1|Xn |≤c = n=1 . P ({|Xn | ≥ c i. we now know that {Xn } and Xn := Xn 1|Xn |≤c are tail equivalent for ∞ c all c > 0 and in particular n=1 Xn is almost surely convergent for all c > 0. 3. According the Borel ∞ zero one law this implies for every c > 0 that n=1 P (|Xn | > c) < ∞. ∞ Since n=1 Xn is almost surely convergent. it follows that limn→∞ Xn = 0 a. and hence for every c > 0.2. ∞ n=1 ∞ n=1 Var Xn 1|Xn |≤c < ∞. we may now conclude that n=1 Yn is almost surely − EXn Letting Yn := Xn convergent. Given c this.46 allows us to conclude ∞ ∞ 2 EYn < ∞.

3 (Scheﬀ´ e’s Lemma). if A ∈ B we have λ (A) + λ (Ac ) = λ (Ω ) = 0. Corollary 13. {Ai }i=1 ⊂ B of Ω n . |fn − g | ≤ Gn ∈ L1 (m) . we have Fact: it is always possible to do this by taking m = µ + ν for example. λ (A) := µ (A) − ν (A) for all A ∈ B . ∞). ν ) := sup |µ (A) − ν (A)| .e. Lemma 13. for large n. dT V (µ. (Ω. Gn → G := 2g a. simply take A = {h > 0} (note Ac = {h ≤ 0}) in Eq. In particular this shows |λ (A)| = |λ (Ac )| and therefore. This is the question we will address in this chapter. Continuing the notation in Scheﬀ´ e’s lemma above.1) to ﬁnd |λ (A)| = 1 2 1 = 2 hdm − A Ac λ TV := sup i=1 |λ (Ai )| : n ∈ N and partitions. and fn → g. that when λ = µ − ν. Ω n→∞ 1 2 |f − g | dm.35 below. Since λ (Ω ) = µ (Ω ) − ν (Ω ) = 1 − 1 = 0. The total variation distance. hdm |h| dm = Ac You are asked to show in Exercise 13. Ω lim dT V (µn . Let µ and ν be two probability measure on a measurable space. ∞ Proof. see Proposition 13. Suppose that m is another positive measure on (Ω. dµn = fn dm with fn : Ω → [0.e. g : Ω → [0.e. ∞). ν ) = ∞ |h| dm + A 1 2 |h| dm. Let λ = µ − ν and h := f − g : Ω → R so that dλ = hdm. ν ) = sup |λ (A)| ≤ A∈B (13. B ) such that there exists measurable functions.2. Ω To prove the converse inequality. Therefore..4. let Gn := fn + g and observe that |fn − g | → 0 m – a. if {µn }n=1 is a sequence of probability measure of the form. dT V (µ. ν ) → 0 as n → ∞. when is µn close to µ for large n. . such that dµ = f dm and dν = gdm. Alternatively put. is an example of a “signed measure. is deﬁned as dT V (µ.1 Total Variation Distance Deﬁnition 13. Let h ∞ := supω∈Ω |h (ω )| when h : Ω → R is a bounded random variable. A∈B |λ (A)| = 1 1 hdm [|λ (A)| + |λ (Ac )|] = hdm + 2 2 Ac A 1 1 ≤ |h| dm + |h| dm = |h| dm. Ω For the second assertion. then dT V (µn .1.. f.1 below. if we let µn (A) := P (Xn ∈ A) and µ (A) := P (X ∈ A) . (13.34.” For signed measures. 13. one usually deﬁnes n 1 2 |h| dm.1 Then dT V (µ.13 Weak Convergence Results Suppose {Xn }n=1 is a sequence of random variables and X is another random variable (possibly deﬁned on a diﬀerent probability space). We would like to understand when. ν ) = 1 lim 2 n→∞ |fn − g | dm = 0. by the dominated convergence theorem 8. λ : B → R deﬁned by. 2 A 2 Ω Ac dT V (µ. The function.1) This shows Remark 13. Xn and X have nearly the “same” distribution. Ω Moreover.a. m . ν ) . ν ) = 1 2 µ − ν TV . 1 For a concrete application of Scheﬀ´ e’s Lemma. B ) . and Ω Gn dm = 2 → 2 = Gdm and n → ∞.

. then F (y +) < F (y −) and (F (y −) . F (y +)) are disjoint intervals of length greater that ε. 1 sup 2 hdµ − Ω Ω hdν : h ∞ ≤1 . . (13. n} so that Example 13. A∈BR m ((F (y −) . Hence it follows that 1 = m ([0. n . We begin by observing that hdµ − Ω Ω hdν = Ω h (f − g ) dm ≤ Ω ∞ |h| |f − g | dm ∞ and F (y ) := P (U ≤ y ) = (y ∧ 1) ∨ 0. i. show µ−ν TV = Ω |f − g | dm = 2dT V (µ. (13. 2 where µX = P ◦ X −1 and µY = P ◦ Y −1 . 1]) ≥ y ∈Cε ≤ h |f − g | dm = 2dT V (µ. From these formula. 2. the next simple example shows this deﬁnition is also too restrictive.2. (13. µ0 ) = |fn (ω ) − f0 (ω )| 2 ω ∈Ω and limn→∞ dT V (µn . Observe that y is the only point of discontinuity of F0 . . ν ) · h ∞ (13. ν ) = 1. f : X → R be a function. . B := 2Ω . This suggest that we should say that Xn converges in distribution to X iﬀ P (Xn ≤ y ) → P (X ≤ y ) for all y ∈ R.4). ν ) = Consequently. U ) = 1 for all n. suppose that ε > 0 is given and let Cε := {y ∈ R : F (y +) − F (y −) ≥ ε} . then Fn (y ) = P (Xn ≤ y ) = 1 i # i ∈ {1. B ) . from the proof of Scheﬀ´ e’s Lemma 13. If we let An = n : i = 1. Observe that if F : R → [0. ν ) h Ω Moreover. However. . Therefore C := ∪∞ k=1 C1/k is at most countable. .8 Let (X. Then it is reasonable to insist that Xn converges of X0 in distribution. y ∈ Cε . it easily follows that F (y ) = limn→∞ Fn (y ) for all y ∈ R. hdµn → Ω Ω hdν if dT V (µn . 1] is a non-decreasing function. Notation 13. F (y +))) ≥ ε · # (Cε ) and hence that # (Cε ) ≤ ε−1 < ∞. F (y +)) and (F (y −) . Suppose that Ω is a (at most) countable set. .3. To see this. These two equations prove Eqs. it follows that dT V (Xn . However. then dT F (µ. (13.7.e. for all bounded and measurable functions. Example 13. .1. Exercise 13. ν ) → 0. we have dT V (µ.144 13 Weak Convergence Results dT V (µ. and ∞ {µn }n=0 are probability measures on (Ω. Under the hypothesis of Scheﬀ´ e’s Lemma 13. . . The set of x ∈ X where f is continuous (discontinuous) at x will be denoted by C (f ) (D (f )). Let us observe that if we let Fn (y ) := P (Xn ≤ y ) and F (y ) := P (U ≤ y ) . Page: 144 job: prob macro: svmonob. if µ and ν are two probability measure on (R.2) 13.3. n} : ≤ y n n Proof. h : Ω → R. If y < y with y. Therefore. d) be a metric space. Notation 13. to U where i P (U ∈ A) = m (A ∩ [0.4) Nevertheless we would like Xn to be close to U in distribution.3) and in particular.2 hdµ − Ω Ω hdν ≤ 2dT V (µ.cls date/time: 23-Feb-2007/15:20 .6. 1]) for all A ∈ BR . then C (F ) is at most countable. More generally. Suppose that P (Xn = 1/n) = 1 for all n and P (X0 = 0) = 1. BR ) such that µ ({x}) = 0 for all x ∈ R while ν concentrates on a countable set. . ν ) = 1 2 hdµ − Ω Ω hdν when h := 1f >g − 1f ≤g . 2. Let fn (ω ) := µn ({ω }) for ω ∈ Ω. Exercise 13. then P (Xn ∈ An ) = 1 while P (U ∈ An ) = 0. ν ) . µY ) = sup |P (X ∈ A) − P (Y ∈ A)| .2 Weak Convergence i 1 for i ∈ {1. 2.2) and (13. Y ) := dT V (µX . Show 1 dT V (µn . Fn (y ) = 1y≥1/n → 1y≥0 = F0 (y ) for all y ∈ R except for y = 0.3) and the latter implies Eq. Suppose that P Xn = n = n Xn is a discrete “approximation” to the uniform distribution.5 Suppose that X and Y are random variables. . let dT V (X. . µ0 ) = 0 iﬀ limn→∞ µn ({ω }) = µ0 ({ω }) for all ω ∈ Ω.

If c := limn→∞ cn exists. Let {F.i. On the other hand. Since F (x − c) = P (X + c ≤ x) . } be a collection of proper distribution v w functions. 3.16. Example 13. 1 2 √ nσN (0.13. (so F (a) ↑ 1) allows us to conclude. Fn converges to F weakly and write. Then Fn → F iﬀ Fn → F. . 2.11. Let {F. } be a collection of right continuous non-increasing functions from R to [0. Suppose X is a non-zero random variables such that X = −X. {Xn }n=1 is said to converge weakly or to converge in distribution to a random variable X (written Xn =⇒ X ) iﬀ Fn (y ) := P (Xn ≤ y ) =⇒ F (y ) := P (X ≤ y ) . Suppose X is a random variable. Notice that F is not a distribution function because all 1 for all of the mass went oﬀ to +∞. then Fn ((a. If Fn → F. Again. 1] and by abuse of notation let us also denote the associated measures. macro: svmonob. 2. L2 (P ) random variables with µ := EX1 and σ 2 = Var (X1 ) .d. then Xn =⇒ X + c. Example 13. F is not a distribution function on R since half the mass went to −∞ while the other half went to +∞. iﬀ Fn (x) → F (x) for all x ∈ C (F ) . σ ) = σN (0. P (Xn = ±n) = 2 1 1 n. Proof. Proof. if cn → c as n → ∞. In the case where Fn and F are proper w and Fn → F. Xn does not converge to X almost surely or in probability. We say F is proper.6 and Fn (y ) := P (Xn ≤ y ) v w and F (y ) := P (Y ≤ y ) . b ∈ C (F ) . Then F (x) = F (a) + lim [Fn (x) − Fn (a)] ≤ F (a) + lim inf Fn (x) .n) + 1[n. .13 (Central Limit Theorem). {cn }n=1 ⊂ R. b ∈ C (F ) and therefore Fn → F. . If Xn and U are as in Example 13. if F is a distribution function of a probability measure. w 2.cls date/time: 23-Feb-2007/15:20 d Likewise. then Sn − nµ d √ =⇒ N (0. Fn converges to F vaguely and write. n→∞ Deﬁnition 13. Then 1. nσ 2 . ∞ d Letting a ↓ −∞.9. we have Sn ∼ = nµ + d b a b a n→∞ e− 2 x dx 1 2 e− 2 x dx. if F (∞) = 1 and F (−∞) = 0. Let F (x) := P (X ≤ x) and Fn (x) := P (Xn ≤ x) = P (X + cn ≤ x) = F (x − cn ) . then for all x ∈ C (F (· − c)) we have Fn (x) → F (x − c) . n Written out explicitly we ﬁnd lim P a< Sn − nµ √ ≤b σ n = P (a < N (0. then Fn → F and Fn → F. if {Xn }n=1 are i. F (x) − F (a) = lim [Fn (x) − Fn (a)] ≥ lim sup [Fn (x) − 1] = lim sup Fn (x) − 1 n→∞ n→∞ n→∞ which upon letting a ↑ ∞.12. iﬀ Fn ((a. i. Observe that Fn (x) → F (x − c) only for x ∈ C (F (· − c)) but this is suﬃcient to assert Xn =⇒ X + c.2 Weak Convergence 145 Deﬁnition 13. Fn → F. n→∞ Lemma 13. using the fact that F is proper. µF and µFn by F and Fn respectively. .e. n→∞ n→∞ w v Example 13. Xn =⇒ X as n → ∞. A sequence of random variables. if we suppose. So now suppose Fn → F and let a < x with a. Suppose that P (Xn = n) = 1 for all n. F (x) ≥ lim sup Fn (x) .∞) → 2 = F (y ) as n → ∞. . 1) = N nµ. Fn → F.14. 1) ≤ b) 1 =√ 2π or equivalently put √ √ 1 lim P nµ + σ na < Sn ≤ nµ + σ nb = √ n→∞ 2π More intuitively. Fn : n = 1. then Fn (y ) = 1y≥n → 0 = F (y ) as n → ∞. d n then Xn := (−1) X = X for all n and therefore. b]) → F ((a. 1) . ∞ Page: 145 job: prob . implies F (x) ≤ lim inf Fn (x) . and Xn = X + cn . Similarly. Fn : n = 1. b]) for all a. b]) for all a. we see that Xn =⇒ X + c. . Example 13. Lemma 13. The central limit theorem (see the ∞ next chapter) states. Clearly. x ∈ C (F ) . then Fn = 2 1[−n.15. we will write Fn =⇒ F. b]) = Fn (b) − Fn (a) → F (b) − F (a) = v v F ((a.10.

Fn =⇒ F.5) 2. =⇒ 2.b] and gε → 1(a. So Xn → c iﬀ macro: svmonob.e. 5. Suppose that {µn }n=0 is a sequence of probability measures on (R.6) and lim inf µn ((a. f dµn → R R ∞ lim inf µn ((a. b)) = µ0 ((a. b]) ≥ lim inf n→∞ n→∞ R gε dµn = R gε dµ0 . b]) .7) mon probability space and c ∈ R.19. Let g ∈ BC (R) . we may use the dominated convergence theorem to pass to the limit as ε ↓ 0 in Eqs. let Fn (y ) := µn ((−∞.7) to conclude. f : R → R (or f : R → C) and Cc (R) for those f ∈ C (R) which have compact support.) Let −∞ < a < b < ∞ with a. Since {|Xn − c| > ε} = {Xn > c + ε} ∪ {Xn < c − ε} it follows Xn → c iﬀ P (Xn > x) → 0 for all x > c and P (Xn < x) → 0 for all x < c. (The reader should compare Theorem 13.5) holds for all f ∈ BC (R) which are uniformly continuous. 3. Theorem 13. Then Xn =⇒ c iﬀ Xn → c. we will now complete the proof.) Proof.9. on this −1 = µn for all n and Yn → Y0 a. f dµ0 for all f ∈ BC (R) . (13.b) as ε ↓ 0. =⇒ 1. b]) . The proof of 4. such that Xn → X0 . Given Skorohod’s Theorem. i. =⇒ 3. Lemma 13. Recall that Xn → c iﬀ for all ε > 0. f (x) ≡ 0 if |x| is suﬃciently large. There exists a probability space (Ω. and 4. Eq. The picture deﬁnition of the trapezoidal functions. Clearly 1. b]) n→∞ and Page: 146 job: prob . P ) and random variables. Suppose that {Xn }n=0 is a sequence of random variables. fε and gε . Then lim sup µn ((a. (13. B .C. Hence we have shown that limn→∞ µn ((a.17.28 below. b ∈ C (F0 ) and for ε > 0. b]) ≤ lim sup n→∞ n→∞ R Corollary 13. let fε (x) ≥ 1(a. =⇒ 5. (13. Therefore it suﬃces to prove 3. Then the following are equivalent.17 with Corollary 13. For all f ∈ BC (R) . Indeed. P (|Xn − c| > ε) → 0. (Recall that example 13. (13.cls date/time: 23-Feb-2007/15:20 P P P Since fε → 1[a.18. b]) ≥ µ0 ((a.6) and (13. will be the content of Skorohod’s Theorem 13. and 5. Yn . 1. space such that P ◦ Yn Proof.16 shows the converse is in general false.5) holds for all f ∈ Cc (R) . then by Corollary 11. Fig. =⇒ 4. =⇒ 5. b]) ≤ µ0 ([a. (3. Proof.b] be the functions in Cc (R) pictured in Figure 13. b]) exists and is equal to µ0 ((a. 4.s. Eq.8) to conclude that E [g (Xn )] → E [g (X0 )] .) In this theorem we will write BC (R) for the bounded continuous functions. lim sup µn ((a. Suppose {Xn }n=1 is a sequence of random variables on a com∞ P P fε dµn = R fε dµ0 (13.T. we have f dµn = E [f (Yn )] → E [f (Y )] = R R ∞ D. g (Xn ) → g (X0 ) and since g is bounded.4. b]) = µ0 ((a.1.b] and gε (x) ≤ 1(a. we may apply the dominated convergence theorem (see Corollary 11.146 13 Weak Convergence Results The next theorem summarizes a number of useful equivalent characterizations of weak convergence. 13. then Xn =⇒ X0 . (13. f dµ0 as n → ∞. by the dominated convergence theorem.1. =⇒ 4. BR ) and for each n. n→∞ where the second equality in each of the equations holds because a and b are points of continuity of F0 . y ]) be the (proper) distribution function associated to µn . These conditions are also equivalent to P (Xn ≤ x) → 1 for all x > c and P P (Xn ≤ x) ≤ P (Xn ≤ x ) → 0 for all x < c (where x < x < c).

Exercise 13. We similarly deﬁne fδ : X → R∪ {−∞} by fδ (x) := Since fδ = − (−f ) . Then D (f ) is a Borel measurable subset of X. y )) . f : X → R be a function. Fn =⇒ F0 . limn→∞ Fn ((a. b]) . y ) < δ } . let f δ : X → R∪ {∞} be deﬁned by. U ⊂ R. αi+1 ). (1. y ∈Bx (δ ) and so by Fatou’s lemma.s. . b]) − µFn ((a. 3. b])| = sup |F (b) − F (a) − (Fn (b) − Fn (a))| a<b a<b ≤ sup |F (b) − Fn (b)| + sup |Fn (a) − Fn (a)| b a → 0 as n → ∞. let Bx (δ ) = {y ∈ X : d (x. b}) = 0 and therefore. f δ ≤ a is closed (or equivalently f δ > a is open) for all a ∈ R. Then F0 (∂A) = F0 ({a. Theorem 13.28 we may choose random variables. b]. such that P (Yn ≤ y ) = Fn (y ) for all y ∈ R and n ∈ N and Yn → Y0 a.) This follows from the observations: 1) C ⊂ R is closed iﬀ U := C c is open. then Fn converges to F uniformly on R. and D (f ) be the set of x ∈ X where f is discontinuous at x. Proof. n→∞ ∞ lim sup |F (x) − Fn (x)| = 0.21. 2) F (U ) = 1 − F (C ) . Suppose that F is a continuous proper distribution function.20 (The Portmanteau Theorem).3 “Derived” Weak Convergence Lemma 13. Then the following are equivalent. as n → ∞.) If F0 (∂A) = 0.13. that |F (x) − Fn (x)| ≤ (F (αi+1 ) − F (αi ))+|F (αi ) − Fn (αi )|+(Fn (αi+1 ) − Fn (αi )) .e. i. x ) < δ − d (x. Since U is open. Since this y is in Bx (δ ) whenever d (x.cls date/time: 23-Feb-2007/15:20 δ δ y ∈Bx (δ ) inf f (y ) . Given δ > 0.e. such that |F (αi+1 ) − F (αi )| ≤ ε for all i. This shows f δ > a is open in X. (2. For x ∈ X and δ > 0. 2. F : R → [0. lim inf n→∞ Fn (U ) ≥ F0 (U ) for open subsets. F (U ) = P (Y ∈ U ) = E [1U (Y )] ≤ lim inf E [1U (Yn )] = lim inf P (Yn ∈ U ) = lim inf Fn (U ) . Let (X. lim supn→∞ Fn (C ) ≤ F0 (C ) for all closed subsets. y ) + d (x.3 “Derived” Weak Convergence n→∞ 147 lim P (Xn ≤ x) = 0 if x < c = F (x) 1 if x > c P 2. x ) < δ ) it follows that f δ (x ) > a for all x ∈ Bx (δ − d (x. We end this section with a few more equivalent characterizations of weak convergence. n→∞ x∈R ∞ {Fn }n=1 where F (x) = P (c ≤ x) = 1x≥c . =⇒ 2. we will denote µFn (A) simply by Fn (A) for all A ∈ BR . Proof. then there exists y ∈ Bx (δ ) such that f (y ) > a. ⇐⇒ 3. Yn . for x ∈ [αi .) Let a. b]) = F0 ((a. 1] is uniformly continuous.17 and 13. if f δ (x) > a. Hints for part 2. 4. it follows that sup |µF ((a. and 3) lim inf n→∞ (−Fn (C )) = − lim supn→∞ Fn (C ) . In particular. Show. then Ao ⊂ A ⊂ A Therefore ¯ ≤ F0 A ¯ = F0 (A) .3. ¯ with F0 A ¯ \ Ao = 0. y ) (because then. d) be a metric space. Given ε > 0. −∞ = α0 < α1 < · · · < αn = ∞. i.e. ⇐⇒ 4. Suppose {Fn }n=0 are proper distribution functions. =⇒ 1. it follows that {fδ ≥ a} = (−f ) ≤ −a macro: svmonob. 13. d (x . Page: 147 job: prob We will begin by showing f δ is lower semi-continuous. and 3. we have shown Xn → c iﬀ Xn =⇒ c. F0 (A) = F0 (Ao ) ≤ lim inf Fn (Ao ) ≤ lim sup Fn A n→∞ n→∞ (4. The combination of Theorem 13. 1. b ∈ C (F0 ) and take A := (a. By abuse of notation. 1. Since C (F ) = R \ {c} . Indeed. y ) ≤ d (x. Fn =⇒ F0 . n→∞ n→∞ n→∞ (2.20 is often called the Portmanteau Theorem. f δ (x) := sup f (y ) . show that there exists. i.) By Theorem 13. Now show.s. C ⊂ R. it follows that 1U (Y ) ≤ lim inf 1U (Yn ) a. limn→∞ Fn (A) = F0 (A) for all A ∈ BR such that F0 (∂A) = 0. If is a sequence of distribution functions converging weakly to F.

c)]| ≤ ε. c)] = lim E [fk (Xn . For the second assertion we take g (x) = (x ∧ M ) ∨ (−M ) in Eq.148 13 Weak Convergence Results is closed for all a ∈ R. and Xn − b =⇒ Z.s. by taking f (x. c)] for all f ∈ BC R2 . and therefore limn→∞ E [f (Xn . Suppose that Xn =⇒ X and P Yn → c where c is a constant. In particular. f is bounded. c) . Suppose that xn → x with x ∈ C (f ) := D (f ) .22. Yn ) − f (Xn . c)] ≤ lim inf E [M − f (Xn . c)]| ≤ E [|f (Xn . P (Y0 ∈ D (g ◦ f )) ≤ P (Y0 ∈ D (f )) = P (X0 ∈ D (f )) = 0. Suppose that {Xn }n=1 are random variables. fδ is upper semi-continuous. M − E [f (X.cls . c)] for all f ∈ BC R2 with f ≥ 0. we ﬁnd E [fk (X. y ) = g (x + y ) and f (x. c) in the sense that E [f (Xn . (13. c)| : |Yn − c| > δ ] ≤ ε + 2M P (|Yn − c| > δ ) → ε as n → ∞. Yn ) =⇒ (X. If Xn =⇒ X0 and P (X0 ∈ D (f )) = 0. Let {Yn }n=0 be random variables on some probability space as in Theorem 13. i. Yn )] ≤ E [f (X. Yn )] . Yn )] → E [f (X.24 (Slutzky’s Theorem). lim sup |E [f (Xn . Theorem 13. c)] and hence we have shown.8) implies E [g (f (Xn ))] = E [g (f (Yn ))] → E [g (f (Y0 ))] = E [g (f (X0 ))] . Then (Xn . Theorem 13. Proof. Yn )] n→∞ n→∞ This proves the ﬁrst assertion. Then applying what we have just proved to fk := ϕk f.25 (δ – method). First suppose that f ∈ Cc R2 . Proof. where f0 ≤ f ≤ f 0 and f0 : X → R∪ {−∞} and f 0 : X → R∪ {∞} are measurable functions. b ∈ R. c)]| n→∞ ≤ lim sup |E [f (Xn . c)| : |Yn − c| ≤ δ ] + E [|f (Xn . y ) ∈ [0. then f (Xn ) =⇒ f (X0 ) . we know E [f (Xn . The proof is now complete since it is easy to see that D (f ) = f > f0 = f − f0 = 0 ∈ BX . Yn ) − f (Xn . Moreover. Let f : R → R be a Borel measurable functions. Now suppose f ∈ BC R2 with f ≥ 0 and let ϕk (x. n→∞ This inequality with f replaced by M − f ≥ 0 then shows. c)]| + lim sup |E [f (Xn . Since. Yn ) − f (Xn . lim sup E [f (Xn . Yn ) − f (Xn . 1] be continuous functions with compact support such that ϕk (x. Then f (xn ) → f (x) as n → ∞. c)] ≤ lim inf E [f (Xn . Hence it follows that g ◦ f ◦ Yn → g ◦ f ◦ Y0 a.e. So an application of the dominated convergence theorem (see Corollary 11. an If g : R → R be a measurable function which is diﬀerentiable at b. an Proof. c)] ≤ lim inf E [f (Xn . n→∞ n→∞ Letting k → ∞ in this inequality then implies that E [f (X. This completes the proof since any f ∈ BC R2 may be written as a diﬀerence of its positive and negative parts.8) where M is a bound on |f | . Then |E [f (Xn . If in addition. fδ ≤ f ≤ f δ for all δ > 0 and f δ ↓ f 0 and fδ ↑ f0 as δ ↓ 0. Ef (Xn ) → Ef (X0 ) . Yn )] . y ) ↑ 1 as k → ∞. an ∈ R\ {0} with limn→∞ an = 0. Yn ) − f (X. Yn )] = E [f (X. For g ∈ BC (R) we observe that D (g ◦ f ) ⊂ D (f ) and therefore. n→∞ n→∞ Hence we have shown. and for ε > 0. Yn )] .23 (Continuous Mapping Theorem). y ) ≤ δ. y )| ≤ ε if (x. Observe that Xn − b = an Xn − b =⇒ 0 · Z = 0 an date/time: 23-Feb-2007/15:20 ∞ Page: 148 job: prob macro: svmonob. Xn =⇒ X. then g (Xn ) − g (b) =⇒ g (b) Z. Theorem 13.8) ∞ c Since ε > 0 was arbitrary. y ) = 1 if |x| ∨ |y | ≤ k and ϕk (x. c) − f (X. Yn ) = Ef (X. 0 0 where M = sup |f | .28. we learn that limn→∞ Ef (Xn . y ) − (x . let δ := δ (ε) be chosen so that |f (x. c)] → E [f (X. Yn )] = M − lim sup E [f (Xn . we learn Xn + Yn =⇒ X + c and Xn · Yn =⇒ X · c respectively. Yn )] ≤ lim inf E [f (Xn . y ) = g (x · y ) with g ∈ BC (R) . y ) − f (x . n→∞ n→∞ Remark 13. (13.

4. then implies job: prob macro: svmonob.9) (13.s. y →x n→∞ ln xdx = [x ln x − x]0 = −1 1 and therefore. suppose now that {f > a} is open for all a ∈ R. limn→∞ Yn = e Let us further observe that . Xn − (−1) 1 √ n = √ 13.i. y ) < δ } . Our goal is to ﬁnd an −bn is weakly convergent to a non-constant random variable. and x ∈ X such that f (x) > a. Since.4). −1 a. 1) . g (Xn ) − g (−1) 1 √ n Taking g (x) := e using g (Xn ) = e Page: 149 x Y + (x) := inf {y ∈ R : F (y ) > x} . d Hence we have shown. where Bx (δ ) := {y ∈ X : 0 < d (x. Let Yn and Y be random variables on a ﬁxed probability space such that Yn = Xn = an Yn + b. Conversely. 1) → R be the function deﬁned by Y (x) = F ← (x) = sup {y ∈ R : F (y ) < x} . This completes the proof since g (b) Y = g (b) Z. 1) = N 0.4 Skorohod and the Convergence of Types Theorems 149 so that Xn =⇒ b and hence Xn → b. Similarly. Xn := ln Yn = n j =1 By the strong law of large numbers. Given a function. Example 13. Then d Uj − e−1 =⇒ N 0. Hence by the central limit theorem. 1] . Hence it follows that lim inf y→x f (y ) ≥ a and then letting a ↑ f (x) then implies lim inf y→x f (y ) ≥ f (x) .10) d lim sup f (y ) := lim sup f (y ) . Notation 13. let Y = F ← : (0. F : R → [0.s. lim Xn = E [ln U1 ] = 0 a.13. let Xn Therefore the δ – method implies. By deﬁnition of the derivative of g at b. Hence we may conclude that Bx (δ ) ⊂ {f > a} which shows {f > a} is open. ≥ ∞ f (x) Solution to Exercise (13. e−2 . a ∈ R. we have g (x + ∆) = g (b) + g (b) ∆ + ε (∆) ∆ where ε (∆) → 0 as ∆ → 0.9) holds. Show f is lower (upper) semi-continuous iﬀ lim inf y→x f (y ) lim supy→x f (y ) ≤ f (x) for all x ∈ X. and bn such that Yna n To this end. =⇒ g (−1) N (0. so that g (Xn ) − g (b) d g (an Yn + b) − g (b) an Yn ε (an Yn ) = = g (b) Yn + an an an = g (b) Yn + Yn ε (an Yn ) → g (b) Y a.s. √ n j =1 n d Xn −b an and Y = Z with Yn → Y a. Suppose that {Un }n=1 are i.26. f : X → R and a point x ∈ X.27 Given a proper distribution function.d. e−2 . 1] and let Yn := j =1 Ujn . lim ε↓0 y ∈Bx (δ ) inf f (y ) = lim inf f (y ) ≥ f (x) > a. so that Var (ln U1 ) = 2 − (−1) = 1. let n 1 ln Uj .4 Skorohod and the Convergence of Types Theorems n (Xn + 1) =⇒ N (0. 1 E ln2 U1 = 0 2 ln2 xdx = 2 it follows that inf y∈Bx (δ) f (y ) > a for some δ > 0.cls date/time: 23-Feb-2007/15:20 . = Yn . 1 n d Exercise 13. there exists δ > 0 such that Bx (δ ) ⊂ {f > a} . let lim inf f (y ) := lim y →x y →x ε↓0 y ∈Bx (δ ) ε↓0 y ∈B (δ ) x inf f (y ) and (13. (13. random variables which are uniformly distributed on [0. 1) .s. Suppose Eq. 1 n 1 P Yn − e 1 √ n −1 =⇒ e−1 N (0. Given x ∈ X and a < f (x) .

So in Page: 150 job: prob ∞ Deﬁnition 13.1) . (13.e. B(0. where E is the countable null set deﬁned as above.1) .11) lim sup Yn (x) ≤ Y (x) . (Ω. Y (x) ≤ Y + (x) and Y (x) < Y + (x) iﬀ x is the height of a “ﬂat spot” of F.) 3. then U ← (γ1 ) < U ← (γ2 ) for all γ1 suﬃciently close to 0 and γ2 suﬃciently close to 1. As a consequence of item 4. Then there ex∞ ists a probability space. B = B(0. We will take Ω := (0. Y and Z. Because of the above comments. and P = m – Lebesgue measure ← ← on Ω and let Yn := Fn and Y := F0 as in Notation 13. Observe that Y is constant iﬀ U (y ) = 1y≥c for some c ∈ R. F (Y (x) −) ≤ x ≤ F (Y (x)) for all x ∈ (0. 1. 1} .29. 1) . x ∈ (0. Conversely. 1) . This is because. U (y ) = P (Y ≤ y ) = P (Z ≤ Ay + B ) = V (Ay + B ) . B . 1) : Y (x) < Y + (x)} . A > 0 and B ∈ R such that Z = AY + B. we have limn→∞ Fn (y ) = F0 (y ) > x and in particular. n→∞ Similarly. where m is Lebesgue measure on (0. Fn (y ) < x for almost all n. E := {x ∈ (0. 1) . are said to be of the same type if there exists constants. This inequality then remains valid as γ1 decreases and γ2 increases.a.cls date/time: 23-Feb-2007/15:20 . We now suppose x ∈ / E. Moreover. Yn (x) → Y (x) for all x ∈ / E. This implies that Yn (x) ≥ y for a.12) Indeed.13) Alternatively put. 5. iﬀ U only takes on the values. F (y0 )] ∩ (0. If y ∈ C (F0 ) with y < Y (x) . 2. {Yn }n=1 such that ← P (Yn ≤ y ) = Fn (y ) for all n ∈ N∪ {∞} and limn→∞ Fn = limn→∞ Yn = Y = F ← a. The set. For the next theorem we will need the following elementary observation. for x ∈ / E and y ∈ C (F0 ) with Y (x) = Y + (x) < y. if U (y ) := P (Y ≤ y ) and V (y ) := P (Z ≤ y ) . (13.28 (Baby Skorohod Theorem).s. we see that Y is B(0. Suppose that {Fn }n=0 is a collection of distribution functions such that Fn =⇒ F0 . P (Yn ≤ y ) = Fn (y ) and P (Y ≤ y ) = F0 (y ) for all y ∈ R. we have limn→∞ Fn (y ) = F0 (y ) < x and in particular. E := {x ∈ (0. then U and V should satisfy. Hence if γ2 > U (y ) then U ← (γ2 ) ≥ y and if γ1 < U (y ) then U ← (γ1 ) ≤ y. for x ∈ / E. Theorem 13. Proof. d (13. that lim sup Yn (x) ≤ Y (x) ≤ lim inf Yn (x) n→∞ n→∞ which shows n→∞ ← lim Fn (x) = lim Yn (x) = Y (x) = F ← (x) for all x ∈ / E. 1) . Y + (x))}x∈E is a collection of pairwise disjoint intervals which is necessarily countable. n→∞ (13. if we suppose that γ1 is not the height of a ﬂat spot of U. then Y (x) ≤ y0 by deﬁnition of Y. 1) . To prove this assertion ﬁrst suppose that Y (x) ≤ y0 . i.27. 1) and x ≤ F (y0 ) . 1) : Y (x) ≤ y0 } = (0. then F (y ) < x and hence F (Y (x) −) ≤ x. Two random variables. n and hence that lim inf n→∞ Yn (x) ≥ y. {0. Proof. If Y is non-constant (a.s. Lemma 13. there exists y ∈ R such that 0 < U (y ) < 1. then according to Eq.4. 1) : Y (x) < Y + (x)} .1) /BR – measurable and m ◦ Y −1 = F. Letting y ↓ Y (x) with y ∈ C (F0 ) then implies We will need the following simple observations about Y and Y + which are easily understood from Figure 13. then F (y ) ≥ x and by right continuity of F it follows that F (Y (x)) ≥ x. {(Y (x) . U ← (γ1 ) < U ← (γ2 ) . macro: svmonob. 4.e.30. {x ∈ (0. (Each such interval contains a rational number. Letting y ↑ Y (x) with y ∈ C (F0 ) then implies lim inf Yn (x) ≥ Y (x) . P ) and random variables. Fn (y ) > x for almost all n. then in fact. of ﬂat spot heights is at most countable.a. Similarly. n→∞ Hence we have shown. So since Y is not constant. F (y0 )] ∩ (0.150 13 Weak Convergence Results order to ﬁnish the proof it suﬃces to show. i. if y > Y (x) . This implies that Yn (x) ≤ y for a. if x ∈ (0. if y < Y (x) . The following inequality holds.11) we have x ≤ F (Y (x)) ≤ F (y0 ) . n and hence that lim supn→∞ Yn (x) ≤ y.) random variable and U (y ) := P (Y ≤ y ) .

If Eq. of (0. then Eq. 2. αn Taking ratios of the last two displayed equations shows. sup {y : Fn (αn y + βn ) < x} = ← (x) − bn Fn = sup {y : Fn (an y + bn ) < x} → U ← (x) and an ← (x) − βn Fn = sup {y : Fn (αn y + βn ) < x} → V ← (x) αn (13. if Eq. by Lemma 13. ∞) . γ1 < γ2 not in Λ such that U ← (γ1 ) < U ← (γ2 ) and V ← (γ1 ) < V ← (γ2 ) . αn αn βn − bn A = lim ∈ (0. Suppose is a sequence of random variables and an .19) Proof. ← ← ← αn := Fn (γ2 ) − Fn (γ1 ) and βn := Fn (γ1 ) ← Fn (x) − βn . (13. 1) such that.16) hold. an αn an an (1) If Fn (y ) := P (Xn ≤ y ) . If the relations in Eq.13. then Xn − βn αn βn − bn X n − bn = + =⇒ AZ + B =: Y.28 (see Eq.cls date/time: 23-Feb-2007/15:20 . (13. (13.14) holds. αn With these identities.30. By assumption we have Fn (an y + bn ) =⇒ U (y ) and Fn (αn y + βn ) =⇒ V (y ) . (13.12)) that there exists an at most countable subset. an V (γ2 ) − V ← (γ1 ) Moreover.14) or (13. βn ∈ R are constants and Y and Z are non-constant random variables.14) is satisﬁed. Xn − bn + bn − βn an Xn − βn = αn an αn Xn − bn an βn − bn an = − an αn an αn −1 =⇒ A (Y − B ) =: Z Similarly. (13.17) for some 0 < γ1 < γ2 < 1. ∞) and B := lim n→∞ an n→∞ an d ∞ {Xn }n=1 P X n − bn ≤y an = Fn (an y + bn ) and P Xn − βn ≤y αn = Fn (αn y + βn ) . exists and Y = AZ + B.. then by Slutsky’s Theorem 13. Eq. U ← (γ2 ) − U ← (γ1 ) αn → A := ← ∈ (0.18) for all x ∈ / Λ. Moreover. (2) Assume the limits in Eq. (13.20. (13. Then 1. Since Y and Z are not constants a. it now follows from the proof of Skorohod’s Theorem 13.14) ← If w := sup {y : Fn (an y + bn ) < x} .31 (Convergence of Types).17) may be written as Fn (βn ) = γ1 and Fn (αn + βn ) = γ2 . ∞) .16) sup {y : Fn (an y + bn ) < x} = ← Fn (x) − bn .20) Page: 151 job: prob macro: svmonob. ← Fn (γ1 ) − bn → U ← (γ1 ) and an ← Fn (γ1 ) − βn F ← (γ1 ) − βn αn = n → AV ← (γ1 ) an αn an (13. (13. If there are some constants. (13. αn ∈ (0.15) implies the others with Z and Y related by Eq.s.15) holds using αn and βn of the form. (13. if X n − bn =⇒ Y an and Xn − βn =⇒ Z. we can choose. then an w + bn = Fn (x) and hence (13. (13. 3. In particular it follows that ← ← F ← (γ2 ) − bn F ← (γ1 ) − bn Fn (γ2 ) − Fn (γ1 ) = n − n an an an ← ← → U (γ2 ) − U (γ1 ) > 0 (13.15) Similarly. Λ. bn .15) is satisﬁed. an > 0 and bn ∈ R and a non-constant random variable Y. such that Eq. then and similarly ← ← Fn (γ2 ) − Fn (γ1 ) → V ← (γ2 ) − V ← (γ1 ) > 0. the limits. If the Fn are invertible functions.13).16) hold then either of the convergences in Eqs. (13. (13. an then Y and Z are of the same type.4 Skorohod and the Convergence of Types Theorems 151 Theorem 13.

19) and (13. (13.n+1) . let Xp denote the number of trials to get success in a sequence of independent trials with success probability p.f. then P (Xp = n) = (1 − p) p = q n−1 p for n ∈ N. 1) . To this end we note that P Mn − b n ≤x an = P (Mn ≤ an x + bn ) = Fn (an x + bn ) = [F (an x + bn )] . Hence we may as well take an to be constant and for simplicity we take an = 1. an Thus we may always center and scale the {Xn } using αn and βn of the form described in Eq. Method 1.5 Weak Convergence Examples Example 13.e. i. . Eq.20)we have From this it follows that bn ∼ λ−1 ln n.152 13 Weak Convergence Results and therefore. 13. this requires an + bn ∼ λ−1 ln n. ∞) that Fn (x) := P (Mn ≤ x) = P ∩n j =1 {Xj ≤ x} n ∞ Notice that F (x) is a distribution function for some random variable. Then n P (Xp > n) = (1 − p) and therefore for x > 0. i. In this case F (x) := P (X1 ≤ x) = 1 − e−λ(x∨0) Consider Mn := max (X1 . we now try to ﬁnd an by requiring.e. (13. Let us see in a couple of ways where the appropriate centering and scaling of the Xp come from in this example. for x ≥ 0 and cn ∈ (0. P (T > x) = e−x for x ≥ 0 or alternatively. Y. macro: svmonob. P Mn − b n ≤1 an = Fn (an + bn ) = [F (an + bn )] → γ2 ∈ (0. 1) . (13. We now compute n→∞ lim P Mn − λ−1 ln n ≤ x = lim = lim n→∞ 1 − e−λ(x+λ 1− e−λx n −1 ln n) n n n→∞ = exp −e−λx . Suppose that {Xn }n=1 are i. . P (T ≤ y ) = 1 − e−y∨0 . Xn ≥ 0 a. =⇒ Y. P (pXp > x) = P Xp > x = j =1 P (Xj ≤ x) = [F (x)] = 1 − e−λx Mn −bn an n n .32. 2.cls date/time: 23-Feb-2007/15:20 .33. . We have. n αn /an → U ← (γ2 ) − U ← (γ1 ) ∈ (0.s. exp (λ) – random variables.17). As above. 1) . and P (Xn ≥ x) = e−λx for all x ≥ 0. Page: 152 job: prob where [x] := n=1 n · 1[n. For n−1 this let q = 1 − p. then according to Eqs. an an an ← Fn ← ← (γ1 ) and βn := (γ2 ) − Fn (3) Now suppose that we deﬁne αn := Fn (γ1 ) . Therefore pXp =⇒ T where T = exp (1) . βn − bn F ← (γ1 ) − βn F ← (γ1 ) − bn = n − n → AV ← (γ1 ) − U ← (γ1 ) := B. For p ∈ (0. 1) and βn − bn → U ← (γ1 ) as n → ∞.i. Our goal is to choose ap > 0 and bp ∈ R such that limp ↓0 Fp (ap x + bp ) exists. Given this.d. Remarks on this example. Xn ) . . by what we have done above. we ﬁrst demand (taking x = 0) that p ↓0 ∞ lim Fp (bp ) = γ1 ∈ (0. If we demand (c. Example 13. 1) . and therefore we have shown Mn − 1 ln n =⇒ Y as n → ∞ λ where P (Y ≤ x) = exp −e−λx .18) above) P Mn − bn ≤0 an = Fn (bn ) = [F (bn )] → γ1 ∈ (0. n then bn → ∞ and we ﬁnd ln γ1 ∼ n ln F (bn ) = n ln 1 − e−λbn ∼ −ne−λbn . However. x p x x = (1 − p)[ p ] = e[ p ] ln(1−p) ∼ e−p[ p ] → e−x as p → 0. Also let Fp (x) = P (Xp ≤ x) = P (Xp ≤ [x]) = 1 − q [x] n d We now wish to ﬁnd an > 0 and bn ∈ R such that 1.

reﬂecting the fact that P (Xp ∈ N) = 1.5 Weak Convergence Examples 153 Since. From this (setting x = 1) we see that pap ∼ c > 0. c ∼ bp ln q = bp ln (1 − p) ∼ −bp p. q bp ∼ 1 − γ1 and hence. Then √ =⇒ N (0. Theorem 13. Hence we might take ap = 1/p as well.cls date/time: 23-Feb-2007/15:20 ∞ f (z ) := E z Xp = n=1 z n q n−1 p = pz . 2q 1 + − p2 p 1 p 2 Hence we have shown.. a success must occur almost surely. i. 1 − qz Observe that f (z ) is well deﬁned for |z | < 1 q and that f (1) = 1. f (z ) = E Xp (Xp − 1) z Xp −k Xp −2 . Observe that |Sn | is an odd integer if n is odd and an even Sm integer if n is even. . 2 p p p2 lim Fp (ap x + bp ) = [1 − exp (− (x + 1))] 1x≥−1 Xp − 1/p = pXp − 1 =⇒ T − 1 1/p Thus. macro: svmonob. and for x > −1 we ﬁnd p−1 (x+1)] (1 − p)[ = exp p−1 (x + 1) ln (1 − p) → exp (− (x + 1)) . (1 − q ) pq Therefore.) We start by observing that S2n = 2k iﬀ # {i ≤ 2n : Xi = 1} = n + k while # {i ≤ 2n : Xi = −1} = 2n − (n + k ) = n − k and therefore. (Sketch of the proof. (z ) = E Xp (Xp − 1) . E [Xp (Xp − 1) . (Center and scale using the ﬁrst moment and the variance of Xp . We then have p−1 (x+1)] Fp (ap x + bp ) = Fp p−1 x + p−1 = 1 − (1 − p)[ ap x+bp ap x+bp d pz p (1 − qz ) + qpz p = = 2 2 dz 1 − qz (1 − qz ) (1 − qz ) d2 pz pq =2 3 dz 2 1 − qz (1 − qz ) p 2 and =q ap x+bp ∼ 1 − F0 (x) it follows that (1 − q ) 2q E [Xp (Xp − 1)] = 2 3 = p2 . we would like to choose ap such that F0 (x) := lim Fp (ap x + bp ) exists. . Method 2. . (Xp − k + 1) z Page: 153 job: prob . .d. Having done this. p ↓0 = q 1−p 2q + p − 1 = 2 = . m Proof. (Xp − k + 1)] = f (k) (1) = Since d dz k |z=1 pz . Moreover..) The generating function is given by ∞ instead.i.. γ1 ∼ Fp (bp ) ∼ 1 − q bp we require. random variables such that P (Xn = ±1) = 1/2 and let Sn := X1 + · · · + Xn – the position of a drunk after n steps. Let {Xn }n=1 be i. This suggests that we take bp = 1/p say. 2 2 σp = Var (Xp ) = EXp − (EXp ) = 2 µp := EXp = = 1 and p which is equal to 0 if x ≤ −1. Xp − √ 1−p p 1 p pXp − 1 =⇒ T − 1 = √ 1−p or again that pXp =⇒ T.e.34. 1 − qz Since. p ↓0 and in particular.13. we have f (z ) = E Xp z f (k ) Xp −1 . F0 (x) ∼ Fp (ap x + bp ) ∼ 1 − q this requires that (1 − p) and hence that ln (1 − F0 (x)) = (ap x + bp ) ln q ∼ (ap x + bp ) (−p) = −pap x − 1. if we had used µp and σp to center and scale Xp we would have considered. 1) as m → ∞.

that n n P U(k. So let us go another route. Proposition 8.n) ∈ (x. 1) k2 n2 −k−1/2 · 1+ k n .n) ∈ / (x. 1) . Since 1 1+ 1 2n (2n) n π (n + k ) (n − k ) 1 1+ 1− k n 2n −2n √ e 4πn n−k −(n−k) k) e 2π (n + k ) · (n − 1+ k n −(n+k) 2π (n − k ) 1 2 2n S S n + X2n+1 S2 n √ 2n+1 = 2√ =√ 2n + 1 2n + 1 2n X2n+1 +√ .n) . U2 . d From this it follows that ρn (x) := 1(0. (13. ∞ =⇒ N (0. √ n! ∼ nn e−n 2πn as n → ∞ and therefore.1 for more details. 1) . 1)) → 0 as n → ∞.cls date/time: 23-Feb-2007/15:20 Page: 154 job: prob .d.n) − k (n) /n √ . Suppose that {Un }n=1 are i. x + ∆]. ±1. for x ∈ (0. x + ∆]. x + ∆]. =⇒ N (0. i. then there must be at least one Ui ∈ (x. πn wherein we have repeatedly used (1 + an ) We now compute P S2 n a≤ √ ≤b 2n = a≤x≤b bn x n/2−1/2 Then dT V (Xn . Let U(k. for ∆ ∈ (0. See Resnick. . for otherwise.n) denote the position of the k th – largest number from the list. {U1 . N (0.n) ≤ x as well and hence U(k.35. Recall Stirling’s formula states. . P U(k. 1) . x + ∆] . l = ebn ln(1+an ) ∼ ebn an when an → 0.n) ∈ (x.21) is the Riemann sum approximation to 1 √ 2π This proves S2n √ 2n b a e−x 2 /2 dx.) Observe that. Observe that if U(k.i.1) (x) dx P U(k. Further let k (n) be chosen so n) that limn→∞ k (n) = ∞ while limn→∞ k( n = 0 and let Xn := √ U(k(n). we see the latter expression in 2n Eq. P (S2n = 2k ) ∼ = (n + n+k −(n+k) k) e 2k with k ∈ {0. . ±n} .n) ≤ x is the probability density for U(k. x = √ 2n 2 √ is the increment of x as k increases by 1 . Un } .154 13 Weak Convergence Results P (S2n = 2k ) = 2n n+k 1 2 2n = (2n)! (n + k )! · (n − k )! 1 2 2n . Let macro: svmonob. √x . .n) ≤ x + ∆ would imply U(k. 1) . (Sketch only. ρn (x) = P S √2n = x 2n e−x a≤x≤b 2 n n−k k · xk−1 (1 − x) .n) ≤ x = P i=1 Xi ≥ k = l=k n l n−l x (1 − x) . It now turns out that ρn (x) is a Beta distribution. . random variables which are uniformly distributed in (0. k 1 =√ 2π /2 2 √ 2n (13. . 2n + 1 S √ 2n+1 2n+1 · 1− k2 n2 −n k n k n −(n−k) 1 =√ πn 1 =√ πn −k 1− −n k n 1− k n · 1+ · 1− · 1− k−1/2 k n k it follows directly (or see Slutsky’s Theorem 13. To do this we are going to estimate.21) Giving a direct computation of this result is not so illuminating. 2n √ So if we let x := 2k/ 2n.e. Proof. k = x n/2 and k/n = P S √2n = x 2n Proposition 13. U(k.2. .20) that as well. Since where the sum is over x of the form. . k ( n) n we have √ −n −x n/2−1/2 1 x2 x x ∼√ 1− · 1+ √ · 1− √ 2n πn 2n 2n √ √ 1 x2 /2 √x √x n/ 2 − 1 / 2 − n/ 2 − 1 / 2 − x x ∼√ e · e 2n · e 2n πn 2 1 ∼ √ e−x /2 .

13. n x = − k (n) at u = 0. j ≤ n) ≤ i<j ≤n we arrive at √ n n! ∼√ (k − 1)! · (n − k )! 2π 1 k (k−1/2) n k (n−k+1/2) n . x] and n − k of these in [x + ∆.5 Weak Convergence Examples 155 Ωi := {Ui ∈ (x.n) ∈ (x. Ω1 + O ∆2 .n) n − 1 k−1 n−k ∈ (x. P U(k. Ωi + O ∆2 on noting the du = x= √ = nP U(k. n! (k − 1)! · (n − k )! (k − 1) √ −1 ne = √ 2π √ −1 ne = √ 2π Since k−1 n (k−1/2) 1 E [F (Xn )] = 0 u − k ( n ) /n du √ ρn (u) F k ( n) n bn = √ nn e−n 2πn 2π (k − 1) (n − k ) 1 k−1 n n−k (n−k) n (n−k) −(n−k) e − √ k ( n) k (n) ρn n k (n) x + k (n) /n F (x) du. x + ∆] = n x (1 − (x + ∆)) ∆ + O ∆2 k−1 k ( n) dx.3 completes the proof. it is then shown in Resnick that 2π (n − k ) k (n) ρn n k (n) x + k (n) /n n e−x /2 → √ 2π 2 (k−1) −(k−1) e k−1 (k−1) n n−k n 1 k−1 (k−1/2) n which upon an application of Scheﬀ´ e’s Lemma 13. x + ∆]) n2 − n 2 ∆ . P (Ui . This leads to the conclusion that P U(k. 1] . After some computations (see Chapter ??). one arrives at 1− = = k n k n (k−1/2) · (k−1/2) k−1 k 1 k (k−1/2) (k−1/2) · 1− k n (k−1/2) ∼ e−1 Page: 155 job: prob macro: svmonob.cls date/time: 23-Feb-2007/15:20 . Un in [0.n) ∈ (x. with x= u − k (n) /n √ k ( n) n ≤ we see that n P U(k. x + ∆] for j = i} . x + ∆] n! n−k = xk−1 (1 − x) . ρn (x) = lim ∆↓0 ∆ (k − 1)! · (n − k )! By Stirling’s formula. It is possible to understand the normalization constants in the deﬁnition of Xn by computing the mean and the variance of U(n. U(k. and 1 − k (n) /n n − k (n) √ = k ( n) k (n) n = n k (n) 1− k (n) n = √ n n k (n) 1− k (n) n =: bn . Now on the set. x + ∆]. x + ∆] iﬀ there are exactly k − 1 of U2 . .36. . x + ∆] and Uj ∈ / (x. n ∼ Using this information. . and therefore.n) ∈ (x. x + ∆] for some i = j with i. Remark 13.n) ∈ (x. x + ∆] = i=1 P U(k. Since P (Ui . x + ∆].n) ∈ (x. Ω1 .k) . . 2 1− By the change of variables formula. Uj ∈ (x. Uj ∈ (x. k (n−k+1/2) n .

x]) = µn ((−∞. BR probability measures on R ¯ . Hence by Cantor’s diagonalization macro: svmonob. (13. µ0 on R ¯ . let x ∈ R and λ ∈ Λ such that λ > x. F (x) ≤ F (y ) = G+ (y ) ≤ G (λ) and therefore. If F = F ˜ (λ) = F ˜ (x) . λ) .156 13 Weak Convergence Results EU(k. As usual.e. 1] is compact. Using this identiﬁcation. Suppose that {Fn }n=1 is a sequence of distribution functions and Λ ⊂ R is a dense set such that G (λ) := limn→∞ Fn (λ) ∈ [0. for each x ∈ R we may ﬁnd ∞ a convergence subsequence of {Fn (x)}n=1 . µn ([−1. Since {Fn (x)}n=1 ⊂ [0. 1]) . since F (y −) = F (y ) for y ∈ C (F ) .6 Compactness and Tightness ˜ are two right continuous Suppose that Λ ⊂ R is a dense set and F and F ˜ on Λ. 1] . Proof. Every sequence of probabil∞ ¯ . Proof. (Note well. 1] . i. then F = F ˜ on R. 1] and [0. we may view them as probability measures on (R. On the other hand. the reader should identify R [−1.22). x] ∩ [−1. F (x) ≤ F (x+) ≤ G+ (x) = F (x) . n→∞ n→∞ k k k+1 − n+1 n+2 n+1 k n−k+1 k = ∼ 2. If. ∞) and then letting x ∈ R tend up to y.n) = = 2 EU( k. n + 1 (n + 2) (n + 1) n Taking the inﬁnum over t ∈ Λ ∩ (y. BR ) . Using the identiﬁcation described above. 1] exists Page: 156 job: prob ∞ . it is possible that F (∞) < 1 and F (−∞) > 0 so that F need not be a distribution function for a measure on (R.e. F : R always a point of continuity. If G : Λ → R is a non-decreasing function. To show F is right continuous. 1]) = 1.n) = = n! n−k xk−1 (1 − x) xdx 0 (k − 1)! · (n − k )! k k ∼ . we may conclude. BR The next theorem deals with weak convergence of measures on R ¯ .39 (Helly’s Selection Theorem). y ∈ R with x < y and and s. BR Hence a probability measure on R ¯ may be identiﬁed with a probability measure on (R. then Fn (x) → F (x) for all x ∈ C (F ) . Fn (s) ≤ Fn (y ) ≤ Fn (t) implies F (x) = G+ (x) ≤ G (s) ≤ lim inf Fn (y ) ≤ lim sup Fn (y ) ≤ G (t) .22) ¯ . So ¯ with as not have to introduce any new machinery. on R ¯ has a sub-sequence which is weakly conver¯ . BR ) which are supported on [−1. Suppose that x. 1] x → tan π ¯. n→∞ n→∞ 13. Then for any y ∈ (x. i.) Proof. for x ∈ R we have functions. as we have already seen. then F (x) := G+ (x) := inf {G (λ) : x < λ ∈ Λ} is a non-decreasing right continuous function. F (x) = lim F (λ) = lim F Λ λ↓x Λ λ↓x This completes the proof. F (x+) = F (x) . Theorem 13. n+1 n 1 n! n−k 2 xk−1 (1 − x) x dx ( k − 1)! · ( n − k )! 0 (k + 1) k and (n + 2) (n + 1) (k + 1) k k2 − (n + 2) (n + 1) (n + 1)2 1 for all λ ∈ Λ. [−1. BR ) which is supported on [−1. we deﬁne F = G+ as in Eq. BR gent to a probability measure.38.n) = = Var U(k. x ∈R 2 Lemma 13. Proposition 13. {µn }n=1 . BR ity measures.37. y ↓x (13. we see that a −∞ should only be considered a point of continuity of a distribution ¯ → [0. t ∈ Λ are chosen so that x < s < y < t. rather than viewing µn as ¯ . F (x) ≤ F (x+) := lim F (y ) ≤ G (λ) . for all x ∈ R.cls date/time: 23-Feb-2007/15:20 ∞ Since λ > x with λ ∈ Λ is arbitrary. Indeed. ¯ . ∞ is function. 1] ⊂ R via the map. Then passing to the limit in the inequality. we may conclude F (y −) ≤ lim inf Fn (y ) ≤ lim sup Fn (y ) ≤ F (y ) for all y ∈ R. 1] iﬀ and only if F (−∞) = 0. let Fn (x) := µn ((−∞.

Proposition 13. Mε ]) ≥ 1 − ε. it follows from Lemma 13. BR (R.24) Observe that the deﬁnition of uniform integrability (see Deﬁnition 11. It is also worth observing α that if α > 0 and C := supλ∈Λ E |Xλ | < ∞.e. Equivalently put. (13. BR ) is tight iﬀ for every ε > 0 there exists Mε < ∞ such that ∞ Fnk }k=1 ∞ {Fn }n=1 we may ﬁnd Mε < ∞ such that Mε . The following notion of tightness is the key to answering this question. there is a weakly convergent subsequence of Γ converging to a probability measure on (R. BR R ¯ .44. A sequence of probability measures ∞ {Pn }n=1 is said to converge weakly to a probability P if limn→∞ Pn (f ) = P (f ) for all for every f ∈ BC (X ). 2. Page: 157 job: prob ∞ 4. µ0 ([−M.22).7 Weak Convergence in Metric Spaces (This section may be skipped. 3. {Xλ : λ ∈ Λ} is tight iﬀ M →∞ λ∈Λ 13. on R ¯ is supported on R. ∞).) Deﬁnition 13. As usual. 1 C α sup P (|Xλ | ≥ M ) ≤ sup E |Xλ | ≤ α → 0 as M → ∞ α M M λ λ and therefore {Xλ : λ ∈ Λ} is tight. P ◦ Xλ : λ ∈ Λ is tight. M ]) = µ0 ([−M.7 Weak Convergence in Metric Spaces 157 argument we may ﬁnd a subsequence. BR ) . Γ. Deﬁnition 13. −Mε ∈ C (F0 ) and µn ([−Mε .42. Mε ]) ≥ 1 − ε k→∞ and by letting ε ↓ 0 we conclude that µ0 (R) = limε↓0 µ0 ([−Mε . Pn (f ) → P (f ) for every f ∈ BC (X ) which is uniformly continuous. In this case ε0 := µ0 ({−∞. M ])) ≤ 1 − ε0 for all M < ∞ µ∈Γ inf µ ([−Mε . The next question we would like to address is when is the limiting measure. Pn (f ) → P (f ) for all f ∈∈ BC (X ). {Xλ : λ ∈ Λ} is tight −1 iﬀ the collection probability measures. (13. Let X be a metric space. Mε ]) = 1.13. limn→∞ Pn (A) = P (A) for all A ∈ B such that P (bd(A)) = 0. ¯ . −1) and Gk (x) = 1 for all x ∈ Q ∩ [1. Mε ]) = lim µnk ([−Mε . For simplicity we will now assume that X is a complete metric space throughout this section. µ0 is supported on [−1. let F0 (x) := µ0 ([−∞. (13. Theorem 13. Hence µ0 may now be transferred back to a measure ¯ .40. Hence it follows that µ0 ([−Mε . BR with µ0 being a probability measure on R ¯ such that µ0 (R) < 1. since Gk (x) = 0 for all x ∈ Q∩ (−∞. suppose there is a subsequence {µnk }k=1 such that µnk =⇒ µ0 ¯ . ∞}) = 1 − ε0 . Let Γ := {µn }n=1 be a sequence of probability measures on ¯ . F0 (x) = 1 for all x ≥ 1 and F0 (x) = 0 for all x < −1 and the corresponding measure. Then Γ is tight.cls date/time: 23-Feb-2007/15:20 . 1] .41. A collection of probability measures. k→∞ Therefore. Suppose δ−n =⇒ δ−∞ and δn =⇒ δ∞ and 1 ⇒ 2 (δn + δ−n ) = 1 ( δ + δ ) . {Gk := of the such that G (x) := limk→∞ Gk (x) exists for all x ∈ Λ := Q. BR µ0 on R ¯ concentrated on R. The following are equivalent: 1. µ0 . BR on R ¯ . it then follows that lim µnk ([−M. macro: svmonob. Suppose that µnk =⇒ µ0 with µ0 being a probability measure on ¯ . We further say that a collection of random variables. Mε ]) ≥ 1 − ε for all n. Example 13. M ]) ≤ µ0 R By choosing M so that −M and M are points of continuity of F0 . i. iﬀ every subsequently limit measure.38 that Gk = Fnk =⇒ F0 . This shows that probability may indeed transfer to the points ∞ −∞ 2 at ±∞. Letting F (x) := G (x+) as in Eq. This is actually weak-* convergence when viewing Pn ∈ BC (X )∗ . ∞ Conversely. If Γ is tight and ε > 0 is given. lim sup Pn (F ) ≤ P (F ) for all F X. BR ) . lim inf n→∞ Pn (G) ≥ P (G) for all G ⊂o X.23) and {µn }n=1 is not tight. Proof. on (R.25) is considerably stronger than the notion of tightness. 5. M ]) ≤ 1 − ε0 . Pn → P as n → ∞. n→∞ w lim sup P (|Xλ | ≥ M ) = 0. x]) . Therefore. then by Chebyschev’s inequality. n∈N ∞ inf µn (([−M. In particular if Γ is tight. Moreover.37 and Proposition 13.43. ∞}) > 0 and hence for all M < ∞ we have ¯ − µ0 ({−∞.

F ) < δ } = Aδ i−1 1 [P (Fi−1 ) − P (Fi )] + [P (Fi−1 ) − P (Fi )] k k i=1 P (Fi ) + 1 . then P (Fδm ) = lim Pn (Fδm ) ≥ lim sup Pn (F ).26) (13. 3. F ) = δ } . 1. 3. =⇒ 2. =⇒ 3.. To ﬁnish the proof we will now show 3.27) Since (i − 1) [P (Fi−1 ) − P (Fi )] k i=1 k k and then letting m → ∞ in this inequality implies item 3. By an aﬃne change of variables it suﬃces to consider f ∈ C (X. Passing to the limit n → ∞ in the equation 0 ≤ Pn (F ) ≤ Pn (fm ) gives 0 ≤ lim sup Pn (F ) ≤ P (fm ) n→∞ i=1 (i − 1) [P (Fi−1 ) − P (Fi )] ≤ P (f ) ≤ k i=1 i [P (Fi−1 ) − P (Fi )] . then 1 − lim inf Pn (G) = lim sup(1 − Pn (G)) = lim sup Pn (Gc ) n→∞ n→∞ c n→∞ = i=1 k−1 (i − 1) (i − 1) P (Fi−1 ) − P (Fi ) k k i=1 i i−1 1 P (Fi ) − P (Fi ) = k k k i=1 k k−1 k ≤ P (G ) = 1 − P (G) ¯ \ Ao . Then for any probability P. let F = Gc .27) becomes. i=1 date/time: 23-Feb-2007/15:20 . (0.cls ≤ 1 k P (Fi ) + 1/k ≤ P (f ) + 1/k. i=1 and in particular the set Λ := {δ > 0 : P (Aδ ) > 0} is at most countable. Let δn ∈ / Λ be chosen so that δn ↓ 0 as n → ∞. For 2. Similarly 4. (13. ⇐⇒ 5. n→∞ n→∞ Using this equation with P = Pn and then with P = P we ﬁnd lim sup Pn (f ) ≤ lim sup n→∞ n→∞ k−1 1 k k−1 Pn (Fi ) + 1/k i=1 Let m → ∞ in this equation to conclude P (F ) ≥ lim supn→∞ Pn (F ) as desired.. which implies 4. Since {Aδ }δ>0 are all disjoint. Conversely. k and let fn (x) := ϕ(nd(x. Assuming item 3. we must have P (Aδ ) ≤ P (X ) ≤ 1 δ>0 Eq. 0 ≤ 1F ≤ fn for all n and fn ↓ 1F as n → ∞. 1)) in which case we have Page: 158 job: prob macro: svmonob. let F set Fδ := {x ∈ X : ρ(x. 1 k k−1 P (Fi ) ≤ P (f ) ≤ i=1 1 k k−1 P (Fi ) + 1/k. k { k ≤f < k } (13. =⇒ 1. Recall that bd(A) = A so if P (bd(A)) = 0 and 3. Then fn ∈ BC (X.158 13 Weak Convergence Results Proof. is obvious. k k where Aδ := {x ∈ X : ρ(x. holds) we have ¯) ≤ P (A ¯) = P (A) and lim sup Pn (A) ≤ lim sup Pn (A n→∞ n→∞ = i=1 P (Fi ) i=1 and i [P (Fi−1 ) − P (Fi )] k i=1 k k lim inf Pn (A) ≥ lim inf Pn (Ao ) ≥ P (Ao ) = P (A) n→∞ n→∞ = X and = i=1 i=1 k−1 from which it follows that limn→∞ Pn (A) = P (A).25) Let Fi := k i k ≤f and notice that Fk = ∅. [0. F )). let 1 if t ≤ 0 ϕ(t) := 1 − t if 0 ≤ t ≤ 1 0 if t ≥ 1 k i=1 (i − 1) 1{ (i−1) ≤f < i } ≤ f ≤ k k k k i=1 i 1 (i−1) i . Then bd(Fδ ) ⊂ Fδ \ {x ∈ X : ρ(x. (and hence also 4. k (13. ⇐⇒ 4. F ) ≤ δ } . =⇒ 3. 1]) is uniformly continuous.

45 (Skorohod Theorem). ¯ (Kε ) ≥ lim sup P ¯ (Kε ) = 1 − ε. Let (X. Letting f ∈ Λ tend to g in C (X ) shows lim supn→∞ |Pnk (g ) − Pnl (g )| = 0 and hence Λ(g ) := limk→∞ Pnk (g ) for all g ∈ C (X ). In this case C (X ) is a Banach space which is separable by the Stone – Weirstrass theorem. see Theo∞ rem ??. It is now clear that Λ(g ) ≥ 0 for all g ≥ 0 so that Λ is a positive linear functional on X and thus there is a probability measure P such that Λ(g ) = P (g ). BX ). we may view P as a measure on B by letting P (A) := Because X0 ∈ BX ∩ BX ¯ X ¯ (A ∩ X0 ) for all A ∈ BX . Then there exists a probability space. choose F ˜ X ¯ such P ˜ ∩ X. Theorem 13. We have also seen that C (X )∗ is metrizable and the unit ball in C (X )∗ is weak . ¯ := P ¯n ∞ such that P ¯ converges weakly to a there is a subsequence P k k =1 k k ¯ ¯ probability measure P on X. Proof. (Ω. General case.46.34 we may assume that X is a subset of ¯ We now extend Pn to X ¯ a compact metric space which we will denote by X. Then that F = F ¯ (F ˜) ≤ P ¯ (F ˜) = P ¯ (F ˜ ∩ X0 ) = P (F ). we have |Pnk (g ) − Pnl (g )| ≤ |Pnk (g ) − Pnk (f )| + |Pnk (f ) − Pnl (f )| + |Pnl (f ) − Pnl (g )| ≤ 2 g − f ∞ + |Pnk (f ) − Pnl (f )| which shows lim sup |Pnk (g ) − Pnl (g )| ≤ 2 g − f n→∞ ∞ ∞ ¯n (A) := P ¯n (A ∩ X ) for all A ∈ BX by setting P ¯ . A collection of probability measures Λ on (X. BX ) is said to be tight if for every ε > 0 there exists a compact set Kε ∈ BX such that P (Kε ) ≥ 1 − ε for all P ∈ Λ. such that µn = P ◦ Yn for all n ∈ N0 := N∪ {0} and limn→∞ Yn = Y a. Replacing f by 1 − f in this inequality also gives lim inf n→∞ Pn (f ) ≥ P (f ) and hence we have shown limn→∞ Pn (f ) = P (f ) as claimed. See Theorem 4. Deﬁnition 13. Given “P ¯n (Kε ) ≥ 1 − ε for all n. w . Since ε > 0. see Exercise ??. we know that C (X )∗ is in one to one correspondence with the complex measures on (X. Yn : Ω → X. Theorem 13. By what we have just proved. P k k→∞ ¯ Since ε > 0 is arbitrary. d) be a separable metric ∞ space and {µn }n=0 be probability measures on (X. lim sup Pk (F ) = lim sup P k k→∞ k→∞ which shows Pk → P. First suppose that X is compact. Page: 159 job: prob macro: svmonob. let Kε ⊂ X be a compact set such that P ¯ Kε is compact in X it is compact in X as well and in particular a closed subset ¯ Therefore by Proposition 13. Let X be a topological space. Corollary ??. ¯ .cls date/time: 23-Feb-2007/15:20 . By Theorem 18. By the Riesz theorem. Suppose X is a separable metrizable space and Λ = {Pn }n=1 is a tight sequence of probability measures on BX .13. Then there exists a subse∞ quence {Pnk }k=1 which is weakly convergent to a probability measure P on BX . BX ) such that µn =⇒ µ0 as n → ∞. lim supn→∞ Pn (f ) ≤ P (f ).44 of X.7 Weak Convergence in Metric Spaces 159 Since k is arbitrary. Given a closed subset F ⊂ X. B . Hence there exists a subsequence {Pnk }k=1 which is weak -* convergent to a probability measure P on X.30 on page 79 of Kallenberg [3]. Alternatively.47. P ) and measurable func−1 tions. this shows with X0 := ∪∞ n=1 K1/n satisﬁes P (X0 ) = 1. Proof.s.” this is where the tightness assumption is going to be used. The main thing we now have to prove is that ¯ (X ) = 1.* compact. Then for g ∈ C (X ) and f ∈ Γ. use the cantor’s diagonaliza∞ tion procedure on a countable dense set Γ ⊂ C (X ) so ﬁnd {Pnk }k=1 such that Λ(f ) := limk→∞ Pnk (f ) exists for all f ∈ Γ.

.

1] ∗ 1[−1. f (λj − λk ) ξj ξ j. if µ := P ◦ X −1 . then we let f (λ) := fX (λ) := E eiλ·X . (Ω.3. Then using the translation invariance of Lebesgue meaRn sure and Tonelli’s theorem.1] ∗ 1[−1. then we have d (µ ∗ ν ) (x) = Rn be the Fourier transform or characteristic function of µ. Notation 14.0] (x) dx so that ν (A) = µ (−A) . Y } are two independent random vectors such that P ◦ X = µ and P ◦ Y −1 = ν. then fX (λ) = µ ˆ (λ) . iﬀ m f (−λ) = f (λ) for all λ ∈ Rn and for all m ∈ N.1] (x) dx and dν (x) = 1[−1.6. . .. µ on (Rn . Rn = m ([0.1] (y − x) 1[0. we have µ ∗ ν (f ) = Rn ×Rn is non-negative. The −1 convolution of µ and ν. To simplify notation we write. ξm ) ∈ Cm .1] (y ) dy R = (14.1]+x (y ) 1[0. B .0] (x − y ) 1[0. Of course. Suppose that dµ (x) = u (x) dx where u (x) ≥ 0 and u (x) dx = 1.k=1 from which it follows that . 1])) = (1 − |x|)+ . Example 14. is the measure. eiλ·x dµ (x) If we further assume that dν (x) = v (x) dx. Let µ and ν be two probability measure on (Rn . . f (x + y ) u (x) dxdν (y ) = Rn ×Rn f (x) u (x − y ) dxdν (y ) ¯k ≥ 0 for all (ξ1 .0] (x) dx where 1[0. = 14.0] (x) = R 1[−1. A function f : Rn → C is said to be positive deﬁnite. Suppose that n = 1. Of course we may give a more direct deﬁnition of the convolution of µ and ν by observing for A ∈ BRn that µ ∗ ν (A) = P (X + Y ∈ A) = Rn u (x − y ) v (y ) dy dx. f ∈ L1 (µ) .4. we will often write µ (f ) for Ω f dµ.1] (y ) dy 1[0.k=1 m m Remark 14. .3) = R dµ (x) Rn dν (y ) 1A (x + y ) 1[0.2) (14.1] (y ) dy = Rn ν (A − x) dµ (x) µ (A − x) dν (x) . In this case d (µ ∗ ν ) (x) = 1[0.1) (14.1.2 Given a measure µ on a measurable space.14 Characteristic Functions (Fourier Transform) Deﬁnition 14. {f (λj − λk )}j. Xn ) : Ω → Rn is a random vector on some probability space (Ω.1 Basic Properties of the Characteristic Function Deﬁnition 14. Given a probability measure. . denoted µ ∗ ν. dµ (x) = 1[0. P ) . let µ ˆ (λ) := Rn d (µ ∗ ν ) (x) = Rn u (x − y ) dν (y ) dx. Deﬁnition 14. {λj }j =1 ⊂ Rn the matrix. 1] ∩ (x + [0. . B ) and a function. More explicitly we require. . BRn ) . . BRn ) .5. u ∗ v (x) = Rn u (x − y ) v (y ) dy = Rn v (x − y ) u (y ) dy. If X = (X1 . P ◦ (X + Y ) where −1 {X.

n} . ∂jm µ ˆ (λ) = Rn d ¯k = µ ˆ (λj − λk ) ξj ξ j. If Rn x dµ (x) < ∞. then µ ˆ ∈ C l (Rn . f : Rn → C.7 For l ∈ N ∪ {0} . then f ∈ C l (Rn . µ is symmetric. .k=1 m ¯k dµ (x) ei(λj −λk )·x ξj ξ eiλj ·x ξj eiλk ·x ξk dµ (x) Rn j. let C l (Rn . .) 4.time continuously diﬀerentiable.e. .8 (Basic Properties of µ ˆ). Item 5. . µ ˆ (λ) = µ ˆ (−λ) for all λ ∈ Rn and in particular. Let µ and ν be two probability measures on (Rn .10 below then implies µ = ν.41 below. . Let H be the subspace of bounded measurable complex functions. ξm ) ∈ Cm . .10 (Injectivity of the Fourier Transform). . Proof. µ ˆ (λ) is continuous.e. . then. More explicitly. . 1. ∂jk f. b ∈ Rn . 2. µ ˆ is a positive deﬁnite function. The proof of items 1. Example 14. l 5. and X : Ω → Rn is a random vector. λ2 According to example 14. l and all j1 . 2. Proof. ixjm ) eiλ·x dµ (x) for all m ≤ l. This proves item 3.162 14 Characteristic Functions (Fourier Transform) m m Notation 14. BRn ) . 2. . . =2 λ2 sin λx − cos λx x=1 dx = 2 |x=0 λ λ2 Proposition 14. It also easy to see that µ ˆ (λ) = µ ˆ (−λ) and µ ˆ (λ) = µ ˆ (−λ) if µ is symmetric. and −iλ eiλx dx = 2 1 (ixj1 . iλ 0 e−iλ − 1 ˆ (λ) = ν ˆ (λ) = µ ˆ (−λ) = µ .4) Page: 162 job: prob macro: svmonob. jk ∈ {1. If X and Y are independent random vectors then fX +Y (λ) = fX (λ) fY (λ) for all λ ∈ Rn .. . Let dµ (x) = 1[0. then µ = ν. BRn ) such that µ ˆ=ν ˆ. then faX +b (λ) = eiλ·b fX (aλ) .1] (x) dx and ν (A) = µ (−A) . i. are elementary and will be left to the reader. Suppose that Λ ⊂ Zd is a ﬁnite set. . The uniqueness Proposition 14. (For the converse of this result. let m ∈ N. m {λj }j =1 ⊂ Rn and (ξ1 . C) iﬀ the partial derivatives.cls date/time: 23-Feb-2007/15:20 . .. 7. .9 (Example 14. . follows by induction using Corollary 8. 2. i. Conversely if µ ˆ (λ) is real then µ ˆ (λ) = µ ˆ (−λ) = eiλ·x dν (x) = ν ˆ (λ) Rn µ ∗ ν (λ) = µ ˆ (λ) ν ˆ (λ) = |µ ˆ (λ)| = eiλ − 1 iλ 2 = 2 [1 − cos λ] . . ∂j1 . Proposition 14. If a ∈ R. C) denote the vector space of functions. Then µ ˆ (λ) = eiλ − 1 . then µ ˆ (λ) is real. 3. Then aλ eiλ·x/(2πL) (14. 6. µ ˆ (0) = 1. C) and ∂j1 . and 7. If µ and ν are two probability measure on (Rn . exist if ∂j := ∂x j and are continuous for k = 1. such that µ (f ) = ν (f ) . .5 we also have d (µ ∗ ν ) (x) = (1 − |x|)+ dx and so directly we ﬁnd µ ∗ ν (λ) = R 1 eiλx (1 − |x|)+ dx = R cos (λx) (1 − |x|)+ dx 1 =2 0 (1 − x) cos λx dx = 2 0 1 1 0 (1 − x) d sin λx λ sin λx = −2 d (1 − x) =2 λ 0 1 − cos λ . and |µ ˆ (λ)| ≤ 1 for all λ. iﬀ µ (−A) = µ (A) for all A ∈ BRn . Therefore if µ is symmetric. ∂ . then µ is symmetric iﬀ X = −X. This may be alternatively expressed as µ ∗ ν (λ) = µ ˆ (λ) ν ˆ (λ) for all λ ∈ Rn .k=1 m 2 = = Rn j =1 e iλj ·x ξj dµ (x) ≥ 0.38. see Bochner’s Theorem 14. . 6. . For item 4. f : Rn → C which are l . µ ˆ is real valued iﬀ µ is symmetric.k=1 Rn j. .5 continued. L > 0 and p (x) = λ∈Λ where ν (A) := µ (−A) . Then H is closed under bounded convergence and complex conjugation.). . (If µ = P ◦ X −1 for some random vector X..

Hence it follows from the bounded convergence theorem that f ∈ H for all f ∈ C (Rn . 2. C) ⊂ H. So for each λ > 0 near 0.e. 1. λ) such that u (0) − u (λ) → −u (0) as λ ↓ 0 λcλ and E X 2m 1 − cos (λX ) 1 − cos (λX ) u (0) − u (λ) ≤ E X 2m = . n. . . This completes the induction argument. For the general induction step we assume the truth of the theorem at level m in which case we know by Lemma 14. . . Then for L > 0 suﬃciently large the function. 2. C) which are L – periodic.14) implies H contains all bounded σ (Cc (Rn .1 Basic Properties of the Characteristic Function 163 with aλ ∈ C.34 below) shows that there is a correspondence between the number of moments of X and the diﬀerentiability of fX . to each λ > 0 with λ near 0. Let X be a random variable. . il The following theorem is a partial converse to this lemma. BR ) . i. any f ∈ C (Rn . C) such that g := f (2m) is diﬀerentiable in a neighborhood of 0 and g (0) = f (2m+2) (0) exists. f (λ) = E eiλX . R)) = BRn – measurable functions and this certainly implies µ = ν. This will be proved by induction on m. If f ∈ C 2m (R. λ↓0 2 λ2 An application of Lemma 14. then µ diﬀerentiable and µ ˆ(l) (λ) = E (iX ) eiλX = R l n 1−cos(λX ) λ2 1 1 − cos (λX ) E X 2 ≤ lim inf E ≤ −u (0) < ∞. X will be a random variable and µ will be a probability measure on (R. fL (x) := λ∈Zn f (x + Lλ) . Cc (Rn . so that p ∈ H.3 or Theorem 9. there exists cλ ∈ (0. n) may be uniformly approximated by a trigonometric polynomial of the form in Eq. C)). The following Lemma is a special case of item 4.12. Since u (λ) := Re f (λ) = E [cos (λX )] . Lemma 14. macro: svmonob. 2 λ λcλ λcλ m m (ix) eiλx dµ (x) for l = 0.11. C) which is L – periodic.4). C) . λ Therefore. If µ = P ◦ X −1 is the distribution of X. Page: 163 job: prob Another use of Fatou’s lemma gives. (i. C) . it follows that u is an even function of λ and hence u = Re f is an odd function of λ and in particular. C) . of Proposition 14. Now suppose f ∈ Cc (Rn . . From the Stone-Weirstrass theorem (see Exercise 14. µ (p) = λ∈Λ aλ µ ˆ λ 2πL = λ∈Λ aλ ν ˆ λ 2πL = ν (p) Proof. An application of the multiplicative system Theorem (see either Theorem 9. we may apply Fatou’s lemma to conclude. Since fL → f boundedly as L → ∞. u (0) = 0. f (x + Lei ) = f (x) for all x ∈ Rd and i = 1. i. We start with m = 0 in which case we automatically we know by Proposition 14.8 or Lemma 14.11 then implies that f ∈ C 2 (R. . (14. see Exercise 14. is continuous and L periodic and hence fL ∈ H. By assumption we know that g is diﬀerentiable in a neighborhood of 0 and that g (0) exists. C) .11 and Theorem 14. 1. Hence the combination of Lemma 14.7 below) or the theory of the Fourier series. .8 below.8. 2. Theorem 14.11 that f ∈ C (R. we may further conclude that f ∈ H as well.cls date/time: 23-Feb-2007/15:20 . Suppose n ∈ N and X is random variables such that E [|X | ] < ˆ (λ) := E eiλX is C n – ∞. there exists 0 < cλ < λ such that u (λ) − u (0) = u (cλ ) = u (cλ ) − u (0) . .14. Then E X 2m+2 < ∞ and f ∈ C 2m+2 (R. u (0) − u (λ) u (cλ ) − u (0) =− → −u (0) as λ ↓ 0. For the most part we are now going to stick to the one dimensional case.e. . l In particular it follows that E Xl µ ˆ(l) (0) = . } .11 may be used to show f ∈ C 2m+2 (R.e. . Then by assumption. m ∈ {0. We now proceed exactly as before but now with u := Re g.12 (see also Corollary 14.11 that f (2m) (λ) = (−1) E X 2m eiλX =: (−1) g (λ) . λcλ cλ Since E and limλ↓0 1 − cos (λX ) u (0) − u (λ) 1 − cos (λX ) ≤E = 2 λ λcλ λcλ 2 =1 2 X . 1 1 − cos (λX ) E X 2m+2 = lim inf E X 2m ≤ −u (0) < ∞ λ↓0 2 λ2 from which Lemma 14. By the mean value theorem.

t ≥ 0. Page: 164 job: prob macro: svmonob.13. n P (Z = n) = e−a a n! . Then ∞ Proposition 14.b] FT (t) := P (T ≤ t) = 1 − e−a(t∨0) . 1 µ ˆ (λ) = √ 2π i =√ 2π i =√ 2π ixe−x R 2 fZ (λ) = −a2 ei2λ − aeiλ exp a eiλ − 1 from which we conclude. iλ (b − a) ae−at eiλt dt = a If a = −c and b = c with c > 0. then µ ˆ (λ) = e−λ x2 dµ (x) = 1. µ := P ◦ T −1 . ∞ eiλx dx = a eiλb − eiλa . dx R − Solving this equation of µ ˆ (λ) then implies µ ˆ (λ) = e−λ 2 /2 µ ˆ (0) = e−λ 2 /2 µ (R) = e−λ 2 /2 . 2 Example 14. the law of T. In partic- R fZ (λ) = E eiλZ = e−a n=0 eiλn aeiλ an = e−a n! n! n=0 ∞ n = exp a eiλ − 1 . . µ ˆ (0) = 0 and µ ˆ (0) = − 3 c and hence it follows that µ ˆ (0) µ ˆ (0) 2 = a−1 and ET 2 = = 2 2 i i a − 1 2 a xdµ (x) = 0 and R 1 x dµ (x) = c2 .”) The distribution function for T is /2 iλx e dx d −x2 /2 iλx e e dx dx R 2 d e−x /2 eiλx dx = −λµ ˆ (λ) . Since FT (t) is piecewise diﬀerentiable. λc Since a =µ ˆ (λ) . EZ = a = Var (Z ) . then P (T ≥ t) = e−at for some a > 0.e. i Therefore. then µ ˆ (λ) = Observe that sin λc . . 2 /2 . Suppose T is a positive random variable such that P (T ≥ t + s|T ≥ s) = P (T ≥ t) for all s. for µ ˆ with respect to λ and then integrating by parts implies. 3! 1 2 and therefore.16. 3 R 2 = a−2 . t ≥ 0. Suppose Z is a Poisson random variable with mean a > 0. 1 EZ = fZ (0) = a and EZ 2 = −fZ (0) = a2 + a.14. 1 µ ˆ (λ) = √ 2π e−x R 2 /2 iλx Diﬀerentiating this result gives. If dµ (x) := √1 e−x 2π ular we have xdµ (x) = 0 and R /2 dx. E eiaT = 0 dµ (t) = FT (t) dt = ae−at 1t≥0 dt. or equivalently P (T ≥ t + s) = P (T ≥ t) P (T ≥ s) for all s.164 14 Characteristic Functions (Fourier Transform) 14. a − iλ a (a − iλ) 3 µ ˆ (λ) = i it follows that ET = and hence Var (T ) = 2 a2 (a − iλ) 2 and µ ˆ (λ) = −2 1 µ ˆ (λ) = 1 − λ2 c2 + .2 Examples Example 14. (Such exponential random variables are often used to model “waiting times. fZ (λ) = iaeiλ exp a eiλ − 1 and e dx. If −∞ < a < b < ∞ and dµ (x) = µ ˆ (λ) = 1 b−a b 1 b−a 1[a. Proof. Diﬀerentiating the formula. (x) dx then Therefore.cls date/time: 23-Feb-2007/15:20 . Example 14.15. has a density. i.

T = n) = n=0 pn µn (A) . on BR and µn =⇒ µ as n → ∞. cX µ := n=0 pn µn . namely the measure n times 14. (14. for ever ε > 0 we may choose aε suﬃciently large so that 1 aε 2 macro: svmonob.19.20 (Tail Estimate). (14.cls 2/aε ∞ ˆn is the Example 14. ∞ ∞ Proof. ν. Yc ≥ 1/2 if |cX | ≥ 2.5).6) Example 14.3 Continuity Theorem 165 Example 14. if {Xk }k=1 are i. In other words. 1 2c where c fX (λ) dλ = −c 1 2c c E eiλX dλ = E −c sin cX . cX Notice that Yc ≥ 0 (see Eq.21 (Continuity Theorem). BR ) and n ∈ N. (14. Let {Xn }n=0 ∪{T } be independent P (T = n) = pn for all n ∈ N0 . −2/aε date/time: 23-Feb-2007/15:20 . Hence we may conclude Yc := 1 − E [Yc ] ≥ E [Yc : |cX | ≥ 2] ≥ E 1 1 : |cX | ≥ 2 = P (|X | ≥ 2/c) .i. Then n=0 pn µ ˆn is the characteristic function of the probability measure. on (R. As an explicit example. By the continuity of f at λ = 0. 2 2 = n=0 P (Xn ∈ A. BR ) and ∞ ∞ ∞ {pn }n=0 ⊂ [0. Suppose that {µn }n=1 is a sequence of probability measure on (R. Recall that the Fourier transform of the uniform distribution on λc [−c. µ. 2 = n=0 E eiλXn P (T = n) = n=0 pn µ ˆn (λ) . Let us also observe that ∞ ∞ Combining this estimate with Eq. If µ is a probability measure on (R.18.7) shows. then µ ˆn is the characteristic function of the probability measure.5) −1 Alternatively put. E eiλXn : T = n 1 2c c µ ˆ (λ) = E eiλXT = n=0 ∞ E eiλXT : T = n = n=0 ∞ (1 − fX (λ)) dλ ≥ −c 1 P (|X | ≥ 2/c) . c] is sin λc and hence 1 2c Therefore. T = n) = n=0 ∞ P (Xn ∈ A.47)) and moreover. (14. In this case. If f is continuous at λ = 0. If µ is a probability measure on (R. BR ) . BR ) then n=0 pn µ characteristic function of a probability measure. BR ) and suppose that f (λ) := limn→∞ µ ˆn (λ) exists for all λ ∈ R. Page: 165 job: prob (1 − Re f (λ)) dλ ≤ ε/2. B . Then for a > 0.14. Let X : (Ω. T = n) sin cX .17. random variables with µ = P ◦ Xk . if n −a . 1 (1 − fX (λ)) dλ = −2/a a 2 2/a (1 − Re fX (λ)) dλ −2/a (14.7) µ (A) = P (XT ∈ A) = n=0 ∞ P (XT ∈ A.6). ∞ µ.d. ∞ Taking a = 2/c in this estimate proves Eq. 1] such that n=0 pn = 1. (14. fXT (λ) = E eiλXT = exp (a (fX1 (λ) − 1)) . ∞ ∞ (1 − fX (λ)) dλ = 1 − E −c sin cX cX = E [Yc ] (14. ∞ ν = n=0 pn µ∗n where µ∗n is deﬁned in Eq. P (|X | ≥ a) ≤ a 2 2/a µ n ∗n := µ ∗ · · · ∗ µ. Then c Here is a more interesting interpretation of −1 = µn and random variables with P ◦ Xn µ (A) = P (XT ∈ A) .3 Continuity Theorem Lemma 14. then f is the characteristic function of a unique probability measure. where XT (ω ) := XT (ω) (ω ) . Indeed. Proof. then n fX1 +···+Xn (λ) = fX (λ) . then a > 0 and pn = a n! e ∞ pn µ ˆn = n=0 an −a n ˆ µ−1) e µ ˆ = e−a eaµ = ea(ˆ n ! n=0 ∞ is the characteristic function of a probability measure. Theorem 14. Suppose that {µn }n=0 are probability measure on (R. P ) → R be a random variable and fX (λ) := E eiλX be its characteristic function.

the condition of a function being positive deﬁnite is preserved under taking pointwise limits.45 below. we could ﬁnd a bounded continuous function. However by Theorem 13. (14. Show for a > 0.d. Therefore taking expectations of Eq. . . One could also use Bochner’s Theorem 14. (14. it follows that µ = ν.166 14 Characteristic Functions (Fourier Transform) According to Lemma 14. (1 + zn ) = eln(1+zn ) n n Remark 14. Suppose that {zn }n=1 ⊂ C satisﬁes. The following lemma will be needed before giving our ﬁrst applications of the continuity theorem.22. g. Since ν ˆ (λ) = liml→∞ µ ˆl (λ) = f (λ) = µ ˆ (λ) . . This leads to a contradiction since. we may ﬁnd a subsequence. Yc ≥ 0 and Yc ≥ 1/2 if c |Xj | ≥ 2 for some j. cXj j =1 d (14. Working as above. Therefore. dλd . k→∞ (1 − fX (λ)) dλ = E [Yc ] ≥ E [Yc : |X |∞ ≥ 2/c] ≥E 1 1 : |X |∞ ≥ 2/c = P (|X |∞ ≥ 2/c) . Since nzn → ξ. µn ({x : |x| ≥ aε }) ≤ → 1 aε 2 1 aε 2 2/aε Solution to Exercise (14. Since. If not.23. if c |X |∞ ≥ 2.cls date/time: 23-Feb-2007/15:20 Page: 166 job: prob . By increasing aε if necessary we can assure that µn ({x : |x| ≥ aε }) ≤ ε for all n and hence ∞ Γ := {µn }n=1 is tight. Suppose now X : (Ω. ∞ By Theorem 13. ε ≤ lim |µ (g ) − µl (g )| = |µ (g ) − ν (g )| = 0.c] d sin cXj =: Yc .e.1). Suppose P ∞ n that {Xn }n=1 are i. µl = µkl of µk such that µl =⇒ ν for some probability measure ν. Indeed.23 that macro: svmonob. We now claim that µn =⇒ µ as n → ∞. it follows that µ ˆ (λ) = lim µ ˆnk (λ) = f (λ) for all λ ∈ R. . f (λ) = 1 + iµλ + o (λ) . µ.8) λ n n = 1 + iµ λ +o n 1 n n where |X |∞ = maxi |Xi | and dλ = dλ1 . B . (1 + zn ) = e and 2 ln (1 + zn ) = zn + O zn = zn + O ∞ that is f is the characteristic function of a probability measure.24 (Weak Law of Large Numbers revisited). limn→∞ nzn = ξ ∈ C. {µnk }k=1 and a probability measure µ on BR such that µnk =⇒ µ as k → ∞. Then S n → EX1 =: µ. Exercise 14. 1 Proposition 14. such that limn→∞ µn (g ) = µ (g ) or equivalently.9) implies. l→∞ 1 n2 .20 and the DCT. P (|X |∞ ≥ a) ≤ 2 a 4 d = en ln(1+zn ) = en(zn +O( n2 )) → eξ as n → ∞. then n lim (1 + zn ) = eξ .9) (1 − Re f (λ)) dλ ≤ ε/2.42 again. Proof.2/a]d (1 − Re fX (λ)) dλ (14. it follows that zn ∼ n → 0 as n → ∞ and therefore ln(1+zn ) by Lemma 14. integrable random variables. Since x → eiλx is a bounded and continuous function. 1 2c d [−c. there is a further subsequence. it follows from Lemma 14. if f (λ) := limn→∞ µ ˆn (λ) is continuous then f is the characteristic function of a probability measure. P ) → Rd is a random vector and fX (λ) := E eiλ·X is its characteristic function.8). i. −2/aε where as before.c]d Hence µn ({x : |x| ≥ aε }) ≤ ε for all suﬃciently large n. there would exists ε > 0 and a subsequence {µk := µnk } such that |µ (g ) − µk (g )| ≥ ε for all k ∈ N. Then by Taylor’s theorem.41 to conclude. 2 2 Taking c = 2/a in this expression implies Eq.2/a]d a 4 d [−2/a. Lemma 14.1. f Sn (λ) = f n (1 − fX (λ)) dλ = 2 [−2/a. n→∞ ξ Proof.42. Let f (λ) := fX1 (λ) = E eiλX1 . we have (1 − Re µ ˆn (λ)) dλ 1 2c d −2/aε 2/aε 1 − eiλ·X dλ = 1 − [−c.i. say n ≥ N.

12) that S2n (λ) = f Sn f√ √ 2n Alternative proof.11) Theorem 14. P S √n ≤ y − P (N (0. then i.26. k even √ Xj n (14. → 0 as n → ∞. If {Xn }n=1 are 2 that EX1 = 0 and EX1 = 1.21 and Proposition 14.6 below as follows.i. using characteristic functions we would conclude from Eq. By Theorem 14. it follows that S ⇒ µ and since µ is constant we n = may apply Lemma 13. we know √ =⇒ L where L is some n 2 random variable with EL = 0 and EL = 1. To rigorously understand this. square integrable random variables such that EX1 = 0 and Sn 2 EX1 = 1. with f (λ) = limn→∞ f √ n fL (λ) . such 3 3 that. 1) ≤ y ) → 0 as n → ∞. It is now a reasonable question to ask “why” is the limiting random variable normal in Theorem 14. However we will give a related result in Theorem 14.d.23 with zn = − 1 2 1+ε λ √ n λ2 .d. = 1 − 1 2 λ √ 2 n 2 2n 1+ε λ √ 2 n f (λ) = f . 1) . (14. (14. Sn (λ) = E e f√ n Sn iλ √ n In particular the rate of convergence is n−1/2 . Page: 167 job: prob macro: svmonob.25.cls date/time: 23-Feb-2007/15:20 . C < ∞.25 and Exercise 13. By the n continuity Theorem 14. Suppose that ∞ {Xn }n=1 are i. it suﬃces to show n→∞ Proof. square integrable random variables such which is the characteristic function of the constant random variable. lim E e Sn iλ √ n =e −λ2 /2 for all λ ∈ R.14. This is a direct consequence of Theorem 14. −λ Sn (λ) − e f√ n 2 /2 n 2 λ − e−λ /2n = f √ n 2 λ ≤n f √ − e−λ /2n n 1 λ λ2 λ2 =n 1− 1+ε √ − 1− +O 2 n 2n n n n λ √ 2 Sn f√ n λ √ 2 .3 Continuity Theorem n→∞ 167 lim f Sn (λ) = eiµλ n ∞ Corollary 14. n Proof.46)) that 1 (14.21. n (14. Berry (1941) and Esseˆ n (1942) showed there exists a constant.25 (The Basic Central Limit Theorem).28 below.3. Then √ =⇒ N (0.12) wherein we have used Lemma 14.27. One way to understand this is.16. Remark 14. Letting f (λ) := E eiλX1 . k odd = f 1+ε λ √ n λ √ n n 1 = 1− 2 λ2 n n √ Xj → e−λ 2 /2 . that 1 n2 f (λ) = f Iterating this equation then shows λ √ 2 2n n λ √ 2 2 .43) and (14. This proof uses Lemma 15. n + 2n k=1. µ. if Sn under the assumptions of Theorem 14.i. we have by Taylor’s theorem (see Eq. We will not prove this theorem here. then S 1 √2n = √ 2n 2 2n k=1. 1) ≤ y ) ≤ C σ n 3 √ / n.19 to conclude Sn n sup P λ∈R → µ. then sup P λ∈R S ρ √n ≤ y − P (N (0. n d d 1 =⇒ √ (L1 + L2 ) 2 where L1 = L = L2 and L1 and L2 are independent.25. if ρ := E |X1 | < ∞. Therefore.10) f (λ) = 1 − (1 + ε (λ)) λ2 2 where ε (λ) → 0 as λ → 0. Sn (λ) = Passing to the limit in this equation then shows. The exact value of the best constant C is still unknown but it is known to be less than 1.

3! (14.14) shows. We now make use of Taylor’s theorem with integral remainder the form. n n Theorem 14.18) using.i. The only property about ∞ normal random variables that we shall use the proof is that if {Nn }n=1 are i.15) with ∆ replaced by δ and subtracting the results then implies 1 f (x + ∆) − f (x + δ ) = f (x) (∆ − δ ) + f (x) ∆2 − δ 2 + ρ (x.14) k=1 1 M 3 3 · E |N1 | + |X1 | . To simplify notation. standard normal random variables. ENk = 2 1 = EXk and the fact that Uk is independent of both Xk and Nk . δ ) δ 3 ≤ M 3 3 |∆| + |δ | . (14. That is we must have L = N (0. (14. Then by a telescoping series argument.168 14 Characteristic Functions (Fourier Transform) An application of Lemma 14.13) where Sn := X1 + · · · + Xn and N = N (0. ≤√ n 3! This completes the proof of Eq. For more in this direction the reader is advised to look up “Stein’s method. (14.16) with x = Uk . 1) and Xn = X1 . ∆) ∆3 − r (x.” macro: svmonob. ENk = 1 = EXk .18) where M 3 3 |Nk | + |Xk | . using Eq. k − 1 k k √ √ ∆ = Nk / n and δ = Xk / n. Let X be independent random variables such that Nn = n=1 d ¯ ¯ n by Xn . ∆) := 2n 1+ε λ √ 2 n 1 2 1 f 0 (x + t∆) (1 − t) dt. 1) . Suppose that ∞ 3 {Xn }n=1 are mean zero variance one i. ∆) . then T N + · · · + Nn d √n := 1 √ = N (0.16) 2 where |ρ (x. we ﬁnd |Rk | = |E [f (Vk ) − f (Vk−1 )]| = |ERk | ≤ ≤ M 3 3 E |Nk | + |Xk | 3! · n3/2 (14. (14. Nn Proof.13) since Tn (λ) = f√ fN Tn d √ = n N because. it follows that √ √ f (Vk ) − f (Vk−1 ) = f Uk + Nk / n − f Uk + Xk / n 1 1 2 2 = √ f (Uk ) (Nk − Xk ) + f (Uk ) Nk − Xk + Rk 2n n (14. √ If we deﬁne Uk := (N1 + · · · + √ Nk−1 + Xk+1 + · · · + Xn ) / n. For this ∞ proof we will assume {Xn }n=1 has third moments.1) (λ) . 1) . 1) .28 (A Non-Characteristic Proof of the CLT). we will denote X Tn := N1 + · · · + Nn and for 0 ≤ k ≤ n.23 then shows 1 f (λ) = lim 1 − n→∞ 2 =e 2 −1 2λ r (x. 1 f (x + ∆) − f (x) = f (x) ∆ + f (x) ∆2 + r (x. (14. ∆)| = r (x. 2 λ √ 2 2 n Taking Eq. (14. ∆) ∆3 2 where Page: 168 job: prob (14. 3! · n3/2 n n Combining this estimate with Eq. Ef S √n n 1 M 3 3 · E |N | + |X1 | − Ef (N ) ≤ √ n 3! d d wherein we have used the simple estimate.d.d random variables such that E |X1 | < 3 (3) ∞.i. √ √ E f Sn / n − f T n / n = k=1 ERk ≤ k=1 E |Rk | [f (Vk ) − f (Vk−1 )] .19) 3! · n3/2 2 Taking expectations of Eq. let √ Vk := (N1 + · · · + Nk + Xk+1 + · · · + Xn ) / n √ √ with the convention that Vn = Sn / n and V0 = Tn / n.15) n λ √ n n 1 λ2 = exp − n 2 n = exp −λ2 /2 = fN (λ) . Let N (0. (14. it follows that √ √ f Sn / n − f Tn / n = f (Vn ) − f (V0 ) = n ∞ d M 3 3 E |N1 | + |X1 | .cls date/time: 23-Feb-2007/15:20 . ¯ n .17) = fN (0. ∆)| ≤ M/3!. It is interesting to give another proof of the central limit theorem. Hence. (14.19). Eq. then Vk = √ Uk + Nk / n and V = U + X / n. Then for f ∈ C (R) with M := supx∈R f (x) < ∞. |r (x. (14.

g )L := 2πL πL b b µ ([a. πL] . In particular.4 A Fourier Transform Inversion Formula Proposition 14. let eL and let n (x) := e 1 (f.22) where ∞ ˆ (λ) = f −∞ f (y ) eiλy dy. f −∞ (14. let S (c) := 1 2π c −c f. The following lemma is needed in the proof of the inversion Theorem 14.30 below. Our next goal is to ﬁnd an inversion formula for the Fourier transform. S (c) = ∞ 1 c 1 c sin λ dλ = sin λ e−λt dt dλ π 0 λ π 0 0 c 1 ∞ = dt dλ sin λe−λt π 0 0 1 ∞ 1 1 = + e−tc [− cos c − t sin c] dt.10 guarantees the injectivity of the Fourier transform on the space of probability measures. we now restrict ourselves to the one dimensional case. (14.29. if f ∈ Cc (R) with supp(f ) ⊂ [−πL. µ ˆ (λ) −∞ c = lim f (x) g ¯ (x) dx 1 c→∞ 2π µ ˆ (λ) −c e−iλa − e−iλb iλ −πL for f. then for x ∈ [−πL. To keep our exposition as simple as possible.cls . n −i L x For L > 0. πL] .23) Hence if we now think that f (x) is a probability density and let dµ (x) := ˆ (λ) . Lemma 14. We will repeat the argument here for the reader’s convenience. b]) = a f (x) dx = a ∞ b 1 2π ∞ µ ˆ (λ) e−iλx dλ dx −∞ = = 1 2π 1 2π µ ˆ (λ) −∞ ∞ a e−iλx dx dλ e−iλa − e−iλb iλ dλ dλ.14. dx) .21) f (y ) ei L y dy e−i L x (14. The ﬁrst assertion has already been dealt with in Example 10. πL] . dx) . λ 1 if y > 0 sgn(y ) = −1 if y < 0 . 2 π 0 1 + t2 c c n∈Z and we are lead to expect.12. Letting L → ∞ in Eq. we should expect f (x) dx so that µ ˆ (λ) = f wherein we have used c dλ sin λe−λt = Im 0 0 dλeiλ e−λt = Im 0 dλe(i−t)λ e(i−t)c − 1 (−i − t) = Im = e(i−t)c − 1 (i − t) = 1 Im 1 + t2 1 e−tc [− cos c − t sin c] + 1 1 + t2 date/time: 23-Feb-2007/15:20 Page: 169 job: prob macro: svmonob. eL n eL L n (x) = 1 2πL πL n∈Z −πL sin λ dλ. f (x) = n∈Z This should provide some motivation for Theorem 14. For c > 0. λ (14. let us ﬁrst recall a few facts about Fourier series.20) then suggests that 1 2πL n 1 x ˆ n e−i L f → L 2π ∞ ˆ (λ) e−iλx dλ f −∞ Proof.4 A Fourier Transform Inversion Formula 169 14.20) n n Then S (c) → π boundedly as c → ∞ and c −c 1 = 2πL where n∈Z n x ˆ n e−i L f L sin λy dλ = sgn(y )S (c |y |) for all y ∈ R. g ∈ L2 ([−πL. By symmetry and Fubini’s theorem. f (x) = 1 2π ∞ ˆ (λ) e−iλx dλ.30 below. Then it is well known (and fairly elementary 2 to prove) that eL n : n ∈ Z is an orthonormal basis for L ([−πL. πL] . 0 if y = 0 (14. To motivate the construction below.

2 is continuous by the dominated convergence theorem. b)) + 1 [µ ({a}) + µ ({b})] .cls date/time: 23-Feb-2007/15:20 . µ ˆ (λ) −c = R dµ (x) e −iλ(a−x) −iλ(b−x) = µ ((a. π 0 1 + t2 2 The the integral in Eq. I (c) := −c c ρ (x) dx = a 1 2π b dx a R dλµ ˆ (λ) e−iλx b µ ˆ (λ) e−iλa − e−iλb iλ dλ dλ = −c R eiλx dµ (x) c e−iλa − e−iλb iλ e −iλa 1 2π 1 = 2π = dλµ ˆ (λ) R a dxe−iλx e−iλa − e−iλb iλ e−iλa − e−iλb dλ iλ dλµ ˆ (λ) R c = R dµ (x) −c c dλeiλx dλ −c −e iλ −e iλ −iλb 1 = lim 2π c→∞ . Example 14. R µ ˆ (λ) −c e−iλa − e−iλb iλ c dλ = µ ((a. (14. and the dµ = ρdm.9 that eiλx (1 − |x|)+ dx = 2 R = 2π R dµ (x) [sgn(x − a)S (c |x − a|) − sgn(x − b)S (c |x − b|)] .23) tends to as c → ∞ by the dominated convergence theorem. Suppose that µ is a probability measure on (R.. it is now easy to verify that µ (A) = A ρ (x) dx for all A ∈ BR or R hdµ = R hρdµ for all bounded measurable functions h : R → R.22) is a consequence of the change of variables. z = λy. is an odd function of λ it follows that I (c) = R dµ (x) −c c dλ Re dλ −c = R dµ (x) e−iλ(a−x) − e−iλ(b−x) iλ sin λ (x − a) − sin λ (x − b) λ Using one of the multiplicative systems theorems. ρ (x) := 1 2π µ ˆ (λ) e−iλx dλ. 2 job: prob This identity could also be veriﬁed directly using residue calculus techniques from complex variables. BR ) and −∞ < a < b < ∞. λ2 (14. Recall from Example 14.29. If µ is a probability measure on (R.170 14 Characteristic Functions (Fourier Transform) 1 ∞ 1 1 dt = . m – a. b)) + 1 (µ ({a}) + µ ({b})) .e. This then implies that ρ ≥ 0.32. Moreover. Proof. BR ) such that µ ˆ ∈ L1 (m) . b)) + 1 [µ ({a}) + µ ({b})] .b) (x) + 1{a} (x) + 1{b} (x) R 1 π R 1 − cos λ −iλx e dλ.24) = µ ((a. The second assertion in Eq.31 that (1 − |x|)+ = 1 dµ (x) 2 · 1(a. then 1 c→∞ 2π lim c and Corollary 14. By Fubini’s theorem and Lemma 14.31. b Proof. Page: 170 macro: svmonob. then dµ = ρdm where ρ is a continuous density on R. 2 Since Im e−iλ(a−x) − e−iλ(b−x) iλ c =− cos (λ (a − x)) − cos (λ (b − x)) λ Letting a ↑ b over a ∈ R such that µ ({a}) = 0 in this identity shows µ ({b}) = 0 for all b ∈ R. Therefore we have shown b µ ((a. b]) = a ρ (x) dx for all − ∞ < a < b < ∞.30 (Fourier Inversion Formula). The function. Now letting c → ∞ in this expression (using the DCT) shows 1 1 lim I (c) = c→∞ 2π 2 1 = 2 dµ (x) [sgn(x − a) − sgn(x − b)] R 1 − cos λ . (14. λ2 Hence it follows1 from Corollary 14. Theorem 14.

again. 1) . Suppose X is a random variable such that u (λ) := fX (λ) continuously diﬀerentiable for λ ∈ (−2ε. we would have on one hand 1 E |X | = √ 2π |x| e−x R 2 λ−1 u (λ) dλ + u (ε) + u (−ε) + u (0) − u (0) ε /2 2 dx = √ 2π and so −1 ∞ 0 xe−x 2 /2 dx = 2 . (14. λ → M λ. we could =− δ ≤|λ|≤ε Now let M = |X | in this expression and then take expectations to ﬁnd 1 E |X | = π 1 − cos λX 1 E dλ = 2 λ π R δ ≤|λ|≤ε (1 − u (λ)) d −λ−1 δ ≤|λ|≤ε R u (λ) − 1 ε u (λ) − 1 −δ |δ + |−ε − λ λ λ−1 u (λ) dλ + λ−1 u (λ) dλ δ ≤|λ|≤ε Suppose that we did not know the value of c := still proceed as above to learn E |X | = 1 c R 1 − Re fX (λ) dλ. λ ε −λ2 /2 d λ 2π c √ 2 1 = c d 1−e R −λ2 /2 λ −1 Passing the limit as δ ↓ 0 using the fact that u (λ) is an odd function. Corollary 14.33. it suﬃces to show ∞> |λ|≤ε Making the change of variables. Evaluating Eq. we have E |X | = 1 π 1 π 1 π 1 − Re fX (λ) dλ.2. Let u (λ) := Re fX (λ) = E [cos λX ] and assume that u ∈ C 1 ((−2ε. We further assume ε 0 ≤2 0 |u (λ)| u (ε) + u (−ε) dλ + < ∞. For all random variables. λ2 1 − cos λ dλ. (Since u is even. C) .34.24) at x = 0 implies ∞ −∞ 1 − u (λ) dλ = λ2 |λ|≤ε 1 − u (λ) dλ + λ2 |λ|>ε 1 − u (λ) dλ. (14. that c = π. 2ε) for some ε > 0.26) Then E |X | < ∞ and fX ∈ C 1 (R. λ (14. λ2 ∞ 1−cos λ dλ λ2 −∞ 1 − u (λ) dλ = lim δ ↓0 λ2 δ ≤|λ|≤ε 1 − u (λ) dλ. Hence if u (λ) were α – H¨ older continuous for some α > 0. λ2 1 − Re fX (λ) dλ. fX (λ) = e 2 1 =− π c = 1 c 1−e R −λ2 /2 =2 0 |u (λ)| u (ε) + u (−ε) dλ + < ∞. + −δ δ → − lim δ ↓0 δ ≤|λ|≤ε We could then evaluate c by making a judicious choice of X. let macro: svmonob. u is odd and u (0) = 0. X. (14.5 Exercises 171 Corollary 14.) Page: 171 job: prob 14. we learn 1 − u (λ) dλ = lim δ ↓0 λ2 ε e−λ R /2 dλ = λ−1 u (λ) dλ + δ ≤|λ|≤ε |λ|≤ε u (ε) + u (−ε) ε from which it follows. λ2 R By an integration by parts we ﬁnd 1 − u (λ) dλ = λ2 = is π. For example if d X = N (0.26) would hold. λ ∈ R. λ2 u (ε) − 1 u (−ε) − 1 − ε −ε u (−δ ) − 1 u (δ ) − 1 − .14. For x.25) π · E |X | = R R Proof.25) Proof. λ ε |u (λ)| dλ < ∞. in the above integral then shows M= 1 − cos (λM ) dλ. λ2 1= Since 0 ≤ 1 − u (λ) ≤ 2 and 2/λ2 is integrable for |λ| > ε.5 Exercises Exercise 14. 2ε) . λ2 (14. Then according to Eq. π ≤ |λ|≤ε ε u (ε) + u (−ε) |u (λ)| dλ + |λ| ε On the other hand. then Eq.cls date/time: 23-Feb-2007/15:20 . C) .

27) Exercise 14. (It is easy to see that ϕ (λ.]). Here is an outline to follow. ν. Show that if ν is a ﬁnite measure on (R. To each ﬁnite and compactly supported measure. let µa (A) = µ (aA) in which case µa (f ) = µ f a−1 · for all bounded measurable f and in particular.e. ϕ (λ) is non-increasing and convex for λ ≥ 0. x) . To ﬁnish the proof it suﬃces to show that ϕ (λ) may be expressed as macro: svmonob. π x2 For a > 0. where µ is the probability measure. on (R. BR ) such that νn =⇒ ν. show k k k=1 k fY (λ) := E eiλY = exp R d Exercise 14. (14. 1 − cos nt g (t) : n2 ln n n≥2 is continuously diﬀerentiable. x) := iλx 1−iλx e − if x = 0 x2 2 −1 2λ if x = 0. C ) even though R |x| dµ (x) = ∞. 1) and Zk being Poisson random variables an with mean ak > 0.172 14 Characteristic Functions (Fourier Transform) ϕ (λ. . Exercise 14. P (Zk = n) = e−ak nk ! for n = 0. {Zk }k=1 ∪ {N } be independent random variables with N = N (0. . (You may ﬁnd the calculus estimates in Section 14. 14. f dνn → R R f dν for all f ∈ BC (R) . Here we say ν is compactly supported if there exists M < ∞ such that ν ({x : |x| ≥ M }) = 0 and we say ν is ﬁnitely supported if there exists a ﬁnite subset. From Example 14.3 in [5]). Let µ be the probability measure on 1 (R.5 (Exercise 2. x) is n n smooth in (λ. 2 .) Let {xk }k=1 ⊂ R \ {0} . BR ) given by n ν = α 2 δ0 + k=1 ak x2 k δxk . With Y := n x ( Z − a ) + αN.3. Because of the continuity theorem and some simple limiting arguments. x) dν (x) (14. BR ) . using Exercises 14. x) dν (x) is the characteristic function of a probability measure on (R.6 (Polya’s Criterioin [1. .8 to be of help. Show. ν. x) dν (x) where ν is the discrete measure on (R. 14. approximate ν by a sequence of ﬁnite measures with compact support as in item 2.28) is the characteristic function of a probability measure on (R.3. BR ) . 305. This last restriction may be removed later by a limiting argument. To do n∈Z this show. BR ) . Solution to Exercise (14. Suppose ϕ (λ) is a non-negative symmetric continuous function such that ϕ (0) = 1.cls date/time: 23-Feb-2007/15:20 Page: 172 job: prob .) 1.] and [2. We will assume that dn > 0 for all n and that ϕ (λ) = 0 for λ suﬃciently large.1. µ ˆa (λ) = µ ˆ a−1 λ . it suﬃces to prove the result for a function ϕ as pictured in Figure 14. 3. i. Problem 26. 104-107.4. we know that (1 − |λ|)+ = µ ˆ (λ) ϕ (λ. BR ) . dµ (x) := 1 1 − cos x dx.6). Fig. BR ) ∞ show there exists a sequence {νn }n=1 of ﬁnitely supported ﬁnite measures on (R.21 that exp R ϕ (λ. 1.32. and the continuity Theorem 14.2. Here is a piecewise linear convex function. on (R. Exercise 14. then f (λ) := exp R ϕ (λ. Show f (λ) is continuous.1. Λ ⊂ R such that ν (R \ Λ) = 0. p. Show ϕ (λ) = ν ˆ (λ) for some probability measure. Now suppose that ν is compactly supported. such that µ ({n}) = p (n) = c n2 ln |n| 1|n|≥2 with c chosen so that 1 p ( n ) = 1 . BR ) .3 on p. For the general case. Show that µ ˆ ∈ C ( R . Please interpret νn =⇒ ν to mean. 2. x) and in fact that ϕ (λ. 0) = limx→0 ϕ (λ.

31) it is now easily shown that dλ n=1 pn 1 − an + ϕ (λ) for λ ∈ / {a1 . if this is the case we ∞ may take. . we know that −sk ≤ −sk+1 or sk ≥ sk+1 for all k and therefore pk ≥ 0 and pk = 0 for all k > N. Proof. x∈Rn for some an > 0 and pn ≥ 0 such that n=1 pn . 2. Let S denote the space of functions f ∈ C ∞ (Rn ) such that f and all of its partial derivatives have rapid decay and let f N. sk = pn n=k 1 for all k an which then implies pk a1 k = sk − sk+1 or equivalently that pk = ak (sk − sk+1 ) . Moreover. there exists Nα < ∞ such sup (1 + |x|) −Nα pk = k=1 k=1 ak (sk − sk+1 ) = k=1 ∞ ak sk − k=2 ak−1 sk ∞ |∂ α g (x)| < ∞. . Suppose that χ : Rn → C is said to be positive deﬁnite with χ (0) = 1.29) + 14. C) is said to have (at most) polynomial growth if there exists N < ∞ such sup (1 + |x|) −N Notice that sn = 0 for all n > N. Indeed.α = sup (1 + |x|)N ∂ α f (x) x∈Rn we must require. ϕ (λ) = − n=k 1 pn when ak−1 < λ < ak an ∞ i. (14. It is pretty clear that we should take an = d1 + · · · + dn for all n ∈ N. C) is said to have rapid decay or rapid decrease if sup (1 + |x|)N |f (x)| < ∞ for N = 1.14. i. such that N 0 = ϕ (aN ) = 1 − n=1 dn sn . .38. . If χ is continuous at 0 then in fact χ is uniformly continuous on all of Rn .6 Appendix: Bochner’s Theorem Deﬁnition 14.35. g ∈ C ∞ (Rn ) is in P iﬀ for all multiindices α. (14. A function f ∈ C (Rn . ν := n=1 pn µan . Working backwards with pk ∞ λ d = deﬁned as in Eq. there exists N ∈ N and C < ∞ such that |f (x)| ≤ C (1 + |x|)N for all x ∈ Rn . (14. (14.36 (Schwartz Test Functions).37 we conclude that 1 χ (x − y ) χ (x) 1 χ (x − y ) χ (x) 1 χ (y ) = χ ¯ (x − y ) 1 χ (y ) A := χ (y − x) χ (−x) χ (−y ) 1 χ ¯ (x) χ ¯ (y ) 1 macro: svmonob. A function χ : Rn → C is said to be positive (semi) m deﬁnite iﬀ the matrices A := {χ(ξk − ξj )}k.37. Since ∞ |f (x)| < ∞. (Notice that any polynomial function on Rn is in P .cls date/time: 23-Feb-2007/15:20 = a1 s1 + k=2 ∞ sk (ak − ak−1 ) = d1 s1 + k=2 = k=1 sk dk = 1 where the last equality follows from Eq. A function f ∈ C (Rn .29) is indeed valid. Taking ξ1 = x.6 Appendix: Bochner’s Theorem ∞ ∞ 173 ϕ (λ) = n=1 pn µ ˆan (λ) = n=1 ∞ pn 1 − λ an (14. .e. Since ϕ is convex. .30) Equivalently. N ∈ N. Deﬁnition 14. ∞ ∞ ∞ ∞ Also let P denote those functions g ∈ C ∞ (Rn ) such that g and all of its derivatives have at most polynomial growth. there is a ﬁrst index.j =1 are positive deﬁnite for all m m ∈ N and {ξj }j =1 ⊂ Rn . Since we are assuming ϕ (λ) = 0 for large λ.31) so that S = f ∈ C ∞ (Rn ) : f N. } and since both functions are equal to 1 at λ = 0 we may conclude that Eq.) sk dk Deﬁnition 14. (14. ξ2 = y and ξ3 = 0 in Deﬁnition 14. . a2 . for each N ∈ N there exists constants CN < ∞ such that |f (x)| ≤ CN (1 + |x|)−N for all x ∈ Rn .α < ∞ for all N and α .e. Page: 173 job: prob . Proposition 14.30).

then for f ∈ S we would have f dµ = Rn Rn is positive deﬁnite from which we conclude χ(ξ − η ) = χ(η − ξ ) (since A = A∗ by deﬁnition) and Page: 174 job: prob macro: svmonob.32) showing A is positive deﬁnite.174 14 Characteristic Functions (Fourier Transform) is positive deﬁnite. C) is positive deﬁnite function. Taking m = 1 and ξ1 = 0 we learn χ(0) |λ| ≥ 0 for all λ ∈ C which proves item 1. and 3. then χ := µ ˆ ∈ C (Rn . C) is a positive deﬁnite function. Combining this inequality with the identity. ξ1 = ξ and ξ2 = η.40. χ(ξ − η )f (ξ )f (η )dξdη ≥ 0. Item 4. follows by approximating the integral in Eq. 0 ≤ det A = 1 + χ (x − y ) χ (y ) χ ¯ (x) + χ (x) χ ¯ (x − y ) χ ¯ (y ) − |χ (x)| − |χ (y )| − |χ (x − y )| .39. 4.η ∈(εZn )∩[−ε−1 . Lemma 14.j =1 m 2 −iξk ·x = = (14. C) is a positive deﬁnite function. χ(η − ξ ) χ(0) and hence |χ(ξ )| ≤ χ(0) for all ξ. Moreover if λ ∈ Cm . If χ ∈ C (Rn .41 (Bochner’s Theorem). In particular. (14. |χ (x) − χ (y )| = |χ (x)| + |χ (y )| − χ (x) χ ¯ (y ) − χ (y ) χ ¯ (x) .ε−1 ]n χ(ξ − η )f (ξ )f (η ) ≥ 0. C).32) by Riemann sums. For all f ∈ S(Rd ). If µ is a ﬁnite positive measure on BRn . χ(ξ − η )f (ξ )f (η )dξdη Rn ×Rn = lim ε−2n ε↓0 ξ. 2. µ ˆ(−ξ ) = Rn eiξ·x dµ(x) = Rn ˆ(−ξ ).j =1 is self-adjoint.j =1 m ¯ j dµ(x) e−i(ξk −ξj )·x λk λ e−iξk ·x λk e−iξj ·x λj dµ(x) Rn k. Rn k=1 e λk dµ(x) ≥ 0 Proof.j =1 Rn k. Since µ is a positive measure (and hence real). If χ(ξ ) = µ ˆ(ξ ). As has already been observed after Deﬁnition ??. m m ¯j = µ ˆ(ξk − ξj )λk λ k.cls (f ∨ ) dµ = Rn ˆ f ∨ (ξ )ˆ µ(ξ )dξ. 3. then 1. |χ(ξ )| ≤ χ(0) for all ξ ∈ Rn . Proof. χ(0) ≥ 0. then there exists a unique positive measure µ on BRn such that χ = µ ˆ. The details are left to the reader. χ(−ξ ) = χ(ξ ) for all ξ ∈ Rn . the dominated convergence theorem implies µ ˆ ∈ C (Rn . Lemma 14. the matrix A := m {µ ˆ(ξk − ξj )}k. Proof. Hence we have |χ (x) − χ (y )| ≤ 1 − |χ (x − y )| + 2 |χ (x − y ) − 1| = (1 − |χ (x − y )|) (1 + |χ (x − y )|) + 2 |χ (x − y ) − 1| ≤ 4 |1 − χ (x − y )| which completes the proof. Suppose χ ∈ C (Rn . Rn ×Rn 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 0 ≤ det χ(0) χ(ξ − η ) 2 2 = |χ(0)| − |χ(ξ − η )| . the matrix A := χ(0) χ(ξ − η ) χ(η − ξ ) χ(0) Theorem 14. Taking m = 2. date/time: 23-Feb-2007/15:20 . e−iξ·x dµ(x) = µ m From this it follows that for any m ∈ N and {ξj }j =1 ⊂ Rn . This proves items 2. gives 0 ≤ 1 − |χ (x − y )| + χ (x − y ) χ (y ) χ ¯ (x) + χ (x) χ ¯ (x − y ) χ ¯ (y ) − |χ (x) − χ (y )| + χ (x) χ ¯ (y ) + χ (y ) χ ¯ (x) = 1 − |χ (x − y )| − |χ (x) − χ (y )| + χ (x − y ) χ (y ) χ ¯ (x) − χ (y ) χ ¯ (x) + χ (x) χ ¯ (x − y ) χ ¯ (y ) − χ (x) χ ¯ (y ) = 1 − |χ (x − y )| − |χ (x) − χ (y )| + 2 Re ((χ (x − y ) − 1) χ (y ) χ ¯ (x)) ≤ 1 − |χ (x − y )| − |χ (x) − χ (y )| + 2 |χ (x − y ) − 1| .

pt (x − y )eiy·ξ dy = Rn ix·ξ ∨ e pt (ξ ) pt (y )ei(y+x)·ξ dy . ∞)) be a function such that ∞ ψ = 1 on K. Let K ⊂ R be a compact set and ψ ∈ Cc (R. If f ∈ Cc (R. (14.35) pt (x) := I (pt (x − ·)) = I ( pt (x − ·) ) pt (x − ·) ∈ S . f = µ(f ) for all f ∈ Cc (Rn . ψ = Rn I (pt (x − ·))ψ (x)dx χ(ξ ) [pt (x − ·)] (ξ )ψ (x)dξ dx Rn Rn ∨ Rn ξ−η )dξ. f | ≤ C (supp(f )) f ∞ We will now show I is positive in the sense if f ∈ S and f ≥ 0 then I (f ) ≥ 0. ψ . (14. R) still satisfying the estimates in Eq. Because of the estimate in Eq. ψ ≥ 0.36) = = Rn Rn Because Rn f ∨ (ξ )dξ = F f ∨ (0) = f (0) = 1. ψ and hence we have proved | I.33) and the fact that [pt (x − ·)] (ξ ) = = ∨ ∞ (Rn .35). R) is a smooth function with supp(f ) ⊂ K. ε (14. e−iηx fε (x)dµ(x) = =e Rn ix·ξ −t|ξ |2 /2 ξ−η ) ε e χ(ξ )ε−n f ∨ ( Rn It . (14. the monotone convergence theorem implies µ(fε ) ↑ µ(1) = µ(Rn ) and therefore µ(Rn ) = µ(1) = χ(0) < ∞. (14. (14.cls date/time: 23-Feb-2007/15:20 . 2 ∞ (Rn . when η = 0. we may apply the approximate δ – function Theorem ?? to Eq. ψ → Rn χ(ξ )ψ ∨ (ξ )dξ = I (ψ ) as t ↓ 0. f ≤ f ∞ I. To ﬁnish the proof we must show µ ˆ(η ) = χ(η ) for all η ∈ Rn given For t > 0 let pt (x) := t−n/2 e−|x| It (x) := I /2 t ∈ S and deﬁne 2 µ(f ) = Rn ∞ χ(ξ )f ∨ (ξ )dξ for all f ∈ Cc (Rn .34). Now knowing the µ is a ﬁnite measure we may use the dominated convergence theorem to concluded µ(e−iηx fε (x)) → µ(e−iηx ) = µ ˆ(η ) as ε ↓ 0 for all η. R+ ) be a radial function such f (0) = 1 and f (x) is decreasing Let f ∈ Cc as |x| increases. then 0 ≤ f ∞ ψ − f ∈ S and hence Page: 175 job: prob macro: svmonob. F −1 e−iηx fε (x) (ξ ) = ε−n f ∨ ( and therefore. then by Theorem ??. Combining this equation with Eq. Let fε (x) := f (εx).34) and moreover this extension is still positive.6 Appendix: Bochner’s Theorem 175 This suggests that we deﬁne I (f ) := Rn 0 ≤ I.14. Replacing f by −f implies. [0. So by the Riesz – Markov Theorem ??.37) = /2 dξ which coupled with the dominated convergence theorem shows I pt . R). it follows that I |DRn has a unique extension I to Cc (Rn . then I (ψ ) = limt↓0 It .36) to ﬁnd e−iηx fε (x)dµ(x) → χ(η ) as ε ↓ 0. Rn χ(ξ )eix·ξ e−t|ξ| χ(ξ )ψ ∨ (ξ )e−t|ξ| Rn 2 2 /2 ψ (x)dξ dx (14. Hence if ψ ≥ 0. ∞ ψ−f = f ∞ I. Using which is non-negative by Eq. there exists a unique Radon – measure µ on Rn such that such that I. R). f f ∞ I. f χ(ξ )f ∨ (ξ )dξ for all f ∈ S . (14. R) where C (K ) is a ﬁnite constant for each compact for all f ∈ DRn := Cc n subset of R . f ≤ and therefore I.37) shows µ ˆ(η ) = χ(η ) for all η ∈ Rn .33) = Rn χ(ξ − η )f ∨ (ξ )f ∨ (η )dη dξ ≥ 0. − I. On the the other hand. from Eq. For general f ∈ S we have I (|f | ) = Rn 2 (14.34) χ(ξ ) |f | 2 ∨ (ξ )dξ = Rn χ(ξ ) f ∨ ¯∨ (ξ )dξ f = Rn ¯∨ (η )dη dξ = χ(ξ )f ∨ (ξ − η )f Rn χ(ξ )f ∨ (ξ − η )f ∨ (−η )dη dξ (14. (14. ψ − I.

|f (x) − pn (x)| = E f (x) − f ( Sn Sn ) ≤ E f (x) − f ( ) n n Sn ≤E f (x) − f ( ) : Sn − x > ε n Sn + E f (x) − f ( ) : Sn − x ≤ ε n (14.d. . f S → f (x) as n → ∞. 1} . there exists polynomials pn on Rd such that pn → f uniformly on K.i. . . .43 (Complex Weierstrass Approximation Theorem).23. Suppose that K ⊂ Cd ∼ = Rd × Rd is a compact rectangle. R). . Then for every f ∈ C (K. bd ] with −∞ < ai < bi < ∞ is a compact rectangle in Rd .7 Appendix: A Multi-dimensional Weirstrass Approximation Theorem The following theorem is the multi-dimensional generalization of Theorem 4. Then there exists polynomials in (z = x + iy. we know that j EXn = x and Var Xn = xj − x2 j = xj (1 − xj ). [ad . z ¯) − f (z )| → 0 as n → ∞ for every f ∈ C (K. it suﬃces to assume f ∈ C ([0. Theorem 14. and δε = sup {|f (y ) − f (x)| : x.38) Here is a version of the complex Weirstrass approximation theorem. let Xn = Xn n=1 values in Rd such that d where pn (x) = ε(·)∈{0. εd ) ∈ {0. This along with the n dominated convergence theorem shows pn (x) := E f Sn n → f (x) as n → ∞. (14. .cls date/time: 23-Feb-2007/15:20 . C) . Since each Xn j with P Xn = 1 = xj . Therefore. By uniform continuity of f on K. 1]. . By a simple scaling and translation of the arguments of f we may d assume with out loss of generality that K = [0. limε↓0 δε = 0. By considering the real and imaginary parts of f separately. z ¯ = x − iy ) . Proof. (14. random vectors with . . 1] . Sn /n → x P n in and by a continuity theorem. . . C). ≤2M P ( Sn − x > ε) + δε . Xn = ε (n)) n d = ε(·)∈{0. b1 ] × . . Suppose that K = [a1 . y ∈ K and y − x ≤ ε} . In fact more is true. By Chebyshev’s inequality. .1}d f (1 − xi ) k=1 i=1 1−εi (k) xi i ε (k ) is a polynomial of degree nd. 4nε2 Sn −x n = j =1 d E j Sn − xj n j Sn n 2 d = j =1 d Var n j Sn − xj n and therefore. ∞ d 1 be i. n→∞ x∈K 1 = n d xj (1 − xj ) ≤ .39) yields the estimate sup |f (x) − pn (x)| ≤ x∈K = j =1 Var d 1 = 2· n j =1 Var k=1 j Xk 2dM + δε nε2 and hence lim sup sup |f (x) − pn (x)| ≤ δε → 0 as ε ↓ 0. then E E Sn = x and n 2 d = d . Xn Given x ∈ K.1}d f ε (1) + · · · + ε (n) n ε (1) + · · · + ε (n) n P (X1 = ε (1) . M = sup {|f (x)| : x ∈ K } . Page: 176 job: prob macro: svmonob. z ¯) for z ∈ Cd . . 4 n j =1 P This shows Sn /n → x in L2 (P ) and hence by Chebyshev’s inequality. .39) P (Xn = ε) = i=1 d (1 − xi ) 1−εi i xε i j is a Bernoulli random variable for all ε = (ε1 .176 14 Characteristic Functions (Fourier Transform) 14. Theorem 14. Eq.42 (Weierstrass Approximation Theorem). P ( Sn − x > ε) ≤ 1 E Sn − x ε2 2 As usual let Sn = Sn := X1 + · · · + Xn ∈ Rd . pn (z. such that supz∈K |qn (z. Suppose ε > 0 is given.

Hence the result now follows from Theorem 14. x − iy ). b) with b − a < 2π.42) such that |F (z ) − p(z. z ¯) = am. f : S 1 → C such that f eix1 . Hint: Use Exercise 14. Solution to Exercise (14.41) implies 2π 2π 2π 0 = lim ε↓0 0 f (θ) pε (θ) dθ = 0 ¯ (θ) dθ = f (θ) f 0 |f (θ)| dθ. if f ∈ C k (R) and z. By applying Theorem 14. φ ˜ := eiθ : θ ∈ J ⊂ S 1 i. Taylor’s theorem with integral remainder states. Since z ¯ = z −1 on S 1 . deﬁne F (z ) := f (eiθ ) where θ ∈ R is chosen so that z = eiθ .43) date/time: 23-Feb-2007/15:20 . Let K = S = {z ∈ C : |z | = 1} and A be the set of polynomials in (z. Hence for any ε > 0 there exists p(z.14.41) ¯ (θ) Choose trigonometric polynomials. 1 ˜ F is continuous when restricted to J. .n z m−n d From this it follows that f ≡ 0. . 1.e.e. ∆) n! ∆n 1 (k ) + ∆k f (z ) + ε (z.7.n z m z ¯n = am. (14. . . ∆) n! k! (14. for any J = (a. z ¯)| ≤ ε for all z. the map θ ∈ J → J ˜ This shows is a homeomorphism. Suppose f ∈ C (R. eixd = F (x) for all x = (x1 .40) satisﬁes ¯(θ) − p (θ) ≤ ε. sup f θ Exercise 14. . It would then follow that 2π 2 |f (θ)| dθ > 0. pε . Since such sets cover S .8).7). Passing to the limit in Eq. Then A is dense in C (S 1 ). By assumption. Letting z ¯ = x − iy as usual. for if |f (θ0 )| > 0 for some θ0 then |f (θ)| ≥ ε > 0 for θ in a neighborhood of θ0 by continuity of f. C) is a 2π – periodic function (i. Example 14. Use Example 14. . y ) on R × R may be written as a polynomial q in (z. namely ¯ z+z ¯ z−z . 2π 2π 0 f (θ) einθ dθ = 0 for all n 0= 0 f (θ) pε (θ) dθ. we have shown polynomials in z and z −1 are dense in C (S 1 ).8. F is well deﬁned since if θ solves eiθ = z then all other solutions are of the form {θ + 2πn : n ∈ Z} . d d Therefore under this identiﬁcation any polynomial p(x. it follows that F is continuous. (14. z ¯) converging uniformly to F on K and hence on S 1 .43 to F restricted to a compact rectangle containing S 1 we may ﬁnd qn (z. z ¯).44.44 that polynomials in z and z −1 are dense in C (S 1 ). Solution to Exercise (14. y ) = q (x + iy. (14. 1] . Taking z = eiθ then implies there exists bn ∈ C and N ∈ N such that Page: 177 job: prob macro: svmonob. ∆ ∈ R or f be holomorphic in a neighborhood of z ∈ C and ∆ ∈ C be suﬃciently small so that f (z + t∆) is deﬁned for t ∈ [0. z ¯) may be thought of as a polynomial p in (x. then k−1 f (z + ∆) = n=0 k−1 f (n) (z ) f (n) (z ) n=0 ∆n + ∆k rk (z.42. z ¯) = p( 2 2i Conversely a polynomial q in (z. y ). we have x = z+¯ 2 and y = 2i . . g : Rd → C.8 Appendix: Some Calculus Estimates We end this section by gathering together a number of calculus estimates that we will need in the future. 0 show again that f ≡ 0. namely p(x. it follows that F (z ) = f ◦ φ−1 (z ) for z ∈ J. ). The mapping (x.40) such that pε (θ) → f uniformly in θ as ε ↓ 0. Hint: start by d showing there exists a unique continuous function. I will write out the solution when d = 1.44 to show that any 2π – periodic continuous function. y ) ∈ Rd × Rd → z = x + iy ∈ Cd is an isomorphism z z −z ¯ of vector spaces. q (z. For z ∈ S 1 . and so by the linearity of the Riemann integral. 2 where Λ is a ﬁnite subset of Z and aλ ∈ C for all λ ∈ Λ. To prove this ﬁrst observe if f ∈ C S 1 then F (z ) = |z | f ( |z z | ) for z = 0 and F (0) = 0 deﬁnes F ∈ C (C) 1 such that F |S = f. xd ) ∈ Rd . Since the map θ → eiθ is a local homeomorphism.7. Since f is 2π – periodic.n z m z −n = am. 0 14. may be uniformly approximated by a trigonometric polynomial of the form p (x) = aλ eiλ·x λ∈Λ 1 N pε (θ) := n=−N bn einθ (14.cls = (14. Exercise 14. f (x + 2π ) = f (x) for all x ∈ R) and 2π f (x) einx dx = 0 for all n ∈ Z. as in Eq. . This example generalizes in an obvious way to d K = S 1 ⊂ Cd . It now follows from Example 14.8 Appendix: Some Calculus Estimates 177 Proof. z ¯) restricted to S 1 .

2 d d ln z = eln z ln z = 1. (14. and 1 ε (z. dz dz Alternatively. ∆) . For y ∈ R.178 14 Characteristic Functions (Fourier Transform) where rk (z.45) 4. k! i. ∆) = 1 k! 1 f (k) (z + t∆) − 0 d dt (1 − t) dt t=1 k 1 k =− f (k) (z + t∆) (1 − t) k! 1 = f (k) (z ) + ∆rk+1 (z. 2 2 Proof.47) |ln (1 + z ) − z | ≤ |z | 2 1 2 (1 − |z |) 2 for |z | < 1. (14. ∆) k! 1 1 g (y ) := cos y − 1 + y 2 /2 ≥ 0 for all y ∈ R. ∆) . Then ln : C \ (−∞.52) cos y = 1 + y 2 0 − cos (ty ) (1 − t) dt ≥ 1 + y 2 0 2 − (1 − t) dt = 1 − y . 2 2 (14. 3. Since eiy − 1 = iy k |ez − 1 − z | ≤ e0∨Re z · 1 ity e dt. 0] → C is a holomorphic function such that eln z = z 3 and if |z | < 1 then (14. ∆) = (k − 1)! f 0 (k ) if Re z ≤ 0.51) eiy − 1 ≤ 2 ∧ |y | for all y ∈ R. For y ∈ R we have 1 1 Lemma 14.46) and if Re z > 0 then |ez − 1 − z | ≤ eRe z |z | /2. 1 (k ) f (z ) ∆k + ∆k+1 rk+1 (z. Since 1 (14. use integration by parts to show. (14. 1 2. let ln z = ln r + iθ. Clearly eln z = z and ln z is continuous. 2 2 (14.49) To prove this. sin y = y 0 cos (ty ) dt and hence |sin y | ≤ |y | .50) eiy − 1 ≤ |y | and hence (14. k! The result now follows by induction. Therefore by the inverse function theorem for holomorphic functions.48) |ez − 1 − z | = z 2 0 etz (1 − t) dt ≤ |z | 2 0 1 et Re z (1 − t) dt.45.44) (14. ∆) = ∆ + k! t=0 1 0 f (k+1) (z + t∆) (1 − t) dt 5. 0 |z | .e. then (z + t∆) − f (k ) (z ) (1 − t) k−1 |ez − 1 − z | ≤ |z | /2 dt → 0 as ∆ → 0. ∆k rk (z. ∆) = 1 k−1 f (k) (z + t∆) (1 − t) dt (k − 1)! 0 1 = f (k) (z ) + ε (z. For z = reiθ with −π < θ < π and r > 0. rk (z. . Combining these into one estimate gives. ln z is holomorphic and z 3 Equivalently put .

.

.

Z y .

Z y .

.

.

.

.

.

.

|cos x| dx.

cos xdx.

≤ .

|sin y | =

≤ |y | 0 0

and for y ≥ 0 we have, cos y − 1 =

0

For the purposes of this lemma it suﬃces to deﬁne ln (1 + z ) = − and to then observe: 1)

P∞

n=1

(−z )n /n

Z

y

Z

− sin xdx ≥

0

X d 1 ln (1 + z ) = (−z )n = , dz 1 + z n=0

∞

y

−xdx = −y 2 /2.

and 2) the functions 1 + z and eln(1+z) both solve f (z ) = and therefore eln(1+z) = 1 + z. 1 f (z ) with f (0) = 1 1+z

This last inequality may also be proved as a simple calculus exercise following from; g (±∞) = ∞ and g (y ) = 0 iﬀ sin y = y which happens iﬀ y = 0.

Page: 178

job: prob

macro: svmonob.cls

date/time: 23-Feb-2007/15:20

14.8 Appendix: Some Calculus Estimates

179

Therefore,

d dz

ln z =

1 z

and

d dz 2

2

**ln z = − z12 . So by Taylor’s theorem,
**

1

**and using Eq. (14.55) with n = 2 implies (14.53) eiy − 1 + iy − y2 2! ≤ |y | . 3!
**

3

ln (1 + z ) = z − z 2

0

1 (1 + tz )

2

(1 − t) dt.

**If t ≥ 0 and |z | < 1, then 1 1 1 n ≤ |tz | = ≤ . (1 + tz ) 1 − t |z | 1 − |z | n=0 and therefore,
**

1 0 ∞

Combining the last two inequalities completes the proof of Eq. (14.56). Equation (14.57) is proved similarly and hence will be omitted. Lemma 14.47. If X is a square integrable random variable, then f (λ) := E eiλX = 1 + iλEX − λ2 E X 2 + r (λ) 2!

1 (1 + tz )

2

(1 − t) dt ≤

1

2 (1 − |z |)

2.

(14.54) where r (λ) := λ2 E X 2 ∧ and (14.55) ε (λ) := E X 2 ∧ |λ| |X | 3!

3 3

**Eq. (14.52) is now a consequence of Eq. (14.53) and Eq. (14.54). Lemma 14.46. For all y ∈ R and n ∈ N∪ {0} ,
**

n

= λ2 ε (λ)

eiy −

k=0

(iy ) k!

k

≤

|y | (n + 1)!

n+1

|λ| |X | 3!

→ 0 as λ → 0.

(14.58)

**and in particular, y2 eiy − 1 + iy − 2! More generally for all n ∈ N we have
**

n

**Proof. Using Eq. (14.56) with y = λX and taking expectations implies, |y | ≤ y2 ∧ . 3!
**

3

(14.56)

f (λ) − 1 + iλEX −

λ2 E X2 2!

≤ E eiλX − 1 + iλX − λ2 ≤ λ2 E X 2 ∧ |λ| |X | 3!

3

X2 2!

eiy −

k=0

(iy ) k!

k

|y | 2 |y | ∧ . ≤ (n + 1)! n!

iy

n+1

n

=: λ2 ε (λ) .

(14.57)

**Proof. By Taylor’s theorem (see Eq. (14.42) with f (y ) = e , x = 0 and ∆ = y ) we have
**

n

The DCT, with X 2 ∈ L1 (P ) being the dominating function, allows us to conclude that limε→0 ε (λ) = 0.

eiy −

k=0

(iy ) k!

k

= ≤

y n+1 n! |y | n!

n+1

1 0 1 0

in+1 eity (1 − t) dt (1 − t) dt =

n

n

|y | (n + 1)!

n+1

which is Eq. (14.55). Using Eq. (14.55) with n = 1 implies eiy − 1 + iy − y2 2! ≤ eiy − (1 + iy ) + ≤ y2 y2 + = y2 2 2 y2 2

Page: 179

job: prob

macro: svmonob.cls

date/time: 23-Feb-2007/15:20

.

k | > ε) = 0.d.k < ∞.k . If {Xn }n=1 are i. In order to do this. 1.k } satisﬁes condition (M ) if 2 Dn := max σn.k t α−2 : |Xn.1) =E 2 Xn.1. For each n ∈ N.k .k (15. Deﬁnition 15. then {Xn.i.k |) = 0 k=1 where ϕ : [0.k : |Xn.k and we say {Xn. = 1. If we let s2 n := k=1 Var (Xk ) = k=1 σk . (15.i.k | > t = 2 sn = = n 2 E Xk : |Xk | > sn t k=1 n 2 E X1 : |X1 | > k=1 (15. then {Xn. Lemma 15.3.k | ] = k=1 1 tα−2 sα n n E |Xk | → 0. The reader should observe that in order for condition (M ) to hold in the setup in Example 15.k }k=1 be independent random variables and let n Remark 15.k (15.k := σk /sn . Suppose are mean zero square integrable random varin n 2 2 ables with σk = Var (Xn ) .4. k=1 α . ∞) → [0.k 1 : |Xn.1) in the limit as n → ∞. tends to zero as n → ∞. let {Xn.d.6).6) lim n→∞ sα n More generally. and Xn. n Our main interest in this chapter is to consider the limiting behavior of Sn as n → ∞. (15. it will be useful to put conditions on the {Xn.k := Xk /sn .3) √ 1 2 E X1 : |X1 | > nσt 2 σ We say {Xn.k | > t ≤ k=1 k=1 n 2 E Xn.5) n→∞ k≤n Xn.k } satisﬁes the Lindeberg Condition (LC) iﬀ n n→∞ ∞ {Xn }n=1 lim 2 E Xn. Example 15. by DCT. (15.k } such that no one term dominates sum deﬁning the sum deﬁning Sn in Eq.k : |Xn. 2. Then {Xn.k ϕ (|Xn. We say that {Xn. n 2 2 2 σn.2.4) n 2 E Xn. lim max P (|Xn. and Var (Sn ) = n k=1 2 σn. The {Xn }n=1 satisfy Liapunov condition. 1.k | > t = 0 for all t > 0.1 it is necessary that limn→∞ s2 n = ∞. then sn = nσ where σ 2 = EX1 n E k=1 2 Xn. {Xn }n=1 are i. then for any t > 0.k : k ≤ n → 0 as n → ∞.1.7) nσt 1 nσ 2 √ lim 2 E Xn.k }k=1 satisfy the above hypothesis n and Sn = s1 k=1 Xk .k ] = 0. if {Xn. √ ∞ 2 and Proof.2) denote the characteristic function of Xn. (15.k } is uniformly asymptotic negligibility (UAN) if for all ε > 0.k | > t α ≤ 1 tα−2 n E [|Xn. we will assume the following standing notation n unless otherwise stated. Assuming Eq. there exists some α > 2 such that n α k=1 E |Xk | = 0.k } satisﬁes the Liapunov condition.k := Xk /sn } satisﬁes (LC ) if either of two conditions hold. 2 σn. Let us continue the notation in Example 15. k=1 (15.k Until further notice we are going to assume E [Xn. ∞) is a non-decreasing function such that ϕ (t) > 0 for all t > 0. (15. which.k } satisﬁes (LC ) .15 Weak Convergence of Random Sums Throughout this chapter.. n n→∞ ∞ Sn := k=1 Xn. ∞ 2. Also let fnk (λ) := E eiλXn.

m 1|Xn. Suppose {Xn.k |≤t + E Xn.182 15 Weak Convergence of Random Sums For the last assertion. k=1 2 The question then becomes under what conditions are these approximations valid.cls . Suppose that ai .6.8) (See Theorem 15. namely that n n We will need the following lemma for our subsequent applications of the continuity theorem.k : |Xn.k 1|Xn. (15.k | > ε → 0.m |>t /2 as n → ∞.11) below that the approximation (B ) is valid.k |) : |Xn. 2 2 2 2 1|Xn.e.k n ∞ To prove this theorem we must show E eiλSn → e−λ 2 ≤t +E 2 2 Xn.k = E Xn.10) is valid if condition (M ) holds. P (|Xn. |b| ≤ 1 and ϕ (|Xn. . then (LC ) =⇒ (M ) =⇒ (U AN ) .9) and therefore using (LC ) we ﬁnd n→∞ k≤n 2 lim max σn.k | > t ≤ k=1 k=1 n 2 E Xn.7 (Lindeberg-Feller CLT (I)). Then n n n n→∞ lim fnk (λ) − exp k=1 k=1 (fnk (λ) − 1) = 0. . Proof. . Theorem 15. (15. bi ∈ C with |ai | . Let a := that n−1 i=1 ai and b := n−1 i=1 bi and observe that |a| . Lemma 15.k | > ε) ≤ k=1 1 ε2 n E |Xn. (15.k } satisﬁes (LC ) .k ≤ t2 for all t > 0. Let {Xn.k | > t ϕ (t) ≤ as n → ∞. lim k=1 1 (fnk (λ) − 1) = − λ2 . It is shown in the estimate Eq. n n→∞ ai − i=1 i=1 bi ≤ i=1 |ai − bi | . 1) .9).k | > ε ε2 E |Xn. let me give an informal explanation for Eq.k = E Xn.k |>t ≤t + m=1 2 E 2 Xn.) Lemma 15. n. .k |>t σn.5. 2 we might expect n 1 2 E |Xn. |bi | ≤ 1 for i = 1. This clearly implies (M ) holds. The proof is now easily completed by induction on n.k | : |Xn.11 for a converse to this theorem. The assertion that (M ) implies (U AN ) follows by Chebyschev’s inequality.k ϕ (|Xn. max P (|Xn. For k ≤ n. It turns out that approximation (A). i.9 below. see Lemma 15. (15.k }n=1 be as above. Using λ2 2 fnk (λ) ∼ 1 − σnk .k | > ε → 0. (15.k 1|Xn. 2 E eiλSn = k=1 Pn fnk (λ) = e k=1 ln fnk (λ) ≤ 1 ε2 n Pn = e k=1 ln(1+fnk (λ)−1) ( A) k=1 Pn ∼ e k=1 (fnk (λ)−1) n = k=1 e(fnk (λ)−1) In fact the same argument shows that (M ) implies n (B ) Pn λ2 2 λ2 ∼ e k=1 − 2 σnk = e− 2 .k | : |Xn. working as above we have n 2 E Xn.k |) → 0 k=1 |an a − bn b| ≤ |an a − an b| + |an b − bn b| = |an | |a − b| + |an − bn | |b| ≤ |a − b| + |an − bn | .k Proof. 2. 2 date/time: 23-Feb-2007/15:20 Page: 182 job: prob macro: svmonob. then Sn =⇒ N (0.k | : |Xn. 1 ϕ (t) n 2 E Xn.k | > ε) ≤ max k≤n k≤n Before starting the formal proof.

|e−u − 1 + u| ≤ u /2 valid for u ≥ 0 (see Eq.k + Bn. Now on to the formal proof of Theorem 15.k 2 σn. and E Xn 1|Xn |≤c converges.k k=1 E e where iλSn −e −λ2 /2 ≤ k=1 fnk (λ) − e = k=1 (An.14.k An.k | > ε 3! 3! 3 3 3 ≤ λ2 E |λ| |Xn. So by the central limit Theorem 15.k ∧ |λ| |Xn. using Lemma 14. n To estimate k=1 Bn.k ≤ lim sup n→∞ λ3 ε + λ2 6 n 2 E Xn.k /2 2 1 λ2 σn. is almost surely convergent.k /2 .k 2 λ2 σn.k /2 .k : |Xn.k ∧ : |Xn.k = 0. it follows that macro: svmonob. we use the estimate.k = k=1 k=1 n 1− 2 λ2 σn.k := fnk (λ) − 1 − Bn.6 to conclude. it n 2 follows that k=1 E Yk 1|Yk |>sn t = 0 for all suﬃciently large n and hence 1 n→∞ s2 n lim ∞ n 2 E Yk 1|Yk |>sn t = 0. and hence for every c > 0.12). The proof given below of Theorem 15.}) = 0. P ({|Xn | ≥ c i. Accord∞ ing the Borel zero one law this implies for every c > 0 that n=1 P (|Xn | > c) < c ∞. 2. If {Xn }n=1 are ∞ independent random variables and the random series.k + λ2 E Xn.k | ≤ ε + λ2 E Xn. 2 λ2 2 − 1 + Xn. {Yn.1 and Remark 15. ∞ 2 ≤ λ2 E Xn.k : |Xn.47.k λ2 2 − 1 + Xn.k = E e iλXn.k | > ε .k := Yk /sn }n=1 satisﬁes (LC ) – see Examples 15.s. With this estimate we ﬁnd.k : |Xn. 14.12.k 2 |λ| |Xn. Var Xn 1|Xn |≤c < ∞. In particular n=1 Xn is almost surely convergent for all c > 0. ∞ n=1 ∞ n=1 ∞ n=1 ∞ 1− 2 λ2 σn. k=1 lim sup n→∞ k=1 An.s. As an application of Theorem 15.k := Now.k 2 and wherein we have used (M ) (which is implied by (LC )) in taking the limit as n → ∞. 1.cls date/time: 23-Feb-2007/15:20 Page: 183 job: prob . n=1 Xn . However. Since n n and since ε > 0 is arbitrary. Theorem 15. then for all c > 0 the following three series converge.k | 2 : |Xn.3.e.k ∧ 2 ≤ λ2 E Xn.7 will not quite follow this route and will not use Lemma 15.k | ≤ ε + λ2 E Xn. c c Fix c > 0.k | 3! 3 ≤E e iλXn. ≤ we may use Lemma 15.k 2 2 − e−λ σn.k : |Xn.7.o.k | 2 : |Xn.49 with z = −u).k . this lemma will be used in the proofs of Theorems 15. let Yn := Xn − E [Xn ] and let n n n c Var (Xk )= k=1 k=1 λ2 2 2 ≤ |λ| ε · E |Xn.11 and 15. For the sake of contradictions.k = k=1 λ4 max σ 2 → 0.k 2 2 n = λ4 8 n 4 σn.11) i.7.k | |λ| |Xn.9 directly. suppose s2 n → ∞ as n → ∞.7 we can give half of the proof of Theorem 12. {Xn } and Xn := Xn 1|Xn |≤c are tail equivalent for all ∞ c c > 0.7.k | > ε 3! Proof.k | ≤ ε + λ2 E Xn..k 2 2 − e−λ σn.8 (Converse assertion in Theorem 12. An. 8 k≤n n.k | > ε 3! 3 |λ| ε 2 2 σn.k | : |Xn. Since Xn → 0 a.15 Weak Convergence of Random Sums 183 if (LC ) is satisﬁed. we may conclude that lim supn→∞ An. it follows that limn→∞ Xn = 0 a. n n k=1 4 2 −λ2 σn.k | > ε k=1 = λ3 ε 6 (15. 3.k 2 P (|Xn | > c) < ∞. Proof. Since |Yk | ≤ 2c.k /2 2 2 E eiλSn = k=1 fnk (λ) and e−λ 2 /2 = k=1 e−λ 2 2 σn. Since n=1 Xn is almost surely convergent. n n n k=1 2 Bn.k ) ≤ λ max σ 2 8 k≤n n. These observations would then constitute a proof of Theorem 15. = 6 From this estimate and (LC ) it follows that n s2 n = Var (Y1 + · · · + Yn ) = k=1 Var (Yk ) = Var Xk 1|Xk |≤c .

k 1 2 |λXn. n.184 15 Weak Convergence of Random Sums 1 s2 n n c c (Xn − E [Xn ]) = k=1 1 s2 n n Yk =⇒ N (0.k (λ) eϕn. Let us now turn to the converse of Theorem 15.k (λ) − k=1 k=1 e ϕn.k (λ)| · 2 k≤n n |ϕn. limn→∞ s2 n ∞ k=1 and so by Slutsky’s theorem. non-random) variables. the estimate in Eq. Page: 184 job: prob macro: svmonob. cn := n 1 c k=1 E [Xn ] . see Theorem 15.k 2 2 E | X | + | λ | ε = + |λ| ε. Dn 2 maxk≤n σn. fSn (λ) − k=1 eϕn. limn→∞ maxk≤n |ϕn.k → 0.k = 0.k | 2 ≤ λ2 2 n 2 σn. to converge to a non-degenerate limit.k (λ) − 1 ≤ 0 and hence eϕn. (Think about this eis2 n ther in terms of characteristic functions or in terms of distribution functions.k (λ) = Re fn. i. (14.k − 1 .k (λ) ≤ k=1 n fn.k (λ)| = k=1 k=1 n E eiλXn.k − 1 − iλXn.cls date/time: 23-Feb-2007/15:20 .k (λ) − 1| ≤ 2 2σn.11) implies that ∞ c c (Xn − E [Xn ]) is convergent a. EeiλX − 1 ≤ E eiλX − 1 ≤ E [2 ∧ |λX |] = E [2 ∧ |λX | : |X | ≥ ε] + E [2 ∧ |λX | : |X | < ε] 2 2 ≤ 2P [|X | ≥ ε] + |λ| ε ≤ 2 E |X | + |λ| ε ε Replacing X by Xn.49) implies n n |ϕn.) Thus we must conclude that ∞ ∞ 2Dn + |λ| ε = |λ| ε → 0 as ε ↓ 0.k (λ) − 1 = E eiλXn. observe that Re ϕn. 1) . n=1 c Since we already know that n=1 Xn is convergent almost surely we may now ∞ conclude n=1 E Xn 1|Xn |≤c is convergent. ε2 Var Xn 1|Xn |≤c = n=1 n=1 c Var (Xn ) = lim s2 n < ∞.k = k=1 ≤ fn.k ε2 ε2 On the other hand we know 1 n→∞ s2 n lim n c Xn = k=1 c Xk = 0 a.k (λ) ≤ e0 = 1 and hence we have from Lemma 15. ∞ fn.7.k (λ)| k=1 2 Lemma 15.k (λ) .11 below.k (λ)| ≤ lim sup n→∞ k≤n n→∞ But it is not possible for constant (i.k (λ)| = |fn. then. 1.e. k=1 E fSn (λ) = E eiλSn = k=1 λ2 . For the ﬁrst item we estimate.k (λ) k=1 n = ≤ ≤ 1 2 |ϕn. 1 s2 n n c E [Xn ]= k=1 1 s2 n n c Xn − k=1 1 s2 n n Yk =⇒ N (0.e. 2 Thus we have shown.k and in the above inequality shows |ϕn. where n := 1 max |ϕn.49).k (λ)| = 0 and n 2.s. ϕn.k } satisﬁes property (M ) . lim sup max |ϕn.k (λ) − eϕn.9.k (λ) = eRe ϕn. k=1 Proof. n n n An application of Kolmogorov’s convergence criteria (Theorem 12.k (λ)| . n→∞ For the second item. k=1 Moreover since EXn. If we deﬁne. k=1 Therefore. 1) .6 and the estimate (14. Suppose that {Xn.s.k (λ) → 0 as n → ∞.k (λ) − 1 − ϕn.k (λ) := fn.

Theorem 15.k 2 Proof.k (λ) = e−λ k=1 2 /2 .1 Inﬁnitely Divisible and Stable Symmetric Distributions n n 185 fn.k = 0 and lim n→∞ lim 2 E Xn.k (λ) ≤ λ2 max |ϕn. Further let f (λ) := E eiλX and u (λ) := Re (f (λ) − 1) .k (λ) = e−λ /2 n→∞ (15.k } satisﬁes (LC ) . n→∞ λ λ 2 E X2 ≥ E X2 − 2 : |X | > c 2 2 c 2 2 Pn Pn 2 lim e k=1 Re ϕn.11 (Lindeberg-Feller CLT (II)).k = 1) = pn.9 implies.10.k }k=1 be independent Bernoulli random variables with P (Xn. let n {Xn. limn→∞ k=1 pn. Proof. By assumption we have n n→∞ k≤n 2 lim max σn. Suppose. 1.6 of Durrett[2] for more on this theorem. limn→∞ max1≤k≤n pn. Theorem 15.k (λ) = e−λ k=1 2 /2 .k } satisﬁes (M ) and also the central limit theorem in Eq. Page: 185 job: prob macro: svmonob.k (λ) − k=1 k=1 eϕn.13) n→∞ lim Re ϕn. For each n ∈ N. Lemma 15. Then for all c > 0.k and P (Xn.14) n→∞ lim E cos (λXn. (See Section 2.8) holds.k ) − 1 + k=1 λ2 2 X =0 2 n.12).k (λ) = lim n and the latter expression tends to zero by item 1. 2 2 c √ In particular if we choose |λ| ≥ 6/ |c| .) n n fn.48)) cos λX − 1 + λ 2 X ≥ 0 and cos λX − 1 ≥ −2.k | > c = 0 k=1 for all c > 0 which is (LC ) .12) from which we may conclude n 2 2 (15.k (λ) = lim e k=1 ϕn.k = a ∈ (0.k . Let X be a random variable such that EX 2 < ∞ and EX = 0. 2 c 2 n→∞ n→∞ eϕn. we consider the following Poisson limit theorem.10 implies n n→∞ λ2 λ2 u (λ) + E X 2 = E cos λX − 1 + X 2 2 2 λ2 ≥ E cos λX − 1 + X 2 : |X | > c 2 λ2 2 ≥ E −2 + X : |X | > c 2 ≥ E −2 which gives Eq. Sn converges to a normal random variable iﬀ (LC ) holds.k (λ)| 4 k≤n The second inequality combined with Lemma 15.k = 0.k = 0) = qn. (14. which by Lemma 15. Suppose {Xn. (15.k := 1 − pn.cls date/time: 23-Feb-2007/15:20 .) Then Sn = k=1 Xn. k=1 We may write this last limit as n (15. we have (see Eq. So under condition (M ) . u (λ) + or equivalently λ λ 2 E cos λX − 1 + X 2 ≥ E X 2 − 2 : |X | > c .k : |Xn.15.1 Inﬁnitely Divisible and Stable Symmetric Distributions To get some indication as to what we might expect to happen when the Lindeberg condition is relaxed.k (λ) = −λ2 /2. then {Xn. Taking the modulus of this equation then implies. (So no one term is dominating the sums in item 1.k =⇒ Z where Z is a Poisson random variable with mean a. then λ2 1 E cos λX − 1 + X 2 ≥ 2 E X 2 : |X | > c . |X | λ2 2 + X : |X | > c c2 2 2 15.12 (A Poisson Limit Theorem). (15. Pn lim e k=1 ϕn. For all λ ∈ R. Therefore. ∞) and 2.

186

15 Weak Convergence of Random Sums

**Proof. Recall from Example 14.14 that for any a > 0, E eiλZ = exp a eiλ − 1 Since E e it follows that E e
**

iλSn iλXn,k iλ

n

n

exp pn,k eiλ − 1

k=1

−

k=1

1 + pn,k eiλ − 1

n

.

iλ

≤ = e pn,k + (1 − pn,k ) = 1 + pn,k e

n iλ

−1 ,

1 2

n

|zn,k | ≤

k=1 n

2

1 max |zn,k | 2 1≤k≤n pn,k .

|zn,k |

k=1

≤ 2 max pn,k

1≤k≤n

=

k=1

1 + pn,k e

−1

.

k=1

**Using the assumptions, we may conclude
**

n n

**Since 1 + pn,k eiλ − 1 lies on the line segment joining 1 to eiλ , it follows that 1 + pn,k eiλ − 1 ≤ 1. Since
**

n

exp pn,k eiλ − 1

k=1

−

k=1

1 + pn,k eiλ − 1

→ 0 as n → ∞.

n

exp pn,k eiλ − 1

k=1

= exp

k=1

pn,k eiλ − 1

→ exp a eiλ − 1

,

**we have shown
**

n n→∞

lim E eiλSn = lim = lim

n→∞

1 + pn,k eiλ − 1

k=1 n

n→∞

exp pn,k eiλ − 1

k=1

= exp a eiλ − 1

.

**The result now follows by an application of the continuity Theorem 14.21. Hence we may apply Lemma 15.6 to ﬁnd
**

n n

**Remark 15.13. Keeping the notation in Theorem 15.12, we have 1 + pn,k eiλ − 1 E [Xn,k ] = pn,k and Var (Xn,k ) = pn,k (1 − pn,k ) and
**

n n

exp pn,k eiλ − 1

k=1 n

−

k=1

≤

k=1 n

exp pn,k eiλ − 1

− 1 + pn,k eiλ − 1

s2 n :=

k=1

Var (Xn,k ) =

k=1

pn,k (1 − pn,k ) .

=

k=1

|exp (zn,k ) − [1 + zn,k ]|

**where zn,k = pn,k eiλ − 1 . Since Re zn,k = pn,k (cos λ − 1) ≤ 0, we may use the calculus estimate in Eq. (14.49) to conclude,
**

Page: 186 job: prob

Under the assumptions of Theorem 15.12, we see that s2 n → a as n → ∞. Let Xn,k −pn,k 2 so that E [Yn,k ] = 0 and σn,k := Var (Yn,k ) = s1 Yn,k := 2 Var (Xn,k ) = sn n 1 p (1 − p ) which satisﬁes condition ( M ) . Let us observe that, for large n,k n,k s2 n n,

macro: svmonob.cls

date/time: 23-Feb-2007/15:20

**15.1 Inﬁnitely Divisible and Stable Symmetric Distributions
**

2 2 E Yn,k : |Yn,k | > t = E Yn,k :

187

Xn,k − pn,k >t sn

n→∞ k≤n

2 lim max σn,k = 0.

2 = E Yn,k : |Xn,k − pn,k | > sn t

**Under condition (M ) , we expect fn,k (λ) ∼ = 1 for n large. Therefore we expect 1 − pn,k sn
**

2

≥E

2 Yn,k

: |Xn,k − pn,k | > 2at

**fn,k (λ) = eln fn,k (λ) = eln[1+(fn,k (λ)−1)] ∼ = e(fn,k (λ)−1) and hence that
**

n n n

2 = E Yn,k : Xn,k = 1 = pn,k

**from which it follows that
**

n n→∞ n 2 E Yn,k : |Yn,k | > t = lim k=1 n→∞

E eiλSn = pn,k

k=1

fn,k (λ) ∼ =

k=1 k=1

e(fn,k (λ)−1) = exp

k=1

(fn,k (λ) − 1) . (15.15)

lim

1 − pn,k sn

2

**= a. This is in fact correct, since Lemma 15.9 indeed implies
**

n n→∞

**Therefore {Yn,k } do not satisfy (LC ) . Nevertheless we have
**

n

lim

E eiλSn − exp

k=1

(fn,k (λ) − 1)

= 0.

(15.16)

Yn,k =

k=1

n k=1

Xn,k − sn

n k=1

pn,k

=⇒

Z −a a

**Since E [Xn,k ] = 0, fn,k (λ) − 1 = E eiλXn,k − 1 = E eiλXn,k − 1 − iλXn,k = eiλx − 1 − iλx dµn,k (x)
**

R −1 where µn,k := P ◦ Xn,k is the law of Xn,k . Therefore we have n n

where Z is a Poisson random variable with mean a. Notice that the limit is not a normal random variable. We wish to characterize the possible limiting distributions of sequences ∞ {Sn }n=1 when we relax the Lindeberg condition (LC ) to condition (M ) . We have the following theorem. Theorem 15.14. Suppose {Xn,k }k=1 satisfy property (M ) and Sn := n =⇒ L for some random variable L. Then the characteristic k=1 Xn,k function fL (λ) := E eiλL must be of the form, fL (λ) = exp

R n

exp

k=1

(fn,k (λ) − 1)

= exp

k=1 R

**eiλx − 1 − iλx dµn,k (x)
**

n

e

iλx

− 1 − iλx dν (x) x2

= exp

R

eiλx − 1 − iλx

k=1

dµn,k (x) (15.17)

= exp

R ∗ := where νn n k=1

where ν – is a ﬁnite positive measure on (R, BR ) such that ν (R) ≤ 1. (Recall iλx 1−iλx that you proved in Exercise 14.4 that exp R e − dν (x) is always the x2 characteristic function of a probability measure.) Proof. As before, let fn,k (λ) = E e the continuity theorem we are assuming lim fSn (λ) = lim

iλXn,k

∗ eiλx − 1 − iλx dνn (x)

**µn,k . Let us further observe that
**

n n

and ϕn,k (λ) := fn,k (λ) − 1. By

R

∗ x2 dνn (x) = k=1 R

x2 dµn,k (x) =

k=1

2 σn,k = 1.

n n→∞ n→∞

fn,k (λ) = f (λ)

k=1

∗ Hence if we deﬁne dν ∗ (x) := x2 dνn (x) , then νn is a probability measure and we have from Eqs. (15.16) and Eq. (15.17) that

where f (λ) is continuous at λ = 0. We are also assuming property (M ) , i.e.

fSn (λ) − exp

R

eiλx − 1 − iλx dνn (x) x2

→ 0.

(15.18)

Page: 187

job: prob

macro: svmonob.cls

date/time: 23-Feb-2007/15:20

188

15 Weak Convergence of Random Sums

iλx 1−iλx ¯ with h (±∞) = 0, there Since h (x) := e − is a continuous function of R x2 is a subsequence, {nl } of {n} such that νnl (h) → ν ¯ (h) for some probability ¯ , BR measure on R ¯ . Combining this with Eq. (15.18) allows us to conclude,

where νs is the ﬁnite measure on (R, BR ) deﬁned by νs (A) := s2 ν s−1 A for all A ∈ BR . The reader should observe that eiλx − 1 − iλx 1 = 2 2 x x and hence (λ, x) →

eiλx −1−iλx x2 ∞

**fL (λ) = lim E eiλSnl = lim exp
**

l→∞ l→∞ R

∗ eiλx − 1 − iλx dνn (x) l

= exp

R

eiλx − 1 − iλx dν (x) . x2

k=2

(iλx) 1 = 2 k! x

k

∞

k=2

ik k k−2 λ x k!

Deﬁnition 15.15. We say that

n {Xn,k }k=1

is smooth. Moreover,

**has bounded variation (BV ) iﬀ
**

n 2 σn,k < ∞.

**sup Var (Sn ) = sup
**

n n {Xn,k }k=1 n k=1

(15.19) and

d eiλx − 1 − iλx ixeiλx − ix eiλx − 1 = = i dλ x2 x2 x d2 eiλx − 1 − iλx ixeiλx = i = −eiλx . dλ2 x2 x Using these remarks and the fact that ν (R) < ∞, it is easy to see that fL (λ) =

R

**Corollary 15.16. Suppose satisfy properties (M ) and (BV ) . If n Sn := k=1 Xn,k =⇒ L for some random variable L, then fL (λ) = exp
**

R

eiλx − 1 − iλx dν (x) x2

(15.20) and fL (λ) =

i

eiλx − 1 dνs (x) fL (λ) x

**where ν – is a ﬁnite positive measure on (R, BR ) .
**

2 Proof. Let s2 n := Var (Sn ) . If limn→∞ sn = 0, then Sn → 0 in L and hence weakly, therefore Eq. (15.20) holds with ν ≡ 0. So let us now suppose ∞ limn→∞ sn = 0. Since {sn }n=1 is bounded, we may by passing to a subsequence if necessary, assume limn→∞ sn = s > 0. By replacing Xn,k by Xn,k /sn and hence Sn by Sn /sn , we then know by Slutsky’s theorem that Sn /sn =⇒ L/s. Hence by an application of Theorem 15.14, we may conclude

−eiλx dνs (x) +

R R

i

eiλx − 1 dνs (x) x

2

fL (λ)

and in particular, fL (0) = 0 and fL (0) = −νs (R) . Therefore the probability measure, µ, on (R, BR ) such that µ ˆ (λ) = fL (λ) has mean zero and variance, νs (R) < ∞. Deﬁnition 15.17. A probability distribution, µ, on (R, BR ) is inﬁnitely divisible iﬀ for all n ∈ N there exists i.i.d. nondegenerate random variables, d n {Xn,k }k=1 , such that Xn,1 +· · ·+Xn,n = µ. This can be formulated in the following two equivalent ways. For all n ∈ N there should exists a non-degenerate probn n ability measure, µn , on (R, BR ) such that µ∗ ˆ (λ) = [g (λ)] n = µ. For all n ∈ N, µ for some non-constant characteristic function, g. Theorem 15.18. The following class of symmetric distributions on (R, BR ) are equal; 1. C1 – all possible limiting distributions under properties (M ) and (BV ) . 2. C2 – all distributions with characteristic functions of the form given in Corollary 15.16.

macro: svmonob.cls date/time: 23-Feb-2007/15:20

fL (λ/s) = fL/s (λ) = exp

R

eiλx − 1 − iλx dν (x) x2

**where ν – is a ﬁnite positive measure on (R, BR ) such that ν (R) ≤ 1. Letting λ → sλ in this expression then implies fL (λ) = exp
**

R

**eiλsx − 1 − iλsx dν (x) x2 eiλsx − 1 − iλsx (sx)
**

2

= exp

R

s dν (x)

2

= exp

R

eiλx − 1 − iλx dνs (x) x2

Page: 188

job: prob

3. C3 – all inﬁnitely divisible distributions with mean zero and ﬁnite variance. Proof. The inclusion, C1 ⊂ C2 , is the content of Corollary 15.16. For C2 ⊂ C3 , observe that if µ ˆ (λ) = exp

R n

eiλx − 1 − iλx dν (x) x2

then µ ˆ (λ) = [ˆ µn (λ)] where µn is the unique probability measure on (R, BR ) such that eiλx − 1 − iλx 1 dν (x) . µ ˆn (λ) = exp x2 n R For C3 ⊂ C1 , simply deﬁne {Xn,k }k=1 to be i.i.d with E eiλXn,k = µ ˆn (λ) . In this case Sn =

n k=1 n

Xn,k = µ.

d

15.1.1 Stable Laws See the ﬁle, dynkin-stable-inﬁnitely-divs.pdf, and Durrett [2, Example 3.10 on p. 106 and Section 2.7.].

.

Part IV Conditional Expectations and Martingales .

.

Checking that · satisﬁes the remaining axioms of a norm is now routine and will be left to the reader. then. then for all x. More generally if A ⊂ H is a set. x+y 2 Fig. ·|· ) be an inner product space. x .) Then 0≤ z 2 Taking the square root of this inequality shows · satisﬁes the triangle inequality. If x. x ∈ H is orthogonal to A (write x ⊥ A) iﬀ x|y = 0 for all y ∈ A. = x + y |x + y = x = x 2 2 + y 2 + x|y + y |x (16.e. the result holds trivially. · .1. Let (H.4. Proof. y ∈ H.e. 2 if x = αy for some α ∈ C. i. z |ax + by = a ¯ z |x + ¯ b z |y . where H is viewed as the normed space (H. = x− x|y y = x y 2 | x|y |2 = x 2− y 2 2 2 2 + | x|y |2 y y 4 2 − 2Re x| x|y y y 2 from which it follows that 0 ≤ y equivalently iﬀ x = y −2 x|y y. y ∈ H | x|y | ≤ x y and equality holds iﬀ x and y are linearly dependent. Proof. Theorem 16. 3. Notice that combining properties (1) and (2) that x → z |x is conjugate linear for ﬁxed z ∈ H.1) + y 2 + 2Re x|y . Let (H. x+y 2 = x ≤ x 2 2 + y + y 2 2 + 2Re x|y + 2 x y = ( x + y )2 . = x y . An inner product on H is a function. 16. The following identity will be used frequently in the sequel without further mention. ·|· ) be an inner product space. ∆y → 0. x|y = y |x . y.1. Let (H. ·|· : H × H → C. · ). x → x|z is linear. then | x + ∆x|y + ∆y − x|y | = | x|∆y + ∆x|y + ∆x|∆y | ≤ x ∆y + y ∆x + ∆x ∆y → 0 as ∆x. Moreover ·|· is continuous on H × H. If y = 0. y ∈ H are orthogonal and write x ⊥ y iﬀ x|y = 0. Let . y ∈ H. using Schwarz’s inequality. (So z is the “orthogonal projection” of x onto y. ·|· ) be an inner product space and x := x|x . then x|y = α ¯ y and hence | x|y | = |α| y 2 Corollary 16. 2. Now suppose that x ∈ H is arbitrary. see Figure 16.2 (Schwarz Inequality).1. Then the Hilbertian norm. is a norm on H. x 2 := x|x ≥ 0 with equality x 2 = 0 iﬀ x = 0. If x. ax + by |z = a x|z + b y |z i.3. from which it follows that ·|· is continuous.16 Hilbert Space Basics Deﬁnition 16. we say x. So assume that y = 0 and observe. Let H be a complex vector space. The picture behind the proof of the Schwarz inequality. x 2 − | x|y |2 with equality iﬀ z = 0 or Deﬁnition 16. let z := x − y −2 x|y y. such that 1.

Suppose that H is a Hilbert space and M ⊂ H is a closed convex subset of H. M ) = inf m∈M m and y. By replacing M by M − x := {m − x : m ∈ M } we may assume x = 0. Then for any x ∈ H there exists a unique y ∈ M such that + x−y 2 =2 x 2 +2 y 2 (16. Deﬁnition 16. y ∈ H. is a Hilbert space – see Theorem 11. H := L2 (µ) with inner product. B . Proposition 16. µ) .2.3) 3. x = 1 for all x ∈ S. If S further satisﬁes. ·|· ) be an inner product space then 1. 2 y 2 +2 z 2 = y+z =4 2 + y−z 2 2 where Nul( ·|x ) = {y ∈ H : y |x = 0} – a closed subspace of H. A subset C of a vector space X is said to be convex if for all x. z ∈ M. it is unique. Let yn ∈ M be chosen such that yn = δn → δ ≡ d(0. Proof. If A ⊂ H is a set. (Notice that any vector subspace of X is convex.4) shows 2 2 2δm + 2δn ≥ 4δ 2 + yn − ym 2 . (16. 2 x x∈S = x∈S x| y ∈S y = x. Uniqueness.cls date/time: 23-Feb-2007/15:20 .) Theorem 16. y ∈ S. I will assume that H is a complex Hilbert space. = x∈S x|x = x∈S Item 3. ·|· ) such that the induced Hilbertian norm is complete. x x∈S = x∈S x 2. and 2. (Parallelogram Law) x+y 2 Deﬁnition 16. is a consequence of the continuity of ·|· and the fact that A⊥ = ∩x∈A Nul( ·|x ) By the parallelogram law and the convexity of M. (Pythagorean Theorem) If S ⊂⊂ H is a ﬁnite orthogonal set. 16.6. see Figure 16. 2. M ) = inf for all x. Proof. y ] := {tx + (1 − t)y : 0 ≤ t ≤ 1} joining x to y is contained in C as well.y ∈S x|y x 2.7. (16. are proved by the following elementary computations. Therefore. For any measure space. (16. then A⊥ is a closed linear subspace of H.4) f (ω ) g ¯ (ω ) dµ (ω ) Hence if y = z = δ. y ∈ C the line segment [x. then S is said to be an orthonormal set. then 2δ 2 + 2δ 2 ≥ 4δ 2 + y − z 2 .5. Existence. if M is a vector subspace of H. f |g = Ω y+z 2 + y−z 2 ≥ 4δ 2 + y − z 2 . if a minimizer for d(0. Page: 194 job: prob macro: svmonob. the real case being easier. x+y 2 + x−y = x 2 2 2 + y 2 + 2Re x|y + x 2 2 + y 2 − 2Re x|y =2 x and +2 y . ·)|M exists.2) x − y = d(x.2. M ). then 2 z ∈M x−z . Moreover. A Hilbert space is an inner product space (H. Items 1.194 16 Hilbert Space Basics A⊥ = {x ∈ H : x ⊥ A} be the set of vectors orthogonal to A. The geometry of convex sets. Example 16. so that y − z 2 = 0. then the point y may also be characterized as the unique point in M such that (x − y ) ⊥ M. Let (H. Fig. Taking y = ym and z = yn in Eq.17 for the completeness assertion. Let δ := d(0.8. A subset S ⊂ H is an orthogonal set if x ⊥ y for all distinct elements x. (Ω.9 (Best Approximation Theorem).

m. is the unique operator A∗ : H → H such that Ax|y = x|A∗ y . 2δ 2 + 2δ 2 ≥ 4δ 2 + lim sup yn − ym 2 . M )2 ≥ x − y 2 . ⊥ 3. the function g (t) := x − (y + tw) 2 = x−y 2 − 2tRe x − y |w + t2 w 2 has a minimum at t = 0 and therefore 0 = g (0) = −2Re x − y |w .5) j (16. i. the PN PM = PM PN = PN . z2 ∈ H and α ∈ C. Let H be a Hilbert space and M ⊂ H be a closed subspace. PM is linear and hence we will write PM x rather than PM (x).e. and Nul(PM ) = M ⊥ . 2 2.e. Therefore PM = PM .cls date/time: 23-Feb-2007/15:20 . i.14 (Riesz Theorem). Let x.6) Theorem 16. y := lim yn ∈ M and because the norm is n→∞ continuous. namely that PN PM = PN . We have already seen. So M ∩ M ⊥ = {0} . then since (x − PM x) and (y − PM y ) are in M . Nul(PM ) = M ⊥ .e.13. The orthogonal projection PM satisﬁes: 1.) A bounded operator A : H → H is self . macro: svmonob. is self-adjoint). x−z 2 = x−y+y−z 2 = x−y 2 + y−z 2 ≥ x−y 2 which shows d(x. If N ⊂ M ⊂ H it is clear that PM PN = PN since PM = Id on N = Ran(PN ) ⊂ M. Because M is closed. Let x1 .1. Then for w ∈ M.n→∞ 2 2. Ran(PM ) = M 5. PM x|y = PM x|PM y + y − PM y = PM x|PM y = PM x + (x − PM x)|PM y = x|PM y . then M ⊥⊥ = span(M ). n→∞ Proof. Since this holds for all n we may conclude that PN PM x = PN x. If x ∈ M ∩ M ⊥ . Then for z ∈ M. 1.16 Hilbert Space Basics 195 Passing to the limit m. x = x|x = 0. let y = PM x so that x−y ∈ M ⊥ . Page: 195 job: prob Recall that j is conjugate linear if j (z1 + αz2 ) = jz1 + αjz ¯ 2 for all z1 . Since w ∈ M is arbitrary. by Pythagorean’s theorem. we have PN PM x|n = PM x|PN n = PM x|n = x|PM n = x|n . denoted A∗ . i. Finally suppose y ∈ M is any point such that (x − y ) ⊥ M. y ∈ H. 4. Then x = y +(x−y ) ∈ 2 M + M ⊥ . Suppose M is a subset of H. Let y ∈ M be the closest point in M to x.e. The orthogonal projection of H onto M is the function PM : H → H such that for x ∈ H. Now suppose M is a closed subspace of H and x ∈ H. 5. (The proof that A∗ exists and is unique will be given in Proposition 16. M ). this implies that (x − y ) ⊥ M. i. Given x ∈ H. Corollary 16. That is to say y is the point in M closest to x. Suppose that A : H → H is a bounded operator. (16. then H = M ⊕ M ⊥ . 1 So y is the desired point in M which is closest to 0.10. Deﬁnition 16. then x ⊥ x. lim supm. n → ∞ in this equation implies. The adjoint of A. More directly. Proof.11.e. The map z ∈ H −→ ·|z ∈ H ∗ is a conjugate linear1 isometric isomorphism. PM (x) is the unique element in M such that x|m = PM (x)|m for all m ∈ M.e.adjoint or Hermitian if A = A∗ . Let H ∗ be the dual space of H (i. i. If M ⊂ H is a proper closed subspace of a Hilbert space H.12 (Projection Theorem). If N ⊂ M ⊂ H ∞ is a projection). PM = PM (PM 4. A := sup { Ax : x ∈ H with x = 1} < ∞. Exercise 16.n→∞ yn − ym 2 = 0. that linear space of continuous linear functionals on H ). Theorem 16. y = lim yn = δ = d(0. Therefore. Deﬁnition 16. Let H be a Hilbert space and M ⊂ H be a closed subspace. i. PM = PM (PM ∗ 3. PM (x) is the unique element in M such that (x − PM (x)) ⊥ M. Taking adjoints gives the other identity.15 below. if x ∈ H and n ∈ N. PM is linear. x2 ∈ H and α ∈ C.e. {yn }n=1 is convergent. Ran(PM ) = M and PM x = 0 iﬀ x = x − 0 ∈ M ⊥ . is another closed subspace. by completeness of H. then PM x1 + αPM x2 ∈ M and PM x1 + αPM x2 − (x1 + αx2 ) = [PM x1 − x1 + α(PM x2 − x2 )] ∈ M ⊥ showing PM x1 + αPM x2 = PM (x1 + αx2 ). Obviously Ran(PM ) = M and PM x = x for all x ∈ M .

A∗ A = A .9) proves A = A∗ A . ∗ ¯ ∗. H = M ⊕ M ⊥ . then for any x ∈ H. Now suppose that K = H. Using this identity back in 2 Eq. Let H and K be Hilbert spaces and A : H → K be a bounded operator.196 16 Hilbert Space Basics Proof. with out loss of generality. 1. x = λx0 . and 4.14. ≤ A∗ A ≤ A A∗ (16. (A + λB ) = A∗ + λB ∗∗ ∗ ∗ 2.15 (Adjoints). f : H/M ∼ = M ⊥ → F is a linear isomorphism. z ∈ H. by Theorem 16. This shows that dim(M ⊥ ) = 1 and hence H = M ⊕ Fx0 where x0 ∈ M ⊥ \ {0} . (16. f (x) = λf (x0 ) = λ x0 |z = λx0 |z = m + λx0 |z = x|z which shows that f = jz. Items 3. This again shows that M ⊥ is spanned by x0 . then (AB ) = B ∗ A∗ . a unique vector z ∈ H (we will denote this z by A∗ (y )) such that Ax|y K = x|z H for all x ∈ H. A∗ = A and 2 4. If K = H. y2 ∈ K and λ ∈ C. | x|z | ≤ x z for all x ∈ H Ax|y1 + λy2 K ¯ Ax|y2 K = Ax|y1 K + λ ∗ ¯ x|A∗ (y2 ) = x|A (y1 ) K + λ = x|A∗ (y1 ) + λA∗ (y2 ) H H with equality when x = z. To see A∗ is linear. i. ∗ 5. This implies that jz H ∗ = ·|z H ∗ = z . Since A∗ y |x H = x|A∗ y H = Ax|y K = y |Ax K ∗ ¯ ∗ is Exercise it follows that A∗∗ = A. for all A. (16. which then implies A ≤ A∗ .2. Replacing A by A∗ in this last inequality shows A∗ ≤ A and hence that A∗ = A . the map x → Ax|y K is in H ∗ and therefore there exists. A := (A ) = A. Then ABh|k = Bh|A∗ k = h|B ∗ A∗ k macro: svmonob. Since A∗ A ≤ A∗ and A 2 A = A 2 = = sup h∈H : h =1 Ah 2 = sup h∈H : h =1 | Ah|Ah | sup A∗ Ah = A∗ A 2 sup h∈H : h =1 | h|A∗ Ah | ≤ 2 (16. is non-zero. λ = f for x = m + λx0 with m ∈ M and λ ∈ F.7) h∈H : h =1 k∈K : k =1 Moreover. This shows there is a unique map A∗ : K → H such that Ax|y K = x|A∗ (y ) for all x ∈ H and y ∈ K. Making use of Schwarz’s inequality (Theorem 16.13.cls date/time: 23-Feb-2007/15:20 Page: 196 job: prob . so that A∗ = A .e. For each y ∈ K. (16. 2 h∈H : h =1 we also have A∗ A ≤ A ≤ A∗ A which shows A Alternatively. Therefore j is isometric and this implies j is injective. choose x0 ∈ M ⊥ \ {0} such that f (x0 ) = 1.e. The assertion that (A + λB ) = A∗ + λB 16.2 ¯(x0 )/ x0 2 .8) Proof. Therefore x − λx0 ∈ M ∩ M ⊥ = {0} . we have A∗ = = = sup k∈K : k =1 A∗ k sup sup | A∗ k |h | | k |Ah | = sup h∈H : h =1 sup sup k∈K : k =1 h∈H : h =1 Ah = A = x|A∗ y H for all x ∈ H and y ∈ K. B ∈ L(H. Moreover. Then there exists a unique bounded operator A∗ : K → H such that Ax|y K and by the uniqueness of A∗ (y1 + λy2 ) we ﬁnd A∗ (y1 + λy2 ) = A∗ (y1 ) + λA∗ (y2 ). for x.8). In particular A ∈ L (H ) has a bounded ∗ −1 ∗ inverse iﬀ A has a bounded inverse and (A∗ ) = A−1 . K ) and λ ∈ C. i. Then M =Nul(f ) – a closed proper subspace of H. 3. For x ∈ M ⊥ we have f (x − λx0 ) = 0 provided that λ := f (x). So let f ∈ H ∗ which we assume. A 2 = A∗ A . This shows A∗ is linear and so we will now write A∗ y instead of A∗ (y ). from Eq. Proposition 16. Then Choose z = λx0 ∈ M ⊥ such that f (x0 ) = x0 |z . Since. by Corollary 16.9) H Alternatively. To ﬁnish the proof we must show that j is surjective. let y1 . The map j is conjugate linear by the axioms of the inner products.2).

X is a Banach3 space. Let A be an m × n matrix thought of as a linear operator from H to K. Theorem). 2. Page: 197 job: prob macro: svmonob.3. then A∗ y |x = y |Ax = 0 for all x ∈ V which shows A∗ y ∈ V ⊥ . Because.cls date/time: 23-Feb-2007/15:20 . A. ¯. and λ ∈ C. C ∈ L(K. and so by the ﬁrst item. ¯z exists. The uniqueness of this extension is Thus T easy to prove and will be left to the reader. K ). The next elementary theorem (referred to as the bounded linear transformation theorem. Lemma 16. n → ∞. K. Let H = Cn and K = Cm equipped with the usual inner products. and S ⊂ Z is a dense linear subspace of Z. then: 1. The main examples for us are Hilbert spaces. Ran(A) = Nul(A∗ )⊥ .e. Proof. for all z ∈ S A Banach space is a complete normed space. Ran(A) = Nul(A∗ )⊥ . T. A(V ) ⊂ V ).2. z |w H = z · w ¯ for z. = A−1 ∗ . Moreover. B ∈ L(H. It is now a simple matter to check that T ¯ : and therefore T Z → X is still linear and that ¯z = lim T n→∞ A∗ = AA−1 = A −1 ∗ ∗ = I ∗ = I and = I = I. Suppose A : H → K is a bounded operator. 3.1. w ∈ H. M ) ∗ ¯ ∗ and (CA)∗ = A∗ C ∗ ∈ L(M. Let z ∈ Z and choose zn ∈ S such that zn → z. or B. Now suppose A(V ) ⊂ V and y ∈ V ⊥ . i. ¯ is an extension of T to all of the Z. there exists C < ∞ such that T z ≤ C z ¯ ∈ L(Z. Since T zm − T zn ≤ C zm − zn → 0 as m.L. Ran(A) = Ran(A)⊥⊥ .e. Similarly if A∗ is Exercise 16. L.16 Hilbert Space Basics 197 which shows (AB ) = B ∗ A∗ .e. if K = H and V ⊂ H is an A – invariant subspace (i. If T : S → X is a bounded linear transformation (i. Let H. If A−1 exists then A−1 A ∗ ∗ ∗ Proof. An element y ∈ K is in Nul(A∗ ) iﬀ 0 = A∗ y |x = y |Ax for all x ∈ H which happens iﬀ y ∈ Ran(A)⊥ . Nul(A∗ ) = Ran(A)⊥ . theorem for short) is often useful. M be Hilbert spaces. Theorem 16. it follows by the completeness of X that limn→∞ T zn =: T if wn ∈ S is another sequence converging to z. −1 ∗ A −1 ∗ A This shows that A∗ is invertible and (A∗ ) invertible then so is A = A∗∗ .17 (B. then T has a unique extension to an element T this extension still satisﬁes ¯z ≤ C z T 3 T zn ≤ lim C zn = C z n→∞ for all x ∈ Z. then V ⊥ is A∗ – invariant. H ).16. X ) and for all z ∈ S ).T. by Exercise 16. then T zn − T wn ≤ C zn − wn → C z − z = 0 ¯z is well deﬁned. Show (A + λB ) = A∗ + λB Exercise 16. Suppose that Z is a normed space. Show the matrix associated to A∗ : K → H is the conjugate transpose of A.

.

f ρdλ = X f Re ρ dλ.1) shows 0 ≤ ν (ρ < 0) = X 1ρ<0 ρ dλ ≤ 0. it follows that λ (ρ > 1) = 0 and hence 0 ≤ ρ ≤ 1. ρdµ = 0. Choose Ai ∈ M such that ν (Ai ) = 0 and µi (Ac i ) = 0 for all i. of the measure ν relative to µ then this decomposition is unique. we have µi (A ) = 0 for all i and therefore. By the Riesz representation Theorem 16. L2 (λ) f → ν (f ) ∈ C is a continuous linear functional on L2 (λ). If there exists a Lebesgue decomposition. c Letting A := ∪i Ai we have ν (A) = 0. The measure ν is absolutely continuous relative to µ (written as ν µ) provided ν (A) = 0 whenever µ (A) = 0. since Ac = ∩i Ac i ⊂ Am for c c all m. If µ1 . λ (ρ < 0) = 0. µ (A ) = 0. 1] such that dν = ρdλ.1) Thus by replacing ρ by Re ρ if necessary we may assume ρ is real. Furthermore if f ∈ L1 (λ) . X We will eventually show that if µ and ν are σ – ﬁnite and ν for some measurable function. (X. and νs ⊥ µ. ρ : X → [0. λ (ρ > α) ≥ ν (ρ > α) = X 1ρ>α ρ dλ ≥ αλ (ρ > α) which is possible iﬀ λ (ρ > α) = 0. suppose that µ is a positive measure and ρ ≥ 0 is a measurable function. Let ν and µ be positive measures on (X. (X. M) is a measurable space. Moreover: if ν is a σ – ﬁnite measure then so are νs and νa . As an example. M). i. then dν = ρdµ Therefore. Similarly for α > 1. Indeed. this inequality continues to hold for all non-negative measurable functions. then (µ1 + µ2 ) ⊥ ν. This shows that µ ⊥ ν. λ and ν are two ﬁnite positive measures on M such that ν (A) ≤ λ(A) for all A ∈ M. ρ ≥ 0. Then the measure. Suppose (X. Let µ and ν be two positive measure on a measurable space. Lemma 17. M).4. Letting α ↓ 1. M) such that µ1 ⊥ ∞ ν and µ2 ⊥ ν. (17. ν := ρµ is absolutely continuous relative to µ. Proof.17 The Radon-Nikodym Theorem Theorem 17. Therefore ρ ≥ 0. Two positive measures νa and νs form a Lebesgue decomposition of ν relative to µ if ν = νa + νs . Then: 1. If f is a non-negative simple function. (17. ν = νs + νa . µ2 and ν are positive measures on (X. 2.. M). then ν (|f |) ≤ λ (|f |) < ∞ and hence f ∈ L1 (ν ) and |ν (f )| ≤ ν (|f |) ≤ λ (|f |) ≤ λ (X ) 1 /2 · f L2 (λ) . λ – a. Moreover. Lemma 17. f : X → R and for such a function we have ν (f ) = Re ν (f ) = Re X Deﬁnition 17. µ.2.a. from which we conclude that 1ρ<0 ρ = 0.3 (Lebesgue Decomposition). . Let µ and ν be two positive measure on a measurable space. there exists a unique ρ ∈ L2 (λ) such that ν (f ) = f ρdλ for all f ∈ L2 (λ). In particular this equation holds for all bounded measurable functions.1 (A Baby Radon-Nikodym Theorem). Proof.5.34 and the MCT. νa µ. µ and ν are mutually singular (written as µ ⊥ ν ) if there exists A ∈ M such that ν (A) = 0 and µ (Ac ) = 0. if µ (A) = 0 then ν (A) = A aν (f = a) ≤ a≥0 aλ (f = a) = λ (f ) .e.e.e. Taking f = 1ρ<0 in Eq. More generally if {µi }i=1 is a sequence of ∞ positive measures such that µi ⊥ ν for all i then µ = i=1 µi is singular relative to ν. We say that ν lives on A and µ lives on Ac . In light of Theorem 6.14. Then there exists a measurable function. It suﬃces to prove the second assertion since we can then take µj ≡ 0 for all j ≥ 3.e. λ – a. then ν (f ) = a≥0 Deﬁnition 17. λ .

Xn (17. x. νs (C ) = ν (C ∩ B ) = ν ˜s (C ) and c νa (C ) = ν (C ∩ B ) = ν ˜a (C ) for all C ∈ M. x. (17.7. Working as above. Hence if we deﬁne νa := 1{h<1} ν and νs := 1{h=1} ν. Hence ν = νa + νs is the desired Lebesgue decomposition of ν. Proof. Proof. So for C ∈ M. µ) and measure νn such that dνn = ρn dµn + dνn with s s νn ⊥ µn . (17. ν = ν ˜a + ν ˜s with ˜ ∈ M such that ν ˜s ⊥ µ and ν ˜a µ. then µ (h = 1) = 0 implies ν (h = 1) = 0 and hence that νs = 0 and we conclude that ν = νa = ρµ. (17. that ν (f ) = λ(f h) = µ(f h) + ν (f h) or equivalently ν (f (1 − h)) = µ(f h).2) Theorem 17.) First suppose that µ and ν are ﬁnite measures and let λ = µ + ν. for all non-negative measurable functions f.1.6) to learn ν (g 1{h<1} ) = µ(g 1{h<1} (1 − h)−1 h) = µ(ρg ). f = 1X and g = 2 · 1X . (17.e. Since µn and νn “live” on Xn there exists An ∈ MXn such that µ (An ) = µn (An ) = 0 and Then f (x) = g (x) for µ – a. The uniqueness assertions follow directly from Lemmas 17. A trivial counterexample is to take M = 2X . Write X = n=1 Xn where Xn ∈ M are chosen so that µ(Xn ) < ∞ and ν (Xn ) < ∞ for all n. Letting n → ∞ then shows that f = g. νs = 0 iﬀ ν µ. Since 1Xn f and 1Xn g are in L1 (µ) for all n.cls date/time: 23-Feb-2007/15:20 . M). Since νs ⊥ µ.6) shows that µ ({h = 1}) = ν (1{h=1} (1 − h)) = 0. dν = hdλ with 0 ≤ h ≤ 1 and this implies. Lemma 17. µ – a. ∞ Existence when µ and ν are σ -ﬁnite measures.e. (17. Lemma 17. f dµ and gdµ are σ – ﬁnite and further satisfy.3) with A being replaced by B to conclude.4) implies 1Xn f dµ = A A∩Xn f dµ = A∩Xn gdµ = A 1Xn gdµ for all A ∈ M. Replacing A by A ∩ Xn in Eq.6 is in general false without the σ – ﬁniteness assumption. By Theorem 17. Let ρ := 1{h<1} h 1−h (17.8 (Radon Nikodym Theorem for Positive Measures). we must have νa (Xn ) < ∞ and νs (Xn ) < ∞.200 17 The Radon-Nikodym Theorem Proof.5) Now suppose we have another Lebesgue decomposition. Then Eq. Suppose µ is a positive measure on (X. g : X → [0. µ – a.6. we also know that νa (A) = 0.6) (17.e. See Remark 17.3) (17. f dµ = A A gdµ for all A ∈ M. If we further assume that ν µ. µn ) ⊂ L1 (X.9 for the motivation for this proof.4) and then take f = g 1{h<1} (1 − h)−1 with g ≥ 0 in Eq.a. By assumption there exists Xn ∈ M such that Xn ↑ X and f dµ < ∞ and Xn gdµ < ∞ for all n. M) and f. i. Moreover. (Von-Neumann’s Proof. µ(A) = ∞ for all non-empty A ∈ M. ∞] are functions such that the measures. (17.6. ν (C ∩ A) = νs (C ∩ A) + νa (C ∩ A) = νs (C ∩ A) = νs (C ) and ν (C ∩ Ac ) = νs (C ∩ Ac ) + νa (C ∩ Ac ) = νa (C ∩ Ac ) = νa (C ) . Page: 200 job: prob macro: svmonob. this equation implies 1Xn f = 1Xn g.2) and (17. Lastly if ν is a σ – ﬁnite measure then there exists Xn ∈ M such that ∞ X = n=1 Xn and ν (Xn ) < ∞ for all n. Existence when µ and ν are both ﬁnite measures. Suppose that µ and ν are σ – ﬁnite positive measures on (X. Let dµn = 1Xn dµ and dνn = 1Xn dν.5 and 17. Remark 17. 0 ≤ h (x) < 1 for µ . Then ν has a unique Lebesgue decomposition ν = νa + νs relative to µ and there exists a unique (modulo sets of µ – measure 0) function ρ : X → [0.e. Since ∞ > ν (Xn ) = νa (Xn ) + νs (Xn ).4) holds yet f = g. Then B = A ∪ A ˜c is a null set for both νs and ν B c = Ac ∩ A ˜s . Therefore we may use Eqs. Then by what we have just proved there exists s s ρn ∈ L1 (X.e. we may choose A ˜) = 0 and A ˜c is ν ˜ is still a µ – null set and and µ(A ˜s – null. ∞) such that dνa = ρdµ. there exists A ∈ M such that µ(A) = 0 and νs (Ac ) = 0 and because νa µ. showing νa and νs are σ – ﬁnite as well. Taking f = 1{h=1} in Eq. we then have νs ⊥ µ (since νs “lives” on {h = 1} while µ (h = 1) = 0) and νa = ρµ and in particular νa µ.

A = {h = 1} and B = {h < 1} and therefore.e. Remark 17. ρ = ρh + h. Suppose that dν = dνs + ρdµ is the Radon-Nikodym decomposition and X = A B such that νs (B ) = 0 and µ(A) = 0. Letting f → 1A f then implies that ν (1A f ) = νs (1A f ) = ν (1A hf ) which show that h = 1. dν = 1{h=1} dν + 1{h<1} dν = 1{h=1} dν + h 1h<1 dµ.e. Then we ﬁnd νs (f ) + µ(ρf ) = ν (f ) = λ(hf ) = ν (hf ) + µ(hf ). it follows that ν = νa + νs with νa = ρµ. ν –a.e. µ = ν – a.e. h In particular it follows that h < 1.4. So up to sets of ν – measure zero. i. s This shows that νn ⊥ µ for all n and so by Lemma 17. µ – a. on B and that ρ = 1− h 1h<1 .s s νn (X \ An ) = νn (Xn \ An ) = 0. Since ∞ ∞ ∞ s (ρn µn + νn )= n=1 ∞ n=1 s (ρn 1Xn µ + νn ) = ρµ + νs .9. µ – a. on B. ρ (1 − h) = h. on B. 1−h . νs := singular relative to µ. Hence this is the desired Lebesgue decomposition of ν relative to µ. ∞ n=1 s νn is ν= n=1 νn = (17.7) where ρ := n=1 1Xn ρn .e. Here is the motivation for the above construction. on A. Also letting f → 1B f implies that µ(ρ1B f ) = ν (h1B f ) + µ(h1B f ) = µ(ρh1B f ) + µ(h1B f ) which implies.e. µ– a.

.

BR ¯ ) – measurable function h : Y → R Proof. there is a map. P ) is the conditional expectation of f. F ) – measurable with h := ai 1Bi – a simple function on R ¯ . iﬀ equality holds in Eq. P ) extends uniquely to a linear contraction from L1 (Ω. If g ∈ Gb and f ∈ L1 (Ω. (18.18 Conditional Expectation In this section let (Ω.1). This extension enjoys the following properties. P ) → L2 (Ω. X. Let hn function. Deﬁnition 18. either F = 0 then G = 0 and F = 0 then G ≤ F and hence G ≤ F a. P ) and G ⊂ B be as above and let f. P ) be a probability space. G . B .e. |EG f | ≤ EG |f | . If A ∈ B and P (A) > 0. we will let E [X |A] := P (A ∩ B ) E [X : A] and P (B |A) := E [1B |A] := P (A) P (A) 0 = E [F : F = 0] ≥ E [G : F = 0] and hence that G1F =0 = 0 a. (F . from Ω → R ¯ such that Hn = hn ◦ F. By the MCT we may now let α ↓ 1 to conclude.s. (18. choose simple functions Hn converging to H. we have shown. B . Hopefully it is clear to the reader that it suﬃces to prove the ﬁrst assertion. Also it is clear that F ≥ G a.e. iﬀ E [F : A] ≥ E [G : A] for all A ∈ B . B . P ). P ) denote orthogonal projection of L2 (Ω.1). i. P – a. B . B .e. If F ∈ L1 (Ω. Let EG : L2 (Ω. Let G ⊂ B be a sub – sigma algebra of B and write f ∈ Gb if f : Ω → C is bounded and f is (G . B . Then to every (σ (F ).s. 0 = E [F : G > F ] . P ) → L2 (Ω. B .e.e. Suppose that F. B . Proof. Taking h := sgn(F ) := ¯ F |F | 1|F |>0 in this identity then implies 0 = E [F h] = E F which implies that F = 0 a. P ) . Pull out property or product rule. P ). F ) is a measurable space and F : Ω → Y is a ¯ . H : Ω → R ¯ such that H = h ◦ F. If f ≥ 0. P ). P ). (18.2) ¯ F 1|F |>0 = E |F | 1|F |>0 = E [|F |] |F | for all integrable random variables. (18. if we take A = {F = 0} in Eq. B . P ) then F = EG f ∈ L1 (Ω. G : Ω → [0. P – a.e. P ) is a measure space and P (Ω ) = 1. P – a. 3. P – a. G .4.3 (Conditional Expectation).1) In particular F = G a. . there EG f ≥ EG g. i.s. P ) and E [F : A] = 0 for all A ∈ B. implies Eq.35 in this section. G . P ). G .2. we may conclude by a simple limiting argument that E [F h] = 0 for all h ∈ Bb . F 1G>F = 0 a.s. Moreover. F = 0 a. P – a. almost surely. iﬀ E [F : A] = 0 for all A ∈ B .s. the only way this can happen is if E [F : G > αF ] = 0. where h := lim sup hn – a measurable function from Y to R n→∞ Lemma 18. then EG (gf ) = g · EG f.s.1) we learn that 5. P ) to L1 (Ω. We will often use the factorization Lemma 6. B . then E [F : G > αF ] ≥ E [G : G > αF ] ≥ E [αF : G > αF ] = αE [F : G > αF ] . If f ∈ L1 (Ω. Lemma 18. For f ∈ L2 (Ω. Monotinicity. 4. Then it follows that be simple functions on R H = lim Hn = lim sup Hn = lim sup hn ◦ F = h ◦ F n→∞ n→∞ n→∞ ¯. Because of this let us repeat it here. Similarly if A := {G > αF } with α > 1 in Eq. P ) onto the closed subspace L2 (Ω. P ) iﬀ E(F h) = E(f h) for all h ∈ Gb . g ∈ L1 (Ω. G . we say that EG f ∈ L2 (Ω. B . Then F ≥ G a. then there exists Bi ∈ F such that 1Ai = 1Bi ◦ F and hence H = h ◦ F ¯ . Since α > 1. Therefore. B . then EG f ≥ 0. More generally if H = ai 1Ai is a simple function. P – a.s. The operator EG : L2 (Ω. for F ∈ L1 (Ω. (18. and B ∈ B . (Ω.1). G . Let B ∈ F such that A = F −1 (B ) then 1A = 1F −1 (B ) = 1B ◦ F and hence the Lemma is valid in this case with h = 1B .e.1. Suppose that (Y. For the converse assertion.s. Let (Ω. First suppose that H = 1A where A ∈ σ (F ) = F −1 (F ). H. For general (σ (F ). ∞] are B – measurable functions.e. BR ¯ ) – measurable function. (18. 2. 1. B . BC ) – measurable. Theorem 18. If f ≥ g.

P ) . (18.s. of E [f |G ] satisﬁes. More precisely you are to show that any version. This then implies that f dP = Q (A) = A A Proof. P ) . Page: 204 job: prob macro: svmonob. M) is a measurable space and X : Ω → X is a measurable map. Thus EG (gf ) = g · EG f. f ∈ L1 (P ) . i. Tower or smoothing property. or equivalently E((F − EG f ) h) = 0 for all h ∈ Gb . for all f ∈ L1 (Ω. P ) to L1 (Ω.s. For general real valued. B . P ). B . P ) and f > 0 a. P ) and therefore EG fn → EG f in L1 (Ω. Exercise 18. (18.s. ±EG f ≤ EG |f | . let Q := f P and observe that Q|G P |G and hence there exists 0 ≤ g ∈ L1 (Ω.. (18. Then 18. Notation 18. B . In this case EG f = Ef a. F = EG f a.s. g. Eq. P ) satisﬁes Eq.e. P ) . for 0 ≤ f ∈ L1 (P ) . b) a. B . B . Ω } . P ) such that dQ|G = gdP |G .6 In the future.7) i. We will often simply denote E [f |σ (X )] simply by E [f |X ] . it follows that |EG f | ≤ EG |f | .4) Taking h = sgn (EG f ) := in Eq. Thus we have shown EG fn ↑ EG f almost surely and in L1 (Ω. of the projection Theorem 16.e.3) E [|EG f | h] = E EG f · sgn (EG f )h = E f · sgn (EG f )h ≤ E [|f | h] = E [EG |f | · h] . G . ±f ≤ |f | and so by Item 2. EG f ≥ 0. E [(g EG f ) h] = E [EG f · hg ] = E [f · hg ] = E [gf · h] = E [EG (gf ) · h] . by a simple limiting argument. b) a. B . (18.3) holds on L2 (Ω. then E [f |G ] ∈ (a. P – a. since if h ∈ Gb . Since h is arbitrary. If f is real. g ∈ (a.5.5) It follows from this equation and the BLT (Theorem 16.e. let h ≥ 0 be a bounded and G – measurable function. (18. Item 5.12. Example 18. Remark 18. Taking h = sgn (F − EG f ) in this identity then shows E [|F − EG f |] = 0. then EG f = Ef a.e. for some a. P ) and the density of L1 probability spaces in L2 – probability spaces shows that Eq. If G0 ⊂ G1 ⊂ B. Eq. Suppose f ∈ L1 (Ω. P ) and h ∈ Gb . G . Moreover. i. There is another standard construction of EG f based on the characterization in Eq.204 18 Conditional Expectation 6. P – a. Moreover.e.s. b such that −∞ ≤ a < b ≤ ∞. Item 6. B .8. By continuity of conditional expectation on L1 (Ω. If f ≥ g a. B . h ≥ 0 then 0 ≤ E(f h) = E(EG f · h) and since this holds for all h ≥ 0 in Gb . B . Then EG0 EG1 f = EG1 EG0 f = EG0 f a. and we may identify g with EG f a.6) EG f 1|E f |>0 EG f G (18. deﬁne EG f = EG f+ − EG f− and then for complex f ∈ L1 (P ) let EG f = EG Re f + iEG Im f. Then by the MCT (or DCT) it follows that fn → f in L1 (Ω.e. Use this result to conclude if f ∈ (a. On the other hand.1.s. i. G = {∅.1 Examples Example 18.e. P ) and hence E(EG f · h) = E( lim EG fn · h) = lim E(EG fn · h) n→∞ n→∞ gdP for all A ∈ G . P ) .3) continues to hold on L1 (Ω. P ) and fn ↑ f a. Now suppose 0 ≤ fn ≤ f ∈ L1 (Ω. This proves item 4. If f. and P (A|X ) := P (A|σ (X )) be conditional probability of A given X.4) shows E(|EG f |) = E(EG f · h) = E(f h) ≤ E(|f h|) ≤ E(|f |). (18. B .. then fn → f in L1 (Ω. if (X. Indeed. P ). (18. E(f h) = E(f · EG h) = E(EG f · h). we may apply this result with f replaced by f − g to complete the proof of both items. Conversely if F ∈ L1 (Ω.4) remains valid for all f ∈ L1 (Ω. if G = B . G .s. is now an easy consequence of the characterization in item 4. On the opposite extreme. B . P ) and h ∈ Gb .7.cls date/time: 23-Feb-2007/15:20 .s. then E(F h) = E(f h) = E(EG f · h) for all h ∈ Gb .. (18.s. B . f ∈ L2 (Ω. Show E [f |G ] > 0 a. if fn := f 1|f |≤n ∈ L1 (Ω.s. For complex f.. B . Items 1 and 2. |EG f | ≤ EG |f | .8. = lim E(fn · h) = E(f · h). n→∞ (18. It goes as follows. Item 3.s. (18.2) and the Radon Nikodym Theorem 17.s. b) a.e. Item 6. P ).s. g :=↑ limn→∞ EG fn exists a. P – a. by the item 5. by item 2. P – a. B . Suppose G is the trivial σ – algebra.17) that EG extends uniquely to a contraction form L1 (Ω. We will further let P (A|G ) := E [1A |G ] be the conditional probability of A given G . g = EG f. we will often write EG f as E [f |G ] . By the deﬁnition of orthogonal projection.2).

y ) dν (y ) if ρ ¯ (x) ∈ (0. ˜ (X ) a. According to Lemma 18.12) E [g (X ) h (X )] = X×Y h (x) g (x) ρ (x. Conversely if EG [f (X )] = E [f (X )] = µ and A ∈ G . g : Ω → R is G – measurable iﬀ g = i=1 λi 1Ai for some λi ∈ R. B . Y ) ∈ U ) = U ρ (x. y ) dν (y ) = g (x) ρ ¯ (x) for µ – a. let E [f |Ai ] := E [1Ai f ] /P (Ai ) if P (Ai ) = 0 and E [f |Ai ] = 0 otherwise. Conversely if EG [f (X )] = E [f (X )] a. Remark 18. such that ˜ (X ) h (X ) E [f · h (X )] = E f for all bounded and measurable functions.) Let G be the σ – algebra generated by P . B . For f ∈ L (Ω. and there exists 0 ≤ ρ ∈ L1 (Ω. P ) is a probability space. where Q (X. E [f (Y ) h (X )] = X×Y h (x) f (y ) ρ (x. B . h : X → R. 1 3. y ) dµ (x) dν (y ) h (x) X Y = f (y ) ρ (x. The following Exercise should help you gain some more intuition about conditional expectations. ∞) δy0 (B ) if ρ ¯ (x) ∈ {0.11) (18. y ) dµ (x) dν (y ) for all U ∈ M ⊗ N . M. X : Ω → X is a measurable function.1 Examples 205 Lemma 18.13). (Recall this means Ω = i=1 Ai . Show: 1.s. and G is a sub-σ -algebra of B . then X is independent of G . Suppose (Ω. B ) := 1 ρ ¯(x) B ρ (x.Fubini theorems. ∞} . for all bounded measurable functions. by independence.s. Y (18. According to Remark 18. and A ∈ G . y ) dν (y ) (18. Then for any bounded (or nonnegative) measurable function.10 (Note well. Then.s. such that E [f (Y ) h (X )] = E [g (X ) h (X )] for all h ∈ Mb . Suppose that (Ω.. X : Ω → X and Y : Ω → Y are measurable functions. Proof.2.9) and x ∈ X and B ∈ N . ∞) Y δy0 (f ) = f (y0 ) if ρ ¯ (X ) ∈ {0. ∞ 2. B . which are to be equal for all h ∈ Mb . f : X → R is a measurable function such that f (X ) ∈ L (Ω. M) is a measurable space.18.s. E [f (X ) : A] = E [f (X ) 1A ] = E [f (X )] E [1A ] = E [µ1A ] = E [µ : A] .9. f : Y → R.e. µ ⊗ ν ) such that P ((X. µ := E [f (X )] .10. ν ) are two σ – ﬁnite measure spaces. Suppose that X is independent of G . P ).s. then EG [f (X )] = E [f (X )] a. B .cls date/time: 23-Feb-2007/15:20 . N . Proof. f : X → R. µ) and (Y. let Q (x. Our goal is to compute E [f (Y ) |X ] . ∞} (18. Show ∞ ∞ {Ai }i=1 Proposition 18. Exercise 18. The following remark is often useful in computing conditional expectations. f ) a. P ) . (18. f : X → R.12) and (18. P ) . f : X → R.8) (Throughout this argument we are going to repeatedly use the Tonelli . P ) is a probability space and P := ⊂B ∞ is a partition of Ω.11. f ) := 1 ρ ¯(X ) f (y ) ρ (X. requires us to demand. So computing E (f |X ) = f ˜ equivalent to ﬁnding a function. f (y ) ρ (x. (X. Suppose (X.) We now compare both sides of this equality.s.13) EG f = i=1 E [f |Ai ] 1Ai a. Therefore EG [f (X )] = µ = E [f (X )] a. y ) dµ (x) dν (y ) h (x) g (x) ρ ¯ (x) dµ (x) X = (18. y ) dν (y ) dµ (x) (18. B ∈ G iﬀ B = ∪i∈Λ Ai for some Λ ⊂ N. where Comparing Eqs. f : X → R.10) where y0 is some arbitrary but ﬁxed point in Y.1. Since this last equation is assumed to hold true for all A ∈ G and all bounded measurable functions. then E [f (X ) 1A ] = E [f (X ) : A] = E [µ : A] = µE [1A ] = E [f (X )] E [1A ] . we are searching for a bounded measurable function.). (18. we have E [f (Y ) |X ] = Q (X. B . x. Let ρ ¯ (x) := Y ρ (x. E (f |X ) = f ˜ ˜ (X ) is for some measurable function.14) Page: 205 job: prob macro: svmonob. g : X → R. y ) dν (y ) if ρ ¯ (X ) ∈ (0. X is independent of G . If X is independent of G and f : X → R is a measurable function such that f (X ) ∈ L (Ω.

10) is an example of a regular conditional distribution of Y given X. y ) dν (y ) = 0 Y (18.17) Page: 206 job: prob macro: svmonob. then a conditional distribution of Y given X will always exists.15) holds when ρ ¯ (x) = 0. f (X. we know that ρ ¯ (x) < ∞ for µ – a. ·) : N → [0. if we let y0 ∈ Y be an arbitrary but ﬁxed point and then deﬁne 1 ¯ (x) ∈ (0. Suppose that (X. ∞) ¯(x) Y f (y ) ρ (x. y ) dν (y ) dµ (x) (by Eq. we should have Q (X. if we require Y to be a “standard Borel space.e. then ρ (x.13. f ) = E [f |X ] a. N ) be measurable spaces and X : Ω → X and Y : Ω → Y be measurable functions. regular conditional distributions do not always exists.s. it is known that all “reasonable” measure spaces are standard Borel spaces. f (x. This is clear for simple functions and Y then for general functions via simple limiting arguments. M) and (Y. f ) a.e. So in most instances of interest a regular conditional distribution of Y given X will exist. Y is isomorphic to a Borel subset of R). Q (x. N ) for each x ∈ X and 2. Q.) Just for added security. we way that Q is the regular conditional distribution of Y given G . on X × Y such that the two assertions are valid. 1A (x. and there exists a regular conditional distribution.cls date/time: 23-Feb-2007/15:20 . A probability kernel.s. Equivalently.s. B ) : X → [0. M ⊗ N ) → R. f (y0 ) if ρ ¯ (x) ∈ {0.12)). Moreover. ρ (x. y ) dν (y ) dµ (x) h (x) Y f (y ) ρ (x. Show: 1.14) will be valid no matter how we choose g (x) at points where ρ ¯ (x) = 0. Hint: let H denote the set of bounded measurable functions.3. y ) dν (y ) if ρ g (x) := ρ . According to Eq. ·)) = E [f (X.. (18. y ) Q (x. the function X x → Q (x. (18. Let (X. dy ) .21. f ) := f (y ) Q (x. X : Ω → X and Y : Ω → Y are measurable functions. When X = Ω and M = G is a sub-σ – algebra of B . dy ) is M/BR – measurable. Unfortunately. the ﬁrst is when ρ ¯ (x) = 0 and the second is when ρ ¯ (x) = ∞. This proposition shows that conditional expectation is a generalization of the notion of performing integration over a partial subset of the variables in the integrand.13) we have E [g (X ) h (X )] = X h (x) g (x) ρ ¯ (x) dµ (x) h (x) g (x) ρ ¯ (x) dµ (x) X∩{0<ρ< ¯ ∞} = = X∩{0<ρ< ¯ ∞} h (x) ρ ¯ (x) h (x) X∩{0<ρ< ¯ ∞} Y 1 ρ ¯ (x) f (y ) ρ (x. (Observe where that when ρ ¯ (x) < ∞. Q (·. 1] is M/BR – measurable for all B ∈ N . Q : X × N → [0. then P ((X. Moreover if ρ ¯ (x) = 0. Q. 1] is a probability measure on (Y. = E [f (Y ) h (X )] wherein we have repeatedly used µ (¯ ρ = ∞) = 0 and Eq. Exercise 18. If Q is a probability kernel on X × Y and f : Y → R is a bounded measurable function or a positive measurable function. ·)) dµ (x) = X dµ (x) Y 1A (x. then x → Q (x. Therefore. N ) be measurable spaces. ∞} then we have shown E [f |X ] = g (X ) = Q (X. A function. If A ∈ M ⊗ N and µ := P ◦ X −1 be the law of X.” (i. See Theorem 18. (18. f : (X × Y. M) and (Y. let us check directly that g (X ) = E [f (Y ) |X ] a. y and therefore f (y ) ρ (x.4 below for more details. f.12.. Remark 18. Whereas to compute the expectation. as desired. Q. B ) is a version of P (Y ∈ B |X ) for each B ∈ N .s. Let (X. 1] is a probability kernel on X × Y iﬀ 1. 2. However.15) and Eq. y ) = 0 for ν – a. y ) dν (y ) dµ (x) Y = = X f (y ) ρ (x. Deﬁnition 18. M) and (Y. Y ) |X ] a. ·) ∈ L1 (ν ) and hence the integral in the deﬁnition of g is well deﬁned. N ) are measurable spaces. deﬁned in Eq. This completes the veriﬁcation that g (X ) = E [f (Y ) |X ] a.206 18 Conditional Expectation There are two possible problems in solving this equation for g (x) as a particular point x. x and so the second problem is not an issue.14. y ) dν (y ) dµ (x) = 1. Since ρ ¯ (x) dµ (x) = X X Y Deﬁnition 18. The probability kernel.s.16) ρ (x. for all f ∈ Nb . on X × Y is said to be a regular conditional distribution of Y given X iﬀ Q (X. (18.e. ·)) is measurable and Q (X. (18. one should integrate over all of the variables. see Section 18. It also gives an example of regular conditional probabilities. Y ) ∈ A) = X Q (x. of Y given X. (18. (18. For all bounded measurable functions.

and |fn | ≤ g a. P ) and f ∧n is increasing. P ) . let gk := inf n≥k fn . Hence. 5. we have E lim EG fn : A = lim E [EG fn : A] = lim E [fn : A] n→∞ n→∞ n→∞ 18.s. up to sets of measure zero. increasing in n. 1. Find a regular conditional distribution of Y given X and prove E [f (X. Remark 18. 2. and |fn | ≤ g ∈ L1 (Ω. Conditional Dominated Convergence (cDCT). we know that fn → f in L1 (Ω. B . 3. Y )] for all x ∈ X. then EG f = lim EG [f ∧ n] ≤ lim EG [g ∧ n] = EG g a. n→∞ n→∞ and so EG still preserves order.s. The other properties are also easily proved by simple limiting arguments. it follows that F :=↑ limn→∞ EG [f ∧ n] exists a.e. If f : Ω → [0. the function F :=↑ limn→∞ EG [f ∧ n] exists a.2 Additional Properties of Conditional Expectations 207 Exercise 18. P ) . Indeed if 0 ≤ g ∈ Gb and f ≥ 0. If 0 ≤ f ≤ g. B . 0 ≤ fn ≤ fn+1 for all n. then EG lim inf fn ≤ lim inf EG [fn ] a. Regarding item 4. Item 2. Suppose that fn → f. above.s. Since EG is a contraction. P ) . again by two applications of the MCT. n→∞ k→∞ k→∞ Item 4.18.s.4 still hold for any B – measurable functions such that 0 ≤ f ≤ g.16. n→∞ n→∞ n→∞ Thus Eq. Item 1. n→∞ n→∞ Similarly by cMCT.s. ∀ bounded measurable f : X × Y → R. As usual it suﬃces to consider the real case. E [F : A] = lim E [EG [f ∧ n] : A] = lim E [f ∧ n : A] = lim E [f : A] . Let fn → f a.. (18. then by cMCT.2. For 0 ≤ fn . satisfying E [f : A] = E [F : A] for all A ∈ G . then EG fn is a.cls date/time: 23-Feb-2007/15:20 Page: 207 job: prob . by two applications of the standard MCT. we start with the fact that 0 ≤ g ± fn a. We have already proved property 2. P ) and Egn → Eg. and is. ∞] is B – measurable. then EG fn → EG f a. |fn | ≤ gn ∈ P L1 (Ω. Hence by cFatou. EG lim inf fn = lim EG gk ≤ lim inf EG fk a. P ) and hence in probability. Conditional Fatou’s Lemma (cFatou). E [f (X. Then by the DCT in Corollary 11. then then limn→∞ EG fn = EG [limn→∞ fn ] a. n→∞ n→∞ = E lim fn : A = E EG n→∞ n→∞ lim fn : A from which it follows that limn→∞ EG fn = EG [limn→∞ fn ] a.8. B . B . Proof.s. i.s. P ) a. Then gk ≤ fk for all k and gk ↑ lim inf n→∞ fn and hence by cMCT and item 1. F : Ω → [0. In addition we now have. If fn → f a.s. (18..s. Conditional Monotone Convergence (cMCT). for any A ∈ G . uniquely determined by as the G – measurable function. Suppose that. (with 0 ≤ g ∈ Gb ).s.19) EG1 EG0 f = EG1 lim EG0 (f ∧ n) = lim EG1 EG0 [f ∧ n] n→∞ n→∞ 4. almost surely. macro: svmonob.15 (Extending EG ). where hf (x) := E [f (x. we have for any A ∈ G .3 and further assume that X and Y are independent. Since f ∧n ∈ L1 (Ω. Y ) |X ] = hf (X ) a.4.2 Additional Properties of Conditional Expectations Theorem 18. that P = lim EG0 (f ∧ n) = EG0 f. EG [gf ] = lim EG [g (f ∧ n)] = lim g EG [(f ∧ n)] = g EG f a. Suppose that. with g ∈ L1 (Ω. Then following the proof of the Dominated convergence theorem. Moreover. and 6. ∞] . n→∞ Item 3. Y ) |X ] |x=X a. EG0 EG1 f = EG0 lim EG1 (f ∧ n) = lim EG0 EG1 (f ∧ n) n→∞ n→∞ = lim EG0 (f ∧ n) = EG0 f n→∞ and (18.s. Suppose again that 0 ≤ fn ∈ L1 (Ω.s. Properties 2. B . B . for all n.s.s.s. P ) . B ..18) holds and this uniquely determines F follows from Lemma 18. almost surely. of Theorem 18. B . it follows that EG fn → EG f in L1 (Ω. 0 ≤ fn ≤ fn+1 for all n. Y ) |X ] = E [f (x.s. Keeping the same notation as in Exercise 18.18) Hence it is consistent to denote F by EG f. gn → g ∈ L1 (Ω.s.

and ϕ : (a.38 (also see Lemma 7. It is easily seen that H is a linear subspace which is closed under bounded convergence. (18. Since this is true for all x ∈ (a. b) . n→∞ n→∞ Fn (x. Taking y = f and then taking conditional expectations imply.s. x∈Λ By Exercise 18. EG [ϕ(f )] ≥ EG ϕ(x) + ϕ− (x)(f − x) = ϕ(x) + ϕ− (x)(EG f − x) a. For n ∈ N. P – a. B . and hence it follows from Corollary 11. X0 ⊂ X where qr (x) ≤ qs (x) for all r ≤ s. Let ν := P ◦ X −1 . Proof. (k + 1) 2−n 1(k2−n . let F (x. Taking expectations of this inequality gives the desired result.cls date/time: 23-Feb-2007/15:20 . To prove this.20. b) → R be a convex function. let qr : X → [0. Corollary 18. b) (and hence all x in the countable set. Assume f ∈ L1 (Ω. f : R → R. X : Ω → X is a measurable function and Y : Ω → R is a random variable. f ) . Then using the basic properties of conditional expectation. By Theorem 11. Q. ∞ where the above equations hold a. M = 1(−∞. y ∈ (a.. −∞ ≤ a < b ≤ ∞. The case p = ∞ and p = 1 have already been covered in Theorem 18.s. t) is measurable. 1) F (·.208 18 Conditional Expectation n→∞ EG (g ± f ) = EG lim inf (g ± fn ) ≤ lim inf EG (g ± fn ) = EG g + n→∞ 18. M) is a measurable space and F : X × R → R is a function such that. B . the function. Let Λ := Q ∩ (a. R). We claim that Q is the desired probability kernel.1. ν (X0 ) = P (X ∈ X0 ) = 1.20) k=−∞ F x.20). b) .s.3 Regular Conditional Distributions lim inf n→∞ EG (fn ) in + case − lim supn→∞ EG (fn ) in − case.19.·) (B ) where µF denotes the probability measure on R determined by a distribution function. For t ∈ R.s. and ϕ(f ) ∈ L1 (Ω. f ) ∈ R is measurable and E [f (Y ) |X ] = Q (X. b) a. 1] is measurable. where ϕ− (x) is the left hand derivative of ϕ at x. EG maps Lp (Ω.t] : t ∈ R . Hence an application of Lemma 18. t) ∈ X × R and therefore F is also M ⊗ BR /BR – measurable. and apply Jensen’s inequality with ϕ (x) = p p p |x| to ﬁnd |EG f | ≤ EG |f | a. Page: 208 job: prob macro: svmonob. Lemma 18. on X × R such that E [f (Y ) |X ] = Q (X. For each r ∈ Q. t) : X → R is M/BR – measurable for all t ∈ R. P ) be a probability space. ·) : R → R is right continuous for all x ∈ X. Then there exits a probability kernel. and limr↓∞ qr (x) = 0 satisﬁes. is M ⊗ BR /BR – measurable.4. ·) is a distribution function on R for each x ∈ X. P .s. Then F (·. f : R → R. Proof. limr↑∞ qr = 1 and limr↓∞ qr = 0.17 (Conditional Jensen’s inequality). B ) = µF (x. B . We will ﬁnish the proof by showing that H contains the multiplicative class. F : R → [0. R) is a random variable satisfying. P . Cancelling EG g from both sides of the equation then implies lim sup EG (fn ) ≤ EG f ≤ lim inf EG (fn ) a. t) := Theorem 18. Let (Ω. Notice that Q x. ν – a.s. The conditional expectation operator. Theorem 18. Λ) we may conclude that EG [ϕ(f )] ≥ sup ϕ(x) + ϕ− (x)(EG f − x) a. b) .t] = F (x. such that X x → Q (x. Suppose that (X. for all bounded measurable functions. t) := 1X0 (x) · inf {qr (x) : r > t} + 1X\X0 (x) · 1t≥0 .19 shows F : X × R → [0. let H be the collection of bounded measurable functions.(k+1)2−n ] (t) . t) = limn→∞ Fn (x. Proof. (18. So now suppose. P ) into Lp (Ω. Then ϕ(EG f ) ≤ EG [ϕ(f )] a. let Q (x. Using the right continuity assumption. For each x ∈ X and B ∈ BR .2 when ϕ is C 1 ) ϕ(y ) ≥ ϕ(x) + ϕ− (x)(y − x) for all for all x.18. t) for all (x. B .s. it follows that F (x. Hence the set. M) is a measurable space.s.39 that x∈Λ sup ϕ(x) + ϕ− (x)(EG f − x) = ϕ (EG f ) a. and 2) F (x. limr↑∞ qr (x) = 1. Combining the last two estimates proves Eq. 1(−∞. then Proof. 1] be a measurable function such that E [1Y ≤r |X ] = qr (X ) a.s. P ) and the map remains a contraction for all 1 ≤ p ≤ ∞.31) and Figure 7. B .s.s. for all r ≤ s.s. f ∈ (a. b) – a countable dense subset of (a. 1 < p < ∞. Then F is M⊗BR /BR – measurable. P – a. Now let r ∈ Q and g : X → R be a bounded measurable function. EG f ∈ (a. t) : X → R is measurable for each t ∈ R and F (x. qr ≤ qs ν – a. Suppose that (X. f ) . 1] .s.

B ) + 1X\X0 (x) δy (B ) for (x. are all isomorphic to (0. M) is a measurable space and (Y. The main goal of this chapter is to prove every Borel subset of a Polish space is a standard Borel space. for all bounded measurable functions. we may let r ↓ t in the above equality (use DCT) to learn. f2 (x2 ) .22.cls date/time: 23-Feb-2007/15:20 . such that τ = τρ . we may conclude that Q X. MXn ) is isomorphic to (Yn . N ) is a standard Borel space. (18. Similar reasoning shows that f −1 is measurable as well.21) This shows f is measurable and by similar considerations. Mn ) and ∞ (Yn . Then and hence Q is a probability kernel on X × Y. Suppose (X. if (Xn . . M) and (Y. Page: 209 job: prob 18. we may assume that Y ∈ BR and N = BY . ⊗∞ n=1 Mn ) are ∞ (Y := n=1 Yn . P – a. For t ∈ R. M) is said to be a standard Borel space if (X. Deﬁnition 18. Corollary 18. Nn ) are isomorphic measure spaces. Lemma 18. Deﬁnition 18. B ) = 1X0 (X ) Q0 (X.s.21. on X × R such that E [f (Y ) |X ] = Q0 (X. Proof. Proof. let fn : Xn → Yn be a measure theoretic isomorphism of all n ∈ N and then deﬁne f (x) = (f1 (x1 ) . This result leads fairly immediately to the following far reaching generalization. t) g (X )] . Moreover. see Corollary 18.s. For the second assertion. On ﬁrst reading. M) and (Y. Again it is clear that f is bijective and measurable. where y is an arbitrary but ﬁxed point in Y. then Q (X. B(0. B ) = 1X0 (X ) E [1B (Y ) |X ] = E [1B (Y ) |X ] a. ⊗∞ n=1 Nn ) are isomorphic. 1) .18.s. with Xn ∈ M and Yn ∈ N . Proof. Theorem 18. E [1Y ≤t · g (X )] = E [F (X.20. x2 . see Corollary 18. If (Xn . N ) are measurable spaces such that ∞ ∞ X = n=1 Xn . t) = E [1Y ≤t |X ] a. Along the way we d will show a number of spaces. we may ﬁnd a probability kernel. Q. G ) × (Y. for all bounded measurable functions. Let us now deﬁne Q (x. (18. . 1(−∞. ) ∈ X. f −1 is measurable as well. A measurable space. we know that P (X ∈ X0 ) = 1. i. you may wish to skip the rest of this section. f : X → Y is the desired measure theoretic isomorphism. Clearly. Then there exits a probability kernel..t] = F (X.26. then −1 −1 f −1 (B ) = ∪∞ (B ∩ Yn ) = ∪∞ n=1 f n=1 fn (B ∩ Yn ) ∈ M. . N ) is a standard Borel space. Y) a. f : X → Y such that f (M) = N and f −1 (N ) = M.21 applied with (X. (0. since ∞ ∞ f −1 n=1 Bn = n=1 −1 fn (Bn ) ∈ ⊗∞ n=1 Nn for all Bn ∈ Mn and n ∈ N. (Y. f : X → Y is a bijection and if B ∈ N . Thus if we let X0 := {x ∈ X : Q0 (x.24. . Moreover if B ∈ BY ⊂ BR .s. then (X := n=1 Xn . Two measurable spaces. By Theorem 18. Moreover we also will see that the a countable product of standard Borel spaces is again a standard Borel space.25 (Polish spaces). Then there exits a probability kernel. on (Ω. f ) .32.21) shows 1 = E [1Y (Y ) |X ] = Q0 (X. including [0. N ) are said to be isomorphic if there exists a bijective map. τ ) which admits a complete metric. Suppose that (X. Q.35 below.. f ) . This completes the proof.a. B ) := 1X0 (x) Q0 (x. Since g was arbitrary. This shows that Q is the desired regular conditional probability. Suppose G is a sub-σ – algebra. This is a special case of Theorem 18. both f and f −1 are measurable. f : Y → R. f : Y → R. G ) and Y : Ω → Ω being the identity map which is B /G – measurable. For each n ∈ N. Q0 . f : R → R. Then deﬁne f : X → Y by f = fn on Xn .4 below. 1]. BB ) where B is a Borel subset of (0. P – a. t) 1X0 (X ) g (X )] = E [F (X. Taking f = 1Y in Eq. f ) . Y = n=1 Yn .s. M) ∼ = (B. In this case we say f is a measure theoretic isomorphism and we will write X ∼ = Y. NYn ) for all n then X ∼ = Y. on X × Y such that E [f (Y ) |X ] = Q (X. (X. let fn : Xn → Yn be a measure theoretic isomorphism. By deﬁnition of a standard Borel space. Therefore. 1) .23. macro: svmonob. M) = (Ω. (X. . Rd .. 1] .4 Appendix: Standard Borel Spaces Deﬁnition 18. P . Suppose that X : Ω → X and Y : Ω → Y are measurable functions.e. B ) ∈ X × BY . In this case Y may also be viewed to be a measurable map form Ω → R such that Y (Ω ) ⊂ Y. [0. for all bounded measurable functions. and RN . N ) such that E [f (Y ) |G ] = Q (·.s.4 Appendix: Standard Borel Spaces 209 E [1Y ≤r · g (X )] = E [E [1Y ≤r |X ] g (X )] = E [qr (X ) g (X )] = E [qr (X ) 1X0 (X ) g (X )] . ) with x = (x1 . 1] . and Y : Ω → Y is a measurable function. ρ.1) . . A Polish space is a separable topological space (X. . see Appendix 18. Y) = 1} .

.28 Suppose (X. Lemma 18. Let Ω := {0.cls date/time: 23-Feb-2007/15:20 . 1] . 1) ∼ = (0. By construction. Similarly if N ∈ N with N ≥ 2 and Λ = {2. If Λ = {1} . (18.2 −n−1 ] ∪ Λ. . R. To prove this it suﬃces. and B := σ (π1 . Proof. . 1] = n=0 (2−n−1 . 4.210 18 Conditional Expectation Proposition 18. (0. (a. is measurable it follows that g is measurable. 1} be projection onto the ith component. It is easy to see by that any bounded open. then N −1 Proposition 18. and (0.22) and so again it follows from what we have proved and Lemma 18. 1) = {0} ∪ n=0 (−2−n . 1} . . 2−n ]. 2−N +1 ∪ n=1 2−n . Let πa : X A → X denote projection operator onto the ath – component of X A (i.26. 1) ∪ Λ. πa (ω ) = ω (a) for all a ∈ A) and let M⊗A := σ (πa : a ∈ A) be the product σ – algebra on X A . x= ∞ ∞ n=1 2 j =1 −j γj (x) and rk+1 (x) = j =k+1 2−j γj (x) . b]. Since each component function. [0. 1). 1) ∪ Λ with the aid of the identities. Let us now show (−1. 1) to construct a map from [0. . 1] . 1} . B ) ∼ = (0. 1) . . 1] = {0} ∪ ∞ [−2 n=0 −n . therefore (2 −n . . . 1) . If ϕ : A → B is a bijection of sets and (X. M⊗B . . M⊗A ∼ = X B . b] . 1] ∼ = (0. X X −1 πb ◦ f −1 = πϕ is measurable as well. γ1 (x) := 1x≥2−1 and r2 (x) := x − 2−1 γ1 (x) ∈ (0. ) be the product σ – algebra on Ω. 1) = n=1 2−n . tan : (−π/2. −2−n ] ∪ [2−n−1 . and k N (0. 2−n−1 ∪ 2−n : n ∈ N x= j =1 2−j γj (x) + rk+1 (x) and (0. Let us now deﬁne g : [0. 1) ∪ Λ ∼ = (0. Similarly (0. } is a countable set. M) is a measurable space and A is a set. The map f : X B → X A deﬁned by f (ω ) = ω ◦ ϕ for all ω ∈ X B is a bijection with f −1 (α) = α ◦ ϕ−1 . 1) ∪ Λ ∼ = (0.30.26 and what we have already proved. 2−1 ). or half open interval is isomorphic to any other such interval using an aﬃne transformation. k (0. . then let γ2 := 1r2 ≥2−2 and r3 = r2 − 2−2 γ2 ∈ 0. b) . Let −∞ < a < b < ∞. ∞ for all k. . 1) = n=0 [2−n−1 . 2.to observe that ∞ Notation 18. We will begin by using a speciﬁc binary digit expansion of a point x ∈ [0.27. . 1) = 0. If a ∈ A and ω ∈ X B .2 −n ] . 1) ∼ = (0. 2−k (18. γ2 (x) . πj ◦ g = γj : [0. M) is a measurable space.26 that (0. 2−N +1 ] ∪ n=1 (2−n . 1) is isomorphic to (0. we ∞ construct {γk (x) . by Lemma 18. we have X X πa ◦ f (ω ) = f (ω ) (a) = ω (ϕ (a)) = πϕ (a) (ω ) . 2−n ) and (0. rk (x)}k=1 such that γk (x) ∈ {0. πi : Ω → {0. 1) → Ω by g (x) := (γ1 (x) . we can show (0. The assertion involving R can be proved using the bijection. closed. 2−2 . 1) ∪ Λ where Λ is a ﬁnite or countable subset of R \ (0. Working inductively. Similarly. then by Lemma 18. . 3. 1] ∪ Λ = (0. The following measurable spaces equipped with there Borel σ – algebras are all isomorphic. b). Proof. [a. Then (Ω. Finally if Λ = {2. 1] because ∞ ∞ (0. then X A . ) . 2−n−1 ∪ 2−n : n = 1. 1) ∼ = [−1. 1) . B(0. π2 . 2−n−1 ] ∪ Λ while N −1 (0. X X and πb are the projection operators on X A and X B respectively. [0.e. 1) → {0. N rk+1 (x) = rk (x) − 2−k γk (x) = x − j =1 2−j γj (x) ∈ 0. π/2) → R. −1 (b) showing f B A A B A B (−1. 1] ∪ Λ = ∞ and rk+1 (x) → 0 as k → ∞. (a. 1) ∪ {1} = (0. 1) → Ω.29. (0. (0. . .1) . To this end. 1} . . let r1 (x) = x. −2 −n−1 ) ∪ (2 −n−1 . 1) . Proof.23) Page: 210 job: prob macro: svmonob. N + 1} . where πa XA XB Thus πa ◦ f = πϕ(a) for all a ∈ A which shows f is measurable. 2−n ) and [−1. [a. . 1].

1} = Ω. j ) = ω (i) (j ) for all ω ∈ Ω = {0.18. let us observe that Ω d ∼ = {0. = {0. then f (g (x)) = x for all x ∈ [0.27 now implies Ω = Ω0 ∪ Λ ∼ = [0. γ2 (x) = 1 iﬀ ω2 = 1. 1). The following spaces are all isomorphic to (0. Notation 18. ⊗n=1 BAn .1}N (α) is measurable for all i. Suppose that (Xn .31. and f in injective on the range of g. we have πi ◦ g −1 : {0. 1) . 1} . MA is generated ˜ where B ˜ := ∞ B ˜ ˜ by sets of the form.e. The inN2 N (18. we see that ⊗∞ BA and where Bn = B n=1 n n MA can both be generated by the same collections of sets. b ∈ Q let d(a. To see job: prob macro: svmonob. d} . = {0. is generated by sets of the form.26 and Proposition 18. 1} g (ω ) (i.1] be Borel sets on [0. ) = (f1 (x1 ) . Our next goal is to show that any Polish space with its Borel σ – algebra is a standard Borel space. Hence it follows that r2 (x) = j =2 2−j ωj and by similar reasoning we learn r2 (x) ≥ 2−2 iﬀ ω2 = 1. 1) .d} This shows Ω ∼ is analogous. 1) ∼ = Rd and (0. So according to Corollary 18.1) → (Ω0 . i. Since single point sets are in B and Λ := Ω \ Ω0 = ∪∞ n=1 {ω ∈ Ω : ωj = 1 for j ≥ n} is a countable set. [0. the σ – algebra. it is not hard to show inductively that γj (x) = ωj for all j.4 Appendix: Standard Borel Spaces 211 Hence if we deﬁne f : Ω → [0. 1] and RN where both of these spaces are equipped with their natural product σ – algebras.. Proof. N N×{1. g (x) = ω. B[0. 1} ΩN −1 −1 πi ◦ g (α) = g (α) (i) (·) = α (i.24) verse. ) is easily seen to me a measure theoretic isomorphism when A is equipped with the product σ – algebra. 1) and to do this it suﬃces to show Ω d ∼ =Ω and Ω N ∼ = Ω. For a. N × {1.. 1] . it sufd ∼ N ∼ ﬁces to show (0. 1] ∼ = RN . .. Indeed N. 2.j ) {0 . An application of Lemma 18. Then f : X → A := n=1 An deﬁned by f (x1 .30. The proof that Ω d ∼ = {0. if ω1 = 0. if ω1 = 1 then x ≥ 2−1 and hence γ1 (x) = 1.. 1] such that there exists a mea∞ surable isomorpohism. Then g is a bijection and since π(i. d N (0. it follows that g is measurable. 1} N N N ˜= A∩B n=1 ˜n ∩ An = B n=1 Bn ˜n ∩ An is the generic element in BA . 1) ∪ N ∼ = [0.j Ω Ω from which it follows that πj ◦ πi ◦ g −1 = π {0. f is surjective. 1). . 1) ∼ = (0. For example.cls date/time: 23-Feb-2007/15:20 . On the other hand. . 1] by f = 2−j πj . ..2. i. f2 (x2 ) .2. B := n=1 Bn where Bn ∈ BAn ⊂ B[0.1] is the product σ – algebra on [0.1] . then (by Eq.1} ΩN πi N −1 N N2 N N N 2 → Ω = {0. . 1) and Rd for any d ∈ N and [0. using Proposition 18. fn : Xn → An . 1)) . B(0. 1). BΩ0 ) . 1} 2 2 N N and Ω N ∼ be deﬁned by = {0. . let g : Ω N → {0. N2 N×{1.27 we know that d N N (0. b) := 1 1 |an − bn | = |πn (a) − πn (b)| .29. 1)) ⊂ Ω0 . n n 2 2 n=1 n=1 ∞ ∞ . 1} .2. 1} ∼ so that γ1 (x) = 0. For example. 1} We may now complete the proof with a couple of applications of Lemma 18. .e. Let An ∈ B[0.23)) rk+1 (x) = 2−k which would contradict Eq. it follows that Λ ∈ B and therefore Ω0 = Ω \ Λ ∈ B . M := ⊗∞ n=1 Mn is again a standard Borel space.1) . Corollary 18. Hence we may now conclude that g : [0. j ) = πi. This shows g is injective. and N2 all have the same cardinality and therefore. 1} is given 2 N {0. 1} Page: 211 → Ω N .1) ∼ = (Ω0 .e. we may conclude that ⊗∞ n=1 BAn = MA .1 } N2 Ω Ω ◦ g (ω ) = πj πi (ω ) .1] .32. Hence g ([0.d} To reduce the problem further. Conversely if ω ∈ Ω0 and x = f (ω ) ∈ [0.d} ∼ N2 {0.. ∞ then X := n=1 Xn equipped with the product σ – algebra. g −1 : {0. In light of Lemma 18. N×{1. Alternatively. to g is given by g −1 (α) (i) (j ) = α (i. x2 . ⊗∞ n=1 BAn .33 Let Q := [0. Since ∞ ∞ Corollary 18. the range of g.. 1) = Ω = (0. We now claim that Ω0 := g ([0. . j ) . Indeed. . A ∩ B n=1 n with Bn ∈ B[0..26 and Proposition 18. consists of those ω ∈ Ω such that ωi = 0 for inﬁnitely many i. if there exists an k ∈ N such that γj (x) = 1 for all j ≥ k.. So. 1]N denote the (inﬁnite dimensional) unit cube in RN . (18. . B[0....31. 1} = {0. Proof. ·) and hence Ω Ω πj ◦ πi ◦ g (α) = α (i.22). . i. (18. The full induction argument is now left to the reader. 1) ∼ = [0. j ∈ N x= j =2 2−j ωj < j =2 2−j = 2−1 ∞ and hence ◦ g is measurable for all i ∈ N and hence g −1 is measurable. BΩ0 ) is a measurable bijection with measurable inverse given by f |Ω0 . Mn ) for n ∈ N are standard Borel spaces. ∞ ∞ The σ – algebra. then ∞ ∞ ∞ j =1 Ω this map is measurable. 1). to ﬁnish the proof it suﬃce to show ⊗∞ n=1 BAn = MA N where M := ⊗∞ n=1 B[0.

We have ¯ ∩ (∩∞ Un ) . G (y )) = 1 |ρ (x. Since Q is a standard Borel space so is Q0 and hence so is X. x) k→∞ k→∞ and therefore x = y. d) and hence X ∼ = Q0 . ρ (x. y ) = d (G (x) . is also complete. an ) = 2ρ (x. d) and for each n ∈ N let Let S Nn := {N ∈ τd : diamd (N ) ∨ diamσ (N ∩ S ) < 1/n} and let Un := ∪Nn ∈ τd . Corollary 18. It is easily seen that d is a metric on Q which. d). Moreover if the metric. i. an ) + ρ (y. b) . ρ (x. G (y )) = ρ (x.) By replacing ρ by 1+ ρ if necessary. an ) − ρ (y. . (18. y ) for all x.e.cls date/time: 23-Feb-2007/15:20 .5 ρ on p. y ∈ X. αk ) = lim ρ (y. 106. X. From the previous paragraph. an )| n 2 n=1 ∞ Now suppose that (X. m−1 → 0 as ∞ m. we may choose n so that 2ρ (x. This shows τγ = τρ . there exists un ∈ N1 ∩ · · · ∩ Nn ∩ S for each n ∈ N. for x.5.5). y ) is small. ε) . and therefore S ⊂ S n=1 ¯ ∩ (∩∞ Un ) and n ∈ N. v ∈ A} and diamd (A) := {sup d (u.24) is measurable relative to the product σ – algebra. ρ (x. Proof.34. δ (ε)) ⊂ Bd (u. γ (x. Therefore. we must show G is injective and γ is a metric on X which is compatible with the topology determined by ρ. σ ) to some element u0 ∈ S. then ρ (x. ) ∈ Q and γ (x.. y ) . Page: 212 job: prob macro: svmonob. Conversely. we may assume that 0 ≤ ρ < 1. A simple argument using the dominated convergence theorem shows y → γ (x. ¯ denote the closure of S inside of (Q. From the deﬁnition of Nn . any separable metrizable space X is homeomorphic to a subset of (Q.e. Solution to Exercise (18. Every Polish space. 1] is continuous. M contains all open balls and hence contains the Borel σ – algebra. by Eq. an )| ≤ 2ρ (x. there exists a continuous injective map G : X → Q such that G : X → G(X ) ⊂ Q is a homeomorphism. σ ) is a complete metric (being the isometric image of a complete metric space) and by what we have just prove. Hence if ε > 0 is given. To every separable metric space (X. then G (X ) is a Gδ –set. ∞ Let D = {an }n=1 be a countable dense subset of X and deﬁne G (x) = (ρ (x. we may ﬁnd δ (ε) such that Bσ (u. d) is the same as the product σ – algebra. Theorem 82. Consequently. v ) : u. un ) = 0 and σ (un . To prove the ﬁrst assertion. v ) : u. since |πn (a) − πn (b)| ≤ 2n d (a. Consequently and Borel subset of X is also a standard Borel space. G is a homeomorphism. it follows that S ⊂ Un ¯ ∩ (∩∞ Un ) . Since D is a dense subset of X. d) and if X is a Polish space then X is homeomorphic to a Gδ – subset of (Q. an ) + ρ (y. Moreover. y ∈ X. we have diamd (Bd (u. Theorem 18.212 18 Conditional Expectation Exercise 18. Conversely. it follows that {un }n=1 is convergent in (S. This completes the proof because we may now shown. an ) + 2n γ (x. S = S n=1 ∞ ¯ ¯ write S = n=1 S1/n where S1/n := u ∈ Q : d u. Let S := G (X ) and σ be the metric on S deﬁned by σ (G (x) . y ) ≤ ρ (x. if u ∈ S and ε > 0 is given. Since (S. In short.34 shows that X is homeomorphic to a measurable (in fact a Gδ ) subset Q0 of (Q. (This proof follows that in Rogers and Williams [4. ε) . d) . ρ). Show d is a metric and that the Borel σ – algebra on (Q. y ) is small if ρ (x. σ ) is complete. δ (ε))) < ε and diamσ (Bd (u. a2 ) . we may choose αk ∈ D such that 0 = lim ρ (x. the G (X ) is the countable intersection of open subsets of (Q. an ) < ε/2 and so if γ (x. n → ∞. a) = ρ (y. Taking δ (ε) = min (δ (ε) . d) is isometric. y ) < ε. B . an ) + |ρ (x. dS ) has the same topology as (S. an ) − ρ (y. i. ρ) is a complete metric space. M. σ ) it follows that d (un . Theorem 18. Since (S. Therefore each ∞ πn is B – measurable and hence M = σ ({πn }n=1 ) ⊂ B . ρ. .]. since N1 ∩ · · · ∩ Nn is an open neighborhood of u ∈ S. αk ) = ρ (y. v ∈ A} . an ) ≤ 2ρ (x. an ) − ρ (x. each of the projection operators. S < 1/n and therefore.35. u0 ) → 0 as well and thus that u = u0 ∈ S. a1 ) . Proof. there exists Nn ∈ Nn such Conversely if u ∈ S n=1 ¯ that u ∈ Nn . we have limn→∞ d (u. um ) ≤ max n−1 . . a3 ) . If G (x) = G (y ) . τσ = τdS . a) for all a ∈ D. y ) is ρ – continuous. Since G : (X. Then (S. with its Borel σ – algebra is a standard Borel space. δ (ε))) < ε where diamσ (A) := {sup σ (u. πn : Q → [0. γ ) → (Q. ∞ ∞ S = ( n=1 Un ) ∩ n=1 S1/n is a Gδ set. it will follow that ρ (x. y ) < 2−(n+1) ε.

Springer-Verlag. Markov processes. L.. Probability: theory and examples. Probability theory. 2002. New York. John Wiley & Sons Inc. MR 2001g:60188 5. and martingales. Rogers and David Williams. 1996. MR MR1609153 (98m:60001) 3. New York. 2000. 7. R. Cambridge. Richard Durrett. MR MR1852999 (2003a:60001) . Belmont. Foundations. C. Diﬀusions. Cambridge Mathematical Library. 2001. Wiley Series in Probability and Mathematical Statistics.References 1. G. second ed. Probability and measure. MR MR1324786 (95k:60001) 2. Duxbury Press.. vol. Patrick Billingsley. second ed. Olav Kallenberg. Reprint of the second (1994) edition.. S. New York University Courant Institute of Mathematical Sciences. Courant Lecture Notes in Mathematics. Vol. MR MR1876169 (2002m:60002) 4. Foundations of modern probability. Probability and its Applications (New York). S.. 1995. third ed. 1. A WileyInterscience Publication. New York. Varadhan. Cambridge University Press. CA.

Dirichlet Principle

Los Perros de Tindalos

Ayuda Ntia 1

Remuneración

C1_Mat021_2008-1

tarea3

Precertamen_MAT_024_-_2013_1°_-_1A_Pauta

MAT270 C2 1S 2006

Cert_2_Mat_270_2°_2012

Pre Certamen 2 (Dificultad Normal)

apuntes_ppt1

20122IWN261V2 Caso Privatizaciones

20122IWN261V2 Ley de Aguas

20122IWN261V2 La Crisis

Guía 1

FORMULARIO 2012-2013

Ayudantia 1 EA

Ayuda Ntia 5

Guía 1

Ayudantía 1

Another Note - El Caso Del Asesino BB de Los Angeles

AuxCurvas - Copia

Int Multiple

Ayudantia 1

- Read and print without ads
- Download to keep your version
- Edit, email or read offline

Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

CANCEL

OK

You've been reading!

NO, THANKS

OK

scribd