Advanced Probability Theory Bing-Yi JING Department of Mathematies, Hong Koug University of Science and Technology Email: majing@ust.hk July, 2012Contents 1 Set Theory LL 12 13 La Ls 16 Lr 18 19 1.10 Sets: Basie set operations Operations of sequence of sets . Indicator fmetions Semiralgebras, Algebras, and o-algebras 1.5.1 Definitions 1.5.2. Relationships. 1.5.3 Some special o-algebras. 1.5.4 How to generate algebras from semi-algebras Generated classes (Minimal classes) . . Monotone class (rm-class), m-class, and L-elass 17.1 Definitions eee 1.7.2 Relationships with o-algebras 1.7.3 Minimal moelass, Avclass and class. 1.7.4 Graphical illustration of different lasses ‘The Monotone Class Theorem (MCT) Produet Spaces Exercise 2 Measure Theory 2 22 28 24 25 26 ar 28 29 2.10 Definitions Properties of measure 22 2.2.2 Case Th algebras, IL Case b: semialgebras, . 2.2.3 Cose IL: o-algobres. Arithmeties with 26 Probability measure. Some examples of measure Extension of set functions (or measures) from semialgebras to algebras ‘Outer measure Extension of measures from semialgebras to o-algebras ‘Complotion of « moasure Const 210.1 General procedaros rction of measures on a a-algebra A 2.10.2 Lebesgue and Lebesgue-Stieltjes mensures 10 20 RBRBSSS 25 28 aL 35 363 2.10.3 Relationship between probability measures and distribution functions 2.10.4 Different types of distributions 2.10.5 Decomposition of distribution fanetions 2.10.6 Further decomposition of a continnous df. F 2.10.7 Some examples of distributions 2.11 Racon-Nikodym theorem 2.11.1 Applications 2.11.2 Propertios 211.3 The essumption of sigmafiniteness 2.12 Exercises Random Variables 3.1 Mappings 3.2. Measurable mapping 3.3 Random Variables (Vectors) . 3.3.1 Random variables 3.3.2 How to check a random variable? 3.3.3 Random vectors 34 Construction of random variables ALL Algebraic operations (4,—,%, +) 34.2 Limiting operations 3.4.3. ‘Transformations 3.5. Approximations of rv. by simple rvs 3.6 c-algebra generated by random variables, 3.6.1 Definition 3.6.2 Some examples 36.3 The o~field generated by a continnous 1. 3.64 A useful theorem 3.7 Distributions and induced distribution functions STL Case E Random variables 3.7.2 Cose Ul: Random veetors . Generating random variables with prescribed distributions Exercises Expectation and Integration AL Bxpectation 4.1L Expectation for simple rvs. 4.1.2 Expectation for uennegative rvs 4.1.3 Expectation for general rv.'s 4.1.4 Summary 4.2 Integration 42.1 Definition 4.2.2 Some properties of integrals. . 4.2.3 Special cases: Lebesguo-Stieltjos integral and Lebesgue intogral 4.3. Measure-theoretic and probabilistic languages 4A How to compute expectation 4 43 45 49 76 7 79 0 SL a st44.1 Expected values of absohitely continuous r.v.'s 44.2 Expected values of discrete r.v.'s 4.5. Relation between expectation and tail probability 1.6 Mo 4.6.1 Young's inequality 4.6.2 Holder's inequality 46.3 Couchy-Schwrarz inequality 4.6.4 Lyapuney’s inequality: ts and Moment inequalities 4.6.5 Minkowski’s inequality 4.6.6 Jonson's inequality 4.6.7 Chebyshev (Markov) inequality 4.7 Exercises Independence 1 Definition Hovr to check independence S21 Discrete nw 's 5.2.2 Absolutely continnous r.v.'s Functions of independent nv.'s- 5.4.1 Transformation properties 2 Convoluiions 5.3.3. An example 534 Conelation 5A Borel-Cantelli Lama and Kolmogorov @— 1 Law 5A. Borel-Cantelli Lemma 54.2 Kolmogorov 0-1 Insis 5.5 Exercises Convergence Concepts 6.1 Modes of convergence 6.1.1 Equivalent definition of as. convergence 6.2 Cauchy Criterion 6.3 Relationships between modes of convergence GA Partial converses 6.4.1 Convergence in probability and distribution to constants are equivalent 6A.2 Dominated convergence in probability implies convergence in amen 6.4.3 Dominated convergence a.s. implies convergence in ancan 6.4.4 Convergence in probability sufficiently fast implies a's. convergence 6.1.5 Convorgonco in man sulficiently fast implies a.s. convergence 6.4.6 Convergence sequences in probability contains as. subsequences s8328 100 100 100 101 102 104 104 106 109 ut m1 12 113 ha 15 15 16 7 his us ns 6.4.7 Convergence in distribution plus uniform integrability implies convergence in moments 118 6.4.8 Convergence in prob plus uniform integrability implies convergence in mean, 6.4.9 Convergence in distribution implies a.s. convergence in another probebility space, 6.5 Convergence of moments; uniform integrability 6.5.1 Definition of uniform intezeability (w.i.) 6.5.2 Convergence in prob. + wi, —> convergence in moan 1s 19 120 120 110 6.5.3 Relationship between L, convergence and convergence of moments 6.5.4 Convergence in dist. + wi, => convergence in mean 6.6 Some closed operations of convergence 6.6.1 Algebraic operations 6.6.2 ‘Transformations (Continuous Mapping) 6.6.3 Sluisky’s Theorem 6.7 Simple limit theorems 6.8 Faton’s Lemma Revisited 6.9 Summary: relationships amongst four modes of converges. 6.10 Exercises ‘Weak Law of Large Numbers 1 Equivalent sequences; truncation 7.2. Weak Law of Large Numbers 7.3 Classical forms of the WLLN TA Exercises Strong Convergence $1 Some maximal inequalities 8.2 The ns. convergence of serias; threeseries theorem: 8.2.1 Review: Cauchy convergent a.s. or in probability 8.2.2 Variance criterion for random series 8, 8.2.4 Kolmogorov two series theorem for absolute random series 83° Strong Laws of Large Numbers (SLLN) 8.3.1 Sovoral useful lemmas: 8.3.2. SLLN for independent rv.'s 8.3.3 SLLN for iid. rv’: necessary and sufficient moment conditions 84 Some applications of the WLLN and SLLN &.5 Some results on LIL Kolmogorov three series theorem for random series. - 8.6 Exercises Weak convergence 9.1 Definition 9.2 Equivalent definitions of weak conver 9.8 Holly’s selection theorem and tightness 9.4 Polya Theorem 9.5 Additional topic: Stable convergence and mixing* 9.6 Exercise 9.7 Some useful theorems Characteristic Functions 10.1 Some examples of characteristic functions: 10.2 Definition and some properties of ets 10.3 Inversion formula 1044 Levy Continuity ‘Theorem . 10.5 Moments of r-v.8 and derivatives of their e.£s 123 124 125 125 126 127 128 130 130 132 134 135 13s 140 141 ui M4 M6 ui Lag 150 150 Wt 14 Ls 159 160 162 162 162 165 168 169 172 172uw 12 10.6 When is 9 fimetion a c.f? 10,7 Bsseen’s Smoothing Leama . 10.8 Characteristic functions and smoothness condition 10.9 Exercises Central Limit Theorems and Related Expansions ALA Central Limit Theorems (CLT) AAA CUT for iid. rvs 11.2. CLT for triangular arrays with finite variances 11.4.3 Central Limit Theorems with infinite variances 11.2 Uniform Borry-Ksseen Bounds 11.2.1, Some useful Temunas 11.2.2 Bary-Hsscen bounds . 11.2.3 A generalization of Berry-Rsseen bounds 11.3 Non-Uniform Borry-Bsseen Bounds . . 114 Edgeworth Expansions 114.41 Heuristic argument for informal Bdgeworth expansions 11.4.2 Hennite polynomials 114.3 Relationship between moments and cumulant LAA Edgeworth expansions 11.5 Nomuniform Edgeworth expansions 11.6 Largo Deviations 11.6.1 Cramer conditioin 11.6.2. Conjugate (or associated) distributions . 11.6.8 Large Deviations 11.64 Cramer-type large deviations LT Saddlepoint Approximations 11.8 Appendix: Some useful elementary inequali 11.9 Exercises Infinitely divisible distributions 12.1 Definitions 12.2 Some examples 12.3 Some properties eves 12.4 Lovy-Khintchine representation of infinitely divisible es 124.1 Proof of necessity 12.4.2. Proof of suficieney 12.4.8. Several examples 12.5 Levy formula 12.6 Kelimogorew fovemala 12.7 Relationship between the sun of independent rv. and id... 12.8 Exercises 190 191 194 197 199 199 199 200) 203) 203, 204 206 ons 208 209) 210 a0 2u au 216 as 218 219 220) 221 223 226 2s 229 229 29 230 231 232 234 235)at lid wart, W.LOG. R Tor HA} 1 Notation and abbreviations almost surely distribution fumetion independent. and identically distribuced infinitely often with respect to without loss of generality’ the set of real numbers. indicator fumetion of the set A end of a proof Unless otherwise specified, all limits are talen as r2—+ 20. limsup |an/bn| <0 ag X dy SS gfby +LChapter 1 Set Theory 2: a space (a nonempty reference set) 1: aa empty set. A (sub)set is a collection of objects (or elements) in ©, denoted by A, B,C. A class (or family) is « collection of subsets of , denoted by ABC DELFI, WEA: wisanclement of A. ACB: tho sot A is contained in the sot 8. Convention: 0. A for any set A 1.2 Basic set operations Let A and B be subsets of 2, define 4° = (w:w@ 4} (complement) AUB = [wiw€A,orweB} (union) ANB = (w:we A, and we B} (intersection) A-B = (wiweA,andw¢B} (difference) AAD = B)U(B— A) (symmettie difference) (Note: sometimes we write AN B = AB. the product of A and B.) Venn diagrams are often helpful.1.3. Operations of sequence of sets Lot Ay, Ao, be subsets of ©, denote their umion and interseetion by URadn = {ww € Ay for some x} OR yay = wut © Ay for all n} If Ay, Ap... are (pairwise) disjoint, hem we wribe Unda = Ae Definition: Let A, Ay, Ap... be subsets of @. Define (@) Infinitely often (i0.) Jimsup4, = Tim Ay = Or Uy An = (e:VE> Len > halos Agh = {w:W © Ay for infinitely many values of n} = {Ay io} (b) Ultimately (ult) limint A, = lina, An = UE dn = (e:DkELV nz hstwe Add fi: € Ay, for all but finitely many valves of n} = {Any ult). Jim, so. dy oF simply Ay, A iE (6) The sequence {4,,} converges to A, written as Tim int Ay = Tiasup Ay = A ‘THeoReM BA We have liminf Ay ¢ lim sup Ay Proof. By definition, linyinf A, = (0:0 € 4p for all but finitely many values of m} {w:u © Ay, for infinitely many values of n) = limsupay. 9 Example. (liming, Ap and limsup, Ay may not be equal). Specify liminfy Ay ond limsup, Ay when Aaj = B, and Ags = C, 9 = 1,2, Solution. Cleasly, Timinf Ay = U1 AEs An = UE(BNC) = BAC, Ay =NEA(BUC)= BUC limsup Ay = fy UETHEOREM 1.3.2 (Monotone sequence of sets converges) W) ALC Ad © Ata then An + A= URE An, written as Ay 1A. 8) If AyD AyD Agen then Ay + A= NZL, Agy triton as Ay | A Proof. We shall only prove a). Lat A= URL; Ay. Clearly, limint 4, = (Lv An) = UR An = A limsup A, = (Ug An) = 1 (UR) An) COROLLARY LBL Let Ay, Az, Be subsets of O. {Ant0.} = lisp Ay = ln UF pn {Anyult.} = Timjnf Ay = Jim Aa Proof, We shall only prove the first one, Let By = US.,Aq. Clearly, By 2 By 2 By.... From the last theorem Fi Be = OR, URa An = limsup dn. lim By = 1.4 Indicator functions Let AC Q define Talw)=Twe eA} = 1 forwe A 0 forwe Ae ‘Thus, 14 “indicates” whether A occurs or not, depending ou whether it is Tor 0. Therefore, the uext theorem is obvious. The indicator function transforms operatious of sets into algebraic operations of 1's and O's, which are often easier to deal with THEOREM L4.1 VA, B CO, we have A=B = las Ta, (meaning E4(se) = Ig) for all w € 9) ACB ~ Is < Jian A=O or 0, resp. = Iy=0 or 1, resp Proof. Let us prove the fitst one only. Clearly, A= B implies that Z1(2) = Ta(w) for all w 9. On the other hand, if A B, then =u € A but w ¢B, or dw € B but w ¢ A. In cither case, for that w, we heve Ia(4) # In(w). The proof is completed. ‘Teor 1.4.2 Tae = win{ly,!e} = dale Tap = mex{lato} = L4+ Io — Lace = Intle-lle In = 1-k Ean = Janne = IA In)tase = Va-Ip Fiwinte Ay = Hint Fa, Fina, ty = imsup ls Tap < Iatlo Proof, We shall only prove a couple of these relationships. (1) First we show that Laup = max{I4, 1p}. Vw € 9. if Laua() = 1, then » € AUB. So either we Aor € B. Socither f4(0) = 1, or Ip(o) = 1. Thus Zaye (o) = max{Z4(2), f9(w)}- Similaely, we ‘ean show that this is trae if Laue) = 0. Thus, we proved Tauy = max{ Ta, la} (2) Noxt wo show that Juniuts ay — liminfy La, Ye © 1, if Imtaty a () = 1, thon o Timinty, Ay, = {An,ult.}. That is, wis contained in all but finite many vales of Ths, [4 (2) L except for a finite many values of n. Tt inuplies that liminf, Fa, (0) = litmy [4,(0) = 1. On the thor hand, if Jima int. an() = 0, then w € (lim infy Ay)® = limsup, As, (by using the DeMorgan Law) = {Aj vio}. Therefore, Lag (we) = 11.0. and so bsp, £4; 48) = 1. Henee, linsupy[l ~ Za, (6)] = 1 fT, () = 1. So lim int, T4, (0) = 0= gangs tg (3). The proof is complete. (3) Finally, we shall prove [4 (2) = [[a(0) —Ip()|- Consider d eases: Var € 0, (2) L() = 1, Lol) = 1, then RES = 0. Ako we note that w € A and w € B,thnsw € ANB. That is, » ¢ AAB, implying LITS=0. So LHS = RIS =0. Similarly, we ean show the following (b) L4{w) = 1, Tg(w) = 0, then LHS=1=RHS. (6) Taw) = 0, T(w) = 1, then LHS—1—H1. (4) 14(u) =, Jaw) = 0, thon LHS=0=RUS. ‘This completes the proof. Indicator functions can sometimes facilitate set operations, as illustrated below Example. Show (AAD)AC = AA(DAC). Proof. Note Thus = aap ~ Tol = |\a — In| — Jol Trans =|[4 ~Tnae|= |Fa ~|To ~ Tell If Tg =0, then Tews = |L4—Ie| = Tens If fy = 1, then Imus Mp —to|—Lal= | to-tal= tins. Tuconen 14.3 hey = Sy ty SS tayeasoay (AY Tayratinninaty =y L Proaf. Set s1 = S38 Ta,, hy Adygy wos Sx = Tay ayrinntys Thea we need (0 show Tupay = 81-82 +89 + (=I) sy (4a) In proof of this, if for some w € ® 1up.,(u) =0, clearly sue) =0, 1 < A 1 (ie, Als closed under complement ad courtable union.) ‘The pair (0 A) is called a measurable space. ‘The sets of A are called measurable sets. [Note: ¢ or = (countable) sum” = “(countable union” RPMARK 1.5.1 IA is an algebra (or a o-algcbra), then 0 € A and 2€ A. However, the same may not hold for semi-algebras. Proof. Assume A is a nonempty algebra or o-algebra. JA € A, so AY € A, Thus, 2 = AUAPE Aand P= 9 €A. For the second part, consider $= (0A, A°}. Its clearly a semialgebra, bub does nol contain 2.REMARK 1.5.2 A ison algebra <> (1). 9€A. (2). A,B EA implies A“ BEA. Proof. "=" part is easy. We now show "=" part. Let 4,B © A Since 9 © A, thus A =O-AEA Also ADB=A-(A-B)EA as (A-B)EA rats and finite intersection thus union, Then is an algebra, So A is closed! under compl . 1.5.2. Relationships An algebra is a semialgebra, and a g-algebra is an algebra, The reverse may not be true, See the next ‘two examples, Example 1: A semi-algebra may not be an algebra. Lot 9 = (90, 5]- Lot $ be a collection of half open intervals in Q, i0., S={(a,)] Sachse} Then S is a scmi-elgcbra, but NOT an algebra, (It is easy to define Lebesgue measure (ie, length, im this ease} on S, and then goneralize to the Borel 2-alyobra.) Proof. For the first part, uote that (a,a] = 0 ¢ S. It is easy to chock that the interscetion of ‘two half open intervals are still half-intorvals (Including @). So it is closed undor intersection. On the other hand, if A = (a,b) € &, thea Av = (00, a] + (4,90), the union of two disjoint half open intervals, Thus, S is a semialgebra. For the second part, note that (7,2 + 1] €.S, but (0,1]U (2,3 ¢S. So it is not closed under finite union, thus not an algebra, Example 2: An algebra may not be a c-algebra. 00,08). Define s {Ufa (ai, bi) = (UP lea (After ordering and merging of intervals.) Again, let 0 co Say Sb, Soo} 00 S01 Sh $02 S daw S dn $20) ‘Then S is an algebra, but NOT a g-algebra, Proof. ‘The first part is easy. For the second part, note that UEAO.1=1/n] = 0,1) 23, or as another example UEaln— ant aya) gS. So it is not closed minder countable union, thus not a algebra, 1.5.3 Some special c-algebras. (1). Trivial c-algebra: {0,1}. This is tho smallest o-algebra (2), Power sets = all subsets of 2, deuoled by P (2). This is the largest o-algebra, often too big to define probability (3), The smallest c-algebra containing Ae: {0,4, 4°.)1.5.4 How to generate algebras from semi-algebras THEOREM L5.1 If S is a semo-algebra, then S = [finite disjoint unions of sets in 8) ts om algebra, called the algebra generated by S. We sometimes also denote S by A(S) Proof, Let A,B €S, thon A= 22, A, and B ‘the definition of S. Thus, 1 By with Ay By €S. Clearly, Ai By € S by ANB= SY ANB 8. So S is closed under (finite) intersection As for complements, uote that Av = (Sy Ai)’ = O%L,4f. The definition of $ implies Af € S. But we have just shown that S is closed under (finite) intersection, s0 A° € 5. So S is also closed under complement, ‘This completes our proof. REMARK 1.5.3. It is easy to define (Lesbeque) measure on the semialgebra S, and then easily extend it fo the algchra S. Can we extend it further to some o-algebra, We shall see later that the answer to this question. is affirmative, Since there are many o-algebrus, ve shall first consider the smallest one containing S. This 1s our nert topic: the c-algebra generated by S, or more generally by any class, 1.6 Generated classes (Minimal classes) (Probability) Measure is defined on e-algebras. Given a class A of subsets of Q, there alvays exists a unique minimal a-algcbra containing A. Minimality is important sinco the power set P(M) is too lange unless @ is finite oF countable. LEMMA 1.6.1 Let (44, 147 €L'} be a collection of a-algebras. Then A=AyarcAy 8 also a o-elgebro. Jie., o-algehras are elased under (possibly uncountable) intersection.) Proof, (1). A&A iunplies Ae A, for ally € T, which in tum implies AS € A, for ally € T ‘Therefore, A® = A, That is, A is closed under complenents, (2). WA, € ALi > 1, then Ay € Ay for all (> Land all y EP. Hence, USA, € Ay for all 7 € P $0 UE ,AL € Merl, =A. So A is closed under countably infinite union. This completes the proof. THEOREM 1.6.1 For any class A, there exists @ unigue minimal a-algebra containing A, denoted by o(A), called the a-algctra generated by A. Tu other words, @) AcolA) 4) For ony o-algebra B with ACB, oA) CB, and o(A) is unique, Proof. Existence. Let Qa = (B:B > A, B is a o-nlgebra on Q}. Clearly. Q. is not empty since it contains the power sot P(A). Define a(A\= [) &. Beda From the above lemma, a(A) is a o-algebra. Uniqueness. Let 2}() be another a-algebta satisfying (a) and (b). By definition, a(4) 046) By symmetry, o)(A) c o(A). Thus, oA) =03(4).REMARK 1.6.1 Similar theorem holds for algebras, monotone classes, A-class, and x-class, see later sections. THVOREM 1.6.2 is a aemi-algebro, and'S = A(S) is an algebra generated by S. Then a(S) = ofS). Proof, By definition, § CSc e(S). $0 o(8) is a a-alyebra containing &, thus o($) < (8). We now show that oS) C o(S). Kor any given A eS, by dafnition, we have A= S71, A, where A, © S.C 9(S). Therofare, A 20(3). This shows that Sc a(S). So a(S) bs a onalgebra tontaining 5. thus a(S) C o(S). This completes our proof. We now introduce the mest important o-algebra in measure theory. Definition: The smallest o-algcbra generated by the collection of all finite open intervals on the real line R= (26,20) (or R= (—26,00] oF R = [=90, 50] as in Chovr anc Teicher, page 11.) is ealled the Borel g-algebra, denoted ly B. The elements of B sue called Borel sets. ‘The pair (R,B) i called the (I-dinensional) Borel measurable space. REMARK 1A (1). Boery “reasonable” subset of Ris « Borel set. However, o(R) # PUR). (2). Kedimensional Borel a-algebra, Borel sets and mensurable space can be similarly defined. (3). Borvl o-elacbra can be generated by many other ways. For instance, i can be gencrated by the class of all finite closed intereals, or the class of all open vets, or the class of all closed sets (4). For AEB, let By = (BNA: Be B}=BNA Then (A,B) is a mensuratie space, and By is enti the Borel a-algebra om A Example: Borel o-algebras gencrated by semialgebras. (1), Let Ry = (ce, 99) (the real line), and define $= {A: A= Mo or (0,8) oF (00,8) oF (a,c), abe R} Then S) is # semisalgebra on Ro. The o(Sp) is called the Borel c-algebras on the real Tine R. (2). Let Ry aud define 0] (half exteuced real li S.= (ab), a,b Ri). ‘Then S is @ scmi-algebra on 2), The a(8)) is called the Borel c-algebras on the half extended real line Ry. (3). Let Re .20, 0] (the extended real Lino), and define Sy ={[a.b), abe Ro} U{R,fo0}. {oo}. ‘Then Sy isa semialgebra ou Ra. The a(S) sealed the Borel e-algebras on the extended real line Note that the introduetion of 90 and —20 simplifies the stuetures of semialgebras,1.7 Monotone class (m-class), m-class, and )-class Checking A to be a a-algobra directly may not be easy in practice. ‘This can be made easier by introducing, new ad more prituitive classes sth as monotone classes, r-rlasses, at A-classos. 1.7.1 Definitions Definition: Let A be a nonempty class of subsets of (1). A is said to be a monotone elass (rn-class) on £ sequence Ay €.A, > 1. That is if lim Ay, ¢ A for evory monotone (0) If Ay € A and A, f, then limy 0 Ay = UBL A: € A, (b) IPAc€ A and 4; |, then Tiny oc Ay = Epi © A, (1e., meclass is closed tinder monotone operations) (The existence of lim Ay, was established in ‘Thcorcm 1.3.2). (2). Aisa n-class on Dif AN BEA, whenever ABE A wx) (i.e. a weclass is closed under finite intersection. Note (3). Ais w A-class on Dif (i). 9EA (i). AB eA for A,B € A and B is « propor subset of A, (Le. BCA) (li). lim Ay € A for every incroasing sequence Ay € A, n > 1. (Ge. Aclass contains Q and is closed mder proper difference and countable in creasing union, Note 4 “imexeasing” limit). (A= lamda = lam +da~lim+diff. Also A= “Large”, see Theorem 1.8.3) Remark. The notion of a m-class and A-class eomes from Dynkin (1961), Theory of Markov Processes Prentice-Hall THEOREM L.7-L If A is a A-class, if 48 an ra-class: Proof, Part (iii) in the definition of \-class implios that A is closed under the limit of increasing sequence of sets in A. By part (i) aud (i) in the definition of class, A is closed under she limit of decreasing sequence of sets in A. Thus, it is on m-class. 91.7.2. Relationships with a-algebras THEOREM L.7.2 Suppose A is an clgebra on 2. Then Aisanmeclass © Ais ao-algcbm, Proof. “—". Suppose that A is a o-algebra, and we shall sbow that A is an melas. Let Ay € A and A, | (or J). From Theorem 1.82, limiy Ay = UR Ay (oF My Aq), which Delongs to A (as A is a o-algebra). Thus A is an meclass SS". Suppose that A is an m-class, and we shall show that it is a algebra, it suffices to show that Us ,A, © A for A, © A. To show this, let B, monotone Ciereasing) and By € Aas A is au algebra. So Titi By = Titin oo UP An = U; Aisanmchss algebra, Since 4 is an UgeyAa. Then By is id € Aas A o-algchra must be an ri-class, but an m-class may not be an algebra, lot alone a a-algebra (sce tho noxt exmnplo). But if A is already an algebra, then an m-class is a ¢-algebra. To cheek an algebra A isa a-algebma, it suffices 10 show that it is an m-elass, which may be easier in practice Example. Let 9 = R, A= {(-90, 2), (-20, a), (-90,00), 0:0 € R}. So Ais an m-class, but is not an algebra (tims nota o-algebra) since itis not elased under differences. e.g. A= (00,0), B = (00,) b>a, then B-~ A= (a,b) @A. THwornn 1.7.8 A is a o-algebra iff it is both « A-class and class, Proof. “+”. Suppose that A is a o-algebra. Clearly, it is a r-elass. Now we show that it is also a Deelass. (i), It is clear that Qe A. (i). A-B=AnBre a (iil), For every increasing sequence Ay, © A,r > 1, by Theoret 1.3.2, Titty oe Aa = which belongs to A (as A is a o-algcbra). mada, So it is A-class, <=", Now suppose that A is both a class and w-class, First, VAL Q-AEA (as Ais a class) (7.2) 80 A is closed Now let us show that A is closed under countable union. For any sequence 4p ¢ A, > 1, using (7.2), we have nicer complements, By = UNA) = (RAS) CA, as Aisa reclass) Ii follows that A is closed under jinite nation, Since B,, is increasing, by the property of A-class, we have UR 1An = i By € A Hence A is closes! under countable wmion. This completes the proof 1.7.3) Minimal m-class, “class and «-class For any class A, we have seen that there exists a nique minimal a-algebra containing A, denoted by ofA}, (the o-algebra generated by A). ‘The same holds true for m-class, A-class aud class Leama 1.7.1 The power set P(A) is an m-cluss (or A-class, or t-class). 10‘The proof is trivial Loom 1.7.2 Let (Ay :7 € 2} be m-classes (or A-classes or n-classes). Then A =Oyerdy is also an in-class (or N-class or F-class). ‘The proof is similar to that of o-algebras. THeoReM 1.7.4 For anv class A, there exists « unique minimal m-cluss (or -class, or x-class) con- taining A, denoted by m{A) (or MA). or ®(A}). called the m-class (or A-class, or x-class) generated by A. In other words, (0) ACm(A), (or Nelass, or n-class). (0) For any m-class (or A-class, or r-class) B with ACB, we havem(A) CB, (or MA) CB, ora) cB) and m(A) [or A(A), or a(A)) is wigue ‘The proof is left as an exercise, 1.7.4 Graphical illustration of different classes ag —algera elas elas, 4 ¥ algebra m= class 4 semi ~ algehra 4 class1.8 The Monotone Class Theorem (MCT) THEOREM L8.L Let A be an algebra. Then, (1) m(A) = o( 4) (2) FB is on meclass and ACB, then (A) CB Proof. (1) First note that Ac m(A) © 0(A) (sinee o(A) &8 an m-class containing A and m(A) i the smallest m-elass containing A). It romains to show thet m(A) 3 o[A). It enifices to show that m(A) is a c-algebra. In fact, by ‘Theorem 1.7.2, we only need to show (A) is an algebra. (*) Proof of (*): Define Cr = {As AE m(A), ANB € m(A) for all BE A} G = {Bs Bem(A), AN BE m(A) for all Ae m(A)} Cy = {As Aem(A), ASE m(A)} We now show that Cj, j = 1,2, are m-classes Proof, First we look at Cy. Suppose A € Ci, j 1, and Ay CAs C Ayes (inereesing). By the definition of C,, we have Ay B © m(A), 7 2 1, Tor all BEA (© in(A)). Ako we have (AB) C (ARAB) C (Ag B) C vn That is, Ay PB foro an increasing sequence in m{c). In ew of m(A} being a mclas, i follows that A,B OJ (A; 7B) € mC), Using the DeMorgan's ase, wo go BURA) =URMBOA)em(A), for all BEA ‘Therefore lim. Ay = U}E2Ay © Cs. Similarly, for'a decreasing sequence of sets D1 3 Da 9 Da > in Cj, we can show that B7 Dy’s form a decreasing sequence in m[A). ‘Thus, Dy) as, we have shown that Cy is a melas Bn( A(BO By) € m( A). ‘Therefore, im D, = 72D; = 6, The proof for Cy is similar and henee omitted. Now we look at Cy. By the identities (UE A) = LA). (PLAY = Uy, show that Cy is a m-class. Secondly, we shall show that m(A} = Cy C5, Clearly, G) C m(A) by definition, = 1,2,3, So we oly need to show helow that C; > (4), #=1,2,3 Proof. (i). Sinco Ais an algebro and A.C m(A); itis clear that A C Cy. Hence, m(AJ C Cy as m(A) Js the smallest m-cliss. Noting CC m(A), we wot mild) = Cy (i). Por any BE A and A € m(A) = C,, we have ANB € m(A) by the definition of Cy, which in ‘torn iinplies that A © Ca (as m(A) Is the Smallest meclass). Noting Co © m(A), we get mA) = C2, which implies that mA) is closed under intersection (ii), Te easy to see that AC Cy, (To see this, Ae AC m(A), then APE AC m(A) as A ia an algebra.) ‘Thus, Cy is an m-cles containing A. Thus, m(A) c Cy. But it is clear that Cy Cm(A} by definition. Then m{A) = Cs, whieh means that m(A) is elosed under complement 2Finally, we shall show (*): m(A) is an algebra, which follows from (ii) and (iil) above, (2) A.C B implies that m(.4] C m(B) ~ B (as B is an m-class). ‘Phe proof then follows easily from (1): ma} a0) THEOREM 1.8.2 Let A be a r-class (1) Thon AA) = ofA) (2) If B is an X-class ond ACB, then (A) CB. Proof. Clearly, A) ¢ oA) as o(A) is a A-class and A(A) is the sinallest I-class. Now lot 1s show that (4) > o(.4). Brom Thoorom 1.7.3, it suffices to show that A(A) is a r-class (which implies that A(A) is a o-algebra containing, 4, tims N(A) > o(A)). To this end, define Cr = {A:AC®, AD Be AA) forall Be A). Clearly, we can show that Ace (C1 is eelas. Proof of (a) and (b). Let AG A, Since Ais a r-class, for any BE A, we have ANB CAE AA). ‘Ths, A.€C). This proves (a). ‘The proof of (b) is loft as an exercise Hence, Cy is a Aecles containing A. SoC, > XA). Thus 4 © A(A) implies that A © ALA) € C1. So for any BE A, by the definition of Cj, we have AB € MA). Define C= {B:BCO, ANB ENA) for all Ae MA} We can also show that Cy is a A-class (Why?). Therefore, Cy is a A-class containing A. Consequently, C MA). So if A,B € MA), then B € A(A) ¢ Cz, and by the definition of C2, we get ANB XA). Thus, A(A) isa r-class.‘The following theorem is the direet consequence of the last two theorems. ‘TioREM 1.8.8 (Monotone Class Theorem) Let ACB be two classes on 2 (1). If Ais a-x-class, and B is 6 \-class, then o(A) CB. (2). If Ais an algebra, and B is an m-class, then a(A) CB. Proof, Applying the last two theorems, we get (1) Bix a Xelass containing A, s0 B > (A) = o( A) (2) Bis a m-class containing A, so BD m(A)= (A). ‘The Monotone Class Theorem is used in the following way: If A hos some property P, in order to show that ¢(A) has the same property P, we ean proceed as follows: (i) Define B = {3 B has property P}, so that AC B. (ii) Show that (1). Aisa solass, and Bis a class, or (2). Ais an algebra, and B is an meclass, (ii) From the Monotone Class Theorem, we get. (A) © B. Therefore, (A) has property P as well.1.9 Product Spaces define for 2 2 For any measmrable spaces (As). 71, (1). r-dim rectangles of the product space of [[!, 2 wy EAC M1 Si L} in A, thon Ais a cvalgobea 3. Let g bea minimal operation, eg. y =o,m, Am. Let A and B be owo classes of subsets of 1. Show that (only need to show one of them) (1). g(g( A) = 9A). (2). AC 9(B), => GA) C 9(8). (3). (ACB, = g(A) cal) ACB, + ofA) cal). 4, Show that the power sot P(M) is a o-algebra (honce it is also a semialgebra, an algebra, a 2-class, aan m-class, and a -class) 5. Define C= {A:AcO, ANBeE NMA) for all Be A), (Bs BO, ANB ENA) forall AE MAI} Show that €; and C, are \-classes 6. Define sup,er A> and infycr Ay to be the stnallest upper bound and largest lower bound of (Ay + ET}, respectively: Then show sup, =U Ay, inf AS = Mery ee ae (Compare these with the defluitions of sup,cp 4) and infer 0). Show that (1) (AAB)A(BAC) = AAC, (2) (AAB)A(CAD) = (AAC)A(BAD), ()AAB=C if A=BAC, ()AAB=CAD iff AAC = BAD. {Anon 2 1, let Br = Ay Basa = BrdAnsi. m2 1. Show 6. 8. (Optioual) Given a sequence of se that Limy cc By €xi8ts iff limty oe AChapter 2 Measure Theory 2.1 Definitions Let O he # space, Aa class, and je=A— R= [-o0, o0] a set fanet Definition: (i). is finite on A it [n(A)] < 00, VASA Gi). 1 is o-finite on A if 3A, CA, such that for cach m, US, As = @ and |j(An)| < 90. Remarks: (8) Convention: 3A € A such that |(A)] <0, otherwise, there is not much point if all values are infinite (b) UE, As = @ can be replaced by DX, As = . See one of the exercises. Definition: Assume that An A, 32} A, CA, DALE A, one Ay ave disjoint, () w additive = WOE, 4) =TE H(A) (ii) 4 is additive > W(S2Zy A) = DE, lA). Definition: is a measure on A if (i) it is nonnegative, ie, YA € A> n(A) > 0, (ii) itis o-ndditive LRBIMA 2.1.1 Assume that O&A and jis additive. Then (i) 1) =0; Gi) a-adiitivity => additive. Proof. Since IA € A sat. (A) < 00, 50 u(A) = u(A) +4(0). Hence (0) = 0, which proves (i). Proof of (ii) follows from (i).Definition: (i), [fis @ measure on a ovalgebra A of subsets of (2, the triplet (2.4.2) is @ measure space. The sets of A are called moasurable sets, or A-moasurable, (ii). A measure space (9, A, P) is a probability space if P() = 1 [Note: [%A) = measurable space # measure space = (0,A 1) Definition: Assume that {An.n = 1} € A and A¢ A, and jis 0 measure, (1) (Contin If Ay S.A implios al Aq) — (A), jis said to be continnous ab A from above. ty from above) (2) (Continuity from below) If Ay 7A implies (Ay) — (A), 4 is said to be continuous ab A from below. (3) (Continuity at A) If Ay + A himplies jy) — (A), 7:8 said to be continous at A. REMARK 2. 1 (8) Additivity isa rother natural requirement while that of -additivity is not. (@#) Measure 11 is continuous from below. To S00 this, lot By, / limy-aco Bo = Usha, Bn. Then wo can write By = By + (Bs — Bh) ++ (Ba — By) =D, AL. Note that a-additivity implies that + (tim Bo) »(E-) Yaw Tim w(Ba). ‘That is, jp $8 continuous from below (#8) However, measure js may NOT be continuons from above, For example, define a measuze jon the Borel measurable space (FB) by WA) = 0 AsO =~, ifAsD. Tis onsy’ to eke that je S o-additive, thus measure, ‘Take Ay = (0,1/n), then Ay + 9 forts a dectensing sequence of sets in B, but y(n) = 20 7 u() = 0 (2) Measure 1 will be continuous from above conditionally on the assumption that Him) < 0c for some finite ma. For ecample, let us show tat the measure given above is indeed continuous from above conditionally at A= 0. In fact, given a decroasing soquence Ay %, @, the assumption j( Ay.) < 00 for some finite am implies that Ac = @ Tor all & > on, cs (Ax) = 0 = (0) for all k > m. (») Finite measures (such as probability measure) are always continuous. Soo the noxt section (Le, Case TTD,2.2 Properties of measure 2.2.1 Caso I: semialgebras. Let A be « semialgebra. For simplicity, we assume that 0} C4 Hence, Lemma 2.1.1 applies, THEOREM 2.2.1 Let jp be @ nonnegative additive set function on a semialgeba A. Let A,B EA and {Ay, Buon 2 1p EA (1), (Monotonicity); ACB => (A) < p(B), (2). (o-Subadditivity] (a). DP An CA, > DP An) < uC}, (b). Perther assume that pis e-additive (hence a measure). Then, BCDB, = WBS I H(Ba) Proof. (1). Note B= 4+(BAA" = A+BO(DN A) = A+ DAB) 1(B) = lA) + ulin B) > wl A) (2). The proof will be given at the ond of the section “Extensions of set funetions” Inter. 2.2.2 Case II: algebras. All the properties for semialgebras also hold for algebras. In addition, we have: TnworeM 2.2.2 (o-subadditivity) Lei be a measure on an alyebru A. Then, ACUPAn, where AEA, {Ann >1}E 4, 5 pla) < (Ay) (Note the difference from the last theorem, Here we have Uj, not SP) Proof. Let By = ANA, € A, and Cy = By ~ U24'B, € A {as it only involves a finite number of operations). So A = An (UAW) =UR(AN A.) WA), (4) (Continuity from above) Ag SA and ji Aga) <0 for some m 21 except for Ay =O pln) > HCA). (3) (Continuity at A) If isa finite measure, and Ay — A, Phen pa(Ay) — jl). [The claim (5) may not be true if jis not a finite measure; sce the example given in Remark 2.1.1 Proof. Even though the proofs of (1) and (2} are implied by those for semi-algebras and algebras, we ‘ull still give very simple proofs for a-algebras (1) eta) = (An) + (a ~ An) > (A) (2) A= UBL Ay = At + (a An) + (As — (A U Aa) 4 80 WA) = Ay) + HCAa An) + p (Aa ~ (As 0 Ad)) + SMA) + (Aa) + 4g) + First assume that i(,,) = 2¢ for some m. Noto Aj, > Ay, for n > m and A= USA, > Ane ‘Then, mouotonicity implies that j(4,,) 2 p(4,,) = 9° form m and p(A) 2 ow. ‘Therefore, linn p( Ay, wd Now assume that forall m, p(y) < 06. Then A= URSyAs = Ay +(Aa— An) + (Ay Aa) WA) = WA) + al — Ay) + uly ~ Aa) + ws) + WA) = ws) Ay) + fin Sad) — Aa) slim wld) (4) Assume that for some m, s1(Am) < oe. (There is no point of discussion if (Am) = 90 for all m > L.) Then Ay — 4p, 2 1 forms an increasing sequonce and Ap — Ay —* Ary — A Ay — OR As. Then from (3), for m= m, Hl Am — Ay = ply) = (An) (Am) = 1A) ‘Therefore, Tun An) = (PE AW) = CA). (3) See one of the exercises at the end of the Chapter. 22.3 Arithmetics with 00 Since a measure can be 00, we need to know arithmetics with it. Far any ar € (—0, 90), define Loot me, ae =ac(e> 0), 20= ox (@ <0) 2.000 =0, w+: = 00, cH = RzE-2-0 4 0 — a0 and co/ae are not defined, 2.4 Probability measure If () = 1, thon p is a probability measure, usually written as P. The probability space is (2,4, P) ‘Then by dofinition, 1. Forany A€ A, we have 0-¢ P(A) <1 2. PQQ) =1 8 POSS A) = EP Pe) Hexe are some more properties: 4 PODT AY) = EY PAD 5. P(B~ A) = P(B) ~ P(A) if AC B. In pantiewsr, P(A) = 1 P(A). 6. P(A) < PCB) AC B. 7. P(AUB) = P(A) + PB) ~ PLAN). 8. P(per Ab) = De PlAR) — Dicy PALM Ag) + 9. If Ay 7 A or An SA, then P{Ay) > PLA). 10. If 4, + A, then P(A,) + PCA) (So probability 1ueasure is continuous, as well as continuous from above aud below.) Proof. We stull only prove a selected few (5). P(B) = P(A+ [B— Al) = P(A) + PCB A). (7), P(AUB) = P(A + [B~ Al) = P(A) + PCB A) and P(B— A) = P(B— ANB) = PUB) — PLAN) (8). By induction. The details are omitted. 2.5 Some examples of measure Example. (2A) is a measurable space. ‘The following set fimnetion 1 on A is a measure: uA) A=0, o =~ Aca, Ag0 ‘This is neither a finite nor a -finite measure. This cxample shows that measures can be rather abstract, even though they are extensions of length, area, volume, Example. (Coming measure). (is « gpace. For any 4c, define p(A) = [A] = mumber of elements in A, (ce if it contains infinitely aany elements)Then jis the counting measure on the measurable space (9, P(M)), where P(A) is the power set Note that (0). 18 a finite measure if |9| < 205 (0). pis a o-finite measure if © ts countable; (0). cis not a c-finite measure if 2 is not countable. Proof of (c). Otherwise, if pis a e-finite sneasure, then 3 Ay) <0, which makes © countable. Contradiction. ff Ay =O suck that p(A,) = Example. (Discrete Probability Spaces). Let be define P(A) = Y vi ‘Then (2, P(Q), P) is a (discrete) probability space. her finite or countably infinite. For any A € PQ), where p(2) 2 0 and en Pw) Example. (Geometric Probability). Continuing from the last exauple, when @ is finite, we may take plw) = 1/10) s0 that P(A) = |A)/(Q|. Note that we have assumed certain kind of symmetry in our definition. Examples include (2). Dipping 0 fair coin: © = (Head, Tait (2). rolling a die: 2 = {1,2,8.4,5,6) 2.6 Extension of sct functions (or measures) from semialgebras to algebras Definition: Let A and B be two classes of subsets of 2 with A CB. If 1 and v are two set fimetions (or measures) defined on A and B, respectively sueh that WA) = (4), forall ACA, » is satd to bo an extension of y from A to 8, and the restrietion from B to A. First we shall extend a set function (or a measure) from 8 semialgebra S to 5 = ACS). THEOREM 2.0.1. (1). Let be @ non-negative additive set function (or measure) on @ scmialgcbraS (containing Q), then p has a unique extension jt to S = A(S), such that jis additeve. (2). Moreoner, ifn is a-additive on S (avhich implies that p. is a measure on S ), then so is Bons. Proof, (1). For every A CS, we can write A= "A; witla A, 8. Define A\=alay nce if A has a distinet portition A= SO", B,, By €S, then ‘Then Fis well-defined on S, HA = Sond) = SMA 4) = SY AB) 23= Lye 1a) Sada 1B) Vein B,) = Yo utBy). That is, 7(A) is the sane intespeetive of the different partitions of A. Clearly, 7i(A) is nonnegative and Fi) = p(@)) =0. Tes easy to soo that 7F is additive. For To prove uniqueness, let fi be another extension of pr to S = A(S) such that fi is add =P Ae AS) with A, eS, we have H(A) = 9 H(A) = SA) = HA) (2). We will show that 7 is e-additive below. Suppose that B, © S and dis DR Ba ES, we wish to show HA) = 5 HB) (6a) Since A,B, € S, we have A=SOAi with eS; and Ba =S° By; with By €S. Note that Aj By; €S (because of semialgebra) and thoy are disjoint. So we have t L BA) = SWAd=Powaina) as aca a a S(EL+m) as A= SDE By = DE TI Bos EEE win 11Byj) a8 pis o-addlitive on S. On the other hand, we have Dre) = YH wiv) TE ay 0d os By CA me nel ja nel jal = EEH(Emo) SSK G wy) ays each ns ae ‘This proves (6.1). We are now ready to prove part 2 of Theorem 2.2.1, stated enalier but not proved. Turonen 2.6.2 (o-Subadditivity) Let lew nonnegative additive set function on a serielyetra Let Ayn © S. Then en(). DF AeA, H(A CP Hl An) (2). Further assume that u is o-uditine (hence « measure). Then, ACIN Ane SS A) SOT wn Proof. Let j1 be the unique extension ji on S = ACS) (from the last theorem) (2). Since I} AC A and A, SOF As € 5, by the monotonicity of j, we get a) = Sry = SMa. Letting m — 0c, we get the desired result. H(A) (2). Since fi is a measure on the algebra § = A(S), applying Theorem 2.2.2, we have ji(A) < EP a(Ay), implying w(A) sO (Ad. 2.7 Outer measure Definition: Let be a measure on a semfalgebra S with @, eS. For any Ac 2, define (A) = inf {5 WAg): A CUS An Ay E s} to be the outer measure of A. j1* is called the outer measure induced by the measure j. Remarks: (0) Recall: a° == inf, a, is the largest lowor bound of the series {a,,n > 1}, Le (8) a* = inky ay is @ lower bound, that is, ay > a for all n> 1 (ii) Ye > 0, a +e cam not be a Tower bound, that is, there exists an dy S46 Am <0" +e (2). p6(A) is defined for all A € @ since AC 0. So the domain of 1°(A) is the power set PO). The range of j* is [0, 9] (2). (A) may not be a measure itself (3). One can think of A as any set on R, and Uj Ay a8 a countable covering of A THEOREM 2.7.1 Let 4 be a measure on a semiolgebra S with 9,0 € S, and yu be the outer measure indeed by (1) p*(A) = H(A) for Ae 8. In particular, 42 (0) = n(@) =. (2) (Monotonicity) BA) SB) for ACB CO. (8) (o-subadaitivity) (PA) S Su(Aw), for (4, 62.Proof, (1). Suppose A € S, Since AC A, by definition, °(A) < (4) On the other hand, if AC USyAn with An € S, shen (A) < SO; 4o(An) by the o-subadditivity of pon S, sce Theorem 2.6.2. Taking inf on both sides, we get (A) < y"(A). So (1) is true (2). Suppose A CB. If B.C US, Byy then A CUR) By. fie if {Ba} is a countable covering of B in S, it is also a countable covering of A in S.) So by definition, w(Al SD WB) ‘Taking inf on both sides, we get u*(A) < u"(B). (3). IEE w(Ag) = 0, (3) is tre. Now assume D p"(A,) < oo. Let e be an arbitrary positive number. For tach A, C , by the definition of °(Ay), thore exists {Ana} € S, k > 1, such that Ay CUB, Ank (Le. & countable covering of Ay in S) and Vaan) Sead + a It is clear that US Aq C UT UPL Ans with Ane € 8. (i former in S.) So, in viewr of the last inequality, wo have Bo (UP Aa) SSS Ane) DOH" (An) +e Ww (Ag) «the latter is a countable covering of the Since «can be arbitrarily small, the proof is complete. We've shown that jis o-subadditive, but it may uot be o-additive, and therefore may not be used as a measure (since a measure must be c-additive). Although it is possible to introduce an inner measure jt, and then a measure induced by p (if innor measure = outer measure), it is unore convenient to introduce the fellowing definition, due to Caratheodory Defi ‘one bias me A set ACO is said to be measurable w.r-t. an outer measure 1° if for any DCO p(D) = (AND) + (AND), “Although this definition is not very intuitive, st works well Tawonen 2.7.2 A set ACO is measurable wrt. an ouier measure 2 iff for any D.C, one has pe (D) > p'(AND) + (AD), Proof. It follows from Theorem 2.7.1 that: j1* is e-subadlditve, i. w(D) < p'(AND) +°(A° MD), ‘Wherefore, 4°(D) = u°(AO.D) + w*(AS ND) > p"(D) > p AND) +e (ASOD) ow Taworea 2.7.3 . Let A° be the class of all x°-mensurable sets (1). The class AY is a a-aigebro, (2). HA =F Ay with (Ay) €AY, ton for any BCS, uA B) = Sonn) QA ule) i w measure space. Parthermore, plas is an extension of w from to 26(ie, Although p* as defined on P(Q), but the restriction of ji" to A* ie @ measure.) (We'll still use pw" to denote 4:*|4- if no confusion occurs.) Proof. (1). We first show that A” is closed under complement. If A €A*, then for any D.C, one has u(D) = u"(AND) + e'(AP AD) = (ASD) + wh((AE ND). which implies that A° € A‘, So A* is closed under complement, It remains to show that 4° is closed under countable union, or countable disjoint union, 1s given in the next lemma. Lena 2.7.1 F is an algebra. F is a o-alyebra => SP Ay € F whenever {An} € F and disjoint, (ic. closed under countable disjoint union.) (The proof of this is left as homework, see homework 1). ‘To use the lemma, take {A,} € A and they are disjoint, We ueed to show that $2} Ay € A*, For auy Dea (DY = WALD) +e 4END) = WALA) + "(APO Aa D) + WAS ABD) = phALAD) + 1°42 AG D)+ "(As ARG) He (AE AGN AGN D) W(ALOD) 4 4(A2 0 ASD) + [43 AGA; DY nse (An AG 1 Yee VAS ASD) HCAS AAS: Mae VASO ASD). Since (An} are disjoint, we have Ag 0 AG. d ARO Aba Mew Af = (4) 5 (S) ‘Therelre, wD) = WALD) + we (AZ ND) + + "(An VD) lJ Staind) +0" (( Letting 2 00, we get roy 2 Syranoran ((Sa)'o0) 2 #((Ea)oo) +0 (Ea pop). (72) ‘This shows that OX, As AY. Thus, AY is a o-aluebre(2). Contimaing from above, since 5°, Ay € A", and so all the inequalities in (72) can be roplacee by equalities. That roy = Spearman (Sa) on) ' AEE) Im particular, we let D = (CH, A.) 1B, we get (Soo) SAB) +O = » (4) 12) +00) Since je* (8) = 0 from Theorem 2.7.1, we get Le (An B)= 9 (Sa) ox) =p" (anB), which proves (2). Kk: Nowe that, from (7.8) and (7.4), we ean not dieectl Erne (4) 0) - since Ay +B = Ay +B dos not necessarily imply Ay = Ap unless |B] < >.) (AND) (3). Clearly, yr is nonnegative on A* and g-ailditive from (2) by taking B= A or 2. Sop" is a measure on A ‘The fact that p"|.4- is an extension of p from S to * follows from Theorem 2.7.1 (1). 2.8 Extension of measures from semialgebras to a-algebras What is the relationship between the two o-algebras A* and a(S}? THEOREM 2.8.1 We have SC AY, hence o(S) AY. Proof. For any A € S, we need to show A € AY, namely W(D) > WAND) HAND) — for any DEL. By the definition of 1*(D), for any arbitrary ¢ > 0, there exists [Ay } © S, UAy DD (Le. there exists a countable covering of J in $), such that WD) +62 HAn) = (DP) (a5) As A€S, we have A° = SJL, By with By € S. So Ay = ANA, FATA, = ANA, +S (Bra) 28So (Ay) = (AO An) + Spy (Bi A). Brom this and (8.5), we have wise > aay) = Mats an) + YN umen de mo = wlAnD)+p'(aenD) where the last inequality holds by the definition of * and the fact that US (AAA,,) and UT, UL, (Ba 4.) are conmable eoverings of AD and A®A.D in S, respectively, since (AN An) = ANUP An) > AND, UL URL (Bk Aa) = URL (AS Aa) = APN Uda) 2 AND. Sineo ¢ ean bo arbitrarily small, we have shown that 4 ¢ A’. Therefore, we showed that $C AY oon both sides of S.C A to get a(S) Co(A*) = AP as AY is ‘To prove the second part, we tal o-algebra, REMARK 2.8.1 From the last theorern and earlier results, we see that Sc Sc aS) c ac PO) |ARK 2.8.2 We have seen from Theorem 2.7.3 that the measure can he extended from S to A. We've also shown that a(S) CA*. In the neat theorem, howener, we shall extend the measure p. directly from § to oS). (It is possible to extend measure from a semialgebra $ to A(S), avud thew extend i again to c-algcbra; sce Durvett (1996) for exaraple.) ‘TizoReM 2.8.2 (Carathoodory Extension Theorem) . Let be w measure on a seinialycbra § with 1,0€ 8. (®. 4 has an extension to a(S), denoted by Heis\+ 80 (%.9(S)sHleis)) # @ measure space Farthermov, tlq(s) = lo(s, es, this extension ean be simply taken to be the restriction of measure p" Lae tw o(S) (i). If is o-tinite, then the extension in (i) is unique. (ie. if. and py ave both extensions of to a(S}, then ju = po) (Remark: The extension from § to a(S} may not be unique if jis not a-finite; see Exercise 2, page 63, Yan S.J. et al. ) Proof. (i). From Theorom 28.1, wo know o(S} C A*. Thon, by Theorem 2.7.3, the restrietion of j on o(S) is am extension of jr from S to a(S) (2). pris o-finite — there exists disjoint {Dy} CS, such that DX, Dj =O and p(D,) < oe for each m. Suppose jx, aanl jig are both exiensions of measure j from 5 16 a(S) ‘Wo shall first show (ANDg) = po(AAD,), YA a(S), and Yn 21 (86) 29Proof in > 1, define M=(A: Ae o(S), (40 = m(AND,)} [This is, we collect all sets satisfying (8.6), and need! toshow that M = a(S). Since M wwe only need to show that M > oS) Obviously, SCM. (To see this, if A ES, then clearly. A € o(S) and also py(A Dy) = pa[AND,) since ADD, © Sond py and pg ore both extensions of p from S to a(S). Hence, AEM.) So jo, and po are both extensions of measure 4 from & to a(S). From Theorem 2.6.1, we know 1 has a unique extension to A(S), i.e, m(A)= 4A), YAE Therefore, 3 = A(S) CM. (To sce this, if A € S, then clearly, A € o(S) ond also (AM Da) = pa(A Da) since A Da © S and pry and jog are both nique esiensions ofp from 8 to'S. Hence, A € M.) From the Monotone Cless Theorem of the last chapter, [is if Maa an mcs containing S, then a(S) CM], it suffices to show that M is an m-class, which implies that M 3 ¢(S) = o(S). By definition, Mc o(S). This shows M = a(S), which in turn proves (8.6) (8), A(s). 1 remains to show that JM is an ra-class Proof, Let {As} € M antl Ay 7 A= UPAR Note (Ax) € M => pn (de 0 Da) = Bo[ALODy} > Lb > L Also {Ay} € MC o(S), by the property of continuity from below for a measure on a g-algebra, wo x06 m(ANDy) = im py(AKODy) = Jim pal, OD, mafAn Dy) ‘Pherefore, A =U Ae € M Now Let {Ac} © Mand Ay \ A= NP Ag. Since pris e-finite, we ave pi,(Ay Ds) S 1:(Dn) = pe(Dp) < 90. By the property of continuity from above for a measure on a o~ algebra, we get M(AN Dy) = Lim py(AKA.Dy) = Tim 224g 0 Dy) = H2(A 0 Dy) Therefore, A= 9% Ay © -M. Thus, is an m-class, Finally, we show the uniqueness. Proof WA. o(S), in view of (8.6) and a-additivity of a measure, we have iA) = (A 7 Du) = Soe (An Da) = S5 wa(A Da) = pol A), ‘Thus, yu and 2 are the same on ofS). ‘The mensute extension theorem has a useful application in probability COROLLARY 2.8.1 If P is a probability defined on a semialgebra S on , then there existe a unique probability space (0,018), P*) such that PYAV=PIA), YES. 302.9 Completion of a measure We have seen that p1also has an extension 1” to AY, which in tum has a restriction j|.4- to the a-algebra o{S). But what is the relationship between these two: Definitions Let (2..A,j1) be a measure space, and NCQ (). Nisa penull sot iff 3B € A with p(B) =0 such that NCB. Gi). (@,.A,p) is @ complete measure space if every penn set N 14 € A and a jenull set N st. B= AAN. = iKB) = Ad) = (A) =0. “N a penull sot” 3B eA and NC Bs. p(B) =0. Thus, AAN CAUNCAUBEA WAU B) < pA} + 0B) = 0. Tha is, 7 is pani set. ‘Therefore, -0aN eA Remarks: (1), We shall call (9,72) to be the completion of (8, .4,)1)) (2). From the proof below, we shall see that one can take A = {AAW AeA, Nisa peu set), = {AUN: AeA, Nisa pail set}, BAAN) = FAUN) =A), for ANN CA and AUN CD. We now introduce 4 lemina useful in the next theorem, LDMMA 2.9.1. Let 4 be @ measure on a semialgebre S, and p° the outer measure induecd by ji. IF A and p(A) 1, {Fags k > Tf such that A CUR, Fae = Bp with Fae CS and WA) < Yl Fau) a(S). So if CC BA and Cc a(S), thon i0(C) S$ °(B) ~ w"(A) =0, which proves (ii). 33We are now roady to state the main theorem of this saction, TuwoREN 2.9.2 Let p be a a-finite measure on a semialgebru S, 4 be the outer measure induced by p and A® the o-alaebra consisis of all the u*-measuruble sets, Then (2.A*,u*|.4+) is the completion of (2,018). le(s)) Proof. For simplicity, we shall write wm a Bo = Ht |o(s) = Blots). |Fq) be the completion of (@,0(S), 4p) as in the last theorem. So @(S) = {AUN : AC ofS), N is po-mull set}, BAAUN) = pol), for AUN € 0(8). Tt suffices to show that oS) = A", (9.12) the proof of which will be given a little later To see why (9-12) is enough, let BE AY, + £ € o(S). > SA € ofS) and a j.y-null set NW such that F = AUN. Since * is a measure on (Q..A°), and N is j1*-measurable with y°(N) = 0, then we have WA) S (BE) 5 (AUN) S°(A) + p"(N) = f(A) = pol). Hence, we have # (AUN) =p(E) = tol) which implies that (02,A*,p*) is imdeod the completion of (12, 6(S).He)- It remains to prove (9.12). Proof of (9.12): (i), We first show that ofS) ¢ AY. If 2 = AUN € (8), where A erin set. Is easy to show that Y € 4 (8) CAP and N is Proof. Wis py-mil set, + NC B € ofS) © AY such that jo(B) = 0. + 0s (MN) S p(B) = 1,(B) = 0, ie, WN) = 0, => VD CO, w(D) = W(DON) +0 = e(DON) + HN) > (DON) + e(DNN). — Nis p'-measurable, ic., N € AY. Since A,N = A* anil A* is a o-algebra, we have UN eat, Ts, (8) cA. MM(ii). Now we show that A* C (8). Let Ae AY Case I: (A) < 06 From Lemma 2.0.1, 3 B € o(8) such that AC Band p"(A) = p"(B) (io. B is a measurable cover of A in 9($)]. Since jis measure on (0, A"), we get w(B~A)=0°(B) (A) Lot C be a moasurable cover of B~ A in o(S), t0., COB-A, Céo(S), p"(C)=""(B- A)=0. Clearly, 4 = (BC) + (ANC) (draw a diagram to iMustrate), and B-c a(S) a(S), ANCcE, C ‘Thus, AC is a py-tnill set. This implies that A ¢ a(S), Case I: p"(A) = oc. Since jy is o-finite, therefore its extension j A= SE, Aa with An € AP and p"(An) < 06. also a-finite. Thus, VA © A", we have Proof. pt is o-th A=4n0=35 (Ey) <0. te, S$ D= TP Hy with By € A and p"(E_) < 00. + AN Fp = SN Ay With Ay = AN By © AY and j°(Ay) S It follows from the last paragraph that A, € 3{S), which implies that A= TD%, Ay € 3S) 45 (8) i a o-algobra, This proves that A" C 3(S) Combining (i) and (i), wo have shown A" = 3S). REMARK 2.9.1 From the proof of the last theorem, we have seen that A” = G{S). Therefore, we can write A In other words, the gap between A* and a(S) is filled with ail all ue -nudl sets (8) + {all po-null sets} 2.10 Construction of measures on a o-algebra A 2.10.1 General procedures Here is one useful way of constructing measures on a o-algebra A. (i), Identify a semialzcbra $ so that A= a(S). (ii). Define a map pS # so that jis. measure on S, (The ne (ii), Extend the measure from § to A = o(S) by the measure extension theorem. Sa 5 a3 A = osj=A Shae Sa leisy theorem is useful iu proving 4: is a measure on S.) => Note Uvat the first two relations are extensions while the Iast one is a restriction. Usnally, 4% can not bo extended further to the power set P(Q) 35The following theorem is useful in finding a measure on S. THEOREM 2.10.1 Let jo be m nonnegative set function on a serni-algehra S with 1,0 8. Asswne that (i) pis additive on S (ie. WA) = Df aA) whenever Ay € S and A= (i) pis o-subadditive on S. (ie. w(A) < DS w(Aj) whenever A, Ay € S ond ACT® Ay for A= DOF Ay, or AC UPAn) Trae s.) Then ys a measure on S. Proof. Since ju is nonnegative and finitely additive, it can be casily shown that s:() = 0. It remains to chocls if pris o-adatitive. That sf A= Soy aq, whore Ay CS, we'd like to shovr Ha) = San) From Theorem 2.6.2, we have (A) 2 YHA.) Combining this with assumption (i), we get (4) =P edn) 2.10.2 Lebesgue and Lebesgue-Stieltjes measures We are now ready to construct Lebesgue and Lebesgue-Stieltjes measures on Lim Borol space (2,8) where £ = [-o0,o0) and B = Borel sets (ie., the o-alebra generaied by all open intervals) THEOREM 2.10.2 (L-S measure) Suppose that F is finite on (00,08) (ie. [F(t)] < oe for |t| < oe). and (i) P is nondecrensing: (ii) P is right continuous. Then there is a unique mensure 4 on (UR, B) with H((a,t)) = (0) ~ F(a), oo sasbsoe (When a ce or —c0, the right-hand & understood to be 0.) Coronary 2.10.1 (Lebesgue measure) There is « wniigue measure X on (RB) with uf(a.b)) =b-a, wo .) Let us come back to the proof of the theorern, (1). Birst we check that sis well-defined, i.e, we should not have 2 ~ ov or (30) ~ (28). Clearly, there is no problem when both a and b are finite, or one of them is finite and the other one is infinite. Finally, when a = 00 and 6 = 00, 4i((a,8]) is also well defines! since F(o0) > —o0 and F(=00) < oo, otherwise, wwe would have either F(Q) = oe for allt € Ror F(t) = —o0 for all t € R, which is excluded from our ‘consideration. (11), Secondly, we show that je defined above is a measure on the semialgebra S by way of Theorem 2.10.1 Cloatly, jis nonnegative. (i), Checking finite additivity (i, checking condition (i) of Theorem 2.101): Let $ DLL Si, where 5; € &. (ia). If S = (0}, or = {ec}, finite additivity is trivial since § = S. 168 then finite additivity holds for R= SO". 8: ifFit hols for R= (—o0} = TNS {-o0}}. (Note that one and only one of §;'s contains {—o0}.) But it is easy to see that R— {00} = [-00,00) and S;— {—oo}’s are all half intervals of the form (a, 0, where —90 0 and sj > 0 so that Fla+s)< Flat Fb, +n) < Fl) + of. Note that [a+ 6.8] © (a,b) © UP(os,bi] © LEPC te +14), Ke, the closed interval has a countable cover by the open intervals (aj, bi-+n4). So there is a finite subeover (01,4), 1S j Sd. By monotonicity of F, wo got F(0)-Fla) = [F@)- Pla=6)]+ (Flo +8) - Fl) < (FO) Flasai] +e < YP) - Flas +e < YFG +m) - Fla) +e s SF) ~ Fay) +26. Lotting ¢ +0, wo got (le, 2)) $Y (as, bi). ()). Case Te -00 = a € b< 00, Le, (00, 0) CUP (as bi) Let <00 < Mc be ce. Thon (Mb) CUP (aj. i}. Fron Case T, we get nay rans Sno) rie) Letting M + oc, we get F(b) — F{-20) < D2 [FG,) ~ Plas) ie atl) $ So pKlassb) (6). Case II: ~26 < a < b= 00. Proof is similar to (b). (4). Case IV: -20 = a (~o0,2] C (00, y)- By the monotonicity of P, wo got F(a) = P((—28,a]) < P\(-se.4) = Flu): (0) Given ity Nr, Ht follows that (90,2) \ (30. Sine P is continuons from above, we have Flan) = P((-2e,al) \ P((-20,1)) = Fle). ‘Thns, limy, (2) Fly), ie. F is right continuous. (2). Givena df. F, it mast be w L-S measure function. Applying Theorem 2.10.2, there exists a unique probability measure P on (R,B) satisfying F(x) ~ F(y) = P((y,2]} for x < y. Letting y \, ~oe, wwe have FG) = lim [F() — P)] = fim P(p,2)) = P00. 2). Romarks: (2) From the above theorem, we can define « distribution fimetion (d.t) by Piz)=P(-x,2), reR. ‘This definition does not involve any random variables (which we have not discussed yet). Ditforont vorsions ofc. involving tv. cam be found in Durrtt, for instance. (2) Pisa di. and Pisa probability measure. ‘Then the following are equivalent (1) P((-0e, bj) = FO), 2) P(A) =F) Fla) (3) Pla, b)) = FH) - Fa-), (®) P((a,b) = FC) =F), (5) P((a,b)) = Plb-) — Pla). (8) Othor usoful relations are (6) P((—2e.8)) = FUb-) 7) Pifa}) = Pla) Fla). 39(4) Let X be a random variable {to be introduced later) on (9, A, P), and let Fy (2) = P(X < 2). Then F is a right continnons, nondecreasing function with F(—90) = 0 and F(oo) = 1 Some authors define such funetions to be dfs. (8). For a dt. F, the set S(P) = {2 F(x 4.6) — F(z—6) > 0, for all > 0} is called the support of F. Furthermore, any point x € $(F) is called point of increase. Te can be shown that (a) each jump point of # belongs to the support and that each isolated point of the support is a jump point. (b) 51 (0) a diserete d.f. can have support (20, 00). F) is a closed set, Proof. (a) « is jump point if F(x) —F(r—) > 0, clearly F(r+e)—F(x—e) > Fo) F(e O for all © > 0. Thus, 2 € S(F). (b]. Let (n,m > 1} € S(P) and eq —+ 2 We need to show that x € S(F). Given ¢ > 0, Ng > O such that for all n> Ng, we have [zy —2| < €/2, oF zy —e/2 0. (©) The diserete c.f, with positive jump size at each rational number is such an example. 2.10.4 Different types of distributions Definition: (i). A df. 6; is called degenerate at tif 62) = 0 4ret (check that it is indeed a d.f) (ii). ALP is called diserete if it can be represented in the form Yoda). (check that itis indeed a df) where {s.r 2 If i a countable set of real numbers, ny > 0 for all j 2 1} and 5p, = (ii). A.A F is ealled continous if itis contimons everywhere Fl) Remarks: (1). Degenerate distribution functions. ‘The unique L-S probobility measure on (R,B) for a degenerate df. é¢(e) = {2 >t} is P((a,b) = 6(b) da) = 1 te (a,b 0 tC) Bull) ~ 6) =1-0= 1.) (In particular, we have P(t) 40Another probability measure for 5,(1) = I{or > t} defined on a different measurable space (O, P(O)), where = {t} and so P() = {0,0} = (0.40). is Put) = 1 Py(@) = 0. (2). Diserete distribution functions. he unique 1-8 probsiility mexsure on (78, B) for a diserete d.f F(2) = S%; Pabag(r) is P((a.b)) = FO) = Fla) = S95.) — D> Pada, (a) = Lelio) o@= pa fecal} since 6,,,(0) ~ 6,,(0) = Lika 1} be any given ensmeration ofthe set of all rational nuunbers, and let (py, > I be a set of positive numbers such that 5p, = 1, 8g, Pa =2-*. Define F(2) = PaBan (7) Since 0 < 6(x) < 1, the sequence on the right hand side of # is absolutely and uniformly convergent, Also sinee éy(1) is inereasnng, we have for x < y, POY) ~ Fla) = Yn (So, (u) ~ bo, ()} > 0. Hence, F is nondecreasing. Due to the uniform convergence, we may deduce that F(a) = Fle) = pa (64, (2) ~ ha, (@-)) = 0 #¢{aan>I} Py B= Om, NEL ‘Therefore, (2) has jumps at all the rational points and nowhere else, Remark: The example shows that the set of points of jump of an increasing funetion may be countably infinite and everywhere dense. 2.10.5 Decomposition of distribution functions Lain 2.10.1 The set of discontinuities of a non-decreasing function és countable aProof. Every discontimions point of an increasing fimetion must be a jump and every jump contains a rational number Lot {aj} be che countable set of points of jump of ad. F and p; = Fla;)— F(aj-) > 0 the size at jump at a). Consider Pala) = S> 9980, (e) = sum of all the jumps of P in (—o0, 1 (after some reordering), but p, # p| for some i > 1. a2In either case, there exists at Ieast one a = a; or a for some 4 > 1, such that Fa) — Fa@-) # Gul) — Gola) (10.16) In view of F(z) = Fle) + Fy(2) = G(x) + Gala), we have 0 = R@)- Fla = [PW ~ Fula) — (Pe) ~ Fal) = [FQ@) — F(a)] - [Ral@) — Fle] # [P@) FG) [ela] Gul@)| from (20.16) = [P@) - Ga(a)) ~ [P-) - Gale) Ga) — G.(-) 0. This iso contradiction. Therefore, we must have Fy = Ga, and consequently P= Gow Tuwowin 2.10.6 Every df con te written as the conver combination of a diserete and a continues Fle) = aF (2) (1 a) Filo). Fi: discrete and Fz: continuous, Such @ decomposition is unique, Proof. Given a df F(z), from the last theorem we can waito Fs) = Fy(z) + Fe(2) uniquely. Let 01> Fy(ee), then 1a = F,(). Ma =0, thon D < Fy(e) < Fa(ce) = a =D for alle, ic. Fala) = 0 for all ». Hence, Fe(r) is & proper @f,, andl we can write Flr) = 0% Fy(r) +1 F.(2) Ia = 1, then 0 < Fe(r) < Fe(oo) = 1-0 = 0 for all x, i, Fe() = 0 for all x, He isa proper df, and we can write F(x) = 1x Fula) +0 % Fel) 160-<.a< 1, thon F ean be written as Pal) Folic) 1a) nae) © YR) F(a) + (=a) FSC) Fl) Fale) + Fela) where it is easy to see that Fy and Py are both d.f’s. Conouiary 2.10.2 Kverg df. F is either w discrete df. or a continuens df, or « combination (or a ‘mizture) of both. 2.10.6 Further decomposition of a continuous df. F ‘We can further decompose a continuous df. as A continuous d.f. = an absolutely continuous df. + a singular d.f. But the shove further analysis of d.£'s requires the theory of Lebesgue measure, and integration, whieh wwe have not done so far. We only list some definitions and conclusions. Definition: 4B@) A function F is called absolutely continuous [in (20,00) and wrt. the Lehesgue measure iff there exists a funetion J in ZL! (ie. J Jat < oe is devined and fice) sueh that for every 2 1, F(a) = 1/2 for 2 ¢ (1/3.2/3}, Fle) = 14 = 1/2 for w € (1/9.2/9) = [1/3*, 2/34), F(@) = 3/4 = 1 = 1/2 for » € [7/9,8/9] = (3? — 2)/32, (8? - 1)/34), It cam be shown that the resulting fmetion # is defined for all all. € A. Further it is nondecreasing continnous, and F(—oc) = Oand F'(o0) = 1, ic. it is a df. (Check carefully about the continuity of F'!!) Also, itis easy to see that F'(zr) =0 for all x © R excopt perhaps for those on the Cantor set (i.0., those points m/3?, m,n €.N’). By definition, F is clearly a singular Lf, which is called “Lebosgue's singular function” REMARK 2.10.1 For Centor ternary set, one can see Question 10, page 25, Chow and Teicher. 442.10.7 Some examples of distributions: Absolutely cont ous df. F's with pad.t. fs (i) Uniform distribution: f(r) = hess =. 1 Viz c a+) (iv) Exponential distribution f(c) = e"1{2 > 0} i) Normal distribution: f(x) (il) Canehy distribution j(x) srete distribution (i) Binomial dfs PLX = = (“re DIOS PS 1, 2 =O are (i) Possion ats PU) = XE, 3.2, (iii) Empirical distribution function (e.d.£.) of observations {11,09} Flo) <2 2). Show that Petia) > SPA) - YX Plain ay). 2. If (A, P) is a probability space, and {4m > 1} € A (ie, they are a sequence of events) Gi) Verify that Pim Ap) = Tim P (02,4; Gi) Show that POZA) = 1 if P(A) = 1. nA 3. W(QA,p) is « measure space, and Ay € A. Gi) Prove that (lim A.) < lin (4,). Analogously, if (U2. then (lim Ay) > lim (Ay Gi) If Js & finite measure, and Jim Ay = lim Ay = A, (ie. lim A, = A), then Tim (Ay) = wl) (iil) ISD An) < 20, then (lina An) = 0. (This is half Borel-Cantelli Lemma when 4 is a probability measure.) Ai) < 00 for some n 2 1 4. At, day... are a sequence of events on the probability space (, A, P) (1) If P(Ag) > 0 and DE, PlAWAG 1) < 20, then P(Ap,é.0) = 0. (2) Tf P(Ag) +0 and SE PLAS Ans) <0, then P(Ay, 5.0) = 0. (3) If Ay’s are independent, provide a simple proof for (1). (4) Optional.) Give examples to show that (1) may not be true if only one of two conditions holds, 5. Let jt bo a nomegative additive sot fumetion on an algebra A and [Aq, > 0) CA. Show (1) is o-aditive (ences measure on.) HP for Ay 7 A, all © A, we have ltt. (An) = nA). (2) if for every Ay SB, we have lina, (An) = 0, then 4 is o-additive (lence a measure). 6, (Alternative definition of v-finite measure). Let jc bea o-fiite measure on a semialgehra S ‘Then there existe disjoint {Bp.m > 1} CS, euch that YI", By = ond (By) < oo for cach n, asChapter 3 Random Variables 3.1 Mappings Since My is arbitrary, it is often convenient 0 consider a fanetion (mapping) X from Q to a simpler spave 2 (exg. 24 =) Definitions Let X:)— M be a mapping, (i), For every subset 15 € M, the inverse image of Fis, xB) {wre €..X (0) € BY = (8 eB} (ii). For every class G C M9, the inverse image of G is XG) = {XB}: Be g} (80 Ae X *(G) means that 3B © Gst. A= NX -1(B)) THEOREM 3.1.1 (Properties of the inverse image) (1). X és @ mopping from 2 to M2. Then () X(,) =, X1(~ =o. (a) X-(B9 = [xB (iii) X“ (rer By PXH(By) for By CM, 7 EL. (wi) X“(MyerB,) X“"(B,) jor B, CM, EL where Tis an index set, wot necessurily count (0) X 1B, = Ba) = X-'(By) = X-*(Ba) for By (02) By C Be CM implies that X-“(By) CX-(B,). (2) IPB is a o-algchra in Qs, then X~(B) is a o-algchra in 2 (8) LetC be a nonempty eloss in M2, then ‘oe a(x") Proof, (1) (i) Note that fem: X ) EM) = XQ) = EM XU) EH} =O. 49(ii) KCBS) = fos X(w) © BY = (ws Xl) ¢ BY = {ww EX BY = (wwe [UB = xX (ByY. (iil) EX (UyerB,) <> Xv) © UyerB, > X(w) © B, for some y ET <> we X-(B,) for some y ET <= W € UserX~ (By). (iv) Similar to (ii), (vy) XB, — Ba) = (5) = X1(Ba) 0 X“1(Ba) = XM) = X~'(Ba) 1B, 9 BS) = XB) NX (vi) X71) = fw: X(w) © By) C (ws X(w) € By} = X~"(Ba). (2). Finst X~1 (B) is nonempty as 1 = X~! (Os) € X~!(B], Noxt, lot A ¢ X~!(B). Thon 3B € B sit A= X-1(B), So A* = X-1(B") © X-1(B), So X~1(B) is closod under complement. Similanly, it can ‘be shown that it is closed! under countable union. ‘Phus it is a o-alechra. (8). Clearly, X (C) © X-1 (o(C)), which is a a-algebra from (2). Thus # (X- (C)) © X-*(a(@)). 1 remains to show XxX? (6(C)) Co (X*(C)) Define G={E:X 4G) €o(x *C))} It sufices to show that o(C) CG. Since CCG, we only need to show that G is a a-algebra. (Proof of CCG: take GEC, then XG) EXC) ea(X-1(0)), hone Ge g.) It romains to show that G is a a-algobra, Lot Ge G + XG) € 6 (X-(C)), > XG) = X-(G)P € @ (X1 C)] = Gee G = G is closed under complement. Similarly, we can show chat it js closed under countable union. The proof is complete. Remark: Clearly, 1() on Mz preserves all the sot operations on 123.2. Measurable mapping Definition: (1) (iA) and (25,8) ore moasmrable spaces, X : 2) —*O is a measurable mapping if X'B)S{XEB)eA YBEB. (2) X is o measurable Function if (2,8) = (R°\B(R") in (1) (3) X isa Borel (measurable) function if(s,.A) = (RP, B(R")) and (2, B) = (R", BCR") in (0). ‘The next theorem is useful in checking if X is measurable or not. THEOREM 8.21 X : (Q1,A) > (M,B) is a measurable mapping if B = o(C) and XC) EA Jor aiced Proof. From Theorem 3.1.1 (iil), we have XB) = X10) = 0 (XO) CalA)=A THEOREM 8.2.2 If X :(Q1.AL) —> (2, Az) and f : (2. Ax) + (Qs.As) are measurable moppings. then F(X) = FX 1 (QA) + (3,Ap) t# also measurable, Proof. VAy € Ay, (f- X © Ag} = (X | PM Ay) © Aa} © Ay A more detailed proof is: Vy € Ay, (FX) Ag) = {ws f Xe) € Ay} = (2: (0) € FM Ay) € Aa} = feiwe Xf As))} EAL (A picture will make this very clear.) 5