0 Up votes0 Down votes

1 views36 pagest

Jan 21, 2019

© © All Rights Reserved

- EC 305 syllabus
- Week-1 ML Notes
- WARNING: Physics Envy May Be Hazardous To Your Wealth!
- 7 Fundamentals of Probability
- Introduction to Probability
- T2Quantitative_v17
- Ieong - Stanford Probability Theory Review for ML
- Linear Programme - Solution ToMartin Service Station
- Solution of HW5
- ch2
- 02 Probability, Bayes Theorem and the Monty Hall Problem
- Chapter 1
- h2p2sol Alt
- mit6 450f09 slide08
- Vivek Sports
- Study Unit 2.1 Basic Probability
- Bosq Nguyen A Course in Stochastic Processes.pdf
- Mathematical Expectation
- EE132B-HW2
- Probability.pdf

You are on page 1of 36

Florian Herzog

2013

Probability space

Probability space

A probability space W is a unique triple W = {Ω, F , P }:

• Ω is its sample space

• F its σ -algebra of events

• P its probability measure

Remarks: (1) The sample space Ω is the set of all possible samples or elementary

events ω : Ω = {ω | ω ∈ Ω}.

(2)The σ -algebra F is the set of all of the considered events A, i.e., subsets of

Ω: F = {A | A ⊆ Ω, A ∈ F }.

(3) The probability measure P assigns a probability P (A) to every event

A ∈ F : P : F → [0, 1].

Sample space

The sample space Ω is sometimes called the universe of all samples or possible

outcomes ω .

Example 1. Sample space

• Toss of a coin (with head and tail): Ω = {H, T }.

• Two tosses of a coin: Ω = {HH, HT, T H, T T }.

• A cubic die: Ω = {ω1, ω2, ω3, ω4, ω5, ω6}.

• The positive integers: Ω = {1, 2, 3, . . . }.

• The reals: Ω = {ω | ω ∈ R}.

Note that the ω s are a mathematical construct and have per se no real or

scientific meaning. The ω s in the die example refer to the numbers of dots

observed when the die is thrown.

Event

subset A, then the event A is said to have occurred. The set of all subsets of

the sample space are denoted by 2Ω.

Example 2. Events

• Head in the coin toss: A = {H}.

• Odd number in the roll of a die: A = {ω1, ω3, ω5}.

• An integer smaller than 5: A = {1, 2, 3, 4}, where Ω = {1, 2, 3, . . . }.

• A real number between 0 and 1: A = [0, 1], where Ω = {ω | ω ∈ R}.

We denote the complementary event of A by Ac = Ω\A. When it is possible

to determine whether an event A has occurred or not, we must also be able to

determine whether Ac has occurred or not.

Probability Measure I

A probability measure P on the countable sample space Ω is a set function

P : F → [0, 1],

• P(Ω) = 1.

• P(ωi) = pi.

• If A1, A2, A3, ... ∈ F are mutually disjoint, then

∞

[ ∞

X

P Ai = P (Ai).

i=1 i=1

Probability

• Sample space: Ω = {ω1, . . . , ωn}, finite!

• Events: F = 2Ω: All subsets of Ω

P

• Probability: P (ωi) = pi ⇒ P (A ∈ Ω) = pi

w i ∈A

Probability axioms of Kolmogorov (1931) for elementary probability:

• P(Ω) = 1.

• If A ∈ Ω then P (A) ≥ 0.

• If A1, A2, A3, ... ∈ Ω are mutually disjoint, then

∞

[ ∞

X

P Ai = P (Ai).

i=1 i=1

Uncountable sample spaces

• Obviously, P(ω) = 0.

• Intuitively, P([0, a]) = a, basic concept: length!

Answer: No! (e.g. Vitali sets, Banach-Tarski paradox)

Answer: No!

Answer: A lot!

Fundamental mathematical tools

Not every subset of [0, 1] has a determinable length ⇒ collect the ones with a

determinable length in F . Such a mathematical construct, which has additional,

desirable properties, is called σ -algebra.

Definition 2. σ -algebra

A collection F of subsets of Ω is called a σ -algebra on Ω if

• Ω ∈ F and ∅ ∈ F (∅ denotes the empty set)

• If A ∈ F then Ω\A = Ac ∈ F : The complementary subset of A is also

in Ω

S∞

• For all Ai ∈ F : i=1 Ai ∈ F

The intuition behind it: collect all events in the σ -algebra F , make sure that

by performing countably many elementary set operation (∪, ∩,c ) on elements

of F yields again an element in F (closeness).

The pair {Ω, F } is called measure space.

Example of σ-algebra

• Ω = {HH, HT, T H, T T } = {ω1, ω2, ω3, ω4}

• Fmin = {∅, Ω} = {∅, {ω1, ω2, ω3, ω4}}.

• F1 = {∅, {ω1, ω2}, {ω3, ω4}, {ω1, ω2, ω3, ω4}}.

• Fmax = {∅, {ω1}, {ω2}, {ω3}, {ω4}, {ω1, ω2}, {ω1, ω3}, {ω1, ω4}, {ω2, ω3},

{ω2, ω4}, {ω3, ω4}, {ω1, ω2, ω3}, {ω1, ω2, ω4}, {ω1, ω3, ω4},

{ω2, ω3, ω4}, Ω}.

Generated σ-algebras

Let C be a class of subsets of Ω. The σ -algebra generated by C , denoted

by σ(C), is the smallest σ -algebra F which includes all elements of C , i.e.,

C ∈ F.

we then just work with the σ -algebra generated by A and have avoided all the

measure theoretic technicalities.

Borel σ-algebra

The Borel σ -algebra includes all subsets of R which are of interest in practical

applications (scientific or engineering).

Definition 4. Borel σ -algebra B(R))

The Borel σ -algebra B(R) is the smallest σ -algebra containing all open

intervals in R. The sets in B(R) are called Borel sets. The extension to the

multi-dimensional case, B(Rn), is straightforward.

• (−∞, a), (b, ∞), (−∞, a) ∪ (b, ∞)

• [a, b] = (−∞, a) ∪ (b, ∞),

S∞ S∞

• (−∞, a] = n=1[a − n, a] and [b, ∞) = n=1[b, b + n],

• (a, b] = (−∞, b] ∩ (a, ∞),

T∞

• {a} = n=1(a − n1 , a + n1 ),

Sn

• {a1, · · · , an} = k=1 ak .

Measure

Definition 5. Measure

Let F be the σ -algebra of Ω and therefore (Ω, F ) be a measurable space.

The map

µ : F → [0, ∞]

is called a measure on (Ω, F ) if µ is countably additive. The measure µ

is countably additive (or σ -additive) if µ(∅) = 0 and for every sequence of

S

disjoint sets (Fi : i ∈ N) in F with F = i∈N Fi we have

X

µ(F ) = µ(Fi).

i∈N

have

µ(F ∪ G) = µ(F ) + µ(G) if and only if F ∩ G = ∅

The triple (Ω, F , µ) is called a measure space.

Lebesgue Measure

The measure of length on the straight line is known as the Lebesgue measure.

Definition 6. Lebesgue measure on B(R)

The Lebesgue measure on B(R), denoted by λ, is defined as the measure on

(R, B(R)) which assigns the measure of each interval to be its length.

Examples:

• Lebesgue measure of one point: λ({a}) = 0.

P∞

• Lebesgue measure of countably many points: λ(A) = i=1 λ({ai }) = 0.

• The Lebesgue measure of a set containing uncountably many points:

– zero

– positive and finite

– infinite

Probability Measure

A probability measure P on the sample space Ω with σ -algebra F is a set

function

P : F → [0, 1],

satisfying the following conditions

• P(Ω) = 1.

• If A ∈ F then P (A) ≥ 0.

• If A1, A2, A3, ... ∈ F are mutually disjoint, then

∞

[ ∞

X

P Ai = P (Ai).

i=1 i=1

F-measurable functions

The function f : Ω → R defined on (Ω, F , P ) is called F -measurable if

−1

f (B) = {ω ∈ Ω : f (ω) ∈ B} ∈ F for all B ∈ B(R),

easier to work with following equivalent condition:

y ∈ R ⇒ {ω ∈ Ω : f (ω) ≤ y} ∈ F

This means that once we know the (random) value X(ω) we know which of

the events in F have happened.

• F = {∅, Ω}: only constant functions are measurable

• F = 2Ω: all functions are measurable

F-measurable functions

X(ω) : Ω 7→ R R

6

t

j X(ω) ∈ R

ω A2

A1

(−∞, a] ∈ B

A3 Y

Ω A4 X −1 : B 7→ F

F-measurable functions - Example

We only know, whether an even or and odd number has shown up:

F = {∅, {ω1, ω3, ω5}, {ω2, ω4, ω6}, Ω} = σ({ω1, ω3, ω5})

1, if ω = ω1, ω2, ω3;

f (ω) =

−1, if ω = ω4, ω5, ω6.

y ∈ R ⇒ {ω ∈ Ω : f (ω) ≤ y} ∈ F

/ F ⇒ f is not F -measurable.

Lebesgue integral I

(Ω, F ) a measure space, µ : Ω → R a measure, f : Ω → R is F -measurable.

• If f is a simple function, i.e., f (x) = ci, for all x ∈ Ai , ci ∈ R

Z n

X

f dµ = ciµ(Ai).

Ω i=1

fn with fn(x) ≤ fn+1(x) which converges to f : limn→∞ fn(x) = f (x).

With this sequence, the Lebesgue integral is defined by

Z Z

f dµ = lim fndµ.

Ω n→∞ Ω

Lebesgue integral II

(Ω, F ) a measure space, µ : Ω → R a measure, f : Ω → R is F -measurable.

• If f is an arbitrary, measurable function, we have f = f + − f − with

+ −

f (x) = max(f (x), 0) and f (x) = max(−f (x), 0),

Z Z Z

+ −

f dµ = f dP − f dP.

Ω Ω Ω

f +dP

R

The Rintegral above may be finite or infinite. It is not defined if Ω

and Ω f −dP are both infinite.

Riemann vs. Lebesgue

The most important concept of the Lebesgue integral is that the limit of

approximate sums (as the Riemann integral): for Ω = R:

f (x) f (x)

6 6

- x - x

∆x

Riemann vs. Lebesgue integral

Let f be a bounded and continuous function on [x1, x2] except at a countable

number of points in [x1, x2]. Then both the Riemann and the Lebesgue integral

with Lebesgue measure µ exist and are the same:

Z x2 Z

f (x) dx = f dµ.

x1 [x1 ,x2 ]

There are more functions which are Lebesgue integrable than Riemann integra-

ble.

Popular example for Riemann vs. Lebesgue

0, x ∈ Q;

f (x) =

1, x ∈ R\Q.

f (x)dx

0

does not exist, since lower and upper sum do not converge to the same value.

However, the Lebesgue integral

Z

f dλ = 1

[0,1]

Random Variable

A real-valued random variable X is a F -measurable function defined on a

probability space (Ω, F , P ) mapping its sample space Ω into the real line R:

X : Ω → R.

Distribution function

The distribution function of a random variable X, defined on a probability space

(Ω, F , P ), is defined by:

Density function

Closely related to the distribution function

R be a nonnegative function, satisfying R f dλ = 1. The function f is called

a density function (with respect to the Lebesgue measure) and the associated

probability measure for a random variable X , defined on (Ω, F , P ), is

Z

P ({ω : ω ∈ A}) = f dλ.

A

for all A ∈ F .

Important Densities I

λx −λ

f (x) = e , x = 0, 1, 2, . . . .

x!

1 x−µ

− 1

f (x) = √ e 2 σ

2πσ

• Multivariate normal density (x, µ ∈ Rn; Σ ∈ Rn×n):

−2

f (x) = p e .

(2π)ndet (Σ)

Important Densities II

Γ( ν+12 )

1 (x − µ)2 − 12 (ν+1)

f (x) = √ 1+

Γ( ν2 ) πνσ ν σ2

Rn×n):

Γ( ν+n ) 1 − 1 (ν+n)

2 T −1 2

f (x) = ν

p 1 + (x − µ) Σ (x − µ) .

n

Γ( 2 ) (πν) det (Σ) ν

Important Densities II

• The chi square distribution with degree-of-freedom (dof) n has the following

density

−x x

n−2

e 2 2 2

f (x) =

2Γ( n2 )

which is abreviated as Z ∼ χ2(n) and where Γ denotes the gamma

function.

• A chi square distributed random variable Y is created by

n

X 2

Y = Xi

i=1

N (0, 1).

Important Densities III

X

Y =q ,

Z

ν

• Another important density is the Laplace distribution:

1 − |x−µ|

p(x) = e σ

2σ

with mean µ and diffusion σ . The variance of this distribution is given as

2σ 2.

Expectation & Variance

The expectation of a random variable X, defined on a probability space

(Ω, F , P ), is defined by:

Z Z

E[X] = XdP = xf dλ.

Ω Ω

With this definition at hand, it does not matter what the sample Ω is. The

calculations for the two familiar cases of a finite Ω and Ω ≡ R with continuous

random variables remain the same.

Definition 14. Variance of a random variable

The variance of a random variable X, defined on a probability space (Ω, F , P ),

is defined by:

Z

2 2 2 2

var(X) = E[(X − E[X]) ] = (X − E[X]) dP = E[X ] − E[X] .

Ω

Normally distributed random variables

variables with parameters µ and σ is often found in the literature. The

following properties are useful when dealing with normally distributed random

variables:

• If X ∼ N (µ, σ 2)) and Y = aX + b, then Y ∼ N (aµ + b, a2σ 2)).

• If X1 ∼ N (µ1, σ12)) and X2 ∼ N (µ2, σ22)) then

X1 + X2 ∼ N (µ1 + µ2, σ12 + σ22) (if X1 and X2 are independent)

Conditional Expectation I

P (A ∩ B) P (B|A)P (A)

P (A|B) = = , P (B) > 0.

P (B) P (B)

A A∩B B

E(XIB )

E(X|B) = , P (B) > 0.

P (B)

Conditional Expectation II

Definition 15. Conditional expectation

Let X be a random variable defined on the probability space (Ω, F , P ) with

E[|X|] < ∞. Furthermore let G be a sub-σ -algebra of F (G ⊆ F ). Then

there exists a random variable Y with the following properties:

1. Y is G -measurable.

2. E[|Y |] < ∞.

3. For all sets G in G we have

Z Z

Y dP = XdP, for all G ∈ G.

G G

Conditional Expectation: Example

Y = E[X|G]

X(ω)

6

X

Y

- ω

G1 G2 G3 G4 G5

Y = E[X|{∅, Ω}] = Ω XdP = E[X].

Conditional Expectation: Properties

• E(E(X|F )) = E(X).

• If X is F -measurable, then E(X|F ) = X .

• Linearity: E(αX1 + βX2|F ) = αE(X1|F ) + βE(X2|F ).

• Positivity: If X ≥ 0 almost surely, then E(X|F ) ≥ 0.

• Tower property: If G is a sub-σ -algebra of F , then

E(ZX|G) = Z · E(X|G).

Summary

operations

• Borel σ -algebra: all the events of practical importance in R

• Lebesgue measure: defined as the length of an interval

• Density: transforms Lebesgue measure in a probability measure

• Measurable function: the σ -algebra of the probability space is ”rich” enough

• Random variable X : a measurable function X : Ω 7→ R

• Expectation, Variance

• Conditional expectation is a piecewise linear approximation of the underlying

random variable.

- EC 305 syllabusUploaded byyannikrock
- Week-1 ML NotesUploaded bySumanth_Yedoti
- WARNING: Physics Envy May Be Hazardous To Your Wealth!Uploaded byCervino Institute
- 7 Fundamentals of ProbabilityUploaded byBeskal Baru
- T2Quantitative_v17Uploaded byCharumathi Raju
- Introduction to ProbabilityUploaded byRochana Ramanayaka
- Ieong - Stanford Probability Theory Review for MLUploaded bytam2012
- Linear Programme - Solution ToMartin Service StationUploaded byRuth Sooklal
- Solution of HW5Uploaded bymiguel
- ch2Uploaded bynarayananx5
- 02 Probability, Bayes Theorem and the Monty Hall ProblemUploaded byuyjco0
- Chapter 1Uploaded bySerge Mandhev Swamotz
- h2p2sol AltUploaded byCatalin Toma
- mit6 450f09 slide08Uploaded byapi-246008426
- Vivek SportsUploaded byVivek Narayanan
- Study Unit 2.1 Basic ProbabilityUploaded bybronwyncloete
- Bosq Nguyen A Course in Stochastic Processes.pdfUploaded byYehandAldo12
- Mathematical ExpectationUploaded byBea Sarreal
- EE132B-HW2Uploaded byDaniel
- Probability.pdfUploaded bySamsul Imran Bahrom
- 1.RandomVariableUploaded byIQBAAL FADLLI
- Introduction to Probability Theory and StatisticsUploaded byKei Yim
- Schuster P. - Stochasticity in Processes. Fundamentals and Applications to Chemistry and Biology - (Springer Series in Synergetics) - 2016Uploaded byfictitious30
- Statistical ModelsUploaded byGiyane Maxmillan
- Mass Volume CurveUploaded byDeepakGujraniya
- Dhruva RamanUploaded byAnonymous mBlMx6
- s to Chas Tic AnalysisUploaded byLeandro Fosque
- Exercises IntegrationUploaded byAnonymous knICax
- bayes-nets-2015.pdfUploaded bySandeep Gupta
- Basic Reliability Mathematics.pdfUploaded byVahid Rezaei

- Lee J. Bain, Max Engelhardt-Introduction to Probability and Mathematical Statistics (2000)Uploaded bynadia_anisa22
- Ch 4 for DisplayUploaded byRahul Saxena
- 12th math ncertUploaded byKumar Pranay
- ch-3-3502Uploaded byahmadskhan
- Monte Carlo SimulationUploaded byNilton de Sousa
- 2.Discrete Random VariablesUploaded byDeephikaKoppulian
- Probability Distribution Theory Class Test 2 NotesUploaded byAnonymous 8nJXGPKnuW
- 500 CompleteUploaded byzedisded
- Tutorial List 1Uploaded byनेपाली नेवरि प्रसन्न
- ProbabilityUploaded byniogan anthony
- Probability and StatisticsUploaded bytaylor
- PTSP JNTU SYLLABUSUploaded byGS Naveen Kumar
- 485-HWUploaded byjiuj78
- Basic Stats and ProbabilityUploaded byMustafa Ghafouri
- Final Under GradUploaded byadityabaid4
- Corelation CoeficientUploaded bynitin30
- Probability DistributionshUploaded byمحمد سلام
- NotesUploaded byaparna1407
- NR-221101-Probability and Random VariablesUploaded bySrinivasa Rao G
- The Martingale Stopping TheoremUploaded byAravindVR
- QueueingUploaded bymaria
- lect1aUploaded byCeyhun Eyigün
- Reduction of Random Variables in Structural Reliability AnalysisUploaded byAbdul Hamid Hamat
- 5935350Uploaded byJorge Pacheco
- ...Siggraph02 - Advanced global illumination.pdfUploaded bymanelf
- 2 ECE II year R15Uploaded byNaveen Yallapu
- johnson_probstats8e_Section6_7.pdfUploaded bygtrin15
- Chapter 8-Random Variables and Probability ModelsUploaded bySomin Lee
- EntropyUploaded byjohnoftheroad
- Mathematical Exploration StatisticsUploaded byanhhuyalex

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.