0 Up votes0 Down votes

123 views24 pagesApr 07, 2011

© Attribution Non-Commercial (BY-NC)

PDF, TXT or read online from Scribd

Attribution Non-Commercial (BY-NC)

123 views

Attribution Non-Commercial (BY-NC)

- Data mining & analysis
- SI lec 5
- Probability-and-Distribution.pdf
- Session 5a
- FCM2063 - Printed Notes
- Prob. review
- ENGR 371 Midterm February 1999
- MATH30-6 Course Syllabus 2Q2012-2013
- 01 as and a Level Mathematics Course Planner Issue 2 (1)
- 4.Sta.probabilitas.1
- 2017- Lecture Series 5 - Random Variables
- Practice Test 3 Key
- Texture TMI
- Syllabus
- 3 Probability
- Quiz2
- Formula-Sheet-CO1.docx
- Textbook2 Taleb
- 23633976 Probability and Statistics
- Discrete_dist_week_3.pptx

You are on page 1of 24

A Concise Course in

Statistical Inference

By Larry Wasserman

Toby Xu

UW Madison

05/29/07

Chapter 1 Overview

Sample Space

Disjoint or Mutually Exclusive

Probability Axioms/Basic Properties

Finite sample spaces

Independent Events

Conditional Probability

Baye’s Theorem

Sample Space

Sample Space Ω is the set of possible outcomes.

Ex: Toss fair coin twice Ω={HH,HT,TH,TT}

Points ω in Ω are called sample outcomes, realizations,

elements.

Subsets of Ω are called events.

Ex: Event that first toss is head is A={HH,HT}

Ac = complement of A (or NOT A)

∅ = complement of Ω

A∪ B = {ω ∈ Ω : ω ∈ A or ω ∈ B or ω ∈ both}

A∩ B = {ω ∈ Ω : ω ∈ A and ω ∈ B}

Disjoint or Mutually Exclusive

Two events (A and B) are mutually exclusive iff A∩ B = ∅

Indicator function of A

Monotone increasing if A1 ⊂ A 2 ⊂ A3 ⋯

Monotone decreasing if A1 ⊃ A2 ⊃ A3 ⋯

Intro to Probability

P(A) = Probability Distribution or Probability Measure

Axioms:

1) P(A)≥0 for every A

2) P(Ω) = 1

If A1,A2…are disjoint then

∞ ∞

P (∪ A i ) = ∑ P ( A i )

i =1 i =1

Statistical Interference:

Frequentist

Bayesian Schools

Basic Properties of Probability

P(∅) = 0

A ⊂ B ⇒ P ( A) ≤ P (B )

0 ≤ P ( A) ≤ 1

A∩ B = ∅ ⇒ P ( A∪ B ) = P ( A) + P ( B ) For Disjoint Probabilities only

P(Ac)=1-P(A)

P ( A∪ B ) = P ( A) + P ( B ) − P ( AB )

A or B = (ABc) or (AB) or (AcB)

Probability on Finite Sample

Spaces

P(A)=A/Ω n n

= = 1

n n! 0 n

= N choose k n n

k k !( n − k )! =

k n − k

• N choose K is counting how many ways can we get a

k-subset out from a set with n elements.

•Ex: There’s 10 people in the Book Club; We want

groups of 2 people. How many possible

combinations?

10 10!

= = 45

2 2!(10 − 2)!

Independent Events

Two events are independent (does not directly affect the probability

of the other from happening) iff: P(AB) = P(A)P(B)

A set of events {Ai : i ε I} is independent if P (∩ A i ) = ∏ P ( A i )

i∈ J i∈ J

Assume A & B are disjoint events, each with +ve probability. Can

they be independent?

Ans: No because:

Independent = P(AB) = P(A)P(B)

But here P(AB)=φ=0 and P(A)P(B)>0

Ex: Toss a fair die A={2,4,6} B{1,2,3,4} A B = {2, 4}

P(AB)=2/6=P(A)P(B)=1/2*2/3

∩

Conditional Probability

Assuming P(B)>0…P(A|B)=P(AB)/P(B)

If A & B are independent then P(A|B)=P(A)

Ex: Draw Ace of clubs (A) and then Queen of Diamonds (B)

P(AB)=P(A)P(B|A)=1/52*1/51=1/2652

P(.|B) satisfies the axioms, for fixed B

P(A|.) does not satisfy the axioms of probability, for fixed A

In general P(A|B) ≠ P(B|A)

Bayes’ Theorem

k

The Law of Total Probability P ( B ) = ∑ P ( B | Ai ) P ( Ai )

i =1

Ω

Where Cj = BAj C1…k are disjoint

each i. If P(B)>0 then, for each i=1,…,k

P( B | Ai ) P( Ai )

P( Ai / B) =

∑ P( B | Aj ) P( Aj )

Additional Examples

Disease Test with + or – outcomes D Dc

probability of one having disease? Ans: not - .001 .891

90%...actually 8% P (+∩ D ) .009

P(D | +) = = ≈ 0.08

P(+) .009 + .099

3 Catergories of Email: A1= “spam”, A2= “low priority”,

A3= “high priority”

P(A1)=.7,P(A2)=.2,P(A3)=.1 Let B be event that email

contains the word “Free”

P(B|A1)=.9, P(B|A2)=.01, P(B|A3)=.01

Q: receive an email with word “free”, what’s the

probability that it is spam?

.9 × .7

P ( A1 | B) = = .995

(.9 × .7) + (.01× .2) + (.01× .1)

Chapter 2 Overview

Random Variable

Cumulative Distribution Function (CDF)

Discrete Vs. Continuous probability functions (Probability Density

Function PDF)

Discrete: Point Mass, Discrete Uniform, Bernoulli, Binomial, Geometric, Poisson

Distribution

Continuous: Uniform, Normal (Gaussian), Exponential, Gamma, Beta, t and Cauchy,

Χ2

Bivariate Distribution

Marginal Distribution

Independent Random Variables

Conditional Distribution

Multivariate Distributions: Multinomial, Multivariate Normal

Transformations of Random Variables.

Random Variable

A random variable is a mapping ω P(ω) X(ω)

X: Ω R TT ¼ 0

That assigns a → real number TH ¼ 1

X(ω) to each outcome ω

Ex: Flip coin twice and let X be HT ¼ 1

number of heads. HH ¼ 2

x P(X=x)

0 ¼

1 ½

2 ¼

Cumulative Distribution

Function

CDF = FX: R→[0,1] Theorem: let X and Y have

CDF F and G. If F(x) = G(x)

FX(x)=P(X≤x) for all x, then P(X∈ A) = P(Y ∈

Ex: Flip a fair coin twice A)

Theorem: CDF needs to

and let X be number of

satisfy:

heads F is non-decreasing: x1<x2,

also F(x1)≤F(x2)

0 x<0

1/4 F is normalized:

0 ≤ x<1

Fx ( x) = lim F ( x) = 0

3/4 1 ≤ x<2 x →−∞

x →+∞

F is right-continuous:

F(x) = F(x+) for all,

where F(x+)= lim F ( y )

y→x

y>x

Discrete Vs. Continuous

Distributions

Def: X is discrete if it take countably many values. We define the

probability function or probability mass function for X by

fX(x)=P(X=x)

Def: X is continuous if there exists a function fX such that fX(x)≥0 for

all x,+∞ and for every a≤b

∫ f ( x )dx = 1

−∞ b

P ( a < X < b ) = ∫ f ( x )dx

a

x

Fx ( x ) = ∫

−∞

fx(t )dt

Note: mention Lemmas (pg 25)

Some Important Discrete

Functions

Point Mass Distribution: if P(X=a)=1 otherwise 0

Discrete Uniform

f(x)= 1

for x=1,...,k

k

0 otherwise

f(x)=Px(1-p)1-x for x ε {0,1}

Binomial (flip coins n time and let X be number of heads)

f(x)=P(X=x)={ n for x=0,…,n

P ^ x(1 − p )^ ( n − x)

x

{0 otherwise

Geometric p ε (0,1) (Think X as the No. of flips needed to get the 1st head)

P(X=k)=p(1-P)k-1, k=1,2,3,…

Poisson (Counting rare events like radioactive decay or traffic accidents)

λ^ x x≥0

f ( x )= e ^ ( − λ )

x!

Some Important Continuous

Distributions

Uniform Distribution

f(x)={1/(b-a) for x ∈ [a,b]

{0 otherwise

Normal (Gaussian)

1 1

f ( x) = exp{ − ( x − µ )^ 2}

σ 2π 2σ ^ 2

if X ~ N(µ, σ2), then Z=(X- µ)/ σ N(0,1)

Exponential (Model lifetimes of electronic components/ waiting

times between rare events)

1

f ( x) = e ^ ( − x / β ), x > 0

β

t distribution is like Normal except with thinker tails

Cauchy is special case of t distribution where ν=1

X2 distribution deals with various degrees of freedom

Bivariate Distribution

Discrete random variables Y=0 Y=1

X=0 1/9 2/9 1/3

function by f(x,y)=P(X=x

and Y=y) X=1 2/9 4/9 2/3

F(1,1)=P(X=1,Y=1)=4/9

1/3 2/3 1

In continuous case, we use

f(x,y) a PDF for the random

variables (X,Y)

(i) f(x,y) ≥ 0 for all (x,y)

(ii) ∞ ∞

∫∫

−∞ −∞

f ( x , y ) dxdy = 1 and ,

A) =

∫∫ f ( x, y )dxdy

Marginal Distributions

Def: if (X,Y) have joint distribution with mass function fX,Y,

then the marginal mass function for X is defined by

fX ( x ) = P ( X = x ) = ∑ P ( X = x , Y = y ) = ∑ f ( x , y )

y y

fY ( x ) = P ( X = x ) = ∑ P ( X = x , Y = y ) = ∑ f ( x , y )

x x

For continuous random variables, the marginal densities are

fX ( x) = ∫ f ( x, y )dy

Y=0 Y=1

fY ( y ) = ∫ f ( x, y )dx X=0 1/10 2/10 3/10

X=1 3/10 4/10 7/10

4/10 6/10

Independent Random

Variables

Def: Two random variables X & Y=0 Y=1

Y are independent iff, for every

X=0 ½ 0 ½

A & B,

X=1 0 ½ ½

P( X ∈ A, Y ∈ B) = P( X ∈ A) P(Y ∈ B) ½ ½ 1

Y=0 Y=1

X=0 ¼ ¼ ½

These are not independent

because

X=1 ¼ ¼ ½

fX(0)fY(1)=(1/2)(1/2)=1/4

½ ½ 1

yet f(0,1)=0

fX(0)fY(0)=f(0,0)

fX(0)fY(1)=f(0,1)

fX(1)fY(0)=f(1,0)

fX(1)fY(1)=f(1,1)

Conditional Distributions

Def:

f(x|y)=P(X=x|Y=y)=P(X=x,Y=y)/P(Y=y)=f(x,y)/f(y)

Assuming that f(y)>0

Def: for continuous random variables,

f(x|y)= f(x,y)/f(y)

Assuming that f(y)>0. Then,

P ( X ∈ A / Y = y ) = ∫ f ( x | y ) dx

A

Multivariate Distribution and IID

Samples

If X1,…,Xn are independent and each has the same marginal

distribution with CDF F, we say that X1,…,Xn are independent

and identically distributed and we write

X1,…Xn ~ F

Random sample size n from F

2 Important Multivariable

Distributions

Multinomial: multivariate version of a Binomial

X ~ Multinomial(n,p)

n

f ( x ) = P1^ ( x1)... Pk ^ ( xk )

x1 ... xk

Where

n n!

=

x1 ... xk x1!...xk !

Transformations of Random

Variables

Three steps for Transformations

1. For each y, find the set Ay={x:r(x)≤y)

2.Find CDF

F(y)=P(Y ≤y)=P(r(X) ≤y)

=P({x;r(x) ≤y})

=

∫ fx ( x ) dx

Ay

3. The PDF is f(y)=F’(y)

- Data mining & analysisUploaded byФранческо Леньяме
- SI lec 5Uploaded byMuhammad Rafeeq
- Probability-and-Distribution.pdfUploaded byckvirtualize
- Session 5aUploaded byRatnam Dubey
- FCM2063 - Printed NotesUploaded byRenuga Subramaniam
- Prob. reviewUploaded bydewddude
- ENGR 371 Midterm February 1999Uploaded byamnesiann
- MATH30-6 Course Syllabus 2Q2012-2013Uploaded byAdrian Gilbert Ratuita
- 01 as and a Level Mathematics Course Planner Issue 2 (1)Uploaded byTha Libra Fraudster
- 4.Sta.probabilitas.1Uploaded bymona
- 2017- Lecture Series 5 - Random VariablesUploaded byMateusz
- Practice Test 3 KeyUploaded bywise_guy398168
- Texture TMIUploaded byAnonymous 6WeX00mx0
- SyllabusUploaded byLisa Vegas
- 3 ProbabilityUploaded byIvan Llovit
- Quiz2Uploaded byRoy Lin
- Formula-Sheet-CO1.docxUploaded byJeffersonTalan
- Textbook2 TalebUploaded byDom DeSicilia
- 23633976 Probability and StatisticsUploaded bydeokarsahil
- Discrete_dist_week_3.pptxUploaded byLibyaFlower
- Discrete-Event SimulationUploaded bymhlin
- Answers to Probability related problems from Problem Set and Slides .rtfUploaded byRam Mohan
- Uniform DistributionUploaded byvishwavichu
- Test 8. ProbabilityUploaded by台大人家教中心
- UntitledUploaded byexammy
- INVENTORY SYSTEMUploaded byVinayKumarSingh
- image_enhancement_spatial_domain.pdfUploaded byJinal Waykar
- Bridge Performance Indicators Based on Traffic Load MonitoringUploaded byRodrigo Astroza
- IT Complete SyllUploaded byHanisha Addanki
- 10.1146@annurev-neuro-071013-014017Uploaded byPablo Carreño

- SMS Unit 6 notes 8th SemUploaded bygvarun_1989
- Control Charts for the Log-Logistic DistributionUploaded byChristo Lembang
- RandomVariables_ProbDistUploaded byAnkur Sinha
- Binomial and PoissonUploaded byjonatan
- Reliability DistributionsUploaded byAhmedBassyouni
- Probability Densities 1Uploaded byArmand Daputra
- Tutorial 3Uploaded byAshwin Kumar
- OC curve.xlsUploaded byAnonymous tv3qpx2
- Send to TyUploaded byvee979
- leac205 (1)Uploaded byAkanksha Gupta
- Statistics 578 Assignemnt 2Uploaded byMia Dee
- c4 Formula SheetUploaded byleolong34
- ENGR371 Course outlineUploaded byCommissar Coprobo
- Hydrological statistics and extremesUploaded byluis chacon
- Multivariate Normal DistributionUploaded byccffff
- Handout_5_2011Uploaded byCassie Desilva
- Chapter 5 in Class ProblemsUploaded byBharat Mendiratta
- hw02Uploaded byAbdullah Wasfi
- FRA Mid Term Final MarksUploaded byHEM BANSAL
- M3.Normal Distribution.Final.pdfUploaded byÖzZíé Ming Ren
- Lecture 4 Sampling DistributionUploaded bySyafiq Cha
- 59913625-Guo-Spring-08-P.txtUploaded byTyrone Barksdale
- UT Dallas Syllabus for se3341.501.09s taught by Michael Baron (mbaron)Uploaded byUT Dallas Provost's Technology Group
- 2rvariables SlidesUploaded bybob
- Lecture ProbUploaded byAnonymous qNFXQTT
- Tabel cdf Distribusi Normal Standar V3.pdfUploaded byhamkane
- Econ 251 examUploaded byjoe
- 6&7Uploaded byAmsalu Walelign
- lecture_notes_ece5510_f09_all.pdfUploaded byRahul Dey
- 6684_01_que_20050622Uploaded byChristopher Williams

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.