This action might not be possible to undo. Are you sure you want to continue?

Welcome to Scribd! Start your free trial and access books, documents and more.Find out more

Lesson 0: Preliminaries

Le Thi Xuan Mai

The university of natural sciences

February 24, 2013

T.X.M. Le

Bayesian statistics

Common distribution

Discrete random variables

T.X.M. Le

Bayesian statistics

Common distribution

**The binomial distribution
**

As its name suggests, the binomial distribution refers to random variables with two outcomes. Example: smoking status: a person does or does not smoke healthy insurance coverage: a person does or does not have health insurance. We now consider the ﬁrst example to demonstrate the calculation of binomial probabilities. Suppose that four adults have been randomly selected and asked whether or not they currently smoke. The random variable of interest in this example is the number of persons who respond yes to the question about smoking (denoted by X ).

T.X.M. Le

Bayesian statistics

Common distribution

Suppose that in the population, the proportion of people who would respond ”yes” to this question is θ, and the probability of a response of ”no” is then 1 − θ. Since each person is independent of all the other persons, the binomial probability can be calculated as follows P (X = 4) = θ4 (the number of 4 responses is θ4 ) P (X = 3) = 4θ3 (1 − θ) (since there are four occurrences of three yes responses) P (X = 2) = 6θ2 (1 − θ)2 P (X = 1) = 4θ(1 − θ)3 P (X = 0) = (1 − θ)4

T.X.M. Le

Bayesian statistics

25)1 × (0. Then the probability of each outcome is as follows 0 P (X = 4) = C4 × (0. Le Bayesian statistics .0469 = P{1 no response} 2 P (X = 2) = C4 × (0.0039 = P{0 no responses} 1 P (X = 3) = C4 × (0.75)1 = 0.X.2109 = P{2 no response} 3 P (X = 1) = C4 × (0.25)2 × (0.25)3 × (0. T.75)3 = 0.M.3164 = P{4 no responses} This lead us to deﬁne a binomial random variable and calculate its probability distribution.75)4 = 0.25)4 × (0.75)2 = 0.Common distribution Suppose θ is 0.25.75)0 = 0.4219 = P{3 no response} 4 P (X = 0) = C4 × (0.25)0 × (0.

1] and y y P (Y = y |θ. E [Y |θ] Var[Y |θ] mode[Y |θ] p (y |θ. = dbinom(y . θ) are independent. 1.Common distribution A random variable Y ∈ {0. n} has a binomial (n. Le Bayesian statistics . = [(n + 1)θ].X. = nθ(1 − θ).M. If Y1 ∼ binomial(n1 . . The binomial(n. T. then Y = Y1 + Y2 ∼ binomial(n1 + n2 . n) = nθ . . n}. θ). . θ). θ) distribution if θ ∈ [0. θ) model assume that Y is a sum of independent binary(θ) random variables. . θ) and Y2 ∼ binomial(n2 . . n) = Cn θ (1 − θ)n−y for y ∈ {0. For this distribution. . 1. n. . When n = 1 this distribution is called the Bernoulli distribution. .

Common distribution Figure : Binomial distribution with n = 10 and θ = 0. T. Le Bayesian statistics .2.X.M.

8. T.X. Le Bayesian statistics .Common distribution Figure : Binomial distribution with n = 10 and θ = 0.M.

T.X. Le Bayesian statistics .2.Common distribution Figure : Binomial distribution with n = 100 and θ = 0.M.

Common distribution Figure : Binomial distribution with n = 100 and θ = 0. Le Bayesian statistics . T.M.8.X.

. 1. mode[Y |r . θ). T. = dnbinom(y . θ] = 0 if r ≤ 1. Le Bayesian statistics .X. For this distribution. = r θ/(1 − θ)2 . θ) distribution for any positive integer r if 0 ≤ θ ≤ 1 and y y r P ( y | r . = [(1 − θ)(r − 1)/θ] if r > 1. θ] mode[Y |r . . θ ) = Cy +r −1 θ (1 − θ ) . E [Y |r . θ] p (y |θ ) = r θ/(1 − θ).M. . θ ] Var[Y |r .} has a negative binomial(r .Common distribution The negative binomial distribution A random variable Y ∈ {0.

Common distribution Figure : Negative binomial distribution.X. Le Bayesian statistics .M. T.

M.Common distribution The name ”negative binomial” can be replaced by ”inverse binomial” because it arises from a series of Bernoulli trials where the number of successes is ﬁxed at the outset. The geometric is the special case of the negative binomial having r = 1. T.X. It is the nuber of failures preceding the ﬁrst success in a series of Bernoulli trials. This distribution is sometimes used to model countably inﬁnite random variables having variance substantially larger than the mean (so that the Poisson model would be inappropriate). Le Bayesian statistics . Y is the number of failure preceding the r th success. In this case.

.X. 1. = θ. 1. E [Y |θ ] Var[Y |θ] mode[Y |θ] p (y |θ) = θ. then Y = Y1 + Y2 ∼ Poisson(θ1 + θ2 ). T. = [θ].M. . .Common distribution The Poisson distribution A random variable Y ∈ {0. Le Bayesian statistics .}. = dpois(y . . then the Poisson model may not be appropriate. θ y e −θ for y ∈ {0. y! If Y1 ∼ Poisson(θ1 ) and Y2 ∼ Poisson(θ2 ) are independent. . If it is observed that a sample mean is very diﬀerent than the sample variance.} has a Poisson(θ) distribution if θ > 0 and P (Y = y |θ) = For this distribution. . θ).

M.X. 10.Common distribution Figure : Poisson distribution with θ = 1. 4. T. Le Bayesian statistics .

Common distribution Continuous random variables T. Le Bayesian statistics .M.X.

p (y |θ. = σ2. σ 2 ] = θ. theta. 1 2πσ 2 e − 2 ( y −θ ) 1 2 /σ 2 for − ∞ < y < ∞. T.X. σ 2 ) = √ For this distribution. E [Y |θ. σ 2 ) = dnorm(y . Le Bayesian statistics . σ 2 ] Var[Y |θ. a2 σ1 + b 2 σ2 ).Common distribution The univariate normal distribution A random variable Y ∈ R has a normal(θ. σ 2 ] = θ.M. σ1 2 2 2 independent. σ 2 ) distribution if σ 2 > 0 and p (y |θ. then 2 2 aX1 + bX2 + c ∼ normal(aθ1 + b θ2 + c . σ 2 ) are If X1 ∼ normal(θ1 . 2 ) and X ∼ normal(θ . mode[Y |θ. sigma).

T.X. Le Bayesian statistics .Common distribution Figure : Some univariate normal distributions.M.

though the distribution is proper for any positive real number ν .X. ν/(ν − 2) if ν > 2. Var[Y |ν ] = ∞ if 1 < ν ≤ 2. Le Bayesian statistics .Common distribution The t distribution A random variable Y ∈ R has a t distribution(ν ) if ν > 0.M. The parameter ν is referred to as the degrees of freedom and is usually taken to be a positive integer. The t is a common heavy-tailed (but still symmetric and unimodal) alternative to the normal distribution. 0 0 dt(y . E [Y |ν ] Var[Y |ν ] mode median p (y |ν ) = = = = = 0 if ν > 1. nu). T. p (y |ν ) = √ ν νπ Γ(ν/2) For this distribution. and y2 Γ[(ν + 1)/2] 1 + ]−(ν +1)/2 .

M.X.Common distribution Figure : Some t distributions. Le Bayesian statistics . T.

X. Its density distribution is of the form 1 for a ≤ y ≤ b b −a p (y |a. E [Y ] = a+b 2 1 12 (b a+b 2 Var [Y ] = median = mode T. b ) = (1) 0 else where For this distribution. b ].Common distribution The continuous uniform probability distribution If a continuous r. Le − a)2 = any value in[a. b ) is constant over that interval and equal to zero elsewhere.M. Bayesian statistics . then Y is said to be uniformly distributed and its distribution is called a continuous uniform probability distribution or continuous rectangular distribution.v Y can assume any value in the interval a ≤ y ≤ b and only these values. and if its probability density function p (y |a.

Common distribution Figure : Uniform distribution. T.X. Le Bayesian statistics .M.

Common distribution The beta distribution A random variable Y ∈ [0. b ). T. Le Bayesian statistics . ab = E [Y ] × E [1 − Y ] (a+b +1)(a+b )2 a−1 (a−1)+(b −1) if a > 1 and b > 1. b ) = dbeta(y . mode[Y |a. b ) distribution if a > 0. The beta distribution is closely related to the gamma distribution. b ] Var[Y |a. b ] = = Γ(a + b ) a−1 y (1 − y )b−1 for 0 ≤ y ≤ 1. b ] = p (y |a. × 1 a+b +1 . b ) = E [Y |a . a. 1] has a beta(a. A multivariate version of the beta distribution is the Dirichlet distribution. Γ(a)Γ(b ) a a+b .X. b > 0 and p (y |a.M.

M. T. Le Bayesian statistics .Common distribution Figure : Beta distribution.X.

b ). ∞) has a gamma(a. b ) = E [Y |a. b ) distribution if p (y |a. σ 2 ) then Y 2 ∼ gamma(1/2. b ) and Y2 ∼ gamma(a2 . If Y ∼ normal(0. b ) and Y1 /(Y1 + Y2 ) ∼ beta(a1 . a −1 b Var[Y |a. a2 ). Γ(a) a b. 1/2) distribution. mode[Y |a. dgamma(y . If Y ∼ normal(0. Le Bayesian statistics . b ] = a .Common distribution The gamma and inverse-gamma distributions A random variable Y ∈ (0. 1/2σ 2 ). b ] = 0 if 0 < a < 1. b > 0. a. The chi-square distribution with ν degrees of freedom is the same as a gamma(ν/2. b ] = p (y |a. b ] = b a a−1 −by y e for y > 0. b ) = if a ≥ 1.M. T. If Y1 ∼ gamma(a1 . then Y1 + Y2 ∼ gamma(a1 + a2 . b2 mode[Y |a. 1) then Y 2 ∼ chi-square(1). a > 0.X. b ) are independent.

M. T.Common distribution Figure : Gamma distributions.X. Le Bayesian statistics .

b ) distribution. b ). The density of Y is p (y |a. b ] = = b a −1 b a −a−1 −b/y y e for y > 0. if a ≥ 2. b ] = ∞ if 0 < a < 2.Common distribution A random variable Y ∈ (0. if X ∼ gamma(a. b ] = b2 (a−1)2 (a−2) b a+1 . b ] Var[Y |a. ∞) has an inverse-gamma(a. b ) and Y = 1/X . mode[X |a. b ) distribution if 1/Y has a gamma(a. T. b ] = ∞ if 0 < a < 1. Le Bayesian statistics . Note that the inverse-gamma density is not simply the gamma density with y replaced by 1/y . with a = ν/2 and b = 1/2. In other words. Γ(a) if a ≥ 1. b ) = For this distribution. The inverse-χ2 (ν ) is a special case of the inverse-gamma distribution. Var[Y |a. E [Y |a. then Y ∼ inverse-gamma(a.M.X. E [Y |a .

the scaled distribution has an extra parameter τ 2 . ν.M. Compared to the inverse-chi-squared distribution.X.Common distribution The scaled inverse chisquare distribution The family of scaled inverse chi-squared distributions is closely related to two other distribution families: the inverse-chi-squared distribution and the inverse gamma distribution. which scales the distribution horizontally and vertically. T. τ 2 ) then f (x . The relationship between the inverse-chi-squared distribution the scaled distribution is that if X ∼ inv-χ2 (ν ) ντ 2 The probability density function of the scaled inverse chi-squared distribution extends over the domain x > 0 and is X ∼ Scale-inv-χ2 (ν. τ 2 ) = ντ 2 (ντ 2 /2)ν/2 −ν/2−1 x exp(− ) Γ(ν/2) 2x where ν is the degrees of freedom parameter and τ 2 is the scale parameter. Le Bayesian statistics .

Le Bayesian statistics .X. τ 2 ) then If X ∼ Scale-inv-χ2 (ν. 2 2 If X ∼ Scale-inv-χ2 (ν.M. τ 2 ) then X ∼ Inv-Gamma T. τ 2 ) then kX ∼ Scale-inv-χ2 (ν. k τ 2 ) If X ∼ Scale-inv-χ2 (ν.Common distribution EX Var (X ) = = mode(X ) = ντ 2 for ν > 2 ν−2 2ν 2 τ 4 for ν > 4 (ν − 2)2 (ν − 4) ντ 2 ν+2 X ∼ inv-χ2 (ν ) ντ 2 ν ντ 2 .

The exponential distribution is a Gamma(1. For this distribution. p (y |λ) = dexp(y . T. mode[Y |λ] = 0. lambda). E [Y |λ] Var[Y |λ] = 1/λ.X.Common distribution Exponential distributions A random variable Y > 0 has an exponential distribution(λ) if λ > 0 and p (y |λ) = λ exp(−λy ). λ) distribution.M. = 1/λ2 . Le Bayesian statistics .

Common distribution Figure : Exponential distributions.M.X. T. Le Bayesian statistics .

σ 2 > 0 and p (y |µ. This distribution is symmetric and unimodal. σ 2 ] = µ.Common distribution The double exponential distribution (Laplace distribution) A random variable Y ∈ R has a Laplace distribution(µ. but has heavier tails and a somewhat diﬀerent shape. being strictly concave up on both sides of µ.M. E [Y |µ. mu. σ 2 ) = For this distribution. = 2σ 2 . 2σ σ mode[Y |µ. |y − µ| 1 exp(− ). Le Bayesian statistics . σ 2 ] Var[Y |µ. σ 2 ] = µ. p (y |µ. σ 2 ) if −∞ < µ < ∞. σ 2 ) = dlaplace(y .X. sigma). T.

X. T.Common distribution Figure : Laplace distributions.M. Le Bayesian statistics .

σ 2 ) = For this distribution. mode[Y |µ. σ 2 ] Var[Y |µ. T.M. more similar to the normal in appearance than the double exponential. σ 2 ] = µ. σ 2 ) if −∞ < µ < ∞. σ 2 ] = µ. p (y |µ. sigma).X. = (π 2 /3)σ 2 . µ exp(− y − σ ) µ 2 σ [1 + exp(− y − σ )] . σ 2 ) = dlogis(y . Le Bayesian statistics . but with even heavier tails.Common distribution The logistic distribution A random variable Y ∈ R has a Logistic distribution(µ. mu. The logistic is another symmetric unimodal distribution. σ 2 > 0 and p (y |µ. E [Y |µ.

M. T.X.Common distribution Figure : Some logistic distributions. Le Bayesian statistics .

This is a special case of the t distribution t (1. µ. though µ is the median of this distribution.Common distribution Cauchy distribution A random variable Y ∈ R has a Cauchy distribution (µ. σπ [1 + ( x − σ ) ] For this distribution. σ 2 ) which has the heaviest possible tails. E [Y ] and Var[Y ] do not exist. σ 2 ) = 1 µ 2 . σ 2 > 0 and p (y |µ. Le Bayesian statistics . −∞ < µ < ∞. σ 2 ) if ν > 0.M. T.X.

Le Bayesian statistics .X.M.Common distribution Figure : Some Cauchy distributions. T.

Σ] = θ.X. T. E [Y |θ. Σ) distribution if Σ is a positive deﬁnite p × p matrix and 1 p (y |θ. Le Bayesian statistics . Σ] Var[Y |θ. Σ] = θ.Common distribution The multivariate normal distribution A random vector Y ∈ R p has a multivariate normal distribution normal(θ.M. Σ) = (2π )−p/2 |Σ|−1/2 exp{− (y − θ)T Σ(y − θ)} for y ∈ R p . 2 For this distribution. mode[Y |θ. = Σ.

M. ncol = p ) = t (t (Z % ∗ %chol(Sigma)) + c (theta)). .d samples from a multivariate normal distribution: Z Y = matrix(rnorm(n ∗ p ).X. Σ2 ) are independent. .Common distribution If Y1 ∼ normal(θ1 . nrow = n. . T.i. . Zp ∼ i. normal(0. Σ). then aY1 + bY2 + c ∼ normal(aθ1 + b θ2 + c . a2 Σ1 + b 2 Σ2 ).d.i. The following R-code will generate an n × p matrix such that the rows are i. If Z is the vector with elements Z1 . 1) then Y = θ + AZ ∼ multivariate normal(θ. Le Bayesian statistics . Σ1 ) and Y2 ∼ normal(θ2 . where and AAT = Σ (A is the Choleski factorization of Σ).

j |ν. M ] = (ν − p − 1)M .X.M. mode[Y |ν. M ) distribution if the integer ν ≥ p . M ] = νM. M ] Var[Yi .j ).Common distribution The Wishart and inverse-Wishart disttributions A random p × p symmetric positive deﬁnite matrix Y has a Wishart(ν. M is a p × p symmetric positive deﬁnite matrix and p (Y |ν.j + mi . Le Bayesian statistics . E [Y |ν. and aj .i mj . exp{tr (A)} = exp( For this distribution. M ) = [2ν p/2 Γp (ν/2)|M |ν/2 ]−1 ×|Y |(ν −p−1)/2 exp{tr (−M −1 Y /2)}. T.j ). where p Γp (ν/2) = π p (p −1)/4 j =1 Γ[(ν + 1 − j )/2]. = ν × (mi2.

Xν ∼ i.d.Common distribution The Wishart distribution is a multivariate version of the gamma distribution.M. Mp×p ). . .X. then Xi XiT ∼ Wishart (ν. . Mp×p ). i =1 The following R-code is used to generate a Wishart distributed random matrix: X Z Y = matrix(rnorm(ν ∗ p ). Le Bayesian statistics . multivariate normal(0. . ν If X1 .i.nrow=nu.ncol=p) #standard normal matr = X % ∗ %chol(M) = t (Z )% ∗ %Z # #Wishart matrix T.

so that Σ ∼ inverse-Wishart(ν. M ] = (ν − p − 1)−1 M −1 . M ). T. For this distribution. M ) and X = Y −1 . S −1 ). If Σ0 were the most probable value of Σ a priori.X. The density of Y is p (Y |ν. M ) =[2ν p/2 Γp (ν/2)|M |ν/2 ]−1 × |Y |−(ν +p+1)/2 exp{tr (−M −1 Y −1 /2)}. then we could set S = (ν0 + p + 1)Σ0 . if X ∼ Wishart(ν. M ) distribution. Le Bayesian statistics .M. mode[Y |ν. we have mode[Y |ν. In other words. S ] = (ν + p + 1)−1 S . M ] = (ν + p + 1)−1 M −1 . [(ν + p − 1)Σ0 ]−1 ) and mode[Σ|ν. S ] = Σ0 . M ) distribution if X −1 has a Wishart(ν.Common distribution A random p × p symmetric positive deﬁnite matrix Y has an inverse-Wishart(ν. then Y ∼ inverse-Wishart(ν. If Σ ∼ inverse-Wishart(ν. E [Y |ν.

- Attribution
- 50 Most Frequently Used UNIX _ Linux Commands (With Examples)
- Huge.pdf
- Parallel Computing — Julia Language 0.4.pdf
- GPUs_ Path into the future _ Parallel Computing Institute.pdf
- Mau co mat
- Moving Koh
- XFEM
- Quyen-Tom-Tat
- Pt Vi phan cap 2 he so hang
- Bai Giang Lap Trinh Huong Doi Tuong Va C
- fopen.pdf
- Add or Delete Captions - Word - Office
- HotelCalifornia_GuitarNick.pdf
- Shortcuts for Win 8
- romance.pdf
- Heaven 2nd mandorin by nicola mandorino
- deserve.pdf
- sell.pdf
- toan_giai_tich_2-2012.doc
- huong_dan_viet_tieu_luan_tong_quan.doc
- BiquyethocvaluyenthimontiengAnh.doc
- vietnam-wastes-too-much-money-to-train-gifted-students.pdf
- Thông Báo Von Neumann – ĐHQG-HCM
- Thông báo tuyển sinh đào tạo trình độ thạc sĩ năm 2013_Trường Đại Học Khoa Học Tự Nhiên

- Eng iSM Chapter 6
- PRP Unit-1
- Chart of Distribution Relationships
- Study Guide
- Chapter 8 (2)
- distribusi Variable.doc
- Output Printout
- Hasil SPSS New
- Lampiran SPSS
- Output SPSS
- Hasil Analisa New - Astri Output1(1)
- 8. Frequencies
- Study Korea
- Crosstabs_2
- Handout Goodness Off It
- Probability doc
- Chap1Sec8
- 1590-6
- 1. Statistik
- Binomial
- notes on bertekas text
- pqt_2
- Random Sequences
- Chp 10 Solutions
- Statistical Properties of the Random PSD With Windowing Discussion
- Concept of Probability and Probability Distribution
- Presentation 6
- Applications of Differentiation
- Average

Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

We've moved you to where you read on your other device.

Get the full title to continue

Get the full title to continue listening from where you left off, or restart the preview.

scribd