Chapter06 Normal 2023su

Basic Quantitative Method
Chapter 6 The Normal Distribution and its Applications
Department of Economics
National Taiwan University
2023.7.21.
(NTU Econ) Basic Quantitative Method 2023.7.21. 1 / 44

Normal Distributions
Section 1

Normal Random Variables
Normal distribution is also called Gaussian distribution, which is

named after German mathematician Johann Carl Friedrich Gauss
(1777–1855)
However, some authors attribute the credit for the discovery of

the normal distribution to de Moivre.
Normal Random Variables
Definition (Normal Distribution)

A random variable X has the normal distribution with two parameters
µ and σ 2 if X has a continuous distribution with the following pdf:
1 1 x−µ 2
f (x) = √ e − 2 ( σ )
σ 2π
where supp(X) = {x∣ − ∞ < x < ∞}, and π ≑ 3.14159. It is denoted by

X ∼ N(µ, σ 2 )
(google “Gaussian integral”)

∞ 1 1 x−µ 2
∫−∞ √ e− 2 ( σ ) = 1
σ 2π
Standard Normal Random Variables
Definition (Standard Normal Distribution)

A random variable Z is called a standard normal random variable, if
µ = 0 and σ = 1 with pdf
1
√ e− 2 z
1 2
2π
It is denoted by Z ∼ N(0, 1)
As a conventional notation, we let ϕ and Φ denote pdf and CDF

of a standard normal random variable,
1 z
ϕ(z) = √ e − 2 z , Φ(z) = ∫ ϕ(w)dw.
1 2
2π −∞

Solid line: N(0, 1) vs. Dashed line: N(0, 9)
0.4
N(0,1)
0.3
f(x)
0.2
0.1
N(0,9)
0.0
−10 −5 0 5 10

Stock Returns (S&P500)
Histogram of Stock Returns

0.12
0.10
0.08
Density
0.06
0.04
0.02
0.00
−20 −10 0 10

Stock Returns (S&P500)
Histogram of Stock Returns

0.12
0.10
0.08
Density
0.06
0.04
0.02
0.00
−20 −10 0 10

Normal vs. Standard Normal
Theorem
Let Z ∼ N(0, 1) and X = σ Z + µ, then
X ∼ N(µ, σ 2 )
Proof: by CDF method.

X−µ
In the same vein, you can show that if X ∼ N(µ, σ 2 ) and Z = σ ,
then Z ∼ N(0, 1).

Properties
Theorem (Invariance Under Linear Transformations)

If X ∼ N(µ, σ 2 ), then
aX + b ∼ N(aµ + b, a2 σ 2 ), a ≠ 0.

Properties
Theorem (Sum of I.I.D. Normal Random Variables)

If {X i }ni=1 ∼i.i.d. N(µ, σ 2 ), and
W = α1 X1 + α2 X2 + ⋯ + α n X n ,
then
n n
W ∼ N (µ ∑ α i , σ 2 ∑ α 2i ) .
i=1 i=1
Consider two special cases:

α i = 1 for all i
α i = n1 for all i

Properties
Theorem
If {X i }ni=1 ∼i.i.d. N(µ, σ 2 ), then
n
Y = ∑ X i ∼ N(nµ, nσ 2 ).
i=1
If {X i }ni=1 ∼i.i.d. N(µ, σ 2 ), then

n
∑i=1 X i σ2
X̄ = ∼ N (µ, ) .
n n

Finding Normal Probabilities
If X ∼ N(5, 16), compute P(X ≤ 3)

For short, we will use the notation: P(N(5, 16) ≤ 3)

Finding Normal Probabilities
However, sometimes what we have is the N(0, 1) table.

When calculating the probability of a normal distribution, we
need to transform it to the standard normal distribution.
Then calculate Φ(a) = P(Z ≤ a)
For instance, if X ∼ N(5, 16),
X−5 3−5
P(X ≤ 3) = P ( ≤ ) = P(Z ≤ −0.5) = Φ(−0.5)
4 4
The following properties will be helpful
P(Z ≤ 0) = P(Z ≥ 0) = 0.5
P(Z ≤ −a) = P(Z ≥ a)
P(−a ≤ Z ≤ 0) = P(0 ≤ Z ≤ a)
We can use the N(0, 1) table to find Φ(−0.5).
Z Table

Using Normal Probabilities to Find Quantiles
Example: Suppose the final grade, X is normally distributed with

mean 70 and standard deviation 10. The instructor wants to give
10% of the class an A+. What cutoff should the instructor use to
determine who gets an A+?
Clearly, X ∼ N(70, 100), and we want to find the constant q such
that P(X > q) = 0.10 or P(X ≤ q) = 0.90.

Using Z Table
Find c such that

X − 70 c − 70 c − 70
0.90 = P(X ≤ c) = P ( ≤ ) = P (Z ≤ )
10 10 10
According to the Z Table, P(Z ≤ 1.28) = 0.90, we have
c − 70
= 1.28
10
That is,
c = 82.8

Bivariate Normal Random Variables
Definition (Bivariate Normal Distribution)

X and Y are bivariate normal distributed if the joint pdf is
eϖ
f XY (x, y) = √
2π (1 − ρ2 )σX σY
where 2 2
1 x − µX x − µX y − µY y − µY
ϖ=− [( ) − 2ρ ( )( )+( ) ]
2(1 − ρ 2 ) σX σX σY σY

Bivariate Normal Random Variables

⎡ ⎤ ⎡ ⎤ ⎡ ⎤
⎢X ⎥ ⎛⎢⎢ µ X ⎥⎥ ⎢⎢ σX2 σXY ⎥⎥⎞
It is denoted by ⎢ ⎥ ∼ N ⎢ ⎥ , ⎢
⎢ ⎥
⎢Y ⎥ ⎝⎢ µY ⎥ ⎢σXY σY2 ⎥⎥⎠
⎣ ⎦ ⎣ ⎦ ⎣ ⎦
Bivariate Normal Density
y
x

Independent vs. Uncorrelated
Theorem
Given that X and Y have a bivariate normal distribution. If X and Y
are independent if and only if
Cov(X, Y) = 0.
Proof: we only need to show the “if” part. Clearly, when

Cov(X, Y) = 0, which implies ρ = 0, it can be shown that
f XY (x, y) = f X (x) fY (y)

Gamma Distributions
Symmetry vs. Asymmetry
Normal distribution is a symmetric distribution.

In some instances, we require a skewed distribution to
characterize the data.
For example,
the size of insurance claims
time-until-default (survival time)
We thus introduce a random variable called Chi-square random
variable to capture the asymmetrical characteristics.

Chi-square Distribution
Section 3

Definition (Chi-square Random Variables)

A random variable X has the Chi-square distribution if the pdf is
k
x 2 −1
e − 2 x , supp(X) = {x∣0 < x < ∞}
1
f (x) = k
2 Γ( k2 )
2
where k is a positive integer called the degree of freedom.
Γ(⋅) is called a Gamma function:

∞
Γ(α) = ∫ x α−1 e −x dx
0

Chi-square distributions (k = 3)
0.25
0.20
0.15
dchisq(x, df = 3)
0.10
0.05
0.00
0 5 10 15 20

Theorem
The MGF of a Chi-square random variable is
k
1 2
M X (t) = ( )
1 − 2t
Proof: by the definition of MGF, and let y = (1 − 2t)x.

Hence,
E(X) = k, Var(X) = 2k

k
x 2 −1 ∞
e − 2 x , Γ(α) = ∫ x α−1 e −x dx
1
f (x) = k
2 2 Γ( k2 ) 0
k
∞ x 2 −1
e − 2 x e tx dx
1
tx
M X (t) = E(e ) = ∫ k
0 2 Γ( k2 )
2
k
∞ x 2 −1
e − 2 (1−2t)x dx, let y = (1 − 2t)x, dx = d y/(1 − 2t)
1
= ∫ k
0 2 2 Γ( k2 )
k
k −1
∞ y 2 −1 ( 1−2t
1
)2 1
e− 2 y(
1
= ∫ k
) dy
0 2 2 Γ( k2 ) 1 − 2t
k k
1 2 ∞ y 2 −1
e− 2 y d y
1
= ( ) ∫ k
1 − 2t 0 2 2 Γ( k ) 2
− k2
= (1 − 2t)
k
1 2 k
M X (t) = ( ) = (1 − 2t)− 2
1 − 2t
dM X (t) k k k
= − (1 − 2t)− 2 −1 (−2) = k(1 − 2t)− 2 −1
dt 2
d M X (t)
2 k k k
= k(− − 1)(1 − 2t)− 2 −2 (−2) = k(k + 2)(1 − 2t)− 2 −2
dt 2 2
dM X (0)
= E(X) = k
dt
d 2 M X (0)
= E(X 2 ) = k(k + 2)
dt 2
Var(X) = E(X 2 ) − E(X)2 = k(k + 2) − k 2 = 2k

Theorem
If X i ∼ χ2 (k i ) for i = 1, 2, . . . n, and they are independent, then
n n
∑ X i ∼ χ 2 (∑ k i )
i=1 i=1
That is, the sum X1 + X2 + ⋯ + X n has the χ2 distribution with

k1 + k2 + ⋯ + k n degrees of freedom.
Proof: by MGF.

If X i ∼ χ2 (k i ) for i = 1, 2, . . . n, and they are independent, then

n n
∑ X i ∼ χ 2 (∑ k i )
i=1 i=1
Proof:
M∑ X i (t) = E(e t ∑ X i ) = E(e tX1 )E(e tX2 )⋯E(e tX n )

k1 k2 kn
1 2 1 2 1 2
= ( ) ( ) ⋯( )
1 − 2t 1 − 2t 1 − 2t
1
2 ∑ i
k
1
= ( )
1 − 2t
n n
∑ X i ∼ χ2 (∑ k i )
i=1 i=1

The following theorem links the normal distribution and Chi-square

distribution.
Theorem
Let Z ∼ N(0, 1). Then the random variable
Y = Z 2 ∼ χ2 (1)
Proof:
1∞
e tz √ e − 2 z dz
2 2 1 2
M Z 2 (t) = E(e tZ ) = ∫
−∞ 2π
1
∞ 1 − 1 (1−2t)z 2 1 2
=∫ √ e 2 dz = ( )
−∞ 2π 1 − 2t

Chi-square Distributions
Corollary
Suppose {Z1 , Z2 . . . , Z k } ∼i.i.d. N(0, 1). Let X = ∑ki=1 Z i2 . Then
X ∼ χ2 (k)
Proof: by the above two theorems.

Degree of freedom: The number of values in the final calculation
of a statistic that are free to vary

Student’s t Distribution
Section 4

Student’s t distributions
The Student’s t distribution was actually published in 1908 by a

British statistician, William Sealy Gosset (1876–1937).

William Sealy Gosset
Gosset, was employed at the Guinness Brewing Co., which

forbade its staffs publishing scientific papers due to an earlier
paper containing trade secrets.
To circumvent this restriction, Gosset used the name “Student”,
and consequently the distribution was named Student’s t
distribution.

Guinness and the 1908 Biometrika Paper

Theorem
Given two independent random variables: Z ∼ N(0, 1) and
W ∼ χ2 (k). Then
Z
U = √ ∼ t(k)
W
k
Proof: beyond the scope of this course.

t distribution is a symmetric distribution.

Limiting distribution
t(k) Ð→ N(0, 1) as k Ð→ ∞.

Comparison: t(2) vs. N(0,1)
Clearly, the Student’s t distribution has a fat tail.
0.4
N(0,1)
0.3
dnorm(x)
0.2
0.1
t(2)
0.0
−4 −2 0 2 4

F Distributions
Section 5
F Distributions

F Distributions
F Distribution
Definition (F Distribution)
Random X has the F distribution with n1 and n2 degrees of freedom
if the probability density function is
Γ( n1 +n
n1 n 1 +n 2
2
2 ) n1 2 n 1 −1 n1 − 2
( ) x 2 (1 + x)
Γ( n21 )Γ( n22 ) n2 n2
with supp(X) = {x∣0 ≤ x < ∞}. It is denoted by
X ∼ F(n1 , n2 )
It is proposed by R.A. Fisher and George W. Snedecor

F Distributions
R.A. Fisher and G. W. Snedecor
Sir Ronald A. Fisher (1890–1962), British statistician,

evolutionary biologist, eugenicist, and geneticist
George W. Snedecor (1881–1974), American mathematician and
statistician

F Distributions
F Distribution (n1 = 10, n2 = 10)
0.6
df(x, df1 = 10, df2 = 10)
0.4
0.2
0.0
0 2 4 6 8

F Distributions
F Distribution
Theorem
Let W1 and W2 be independent Chi-square random variables:
W1 ∼ χ2 (n1 ), and W2 ∼ χ2 (n2 ),
then
W1 /n1
X= ∼ F(n1 , n2 )
W2 /n2

F Distributions
F Distribution
Theorem
If t ∼ t(k), then
t 2 ∼ F(1, k)

Chapter06 Normal 2023su

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter06 Normal 2023su

Uploaded by

Copyright:

Available Formats

Basic Quantitative Method

Chapter 6 The Normal Distribution and its Applications

(NTU Econ) Basic Quantitative Method 2023.7.21. 1 / 44

(NTU Econ) Basic Quantitative Method 2023.7.21. 2 / 44

Normal Random Variables

Normal distribution is also called Gaussian distribution, which is

However, some authors attribute the credit for the discovery of

Normal Random Variables

Definition (Normal Distribution)

where supp(X) = {x∣ − ∞ < x < ∞}, and π ≑ 3.14159. It is denoted by

(google “Gaussian integral”)

Standard Normal Random Variables

Definition (Standard Normal Distribution)

As a conventional notation, we let ϕ and Φ denote pdf and CDF

(NTU Econ) Basic Quantitative Method 2023.7.21. 5 / 44

(NTU Econ) Basic Quantitative Method 2023.7.21. 6 / 44

Stock Returns (S&P500)

Histogram of Stock Returns

(NTU Econ) Basic Quantitative Method 2023.7.21. 7 / 44

Stock Returns (S&P500)

Histogram of Stock Returns

(NTU Econ) Basic Quantitative Method 2023.7.21. 8 / 44

Normal vs. Standard Normal

Proof: by CDF method.

(NTU Econ) Basic Quantitative Method 2023.7.21. 9 / 44

Theorem (Invariance Under Linear Transformations)

(NTU Econ) Basic Quantitative Method 2023.7.21. 10 / 44

Theorem (Sum of I.I.D. Normal Random Variables)

Consider two special cases:

(NTU Econ) Basic Quantitative Method 2023.7.21. 11 / 44

If {X i }ni=1 ∼i.i.d. N(µ, σ 2 ), then

(NTU Econ) Basic Quantitative Method 2023.7.21. 12 / 44

Finding Normal Probabilities

If X ∼ N(5, 16), compute P(X ≤ 3)

(NTU Econ) Basic Quantitative Method 2023.7.21. 13 / 44

Finding Normal Probabilities

However, sometimes what we have is the N(0, 1) table.

(NTU Econ) Basic Quantitative Method 2023.7.21. 15 / 44

Using Normal Probabilities to Find Quantiles

Example: Suppose the final grade, X is normally distributed with

(NTU Econ) Basic Quantitative Method 2023.7.21. 16 / 44

Find c such that

(NTU Econ) Basic Quantitative Method 2023.7.21. 17 / 44

Bivariate Normal Random Variables

Definition (Bivariate Normal Distribution)

(NTU Econ) Basic Quantitative Method 2023.7.21. 18 / 44

Bivariate Normal Random Variables

(NTU Econ) Basic Quantitative Method 2023.7.21. 19 / 44

Independent vs. Uncorrelated

Proof: we only need to show the “if” part. Clearly, when

f XY (x, y) = f X (x) fY (y)

(NTU Econ) Basic Quantitative Method 2023.7.21. 20 / 44

Symmetry vs. Asymmetry

Normal distribution is a symmetric distribution.

(NTU Econ) Basic Quantitative Method 2023.7.21. 21 / 44

(NTU Econ) Basic Quantitative Method 2023.7.21. 22 / 44

Definition (Chi-square Random Variables)

where k is a positive integer called the degree of freedom.

Γ(⋅) is called a Gamma function:

(NTU Econ) Basic Quantitative Method 2023.7.21. 23 / 44

(NTU Econ) Basic Quantitative Method 2023.7.21. 24 / 44

Proof: by the definition of MGF, and let y = (1 − 2t)x.

(NTU Econ) Basic Quantitative Method 2023.7.21. 25 / 44

(NTU Econ) Basic Quantitative Method 2023.7.21. 27 / 44