The normal distribution

Semester 2021-2022

Course name Instructor’s name: Worksheet created by

Random signals and noise Dr. Montasir Qasymeh Name: Mohammad Tahmid ID: 1076344
Hassan
One of the most important continuous variable distributions is the normal distribution. It is a bell-shaped curve.
2
The probability density function is of the form 𝑒 −𝑥

It is important to find a function of this type that has a standard deviation of one and a mean of zero.
2
The function 𝑒 −𝑥 is even, so the average value would be a 0:
∞
−𝑥 2
1 −𝑥 2 ∞
∫ 𝑥𝑒 𝑑𝑥 = [− 𝑒 ] = (0 − 0) = 0
−∞ 2 −∞

The integral of any probability density function from negative infinity to positive infinity must equal one.
∞
2
𝑘 ∫ 𝑒 −𝑥 𝑑𝑥 = 1
−∞

Using polar coordinates,

∞ ∞
2 2
𝑘 ∫ 𝑒 −𝑥 𝑑𝑥 ∗ 𝑘 ∫ 𝑒 −𝑦 𝑑𝑦 = 12
−∞ −∞
∞ ∞
2 2
𝑘 ∫ ∫ 𝑒 −𝑦 𝑒 −𝑥 𝑑𝑥 𝑑𝑦 ∗ 𝑘 = 1
−∞ −∞
∞ ∞
2∫ 2 2
𝑘 ∫ 𝑒 −𝑦 𝑒 −𝑥 𝑑𝑥 𝑑𝑦 = 1
−∞ −∞

Using the substitution 𝑥 2 + 𝑦 2 = 𝑟 2

𝑥 = 𝑟 cos 𝜃

𝑦 = 𝑟 sin 𝜃
And 𝑑𝑥𝑑𝑦 = 𝑟𝑑𝑟𝑑𝜃
2𝜋 ∞
2
𝑘2 ∫ ∫ 𝑟𝑒 −𝑟 𝑑𝑟𝑑𝜃 = 1
0 0

1
𝑘 2 ∗ 2𝜋 ∗ =1
2
𝑘2 ∗ 𝜋 = 1
 1
𝑘=
√𝜋
The variance of this is:
∞ ∞
2 2
𝑘 ∫ 𝑥 2 𝑒 −𝑥 𝑑𝑥 ∗ 𝑘 ∫ 𝑦 2 𝑒 −𝑦 𝑑𝑦 = 12
−∞ −∞
∞ ∞
2 2
𝑘 ∫ ∫ 𝑒 −𝑦 𝑒 −𝑥 𝑥 2 𝑦 2 𝑑𝑥 𝑑𝑦 ∗ 𝑘 = 1
−∞ −∞
∞ ∞
2 2
𝑘 2 ∫ ∫ 𝑥 2 𝑦 2 𝑒 −𝑦 𝑒 −𝑥 𝑑𝑥 𝑑𝑦 = 1
−∞ −∞
∞ ∞
2 2
𝑘 2 ∫ ∫ 𝑥 2 𝑦 2 𝑒 −𝑦 𝑒 −𝑥 𝑑𝑥 𝑑𝑦 = 1
−∞ −∞

2𝜋 ∞ (sin 2𝜃)2 −𝑟 2
2 ∫ 𝑟4
𝑘 ∫ 𝑒 𝑟 𝑑𝑟 𝑑𝜃 = 1
0 0 4
2𝜋 ∞
1 − cos 4𝜃 −𝑟2
𝑘2 ∫ ∫ 𝑟4 𝑒 𝑟 𝑑𝑟 𝑑𝜃 = 1
0 0 8

1 2𝜋 ∞ 4 1 − cos 4𝜃 −𝑟2
∫ ∫ 𝑟 𝑒 𝑟 𝑑𝑟 𝑑𝜃 = 1
𝜋 0 0 8

2 1 2 2 1 2 2 1 2 2 2
∫ 𝑟 5 𝑒 −𝑟 𝑑𝑟 = − 𝑟 4 𝑒 −𝑟 + ∫ 2𝑟 3 𝑒 −𝑟 𝑑𝑟 = − 𝑟 4 𝑒 −𝑟 + ∫ 2𝑟 3 𝑒 −𝑟 𝑑𝑟 = − 𝑟 4 𝑒 −𝑟 − 𝑟 2 𝑒 −𝑟 − ∫ 2𝑟𝑒 −𝑟 𝑑𝑟
2 2 2
1 2 2 2
= − 𝑟 4 𝑒 −𝑟 − 𝑟 2 𝑒 −𝑟 − 𝑒 −𝑟
2
∞
1 4 −𝑟 2 2 −𝑟 2 −𝑟 2
[= − 𝑟 𝑒 −𝑟 𝑒 −𝑒 ] = 1
2 0
2𝜋
1 1 − cos 4𝜃 1
∗∫ 𝑑𝜃 ∗ 1 =
𝜋 0 8 4

The desired constant is

1
2
1
The variance of this distribution is 2, as the mean is 0.

We need a variance of 1.

It is known that this integral

∞
1 2
∫ 𝑒 −𝑥 𝑑𝑥 = 1
√𝜋 −∞

1
The variance is however.
2

Let a substitution be done.

1
𝑥= 𝑦
√2
 1
𝑑𝑥 = 𝑑𝑦
√2
∞ 𝑦2
1
∫ 𝑒 − 2 𝑑𝑦 = 1
√2𝜋 −∞
1
The integral is 1, the expectation is 0 and the variance is 2 ∗ 2 = 1.

This is the probability density function needed. It is for the standardized normally distributed variable, Z.

This is a probability density function that cannot be easily integrated. Hence, the cumulative density function is found
numerically for different values of Z and then tabulated in tables, for the standard normally distributed variable.

Remember, the cumulative density function tells us the probability of a variable taking a value less than a given number.

In addition, remember that the probability density function for the standard normal variable is symmetric about zero.

The symbol for the cumulative probability function is Φ(𝑧)

Therefore, for any positive number f,

𝑃(𝑧 < −𝑓) = 𝑃 (𝑧 > 𝑓) = 1 − Φ(𝑓)

By the same symmetry

𝑃(𝑧 > −𝑓) = 𝑃 (𝑧 < 𝑓) = Φ(𝑓)

To calculate 𝑃(𝑧 < |𝑓|), where

𝑃 (−𝑓 < 𝑧 < 𝑓) = 1 − 𝑃 (𝑧 < −𝑓) − 𝑃(𝑧 > 𝑓) = 1 − (1 − Φ(𝑓)) − (1 − Φ(𝑓)) = 2Φ(𝑓) − 1

Therefore, 𝑃(𝑧 > |𝑓|) = 1 − (2Φ(𝑓) − 1) = 2 − 2Φ(𝑓)

It is possible to go from a probability value to a z-value. It is due to the cumulative density function of the standard
normally distributed variable being one-to one as the probability density function is never 0. This allows for finding
Φ−1 (𝑝) for an y probability 𝑝.

Sometimes, the normal distribution in question is not with a mean of 0 or with a variance of 1. However, the tables are
for the standardized normal variable only. To be able to use the tables, it is important to standardize the given normal
variable, by subtracting the mean to center the distribution about zero and then dividing by the square root of the
variance to reduce the variance to 1.

In other words, if there is a normally distributed variable 𝑋 with a mean 𝜇 and a variance 𝜎 2 ,
𝑋−𝜇
𝑍=
𝜎
One important aspect is that the normal distribution can be used to approximate other distributions. If there is a
sufficiently large number of trials present, the resulting distribution eventually converges towards a normal distribution.

For a binomial distribution, the mean to be used is 𝑛𝑝 and the variance to be used is 𝑛𝑝(1 − 𝑝).

𝑛𝑝 and 𝑛(1 − 𝑝) must both be at least 5 for the approximation to work properly

For a Poisson distribution, the mean and variance are equal to the parameter 𝜆, which should be large.
To understand why this is the case, it is important to invoke the central limit theorem. This theorem states that the
average of the samples taken from a given distribution tends to a normal distribution as the number of samples rises to
infinity.

To explain this, it is important to explain something called a moment generating function. A moment generating function
is a function that can be used to determine the moments of a given function. The nth moment is the expected value of
the nth power of the random variable.

The moment generating function is

𝐸(𝑒 𝑡𝑋 ) = 𝑀𝑋 (𝑡)
The expectation of the exponential of the random variable X times a scalar t.

To find the nth moment, differentiate the moment generating function n times with respect to t and then evaluate the
nth derivative at t=0.

Now, if there is a random variable X and the normalized version is Y, then

𝑋 − 𝜇𝑋
𝑌=
𝜎𝑋
This means the mean of Y is 0 and the standard deviation and variance is 1.

The sum of n independent samples of Y can be written as

𝑋1 + 𝑋2 +. . +𝑋𝑛 − 𝑛𝜇𝑋 𝑌1 + 𝑌2 + ⋯ + 𝑌𝑛
𝑌𝑛 = =
𝜎𝑋 1
This sum has a mean of 0 and a variance of n

The normalized sum with a variance of 1 is

𝑌1 + 𝑌2 + ⋯ + 𝑌𝑛
√𝑛
The moment generating function is
𝑌 +𝑌 +⋯+𝑌𝑛
𝑡( 1 2 )
𝐸(𝑒 √𝑛 )

As each Y is identical and independent, this means

𝑛 𝑛
𝑌 +𝑌 +⋯+𝑌𝑛 𝑡𝑌
𝑡( 1 2 ) 𝑡
𝐸 (𝑒 √𝑛 ) = (𝐸 (𝑒 √𝑛 )) = (𝑀𝑌 ( ))
√𝑛

Now, as n reaches infinity

𝑡𝑌 𝑡𝑌
𝑡 ln(𝐸(𝑒 √𝑛 )) ln(𝐸(𝑒 √∞ ))
𝑛 ln(𝑀𝑌 ( )) ln(𝐸(𝑒 0 ))
𝑡 𝑡 √𝑛 ln(1)
lim (𝑀𝑌 ( )) ⇒ lim 𝑛 ln (𝑀𝑌 ( )) = lim 1 = lim 1 = 1 = 0
= 0
=
𝑛→∞ √𝑛 𝑛→∞ √𝑛 𝑛→∞ 𝑛→∞
𝑛 𝑛 ∞
0
0
(𝑖𝑛𝑑𝑒𝑡𝑒𝑟𝑚𝑖𝑛𝑎𝑡𝑒)

Differentiating with respect to n gives

 𝑡
𝑀𝑌′ (
)
√𝑛 ∗ − 1 ∗ 𝑡
𝑡 2 𝑛√𝑛 𝑡 𝑡
𝑀𝑌 ( ) 𝑡𝑀𝑌′ ( ) 𝑡𝑀𝑌′ ( )
lim
√𝑛 1
= lim ( ∗
1
√𝑛 𝑛) = lim ( ∗ √𝑛 ) = lim (1 ∗ 𝑡 ∗ (𝐸 (𝑌) = 0)) = 0
1 𝑡 √ 1 1
𝑛→∞ 𝑛→∞ 2 𝑛→∞ 2 𝑛→∞ 2 0
− 2 𝑀𝑌 ( ) 1∗ ∗1
𝑛 √ 𝑛 √ 𝑛 √ 𝑛
𝑡 −1 𝑡
𝑡 2 𝑀𝑌′′ ( )∗ 𝑡 2 (𝑀𝑌′′ ( ) = 𝐸(𝑌 2 ) = 1)
1 1
√𝑛 2𝑛√𝑛 ) = lim ( ∗ √𝑛 1
lim ( ∗ ) = 𝑡2
𝑛→∞ 2 −1 𝑛→∞ 2 1 2
1∗
2𝑛√𝑛
𝑡2
𝑒 2 is the MGF of the normal distribution.
∞ ∞ ∞ 2 ∞
1 𝑦2 1 𝑦 2 −2𝑡𝑦 1 𝑦 2 −2𝑡𝑦+𝑡 2 −𝑡 2 1 (𝑦−𝑡)2−𝑡 2
∫ 𝑒 𝑡𝑦 𝑒 − 2 𝑑𝑦 = ∫ 𝑒− 2 𝑑𝑦 = ∫ 𝑒− 2 𝑑𝑦 = ∫ 𝑒− 2 𝑑𝑦
√2𝜋 −∞ √2𝜋 −∞ √2𝜋 −∞ √2𝜋 −∞
𝑡2 1 ∞ (𝑦−𝑡)2 𝑡2
= 𝑒2 ∫ 𝑒 − 2 𝑑𝑦 = 𝑒 2
√2𝜋 −∞
So, the sum of a large number of independent identically distributed random variables tends towards a normal
distribution.

Reference

H. Pishro-Nik, Introduction to Probability. Statistics and Random Processes. .