Week 1

IE 212 STATISTICS
Week 4: Sampling Distributions

Dr. Başak GEVER

UNIVERSITY OF TURKISH AERONAUTICAL ASSOCIATION
IE 212 STATISTICS WEEK 1

SAMPLING DISTRIBUTIONS
❖ The field of statistical inference is basically concerned with
generalizations and predictions.
❖ We computed a statistic from a sample selected from the
population, and from this statistic we made various statements
concerning the values of population parameters that may or may not
be true.
❖ Since a statistic is a random variable that depends only on the
observed sample, it must have a probability distribution.

DR. BAŞAK GEVER 1

IE 212 STATISTICS WEEK 1

Definition: The probability distribution of a statistic is called a sampling

distribution.

❖ The sampling distribution of a statistic depends on the distribution

of the population, the size of the samples, and the method of
choosing the samples.
❖ We study several of the important sampling distributions of
frequently used statistics.

DR. BAŞAK GEVER 2

IE 212 STATISTICS WEEK 1

1. Normal Distribution: Suppose that 𝑋 ∈ 𝑁𝑜𝑟𝑚𝑎𝑙(𝜇, 𝜎 2 ). Then pdf

and cdf of 𝑋 are as follows:

1 (𝑥 − 𝜇)2
𝑓𝑋 (𝑥) = exp {− 2
} , 𝑥 ∈ 𝑅;
√2𝜋𝜎 2𝜎
𝑥
𝐹𝑋 (𝑥) ≡ ∫ 𝑓𝑋 (𝑢)𝑑𝑢 , 𝑥 ∈ 𝑅.
−∞

➢ The first important sampling distribution to be considered is that of

the mean 𝑋̅.

DR. BAŞAK GEVER 3

IE 212 STATISTICS WEEK 1

Theorem: If 𝑋1 , 𝑋2 , … , 𝑋𝑛 are independent random variables having

normal distributions with means 𝜇1 , 𝜇2 , … , 𝜇𝑛 and finite variances

𝜎12 , 𝜎22 , … , 𝜎𝑛2 , respectively, then the random variable

𝑌 = 𝑎1 𝑋1 + 𝑎2 𝑋2 + ⋯ + 𝑎𝑛 𝑋𝑛

has a normal distribution with mean 𝜇𝑌 = 𝑎1 𝜇1 + 𝑎2 𝜇2 + ⋯ + 𝑎𝑛 𝜇𝑛 =

∑𝑛𝑖=1 𝑎𝑖 𝜇𝑖 and variance 𝜎𝑌2 = 𝑎12 𝜎12 + 𝑎22 𝜎22 + ⋯ + 𝑎𝑛2 𝜎𝑛2 = ∑𝑛𝑖=1 𝑎𝑖 𝜎𝑖2 .□

DR. BAŞAK GEVER 4

IE 212 STATISTICS WEEK 1

Theorem (Central Limit Theorem): If 𝑋̅ is the mean of a random sample

of size 𝑛 taken from a population with mean 𝜇 and finite variance 𝜎 2 ,
then limiting form of the distribution of

𝑋̅ − 𝜇
𝑍= 𝜎
√𝑛
as 𝑛 → ∞, is the standard normal distribution with parameters 0 and 1,
respectively.□

➢ Let us determine the distribution of the sample mean 𝑋̅:

DR. BAŞAK GEVER 5

IE 212 STATISTICS WEEK 1

Theorem: Suppose that a random sample of 𝑛 observations is taken

from a normal population with mean 𝜇 and variance 𝜎 2 . Each

observation 𝑋𝑖 , 𝑖 = 1, 2, . . . , 𝑛, of the random sample will then have the

same normal distribution as the population being sampled. Then the

distribution of 𝑋̅ is Normal distribution with parameters

1
𝜇𝑋̅ = 𝐸 (𝑋̅) = 𝐸 ( ∑𝑛𝑖=1 𝑋𝑖 ) = 𝜇 and
𝑛

1 𝜎2
𝜎𝑋2̅ = 𝑉𝑎𝑟(𝑋̅) = 𝑉𝑎𝑟 ( ∑𝑛𝑖=1 𝑋𝑖 ) = .□
𝑛 𝑛
DR. BAŞAK GEVER 6
IE 212 STATISTICS WEEK 1

❖ If we are sampling from a population with unknown distribution,

either finite or infinite, the sampling distribution of 𝑋̅ will still be
approximately normal with mean 𝜇 and variance 𝜎 2 /𝑛, provided that
the sample size is large.
❖ This amazing result is an immediate consequence of the Central
Limit Theorem.
❖ The normal approximation for 𝑋̅ will generally be good if 𝑛 ≥ 30,
provided the population distribution is not terribly skewed.

DR. BAŞAK GEVER 7

IE 212 STATISTICS WEEK 1

❖ If 𝑛 < 30, the approximation is good only if the population is not

too different from a normal distribution and if the population is
known to be normal, the sampling distribution of 𝑋̅ will follow a
normal distribution exactly, no matter how small the size of the
samples.
❖ The sample size 𝑛 = 30 is a guideline to use for the Central Limit
Theorem.
❖ However, as the statement of the theorem implies, the
presumption of normality on the distribution of 𝑋̅ becomes more
accurate as 𝑛 grows larger.
DR. BAŞAK GEVER 8
IE 212 STATISTICS WEEK 1

Example: An electrical firm manufactures light bulbs that have a length

of life that is approximately normally distributed, with mean equal to
800 hours and a standard deviation of 40 hours. Find the probability
that a random sample of 16 bulbs will have an average life of less than
775 hours.

DR. BAŞAK GEVER 9

IE 212 STATISTICS WEEK 1

Solution: The sampling distribution of 𝑋̅ will be approximately normal,

40
with 𝜇𝑋̅ = 800 and 𝜎𝑋̅ = = 10. The desired probability is given by
√16
the area of the shaded region in the following figure.

775−800
Corresponding to 𝑥̅ = 775, we find that 𝑧 = = −2.5
10
−2.5
and therefore, 𝑃(𝑋̅ < 775) = 𝑃(𝑍 < −2.5) = ∫−∞ 𝜑𝑍 (𝑢)𝑑𝑢
−2.5 1 −𝑢 2 /2 1 𝑥
= ∫−∞ √2𝜋 𝑒 𝑑𝑢 = {erf ( ) + 1} = 0.0062. (Go to Z Table)
2 √2

DR. BAŞAK GEVER 10

IE 212 STATISTICS WEEK 1

❖ If a random sample of size 𝑛 drawn from a normal population with

mean 𝜇 and variance 𝜎 2 , and the sample variance is computed, we

obtain a value of the statistics:

𝑛
1
𝑆2 = ∑(𝑋𝑖 − 𝑋̅)2 .
𝑛−1
𝑖=1

❖ If the variability in sample is to be studied, clearly the sampling

distribution of 𝑆 2 will be used in learning about the parametric

counterpart, the population variance 𝜎 2 .

DR. BAŞAK GEVER 11
IE 212 STATISTICS WEEK 1

Theorem: If 𝑋1 , 𝑋2 , … , 𝑋𝑛 are independent random variables having

𝐸 (𝑋𝑖 ) = 𝜇, 𝑖 = 1,2, … , 𝑛 and 𝑉𝑎𝑟(𝑋𝑖 ) = 𝜎𝑋2 < ∞, 𝑖 = 1,2, … , 𝑛, then
𝜎𝑋2
𝑉𝑎𝑟(𝑋̅) = ; 𝐸 (𝑆 2 ) = 𝜎𝑋2 .
𝑛
2
1 𝑛 1 𝑛 1 𝜎𝑋
Proof: 𝑉𝑎𝑟(𝑋̅) = 𝑉𝑎𝑟 ( ∑𝑖=1 𝑋𝑖 ) = 2
∑ 𝑖=1 𝑉𝑎𝑟(𝑋𝑖 ) = 𝑛𝜎𝑋
2
= .
𝑛 𝑛 𝑛2 𝑛
𝑛 𝑛
1 1
𝐸 (𝑆 2 ) = 𝐸{ ̅ 2
∑(𝑋𝑖 − 𝑋) } = 𝐸 {∑(𝑋𝑖 − 𝑋̅)2 } ⇒
𝑛−1 𝑛−1
𝑖=1 𝑖=1

➢ Include 𝑋𝑖 − 𝑋̅ = 𝑋𝑖 − 𝜇 + 𝜇 − 𝑋̅:
𝑛
1
⇒ 𝐸 {∑[(𝑋𝑖 − 𝜇) + (𝜇 − 𝑋̅)]2 }
𝑛−1
𝑖=1
DR. BAŞAK GEVER 12
IE 212 STATISTICS WEEK 1

𝑛
1
= 𝐸 {∑[(𝑋𝑖 − 𝜇)2 + (𝜇 − 𝑋̅)2 + 2(𝑋𝑖 − 𝜇)(𝜇 − 𝑋̅)]}
𝑛−1
𝑖=1
𝑛 𝑛 𝑛
1
= 𝐸 {∑(𝑋𝑖 − 𝜇)2 + ∑(𝜇 − 𝑋̅)2 + ∑ 2(𝑋𝑖 − 𝜇)(𝜇 − 𝑋̅)}
𝑛−1
𝑖=1 𝑖=1 𝑖=1
𝑛 𝑛
1
= 𝐸 {∑(𝑋𝑖 − 𝜇)2 + 𝑛(𝜇 − 𝑋̅)2 + 2(𝜇 − 𝑋̅) ∑(𝑋𝑖 − 𝜇)}
𝑛−1
𝑖=1 𝑖=1
𝑛
1
= 𝐸 {∑(𝑋𝑖 − 𝜇)2 + 𝑛(𝑋̅ − 𝜇)2 − 2𝑛(𝑋̅ − 𝜇)2 }
𝑛−1
𝑖=1

DR. BAŞAK GEVER 13

IE 212 STATISTICS WEEK 1

𝑛
1
= {∑ 𝐸 (𝑋𝑖 − 𝜇)2 − 𝑛𝐸 (𝑋̅ − 𝜇)2 }
𝑛−1
𝑖=1
𝑛 2
1 1 𝑛𝜎𝑋
= {∑ 𝑉𝑎𝑟(𝑋𝑖 ) − 𝑛𝑉𝑎𝑟(𝑋̅)} = {𝑛𝜎𝑋2 − }
𝑛−1 𝑛−1 𝑛
𝑖=1
1
= 𝜎𝑋2 (𝑛 − 1) = 𝜎𝑋2 . □
𝑛−1

DR. BAŞAK GEVER 14

IE 212 STATISTICS WEEK 1

2. Chi-Square Distribution: Suppose that 𝑋 ∈ 𝐶ℎ𝑖 − 𝑆𝑞𝑢𝑎𝑟𝑒𝑑 with

degress of freedom 𝜈. Then pdf of 𝑋 is as follows:

1 𝑛
𝑓𝑋 (𝑥) = 𝑛 𝑥 2 −1 𝑒 −𝑥/2 , 𝑥 > 0. □
𝑛
2 Γ( )
2
2
❖ If you recall, Chi-Squared distribution is a speacial case of Gamma
distribution.
❖ When the parameters of Gamma distribution 𝛼 = 𝑛/2 and 𝜆 = 1/
2, then we obtain Chi-Squared distribution with degrees of freedom
𝜈 = 𝑛.

DR. BAŞAK GEVER 15

IE 212 STATISTICS WEEK 1

❖ Another relationship can be constructed between chi-squared and

standard normal distribution as follows:

Theorem: If 𝑍𝑖 , 𝑖 = 1,2, . . , 𝑛 are independent random variables having

standard normal distributions with mean 0 and variance 1 then, the
distribution of the following random variable is chi – squared
distribution with degrees of freedom 𝜈 = 𝑛:
𝑛

𝑈 = ∑ 𝑍𝑖2 .
𝑖=1

DR. BAŞAK GEVER 16

IE 212 STATISTICS WEEK 1

Theorem: If 𝑆 2 is the variance of a random sample of size n taken from

a normal population having the variance 𝜎 2 , then the statistic
𝑛
( 𝑛 − 1 )2
𝑆 ( 𝑋𝑖 − 𝑋̅ ) 2
𝜒𝜈2 = = ∑
𝜎2 𝜎2
𝑖=1

has a chi-squared distribution with 𝜈 = 𝑛 − 1 degrees of freedom.□

❖ The values of the random variable 𝜒𝜈2 are calculated from each
sample by the formula
( 𝑛 − 1 ) 𝑠 2
𝜒𝜈2 = 2
.
𝜎
❖ The probability that a random sample produces a 𝜒𝜈2 value greater
than some specified value is equal to the area under the curve to the
right of this value.
DR. BAŞAK GEVER 17
IE 212 STATISTICS WEEK 1

2 represent the 𝜒 2 value above which we

❖ It is customary to let 𝜒𝛼,𝜈 𝜈
find an area of 𝛼:
2
𝜒𝛼,𝜈 ≑ 𝜒 2 : 𝑃{Χ > 𝜒 2 : 𝜈 = 𝑛 − 1} = 𝛼

Example: 𝜒 2 value with 𝜈 = 7 d.f. leaving an area 0.05: 𝑃{𝑋 > 𝜒 2 : 𝜈 =

2
7} = 0.05, 𝜒𝜈=7;𝛼=0.05 = 14.067. Owing to lack of symmetry, we must
2
also use the tables to find 𝜒𝜈=7;𝛼=0.95 = 2.167.
DR. BAŞAK GEVER 18
IE 212 STATISTICS WEEK 1

3. T – Student Distribution

❖ If the sample size is large enough, say n ≥ 30, the distribution of T

does not differ considerably from the standard normal.
❖ However, for n < 30, it is useful to deal with the exact distribution of
𝑇.

DR. BAŞAK GEVER 19

IE 212 STATISTICS WEEK 1

Theorem: Let 𝑍 be a standard normal random variable and 𝑉 a chi-

squared random variable with 𝜈 degrees of freedom. If 𝑍 and 𝑉 are
independent, then the distribution of the random variable 𝑇, where
𝑍
𝑇= ,is given by the density function
√𝑉/𝜈

(𝜈 + 1 ) (𝜈+1)
Γ[ ] 𝑡2
−
2
2
ℎ(𝑡) = 𝜈 (1 + ) , 𝑡 ∈ 𝑅.
Γ ( ) √𝜋𝜈 𝜈
2
This is known as the t-distribution with 𝜈 degrees of freedom.□
➢ From the foregoing and the theorem above we have the following
corollary.

DR. BAŞAK GEVER 20

IE 212 STATISTICS WEEK 1

Corollary: Let 𝑋1 , 𝑋2 , … , 𝑋𝑛 be independent random variables that are

all normal with mean 𝜇 and standard deviation 𝜎. Let
𝑛 𝑛
1 1
𝑋̅ = ∑ 𝑋𝑖 ; 𝑆 =
2 ∑(𝑋𝑖 − 𝑋̅)2 .
𝑛 𝑛−1
𝑖=1 𝑖=1
𝑋̅−𝜇
Then the random variable 𝑇 = has a t-distribution with 𝜈 = 𝑛 − 1
𝑆/√𝑛
degrees of freedom.

DR. BAŞAK GEVER 21

IE 212 STATISTICS WEEK 1

❖ The distribution of 𝑇 is similar to the distribution of 𝑍 in that they

both are symmetric about a mean of zero.
❖ Both distributions are bell shaped, but the t-distribution is more
variable, owing to the fact that the T-values depend on the
fluctuations of two quantities, 𝑋̅ and 𝑆 2 , whereas the 𝑍-values
depend only on the changes in 𝑋̅ from sample to sample.
❖ The distribution of 𝑇 differs from that of 𝑍 in that the variance of 𝑇
depends on the sample size 𝑛 and is always greater than 1.
❖ Only when the sample size 𝑛 → ∞ will the two distributions
become the same.
DR. BAŞAK GEVER 22
IE 212 STATISTICS WEEK 1

❖ It is customary to let 𝑡𝛼 represent the t-value above which we find

an area equal to 𝛼:
𝑡𝜈;𝛼 ≑ 𝑡: 𝑃{𝑇 > 𝑡: 𝜈 } = 𝛼
❖ Hence the t-value with 10 degrees of freedom leaving an area of
0.025 to the right is 𝑡 = 2.228.

❖ Since t-distribution is symmetric about a mean of zero, we have

𝑡1−𝛼 = −𝑡𝛼 ; that is, t-value leaving an area of 1 − 𝛼 to the right and
therefore an area of 𝛼 to the left is equal to the negative t-value that
leaves an area of 𝛼 in the right tail of the distribution.
DR. BAŞAK GEVER 23
IE 212 STATISTICS WEEK 1

❖ That is, 𝑡0.95 = −𝑡0.05 ; 𝑡0.99 = −𝑡0.01 and so forth.

Example: The t-distribution with 𝜈 = 14 degrees of freedom that
leaves an area of 0.025 to the left, and therefore an area of 0.975 to
the right, is 𝑃{𝑇 > 𝑡: 𝜈 = 14; 𝛼 = 0.975} = 𝑡𝜈=14;𝛼=0.975 =
−𝑡𝜈=14;1−𝛼=0.025 = −2.145.

Example: Find k such that 𝑃{𝑘 < 𝑇 < −1.761} = 0.045 for a random
𝑋̅−𝜇
sample of size 15 selected from a normal distribution and .
𝑆/√𝑛

DR. BAŞAK GEVER 24

IE 212 STATISTICS WEEK 1

Solution:
𝑃{𝑘 < 𝑇 < −1.761} = 0.045 ⇒ 𝑃{𝑇 < −1.761} − 𝑃{𝑇 < 𝑘} = 0.045
Because of the symmetry, 𝑃{𝑇 < −1.761} = 𝑃{𝑇 > 1.761} can be
written. From t-table
𝑃{𝑇 > 1.761} = 0.05.
Then 𝑃{𝑇 < −1.761} − 𝑃{𝑇 < 𝑘} = 0.045 ⇒ 0.05 − 𝑃{𝑇 < 𝑘} =
0.045 ⇒
𝑃{𝑇 < 𝑘} = 0.005 = 𝑃{𝑇 > −𝑘} = 0.005 ⇒ 𝑘 = −𝑡14;0.005 = −2.977

DR. BAŞAK GEVER 25

IE 212 STATISTICS WEEK 1

DR. BAŞAK GEVER 26

IE 212 STATISTICS WEEK 1

4. F Distribution
❖ We have motivated the t-distribution in part by its application to
problems in which there is comparative sampling (i.e., a comparison
between two sample means).
❖ While it is of interest to let sample information shed light on two
population means, it is often the case that a comparison of variability
is equally important, if not more so.
❖ The F-distribution finds enormous application in comparing sample
variances.
❖ Applications of the F-distribution are found in problems involving
two or more samples.

DR. BAŞAK GEVER 27

IE 212 STATISTICS WEEK 1

Theorem: Let U and V be two independent random variables having

chi-squared distributions with 𝜈1 and 𝜈2 degrees of freedom,
𝑈/𝜈1
respectively. Then the distribution of the random variable 𝐹 = is
𝑉/𝜈2
given by the density function
𝜈1
(𝜈1 + 𝜈2 ) 𝜈1 2
Γ[ ]( ) 𝑓 (𝜈1 /2)−1
2 𝜈2
ℎ(𝑓) = 𝜈1 𝜈2 (𝜈1 +𝜈2 )/2
,𝑓 > 0
Γ( )Γ( ) 𝜈1 𝑓
2 2 (1 + 𝜈 )
2

This is known as the F-distribution with 𝜈1 and 𝜈2 degrees of freedom.

DR. BAŞAK GEVER 28

IE 212 STATISTICS WEEK 1

DR. BAŞAK GEVER 29