Sampling Distributions and Point Estimation of Parameters: Engdat1

Sampling Distributions and Point
Estimation of Parameters
ENGDAT1
1
Point Estimation
• A point estimate is a reasonable value of a
population parameter.
• Data collected, X1, X2,…, Xn are random
variables.
• Functions of these random variables, x-bar
and s2, are also random variables called
statistics.
• Statistics have their unique distributions that
are called sampling distributions.
2
© John Wiley & Sons, Inc. Applied Statistics and Probability for Engineers, by Montgomery and Runger.
Point Estimator
A point estimate of some population parameter θ
!.
is a single numerical value Q
! is called the point estimator.
The statistic Q
As an example,suppose the random variable X is normally distributed with

an unknown mean µ. The sample mean is a point estimator of the unknown
population mean µ. That is, µ! = X . After the sample has been selected,
the numerical value x is the point estimate of µ.
Thus if x1 = 25, x2 = 30, x3 = 29, x4 = 31, the point estimate of µ is
25 + 30 + 29 + 31
x= = 28.75
4
3
Some Parameters & Their Statistics
Parameter Measure Statistic
μ Mean of a single population x-bar
σ2 Variance of a single population s2
σ Standard deviation of a single population s
p Proportion of a single population p -hat
μ1 - μ2 Difference in means of two populations x bar1 - x bar2
p1 - p2 Difference in proportions of two populations p hat1 - p hat2
• There could be choices for the point estimator of a parameter.

• To estimate the mean of a population, we could choose the:
– Sample mean.
– Sample median.
– Average of the largest & smallest observations of the sample.
• We need to develop criteria to compare estimates using statistical
properties.
4
Some Definitions
• The random variables X1, X2,…,Xn are a random
sample of size n if:
a) The Xi are independent random variables.
b) Every Xi has the same probability distribution.
• A statistic is any function of the observations
in a random sample.
• The probability distribution of a statistic is
called a sampling distribution.
5
Sampling Distribution of the Sample Mean
• A random sample of size n is taken from a normal
population with mean μ and variance σ2.
• The observations, X1, X2,…,Xn, are normally and
independently distributed.
• A linear function (X-bar) of normal and independent
random variables is itself normally distributed.
X 1 + X 2 + ... + X n
X = has a normal distribution
n
µ + µ + ... + µ
with mean µ X = =µ
n
s 2 + s 2 + ... + s 2
and variance s X =
2
n2
6
Sampling
Distributions
• The probability
distribution of a
statistic is called a
sampling distribution.
• The sampling
distribution of a Law of Large Numbers
statistic predicts the
behavior of the ¡ As the number of observation
drawn increases, the sample
statistic in the long statistic get closer and closer to
run. the population parameter
Central Limit Theorem
If X 1 , X 2 ,..., X n is a random sample of size n is
taken from a population (either finite or infinite)
with mean µ and finite variance s , and 2
if X is the sample mean,

then the limiting form of the distribution of
X -µ
Z= (7-1)
s n
as n ® ¥, is the standard normal distribution.
8
Example 7-1: Resistors
An electronics company
manufactures resistors having a
mean resistance of 100 ohms
and a standard deviation of 10
ohms. The distribution of
resistance is normal. What is
the probability that a random
sample of n = 25 resistors will Figure 7-2 Desired probability is
have an average resistance of shaded
less than 95 ohms?
9
Example 7-2: Central Limit Theorem
Suppose that a random variable X has a continuous
uniform distribution:
ì1 2, 4 £ x £ 6
f ( x) = í
î0, otherwise
Find the distribution of the sample mean of a random
sample of size n = 40.
Figure 7-3 Distributions of

X and X-bar
10
Two Populations
We have two independent normal populations. What is
the distribution of the difference of the sample means?
The sampling distribution of X 1 - X 2 is:
µ X - X = µ X - µ X = µ1 - µ 2
1 2 1 2
s 12 s 22
s X2 - X = s X2 - s X2 = +
1 2 1 2
n1 n2
The distribution of X 1 - X 2 is normal if:
(1) n1 and n2 are both greater than 30,
regardless of the distributions of X 1 and X 2 .
(2) n1 and n2 are less than 30,
while the distributions are somewhat normal.
11
Sampling Distribution of a Difference in Sample Means
• If we have two independent populations with means μ1 and
μ2, and variances σ12 and σ22,
• And if X-bar1 and X-bar2 are the sample means of two
independent random samples of sizes n1 and n2 from these
populations:
• Then the sampling distribution of:
Z=
( X 1 - X 2 ) - ( µ1 - µ2 )
(7-4)
s 2
s 2
1
+ 2
n1 n2
is approximately standard normal, if the conditions of the
central limit theorem apply.
• If the two populations are normal, then the sampling
distribution is exactly standard normal.
12
Example 7-3: Aircraft Engine Life
The effective life of a component
used in jet-turbine aircraft
engines is a normal-distributed
random variable with
parameters shown (old). The
engine manufacturer Figure 7-4 Sampling distribution of
introduces an improvement the sample mean difference.
into the manufacturing
process for this component Process
that changes the parameters Old (1) New (2) Diff (2-1)
as shown (new). x -bar = 5,000 5,050 50
Random samples are selected s= 40 30 50
from the “old” process and n= 16 25
“new” process as shown. Calculations
What is the probability the s / √n = 10 6 11.7
difference in the two sample z= -2.14
means is at least 25 hours? P(xbar2-xbar1 > 25) = P(Z > z) = 0.9840
= 1 - NORMSDIST(z)
13
General Concepts of Point Estimation
• We want point estimators that are:
– Are unbiased.
– Have a minimal variance.
• We use the standard error of the estimator to
calculate its mean square error.
14
Unbiased Estimators Defined
! is an unbiased estimator
The point estimator Q
for the parameter θ if:
E Q ( )
! =θ (7-5)
If the estimator is not unbiased, then the difference:
E Q! -θ
( )
(7-6)
!.
is called the bias of the estimator Q
!
The mean of the sampling distribution of Q
is equal to θ.
15
Choosing Among Unbiased Estimators
! 1 and Q
Suppose that Q ! 2 are unbiased estimators of θ.
The variance of Q! 1 is less than the variance of Q

! 2.
! 1 is preferable.
\ Q
Figure 7-5 The sampling distributions

of two unbiased estimators.
16
Minimum Variance Unbiased Estimators
• If we consider all unbiased estimators of θ, the
one with the smallest variance is called the
minimum variance unbiased estimator (MVUE).
• If X1, X2,…, Xn is a random sample of size n from a
normal distribution with mean μ and variance σ2,
then the sample X-bar is the MVUE for μ.
• The sample mean and a single observation are
unbiased estimators of μ. The variance of the:
– Sample mean is σ2/n
– Single observation is σ2
– Since σ2/n ≤ σ2, the sample mean is preferred.
17
Standard Error of an Estimator
! is its standard deviation, given by
The standard error of an estimator Q
! .
s Q! = V Q ( )
If the standard error involves unknown parameters that can be estimated,
substitution of these values into s Q!
produces an estimated standard error, denoted by s Q! .
!
Equivalent notation: s Q! = sQ! = se Q ( )
If the X i are ~N ( µ , s ) , then X is normally distributed,
s s
and s X = . If s is not known, then s X = .
n n
18
Example 7-5: Thermal Conductivity
• These observations are 10 xi
measurements of thermal 41.60
conductivity of Armco iron. 41.48
42.34
• Since σ is not known, we use s to 41.95
calculate the standard error. 41.86
42.18
• Since the standard error is 0.2% of 41.72
the mean, the mean estimate is fairly 42.26
41.81
precise. We can be very confident 42.04
that the true population mean is 41.924 = Mean
41.924 ± 2(0.0898). 0.284 = Std dev (s )
0.0898 = Std error
19
Mean Squared Error
!
The mean squared error of an estimator Q
of the parameter θ is defined as:
( ) ( )
2
! =E Q
MSE Q ! -θ (7-7)
( ) ( )
2 2
Can be rewritten as = E éQ!-E Q! ù + éθ - E Q
! ù
ë û ë û
= V Q + ( bias )
! 2
( )
Conclusion: The mean squared error (MSE) of the
estimator is equal to the variance of the
estimator plus the bias squared. It measures
both characteristics.
20
Relative Efficiency
• The MSE is an important criterion for
comparing two estimators.
Relative efficiency =
( ) MSE Q1
MSE ( Q ) 2
• If the relative efficiency is less than 1, we

conclude that the 1st estimator is superior to
the 2nd estimator.
21
Optimal Estimator
• A biased estimator can be
preferred to an unbiased
estimator if it has a
smaller MSE.
• Biased estimators are
occasionally used in linear
Figure 7-6 A biased estimator has a
regression. smaller variance than the unbiased
estimator.
• An estimator whose MSE
is smaller than that of any
other estimator is called
an optimal estimator.
22

Sampling Distributions and Point Estimation of Parameters: Engdat1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Sampling Distributions and Point Estimation of Parameters: Engdat1

Uploaded by

Copyright:

Available Formats

Sampling Distributions and Point

As an example,suppose the random variable X is normally distributed with

• There could be choices for the point estimator of a parameter.

if X is the sample mean,

Figure 7-3 Distributions of

The variance of Q! 1 is less than the variance of Q

Figure 7-5 The sampling distributions

• If the relative efficiency is less than 1, we

You might also like