7/12/2012
Unit 6 Outline: Introduction to Inference
Unit 6: Introduction to Inference
Chapter 6 in IPS
Confidence Intervals - General Idea
We want to make a statement (inference) about a population parameter
(e.g. μ or p; unknown value) using information from observed sample
data (statistic; an estimate such as x or pˆ )
The general form of a CI is: Estimate + margin of error
Estimate + (z-value)×(SD of estimate)
Usually written as: (Lower confidence limit, Upper confidence limit)
(Est. – z*×(SD of est.), Est. + z*×(SD of est.))
Example: σ is known to be 5, and we sample x to be 18:
18 + 1.96×(5) = (8.2, 27.8)
3 4
• Confidence Intervals
• Hypothesis Tests
• Power and Sample Size
Example: Confidence interval for population
mean rent in Cambridge
• Suppose we wish to make a statement about rent
of one-bedroom apartments in Cambridge
• We take a random sample of properties; perhaps
from the property listings at Cambridge Property
Tax Office, perhaps from realty listings (not
ideal)
– keeping in mind good sampling principles of Stat
S100 as we collect the apartment rents in the sample
1
 7/12/2012

Regardless of the population distribution from Logic behind confidence intervals

which the individual random variables are drawn, Population of • Draw a sample of 100 rents X1,…X100 from 1-BR apartments, and
the sample mean is approximately normally rents: mean = μ, calculate the sample mean (say $1250).
distributed with mean μ and standard dev. (σ/√n) sd = σ … a bit
• Assume the standard deviation across individual apartment rents in the
right-skewed
Approximation improves with increasing n. population is σ = 300. What is the standard deviation of the sample
mean?  300 300
Random Observed X     30
Variable Values n 100 10
Apt (rents) Pop. Mean Pop. SD (rents)
• Thus with 95% probability, x will lie within plus or minus 1.96(σ/√n) =
1 X1 µ σ x1 = $920 1.96(30) = 58.8 dollars of the true population mean price, μ.
2 X2 µ σ x2 = $800 • So we say with 95% confidence, the unknown mean, μ, will lie within
plus or minus 58.8 dollars of the sample mean ( x )
… … … … …
• We express this by saying that the 95% confidence interval for the true
n Xn µ σ xn = $1500 population mean rent (1-BR Cambridge Apts) is:
Sample 
Mean X µ x (1250 – 58.8, 1250 + 58.8) or ($1,191.20, $1,308.80)
n
• So why a confidence interval and not a probability interval? Because
   only random things have probability. And even though we are
X has a N   ,  distribution estimating the unknown μ, it is a fixed number. What was random is
 n the sampling procedure we used to calculate x. 6

Varying the level of confidence

More generally, the 95% Suppose we know , the population standard deviation, but not the mean μ,
confidence interval for the and have drawn a random sample of size n from the population to estimate μ.
population mean is: The 95% confidence interval is:

       
 x  1.96 , x  1.96   x  1.96 , x  1.96 
 n n  n n
Can we be more confident? Suppose z* and C are related as in Figure 6.4
IPS Figure 6.3 shows the from IPS. We can say that the sample mean will be within the population
confidence intervals based on mean, plus or minus z*σ/√n with 100*C% confidence.
25 random samples. Note
that all but 1 contain the true A confidence interval for the mean with
mean μ confidence level C will have the form:

   
 x  z* , x  z* 
 n n
7 8

2
 7/12/2012

Alternative common notation

A few values of z* and C
In many texts, the region of area C is instead labeled with area
1-. The two regions in the tail then each have area (/2). The
• These are the most common confidence z point to the right (labeled z* below) is then denoted by z(1- /2)
coefficients and z* values. area 1-
The formula for the confidence
interval becomes
z* 1.645 1.960 2.576
   
x  z , x  z  
  
 1 

n  1  n 
C 90% 95% 99%   2  2

Some alternative notation on the next slide

In this case, 1- is the confidence
coefficient, and  is the error rate.
area /2
9 10

Looking Ahead: Confidence Intervals for

Controlling the margin of error a Population Mean, unknown SD
The margin of error m in a confidence z * • So what are we doing with these confidence interval
interval for the mean is In the rents m calculations?
example, m = 58.8 n
– We are trying to determine where the true unknown mean, μ,
is based on a sample mean, x
If we want to choose a sample size n – But this calculation (so far) assumes we know the true population
 z * 
2
standard deviation: σ
to get a given margin of error, we n 
solve for n  m  – This doesn’t often happen in real life. If we are trying to estimate
μ, we will also probably have to estimate σ .
– What’s our sample-based estimate of σ ?
How many apartments should 2
 1.96 * 300  – s
have been sampled to have a n   864.36  865
margin of error of  $20?  20  – This throws off everything. The calculation is no longer based on
a normal distribution, but a t-distribution. More on this after the
midterm…

11 12

3
 7/12/2012

Statistical Decision Making Testing a hypothesis about

a population mean
(aka Hypothesis Testing)
1. Formulate a Null Hypothesis and an Alternative Hypothesis
• The null hypothesis (H0) assumes a distribution for the population
• A discussion of this material can be found in IPS, 6.2
that reflects no change from the past or is nothing interesting going
• Also called Significance Testing or Testing Statistical Hypotheses
on. If the null hypothesis is true, any discrepancy between the
• We look at it from two aspects: observed data and the hypothesized distribution is due only to
– Some general principles (motivated by examples) chance variation (Cambridge 1-BR Rent H0: μ = 1200)
– Specific forms of tests in commonly arising situations • The alternative hypothesis (HA) states that there is a real difference
(comparing two groups, cross-classified data, regression
between the distribution of the observed data and the null-
considered more formally)
hypothesized distribution. An observed discrepancy between the
• Today’s Example: Cambridge officials claim that rents have not
observed data and the null-hypothesis distribution is not due to
changed since the year 2000 when the true population average rent
was reported to be $1200.
chance variation. (Cambridge 1-BR Rent HA: μ ≠ 1200)
– Does the data support the claim? • We set things up this way because it is easier to disprove
something than prove it (we are usually hoping to disprove H0)

13 14

Testing a hypothesis…(cont.)
Testing a hypothesis…(cont.)
2. Calculate the value of the test statistic on which the test will be
based.
• The test statistic measures the difference between the observed 3. Find the probability of getting this test statistic or a more
data and what is expected given the null hypothesis is true. extreme one if the null hypothesis were true.
• This is called the p-value (stands for probability-value)
• The test statistic answers the question, “How many standard
deviations from the hypothesized value is the observed sample • The smaller the p-value, the stronger the evidence against
the null hypothesis.
value?”
• Almost always of the form
4. Come to a conclusion about your hypotheses.
(observed statistic) - (its expected value)
• If the p-value is as small or smaller than the pre-specified
(standard deviation of statistic) level of the test or alpha (), usually 0.05, we say the result
• When will this value be large (in magnitude)? When will it be is statistically significant at level .
close to zero?

15 16

4
 7/12/2012

Testing a hypothesis about  Mechanics of testing

(1-sided vs. 2-sided alternatives)
(when  is known)
σ known
• Collect a simple random sample from a population with Reject H0: µ = µ0 in favor of the one-sided
– Unknown mean  alternative HA: µ > µ0 whenever
– known standard deviation  (rarely happens, but good place to start) X  0
Z  z*
• If the population is normal or the sample size is large enough then 
– The sampling distribution of the sample mean ( x ) is: n
Reject H0: µ = µ0 in favor of the one-sided
• Approximately Normal
 x alternative HA: µ < µ0 whenever
• With mean 
n  n X  0
• Standard deviation Z  z*

n
This is equivalent to saying that the standardized sample mean has a sampling Reject H0: µ = µ0 in favor of the two-sided
distribution that is approximately standard normal. This will be the test alternative HA: µ ≠ µ0 whenever
statistic. It will be computed under the assumption that the null hypothesis is
true (so  is the null value). The p-value will be calculated under this null X  0
Z  z*
sampling distribution. 
n
17

Connection between Hypothesis Tests Cambridge 1-BR Rents Example as

and Confidence Intervals
• There is a close connection between confidence intervals and
two-sided tests of hypotheses
– A level  two-sided significance test rejects a hypothesis
H0:  = 0 exactly when the value 0 falls outside a level
(1 - ) confidence interval for 
– Rationale (use .95 (95%) confidence level,  = 0.05 level
test to be concrete): Suppose a value 0 falls outside a
95% confidence level. Then we are 95% certain that 0 is
not consistent with the data. This is equivalent to saying
that the data are not consistent with the hypothesis
H0:  = 0 at the 5% significance level.
– Note:  and confidence level should add to one
a Hypothesis Test
The true population average rent in Cambridge in 2000
was $1200. Let’s use the data that we collected to test
whether there is evidence of a change in the average rent
since 2000.
Follow these 4 easy steps:
1. Set-up your hypotheses (and α-level)
2. Gather your data and calculate the test-statistic
3. Calculate the p-value (based on the appropriate sampling
distribution)
4. Come to a conclusion about your hypotheses

19 20

5
 7/12/2012

Cambridge Rent Data Solution…

15
As a 2-sided test: As a 1-sided test:
1. H 0 :   1200 1. H 0 :   1200
H A :   1200 H A :   1200
  0.05

10
  0.05
Percent
2. 2.
x   1250  1200 x   1250  1200
z   1.670 z   1.670
5  / n 300 / 100  / n 300 / 100

3. p  value  P (| z | 1.67) 3. p  value  P ( z  1.67)

 2 P ( z  1.67)  2(1  0.9525)  1  0.9525  0.0475
0

500 1000 1500 2000 2500  2(0.0475)  0.0950

rent
4. We cannot reject the null hypothesis. 4. We can reject the null hypothesis. It
. summarize rent There is not enough evidence to suggest that appears that the average rent for 1-BR
1-BR rents have changed since 2000. aptartments have increased since 2000.
Variable | Obs Mean Std.Dev. Min Max
----------+----------------------------------------- Most of this can be done in Stata…
rent | 100 1250 316.522 730 2400
21 22

Error rates in hypothesis testing

Lecture Outline
• In the Cambridge Rent example, the null hypothesis to be tested
• Confidence Intervals was H0:  = $1200, with the two-sided alternative HA:  ≠ $1200.
• Think of this as strictly a decision problem with only two
possibilities for the decision maker, H0 or HA. Because of the lack
of complete predictability (i.e., the presence of randomness in the
• Hypothesis Tests sample), all four of the following branches are possible (before
actually doing the study):

Conclude H0 Conclude H0
• Power and Sample Size H0 true HA true

Conclude HA Conclude HA

Correct Decision Error

23 24

6
 7/12/2012

Error rates in hypothesis testing….

Terminology on types of errors
• P(type I error) = P(conclude HA when H0 is true) is labeled
…this is what we set to 0.05, typically
• We have been using this implicitly already.
– Suppose we reject a null hypothesis whenever the p-value is
less than 0.05.
– The p-value is calculated assuming the null hypothesis is
true, so if the null hypothesis really is true, we will reject it
less than 5% of the time.
• The approach we use controls P(type I error), but we have
been silent about the probability of a type II error. More on
that later.

25 26

Error rates in hypothesis testing, Statistical Power

Statistical Power • Picture to the left is from IPS, ex.
6.17 (don’t worry about #’s)
• P(type II error) = P(Conclude H0 when HA is true) is • Statistical power is something that
typically labeled  is calculated before gathering data.
–Analogous to calculating sample
• In decision theoretic approach to hypothesis testing, size to obtain desired margin of
– the type I error rate  is fixed (usually 0.05) error in confidence interval.
– 1 -  = P (Conclude HA | HA is true)
= P(correct decision when HA is true) 3 things to think about:
• What happens to power as the mean
1 -  is called the statistical power of the test. Computing
under the alternative is further from
this probability can be subtle, and depends closely on the null hypothesis?
particular problem • What happens to power as standard
deviation increases?
• What happens to power as sample
Note: Power is always between 0 and 1…it’s a probability size increases?

27 28

7
 7/12/2012

Has the cost of living changed in Maine? Before data are collected
• The US Department of Housing and Urban Development publishes a • Suppose we `know’ (perhaps from previous studies) that the
table of Fair Market Rents (FMR) by state to set amounts paid in standard deviation for the rents of all two-bedroom apartments in
voucher programs, Section 8 housing, and other programs Maine is σ = $72
• The data used here oversimplifies the technical details in the • Suppose we wish to design our study so that the margin of error
definition of FMR, but is a good illustration. in a 95% confidence interval for the mean is $25, that is, the
confidence interval will be (approximately) of the form: sample
• In 1997, FMR for two-bedroom apartment in Maine was $590. mean  $25
• Suppose we wish to do a study to examine how rents in Maine – How large should our sample of rents be?
compare to FMR. We’ll collect a random sample of rents of two- • Recall the formula for margin of error (m) in a confidence
bedroom apartments. interval for a mean:   
m  z*  
• More specifically, we would like to `decide’ (in 1997), based on data,  n
whether or not rents in Maine are different than the fair market rent • From this we can get * 2
 ( z )( )   1.96(72) 
2

• We’ll eventually use a data set that is a random sample of rents for 32 (just solve for n): n        31.8  32
two-bedroom apartments in Maine in 1997  m   25 
• Start with some pre-data calculations… • So what is the Power for this test?

29 30

Calculating Power
• When calculating power, it’s a 2-step Process
– First: determine the rejection region assuming the Null
hypothesis is true (what values of x are needed to reject the
test).
– Second: calculate the probability of finding a sample statistic
(in this case, the sample mean) that falls in the rejection
region if the alternative hypothesis is actually true
• What are our hypothesis here?
– H0: μ = 590
– HA: μ ≠ 590
• Let’s do the calculation…
Calculating Power
• First: determine the rejection region assuming the Null
hypothesis is true (always helps to draw the picture):
– We will reject when |z-stat| > 1.96 (far out in tails)
– What does this mean in terms of the sample mean?
• x > μ + (1.96)*(σ/√n) = 590 + 1.96*72/√32 = 615
• x < μ - (1.96)*(σ√n) = 590 - 1.96*72/√32 = 565
• Second: calculate the probability of finding a sample mean
that falls in the rejection region if HA is actually true…so we
have to pick a specific μ within HA…let’s use μ = 625 here:
P ( x  615 |  A  625)  P ( x  565 |  A  625)  P ( x  615 |  A  625)  0
 x   A 615  625 
 P    P ( z  0.79)  0.7840
 / n 72 / 32 

• A picture can be worth a thousand words…

31 32

8
 7/12/2012

Maine rents
• Next step: take a sample of 32 rents and see what happens
• Summary of data below…apparently bad luck on the standard
deviation…we will return to this

. summarize rent, detail

rent
-------------------------------------------------------------
Percentiles Smallest
1% 250 250
5% 510 510
10% 530 510 Obs 32
25% 550 530 Sum of Wgt. 32

50% 590 Mean 603.125

Largest Std. Dev. 97.69728
75% 655 710
90% 710 710 Variance 9544.758
95% 770 770 Skewness -1.059636
99% 800 800 Kurtosis 6.791641
33 33 34

Maine rents… From the testing perspective

• We can go ahead with the confidence interval, using the normal distribution
the multiplier (z*) in the formula:
• Suppose policy maker in Maine wants to use these data to
    72  decide if the average rent in Maine are different than the
x  z *   603.13  1.96   (578.18, 628.08) FMR. If so, Maine will implement an expensive rent
 n  32 
subsidy program.
• An apparent inconsistency: we used a putative σ = 72 to choose a sample • Wishes to test the null hypothesis H0: mean rent = 0 =
size, but this may have been wishful thinking since s = 97.7 $590 vs. the alternative hypothesis HA:  ≠ $590.
• Want to conduct a decision theoretic type test with type I
error  = 0.05
• Note that this is a two-sided hypothesis and we will
construct a two-tailed p-value.
• Would this null hypothesis be rejected?
• Let’s check in Stata…

35 36