Lec5 3.

April 29, 2005

The Power of a Statistical Test

Denition Ways to Increase the Power Two Types of Error Revisited

The power of a statistical test The power of a signicance test measures its ability to detect an alternative hypothesis. The power against a specic alternative is calculated as the probability that the test will reject H0 when that specic alternative is true. This calculation requires knowledge of the sampling distribution of the test statistic under the alternative hypothesis.

Example: Does exercise make strong bones? Can a 6-month exercise program increase the total body bone mineral content (TBBMC) of young women? A team of researchers is planning a study to examine this question. Based on the results of a previous study, they are willing to assume that = 2 for the percent change in TBBMC over the 6-month period. A change in TBBMC of 1% change would be considered important, and the researcher would like to have a reasonable chance of detecting a change this large or larger. Are 25 subjects a large enough sample for this project?

1. State the hypotheses: let denote the mean percent change. H0 : = 0 Ha : > 0 2. Calculate the rejection region: The z test rejects H0 at the = 0.05 level whenever z= x x 0 = 1.645 / n 2/ 25

that is, we reject H0 when x 1.645 2 = 0.658 25 3. Compute the power at a specic alternative: The power of the test at alternative = 1 is P ( 0.658| = 1) x

0.658 1 x | = 1) = P( / n 2/ 25 = P (Z 0.855) = 0.80

Ways to Increase the Power Increase . A 5% test of signicance will have a greater chance of rejecting the alternative than a 1% test because the strength of evidence required for rejection is less. Consider a particular alternative that is farther away from 0 . Values of that are in Ha but lie close to the hypothesized value 0 are harder to detect than values of that are far from 0 . Increase the sample size. More data will provide more information about x so we have a better chance of distinguishing values of . Decrease . This has the same eect as increasing the sample size: it provides more information about . Improving the measurement process and restricting attention to a subpopulation are two common ways to decrease .

Example: Does exercise make strong bones? (cont.)

Change the signicance level to = 0.01. The z test rejects H0 at the = 0.01 level whenever z= x x 0 = 2.32 / n 2/ 25

The power of the test at alternative = 1 is

that is, we reject H0 when x 2.32 2 = 0.928 25 0.928 1 x | = 1) P ( 0.928| = 1) = P ( x / n 2/ 25 = P (Z 0.18) = 0.57

Change the alternative to = 2.

The power of the test at alternative = 2 at = 0.01 level is P ( 0.928| = 2) = P ( x x 0.928 2 | = 2) / n 2/ 25

= P (Z 2.68) = 0.996

 Increase the sample size n = 100.

The z test rejects H0 at the = 0.01 level whenever x x 0 = z= 2.32 / n 2/ 100 The power of the test at alternative = 1 is P ( 0.464| = 1) = P ( x
2 that is, we reject H0 when x 2.32 100 = 0.464

0.464 1 x | = 1) / n 2/ 100 = P (Z 2.68) = 0.996

Decrease = 1

The z test rejects H0 at the = 0.01 level whenever x 0 x = z= 2.32 / n 1/ 100 The power of the test at alternative = 1 is P ( 0.232| = 1) = P ( x
1 that is, we reject H0 when x 2.32 100 = 0.232

0.232 1 x | = 1) / n 1/ 100 = P (Z 7.68) 1

Two Types of Errors Revisited

Recall that the following four outcomes are possible when conducting a test: Reality Our Decision H0 (Prob = 1 ) Type II Error (Prob = ) Ha Type I Error (Prob = ) (Prob = 1 )

H0 Ha

The signicance level of any xed level test is the probability of a Type I error. The power of a xed level test against a particular alternative is 1 for that alternative. In practice, we rst choose an and consider only tests with probability of Type I error no greater than . Among all level , we select one that makes the probability of Type II error as small as possible (i.e. the most powerful possible test).