NULL HYPOTHESIS

SIGNIFICANCE TESTING (NHST)

• Null hypothesis testing is used to determine whether mean differences
among groups in an experiment are greater than the differences that are
expected simply because of error variation.

• The first step in null hypothesis testing is to assume that the groups do not
differ—that is, that the independent variable did not have an effect (the null
hypothesis).

• Probability theory is used to estimate the likelihood of the experiment’s

observed outcome, assuming the null hypothesis is true.

• A statistically significant outcome is one that has a small likelihood of

occurring if the null hypothesis were true.

• Because decisions about the outcome of an experiment are based on probabilities,

Type I (rejecting a true null hypothesis) or Type II (failing to reject
a false null hypothesis) errors may occur
EXPERIMENTAL SENSITIVITY AND
STATISTICAL POWER
• Sensitivity refers to the likelihood that an experiment will detect the effect
of an independent variable when, in fact, the independent variable truly
has an effect.
• Power refers to the likelihood that a statistical test will allow researchers to
reject correctly the null hypothesis of no group differences.

• The power of statistical tests is influenced by the level of statistical significance,

the size of the treatment effect, and the sample size.

• The primary way for researchers to increase statistical power is to increase

sample size.

• Repeated measures designs are likely to be more sensitive and to have more
statistical power than independent groups designs because estimates of
error variation are likely to be smaller in repeated measures designs.
• Type II errors are more common in psychological research using NHST
than are Type I errors.

• When results are not statistically significant (i.e., p .05), it is incorrect to

conclude that the null hypothesis is true.
NHST: COMPARING TWO MEANS
• The appropriate inferential test when comparing two
means obtained from different groups of subjects is a
t-test for independent groups.
• A measure of effect size should always be reported
when NHST is used.
• The appropriate inferential test when comparing two
means obtained from the same subjects (or matched
groups) is a repeated measures (within subjects) t-test.
STATISTICAL SIGNIFICANCE AND SCIENTIFIC
OR PRACTICAL SIGNIFICANCE
• We must recognize the fact that statistical significance
is not the same as scientific significance.
• We also must acknowledge that statistical significance
is not the same as practical or clinical significance.