Hypothesis Testing

• Hypothesis testing is a way of examining a

statement about a relationship between
independent and dependent variables:
• Independent variable: the variable whose effects
the experimenter is interested in studying.
• Dependent variable: the variable that the
experimenter measures (the data).
Independent and Dependent Variables - Example
• If an experimenter is interested in researching how
hours of studying for an exam affect performance
on a test, the variables are as follows:
Independent Variable (IV): hours spent studying
Dependent Variable (DV): performance on test (e.g.,
grade received).
 Independent Variables
• There are 2 broad types of IVs:
1. Treatment Variable: a treatment the experimenter
applies to previously undifferentiated participants.
• E.g., certain participants are told to study for 5 hours
and others are told to study for 2 hours.
2. Categorical Variable: A characteristic that is inherent
to, or pre-exists, in the participant.
• E.g., sex – you can’t assign someone a sex.
 Levels of IV
• We also talk about the levels of IVs – how we break
down the IV.
• E.g., if we are interested in studying the IV of hours spent
studying, it could have 2 levels – 2 hours and 5 hours.
• Studying the IV of sex has 2 levels – male and female.
• The levels of an IV are compared on their DV scores to
look for a difference in outcome.
• E.g., is there a difference in test performance between
those who study for 5 hours and those who study for 2
hours?
 Null Hypothesis
• Tests of hypotheses are decisions to retain or
reject a null hypothesis (Ho).
• Null hypothesis (Ho) : a statement of relationship
between the IV and DV, usually a statement of no
difference or no relationship.
• We assume there is no relationship between IV
and DV.
 Null Hypothesis Examples
• Men and women do not differ in IQ (men =
women)
• Hours spent studying do not affect test
performance (2 hours = 5 hours)
• Height does not affect weight (short = tall)
 Null Hypotheses
• Null hypotheses contain 3 components:
The IV comparison being made
The DV being measured
The null relationship between IV and DV (e.g.,
“do not differ”).
 Alternative Hypothesis
• Although not directly tested, the Alternative
Hypothesis (Ha) does state a relationship, or effect,
of the IV on the DV.
• This is often called the Research Hypothesis.
• E.g.,
Ha: Men and women do differ in IQ (men  women)
Ha: Women have higher IQs than men (women > men)
 Directional Ha
• Ha: Women have higher IQs than men (women >
men) is a directional alternative hypothesis.
• We state that one level of the IV will have greater
(or lesser) DV scores than the other level.
• When we make a directional alternative
hypothesis, we have a reason (either based on past
research or a theory) to predict the direction of the
results.
• (Note: the above example is hypothetical only).
 Non-Directional Ha
• A non-directional alternative hypothesis does not
state the expected direction of effect:
Ha: Men and women have differing IQs (women 
men).
• We make a non-directional alternative hypothesis
when we have no reason to predict the direction of
the results.
• For instance, since there is no theory or research
body that would suggest that women should have
higher IQs than men, we would only predict that
their IQs are different than men’s.
 Confidence Level
• Confidence Level: a criterion level of probability
(alpha ), set by the experimenter, which acts as
the reference for deciding whether to reject or
retain the null hypothesis.
• Significant Result at .05: we determine the null
hypothesis is not true but there is a 5% chance that
the null hypothesis is actually true.
 Confidence Level
• The confidence level is set by the experimenter,
but generally the convention is to use  = 0.05
and  = 0.01.
• For  = 0.05, this means that there is a 5% chance
we will reject the null hypothesis when it is
actually true.
 Rejecting the Null Hypothesis
• If the likelihood of observing the outcome is
below the confidence level ( = 0.05 or  =
0.01), then we say that the result is significant
and we reject the null hypothesis.
• Significant results reject Ho (there is a
difference).
• Non-significant results retain Ho (there is no
difference).
 Type I and Type II Errors
• When we decide to retain or reject the null
hypothesis, we never do so with 100% certainty
we are making the right decision – we make the
decision with a probability of being correct (the
alpha level).
• We can make an incorrect decision, resulting in 2
types of errors, Type I or Type II.
 Type I Errors
• Type I Error: Rejection of the null hypothesis
when it is true.
• We conclude that the IV affects or is related to the
DV when in reality the result was due to sampling
error.
• We see something that is not really there.
 Type I Error Example
• If our null hypothesis is that men and women do
not differ in IQ, the Type I error is:
Finding a result that men and women do differ in
IQ, when in reality they do not.
• We find this difference because of sampling
error.
 Type II Errors
• Type II Error: Retention of the null hypothesis
when it is false.
• We conclude that the IV does not affect or is not
related to the DV when in reality there is an effect
or relationship.
• We fail to see something that is really there.
 Type II error Example
• If our null hypothesis is that men and women do
not differ in IQ, the Type II error is:
Finding a result that men and women do not
differ in IQ, when in reality they do.
Type I and Type II Errors
 One vs. Two Tailed Tests
• The “tails” of a test set up our rejection region – they
determine how we decide to retain or reject Ho.
• When we use a one-tailed test, we are testing the null
hypothesis for a directional alternative hypothesis
(e.g., Ha: women will have higher IQs than men).
• We are only interested in whether or not women have
higher IQs than men, not lower.
 Two-Tailed Tests

• When we use a two-tailed test, we are testing the

null hypothesis for a non-directional alternative
hypothesis (e.g., Ha: women and men will have
different IQs).
• Here, we are interested in whether or not women
have higher or lower IQs than men.
 One vs. Two Tailed Tests (using  = 0.05)

2.5% 2.5% 5% 5%