CTIS 365

APPLIED DATA ANALYSIS

Dr. Seyid Amjad Ali

syedali@bilkent.edu.tr
C202, 290 5335

CTIS 365: Applied Data Analysis Slide 1

HYPOTHESIS TESTING
Hypothesis testing

Statistical procedure that uses sample data to evaluate

hypothesis about a population parameter.

In statistics, hypothesis is a claim or statement about a

property of a population.

CTIS 365: Applied Data Analysis Slide 2

HYPOTHESIS TESTING
Core logic of hypothesis testing
1. We state a hypothesis about a population. It is usually
based on existing research literature, on observation, or
through reasoning.

2. We collect data through a random sample from the

population.

3. We compare the sample data with the prediction that

was made from the hypothesis. If consistent, the
hypothesis is reasonable; if not, the hypothesis is wrong.

CTIS 365: Applied Data Analysis Slide 3

HYPOTHESIS TESTING
Basic steps in hypothesis testing

STEP 1. State the hypotheses

STEP 2. Set the criteria for a decision
STEP 3. Collect data and compute sample statistics
STEP 4. Make a decision

CTIS 365: Applied Data Analysis Slide 4

HYPOTHESIS TESTING
Example 8
Suppose a researcher is interested in the effect of
simulation during infancy on human development.
Specifically, he thinks that increased handling early in life
will result in difference in body weights.

CTIS 365: Applied Data Analysis Slide 5

HYPOTHESIS TESTING
Example
What does he know about the population?
It is known from national health statistics that the mean
weight for 2-year old children is  = 26 pounds. The
distribution is normal with  = 4 pounds.

CTIS 365: Applied Data Analysis Slide 6

HYPOTHESIS TESTING
The researcher wants to know through analyzing the data
whether the treated popn. on average has a significantly
different body weight compared to the original population

CTIS 365: Applied Data Analysis Slide 7

HYPOTHESIS TESTING
STEP 1. STATE THE HYPOTHESES

The null hypothesis (H0) states that in the general

population there is no change, no difference, or no
relationship. In the context of an experimental study, H0
predicts that the treatment will have no effect on the
dependent variable for the population.

H0:  after treatment(=  before treatment )= 26

CTIS 365: Applied Data Analysis Slide 8

HYPOTHESIS TESTING
STEP 1. STATE THE HYPOTHESES

The alternative hypothesis (H1) states that there is a

change, a difference, or a relationship for the general
population. In the context of an experimental study, H1
predicts that the treatment will have an effect on the
dependent variable for the population.

H1:  after treatment (≠  before treatment) ≠ 26

CTIS 365: Applied Data Analysis Slide 9

HYPOTHESIS TESTING
STEP 2. SET THE CRITERIA FOR A DECISION

Determine the cutoff sample score on the comparison

distribution at which the null hypothesis should be rejected
Also called the “critical value”
In general, researchers use a cutoff with probability of 5% or
less or sometimes 1% or less

CTIS 365: Applied Data Analysis Slide 10

HYPOTHESIS TESTING
STEP 2. SET THE CRITERIA FOR A DECISION

The alpha level or level of significance is a probability

value that is used to define the very unlikely sample
outcomes if the null hypothesis is true.
Commonly used alpha levels are:  = .05,  = .01,  = .001

CTIS 365: Applied Data Analysis Slide 11

HYPOTHESIS TESTING
STEP 2. SET THE CRITERIA FOR A DECISION

The critical region is composed of extreme sample values

that are very unlikely to be obtained if the null hypothesis
is true. The boundaries for the critical region are
determined by the alpha level. If sample data fall in the
critical region, the null hypothesis is rejected.

CTIS 365: Applied Data Analysis Slide 12

HYPOTHESIS TESTING
STEP 2. SET THE CRITERIA FOR A DECISION

CTIS 365: Applied Data Analysis Slide 13

HYPOTHESIS TESTING
STEP 2. SET THE CRITERIA FOR A DECISION

CTIS 365: Applied Data Analysis Slide 14

HYPOTHESIS TESTING
STEP 3. COLLECT DATA AND COMPUTE
SAMPLE STATISTICS

- A sample of 16 newborn infants was selected.

- Parents were trained to provide daily additional handling
to their baby.
- The body weight for each of the 16 infants were weighed
at age 2.
- The mean weight for the treated sample was computed:
M = 30 pounds and SD = 3.8

CTIS 365: Applied Data Analysis Slide 15

HYPOTHESIS TESTING
STEP 3. COLLECT DATA AND COMPUTE
SAMPLE STATISTICS
We want to know through data
analysis whether the treated
sample mean M = 30 pounds differs
significantly from the population
mean m = 26 pounds.

CTIS 365: Applied Data Analysis Slide 16

HYPOTHESIS TESTING
STEP 3. COLLECT DATA AND COMPUTE
SAMPLE STATISTICS

The comparison is established by computing a z-score that

describes exactly the location of sample mean relative to
the hypothesized population mean.

M  M 
z 
M 
n

CTIS 365: Applied Data Analysis Slide 17

HYPOTHESIS TESTING
STEP 3. COLLECT DATA AND COMPUTE
SAMPLE STATISTICS

M   M   30  26
z    4.00
M  n 4 16

With an alpha level of .05, this calculated z-score is far

beyond the boundary of 1.96.

CTIS 365: Applied Data Analysis Slide 18

HYPOTHESIS TESTING
STEP 4. MAKE A DECISION

z-score of 4 falls in
the critical region.
Thus, we reject the
null hypothesis and
conclude that special
handling did have an
effect on the weight
of infants

CTIS 365: Applied Data Analysis Slide 19

HYPOTHESIS TESTING
STEP 4. MAKE A DECISION

If calculated z-score
was not in the critical
region, then we
would fail to reject the
null hypothesis and
conclude that the
treatment does not
have an effect.

CTIS 365: Applied Data Analysis Slide 20

HYPOTHESIS TESTING
STEP 4. MAKE A DECISION

If we set  = .01, what

would be our decision in
this research study?

For  = .01, the corresponding

critical z-value is 2.58. Because
the calculated z-statistic > the
critical z, we reject H0.

CTIS 365: Applied Data Analysis Slide 21

HYPOTHESIS TESTING
ANOTHER EXAMPLE

Suppose a researcher is interested in the effect of prenatal

alcohol on birth weight. A random sample of n = 16
pregnant rats is obtained. The mother rats are given daily
doses of alcohol. The average weight at birth for the
sample is 15 grams. It is known that regular newborn rats
have an average weight of  = 18 grams. The distribution
of weights is normal with  = 4.

CTIS 365: Applied Data Analysis Slide 22

HYPOTHESIS TESTING
STEP 1: State the hypotheses
The null hypothesis states that exposure to alcohol has no
effect on birth weight.
H0: alcohol exposure = 18

The alternative hypothesis states that exposure to alcohol has

effect on birth weight.
H1: alcohol exposure ≠ 18

Note that null and alternative hypotheses are opposite to

each other. They cannot be both true.
CTIS 365: Applied Data Analysis Slide 23
HYPOTHESIS TESTING
STEP 2: Set the criteria for a decision
1. Construct the distribution of
sample means. The distribution
of sample means corresponds to
all the possible outcomes that
could be obtained if H0 is true;
2. Identify the boundaries of
the critical region (the critical z-
values, defined by  level) that
separates the extreme outcomes
from the high-probability
outcomes.

CTIS 365: Applied Data Analysis Slide 24

HYPOTHESIS TESTING
STEP 3: Collect the data and compute the test
statistic
M   15  18
z   3.00
M 4
16

Note that test statistic indicates that the sample data are converted
into a single, specific statistic that is used to test hypothesis. It can be
z, t, F,... We will talk about each later.

CTIS 365: Applied Data Analysis Slide 25

HYPOTHESIS TESTING
STEP 4: Make a decision

The calculated z-score has a value of -3.00, which is

beyond the boundary of -1.96. This means, the sample
mean is located in the critical region. Our decision is to
reject the null hypothesis, and conclude that exposure to
alcohol has a significant effect on birth weight.

CTIS 365: Applied Data Analysis Slide 26

HYPOTHESIS TESTING
REPORTING THE RESULTS

The prenatal exposure to alcohol has a significant effect on

birth weight, z = - 3.00, p < .05.

Findings are said to be significant or statistically significant

when the null hypothesis has been rejected. Difference is
not due to chance.

CTIS 365: Applied Data Analysis Slide 27

HYPOTHESIS TESTING
DIRECTIONAL VS. NONDIRECTIONAL

One-tailed test vs. Two-tailed test

In directional (one-tailed)
hypothesis test, the statistical
hypotheses specify either
decrease or increase in the
population mean score and
places the critical region in one H0: treated ≤ 26
tail of distribution.
H1: treated > 26

CTIS 365: Applied Data Analysis Slide 28

HYPOTHESIS TESTING

CTIS 365: Applied Data Analysis Slide 29

HYPOTHESIS TESTING
One-tailed test vs. Two-tailed test

• Two-tailed test is more conservative than one-tailed test

• One-tailed test is used when there is prior justification
for making a directional prediction.
• Some believe that two-tailed tests should always be
used, even when an experimenter makes a directional
prediction

CTIS 365: Applied Data Analysis Slide 30

HYPOTHESIS TESTING
EFFECT SIZE

Effect size is a name given to a family of indices that measure

the magnitude of a treatment effect. Unlike significance tests,
these indices are independent of sample size.

The most commonly used are eta-squared, Cohen’s d, and

R2.

CTIS 365: Applied Data Analysis Slide 31

HYPOTHESIS TESTING
EFFECT SIZE

Cohen’s d = |mean difference|/ standard deviation

CTIS 365: Applied Data Analysis Slide 32

HYPOTHESIS TESTING
EFFECT SIZE

Cohen’s d = mean difference / standard deviation

For the previous example, Cohen’s d can be calculated as:

15 - 18
Cohen' s d   0.75
4

According to standards suggested by Cohen, this is a

medium effect.

CTIS 365: Applied Data Analysis Slide 33

HYPOTHESIS TESTING
ASSUMPTIONS OF THE Z-TEST
•Random sampling
•Independent observations
•Normal sampling distribution
•The value of  is unchanged by the treatment

CTIS 365: Applied Data Analysis Slide 34

HYPOTHESIS TESTING
Type I error

• Reject the null hypothesis when it is true

• Conclude that a treatment has an effect when in

fact it does not
• How to control and minimize the risk of
committing a Type I error?
• The probability of a Type I error is equal to the
alpha level

CTIS 365: Applied Data Analysis Slide 35

HYPOTHESIS TESTING
Type II error

• Fail to reject the null when it is false

• Conclude that a treatment does not have an

effect when in fact it does
• Represented by 

CTIS 365: Applied Data Analysis Slide 36

HYPOTHESIS TESTING
Errors in Hypothesis Testing

Type I error and Type II error

CTIS 365: Applied Data Analysis Slide 37

HYPOTHESIS TESTING
Selecting an alpha level

CTIS 365: Applied Data Analysis Slide 38

HYPOTHESIS TESTING
STATISTICAL POWER

Probability that the test will correctly rejects a false

null hypothesis; i.e., probability of detecting a
treatment effect when one really exists.

Power
= p (reject a false H0)
=1-b

CTIS 365: Applied Data Analysis Slide 39

HYPOTHESIS TESTING
FACTORS THAT AFFECT POWER
Treatment effect
Alpha level
Sample size
One-tailed vs. two-tailed test

CTIS 365: Applied Data Analysis Slide 40

HYPOTHESIS TESTING
FACTORS THAT AFFECT POWER
 Treatment effect: the larger the effect, the higher
the power

CTIS 365: Applied Data Analysis Slide 41

HYPOTHESIS TESTING

CTIS 365: Applied Data Analysis Slide 42

HYPOTHESIS TESTING

CTIS 365: Applied Data Analysis Slide 43

HYPOTHESIS TESTING

CTIS 365: Applied Data Analysis Slide 44

HYPOTHESIS TESTING
FACTORS THAT AFFECT POWER
 Treatment effect: the larger the effect, the higher
the power
 Alpha level: the larger the alpha, the higher the
power

CTIS 365: Applied Data Analysis Slide 45

HYPOTHESIS TESTING

a=.10
a=.05

CTIS 365: Applied Data Analysis Slide 46

HYPOTHESIS TESTING
FACTORS THAT AFFECT POWER
 Treatment effect: the larger the effect, the
higher the power
 Alpha level: the larger the alpha, the higher
the power
 Sample size: the larger the sample size, the
higher the power

CTIS 365: Applied Data Analysis Slide 47

HYPOTHESIS TESTING

CTIS 365: Applied Data Analysis Slide 48

HYPOTHESIS TESTING
FACTORS THAT AFFECT POWER
 Treatment effect: the larger the effect, the higher the
power
 Alpha level: the larger the alpha, the higher the
power
 Sample size: the larger the sample size, the higher
the power
 One-tailed vs. two-tailed test: everything equal,
one-tailed test is more powerful than two-tailed test.

CTIS 365: Applied Data Analysis Slide 49

HYPOTHESIS TESTING

a=.05, one-tailed

a=.05, two-tailed
CTIS 365: Applied Data Analysis Slide 50