Type 1 and Type 2 Errors

Objectives
What is the difference between statistical error and
systematic error?
What is Ho and Ha?
What is a type 1 & 2error?
Give an example of a study showing type 1 and type 2 error
How to minimize type 1 and type 2 errors?
What does P value and Power of a study mean?
How to use these concepts while interpreting a study?
How is the calculation of type 1 and type 2 done?
Other types of errors?
Statistical error vs Systemic error
Statistical error: the difference between a measured
value and actual value is due to Random(Chance)
element.
• All continuous variables have statistical uncertainty.
• Intrinsic randomness of the Universe.
• Sometimes you can measure it with the variation.
• Sometimes you need to assign it based upon the precision
of your device.
Statistical error vs Systemic error
• Systematic error: the difference between a measured
value and actual value is due to a Non random element

• In addition, measurement devices can have systematic

uncertainties.
• These are generally harder to get a handle on.
• One way is to try and measure a different way.
• Statistical and systematic uncertainties are related to the
ideas of accuracy and precision.
Etymology
1933 -Jerzy Neyman and Egon Pearson described the
concept of the two types of errors
i)Rejecting null hypothesis when it is true
ii)Accepting null hypothesis when alternate
hypothesis is true
Coined the terms type 1 and type 2 errors
2 kinds of mistakes
Type 1 –We say something is true when it is not true
(false positive)

Type 2 – We say something is not true when it is true

(false negative)
 Type 1 –XYZ is a good
actress
Error

Type 2 –Amitabh Bachhan is not

a good actor
Inference
The process of drawing a conclusion about a larger
group of people based on data from a smaller
subgroup
The Role of Chance
A subgroup (study sample) may differ from the larger
group by chance alone

Statistical tests can be used to determine if observed

differences are explained by chance
Null Hypothesis(Ho)
Similar to a trial –Every person is NOT GUILTY unless
proven otherwise

Corresponds to a presumed ‘default state of nature’

Null hypothesis
British statistician Sir Ronald Aylmer Fisher(1890–
1962) stressed that the "null hypothesis":
...is never proved or established, but is possibly
disproved, in the course of experimentation. Every
experiment may be said to exist only in order to give
the facts a chance of disproving the null hypothesis.
(1935)
Alternate Hypothesis(Ha)

Alternate hypothesis (Ha)- Alternative to null

hypothesis
The Screen Test
Hollywood producer who doesn’t know about bollywood
(or how to do a scientific study) wants to study
popularity of actress Rakhi Sawant and Amitabh
Bachchan
 Sawant Study- α error
Null hypothesis(Ho): Not more than 50% of public
like Rakhi Sawant
Alternate hypothesis(Ha):More than 50% of public
like Rakhi Sawant
Results: Study conducted shows more than 60% like
Rakhi Sawant
Conclusion: Ho rejected (Positive Study)
Result :Picture flops because of type 1 error
How could this α error happen?
Accuracy of study –Randomization?
Types of questions-vague?
Frequency –Sample size
how many people are aware about the actress and her
Accomplishments
Lack of specificity increases chances of α error
B Big B study -(Big)β error
Null hypothesis :Not more than 50% like Amitabh
Bachhan
Alternate Hypotheis: More than 50% like Amitabh
Bacchan
Study shows not more than 30% like Big B
Conclusion- Null hypothesis accepted (Negative
study)
Consequence –Picture flops due to type 2 error
How could this β error happen?
Accuracy- was criteria selected too strict?
Randomization?
Frequency – how many in study population familar
with Big B?
High Specificity=Higher chance of type 2 error
Type 1 (α)-Error can only happen in a positive study
( False positive)

Type 2 (β)Error can only happen in a negative study

(False negative)

You cannot make a Type 1 and Type 2 error in the same

study (because you can’t reject AND accept Ho at the
same time!)
 TRUTH
Good actor Not Good
actor

STUDY Good actor (1-beta) Type I error

Power (alpha error)
FP error
‘P’ value
Not Good Type II error (1-alpha)
actor (beta error) Confidence
FN error
 The Judges Dilemma
Not
Guilty Guilty

Type I error

Type II Error
 TRUTH
Guilty Innocent

JUDGE Guilty (1-beta) Type I error

Power (alpha error)
FP error

Innocent Type II error (1-alpha)

(beta error) Confidence
FN error
Which is a more serious error?
Conventionally
Accepted alpha error is 0.05
Accepted Beta error is 0.2

Alpha error is “error of commission” – it changes

practice
Beta error is “error of omission”
Is there an association?

Is it due to chance?

Is it due to bias?

Is it due to confounding?

Is it causal?
‘p’ value < 0.05 – what does it
really mean?
“p” is the probability that the result is just by chance
It is nothing but alpha error
That is… when we conclude that there is an
association but in reality there is no association.
Lower the p value more confident we are that the
result is true…
Comparison with screening test
P value Sensitivity
High p value = high sensitivity, greater chances of
making false positive error

Power of Study Specificity

High Power = high specificity, greater chances of making
a false negative error
Chance implies probability
P value –Probability of making a Type 1 error
Probability that the difference observed between the
two groups is due to chance alone

E.g.1 : ‘Only 11% of husbands reported having unmet

need for family planning as compared to 17.5% of wives
(P value<0.001)’
E.g2: ‘the prevalence of BP in males (44.68%) was less
compared to females (55.22%),
(P>0.05)

E.g.3 Average pre-test score was 25.09 (SD 4.02)

And Post test sore was 50.64(6.99), P value <0.01
P value
More significance in hypothesis generating
studies(descriptive,analytical)

Level of significance chosen according to needs of

study- 0.05/0.01/0.001(greater resources, greater
sample size, lower p value)

Basically tells you if the difference between the two

groups is significant.
To reduce chance of α error
Choose lowest possible level of
significance,according to possible sample size
Power
More important in Experimental Studies
(Hypothesis confirming)

The probability of NOT making a Type 2 error

Power of 95% = less than 5% chance of making a type

2 error

Power increases with increasing sample size

“The sample size for this trial was determined using
GPower [58]. This indicated that complete data from
62 participants would provide 80% power to detect a
minimal clinically important improvement of 20% in
pain outcomes”

Interpretation
To reduce chances of β error
 Choose highest possible Power of study depending on
sample size

Usually power of study is set at 85%, but for really

sophisticated studies it may be set at 95%
Minimizing Error
Minimizing errors is not simple —reducing one type
of error generally results in increasing the other type
Based on the real-life application, one type of error
may be more serious than the other.
(In such cases, a compromise should be reached to
reduce the more serious type of error.)
Sample size
The only way to reduce both types of errors is to
increase the sample size and this may or may not be
feasible
Bayes Theorem
The probability that an observed positive result is a
false positive (and not a true positive) may be
calculated using Bayes's theorem.
The key concept of is that the true rates of false
positives and false negatives are a function of the
accuracy of the test , and also the actual rate or
frequency of occurrence of state within the test
population.
 Scenarios
A particular compound is not hazardous in drinking water
if it is present at a rate of no more than 25ppm. The
Community Medicine believes that a certain water source
does not meet this standard.
μ: mean amount of the compound (in ppm)
H0: μ < 25
Ha: μ > 25

If the Invesigators decides to gather data and formally conduct

this test, describe type I and type II errors in the context of this
scenario and the consequences of each.
 Scenarios
Type I error:
Stating that the evidence indicates the water is unsafe when,
in fact, it is safe.
The Investigators will initiate a clean-up where none is
required (Rs. wasted).
Type II error:
Stating that there is no evidence that the water is unsafe
when, in fact, it is unsafe.
The opportunity to note (and repair) a potential health risk
will be missed.
Other Types of Errors
Type III error
(In systems theory):Asking the wrong question and
using the wrong null hypothesis

, Frederick Mosteller(1948)
"correctly rejecting the null hypothesis for the wrong
reason
Type IV?
Marascuilo and Levin (1940)- Incorrect interpretation
of correctly rejected Ho

 Raiffa, Harvard economist (1969)- Solving the right

problem too late
Summary
Two kinds of errors in hypothesis testing –type 1 and
type 2
Type 1(α) is false positive, can only occur in a positive
study
P value –denotes probability of making type 1 error
Type 2(β) is false negative, can only occur in negative
study
Power – denotes probability of not making type 2 error
Actual mathematical calculations done using Bayes
theorem
Reducing probability of making T1 error increases
probability of making T2 error and vice versa
Increasing sample size reduces both types of errors
Type 1 (false +) – possibility shown by pvalue(Sensitivity)
Type 2 (false -) –possibiltiy shown by Power(Specificity)
References
Park’s Textbook of Preventive and Social medicine 18th
edition
‘Type 1 and Type 2 errors’ Wikipedia
IJCM, Jun-Sep, 2009 (for article excerpts)
Magnet therapy for the relief of pain and
inflammation in rheumatoid arthritis : A randomised
placebo-controlled crossover trial(on pub med central)
AP Stat Review, April 18, 2009