Accuracy of Observations and Measurements

EPIDEMIOLOGY
Dr. Ingrid Mejia Paneda || January 26, 2017

Instrumental variation
Content Outline:
I. Sources of variations  variation in the observation of a phenomena caused
a. Inter-observer Variation by the instruments or equipment being used
b. Intra-observer variation  ex. Use of a Littmann stethoscope and a non-branded
c. Inter-individual variation one
d. Intra-individual variation
e. Instrument variation II. ACCURACY AND PRECISION
II. Accuracy vs Precision
III. 2x2 table Accuracy
IV. Sensitivity and Specificity  Accuracy is the ability of a measurement to be correct
V. Predictive values on the average.
a. Positive Predictive value  It is sometimes termed as validity in research. Gives
b. Negative Predictive value you something that is closest to the true value.
VI. Relationship of Disease Prevalence to
Predictive value Formula:
VII. Clinical Applications
a. Screening Tests
b. Confirmatory tests
VIII. Exercise 3

Precision
LEGEND:
‼ Precision is the ability of a measurement to give the
same or a very similar result with repeated
Presentation Mentioned in Remember Book measurements of the same thing.
the lecture  It is sometimes termed as repeatability/
reproducibility in research.
I. SOURCES OF VARIATION
Inter-observer variation
difference in results obtained by two or more observers
examining the same phenomena
 Can be attributed to difference in technique,
knowledge and skills, and physical and mental
factors.
 ex. Two radiologists having different observations on
the same x-ray
Can be solved through a 3rd opinion and/or additional
diagnostic tests

Intra-observer variation
difference incurred by one observer when observing
the same phenomenon more than once
 Can be attributed to change in environment, change
in mood, physical state, etc.
 ex. Nurse taking the vital signs of a patient at the start
and end of her shift.
Inter-individual variation
 variation in the observation when observing two or
more individuals; observer can be one or more
 Can be attributed to mood, genetics, age, sex,
behavior
ex. Taking the respiration rate of a patient who is
crying and another who is calm
Intra-individual variation
 difference in observation within one person, subject or
sample
 Can be attributed to change in environment, change
in mood, physical state, etc.
 ex. Manny Pacquiao's heart rate while singing and
boxing
Figure 1. Accuracy and Precision through a Bull's Eye
III. 2X2 TABLE
 2 rows and 2 columns
 Column Headings= (+) Disease, (-) Disease
 Stubs or Subheading= (+) Test Result, (-) Test Result
 Each column usually refers to a sub population
 One can use letters or other names to classify every
set of participants
 True Positives (TP) - Positive test result, has disease
(Group A)
 False Positives (FP) - Positive test result, no disease
(Group B)
o “type I error or alpha (α) error” – patient will be
given needless therapy which may be harmful to
the patient
 False Negatives (FN) - Negative test result, has
disease (Group C)
o ”type II error or beta (β) error”–patient is not
treated and may suffer severe consequences
 True Negatives (TN) - negative test result, no
disease (Group D)

E1 T4 | Gill | Mendoza | Mojica | Molina | Morales |Nidua 1 of 5

 Accuracy of Observations and Measurements
!! The word positive has a letter "o" in it, which represents one,  Formula:
and the first letter of the alphabet, a. Therefore, it is a type 1, True Negatives (TP)
alpha error. Specificity = x100
True Positives (TP) + False Negatives (FN)

 Interpretation:
o “There is a __% probability of subjects
without disease testing negative using the
tool”.

 2x2 contingency table:

A (TP) B (FP)

C (FN) D (TN)(numerator)

V. PREDICTIVE VALUE
 Tests used for determining presence or absence of
Figure 2. Summary of the formulas for accuracy determination disease
IV. SENSITIVITY AND SPECIFICITY Positive Predictive Value
 Characteristics of diagnostic tests (screening or  Probability that the subject HAS the disease when the
confirmatory) test is POSITIVE.
 Tests used for diagnostic tools  Formula:
True Positives (TP)
Sensitivity PPV = x100
 Ability of a tool to yield a POSITIVE result or to detect True Positives (TP) + False Positives (FP)
disease when the disease is actually PRESENT.
 You are sensitive to the needs of the people and  Interpretation:
would like to catch/treat everyone who has the o “There is a __% probability of subjects, who
disease. tested positive, having the disease”.
o High sensitivity when TP is high and FN  2x2 contingency table:
is low. (2019A)
 Formula: A (TP) (numerator) B (FP)
True Positives (TP)
Sensitivity = x100
True Positives (TP) + False Negatives (FN) C (FN) D (TN)

 Interpretation:
o “There is a __% probability of subjects with Negative Predictive Value
disease testing positive using the tool”.  Probability that the subject DOES NOT HAVE the
 2x2 contingency table: disease when the test is NEGATIVE.
 Formula:
A (TP) (numerator) B (FP)
True Negatives (TN)
NPV = x100
True Negatives (TN) + False Negatives (FN)
C (FN) D (TN)

Specificity  Interpretation:
 Ability of a tool to yield NEGATIVE result when o “There is a __% probability of subjects who
disease is actually ABSENT. tested negative, not having the disease”.
 Well, you’re not Santa Claus; you have financial  2x2 contingency table:
constraints too, so you try to be specific, and exclude
everyone who does not have the disease in order to A (TP) B (FP)
avoid wasting resources
o High specificity when TN is high and FP
is low. (2019A) C (FN) D (TN) (numerator)

2 of 5
E1 T4
 Accuracy of Observations and Measurements
VI. RELATIONSHIP OF DISEASE PREVALENCE TO PREDICTIVE
VALUES
 Prevalence – the total number of people who are
positive for the disease in the entire population being
observed.
𝑇𝑃 + 𝐹𝑁
𝑃𝑟𝑒𝑣𝑎𝑙𝑒𝑛𝑐𝑒 =
𝑇𝑃 + 𝐹𝑃 + 𝑇𝑁 + 𝐹𝑁
 From the formula, we can derive these relationships:
o ↑ Prevalence: ↑ TP ↑ FN
o ↑ Prevalence: ↓ FP ↓ TN
 Recall the formula for Positive Predictive
Value(PPV) and Negative Predictive Value(NPV):
they are directly related to TP and TN, respectively.
 Consequently:
o ↑ Prevalence: ↑ PPV ↓ NPV
o ↓ Prevalence: ↓ PPV ↑ NPV or,
o Prevalence is directly related to PPV and
inversely related to NPV.
Figure 3. Relationship between disease prevalence and predictive
Hypothetical Example value in a test with 95% sensitivity and 85% specificity. From
 100 people are tested for disease. 15 people have the Mausener JS, Kramer S: Mausner and Bahn Epidemiology: An
disease; 85 people are not diseased. Introductory Text (1985)
o Prevalence = 15%
 In practical applications, when a disease has a high
Table 2. Screening Test X prevalence, and a patient consults with symptoms:
Truth o Do a screening test, and if positive, start
Test Result Disease Non Disease Total treatment.
Positive 10 (TP) 40 (FP) 50 o The high prevalence means there is a high
Negative 5 (FN) 45 (TN) 50 PPV, or that there is a low chance of a false
Total 15 85 100 positive.
o Sensitivity = 10/15 x 100 = 67%  When a disease has a low prevalence,
o Specificity = 45/85 x 100 = 53% o Do both screening and confirmatory tests
o PPV = 10/50 x 100 = 20% before starting treatment.
o NPV = 45/50 x 100 = 90% o The low prevalence means there is a high
NPV, and not PPV.
Table 3. Screening Test X, Increased Prevalence  In any case, specificity and sensitivity are not affected
Truth since they are inherent characteristics of the
Test Result Disease Non Disease Total diagnostic tests.
Positive 20 (TP) 33 (FP) 53
Negative 10 (FN) 37 (TN) 47 VII. CLINICAL APPLICATIONS:
Total 30 70 100 Specificity
 Used to “rule out” false hypothesis/ differential
 Using the same test but in a different population, diagnosis
o Prevalence = 30%  Must have high sensitivity and with low false
 We maintain the sensitivity and specificity because negatives (Type II error rate) so that you won’t be
these are inherent characteristics of the test. sending home people who actually have the disease
o Sensitivity = 20/30 x 100 = 67%
o Specificity = 37/70 x 100 = 53% !! SNOUT: SeNsitivity-> rule OUT
 Knowing the relation of Prevalence to PPV and NPV,
the latter values should change.
o PPV = 20/53 x 100 = 38% CONFIRMATORY TESTS
o NPV = 37/47 x 100 = 79%  Used to “rule in” true hypothesis/ diagnosis
 Using the same test in a population with higher  Must have high specificity and with low false positives
prevalence yields an increase in PPV and a decrease (Type I error rate) so that you do not mistakenly
in NPV. diagnose someone with a disease he/she does not
have

!! SPIN: SPecific -> rule IN

3 of 5
E1 T4
 Accuracy of Observations and Measurements
VIII. EXERCISE 3
Part 1
1. Give your observations on the blood pressure
readings obtained by the readers.
 An increase in activity increases blood pressure
 Change of position from sitting to standing caused a
decrease in blood pressure
 Blood pressure measurements may show differences
but are still within a range
 Due to differences in weight, height, body fat, muscle
mass, gender, etc., blood pressure may vary
 The use of different tools for blood pressure
measurement showed differences in blood pressure
 Using the aneroid sphygmomanometer showed blood
pressure readings in multiples of 5, compared to the
digital sphygmomanometer’s readings that do not
exactly have a trend in its values
2. What are the possible sources of variations in
observations and measurements? Explain briefly
the meaning of such variations.
 Intra-observer variation: within the observer;
chances of error referring to changes in the condition
of the observer analyzing the data
o Example: focus vs. distraction
 Inter-observer variation: between observers;
chances of error referring to differences in
characteristics among observers analyzing the data
o Example: technique or mastery of the
observer
 Intra-individual variation: within subject; chances of
error referring to the changes in the condition of the
subject
o Example: ability to relax, changes in mood
 Inter-individual variation: between subjects;
chances of error referring to the differences in
characteristics among different subjects being
observed
o Example: weight, height, body fat, muscle
mass, gender
 Instrument variation: chances of error referring to
technical differences in the instrument used to gather
the data
o Example: aneroid sphyg vs. digital sphyg
Part 2
1. How accurate is the Doppler-guided biopsy in the
diagnosis of prostate cancer? Show all pertinent
calculations.
Accuracy = (TP + TN) / (TP + TN + FP + FN) x 100
= (19 + 17) / (19 + 17 + 11 + 3) x 100
= 72%
The accuracy of the Doppler-guided biopsy in diagnosing
prostate cancer is 72%.
2. What is the probability of having prostate cancer
among patients with a positive Doppler-guided
biopsy? Show your pertinent calculations.
PPV = TP / (TP + FP) x 100
= 19 / (19 + 11) x 100
= 63.33%
= 63%
There is a 63% chance of having prostate cancer if you
test positive for the Doppler-guided biopsy.
3. Based on the calculations that you have done, is
the Doppler-guided biopsy an ideal screening tool
or confirmatory test, or both?
The Doppler-guided biopsy is an ideal screening tool, but
not an ideal confirmatory test because it has a high
sensitivity and low number of false negatives.
4. Define the following:
a. Sensitivity
 Probability of testing positive when disease is
present
 Sensitivity = TP / TP + FN
b. Specificity
 Probability of testing negative when disease is
absent
 Specificity = TN / FP + TN
c. Positive predictive value
 Probability of having a disease when test is
positive
 Positive predictive value = TP / TP + FP

3. How can these sources of variations be d. Negative predictive value

minimized? What are the ways of enhancing the  Probability of not having a disease when test is
accuracy of measurements? negative
 Proper calibration of instruments  Negative predictive value = TN / FN + TN
 Readers are knowledgeable in operating the
instruments used e. True positives
 Consistent atmosphere in the work area (minimize  Individuals who test positive for a disease and
distractions, maintain temperature) actually have the disease
 Screening tests should be performed in order to rule
out false data, coming up with accurate and precise f. False positives
results.  Individuals who test positive for a disease but
o Accurate: measurements are close to the do not actually have the disease
target result
o Precise: results are close to one another g. True negatives
 Individuals who test negative for a disease
and do not have the disease

4 of 5
E1 T4
 Accuracy of Observations and Measurements

h. False negatives
 Individuals who test negative for a disease but
actually have the disease

i. Type 1 error / α error

 Individuals testing positive for a disease when
they do not really have the disease; false
positive

j. Type 2 error / β error

 Individuals testing negative for a disease
when they have the disease; false negative

5. Explain the influences of the prevalence of

disease on the number of true positives and true
negatives.
 Prevalence is the total number of people positive
for a disease in the population being observed.
The formula for prevalence is:

 Prevalence = TP + FN / (TP + FP + TN + FN)

 From this equation, we can see that an in order to

increase in prevalence, either we increase in TP
+ FN, or a decrease in FP and TN, or both.

REFERENCES:
1. Lecture Notes
2. Powerpoint presentation - Dr. Paneda
3. 2019A and C trans
4. Butacan. 2016. Plenary on Accuracy of observations
and measurements.

5 of 5
E1 T4