Quality Control –

Introduction
The Quality System
Organization Personnel Equipment

Process
Purchasing Control Information
(QC & EQA) &
& Inventory Specimen
Management
Management

Documents Occurrence
Assessment
& Records Management

Process Customer Facilities &

Improvement Service Safety

2
The Quality Assurance Cycle

Patient/Client Prep
Sample Collection
Personnel Competency
Reporting Test Evaluations
•Data and Lab
Management
•Safety
•Customer
Service Sample Receipt
and Accessioning

Record Keeping

Quality Control Sample Transport

Testing

3
 Quality Control
• Definitions
• Qualitative Quality Control
• Quantitative QC – How to implement
ƒ Selection and managing control materials
ƒ Analysis of QC data
ƒ Monitoring quality control data

4
 What is Quality Control?
• Process or system for monitoring the quality
of laboratory testing, and the accuracy and
precision of results
• Routinely collect and analyze data from every
test run or procedure
• Allows for immediate corrective action

5
 Designing a QC Program –
• Establish written policies and procedures
ƒ Corrective action procedures
• Train all staff
• Design forms
• Assure complete documentation and review

6
 Qualitative vs.Quantitative
• Quantitative test
ƒ measures the amount of a substance
present
• Qualitative test
ƒ determines whether the substance being
tested for is present or absent

7
 Qualitative QC
• Quality control is performed for both, system
is somewhat different
• Controls available
ƒ Blood Bank/Serology/Micro
ƒ RPR/TPHA
ƒ Dipstick technology
ƒ Pregnancy

8
 Stains, Reagents, Antisera
• Label containers
ƒ contents
ƒ concentration
ƒ date prepared
ƒ placed in service
ƒ expiration date/shelf life
ƒ preparer
9
 Media Preparation
• Record amount prepared
• Source
• Lot number
• Sterilization method
• Preparation date
• Preparer
• pH
• Expiration date
10
 Microbiology QC
• Check:
ƒ Sterility
ƒ Ability to support growth
ƒ Selective or inhibitory characteristics of the medium
ƒ Biochemical response
• Frequency
ƒ Test QC organisms with each new batch or lot number
• Check for growth of fastidious organisms on media of
choice – incubate at time and temp recommended
• RECORD Results on Media QC form
11
Quality Control: Stains and Reagents
• Gram stain QC
ƒ Use gram positive and gram negative
organisms to check stain daily
• Other :
ƒ Check as used – positive and negative
reactions

12
 Stock QC organisms
• Organisms to be maintained must be
adequate to check all media and test systems.
ƒ E. coli – MacConkey, EMB, susceptibility
tests
ƒ Staphylococcus aureus – Blood agar,
Mannitol Salt, susceptibility tests
ƒ Neisseria gonorrhoeae – chocolate, Martin-
Lewis
13
 Detecting Errors
• Many organisms have predictable
antimicrobial test results
ƒ Staphylococcus spp. are usually
susceptible to vancomycin
ƒ Streptococcus pyogenes are always
susceptible to penicillin
ƒ Klebsiella pneumoniae are resistant to
ampicillin
14
 Sources of Error
• If you encounter an unusual pattern
ƒ rule out error by checking identification of
organisms
ƒ repeat antimicrobial susceptibility test

• Report if repeat testing yields same result, or refer

the isolate to a reference laboratory for confirmation

15
 Quality Control –
Quantitative Tests
How to implement a laboratory
quality control program
 Implementing a QC Program –
Quantitative Tests
• Select high quality controls
• Collect at least 20 control values over a period of
20-30
days for each level of control
• Perform statistical analysis
• Develop Levey-Jennings chart
• Monitor control values using the Levey-Jennings
chart and/or Westgard rules
• Take immediate corrective action, if needed
ƒ Record actions taken
17
 Selecting Control Materials
Calibrators
• Has a known concentration of the substance
(analyte) being measured
• Used to adjust instrument, kit, test system in
order to standardize the assay
• Sometimes called a standard, although
usually not a true standard
• This is not a control

18
 Selecting Control Materials
Controls
• Known concentration of the analyte
ƒ Use 2 or three levels of controls
ƒ Include with patient samples when
performing a test
• Used to validate reliability of the test system

19
 Control Materials
Important Characteristics
• Values cover medical decision points
• Similar to the test specimen (matrix)
• Available in large quantity
• Stored in small aliquots
• Ideally, should last for at least 1 year
• Often use biological material, consider bio-
hazardous

20
 Managing Control Materials
• Sufficient material from same lot number or
serum pool for one year’s testing
• May be frozen, freeze-dried, or chemically
preserved
• Requires very accurate reconstitution if this
step is necessary
• Always store as recommended by
manufacturer
21
 Sources of QC Samples
• Appropriate diagnostic sample
• Obtained from:
ƒ Another laboratory
ƒ EQA provider
• Commercial product

22
 Types of Control Materials
• Assayed
ƒ mean calculated by the manufacturer
ƒ must verify in the laboratory
• Unassayed
ƒ less expensive
ƒ must perform data analysis
• “Homemade” or “In-house”
ƒ pooled sera collected in the laboratory
ƒ characterized
ƒ preserved in small quantities for daily use

23
Preparing In-House Controls
 Criteria for Developing
Quality Controls for HIV
• Low positive
• Between the cut off and positive control
• At a level where variability can be followed
• Generally ~2 times the cut off

25
 Production of a QC Sample -
Production Protocol
• Materials
• Calculation of Volume
ƒstock sample
ƒ diluent
ƒ QC batch
• Method
• Validation Acceptance Criteria
ƒ batch
ƒ stability 26
 Process for Preparing
In-house Controls
• Serial dilution of high positive stock sample
• Select suitable dilution
• Produce large batch
• Test stability
• Test batch variation
• Dispense, label, store

27
 Making Suitable Dilutions
100 ul serum
in tube 1
Mix and Transfer Discard

100ul diluent in Each tube is a 1:2 dilution

each tube
of the previous tube
28
Selecting a Suitable Sample Dilution
Serial Dilutions on Abbott AxSYM HIV-1/HIV-2 MEIA
20

18

16

14
S/Co Ratio

12

10

4 Pos Cont 3.3

2 Cut Off 1.0
0
Neg Cont 0.38
2048

8192

65536
1024

4096

16384
32768
2
4
8

32
64

256

131072
262144
524288
16

128

512

Doubling Dilutions
29
 Batch Production

• Prepare positive sample

ƒ centrifuge
ƒ heat inactivate
• Mix positive sample in diluent
ƒ magnetic stirrer
• Bottle batch in numbered lots of suitable
volume
30
 Stability Testing

• Assess the rate of deterioration

QC Sample Day 7 Day 14 Day 21 Day 28
Storage

-20c 9 9 9 9

4c 9 9 9 9

16-25°C 9 9 9 9
31
 Batch Validation
• Dispense aliquots
• Test aliquots
• Confirm desired titre level
ƒ compare against target value
• Confirm minimal batch variation
ƒ acceptable if CV <20%
ƒ aim for <10%
32
 Storage of QC Samples
• Validated batch aliquoted into smaller ‘user
friendly’ volumes for storage
• Establish a storage protocol:
ƒ store at -20oC
ƒ in use vials stored at 4oC
ƒ use 0.5 ml vial maximum of one week
ƒ freeze-dried
(requires accurate reconstitution)
ƒ chemically preserved

33
34
Quality Control -Quantitative

Analysis of QC Data
 How to carry out this analysis?
• Need tools for data management and analysis
ƒ Basic statistics skills
ƒ Manual methods
ƒ Graph paper
ƒ Calculator
ƒ Computer helpful
ƒ Spreadsheet
• Important skills for laboratory personnel

36
 Analysis of Control Materials
• Need data set of at least 20 points, obtained
over a 30 day period
• Calculate mean, standard deviation,
coefficient of variation; determine target
ranges
• Develop Levey-Jennings charts, plot results

37
 Establishing Control Ranges
• Select appropriate controls
• Assay them repeatedly over time
ƒ at least 20 data points
• Make sure any procedural variation is represented:
ƒ different operators
ƒ different times of day
• Determine the degree of variability in the data to establish
acceptable range

38
 Measurement of Variability
• A certain amount of variability will naturally
occur when a control is tested repeatedly.
• Variability is affected by operator technique,
environmental conditions, and the
performance characteristics of the assay
method.
• The goal is to differentiate between
variability due to chance from that due to
error.
39
Measures of Central Tendency
• Data are frequently distributed about a
central value or a central location
• There are several terms to describe that
central location, or the ‘central tendency’
of a set of data

40
 Measures of Central Tendency
• Median = the value at the center (midpoint)
of the observations
• Mode = the value which occurs with the
greatest frequency
• Mean = the calculated average of the
values

41
 Calculation of Mean

X +X +X ... + X n
(X ) = 1 2 3
n
X = Mean
X1 = First result
X2 = Second result
Xn = Last result in series
n – Total number of results

42
 Calculation of Mean: Outliers

1. 192 mg/dL 7. 200 mg/dL

2. 194 mg/dL 8. 200 mg/dL
3. 196 mg/dL 9. 202 mg/dL
4. 196 mg/dL 10. 255 mg/dL
5. 160 mg/dL 11. 204 mg/dL
6. 196 mg/dL 12. 208 mg/dL
13. 212 mg/dL
43
 Calculation of Mean
1) 192 mg/dL
2) 194 mg/dL
3) 196 mg/dL • Mean = the calculated
average of the values
4) 196 mg/dL
• The sum of the values (X11
5) 196 mg/dL + X22 + X33 … X11
11) divided
6) 200 mg/dL by the number (n) of
7) 200 mg/dL observations
8) 202 mg/dL • The mean of these 11
9) 204 mg/dL observations is (2200 ÷
10) 208 mg/dL 11) = 200 mg/dL
11) 212 mg/dL
Sum = 2,200 mg/dL 44
 Calculation of Mean:
ELISA Tests
• Collect optical density (OD) values for controls
for each assay run
• Collect cutoff (CO) value for each run
• Calculate ratio of OD to CO (OD/CO) for each
data point or observation
ƒ This ratio standardizes data
• Use these ratio values to calculate the mean
45
 Normal Distribution
• All values are symmetrically distributed
around the mean
• Characteristic “bell-shaped” curve
• Assumed for all quality control statistics

46
 Normal Distribution

X
Frequency

4.7’ 4.8’ 4.9’ Mean 5.1’ 5.2’ 5.3’

47
 Normal Distribution

16
14
# o f O b s e rv a tio n s

Mean
12
10
8
6
4
2
0
192 194 196 198 200 202 204 206 208 210 212
Serum glucose (mg/dL)

48
 Accuracy and Precision
• The degree of fluctuation in the measurements is
indicative of the “precision” of the assay.
• The closeness of measurements to the true value
is indicative of the “accuracy” of the assay.
• Quality Control is used to monitor both the precision
and the accuracy of the assay in order to provide
reliable results.

49
 Precision and Accuracy
• Precise and inaccurate • Precise and accurate

50
Imprecise and inaccurate

51
 Measures of Dispersion
or Variability
• There are several terms that describe the
dispersion or variability of the data around
the mean:
Range
Variance
Standard Deviation
Coefficient of Variation

52
 Range
• Range refers to the difference or spread
between the highest and lowest observations.
• It is the simplest measure of dispersion.
• It makes no assumption about the shape of
the distribution or the central tendency of the
data.

53
 Calculation of Variance (S2)

2 ∑ (X − X )
S = N −1 1
2
= mg /dl
2 2

54
 Calculation of Variance
• Variance is a measure of variability about the
mean.
• It is calculated as the average squared
deviation from the mean.
ƒ the sum of the deviations from the mean,
squared, divided by the number of
observations (corrected for degrees of
freedom)
55
 Degrees of Freedom
• Represents the number of independent
data points that are contained in a data set.

• The mean is calculated first, so the

variance calculation has lost one degree of
freedom (n-1)

56
Calculation of Standard Deviation

(x − x )
S= = mg/dl
2
1

N −1

variance
57
Calculation of Standard Deviation
• The standard deviation (SD) is the square root of
the variance
ƒ it is the square root of the average squared
deviation from the mean
• SD is commonly used (rather than the variance)
since it has the same units as the mean and the
original observations
• SD is the principle calculation used in the
laboratory to measure dispersion of a group of
values around a mean
58
 Standard Deviation and Probability
• For a set of data with a
normal distribution, a value X
will fall within a range of:

Frequency
ƒ +/- 1 SD 68.2% of the
time 68.2%

ƒ +/- 2 SD 95.5% of the

time 95.5%
99.7%
ƒ +/- 3 SD 99.7% of the
time -3s- 2s -1s Mean +1s +2s +3s

59
 Standard Deviation and Probability
• In general, laboratories use the +/- 2 SD criteria for
the limits of the acceptable range for a test
• When the QC measurement falls within that range,
there is 95.5% confidence that the measurement is
correct
• Only 4.5% of the time will a value fall outside of
that range due to chance; more likely it will be due
to error

60
 Calculation of
Coefficient of Variation

• The coefficient of
variation (CV) is the
standard deviation (SD) SD
expressed as a
CV = x 100
percentage of the
mean
mean
• Ideally should be less
than 5%

61
Monitoring QC Data
 Monitoring QC Data
• Use Levey-Jennings chart
• Plot control values each run, make decision
regarding acceptability of run
• Monitor over time to evaluate the precision
and accuracy of repeated measurements
• Review charts at defined intervals, take
necessary action, and document

63
 Levey-Jennings Chart

• A graphical method for displaying control

results and evaluating whether a procedure
is in-control or out-of-control
• Control values are plotted versus time
• Lines are drawn from point to point to accent
any trends, shifts, or random excursions

64
 Levey-Jennings Chart
1 .2
-2 0 -1 5 -1 0 -5 0
-2 0 -1 5 -1 0 -5 0

+3SD
+3SD

+2SD
+2SD
0 .8

+1SD
+1SD

Mean
Mean
0 .4

-1SD
-1SD

-2SD
-2SD

-3SD
0

-3SD

65
 Levey-Jennings Chart -
Record
Record Time
Time on
on X-Axis
X-Axis and
and the
the Control
Control Values
Values on
on Y-Axis
Y-Axis
Co n tro l V alu es (e.g . m g /d L )

115
110
105
100
95
90
85
80
11 22 33 44 55 66 77 88 99 10
10 11
11 12
12 13
13 14
14 15
15 16
16 17
17 18
18 19
19 20
20 21
21 22
22 23
23 24
24

Time (e.g. day, date, run number)

66
 Levey-Jennings Chart -
Plot
Plot Control
Control Values
Values for
for Each
Each Run
Run
Co n tro l V alu es (e.g . m g /d L )

115
110
105
100
95
90
85
80
11 22 33 44 55 66 77 88 99 10
10 11
11 12
12 13
13 14
14 15
15 16
16 17
17 18
18 19
19 20
20 21
21 22
22 23
23 24
24

Time (e.g. day, date, run number)

67
 Levey-Jennings Chart
Calculate
Calculate the
the Mean
Mean and
and Standard
Standard Deviation;
Deviation;
Record
Record the
the Mean
Mean and
and +/-
+/- 1,2
1,2 and
and 33 SD
SD Control
Control Limits
Limits
115
+3SD
110
+2SD
105
+1SD
100
Mean
95
-1SD
90
-2SD
-3SD 85
80
11 22 33 44 55 66 77 88 99 10
10 11
11 12
12 13
13 14
14 15
15 16
16 17
17 18
18 19
19 20
20 21
21 22
22 23
23 24
24

Day

68
 Levey-Jennings Chart -
Record and Evaluate the Control Values
115
+3SD 115
110
110
+2SD
105
105
+1SD
Mean 100
100
95
95
-1SD
90
90
-2SD
-3SD 85
85
80
80
11 22 33 44 55 66 77 88 99 10
10 11
11 12
12 13
13 14
14 15
15 16
16 17
17 18
18 19
19 20
20 21
21 22
22 23
23 24
24

Day

69
 Findings Over Time
• Ideally should have control values clustered about
the mean (+/-2 SD) with little variation in the upward
or downward direction
• Imprecision = large amount of scatter about the
mean. Usually caused by errors in technique
• Inaccuracy = may see as a trend or a shift, usually
caused by change in the testing process
• Random error = no pattern. Usually poor technique,
malfunctioning equipment

70
Statistical Quality Control Exercise

• Hypothetical control values (2 levels of

control)
• Calculation of mean
• Calculation of standard deviation
• Creation of a Levey-Jennings chart

71
 When does the Control Value
Indicate a Problem?
• Consider using Westgard Control Rules
• Uses premise that 95.5% of control values
should fall within ±2SD
• Commonly applied when two levels of control
are used
• Use in a sequential fashion

72
 Westgard Rules
• “Multirule Quality Control”
• Uses a combination of decision criteria or
control rules
• Allows determination of whether an analytical
run is “in-control” or “out-of-control”

73
 Westgard Rules
(Generally used where 2 levels of control
material are analyzed per run)
• 12S rule • R4S rule
• 13S rule • 41S rule
• 22S rule • 10X rule

74
 Westgard – 12S Rule
• “warning rule”
• One of two control results falls outside ±2SD
• Alerts tech to possible problems
• Not cause for rejecting a run
• Must then evaluate the 13S rule

75
 12S
2S
Rule == AA warning
warning to
to trigger
trigger careful
careful inspection
inspection
of
of the
the control
control data
data
+3SD

+2SD

+1SD
12S rule
Mean violation
-1SD
-2SD
-3SD

11 22 33 44 55 66 77 88 99 10
10 11
11 12
12 13
13 14
14 15
15 16
16 17
17 18
18 19
19 20
20 21
21 22
22 23
23 24
24

Day

76
 Westgard – 13S Rule
• If either of the two control results falls outside
of ±3SD, rule is violated
• Run must be rejected
• If 13S not violated, check 22S

77
 13S
3S
Rule == Reject
Reject the
the run
run when
when aa single
single control
control
measurement
measurement exceeds
exceeds the
the +3SD
+3SD or
or -3SD
-3SD control
control limit
limit
+3SD

+2SD

+1SD

Mean
13S rule
-1SD violation
-2SD
-3SD

11 22 33 44 55 66 77 88 99 10
10 11
11 12
12 13
13 14
14 15
15 16
16 17
17 18
18 19
19 20
20 21
21 22
22 23
23 24
24

Day

78
 Westgard – 22S Rule
• 2 consecutive control values for the same
level fall outside of ±2SD in the same
direction, or
• Both controls in the same run exceed ±2SD
• Patient results cannot be reported
• Requires corrective action

79
 22S
2S
Rule == Reject
Reject the
the run
run when
when 22 consecutive
consecutive control
control
measurements
measurements exceed
exceed the
the same
same
+2SD
+2SD or
or -2SD
-2SD control
control limit
limit
+3SD

+2SD

+1SD

Mean
22S rule
-1SD violation
-2SD
-3SD

11 22 33 44 55 66 77 88 99 10
10 11
11 12
12 13
13 14
14 15
15 16
16 17
17 18
18 19
19 20
20 21
21 22
22 23
23 24
24

Day

80
 Westgard – R4S Rule
• One control exceeds the mean by –2SD, and
the other control exceeds the mean by +2SD
• The range between the two results will
therefore exceed 4 SD
• Random error has occurred, test run must be
rejected

81
 R4S
4S
Rule == Reject
Reject the
the run
run when
when 11 control
control
measurement
measurement exceed
exceed the
the +2SD
+2SD and
and the
the other
other
exceeds
exceeds the
the -2SD
-2SD control
control limit
limit
+3SD

+2SD

+1SD

Mean
R4S rule
-1SD violation
-2SD
-3SD

11 22 33 44 55 66 77 88 99 10
10 11
Day
11 12
12 13
13 14
14 15
15 16
16 17
17 18
18 19
19 20
20 21
21 22
22 23
23 24
24

82
 Westgard – 41S Rule
• Requires control data from previous runs
• Four consecutive QC results for one level of
control are outside ±1SD, or
• Both levels of control have consecutive results
that are outside ±1SD

83
 Westgard – 10X Rule
• Requires control data from previous runs
• Ten consecutive QC results for one level of
control are on one side of the mean, or
• Both levels of control have five consecutive
results that are on the same side of the mean

84
 10xx Rule == Reject
Reject the
the run
run when
when 10
10 consecutive
consecutive control
control
measurements
measurements fall
fall on
on one
one side
side of
of the
the mean
mean
+3SD

+2SD

+1SD

Mean

-1SD
-2SD
10x rule
-3SD
violation

11 22 33 44 55 66 77 88 99 10
10 11
11 12
12 13
13 14
14 15
15 16
16 17
17 18
18 19
19 20
20 21
21 22
22 23
23 24
24

Day

85
Westgard Multirule QC

86
 When a rule is violated
• Warning rule = use other rules to inspect the
control points
• Rejection rule = “out of control”
ƒ Stop testing
ƒ Identify and correct problem
ƒ Repeat testing on patient samples and controls
ƒ Do not report patient results until problem is
solved and controls indicate proper
performance
87
 Solving “out-of-control” problems
• Policies and procedures for remedial action
• Troubleshooting
• Alternatives to run rejection

88
 Summary
• Why QC program?
ƒ Validates test accuracy and reliability

89
 Summary:
How to implement a QC program?
ƒ Establish written policies and procedures
ƒ Assign responsibility for monitoring and reviewing
ƒ Train staff
ƒ Obtain control materials
ƒ Collect data
ƒ Set target values (mean, SD)
ƒ Establish Levey-Jennings charts
ƒ Routinely plot control data
ƒ Establish and implement troubleshooting and corrective
action protocols
ƒ Establish and maintain system for documentation
90