External Quality

Assessment: Trends
and Developments

Paulo Pereira, Ph.D.

 Contents
1. What is external quality asessment?
2. What are EQA principles?
3. Which require EQA?
4. What are the technical guides for EQA?
5. How important is the homogeneity of the group's results?
6. Case Study
What is external quality asessment?
• External quality assessment (EQA) is "a program in
which multiple samples are periodically sent to
members of a group of laboratories for analysis
and/or identification, in which each laboratory's
results are compared to those of other laboratories
in the group and/or to an assigned value, and the
participating laboratory and others are informed"
(CLSI H26, C24, I/LA20)
• In qualitatives the assigned value is the true result
• EQA is a matter of bias
What are EQA principles?
• Quantitative bias assessment (bias %)
• Qualitative bias assessment (false results)
• Evaluation of participants' performance
against pre-established criteria through
comparisons between laboratories
• Evaluation of analytical performance that
includes a sample for which the analyst
does not know the expected measurement
result (blind controls)
• Internal quality control is proactive
• EQA is retrospective
Which standards require EQA?
- ISO 15189 "Clinical Laboratories –
Particular Requirements for Quality and
Competence"
Sub-clause 5.6 "Quality assurance of
analysis results" - 5.6.3 Comparisons
between laboratories
- ISO 9001 "Quality Management Systems -
Requirements"
9 "Performance evaluation"
What are the technical guides for EQA?
• ISO/IEC 17043 "Conformity Assessment
- General Requirements for Proficiency
Testing" (2010)
• This International Standard specifies the
general requirements for the
competence of suppliers of proficiency
testing programs and for the
development and operation of
proficiency testing programs.
• These requirements are general for all
types of proficiency testing
programmes, and can be used as a basis
for defining specific technical
requirements for particular fields of
application.
• ISO 13528 "Statistical methods for use
in proficiency testing by comparison
between laboratories" (2015)
• Includes different mathematical models
to determine estimates used in EQA
schemes
 σ𝑛
𝑖=0 𝑥𝑖
Average: 𝑥 =
𝑛

σ𝑛
𝑖=0(𝑥𝑖 −𝑥)
2
Standard deviation: 𝑠 =
𝑛−1

𝑥−𝑥
𝑍score =
𝑠
𝑥−𝑥
% 𝐸𝑟𝑟𝑜𝑟 = ∗ 100
𝑥
1.25∗𝑠
Measurement uncertainty: 𝑢𝑥 =
𝑛
• EURACHEM "Selection, use and
interpretation of proficiency testing
schemes by laboratories" (2021)
• Free Download
https://www.eurachem.org/index.php/
publications/guides/usingpt
• Understand the basic statistics and
performance score used by EQA
providers;
• Use and interpret the results of the EQS
to improve the overall performance of
the laboratory
• EFLM (European Federation of Clinical
Chemistry and Laboratory Medicine)
"Analytical Performance Specifications for
External Quality Assessment: Definitions
and Descriptions" (2017)
• Open access article
• Specific recommendations are for medical
laboratories
 How important is
the homogeneity of the group's results?
 A. Truly biased estimates
𝑥0
Low risk of falsely biased
results
High probability of occurrence of
Number of results, n

the average

Frequency of results
Homogeneous results
𝑏0
𝑠0

Analyte concentration
Low dispersion of results
 B. Falsely biased estimates

High risk of falsely biased

results
Number of results, n

Low probability of occurrence of the

average
𝑥0

High dispersion of results Heterogeneous results

𝑏0
𝑠0

Analyte concentration
Case Study
CASE 1: Alanine aminotransferase (ALT) (IU/L)

• Performance evaluation criteria:

- If, z ≤ 2, "Satisfactory Performance"
- If, z ∈ ]2,3[, "Questionable Performance"
- If, z ≥ 3, "Unsatisfactory Performance"
• Evaluation of EQA in 6 exercises
• The criterion is common to all quantitative
CASE 1: Alanine aminotransferase (ALT) (IU/L)
Your No.
Exercise Average % Error sgroup z-score ux Performance Action
result labs
Satisfactory Maintenance of
1 39 40 2,6% 4% 1,560 10 0,641 1,6% 0,617
performance practices
Satisfactory Maintenance of
2 41 39 -4,9% 5% 2,050 12 0,976 1,8% 0,740
performance practices
Satisfactory Maintenance of
3 43 42 -2,3% 6% 2,580 11 0,388 2,3% 0,972
performance practices
Unsatisfactory Corrective maintenance
4 38 33 -13,2% 4% 1,520 14 3,289 1,3% 0,508
performance / calibration
Satisfactory Maintenance of
5 41 40 -2,4% 5% 2,050 12 0,488 1,8% 0,740
performance practices
Satisfactory Maintenance of
6 42 40 -4,8% 5% 2,100 11 0,952 1,9% 0,791
performance practices
CASE 2: Immunoassay for anti-IgG / IgM SARS-CoV-2 (binary result)

• Misclassification Index Score (MIS)

• Corresponds to the number of discrepancies in the result of an analyte
reported in the two samples sent for evaluation, with respect to the result
defined by consensus
• Performance evaluation criteria:
- If, MIS = 0 No misclassification, "good performance"
- If, MIS = 1 Misclassification of a sample result, "indeterminate
performance"
- If, MIS = 2 Misclassification of results from two samples, "poor
performance"
CASE 2: Immunoassay for anti-IgG / IgM SARS-CoV-2 (binary result)
True Your
Exercise Agreement No. labs Performance MIS
result result
Satisfactory Maintenance of practices
1 + + Yes 7 -
performance
Satisfactory
1 + + Yes 9 - Maintenance of practices
performance
Questionable Indeterminate Corrective maintenance,
2 - + NO 12
performance performance low clinical specificity
Satisfactory Indeterminate Corrective maintenance,
2 + + Yes 9
performance performance low clinical specificity
Unsatisfactory Poor Corrective maintenance,
4 + - NO 7
performance performance low clinical sensitivity
Unsatisfactory Poor Corrective maintenance,
4 + - NO 11
performance performance low clinical sensitivity
External Quality
Assessment: Trends
and Developments

Paulo Pereira, Ph.D.