Validation and Verification of

Molecular Biology Assay

Ms. Coosje Tuijn
Ms. Zaruhi Davtyan
Dr. Arsen Zakaryan

Online, ISTC, Central Asia, South-East

May 2022 and Eastern European countries
Session objectives
At the end of this presentation participants should:
• Describe what is method validation and verification
• Identify when to perform method validation and verification
• Recognize what parameters needed for validation and
verification
• Define how to perform method validation and verification
• Define and calculate all parameters needed for validation and
verification
What is Validation and Verification?
According to ISO:
• Validation is defined as “confirmation, through the
provision of objective evidence that the requirements for a
specific intended use or application have been fulfilled”
– It examines the whole process that is being used to check that results
are correct and consistent
• Verification is defined as “verification as the confirmation,
through provision of objective evidence that specified
requirements have been fulfilled” 
– It can also be described as the confirmation of whether or not a
product (diagnostic kit or equipment) complies with a regulation,
requirement, specification, or imposed condition 
What are the Purposes of Validation and
Verification
• To quantifiably characterize system performance
• To assess potential for error
• To identify method-to-method differences
• To meet regulatory guidelines
• The method validation and verification are essential requirements
of accreditation for ISO 17025 and ISO 15189
• Validated test methods or equipment do not need any further
validation after it is done once
– Ongoing fitness for purpose is monitored through the laboratory’s
verification, quality assurance which may include personnel competency
assessment, quality control, proficiency testing, etc 
When to Do Method Validation Studies?
• Before introduction into routine use:
– Non-standard method
– Methods designed/developed in the laboratory
– Standard methods used outside intended scope and validated
methods subsequently modified
• When condition is changed under which an original
validation was done
• When the performance of existing methods has been
shown to be unsatisfactory or
• Whenever the method is changed or modified beyond the
original specification
When to Do Method Verification Studies?
• When laboratory placing a new validated commercial
method/equipment with defined performance (from
manufacturer) for routine use or
• In a case where a previously validated method is
modified and then revalidated before use
• At regular intervals to assess on-going system
performance
• When troubleshooting questions on system
performance arise
When to do validation or verification?
For example:
• If device or kit marked as IVD “for in vitro
diagnostics” the clinical laboratory must only verify
the performance specifications
• If the device or kit marked as RUO “for research use
only” the clinical laboratory should conduct validation
to establish performance specifications before
reporting the patient results
Who Should Do Validation and Verification?
• Validation of commercial assay kits as well as the
equipment used should be performed by the
manufacturer to ensure that they achieve the stated
performance. The user should obtain this information
from the manufacturer*, however
• Verification should be performed by the lab, confirming
through review (published and unpublished evaluations,
EQA data, etc.) and testing that the equipment and the
kits meet the written specification requirements
* If such documents are not available the lab should perform the
validation study
What are the Parameters for Validation and
Verification Studies of qPCR?
• Analytical sensitivity (limit of detection)
• Analytical specificity (cross-reactivity and
interference)
• Accuracy
• Precision (repeatability intra-assay and reproducibility
inter-assay)
• Linearity
• Reference Interval (for latent infections only)
 Analytical sensitivity (limit of
• detection)
Analytical sensitivity refers to the minimum number of copies in a
sample that can be measured accurately with an assay
• Analytical sensitivity is often referred to as the “limit of
detection”(LOD)
• LOD is the lowest actual concentration of analyte in a specimen
that can be consistently detected (>95% probability is commonly
used)
• LOD is expressed as a concentration (usually copies/ml; copies/ug
DNA), the lower the detectable concentration of analyte, the greater
the analytical sensitivity of the assay
• Analytical sensitivity of an assay could be different from the
diagnostic or clinical sensitivity (very low concentrations
 Analytical Sensitivity Calculation
• The LOD is generally determined in one of two ways:
– statistically, by calculating the point at which a signal can be
distinguished from background (Cq), or
– empirically, by testing serial dilutions of samples with a known
concentration of the target in the analytical range of the
expected detection limit
• The LOD is estimated as a 95% one-sided confidence limit by
calculating the mean of the blank (specimens not containing the
analyte of interest) plus 1.645 times the standard deviation of the
blank plus 1.645 times the standard deviation of a low
concentration sample(s)
– LOD = mean blank + 1.645SDblank +1.645SDlow-
concentration sample
 Analytical Sensitivity Challenges
• LOD estimates in qPCR analyses are complicated by the
logarithmic nature of Cq, because Cq is undefined when
the template concentration is zero
• When attempted, the recommended number of blank and
low-level samples to be used to establish the LOD
• It may be necessary to spike several samples whose
concentrations are in the analytical range of the expected
detection limit
• Appropriate determination and modeling of the LOD in
the qPCR is a still focus of continued research
Analytical specificity
• Analytical specificity refers to the qPCR assay
detecting the appropriate target sequence rather than
other, nonspecific targets also present in a sample
• Diagnostic specificity is the percentage of individuals
who do not have a given condition and are identified
by the assay as negative for the condition
Diagnostic specificity
• In some situations, the diagnostic specificity of a
molecular assay can be diminished without loss of
analytical specificity
• Situations that contribute to diminished diagnostic
specificity of infectious disease molecular assays include
false-positive reactions that occur because of sample
contamination or detection of nucleic acid fragments
from organisms that are not viable and are therefore not
capable of causing disease
• False-positive results can also be caused by interfering
substances or organisms that are genetically similar
Analytical specificity (cross-reactivity and
interference)
• Analytical specificity is the parameter that needs to be
determined for validation of molecular assays
• Analytical specificity refers to the ability of an assay to
detect only the intended target and that quantification of
the target is not affected by other conditions (cross-
reactivity, contamination and other)
• The two aspects of analytical specificity are:
– Cross-reactivity and
– Interference
Analytical specificity (cross-reactivity and
interference)
• Cross-reactivity – similar genetic structure, normal
flora organisms presented in the sample or organisms
that cause similar disease states or clinically relevant
coinfections should be tested

• Interference - interfering substances affecting PCR

should be tested (endogenous and exogenous substances
present in specimens have the potential to affect on
polymerase activity and interfere with or inhibit
amplification of nucleic acid)
Analytical Specificity Study Design
• Interference studies begin by compiling a list of cross-
reacting or interfering substances that have the potential
to affect the assay being evaluated
• Two approaches for conducting interference studies are:
– analyzing the effect of potentially interfering
substances added to specimens containing the analyte
of interest
– evaluating the bias of representative patient specimens
containing the potential interfering substance using
both the test method and a comparative reference
method
Analytical Specificity Calculation
• Data analysis and criteria for acceptance commonly
applied to a paired t test and a a P value
• Analysis is based on the difference between the means
of the test and control samples and the allowable error
that is clinically significant for the test
• If the calculated P value is below the threshold chosen
for statistical significance (e.g., P < 0.05), then the
influence attributed to the presence of the interfering
substance is significant
Accuracy

• Accuracy refers to the closeness of the agreement

between the results of a single measurement and the true
value of the analyte
• Accuracy is also used as a measure of how well a test
correctly identifies or excludes a condition. So, accuracy
is the proportion of true results (both true positives and
true negatives) in the distribution
Accuracy Studies
Studies,Performance
cont’d:
• A typical Accuracy Study includes:
– At least 100 different samples for method comparison study
with “gold standard”
– good distribution of sample values ranging from low to high
– same samples tested by different methods (for example
qPCR vs Gold Standard) or on both instruments
• For quantitative tests, statistical analysis of
accuracy study results typically includes:
– Regression analysis
– Bias analysis
Accuracy Studies
Studies,Performance
cont’d:
Challenges
• Molecular methods often prove to be more sensitive
than current “gold standard” methods
• Assessment of accuracy is challenging when the new
method has lower detection limits than the old method
• It is important to use well-characterized specimens or
reference material with known target values (not always
available)
• For analytes that do not have an independent test to
measure, the gold standard may be a clinical diagnosis
determined by definitive clinical methods
Regression Analysis: Plotting the Comparative
Data
Accuracy Studies Analysis

The Equation of the Line of Best Fit:

y = mx + b
m = Slope
b = Y Intercept
Accuracy Studies Analysis
• Regression Statistics Review:
– Correlation Coefficient (r) - characterizes the dispersion of
results around the line of best fit
– Slope - The “lean” of the line of best fit (proportional bias)
– Y-Intercept - the point at which the line of best fit intersects
the Y axis. (constant bias)
• Acceptability Criteria:
– Correlation Coefficient (r) - the closer to 1.0 the better
– Slope - The closer to 1.0 the better
– Y-Intercept - the closer to zero the better
 The Acceptability criteria should be set by the lab
Accuracy - qualitative tests
• For qualitative tests the following formula is applied:

Accuracy= X 100%

• Each laboratory should decide the acceptance range

(generally > 95%)
Precision
• Precision - closeness of results of replicate
measurements
• “Repeatability” and “Reproducibility” are considered
to be the extreme measures of precision and defined as
level of concordance of the individual test results for:
– Repeatability: intra-assay precision (a single run carried out
under the same conditions: same operator, reagent lots,
instrument, laboratory, time, etc.)
– Reproducibility: inter-assay precision (from one run to
another carried out under different condition: different
operators, reagent, lots, time, laboratory, etc.)
Precision Study Performance

Repeatability Reproducibility 
• same samples (high, mid • series of sample
low) measurement (high, mid,
• one lab low)  
• short period of time • different labs 

• constant conditions • different times/days  

(operator, reagent, lot, • variable conditions
equipment) (operator, reagent, lot,
equipment)
Precision Calculation
• Precision characterised in terms of the:
– Mean
– Standard deviation (SD) for Cq variance or for copy number or
concentration measurements may be used
– Coefficient of variation (CV) (not be used with Cq as values generated
from different runs are subject to inherent inter-run variation hence,
reporting inter-run Cq variation is not appropriate)
• Expressed as the percent coefficient of variation (%CV), where:
– %CV = (standard deviation of measurements / mean) x 100 
• Acceptability expectations could be set by the lab (recommended:
+/- 15%, except for lower limits where precision should not
exceed 20%, and 2 or 3 SD)
Precision vs Accuracy
Linearity*
• Linearity is defined as the determination of the linear range of
quantification for a test or test system 
• The lower limit of linearity is referred to as the lower limit of
quantification (LLOQ) and the upper limit of linearity as the
upper limit of quantification (ULOQ). Both are the boundaries
of the reportable range
• Linearity testing challenges the entire equipment calibration
range, including the extremes, and can detect problems such as
reagent or equipment deterioration earlier than quality control
or proficiency testing failures
*Reportable range and other terms, such as measuring interval, analytical
measurement range, and linear range, are used interchangeably to refer to the same
performance characteristic of quantitative assays
Linearity Study Design
• A linearity experiment involves testing a series of
samples of known concentrations or a series of known
dilutions of a highly elevated patient specimen or
standard with concentrations across the anticipated
measuring range
• The measured values are compared to the assigned
values, typically by plotting the measured values on
the y axis and the assigned values on the x axis
• The reportable range is assessed by fitting a regression
line through the points in the linear range
Linearity data plotting
ULOQ
Linear regression analysis is
commonly recommended
and accepted for linearity
data analysis
LLOQ
Revalidation

• Validation of an analytical method is a one-time

process unless the conditions under which the method
was developed have changed. Revalidation is required
if the existing method is modified (e.g., by addition of
new sample type or changes in a critical component or
reagent that may affect the assay)
How to Do Method Validation and Verification
Consult appropriate reference guides.
• CLSI evaluation document series
• UK Standards (evaluation, validation and
verification of diagnostic tests)
• Manufacturer’s package inserts, evaluation
protocol manuals and supplemental protocols
 Where to Find the Method Validation Verification
Regulations

Clinical and Laboratory Standards Institute (CLSI)

Clinical Laboratory Improvement Act (CLIA)
College of American Pathologist (CAP)
Joint Commission on Accreditation of Healthcare
Organizations (JCAHO)
UK Standards for Microbiology Investigations (Evaluations,
validations and verifications of diagnostic tests)
Papers for Molecular Biology Assay
Validation
Some References for Regulatory Guidelines for
Method Validation:
• CLSI EP12-A2 (January 2008)
• CLSI EP05-A3 (October 2014)
• CLIA 88 Final Rule - (January 2003)
• CAP Laboratory General Checklist (April 2014)
• NCCLS - C30A-2 - (Item 6.3.2)
• UK Standards Evaluations, validations and
verifications of diagnostic tests – March 2017
Objectives
At the end of this presentation participants should:
• Describe what is method validation and verification
• Identify when to perform method validation and verification
• Recognize what parameters needed for validation and
verification
• Define how to perform method validation and verification
• Define and calculate all parameters needed for validation and
verification
 Thank you for your attention!

Questions? Remarks? Discussions?

Contacts:
tuijn@iqls.net
davtian@iqls.net
zakaryan@iqls.net