Common Issues in Qualification and

Validation of Analytical Procedures

Alexey Khrenov, PhD
OTAT/CBER/FDA
CMC Strategy Forum

January 29, 2018 - Washington, DC

 Disclaimer

• These comments are an informal

communication and represent my own best
judgment. These comments do not bind or
obligate FDA.

www.fda.gov 2
 Purpose of Qualification/Validation
• Qualification/validation is the process of
demonstrating that the analytical procedure is
suitable for its intended use.
• What does “suitable for its intended use” mean
for regulatory reviewers?
– Can we (and you) trust your past and future
data?

www.fda.gov 3
 Validation vs. Qualification
• Qualification
– Confirm that the analytical procedure (as performed) is suitable for the
specified purpose, and the results are reliable and reproducible.
– Analytical procedures must be qualified, but specifics of qualification are at
the company’s discretion based on scientific judgement.
– Qualification reports are not expected, but the company may be requested
to provide data to prove that the methods are sufficiently qualified.
• Validation
– Can be performed after gaining sufficient knowledge of the analytical
procedure.
– Formal exercise to determine the ability of the method to meet the
acceptance criteria pre-defined in the validation protocol for each of the
test parameters.
– Validated analytical procedure must be described in sufficient details for it
to be performed in FDA laboratory, and produce results comparable to those
obtained by the company.
– Validation reports must be included in the BLA/NDA. 4
www.fda.gov
 Validation vs. Qualification (contd.)
– Qualified analytical procedures should always be used in the regulated
environment (IND/BLA stage).
– Coordination of the development of analytical methods, manufacturing
process, and product is required
• Analytical procedure may need to be re-qualified or re-validated if
the acceptance criteria or matrix changes.
• If the analytical procedure is changed, bridging study needs to be
performed.
• Do NOT change the manufacturing process and analytical procedure
at the same time!
– Commercial process must be supported by fully validated analytical
procedures, i.e., the analytical procedures must be validated BEFORE the
commencement of the PPQ campaign.
– Analytical procedures used in clinical trials must be validated.
www.fda.gov 5
 Risks of Inadequate Validation
• Validation failures result in additional development work and delay
regulatory submission.
• Data excluded from review of license application
• The later the procedure is validated, the greater the risk.

Risk mitigation strategy

• Define and qualify the performance characteristics of the analytical
procedure that are critical to its intended use from the start.
• Develop system suitability procedures and criteria.
• Monitor and trend performance of analytical procedure.
• Keep gaining knowledge about capabilities of analytical procedures.
• Validate the procedure early in product development.

www.fda.gov 6
 Analytical procedure lifecycle
Development Qualification
• Initial assessment of • Assessment of the
the suitability of performance
analytical procedure characteristics of
• Optimization of analytical procedure
analytical procedure • Assessment of the
details capability of analytical
procedure to support
specification
• Gaining knowledge for
setting final specification
acceptance criteria and
design of validation
protocol
• Performance
characteristics are
inadequate to control
the product and the
manufacturing process.
www.fda.gov
Validation
• Confirmation that the
performance
characteristics of
analytical procedure
are established
correctly.
• Confirmation that
analytical procedure is
capable to support
specification
acceptance criteria.
• Failure to meet
validation acceptance
criteria
• Changes in specification
• Changes in the process
or product
• Modification of the
analytical procedure
7
Considerations for Setting Specifications
• Acceptance criteria for different
specification parameters may be limited by
different factors.
Clinical • Acceptance criteria are frequently adjusted
Risk
assessment
safety
/efficacy
when more data become available.
• Changes in the manufacturing process/
analytical procedures require re-assessment
Process Analytical
capability capability of the acceptance criteria.
• Sound statistical approaches should be used
where possible.
• Data to be included in the justification of
specification should be carefully selected
based on their relevance to the commercial
manufacturing process.
www.fda.gov 8
 Full Validation
Validation characteristics Additionally, for cross-
• Accuracy validation
• Precision • Comparability
– Repeatability
– Intermediate Precision
• Specificity
• Detection Limit
• Quantitation Limit
• Linearity
• Range
• Robustness

www.fda.gov 9
 Validation Acceptance Criteria
• Should be based on intended use, knowledge of
procedure performance, and capability to confirm
the suitability for its intended use.

• Should be specific for the analytical procedure.

• Should be specific for the validation parameter.

www.fda.gov 10
 The Big Problem
“We must do validation because FDA wants
validation report”

Leads to
• Poor design of the validation study
• Lack of scientific assessment of data
• Disconnect between validation and intended
use of the method
www.fda.gov 11
 Acceptance criteria issues
• Acceptance criteria (AC) are too wide
Risk: Analytical procedure may be found unsuitable and data excluded
from review.

Example 1 Actual variation of peak position

was 1/200 of AC
AC for peak position
Actual RSD% for peak area was
1/25 of AC

Anything will PASS!

Review of validation is based on the intended use of the analytical

procedure. Meeting an irrelevant AC does not confirm suitability.
www.fda.gov 12
 Acceptance criteria issues (contd.)
• Same AC are used for multiple validation parameters and methods
Example 2
– 1/6 of the specification range was used as the range where the
results should fall for Precision validation.
– The same value was then used to calculate the AC for standard
deviation (SD) when validating the Accuracy, Repeatability, and
Intermediate Precision for multiple methods.
– As a result, an acceptable SD equal to 1/6 of specification range (±
3 SD covered the whole specification range) was set as the AC for
multiple methods and parameters.

As the approach was defined in the SOP, it resulted in a Form 483

Observation during the facility inspection.

www.fda.gov 13
 Typical issues: Linearity
• Lack of visual inspection and qualitative assessment of the
linearity data.
Risk: Analytical procedure may be found unsuitable and data
excluded from review.
Example 3
No evidence that visual inspection was
performed of a plot.

Data show a loss of sensitivity at the lower limit

of range.

Procedure was re-validated.

Linearity should also be evaluated by visual inspection of a plot of signals as a
function of analyte concentration or content. A plot of the data should be
included. (ICH Q2)

www.fda.gov 14
 Typical issues: Range
• Validated range does not cover 80% - 120% of specification
range.
Risk: Re-validation of the procedure may be required, part of data may be
excluded from review.
Example 4
During validation, the matrix containing the analyte was spiked
with analyte solution. The analyte in the matrix was not taken into
account, and the low part of the range was not validated as
intended.

Supplemental validation was performed.

www.fda.gov 15
 Typical issues: Range (contd.)
Example 5
During validation, the dilution scheme of the standard solution used for
validation was incorrect, and an incorrect range was validated.

Supplemental validation was performed.

Example 6
After change of specification acceptance criteria, the validated range
became insufficient to support specification. The new range was not re-
validated before the submission of the license application.

The analytical procedure was re-validated during application review.

www.fda.gov 16
 Typical issues: Specificity
• Specificity is validated by showing lack of response to unrelated
or irrelevant substances.
Risk: Procedure may be found not specific enough to distinguish
between the target analyte and related substances potentially
present. Assay may be deemed unsuitable and data may be
excluded from review.

www.fda.gov 17
 Typical issues: Accuracy
• Accuracy was not validated over the entire range of the
analytical procedure.
Risk: Results at the edges of the range may not be accurate,
leading to false OOS results.

• Accuracy is validated using the number of repeats different

from the routine procedure (for the methods using replicates).
Risk: Accuracy of the assay in general is overestimated.

www.fda.gov 18
 Typical issues: Accuracy (Contd.)
• Accuracy is calculated incorrectly.
Risk: Validation exercise may fail acceptance criteria.

Example 7
For the validation of accuracy, different volumes of the standard
solution were spiked into the matrix with fixed concentration of
analyte to obtain fixed final volume.
When the expected amounts of analyte were calculated, the
changes in the amount of analyte coming from the variable
volumes of the matrix were not taken into account. As a result,
accuracy was underestimated.
Results were re-calculated and the validation report was revised.

www.fda.gov 19
 Typical issues: Precision
• For multi-step procedures variability only for final step is
assessed.
Risk: Precision of the procedure may be overestimated.
Example 8
For the validation of a procedure involving complex sample
preparation, including chromatographic separation of sampled
fractions, followed by separate analysis of each fraction, only the
data regarding the precision of final analysis were included.

Data were generated outside of the validation study, and only

provided in response to an Information Request.

www.fda.gov 20
 Typical issues: Precision (contd.)
• Validation study design and AC do not allow for identification and control of
source of variability, and establishment of suitability for intended use.
Risk: Unsuitable procedure may be validated.

Example 9
Intermediate Precision validation was designed to have two data sets:
(Analyst 1, Day 1, Instrument 1) and (Analyst 2, Day 2, Instrument 2). The
single AC for SD was set at 5.8, applicable both to each set and combined
set. Specification limit was set at 15.
SDs of 0.6 and 0.8 was obtained for each set, and an SD of 3 for the
combined set. The difference of means between the sets was ~5.
OOS/OOT results were observed in routine testing and attributed to the
variability of the test. The procedure was found to be unsuitable for its
purpose due to low Intermediate Precision.

Company performed investigation, and found the root cause for the high
variability. The procedure was modified and re-validated.
www.fda.gov 21
 Typical issues: Robustness
• Parameters used to validate robustness are not relevant or set
of parameters is not complete.
Risk: Robustness of the procedure is not established and adequate
control are not established to control for potential causes of
variability.

Example 10
During the validation of robustness, only the effect of the
concentration of reagents was assessed. The method is known to
be sensitive to temperature and time, etc.

The procedure was re-validated.

www.fda.gov 22
 Typical issues: Robustness
Example 11
During the validation of robustness for test method using a
commercially available kit, only 2 lots of the kit were used, and it
was concluded that it is not a factor affecting assay robustness.

The procedure was re-validated.

www.fda.gov 23
 Typical issues: Comparability
• Acceptance criteria and statistical analysis are inappropriate to
establish comparability.
Risk: New procedure may produce significantly different results and may
not be able to support specification.

Example 12
The company proposed to use a new method to test an existing
specification parameter. The company also proposed to maintain the
existing acceptance criteria. Comparability was established based on the
mean difference of 5.8% from testing 6 samples by both methods,
meeting the acceptance criterion of < 6%. During review, the paired t-
test was applied to the data, which produced a p-value of less than 10-6.

The supplement was withdrawn.

www.fda.gov 24
 Special Considerations for Reference
Material Based Assays
• Manufacturers are recommended to develop in-house reference
material(s) early in the development cycle to ensure reliability of
test results.
• Assay should be validated using the same reference material as
used in routine testing.
• In case of a change in the reference material, calibration of the
old material against the new one must be performed.
• Commercially available critical reagents should be qualified
using the in-house material.
• Use of reference materials provided with commercially available
kits is discouraged because their quality cannot be assured.
www.fda.gov 25
 Data Integrity Issues
Example 13
• Bridging study was performed for two methods, and the data were
submitted. The company stated that “similar profile” was obtained.

• Upon review, it was found that the data for the new method
were not fitted correctly in Excel.

www.fda.gov 26
 Data Integrity Issues (contd.)
Example 14
• A new procedure was submitted for approval along with the validation
summary. The summary stated that validation was successful with
one minor deviation. The summary included raw data for all
parameters except Intermediate Precision. The original validation
report was requested from the company. During review of report, it
was found that:
– Accuracy validation failed, supposedly because a wrong concentration
of the stock solution was used. The data were then recalculated using
the “correct” concentration to obtain acceptable results.
– Intermediate Precision validation failed significantly. Acceptance
criteria were “met” by using incorrect datasets in the calculation of
statistical parameters.

The analytical procedure was deemed unsuitable for its intended

purpose.
www.fda.gov 27
Thank you!