Professional Documents
Culture Documents
MHRA Presentation, October 2017 - OOS & OOT Investigations
MHRA Presentation, October 2017 - OOS & OOT Investigations
October 2017
How to use the Investigation Flowchart
Navigation:
Clicking on those Process Step boxes with a heavy border in a process flow will take
you to a new page with further detail about that investigation step.
2
Overview
Laboratory Analysis
Phase la Investigation
Phase Ib Investigation
3
Laboratory Analysis
Investigations of "Out of Specification (OOS) / Out of Trend (OOT)/ Atypical results" have to be done
in cases of:
• Batch release testing and testing of starting materials.
• In-Process Control testing: if data is used for batch calculations/decisions and if in a dossier
and on Certificates of Analysis.
• Stability studies on marketed batches of finished products and or active pharmaceutical
ingredients, on-going / follow up stability (no stress tests)
• Previous released batch used as reference sample in an OOS investigation showing OOS or
suspect results.
• Batches for clinical trials.
All solutions and reagents should be retained until all data has been second person verified as
being within the defined acceptance criteria.
Pharmacopoeia have specific criteria for additional analyses of specific tests (i.e. dissolution level
specification for S1, S2 & S3 testing; Uniformity of dosage units specification for testing of 20
additional units; Sterility Testing).
However if the sample test criteria is usually the first level of testing and a sample has to be
tested to the next level this should be investigated as it is not following the normal trend.
The OOS process is not applicable for In-process testing while trying to achieve a manufacturing
process end-point i.e. adjustment of the manufacturing process. (e.g. pH, viscosity), and for studies
conducted at variable parameters to check the impact of drift (e.g. process validation at variable
parameters).
4
OOS / OOT Result
5
Phase la Investigation
No Error found
Obvious Error
No Further Investigation
Required
6
Phase la Investigation
Definition:
Phase la investigation is to determine whether there has been a clear obvious errors
due to external circumstances such as power failure or those that the analyst has
detected prior to generating data such as spilling sample that will negate the
requirement of a Phase Ib investigation.
For microbiological analysis this may be after the analysis has been completed and
reviewed during reading of the samples.
It is expected that these issues are trended even if a laboratory investigation lb or ll was
not raised.
7
Phase la Investigation - Obvious Error
Examples
Calculation error –
analyst and supervisor to review, both initial and date correction.
Power outage –
analyst and supervisor document the event, annotate “power failure; analysis to be repeated” on
all associated analytical documentation.
Equipment failure –
analyst and supervisor document the event, annotate “equipment failure; analysis to be
repeated” cross reference the maintenance record.
Testing errors –
for example, spilling of the sample solution, incomplete transfer of a sample; the analyst must
document immediately.
for microbiology it could be growth on a plate not in the test sample area, negative or positive
controls failing.
If no error was noted, and none of the above conditions were met Phase Ib investigation must
take place.
8
Phase Ib Investigation
Assignable Cause
No Assignable Cause or Evidence
(Root cause identified)
of Error Remains Unclear
Test Data
Contact:- Production/ Invalidated Generate CAPA
QA/Contract Giver/MAH/QP Repeat Analysis
Close Investigation
9
Phase Ib Investigation - Definitions
Specification –
A specification is defined as a list of tests, references to analytical procedures, and
appropriate acceptance criteria which are numerical limits, ranges, or other criteria
for the tests described. It establishes the set of criteria to which a drug substance,
drug product or materials at other stages of its manufacture should conform to be
considered acceptable for its intended use. “Conformance to specification” means
that the drug substance and drug product, when tested according to the listed
analytical procedures, will meet the acceptance criteria. Specifications are critical
quality standards that are proposed and justified by the manufacturer and approved
by regulatory authorities as conditions of approval.
Acceptance Criteria –
Numerical limits, ranges, or other suitable measures for acceptance of the results of
analytical procedures which the drug substance or drug product or materials at other
stages of their manufacture should meet.
Internal Specification –
Are also action limits within regulatory specifications.
10
Phase Ib Investigation - Definitions
Assignable Cause –
An identified reason for obtaining an OOS or aberrant/anomalous result.
No Assignable Cause –
When no reason could be identified.
Invalidated test –
A test is considered invalid when the investigation has determined the
assignable cause.
Reportable result –
Is the final analytical result. This result is appropriately defined in the written approved
test method and derived from one full execution of that method, starting from the original
sample.
11
Phase Ib Investigation – Definitions continued
Hypothesis/Investigative Testing –
Is testing performed to help confirm or discount a possible root cause i.e what might
have happened that can be tested:- for example it may include further testing
regarding sample filtration, sonication /extraction; and potential equipment failures
etc. Multiple hypothesis can be explored.
12
Investigation by Analyst and Supervisor
For microbiological analysis where possible once a suspect result has been identified
ensure all items related to the test failure are retained such as other environmental plates,
dilutions, ampoules/vials of product, temperature data, auto-pipettes, reagents – growth
media. No implicated test environmental plates should be destroyed until the
investigation has been completed.
This initial hypothesis testing can include the original working stock solutions but should
not include another preparation from the original sample (see: re-testing)
13
Investigation by Analyst and Supervisor continued
The checklist may not be all-inclusive, but should be a good guideline to cover the
pertinent areas that need to be covered in any laboratory investigation:-
14
Investigation by Analyst and Supervisor continued
15
Investigation by Analyst and Supervisor continued
16
Phase II Investigation
Manufacturing
Investigation
Assignable No Assignable
Cause Cause Identified
Disposition Batch
17
Phase II
Conducting Failure Investigation
No Assignable Cause
OOS Results Obtained Assignable Cause Invalidate
Confirm OOS Original
Results
Report All Results
18
Unknown Cause
No Assignable Cause
Hypothesis Testing
Averaging
Re-Test
Outlier test
Re-Sampling
Stability
Microbiology
Home
19
Phase II Investigation
Phase II investigation
Conducted when the phase I investigations did not reveal an assignable laboratory
error. Phase II investigations are driven by written and approved instructions against
hypothesis. Prior to further testing a manufacturing investigation should be
started to determine whether there was a possible manufacturing root cause.
If not already notified the contract giver/MAH/QP (in accordance with the
responsibilities in the TA) should be notified along with production and QA if a
manufacturing site.
If the investigation determines analyst error all analysis using the same technique
performed by the concerned analyst should be reviewed.
20
Phase II Investigation - Definitions
Hypothesis/Investigative Testing –
Is testing performed to help confirm or discount a possible root cause i.e what might
have happened that can be tested:- for example it may include further testing regarding
sample filtration, sonication /extraction; and potential equipment failures etc. Multiple
hypothesis can be explored.
Re-Test –
Performing the test over again using material from the original sample composite, if it
has not been compromised and/or is still available. If not, a new sample will be used.
Re-sample –
A new sample from the original container where possible, required in the event of
insufficient material remaining from original sample composite or proven issue with
original sample integrity.
21
Phase II Investigation - Unknown Cause / No Assignable
Cause
Hypothesis Testing (Applicable to Phase Ia and Phase II):
Should be started as part of Phase Ia and continue into Phase II if no assignable cause
found.
This Hypothesis testing may continue from the re-measurement of the original
preparations.
Investigational testing may not be used to replace an original suspect analytical results.
It may only be used to confirm or discount a probable cause.
22
Phase II Investigation - Unknown Cause / No Assignable
Cause
If no assignable cause that could explain the results can be identified during the manufacturing
investigation or the assay failure investigation retesting may be considered. Part of the
investigation may involve retesting a portion of the original sample.
Retesting:
• Performed on the original sample not a different sample.
• Can be a 2nd aliquot from the same sample that was the source of the original failure.
• If insufficient quantity of the original sample remains to perform all further testing then the
procedure for obtaining a resample must be discussed and agreed by QA/Contract Giver/QA
equivalent. The process of obtaining the resample should be recorded within the laboratory
investigation.
• The decision to retest should be based on sound scientific judgement. The test plan must be
approved before re testing occurs.
• The minimum number of retests should be documented within the procedure and be based
upon scientifically sound principles. Any statistical review with regards to %RSD and
repeatability should relate to the values obtained during method validation (accuracy,
precision, and intermediate precision). The number of retests should be statistically valid;
papers have suggested 5, 7, or 9.
• The retests should be performed by a different analyst where possible. The second analyst
should be at least as experienced and qualified in the method as the original analyst.
23
Phase II Investigation - Unknown Cause / No
Assignable Cause
Averaging:
• The validity of averaging depends upon the sample and its purpose. Using averages can
provide more accurate results. For example, in the case of microbiological assays, the use of
averages because of the innate variability of the microbiological test system. The kinetic scan
of individual wells, or endotoxin data from a number of consecutive measurements, or with
HPLC consecutive replicate injections from the same preparation (the determination is
considered one test and one result), however, unexpected variation in replicate
determinations should trigger investigation and documentation requirements.
• Averaging cannot be used in cases when testing is intended to measure variability within the
product, such as powder blend/mixture uniformity or dosage form content uniformity.
• Reliance on averaging has the disadvantage of hiding variability among individual test results.
For this reason, all individual test results should normally be reported as separate values.
Where averaging of separate tests is appropriately specified by the test method, a single
averaged result can be reported as the final test result. In some cases, a statistical treatment
of the variability of results is reported. For example, in a test for dosage form content
uniformity, the standard deviation (or relative standard deviation) is reported with the
individual unit dose test results.
24
Phase II Investigation - Unknown Cause / No
Assignable Cause
Averaging continued:
• Consideration of the 95% Confidence Limits (CL 95% ) of the mean would show the
variability when averaging is used.
25
Phase II Investigation - Unknown Cause / No Assignable
Cause
Averaging continued:
• Consideration of using 95% Confidence Limits (CL 95% ) of the mean would show the
variability when averaging is used.
26
Phase II Investigation - Unknown Cause / No Assignable
Cause
Re-sampling:
• Should rarely occur!
• If insufficient quantity of the original sample remains to perform all further testing then the
procedure for obtaining a resample must be discussed and agreed by QA/Contract Giver/QA
equivalent. The process of obtaining the resample should be recorded within the laboratory
investigation.
• Re-sampling should be performed by the same qualified methods that were used for the
initial sample. However, if the investigation determines that the initial sampling method was
in error, a new accurate sampling method shall be developed, qualified and documented.
• Will occur when the original sample was not truly representative of the batch or there was a
documented/traceable lab error in its preparation.
27
Phase II Investigation - Unknown Cause / No Assignable
Cause
Outlier test:
• An outlier may result from a deviation from prescribed test methods, or it may be
the result of variability in the sample. It should never be assumed that the reason
for an outlier is error in the testing procedure, rather than inherent variability in the
sample being tested.
• Statistical analysis for Outlier test results can be as part of the investigation
and analysis. However for validated chemical tests with relatively small variance
and that the sample was considered homogeneous it cannot be used to justify
the rejection of data.
• While OOS guidance is not directly intended for bioassay analysis, it can be used
as a starting point for the investigation. Compendia such as the BP; PhEur and
USP, provide guidance on outliers for these types of analysis.
28
Phase II Investigation - Unknown Cause / No Assignable
Cause
Microbiological investigations:
• These are difficult to perform as the result can be 1 to 2 weeks after the analysis was
performed and may be weeks after the batch was manufactured.
• It is important to evaluate the test conditions carefully and determine what the
boundary of samples/products/manufacturing area is. It you do not determine the
boundary of the suspect results it is difficult to determine if it one or more batches
impacted.
• The laboratory and manufacturing investigations need to be in depth.
• The investigations should clearly state the hypothesis and who will be responsible for
the identified tasks.
• Are the organisms of an expected type, determine likely source – would it be likely to
be found where it was?
• Review the media – prepared in house or bought in pre-prepared, supplier history,
sterilisation history
• Equipment/utilities used – validation, maintenance and cleaning status.
• Evaluate area/environmental trends for test area and support areas.
29
Phase II Investigation - Unknown Cause / No Assignable
Cause
Microbiological investigations continued:
30
Phase II Investigation - Unknown Cause / No Assignable
Cause
Microbiological investigations continued:
• The investigation may include working closely with the manufacturing team
• During the investigation it is an advantage to go and look at where the contamination
occurred.
• Ask how relevant plant is cleaned, tested for integrity, checked for wear, checked for
material suitability and maintained at the occurrence site may reveal possible
causes.
• Where possible talk directly to the staff involved as some information may be missed
if not looked at from the chemist/ microbiologist point of view.
• Look for other documentation such as deviations and engineering notifications
around the area of concern (this is applicable to the laboratory as well as
manufacturing).
• Trending can have species drift which may also be worthy of an action limit style
investigation.
• Statistical analysis for microbiology can include lots of zero results so recovery rates
or similar may have to be used.
• If a sample is invalidated the remaining level of assurance needs to be carefully
considered, is their sufficient residual information?
• Corrective actions may be appropriate for more than one root cause.
31
Phase II Investigation - Unknown Cause / No Assignable
Cause
Stability – OOS/OOT:
Stability OOS/OOT situations should be escalated as soon as the suspect result is found. Follow the
investigation as above for Phase I and Phase II. For OOS Situations Regulatory agencies will require
notification within a short time point of discovery due to recall potential.
If abnormal results are found at any stability interval which predicts that the test results may be OOS before the
next testing interval, schedule additional testing before the next scheduled testing interval. This will help better
determine appropriate actions to be taken.
OOT
To facilitate the prompt identification of potential issues, and to ensure data quality, it is advantageous to use
objective (often statistical) methods that detect potential out-of-trend (OOT) stability data quickly.
OOT alerts can be classified into three categories to help identify the appropriate depth for an investigation. OOT
stability alerts can be referred to as:
– analytical,
– process control, and
– compliance alerts,
As the alert level increases from analytical to process control to compliance alert, the depth of investigation
should increase.
32
Phase II Investigation - Unknown Cause / No Assignable
Cause
Stability:
• A compliance alert defines a case in which an OOT result suggests the potential or likelihood
for OOS results to occur before the expiration date within the same stability study (or for other
studies) on the same product.
• An analytical alert is observed when a single result is aberrant but within specification limits
(i.e., outside normal analytical or sampling variation and normal change over time).
33
Phase III Investigation
To determine:
34
Phase III Investigation
The phase 3 investigation should review the completed manufacturing investigation and combined
laboratory investigation into the suspect analytical results, and/or method validation for possible
causes into the results obtained.
The investigation report should contain a summary of the investigations performed; and a detailed
conclusion.
For microbiological investigations ,where appropriate, use risk analysis tools to support the
decisions taken and conclusions drawn. It may not have been possible to determine the actual root
cause therefore a robust most probable root cause may have to be given.
Once a batch has been rejected there is no limit to further testing to determine the cause of
failure, so that corrective action can be taken.
The impact of OOS result on other batches, on going stability studies, validated processes
and testing procedures should be determined by Quality Control and Quality Assurance and
be documented in the conclusion, along with appropriate corrective and preventive actions.
35
Batch Disposition
Conclusion:
• If no laboratory or calculation errors are identified in the Phase I and Phase II
there is no scientific basis for invalidating initial OOS results in favour of passing
retest results. All test results, both passing and suspect, should be reported (in all
QC documents and any Certificates of Analysis) and all data has to be considered
in batch release decisions.
• If the investigation determines that the initial sampling method was inherently
inadequate, a new accurate sampling method must be developed, documented,
and reviewed and approved by the Quality Assurance responsible for release. A
consideration should be given to other lots sampled by the same method.
• An initial OOS result does not necessarily mean the subject batch fails and must
be rejected. The OOS result should be investigated, and the findings of the
investigation, including retest results, should be interpreted to evaluate the batch
and reach a decision regarding release or rejection which should be fully
documented.
36
Batch Disposition
Conclusion continued:
• In those cases where the investigation indicates an OOS result is caused by a factor
affecting the batch quality (i.e., an OOS result is confirmed), the result should be used
in evaluating the quality of the batch or lot. A confirmed OOS result indicates that the
batch does not meet established standards or specifications and should result in the
batch's rejection and proper disposition. Other lots should be reviewed to assess
impact.
• the OOS result should be given full consideration (most probable cause
determined) in the batch or lot disposition decision by the certifying QP and the
potential for a batch specific variation also needs considering.
• Any decision to release a batch, in spite of an initial OOS result that has not been
invalidated, should come only after a full investigation has shown that the OOS result
does not reflect the quality of the batch. In making such a decision, Quality
Assurance/QP should always err on the side of caution.
37
Out of Specification & Out of Trend Investigations
38