Professional Documents
Culture Documents
6 ValidationTestingof IMDA300/350Systems
Know Now. Act Now. Introduction
This fact sheet provides a summary of test parameters and results for BioVigilants IMD-A 300 and IMD-A 350 systems based on USP <1223> and EP 5.1.6 validation guidelines. The IMD-A series of systems is designed to continuously monitor air and instantaneously detect the presence of both single and agglomerated microbes in real time. Testing was performed to demonstrate that the IMD-A systems are equivalent to, if not better than, the compendial microbiological method as per USP <1223> and EP 5.1.6 guidelines. Comparison of the IMD-A systems to the compendial method was gauged through the sideby-side testing of four IMD-A systems, two of each model, and three air samplers common to the pharmaceutical manufacturing environment: the SAS Super 100, MAS 100NT, and SMA. In order to stringently assess all systems, a highly homogenous, state-of-the-art aerosol test chamber was employed in order to ensure that these systems were sampling the same aerosol environment concurrently throughout the extensive test trials. BioVigilant has maintained close communications with the FDA regarding the test plan, statistics utilized, and the subsequent results obtained, now summarized in this fact sheet.
BioVigilantTestsper USP<1223>/EP5.1.6
Accuracy Precision Linearity Limit of Detection Limit of Quantification Ruggedness Robustness Range Specificity
IMDAUserTests
Follow FDAs PAT Guidance Communicate test plan to FDA and share results IQ/OQ/PQ Side-by-side testing of the IMD-A system with conventional methods
In addition to the USP <1223> and EP 5.1.6 test results, BioVigilants DMF submissions contain information that the FDA can review to approve the use of the rapid microbiological method enabled by BioVigilants systems for environmental monitoring in the manufacturing, processing, packaging or storage of drug products. Drug makers reference these filings for FDA submission applications.
Background
In the United States and Europe respectively, the USP <1223> Validation of Alternative Microbiological Methods 1 standard, and EP 5.1.6 Alternative methods 2 for control of microbiological quality standard guide the validation or implementation of alternative microbiological methods such as BioVigilants IMD-A systems. Results from testing to these standards are filed as part of BioVigilants Drug Master File (DMF) submissions to the U.S. FDA, which supplement the testing drug makers may perform to validate IMD-A systems for use in their manufacturing areas.
DMF Contents
USP <1223> test information and results Environmental & other performance tests Published information on comparative traditional methods IMD-A model overview including Technology Functionality Calibration Intended use Components
General Information Chapter <1223> Validation of Alternative Microbiological Methods. United States Pharmacopeia 32 National Formulary 27: 2009.
BioVigilants IQ/OQ/PQ
Chapter 5.1.6 Alternative methods for control of microbiological quality. European Pharmacopoeia Sixth Edition, Supplement 6.5: 2009. Information in this document is subject to change without notice. While the information contained herein is believed to be accurate and reliable, BioVigilant Systems, Inc., assumes no responsibility for errors or omissions. IMD, IMD-A, PHARMAMASTER, BIOVIGILANT, the BioVigilant logo, and the term Instantaneous Microbial Detection are the trademarks or registered trademarks of BioVigilant Systems, Inc. in the United States and/or other countries. 2011 BioVigilant Systems, Inc. All rights reserved. Printed in U.S.A.
Test Microbes
Five indicator microbes, those common to the pharmaceutical manufacturing environment, were chosen including gram negative and gram positive, vegetative bacteria, and spore-state bacteria, as noted earlier.
Test Facility
Biological challenge testing was performed at the Yamatake g-Lab facility in Fujisawa, Japan. The general laboratory area provides a Grade B/ISO 7 environment with the capabilities in place to handle the aerosol testing of BSL-2 organisms. Laboratory personnel have a broad range of experience including microbiology, biochemistry, and industrial hygiene, and specific expertise in fundamental research on bacterial culturability and viability associated with 3 biological aerosol testing.
Figure 1: Aerosol test chamber in the Yamatake g-Lab facility
Nebulizer A Salter Laboratories 8900-series nebulizer was located inside the test chamber and utilized to disseminate all bacterial suspensions during testing. Particle Counter An ISO-21501-4-compliant Kanomax 3900 particle counter was used as a reference instrument for microbial aerosol concentrations and to establish background particle count levels during testing.
Hasegawa, N. et al., A study of bacterial culturability during bioaerosol challenge test using a test chamber, Journal of Aerosol Science, 42, 6, 397-407 (2011).
BioVigilant.com
2005 W. Ruthrauff Road, Suite 151 Tucson AZ 85705 Telephone (520) 292-2342 Fax (520) 292-2365
Page 3 of 8
Air Samplers (Compendial Method) Three air samplers representing the traditional air sampling method were utilized during testing: the SAS Super 100, MAS 100NT, and SMA. These air samplers were chosen due to their prevalence in pharmaceutical manufacturing environments. IMD-A 300 and IMD-A 350 (Alternative Microbiological Method) Testing was completed with two of each IMD-A model: IMD-A 300 and IMD-A 350. The IMD-A 300 operates at 1.15 LPM, while the IMD-A 350 operates at 28.3 LPM. Both systems have a particle size detection range from 0.5 m to 10 m, and operate based on a Mie scatter detection method for particle sizing and enumeration, and intrinsic fluorescence detection for biologic classification. with the JIS 3836 Annex 2 standard for evaluation of 4 A Kanomax 3900 particle chamber uniformity. counter was utilized to obtain 10, one-minute samples at each of seven locations within the chamber, including all five sampling locations in the base of the chamber and two air-sampler sampling port locations. The results of this test are shown in Figure 2. The mean number of particles detected at each of the seven locations was within a range of 10% of the average number of particles across all sampling locations.
1.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 S1 S2 S3 S4 S5 Location S6 S7
Figure 2: Aerosol test chamber static uniformity test. The red line marks the average normalized particle counts across all seven sampling locations. The blue lines represent plus and minus 10 percent of this average value.
Cleanliness Test An evaluation of the level of particle and biologic cleanliness in the chamber was performed. It was confirmed that the chamber complied with ISO Class 4 particulate levels and contained fewer than 1 cfu/m3. Laser-Particle Visualization Evaluation A laser-particle visualization system from Shin Nippon Air Technologies was utilized to aid in the evaluation and confirmation of chamber homogeneity.
Japanese Industrial Standard: Testing methods for collection efficiency of airborne microbe samplers. JIS K 3836: 1995.
BioVigilant.com
2005 W. Ruthrauff Road, Suite 151 Tucson AZ 85705 Telephone (520) 292-2342 Fax (520) 292-2365
Page 4 of 8
comparison, alternative acceptance criteria were established. These alternate criteria were applied on an as-needed basis and are reported herein if utilized. Accuracy The USP<1223> and EP 5.1.6 Accuracy metric is used to confirm that the IMD-A systems produce results equivalent to or better than those obtained by the traditional method. As stated in EP 5.1.6 and very similarly stated in USP <1223>, at least five suspensions across the range of the test must be analyzed for each test micro-organism. As such, five different concentrations across the operational range of the traditional method were analyzed for each of the five microbes tested. Furthermore, 12 replicates were performed at each titer to obtain a statistically relevant sample size. The Accuracy metric was assessed with the following primary and alternate acceptance criteria: 1. The mean, normalized IMD-A count for each organism and dilution is greater than or equivalent to 70% of the mean, normalized recovery of the reference air sampler. If the primary criterion is not met, a statistical evaluation of the difference is performed using a T-test with a 95% confidence interval. The Specificity 1 metric was assessed based on the following acceptance criteria: 1. The IMD-A systems produce quantitative (>50% detectability of replicates) results for each of the 5 concentrations tested for each microbe, which is corroborated by some level of positive growth for each reference instrument.
Limit of Detection The Limit of Detection (LOD) metric is utilized to determine the ability of the IMD-A systems to detect the presence of low numbers of microorganisms. Both USP <1223> and EP 5.1.6 recommend an LOD concentration that results in at least 50% of samples showing growth in the compendial test. The LOD titer, thus, is defined as the lowest titer resulting in at least 50% of replicates yielding growth. This was evaluated for each air sampler and IMD-A system with the following acceptance criteria: 1. The LOD of the IMD-A system is equal to or better than the LOD of the conventional air sampler(s).
2.
Precision The USP<1223> and EP 5.1.6 Precision metric is utilized to show that the IMD-A systems repeatedly produce results that are equivalent across multiple samplings, and to compare the degree of agreement between test results on the IMD-A systems to those of the reference method through the use of relative standard deviation (RSD) measurements. As stated in USP <1223>, at least five suspensions across the range of the test should be analyzed. For each suspension at least 10 replicates should be assayed in order to be able to calculate statistically significant estimates of the standard deviation or relative standard deviation. Again, data from 12 replicates for each of five suspensions across the range of the test have been used in the evaluation of Precision. The following primary and alternate acceptance criteria were applied in the analysis of Precision: 1. The RSD of the IMD-A system is less than or equal to the RSD of the reference air sampler (RAS). If the RSDIMD-A>RSDRAS, a statistical calculation is performed to assess the relative Precision of the means between the IMD-A system and the air sampler(s).
Limit of Quantification Both USP <1223> and EP 5.1.6 define the limit of quantification (LOQ) as the lowest number of microorganisms that can be accurately counted. The LOQ is utilized to confirm that the IMD-A systems have an LOQ not greater than that of the reference method for environmental monitoring. This was assessed by comparing the lowest concentrations of microbes which could be accurately counted by the reference method and the IMD-A system to confirm that the IMD-A system was at least as sensitive at low microbial levels. 1. The LOQ of the IMD-A system is not greater than the LOQ for the conventional air sampler(s) tested, i.e. it is at least as sensitive as the traditional method.
2.
Specificity 1 As stated in USP <1223>, and similarly stated in EP 5.1.6, the qualitative Specificity 1 metric assesses the ability of a quantitative microbiological method to detect a range of microorganisms that may be present in the test article. Consequently, five indicator microbes common to the pharmaceutical manufacturing environment were chosen for testing.
Linearity The Linearity metric is employed to ensure that the IMD-A systems produce results that are proportional to the concentration of microorganisms present in a sample and equivalent to or better than results obtained by the reference method. Both USP <1223> and EP 5.1.6 recommend assessing a systems linearity by calculating the coefficient of determination, 2 R , from a linear regression analysis of data obtained. A data set including at least five concentrations and a minimum of five replicates is recommended in USP <1223>. As such, 12 replicates at five different concentrations were evaluated for each IMD-A 2 system and all three air samplers. The R values for all systems were calculated in reference to the particle counts obtained on the Kanomax particle counter.
BioVigilant.com
2005 W. Ruthrauff Road, Suite 151 Tucson AZ 85705 Telephone (520) 292-2342 Fax (520) 292-2365
Page 5 of 8
The following primary and alternate acceptance criteria were used: 1. 2.
2 The IMD-A systems R value is greater than or 2 equivalent to the reference air samplers R value.
If the IMD-A systems R2 value is less than the 2 2 reference air samplers R value, the IMD-A R value should not be less than 0.90.
Range As defined in USP <1223> and EP 5.1.6, the operational range of a quantitative microbiological method is the interval between the upper and lower levels of microorganisms that have been demonstrated to be determined with Precision, Accuracy, and Linearity. Range is determined from the data acquired for Precision, Accuracy, and Linearity, and values are reported. Robustness The Robustness test is utilized to show that the IMDA systems are capable of operating within the temperature and humidity limits tested and that the change in operating parameters has little or no effect on their performance. 1. The performance of the IMD-A systems evaluated using reference particles stays within manufacturing tolerances across the operating conditions tested.
All IMD-A passing rates are within 1% for the three air samplers tested. Additional Test Highlights
IMD- A 300 (2 systems) Accuracy, Precision, Linearity, Range, LOD, LOQ Comparison testing: 345 comparison tests each 97% overall rate of passing The majority of metrics not passed were related to Precision at low concentrations due to the disparity in flow rate between the IMD-A 300 (1.15 LPM) and SAS (100 LPM), MAS (100 LPM) and SMA (28.3 LPM) air samplers, and the low total microbial counts (low number statistics). All Robustness and Ruggedness tests passed All Specificity 1 and Specificity 2 tests passed
Ruggedness The Ruggedness assessments in USP <1223> and EP 5.1.6 are utilized to assess the repeatability in performance of like-model IMD-A systems and to confirm that the systems are intrinsically resistant to influences exerted by operational and environmental variables. In this regard, Ruggedness is defined as the degree of precision of test results, and a measure of relative standard deviation is used in the evaluation of data toward assessment of the Ruggedness metric. To provide context for interpretation, calculated RSD values may be compared to guidelines developed for the approximate expected RSD, as listed in the tables of Summary Test Results which follow. The following acceptance criterion was evaluated for Ruggedness: 1. Degree of precision of IMD-A test results obtained by analysis of the same samples under a variety of conditions is determined. The calculated RSD for datasets compared is reported.
IMD-A 350 (2 systems) Accuracy, Precision, Linearity, Range, LOD, LOQ Comparison testing: 345 comparison tests each 99% overall rate of passing All Robustness and Ruggedness tests passed All Specificity 1 and Specificity 2 tests passed
Specificity 2 The quantitative Specificity 2 test is utilized to challenge the alternative technology in a manner that would encourage false positive results. As a result, the IMD-A systems were challenged with five common pharmaceutical cleanroom materials with the potential to elicit a fluorescence response on the IMDA system and thus be considered interferents. 1. Evaluate the statistical significance between the mean biological counts observed during material testing and the biological counts observed during
Low R2 values were observed for M. lylae on all IMD-A systems and air sampler instruments tested. M. lylae appears to be susceptible, both in terms of fluorescence and viability, to stresses induced during nebulization, affecting the resulting linearity measured by all samplers. For all other organisms, R2 values 0.90 were obtained.
BioVigilant.com
2005 W. Ruthrauff Road, Suite 151 Tucson AZ 85705 Telephone (520) 292-2342 Fax (520) 292-2365
Page 6 of 8
Met/exceededacceptance criteria
IMDA300SummaryofUSP<1223>&EP5.1.6MicrobialChallengeTestingResults
IMDA300ComparisontoThreeAirSamplersOverFiveConcentrations SASSuper100(100LPM)
T1 T2 T3 T4 T5 T1
Legend:
MAS100NT(100LPM)
T2 T3 T4 T5 T1 T2
SMA
T3 T4 T5
TestFunction
Accuracy(IMD A%)
Precision (%RSD)
Specificity1
Limitof Detection
Appliestothelowest concentrationonly
Appliestothelowest concentrationonly
Appliestothelowest concentrationonly
Limitof Quantification
Appliestothelowest concentrationonly
R2IMDA300#1=1.00R2IMDA300#2=0.99 R
2 IMDA300#1= 0.97R IMDA300#2 = 0.95 2
Appliestothelowest concentrationonly
R2IMDA300#1=1.00R2IMDA300#2=0.99 R
2 IMDA300 #1 = 0.97 R IMDA300#2 = 0.95 2
Appliestothelowest concentrationonly
R2IMDA300#1=1.00R2IMDA300#2=0.99 R2IMDA300#1=0.97R2IMDA300#2=0.95 R2IMDA300#1=0.99R2IMDA300#2=0.99 R2IMDA300#1=0.68R2IMDA300#2=0.61 R2IMDA300#1=0.95R2IMDA300#2=0.94
Linearity
(Underlined valuesdonot meetcriteria)
R2IMDA300#1=0.99R2IMDA300#2=0.99 R2IMDA300#1=0.68R2IMDA300#2=0.61 R
2 IMDA300#1= 0.95R IMDA300#2 = 0.94 2
R2IMDA300#1=0.99R2IMDA300#2=0.99 R2IMDA300#1=0.68R2IMDA300#2=0.61 R
2 IMDA300 #1 = 0.93 R IMDA300#2 = 0.90 2
Range
Legend:
TestFunction
Met/exceededacceptancecriteria Didnotmeetacceptancecriteria
SummaryofUSP<1223>&EnvironmentalTestingResults
IMDA300
TestedMaterial 70%IPA,WFI TrypticSoyBroth Specificity CleanroomPaper Verification2 CleanroomTyvekGown SiliconeSpray Riboflavin NormalizedCounts Ruggedness <10[#particle/Lorbio/L] Verification 1030[#particle/Lorbio/L] (Sidebysidetesting) >30[#particle/Lorbio/L] RobustnessVerification OperationalVibration OperationalAltitude SurvivabilityEnvironmental
ApproximateExpectedRSD(%) ExperimentallyDeterminedRSD(%) <50% <43%* <35% <13% <21% <9% Operatingtemperaturerangeof1530C,andoperatinghumidityrangeof1080%RHnoncondensing Horizontalandverticalvibrationtestingsuccessfullycompletedfrom5Hzto300Hzat0.25gto1.00g Successfullytestedto abarometricpressureequivalentto6500feetinaltitude Successfullycompleted
Notes:
A single value is reported under Accuracy in instances where only one of the two IMDA 300 systems tested did not meet the acceptance criteria. Two values are representative of nonpassing values for both IMDA300systems. A70%acceptancecriteriaisrequiredinorderto meettheAccuracymetric. As with Accuracy, only nonpassing values are displayed for Precision.Two values indicate that only one IMDA system did not meet the Precision acceptance criteria, while three values indicate that both (2) IMDA300systemsdidnotmeetthePrecisionmetric.Inbothcasesthefinalnumberprovidedrepresentstheairsampler%RSDforcomparison. *ThisRSDvaluedoesnotnecessarilyapplytoverylowcountenvironments.AhigherRSDmaybeobservedinenvironmentswhereapproximately<2particles/L areobserveddueto theminimalnumberof particlespresentintheenvironmentandresultingsmallnumberstatistics.
BioVigilant.com
2005 W. Ruthrauff Road, Suite 151 Tucson AZ 85705 Telephone (520) 292-2342 Fax (520) 292-2365
Page 7 of 8
Met/exceededacceptancecriteria
IMDA350SummaryofUSP<1223>&EP5.1.6MicrobialChallengeTestingResults
IMDA350ComparisontoThreeAirSamplersOverFiveConcentrations SASSuper100(100LPM)
T1 T2 T3 T4 T5 T1
Legend:
MAS100NT(100LPM)
T2 T3 T4 T5 T1 T2
SMA
T3 T4 T5
TestFunction
Accuracy(IMD A%)
Precision (%RSD)
Specificity1
Limitof Detection
Appliestothelowest concentrationonly
Appliestothelowest concentrationonly
Appliestothelowest concentrationonly
Limitof Quantification
Appliestothelowest concentrationonly
R2IMDA350#1=1.00R2IMDA350#2=1.00 R2IMDA350#1=0.95R2IMDA350#2=0.96 R2IMDA350#1=0.99R2IMDA350#2=0.99 R R
2 2 IMDA350#1=0.48 R IMDA350#2 =0.64 IMDA350#1=0.94 R IMDA350#2 =0.93 2 2
Appliestothelowest concentrationonly
R2IMDA350#1=1.00R2IMDA350#2=1.00 R2IMDA350#1=0.95R2IMDA350#2=0.96 R2IMDA350#1=0.99R2IMDA350#2=0.99 R R
2 2 IMDA350#1=0.48 R IMDA350#2 =0.64 IMDA350#1=0.92 R IMDA350#2 =0.90 2 2
Appliestothelowest concentrationonly
R2IMDA350#1=1.00R2IMDA350#2=1.00 R2IMDA350#1=0.95R2IMDA350#2=0.96 R2IMDA350#1=0.99R2IMDA350#2=0.99 R2IMDA350#1=0.48R2IMDA350#2=0.64 R2IMDA350#1=0.94R2IMDA350#2=0.93
Linearity
Range
Legend:
TestFunction
SummaryofUSP<1223>&EnvironmentalTestingResults
TestedMaterial IMDA350 70%IPA,WFI TrypticSoyBroth Specificity CleanroomPaper Verification2 CleanroomTyvekGown SiliconeSpray AnalysisnotpossibleontheIMDA350duetothefloodingofsystemwithsiliconeparticles Riboflavin ApproximateExpectedRSD(%) NormalizedCounts ExperimentallyDeterminedRSD(%) Ruggedness <10[#particle/Lorbio/L] <15%* <50% Verification 1030[#particle/L orbio/L] <3% <35% (Sidebyside testing) >30[#particle/Lorbio/L] <1% <21% RobustnessVerification Operatingtemperaturerangeof1530C,andoperatinghumidityrangeof1080%RHnoncondensing OperationalVibration Horizontalandverticalvibrationtestingsuccessfullycompletedfrom5Hzto300Hzat0.25gto1.00g OperationalAltitude Successfullytestedtoabarometricpressureequivalentto 6500feetinaltitude SurvivabilityEnvironmental Successfullycompleted Notes: A single value is reported under Accuracy in instances where only one of the two IMDA 350 systems tested did not meet the acceptance criteria. Two values are representative of nonpassing values for both IMDA350systems. A70%acceptancecriteriaisrequiredinordertomeettheAccuracymetric. As with Accuracy, only nonpassing values are displayed for Precision.Two values indicate that only one IMDA system did not meet the Precision acceptance criteria, while three values indicate that both (2) IMDA350systemsdidnotmeetthePrecisionmetric.Inbothcasesthefinalnumberprovidedrepresentstheairsampler%RSDforcomparison. *ThisRSDvaluedoesnotnecessarilyapplyto verylowcountenvironments.AhigherRSDmaybeobservedinenvironmentswhereapproximately<2particles/Lareobservedduetotheminimalnumberof particlespresentintheenvironmentandresultingsmallnumberstatistics.
BioVigilant.com
2005 W. Ruthrauff Road, Suite 151 Tucson AZ 85705 Telephone (520) 292-2342 Fax (520) 292-2365
Page 8 of 8
B. atrophaeus
1000.00
100.00
10.00
Concentration
IMD-A 300 (#1) IMD-A 300 (#2) IMD-A 350 (#1) IMD-A 350 (#2) Kanomax
B. atrophaeus
1000.00
100.00
10.00
1.00
Concentration
IMD-A 300 (#1) IMD-A 300 (#2) IMD-A 350 (#1) IMD-A 350 (#2) SAS MAS SMA
S. epidermidis
1000.00
100.00
10.00
1.00
Concentration
IMD-A 300 (#1) IMD-A 300 (#2) IMD-A 350 (#1) IMD-A 350 (#2) SAS MAS SMA
LI007August5,2011
BioVigilant.com
2005 W. Ruthrauff Road, Suite 151 Tucson AZ 85705 Telephone (520) 292-2342 Fax (520) 292-2365