Performance Evaluation Plan

Autof ms1000 / Autof ms2000 / Autof ms600 / Autof ms800 /

Autof ms1600 / Autof ms2600

(Autofms11011012, Autofms11021012, Autofms11031012,

Autofms11041012, Autofms11051012, Autofms11061012)

Author(s) Approver

Name: Liu Meili Name: Zhao Gaoling

Function/Department: RA Manager Function/Department: R&D Supervisor

Signature:/Signed/ Signature: /Signed/

Date: 2022.9.3 Date: 2022.9.3

 20211201:01

Table of contents

1. Scope......................................................................................................................................2
2. General Device Information...................................................................................................2
3. General Safety and Performance Requirements.....................................................................3
4. Device Characteristics............................................................................................................5
4.1. Scientific Validity....................................................................................................5
4.2. Analytical Performance............................................................................................6
4.3. Clinical Performance................................................................................................6
4.3.1 Clinical site 1............................................................................................................6
4.3.2 Clinical site 2............................................................................................................6
4.3.3 Clinical site 3............................................................................................................6
5. Specification and Methods.....................................................................................................6
5.1 Scientific Validity....................................................................................................7
5.2 Analytical performance............................................................................................7
5.2.1. Sample preparation methods...........................................................................7
5.2.2. Quality Control...............................................................................................7
5.2.3. Precision.........................................................................................................8
5.2.4. Media and colony stability............................................................................11
5.2.5. Sample stability prior to Matrix application.................................................14
5.2.6. Sample stability post Matrix application......................................................17
5.2.7. Matrix stability..............................................................................................19
5.2.8. Carry-Over and Cross contamination...........................................................22
5.3 Clinical performance study....................................................................................24
5.3.1. Objective.......................................................................................................24
6. State of the art.......................................................................................................................24
7. Post-Market Performance Follow-up Planning....................................................................24
8. References............................................................................................................................25

/ 28
 20211201:01

1. Scope

This plan describes how the performance will be established in compliance with the IVDR
2017/746. The performance evaluation plan specifies the characteristics and the performance the
device must achieve as well as the process and criteria applied to generate the necessary
evidence.
Where any of the below-mentioned elements are not deemed appropriate due to the specific
device characteristics a justification shall be provided.

2. General Device Information

Product(s) to which the Autof ms1000 / Autof ms2000 / Autof ms600 / Autof
Performance Evaluation Plan ms800 / Autof ms1600 / Autof ms2600
applies (official product name(s)
+ catalogue number(s)) (REF 01101/ REF 01102 / REF 01103 / REF 01104/ REF
01105 / REF 01106)
Product Description The Autof ms instrument is a matrix-assisted laser
desorption/ionization coupled with time-of-flight mass
spectrometry unit intended to identify microorganisms from
isolated colonies. The technological principle of the Autof
ms1000 is based on unique ribosomal protein patterns of
microorganisms obtained via mass spectrometry. This type
of mass spectrometry involves the use of a liquid
matrix, commonly α-Cyano-4-hydroxycinnamic acid.
When ablated by the laser, the matrix absorbs energy and
the analyte molecules are ionized by proton transfer with
matrix molecules. This technique is considered soft
ionization because the matrix mediates the energy
absorption to ionize the large analyte biomolecules with
relatively no fragmentation or decomposition. The charged
ions are accelerated by an electrostatic field that transmits a
constant kinetic energy in the time of flight analyzer. The
ions arrive at the detector based on their m/z (mass-to-
charge ratio) and a spectrum is created.
The test organism's spectrum (measured mass peaks) is
compared with a reference spectra library. The spectra
library is a database of compiled m/z measurements based
on one or more reference organisms. The Autof ms system
software determines a probability ranking of the organism
identification based on the test spectrum results compared
to the library results.
Technology The
Automated Mass Spectrometry Microbial Identification
System is a mass spectrometer systems using matrix-
assisted laser desorption/ionization – time of flight
(MALDI-TOF) for the identification of microorganisms
cultured from human specimens.
The
Automated Mass Spectrometry Microbial Identification
System is a qualitative in vitro diagnostic device indicated
for use in conjunction with other clinical and laboratory
findings to aid in the diagnosis of bacterial and fungal

3 / 28
 20211201:01

infections and for identification of nucleic acid. The system

is intended for in vitro diagnostic use.

3. General Safety and Performance Requirements

The table below provides a list of the General Safety and Performance Requirements of the IVD
Regulation - Annex I section 1 to 9 that require support from relevant Scientific Validity,
Analytical and Clinical Performance Data.

Annex I General Safety and Performance Requirements In which report will

sections requiring support from relevant scientific validity, this be addressed
analytical performance and clinical performance data. (SVR, APR, CPR,
PER)
1 Devices shall achieve the performance intended by their Clinical performance
manufacturer and shall be designed and manufactured in report
such a way that, during normal conditions of use, they are Performance
suitable for their intended purpose. They shall be safe and evaluation report
effective and shall not compromise the clinical condition or
the safety of patients, or the safety and health of users or,
where applicable, other persons, provided that any risks
which may be associated with their use constitute acceptable
risks when weighed against the benefits to the patient and
are compatible with a high level of protection of health and
safety, taking into account the generally acknowledged state
of the art.
2 The requirement in this Annex to reduce risks as far as Analytical
possible means the reduction of risks as far as possible performance report
without adversely affecting the benefit-risk ratio. Clinical performance
report
Performance evaluation
report
3 Manufacturers shall establish, implement, document and Analytical
maintain a risk management system. Risk management shall performance report
be understood as a continuous iterative process throughout Clinical performance
the entire lifecycle of a device, requiring regular systematic report
updating. In carrying out risk management manufacturers Performance
shall: evaluation report
(a) establish and document a risk management plan for each Scientific validity
device; report
(b) identify and analyse the known and foreseeable hazards
associated with each device;
(c) estimate and evaluate the risks associated with, and
occurring during, the intended use and during reasonably
foreseeable misuse;
(d) eliminate or control the risks referred to in point (c) in
accordance with the requirements of Section 4;
(e) evaluate the impact of information from the production
phase and, in particular, from the post-market surveillance
system, on hazards and the frequency of occurrence thereof,
on estimates of their associated risks, as well as on the
overall risk, the benefit-risk ratio and risk acceptability; and
(f) based on the evaluation of the impact of the information
4 / 28
 20211201:01

referred to in point (e), if necessary amend control measures

in line with the requirements of Section 4.
4 Risk control measures adopted by manufacturers for the
design and manufacture of the devices shall conform to
safety principles, taking account of the generally
acknowledged state of the art. To reduce risks, the
manufacturers shall manage risks so that the residual risk
associated with each hazard as well as the overall residual Analytical
risk is judged acceptable. In selecting the most appropriate performance report
solutions, manufacturers shall, in the following order of Clinical performance
priority: report
(a) eliminate or reduce risks as far as possible through safe Performance
design and manufacture; evaluation report
(b) where appropriate, take adequate protection measures, Scientific validity
including alarms if necessary, in relation to risks that cannot report
be eliminated; and
(c) provide information for safety
(warnings/precautions/contra-indications) and, where
appropriate, training to users.
Manufacturers shall inform users of any residual risks.
5 In eliminating or reducing risks related to use error, the
manufacturer shall:
Analytical
(a) reduce as far as possible the risks related to the
performance report
ergonomic features of the device and the environment in
Clinical performance
which the device is intended to be used (design for patient
report
safety), and
Performance
(b) give consideration to the technical knowledge,
evaluation report
experience, education, training and use environment, where
Scientific validity
applicable, and the medical and physical conditions of
report
intended users (design for lay, professional, disabled or
other users).
6 The characteristics and performance of a device shall not be Analytical
adversely affected to such a degree that the health or safety performance report
of the patient or the user and, where applicable, of other Clinical performance
persons are compromised during the lifetime of the device, report
as indicated by the manufacturer, when the device is Performance
subjected to the stresses which can occur during normal evaluation report
conditions of use and has been properly Scientific validity
maintained in accordance with the manufacturer's report
instructions.
7 Devices shall be designed, manufactured and packaged in Analytical
such a way that their characteristics and performance during performance report
their intended use are not adversely affected during transport Clinical performance
and storage, for example, through fluctuations of report
temperature and humidity, taking account of the instructions Performance
and information provided by the manufacturer. evaluation report
Scientific validity
report
8 All known and foreseeable risks, and any undesirable effects Analytical
shall be minimised and be acceptable when weighed against performance report
the evaluated potential benefits to the patients and/or the Clinical performance
user arising from the intended performance of the device report

5 / 28
 20211201:01

during normal conditions of use. Performance

evaluation report
Scientific validity
report
9 Performance characteristics
9.1 Devices shall be designed and manufactured in such a way Analytical
that they are suitable for the purposes referred to in point (2) performance report
of Article 2, as specified by the manufacturer, and suitable Clinical performance
with regard to the performance they are intended to achieve, report
taking account of the generally acknowledged state of the Performance
art. They shall achieve the performances, as stated by the evaluation report
manufacturer and in particular, where applicable:
(a) the analytical performance, such as, analytical
sensitivity, analytical specificity, trueness (bias), precision
(repeatability and reproducibility), accuracy (resulting from
trueness and precision), limits of detection and quantitation,
measuring range, linearity, cut-off, including determination
of appropriate criteria for specimen collection and handling
and control of known relevant endogenous and exogenous
interference, cross- reactions; and
(b) the clinical performance, such as diagnostic sensitivity,
diagnostic specificity, positive predictive value, negative
predictive value, likelihood ratio, expected values in normal
and affected populations.
9.2 The performance characteristics of the device shall be Performance
maintained during the lifetime of the device as indicated by evaluation report
the manufacturer.
9.3 Where the performance of devices depends on the use of Performance
calibrators and/or control materials, the metrological evaluation report
traceability of values assigned to calibrators and/or control
materials shall be assured through suitable reference
measurement procedures and/or suitable reference materials
of a higher metrological order. Where available,
metrological traceability of values assigned to calibrators
and control materials shall be assured to certified reference
materials or reference measurement procedures.
9.4 The characteristics and performances of the device shall be Not applicable
specifically checked in the event that they may be affected
when the device is used for the intended use under normal
conditions:
(a) for devices for self-testing, performances obtained by
laypersons;
(b) for devices for near-patient testing, performances
obtained in relevant environments (for example, patient
home, emergency units, ambulances).

4. Device Characteristics

Performance evaluation of this device is a continuous process by which data are assessed and
analysed to demonstrate the scientific validity, analytical performance and clinical
performance of that device for its intended purpose as stated by the manufacturer.

6 / 28
 20211201:01

4.1. Scientific Validity

All claims that summarizes the goal of the scientific validity are listed as below. These claims
resemble an intended purpose statement that includes the identity of analyte(s), clinical
condition(s), test function(s), specimen type(s) and target population.

Claims
Claim 1 ： MALDI-TOF MS can give the identification results quickly with little bacteria.

4.2. Analytical Performance

Performance Characteristic Target specification

Specimen type Microorganisms cultured from human specimens
Analytical Performance
Precision The identification score of bacteria and yeast should be > 9.0
Media and colony stability The identification score of bacteria and yeast should be > 9.0
Sample stability prior to Matrix The identification score of bacteria and yeast should be > 9.0
application
Sample stability post Matrix The identification score of bacteria and yeast should be >
application 9.0. The identification result of blank spot (matrix only)
should be deliver a result of “No Spectrum”
Matrix Stability The identification score should be > 9.0
Carry-Over and Cross The identification score should be > 9.0.
contamination

4.3. Clinical Performance

4.3.1 Clinical site 1

Clinical site Henan Provincial People’s Hospital

Instrument Autof ms 1000 Microflex
Identification ATCC strains 100%(12/12) 100%(12/12)
accuracy Clinical strains 93.98% (437/465) 93.12% (433/465)
Repeatability Clinical strains 100% /

4.3.2 Clinical site 2

Clinical site Shanghai Center for Clinical Laboratory

Instrument Autof ms 1000 Microflex
Identification ATCC strains 97.67%(168/172) /
accuracy Clinical strains 100%(176/176) /
Repeatability Clinical strains 100% /

4.3.3 Clinical site 3

Clinical site Suzhou Municipal Hospital

Instrument Autof ms 1000 Microflex
Identification ATCC strains 94.23% 84.62%
accuracy Clinical strains 95.65% 94.02%
Repeatability Clinical strains 100% /

7 / 28
 20211201:01

5. Specification and Methods

Specification of methods, including the appropriate statistical tools, used for the examination of
the analytical and clinical performance of the device shall be provided as below.

5.1 Scientific Validity

The literature search will be done by using the online PubMed service.
The PubMed database and 510(k) Premarket Notification will be challenged with the following
terms:
(a) MALDI-TOF
(b) Bacterial and/or fungi infections
(c) Identification Justification:

(a) Qualitative identifying the bacteria and fungi isolated from sample by MALDI-TOF
mass spectrometry method;
(b) Aid in the diagnosis of bacterial and fungi infections;
Selection of the literature is done at three stages: based on the title, abstract and, finally, full
article. Only articles that were selected at one given stage will be evaluated in the next one.
Selection is done by evaluating the article against the exclusion criteria in a qualitative manner:
if no exclusion criterion applies, the article is retained for further evaluation.

Scientific Validity of the assay is to be demonstrated through the performance of a systematic

literature review designed in accordance with the [Systematic Literature Study Procedurexxxx].

More detailed information can be found in the device’s Scientific Validity Plan [Document
reference- Autofms11011015].

5.2 Analytical performance

Analytical performance is mainly used to Precision, Media and colony stability, Sample stability
prior to Matrix application, Sample stability post to Matrix application, Matrix stability and
Carry-Over and Cross contamination. Check if the design criteria can be met through the test
results. Many of the Data Tables listed in the following protocols are formatted as summary
tables. Organism ID and Score will be recorded for each test spot in all Protocols. In order to
simplify the explanations in the following protocols, only the Summary tables are shown.

5.2.1. Sample preparation methods

Sample preparation is completed with bacteria or fungi prior to testing with the Autof ms1000.
Each analytical study will mention which sample preparation method(s) will be used. The
preparation methods are described in the Autof ms1000 Operating Manual.

5.2.2. Quality Control

Perform Quality control every day the instrument is used, prior to performing any testing to
ensure the instrument is correctly calibrated and the reagents are acceptable to use.
1) Testing via direct transfer (DT), extended direct transfer (eDT), and extraction method
(Ext) will be performed in accordance with Autof ms1000 operating manual.
2) QC will be run for the preparation method used that day. (ex: If only DT will be

8 / 28
 20211201:01

performed that day, only QC for DT will be performed, etc.).

3) The following organisms will be used every day analytical testing is performed.
a. Staphylococcus aureus (ATCC 25923)
b. Candida albicans (ATCC 10231)
c. Pseudomonas aeruginosa (ATCC 27853)
d. Klebsiella pneumoniae (ATCC 13883)
e. Bacteroides fragilis (ATCC 25285)
4) Identify the sample with the Autof ms1000 as described in the Operating Manual.
If Autof ms1000 correctly identifies the organisms with an identification score of ≥9.00, then the
system and reagents are performing as expected and additional performance testing for the day
may be completed.

5.2.3. Precision

5.2.3.1. Purpose
The purpose of this protocol is to test the precision of Autof ms1000 using standard reference
strains.

5.2.3.2. Equipment and Materials

Equipment

No. Instrument no. Instrument name Instrument type

01 QT-690 Centrifuge Sigma centrifuge
02 JC-343 Automated mass spectrometry Autof ms1000
microbial identification system
03 A-175 Incubator PYX-DHS.500.BS-Ⅱ
04 A-163 Refrigerator BD/BC-518C
Material and reagent

Material
No. Material Lots Supplier
01 Sample Pretreatment Reagent 3 separate Autobio Diagnostics Co., Ltd.
02 Calibrator for use with AUTOF MS 3 separate Autobio Diagnostics Co., Ltd.
03 Target plate 3 separate Autobio Labtec Instruments Co.,
Ltd.
04 Absolute ethyl alcohol -- Sigma-Aldrich
05 Accessories (Inoculating loop, -- /
Micropipettor, Pipette tip )

Organisms

No. Organism Strain Organism type

01 Enterococcus faecalis ATCC 51299 Gram positive
02 Staphylococcus aureus ATCC 29213 Gram positive
03 Enterococcus faecium ATCC 19434 Gram positive
04 Streptococcus pneumoniae ATCC 10015 Gram positive
05 Clostridium beijerinckii ATCC 8260 Anaerobe
07 Escherichia coli ATCC 25922 Gram negative
08 Pseudomonas aeruginosa ATCC 27853 Gram negative
09 Acinetobacter baumannii ATCC 19606 Gram negative

9 / 28
 20211201:01

10 Aggregatibacter aphrophilus ATCC 33389 Gram negative

11 Bacteroides fragilis ATCC 25285 Anaerobe
12 Candida parapsilosis ATCC 22019 Yeast
13 Cryptococcus gattii ATCC MYA-4560 Yeast
14 Rhodotorula mucilaginosa ATCC 66034 Yeast
15 Trichosporon asahii ATCC 90039 Yeast

5.2.3.3. Procedure
Proposed Study Description:
1) Obtain appropriate organisms from the organism table above.
2) Inoculate from a frozen stock vial of each microorganism to the appropriate media (Ex: Blood
Agar) per organism, streaking for isolated colonies.
3) Incubate the media overnight at the appropriate temperature and atmospheric conditions. This
is considered the primary culture.
4) Inoculate fresh colonies from the primary culture onto another media plate.
5) Incubate the media overnight at the appropriate temperature and atmospheric conditions. This
is considered the secondary culture.
6) Prepare each organism, in triplicate, from the secondary culture using each sample
preparation method (DT, eDT and Ext) as described in the Operating manual.
7) Identify organisms with the Autof ms1000 as described in the Operating Manual.
8) Repeat Step 6 once within the same day.
9) Repeat Steps 1-7 on twelve total separate days.
a) Note: Use a separate set of material lots on days 1-4, days 5-8, and days 9-12.

5.2.3.4. Interpretation of Results

Results: Read and record the identification and score from each test.
Acceptance Criteria: The identification score of bacteria and yeast should be > 9.0. Example
Summary Data table for Enterococcus faecalis

Day Run # of “Reliable” Identification Results # of false identifications

DT method eDT method Ext method
1 A /3 /3 /3
1 B /3 /3 /3
2 A /3 /3 /3
2 B /3 /3 /3
3 A /3 /3 /3
3 B /3 /3 /3
4 A /3 /3 /3
4 B /3 /3 /3
5 A /3 /3 /3
5 B /3 /3 /3
6 A /3 /3 /3
6 B /3 /3 /3
7 A /3 /3 /3
7 B /3 /3 /3
8 A /3 /3 /3
8 B /3 /3 /3
9 A /3 /3 /3
9 B /3 /3 /3
10 A /3 /3 /3
10 B /3 /3 /3

10 / 28
 20211201:01

11 A /3 /3 /3
11 B /3 /3 /3
12 A /3 /3 /3
12 B /3 /3 /3

11 / 28
 20211201:01

5.2.4. Media and colony stability

5.2.4.1. Purpose
The purpose of this protocol is to determine the age of culture that can be used for identification
by the Autof ms1000. Eighteen microorganisms including bacteria and yeast will be tested on at
least twenty different types of agar media. After each culture has grown on each of the media
types, the colonies will be tested using all three sample preparation methods in duplicate and
tested with the Autof ms1000.
5.2.4.2. Equipment and Materials
Equipment

No. Instrument no. Instrument name Instrument type

01 QT-690 Centrifuge Sigma centrifuge
02 JC-343 Automated mass spectrometry Autof ms1000
microbial identification system
03 A-175 Incubator PYX-DHS.500.BS-II
04 A-163 Refrigerator BD/BC-518C

Material and reagent

No. Material Supplier
01 Sample Pretreatment Reagent Autobio Diagnostics Co., Ltd.
02 Calibrator for use with AUTOF MS Autobio Diagnostics Co., Ltd.
03 Absolute ethyl alcohol Sigma-Aldrich
04 Accessories(Inoculating loop, Micropipettor, /
Pipette tip )

Organisms
No. Organism Strain
01 Klebsiella pneumoniae ATCC 13883
02 Pseudomonas aeruginosa ATCC 10145
03 Escherichia coli ATCC 25922
04 Legionella pneumophila ATCC 33152
05 Bordetella pertussis ATCC 9797
06 Streptococcus agalactiae ATCC 13813
07 Staphylococcus aureus ATCC 29213
08 Enterococcus faecalis ATCC 29212
09 Nocardia spp. ATCC 31319
10 Neisseria gonorrhoeae ATCC 19424
11 Campylobacter jejuni ATCC 33291
12 Candida tropicalis ATCC 13803
13 Candida albicans ATCC 10231
14 Clostridium perfringens ATCC 13124
15 Peptostreptococcus anaerobius ATCC 27337
16 Clostridiodes (Clostridium) difficile ATCC 9689
17 Bacteroides fragilis ATCC 25285
18 Bacteroides thetaiotaomicron ATCC 29741

5.2.4.3. Procedure
1) Inoculate organism from a frozen stock vial to the appropriate media (Ex: Blood Agar) per
organism, streaking for isolated colonies.

12 / 28
 20211201:01

2) Incubate the media overnight at the appropriate temperature and atmospheric conditions.
This is considered the primary culture.
3) Inoculate fresh colonies from the primary culture onto three plates of each media type
required per organism.
4) Incubate the media overnight at the appropriate temperature and atmospheric conditions
for the organism. This is considered the secondary culture.
a) Pull the appropriate media type/organism from the incubator at the appropriate time
point.
i. Pull one plate of each media type/organism from the incubator for testing after
18 h of incubation. (Plate 1)
ii. Pull a second plate of each media type/organism from the incubator for testing
after 24 h of incubation. (Plate 2)
iii. Pull a third plate of each media type/organism from the incubator for testing
after 48 h of incubation. (Plate 3)
iv. Note: some organisms may require longer minimum incubation times such as
Legionella
spp. If so, note the time points tested for those organisms in the data table.
b) Test each organism from the appropriate secondary culture using all sample
preparation methods (DT, eDT and Ext) with the Autof ms1000 as described in the
Operating Manual.
i. Perform the test immediately following removal from the incubator. This is the
0hr test.
c) Following the initial 0 hr test, hold secondary plates at room temperature (25±2℃)
for 12 hours.
d) Following the 12 hr hold at room temperature (25±2 ℃ ), test each organism from
the appropriate secondary culture using all sample preparation methods (DT, eDT
and Ext) with the Autof ms1000 as described in the Operating Manual. This is the
12 hr test.

13 / 28
 20211201:01

5.2.4.4. Interpretation of Results

Data analysis: Read and record the identification and score from each test.
Example data table per organism and media type
Escherichia coli, BA
Testing Condition Identification Score
18hr/35±2℃, 0 hr/25 ±2℃
18hr/35±2℃, 12 hr/25 ±2℃
24hr/35±2℃, 0 hr/25 ±2℃
24hr/35±2℃, 12 hr/25 ±2℃
48hr/35±2℃, 0 hr/25 ±2℃
48hr/35±2℃, 12 hr/25 ±2℃
Note: This data table will be repeated per organism and media type.
Example summary table of results (not showing all media types)

Microorganism Correct Identification result

BA TSA MAC CHOC SAB
DT eDT Ext DT eDT Ext DT eDT Ext DT eDT Ext DT eDT Ext
Bacteroides fragilis /6 /6 /6 /6 /6 /6 /6 /6 /6
Candida albicans /6 /6 /6 /6 /6 /6 /6 /6 /6 /6 /6 /6
Candida tropicalis /6 /6 /6 /6 /6 /6 /6 /6 /6 /6 /6 /6
Enterococcus faecalis /6 /6 /6 /6 /6 /6 /6 /6 /6
Escherichia coli /6 /6 /6 /6 /6 /6 /6 /6 /6 /6 /6 /6
Klebsiella pneumoniae /6 /6 /6 /6 /6 /6 /6 /6 /6 /6 /6 /6
Legionella pneumophila /6 /6 /6 /6 /6 /6 /6 /6 /6
Neisseria gonorrhoeae /6 /6 /6 /6 /6 /6 /6 /6 /6
Nocardia spp. /6 /6 /6 /6 /6 /6 /6 /6 /6
Pseudomonas aeruginosa /6 /6 /6 /6 /6 /6 /6 /6 /6 /6 /6 /6
Staphylococcus aureus /6 /6 /6 /6 /6 /6 /6 /6 /6
Streptococcus agalactiae /6 /6 /6 /6 /6 /6 /6 /6 /6
Acceptance Criteria: The identification score of bacteria and yeast should be > 9.0 to be considered a correct identification result.

14 / 28
 20211201:01

15 / 28
 20211201:01

5.2.5. Sample stability prior to Matrix application

5.2.5.1. Purpose
This study will assess isolate stability on the target plate prior to matrix overlay via Direct
Transfer (DT) and Extended Direct Transfer (eDT) method (Phase 1). In addition, the study will
confirm the stability of extracted material (from the Ext method) prior to target plate inoculation
(Phase 2).
5.2.5.2. Equipment and Materials
Equipment

No. Instrument no. Instrument name Instrument type

01 QT-690 Centrifuge Sigma centrifuge
02 JC-343 Automated mass spectrometry Autof ms1000
microbial identification system
03 A-175 Incubator PYX-DHS.500.BS-II
04 A-163 Refrigerator BD/BC-518C

Material and reagent

No. Material Supplier
01 Sample Pretreatment Reagent Autobio Diagnostics Co., Ltd.
02 Calibrator for use with AUTOF MS Autobio Diagnostics Co., Ltd.
03 Absolute ethyl alcohol Sigma-Aldrich
04 Accessories(Inoculating loop, /
Micropipettor, Pipette tip )

Organisms
Organism Group No. Organism Strain
Gram Positive 01 Escherichia coli ATCC 25922
Gram Positive 02 Pseudomonas ATCC 10145
aeruginosa
Gram Positive 03 Klebsiella pneumoniae ATCC 13883
Gram Negative 04 Enterococcus faecalis ATCC 29212
Gram Negative 05 Enterococcus faecium ATCC 9562
Gram Negative 06 Staphylococcus aureus ATCC 29213
Yeast 07 Candida albicans ATCC 10231
Yeast 08 Candida parapsilosis ATCC 22019
Yeast 09 Saccharomyces ATCC 9763
cerevisiae

5.2.5.3. Procedure
1. Obtain appropriate organisms from the organism table above.
2. Inoculate from a frozen stock vial of each microorganism to the appropriate media (Ex:
Blood Agar) per organism, streaking for isolated colonies.
3. Incubate the media overnight at the appropriate temperature and atmospheric conditions. This
is considered the primary culture.
4. Inoculate fresh colonies from the primary culture onto three plates of each media listed in the
organism table listed above.
5. Incubate the media overnight at the appropriate temperature and atmospheric conditions for
the organism. This is considered the secondary culture.
6. Phase 1
1) DT Method

16 / 25
 20211201:01

i. Smear each organism to 15 spots on the target plate, as described in the

Autof ms1000 Operating Manual.
ii. Overlay 3 spots with matrix for each of five time points following plating
(0min, 15 min, 30 min, 60 min, and 120 min).
iii. After the matrix overlay has dried, identify the organism as described in the
Autof ms1000 Operating Manual.
2) eDT method
i. Smear each organism to 15 spots on the target plate, followed by Lysate 1,
as described in the Autof ms1000 Operating Manual.
ii. Overlay 3 spots with matrix for each of five time points following plating
(0min, 15 min, 30 min, 60 min, and 120 min).
iii. After the matrix overlay has dried, identify the organism as described in the
Autof ms1000 Operating Manual.
3) Ext Method
i. Extract each organism, in duplicate, following the extraction method as
described in the Autof ms1000 Operating Manual.
ii. Plate each extract to 15 spots on the target plate.
iii. Overlay 3 spots of each extract with matrix for each of five time points
following plating (0min, 15 min, 30 min, 60 min, and 120 min).
iv. After the matrix overlay has dried, identify the organism as described in the
Autof ms1000 Operating Manual.
v. Use the prepared extracts in Phase 2, below.
7. Phase 2
i. Hold the extracts prepared in step 6.3)i for 24 hours at room temperature.
ii. Repeat steps 6.3)ii-iv., except for three spots of each extract will instead be
overlaid with matrix at five different time points following plating: 0 h, 1 h,
4 h, 8 h, and 24 h.
5.2.5.4. Interpretation of Results
Example Results Table for sample stability prior to the addition of Matrix (Phase 1): Read and
record the identification and score per test. Note: this table should be repeated for each
organism.
Time Before
Method Matrix Correct ID False ID
Application
0 min /3
15 min /3
Direct Transfer (DT) 30 min /3
60 min /3
120 min /3
0 min /3
15 min /3
Extended Direct Transfer
30 min /3
(eDT)
60 min /3
120 min /3
0 min /6
15 min /6
Extraction (EXT) 30 min /6
60 min /6
120 min /6

Example of Results Table for extracted sample stability at room temperature for 24 hours prior

17 / 25
 20211201:01

plating to the target plate (Phase 2): Read and record the identification and score per test.
Time before
Method matrix Correct ID False ID
application
0 hr /6
1 hr /6
Extraction (EXT) after 24
4 hr /6
hours at room temperature
8 hr /6
24hr /6
Note: This table will be repeated for each organism.
Acceptance Criteria: The identification score of bacteria and yeast should be > 9.0.

18 / 25
 20211201:01

5.2.6. Sample stability post Matrix application

5.2.6.1. Purpose
The purpose of this protocol is to test the sample stability after Matrix application. This study
will assess the stability of test microorganisms on the spotted target plate following matrix
addition at various temperature and relative humidity conditions. In addition, the study will
serve to prove that matrix alone will not result in an Autof ms1000 identification.
5.2.6.2. Equipment and Materials
Equipment

No. Instrument no. Instrument name Instrument type

01 QT-690 Centrifuge Sigma centrifuge
02 JC-343 Automated mass Autof ms1000
spectrometry microbial
identification system
03 A-175 Incubator PYX-DHS.500.BS-II
04 A-163 Refrigerator BD/BC-518C

Material and reagent

No Material Supplier
.
01 Sample Pretreatment Reagent Autobio Diagnostics Co.,
Ltd.
02 Calibrator for use with AUTOF Autobio Diagnostics Co.,
MS Ltd.
03 Absolute ethyl alcohol Sigma-Aldrich
04 Accessories(Inoculating loop, /
Micropipettor, Pipette tip )

Organisms
Organism Group No. Organism Strain
Gram Negative 01 Escherichia coli ATCC 25922
Gram Negative 02 Pseudomonas aeruginosa ATCC 10145
Gram Negative 03 Klebsiella pneumoniae ATCC 13883
Gram Positive 04 Enterococcus faecalis ATCC 29212
Gram Positive 05 Enterococcus faecium ATCC 9562
Gram Positive 06 Staphylococcus aureus ATCC 29213
Yeast 07 Candida albicans ATCC 10231
Yeast 08 Candida parapsilosis ATCC 22019
Saccharomyces
Yeast 09 ATCC 9763
cerevisiae

5.2.6.3. Procedure
1. Obtain appropriate organisms from the organism table above.
2. Inoculate from a frozen stock vial of each microorganism to the appropriate media (Ex:
Blood Agar) per organism, streaking for isolated colonies.
3. Incubate the media overnight at the appropriate temperature and atmospheric conditions.
This is considered the primary culture.
4. Inoculate fresh colonies from the primary culture onto three plates of each media listed in
the organism table listed above.
5. Incubate the media overnight at the appropriate temperature and atmospheric conditions for

19 / 25
 20211201:01

the organism. This is considered the secondary culture.

6. From the secondary culture, use all three (DT, eDT, and Ext) sample preparation methods as
described in the Autof ms1000 Operating Manual.
7. Inoculate each sample preparation to four spots on six separate target plates.
8. Add Matrix to four blank spots on each of the six target plates.
9. Identify organisms with the Autof ms1000 as described in the Operating Manual. This is the
0h identification.
10. Hold each plate at one of the following conditions:
1) 20 ± 1°C, 30 ± 5% relative humidity
2) 20 ± 1°C, 40 ± 5% relative humidity
3) 20 ± 1°C, 70 ± 5% relative humidity
4) 25 ± 1°C, 30 ± 5% relative humidity
5) 25 ± 1°C, 40 ± 5% relative humidity
6) 25 ± 1°C, 70 ± 5% relative humidity
11. Identify organisms with the Autof ms1000 as described in the Operating Manual at 4±1 h,
8±1 h, and 24±1 h.
12. Following identification at each time point, return the target plate to the appropriate
temperature/humidity condition.
5.2.6.4. Interpretation of Results
Results: Read and record the identification and score from each test.
Acceptance Criteria: The identification score of bacteria and yeast should be > 9.0. The
identification result of blank spot (matrix only) should be deliver a result of “No Spectrum”.
Example results table

Identification result and Score

Temperature 20±1°C
Relative humidity 40±5% 70±5%
Sample EC PSA KP Matri EC PSA KP Matrix
x
Metho Time Replicate
d
DT 0 1
/eDT/ hour 2
Ext 3
4
4 1
hours 2
3
4
8 1
hours 2
3
4
24 1
hours 2
3
Identification result and Score
Temperature 20±1°C
Relative humidity 40±5% 70±5%
Sample EC PSA KP Matri EC PSA KP Matrix
x

20 / 25
 20211201:01

Metho Time Replicate

d
4

21 / 25
 20211201:01

5.2.7. Matrix stability

5.2.7.1. Purpose
The purpose of this protocol is to test the Matrix stability by assessing the accelerated/shipping
stability, real time stability and reconstitution stability of the matrix with Escherichia coli,
Staphylococcus aureus and Candida albicans.
5.2.7.2. Equipment and Materials
Equipment

No. Instrument no. Instrument name Instrument type

01 QT-690 Centrifuge Sigma centrifuge
02 JC-343 Automated mass spectrometry Autof ms1000
microbial identification system
03 A-175 Incubator PYX-DHS.500.BS-II
04 A-163 Refrigerator BD/BC-518C

Material and reagent

No. Material Supplier
01 Sample Pretreatment Reagent Autobio Diagnostics Co.,
Ltd.
02 Calibrator for use with Autof ms1000 Autobio Diagnostics Co.,
Ltd.
03 Absolute ethyl alcohol Sigma-Aldrich
04 Accessories(Inoculating loop, /
Micropipettor, Pipette tip )

Organisms
No. Organism Strain
01 Escherichia coli ATCC 25922
02 Staphylococcus aureus ATCC 29213
03 Candida albicans ATCC 10231

5.2.7.3. Procedure
1. Bring matrix to room temperature and reconstitute based on the following study guidelines:
1) Accelerated/shipping stability: store seven vials from three lots of un-reconstituted matrix
at 35±2°C. Bring the matrix to room temperature and reconstitute as described in the Matrix
IFU at 0 day, 3 weeks, 6 weeks, 9 weeks, 12, weeks, 15 weeks, and 18 weeks.
2) Real time stability: store six vials from three lots of un- reconstituted matrix at 2-8°C.
Reconstitute the matrix as described in the Matrix IFU at 0 months, 3 months, 6 months, 9
months, 12 months, and 18 months.
3) Reconstitution stability: Reconstitute two vials from each of the three lots of matrix and
store one vial at 20±1°C and one vial at at 25±1°C. Test the reconstituted matrix (step 2 and
3) after 0 days, 1 day, 3 days, and 7 days at each storage temperature.
4) Reconstitution at stressed temperature: Reconstitute two vials from each of the three lots
of matrix and store one vial of reconstituted matrix at 15±1°C and one vial at 30±1°C. Test
the reconstituted matrix (step 2 and 3) after 0 hours, 1 hours, 4 hours, 8 hours, and 12 hours.

22 / 25
 20211201:01

2. For each time point for testing the reconstituted matrix, prepare the three freshly cultured
microorganisms in duplicate per method (DT, eDT, and Ext) as described in the Autof
ms1000 Operating Manual.
3. Identify the organisms, as described in the Autof ms1000 Operating Manual.

23 / 25
 20211201:01

5.2.7.4. Interpretation of Results

Read and record the identification and score from each test.
Acceptance Criteria: The identification score should be > 9.0 and the identification result should
be Escherichia coli, Staphylococcus aureus and Candida albicans.
Shipping/Accelerated Stability Study Results Table:

Identification result
Lot # Escherichia coli Staphylococcus aureus Candida albicans
DT eDT EXT DT eDT EXT DT eDT EXT
Lot 1
Weeks Lot 2
Lot 3

Real Time Stability Study Results Table:

Time Lot # Identification result
Escherichia coli Staphylococcus aureus Candida albicans
DT eDT EXT DT eDT EXT DT eDT EXT
Month Lot 1
Lot 2
Lot 3

Reconstitution Stability Study Results Table

Time Lot # Identification result
Escherichia coli Staphylococcus aureus Candida albicans
DT eDT EXT DT eDT EXT DT eDT EXT
Day Lot 1
Lot 2
Lot 3

Reconstitution at stressed temperature Stability Study Results Table

Time Lot # Identification result
Escherichia coli Staphylococcus aureus Candida albicans
DT eDT EXT DT eDT EXT DT eDT EXT
Hour Lot 1
Lot 2
Lot 3

24 / 25
 20211201:01

5.2.8. Carry-Over and Cross contamination

5.2.8.1 Purpose

The purpose of this protocol is to verify there is no risk of carry-over and cross contamination,
defined as microbial sample convergence between adjacent blank target spots.
5.2.8.2 Equipment and Materials

Equipment

No. Instrument Instrument name Instrument

no. type
01 QT-690 Centrifuge Sigma
centrifuge
02 JC-343 Automated mass spectrometry microbial identification system Autof
ms1000
03 A-175 Incubator PYX-
DHS.500.
BS-II
04 A-163 Refrigerator BD/BC-
518C

Material and reagent

No. Material Supplier
01 Sample Pretreatment Reagent Autobio Diagnostics Co., Ltd.
02 Calibrator for use with AUTOF MS Autobio Diagnostics Co., Ltd.
03 Absolute ethyl alcohol Sigma-Aldrich
04 Accessories /
(Inoculating
loop, Micropipettor, Pipette tip )

Organisms
No. Organism Strain
01 Escherichia coli ATCC 25922
02 Klebsiella pneumoniae ATCC 13883

5.2.8.3 Procedure

1. Inoculate four spots, for each of the organism preparation methods (DT, eDT, and Ext) as
described in the Autof ms1000 Operating Manual, on each of two separate target plates, as
described in Layout-1, below.
2. Apply Matrix to four blank spots that do not contain microorganisms.

Target Plate organism layout 1 (Before Cleaning)

1 2 3 4 5 6 7 8 9 10 11 12
A KP- KP- KP- KP- KP-e KP-e KP-e KP-e KP- KP- KP- KP-
DT DT DT DT DT DT DT DT Ext Ext Ext Ext
B EC-DT EC-DT EC-DT EC-DT EC- EC- EC- EC- EC- EC- EC- EC-
eDT eDT eDT eDT Ext Ext Ext Ext
C Mat Mat Mat Mat
r ix r ix r ix r ix

25 / 25
 20211201:01

only only only only

3. Identify the organisms, as described in the Autof ms1000 Operating Manual.

4. Clean the two target plates with the specified cleaning procedure.
5. Inoculate four spots, for each of the organism preparation methods (DT, eDT, and Ext) as
described in the Autof ms1000 Operating Manual, on each of two separate target plates, as
described in Layout-2.

Target Plate organism layout 2 (After cleaning)

1 2 3 4 5 6 7 8 9 10 11 12
A EC- EC- EC- EC- EC- EC- EC- EC- EC-e EC-e EC-e EC-e
Ext Ext Ext Ext DT DT DT DT DT DT DT DT
B KP- KP- KP- KP- KP- KP- KP- KP- KP-e KP-e KP-e KP-e
Ext Ext Ext Ext DT DT DT DT DT DT DT DT
C Matr Matr Matr Matr
ix ix ix ix
only only only only

6. Identify the organisms, as described in the Autof ms1000 Operating Manual.

7. Repeat steps 4-6 using Layout 3

Target Plate organism layout 3 (After cleaning)

1 2 3 4 5 6 7 8 9 10 11 12
A KP-e KP-e KP-e KP-e KP KP- KP- KP- KP- KP- KP- KP-
DT DT DT DT - Ext Ext Ext DT DT DT DT
Ext
B EC-e EC-e EC-e EC-e EC EC- EC- EC- EC- EC- EC- EC-
DT DT DT DT - Ext Ext Ext DT DT DT DT
Ext
C Matr Matr Matr Matr
ix ix ix ix
only only only only

8. Repeat steps 4-6 using Layout 4

Target Plate organism layout 4 (After cleaning)
1 2 3 4 5 6 7 8 9 10 11 12
A EC- EC- EC- EC- EC-e EC-e EC-e EC-e EC- EC- EC- EC-
DT DT DT DT DT DT DT DT Ext Ext Ext Ext
B KP- KP- KP- KP- KP-e KP-e KP-e KP-e KP- KP- KP- KP-
DT DT DT DT DT DT DT DT Ext Ext Ext Ext
C Matr Matr Matr Matr
ix ix ix ix
only only only only

9. Repeat steps 4-6 using Layout 5

Target Plate organism layout 5 (After cleaning)
1 2 3 4 5 6 7 8 9 10 11 12
A KP- KP- KP- KP- KP- KP- KP- KP- KP-e KP-e KP-e KP-e
Ext Ext Ext Ext DT DT DT DT DT DT DT DT
B EC- EC- EC- EC- EC- EC- EC- EC- EC-e EC-e EC-e EC-e

26 / 25
 20211201:01

Ext Ext Ext Ext DT DT DT DT DT DT DT DT

C Mat Mat Mat Mat
r ix r ix r ix r ix
only only only only

10. Repeat steps 4-6 using Layout 6

Target Plate organism layout 2 (After cleaning)
1 2 3 4 5 6 7 8 9 10 11 12
A EC- EC- EC- EC- EC- EC- EC- EC- EC-e EC-e EC-e EC-e
Ext Ext Ext Ext DT DT DT DT DT DT DT DT
B KP- KP- KP- KP- KP- KP- KP- KP- KP-e KP-e KP-e KP-e
Ext Ext Ext Ext DT DT DT DT DT DT DT DT
C Matr Matr Matr Matr
ix ix ix ix
only only only only
5.2.8.4 Interpretation of Results

Results: Read and record the identification and score from each test. Acceptance Criteria: The
identification score should be > 9.0.
Example Results Tables (one table per layout)

Identification result before cleaning (Layout #)

Target Target Plate 1 Target Plate 2
plate DT eDT Ext Matrix DT eDT Ext Matrix
positio E K E K E K E K E KP EC KP
n C P C P C P C P C
1
2
3
4
5.3 Clinical performance study

5.3.1. Objective

The objective of this study is to verify the performance of Autof ms1000 in an independent
clinical laboratory setting. The goal is to support the claim that Autof ms1000 is both safe and
effective for use in identification of microorganisms.
186 species of ATCC strains and 354 clinical species were studied in parallel with the reference
device in 3 clinical sites. Identification accuracy and repeatability of identification of Autof ms
shall be calculated for ATCC strains and clinical species respectively.
6. State of the art

Comparing current state of the

Existing relevant standards, Common Specifications, guidance or best practices documents that
are relevant for the performance evaluation of Autof ms 1000 will be identified once possible.

7. Post-Market Performance Follow-up Planning

Performance data from users in both domestic and overseas marketing area, any changes of the
device that may influence the performance of the device and performance date from similar
devices on the market will be analysed and reviewed. PMPF plan and report will be updated for

27 / 25
 20211201:01

every year for Autof ms 1000.

A PMPF Plan is written in accordance with the [RA-SMP011 Post-Market Performance Follow-
Up Procedure].
More detailed information can be found in the device’s PMPF Plan [Document reference
MS0001-28].

8. References

Following QMS documents were made and following standards, guidelines were referenced
within the plan throughout the performance evaluation of the device.

Document reference Document name

ZH-SMP-036 SMP of performance evaluation for IVDR devices
ZH-SMP-037 SMP of clinical evaluation for IVDR devices
SMP of European medical device vigilance system management for
ZH-SMP-022
IVDR devices
ZG-SMP-104 SMP of PMS management

Following standards and guidance will be referenced for performance evaluation of this assay.

No. Standards/guidance name

1 EN ISO 13485:2016
2 EN ISO 14971:2019
3 EN ISO 18113-1:2011
4 EN ISO 18113-2:2011
5 EN ISO 15223-1:2016
6 EN 13612:2002
7 EN ISO 23640:2015
8 Regulation (EU) 2017/746
9 IEC 61010-1:2010
10 IEC 61010-2-101:2018
11 IEC 60825-1
12 BS EN 61326-1:2013
13 BS EN 61326-2-6:2013
14 IEC 62304:2015
15 IEC/TR 80001-2-3:2012

28 / 25