Health Policy

FDA-cleared articial intelligence and machine learning-

based medical devices and their 510(k) predicate networks
Urs J Muehlematter, Christian Bluethgen, Kerstin N Vokinger

The US Food and Drug Administration is clearing an increasing number of articial intelligence and machine learning Lancet Digit Health 2023;
(AI/ML)-based medical devices through the 510(k) pathway. This pathway allows clearance if the device is substantially 5: e618–26

equivalent to a former cleared device (ie, predicate). We analysed the predicate networks of cleared AI/ML-based Institute for Diagnostic and
Interventional Radiology
medical devices (cleared between 2019 and 2021), their underlying tasks, and recalls. More than a third of cleared
(U J Muehlematter MD,
AI/ML-based medical devices originated from non-AI/ML-based medical devices in the rst generation. Devices with C Bluethgen MD) and
the longest time since the last predicate device with an AI/ML component were haematology (2001), radiology (2001), Department of Nuclear
and cardiovascular devices (2008). Especially for devices in radiology, the AI/ML tasks changed frequently along the Medicine (U J Muehlematter),
University Hospital Zurich and
device’s predicate network, raising safety concerns. To date, only a few recalls might have aected the AI/ML University of Zurich, Zurich,
components. To improve patient care, a stronger focus should be placed on the distinctive characteristics of AI/ML Switzerland; Center for
when dening substantial equivalence between a new AI/ML-based medical device and predicate devices. Artificial Intelligence in
Medicine and Imaging,
Stanford University, Stanford,
Introduction for medical devices and device-led combination products CA, USA (C Bluethgen); Faculty
The US Food and Drug Administration (FDA) has cleared targeting an underlying disease or condition with the of Law, University of Zurich,
an increasing number of articial intelligence and goal of providing patients with timely access while Zurich, Switzerland
machine learning (AI/ML)-based medical devices over the preserving statutory standards.19 (Prof K N Vokinger MD)

past 5 years.1–3 An example is the use of smart
electrocardiogram (ECG) devices for estimating the
probability of a heart attack.4 AI/ML-based medical devices
involve new techniques or distinctive characteristics.4–6
In 2019, the FDA intensied discussions on policies in the
clearance of these devices.4–6 There is no specic clearance
pathway for AI/ML-based medical devices. On the basis of
the risks of the devices, the FDA clears medical devices
through three main pathways: approval through the
premarket approval pathway, authorisation via the de novo
premarket review, and clearance via the 510(k) pathway.3,7
For simplicity, we use the term clearance to refer to the
marketing authorisation of the devices via all pathways.
The 510(k) pathway allows the clearance of a new
medical device if the manufacturer shows that the device
is substantially equivalent to one or more legally
marketed devices (referred to as predicate devices).
Predicate devices are other 510(k)-cleared devices, devices
cleared through the de novo pathway, or devices initially
cleared through the premarket approval that have been
reclassied.8–10 Manufacturers might also refer to
reference devices; however, these devices can be used
only to support scientic methodology and are not
considered as predicate devices.11 Devices cleared through
the de novo pathway represent novel technologies in
contrast to 510(k)-cleared devices.12,13
A lenient interpretation of substantial equivalence
facilitates clearance of generations of devices that claim
substantial equivalence to each other with iterative
design changes, resulting in devices dissimilar from
original predicates—ie, predicate creep—posing safety
risks to patients.2,12,14–17 Scientists and policy makers have
debated whether and how the 510(k) pathway should be
restructured to improve the quality of devices for
patients.15,16,18 For example, in 2021 the FDA introduced
the Safer Technologies Program, a voluntary programme
The majority of AI/ML-based medical devices have Correspondence to:
Prof Kerstin N Vokinger, Faculty
been cleared through the 510(k) pathway.1,3 To the best of
of Law, University of Zurich,
our knowledge, the predicate networks of AI/ML-based 8001 Zurich, Switzerland
medical devices cleared through the 510(k) pathway have lst.vokinger@rwi.uzh.ch
not yet been systematically assessed. In this study, we
analysed AI/ML-based medical devices and their
predicate networks across medical specialties; more
specically we analysed length of time and number of
generations until a de novo medical device or
a non-AI/ML medical device was reached (including the
role of reference devices and patterns in the reporting of
technical terms attributable to AI/ML), tasks of the
underlying AI/ML, and number of recalls.
Methods
Search strategy and selection criteria
We used the FDA’s publicly available list of AI/ML-based
medical devices to identify all AI/ML-based devices
cleared via the 510(k) pathway between Jan 9, 2019, and
Dec 29, 2021, and included them in our primary study
cohort.1 UJM and CB did the search, and all AI/ML-based
devices cleared via the 510(k) pathway were included, with
no restrictions. The FDA’s 510(k) premarket notication
and de novo databases served as data sources to identify
the predicate devices of the devices included in our
primary study cohort.20,21 We (UJM and CB) identied the
predicate devices for each device in the primary study
cohort until a de novo or non-AI/ML-based predicate
device was reached.
Data extraction and analysis
Predicate devices were identied by use of natural
language processing in R, applying a similar method as
a previous study that identied 510(k) predicate devices.22
In summary, we used 510(k) documents (ie, statement,
summary, and decision summary) as input data and

www.thelancet.com/digital-health Vol 5 September 2023 e618

 Health Policy
extracted the text using the tesseract package. From this
text, we identied strings similar to the medical device
control numbers (ie, 510[k] number, de novo number, and
premarket approval number). We then removed duplicates
and created a list of potential predicate devices associated
with a medical device in our primary study cohort as the
nal output. To maximise the sensitivity of this method,
we did not apply text cleaning or anchor keywords.
Publicly accessible 510(k) documents were continuously
web-scraped from the FDA’s 510(k) premarket notication
database and searched for predicate devices.
In a manual data extraction process, UJM and CB (who
have expertise in the eld of AI/ML) independently
assessed the automatically extracted predicate devices.
Divergences were resolved through discussions between
all authors. Devices that were falsely identied as
predicates and duplicates were manually removed and
devices that were missed by the automated extraction
process were manually added.
Each predicate device was assessed for whether it
included an AI/ML component by use of the afore-
mentioned FDA publicly available documents. These
documents include information about the intended use
and the technological characteristics.23 AI/ML tasks were
also identied from these documents.
There is no unied denition of AI.24 The FDA denes
AI as the science and engineering of making intelligent
machines, and they outline that AI can use dierent
techniques, such as models based on statistical analysis
of data, expert systems that primarily rely on if–then
statements, or machine learning.4 Whereas early
AI systems were generally based on decision rules, recent
AI research focuses on machine learning methods.25
Thus, aligned with our previous study,3 we considered
only medical devices using machine learning as AI/ML-
based medical devices. Machine learning can be
implemented in a system that has the capacity to learn
from training on a specic task by tracking performance
measures.4
We categorised predicate devices as AI/ML, unclear
AI/ML, or non-AI/ML. A device was qualied as AI/ML
if the underlying AI/ML technique was explicitly stated
(eg, deep learning neural networks for K203235;
See Online for appendix appendix p 11) or if the device was listed in the FDA’s
published list of AI/ML-based medical devices. A
previous study showed that many AI/ML-based medical
devices on the FDA published list (152 [44·3%] of 343
devices) did not mention AI/ML-related keywords in
their 510(k) documents.26 To mitigate potential bias, we
qualied devices that stated tasks usually solved by
AI/ML at the time of their clearance as AI/ML when the
documents allowed us to implicitly draw this conclusion
(eg, stating arrhythmia analysis as the task at a time
when this task was usually solved with machine
learning27). Devices for which the documents described
tasks that could have either been solved with or without
AI/ML (eg, stating automatic segmentation as the task
e619
without information for the specic segmentation
technique) were qualied as unclear AI/ML. Devices
that did not meet the inclusion criteria for AI/ML or
unclear AI/ML were considered as non-AI/ML.
Additional information, especially reference devices,
extracted from the FDA’s 510(k) premarket notication
and de novo database is listed in the appendix (p 1). We
calculated the frequencies of explicit reporting of
technical terms attributable to AI/ML in the publicly
available documents to evaluate how the explicit
reporting of these technical terms has evolved over
time.
We traced back the predicate networks of the primary
AI/ML-based medical devices to a non-AI/ML-based
predicate device, a predicate device with missing publicly
available documents, or an underlying de novo device. We
did not include reference devices in the predicate networks
because they are not considered to be predicate devices.11
The rst generation was dened as all predicate devices
for which the devices in our primary study cohort claimed
substantial equivalence for clearance, the second
generation included all predicate devices for which the
rst-generation devices claimed substantial equivalence,
and so on.
We veried which AI/ML-based medical devices were
aected by a recall by searching the openFDA Device
Recalls data set.28 Ambiguities in the data extraction
process were resolved by discussion between all authors.
This study adheres to the Standards for Reporting
Qualitative Research guidelines.
Statistical analysis
Descriptive statistics were calculated to assess the
predicate networks of AI/ML-based medical devices in
our study cohort. Dierences in the number of predicate
generations between medical specialties were analysed
by use of the Wilcoxon rank-sum test and dierences in
the number of AI/ML tasks were analysed by use of the
χ² test. To avoid confounding, devices that occurred in
more than one predicate network due to connections
between the networks were counted only once. Two-tailed
p values of less than 0·05 were considered to indicate
signicant dierences.
All statistical analyses were done in R, version 4.2.3.
Results
Overview of study cohort and medical specialties
The complete study cohort with 755 medical devices was
composed of the primary study cohort of 285 devices
(AI/ML-based medical devices cleared by the FDA
between Jan 9, 2019, and Dec 29, 2021) and their predicate
networks, with 470 unique medical devices (456 cleared
through the 510(k) pathway, 13 through the de novo
pathway, and one through the premarket approval
pathway; gure 1; appendix pp 2–15, 16–36). 85 (18·1%)
of 470 devices were not identied in the automated
extraction process, mainly because some of the FDA’s
www.thelancet.com/digital-health Vol 5 September 2023