How to Design AI-Driven Clinical Trials in

Nuclear Medicine
Gaspar Delso, PhD,* Davide Cirillo, PhD,† Joshua D Kaggie, PhD,z
Alfonso Valencia, PhD Prof.,† Ur Metser, MD Prof.,x and Patrick Veit-Haibach, MD Prof.x

Artificial intelligence (AI) is an overarching term for a multitude of technologies which are
currently being discussed and introduced in several areas of medicine and in medical imag-
ing specifically. There is, however, limited literature and information about how AI techni-
ques can be integrated into the design of clinical imaging trials. This article will present
several aspects of AI being used in trials today and how imaging departments and espe-
cially nuclear medicine departments can prepare themselves to be at the forefront of AI-
driven clinical trials. Beginning with some basic explanation on AI techniques currently
being used and existing challenges of its implementation, it will also cover the logistical
prerequisites which have to be in place in nuclear medicine departments to participate suc-
cessfully in AI-driven clinical trials.
Semin Nucl Med 51:112-119 © 2020 Elsevier Inc. All rights reserved.

Introduction success and sustainability of these activities, AI is gaining an

increasing importance in the definition of the future of phar-

I n this manuscript we present a brief overview of the vari-

ous hurdles hampering the efficacy of clinical trials and
how new artificial intelligence (AI) and machine learning
techniques are rising to meet the challenge. We will then
focus on the steps that Nuclear Medicine departments can
take to position themselves favorably toward what is most
likely to become the future of clinical trials.
The pathway for a candidate compound to become a regu-
latory approved drug is a notoriously lengthy undertaking.
Patient cohort recruitment and clinical endpoint selection are
among the main causes of high failure rates in the late stages
of trials and can cause significant human and financial costs.
The overall progressing decline of new drugs approval and
return on investment has been fueling a long-standing debate
from both research and pharmaceutical industry around new
ways to optimize drug development.1,2 Among the most
promising avenues towards an effective improvement of
*GE Healthcare, Chicago, IL.
y
Barcelona Supercomputing Centre, Barcelona, ES.
z
Department of Radiology, University of Cambridge, Cambridge, UK.
x
Joint Department of Medical Imaging, University Health Network, Toronto,
CA.
Address reprint requests to: Patrick Veit-Haibach, MD, Department of
Medical Imaging, University of Toronto, Toronto General Hospital, 1
PMB-275, 585 University Avenue, Toronto, ON M5G 2N2. E-mail:
Patrick.Veit-Haibach@uhn.ca
maceutical clinical research and healthcare,3 further fueled
by an increasing interest in real world data mining.4
Diagnostic Nuclear Medicine plays an important part in
many clinical trials as patients undergo imaging-derived ther-
apy decisions during the trial. Several AI techniques have
been developed to improve nuclear medicine imaging and
are therefore now part of clinical trials. Additionally, Thera-
peutic Nuclear Medicine (Theranostic) is gaining momentum
as a therapeutic option in different diseases and therefore
must prove its clinical value in trials of its own.
Standard design of clinical trials
Clinical trials study specific aspects of biomedical interven-
tions and, generally, evaluate their impact on human health.
This includes the testing of new compounds, medical devices
and procedures, as well as the comparative evaluation of
known treatments.
The goal of clinical trials is to generate data on safety and
efficacy of a medical technique, such as a new drug, device
or procedure, under rigorously controlled conditions, ensur-
ing the scientific validity and reproducibility of the results, as
well as the safety of its participants.5-8 The proposed study
protocol must be reviewed by a Research Ethics Committee

112 https://doi.org/10.1053/j.semnuclmed.2020.09.003
0001-2998/© 2020 Elsevier Inc. All rights reserved.
How to Design AI-Driven Clinical Trials in Nuclear Medicine
Figure 1 Schematic overview over the different phases of a standard
clinical trial and the specific requirements and timelines per trials
phase.
and submitted for approval to the local or even higher regula-
tory authorities, who are responsible for assessing the relative
risks and benefits of the trial. Clinical trials can involve multi-
ple centers, at a local, national, and multinational level,
which add to the complexity of the regulatory requirements.
A traditional clinical trial is structured as a sequential set of
phases of increasing magnitude (see Fig. 1). These are often
in the form of randomized controlled trials. Success in each
phase is required before proceeding to the next. The benefits
of this structure are increased participant safety and reduced
costs in the event of early failure. However, it is also a
lengthy and rigid process, well suited for testing compounds
intended for mass marketing, but less so in the case of
complex therapies, rare diseases or very specific patient
populations.
In the case of medical devices, preclinical and clinical trials
are required in order to obtain premarket approval, except
when substantial equivalence to an already marketed device
can be proven via the 510(k) pathway (in the United States)
Similar, but not identical regulations apply in other regions: for
example, the new Medical Device Regulation 2017/745, set to
become fully effective in the E.U. in 2021, contemplates device
categories roughly equivalent to those used by the FDA.
While the methodology behind these studies has remained
essentially unchanged since their introduction seventy years
ago,9 clinical trials have become increasingly complex, time-
consuming and costly due to the continuous evolution and
proliferation of standards and requirements. Adaptive
designs have been proposed to provide increased flexibility,
albeit at a cost in logistic complexity.10
The unfortunate consequence of this is that clinical trial
costs have steadily increased,11 in some cases reaching stag-
gering figures in the hundreds of millions of dollars, while
their success ratio has decreased (ie, the so-called Eroom's
law, in reverse to the widely known Moore’s law).
113
Several factors contribute to this decrease of return on
investment, that elucidate more than the classical explanation
of disappearing “low-hanging-fruit”:
 The increased number of ongoing clinical trials has
increased the difficulty to meet the recruitment targets
defined in the trial protocol. Suitable, willing subjects
are becoming a rare commodity in a market saturated
with clinical trials for the most common diseases. This
is compounded with the lack of appropriate systems
for large-scale screening of medical records. Even
potential volunteers and healthcare practitioners
actively looking to enroll their patients are struggling
to identify appropriate trials, both due to the technical
language of the admission criteria and to the sheer
amount of information to navigate. This is particularly
critical in Nuclear Medicine trials, where the proce-
dures involved may cause harm and pathologies are
evolving.
 Highly specific recruitment requirements can eventu-
ally lead to struggles in fulfilling targets or suboptimal
patient selection. Poor patient selection will compro-
mise the validity of the study outcomes and increase
the probability of protocol deviations and participant
dropout. Participant retention is a critical consideration
as trials move toward more targeted patient popula-
tions, whether due to the ethnicity and social circum-
stances of the subjects or to the rarity of their
condition.
 Additional confounding factors as studies become
increasingly complex are patient monitoring and the
optimization of dosing regimens. These affect patient
safety and trial outcomes and require a significant
investment in terms of subject-matter expert time,
while introducing an undesirable degree of subjectivity
to the study.
Potential of AI techniques
Over 450 interventional studies (clinical trials) include the
terms “machine learning,” “deep learning,” or “artificial intel-
ligence,” as listed by the U.S. National Library of Medicine as
of July 2020. Roughly one-third of these trials are currently
recruiting (Fig. 2).
With the continuous growth of computational power and
available data (sometimes termed “Real World Data”), AI
promises its unprecedented potential to achieve a faster,
cheaper and more effective biomedical development process.
Indeed, in response to the digitization of health information
and the advancement in high-throughput technologies, the
generation of vast and complex volumes of molecular and
clinical datasets from thousands of patients is setting out new
challenges to computational analysis and interpretation of
biomedical data, as it grows ever larger. AI enables effective
patient-trial matching with improved adherence and efficient
endpoint assessment by operating on heterogeneous data,
including structured and unstructured datasets, imaging and
114
Figure 2 Graphic overview of the three focus areas for AI-based
enhancement of clinical trials.
radiopharmaceutical data, Electronic Medical Records, mul-
tiomics data (genomics, radiomics, proteomics, metabolo-
mics, etc.), social media, and many others.12,13
Big data in medicine and healthcare refers to the con-
tinuous generation and acquisition of an increasing varied
ecosystem of biomedical data types and data governance
solutions, ranging from digital data collection devices to
federated infrastructures for long-term storage and secure
data sharing. The main characteristics of the so-called
biomedical big data14,15 such as volume, velocity, variety,
and veracity, are exposing the limitations of traditional
data processing.16 In addition to such wealth of data and
data handling solutions, the acceleration of effective
development is also promoted by the convergence of
cheaper hardware, the availability of pretrained models
and open-source, off-the-shelf applications. Indeed, while
training large-scale deep-learning models is still expensive
and in general cannot be done locally, affordable GPU-
based hardware is now available for inferencing (ie, using
an already trained model), as well as prototyping and
adapting existing models.
The extraordinary recent developments in the field of AI
have resulted in a plethora of data-driven hypothesis-gen-
erating solutions that seem an ideal match for the health-
care field. This paradigm shift in biomedical research and
development is encouraging both industries and healthcare
systems to adopt innovative technologies that focus on col-
lecting, storing and analyzing biomedical information.17
For instance, intelligent drug design for personalized ther-
apies represents an illustrative asset that is expected to
thrive from the application of AI-aided big data analytics
and provide actionable insights and benefits not only to
the patients and clinical practitioners but also to policy
makers and strategic business decision actors.18 Albeit at a
slower pace, AI is also permeating the Nuclear Medicine
field, from therapy planning to scanning, interpretation
and reporting, starting with cognitively-undemanding
labor-intensive tasks.19
AI holds the potential to revolutionize clinical trials and
have a significant economic impact in healthcare by facilitat-
ing our understanding of disease mechanisms, the detection
of new drug targets and biomarkers, and thus the identifica-
tion of suitable patients.20 For example, AI-driven tools will
be used to select homogenous cohorts of patients, predicted
G. Delso et al.
to be more likely to respond to the treatment and display
measurable endpoints.
Three focus areas for AI-based technologies in clinical tri-
als have been identified, spanning all stages of trial design,
startup, conduct, and closeup.
Focus on the biology
Several research efforts are focused on the implementation
of AI systems for the discovery of new compounds, syn-
ergistic drug combinations and uses (ie, drug repurpos-
ing), as well as a better understanding of compounds’
mode-of-action.21 All those applications are based on
an ample spectrum of AI approaches, including super-
vised and unsupervised machine learning methods,
aiming to maximizing the therapeutic benefit against a
given disease target by exploring the so-called chemical
space, which is estimated to contain 1060 organic mole-
cules of interest for drug discovery.22 Examples of such
methods include generative models based on neural
networks,23,24 in which the network is trained to gener-
ate data that mimics that of the system of interest.`
Focus on the patients
AI can improve trials by boosting patient recruitment,
screening, enrollment, and monitoring, which will fos-
ter empowered patients and personalize approaches.
Digital technologies assessing the patient’s experience
have the potential to translate meaningful changes in
disease (eg, quantitative improvements by measuring
physical parameters), such as wearable devices and
mobile phone apps, which can enhance patient-centric
innovations by determining useful digital endpoints.25
Digital phenotypes, such as real-time monitored activity
and mobility, have clinical utility and can be validated
in randomized control interventional studies. The gen-
eration of real-world evidence through AI-enhanced
processing of real-world data can inform both trial prep-
aration and outcome analysis. Finally, ethically-
informed AI can enhance patient engagement through
highly tailored developments that must respect equity
and inclusiveness.26,27
Focus on the processes
AI can be directly applied to evaluate the performance of
clinical trial pipelines to identify common characteristics
associated with regulatory approval or refusal, including
not only efficacy and safety issues but also strategic and
financial aspects.28 Using AI to reason on the attrition
rates in drug development can support decision-making
and process management in all its aspects by planning
and forecasting business models thus reducing risk and
time expended from strategy design to execution. The
How to Design AI-Driven Clinical Trials in Nuclear Medicine
identification of potential investigators, including spe-
cialists, committees and scientific boards, as well as the
selection of high-performance sites, based on budget,
qualifications and patient population, are planning
aspects of clinical trials that can be informed and opti-
mized by AI. Moreover, AI can reduce the impact of
human error in data collection, discover trends and pin-
point relevant insights, and source quality information
to optimize production, supply chain and logistics. For
instance, pharmaceutical manufacturers consider AI
essential in specific areas such as cost-efficiently optimi-
zation of compound administration and dosing
regimens.29
As for nuclear medicine, several areas have benefited from AI
applications, including nuclear cardiology, oncology, and neu-
rology.30-37 As with other imaging modalities, the most visible
aspect has been a proliferation of segmentation and classifica-
tion applications for diagnostic support.38 However, AI is also
being applied in creative new ways: image quality enhancement
from low dose acquisitions39; reconstruction40,41; attenuation
correction for hybrid imaging42; spatial normalization43; multi-
modal data analysis44; predicting patient response to therapy45;
implementation of AI-powered imaging biomarkers,46 opening
up the possibility of large-scale population screening with a
radical impact in clinical trial improvement.
Challenges of integrating AI into
trials
Despite the undeniable potential of AI methods to streamline
clinical trials, a word of caution is required. The recent prolif-
eration of enthusiastic reports about the topic, often listing
inspiring but episodic success stories, may offer a misleading
picture of the considerable amount of work still required to
enable this vision. In the following sections we will focus on
the case of Nuclear Medicine departments participating in
clinical trials driven by pharmaceutical and medical technol-
ogy companies.
While the extraordinary leap forward of the AI field over
the past ten years has been of a technical nature, it is impor-
tant to realize that the requirements for its successful integra-
tion in practice are unquestionably multidisciplinary. Drug
and medical device manufacturers need to establish partner-
ships with academical institutions, clinical centers and ser-
vice providers in order to secure the required expertise and
infrastructure to support these new technologies. This com-
prises, from an in-depth understanding of the principles of
AI in order to select the best suited model for a given job, to
the ability to recruit and process patients for the trial, to the
technical means to provide long-term storage and curation of
the resulting data. Good examples of efforts toward this goal
are the MELLODDY Consortium,47 using part of the chemi-
cal libraries of 10 pharmaceutical firms to create a deep learn-
ing-based drug discovery service; or the MLPDS consortium,
a collaboration between biotechnology and pharma
115
companies and several departments of the Massachusetts
Institute of Technology to automate small molecule discov-
ery and synthesis. Similar consortia are being created, albeit
at a slower pace, for the field of radiopharmaceutical devel-
opment (eg, Isotope4life).
Another well-known limitation of modern AI is its depen-
dency on vast amounts of training data. Clinical trial data is
especially sensitive as it is connected to personal health infor-
mation that is heavily protected against misuse. In practical
terms, hospitals differ in their willingness to integrate new AI
systems as these can cause risk if a patient’s information is
leaked or misdiagnosed by these systems. Hospitals can also
differ in their business models, being for-profit or non-for-
profit, which can cause varied engagement with commercial
institutions. Data acquisition is one of the most significant
hurdles in AI training. At present, AI development may have
intellectual property surrounding their code, their data, or
future developments, that a hospital or commercial entity
may want to protect. Especially in imaging and nuclear medi-
cine, there are large variations across institutions even in the
same jurisdiction concerning acquisition of imaging data,
reconstruction, transfer and storage.48-54 Even in optimally
functioning consortia adhering to strict data harmonization
policies, intrinsic differences in data quality (eg, due to scan-
ner performance) may impair the use of common AI models
to extract and analyze the resulting biomarkers.55 Addition-
ally, nuclear medicine data as semiquantitative (and partly
qualitative-only) modality, is prone to subjective interpreta-
tion and thus, automated mining of data currently poses a
comparability and curation problem, introducing a potential
bias for clinical trials.
Significant efforts, driven by both public and private
investment, have been dedicated in recent years to the
harmonization and standardization of biomedical data, the
networking of databases and the development of open-
source data management tools. Arguably, the biomedical
part of this effort56-59 is far more advanced than its healthcare
counterpart, with FAIR60 and AAI61 standards in place and
implemented and journals, agencies, and institutions com-
mitted to make data openly accessible. While their success
gives us reason to hope for a similar outcome in the medical
field, there is still much work ahead.
Natural language processing algorithms have only recently
started to curate into structured, searchable databases the
staggering amount of usable data encoded in physician’s
notes, which may still not be digital or easily interpreted by
external parties. Challenges are inherent with data collection
as new discovery techniques or diseases can emerge that may
not harmonize with data from older trials. Indeed, proven
clinical care often lags behind new methods as the burden of
proof is high for both clinical methods and drug trials in
order to reduce harms to the patients. With these constantly
developing new methods, companies may also have con-
stantly developing proprietary and changing formats that AI
models are ill-equipped to handle.
The technical, administrative, and legal resources required
to expand AI efforts across clinical centers and even country
borders, addressing such issues as harmonization, data
116
ownership and protection from malicious access, can become
daunting, although not impossible.62,63 This is discussed in
“Clinical and ethical considerations of AI-enhanced trials.”
Departmental prerequisites for
AI implementation
Pharmaceutical and medical device companies are actively
looking for partnerships that will allow them to enhance their
clinical trials with promising new technologies. There are
several steps that clinical sites can undertake to enhance their
eligibility as participants of those trials. These are, for the
most part, as self-evident as going paperless and, in many
cases, can take just as much effort and resources. We provide
in this section a summary of those steps, with a focus on
Nuclear Medicine departments.
There is a technical aspect to this transformation, to ensure
that the more general hardware and software infrastructure
requirements are satisfied. But, more importantly, there is a
logistic and cultural aspect in the drive towards the generation
of curated, accessible, standardized and, in summary, AI-
friendly data. It might be objected that AI algorithms have
been demonstrated to successfully extract meaningful informa-
tion from unstructured datasets (a commercial example could
be the web-based healthcare data mining tool by Roam Analyt-
ics, San Mateo, CA). However feasible, unstructured data man-
agement remains an inefficient alternative to proper data
collection and curation practices. In any case, as a general
rule, the better the data are organized and structured, the eas-
ier future applications will be, AI-based and otherwise.
An advisable first step would be cataloguing the various
data sources controlled by the department and implementing
protocols to manage their lifecycle.64,65 This includes all data
typically included in medical records (personal statistics,
demographics, medical history, medication and allergies,
vital signs, test results, imaging, billing information, etc.),
but also data about the facilities (medical devices, isotope,
and tracer production), personnel and expertise, research
protocols, ethical approval documentation, informed consent
forms, information, and divulgation materials, etc.
For Nuclear Medicine departments specifically, this means
that there ideally needs to exist a centralized, standardized
system which can capture the multiple processes which are
unique for Nuclear Medicine and mine them for predictive/
diagnostic/prescriptive analytics. Data sources to be captured
in here are, for example, the radiopharmacy. The steps to be
documented in a standardized way range from the produc-
tion processes (several steps for low-volume tracers are not
automated), the status and delivery of precursors, the equip-
ment used (ie, reagents and hotcells), to the quality control
and shipping logistics. For the imaging equipment, a separa-
tion of records might be needed that is, for preclinical vs.
clinical systems. Also, differences between research acquisi-
tion vs standard clinical acquisition (injected tracer doses,
acquisition time, reconstruction) must be monitored and
documented to be comprehensible and minable in an effi-
cient way. The documentation can be even extended to the
G. Delso et al.
Research Ethics evaluation process, which also captures
Nuclear Medicine specific topics like radiation burden and
radiopharmaceutical safety.
Imaging in nuclear medicine is particularly challenging
from the point of view of AI: costly examinations with rela-
tively low throughput and involving ionizing radiation. The
field of synthetic data generation offers an opportunity in
this case.66 A key to best take advantage of synthetic data
generation for AI purposes is controlling the image genera-
tion model. Indeed, imaging modalities in nuclear medicine
often involve a complex reconstruction process to transform
the data acquired by the scanner into human-readable
images. Access to this reconstruction and its internal parame-
ters, as well as a firm grasp of the underlying physics, will
provide a huge boost to the amount and quality of the syn-
thetic data that can be produced.
There are many good reasons for the adoption of an elec-
tronic health record (EHR) software, too many to enumerate
in this article or to advise on a specific implementation
(locally hosted vs cloud-based, proprietary vs open, etc.)
Two important considerations are worth mentioning, when
considering such a move: first, this is an effort that will
require the engagement of the department staff, preferably
with someone to champion it. Second, make sure to be aware
of the data model used by the EHRs. There are several pro-
posed standards to structure health record data and stan-
dardize database querying and export, including Common
Data Models intended to enable the combination of data
from multiple sources (eg, Sentinel, OMOP, PCORNet, Vir-
tual Data Warehouse). While we do not wish to advocate for
one specifically, it is likely that clinical trial organizers will
prioritize sites with readily accessible data stored following a
well-known data model. Initiatives such as the Observational
Health Data Sciences and Informatics (OHDSI) and the Euro-
pean Health Data & Evidence Network (EHDEN) are good
sources of information on this respect.
Several recommendations are valid before engaging in any
kind of clinical trial: the adoption of a document manage-
ment system and the associated practices are highly advis-
able; when data measurements are generated, calibration and
quality control measurements will have to be periodically
recorded; considering the expensive nature of the data, it is
important to also consider the potential effect of unforeseen
disastrous events, like a fire, such that data or models should
be backed up in multiple locations. This work is typically
done by clinical research coordinators or physicians them-
selves (depending on the jurisdiction), trained profoundly in
GCP trial coordination and documentation. For nuclear
medicine studies specifically, the correct and compliant
documentation of the most sensitive data (ie, radiopharma-
ceutical or contrast media dosages, acquisition time, recon-
struction parameters) are of utmost importance. Here, a
standardized documentation system (paper based earlier,
currently moving into electronic system administration) is
required by the authorities and is vigilantly monitored. AI
techniques and machine learning algorithm are prone to
deliver significantly improved results when electronic based
and standardized systems are available as a database.
How to Design AI-Driven Clinical Trials in Nuclear Medicine
Data consistency is a key factor affecting the performance
of trained methods. Sites will have to be particularly vigilant
to ensure that no significant changes are introduced in the
data generation pathway during a study (eg, from the way
the patient weight is measured to the activity of patients dur-
ing radiotracer uptake time). This may include events such
as equipment servicing or upgrades, re-calibration, changes
in compound production or supplier, new personnel, etc.
This is particularly critical when AI models of the “black
box” type are used, as they are particularly sensitive to data
inconsistencies and biases during their training.
Personnel awareness and involvement will be a key factor
in order to ensure not just data consistency but, more impor-
tantly, provable consistency, in the event of audits. Clear,
comprehensive and well-documented protocols are invalu-
able in this effort. New professional profiles, such as Biomed-
ical Engineers, combining the life sciences and technology
backgrounds required by this kind of endeavor, are increas-
ingly available and slowly being integrated into the field.
Nuclear Medicine is a particularly challenging field in this
sense, due to the overlap of nuclear physics and hybrid imag-
ing modalities, each requiring specialized competencies and a
considerable technical background.
Reference data, such as phantom measurements and
healthy volunteer databases can also provide quantitative
metrics of data consistency, nowadays easily automated and
tracked. Reference datasets can also be of great value for
study sponsors trying to determine whether a site is suited
for their study, or in the case that an AI model needs site-spe-
cific re-training.
Current examples of AI-related clinical trials in the field
of Nuclear Medicine are mostly focused on the evaluation
of AI clinical tools. These range from the relatively small
single-site studies (eg, “Symmetricity-Driven Learning
Framework for Pediatric Temporal Lobe Epilepsy Detection
Using 18F-FDG-PET Imaging” NCT04169581; “Usefulness
of Deep-Learning Image Reconstruction for Cardiac Com-
puted Tomography Angiography - a Prospective, Non-
randomized Observational Trial” NCT03980470), to large
multisite collaborations (eg, “Automated Quantification of
Coronary Artery Calcifications on Radiotherapy Planning
CTs for Cardiovascular Risk Prediction in Breast Cancer
Patients: the BRAGATSTON Study” NCT03206333). Evi-
dence of the usefulness of AI tools for clinical trial support,
on the other hand, is mainly limited to preliminary study
results.
Clinical and ethical
considerations of AI-enhanced
trials
The ethical implications of the increasing use of AI in health-
care are a complex topic still being widely debated.67 Depart-
ments entering clinical trials enhanced by AI will, in most
cases, be working with predictive decision-support systems.
In which case they may want to consider some of the
117
following topics and engage their ethical and legal depart-
ments in the discussion:
- Security: Gathering large amounts of well-structured
data in electronic repositories carries an inevitable risk
of exposing confidential patient information, either
accidentally or through a malicious security breach.
This is particularly critical in the case of Nuclear Medi-
cine, given the severity of the conditions involved. Per-
sonnel training and strict adherence to the relevant
data protection regulations (GDPR, CCPA, etc.) will go
a long way towards minimizing these risks. In general,
straightforward, well-designed data pipelines and pro-
tocols yield better results than excessive security meas-
ures that impair workflow and often motivate
personnel to find workarounds.
- Reliability: When AI is used to drive healthcare deci-
sion-making there is an inherent risk of error. More to
the point, there is a risk of errors being committed in a
different manner as a human or a conventional data
processing algorithm would. A human will (in most
cases) acknowledge uncertainty and account for it in
the decision-making process. Conventional processing
can fail more or less graciously, depending on the qual-
ity of its programming, but failure will more often than
not be recognizable. That is not necessarily the case
with AI, particularly with deep learning models. Fail-
ure can sometimes come in unexpected ways, even for
apparently minor variations of the model input (an
effect poetically termed “artificial stupidity”). Until suf-
ficiently tested in real-world scenarios, AI should be
considered as complementary to, not independent
from, expert decision-making. Notice how, even in
that case, unwanted bias may arise if experts find it eas-
ier (or safer) to agree with the system rather than hav-
ing to justify their disagreement, hence transforming
the initial complementary AI in the de-facto decision
system.
- Training: Due to the complementary nature of the role
of AI in the immediate future, it is important to train
clinical personnel to team up with these new tools. AI
does have the potential to identify patterns that elude
human experts and the necessary supervisory role of
the latter cannot come at a cost of overlooking and
overruling the insights that the algorithm can provide.
Clinicians will have to strike a balance between trusting
the machine and falling into the complacency of
blindly trusting it.
- Bias: A known example of the limitations of AI algo-
rithms is their dependency with the database used to
train them. Subtle biases in the database can be inad-
vertently encoded in the resulting model, leading to
infamous cases such as algorithms basing their deci-
sions on race, gender, or socioeconomic information.
Avoiding these unfortunate situations depends on the
same factors that have already been discussed: Suitable
personnel training to ensure unbiased recruitment,
good record keeping practices, expert supervision of
118
AI output, etc. In those cases where pretrained AI
models are to be applied, re-training should be consid-
ered, in order to adapt the model to the local popula-
tion (eg, certain tracers are only available in certain
countries, and tracer dosing guidelines may vary
depending on the jurisdiction).
- Explainability: A whole field has appeared recently
around the concept of explainable AI, that is, the devel-
opment of AI models capable of providing the reasons
for its decisions.68 Scientists working in this field work
towards more transparent AI, as opposed to the infa-
mous “black box” behavior of many current models.
- Liability: Questions have been raised about the possible
legal implications of using AI algorithms of the “black
box” type in clinical decision making. It is a concern
that, in case of malpractice litigation, AI users would
not be able to provide the rationale for the algorithm
decision. Thus, ultimately the physician (at least cur-
rently) has the last say in diagnostic as well as thera-
peutic decision making.
Conclusion
Industry stakeholders are looking towards AI hoping to
break the diminishing returns trend of modern clinical trials.
Modern AI has the potential to provide more meaningful and
cost-effective trial outcomes. Its applications range from
strategic analysis to compound development, participant
recruitment and clinical decision-making. Establishing col-
laborations with academic, clinical and technology providers
will be essential to enable these technologies.
Nuclear medicine departments wishing to enter this field
will need to undergo several changes, both in terms of tech-
nical infrastructure and in terms of professional practices.
From patient selection to radiotracer production, dosing,
diagnosis and therapeutic outcome prediction, it will ulti-
mately be in the hands of the qualified experts to make sure
that the right AI tools and the right data are being used, effi-
ciently, securely and without biases. Team engagement and
training will be key factors for these new technologies to take
root and yield their full potential.
References
1. Scannell JW, Blanckley A, Boldon H, et al: Diagnosing the decline in phar-
maceutical R&D efficiency. Nat Rev Drug Discov 11:191-200, 2012
2. Mullard A: R&D returns continue to fall. Nat Rev Drug Discov 16:9,
2017
3. Topol EJ: High-performance medicine: the convergence of human and
artificial intelligence. Nat Med 25:44-56, 2019
4. Ramagopalan SV, Simpson A, Sammon C: Can real-world data really
replace randomised clinical trials? BMC Med 18, 2020. https://www.
ncbi.nlm.nih.gov/pmc/articles/PMC6961357/
5. WMA - The World Medical Association-WMA Declaration of Helsinki –
Ethical Principles for Medical Research Involving Human Subjects.
https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-
principles-for-medical-research-involving-human-subjects/
6. The Medicines for Human Use (Clinical Trials) Regulations 2004.
Queen's Printer of Acts of Parliament. https://www.legislation.gov.uk/
uksi/2004/1031/contents/made
G. Delso et al.
7. Ervine C: Directive 2004/39/Ec of the European Parliament and of the
Council of 21 April 2004. Core Statutes on Company Law. London:
Macmillan Education UK, 757-759, 2015 http://link.springer.com/
10.1007/978-1-137-54507-7_21
8. EUR-Lex - 32005L0028 - EN - EUR-Lex. https://eur-lex.europa.eu/eli/
dir/2005/28/oj
9. Stolberg HO, Norman G, Trop I: Randomized Controlled Trials. AM J
Roentgenol 183:1539-1544, 2004
10. Pallmann P, Bedding AW, Choodari-Oskooei B, et al: Adaptive designs
in clinical trials: why use them, and how to run and report them. BMC
Med 16:29, 2018
11. Moore TJ, Zhang H, Anderson G, et al: Estimated Costs of Pivotal Trials
for Novel Therapeutic Agents Approved by the US Food and Drug
Administration, 2015-2016. JAMA Intern Med 178:1451-1457, 2018
12. Harrer S, Shah P, Antony B, et al: Artificial Intelligence for Clinical Trial
Design. Trends Pharmacol Sci 40:577-591, 2019
13. Uribe CF, Mathotaarachchi S, Gaudet V, et al: Machine Learning in
Nuclear Medicine: Part 1—Introduction. J Nucl Med 60:451-458, 2019
14. Bender E: Big data in biomedicine. Nature 527(7576):S1, 2015
15. Stephens ZD, Lee SY, Faghri F, et al: Big Data: Astronomical or Genomi-
cal? PLOS Biol 13:e1002195, 2015
16. Big hopes for big data. Nat Med 26:1, 2020
17. Shah P, Kendall F, Khozin S, et al: Artificial intelligence and machine
learning in clinical development: a translational perspective. Npj Digit
Med 2:1-5, 2019
18. Schneider P, Walters WP, Plowright AT, et al: Rethinking drug design in
the artificial intelligence era. Nat Rev Drug Discov 19:353-364, 2020
19. Nensa F, Demircioglu A, Rischpler C: Artificial Intelligence in Nuclear
Medicine. J Nucl Med 60(Supplement 2):29S-37S, 2019
20. Wolff J, Pauling J, Keck A, et al: The Economic Impact of Artificial Intel-
ligence in Health Care: Systematic Review. J Med Internet Res 22:
e16866, 2020
21. Mak K-K, Pichika MR: Artificial intelligence in drug development: pres-
ent status and future prospects. Drug Discov Today 24:773-780, 2019
22. Reymond J-L, van Deursen R, Blum LC, et al: Chemical space as a
source for new drugs. MedChemComm 1:30-38, 2010
23. Kadurin A, Nikolenko S, Khrabrov K, et al: druGAN: An Advanced
Generative Adversarial Autoencoder Model for de Novo Generation of
New Molecules with Desired Molecular Properties in Silico. Mol Pharm
14:3098-3104, 2017
24. Born J, Manica M, Oskooei A, et al: Designing Anticancer Drugs From
Transcriptomic Data via Reinforcement Learning editor. In: Schwartz R
(ed): Research in Computational Molecular Biology, Cham: Springer
International Publishing, 231-233, 2020
25. Boehme P, Hansen A, Roubenoff R, et al: How soon will digital end-
points become a cornerstone for future drug development? Drug Discov
Today 24:16-19, 2019
26. Cirillo D, Catuara-Solarz S, Morey C, et al: Sex and gender differences
and biases in artificial intelligence for biomedicine and healthcare. Npj
Digit Med 3:1-11, 2020
27. Saltelli A, Bammer G, Bruno I, et al: Five ways to ensure that models
serve society: a manifesto. Nature 582(7813):482-484, 2020
28. Feijoo F, Palopoli M, Bernstein J, et al: Key indicators of phase transition
for clinical trials through machine learning. Drug Discov Today 25:414-
421, 2020
29. Litster J, Bogle IDL: Smart Process Manufacturing for Formulated Prod-
ucts. Engineering 5:1003-1009, 2019
30. Wang H, Zhou Z, Li Y, et al: Comparison of machine learning
methods for classifying mediastinal lymph node metastasis of non-
small cell lung cancer from 18F-FDG PET/CT images. EJNMMI Res
7:11, 2017
31. Hosny A, Parmar C, Quackenbush J, et al: Artificial intelligence in radi-
ology. Nat Rev Cancer 18:500-510, 2018
32. Choi H: Deep Learning in Nuclear Medicine and Molecular Imaging:
Current Perspectives and Future Directions. Nucl Med Mol Imaging
52:109-118, 2018
33. Liu X, Chen K, Wu T, et al: Use of multimodality imaging and artificial
intelligence for diagnosis and prognosis of early stages of Alzheimer's
disease. Transl Res 194:56-67, 2018
How to Design AI-Driven Clinical Trials in Nuclear Medicine
34. Hall M: Artificial intelligence and nuclear medicine. Nucl Med Commun
40:1-2, 2019
35. Visvikis D, Cheze C, Jaouen V, et al: Artificial intelligence, machine
(deep) learning and radio(geno)mics: definitions and nuclear medicine
imaging applications. Eur J Nucl Med Mol Imaging 46:2630-2637,
2019
36. Duffy IR, Boyle AJ, Vasdev N: Improving PET Imaging Acquisition and
Analysis With Machine Learning: A Narrative Review With Focus on Alz-
heimer's Disease and Oncology. Mol Imaging 18:1536012119869070, 2019
37. Tseng H-H, Wei L, Cui S, et al: Machine Learning and Imaging Infor-
matics in Oncology. Oncology 98:344-362, 2020
38. Hatt M, Laurent B, Ouahabi A, et al: The first MICCAI challenge on PET
tumor segmentation. Med Image Anal 44:177-195, 2018
39. Xiang L, Qiao Y, Nie D, et al: Deep Auto-context Convolutional Neural
Networks for Standard-Dose PET Image Estimation from Low-Dose
PET/MRI. Neurocomputing 267:406-416, 2017
40. Kim K, Wu D, Gong K, et al: Penalized PET reconstruction using deep
learning prior and local linear fitting. IEEE Trans Med Imaging
37:1478-1487, 2018
41. Häggström I, Schmidtlein CR, Campanella G, et al: DeepPET: A deep
encoder-decoder network for directly solving the PET image reconstruc-
tion inverse problem. Med Image Anal 54:253-262, 2019
42. Liu F, Jang H, Kijowski R, et al: Deep Learning MR Imaging-based Atten-
uation Correction for PET/MR Imaging. Radiology 286:676-684, 2018
43. Kang SK, Seo S, Shin SA, et al: Adaptive template generation for amyloid
PET using a deep learning approach. Hum Brain Mapp 39:3769-3778,
2018
44. Suk H-I, Lee S-W, Shen D: Latent feature representation with stacked auto-
encoder for AD/MCI diagnosis. Brain Struct Funct 220:841-859, 2015
45. Huang C, Clayton EA, Matyunina LV, et al: Machine learning predicts
individual cancer patient responses to therapeutic drugs with high accu-
racy. Sci Rep 8:16444, 2018
46. Weiss J, Hoffmann U, Aerts HJWL: Artificial intelligence-derived imag-
ing biomarkers to improve population health. Lancet Digit Health 2:
e154-e155, 2020
47. MachinE Learning Ledger Orchestration for Drug DiscoverY | MEL-
LODDY Project | H2020 | CORDIS | European Commission. https://cor-
dis.europa.eu/project/id/831472
48. Makris NE, Huisman MC, Kinahan PE, et al: Evaluation of strategies
towards harmonization of FDG PET/CT studies in multicentre trials:
comparison of scanner validation phantoms and data analysis proce-
dures. Eur J Nucl Med Mol Imaging 40:1507-1515, 2013
49. Treves ST, Lassmann M: International guidelines for pediatric radio-
pharmaceutical administered activities. J Nucl Med 55:869-870, 2014
50. Buvat I, Frey E, Green A, et al: Quantitative Nuclear Medicine Imaging:
Concepts. Requirements and Methods. Hum Health Rep: 9, 2014.
http://lup.lub.lu.se/record/4390492
51. Boellaard R, Delgado-Bolton R, Oyen WJ, et al: FDG PET/CT: EANM
procedure guidelines for tumour PET imaging: version 2.0. Eur J Nucl
Med Mol Imaging: 42, 2015. https://doi.org/10.1007/s00259-014-
2961-x
119
52. Zimmerman BE, Grošev D, Buvat I, et al: Multi-centre evaluation of
accuracy and reproducibility of planar and SPECT image quantification:
An IAEA phantom study. Z Für Med Phys 27:98-112, 2017
53. Schwarz SW, Decristoforo C, Goodbody AE, et al: Harmonization of
United States, European Union and Canadian First-in-Human Regula-
tory Requirements for Radiopharmaceuticals—Is This Possible? J Nucl
Med 118:209460, 2018. Sep 27jnumed
54. Mada MO, Hindmarch P, Stirling J, et al: Competencies and training of
radiographers and technologists for PET/MR imaging - a study from the
UK MR-PET network. Eur J Hybrid Imaging 4:1, 2020
55. Zhang J, Jacko R, Vu T, et al: Artificial Intelligence Integration in Cloud-
based Real-time Data Quality Assurance for Multi-Institutional Clinical
Trials. J Nucl Med 59(supplement 1):1280, 2018
56. Zou D, Ma L, Yu J, et al: Biological Databases for Human Research.
Genomics Proteomics Bioinformatics 13:55-63, 2015
57. Global Alliance for Genomics and Health. GENOMICS. A federated eco-
system for sharing genomic, clinical data. Science 352(6291):1278-
1280, 2016
58. Blomberg N, Consortium E: ELIXIR position paper on FAIR data man-
agement in the life sciences. F1000 Research 6, 2017. https://
f1000research.com/documents/6-1857
59. The Alliance of Genome Resources: Building a Modern Data Ecosystem
for Model Organism Databases. Genetics 213:1189-1196, 2019
60. Wilkinson MD, Dumontier M, Aalbersberg I, et al: The FAIR Guiding
Principles for scientific data management and stewardship. Sci Data 3,
2016. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4792175/
61. Linden M, Prochazka M, Lappalainen I, et al: Common ELIXIR Service
for Researcher Authentication and Authorisation. F1000 Research: 7,
2018. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6124379/
62. Bowles KH, Potashnik S, Ratcliffe SJ, et al: Conducting Research Using
the Electronic Health Record Across Multi-Hospital Systems: Semantic
Harmonization Implications for Administrators. J Nurs Adm 43:355-
360, 2013
63. Geneviève LD, Martani A, Mallet MC, et al: Factors influencing harmo-
nized health data collection, sharing and linkage in Denmark and Swit-
zerland: A systematic review. PLOS ONE 14:e0226015, 2019
64. Dondi M, Kashyap R, Pascual T, et al: Quality Management in Nuclear
Medicine for Better Patient Care: The IAEA Program. Semin Nucl Med
43:167-171, 2013
65. Dondi M, Torres L, Marengo M, et al: Comprehensive Auditing in
Nuclear Medicine Through the International Atomic Energy Agency
Quality Management Audits in Nuclear Medicine (QUANUM) Program.
Part 1: the QUANUM Program and Methodology. Semin Nucl Med
47:680-686, 2017
66. Frangi AF, Tsaftaris SA, Prince JL: Simulation and Synthesis in Medical
Imaging. IEEE Trans Med Imaging 37:673-679, 2018
67. Rigby MJ: Ethical Dimensions of Using Artificial Intelligence in Health
Care. AMA J Ethics 21:121-124, 2019
68. Barredo Arrieta A, Díaz-Rodríguez N, Del Ser J, et al: Explainable Artifi-
cial Intelligence (XAI): Concepts, taxonomies, opportunities and chal-
lenges toward responsible AI. Inf Fusion 58:82-115, 2020