Professional Documents
Culture Documents
14, 2019
STATE-OF-THE-ART REVIEW
Partha Sardar, MD,a J. Dawn Abbott, MD,a Amartya Kundu, MD,b Herbert D. Aronow, MD,a Juan F. Granada, MD,c
Jay Giri, MD, MPHd,e
ABSTRACT
Access to big data analyzed by supercomputers using advanced mathematical algorithms (i.e., deep machine learning)
has allowed for enhancement of cognitive output (i.e., visual imaging interpretation) to previously unseen levels and
promises to fundamentally change the practice of medicine. This field, known as “artificial intelligence” (AI), is making
significant progress in areas such as automated clinical decision making, medical imaging analysis, and interventional
procedures, and has the potential to dramatically influence the practice of interventional cardiology. The unique nature of
interventional cardiology makes it an ideal target for the development of AI-based technologies designed to improve
real-time clinical decision making, streamline workflow in the catheterization laboratory, and standardize catheter-based
procedures through advanced robotics. This review provides an introduction to AI by highlighting its scope,
potential applications, and limitations in interventional cardiology. (J Am Coll Cardiol Intv 2019;12:1293–303)
© 2019 Published by Elsevier on behalf of the American College of Cardiology Foundation.
From the aCardiovascular Institute, Warren Alpert Medical School at Brown University, Providence, Rhode Island; bDivision of
Cardiovascular Medicine, University of Massachusetts Medical School, Worcester, Massachusetts; cCardiovascular Research
Foundation, Columbia University Medical Center, New York, New York; dPenn Cardiovascular Outcomes, Quality and Evaluative
Research Center, Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania; and the
e
Cardiovascular Medicine Division, University of Pennsylvania, Philadelphia, Pennsylvania. Dr. Granada has received institutional
grant/research support (to Skirball Center for Innovation) from Abbott Vascular, Amaranth Medical, Amber Medical, Amgen,
Baylis, BIO2 Medical, Bristol-Myers Squibb, Boston Scientific, Cagent Vascular, Caliber Therapeutics, Cephea, Columbia Medical,
Corindus Vascular, Celyad, Freudenberg Medical, Intact Vascular, JenaValve, Keystone Heart, LimFlow Medical, LoneStar Heart,
Marvel Medical, Medtronic, Meril Life Sciences, MicroVention, Motus GI, Navigate Cardiac Structures, New York University,
OrbusNeich Medical, SoundBite Medical, Spectranetics, Toray Industries, Vetex Medical, Volcano (Philips), and Zimmer Biomet.
Dr. Giri has served on an advisory board for AstraZeneca; and has received research support to the institution from Recor Medical
and Abbott Vascular. All other authors have reported that they have no relationships relevant to the contents of this paper to
disclose.
Manuscript received August 8, 2018; revised manuscript received February 26, 2019, accepted April 2, 2019.
However, some successes have already been seen pre-specified environment to maximize a reward
with supervised learning with models developed (1,2) (Figure 1). The program thus identifies the
to successfully improve heart failure readmission appropriate behavior using a “reward criteria” to
prediction and accurately grade echocardiographic influence the decision-making process. It attempts to
mitral regurgitation (8). accomplish a task (e.g., driving a car, inferring
medical decisions) while learning from its own suc-
UNSUPERVISED LEARNING. Unsupervised learning
cess and failures (2,6). The main aim of reinforce-
seeks to discover underlying structure or relation-
ment learning is to maximize the accuracy of
ships among variables in a dataset (Figure 1). The
algorithms using trial and error. Many clinical
training of dataset is conducted without any spe-
problems can be formatted to fit the format of a
cific labels, and the algorithm clusters data to
reinforcement learning problem. Hence, reinforce-
reveal underlying patterns. It seeks to identify
ment learning algorithms may be used to aid clinical
novel disease mechanisms, genotypes, or pheno-
decision making, intelligently segment medical im-
types from hidden patterns present in the data
aging data, and select personalized medications
(2,7). Unsupervised learning algorithms including
(1,2). Application of reinforcement learning to med-
artificial neural networks that analyzed surface
icine and cardiology thus far has been limited.
echocardiograms have already been found to be
Studies have demonstrated promising results with
useful in the automated discrimination of hyper-
reinforcement learning in optimization of treatment
trophic cardiomyopathy from physiological hyper-
decisions for chronic illnesses and recommending
trophy seen in athletes (10). Shah et al. (11) used
mechanical ventilator weaning protocols that have
agglomerative hierarchical clustering, a commonly
led to superior clinical outcomes (7).
used unsupervised learning tool for analysis of
phenotypic data, and the phenomapping resulted in NATURAL LANGUAGE PROCESSING. NLP, a conflu-
a novel classification of heart failure with preserved ence of AI and linguistics, focused upon developing a
ejection fraction. computer’s ability to understand human language (6)
(Central Illustration, Table 1). ML and DL have some
REINFORCEMENT LEARNING. Reinforcement learning, overlap with NLP; however, NLP has a strong lin-
based on behavioral psychology, uses an alternate guistics component (not represented in images) that
approach where a software program acts in a requires an understanding of how we use language.
1296 Sardar et al. JACC: CARDIOVASCULAR INTERVENTIONS VOL. 12, NO. 14, 2019
Applications of artificial intelligence (AI) includes AI-guided diagnosis, image interpretation, clinical decision support, data driven therapeutics, research
and development, population health, efficient administration, workflow and regulation, and AI-assisted interventional procedures. CT-FFR ¼ computed
tomography fractional flow reserve; EHR ¼ electronic health record; IVUS ¼ intravascular ultrasound; OCT ¼ optical coherence tomography.
interventions (19). This system integrates the cor- CEREBRIA-1 (Machine Learning vs Expert Human
egistration of angiographic images and echo images Opinion to Determine Physiologically Optimized
into the workflow via ML-based probe detection and Coronary Revascularization Strategies), evaluated
automated registration updates, enabling clinical whether an ML algorithm based on computational
teams to identify soft tissue–based structures that are interpretation of pressure-wire pull back data would
provided directly from the integrated ultrasound be similar to expert human interpretation for treat-
system. TrueFusion can enable echocardiographers ment strategies in patients with stable coronary ar-
and interventionalists to better communicate and tery disease (20). The study included 1,008
achieve more intuitive anatomical orientation during instantaneous wave-free ratio pullback traces,
challenging procedures. This may result in reduced including 317 duplicates, which were analyzed by
contrast usage, procedure time, and radiation expo- both the ML algorithm and a multinational team of
sure (19). In the future, via DL, automated diagnosis interventionalists. They found that the computer-
of imaging-based pathologies may be possible inde- based ML program was noninferior to the expert
pendent of an imaging specialist. consensus decision for both appropriateness of PCI
and determining PCI strategy (20).
CLINICAL DECISION SUPPORT. Clinical decision IBM’s Watson for Health applies cognitive tech-
support systems with cognitive computing are under nology to extract and analyze information from the
development and include self-learning systems using EMR, lab reports, imaging reports, the published
ML, pattern recognition, and NLP to mimic human medical reports, guidelines, and various Internet
thought processes (Figure 2). A multinational study, sources (2,3). This technology combines ML and
1298 Sardar et al. JACC: CARDIOVASCULAR INTERVENTIONS VOL. 12, NO. 14, 2019
Artificial intelligence-enabled future catheterization laboratory with clinical decision support system, voice-powered virtual assistant,
augmented reality platforms, and semiautonomous/autonomous robotic system.
systems neuroscience to build powerful, general- anxiety and stress (22). The FDA-approved True 3D
purpose learning algorithms into neural networks system developed by EchoPixel (Santa Clara, Califor-
that mimic the human brain. The IBM Watson for nia) renders patient-specific anatomy in an intuitive,
Oncology cognitive computing system can provide interactive VR format (23). SentiAR Inc, a St. Louis,
confidence-ranked, evidence-based treatment rec- Missouri–based company, received a $2.2 million
ommendations for cancer. Treatment recommenda- research grant from the National Institutes of Health
tions by Watson and a tumor board were concordant to develop an AR cardiac hologram technology that
in 96% of lung, 81% of colon, and 93% of rectal cancer allows real-time viewing, measurement and manip-
cases (21). Currently, IBM is developing Medical ulation of patient anatomy in a holographic display
Sieve, an automated cognitive assistant for cardiolo- for procedural guidance (24) (Figure 3). Similar
gists and radiologists designed to aid clinical AR systems can be used to overlay important infor-
decision-making. This IBM project has addressed mation required during procedures that is typically
many modalities of cardiac imaging including displayed on multiple monitors stacked around the
automatic detection of coronary stenoses in interventionist.
angiography.
VOICE-POWERED VIRTUAL ASSISTANTS. Voice-powered
virtual assistants, such as Apple’s Siri, Amazon’s
VIRTUAL REALITY, AUGMENTED REALITY, Alexa, and Google’s Assistant, employ AI speech
AND AI recognition that has now improved to the point of
exceeding human accuracy in transcribing conversa-
Although there are technological differences in vir- tion. Compared with typing on keyboards for
tual reality (VR)/augmented reality (AR) and AI, searching data in EMR or online medical information,
combined application of these technologies may be voice is easier, faster, more convenient, and with the
useful for interventional procedures (Figure 2). VR help of a unique voiceprint, can eliminate the need
platforms are currently being used in periprocedural for passwords (25). Voice-powered virtual assistants
planning of structural heart interventions, as well use speech recognition and NLP to “understand”
as pre-procedure patient experiences to decrease and process spoken data for output. AI empowered,
JACC: CARDIOVASCULAR INTERVENTIONS VOL. 12, NO. 14, 2019 Sardar et al. 1299
JULY 22, 2019:1293–303 Artificial Intelligence and Interventional Cardiology
addition of ML does not improve results (2,5). For generative adversarial networks have been used to
instance, Frizzell et al. (33) reported that a number of compensate for this deficiency and synthetically
ML algorithms did not improve prediction of 30-day produce large image datasets, including angiograms
heart failure readmissions compared with more and echocardiograms, at high resolution that could be
traditional prediction models. “As more control is used to help train deep neural networks (34).
ceded to algorithms, it is important to note that these There are concerns that robotization may lead to an
new algorithmic decision-making tools come with no increase in unnecessary interventions, take focus
guarantees of fairness, equitability, or even veracity” away from patient expectations, or exacerbate exist-
(32). Issues likely to arise with the application of ML/ ing socioeconomic biases related to care delivery.
DL on various datasets include: 1) issues with data Wireless connectivity of wearable and implantable
integrity (just a newer version of the classic adage: devices, cloud-based AI technologies, and robots face
“garbage in, garbage out”); 2) issues related to lack of cyber security risks. A proposed solution to mitigate
diversity in training datasets; and 3) an impaired or this issue is a custom-built “health care blockchain”
absent ability to fully evaluate for methodological technology that can store health information on an
bias in analysis. Of particular concern is the latter encrypted digital ledger in order to minimize cyber
issue as it relates to neural networks, which are based security risks (35).
on a ‘‘black box’’ design (1,5). Although the auto- One looming question is whether AI technologies
mated nature of neural networks allows for detection could eventually replace a human interventional
of patterns missed by humans, human scientists are cardiologist. This does not look possible in the fore-
left with little ability to assess how or why the com- seeable future. Current technologies have several
puter discerned such patterns. Human physicians, shortcomings in comparison to activities of the hu-
therefore, must critically evaluate the predictions man brain. Structurally, bio-inspired neural networks
generated by AI and interpret the data in clinically resemble at best the outer layers of the retina or the
meaningful ways (1,8). Large, well-curated datasets visual cortex where images are just sensed or repre-
are required for training of DL algorithms to provide sented (36). The capacity of complex decision making
diagnostic and predictive capabilities. However, the or performing a procedure independently would
lack of large datasets of carefully annotated images be quite arduous for current AI/ML algorithms. AI,
and videos have been limiting across various disci- although widely publicized for its amazing perfor-
plines in medicine including IC. Interestingly, mance, is actually quite shallow in intelligence in its
1302 Sardar et al. JACC: CARDIOVASCULAR INTERVENTIONS VOL. 12, NO. 14, 2019
current form (1,36). Robotic systems geared towards AI, physicians should seek to expand involvement in
the technical performance of cardiovascular in- local, national, or international clinical data regis-
terventions are far too primitive to operate indepen- tries. As data cleaning techniques improve, registries
dently on patients. In the aviation industry, pilots could become linked to expand their utility and in-
embraced autopilot technology; however, most pas- crease the availability of clinical, genomic, proteomic,
sengers would not want to do away with pilots. radiographic, and angiographic data available for AI-
Similarly, we anticipate AI will assist rather than based analysis. Interventionalists have the clinical
replace the human operator in the catheterization insight that can guide data scientists and engineers to
laboratory. The societal and ethical complexities of answer the right questions with the right data;
applications of AI require further reflection, proof of whereas engineers can provide automated, compu-
their medical utility, economic valuing, and devel- tational solutions to data analytics problems that
opment of interdisciplinary strategies for their wider would otherwise be too costly or time-consuming for
application (37). manual methods. If appropriately developed and
implemented, AI has the potential to revolutionize
SYNERGIZING IC AND AI the way IC is taught and practiced.
REFERENCES
1. Shameer K, Johnson KW, Glicksberg BS, 5. Hashimoto DA, Rosman G, Rus D, Meireles OR. large cohort of all-comers undergoing percuta-
Dudley JT, Sengupta PP. Machine learning in car- Artificial intelligence in surgery: promises and neous coronary intervention: Comparison of five
diovascular medicine: are we there yet? Heart perils. Ann Surg 2018;268:70–6. contrast media. Int J Cardiol 2018;273:69–73.
2018;104:1156–64.
6. Jiang F, Jiang Y, Zhi H, et al. Artificial intelli- 10. Narula S, Shameer K, Salem Omar AM,
2. Krittanawong C, Zhang H, Wang Z, Aydar M, gence in healthcare: past, present and future. Dudley JT, Sengupta PP. Machine-learning algo-
Kitai T. Artificial intelligence in precision cardio- Stroke Vasc Neurol 2017;2:230–43. rithms to automate morphological and functional
vascular medicine. J Am Coll Cardiol 2017;69: 7. Johnson KW, Torres Soto J, Glicksberg BS, et al. assessments in 2D echocardiography. J Am Coll
2657–64. Artificial intelligence in cardiology. J Am Coll Cardiol 2016;68:2287–95.
3. Syeda-Mahmood T. Role of big data and ma- Cardiol 2018;71:2668–79. 11. Shah SJ, Katz DH, Selvaraj S, et al. Pheno-
chine learning in diagnostic decision support in 8. Al’Aref SJ, Anchouche K, Singh G, et al. Clinical mapping for novel classification of heart failure
radiology. J Am Coll Radiol 2018;15 Pt B:569–76. applications of machine learning in cardiovascular with preserved ejection fraction. Circulation 2015;
disease and its relevance to cardiac imaging. Eur 131:269–79.
4. Henglin M, Stein G, Hushcha PV, Snoek J,
Wiltschko AB, Cheng S. Machine learning Heart J 2019;40:1975–86. 12. Small AM, Kiss DH, Zlatsin Y, et al. Text mining
approaches in cardiovascular imaging. Circ Car- 9. Azzalini L, Vilca LM, Lombardo F, et al. Inci- applied to electronic cardiovascular procedure
diovasc Imaging 2017;10:e005614. dence of contrast-induced acute kidney injury in a reports to identify patients with trileaflet aortic
JACC: CARDIOVASCULAR INTERVENTIONS VOL. 12, NO. 14, 2019 Sardar et al. 1303
JULY 22, 2019:1293–303 Artificial Intelligence and Interventional Cardiology
stenosis and coronary artery disease. J Biomed optimized coronary revascularization strategies. patients presenting with chest pain in the emer-
Inform 2017;72:77–84. Paper presented at: Transcatheter Cardiovascular gency department (abstr). J Am Coll Cardiol 2018;
Therapeutics Symposium (TCT) 18; September 24, 71 Suppl:A225.
13. Sohn S, Ye Z, Liu H, Chute CG, Kullo IJ. Iden-
2018; San Diego, CA.
tifying abdominal aortic aneurysm cases and con- 29. Corindus Receives FDA Clearance for First
trols using natural language processing of 21. Somashekhar SP, Sepúlveda MJ, Norden AD, Automated Robotic Movement in technIQ Series
radiology reports. AMIA Jt Summits Transl Sci Proc et al. Early experience with IBM Watson for for CorPath GRX Platform [press release]. March 5,
2013;2013:249–53. Oncology (WFO) cognitive computing system for 2018. Available at: https://goo.gl/Sb1B2i.
lung and colorectal cancer treatment (abstr). Accessed October 10, 2018.
14. Du T, Xie L, Liu X, et al. Intelligent recognition
J Clin Oncol 2017;35:15 Suppl:8527.
of coronary angiography by deep learning tech- 30. Two Industry Leaders Make for One Extraor-
nology: a novel computer-aided diagnostic sys- 22. Project Brave Heart: Studying the Impact of dinary Verb. Verb Surgical website. Available at:
tem. Paper presented at: Transcatheter Virtual Reality Preparation and Relaxation Therapy. http://www.verbsurgical.com/about/. Accessed
Cardiovascular Therapeutics Symposium (TCT) 18; Lucile Packard Children’s Hospital Stanford web- October 19, 2018.
September 21–25, 2018; San Diego, CA. site. Available at: http://www.stanfordchildrens.
31. Elson DS, Cleary K, Dupont P, Merrifield R,
org/en/innovation/virtual-reality/anxiety-research.
15. Ciusdel C, Turcea A, Puiu A, et al. An artificial Riviere C. Medical robotics. Ann Biomed Eng 2018;
Accessed October 10, 2018.
intelligence based solution for fully automated 46:1433–6.
cardiac phase and end-diastolic frame detection 23. Silva JNA, Southworth M, Raptis C, Silva J.
32. Beam AL, Kohane IS. Big data and
on coronary angiographies. Paper presented at: Emerging applications of virtual reality in cardio-
machine learning in health care. JAMA 2018;319:
Transcatheter Cardiovascular Therapeutics Sym- vascular medicine. J Am Coll Cardiol Basic Trans
1317–8.
posium (TCT) 18; September 21–25, 2018; San Science 2018;3:420–30.
Diego, CA. 33. Frizzell JD, Liang L, Schulte PJ, et al. Predic-
24. Improving Visualization and Interaction During
tion of 30-day all-cause readmissions in patients
16. Molony D, Hosseini H, Samady H. Deep IVUS: a Transcatheter Ablation Using A Mixed Reality
hospitalized for heart failure: comparison of ma-
machine learning framework for fully automatic System: First-In-Human Experience. SentiAR
chine learning and other statistical approaches.
IVUS segmentation. Paper presented at: Trans- Technology website. Available at: https://sentiar.
JAMA Cardiol 2017;2:204–9.
catheter Cardiovascular Therapeutics Symposium com/#!info-graph. Accessed October 10, 2018.
(TCT) 18; September 21–25, 2018; San Diego, CA. 34. Topol EJ. High-performance medicine: the
25. Steinhubl SR, Topol EJ. Now we’re talking:
convergence of human and artificial intelligence.
17. Fairbairn TA, Nieman K, Akasaka T, et al. Real- bringing a voice to digital medicine. Lancet 2018;
Nat Med 2019;25:44–56.
world clinical utility and impact on clinical 392:627.
decision-making of coronary computed tomogra- 35. Ichikawa D, Kashiyama M, Ueno T.
26. Boone D. Cardiovascular disease and percuta-
phy angiography-derived fractional flow reserve: Tamper-Resistant Mobile Health Using Block-
neous coronary intervention in hematological
lessons from the ADVANCE registry. Eur Heart J chain Technology. JMIR Mhealth Uhealth 2017;5:
malignancies: knowledge is power. Paper pre-
2018;39:3701–11. e111.
sented at: Transcatheter Cardiovascular Thera-
18. Fearon WF, Achenbach S, Engstrøm T. Accu- peutics Symposium (TCT) 18; September 21–25, 36. Sengupta PP, Adjeroh DA. Will artificial intel-
racy of fractional flow reserve derived from coro- 2018; San Diego, CA. ligence replace the human echocardiographer?
nary angiography. Circulation 2019;139:477–84. Circulation 2018;138:1639–42.
27. Ganim N, Monlezun D, Balan P. Racial dispar-
19. Siemens Healthineers. Siemens Healthineers ities in procedural complications after trans- 37. He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K.
Receives FDA Clearance for TrueFusion Structural catheter aortic valve replacement (TAVR): a The practical implementation of artificial intelli-
Heart Disease Feature. September 6, 2017. Available retrospective single-center case-control study. gence technologies in medicine. Nat Med 2019;25:
at: https://usa.healthcare.siemens.com/news/ Paper presented at: Transcatheter Cardiovascular 30–6.
fdaclearanceoftruefusionfeature.html. Accessed Therapeutics Symposium (TCT) 18; September
October 19, 2018. 21–25, 2018; San Diego, CA.
20. Davies J. CEREBRIA-1: machine learning vs 28. Lindholm D, Holzmann M. Machine learning KEY WORDS artificial intelligence,
expert human opinion to determine physiologically for improved detection of myocardial infarction in interventional cardiology