@ ESC European Heart Journal (2024) 45, 49204934 FASTTRACK - CLINICAL RESEARCH

European Society httpsi/doi.org/10.1093/eurheartj/ehae595 Arrhythmias

of Cardiology

Prediction of incident atrial fibrillation using

deep learning, clinical models, and polygenic
scores
Gilbert Jabbour ® 1’2’3, Alexis Nolin-LapaIme1’z’3’4, Olivier Tastetm,
Denis Corbinm, Paloma Jorda ® 1’2, Ac 1’3, Jacques Delfratem,
David Busseuil', Julie G. Hussin ® 1245 Marie-Pierre Dubé ® ""%%,
Jean-Claude Tardif ® 1:2.56 |éna Rivard ® 2, Laurent Macle © "2,
Julia Cadrin-Tourigny ® "2, Paul Khairy ® ""*, Robert Avram ® ?3+t,
and Rafik Tadros ® "2+t
"Montreal Heart Institute Research Centre, 5000 Belanger St, Montreal, Quebec H1T 1C8, Canada; 2Facuby of Medicine, Université de Montréal, 2900 Edouard Montpetit Bivd, Montreal, Quebec
H3T 1)4, Canada; *HeartWiseA, 5000 Belanger St, Montreal, Quebec HIT 1C8, Canada; *Quebec Artificial Intelligence Institute (MILA), Montreal, Quebec, Canada; *Université de Montréal
Beauleus Saucier Pharmacogenomics Center, Montreal, Quebec H1T 1C8, Canada; and “Montreal Health Innovations Coordinating Genter, 5000 Belanger St, Montreal, Quebec H1T 1C8, Canada
Received 15 Juiy 2024; revised 8 August 2024; accepted 21 August 2024; online publish-ahead-of
rit 1 September 2024

See the editorial comment for this article ‘Another piece in the puzzle of atrial fibrillation risk: clinical, genetic, and
electrocardiogram-based artificial intelligence’, by S. Kany et al., https://doi.org/10.1093/eurheartjlehae691.

Abstract

Background and Deep learning applied to electrocardiograms (ECG-A) is an emerging approach for predicting atrial fibrillation or flutter
Aims (AF). This study introduces an ECG-Al model developed and tested at a tertiary cardiac centre, comparing its performance
with clinical models and AF polygenic score (PGS).
Methods Electrocardiograms in sinus rhythm from the Montreal Heart Institute were analysed, excluding those from patients with
pre-existing AF. The primary outcome was incident AF at 5 years. An ECG-Al model was developed by splitting patients
into non-overlapping data sets: 70% for training, 10% for validation, and 20% for testing, The performance of ECG-AI, clinical
models, and PGS was assessed in the test data set. The ECG-Al model was externally validated in the Medical Information
Mart for Intensive Care-IV (MIMIC-IV) hospital data set.
Results A total of 669 782 ECGs from 145 323 patients were included. Mean age was 61 + 15 years, and 58% were male. The p
mary outcome was observed in 15% of patients, and the ECG-Al model showed an area under the receiver operating char-
acteristic (AUC-ROC) curve of .78. In time-to-event analysis including the first ECG, ECG-Al inference of high riskidentified
26% of the population with a 4.3-fold increased risk of incidentAF (95% confidence interval: 4.02—4.57). In a subgroup ana-
Iysis of 2301 patients, ECG-Al outperformed CHARGE-AF (AUC-ROC = 62) and PGS (AUC-ROC = 59). Adding PGS and
CHARGE-AF to ECG-Al improved goodness of fit (likelihood ratio test P <.001), with minimal changes to the AUC-ROC
(:76-77). In the external validation cohort (mean age 59 18 years, 47% male, median follow-up 1.1 year), ECG-Al model
performance remained consistent (AUC-ROC = 77)
provides an accurate tool to predict new-onset AF in a tertiary cardiac centre, surpassing clinical an

* Corresponding authors. Emails rafi adros@umontrealca (R.T.; robert.awammd@gmail.com (RA)

t These authors jointly supervised the study and share senior authorship.
© The Author(s) 2024, Published by Oxford University Press on behalf of the European Society of Cardiology.
Thisis an Open Access articke distributed under the terms of the Creative Commons Attribution-NonCommercial License (https:/creativecommons orgficenses/by-nc/4.0), which permits
non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact reprints@oup.com for reprints and
translation rights for reprints. Allother permissions can be obtained through our RightsLink service viathe Permissions ink o the article page on our site—for further information please contact
journals permissions@oup.com.
Prediction of atrial fibrillation using deep learning 4921

Structured Graphical Abstract

Key Question
Can an ECG-based deep learning model (ECG-AI) predict 5-year incident atrial fibrillation or atrial flutter (AF) in a tertiary cardiac
centre? How does the performance of ECG-Al compare to clinical prediction models and polygenic scores (AF-PGS)?
Key nding
ECG-Al demonstrated good performance in predicting incident AF overall and across patient subgroups. While ECG-Al outperformed
clinical models and AF-PGS, adding clinical and PGS prediction to ECG-Al minimally changed the AUC but improved goodness-of-it.
ECG-Al showed consistent performance inan external cohort.
Take Home Message
ECG-Al outperforms existing clinical and polygenic risk scores in predicting new-onset AF in a tertiary cardiac centre population.
ECG-Al provides a clinically useful tool to identify patients who may benefit from more intensive AF screening to help prevent
AF-related complications.

ims Development cohort External validation

Develop an ECG-based deep | Montreal Heart Institute - Beth Israel Deaconess

learning model (ECG-AI) to S Montreal, Canada (MHI) fi%fi Medical Center, Boston,
A GTHIER,
predict 5-year incident atrial
fibrillation or atrial flutter
(AF) in a cardiac care setting
8 145 323 patients ‘ 109 870 patients
Compare ECG-Al to clinical
and polygenic scores (PGS) —~|~ 669782 ECG N~ 437323ECG

Key findings

@ 03
ECG-Al outperformed clinical and polygenic scores in predicting new-onset AF and
showed strong generalizability on MIMIC-IV (AUC-ROC = 0.77; HR = 4.6)
10
08 08

MH o MHIAF-free *° HR =429
AUC-ROC' probability (95% C14.02-457)
04| p<oont
06 02| —ECGAllowrisk
— ECG-Al high risk
[ 00
PGS CHARGE-AF ECG-Al ECG-Al+ ECG-Al+ 0 5 ) 15
PGS PGS+ Years
CHARGEAF

An ECG-Al model trained at the MHI (a tertiary cardiac centre) predicts 5-year incident atrial fibrillation or flutter (AF) in an internal independent
test data set (MHI; AUC-ROC .78) and an external population (MIMIC-IV; AUC-ROC .77). The ECG-Al outperforms existing clinical (CHARGE-AF)
and polygenic scores (PGS). Adding PGS and CHARGE-AF to ECG-Al improved goodness ofit (ikelihood ratio test P <.001), with minimal changes
to the AUC-ROC (.76-77). Created with Biorender.com. HR, hazard ratio; MIMIC-IV, Medical Information Mart for Intensive Care-IV; AUC-ROC,
area under the receiver operating characteristic curve.
Keywords Atrial fibrillation + Deep learning + Electrocardiogram + Polygenic scores

Introduction with a lower risk of adverse cardiovascular outcomes compared with

initial rate-control strategies.” Importantly, it has been estimated that
Atrial fibrillation (AF) is the most common sustained cardiac arrhythmia in about one-third of patients, AF is asymptomatic, contributing to
inadults’ and is associated with an increased risk of stroke, heart failure, under-detection in a large proportion of patients.' Early detection of
cognitive decline, hospitalizations, and death.® Oral anticoagulation AF provides an opportunity to implement measures aimed at the pri-
(OAC) significantly reduces the risk of stroke in patients with AF.”® mary prevention of AF-related morbidity and mortality with an early
Early rhythm-control strategies have also been shown to be associated initiation of appropriate therapy, including OAC when indicated, and
4922 Jabbour et al.

W 2598
£GG vaute
para 004s 18 0yors0772072
o [T e — VicY
159608 s
1,430,107 ECG N 795,546 ECG.
Postop <30
o [E—
Vs Notin s
Excluded ECG Not in sinus rhythm fostt

[—— s e
167,509 ECG ECGs with baseline AF

o sece
Development set

145333 No oollowp 127,936

o< s
“
[ No olowap
Training et Valiacionzet Testoet
iAo
rincee Ai Corary
e AL
s aner
01726 prtes 155 s 2065 s
0570 s
prop— r—
Siserece
7kt g - i g 15571 Siee
preens
[T e T
S0T eee e1155 s S
e |
Figure 1 Electrocardiogram and patient flowchart for the Montreal Heart Institute cohort and the external validation cohort Medical Information
Mart for Intensive Care-IV (MIMIC-IV). A single ResNet-50 model initialized with random weights was trained using the training set. Hyperparameter
tuning was performed using the validation set. The best performing model in the validation set was selected based on the lowest loss, and then, this
model performance was reported on three subgroups within the test set, i.e. ‘MHI All-Comers’, ‘"MHI Hospitalized’, and ‘MHI Biobank'. For the latter
group, after removing patients with missing data, CHARGE-AF and AF-PGS scores were available for 2301 out of the 2370 patients. External validation
was performed in the MIMIC-IV data set from the Beth Israel Deaconess Medical Center in Boston, USA

risk factor modification. Improving the ability to identify individuals at management strategies. Furthermore, the study evaluates the ECG-Al
high risk of developing AF can help define populations in whom more model across diverse populations, including variations in socio-economic
intensive AF screening would be cost-effective.’ status, age, and sex, thereby addressing the important issue of bias in Al
Simple predictive scores for new-onset AF, such as CHARGE-AF applications.>" A comprehensive assessment of the ECG-Al performance
(Cohorts for Aging and Research in Genomic Epidemiology-Atrial using discrimination, calibration, decision curve, and time-to-event ana-
Fibrillation) and HATCH, have been explored for that purpose.’>"* Iyses and confirming its generalizability in an external cohort was con-
Moreover, the CHA;DS,-VASc score, which was developed to predict ducted. Finally, a novel approach of combining ECG-Al, AF-PGS, and
the risk of stroke in AF patients, has also been shown to predict incident traditional clinical prediction in AF risk stratification is assessed.
AF." Recent technological advances in artificial intelligence (Al) have
enabled the development of novel AF prediction models, which have
shown promising performances in predicting new-onset AF using single Methods
or multiple lead sinus rhythm electrocardiograms (ECGs) in primary Study population
care settings or across diverse hospital network populations.’>2°
In a retrospective cohort, all ECGs in the MHI database acquired between
Furthermore, recent studies suggest a good predictive ability of AF 2004 and 2022 were considered. Electrocardiograms were excluded if they
polygenic scores (AF-PGS) ina primarily general population, with a pos- had invalid metadata or waveform (ie. missing derivations or erroneous sig-
sible additive value to clinical AF prediction models.2'">* nals defined by a maximum voltage > 10 mV) or showed no sinus rhythm
In this study, an open-weight ECG-Al model is introduced to predict (see Supplementary data online, Figure S1). Electrocardiograms were also
incident AF at the Montreal Heart Institute (MHI), Canada. The analysis excluded if performed within 30 days of cardiac surgery, were acquired
plan and reporting adhere to key quality criteria recently developed for in patients with pre-existing AF or atrial flutter (using the same definition
clinical Al prediction modelling studies.?*~22 This study stands out by ex- as the outcome, see next section), or were acquired in patients without
ploring innovative risk markers of AF (ECG-Al and AF-PGS) within ater- subsequent follow-up at MHI (Figure ). The remaining ECGs were then
tiary cardiac care institution with a high prevalence of heart failure (HF) randomly split by distributing patients into non-overlapping training
and coronary artery disease (CAD), where AF is more prevalent, is asso- (70%), validation (10%), and test (20%) sets, stratified according to age,
sex, and outcome ensuring balanced distribution of these variables among
ciated with increased risk for complications, and may be mediated by dis- data sets (see Supplementary data online, Tables S1-53). In each set, three
tinct risk markers compared with the general population.””* Moreover, groups were defined (Table 7). The ‘MHI All-Comers' group included all the
unlie most previously published literature on ECG-Al for incident AF ECGs in each set. Two subgroups of the ‘MHI All-Comers' are defined: the
prediction,the presented ECG-Almodel is open-weight (i.e. publicly avail- ‘MHI Hospitalized’ group included the ECGs of patients who were hospita-
able) to spur further innovation and improve accessibility in ECG-Al tech- lized at the MHI and the MHI Biobank'included the subset of patients in the
nology, potentially accelerating advancements in AF prediction and MHI hospital biobank, a prospective hospital-based cohort of >20 000
Prediction of atrial fibrillation using deep learning 4923

Table 1 Demographic and clinical characteristics of the development and external validation data sets

MHI All-Comers MHI Hospitalized MHI Biobank MIMIC-IV

"8G 69782 475986 104850 437323
Patients 145323 76 680 11622 109870
ECG per patient 20/(Q1:10,Q3:50) 30 (Q1: 20, Q3: 80) 6.0 (Q1:30, Q3:120) 20 (Q1: 1.0, Q3: 40)
Patient-level data
Age (years) 613(x 152) 643 (+ 13.7) 632 (x 116) 592 (+179)
Male 84087 (57.9%) 50114 (65.4%) 7326 (63.0%) 51627 (47%)
cMD 32(x 14) 32(x13) 32(x14)
MHI Hospitalized 76680 (52.8%) 76 680 (100.0%) 7919 (68.1%)
Follow-up (years) 32/(Q1:.3,Q3:86) 40(Q1: 3,Q3:97) 96 (Q1: 46, Q3:136) 141 (Q1:.03, Q3: 47)
5-year incident AF 22695 (15.6%) 18492 (24.1%) 2846 (24.5%) 16610 (15.1%)
ECG-level data
Age (years) 628 (+ 148) 642 (+ 14.1) 640 (120 612 ( 165)
Follow-up (years) 42 (Q1:12,Q3:83) 43 (Q1:11,Q3: 86) 66 (Q1:32, Q3:105) 17 (Q1:.2, Q3: 45)
5-year incident AF 80183 (12.0%) 70230 (14.8%) 13772 (131%) 65301 (14.9%)
Years to incident AF 20 (Q1: 1,Q3:5.6) 19 (Q1:.1,Q3:55) 33(Q1:7,Q3:70) 11(Q1:.1,Q3:33)

"MHI All-Comers' group includes allthe development set electrocardiograms."MHI Hospitalized’ group includes the electrocardiograms of patients who have been hospitalized at the MHI
"MHI Biobank’includes the subsetof patients in the Montreal Heart Institute hospital biobank "MIMIC-IV"includesall elgible patients and electrocardiograms of the external validation data
set. The number of electrocardiograms per patient, folow-up duration, and time to incident atrial fibrillation are provided in quartiles format, indicating the median, 25th percentile (Q1),
and 75th percentile (Q3). Age and Canadian Index for Multiple Deprivation (CIMD) are presented as mean + standard deviation

participants of which 16876 have available genotypic data. Detailed co- per lead at a sampling rate of 250 Hz. Muttiple ECG recordings from
morbidities are only reported for the ‘MHI Hospitalized” group, in whom the same patient were independently fed into the training model.
dinical diagnoses were ascertained using International Classffication of Hyperparameters were optimized on the validation set using a Bayesian
Diseases (ICD) codes (see Supplementary data online, Table $4). grid-search approach. The best performing model in the validation set
was selected based on the lowest loss, and then, this model performance
Outcome was reported on three subgroups within the internal MHI test set,
The primary outcome, termed ‘incident AF, included new-onset AF or atrial ie. ‘MHI All-Comers, "MHI Hospitalized, and ‘MHI Biobank'. Using
flutter. Incident AF at 5 years was modelled as binary outcome and was de- Tensorflow's GradientTape (version 29.1), the gradient of the model’s
termined based on avallable outpatient and inpatient clinical and prediction was computed with respect to the input ECG sample, resulting
medico-administrative databases and ECG diagnoses, which incorporated in a saliency map that highlights
the most influential parts of the ECG signal,
ECG acquisitions, hospitalization records, emergency room visits, AF clinic thereby providing explainability.*** The ECG-Al development details are
visits, and electrophysiology procedures (see Supplementary data online, provided in the Supplementary data online, Note S1.
Figure $2). The sensitivity and specificity of this definition of incident AF
Were assessed in 200 randomly selected patients using manual chart reviews Clinical risk models
as a gold standard. The same clinical and administrative databases were used Four different clinical risk models were tested, including ‘Age & Sex,
as eligible follow-up encounters to establish maximum follow-up time, with HATCH, CHA,DS,-VASc, and CHARGE-AF (details in Supplementary
censoringat the date of last follow-up at MHI, heart transplantation, or death. data online, Tables S5 and $6)."*"* Clinical risk scores were incorporated
into logistic regression (LR) models, which were fitted using the training
Electrocardiogram acquisition and validation sets. The CHARGE-AF score was only calculated for patients
Electrocardiograms were retrieved in XML format using the MUSE in the MHI biobank cohort at the time of inclusion in the biobank, since
Cardiology Information System (GE Healthcare, Chicago, IL). Each XML some components were not available in the other patient subgroups. The
file contains data for 12 ECG derivations, with each derivation capturing ECG-Al prediction based on the single ECG closest to inclusion in the
voltage readings over a 10 s period sampled at 250 Hz. Each voktage was MHI biobank (and CHARGE-AF calculation) was used to compare the pre-
standardized by removingthe mean and scalingto unit variance of the train- dictions of ECG-Al with those of CHARGE-AF.
ing set population voltages. Since this scaling method is inherently sensitive
to outliers, ECGs with extreme voltage values (>10 mV) were considered Polygenic score calculation in the Montreal
to be outliers and discarded from the data set. Heart Institute biobank
The predictive ability of AF-PGS, both alone and in combination with
Electrocardiogram-based deep learning model ECG-Al and CHARGE-AF, was assessed in the MHI biobank cohort. The
A single ResNet-50 model*? initialized with random weights was trained in previously published AF-PGS from Khera et al’® (PGS catalogue ID
the training set using four A6000 GPUs (NVIDIA, Santa Clara, CA, USA). PGS000016) was converted to GRCh38 genomic build. The MHI biobank
The model receives a single 12-lead ECG as input, with a duration of 10's cohort previously underwent array genotyping on the llumina Global
4924 Jabbour et al.

Screening Array followed by standard genotypic quality control and Confidence intervals (Cls) are reported using bootstrapping with 1000
genome-wide imputation on the TOPMed reference panel. The AF-PGS iterations. For normally distributed data, results are presented as mean +
was computed using weights from PGS000016, including 6 502 964 single- standard deviation. For non-normally distributed data, results are presented
nucleotide polymorphisms out of 6 730 541 in the original score (97%). The using quartiles. The DeLong method was used to statistically compare the
raw AF-PGS was standardized using a standard scaler, followed by a logistic ROC curves of different predictive models.*” To evaluate the improvement
transformation to convertthe values toa range between Oand 1. The single inthe model’s goodness of fit with the addition of new variables to ECG-Al,
ECG dlosest to enrolment in the MHI biobank was used to compare a log-likelihood ratio test (LRT) was conducted.
ECG-Al predictions with AF-PGS predictions. Data analysis and visualization were performed using Python (version
3.8) with the followinglibraries: scikit-learn (version 1.3.2), Iifelines (version
Statistical analysis, performance metrics, and 0.27.8), matplotiib (version 3.7.5), and seaborn (version 0.13.2).
reporting
After ECG-Al training, a LR model was used to integrate ECG-Al probability Subgroup analyses
predictions with clinical and polygenic scores. The LR model was fitted on The study aimed to ensure that ECG-Al performance remained consistent
the training and validation sets and tested on the test set. Model across dverse patient populations. For this purpose, pre-defined subgroup ana-
performance was assessed using several metrics in the test set. Iyses were performed by stratifyingthe testing data set by sex (male and female),
Discrimination performance is reported using the area under the receiver age (<65 and 265 years), and socio-economic status. The latter was assessed
operating characteristic (AUC-ROC) curve, the area under the preci- usingthe Canadian Index for Muttiple Deprivation (CIMD), a measure of socio-
sion-recall curve (PRC), and the diagnostic odds ratio (DOR). The DOR economic conditions based on the 2021 Canadian Census of Population micro-
is the ratio of the odds of disease in test positives relative to the odds of dataand derived using patient postal codes.™ The composite summary score
disease in test negatives.”® ranges from 1 to 5, with 1 representing the least deprived and 5 representing
Calibration was assessed using calibration curves by fitting a spline to the the most deprived. Furthermore, the performance of ECG-Al was tested in
alibration data using the UnivariateSpline function from the SciPy Python subgroups with and without the two most common cardiac conditions, name-
library (version 1.10.1) with a smoothing factor of 1. To quantify the calibra- Iy, HFand CAD, defined using ICD codes inthe MHI Hospitalized" subgroupof
tion performance, the estimated calibration index (ECI) was computed as the test data set (see Supplementary data online, Table S4).
the root mean squared difference between the mean predicted probabil-
ities and the spline-fitted calibration curve.*” External validation
Decision analysis curves (DCAS) were constructed by plotting net bene-
fit: (NB) against various threshold probabilties, considering both discrimin- To investigate the generalizability of the ECG-Al model outside the MHI, an
ation and calibration”® The NB was computed at different decision
externalvalidation analysis was performed using the Medical Information Mart
thresholds as follows, where N is the total number of samples and t is for Intensive Care (MIMIC)-1V, a large de-identified data set of patients admit-
ted to the emergency department or an intensive care unit at the Beth Israel
the threshold probability:
Deaconess Medical Center in Boston, USA.*'™*? Similar ECG inclusion/exclu-
sion criteria from the MHI data set were used. Electrocardiogram voltages
o= 1- ()65 were standardized using a standard scaler, adjusting for the external validation
set by removing the mean and scaling to unit variance. Incident AF at 5 years
Inadeployment scenario, a predictive model would be used to guide down- (the primary outcome) was modelled as a binary outcome and determined
stream intensive AF screening in high-risk populations. Therefore, the NB based on ECG and hospitalization diagnoses. Performance of ECG-Al in
was compared with the default policies of ‘Screen None' or ‘Screen All this external data set was also assessed using AUC-ROC, PRC, calibration,
for AF. The ‘Screen None' NB is O given that TP and FP are 0. ‘Screen DCA, and time-to-event analyses as described above.
All'implies no perceived downsides of over-screening To calculate the
‘Screen All' NB, (TP/N) was replaced by the prevalence and (FP/N) by
(1 = prevalence). For instance, with a 10% AF event rate, ‘Screen All'implies Results
10% correctand 90% incorrect dlassification at agiven threshold. Sensitivity,
specificity, and DOR were calculated at the event rate threshold in each Description of the Montreal Heart
group that maximizes the NB and considered to be an optimal threshold. Institute study population
Finally, time-to-event analyses were conducted in which time 0 was defined Atotal of 669 782 ECGs (47% of the screened ECGs) acquired from 145
as the date of each patient's first ECG. For the MHI Biobank subset, the ECG 323 patients met the inclusion criteria (Figure 7). In the ‘MHI Al-Comers’
closest to biobank enrolment was used. This approach was chosen to simulate
a prospective deployment scenario with the longest possible follow-up and to group, the mean age was 61 15 years and 58% of the patients were
provide a fair comparison with CHARGE-AF, calculated at biobank enrolment. male. Each patient had a median of two ECGs [first quartile (Q1): 1, third
An exploratory analysis was also conducted by choosing the ECG generating quartile (Q3): 5]. The 5-year incident AF outcome was observed in 12.0%
the highest predicted AF probability as time 0. Stratification into high-risk and of ECGsand 15.6% of patients. In avalidation study where allavailable med-
low-risk groups was determined using predictions from the ECG-Al model at ical records were manually retrieved and reviewed for a randomly selected
the chosen optimal classification threshold. Survival curves were estimated subset of 200 patients, the specificity and sensitivity for classifying the pri-
using the Kaplan-Meier (KM) method. The survival distributions between high- mary AF outcome were 100% and 91% (95% Cl: 83.9-986), respectively
risk and low-risk groups were compared using the log-rank test. Hazard ratios (see Supplementary data online, Table $7). The median time to incidert AF
(HR) between the two groups were calculated by fitting a Cox proportional was 2 years (Q1:.1,Q3: 5.6). The MHI Hospitalized' group included 71%
hazards model after verifying proportionality assumptions. of the ECGs and 53% of the patients from the ‘MHI All-Comers’ group
All results are reported on the test set which excludes patients included
in the training and validation sets. The ECG-Al prediction was reported at (Table 1). Clinical characteristics for the ‘MHI Hospitalized” group were
the ECG level, whereby multiple ECG recordings from the same patient comparable among the training, validation, and test sets (Table 2). The
were independently fed into the training model. The ECG-Al prediction MHI cohort had a prevalence of CAD of 71.4% and HF of 13.4%.
was also reported at the patient level by averaging the model's probability Performance results for each of the pre-defined prediction modelsin
outputs for ECGs grouped according to both their 5-year AF outcome and the test setare summarized in Supplementary data online, Table S8, and
the patient's identity (see Supplementary data online, Figure S3). further described below.
Prediction of atrial fibrillation using deep learning 4925

Table 2 Detailed patient comorbidities for MHI Hospi lized group (overall, training, validation, and test subgroups)
and the external validation cohort (MIMIC-IV)

Overall Validation Test MIMIC-IV

“Number of patients 76680 7685 15271 109870
Heart failure 10260 714 999 2047 15273
(134%) (13.4%) (13.0%) (134%) (139%)
Coronary artery disease 54725 38362 5455 10908 25643
714%) 714%) 71.0%) 714%) 233%)
Chronic obstructive pulionary disease 8952 6285 892 1775 10479
(17%) (17%) (1:6%) (116%) ©0.5%)
Hypertension 48285 33777 4805 9703 60705
63.0%) ©29%) (625%) (635%) (553%)
Diabetes 20682 14507 2086 4089 25630
7.0%) 7.0%) 7.1%) (68%) 233%)
Stroke 847 594 B 71 4589
(11%) (1.1%) (1.1%) (11%) @2%)
Dysipidacria 50978 35688 s167 10123 44440
(665%) (664%) (67.2%) (663%) (40.4%)
Obesity 20621 14520 2059 4042 15003
(265%) 7.0%) (65%) (265%) (137%)
Chronic kidney disease 10360 72% 1049 201 15750
(135%) (136%) (136%) (132%) (143%)
Sleep apnoea 448 31 458 902 11101
8% 8% ©.0%) (59%) (10.1%)
Hyperthyroidism 358 255 n 7 1064
(:5%) (5% (4% (5% (1.0%)
Vascular disease 8169 Se94 829 1646 8021
(107%) (106%) (108%) (108%) 7.3%)
MIMICHV, Medical Information Mart for Intensive Care-IV.

Atrial fibrillation prediction in MHI the patient level. Supplementary data online, Table $9, provides the clas-
sification metrics at the patient level for various thresholds.
All-Comers using ECG-based deep
Subgroup analyses are shown in Figure 3. The ECG-Al demonstrated
learning, age, and sex better discrimination performance in female patients (AUC-ROC .77),
The ECG-Al model demonstrated good discriminative ability in the test compared with male patients (AUC-ROC .735; DeLong P <.001). The
set to identify incident AF with significantly better performance com- model did not show significant differences in discrimination perform-
pared with the Age & Sex LR model, with a higher AUC-ROC curve ance across other subgroups, including age, CIMD, follow-up duration,
of .75 (95% Cl: 745-753) vs. .63 (95% Cl: 627-.636; P < 001) and im- and time interval to AF diagnosis (Figure 3).
proved predsion-reall area of 31 (95% Cl:.30-32) vs. .17 (95% Cl: Time-to-event analysis results showed that patients with a high-risk
168-176) (Figure 2). The ECG-Al model also exhibited the best cali- ECG, as predicted by ECG-Al witha threshold probability of >12%, had
bration with an ECI of .086. Adding age and sex to the ECG-Al model significantly lower incident-free probabilties compared with those hav-
as a post-training LR model yielded a similar AUC-ROC (75) and did ing alow-risk ECG with aHR of 4.29 (95% Cl: 4.02-457; P <.001) at an
not provide an overall better fit (LRT statistic < 0, P = 1). At the patient extended follow-up of up to 15 years (Figure 4). In sensitivity analyses,
level, the ECG-Al model showed the highest AUC-ROC of .78 (95% Cl: KM curves were plotted after removing cases where AF was diagnosed
768-783) and precision—recall area of 42 (95% Cl: 41-.44). within 30 days (see Supplementary data online, Figure S4) and within 1
By means of the DCA, ECG-Al model consistently showed the high- year (see Supplementary data online, Figure S5) from the index ECG.
est NB across a range of threshold probabiltties, with significant im- The model also demonstrated consistent discrimination performance
provement over the Age & Sex model, with the highest separation after excluding ECGs with a time to AF diagnosis of <1 year
observed at a probability threshold corresponding to the event rate (Figure 3), which could represent pre-existing paroxysmal AF rather
(ie. 12% at the ECG level and 15% at the patient level). Using a classi- than true incident AF. Time-to-event subgroup analyses showed con-
fication threshold of 12%, the ECG-Al model showed a sensitivity of sistent results when stratifying for age and sex (see Supplementary
66%, a specificity of 75%, and a negative predictive value of 93% at data online, Figure S6).
4926 Jabbour et al.

A ROC Curve B Precision-Recall Curve

0 0 — hge &sex:0.17 (0.168,0.176]
— ECoAr031(0303, 0318)
— ECGAI+ Age & Sex: 031 0306, 0.320)
— ECGAI PatentLevel: 0.42 0,405, 0.436]
o os

Zos
: §os
H &

0z
— Age bsex:063 (0627, 06361
— EcoAr07510745, 0753) 0z
— ECGAI+ Age & Sex: 075 0745, 0754]
00 — ECGAI PatientLevel:0.78 (0.768,0.763]
o o or oe os o w oz o8 o
Flse ositive Rate Recal
[+ Calibration Curve D . Decision Curve Analysis
o — hgeiser
-
e ecow+ agesex
" eco raentieve
ScreanAl
oo - Sceen one

M 010
Z0e
& goos

goe
8 o0

— Age & €61 = 0349 10,349, 0.349) o002

y eCoar ECI= 0086 0050, 01251
,, ECGAI + Age & Sox: EC1 = 0124 (0.095,0155]
ool ¢ — £CO.A PotientLevet £€1 - 0.095 (0,070, 0.136] 000
04 06 o8 0 010 015 oz o 030
Predicted 5 year Risk reshold Probabilty
Figure 2 MHI All-Comers test set (29 065 patients, 135 544 ECG) performance assessment of the four models: (i) Age & Sex logistic regression, (i)
Electrocardiogram-based deep learing (ECG-A), (ii) ECG-Al + Age & Sex, and (iv) ECG-AI patient level. ECG-Al and ECG-Al + Age & Sex overlap in
A,B,and D. (A) The receiver operating characteristic curve, plottingthe true positive rate against the false positive rate for each model, with the area under
the curve indicating discriminatory power and reported i the legend. (B) The precision-recall curve, plotting precision against recall, with the area under the
curve reported in the legend. (C) The calibration curve, showing the relationship between predicted and observed 5-year AF risk; the slope and intercept
are calculated using linear regression, and the curve is plotted using a univariate spline with smoothing factor of 1. The estimated calibration index (ECI,
reported in the legend) is the root mean squared difference between the mean predicted proba s and the spline-fitted calibration curve. (D) The de-
cision curve analysis, plotting net benefit against threshold probability. The ‘Screen All’ line is different for patient-level and ECG-level curve.

Saliency maps highlighted the P-wave area as having the highest influ- achieved an AUC-ROC of 73 (95% Cl: .725-735), indicating superior
ence on the model’s prediction (Figure 5). Signal artefacts and ectopic discrimination compared with the CHA,DS,-VASc (AUC-ROC = 55,
beats appeared to contribute less to the model's prediction (see 95% Cl: .548-558) and HATCH (AUC-ROC=.52, 95% CI:
Supplementary data online, Figures S7-59). 515-524) models (see Supplementary data online, Figure S10).
Similarly, the PRC revealed that the ECG-Al model had the highest pre-
Atrial fibrillation prediction in MHI cision—recall area of .34 (95% Cl: .33-35), outperforming the clinical
Hospitalized patients using ECG-based models. Adding CHA,DS,-VASc or HATCH clinical scores to
ECG-Al after training did not provide an overall better fit on the test
deep learning, HATCH, and set (LRT statistic < 0, P=1). A sensitivity analysis was also performed,
CHA;DS,-VASc restricting cases to those with a follow-up duration of over 1 year,
The ECG-AI model was then compared with the traditional dinical risk which did not significantly impact the discrimination performance of
scores to predict incident AF in the MHI Hospitalized cohort, where clinical risk models and ECG-Al (see Supplementary data online,
clinical data for each patient were adjudicated. The ECG-Al model Table $10). The ECG-Al model performance was similar in different
Prediction of atrial fibrillation using deep learning 4927

A ECG-Level B ECG-Level c ECG-Level

overa [ . overs . overa .
sgenes| i . roenes . roezes .
sge<es| i o rge<es . roe<es -

remate | . remale . remale .

amo<a| i N a2 . awo<2 .
oatyer | . fuz 1year . Fuz1yenr M

Tmetont <300ms | | . TimetoA < 30 days . Timetoss<30dms | ¢ w

b Patient-Level E Patient-Level F
overait | . overall . overail
sgemes| i o raezes . e
nae<es| i [ nae<es - rae<es
e | o wate .

am<z| i . a2 e amo<2 \ ‘.
a2 | i = oz . amon2 i .
s tye | ‘o fux 1year . Fu 1 yen { .
Fuzsyeas | | e FUzsyears . FU =5 years | .
Diagnostic 0dds Ratio ROC AUC PRC AUC

Figure 3 Electrocardiogram-based deep learning (ECG-Al) discrimination performance metrics overall and in subgroups of the MHI All-Comers test
set (29 065 patients, 135 544 ECG) at the ECG level (A-C) and patient level (D—F). (A and D) The diagnostic odds ratio which s calculated as (sensitivity/
(1 = sensitivity))/(specificity/(1 — specificity)) at an optimal threshold of 12% for ECG level and 15% for patient level. (B and E) The receiver operating
characteristic area under the curve (ROC AUC). (C andF) The precision-recall curve area under the curve (PRC AUC). The dashed lines represent
prevalence, indicating
the proportion of true positive cases within the population, important for interpreting precision—recall curve which is sensitive to
class imbalance. Confidence intervals for all metrics were derived from 1000 bootstrap iterations. CIMD, Canadian Index for Mutiple Deprivation; FU,
follow-up)

subgroups including in cases with actionable AF, ie. where OAC would 2301 patients with complete AF-PGS data and CHARGE-AF score
be recommended based on the CHA,DS,-VASc score, with an were included from the test set. The AF-PGS and CHARGE-AF models
ECGlevel AUCROC of 728 (95% Ck 722-734) (see showed poorer discrimination performances, with respective AUC-
Supplementary data online, figure S11). Discrimination performance ROC of 59 (95% Cl: .57-63; Delong P<.001) and .62 (95% Cl:
was reduced in patients with a history of HF (AUC-ROC of .69, 95% 160-65; Delong P <.001) compared with ECG-Al (AUCROC of .76,
CI: .68-70) compared with the overall resuits, while performance was 95% Cl: 74-79) (see Supplementary data online, Figure $13). While
better in patients with a history of CAD, with an AUCROC of .75 the addition of AF-PGS and/or CHARGE-AF to ECG-Al as a post-
(95% CI: .746-758) (see Supplementary data online, Figure S11). The training set yielded similar AUC-ROC compared with ECG-AI, this add-
time-to-event analysis also showed that ECG-Al prediction was asso- ition improved the calibration performance with a reduction of ECI from
ciated with a long-term hazard of developing AF in the ‘MHI 157 (95% Cl: 125-198) using ‘ECG-Al alone’ to 095 (95% Cl:
Hospitalized group (HR 3.0, 95% Cl: 2.79-3.22; P < 001), in patients 1052-147) using ‘ECG-Al + AF-PGS’ and 079 (95% Cl: 046-116) using
with history of CAD (HR 3.49, 95% CI: 3.09-3.95; P < 001) and HF ‘ECG-Al + AF-PGS + CHARGE-AF’ (see Supplementary data online,
(HR 441, 95% Cl: 2.74-7.10; P < .001) (Figure 4A-D), and in patients Figure $13). The LRT further confirmed that the more complex models
without documented diagnoses of HF or CAD (see Supplementary provided a significantly better overall fit (P=.0002 for ECG-Al+
data online, figure $12). AF-PGS; P <0001 for ‘ECG-Al + AF-PGS + CHARGE-AF), compared
to 'ECG-Al alone’. The DCA, which is influenced by both discrimination
Atrial fibrillation prediction in MHI and calibration, also showed an improved NB in models that add AF-PGS
andlor CHARGE-AF to ECG-Al (see Supplementary data online,
Biobank patients using ECG-based deep Figure $13). Time-to-event analyses in the MHI Biobank group showed
learning, AF-PGS, and CHARGE-AF superior HR when ECG-Al was used to stratify patients into high-risk
A single ECG per patient, acquired closest to patient enrolment in the and low-risk groups [HR 451 (95% Cl: 3.76-5.40); P <.001] compared
MHI Biobank, was used to compare ECG-Al predictions with AF-PGS with AF-PGS [HR 185 (95% Cl 144-236) P<.001] and
and CHARGE-AF predictions in the MHI Biobank group. A total of CHARGE-AF [HR 2,50 (95% C: 1.81-3.46); P <.001] (Figure 6).
4928 Jabbour et al.

A‘ N All-Comers B o MHI Hospitalised

ECG-Al High Risk
09 = Eco LowRisk
208 >

Bor ]

fos §
g, 2
o3 HR=3.00 (95% C1 2.79-3.22)
02 o o2 o 20001
B o W B I 3
Years Years
At risk At risk
20751 o151 360 95 o3 s085 2 s19
Ties 2232 w08 B\ sos 1675 i) RS
c 10 cap b 10 HE
09 0s
o8 08
Bo7 Fo7

gos Sos
Sos
o4 <04

o3 HR=3.49 (95°%C1 3.09-3.95) o3 HR=4.41 (95% C1 2.74-7.10)

02 o 20,001 o2 o 20001
B 0 W B I s
Years Years
At risk At risk
Se07 2475 231 300 30 5 3 u
352 1% 33 F 168 3 12
Figure 4 Incident atrial fibrillation—free probability: Kaplan-Meier curves using electrocardiogram-based deep learning (ECG-Al) to stratify patients at
dlassification threshold of 12%. Index electrocardiograms with calculated timeto atrial fibrillation diagnosis of O were removed. Hazard ratios (HR) were
calculated by fitting a Cox proportional hazards model. P-values are calculated using the log-rank test. (4) KM curves of patients in the ‘MHI All-Comers'
group. Only the first electrocardiogram of each patient was used. (B) KM curves of patients in the ‘MHI Hospitalized" group. Only the first electrocar-
diogram of each patient was used. (C) KM curves of patients with a prior history of CAD. Only the first electrocardiogram acquired after the earliest
record of coronary artery disease diagnosis was used. (D) KM curves of patients with a prior history of heart failure. Only the first electrocardiogram
acquired after the earliest record of heart failure diagnosis was used

External validation of the ECG-based deep Discussion

learning model in the Medical Information The development of an ECG-based deep learning model (ECG-A) de-
Mart for Intensive Care-IV data set signed to predict 5-year incident AF risk at an academic cardiac centre
A total of 437323 ECGs recorded in 109870 patients from the was presented. The study aimed to adhere to key quality criteria re-
MIMIC-IV data set were used to externally validate the ECG-Al mod- cently developed for clinical Al prediction modelling studies?*2® The
el. Patients were aged 59 + 18 years, 47% were males, and median ECG-Al demonstrated superior discrimination and calibration per-
follow-up was 1.1 years (Q1: .03, Q3: 4.7) (Table 1). The S-year formance compared with both clinical and polygenic scores
incidence of AF was 15.1%. As shown in Figure 7, ECG-Al demon- (Structured Graphical Abstract). Recent studies reported on the develop-
strated good discrimination, calibration, and net clinical benefit. ment of incident AF prediction models using 12-lead ECG during nor-
Time-to-event analyses demonstrated that the 32% of patients with mal sinus rhythm in a diverse hospital network population.’>™"® To our
ECG-Al predicted high AF risk had a 4.6-fold increased hazard (95% knowledge, this study represents the first attempt to predict incident
Cl 445-474) of developing AF during long-term follow-up AF in a cardiac care centre using an ECG-Al model and comparing it
(P <.001), with consistent results when excluding ECGs with time to both clinical and polygenic scores. A multifaceted performance as-
to AF < 1 year (Figure 8). sessment was conducted, evaluating discrimination, calibration, and
Prediction of atrial fibrillation using deep learning 4929

Figure 5 Saliency maps for two electrocardiogram derivations, Il and V1, which visualize the importance of different segments of the electrocardio-
gram signals in predicting atrial fibrillation using electrocardiogram-based deep learning (ECG-AI). The saliency maps were generated using
TensorFlow’s GradientTape to compute the gradient of the model's prediction with respect to the input electrocardiogram sample, providing explain-
ability. The maps show regions of low to high saliency, indicated by the colour gradient from light (low saliency) to dark (high saliency). The derivations Il
and V1 are shown, with notable high saliency around the P-wave that the model found most relevant for predicting atrial fibrillation

NB using decision curve analysis to comprehensively assess the predict- how AF is recorded across different settings, potentially leading to under-
ive capabilities of the models.2” This is crucial since solely reporting reporting or misclassification of AF cases. The 91% sensitivity of our AF
AUC-ROC can be overly optimistic in the setting of imbalanced data outcome adjudication validation study indicated a modest risk of classifica-
with a relatively small event rate,** and the NB better reflects the impact tion bias (see Supplementary data online, Table S7), which could have also
of clinical implementation. Reassuringly, the ECG-Al model demonstrated negatively impacted the performance of ECG-AI There was no significant
good performance in an independent external validation cohort change in the discrimination performance of clinical risk models in the sen-
(MIMIC-V), supporting the generalizabilty of the model across different sitivity analysis restricting cases tothose with afollow-up duration of over 1
dlinical settings. Importantly, while most published ECG-Al models remain year, suggesting that short-term follow-up bias had a negiigible impact on
proprietary, this study not only shares the open weights of the model but our findings (see Supplementary data online, Table 510). Second, the popu-
also provides the complete code for replicating the validation using the lation in our study, drawn from a cardiac care centre, is significantly
MIMIC-IV data set. This comprehensive approach promotes transparency different from the primarily general population included in the
and reproducibility, allowing researchers to independently validate our meta-analysis.*® Clinical risk models may perform less well ina cardiac cen-
findings. Furthermore, our open-source code enables other investigators tre patient population owingto the higher prevalence of cardiovascular co-
to retrain and fine-tune the model on their own data sets, facilitating adap- morbidities. Consistent with our findings, a recent study by Marston etal.*
tation to diverse dinical contexts. By establishing this robust, accessible reported that the CHARGE-AF dlinical score achieved a C-index of 65 in
baseline, the aim is to accelerate future research and improvement in predicting incident AF among patients with cardiovascular conditions.
ECG-based AF risk prediction across various heafthcare settings. Ourfindings indicate that the ECG-Al model outperforms traditional
clinical risk models, CHA;DS,-VASc, HATCH, and CHARGE-AF,
Electrocardiogram-based deep learning across various performance metrics. Saliency maps revealed that the
P-wave area had the most significant impact on ECG-AT's prediction
of AF risk, which i similar to the finding of Khurshid et l."® The patho-
5-year incident atrial fibrillation physiological plausibility of the predictive ability of ECG-AI can thus be
Atrial fibrillation diinical risk scores are a combination of established AF attributed to the assumption of an underlying ECG signature indicative
risk factors. In a recent meta-analysis, CHARGE-AF achieved a of significant atrial myopathy, which represents a vulnerable substrate
C-statistic of .71 (95% Cl: 66-76), HATCH 67 (95% Cl: .61-73), for AF. In fact, Verbrugge et al.* recently found that a higher deep
and CHA,DS-VASc .69 (95% CI: 64-74)."° Further, the CHARGE- learning probability of paroxysmal AF is associated with greater atrial
AF score performance has been consistently reported inthe .7—8 range myopathy on echocardiography and invasive haemodynamic testing.
across bothambulatory and general healthcare populations.'>**¢~* | The ECG-Al model maintained good performance across different
our study, the performance of these three clinical models was lower demographic groups, including age, sex, and socio-economic status
than previously reported. This discrepancy could be attributed to sev- (CIMD). This confirmation is important since current deep learning mod-
eral factors. First, classification bias may arise from inconsistencies in els have limited explainability and, therefore, carry the potential to reflect
4930 Jabbour et al.

A
w0 ECG-Al
—ecommn
B
1 AF-PGS

" 0
z z
£0s o6
¢Zos ]Zos

o o
HRe8.51 (95%
etyC1 3.76.5.40) HRe1.85 (95%
oo€1 1.44.2.36)
h s ® s % s © s
Atk
s5 0YWAtk
Aw01 257w o 029w 27% "i
c 10 CHARGE-AF D 10 ECG-Al + AF-PGS + CHARGE-AF

0 o
z z
£0s 2
¢é 0.4 ]go

02 .
HRe2.50 (95%
eyC1 1.81:3.46) HRe5.83 (95%
P €1 4.53-6.51)
h s ® s % s © s
Atk
oo o1% 2083 PPYoAt 030e n4 "i
Figure 6 Incident atrial ibrillation-free probability: Kaplan-Meier curves using different models to stratify patients in the MHI Biobank group. Index
electrocardiograms with calculated time to AF diagnosis equal to O days were removed. Hazard ratios were calculated by fitting a Cox proportional
hazards model. P-values are calculated using the log-rank test. (4) Electrocardiogram-based deep learning (ECG-Al) model. Classification threshold =
12%. (B) AF-polygenic score (AF-PGS) model. Classification threshold = top decile (10%) of PGS. (C) CHARGE-AF score. Classification threshol
21% based on the decision curve analysis. (D) ECG-Al + AF-PGS + CHARGE-AF model. AF-PGS and CHARGE-AF are added to ECG-Al post-training
using a logistic regression. Classification threshold = 21% based on the decision curve analysis

and perpetuate health disparities.”™" Of interest, ECG-Al showed sig- studies can be attributed to differences in the clinical setting, where
nificantly better discrimination performance in females compared with our study involved patients seen in a tertiary cardiac institute while
males. This better performance in females was also observed in a prior prior studies have mostly considered application of AF-PGS in the gen-
study for clinical AF prediction.® Accurate prediction of AF in females eral population. In this study, the first attempt to combine ECG-Al with
is of dlinical interest since female AF patients are at increased risk for ad- AF-PGS to predict AF risk is reported. Interestingly, while this combin-
verse outcomes including stroke and death compared with males.” The ation did not significantly enhance discrimination performance, it did
ECG-Al also maintained consistent performance in survival analysis when improve calibration performance, resuiting in an overall better fit and
selecting the single ECG with the highest AF probability per patient (see NB for the combined model compared with ECG-Al and with greater
Supplementary data online, Figure S14). benefit than when only the CHARGE-AF was added. This finding sug-
gests that AF-PGS potentially provides additional predictive value be-
yond ECG-Al in assessing AF risk in a cardiac care centre population.
Electrocardiogram-based deep learning
outperforms AF-PGS in predicting 5-year
Electrocardiogram-based deep learning
incident atrial fibrillation
Genetic variation has been shown to play an important role in deter- prediction is associated with long-term
mining long-term AF risk. In the MHI Biobank cohort, AF-PGS showed atrial fibrillation risk
poor discrimination performances compared with ECG-AL The lower The time-to-event analysis demonstrated a sustained separation of the
performance of AF-PGS in our study compared with prior AF-PGS KM incidence-free survival curves for up to 15 years after the index
Prediction of atrial fibrillation using deep learning 4931

A RoC curve B Precision-Recall Curve

0 — ECGA1:0.30(0.296, 0.303]
— ECGAIPatient.Level: 0,39 [0.386, 0.402]

08
True positive Rate

Precision
04

0z
— ECGA:0.71(0.704, 0.708)
00 —— ECGAIPatient Level: 0.77 [0.765, 0.773]
00 0z 00 06 0 To 00 [ 04 06 o8 0
False Positive Rate. Recall

Calibration Curve Decision Curve Analysis

)
—— ECGAI Patent Level
- Screen Al
—= Screen None.
Observed 5-year Risk.

Net genefic

,' — ECGAI ECI = 0,141 10.126,0.158]

oo| 7 —— ECGA patientLevel: €I - 0,071 (0.060,0.099]
0 02 04 06 0 10 01 o1 025
Predicted 5-year Risk Threshold Probabilty
Figure 7 Performance assessment of electrocardiogram-based deep learning (ECG-AI) in the Medical Information Mart for Intensive Care-IV
(MIMIC-V) external validation data set (109 870 patients, 437 323 ECG). (4) The receiver operating characteristic curve, plotting the true positive
rate against the false positive rate for each model, with the area under the curve indicating discriminatory power (reported in legend). (B) The preci-
sion—recall curve, plotting precision against recall with the area under the curve reported in legend. (C) The calibration curve, showing the refationship
between predicted and observed 5-year atrial ibrillation risk; the slope and intercept are calculated using linear regression,and the curve is plotted using
a univariate spline with smoothing factor of 1. The estimated calibration index (ECI, reported in legend) is the root mean squared difference between
the mean predicted probabilities and the spline-fitted calibration curve. (D) The decision curve analysis, plotting net benefit against threshold probability.

ECG. This finding further supports the potential of ECG-AI to stratify Limitations and future directions
AF risk. Additionally, these resuits are consistent with those ofa previ- The results should be interpreted in the context of our study design,
ous study evaluatingAF prediction using ECG-AV, thereby strengthening which was retrospective and confined to a single tertiary cardiac refer-
the overall body of evidence.'” The ECG-Al model maintained its ability ral centre. In Canada, coronary angiography is largely performed in aca-
to discriminate between high- and low-risk populations for new-onset demic centres like MHI, while HF care is more widely distributed,
AF after further stratifying the time-to-event analysis by age and sex contributing to the higher relative prevalence of CAD in the MHI popu-
groups, two established risk factors for AF. The model's superiority lation. External validation was conducted with the MIMIC-IV cohort,
over age and sex alone was most evident in younger age groups and fe- which includes patients admitted to the emergency department or an
male patients, aligning with the findings of Raghunath et al.”” Finally, intensive care unit, a population that differs from the MHI cohort and
stratifying patient using ECG-Al was associated with a higher HR of in- has a short median follow-up of 1.1 years. Despite these differences,
cident AF compared with AF-PGS and CHARGE-AF further affirming the consistent performance of ECG-Al in patients without CAD and
the advantage of ECG-AI (Figure 6). in the MIMIC-IV cohort is reassuring for its generalizability. Although
4932 Jabbour et al.

10 MIMIC IV
—— ECGAI Hign Risk
— ECG-Al Low Risk
Incident.free Probability

HR=4.59 (95% C1 4.45-4.74)

p<0.001
o 4 6
Years
At risk
74696 21581 13653
35174 ‘sas6 3756

MIMIC IV 1-year blanking

Incident.free Probability

HR=3.76 (95% C13.60-3.94)

p<0.001
o 4 6
Years
At risk
71705 31980 21581 13653 7036 1962
28455 10502 ‘6486 3756 1908 480
s using electrocardiogram-based deep learning (ECG-Al) to stratify patients in
Figure 8 Incident-free atrial fibrillation probability: Kaplan-Meier curve:
the Medical Information Mart for Intensive Care-IV (MIMIC-1V) external validation cohort. Electrocardiograms with calculated time to atrial ibrilation
diagnosis equal to O days were removed. Hazard ratios (HR) were calculated by fittinga Cox proportional hazards model. P-values are calculated using
the log-rank test. (4) ECG-Al model. Classification threshold = 12%. (B) ECG-Al model when excluding ECGs with time to AF < 1 year.

the AF classification bias was mitigated by using multiple sources and study did not account for right censoring when modellingAF as a binary
conducting a validation study against manual adjudication, a residual outcome. Despite consistent results from sensitivity analyses with
classification bias may be present and may have negatively impacted follow-up restrictions, future studies could incorporate time-to-event
ECG-Al performance. The 5-year AF incidence in our cohort is higher analysis in prospective cohorts where right censoring can be properly
than in general healthcare-related data sets.'®*® This discrepancy can accounted for. The study did not assess performance bias related to
be attributed to our study population consisting of cardiac patients, ethnicity, as such data were not available in the entire population and
who have a higher risk of developing AF and who may have undergone the MHI Biobank subgroup (where genetic ancestry could have been
more intensive monitoring leading to more frequent detection of AF inferred) is predominantly of European ancestry. Another limitation
compared with general healthcare-related data sets. Furthermore, in of this study is that new-onset AF and atrial flutter were analysed in
a retrospective analysis, it is challenging to distinguish between the de- a combined manner, which may affect the granularity and specificity
tection of pre-existing paroxysmal AF and the prediction of truly inci- of the results. Future work could consider separating these conditions,
dent AF, despite purposefully exduding patients with known AF at which can be beneficial in specific clinical scenarios, such as predicting
baseline. It cannot be excluded that some sinus rhythm ECG recorded new-onset AF in patients referred for atrial flutter ablation procedure.
after AF diagnosis may have been included. Reassuringly, the sensitivity Our study leverages a ResNet-50 architecture for ECG-Al predic-
analysis excluding ECG with a time to AF diagnosis of <1 year (to min- tion of incident AF, building upon previous seminal works that uti-
imize inclusion of patients with pre-existing paroxysmal AF) showed lized less complex convolutional neural network structures.’*™"”
consistent results with the overall All-Comers cohort (Figure 3; Importantly, while most published ECG-Al models remain proprietary,
Supplementary data online, Figure S5). Furthermore, this retrospective the open weights of the MHI ECG-Al model were shared. This
Prediction of atrial fibrillation using deep learning 4933

open-weight approach promotes transparency and establishes a robust publicly available. Please visit our repository at https:/github.com/
baseline for future research. As the field of Al rapidly evolves, with re- HeartWise-Allecg-ai-af-mhi.
cent advancements suggesting that more complex models could en-
hance predictive performance, our open-weight ResNet-50 model Funding
serves as a valuable benchmark. Researchers can now use this model
to validate its performance in diverse populations and directly compare P is supported by a fellowship grant from Takeda Canada and CIHR
new, potentially more sophisticated architectures against our model, Institute of Genetics (TAK401439). JH. LR, JC.-T, and RA. are sup-
accelerating progress in the field. ported by the Fonds de Recherche du Québec. JH. is funded by a
While adding clinical and genetic prediction to ECG-Al improved Natural Sciences and Engineering Research Council of Canada (NSERC)
performance in the MHI Biobank subgroup, further analysis is war- Discovery Grant (RGPIN-2022-04262). M-P.D, -C.T., and RT. are sup-
ranted to ensure the generalizability of this observation in diverse po- ported by the Canada Research Chairs programme. J.C.-T.and RT. are
pulations. Future developments also hold the promise of training a supported by the Philippa and Marvin Carsley Chair. PX. is supported
deep learning model end to end with all these modalities combined.> by the André Chagnon Research Chair.
Furthermore, the added clinical gain of ECG-AI remains uncertain be-
yond net clinical benefit modelling, and the cost-effectiveness remains Ethical Approval
unexplored. Finally, inherent challenges with ECG-AI persist, such as The study complies with the Declaration of Helsinki. The protocols
workflow integration and the acceptance of Al by both clinicians and were approved by the MHI ethics committee (submission IDs 2023-
patients in the medical field.>* 3160,2023-1756, and 2021-2928). All MHI Biobank participants signed
an informed consent. For other MHI patients, waiver of informed con-
sent was obtained from the ethics committee.
Conclusion
Our study contributes to the growing body of evidence demonstrating Pre-registered Clinical Trial Number
that deep learning applied to a resting 12-lead ECG during sinus rhythm None supplied.
can effectively predict the risk of new-onset AF with high performance
in a tertiary cardiac care centre population. This prediction outper-
forms existing clinical and polygenic scores. Our study demonstrates References
the potential of ECG-Al models to significantly enhance AF prediction 1. Tsao CW, Aday AW, Aimarzoog ZI, Anderson CAM, Arora P, Avery CL, et al. Heart
Disease and Stroke Statistics-2023 Update: a report from the American Heart
in a cardiac care setting, paving the way for tailored strategies for early Association. Ciraulation 2023;147¢93-621. https:/doiorg/10.1161/CIR 00000000000
detection of AF to prevent adverse outcomes. 01123
Goette A, Kalman JM, Aguinaga L, Akar }, Cabrera JA, Chen SA, et a. EHRA/HRS/
/APHRS/SOLAECE expert consensus on atrial cardiomyopathies: definition, character-
ization, and dlinical implication. Europace 2016;18:1455-90. https./doi.org/10.1093/
Acknowledgements europace/euw161
Qin D, Mansour MC, Ruskin JN, Heist EK. Atrial fibrilation-mediated cardiomyopathy.
We wish to thank ulie Todd for her assistance in big data extraction Circ Amhythm Electrophysiol 2019;12:¢007809. https:/doi.org/10.1161/CIRCEP.119.
and Sewanou Hermann Honfo and Luca Scimeca for the valuable 007809
discussions on statistical and deep learning modelling, as well as Santhanakrishnan R, Wang N, Larson MG, Magnani JW, McManus DD, Lubitz SA, et al
IS

Amélie Jeuken, Léanie Moreau, Breanna Chen, Agnés Hage-Chehine, Atrial fibrillation begets heart failure and vice versa: temporal associations and differ-
ences in preserved versus reduced ejection fraction. Circulation 2016;133:484-92.
Louise-Sabine Louis-Aimé, and Samuel Moussa for their assistance in https://doi.org/10.1161/CIRCULATIONAHA 115.018614
data collection. The Structured Graphical Abstract was created with Koh YH, Lew LZW, Franke KB, Elliott AD, Lau DH, ThiyagarajahA, etal. Predictive role

Biorender.com. of atrial fibrillation in cognitive decline: a systematic review and meta-analysis of 2.8 mil-
lion individuals. Europace 2022,24:1229-39. https:/doi.org/10.1093/europace/euac003
Papanastasiou CA, Theochari CA, Zareifopoulos N, Arfaras-Melainis A, Giannakoulas
G, Karamitsos TD, et al. Atrial fibrillation is associated with cognitive impairment, al-
Supplementary data ‘cause dementia, vascular dementia, and Alzheimer’s disease: a systematic review and
meta-analysis. | Gen Intern Med 2021;36:3122-35. https://doi.org/10.1007/s11606-
Supplementary data are available at European Heart Journal orline. 021-06954-8
Hart RG, Pearce LA, Aguilar MI. Meta-analysis: antithrombotic therapy to prevent
stroke in patients who have nonvalvular atrial fibrilation. Ann Intem Med 2007;146:
857-67. https:/doiorg/10.7326/0003-4819-146-12-200706 190-00007
Declarations Ruff CT, Giugliano RP, Braunwald E, Hoffman EB, Deenadayalu N, Ezekowitz MD, et al
Comparison of the efficacy and safety of new oral anticoagulants with warfarin in pa-
Disclosure of Interest tients with atrial fibrillation: a meta-analysis of randomised trials. Lancet 2014;383:
955-62. https:/doiorg/10.1016/50140-6736(13)62343-0
M.-PD. holds a minor equity interest in DalCor Pharmaceuticals. Kirchhof P, Camm A|, Goette A, Brandes A, Eckardt L, Evan A et al. Early rhythm-
J-C.T. reports research grants from AstraZeneca, Boehringer control therapy in patients with atrial fibrilation. N Engl | Med 2020;383:1305-16.
Ingelheim, Ceapro, DalCor Pharmaceuticals, Esperion, Merck, https://doi.org/10.1056/NEJMoa2019422
Dilaveris PE, Kennedy HL Silent atrial fibrillation: epidemiology, diagnosis, and linical
Novartis, Novo Nordisk, and Pfizer; honoraria from DalCor
3

impact. Clin Cardiol 2017:40:413-8. https:/idoiorg/10.1002/clc22667

Pharmaceuticals, Pendopharm, and Pfizer; and minor equity interest Petz| AM, Jabbour G, Cadrin-Tourigny }, Purerfellner H, Macle L, Khairy P, et al
in DalCor Pharmaceuticals. Innovative approaches to atrial fibrillation prediction: should polygenic scores and ma-
chine learning be implemented in clinical practice? Europace 2024;26:eu2e201. httpsi
doi.org/10.1093/europace/euae201
Data Availability AlonsoA, Krithe BP, Aspelund T, Stepas KA, Pencina M), Moser CB, et al. Simple risk
model predicts incidence of atrial fibrillation in a racially and geographically diverse
Theassociated code required to run the trained ResNet-50 model used population: the CHARGE-AF consortium. | Am Heart Assoc 2013;2:000102. httpsi
in this study is available on GitHub. The ECG-Al model is also made doi.org/10.1161/JAHA.112.000102
4934 Jabbour et al.

13 SuenariK, Chao T-F, Lius C, KiharaY, Chen T+, Chen S-A. Usefulness
of HATCH score 4 Simonyan K, Vedaldi A, Zisserman A 2013. Deep inside convolutional networks: visua-
in the prediction ofnew-onsetatrialfibrillation for Asians. Medidie (Batimore) 2017:96: lising image classification models and saliency maps. arXiv, arXiv:13126034, preprint: not
€5597. https:/doi.org/10.1097/MD.0000000000005597 peer reviewed.
Zuo ML, Lius, Chan K-H, Lau KK, Chong B-H, Lam K-F, et al The CHADS2and CHA Khera AV, Chaffin M, Aragam KG, Haas ME, Roselli C, Choi SH, et dl. Genome-wide
2DS 2-VASC scores predict new occurrence of atrial fibrillation and ischemic stroke. | polygenic scores for common diseases identify individuals with risk equivalentto mono-
Interv Card Electrophysiol 2013;37:47-54. https:fdoi org/10.1007/510840-012-9776-0 genic mutations. Nat Genet 2018:50:1219-24. httpsidoiorg/10.1038/541588-018-
AttaZI|, Noseworthy PA, Lopez-fimenez F, Asirvatham 5}, DeshmukhA, Gersh B}, et al. 01832
An artificial intelligence-enabled ECG algorithm for the identification of patients with Glas AS, Limer |G, Prins MH, Bonsel GJ, Bossuyt PM. The diagnostic odds ratio: a single
atrial fibrillation during sinus rhythi a retrospective analysis of outcome prediction. indicator of test performance. J Clin Epidemiol 2003;56:1129-35. httpsidoiorg/10.
Lancet 2019,394:861-7. https/doiorg/10.1016/50140-6736(19)31721-0 1016/50895-4356(03)00177-X
Khurshid S, Friedman , Reeder C, Di Achille P, Diamant N, Singh P, et al ECG-based Van Hoorde K, Van Huffel S, Timmerman D, Bourne T, Van Calster B. A spline-based
deep learning and dlinical risk factors to predict atrial fibrilation. Craulation 2022,145: t0ol to assess and visualize the calibration of multiclass risk predictions. | Biomed Inform
122-33. httpsi/doi org/10.116ICIRCULATIONAHA 121057480 2015,54:283-93. httpst/doi org/10.1016/jbi 201412016
Raghunath S, Pfeffer JM, Ulloa-Cerna AE, Nemani A, Carbonati T, Jing L, et dl. Deep Vickers A, Elkin EB. Decision curve analysis: a novel method for evaluating prediction
neural networks can predict new-onset atrial fibrilltion from the 12-lead ECG and models. Med Deds Making 2006,26:565-74. https:/doi org/10.1177/0272989X0629
help identify those at risk of atrial fibrillation-related stroke. Circuiation 2021:143: 5361
1287-98. httpsifdoi org/10.1161/CIRCULATIONAHA 120047829 DeLong ER, DeLong DM, Clarke-Pearson DL Comparingthe areas under two or more
Yuan N, Duffy G, Dhruva S5, Oesterle A, Pellegrini CN, Theurer | et al Deep learningof correlated receiver operating characteristic curves: a nonparametric approach.
Biometrics 1988:44:837—45. https:doi org/102307/2531595
electrocardiograms in sinus rhythm from US veterans to predict atrial fibrillation. JAVA P Statistics Canada. 2023. The Canadian Index of Multiple Deprivation, 2021. httpsy/
Cardol 2023;8:1131-9. https:fdoi org/10.100 1 /jamacardio 2023 3701

B
www150,statcan gc ca/n1/publ45-20-0001/452000012023002-eng htm.
Hygrel T, Viberg F, Dahlberg E, Chariton PH, Kemp Gudmundsdottir K, Mant},etal An 4 JohnsonA, BulgareliL, PollardT, Gow B, MoodyB, Horngs, et al. MIMIC-IV version 3.0.
artifical inteligence-based model for prediction of atrial fibrilltion from single-lead si- PhysioNet 2024. https/doiorg/10.13026/hxp0-hgs9 (15 August 2024, date last
nus rhythm electrocardiograms faciltating screening. Europace 2023,2511332-8. https/ accessed)
doiorg/10.1093/europaceleuad03s 42 Johnson AEWV, Bulgareli L, Shen L, Gayles A, ShammoutA, Hormg S, et d. MIMIC-IV, a
X Attia Z|, Harmon DM, Behr ER, Friedman PA. Application of artfical intelligence to the freely accessible electronic health record dataset. Sci Data 2023;10:1. https/idoi org/10.
B

electrocardiogram. Eur Heart ) 2021:42:4717-30. hitpsi/doi.org/10.1093/eurheartj/ 1038/541597-022-01899x

ehab649 4 Goldberger AL, Amaral LAN, Gass L, Hausdorff JM, hanov PC, Mark RG, et al
b Weng L-C, Preis SR, Hulme OL, Larson MG, Choi SH, Wang B, etal. Genetic predispos- PhysioBank, PhysicToolkit, and PhysioNet: components of a new research resource
ition, clinical risk factor burden, and lfetime risk of atrial fbrillation. Circulation 2018; for complex physiologic signals. Ciraulation 200010121520, https/idoiorg/10.
137:1027-38. httpsy/doi.org/10.1161/CRCULATIONAHA 117.031431 1161/01.CR101.23215
2 Muse ED, Wineinger NE, Spencer EG, Peters M, Henderson R, ZhangY, et ai Validation P Avram R Olgin JE, Tison GH. The rise of open-sourced machine learning in small and
S

of a genetic risk score for atrialfbrilltion: a prospective multicenter cohort study. PLoS imbalanced datasets: predicting in-stent restenosis. Can J Cardiol 202036:1574-6.
Med 2018151002525, https:idoiorg/10.1371/journal pmed. 1002525 https:doi org/10.1016/jcjca 202002002
2 Lazarte), Dron}5, Mclntyre AD, Skanes AC, Gula L), Tang AS, et al. Evaluating polygenic 4 Himmelreich JCL, Veelers L, Lucassen WAM, Schnabel RB, Rienstra M, van Weert H,
risk scores in “lone” atrial fbrilltion. JC Open 2021:3751-7. https:Jdoi.org/10.1016/; et al Prediction models for atrial fibrilltion applicable in the community: a systematic
€jc0202102001 review and meta-analysis. Europace 202022:684-94. https:/doi org/10.1093/europace/
2 Borschel CS, Ohlrogge AH, Geelhoed B, Niiranen T, Havulinna AS, Palosaari T, et al. eaa005
®

Risk prediction of atrial fibrilation in the community combining biomarkers and genet- 46, Christophersen IE, Yin X, Larson MG, Lubitz SA, Magnani JW, McManus DD, et al. A
ics. Europace 202123:674-81. https//doiorg/10.1093/europacefeuaa334 comparison of the CHARGE-AF and the CHA2DS2-VAS risk scores for prediction
2 Khurshid S, Mars N, Haggerty CM, Huang Q, Weng L-C, Hartzel DN, et dl. Predictive of atrial fibrillion in the Framingham Heart Study. Am Heart | 2016:178:45-54
accuracy ofa clinical and genetic risk model for atrial fbrilation. Circ Genom Preds Med https:/doi org/10.1016/jahj 2016 05.004
2021:44:2003355. https:/doi org/10.1161/CIRCGEN.121 003355, 47. Hulme OL, Khurshid S, Weng L-C, Anderson CD, Wang EY, Ashburner JM, et al
2% van Royen FS, Asselbergs FW, Alfonso F, Vardas P, van Smeden M. Five critical quality Development and validation of a prediction model for atrial fibrillation using electronic
criteria for artificil intelligence-based prediction models. Eur Heart ) 2023:44:4831-4. health records JACC Cln Electrophysiol 2019,5:1331-41. https/doi org/10.1016/}jacep.
https:fdoiorg/10.1093/eurheartjlehad727 201907016
2 van Smeden M, Heinze G, Van Calster B, Asselbergs FW, Vardas PE, Bruining N, et al. 4 Khurshid S, Kartoun U, Ashburner M, Trinquart L, Philippakis A, Khera AV, et al
Critical appraisal of artificialintelligence-based prediction models for cardiovascular dis- Performance of atrial fibrilation risk prediction models in over 4 milion individuals
ease. Eur Heart | 2022:43:2921-30. https//doi.org/10.1093/eurheartjlehac238 Circ Antythm Electrophysiol 2021;14:6008997. htps:/doi.org/10.1161/CIRCEP.120.
2 Collins GS, Moons KGM, Dhiman P, Riley RD, Beam AL, Van Calster B, etal. TRIPOD + 008997
Al statement: updated guidance for reporting clinical prediction models that use regres- 49, Marston NA, Garfinkel AC, Kamanu FK, Melloni GM, Roseli C, Jarolim P, et al. A poly-
sion or machine learning methods. BM) 2024385:€078378. https:/dol.org/10.113¢/ genic isk score predicts atrialfbrilltion i cardiovascular disease. Eur Heart | 2023
bmj-2023-078378 221-31. https/doiorg/10.1093/eur heartj/ehac460
2 Zafrir B, Lund LH, Laroche C, Ruschitzka F, Crespo-Leiro MG, Coats AJS, et al. 50. Verbrugge FH, Reddy YNV, Attia ZI, Friedman PA, Noseworthy PA, Lopez-fimenez F,
Prognostic implications of atrial fibrilation in heart failure with reduced, mid-range, et al. Detection of left atrial myopathy using artificial intelligence-enabled lectrocardiog-
raphy. Circ Heart Fail 2022:45:2008176. https:/doi org/10.1161/CIRCHEARTFAILURE.
and preserved ejection fraction: a report from 14 964 patients in the European 120008176
Society of Cardiology Heart Failure Long-Term Registry. Eur Heart ) 201839: 5 Noseworthy PA, Attia ZI, Brewer LC, Hayes SN, Yao X, Kapa S, et al. Assessing and
4277-84. hitpsidoi.org/10.1093/eurheartjlehy626 mitigating bias in medical artificial intelligence: the effects of race and ethnicity on a
Fauchier L, BissonA, Bodin A, Herbert |, Angoulvant D, Danchin N, et al Outcomes in deep learning model for ECG analysis. Circ Amhythm Electrophysiol 2020;13:2007988.
g

patients with acute myocardil infarction and new atrial fibrilation: a nationwide ana- httpsidoi org/10.1161/CIRCEP.119.007988
Iysis. Clin Res Cardiol 2021;110:1431-8. https:/doi.org/10.1007/500392-021-01805-2 52 Tadros R, Ton AT, Fiset C, Nattel S, Sex differences in cardiac electrophysiology and
Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm dlinical arrhythmias: epidemiology, therapeutics, and mechanisms. Can J Cardiol 2014;
Used to manage the health of populations. Science 2019,366:447-53. https:/doi.org/10. 30783-92. https:/doi org/10.1016/jcjca 2014 03.032
1126/science aax2342 5: Boonstra M), Weissenbacher D, Moore JH, Gonzalez-Hernandez G, Asselbergs FW.
He K, ZhangX, Ren, Sun |. Deep residual learningfor image recognition. In: Proceedings Atificil intelligence: revolutionizing cardiology with large language models. Eur Heart
of the IEEE Corference on Computer Vision and Pattem Recognition. New York City: IEEE, 2024:45:33245. https:idoi.org/10.1093/eurheartjlehad838
2016, 770-778. 5- Leclercq C, Witt H, Hindricks G, Katra RP, Albert D, Belliger A, et . Wearables, tele-
%

Abadi M, Agarwal A Barham P, Brevdo E, Chen Z, Citro C, et . 2016. Tensorflow: medicine, and articial intelligence in arrhythmias and heart failure: proceedings of the
large-scale machine leaming on heterogeneous distributed systems. arXiv, arXiv European Society of Cardiology Cardiovascular Round Table. Europace 2022.24:
160304467, preprint: not peer reviewed 1372-83. https//doi org/10.1093/europace/euac052