Professional Documents
Culture Documents
Abstract
arXiv:1707.01836v1 [cs.CV] 6 Jul 2017
conv
2. Model
Problem Formulation
BN
The ECG arrhythmia detection task is a sequence-to-
ReLU
sequence task which takes as input an ECG signal X =
[x1 , ..xk ], and outputs a sequence of labels r = [r1 , ...rn ], dense
such that each ri can take on one of m different rhythm
Softmax
classes. Each output label corresponds to a segment of the
input. Together the output labels cover the full sequence.
Figure 2. The architecture of the network. The first and last layer
For a single example in the training set, we optimize the are special-cased due to the pre-activation residual blocks. Over-
cross-entropy objective function all, the network contains 33 layers of convolution followed by a
n fully-connected layer and a softmax.
1X
L(X, r) = log p(R = ri | X)
n i=1
where p() is the probability the network assigns to the i-th sampled at 200Hz, and the model outputs a new prediction
output taking on the value ri . once every second. We arrive at an architecture which is 33
layers of convolution followed by a fully connected layer
Model Architecture and Training and a softmax.
We use a convolutional neural network for the sequence-to- In order to make the optimization of such a network
sequence learning task. The high-level architecture of the tractable, we employ shortcut connections in a similar man-
network is shown in Figure 2. The network takes as input ner to those found in the Residual Network architecture (He
a time-series of raw ECG signal, and outputs a sequence et al., 2015b). The shortcut connections between neural-
of label predictions. The 30 second long ECG signal is network layers optimize training by allowing information
to propagate well in very deep neural networks. Before
1
iRhythm Technologies, San Francisco, California the input is fed into the network, it is normalized using a
Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks
Seq Set In both the Sequence and the Set case, we compute the
Model Cardiol. Model Cardiol. F1 score for each class separately. We then compute the
overall F1 (and precision and recall) as the class-frequency
Class-level F1 Score
weighted mean.
AFIB 0.604 0.515 0.667 0.544
AFL 0.687 0.635 0.679 0.646
AVB TYPE2 0.689 0.535 0.656 0.529 Model vs. Cardiologist Performance
BIGEMINY 0.897 0.837 0.870 0.849 We assess the cardiologist performance on the test set. Re-
CHB 0.843 0.701 0.852 0.685
EAR 0.519 0.476 0.571 0.529 call that each of the records in the test set has a ground
IVR 0.761 0.632 0.774 0.720 truth label from a committee of three cardiologists as well
JUNCTIONAL 0.670 0.684 0.783 0.674 as individual labels from a disjoint set of 6 other cardiolo-
NOISE 0.823 0.768 0.704 0.689 gists. To assess cardiologist performance for each class, we
SINUS 0.879 0.847 0.939 0.907 take the average of all the individual cardiologist F1 scores
SVT 0.477 0.449 0.658 0.556
TRIGEMINY 0.908 0.843 0.870 0.816 using the group label as the ground truth annotation.
VT 0.506 0.566 0.694 0.769 Table 1 shows the breakdown of both cardiologist and
WENCKEBACH 0.709 0.593 0.806 0.736
model scores across the different rhythm classes. The
Aggregate Results model outperforms the average cardiologist performance
Precision (PPV) 0.800 0.723 0.809 0.763 on most rhythms, noticeably outperforming the cardiolo-
Recall (Sensitivity) 0.784 0.724 0.827 0.744 gists in the AV Block set of arrhythmias which includes
F1 0.776 0.719 0.809 0.751 Mobitz I (Wenckebach), Mobitz II (AVB Type2) and com-
plete heart block (CHB). This is especially useful given
Table 1. The top part of the table gives a class-level comparison of the severity of Mobitz II and complete heart block and the
the expert to the model F1 score for both the Sequence and the Set importance of distinguishing these two from Wenckebach
metrics. The bottom part of the table shows aggregate results over which is usually considered benign.
the full test set for precision, recall and F1 for both the Sequence
and Set metrics. Table 1 also compares the aggregate precision, recall and
F1 for both model and cardiologist compared to the ground
truth annotations. The aggregate scores for the cardiolo-
gist are computed by taking the mean of the individual car-
ical, requiring immediate attention (Dubin, 1996). diologist scores. The model outperforms the cardiologist
Table 2 in the Appendix also shows the number of unique average in both precision and recall.
patients in the training (including validation) set and test
set for each rhythm type. 5. Analysis
The model outperforms the average cardiologist score on
4. Results both the sequence and the set F1 metrics. Figure 4 shows
Evaluation Metrics a confusion matrix of the model predictions on the test set.
Many arrhythmias are confused with the sinus rhythm. We
We use two metrics to measure model accuracy, using the expect that part of this is due to the sometimes ambiguous
cardiologist committee annotations as the ground truth. location of the exact onset and offset of the arrhythmia in
Sequence Level Accuracy (F1): We measure the aver- the ECG record.
age overlap between the prediction and the ground truth Often the mistakes made by the model are understand-
sequence labels. For every record, a model is required to able. For example, confusing Wenckebach and AVB Type2
make a prediction approximately once per second (every makes sense given that the two rhythms in general have
256 samples). These predictions are compared against the very similar ECG morphologies. Similarly, Supraventric-
ground truth annotation. ular Tachycardia (SVT) and Atrial Fibrillation (AFIB) are
Set Level Accuracy (F1): Instead of treating the labels for often confused with Atrial Flutter (AFL) which is under-
a record as a sequence, we consider the set of unique ar- standable given that they are all atrial arrhythmias. We also
rhythmias present in each 30 second record as the ground note that Idioventricular Rhythm (IVR) is sometimes mis-
truth annotation. Set Level Accuracy, unlike Sequence taken as Ventricular Tachycardia (VT), which again makes
Level Accuracy, does not penalize for time-misalignment sense given that the two only differ in heart-rate and are
within a record. We report the F1 score between the unique difficult to distinguish close to the 100 beats per minute de-
class labels from the ground truth and those from the model lineation.
prediction.
Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks
AFIB 1.0 beat detection (Coast et al., 1990). Artificial neural net-
AFL 0.9 works have also been used for the task of beat detection
AVB_TYPE2 0.8 (Melo et al., 2000). While these models have achieved
BIGEMINY 0.7
CHB high-accuracy for some beat types, they are not yet suffi-
EAR 0.6
True label
He, Kaiming, Zhang, Xiangyu, Ren, Shaoqing, and Sun, Shah, Atman P and Rubin, Stanley A. Errors in the
Jian. Identity mappings in deep residual networks. In computerized electrocardiogram interpretation of car-
European Conference on Computer Vision, pp. 630645. diac rhythm. Journal of electrocardiology, 40(5):385
Springer, 2016c. 390, 2007.
Heden, Bo, Ohlsson, Mattias, Holst, Holger, Mjoman, Mat- Srivastava, Nitish, Hinton, Geoffrey E, Krizhevsky, Alex,
tias, Rittner, Ralf, Pahlm, Olle, Peterson, Carsten, and Sutskever, Ilya, and Salakhutdinov, Ruslan. Dropout:
Edenbrandt, Lars. Detection of frequently overlooked a simple way to prevent neural networks from overfit-
electrocardiographic lead reversals using artificial neu- ting. Journal of Machine Learning Research, 15(1):
ral networks. The American journal of cardiology, 78 19291958, 2014.
(5):600604, 1996.
Turakhia, Mintu P, Hoang, Donald D, Zimetbaum, Peter,
Ioffe, Sergey and Szegedy, Christian. Batch normalization: Miller, Jared D, Froelicher, Victor F, Kumar, Uday N,
Accelerating deep network training by reducing internal Xu, Xiangyan, Yang, Felix, and Heidenreich, Paul A.
covariate shift. arXiv preprint arXiv:1502.03167, 2015. Diagnostic utility of a novel leadless arrhythmia moni-
Kingma, Diederik and Ba, Jimmy. Adam: A toring device. The American journal of cardiology, 112
method for stochastic optimization. arXiv preprint (4):520524, 2013.
arXiv:1412.6980, 2014. Xiong, Wayne, Droppo, Jasha, Huang, Xuedong, Seide,
Laguna, Pablo, Mark, Roger G, Goldberg, A, and Moody, Frank, Seltzer, Mike, Stolcke, Andreas, Yu, Dong,
George B. A database for evaluation of algorithms for and Zweig, Geoffrey. Achieving human parity in
measurement of qt and other waveform intervals in the conversational speech recognition. arXiv preprint
ecg. In Computers in Cardiology 1997, pp. 673676. arXiv:1610.05256, 2016.
IEEE, 1997.
Li, Cuiwei, Zheng, Chongxun, and Tai, Changfeng. De-
tection of ECG characteristic points using wavelet trans-
forms. IEEE Transactions on biomedical Engineering,
42(1):2128, 1995.
Martnez, Juan Pablo, Almeida, Rute, Olmos, Salvador,
Rocha, Ana Paula, and Laguna, Pablo. A wavelet-
based ECG delineator: evaluation on standard databases.
IEEE Transactions on biomedical engineering, 51(4):
570581, 2004.
Melo, SL, Caloba, LP, and Nadal, J. Arrhythmia analysis
using artificial neural network and decimated electrocar-
diographic data. In Computers in Cardiology 2000, pp.
7376. IEEE, 2000.
Moody, George B and Mark, Roger G. A new method for
detecting atrial fibrillation using RR intervals. Comput-
ers in Cardiology, 10(1):227230, 1983.
Moody, George B and Mark, Roger G. The impact of
the MIT-BIH arrhythmia database. IEEE Engineering
in Medicine and Biology Magazine, 20(3):4550, 2001.
Pan, Jiapu and Tompkins, Willis J. A real-time QRS detec-
tion algorithm. IEEE transactions on biomedical engi-
neering, (3):230236, 1985.
Sainath, Tara N, Mohamed, Abdel-rahman, Kingsbury,
Brian, and Ramabhadran, Bhuvana. Deep convolutional
neural networks for lvcsr. In Acoustics, speech and sig-
nal processing (ICASSP), 2013 IEEE international con-
ference on, pp. 86148618. IEEE, 2013.
Appendix
Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks
Atrial Fibrilla-
AFIB 4638 44
tion
Second degree
AVB TYPE2 AV Block Type 1905 28
2 (Mobitz II)
Ventricular
BIGEMINY 2855 22
Bigeminy
Complete Heart
CHB 843 26
Block
Ectopic Atrial
EAR 2623 22
Rhythm
Idioventricular
IVR 1962 34
Rhythm
Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks
Junctional
JUNCTIONAL 2030 36
Rhythm
Supraventricular
SVT 6301 34
Tachycardia
Ventricular
TRIGEMINY 2864 21
Trigeminy
Ventricular
VT 4827 17
Tachycardia
Wenckebach
WENCKEBACH 2051 29
(Mobitz I)
Table 2. A list of all of the rhythm types which the model classifies. For each rhythm we give the label name, a more descriptive name
and an example chosen from the training set. We also give the total number of patients with each rhythm for both the training and test
sets.