Artificial Intelligence-based Chest X-ray Image

Analysis for 2019 Novel Coronavirus Case

JLK INSPECTION, Inc. (E-mail: support@jlk-inspection.com, Tel: +82-70-4651-4051)

To demonstration the performance of AI-based pneumonia

prediction model in a COVID-19 infected case, the CXR cases
I. INTRODUCTION presented in the reference 1 were used. The COVID-19 case
The 2019 novel coronavirus (2019-nCOV; COVID-19) that presented in the paper has follow-up CXR images from 4 to 10
began in Wuhan, China, has rapidly and widely spread in the days after the onset of illness. The probabilities of pneumonia
world[1,2]. The WHO interim guidance suggests a real-time classification for each CXR images were obtained from the
reverse-transcriptase–polymerase-chain-reaction (RT-PCR) trained classification model. The localized heatmaps were
assay to diagnose COVID-19 infection[3]. However, as the described with combining Grad-CAM results of the
number of suspected patients is increasing over the limited classification models.
screening ability, the diagnostic time delay due to the running
time of real-time RT-PCR makes it difficult to treat the patients III. RESULT
timely, adapting to the severity. For the faster monitoring of the The prediction results of the pneumonia classification models
patient's severity, chest imaging such as chest x-ray (CXR) and on the test dataset are using, AUROC (Area Under the Receiver
computed tomography (CT) scan can be an alternative method. Operating Characteristic Curve; AUC)[7].
In this paper, we present artificial intelligence-based (AI-based)
analysis results of CXR images of a COVID-19 infected patient
with a pneumonia sensitive AI classification model. The
changing of AI prediction probabilities was demonstrated with
progression from initial mild symptoms to pneumonia. The AI-
weighted area was described by localized heatmap on the CXR
images. Furthermore, we discussed the possibilities of triage of
AI-based CXR analysis for rapid and quantitative analysis at
the initial stage of pneumonia in the classification of many
suspected patients for COVID-19.

II. METHOD
The pneumonia classification model consists of a pre-
processing module and an ensemble classification module. In
the pre-processing module, the input CXR image is resized to
Figure 1. ROC curves of models
256 × 256 and processed for histogram equalization[4,5]. To
eliminate undesirable noise in the image, a U-net[6] based lung
segmentation model segments both sides of the lungs and crop Figure 1 shows the ROC curves and AUC scores of each model.
the region of interest (ROI) from the image. In classification Model 1 and Model 2 having different preprocessing
module, an ensemble model of two neural networks with parameters on histogram equalization distribute different AUC
DenseNet structure classify the image into pneumonia or performances with 0.97 and 0.96, respectively. As expected, the
normal case. We used 79,672 CXR images based dataset from ensemble model of these two models performed better than
Korean National Tuberculosis Association (KNTA) with each classification model with 0.98 AUC.
mixing up several abnormal cases from CXR dataset of Table 1 shows the performance of sensitivity, specificity, and
National Institutes of Health (NIH) for training the ensemble AUC of each model.
classification model. To test the performance of pneumonia Table 1. Sensitivity, Specificity, and AUC of Models
classification models, separate dataset of 424 CXR images Sensitivity Specificity AUC
(from Gachon University Gil Medical Center, Incheon, South
Model 1 0.9717 0.7642 0.9725
Korea) was used. For test dataset, the diagnosis of pneumonia
Model 2 0.6462 0.9953 0.9599
was based on both of the chest CT findings and clinical
Ensemble Model 0.9623 0.9009 0.9826
information. The two classification models were trained and
validated using same dataset consists of pneumonia and normal
cases.
Table 2 shows AI-based pneumonia prediction results for the
follow-up CXR images of the COVID-19 infection case (Illness
days 4, 7, 9, and 10) from reference 1. The probability values
of the ensemble model having the threshold of 0.5 for
pneumonia. As shown Table 2, the pneumonia probability of AI
prediction was increased with progression to pneumonia.
Table 2. AI prediction results of pneumonia for COVID-19 infection case
AI Predicted Pneumonia
CXR image
(prob.)
Illness day 4 Negative (0.2746)
Illness day 7 Positive (0.5181)
Illness day 9 Positive (0.7648)
Illness day 10 Positive (0.6086)
Figure 2 shows the CXR image and AI result in illness day 4
from reference 1. AI predicts this CXR as a low probability at
pneumonia and the localized heatmap lesion is not visualized
on the CXR image. As mentioned in reference 1, any
abnormalities were not found in this CXR image. In the CXR
image of very early COVID-19 patients, both the radiologist
and the AI have not found abnormalities because of no
progression of pneumonia.
Figure 2. Illness day 4 CXR image (left) and AI result image with localized
lesion heatmap by AI (right)
Figure 3 shows the CXR image and AI result in illness day 7.
AI predicts the case as pneumonia positive (>0.5), and the
localized heatmap lesion on left lower lung zone indicates AI-
weighted pneumonia patterns. As reference 1, the radiologist
reported as showing no evidence of infiltrates or abnormalities
in this CXR image. However, the pneumonia sensitive AI
predicts this case as pneumonia positive with the probability of
~0.52 just over the pneumonia threshold of 0.5.
Figure 3. Illness day 7 CXR (left) and AI result image with localized lesion
heatmap by AI (right)
Figure 4 shows the CXR image and AI result in illness day 9.
AI predicts the case as a pneumonia case with a high probability
of ~0.76. Reference 1 reported that evidence of pneumonia in
the left lower lung zone was shown. However, the heatmap
result shows that the pneumonia sensitive AI referred to the
both lower lung zone. Reference 1 reported that these
radiographic findings coincided with a change in respiratory
status starting on the evening of this day.
Figure 4. Illness day 9 CXR (left) and AI result image with localized lesion
heatmap by AI (right)
Figure 5 shows the CXR image and AI result in illness day 10.
Reference 1 reported that the patient was started on
supplemental oxygen, treatment for pneumonia from this day.
Whether the CXR image acquired timing was not reported, the
AI predicted probability decreased by ~0.61, and the heatmap
level of the AI-weighted probability reduced compared to those
of illness day 9. Reference 1 also reported that basilar streaky
opacities in both lungs, a finding consistent with atypical
pneumonia, and rales were noted in both lungs on auscultation.

Figure 4. Illness day 10 chest x-ray image (left) and AI result image with
localized lesion heatmap by AI (right)
Figure 5. The AI predicted probability graph of COVID-19 case from
illness day 4 to 10
Figure 6 shows the changing of AI prediction results as illness
day increases. In illness day 7, AI detected pneumonia patterns
in CXR images. The probability value increases linearly from
illness days 4 to 9 with the progression from mild to severe
pneumonia.
IV. CONCLUSION
The ensemble model combined different two models of
sensitivities and specificities performed 0.98 AUC for the
pneumonia test dataset of a completely different data source
from the training data source. Applying the ensemble model for
the CXR images of the COVID-19 infected case from reference
1, the pneumonia classification probabilities of AI models were
changed with the progression of pneumonia, as shown in Table
3. In illness day 7, AI detected pneumonia patterns in CXR
images earlier than the report of reference 1. These features can
lead to the application possibility for screening and quantitative
analysis for image-based early triage with the progression of
pneumonia. Furthermore, the AI-weighted heatmap can
describe pneumonia suspected patterns on the CXR images,
which leads to increase efficiency and performance of CXR
reading even in the early stage of pneumonia.
Table 3. AI prediction results of pneumonia for COVID-19 infection case
AI Predicted Radiograph
CXR image
Pneumonia (prob.) reading in ref. 1
Illness day 4 Negative (0.2746) No finding
Illness day 7 Positive (0.5181) No finding
Illness day 9 Positive (0.7648) Abnormal
Illness day 10 Positive (0.6086) Abnormal
WHO guides to recognize and sort all patients at first point of
contact as a mild, moderate, or severe illness, including severe
pneumonia, ARDS, sepsis and septic shock[3]. A fast triage of
diagnosis and treatment is needed to control a large number of
suspected or infected patients effectively and minimize the
losses. The CXR images are the most widely used image
modality for monitoring lung diseases due to simple and rapid
image acquirement. The AI-based pneumonia classification
probability of CXR images can be one of the fast and
quantitative image-based triage tool for diagnosis and treatment
with the fully-automated image analysis function to control a
large number of suspected or infected patients effectively. A
more accurate triage analysis can be provided by combining the
patient's information such as fever, respiration condition, and
underlying condition.
V. REFERENCES
[1] Holshue, Michelle L., et al. "First case of 2019 novel coronavirus in the
United States." New England Journal of Medicine (2020).
[2] Ng, Ming-Yen, et al. "Imaging profile of the COVID-19 infection:
radiologic findings and literature review." Radiology: Cardiothoracic
Imaging 2.1 (2020): e200034.
[3] World Health Organization. Clinical management of severe acute
respiratory infection when novel coronavirus (2019-nCoV) infection is
suspected: interim guidance. January 28, 2020.
[4] A. K. Jain, Fundamentals of digital image processing. Englewood Cliffs,
NJ: Prentice Hall (1989).
[5] W. K. Pratt, "Digital image processing john wiley & sons," Inc., New
York (1991).
[6] O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks
for biomedical image segmentation," in International Conference on
Medical image computing and computer-assisted intervention (2015):
Springer, pp. 234-241.
[7] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely
connected convolutional networks," in Proceedings of the IEEE
conference on computer vision and pattern recognition (2017).
[8] V. Bewick, L. Cheek, and J. Ball, "Statistics review 13: receiver operating
characteristic curves," Critical care (2004).