Research

JAMA Otolaryngology–Head & Neck Surgery | Original Investigation

A Deep Learning Approach to Predict Conductive Hearing Loss

in Patients With Otitis Media With Effusion Using Otoscopic Images
Junbo Zeng, MD; Weibiao Kang, MD; Suijun Chen, MD; Yi Lin, MEng; Wenting Deng, MD; Yajing Wang, MD;
Guisheng Chen, MD; Kai Ma, PhD; Fei Zhao, MD, PhD; Yefeng Zheng, PhD; Maojin Liang, MD; Linqi Zeng, MD;
Weijie Ye, MD; Peng Li, PhD; Yubin Chen, MD; Guoping Chen, MD; Jinliang Gao, MD; Minjian Wu, MD;
Yuejia Su, MD; Yiqing Zheng, MD; Yuexin Cai, MD, PhD

Supplemental content
IMPORTANCE Otitis media with effusion (OME) is one of the most common causes of
acquired conductive hearing loss (CHL). Persistent hearing loss is associated with poor
childhood speech and language development and other adverse consequence. However, to
obtain accurate and reliable hearing thresholds largely requires a high degree of cooperation
from the patients.
OBJECTIVE To predict CHL from otoscopic images using deep learning (DL) techniques and a
logistic regression model based on tympanic membrane features.

DESIGN, SETTING, AND PARTICIPANTS A retrospective diagnostic/prognostic study was

conducted using 2790 otoscopic images obtained from multiple centers between January
2015 and November 2020. Participants were aged between 4 and 89 years. Of 1239
participants, there were 209 ears from children and adolescents (aged 4-18 years [16.87%]),
804 ears from adults (aged 18-60 years [64.89%]), and 226 ears from older people (aged
>60 years, [18.24%]). Overall, 679 ears (54.8%) were from men. The 2790 otoscopic images
were randomly assigned into a training set (2232 [80%]), and validation set (558 [20%]). The
DL model was developed to predict an average air-bone gap greater than 10 dB. A logistic
regression model was also developed based on otoscopic features.

MAIN OUTCOMES AND MEASURES The performance of the DL model in predicting CHL was
measured using the area under the receiver operating curve (AUC), accuracy, and F1 score (a
measure of the quality of a classifier, which is the harmonic mean of precision and recall; a
higher F1 score means better performance). In addition, these evaluation parameters were
compared to results obtained from the logistic regression model and predictions made by
three otologists.

RESULTS The performance of the DL model in predicting CHL showed the AUC of 0.74,
accuracy of 81%, and F1 score of 0.89. This was better than the results from the logistic
regression model (ie, AUC of 0.60, accuracy of 76%, and F1 score of 0.82), and much
improved on the performance of the 3 otologists; accuracy of 16%, 30%, 39%, and F1 scores
of 0.09, 0.18, and 0.25, respectively. Furthermore, the DL model took 2.5 seconds to predict
from 205 otoscopic images, whereas the 3 otologists spent 633 seconds, 645 seconds, and
692 seconds, respectively.

CONCLUSIONS AND RELEVANCE The model in this diagnostic/prognostic study provided

greater accuracy in prediction of CHL in ears with OME than those obtained from the logistic
regression model and otologists. This indicates great potential for the use of artificial
intelligence tools to facilitate CHL evaluation when CHL is unable to be measured.

Author Affiliations: Author

affiliations are listed at the end of this
article.
Corresponding Author: Yuexin Cai,
MD, PhD (caiyx25@mail.sysu.edu.
cn), and Yiqing Zheng, MD (zhengyiq
@mail.sysu.edu.cn), Department of
Otolaryngology, Sun Yat-sen
Memorial Hospital, Sun Yat-sen
JAMA Otolaryngol Head Neck Surg. 2022;148(7):612-620. doi:10.1001/jamaoto.2022.0900 University, 107# Yanjiang West Rd,
Published online May 19, 2022. Guangzhou 510120, China.

612 (Reprinted) jamaotolaryngology.com

© 2022 American Medical Association. All rights reserved.

Downloaded from jamanetwork.com by Abdullah Fatih on 02/29/2024
 A Deep Learning Approach to Predict Conductive Hearing Loss in Patients With Otitis Media With Effusion
O
titis media with effusion (OME) is one of the most com-
mon middle ear conditions presenting at otolaryngol-
ogy and audiology clinics. Evidence suggests that ap-
proximately 90% of preschool children will experience OME
at least once, and 30% to 40% are likely to have recurrent
episodes.1,2 In the natural course of OME, more than 20% of
children will have a conductive hearing loss (CHL) of more
than 35 dB and 5% to 10% more than 50 dB.3-5 The CHL in pa-
tients with OME is largely caused by fluid accumulation in the
middle ear cavity, which increases stiffness and mass of the
tympanum.6 Without timely and appropriate intervention, per-
sistent CHL in preschool children with OME is associated with
poor speech and language development and other adverse con-
sequences, for example behavior issues and educational
difficulties.6-8
According to the clinical practice guidelines recom-
mended by the American Academy of Otolaryngology in 2016,6
otoscopy, pure tone audiometry (PTA), and tympanometry are
suggested as essential tools for OME diagnosis. Although PTA
is considered the gold standard for assessing hearing sensitiv-
ity, it has several limitations owing to the subjective nature of its
measurement, requiring a high degree of cooperation from the
patients to achieve accurate and reliable hearing thresholds.
Indeed, Browning et al9 and Kemaloglu et al10 have indicated
that younger children are unlikely to be able to concentrate long
enough to complete the test.
The otoscope is a fundamental tool used in otologic prac-
tice to examine tympanic membrane condition by observing
its shape, color, and integrity.11,12 However, otoscopic exami-
nation and otoscopic images are currently unable to provide
information on hearing levels. A number of studies have shown
that deep learning (DL) is an effective tool to classify patho-
logic conditions from medical images, information that can
then support diagnosis and treatment.13-15 In the field of otol-
ogy, previous DL studies have mainly used otoscopic images
as input data and focused on how to achieve a high diagnos-
tic accuracy in classifying middle ear disorders.16-22
The aim of this study was to predict the degree of CHL from
otoscopic images using DL techniques and a logistic regres-
sion model based on tympanic membrane features. The im-
portance of this novel research is that it could enable a quick
and accurate prediction of the degree of CHL, and thus pro-
vide solutions to reduce patient waiting time and avoid un-
necessary referral.
Methods
Study Design and Data Set
In this multicenter study, otoscopic images of OME cases were
obtained from 3 tertiary care medical centers in south China,
including Sun Yat-sen Memorial Hospital of Sun Yat-sen
University, The Third Affiliated Hospital of Sun Yat-sen Uni-
versity, and Zhongshan People’s Hospital. All data were col-
lected between January 2015 and November 2020 and re-
viewed retrospectively. Cases of OME were diagnosed
according to clinical otologic practice that included medical
history, physical examination with otoscopes, and audiologi-
jamaotolaryngology.com (Reprinted) JAMA Otolaryngology–Head & Neck Surgery July 2022 Volume 148, Number 7
© 2022 American Medical Association. All rights reserved.
Downloaded from jamanetwork.com by Abdullah Fatih on 02/29/2024
Original Investigation Research
Key Points
Question How effective are deep learning (DL) models in
predicting conductive hearing loss (CHL) from otoscopic images of
ears with otitis media with effusion?
Findings In this diagnostic/prognostic study including 2790
otoscopic images from 1239 patients, a DL model was developed
using otoscopic images and validated in multiple centers. The DL
model predicted CHL with an accuracy of 81%.
Meaning This research to predict CHL using a DL model revealed
great potential for the quick and accurate prediction of degree of
conductive hearing loss, and offers opportunities to reduce
patient waiting time and avoid unnecessary referral.
cal tests (PTA and tympanometry). Inclusion criteria re-
quired that otoscopic images and audiological assessment re-
sults were measured at the same time and on individual OME
ears. Ears with OME with a history of middle ear surgery (eg,
grommet insertion) were excluded. The high incidence of na-
sopharyngeal tumors in the south of China is associated with
8% to 29% of patients there having OME after radiotherapy.23,24
However, owing to the low incidence in the other regions of
China, to avoid confusion, radiation-induced OME ears were ex-
cluded in the present study. In addition, otoscopic images of OME
ears labeled with an incorrect ear side were excluded. This study
was approved by the institutional review board of Sun Yat-sen
Memorial Hospital (reference number SYSEC-KY-KS-2021-191),
and written informed consent was waived owing to the retro-
spective nature of the study.
Quality Control for Otoscopic Images
All otoscopic images were taken by otologists in routine
otologic practice. They were captured in standard high-
resolution using 2 different otoscopes, a 4-mm STORZ 0° en-
doscope (KARL STORZ, Germany) or a 2.7-mm TIAN SONG 0°
endoscope (TIAN SONG, China). Images were stored in JPEG
or PNG format in the range 500 × 500 to 700 × 700 pixels. Only
otoscopic images where the pars flaccida and pars tensa were
captured intact were included in this study. Furthermore, only
otoscopic images taken using standard white light were deemed
eligible for developing the DL model. Overexposed and un-
derexposed images, and images where the tympanic mem-
brane was largely blocked by earwax were excluded. The 1 to
3 best quality otoscopic images captured from different angles
of each ear were selected for analysis. Overall, PTA hearing
thresholds at frequencies of 500, 1000, 2000, and 4000 Hz were
measured in left and right ears. The air-bone gap (ABG) was cal-
culated at each frequency and an average of the 4 frequencies
calculated. We chose averaged ABG across the 4 frequencies
greater than 10 dB as the criteria of conductive hearing loss.25
The severity of OME and changes to the tympanic membrane is
associated with different degrees of conductive hearing loss at
different individual frequencies.26-28 Consequently, the DL
model was used to predict CHL using different criteria: ABG
greater than 10 dB at each frequency, averaged air conduction
hearing threshold more than 500, 1000, 2000, 4000 Hz greater
than 25 dB, and averaged ABG of the 4 frequencies greater than
613
 Research Original Investigation A Deep Learning Approach to Predict Conductive Hearing Loss in Patients With Otitis Media With Effusion
20 dB. Changes in the tympanic membrane used to develop
the logistic regression model included attic retraction pocket,
atelectasis, type of fluid (full, air-fluid level, bubble), color of
tympanic membrane, color of fluid, myringosclerosis, and
opacification.16,25,27 An otologist (J.B.Z.) was assigned to col-
lect otoscopic images, other clinical data, and record changes
in the tympanic membrane. Another otologist (Y.X.C.) with
more than 10 years of clinical experience reviewed the data in-
dependently. Any discrepancies were discussed with an-
other experienced otologist (S.J.C.) until consensus was
reached. The detailed criteria for the otologist to predict CHL
on the basis of otoscopic images are described in the eMethods
in the Supplement.
Convolutional Neural Network Model
Development and Validation
Convolutional neural network (CNN) model development and
validation included data preprocessing, strategies to mini-
mize overfittings, model selection, construction of a predic-
tion system, and main hyperparameters and are detailed in
the eResults in the Supplement. To evaluate the performance
of the DL model, the discriminative performance was
assessed from the area under the receiver operating character-
istic (AUC) curve, accuracy, F1 score (a measure of the quality
of a classifier, which is the harmonic mean of precision and
recall, higher F1 score means better performance) and com-
parison with otologists’ prediction and the outcomes
obtained using the logistic regression model, all detailed in
eFigure 1 in the Supplement.
Deep Learning Model Interpretability
To interpret how the DL model produced the diagnostic out-
puts it was crucial to visualize what the discriminative
regions in the otoscopic images used were by the DL model.
A gradient-weighted heatmap was obtained from the final
layer of the neural network, which highlighted the discrimi-
native regions in the original otoscopic image using blue to
red colors.
Experimental Equipment and Statistical Analysis
The experiments were performed using Python (version 3.6 in
Keras, version 2.2.4) to develop the diagnostic model based
on TensorFlow with 4 Titan XP 256 GB GPU (version 1.12.0).
All statistical analyses were carried out using SPSS statistical
software (version 26, IBM). The categorical variables were com-
pared using the χ2 test and Fisher exact test. The continuous
variables were compared using the Kruskal-Wallis rank-sum
test. The AUC, accuracy, and F1 score were calculated, and 95%
CIs of AUC were estimated using bootstrapping with 1000 rep-
licates. Two-sided P < .05 indicated statistical significance.
Results
Summary of Otoscopic Images
After application of inclusion and exclusion criteria, a total of
1239 ears with clinical data and 2790 otoscopic images were
taken from the patients for use in model development and in-
614 JAMA Otolaryngology–Head & Neck Surgery July 2022 Volume 148, Number 7 (Reprinted)
© 2022 American Medical Association. All rights reserved.
Downloaded from jamanetwork.com by Abdullah Fatih on 02/29/2024
ternal validation. Within the data set, 738 ears with 1678 oto-
scopic images were from Sun Yat-sen Memorial Hospital of Sun
Yat-sen University; and 330 ears with 721 otoscopic images were
from The Third Affiliated Hospital of Sun Yat-sen University;
and 171 ears with 391 otoscopic images were from Zhongshan
people’s hospital. The OME ears with otoscopic images were
randomly split into the training set (1034 ears with 2232
otoscopic images) and validation set (205 ears with 558
otoscopic images).
Summary of Clinical Characteristics
Clinical diagnosis of CHL was taken on the basis of the aver-
aged ABG across 4 frequencies being greater than 10 dB. Over-
all, 996 ears (80.4%) demonstrated CHL. The data set com-
prised 1239 participants aged between 4 and 89 years. This
included 209 ears from children and adolescents (aged 4-18
years 16.9%), 804 ears from adults (aged 18-60 years, 64.9%),
and 226 ears from older people (aged >60 years, 18.2%); 679
ears (54.8%) were from men. Ears with tympanogram A (5.0%)
were further confirmed with tympanotomy, whereas 835 ears
(67.40%) with tympanogram B, and 342 ears (27.6%) with tym-
panogram C did not need further testing to confirm.
Atelectasis (753 [60.7%]) and attic retraction pocket (922
[74.3%]) of any stage were most commonly seen. Pearl gray (736
[59.5%]) was the most common tympanic membrane color, and
amber (1088 [87.8%]) the most common fluid color. Full (898
[72.5%]) was the most common type of fluid status. Statisti-
cal analysis revealed that demographic factors, type of tym-
panogram, sex, changes of tympanic membrane, and age were
not significantly different between the training and valida-
tion sets. Detailed characteristics of the ears in the training and
validation sets are summarized and compared in Table 1.
Evaluation of DL Model Performance
To commence, receiver operating characteristic (ROC) curves
were plotted and AUC of the DL model calculated to show di-
agnostic performance. The DL diagnostic model achieved an
AUC of 0.74 (95% CI, 0.67-0.80) with ABG over 4 frequencies
greater than 10 dB. The F1 score was 0.97 (95% CI, 0.95-
0.99), and accuracy 81% (95% CI, 77%-85%). With ABG at 0.5
kHz, the AUC was 0.74 (95% CI, 0.65-0.83), 1 kHz 0.79 (95%
CI, 0.68-0.85), 2 kHz 0.68 (95% CI, 0.63-0.75), and at 4 kHz
0.65 (95% CI, 0.55-0.78), respectively. When the DL model was
applied to diagnose those with a CHL using averaged ABG
across the 4 frequencies greater than 20 dB, the AUC was 0.69
(95% CI, 0.60-0.78), with F1 score of 0.83 (95% CI, 0.77-
0.88), and accuracy of 73% (95% CI, 65%-80%) (Table 2). When
human and artificial intelligence were compared, the perfor-
mance of the 3 otologists in predicting CHL by reviewing the
otoscopic images was lower compared with the DL model in
terms of accuracy (81% vs 30%, 16%, and 39%). The DL model
took just 2.5 seconds to complete its prediction of CHL, whereas
the 3 otologists took 633 seconds, 645 seconds, and 692 sec-
onds, respectively. The logistic regression model based on tym-
panic membrane changes provided a lower AUC of 0.62 (95%
CI, 0.45-0.75), F1 score of 0.82 (95% CI, 0.79-0.85), and accu-
racy of 76% (95% CI, 70%-81%) (eFigure 2 in the Supplement;
Table 3).
jamaotolaryngology.com
 A Deep Learning Approach to Predict Conductive Hearing Loss in Patients With Otitis Media With Effusion Original Investigation Research

Table 1. Basic Characteristics of Participants

No. (%)
Characteristic Training set (n = 1024) Validation set (n = 215) P value
Age, median (IQR), y 44 (27-58) 44 (25-60) .93
Male sex 562 (54.9) 117 (54.4) .90
Tympanogram type
Tympanogram A 52 (5.0) 10 (4.7)
Tympanogram B 698 (68.2) 137 (13.4) .35
Tympanogram C 274 (26.8) 68 (31.6)
Pathologic features of tympanic membranes
Atelectasis 633 (61.8) 120 (55.9)
Stage I 562 (54.9) 106 (49.3)
Stage II 51 (5.0) 11 (5.1)
.44
Stage III 10 (1.0) 1 (0.5)
Stage IV 10 (1.0) 2 (0.9)
Attic retraction pocket 768 (75) 154 (71.6)
Stage I 561 (54.8) 111 (51.6)
Stage II 189 (18.5) 38 (17.7)
.30
Stage III 9 (0.9) 4 (1.9)
Stage IV 9 (0.9) 1 (0.5)
Color of tympanic membrane
Pearl gray 593 (57.9) 143 (66.5)
White 41 (4) 19 (8.8) .07
Transparent 179 (17.5) 53 (24.7)
Color of fluid
Amber 904 (88.3) 184 (85.6)
Transparent 5 (0.5) 2 (0.9)
.54
Yellow-white 39 (3.8) 12 (5.7)
Dark 76 (7.6) 17 (7.9)
Volume of fluid
Bubble 89 (8.7) 17 (7.9)
Air-fluid level 192 (18.8) 43 (20.0) .87
Full 743 (72.6) 155 (72.1)
Atrophy 97 (9.5) 20 (9.3) .94
Opacification
Marginal opacification 117 (11.4) 17 (7.9)
.01
Full opacification 53 (5.2) 23 (10.7)
Myringosclerosis 56 (5.5) 9 (4.2) .44

Table 2. Performance in Different Characteristics Groups

Group AUC (95%CI) F1 score (95% CI) Accuracy, % (95% CI)

Male 0.75 (0.62-0.86) 0.91 (0.88-0.95) 84 (79-90)
Female 0.71 (0.60-0.80) 0.86 (0.81-0.91) 76 (68-83)
Age, y
<18 0.65 (0.47-0.81) 0.83 (0.76-0.90) 72 (62-82)
18-60 0.76 (0.67-0.90) 0.89 (0.85-0.93) 80 (75-87)
>60 0.78 (0.63-0.90) 0.92 (0.88-0.96) 86 (78-93)
Tympanogram A 0.78 (0.44-1.00) 0.58 (0.21-0.80) 49 (20-73)
Tympanogram B 0.74 (0.64-0.84) 0.93 (0.90-0.95) 87 (82-91)
Abbreviation: AUC, area under the
Tympanogram C 0.72 (0.60-0.82) 0.85 (0.78-0.90) 74 (66-82)
receiver operating curve.

Performance of the model was lower in the children and C had a higher accuracy than tympanogram A (87%, 74%, 49%,
adolescent (aged 4-18 years) group than in the adult (aged 18-60 respectively) (Table 2). The clinical characteristics of OME ears
years) and older (aged >60 years) groups. Tympanogram B and where CHL was predicted rightly or wrongly by the DL are

jamaotolaryngology.com (Reprinted) JAMA Otolaryngology–Head & Neck Surgery July 2022 Volume 148, Number 7 615

© 2022 American Medical Association. All rights reserved.

Downloaded from jamanetwork.com by Abdullah Fatih on 02/29/2024
 Research Original Investigation A Deep Learning Approach to Predict Conductive Hearing Loss in Patients With Otitis Media With Effusion

Table 3. Performance of the DL Model, Logistic Regression Model, and Otologists

Group AUC (95% CI) F1 score (95% CI) Accuracy, % (95% CI)
Otologist 1 (5 y) NA 0.18 (0.20-0.38) 30 (18-33)
Otologist 2 (10 y) NA 0.09 (0.06-0.24) 16 (11-23)
Otologist 3 (10 y) NA 0.25 (0.22-0.47) 39 (22-48)
ABG of 500 Hz >10 dBa 0.74 (0.65-0.83) 0.87 (0.81-0.90) 78 (70-84)
ABG of 1000 Hz >10 dBa 0.79 (0.68-0.85) 0.92 (0.86-0.94) 85 (78-88)
ABG of 2000 Hz >10 dBa 0.68 (0.63-0.75) 0.53 (0.48-0.59) 66 (60-73)
ABG of 4000 Hz >10 dBa 0.65 (0.55-0.78) 0.81 (0.75-0.87) 72 (63-80) Abbreviations: ABG, air-bone gap;
Averaged ABG>10 dBa 0.74 (0.67-0.81) 0.89 (0.86-0.92) 81 (77-85) AUC, area under the receiver
operating curve; DL, deep learning;
Averaged ABG>20 dBa 0.69 (0.60-0.78) 0.83 (0.77-0.88) 73 (65-80)
NA, not applicable.
Logistic regression model 0.62 (0.45-0.75) 0.82 (0.79-0.85) 76 (70-81) a
Deep learning model.

Table 4. Characteristics of the DL Model Missed and Identified CHL

No. (%)
Characteristic Missed (n = 40) Identified (n = 175) P value
Age, median (IQR), y 39 (18-65) 46 (29-60) .11
Male sex 16 (40) 101 (57.8) .04
Tympanogram
Tympanogram A 6 (15) 4 (2.3)
Tympanogram B 19 (47.5) 118 (67.4) .001
Tympanogram C 15 (37.5) 53 (30.3)
Features of tympanic membranes
Atelectasis 22 (55) 98 (56)
Stage I 21 (52.5) 85 (48.6)
Stage II 1 (2.5) 10 (5.7)
.01
Stage III 0 1 (0.6)
Stage IV 0 2 (1.1)
Attic retraction pocket 24 (60) 130 (74.3)
Stage I 22 (55) 89 (50.9)
Stage II 2 (5) 36 (20.6)
.07
Stage III 0 4 (2.3)
Stage IV 0 1 (0.6)
Color of tympanic membranes
Pearl gray 26 (65) 117 (66.9)
White 3 (7.5) 16 (9.1) .87
Transparent 11 (27.5) 42 (24.0)
Color of fluid
Amber 34 (85) 150 (85.7)
Transparent 0 2 (1.1)
.85
Yellow-white 3 (7.5) 9 (5.1)
Dark 3 (7.5) 14 (8.0)
Volume of fluid
Bubble 8 (20) 9 (5.1)
Air-fluid level 16 (40) 27 (15.4) <.001
Full 16 (40) 139 (79.4)
Atrophy 1 (2.5) 19 (10.9) .10
Opacification
Marginal opacification 3 (7.5) 14 (8.0)
.70
Full opacification 3 (7.5) 21 (12.0)
Abbreviations: CHL, conductive
Myringosclerosis 0 9 (5.1) .14
hearing loss; DL, deep learning.

illustrated in Table 4. A higher percentage of wrong cases was traction pocket (55.0% vs 50.9%), and bubble (20.0% vs 5.1%),
seen in stage I atelectasis (52.5% vs 48.6%), stage I attic re- with few errors in myringosclerosis (0% vs 5.1%).

616 JAMA Otolaryngology–Head & Neck Surgery July 2022 Volume 148, Number 7 (Reprinted) jamaotolaryngology.com

© 2022 American Medical Association. All rights reserved.

Downloaded from jamanetwork.com by Abdullah Fatih on 02/29/2024
 A Deep Learning Approach to Predict Conductive Hearing Loss in Patients With Otitis Media With Effusion Original Investigation Research

Figure. Heatmap Illustration Overlaid by Deep Learning on Original Otoscopic Images

A No tympanic membrane changes

B Air-fluid level

C Atelectasis

D Attic retraction pocket

The red color represents a high

discriminative region of the otoscopic
images for identification of
conductive hearing loss. A, No
tympanic membrane changes;
B, air-fluid level; C and D, atelectasis
and attic retraction pocket.

The DL Model Interpretability

The heatmap (Figure) highlights the changes in the tympanic Discussion
membrane in the otoscopic images. These regions are what
otologists based their estimates of CHL on, such as atelecta- The importance of this novel research to predict hearing level
sis, attic retraction pocket, and air-fluid level. using a DL model is that it provides solutions to reduce pa-

jamaotolaryngology.com (Reprinted) JAMA Otolaryngology–Head & Neck Surgery July 2022 Volume 148, Number 7 617

© 2022 American Medical Association. All rights reserved.

Downloaded from jamanetwork.com by Abdullah Fatih on 02/29/2024
 Research Original Investigation A Deep Learning Approach to Predict Conductive Hearing Loss in Patients With Otitis Media With Effusion
tient waiting time and avoid unnecessary referral. In the pre-
sent study, we have demonstrated that DL was used to pre-
dict CHL in averaged ABG greater than 10 dB (accuracy of 81%).
Notably, the DL model acquired the highest accuracy at 1 kHz
(85%). The DL model provided values of AUC, F1 score, and ac-
curacy greater than the logistic regression model and otolo-
gists reviewing otoscopic images. Moreover, the DL model only
took 2.5 seconds to complete prediction of CHL in the valida-
tion set, whereas the 3 otologists took 633 seconds, 645 sec-
onds, and 692 seconds, respectively. The study also demon-
strated that the DL model used features changes in the
tympanic membrane to predict CHL, such as atelectasis, attic
retraction pocket, and air-fluid level.
Evidence has shown that the applications of artificially in-
telligent technology in health care may be an alternative so-
lution, or provide additional support, for diagnostic decision
making. In the field of ENT and clinical audiology, previous
studies have shown DL models to be useful diagnostic tools
to assist otologists in terms of distinguishing various otologic
diseases on the basis of otoscopic images.17-22 For example,
Cai et al22 proved the capability of artificial intelligence to
diagnose OME automatically with an accuracy of 93%. In ad-
dition, a recent literature review29 indicated that DL models
have achieved accuracy from 75.3% to 99% in predicting noise-
induced hearing loss. However, it is noteworthy that the DL
algorithm for predicting the degree of CHL using otoscopic im-
ages did not attain the excellent performance seen in auto-
mated diagnosis of ear diseases in previous studies (ie, accu-
racy over 90%).18-22 The possible explanation is that OME may
cause structural alterations in the tympanic cavity,12 not just
tympanic membrane changes. The otoscopic images only pro-
vide features of the tympanic membrane and fluid, without
providing other features associated with CHL, such as status
of the structures in the tympanic cavity and mobility of the
tympanic membrane.25,30 Similar experiences occurred in
studies in other fields. For example, Noorbakhsh et al31 used
hematoxylin and eosin–scanned images to develop CNN ar-
chitectures for tumor or normal and gene variation classifica-
tion. Their CNN architectures acquired an AUC of 0.99 in 19
cancer type diagnosis, with only AUCs of 0.65 to 0.80 in TP53
variations. In addition, Coudary et al32 developed a DL model
that classified lung histopathologic slides into adenocarci-
noma (LUAD), squamous cell carcinoma, and normal lung tis-
sue. This model showed an averaged AUC of 0.97. When DL
was used to predict commonly mutated genes in LUAD, the
AUCs were decreased, ranging from 0.64 to 0.86.
The heatmap in the Figure shows that tympanic mem-
brane changes such as atelectasis and attic retraction pocket
are the distinctive regions used by the DL model to predict CHL.
Previous studies have correlated these features with CHL.25,27
Tympanic membrane changes and fluid could limit the mo-
bility of the tympanic membrane and cause erosion of middle
ear structures.6,25,27,30,33 However, otologists and the logistic
regression model based on tympanic membrane changes show
a worse performance in predicting CHL. The possible reason
is that these tympanic membrane changes were classified for
OME diagnostic assessment, rather than CHL assessment.34-36
Moreover, the manual classification of tympanic membrane
618 JAMA Otolaryngology–Head & Neck Surgery July 2022 Volume 148, Number 7 (Reprinted)
© 2022 American Medical Association. All rights reserved.
Downloaded from jamanetwork.com by Abdullah Fatih on 02/29/2024
changes did not contain sufficient detail to reflect the condi-
tion of tympanic membrane, such as air-fluid level, not sup-
plying information on how much fluid was in the tympanic cav-
ity. However, the DL model used the whole otoscopic image
as input data to avoid missing important detailed informa-
tion. As a result, the DL model offered a faster and more
accurate prediction of CHL than the otologists, which may be
useful when applied to the clinical environment.
Although the group younger than 18 years showed the low-
est accuracy level (72%), it was still significantly higher than
the prediction perceived by parents using a symptom ques-
tionnaire in the study by Swierniak et al,37 with an accuracy
of 17.8%. Previous studies have suggested that the diagnostic
accuracy for detecting OME using otoscopy is between 60%
and 70%,38 whereas the sensitivity and specificity of tympa-
nometry can reach to 70% to 90% for the detection of OME,
but it is dependent on patient cooperation.39,40 Further-
more, it should be noted that there are several influencing
factors that affect the observation of the tympanic mem-
brane and thus the accuracy in diagnosis during otoscopic
examination, such as in the conditions when the external
ear canal is partly or completely blocked by cerumen, or when
children are unable or unwilling to cooperate with the
examination.41-44 Therefore, it is crucial to obtain high-quality
otoscopic images as input data to ensure a reliable and accurate
DL model for predicting CHL in ears with OME.
These results that explore the relationship between oto-
scopic images and CHL provide great potential for internal
medicien physicians, general practitioners, pediatricians, and
other clinicians in rural areas to use the current DL model to
predict CHL by sending their otoscopic images to a central-
ized server for diagnosis of OME. Patients can then be triaged
to otologists through new medical patterns.
Limitations and Future Studies
Although our algorithm achieved an overall accuracy greater
than 80%, there are some limitations to this study. First, ow-
ing to the nature of retrospective data, lack of a standardized
otoscopic image acquisition protocol may have limited their
quality and consistency. Therefore, to improve the DL model
performance, further prospective randomized studies with
standardized otoscopic image acquisition and annotation
protocols are recommended. Second, the mobility of the tym-
panic membrane, status of structures in the tympanic cavity
and other clinical factors associated with CHL could have been
included in the prediction model. Classifying on the basis of
morphological features of the tympanic membrane and their
changes could improve performance accuracy in evaluating
hearing loss.
Conclusions
In this diagnostic/prognostic study, a clinically useful DL model
has been developed using DL training on 2790 otoscopic im-
ages from patients located in multiple centers. These results first
show that DL algorithms can predict CHL using otoscopic im-
ages to an accuracy of 81%. The DL model acquired a better per-
jamaotolaryngology.com
 A Deep Learning Approach to Predict Conductive Hearing Loss in Patients With Otitis Media With Effusion
formance in less time than a logistic regression model and otolo-
gists at predicting CHL. We intend to improve the performance
and application of this DL model with a larger prospective study.
ARTICLE INFORMATION
Accepted for Publication: April 1, 2022.
Published Online: May 19, 2022.
doi:10.1001/jamaoto.2022.0900
Author Affiliations: Department of
Otolaryngology, Sun Yat-sen Memorial Hospital,
Sun Yat-sen University, Guangzhou, China (J. Zeng,
S. Chen, Deng, Wang, Guisheng Chen, Liang, Wu,
Su, Yiqing Zheng, Cai); The second Hospital,
Medical College, Shantou University, Shantou,
Guangdong Province, China (Kang); Jarvis Lab,
Tencent, Shen Zhen city, Guangdong Province,
China (Lin, Ma, Yefeng Zheng); Hong Kong
University of Science and Technology, Hong Kong,
China (Lin); Centre for Speech and Language
Therapy and Hearing Science, Cardiff School of
Sport and Health Sciences, Cardiff Metropolitan
University, Wales, United Kingdom (Zhao);
Zhongshan School of Medicine, Sun Yat-sen
University, Guangzhou, China (L. Zeng, Ye);
Department of Otolaryngology–Head and Neck
Surgery, The Third Affiliated Hospital of Sun Yat-sen
University, Guangzhou, China (Li, Y. Chen);
Department of Otolaryngology, Zhongshan City
People’s Hospital, Zhongshan Affiliated Hospital of
Sun Yat-sen University, Zhongshan, Guangdong
Province, China (Guoping Chen); Department of
Otolaryngology, Shenzhen Baoan Women’s and
Children’s Hospital, Shenzhen, Guangdong
Province, China (Gao); Shenzhen-Shanwei Central
Hospital, Sun Yat-sen Memorial Hospital, Sun
Yat-sen University, Shanwei City, Guangdong
Province, China (Yiqing Zheng, Cai).
Author Contributions: Drs J. Zeng and Cai had full
access to all of the data in the study and take
responsibility for the integrity of the data and the
accuracy of the data analysis. Drs J. Zeng and Kang
contributed equally to the work.
Concept and design: J. Zeng, S. Chen, Ma, Zhao,
Yefeng Zheng, Yiqing Zheng, Cai.
Acquisition, analysis, or interpretation of data: J.
Zeng, Kang, Deng, Wang, Guisheng Chen, Zhao,
Liang, I. Zeng, Ye, Li, Y. Chen, Guoping Chen, Gao,
Su, YiQing Zheng, Cai.
Drafting of the manuscript: J. Zeng, Kang, Zhao, I.
Zeng, Ye, Yiqing Zheng, Cai.
Critical revision of the manuscript for important
intellectual content: J. Zeng, S. Chen, Deng, Wang,
Guisheng Chen, Ma, Zhao, Yefeng Zheng, Liang, Li,
Y. Chen, Guoping Chen, Gao, Su, Yiqing Zheng, Cai.
Statistical analysis: J. Zeng, Wang, Ye.
Obtained funding: Cai.
Administrative, technical, or material support: Kang,
S. Chen, Deng, Guisheng Chen, Yefeng Zheng,
Liang, Li, Y. Chen, Guoping Chen, Gao, Su, Yiqing
Zheng, Cai.
Supervision: J. Zeng, Kang, Ma, Zhao, Yefeng Zheng,
I. Zeng, Yiqing Zheng, Cai.
Conflict of Interest Disclosures: None reported.
Funding/Support: This work was funded by the
Medical artificial intelligence project of Sun Yat-sen
Memorial Hospital (YXYGZN201904), Key R&D
Program of Guangdong Province, China (Grant No.
2018B030339001), National Natural Science
Foundation of China (Grant No. 82071062), and
Guangdong Basic and Applied Basic Research
jamaotolaryngology.com
© 2022 American Medical Association. All rights reserved.
Downloaded from jamanetwork.com by Abdullah Fatih on 02/29/2024
A standardized otoscopic images acquisition and annotation pro-
tocol should be discussed and published, which will further en-
hance DL used on otoscopic images.
Foundation (Grant No. 2021A1515012038). Dr Zhao
was funded by NIHR (AI Award, 02305), Sêr Cymru
III Enhancing Competitiveness Infrastructure Award
(MA/KW/5554/19), Cardiff Metropolitan University
The Global Academies Research and Innovation
Development Fund.
Role of the Funder/Sponsor: The funding sources
had no role in the design and conduct of the study;
collection, management, analysis, and
interpretation of the data; preparation, review, or
approval of the manuscript; and decision to submit
the manuscript for publication.
Additional Contributions: We are thankful for the
proofreading of Professor Christopher Wigham,
PhD, Swansea University. He was not compensated.
REFERENCES
1. Tos M. Epidemiology and natural history of
secretory otitis. Am J Otol. 1984;5(6):459-462.
2. Williamson IG, Dunleavey J, Bain J, Robinson D.
The natural history of otitis media with effusion--a
three-year study of the incidence and prevalence of
abnormal tympanograms in four South West
Hampshire infant and first schools. J Laryngol Otol.
1994;108(11):930-934.
3. Fria TJ, Cantekin EI, Eichler JA. Hearing acuity of
children with otitis media with effusion. Arch
Otolaryngol. 1985;111(1):10-16. doi:10.1001/archotol.
1985.00800030044003
4. Roberts J, Hunter L, Gravel J, et al. Otitis media,
hearing loss, and language learning: controversies
and current research. J Dev Behav Pediatr. 2004;25
(2):110-122. doi:10.1097/00004703-200404000-
00007
5. Vanneste P, Page C. Otitis media with effusion in
children: pathophysiology, diagnosis, and
treatment. A review. J Otol. 2019;14(2):33-39. doi:
10.1016/j.joto.2019.01.005
6. Rosenfeld RM, Shin JJ, Schwartz SR, et al.
Clinical practice guideline: otitis media with effusion
executive summary (update). Otolaryngol Head
Neck Surg. 2016;154(2):201-214. doi:10.1177/
0194599815624407
7. Ito M, Takahashi H, Iino Y, et al. Clinical practice
guidelines for the diagnosis and management of
otitis media with effusion (OME) in children in
Japan, 2015. Auris Nasus Larynx. 2017;44(5):501-508.
doi:10.1016/j.anl.2017.03.018
8. Bess FH, Dodd-Murphy J, Parker RA. Children
with minimal sensorineural hearing loss:
prevalence, educational performance, and
functional status. Ear Hear. 1998;19(5):339-354.
doi:10.1097/00003446-199810000-00001
9. Browning GG; MRC Multi-centre Otitis Media
Study Group. Influence of age, type of audiometry
and child’s concentration on hearing thresholds. Br
J Audiol. 2000;34(4):231-240. doi:10.3109/
03005364000000133
10. Kemaloğlu YK, Gündüz B, Gökmen S, Yilmaz M.
Pure tone audiometry in children. Int J Pediatr
Otorhinolaryngol. 2005;69(2):209-214. doi:10.1016/
j.ijporl.2004.08.018
(Reprinted) JAMA Otolaryngology–Head & Neck Surgery July 2022 Volume 148, Number 7
Original Investigation Research
11. Lieberthal AS, Carroll AE, Chonmaitree T, et al.
The diagnosis and management of acute otitis
media. Pediatrics. 2013;131(3):e964-e999. doi:10.
1542/peds.2012-3488
12. Areias B, Parente MPL, Santos C, Gentil F, Natal
Jorge RM. The human otitis media with effusion:
a numerical-based study. Comput Methods Biomech
Biomed Engin. 2017;20(9):958-966. doi:10.1080/
10255842.2017.1316382
13. Gargeya R, Leng T. Automated identification of
diabetic retinopathy using deep learning.
Ophthalmology. 2017;124(7):962-969. doi:10.1016/
j.ophtha.2017.02.008
14. Ai T, Yang Z, Hou H, et al. Correlation of chest
CT and RT-PCR testing for coronavirus disease 2019
(COVID-19) in China: a report of 1014 cases. Radiology.
2020;296(2):E32-E40. doi:10.1148/radiol.
2020200642
15. Bai HX, Wang R, Xiong Z, et al. Artificial
intelligence augmentation of radiologist
performance in distinguishing COVID-19 from
pneumonia of other origin at chest CT. Radiology.
2020;296(3):E156-E165. doi:10.1148/radiol.
2020201491
16. Myburgh HC, van Zijl WH, Swanepoel D,
Hellström S, Laurent C. Otitis media diagnosis for
developing countries using tympanic membrane
image-analysis. EBioMedicine. 2016;5:156-160.
doi:10.1016/j.ebiom.2016.02.017
17. Myburgh HC, Jose S, Swanepoel DW, Laurent C.
Towards low cost automated smartphone- and
cloud-based otitis media diagnosis. Biomed Signal
Process Control. 2018;39:34-52. doi:10.1016/
j.bspc.2017.07.015
18. Cha D, Pae C, Seong SB, Choi JY, Park HJ.
Automated diagnosis of ear disease using ensemble
deep learning with a big otoendoscopy image
database. EBioMedicine. 2019;45:606-614. doi:10.
1016/j.ebiom.2019.06.050
19. Lee JY, Choi S-H, Chung JW. Automated
classification of the tympanic membrane using a
convolutional neural network. Appl Sci (Basel).
2019;9(9). doi:10.3390/app9091827
20. Khan MA, Kwon S, Choo J, et al. Automatic
detection of tympanic membrane and middle ear
infection from oto-endoscopic images via
convolutional neural networks. Neural Netw. 2020;
126:384-394. doi:10.1016/j.neunet.2020.03.023
21. Wu Z, Lin Z, Li L, et al. Deep learning for
classification of pediatric otitis media. Laryngoscope.
2021;131(7):E2344-E2351. doi:10.1002/lary.29302
22. Cai Y, Yu JG, Chen Y, et al. Investigating the use
of a two-stage attention-aware convolutional
neural network for the automated diagnosis of
otitis media from tympanic membrane images:
a prediction model development and validation
study. BMJ Open. 2021;11(1):e041139. doi:10.1136/
bmjopen-2020-041139
23. Chao WY, Wang CF, Chang SJ. Ventilation tube
in adults with middle-ear effusion. J Otolaryngol.
1999;28(5):278-281.
24. Christensen JG, Wessel I, Gothelf AB, Homøe P.
Otitis media with effusion after radiotherapy of the
619
 Research Original Investigation
head and neck: a systematic review. Acta Oncol.
2018;57(8):1011-1016. doi:10.1080/0284186X.2018.
1468085
25. Ungkanont K, Charuluxananan S, Komoltri C.
Association of otoscopic findings and hearing level
in pediatric patients with otitis media with effusion.
Int J Pediatr Otorhinolaryngol. 2010;74(9):1063-1066.
doi:10.1016/j.ijporl.2010.06.006
26. Cai T, McPherson B, Li C, Yang F. Pure tone
hearing profiles in children with otitis media with
effusion. Disabil Rehabil. 2018;40(10):1166-1175.
doi:10.1080/09638288.2017.1290698
27. Canali I, Rosito LPS, Longo VD, da Costa SS.
Audiometric pattern in moderate and severe
tympanic membrane retraction. Otol Neurotol.
2021;42(6):e716-e723. doi:10.1097/MAO.
0000000000003099
28. Cai T, McPherson B. Hearing loss in children
with otitis media with effusion: a systematic review.
Int J Audiol. 2017;56(2):65-76. doi:10.1080/
14992027.2016.1250960
29. Chen F, Cao Z, Grais EM, Zhao F. Contributions
and limitations of using machine learning to predict
noise-induced hearing loss. Int Arch Occup Environ
Health. 2021;94(5):1097-1111. doi:10.1007/
s00420-020-01648-w
30. Takahashi H, Honjo I, Hasebe S, Sudo M,
Tanabe M. The diagnostic and prognostic value of
eardrum mobility in otitis media with effusion. Eur
Arch Otorhinolaryngol. 1999;256(4):189-191. doi:10.
1007/s004050050137
31. Noorbakhsh J, Farahmand S, Foroughi Pour A,
et al. Deep learning-based cross-classifications
620 JAMA Otolaryngology–Head & Neck Surgery July 2022 Volume 148, Number 7 (Reprinted)
© 2022 American Medical Association. All rights reserved.
Downloaded from jamanetwork.com by Abdullah Fatih on 02/29/2024
A Deep Learning Approach to Predict Conductive Hearing Loss in Patients With Otitis Media With Effusion
reveal conserved spatial behaviors within tumor
histological images. Nat Commun. 2020;11(1):6367.
doi:10.1038/s41467-020-20030-5
32. Coudray N, Ocampo PS, Sakellaropoulos T, et al.
Classification and mutation prediction from
non-small cell lung cancer histopathology images
using deep learning. Nat Med. 2018;24(10):1559-1567.
doi:10.1038/s41591-018-0177-5
33. Li Y, Hunter LL, Margolis RH, et al. Prospective
study of tympanic membrane retraction, hearing
loss, and multifrequency tympanometry.
Otolaryngol Head Neck Surg. 1999;121(5):514-522.
doi:10.1016/S0194-5998(99)70049-5
34. Kaleida PH, Fireman P. Diagnostic assessment
of otitis media. Clin Allergy Immunol. 2000;15:
247-262.
35. Sadé J, Berco E. Atelectasis and secretory otitis
media. Ann Otol Rhinol Laryngol. 1976;85(2)(suppl 25
Pt 2):66-72. doi:10.1177/00034894760850S214
36. Tos M, Stangerup SE, Larsen P. Dynamics of
eardrum changes following secretory otitis. a
prospective study. Arch Otolaryngol Head Neck Surg.
1987;113(4):380-385. doi:10.1001/archotol.1987.
01860040042014
37. Swierniak W, Gos E, Skarzynski PH, Czajka N,
Skarzynski H. The accuracy of parental suspicion of
hearing loss in children. Int J Pediatr Otorhinolaryngol.
2021;141:110552. doi:10.1016/j.ijporl.2020.110552
38. American Academy of Family Physicians;
American Academy of Otolaryngology-Head and
Neck Surgery; American Academy of Pediatrics
Subcommittee on Otitis Media With Effusion. Otitis
media with effusion. Pediatrics. 2004;113(5):1412-
1429. doi:10.1542/peds.113.5.1412
39. Watters GW, Jones JE, Freeland AP. The
predictive value of tympanometry in the diagnosis
of middle ear effusion. Clin Otolaryngol Allied Sci.
1997;22(4):343-345. doi:10.1046/j.1365-2273.1997.
00023.x
40. Anwar K, Khan S, Rehman HU, Javaid M,
Shahabi I. Otitis media with effusion: accuracy of
tympanometry in detecting fluid in the middle ears
of children at myringotomies. Pak J Med Sci. 2016;
32(2):466-470. doi:10.12669/pjms.322.9009
41. Brodie KD, David AP, Kriss H, Chan DK.
Outcomes of an early childhood hearing screening
program in a low-income setting. JAMA Otolaryngol
Head Neck Surg. 2022. doi:10.1001/jamaoto.
2021.4430
42. McManus B, Harbarger C, Grillis A, et al.
Otoscopy and tympanometry outcomes from the
National Health and Nutrition Examination Survey
(NHANES). Am J Otolaryngol. 2022;43(2):103332.
doi:10.1016/j.amjoto.2021.103332
43. Esposito S, Bianchini S, Argentiero A, Gobbi R,
Vicini C, Principi N. New approaches and
technologies to improve accuracy of acute otitis
media diagnosis. Diagnostics (Basel). 2021;11(12):
2392. doi:10.3390/diagnostics11122392
44. Marchisio P, Pipolo C, Landi M, et al; Italian
Earwax Study Group. Cerumen: A fundamental but
neglected problem by pediatricians. Int J Pediatr
Otorhinolaryngol. 2016;87:55-60. doi:10.1016/
j.ijporl.2016.05.014
jamaotolaryngology.com