574 IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 18, NO.
2, APRIL 2021
Diabetic Retinopathy Diagnosis Using Multichannel
Generative Adversarial Network
With Semisupervision
Shuqiang Wang , Xiangyu Wang, Yong Hu, Yanyan Shen , Zhile Yang, Min Gan, and Baiying Lei
Abstract— Diabetic retinopathy (DR) is one of the major causes
of blindness. It is of great significance to apply deep-learning
techniques for DR recognition. However, deep-learning algo-
rithms often depend on large amounts of labeled data, which
is expensive and time-consuming to obtain in the medical
imaging area. In addition, the DR features are inconspicuous
and spread out over high-resolution fundus images. Therefore,
it is a big challenge to learn the distribution of such DR
features. This article proposes a multichannel-based generative
adversarial network (MGAN) with semisupervision to grade DR.
The multichannel generative model is developed to generate a
series of subfundus images corresponding to the scattering DR
features. By minimizing the dependence on labeled data, the pro-
posed semisupervised MGAN can identify the inconspicuous
lesion features by using high-resolution fundus images without
compression. Experimental results on the public Messidor data
set show that the proposed model can grade DR effectively.
Note to Practitioners—This article is motivated by the chal-
lenging problem due to the inadequacy of labeled data in
medical image analysis and the dispersion of efficient features in
Manuscript received August 23, 2019; revised December 3, 2019 and
January 31, 2020; accepted March 4, 2020. Date of publication April 9,
2020; date of current version April 7, 2021. This article was recommended
for publication by Lead Guest Editor A. Si and Editor M. Zhang upon
evaluation of the reviewers’ comments. This work was supported by the
National Natural Science Foundations of China under Grant 61872351 and
Grant 61771465, by the International Science and Technology Cooperation
Projects of Guangdong under Grant 2019A050510030, by the Strategic
Priority CAS Project under Grant XDB38000000, by the Major Projects
from General Logistics Department of People’s Liberation Army under
Grant AWS13C008, and by the Shenzhen Key Basic Research Projects
under Grant JCYJ20180507182506416. (Shuqiang Wang and Xiangyu Wang
contributed equally to this work.) (Corresponding authors: Min Gan;
Baiying Lei.)
Shuqiang Wang is with the Shenzhen Institutes of Advanced Technology,
Chinese Academy of Sciences, Shenzhen 518055, China, and also with the
Joint Engineering Research Center for Health Big Data Intelligent Analysis
Technology, Shenzhen 518060, China.
Xiangyu Wang is with the College of Data Science, University of Science
and Technology of China, Hefei 230026, China.
Yong Hu is with the Department of Orthopedics and Traumatology,
The University of Hong Kong, Hong Kong.
Yanyan Shen and Zhile Yang are with the Shenzhen Institutes of Advanced
Technology, Chinese Academy of Sciences, Shenzhen 518055, China.
Min Gan is with the College of Mathematics and Computer Science, Fuzhou
University, Fuzhou 350116, China (e-mail: aganmin@aliyun.com).
Baiying Lei is with the School of Biomedical Engineering, Health
Science Center, Shenzhen University, Shenzhen 518060, China (e-mail:
leiby@szu.edu.cn).
This article has supplementary downloadable material available at
https://ieeexplore.ieee.org, provided by the authors.
Color versions of one or more of the figures in this article are available
online at https://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TASE.2020.2981637
1545-5955 © 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: VIT University. Downloaded on December 02,2021 at 10:52:28 UTC from IEEE Xplore. Restrictions apply.
high-resolution medical images. As for the inadequacy of labeled
data in medical image analysis, the reasons mainly include the
followings: 1) the high-quality annotation of medical imaging
sample depends heavily on scarce medical expertise which is
very expensive and 2) comparing with natural issues, it is more
difficult to collect medical images because of privacy issues. It is of
great significance to apply deep-learning techniques for diabetic
retinopathy (DR) recognition. In this article, the multichannel
generative adversarial network (GAN) with semisupervision is
developed for DR-aided diagnosis. The proposed model can
deal with DR classification problem with inadequacy of labeled
data in the following ways: 1) the multichannel generative
scheme is proposed to generate a series of subfundus images
corresponding to the scattering DR features and 2) the proposed
multichannel-based GAN (MGAN) model with semisupervision
can make full use of both labeled data and unlabeled data.
The experimental results demonstrate that the proposed model
outperforms the other representative models in terms of accuracy,
area under ROC curve (AUC), sensitivity, and specificity.
Index Terms— Computer-aided diagnosis (CAD), diabetic
retinopathy (DR), generative adversarial network (GAN), multi-
channel, semisupervised learning.
I. I NTRODUCTION
D IABETIC retinopathy (DR) is one of the complications
caused by diabetes [1]. The high blood sugar level
damages the blood vessels of the light-sensitive tissue at
the retina. It is one of the main causes of blindness. The
World Health Organization predicts that global diabetics will
reach 4.4% of the population in 2030 and about half of
them will have DR complications [2]. Early diagnosis through
regular screening is important for preventing DR. However,
it is time-consuming for ophthalmologists to diagnose effi-
ciently. In order to reduce the cost of regular screening,
the technology for capturing color fundus images is often
adopted. This approach offers the possibility to make use of
computer-aided diagnosis (CAD) [3], [4] technology, which
has been widely studied in artificial intelligence for healthcare
applications [5]–[9]
For traditional machine learning, most of the efficient fea-
tures need to be identified by an expert manually. Besides,
the performance of the traditional machine-learning meth-
ods often depends on how accurately the features are
extracted [10]–[13]. In recent years, deep-learning technol-
ogy has been widely applied in the field of medical image
analysis [6], [14]. It can learn the high-level features from
images automatically. However, deep-learning models usually
depend on a large amount of labeled data, which is a big
WANG et al.: DR DIAGNOSIS USING MGAN WITH SEMISUPERVISION
challenge to obtain in the medical imaging area. For the
labeling process of DR images, the grading of DR requires
the clinician to extract the lesions and measure the area of
the lesions manually, which is highly time-consuming. Due to
the lack of high-quality labeled data in real applications, it is
difficult to apply the general deep-learning method (such as
GoogleLeNet and ResNet) for DR diagnosis. On the other
hand, hospitals can produce a large number of unlabeled
data containing important potential information in which the
machine-learning model can benefit. Therefore, it is feasible
to employ semisupervised learning for DR recognition task.
As for semisupervised learning, generative adversarial network
(GAN) [15] not only works well in a wide variety of appli-
cations for image generation but also can achieve excellent
performance in semisupervised classification [16].
The high-resolution fundus images are usually employed
to diagnose DR in the clinical treatment. However, there are
several challenges for employing the high-resolution fundus
images and the general GAN model to recognize DR. First,
the proportion of effective DR features is pretty low in
high-resolution fundus images. Feeding such images into a
classifier directly will result in numerous redundant infor-
mation. Due to the microminiature of DR characteristics,
reducing the size of fundus image beforehand will result in
losing the information of tiny lesions. Second, due to the
microminiature of DR features, it is difficult for the general
GAN model to generate high-resolution images, including
detailed semantic information. Third, the locations of DR
lesion are often spread out and exudates may appear in various
locations depending on the patient. Moreover, the lesion area
might surround with noise, such as blood vessels and imaging
shadows. Thus, it is unappealing to extract the lesion by
employing the detection of region of interest (ROI). Based on
the earlier analysis, this article proposes a multichannel-based
semisupervised GAN (SSGAN) for grading DR. The proposed
model can make full use of labeled data and unlabeled data
to recognize DR automatically without losing the original
DR features.
The main contributions of this article are summarized as
follows.
1) A multichannel-based GAN (MGAN) model is pro-
posed, which can generate a series of subfundus images,
including effective local features. All the subfundus
images are then combined to obtain the most represen-
tative features of the entire fundus image. In this way,
the proposed model can deal with the challenge that the
effective DR features (e.g., exudates, microaneurysms,
and bleeding points) are diffuse in the high-resolution
fundus images.
2) The feature extraction scheme is incorporated into the
proposed MGAN framework. This scheme can reduce
the noise from the original fundus images and extract
the scattering lesion features, which can improve the
performance of discriminator.
3) The proposed MGAN can employ both labeled data and
unlabeled data. As far as we know, it is the first time
that the SSGAN is employed for grading DR.
Authorized licensed use limited to: VIT University. Downloaded on December 02,2021 at 10:52:28 UTC from IEEE Xplore. Restrictions apply.
575
The rest of this article is organized as follows. In Section II,
the related works on DR assessment are reviewed. In
Section III, the proposed model is described in detail.
In Section IV, the model is evaluated, and the factors influ-
encing model performance are also analyzed. In Section V,
the experimental results compared with the existing methods
are presented and discussed. Conclusions are summarized in
Section VI.
II. R ELATED W ORKS
In the recent years, CAD technology using machine learning
has been applied to diagnose various diseases. Extensive
research has been carried out on assisted DR diagnosis [18].
For example, Seoud et al. [19] proposed an automatic DR
grading system. In their work, a red lesion detection is
adopted to generate a DR lesion probability map, which
is represented by 35 features, including location, size, and
probability information. Pratt et al. [20] used a convolutional
neural network (CNN) model and color fundus images
for DR classification. They employed data augmentation to
expand the training set and used 80 000 samples to train
the CNN model. Gulshan et al. [21] employed the Inception-
v3 architecture-based model to automate detection of DR and
diabetic macular edema. Haloi et al. [22] proposed a microa-
neurysm detection system for DR detection. They used deep
neural network (DNN) to identify microaneurysm without the
preprocessing steps. The proposed model was evaluated on the
Retinopathy Online Challenge (ROC) and Diaretdb1v2 data-
base.
Costa et al. [23] developed a weakly supervised DR detec-
tion system. They used multiple instance learning (MIL)
algorithms for the joint optimization of the instance encod-
ing and the classification. Shan and Li [24] used the stacked
sparse autoencoder (SSAE) and fundus images for microa-
neurysms classification. Gargeya and Leng [25] developed a
data-driven method using the deep residual learning mech-
anism to learn discriminative features for DR detection.
Antal and Hajdu [26] developed an ensemble-based algorithm
for the DR screening. Costa and Campilho [27] presented the
bag-of-visual-words (BoVW)-based model for DR detection,
and the model was tested on the Messidor and DR2 data
sets. Vo and Verma [28] proposed a DR detection model
by combing kernels with multiple losses network (CKML
Net) and VGGNet with Extra Kernel (VNXK). The exper-
imental results on the EyePACS and Messidor data sets
showed the efficiency of the proposed model. Two pre-
trained [29] CNN models were employed to identify the grade
of DR on fluorescein angiography photographs. To detect
microaneurysm from fundus images, Dai et al. [30] pro-
posed a multisieving CNN framework integrating with the
image-to-text mapping scheme for guiding clinical report.
Cao et al. [31] developed a model by integrating random
forest (RF), neural network (NN), and support vector machine
(SVM) to detect microaneurysm. Moreover, principal com-
ponent analysis (PCA) was employed to reduce the dimen-
sionality of DR image patches. Based on top-performing
supervised CNN, Gondal et al. [32] presented a weakly
576 IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 18, NO. 2, APRIL 2021

TABLE I
R ETINOPATHY G RADE IN M ESSIDOR D ATA S ET

TABLE II
C LASSIFICATION TASK D ESCRIPTION

supervised object localization method to detect DR lesions on
image level. By introducing a heatmap optimization procedure,
Quellec et al. [33] proposed a ConvNets framework to detect
lesions in the context of DR screening. Zhou et al. [34] pro-
posed an MIL-based model for DR detection. Zeng et al. [35]
trained a binocular Siamese-Like CNN with a transfer learning
technique to classify color retinal fundus photographs into
two grades. By employing transfer learning, Li et al. [36]
presented a CNN-based model for DR fundus image classifica-
tion. Similarly, Khandelwal and Mishra [37], Raju et al. [38],
Xu et al. [39] and Ghosh et al. [40] also adopted CNN-based
approach for automatic recognition of DR. Furthermore,
Qummar et al. [41] presented an ensemble of five CNN
models to encode the DR features and improve the clas-
sification performance. Brown et al. [42] employed CNN to
diagnose plus disease in retinopathy of prematurity (ROP)
from retinal photographs. Li et al. [43] developed a novel
deep network OCTD_Net for early-stage DR detection. The
OCTD_Net consisted of two independent networks. One net-
work extracted features from the original optical coherence
tomography (OCT) images, and the other extracted retinal
layer information. Poplin et al. [44] developed a DNN-based
model to predict cardiovascular risk factors from retinal fundus
photographs. Wang et al. [17] proposed a novel zoom-in-net
for DR recognition by employing both the whole retinal
image and its suspicious lesion patches generated by atten-
tion maps. De Fauw et al. [45] first applied a U-Net-based
segmentation network on t3-D OCT scans to generate the
tissue segmentation map, and then, they predict the retinal
disease using a classification network. Based on the pixel-wise
score propagation model, de La Torre et al. [46] presented a
DNN-based interpretable model for DR assessment. With this
model, the generated visual maps can be interpreted.
Most of the abovementioned works are based on super-
vised DNN of which the performance often depends on a
large amount of labeled samples. However, plenty of labeled
samples are always not available for image processing in the
medical field. Some of the abovementioned works resorted to
transfer leaning scheme to deal with this issue, while transfer
learning does not always work in medical image processing.
This article proposes a multichannel based SSGAN for grading
DR. The proposed model can make full use of labeled data and
unlabeled data to recognize DR automatically without losing
the original DR features.
III. M ATERIALS AND M ETHODS
A. Data Set
The proposed model is evaluated on publicly avail-
able Messidor data set (http://www.adcis.net/en/third-party/
messidor/) [47], which contains approximately 1200 digital
fundus images. Each image is obtained by using a Topcon
TRC NW6 nonmydriatic camera and 45◦ of view centered
on the fovea. The ratio of images with pupil dilation to
images without dilation is 2:1. The size of the fundus image
is 1440 × 960, 2240 × 1488, or 2304 × 1536. The label
for each image is provided by ophthalmologists. According
to the number of microaneurysms, hemorrhages, and the
existence of neovascularization, each image is classified as
one of four lesion grades (R0, R1, R2, and R3), as described
in Table I. R0 represents a normal, no lesion fundus image,
R1 and R2 represent the mild and severe nonproliferative
retinal image, respectively, and R3 represents the proliferative
retinopathy image. In the Messidor data set, R0, R1, R2,
and R3 account for 45.5%, 12.75%, 20.58%, and 21.67%
of the total data set, respectively. In addition, the severity
of the lesion is classified according to class labels, as shown
in Table II.
B. DR Feature Extraction
The framework of the proposed DR grading model is shown
in Fig. 1. The proposed model consists of four parts.
1) The DR feature extractor is designed to extract disper-
sive lesion features.
2) The multichannel generative model is designed to gen-
erate a series of subfundus images, including effective
local features.

Authorized licensed use limited to: VIT University. Downloaded on December 02,2021 at 10:52:28 UTC from IEEE Xplore. Restrictions apply.
WANG et al.: DR DIAGNOSIS USING MGAN WITH SEMISUPERVISION
Fig. 1. Framework of the DR grading model.
Fig. 2. Framework of the SSGAN with multichannel input.
3) The discriminator is designed to predict C + 1 classes
by using convolutional layers accounting for multiscale
DR features matching.
4) In order to improve the robustness and accuracy of
the proposed model, an ensemble-based framework is
designed by integrating various trained discriminators.
Each part will be detailed as follows.
Since the fundus image has high resolution with millions
of pixels, feeding such high-resolution fundus images into
GAN directly will cause the number of parameters increasing
Authorized licensed use limited to: VIT University. Downloaded on December 02,2021 at 10:52:28 UTC from IEEE Xplore. Restrictions apply.
577
drastically. Moreover, compressing the fundus image will
destroy tiny DR lesion features, such as bleeding points with
only a few pixels. To solve this problem, a feature extractor
is designed to extract the most representative DR features
from the original fundus image by transferring a portion of
the pretrained network structure.
The feature extractor is shown in Fig. 2. First, the image is
normalized to eliminate the effects of different scales and illu-
minations. Second, the data augmentation strategy is applied
to increase the amount of training data. Third, the large-scale
578 IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 18, NO. 2, APRIL 2021
natural data set is used to pretrain the model and the fundus
image is used to fine-tune the network parameters. In order to
extract the most representative DR features, the densely con-
nected convolutional network (DenseNet) [48] is pretrained to
extract DR features from high-resolution fundus images. The
main idea of DenseNet is to connect each layer’s output to the
subsequent layers by creating a short path from preceding lay-
ers to subsequent layers. Therefore, compared with using other
pretrained models, DenseNet can extract and transfer features
efficiently with fewer parameters. For the feature extractor,
we transfer a portion of the pretrained model to represent
fundus image features. The framework includes a convolution
layer learning the basic features of the retinopathy image and
several dense blocks containing several composite layers. Each
composite layer includes three consecutive operations: batch
normalization (BN), rectifying by linear unit (ReLU) [49],
and convolution operation with a 3 × 3 convolution kernel.
To remove the redundant information, a 1 × 1 convolution
layer is designed in prior to each composite layer. It can
enhance the expression ability of the feature extractor and
decrease computation cost. Likewise, a 1 × 1 convolution
layer and a pooling layer are designed among dense blocks.
They can preserve the effective features, discard redundant
information, and effectively reduce the size of samples.
C. Multichannel-Based SSGAN
The proposed framework includes a multichannel generative
model and a discriminator. Moreover, to improve the robust-
ness and accuracy of the model, the ensemble-based scheme is
employed to integrate a series of well-trained discriminators.
The proposed framework is shown in Fig. 2. Each part will
be detailed as follows.
1) Discriminator: For the proposed multichannel based
SSGAN, the input to the discriminator includes the real DR
features extracted by the feature extractor, and DR features
generated by the multichannel generative model. For the
proposed model, the discriminator is mainly composed of
several convolution layers and a fully connected layer with
a softmax activation function, which predicts the class labels
of the real samples and generated samples.
Supposing that the input DR sample is x and the last layer
outputs a vector (o1 , o2 , . . . , oC ), the probability pd (y = j |x)
that x belongs to class j is given by
exp(o j )
pd (y = j |x) = C . (1)
c=1 exp(oc )
The semisupervised learning strategy is introduced into
discriminator to make full use of the unlabeled samples. The
discriminator outputs C + 1 classes [16], where C represents
the number of classes for real sample, and the extra class
indicates whether the input is real sample or fake sample from
the generator.
In this article, there are two types of the input data for the
discriminator. One is the labeled DR data, and the other is
the unlabeled DR data. For the labeled data, there are four
categories (R0, R1, R2, and R3). For the unlabeled data, each
sample can be labeled as “Real” or “Fake.” In order to make
full use of the label information, different loss functions are
Authorized licensed use limited to: VIT University. Downloaded on December 02,2021 at 10:52:28 UTC from IEEE Xplore. Restrictions apply.
defined for different label types. The labeled samples only
need to be predicted as a certain class since they are all real.
The corresponding loss function is defined as
L Dr−labeled = −E x,y∼ pdata (x,y) log pd (y|x, y < C + 1)

= −o y + log[ exp o j ] (2)
j =1
where o represents the probability of the corresponding cate-
gory and y represents the ground truth. The unlabeled samples
can only be classified as real sample or generated sample
without one certain class label. The (C + 1)th class is
employed to indicate whether a sample is generated by the
generator. For the generated DR samples, the loss function is
defined as
L Dgenerated = E x∼Gen(z) log[ pd (y = C + 1|x)]
⎡ ⎤

= log⎣1 + exp o j ⎦. (3)
j =1
In the same manner, the corresponding loss function for the
real unlabeled DR sample is defined as
L Dr−unlabeled
= −E x,y∼ pdata (x,y) log[1 − pd (y = C + 1|x)]
⎡ ⎤ ⎡ ⎤
 
= − log⎣ exp o j ⎦ + log⎣1 + exp o j ⎦. (4)
j =1 j =1
The total loss function for the discriminator is the sum of the
abovementioned three loss functions
(Loss)(Dis) = L Dr−labeled + L Dgenerated + L Dr−unlabeled . (5)
2) Multichannel-Based Generative Model: For the general
GAN, it is difficult to generate a high-resolution fundus image
directly since the number of parameters for the generator will
be increased dramatically. In this article, a multichannel-based
generative model is proposed to learn the distribution of DR
features. The multichannel-based generative model contains
multiple parallel generators to generate a series of subfundus
images, including effective DR features directly. It can be
assumed that one entire DR sample includes M features and
the generative model includes n generators. Each generator
accounts for N = M/n features. Let Geni denotes the i th
generator in set Gen= {Gen1 , Gen2 , . . . , Genn }. The objec-
tive function for the proposed multichannel-based SSGAN is
defined as
min max V (Dis, {Gen1 , Gen2 , . . . , Genn })
{Gen1 ,Gen2 ,...,Genn } Dis
= E x∼Pdata(x) [log(Dis(x))]
+ E z∼ pz (z) [log(1 − Dis(Gen(z)))]
+ ||E φ∼Pr { f (φ1 ), f (φ2 ), . . . , f (φn )}
− E z∼Pz (z) { f (Gen1 ), f (Gen2 ), . . . , f (Genn )}||22 (6)
where f (φi ) represents the real DR features corresponding to
the i th generator. For the multichannel generative model, each
generator contains several deconvolution layers. By choosing
a noise variable pz (z) following a Gaussian distribution as
WANG et al.: DR DIAGNOSIS USING MGAN WITH SEMISUPERVISION 579

the input for Geni , the output of each layer of Geni can be
expressed as
⎛ ⎞
⎜  ⎟
bi sx+ p,sy+q,l = gli ⎜
⎝
i
ax,y,d · wip,q,d,l + cli ⎟
⎠ (7)
p∈{0,1...,k−1}
q∈{0,1...,k−1}
d∈{1...,D}

where a represents the input of layer 1, w and c represent

the weight and bias of the deconvolution layer, respectively,
i indicates the i th generator, s represents the stride of the
deconvolution operation, x and y denote the size of the input
features, respectively, D and k indicate the number of input Fig. 3. Architecture of the ensemble-based discriminative model.
features and the size of deconvolution kernels, respectively,
and gli (·) represents the activation function in layer l. The
ReLU is chosen as the activation function. loss is introduced into each generator. The corresponding loss
Each generator maps the input pz (z) to the DR feature space function is defined as
Geni (z; Wi ). Wi represents the parameter in Geni . Each gener-
ator generates a subfundus image, including a part of effective LG W = E z∼ pz (z) [log(1 − Dis(Gen(z)))] (11)
DR features. The output of the multichannel generative model
where Gen(z) indicates the combined features generated by
is given by
the multichannel generative model. The loss function of each
Gen(z) = Gen1 (z; W1 )⊕Gen2 (z; W2 )⊕. . .⊕Genn (z; Wn ) (8) generator is defined as

where ⊕ denotes the connection operation of feature maps. (Loss)(Geni ) = E z∼ pz (z) [log(1 − Dis(Gen(z)))]
To simulate the basic DR features, the multichannel generative + ||E x∼Pr f (φi ) − E z∼Pz (z) f (Geni (z))||22 . (12)
model generates a matrix with the same size as the output
For optimization purpose, the loss of each generator is
of feature extractor. As elaborated earlier, the multichannel
minimized as
generative model aims to learn the distribution of real DR
features. This is achieved by optimizing the loss function of G opt = arg min(Loss)(Geni ). (13)
G
the multichannel generative model. Each generator simulates
a subfundus image, including the local DR features and also 3) Ensemble Discriminators: To improve the robustness
measures the distribution deviation between the final generated and accuracy of the proposed model, we devise an ensem-
features and the real DR features. Thus, the loss function of ble scheme into the model. The ensemble discriminative
each generator consists of two parts model includes multiple discriminators with different struc-
tures, as shown in Fig. 3. Each discriminator is optimized
(Loss)(Geni ) = LG W + LG iM (9) by training independent multichannel generative networks.
For the input DR features, each discriminator outputs an
where LG W represents the distance between the distribution
independent classification result. The weight parameter is
of generated DR features and the distribution of real DR
assigned to each discriminator by using weighted-based fusion
features. LG iM represents the multichannel features matching
scheme [50], [51]. The classification probability of the kth
loss between the generated features of each subfundus image
input DR features given by discriminator i can be expressed
and the real local features.
as
The distribution of the generated DR features is expected  
to approximate the real distribution of DR features. Therefore, exp(o 1 ) exp(o 2 ) exp(o C )
Pki = C i
, C i
,. . . , C i
.
the multichannel feature matching mechanism is introduced
c=1 exp(oci ) c=1 exp(oci ) c=1 exp(oci )
into the proposed model, while one discriminator is employed (14)
to specify the distribution approximating which is the target of
the multichannel generators. More specifically, each generator Pki can be normalized by
is trained to approximate the expected state accounting for  i i 
the real DR features from the intermediate layers of the α1 , α2 , . . . , αCi
discriminator. Therefore, LG iM can be given by Pki
=   . (15)
exp(o1i ) exp(o2i ) exp(oCi )
LG iM = ||E x∼Pr f (φi ) − E z∼Pz (z) f (Gen i
(z))||22 (10) max C , C exp(o , . . . , C exp(o
c=1 exp(oci ) c=1 ci ) c=1 ci )

where f (·) denotes the activation on an intermediate layer Supposing that the proposed model employs L discrimina-
of the discriminator, E x∼Pr f (·) represents the expectation of tors, the final class label is determined by
real DR features extracted by the intermediate layers of the  L 
 
L 
L
discriminator, and Geni (z) represents the generated features of ypredict = arg max α1i , α2i , . . . , αCi . (16)
a subfundus image generated by generator i. The discriminant i=1 i=1 i=1

Authorized licensed use limited to: VIT University. Downloaded on December 02,2021 at 10:52:28 UTC from IEEE Xplore. Restrictions apply.
580 IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 18, NO. 2, APRIL 2021
Fig. 4. Effect of using different numbers of labeled training data. (a) DR. (b) Normal/abnormal. (c) Referable/nonreferable.
IV. E XPERIMENTS AND R ESULTS
A. Experiment Configuration
All the experiments are implemented using NVIDIA Tesla
P100 GPU. The weights of the generators and discriminators
are initialized with Xavier uniform distribution, and the initial
value of bias was set as 0. The model was trained by using
minibatch with a size of 16.
B. Data Preprocessing and Augmentation
The useless black boundary region is removed by a prede-
fined threshold, and all the fundus images are normalized to
the size of 1440 × 1440. The training set is augmented with
several geometric transformations, including random rotation
(range from −90◦ to 90◦ ) and horizontal and vertical flips.
The developed model is pretrained using the ImageNet [52]
data set.
C. Evaluation Criteria
The proposed model is evaluated individually and compar-
atively using the following metrics: sensitivity (SN), speci-
ficity (SP), area under ROC curve (AUC), and accuracy. These
metrics are reviewed in the following.
In medical statistic tasks, diagnostic indicators usually have
two basic characteristics: sensitivity and specificity. Sensitivity
refers to the probability that it will not miss a diagnosis, and
specificity refers to the probability of not being misdiagnosed.
Sensitivity and specificity are calculated as follows:
TP TN
Sensitivity = , Specificity =
TP + FN TN + FP
where the true positives (TPs) value is the number of samples
that are correctly classified as the positive class, the false neg-
atives (FNs) value is the number of samples belonging to the
positive class, but they are erroneously predicted as negative
class, the true negatives (TNs) value is the number of samples
that are correctly classified as the negative class, and the false
positives (FPs) value is the number of instances belonging to
negative class, but they are erroneously predicted as positive
class. In (17), sensitivity indicates how many positive samples
can be detected from all TP samples, and specificity indicates
how many negative samples can be detected from all TN
Authorized licensed use limited to: VIT University. Downloaded on December 02,2021 at 10:52:28 UTC from IEEE Xplore. Restrictions apply.
samples. For the DR grading, it is hoped to reduce the rate of
missed diagnosis by increasing sensitivity.
The AUC is a measure frequently used to evaluate the
performance of classifiers. It is an indicator of the probability
that a classifier can correctly classify samples. Note that an
AUC value of 0.5 indicates a random classifier (guessing). The
main advantage of AUC is its ability to evaluate the grading
performance for unbalanced data sets.
D. Parameter Analysis
The experiment was evaluated using a tenfold cross vali-
dation to weaken the likelihood of overfitting. For each fold,
the proportion of training, validation, and testing data is 70%,
10%, and 20%, respectively. Meanwhile, the training set has
been augmented by ten times.
In the following, we present the perspectives on evaluating
the proposed multichannel-based SSGAN. The main objectives
of the evaluation are as follows.
1) To investigate the effect of using different numbers of
labeled training samples in 1).
2) To investigate how the feature extractor influences the
performance of the proposed model in 2).
3) To investigate the effect of using different numbers of
generators in 3).
4) To compare the effectiveness of four optimization algo-
rithms (gradient descent, momentum, Adagrad, and
Adadelta) in 4).
5) To compare the effectiveness of five different cost func-
tions employed in the proposed model in 5).
(17) Each of these objectives is considered in a separate section
in the following.
1) Number of Labeled Training Data: For DR diagnosis,
there are a very few high-quality samples with labels, and
most of the DR samples are unlabeled. We first test how
the number of labeled samples influences the performance of
the proposed model. The metrics, including AUC, SN, SP,
and accuracy, are employed. The experimental results are
shown in Fig. 4. From Fig. 4(a), it can be observed that the
classification accuracy raises greatly (from 70.17% to 78%)
as the number of labeled data increases from 5 to 20.
The classification accuracy increases mildly (from 79.37% to
80.98%) as the number of labeled data increases continually
WANG et al.: DR DIAGNOSIS USING MGAN WITH SEMISUPERVISION
Fig. 5. Effect of using feature extractor.
from 50 to 200. The abovementioned experimental results
show that the proposed semisupervised model can work well
using a relatively small number of labeled samples. However,
a certain amount of labeled samples is essential if one wants to
obtain a higher classification accuracy by using the proposed
semisupervised model. The similar trend and conclusion can
also be observed from Fig. 4(b) and (c). It demonstrates
that the proposed multichannel-based SSGAN can generate
samples following the similar distribution of real DR samples,
and the performance of discriminators can be significantly
improved. In reality, especially in the field of DR diagno-
sis, labeled data is time-consuming for clinicians. Therefore,
the proposed multichannel-based SSGAN has great potential
for DR diagnosis.
2) Feature Extractor: In order to verify the influence of
the feature extractor, a comparison experiment was carried
out to test the model performance with/without the feature
extractor. The experimental results are shown in Fig. 5. From
Fig. 5, it can be seen that the proposed multichannel-based
SSGAN with feature extractor can achieve a higher classifi-
cation accuracy. The reasons mainly include the followings:
1) some redundant features are removed by feature extractor
so that the discriminator can directly process the effective
features and 2) the DR features extracted by pretrained feature
extractor are highly representative so that the discriminator
can learn the distribution of DR features more accurately and
the classification accuracy of the proposed model is improved.
Fig. 6 shows the feature maps and the corresponding scattered
features of DR. For sample 1, Fig. 6(a) and (b) indicates the
feature maps obtained from the feature extractor and Fig. 6(c)
indicates the scattered DR features. For sample 2, Fig. 6(d)
and (e) indicates the feature maps obtained from the feature
extractor and Fig. 6(f) indicates the scattered DR features.
3) Number of Generators: The proposed model is based on
the multichannel GAN. The number of channels/generators
plays an important role in the model performance. Several
groups of experiments are carried out to test its influence. The
experimental results are shown in Fig. 7. Fig. 7(a) shows that
the grading accuracy raised greatly as the number of generators
increases from 1 to 4, but the accuracy is decreased gradually
Authorized licensed use limited to: VIT University. Downloaded on December 02,2021 at 10:52:28 UTC from IEEE Xplore. Restrictions apply.
581
Fig. 6. Displays of DR features. (a) and (b) Feature maps from the feature
extractor and (c) scattered DR features for sample 1. (d) and (e) Feature maps
from the feature extractor and (f) scattered DR features for sample 2.
when the number of generators keeps increasing. Therefore,
multichannel-based generative model can improve the model
performance. However, the generator is not the more the
better. The similar trend and conclusion can be observed from
Fig. 7(b) and (c). Based on Fig. 7, the following conclusions
can be obtained: 1) the proposed multichannel-based gener-
ative model can benefit from a certain number of generators
since they can help generate a series of subfundus images
including effective local features and 2) excessive generators
can weaken the performance of the multichannel-based model
since they introduce excessive parameters, which results in
overfitting. Moreover, excessive generators may lead to miss-
ing the correlation between local features and global features.
4) Optimization Algorithm: To test the influence of the opti-
mization algorithm, several groups of experiments are carried
out using gradient descent, Adam, momentum, Adagrad, and
Adadelta, respectively. The experimental results are shown
in Fig. 8(a). Fig. 8(a) shows that, compared with the other four
optimization algorithms (gradient descent, momentum, Ada-
grad, and Adadelt), Adam algorithm achieves the best accuracy
with respect to classifications of DR, normal/abnormal, and
referable/nonreferable, respectively. The reason is that Adam
can update adaptive learning rates based on the training data.
The learning rate is stable within a certain range, which makes
the parameters relatively stable. Discriminator with relatively
stable parameters is important for training GAN [53].
5) Cost Function: In this article, the multichannel feature
matching mechanism is introduced into the proposed model.
Several comparative experiments using this mechanism are
conducted to demonstrate its advantage. The employed mech-
anisms include minibatch discrimination (MBD), historical
averaging (HisA), one-sided label smoothing (OSLS), and
virtual BN (VBN) [54]. The experimental results are shown
in Fig. 8(b). Fig. 8(b) shows that the proposed model with
multichannel feature matching mechanism achieves the highest
accuracy among all the methods. The multichannel feature
matching mechanism can minimize the difference between the
generated DR features and the real DR features. Fig. 9 shows
582 IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 18, NO. 2, APRIL 2021

Fig. 7. Effect of using different numbers of generators. (a) DR. (b) Normal/abnormal. (c) Referable/nonreferable.

Fig. 8. (a) Effect of optimization methods. (b) Effect of five different loss functions.

Fig. 10. Confusion matrix of DR classification results.

Fig. 9. Trend of loss function for the training process.
TABLE III
C OMPARISON W ITH O THER M ETHODS FOR DR G RADING
the discriminator loss for DR classification in the training
process.

V. D ISCUSSION
This section provides a comparative evaluation of the pro-
posed multichannel-based SSGAN with other representative
methods in terms of accuracy, AUC, sensitivity, and specificity.
Several groups of experiments were conducted using differ- human experts, SSGAN, and other representative methods.
ent classification methods. The comparable methods include The experimental results are presented in Tables III–V.

Authorized licensed use limited to: VIT University. Downloaded on December 02,2021 at 10:52:28 UTC from IEEE Xplore. Restrictions apply.
WANG et al.: DR DIAGNOSIS USING MGAN WITH SEMISUPERVISION 583

TABLE IV
C OMPARISON W ITH O THER M ETHODS ON N ORMAL /A BNORMAL

TABLE V
C OMPARISON W ITH O THER M ETHODS ON R EFERABLE /N ONREFERABLE

Fig. 11. (a)–(d) and (f)–(i) Generated samples versus (e) and (j) real samples.

From Table III, it can be observed that, for the TABLE VI

four-category grading of DR, the proposed framework using
multichannel-based SSGAN achieves the highest accuracy
of 84.23% among all the methods. For the binary classifi-
cation of DR, Table IV shows that, compared with the other
methods, the proposed method achieves the highest accuracy
of 96.6%, the highest sensitivity of 97.0%, the highest speci-
ficity of 96.8%, and the highest AUC of 98.3%. Table V
also shows that the proposed method outperforms the other
representative methods [16], [17], [28] in terms of accuracy,
AUC, sensitivity, and specificity, respectively.
To demonstrate the advantage of the proposed method,
we only employ 100 labeled DR samples and a large number
of unlabeled DR samples to train the model. To show the
classification performance in the four categories (R0, R1, R2,
and R3), we employ the confusion matrix to analyze the exper-
imental results that are shown in Fig. 10. From Fig. 10, it can
be observed that although we only employ 100 labeled DR
samples to train the model, the proposed model performs well.
R ESULTS OF DR C LASSIFICATION ON D IFFERENT D ATA S ETS
It demonstrates that the proposed method can deal with a
classification problem when the labeled samples are limited.
The proposed model is tested on different data sets to
demonstrate its generalization capability. The experimental
results are shown in Table VI. The comparison between
the generated images and real images is given in Fig. 11,
where Fig. 11(a)–(d) is the generated images corresponding
to the real image Fig. 11(e) at different stages. Similarly,
Fig. 11(f)–(i) is generated images corresponding to the real
image Fig. 11(j) at different stages. Note that, in this article,

Authorized licensed use limited to: VIT University. Downloaded on December 02,2021 at 10:52:28 UTC from IEEE Xplore. Restrictions apply.
584 IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 18, NO. 2, APRIL 2021
TABLE VII
C ONFIGURATIONS OF THE P ROPOSED M ODEL
we aim to obtain a good discriminator. It is well known that
good semisupervised classification performance and a good
generator cannot be obtained at the same time [56]. Given the
discriminator objective, good semisupervised learning often
requires a bad generator. Therefore, in this article, it is not
important whether the generated images are uniform with the
real images.
VI. C ONCLUSION
The inadequacy of labeled data is a challenge for using
deep-learning technology in medical image analysis. The
reasons mainly include the followings: 1) the high-quality
annotation of medical imaging sample depends heavily on
scarce medical expertise which is very expensive; 2) compared
with natural issues, it is more difficult to collect medical
images because of privacy issues, and 3) there are many kinds
of diseases, and so many different medical data sets need to be
collected. Actually, collecting sufficient DR samples with label
is difficult. In this article, the multichannel GAN with semi-
supervision is developed to assess DR. The proposed model
can deal with a DR classification problem with inadequacy
of labeled data in the following ways. First, the multichannel
generative scheme is proposed to generate a series of sub-
fundus images corresponding to the scattering DR features.
Second, the proposed multichannel-based GAN model with
semisupervision can make full use of both labeled data and
unlabeled data. Third, the DR feature extractor is introduced
into the proposed model to weaken noise and extract represen-
tative DR features, some of which are tiny and spread out over
high-resolution fundus images. Experiments are conducted
by using the Messidor data set. The experimental results
demonstrate that the developed model outperforms the other
representative models [16], [17], [28] in terms of accuracy,
AUC, sensitivity, and specificity. In particular, Fig. 10 shows
the promising performance of the proposed model even if only
100 labeled samples are employed. However, this article still
has some limitations. For example, biological methods and
pathological analysis are not considered in this article. These
issues will be addressed in the future work.
A PPENDIX :
D ETAILS OF N ETWORK S TRUCTURE
The configurations of the proposed model are given
in Table VII. In this article, each discriminator is cotrained
Authorized licensed use limited to: VIT University. Downloaded on December 02,2021 at 10:52:28 UTC from IEEE Xplore. Restrictions apply.
with four generators. Five discriminators are trained for the
ensemble model. In Table VII, “256, 3 × 3conv, stride 2”
means 256 convolution kernels of size 3 × 3 and the stride
is 2.
R EFERENCES
[1] O. Faust, R. Acharya U., E. Y. K. Ng, K.-H. Ng, and J. S. Suri, “Algo-
rithms for the automated detection of diabetic retinopathy using digital
fundus images: A review,” J. Med. Syst., vol. 36, no. 1, pp. 145–157,
Feb. 2012.
[2] S. Haneda and H. Yamashita, “International clinical diabetic retinopathy
disease severity scale,” Nihon Rinsho. Jpn. J. Clin. Med., vol. 68, p. 228,
Nov. 2010.
[3] S. Wang, Y. Hu, Y. Shen, and H. Li, “Classification of diffusion tensor
metrics for the diagnosis of a myelopathic cord using machine learning,”
Int. J. Neural Syst., vol. 28, no. 2, Mar. 2018, Art. no. 1750036.
[4] S. Wang et al., “Skeletal maturity recognition using a fully automated
system with convolutional neural networks,” IEEE Access, vol. 6,
pp. 29979–29993, 2018.
[5] M. Zhang et al., “Adaptive patient-cooperative control of a compliant
ankle rehabilitation robot (CARR) with enhanced training safety,” IEEE
Trans. Ind. Electron., vol. 65, no. 2, pp. 1398–1407, Feb. 2018.
[6] H. Greenspan, B. van Ginneken, and R. M. Summers, “Guest editorial
deep learning in medical imaging: Overview and future promise of
an exciting new technique,” IEEE Trans. Med. Imag., vol. 35, no. 5,
pp. 1153–1159, May 2016.
[7] B. Zhong, W. Niu, E. Broadbent, A. McDaid, T. M. C. Lee, and
M. Zhang, “Bringing psychological strategies to robot-assisted physio-
therapy for enhanced treatment efficacy,” Frontiers Neurosci., vol. 13,
p. 984, Sep. 2019.
[8] A. K. AlZubaidi, F. B. Sideseq, A. Faeq, and M. Basil, “Computer
aided diagnosis in digital pathology application: Review and perspective
approach in lung cancer classification,” in Proc. Annu. Conf. New Trends
Inf. Commun. Technol. Appl. (NTICT), Mar. 2017, pp. 219–224.
[9] M. Zhang, A. McDaid, A. J. Veale, Y. Peng, and S. Q. Xie, “Adaptive
trajectory tracking control of a parallel ankle rehabilitation robot with
joint-space force distribution,” IEEE Access, vol. 7, pp. 85812–85820,
2019.
[10] A. Sopharak et al., “Machine learning approach to automatic exudate
detection in retinal images from diabetic patients,” J. Modern Opt.,
vol. 57, no. 2, pp. 124–135, Jan. 2010.
[11] R. Priya and P. Aruna, “Diagnosis of diabetic retinopathy using
machine learning techniques,” ICTACT J. Soft Comput., vol. 3, no. 4,
pp. 563–575, Jul. 2013.
[12] J. Krause et al., “Grader variability and the importance of reference stan-
dards for evaluating machine learning models for diabetic retinopathy,”
Ophthalmology, vol. 125, no. 8, pp. 1264–1272, Aug. 2018.
[13] M. Zhang, W. Meng, C. Davies, Y. Zhang, and S. Xie, “A robot-
driven computational model for estimating passive ankle torque with
subject-specific adaptation,” IEEE Trans. Biomed. Eng., vol. 63, no. 4,
pp. 814–821, Aug. 2016.
[14] G. Litjens et al., “A survey on deep learning in medical image analysis,”
Med. Image Anal., vol. 42, pp. 60–88, Dec. 2017.
[15] H. Huang et al., “Introvae: Introspective variational autoencoders for
photographic image synthesis,” in Proc. Adv. Neural Inf. Process. Syst.,
2018, pp. 52–63.
WANG et al.: DR DIAGNOSIS USING MGAN WITH SEMISUPERVISION
[16] A. Odena, “Semi-supervised learning with generative adversarial
networks,” 2016, arXiv:1606.01583. [Online]. Available: http://arxiv.
org/abs/1606.01583
[17] Z. Wang, Y. Yin, J. Shi, W. Fang, H. Li, and X. Wang, “Zoom-in-
net: Deep mining lesions for diabetic retinopathy detection,” in Proc.
Int. Conf. Med. Image Comput. Comput.-Assist. Intervent. Quebec City,
QC, Canada: Springer, 2017, pp. 267–275.
[18] C. I. Sánchez, M. Niemeijer, A. V. Dumitrescu,
M. S. A. Suttorp-Schulten, M. D. Abràmoff, and B. van Ginneken,
“Evaluation of a computer-aided diagnosis system for diabetic
retinopathy screening on public data,” Investigative Opthalmol. Vis.
Sci., vol. 52, no. 7, pp. 4866–4871, Jun. 2011.
[19] L. Seoud, J. Chelbi, and F. Cheriet, “Automatic grading of diabetic
retinopathy on a public database,” in Proc. Ophthalmic Med. Image
Anal. 2nd Int. Workshop (OMIA), Munich, Germany, 2015, pp. 97–104.
[20] H. Pratt, F. Coenen, D. M. Broadbent, S. P. Harding, and Y. Zheng,
“Convolutional neural networks for diabetic retinopathy,” Procedia Com-
put. Sci., vol. 90, pp. 200–205, Jan. 2016.
[21] V. Gulshan et al., “Development and validation of a deep learning
algorithm for detection of diabetic retinopathy in retinal fundus pho-
tographs,” JAMA, vol. 316, no. 22, pp. 2402–2410, Dec. 2016.
[22] M. Haloi, “Improved microaneurysm detection using deep neural net-
works,” 2015, arXiv:1505.04424. [Online]. Available: http://arxiv.org/
abs/1505.04424
[23] P. Costa, A. Galdran, A. Smailagic, and A. Campilho, “A weakly-
supervised framework for interpretable diabetic retinopathy detection
on retinal images,” IEEE Access, vol. 6, pp. 18747–18758, 2018.
[24] J. Shan and L. Li, “A deep learning method for microaneurysm detection
in fundus images,” in Proc. IEEE 1st Int. Conf. Connected Health, Appl.,
Syst. Eng. Technol. (CHASE), Jun. 2016, pp. 357–358.
[25] R. Gargeya and T. Leng, “Automated identification of diabetic retinopa-
thy using deep learning,” Ophthalmology, vol. 124, no. 7, pp. 962–969,
Jul. 2017.
[26] B. Antal and A. Hajdu, “An ensemble-based system for auto-
matic screening of diabetic retinopathy,” Knowl.-Based Syst., vol. 60,
pp. 20–27, Apr. 2014.
[27] P. Costa and A. Campilho, “Convolutional bag of words for diabetic
retinopathy detection from eye fundus images,” IPSJ Trans. Comput.
Vis. Appl., vol. 9, no. 1, p. 10, 2017.
[28] H. H. Vo and A. Verma, “New deep neural nets for fine-grained diabetic
retinopathy recognition on hybrid color space,” in Proc. IEEE Int. Symp.
Multimedia (ISM), Dec. 2016, pp. 209–215.
[29] S. Jialin Pan and Q. Yang, “A survey on transfer learning,” IEEE Trans.
Knowl. Data Eng., vol. 22, no. 10, pp. 1345–1359, Oct. 2010.
[30] L. Dai et al., “Clinical report guided retinal microaneurysm detection
with multi-sieving deep learning,” IEEE Trans. Med. Imag., vol. 37,
no. 5, pp. 1149–1161, May 2018.
[31] W. Cao, N. Czarnek, J. Shan, and L. Li, “Microaneurysm detection
using principal component analysis and machine learning methods,”
IEEE Trans. Nanobiosci., vol. 17, no. 3, pp. 191–198, Jul. 2018.
[32] W. M. Gondal, J. M. Köhler, R. Grzeszick, G. A. Fink, and M. Hirsch,
“Weakly-supervised localization of diabetic retinopathy lesions in retinal
fundus images,” in Proc. IEEE Int. Conf. Image Process. (ICIP),
Sep. 2017, pp. 2069–2073.
[33] G. Quellec, K. Charrière, Y. Boudi, B. Cochener, and M. Lamard, “Deep
image mining for diabetic retinopathy screening,” Med. Image Anal.,
vol. 39, pp. 178–193, Jul. 2017.
[34] L. Zhou, Y. Zhao, J. Yang, Q. Yu, and X. Xu, “Deep multiple
instance learning for automatic detection of diabetic retinopathy in
retinal images,” IET Image Process., vol. 12, no. 4, pp. 563–571,
Apr. 2018.
[35] X. Zeng, H. Chen, Y. Luo, and W. Ye, “Automated diabetic retinopathy
detection based on binocular Siamese-like convolutional neural net-
work,” IEEE Access, vol. 7, pp. 30744–30753, 2019.
Authorized licensed use limited to: VIT University. Downloaded on December 02,2021 at 10:52:28 UTC from IEEE Xplore. Restrictions apply.
585
[36] X. Li, T. Pang, B. Xiong, W. Liu, P. Liang, and T. Wang, “Convolutional
neural networks based transfer learning for diabetic retinopathy fundus
image classification,” in Proc. 10th Int. Congr. Image Signal Process.,
Biomed. Eng. Informat. (CISP-BMEI), Oct. 2017, pp. 1–11.
[37] A. Khandelwal and A. K. Mishra, “Design simulation and analysis of
enhanced diabetic retinopathy using convolutional neural network,” Int.
J. Res. Anal. Rev., vol. 6, no. 3, pp. 660–663, 2019.
[38] M. Raju, V. Pagidimarri, R. Barreto, A. Kadam, V. Kasivajjala, and
A. Aswath, “Development of a deep learning algorithm for automatic
diagnosis of diabetic retinopathy,” in Proc. MedInfo, 2017, pp. 559–563.
[39] K. Xu, D. Feng, and H. Mi, “Deep convolutional neural network-based
early automated detection of diabetic retinopathy using fundus image,”
Molecules, vol. 22, no. 12, p. 2054, 2017.
[40] R. Ghosh, K. Ghosh, and S. Maitra, “Automatic detection and classifi-
cation of diabetic retinopathy stages using CNN,” in Proc. 4th Int. Conf.
Signal Process. Integr. Netw. (SPIN), Feb. 2017, pp. 550–554.
[41] S. Qummar et al., “A deep learning ensemble approach for diabetic
retinopathy detection,” IEEE Access, vol. 7, pp. 150530–150539, 2019.
[42] J. M. Brown et al., “Automated diagnosis of plus disease in retinopa-
thy of prematurity using deep convolutional neural networks,” JAMA
Ophthalmol., vol. 136, no. 7, pp. 803–810, Jul. 2018.
[43] X. Li, L. Shen, M. Shen, F. Tan, and C. S. Qiu, “Deep learning
based early stage diabetic retinopathy detection using optical coherence
tomography,” Neurocomputing, vol. 369, pp. 134–144, Dec. 2019.
[44] R. Poplin et al., “Prediction of cardiovascular risk factors from retinal
fundus photographs via deep learning,” Nature Biomed. Eng., vol. 2,
no. 3, pp. 158–164, Mar. 2018.
[45] J. De Fauw et al., “Clinically applicable deep learning for diagnosis and
referral in retinal disease,” Nature Med., vol. 24, no. 9, pp. 1342–1350,
Sep. 2018.
[46] J. de la Torre, A. Valls, and D. Puig, “A deep learning interpretable
classifier for diabetic retinopathy disease grading,” Neurocomputing,
Apr. 2019, doi: 10.1016/j.neucom.2018.07.102.
[47] E. Decencière et al., “Feedback on a publicly distributed image data-
base: The Messidor database,” Image Anal. Stereology, vol. 33, no. 3,
pp. 231–234, 2014.
[48] G. Huang, Z. Liu, L. V. D. Maaten, and K. Q. Weinberger, “Densely
connected convolutional networks,” in Proc. CVPR, Jul. 2017, vol. 1,
no. 2, p. 3.
[49] X. Glorot, A. Bordes, and Y. Bengio, “Deep sparse rectifier neural net-
works,” in Proc. 14th Int. Conf. Artif. Intell. Statist., 2011, pp. 315–323.
[50] X. Frazao and L. A. Alexandre, “Weighted convolutional neural network
ensemble,” in Iberoamerican Congress on Pattern Recognition. Puerto
Vallarta, Mexico: Springer, 2014, pp. 674–681.
[51] G. Wen, Z. Hou, H. Li, D. Li, L. Jiang, and E. Xun, “Ensemble of
deep neural networks with probability-based fusion for facial expression
recognition,” Cognit. Comput., vol. 9, no. 5, pp. 597–610, Oct. 2017.
[52] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet:
A large-scale hierarchical image database,” in Proc. IEEE Conf. Comput.
Vis. Pattern Recognit., Jun. 2009, pp. 248–255.
[53] M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein GAN,” 2017,
arXiv:1701.07875. [Online]. Available: http://arxiv.org/abs/1701.07875
[54] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and
X. Chen, “Improved techniques for training GANs,” in Proc. Adv. Neural
Inf. Process. Syst., 2016, pp. 2234–2242.
[55] D. Y. Carson Lam, M. Guo, and T. Lindsey, “Automated detection
of diabetic retinopathy using deep learning,” in Proc. AMIA Summits
Transl. Sci., 2018, p. 147.
[56] Z. Dai, Z. Yang, F. Yang, W. W. Cohen, and R. Salakhutdinov,
“Good semi-supervised learning that requires a bad GAN,” 2017,
arXiv:1705.09783. [Online]. Available: http://arxiv.org/abs/1705.09783
[57] M. D. Abràmoff et al., “Automated analysis of retinal images for
detection of referable diabetic retinopathy,” JAMA Ophthalmol., vol. 131,
no. 3, pp. 351–357, Mar. 2013.