Supervised Binary Convolutional Neural Networks with Model Stacking for Diagnosis of

Sirihaasa Nallamothu
University High School Fundus & Eyelid Diseases
Mentors: Sateesh Nallamothu and Cory Culbertson

Overview
Overview Model Topology Results
A total of 6 binary convolutional neural networks for diagnosis of retinal ailments through fundus images along with 1 binary convolutional neural network for diagnosis of
Retinal and ocular health is often overlooked and seen as a privilege not a necessity. Although there are bacterium trachoma were created and trained. The model architecture between these neural networks was similar, however number of dense layers, max pooling/conv2d After diligent construction of model architecture for each of the binary convolutional
a variety of machine learning based and ophthalmologist to diagnose these ailments they require instances and convolutions were varied in the model construction to yield favorable results. The network architecture begins with a batch normalization computation to neural networks, and comparing both the model training accuracy and individual model
expensive equipment and resources, and often times overlook specific retinal diseases. This standardize the input. Next, there is a Conv2D layer (3x3 window) which applies feature maps, the activation function and the multiplicative matrix along with a max pooling
layer (2x2 window) which finds the largest filter value result to preserve and identify prevalent aspects of the image. The combination of the Conv2D and MaxPooling layers, classifications to a multiclassification neural network, I have determined that model
comprehensive research posters details the creation of six binary convolutional neural networks to create convolutions which help to filter and identify important features and create feature maps which will ultimately help with classification of the input image. The ReLU stacking is more accurate and reliable. I have also conclusively determined that model
diagnose Diabetic Retinopathy, Age Related Macular Degeneration, Myopia, Glaucoma, Ocular activation function is used consistently throughout the convolutions. After the convolutions a neuron ‘dropout’ computation which randomly ignores 20% of the model’s
stacking produces confidence percentages with a greater magnitude than
Hypertension, and Cataracts with fundus images, along with Trachoma through inner under eye lid neurons. Lastly, the data is flattened and passed through the dense layers. A Sigmoid algorithm was applied on the last dense layer to procure an optimized binary output.
images. In order to produce a conclusive diagnosis model stacking was implemented, and the diagnosis [Model summaries are featured in code repositories] multiclassification networks, as the model is looking to identify whether or not the fundus
results were compared to a trial multiclassification neural network. Analysis was also completed on the image has that ailment, not completely classify it. Because of this, the threshold value I
visualizations of the convolutions to identify which features of the image the model was detecting, and produced [0.70] is significantly high, as the neural networks consistently produce
to determine if these identification methods were different compared to traditional classification. confidence percentages that are extremely close to 1 or 0. Each model was also trained to
The neural networks produced optimal results for both training and testing data accuracies along with *Scaled by 0.5 for over 90% training data accuracy and less than 20% loss.
model classification, and show promise for implementation and for further research to be completed. number feature maps and
neurons Although each of the models produced high training
and validation data accuracies, the size of the training
dataset seems to have a small amount of correlation on
the overall accuracies of the training dataset. When
passing in a separate testing data set into the binary
Background models via a function, the accuracies that were
produced also seemed to correlate with the size of the
training dataset, despite implemented class weights. As
There have been many attempts to approach diagnose of retinal ailments both through the size of the training dataset increases, testing data
fundus images and Optical Coherence Tomography.
Google’s Algorithm to diagnose Diabetic Retinopathy through fundus images is highly
Model Fitting & Training accuracies seem to decrease, by a few percentage
points.

accurate, utilizes a large dataset, and has revolutionized the face of telemedicine and When training and fitting the binary convolutional neural networks, an emphasis was put on implementing the correct optimization algorithms and loss functions to
accessibility (Google, 2020). Although there are many specialized models to diagnose improve the accuracy of the model classifications.
more commonly known retinal diseases such as Diabetic Retinopathy or Glaucoma, there Loss Function: Binary Crossentropy was the loss function implemented in order to determine the overall error between the singular output of the model and the target

is a lack of algorithms created to diagnose other life threatening ailments such as Myopia
or Age Related Macular Degeneration. There are also a multitude of algorithms that use
optical coherence tomography images and 3D cross sections to diagnose specific retinal
diseases(Mehmet, 2019). It should be noted that OCT technology is incredibly expensive
and therefore inaccessible to the general public.
Most existing algorithms that do attempt to diagnose multiple retinal diseases use
multiclassification convolutional neural networks, that provide a multitude of confidence
Normal vs. Glaucoma
percentages, which may dilute the overall diagnosis, and provide an inaccurate
breakdown of the ailments in the fundus image.
Additionally, there are no established convolutional neural networks to diagnose
Trachoma, a bacterial infection that causes scarring in the under eye lid and is the leading
cause of blindness in underdeveloped countries.
Current methods to diagnose retinal diseases include handheld fundus imaging devices,
OCT technology. Each of these methods has their warrants, however many of them
require a optical specialist to interpret and diagnose these diseases or they require
Normal vs. Cataracts
expensive and inaccessible technology.
It is clear that there is a need for a comprehensive, accessible algorithm that can diagnose
a multitude of retinal ailments that can provide accurate results.
value.
Optimizer: ‘Adam’ was used as the primary optimizer for the neural networks. This algorithm was used to calculate the ‘gradient descent’, which helps the model adjust
the filters in the convolutions for the upcoming iterations. Along with this, a learning rate of 0.001 to control how quickly and ‘incrementally’ the model adapts to the
problem set.
Class Weights & Callbacks: Class weights were implemented to prevent bias in the training datasets, specifically to prevent preference to the ‘Normal’ training dataset,
which had substantially larger amounts of images. Callbacks were implemented to terminate model training when it reached a certain accuracy to prevent overfitting.
Model Fitting Params: Epoch numbers varied based on callback cutoffs, however 70 was the average number to train each model. Batch size was set to 32 images at a
time, and 15% of the inputted training dataset was set aside for model validation metrics.
Metrics: Area under the curve, which finds the model’s ability to distinguish between the two training classes, along with Accuracy, Loss and Validation Accuracy & Loss
were implemented in the model.
Normal vs. Trachoma
Normal vs. AMD
Conclusion & Discussion
The binary classification convolutional neural networks with model stacking were able to successfully
classify fundus images with specific retinal diseases. Additionally, the Trachoma model was successful
in identifying under eye lid images that had bacterium trachomatis. In the future these neural networks
can be implemented in a mobile hand held device, along with a 20D ophthalmic lens to provide fundus
images for the models to classify.
Perhaps a broader and more prevalent implication of these models is that they have an ability to identify
new features or aspects of a fundus image to focus on for diagnosis. In order to gather greater insight
into the features that the convolutions are narrowing on, visualization was completed using a ‘green-
blue’ feature map. Many of the fundus image feature detectors were narrowing in on the optic disc and
fovea region along with pigmentation, which is often overlooked for the arteries and veins.
Additionally the Trachoma convolutional neural network may be able to provide further insights into
what specific features of the inner eye lid to focus on when diagnosing this ailment. The convolutions
are narrowing in on scarring in the lower region of the inner eye lid, where scarring is often the heaviest.
An in-depth analysis of of these convolutions may lead to more conclusive results.
Diabetic Retinopathy Trachoma

Motivation & Approach Normal vs. Diabetic Retinopathy Normal vs. Myopia

Over 200 million people worldwide suffer from retinal and ocular diseases, with 170 million people
living with Age-Related Macular Degeneration [AMD] and 1.5 million living with Diabetic Retinopathy
among other horrific eye disorders and diseases. 44 countries face the prevalent health problem of
Trachoma which causes irreversible blindness. 80% of these diseases can be detected, prevented, or
cured. Ocular and Retinal care is often seen as a privilege, rather than a right or necessity. The goal is to
create accessible and accurate stacked binary convolutional neural networks to diagnose Diabetic
Retinopathy, Age Related Macular Degeneration, Myopia, Glaucoma, Ocular Hypertension, and
Cataracts with fundus images. The Fundus is the essentially the back surface of the eye that houses blood
Individual Model Accuracies: [converted to percentages]
vessels and arteries which can be seen via a 20D ophthalmic lens and steady light source. Users will also Glaucoma: 93% Hypertension: 94% Cataracts: 97% Trachoma: 97% Myopia: 93% Diabetic Retinopathy: 95%
be able to receive confidence percentages on diagnosis of bacterium Trachoma. This will be done by AMD: 95%
using a cohesive dataset, class weights, intricate model topography, and calculated insights into the
algorithm's calculations through model convolutions. Accuracies of model stacking and
multiclassification models will be compared for this specific use case. Full Convolutions are

Model Stacking vs.

featured on a separate
document in ‘Links’
section.

Multiclassification
Dataset In order to draw conclusions on whether multiclassification models or binary stacked models provided more

The dataset utilized was a combination of data from Shangong Medical Technology Co., Ltd, National Eye Institute, and Joint
Shantou International Eye Centre (JSIEC).
The dataset is compromised of 7,000 images in the training data set and 890 images in the testing dataset, for ailments with
fundus images exclusively. ‘Trachoma’ images was compiled with 1483 training images and 20 testing images. All images were
resized to 150 px to maintain consistency. For this specific use case, RGB and 3 color channels were implemented as fundus
pigmentation is prevalent in identifying the ailment.
Class weights were implemented when training and fitting the model to balance out classes and prevent bias towards datasets
that were substantially larger. Training and Testing datasets were compiled separately. Training dataset samples and breakdown
featured below.
Additional Resources
accurate classifications, I created a multiclassification model to compare training accuracies and model
diagnosis to the binary stacked method.
Although initial testing with the multiclassification model showed promising results, in this specific use case it
is clear that binary classification model stacking provides clearer, less ‘diluted’ results, with a stronger
polarization to one classification rather than multiple. Each of the confidences are provided on a scale of 0 to
1, the closer the number is to 1, the higher the probability that the fundus image has that specific ailment. If all Neural Network Model Demonstration: https://www.youtube.com/watch?v=TQHd95bI_2Q 
values are close to 0, the fundus is healthy. The threshold value I determined based off of model stacking Poster Presentation and Research Video: https://www.youtube.com/watch?v=BsjGtkZOfUA&t=2s 
results was 0.65. All raw value featured below can be converted to percentages by multiplying by 100.
Github Repository:
https://github.com/SirihaasaNallamothu28/Fovea-ConvolutionalNueralNetworksDiagnosisofFundus-RetinalDiseases  
Research Grant Video: https://www.youtube.com/watch?v=0U2_JKBoiG0&t=15 
Multiclassification Metrics Repositories & Additional Links
Dataset Sorting Script: https://github.com/SirihaasaNallamothu28/Fovea-DataSetSortingScript
Description of Retinal Ailments
https://tinyurl.com/fundusdiseasesdescription 
Model Convolutions: https://www.scribd.com/document/528733030/Model-Convolutions-and-Data-Nallamothu
Ailment: Diabetic Retinopathy Ailment: Glaucoma
Model Stacking Diagnosis Model Stacking Diagnosis

Sources
Diabetic Retinopathy Glaucoma Diabetic Retinopathy Glaucoma

Normal Glaucoma Myopia Diabetic Retinopathy AMD Myopia AMD Myopia AMD Accuracy: 68%, Loss: 97%, Validation Accuracy: 54% , Validation Loss: 100%
25% 22% 14% 17% 8% Testing Dataset Classifications Sertkaya, Mehmet & Ergen, Burhan & Toğaçar, Mesut. (2019). Diagnosis of Eye Retinal Diseases Based on Convolutional Neural Networks
Ocular Hypertension Cataracts Ocular Hypertension Ailment: Trachoma Metrics for Number of Classifications for Binary and Multiclassification models Using Optical Coherence Images. 1-5. 10.1109/ELECTRONICS.2019.8765579.
Cataracts
Model Diagnosis: got correct out of 3 testing data set images of each ailment, based on thresholds. Diagnosing diabetic retinopathy with machine learning. Google. (n.d.). Retrieved September 28, 2021, from
Multiclassification Distribution: Multiclassification Distribution:
https://about.google/intl/ALL_us/stories/seeingpotential/.
Normal Diabetic Retinopathy
Ocular disease recognition. Kaggle. (2020, September 24). Retrieved July 16, 2020, from https://www.kaggle.com/andrewmvd/ocular-disease-
Normal Diabetic Retinopathy
recognition-odir5k.
Lehrstuhl für Mustererkennung, F.-A.-U. E.-N. (n.d.). High-Resolution Fundus (HRF) Image Database. Retrieved September 28, 2021, from
Glaucoma Ocular Hypertension Ocular Hypertension https://www5.cs.fau.de/research/data/fundus-images/.
Glaucoma
Linchundan. (2019, June 18). 1000 fundus images with 39 Categories. Kaggle. Retrieved September 28, 2021, from
Hypertension Cataracts Trachoma Healthy AMD Myopia AMD Myopia https://www.kaggle.com/linchundan/fundusimage1000.
7% 42% 57% Cataracts
7% Cataracts