Automatic Recognition of Fish Diseases in Fish

Farms
Ahmed Waleed, Hadeer Medhat, Mariam Esmail,
Kareem Osama, Radwa Samy, Taraggy M Ghanim
Faculty of Computer Science,Misr International University
Cairo, Egypt
ahmed1608592, hadeer1605917, mariam1605986,
kareem1608590, radwa.mohamed, taraggy.ghanim{@miuegypt.edu.eg}

Abstract—Fish diseases are the major cause for increasing

mortality in fish farms. Automatic identification of diseased fish at
early stages is necessary step to prevent spreading disease. Fish
disease diagnosis suffers from some limitations that need high
level of expertise to be resolved. Recognition of fish abnormal be-
haviors helps in early prediction of fish diseases. Fish behavior is
evaluated by analyzing fish trajectories in videos. Abnormalities
may be due to environmental changes. This paper introduces
a survey on what computer vision techniques propose in that
field. A comprehensive comparison between different automatic
recognition systems is included. Finally, our approach is proposed
to automatically recognize and identify three different types of
fish diseases. These diseases are Epizootic ulcerative syndrome
(EUS), Ichthyophthirius (Ich) and Columnaris. Our approach Fig. 1: Different type of fish diseases.
shows the effect of different color spaces on the Convolutional
Neural Networkk CNN final performance.

I. I NTRODUCTION
Fish is a major component in global food supply. It makes
a big impact in economy. It is considered to be one of the
main resources for countries income. As a living organism, fish
suffers from various diseases that cause their death. Infection
(a) Earthen ponds (b) Tank indoor systems
spreads quickly in fish farms. Therefore, early identification
and diagnosis of fish diseases is an important step to prevent Fig. 2: Extensive ponds [2]
and control spreading of diseases. Late detection of diseased
fish can cause annihilation to the whole farm fish [1].
Diagnosing fish diseases depends on the expertise of experts Intensive pond relies on technology to raise fish in artificial
in the domain. Difficulties due to fast fish movement may tanks at very high densities Some intensive ponds are com-
cause tracking the infected fish to be impossible by human pletely closed system, as shown in Fig. 3a, this type of ponds
vision. Poor quality of unclear water in Earthen ponds also is called integrated recycling system. Another type is cage
cause limitations in diagnosis and tracking. Image processing ponds, this is shown in Fig. 3b.
and computer vision techniques support this field of research.
Vision can be improved and tracking fish becomes easier by
slowing down motion in videos, also automatic fish diagnosis
is necessary and needed. Different types of Fish diseases are
shown in Fig. 1.
II. BACKGROUND
(a) Integrated Recycling Ponds (b) Cage Ponds
There are different types of ponds of fish farms. These
types are earthen ponds, concrete ponds and floating cages. Fig. 3: Intensive Ponds [8]
These ponds are categorized to either extensive or intensive.
Extensive ponds are much like natural ecosystems with respect The environment in fish farm is based on the measured
to nutrient inputs, nutrient cycling, species diversity, oxygen amount of oxygen, ammonia-nitrogen, alkalinity and PH. If
dynamics, and level of human intervention. examples are any disruption happened in these measured values, it will lead
shown in Fig.3 . to fish diseases and consequent death.


ª
*&&& 
 TABLE I: Fish Diseases
Fish Diseases
Disease Description Symptoms Causes
Ichthyophthirius (Ich) [3] 1)It affects fish that are stressed. 1)It appears on the body, fins and gills 1)Change in the level of PH.
of fish as white nodules of up to 1 mm. 2)Low in the rate of the oxygen.
2)Its fungal infection. 2)Starting on the head and spreading over 3)High temperature.
3)Its morbidity rate may reach 100 Percent- the whole body.
age 2)Rapid breathing or gasping at the surface
of the water.
3)Darkness of the body color.
4)Resting on the bottom.
Motile Aeromonas septicemia MAS is almost appears in freshwater - 1)Hemorrhage at fins and on the skin . 1)Change in water temperature.
(MAS) [4] bacterial infection 2)Shallow to deep skin ulcers. 2)Infection occurred by water.
Most outbreak in summer and
spring.
Saprolegniasis [5] 1)Saprolegnia is a genus of water moulds. Cotton moulds(white or grey fibrous patches 1)Low water temperatures.
2)It effects on fish and fish eggs. they form like ”Wool)”. 2)Concurrent infections
3)It is found most frequently in freshwater
Columnaris [6] Its a fungal infection.Results from an infec- 1)Causing frayed and ragged fins. Low dissolved oxygen levels in the
tion caused by the ”Gram-negative”. 2)Fish will breathe rapidly. water column.
3)Gills will change colour.
4)The appearance of ulcerations on the skin.
5)White or cloudy,fungus-like patches.
Epizootic ulcerative 1) It is a disease caused by a fungal Red spots appear and are hardly identified. 1)Low temperature.
syndrome(EUS) [7] pathogen called Aphanomyces invadans. 2)Heavy rainfall in tropical and
2)It influences hundreds of different fish sub-tropical waters.
types.

There are two causes of fish diseases, infectious or non-

infectious illnesses. Fish infections may also cause exceeding
mortality. Mortality may reach to 90 percentage of stocks.
Non-infectious diseases may occur due to pollution of water.
Other environmental diseases may occur due to low dissolved
oxygen, high alkali, high nitrite.Infectious diseases can be
classified as viral, bacterial, fungal, parasitic, as shown in
Table I
III. S IMILAR S YSTEMS
Different approaches [9], [10], [11] [12] applied computer
vision techniques to detect and identify fish diseases. Some
approaches [9] [11] were proposed to classify epizootic ul-
cerative syndrome (EUS) diseased fish. Color segmentation
methodology is often applied on fish images to extract dam-
aged skin [10]. Another approach [12] extracted infected
regions and send notifications to fish farmers. Notification
include diagnosed disease and the suggested treatment method.
An Approach [9] applied histogram equalization followed
by edge detection for segmentation. Canny’s edge detector
achieved the best results, compared to other edge detectors.
Features from Accelerated Segment Test (FAST) [13] outper-
formed other features like Histogram of gradient (HOG) [14]
features when been applied on EUS infected images. Principal
Component Analysis (PCA) [15], [11] was applied after FAST
features to reduce feature vector dimensions. Neural Network
[16] achieved better recognition results compared to K-Nearest
Neighbor (k-NN) [17] when applied as a classifier.
K-Means clustering [18] was applied for segmenting dis-
eased area in hue-saturation-value (HSV) color space of fish
diseased images, based on the hue values. Morphological
operations [19] were applied to compute the diseased area,
as shown in equation 1.
A  B = (A  B) ⊕ B (1)
Where  and ⊕ represent erosion and dilation, respectively
and B is the structuring element.
Another usage of morphological operations was noise
removal and edge detection during preprocessing stage [12].
Morphological erosion and dilation operations were applied
to delete noise as shown in equations 2 and 3. Diseased area
were detected by discriminating small and large connected
components.
Erosion : I ⊗ SE = X : SE + X < I (2)
C
Dilation : I ⊕ SD = I c  (−SD ) (3)
where I is a source image, SE and SD are the structuring
elements for erosion and dilation operations,respectively. Two
types of features were extracted [12], the first is based on the
polar coordinates and the second set is based on the geomet-
rical features as defined in equations 4 and 5, respectively.
X = r cos θ, Y = r sin θ (4)
Where ris the radial distance from the origin and θ is the
counterclockwise angle with the x-xis, In terms of X, Y, r and
θ the geometric features were calculated by
 y
r = x2 + y 2 , θ = tan− 1( ) (5)
x
Convolutional Neural Networks (CNN) [20], were applied
for fish disease classification in another approach [21].Images
were enhanced by applying Gaussian blurring [22] and mor-
phological operations. Otsus thresholding [23] and Pyramid


 (a) ICH (b) Columnaris (c) EUS
Fig. 4: Infectious diseases

Mean Shifting (PMS) [24], defined in equation 6, were then

applied to enhance classification by CNN.

K(xi − x)xi
m(x) = xi ∈Nn (6)
xi ∈Nn K(xi − x)

IV. DATA - SETS

Approach [9] collected their database from some sources
as NGRF, Lucknow and CIFRI (academic institution). The
experiments were done on EUS diseased fish images. Ap-
proach [10] experiments was done on fish images with various
infections, as white spots and other damaged skin areas.
Approach [11] collected images of EUS infected fish from the
different part of the Barak Valley and Assam. Approach [12]
applied their approach on 60 different parasites microscopic
images, that were collected from National Bureau of Fish
Genetic Resources (NBFGR, Lucknow) and ICAR-Central
Inland Fisheries Research Institute (CIFRI), Kolkata.
Our data-set includes fish photos with the three types
of fish diseases, shown in Fig. 4. Our data were collected
from different internet resources. Due to limitation of these
resources and lack of diseased images, we applied image data
augmentation [25] to increase our samples. Augmentation was
based on zooming, shearing, rotating, and applying prepro- Fig. 5: Proposed System Overview
cessing function. Our collected data-set include 15 images
per disease. After applying data augmentation, samples are
increased to 800 images per disease. 550 images were used as infections. The overview of our proposed approach is shown
training samples per disease, and the remaining samples were in Fig. 5.
used for testing.
In our recognition system, we applied different color spaces
V. P ROPOSED A PPROACH on input images during pre-processing phase, as shown in Fig.
6. Three different color spaces were applied; RGB as shown
Our proposed approach aims to detect and diagnose fish dis- in Fig. 6a, Ycbcr as shown in Fig. 6b and XYZ as shown in
eases in fish farms automatically. Raspberry Pi kit is used and Fig. 6c.
connected to sensors, camera and a personal computer (PC).
The proposed system is presented in three consequent stages.
During the first stage, water quality is examined by measuring
water temperature and rate of pH, while fish captures are
acquired by the camera.The kit gets the sensors’ measurements
and acquired fish captures. In the second stage, all inputs are
passed to the PC for processing. This processing concerns (a) RGB (b) Ycbcr (c) XYZ
detecting any abnormal farm environment and fish infections.
Infection detection starts by pre-processing, then segmentation Fig. 6: Infected Area in Different Color Spaces
of infected areas and finally classification. Finally in the third
Stage, the kit is responsible for sending notifications to farm For segmentation, we built a Gaussian distribution in the
owners’ mobile phones, using GSM800 component, to notify Ycbcr color space, this is defined in equation 7. The values μ
them of any improper changes in farm environment or detected and σ are the mean and standard deviation of the cb and cr


values of our training infected crops. TABLE II: Alexnet Layers Description
√
f (x) = (1/σ 2π)e( 1/2(x − μ/σ)2 ) (7) Input Output Layers Stride Kernel size No. of
size size parameters
Gaussian functions [26] are commonly applied to represent 227x227x3 55x55x96 convolution 4 11x11 34944
55x55x96 27x27x96 maxpool 2 3x3 0
normal distributions, in the field of signal processing. The built 27x27x96 27x27x256 convolution 1 5x5 614656
distribution is then applied on test images, to measure the 27x27x256 13x13x256 maxpool 2 3x3 0
probability of including any infected area, as defined in Fig. 13x13x256 13x13x384 convolution 1 3x3 885120
7 13x13x384 13x13x384 convolution 1 3x3 1327488
13x13x384 13x13x256 convolution 1 3x3 884992
13x13x256 6x6x256 maxpool 2 3x3 0

vector one corresponding to each disease class. The detailed

architecture of the Alexnet CNN is shown in Fig. 8, and the
detailed description of each layer is shown in table II.
Fig. 7: Segmentation Block Diagram

Conversion from RGB color space to Ycbcr is defined by

equations 8, 9, 10.
65.738R 129.057G 25.064B
Y = 16 + + + (8)
256 256 256
37.945R 74.494G 112.439B
Cb = 128 − − + (9)
256 256 256
112.439R 94.154G 18.285B Fig. 8: Alex-net CNN Architecture
Cr = 128 + − − (10)
256 256 256
The XYZ color space [27] was used in the phases of feature
extraction and classification by convolution neural networks VI. E XPERIMENTAL R ESULTS
(CNN). XYZ values are generated from RGB extrapolations In our experiments, we compared between the performance
in order to avoid negative values. Y represents luminance in- of the ResNet18, ResNet50, ResNet101 and Alex-Net CNN ar-
formation, Z is the blue component, while X is a combination chitectures. Each architecture is trained on 2160 samples, and
between the orthogonal and the non-negative cone response finally tested on 314 samples. Experiments are done at learning
curves. To convert from RGB to XYZ, the equations 11, 12 rate= 0.001, training and testing accuracy are summarized in
and 13 are applied consequently. Fig. 9 and 10, respectively. Experiments were achieved in the
If R, G, B ≥ 0.04045, different color spaces: RGB, YCBCR and XYZ. It is shown
r, g, b = ((R, G, B + 0.055)/1.055)(2.4) ) (11) that Alexnet CNN outperforms the other architectures during
testing phase. The highest testing accuracy is achieved by
otherwise, Alexnet in the XYZ color spaces. The ResNet101 architecture
r, g, b = (R, G, B/12.92) (12) achieved the highest training accuracy in the RGB color space.
Then, matrix M is used to convert to XYZ.
   
X r
Z = (M ) b (13)
Y g
During feature extraction and classification phase,the effect
of different convolutional neural networks (CNN) [28] archi-
tectures was recorded. Our approach classified the diseases to
eihter columnaris, ichthyophthirius (ICH) or Epizootic ulcera-
tive syndrome (EUS). The CNN extract features automatically
in an efficient manner, and finally achieve classification using
the final fully connected layers [29].
The best classification results were achieved by Alexnet
CNN. Alexnet CNN is a deep neural network architecture
composed of eight layers. The first five layers are convolu-
tional layers (CONV) layers, and the final three layers are Fig. 9: Training Accuracy of Different CNN Architectures in
fully connected (FC) layers. The input images are normalized RGB, XYZ and YCBCR Color Spaces
to size 227x227x3, while the output outputs a 3x1 probability


 TABLE III: Similar systems comparison
System Function Data Algorithms Accuracy
[9] Automatically detects or diagnoses Images of the EUS infected fish Cannys edge detection algorithm– 86 %
the EUS diseased fish collected from sources as NGRF, Fast algorithm–PCA–Neural Net-
Lucknow and CIFRI work
[12] Extract pathogen area from the mi- Microscopic images of diseased 3x3 mean filter and edge sharp- 90 %
croscopic images of infected fish fishes collected from National Bu- ening filter–Morphological erosion
and sending notification about di- reau of Fish Genetic Resources and dilation operations–Polar and
agnosed disease and treatment to (NBFGR, Lucknow) and ICAR- geometric feature–PCA
the fish farmers Central Inland Fisheries Research
Institute (CIFRI), Kolkata
[11] Detects or diagnoses the EUS dis- Fish effected with (EUS) were col- PCA–K-Means Clustering–HSV 90 %
eased fish lected from the different part of the
Barak Valley, Assam
Proposed Approach Detecting fish diseases Collecting our own data (extract CNN (Alex-Net with XYZ ) 99.0446%
images from videos)
CNN (ResNet18 with XYZ ) 95.8599%
CNN (ResNet50 with XYZ ) 96.4968%
CNN (ResNet101 with XYZ ) 97.4522%

TABLE V: ResNet50 in RGB color space

ResNet50
Learning Rates Training Time Recognition Ac-
minutes:seconds curacy
0.1 14:59 94.2675%
0.01 17:9 93.3121 %
0.001 18:35 96.4%
0.0001 20:50 98.815%

TABLE VI: ResNet101 in RGB color space

ResNet101
Learning Rates Training Time Recognition Ac-
minutes:seconds curacy
Fig. 10: Testing Accuracy of Different CNN architectures in 0.1 16:29 94.2675%
RGB, XYZ and YCBCR Color spaces 0.01 14:49 92.3567 %
0.001 16:46 92.0%
0.0001 19:47 93.9490%
TABLE IV: ResNet18 in RGB color space
ResNet18
Learning Rates Training Time Recognition Ac- different types of fish diseases automatically. Different CNN
minutes:seconds curacy
0.1 24:4 92.0382%
architectures were applied on our collected data-set images in
0.01 14:43 90.7643% different color spaces. The Alexnet architecture achieved the
0.001 16:48 92.03% superior results in the XYZ color space.
0.0001 20:48 92.038%
VIII. F UTURE WORK
Our future work is implementing the Raspberry hardware
Table III summarizes the comparative results with sim- circuit to read temperature and ph sensor measurements, and
ilar systems. EUS disease was identified by two different to send notifications via GSM800 to users mobile phones. We
approaches [9], [11]. Colored segmentation was applied for are also planning to increase our dataset and including more
marking infected regions [10]. It was emphasized that com- fish diseases
bination of features [12] improved recognition system perfor-
mance, by achieving more convenient and rapid diagnosis of R EFERENCES
fish disease. Our proposed method outperformed the similar [1] FAO, The state of world fisheries and aquaculture 2018-meeting the
system, by diagnosing three different diseases with high accu- sustainable development goals (2018).
racy. Detailed experiments on the four CNN architectures in
the RGB color space are shown in Tables. IV, V, VI, VII. The TABLE VII: AlexNet in RGB color space
tables show the relation between the different learning rates
AlexNet
with their corresponding training time and achieved accuracy. Learning Rates Training Time Recognition Ac-
minutes:seconds curacy
VII. C ONCLUSION 0.1 9:33 90.4459%
0.01 7:40 90.1274 %
Identification of diseased fish at early stages is important 0.001 11:5 89.1720%
to prevent spreading fish diseases. The system diagnose three 0.0001 10:22 92.3567 %


 [2] A. Wezel, J. Robin, M. Guerin, F. Arthaud, D. Vallod, Management [25] A. Mikołajczyk, M. Grochowski, Data augmentation for improving
effects on water quality, sediments and fish production in extensive deep learning in image classification problem, in: 2018 international
fish ponds in the dombes region, france, Limnologica-Ecology and interdisciplinary PhD workshop (IIPhDW), IEEE, 2018, pp. 117–122.
Management of Inland Waters 43 (3) (2013) 210–218. [26] K. Wu, L. Huang, H. Lin, X. Kong, Face detection based on ycbcr
[3] R. Francis-Floyd, P. Reed, Ichthyophthirius multifiliis (white spot) gaussian model and kl transform, in: 2011 International Symposium on
infections in fish, Florida Cooperative Extension Service, Institute of Computer Science and Society, IEEE, 2011, pp. 100–103.
Food and Agricultural , 1991. [27] Z. Lu, X. Jiang, A. Kot, An effective color space for face recognition, in:
[4] F. K. Saikot, R. Zaman, M. Khalequzzaman, Pathogenecity test of 2016 IEEE International Conference on Acoustics, Speech and Signal
aeromonas isolated from motile aeromonas septicemia (mas) infected Processing (ICASSP), IEEE, 2016, pp. 2019–2023.
nile tilapia on some freshwater fish, Science International 2013 (2013) [28] H.-C. Shin, H. R. Roth, M. Gao, L. Lu, Z. Xu, I. Nogues, J. Yao,
325–329. D. Mollura, R. M. Summers, Deep convolutional neural networks for
[5] H. Ferguson, J. Morales, V. Ostland, Streptococcosis in aquarium fish, computer-aided detection: Cnn architectures, dataset characteristics and
Diseases of Aquatic Organisms 19 (1) (1994) 1–6. transfer learning, IEEE transactions on medical imaging 35 (5) (2016)
[6] A. M. Declercq, F. Haesebrouck, W. Van den Broeck, P. Bossier, 1285–1298.
A. Decostere, Columnaris disease in fish: a review with emphasis on [29] D. Gupta, Architecture of convolutional neural networks (cnns) demys-
bacterium-host interactions, Veterinary research 44 (1) (2013) 1–17. tified, Analytics Vidhya.
[7] J. Lilley, R. Callinan, S. Chinabut, S. Kanchanakhan, I. MacRae,
M. Phillips, Epizootic ulcerative syndrome (eus) technical handbook,
Tech. rep., Aquatic Animal Health Research Institute (1998).
[8] N. R. Bromage, C. J. Shepherd, Intensive fish farming, BSP Professional
Books, 1988.
[9] S. Malik, T. Kumar, A. Sahoo, Image processing techniques for identi-
fication of fish disease, in: 2017 IEEE 2nd International Conference on
Signal and Image Processing (ICSIP), IEEE, 2017, pp. 55–59.
[10] V. Lyubchenko, R. Matarneh, O. Kobylin, V. Lyashenko, Digital image
processing techniques for detection and diagnosis of fish diseases,
International Journal of Advanced Research in Computer Science and
Software Engineering (2016) 79–83.
[11] H. Chakravorty, R. Paul, P. Das, Image processing technique to detect
fish disease, International Journal of Computer Science and Security
(IJCSS) 9 (2) (2015) 121.
[12] J.-S. Park, M.-J. Oh, S. Han, Fish disease diagnosis system based on
image processing of pathogens’ microscopic images, in: 2007 Frontiers
in the Convergence of Bioscience and Information Technologies, IEEE,
2007, pp. 878–883.
[13] A. Audi, M. Deseilligny, C. Meynard, C. Thom, Implementation of an
imu aided image stacking algorithm in a digital camera for unmanned
aerial vehicles, Sensors 17 (2017) 1646. doi:10.3390/s17071646.
[14] E. Fotiadis, M. Garzón, A. Barrientos, Human detection from a mobile
robot using fusion of laser and vision information, Sensors 13 (9) (2013)
11603–11635.
[15] A. Tharwat, Principal component analysis-a tutorial., IJAPR 3 (3) (2016)
197–240.
[16] J. Schmidhuber, Deep learning in neural networks: An overview, Neural
networks 61 (2015) 85–117.
[17] P. Cunningham, S. J. Delany, k-nearest neighbour classifiers, Multiple
Classifier Systems 34 (8) (2007) 1–17.
[18] S. Na, L. Xumin, G. Yong, Research on k-means clustering algorithm:
An improved k-means clustering algorithm, in: 2010 Third International
Symposium on intelligent information technology and security informat-
ics, IEEE, 2010, pp. 63–67.
[19] S. Ravi, A. Khan, Morphological operations for image processing:
understanding and its applications, in: Proc. 2nd National Conference on
VLSI, Signal processing and Communications NCVSComs-2013, 2013.
[20] S. Liu, W. Deng, Very deep convolutional neural network based image
classification using small training sample size, in: 2015 3rd IAPR Asian
conference on pattern recognition (ACPR), IEEE, 2015, pp. 730–734.
[21] D. Rathi, S. Jain, S. Indu, Underwater fish species classification using
convolutional neural network and deep learning, in: 2017 Ninth Interna-
tional Conference on Advances in Pattern Recognition (ICAPR), IEEE,
2017, pp. 1–6.
[22] E. S. Gedraite, M. Hadad, Investigation on the effect of a gaussian blur in
image filtering and segmentation, in: Proceedings ELMAR-2011, IEEE,
2011, pp. 393–396.
[23] P. M. Bhargavi, V. S. K. Mayee, T. Manaswini, S. Manvitha, A
comparison of image segmentation techniques, otsu and watershed for
x-ray images, IJRET: International Journal of Research in Engineering
and Technology 4 (04).
[24] M. Stojmenovic, A. S. Montero, A. Nayak, Link shifting based pyramid
segmentation for elongated regions, in: International Conference on
Signal Processing, Image Processing, and Pattern Recognition, Springer,
2009, pp. 141–152.