Professional Documents
Culture Documents
0:
RESEARCH CHALLENGES AND OPPORTUNITIES
Received February 28, 2019, accepted March 21, 2019, date of publication March 28, 2019, date of current version April 17, 2019.
Digital Object Identifier 10.1109/ACCESS.2019.2908040
Corresponding authors: M Ikram Ullah Lali (ikramullah@uog.edu.pk) and Khursheed Aurangzeb (kaurangzeb@ksu.edu.sa)
This work was supported by the Deanship of Scientific Research at King Saud University for funding this work through research group
under Grant RG-1438-034.
ABSTRACT Agriculture is a major part of the world economy as it provides food safety. However, in recent
years, it has been noted that plants are extensively infected by different diseases. This causes enormous
economic losses in agriculture industry around the world. The manual inspection of fruit diseases is a
difficult process which can be minimized by using automated methods for detection of plant diseases at the
earlier stage. In this paper, a new method is implemented for apple diseases identification and recognition.
Three pipeline procedures are followed by preprocessing, spot segmentation, and features extraction, and
classification. In the first step, the apple leaf spots are enhanced by a hybrid method which is the conjunction
of 3D box filtering, de-correlation, 3D-Gaussian filter, and 3D-median filter. After that, the lesion spots
are segmented by the strong correlation-based method and optimized their results by fusion of expectation
maximization (EM) segmentation. Finally, the color, color histogram, and local binary pattern (LBP) features
are fused by comparison-based parallel fusion. The extracted features are optimized by genetic algorithm
and classified by One-vs-All M-SVM. The experimental results are performed on plant village dataset. The
proposed methodology is tested for four types of apple disease classes including healthy leaves, Blackrot,
Rust, and Scab. The classification accuracy shows the improvement of our method on selected apple diseases.
Moreover, the good preprocessing step always produced prominent features which later achieved significant
classification accuracy.
INDEX TERMS Symptoms enhancement, symptoms segmentation, feature extraction, optimal features,
recognition.
2169-3536 2019 IEEE. Translations and content mining are permitted for academic research only.
VOLUME 7, 2019 Personal use is also permitted, but republication/redistribution requires IEEE permission. 46261
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
M. A. Khan et al.: Optimized Method for Segmentation and Classification of Apple Diseases
nutrient values. However, its production and quality are dam- TABLE 1. List of abbreviations and their description.
aged by the attack of different diseases like black rot, rust,and
blight. Therefore, it is essential to develop an automated com-
puterized system for detection and classification of apple leaf
symptoms at early stage. The early detection of these symp-
toms are helpful to improve the fruits quality and production.
In the field of computer vision, it is an active research area
to find out the lesion spot [4] and recently several methods
are introduced for fruits diseases detection through image
processing and machine learning algorithms [5]. Normally,
the apple lesion spots are examined by their color, texture,
and shape, therefore, the features like color, texture, and
shape are essential for classification of apple diseases. A lot
of segmentation, features extraction, and classification tech-
niques are proposed in literature for fruits diseases segmenta-
tion and recognition such as combination of features based
symptoms identification [6], color based segmentation [7],
correlation based features selection [4], optimized weighted
segmentation [8], texture features [9], shape features [10],
Support Vector Machine (SVM) [11], and few more [12].
A. PROBLEM STATEMENT segmentation, and feature selection blocks, which are enu-
Various global challenges exist in the agriculture domain merated below.
such as change of environment and appearance of multiple a) A hybrid contrast stretching technique is proposed in
symptoms in one image. The complex background and mul- which a 3D-Box filter is applied, which initially blurs the
tiple symptoms regions are not easy to detect from an image. boundary of a lesion spot, followed by an implementation of a
The presence of symptom regions on the boundary of image, de-correlation function to reduce image distortion. This com-
various characteristics of leaf symptoms, the existence of plete procedure is facilitated by 3D-Gaussian and 3D-Median
noise, change in lighting condition, and color based similarity filter - making the foreground and background maximally
among various symptoms. Each lesion spot has a different differentiable.
shape, color, and texture which makes the detection process b) A diseased spot segmentation algorithm is proposed in
difficult which later utilized in the classification phase. The which ’S’ channel of HSI color space is considered, which is
challenges which researchers faced in this domain using com- later used to identify set of clusters having higher pixel values.
puter vision and machine learning techniques are: a) correctly A strong correlation (SC) based method is later proposed to
segmentation of leaf symptoms in the image which exist select a strong cluster, followed by an optimization step using
in various symptoms and most are similar to background expectation maximization (EM) algorithm. This step plays its
regions, (b) extraction of sufficient number of features for role especially under the scenario when the infected regions
classification because the irrelevant features do not produce are small in size.
good accuracy and also effects on the system execution time, c) A method is designed to utilize color and tex-
and (c) selection of most prominent features. The justification ture features, which are later fused through parallel
behind these key challenges of any computer vision system approach. An addition of feature selection step using
are: (a) the correct symptoms segmentation is performed only genetic algorithm (GA) further expedite the process for
through clear difference among symptoms pixels and healthy getting improved classification results using support vector
pixels, which are defined through a good preprocessing step, machines.
(b) a good computer-based automated method always needs
a minimum good number of features for sufficient accuracy C. ABBREVIATIONS AND ACRONYMS
along better execution time.
II. RELATED WORK
The common apple plant diseases are apple rust, apple rot,
B. CONTRIBUTIONS apple scab, apple greening, and powdery mildew. Recently,
A new automated apple leaf diseases method is implemented several researchers introduced various techniques such as
for detection and recognition through symptoms segmenta- Dubey and Jalal [13] described a new computer vision
tion and best features selection. Three pipeline procedures based method using multiple features. The described method
including image enhancement, lesion spot segmentation, followed three major steps- i) diseased regions are seg-
and prominent features selection & classification are uti- mented by K-Means, ii) color, LBP and shape features are
lized. Our major contributions are in contrast enhancement, extracted using segmented lesion spots. Finally, classification
is performed through MSVM and experiments are done on few more. Ali et al. [19] described an automated method for
four types of apple diseases including apple blotch, rot, detection and classification of citrus diseases. The described
scab, and healthy. For each disease class, 80 images are method is based on two major steps including symptoms seg-
collected from Google and achieved maximum classification mentation and feature extraction. In segmentation, the Delta
accuracy on different feature vectors 98.75%, 95%, 96.25%, E method is utilized and later on, texture and color features
and 100%, respectively. Khan et al. [6] presented an auto- are extracted. The extracted features are reduced by PCA
mated system for multiple fruit diseases using deep learning. and fed to SVM for classification, which achieved maximum
The presented method process into two primary steps- dis- recognition accuracy 99.9%.
ease segmentation through correlation based approach and
classification of fruit diseases by deep features extraction III. PROPOSED METHODOLOGY
from pre-trained CNN models like VGG16 and AlexNet. Three primary steps are incorporated int he proposed system-
Chuanlei et al. [4] introduced a new image processing based contrast enhancement and symptom segmentation, multi-type
technique for the recognition of apple diseases. In the features extraction, and prominent features selection and
presented method, RGB color transformation structure is recognition, main architecture demonstrated in Figure. 1.
designed and segment the symptoms through region growing.
Finally, features are extracted for each spot with reduced A. DISEASE SPOT ENHANCEMENT
dimensions of extracted features by GA and correlation coef- Preprocessing step has gained much importance in CV and
ficients. For experimental results, 90 diseased leaves images image processing because of low contrast images effects
are selected from three types of disease classes including overall accuracy. This step is essential for later work on
powdery mildew, mosaic, and rust with classification rate disease spot segmentation and feature extraction because dis-
is 90%. Dubey and Jalal [14] utilized a K-Means clustering eases can be usually identified according to the symptoms
method for apple fruit diseases segmentation. After that, shape, color, and texture. In this article, Plant Village dataset
extract texture (LBP, CLBP), color and global color his- is used for evaluation. The Plant Village dataset contains
togram features and combined using simple concatenation several types of problems such as noise, brightness, and
method. For experimental results, three types of apple fruit change of the origin. In this article, an optimized algorithm
disease images are selected and achieved recognition accu- is implemented to address these problems and to improve
racy up to 93%. Also, the described method performs well for the performance of quantitative image analysis. The pro-
CLBP features and achieved recognition accuracy 89.88%, posed method consists of three pipeline procedures including
90.71%, 96.66%, and 99.33% for apple blotch, rot, scab, and 3D-Box filtering, De-correlation, and the combination of a
healthy. Ormani et al. [15] introduced an automated tech- 3D-Gaussian and 3D-Median filter.
nique for detection of apple diseases. The introduced method
is based on three series of steps. In the first step, the lesion 1) 3D-BOX FILTERING
spots are segmented by K-Means clustering method. Then, For smoothing the disease spot region in the image,
the color, GLCM, and wavelet features are extracted from 3D-Box filter operation is performed. The 3D-box filter is
segmented spots, which are classified by support vector used equally weighting a rectangular neighborhood of pixels
regression (SVR). The major advantage of SVR algorithm in three major steps: a) compute filter size; b) padding, and c)
is to minimize the error rate and increase the classification normalization factor. Let φxyz is an RGB input image having
accuracy. Shuaibu et al. [16] presented an optimized stochas- dimension 256 × 256. Let ϕabc be an intensity value of size
tic algorithm for identification of apple marssonina disease. a × b × c 3-dimensional image ϕ at position (m, n, o), while
A PSO algorithm is utilized to find out the optimal features ϕ is 3D-box filter into φ. Then, the size of the filter is defined
based on their highest discrimination. The selected features as follows:
are classified by SVM and achieved 100% accuracy. Sing
a1 X
b1 X
c1
and Misra [17] described a two step automated technique. X
ϕijk = (ϕmno ) (1)
The disease lesions spot are segmented by an optimized
m=1 n=1 o=1
genetic algorithm. The GA perform significant well with
SVM and achieved the segmentation accuracy 95.71%, which where, ϕijk denotes the size of filter, a1 = 4, b1 = 4, and
is significantly better as compared to MDC with K-Means c1 = 3. After that, we perform padding on φxyz using filter
and MDC with the proposed method. The major advantage size as:
of GA is that it can process a large number of variables ϕijk − 1
at the same time. Barbedo [18] introduced a new method PD = (2)
2
for segmentation of disease spots on plant leaves based on
where, PD denotes the padding window, which is performed
manipulation of a color channel and boolean operation, which
on RGB image φxyz as follows:
is performed on the binary mask. The introduced method
resolves many challenges such as variability and change in N
X
condition. For experimental results, 77 disease images are Pd (φ) = (φxyz , PD ) (3)
selected of 11 plants including citrus, coffee, corn, cotton, and x,y,z=1
where, Pd (φ) is padding image. Each padding window is image DC(x, y) as follows:
updated horizontally based on adding a padding window on
vj = T (x, y) × (ui − µ) + µt (6)
the right side and subtracting out a value from the left side.
The updating of padding window is defined as follows: where, T (x, y) is an nBands − by − nBands matrix, µ denotes
P the mean of BF(x, y), and µt is mean of target output image
Pd (φ)
Update(Pd (φ)) = (4) DC(x, y). Then, calculate the covariance matrix of ui and vj
t as follows:
where t is a constant parameter for both left and right window
Cov(ui , vj ) = E (ui − µi )(vj − µj )
(7)
at the time of updating. Finally, performed normalization
Cov(ui , vj ) = E (ui vj )(µi µj )
on the image after completion of all padding iterations as (8)
follows: where, µ = E(ui ) or E(vj ). After that,we compute target out-
1 put T (x, y) to perform an eigen decomposition of covariance
NLφ = Q (5) matrix as:
(Pd (φ))
Q T (x, y) = σt × cov(ui , vj )σ −1 (9)
where, NLφ denotes the normalization factor, and ()
denotes the product of padding image. The effects of where, σt denotes the standard deviation (SD) of target out-
3D-bx filtering operation on the original RGB image is shown put, and σ −1 is SD of input
q 3D-box filter image. The SD (σ )
in Figure 2. is computed as: σ = E(Cov(ui , vj ))2 − [E(Cov(ui , vj ))]2 .
2) COLOR DIFFERENTIATION Hence, the final enhanced decorrelation image is obtained as
Color differentiation is used to improve the color difference follows:
between each pixel in the image. Therefore, to address this DCF (x, y) = µt + σt × cor(ui , vj )σ −1 × (ui − µ) (10)
problem, we used decorrelation. Decorrelation is a general
term for any process that is used to reduce the serial corre- where, cor(ui , vj ) = σ × cov(ui , vj )σ −1 . The effects of
lation between two similar processes. It is done on the basis decorrelation stretching is shown in Figure 3.
of band to band correlation. Therefore, the excessive color
improves the visual perception of an image and also make it 3) IMAGE NORMALIZATION
easier for feature discrimination. In general, the image normalization is a process that improves
Let BF(x, y) denotes the 3D-box filter image, which is the range of pixel intensity values which is sometimes called
obtained from equation 1 to 5. Let DC(x, y) denotes the target contrast stretching [21]. Normalization is performed in form
output decorrelation image. Then, the vector x ∈ 8ui ×vj , of smoothing the image and removing the noise factor. The
where ui and vj denotes the nbands − by − 1 vector of 3D- enhanced decorrelation image (DCF (x, y)) is used for this
box filter image BF(x, y) and target output image DC(x, y). processes because it contains too much noise as shown in
Therefore, the vector ui containing the pixel value of BF(x, y) Figure 4. Therefore, we used 3D-Gaussian and 3D Median
is transformed into the corresponding pixel vj in the output filter to remove these artifacts. The Gaussian filter smoothes
FIGURE 2. 3D-box filtering operation effects. a) Original image; b) 3D-box filter image, and c) histogram
of 3D-box filter image.
FIGURE 3. Color differentiation effects using decorrelation. a) 3D-box filter image; b) decorrelation image, and c) histogram of
decorrelation image.
the disease spot region but at the same time it reduces the process, the 3D Gaussian filter [20] is defined as:
disease information, therefore, an extra 3D Median filter is 2 +y2 +z2
performed after performing the 3D-Gaussian filter on the 1 −( x )
G33(x, y, z) = e 2π σ 2 (14)
3
image. σ 3 (2π) 2
Let DCF (x, y) denotes the three dimensional enhanced
decorrelation image having dimension 256 × 256 × 3. Let where, G33(x, y, z) is 3D-Gaussian filter image, whose
G(x, y) denotes the 2D gaussian filter, which is defined as effects are given in Figure 4(d). Finally, a 3D-Median filter
follows: (3 × 3 × 3) is performed on Gaussian filtered image, which
efficiently removes the remaining noise after performing 3D-
1 2 2 Gaussian filter. The effects of enhancement after performing
−( x +y )
G(x, y) = √ e 2σ 2 (11) 3D-Median filter image are shown in Figure 4(e).
σ 2π
where, σ 2 denotes the variance of given image and σ is SD. IV. DISEASE SEGMENTATION
The variance σ 2 and σ is computed as follows: Plant disease is one of the major factors which impacts to
diminish quantity and reduce the quality of the product.
P
(DCF (x, y))2
P
(DCF (x, y)) 2 Therefore, to control the plant diseases, it is much important
σ =
2
−( ) (12) to diagnose them at early stages. Several image process-
N N
s ing methods have been utilized for segmentation of such
(DCF (x, y))2
P P
(DCF (x, y)) 2 diseases in agriculture such as Fuzzy-C Means, K-Means
σ2 = −( ) (13)
N N Clustering, Otus segmentation, and color segmentation. From
all segmentation methods, the color segmentation is much
where, N denotes the total number of pixels in the input important because each disease spot has a different color.
DCF (x, y) image. To following above 2D gaussian filter Moreover, there are several challenges including the shape
FIGURE 4. Image Normalization effects. a) Original image; b) 3D-box filtering image; c) decorrelation image; d) 3D-Gaussian image,
and e) 3D-Median filtered image.
of disease spot, change in color and texture, and change in and extract saturated channel (S) as as follows:
origin and scale. To address these challenges, we present
an optimized disease segmentation method based on Strong 3 × min(R(u, v), G(u, v), B(u, v))
S(u, v) = 1 − (15)
Correlated Pixels (SCP) and Expectation Maximization (EM) G33(x, y, z)
algorithm. The optimization is performed based on the fusion
of SCP and EM by entropy method as shown in Figure 5. where, S(u, v) is saturated channel and overall effects are
The detailed description of the optimized method is given shown in Figure 5, which shows that the disease spots are
below. clearly highlighted after performing HSI transformation.
As G33(x, y, z) is an enhanced RGB image, where After that, we calculate the histogram of selected S channel
(x, y, z) denotes its pixel values. Suppose, R(u, v) = based on their mesh representation. Then, the number of bins
G33(x,y,z) , , G(u, v) = G33(x,y,z) , and B(u, v) = G33(x,y,z)
G33(red) G33(green) G33(blue)
based on the higher brightness probability value is calculated.
denote the extracted red, green, and blue channel from The higher brightness value is computed based on their his-
enhanced image. Then, HSI transformation [22] is performed togram as given in Figure 13(c). The probability is defined as
follows:
pi
Prob(pix) = P (16)
(P(pix))
where, Prob(pix) denotes the probability value of image
pixels, pi denotes the intensity value of each pixel, and
P
(P(pix)) denotes the intensity value of all pixels in the FIGURE 6. Initial disease spot extraction effects. a) Selected S channel; b)
mesh representation; c) histogram of mesh representation; d) initial
image. Then, we select the lower intensity value probability binary image.
pixels in one cluster and later in other clusters. Let C1 denotes
the cluster which have higher probability intensity values (in where, ED(xi ) denotes the distance pixel values, which are
other words those pixels which pixels range is near to 180 or later utilized in threshold function to convert image into
greater), which means that those pixels which are ≥ 180. Let binary form.
C2 denotes those pixels, which pixel range is < 180. Then (
select the higher intensity value pixels as follows: 1 if xi ≤ 0.35
T (xi ) = (19)
( 0 Otherwise
C1 if Prob(bin) ≥ 180
SL(bin) = (17) The threshold function T (xi ) gives a binary image, which is
C2 if Prob(bin) < 180 shown in Figure 6. In Figure 13(a) the highlighted regions
are disease spots, (b) denotes the mesh representation of
where SL(bin) denotes the clustered matrix. Then, C1 is
highlighted regions, (c) represents the histogram of mesh
selected based on their higher pixel values and Euclidean
representation, and (d) shows its initial binary representa-
Distance (ED) between each pixel is calculated. In pattern
tion after performing threshold function. Finally, we perform
recognition and computer vision, the ED playa an important
correlation coefficient (CC) to measure the strength between
role to measure the distance between objects in the image.
pixels, which are computed after threshold function. Through
Therefore, we calculate the ED between selected image pixels
CC, select the strong correlated pixels as follows: As we know
and select the minimum distance pixels. Let 1 be the pixel
that strong correlation between two variable is (Strongr = 1)
values of cluster C1 , where C1 denotes the high-intensity
and weak correlation is defined by Weakr = 0, then e r is
value pixels. Let xi ∈ 1, where i = x1 , x2 , . . . xn number
computed as follows:
of pixels. Then ED between xi pixel is calculated as follows: P P P
n Txy − ( Tx )( Ty )
q r=p P (20)
ED(xi ) = (x2 − x1 )2 + (x3 − x2 )2 , . . . (xn − xn−1 )2
e q P
n( Tx2 ) − ( Tx )2 n( Ty2 ) − ( Ty )2
P P
X n q
ED(xi ) = (xi − xi−1 )2 (18) where, e
r denotes the correlation value of threshold image,
i=0 which is utilized as a threshold value to obtain final
FIGURE 9. Image segmentation effects. a) HSI image; b) selected S Channel; c) SCP segmented image; d) EM
segmentation; e) optimized EMI segmentation, and f)mapped EMI on segmented spots.
extracted LBP vector is 1×59, which is later fused with color, presented red, green, and blue channel, respectively, where
and CH features as shown in Figure 10. (u, v) denotes the pixel of red channel. Then, mean, variance,
SD, skewness, kurtosis, entropy, energy, and mode of R(u, v)
2) COLOR HISTOGRAM FEATURES is calculated as follows [25]:
As G33(x, y) be an RGB enhanced image having size J × L, X X R(u, v)
where J = 256, and L = 256. Then, we perform conver- µc = (26)
u v
Z
sion of three color spaces including RGB, HSV, and HSI
(R(u, v))2
P
for CH feature extraction. Initially, we divide each image σc2 = − µ2c (27)
into their respective three channels. For example, the RGB s Z
image consists of three channels such as red, green, and P
(R(u, v))2
blue. Thereafter, we perform histogram equalization on each σc = − µ2c (28)
Z
color channel and extract 8 different feature including mean,
Z −1
variance, SD, skewness, kurtosis, entropy, energy, and mode 1 X
SKc = (R(z1 ) − µc )3 Probg (29)
for all channels. The mean of the channel describes its general σc3 z
1
brightness, however, the variance and SD are used to find µc
out how each pixel in the image varies from the neighboring Modec = (30)
σc × SKc
pixels or central pixels, and contrast of an image, respectively.
Z −1
Moreover, the skewness finds the asymmetry in their intensity X
Entropyc = − Prob(R(u, v)). log2 Prob(R(u, v)) (31)
values, entropy explains that how many bits are needed to
z1 =0
code the image data, and energy tells about the smoothing
Z −1
pixels in the given channel. These selected features are cal- X
Energyc = [Prob(R(u, v))]2 (32)
culated as follows:
z1
Let G33(x, y) is a three-dimensional enhanced RGB
G33(x) G33(y) Z −1 (R(u,v))i −µc )
image, then R(u, v) = G33(x,y,z) , G(u, v) = G33(x,y,z) , X
Z
Kurc = (33)
G33(z)
and B(u, v) = G33(x,y,z) , where R(u, v), G(u, v), and B(u, v) z1
σ 4
c
where, µc is mean feature for red channel, Z denotes the total B. FEATURE SELECTION
number of pixels in R(u, v) channel, σc2 is variance feature For computational purposes, images are represented by col-
for red channel, σc2 is SD feature for red channel, SKc is lection of features. To build an efficient classification model,
skewness feature, z1 ∈ (u, v), Probg is probability of gray it is essential that features contain much information and
levels Z −1, Modec is mode feature for red channel, Entropyc number of features are kept as small as possible. However,
is entropy feature, Prob(R(u, v)) probability of red channel it is difficult to distinguish that which features are appropriate
pixels, Energyc is energy feature for red channel, and Kurc for accurate classification of images. Because, the extracted
is kurtosis feature for red channel. Hence, for one channel, features are large in number and it contains many irrelevant
total 8 features are extracted. Similarly, the length of all three features, which not only degrade the system accuracy but
color spaces, the feature vector (FV) is 1 × 72 as shown also effect on the system execution time. To remove the
in Figure 10. Finally, the simple color features are extracted adverse consequences of unnecessary and redundant features,
from the original RGB image. To follow above CH features many feature selection techniques have been proposed such
extraction process, 8 similar feature parameters are extracted as competitive swarm optimizer based algorithm [26].
for each channel. The size of extracted feature vector for one As given FV (i) is fused feature vector having dimension
channel is 1 × 8 and length of final color FV is 1 × 72, which 1 × 72. Let ψ(OP(i)) denotes the optimized feature vec-
is later fused with CH and LBP feature vector. tor having dimension A × B, where A denotes the index
For feature fusion, a high-value feature based parallel of extracted feature and B denotes the length of optimized
fusion is employed. The fusion process is completed in two feature vector. Generally, the GA is comprised of five major
steps: a) same length of each feature vector, and b) high-value steps including initialization, crossover, mutation, selection,
feature based fusion. In the first step, make the same size of and fitness function. In initialization step, the selected num-
each feature vector based on ‘‘0’’ appending. After, that each ber of iterations (It = 500), number of populations (NP =
index of given feature vectors is compared to each other and 20), crossover rate (CP = 0.4), mutation rate (MP = 0.001),
the higher-value feature is selected for a fused vector. The and selection of parent pressure is β = 6. Amongst, sev-
length of a final fused vector is 1 × 72. The detailed flow of eral crossover techniques, we select the uniform crossover
color feature extraction and fusion is given in Figure 10. technique [27] with crossover rate 0.4. The crossover func-
tion is defined as follows:
ψ(CH )sl if ψ(CH )i ≥ ψ(CL)i & ψ(SF)i
FV (i) = ψ(CL)sl if ψ(CL)i ≥ ψ(CH )i & ψ(SF)i ψCR = CROSS(h1 , h2 )
ψ(SF)sl if ψ(SF)i ≥ ψ(CH )i &ψ(CL)i h1 = α × f1 + (1 − α) × f2
(34) h2 = α × f2 + (1 − α) × f1 (35)
where, FV (i) denotes the final fused feature vector, ψ(CH )sl , where, h1 , h2 denotes the given input features, and f1 , f2
ψ(CL)sl , and ψ(SF)sl denotes the selected color histogram, are new offspring individuals after crossover operation.
color, and LBP feature, respectively. In the next step, the offsprings are mutated using uniform
Algorithm 1 Features Fusion and Selection TABLE 2. Brief description of selected training and testing images of
1: Input : ψ(CL), ψ(CH ), ψ(SF)
various apple leaf diseases.
FIGURE 11. Annotated sample for each selected apple disease and healthy leaf.
(Plant Village) and few of sample images are shown in we opt a ratio of 50:50 ratio and results are evaluated using
Figure 12. The ground truth of selected images is drawn from 10-fold cross-validation (10CV). The best achieved accuracy
an expert in this area from Agriculture University Fasilabad, of 92.9% for M-SVM as given in Table 4. The classification
Pakistan. The accuracy of segmented images is computed by accuracy of M-SVM is also verified through Table 5. More-
their corresponding ground truth images and few of segmen- over, the graphical comparison of classification methods is
tation results in numerical form are also presented in Table 3. given in Figure 14, which shows that the M-SVM perform
The average accuracy is 93.916% which is good as compare significantly well for Test 1 in terms of specificity, accuracy,
to other methods as plotted in Figure 14. The maximum INF, and G-measure. Moreover, the testing execution time
segmentation accuracy on our method is achieved as 97.79%. of M-SVM is 7.862 seconds, which is better as compared
Moreover, the sample visual segmentation results with their to other classification methods. The worst execution time is
corresponding ground truth images are shown in Figure 13. 69.260 seconds for ESDA but its classification rate is 91.50%,
The above discussion clearly showed that the proposed seg- which is better as compared to CT, L-SVM, F-KNN, W-KNN,
mentation method performs well on selected images. and EBT.
TABLE 3. Accuracy results for proposed optimized segmentation method. TABLE 6. Classification results for PCA based feature reduction.
FIGURE 13. Comparison of proposed segmented effects with ground truth images. a)
Original RGB image; b) selected S channel; c) optimized EMI segmentation; d) mapped
on original image, and e) ground truth image.
FIGURE 14. Graphical comparison of M-SVM with other classification methods in terms of accuracy,
specificity, Informedness, and G-Measure for Test 1.
FIGURE 15. Graphical comparison of M-SVM with other classification methods in terms of accuracy,
specificity, Informedness, and G-Measure for Test 2.
FIGURE 16. Graphical comparison of M-SVM with other classification methods in terms of accuracy,
specificity, Informedness, and G-Measure for Test 3.
For classification results, three different experiments are done Existing well-known techniques such as Yolo, RCNN,
on select dataset such as without feature selection, PCA based F-RCNN, and SSD models show improved performance in
feature reduction, and proposed feature selection based anal- object detection for both real-time and static image. For the
ysis. The maximum classification accuracy of Test 1 is 92.9% case when the object/s of interest is too small, the overall per-
as presented in Table 3 and Table 4. In Test 2, the PCA based formance will be compromised in terms of overall accuracy.
feature reduction method performs well as compare to Test 1 In [34], authors validated YOLO CNN model on the traffic
and achieved maximum classification accuracy is 94.3% as signs and concluded that a set of stop signs which are quite far,
given in Table 5 and Table 6. The proposed achieved maxi- as well as small in size are not easy to detect, hence, showed
mum classification accuracy 97.20% as presented in Table 7, only 30% accuracy. Whereas, in this work, our primary objec-
which is confirmed by their confusion matrix in Table 8. tive is to detect the region of interest (infected) and then
The proposed method performs significantly well as com- extract its features for recognition. Due to a fact, the features
pare to Test 1 and Test 2 in terms of execution time, INF, from cropped images give better accuracy, as the irrelevant
G-M, Sensitivity, and Precision. Moreover, the graphical rep- information (background) is discarded, and only the rele-
resentation of all three Tests is shown in Figure 12, 13, and 14. vant information is considered. For the aforementioned CNN
REFERENCES
[1] S. Savary, A. Ficke, J-N. Aubertot, and C. Hollier, ‘‘Crop losses due to
diseases and their implications for global food production losses and food
security,’’ Food Secur., vol. 4, no. 4, pp. 519–537, 2012.
[2] K. R. Gavhale, and U. Gawande, ‘‘An overview of the research on plant
leaves disease detection using image processing techniques,’’ J. Comput.
Eng., vol. 16, no. 1, pp. 10–16, 2014.
models, a large number of ground truth images are required [3] B. J. Samajpati, and S. D. Degadwala, ‘‘Hybrid approach for apple
fruit diseases detection and classification using random forest classi-
for training, which in our used dataset, are not publicly avail- fier,’’ in Proc. Int. Conf. Commun. Signal Process. (ICCSP), Apr. 2016,
able. Moreover, our algorithm detects minuscule symptoms pp. 1015–1019.
from the leaf images more accurately - compared to set of [4] Z. Chuanlei, Z. Shanwen, Y. Jucheng, S. Yancui, and C. Jia, ‘‘Apple leaf
disease identification using genetic algorithm and correlation based feature
CNN models, as well as other algorithms like thresholding selection method,’’ Int. J. Agricult. Biol. Eng., vol. 10, no. 2, pp. 74–83,
etc. The recognition accuracy of existing techniques are also 2017.
presented in Table 10. For comparison, we are only consid- [5] A. Camargoa and J. S. Smith, ‘‘An image-processing based algorithm
to automatically identify plant disease visual symptoms,’’ Biosyst. Eng.,
ering the methods tested on plant village dataset. Recently,
vol. 102, no. 1, pp. 9–21, 2009.
Khan et al. [6] presented an automated approach for fruit [6] M. A. Khan et al., ‘‘CCDF: Automatic system for segmentation and
diseases by achieving an accuracy of 98.10% on apple black recognition of fruit crops diseases based on correlation coefficient and
rot and for apple scab, 96.90%. In this work, we achieve an deep CNN features,’’ Comput. Electron. Agricult., vol. 155, pp. 220–236,
Dec. 2018.
average accuracy of 98.10% on of apple black rot, on apple [7] L. J. Rozario, T. Rahman, and M. S. Uddin, ‘‘Segmentation of the region
scab its 97.30%, and 94.6% on apple rust. From the results, of defects in fruits and vegetables,’’ Int. J. Comput. Sci. Inf. Secur., vol. 14,
it can comfortable concluded that the proposed method out- p. 399, May 2016.
[8] M. Sharif, M. A. Khan, Z. Iqbal, M. F. Azam, M. I. U. Lali, and
performs existing methods - validated on same dataset. M. Y. Javed, ‘‘Detection and classification of citrus diseases in agriculture
based on optimized weighted segmentation and feature selection,’’ Com-
put. Electron. Agricult., vol. 150, pp. 220–234, Jul. 2018.
VII. CONCLUSION
[9] A. T. Sapkal and U. V. Kulkarni, ‘‘Comparative study of leaf disease
In this article, an optimized automated computer-based diagnosis system using texture features and deep learning features,’’ Int.
method is proposed and validated for recognition of apple J. Appl. Eng. Res., vol. 13, no. 19, pp. 14334–14340, 2018.
diseases. The lesion spot contrast stretching, lesion segmen- [10] A. M. AlShahrani, M. A. Al-Abadi, A. S. Al-Malki, A. S. Ashour, and
N. Dey, ‘‘Automated system for crops recognition and classification,’’
tation, and prominent features selection & recognition steps in Computer Vision: Concepts, Methodologies, Tools, and Applications.
are utilized. The contrast of infected spot is enhanced and Pennsylvania, PA, USA: IGI Global, 2018, pp. 1208–1223.
[11] P. Moghadam, D. Ward, E. Goan, S. Jayawardena, P. Sikka, and [24] S. Zhanga, X. Wu, Z. Youa, and L. Zhang, ‘‘Leaf image based cucumber
E. Hernandez, ‘‘Plant disease detection using hyperspectral imaging,’’ in disease recognition using sparse representation classification,’’ Comput.
Proc. Int. Conf. Digit. Image Comput. Techn. Appl. (DICTA), Dec. 2017, Electron. Agricult., vol. 134, pp. 135–141, Mar. 2017.
pp. 1–8. [25] S. Sergyan ‘‘Color histogram features based image classification in
[12] K. P. Ferentinos, ‘‘Deep learning models for plant disease detection and content-based image retrieval systems,’’ in Proc. 6th Int. Symp. Appl.
diagnosis,’’ Comput. Electron. Agricult., vol. 145, pp. 311–318, Feb. 2018. Mach. Intell. Inform., Jan. 2008, pp. 221–224.
[13] S. R. Dubey and A. S. Jalal, ‘‘Apple disease classification using color, [26] S. kai, G. Cheng, and Y. Jin ‘‘Feature selection for high-dimensional
texture and shape features from images,’’ Signal, Image Video Process., classification using a competitive swarm optimizer,’’ Soft Comput., vol. 22,
vol. 10, no. 5, pp. 819–826, 2016. no. 3, pp. 811–822, 2018.
[14] S. R. Dubey and A. S. Jalal, ‘‘Detection and classification of apple fruit [27] S. Kumara, D. E. Goldberg, and G. Kendall. ‘‘Genetic algorithms,’’ in
diseases using complete local binary patterns,’’ in Proc. 3rd Int. Conf. Search Methodologies. New York, Ny, USA: Springer, 2014, pp. 93–117.
Comput. Commun. Technol., Nov. 2012, pp. 346–351. [28] S. N. Sivanandam, and S. N. Deepa. Introduction to Genetic Algorithms.
[15] E. Omrani, B. Khoshnevisan, S. Shamshirband, H. Saboohi, N. B. Anuar, New York, NY, USA: Springer, 2007.
and M. H. N. M. Nasir, ‘‘Potential of radial basis function-based support [29] A. Lipowski and D. Lipowska, ‘‘Roulette-wheel selection via stochastic
vector regression for apple disease detection,’’ Measurement, vol. 55, acceptance,’’ Phys. A, Stat. Mech. Appl., vol. 391, no. 6, pp. 2193–2196,
pp. 512–519, Sep. 2014. 2012.
[16] M. Shuaibu, W. S. Lee, Y. K. Hong, and S. Kim, ‘‘Detection of apple [30] Y. Liu and Y. F. Zheng, ‘‘One-against-all multi-class SVM classifica-
marssonina blotch disease using particle swarm optimization,’’ Trans. tion using reliability measures,’’ in Proc. IEEE Int. Conf. Neural Netw.,
Amer. Soc. Agricult. Biol. Eng., vol. 60, no. 2, pp. 303–312, 2017. Jul. 2005, pp. 849–854.
[17] V. Singh and A. K. Misra, ‘‘Detection of plant leaf diseases using image [31] G. Wang, Y. Sun, and J. Wang, ‘‘Automatic image-based plant disease
segmentation and soft computing techniques,’’ Inf. Process. Agricult., severity estimation using deep learning,’’ Comput. Intell. Neurosci., vol. 9.
vol. 4, no. 1, pp. 41–49, 2017. no. 4, pp. 2193–2196, 2017.
[18] J. G. A. Barbedo, ‘‘A new automatic method for disease symptom segmen- [32] A. Kamilaris and F. X. Boldú, ‘‘Deep learning in agriculture: A survey,’’
tation in digital photographs of plant leaves,’’ Eur. J. Plant Pathol., vol. 147, Comput. Electron. Agricult., vol. 147, pp. 70–90, Apr. 2018.
no. 2, pp. 349–364, 2017. [33] T. Wiesner-Hanks et al., ‘‘Image set for deep learning: Field images of
[19] H. Ali, M. I. Lali, M. Z. Nawaz, M. Sharif, and B. A. Saleem, ‘‘Symptom maize annotated with disease symptoms,’’ BMC Res. Notes, vol. 11, no. 1,
based automated detection of citrus diseases using color histogram and p. 440, 2018.
textural descriptors,’’ Comput. Electron. Agricult., vol. 138, pp. 92–104, [34] J. Lu, H. Sibai, E. Fabry, and D. Forsyth. (2017). ‘‘No need to worry
Jun. 2017. about adversarial examples in object detection in autonomous vehicles.’’
[20] V. Ugarte, U. Sinha, V. Malis, R. Csapo, and S. Sinha, ‘‘3D multimodal [Online]. Available: https://arxiv.org/abs/1707.03501
spatial fuzzy segmentation of intramuscular connective and adipose tissue [35] Z. Iqbal, M. A. Khan, M. Sharif, J. H. Shah, M. H. Rehman, and K. Javed,
from ultrashort TE MR images of calf muscle,’’ Magn. Reson. Med., ‘‘An automated detection and classification of citrus plant diseases using
vol. 77, no. 2, pp. 870–883, 2017. image processing techniques: A review,’’ Comput. Electron. Agricult.,
[21] C. R. Gonzalez, ‘‘Richard Eugene woods,’’ Digital Image Processing. vol. 153, pp. 12–32, Oct. 2018.
vol. 85, 2007.
[22] Y. Mitani and Y. Hamamoto, ‘‘A consideration of pan-sharpen images by
HSI transformation approach,’’ in Proc. Annu. Conf. SICE, Aug. 2010,
pp. 1283–1284.
[23] O. Timo, M. Pietikäinen, and T. Mäenpää, ‘‘Multiresolution gray-scale and Authors’ photographs and biographies not available at the time of
rotation invariant texture classification with local binary patterns,’’ IEEE publication.
Trans. Pattern Anal. Mach. Intell., vol. 24, no. 7, pp. 971–987, Jul. 2002.