2014 18th International Conference on Information Visualisation
Detecting Mango Fruits by using Randomized Hough Transform and
Backpropagation Neural Network
1
Kutiba Nanaa, 2 Mohamed Rizon, 1 Mohd Nordin Abd Rahman, 3 Yahaya Ibrahim and 1 Azim Zaliha Abd Aziz
1
Faculty of Informatics and Computing,
Faculty of Design and Engineering Technology,
3
Faculty of Applied Social Sciences,
Universiti Sultan Zainal Abidin (UniSZA), Terengganu, Malaysia.
2
methods implemented to extract tomato [2], apple [3] and
eggplant [4].
In a study aimed to estimate the appropriate time for
harvesting olives [5], spectrum data have been analyzed for
each of olives, two kinds of leaves and branches. And then a
model of olive fruit has been built by using Principal
Component Analysis method (PCA). Cross correlation
method has been applied in order to extract olive fruits. In
fact, the use of PCA method uses a set of uncorrelated
variables in order to describe a set of observations.
Backpropagation Neural Network has classified the olive
fruit in order to take a decision of harvest. The inputs of
neural network consist of shape, area, deviation, range, the
average correlation and average values of color while the
unique output of neural network estimated if the time is
appropriate for the harvest. However, the correct
classification ranked up to more than 95%.
In another study hired MATLAB GUI to implement,
aimed to determine the ripeness of mango fruits, color image
has been converted to gray then to binary color image by
applying a threshold determined using histogram
equalization. Watershed method has applied to isolate each
mango fruit into separate image. Then extracting quantity
properties to use as parameters of 140 input neurons in
Backpropagation Neural Network which assess the ripeness
of mango [6] while the output neuron of the neural network
indicates ripe mango (if the output is equal or greater than
0.5) and unripe mango (if output is less than 0.5). However,
it is difficult to extract accurate quantity properties carelessly
shape of object. In our paper we focus on oval shapes to
contribute in mango detection as a substantial step for the
researches which involved in studying maturity, diseases and
quality
AbstractA new method for mango detection is presented
in this paper. This method is based on preprocessing operators
on image which includes converting to gray image, finding
edges, calculating distances to edges, opening morphology and
converting to binary color image. To take advantage of oval
shaped mango fruit, we apply Randomized Hough Transform
method to detect potential places for mango fruit in input
images. By using Backpropagation Neural Network, we
recognize mango fruits from these potential places. The dataset
used to implementing this paper is 50 RGB images captured of
mango fruits on trees. As shown in experimental results, in the
case of clear fruit in input images, the detection rates up to
96.26% while it decreases in the case of partially covering or
overlapping. However, this method can be applied to detect
other fruits in varied sizes and colors.
Keywords--- Detecting Mango, image recognition, feature
extraction, image segmentation, neural network, watershed
algorithm, Randomized Hough Transform, detecting Fruits
I.
INTRODUCTION
Computer vision provided many solutions to reduce the
human effort in the field of agriculture. Researchers have
focused on several topics as finding weeds, searching for
fruits, monitoring the health status of trees, automatic
classification of fruits, determining fruit ripeness, detecting
diseases in trees and fruit, etc. the survey [1] discussed
various researches involved to these topics.
In systems designed to monitor fruits of harvest, it is
necessary to create an effective methods for detecting these
fruits as well as the predictability of kind and quality. These
systems substantially rely on studying fruits shape and
discussing images of fruits on trees by focusing on several
geometric factors such as density, color, edges and rotation
of fruit in addition to the amount of properties such as height,
width, thickness, and the possibility of presence spots on the
fruit.
Practically, extracting fruits from images requires initial
processes in order to determine the initial objects and then
verify whether the objects represent fruit or do not. These
processes isolate areas of fruit from the background
(background contains sky, leaves, branches, ground, noise,
etc.), whether by converting color image to gray or binary
color image by applying a certain threshold, by using certain
morphological conversion such as watershed or by applying
some of color segmentation methods such as in the proposed
1550-6037/14 $31.00 2014 IEEE
DOI 10.1109/IV.2014.54
II.
PROPOSED METHOD
The proposed method is based on three steps. The first
step performs preprocessing and normalizing input image.
Second step aims funding oval shapes image, which may
indicates mango in input image. Final step is verifying each
oval shape whether represents mango or does not by using
neural network.
A. Prepocessing Input imags
In this step, we perform some necessary operators in
order to simply detecting ellipses in the input image. Firstly,
388
we convert color of the input image of size to gray
image of same size. Usually, getting gray image from
RGB images is done by converting RGB color space to
YCrCb color space and considering Y values as gray values
of RGB image. In this paper we did not rely on YCrCb to get
grey values. Instead, we consider green values of RGB
represent gray values inasmuch as the dominant color in the
input image is green. However, Equation 1 shows creating
the gray image from the input image.
. Each image represents only one region. For each
image we apply canny edge detector operator to verify if it
represents an ellipse.
(1)
In order to detect edges in , the operator of canny edge
detector has been applied. In fact, canny operator uses four
filters to detect horizontal, vertical and two diagonal edges.
As observed from the image produced by applying canny
operator process , edges in leaves region are abundant
comparing to edges in mango region due to overlapping
leaves. . To delete leaves region from discussion, we apply a
function ( ) on each pixel to calculate its
distance to edges. This function compounds of eight
parameters which represent the distance to edges on the
vertical axis, the horizontal axis and two diagonal axes, as in
Equation 2 and Equation 3.
(2)
Figure 1. The generated images during preprocessing input image.
(3)
Figure 1 shows implementing this step on the input image
and generates the images and . By
applying watershed method on , there are 15 isolated
images of single potential mango region have
been generated. Figure 1 shows two images and .
B. Finding Oval Shapes
To take advantage of the elliptical shaped of mango fruit;
in this step we seek to find elliptical shape in the isolated
image . However, Equation 4 represents the
equation of ellipse.
After finding the matrix which represents the distance to
edges for each pixel, we normalize the values of the matrix
to locate in the range [0, 255] to form the image. Then we
convert the image into binary color image by
considering threshold equals 128. Open morphological
operator is applied a certain number of times (which involve
dilation of erosion) to remove small patches (which involve
the noise) and produce the image .
White regions in the image reflect the potential
regions for mango fruits. To study each area separately we
use watershed method which generates multiple images
(4)
The requirement to implementing this step is to find
values of the parameters and . The variables and
are the coordinates of edges in isolated images.
Randomized Hough Transform [7] has been proposed to find
these parameters and estimate the rotation degree of ellipse.
Furthermore, MATLAB files are provided to implement
(findellipseParams.m [8] and fit_ellipse.m [9]).
To verifying the corresponding between the produced
ellipse and the object in the isolated image, we comparing
389
training patterns are taken from 20% of mango image
dataset.
their areas according to Equation 5 and ignore the images
which do not attain the corresponding ratio of 0.85. In figure
2 shows produced ellipses (bordered in red) for objects (in
white). we notice that the images in left column are accepted
as oval shapes and the images in right column are rejected.
III.
EXPERIMENTAL RESULTS
The used Dataset consists of 50 images taken by digital
camera with a resolution of 12 mega pixels. All images have
been converted to size of 430 * 320. These images contain
187 mango fruits. By accessing to the images, 126 mango
fruit appear fully, 28 mango fruit appear partially and 33
mango fruits appear as overlapped by other mango fruit.
After applying the pro-processing and Hough Transform
method which discussed in previous section, there are 256
candidate oval shapes may represent mango fruits. Actually,
only 184 of mango fruits were embedded within candidate
oval shapes. In other word, only 71.88% of candidate shapes
represent mango fruit. Wherefore, that it is necessary to
employ Neural Networks in order to classify mango fruits
from candidate oval shapes.
On the other hand, some parts of mango fruits do not
enough appear, that produces ignoring the fruit in addition to
some overlapped fruits has been detected as only one fruit.
.
(5)
Figure 2. Fitting ellipse on object shape.
C. Nueral Network
Neural Network is used to find a non liner correlation
between
Non-descriptive
elements.
We
used
Backpropagation Neural Network to implement our
proposed method which learn through training patterns
composed of input patterns associated with appreciate output
patterns.
Normalizing input patterns is essential process before
forwarding into neural network. Normalizing has been done
by cropping the ellipse and rotating in order to locate
vertically. Angle of rotation is estimated by Randomized
Hough Transform method. By using affine method, detected
oval shape resized into image of size 3015.
Colors of the pixels which locate inside the ellipse
correspond to its color in the gray image while colors of
the pixels which locate outside the ellipse convert to black
(the value equals 0).
Three layers form the proposed neural network. Input
layer has 450 neurons. Inputs are color values of the pixels
included in cropped image after converting the value range
from [0,255] into [-1, 1]. Only one hidden layer is used. In
general, use of hidden layer can approximate non-linear
separable decision as in our paper. Hidden layer has 50
neurons. Output layer has a unique neuron which indicates to
corresponding the input ellipse to mango when the output
greater than 0.5 otherwise it indicates to not corresponding.
The activation function used is binary sigmoid and the
Figure 3. Sample of experimental results.
In order to train the neural network, we considered 10
images as training set. We associate each cropped image
with the value (1) when the cropped ellipse image represents
mango fruit otherwise we associate it with the value (-1).
After training, the neural network classified 185 cropped
images as mango image. In fact, only 180 results of
classification are correct. There are five incorrect results and
failing to classify seven pattern of mango image. Generally,
our proposed method detects 96.26% of mango in database.
The ratio of incorrect to correction detection is 2.78%.
390
Figure .3 show samples of detection results. Comparing to
previous studies, detecting olives [5] reported as 95% of
detecting rate and detecting ripped mango [6] reported as
90% of detecting rate. Furthermore, we used 20% of dataset
as training set to implement the experiences while the
training set in [6] was 70% of database
IV.
REFERENCES
[1]
[2]
CONCLUSIONS
[3]
This paper presents a new method for detecting mango
fruits from images. The main contribution is to employ the
elliptical mango shape in detection method. method is based
on preprocessing operators on image includes converting to
gray image, finding edges, calculating distances to edges,
opening morphology and converting to binary color image.
To take advantage of the elliptical shape of mango fruit, we
employ Randomized Hough Transform to find oval shapes in
input image. Backpropagation Neural Network is used to
classify the mango fruit from the proposed oval shapes.
Three layers form the proposed neural network. Input layer
consist of 450 neurons used to forward values of the cropped
oval shape image, One hidden layer includes 50 neurons, and
output layer include a unique neuron classify the input
cropped image as a mango image in the case of the output
greater than 0.5 . Experimental results show that mango
detection rate up to 96.26% in the case of clear appearance of
mango while it reported as 90% in the study involved
detecting ripped mango. However the detection rate is
reduced in the case of overlapping mango or patricianly
covered.
[4]
[5]
[6]
[7]
[8]
[9]
391
A. Vibhute and S. K. Bodhe. Applications of Image Processing in
Agriculture: A Survey, International Journal of Computer
Applications 52, 2012.
A. Gastlum-Barrios, R. A. Brquez-Lpez, E. Rico-Garca, M.
Toledano-Ayala and G. M. Soto-Zaraza, Tomato quality evaluation
with image processing: A review, African Journal of Agricultural
Research, vol. 6(14), pp. 3333-3339, 2011.
Z. Xiao-bo, Z. Jie-wen, L. Yanxiaoa and M. Holmes, In-line
detection of apple defects using three color cameras system,
Computers and Electronics in Agriculture, vol. 70, pp.129134, 2010.
D. Unay and B. Gosselin, Artificial neural network-based
segmentation and apple grading by machine vision, In Proc. IEEE
Int. Conference on Image Processing, vol. 2, [Link]-630, 2005.
G., Gabriel, S. Best, J. Ceroni and G. Lefranca, Olive Fruits
Recognition Using Neural Networks, Procedia Computer Science,
vol. 17, pp. 412-419, 2013.
M. Chhabra, A. Cupta, P. Mehrotra and S. Reel, Automated
Detection of Fully and Partially Riped Mango by Machine Vision,
IJCA Proceedings on International Conference on Recent Trends in
Information Technology and Computer Science, pp. 25-31, 2012.
[Link], S, Ellipse detection using randomized Hough transform,
[Link]/res/vision/[Link], May 2002.
[Link] 2002.
[Link] 2007.
