Automatic
Department of Electrical Engineering
Sameen Anas
Graduate School of Science and Engineering
Karachi Institute of Economics and Technology
Karachi, Pakistan
Sameen.arif32@gmail.com
Syed Muhammad G Monir
Abstract—This paper presents an efficient method for
automatic grading of date fruit based on computer vision. Color
images of date fruits are used to extract size, shape and wrinkle
feature for classification. Prior to classification, a support vector
machine (SVM) based image segmentation technique is used for
efficient removal of background pixels and shadows. Size and
wrinkles are classified on basis of mean-variance relationship and
threshold value method respectively, whereas backpropagation
and radial basis function (RBF) neural networks are used for
shape classification giving accuracy of 77% and 96.3%
respectively.
Keywords—back propagation; Fruit classification; image
segmentation; pattern recognition; radial basis function; support
vector machine.
I. INTRODUCTION
Date fruit is one of the most consumed fruit in South Asia
and Middle-East. Saudi Arabia alone produces around 400
types of dates. Different types of dates have different features
which are difficult to classify by humans [1].
Manual grading and sorting of fruits is a repetitive, labor
intensive and time consuming process and it is carried out by
humans manually through visual inspection. Also there has to
be standardization in date marketing. The dates are graded on
basis of their size, shape, color and texture. All these feature
are difficult to be caught my human eye. It is hard to classify
dates that are in similar colors yet in different types. However,
due to the different perceptions of the employees,
classification mistakes were made and it leads to
non-consistent and non-uniform results.
In short the demerits of manual classification are that it is
highly expensive, labors demand higher pay to inspect dates.
By mishandling, damage can occur. According to different
perception of human labors, same date fruit can be classified
as of different size and shape etc. this decreases accuracy and
leads to non standardization. There must be an automated
Muhammad Ali Jinnah University
Karachi, Pakistan
monir@jinnah.edu
grading system which is robust enough to classify dates on
basis of their individual features and sorts the dates with
minimum error, takes less time and is consistent. Image
processing along with neural networks is the emerging
technique that functions like a human brain and gives very
satisfying results.
In this paper, an automatic date sorter algorithm is
presented using digital image processing tools and neural
networks for successful classification. Sorting is done on the
basis of size, shape and wrinkles. A comparison is carried out
between back propagation and RBF classifier. Also an
efficient method for image segmentation is implemented. The
paper is organized as follows:
A literature review is presented briefly on the related work
in agriculture industry for fruit grading based on external
quality factors in section 2. In section 3, the proposed
methodology is discussed containing detailed explanation of
image processing and pattern recognition tools used to build
the date sorting algorithm along with classification methods
for each feature (color, size, shape, wrinkles). Classification
methods are discussed in section 4, followed by
implementation of these methods in section 5. Section 6
provides the results of experiments performed. Finally
conclusions are drawn is section 7.
II. LITERATURE REVIEW
Presently, insufficient research work has been carried out
regarding date sorting. Extensive research is found on fruit
sorting based on their external features such as color, texture,
size and shape. These features happen to be a benchmark for
visual inspection of fruits. The present research has been done
of apples, eggplants, oranges and leaf classification [1].
In 2014 Mohana and Prabhakar graded dates on basis of
shape and texture. They used contour based shape feature via
local binary pattern and curvelet transform. k-NN neighbor
classifier with the grading accuracy of 96.45% was used [2]. A Ohali’s results are based on only 1st coefficient of Fourier
paper written dedicatedly on shape recognition compares transform and his results of 80% accuracy cannot be achieved
different shape signatures and finds that among complex with this little information of shape profile.
coordinates, centroid distance and curvature signature,
The proposed method overcomes this shortcoming by
centroid distance is the significantly better than other shape
taking more than one coefficient of Fourier transform. The 1st
signatures and it gives better results [3]. D.J Lee et al also
value is infact totally neglected and further coefficients are
proposed date size sorting algorithm using connected
used to represent shape feature vector. Experimentally it is
component analysis [4]. D. Zhang et al used short wave
found that 16 coefficients are enough. Less than 16 causes
infrared imaging to classify. Ellipse fitting is used on
inaccurate results and more than 16 causes extra time.
segmented fruit image and fruit size is measures based on
major axis length. The training stage includes the gray-scale Also Ohali incorporated the values of size and wrinkle
histogram normalization and fruit skin threshold for each prior ratio in the input feature vector of BPNN along with one
quality class. Normalized histogram of the test fruit is Fourier coefficient (shape). Whereas the classification on basis
compared with the four predefined classes. To separate of size and number of wrinkles has already been made by
delaminated skin from normal fruit, skin threshold is using mean average relationship and using threshold in his
calculated according to histogram. For grade determination, paper. In proposed method, grading on basis of size and
four levels of delamination are compared based on the wrinkles is done using mean-variance relationship and
delamination percentage. In training stage, 40 date samples are threshold value method respectively. Whereas for shape
used for training. Total 1200 date samples are used for testing. classification RBF is used whose success rate is far better than
The accuracy of their approach was 95% for jumbo and extra BPNN. Fig.1 and fig. 2 show the overall methodology adopted
fancy classes and similarly, for fancy and confection classes to classify dates.
98% accuracy has been achieved [5].
The proposed method uses a better approach for image
A study on external quality features is carried out by segmentation using support vector machine (SVM). Image
Yousef Al Ohali in 2011. Their research focuses on shape, size segmentation is performed by Otsu’s method of thresholding
and texture of dates The system used total 1860 dates which along with SVM. The grayscale image obtained by svm
includes 3 different grades of date fruits such that each grade training of foreground and background is used in Otsu’s
consist of 620 samples. Simple threshold is used in method. The threshold value of svm grayscale image is used to
segmentation. Segmentation is followed by edge detection binarize the image.
using sobel operator. The external features of date fruit are
extracted and then classified by back propagation neural
network. The classification accuracy of 80% was achieved.
For shape feature detection they used centroid distance
formula whose 1st Fourier coefficient is used as the
irregularity measure. For size the area covered by date is
found and wrinkles are found by sobel edge detection
algorithm [1].
III. PROPOSED METHOD
Automatic date sorter algorithm is presented using
computer vision tools and neural networks for successful
classification. A paper written in 2011 by Yousef Al Ohali is
used as a benchmark and a better approach is presented
dealing with the external quality features such as size,
wrinkles and shape. A comparison is carried out between his
work and proposed solution. The areas in which improvements
are done are discussed in following paragraphs.
For shape classification Ohali used centroid distance
formula to find boundary profile of dates and took Fourier
transform of centroid distance. Only 1st coefficient of Fourier
transform is used as the irregularity measure to classify shape.
This is a wrong approach because the first coefficient is the
average value of shape profile and it doesn’t give detailed
shape information. In order to have a detailed and accurate
shape profile of date, more than one coefficients are necessary.

A. Image Acquisition and Database
A 16 mega pixel digital camera is used to acquire date
images. Samples of different types of date fruit variety were
selected for the experiment. All date fruit samples were
individually numbered totaling 270 images. Half of the images
are used for database building purpose and the other half for
testing purpose.
A. Preprocessing
Preprocessing stage comprises of image input to the
system, its segmentation followed by binarization. Finally the
date fruit boundary and wrinkles is found using sobel edge
operator.
1) SVM Based Segmentation:
The first and most important step in image processing is
image segmentation. There are many methods of image
segmentation but they fail to give accurate outcomes if
lighting conditions in image are variable, or if image has
shadows in it. To avoid the issues of non uniform lighting
conditions and shadows in captured images, this paper uses an
automatic adjustable algorithm for segmentation of color
images using linear support vector machine (SVM) and Otsu’s
thresholding method [6]. The segmentation algorithm used
consists of three main steps:
● The optimal linear separation hyperplane in the 3-D
RGB space is calculated by using linear SVM
● A contrast enhanced grayscale image is calculated
from the linear hyperplane
● The optimal threshold around the fruit boundary is
automatically estimated using Otsu’s method [6].
The figure below shows segmented date by using simple
Otsu’s method and then by using Otsu along with SVM. The
proposed SVM based segmentation segments image without
any shadow. Whereas simple basic Otsu based segmentation
yields results with shadow o date and variable lighting in
corner too.
B. Feature Definition and Extraction
The external quality factors of dates are referred to as
features. These features are size, shape and wrinkles. The
properties, usefulness and extraction mechanism of these
features are as follows:
1) Size:
The fruit size is one of the main quality attribute used
by farmers. The bigger size fruit is considered of better
quality. The size is estimated by calculating the area covered
by the fruit image. To compute the area, first the fruit image is
binarized to separate the fruit image from its background. The
number of pixels that cover the fruit image is counted and
considered as an estimate of size [1].
2) Wrinkles:
The Better quality dates have more number of wrinkles
present on their skin. . E.g. Ajwa date has more wrinkles
whereas Irani date has no or very less wrinkles on its skin.
Wrinkles are extracted by using sobel edge detection
algorithm which finds the number of edges on each date. This
equates to the total number of wrinkles present of fruit skin.
Total numbers of pixels which body the edges are counted. A
wrinkle ratio between edge pixel count and total fruit area
pixel count is calculated by using eq.1. Wrinkle ratio gives the
percentage of area covered by wrinkles in entire date fruit.
𝐴𝑟𝑒𝑎 𝐶𝑜𝑣𝑒𝑟𝑒𝑑 𝑏𝑦 𝐸𝑑𝑔𝑒𝑠
𝑊𝑅 = 𝑇𝑜𝑡𝑎𝑙 𝐴𝑟𝑒𝑎 𝑜𝑓 𝐹𝑟𝑢𝑖𝑡
(1)
 3) Shape:

The shape of date fruit is also one of the main feature of

quality assessment. Fruits having irregular shapes are
considered of better quality. The method used to find shape of
date fruit is to find the boundary profile of date fruit. A
comparative study on robustness, computational complexity,
and convergence speed and retrieval performance showed that
boundary-to-centroid distance function is the best shape
signature [7]. So this research uses centroid distance as a
shape signature. The process leading to shape feature
extraction can be divided in the following stages:
Image capturing, segmentation or binarization, finding
shape signature, finding fourier descriptors of shape signature
which constitutes the shape feature vector.
The estimation steps are described below.
● Use boundary tracing algorithm to find the outermost
boundary pixels of date fruit image. The boundary
tracing algorithm is applied to the binarized fruit
image, in which fruit is represented by white pixels
and background by black pixels.
● Link the outermost edge points to form the outermost \
profile of the fruit image. II. CLASSIFICATION
● Compute the centroid (Xc, Yc) of the boundary I first visually examined the fruits that are used in this
profile. experiment and graded them manually according to their
● Find centroid distance by using the formula: features for size and wrinkles 3 grades are assigned, whereas
or shape two grades are assigned. Date size is categorized as
2 2 small, medium and large. Wrinkles are categorized as less,
𝑟(𝑖) = (𝑋𝑖 − 𝑋𝑐) + (𝑌𝑖 − 𝑌𝑐) (2)
medium and more. Shape is categorized as regular and
Where i=1,2,3… N, N=total number of points in the irregular. The fruits having regular shape, large size, high
boundary profile, Xi and Yi are the Cartesian percentage of wrinkle ratio are branded as of the best quality.
coordinates at particular boundary pixel. A. Size Classification
● Compute the Fourier coefficients of r(i). Date is categorized as big, medium and small using the
𝑁 −2π𝑛𝑖
average area and variance relationship.
𝑎𝑛 = ∑ 𝑟(𝑖)𝑒 𝑁
(3) 2
𝑖=1
𝐴∓𝑘σ (4)
2
Experiment in this study showed that considering only first Where 𝐴 is average area of the training samples σ is
16 coefficients provide good estimation of shape, truncate the variance of the training samples and 𝑘 is the extent of spread
Fourier series to 16 coefficients to make classification fast. of data [1]. The value of k depends on user as how a size of
These Fourier coefficients are called Fourier descriptors and date is perceived. Following table defines the criteria of date
are used as an input to the neural network classifier for shape size classification for k=1.45. A is pixel count of test fruit
classification.
TABLE I. CRITERIA OF DATE SIZE CLASSIFICATION
The following two figures show the centroid distance
curves of two types of dates. The fist figure show curve for Size Classification Decision Rules
date belonging to class 1(regular) and the other figure shows Criteria Grade
class 2 (irregular). The shape signature of regular date is 𝐴 < 𝐴 − 𝑘σ
2
Small
smooth as compared to irregular date. 2 2
𝐴 − 𝑘σ 𝐴 < 𝐴 + 𝑘σ Medium
2 2
𝐴 − 𝑘σ 𝐴 > 𝐴 + 𝑘σ Big
 This type of classifier is simple to implement, accurate and
is less memory intensive.
B. Wrinkle Classification
For wrinkles the classification is simply carried out by
finding wrinkle ratio which is the percentage of pixels covered
by wrinkles to the total pixels of date. A simple threshold is
set to classify date having more or less wrinkles. If the value
of wrinkle ratio is beyond the predefined threshold, that date
has more wrinkles. Whereas if the ratio is less than threshold
value, the date has less wrinkles. The threshold value is user
defined and depends on the perception of the person designing
the system
C. Shape Classification
For shape classification, back propagation neural network
(BPNN) used by Ohali is implemented and compared with the
proposed RBF neural network.
1) Back Propogation Neural Network
The back propagation neural network has an input
layer, minimum of one hidden layer and an output layer. A
training input pattern is presented to the network input layer.
The network propagates the input pattern from layer to layer
until output pattern is generated by the output layer. If this
pattern is different from the desired output, an error is
calculated and then propagated backward through the network
from the output layer to the input layer. The weights are
modified as the error is propagated.
2) Radial Base Neural Network
The RBF neural network uses radial basis
functions as activation functions. It’s output is a linear
combination of radial basis functions of the inputs and neuron
parameters. RBF has an input layer, a hidden layer and an
output layer. Gaussian transfer function is used as activation
function in hidden layer. The output of Gaussian transfer
function is inversely proportional to the distance from the
center of the neuron [8].
Classification decision is performed by measuring the
input’s similarity to examples from the training set. Neurons
compute the Euclidean distance between the input and its
prototype and if both are equal, then the output of that RBF
neuron will be 1. Whereas if there is vast difference between
input and prototype from training dataset, the neuron’s output
is close to zero..  The shape of the RBF neuron’s response is a
bell curve [8].
In RBF network shown in fig. 5, output Zj of hidden
neurons is obtained by calculating the closeness of the
input x to an n-dimensional parameter vector j associated with
the jth hidden unit. Here, the response characteristics of the jth
hidden unit are given by eq. 5 [8]:
∥𝑋−μ𝑗∥
𝑍𝑗(𝑥) = 𝐾( 2 )
σ
(5)
Where K is a positive radially-symmetric function with
maximum value at its "center" j and which drops off rapidly to
zero away from the center. The parameter j is the "width" of
the receptive field in the input space for unit j. zj has greater
value only when the "distance" ∥𝑋 − μ𝑗∥ is smaller than the
width j. the output of the RBF network to input x is
the L-dimensional activity vector y whose lth component is
given by eq.6 [8].
𝐽
𝑦𝑙(𝑥) = ∑ 𝑤𝑙𝑗𝑍𝑗(𝑥) (6)
𝑗=1
The approach used here is as follows:
● Input matrix and target class are input to the system
● The input vector with the greatest error is found. At
each iteration the input vector which will result in
lowering the network error the most, is used to create
a radbas neuron.
● A radial basis network is iteratively created one
neuron at a time. A radbas neuron is added with
weights equal to that vector.
● Neurons are added to the network until the
sum-squared error falls beneath an error threshold
● The error of the new network is checked, and if low
enough classification is finished. Otherwise the next
neuron is added. This procedure is repeated until the
error goal is met.
III. IMPLEMENTATION
To classify dates on basis of shape first back propagation is
implemented which is used in Ohali’s paper. After that, radial
basis function neural network is implemented to compare both
neural networks
During the training phase, the network is given sample
inputs and the target classes. An input vector of size 15 by 135
is input to backpropogation neural network. 15 denote the 1st
16 Fourier coefficients of shape minus the dc component. And
total 135 dates are used for training. Input layer has total 15
neurons representing 15 Fourier coefficient of date. The input
feature vector is transferred to hidden layer via input layer.
Hidden layer has 10 neurons. The activation function used in
hidden layer is tan sigmoid. The output layer has 2 neurons
representing class 1 as regular and class 2 as irregular. Batch
training is used.
2) Radial Base Function Network
In RBF, input layer takes one date at a time having 15
Fourier coefficients. The hidden layer has total 135 neurons A confusion matrix of RBF classifier is shown in fig.7.
(equal to no. of dates). Hidden layer uses Gaussian as First two diagonal cells show the number and percentage of
activation function. Output layer has 2 neurons denoting class correct classifications by the trained network. 72 dates are
1 and class 2. correctly classified as class 1 (regular). This corresponds to
53.3% of all 135 dates. Similarly, 58 dates are correctly
IV. EXPERIMENTS AND RESULTS classified as class 2(irregular). This corresponds to 43% of all
dates. 2 dates of class 2(irregular) are incorrectly classified as
Both BPNN and RBF are trained on 135 date samples (75 class 1(regular) and this corresponds to 1.5% of all 135 dates
samples of class 1, 60 samples of class 2). The testing was in the data. Similarly, 3 dates of class1 (regular) are incorrectly
done of 135 samples. Experiment in this study showed that classified as class 2(irregular) and this corresponds to 2.2% of
considering only first 16 coefficients provide good estimation all data. Out of 74 class 1 (regular) predictions, 97.3% are
of shape. The confusion matrix and results of both neural correct and 2.7% are wrong. Out of 61 class 2(irregular)
networks is shown below: predictions, 95.1 % are correct and 4.9% are wrong. Out of 75
class 1 dates (regular), 96% are correctly predicted as class 1
1) Back Propogation Neural Network
and 4 % are predicted as class2. Out of 60 class 2(irregular)
A confusion matrix of BPNN classifier is shown in fig.6. dates, 96.7% are correctly classified as class 2 and 3.7% are
First two diagonal cells show the number and percentage of classified as class 1.
correct classifications by the trained network. 67 dates are
Overall, 77% of the predictions are correct and 23% are wrong
correctly classified as class 1 (regular). This corresponds to
classifications. Results of both BPNN and RBF classifiers are
49.6% of all 135 dates. Similarly, 37 dates are correctly
stated in table 2.
classified as class 2(irregular). This corresponds to 27.4% of
all dates. 23 dates of class 2(irregular) are incorrectly
classified as class 1(regular) and this corresponds to 17% of
all 135 dates in the data. Similarly, 8 dates of class1 (regular)
are incorrectly classified as class 2(irregular) and this
corresponds to 5.9% of all data. Out of 90 class 1 (regular)
predictions, 74.4% are correct and 25.6% are wrong. Out of 45
class 2(irregular) predictions, 82.2 % are correct and 17.8%
are wrong. Out of 75 class 1 dates (regular), 89.3% are
correctly predicted as class 1 and 10.7 % are predicted as
class2. Out of 60 class 2(irregular) dates, 61.7% are correctly
classified as class 2 and 38.3% are classified as class 1.
Overall, 96.3% of the predictions are correct and 3.7% are
wrong classifications.

TABLE II. RESULTS OF SHAPE CLASSIFICATION USING BPNN AND RBF

Shape Classification Accuracy Using BPNN and RBF

Classifier Success % Error Correctly Misclassifie
 Rate Classified d Samples
Samples [1] Yousef Al Ohali, “Computer vision based date fruit grading system:
BPNN 77% 23% 104 31 Design and implementation,” Journal of King Saud University -
Computer and Information Sciences,vol. 23,pp. 29-36, 1//2011.
RBF 96.3% 3.7% 130 5
[2] Mohana S.H. and Prabhakar C.J., “a novel technique for grading of dates
using shape and texture features,” An International Journal (MLAIJ)
Vol.1, No.2, December 2014.
V. CONCLUSION [3] Dengsheng Zhang and Guojun Lu, “A Comparative Study of Fourier
Descriptors for Shape Representation and Retrieval,” The 5th Asian
In this paper size, shape and wrinkle based grading of Conference on Computer Vision, 23—25 January 2002, Melbourne,
dates is performed using image processing and neural Australia.
network. A better method of image segmentation using SVM [4] Dah-Jye Lee, Robert Schoenberger, James Archibald and Steve
and shape classification is proposed and results are compared McCollum, “Development of a machine vision system for automatic
with a benchmark paper written by Yousef Al Ohali. SVM date grading using digital reflective near-infrared imaging,” Journal of
Food Engineering, October 2007.
based segmentation gives accurate results by eliminating
[5] Dong Zhang, Dah-Jye Lee, Beau J.Tippetts and Kirt D. Lillywhite,
shadows in images. For shape classification it is proved that “Date quality evaluation using short-wave infrared imaging,” Journal of
first 16 Fourier descriptors If used as shape signature gives Food Engineering, Volume 141, November 2014, pp 74–84
better accuracy as compared to using only first fourier [6] Akira Mizushima and Renfu Lu, “An image segmentation method for
descriptor used in benchmark paper. Further, a comparison is apple sorting and grading using support vector machine and Otsu's
made between BPNN and RBF showing RBF out method,” Journal Computers and Electronics in Agriculture, Vol. 94, pp.
outnumbering correctly classified sample and showing 29-37, June 2013.
accuracy of 96.3 %. Results obtained overall of all features are [7] H.K. Mebatsion, Frédéric Boudon, Christophe Godin and Nadia Bertin,
“A novel profile based model for virtual representation of
quite satisfactory.
quasi-symmetric plant organs,” Computers and Electronics in
Agriculture ,vol. 75(1), pp. 113-124 ,January 2011.
[8] Mohamad H. Hassoun, Fundamentals of Artificial Neural Networks, A
REFERENCES Bradford book, MIT press, Cambridge, Massacheutts.

[1] Y. Al Ohali, "Computer vision based date fruit grading system: Design and implementation," Journal of King Saud
University - Computer and Information Sciences, vol. 23, pp. 29-36, 1// 2011.
[2] M. S. H. a. P. C.J, "A Novel Technique for Grading of Dates using Shape and Texture Features," Machine Learning and
Applications: An International Journal (MLAIJ), vol. Vol.1, p. 15, December 2014 2014.
[3] G. L. Dengsheng Zhang "A comparative Study of Fourier Descriptors for Shape Representation and Retrieval "
presented at the The 5th Asian Conference on Computer Vision, Melbourne, Australia, 2002.
[4] D.-J. Lee, R. Schoenberger, J. Archibald, and S. McCollum, "Development of a machine vision system for automatic
date grading using digital reflective near-infrared imaging," Journal of Food Engineering, vol. 86, pp. 388-398, 6// 2008.
[5] D. Zhang, D.-J. Lee, B. J. Tippetts, and K. D. Lillywhite, "Date quality evaluation using short-wave infrared imaging,"
Journal of Food Engineering, vol. 141, pp. 74-84, 11// 2014.
[6] A. Mizushima and R. Lu, "An image segmentation method for apple sorting and grading using support vector machine
and Otsu's method," Comput. Electron. Agric., vol. 94, pp. 29-37, 2013.
[7] H. K. Mebatsion, F. Boudon, C. Godin, C. Pradal, M. Génard, C. Goz-Bac, et al., "A novel profile based model for
virtual representation of quasi-symmetric plant organs," Computers and Electronics in Agriculture, vol. 75, pp. 113-124,
1// 2011.
[8] M. Hassoun. (1995). Fundamentals of Artificial Neural Networks (MIT Press)