An Approach towards Classification of Fruits and

Vegetables using Fractal Analysis

Susovan Jana1, Ranjan Parekh2, and Bijan Sarkar1

1
Department of Production Engineering, Jadavpur University, Kolkata, India
2
School of Education Technology, Jadavpur University, Kolkata, India
jana.susovan2@gmail.com, rparekh@school.jdvu.ac.in,
bijan.sarkar@jadavpuruniversity.in

Abstract. Agriculture-related works include harvesting, sorting, and packaging

etc. Those works require lots of time and a huge number of expert resources for
manual execution. Automation may be the solution to this problem. There are
lots of challenges for the automation of those works with the help of image pro-
cessing. One of the major challenges is the identification of fruit and vegetable
class accurately from various viewing positions. In this paper, a viewpoint in-
dependent solution is proposed for fruit and vegetable classification. Firstly, in-
put RGB color image is converted to a grayscale image. Multiple threshold val-
ues are calculated from the grayscale image using multi-level thresholding
technique. Then a set of the binary images is generated using those threshold
values. In the next step, the border image is extracted from each of the binary
images. Finally, the fractal dimension is computed from the border image and
used to classify the fruit and vegetable. The proposed method was tested on a
dataset of 1080 images, which contains 15 classes of fruits and vegetables.
Complete 360° viewing positions are considered for experimentation. The range
of overall system accuracy is 97.78% to 100% using k-NN classifier.

Keywords: Viewpoint Independent, Multi-level Thresholding, Fractal Dimen-

sion, Classification.

1 Introduction

Computer vision is one of the superior technologies for automating the process in
various industries. Agriculture is one of the major application areas of this technolo-
gy. Automation is necessary for every aspect of planting, treatment, pest control, har-
vesting, sorting and packing. The reason behind this automation is not only to make
the process faster but also reducing the human effort in repetitive works. Among the
agricultural products, fruits and vegetables are necessary for everyday life. In recent
days there is a trend of buying fresh fruit and vegetables from a supermarket in India
as well as in other developing countries. There are challenges to identify a fruit and
vegetable class in supermarkets. Barcode or Catalogue based approach partially
solves the problem, but barcode-based tracking is not possible when the number of
fruit and vegetable is large. Searching preferred item from the large catalog is time-

adfa, p. 1, 2011.
© Springer-Verlag Berlin Heidelberg 2011
consuming. The image-based solution may be effective to address that problem. In
this universe, there are lots of species of fruit and vegetable [1]. All the species are
not consumable. Among the consumable fruits and vegetables though there are varia-
tions of taste but lots of similarity in outlook. Categorizing based on an image feature
is very challenging. There are many existing works for automating this task based on
image analysis. Fruit and vegetable color, shape, size and texture [2] features were
used to recognize the object. Fruit and vegetable image may be captured from various
positions in real life scenario. This makes the classification task more critical. The
motivation of this work is to propose a system framework, which will be able to clas-
sify specific fruit and vegetable though the image is captured from various positions.
In section 2, a literature review is done on image-based fruit and vegetable classifi-
cation. Section 3 illustrates the proposed method in details with equations and dia-
grams. Section 4 brings the experimentations and results of this work. A comparative
analysis is introduced in section 5 followed by the conclusion and future scope of
research in section 6.

2 Previous Works

It is needed to classify the class of fruit and vegetable accurately for every aspect of
agricultural works. There are some previous works of classification among different
fruit classes based on visual features like color, texture, shape, and size. A fruit and
vegetable classification method was proposed using CENTRIST and color feature [3].
Census transformed histogram and hue, saturation color histogram were extracted and
used as a classification feature after normalization. Rachmawati et al. also proposed
an RGB color histogram quantization [4] technique to classify 32 classes of fruit. In a
texture based approach [5], Improved Sum and Difference Histogram (ISADH) fea-
ture was used for recognition of fruit from a color image. Multiple region-based fea-
tures were used for classification of fruits in a shape based approach [6]. This ap-
proach is geometrical transformation invariant. A combined features based approach
[7] was proposed for fruit classification using color and shape features. Color means
for each of RGB component, area, perimeter, and roundness of the object were com-
bined in the feature vector. A nearly similar mixed approach was proposed combining
color, shape, and texture based features [8]. Zawbaa et al. [9] proposed two sets of
classification features to classify 3 fruit class. The first set combined color (variance,
mean, kurtosis, and skewness) and shape (eccentricity, centroid, and Euler number)
features. The second set used only SIFT feature. In a work [10], 64 color feature, 7
texture feature and 8 shape features were extracted from fruit image. Then PCA was
used to reduce the dimension of the feature vector. The classification of the subtypes
of particular fruit and vegetable class is also a tough job. Various shape features were
adopted to predict subtypes [11] using different classification algorithm.
It is observed that most of the previous work was tested on less number of fruit and
vegetable classes. All the approaches considered only one viewing position but result
degrades when multiple viewing positions are introduced. This is one of the most
difficult obstacles to recognize fruit and vegetable properly from an image. In this
paper, a new classification approach is proposed to address the problem of viewing
position change.

3 Proposed Method

This approach adopted few steps to normalize the effect of viewpoint in classification
as well as selects a feature which is able to discriminate among multiple classes of
fruit and vegetables irrespective of the viewing positions.

3.1 System Overview

Fig. 1 shows the flow of the different process in the proposed system. Input image (!)
is converted to grayscale image (!" ). Multilevel Otsu thresholding is applied on !" to
get a set (#), which contains number of threshold values same as the number of
threshold level$(%& ). The graysacle image is binarized (!'* ) using the threshold val-
ues. The border image (!' ) is extracted from the binary images (!'* ). The fractal di-
mension (+) is computed from every border image and stored as feature vector (,-)
in database. Finally, the class label (.) prediction has been done by the trained classi-
fier using$,-.

Fig. 1. Process flow diagram of the proposed system

3.2 Conversion to Grayscale

RGB input image (!) is split into three components i.e. IR, IG, IB. A grayscale image
(!" ) is generated using these components and Eqn (1).

!" = 0/2989 × I1 3 0/5870 × I4 3 0/66:0 × I; (1)

3.3 Multi-level Threshold
Multilevel Otsu thresholding technique [12] is applied on the grayscale image. It’s a
very fast algorithm in terms of processing speed and requires low storage space. As-
suming, the grayscale image (!" ) contains < number of pixels with gray level from 1
to L. Total count of the pixel for gray level . is denoted by >? . The probability of gray
level . in the image is @? . Refer to Eqn. (2).
AB
@? = $ (2)
C

The number of threshold level (%& ) is an input to the system. Output is a set # of %&
threshold values${#D E $#F E $#G E … E #HJ }, based on the gray level histogram distribution. It
divides the image into K = (%& 3 6) number of classes. The LM in Eqn. (3) represents
cumulative probability of N &O class and PM in Eqn. (4) depicts mean intensity of N &O
class. Therefore, the mean intensity of the whole image is P Q and between class vari-
ance is RS F . Refer to Eqn. (5) and Eqn. (6).
LM = $ T?UVW @? (3)
?$/$$XB
PM = $ T?UVW (4)
YW

P Q = TZ?[D .$/ @? = T\
M[D PM LM (5)

RS F = T\
M[D LM (PM ] P Q )F (6)
The optimal threshold values ^#D _ E #F _ E #G _ E … E #HJ _ ` are chosen by maximizing the be-
tween class variance (RS F ). Refer to Eqn. (7).

^#D _ E #F _ E #G _ E … E #HJ _ ` = abc$<de^RS F f#D E #F E #G E … E #HJ g`E 6 h #D i j i #HJ i k

(7)

3.4 Binarization
The grayscale image is binarized using Eqn. (8) for each threshold value of set$#. It
gives the same number of binary image as number of threshold level. Fig. 2 shows the
sample images after binarization with a fixed threshold value for pineapple fruit
where number of threshold level is 4.
6$$$$.>$!" (eE l) n #
!'* (eE l) = m (8)
0E$$$$$$$$$$$opqrbs.tr

Fig. 2. Sample binary images using Eqn. (8) and uv = :