AUTOMATIC FRUIT SORTING PANEL

B.S. ELECTRONIC ENGINEERING, BATCH 2016

Internal Advisor
Sir Saeed Azhar
Assistant professor
Electronic Engineering Department
SSUET, Karachi

Submitted by

M.UMERFAROOQ 2016-EE-189
M.A. HASEEB 2016-EE-077
M.SAAD KAMAL 2016-EE-192
SHAFIQ-UR-REHMAN 2016-EE-131

DEPARTMENT OF ELECTRONIC ENGINEERING

SIR SYED UNIVERSITY OF ENGINEERING AND
TECHNOLOGY, KARACH
 AUTOMATIC FRUIT SORTING PANEL
Abstract—This paper discusses the development of portable
fruit sorting and grading machine based on computer vision for
small agro-industries. The mechanical system is designed from
low cost material in the form of inclined and segmented plane to
substitute the utilization of conveyor belt. In this case, motor
servos are used as gate opener and director for the mechanical
system. The autonomous system collects video image from a
webcam placed on the top of analysis area, then the image will be
analyzed due to the process of computer vision. Firstly, the
computer vision algorithm transforms the RGB (Red, Green, and
Blue) color space to HSV (Hue, Saturation, and Value) color
space of the image to facilitate the processes of color
segmentation that are robust to the light intensity
fluctuation.Then the system will cluster fruit quality according to
the level of maturity and its dimension. In the end, the
autonomous system will actuate the servos to move the fruit to a
specific bin according to their quality grade. Then the result of
fruit analysis data will be displayed on PC’s monitor. The system
can do the task in 500 ms with precision result.
Keywords—portable fruit sorting and grading machine;
computer vision; HSV color space; servo-based sorter; low cost
mechanical design
I. INTRODUCTION
In agrarian country - like Pakistan, agro-industrial
companies are the major and vital sectors which drive the
economic growth. The supporting technology has to be
developed to boost the production of this sector. One of
supporting technology to be well-developed is in sorting and
grading processes [1]. Sorting and grading are important
process to maintain quality of products. In present common
scenario, sorting and grading of fruit according to maturity
level are performed manually before transportation. This
manual sorting by visual inspection is labor intensive, time
consuming, and suffers from the problem of inconsistency and
inaccuracy in judgement by different human [2]. To increase
the accuracy, precision, continuity, and uniformity of sorting
and grading, the development of novel technology is needed
[3].
For small/home agro-industrial companies in Pakistan
which are mostly in micro-business, the procurement of high-
cost sorting machine for their needs is burdensome, whereas
they need that machine to maintain and increase their product’s
c 2013 IEEE
978-1-4799-1078-6/13/$31.00 §
quality. The rapid development of computer vision and image
processing bring through the solution to fabricate a low-cost
and portable fruit’s sorting and grading machine for
small/home agro-industrial companies. Even more the
availability of low-cost and smart robotics servo, dynamixel
ax-12, can be applied as actuator in machine’s mechanical
system.
[4]. They used RGB color space, edge detection, and
boundary tracing to determine color quality and dimension of
mangoes. Different from them, this project uses HSV color
space and natural color segmentation to determine fruit
maturity and geometrics approach to determine fruit
dimension. [5]. HSV color space was more reliable to be
exploited in machine vision fruit grading and sorting.
II. COMPUTER VISION
Computer vision is the transformation of data from a still
video camera into either a decision or a new representation. All
such transformations are done for achieving some particular
goal [7]. In other words, computer vision is the representation
of image in another form after undergoing computation. The
new representation of image can be decision data, analytical
data including parameters, graphic describing specific
information, or another representation. The computer vision is
related to image processing techniques to manipulate the image
data by applying image enhancement, image restoration, image
analyzing, etc. In this project, the computer vision is needed to
transform the image of fruit which is captured webcam to be a
numerical value of HSV color components of every pixel
covering the entire image that will be used to compute
analytical data to determine the quality of fruit.
III. COLOR SPACE
A color space is a method by which we can specify, create,
and visualize color [8] into unique data representation. It is a
definition of color into separated components which construct
the color. The RGB color space defines specific color into
combinations of Red, Green, and Blue, otherwise the HSV
color space represents specific color into Hue, Saturation, and
Value. All that color representation is used in particular
71
condition according to the need, different color space is better
for different applications. Commonly, a color space is
represented as 3-D data representation.
RGB color space, the color red, green, and blue are
mapped onto a Cartesian coordinate system which results a 3-
D cube. The vertices of cube are the primary colors (red,
green, and blue) and the secondary colors (cyan, yellow, and
magenta) of light. The origin of the coordinate system is black
where the red, green, and blue (RGB) color components are all
0.0. The diagonally opposite corner of the cube is white, where
the RGB color components are at their maximum value. It is
the most commonly used color space in computer systems,
televisions, and videos [9]. The value of RGB in the system
are between 0- 255 to represent minimum value until
maximum value.
The HSV color space (visualized in Fig.1 right) is unique
color representation which defines colors according to Hue,
Saturation, and Value of image. Hue is the basic color type in
the range of 0-360°, expressed as color hexagon using red
axis as zero reference. Saturation is the vibrancy of color
ranges from 0-100% represented in the V axis. Value is the
representation of brightness from 0-100% which is measured
as the distance along the side axis of the cone. The advantage
of using HSV color space in this project is the effectiveness of
basic color determination using Hue component.
72
IV. SYSTEM DESCRIPTION
A. Mechanical System
The mechanical system is designed in the form of inclined
and segmented plane to minimize the utilization of high-cost
conveyor belt. By adopting the concept of LEGO toy building,
this mechanical system is designed to be fast and easily
mounted and separated. The advantages of using this
mechanical concept are low power, portable, and easy to
arrange. By using this concept, the system is not only
upgradable but also customizable according to the user’s needs.
The mechanical system is designed to be upgradable by adding
the additional part and made of thin and light woods so that it
can be easy to be brought and transported. The main system
can be partitioned into two parts.
The camera holder that holds webcam as the main and a
single sensor used in the system, is placed on the top of
analysis-region with slant angle 60° to the bottom so that it is
perpendicular to the analysis-region. The camera holder is
made of aluminium and strongly attached to the side of
inclined and segmented plane. The motor servos - ax-12
dynamixels are utilized as the gate opener and director. Those
motor servos are low-cost and easy to find in the market.
Moreover, the dynamixel is easy to be controlled using serial
communication from usb2dynamixel. For the minimum
system, only 4 dynamixels are used, but it can be added by
additional parts to increase the number of quality. The main
purpose of this mechanical design is to achieve the low-cost,
low-power, and portable mechanical system for sorting and
grading machine so that it can be employed flexibly by the
user e.g. within a room, a moving truck container, a garden,
etc.

B. Sorting and Grading Mechanism
\\
Flowchart of System
The flowchart of system is described. The autonomous
system is started with system initialization e.g. camera
initialization and servo motor initialization. Then the camera
(which covers analysis area and queuing area) will check
analysis area whether a fruit exists or not. If there is not a fruit,
the camera will check queuing area. If there is a fruit in
queuing area then the mechanical system will control the motor
servo to load fruit from queuing area toward analysis area,
otherwise if there is not a fruit, the system enters idle state to
wait until user reloads fruits. If a fruit exists on analysis area,
the system will analyze its quality. The quality analysis
consists of 1) color quality analysis and 2) dimension of fruit
analysis. Both of them will determine fruit quality. Then the
autonomous system will sort the fruit according to its quality to
specific bin represent fruit’s quality number.
73
The method of sorting is based on color quality or
dimension of fruit. The system will provide the choice in color
or dimension mode in the beginning of program so that the user
more flexible to choose which method they need. However, the
data of color and dimension of every fruit analyzed will be
displayed in PC’s monitor over time.
C. Computer Vision Algorithm
visualizes the method of computer vision algorithm done
by the autonomous machine. By grabbing video image from
the camera, the computer vision algorithm crops some Region
of Interest (ROI) within the image. The ROI covers the region
of image which consists of the image of fruit. After take the
ROI, the machine transforms the ROI image from RGB color
space into HSV color space. The data in ROI-HSV is then
processed to do feature extraction. The feature extraction is
performed to segment the fruit image from the background and
take important information data. The data taken from feature
extraction are stored in the memory to be computed by the
computation algorithm. As the result of computation, the
machine controls the servos and displays the parameter data on
PC’s monitor.
Grabbing Image Cropping ROI Feature Extraction
HSV Transform
Computation:
 Interested-Color Percentage
 Fruit Color Defect
 Fruit Size
Command data
Displaying Data in Servo Control
Manipulated Image
Computer Vision Algorithm
V. FEATURE EXTRACTION
A. RGB to HSV Transformation
The origin of image data of C920 is RGB, so that the
system needs to transform RGB to HSV color space. The RGB
to HSV transformation is done by applying the formula of
RGB-HSV conversion as in (1). The conversion results the data
of Hue, Saturation, and Value, which are still in 3-D color
space with another representation. To use the only Hue
component from HSV, the algorithm is designed to divide 3-D
array data to be 1-D array data of HSV component. The array
of HSV = {H,S,V} then is divided into HSV{0} = H, HSV{1}
= S, and HSV{2} = V. From now on, the colors of every pixel
in the image are represented as separated HSV component.

B. Natural Ripeness Color Segmentation
Segmentation is the technique used to partition the image
into several regions [11] which include important information.
In this project, the segmentation covers the processes of
thresholding and clustering which is used to group and class
the natural color of sorted and graded fruit. According to [10],
the maturity of fruit can be determined from the distribution of
specific color component. By analyzing the fruit’s image
within HSV color space in the range of Hue, the natural
ripeness level of fruit can be monitored in the line of Hue
number from H = 29° to H = 60° with S=V= 100% for the
whole range, as shown in Fig. 5. The emergence of yellow to
red component is the signal of maturity. The S and V are still
set in the whole region, whereas they can be used to analyze
another parameter of fruit quality, that is fruit defect. The
darker the color of fruit color, the worse the quality. If the dark
level of color exceed the threshold, the fruit quality is bad
(indication of fruit decay). The value of H is still in degree of
Hue from hexagonal HSV color space. The computer vision
algorithm represents the degree from 0° - 360° into byte
representation from 0 until 180. So that, it can be said that the
characteristic of orange fruit ripeness is the emergence of H
from 15 – 30 in the range of byte value.
VI. CALCULATION AND CALIBRATION
The calculation of categorized color component percentage
of green, yellow, orange (g, y, o, in N-number) is performed
by using (2). Then by comparing the composition of
categorized color component, the color decision can be
resulted. The calculation of fruit size is done by applying (3).
The K is the constant of calibration resulted in machine
calibration. The distance between camera and the base of
queuing and analysis region is 300 mm. So that by
implementing spherical area approaches, the value of K can be
determined. In this project, calibration of K is found as 1/102.
Every pixel of image are the representation of 1/102 cm or
0.098 mm. It is very precise measurement than using a ruler or
vernier caliper. The calibration is held in a room with light
intensity 200 lux.

74
 The fruit samples used in experiment have dimension 40
mm to 75 mm. The color diversity is graded from green to
orange. The high quality fruits have characteristics of Hue
color from 60° to 29° or in byte representation 30 - 15 (see Fig.
5) and have upper class diameter > 60 mm. The combination of
color class and diameter class are used to determine fruit
quality.
The calibration of system uses a sample of an orange fruit.
First, the orange fruit is measured using vernier caliper and the
second step the fruit diameter analysis algorithm will estimate
fruit diameter based on fruit’s bounding box. From the
measurement using vernier caliper, orange fruit sample had
diameter of 7.3 cm as shown in Fig. 6. Then, by using visual
measurement technique the system estimated fruit diameter.
The measurement data is sampled every 10 ms along 500 ms of
total sampling time. Fig. 7 shows the results of sample orange
fruit’s diameter estimation. The error of measurement is
relatively low with deviation no more than 0.003 cm.
VII. RESULT AND DISCUSSION
The experiment data are shown in Table 1. The data are
taken from the first 10 fruit samples after calibration. As shown
in Table 1, the color classification of all fruits are successfully
done (there is no misjudging of fruit color) in the room with
light intensity from 150 – 500 lux. The fluctuating results of
size measurement data are small (below 2%) so that the
calibration is successful and the size measurement results are
valid. Overall, the machine works relatively fast with sorting
speed 500 ms (Processor 2.2 GHz, servo serial communication
1Mbps).
75
VIII. CONCLUSION
This paper introducing a new design of autonomous fruit
sorting and grading using computer vision so that it is portable,
fast, low-cost, and user customizable. The test has been
conducted for orange fruit in the size between 25 mm – 75 mm
and indicates good result in color classification, sizing, and
sorting. The fluctuation of lighting condition does not affect the
precision of system while the light intensity is enough (150-
500 lux). For future work, the techniques of sorting can be
upgraded to be multiple-analysis using parallel-threading and
plural-sliding path to improve the quantity of fruit per time
sorting.
ACKNOWLEDGMENT
We would want to appreciate and thank all of those people who
helped us during this journey of completing final year project.
Without everyone’s support, we could not have accomplished
our work.
We express our sincere gratitude to Mr. SAEED (Dept. of
Electronics Engineering) for his amazing support and guidance,
continuous encouragement and supervision throughout the
course of the present work. It’s of no doubt that we have been
very fortunate to work under such a supportive and cooperative
teacher/internal
And last but not the least we are extremely grateful to the whole
faculty and administration of university, for providing us all the
facilities that were mandatory, without which this project could
not be a success.
REFERENCES
 H. A. Eissa and A. a. A. halik, “Understanding Color
Image Processing by Machine Vision for Biological
Materials,” Struct. Funct. Food Eng., pp. 227–274,
2012.
 S. Yogitha and P. Sakthivel, “RESEARCH ARTICLE
A Distributed Computer Machine Vision System for
Automated Inspection and Grading of Fruits,” vol. 3,
no. 4, pp. 522–527, 2014.
 S. Nandi, B. Tudu, and C. Koley, “A Machine Vision-
Based Maturity Prediction System for Sorting of
Harvested Mangoes.,” IEEE Trans. Instrum. Meas., vol.
63, no. 7, pp. 1722–1730, 2014.
 R. Mahendran, “Application of Computer Vision
Technique on Sorting and Grading of Fruits and
Vegetables,” J. Food Process. Technol., pp. 1–7, 2012.
 O. Marques, "Histogram Processing," in Practical
Image and Video Processing Using MATLAB, Florida,
John Wiley, 2011, pp. 176-180
 O. Marques, "Image Segmentation," in Practical Image
and Video Processing Using MATLAB, Florida, John
Wiley, 2011, pp. 365-371
 O. Marques, "Morphological Image Processing," in
Practical Image and Video Processing Using
MATLAB, Florida, John Wiley, 2011, pp. 299-325
 O. Marques, "Feature Extraction And Representation,"
in Practical Image and Video Processing Using
MATLAB, Florida, John Wiley, 2011, pp. 447-466.
 NM Ali, MS Karis, AFZ Abidin, B Bakri, NR Abd
Razif, “Traffic Sign Detection and Recognition:
Review and Analysis”, JurnalTeknologi 77(20), 107-
113, 2015
 NM Ali, YM Mustafah, NKA Md Rashid,
“Performance Analysis of Robust Road Sign
Identification”, IOP Conference Series: Materials
Science and Engineering 53(1), 012017, 2013
 NM Ali, MS Karis, J Safei, “ Hidden Nodes of Neural
 Network: Useful Application in Traffic Sign Recognition”,
in IEEE Conference on Smart Instrumentation,
Measurement and Applications (ICSIMA), 2014, pp. 1 -4.
 NM Ali, NKA Md Rashid, YM Mustafah, “Performance
Comparison between RGB and HSV Color Segmentations
for Road Sign Detection”, Applied Mechanics and
Materials, 393, pp. 550-555.
 MS Karis, N Hasim, N Mohd Ali, MF Baharom, A
Ahmad, N Abas, MZ Mohamed, Z Mokhtar, “Comparative
Study for Bitumen Containment Strategy using Different
Radar Types for Level Detection”, 2014 2nd
 International Conference on Electronic Design (ICED), pp.
161-166

76