See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/336020266

Automated Sorting and Grading of Agricultural Products based on Image

Processing

Conference Paper · November 2019

DOI: 10.1109/ICICT47744.2019.9001971

CITATIONS READS

18 2,376

6 authors, including:

Hafiz Muhammad Tayyab Abbas Usama Shakoor

National University of Sciences and Technology Institute of Space Technology
1 PUBLICATION 18 CITATIONS 1 PUBLICATION 18 CITATIONS

SEE PROFILE SEE PROFILE

Jaleed Khan Khurram Khurshid

University of Galway Institute of Space Technology
34 PUBLICATIONS 1,165 CITATIONS 130 PUBLICATIONS 2,415 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Quantum Machine Learning Paradigm for future generation mobile networks View project

HYPERSPECTRAL IMAGE ANALYSIS FOR WRITER IDENTIFICATION USING DEEP LEARNING View project

All content following this page was uploaded by Hafiz Muhammad Tayyab Abbas on 31 January 2021.

The user has requested enhancement of the downloaded file.

 Automated Sorting and Grading of Agricultural
Products based on Image Processing
Hafiz Muhammad Tayyab Abbas, Usama Shakoor, Muhammad Jaleed Khan, Mushtaq Ahmed, Khurram Khurshid
Artificial Intelligence and Computer Vision (iVision) Lab,
Department of Electrical Engineering, Institute of Space Technology, Islamabad 44000, Pakistan.
hafiztayyab13@gmail.com, usamashakoorpk@gmail.com, mjk093@gmail.com, qazi@ist.edu.pk,
khurram.khurshid@ist.edu.pk
Abstract— Demand of good quality fruits is increasing with the
rise in population. GDP of the many countries rely upon its
export. Major part of GDP of Pakistan depends on its fruit
export business. The present quality detection, sortation &
dispensing systems have the downside of low yield, time
consumption, high cost and complication. The study purpose of
machine-controlled fruit sorting using image processing is to
create higher quality fruit sorting, quality maintenance,
production and to cut back labor concentration. It is a
necessity for machine-controlled system that quick and quality
feature detection as well as quick dispensing of fruits is carried
out. A comprehensive review of current work related to
automated sorting and grading of agricultural products is
presented in this paper. We also propose a complete end to end
system of automated and efficient fruit sorting and grading
based on image processing. Effectiveness of the proposed
structure is demonstrated by the initial experimental results.
Keywords—image processing, fruit sorting, fruit grading,
machine vision
I. INTRODUCTION
The main aim of this study and project is to design and
fabricate a fruit sorting and dispensing machine that is
capable to effectively distinguish between a good quality and
a bad quality fruit. Planning a complete machine-controlled
system is our main objective. We have designed the system
based on analysis of images of the fruit. Image processing
gives precise, accurate, reliable and quantitative data
regarding giant volume of fruits which can't be dispensed by
individuals [1]. With the development in the field of image
processing, software code and hardware, it becomes enticing
to find the standard of object, by exploitation computer
vision technology [1]. Our planned system collects image
from camera and then applies certain image processing
techniques [2] to extract features like texture, color and size.
Decision outcome is then sent to the controller of Arduino
compatible with laptop. The controller is programmed with
Arduino where we can see the step by step results [3]. There
is a linear conveyor arrangement to position the item for
sorting mechanism [4]. The major benefits of machine-
controlled fruit sorting system is value effective, reliable,
larger product stability, safety, speed and correct sorting
results that improve the standard of the merchandise and get
rid of inconsistent manual analysis and scale back
dependence on accessible ancient review [3][4]. The main
aim is to design an automatic fruit sorting system that
improves the standard, enhances the production efficiency
and reduces labor cost of the method.
In this paper, we present a comprehensive review of
present systems related to automated sorting and grading of
agricultural products and we also propose a complete
hardware system for automated and efficient fruit sorting and
grading based on image processing.
II. LITERATURE REVIEW
After harvesting of fruits, they are washed, sorted,
graded, packed and stored. Out of all these stages, sorting
and grading of fruit are very important steps. Sorting means
separation based on a single measurable property of some
material while grading is the assessment of the overall
quality of food using a number of characteristics. This
section reviews the related fundamentals and recent
literature on this topic.
A. Understanding Color Image processing by Machine
Vision for Biological Materials
Fruits after harvesting are washed properly. After that, they
are sorted so that the rotten fruits are taken off. Quality
evaluation of fruits is a very important consideration for
production of vegetables and fruits. A lot of research has
been conducted to make the process of sorting efficient and
automatic. Sorting is a process which is done based on the
single measurable property of materials which are in raw
form. Grading is considered to be the evaluation of quality
of food materials based upon certain attributes. Food
products can be separated and sorted based upon certain
attributes like their appearance, sizes and colors. At present
in most rural parts of Pakistan, fruits are sorted manually
which is a very time consuming process. Therefore we need
a low cost, automatic and effective method that can sort the
food products automatically based on machine vision. Image
processing and machine vision have a wide ranges of
applications in many industries specially in the fruit
industries for evaluating the size, color and weight of the
food products. Mainly the food products are evaluated on
the basis of following parameters.
• Color
• Homogeneity
• Size
• External defects[5].
B. An Apple Grading System in step with European Fruit
Quality Standards exploitation Dennis Gabor Filter and
Artificial Neural Networks
In recent years, application of image processing and
machine vision has increased in the fruit industries. By
using these techniques we can now grade the fruits more
easily. Apples can also be graded using their texture feature
analysis with the help of artificial intelligence and image
processing. Golden delicious and red delicious are the two
kinds of apple that can be separated by using Gabor filter.
By using this kind of techniques mean and variance of the
values of energy are collected which further present the
intensity of the image and contrast of the image
respectively. Gabor filter are space frequency filters using
which the local and global information can be presented.
These types of filters are very useful for working on the
quasi periodic signals which are actually the textures. By
using the texture features several parameters of the object in
an image can be measured like regularity of the object, its
smoothness and coarseness[6].
C. Virtual critic for Apple Quality Assessment exploitation
Fruit Size and Illumination Features
This study mainly focuses on the virtual grader which can
be used for the assessment of apple quality using artificial
intelligence and machine vision. The idea of the virtual
grader was extracted in the shape of K Nearest Neighbor
classifier which is basically based on the design of
Euclidean distance matric. K NN classifiers have very vast
ranges of applications because they are considered to be
very roubt especially in the noisy environment. While
measuring the fruit size with the help of that type of
technique the illumination of surface of fruit is one of the
most important parameters affecting the quality of results.
Therefore the performance of the system is evaluated taking
different values of the illumination of the surface of fruit.
The system talked above can be successfully implemented
for the grading of the type of apple red delicious using size
of fruit. It can be observed experimentally that K-NN
classifier used achieves very good results[7].
D. Automation of Object Sorting using an Industrial Rob
arm and MATLAB based Image processing
In these recent years, automated techniques are used for
increase in the production in the industry, for increasing
accuracy, for elimination in the human error and for safety
purposes. With the development in the artificial intelligence,
machine vision can be used to make most of the processes in
the industry related to production automatic. By using
machine vision, they can be developed to perform the tasks
equivalent to human beings. Human visions have some
limitations like we are limited when we are detecting
various shades of different colors or when we are
determining high precise dimensions. While in automated
machine vision based systems identification of the parts of
the colors can be done more affectively. So the objects
having different colors detected by the system can be sorted
more affectively with the help of robotic arm which pick
and places the object depending on its color. If the desired
color is not found then that object is rejected by the robotic
arm. MATLAB is used in the complete mentioned system
for image processing which is interfaced with the
microcontrollers to run the different motors in the system.
Machine visions have vast ranges of applications in the
pharmaceuticals industry, agriculture industry and also in
automobile industry. This technology is far better than the
manual sorting that was used initially[8].
E. Enhanced Techniques for sorting and grading the Fruit
Quality using MSP430 Controller
This study present detection of quality of fruits. Size and
color of fruits, are some features used for the detection of
2
pixels. For removing background Gaussian Mixture Model
is used and Support Vector Machine is used for the
classification of different colors. At present fruits are mostly
sorted with the help of hand which is very slow process as it
requires a lot of time and effort. Also the manual sorting
techniques has very slow speed, complex and also very
expensive. The present system for the removal of
background is based on the histogram approach. It has one
drawback that it sometimes also removes the pixels of fruits
when it sees them as clusters. This defect can be removed
with the help of GMM which is actually pixel wise
classification method. Image processing gives us reliable,
consistent and accurate information about the large volumes
of fruits which may not be achieved by employing human
beings. The hardware prototype can be created by using
MSP430 microcontroller which is a low power controller. It
is very useful specially in the fruit industries for the
detection and grading of fruits[9].
F. Automated Skin Defect Identification System for Fruit
Grading Based on Discrete Curvelet Transform
The main aim of carrying out this study is to develop the
technique for the assessment of quality of the fruits by
considering their texture analysis on the basis of Curvelet
Transform. These techniques have multiresolution approach
and they are capable for the assessment of surface of fruit
both at high resolution and low resolution. CCD camera is
used for the acquisition of image of the fruit and lemon and
guava were analyzed by experimentation.
Energy, standard deviation, entropy and mean are some
measures based on the textures that are used to sort texture
of the surface of fruits. Almost 96% accuracy is achieved
using SVM classification method. Texture features based on
curvelet gives fine insight to guess about damages in the
fruit skin[10].
G. Design and Development of Object Recognition and
Sorting Robot for Material Handling in Packaging and
Logistic Industries
Color is that the most commonly used feature to tell
differences between objects, sorting, recognizing and
pursuit. Typically automaton is mounted with a camera or
the camera is mounted within the space to notice the thing.
This technology is mainly applied in material handling in
provision and packaging business where the objects moving
on a belt are separated by using a color detection automaton.
Robot has been developed with success and testing was
carried out using the dummy industrial setting developed
within the laboratory. The target is met by sorting the
objects supported the color feature from a gaggle of objects.
A GUI in MATLAB was created to show the video of the
incoming object and displaying the color of the thing to the
user. The user must choose the COM Port initial. The
automaton got the signal regarding the position of the thing
within the space through the serial communication. Once the
color of the thing was known, the larva picks the thing and
automaton reaches the required location. Color detection
beside pattern recognition and Speech recognition can play
an important role in several industries and also can increase
the accuracy of the task in supplying and packaging
business.
The robot was successfully developed and texting was also
successfully done in the laboratory .the object was
successfully sorted between different objects on the basis of
color features. A GUI in MATLAB was successfully created
in order to display the video of the incoming object that is to
be sorted and its color is displayed to the user with the help
of that GUI. The developed Eye –Bot is able to detect the
color of the object is also able to place in the desired
location. The color detection capability can also be extended
to blue and green[11].
H. Size Properties of Mangoes using Image Analysis
This study mainly focuses on the development of the
algorithm to effectively detect, sort and grade the mangoes.
First the image is taken and then from those images different
features of the mangoes are extracted and then the mango
class is identified. The features which are extracted are area,
percent defect and roundness. Either mango is export, local
or rejects the parameters that decide that thing are roundness
and percent defect. For segmentation, an optimal threshold
was used and for the determination of perimeter the tracing
of boundary was used.
The algorithm is capable to segment the mango correctly
even we change its position. There is one limitation that it
cannot correctly identify the stem as it can be green or
brown. The pixel of the green stem was identified to be
healthy and pixels of the brown stem are identified as
defective[12].
III. PROPOSED FRUIT SORTING AND GRADING SYSTEM
We have developed a complete hardware system that
involves a mechanical part as well as a camera based image
processing part. These two processes are interconnected to
attain the ultimate goal of sorting of fruits so that when a
rotten apple comes, it is taken away. The classification of
fruits is carried out by conventional image processing
techniques using MATLAB. Dispensing part is completed
on the conveyor belt arrangement with the assistance of
Arduino. Below mentioned Fig.1 is the block diagram of our
proposed model. An object (which in our case is apple)
comes on a conveyer belt where the image acquisition takes
place with the help of camera. The implemented feature
based classifier decides the maturity of fruits and the
decision is sent to Arduino that effects the separation of
spoilt fruit using mechanical separators that act as
dispensing mechanism.
The following steps are involved:
A. Image Acquisition
Image acquisition is done through a low cost camera that
takes the pictures of fruit on conveyer belt from affixed
location.
B. Image Processing
Image processing involves the application of various pre-
processing steps to boost quality of the image. This is done
so that the image is clear and the fruit is easily segmented
out. Binarization of the image is also carried out as certain
features are extracted in binary domain [13]. We then
remove the noise from the picture with the help of median
filter as shown in figure 2.
3
Object
Arrival
Image
Acquisition
Image
Processing
Feature
Extraction
Image
Classification
Dispensing
Mechanism
Fig. 1. Block diagram of the proposed system
C. Feature Extraction
Certain features are to be extracted to detect apple skin
features for identification of quality (e.g., the position and
nature of defect). Training data is collected and send to the
classifier. Mainly three features are extracted that are as
following:
1) Edge Features
The edge features can be evaluated by applying different
kind of filters like sobel, canny and pewit edge detection
filters. Out of all these, we prefer sobel edge filter in our
work as the visual results were better.
2) Color Features
For the determination of color features in our work, we use
HSV color space. Hue, Saturation, and Value (HSV) is a
color model that is often used in graphics and paint
programs in place of the RGB color model. In using this
color space, a color is specified and then white or black is
added to easily make color alterations.
3) Texture Features
To determine texture features we use entropy filter and
standard deviation filter which gives us the texture analysis
of the picture. Entropy filter highlights edges by brightening
pixels which have dissimilar neighbors. It sets each pixel in
the current image equal to the entropy (measure of disorder)
of the pixel values in its specified neighborhood, whereas
standard deviation filter performs standard deviation
filtering and returns the filtered image. The value of each
output pixel is the standard deviation of the 3-by-3
neighborhood around the corresponding input pixel.
D. Classification
The extracted features help us in classification of the object
as good or bad. For classification, fusion of the three feature
map images is used. As discussed above, we take HSV
transform of the image and extract V component of the HSV
transform as V component gives us the max info about
variation in the color of the image to detect the rotten parts
as apple parts which are rotten show different color in the
image. We convert the V component to BW so that the
rotten parts are shown by white and remaining parts of the
image remain black. Then with the help of morphological
operation and applying connected components analysis we
extract the rotten parts of the image in the form of connected
components.
The fused image, that is formed by logical AND of the three
feature images, is analysed to see if any larger areas exist. If
there are unidentified large areas in the final binary image,
then we assume it to be a rotten mark on the fruit. We took
that decision by computing the area of white pixels in the
resultant image. If area is zero mean there are no white
pixels in the image means no rotten parts. It means we select
the apple as good and if there are white pixels in the image
means rotten parts then area will be non-zero so in that case
we decide that thee apple as bad. The fruits not satisfying
the factors are discarded and can be dumbed.
E. Dispensing
Once the classifier takes a decision, the decision is passed to
the hardware mechanical unit. The operation of dispensing
is done through gate model (providing 2 ways on conveyor).
The management of gate model is done through Arduino.
IV. EXPERIMENTS & RESULTS
Fig. 2 depicts the results of the complete system. The GUI
of the complete project is made in which step-wise results of
all the intermediate images from very first input image to
last one resultant image are shown. The GUI gives a quick
overview of all the intermediate steps involved and is easy
to use by the person operating the dispensing system.
4
Fig. 2. Visual Demonstration of Experimental Results
V. CONCLUSION
Automatic fruit sorting using image processing is a strong
and economical system as compared to the existing manual
sorting that is not economical and takes a lot of time and
labour. The system accelerates the process and helps the
fruit business. However, this classification will be more
robust if more features are added including circumference,
form and shape detection. Multiple cameras can also be
installed to enhance visibility from all angles at once. Our
aim is to further enhance this project and create a process
primarily based on computer vision for fruit sorting. In later
stage, we will add system to incorporate all kinds of fruits
and vegetables. The system can further be enhanced in other
areas in addition to food and beverages like in
pharmaceutical, paints and cement industries.
REFERENCES
[1] A. Vibhute, “Application of Image Processing in Agriculture : A
Survey Applications of Image Processing in Agriculture : A Survey,”
no. May, 2014.
[2] A. Mishra, P. Asthana, and P. Khanna, “THE QUALITY
IDENTIFICATION OF FRUITS IN IMAGE PROCESSING USING
MATLAB,” pp. 2319–2322, 2014.
[3] M. Uesugi, F. Application, P. Data, P. E. E. Hoffman, and A. E. J.
Rolla, “United States Patent ( 19 ),” no. 19, 1977.
[4] S. Sheth, R. Kher, R. Shah, P. Dudhat, and P. Jani, “Automatic
Sorting System Using Machine vision.”
[5] A. H. A. Eissa and A. A. A. Khalik, “Understanding Color Image
Processing by Machine Vision for Biological Materials,” 1999.
[6] K. A. Vakilian and J. Massah, “AN APPLE GRADING SYSTEM
ACCORDING TO EUROPEAN FRUIT QUALITY STANDARDS
USING GABOR FILTER AND ARTIFICIAL NEURAL,” vol. 17,
no. 1, pp. 75–85, 2016.
[7] A. Pal, S. Chauhan, and A. P. Singh, “Virtual Grader for Apple
Quality Assessment,” vol. 14, no. 4, 2014.
[8] P. D. B. Rane, G. S. S, N. D. V, and S. J. U, “― Automation of
Object Sorting Using an Industrial Roboarm and MATLAB Based
Image Processing ‖,” vol. 5, no. 2, pp. 187–191, 2015.
[9] J. Ramprabhy and S. Nandhini, “Enhanced Technique for Shorting
and Grading the Fuit Quality Using MSP430 Controller,” Int. J. Adv.
Eng. Techology, vol. 7, no. 5, pp. 1483–1488, 2014.
[10] R. Thendral and A. Suhasini, “Automated skin defect identification
system for orange fruit grading based on genetic algorithm,” Curr.
Sci., vol. 112, no. 8, pp. 1704–1711, 2017.
[11] V. Devalla, “Design and Development of Object Recognition and
Sorting Robot for Material Handling in Packaging and Logistic
Industries,” Int. J. Sci. Adv. Technol., vol. 2, no. 9, pp. 30–35, 2012.
[12] T. U. G. Jr, “Size Properties of Mangoes using Image Analysis,” vol.
6, no. 2, pp. 31–42, 2014.
[13] K. Khurshid, I. Siddiqi, C. Faure, N. Vincent, Comparison of Niblack
inspired Binarization methods for ancient documents, 16th Document
Recognition and Retrieval Conference (DRR), January 2009, USA
 5

View publication stats