Proceedings of 2020 IEEE Applied Signal Processing Conference (ASPCON)

A novel approach to detect and classify fruits using

ShuffleNet V2
Sourodip
Ghosh
Dept.
of
Electronics
Engineering

KIIT
University














Bhubaneshwar,
India
sourodip.ghosh02@gmail.com
Md.
Jashim
Mondal Sourish
Sen
Dept
of
Electronics
Engineering
Dept.
of
Electronics
Engineering

KIIT
University












KIIT
University
















Bhubaneshwar,
India
Bhubaneshwar,
India

jashimmondal007@gmail.com sourishsen7@gmail.com

Soham
Chatterjee Nilanjan
Kar
Roy Suprava
Patnaik

Dept
of
Electronics
Engineering
Dept
of
Electronics
Engineering
Dept
of
Electronics
Engineering

KIIT
University













KIIT
University


















KIIT
University




















Bhubaneshwar,
India
Bhubaneshwar,
India
Bhubaneshwar,
India
1704302@kiit.ac.in nilan3107@gmail.com suprava.patnaikfet@kiit.ac.in

Abstract—In
the
proposed
context,
we
show
an
identification

and
classification
a pproach
o f
o rganic
p roducts
b etween
41

unique
classes.
We
have
utilized
a
pre-trained
Convolutional

Neural
Network
design,
the
ShuffleNet
V 2,
c hosen
a s
f or
the

proficient
p resentation
e xtent
o f
building
c onvolutional
b locks
at

ease,
by
using
more
feature
channels.
The
model,
when
tried
on

the
proposed
dataset,
accomplished
a
test
accuracy
of
96.24%
accordingly
making
a
stride
further
in
the
exploration
proposed

by
past
authors
surveying
the
organic
product
detection
via

Convolutional
learning
and
feature
re-usability
technique.
The

outcomes
are
assessed
utilizing
various
assessment
parameters,

like
Precision,
Sensitivity,
F-Score,
and
ROC
score.
Moreover,
a

visual
of
the
predicted
images
was
performed
to
anticipate
the

evaluation.
Index
Terms—Convolutional
Neural
Networks,
Fruit
classifi-
cation,
ShuffleNet
V2
I.
I NTRODUCTION
In
speaking
to
the
origin
for
the
human
mind,
images
are
the
most
fundamental
technique
in
physical
grouping

of
fruits.
Components
influencing
f ruits
c an
b e
measured

outwardly
which
are
difficult,
costly
and
are
handily
influenced

by
physical
variables,
including
conflicting
a ssessment
and
abstract
outcomes.
The
fruit
examiners
have
done
the
quality

assessment
by
experience
and
observing.
This
strategy
is
fundamentally
conflicting,
fl ighty
an d
ch oices
ar e
on ly
from
time
to
time
equivalent
among
examiners.
In
this
kind
of

condition,
the
examination
of
natural
products
for
a
few
prospective
rules
is
a
persistent
assignment;
machine
vision

frameworks
are
optimum
in
organic
product
assessment.
The
computer-aided
procedures
used
to
perceive
a
fruit
depend
on
four
essential
highlights
which
portray
the
item:
intensity,

color,
shape
and
texture,
Arivazhagan
et
al.
[1]
propose
an
effective
combination
of
shading
and
surface
features
for
fruit
recognition.
The
acknowledgment
is
finished
by
the
base
separation
classifier
dependent
on
the
measurable
and
co-
event
features
obtained
from
the
Wavelet
transformed
sub-
bands.
Zawbaa
et
al.
[2]
propose
a
natural
fruit
detection
framework
to
group
and
distinguish
between
classes
of
the

fruits.
In
this
paper,
we
propose
a
classification
a nalysis
of

fruits
using
ShuffleNet
V 2,
a
l ight-weight
C NN
architecture,

and
evaluate
the
performance
with
feature
assessments
and

evaluation
metrics.
II.
L ITERATURE
S URVEY
In
this
segment,
we
will
get
acquainted
with
past
under-
takings
on
Fruit
Recognition
using
Deep
Learning.
Bargoti
et

al.
[3]
use
a
framework
to
perceive
fruits
in
an
estate.
They

employ
usage
of
Faster
R-CNN
framework
across
detection

of
orchards,
including
apples,
almonds
and
mangoes.
They

achieve
an
F1-Score,
greater
than
0.9
for
apples
and
mangoes.

The
quicker
Region-based
convolutional
association
has
been

explained
in
nuances
by
Ren
et
al.
[4].
Puttemans
et
al.
[5]

worked
on
methods
related
to
the
customized
gathering
of

fruits
with
a
technique
for
recognizing
prepared
strawberries

and
apples
from
estates.
Barth
et
al.
[6]
portrays
a
technique

to
exploration
of
images.
They
created
a
synthetic
dataset,

by
using
emperical
measurements
and
creating
3D
plant

models.
Deep
Learning
models
were
trained
on
these
synthetic

datasets.
This
was
the
first
synthetic
approach,
enhancing

further
step
in
Deep
Learning
using
3D
synthetic
models.
An
advancing
survey
of
working
found
that
Convolutional

Neural
Networks
have
expanded
a
stunning
ground
in
pic-
ture
portrayal
and
item
location.
CNN
has
been
exhaustively

applied
to
design
attestation
issues,
for
example,
image
clas-
sification
and
object
detection
[7]
[8]
[9].
A
Quicker
Region-
based
CNN
approach
has
been
utilized
for
tweaked
customized

fruit
detection
and
acknowledgment
from
pictures.
The
system

is
prepared
to
utilize
RGB
and
NIR
(close
infra-red)
pictures

and
utilizing
the
blend
of
these
photographs
the
structure

can
have
better
execution
[10].
Sunderhauf
et
al.
[11]
in

2014,
presented
a
CNN
model
to
extract
features
from
plant

classification
datasets
with
a
score
of
0.249
on
LifeCLEF
2014

program.
Naik
et
al.
[12]
highlighted
the
excess
time
and

‹,(((

163
ISBN: 978-1-7281-6882-1 PART: CFP20P52-ART
Authorized licensed use limited to: KIIT University. Downloaded on February 10,2022 at 03:48:35 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of 2020 IEEE Applied Signal Processing Conference (ASPCON)

energy
loss
in
controlling
fruit
quality
control,
and
therefore
TABLE I: Number of images for proposed method from the
discuss
using
methods
like
SURF,
HOG,
LBP,
alongwith
some
FRUITS 360 dataset.
discussions
about
machine
learning
algorithms
like
SVM,
Class
Class Training images Validation images Test images
ANN,
KNN,
and
a
few
CNN
architectures.
The
review
con- index
tains
detailed
comparison
in
the
form
of
a
table,
to
summarize
0 Apple Braeburn 492 75 89
1 Apple Crimson Snow 444 66 82
the
best
classifiers
f or
f ruit
d etection
t ill
2 017.
Pre-handling,
2 Golden Apple-I 480 74 86
and
grayscale
image
processing
was
applied
so
as
to
develop
3 Golden Apple-II 492 86 78
4 Golden Apple-III 481 73 88
information.
The
accuracy
achieved
on
the
testing
set
was
5 Granny Smith Apple 492 80 84
96.3%. 6 Apple Pink Lady 456 80 72
7 Red Apple-I 492 83 81
8 Red Apple-II 492 88 76
III.
PROPOSED
METHOD 9 Red Apple-III 429 80 64
10 Red Delicious Apple 490 75 91
In
this
research,
we
present
a
fruit
classification
approach
11 Red Yellow Apple I 492 83 81
using
ShuffleNet
V2
[13].
This
architecture
unravels
the
com- 12 Red Yellow Apple II 672 107 112
plexity
level
by
limiting
MAC
to
a
great
extent.
This
permits
13 Banana 490 78 88
14 Banana Lady Finger 450 68 86
the
model
to
arrive
at
a
lower
bound
when
the
total
number
15 Banana Red 490 84 82
of
input
and
output
channels
are
equivalent.
The
reusing
of
16 Cherry I 492 83 81
17 Cherry II 738 120 126
features
through
mapping
is
a
preferred
position
additionally
18 Cherry Rainier 738 113 133
found
in
DenseNet
[14].
The
exceptionally
proficient
model
19 Wax Black Cherry 492 81 85
20 Wax Red Cherry 492 86 78
makes
building
blocks
simpler,
by
utilizing
more
feature
21 Wax Yellow Cherry 492 85 79
channels.
Likewise,
the
model
can
access
a
huge
system
22 Grape Blue 984 157 171
23 Grape Pink 492 87 77
effortlessly.
This
permits
the
model
to
accomplish
the
best
24 White Grape 490 85 81
in
class
accuracy
tradeoff
and
speed
in
the
computation.
The
25 White Grape II 490 74 92
26 White Grape III 492 89 75
light-weight
CNN
model
is
an
advantage
for
situations
where
27 White Grape IV 471 80 78
computation
complexity,
MAC
and
system
specifications
are
28 Grapefruit Pink 490 81 85
29 Grapefruit White 492 83 81
not
very
efficient.
It
allows
models
to
run
in
a
compact,
mobile
30 Guava 490 71 95
enviroment
with
limited
scope
of
FLOPs.
The
work
process
31 Lychee 490 85 81
32 Mango 490 74 92
for
the
proposed
strategy
is
demonstrated
in
Fig.
1. 33 Mango Red 426 69 73
34 Orange 479 78 82
35 Pineapple 490 96 70
36 Pineapple Mini 493 82 81
37 Raspberry 490 77 89
38 Redcurrant 492 90 74
Fig. 1: Workflow for this experiment 39 Strawberry 492 76 88
40 Strawberry Wedge 738 118 128

A. Data
The
dataset
utilized
for
model
training
is
the
Fruits
360
data

[15].
This
is
an
extensively
utilized
dataset
for
the
grouping
of

organic
products
in
a
deep
learning
approach.
It
contains
top-
notch
surveyed
pictures
of
131
classes
altogether.
We
have

restricted
the
number
of
classes
to
41,
to
keep
up
with
the

extent
of
the
research.
The
rest
of
the
classes
were
discarded
as

a
majority
of
classes
in
this
dataset
contains
vegetable
images.

Also,
the
41
classes
were
manually
selected
with
respect
to

their
prevalence
when
compared
to
the
other
rare
classes

of
fruits.
The
quantity
of
images
is
29,347
altogether.
This

incorporates
training
and
test
directories
previously
designed

by
the
authors.
The
images
were
captured
from
6
different

cameras,
the
pictures
were
caught
in
a
rotational
premise,

to
prepare
models
with
a
feature
evaluation
ability.
The
total

number
of
pictures
per
class
is
referenced
in
Table
I.
average CPU. This enhances the opportunity for experimenting
with extremely huge datasets, like the data identified with our
proposed investigation.
C. Data pre-processing
The images in the various classes of fruits were high-
dimensional,
perfectly
suitable
to
assess
features,
fundamental

for
model
building.
The
pictures
were
initially
of
size
100
x

100.
They
were
resized
to
32
x
32
to
forestall
information

regarding
input
layer
criteria
in
ShuffleNet
V 2
architecture.

The
images
were
converted
from
BGR
channel
encoding
to

RGB
to
get
to
access
information
from
the
OpenCV
library.

The
pixel
values
were
standardized
to
range
0-1.
The
final

number
of
images
before
model
training
is
shown
in
Table
II.

B. System specifications TABLE
II:
Number
of
Images
after
Data
Pre-processing
The model was created utilizing a single 12GB NVIDIA
Training Testing Validation
Tesla K80 GPU. It provides 1.8TFlops 12GB RAM with 4992
Set Set Set
CUDA cores, which makes the entire calculation time 10x
quicker than on a typical CPU. The training data had high No. of images 22,232 3,615 3,500
batches of information, hence GPU was preferred over an

164
ISBN: 978-1-7281-6882-1 PART: CFP20P52-ART
Authorized licensed use limited to: KIIT University. Downloaded on February 10,2022 at 03:48:35 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of 2020 IEEE Applied Signal Processing Conference (ASPCON)

D. Model Architecture
A
light-weight
CNN
architecture,
ShuffleNet
V 2
w as
used

as
the
architecture
of
the
model
training.
The
perception

behind
selecting
this
model
is
that
it
incorporates
feature
re-
usability
with
low
trainable
parameters
for
large
data-sets
like

the
Fruits
360
dataset.
This
reduces
model
complexity
and

training
time,
with
a
state
of
the
art
model
performance
and

evaluation
assessment
parameters.
Parameters
like
scale
factor,
pooling,
num
shuffle
u nits,
bot-
tleneck
ratio
were
set,
the
number
of
classes
was
41.
The

number
of
hidden
layers
was
rectified
w ith
c ustom
inputs

using
hyperparameter
tuning
to
get
the
optimum
results
from

Fig. 2: Accuracy and Loss graph for training and validation
the
model.
The
network
was
created
by
stacking
6
building

for ShuffleNet V2
blocks,
each
containing
two
convolutional
layers.
No
layers

were
frozen
during
the
model
execution.
For
each
activation

layer,
’ReLU’
function
was
used,
followed
by
a
final
’softmax’

layer,
with
41
neurons,
corresponding
to
the
number
of
output
E. Result Analysis
classes.
The
optimizer,
Adam
[16]
was
used
for
the
error
rate
The
ShuffleNet
V2
model
was
evaluated
on
the
test
set.
The

reduction.
The
equations
for
update
weights
are
mentioned
model
correctly
classifies
3 ,479
i mages
o ut
o f
3 ,615
images

below. in
the
test
set.
Therefore,
the
test
accuracy
achieved
was

Initialisation of weights: 96.24%.
The
confusion
matrix
denotes
the
class-wise
detection

performance
of
the
model,
as
shown
in
Fig.
3.
ρm ← 1, ρv ← 1, m ← 0, v ← 0 (1)
Update rules for Adam Optimiser:
ρm ← βm ρm (2)

ρ v ← β v ρv (3)

m ← βm m + (1 − βm )∇w J (4)

v ← βv v + (1 − βv )(∇w J  ∇w J) (5)
√
m 1 − ρv
w ← w − α( √ ) (6)
v +  1 − ρm
where
m,
v
represents
first
and
second
moment
vector,
re-
spectively.
Similarly,
βm,
and
βv
represents
exponential
decay

rate
for
first
and
second
moment
vector,
respectively.
ρm
and
ρv

specifies
the
adaptive
learning
rate
time
decay
factor.
This

parameter
is
similar
to
momentum
and
relates
to
the
memory

for
prior
weight
updates.
α
in
Eq.
6
represents
learning
rate
or

step
size.
∇wJ
represents
gradient
of
cost
function,
J.

in
Eq.
6

is
a
small
value
to
prevent
division
by
zero
condition.
The
Eq.
1

denotes
initial
weights
before
update,
and
Eq.
2
to
Eq.
5
denote

conditions
obeying
weight
and
bias
change.
The
final
change
in

weight
parameter
is
shown
in
Eq.
6.
In
Eq.
5,

refers
to

element-wise
multiplication
and
in
Eq.
6,
the
operations
under

the
root
are
also
handled
element-wise. Fig.
3:
Confusion
Matrix
for
ShuffleNet
V2
Adam
optimizer
is
a
blend
of
RMSProp
and
AdaGrad
with

moments.
This
fuses
in
changing
the
primary
moving
average
Table
III
contains
Recall
and
Precision
values
in
classifying

to
Nesterov
accelerated
momentum,
along
these
lines
radically
the
test
data,
class
wise.
All
the
classes
had
high
chance
on

uniting
to
global
minima
in
any
event
train
time.
In
the
getting
predicted,
with
no
interference.
The
Receiver
Operat-
particular
dataset,
Adam
outperformed
all
different
optimizers,
ing
Characteristics
(ROC)
scores
of
the
model
was
calculated

with
its
refreshed
weights. to
be
99.64%,
suggesting
the
model
had
an
enhanced
chance

The
model
was
trained
on
30
epochs.
The
training
and
of
locating
the
best
threshold
while
categorizing
classes
based

validation
accuracy
and
loss
are
shown
in
Fig.
2. on
their
feature
differences.

165
ISBN: 978-1-7281-6882-1 PART: CFP20P52-ART
Authorized licensed use limited to: KIIT University. Downloaded on February 10,2022 at 03:48:35 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of 2020 IEEE Applied Signal Processing Conference (ASPCON)

TABLE III: Precision and Sensitivity scores of ShuffleNet V2 label
in
Fig.
4
indicates
mis-classification,
prediction
of
an

model Apple
Pink
Lady
label
which
was
actually,
an
image
label

of
class
Apple
Crimson
Snow.
The
Precision
of
Apple
Pink

Class
Class Precision Sensitivity Lady
class
(index
6
on
Tab.
III)
and
the
Sensitivity
of
Apple

index
Crimson
Snow
class
(index
1
on
Tab.
III)
have
values
0.92

0 Apple Braeburn 1.00 1.00 and
0.93
respectively.
Moreover,
the
features
of
the
classes

1 Apple Crimson Snow 1.00 0.93 were
very
closely
related.
Therefore,
the
model
had
a
slightly

2 Golden Apple-I 1.00 1.00 lower
chance
of
predicting
the
class
label
of
this
test
image

3 Golden Apple-II 1.00 1.00 when
compared
to
the
rest
of
the
classes.
4 Golden Apple-III 1.00 1.00
5 Granny Smith Apple 1.00 1.00
6 Apple Pink Lady 0.92 1.00
7 Red Apple-I 1.00 1.00
8 Red Apple-II 1.00 0.91
9 Red Apple-III 1.00 1.00
Fig. 4: Prediction results (if predicted label == true label: color
10 Red Delicious Apple 1.00 1.00
= green, else color = red)
11 Red Yellow Apple I 1.00 1.00
12 Red Yellow Apple II 1.00 1.00
13 Banana 1.00 1.00 C ONCLUSION
14 Banana Lady Finger 1.00 1.00
Convolutional neural networks and Image Processing have
15 Banana Red 0.90 1.00
been widely used to address the organic product classification
16 Cherry I 1.00 1.00
problem. We introduce a classification and analysis approach
17 Cherry II 0.98 1.00
of fruits using ShuffleNet V2, a pre-trained convolutional
18 Cherry Rainier 1.00 1.00
neural network architecture. The experiments showed a high-
19 Wax Black Cherry 1.00 1.00
performance analysis using several model assessment parame-
20 Wax Red Cherry 1.00 1.00
ters. The accuracy and misclassification range is optimized to
21 Wax Yellow Cherry 1.00 1.00
a very high extent in terms of previous methods involved, with
22 Grape Blue 1.00 1.00
an advanced step to assess the fruit classification problem.
23 Grape Pink 1.00 1.00
24 White Grape 1.00 1.00 R EFERENCES
25 White Grape II 1.00 1.00 [1] S. Arivazhagan, R. N. Shebiah, S. S. Nidhyanandhan, and L. Ganesan,
26 White Grape III 1.00 1.00 “Fruit recognition using color and texture features,” Journal of Emerging
27 White Grape IV 1.00 1.00 Trends in Computing and Information Sciences, vol. 1, no. 2, pp. 90–94,
2010.
28 Grapefruit Pink 1.00 1.00 [2] H. M. Zawbaa, M. Abbass, M. Hazman, and A. E. Hassenian, “Au-
29 Grapefruit White 1.00 1.00 tomatic fruit image recognition system based on shape and color
30 Guava 1.00 0.98 features,” in International Conference on Advanced Machine Learning
Technologies and Applications, pp. 278–290, Springer, 2014.
31 Lychee 1.00 0.98 [3] S. Bargoti and J. Underwood, “Deep fruit detection in orchards,” in 2017
32 Mango 0.99 1.00 IEEE International Conference on Robotics and Automation (ICRA),
33 Mango Red 1.00 1.00 pp. 3626–3633, IEEE, 2017.
[4] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time
34 Orange 0.93 1.00 object detection with region proposal networks,” in Advances in neural
35 Pineapple 1.00 1.00 information processing systems, pp. 91–99, 2015.
36 Pineapple Mini 1.00 1.00 [5] S. Puttemans, Y. Vanbrabant, L. Tits, and T. Goedemé, “Automated
visual fruit detection for harvest estimation and robotic harvesting,” in
37 Raspberry 1.00 1.00 2016 Sixth International Conference on Image Processing Theory, Tools
38 Redcurrant 1.00 1.00 and Applications (IPTA), pp. 1–6, IEEE, 2016.
39 Strawberry 1.00 1.00 [6] R. Barth, J. IJsselmuiden, J. Hemming, and E. J. Van Henten, “Data
synthesis methods for semantic segmentation in agriculture: A capsicum
40 Strawberry Wedge 1.00 0.94 annuum dataset,” Computers and electronics in agriculture, vol. 144,
pp. 284–296, 2018.
[7] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification
with deep convolutional neural networks,” in Advances in neural infor-
The model had an average precision of 0.993 and sensitivity mation processing systems, pp. 1097–1105, 2012.
of 0.992, therefore the average F Score was evaluated as 0.993. [8] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan,
The low validation loss ensures a very low chance of model V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,”
in Proceedings of the IEEE conference on computer vision and pattern
overfitting. recognition, pp. 1–9, 2015.
A test-prediction visualization was generated to mark labels [9] K. Simonyan and A. Zisserman, “Very deep convolutional networks for
with green color if the prediction label matches true label, large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
[10] J. Dai, Y. Li, K. He, and J. Sun, “R-fcn: Object detection via region-
red if not. Out of 41 classes, the model had a high chance based fully convolutional networks,” in Advances in neural information
of predicting correct labels as demonstrated in Fig. 4. A red processing systems, pp. 379–387, 2016.

166
ISBN: 978-1-7281-6882-1 PART: CFP20P52-ART
Authorized licensed use limited to: KIIT University. Downloaded on February 10,2022 at 03:48:35 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of 2020 IEEE Applied Signal Processing Conference (ASPCON)

[11] N. Sünderhauf, C. McCool, B. Upcroft, and T. Perez, “Fine-grained
plant classification using convolutional neural networks for feature
extraction.,” in CLEF (Working Notes), pp. 756–762, 2014.
[12] S. Naik and B. Patel, “Machine vision based fruit classification and
grading-a review,” International Journal of Computer Applications,
vol. 170, no. 9, pp. 22–34, 2017.
[13] N. Ma, X. Zhang, H.-T. Zheng, and J. Sun, “Shufflenet v2: Practical
guidelines for efficient cnn architecture design,” in Proceedings of the
European Conference on Computer Vision (ECCV), pp. 116–131, 2018.
[14] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely
connected convolutional networks,” in Proceedings of the IEEE confer-
ence on computer vision and pattern recognition, pp. 4700–4708, 2017.
[15] H. Mureşan and M. Oltean, “Fruit recognition from images using deep
learning,” Acta Universitatis Sapientiae, Informatica, vol. 10, no. 1,
pp. 26–42, 2018.
[16] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
arXiv preprint arXiv:1412.6980, 2014.

167
ISBN: 978-1-7281-6882-1 PART: CFP20P52-ART
Authorized licensed use limited to: KIIT University. Downloaded on February 10,2022 at 03:48:35 UTC from IEEE Xplore. Restrictions apply.