USE OF ARTIFICIAL INTELLIGENCE FOR DETERMINATION OF

VARIOUS THINGS IN APPLE FRUIT PRODUCTION

Goran FRUK1,* , Ana Marija ANTOLKOVIĆ1, Tomislav KARAŽIJA2, Antonio VIDUKA2, Rea VRTODUŠIĆ1, Marko PETEK2,
1
Martina SKENDROVIĆ BABOJELIĆ
1University of Zagreb Faculty of Agriculture, Division of Horticulture and Landscape Architecture, Department of Pomology, Svetošimunska cesta 25, Zagreb, Croatia (gfruk@agr.hr)
2
University of Zagreb Faculty of Agriculture, Division of Agroecology, Department of Plant Nutrition, Svetošimunska cesta 25, Zagreb, Croatia

SUMMARY MATERIALS AND METHODS

Fruits are important sources of minerals and vitamins necessary for human health and in Photo material
some cases even a source of sugar for satisfaction and pleasure. Climate change and global Photos were taken in five production apple
transport in the last decades raised problems in fruit production regarding the faster spread orchard near Zagreb, Croatia (Čehi, Mičevec,
of pests and diseases and their ability to overwinter in new environments. On the other
Petrovina, Staro Čiče and Ribnica). Camera used
hand, a hardworking environment and tasks in agricultural production decreased the
for taking photos is custom made digital RGB
availability of manpower. The rapid development of information technologies (sensors,
machine learning, IoT, etc.) and the lack of manpower in agriculture empowered the
camera with CMOS sensor. Pictures were taken
development of new tools for fruit growers. In the last 10 to 15 years lots of new tools and in five intervals through the day (8-10h, 10-12h,
automatic detection were developed for use in agriculture. The aim of the RDI (Research, Figure 1 RGB camera 12-14h, 14-16h, 16-18h). A total of 11,602
Development and Innovation) project AgriArt comprehensive management system in the photos were taken.
field of precision agriculture was to develop a system for the automatic detection of various Processing photo material
things in apple fruit production. A total of 11,602 photos were taken and 153,154 All photos were subjected for annotating objects
annotations of 42 classes were annotated. The average precision of detection for classes (42 classes) on each photo for machine learning.
was calculated based on the confusion matrix. The best results were obtained with the For this study 30 randomly selected photos
detection of ‘Healthy leaf’ (0.74), ‘Juvenile fruit’ (0.72), ‘Leaf apple scab’ (0.70), and ‘Leaf were choosen and processed both by human
miner’ (0.68). Good results were obtained with the detection of ‘Flower cluster’ (0.63), ‘Fruit (trained expert) and artificial intelligence (AI) in
apple scab’ (0.63), ‘Leaf aphid’ (0.55), ‘Fruit’ (0.53), ‘Leaf mildew’ (0.52), and ‘Generative spur’ 32 classes.
(0.50). Other classes had average precision lower than 0.50 (‘Deformed fruit’, ‘Fruit frost Results analysis
damage’, ‘Fruit mildew’, ‘Spur woolly aphid’, ‘Undeveloped leaf’, ‘Juvenile leaf’) precision or a Machine processed (anotated) photos were
small number of annotations (‘Deformed leaf’, ‘Fruit monilinia,’ ‘N deficiency’ and ‘P
taken into confusion matrix analysis and
deficiency’) that are not suitable for this analysis. The usage of RGB cameras in combination
average precision was calculated.
with machine learning techniques and automatic detection shows promising results.
Keywords: machine learning, IoT, Malus x domestica, automatic detection, pest detection, Figure 2 AI annotated Comparison between number of annotations
disease detection, fruit damage detection photo made by AI and Human are shown.

RESULTS AND DISCUSSION 800

Confusion matrix and average precision (Table 1) showed the best 750

results with the detection of ‘Healthy leaf’ (0.74), ‘Juvenile fruit’ (0.72), 700

‘Leaf apple scab’ (0.70), and ‘Leaf miner’ (0.68). Good results were 650

obtained with the detection of ‘Flower cluster’ (0.63), ‘Fruit apple scab’ 600

(0.63), ‘Leaf aphid’ (0.55), ‘Fruit’ (0.53), ‘Leaf mildew’ (0.52), and 550

‘Generative spur’ (0.50). Other classes had average precision lower 500

than 0.50 or a small number of annotations (less than 1000) that are 450

not suitable for this analysis. 400

350
Table 1 Average precision and number of annotations 300
Average No. of Average No. of
Class Class
precision annotations precision annotations 250

Leaf apple
Deformed fruit 0.40 2431 0.70 8361 200
scab
150
Deformed leaf 0.39 740 Leaf mildew 0.52 2294
100

Flower cluster 0.63 18699 Leaf miner 0.68 8493 50

Mg/Fe
Fruit 0.53 2659 0.60 1451 0
deficiency Healthy Juvenile Fruit Leaf Leaf Fruit Red Red Red Green Green Spur Generati Flower Fruit DeformeDeforme Leaf N Mg/Fe Zn/Mn Fruit Leaf Old leaf JuvenileUndevel Fruit
leaf leaf apple apple mildew mildew apple A apple B apple C apple apple C woolly ve spur cluster frost d fruit d leaf miner deficien deficien deficien monilini aphid fruit oped
Fruit apple
0.63 4089 N deficiency 0.83 518
scab scab AB aphid damage cy cy cy a leaf
scab
AI annotations Human annotations
Fruit frost
0.43 4852 Old leaf mix 0.66 2408
damage Figure 2 Comparision of AI and Human annotations in 30 photos
Fruit mildew 0.43 1117 P deficiency 0.00 26

Fruit monilinia 0.72 251 Red fruit A 0.68 13164

Comparision of AI and Human detection on the same photos, showed interesting results. In general
Generative
spur
0.50 25067 Red fruit B 0.49 6643 AI and Human detection have similar trend. Thus is specially visible in detecting ‘Healty leaf’,
Green fruit AB 0.65 5818 Red fruit C 0.30 3973 ‘Flower cluster’ and ‘Fruit’. However in some important classes like ‘Apple leaf scab’, ‘Spur wooly
Spur wooly
aphid’, ‘Generative spur’ and ‘Leaf mildew’, there was big discrepancy between AI and human
Green fruit C 0.20 1277 0.45 2749
aphid detection. This discrepancy shows need for optimizing processes in automatic detection.
Undeveloped Unfortunately, some classes had low level of appereance thus there was not enough samples
Healthy leaf 0.74 18671 0.46 2741
leaf
Juvenile fruit 0.72 3137 Yellow fruit AB 0.60 391
annotation for machine learning(ML) nor for testing between AI and Human annotation.
Appereance of some of studied classes can be easely induced for poviding enough samples for ML.
Juvenile leaf 0.47 6578 Yellow fruit C 0.53 500
 
ZN/Mn
Leaf aphid 0.55 3347 0.31 461
deficiency

CONCLUSION ACKNOWLEDGEMENT
It was noticed various success in usage of AI in automatic detection of studied classes.  
Noticed lack of success in automatic detection of some classes (nutrition deficiency) is due This research was supported by European structural and investment funds
to low number of annotations provided for machine learning. through project AgriART comprehensive management system in the field of
precision agriculture (KK.01.2.1.02.0290)
In some classes (‘Leaf miner’ and ‘Leaf apple scab’, ‘Spur wooly aphid’, ‘Generative spur’) it
was noticed high difference in detection between AI and Human despite high number of
annotations for ML. Some of them at the same time showed low average precision
detection.
Usage of RGB cameras and automatic detection with artificial inteligence shows promising
results in detection in some classes (‘Flower cluster’ and‘Fruit’) while for other needs to be
optimized.

V Balkan Symposium on Fruit Growing, Zagreb, Croatia, June 18-21, 2023