PROPOSAL

TRANSFER LEARNING IMAGE CLASSIFICATION USING

RESNET-50 BASED CNN MODEL TO DETECT AND OBSERVE
NUTRITIONAL DEFICIENCY
IN APPLE PRODUCE

Fadel Thariq Gifari

18/423103/PA/18186

Undergraduate Program in Computer Science

Department of Computer Science and Electronics
Faculty of Mathematics and Natural Sciences
Universitas Gadjah Mada
Yogyakarta
2023
 1

APPROVAL PAGE

BACHELOR’S THESIS PROPOSAL

Transfer Learning Image Classification Using

ResNet-50 based CNN model to detect and Observe
Nutritional Deficiency in Crop Produce

Proposed by:

Fadel Thariq Gifari

18/423103/PA/18186

Supervisor:

Dr. Nur Rokhman,S.Si., M.Kom

Secondary Supervisor:

Azhari , Drs., MT., Dr

 2

TABLE OF CONTENTS

Approval Page...................................................................................................1
Table of Contents..............................................................................................2
List of Figures...................................................................................................3
Abstract............................................................................................................ 4
1. Introduction...........................................................................................5
1.1 Background.................................................................................................5
1.2 Research Problem.......................................................................................6
1.3 Research Scope.......................................................................................... 7
1.4 Research Objective...................................................................................7
1.6 Research Advantages............................................................................... 7
1.7 Proposal Organization.................................................................................. 8
2. Literature Review...................................................................................9
3. Basic Theories...................................................................................... 15
3.1 Image Classification using Transfer Learning.......................................... 15
3.2 ResNet-50 Implementations and Architecture:.......................................... 16
3.3 Fruits 360 Dataset.................................................................................. 17
3.4 Data Preprocessing for images.................................................................. 19
3.4.1 Histogram Equalization..................... 20
3.4.2 Affine Transformations.................................................. 20
3.5 Optimization Methods for ResNet-50............................................. 21
3.7 Example of setting parameters.............................................. 21
3.8 Example of Importing the required libraries.............................................. 22
3.9 Example and Preprocessing specifics.............................................. 23
4. Research Methodology........................................................................25
4.1 Research Description................................................................................25
4.2 Research Stages........................................................................................26
4.3 Research Workflow...................................................................................28
4.4 Dataset Preprocessing.............................................................................. 28
 3
4.4 Google Collab usage................................................................................30
4.6 Implementation and Tools.................................. 34
4.6.1 Tools and Materials........................................................................ 30
4.6.2 Implementation of Machine Learning Models of ResNet-50.............. 31
4.6.3 Model Evaluation Plan....................................................................31
5. Research Schedule............................................................................... 32
References.........................................................................................................33

LIST OF FIGURES

Figure 1.1 : Confusion Matrix 10

Figure 3.1 :Image detection on horses 16

Figure 3.2 :Comparison between VGG-16 and ResNet 50 17

Figure 4.1 : Research Stages 29

Figure 4.2 : Research Workflow 31

LIST OF TABLES
Figure 1.1 : Confusion Matrix 10

Figure 3.1 :Image detection on horses 16

Figure 3.2 :Comparison between VGG-16 and ResNet 50 17

Figure 4.1 : Research Stages 29

Figure 4.2 : Research Workflow 31

 4

ABSTRACT

Transfer Learning Image Classification Using ResNet50

based CNN model to detect and Observe Nutritional
Deficiency in Crops
By:

Fadel Thariq Gifari

18/423103/PA/18186

Agriculture is an everlasting industry that is essential in the survival of

humans in its entirety,it also generates products that will decrease the amount of
poverty that exists within this world. According to Worldbank.org Statistics infer
that “25% of GDP” is of agriculture in countries that are in the “developing”
phase.There are several factors that will result in the ramification of Agriculture
Produce, including the main factors which are : weather and climate
change,viability of the land,the level of synchronization the chosen crops has
with the soil, to simulate all the data acquired is proven to be difficult, but quite
detrimental to the outcome for the Industry, due to the instability of the factors
listed beforehand. Utilizing multiple pre-trained models as reference for transfer
learning using a new model on a fruits dataset allows for a better understanding of
fruit nutritional deficiency, without having to manually scan it with unpredictable
variables.A suitable architecture or mode would be a ResNet 50 model coupled
with CNN which has high accuracy for image classification in particular.

Keywords: high accuracy, VGG-16, CNN, models, nutritional deficiency, machine learning, variables, transfer learning.
 5

CHAPTER 1

INTRODUCTION

1.1 Background
In researching Agricultural progression for fruits specifically, nutrition such as
potassium,calcium,and nitrogen are detrimental in producing healthy crops from
well attended plants.Using image classification will allow the process of
identifying a type of deficiency on a crop via the images of the leaves itself will
shorten the time required to produce a counter plan for the owner of the plant to
increase its nutrients level. A simulation of all unlabeled data from an available
dataset using an untrained model that will eventually be trained using the same
dataset.
Existing research based on crop image classification existed, commonly
used to detect the overall ripeness of the produce and if it is rotten or fresh and to
gauge the ripeness of the fruit based on its chlorophyll color. As for this research
which aims to analyze the leaves for the deficiency based on its colorization
,which will provide a basis of criteria for th e model to train on segmenting the
colors of the leaves for the nutrient deficiency classification.
Regarding the results for past researches related to image classification for
crops or its plants, there has been a trend of a comparison between two statuses of
the plant or crop, either ripe or unripe,rotten or fresh,viable or unviable for the
land and so forth, but there is little to none , when it comes to observing the
natural discrepancy of the produce, using its original leaves by utilizing
Algorithms such as CNN and ResNet 50, but rather than entirely creating a new
model from scratch, utilizing a pre-trained model as a basis for the new model and
the available dataset filled with pictures of different leafs with multiple
colorization on the surface.
 6

1.2 Research Problem

For Research methods and complications regarding image classification in
Agricultural uses, it is prevalent that previous researches touched on the
landscaped identification of an apple’s viability using a dataset that contains a
close up view that shows the intermittent colors on the different types of apples.
To put it simply a specification of result based on colorization of the leaf, in this
case nutrient deficiency classification that focuses on parts of the images on the
leaf based on a variety induced dataset with multiple leafs and crops all from
differing environments.
Although this research aims to be more specific, previous research that
focused on the colorization will prove to be useful in creating a reference point
for the intensity of filters used in the model that will then translate into the
classification of each nutrient, and in each iteration the overall accuracy will be
heightened.

1.3 Research Scope

With this Research it aims to contribute in creating a shortcut in nutrient
detection and classification on different type of leaves for different types of crops,
that will potentially benefit farmers in providing sufficient recovery options for
the crop on the plot of land.Although this Research uses already provided dataset,
the model that is created primarily and initially from transfer learning via ResNet
50 architecture will be able to reduce the epoch loss and increase accuracy using a
CNN esque Algorithm to classify each loss of nutrient within a leaf.
ResNet 50 is applicable in this sense due to its adaptable nature with
multiple libraries suitable for image classification such as numpy ,keras, and
mahotas that deals with image scaling and transformation for increasement of
segmentation between the images.
 7

1.4 Research Objective

The primary objective of this research is to create a model that adheres to to
a sustainable and modifiable result of classification, furthermore the simplicity of
the results will allow the model to improve its accuracy overtime with a specified
dataset rather than experimenting with a variety of datasets that does not meet the
criteria for the nutrients colours.The model is expectantly to have a reduced epoch
loss and have an accuracy between 70% and 80 %.
It is also important the the data preprocessing phase extract the desired
features such as colors and leaf shapes, followed by resizing each image
uniformly in accordance with the initial filters used to highlight the colour
anomalies.Blurry/Unclear images will also be augmented using pixelating
techniques that expand a select few of the images and finally smoothing the edges
for increased recognition.

1.5 Research Advantages

One of the main benefits procured from this research is regarding a

pseudo-digitized solutions for a short term classification of each nutrient that has
their own unique properties characteristics when it is lacing in a plant and its
produce.The model has the potential to be a basis for the initial phase of similarly
induced applications that has a goal relating to agricultural produce, while
requiring detection of a certain aspect of its overall growth and properties.

From there the results of this model can stem into multiple uses such as an
automated system that creates a step by step process into restoring the health of
the plant or a personalized application open to public use for personal
greenhouses and house plants.It is also possible to create a commercially
introductory businesses that provide the nutrients need based on what the owner
has gain knowledge of.
 8

1.6 Proposal Organization

This proposal will provide Chapter II, containing all the literature review
that are comparable to this research topic, followed by Chapter III that discusses
the initial methods and theories to pre process the data and create the
model,furthermore Chapter IV will delve into the specifics of the methods that are
to be used and implemented throughout the course of the model and its result, and
finally Chapter V provides the research schedule in order until the final result is
admitted.
 9

CHAPTER 2

LITERATURE REVIEW

The simplest of Convolutional Neural network revolves around using

tensorflow in order to detect and classify certain images, utilizing hidden layers
between the input and output layer to produce a filter.The layers itself includes
aspects that uphold its structure.To create the hidden layer, a kernel size of
1x1,2x2,3x3,4x4 for Small images and 5x5 for larger images has to be set in order
to properly set the detection parameters for the inputted picture for further
accuracy in detecting the anomalies within each picture,after setting the Kernel
Size a padding is required to Uniformize the pixel existing around and within the
picture.The last one is Stride that mainly decreases the amount of pixel
implemented, while also reducing the amount of time require to finish the training
data,finally a Pooling Layer to lower the size of the image in order for the image
to be used in future layers to reduce the discrepancy between the image and the
layer.

For Agriculture, using the Convolutional Neural Network for its economic
and practical analysis is highly intensified in the industry. On the practical side,
Convolutional Neural Network is able to detect the anomalies within a specific
crop field and segment all the desired results, in order to classify the percentage of
each factor revolving the practice of agriculture, specifically to identify the
components that increase or decrease the overall growth of a crop field.For
example in a research conducted by Cambridge University A set of Aerial Photos
were taken of an abandoned Plantation Field, using a training data ratio of 85:15 ,
it was possible to remotely label and differentiate SugarCane from Soil on the
plantation field, while the miscellaneous items were labeled as others.
 10

Figure 1.1 Confusion Matrix of Land

From the result, it can be deduced that the matrix displays labels in an
orderly manner,Data as such can be applied in researching an adaptable and
feasible land, suitable for massively produced crops(Kamilaris,2018). There is a
segmentation between pictures to test the overall accuracy and loss rate of the
CNN model, it is highly admissible to create a program that has all of the
attributes for a CNN and a sample dataset unrelated to the topic to avoid any
baseline errors, in addition, multiple parameters that were defined including
sigmoid and feedforward functions that increases the overall efficiency of the
model before training it iteratively, by defining each parameter individually, it
will allow the program to predict the overall accuracy of the model when
presented with 3 classes containing 100 images each.While also using a form of
kernel that allows each image to be separated individually. With the goal of
preventing errors during classification.

In Essence,represents the CNN model’s epoch compilations.From these

results it can be summarized that the cnn model is able to procure the proper
results from this sample datasets to be used for further research, and proven to be
viable for image classification due to its low epoch loss and high accuracy while
also maintaining its original model without any drawbacks. The reason this model
was used is due to the similarity of factors to be detected within that of weather
forecast images, including the air currents, heat precipitation, and rain
 11

precipitation that plays a role in the overall growth of the crops. According to a
research done by Dong- a University that focused on the contents within the
crops, mainly Tomatoes, to determine nutrient deficiency within the crops.

Table 1.1 Deficiency table

The Table “1.1” above indicates the deficiency between the three
micronutrients are prevalent in the tomatoes,by adding multiple Convolutional
layers , in supervised learning the data has been labeled for each form of tomato,
to indicate its deficiency.According to the researchers, in order to set the proper
boundaries for the Image Segmentation to successfully Operate,first it is required
to “tune the Inception‐ResNet v2 model based on the CNN architecture”
(Tran,2019,p.6),followed by the implementation of normalization in order to
display the vanishing gradient prominent within the specified images.The objects
that are displayed within the image are the Tomato fruit and Leaves. Figure 2.2
For the experimental results, the images showed that the nitrogen deficiency is
detected only in the tomato leaves, while calcium and other nutrients are found on
the Tomato Fruit itself with the accuracy resulting in 87.27 % , with each of the
values producing its own individual maximum value of 0.989,0.999, and 1.0
(Tran,2019,). The results of the supervised learning proved to be substantially
accurate, all the data acquired from the Tomato Research is viably utilized by
farmers, to determine what type of tomato seed is preferable compared to the
other existing ones,from this it will be possible to set the tomato seeds so that it
will result in a hybrid produce, that will improve and avoid any deficiency
evolving within the tomatoes.

Another research focused on detecting the ripeness of fruits titled: “

Implementation of Transfer Learning Using VGG16 on Fruit Ripeness Detection”
by researchers from ITB. The VGG model is regarded as the most prevalent deep
learning architecture, as mentioned in the sub-chapter on deep learning. the
VGG16 model's weight, which was utilized to feature-extract data
 12

from the fruit ripeness dataset. Because there are only 140 photos in each dataset
for each image category, the Data Augmentation procedure is carried out using
ImageDataGenerator from the Keras library in order to reduce overfitting. ReLu
activation is the activation method employed in the MLP block. The MLP block
output layer employs the 8 class softmax activation function. The second design
uses the VGG16 model and Batch Normalisation for transfer learning.With the
architecture outlined in the proposed transfer learning sub-chapter, this research
demonstrates the impact of regularization approaches on transfer learning in
decreasing overfitting of the fruit maturity dataset. For picture training, validation,
and testing, there are 100, 20 and 20 photos for each category, respectively. The
utilized ModelCheckpoint's monitor = val accuracy, mode = max property. Table
1 compares the accuracy, precision, recall, and F-Measure values for dropouts
between 0.3 and 0.7.

The recommended performance only achieves an accuracy value of 84

percent, according to the testing findings on the same dataset. The Dropout and
Batch Normalisation procedures are the same, except the Regularizer kernel is
used in place of the Regularizer type. The next research is regarding Fruit
Classification using SVM instead of a traditional CNN method for image
classification.

The next research focused on the classification regarding a dataset that

contained 2800 images of flowers that are to be categorized in 4 classes namely
Dandelion,Daisy,Sunflower and Tulip.The dataset then is split into percentages
for Training it is 60% , 20% is allocated for Validation , and Testing is presented
with 20%.Although the dataset was ready to be inputted to the Architecture, the
researchers opted to Augment the images first with methods such as rotation,
shifting , resizing rescaling , and zooming. Before being coupled to a multilayer
perceptron (MLP) classifier, VGG16 consists of five convolutional blocks.

The early and bottom layers of a pre-trained model architecture simply

encode general, reusable characteristics, whereas the upper levels encode
specialized features based on the learned dataset. Retraining VGG16 is intended
to help the model become more specialised to certain characteristics in the
dataset. Thus that is the reason why In this study, the model architecture is a
16-layer VGGnet (VGG16). The pre-trained architecture is obtained using the
transfer
 13

learning technique from the ImageNet dataset. 1,000 distinct classes and 1,000
different levels of spatial hierarchy are included in the 1.4 million tagged photos
in ImageNet. Pre-trained networks can serve as good generic models of the visual
world since they are trained on such a huge and diverse set of data involving
diverse flowers. All the VGG16 blocks are connected to three ML P layes. Two
hidden layers and one output layer make up the three MLP layers. Two hidden
layers employ a rectified linear unit (ReLU) activation function with a dropout
rate of 40%. Four nodes make up the output layer, each of which directly
represents one of the four classes and has a softmax activation function. The
stochastic gradient descent (SGD) optimizer is used to train the model
architecture, with a learning rate of 5e-3 (0.005) and a total of 40 epochs.

Table 2.1 : Comparison table of different research projects and techniques

using CNN and Transfer Learning methods.

Researcher(s) Goal Method Computational Accuracy

Time

A. Kamilaris et al. Reviewing CNN CNN undefined 98%

(2018) use in Agriculture

Mohamad Aqib Plant Classificatio n VGG16,Data Testing sets = 28 99%

Haqmi Abas et using VGG16 Augmentation,
al. (2018) Transfer Learning Transfer Learning
and data
augmentatio n

Zhongqi Miao et al. A Transfer Avg = 53.833s 83%

(2018) comparison of Learning,
visual features VGG-16,
used by humans ResNet 50, Image
and machines to Classification
classify wildlife
 14

Purbaditya et al. Deep Learning undefined Undifined

Detection of
(2018) Extraction
Harbour
Porpoise
with
Low-Level
Feature
Extraction
and Deep
Learning
Based
Classificati
on

.
 15

CHAPTER 3

BASIC THEORIES

In this proposed chapter all the pre-ordained theories regarding the

Orchestration of the Architecture and how Transfer Learning will be implemented
from the training of the dataset up until the testing.In the first section which is
section 3.1 the discussion about transfer learning in image classification is
presented, followed by section 3.2 that mentions and explains the procedures for
transfer learning in CNN and deep learning.3.3 will discuss the steps of data pre
processing and augmentation, next is section 3.4 that contains the comparison of
which architecture is best for this research’s dataset.the final section will be 3.5, it
will explore the effectiveness of ResNet 50 Pre trained model in accordance with
similar datasets in comparison with other Pre trained models such as VGG-16 and
InceptionV3

3.1 Image Classification using Transfer Learning

Transfer learning has been introduced by computer vision as a way to speed

up the process of finalizing a model using pre-trained models instead of creating
one from scratch.Therefore it is fair to explore the origins of transfer learning and
its improvements over the years of existing in image classification. Specifically
VGG-16 that is commonly trained with ImageNet using Flower datasets thats
similar to the leaf dataset that research aims to utilise.It is also important to note
that in image categorization, convolutional neural networks are often utilised.
Transfer learning has made it feasible to train the model with constrained
resources using the pre-trained models. By adding more layers that are tailored to
the job, it does assist to concentrate on collecting more precise qualities. Results
from a model run on 10 randomly selected samples with five iterations
demonstrate 90% accuracy.
 16

As for transfer learning using Keras (ResNet50) it has better sensitivity than
VGG-16 regarding the edges of an object as shown below.

Figure 3.1 The Comparison shows that ResNet50 has an advantage

While reading about Transfer Learning comparison with ResNet 50 and

VGG-16 it is important to note that both features a different hierarchical
clustering to identify, which object corresponds to what features as shown within
the visualization below.
 17

Figure 3.2 Comparison between VGG-16 and Resnet-50

The Image above indicates that there are some features that are similar
within the object, therefore each architecture responds in accordance to their
accuracy and inner workings. It can be said ResNet-50 is conditionally better for
this research due to the fact that it picks up more features on the animal, than the
latter architecture.

3.2 ResNet-50 Implementations and Architecture

All of the implementations relating to ResNet-50 has 152 layers and is

catered more towards deep learning, while avoiding human error as often as
possible. The unique element of ResNet is to utilize a skip connection in order to
backpropagate the gradient to the earlier apparent layers.This way the ResNet
blocks will be allowed to stack on top of each other, which is effective and
required for deep learning/transfer learning.There are several blacks in ResNet
that are important in managing all the blocks in between, for example the identity
block provides the same dimension as the output activation, meaning no changes
are made in the input and output.Next is the convolutional block, its role is to use
 18

a CONV2D layer to match up the inputs and outputs In detail the ResNet-50
model contains 5 stages that consists of both a convolution block and an identity
block,each has 3 convolutional layers that accommodate their functions.

Figure 3.3 a Visualization of ReLU layers

As seen above there are multiple layers such as ReLU and others that take
into account all the functions ResNet-50 can and are able to do.
 19

3.3 Fruits 360 Dataset

The Fruits 360 Dataset provides all the necessary content that adhere to this
research, ranging from a variety of fruit images that has a 360 point of view for
each of the object.Each images is 100x100 pixels all uniformed followed by a
training set size of 67692 images and a test set size of 22

Figure 3.4 Fruits 360 Dataset

It can be inferred from the dataset above that each fruit is labelled
differently based on its species and angle.That will allow the ResNet algorithm to
review its accuracy based on each angle and filters top detect the nutrional
deficiency. In this dataset only the types of Apples will be taken into account such
as: Braeburn,Crimson Snow,Golden,etc.
 20

3.4 Data Preprocessing for images

For the Preprocessing stage of the dataset the images are transformed using
a multitude of functions such as the histogram equalization using the appropriate
functions and values.

● Transform the image using PBT (pixel brightness corrections) - using

histogram equalization [ pY (z)dz) = pY (y) = pX(T−1(y)) d dy
(T−1(y))].
● Modifying the positions of the pixels using affine transformation such as
scaling,translation,rotation,shearing.
● Interpolation methods will be used, specifically nearest neighbor
interpolation.
● For image enhancement to procure segmentation, the low pass filtering
will transform the images into a smoother state

3.4.1 Histogram Equalization

Histogram Equalization will allow the images in the dataset to improve its
own individual contrast, in essence it stretches out the overall intensity of each
image.The histogram will also assist the model in identifying the color balance in
the form of a graph.
 21

Figure 3.4 Histogram Equalization in graph form

As seen from the image above it can be inferred that an equalization helps
spread the red,green,blue aspect of an Image.Furthermore the specific equation in
which the equalization is calculated is a as follows.

In the equation above,let f be a given picture in computer graphics that is

represented as a mr by mc matrix with integer pixel intensities that range from 0
to L - 1. L, frequently 256, is the total number of potential intensity values. Let p
stand for f's histogram, which has a bin for each potential intensity.

3.4.2 Affine Transformations

Contrary to the conventional geometry unit circle of angles, which is

measured in counter clockwise rotation with 0 starting from the positive X axis,
affine transformation employs an angle of rotation that is clockwise. As a result,
you'll notice that the angle's negative value is frequently used. Using something as
advanced as the OpenCV library for this transformation, will provide a more
efficient and sustainable way of resizing and managing the images
 22

3.5 Optimization Methods for ResNet-50

The optimization method that is applicable for ResNet 50 is one that can be
divided into multiple parts, including but not limited to padding and
stride.Padding in ResNet-50 refers to when a fixed padding is commonly used
rather than a personalized one with customized padding. In order to prevent the
loss of pixels within the image, this formula is used in python for the model.

3.7 Example of Importing the required libraries

As shown in the above example, importing the libraries first for the normalization
and pre-processing of the dataset should be of utter importance, as the keras is
needed for standardization of each image in the dataset.
Next the training of the dataset would result in the form of phosphorous
deficiency that correlate to white on the apple, green for nitrogen ,and yellow for
iron.So for example each singular picture of an apple will result in the example
below:

Model Prediction : NITROGEN

YELLOW : 86%
WHITE: 7%
GREEN :7%
 23

Import keras
import numpy as
np import PIL
Import matplotlib
import tensorflow as
tf
from tensorflow.keras import
layers,Dense,Flatten from
tensorflow.keras.models import Sequential Pip
install col
 24

CHAPTER 4

RESEARCH METHODOLOGY

The suggested research concept, implementation, and methodology will be

covered in this chapter. The overall description of the planned research will be
found in Section 4.1. The computer requirements, programming languages, and
supporting libraries that will be utilized both for the study implementation
procedure and for comparison are described in Section 4.2. The steps used in the
suggested research will be described in Section 4.3. Finally, Section 4.4 will
describe the planned research's procedure and how its objective will be
accomplished.

4.1 Research Description

The Research explores the possibilities of using ResNet-50 for color

detection in order to gather nutritional deficiency within a fruit or vegetables that
are provided by the Fruits 360 dataset.Before the dataset is put into training, a
dataset preprocessing is required in order to resize,remove noise,and smoothing
out the edges of the picture.

In detail the model will detect the different colourization that are apparent
on the outside of the fruit, thus focusing only on its outer filtration of skin color
that corresponds to a certain nutrient.The percentage in which the the nutritional
deficiency is detected will be based on how frequent the decolorization is apparent
of the skin of the produce,and from there the model will classify either the
 25

produce is nutritionally deficient or not.

To start the research it is advisable to use Google Collab to import all the
necessary libraries, which in this case is Keras, with a Convolutional Neural
Network approach that will compare the accuracy of Training vs. Test data. The
overall performance of the model will be based on multiple iterations of its
training and test results, while also increasing its accuracy.

4.2 Research Stages

The stages are including but not limited to : Literature review, dataset
preparation, application design, model design, implementation, model
assessment,result comparison , and research report all in which will be included in
this 4.2 section.
 26

For the Literature study it is simply knowledge gathering of similar topics

that covers from its basics and architecture to its results.From the gathered
information, the methods and errors that each research used and experience can be
a point of reference for this research in order to optimize the model after cross
validation.

As explained before, the data preprocessing stage will simply be cleaning

the dataset images in a way that it will resize,filter,smooth its edges and overall
content, until all images are in uniform and ready to be implemented into the
model itself.

The next stage requires to use transfer learning from previously similar
iterated models, in order to create a sophisticated model that will do a new task
which is detecting colourization and decolorization, that will determine whether
or not the product falls under the nutritional deficiency category.
Optimisation will allow this research to produce a maximum and minimum
function for the model based on multiple iterations of the model.The Optimisation
method that will be used, is referred to as stride optimization.
Model deployment happens when the model is already as optimised as
possible with high accuracy results through multiple iteration and altering
optimization methods.
 27

4.3 Research Workflow

This research is divided into several stages consisting of data
preprocessing, application and model design, implementation, and model
evaluation. Details of the plan for each stage of the research will be discussed in
the next subsection. The research workflow to be carried out is summarized as a
flowchart in Figure 4.2.

4.4 Dataset Preprocessing

For the detection method to be as effective as possible, the dataset must first
undergo preprocessing. Assisting with categorical characteristics, performing
affine transformations, and performing normalization in the form of histogram
equalization are all required. If categorical characteristics cannot be discovered,
the picture will be placed into an equalized adaptive histogram

Scenario Accuracy on training set Accuracy on test set

Grayscale 99.92% 94.45%

RGB 99.87% 95.89%

HSV 99.89% 95.31%

HSV + Grayscale 99.85% 96.09%

HSV + Grayscale+ 99.60% 96.13%

hue/saturation change+flips

Table 4.3 : Comparison table of different research projects and techniques

using CNN and Transfer Learning methods.

The Example above shows an example of a research that focused on using

multiple filters/scenarios, and if the fruits will still be labeled correctly under
those circumstances, followed by its accuracy rate as the main important output.
 28

Normalisation is performed by using the First normal form. The table 4.3
below shows a brief example of the first normal form used in the dataset of this
research.

Standardization is performed only if needed in case there are values that

are

distant from the normal form. This is done by using the z-score conversion. The
following shows an example of z-score conversion used for standardization in
measuring with the order value in the dataset of this research. (average order = 16,
standard deviation = 2, current order = 15). Below is the basic equation for
Standardization:
 29

4.5 Google Collab usage

The model will mainly be created in Google Collab as it is most preferable when
creating a python based machine learning program,because it already has built in
libraries and commands for image formatting and classification.

4.6 Implementation

4.6.1 Tools and Materials

The development of this application and writing in this research will be
carried out using a computer with the following specifications.
1. Processor : 8th Gen Intel Core i7-8750H 2.2 GHz (up to
4.1 GHz)z
2. RAM : 16 GB
3. GPU : NVIDIA GeForce GTX 1060
4. Operating System : Windows 11 Home 64-bit v. 21H1

This research will use the following software and supporting libraries.

1. Python 3.10.0
2. Anaconda
2.1.1 3. NumPy
1.22.3
4. Keras 2.7
5. Matplotlib 3.1
6. TensorFlow 2.0
7. ResNet-50
8. Fruits 360 Dataset
 30

4.6.2 Implementation of Machine Learning Models of ResNet-50

Keras is the perfect simplified library to make use of all the preprocessing
techniques for each picture in the dataset. Implementing Machine Learning in
color detection using ResNet-50 will allow the model to systematically and
automatically classify which percentage of absence of color adhere to which
category of deficiency.

4.6.3 Model Evaluation Plan

The evaluation for the Model will be based on how many deficiencies are
constantly shown and detected within all the overall types of apples and its
subspecies. The percentage based on the consistency of deficiency taken from the
detection of the colors will be the basis on the accuracy of the model.

Table 4.4 : Proposed Nutritional deficiency results with ResNet-50

No. Model Indicator Success Rate

1 Transfer Learning Computational time seconds

ResNet-50 (millisecond)

Accuracy To be
researche
d

To be
Color researche
Ratio d
 31

Table 5.1 : Proposed Research Schedule

2023
Kegiatan May June July
1 2 3 4 1 2 3 4 1 2 3 4

Model and
Datasete Research

Research Libraries and

how to implement

Python code and

Dataset pre-processing

Test & training dataset

Model accuracy review

and efficiency

Thesis completion
overtime
 32

REFERENCES

Bank, W., 2021. Overview. [online] World Bank. Available at: [Accessed 18 April
2021]. Chui, M. et al., 2019. Notes from the AI frontier: Applications and value
of deep learning.

Bhattacharya, P., Wulf, S. and Zölzer, U. (2018) (PDF) detection of Harbour Porpoise with
low-level feature extraction ... Available at:
https://www.researchgate.net/publication/332902381_Detection_of_Harbour_Porpoise_w
i th_Low-Level_Feature_Extraction_and_Deep_Learning_Based_Classification
(Accessed: November 4, 2022).

Garg, A. (2022) Image classification using resnet-50 Deep learning model, Analytics
Vidhya. Available at:
https://www.analyticsvidhya.com/blog/2022/09/image-classification-in-stl-10-dataset-usi
n g-resnet-50-deep-learning-model/ (Accessed: November 3, 2022).

McKinsey & Company. Available at: [Accessed April 18, 2021]. Jan, B. et al.,
2017. Deep learning in big data Analytics: A comparative study. Computers &
Electrical Engineering. Available at: [Accessed April 18, 2021]. Kamilaris, A. &
Prenafeta-Boldú, F.X., 2018. A review of the use of convolutional neural networks
in agriculture. The Journal of Agricultural Science, 156(3), pp.312–322.

Kamilaris, A., Kartakoullis, A. and Prenafeta-Boldú, F., 2017. A review on the

practice of big data analysis in agriculture. [online]
Www-sciencedirect-com.access.library.uni sa.edu.au. Available at: [Accessed 18
April 2021].

Phung & Rhee, 2019. A High-Accuracy Model Average Ensemble of

Convolutional Neural Networks for Classification of Cloud Image Patches on
Small Datasets. Applied Sciences, 9(21), p.4500.

Sharma, M. et al., 2017. Effects of Reactive Oxygen Species on Crop Productivity.

Reactive Oxygen Species in Plants, pp.117–136. Tran, T.-T. et al., 2019. A
Comparative Study of Deep CNN in Forecasting and Classifying the Macronutrient
Deficiencies on Development of Tomato Plant. Applied Sciences, 9(8), p.1-19

Kudrov, M. (no date) Example of the resnet-50 result classification. | download scientific

... Available at:

 33

https://www.researchgate.net/figure/Example-of-the-ResNet-50-result-classification_fig5_

339903705 (Accessed: November 3, 2022).

SHARMA, A. (2021) Rice_disease_detection, Kaggle. Kaggle. Available at:

https://www.kaggle.com/code/f0rtaza/rice-disease-detection (Accessed: November

4, 2022).