Ecological Informatics 76 (2023) 102096

Contents lists available at ScienceDirect

Ecological Informatics
journal homepage: www.elsevier.com/locate/ecolinf

Recognition method of soybean leaf diseases using residual neural network

based on transfer learning
Miao Yu, Xiaodan Ma, Haiou Guan *
College of Information and Electrical Engineering, Heilongjiang Bayi Agricultural University Daqing 163319, China

A R T I C L E I N F O A B S T R A C T

Keywords: Leaf disease is an important factor restricting the high quality and high yield of the soybean plant. Insufficient
Soybean leaf disease control of soybean diseases will destroy the local ecological environment and break the stability of the food
Image processing chain. To overcome the low accuracy in recognizing soybean leaf diseases using traditional deep learning models
Deep learning model
and complexity in chemical analysis operations, in this study, a recognition model of soybean leaf diseases was
Transfer learning
Recognition model
proposed based on an improved deep learning model. First, four types of soybean diseases (Septoria Glycines
Hemmi, Soybean Brown Leaf Spot, Soybean Frogeye Leaf Spot, and Soybean Phyllosticta Leaf Spot) were taken as
research objects. Second, image preprocessing and data expansion of original images were carried out using
image registration, image segmentation, region calibration and data enhancement. The data set containing 53,
250 samples was randomly divided into the training set, verification set, and test set according to the ratio of
7:2:1. Third, the convolution layer weight of the pre-training model based on the ImageNet open data set was
transferred to the convolution layer of the ResNet18 model to reconstruct the global average pooling layer and
the fully connected layer for constructing recognition model of TRNet18 model. Finally, the recognition accuracy
of the four leaf diseases reached 99.53%, the Macro-F1 was 99.54%, and the average recognition time was
0.047184 s. Compared with AlexNet, ResNet18, ResNet50, and TRNet50 models, the recognition accuracy and
Macro-F1 of the TRNet18 model were improved by 6.03% and 5.99% respectively, and the model recognition
time was saved by 16.67%, The results showed that the proposed TRNet18 model had higher classification ac­
curacy and stronger robustness, which can not only provide a reference for accurate recognition of other crop
diseases, but also be transplanted to the mobile terminal for recognition of crop leaf diseases.

1. Introduction
Soybean disease is an important factor restricting the high quality
and high yield of the soybean plant (Cen et al., 2020; Meng et al., 2022).
Rapid characterization and detection of phenotypic traits for soybean
diseases have important theoretical significance and practical value for
variety selection, scientific cultivation, and fine management. Analysis
of relevant survey data showed that the yield loss caused by diseases
accounts for about 10% each year, and even more than 30% in severe
cases for the soybean plant (Chang et al., 2018; Guo et al., 2021). Real-
time monitoring and rapid recognizing of soybean diseases have become
extremely important to improve soybean quality and yield.
In recent years, deep learning models integrated with machine vision
technology have been widely applied in the recognition of crop diseases
(Atila et al., 2021; Dai et al., 2022; Feng et al., 2021; Li et al., 2020; Liu
et al., 2020; Schirrmann et al., 2021). A disease and insect recognition
model based on the deep learning method was established using 10 crop
diseases and pests in the open-source datasets using Kaggle1 and
Forestry2 as classification data (Zhao et al., 2022). Transfer learning
method was used to recognize and classify the disease images of maize
leaves, and apple leaves in the PlantVillage open dataset (Dong et al.,
2022; Liu et al., 2021; Zhao et al., 2021; Zhou et al., 2021). On the open
dataset, a residual network-based CDCNNv2 algorithm was proposed
using transfer learning methods and deep learning to classify the
severity of crop diseases and insect pests (Yu et al., 2020). On 19 types of
pepper disease and insect pests, a recognition method was proposed
based on transfer learning and deep feature diagnosis (Gu et al., 2021).
The IP102 agricultural pest and disease image dataset was used as the
experimental dataset to compare the classification accuracy of the
ResNeXt-50 model under different combinations of the learning rate,
transfer learning, and data augmentation (Li et al., 2022). Model
training based on large public datasets promoted the technical

* Corresponding author.
E-mail addresses: mxd@cau.edu.cn (X. Ma), gho@cau.edu.cn (H. Guan).

https://doi.org/10.1016/j.ecoinf.2023.102096
Received 2 December 2022; Received in revised form 31 March 2023; Accepted 1 April 2023
Available online 5 April 2023
1574-9541/© 2023 Elsevier B.V. All rights reserved.
M. Yu et al. Ecological Informatics 76 (2023) 102096

development and innovation of traditional deep-learning models in the 2. Materials and methods
field of crop diseases. At the same time, fast training algorithms of deep
learning have become technical keys in related fields. At present, a lot of 2.1. Experimental materials
research results in crop disease diagnosis were achieved based on the
transfer learning model. A corn disease recognition method based on a The images of soybean leaf diseases were from 2006 to 2021 in the
deep learning network was proposed, and the classification accuracy science and technology park of the 850 Farm in Heilongjiang Province of
obtained by ResNet50 was 92.82% (Wang et al., 2022a, 2022b). Using China. The plant protection experts of Heilongjiang Bayi Agricultural
62 wild plant images from the Kaggle dataset, a method based on the University and the farm plant protection station jointly acquired four
ResNet101 network and transfer learning was proposed, with a test soybean leaf diseases. The number of disease samples was shown in
accuracy of 85.6% (Li et al., 2021). Using the cassava disease image Table 1. The test area belongs to the second accumulated temperature
dataset, transfer learning was applied to train Inception v3, and the best zone. The main soybean varieties in the test area are Kenfeng 17, Ken­
model achieved an accuracy of 93% (Amanda et al., 2017). Through nong 23, Heinong 44, Kendou 94, Xiannong 1, and Dongnongdou 252.
transfer learning combined with fine-tuning, three different deep- The images of soybean diseased leaves were acquired on the above
learning models were trained with a model accuracy of more than soybean plants. The image data included SGH, SBLS, SFLS, and SPLS in
80% (Wang et al., 2022a, 2022b). Using pre-trained transfer learning four growth stages, including the full flowering stage (R2), beginning
models including DenseNet, ResNet, VGG and InceptionNet, the non- pod stage (R3), full pod stage (R4), and beginning seed stage (R5).
invasive detection and classification of aflatoxy-contaminated dry figs
obtained the training accuracy of 98.57% and the validation accuracy of 2.2. Image acquisition method
97.50% (Kılıç and İnner, 2022). The model structures generated by
different deep learning algorithms were used to segment and automat­ In this study, the images of soybean leaf diseases were acquired by
ically identify bacterial spot diseases in peach crops, and the accuracy of using a digital camera (SONY's DSLR-A350, Japan). The exposure speed
corresponding peach leaves (bacteria and health) reached 98.75% of the camera was set at 1/1000 s, and the aperture value was F8.0. In
(Yadav et al., 2021). The acquired leaf image datasets of different types the preparation stage of the experiment, to ensure the true and reliable
of mung bean were segmented and enhanced. The test accuracy of color of the acquired diseased soybean leaf images, a standard template
VirLeafNet-1, VirLeafNet-2 and VirLeafNet-3 on different leaf images was made based on 18% gray cards. During the shooting process, the
were 91.234%, 96.429% and 97.403%, respectively (Joshi et al., 2020). infected soybean leaves on the plant were laid flat on the template, and
Combining multiple loss functions, an effective loss fusion convolutional the lens was kept perpendicular to the standard template. The shooting
neural network model was proposed to recognize the disease samples in distance was always kept at 40 cm. To show the growth state of infected
the public data set, and the discrimination accuracy reached 98.93% leaves of soybean plants under natural conditions, the weather condition
(Gokulnath and Usha Devi, 2021). An efficient tomato leaf disease of a windless and sunny day was selected to complete the data acqui­
segmentation model was developed using an enhanced radial basis sition. A total of 726 images of four types of soybean diseases were ac­
function neural network with a segmentation accuracy of 98.92% (Patil quired, including 203 images of SGH, 133 images of SBLS, 277 images of
Manoj and Manohar, 2022). The accuracy of disease diagnosis models in SFLS, and 113 images of SPLS. During the whole acquisition experiment,
the existing research needs to be further improved to provide more soybean leaves did not need to be picked to avoid adverse effects on
reliable basis for the precise prevention and control of crop diseases in soybean plant growth.
the field, so as to improve the yield and quality of crops. The recognition
accuracy of existing disease diagnosis models based on traditional deep
learning needs to be further improved to provide a more reliable basis 2.3. Overall process of establishing the recognition model
for the precise prevention and control of crop diseases in the field, to
improve the yield and quality of crops. In crop disease diagnosis tech­ The overall process of establishing the recognition model was shown
nology, the use of transfer learning to improve the rapid training algo­ in Fig. 1. First, the original data of the four leaf diseases were obtained
rithm of deep learning models is rarely reported in recognition of and preprocessed using a digital camera (Fig. 1 (a)) and a standard gray
soybean leaf diseases. card (Fig. 1 (b)). Second, the original image was registered after using
Therefore, to improve the recognition accuracy and efficiency, a image registration technology (Fig. 1 (c)). The leaf area was extracted
recognition model of the soybean leaf disease was proposed based on (Fig. 1 (d)) by using gray value analysis and (the OTSU) segmentation
transfer learning improved by residual network. First, four leaf disease method (Fig. 1 (e)). Third, the extracted leaf area was labeled based on
varieties of Septoria Glycines Hemmi (SGH), Soybean Brown Leaf Spot region calibration (Fig. 1 (f)), and finally enhanced (Fig. 1 (g)). In
(SBLS), Soybean Frogeye Leaf Spot (SFLS), and Soybean Phyllosticta addition, the network model was constructed and trained. Further, the
Leaf Spot (SPLS) were taken as the research objects. Then, the image
data was expanded using rotation, mirroring, noise adding, and filtering. Table 1
Third, the weight of the convolutional layer of the pre-trained model Principle table of image enhancement methods.
was transferred to the convolutional layer of the residual neural network Image enhancement Computational equation Parameter description
(ResNet18), the network layer of the pre-trained model was frozen, and method
the global average pooling layer and fully connected layer were created
[x' y' ] =
after the convolutional layer to establish a transfer learning residual Image rotation [
cosβ sinβ
] β: Rotation angle
[x y]
network TRNet18. In addition, to verify the accuracy and efficiency of − sinβ cosβ
the proposed model, the traditional convolutional neural network Horizontal mirror
⎡ '⎤ ⎡
x 1 0 0
⎤⎡ ⎤
x
W: Width of image
models including TRNet18, AlexNet, ResNet50 and ResNet50 were transformation ⎣ y' ⎦ = ⎣ 0 − 1 W ⎦⎣ y ⎦
1 0 0 1 1
compared. The results showed that the proposed transfer learning SNR∈[0,1]: Specifies the
network model based on ResNet18 achieved accurate and rapid recog­ Salt and pepper noise NP = SP × (1 − SNR) signal-to-noise ratio
nition of soybean leaf diseases, which can provide a theoretical basis and SP: Total number of pixels
X - mean: Mean
technical support for intelligent mining and analysis of the phenotypic valuesigma: Standard
big data of crop diseases. Pout = Pin + X - mean + deviation
Gaussian noise
sigma × G(d) Pin: Input pixel
G(d): Gaussian sampling
distribution formula

2
M. Yu et al. Ecological Informatics 76 (2023) 102096

y y

y y

Fig. 1. The Overall process of establishing the recognition model.

classical convolutional neural network AlexNet (Fig. 1 (h)), residual acquired images were shown in Fig. 2 (a). Since the traditional module
series ResNet18 (Fig. 1 (I)) and ResNet50 were selected in the study, and and diseased leaves were under the same light source, the registration
transfer learning was combined based on ResNet18 and ResNet50. A law of a standard gray card was consistent with the target. The marking
total of five network models were constructed in the investigation. module adopted a standard gray card with a relatively regular shape,
Finally, the network model was evaluated including the accuracy and which was convenient for automatic acquisition and analysis by com­
loss value of the training process (K), the output value (L), running time, puter. The RGB mean values of the marker module and the background
macro-F1, and accuracy (M) of the test process. were counted, and the color registration of the non-standard image was
performed in turn according to the supervised color registration method
3. Image preprocessing of the standard gray card.
Set the color values of the marker block of the standard gray card in
3.1. Image registration the common space asR0、G0、B0and the color values of the acquired
image as R, G, and B, respectively.
Image registration refers to the discipline of distorted images. To ⎧
acquire color images of leaves, the digital camera was perpendicular to ⎨ R0 = a1 v1 + a1 v2 + … + a1 vj
G0 = a2 v1 + a2 v2 + … + a2 vj (1)
the leaf surface, and the acquisition of the diseased leaf image will ⎩
B0 = a3 v1 + a3 v2 + … + a3 vj
produce a slight geometric distortion to meet the basic needs of model
training and disease recognition. During the shooting process, many The matrix form of Eq. (1) was as follows:
factors such as image sensor and illumination will lead to the distortion
X = AT × V (2)
of the color value for the acquired images. Thus, there was a specific
deviation between the actual value of the diseased leaves in the natural The matrix Acan be optimized by the least square method, which was
environment. When soybean disease occurs, the color of the leaves is as follows.
reflected (Ma et al., 2019), and the color characteristics of the leaves will ( )− 1 ( )
change accordingly. Therefore, the obtained image was subjected to A = V × VT V × XT (3)
color registration for subsequent analysis to ensure the authenticity and
reliability of the analysis results. XOUT = AT × VT (4)
Four mark-blocks of the standard gray card were placed in the Therefore, A was input into Eq. (4), the R, G, and B values of each
shooting environment and photographed with the leaf image. The pixel of the registered image can be calculated to achieve color

3
M. Yu et al. Ecological Informatics 76 (2023) 102096

Fig. 2. Color correction effect.

registration. Where XOUT was the R, G, and B three-valued matrix of the
registered image, and VT was the matrix composed of the items of the
empirical equation corresponding to the three-valued color of the
captured image.
Image color correction steps were as follows.
(1) First, the acquired image was read and clipped to avoid the in­
fluence of the background outside the whiteboard on the color, and
three channel values of the clipped image were obtained.
(2) Read the standard template scanned using the scanner to obtain
the standard gray card marker block.
(3) Used Eq. (1) Eq. (4) to complete color registration.
Diseased images of soybean plants before and after color registration
were shown in Fig. 2.
In Fig. 2 (a), the R, G, and B mean values of the original image were
101,107,103, respectively. The R, G, and B mean values of the scanned
standard gray card marker block were 110,137,135, respectively. The R,
G, and B mean values of the registered image in Fig. 2 (b) were 199, 208
and 197, respectively. The determination coefficient R2 = 0.86 in the
experiment process met the basic requirements of image registration.
3.2. Image segmentation
OTSU is a global-based binarization algorithm, which divides the
images into two parts, foreground, and background according to gray
characteristics. When the best threshold is obtained, the difference is the
largest between the two parts, which indicates that the difference be­
tween the two parts of the image is more remarkable. When some targets
and backgrounds are misclassified, the between-class variance of the
two parts becomes smaller. In short, when the threshold segmentation
makes the maximum variance between classes, the probability of
misclassification is the smallest.
T was the image segmentation threshold. The proportion of fore­
ground pixels in the image was w0, and the average gray level wasu0. The
ratio of background pixels to the image was w1, the average gray level
was u1, and the total intermediate gray level of the image was u. The
calculation Equation was as follows.
u = w0 × u0 + w1 × u1 (5)
The variance of foreground and background was g, and the calcula­
tion Equation was as follows.
The gray histogram of the soybean diseased leaf image acquired in
this experiment was shown in Fig. 3.
The gray histogram in Fig. 3 had prominent double peaks. The seg­
mentation threshold T = 0.6235 obtained by the OTSU was converted
between [0, 255]. The converted value t = 158.9925 was the position of
the trough between the two peaks in the gray histogram, indicating that
the sample data in this experiment conformed to the best global
threshold processing characteristics.
The effect of using the OTSU to segment the soybean diseased leaf
image was shown in Fig. 4.
In the evaluation index of the segmentation effect of soybean disease
image, TP in soybean disease image referred to the number of pixels of
actual diseased soybean leaves that were correctly predicted to be
diseased soybean leaves in the segmented image. FP referred to the
number of pixels of the actual image background that were incorrectly
predicted as soybean diseased leaf pixels. TN referred to the number of
pixels of the actual image background that were correctly predicted as
image background pixels. FN referred to the number of pixels of actual
diseased soybean leaves that were incorrectly predicted as image
background pixels.
Precision represented the proportion of TP to TP + FP. The perfect
segmentation value of Precision was 1, and the score of maximum inter-
class variance method was 0.9979.
TPR represented the proportion of TP to TP + FN. The perfect seg­
mentation value of TPR was 1, and the maximum inter-class variance
method score was 1.0000.
FPR represented the proportion of FP in FP + TN. The perfect

g = w0 × (u0 − u)2 + w1 × (u1 − u)2 (6)

Eq. (7) can be obtained through the combination of Eq. (5) and Eq.
(6).

g = w0 × w1 × (u0 − u1 )2 (7)
When the variance g was the largest, the difference between fore­
ground and background was the largest. At the same time, the gray T
was the optimal threshold. Fig. 3. Gray histogram.

4
M. Yu et al.
Fig. 4. Segmented image.
segmentation value of FPR was 0, and the maximum inter-class variance
method score was 0.0006.
The above three segmentation evaluation indexes showed that the
OTSU performed better in segmentation of the data set samples.
3.3. Region labeling
Each original acquired image contained 1–6 diseased single leaves.
The segmented diseased leaf images were processed by the region cali­
bration method, and the processing steps were as follows.
(1) Read the diseased image with the segmented background;
(2) Set the number of diseased leaves to be calibrated on the image as
m (m ∈ [1, 6]);
(3) In the regional calibration coordinate system of the image, the X-
axis direction was from left to right, and the Y-axis was from top to
bottom. As shown in Fig. 5, select the starting point (x1, y1)and ending
point (x2, y2) of the rectangular area were selected, where a diseased leaf
was located in the image, and a diseased leaf was cut out for each set of
values selected. Repeated m times until all diseased leaves on the image
were taken.
Fig. 5. Legend of area calibration.
5
Ecological Informatics 76 (2023) 102096
The single leaf image obtained by the segmented leaf image using the
region calibration method was shown in Fig. 6.
3.4. Sample expansion and partitioning
The single diseased leaf image was set as the object, the size of the
image was scaled to 224 × 224, and then data enhancement methods
were adopted, such as Gaussian filtering, image brightness trans­
formation, image flip (90∘and180∘), noise addition, image rotation, and
rotation (Yu et al., 2022) to expand the images of the data set and
provide data basis for model training. All kinds of sample sizes are
balanced to improve the generalization ability of the model. The
calculation Equation and parameter description of the four image
enhancement methods were shown in Table 1.
The single-leaf images after data enhancement were shown in Fig. 7.
726 soybean-diseased images after image registration were set as the
object, OTSU was first used to remove the sample background. Then the
single diseased leaf image was obtained using region calibration.
Further, a total of 53, 250 expanded images of four types of soybean
diseases were obtained by data enhancement method, and each kind of
image was randomly divided into the training set, validation set, and test
set according to 7:2:1, which was used for the training, validation, and
performance testing of the model in this experiment. The coding labels
and division of data sample sizes of different disease images were shown
in Table 2.
4. Network models
In recent years, the convolutional neural network (CNN) has ach­
ieved good results in image classification and recognition. It is more
excellent and the recognition rate is higher than the traditional classi­
fication method.
4.1. Classic convolutional neural network AlexNet
The AlexNet (Alex et al., 2017) consists of five convolutional layers,
three max-pooling layers, and three fully connected layers. The network
structure was shown in Fig. 8:
The network structure parameters of each layer of the AlexNet were
detailed in Table 3.
4.2. Residual neural network
To improve the accuracy of the model, deep neural networks are
constantly developing. With the deepening and widening of the neural
network, the following problems have arisen: (1) Because of the large
amount of data required for model training, it isn't easy to apply in
practice; (2) With more and more network parameters, model training
time is too long; (3) Gradient disappearance; (4) The model is difficult to
optimize.
However, this is not the case when the residual neural network is
trained. The core residual module can transmit information backward
through identity mapping when the network is optimal. In theory, the
network will always be optimal.
4.2.1. ResNet18
When scholars tried to develop a deeper network layer, the problems
of gradient disappearance and gradient explosion appeared. To solve
this problem, identity mapping was introduced into the structure of the
residual network. In a series of residual networks, ResNet18 is charac­
terized by low computational burden, and its unit structure BasicBlock
was shown in Fig. 9.
In Fig. 9, BasicBlock mainly included three parts. The input x of
BasicBlock first passed through the first part composed of the convolu­
tion layer with a convolution kernel size of 3 × 3, Batch Normalization
(BN)layer, and activation function Relu. Then it passed through the
M. Yu et al. Ecological Informatics 76 (2023) 102096

Fig. 6. Region labeling results.

Fig. 7. Data enhancement results.

Table 2
Disease sample pretreatment, coding, and quantity division.
Soybean disease species Region labeling Data augmentation Training sets Validation set Testing set Coding label

SGH 203 13,804 9664 2760 1380 [1,0,0,0]

SBLS 360 12,960 9072 2592 1296 [0,1,0,0]
SFLS 1491 11,928 8351 2385 1192 [0,0,1,0]
SPLS 429 14,558 10,912 2911 1455 [0,0,0,1]
Total 2483 53, 250 37, 279 10,648 5323 –

Fig. 8. Network structure of AlexNet.

second part composed of the convolution layer with a convolution
kernel size of 3 × 3 and BN layer. F(x) represented the residual learning
of x, ⊗ represented the superposition of residual learning and identity
mapping x, that was, the third part was F(x) + x and activation function
Relu, so the actual output of BasicBlock was H(x) = F(x) + x through
Relu. The goal of BasicBlock optimization was F(x) = H(x) − x, and the
ideal optimization was to approximate it to 0. BasicBlock offset the
deficiency of information distortion and loss of image information pro­
cessing, and greatly enhanced the learning capacity of the network.
Although the residual network can effectively solve the gradient in­
crease brought about by a series of problems, such as gradient
disappearance and explosion, etc. Still, if we want to enhance the ac­
curacy of the training model further, and increase the depth of the
network at the same time, it will increase the width of the network and
complexity, improve the memory model of training need, and increase
the difficulty of the training model.
The overall network structure of the ResNet18 included an input
layer, convolution layer (Conv), BasicBlock, pooling layer (Maxpool and
Avgpool), and fully connected layer (Fc). In addition to the output
discrimination layer, ReLU was used as the activation function in other
layers of the network. Softmax was used to classify soybean disease in
network model. The network structure was shown in Fig. 10.

6
M. Yu et al. Ecological Informatics 76 (2023) 102096

Table 3 the Bottleneck passed through the first part, which composed of a
Specific structural parameters of the AlexNet model. convolution layer with a convolution kernel size of 1 × 1, a BN layer,
Type Input size Stride Output size and an activation function Relu. The 1 × 1 convolution layer reduced the
number of convolution channels to 1/4 and passed through the second
Conv 224 × 224 × 3 4 55× 55× 48
Max Pool 55× 55× 48 2 27 × 27 × 48 part, which composed of the convolution layer with a convolution
Conv 27 × 27 × 48 1 27 × 27 × 128 kernel size of 3 × 3, BN layer, and activation function Relu. Finally, the
Max Pool 27 × 27 × 128 2 13 × 13 × 128 third part comprised a convolution layer with a convolution kernel size
Conv 13 × 13 × 128 1 13 × 13 × 192 of 1 × 1 and a BN layer. At this time, the 1 × 1 convolution layer was
Conv 13 × 13 × 192 1 13 × 13 × 192
Conv 13 × 13 × 192 1 13 × 13 × 128
used to recover the number of channels, so that the number of output
Max Pool 13 × 13 × 128 2 6 × 6 × 128 channels of the Bottleneck was equal to the number of input channels of
FC 6 × 6 × 128 – 1 × 1 × 2048 the Bottleneck. F(x) denoted the residual learning of x, and ⊗ denoted
FC 1 × 1 × 2048 – 1 × 1 × 2048 the superposition of residual learning and identity mapping x. The
FC 1 × 1 × 2048 1×1×4
–
fourth part was F(x) + x and activation function Relu, so the actual
output of Bottleneck was H(x) = F(x) + x through Relu. Similar to
BasicBlock, the goal of Bottleneck optimization was F(x) = H(x) − x, and
the ideal optimization was to approximate it to 0. Compared to the
BasicBlock, the two 1 × 1 convolutions in the Bottleneck effectively
reduced the number of parameters and computational complexity of the
convolution, simplifying the training network process.
The overall network structure of the ResNet50 was composed of the
input layer, Bottleneck, pooling layer, and fully connected layer. In
addition to the output discrimination layer, the other layers used ReLU
as the activation function. The network model used Softmax to classify
soybean leaf diseases. The ResNet50 network structure was shown in
Fig. 12.
The network structure parameters of each layer of the ResNet50 were
detailed in Table 5.

4.3. Residual network based on transfer learning

Transfer learning is a kind of machine learning method that means

the application of a pre-trained model in a new task. The domain mainly
consists of two parts: the feature space and the probability distribution
of generating these data. The domain of the pre-training model is called
the source domain, and the domain of the new task is called the target

Table 4
Specific structural parameters of ResNet18 model.
Fig. 9. BasicBlock.
Type Input size Stride Output size

The network structure parameters of each layer of the ResNet18 were Conv 224 × 224 × 3 2 112 × 112 × 64
Max Pool 112 × 112 × 64 2 56 × 56 × 64
detailed in Table 4.
Layer BasicBlock1/2 56 × 56 × 64 1 56 × 56 × 64
BasicBlock3 56 × 56 × 64 2 28 × 28 × 128
Layer
4.2.2. ResNet50 BasicBlock4 28 × 28 × 128 1 28 × 28 × 128
In a series of residual neural networks, although the residual struc­ Layer
BasicBlock5 28 × 28 × 128 2 14 × 14 × 256
ture is composed of convolution layers, ResNet50 ‘s Bottleneck and BasicBlock6 14 × 14 × 256 1 14 × 14 × 256
BasicBlock7 14 × 14 × 256 2 7 × 7 × 512
ResNet18 ‘s BasicBlock differ in convolution layers and convolution Layer
BasicBlock8 7 × 7 × 512 1 7 × 7 × 512
kernel size. The structure of the Bottleneck was shown in Fig. 11. Avg Pool 7 × 7 × 512 – 1 × 1 × 512
In Fig. 11, Bottleneck was summarized in four parts. The input x of FC 1 × 1 × 512 – 1×1×4

Fig. 10. Network structure of Resnet18.

7
M. Yu et al. Ecological Informatics 76 (2023) 102096

training network were directly used by other layers. Then the data set
composed of the soybean disease image of this experiment was used to
retrain the whole model. The residual neural network based on transfer
learning was constructed to realize the disease recognition of self-
collected soybean images.

4.3.1. TRNet18
In this paper, the method of combining transfer learning and residual
neural network was used to select the pre-trained weight file of
ResNet18 on the training model dataset ImageNet, which had a specific
correlation and similarity with the soybean disease dataset of this
experiment. The TRNet18 was constructed to realize the disease
recognition of self-collected soybean images. The network structure of
the TRNet18 was shown in Fig. 13.
In model construction, a Fine-tuning strategy was applied in the new
target task by adjusting one or more layers of the pre-trained model in
the dataset ImageNet. In this experiment, the generic features extracted
by the convolutional layer of ResNet18's pre-training model were
retained. The specific steps of constructing the TRNet18 model were as
follows.
(1) The mean value after the input image misused the RGB channel
was 123.68,116.78 and 103.94, respectively. The same preprocessing as
the pre-training model data set was performed to ensure the correct call
of the pre-training model.

Table 5
Specific structural parameters of the ResNet50 model.
Type Input size Stride Output size
Fig. 11. Bottleneck.
Conv 224 × 224 × 3 2 112 × 112 × 64
Max Pool 112 × 112 × 64 2 56 × 56 × 64
domain. There is a certain similarity or correlation between the pre- BottleBlock1 56 × 56 × 64 1 56 × 56 × 256
trained task and the new task, so similar tasks can better perform Layer BottleBlock2 56 × 56 × 256 1 56 × 56 × 256
BottleBlock3 56 × 56 × 256 1 56 × 56 × 256
parameter migration. This process reduces the parameters of the
BottleBlock4 56 × 56 × 256 2 28 × 28 × 512
training model, shortens the training time, accelerates the convergence BottleBlock5 28 × 28 × 512 1 28 × 28 × 512
speed and reflects the timeliness, and can effectively solve the over- Layer
BottleBlock6 28 × 28 × 512 1 28 × 28 × 512
fitting problem. BottleBlock7 28 × 28 × 512 1 28 × 28 × 512
Feature transfer and model transfer are two standard methods of BottleBlock8 28 × 28 × 512 2 14 × 14 × 1024
BottleBlock9 14 × 14 × 1024 1 14 × 14 × 1024
transfer learning. The experiment in this study adopted the method of
BottleBlock10 14 × 14 × 1024 1 14 × 14 × 1024
model migration. To accelerate the convergence and enhance the Layer
BottleBlock11 14 × 14 × 1024 1 14 × 14 × 1024
generalization ability of the model, this paper combined the idea of BottleBlock12 14 × 14 × 1024 1 14 × 14 × 1024
transfer learning and residual neural network. They selected the pre- BottleBlock13 14 × 14 × 1024 1 14 × 14 × 1024
BottleBlock14 14 × 14 × 1024 2 7 × 7 × 2048
trained weight file on the training model data set ImageNet that had
Layer BottleBlock15 7 × 7 × 2048 1 7 × 7 × 2048
specific correlation and similarity with the soybean disease data set of BottleBlock16 7 × 7 × 2048 1 7 × 7 × 2048
this experiment. First, the pre-training model was reinitialized, and the Avg Pool 7 × 7 × 2048 – 1 × 1 × 2048
parameter layer was connected, the weight parameters of the pre- FC 1 × 1 × 2048 – 1×1×4

Fig. 12. Network structure of Resnet50.

8
M. Yu et al. Ecological Informatics 76 (2023) 102096

Fig. 13. Network structure of the TRNet18.

(2) The convolutional layer weights of the pre-trained model were
transferred to the convolutional layer of the TRNet18 model so that the
weights were updated synchronously with training.
(3) A global average pooling layer was added after the convolution
layer to extract the channel content, and the dimension of the feature
map was changed from 4 dimensions to 2 dimensions. The 2-dimen­
sional matrix was then flattened into a 1-dimensional matrix as input
to the subsequent fully connected layer to accelerate the calculation.
(4) The global average pooling layer was followed by a fully con­
nected layer. The research object of the experiment was four kinds of
soybean diseases, so the output dimension of the fully connected layer
was set to 4, and Softmax was used as the activation function.
4.3.2. TRNet50
The pre-trained weight file of ResNet50 was used on the data set
ImageNet, and TRNet50 was constructed to realize disease recognition
of self-collected soybean images. The network structure of the TRNet50
diagram was shown in Fig. 14.
The steps to construct the TRNet50 model were different from those
of TRNet18. The global average pooling layer was followed by a com­
bination of two Dropout layers and fully connected layers. Two Dropout
layers were temporarily discarded from the network at a probability of
0.5 to prevent network overfitting. The output dimension of the first
fully connected layer was 1024, and the activation function was ReLU.
To meet the experimental requirements, the output dimension of the
second fully connected layer was 4, using Softmax as the activation

Fig. 14. Network Framework of TRNet50.

9
M. Yu et al. Ecological Informatics 76 (2023) 102096

function. the difference between the actual value and the predicted value of the
disease type. The smaller the difference, the better the model training
effect. The cross-entropy loss equation was as follows:
4.4. 4.4 Learning algorithm
1∑
C= − [ylna + (1 − y)ln(1 − a) ] (10)
4.4.1. Batch normalization algorithm n x
The nature of the neural network learning process is to learn the data Where, C was on behalf of the cross-entropy loss, y was the actual
distribution. In the process of network training, the parameters are value, α was the predicted value, and when z =
∑
ωj × xj + b, α = σ(z).
constantly updated. In addition to the data of the input layer, the input The derivation equations of cross-entropy loss for weight and bias
data distribution of each subsequent network layer is constantly terms were as follows:
changing. The purpose of the batch normalization (BN) algorithm is to
solve the problem that the data distribution of intermediate layers ∂C 1∑
= xj (σ (z) − y ) (11)
changes during the training process. Therefore, in the input of each ∂ωj n x
layer, the BN layer can be added, and then entered into the next layer of
the network. This BN layer is a learnable and parametric network layer, ∂C 1∑
= (σ (z) − y ) (12)
also known as transformation reconstruction, which introduces learn­ ∂b n x
able parameters γ and β.
In the derivative of cross-entropy loss to weight ωj and bias term b,
The normalization formula of γ and β introduced in transformation
the weight update was affected by the error σ (z) − y. When the error was
reconstruction was as follows.
large, the weight updated quickly.
∧(k)
y(k) = γ (k) x + β(k) (8)
5. Result analysis
(k)
E[x ] referred to the average of the neurons in each batch of
√̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
training data. Var[x(k) ]was one standard deviation of the activation of 5.1. Process of simulation experiment
√̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
neurons x(k) for each batch of data. When γ (k) = Var[x(k) ] andβ(k) = E
(k)
[x ], the distribution of features learned by the original network could In this experiment, a deep learning environment based on Python
be recovered. and Tensorflow - GPU as well as CUDA was built under the Windows
Finally, the forward conduction process of the BN network layer was operating system. The specific versions of the tools used were shown in
as follows: Table 7.
( ) ( ) ( ) The Original model included Alexnet, ResNet18, ResNet50, and the
1 ∑ 1 ∑ transferred model included TRNet18 and TRNet50. The specific steps
m m
∧ x i − μВ (
μВ = xi → σ2В = (xi − μВ )2 → xi = √̅̅̅̅̅̅̅̅̅̅̅̅̅ → yi
m i=1 m i=1 σ2В + ε were as follows:
∧ ) (1) The self-collected soybean diseased leaf images were set as the
= γxi + β ≡ BN γ,β (xi ) object, and image processing techniques including color correction,
(9) threshold segmentation, region calibration, data enhancement, and size
normalization were used to preprocess the disease image. The pre­
4.4.2. Adam algorithm processed image was randomly divided into the training set, verification
In this study, the adaptive motion estimation (Adam) optimizer was set, and test set, according to the ratio of 7:2:1.
used to train and optimize the network parameters. The optimization (2) Pre-trained ResNet18 and ResNet50 on the public dataset
steps were as follows: ImageNet, and saved the weights generated during the pre-training
(1) To search and optimize for each parameter, we must maintain a process as characteristics for the classification of soybean leaf dis­
moment vectormt and an exponentially weighted infinity normvt, which eases. Then, the ResNet18 and ResNet50 were fine-tuned by using the
initialized m0 = 0，v0 = 0 at the beginning of the search. fine-tuning strategy, according to the number of nodes to construct a
(2) The algorithm was executed iteratively in time t starting from t = transfer learning network model for the recognition of soybean leaf
1, and a new set of parameter values x was calculated in each iteration, diseases.
which was from xt− 1toxt. (3) The divided training set and validation set were used as the input
(3) The deviation of the gradient average mt/vt was corrected to of the five networks to train and verify the model, and the models
reduce the influence of the deviation on the initial stage of training. generated by the five networks during the training process were called
The specific process of updating parameters was shown in Table 6. and tested on the divided test set.
The simulation process of 5 kinds of networks in the soybean disease
4.4.3. Cross-entropy loss function experiment was shown in Fig. 15.
In this paper, the cross-entropy loss function was used to represent

5.2. Analysis of recognition results

Table 6
Algorithm for updating parameters.
5.2.1. Training set and validation set
Algorithm1: Parameter update algorithm The training set and validation set were divided to complete the
Input: Hyper-parameter β1,β2, x,m0andv0
▹β1、β2 were used to control the first-order and second-order momenta.
if 0 < time < t + 1 then Table 7
xt− 1 → xt Experimental environment.
gt = f'(xt− 1) ▹gt was the gradient of the current time step
Operating system Windows 10 64 bit
mt = β1mt− 1 + (1 − β1)gt…vt = β2vt− 1 + (1 − β2)g2t
mt vt CPU lntel(R) Core(TM) 3.20GHz
m't = …v't =
1 − βt1 1 − βt2 GPU NVIDIA GeForce GTX 1050 Ti (4096 MB)
m' Deep learning framework Tensorflow-GPU 2.0
xt = xt− 1 − α √̅̅̅̅ t ▹α was the learning rate andε was a smoothing term
v't + ε Size of running memory 16GB
end if CUDA's version 10.0
output:data Python's version 3.7.3

10
M. Yu et al. Ecological Informatics 76 (2023) 102096

Fig. 15. The process of the simulation experiment.

Fig. 16. Curves of the model during training.

11
M. Yu et al.
training process together according to Table 2. The training set was used
for updating model parameters. The validation set was used to assess the
effect of the model obtained by training. Then the model was adjusted
according to the impact of the model, so that the model effect was
optimal by the validation set.
5.2.2. Training process and result analysis
In the training process, the input size of the sample was 224 × 224
pixels, Batch_size = 64, and epoch =300. The activation function was
ReLu, BN was added for data normalization, and the optimizer was
Adam. The accuracies of AlexNet, ResNet18, TRNet18, ResNet50, and
TRNet50 on the training set were 100%, 100%, 98.95%, 95.10%, and
99.83%, respectively. The accuracies on the validation set were 92.68%,
98.62%, 99.60%, 96.30, and 99.16%, respectively. In the training pro­
cess, the change of accuracy and loss values of the five network models
in the training set and the validation set were shown in Fig. 16.
In Fig. 16 (a), the training accuracy of AlexNet went from 0.430 to 1,
with a range of 0.570, which approached 0.99 and converged at Epoch
= 46. The training accuracy of ResNet18 went from 0.785 to 1 with a
range of 0.215, which approached 0.99, and converged at Epoch = 13.
The training accuracy of TRNet18 varied from 0.790 to 0.990 with a
range of 0.2, closed to 0.98, and converged when Epoch = 108. The
training accuracy of ResNet50 ranged from 0.757 to 0.951, with a
variation of 0.194, which closed to 0.95 and converged at Epoch = 289.
The training accuracy of TRNet50 ranged from 0.680 to 0.998, with a
deviation of 0.318, which closed to 0.99 and converged at Epoch = 19.
The smaller the change range of the training accuracy on the training
set, the better the strength of the network. The stability of the network
model can be weakened in the order of ResNet50, TRNet18, ResNet18,
TRNet50, and AlexNet. The closer the value of the training accuracy
curve was to 1 and the smaller the Epoch was, the faster the convergence
speed was. Then the convergence speed of the network model can be
slowed down in the order of ResNet18, TRNet50, AlexNet, TRNet18, and
ResNet50.
In Fig. 16 (b), the training loss value of AlexNet decreased from 1.023
to 8.45 × 10− 7, with a change of 1.023, which approached 0.001 and
converged at Epoch = 47. The training loss value of ResNet18 decreased
from 0.495 to 3.010 × 10− 7, with a change of 0.495, which approached
0.001 and converged at Epoch = 35. The training loss value of TRNet18
decreased from 0.493 to 0.028, with a change of 0.465, which
approached 0.03 and converged at Epoch = 179. The training loss value
of ResNet50 decreased from 0.587 to 0.131, with a change of 0.457,
which approached 0.13 and converged at Epoch = 225. The training loss
of TRNet50 fell from 0.706 to 0.007, with a variation of 0.698, which
approached 0.01 and converged at Epoch = 87. The smaller the change
range of the training loss curve on the training set, the better the stability
of the network. The strength of the network model can be weakened in
the order of ResNet50, TRNet18, ResNet18, TRNet50, and AlexNet. The
closer the value of the training loss curve was to 0 and the smaller the
Epoch was, the faster the convergence speed was. Then the convergence
speed of the network model can be slowed down in the order of
ResNet18, AlexNet, TRNet50, TRNet18, and ResNet50.
In Fig. 16 (c) the training accuracy of AlexNet went from 0.487 to
0.927, with a range of 0.440, which closed to 0.92 and converged at
Epoch = 258 The training accuracy of ResNet18 went from 0.783 to
0.986, with a range of 0.203 which closed to 0.98 and converged when
Epoch = 26. The training accuracy of TRNet18 went from 0.881 to 0.996
with a range of 0.115, which closed to 0.99 and converged when Epoch
= 49. The training accuracy of ResNet50 went from 0.838 to 0.963 with
a range of 0.125, closed to 0.96, which converged when Epoch = 179.
The training accuracy of TRNet50 went from 0.732 to 0.992, with a
variation of 0.260, which approached 0.99 and converged at Epoch =
82. The smaller the interpretation of the verification accuracy curve on
the verification set, the better the stability of the network, and the sta­
bility of the network model can be weakened in the order of TRNet18,
ResNet50, ResNet18, TRNet50, and AlexNet. The closer the value of the
12
Ecological Informatics 76 (2023) 102096
verification accuracy curve was to 1 and the smaller the Epoch was, the
faster the convergence speed was. Then the convergence speed of the
network model can be slowed down in the order of TRNet18, TRNet50,
ResNet18, ResNet50, and AlexNet.
In Fig. 16 (d), the training loss value of AlexNet decreased from
1.082 to 0.348, with a change of 0.734, which approached 0.4 and
converged at Epoch = 111. The training loss value of ResNet18
decreased from 0.466 to 0.075, with a change of 0.391, which
approached 0.07 and converged at Epoch = 59. The training loss value
of TRNet18 decreased from 0.282 to 0.011, with a change of 0.271,
which approached 0.01 and converged at Epoch = 82. The training loss
value of ResNet50 decreased from 0.391 to 0.099, with a change range
of 0.292, which approach 0.1 and converged at Epoch = 142. TRNet50's
training loss decreased from 0.599 to 0.045, with a variation of 0.555,
which approached 0.04 and converged at Epoch =48. On the verifica­
tion set, the smaller the change range of the loss curve was, the better the
stability of the network was. The strength of the network model can be
weakened in the order of TRNet18, ResNet50, ResNet18, TRNet50, and
AlexNet. The value of the loss curve at the time of convergence is closer
to 0 and the smaller the Epoch, the faster the convergence speed was, the
convergence speed of the network model can be slowed down in the
order of TRNet18, TRNet50, ResNet18, AlexNet, and ResNet50.
From the above analysis of the curves in Fig. 16, the optimal network
model TRNet18 in training can be comprehensively evaluated, which
showed that based on ResNet18, the transfer learning fine-tuning
method could learn the characteristics of soybean diseased areas more
efficiently than other models. The TRNet18 model improved the
recognition accuracy of the algorithm faster, accelerated the training
and verification process, and played a positive role in reducing the loss
value.
5.2.3. Testing set
The test set divided in Table 2 was used to complete the test process.
In this study, the test set was the unknown data set except the training
set and the validation set to evaluate the generalization ability of the
final model, and test whether the model could give the correct output.
Unbiased estimation meant that the expected value of the estimator was
equal to the actual value of the estimator. The expectation of the sample
mean was a fair estimate of the population mean.
Supposed that the mean of a random variable X was μ, there were n
∑
samples X1, ⋯⋯, Xn and the sample mean X = 1n ni=1 Xi , then: the un­
biased estimate of the population mean X was expressed as follows:
1∑n
E(X) = E(Xi ) = μ (13)
n i=1
In deep learning, the performance evaluation index of the model was
set to X. On the test set with n data, the n index values X1, ⋯⋯, Xn
obtained by n data were calculated, and then divided by n to get the
expectation of the mean value of the sample (marked value). The
expectation of the sample standard was the unbiased estimation of the
overall mean, so calculating the average value of the performance
evaluation index on the test set was a fair estimation of the overall
performance of the model. This process was also to evaluate the
generalization performance of the model.
5.2.4. Simulation test and result analysis
The four types of leaf disease samples in the test set were treated the
same in the training process. The test accuracy of the five network
models of AlexNet, ResNet18, TRNet18, ResNet50, and TRNet50 was
76.54%, 98.42%, 99.53%, 95.89%, and 98.42%, respectively. The test
on the test set was shown in Table 8.
According to Table 8, from the analysis of the classification accuracy
of the model on the test set, the classification accuracy of AlexNet was
the lowest, and the classification accuracy of ResNet18 and ResNet50
was 21.88% and 19.35%, higher than that of AlexNet, respectively. The
M. Yu et al. Ecological Informatics 76 (2023) 102096

Table 8
Disease test results.
Disease category Number of tests Accuracy/%

AlexNet ResNet18 TRNet18 ResNet50 TRNet50

SGH 1380 100 100 100 99.78 100

SBLS 1296 59.57 97.92 99.23 93.67 98.77
SFLS 1192 77.01 98.99 99.58 97.57 97.40
SPLS 1455 69.00 96.91 99.31 92.78 97.46
Total 5323 76.54 98.42 99.53 95.89 98.42

reason was that the structure in the residual neural network model could
extract disease features in more detail and improve the recognition ef­
fect. The classification accuracy of ResNet18 was 2.53% higher than that
of ResNet50. In the migration model based on the residual neural
network, the classification accuracy of TRNet18 was 1.11% higher than
that of TRNet50. Compared with the original model, the classification
accuracy of TRNet18 was 1.11%, higher than that of ResNet18, and the
classification accuracy of TRNet50 was 2.53%, higher than that of
ResNet50. This was because the TRNet18 and TRNet50 models used
transfer learning to fine-tune the pre-training model, retaining a large
amount of low-level general feature information of the pre-training
model. In contrast, the high-level learned the features of specific dis­
ease classification tasks, and the high-level and low-level collaboration
significantly improved the recognition accuracy.
In artificial intelligence, the confusion matrix was a visualization
tool, especially for supervised learning. Each column of the confusion
matrix represented the actual category, each row represented the
probability that a column category would be predicted as a row cate­
gory, and the diagonal value was the probability of the correct
prediction. The performance of the five network models on each type of
soybean disease in the test set and the visualization results was shown in
Fig. 17.
Figs. 17 (a), (b), (c), (d), and (e) showed that there were 6, 4, 6, 10,
and 9 misidentifications in the recognition process of four diseases in the
test set. The recognition error rates were 6.18%, 1.81%, 16.2%, and
6.37%, respectively. According to the sum value of the recognition error
rate, the model was sorted from large to small: AlexNet, ResNet50,
TRNet50, ResNet18, and TRNet18. Compared with the AlexNet model,
the residual neural network model and the residual neural network
model with transfer learning significantly reduced the error rate.
Compared with the original residual neural network, the recognition
error rate of the model was further reduced after introducing the transfer
learning method. ResNet18 performed best on the test set.
In the process of the simulation experiment, the decoding mapping
rule corresponding to the actual output value of the network model and
the coding of soybean disease type was as follows:

Fig. 17. The performance of the models.

13
M. Yu et al.
⎧ y1 , [1, 0, 0, 0], SGH
⎪
⎪
⎨ y2 , [0, 1, 0, 0], SBLS
y = MAX(y1 , y2 , y3 , y4 ) (14)
⎪
⎩ y3 , [0, 0, 1, 0], SFLS
⎪ in this area was 1192, decoded as [0,0,1,0], which should be SFLS.
y4 , [0, 0, 0, 1], SPLS
By comparing the maximum value of the network output as a
favorable segment, the output value was established for the coding
vector, and then analyzed for the soybean disease category. A model
method was established to recognize the soybean disease category
quickly. 5323 image samples of 4 soybean disease types were used as the
prediction set, the network calculation results of the optimal soybean
disease recognition model TRNet18 selected here were shown in Fig. 18.
In Fig. 18, the four colors represented the four-column node values of
the actual output of the TRNet18 network, and the network with only
one soybean disease type in each column had the largest real output
value. When the x ∈ [1, 1380] (Fig. 18(a)), the value of the corre­
sponding red area was the largest, and the test sample size of the disease
image in this area was 1380, decoded as [1,0,0,0], which should be SGH.
When the x ∈ [1381, 2676](Fig. 18(b)), the corresponding blue area had
the largest value, and the test sample size of the disease image in this
area was 1296, decoded as [0,1,0,0], which should be SBLS. However,
ten abnormal points were generated in this interval to judge the SBLS, so
there was a misdiagnosis in the SBLS. After the analysis of the network
output value, it was found that the misdiagnosis between SBLS and SPLS
Fig. 18. Calculation results of the TRNet18 model.
14
Ecological Informatics 76 (2023) 102096
occurred due to the similarity in the symptoms of the disease leaves.
When the x ∈ [2677, 3868](Fig. 18(c)), the value of the corresponding
green area was the largest, and the test sample size of the disease image
However, five abnormal points were generated in this interval, which
was judged as SFLS. When the abscissa x ∈ [3869, 5323](Fig. 18(d)), the
value of the corresponding magenta region was the largest. The test
sample size of the disease image in this region was 1455, decoded as
[0,0,0,1], which should be SPLS. In the test process of this experiment,
SFLS was misdiagnosed as SPLS because some images were seriously
distorted in the data enhancement process, resulting in the error of
disease diagnosis. In the recognition process, the four output values of
the network had errors, which was the reason why the recognition ac­
curacy of the model did not reach 100%.
In Table 9, the training models of ResNet18 and TRNet18 were
equivalent and the smallest, followed by AlexNet. TheTRNet50 was
slightly more prominent and the largest than ResNet50. When the model
recognized the type of soybean disease on the test set, the average
recognition time of TRNet18 was 51.03%, less than that of ResNet18,
and the average recognition time of TRNet50 was 36.33%, less than that
of ResNet50. The above analysis showed that the model after transfer
learning improved the recognition accuracy on the test set. Based on
ensuring that the training model was not much different, it also reduced
the average recognition time of each test set disease image, The transfer
learning using Fine-tuning showed a significant advantage in the
M. Yu et al. Ecological Informatics 76 (2023) 102096

Table 9 Table 10
Comparison of training and recognition performance of different models. Model performance evaluation.
Model AlexNet ResNet18 TRNet18 ResNet50 TRNet50 Model Disease type Precision Recall F1-score Macro-F1
Perfor-
SGH 100 100 100
mance index
SBLS 59.57 60.81 60.18
AlexNet 76.72
Model size /MB 55.69 42.76 42.70 90.14 97.95 SFLS 77.01 88.16 82.21
Average SPLS 69.01 60.49 64.47
identification 0.065740 0.096357 0.047184 0.087842 0.055930 SGH 100 100 100
time /s SBLS 97.92 98.91 98.41
ResNet18 98.08
SFLS 98.99 95.10 97.01
SPLS 96.91 96.91 96.91
SGH 100 100 100
training process. TRNet18
SBLS 99.23 99.58 99.4
99.54
SFLS 99.58 99.57 99.57
Since Precision and Recall were contradictory indexes and each has
SPLS 99.31 99.04 99.17
its limitations. It was not rigorous to use one of them to evaluate the SGH 99.78 99.78 99.78
performance of a model. F1-score was an index of the classification SBLS 93.67 93.32 93.49
ResNet50 95.96
problem. It was the harmonic mean of Precision and Recall. Its value SFLS 97.57 98.42 97.99
range was [0,1]. F1-score was often used as the final evaluation method SPLS 92.78 92.34 92.56
SGH 100 98.84 99.42
to solve multi-classification problems in many machine learning
SBLS 98.77 98.21 98.49
competitions. TRNet50
SFLS 97.4 98.44 97.92
98.41
The Equation of the F1-score was as follows: SPLS 97.46 98.13 97.79

2 × Precision × Recall
F1 =
Precision + Recall
Where, TP represented a positive sample predicted by the model as a
positive class. FP represented a negative sample predicted by the model
as a positive class. And FN represented a positive sample predicted by
the model as a negative class. The closer the value of Precision, Recall,
and F1 was to 1, the better the model's performance.
The specific steps of using the F1-score to evaluate the classification
effect of the model were as follows:
(1) Calculation of Precision for each disease type.
Precision represented the ratio of each class with genuinely positive
samples predicted to be positive. The calculation method was as follows:
Precisionk =
TP
TP + FP
Where, k ∈ [0, 1, 2, 3]had four values, indicating that the type of
soybean disease used in this experiment was 4.
(2) Calculation of Recall for each disease type.
Recall represented the proportion of the sample that was predicted to
be positive in each type of the sample that was positive. The calculation
equation was:
TP
Recallk =
TP + FN
(3) Calculation of F1-score of each type based on the Precision and
Recall for the calculated disease type. The calculation method was as
follows:
2 × Precisionk × Recallk
F1k =
Precisionk + Recallk
(4)The final evaluation results were obtained by calculating the
average macro Macro-F1 of the F1-score for each disease type. The
calculation method was as follows:
1∑
Macro-F1 = F1k
n
According to the confusion matrix and Eqs. (17), (18), and (19), the
performance evaluation values of ResNet18, TRNet18, ResNet50, and
TRNet50 were shown in Table 10.
Table 10 listed the performance evaluation results of the five models.
First, compared with AlexNet, Macro-F1 of ResNet18 increased by
21.69%, reflecting the advantage of the residual structure. Then,
compared with ResNet50, Macro-F1 of ResNet18 increased by 2.45%
while reducing the number of parameters. After introducing transfer
learning to ResNet50, TRNet50 was constructed, and the Macro-F1
(15)
increased by 2.45%. Finally, the performance of the TRNet18 network
model was improved to 99.54% compared with ResNet18 under the
combination of transfer learning and residual neural network. There­
fore, the TRNet18 based on ResNet18 proposed in this paper had better
performance.
6. Discussion
The accuracy of crop disease diagnosis in ecological informatics
needs to be improved, there are few studies on soybean disease diag­
nosis, and related studies are mainly trained on large public datasets. In
this paper, a soybean disease recognition model based on transfer
(16) learning was constructed on the basis of residual neural network
ResNet18. The accuracy of the model was 99.53%, the model size was
42.70 MB, and the average disease recognition time was 0.047184 s. It
was an accurate, fast and efficient disease recognition model. The scale
of this model is 53.08%, smaller than that of the model in reference
(Dong et al., 2022), which was easy to be transplanted to the portable
mobile equipment in the agricultural field, so as to promote the intel­
ligent process of actual field operations. The average disease diagnosis
time of the model on the test set was shorter, which saved 82.25%
(17) compared with the references (Gu et al., 2021; Yadav et al., 2021), and
realized the rapid diagnosis of crop diseases. The sample size of the
proposed model was 29.21%, less than that of reference (Li et al., 2022),
and the disease recognition rate was 6.88%, higher than that of refer­
ences (Amanda et al., 2017; Bevers et al., 2022; Dong et al., 2022;
Gokulnath and Usha Devi, 2021; Gu et al., 2021; Jamali et al., 2023;
(18) Joshi et al., 2020; Kılıç and İnner, 2022; Li et al., 2021; Li et al., 2022; Li
et al., 2023; Liu et al., 2021; Patil Manoj and Manohar, 2022; Wang
et al., 2022a, 2022b; Yu et al., 2020; Zhao et al., 2021; Zhou et al.,
2021). This model could provide a basis for guiding agrological opera­
tions such as timely and precise application of medicine and fertilization
in the early stage of crop diseases to minimize the cost of ecological
(19) environment pollution and maximize crop yield and quality.
The proposed model achieved the recognition of different soybean
leaf diseases, but the simulation results showed that three diseases were
incorrectly recognized. In the process of disease image acquisition, the
time period with uniform illumination should be selected under the
same sun angle to reduce the interference of external factors on the
disease image set. If there was a large geometric deviation in the images,
it can be corrected according to the bilinear projection mapping algo­
rithm (Guan et al., 2018) of our research group. In this study, the single
leaf image of soybean disease was used as the input of the network
model, but in the early stage of disease infection, the regional

15
M. Yu et al.
characteristics of the disease spots were not obvious, which was not
conducive to the diagnosis of the disease. There are similarities in some
characteristics of different disease performance traits, which increased
the recognition difficulty. It is necessary to optimize more effective al­
gorithms for distinguishing characteristics to achieve accurate disease
recognition. In this study, soybean disease images were acquired in the
field environment, but the color image band was difficult to reflect the
internal differences of infection by multiple pathogens. Therefore, in the
natural field environment, the disease diagnosis of soybean plants at the
early stage with infection by pathogens and cross infection by multiple
pathogens is still a further problem to be solved in this study.
The simple structure and strong generalization ability of the model
proposed in this study were conducive to its practical application and
promotion. The non-destructive soybean disease diagnosis method in
this paper can also be linked with genetics to solve the biological stress
problem of crops. Based on the principle of early detection and early
treatment, every stage of monitoring, forecasting and diagnosis of soy­
bean plant diseases is extremely important for crop quality and high
yield. The research results in this study could provide a basis for guiding
agronomic operations such as timely and precise application of medicine
and fertilization in the early stage of soybean diseases, so as to maximize
the yield and quality of soybean with the minimum cost of ecological
environment pollution. Furthermore, the multivariate deep learning
model could not only be combined with the prior knowledge of plant
protection experts to expand the scope of plant disease detection, but
also was integrated into a portable mobile terminal to promote the
intelligent process of actual field operations.
7. Conclusion
A recognition method for soybean leaf diseases was proposed based
on transfer learning. In this study, the digital camera was used to acquire
digital images of diseased soybean leaves at different growth stages (R2
~ R5). The standard gray card was used to register the image's color to
reduce the error caused by the color deviation. OTSU was used to
effectively extract diseased soybean leaf images and reduce the influ­
ence of background in the process of disease image data recognition. In
addition, the data enhancement method was applied to expand the
image samples of the disease single leaf image, which provided sufficient
and reliable experimental data sets for the neural network to extract
image feature information. The weights of the convolutional layer of the
pre-training model were transferred to the convolutional layer of the
ResNet18 model to establish the recognition model (TRNet18). The
average recognition rate was 99.53% with the recognition time of
0.047184 s. Compared with the traditional models, the disease recog­
nition accuracy was increased by 1.11%–22.99%, the recognition time
was saved by 15.64%–51.03%, and the Macro-F1 value was relatively
increased by 1.13%-22.82. This result can provide not only a reliable
basis for disease prevention and control in soybean planting and
exemplary management, but also technical support for the soybean field
application process.
Author contributions
Conceptualization, H.G.; methodology, H.G., X.M. and M.Y.; vali­
dation, M.Y.; formal analysis, H.G., X.M., M.Y.; resources, H.G. and X.
M.; writing—original draft preparation, M.Y.; writing—review and
editing, H.G. and M.Y.; visualization, M.Y. All authors have read and
agreed to the published version of the manuscript.
Funding
This research was funded by the National Natural Science Founda­
tion of China (grant number: 31601220), Natural Science Foundation of
Heilongjiang Province, China (grant number: LH2021C062 and
LH2020C080), Heilongjiang Bayi Agricultural University Support
16
Ecological Informatics 76 (2023) 102096
Program for San Heng San Zong, China, (grant number: TDJH202101
and ZRCQC202006).
Institutional review board statement
Not applicable.
Informed consent statement
Not applicable.
Declaration of Competing Interest
The authors declare no conflict of interest.
Data availability
The authors do not have permission to share data.
References
Alex, K., Ilya, S., Geoffrey, E.H., 2017. ImageNet classification with deep convolutional
neural networks. Commun. ACM 60 (6), 84–90. https://doi.org/10.1145/3065386.
Amanda, R., Kelsee, B., Peter, M.C., Babuali, A., James, L., David, P.H., 2017. Deep
learning for image-based cassava disease detection. Front. Plant Sci. 8, 1852.
https://doi.org/10.3389/fpls.2017.01852.
Atila, U., Ucar, M., Akyol, K., Ucar, E., 2021. Plant leaf disease classification using
EfficientNet deep learning model. Ecol. Informat. 61, 101182 https://doi.org/
10.1016/j.ecoinf.2020.101182.
Bevers, N., Sikora, Edward J., Hardy, Nate B., 2022. Soybean disease identification using
original field images and transfer learning with convolutional neural networks.
Comput. Electron. Agric. 203, 107449 https://doi.org/10.1016/J.
COMPAG.2022.107449.
Cen, H.Y., Zhu, Y.M., Sun, D.W., Zhai, L., Wan, L., Ma, Z.H., 2020. Current status and
future perspective of the application of deep learning in plant phenotype research.
Transact. Chinese Soc. Agricult. Eng. 36, 1–16. https://doi.org/10.11975/j.
issn.1002-6819.2020.09.001.
Chang, K.F., Hwang, S.F., Ahmed, H.U., Strelkov, S.E., Harding, M.W., Conner, R.L.,
McLaren, D.L., Gossen, B.D., Turnbull, G.D., 2018. Disease reaction to Rhizoctonia
solani and yield losses in soybean. Can. J. Plant Sci. 98 (1), 115–124. https://doi.
org/10.3969/j.issn.1674-3547.2021.06.006.
Dai, F., Wang, F.C., Yang, D.Z., Lin, S.M., Chen, X., Lan, Y.B., Deng, X.L., 2022. Detection
method of citrus psyllids with field high-definition camera based on improved
cascade region-based convolution neural networks. Front. Plant Sci. 12, 816272
https://doi.org/10.3389/fpls.2021.816272.
Dong, P., Wei, M.H., Shi, L., Guo, W., 2022. Research and application of transfer learning
in identification of maize leaf diseases. J. Chinese Agricult. Mechanizat. 43 (03),
146–152. https://doi.org/10.13733/j.jcam.issn.2095–5553.2022.03.020.
Feng, X., Li, D.D., Wang, W.J., Zhang, G.Q., Liu, H.J., Sun, Y.S., Liang, S., Yang, Y.,
Zhang, H.C., Zhang, H., 2021. Image recognition of wheat leaf diseases based on
lightweight convolutional neural network and transfer learning. J. Henan Agricult.
Sci. 50 (04), 174–180. https://doi.org/10.15933/j.cnki.1004-3268.2021.04.023.
Gokulnath, B.V., Usha Devi, G., 2021. Identifying and classifying plant disease using
resilient LF-CNN[J]. Ecolog. Informat. 63, 101283 https://doi.org/10.1016/J.
ECOINF.2021.101283.
Gu, Y.H., Yin, H.L., Jin, D., Park, J.H., Yoo, S.J., 2021. Image-based hot pepper disease
and pest diagnosis using transfer learning and fine-tuning. Front. Plant Sci. 12,
724487 https://doi.org/10.3389/FPLS. 2021.724487.
Guan, H.O., Liu, M., Ma, X.D., 2018. Automatic soybean disease diagnosis model based
on image correction technology. J. Jiangsu Univ. Nat. Sci. Edit. 39 (4), 409–413,
430. https://doi.org/10.3969/j.issn.1671-7775.2018.04.007, 430.
Guo, D., Zhang, H., Yang, J.T., Yu, L.Y., Wu, B.J., Li, M.M., Huang, W., Yuan, Z.C., 2021.
Occurrence status and green control counter measure for diseases and insect pests of
soybeans in Shandong Province. Soybean Sci. Technol. 4, 27–30. https://doi.org/
10.3969/j.issn.1674-3547.2021.04.006.
Jamali, M., Soufizadeh, S., Yeganeh, B., Emam, Y., 2023. Wheat leaf traits monitoring
based on machine learning algorithms and high-resolution satellite imagery. Ecol.
Informat. 74, 101967 https://doi.org/10.1016/J. ECOINF.2022.101967.
Joshi, R.C., Kaushik, M., Dutta, M.K., Srivastava, A., Choudhary, N., 2020. VirLeafNet:
Automatic analysis and viral disease diagnosis using deep-learning in Vigna mungo
plant[J]. Ecolog. Informat. 61, 101197 https://doi.org/10.1016/J.
ECOINF.2020.101197.
Kılıç, C., İnner, B., 2022. A novel method for non-invasive detection of aflatoxin
contaminated dried figs with deep transfer learning approach[J]. Ecolog. Informat.
70, 101728 https://doi.org/10.1016/J. ECOINF.2022. 101728.
Li, J.H., Lin, L.J., Tian, K., Al, A.A., 2020. Detection of leaf diseases of balsam pear in the
field based on improved Faster R-CNN. Transact. Chinese Soc. Agricult. Eng. 36 (12),
179–185. https://doi.org/10.11975/j.issn.1002-6819.2020.12.022.
M. Yu et al.
Li, L.P., Shi, F.P., Tian, W.B., Chen, L., 2021. Wild plant image recognition method based
on residual network and transfer learning. Radio Eng. 51 (09), 857–863. https://doi.
org/10.3969/j.issn.1003-3106.2021.09.003.
Li, C., Zhen, T., Li, Z.H., 2022. Image classification of pests with residual neural network
based on transfer learning. Appl. Sci. 12 (9), 4356. https://doi.org/10.3390/
APP12094356.
Li, E.L., Wang, L.W., Xie, Q.J., Gao, R., Su, Z.B., Li, Y.G., 2023. A novel deep learning
method for maize disease identification based on small sample-size and complex
background datasets. Ecol. Informat. 75, 102011 https://doi.org/10.1016/J.
ECOINF.2023.102011.
Liu, B., Ding, Z.F., Tian, L.L., He, D.J., Li, S.Q., Wang, H.Y., 2020. Grape leaf disease
identification using improved deep convolutional neural networks. Front. Plant Sci.
11, 1082. https://doi.org/10.3389/fpls.2020.01082.
Liu, A.Y., Wu, Y.Z., Zhu, X.N., Fan, G.H., Yue, Y., Zhang, Y.H., 2021. Corn disease
recognition based on deep residual network. Jiangsu J. Agricult. Sci. 37 (01), 67–74.
https://doi.org/10.3969/j.issn.1000-4440.2021.01.009.
Ma, X.D., Zhu, K.X., Guan, H.O., Feng, J.R., Liu, M., Zheng, M., 2019. Crop image feature
extraction and its applicationin disease diagnosis. J. Heilongjiang Bayi Agricult.
Univ. 31 (2), 93–99. https://doi.org/10.3969/j.issn.1002-2090. 2019. 02. 015.
Meng, F.L., Yu, J.Y., Li, C.J., Huang, M.H., Zhao, L., Wang, X., Jiang, Y., Qin, R.F.,
Wang, C.L., 2022. Research progress on occurrence and management of soybean cyst
nematode in Northeast China. J. Northeast. Agric. Univ. 53 (01), 87–94. https://doi.
org/10.19720/j.cnki.issn.1005-9369.2022.01.010.
Patil Manoj, A., Manohar, M., 2022. Enhanced radial basis function neural network for
tomato plant disease leaf image segmentation[J]. Ecolog. Informat. 70, 101752
https://doi.org/10.1016/J. ECOINF.2022. 101752.
Schirrmann, M., Landwehr, N., Giebel, A., Dammer, K., 2021. Early detection of stripe
rust in winter wheat using deep residual neural networks. Front. Plant Sci. 12, 475.
https://doi.org/10.3389/fpls.2021.469689.
17
Ecological Informatics 76 (2023) 102096
Wang, C., Wang, C.Q., Liu, J.M., 2022a. Identification of maize leaf diseases based on
deep learning. Modern Agricult. Res. 28 (06), 102–106. https://doi.org/10.19704/j.
cnki.xdnyyj.2022.06.020.
Wang, K.L., Chen, K.Y., Du, H.Y., Liu, S., Xu, J.W., Zhao, J.F., Chen, H.L., Liu, Y.J.,
Liu, Y., 2022b. New image dataset and new negative sample judgment method for
crop pest recognition based on deep learning models. Ecol. Informat. 69, 101620
https://doi.org/10.1016/J.ECOINF.2022.101620.
Yadav, S., Sengar, N., Singh, A., Singh, A., Dutta, M.K., 2021. Identification of disease
using deep learning and evaluation of bacteriosis in peach leaf[J]. Ecolog. Informat.
61, 101247 https://doi.org/10.1016/J.ECOINF.2021.101247.
Yu, X.D., Yang, M.J., Zhang, H.Q., Li, D., Tang, Y.Q., Yu, X., 2020. Research and
application of crop diseases detection method based on transfer learning. Transact.
Chinese Soc. Agricult. Mach. 51 (10), 252–258. https://doi.org/10.6041/j.
issn.1000-1298.2020.10.028.
Yu, M., Ma, X.D., Guan, H.O., Liu, M., Zhang, T., 2022. A recognition method of soybean
leaf diseases based on an improved deep learning model. Front. Plant Sci. 13,
878834. https://doi.org/10.3389/fpls.2022.878834.
Zhao, H.Q., Yang, Y.F., Liu, Z.L., Song, R., 2021. Step-by-step identification method of
crop leaf diseases based on transfer learning. Bull. Surv. Mapp. 07, 34–38. https://
doi.org/10.13474/j.cnki.11-2246.2021.0205.
Zhao, S.Y., Liu, J.Z., Bai, Z.C., Hu, C.H., Jin, Y.J., 2022. Crop pest recognition in real
agricultural environment using convolutional neural networks by a parallel attention
mechanism. Front. Plant Sci. 13, 839572 https://doi.org/10.3389/
fpls.2022.839572.
Zhou, H.W., Shen, H.Y., Yuan, X.P., Li, X.D., 2021. Research on identification method of
apple leaf diseases based on transfer learning. J. Chinese Agricult. Mechanizat. 42
(11), 151–158. https://doi.org/10.13733/j.jcam. issn. 2095–5553.2021.11.23.