Expert Systems With Applications

Resnet-Attention Based Spatial- Channel Network for Plant Growth Prediction from
Time Series Data
--Manuscript Draft--

Manuscript Number: ESWA-D-23-08549

Article Type: Full length article

Keywords: Phenotyping; Plant Growth; Future Image Frames; Deep Learning; Time Series

Abstract: Phenotyping undergoes the quantitative analysis of bio-chemical, physiological and

anatomical plant features. The natural way of plant growth is time consuming process
and slows down the experimentation of phenotyping. Artificial intelligence (AI) based
models provide an automatic way and overcoming Phenotyping challenges. This work
introduces an enhanced deep learning (DL) model for predicting the growth of plants
by making segmentation ground truth into the future. This work has three major stages;
initially, the entire annotations are automatic and generate images with XML based
annotations. Then, dual tree complex wavelet transform (DT-CWT) is provided to the
original data for reducing the overfitting and reducing noise. The DL model ResNet-
Attention based Spatial- channel module (R-At-SCM) is used for extracting features
and used to forecast future image frames. The experimentation is carried out on two
benchmark datasets like Arabidopsis thaliana and Brassica rapa. The results depict
better performance in terms of the error measures like MSE, RMSE, MAE and MAPE
values. This model is well suited and trainable for various types of plants and
mutations.

Powered by Editorial Manager® and ProduXion Manager® from Aries Systems Corporation
Manuscript Click here to access/download;Manuscript;Manuscript.docx

Click here to view linked References

Resnet-Attention Based Spatial- Channel Network for Plant

Growth Prediction from Time Series Data

Dr. Subbiah Narayanan1*, Dr. Dhamodaran Sridevi2

1*
Assistant Professor (Sel.G), Department of Information Technology, SRM Valliammai

Engineering College, Chengalpattu, Tamilnadu, India

1*
Email: drsnarayanan0027@gmail.com
2
Assistant Professor (Sel.G), Department of Information Technology, SRM Valliammai

Engineering College, Chengalpattu, Tamilnadu, India

2
Email: dsridevi443@hotmail.com
Abstract

Phenotyping undergoes the quantitative analysis of bio-chemical, physiological and

anatomical plant features. The natural way of plant growth is time consuming process and

slows down the experimentation of phenotyping. Artificial intelligence (AI) based models

provide an automatic way and overcoming Phenotyping challenges. This work introduces an

enhanced deep learning (DL) model for predicting the growth of plants by making

segmentation ground truth into the future. This work has three major stages; initially, the

entire annotations are automatic and generate images with XML based annotations. Then,

dual tree complex wavelet transform (DT-CWT) is provided to the original data for reducing

the overfitting and reducing noise. The DL model ResNet-Attention based Spatial- channel

module (R-At-SCM) is used for extracting features and used to forecast future image frames.

The experimentation is carried out on two benchmark datasets like Arabidopsis thaliana and

Brassica rapa. The results depict better performance in terms of the error measures like MSE,

RMSE, MAE and MAPE values. This model is well suited and trainable for various types of

plants and mutations.

Keywords: Phenotyping; Plant Growth; Future Image Frames; Deep Learning; Time Series.

1. Introduction

Plant growth is caused due to the internal and external terms like nutrition supply and

environmental factors (Lynch et al,2012). For improving plant harvesting in commercial

agriculture, it is necessary to know how the environmental factor affects the plant's growth.

Plant phenotype is a challenge to deal with this issue in the agricultural field. Generally, a

plant phenotype is the physical and biochemical behaviors and is affected due to interactions

among environmental factors and genetic characteristics (Minervini et al,2015) Every plant

differs by species; hence it is essential to compute the relation between environmental factors

and the phenotype of every species of plant. For solving this issue, the emergence of plant
phenotypingmodels for different plant species has been carried out for years(Li et

al,2018,Rochefort et al,2019,Li et al,2018).

The automatic plant phenotyping models based on the image have been introduced

because of the advancement of several kinds of low cost cameras (Tsaftaris and

Noutsos,2009). `These models have shown better achievements in enhancing the efficiency,

throughput and scale of phenomic study. The stages like segmentation, extraction of features

and analyzing data are the major for progress in plant phenotyping models based on images

(Sakurai et al, 2018). The benefits of models based on image have the characteristics like:

they are in non-destruction manner and also permit to observe phenotype of plants in high

throughput. The simple models of plants like convex hull, height and centre of mass have

been computed from the images in earlier days. The recent emergence of data mining

techniques like deep learning (DL) models considered pixel-by-pixel segmentation of plant

region in an image. It is essential to analyze the phenotyping technologies to clarify the

process in agriculture using computer vision approaches (Li, et al, 2014 ,Alhaityand,Abbod

2020).

Plant growth modelling is a complex process because of massive data variations like

description level, scale of interest and combination of environmental variables.Themultiple

stage time series prediction is the predicting the time series in several time stages ahead to the

future (Schmidhuber ,2015,Sorjamaa et al, 2007,Taieb et al ,2012). When comparing the

single stage ahead prediction, multi-stage ahead prediction can ensure more advantages. In

the existing works, there are three major models are introduced to manage multi-stage ahead

prediction processes. They are recursive models, direct models and multi-output prediction

model (Liakos et al, 2018,Benos et al,2021, Singh et al, 2016, pallathadka et al,2021)

Existing image processing and evaluating models have shown better performance in plant

phenotyping. Further, DL based models have depicted excellent accuracy. These models can
compute various plant traits effectively. These models achieved enhanced performance when

compared to existing phenotyping models. The experts depend highly on these extraordinary

outcomes for capturing complicated features and plant structures on below and above the

ground. DL models like Convolution Neural Networks (CNN) have shown remarkable

advantages in the field of crop yield prediction, plant disease prediction and plant growth

prediction (Darwish et al, 2020,Francis et al, 2019,Adams et al ,2020).

Motivation

Precision agriculture is a research field that aims at protecting the production of food.

Precious agriculture is the process of monitoring plant growth in plant factory production for

observing the characteristics and predicting the estimation of yield prediction. Phenotyping

undergoes the quantitative analysis of anatomical, physiological and biochemical

characteristics of plants. Recently, time series analysis and prediction are considered as a hot

research topic in several applications like financial stock forecasting, anomaly detection,

medical imaging, plant growth prediction and smart agriculture. Generally, time series data

are generated as a series of observed data collected in sequential order. But, the complexity is

somewhat high and makes the analysis process as a complex process. Generally, the growth

of plants is based on measurements of variation in plants, their size and structure. Various

traditional solutions for automatic plant phenotyping are either costly or invasive. Generally,

invasive models are not suggested since they need to uproot and cut plants. It is also complex

to use devices that rely on costly equipment like tomographs and multi-spectral cameras.

Hence, Machine learning (ML) and deep learning (DL) are used for analysing the data.

Main Contributions

The forecast contributions of the proposed work are:

 To introduce new databases with new pre-processing stages to make the data

annotation.
  To introducedeep learning (DL) model ResNet-Attention based Spatial- channel

module (R-At-SCM) is used for extracting features and used to forecast future image

frames.

 The performance of the proposed plant growth prediction is compared on a

benchmark database and compared with other baseline models.

The remainder of the paper is organized as follows: Section 2 presents a recent related work

based on plant growth prediction using deep learning models, Section 3 presents the proposed

plant growth prediction, Section 4 is the results and discussions and the entire work is

concluded in Section 5.

2. Related Works

Some of the recent research works based on plant growth prediction using deep learning

models are listed below.

Son et al. (Son etal ,2020 ) used ML models for rice crop yield prediction in the country

of Taiwan by time series satellite database. This work utilized the data from the period of

2000 to 2018 and it has the stages like data pre-processing for generating smooth time series

NDVI (Normalized Difference Vegetation Index), the models establishment for yield

prediction and evaluation of the models. The outcomes were compared over government

yield statistics and provided better performance with better MAE and RMSE values between

11.7% and 7.1% respectively.

Kim et al. (Kim et al ,2022) introduced a novel model on the basis of the prediction of the

plant using DL and Spatial Transformation. This work has dual subnets to estimate the shape

and RGB images. That is, the plant image shape was learned and the RGB channel was

reconstructed. The growth characteristics were computed by a set of affine transform

variables. This work used environmental factors in closed environmental conditions and

aimed for maximizing the harvest of the plant.

 Chang et al. (Chang etal, 2021)introduced multi-DL model for extracting high quality

high-throughput phenotyping platform (HTPP) data. This model established an association

among crop yield performance, HTTP data and selected essential intervals of time using DL

models. This work considered Arabidopsis and features were extracted using U-Net with SE-

ResXt101 encoder. At last, projected area (PA) was declared to be fresh weight (FW) and the

correlation coefficient (CC) among FW and predicted FW was computed as 0.85.

Yasrab et al. (Yasrab etal, 2021)introduced the prediction of plant growth using the DL

model from time series data. This work was introduced for generating future growth frames

of roots and crop leaves. This work considered two benchmark datasets and the images were

resized. The DL model Generative Adversarial Network (GAN) was used for providing a

forecast for the future of the input images. In the shooting process, the system generated the

correct segmentation mask and in roots, the system provided an extension to the RootNav

model. This model achieved higher SSIM and mIoU values of 94.6% for shoots and 76.8%

for roots.

Alhnaity et al. (Alhaity etal, 2019)introduced the LSTM model for predicting plant

growth and yield in greenhouse environmental factors. Two criteria like tomato yield

prediction and Ficusbenjamina stem growth were considered. The parameters like growth,

stem diameter values and former yield were utilized by the LSTM model for predicting the

growth. The performances of RMSE, MAE and MSE were considered and compared with

SVR and RF models. The LSTM model achieved better RMSE and MAE values of 0.073 and

0.03 for the tomato yield prediction.

Sakurai et al. (Sakurai et al,2019) introduced CNN-LSTM model for the prediction of

plant growth in future images from the prior images and the loss function was used for

optimizing the change of leaves among frames. For capturing the long-range dependencies,

this network was employed. The experimentations were considered for conditions like with
and without loss functions. Further, this model evaluated outcomes by the weighting

coverage score of each leaves as quantitative analysis and predicted values were considered

as qualitative analysis.

Rizkiana et al.(Rizkiana et al,2021)introduced an embedding sensing model for the

prediction of plant growth. This model has two phases; in the first phase, the image was pre-

processed by filters and statistical approaches were used for time series cropping. Then the

prediction phase was carried out by ML model. This model was executed on embedding

devices. The evaluation was carried out on real time data and considered 16 plants. Further,

the monitoring system was designed and computed and predicted the leaf area. Finally, it was

proved that the accuracy increased with the growth of the database.

Uchiyama et al. (Uchiyama et al ,2017) introduced ANN (artificial neural network) for

the prediction of lettuce. This network was analyzed by a various number of nodes at the

hidden layer by considering the inputs like temperature, light intensity and humidity. The

lettuce plant was tested in plant factory production. This model achieveda better R2 value of

0.98 for training and 0.72 for testing and an RMSE value of 0.03. among all comparisons,

architecture 4-7-1 obtained better results.

3. Proposed Methodology

Plant growth prediction into future has the capacity to improve the plant cycles, and predicts

phenotype and processes more effectively. Attention based models are used for forecasting

the future frames in several fields. Hence, in this work ResNet-Attention based Spatial-

channel module (R-At-SCM) is used for capturing phenological features from the root and

shoot images and also predicting future phenological features on the basis of present growth

strategies. This model has ability to correctly predict plant models and also minimizes the

time needed for growing and measuring plants. Figure 1 shows the framework of the

proposed plant growth prediction.

3.1 Dataset collection

In this work, the databases like Arabidopsis thalianaand Brassica rapa are considered for the

plant growth prediction process. These datasets are divided into 70% for training and 30% for

testing.

Arabidopsis thaliana: This (Wilson et al,2015)database is obtained from Arabidopsis seeds

and these plants are grown in the regulated environmental condition for 7 days. This database

has 58 plates of plants and every plate has 5 plants and they are rotated by 900. The digital

cameras are utilized for obtaining the image from vertical leaf orientation. In this database,

2502 images (47 plants) are used for training and 694 images (11 plants) are used for testing.

Brassica rapa: It (Nesteruk et al, 2020) is one kind of mustard spinach vegetable and in this

work, the RGB-database of Brassica (Komatsuna) is utilized. The plants in this database in

grown through a hydroponic culture toolkit. These plants are grown in the regulated

environmental condition and lighting and temperature were set to 2400 lux and 280. The

images are obtained by Intel RealSense SR300 camera with the resolution of 640 x 480

pixels. In this database, 480 images (4 plants) are used for training and 120 images (1 plant)

are used for testing.

Pre-processing

Dimensionality reduction
Input image
using DTCWT

RGB segmentation masks

Growth prediction using

R-At-SCM

Future root frames

Figure 1: Framework of proposed plant growth prediction

3.2 Pre-processing

ResNet-Attention based Spatial- channel module (R-At-SCM) model trains on the databases

and predicts the multi-stage growth. Database annotations are time consumption in the

biological field because of the complicated nature of the plants. When the database

annotations are already available, then dimensionality reduction and prediction processes are

carried out directly. For instance, the Brassica database has segmentation masks and no pre-

processing of data is needed. Further, Arabidopsis doesn’t have segmented masks. Hence,

RootNav 2.0 is used for this process and root topologies are saved in XML format. This

format ensures an easy way of using root data modeling (RDM) with various modeling and it

preserves the time series information.

3.3 Dimensionality reduction using DTCWT

DTCWT is used for denoising of images while managing the non-stationary nature of

obtained time series data. The process of DTCWT to represent, decompose and reconstruct

the original data is given in the following section:

In this algorithm, two parallel trees ( x, y ) are utilized and the one branch of x is

cancelled by the respective branch of y . The DT-CWT has the characteristics like good

reconstruction, shift invariance and better directional selectivity. WhenDT-CWT is applied to

2-D signals, four trees structures (2 trees for row and 2 trees for column) are obtained. Then,

the complex filters are used for separating negative and positive frequency elements in one

dimensional. In this work, DT-CWT is applied to two dimensional signals to obtain four trees

structures. The image f (a, b) is decomposed by complex wavelet and scaling function and it

is expressed as:

f ( a, b)  B jo , l jo , l (a, b)    G kj ,l ,  jk,l (a, b)

k j 1 lZ 2
(1)
lZ 2
where jo is the decomposition stage, B jo , l and G kj ,l are is the scaling and wavelet terms.  jo ,

l ( a , b) and  jk,l (a, b) are the scaling and wavelet functions.

3.4 ResNet-Attention based Spatial- channel module (R-At-SCM)

Once the images are denoised using DTCWT, prediction of future frame process is carried

out. The enhanced DL model R-At-SCM is used to forecast future image frames. This DL

model has three layers like core network, sub-networks and embedded layers. Initially, in the

core network, 3 residual blocks of ResNet-152 is utilized. Then, feature maps are resized and

given to the sub-networks. Every sub-network has four blocks of ResNet-152 and Attention

based Spatial- channel module. Then, sub-networks are used for extracting high dimensional

features ( Fe1 , Fe2 ,....., FeN ) . Figure 2 shows the structure of R-At-SCM model. Here, the

input image is provided into ResNet-152 (core network). The scale networks are generated

using bilinear model. Every scale feature is given to the respective sub-networks and

followed by attention based spatial channel module. These networks are trained and an

embedded layer is used to fuse these features for producing the final output.

The embedded layer has 2 convolutional and 1 fully connected (FC) layer. Once the feature is

resized, the convolutional layer is utilized for embedding the comprehensive features ( Fen )

into final features ( F final ) . Then, this F final is provided into FC layer. The loss function used

in this cross entropy loss and it is expressed as:

D
Loss   log( p(d )) q(d ) (2)
d 1

where D is the number of plants in the training set, p (d ) and q (d ) are the predicted

probability and ground truth.

 FC

Attention based
Scale 1 sub-
spatial channel
network
module
FC
Attention based
ResNet- Scale 2 sub- spatial channel
152 network module

…………

…………

……...
FC

Attention based
Scale n sub- spatial channel
network module
FC

Embedded
layer

Figure 2: Structure of R-At-SCM model

Attention based Spatial- channel module: There are 2 modules in this layer; they are

Spatial attention module (SAM) and Channel attention module (CAM) as shown in Figure

3.SAM is used for selecting discriminative pixels and CAM is used for selecting the essential

channels. After combining the features, the tensor product and convolutional layer are

followed. The feature maps are normalized by feature maps using sigmoid function. Then, the

hwd
weights of final maps and original maps are fused to obtain the final features. Let g  R is

the input of the module, where h is the height, w is the width and d is channel. Once the

h w1
process of SAM and CAM are completed, the attention maps like S  R (spatial) and

d  R11d . For normalizing the attention map, sigmoid is used. At last, element wise

multiplication is used to fuse the final maps weights and it produces final attention feature

maps.
 SAM Spatial maps

Reduce
Resize

Conv(1,3*3,2) Conv(1,1*1,1) Conv(1,1*1,1)

CAM Attention
Channel maps Sigmoid
maps
Pool
(h*w*1) Conv(1,1*1,1) Conv(1,1*1,1)
Original Attentive
feature maps
maps

Figure 3: Attention-based Spatial- channel module

Spatial attention module (SAM): This layer is used to automatically find the discriminative

hwd
pixels of root images. Group of tensors g  R is set into SAM and it has 4 layers. For

compressing the input feature maps, channel-stage average-pooling layer is used and it is

expressed as:

1 d
s k ,l   g k ,l , m
d m 1
(3)

where g k ,l ,m is the m th channel at the direction of ( k , l ) . After the layer of channel-stage

average-pooling, convolutional filter with 3x 3 filter and resized filter are provided. At last,

scale convolution layer with a 1x 1 filter is used for learning the feature scale to fuse with

spatial features and the activation function used in ReLU.

Channel attention module (CAM): This layer is used to automatically find the discriminative

feature maps channel. CAM has 3 layers and average-pooling layer is used for aggregating

spatial features into channel c m . It is expressed as:

1 h w
cm   g k ,l ,m
hw k 1 l 1
(4)

Finally, 2 convolutional layers are used for learning feature scale to fuse with spatial features

for predicting the future frames. The training process of R-At-SCM is given in Algorithm 1.
Algorithm 1: Training process of R-At-SCM

Input: root image training data

Output: F final

Step 1: m  1

Step 2: while m  M do

Step 3: minimize Loss

Step 4: Update the parameters of core network

Step 5: m  1

Step 6: end while

Step 7: minimize Loss

Step 8: Return F final

4. Results and Discussion

In this work, the entire implementations have been processed in a system with 8 GB RAM

and an Intel Core i5 CPU with 3.0 GHz speed. For evaluating the proposed model, the

simulation tool Python is exploited. Table 1 shows the Hyperparameters of R-At-SCM.

Table 1: Range of Hyperparameters of R-At-SCM

Hyperparameters Range

Size of batch 32

Learning rate 0.001

Optimizer adam

pooling average

Convolutional 3 depth 8

Convolutional 4 depth 23
4. 1 Performance measures

The measures used for comparing the performance are Mean Squared Error (MSE), Peak

Signal to Noise Ratio (PSNR), Structural Similarity Index (SSIM), Root Mean square error

(RMSE):, Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), R-

squared (R2) and mean intersection of union (mIoU). The expressions are given below:

MSE: It is utilized for calculating the mean squared variation among the predicted and GT

images. When the value of MSE is less, the prediction is better and it is expressed as:

1 n
MSE (a, b)   (ai  bi ) 2
n i 1
(5)

where ai and bi are the GT and predicted images.

RMSE: It measures the square root of difference among the predicted and GT images and it

is indicated as:

RMSE 
1 n

 a b
ni 1 i i
2  (6)

MAE: It is the difference among the predicted and GT images and it is indicated as:

 ai  bi 
1 n
MAE 
2
(7)
n i 1

MAPE: It measures the deviation from GT images in terms of the percentage and it is

expressed as:

1 n ai  bi
MAPE    100 (8)
ni 1 b
i

R2: It measures the linear relation among the predicted and GT images and it is indicated as:

1/ 2
 1 n

   a  b 2 
 n

i i 

R 2  1   i  1 
(9)
   bi
i 1

  
  
PSNR: This measure is used for finding the image quality and it is the ratio of maximum

feasible power of signal and distorting noise power. It is defined as:

P2
PSNR  10 log 10 (10)
MSE

SSIM: This measure is used for finding the similarity between the predicted and GT images.

It is expressed as:

( 2cd  e1 )( 2cd  e2 )
SSIM 
( c2  d2  e1 )( c2  d2  e2 ) (11)

where  c and  d are the average of c and d . c 2 and  are the variance and covariance.

mIoU: It is the average of IoU of the segmented roots over the entire images of the dataset

and it is expressed as:

1 n T p  Tn
mIOU  
n 1 T p  Tn  F p  Fn
(12)

where true positive (T p ) false positive ( Fp ) true negative (Tn ) false negative ( Fn )

(a)Experimental results of Arabidopsis thaliana dataset

The first database of root images is utilized for analyzing the performance on complicated

RDM images (Wilson et al,2015) The model is re-trained for generating 3 future

segmentation ( m  1 to m  3) masks with six inputs. Figure 4 depicts sample images of the

input and future frames. The rates of root growth of GT images and predicted frames are

computed. The mean growth of predicted frames is varied from time of m  1 to m  2 and

m  2 to m  3 . Figure 5 shows the GT image, processed GT image, predicted frame and

processed frame. After computing these frames, the performance of mIoU is computed.

m2 m 1 m m 1 m2 m3

 (a)

(b)

Figure 4: Samples images of (a) input (b) future frames

(a) (b) (c) (d)

Figure 5: Qualitative analysis of (a) GT (b) processed GT (c) predicted frame (d) processed

frame

The metric mIoU is utilized for evaluating segmentation performance and used to find the

part of the overlapped region between segmentation and GT images. This part is identified

for every class and the average value is obtained to find mIoU. The performance of the

predicted frames is measured by computing the values of mIoU.

Table 2: Performance comparison of mIOU of proposed and existing approaches

Methods m 1 m2 m3

ResNet-152 0.891 0.883 0.828

LSTM 0.823 0.945 0.912

Bi-LSTM 0.945 0.962 0.945

proposed 0.984 0.989 0.991

Table 2 shows the performance comparison of mIOU of proposed and existing approaches. In

this metric, when the value of mIoU is high, the plant growth prediction is better. The DL

models like ResNet-152, LSTM and Bi-LSTM are compared with the proposed R-At-SCM

model for the time steps of m  1 , m  2 and m  3 . For the time steps of m  3 , the

proposed R-At-SCM achieved a better mIoU value. Hence, it is proved that this model can be

efficiently used for plant growth prediction.

(b) Experimental results ofBrassica rapadataset

The second database considered for this work is Brassica rapa and the test set has full growth

pattern of the plant. R-At-SCM architecture is trained for forecasting 6 future frames using 6

inputs. Figure 6 depicts the qualitative analysis between the GT and predicted frames. Figure

7 shows the GT, processed GT, predicted frame and processed frame.

(a)

(b)

(c)

Figure 6: Qualitative analysis (a) Input frames (b) GT (c) Predicted frames
 (a) (b) (c)

Figure 7: Qualitative analysis of (a) GT (b) processed GT (c) predicted frame (d) processed

frame

(a)

(b)

Figure 8: Accuracy and loss curves of the proposed model on (a) Arabidopsis thaliana and

(b) Brassica rapa datasets

Figure 8 shows the accuracy and loss curves of the proposed model on the datasets like

Arabidopsis thaliana and Brassica rapa. Here, the performance is carried out by varying the

training epoch from 0 to 100.

Table 3: Comparison of Image quality of various approaches on Brassica rapa dataset

PSNR

Methods m 1 m2 m3 m4 m5 m6

ResNet-152 23.5 29.1 32.4 41.3 26.4 36.2

LSTM 34.6 37.6 33.8 39.7 29.3 37.4

Bi-LSTM 31.8 38.6 34.6 38.1 36.9 42.1

proposed 46.2 47.9 49.8 48.3 51.3 45.3

SSIM

Methods m 1 m2 m3 m4 m5 m6

ResNet-152 0.972 0.941 0.923 0.882 0.923 0.951

LSTM 0.934 0.953 0.958 0.886 0.904 0.962

Bi-LSTM 0.956 0.961 0.971 0.894 0.912 0.974

proposed 0.983 0.991 0.993 0.981 0.989 0.983

Table 3 depicts the comparison of Image quality of various approaches like ResNet-152,

LSTM, Bi-LSTM and the proposed R-At-SCM on the Brassica rapa dataset. The

performance is carried out by varying the frames from m  1 to m  6 . The frame m  3

achieved better SSIM value of 0.993 and the frame m  5 achieved a better PSNR value of

51.3.

Table 4: Comparison of error measures of various approaches on Arabidopsis thaliana

dataset

MSE

Methods m 1 m2 m3 m4 m5 m6

ResNet-152 0.234 0.192 0.239 0.189 0.235 0.347

LSTM 0.312 0.162 0.271 0.183 0.289 0.267

Bi-LSTM 0.117 0.046 0.145 0.017 0.138 0.125

proposed 0.091 0.003 0.102 0.007 0.009 0.018

MAPE

Methods m 1 m2 m3 m4 m5 m6

ResNet-152 12.45 17.4 11.9 34.23 18.89 23.8

LSTM 10.83 15.56 8.39 38.21 20.8 25.1

Bi-LSTM 14.59 14.45 9.78 41.33 12.67 14.23

proposed 9.99 7.34 5.49 12.23 8.23 11.2

MAE

Methods m 1 m2 m3 m4 m5 m6

ResNet-152 0.271 0.261 0.271 0.029 0.235 0.031

LSTM 0.233 0.245 0.252 0.014 0.184 0.029

Bi-LSTM 0.134 0.189 0.061 0.009 0.093 0.015

proposed 0.083 0.156 0.001 0.004 0.024 0.002

R2

Methods m 1 m2 m3 m4 m5 m6

ResNet-152 0.907 0.887 0.972 0.912 0.813 0.896

LSTM 0.891 0.873 0.915 0.892 0.892 0.894

Bi-LSTM 0.882 0.829 0.925 0.903 0.919 0.925

proposed 0.991 0.963 0.994 0.935 0.941 0.976

Table 4 shows the comparison of error measures of various approaches on the Arabidopsis

thaliana dataset. The error measures like MSE, MAPE, MAE and R2 are compared. From the

above comparison, it is observed thatthe values of error metrics are found to low and R2

values are more for the proposed R-At-SCM. In the case of MSE, the proposed R-At-SCM

achieved very less value of 0.03 for the frame m  2 , whereas the other models like ResNet-

152, LSTM and Bi-LSTM achieved higher MSE values of 0.192, 0.162 and 0.046. This

proposed R-At-SCM attained better results because of the attention spatial channel module.

The other DL approaches achieved poor results because of complex architecture and takes
more time for completing the process. Thus theproposed R-At-SCM proved its supremacy

when compared to other approaches.

5. Conclusion

This work introduces a phenotyping and plant growth prediction model. This work was

carried out for generating future frames of roots and leaves of the plants. The proposed DL

model R-At-SCM has the capacity of predicting different growth frames into the future. This

model combined spatial and temporal features of the root system for providing efficient root

growth prediction. The introduced root masks were verified by RootNav 2.0 and the extracted

model was stored in XML format. The denoising and reduction were done by DTCWT. The

experimental results were carried out for image quality and error measures on the two

benchmark datasets like Brassica rapa and Arabidopsis thaliana datasets. The proposed model

achieved a better mIoU value of 0.991 for m  3 frame. This model reduced the complexity of

predicting issues and predicted the future growth of the plant.

Compliance with Ethical Standards

Disclosure of potential conflicts of interest

Authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants performed by any of the

authors.

Data Availability Statement

Data sharing does not apply to this article as no new data has been created or analyzed in this

study.

Funding Information

This research did not receive any specific grant from funding agencies in the public,

commercial, or not-for-profit sectors

Declarations

Conflicts of interest/Competing interests: Not Applicable

Code availability: Not Applicable

Consent to participate: Not Applicable

Consent for publication: Not Applicable

Acknowledgement: None

References

Adams, J. R. (2020). Plant segmentation by supervised machine learning methods and

phenotypic trait extraction of soybean plants using deep convolutional neural networks

with transfer learning (Doctoral dissertation, The University of Nebraska-Lincoln).

Adams, J., Qiu, Y., Xu, Y., &Schnable, J. C. (2020).Plant segmentation by supervised

machine learning methods. The Plant Phenome Journal, 3(1), e20001..

Alhnaity, B., &Abbod, M. (2020).A new hybrid financial time series prediction

model. Engineering Applications of Artificial Intelligence, 95, 103873.

Alhnaity, B., Pearson, S., Leontidis, G., &Kollias, S. (2019, June). Using deep learning to

predict plant growth and yield in greenhouse environments. In International Symposium

on Advanced Technologies and Management for Innovative Greenhouses: GreenSys2019

1296 (pp. 425-432).

Benos, L., Tagarakis, A. C., Dolias, G., Berruto, R., Kateris, D., &Bochtis, D. (2021).

Machine learning in agriculture: A comprehensive updated review. Sensors, 21(11), 3758.

Chang, S., Lee, U., Hong, M. J., Jo, Y. D., & Kim, J. B. (2021).Time-Series Growth

Prediction Model Based on U-Net and Machine Learning in Arabidopsis. Frontiers in

plant science, 12.2.

Darwish, A., Ezzat, D., &Hassanien, A. E. (2020). An optimized model based on

convolutional neural networks and orthogonal learning particle swarm optimization

algorithm for plant diseases diagnosis. Swarm and evolutionary computation, 52, 100616.

Francis, M., &Deisy, C. (2019, March).Disease detection and classification in agricultural

plants using convolutional neural networks—a visual understanding. In 2019 6th

International Conference on Signal Processing and Integrated Networks (SPIN) (pp.

1063-1068). IEEE.

Kim, T., Lee, S. H., & Kim, J. O. (2022). A Novel Shape Based Plant Growth Prediction

Algorithm Using Deep Learning and Spatial Transformation. IEEE Access, 10, 37731-

37742.

Li, L., Zhang, Q., & Huang, D. (2014).A review of imaging techniques for plant

phenotyping. Sensors, 14(11), 20078-20111.

Li, Y., Wu, X., Chen, T., Wang, W., Liu, G., Zhang, W., ...& Zhang, G. (2018). Plant

phenotypic traits eventually shape its microbiota: a common garden test. Frontiers in

microbiology, 9, 2479.

Li, Y., Wu, X., Chen, T., Wang, W., Liu, G., Zhang, W., ...& Zhang, G. (2018). Plant

phenotypic traits eventually shape its microbiota: a common garden test. Frontiers in

microbiology, 9, 2479.

Liakos, K. G., Busato, P., Moshou, D., Pearson, S., &Bochtis, D. (2018). Machine learning in

agriculture: A review. Sensors, 18(8), 2674.

Lynch, J., Marschner, P., &Rengel, Z. (2012). Effect of internal and external factors on root

growth and development.In Marschner's mineral nutrition of higher plants (pp. 331-

346).Academic Press.
Minervini, M., Scharr, H., &Tsaftaris, S. A. (2015). Image analysis: the new bottleneck in

plant phenotyping [applications corner]. IEEE signal processing magazine, 32(4), 126-

131.

Nesteruk, S., Shadrin, D., Kovalenko, V., Rodríguez-Sánchez, A., &Somov, A. (2020,

June).Plant growth prediction through intelligent embedded sensing.In 2020 IEEE 29th

International Symposium on Industrial Electronics (ISIE) (pp. 411-416). IEEE..

Pallathadka, H., Mustafa, M., Sanchez, D. T., Sajja, G. S., Gour, S., &Naved, M.

(2021).Impact of machine learning on management, healthcare and agriculture. Materials

Today: Proceedings.

Rizkiana, A., Nugroho, A. P., Salma, N. M., Afif, S., Masithoh, R. E., Sutiarso, L., &

Okayasu, T. (2021, April). Plant growth prediction model for lettuce (Lactucasativa.) in

plant factories using artificial neural network. In IOP Conference Series: Earth and

Environmental Science (Vol. 733, No. 1, p. 012027). IOP Publishing.

Rochefort, A., Briand, M., Marais, C., Wagner, M. H., Laperche, A., Vallée, P.,

...&Sarniguet, A. (2019). Influence of environment and host plant genotype on the

structure and diversity of the Brassica napus seed microbiota. Phytobiomes Journal, 3(4),

326-336.

Sakurai, S., Uchiyama, H., Shimada, A., & Taniguchi, R. I. (2019, February).Plant Growth

Prediction using Convolutional LSTM. In VISIGRAPP (5: VISAPP) (pp. 105-113).

Sakurai, S., Uchiyama, H., Shimada, A., Arita, D., & Taniguchi, R. I. (2018).Two-step

Transfer Learning for Semantic Plant Segmentation.In ICPRAM (pp. 332-339).

Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural

networks, 61, 85-117.

Singh, A., Ganapathysubramanian, B., Singh, A. K., &Sarkar, S. (2016). Machine learning

for high-throughput stress phenotyping in plants. Trends in plant science, 21(2), 110-124.
Son, N. T., Chen, C. F., Chen, C. R., Guo, H. Y., Cheng, Y. S., Chen, S. L., ... & Chen, S. H.

(2020). Machine learning approaches for rice crop yield predictions using time-series

satellite data in Taiwan. International Journal of Remote Sensing, 41(20), 7868-7888..

Sorjamaa, A., Hao, J., Reyhani, N., Ji, Y., &Lendasse, A. (2007).Methodology for long-term

prediction of time series. Neurocomputing, 70(16-18), 2861-2869.

Taieb, S. B., Bontempi, G., Atiya, A. F., &Sorjamaa, A. (2012). A review and comparison of

strategies for multi-step ahead time series forecasting based on the NN5 forecasting

competition. Expert systems with applications, 39(8), 7067-7083.

Tsaftaris, S. A., &Noutsos, C. (2009).Plant phenotyping with low cost digital cameras and

image analytics.In Information Technologies in Environmental Engineering (pp. 238-

251).Springer, Berlin, Heidelberg.

Uchiyama, H., Sakurai, S., Mishima, M., Arita, D., Okayasu, T., Shimada, A., & Taniguchi,

R. I. (2017). An easy-to-setup 3D phenotyping platform for KOMATSUNA

dataset.In Proceedings of the IEEE International Conference on Computer Vision

Workshops (pp. 2038-2045).

Wilson, M. H., Holman, T. J., Sørensen, I., Cancho-Sanchez, E., Wells, D. M., Swarup, R.,

...&Hodgman, T. C. (2015). Multi-omics analysis identifies genes mediating the

extension of cell walls in the Arabidopsis thaliana root elongation zone. Frontiers in cell

and developme

Yasrab, R., Zhang, J., Smyth, P., & Pound, M. P. (2021).Predicting plant growth from time-

series data using deep learning. Remote Sensing, 13(3), 331..

ORCID Information

ORCID Information
Dr. S. Narayanan:

https://orcid.org/0000-0002-8882-7322
Highlights

Highlights

 Phenotyping undergoes the quantitative analysis of bio-chemical

 natural way of plant growth is time consuming process

 the entire annotations are automatic and generate images with XML

 DT-CWT provided to the original data for reducing the overfitting & reducing noise

 This model is well suited and trainable for various types of plants and mutations
Declaration of Interest Statement

Declaration of Interests Statement

21-Jul-2023
Dear Editors in chief,

☑ The authors declare that they have no known competing financial interests or personal relationships
that could have appeared to influence the work reported in this paper

Thank You
Sincerely,
Dr. Subbiah Narayanan
Dr. Dhamodaran Sridevi