b i o s y s t e m s e n g i n e e r i n g 2 1 9 ( 2 0 2 2 ) 1 6 5 e1 7 6

Available online at www.sciencedirect.com

ScienceDirect

journal homepage: www.elsevier.com/locate/issn/15375110

Research Paper

Improving rice nitrogen stress diagnosis by

denoising strips in hyperspectral images via deep
learning

Yueming Zhu a,b, Alwaseela Abdalla a,b, Zheng Tang a,b, Haiyan Cen a,b,*
a
College of Biosystems Engineering and Food Science, and State Key Laboratory of Modern Optical Instrumentation,
Zhejiang University, Hangzhou, 310058, China
b
Key Laboratory of Spectroscopy Sensing, Ministry of Agriculture and Rural Affairs, Hangzhou, 310058, China

article info
It is critical to assess crop nitrogen status for precision fertilization and crop management.
Article history: Hyperspectral imaging (HSI) has been proved as one of the valuable sensing technologies
Received 3 October 2021 for plant nitrogen stress detection, while exploring efficient methods for hyperspectral
Received in revised form image analysis remains challenging due to the high dimensionality, information redun-
1 April 2022 dancy, and noise interference. This study aims to develop a de-striping convolution neural
Accepted 3 May 2022 network (DS-CNN) to remove strip noise in hyperspectral images and design a nitrogen
Published online 20 May 2022 diagnosis CNN (ND-CNN) for rice leaves. The DS-CNN included an image encoder, a strip
dropout bottleneck, and a decoder to remove the strip noise in hyperspectral images.
Keywords: Activation function Leaky-ReLU was introduced to activate the potential neurons to keep
Deep learning the real text features of the hyperspectral image. Element-wise feature addition mecha-
Hyperspectral imaging nism was also applied to guarantee the full range of band images that reached the best
Strip denoising structural and textural similarities. We constructed six datasets with different noise scales
Rice crop to explore the potential of DS-CNN. The best performance of DS-CNN was on the lowest
Nitrogen strip noise dataset (s ¼ 0.02), with the mean squared error (MSE) lower than 2  104,
highest structure similarity index metric (SSIM) of 0.99, and peak signal-to-noise ratio
(PSNR) of around 36 dB on the validation dataset. Moreover, we compared the nitrogen
diagnosis performance before and after denoising. The ND-CNN developed from the
denoised dataset not only avoided the overfitting but also improved the accuracy of ni-
trogen stress diagnosis of rice leaves.
© 2022 IAgrE. Published by Elsevier Ltd. All rights reserved.

nitrogen status of the crops can be estimated using either

1. Introduction laboratory-based chemical analysis, visual observation by
experts, or soil plant analysis development (SPAD) chlorophyll
The measurement of nitrogen (N) stress is a prerequisite for meter. Although chemical analysis is the most accurate and
the quantitative assessment of crops. Traditionally, the precise method, it has the limitations of being invasive,

* Corresponding author. College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou, China.
E-mail address: hycen@zju.edu.cn (H. Cen).
https://doi.org/10.1016/j.biosystemseng.2022.05.001
1537-5110/© 2022 IAgrE. Published by Elsevier Ltd. All rights reserved.
166 b i o s y s t e m s e n g i n e e r i n g 2 1 9 ( 2 0 2 2 ) 1 6 5 e1 7 6
destructive, and preventing instantaneous nitrogen mea-
surements. The SPAD is a simple and portable diagnostic tool
that measures the greenness or relative content of leaves
using an empirical formula. This method, however, has
limited success in the large-scale area because it is time-
consuming (Abdalla, Cen, Wan, Mehmood, & He, 2021). As
an alternative technique, hyperspectral imaging (HSI) tech-
niques combined with machine learning (and its subclass
deep learning) have been considered one of the most impor-
tant remote and proximal techniques to instantaneously
monitor plant nutrient status none destructively and none-
invasively. Compared to other imaging techniques, the
advantage of HSI can extract both physiological and structural
information associated with nitrogen status (Jung, Song,
Hong, & Shim, 2021; Sabzi, Pourdarbani, Rohban, Garcı́a-
Mateos, & Arribas, 2021).
Nitrogen stress can be sensed by spectral reflectance in
both visible and near-infrared (NIR) regions (400e1000 nm), A
significant correlation between plant nitrogen status and
spectral reflectance has been reported in recent literature. For
example, Wang et al. (2021) employed unmanned aerial
vehicle-based hyperspectral imagery to estimate the nitrogen
of the paddy rice at leaves and plant levels taking into account
the growth stage, high correction between vegetation indices
that are derived from the hyperspectral images, and nitrogen
status have been found. Similar works have been reported by
Li, Li, Elsayed, Hu, and Schmidhalter (2021), their results
showed that the reflectance in the 550 nm wavelength region
has a significant influence on differentiating between various
N treatments. While other studies indicate that the total N in
plants can be determined at 671 nm and 780 nm wavelengths
(Tros et al., 2021).
The application of HSI, however, is hampered by different
sources of noise, including but limited to, atmospheric effects
and internal sensor noises. The atmospheric effects can be
eliminated by using close-range HSI under controlled illumi-
nation conditions, but the internal hyperspectral sensor
noises, such as thermal-driven noise (i.e., strip noise), quan-
tization noise, and shot (i.e., photon) noise are the main
challenges associated with the use of HSI for extraction
meaningful information from the plants. The strip noise
resulting from an increasing temperature of the sensor (i.e.,
due to poor heat dissipation) can cause severe degradation in
image quality, but also significantly influence the perfor-
mance of HSI-based nutrient status diagnosis of the crops (Liu,
Cai, Li, Ning, & Jiang, 2022; Sellami & Tabbone, 2022) and other
level tasks based on hyperspectral imaging. Therefore, strip
noise removal has become an essential and inevitable pre-
processing step before the subsequent image analysis and
applications of the HSI. Eliminating the strip noise may
significantly improve the performance and robustness of HSI
applications (Dao, Mantripragada, He, & Qureshi, 2021).
Generally, strip noise can be divided into two classes; peri-
odical and non-periodical noises. A periodical stripe noise is
more prevalent and difficult to handle. Recently, several de-
stripping algorithms have been developed for remote
sensing applications and can be grouped into several cate-
gories; such as filtering-based methods, statistical-based
methods, and learning-based methods. Filtering-based
methods, such as Fourier transform (Chen, Shao, Guo,
Wang, & Zhu, 2003; Pan & Chang, 1992)and wavelet analysis
(Torres & Infante, 2001), can only perform well on the periodic
stripe noise and may affect the structural details of the image
that have similar frequencies as the stripe resulting in blur-
ring and or ringing artifacts of the output images. The
statistical-based methods, such as histogram equalization
and moment matching, rely on the statistical properties of the
digital sensor and assume that the mean and standard devi-
ation of the digital sensors are consistent and the image scene
is homogenous, when such assumption is violated the image
details can be damaged during destriping. Hence, there is a
need to have a common and robust de-striping framework
that will directly estimate stripe-free images from the stripe
images without considering the intrinsic properties of the
stripe noise. Recent advances in deep learning networks,
particularly convolutional neural networks (CNN), have
attracted the attention of researchers to apply this approach
to solve such an ill-posed problem. The first deep learning
method (named SNRCNN) is proposed by Kuang, Sui, Chen,
and Gu (2017) to remove the stripe noise from infrared im-
ages. It is a simple and shallow CNN architecture, which is
evaluated on simulated non-uniform stripes and tends to
over-smooth the output image. Deshpande, Patale, and Roy
(2021) proposed a global residual-based deep neural network
to remove the stripe noise from remote sensing imagery, the
image quality is further improved using multilevel wavelet
decomposition and frequency domain filtering as a post-
processing step. A wavelet-based deep neural network pro-
posed by Guan, Lai, and Xiong (2019) separates the stripe noise
from the scene. This method preserves the image details
without using post-processing but the details along the stripe
direction are over-smoothed. This over-smoothing cause
degradation of image quality in remotely sensed data and
reduce its usefulness for any application in the automated
analysis process. Therefore, for remote sensing appl, there is a
need to have a common and robust method to remove the
stripe noise from hyperspectral images while maintaining the
structural and textural details without using post-processing
to the output image. With this motivation, a new deep
learning network for hyperspectral image de-striping with
improved performance has been proposed in this study,
namely a de-striping convolution neural network (DS-CNN).
The main contributions of this article can be summarized
as follows; (1) we proposed a de-striping convolution neural
network (DS-CNN) for removing strip noise in the HSI while
preserving the structural details of the output image. Our DS-
CNN architecture consists of an encoder, a strip dropout
bottleneck, and a decoder. (2) a new activation function, so-
called Leaky-ReLU, was proposed to activate the potential
neurons that help in maintaining good image quality and
preventing the over-smoothing problem. Additionally, an
element-wise feature addition mechanism was applied to
ensure that the bands have the same structural and textural
details. (3) extensive experiments on simulated and real data
demonstrate that our method outperforms many existing
state-of-the-art de-stripping methods in terms of quantitative
assessments. (4) finally, as proof of concept and to represent
the practical challenge of the proposed method in real plant
phenotyping applications, the nitrogen status of the plants
was diagnosed using real de-stripped HSI.
 b i o s y s t e m s e n g i n e e r i n g 2 1 9 ( 2 0 2 2 ) 1 6 5 e1 7 6
2. Materials and methods
2.1. Experimental design
The experiment was implemented at the rice-production
Functional Area (29 310 5.3500 N, 120 60 6.1200 E) which is located
at Anhua Town, Zhuji City, Zhejiang Province, China with an
annual temperature of 16.3  C and 16 m altitude above the sea.
The rice (Yongyou 1540) in the experimental site was divided
into 25 subplots, each of them with 18  10 m2 , all of the rice
plants were transplanted in the June of 2019 at the pre-
planning stage. Five different stress of N fertilizers (N0, N1,
N2, N3, and N4 ¼ 0: 0.5: 1: 1.5: 2) were applied in the form of
urea, with five repetitions at different growth stages including
40% at pre-planting stage, 30% at the tillering stage and 30% at
the booting stage.
2.2. Hyperspectral image acquisition
The hyperspectral imaging system used in this study was
developed by AgroOptics and Imaging Lab at Zhejiang Uni-
versity, it covers the visible and near-infrared spectral range
of 400e1000 nm. The system consists of a CMOS camera (Zyla-
4.2 P, Andor Technology, UK.), an imaging spectrograph
(ImSpector V10E, Spectral Imaging Ltd., Oulu, Finland) with a
spectral resolution of 2.8 nm, a high-quality lens (Schneider
Kreuznach Xenoplan 1.4/17e0903), two tungsten halogen
lamps (Fibre-Lite DC950 Illuminator, 150 W, Dolan Jenner In-
dustries Inc. Boxborough, MA, USA.) and a stepper motor
driven conveyer belt.
Hyperspectral images of rice leaves were collected during
the initial heading stage and initial filling stage on 28 August
and 21 September 2019, respectively. The nitrogen supplica-
tion at these two growth stages is very crucial to the final yield
production in terms of both quality and quantity (Cen et al.,
2019).
Before the imaging acquisition, the surface of the leaves
was cleaned to avoid the interference of dust and water, the
resolution parameter was set to 1024  237 (spatial  spectral)
mode. The white and dark reference images were used for
hyperspectral image correction to remove the environmental
effects by equation (1):
IO  B
IC ¼  100%
WB
where IC is the corrected hyperspectral image, IO is the original
raw image, B is the image collected under dark conditions
(with 0% reflectance), and W is the reflectance image obtained
by the standard whiteboard (Isuzu Optics Corp., Taiwan,
China) with a white surface (99% reflectance) under the
halogen lamp. After calibration, the image segmentation was
performed using the threshold method to mask out the region
of interest (ROI).
2.3. DS-CNN architecture
The noise removal can be expressed as an inverse task
formulated as an energy minimization problem (Chen,
Papandreou, Kokkinos, Murphy, & Yuille, 2018) as:
167
N* ¼ minn Eðo; sÞ þ RðnÞ (2)
where E(o,s) is a task-related metric, s is the strip noise cor-
rupted image, o is the original clean image, R(n) is a regulari-
zation function learned from data (Sidorov & Hardeberg, 2019).
While, Lempitsky, Vedaldi, and Ulyanov (2018) consider that
this regularization can be extracted with the CNN network
parameters optimization process, which can be applied to the
noise removal problem depicted by the equation below.


n* ¼ fq* ðsÞ; where q* ¼ argminn E fq ðsÞ; n0 (3)
in which f is a CNN architecture, s is a specific input of noise,
meanwhile, an expected clean image can be restored with the
optimization of the neural network f, the weights, and other
parameters can be learned from the training and optimization
process. Generally, we can optimize f's performance by mean
squared error (MSE) for loss function L which can be depicted
as:
1 Xn
LðtÞ ¼ kMðSi ; tÞ  ðSi  Oi Þk2f (4)
2n i¼1
where Si and Oi are the striped and clean image pairs in the
dataset, t are the parameters to be trained, f is the Frobenius
norm function. The whole architecture of DS-CNN is shown in
Fig. 1 which will be explained in the following.
The proposed DS-CNN employs 3 functional modules
namely encoder, dropout block, and decoder. The input of DS-
CNN is optimized as 128  128  1, note that we have a single
channel feature map, each encoder layer in the encoder block
owns a different set of filters with optimized contractible sizes
and numbers, as depicted in order for this up-sampling
computation is applied to save essential features of the
input image, these abundant features from the encoder are
sent to the decoder by layer addition for twice. The multiple
filters with different sizes and numbers are also applied at the
beginning of the dropout block after the dense layers, three
deconvolution layers are applied to drop out more noise fea-
tures. In the decoding block, the batch-normalization layer is
applied twice to activate the expected parameters. The final
denoised image is reconstructed as the same size as the input,
the more detailed explanation of DS-CNN will be explained in
the following.
The encoder itself can be applied for image feature
extraction as shown in Fig. 1, we tuned different kernel sizes
(1)
for down-sampling with stride 2 rather than pooling as pre-
vious research reported (Mao, Shen, & Yang, 2016) for the four
convolutional layers in the encoder block. During the training,
the MSE of the model on different filter sizes are recorded and
compared, the PSNR and SSIM are also evaluated as image
quality monitors. The training time and resource cost increase
rapidly as the filter size increases the computation
complexity. Filter size 3  3 showed an outperforming de-strip
result than others, which is an outperformed filter size for DS-
CNN after tuning.
Dropout is arguably the most prominent regularization
technique practically (Cai et al., 2019) to pre overfitting, we
construct the model dropout as a bottleneck structure, two
convolutional layers (4  4 followed by 2  2 convolution)
with Leaky-ReLU activation function are arranged after both
168 b i o s y s t e m s e n g i n e e r i n g 2 1 9 ( 2 0 2 2 ) 1 6 5 e1 7 6
Fig. 1 e The architecture design of the De-Striping Convolution Neural Network (DS-CNN).
of the convolutional layers, one flatten layer and two dense
layers are followed as final feature representation, which
was transferred to the final decoder block by the adversely
convolutional computation, this design was proved to be
stable and effective after practical tuning to reach the
advanced strip removal performance evaluated by MSE,
PSNR and sent SSIM.
The decoder block takes features from the encoder and
dropout blocks to reconstruct the clean image, we used the
conv2D transpose layer as a basic unit according to the
encoder dimensions and arrangements, essential features
from the encoder and dropout block are combined twice ac-
cording to the feature map dimension.
Skip connection was used twice in the model to transfer
the textural feature across the dropout block, it ensures the
maximum flow of features between the encoder and decoder
block. Usually, gradient information can be lost when passing
through many layers, the motivation behind layer addition is
that it has an uninterrupted gradient flow from the forward
layer to the after, which can also contribute to tackling the
vanishing gradient problem. It enables an alternative way to
ensure feature reusability in the same dimension as the
former layers. Flatten and dense layers are also applied to
prevent overfitting after the two skip connections to enforce
the DS-CNN reached its gradient edge as early as possible.
Activation functions should be selected according to the
practical performance, previous studies have reported the
ReLU activation function as follow:
 but also have degradation problems in the training accuracy.
0 jx < 0
f ðxÞ ¼
x jx  0
x is the feature value in the arbitrary index from the con-
volutional layers, it is a widely used activation function
because it costs less but works faster than other activation
functions such as tanh, sigmoid, max out, etc. (LeCun, Bengio, &
Hinton, 2015). But ReLU sets the x to zero when x<0, thus
making the neuron dead which improves the sparseness of
the network. However, ReLU has been proved limitations for
image denoising, because it fails to identify the noise from the
image text feature, especially under heavy and complicated
noise conditions, it cannot retain the positive image features
as well as remove the noise features effectively (Shang, Sohn,
Almeida, & Lee, 2016).
At the end of each convolution layer followed by the leaky-
ReLU (leaky version of Rectified Linear Unit) function, a non-
saturating activation function that overcomes the “dying
ReLU” problem given by

ax jx < 0
f ðxÞ ¼ (6)
x jx  0
where a is a number that can be very small (we used 0.01) to
avoid negative values than ReLU, our tuning proved leaky-ReLU
to be more effective and stable for strip denoising.
Our model tries to reduce the model complexity by using
very small receptive fields (3  3) in every convolutional layer
resulting in a model size of only 23 M. Small size receptive
fields allow DS-CNN to have a large number of convolutional
layers. Training such a deep network may have a lot of com-
plications not only overfitting and high computational cost
(5) To resolve this issue, we added a skip connection to our
network, which means that some of the layers in the network
 b i o s y s t e m s e n g i n e e r i n g 2 1 9 ( 2 0 2 2 ) 1 6 5 e1 7 6
are skipped and feed the output of one layer as the input to the
next layers. Such a technique was introduced to solve the
degradation problem in different deep learning architectures
such as ResNet and U-Nets. Another method used to boost the
performance of the DS-CNN and improve the convergence is
that each convolutional layer was followed by a Leaky ReLU
(LReLU) activation function which prevents neurons that have
values less than zero to die during the training progress and
no training will happen to these neurons.
2.4. ND-CNN architecture
The nitrogen diagnosis model namely ND-CNN which is
designed to diagnose the nitrogen status of rice plants is
constructed by two convolutional up-sampling blocks as
feature generators, each of that contains two convolutional
layers and one max-pooling layer, a fully connected layer, and
two dense layers at the end as dropout function to handle with
the final prediction (see Fig. 2).
The convolutional kernel size is set to 3  3 which has been
proved an excellent performance for image classification, the
max-pooling layer can reduce the dimensionality of feature
maps with a pool size of 2  2, dense layers are applied to
dropout the neurons before the final classification. ReLU is
applied as an activation function between each of the
convolution layers. Softmax is applied as activation for final
classification, the learning rate is set to 1  103.
3. Model training
3.1. Simulated dataset construction
The HSI image acquisition is time-consuming and requires a
large amount of storage capacity making it difficult to obtain a
large number of images at one time, our total number of
training data contains 98 clean HSI images with a spatial and
spectral resolution of (1024  1024) and 237, respectively.
Fig. 2 e Framework of nitrogen diagnosis CNN (ND-CNN).
169
Therefore, the data augmentation was adopted to artificially
increase the number of the training dataset, 10 times image
augmentation was applied with a spatial dimension of
128  128 covering all of the bands by random cropping. We
simulated strip noised data by adding different scales of strip
noise on the clean hyperspectral images, for each hyper-
spectral image, we add random strip noise level s to each band
image x, so the simulated striped image y at waveband w
Yw¼Xwþs where Xw is the noise-free image and s is the
simulated strip noise. Totally six noisy datasets were gener-
ated according to the noise level s ¼ 0.02, 0.05, 0.10, 0.15, 0.20,
and 0.25, with the clean image as a reference to calculate PSNR
and SSIM, DS-CNN was trained on all the datasets to exploring
the ultimate denoising capability.
3.2. Implementation details
ROI mask process for HSI preprocessing was built with the
Environment for Visualizing Images (ENVI) software (ENVI 4.6)
(Research Systems Inc., Boulder Co., USA) and batch pro-
cessing with MATLAB R2018b (The MathWorks, Natick, MA,
USA). The DS-CNN was implemented by a deep learning
framework namely Keras, it offers a GPU version to speed up
the large-scale image computation. All experiments were
conducted on the Windows server with an Intel(R) Core(TM)
CPU mode i7-7800X @ 3.50 GHz CPU, the memory of 32 GB, a
GPU namely NVIDIA GeForce GTX 1080 Ti with 11 GB memory.
Considering the GPU resource and memory limitation, the
training batch size was set to 128 during the training process.
4. Results and discussion
To evaluate the validity and generality of DS-CNN, one loss
indicator and two image quality indicators were introduced to
monitor the model's performance. We used mean squared
error (MSE) (X. Liu, Tanaka, & Okutomi, 2013) as back-
propagation, which can be defined as follows:
170 b i o s y s t e m s e n g i n e e r i n g 2 1 9 ( 2 0 2 2 ) 1 6 5 e1 7 6
For a given reference image X and processed image Y, both
of the same size m  n, the MSE between X and Y is defined by:
m1 X
1 X n1
2 ples on trained DS-CNN, it is clear that the DS-CNN reached
MSE ¼ ðXði; jÞ  Yði; jÞÞ
mn i¼0 j¼0
For DS-CNN, the minimization of the MSE was imple-
mented by the Adam optimizer. Besides, peak signal-to-noise
ratio (PSNR) and structure similarity index metric (SSIM) (Hore
& Ziou, 2010) were introduced to evaluate the denoising
quality. PSNR is an objective index, higher PSNR usually in-
dicates an increased image quality after denoising, which is
defined as the following:
 2
 HSI. Here, we have selected the most four relevant methods to
255
PSNRðX; YÞ ¼ 10 log10
MSEðX; YÞ
The SSIM is a quality metric used for image processing
expressed by loss of correlation, image luminance distortion,
and image contrast distortion, it is independent of both image
brightness and contrast, which can be used to evaluate the
initial feature and structural difference between two images.
The SSIM is defined as follows:
SSIMðX; YÞ ¼ lðX; YÞcðX; YÞsðX; YÞ
where
2uX uY þ C1
lðX; YÞ ¼
u2X þ u2Y þ C1
2sX sY þ C2
cðX; YÞ ¼
s2X þ s2Y þ C2
sXY þ C3
sðX; YÞ ¼
sX sY þ C3
Luminance l (X, Y) comparison function indicates the dif-
ference of the two images’ mean luminance (mX and mY Þ, the
maximal of this index is 1 and equal to 1 only if mX ¼ mY . cðX; YÞ
is the indicator of contrast of the image after and before image
processing, this contrast is presented by the standard devia-
tion sX and sY , this indicator is maximal to 1 and equal to 1
only if sX ¼ sY . The expression sðX; YÞ reveals the correlation
coefficient after image denoising. The covariance between X
and Y is sXY , the reasonable values of SSIM are in [0,1]. C1 ,
C2 and C3 are positive constants used to avoid a null
denominator.
4.1. Training results on denoising
The MSE is applied as loss indicator for DS-CNN, Adam is used
as optimization function, it adapts the learning rate of every
parameter, which employs an exponentially decaying average
of previous gradients in the network, the learning rate was set
to 1  103 to initialize the training, the model reached its
balanced peak after 1250 trainging epochs within 26 h. As
showed in Fig. 3 (a), the average loss decreased to 1  104 after
500 epochs which manifests that DS-CNN can learn the dif-
ference between the noised image and clean image, in order to
produce an high quality estimated image, DS-CNN should
drop the noise without change the text feature at the same
time, the PSNR and SSIM curve indicate the model can remove
the strips effectively and produce an image with high quality,
see Fig. 3 (b) and (c).
Table 1 reports the quantitative indicators of testing sam-
(7)
outstanding performance on the dataset with noise level
s ¼ 0.02, it is also clear that image contaminated with slight
noise can be removed with higher image quality indicated by
the quantitative index from Table 1.
4.2. Hyperspectral strip removal performance
comparison
There are several reported methods for stripe noise removal in
(8) demonstrate the contributions of our work. One is filtering-
based method and others are learning-based methods as fol-
lows. (1) Wavelet-Domain Low-Rank/Group-Sparse (WDLRGS)
destriping method proposed by N. Liu, Li, Tao, Fowler, and
Yang (2019). This method leveraged a group-sparse model of
wavelet coefficients of a hyperspectral image with a low-rank
stripe model to destripe subbands of a 2-diminational discrete
wavelet transform (2D-DWT). The destriping process, as
formulated by an Alternating Direction Method of Multipliers -
(9)
based low-rank/group-sparse optimization, was deployed
exclusively in the 2D-DWT subbands wherein the stripes from
a pushbroom sensor are concentrated. (2) Wavelet Deep
(10) Neural Network (SNRWDNN) proposed by Shan, Liu, Mu, Cao,
and He (2019) for Stripe Noise Removal. This approach treats
the stripe noise removal as a transform coefficients prediction
(11) problem in wavelet domain and utilize the directional char-
acteristics of the stripe to suppress the noise with less detail
loss. (3) Dual deep convolutional neural network (DDCNN) for
(12) hyperspectral image denoising proposed by Shan et al. (2019).
This approach utilizes a new activation function exchange
information between the two CNNs. (4) Residual DNN with
post filtering (RDCNN) proposed by Deshpande et al. (2021), it
can adopt residual learning strategy in the context of global
residual learning method to destrip and denoise images from
the estimated noise residue. It works using a single residual
unit to predict the residual noise from the known noise pat-
terns and reiterates model training to minimize the loss
function. These methods are designed either an excellent strip
noise filter or a functional optimizer to solving the inverse
problem.
As shown in Table 2, the proposed method shows
improved performance in terms of both Peak Signal-to-Noise
Ratio (PSNR) and Structural Similarity Index Metric(SSIM)
quality measures on all noise scales when compared with the
state-of-the-art methods.
4.3. Hyperspectral strip removal visualization
For the simulated noise scale datasets, a DS-CNN denoising
result is shown in Fig. 4, the original case in Fig. 4 is the clean
image, different noise scale images are shown under original
case in Fig. 4(a), the denoised image with corresponding model
are shown in (b) under original case. The outperform of lower
strip noise scale denoising performance in the Fig. 4 also
conclued that DS-CNN produced an outstanding performance
on the dataset with lower strip noise.
 b i o s y s t e m s e n g i n e e r i n g 2 1 9 ( 2 0 2 2 ) 1 6 5 e1 7 6 171

Fig. 3 e De-Striping Convolution Neural Network training curves. (a) Average loss (b) Average peak signal to noise ratio
(PSNR) and (c) Average structural similarity index metric (SSIM) on the dataset with strip noise scace s ¼ 0.02.

strip removal and Fig. 5(b) illstrates a promising image details

Table 1 e The validated peak signal to noise ratio (PSNR),
Average structural similarity (SSIM), and mean squared
error (MSE) on the datasets with coordinate noise scale.
s ¼ 0.02 s ¼ 0.05 s ¼ 0.1 s ¼ 0.15 s ¼ 0.2 s ¼ 0.25
PSNR 33.82 32.53 30.09 28.77 27.61 27.90
SSIM 0.96 0.93 0.88 0.85 0.81 0.81
MSE 41.98 70.77 118.90 146.78 177.28 181.79
An important benefit of the proposed method is the
robustness in the accuracy of generated noise-free images
from the images subjected to different levels ranging from
0.02 to 0.25. Fig. 4 shows the generated noise-free image from
the noisy image (Fig. 4a) compared to the noise-free image of
the same subject (Fig. 4b). Visually, it can be observed that the
denoised images generated by the proposed model have very
good quality in all selected bands; all the strip noises were
effectively removed from the input image. It is worth
mentioning that when the noise scale is large, the quality is
slightly degraded while maintaining the texture and struc-
tural information of the image and there is no over-smoothing
or distortion was observed in the denoised image indicating
that the generated noise-free images can be used to describe
any biological phenomena such as nutrient health status of
the plants, as further explained in section 5. These visualiza-
tion results agree well with quantitative evaluation as pre-
sented in Table 2.
Figure 5 shows the denoising results simulated by the
hyperspectral data set image with false color (a) and RGB
channels (b) from simulated dataset. The size of the selected
image is 128  128, the original image in Fig. 5(a) is the 66th
band. The RGB image in Fig. 5(b) consists 21st, 62nd and 104th
band image, the Fig. 5(a) manifests a clear text features after
after denoising.
As the strip noise are varying among different spectral
regions, we have tested systems on another band in the visible
region to further confirm its robustness. Fig. 5 visualized the
noise removal using different spectrum bands including the
21st, 62nd, and 104th bands (482.49 nm, 558.91 nm, and
671.94 nm, respectively) which represent RGB image as shown
on the left side of Fig. 5 and 66th band (482.49 nm) indicating
that the noise was removed from all bands and the textural
feature was fully recovered after noise removal without
causing over-smoothing or damaging the boundaries of the
image.
4.4. Application on real hyperspectral images
To represent the practical challenge of our proposed deep
learning model, it was also evaluated on real hyperspectral
images contaminated by strip noise generated by hyper-
spectral systems during the experiment. As shown in Fig. 6,
our proposed model has the unprecedented capability of
removing the noise that is generated due to the malfunction of
the hyperspectral imaging systems.
5. Improvement of nitrogen stress diagnosis
As one of the potential technologies, HSI has been applied to
various aspects of plant research. Literature review shows
that well-known nitrogen absorption bands can produce
comparative results on plant nitrogen evaluation (Berger et al.,
2020; Sun et al., 2020), the nitrogen sensitive absorption bands
are mostly located in the VIS area, which showed lower

Table 2 e The average PSNR (in dB)/SSIM value of stripe noise cancellation performance on set of 10 hyperspectral images.
Noise Level PSNR/SSIM after strip noise cancellation
SNRWDNN WDLRGS DDCNN RDCNN Ours
s ¼ 0.05 27.04/0.79 29.74/0.91 29.52/0.94 33.18/0.95 33.82/0.96
s ¼ 0.1 24.50/0.82 29.34/0.87 26.67/0.90 30.07/0.93 32.53/0.94
s ¼ 0.15 22.87/0.60 27.88/0.85 27.37/0.85 28.17/0.87 30.09/0.88
s ¼ 0.20 20.35/0.49 24.57/0.82 25.84/0.83 26.45/0.85 28.77/0.86
s ¼ 0.25 18.13/0.45 22.63/0.78 23.36/0.80 25.29/0.80 27.61/0.82
172 b i o s y s t e m s e n g i n e e r i n g 2 1 9 ( 2 0 2 2 ) 1 6 5 e1 7 6
Fig. 4 e De-Striping Convolution Neural Network (DS-CNN) Denoising visulization. (a) The orginal clean image and 6 noise
scale images on selected bands, and (b) the original clean image and DS-CNN denoised image respectively.
reflectance and transmission but higher absorption by chlo-
rophyll a and chlorophyll b, plant leave reflectance also
showed red edge between the visible area and near-infrared
area (720e950 nm) (Berger et al., 2020; Wheeler, Levia, &
Vargas, 2020). Generally, an increase of nitrogen content
supply for the crop can be revealed by near-infrared region
reflectance increase and visible region reflectance decrease
(Liang et al., 2018; Mishra et al., 2017), these advanced un-
derstanding of plant spectral reflectance can be the guidance
for further research to reveal the relationship between the
absorption band images and the nitrogen contents of plant.
Note that our initial purpose was evaluating the nitrogen
stress from rice leaves, inspired by the spectral sensitivities
and strong capabilities of machined learning, we generated
the dataset with the sensitive band images for plant nitrogen
stress diagnosis. We concatenated band image R560 from the
visible area, band image R810, and R950 from the near-infrared
area as ND-CNN input images, with 10 times random cropping
as data augmentation with the hyperspectral image before
and after denoising.
The ND-CNN model was firstly trained on the noisy dataset
as mentioned at the beginning which led to overfitting,
although we tried to adjust hyperparameters such as con-
volutional layers, filters even activation functions, overfitting
remain unchanged see Fig. 7(a). While the accuracy of the ND-
CNN remarkably increased on the denoised dataset by DS-CNN
shown in Fig. 7(b), ND-CNN successfully avoided overfitting on
the dataset (b) with higher accuracy and lower loss.
As shown in Fig. 7, in terms of the training process, the
stripe-free image dataset showed a remarkable improvement
in both accuracy and validation loss, and a trivial difference
between the training and validation accuracy was observed in
the stripe-free dataset compared to the dataset that is
contaminated by the stripe noise; indicated that model
trained on the noisy dataset more susceptible to overfitting
problem.
In terms of classification accuracy (Fig. 8), the stripe-free
image dataset has also shown significant improvement in
distinguishing five levels of nitrogen stresses. The accuracy
was improved from 84.44% to 99.56%.
The classification confusion matrix in Fig. 8 showed the
ND-CNN performance on the dataset (a) before strip removal
and dataset (b) after strip removal by DS-CNN, it is a
remarkable improvement for nitrogen stress diagnosis on the
dataset (b) compared to (a), for the nitrogen stress diagnosis
on the dataset (a) without remove strips, five grades of nitro-
gen stress were misclassified to some extent, which had been
improved substantially on the dataset after denoised, except a
few samples from N0 and N3 been misclassified, the rest 3
classes (N1, N2, and N4) are predicted with 100% accuracy.
To show the improvements for nitrogen diagnosis practi-
cally, t-distributed stochastic neighbor embedding (t-SNE)
(Van der Maaten & Hinton, 2008) visualization of feature
extraction by ND-CNN on the dataset (a) before strip removal
and (b) dataset after strip removal by DS-CNN are shown in
Fig. 9, here we observed that the results of confusion matrix in
 b i o s y s t e m s e n g i n e e r i n g 2 1 9 ( 2 0 2 2 ) 1 6 5 e1 7 6 173

Fig. 5 e Example of De-Striping Convolution Neural Network (DS-CNN) validation performance on the 6 noise levels. (a) The
denoising comparison with specific 66th band on one of the hyperspectral image patch (b) selected RGB (R:21st, G:62nd,
B:104th band) image before and after denoised on real leaf HSI image patch.

Fig. 8 is consistent with the t-SNE 2D map in Fig. 9, on the
dataset (a) the 5 feature maps corresponding to 5 nitrogen
stress overlapped each other which means a high number of
misclassifications, the whole map are distributed in a larger
space than the map extracted from the dataset (b). The t-SNE
feature clustering of ND-CNN on the dataset after strip
removal can be distributed in a map within 80  80 area rather
than 160  160 area compared to the clustering on the dataset
before strip removal, Besides, the distribution of feature
groups from the dataset (b) is separated without any over-
lapping, which indicates an out-performing nitrogen diag-
nosis result on the dataset (b).
174 b i o s y s t e m s e n g i n e e r i n g 2 1 9 ( 2 0 2 2 ) 1 6 5 e1 7 6

Fig. 6 e De-Striping Convolution Neural Network (DS-CNN) denoising on real strip noised hyperspectral image case. Top row
for noised hyperspectral band images, and the bottom row for DS-CNN denoised image. Note that the false color is chosen
for clear visualization. (For interpretation of the references to color in this figure legend, the reader is referred to the Web
version of this article.)

Fig. 7 e The accuracy and loss of nitrogen diagnosis CNN (ND-CNN) on dataset before (a) after (b) denoising.

Fig. 8 e The confusion matrix of Nitrogen Diagnosis CNN (ND-CNN) on (a) test dataset before denoising and after (b).
 b i o s y s t e m s e n g i n e e r i n g 2 1 9 ( 2 0 2 2 ) 1 6 5 e1 7 6
Fig. 9 e 2D t-distributed stochastic neighbor embedding (t-SNE) feature visualizations after Nitrogen Diagnosis
Conculutional Neuron Network (ND-CNN) on the dataset (a) before and (b) after denoising by De-Striping Convolutional
Neural Network (DS-CNN).
6. Conclusions
This paper proposed an advanced deep learning networks
architecture for stripe noise removal in a hyperspectral image
acquired using a close-range imaging system. Extensive ex-
periments on simulated and real data demonstrate that the
proposed model can effectively remove the stripe noise from
the input image without employing any postprocessing step
while maintaining structural and textural details of the output
image. Integration of the element-wise feature concatenation
mechanism and dropout block helps to boost deep learning
performance and hence improve the image quality and over-
smoothing issues. Our method outperforms several existing
state-of-the-art destriping methods in terms of both peak
signal-to-noise ratio (PSNR) and structural similarity index
metric (SSIM) quality measures. To represent the practical
challenge of the proposed DS-CNN for stripe noise removal,
we developed another deep learning model, so-called ND-
CNN, to diagnose the nutrient status of the plants using the
stripe-free images generated by the DS-CNN model. The re-
sults showed that the nutrient status diagnosis was signifi-
cantly improved after stripe noise removal. Considering all
types of noise in one study might be a broad topic and we also
believe that there is no universal deep learning model that is
suitable for all types of noises. According to our experience,
training a single deep learning model to perform multiple
tasks without defining a specific objective function (such as
ones used to solve cross-domain problems) would signifi-
cantly reduce the accuracy. In the future, the proposed model
can be tuned and extended to be universally suitable for many
types of noises.
Declaration of competing interest
The authors declare that they have no known competing
financial interests or personal relationships that could have
appeared to influence the work reported in this paper.
175
Acknowledgments
This research was supported by the Key Research and Devel-
opment Program of China (2021YFD2000104).
references
Abdalla, A., Cen, H., Wan, L., Mehmood, K., & He, Y. (2021).
Nutrient status diagnosis of infield oilseed rape via deep
learning-enabled dynamic model. IEEE Transactions on
Industrial Informatics, 17(6), 4379e4389. https://doi.org/10.1109/
TII.2020.3009736
ret, J.-B., Hank, T., Wocher, M., Mauser, W.,
Berger, K., Verrelst, J., Fe
et al. (2020). Retrieval of aboveground crop nitrogen content
with a hybrid machine learning method. International Journal of
Applied Earth Observation and Geoinformation, 92, 102174. https://
doi.org/10.1016/j.jag.2020.102174
ret, J.-B., Wang, Z., Wocher, M.,
Berger, K., Verrelst, J., Fe
Strathmann, M., et al. (2020). Crop nitrogen monitoring:
Recent progress and principal developments in the context of
imaging spectroscopy missions. Remote Sensing of Environment,
242, 111758. https://doi.org/10.1016/j.rse.2020.111758
Cai, S., Shu, Y., Chen, G., Ooi, B. C., Wang, W., & Zhang, M. (2019).
Effective and efficient dropout for deep convolutional neural
networks. ArXiv, 1e12. Retrieved from http://arxiv.org/abs/
1904.03392.
Cen, H., Wan, L., Zhu, J., Li, Y., Li, X., Zhu, Y., et al. (2019). Dynamic
monitoring of biomass of rice under different nitrogen
treatments using a lightweight UAV with dual image-frame
snapshot cameras. Plant Methods, 15(1), 32.
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., &
Yuille, A. L. (2018). DeepLab: Semantic image segmentation
with deep convolutional nets, atrous convolution, and fully
connected CRFs. IEEE Transactions on Pattern Analysis and
Machine Intelligence, 40(4), 834e848. https://doi.org/10.1109/
TPAMI.2017.2699184
Chen, J., Shao, Y., Guo, H., Wang, W., & Zhu, B. (2003). Destriping
CMODIS data by power filtering. IEEE Transactions on Geoscience
and Remote Sensing, 41(9), 2119e2124. https://doi.org/10.1109/
TGRS.2003.817206
176 b i o s y s t e m s e n g i n e e r i n g 2 1 9 ( 2 0 2 2 ) 1 6 5 e1 7 6
Dao, P. D., Mantripragada, K., He, Y., & Qureshi, F. Z. (2021).
Improving hyperspectral image segmentation by applying
inverse noise weighting and outlier removal for optimal
scale selection. ISPRS Journal of Photogrammetry and Remote
Sensing, 171, 348e366. https://doi.org/10.1016/j.isprsjprs.
2020.11.013
Deshpande, A. M., Patale, S. R., & Roy, S. (2021). Removal of line
striping and shot noise from remote sensing imagery using a
deep neural network with post-processing for improved
restoration quality. International Journal of Remote Sensing,
42(19), 7357e7380. https://doi.org/10.1080/
01431161.2021.1957512
Guan, J., Lai, R., & Xiong, A. (2019). Wavelet deep neural network
for stripe noise removal. IEEE Access, 7, 44544e44554. https://
doi.org/10.1109/ACCESS.2019.2908720
Hore, A., & Ziou, D. (2010). Image quality metrics: PSNR vs. SSIM.
In In 2010 20th international Conference on pattern
recognition (pp. 2366e2369). IEEE. https://doi.org/10.1109/
ICPR.2010.579.
Jung, J. G., Song, K. E., Hong, S. H., & Shim, S. I. (2021).
Hyperspectral characteristics of an individual leaf of wheat
grown under nitrogen gradient. Plants, 10(11), 2291. https://
doi.org/10.3390/plants10112291
Kuang, X., Sui, X., Chen, Q., & Gu, G. (2017). Single infrared image
stripe noise removal using deep convolutional networks. IEEE
Photonics Journal, 9(4), 76924e76934. https://doi.org/10.1109/
JPHOT.2017.2717948
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning.
Nature, 521(7553), 436e444. https://doi.org/10.1038/
nature14539
Lempitsky, V., Vedaldi, A., & Ulyanov, D. (2018). Deep image prior.
In 2018 IEEE/CVF conference on computer vision and pattern
recognition (pp. 9446e9454). IEEE. https://doi.org/10.1109/
CVPR.2018.00984.
Liang, L., Di, L., Huang, T., Wang, J., Lin, L., Wang, L., et al. (2018).
Estimation of leaf nitrogen content in wheat using new
hyperspectral indices and a random forest regression
algorithm. Remote Sensing, 10(12), 1940. https://doi.org/10.3390/
rs10121940
Li, F., Li, D., Elsayed, S., Hu, Y., & Schmidhalter, U. (2021). Using
optimized three-band spectral indices to assess canopy N
uptake in corn and wheat. European Journal of Agronomy, 127,
126286. https://doi.org/10.1016/j.eja.2021.126286
Liu, R., Cai, W., Li, G., Ning, X., & Jiang, Y. (2022). Hybrid dilated
convolution guided feature filtering and enhancement
strategy for hyperspectral image classification. IEEE Geoscience
and Remote Sensing Letters, 19. https://doi.org/10.1109/
LGRS.2021.3100407
Liu, N., Li, W., Tao, R., Fowler, J. E., & Yang, L. (2019).
Hyperspectral stripes removal with wavelet-domain low-
rank/group-sparse decomposition. In 2019 10th Workshop on
hyperspectral Imaging and signal processing: Evolution in remote
sensing (WHISPERS) Vol. 2019eSepte (pp. 1e4). IEEE. https://
doi.org/10.1109/WHISPERS.2019.8921401.
Liu, X., Tanaka, M., & Okutomi, M. (2013). Single-image noise level
estimation for blind denoising. IEEE Transactions on Image
Processing, 22(12), 5226e5237. https://doi.org/10.1109/
TIP.2013.2283400
Mao, X.-J., Shen, C., & Yang, Y.-B. (2016). Image restoration using
convolutional auto-encoders with symmetric skip
connections. Retrieved from http://arxiv.org/abs/1606.08921.
Mishra, P., Asaari, M. S. M., Herrero-Langreo, A., Lohumi, S.,
Diezma, B., & Scheunders, P. (2017). Close range hyperspectral
imaging of plants: A review. Biosystems Engineering, 164, 49e67.
https://doi.org/10.1016/j.biosystemseng.2017.09.009
Pan, J.-J., & Chang, C.-I. (1992). Destriping of Landsat MSS images
by filtering techniques. Photogrammetric Engineering & Remote
Sensing, 58(10), 1417e1423.
Sabzi, S., Pourdarbani, R., Rohban, M. H., Garcı́a-Mateos, G., &
Arribas, J. I. (2021). Estimation of nitrogen content in cucumber
plant (Cucumis sativus L.) leaves using hyperspectral imaging
data with neural network and partial least squares
regressions. Chemometrics and Intelligent Laboratory Systems,
217, 104404. https://doi.org/10.1016/j.chemolab.2021.104404
Sellami, A., & Tabbone, S. (2022). Deep neural networks-based
relevant latent representation learning for hyperspectral
image classification. Pattern Recognition, 121, 108224. https://
doi.org/10.1016/j.patcog.2021.108224
Shang, W., Sohn, K., Almeida, D., & Lee, H. (2016). Understanding
and improving convolutional neural networks via
concatenated rectified linear units. 33rd International Conference
on Machine Learning, ICML, 5, 3276e3284, 2016.
Shan, W., Liu, P., Mu, L., Cao, C., & He, G. (2019). Hyperspectral
image denoising with dual deep CNN. IEEE Access, 7,
171297e171312. https://doi.org/10.1109/ACCESS.2019.2955810
Sidorov, O., & Hardeberg, J. Y. (2019). Deep hyperspectral prior:
Single-image denoising, inpainting, super-resolution. In 2019
IEEE/CVF international conference on computer vision workshop
(ICCVW) (pp. 3844e3851). IEEE. https://doi.org/10.1109/
ICCVW.2019.00477.
Sun, D., Xu, H., Weng, H., Zhou, W., Liang, Y., Dong, X., et al.
(2020). Optimal temporal-spatial fluorescence techniques for
phenotyping nitrogen status in oilseed rape (Brassica napus
L.). Journal of Experimental Botany. https://doi.org/10.1093/jxb/
eraa372
Torres, J., & Infante, S. O. (2001). Wavelet analysis for the
elimination of striping noise in satellite images. Optical
Engineering, 40(7), 1309e1314. https://doi.org/10.1117/1.1383996
Tros, M., Mascoli, V., Shen, G., Ho, M. Y., Bersanini, L., Gisriel, C. J.,
et al. (2021). Breaking the red limit: Efficient trapping of long-
wavelength excitations in chlorophyll-f-containing
photosystem I. Inside Cosmetics, 7(1), 155e173. https://doi.org/
10.1016/J.CHEMPR.2020.10.024
Van der Maaten, L., & Hinton, G. (2008). Visualizing Data using t-
SNE. Journal of Machine Learning Research, 9(86), 2579e2605.
Wang, L., Chen, S., Li, D., Wang, C., Jiang, H., Zheng, Q., et al.
(2021). Estimation of paddy rice nitrogen content and
accumulation both at leaf and plant levels from UAV
hyperspectral imagery. Remote Sensing 2021, 13(15), 2956.
https://doi.org/10.3390/RS13152956
Wheeler, K. I., Levia, D. F., & Vargas, R. (2020). Visible and near-
infrared hyperspectral indices explain more variation in
lower-crown leaf nitrogen concentrations in autumn than in
summer. Oecologia, 192(1), 13e27. https://doi.org/10.1007/
s00442-019-04554-2