Expert Systems With Applications 231 (2023) 120628

Contents lists available at ScienceDirect

Expert Systems With Applications

journal homepage: www.elsevier.com/locate/eswa

A parallel and serial denoising network

Qi Zhang a , Jingyu Xiao b , Chunwei Tian c,d ,∗, Jiayu Xu e ,∗∗, Shichao Zhang b , Chia-Wen Lin f
a
School of Economics and Management, Harbin Institute of Technology at Weihai, Weihai, 264209, China
b
School of Computer Science, Central South University, Changsha, 410083, China
c
School of Software, Northwestern Polytechnical University, Xi’an, 710129, Shaanxi, China
d
Research & Development Institute of Northwestern Polytechnical University in Shenzhen, 518057, China
e Shenzhen Research Institute, Guangdong Databeyond Technology Co., Ltd, Shenzhen, Guangdong, China
f Department of Electrical Engineering and the Institute of Communications Engineering, National Tsing Hua University, Hsinchu 300, Taiwan

ARTICLE INFO ABSTRACT

Keywords: Convolutional neural networks (CNNs) have performed well in image denoising. Although some CNNs enlarge
Deformable convolution convolutional kernels and increase stacked convolutional layers to overcome the locality defect of convolutional
Heterogenous networks operations, they may increase computational costs. In this paper, we propose a parallel and serial denoising
Enhanced residual dense architecture
network (PSDNet) for image denoising to preserve image texture. Specifically, the proposed PSDNet contains
CNN
a parallel block (PB), a serial block (SB), and a reconstruction block (RB). A PB uses two heterogeneous
Image denoising
sub-networks with a deformable convolution in a parallel way to extract comparative information for better-
recovering image texture. A SB utilizes an enhanced residual dense architecture via combinations of a batch
normalization, ReLU, and convolutional layer in a serial way to refine obtained features for obtaining more
accurate noise information. A RB is responsible for reconstructing images. Experimental results reveal that our
PSDNet is very effective in image denoising, according to quantitative analysis and visual analysis. Codes can
be obtained at https://github.com/hellloxiaotian/PSDNet.

1. Introduction
It is known that collected images often are damaged by inevitable
noise when images are captured and transferred by digital camera de-
vices (Healey & Kondepudy, 1994). Motivated by that, image denoising
techniques are developed (Buades, Coll, & Morel, 2005). Presented
image denoising techniques usually depend on the degradation equa-
tion of 𝑥 = 𝑦 − 𝜇 to obtain clean images (Liu, Zhang, Zhang, Lin, &
Zuo, 2018), where 𝑦 and 𝜇 are used to symbol given noisy images
and noise, respectively. 𝑥 denotes a recovered clean image. Noise is
represented via additive white Gaussian noise with standard deviation
𝜎 to test performance of an obtained denoiser in general. According to
the principles of mentioned degradation equation, a lot of denoising
methods are proposed (Rajni & Anutam, 2014). For instance, Markov
random field (MRF) method exploited wavelet transform and smooth-
ness measure to filter noise for obtaining a denoiser (Malfait & Roose,
1997). A weighted nuclear norm minimization (WNNM) resorted to
low-rank clustering correlation to suppress noise for restoring high-
quality images (Gu, Zhang, Zuo, & Feng, 2014). Wang, Yang, and Fu
(2010) ingeniously transformed the noisy images into the frequency
domain with wavelet transformation, and then optimize the model
with feature vectors obtained in the wavelet domain. Besides, there
are other popular denoising methods, i.e., a cascade of shrinkage fields
(CSF) (Schmidt & Roth, 2014) and block-matching and 3D filtering
(BM3D) (Dabov, Foi, Katkovnik, & Egiazarian, 2007). Although these
denoising methods can recover details of damaged images, they relied
on manual tuning parameters and complex optimization methods to ob-
tain denoising performance, which may increase training time (Lucas,
Iliadis, Molina, & Katsaggelos, 2018).
To solve these drawbacks, CNNs of automatically learning fea-
tures (Jafarbigloo & Danyali, 2021) are presented for image denois-
ing (Tian et al., 2021). For example, Zhang, Zuo, Chen, Meng and
Zhang (2017) used batch normalization (BN) (Ioffe & Szegedy, 2015)
and a residual learning operation (RL) (He, Zhang, Ren, & Sun, 2016)
as components to plug in a CNN for image denoising. Subsequently,
researchers are absorbed in improving network architectures to extract
more useful information in image denoising (Chang, Yan, Fang, Zhong,
& Liao, 2018). Yuan et al. combined RL and multi-scale techniques into
a CNN to mine rich features for overcoming the challenge of insufficient

∗ Corresponding author at: School of Software, Northwestern Polytechnical University, Xi’an, 710129, Shaanxi, China.
∗∗ Corresponding author.
E-mail addresses: hit_zq910057@163.com (Q. Zhang), jyxiao@csu.edu.cn (J. Xiao), chunweitian@163.com (C. Tian), xujiayu@databeyond.cn (J. Xu),
zhangsc@csu.edu.cn (S. Zhang), cwlin@ee.nthu.edu.tw (C.-W. Lin).

https://doi.org/10.1016/j.eswa.2023.120628
Received 7 May 2022; Received in revised form 3 April 2023; Accepted 27 May 2023
Available online 7 June 2023
0957-4174/© 2023 Published by Elsevier Ltd.
Q. Zhang et al.
hyperspectral noisy images (Yuan, Zhang, Li, Shen, & Zhang, 2018).
Inspired by UNet (Ronneberger, Fischer, & Brox, 2015), Yu et al.
proposed a denoising architecture that iteratively up- and down-scales
features to boost both the usage efficiency of GPU memory and the
size of receptive field for image denoising (Yu, Park, & Jeong, 2019).
To decrease the computational overhead of CNN architecture, Panda
et al. utilized dilated convolutions and exponential linear units to build
an image denoiser (Panda, Naskar, & Pal, 2018). These models have
revealed the great complexity of CNNs. Consequently, some researchers
sought to integrate conventional algorithms, e.g., K-SVD (Chen, Pu,
Tong, & Wu, 2022), into CNNs to address image denoising (Wang,
Qin, & Zhu, 2017). For instance, Tian et al. jointly leveraged signal
processing algorithms and discriminative learning in CNNs to extract
frequency and semantic information for image denoising (Tian et al.,
2022). In addition, Li and Wu (2019) exploited the split Bergman iter-
ation algorithm to covert depth image inpainting problem into image
denoising problem. Also, Jia et al. converted the denoising process of
CNN into a fractional-order differential optimization problem to give
long-term memory ability to CNNs (Jia, Liu, Feng, & Zhang, 2019).
To make the denoising model interpretable, Herbreteau et al. managed
to implement the traditional Discrete Cosine Transform algorithm in
the form of CNN through the combination of convolutions and hard
shrinkage function (Herbreteau & Kervrann, 2022). Besides, there are
many other popular denoising methods for blind denoising (Zhao,
Shao, Bao, & Li, 2019), real damaged image denoising (Yu, Fu, &
Ge, 2022), and magnetic resonance imaging (MRI) denoising (Chen,
Zhou, & Adnan, 2021). Although these methods are effective for image
denoising, they may mainly focus on pixel-wise accuracy instead of
texture precise.
In this paper, we propose a parallel and serial denoising network
(PSDNet) to overcome these challenges. The core consideration of
our work is how to build a model architecture capturing semantic
information. Firstly, we utilized two parallel but shallow modules to
mine contrasting features, which is like two eyes seeing one object from
different angles. Then, a serial architecture powerfully filtered low-
quality information from those contrasting features. The merit of this
arrangement is that we have well-balanced the acquisition and purifi-
cation of diverse features. Specifically, the proposed PSDNet contains
a parallel block (PB), a serial block (SB), and a reconstruction block
(RB). A PB uses two heterogeneous sub-networks with a deformable
convolution in a parallel way to extract comparative information for
suppressing noise. A SB utilizes an enhanced residual dense architecture
via combinations of a batch normalization, ReLU, and convolutional
layer in a serial way to refine obtained features for obtaining more
accurate noise information. A RB is responsible for reconstructing
images. Experimental results reveal that our PSDNet is very effective
in image denoising, i.e., synthesized noisy image and real noisy image
denoising.
Our main contributions can be summarized as follows.
(1) Two heterogeneous sub-networks with a deformable convolution
are designed to obtain comparative information for filtering noise.
(2) We use combinations of a batch nomination, ReLU, and con-
volutional layer to enhance a residual dense architecture to remove
redundant features to improve denoising performance.
(3) Combining a parallel and serial way can achieve an effective
CNN for improving the effects of images denoising, which only takes
denoising time of 12.41% for noisy images with 256 × 256, parameters
of 6.87% and GFLOPs of 5.06% of the popular denoising method
(MPRNet). That is suitable to mobile digital devices.
(4) The proposed network is effective for both synthesized noisy
image and real noisy image denoising.
The remainder of this paper is listed as follows. Section 2 discusses
the related work of PSDNet and Section 3 demonstrates the proposed
method in detail. Section 4 shows an analysis of the deigned network
and its experimental results in image denoising. Section 5 gives a
conclusion of this paper.
2
Expert Systems With Applications 231 (2023) 120628
2. Related work
2.1. Deep convolutional networks for image denoising
The strong generalization abilities of deep CNNs are very effective
in image denoising (Zhao, Lv, Liu, & Qin, 2017). For instance, Jain et al.
used a CNN in an unsupervised way to deal with image denoising (Jain
& Seung, 2008). Wang et al. adopted a L2 loss of total variation to
eliminate the effects of noise for image denoising (Wang et al., 2010).
Tai et al. integrated recursive mechanisms into a CNN to facilitate
more hierarchical features to act deep layers for recovering images (Tai,
Yang, Liu, & Xu, 2017). To consider computational costs and denoising
results, Zhang et al. embedded a CNN into an optimization method to
predict a clean image (Zhang, Zuo, Gu & Zhang, 2017). For limited
hardware resources, Tian et al. used batch renormalization, dilated
convolution, and residual learning operations in two sub-networks to
extract complementary features in image denoising (Tian, Xu & Zuo,
2020). To reduce denoising computational costs, Cho et al. applied
separable convolution and the gradient prior in a CNN to reduce the
complexities for obtaining denoiser (Cho & Kang, 2018). To improve
denoising performance, a dual-stage CNN used residual operations to
coarsely remove noise (Sun, Kottayil, Mukherjee, & Cheng, 2018).
Then, a multi-scale feature selection layer was used to recover details
of damaged images (Sun et al., 2018). The combination of the gener-
ative adversarial network and the nearest neighbor algorithm is very
effective for image denoising (ZhiPing, YuanQi, Yi, & XiangBo, 2018).
Besides, Gai and Bao (2019) proposed a novel adaptive residual convo-
lutional neural network as well as an improved loss function, namely,
the joint use of perceptual loss (Wu, Chen, Ji, & Zhan, 2018) and
mean square error (MSE) (Ephraim & Malah, 1984), to boost denoising
performance. To address the camera relocalization problem, Jin, Yu,
Li, and Fei (2021) designed a framework based on CNN and short-term
memory (LSTM) (Jonides, Lewis, Nee, Lustig, Berman, & Moore, 2008)
to learn accurate six-degrees of freedom pose of images, which can
further improve the accuracy and real-time of the camera relocalization
model. Motivated by these, we also use a deep CNN for image denoising
in this paper.
2.2. Residual dense learning architectures for image restoration
To address the long-term decency problem, extracting hierarchal
features via dense residual learning architectures is presented for image
restoration (Zhang, Tian, Kong, Zhong & Fu, 2018). He et al. used
dense residual learning architectures to mine local and global features
to relieve the vanishing gradient problem in image restoration (He
et al., 2016). Kim et al. used a cascading network with a group residual
dense block for real noisy image denoising (Kim, Ryun Chung, &
Jung, 2019). Bao et al. used a U-Net with multi-scale residual dense
blocks to prevent overfitting of the denoising network (Bao, Yang,
Wang, Bai, & Lee, 2020). Using residual dense techniques and global
residual learning operations in a U-Net to accelerate training speed in
image denoising (Gurrola-Ramos, Dalmau, & Alarcón, 2021). To further
improve image denoising, Park et al. integrated a dense connection
and residual learning operation into a U-Net to improve denoising
effects (Park, Yu, & Jeong, 2019). Mentioned residual dense learning
techniques are effective in image restoration, thus, we use residual
dense learning techniques in this paper. Differing from these methods,
we use the combination of BN+ReLU+Conv rather than Conv+ReLU
and Conv+BN+ReLU to enhance a residual dense architecture in image
denoising, where BN+ReLU+Conv denotes the combination of BN,
ReLU, and convolutional layers. Conv+ReLU and Conv+BN+ReLU are
used to represent the combination of a convolutional layer and ReLU,
a combination of a convolutional layer, BN, and ReLU, respectively.
Q. Zhang et al.
Fig. 1. Network architecture of PSDNet.
2.3. Deformable convolutions for image processing
Because convolution operation can make locations of extracted tar-
gets suffer from offsets, the performance of image applications will de-
crease (Dai, Qi, Xiong, Li, Zhang, Hu, et al., 2017). To address this issue,
deformable convolution techniques are presented (Dai et al., 2017).
Zhu et al. used deformable convolutions into a CNN to adaptively
sample obtained features for hyperspectral image (HSI) classification
under complex conditions (Zhu, Fang, & Ghamisi, 2018). Thomas et al.
used kernel point convolution and deformable to adaptively adjust
convolution kernels for recovering deformable targets for image clas-
sification and segmentation (Thomas et al., 2019). To extract unevenly
distributed context information, combining location-aware deformable
convolution and attention can overcome offset estimation from con-
volutional operations in object detection (Zhang & Kim, 2019). Shim
et al. used multi-scale architecture with nonlocal blocks in unsupervised
ways to overcome offsets from convolutional operations for image
super-resolution (Shim, Park, & Kweon, 2020). According to mentioned
illustrations, we can see that deformable convolutions can not only be
effective for high-level vision tasks, i.e., image classification and object
detection, but also better deal with low-level vision tasks, i.e., image
super-resolution. Thus, we also a deformable convolution into a CNN
to tackle the image denoising problem in this paper.
3. The proposed method
In this paper, we adopt an effective CNN to train a denoiser.
We first present a PSDNet architecture in this section. Subsequently,
we present detailed information on training for a PSDNet denoising
model in Section 4. Specifically, this section successively introduces
network architecture, loss function, parallel block, serial block, and
reconstruction block.
3.1. Network architecture
17-layer PSDNet contains three blocks, i.e., a 11-layer parallel block
as well as PB, a 6-layer serial block as well as SB, and a reconstruction
block as well as RB as shown in Fig. 1. A PB designed two het-
erogeneous sub-networks with a deformable convolution in a parallel
way to mine comparative wide information and context information
for suppressing noise. A SB utilizes simple components, i.e., batch
3
Expert Systems With Applications 231 (2023) 120628
normalization, ReLU, and convolution layers to design an enhanced
residual dense architecture in a serial way to remove redundant fea-
tures for improving denoising effects. A RB resorts to a residual learning
operation to act between a given image and predicted noisy mapping to
construct a clean image. The mentioned procedure can be formulated
as follows.
𝐼𝑃 = 𝑃 𝑆𝐷𝑁𝑒𝑡(𝐼𝑛 )
(1)
= 𝑅𝐵(𝑆𝐵(𝑃 𝐵(𝐼𝑛 ))),
where 𝐼𝑃 and 𝐼𝑛 are used to stand for noisy images and clean images,
respectively. 𝑃 𝑆𝐷𝑁𝑒𝑡 is utilized to express a function of PSDNet. 𝑃 𝐵,
𝑆𝐵, and 𝑅𝐵 are functions of PB, SB, and RB, respectively.
3.2. Loss function
PSDNet uses the MSE as a loss function to train a denoiser, according
to a degradation equation of 𝑥 = 𝑦 − 𝜇. That is, we use paired data,
i.e., clean image and noisy image acts a CNN via MSE to train a robust
denoising method. The process above can be explained as
∑
𝑁
1 ‖ ‖2
𝑙(𝜃) = ‖𝑃 𝑆𝐷𝑁𝑒𝑡(𝐼𝑛𝑖 ) − 𝐼𝑝𝑖 ‖ , (2)
2𝑁 ‖ ‖
𝑖=1
where 𝐼𝑝𝑖 and 𝐼𝑛𝑖 denote the 𝑖th noisy image and referenced clean image,
respectively. 𝑁 is the number of noisy images. 𝜃 denotes the learned
parameters and 𝑙 is the loss function. The procedure can be optimized
via Adam (Kingma & Ba, 2014).
3.3. Parallel block
11-layer parallel block uses two heterogeneous sub-networks with
a deformable convolution in a parallel way to extract comparative
information to suppress noise. The first sub-network is composed of one
Deformconv+ReLU and ten Conv+BN+ReLU, respectively. The Deform-
conv+ReLU denotes a combination of a deformable convolution (Dai
et al., 2017) and ReLU, which can be used to correct locations of noise
features by a convolutional operation. Also, its input channel number
and output channel number are 3 and 64 when a given noisy image is
color. If a given noisy image is gray, its input channel number and out-
put channel number are 1 and 64, respectively. Ten Conv+BN+ReLU
are used to learn deep noisy features, where Conv+BN+ReLU is combi-
nation of a convolutional layer, BN, and ReLU. Mentioned kernel sizes
Q. Zhang et al.
are 3 × 3. BN and ReLU are applied to normalize training samples and
map linear features into non-linear features, respectively. And its input
channel number and output channel number are 64, respectively. The
second sub-network designs a 5-layer novel network architecture to ex-
tract different features in image denoising. It includes Conv+BN+ReLU,
Conv+BN, Conv+BN+ReLU, Conv+BN and Conv+BN+ReLU, where
Conv+BN stands for a combination of a convolutional layer and BN.
Input channel number and output channel number are 3 and 64 for
color noisy images. Input channel number and output channel number
are 1 and 64 for gray noisy images. All sizes of convolutional kernels
are 3 × 3. The first sub-network and second sub-network have heteroge-
neous architectures, which facilitates obtained features richer in image
denoising. Obtained features can be fused via a residual operation.
Mentioned illustrations can be symbolized as
𝐼𝑃 𝐵 = 𝑃 𝐵(𝐼𝑛 )
= 𝑓1 (𝐼𝑛 ) + 𝑓2 (𝐼𝑛 )
= 10𝑅𝐵𝐶(𝐷𝑅(𝐼𝑛 )) + 𝑅𝐵𝐶(𝐶𝐵(𝑅𝐵𝐶(𝐶𝐵(𝑅𝐵𝐶(𝐼𝑛 )))))
where 𝑓1 and 𝑓2 denote the functions of first and second sub-networks,
respectively. 𝐷𝑅 are expressed as a combination of a deformable
convolution and ReLU, where a deformable convolution is shown in Dai
et al. (2017). 𝑅𝐵𝐶 stands for a combination of a convolution, BN, and
ReLU. 10𝑅𝐵𝐶 is used to represent ten combinations of a convolution,
BN, and ReLU. 𝐶𝐵 is defined as a combination of a convolution and
BN. + is a residual operation as well as ⊕ in Fig. 1. 𝐼𝑃 𝐵 is the output
of PB and acts SB.
3.4. Serial block
To prevent over-enhancement phenomenon, a 6-layer serial block
as well as SB is designed. SB is composed of six BN+ReLU+Conv,
where each BN+ReLU+Conv denotes a combination of BN, ReLU, and
a convolutional layer. BN (Ioffe & Szegedy, 2015) is used to improve
the training speed of trained denoising model via a normalization
operation. A convolutional layer is used to learn useful information and
convert non-linear features into linear features. Every convolution acts
every subsequent layer except for the last convolution via concatena-
tion operations. Mentioned kernel sizes are 3 × 3. The input channel
number and the output channel number of the first BN+ReLU+Conv
are both 64. The input channel number and the output channel num-
ber of the second BN+ReLU+Conv are 128 and 64, respectively. The
input channel number and the output channel number of the third
BN+ReLU+Conv are 192 and 64, respectively. The input channel num-
ber and the output channel number of the fourth BN+ReLU+Conv are
256 and 64, respectively. The input and the output channel number
of the fifth BN+ReLU+Conv are 320 and 64, respectively. The input
and the output channel number of the sixth BN+ReLU+Conv are 384
and 3 for color noisy images. If recovered images are gray, the output
channel number of final BN+ReLU+Conv is 1. The last convolutional
layer is used to obtain noisy mappings.
𝐼𝑆𝐵 = 𝑆𝐵(𝐼𝑃 𝐵 )
= 𝐶𝐵𝑅(𝐶𝐴(𝐼𝑃 𝐵 , 𝑂5 ))
𝑂1 = 𝐶𝐵𝑅(𝐼𝑃 𝐵 )
𝑂2 = 𝐶𝐵𝑅(𝐶𝐴(𝐼𝑃 𝐵 , 𝑂1 ))
𝑂3 = 𝐶𝐵𝑅(𝐶𝐴(𝐼𝑃 𝐵 , 𝑂1 , 𝑂2 ))
𝑂4 = 𝐶𝐵𝑅(𝐶𝐴(𝐼𝑃 𝐵 , 𝑂1 , 𝑂2 , 𝑂3 ))
𝑂5 = 𝐶𝐵𝑅(𝐶𝐴(𝐼𝑃 𝐵 , 𝑂1 , 𝑂2 , 𝑂3 , 𝑂4 ))
where 𝑆𝐵 and 𝐶𝑅𝐵 are functions of SB and the BN+ReLU+Conv,
respectively. BN+ReLU+ Conv is equal to a combination of BN, ReLU,
and a convolution. Besides, 𝐶𝐴 expresses a concatenation operation.
𝑂1 , 𝑂2 , 𝑂3 , 𝑂4 , 𝑂5 are the outputs of the first, second, third, fourth and
fifth BN+ReLU+Conv, respectively. 𝐼𝑆𝐵 is an output of SB and acts a
reconstruction block.
Expert Systems With Applications 231 (2023) 120628
3.5. Reconstruction block
The reconstruction block uses a residual operation to act a given
noisy image and predicted noisy mapping for obtaining a clean image.
This process can be expressed as follows.
𝐼𝑃 = 𝑅𝐵(𝐼𝑛 , 𝐼𝑆𝐵 )
(6)
= 𝐼𝑛 − 𝐼𝑆𝐵
where - denotes a residual operation as well as ⊕ in Fig. 1.
4. Experiments
4.1. Training and test datasets
We use synthetic and real noisy images to test effectiveness of the
(3) proposed PSDNet in image denoising. Synthetic noisy images can be
divided into gray and color synthetic noisy images.
In terms of gray synthetic noisy images, 400 natural images from the
Berkeley segmentation dataset (BSD) (Martin, Fowlkes, Tal, & Malik,
2001) are cropped into 180 × 180 as training images, according to
a denoising CNN, i.e., DnCNN (Zhang, Zuo, Chen et al., 2017). To
accelerate training speed, gray Gaussian noisy images with sizes of
180 × 180 can be cut as image patches of 40 × 40 in steps of 10. Also,
public Set12 (Zhang, Zuo, Chen et al., 2017) and BSD68 (Roth & Black,
2005) are chosen as test datasets to verify the denoising performance
for gray synthetic noisy images, where Set12 and BSD68 are derived
from the BSD.
In terms of color synthetic noisy images, we cut the same 400 color
images as gray noisy images as image patches of 40 × 40 in steps of
width of 20 and height of 30 as a training dataset. CBSD68 (Roth &
Black, 2005) is used to evaluate the denoising performance of PSDNet
for color synthetic Gaussian noisy images.
Before the training phase, we augment the training dataset through
scaling, rotating, and flipping the images. First, the images are ran-
domly resized with one of the bicubic interpolation factors: 1, 0.9, 0.8,
and 0.7. Then, we randomly pick one of the following four modes to
enhance the robustness of the target model: no rotating, rotating by
90◦ counterclockwise, rotating by 180◦ counterclockwise, rotating by
270◦ counterclockwise. Finally, for each image, we randomly decide
whether to conduct a horizontal flip to it or not.
In terms of real noisy images, 80 natural noisy images from the
PolyU training dataset (Xu, Li, Liang, Zhang, & Zhang, 2018) are used
to train a denoiser and CC (Nam, Hwang, Matsushita, & Kim, 2016) is
used to evaluate its denoising performance.
Note that all the training images are also cropped into 40 × 40
patches in steps of width of 20 and height of 30, as the color synthetic
noisy images.
4.2. Implementation details
The codes are conducted by Python of 3.7.10 and Pytorch of
(4) 1.1.0 (Pytorch, 2018). And they run on an Ubuntu of 18.04 with a
Nvidia Titan XP GPU, an Intel Xeon Gold 6140 CPU, and a RAM of
size 51G. To improve the training and test speed, Nvidia CUDA of 10.1
and cuDNN of 7 are employed to accelerate the speed of the mentioned
(5) GPU. Besides, the training epochs are 180, and the initial learning rate
is 1e-3. The learning rate decays by 0.2 times at milestones of 30, 60,
and 90, respectively. Other parameters are the same as Zhang, Zuo,
Chen et al. (2017).
4.3. Network analysis
We propose a 17-layer parallel and serial denoising network in
image denoising. It mainly depended on a 11-layer parallel block as
4
Q. Zhang et al. Expert Systems With Applications 231 (2023) 120628

Table 1
PSNR (dB) results of several networks on BSD68 for noise level of 25.
Methods PSNR (dB)
PSDNet 29.290
Method1: PSDNet without the second network in the PB 29.265
Method2: Method1 without deformconv+ReLU in the PB 29.260
Method3: Method2 without BN in the PB 29.253
Method4: Method3 without residual dense operations in the SB 29.226
Method5: Method4 without BN 29.031
Method6: Method5 without SB 28.185
Method7: Method4 with Conv+BN+ReLU rather than BN+ReLU+Conv 29.163

Table 2
Average PSNR (dB) of different methods on BSD68 with different noise levels of 15, 25, and 50. The best and second denoising results are marked by
red and blue lines, respectively.
Methods BM3D WNNM EPLL TNRD CSF MLP NLED TSLCSNet RED30
𝜎 = 15 31.07 31.37 31.21 31.42 31.24 – 31.43 – –
𝜎 = 25 28.57 28.83 28.68 28.92 28.74 28.96 28.93 – –
𝜎 = 50 25.62 25.87 25.67 25.97 – 26.03 – 26.24 26.35
MemNet IDCNN DnCNN IRCNN ECNDNet FFDNet ADNet HDCNN PSDNet PSDNet-B
– 30.78 31.72 31.63 31.71 31.63 31.74 31.74 31.78 31.63
– 28.61 29.23 29.15 29.22 29.19 29.25 29.25 29.29 29.18
26.35 25.78 26.23 26.19 26.23 26.29 26.29 26.23 26.31 26.26

well as PB, a 6-layer serial block as SB, and a reconstruction block
as well RB to obtain a good denoising performance. To show good
performance of key techniques, we test the validity and reasonableness
of key techniques as follows.
Parallel block: According to GoogLeNet (Szegedy et al., 2015), we
can see that enlarging the width of a deep CNN can extract more wide
features to improve performance in image classification. Inspired by
that, we use two sub-networks to extract commentary features in image
denoising. According to RDN, we can see that network architecture has
a bigger difference, and its performance is better (Zhang, Tian et al.,
2018). Motivated by that, we design two heterogeneous sub-networks
in a parallel way to remove noise. The first network is composed of
a deformable convolutional layer and ReLU and ten combinations of
a convolutional layer, BN, and ReLU. The second network consists of
three combinations of convolutional layer, BN and ReLU, two combi-
nations of convolutional layer and BN. The effectiveness of the two
sub-networks is verified by PSDNet and Method1 as shown in Table 1,
where Method1 is PSDNet without the second network in the PB. To
overcome the effect of target offset by a convolutional operation, a
deformable convolution is used in a CNN as the first layer in the first
sub-network. As shown in Table 1, Method1 has obtained a higher
PSNR than that of Method2, where Method2 denotes Method1 without
a deformconv+ReLU. To deal with unevenly distributed data, BN is
fused into each layer from the 2nd–10th layers in PB to normal-
ize data and accelerate training speed in image denoising. Method3
has exceeded Method2 in terms of PSNR as given in Table 1, which
shows BN is effective for PB in image denoising. Method 3 stands for
Method2 without BN in the PB. More detailed information on a parallel
block is as illustrated in Section 3.3. To prevent the over-enhancement
phenomenon, a 6-layer serial block is presented.
Serial block: According to Tian et al. (2020), we can see that
stacked convolutional layers can be used to refine features. Motivated
by that, we design a 6-layer serial block to remove redundant fea-
tures in image denoising. Taking a long-term dependency of a deep
network into consideration, we use residual dense operations into a
SB to enhance the abilities of shallow layers to deep layers. As shown
in Table 1, we can see that Method4 outperforms Method3, which
shows the superiority of residual dense operations in a SB for image
denoising. Method4 is Method3 without residual dense operations in
the SB. To further improve the effect of image denoising, we enlarge
the difference of architecture via stacked six BN+ReLU+Conv in the
SB. In Table 1, we can see that Method 4 has a higher PSNR than that
of Method6, which verifies the validity of stacked six BN+ReLU+Conv
for image denoising. The effectiveness of BN is tested via Method4
and Method5 in Table 1. Method5 is defined as Method4 without BN.
Method6 is symboled as Method5 without SB. Besides, we design a
comparative experiment to compare denoising performance between
the proposed stacked BN+ReLU+Conv and stacked Conv+BN+ReLU
in Table 1, where stacked BN+ReLU+Conv is more competitive for
image denoising. Finally, we compare Method4 and Method6 in terms
of PSNR to test the validity of the combination of parallel and serial
ways in the proposed PSDNet for image denoising as listed in Table 1.
Reconstruction block is only used to reconstruct clean images. Thus, we
do not test its effectiveness in image denoising.
4.4. Comparisons with the state-of-the-art methods
We compare denoising performance of PSDNet via 25 denoising
methods as comparative methods, i.e., WNNM (Gu et al., 2014), CSF
(Schmidt & Roth, 2014), BM3D (Dabov et al., 2007), DnCNN (Zhang,
Zuo, Chen et al., 2017), memory network (MemNet) (Tai et al., 2017),
image restoration CNN (IRCNN) (Zhang, Zuo, Gu et al., 2017), en-
hanced CNN denoising network (ECNDNet) (Tian et al., 2019), ex-
pected patch log likelihood (EPLL) (Zoran & Weiss, 2011), trainable
nonlinear reaction diffusion (TNRD) (Chen & Pock, 2016), multi-layer
perceptron (MLP) (Burger et al., 2012), non-local ensemble denoiser
(NLED) (Yang et al., 2020), two-stream learning-based compressive
sensing network (TSLCSNet) (Lee, Ku, Kim, & Ko, 2021), residual
encoder–decoder network (RED30) (Mao, Shen, & Yang, 2016), im-
age denoising CNN (IDCNN) (Ofir & Keller, 2021), fast and flexible
denoising convolutional neural network (FFDNet) (Zhang, Zuo et al.,
2018), attention-guided denoising convolutional neural network (AD-
Net) (Tian, Xu, Li et al., 2020), image denoising based on symmetric
generative adversarial network (ID-AGAN) (Wang et al., 2021), learned
K-SVD algorithm (LKSVD) (Scetbon et al., 2021), median filter aided
CNN (MFAC) (Dey et al., 2021), noise fusion convolutional neural
network (NFCNN) (Xu & Xie, 2022), guided intra-patch smoothing
adaptive weighted graph filtering (AWGF-GPS) (Tang et al., 2021), and
denoising blind-spot network (D-BSN) (Wu et al., 2020), hybrid denois-
ing CNN (HDCNN) (Zheng et al., 2022), Residual non-local attention
network (RNAN) (Zhang et al., 2019), Batch-renormalization denoising
network (BRDNet) (Tian, Xu & Zuo, 2020) for both synthetic noisy
image and real image denoising. To guarantee fairness of testing our
proposed PSDNet, we choose use same noise levels (i.e., 𝜎=15,25 and
50) and index, i.e., PSNR on public datasets (BSD68, Set12, CBSD68

5
Q. Zhang et al. Expert Systems With Applications 231 (2023) 120628

Table 3
Average PSNR (dB) results of different methods on Set12 with noise levels of 15, 25, and 50. The best and second denoising results are marked by red and blue lines, respectively.
Images C.man House Peppers Starfish Monarch Airplane Parrot Lena Barbara Boat Man Couple Average
Noise level 15
BM3D (Dabov et al., 2007) 31.91 34.93 32.69 31.14 31.85 31.07 31.37 34.26 33.10 32.13 31.92 32.10 32.37
WNNM (Gu et al., 2014) 32.17 35.13 32.99 31.82 32.71 31.39 31.62 34.27 33.60 32.27 32.11 32.17 32.70
EPLL (Zoran & Weiss, 2011) 31.85 34.17 32.64 31.13 32.10 31.19 31.42 33.92 31.38 31.93 32.00 31.93 32.14
CSF (Schmidt & Roth, 2014) 31.95 34.39 32.85 31.55 32.33 31.33 31.37 34.06 31.92 32.01 32.08 31.98 32.32
TNRD (Chen & Pock, 2016) 32.19 34.53 33.04 31.75 32.56 31.46 31.63 34.24 32.13 32.14 32.23 32.11 32.50
DnCNN (Zhang, Zuo, Chen et al., 2017) 32.61 34.97 33.30 32.20 33.09 31.70 31.83 34.62 32.64 32.42 32.46 32.47 32.86
IRCNN (Zhang, Zuo, Gu et al., 2017) 32.55 34.89 33.31 32.02 32.82 31.70 31.84 34.53 32.43 32.34 32.40 32.40 32.77
FFDNet (Zhang, Zuo & Zhang, 2018) 32.43 35.07 33.25 31.99 32.66 31.57 31.81 34.62 32.54 32.38 32.41 32.46 32.77
ECNDNet (Tian et al., 2019) 32.56 34.97 33.25 32.17 33.11 31.70 31.82 34.52 32.41 32.37 32.39 32.39 32.81
ID-AGAN (Wang, Chang, & Zhao, 2021) 32.21 34.99 33.03 32.28 32.75 31.62 31.61 34.63 32.23 32.34 32.42 32.42 32.71
LKSVD (Scetbon, Elad, & Milanfar, 2021) 32.16 34.59 32.92 31.78 32.78 31.54 31.66 34.24 32.22 32.18 32.22 32.11 32.53
MFAC (Dey, Bhattacharya, Schwenker, & Sarkar, 2021) 32.60 34.78 33.27 32.00 32.81 31.71 31.84 34.35 32.13 32.32 32.34 32.30 32.70
NFCNN (Xu & Xie, 2022) – – – – – – – – – – – – 32.88
NLED (Yang, Xu, Quan, & Ji, 2020) 32.28 34.76 33.10 31.75 32.71 31.59 31.70 34.35 32.53 32.16 32.22 32.13 32.61
HDCNN (Zheng, Zhi, Zeng, Tian, & You, 2022) 32.51 35.17 33.22 32.23 33.20 31.69 31.86 34.57 32.60 32.39 32.36 32.46 32.86
PSDNet 32.64 35.04 33.30 32.23 33.20 31.67 31.95 34.57 32.69 32.43 32.40 32.48 32.89
PSDNet-B 32.12 34.94 33.13 32.05 32.96 31.55 31.66 34.51 32.27 32.32 32.29 32.38 32.68
Noise level 25
BM3D (Dabov et al., 2007) 29.45 32.85 30.16 28.56 29.25 28.42 28.93 32.07 30.71 29.90 29.61 29.71 29.97
WNNM (Gu et al., 2014) 29.64 33.22 30.42 29.03 29.84 28.69 29.15 32.24 31.24 30.03 29.76 29.82 30.26
EPLL (Zoran & Weiss, 2011) 29.26 32.17 30.17 28.51 29.39 28.61 28.95 31.73 28.61 29.74 29.66 29.53 29.69
CSF (Schmidt & Roth, 2014) 29.48 32.39 30.32 28.80 29.62 28.72 28.90 31.79 29.03 29.76 29.71 29.53 29.84
TNRD (Chen & Pock, 2016) 29.72 32.53 30.57 29.02 29.85 28.88 29.18 32.00 29.41 29.91 29.87 29.71 30.06
DnCNN (Zhang, Zuo, Chen et al., 2017) 30.18 33.06 30.87 29.41 30.28 29.13 29.43 32.44 30.00 30.21 30.10 30.12 30.43
IRCNN (Zhang, Zuo, Gu et al., 2017) 30.08 33.06 30.88 29.27 30.09 29.12 29.47 32.43 29.92 30.17 30.04 30.08 30.38
FFDNet (Zhang, Zuo et al., 2018) 30.10 33.28 30.93 29.32 30.08 29.04 29.44 32.57 30.01 30.25 30.11 30.20 30.44
ECNDNet (Tian et al., 2019) 30.11 33.08 30.85 29.43 30.30 29.07 29.38 32.38 29.84 30.14 30.03 30.03 30.39
ID-AGAN (Wang et al., 2021) 29.92 33.23 30.71 29.85 30.14 29.07 29.14 32.61 29.77 30.22 30.12 30.10 30.41
LKSVD (Scetbon et al., 2021) 29.70 32.53 30.35 28.99 30.15 28.92 29.13 31.99 29.36 29.95 29.85 29.71 30.05
MFAC (Dey et al., 2021) 30.15 32.80 30.80 29.31 29.94 29.09 29.23 32.17 29.45 30.10 29.98 30.00 30.25
NFCNN (Xu & Xie, 2022) – – – – – – – – – – – – 30.42
NLED (Yang et al., 2020) 29.75 32.81 30.66 29.09 30.03 28.99 29.29 32.18 30.11 29.90 29.86 29.74 30.20
HDCNN (Zheng et al., 2022) 30.03 33.28 30.75 29.42 30.37 29.11 29.43 32.53 30.03 30.23 30.01 30.14 30.44
PSDNet 30.22 33.13 30.84 29.45 30.41 29.09 29.54 32.45 30.12 30.21 30.06 30.12 30.47
PSDNet-B 29.97 33.08 30.74 29.32 30.28 29.00 29.35 32.34 29.77 30.14 29.98 30.04 30.33
BM3D (Dabov et al., 2007) 26.13 29.69 26.68 25.04 25.82 25.10 25.90 29.05 27.22 26.78 26.81 26.46 26.72
WNNM (Gu et al., 2014) 26.45 30.33 26.95 25.44 26.32 25.42 26.14 29.25 27.79 26.97 26.94 26.64 27.05
EPLL (Zoran & Weiss, 2011) 26.10 29.12 26.80 25.12 25.94 25.31 25.95 28.68 24.83 26.74 26.79 26.30 26.47
MLP (Burger, Schuler, & Harmeling, 2012) 26.37 29.64 26.68 25.43 26.26 25.56 26.12 29.32 25.24 27.03 27.06 26.67 26.78
TNRD (Chen & Pock, 2016) 26.62 29.48 27.10 25.42 26.31 25.59 26.16 28.93 25.70 26.94 26.98 26.50 26.81
DnCNN (Zhang, Zuo, Chen et al., 2017) 27.03 30.00 27.32 25.70 26.78 25.87 26.48 29.39 26.22 27.20 27.24 26.90 27.18
IRCNN (Zhang, Zuo, Gu et al., 2017) 26.88 29.96 27.33 25.57 26.61 25.89 26.55 29.40 26.24 27.17 27.17 26.88 27.14
FFDNet (Zhang, Zuo et al., 2018) 27.05 30.37 27.54 25.75 26.81 25.89 26.57 29.66 26.45 27.33 27.29 27.08 27.32
ECNDNet (Tian et al., 2019) 27.07 30.12 27.30 25.72 26.82 25.79 26.32 29.29 26.26 27.16 27.11 26.84 27.15
ID-AGAN (Wang et al., 2021) 27.07 30.57 27.48 26.43 26.90 25.95 26.36 29.62 26.54 27.35 27.33 27.04 27.39
LKSVD (Scetbon et al., 2021) 26.68 29.37 26.96 25.38 26.54 25.62 25.99 28.85 25.73 26.99 26.95 26.55 26.80
MFAC (Dey et al., 2021) 27.17 29.80 27.30 25.56 26.60 25.78 26.26 29.11 25.73 27.12 27.09 26.81 27.03
AWGF-GPS (Tang, Sun, Jiang, Chen, & Zhou, 2021) 26.53 30.45 27.06 25.51 26.45 25.55 26.19 29.28 27.69 27.00 27.00 26.68 27.12
HDCNN (Zheng et al., 2022) 27.20 30.04 27.47 25.73 26.89 25.82 26.29 29.50 26.14 27.16 27.23 26.93 27.20
PSDNet 27.17 30.24 27.36 25.73 26.84 25.89 26.45 29.33 26.44 27.27 27.22 26.92 27.24
PSDNet-B 27.14 30.09 27.29 25.66 26.78 25.82 26.46 29.23 26.34 27.20 27.18 26.83 27.17

Table 4 Table 5
PSNR (dB) results of different methods on CBSD68 dataset with noise levels of 15, 25, SSIM results of different methods on BSD68 and Set12 datasets with noise levels of 15,
35, 50, and 75. The best and second denoising results are marked by red and blue 25, and 50. The best and second denoising results are marked by red and blue lines,
lines, respectively. respectively.
Methods 𝜎=15 𝜎=25 𝜎=35 𝜎=50 𝜎=75 Datasets BSD68 Set12
CBM3D (Dabov et al., 2007) 33.52 30.71 28.89 27.38 25.74 Methods 𝜎=15 𝜎=25 𝜎=50 𝜎=15 𝜎=25 𝜎=50
DnCNN (Zhang, Zuo, Chen et al., 2017) 33.98 31.31 29.65 28.01 –
BM3D (Dabov et al., 2007) 0.872 0.801 0.686 0.895 0.850 0.768
IRCNN (Zhang, Zuo, Gu et al., 2017) 33.86 31.16 29.50 27.86 –
WNNM (Gu et al., 2014) 0.877 0.809 0.698 0.898 0.856 0.778
FFDNet (Zhang, Zuo et al., 2018) 33.80 31.18 29.57 27.96 26.24
DnCNN (Zhang, Zuo, Chen et al., 2017) 0.891 0.828 0.718 0.902 0.862 0.783
ADNet (Tian et al., 2020) 33.99 31.31 29.66 28.04 26.33
IRCNN (Zhang, Zuo, Gu et al., 2017) 0.888 0.825 0.717 0.901 0.860 0.780
NFCNN (Xu & Xie, 2022) 33.91 31.24 – 27.94 26.41
FFDNet (Zhang, Zuo et al., 2018) 0.890 0.829 0.724 0.902 0.863 0.790
D-BSN (Wu, Liu, Cao, Ren, & Zuo, 2020) 33.56 30.61 – 27.41 –
ADNet (Tian, Xu, Li et al., 2020) 0.892 0.829 0.722 0.905 0.865 0.791
PSDNet 34.00 31.37 29.71 28.02 26.38
PSDNet 0.897 0.835 0.722 0.906 0.864 0.784
PSDNet-B 34.01 31.35 29.70 28.05 –

and CC) as popular methods, i.e., DnCNN (Zhang, Zuo, Chen et al., and a random seed for training process. noise level is standard devi-
2017) and ADNet (Tian, Xu, Li et al., 2020) to conduct comparative ation, i.e., 𝜎. Also, a seed is fixed, when these denoising models are
experiments, where noise is generated by Numpy (Harris et al., 2020) tested.

6
Q. Zhang et al. Expert Systems With Applications 231 (2023) 120628

Table 6
SSIM results of different methods on CBSD68 dataset with noise levels of 15, 25, and 50. The best and second denoising results are marked by red and blue lines, respectively.
Methods CBM3D (Dabov et al., DnCNN (Zhang, Zuo, IRCNN (Zhang, Zuo, Gu FFDNet (Zhang, Zuo RNAN (Zhang, Li, Li, BRDNet (Tian, Xu & PSDNet
2007) Chen et al., 2017) et al., 2017) et al., 2018) Zhong, & Fu, 2019) Zuo, 2020)
𝜎=15 0.9215 0.9290 0.9285 0.9290 – 0.9291 0.9326
𝜎=25 0.8672 0.8830 0.8824 0.8821 – 0.8847 0.8892
𝜎=50 0.7626 0.7896 0.7898 0.7887 0.8018 0.7950 0.7950

Table 7 model has achieved the highest structural similarity (SSIM) (Hore &
PSNR (dB) results of different methods on CC dataset. The best and second denoising Ziou, 2010) performance at all noise levels of 15, 25, and 50. Also, our
results are marked by red and blue lines, respectively.
method has an improvement of 0.006 in terms of SSIM for noise level
Camera settings DnCNN GAT-BM3D TID PSDNet
of 25 than that of the second ADNet on BSD68 for image denoising in
Canon 5D ISO = 3200_1 37.26 31.23 37.22 37.26 Table 5. Thus, it is competitive for gray and color synthetic noisy image
Canon 5D ISO = 3200_2 37.26 30.55 34.54 34.63
denoising.
Canon 5D ISO = 3200_3 34.09 27.74 34.25 33.86
For real noisy images, we choose DnCNN (Zhang, Zuo, Chen et al.,
Nikon D600 ISO = 3200_1 33.62 28.55 32.99 33.85
2017), GAT-BM3D (Makitalo & Foi, 2012), and TID (Luo, Chan, &
Nikon D600 ISO = 3200_2 34.48 32.01 32.99 34.81
Nikon D600 ISO = 3200_3 35.41 39.78 35.58 35.60
Nguyen, 2015) as comparative methods on CC to compare denoising
performance with the proposed PSDNet. As shown in Table 7, our
Nikon D800 ISO = 1600_1 37.95 32.24 34.49 36.83
Nikon D800 ISO = 1600_2 36.08 33.86 34.49 36.93
PSDNet has the highest average PSNR on different ISOs. For instance,
Nikon D800 ISO = 1600_3 35.48 33.90 35.26 36.56 our PSDNet has an improvement of 1.42 dB than that of GAT-BM3D
for Nikon D800 ISO = 3200_1 in Table 7. Because denoising time
Nikon D800 ISO = 3200_1 34.08 36.49 33.70 35.07
Nikon D800 ISO = 3200_2 33.70 32.91 31.04 34.15 and complexity are also important indexes for real digital devices, we
Nikon D800 ISO = 3200_3 33.31 40.20 33.07 34.50 conduct experiments via different methods to test running time and
Nikon D800 ISO = 6400_1 29.83 29.84 29.40 30.32 complexity. That is, we choose four denoising methods with noisy
Nikon D800 ISO = 6400_2 30.55 27.94 29.40 30.78 images of different sizes to test running time.s As shown in Table 8, our
Nikon D800 ISO = 6400_3 30.09 29.15 29.21 30.12 PSDNet has obtained the fastest denoising time for noisy images with
Average 33.86 32.43 33.36 34.35 256 × 256 and 512 × 512. As shown in Table 9, we can see that our
PSDNet has obtained the second least parameters and the first Flops,
which shows its superiority in complexity. Also, the best and second
Table 8 denoising results are marked by red and blue lines, respectively. Ac-
Running time of popular methods for noisy images of two sizes: 256 × 256, 512 × 512.
cording to mentioned illustrations, we can see that PSDNet is effective
The best and second denoising results are marked by red and blue lines, respectively.
in image denoising in terms of quantitative analysis. Tables 1–9 reveal
Methods Device 256×256 512×512
that the red line is used to represent the best denoising effect, the blue
DIDN (Yu et al., 2019) GPU 1.258 0.3290
line is used to express the second denoising effect.
MLEFGN (Fang, Li, Yuan, Zeng, & Zhang, 2020) GPU 0.0574 0.2076
SRMNet (Fan, Liu, Liu, & Chiu, 2022) GPU 0.0387 0.1436
To evaluate the abilities to recover texture information from our
MPRNet (Zamir et al., 2021) GPU 0.1047 0.2610 PSDNet, we choose BM3D, IRCNN, ADNet, DnCNN, FFDNet, and PS-
PSDNet GPU 0.0130 0.0517 DNet on BSD68 for a noise level of 50, Set12 for a noise level of 25,
Kodak24 (Franzen, 1999) for the noise level of 35 and CBSD68 for the
noise level of 15 to conduct visual experiments. That is, we choose
Table 9
an area of recovered images of different methods to amplify it as an
Complexities of different denoising networks.
observation area. The observation area is clearer, its recovered detailed
Methods Parameters(M) GFLOPs
information is better, which shows the corresponding denoising method
RNAN (Zhang et al., 2019) 8.96 12.1 is more effective. From Figs. 2 and 3, we can see that our PSDNet
VDN (Yue, Yong, Zhao, Meng, & Zhang, 2019) 7.82 2.03
MPRNet (Zamir et al., 2021) 15.73 34.02
has obtained clearer observation areas, which illustrates our PSDNet
DIDN (Yu et al., 2019) 164.56 27.38 is more effective than other popular denoising methods, i.e., DnCNN
RDUNet (Gurrola-Ramos et al., 2021) 166.37 35.64 for gray noisy image denoising. From Figs. 4 and 5, we can see that
DHDN (Park et al., 2019) 168.19 49.79 our PSDNet has clearer observation areas, which shows our PSDNet
PSDNet 1.08 1.72 has better denoising performance than these other popular denoising
methods, i.e., FFDNet for color noisy image denoising. According to
mentioned presentations, it is known that our PSDNet has merits in
qualitative analysis for image denoising. In a summary, the proposed
Synthetic noisy images include gray and color synthetic noisy im-
PSDNet is suitable for image denoising, according to quantitative and
ages. For gray synthetic noisy image denoising, noise levels of 15, 25,
qualitative analysis.
and 50 are used to conduct gray synthetic noisy images. For color
Recovering more detailed information is very important for image
synthetic noisy image denoising, noise levels of 15, 25, 35, 50, and
denoising (Tian, Xu, Li et al., 2020). Little improvement is very helpful
75 are used to conduct color synthetic noisy images. Additionally, we image denoising (Tian et al., 2022). Also, the proposed method has
train a blind denoising method as PSDNet-B via noise levels from 0 to obtained competitive results in terms of PSNR for gray, color and
55. real noisy image denoising. Stable performance is very beneficial for
As shown in Tables 2 and 3, we can see that the proposed PSDNet real applications. Besides, heterogeneous architecture including simple
has obtained the best gray synthetic image denoising performance for components can achieve good denoising performance by parallel and
noise levels of 15 and 25. Our PSDNet has an improvement of 0.04 dB serial ways achieve a lightweight CNN, which also obtains faster de-
for noise levels of 15 and 25 for image denoising than that of the second noising time and smaller complexity, which is very helpful for mobile
denoising method in Table 2. Also, it has an improvement of 0.07 dB for phones and cameras with less memory. Thus, our proposed PSDNet is
noise level of 50 on Set12 for image denoising than that of the second suitable to be deployed on mobile devices, i.e., phones and cameras
denoising method in Table 3. As shown in Table 4, we can see that the in real world. Our model is trained by a supervised manner, which
proposed method has obtained the best denoising performance for noise still suffers from challenges of unknown noisy images captured in
levels of 15, 25, and 35. And it obtained the second denoising result for complex scenes. Thus, we will propose unsupervised denoising methods
the noise level of 75. Moreover, Tables 5 and 6 reveal that our PSDNet to address the mentioned problem in the future.

7
Q. Zhang et al. Expert Systems With Applications 231 (2023) 120628

Fig. 2. Denoising results of different methods on one image from BSD68 when noise level 50. (a) Original image, (b) Noisy image/14.16 dB, (c) BM3D (Dabov et al., 2007)/23.45 dB,
(d) HDCNN (Zheng et al., 2022)/24.10 dB, (e) ECNDNet (Tian et al., 2019) /24.50 dB, (f) ADNet (Tian, Xu, Li et al., 2020)/24.51 dB, (g) DnCNN (Zhang, Zuo, Chen et al.,
2017)/24.52 dB, and (h) PSDNet/24.69 dB.

Fig. 3. Denoising results of different methods on one image from Set12 when noise level is 25. (a) Original image, (b) Noisy image/20.17 dB, (c) BM3D (Dabov et al.,
2007)/29.25 dB, (d) DnCNN (Zhang, Zuo, Chen et al., 2017)/30.28 dB, (e) ECNDNet (Tian et al., 2019)/30.30 dB, (f) HDCNN (Zheng et al., 2022)/30.37 dB, (g) ADNet (Tian,
Xu, Li et al., 2020)/30.39 dB and (h) PSDNet/30.41 dB.

Fig. 4. Denoising results of different methods on one image from Kodak24 when noise level 35. (a) Original image, (b) Noisy image/17.25 dB, (c) CBM3D (Dabov et al.,
2007)/26.78 dB, (d) IRCNN (Zhang, Zuo, Gu et al., 2017)/27.67 dB, (e) FFDNet (Zhang, Zuo et al., 2018)/27.69 dB, (f) DnCNN (Zhang, Zuo, Chen et al., 2017)/27.72 dB, (g)
ADNet (Tian, Xu, Li et al., 2020)/27.77 dB and (h) PSDNet/27.81 dB.

5. Conclusion in a parallel way are designed to improve performance of image de-

noising. Taking into target bias issue from convolutional operations,
In this paper, we propose a parallel and serial denoising network a deformable convolution is used into a sub-network. That can not
as well as PSDNet for image denoising. To extract complementary only address target bias issue, but also improve denoisng effect via
information, two heterogeneous sub-networks with simple components enlarging difference of the network. To prevent over-enhancement

8
Q. Zhang et al.
Fig. 5. Denoising results of different methods on one image from CBSD68 when noise level is 15. (a) Original image, (b) Noisy image/24.62 dB, (c) CBM3D (Dabov et al.,
2007)/33.07 dB, (d) DnCNN (Zhang, Zuo, Chen et al., 2017)/33.69 dB, (e) FFDNet (Zhang, Zuo et al., 2018)/33.72 dB, (f) IRCNN (Zhang, Zuo, Gu et al., 2017)/33.82 dB, (g)
ADNet (Tian, Xu, Li et al., 2020)/33.96 dB and (h) PSDNet/34.04 dB.
and long-term dependency phenomenon, a novel architecture based
a residual dense idea is designed to remove redundant features in a
serial way. Additionally, a combination of parallel and serial ways can
combine wide and deep features to facilitate robust noisy features for
image denoising, which can make the proposed PSDNet effective for
synthetic and real noisy image denoising. The proposed PSDNet has
fast denoising time, which is suitable to mobile digital device. We will
design a CNN for complex low-level vision task, i.e., low-resolution
image blind denoising in the future.
CRediT authorship contribution statement
Qi Zhang: Wrote this manuscript, Offered key ideas, Conducted
some experiments. Jingyu Xiao: Conducted some experiments. Chun-
wei Tian: Revised the manuscript, Provided valuable comments for this
manuscript. Jiayu Xu: Revised the experiment part of this manuscript.
Shichao Zhang: Provided the partial experimental analysis. Chia-Wen
Lin: Revised the manuscript, Offered valuable comments.
Declaration of competing interest
The authors declare that they have no known competing finan-
cial interests or personal relationships that could have appeared to
influence the work reported in this paper.
Data availability
Data will be made available on request.
Acknowledgments
This work was supported in part by the Guangdong Basic and
Applied Basic Research Foundation under Grant 2021A1515110079, in
part by Central High School Young Teachers Development Fund under
Grant IDGA10002145 in part by National Natural Science Foundation
of China under Grant 62201468 and in part by China Postdoctoral Sci-
ence Foundation under Grant 2022TQ0259 and 2022M72259, in part
by Shenzhen Municipal Science and Technology Innovation Council
under Grant JSGG20220831105002004.
9
Expert Systems With Applications 231 (2023) 120628
References
Bao, L., Yang, Z., Wang, S., Bai, D., & Lee, J. (2020). Real image denoising based
on multi-scale residual dense block and cascaded U-net with block-connection. In
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
workshops (pp. 448–449).
Buades, A., Coll, B., & Morel, J.-M. (2005). A review of image denoising algorithms,
with a new one. Multiscale Modeling and Simulation, 4(2), 490–530.
Burger, H. C., Schuler, C. J., & Harmeling, S. (2012). Image denoising: Can plain neural
networks compete with BM3D? In 2012 IEEE conference on computer vision and
pattern recognition (pp. 2392–2399). IEEE.
Chang, Y., Yan, L., Fang, H., Zhong, S., & Liao, W. (2018). HSI-DeNet: Hyperspectral
image restoration via convolutional neural network. IEEE Transactions on Geoscience
and Remote Sensing, 57(2), 667–682.
Chen, Y., & Pock, T. (2016). Trainable nonlinear reaction diffusion: A flexible frame-
work for fast and effective image restoration. IEEE Transactions on Pattern Analysis
and Machine Intelligence, 39(6), 1256–1272.
Chen, R., Pu, D., Tong, Y., & Wu, M. (2022). Image-denoising algorithm based on
improved K-singular value decomposition and atom optimization. CAAI Transactions
on Intelligence Technology, 7(1), 117–127.
Chen, Z., Zhou, Z., & Adnan, S. (2021). Joint low-rank prior and difference of Gaussian
filter for magnetic resonance image denoising. Medical & Biological Engineering &
Computing, 59(3), 607–620.
Cho, S. I., & Kang, S.-J. (2018). Gradient prior-aided CNN denoiser with separa-
ble convolution-based optimization of feature dimension. IEEE Transactions on
Multimedia, 21(2), 484–493.
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse
3-D transform-domain collaborative filtering. IEEE Transactions on Image Processing,
16(8), 2080–2095.
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., et al. (2017). Deformable
convolutional networks. In Proceedings of the IEEE international conference on
computer vision (pp. 764–773).
Dey, S., Bhattacharya, R., Schwenker, F., & Sarkar, R. (2021). Median filter aided CNN
based image denoising: An ensemble aprroach. Algorithms, 14(4), 109.
Ephraim, Y., & Malah, D. (1984). Speech enhancement using a minimum-mean square
error short-time spectral amplitude estimator. IEEE Transactions on Acoustics, Speech
and Signal Processing, 32(6), 1109–1121.
Fan, C.-M., Liu, T.-J., Liu, K.-H., & Chiu, C.-H. (2022). Selective residual M-net for real
image denoising. In 2022 30th European signal processing conference (pp. 469–473).
IEEE.
Fang, F., Li, J., Yuan, Y., Zeng, T., & Zhang, G. (2020). Multilevel edge features guided
network for image denoising. IEEE Transactions on Neural Networks and Learning
Systems, 32(9), 3956–3970.
Franzen, R. (1999). Kodak24. URL http://www.r0k.us/graphics/kodak/.
Gai, S., & Bao, Z. (2019). New image denoising algorithm via improved deep
convolutional neural network with perceptive loss. Expert Systems with Applications,
138, Article 112815.
Gu, S., Zhang, L., Zuo, W., & Feng, X. (2014). Weighted nuclear norm minimization with
application to image denoising. In Proceedings of the IEEE conference on computer
vision and pattern recognition (pp. 2862–2869).
Gurrola-Ramos, J., Dalmau, O., & Alarcón, T. E. (2021). A residual dense U-net neural
network for image denoising. IEEE Access, 9, 31742–31754.
Harris, C. R., Millman, K. J., Van Der Walt, S. J., Gommers, R., Virtanen, P.,
Cournapeau, D., et al. (2020). Array programming with numpy. Nature, 585(7825),
357–362.
Q. Zhang et al.
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image
recognition. In 2016 IEEE conference on computer vision and pattern recognition (pp.
770–778). http://dx.doi.org/10.1109/CVPR.2016.90.
Healey, G. E., & Kondepudy, R. (1994). Radiometric CCD camera calibration and noise
estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(3),
267–276.
Herbreteau, S., & Kervrann, C. (2022). DCT2net: an interpretable shallow CNN for
image denoising. IEEE Transactions on Image Processing, 31, 4292–4305.
Hore, A., & Ziou, D. (2010). Image quality metrics: PSNR vs. SSIM. In 2010 20th
international conference on pattern recognition (pp. 2366–2369). IEEE.
Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training
by reducing internal covariate shift. In International conference on machine learning
(pp. 448–456). PMLR.
Jafarbigloo, S. K., & Danyali, H. (2021). Nuclear atypia grading in breast cancer
histopathological images based on CNN feature extraction and LSTM classification.
CAAI Transactions on Intelligence Technology, 6(4), 426–439.
Jain, V., & Seung, S. (2008). Natural image denoising with convolutional networks.
Advances in Neural Information Processing Systems, 21.
Jia, X., Liu, S., Feng, X., & Zhang, L. (2019). Focnet: A fractional optimal control
network for image denoising. In Proceedings of the IEEE/CVF conference on computer
vision and pattern recognition (pp. 6054–6063).
Jin, Y., Yu, L., Li, G., & Fei, S. (2021). A 6-DOFs event-based camera relocalization
system by CNN-LSTM and image denoising. Expert Systems with Applications, 170,
Article 114535.
Jonides, J., Lewis, R. L., Nee, D. E., Lustig, C. A., Berman, M. G., & Moore, K. S.
(2008). The mind and brain of short-term memory. Annual Review of Psychology,
59, 193–224.
Kim, D.-W., Ryun Chung, J., & Jung, S.-W. (2019). Grdn: Grouped residual dense
network for real image denoising and gan-based real-world noise modeling. In
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
workshops.
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv
preprint arXiv:1412.6980.
Lee, B., Ku, B., Kim, W., & Ko, H. (2021). Two-stream learning-based compressive
sensing network with high-frequency compensation for effective image denoising.
IEEE Access, 9, 91974–91982.
Li, Z., & Wu, J. (2019). Learning deep cnn denoiser priors for depth image inpainting.
Applied Sciences, 9(6), 1103.
Liu, P., Zhang, H., Zhang, K., Lin, L., & Zuo, W. (2018). Multi-level wavelet-CNN for
image restoration. In Proceedings of the IEEE conference on computer vision and pattern
recognition workshops (pp. 773–782).
Lucas, A., Iliadis, M., Molina, R., & Katsaggelos, A. K. (2018). Using deep neural
networks for inverse problems in imaging: beyond analytical methods. IEEE Signal
Processing Magazine, 35(1), 20–36.
Luo, E., Chan, S. H., & Nguyen, T. Q. (2015). Adaptive image denoising by targeted
databases. IEEE Transactions on Image Processing, 24(7), 2167–2181.
Makitalo, M., & Foi, A. (2012). Optimal inversion of the generalized anscombe
transformation for Poisson-Gaussian noise. IEEE Transactions on Image Processing,
22(1), 91–103.
Malfait, M., & Roose, D. (1997). Wavelet-based image denoising using a Markov random
field a priori model. IEEE Transactions on Image Processing, 6(4), 549–565.
Mao, X., Shen, C., & Yang, Y.-B. (2016). Image restoration using very deep convo-
lutional encoder-decoder networks with symmetric skip connections. Advances in
Neural Information Processing Systems, 29.
Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A database of human segmented
natural images and its application to evaluating segmentation algorithms and
measuring ecological statistics. Vol. 2, In Proceedings eighth IEEE international
conference on computer vision. ICCV 2001 (pp. 416–423). IEEE.
Nam, S., Hwang, Y., Matsushita, Y., & Kim, S. J. (2016). A holistic approach to cross-
channel image noise modeling and its application to image denoising. In Proceedings
of the IEEE conference on computer vision and pattern recognition (pp. 1683–1691).
Ofir, N., & Keller, Y. (2021). Multi-scale processing of noisy images using edge
preservation losses. In 2020 25th international conference on pattern recognition (pp.
1–8). IEEE.
Panda, A., Naskar, R., & Pal, S. (2018). Exponential linear unit dilated residual network
for digital image denoising. Journal of Electronic Imaging, 27(5), Article 053024.
Park, B., Yu, S., & Jeong, J. (2019). Densely connected hierarchical network for image
denoising. In Proceedings of the IEEE/CVF conference on computer vision and pattern
recognition workshops.
Pytorch, A. D. I. (2018). Pytorch.
Rajni, R., & Anutam, A. (2014). Image denoising techniques-an overview. International
Journal of Computer Applications, 86(16), 13–17.
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks
for biomedical image segmentation. In International conference on medical image
computing and computer-assisted intervention (pp. 234–241). Springer.
Roth, S., & Black, M. J. (2005). Fields of experts: A framework for learning image
priors. Vol. 2, In 2005 IEEE computer society conference on computer vision and pattern
recognition (CVPR’05) (pp. 860–867). IEEE.
Scetbon, M., Elad, M., & Milanfar, P. (2021). Deep k-svd denoising. IEEE Transactions
on Image Processing, 30, 5944–5955.
10
Expert Systems With Applications 231 (2023) 120628
Schmidt, U., & Roth, S. (2014). Shrinkage fields for effective image restoration. In
Proceedings of the IEEE conference on computer vision and pattern recognition (pp.
2774–2781).
Shim, G., Park, J., & Kweon, I. S. (2020). Robust reference-based super-resolution with
similarity-aware deformable convolution. In Proceedings of the IEEE/CVF conference
on computer vision and pattern recognition (pp. 8425–8434).
Sun, X., Kottayil, N. K., Mukherjee, S., & Cheng, I. (2018). Adversarial training for dual-
stage image denoising enhanced with feature matching. In International conference
on smart multimedia (pp. 357–366). Springer.
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., et al. (2015). Going
deeper with convolutions. In Proceedings of the IEEE conference on computer vision
and pattern recognition (pp. 1–9).
Tai, Y., Yang, J., Liu, X., & Xu, C. (2017). Memnet: A persistent memory network for
image restoration. In Proceedings of the IEEE international conference on computer
vision (pp. 4539–4547).
Tang, Y., Sun, J., Jiang, A., Chen, Y., & Zhou, L. (2021). Adaptive graph filtering
with intra-patch pixel smoothing for image denoising. Circuits, Systems, and Signal
Processing, 40(11), 5381–5400.
Thomas, H., Qi, C. R., Deschaud, J.-E., Marcotegui, B., Goulette, F., & Guibas, L.
J. (2019). Kpconv: Flexible and deformable convolution for point clouds. In
Proceedings of the IEEE/CVF international conference on computer vision (pp.
6411–6420).
Tian, C., Xu, Y., Fei, L., Wang, J., Wen, J., & Luo, N. (2019). Enhanced CNN for image
denoising. CAAI Transactions on Intelligence Technology, 4(1), 17–23.
Tian, C., Xu, Y., Li, Z., Zuo, W., Fei, L., & Liu, H. (2020). Attention-guided CNN for
image denoising. Neural Networks, 124, 117–129.
Tian, C., Xu, Y., & Zuo, W. (2020). Image denoising using deep CNN with batch
renormalization. Neural Networks, 121, 461–473.
Tian, C., Xu, Y., Zuo, W., Du, B., Lin, C.-W., & Zhang, D. (2021). Designing and training
of a dual CNN for image denoising. Knowledge-Based Systems, 226, Article 106949.
Tian, C., Xu, Y., Zuo, W., Zhang, B., Fei, L., & Lin, C.-W. (2020). Coarse-to-fine CNN
for image super-resolution. IEEE Transactions on Multimedia, 23, 1489–1502.
Tian, C., Zheng, M., Zuo, W., Zhang, B., Zhang, Y., & Zhang, D. (2022). Multi-stage
image denoising with the wavelet transform. Pattern Recognition, Article 109050.
Wang, Y., Chang, D., & Zhao, Y. (2021). A new blind image denoising method based on
asymmetric generative adversarial network. IET Image Processing, 15(6), 1260–1272.
Wang, T., Qin, Z., & Zhu, M. (2017). An ELU network with total variation for image
denoising. In International conference on neural information processing (pp. 227–237).
Springer.
Wang, X.-Y., Yang, H.-Y., & Fu, Z.-K. (2010). A new wavelet-based image denoising
using undecimated discrete wavelet transform and least squares support vector
machine. Expert Systems with Applications, 37(10), 7040–7049.
Wu, C., Chen, X., Ji, D., & Zhan, S. (2018). Image denoising via residual network based
on perceptual 1oss. Journal of Image and Graphics, 23(10), 1483–1491.
Wu, X., Liu, M., Cao, Y., Ren, D., & Zuo, W. (2020). Unpaired learning of deep image
denoising. In European conference on computer vision (pp. 352–368). Springer.
Xu, J., Li, H., Liang, Z., Zhang, D., & Zhang, L. (2018). Real-world noisy image
denoising: A new benchmark. arXiv preprint arXiv:1804.02603.
Xu, M., & Xie, X. (2022). NFCNN: toward a noise fusion convolutional neural network
for image denoising. Signal, Image and Video Processing, 16(1), 175–183.
Yang, X., Xu, Y., Quan, Y., & Ji, H. (2020). Image denoising via sequential ensemble
learning. IEEE Transactions on Image Processing, 29, 5038–5049.
Yu, X., Fu, Z., & Ge, C. (2022). A multi-scale generative adversarial network for
real-world image denoising. Signal, Image and Video Processing, 16(1), 257–264.
Yu, S., Park, B., & Jeong, J. (2019). Deep iterative down-up cnn for image denoising.
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
workshops.
Yuan, Q., Zhang, Q., Li, J., Shen, H., & Zhang, L. (2018). Hyperspectral image denoising
employing a spatial–spectral deep residual convolutional neural network. IEEE
Transactions on Geoscience and Remote Sensing, 57(2), 1205–1218.
Yue, Z., Yong, H., Zhao, Q., Meng, D., & Zhang, L. (2019). Variational denoising
network: Toward blind noise modeling and removal. Advances in Neural Information
Processing Systems, 32.
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., et al. (2021).
Multi-stage progressive image restoration. In Proceedings of the IEEE/CVF conference
on computer vision and pattern recognition (pp. 14821–14831).
Zhang, C., & Kim, J. (2019). Object detection with location-aware deformable convo-
lution and backward attention filtering. In Proceedings of the IEEE/CVF conference
on computer vision and pattern recognition (pp. 9452–9461).
Zhang, Y., Li, K., Li, K., Zhong, B., & Fu, Y. (2019). Residual non-local attention
networks for image restoration. arXiv preprint arXiv:1903.10082.
Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. (2018). Residual dense network for
image super-resolution. In Proceedings of the IEEE conference on computer vision and
pattern recognition (pp. 2472–2481).
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017). Beyond a gaussian denoiser:
Residual learning of deep cnn for image denoising. IEEE Transactions on Image
Processing, 26(7), 3142–3155.
Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017). Learning deep CNN denoiser prior
for image restoration. In Proceedings of the IEEE conference on computer vision and
pattern recognition (pp. 3929–3938).
Q. Zhang et al.
Zhang, K., Zuo, W., & Zhang, L. (2018). Ffdnet: Toward a fast and flexible solution
for CNN-based image denoising. IEEE Transactions on Image Processing, 27(9),
4608–4622.
Zhao, W., Lv, Y., Liu, Q., & Qin, B. (2017). Detail-preserving image denoising via
adaptive clustering and progressive PCA thresholding. IEEE Access, 6, 6303–6315.
Zhao, H., Shao, W., Bao, B., & Li, H. (2019). A simple and robust deep convolutional
approach to blind image denoising. In Proceedings of the IEEE/CVF international
conference on computer vision workshops.
Zheng, M., Zhi, K., Zeng, J., Tian, C., & You, L. (2022). A hybrid CNN for image
denoising. Journal of Artificial Intelligence and Technology.
11
Expert Systems With Applications 231 (2023) 120628
ZhiPing, Q., YuanQi, Z., Yi, S., & XiangBo, L. (2018). A new generative adversarial
network for texture preserving image denoising. In 2018 eighth international
conference on image processing theory, tools and applications (pp. 1–5). IEEE.
Zhu, J., Fang, L., & Ghamisi, P. (2018). Deformable convolutional neural networks for
hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 15(8),
1254–1258.
Zoran, D., & Weiss, Y. (2011). From learning models of natural image patches to whole
image restoration. In 2011 international conference on computer vision (pp. 479–486).
IEEE.