8514 IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 17, NO.
12, DECEMBER 2021
Multivariate Air Quality Forecasting With Nested
Long Short Term Memory Neural Network
Ning Jin , Yongkang Zeng , Ke Yan , Member, IEEE, and Zhiwei Ji
Abstract—Artificial intelligence-based air quality index
(AQI) forecasting is a hot research topic in the fields of
sustainable and smart industrial environment design. There
are mainly two obstacles that hinder the existing machine
learning (ML) and deep learning (DL) technologies provid-
ing accurate forecasting results to protect the environment,
which include the intercorrelation between different AQI
components and the highly volatile AQI pattern changes.
In this article, a novel DL framework combining multiple
nested long short term memory networks (MTMC-NLSTM)
is proposed for accurate AQI forecasting enlightened with
the federated learning. The performance of the proposed
MTMC-NLSTM model is compared with conventional ML
models, DL methods, as well as hybrid DL models. The
experimental results show that the performance of the pro-
posed method is superior to those of all compared models.
Index Terms—Discrete stationary wavelet transform
(DSWT), multichannel neural network, multitask neural net-
work, nested long short term memory (NLSTM).
I. INTRODUCTION
IR quality control, as one of the most important and long-
A existed issues in the process of human civilization, has
received increasing attention globally in recent years [1]. Air pol-
lution not only destroys the ecological balance of nature, but also
affects people’s health, economy, and civilization developments
[2]. Air quality index (AQI), as one of the major indicators of the
air quality, is comprised of six major pollution components—
fine particulate matter (PM2.5 ), respirable particulate matter
(PM10 ), sulfur dioxide (SO2 ), nitrogen dioxide (NO2 ), carbon
monoxide (CO), and ozone (O3 ) [3]. The research and forecast-
ing techniques of AQI enable us to better prevent and control air
pollution issues. With the guidance of more precise AQI fore-
casting, more effective measures can be taken, including sending
Manuscript received November 27, 2020; revised February 27, 2021;
accepted March 9, 2021. Date of publication March 17, 2021; date of
current version August 20, 2021. Paper no. TII-20-5352. (Corresponding
author: Ke Yan.)
Ning Jin and Yongkang Zeng are with the Key Laboratory of Elec-
tromagnetic Wave Information Technology and Metrology of Zhe-
jiang Province, College of Information Engineering, China Jiliang Uni-
versity, Hangzhou 310018, China (e-mail: jinning1117@cjlu.edu.cn;
ykzeng899@gmail.com).
Ke Yan is with the National University of Singapore, Singapore
117566, Singapore (e-mail: yanke@cjlu.edu.cn).
Zhiwei Ji is with the School of Artificial Intelligence, Nanjing Agricul-
tural University, Nanjing 210095, China (e-mail: zhiwei.ji@njau.edu.cn).
Color versions of one or more figures in this article are available at
https://doi.org/10.1109/TII.2021.3065425.
Digital Object Identifier 10.1109/TII.2021.3065425
1551-3203 © 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Hyderabad IG Memorial Library. Downloaded on September 13,2022 at 04:36:42 UTC from IEEE Xplore. Restrictions apply.
travel warnings, providing self-protection suggestions, reorga-
nizing industrial production, and promoting relevant legislation.
Such guidance is vital to ensure social welfare and protect the
environment. However, the internal factors, including industrial
production, energy extraction, transportation, etc., make the AQI
change irregular, and difficult for forecasting techniques [4].
Data-driven time series forecasting methods, including deep
learning (DL) and big data analysis, provide crucial forecasting
results for many industrial areas [5]–[7]. For the AQI forecasting
problem, traditional methods include regression analysis, sup-
port vector machine (SVM), and random forest (RF) [8], [9].
For more precise control of the environmental indices, and the
next-generation smart environment setup, DL techniques, such
as the long short term memory (LSTM), are adopted for the AQI
forecasting problem [10]–[12].
In this article, study, the LSTM network was upgraded to
the nested LSTM network (NLSTM), which adds an extra
LSTM unit nested in every LSTM unit [13], [14]. An innovative
multitask multichannel (MTMC) NLSTM network (MTMC-
NLSTM) is proposed for multivariate AQI data forecasting.
Discrete stationary wavelet transform (DSWT) is employed
to decompose the original data into multiple subsignals ac-
cording to frequency. Each subsignal is then attached with an
NLSTM model for short-term AQI prediction. The proposed
MTMC-NLSTM framework has the following contributions in
the related fields.
1) Adopting NLSTM in AQI component prediction. The NL-
STM neural network was proposed in 2018. To the best
of our knowledge, it is the first work utilizing NLSTM for
AQI component prediction. The original NLSTM is also
further extended with a multichannel NLSTM structure.
2) A multitask multichannel NLSTM neural network enlight-
ened with federated learning. Instead of training and
testing the six components of the AQI separately, an
MTMC algorithm is proposed to forecast the six AQI
components, i.e., PM2.5 , PM10 , SO2 , NO2 , CO, and O3 ,
simultaneously, enlightened with the federated learning.
The proposed MTMC learning structure enhances the
data-driven prediction performance by considering the
internal correlation between different components of the
AQI concurrently.
3) A comprehensive comparative study. In Section IV, the
performance of the proposed MTMC-NLSTM model is
compared with those of the conventional machine learn-
ing (ML) models, including the multilayer perception
(MLP), the support vector regression (SVR), and various
 JIN et al.: MULTIVARIATE AIR QUALITY FORECASTING WITH NESTED LSTM NEURAL NETWORK
Fig. 1. General flowchart of the proposed AQI forecasting framework.
DL models. The performance of the proposed method is
superior to all compared existing models.
In the experimental simulation phase, a real-world dataset
collected in Beijing, China, is utilized to justify the superior
performance of the proposed method with four evaluation met-
rics. The performances of the multiple existing time series
forecasting methods are compared. The results show that the
proposed method outperforms the compared methods in both
fitting performance and prediction errors.
II. RELATED WORKS
DL techniques for time series data are among the top impor-
tant and popular topics in AI. Wen et al. [15] implemented an
LSTM model to predict solar energy generation in short term.
The proposed singular LSTM neural network outperformed
existing models in results. Das et al. [16] employed bidirectional
LSTM, a state-of-art method, to predict electric loads of different
household devices. Yan et al. [17] combined stationary wavelet
transform (SWT) with LSTM to predict energy consumption
for individual households. Sugiartawan et al. [18] proposed a
similar method predicting the growth in the number of visitors
and tourism investments. Zheng et al. [19] proposed a hybrid DL
model combining LSTM and empirical mode decomposition
(EMD), decomposing the original data into multiple intrinsic
mode functions (IMFs) for better forecasting analysis. Zang et
al. [20] employed a novel data preprocessing method variational
mode decomposition (VMD) and convolutional neural network
(CNN) in photovoltaic power forecasting and achieved better
performance.
While the research of air quality prediction receives increas-
ingly attention, various ML and DL methods have been adopted
and reinvented for AQI forecasting. Kok et al. [21] proposed
LSTM neural networks for air quality prediction in smart cities
and compared the performance of LSTM to that of SVR. Song
et al. [22] proposed an air quality prediction method based on
the hybrid model of LSTM and Kalman filtering to predict the
concentration of air pollutants.
While the time series data become increasing volatile, feder-
ated learning enlightened models are proposed to handle those
tedious problems. Cheng et al. [23] developed a multitask and
multiview learning model to perform short-term traffic forecast-
ing. Particle swarm algorithm was implemented to optimize the
model. Tian et al. [24] developed a multichannel DL framework
to conduct electrical load time series forecasting. The framework
consists two parallel channels and a feature fusion module.
One of the channels is composed of CNN layers and another
is composed of LSTM layers. The two channels are jointed in
feature fusion module and the final output is set after it. Shao et
al. [25] proposed a similar multichannel DL framework based
Authorized licensed use limited to: University of Hyderabad IG Memorial Library. Downloaded on September 13,2022 at 04:36:42 UTC from IEEE Xplore. Restrictions apply.
8515
on CNN and LSTM. The statistic components were included as
part of the features for better performance.
III. METHODOLOGY
The general flowchart of the proposed AQI forecasting frame-
work is illustrated in Fig. 1. The input AQI data consist of six
components, including PM2.5 , PM10 , SO2 , NO2 , CO, and O3 .
For each component, data normalization is applied followed by a
three-stage wavelet transform, dividing the original volatile data
into more stationary subsignals (see Fig. 2). The training part of
the subsignals are learned by the proposed MTMC-NLSTM DL
neural network to produce forecasting results. The forecasted
results are evaluated by four different evaluation metrics.
A. Discrete Stationary Wavelet Transform
Original AQI dataset contains extremely frequent and dras-
tic fluctuations. Without data preprocessing, these fluctuations
cannot be well-learned and predicted merely by DL model.
Wavelet transform (WT) is a data preprocessing method
with excellent performance for data stabilization [26]. In this
article, DSWT with basis function of Daubechies wavelet is
implemented to decompose the original data into multiple sub-
signals including the denoised lower frequency component and
deseasonalized higher frequency component (see Fig. 2) [13].
After decomposition, the information mixed in the original
data is resolved into several subsignals, resulting in improved
prediction accuracy.
By wavelet transform decomposition of level m, the original
data series can be decomposed into
m

y (t) = Am (t) + Di (t) (1)
i=1
where Am is an approximate information set, indicating the
overall pattern and longer term trend characteristics of the orig-
inal data. Di are high-frequency information sets, representing
detailed high-frequency fluctuations caused by sorts of transient
factors, which are a noise portion of the original data.
The variable m is initialized with 1. With each level of de-
composition, a low-frequency data Amt is to be decomposed
into a higher frequency set D(m+1) and a lower frequency set
A(m+1) . In the proposed model, decomposition level of WT is
set to be three, decomposing the original raw data y(t) into four
subsignals: A3 , D1 , D2 , and D3 (see Fig. 2). The original data
series y(t) is decomposed into
y (t) = A3 (t) + D3 (t) + D2 (t) + D1 (t) . (2)
It is noted that when the times of decomposition increase beyond
three, the prediction accuracy tends to be stable. However, the
complexity of the network and the time of training surges, thus
the overall efficiency and performance decrease. In Fig. 2, the
 8516 IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 17, NO. 12, DECEMBER 2021

Fig. 2. Decomposition component after wavelet transform. The signals from left to right represent original data, A3 , D3 , D2 , and D1 .

four subsignals are modeled separately with the NLSTM neural

networks for better forecasting performance.

B. NLSTM Neural Network

The very recently proposed NLSTM neural network [14] is
adopted to improve the prediction performance by memorizing
additional information in the historical data.
An NLSTM memory cell nests one more LSTM memory cell
into the original LSTM cell. The external storage cell is free to
selectively read and write the relevant long-term information of
the internal cell. This structure, in overall, improves the robust-
ness of the original LSTM neural network structure, enabling
memorizing and processing longer term history information. In
LSTM, the output gate follows a principle that information that Fig. 3. Structure of an NLSTM memory cell.
is not relevant to the current time step is still worth remembering.
 
f˜t = σ̃f
Following the abovementioned logic, NLSTM is more advan-
x̃t W̃xf + h̃t−1 W̃hf + b̃f (12)
tageous in the prediction of the time series data with volatile
changes [14].  
c̃t = f˜t  c̃t−1 + ĩt  σ̃c x̃t W̃xc + h̃t−1 W̃hc + b̃c (13)
The structure of a common LSTM memory cell is as follows:
 
it = σi (xt Wxi + ht−1 Whi + bi ) (3) õt = σ̃o x̃t W̃xo + h̃t−1 W̃ho + b̃o (14)
ft = σf (xt Wxf + ht−1 Whf + bf ) (4)
h̃t = õt  σ̃h (c̃t ) . (15)
ct = ft  ct−1 + it  σc (xt Wxc + ht−1 Whc + bc ) (5)
ot = σo (xt Wxo + ht−1 Who + bo ) (6) Due to the existence of an inner memory, the update of the
outer memory cell status ct becomes
ht = ot  σh (ct ) . (7)
ct = h̃t . (16)
Consider (9), the update of the memory cell state ct is made
by adding two parts, that is With the structure of outer and inner memory, the NLSTM
network can form a hierarchy of memory, and can remember
h̃t−1 = ft  ct−1 (8)
and process longer term information. Therefore, the NLSTM
x̃t = it  σc (xt Wxc + ht−1 Whc + bc ) . (9) network is applied to achieve better prediction with higher
accuracy and lower error instead of LSTM or other extensions.
Using another LSTM cell to replace the ct calculation equa-
tion in the ordinary LSTM model constitutes an NLSTM cell.
C. MTMC-NLSTM Neural Network Structure
The outer LSTM is called the outer memory, and the inner
LSTM is called the inner memory. In an ordinary LSTM cell, In this article, the multiple AQI time series components are
the memory cell status ct is updated as follows: trained and predicted simultaneously using a novel MTMC-
NLSTM neural network structure, considering the correlations
ct = h̃t−1 + x̃t . (10)
between different AQI components.
In the NLSTM cell depicted in Fig. 3, this process is replaced Referring to the overall flowchart shown in Fig. 1, each
by an inner LSTM cell, where x̃t and h̃t−1 are short-term and AQI component data are decomposed into multiple subsignals
long-term memory inputs, respectively. The structure of the using WT and inputted into the MTMC-NLSTM model. For
inner LSTM cell, identical with a common one, is as follows: each AQI component, a multichannel NLSTM neural network
  module is attached to learn the subsignals separately, and each
ĩt = σ̃i x̃t W̃xi + h̃t−1 W̃hi + b̃i (11) subsignal corresponds to one of the channels. Subsequently, all

Authorized licensed use limited to: University of Hyderabad IG Memorial Library. Downloaded on September 13,2022 at 04:36:42 UTC from IEEE Xplore. Restrictions apply.
 JIN et al.: MULTIVARIATE AIR QUALITY FORECASTING WITH NESTED LSTM NEURAL NETWORK
Fig. 4. Internal structure of the proposed MTMC-NLSTM model. The blue boxes represent composing layers of the neural network.
AQI components are input into the multitask NLSTM neural
network, where each component corresponds to one of the
task channels. All outputs of the multichannel models are
concatenated using a feature fusion layer for the dense layers’
outputs. Internally, the prediction output of each component is
influenced by other components in the same AQI. Considering
the physical properties of the AQI, the correlative influence
potentially improves the final prediction results of the AQI
components. The overall network structure of the proposed
MTMC-NLSTM model is illustrated in Fig. 4.
We introduce the multichannel NLSTM network and the mul-
titask structure separately. Each multichannel NLSTM network
consists of four channels. Each channel has two layers, namely,
an NLSTM layer and a full-connected layer (dense layer). The
number of units for each NLSTM layer and the number of units
for the dense layer are shown in Fig. 3. The “ADD” operation
sums the four prediction outputs and outputs a 16×1 vector as
the prediction result for each AQI component.
In the multitask framework, in the feature fuse layer, six 16×1
vectors as the outputs of the six multichannel NLSTM networks
are concatenated into a 96×1 vector. The concatenated vector is
then input into different dense layers for final prediction results.
The proposed MTMC framework is, in overall, multi-input
multioutput (MIMO), and is composed of six parallel multichan-
nel NLSTM modules, a feature fusion layer and an output layer.
In the multichannel modules, the variables are learned separately
and in parallel. In the feature fusion layer, the features of all
variables are merged. In the output layer, the final predictions of
the AQI components are generated.
It is noted that in this MTMC-NLSTM neural network struc-
ture, the proposed model enables the NLSTM neural networks
to comprehensively learn and analyze the features between
different AQI variables.
D. Proposed Forecasting Model
The detailed procedure of the proposed forecasting model is
presented in Algorithm 1. The overall process of the proposed
forecasting method involves the following three main steps.
Authorized licensed use limited to: University of Hyderabad IG Memorial Library. Downloaded on September 13,2022 at 04:36:42 UTC from IEEE Xplore. Restrictions apply.
8517
Step 1: The six AQI time series data are normalized using
zero-score normalization. Each sample of the normalized AQI
data is then decomposed into four subsignals using wavelet
transform.
Step 2: The data processed in step 1 is divided into training set
and testing set as input of the MTMC neural network. The
network has 24 inputs corresponding to the 24 subsignals,
and six outputs corresponding to the six AQIs’ predicted
value. The network is trained by fitting the actual value of
the six AQIs’ based on the 24 subsignals from the training
set. During the training process, 5% of the training set is split
to be validation set. When the neural network is trained, the
predictions of the AQIs are generated based on the testing set.
Step 3: The predictions obtained in step 2 are denormalized
to the original magnitude. The forecasting results are then
evaluated to check the forecasting performance according to
the seriousness of error and effectiveness of fitting.
The detailed hyperparameter settings of the proposed multi-
channel NLSTM model (see Fig. 4) can be found in Table I. The
RMSprop optimizer is adopted to minimize the loss function of
mean-squared error (MSE). The learning rate is set 0.001 without
weight decay. The neural network is trained by 16 epochs of
training set and the result is output. All hyperparameters are
tuned and optimized in Section IV.
IV. EXPERIMENTAL PROCESS AND RESULTS
A. Data Description
The original AQI data employed in this article are collected
from 12 observing stations around Beijing from year 2013 to
2017. The data are accessible at The University of California,
Irvine (UCI) Machine Learning Repository [27]. The original
sensor data are collected in an hourly time step, across 1461
days. The total number of hourly data samples is 35 064. Each
hourly recorded dataset consists of multiple variables and six of
them are utilized in this research, namely, PM2.5 , PM10 , SO2 ,
NO2 , CO, and O3 . The six AQI components data are visualized
in Fig. 5. It is evident that these components have vital influence
 8518 IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 17, NO. 12, DECEMBER 2021

Fig. 5. Visualization of the six AQI components.

data samples (50 days) of the remaining data are used for testing.
Algorithm 1: Overall Process of the Proposed Model,
The proportion between training, validation, and testing dataset
Where i is the Index for the AQI Components and k is the
is approximately 19:1:1.
Index for the Decomposed Subsignals.
Input: Original data of the 6 AQIs F 1∼6 , size of training
set (including validation set) ntrain , size of testing set B. Experimental Setup
ntest . The experiments were conducted in a server equipped with
Output: The prediction of the 6 AQIs F̂ 1∼6 . Intel i7-8700K CPU and NVIDIA GeForce GTX 1080 GPU.
for i = 1 to 6 do The software environment is Python 3.6. The source code and
Data normalization using Zero-Score method: dataset of this article is freely available for download.1
F S i = f S i (F i ) Multiple existing methods are compared to further verify the
Data decomposing using Wavelet superiority of the proposed method in multivariate AQI time
transform:{F SD i1∼i4 } = f W T (F S i ) series forecasting. Using the same evaluation metrics, the pre-
According to ntrain and ntest, divide F S i into: diction errors and fitting effect of different methods predicting
{T rainY i , T estY i } each AQI variable are calculated and compared. The universality
Obtain Actual value: RealY i = f S i −1 (T estY i ) of the methods is challenged because different AQI components
for k = 1 to 4 do show different patterns and characteristics.
According to ntrain and ntest, divide F SD ik The comparative methods include decision tree, random for-
into:{T rainX ik , T estX ik } est, multilayer perception (MLP), SVR, the traditional LSTM
end neural network [28], NLSTM, and the stacked LSTM (SLSTM)
end [29]. The proposed method is also compared with the hybrid
Obtain training set: DL models combining various LSTM extensions and data pre-
TrainX = {T rainX ik } , TrainY = processing methods, namely, EMD and VMD [19], [20]. EMD
{T rainY i } and VMD are implemented with similar purpose of wavelet
Obtain network testing set: transform [17], [18]. A multitask NLSTM model not combining
TestX = {T estX ik } , TestY = wavelet transform is also compared with the proposed MTMC
{T estY i } learning framework. The four evaluation metrics introduced
where i ∈ {1, 2, 3, 4, 5, 6}, j ∈ {1, 2, 3, 4} above are applied to quantify the effect of forecasting, and
Build the proposed neural network N N according to evaluate the performances of these forecasting methods.
Table I.
Input TrainX and TrainY, and fit TrainY by
C. Evaluation Metrics
minimizing loss function:
M SE(N N (TrainX), TrainY) Four evaluation metrics, namely, mean absolute error (MAE),
Input TestX to obtain the forecasting results: root-mean squared error (RMSE), mean absolute percentage
F̂ S 1∼6 = N N (TestX) error (MAPE), and r-square (R2 ), are employed to evaluate the
for i = 1 to 6 do accuracy of the prediction and verify the effectiveness of the
Denormalization: F̂ i = f S i −1 (F̂ S i ) propose method. The specific formulas of the abovementioned
end four metrics are listed as follows:
  1   
n

MAE f, fˆ = fi − fi  (17)
n i=1
upon the air quality status and are important components of air
pollutants [23].
The training set in the experiment consists of the first 22 800
(950 days) data samples in the dataset. For the purpose of 1 [Online]. Available: https://github.com/761133412/AQI_Predction_
validation, 1200 data samples (50 days) are utilized, and 1200 MTMC.git

Authorized licensed use limited to: University of Hyderabad IG Memorial Library. Downloaded on September 13,2022 at 04:36:42 UTC from IEEE Xplore. Restrictions apply.
 JIN et al.: MULTIVARIATE AIR QUALITY FORECASTING WITH NESTED LSTM NEURAL NETWORK 8519

TABLE I
HYPERPARAMETER SETTING OF THE MULTICHANNEL NLSTM MODEL

TABLE II TABLE III

FORECASTING RESULTS OF THE PROPOSED MODEL AND THE COMPARED FORECASTING RESULTS OF THE PROPOSED MODEL AND THE COMPARED
METHODS (AQI COMPONENT PM2.5 ) METHODS (AQI COMPONENT PM10 )

 TABLE IV
  1  n   2 FORECASTING RESULTS OF THE PROPOSED MODEL AND THE COMPARED
 
RMSE f, fˆ = fi − fi 
METHODS (AQI COMPONENT SO2 )
(18)
n i=1
 
  n f − f i 
1 i
MAPE f, fˆ = (19)
n i=1 fi

n  
2
  i = 1 fi − fi
R2 f, fˆ = 1 −
n   (20)
¯2
i = 1 fi − f

where f refers to the actual data value and fˆ is the predicted

value.
Amongst the four metrics, MAE, RMSE, MAPE are to eval-
uate the level of the error for prediction results. Lower MAE,
RMSE, and MAPE values refer to higher forecasting accuracy.
R2 evaluates the fitting conditions between the prediction results
and the actual data. While the values of R2 are closer to 1, the
better fitting of the prediction model/results is reflected.
a “-“ sign. We also display prediction results of all compared
methods visually in Fig. 6.
D. Results of Experiment According to the prediction results shown in Tables II–VII,
The prediction results of the six AQI components produced the proposed model outperforms all compared methods in terms
by all compared methods with four evaluation metrics are shown of the four evaluation metrics, including, MAE, RMSE, MAPE,
in Tables II –VII, respectively. The averaged training time of all and R2 . The proposed MCMT-NLSTM neural network frame-
compared methods is recorded in seconds. While the training work provides better fitting to the original data, resulting in more
time for most of the ML methods is short, it is denoted using accurate prediction results. The training time of the proposed DL

Authorized licensed use limited to: University of Hyderabad IG Memorial Library. Downloaded on September 13,2022 at 04:36:42 UTC from IEEE Xplore. Restrictions apply.
 8520 IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 17, NO. 12, DECEMBER 2021

TABLE V
FORECASTING RESULTS OF THE PROPOSED MODEL AND THE COMPARED
METHODS (AQI COMPONENT NO2 )

TABLE VI
FORECASTING RESULTS OF THE PROPOSED MODEL AND THE COMPARED
METHODS (AQI COMPONENT CO)

TABLE VII
FORECASTING RESULTS OF THE PROPOSED MODEL AND THE COMPARED
METHODS (AQI COMPONENT O3 )

Fig. 6. Comparison of final prediction result and the actual value.

The six subfigures share the same legend and from top to bottom
are the forecasting results for PM2.5 , PM10 , SO2 , NO2 , CO, and O3 ,
respectively.

model is less than 2 min, which is within the acceptable range

for real-time air pollution monitoring.
Most of the outperformance is achieved with the unique
inner memory structure of the NLSTM neural network without
increasing the training time drastically. Although the SLSTM
neural networks also improve the prediction accuracy with stack-
ing multiple LSTM layers, the stacking architecture doubles the

Authorized licensed use limited to: University of Hyderabad IG Memorial Library. Downloaded on September 13,2022 at 04:36:42 UTC from IEEE Xplore. Restrictions apply.
 JIN et al.: MULTIVARIATE AIR QUALITY FORECASTING WITH NESTED LSTM NEURAL NETWORK
number of parameters in the optimization process. The compu-
tational time of the SLSTM training process is much longer than
NLSTM. Therefore, according to our results, NLSTM model is
more suitable for AQI time series forecasting compared with
other LSTM extensions.
In the proposed and compared methods, the technique of data
decomposition using EMD, WT, and VMD are implemented to
stabilize the original AQI variable data. According to Tables II–
VII, after combining DL models with data preprocessing meth-
ods, including DSWT (WT), EMD, and VMD, the prediction
performance is improved notably and the prediction latency is
reduced. The result justifies the necessity of the implementation
of data preprocessing methods to stabilize the original data.
Compared to EMD and VMD, DL models combined with
WT perform more accurate prediction and produce less error.
VMD, EMD, and any other widely employed data processing
techniques, including Kalman filtering [22] and Fourier trans-
form [30], utilize a global and infinitely long wave to fit the
original data series, and decompose the original data.
Different from those methods, WT utilizes a local and adap-
tively long wavelet to analyze the local fluctuations in the
original data. In this study, the air pollutant concentration record
consists of plentiful frequent and drastic fluctuations, which
have no obvious global temporal pattern. The characteristics of
the dataset make WT more suitable for decomposing the AQI
data. In Tables II–VII, a model of multitask NLSTM that is not
attached with WT is compared with other methods. The results of
the multitask model are not comparable to those of the proposed
model. In the multitask model, the history information of all AQI
variables are accessible to the neural network. However, without
the WT data preprocessing, the comprehensive and multivariate
information is too mixed and too concentrated, making the DL
model overwhelmed at certain points. The obstructed NLSTM
networks cannot recognize the temporal features without WT,
and subsequently, produce less accurate prediction.
One of the contributions of this article is that an innovative
method of data preprocessing by DSWT is proposed. In the
DSWT, the high-frequency signal is not simply considered
as noise and filtered out, but learned and analyzed by neural
networks respectfully. This process significantly increases the
attentions of DL networks for every part of the features. And
since more than one LSTM network is implemented to learn
parts of the data, the prediction accuracy can be significantly
improved.
Furthermore, the prediction vectors of the subsignals are first
summed to produce the final prediction results, and then opti-
mized by the DL model. In the existing studies, the reconstruc-
tion of the data is by simply summing the results of the subsignals
without any further optimization, causing insensitivity to the
peaks [17], [18]. In the proposed method, the summation of
the subsignal prediction results is optimized during the training
processing.
V. CONCLUSION
The contribution of this article was an innovative MTMC
learning framework for multivariate air pollutant concentration
Authorized licensed use limited to: University of Hyderabad IG Memorial Library. Downloaded on September 13,2022 at 04:36:42 UTC from IEEE Xplore. Restrictions apply.
8521
prediction. The training and forecasting process of the multi-
variate AQI data was performed in parallel, enlightened with
the federated learning. According to the experimental results,
the proposed method was capable of tracing the actual AQI
very closely, with MAE, RMSE, and MAPE values all less
than three. The accurate prediction of the air pollution supported
environmental management decisions and was crucial for human
health by providing timely environmental information to the
government.
In the experimental phase, a real-world dataset collected by
weather stations located around Beijing, China, was utilized. A
comprehensive comparative study with various existing methods
in the literature was conducted and the results and comparisons
demonstrated that the proposed method could outperform most
of the existing methods compared and showed notable supe-
riority in multivariate AQI time series forecasting. According
to Tables II–VII and Fig. 6, the proposed MTMC model could
perform predictions with far less error and almost no latency
in trend compared to the other methods. The result proved that
the proposed DL framework is very suitable for multivariate
AQI time series forecasting. The proposed method separated
the features and trends by decomposing the original data into
high-frequency part and low-frequency part to learn them, re-
spectively, in a multichannel module. The prediction of each
variable was produced by merging and optimizing the prediction
of the subsignals during the training process. The capability of
learning multiple AQI variables in parallel was obtained by the
multitask structure. In the multitask architecture, the prediction
of each variable was performed and optimized after analysis of
the information of the other variables.
REFERENCES
[1] L. Wan, Y. Sun, I. Lee, W. Zhao, and F. Xia, “Industrial pollution areas
detection and location via satellite-based IIoT,” IEEE Trans. Ind. Inform.,
vol. 17, no. 3, pp. 1785–1794, Mar. 2021.
[2] S. Lei, Y. Hou, X. Wang, and K. Liu, “Unit commitment incorporating
spatial distribution control of air pollutant dispersion,” IEEE Trans. Ind.
Inform., vol. 13, no. 3, pp. 995–1005, Jun. 2017.
[3] M. M. Rathore, A. Ahmad, A. Paul, and S. Rho, “Urban planning and
building smart cities based on the Internet of things using big data analyt-
ics,” Comput. Netw., vol. 101, no. C, pp. 63–80, 2016.
[4] X. Lü, T. Lu, C. J. Kibert, and M. Viljanen, “Modeling and forecasting en-
ergy consumption for heterogeneous buildings using a physical–statistical
approach,” Appl. Energy, vol. 144, pp. 261–275, 2015.
[5] I. Alawe, A. Ksentini, Y. Hadjadj-Aoul, and P. Bertin, “Improving traffic
forecasting for 5G core network scalability: A machine learning approach,”
IEEE Netw., vol. 32, no. 6, pp. 42–49, Nov./Dec. 2018.
[6] L. Ren, Z. Meng, X. Wang, L. Zhang, and L. T. Yang, “A data-driven
approach of product quality prediction for complex production systems,”
IEEE Trans. Ind. Inform., to be published, doi: 10.1109/TII.2020.3001054.
[7] J. Chang, Y. Du, X. Chen, E. G. Lim, and K. Yan, “Forecasting based
virtual inertia control of PV systems for islanded micro-grid,” in Proc.
IEEE 29th Australas. Univ. Power Eng. Conf., 2019, pp. 1–6.
[8] Y. Han, J. C. Lam, V. O. Li, and Q. Zhang, “A domain-specific Bayesian
deep-learning approach for air pollution forecast,” IEEE Trans. Big Data,
to be published, doi: 10.1109/TBDATA.2020.3005368.
[9] K. Gu, J. Qiao, and W. Lin, “Recurrent air quality predictor based on
meteorology-and pollution-related factors,” IEEE Trans. Ind. Inform.,
vol. 14, no. 9, pp. 3946–3955, Sep. 2018.
[10] Z. Zhao, W. Chen, X. Wu, P. C. Chen, and J. Liu, “LSTM network: A deep
learning approach for short-term traffic forecast,” IET Intell. Transport
Syst., vol. 11, no. 2, pp. 68–75, 2017.
 8522 IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 17, NO. 12, DECEMBER 2021
[11] H. Shi, M. Xu, and R. Li, “Deep learning for household load forecasting—
A novel pooling deep RNN,” IEEE Trans. Smart Grid, vol. 9, no. 5,
pp. 5271–5280, Sep. 2017.
[12] M. Ma and Z. Mao, “Deep convolution-based LSTM network for re-
maining useful life prediction,” IEEE Trans. Ind. Inform., vol. 17, no. 3,
pp. 1658–1667, Mar. 2021.
[13] F. A. Gers, J. Schmidhuber, and F. Cummins, “Learning to forget:
Continual prediction with LSTM,” Neural Comput., vol. 12, no. 10,
pp. 2451–2471, 2000.
[14] J. R. A. Moniz and D. Krueger, “Nested LSTMs,” 2018, arXiv:1801.10308.
[15] L. Wen, K. Zhou, S. Yang, and X. Lu, “Optimal load dispatch of community
microgrid with deep learning based solar power and load forecasting,”
Energy, vol. 171, pp. 1053–1065, Mar. 2019.
[16] A. Das, M. K. Annaqeeb, E. Azar, V. Novakovic, and M. B. Kjærgaard,
“Occupant-centric miscellaneous electric loads prediction in buildings
using state-of-the-art deep learning methods,” Appl. Energy, vol. 269,
2020, Art. no. 115135.
[17] K. Yan, W. Li, Z. Ji, M. Qi, and Y. Du, “A hybrid LSTM neural network for
energy consumption forecasting of individual households,” IEEE Access,
vol. 7, pp. 157633–157642, 2019.
[18] P. Sugiartawan, R. Pulungan, and A. K. Sari, “Prediction by a hybrid of
wavelet transform and long-short-term-memory neural network,” Int. J.
Adv. Comput. Sci. Appl., vol. 8, no. 2, pp. 326–332, 2017.
[19] Z. Huiting, Y. Jiabin, and C. Long, “Short-term load forecasting using
EMD-LSTM neural networks with a XGBoost algorithm for feature
importance evaluation,” Energies, vol. 10, no. 8, 2017, Art. no. 1168.
[20] H. Zang et al., “Hybrid method for short-term photovoltaic power forecast-
ing based on deep convolutional neural network,” IET Gener., Transmiss.
Distrib., vol. 12, no. 20, pp. 4557–4567, 2018.
[21] I. Kok, M. U. Simsek, and S. Ozdemir, “A deep learning model for air
quality prediction in smart cities,” in Proc. IEEE Int. Conf. Big Data,
Boston, MA, USA, 2017, pp. 1983–1990.
[22] X. Song, J. Huang, and D. Song, “Air quality prediction based on LSTM-
Kalman model,” in Proc. IEEE 8th Joint Int. Inf. Technol. Artif. Intell.
Conf., 2019, pp. 695–699.
[23] S. Cheng, F. Lu, P. Peng, and S. Wu, “Multi-task and multi-view learning
based on particle swarm optimization for short-term traffic forecasting,”
Knowl. Based Syst., vol. 180, no. 15, pp. 116–132, Sep. 2019.
[24] C. Tian, J. Ma, C. Zhang, and P. Zhan, “A deep neural network
model for short-term load forecast based on long short-term memory
network and convolutional neural network,” Energies, vol. 11, 2018,
Art. no. 3493.
[25] S. Xiaorui, C.-S. Kim, and P. Sontakke, “Accurate deep model for electric-
ity consumption forecasting using multi-channel and multi-scale feature
fusion CNN-LSTM,” Energies, vol. 13, 2020, Art. no. 1881.
[26] A. J. Conejo, M. A. Plazas, R. Espinola, and A. B. Molina, “Day-ahead
electricity price forecasting using the wavelet transform and ARIMA
models,” IEEE Trans. Power Syst., vol. 20, no. 2, pp. 1035–1042,
May 2005.
[27] S. Zhang, B. Guo, A. Dong, J. He, Z. Xu, and S. X. Chen, “Cautionary
tales on air-quality improvement in Beijing,” Proc. Roy. Soc. A, Math.,
Phys. Eng. Sci., vol. 473, no. 2205, 2017, Art. no. 20170457.
[28] Q. Wang, S. Bu, and Z. He, “Achieving predictive and proactive mainte-
nance for high-speed railway power equipment with LSTM-RNN,” IEEE
Trans. Ind. Inform., vol. 16, no. 10, pp. 6509–6517, Oct. 2020.
[29] K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, and J. Schmidhu-
ber, “LSTM: A search space odyssey,” IEEE Trans. Neural Netw. Learn.
Syst., vol. 28, no. 10, pp. 2222–2232, Oct. 2017.
[30] R. N. Bracewell, The Fourier Transform and its Applications. New York,
NY, USA: McGraw-Hill, 2005.
Authorized licensed use limited to: University of Hyderabad IG Memorial Library. Downloaded on September 13,2022 at 04:36:42 UTC from IEEE Xplore. Restrictions apply.
Ning Jin received the B.S. degree and the mas-
ter’s degree in information and electronic en-
gineering from Zhejiang University, Hangzhou,
China, in 1988 and 1991, respectively.
She is currently a Professor and the Dean
with the Information Engineering College, China
Jiliang University, Hangzhou. She is a Senior
Member of the China Institute of Communica-
tions (CIC) and the Chair of the Hangzhou In-
stitute of Electronics, Hangzhou. Her research
interests include intelligent systems, wireless
networks, and signal processing.
Yongkang Zeng is currently working toward the
postgraduation degree in automation and con-
trol engineering with the Key Laboratory of Elec-
tromagnetic Wave Information Technology and
Metrology of Zhejiang Province, College of In-
formation Engineering, China Jiliang University,
Hangzhou, China.
His main research interests include AI, ma-
chine learning, and deep learning technologies
for multiple industrial applications.
Ke Yan (Member, IEEE) received the bachelor’s
and Ph.D. degrees in computer science from the
School of Computing (SoC), National University
of Singapore (NUS), Singapore, in 2006 and
2012, respectively.
He is currently an Assistant Professor with
NUS. He has authored or coauthored more than
70 full length papers with highly ranked con-
ferences and journals, including the Associa-
tion for the Advancement of Artificial Intelligence
(AAAI), IEEE TRANSACTIONS ON INDUSTRIAL IN-
FORMATICS (TII), IEEE TRANSACTIONS ON SUSTAINABLE ENERGY (TSE),
IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS: SYSTEMS
(SMCA), and Applied Energy (AE). He is actively engaged in cross-
discipline research fields. His research interests include machine learn-
ing, artificial intelligence, cyber intelligence, applied mathematics, sus-
tainability, and applied energy.
Zhiwei Ji received the bachelor’s degree from
Zhejiang A&F University (ZAFU), Linan, China,
in 2003, the master’s degree from Shanghai
University, Shanghai, China, in 2009, and the
Ph.D. degree from the School of Electronics
and Information Engineering, Tongji University,
Shanghai, in 2016.
He is currently a Full Professor with the
School of Artificial Intelligence, Nanjing Agricul-
tural University (NJAU), Nanjing, China. Prior to
this position, he was an Assistant Professor with
the University of Texas Health Science Center, Houston (UTHealth),
Houston, TX, USA. He has been working on systems biology, bioinfor-
matics, pattern recognition, big data analysis and modeling for over ten
years. He is currently the Director of the Center for Data Science and
Intelligent Computing, NJAU. He has authored or coauthored more than
50 papers in top journals, e.g., PloS Computational Biology, Information
Sciences, during recent years.