Applied Energy 329 (2023) 120288

Contents lists available at ScienceDirect

Applied Energy
journal homepage: www.elsevier.com/locate/apenergy

A proactive 2-stage indoor CO2-based demand-controlled ventilation

method considering control performance and energy efficiency
Chunxiao Li , Can Cui *, Ming Li
College of Engineering, Ocean University of China, No. 238 Songling Road, Laoshan District, Qingdao, China

H I G H L I G H T S

• The proposed proactive 2-stage demand-controlled ventilation (P2S-DCV) method improves the DCV performance.
• A deep neural network -based prediction model is proposed to predict future indoor CO2 and applies ventilation control in advance.
• A reinforcement learning-based method is proposed to achieve accurate and rapid airflow control.
• The fan pressure and damper positions are optimized to minimize energy consumption.
• The proposed P2S-DCV method has good generalization ability under various conditions.

A R T I C L E I N F O A B S T R A C T

Keywords: This paper presents a novel method, named proactive 2-stage demand-controlled ventilation (P2S-DCV) method,
Proactive 2-stage demand-controlled ventila­ to maintain indoor air quality (IAQ) and reduce the energy consumption of multi-zone ventilation systems. The
tion proposed P2S-DCV method applies a proactive control scheme, which predicts future indoor CO2 concentration
Multi-zone ventilation systems
and supplies proper ventilation to each zone. The method includes two stages. In Stage I, a DNN prediction model
Indoor air quality
Energy efficiency
is established to predict the future CO2 concentration to calculate the corresponding demand airflow. In Stage II,
Predicted indoor CO2 a reinforcement learning method is designed to achieve rapid and accurate control, and further reduce the energy
Reinforcement learning consumption by optimizing the fan pressure and damper positions. A 5-zone ventilation system is established to
validate the proposed P2S-DCV method. The experiment verifies that: a) it can maintain comfortable IAQ via
predicting the change of future indoor CO2 and applying effective ventilation control in advance; b) it can
improve the control performance, the accuracy is maintained within 8 % (satisfied the ASHRAE Standards), and
the control time is maintained within minutes. It can reduce the regulating time by 83.62 % compared with
ASHRAE Ratio method, and up to 51.68 % compared with PID method; c) it can reduce the fan energy con­
sumption by 16.4 % compared with ASHRAE Ratio method, and up to 21.8 % compared with PID method; d) it
has good generalization ability for various IAQ requirements and ventilation systems with different topologies.

However, insufficient ventilation degrades the IAQ, which can reduce

1. Introduction
It is estimated that modern people spend 80–90 % of their time at
indoor activities [1], and indoor air quality (IAQ) is one of the most
important factors affecting the health of indoor occupants [2–4]. How­
ever, due to poor indoor air circulation, the indoor pollutant concen­
trations may be 2–5 times higher than outdoor air pollutant
concentrations [5], which results in unsatisfactory IAQ. Poor IAQ can
lead to high morbidity and mortality, including sick building syndrome
[6–8]. As an important part of HVAC system, ventilation system pro­
vides treated air to a zone to promote indoor air circulation [9].
the comfort of occupants in buildings [10]. Still, excessive ventilation
can deliver additional unnecessary treated air to a zone, which can
waste energy [11]. Therefore, it is of great significance to develop an
efficient control method for ventilation systems to supply the appro­
priate amount of air to each zone to maintain satisfactory IAQ while
reducing the energy consumption of the ventilation system [12].
In recent years, the demand-controlled ventilation (DCV) methods
have been widely used for ventilation system control. DCV methods
dynamically adjust the ventilation airflow according to the ventilation
demand of the indoor environment. Currently, the existing DCV method
are mainly based on indoor occupancy [13], indoor sensors [14], or

* Corresponding author.
E-mail address: cuican@ouc.edu.cn (C. Cui).

https://doi.org/10.1016/j.apenergy.2022.120288
Received 13 July 2022; Received in revised form 29 October 2022; Accepted 1 November 2022
Available online 10 November 2022
0306-2619/© 2022 Elsevier Ltd. All rights reserved.
C. Li et al. Applied Energy 329 (2023) 120288

Nomenclature H Relative humidity (Rh)

P Air pressure (Pa)
pfan Fan static pressure (Pa) AL Indoor activity level
pmax Maximum fan static pressure (Pa) δmax Error limit (%)
s Fan speed δ Maximum relative error of airflow
θ Damper position (◦ ) δi,t Relative error of airflow in zone i at time t
θi,t Damper position of zone i (◦ ) Tcontrol Control interval (min)
min
θ Minimum value of the damper position (◦ ) Tregulate Regulating time (min)
θmax Maximum value of the damper position (◦ ) Tresponse Response time (min)
qtol Total flow rate (m3/h) Tmax
response Maximum response time (min)
max
q Maximum flow rate (m3/h) ρX1 X2 Correlation between data X1 and X2
qi,t Actual flow in zone i at time t (m3/h) η Fan efficiency
qi,t* Demand flow for zone i at time t (m3/h) dhis Historical environmental data
Ra Demand airflow per unit volume (m3/h•m2) mjk Output of hidden neuron j from layer k
P Fan power (W) Wijk Weight value between input i and hidden neuron j in layer
Δpstraight duct Pressure loss in rough pipe (Pa) k
Δpfitting Pressure loss in duct fitting (Pa) bjk Bias value for hidden neuron j in layer k
f Darcy friction factor τ Time step
L Length of the duct (m) ei,t Airflow error
Dh Hydraulic diameter (m) Ei Specified accuracy level
ρ Air density (kg/m3) at RL action
v Airflow velocity (m/s) st RL state
Cf Fitting loss coefficient rt RL reward
Cf damper Damper loss coefficient ci RL reward parameters
ξ Duct system resistance π RL policy
M Indoor CO2 release rate (ppm) π* Optimal RL policy
Vi Volume of zone i (m3) βi Positive weight coefficient
Ci,t CO2 concentration of zone i at time t (ppm) Na Adjustment times
Cimax Maximum allowable CO2 concentration for zone i (ppm) Nmax
a Rapidity threshold
Ci,t+τ CO2 concentration of zone i at time t + τ (ppm) ω Parameter of soft Q network
Cout Outdoor CO2 concentration (ppm) ϑ Parameter of soft state value function
Ci,tmix Mixed CO2 concentration of zone i at time t (ppm) φ Parameter of policy
Cs Supplied CO2 concentration (ppm) ω’ Parameter of target soft Q network
Cm Measured CO2 concentration (ppm) λ Update step
Cpre Predicted CO2 concentration (ppm) μt Mean of Gaussian distribution
Ce Excess rate of CO2 (%) σt Standard deviation of Gaussian distribution
χ Relative error of CO2 concentration D Replay buffer
Ra Demand airflow per unit area (m3/h•m2) Dq Target airflow database
NZ Number of zones γ Discount factor
T Temperature (◦ C) α Temperature coefficient

indoor pollutant concentration (e.g., CO2 [15]). Studies have shown that
25–40 % of energy can be saved through DCV method in different oc­
casions, i.e., lab [16], office building [17–19], classroom[20].
To effectively maintain IAQ and reduce the energy consumption of
ventilation systems, some DCV methods have been proposed. Anand
et al. [21] proposed a ventilation system control method based on indoor
occupancy. Experiments in different zones showed that the proposed
method can maintain IAQ while reducing system energy consumption.
Nam et al. [22] proposed an intelligent ventilation system based on the
predicted occupancy over the day. The proposed method was validated
in a Korean subway. Wang et al. [23] proposed a graph-based occupancy
detection method. The method adjusts demand airflow according to
dynamic changes in the location as well as the number and activities of
occupants. Zhuang et al. [24] proposed a DCV method based on occu­
pancy prediction, and the experimental results showed that the pro­
posed method provided a more flexible ventilation control scheme. Wei
et al. [25] proposed a DCV method based on the number of people and
occupancy activities, which is verified through a building energy model.
However, due to the randomness and complexity of occupancy [26],
it is difficult to accurately predict the demand airflow of each zone,
which may result in excessive or insufficient ventilation. To solve this
problem, some studies have proposed CO2-based DCV methods.
Kabirikopaei et al. [27] proposed a method to calculate the demand
airflow using real-time CO2 and evaluated the impact of CO2 measure­
ment accuracy on demand airflow. Experiments showed that there is a
strong relationship between demand airflow and CO2 measurement or
prediction accuracy. López et al. [28] evaluated a CO2-based IAQ
calculation method with experiments in near-zero-energy buildings in
France. The results showed that the IAQ index based on CO2 can well
reflect the indoor environment. Lu et al. [29] reviewed several CO2-
based DCV methods and clarified the role of CO2 in ventilation control.
The results demonstrated that the CO2-based DCV methods are effective
to control IAQ.
Although the above studies have improved DCV to a certain extent,
the actual performance of DCV methods may be deteriorated in practical
applications due to the following challenges. Most DCV methods are
passively ventilated, meaning that the ventilation changes after the
detected CO2 exceeds acceptable thresholds. However, there is time
delay in the IAQ response after regulating the ventilation flow due to the
spatial and temporal coupling [30]. Actually, the ventilation should
response in advance to cater to the IAQ requirement under changing
demand. In addition, for the realization of the demand flow, most DCV
methods use ASHRAE Ratio method [31] or PID method to control the
airflow. However, the flow adjustment in a practical multi-zone

2
C. Li et al. Applied Energy 329 (2023) 120288

satisfying the ASHRAE Standards and the regulating time controlled

within minutes. c) It can greatly reduce the energy consumption of the
ventilation system in DCV applications. d) It has good generalization
ability under different IAQ requirements and different ventilation
systems.
The rest of the paper is organized as follows. Section 2 introduces the
ventilation system and the control objectives of the proposed P2S-DCV
method. Section 3 introduces the proposed P2S-DCV method,
including the airflow calculation method and airflow control method.
Section 4 presents the results and relevant discussions. Finally, this
paper is concluded in Section 5.

2. Problem description

2.1. Introduction of the ventilation system and ventilation control

Mechanical ventilation systems use electric fans to supply treated air

to maintain the IAQ of conditioned zones. Fig. 1 shows a typical multi-
zone ventilation system. As shown in Fig. 1, the ventilation system
mainly consists of fans, ductwork and terminal devices.
The fan provides increased stagnation pressure from the fan inlet,
denoted by pfan, which drives air to designated zones. The fan curve can
be approximated by a quadratic formula:
( )
q2tol
pfan = pmax 1 − max (1)
q
where qtol is the actual total flow rate pumped by the fan, pmax and
Fig. 1. A typical multi-zone ventilation system. max
q are the maximum static pressure and the maximum flow rate of the
fan respectively.
ventilation system is quite difficult and may not even converge in large There are two types of losses in ventilation systems, which are the
systems, due to strong coupling effect [32] and nonlinearity [33]. pressure loss in straight rough pipes and the pressure loss in duct fittings.
Hence, it is impractical to neglect the regulating time and control ac­ The pressure loss in straight rough pipes Δpstraight duct is character­
curacy for DCV methods. ized by the Darcy–Weisbach equation:
To tackle the above-mentioned challenges, this paper proposes a
proactive 2-stage demand-controlled ventilation (P2S-DCV) method for Δpstraight duct = f
L 1 2
ρv (2)
multi-zone ventilation systems. The novelty of this paper includes: Dh 2
where f is the Darcy friction factor [34], L is the length of the duct, Dh
a) A novel proactive ventilation scheme for multi-zone DCV system: A is the hydraulic diameter, ρ is the density of air, and v is the mean flow
proactive ventilation scheme is designed to calculate the demand velocity.
airflow of each zone based on the predicted indoor CO2 concentra­ The pressure loss caused by duct fittings Δpfitting can be characterized
tion. A DNN model is established to predict the future indoor CO2 by the fitting loss coefficient Cf:
concentration. Then, the corresponding demand airflow is calculated
by the predicted CO2 concentration to achieve proactive ventilation. 1
Δpfitting = Cf ρv2 (3)
b) A novel airflow control method considering control performance: An 2
airflow control method is proposed based on reinforcement learning where Cf can be estimated from the ASHRAE duct fitting database
(RL), which is used to achieve the calculated demand airflow of each (DFDB) [34]. Among the duct fittings, the damper is commonly used
zone rapidly and accurately. RL-based airflow control method fitting for regulating the flow, which mechanism is to vary the angle of
simultaneously regulates the damper positions and fan pressure in the damper blade θ to change the damper coefficient Cf damper to further
each zone to achieve rapid and accurate control of airflow. change the duct flow, as shown in Table 1.
c) A novel airflow control method for further energy saving by opti­ The fan provides increased stagnation pressure from the fan inlet,
mizing fan and dampers: The proposed P2S-DCV method optimizes denoted by pfan, which drives air to designated zones. If the terminal
the fan static pressure and damper positions to reduce the fan power, discharge pressure is atmospheric (assuming the pressure at the system
which further increases the energy efficiency of ventilation system. outlet is negligible), pfan equals the sum of all losses:
d) The performance of the proposed P2S-DCV method has been verified ∑( )
using real office data and has been tested under different scenarios. pfan = Δpstraight duct + Δpfitting (4)

The proposed P2S-DCV method has the following advantages. a) It
can maintain comfortable IAQ via predicting the change of future indoor
CO2 and applying effective ventilation control in advance. b) It can
significantly improve the control performance, with the airflow error
Hence, the fan provides power for the ventilation system. The fan
power consumption can be described by Eq. (5), which indicates that
higher fan static pressure is associated with higher energy consumption:

Table 1
ASHRAE CD9-1 Table (Butterfly damper) [34].
θ 0◦ 10◦ 20◦ 30◦ 40◦ 50◦ 60◦ 65◦ 70◦ 80◦ 85◦ 90◦
damper
Cf 0.19 0.67 1.76 4.38 11.2 32.0 113 619 2010 10,350 99,999 99,999

3
C. Li et al. Applied Energy 329 (2023) 120288

that the CO2 concentration of each zone satisfies the specified level at
any time:
Ci,t ⩽Cimax , ∀i ∈ NZ , t > 0 (6)
max
where Ci,t denotes the CO2 concentration of zone i at time t, Ci
denotes the maximum allowable CO2 concentration for zone i, and NZ
denotes the number of zones.

3. b) Objective 2: Rapid and accurate control of airflow

To avoid practical DCV performance degraded by the control per­

formance of airflow, the regulating time and the final airflow accuracy
should be considered. For the airflow accuracy, the final error for airflow
should be controlled within tolerance:
⃒ ⃒
Fig. 2. Graph illustration of ventilation control.
⃒ ⃒
⃒qi,t − q*i,t ⃒
< δmax , ∀i ∈ NZ (7)
q*i,t
pfan ⋅qtol
P= (5)
η where qi,t denotes the actual flow in zone i at time t, qi,t* denotes the
demand flow for zone i at time t, and δmax is the specified error limit
where P is the fan power, qtol is the airflow delivered by the fan and η
(usually set to 10 % by industrial standards [35]).
is the fan efficiency.
Considering that there is response time for sensing indoor CO2 con­
Ventilation control can be described by Fig. 2, which is operating the
centration, the control interval Tcontrol for practical ventilation systems
fan and dampers. In Fig. 2, the black lines represent the fan speed curves
is given by:
under different fan speeds s. The red lines represent a family of system
resistance curves, approximated as a quartic model with respect to the Tregulate + Tresponse ⩽Tcontrol (8)
system resistance ξ. For a ventilation system operating under fan speed s
where Tregulate is the regulating time of ventilation system, Tresponse is
and system resistance ξ, the operating point is the intersection of the two
the response time of indoor CO2 concentration.
curves. Therefore, regulating the zone dampers will change the system
Assume the indoor CO2 concentration stabilizes after time Tmax response.
resistance curves, and the operating point shifts along the red lines. By
To reduce the CO2 concentration to the specified threshold Cimax within
regulating the fan speed, the operating point will shift along the black
the time step τ, we have:
lines. Fig. 2 shows that for the same demand airflow, the system can
operate under different fan pressures, denoted by points A and A’, with max
Tregulate + Tresponse ⩽τ (9)
point A yielding higher energy consumption.
This paper sets Tcontrol = τ, which leads to:
max
Tregulate < Tcontrol − Tresponse (10)
2.2. Objectives for the proposed P2S-DCV method
In sum, Eq. (7) and Eq. (10) are the two goals for Objective 2,
a) Objective 1: Real-time capability for maintaining IAQ. specifying the final airflow accuracy and the regulating time
respectively.
In this study, the CO2 concentration is adopted as the IAQ indicator.
Considering there is time delay in the response of IAQ, the future CO2
concentration is predicted based on historical data (the corresponding
theory will be introduced detailly in Section 3.2.1) to achieve proactive
ventilation. Hence, the first objective for ventilation control is to ensure

Fig. 3. Graphical framework of the proposed P2S-DCV method.

4
C. Li et al.
Fig. 4. Correlations between future CO2 and different indoor environmental
parameters under different history window sizes.
4. c) Objective 3: Improved energy efficiency of the ventilation
system
The energy efficiency is another important factor that needs to be
considered in ventilation control. For a given operating point, the energy
saving of fan can be achieved by reducing fan static pressure pfan. Ac­
cording to Eq. (4), pfan equals the sum of pressure loss Δpstraight duct and
Δpfitting. Eq. (11) indicates that fan static pressure can be reduced by
reducing the resistance of the dampers:
∑( )
pfan = Δpstraight duct + Δpfitting ⇒ pfan ∝Δpdamper ⇒ pfan ∝Cfdamper
(11)
A commonly used practice is to make at least one damper fully open
during ventilation control [3135], which leads to:
∃θi →θ min
, ∀θ min
< θi < θ max
, i ∈ NZ (12)
where θi is the damper position of zone i. According to the ASHRAE
definition, θmin = 0◦ indicates the damper is fully open, and θmax = 90◦
indicates the damper is fully closed, which imposes a dynamic damper
motion constraint.
5. The proposed P2S-DCV method for multi-zone ventilation
system
5.1. Framework of the proposed P2S-DCV method
To maintain IAQ and reduce ventilation energy consumption, this
paper proposes a proactive 2-stage demand-controlled ventilation (P2S-
DCV) method. The framework of the P2S-DCV method is depicted in
Fig. 3. The proposed P2S-DCV method includes two stages. In Stage I, a
CO2 prediction model is established to predict the future indoor CO2
concentration based on historical data. The predicted indoor CO2 con­
centration is used to calculate the corresponding demand airflow based
on the indoor environment to achieve proactive ventilation. In Stage II, a
RL-based airflow control method is designed to achieve rapid and ac­
curate tracking of the demand airflow for each zone. To further reduce
the energy consumption of the system, the fan pressure and damper
positions are optimized in Stage II. The details of the two stages are
introduced in Section 3.2 and Section 3.3.
5.2. Stage I: Calculation of demand ventilation for multi-zone ventilation
system
This section introduces the first stage of the proposed P2S-DCV
method, which is the calculation of the demand ventilation for multi-
zone ventilation systems. First, a prediction model is established to
predict indoor CO2 concentrations. Next, the proposed P2S-DCV method
uses the predicted CO2 concentration to calculate the demand airflow.
5
Applied Energy 329 (2023) 120288
5.2.1. Prediction model for indoor CO2 concentration
Accurate CO2 concentration prediction is key to achieving CO2-based
DCV [37]. However, indoor CO2 concentration is affected by many
factors, which makes it difficult to accurately predict. A data-driven
method uses machine learning techniques to learn complex relation­
ships from data. Therefore, this paper builds a data-based prediction
model to predict future indoor CO2.
The data used in this paper is provided by VVT [38], which is real-life
and long-term data captured over a period of a year in 2019, including
all four seasons. It comprises approximately 22.6 million samples ac­
quired from a total of 62 sensor nodes in 13 zones at the VTT premises.
Measurements in the database include temperature (T), relative hu­
midity (H), air pressure (P), CO2 and activity level (AL).
Before data analysis, we need to determine the prediction interval of
CO2 concentration, i.e., the size of the prediction window. The predic­
tion window is actually a trade-off between the CO2 concentration
prediction performance and control performance, which should be set
properly depending on cases.
It is found from previous experiments that a smaller prediction
window obtains better prediction results. When prediction window is
too long, the prediction performance will degrade due to the increased
complexity and uncertainty of the indoor environment. On the other
hand, the prediction window should also be big enough to guarantee
that the adjustment of airflow can be fulfilled within specified time.
For the studied case and the corresponding ventilation range (40 m3/
h − 140 m3/h), the response time Tresponse of CO2 concentration is set to
10 min, by estimating τ according to the dynamic change of indoor CO2
concentration:
( qτ) (M )[ ( qτ) ]
Ct+τ = Ct exp − + + Cs 1 − exp − (13)
V q V
where Ct+τ is the CO2 concentration at time t + τ, Ct is the CO2
concentration at time t, Cs is the CO2 concentration of the supplied air, q
is the ventilation airflow, V is the zone volume, M is the indoor CO2
release rate, and τ is the time step.
As previously mentioned, it requires time for the regulation of
ventilation system. Then, the prediction window is finally set to 15 min,
i.e., the P2S-DCV method will predict the next 15-minute CO2
concentration.
Next, the data is analyzed to determine the input of the prediction
model. Considering that indoor CO2 concentration is a time-dependent
variable, a moving time window is designed to enhance the perfor­
mance of the prediction model. The moving time window is used to
record historical environmental parameters to predict future CO2
concentrations.
This paper determines the size of the history window according to
the prediction performance and the value of Pearson correlation coef­
ficient calculated according to Eq. (14):
Cov(X , X )
1 2
ρX1 X2 = √̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
̅ (14)
Var[X1 ]Var[X2 ]
where ρX1 X2 is the correlation between data X1 and X2, Cov(X1, X2) is
the covariance of X1 and X2, Var[X1] is the variance of X1, and Var[X2] is
the variance of X2. ρX1 X2 is used to characterize the degree of correlation
between X1 and X2, and |ρX1 X2 | close to 1 means that X1 and X2 are highly
correlated.
This paper chooses 0.5 as the critical value, i.e., a significant corre­
lation is supposed to exist between data when Pearson correlation co­
efficient is greater than 0.5.
Fig. 4 compares the correlation between future CO2 with different
historical data (indoor environmental parameters) under different
window sizes. For the indoor environmental parameters, temperature
(T), relative humidity (H), air pressure (P), CO2 and activity level (AL)
are analyzed. For the size of the historical window, four sizes are tested,
which are 2, 5, 8 and 10 min.
C. Li et al. Applied Energy 329 (2023) 120288

Fig. 5. Results of predicting future CO2 concentration using different methods.

are linear regression (LR) [39], support vector machine (SVM) [40],
Table 2
long short-term memory (LSTM) [41] and deep neural network (DNN)
Prediction performance of different methods.
[42]. The four methods share the same data and data-processing, and the
LR SVM LSTM DNN hyperparameters of these methods were also optimized respectively. We
Train R2 0.96 0.95 0.88 0.96 used 5000 sets of data to train and validate the model, and 400 sets of
MAE 26.15 28.84 27.8 24.59 data to test the model. The prediction performance of the model was
RMSE 38.80 41.25 38.16 37.25
evaluated by the coefficient of determination (R2), mean absolute error
Test R2 0.80 0.81 0.51 0.82
MAE 35.60 36.27 67.43 35.01 (MAE) and root mean square error (RMSE) metrics [4344]. The results
RMSE 51.16 51.00 81.68 50.00 are given in Fig. 5 and Table 2.
In Fig. 5, the relative error of CO2 concentration is denoted by χ ,
which is calculated by Eq. (16).
It can be found from Fig. 4 that the historical data and predicted data ⃒ ⃒
are strongly correlated in CO2 concentration and AL. Hence, these two ⃒Cm − Cpre ⃒
χ= (16)
parameters are chosen as the input of the prediction model. Based on Cm
Fig. 4, the historical window size is set to 5-minute, to guarantee the
where Cm is the measured CO2 concentration, Cpre is the predicted
forecast range without compromising the prediction accuracy.
CO2 concentration.
In sum, the input of the prediction model is the CO2 concentration
According to Fig. 5 and Table 2, it is found that the DNN obtains the
and AL of the past 5 min, and the output of the prediction model is the
best prediction results among all tested methods. For LSTM, its predic­
next 15-minute CO2 concentration.
tion results have large inconsistence with the actual data (the purple line
After data analysis, we built a prediction model to predict future
in Fig. 5) due to multi-step prediction in the studied case. For LR, SVM
indoor CO2 concentration. Specifically, the prediction model is used to
and DNN, they can capture the trend of future CO2. A close look at
represent the relationship between historical data and the future CO2
Table 2 shows that the DNN has the best performance. Considering that
concentration.
DNN can well fit nonlinear data and has good generalization ability with
Ci,t+τ = F(dhis ) (15) flexible structure and low training cost, this paper adopts DNN-based
prediction model to predict the future CO2 concentration.
where Ci,t+τ represents the predicted CO2 concentration of zone i at
DNN is a multilayer structure with an input layer, hidden layers and
time t + τ, dhis is the historical data, and F(•) is the function which
an output layer. Each neuron is connected to neurons in neighbor layers
predicts the future CO2 concentration Ci,t+τ according to historical data.
with weighted values. The weighted values indicate the influence of
For the selection of F(•) function, we compared four methods, which
each input neuron on each output neuron. The output of the neurons can

Table 3
The structures and training parameters of the DNNs.
Description Narrow neural network Medium neural network Two-layer neural network Three-layer neural network

Input The CO2 concentration and activity level over past 5 min
Output The next 15-minute CO2 concentration
Normalization z-score normalization method
Main Iteration limit: 1000
components Early stopping: 10 patience
Loss function: mean squared error
Parameter optimizer: Adam
Sub- Learning rate: 0.001 Learning rate: 0.001 Learning rate: 0.001 Learning rate: 0.001
components Gradient decay factor: 0.9 Gradient decay factor: 0.9 Gradient decay factor: 0.9 Gradient decay factor: 0.9
Regularization parameter: 0.006 Regularization parameter: 0.006 Regularization parameter: 0.01 Regularization parameter: 0.01
Input layer: 12 neuronsFirst hidden Input layer: 12 neuronsFirst hidden Input layer: 12 neuronsFirst hidden Input layer: 12 neuronsFirst hidden
layer: 10 neurons layer: 16 neurons layer: 16 neurons layer: 32 neurons
(ReLU)Output layer: 1 neuron (ReLU)Output layer: 1 neuron (ReLU)Second hidden layer: 8 (ReLU)
(Linear) (Linear) neurons Second hidden layer: 16neurons
(ReLU)Output layer: 1 neuron (ReLU)Third hidden layer: 16
(Linear) neurons
(ReLU)Output layer: 1 neuron
(Linear)
Validation 10-fold Cross-validation

6
C. Li et al. Applied Energy 329 (2023) 120288

Fig. 6. Results of predicting future CO2 concentrations using different DNNs.

Fig. 6 compares different DNN models in predicting the zone CO2 con­
Table 4
centration for 400 min. Table 4 shows the prediction performance of
Prediction performance of different DNNs.
models.
Narrow Medium Two-layer Three-layer According to Fig. 6 and Table 4, it is found that the complex network
structure is prone to overfitting. Fig. 6 and Table 4 show that the DNN
neural neural neural neural
network network network network structure with 12 input neurons, 10 hidden neurons and one output
neuron exhibits best fitting performance and can be used to predict the
Train R2 0.96 0.96 0.96 0.96
MAE 24.59 24.06 25.44 23.87 change of CO2 concentration. Therefore, this paper sets narrow neural
RMSE 37.25 36.35 38.51 36.26 network as the prediction model of this paper.
Test R2 0.82 0.78 0.77 0.73
MAE 35.01 37.86 39.45 43.46 5.2.2. Calculation of demand ventilation
RMSE 50.00 54.13 55.49 60.68
Indoor CO2 concentration is commonly used to assess IAQ, which can
be used as an indicator for measuring IAQ. Most ventilation standards
recommend that the indoor CO2 concentration should not be higher than
Table 5 1000 ppm [47]. Indoor CO2 concentrations below 800 ppm indicate
Division of indoor CO2 concentration level.
excellent IAQ [48]. When the indoor CO2 concentration exceeds 1000
CO2 level Good Moderate Unhealthy ppm, it means that IAQ is unhealthy. When the indoor CO2 concentra­
Range of CO2 Ci,t < 800 800 ppm < Ci,t < Ci,t greater than tion is between 800 and 1000 ppm, it means that IAQ is moderate. When
concentrations ppm 900 ppm 1000 ppm the indoor CO2 concentration is lower than 800 ppm, it means that IAQ
is good. Table 5 gives the criteria of IAQ under different CO2 concen­
tration level.
be represented by Eq. (17):
To achieve proactive ventilation, this paper calculates the future
∑
Nk− 1 demand airflow based on the CO2 concentration predicted by DNN
mkj = Wijk mk−i 1 + bkj (17) model. In this paper, the steady-state equation is adopted to describe the
evolution of CO2 concentration. According to the mass conservation of
i=1

where mkj is the output of hidden neuron j from layer k. Wkij is the
weight value for the connection between input i and hidden neuron j in
layer k. bkj is the bias value for hidden neuron j in layer k.
To obtain proper structure and parameters of DNN, four different
structures were compared, which are narrow neural network with one
hidden layer, medium neural network with one hidden layer, neural
network with two hidden layers, and neural network with three hidden
layers.
This paper uses the 5000 sets of data (Zone 1 data from VVT data­
base) for training and 10-fold cross-validation. Specifically, this paper
initializes the weights value with the Glorot initializer [45] and ini­
tializes the bias of each layer to zero. During the training process, the
weighted coefficients and the bias values are determined by minimizing
the mean squared errors between predicted values and observed data.
The Adam algorithm [46], which is a variant of gradient descent algo­
rithm, is used to update the parameters in the network. In addition, to
avoid overfitting of the prediction model, this paper adopts L2 regula­
rization. We built DNN models in regression learner in MATLAB 2021b.
The structures and training parameters for the four tested DNNs are
given in Table 3.
After training and validation, this paper tested the model under 400
sets of data (Zone 2 data from VVT database). The performance of the
prediction model was evaluated by the coefficient of determination (R2),
mean absolute error (MAE) and root mean square error (RMSE) [4344].
indoor CO2 concentrations, the dynamics of Ci,t+τ can be described by:
(Ci,t+τ − Cimax )Vi = qi,t (Cout − Ci,t
mix
(18)
)τ
where Ci,t+τ is the predicted CO2 concentration of zone i at time t + τ,
Cimax is the maximum allowable CO2 concentration of zone i, Vi is the
volume of zone i, qi,t is the airflow entering zone i at time t, Cout is the
outdoor CO2 concentration, Ci,tmix is the mixed CO2 concentration of
zone i at time t, and τ is the duration of a time slot. In this paper, V is
obtained from the VVT database, which are 89.6, 38.3, 38.9, 89.6 and
68.8 m3 for each zone respectively, and Cout is set to 450 ppm.
Then, the demand airflow of zone i at time t can be calculated by Eq.
(19):
(Ci,t+τ − Cimax )Vi
qi,t = (19)
mix
(Cout − Ci,t )τ
5.3. Stage II: Control of ventilation for multi-zone ventilation system
To guarantee the practical performance of DCV, the proposed P2S-
DCV method introduces Stage II to improve the control performance
of airflow. For the ventilation control, the ventilation system regulates
fan pressure and damper positions to achieve the demand airflow of
each zone. The system state at next step is only relevant to the current
system state and control action and independent of the previous state.
Therefore, the ventilation system control process can be treated as a

7
C. Li et al. Applied Energy 329 (2023) 120288

Fig. 7. Flow chart of the RL-based airflow control method.

Markov decision process (MDP) problem. To properly solve this prob­ system, the reward function of the agent is designed as four parts, i.e.,
lem, this paper proposes a RL-based airflow control method in Stage II of the penalty of the airflow violation, the reward for achieving the de­
P2S-DCV method. Before introducing the proposed RL-based airflow mand airflow, the penalty of the regulating time, and the penalty of the
control method, the key elements are designed first. energy consumption of the ventilation system rapidly and accurately.
To prevent poor airflow control, the P2S-DCV method applies a
a) Key elements. penalty for airflow violation:
r1,t = − c1 Sgn(max(δi,t − E1 )) − c2 Sgn(max(δi,t − E2 )) (24)
State: The agent takes control action based on system state st. The
airflow qt = [q1,t, q2,t, …, qNz,t] is controlled by the fan pressure (pfan,t) where Sgn(•) is the signal (logical) function, E1 and E2 are the
and damper positions (θt = [θ1,t, θ2,t, …, θNz,t]). Considering that the specified accuracy level, and E1 < E2. δi,t is the relative error, which is
common aim of the agent is to minimize the error between the existing defined by Eq.:
airflow and the demand airflow, the state of the agent at time t is ⃒ ⃒
⃒ ⃒
designed as follows: ⃒qi,t − q*i,t ⃒
δi,t = (25)
[ ] q*i,t
st = pfan , θ1,t , θ2,t , … , θNZ ,t , e1,t , e2,t , … , eNZ ,t (20)
To encourage achieving the demand airflow in all zones, a reward
where ei,t = qi,t - qi* is the airflow error.
term is introduced:
Action: at represents the control action of the agent in state st. It is
designed to increment the fan pressure and damper positions: r2,t = c3 Sgn(max(E1 − δi,t )) (26)
[ ]
at = pfan , Δθ1,t , Δθ2,t , … , ΔθNZ ,t (21) which means that r2,t = c3 when δi,t < E1.
To improve the rapidity of the control method, the third part of rt can
Considering the constraints of variables present in the actual physical
be designed by:
model, at is only valid when inequalities (22) and (23) are satisfied.
r3,t = − c4 (27)
θmin ⩽θi,t + at ⩽θmax (22)
where c4 ≪ 1. r3,t is used to penalize each action to accelerate
0 < pfan,t + at ⩽pmax (23) convergence.
Based on the analysis in Section 2.1, a higher fan pressure indicates
Reward function: The transition of the environment from state st to
higher energy consumption. Therefore, the fourth part of rt is designed
st+1 is triggered by action at. Then, reward rt is given to encourage or
as follows:
punish the agent’s actions. Since our aim is to rapidly and accurately
control the airflow and reduce the energy consumption of the ventilation r4,t = − pfan,t (28)

8
C. Li et al.
Fig. 8. Architectures of the critic and actor networks.
Taking the four parts into consideration, the reward function is
designed as follows:
rt = β1 r1,t + β2 r2,t + β3 r3,t + β4 r4,t
where β1, β2, β3, and β4 are the positive weight coefficients of the four
parts.
IsDone: The stopping criterion for RL training is called IsDone,
which should be properly set to achieve the three designed control ob­
jectives. Objective 1 can be achieved by Stage I. Let Na denote the
number of adjustment times in the airflow control process, Namax denote
the rapidity threshold, and δmax denote the accuracy threshold. To
achieve Objective 2, we set max(δi,t) < δmax and Na < Namax, which
means that the accuracy and rapidity of airflow control should meet the
requirements. To achieve Objective 3, the P2S-DCV method requires at
least one damper be nearly fully open during airflow control, as previ­
ously discussed. Therefore, the stopping criterion for the training is:
IsDone = {δi,t , Na , θi |δi,t < δmax , Na < Namax , ∃θi →0, i ∈ NZ }
b) RL-based ventilation control method
To realize the airflow control of ventilation system, we design a RL-
based control method. Specifically, the soft actor-critic (SAC) [49] based
on the actor-critic (AC) framework [50] is used for learning airflow
Applied Energy 329 (2023) 120288
control method. Fig. 7 shows the flowchart of the proposed airflow
control method. The following will introduce the proposed RL-based
ventilation control method step by step.
The critic learns to provide action value estimate Q(st, at) for action
at, which is chosen based on state st, and the actor learns a policy π to
choose an action based on the Q value [50]. The SAC actor aims to
maximize the reward while also maximizing the entropy:
∑
T
π* = argmax E(st ,at )∼ρπ [r(st , at ) + αH(π(⋅|st ))] (31)
π
t=0
where α is the temperature parameter that balances the importance
of the entropy term against the system reward and H(•) is the policy
entropy, which is defined as:
∑
H(X) = − P(xi )logP(xi ) (32)
xi ∈X
The entropy term will increase the reward for actions with lower
probability to be chosen, which can enhance the exploration of the
agent. According to the Bellman backup, the soft action value function Q
(st, at) and the soft value function V(st) are defined as:
Q(st , at ) = r(st , at ) + γEst+1 ρ [V(st+1 )] (33)
V(st+1 ) = Eat π [Q(st , at ) − αlogπ (at |st )] (34)
In the policy improvement step, the policy is updated to the expo­
nential form of the soft Q function to guarantee that it is updated with
the soft Q value. In addition, a policy update range Π is established to
ensure that the πold of each update is better than πnew. Then, the KL
divergence is used to constrain the range of policy updates.
In the policy update step, the policy is updated as follows:
⎛ ⎞
(1 )
⎜ exp αQπold (st , at ) ⎟
πnew = argminDKL ⎜⎝π(⋅|st )||
⎟
⎠ (35)
π ∈Π Zπold (st )
where πnew is the updated policy, πold is the old policy, Π is the
feasible sets of policy function, Qπold (st , at ) is the Q value function of the
old policy, and Zπold (st ) is the partition term to normalize the distribution
that does not contribute to the gradient of the new policy and therefore
can be ignored, DKL(X||Y) is the KL divergence to measure the difference
of distribution X and Y. Specifically, the KL divergence is calculated by
Eq.:
∑n (
Xi
)
DKL (X||Y) = Xi log (36)
(29) i=1
Yi
DKL is used to measure the degree of difference between two prob­
ability distributions. In Eq. (35), DKL is used to constrain the update
range of the policy π, ensuring thatQπnew (st , at ) > Qπold (st , at ).
A neural network fitting policy and soft Q function were applied in
this study. The input of the policy neural network is the state, and the
output is the mean and standard deviation of the Gaussian distribution.
The probability of choosing an action is consistent with the Gaussian
distribution. The soft Q network outputs the Q value of the state-action
pair. A replay buffer D = <st, at, rt, st+1 > is used to store historical
experience, and the neural networks use the data in the replay buffer D
to update the parameters. Next, the updating method of neural network
parameters is introduced.
In SAC, to speed up the network parameter update, a target soft Q
(30)
network Q’(st, at) is built. The structure of the target soft Q network is the
same as that of the soft Q network. Specifically, the soft Q network pa­
rameters can be trained by minimizing the loss function:
[ ]
1
JQ (ω) = E(st ,at )D̃ (Qω (st , at ) − (r(st , at ) + γEst+1 P̃[Vϑ (st+1 )] ) )2 (37)
2
where ω is the parameter of the soft Q network Qω(st, at), ϑ is the
9
C. Li et al. Applied Energy 329 (2023) 120288

Fig. 9. Changes in airflow and CO2 concentration in the five zones over 400 min under the different strategies. (a) Zone 1. (b) Zone 2. (c) Zone 3. (d) Zone 4. (e)
Zone 5.

parameter of the soft state value functionVϑ (st+1 ), and Vϑ (st+1 ) can be The parameters φ of policy network π φ (at |st ) are updated by directly
computed by: minimizing the expected KL divergence in Eq. (35):
Vϑ (st+1 ) = Eat π [Qω (st , at ) − αlogπφ (at+1 |st+1 )] (38) Jπ (φ) = Est D [Eat πφ [αlog(π φ (at |st )) − Qω (st , at )]] (40)

where φ represents the parameters of policy π and ω represents the

’
To reduce the calculation, a Gaussian distribution is used to repre­
parameters of the target soft Q network Q’ω’ (st, at), which can be updated sent the policy.
by:
at = fφ (μt , σ t ) (41)
(39)
′ ′
ω = λω + (1 − λ)ω
where μt is the mean of a Gaussian distribution and σ t is the standard
where λ is the update step. deviation of the Gaussian distribution.

10
C. Li et al.
Fig. 9. (continued).
In addition, to improve the generalization ability of the proposed
P2S-DCV method, we set a target airflow database Dq so that the pro­
posed P2S-DCV method can be trained under various airflow targets.
Fig. 8 shows sketches of the designed critic and actor networks in
Stage II. The input of critic network is state and action, and the output is
Q value of state-action pair. The input of actor network is state, and the
output is the mean and standard deviation of the probability of a
Gaussian distribution which generates possible actions. The hidden
layers of critic and actor networks are fully connected (FC) layers. The
activation function of each hidden layer is ReLU function. In addition,
temperature coefficient α is 0.01. The discount factor γ is 0.99. The
learning rate of critic and actor network are both 0.001. The target soft Q
function updates with λ = 0.001. The experience buffer size is 105 and
the mini batch size is 64.
6. Results and discussion
6.1. Performance evaluation of the proposed P2S-DCV method
This section evaluates the performance of the proposed P2S-DCV
method in 4 aspects. First, the ability for maintaining IAQ is tested by
comparing with the existing DCV methods (DCV based on occupancy
and DCV based on timely CO2 concentration). Considering the proposed
P2S-DCV method introduces Stage II for airflow control, the control
performance will be accessed specifically, by comparing with traditional
airflow control methods (ASHRAE Ratio method [31] and PID method).
In addition, the energy-saving performance will also be evaluated.
Finally, the generalization of P2S-DCV method will be tested.
11
Applied Energy 329 (2023) 120288
6.1.1. Ability of the proposed P2S-DCV method in maintaining IAQ
To verify the ability of P2S-DCV method in maintaining IAQ, the P2S-
DCV method is compared with other DCV methods. The implementation
details of other DCV methods are as follows:
Occupancy-based DCV method: The indoor demand airflow is
calculated by ASHRAE Standard 62.1–2010 [12] based on the number of
occupants in each zone. The number of occupants is determined based
on the maximum number of occupants that the zone can accommodate.
According to this method, the demand airflow of the five zones during
the operation of the ventilation system is determined as 134.3, 56.4,
57.0, 134.3, and 96.8 m3/h (based on VVT data [38]).
Timely detection-based DCV method: Timely detection-based DCV
method is similar to P2S-DCV method. It calculates the demand airflow
according to the real-time indoor CO2 concentration. It changes the
airflow supplied by the ventilation system when the CO2 concentration
exceeds the threshold value.
The proposed P2S-DCV method: The demand airflow of P2S-DCV
method is calculated based on the predicted CO2. Then, the airflow
supplied by the ventilation system is changed in advance. The demand
airflow for the P2S-DCV method at high indoor activity level is calcu­
lated according to Eq. (19).
Notably, the airflow supplied by the ventilation system should also
dilute the indoor pollutants associated with the building. According to
ASHRAE Standard 62.1–2010 [12], the airflow supplied indoor is:
qi,t = AZ Ra (42)
where AZ is the area of the zone and Ra is the demand airflow per unit
area. ASHRAE provides recommended values of Ra for different types of
zones. For the office space, the recommended value of Ra is 1.08 m3/
h•m2. For the five zones, the demand airflows are 96.8, 41.4, 42.0, 96.8,
C. Li et al. Applied Energy 329 (2023) 120288

Table 6 ventilation, which will lead to higher ventilation system energy con­
Comparison of different DCV methods in terms of IAQ index and supplied sumption based on Eq. (5). The timely detection-based DCV method will
airflow. cause poor IAQ during some time periods (see the zoomed part of the
Ce (%) qtol (m3/h) right subfigure), since it responds after the CO2 concentration exceeds
Occupancy-based DCV method 0 478.7
the threshold. In comparison, the proposed P2S-DCV method predicts
Timely detection-based DCV method 9.46 353.0
P2S-DCV method 0 352.5
Table 7
3 Comparison of the control performance for different DCV methods.
and 74.3 m /h (based on VVT data [38]).
During the test period, the CO2 concentration threshold is set to Relative error (%) Mean regulating time (min)
Maximum Mean
1000 ppm. The above three strategies were used to regulate the indoor
environment. Fig. 9 shows the demand airflow and the CO2 concentra­ ASHRAE Ratio method 9.9 4.0 12.27
PID method 9.8 5.11 4.16
tion changes in the five zones using the different strategies. Based on the
P2S-DCV method 7.9 4.57 2.01
results, the occupancy-based DCV methods will cause excessive

Fig. 10. Comparison of relative errors of airflow for different DCV methods under 20 test cases. (a) Maximum relative error of all zones. (b) Mean relative errors of
all zones.

Fig. 11. Comparison of regulating time for different DCV methods under 20 test cases.

12
C. Li et al. Applied Energy 329 (2023) 120288

Table 8
∑ NZ max(Ci,t
max − Cmax ,0
)
Fan pressure and damper positions for different cases. i=1 Cimax
i
× 100%
Ce = (43)
Case Demand Airflow qi* (m3/h) pfan [θ1, θ2, θ3, θ4, θ5] δ NZ
(Pa) (%)
qtol denotes the total supplied ventilation per hour, which is associ­
1 [143.7, 112.5, 266.1, 218.8, 109.9 [27◦ , 22◦ , 0◦ , 2◦ , 7.7
125.0] 11◦ ]
ated with the cost of maintaining IAQ:
2 [175.9, 144.7, 183.8, 242.2, 115.3 [23◦ , 21◦ , 20◦ , 2◦ , 7.6 ∑NZ ∑T
qi,t Δt
118.8] 15◦ ] qtol = i=1 t=0 (44)
3 [148.7, 177.8, 137.8, 256.2, 113.1 [30◦ , 13◦ , 26◦ , 0◦ , 7.8 T
101.7] 16◦ ]
As shown in Table 6, Occupancy-based DCV method and P2S-DCV
4 [172.8, 187.2, 157.7, 233.3, 119.8 [25◦ , 14◦ , 24◦ , 2◦ , 7.8
135.7] 12◦ ] method can maintain satisfactory IAQ. However, compared with the
5 [228.6, 115.7, 175.4, 179.1, 125.1 [19◦ , 29◦ , 21◦ , 5◦ , 7.3 Occupancy-based DCV method, the proposed P2S-DCV method signifi­
208.8] 2◦ ] cantly reduces the ventilation airflow and avoids excessive ventilation.
6 [245.7, 180.8, 245.7, 109.0, 114.0 [12◦ , 1◦ , 5◦ , 14◦ , 2◦ ] 6.3 Hence, the proposed P2S-DCV method is more economic.
222.1]
7 [283.5, 237.8, 156.8, 190.9, 105.9 [1◦ , 0◦ , 23◦ , 12◦ , 7.5
109.2] 22◦ ] 6.1.2. Control performance of the proposed P2S-DCV method
8 [283.6, 238.0, 128.3, 178.2, 110.0 [5◦ , 2◦ , 29◦ , 14◦ , 7.9 This section validated the control performance of the proposed P2S-
150.0] 18◦ ] DCV method in terms of accuracy and regulating time. Comparisons are
9 [258.9, 107.2, 253.5, 168.9, 105.9 [10◦ , 22◦ , 2◦ , 6◦ , 7◦ ] 7.8
made with two other airflow control methods of DCV, which is ASHRAE
168.9]
10 [269.6, 170.9, 132.4, 219.5, 104.0 [9◦ , 15◦ , 27◦ , 1◦ , 7.2 Ratio method and the PID method. Specifically, ASHRAE Ratio method,
125.5] 10◦ ] also known as proportional method, is provided by ASHRAE as a typical
airflow control method.
20 groups of comparative experiments are conducted. The three
11 [194.5, 237.3, 157.7, 153.3, 116.2 [21◦ , 1◦ , 21◦ , 16◦ , 7.5
methods are used to control the same demand airflow under the same
182.1] 12◦ ]
12 [168.5, 148.9, 260.2, 213.9, 111.4 [24◦ , 5◦ , 0◦ , 1◦ , 9◦ ] 7.3 initial fan and damper settings, and the demand airflow is randomly set.
122.3]
13 [142.0, 244.9, 132.5, 214.6, 114.6 [30◦ , 1◦ , 29◦ , 5◦ , 7.4 a) Evaluation of the control accuracy.
145.4] 11◦ ]
14 [144.8, 165.3, 184.2, 146.3, 113.8 [29◦ , 13◦ , 16◦ , 6◦ , 7.4
215.9] 1◦ ]
Fig. 10 shows the relative errors of the measured flow and demand
15 [138.3, 198.9, 199.9, 219.7, 125.0 [30◦ , 3◦ , 16◦ , 4◦ , 6.3 flow using three methods for 20 test cases. It is observed that the relative
151.7] 11◦ ] error of the P2S-DCV method of all test cases are lower than 10 %, which
16 [223.9, 133.7, 236.6, 122.1, 107.5 [12◦ , 16◦ , 4◦ , 13◦ , 7.3 satisfies the ASHRAE Standard [35].
213.7] 3◦ ]
17 [273.0, 188.3, 226.5, 116.9, 110.5 [9◦ , 2◦ , 10◦ , 10◦ , 0◦ ] 7.9
204.5] b) Evaluation of the regulating time.
18 [196.5, 181.7, 208.4, 223.8, 108.3 [18 , 3 , 12 , 0 ,
◦ ◦ ◦ ◦
4.9
103.2] 13◦ ] Fig. 11 shows the regulating time Tregulate of the three methods in the
19 [135.8, 185.4, 252.6, 130.0, 119.5 [29◦ , 2◦ , 5◦ , 13◦ , 3◦ ] 7.5
comparative experiment, where the time step of the ventilation control
223.4]
20 [204.2, 121.7, 158.0, 156.8, 110.1 [7◦ , 1◦ , 28◦ , 6◦ , 0◦ ] 7.9
is 15 s. It can be seen that the regulating time of the proposed P2S-DCV
227.2] method is significantly shorter than the other two methods. The reason
is that the P2S-DCV method simultaneously adjusts damper positions of
multiple zones in a single step and optimizes the control policy by
the future indoor CO2 concentration and applies ventilation control in Objective 2, which greatly improves the efficiency.
advance to achieve proactive ventilation. It can avoid excessive venti­ Table 7 summarizes the control performance for different DCV
lation and can maintain the indoor CO2 concentration below the speci­ methods. It can be concluded that the P2S-DCV method demonstrates
fied threshold (1000 ppm). preferable performance in terms of both regulating time and accuracy.
To better evaluate the DCV methods, Table 6 gives the IAQ index
(based on CO2 concentration) and the total supplied airflow for the three 6.1.3. Energy-saving performance of the proposed P2S-DCV method
tested methods. Compared with other DCV methods, the P2S-DCV method has
Specifically, Ce represents the average excess rate of CO2 concen­ additional energy-saving performance, owing to the designed Objective
tration in the zone, which is to evaluate the degree of CO2 concentration 3. Table 8 lists the fan pressure and damper positions for the tested cases.
exceeding the standard: We can draw the following conclusions: a) with the proposed P2S-DCV
method, the airflow error can be controlled within 8.0 %; b) all damper

Fig. 12. Fan pressure with different methods under 20 tests.

13
C. Li et al. Applied Energy 329 (2023) 120288

Fig. 13. Energy-saving of the proposed P2S-DCV method in comparison with the ASHRAE Ratio method and PID method.

Fig. 14. Generalization test under different IAQ requirements.(a) CO2 concentration threshold of 800 ppm. (b) CO2 concentration threshold of 600 ppm.

14
C. Li et al.
Fig. 14. (continued).
positions can be kept at small positions (<30◦ ), which reduces the sys­
tem resistance and power consumption; c) at least one damper is close to
fully open which satisfies ASHRAE Standard for energy-saving purpose.
To further quantify the energy-saving performance of the P2S-DCV
method, ς is introduced to evaluate the energy conservation, which is
given by Eq. (45):
abs(Pcom − PP2S )
ϛ= (45)
Pcom
where ς is the power reduction, Pcom and PP2S are the fan power of
the compared method and the proposed P2S-DCV method respectively,
estimated by Eq. (5).
Suppose the fan efficiency η remains constant, then ς can be related
to the fan static pressure. Then, Eq. (45) can be transformed to:
15
Applied Energy 329 (2023) 120288
abs(pfan,com − pfan,P2S )
ϛ≈ (46)
pfan,com
where pfan,com is the fan pressure of compared method, and pfan,P2S is
the fan pressure of the proposed P2S-DCV method.
Then, this paper compared the fan power of the proposed method
with ASHRAE Ratio method and PID method under 20 tests. In each test,
the demand airflow is kept same for all three methods, i.e., the total
airflow qtol is same. During the comparative experiment, the fan pressure
for each control method were recorded, which are shown in Fig. 12. The
corresponding energy conservation can be estimated by Eq. (46).
Fig. 13 plots the energy-saving for the proposed P2S-DCV method in
comparison with the other two methods. As shown, it is estimated that
the proposed P2S-DCV method can save up to 16.4 % energy compared
C. Li et al. Applied Energy 329 (2023) 120288

Fig. 15. Ventilation systems with different topologies.(a) Ventilation system I. (b) Ventilation system II. (c) Ventilation system III.

6.2. Further validation of the generalization ability of the proposed P2S-

Table 9 DCV method
Results for different ventilation systems (a) Ventilation system I. (b) Ventilation
system II. (c) Ventilation system III.
In this section, the generalization ability of the P2S-DCV method is
Ventilation Case Demand Airflow qi* pfan [θ1, θ2, θ3, θ4, δ evaluated on the different IAQ requirements and different ventilation
system (m3/h) (Pa) θ5] (%)
systems.
System I 1 [200.4, 148.9, 160.7, 76.8 [12◦ , 9◦ , 13◦ , 6.4
111.2, 143.0] 0◦ , 23◦ ]
6.2.1. Generalization ability under different IAQ requirements
2 [143.2, 160.5, 177.4, 87.3 [25◦ , 3◦ , 13◦ , 7.3
93.2, 174.6] 16◦ , 30◦ ]
This section tests the performance of the P2S-DCV method under
3 [132.2, 145.1, 136.9, 91.4 [27◦ , 18◦ , 6.5 different CO2 concentration thresholds, which are 800 ppm and 600
98.3, 184.5] 24◦ , 4◦ , 17◦ ] ppm. Fig. 14(a) and (b) depict the dynamic curves of the CO2 concen­
4 [137.8, 111.4, 143.4, 98.8 [28◦ , 28◦ , 7.8 tration of each zone when P2S-DCV method is applied to the thresholds
136.6, 134.6] 26◦ , 4◦ , 28◦ ]
of 800 ppm and 600 ppm, respectively. As shown in Fig. 14, the P2S-
5 [131.7, 113.2, 187.1, 87.8 [26◦ , 19◦ , 7.9
119.1, 148.8] 10◦ , 1◦ , 23◦ ] DCV method can control the indoor CO2 concentration below the
System II 1 [184.2, 184.2, 138.2, 66.0 [8◦ , 3◦ , 6◦ , 7.4 required threshold under all test cases. The proposed P2S-DCV method
206.7, 138.1] 8◦ , 20◦ ] demonstrates good generality ability under different CO2 requirements.
2 [149.4, 209.5, 91.1, 81.4 [15◦ , 2◦ , 19◦ , 6.0
214.3, 214.3] 3◦ , 2◦ ]
3 [203.9, 203.9, 146.9, 80.0 [2◦ , 2◦ , 9◦ , 7.4
6.2.2. Generalization ability under different ventilation systems
167.3, 167.2] 16◦ , 17◦ ] This section tests the performance of the P2S-DCV method under
4 [125.2, 146.9, 178.1, 71.9 [21◦ , 21◦ , 2◦ , 7.5 three other different ventilation systems. Fig. 15 depicts the topologies
166.9, 153.3] 14◦ , 18◦ ] of tested ventilation systems, which are named as Ventilation system I,
5 [170.0, 134.3, 155.6, 71.5 [10◦ , 20◦ , 0◦ , 7.8
Ventilation system II, and Ventilation system III. These ventilation sys­
152.6, 209.7] 16◦ , 11◦ ]
System III 1 [184.4, 149.3, 128.0, 65.6 [7◦ , 0◦ , 4◦ , 6.7 tems have different duct component parameters and topologies. The
194.5, 150.2] 11◦ , 20◦ ] agent preciously trained in Section 4.1.2 is directly used for airflow
2 [197.1, 137.2, 137.2, 78.5 [7◦ , 2◦ , 2◦ , 2.8 control of these ventilation systems.
183.8, 198.7] 11◦ , 12◦ ]
Table 9 depicts the tests results. It shows that the airflow error of
3 [205.2, 162.8, 130.0, 77.8 [4◦ , 4◦ , 10◦ , 5.3
162.2, 179.7] 15◦ , 15◦ ]
P2S-DCV method is<10 % under all tested systems. Generally, the
4 [172.8, 171.5, 112.9, 68.7 [9◦ , 3◦ , 12◦ , 6.7 control can be fulfilled within 10 steps, which is relatively efficient. In
157.4, 156.4] 18◦ , 19◦ ] addition, the damper positions also satisfy the requirement of the
5 [168.1, 98.2, 158.2, 71.9 [11◦ , 9◦ , 2◦ , 4.2 ASHRAE Standard that at least one damper is close to fully open.
147.1, 214.9] 15◦ , 10◦ ]
In conclusion, the P2S-DCV method is applicable over different IAQ
requirements and ventilation systems.
with the ASHRAE Ratio method (Test 5), and up to 25.6 % energy
compared with the PID method (Test 10). On average, the proposed P2S- 7. Conclusions
DCV method can improve the energy-saving by 4.85 % in comparison
with ASHRAE Ratio method, and 19.62 % in comparison with PID A proactive 2-stage demand-controlled ventilation (P2S-DCV)
method. In conclusion, the proposed P2S-DCV method shows good method is proposed to maintain IAQ and reduce the energy consumption
improvement in the energy conservation. of multi-zone ventilation systems. The proposed P2S-DCV method con­
sists of two stages. In Stage I, the demand airflow is calculated based on
the proposed indoor CO2 prediction model. In Stage II, an RL-based
control method is designed to control the fan pressure and damper

16
C. Li et al.
positions to achieve the demand airflow for each zone. In addition, the
P2S-DCV method achieve higher energy efficiency by optimizing the fan
pressure and damper positions during the control process.
Compared with other DCV methods, the P2S-DCV method can
maintain IAQ within the specified range, avoiding poor IAQ and
excessive ventilation. In addition, with the proposed P2S-DCV method,
the airflow control process can be fulfilled within minutes, and the flow
error satisfies the ASHRAE Standard (within 10 %). In terms of the
energy-saving performance, the P2S-DCV method can reduce energy
consumption by 16.4 % compared with the ASHRAE Ratio method and
21.8 % compared with the PID method. Furthermore, the results
demonstrate that the P2S-DCV method has a good generalization ability
over various IAQ requirements and ventilation systems with different
topologies. Future work will further investigate the generalization of the
proposed method under different-sized ventilation systems and other
possible factors.
CRediT authorship contribution statement
Chunxiao Li: Methodology, Software, Validation, Investigation,
Data curation, Writing – original draft. Can Cui: Conceptualization,
Writing – review & editing, Project administration, Funding acquisition.
Ming Li: Writing – review & editing, Supervision, Project
administration.
Declaration of Competing Interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
Data availability
Data will be made available on request.
Acknowledgements
This work was supported in part by the Shandong Provincial Natural
Science Foundation (Grant No. ZR2021QE291), the Shandong Provin­
cial Science Fund for Excellent Young Scholars (Overseas) (Grant No.
2022HWYQ-064), and in part by the start-up grant of Prof. Cui Can at
Ocean University of China.
References
[1] Paleologos KE, Selim MYE, Mohamed AMO. Chapter 8 - Indoor air quality:
pollutants, health effects, and regulations. In: Mohamed AMO, Paleologos EK,
Howari FM, editors. Pollution assessment for sustainable practices in applied
sciences and engineering. Oxford, UK: Butterworth-Heinemann; 2021. p. 405–89.
[2] Cao SJ, Ren C. Ventilation control strategy using low-dimensional linear
ventilation models and artificial neural network. Build Environ 2018;144:316–33.
https://doi.org/10.1016/j.buildenv.2018.08.032.
[3] Khovalyg D, Kazanci OB, Halvorsen H, Gundlach I, Bahnfleth WP, Toftum J, et al.
Critical review of standards for indoor thermal environment and air quality. Energy
Build 2020;213:109819.
[4] Shin MS, Rhee KN, Lee ET, Jung GJ. Performance evaluation of CO2-based
ventilation control to reduce CO2 concentration and condensation risk in
residential buildings. Build Environ 2018;142:451–63. https://doi.org/10.1016/j.
buildenv.2018.06.042.
[5] Pitarma R, Marques G, Ferreira BR. Monitoring indoor air quality for enhanced
occupational health. J Med Syst 2016;41:23. https://doi.org/10.1007/s10916-016-
0667-2.
[6] Joshi SM. The sick building syndrome. Indian J Occup Environ Med 2008;12:61–4.
https://doi.org/10.4103/0019-5278.43262.
[7] Persily AK. Indoor carbon dioxide concentrations in ventilation and indoor air
quality standards. In: In: 36th AIVC conference effective ventilation in high
performance buildings. Gaithersburg, Maryland: NIST; 2015. p. 810–9.
[8] Lowther SD, Dimitroulopoulou S, Foxall K, Shrubsole C, Cheek E, Gadeberg B, et al.
Low level carbon dioxide indoors—a pollution indicator or a pollutant? A health-
based perspective Environments 2021;8(11):125.
[9] Cakyova K, Figueiredo A, Oliveira R, Rebelo F, Vicente R, Fokaides P. Simulation of
passive ventilation strategies towards indoor CO2 concentration reduction for
17
Applied Energy 329 (2023) 120288
passive houses. J Build Eng 2021;43:103108. https://doi.org/10.1016/j.
jobe.2021.103108.
[10] Zucker G, Sporr A, Kollmann S, Wendt A, Chaido LS, Fernbach A. A cognitive
system architecture for building energy management. IEEE Trans Ind Inform 2018;
14:2521–9. https://doi.org/10.1109/TII.2018.2815739.
[11] Fix AJ, Pamintuan BC, Braun JE, Warsinger DM. Vapor-selective active membrane
energy exchanger with mechanical ventilation and indoor air recirculation. Appl
Energy 2022;312:118768. https://doi.org/10.1016/j.apenergy.2022.118768.
[12] Ashrae. Standard 62.1-2010, ventilation for acceptable indoor air quality. Atlanta,
Georgia: ASHRAE; 2010.
[13] Ye Y, Chen Y, Zhang J, Pang Z, O’Neill Z, Dong B, et al. Energy-saving potential
evaluation for primary schools with occupant-centric controls. Appl Energy 2021;
293:116854. https://doi.org/10.1016/j.apenergy.2021.116854.
[14] Lu X, O’Neill Z, Li Y, Niu F. A novel simulation-based framework for sensor error
impact analysis in smart building systems: A case study for a demand-controlled
ventilation system. Appl Energy 2020;263:114638. https://doi.org/10.1016/j.
apenergy.2020.114638.
[15] Kim MK, Baldini L, Leibundgut H, Wurzbacher JA. Evaluation of the humidity
performance of a carbon dioxide (CO2) capture device as a novel ventilation
strategy in buildings. Appl Energy 2020;259:112869. https://doi.org/10.1016/j.
apenergy.2019.03.074.
[16] Kong M, Dong B, Zhang R, O’Neill Z. HVAC energy savings, thermal comfort and
air quality for occupant-centric control through a side-by-side experimental study.
Appl Energy 2022;306:117987. https://doi.org/10.1016/j.apenergy.2021.117987.
[17] Li W, Wang S. A multi-agent based distributed approach for optimal control of
multi-zone ventilation systems considering indoor air quality and energy use. Appl
Energy 2020;275:115371. https://doi.org/10.1016/j.apenergy.2020.115371.
[18] Li W, Wang S, Koo C. A real-time optimal control strategy for multi-zone VAV air-
conditioning systems adopting a multi-agent based distributed optimization
method. Appl Energy 2021;287:116605. https://doi.org/10.1016/j.
apenergy.2021.116605.
[19] Li B, Wu B, Peng Y, Cai W. Tube-based robust model predictive control of multi-
zone demand-controlled ventilation systems for energy saving and indoor air
quality. Appl Energy 2022;307:118297. https://doi.org/10.1016/j.
apenergy.2021.118297.
[20] Zhang S, Ai Z, Lin Z. Novel demand-controlled optimization of constant-air-volume
mechanical ventilation for indoor air quality, durability and energy saving. Appl
Energy 2021;293:116954. https://doi.org/10.1016/j.apenergy.2021.116954.
[21] Anand P, Sekhar C, Cheong D, Santamouris M, Kondepudi S. Occupancy-based
zone-level VAV system control implications on thermal comfort, ventilation,
indoor air quality and building energy efficiency. Energy Build 2019;204:109473.
https://doi.org/10.1016/j.enbuild.2019.109473.
[22] Nam KiJeon, Heo SungKu, Li Q, Loy-Benitez J, Kim MinJeong, Park DuckShin,
et al. A proactive energy-efficient optimal ventilation system using artificial
intelligent techniques under outdoor air quality conditions. Appl Energy 2020;266:
114893.
[23] Wang H, Wang G, Li X. Image-based occupancy positioning system using pose-
estimation model for demand-oriented ventilation. J Build Eng 2021;39:102220.
https://doi.org/10.1016/j.jobe.2021.102220.
[24] Zhuang C, Choudhary R, Mavrogianni A. Probabilistic occupancy forecasting for
risk-aware optimal ventilation through autoencoder Bayesian deep neural
networks. Build Environ 2022;219:109207. https://doi.org/10.1016/j.
buildenv.2022.109207.
[25] Wei S, Tien PW, Chow TW, Wu Y, Calautit JK. Deep learning and computer vision
based occupancy CO2 level prediction for demand-controlled ventilation (DCV).
J Build Eng 2022;56:104715. https://doi.org/10.1016/j.jobe.2022.104715.
[26] Clark JD, Less BD, Dutton SM, Walker IS, Sherman MH. Efficacy of occupancy-
based smart ventilation control strategies in energy-efficient homes in the United
States. Build Environ 2019;156:253–67. https://doi.org/10.1016/j.
buildenv.2019.03.002.
[27] Kabirikopaei A, Lau J. Uncertainty analysis of various CO2-Based tracer-gas
methods for estimating seasonal ventilation rates in classrooms with different
mechanical systems. Build Environ 2020;179:107003. https://doi.org/10.1016/j.
buildenv.2020.107003.
[28] López MJR, Guyot G, Golly B, Ondarts M, Wurtz F, Gonze E. Relevance of CO2-
based IAQ indicators: Feedback from long-term monitoring of three nearly zero-
energy houses. J Build Eng 2021;44:103350. https://doi.org/10.1016/j.
jobe.2021.103350.
[29] Lu X, Pang Z, Fu Y, O’Neill Z. The nexus of the indoor CO2 concentration and
ventilation demands underlying CO2-based demand-controlled ventilation in
commercial buildings: a critical review. Build Environ 2022;218:109116. https://
doi.org/10.1016/j.buildenv.2022.109116.
[30] Zheng J, Guo X, Hu S, Wu F, Lao C, Ma H, et al. Study on long-term variation
characteristics of indoor CO2 concentrations based on a data-driven method.
Energy Build 2022;256:111760.
[31] Nebb. Procedural standard for testing adjusting and balancing of environmental
systems. Maryland: NEBB; 2015.
[32] Cui C, Zhang X, Cai W. An energy-saving oriented air balancing method for
demand controlled ventilation systems with branch and black-box model. Appl
Energy 2020;264:114734. https://doi.org/10.1016/j.apenergy.2020.114734.
[33] Cheng F, Cui C, Cai W, Zhang X, Ge Y, Li B. A novel data-driven air balancing
method with energy-saving constraint strategy to minimize the energy
consumption of ventilation system. Energy 2022;239:122146. https://doi.org/
10.1016/j.energy.2021.122146.
[34] Ashrae. ASHRAE duct fitting database version 6.0. Atlanta, Georgia: ASHRAE;
2014.
C. Li et al.
[35] Ashrae. Heating, ventilating, and air-conditioning applications. Atlanta, Georgia:
ASHRAE; 2015.
[37] Liu X. ASTM and ASHRAE standards for the assessment of indoor air quality. In:
Zhang Y, HOPKE PK, Mandin C, editors. Handbook of Indoor Air Quality.
Singapore: Springer Singapore; 2021. p. 1–36.
[38] Kallio J, Tervonen J, Räsänen P, Mäkynen R, Koivusaari J, Peltola J. Forecasting
office indoor CO2 concentration using machine learning with a one-year dataset.
Build Environ 2021;187:107409. https://doi.org/10.1016/j.
buildenv.2020.107409.
[39] Lu T, Lü X, Salonen H, Zhang Q. Novel hybrid modeling approach for utilizing
simple linear regression models to solve multi-input nonlinear problems of indoor
humidity modeling. Build Environ 2022;213:108856. https://doi.org/10.1016/j.
buildenv.2022.108856.
[40] Blanco V, Japón A, Puerto J. A mathematical programming approach to SVM-based
classification with label noise. Comput Ind Eng 2022;172:108611.
[41] Zha W, Liu Y, Wan Y, Luo R, Li D, Yang S, et al. Forecasting monthly gas field
production based on the CNN-LSTM model. Energy 2022;260:124889. https://doi.
org/10.1016/j.energy.2022.124889.
[42] Liang Y, Wu C, Ji X, Zhang M, Li Y, He J, et al. Estimation of the influences of
spatiotemporal variations in air density on wind energy assessment in China based
18
Applied Energy 329 (2023) 120288
on deep neural network. Energy 2022;239:122210. https://doi.org/10.1016/j.
energy.2021.122210.
[43] Hyndman R, Koehler A. Another look at measures of forecast accuracy. Int J
Forecast 2006;22:679–88. https://doi.org/10.1016/j.ijforecast.2006.03.001.
[44] Nam KiJeon, Kim MinJeong, Lee S, Hwangbo S, Yoo ChangKyoo. Interpretation
and diagnosis of fouling progress in membrane bioreactor plants using a periodic
pattern recognition method. Korean J Chem Eng 2017;34(11):2966–77.
[45] Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward
neural networks. in: AISTATS; 2010.
[46] Kingma. D, Ba. J. Adam: A method for stochastic optimization. arXiv preprintarXiv:
1412.6980. 2014. https://doi.org/10.48550/arXiv.1412.6980.
[47] Chenari B, Dias Carrilho J, Gameiro da Silva M. Towards sustainable, energy-
efficient and healthy ventilation strategies in buildings: a review. Renew Sustain
Energy Rev 2016;59:1426–47. https://doi.org/10.1016/j.rser.2016.01.074.
[48] Kapsalaki M. ASHRAE position document on indoor carbon dioxide. Atlanta,
Georgia: ASHRAE; 2022.
[49] Haarnoja T, Zhou A, Hartikainen K, Tucker G, Ha S, Tan J, et al. Soft actor-critic
algorithms and applications. arXiv preprint arXiv:1812.05905, 2018.
[50] Konda V, Tsitsiklis J. Actor-critic algorithms. In: Solla SA, Leen TK, Muller KR,
editors. Advances in neural information processing systems. Cambridge, MA: MIT
Press; 1999. p. 1008–14.