Sensors and Actuators B 267 (2018) 245–254

Contents lists available at ScienceDirect

Sensors and Actuators B: Chemical

journal homepage: www.elsevier.com/locate/snb

Using statistical methods to carry out in field calibrations of low cost

air quality sensors
José María Cordero ∗ , Rafael Borge, Adolfo Narros
ETSII-UPM, José Gutiérrez Abascal 2, 28006 Madrid, Spain

a r t i c l e i n f o a b s t r a c t

Article history: The poor air quality found in big cities is harmful to human health. The aim of the LIFE PHOTOSCALING
Received 28 November 2017 Project (LPP) is to assess the effectiveness of different photocatalytic pavements in reducing NO2 pol-
Received in revised form 14 March 2018 lution. The objective of this preliminary study is to determine how well low cost AQmesh sensors can
Accepted 4 April 2018
accurately enough measure NO2 concentrations to be able to determine the effects of using photocat-
Available online 11 April 2018
alytic pavements. Data was collected from AQmesh sensors that were installed in two Air Quality Stations
in order to monitor NO2 under traffic and urban background conditions. The NO2 measurements were
Keywords:
unreliable, resulting in an unsatisfactory level of accuracy. A two-step calibration method was devised
Electrochemical sensors
Neuronal networks
in order to overcome this limitation. This method consisted of supervised statistical machine learning
Uncertainty regression algorithms. A first Multivariate Linear Regression provided a new explanatory variable that
Nitrogen dioxide contained valuable information about the error. This variable was fed into more sophisticated equations,
Air quality such as Random Forests, Support Vector Machines and Artificial Neural Networks. The various models
were evaluated by calculating statistics of errors and relative expanded uncertainties, and through Taylor
Diagrams. After a careful calibration, AQmesh sensors met the Air Quality Directive’s standards of accu-
racy at high concentrations of NO2 . However, we found that each individual sensor behaves differently
and thus, each unit requires the development and application of a specific calibration model.
© 2018 Elsevier B.V. All rights reserved.

1. Introduction under adverse weather conditions, pollutants cannot dissipate,

Worldwide, poor air quality leads to 3 million deaths annually
[1]. Ambient air pollution is a cause of premature deaths and vari-
ous respiratory conditions, such as throat irritation, congestion and
asthma [2]. Motor vehicles are the principle source of NO2 . High
concentrations of this pollutant are routinely detected in areas of
traffic. Therefore, the source of NO2 emissions is found to be prox-
imal to pedestrians, causing a harmful situation.
Within the framework of its current sustainable development
initiatives, the European Union (EU) approved The Air Quality
Directive (AQD) [3]. The AQD established thresholds for different
types of pollutants. EU member states are bound by this directive
to apply the necessary measures to bring pollutant concentrations
down below these thresholds.
However, NO2 is routinely detected at higher than permitted
levels at Air Quality Stations (AQS) in large cities that have high
motor vehicle traffic, such as Madrid. AQS networks continuously
monitor pollutants. From these monitoring activities we know that,
∗ Corresponding author.
E-mail address: jmcordero@etsii.upm.es (J.M. Cordero).
leading to levels of NO2 well above the AQD thresholds. Mea-
sures are taken to minimize the impact of these weather events,
such as restricting motor vehicle traffic. As a consequence, the nor-
mal circulation of motor vehicle traffic is altered, disrupting the
daily routines of the citizenry. Alternatives to the traffic restrictions
are technologies for reducing the NO2 in the atmosphere such as
installing photocatalytic pavements or paints in roads, sidewalks
and facades [4–6] or photocatalytic urban textiles in sunshades
[7]. These technologies rely on TiO2 -based materials that are acti-
vated by UVA radiation. Upon activation, these materials are able
to oxidize NO2 into nitrates.
The high cost of AQS networks means that pollutants can only
be measured in a limited number of locations at a time. This makes
it impossible to model a complete time space distribution. Passive
samplers have been used to study the distribution of pollutants at
the microscale level [8]. However, passive sampling is a cumulative
method and cannot be used to carry out absolute measurements
at discrete time points. As a consequence, complex distributions in
the horizontal and vertical profiles cannot be performed via passive
sampling. In addition, the effect of other meteorological or ambi-
ent variables on NO2 measurements cannot be determined using
passive methods.

https://doi.org/10.1016/j.snb.2018.04.021
0925-4005/© 2018 Elsevier B.V. All rights reserved.
246 J.M. Cordero et al. / Sensors and Actuators B 267 (2018) 245–254
Low-cost sensors (LCS) are being developed through initiatives
such as AirSensEUR [9,10] and are based in both electrochemical
or solid state microsensors designed to monitor urban ambient air
gases like NOx , CO, SO2 or O3 . Networks of mobile or fixed LCSs gas
sensors permit the mapping of wider areas of the city [11] with a
high time resolution. The sensors are normally calibrated against
the measurements recorded in nearby AQSs in a co-location cam-
paign. LCSs are versatile, autonomous, and can make measurements
at several points over wide areas [12], as well as remote spaces [13],
reducing the cost of detection in comparison to AQSs and allowing
the generation of large observation datasets. However, this technol-
ogy has its challenges as well: LCSs show cross-sensitivity between
the target gas and other gases in the urban atmosphere. In particu-
lar, NO2 and O3 electrochemical sensors are prone to show mutual
cross-sensitivities [14]. LCSs are also dependent on weather condi-
tions, and suffer from short term and long term drift. In addition,
units of the same type are highly variable [15], poorly selective, and
not very stable [16–18].
LCSs have been extensively checked and calibrated in labora-
tory. However, the results change dramatically when the devices
are deployed in urban sites [15,17–19]. This is because the con-
ditions of the urban atmosphere cannot be exactly reproduced in
the laboratory. Consequently, in-field calibrations of these devices,
under the conditions in which the LCSs will be used, are essential
for obtaining accurate readings in the urban atmosphere. There-
fore, a calibration method has to be designed for each individual
device under the particular conditions it will be used.
These limitations make the calibration of these devices difficult,
and complex equations are required to assure the reliability of these
instruments. As part of the MacPoll Project, the Joint Research Cen-
tre has published several reports on this subject and has designed
a calibration protocol for LCSs [20]. In most of these studies, sta-
tistical methods, such as linear regression (LR), multivariate linear
regression (MLR) and artificial neural networks (ANN) [13,21,23,25]
are used to fit the signals. A recent work provides a comprehen-
sive comparison of several machine learning algorithms, including
training, optimization and validation [22]. Some ANN studies have
relied on predictive variables from different sensors as inputs
[19,20], an approach that solved the primary problem of interfer-
ence between sensor measurements and external variables. This
strategy has led to more accurate results. Another approach has
been to use a reference station to update the parameters of the
calibrated sensors, thereby correcting for the drift [24]. In other
study [26] other sophisticated machine learning algorithms such
as boosted regression trees and Gaussian process emulators were
successfully applied to process the raw signal of their LCSs, solv-
ing the main complex interferences in the measurements. Here we
report the results of using ANNs and less commonly utilized meth-
ods, such as Support Vector Machines (SVM) and Random Forests
(RF), to fit the data using AQS reference data as objective variables.
The goal of the LIFE PHOTOSCALING Project (LPP) is to achieve
the sustainable use photocatalytic technologies on urban pave-
ments. As part of the LPP, different photocatalytic materials will be
tested for using as pavements [27]. To evaluate the effectiveness of
these materials, accurate and precise methods for measuring NO2
are crucial. The objective of this study is to validate the LCS gas sen-
sor technology for use in the context of the LPP. We report here a
more advanced method for calibrating LCS devices, making them
more accurate and precise.
AQmesh units (PODs) were used as model LCSs. Manufacturer
calibration of these devices was not found to be satisfactory in
terms of accuracy. Four PODs were installed in two AQSs in the
Community of Madrid. These AQSs covered two different ambient
conditions: traffic and urban background. Data was gathered from
both the PODs and AQSs. Various statistical methods were applied
to fit the data. Interactions between variables that obscured the cor-
rect measurements were evaluated. This process had to be repeated
for every POD under each ambient condition. This study should be
repeated in order to account time drift [20] and seasonally variable
meteorological conditions.
2. Materials and methods
2.1. Air quality stations
Reference measurements were collected from two AQSs. One of
these was located in a traffic area in a neighbourhood of Arturo Soria
street in the city of Madrid. The other was located in an urban back-
ground area in the town of Arganda del Rey, located about 25 km
southeast of Madrid city. As a consequence, the behaviour of the
AQmesh PODs could be studied under two different environmental
conditions. The measurement techniques utilized were:
• Chemiluminescence, to measure levels NO and NO2 .
• UV absorption, to measure levels of O3 .
Meteorological parameters were recorded only in Arganda del
Rey because the Arturo Soria AQS did not have a meteorological
station.
The Arturo Soria AQS is managed by the Madrid City Council and
the Arganda del Rey AQS is managed by the Comunity of Madrid
autonomous government.
2.2. AQmesh PODs
AQmesh PODs were selected as model LCSs. Our group had
extensive experience with LCSs that were either electrochemical
or MeO based. AQmesh was chosen because they provide accu-
rate readings over a wide range of values: from 0 to 4000 ppb for
NO and NO2 , and from 0 to 1800 ppb for O3. All the gas sensors
in the AQmesh PODs tested were Alphasense electrochemical sen-
sors [28]. Detailed technical specifications can be found in reference
[29]. The POD’s supplier applies a proprietary algorithm which con-
verts the electrical signals into concentration units. The user could
download the resulting information from a website. From here
on in, we will refer to this downloaded data as “raw” despite the
prior treatment. However, treating the data with the proprietary
algorithm alone did not result in the required level of accuracy.
Therefore, the data required further treatment.
2.3. Calibration by statistical models
Four PODs were placed on an iron stand near the sampling points
of selected AQ stations. This operation was carried out between
11–28 March 2017 in Arturo Soria, and between 28 March to 10
April 2017 in Arganda del Rey. Once the operation was finished,
both the data from the reference stations and the data from the
PODs were utilized for the calibration. The calibration of the PODs
was carried out using statistical machine learning methods.
The data treatment was performed using the open source lan-
guage R and using the IDE Rstudio software [30,31].
At first sight, an acceptable correlation was observed between
the station and the POD raw data. Only a small variability between
the data sets was observed. For being low cost sensors, AQmesh per-
formed relatively well, but did not provide the accuracy required
for use by the LIFE PHOTOSCALING Project. Therefore, the data was
treated further.
To carry out the data treatment, a novel regression algorithm
was applied in two steps (see Fig. 1). First, a MLR model was fit-
ted using explanatory variables measured by the sensors (CNO2 ,
CNO , CO3 , T). RH was discarded applying a backward elimination
 J.M. Cordero et al. / Sensors and Actuators B 267 (2018) 245–254 247

Fig. 1. The flux diagram of the regression process.

Fig. 2. A representative diagnostic of the MLR model. The results for POD 1 are shown. The residuals vs. fitted values plot is seen on the left. The histogram of the residuals
is seen on the right.

process due to its high p-value. MLR is a suitable model for electro- Random Forests were calculated using the RandomForest soft-
chemical sensors because their response to gas concentrations is ware package [36]. Random Forests are decision trees combined
linear and the cross-sensitivities are additive. From this model, we with a bagging process. Several decision trees are calculated and
derived a new variable called diff, which is the difference between averaged. The individual trees are computed via samples selected
the model predictions and the measure of the AQmesh itself. The from an initial input set [37]. It was found that using more than 500
new variable diff was used to account for the error between pre- trees did not improve the accuracy of the model in this study case,
dicted and observed data and provided useful information about so we set this hyperparameter to 500.
interference among variables. In the second step, along with the A subsequent complete calibration will be required in the future
same explanatory variables, diff was fed into more sophisticated on a quarterly basis because of the expected drift and changes in
regression methods such as SVMs, RFs and ANNs. meteorological conditions.
Using the Neuralnet software package [32], ANNs were calcu-
lated using an algorithm that had resilient back-propagation and
weight backtracking features [33]. Several ANNs architectures were 2.4. Model evaluation
tested and the one that showed the best fit with a reasonable degree
of complexity was selected. As a result, we used a configuration The model learning process was carried out randomly splitting
including 2 hidden layers, one with 10 nodes and the other with 5 the dataset into two smaller datasets. The training set, used for
nodes. The inputs were scaled to have a mean of zero and standard the model learning and containing the 75% of the data while the
deviation of 1. test set, with the remaining 25% of the data. Once the models were
SVMs were fitted using the e1071 software package [34]. SVMs trained, they were preliminary evaluated using the test dataset. The
are a popular nonparametric technique primarily developed to errors calculated by means of the following equations were used to
solve classification problems. This technique makes use of hyper- validate the models:
planes to search for the maximum separation between classes using n  2
kernel functions. The SVMs were then modified so that they could SSE i=1
y − yi,pred
i
r2 = 1 − =1−  n
(1)
be applied to regression problems [35]. In our case, the regression SST
i=1 (yi
− ȳ)2
problem was linear so a linear kernel was set in the algorithm. Other
authors have used other kernel function to test their impact in the 1 n  2
MSE =  y − yi (2)
performance of their models n i=1 i,pred
248 J.M. Cordero et al. / Sensors and Actuators B 267 (2018) 245–254

Fig. 3. Time series from the Arganda del Rey AQS. The T plot is on the left, and RH plot is on the right. AQS measurements are plotted with a black line. All other plot colours
represent POD measurements.

Fig. 4. Time series of NO2 concentration in Arturo Soria AQS for each POD. Blue lines represent the individual PODs, black lines represent the AQS. (For interpretation of the
references to colour in this figure legend, the reader is referred to the web version of this article.)


100 n yi,pred − yi
1 n  2 MAPE = | | (5)
RMSE = yi,pred − yi (3) n i=1 yi
n i=1

Where:
SSE , is the sum of squared error values
1 n SST , is the total sum of squares
MAE = |yi,pred − yi | (4)
n i=1 yi,pred , is the ith value predicted by the model
 J.M. Cordero et al. / Sensors and Actuators B 267 (2018) 245–254 249

Fig. 5. The top plots represent the Arturo Soria data and the bottom plots represent the Arganda del Rey data. Plots on the left side show a comparison between the “raw”
NO2 measurements from four PODs. Mean concentration values and standard deviations calculated from the POD data and are shown in the plots on the right side.

yi , is the reference ith value The MLR model requirements were validated by diagnosing the
ȳ, is the global mean of the reference data residuals from the first step of the regression procedure (see Fig. 2
n, is the size of the sample for a representative example).
This procedure gave a preliminary insight on the model perfor- The residuals vs. fitted values plot clearly showed constant vari-
mance. Other authors [24] split the dataset into train, validation ance, indicating homoscedasticity. The histograms of the residuals
and test sets for a more accurate evaluation. Our dataset is small resembled normal distributions. We concluded that the data was
to proceed in the same manner. This issue is overcame with the independent and identically distributed. In this way, the MLR mod-
application of the k-fold cross-validation method to the final mod- els were validated.
els. This method consists of splitting the dataset into k folds. Then, In line with AQD guidelines, the relative expanded uncertainty
all but one of them is used for training and the other as test data. The was calculated and plotted vs. the NO2 concentration. The thresh-
models are fitted with this configuration for all the possible com- olds for the relative expanded uncertainty for NO2 are 15% for the
binations among training and testing datasets. Using this method reference and 25% for the indicative methods. This plot served as
allows to calculate RMSE and r2 for k models along with their stan- an indication of the validity of the different models under the con-
dard deviation. dition of a real atmosphere. The following equations were used in
Furthermore, we performed a grid search to find the best hyper- these calculations [39]:
parameters by using the R caret package [38]. This procedure 
RSS 2
changes the values of selected hyperparameters and runs the model 2 (n−2)
− u2 (xi ) + [b0 + (b1 − 1) xi ]
for each one of them to compare the response and to identify the Ur (yi ) = (6)
yi
value that yields an optimal error metric (the lowest RMSE in this
case). We only found significant improvements regarding hyperpa- 2
RSS = (yi − bo − b1 xi ) (7)
rameters discussed in Section 2.3. The remaining hyperparameters
were set to the default values. Ur , is the relative expanded uncertainty.
250 J.M. Cordero et al. / Sensors and Actuators B 267 (2018) 245–254

Table 1
Errors of the linear regression between “raw” and reference data.

POD MSE, (␮g/m3 )2 RMSE, ␮g/m3 MAE, ␮g/m3 MAPE, %

Arturo Soria Arganda del Rey Arturo Soria Arganda del Rey Arturo Soria Arganda del Rey Arturo Soria Arganda del Rey

1 55.9 77.9 7.5 8.8 3.8 7.3 11.3 49.9

2 117.8 73.7 10.9 8.6 8.5 7.3 15.5 14.6
3 45.8 72.2 6.8 8.5 3.4 7.1 30.0 49.6
4 100.1 166.8 10.0 12.9 8.2 10.8 22.9 88.7

Table 2
The r2 of the linear regression between “raw” and reference data sorted according to type of pollutant.

POD NO2 O3 NO

Arturo Soria Arganda del Rey Arturo Soria Arganda del Rey Arturo Soria Arganda del Rey

1 0.89 0.72 0.95 0.95 0.93 0.70

2 0.85 0.65 0.88 0.93 0.70 0.18
3 0.89 0.71 0.93 0.94 0.92 0.66
4 0.83 0.56 0.95 0.94 0.93 0.56

shows the temperature (T) and relative humidity (RH) measured

Fig. 6. Scatter plot of the variables used for the MLR model. Regression lines are
coloured in green, the non-parametric regression smoothers (GAM) are coloured in
red and the confidence intervals for the smoothers are represented by dotted red
lines. Gaseous compounds are measured in ␮g/m3 and temperature is measured
in Celsius. (For interpretation of the references to colour in this figure legend, the
reader is referred to the web version of this article.)
u2 (xi ) , is the random uncertainty of the standard method.
RSS, is the sum of (relative) residuals.
xi , is the average result of the reference method over period i.
yi , is the average result of the model over period i.
xbo , b1 , are the coefficients of the regression y = bo + b1
Finally, the Taylor Diagram was plotted. It represented three
variables: the standard deviation, the centred root mean squared
error, and the r2 coefficient.
The evaluation of the accuracy of the models and the plots were
used to select the best one for each POD.
3. Results and discussion
The data from the AQS reference stations generally corre-
sponded well with the “raw” data from the AQmesh POD. Fig. 3
by the AQS in Arganda del Rey in comparison to the POD data.
The T and RH were comparable, with maximum standard devi-
ations of ± 0.8 ◦ C and ± 4.8%, respectively. The data from the Arturo
Soria AQS gave similar results: maximum standard deviations
of ± 1.2 ◦ C and ± 4.0%, respectively. Therefore, the quality of these
measures was both very high and location independent.
Fig. 4 shows the time series for concentrations of NO2 in Arturo
Soria. The corresponding O3 and NO time series are included as
supplementary material.
As seen in Fig. 4, the POD data had the same trend as the refer-
ence station data. Nevertheless, the absolute values of all the PODs
data were lower. In addition, the data varied from one sensor to
another, a result that indicated inter-comparability issues (Fig. 5).
For example, POD 4, consistently generated the highest values of all
the PODs (Fig. 5). This effect had to be corrected because the success
of our future project will depend on the robustness of the POD mea-
surements. The maximum standard deviation for the Arturo Soria
POD data was ± 46.57 ␮g/m3 and the minimum standard deviation
was ± 1.78 ␮g/m3 (Fig. 5), In the case of Arganda del Rey, the stan-
dard deviation varied from ± 12.70 ␮g/m3 to ± 0.89 ␮g/m3 . These
values were considered high because they were of the same mag-
nitude as the measurements we planned to make.
Eqs. (1)–(5) were used to calculate the statistical errors between
the “raw” POD and the AQS data (Table 1).
The r2 coefficients for “raw” data are sorted according to type of
pollutant in Table 2. In Arganda del Rey, this coefficient was lower
for NO2 and NO. This effect was due to the lower concentrations of
these species in the context of an urban background environment.
The bias and high error observed in the time series meant that
further treatment of POD data was required to correct the unde-
sirable effects of extrinsic and intrinsic variables on the PODs.
Therefore, an MLR model was fitted to the data using a two-step
regression procedure. Fig. 6 shows how well AQS NO2 measure-
ments correlated with POD variables. It represents scatter plots of
the concentrations of NO2 (both reference AQS and AQmesh read-
ings), NO, O3 and T. For example, Fig. 6-e shows the concentration
of NO2 measured by the AQmesh versus the NO2 measured by
the reference AQS. This plot is very useful to identify the type of
relationship among relevant variables.
There was a well-defined linear relationship between reference
AQS NO2 values and the AQmesh gas measurements (CNO2 , CNO ,
CO3 ), (Fig. 6a–c), so they were selected as input features for MLR
models. The temperature variable seemed to have poor explana-
tory performance (Fig. 6d) for this particular dataset. Nevertheless,
this variable was statistically significant at a 95% level of confidence
 J.M. Cordero et al. / Sensors and Actuators B 267 (2018) 245–254 251

Fig. 7. AQmesh NO2 measures vs. AQS for “raw” data and different models. Bands of 95% confidence are seen in blue. The bands of prediction are depicted with red lines.
POD1 is used as the representative example. From this plot, we can see that the bands of the intervals of “raw” data were already greatly reduced when the primary MLR
model was applied. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Table 3 prediction intervals (Fig. 7). As shown in Fig. 7, MLR, SVM, RF and
Representative MLR regression summary. Results from the analysis of POD1 data
ANN seemed to improve the predictions, since the confidence and
are shown.
prediction intervals become narrower in comparison to the “raw”
Parameter Estimate Std. Error t value Pr (>|t|) data. However, some variability was observed: the slopes and the
a 11.5426675 1.47 7.85 7.97 10−14 dispersion of the data points differed from one method to another.
b 0.72700464 0.03 22.75 <2.22 10−17 Due to the variability, further analysis will be needed in order to
c 0.09725934 0.01 9.21 <2.22 10−17 confirm the validity of these models.
d −0.43709431 0.04 −11.90 <2.22 10−17
The error in the predictions decreased in proportion to the
e 1.54167776 0.09 16.44 <2.22 10−17
sophistication of regression methods (Table 4). Thus, the coeffi-
cient of determination r2 improved with respect to the original MLR
model. To further evaluate our models accuracy, k-Fold cross val-
(Table 3), and we decided to keep it in the model to allow consider-
idation method was used and its results are shown in (Table 5)
ing its effect over measured NO2 concentration under a wider range
using four folds. Consistent low RMSE, high r2 values point out the
of values (future campaigns over different seasons of the year). The
robustness of our validation approach.
p-values for all the other variables showed statistical significance
To have a better picture of the accuracy of the statistical models,
so they were included in the MLR model. The following MLR model
Taylor Diagrams for each POD were obtained (see Fig. 8).
was then used:
In the case of POD1, the SVM model performed well because
NO2,predicted = a + bNO2 + cNO + dO3 + eT its variability in terms of standard deviation was similar to the
variability of observed data (Fig. 8). The r2 and RMSE values were
Where a, b, c, d and e are the model coefficients. comparable in all the models. However, the RF model offered the
In fact, all the coefficients, as well as the overall model, were highest r2 and the lowest centred RMSE of all the models. The vari-
statistically significant at a 95% confidence level, as determined by ance of the RF model was low in comparison to the observed data.
t-tests and F-test, respectively (Table 3). In the case of POD2, the RF and SVM models performed better than
As part of the regression procedure, the diff variable was com- the ANN and MLR models. For POD 3, a pattern similar to POD1
puted. SVM, RF and ANN models were fitted using this variable emerged, but this time ANN performed slightly better than SVM.
and the same inputs of the MLR. Predicted versus observed values In the case of POD 4, SVM and RF again gave the highest r2 and
from all four methods were plotted along with 95% confidence and the lowest centred RMSE, but the RF model showed less variance
252 J.M. Cordero et al. / Sensors and Actuators B 267 (2018) 245–254

Table 4
Statistics of error calculated for each of the four PODs and the test set, sorted according to model (MLR, RF, SVM, and ANN).

Model POD* MSE, (␮g/m3 )2 RMSE, ␮g/m3 MAE, ␮g/m3 MAPE, % r2

MLR 1 30.7 5.5 4.4 32.7 0.81

2 53.5 7.3 5.4 12.3 0.93
3 30.7 5.5 4.6 35.4 0.85
4 51.9 7.2 5.9 14.4 0.90
RF 1 18.0 4.3 3.5 26.5 0.90
2 79.1 8.9 6.3 18.5 0.86
3 25.3 5.0 4.1 32.9 0.85
4 46.0 6.8 5.5 16.0 0.95
SVM 1 29.0 5.4 3.9 28.9 0.81
2 61.7 7.9 4.8 24.6 0.89
3 35.8 6.0 4.4 34.3 0.79
4 86.3 9.3 8.2 19.8 0.95
ANN 1 144.3 12.0 11.7 45.9 0.88
2 196.1 14.0 11.1 93.4 0.62
3 36.7 6.1 5.2 40.8 0.88
4 96.9 9.8 8.3 20.2 0.86
*
The errors shown correspond to the future locations of the PODs. The errors from POD1 and POD3 were calculated with respect to the Arganda del Rey calibration,
whereas the errors from POD2 and POD4 were calculated with respect to Arturo Soria calibration.

than the observed data. The variation in the results means that a
model should be chosen for each specific sensor model and condi-
tion under which the sensor is used.
According to the analysis of relative expanded uncertainties
(Fig. 9), the uncertainty varied according to the concentrations
measured. In the case of POD1, POD2 and POD3, the RF model
resulted in the least uncertainty at concentrations below about
40 ␮g/m3 . Above that concentration, ANNs and SVMs had lower
uncertainty, with the exception of the ANN model construction for
POD2. When taking into account the full range of concentrations,
the RF model was considered best for POD4. At above 40 ␮g/m3 and
with the correct choice of model, the uncertainty of AQmesh PODs
was below 25%. These results demonstrate that AQmesh PODs are
valid for use as indicators.

4. Conclusions

In this study, we assessed the performance of four PODs co-

located in different AQSs. The results were positive in terms
of trends and absolute values for O3 and NO2 measurements.
However, the results were less promising in the case of NO mea-
surements. The inter-comparability of the temperature and relative
Fig. 8. Taylor Diagrams for each POD analysed according to each of the four models
(ANN, RF, MLR, and SVM). These plots revealed interesting features that could not humidity variables was excellent. However, each type of sensor
be seen with the previous statistical tools. measured NO2 differently. For example, POD 4 always gave the
highest NO2 values. Importantly, NO2 was to be the principal pollu-
tant studied in the LIFE PHOTOSCALING Project. Therefore, a further
treatment of the data was required improve the reliability of the
Table 5 NO2 measurements.
Statistics of error calculated for each of the four PODs after applying k-fold cross-
We successfully applied an algorithm of machine learning in
validation, sorted according to model (RF, SVM, and ANN).
two steps, the first of which included an MLR fitting. By then sub-
Model POD* RMSE, ␮g/m3 r2 tracting the NO2 predictions from POD measurements, we created a
RF 1 4.88 ± 0.79 0.91 ± 0.02 useful new variable: diff. This variable provided information about
2 8.67 ± 1.92 0.96 ± 0.02 interferences that increase the error. In a second step, this new vari-
3 4.58 ± 0.68 0.91 ± 0.02 able was fed to more complex and non-linear statistical methods
4 7.44 ± 1.23 0.97 ± 0.02
SVM 1 4.09 ± 0.58 0.92 ± 0.02
of regression: RFs, SVMs and ANNs.
2 8.26 ± 0.76 0.95 ± 0.01 The error statistics RMSE, MSE, MAE and MAPE, and the r2 coef-
3 4.16 ± 0.68 0.92 ± 0.03 ficient were used to assess which model was best. However, other
4 5.68 ± 0.46 0.97 ± 0.01 statistical tools, such as Taylor Diagrams were employed to sup-
ANN 1 3.36 ± 0.73 0.96 ± 0.01
port the final decision. The Taylor Diagrams indicated that the SVM
2 6.34 ± 1.05 0.98 ± 0.01
3 3.40 ± 0.53 0.96 ± 0.04 model had the smallest variance with respect to the observed data,
4 7.09 ± 2.30 0.93 ± 0.12 but was not always the model with the highest r2 and the lowest
*
The errors shown correspond to the future locations of the PODs. The errors from
centred RMSE.
POD1 and POD3 were calculated with respect to the Arganda del Rey calibration, Analysing the relative expanded uncertainty was useful in visu-
whereas the errors from POD2 and POD4 were calculated with respect to Arturo alizing which model was best for each POD and for showing how
Soria calibration. the choice of ideal model was dependent on the concentration of
NO2 measured. Generally, SVMs and ANNs performed well for all
 J.M. Cordero et al. / Sensors and Actuators B 267 (2018) 245–254 253

Fig. 9. Relative expanded uncertainties in percent for each POD with respect to each calibration method. The horizontal red line shows the limit of 25% for the indicative
methods. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

the PODs at concentrations higher than 40 ␮g/m3 , but RFs per-
formed better below this threshold. In this study, AQmesh PODs
were shown to be useful indicators at high concentrations because
their relative expanded uncertainties were below 25%.
From our results, we cannot propose a specific model to be gen-
erally used for future applications. Instead, we created base R-code
for a two-stage calibration procedure to automatically select the
best model for each sensor. This approach allows obtaining the
most accurate results and even meet the AQD guidelines regarding
maximum uncertainty for indicative measurements. Consequently,
we deem AQmesh sensors suitable to estimate photocatalytic
efficiencies within the LIFE-PHOTOSCALING Project. Nonetheless,
further research is needed to gain a better understanding of the
performance of the methodology presented under a wider range
of conditions and thus, its applicability for other urban air quality-
related applications.
Acknowledgements
This work has been partially funded by the European LIFE
PHOTOSCALING project entitled “Sustainability of photocatalytic
technologies on urban pavements: from laboratory tests to in field
compliance criteria (LIFE13 ENV/ES/001221)”. This work was a joint
research effort in collaboration with the TECNAIRE-CM (innova-
tive technologies for the assessment and improvement of urban
air quality) scientific programme funded by the Directorate Gen-
eral for Universities and Research of the Greater Madrid Region
(S2013/MAE-2972). We thank the General Directorate of Sus-
tainability (Madrid City Council) and the General Directorate of
Environment (Madrid Greater Region) for granting us access to the
air quality monitoring stations and corresponding data used for cal-
ibration purposes in this study. We also want to thank Envirodata
for lending us one of the PODs used in this work.
254 J.M. Cordero et al. / Sensors and Actuators B 267 (2018) 245–254
Appendix A. Supplementary data
Supplementary data associated with this article can be found, in
the online version, at https://doi.org/10.1016/j.snb.2018.04.021.
References
[1] World Health Organization, Ambient Air Pollution: A global assessment of
exposure and burden of disease, 2016, pp. 1–131 (9789241511353).
[2] C. Guerreiro, A. Gonzalez Ortiz, F. de Leeuw, M. Viana, J. Horalek, Air quality in
Europe — 2016 report, 2016, http://dx.doi.org/10.2800/413142.
[3] Directive 2008/50/EC of the European Parliament and of the Council of 21
May 2008 on ambient air quality and cleaner air for Europe, 2008.
[4] LIFE-MINOx STREET, (n.d.). http://www.lifeminoxstreet.com/life/inicio/.
[5] LIFE-EQUINOX, (n.d.). http://life-equinox.eu/.
[6] LIFE-DIGITALIFE, (n.d.). http://digitalife.active-ceramic.com/.
[7] LIFE-PHOTOCITYTEX, (n.d.). http://www.ceam.es/PHOTOCITYTEX/home.htm.
[8] J.L. Santiago, R. Borge, F. Martin, D. de la Paz, A. Martilli, J. Lumbreras, B.
Sanchez, Evaluation of a CFD-based approach to estimate pollutant
distribution within a real urban canopy by means of passive samplers, Sci.
Total Environ. 576 (2017) 46–58, http://dx.doi.org/10.1016/j.scitotenv.2016.
09.234.
[9] AIRSENSEUR. http://www.airsenseur.org/website/.
[10] A. Kotsev, S. Schade, M. Craglia, M. Gerboles, L. Spinelle, M. Signorini, Next
generation air quality platform: openness and interoperability for the
internet of things, Sensors (Switzerland) 16 (2016) 1–16, http://dx.doi.org/10.
3390/s16030403.
[11] D. Hasenfratz, O. Saukh, S. Sturzenegger, L. Thiele, Participatory air pollution
monitoring using smartphones, in: 2nd Int. Work. Mob. Sens., 2012, pp. 1–5
(//ftp.tik.ee.ethz.ch/pub/people/hdavid/HSST2012.pdf).
[12] G. Miskell, J.A. Salmond, D.E. Williams, Use of a handheld low-cost sensor to
explore the effect of urban design features on local-scale spatial and temporal
air quality variability, Sci. Total Environ. 619 (2018) 480–490, http://dx.doi.
org/10.1016/j.scitotenv.2017.11.024.
[13] L. Spinelle, M. Gerboles, M.G. Villani, M. Aleixandre, F. Bonavitacola, Field
calibration of a cluster of low-cost available sensors for air quality
monitoring. Part A: Ozone and nitrogen dioxide, Sens. Actuators, B Chem. 215
(2015) 249–257, http://dx.doi.org/10.1016/j.snb.2015.03.031.
[14] R. Baron, J. Saffell, Amperometric gas sensors as a low cost emerging
technology platform for air quality monitoring applications: a review, ACS
Sens. 2 (2017) 1553–1566, http://dx.doi.org/10.1021/acssensors.7b00620.
[15] N. Castell, F.R. Dauge, P. Schneider, M. Vogt, U. Lerner, B. Fishbain, D. Broday,
A. Bartonova, Can commercial low-cost sensor platforms contribute to air
quality monitoring and exposure estimates? Environ. Int. 99 (2017) 293–302,
http://dx.doi.org/10.1016/j.envint.2016.12.007.
[16] M.C. Carotta, G. Martinelli, L. Crema, M. Gallana, M. Merli, G. Ghiotti, E.
Traversa, Array of thick film sensors for atmospheric pollutant monitoring,
Sens. Actuators B Chem. 68 (2000) 1–8, http://dx.doi.org/10.1016/S0925-
4005(00)00439-1.
[17] C. Borrego, A.M. Costa, J. Ginja, M. Amorim, M. Coutinho, K. Karatzas, T.
Sioumis, N. Katsifarakis, K. Konstantinidis, S. De Vito, E. Esposito, P. Smith,
P.G.N. Andr, W. Reimringer, R.P. Otjes, O. Von Sicard, R. Pohle, B. Elen,
Assessment of air quality microsensors versus reference methods: the
EuNetAir joint exercise, Atmos. Environ. 147 (2016) 246–263, http://dx.doi.
org/10.1016/j.atmosenv.2016.09.050.
[18] N. Masson, R. Piedrahita, M. Hannigan, Approach for quantification of metal
oxide type semiconductor gas sensors used for ambient air quality
monitoring, Sens. Actuators B. Chem. 208 (2015) 339–345, http://dx.doi.org/
10.1016/j.snb.2014.11.032.
[19] M. Kamionka, P. Breuil, C. Pijolat, Calibration of a multivariate gas sensing
device for atmospheric pollution measurement, Sens. Actuators B Chem. 118
(2006) 323–327, http://dx.doi.org/10.1016/j.snb.2006.04.058.
[20] JRC Technical report, Protocol of evaluation and calibration of low-cost gas
sensors for the monitoring of air pollution, 2013. doi: 10.2788/9916.
[21] L. Spinelle, M. Gerboles, M.G. Villani, M. Aleixandre, F. Bonavitacola, Field
calibration of a cluster of low-cost commercially available sensors for air
quality monitoring. Part B: NO, CO and CO2 , Sens. Actuators B Chem. 238
(2017) 706–715, http://dx.doi.org/10.1016/j.snb.2016.07.036.
[22] S. De Vito, E. Esposito, M. Salvato, O. Popoola, F. Formisano, R. Jones, G. Di
Francia, Chemical calibrating chemical multisensory devices for real world
applications: an in-depth comparison of quantitative machine learning
approaches, Sens. Actuators B Chem. 255 (2018) 1191–1210, http://dx.doi.
org/10.1016/j.snb.2017.07.155.
[23] G. Huyberechts, P. Szeco, Simultaneous quantification of carbon monoxide
and methane in humid air using a sensor array and an artificial neural
network, Sens. Actuator B-Chem. 45 (1997) 123–130.
[24] E. Esposito, S. De Vito, M. Salvato, V. Bright, R.L. Jones, O. Popoola, Dynamic
neural network architectures for on field stochastic calibration of indicative
low cost air quality sensing systems, Sens. Actuators B Chem. 231 (2016)
701–713, http://dx.doi.org/10.1016/j.snb.2016.03.038.
[25] K. Brudzewski, Gas analysis system composed of a solid-state sensor array
and hybrid neural network structure, Sens. Actuators B 55 (1999) 38–46,
http://dx.doi.org/10.1016/S0925-4005(99)00040-4.
[26] A.C. Lewis, J.D. Lee, P.M. Edwards, M.D. Shaw, M.J. Evans, S.J. Moller, K.R.
Smith, J.W. Buckley, M. Ellis, S.R. Gillot, A. White, Evaluating the performance
of low cost chemical sensors for air pollution research, Faraday Discuss. 189
(2016) 85–103, http://dx.doi.org/10.1039/c5fd00201j.
[27] LIFE-PHOTOSCALING. http://www.life-photoscaling.eu.
[28] Alphasense. http://www.alphasense.com/.
[29] AQmesh. http://www.aqmesh.com/.
[30] CRAN. https://cran.r-project.org/.
[31] Rstudio. https://www.rstudio.com/.
[32] S. Fritsch, F. Guenther, Neuralnet: Training of Neural Networks, 2016
(https://cran.r-project.org/package=neuralnet).
[33] M. Riedmiller, Advanced supervised learning in multi-layer perceptrons —
from backpropagation to adaptive learning algorithms, Comput. Stand.
Interfaces 16 (1994) 265–278, http://dx.doi.org/10.1016/0920-
5489(94)90017-5.
[34] D. Meyer, E. Dimitriadou, K. Hornik, A. Weingessel, F. Leisch, Misc Functions of
the Department of Statistics, Probability Theory Group (Formerly: E1071), TU
Wien, 2017, pp. e1071 (https://cran.r-project.org/package=e1071).
[35] H. Drucker, C.J.C. Burges, L. Kaufman, A. Smola, V. Vapnik, Support vector
regression machines, in: M.C. Mozer, M.I. Jordan, T. Petsche (Eds.), Adv.
NEURAL Inf. Process. Syst. 9 Proc. 1996 Conf., MIT Press, Five Cambridge
Center, Cambridge, MA 02142, 1997, pp. 155–161.
[36] A. Liaw, M. Wiener, Classification and Regression by random Forest, 2002
(http://cran.r-project.org/doc/Rnews).
[37] L. Breiman, Random forests, Mach. Learn. 45 (2001) 5–32, http://dx.doi.org/
10.1023/A:1010933404324.
[38] M. Kuhn, Caret: Classification and Regression Training, 2017 (https://cran.r-
project.org/package=caret).
[39] EC working group, Guide To The Demonstration Of Equivalence Of Ambient
Air Monitoring Methods, 2010.
Biographies
Dr José María Cordero is a PhD researcher in the Department of Chemical and Envi-
ronmental Engineering at the Technical University of Madrid (UPM). He completed
his doctorate in the INCAR-CSIC, Spain, studying the sulphation kinetics and capaci-
ties of CaO in Calcium Looping reactors to capture CO2 . He is currently investigating
the sustainable use of photocatalytic pavements for NO2 mitigation in the City of
Madrid.
Dr Rafael Borge is an Associate Professor in the Department of Chemical and
Environmental Engineering at the Technical University of Madrid (UPM). He has
extensive experience on air quality modelling and air quality management. He
currently leads the Laboratory of Environmental Modelling and coordinates the
TECNAIRE-CM research project that aims at the development and integration of
new technologies and methods for multiscale modelling and measuring of urban air
pollution.
Dr Adolfo Narros is an Associate Professor in the Department of Chemical and Envi-
ronmental Engineering at the Technical University of Madrid (UPM). His expertise is
in chemical analysis of air pollutants and in the use of sensors in the measurement
of the air quality.