Metrics for Performance Prediction

of Wireless Sensor Networks

Frank Oldewurtel and Petri M ah onen
Institute for Networked Systems, RWTH Aachen University
Kackertstrasse 9, D-52072 Aachen, Germany
Email: fol@inets.rwth-aachen.de
AbstractWe present the derivation of a novel metric for the
performance prediction of Wireless Sensor Networks (WSNs). In
particular, the metric can be applied in, for example, monitoring
and surveillance applications where WSNs are used. Those
networks execute protocols or techniques that are often sensitive
to spatial correlation. The proposed metric is based on the
correlation in the sensed phenomenon and the correlation in the
location of the sensor nodes. The main application area of the
performance prediction metric lies in the design and optimisation
of WSNs prior to the costly deployment phase. Since extensive
simulations can be completely avoided it serves as a rapid and
lightweight evaluation tool for comparative analysis of WSNs.
The concept is applicable and also extensible through merging
of selected performance criteria.
I. INTRODUCTION
Wireless Sensor Networks (WSNs) can greatly enhance
our capability to control and monitor physical phenomena.
Typically, they consist of a large number of nodes, which
are usually resource-constrained sensing devices. Sensor nodes
are battery-operated and have very low capabilities in terms
of computation and communication. A major criterion to
characterise the performance and the operational lifetime of
WSNs is their ability to save energy. Hence, energy con-
sumption is the paramount performance criterion considered.
Depending on the application other criteria can be included
in the overall performance evaluation. Many known phenom-
ena, for example, forest re or rainfall areas, exhibit certain
subregions or a collection of interesting locations that are
more important than other regions to the application user.
Those interesting points may represent local deviations of the
phenomena and can be seen as activities or events. Often most
of the information regarding phenomena can be obtained and
analysed by monitoring such sets of interesting points. In such
scenarios the effectiveness of WSNs is often determined to a
large extent by the sensing coverage provided by the actual
node deployment.
Performance prediction of WSNs is a difcult but highly
important task. Extensive simulations using environments with
increasing level of detail is the primary approach for tackling
this task. The major drawbacks are signicant implementation
efforts and long simulation times. Hence, the goal of our
approach is to overcome this situation by deriving a metric
that enables rapid and lightweight prediction of the overall
performance under selected criteria. Similar to simulations,
this metric makes use of input data such as the phenomenon
under study and the chosen network topology. In contrast to
simulations, the proposed metric allows quick and lightweight
evaluation by exploiting two concepts borrowed from the eld
of spatial statistics.
Recently there has been work on the estimation of the
phenomenon itself facing the energy-distortion trade-off, see,
for example [1]. Furthermore, in the area of power proling
sensor nodes have been investigated using predicted opera-
tion states and their associated state holding times, see, for
example [2]. Within our scope it has been shown that the
correlation in both the phenomenon data and the locations
of the nodes strongly affect energy consumption, operational
lifetime and sensing coverage [3][6]. However, to the best of
our knowledge there is no approach reported that addresses
the performance prediction of correlation-sensitive WSNs as
proposed in our work. In this paper, we derive the performance
prediction metric considering energy consumption as the major
performance criterion. In addition, we extend this metric by
including sensing coverage as second criterion exemplarily.
Furthermore, we evaluate the proposed concept through exten-
sive simulations and apply the performance prediction metric
to our use cases.
The remainder of the paper is structured as follows. In
Section II the reference system model is briey described.
Section III introduces two concepts borrowed from the eld
of spatial statistics. In Section IV we derive the performance
prediction metric and evaluate the proposed metric considering
energy consumption as major criterion. Section V extends
the proposed metric by taking into account sensing coverage
as second performance criterion. Finally, Section VI draws
conclusions.
II. REFERENCE SYSTEM MODEL
In this section we motivate the reference system model
which consists of ve components.
A. Phenomenon model and Deployment model
The physical phenomena monitored by WSNs, e.g., temper-
ature and light intensity usually yields sensed data exhibiting
strong correlations in the spatial domain [7], [8]. In our studies
we make use of synthetically generated and spatially correlated
data elds h(x, y) [9]. The model used is independent on the
node density, the number of nodes or the topology. The gener-
ated data sampled from the model shows good correspondence
when statistically compared with experimental data.
The node deployment model consists of the specication of
the total number of nodes N, and the coordinates (x
i
, y
i
)
N
i=1
of the individual nodes. We assume a xed region A with
area |A| and that the coordinates are dened by a random
point process (PP) [10]. PPs were successfully applied to
deployment modeling in, for example, [3], [6].
B. Communication model
In terms of communication we assume that each sensor
node has an omni-directional transmission range and utilizes
erroneous links. For modeling error characteristics a widely
used model is the Gilbert-Elliot bit error model [11]. This
model is fundamentally based on a two-state Markov model
that takes bit error bursts into account. In the case of packet
errors, generated by this model, retransmissions are initiated.
The maximum number of packet retransmissions is based
on the IEEE 802.15.4 specication and is thus set to three.
The physical parameters such as transmission ranges, packet
header size and payload size were determined by the Telos
platform running the embedded operating system TinyOS 2.x.
We apply Distributed Source Coding (DSC) achieving high
energy-efciency through exploitation of spatial correlations
in the sensed phenomena [12], [13]. In terms of mobility the
sensor nodes are quasi-stationary at known positions and the
overall network density is constant in all cases. Regarding
cluster head selection we focus on the closest-to-center of
gravity scheme. Throughout the scenarios the shortest path
multihop routing protocol is applied.
C. Observation model and Energy model
The observation model s(x, y) describes the observations
or measurements of the sensed phenomena at the node loca-
tions and can therewith directly and explicitly formulated as
s(x, y)=h(x
i
, y
i
) for i {1, . . . , N} and undened otherwise.
We apply the energy model e(x
i
, y
i
) introduced in our
previous work [3]. The model takes into account packet header
overhead and the additional energy consumption due to the
DSC-related signal processing such as encoding, joint decod-
ing and entropy tracking. The entropy tracking algorithm esti-
mates the underlying joint probability density function (PDF)
of the sensor readings observed by the node pair. The energy
model is essentially based on measurements obtained from real
experiments using the Telos sensor node platform [12].
III. SPATIAL STATISTICS
We shall now outline the techniques used to characterise
correlations both in the spatial phenomena being sensed by
the WSN, and in the locations of the nodes themselves. We
denote the value of the phenomena under study at x R
2
by
h(x). In order to characterise h using statistics, we assume it to
be a realisation of a random eld, that is, a stochastic process
dened on a region of the plane. For most physical phenomena
of interest, such as humidity, atmospheric pressure or the
strength of the magnetic eld, nearby samples of h are heavily
correlated. We can characterise these correlations by means of
the second-order semivariogram (also called semivariance) [9]
(s t) =
1
2
Var{h(s) h(t)}. (1)
The reader should note that the value of the semivariogram as
a function of the distance |st| is sufcient for stationary and
isotropic h. In our context the two-dimensional isotropic case
is assumed in which the semivariogram depends only upon the
Euclidean distance r between any two locations.
To facilitate comparison to the correlations in node loca-
tions, we use instead of the semivariogram the correlogram
dened by
C(s, t) E{h(s)h(t)} (s)(t), (2)
where (s) denotes the mean value of h at s. The be-
haviour of the correlogram is opposite to that of the semi-
variogram. Indeed, the semivariogram and the correlogram
are related for sufciently well-behaved h by the relation
(h) = C(0) C(h).
The correlations between measurements of h depend not
only on correlations in h (characterised by the correlogram)
but also on correlations in the locations of the sensor nodes
carrying out the measurements. Suppose the nodes have overall
area density
A
. Intuitively, sensor nodes grouped together
(high location correlations) measure more highly correlated
values of the phenomena than nodes deployed on, say, a grid.
Such correlations in node locations can be measured by the
pair correlation function g(r), dened in terms of the joint
probability density
dP =
2
A
g(r) dA
1
dA
2
(3)
of nding one point in each of the two area elements dA
1
and dA
2
separated by the distance r. In the case of totally
random locations we obviously have g(r)=1. High values of
g(r) indicate clustering at the relevant length scale, whereas
values below one indicate regularity (as is the case for grid,
for example). Integrating the pair correlation function from
zero to a distance r directly yields the expected number of
neighbors at distance r for an arbitrary node in the network.
It is thus clearly related to the level of correlation exhibited by
the measurements of the nodes in the network. Pair correlation
functions have been also successfully used to characterise
wireless networks in [14].
IV. PERFORMANCE PREDICTION METRIC
Simulation work in the context of correlation-sensitive
WSNs takes into account the phenomena under study and
the chosen network topology as major input data. While
the correlogram characterises the correlation structure in the
phenomena data the pair correlation function describes the
correlation in the node locations. Considering average energy
consumption

E as the performance criterion we thus argue that
the performance of WSNs can be expressed as a linear function
of the form e = k f(C(r), g(r)), where k is a constant.
0 10 20 30 40
0
5
10
15
20
25
30
distance r
c
o
r
r
e
l
a
t
i
o
n


d
ph
= 10
d
ph
= 9
d
ph
= 7
d
ph
= 5
d
ph
= 3
Fig. 1. Correlograms of phenomena exhibiting different correlation distances.
Figure 1 depicts the (empirical) correlograms of the consid-
ered phenomena exhibiting varying correlation distance d
ph
.
If the distance between data elements of the phenomena is
more than d
ph
, then they cannot be directly derived from each
other. Hence, we expect that by decreasing d
ph
the correlogram
will converge earlier. In fact, we observe that the shape (in
particular the slope and the convergence behaviour) of the
curve changes as expected dependent on d
ph
. We see that the
correlogram decreases overall in value as distance is increased,
corresponding to the intuition that far away values are less
correlated than nearby ones.
The pair correlation functions of various topologies are
depicted in Figure 2. The topologies are sampled from two
types of cluster point processes [10]. The Thomas PP is based
on a Poisson PP (uniform random distribution) of intensity
T
which is used to generate cluster centers. Then each parent or
cluster center point is replaced by a cluster of points. The num-
ber of points in each cluster is a Poisson distributed random
variable with mean value
T
. The locations of nodes in each
cluster are sampled from a normal distribution with variance

2
T
and the mean located at the cluster center. Another cluster
process is the Matern PP. As for the Thomas PP, the number
of parent points are distributed according to a Poisson PP with
intensity
M
. The number of cluster members in each cluster
is also sampled from a Poisson distribution with mean
M
.
The only difference lies in how the cluster points themselves
are distributed. While for the Thomas PP a normal distribution
is used, the cluster points of a Matern PP are uniformly
distributed over a disc of radius R
M
with the respective parent
point as center. Again, the parent points do not occur in the
resulting realisation of the PP. In our simulations we used the
parameters (
T
,
T
) =(
M
,
M
) =(8.4 10
4
, 12) while the
third parameter,
T
in the Thomas case and R
M
in the Matern
case, controls the cluster spread.
From our analysis we derive the inner product as candi-
date solution for the linear function leading to the proposed
performance prediction metric dened as
=

C(r), g(r)
_
. (4)
0 5 10 15 20 25
0
2
4
6
8
10
12
distance r
p
a
i
r

c
o
r
r
e
l
a
t
i
o
n


Matern, R
M
= 11
Matern, R
M
= 14
Thomas,
T
= 3
Thomas,
T
= 4
Thomas,
T
= 5
Fig. 2. Pair correlation functions of the considered topologies; sampled from
different types of random point processes with varying cluster spread.
Maximising leads to large metric values indicating minimal
energy consumption. For evaluation we conducted extensive
simulations (2000 runs each) obtaining average energy con-
sumption values

E and computed the respective -metric
values using equation 4. Figure 3 shows the energy consumed
vs. the prediction metric. Interconnected data points belong
to phenomena with identical d
ph
. Markers indicate specic
topologies. From Figure 3 we observe that the energy con-
sumption is directly related to the performance prediction
metric, i.e.

E . This interesting relation holds across all
considered topologies with arbitrary but xed d
ph
. Further-
more, Figure 4 is similar to Figure 3 but in contrast shows
interconnected data points which belong to arbitrary but xed
topologies deployed on phenomena with varying d
ph
. We have
found that

E is again directly related to for any particular
topology across phenomena exhibiting different d
ph
.
As a result, the inner product can be used as a lightweight
and powerful metric for the performance prediction of WSNs.
It is based on C(r) and g(r) solely. While only the correla-
tion structure of the phenomenon needs to be estimated the
pair correlation functions of candidate topologies need to be
stored using a minimal amount of memory. Using prior
to the costly WSN deployment is a lightweight approach to
signicantly improve the cost-efciency of the system. Since
extensive simulations can be completely avoided during the
system design phase this metric enables rapid comparative
performance evaluation of various deployment strategies.
V. EXTENDED METRIC AND ITS APPLICATION
Depending on the application other performance criteria can
play an important role. Since the effectiveness of WSNs is
often dependent on the sensing coverage we extend the per-
formance prediction metric by considering sensing coverage as
second criterion. Full sensing coverage (100 %) is achieved if
the deployed nodes cover at least the collection of interesting
locations of the phenomenon. We denote all those locations
as the set of points of interest (POI). POI is essentially
a point cloud which is selected based on signicant local
0 200 400 600 800 1000
550
600
650
700
750
800
850
900
performance prediction metric
a
v
e
r
a
g
e

e
n
e
r
g
y

c
o
n
s
u
m
e
d

[

J
]


Matern, R
M
= 11
Matern, R
M
= 14
Thomas,
T
= 3
Thomas,
T
= 4
Thomas,
T
= 5
d
ph
= 3
d
ph
= 7
d
ph
= 5
d
ph
= 10
d
ph
= 9
Fig. 3. Direct relationship between performance prediction metric and
energy consumption according to varying correlation distance d
ph
; markers
indicate specic topologies.
deviations of the actual neighbouring phenomenon values.
Those deviations represent the highest information content and
are assumed to be most important to the application user. It
is noteworthy that nodes deployed in the area that do not
cover the POI still provide less but useful information. In
order to identify the POI we particularly apply the Laplace
operator and low-pass ltering to the entire phenomenon.
The Laplace operator on the function f is in the two-
dimensional cartesian case a second order differential operator
of the form f =
2
f =

2
f
x
2
+

2
f
y
2
. From the POI we can
derive a priori information which can be incorporated into the
random deployment models in order to improve the sensing
coverage of WSNs. The analysis of enhanced deployment
models exploiting a priori information has been presented in
our previous works [6], [15] and is not the focus of this paper.
However, in Table I we include a fraction of our results
adopted from [6] to be self-contained. Focusing on phenomena
with d
ph
=10 exemplarily, Table I lists simulation results (2000
runs) such as average energy consumption

E and average
sensing coverage v in respect to various deployment models.
The parameters of the models where estimated from the POI
using the minimum contrast method [16]. The term parent-
modied refers to the case where the parent point locations
(cluster centers) are restricted to lie on the rst principal
component of the POI. In addition, the term non-symmetric
refers to case where the shape of the node distribution in each
cluster is dependent on the shape factor . Both options are
examples of how phenomena-related a priori information can
be exploited in order to improve the sensing coverage of the
deployment.
From our analysis, we nd that the degree of similarity
between the PDFs of the POI and the chosen deployment
model is directly related to sensing coverage. As similarity
measures to estimate the distance between distributions we
take into account the Kullback-Leibler divergence [17] and
the Hellinger metric [18]. The symmetric Kullback-Leibler
0 200 400 600 800 1000
550
600
650
700
750
800
850
900
performance prediction metric
a
v
e
r
a
g
e

e
n
e
r
g
y

c
o
n
s
u
m
e
d

[

J
]


Matern, R
M
= 11
Matern, R
M
= 14
Thomas,
T
= 3
Thomas,
T
= 4
Thomas,
T
= 5
Fig. 4. Direct relationship between performance prediciton metric and
energy consumption according to various topologies.
divergence of two PDFs p and q is dened as
KL(p, q) = D(p||q) +D(q||p), with (5)
D(p||q) =

p(x) log
_
p(x)/q(x)
_
.
Furthermore, the Hellinger metric is dened as
HM =
_

j
_
_
p(x
j
)
_
q(x
j
)
_
2
_1
2
. (6)
It is preferred to not rely on similarity measures that operate
on PDFs solely since those may be misleading. Hence, we
use in addition the Kolmogorov-Smirnov statistic [19] and the
Area metric which operate on cumulative distribution functions
(CDFs). The weighted Kolmogorov-Smirnov metric is dened
as
KS = max
x
_
|I(x) M(x)|/
_
M(x)
_
1 M(x)
_
_
, (7)
where I is the CDF of the POI and M is the CDF of the
chosen deployment model. Furthermore, the Area metric Q as
applied in, for example, [20] is dened as
Q(I, M) =
1
J
J

j=1

log(I
1
(j/J)) log(M
1
(j/J))

log(I
1
(1/J)) log(M
1
(1/J))

2J

log(I
1
(1)) log(M
1
(1))

2J
,
(8)
where J denotes the size of the observation window. Since
an increasing degree of similarity of two distribution func-
tions implies decreasing values in the distance between those
functions we denote the similarity measure in the case of
PDF-based measures =1/KL and =1/HM, respectively.
Similarly in the case of CDF-based measures it follows
=1/KS and =1/Q, respectively. The extended prediction
metric can then be dened as
= , (9)
TABLE I
AVERAGE ENERGY CONSUMPTION

E AND SENSING COVERAGE v
No. Model description

E[J] v[%]
1 Poisson (
P
= 0.0313) 1066.20 2.99
2 Thomas (
T
,
T
,
T
)(1.1210
3
, 27.9, 5.0) 797.43 3.29
5 Parent-mod. non-sym. Thomas ( = 2) 802.37 9.10
6 Parent-mod. non-sym. Thomas ( = 3) 804.03 9.23
7 Matern (
M
,
M
, R
M
)(1.1510
3
, 27.1, 9.4) 678.40 3.50
10 Parent-mod. non-sym. Matern ( = 2) 626.27 9.54
11 Parent-mod. non-sym. Matern ( = 3) 628.55 10.53
where denotes the introduced performance prediction metric
according to energy consumption. For the evaluation of the
deployment strategies we develop the -metric
= v/

E, (10)
which takes into account simulation results such as energy
consumption

E and sensing coverage v.
Figure 5 shows the direct relationship between the extended
prediction metric and the -metric considering the deploy-
ment strategies achieving largest v-values. Markers indicate
the topology type 5, 6, 10 and 11 from Table I. We can see
the behaviour of all considered similarity measures in respect
to the best performing deployment strategies. While all curves
behave similarly we observe that the CDF-based curves allow
a more clear distinction. Figure 5 shows that the extended
prediction metric can be used instead of the -metric which
captures the overall performance of WSNs from simulations
viewpoint. Thus, the concept of the performance prediction
metric is applicable and can also be extended through merging
of selected performance criteria.
VI. CONCLUSIONS
In this paper we presented the derivation of a novel metric
for the performance prediction of Wireless Sensor Networks
(WSNs). The proposed metric is based on the correlation in
the sensed phenomenon and the correlation in the location of
the sensor nodes. The main application area of the perfor-
mance prediction metric lies in the design and optimization of
WSNs prior to the costly deployment phase. Since extensive
simulations can be completely avoided it serves as a rapid
and lightweight evaluation tool for comparative analysis of
WSNs. We have shown that this concept is applicable and also
extensible through merging of selected performance criteria.
ACKNOWLEDGMENT
This work was nancially supported by the German Re-
search Foundation (DFG) through the UMIC-research centre
at the RWTH Aachen University. We would like to thank Janne
Riihij arvi for stimulating discussions on spatial statistics.
REFERENCES
[1] W. Bajwa, J. Haupt, A. Sayeed, and R. Nowak, Joint Source-Channel
Communication for Distributed Estimation in Sensor Networks, IEEE
Trans. on Information Theory, vol. 53, no. 10, pp. 36293653, 2007.
[2] V. Shnayder, M. Hempstead, B. Chen, G. W. Allen, and M. Welsh,
Simulating the power consumption of large-scale sensor network ap-
plications, in Proceedings of the SENSYS, Baltimore, USA, 2004, pp.
188200.
11 12 13 14 15 16 17
10
3.2
10
3.4
10
3.6
10
3.8
metric 1000

m
e
t
r
i
c


KL
HM
KS
Q
type 5
type 6
type 10
type 11
Fig. 5. Relation between and -metric in respect to the considered
similarity measures; markers indicate the topology type.
[3] F. Oldewurtel and P. M ah onen, Efciency Analysis and Derivation of
Enhanced Deployment Models for Sensor Networks, Int. Journal of Ad
Hoc and Ubiquitous Computing (IJAHUC), 2010, accepted.
[4] Y. Zou and K. Chakrabarty, Sensor Deployment and Target Localization
in Distributed Sensor Networks, ACM Trans. on Embedded Computing
Systems (TECS), vol. 3, no. 1, pp. 6191, 2004.
[5] G. Wang, G. Cao, and T. L. Porta, Movement-Assisted Sensor Deploy-
ment, IEEE Trans. on Mobile Computing, vol. 5, no. 6, pp. 640652,
2006.
[6] F. Oldewurtel and P. M ah onen, Analysis of Enhanced Deployment
Models for Sensor Networks, in Proceedings of the VTC spring, Taipei,
Taiwan, 2010, accepted.
[7] S. Pattem, B. Krishnamachari, and R. Govindan, The Impact of Spatial
Correlation on Routing with Compression in Wireless Sensor Networks,
ACM Trans. on Sensor Networks (TOSN), vol. 4, no. 4, pp. 133, 2008.
[8] F. Oldewurtel, J. Ansari, and P. M ah onen, Cross-Layer Design for Dis-
tributed Source Coding in Wireless Sensor Networks, in Proceedings
of the SENSORCOMM, Cap Esterel, France, 2008, pp. 435443.
[9] A. Jindal and K. Psounis, Modeling Spatially Correlated Data in Sensor
Networks, ACM Trans. on Sensor Networks (TOSN), vol. 2, no. 4, pp.
466499, 2006.
[10] D. Stoyan, W. S. Kendall, and J. Mecke, Stochastic Geometry and its
Applications. Wiley, 1995.
[11] J.-P. Ebert, A. Willig, and A. Wolisz, A Gilbert-Elliot Bit Error Model
and the Efcient Use in Packet Level Simulation, in TKN technical
report TKN-99-002, 1999.
[12] F. Oldewurtel, M. Foks, and P. M ah onen, On a Practical Distributed
Source Coding Scheme for Wireless Sensor Networks, in Proceedings
of the VTC spring, Marina Bay, Singapore, March 2008, pp. 228232.
[13] F. Oldewurtel, J. Riihij arvi, and P. M ah onen, Efciency of Distributed
Compression and its Dependence on Sensor Node Deployments, in
Proceedings of the VTC spring, Taipei, Taiwan, 2010, accepted.
[14] J. Riihij arvi, P. M ah onen, and M. R ubsamen, Characterizing Wire-
less Networks by Spatial Correlations, IEEE Communications Letters,
vol. 11, no. 1, pp. 3739, 2007.
[15] F. Oldewurtel and P. M ah onen, Estimation and Evaluation of Deploy-
ment Models for Sensor Networks, in Proceedings of the VTC fall,
Anchorage, USA, 2009, pp. 15.
[16] P. J. Diggle and R. J. Gratton, Monte Carlo Methods of Inference for
Implicit Statistical Models, Journal of the Royal Statistical Society,
vol. 46, no. 2, pp. 193227, 1984.
[17] T. M. Cover and J. A. Thomas, Elements of Information Theory. Wiley,
2006.
[18] A. Ferrante, M. Pavon, and F. Ramponi, Hellinger versus Kullback-
Leibler Multivariable Spectrum Approximation, IEEE Trans. on Auto-
matic Control, vol. 53, no. 4, pp. 954967, 2008.
[19] F. Massey, The Kolmogorov-Smirnov test for Goodness of Fit, Journal
of the American Statistical Assoc., vol. 46, no. 253, pp. 6878, 1951.
[20] M. Wellens, J. Riihij arvi, and P. M ah onen, Empirical Time and Fre-
quency Domain Models of Spectrum Use, Elsevier Physical Commu-
nication, vol. 2, no. 1-2, pp. 1032, 2009.