35th Annual IEEE Conference on Local Computer Networks
TestbedProfiler: A Validation Tool for Wireless
Sensor Network Testbed Deployment
Chad S. Metcalf Tracy Camp
Arch Rock Corporation Colorado School of Mines
metcalfc@gmail.com tcamp@mines.edu
Abstract—We present TestbedProfiler, a wireless sensor net-
work application suite developed to help guide the installation
of WSN testbeds. TestbedProfiler can be used to assist the
deployment of a WSN testbed by evaluating proposed mote
locations in terms of connectivity and signal strength.
We developed TestbedProfiler and used it to evaluate the
Casino Lab and the Edgar Mine testbeds owned by the Colorado
School of Mines. We found TestbedProfiler to be instrumental in
understanding how the motes communicate and how the testbeds
could be improved. We believe that TestbedProfiler can improve
testbed deployments, leading to improved real-life wireless sensor
network applications.
Index Terms—WSN testbed; WSN testbed deployment; wire-
less sensor network testbed
I. I NTRODUCTION
A thorough evaluation of a wireless sensor network proto-
col or application requires a combination of simulation and
implementation. While simulation [1] enables the exploration
of a large parameter space, implementation on a real-world
testbed enables the evaluation of a protocol or application in
a real network environment; however, just as it is improper to
simply simulate a protocol, collect results, and declare victory,
it is improper to do the same on a testbed for the same reason.
Borrowing a concept from software engineering, we assert that
the deployment of a testbed must be verified and validated.
According to the Capability Maturity Model Integrated [2],
“Verification confirms that work products properly reflect the
requirements specified for them.” In other words, verification
ensures that you built the testbed correctly (e.g., power,
programming interface, back-channel communications). “Val-
idation confirms that the product, as provided, will fulfill its
intended use.” In other words, validation ensures that you built
the correct testbed (e.g., that you can obtain a desired average
neighbor count).
Many of today’s wireless sensor network testbeds are only
verified. But, even though the testbed is able to transmit
packets, it does not mean that the testbed accurately reflects
the intended environment.
Our TestbedProfiler application suite addresses these chal-
lenges by enabling a testbed designer to easily understand the
communication environment of his/her testbed. TestbedProfiler
analyzes mote connectivity, packet delivery rates, average
neighbor counts, and typical received signal strength indicator
(RSSI) and link quality indicator (LQI) values. Testbed design-
ers can then use the information provided by TestbedProfiler to
978-1-4244-8389-1/10/$26.00 ©2010 IEEE
Authorized licensed use limited to: ULB Darmstadt. Downloaded on December 09,2022 at 18:48:35 UTC from IEEE Xplore. Restrictions apply.
LCN 2010, Denver, Colorado
Michael Colagrosso Oliver Chase
Labjack Corporation Colorado School of Mines
mike.colagrosso@gmail.com ochase@mines.edu
adjust the deployed mote positions and achieve an environment
that more closely reflects the intended environment, in other
words validating his/her testbed.
This paper describes our TestbedProfiler application suite, as
well as our experiences using TestbedProfiler with the Casino
Lab and Edgar Mine testbeds of the Colorado School of Mines.
In Section II, we present related tools, and in Section III we
provide a technical overview of our TestbedProfiler application
suite. We then discuss our experiences using TestbedProfiler
with Casino Lab in Section IV and the Edgar Mine in
Section V. Finally, we present our conclusions in Section VI.
II. R ELATED W ORK
The most closely related tool to TestbedProfiler is SCALE
[3], a communication assessment tool considering packet de-
livery rates. TestbedProfiler outputs packet send rate as well,
but is primarily interested in overall connectivity and neighbor
counts. In addition, SCALE was written only for testbeds using
Mica1 and Mica2 hardware. TestbedProfiler was written for
any mote hardware that supports the Chipcon CC2420 radio
[4], which includes the two most common mote types used in
today’s testbeds (i.e., Telos and MicaZ motes).
MoteLab’s similar Connectivity Daemon collects radio con-
nectivity between motes and presents this information to the
user through a web interface. The tool is primarily intended to
visualize testbeds after deployment and requires that MoteLab
be installed on the testbed. TestbedProfiler is designed to be
used during deployment, has no installation, and provides a
broader range of statistics.
Lastly, SeeDTV [5] is a recently developed validation tool
that has different goals than TestbedProfiler. SeeDTV is a
handheld tool to use while walking around a deployment, to
visualize what is happening at a specific location. It is used for
debugging a deployed sensor network, while TestbedProfiler
is designed to visualize the network as a whole, specifically
during design and deployment (instead of during support and
diagnostics). Additionally, a primary goal of TestbedProfiler
is to validate that the profile of a given testbed matches the
profile of the intended environment.
III. T ECHNICAL D ETAILS
TestbedProfiler is a collection of software tools for ana-
lyzing a testbed of Crossbow’s MicaZ motes or Telos motes.
Our only assumption is that the testbed has a back-channel
188
for reporting data, to avoid interference with radio communi-
cations of the actual application. In TestbedProfiler, a central
server handles the tasking of the motes and data collection.
The application suite also includes a collection of analysis
scripts to assist users in understanding the collected data.
TestbedProfiler consists of three different software com-
ponents. A TinyOS application runs on the mote hardware,
sending broadcast packets to elicit connectivity information
and reporting statistics to the central server. The Profiler
Server controls the execution of the profile experiment. It tasks
individual motes to begin communications and stores all data
collected during the experiment. Finally, the post-experiment
analysis scripts provide average statistics and visualization of
the data collected during an experiment.
We suggest that the user initially run TestbedProfiler to
generate a profile for a testbed. The user then executes the
analysis scripts to uncover the environment for network com-
munications. If the user determines the testbed environment
does not match the profile of the intended environment, then
the user can use the visualization produced to determine the
motes in the testbed that are causing complications. Finally,
the user repeats these steps until the desired profile is achieved.
The rest of this section presents details of the CC2420 radio
and the three software components previously listed. Further
information is available in [6], [7].
A. Chipcon CC2420 Radio
Many of the motes in today’s testbeds use the Chipcon
CC2420 for wireless communications. The CC2420 is a
2.4 GHz IEEE 802.15.4 radio frequency transceiver that was
designed for low-power wireless applications. The chip is
capable of transmitting on 15 channels (11–26) separated
by 5 MHz, and it utilizes a digital Direct Sequence Spread
Spectrum (DSSS) modem [4] to transmit data at rates up to
250 kbps.
The CC2420 has a programmable power output with 32
levels, though only 8 levels are officially documented [8]. The
resulting output powers range from -25 dBm (< 10 μW) to
0 dBm (∼ 1 mW). TestbedProfiler can cycle through all of
these levels when creating its profile.
The CC2420 also provides a digital received signal strength
indicator (RSSI) register that TestbedProfiler uses to assess
the network. Upon each packet reception, the CC2420 will
calculate a link quality indicator (LQI).
B. TinyOS Application
Within our TestbedProfiler application suite is a TinyOS
application (called TestbedProfilerC) that executes on the mote
hardware. TestbedProfilerC uses the radio to communicate to
other motes and the serial port to communicate to the PC (i.e.,
the Profiler Server). It also has a timer to control the flow of
execution and is capable of sending and receiving variable
sized packets. See [7] for details on its configuration.
The TinyOS application itself is quite simple. A mote waits
until it is instructed over its serial port to begin sending
packets. Once instructed, the mote begins broadcasting packets
189
Authorized licensed use limited to: ULB Darmstadt. Downloaded on December 09,2022 at 18:48:35 UTC from IEEE Xplore. Restrictions apply.
of different sizes at the given power level to test whether there
is a difference in network behavior for different sized packets.
For this experiment the payload sizes were 25, 50, 75, and
100 bytes, chosen to represent a wide range of packet sizes.
When a mote hears a broadcast packet, it reports the sender’s
ID, receiver’s ID, sequence number of the packet, the received
RSSI value, the received LQI value, the payload size, and the
transmitted power level back to the Profiler Server over its
serial connection. The post-experiment analysis scripts then
use this information to generate the testbed’s overall profile.
C. Profiler Server
The Profiler Server is responsible for overseeing the execu-
tion of the experiment, tasking the motes, and recording the
data received. The most critical job that this server plays is
to ensure that one, and only one, mote is broadcasting at a
time, done by a sequential tasking of the motes. The Profiler
Server transmits a command to a mote and then waits until a
complete response is returned before sending a new command
to a different mote. We average data for each power level
together over time to avoid environmental differences that
might affect the data. Profiler Server also launches a second
thread to receive incoming packets from the motes and saves
the data to a file for later processing.
D. Post-Experiment Analysis Scripts
We include two Python scripts in our TestbedProfiler appli-
cation suite:
• CreateDigraph uses Graphviz to print directed graphs that
represent connectivity of a profiled network at a given
power level.
• CreateProfile calculates statistics for each node in the
testbed. A node’s profile includes a summary of its
connectivity to other nodes with the packet reception
rates, the average RSSI values, and the average LQI
values for the node’s neighbors.
These two analysis scripts allow a testbed designer to better
understand the testbed’s connectivity. The designer can then
use this information to identify the motes that are problematic
and then adjust to achieve a more realistic configuration.
IV. C ASINO L AB A NALYSIS
The Colorado School of Mines has a wireless sensor net-
work testbed in its engineering building, serving both research
and educational purposes. During the deployment, the Casino
Lab was thoroughly verified, but not validated. We used our
TestbedProfiler application suite to help us fully understand the
topology of the lab at various power levels. Previous empirical
evidence suggested that setting the CC2420 radio with a power
level of only three or four was sufficient for single hop
communication between any two motes. Some experiments
also suggested that dead spots within the Casino Lab topology
existed. TestbedProfiler was helpful in understanding these
results, and was evidence that testbed designers should validate
their testbed during deployment.
 Fig. 2. The output of the CreateDigraph analysis script at power level one.
Fig. 1. The Casino Lab environment at the Colorado School of Mines.

Power Average Power Average Power Average

A. Casino Lab Environment Level Neighbor Level Neighbor Level Neighbor
The Casino Lab is a wireless sensor network testbed with Count Count Count
52 Tmote Sky motes hung from the ceiling of a light industrial 01 1.909 12 49.124 22 49.832
environment as shown in Fig. 1. The motes are connected via 02 3.922 13 49.144 23 49.915
USB to 26 Tmote Connect Ethernet bridges which provide 03 33.295 14 49.152 24 49.978
power and a communications back-channel. They are arranged 04 42.982 15 49.396 25 49.970
in a grid with four columns of thirteen motes, as in a deck of 05 43.140 16 49.564 26 50.007
playing cards. Note that the columns are not straight lines and 06 43.364 17 49.592 27 49.331
the motes are deployed around obstacles such as light fixtures, 07 46.196 18 49.622 28 47.133
pipes, and pillars. 08 47.892 19 49.750 29 46.870
09 47.807 20 49.840 30 46.901
B. Methodology 10 47.855 21 49.829 31 45.421
We executed our TestbedProfiler application suite with 100 11 48.601
iterations per packet size per power level. We exercised all 31
nonzero power levels in this experiment and used a large range Fig. 3. The average neighbor counts for all 31 power levels in the Casino
Lab; we note that many of the available power levels are redundant.
of payload sizes from 25 to 100. Post-experiment analysis
showed that the 6 of clubs did not respond, so we omitted
it from our analysis.
is 3.922 (see Fig. 3), which is almost twice that of power level
With this setup, it took 11 hours for our TestbedProfiler to
one. There are, however, three motes with limited communi-
execute, and the amount of data returned to the Profiler Server
cation capability and five motes completely partitioned.
from the motes was overwhelming; over all of the full Casino
At power level three, the network connectivity increases by
Lab profiles we cataloged an average of 26,884,218 ± 280,390
roughly an order of magnitude. The average neighbor count
received messages occupying on average 575.4 ± 6.27 MB.
at power level three is 33.295 (see Fig. 3). The Casino Lab
Our post-experiment analysis scripts, especially CreateDi-
testbed goes from sparsely to densely connected in just three
graph, were vital in processing all this data and understanding
of the 31 power levels. Many of the available power levels are
the topology of our testbed.
redundant in regards to average neighbor count.
C. Results TestbedProfiler also provided us with a great deal of in-
Our post-experiment analysis scripts show that the network formation that helped explain some of the odd behavior we
in the Casino Lab is heavily partitioned at the lowest power had seen in past experiments. Typically experiments in Casino
level; Fig. 2, created by CreateDigraph, shows that three small Lab were done at the lower powers, and it was believed that
networks exist, two of which are connected by one node. If there were dead spots. Fig. 2 clearly shows where these dead
this node (node 43) fails, all three networks will be partitioned. spots are and how many nodes are involved. The results from
We also note that several of the connections between nodes TestbedProfiler allowed us to reexamine previous results and
are asymmetrical. Furthermore, 12 motes have zero commu- see more clearly what was happening.
nication capability. As shown in Fig. 3, the average neighbor
count for power level 1 is only 1.902. V. I MPROVING THE E DGAR M INE D EPLOYMENT
At power level two, the network connectivity increases The Edgar Mine is an underground mineral and hard-rock
substantially. The average neighbor count at power level two mine in Idaho Springs, Colorado that the Colorado School

190
Authorized licensed use limited to: ULB Darmstadt. Downloaded on December 09,2022 at 18:48:35 UTC from IEEE Xplore. Restrictions apply.

Fig. 4. A map of the Edgar Mine with our deployed motes. Each square on
the map is 30 meters on a side.
Mote Neighbors Mote Neighbors
1 2, 3 6 5, 7
2 1, 3 7 5, 6, 8, 9, 10
3 1, 2, 4 8 7, 9, 10
4 3, 5, 6 9 7, 8, 10
5 4, 6, 7 10 7, 8, 9
Fig. 5. The Edgar Mine neighbor table for all the motes in Fig. 4, which
we revised and improved after consulting TestbedProfiler.
of Mines (CSM) uses for engineering instruction and safety
training. CSM researchers are developing a WSN to track mine
operators and equipment and to monitor air quality. Our initial
testbed, mapped in Fig. 4, comprises 10 motes, with mote
number 1 about 300 meters into the mine tunnel. The ribs
(walls of the tunnels) are rough and uneven, causing signifi-
cant multi-path and signal degradation. These unpredictable
features plagued our initial mine deployment and required
interactively trying new positions and checking the results with
TestbedProfiler.
We initially collected RSSI samples measured from several
locations throughout the mine tunnels, and we determined
that motes should be placed approximately 18 meters apart
so we can track mine operators to an accuracy of at least
15 meters, obtain a sufficient resolution in air-quality sensing,
and provide 2–3 neighbors to each mote. In our initial mote
deployment, we placed the devices at temporary locations
with this spacing and then consulted TestbedProfiler before
permanently installing them. TestbedProfiler showed us weak
connections among motes 1, 2, and 3. Mote 1 had only a weak
signal to mote 2, mote 2 had only a weak signal to mote 1,
and mote 3 couldn’t communicate with either mote.
Due to the results from TestbedProfiler, we decided to move
motes 2 and 3 closer to mote 1 in order to ensure good
connectivity. We kept the spacing of motes 2 and 3 roughly
the same, but varied their positions. Through iterating with
TestbedProfiler, we found that raising mote 3’s position and
lowering mote 2’s position resulted in full connectivity among
motes 1, 2, and 3 with redundant links. We show the full,
revised neighbor table in Fig. 5.
191
Authorized licensed use limited to: ULB Darmstadt. Downloaded on December 09,2022 at 18:48:35 UTC from IEEE Xplore. Restrictions apply.
VI. C ONCLUSIONS
We created the TestbedProfiler application suite to give
testbed designers a tool that can be used to help ensure a
testbed deployment replicates the intended environment. We
suggest that TestbedProfiler be used several times during the
deployment phase; that is, before the final positions of the
motes have been determined, testbed designers should iter-
atively generate a profile of the testbed’s current deployment
with TestbedProfiler, identify problematic areas, and reposition
motes as needed. We used TestbedProfiler in this manner
successfully at the Edgar Mine. We also found TestbedProfiler
beneficial in explaining previous results in our Casino Lab
WSN.
Given the amount of labor involved in deploying a
testbed, we encourage testbed designers to use our Testbed-
Profiler application suite and validate the testbed’s in-
tended use exists. We have made our TestbedProfiler ap-
plication suite freely available to other researchers at
http://toilers.mines.edu. If you use TestbedProfiler
at your testbed site, please email us your testbed’s profile. We
would like to make these profiles available to all researchers,
leading to better understanding of existing testbeds and the
results that they provide.
ACKNOWLEDGMENTS
The authors would like to thank Keith Hellman, Kerri Stone,
Bryan Whalen, Nate Van Vorst, and Wade Simmons for their
work in deploying the Casino Lab. We would also like to
thank Chris Turiano, Manoja Weiss, and Mark Kuchta for
their contribution to the Edgar Mine deployment. This work
supported in part by NSF Grant CNS-0848130.
R EFERENCES
[1] P. Levis, N. Lee, M. Welsh, and D. Culler, “TOSSIM: Accurate and
Scalable Simulation of Entire TinyOS Applications,” in SenSys ’03:
Proceedings of the 1st International Conference on Embedded Networked
Sensor Systems. New York, NY, USA: ACM Press, 2003, pp. 126–137.
[2] Software Engineering Institute, “The Software Capability Maturity Model
Integrated (CMMI-SW) version 1.1,” Aug. 2002, Viewed on August
30, 2007. [Online]. Available: http://www.sei.cmu.edu/pub/documents/
02.reports/pdf/02tr028.pdf.
[3] A. Cerpa, N. Busek, and D. Estrin, “SCALE: A Tool for Simple
Connectivity Assessment in Lossy Environments,” Technical Report
0021, UCLA Center for Embedded Network Sensing (CENS), September
2003., 2003, Viewed on September 27, 2007. [Online]. Available:
http://lecs.cs.ucla.edu/Publications/papers/scale-tech-report.pdf.
[4] Chipcon, “CC2420 Data Sheet,” Apr. 2006, Viewed on June 18, 2007.
[Online]. Available: http://www.chipcon.com/files/CC2420 Data Sheet
1 4.pdf.
[5] H. Liu, L. Selavo, and J. Stankovic, “SeeDTV: Deployment-Time Vali-
dation for Wireless Sensor Networks,” in Proceedings of EmNets 2007,
2007, pp. 23–27.
[6] C. Metcalf, T. Camp, and M. Colagrosso, “TestbedProfiler: A validation
tool for wireless sensor network testbed deployment,” 2010, Technical
Report, Colorado School of Mines, MCS-10-02.
[7] C. Metcalf, “TOSSIM live: Towards a testbed in a thread,” Master’s thesis,
Colorado School of Mines, 2008.
[8] J.-H. Hauer, V. Handziski, and A. Wolisz, “Experimental Study of the
Impact of WLAN Interference on IEEE 802.15.4 Body Area Networks,”
in Wireless Sensor Networks: 6th European Conference, EWSN 2009
Cork, Ireland, February 2009 Proceedings. Berlin, Germany: Springer,
2009, p. 20.