Chapter 15:

Wireless Multimedia Sensor Networks

Wireless Sensor Networks

1
Akyildiz/Vuran
Wireless Multimedia Sensor Networks (WMSNs)
Networks of wirelessly interconnected devices that allow
retrieving video and audio streams, still images, and scalar
sensor data.

Also able to store process in real-time, correlate and fuse

multimedia data originated from heterogeneous sources.

Wireless Sensor Networks

2
Akyildiz/Vuran
Wireless Multimedia Sensor Networks
E. Gurses, O. B. Akan, "Multimedia Communication in Wireless Sensor Networks," Annals of Telecommunications , vol. 60,
no. 7-8, pp. 799-827, July-August 2005.
I.F. Akyildiz, T. Melodia, K. Chowdhury, A Survey on Wireless Multimedia Sensor Networks, Computer Networks
(Elsevier), March 2007.
S. Misra, M. Reisslein, and G. Xue. A Survey of Multimedia Streaming in Wireless Sensor Networks, IEEE Communications
Surveys and Tutorials, 10(3), 2008

Sink
Internet LEGEND
Multimedia
processing hub
Video sensor
Audio sensor
Gateway High end video
sensor
Scalar sensor

Wireless gateway

Storage hub

(a) Single-tier flat, homogeneous (b) Single-tier clustered, (c) Multi-tier, heterogeneous sensors,
sensors, distributed processing, heterogeneous sensors, distributed processing, distributed storage
centralized storage centralized processing,
centralized storage

Wireless Sensor Networks

3
Akyildiz/Vuran
Video Sensors
High-end
Stargate board interfaced with a
medium resolution camera
Stargate hosts an 802.11 card and
a MICAz mote that functions as
gateway to the sensor network
Low-end
MicaZ interfaced with Cyclops low
resolution camera
CmuCam3

Wireless Sensor Networks

4
Akyildiz/Vuran
Stargate + Garcia = Multimedia Mobile
Sensor
Mobile
Onboard IR Sensors
Pan-tilt Camera
Stargate
Connects to a MICAz network
Onboard Linux Operating System

Wireless Sensor Networks

5
Akyildiz/Vuran
Multimedia Sensor Hardware
MEMORY SENSORS

CPU CAMERA
AUDIO

POWER UNIT SCALAR SENSORS

ENERGY HARVESTING

COMMUNICATION COORDINATION
SUBSYSTEM SUBSYSTEM

SYSTEM SOFTWARE AND LOCATION

MIDDLEWARE MANAGEMENT

MOTION CONTROLLER MOBILITY/ ACTUATION

NETWORK UNIT
INTERFACE
STACK NETWORK SERVOS
SYNCHRONIZATION
MOTORS

Wireless Sensor Networks

6
Akyildiz/Vuran
APPLICATIONS
Tracking
Home Automation
Environmental monitoring
Multimedia Surveillance Sensor Networks (against crime
and terrorist attacks, law enforcement agencies to monitor
areas, public events, private properties and borders).

Wireless Sensor Networks

7
Akyildiz/Vuran
APPLICATIONS
Storage of Potentially Relevant Activities.
Thefts, car accidents, traffic violations
Make video/audio streams or reports available for future query
Traffic Avoidance, Enforcement and Control Systems
Smart parking advice systems
Monitor the flow of vehicular traffic on highways (avg. speed, no. of
cars)
Monitor accidents for subsequent accident scene analysis

Wireless Sensor Networks

8
Akyildiz/Vuran
APPLICATIONS
Advanced Health Care Delivery
Patients will carry medical sensors to monitor parameters such as
body temperature, blood pressure, pulse oximetry, breathing
activity
Automated Assistance for the Elderly and Family Monitors

Environmental Monitoring (acoustic and video feeds)

Person Locator Services (locate missing persons)
Industrial Process Control
Wireless Sensor Networks
9
Akyildiz/Vuran
WMSNs CHALLENGES
Resource Constraints
Sensor devices are constrained in terms of battery,
memory processing capability, and achievable data rate
Variable Channel Capacity
Application-Specific QoS Requirements
High Bandwidth Demand
Data rate for sensors 250Kbps; much more higher rates
are required for multimedia sensors

Wireless Sensor Networks

10
Akyildiz/Vuran
WMSNs CHALLENGES
Multimedia Source Coding Techniques
Uncompressed raw video streams require excessive BW
e.g., a single monochrome frame in the NTSC-based Quarter
Common Intermediate Format (QCIF, 176x120), requires around 21
Kbytes, and at 30 frames/sec, a video stream requires over 5 Mbps!

Traditional video coding techniques are based on the idea of

reducing the bit rate generated by the source encoder by exploiting
source statistics

Wireless Sensor Networks

11
Akyildiz/Vuran
WMSNs CHALLENGES
Encoders --> intra-frame compression techniques
Reduce redundancy within one frame
Leverage inter-frame compression (also known as predictive
encoding or motion estimation)
Predictive encoding requires
Complex encoders
Powerful processing algorithms
High energy
Not suited for low-cost multimedia sensors

Wireless Sensor Networks

12
Akyildiz/Vuran
WMSNs CHALLENGES
Multimedia In-network Processing.
Need for new multimedia in-network processing
algorithms
New architectures for collaborative, distributed, and
resource-constrained processing
Increase the system scalability
Reduce redundant information
Merge data from multiple views, on different media,
and with multiple resolutions

Wireless Sensor Networks

13
Akyildiz/Vuran
WMSNs CHALLENGES
Power Consumption
Multimedia applications produce high volumes of data, which
require high transmission rates, and extensive processing

Integration with Internet (IP) Architecture and Other Wireless

Technologies (to retrieve useful information from anywhere and at any
time)

Protocols, algorithms and architectures

Maximize the network lifetime AND
Provide the QoS required by the application
Wireless Sensor Networks
14
Akyildiz/Vuran
Examples of Deployed WMSNs
Panoptes
W. Feng, B., Code, E. Kaiser, M. Shea, L. Bavoil, Panoptes: Scalable Low Power
Video Sensor Networking Technologies, Proc. of ACM Multimedia Conf., Nov
2003.
SensEye
P. Kulkarni, D. Ganesan, P. Shenoy,SensEye: A Multi-tier Camera Sensor
Network, Proc. of ACM Multimedia, Nov. 2005
IrisNet
IrisNet (Internet-scale Resource-Intensive Sensor Network Services)
S. Nath, Y. Ke, P. Gibbons, B. Karp and S. Seshan, A Distributed Filtering
Architecture for Multimedia Sensors, Intel Tech. Rept, Aug 2004.

Wireless Sensor Networks

15
Akyildiz/Vuran
Panoptes
Environmental observation and surveillance applications

Intel StrongARM PDA platforms with a Logitech webcam

Video sensors are high-end devices with Linux, 64 Mbytes of
memory connected through 802.11 cards.

Spatial compression (but not temporal), distributed filtering,

buffering, and adaptive priorities for the video stream.

Wireless Sensor Networks

16
Akyildiz/Vuran
SensEye Video stream
Tier 3
handoff

Webcam + Stargate
wakeup
Tier 2

Low-res cam + Mote

wakeup
Tier 1

Scalar Sensors + Mote

Wireless Sensor Networks

17
Akyildiz/Vuran
IrisNet
Wide-area sensor network
Internet-like queries to video and other data on this infrastructure
Video sensors and scalar sensors are spread throughout the
environment, and collect potentially useful data.
User views the sensor network as a single unit that can be queried
Each query operates over data collected from the sensor network
Allows simple Google-like queries as well as more complex queries
involving arithmetic and database operators

Wireless Sensor Networks

18
Akyildiz/Vuran
COMMUNICATION PROTOCOLS

Task Management Plane

Power Management Plane
Application Layer

Transport Layer

Network Layer

Data Link Layer

Physical Layer

Wireless Sensor Networks

19
Akyildiz/Vuran
Application Layer
Provide traffic management and admission control
functionalities
Perform source coding according to application
requirements and hardware constraints, by leveraging
advanced multimedia encoding techniques
Provide flexible and efficient system software
Provide primitives for applications to leverage collaborative,
advanced in-network multimedia processing techniques

Wireless Sensor Networks

20
Akyildiz/Vuran
TRAFFIC CLASSES
Differentiation between traffic types
Integrated Traffic: (AUDIO, VIDEO, DATA, STILL IMAGE)
Delay in/sensitive
Jitter in/sensitive
Loss in/sensitive
Different data rate requirements
How to guarantee delay bounds and jitter bounds?
How to realize data aggregation?
Explore the tradeoffs between quality and energy consumption!!

Wireless Sensor Networks

21
Akyildiz/Vuran
TRAFFIC CLASSES
Real-time, Loss-tolerant, Multimedia Streams
Delay-tolerant, Loss-tolerant, Multimedia Streams
Real-time, Loss-tolerant, Data
Real-time, Loss-intolerant, Data
Delay-tolerant, Loss-intolerant, Data
Delay-tolerant, Loss-tolerant, Data

Wireless Sensor Networks

22
Akyildiz/Vuran
OPEN RESEARCH ISSUES ON
APPLICATION LAYER
While theoretical results on Slepian-Wolf and Wiener-Ziv
coding exist since thirty years, there is still a lack of
practical solutions

The net benefits and the practicality of these techniques

still need to be demonstrated

Need to fully explore the tradeoffs between the achieved

fidelity and energy consumption

Wireless Sensor Networks

23
Akyildiz/Vuran
 Transport Layer Protocols Overview

TCP/UDP and TCP Friendly Schemes Application Specific and Non-standard Protocols
TCP may be preferred over UDP
unlike traditional wireless networks
Compatible with the TCP rate control
Reliability Congestion Control Use of Multipath
mechanism, e.g.. STCP, MPEG-TFRCP
Better load balancing and
Per-packet delivery Spatio-temporal reporting
robustness to channel state
guarantee for selected Adjusting of reporting
variability.
packet types frequency based on
Need to regulate multiple
Redundancy by caching current congestion levels
sources monitoring the same
at intermediate nodes e.g. ESRT
event
e.g. RMST, PSFQ, (RT)2
e.g. CODA, MRTP
Wireless Sensor Networks
24
Akyildiz/Vuran
OPEN RESEARCH ISSUES IN
TRANSPORT LAYER
Tradeoff between Reliability and Congestion Control
Real-Time Communication Support
Relation between Multimedia Coding Rate and Reliability

Wireless Sensor Networks

25
Akyildiz/Vuran
NETWORK LAYER
(ROUTING SOLUTIONS)
Network conditions that leverage channel and link statistics
Construct paths based on packet priorities
Specialized protocols for real-time streaming that use
spatio-temporal forwarding

Wireless Sensor Networks

26
Akyildiz/Vuran
Routing Algorithms Overview

Network Condition Traffic Class Based Real Time Streaming

Based Metrics Metrics Based Metrics
Position wrt sink QoS profiles/Traffic
Radio characteristics
Spatio-temporal
classes
character
Error rate
Dropping rate
Residual energy Probabilistic delay
Latency tolerance
Backlogged packets guarantees
Desired bandwidth

Wireless Sensor Networks

27
Akyildiz/Vuran
Recent Solutions
QoS Routing Based on Network Conditions
L. Savidge, H. Lee, H. Aghajan, A. Goldsmith, QoS Based
Geographic Routing for Event Driven Image Sensor Networks,
Proc of BASENETs, Oct. 2005
K. Akkaya and M Younis, An Energy Aware QoS Routing Protocol
for WSNs, Proc. of ICDSW, 2003.
Routing Protocols with Support for Streaming
T. He, J. Stankovic, C. Lu, T. Abdelzaher, A Spatio Temporal
Communication Protocol for WSNs IEEE Tr. on Parallel and
Distributed Systems, Oct 2005.

Wireless Sensor Networks

28
Akyildiz/Vuran
Recent Solutions
MMSPEED:
E. Felemban, C. G Lee, E. Ekici MMSPEED: Multipath
Multi-speed Protocol of QoS Guarantees of Reliability and
Timeliness in WSNs, IEEE Tr. on Mobile Computing,
June 2006

Wireless Sensor Networks

29
Akyildiz/Vuran
Open Research Issues in NETWORK
LAYER
While current research directions make an effort to provide real-time
streaming, they are still best effort services.

Giving firm delay guarantees in a dynamically changing network is a

difficult problem and yet is important for seamless viewing of the
multimedia frames.

MMSPEED takes the step towards this end by adopting a probabilistic

approach more research needed !!!

Wireless Sensor Networks

30
Akyildiz/Vuran
Open Research Issues in NETWORK
LAYER
Identification of the optimal routing metrics
Most routing protocols that consider more than one metric, such as
energy, delay etc., form a cost function that is then minimized.

The choice of the weights for these metrics need to be undertaken

Further work is needed to shift this decision making process and
network tuning from the user end into the network

Wireless Sensor Networks

31
Akyildiz/Vuran
MAC LAYER
Channel access policies
Scheduling and buffer management
Error control

Wireless Sensor Networks

32
Akyildiz/Vuran
Overview of MAC Layer Protocols

Contention Free
Contention Based
Coordinate sleep/awake
cycles
Single Channel Multi-channel
TDMA -like
Bursty nature of
Better bandwidth utilization
Better control for multimedia Hardware assumptions scheduling may lead to
design parameters
Channel switching delay may be jitters
Simple hardware, operation
a consideration in end to end e.g.. S-MAC, T-MAC
MIMO technology
e.g.. STE, EDD
latency
e.g.. STEM, RATE-EST, CMAC

Wireless Sensor Networks

33
Akyildiz/Vuran
CHANNEL ACCESS POLICIES
Main causes of energy loss
Packet collisions
Subsequent re-transmissions
Overhearing packets
Idle listening
CHANNEL ACCESS IMPORTANT

Wireless Sensor Networks

34
Akyildiz/Vuran
Contention-Based MAC Protocols
Most contention-based protocols have a single-radio architecture.
Alternate between sleep cycles (low power modes with transceiver
switched off) and listen cycles (for channel contention and data
transmission).

Primary concern saving energy, at the cost of latency and by

leading to throughput degradation

NOT AN ACCEPTABLE TRADEOFF FOR WMSN !

Wireless Sensor Networks

35
Akyildiz/Vuran
Problems of Contention-Based MAC
Protocols
A sophisticated duty cycle calculation based on permissible
end-to-end delay needs to be implemented

Coordinating the sleep-awake cycles between neighbors is

generally accomplished through schedule exchanges.

Overhead of passing frequent schedules also needs

investigation in light of the ongoing high data rate
video/audio messaging.

Wireless Sensor Networks

36
Akyildiz/Vuran
Problems of Contention-Based MAC
Protocols
Video traffic exhibits an inherent bursty nature and can lead
to sudden buffer overflow at the receiver.

By choosing to send a burst of data during the listen cycle,

T-MAC shows performance improvement over S-MAC, but
at the cost of monopolizing a bottleneck node.

Such an operation could well lead to strong jitters and

result in discontinuous real-time playback.

Wireless Sensor Networks

37
Akyildiz/Vuran
Contention-Free Protocols
Clusterhead (CH) or sink helps in slot assignment, querying
particular sensors and maintaining time schedules.

These protocols can be easily adapted for multimedia

transmission !!!

Wireless Sensor Networks

38
Akyildiz/Vuran
OPEN MAC RESEARCH ISSUES
TDMA schedules within a cluster can be easily devised
Problem is more difficult when individual CHs are not in
direct range of the sink
inter-cluster multi-hop communication needed!!

Wireless Sensor Networks

39
Akyildiz/Vuran
OPEN MAC RESEARCH ISSUES
Need non-overlapping slot assignment for all neighboring
clusters
NP-complete by reduction to an instance of graph coloring
Development of efficient heuristics is an open issue

Wireless Sensor Networks

40
Akyildiz/Vuran
OPEN MAC RESEARCH ISSUES
Effect of clock drift if the slot duration is small and rigid
time synchronization is required for best performance

Network scalability

Wireless Sensor Networks

41
Akyildiz/Vuran
OPEN MAC RESEARCH ISSUES
TDMA schedules must be able to accommodate high node
densities

As channel capacity in TDMA is fixed, only slot durations or

number of slots in a frame may be changed keeping in mind
the number of users and their respective traffic types.

Wireless Sensor Networks

42
Akyildiz/Vuran
FURTHER LINK LAYER OPEN
RESEARCH PROBLEMS
ERROR CONTROL
FEC Which one?
ARQ Depends !!
Hybrid ARQ ??

Multimedia Packet Size Optimization !!

Wireless Sensor Networks

43
Akyildiz/Vuran
PHYSICAL LAYER:
Ultra Wide Band Communication
UWB: as a signal with either a fractional bandwidth of
20% of the center frequency or 500 MHz (when the center
frequency is above 6 GHz).
The FCC calculates the fractional bandwidth as
2(fH fL ) / (fH + fL)
where fH represents the upper frequency of the -10 dB
emission limit and fL represents the lower frequency limit of
the -10 dB emission limit

Wireless Sensor Networks

44
Akyildiz/Vuran
PHYSICAL LAYER:
Ultra Wide Band Communication
Low Power Consumption
High Data Rate
Higher immunity to the multi-path fading

Wireless Sensor Networks

45
Akyildiz/Vuran
PHYSICAL LAYER:
Ultra Wide Band Communication
Time-Hopping Impulse Radio UWB (TH-IR-UWB)
MultiCarrier UWB (MC-UWB) based on OFDM

Wireless Sensor Networks

46
Akyildiz/Vuran
Time-Hopping Impulse Radio UWB
(TH-IR-UWB)
Send very short duration pulses (in the order of hundreds of
picoseconds) to convey information
Time is divided into frames, each of which is composed of
several chips of very short duration
Each sender transmits one pulse in a chip per frame only,
and multi-user access is provided by pseudo-random time
hopping sequences (THS) that determine in which chip each
user should transmit

Wireless Sensor Networks

47
Akyildiz/Vuran
MultiCarrier UWB (MC-UWB) based on
OFDM
Well-suited to avoid interference because its carrier
frequencies can be precisely chosen to avoid narrowband
interference to or from narrowband systems
However, implementing a MC-UWB front-end power
amplifier can be challenging due to the continuous
variations in power over a very wide bandwidth
Moreover, when OFDM is used, high-speed FFT processing
is necessary, which requires significant processing power
and leads to complex transceivers

Wireless Sensor Networks

48
Akyildiz/Vuran
TH-IR-UWB
TH-IR-UWB signals require fast switching times for the
transmitter and receiver and highly precise synchronization
The RF front-end of an TH-IR-UWB system may resemble a
digital circuit, thus circumventing many of the problems
associated with mixed-signal integrated circuits.
Simple TH-IR-UWB systems can be very inexpensive to
construct

Wireless Sensor Networks

49
Akyildiz/Vuran
WHY IR-UWB for WMSNs?
It enables high data rate, very low power wireless
communications, on simple-design, low-cost radios
(carrierless, baseband communications)
Its fine delay resolution properties are appropriate for
wireless communications in dense multipath environment,
by exploiting more resolvable paths

Wireless Sensor Networks

50
Akyildiz/Vuran
WHY IR-UWB for WMSNs?
Provides large processing gain in presence of interference
Provides flexibility, as data rate can be traded for power
spectral density and multi-path performance

Finding suitable codes for THS is trivial (as opposed to

CDMA codes), and no assignment protocol is necessary

Wireless Sensor Networks

51
Akyildiz/Vuran
WHY IR-UWB for WMSNs?
Integrated MAC/PHY solutions are possible
Interference mitigation techniques allow realizing MAC
protocols that do not require mutual temporal exclusion
between different transmitters.

Simultaneous communications of neighboring devices are

feasible without complex receivers as required by CDMA

Wireless Sensor Networks

52
Akyildiz/Vuran
OPEN RESEARCH ISSUES ON
PHYSICAL LAYER
How to efficiently share the medium in UWB multi-hop
networks?
How to provide provable latency and throughput bounds to
multimedia flows in an UWB environment
Develop analytical models to quantitatively compare
different variants of UWB to determine trade-offs in their
applicability to high data rate and low power consumption
devices such as multimedia sensors

Wireless Sensor Networks

53
Akyildiz/Vuran
OPEN RESEARCH ISSUES ON
PHYSICAL LAYER
Integrate UWB with advanced cognitive radio techniques to
increase the spectrum utilization

For example, UWB pulses could be adaptively shaped to

occupy portions of the spectrum that are subject to lower
interference

Wireless Sensor Networks

54
Akyildiz/Vuran