Chapter 2

Performance Metrics

2: Performance 1-1
Chapter 2: goals

• Identify KPI (Key Performance Indicators)

for MM networks
• Describe relevant QoS parameters
• Survey the effect of QoS parameters over
MM services and applications
• Learn how QoS parameters can be
estimated

2: Performance 1-2
Chapter 2 outline
1.1 Introduction 1.5 Jitter
1.2 Network 1.6 Packet Loss
measurements 1.7 Multimedia and
1.3 Bandwidth and WiFi
capacity
1.4 Delay

2: Performance 1-3
Introduction
• Multimedia applications and services need specific
QoS levels
• QoS depends on several parameters:
– Bandwidth or data rate: a measure of transmission speed.
– Latency (maximum frame/packet delay): time needed
from transmission to reception.
– RTT (Round Trip Time): time needed from transmission to
reception and back again
– Jitter: a measure of smoothness of the audio/video playback,
related to the variance of frame/packet delays.
– Packet Loss: a measure (in percentage) of error rate of the
packetized data transmission.
– Others: Availability, Sync Skew, etc.

2: Performance 1-4
Multimedia Service Classes
• Real-Time (also Conversational): two-way traffic, low
latency and jitter, possibly with prioritized delivery, e.g.,
voice telephony and video telephony.

• Priority Data: two-way traffic, low loss and low latency,

with prioritized delivery, e.g., E-commerce applications.

• Silver: moderate latency and jitter, strict ordering and

sync. One-way traffic, e.g., streaming video, or two-way
traffic (also Interactive), e.g., web surfing, Internet games.

• Best Effort: (also Background): no real-time requirement,

e.g., downloading or transferring large files (movies).
• Bronze: no guarantees for transmission.

2: Performance 1-5
Requirement on Network
Bandwidth

2: Performance 1-6
Tolerance of Latency and Jitter
in Digital Audio and Video

2: Performance 1-7
QoS vs QoE
• QoS (Quality of • QoE (Quality of
Service) Experience)
– Performance level of a – Measure of a customer's
service or application experience with a service
– How does the service or application
works? – How does the service
– Measurable and works from the point of
objective view of a user?
– Metrics: – Measurable but
• Bandwidth subjective
• Delay – Metrics:
• Jitter • MOS (Mean Opinion Score)
• etc. • QoS to QoE mapping PSQA
(Pseudo-Subjective Quality
Assessment)

2: Performance 1-8
Chapter 2 outline
1.1 Introduction 1.5 Jitter
1.2 Network 1.6 Packet Loss
measurements 1.7 Multimedia and
1.3 Bandwidth and WiFi
capacity
1.4 Delay

2: Performance 1-9
Why measure?

• QoS parameters estimation

• Anomaly detection
• Dimensioning of applications, services or
networks
• SLA (Service Level Agreement) validation
– Service contract between service provider and
user. Service definition (scope, quality and
responsibilities)
• Testing

2: Performance 1-
10
Types of measurements
• Active • Passive
– Traffic injection – Traffic capture and
– Intrusive analysis
– Accurate – Non-intrusive
– Analysis of specific – Dependent on traversing
situations traffic
– Two types: – Big picture
• File transfer – Tools:
• Packet Pair/ Packet Train • Wireshark
– Tools: • NetFlow
• SpeedTest • MRTG (Multi Router
• iperf Traffic Grahp)
• ping
• traceroute

2: Performance 1-11
File Transfer
• To measure a file download is performed:
– Binary file
– Random content
– Without optimizations
• Use of HTTP protocol
• File download size  8 times the estimated
bandwidth
• Integrity check after file reception
• Defined in ETSI EG 202 057-4
• Not suitable for multimedia services

2: Performance 1-12
File Transfer: QoS estimation
• How to estimate QoS parameters?
– Bandwidth: (N= file size, T= download time)
– RTT: estimation using TCP RTT (in handsake)
– One Way Delay (OWD): No possibility of measuring.
Estimation using RTT/2 (only for symetric networks)
– Jitter: OWD variation
– Packet Loss: TCP analysis (retransmissions, sequence
numbers)

2: Performance 1-13
File Transfer: TCP problems
• Closer to user perception of the service
• Highly affected by TCP (window and concurrent
TCP flows), concurrent traffic and CPU load

2: Performance 1-14
Packet Pair
• TCP measurements are unfeasible in MM
environments
• How can we measure these services?
– Use UDP RTP-like packets (with departure timestamp and
sequence numbers)
– Packet rate and size adaptation
• Send multiple equally sized packet pairs from a
source to a destination back-to-back (at maximum
rate)
• At receiver side, analyze time between packets
(inter-arrival or inter-packet)

2: Performance 1-15
Packet Pair (II)

Δr

Source Intermediate Destination

Hops

Tight
Link

2: Performance 1-16
Packet Pair: QoS estimation
• How to estimate QoS parameters?
– Bandwidth: (L=packet size Δr=inter-arrival)
– One Way Delay: Difference between arrival time and
departure time
– Jitter: Variation of OWD
– Packet Loss: Count gaps in sequence numbers
• Advantages:
– Barely intrusive (small measurement time)
• Problems:
– What packet size do we use?
– Time synchronization (source and destination)
– Affected by concurrent traffic
2: Performance 1-17
Concurrent Traffic Effects
• Inter-arrival expansion:
Inter-arrival

Source Intermediate Destination

Hops
• Inter-arrival contraction
Maximum dispersion

Source Intermediate Destination

Hops
2: Performance 1-18
Packet Train
• To reduce the impact of concurrent traffic N
equally-sized (of L bytes) packets are sent back-
to-back from a source to a destination
• At destination point, inter-arrival of each pair of
packets is analyzed

ΔrN Δr2 Δr1

N packets

… …

2: Performance 1-19
Packet Train: QoS estimation
• How to estimate QoS parameters?
– Bandwidth: (Δri=i-th inter-arrival)
– One Way Delay: Difference between arrival time and
departure time. Average of individual delays or percentile
50 or percentile 95
– Jitter: Variation of OWD
– Packet Loss: Count gaps in sequence numbers
• Advantages:
– Immune to cross traffic (except CBR [Constant-Bit-Rate]
traffic)
• Beware shapers/policers and queueing disciplines
(modification of inter-arrivals)

2: Performance 1-20
Chapter 2 outline
1.1 Introduction 1.5 Jitter
1.2 Network 1.6 Packet Loss
measurements 1.7 Multimedia and
1.3 Bandwidth and WiFi
capacity
1.4 Delay

2: Performance 1-
21
Bandwidth and capacity
• Capacity: constant transmission rate of a link or
end-to-end path
• Available Bandwidth: non-used capacity of a link
or end-to-end path in a given time
• If link or end-to-end path is saturated there is no
available bandwidth
• When deploying MM services enough bandwidth is
necessary to transport data without losses

2: Performance 1-22
Bandwidth and MM services
• VoIP (Voice over IP):
– Codec dependent
– Bandwidth between 8 and 96 Kb/s per direction (caller,
calle)
– Packet losses if not enough bandwidth
• VoD (Video on Demand):
– Mostly dependent on the codec and the format
of the video (HD/SD)
– 1 SD channel (2-3 Mbit/s)
– 1 HD channel (10-20 Mbit/s)
• Statistical multiplexing (channel saturation)

2: Performance 1-23
Chapter 2 outline
1.1 Introduction 1.5 Jitter
1.2 Network 1.6 Packet Loss
measurements 1.7 Multimedia and
1.3 Bandwidth and WiFi
capacity
1.4 Delay

2: Performance 1-
24
Delay
• Time between emission of the first bit of a
packet and the reception of the last bit of
such packet [RFC2679]
• Negative delays?
– Only if source and destination points are not
synchronized
– Use of synchronization mechanisms (NTP, PTP,
GPS)
• Several measurements, only one delay?
– Percentile 50
– Minimum/Maximum/Average
2: Performance 1-25
Delay of a packet i

2: Performance 1-26
Delay of a packet i
• ti : packet generation time at sender side
• ai : packet reception time at receiver side
• pi : playout time at receiver side
• Dprop : propagation delay. Dependent on distance
and propagation speed
• vi : queueing and processing delay
• bi : waiting time at receiver until play-out time.
Buffer dependent.bi = pi-ai
• di : total play-out delay di = pi - ti
• ni : total network only delay ni = ai - ti

2: Performance 1-27
Delay and MM services
• VoIP:
– Echo cancellation problems (delay estimation)
– Audio freezes
– MtE (Mouth-to-Ear) delay < 150 ms [ITU-T
G.114]
• VoD:
– Delay in video reproduction
– Increase in interactivity times (round-trip)
– Delay < 150 ms for optimal experience
– 4-5 seconds of buffering allowed

2: Performance 1-28
Chapter 2 outline
1.1 Introduction 1.5 Jitter
1.2 Network 1.6 Packet Loss
measurements 1.7 Multimedia and
1.3 Bandwidth and WiFi
capacity
1.4 Delay

2: Performance 1-
29
Jitter
• Packet Delay Variation [RFC 3393]
– Over a stream of packets
– Difference between delays of selected packets
(deterministic o random)
• Multiple definitions
– Standard deviation:
– Percentile range: difference between
percentile 50 and 95
– Coefficient of variation:

2: Performance 1-30
Jitter and MM services
• VoIP:
– Jitter < 30 ms
– Use of buffer to compensate jitter
– Adaptive estimation algorithms
• VoD:
– STB (Set-top box) decoding tolerance of ±500ns
– Use of buffers to compensate jitter
• If buffer small  video freezes
• If buffer large  increment in interactivity delay
– Buffer size < 100 ms + maximum jitter
– Jitter < 10 ms

2: Performance 1-31
Chapter 2 outline
1.1 Introduction 1.5 Jitter
1.2 Network 1.6 Packet Loss
measurements 1.7 Multimedia and
1.3 Bandwidth and WiFi
capacity
1.4 Delay

2: Performance 1-
32
Packet Loss
• Percentage of packets sent but not received
or received corrupt [RFC 2680]
– Over a stream of packets
– Use of sequence numbers or common identifiers
between sender and receiver
– Differentiate delay loss and transport loss
– Application-specific threshold for marking packet
as delay loss

2: Performance 1-33
Packet Loss
• VoIP:
– 20-30 ms of audio samples can be compensated
(Packet Loss Concealment)
– Packetization intervals near 20 ms
– Gaps in communications and lack of fluency in
the conversation
• VoD:
– Dependent on which frame type (I, B, P) is lost
– Gaps in the reproduction and “pixelation”
• Generally, packet loss should be less than
1%
2: Performance 1-34
Chapter 2 outline
1.1 Introduction 1.5 Jitter
1.2 Network 1.6 Packet Loss
measurements 1.7 Multimedia and
1.3 Bandwidth and WiFi
capacity
1.4 Delay

2: Performance 1-
35
Multimedia and WiFi
• Wireless networks suffer from:
– Bit errors as they are prone to interference and
transmission medium degradation.
– Increased delays (due to backoff)
• WiFi Multimedia (WMM) and Wireless
Multimedia Extensions (WME):
– 802.11e specification
– Definition of 4 categories for priorization:
• Voice
• Video
• Best-effort
• Background
2: Performance 1-36
Multimedia and WiFi
• Variable effective bandwidth:
– Distance to AP
– Number of clients connected
– Quality of the signal (retransmissions)
• Increased Jitter
• Larger number of concurrent transmissions:
– Increased packet losses and delays

2: Performance 1-37