You are on page 1of 85

RTP: Multimedia Streaming over IP

Colin Perkins USC Information Sciences Institute

Internet Multimedia
Internet Multimedia has long history:
RFC 741, "Network Voice Protocol", 1977 First video experiments in the early 1980s

Modern standards development began in 1992:


Developing from teleconferencing systems Audiocast of IETF meetings
20 sites on 3 continents Precursors to RTP and the present standards

Standardized RTP in 1996

Widespread availability of suitable networks in the last couple of years Unusual characteristics, worth exploring
Copyright 2002 Colin Perkins

Talk Outline
Use of IP for real-time traffic Protocols for real-time multimedia over IP
Outline of signalling protocols Outline of media transport protocols

RTP: Real-time Transport Protocol


RTP data transfer protocol RTP control protocol

Robustness
Playout and timing correction Error correction Security Congestion control

Security Conclusions

Copyright 2002 Colin Perkins

Talk Outline
Use of IP for real-time traffic Protocols for real-time multimedia over IP
Outline of signalling protocols Outline of media transport protocols

RTP: Real-time Transport Protocol


RTP data transfer protocol RTP control protocol

Robustness
Playout and timing correction Error correction Security Congestion control

Security Conclusions

Copyright 2002 Colin Perkins

The IP Protocol Stack


Application programs HTML MIME Media codecs HTTP SMTP RTP FTP SIP RTSP TCP UDP

Applications can't see the link layers


Just see the performance of the IP layer Must assume lowest common denominator behaviour

IP
ADSL Ethernet PPP Twisted Pair Optical Fibre Wireless

Link layer can't tell the needs of the application


Just see a series of packets Optimisations for particular traffic classes are risky Is the traffic really what you think?

IP forms an abstraction layer


Applications and transport protocols above Assorted link technologies below

Decoupling applications from the network

Copyright 2002 Colin Perkins

IP Service Model
The IP service model is limited Best effort packet transport Fragmentation Routing and addressing

Copyright 2002 Colin Perkins

Best Effort Packet Transport


Performance not guaranteed Packets can be
lost delayed reordered duplicated corrupted

Many causes of problems:


Congestion may cause loss Packet corruption may cause loss Route changeover may cause loss Queuing delay Multi-path IP routing may reorder Link-layer striping may reorder Spurious retransmission and router bugs cause duplicates

and the transport protocol must compensate Checksum to catch bit errors

Assumption: Significant packet loss

Copyright 2002 Colin Perkins

Packet Loss Patterns

45

Packet Loss Rate (percent)

40 35 30 25 20 15 10 5 0 8:00 10:00 12:00 14:00 16:00 18:00

Time of day

Copyright 2002 Colin Perkins

Timing Disruption

45 line: clocks are synchronized Discontinuity due to route change.

Reception time

Queuing jitter causes variation in inter-packet arrival time. Non-45 slope shows presence of clock skew Transmission time

Copyright 2002 Colin Perkins

Best Effort Transport


Performance can be bad
Applications should be prepared to compensate Doesn't have to be!

Loss and jitter can be made arbitrarily low through careful engineering
Most backbone networks have very good performance Interconnects and customer LANs are currently the main trouble spots Explicit QoS doesn't appear necessary

Copyright 2002 Colin Perkins

Fragmentation
IP fragments packets that exceed the MTU
Often >1500 octets

This causes an undesirable loss multiplier effect


Loss of one fragment means the others must also be discarded Better for the application to fragment, and generate small IP packets

Original Fragments

Packet loss Reconstructed

Copyright 2002 Colin Perkins

Routing and Addressing


Most communication is unicast
Point-to-point

Inefficient for many receivers


The server must generate n copies of the data, for n clients Significant scaling bottleneck

The use of IP with best effort transport implies heterogeneity Multicast makes this an order of magnitude worse
Each receiver sees different loss characteristics Hard to get timely feedback in a scalable manner

Multicast allows the server to scale


Adds network complexity

Copyright 2002 Colin Perkins

Properties of IP: Summary


Best effort, unreliable, packet delivery service
Much of the network performs well Peering points and customer premises often lacking

Provides a fragmentation service


Typically best avoided

Provides a multicast service


In some parts of the network
10% of broadband connections

At best: simple, effective, service

Copyright 2002 Colin Perkins

Transport Protocols
The IP service, by itself, is very limited
Just (tries to) deliver packets

Always augmented by a transport protocol


TCP UDP
(others in development)

Copyright 2002 Colin Perkins

TCP
Enhances the raw IP service
Point-to-point and connection oriented Service selection through ports Reliable, in-order, delivery Rate adaptation to match network capacity

Copyright 2002 Colin Perkins

TCP
Rate adaptation matches throughput to capacity
Congestion Window Size

High link utilization Fair share between flows

Retransmission ensures that no data is lost


Reliable, in-order, delivery

Time Slow start Slow start Congestion avoidance Congestion avoidance

Copyright 2002 Colin Perkins

UDP
Uses the services of IP
best effort (unreliable) but timely fragmentation routing and addressing unicast and multicast

Provides ports, in addition to IP addressing, but no other services

Copyright 2002 Colin Perkins

Reliability/Timeliness Tradeoff

Reliable Not timely

Unreliable Timely

TCP

UDP RTP

Protocols built on unreliable packet networks must make a fundamental trade-off:


Timely Reliable

Multimedia systems choose their transport carefully:


TCP for signalling UDP for media data

TCP is at one extreme, UDP the other

Application level protocols can blur the boundary


E.g. RTP for multimedia data Essential for performance

Copyright 2002 Colin Perkins

Talk Outline
Use of IP for real-time traffic Protocols for real-time multimedia over IP
Outline of signalling protocols Outline of media transport protocols

RTP: Real-time Transport Protocol


RTP data transfer protocol RTP control protocol

Robustness
Playout and timing correction Error correction Security Congestion control

Security Conclusions

Copyright 2002 Colin Perkins

The Multimedia Protocol Framework


Two fundamental components of the framework: Signalling protocols Media transport protocols

Copyright 2002 Colin Perkins

Signalling Protocols
The first stage of any multimedia session is signalling User location Session initiation, call setup and teardown Media negotiation Conference control Many signalling protocols exist Teleconferencing: H.323 Telephony: SIP Streaming: RTSP, SAP

Copyright 2002 Colin Perkins

Signalling Protocols: H.323


Original signalling protocol for IP-based multimedia
Extension of H.320 ISDN conferencing to IP Tightly coupled, small group, video conferencing

Flexible media negotiation and call control Reputation for complexity


ASN.1 encoding Many RTT call setup
(mostly fixed in later versions)

Copyright 2002 Colin Perkins

Signalling Protocols: SIP


RFC 2543 (recently updated) SIP is used to invite someone to join a session
Media negotiation User location Call setup and teardown

Considerable overlap with H.323


More flexible integration with other Internet services
Email, Web, streaming media, recording, agents, etc.

More limited media negotiation and call control


Extensions underway

Very different style


Protocol operation is based on HTTP Reuses much existing infrastructure
Copyright 2002 Colin Perkins

Signalling Protocols: RTSP


RFC 2326 Designed for control of a media on demand server
Point-to-point VCR-style remote control Record/play/rewind/fast-forward

Widespread commercial use


RealAudio, QuickTime

May also be useful for controlling other devices


Voice mail Interactive voice response

Leverages HTTP and SIP infrastructure

Copyright 2002 Colin Perkins

Signalling Protocols: SAP


RFC 2974 A multicast announce-listen protocol for wide area announcement of multimedia sessions
Announcers periodically multicast SDP descriptions to a well known group Inter-announcement interval is 10+ minutes Listeners slowly build up a cache of sessions

Suitable for announcing long-lived public sessions


E.g. radio/TV station, event coverage

Mostly used with IP multicast


Talk of use with cable networks

Copyright 2002 Colin Perkins

Media Transport Protocols


Once the session has been setup, media flows Convergence on a single media transport protocol for: Voice over IP Teleconferencing Streaming media Real-time Transport Protocol, RTP Flexible, supports many codecs and media types Extensible to new scenarios

Copyright 2002 Colin Perkins

The Multimedia Protocol Framework

Call control Media negotiation RTSP TCP SIP

Light weight sessions SAP UDP IP

Media codecs RTP

Media codecs RTP UDP

RAS

Call control Media negotiation H225.0 TCP IP H.245

IETF Multimedia Protocol Stack

ITU Teleconferencing Protocols

Copyright 2002 Colin Perkins

Design Choices
Depending on the scenario, implement:
An appropriate signalling protocol
RTSP SIP/H.323 SAP

Media transport using RTP


One or more codecs
MPEG H.263

Error correction and concealment Congestion Control

Copyright 2002 Colin Perkins

Talk Outline
Use of IP for real-time traffic Protocols for real-time multimedia over IP
Outline of signalling protocols Outline of media transport protocols

RTP: Real-time Transport Protocol


RTP data transfer protocol RTP control protocol

Robustness
Playout and timing correction Error correction Congestion control

Security Conclusions

Copyright 2002 Colin Perkins

RTP: Real-time Transport Protocol


The standard for real-time transport over IP networks
Streaming audio and video Voice over IP

Published as an IETF proposed standard


RFCs 1889 and 1890 in January 1986 Adopted by ITU as part of H.323 Adopted by 3GPP for next generation cellular telephony Widespread use in streaming: QuickTime, Real, Microsoft
(HTTP streaming still common)

Recently revised for draft standard status


Work complete, awaiting publication Changes include:
Clarifications and bug-fixes based on experience Scalability improvements Support for new codecs

100% backwards compatible


Copyright 2002 Colin Perkins

Philosophy of RTP
The challenge:
build a mechanism for robust, real-time media delivery above an unreliable and unpredictable transport layer without changing the transport layer

Push responsibility for media delivery onto the end-points where possible

Make the system robust to network problems; media data should be loss tolerant

The end-to-end argument

Application level framing

Copyright 2002 Colin Perkins

The End-to-end Argument


Two options for ensuring reliability
Pass responsibility hop-by-hop, along with the data
Email

Responsibility remains with the end points, which ensure delivery even if the intermediate steps are unreliable

Both TCP and RTP take the second approach Consequences:


Intelligence tends to "bubble-up" the protocol stack to the end points The intermediate systems can be simple, and need not be robust
They can simply discard data they cannot deliver, since it will be recovered end-to-end

The network is dumb, but end-points are smart


Copyright 2002 Colin Perkins

Application Level Framing


Only the application has sufficient knowledge of its data to make an informed decision about how that data should be transported Implications:
The transport protocol should accept data in application meaningful chunks ("ADUs")
The application must understand the data, The application must be able to process ADUs independently, in arbitrary order, and in the presence of loss

The transport protocol should expose details of delivery, allowing the applications to react intelligently if there are problems
Blind retransmission is not always appropriate Maybe the data is stable, and an updated version can be sent Maybe the data is obsolete, and doesn't need to be resent Maybe an alternate representation of the data can be sent

Copyright 2002 Colin Perkins

Philosophy of RTP
The philosophy of RTP implies smart, network-aware, applications that are capable of reacting to problems end-to-end.
Both sender and receiver are intelligent The network is dumb and can be unreliable

Similar principles apply to the signalling protocols


Mostly end-to-end operation, limited support for network state

Fits well with the IP service Contrast with traditional applications:


Telephone network is smart, end-points are dumb MPEG sender is smart, receiver relatively dumb

Copyright 2002 Colin Perkins

Protocol Components
Payload Payload Payload Payload Format Format Format Format RTP Data Transfer Protocol UDP IP

RTP Profile

RTP Control Protocol

Copyright 2002 Colin Perkins

Protocol Components

RTP Data Transfer Protocol

Transports application data units


Audio Video

Provides:
Source identification Payload identification Media sequencing Timing recovery

One RTP stream transports each media type


Copyright 2002 Colin Perkins

Extensions allow for error correction

Protocol Components

RTP Control Protocol

Reception quality feedback


Packet loss fraction Average timing jitter

Mapping from media clock to external time-base


E.g. for lip synchronization

Optional source description


Name, location, email address, phone number
Copyright 2002 Colin Perkins

Loosely coupled membership management

Protocol Components
Payload Payload Payload Payload Format Format Format Format

Provide the adaptation layer between a particular codec and RTP Optimised for robustness to packet loss

Many payload formats exist, with more being developed:


H.261, H.263, M-JPEG, MPEG-2, MPEG-4, BT.656, SMPTE-292

Copyright 2002 Colin Perkins

Protocol Components

RTP Profile

Define use of RTP in particular application scenarios


"Reasonable defaults" Adaptation to unusual conditions
Single source multicast Operation without back channel Authenticated and secure operation
Copyright 2002 Colin Perkins

Provides a namespace for payload formats

Combining the Pieces


A multimedia session comprises several RTP sessions
One for each media type

Each RTP session:


Implements a particular RTP profile Includes an RTP data flow
Transporting a single media type according to one or more payload formats
E.g. Audio switching between G.729 and Fax E.g. Video using MPEG

Includes an RTP control protocol flow


Providing reception quality feedback, user information, etc.

Is defined by:
Source and destination IP addresses A pair of UDP ports: one for RTP, one for RTCP

Copyright 2002 Colin Perkins

RTP Data Transfer Protocol


The RTP Data Transfer Protocol delivers a single media stream from sender to one, or more, receivers
Few assumptions about the underlying transport Usually runs over UDP/IP

RTP provides: Source identification Media identification Media transport


Padding, if necessary Marking of significant events

Typically implemented in an application or as a library


User level, not part of the kernel

Sequencing Timing recovery

Copyright 2002 Colin Perkins

Packet Format

V PX

CC

PT

Sequence Number

Timestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers

Payload data Padding

Copyright 2002 Colin Perkins

Source Identification
Each packet carries a 32 bit synchronization source
Randomly chosen at startup, with collision detection
V P X CC M PT Timestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers Sequence Number

Provides a transport layer independent identifier


Supports gateways IPv4, IPv6, ATM

Payload data Padding

Identifies a single timesynchronized media flow


Mapped to a persistent identifier using RTCP

Copyright 2002 Colin Perkins

Source Identification
Each packet may include a list of contributing sources
Allows data from up to 16 sources to be identified Each CSRC is the SSRC of a mixed participant
V P X CC M PT Timestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers Sequence Number

Payload data

Allows RTP to support mixers and translators


Mixers combine several flows into one
E.g. Conferencing MCU

Padding

Translators change the format of a flow, or gateway between different networks


Transcode to a lower bit-rate Gateway between unicast and multicast

Copyright 2002 Colin Perkins

Communication Models
Mixers and translators greatly expand the range of communications models available to RTP
RTP end point RTP Translator or mixer Multicast group, the network replicates data as necessary, with no translation or mixing.

Point-to-point communication via unicast

Four participants communicating via a multicast group

Replicated unicast: a group of three using an RTP translator/mixer to mediate communications.


Copyright 2002 Colin Perkins

Translated: multicast to unicast. Two participants communicating via a multicast group, with a third linked to the session by an RTP translator.

Media Identification
Each packet carries a 7 bit payload type field Mapped to a payload format during session setup
Allows flexible signalling of codec type and parameters Mapping can be static, if the profile allows
V P X CC M PT Timestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers Sequence Number

Payload data Padding

Each flow carries only one type of media


Only audio, or only video

The payload type allows the sender to switch between a set of payload formats
E.g. a flow carries only audio but may switch between fax and voice at any time

Copyright 2002 Colin Perkins

Media Transport and Payload Formats


Packets contain a block of payload data, described by a payload format Payload formats describe the mapping between codec output and RTP packets
Chosen so that each packet is independently decodable Application level framing
V P X CC M PT Timestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers Sequence Number

Payload data Padding

The payload data typically includes a payload header to ease parsing


E.g. The H.261 payload format copies some information from the GOB header so each set of macro-blocks can be decoded independently

Copyright 2002 Colin Perkins

Media Transport: Marker


Each packet includes a bit to mark significant events
Start of talk spurt for audio Last packet of frame for video
V P X CC M PT Timestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers Sequence Number

A hint that special processing may be required

Payload data Padding

Copyright 2002 Colin Perkins

Media Transport: Padding


Each packet may be padded beyond its natural size Rarely used, but needed by some encryption algorithms
DES in CBC modes operates on 64 bit blocks
V P X CC M PT Timestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers Sequence Number

Payload data Padding

The SRTP profile provides a better security solution

Copyright 2002 Colin Perkins

Sequencing
Each packet contains a 16 bit sequence number
Random initial value Increments monotonically with each packet sent Wraps around to zero when the limit is reached
V P X CC M PT Timestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers Sequence Number

Payload data Padding

Used to detect packet loss


Is not used to determine playout order

Basic RTP does not provide error correction


The receiver is expected to conceal the error, and to continue processing Extensions provide forward error correction and limited retransmission
Copyright 2002 Colin Perkins

Timing Recovery
Each packet contains a 32 bit timestamp Indicates the sampling instant of the oldest payload data
Determines playout order
Payload data V P X CC M PT Timestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers Sequence Number

The clock rate is defined by the payload format:


Audio clock is sampling rate Video clock is 90kHz, indicating the frame time Mapping to codec time-base is also defined

Padding

Time code not carried directly, but mapping to wall clock time via RTCP sender reports

No requirements on stability or accuracy of clock


Implies receiver adaptation

Copyright 2002 Colin Perkins

RTP Control Protocol (RTCP)


Each RTP data flow has an associated control flow The control flow provides:
Time-base management Quality of service feedback Member identification and management

Copyright 2002 Colin Perkins

Time-base Management
Timestamps map between the RTP timeline and NTP wall-clock time
If a common NTP clock is used for multiple streams, a receiver can synchronize them

No explicit transport of SMPTE (or similar) time-codes


Can be derived from NTP timestamps
Accuracy limited by NTP resolution, unless external clock provided RTSP provides a mapping function

Also allows receivers to estimate data/packet rate and possibly clock skew

Copyright 2002 Colin Perkins

Reception Quality Reporting


Quality of service feedback from each receiver:
Loss fraction Cumulative number of packets lost Highest sequence number received Inter-arrival jitter Round-trip time

Many uses:
Loss rate can be used to select amount of FEC to employ Jitter gives estimate of playout buffer delay at receiver

Copyright 2002 Colin Perkins

Membership Management
RTCP provides a canonical name, mapping SSRC to a persistent identifier
Used to associate streams for synchronisation

Augments the membership management provided by the signalling protocol


Primarily using the explicit leave indication

RTCP can optionally deliver source description data:


Name Email address Phone number Location (extend with metadata)

Provides loosely coupled presence information


Explicit leave message
Copyright 2002 Colin Perkins

RTCP reporting interval


RTCP is a low-rate reporting protocol
Not intended for uses that require instant feedback Scalable to very large sessions
Statistical summary of group conditions

Packets are sent periodically


The interval between packets is adjusted to limit RTCP to once per 5 seconds, or 5% of the data rate Randomized to avoid synchronization

Copyright 2002 Colin Perkins

RTP: Summary
Flexible and extensible media transfer protocol
Supports a range of codecs Allows detection of network problems Allows recovery of media timing

Associated, low rate, reporting of reception quality, time-base, and presence information

Copyright 2002 Colin Perkins

Talk Outline
Use of IP for real-time traffic Protocols for real-time multimedia over IP
Outline of signalling protocols Outline of media transport protocols

RTP: Real-time Transport Protocol


RTP data transfer protocol RTP control protocol

Robustness
Playout and timing correction Error correction Congestion control

Security Conclusions

Copyright 2002 Colin Perkins

Robustness
RTP operates over UDP/IP
Best effort delivery Packets can be lost, delayed, reordered, duplicated, etc.

Applications are responsible for correct playout


Timing recovery Error concealment Congestion control

Copyright 2002 Colin Perkins

Timing Recovery
The network can seriously disrupt media timing Receivers must include a jitter compensation buffer to reconstruct the media for playout
Router
Constant inter-packet spacing

Original media stream

Sender

Internet

Network induces timing jitter into the media stream

Received media stream

Router
Variable inter-packet spacing

Receiver

Copyright 2002 Colin Perkins

Playout and Timing Correction

1st talk spurt

2nd talk spurt

Transmission
Jitter affects inter-packet timing

Network Transit Reception Playout Buffer


Network transit delay

Playout
Packet discarded due to late arrival

Playout buffering delay added to compensate for jitter.

Copyright 2002 Colin Perkins

Playout and Timing Correction


RTP does not specify a standard playout buffer or timing reconstruction algorithm
Provides the necessary information; allowing product differentiation

Many trade-offs to consider:


latency versus quality speed of reaction to change buffering ability

Typical design:
Streaming applications use large delay (10+ seconds) Interactive applications try to keep delay low (tens of milliseconds)

Compare with MPEG, where the buffer model is closely defined

Copyright 2002 Colin Perkins

Error Correction
Limited Retransmission
RTCP feedback profile

Unequal error protection Interleaving

Forward Error Correction


Media specific Media independent

Reliable Not timely

Unreliable Timely

TCP

UDP RTP

Add limited reliability to RTP


Copyright 2002 Colin Perkins

Retransmission in RTP
Why must retransmission be limited?
Timely versus reliable We don't want to re-invent TCP

How to implement retransmission?


RTCP provides a back channel Modify the RTCP timing rules to allow early feedback
Keep the fundamental scaling rules to avoid potential for implosion

Copyright 2002 Colin Perkins

RTCP Feedback Profile


RTCP reports sent as usual Feedback can be sent early
Ignore 5 second rule Borrow bandwidth from the next reporting interval Delays next report Send minimal report packet

Transport layer feedback


NACK, ACK sequence number

Payload specific feedback


Reference picture selection MPEG-4 NEWPRED

Under development in IETF

Immediate FB mode

Early RTCP mode

Regular RTCP mode

Report every relevant event immediately

Report many of the events but not all

Just regular RTCP packets

Group size

Copyright 2002 Colin Perkins

Send feedback + regular RTCP packets

Forward Error Correction


Retransmission relies on feedback from receivers
Must request that lost packets are resent

Alternative:
Sender adds redundant data to the media stream Receivers use this to correct errors, without contacting sender

Works well in many cases Two scenarios where it is inefficient:


Round-trip time is large Many receivers, independent loss events

Forward Error Correction


Well known technique at the link layer Also be applicable at the IP level and above

RTP

RTP

RTP

FEC

RTP

RTP

RTP

FEC

RTP

Copyright 2002 Colin Perkins

Media Specific FEC


Some codecs may naturally be loss tolerant Design payload format to take advantage of this

Copyright 2002 Colin Perkins

Media Specific FEC


Some codecs may naturally be loss tolerant Design payload format to take advantage of this
Audio/video redundancy RFC 2198, AMR, H.263+

Original Stream

Redundant data added

Packet lost in transport

Reconstructed stream

Copyright 2002 Colin Perkins

Media Independent FEC


Media specific FEC needs to be produced by the encoder
Part of the payload format Either off-line or on-line

Better if the FEC can be derived from pre-compressed media


Less load on sender Perhaps less network efficient

Compression performed at time of transmission


To pick appropriate FEC Undesirable Too much load on sender to support many streams

Compress
Source image

Packetize
RTP RTP RTP RTP FEC

FEC
Copyright 2002 Colin Perkins

Parity FEC
Parity codes to protect serial communication well known Can apply same technique to packet networks
Generate parity packet

Standard for parity FEC:


RFC 2733 Flexible parity operation

Standards for Reed-Solomon coding under development

Bit stream A

1 Calculate parity to recover B = A XOR (A XOR B)

Parity code: A XOR B

Bit stream B
Copyright 2002 Colin Perkins

Transmission loses B

Unequal Error Protection


Not all data in the packets is equally important
Headers and codec state updates are vital Media data is of variable importance
Data used for predication Data used in a single frame

Some links have high bit error rate


Causes packet corruption Detected by UDP checksum Packet discarded

Seriously impacts wireless link performance


Cellular, especially

Partial Checksum at the UDP level

Copyright 2002 Colin Perkins

Interleaving
Packet loss concealment and correction work best when loss is isolated
Single packet losses

Can interleave, to make bursts of loss appear as random loss


Adds considerable delay

Packet loss on the Internet is bursty

Popular with streaming apps


Part of many audio payload formats

10 11 12 13 14

15 16

Original stream

13

10

14

11 15

12 16

Interleaved stream

13

10

14

12 16

Packet loss

1
Copyright 2002 Colin Perkins

10

12 13 14

16

Reconstructed stream

Error Correction: Summary


Extensive, and ongoing, research and standards work developing error correction for RTP
Retransmission Media/codec specific FEC Media independent FEC Partial checksum Interleaving

Network doesn't have to be perfect


If it's okay for data traffic, it's probably okay for video Unless you have very strict requirements

Acceptable to overprovision for QoS


No real need for RSVP or differentiated services

Well designed applications can tolerate significant loss


Often, 5% loss acceptable

Copyright 2002 Colin Perkins

Congestion Control
But the preceding assumed traffic is well behaved assumed flows respond to congestion in the network or, QoS and flow admission control employed

Copyright 2002 Colin Perkins

Congestion Control
An IP network provides a best-effort packet-switched service.
No admission control The network accepts all packets and tries to deliver them.
Normal operation Packets delivered Congestion Collapse

However, no guarantee of delivery provided


Excess packets discarded if links become congested.

Packets sent

The transport protocol must detect loss, and reduce its rate to allow the congestion to clear
TCP does this automatically RTP does not
Copyright 2002 Colin Perkins

Congestion Control
Adaptation must be done by the application
Possible rate changes depend on the codec Complex feedback loop between codec and network
Normal operation Packets delivered Congestion Collapse

For RTP, implies senders should observe receiver feedback


If loss fraction is non-zero, consider sending less As loss decreases, consider sending faster

Packets sent

Copyright 2002 Colin Perkins

TCP Friendly Rate Control


Possible to predict the longterm average throughput of TCP
T= R s 3p 2p + 3 p(1 + 32 p 2 ) Trto 8 3

The "best current practice" congestion control scheme for multimedia flows With appropriate parameters:
Fair to TCP on average Slowly changing rate

Derive throughput in terms of observed loss rate, RTT and packet size
Measurable qualities in RTP

Adapt sending rate to match


Driven by reception of RTCP Drives codec operation

Copyright 2002 Colin Perkins

Limitations of Congestion Control


TCP friendly algorithms are new, and evolving
Very limited deployment Not clear that they have reached their final form

Human factors aspects play a key role


Congestion control implies variable quality Subjectively very annoying, unless the rate of change is slow Can have a significant impact on congestion control

Interactions between codec and network are not well defined


Unclear how slow response, or limited adaptability, will impact fairness

Copyright 2002 Colin Perkins

Talk Outline
Use of IP for real-time traffic Protocols for real-time multimedia over IP
Outline of signalling protocols Outline of media transport protocols

RTP: Real-time Transport Protocol


RTP data transfer protocol RTP control protocol

Robustness
Playout and timing correction Error correction Congestion control

Security Conclusions

Copyright 2002 Colin Perkins

Security
Several aspects to multimedia security Confidentiality of the media RTP can help here Authentication of the sender Watermarking Storage

Copyright 2002 Colin Perkins

Security in RTP
Basic RTP provides limited security
Packets may be encrypted
DES is specified; algorithm may be negotiated during session setup

A secure RTP profile is under development


Encryption for confidentiality
Encrypts only the payload data, not the headers AES in counter or F8 mode Robust to bit errors, allows header compression, suitable for cellular wireless

Does not support sender authentication

Sender authentication
Adds a trailer to each packet, containing authentication code HMAC-SHA1

Copyright 2002 Colin Perkins

Watermarking, Storage and DRM


RTP is concerned only with the transmission of media Does not consider:
The contents of the video image How the image is captured, generated and stored

Implications:
Receivers may store the payload
RTP cannot influence the process The copy may not be perfect

Watermarks or other embedded data can be inserted at source


But must be robust to packet loss

Copyright 2002 Colin Perkins

Talk Outline
Use of IP for real-time traffic Protocols for real-time multimedia over IP
Outline of signalling protocols Outline of media transport protocols

RTP: Real-time Transport Protocol


RTP data transfer protocol RTP control protocol

Robustness
Playout and timing correction Error correction Congestion control

Security Conclusions

Copyright 2002 Colin Perkins

Conclusions
IP provides a non-optimal service for video transport
Careful network engineering can solve many problems

Multimedia protocol framework comprises:


Signalling: H.323, SIP, RTSP, SAP Media Transport: RTP + codecs

RTP provides:
Robust, flexible and extensible media transport Range of error correction schemes Range of security solutions

Limitations:
Congestion control

Copyright 2002 Colin Perkins

For More Information


IETF Audio/Video Transport Working Group http://www.ietf.org/html.charters/avt-charter.html

Colin Perkins http://www.east.isi.edu/~csp/

Copyright 2002 Colin Perkins

You might also like