Professional Documents
Culture Documents
Internet Multimedia
Internet Multimedia has long history:
RFC 741, "Network Voice Protocol", 1977 First video experiments in the early 1980s
Widespread availability of suitable networks in the last couple of years Unusual characteristics, worth exploring
Copyright 2002 Colin Perkins
Talk Outline
Use of IP for real-time traffic Protocols for real-time multimedia over IP
Outline of signalling protocols Outline of media transport protocols
Robustness
Playout and timing correction Error correction Security Congestion control
Security Conclusions
Talk Outline
Use of IP for real-time traffic Protocols for real-time multimedia over IP
Outline of signalling protocols Outline of media transport protocols
Robustness
Playout and timing correction Error correction Security Congestion control
Security Conclusions
IP
ADSL Ethernet PPP Twisted Pair Optical Fibre Wireless
IP Service Model
The IP service model is limited Best effort packet transport Fragmentation Routing and addressing
and the transport protocol must compensate Checksum to catch bit errors
45
Time of day
Timing Disruption
Reception time
Queuing jitter causes variation in inter-packet arrival time. Non-45 slope shows presence of clock skew Transmission time
Loss and jitter can be made arbitrarily low through careful engineering
Most backbone networks have very good performance Interconnects and customer LANs are currently the main trouble spots Explicit QoS doesn't appear necessary
Fragmentation
IP fragments packets that exceed the MTU
Often >1500 octets
Original Fragments
The use of IP with best effort transport implies heterogeneity Multicast makes this an order of magnitude worse
Each receiver sees different loss characteristics Hard to get timely feedback in a scalable manner
Transport Protocols
The IP service, by itself, is very limited
Just (tries to) deliver packets
TCP
Enhances the raw IP service
Point-to-point and connection oriented Service selection through ports Reliable, in-order, delivery Rate adaptation to match network capacity
TCP
Rate adaptation matches throughput to capacity
Congestion Window Size
UDP
Uses the services of IP
best effort (unreliable) but timely fragmentation routing and addressing unicast and multicast
Reliability/Timeliness Tradeoff
Unreliable Timely
TCP
UDP RTP
Talk Outline
Use of IP for real-time traffic Protocols for real-time multimedia over IP
Outline of signalling protocols Outline of media transport protocols
Robustness
Playout and timing correction Error correction Security Congestion control
Security Conclusions
Signalling Protocols
The first stage of any multimedia session is signalling User location Session initiation, call setup and teardown Media negotiation Conference control Many signalling protocols exist Teleconferencing: H.323 Telephony: SIP Streaming: RTSP, SAP
RAS
Design Choices
Depending on the scenario, implement:
An appropriate signalling protocol
RTSP SIP/H.323 SAP
Talk Outline
Use of IP for real-time traffic Protocols for real-time multimedia over IP
Outline of signalling protocols Outline of media transport protocols
Robustness
Playout and timing correction Error correction Congestion control
Security Conclusions
Philosophy of RTP
The challenge:
build a mechanism for robust, real-time media delivery above an unreliable and unpredictable transport layer without changing the transport layer
Push responsibility for media delivery onto the end-points where possible
Make the system robust to network problems; media data should be loss tolerant
Responsibility remains with the end points, which ensure delivery even if the intermediate steps are unreliable
The transport protocol should expose details of delivery, allowing the applications to react intelligently if there are problems
Blind retransmission is not always appropriate Maybe the data is stable, and an updated version can be sent Maybe the data is obsolete, and doesn't need to be resent Maybe an alternate representation of the data can be sent
Philosophy of RTP
The philosophy of RTP implies smart, network-aware, applications that are capable of reacting to problems end-to-end.
Both sender and receiver are intelligent The network is dumb and can be unreliable
Protocol Components
Payload Payload Payload Payload Format Format Format Format RTP Data Transfer Protocol UDP IP
RTP Profile
Protocol Components
Provides:
Source identification Payload identification Media sequencing Timing recovery
Protocol Components
Protocol Components
Payload Payload Payload Payload Format Format Format Format
Provide the adaptation layer between a particular codec and RTP Optimised for robustness to packet loss
Protocol Components
RTP Profile
Is defined by:
Source and destination IP addresses A pair of UDP ports: one for RTP, one for RTCP
Packet Format
V PX
CC
PT
Sequence Number
Source Identification
Each packet carries a 32 bit synchronization source
Randomly chosen at startup, with collision detection
V P X CC M PT Timestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers Sequence Number
Source Identification
Each packet may include a list of contributing sources
Allows data from up to 16 sources to be identified Each CSRC is the SSRC of a mixed participant
V P X CC M PT Timestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers Sequence Number
Payload data
Padding
Communication Models
Mixers and translators greatly expand the range of communications models available to RTP
RTP end point RTP Translator or mixer Multicast group, the network replicates data as necessary, with no translation or mixing.
Translated: multicast to unicast. Two participants communicating via a multicast group, with a third linked to the session by an RTP translator.
Media Identification
Each packet carries a 7 bit payload type field Mapped to a payload format during session setup
Allows flexible signalling of codec type and parameters Mapping can be static, if the profile allows
V P X CC M PT Timestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers Sequence Number
The payload type allows the sender to switch between a set of payload formats
E.g. a flow carries only audio but may switch between fax and voice at any time
Sequencing
Each packet contains a 16 bit sequence number
Random initial value Increments monotonically with each packet sent Wraps around to zero when the limit is reached
V P X CC M PT Timestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers Sequence Number
Timing Recovery
Each packet contains a 32 bit timestamp Indicates the sampling instant of the oldest payload data
Determines playout order
Payload data V P X CC M PT Timestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers Sequence Number
Padding
Time code not carried directly, but mapping to wall clock time via RTCP sender reports
Time-base Management
Timestamps map between the RTP timeline and NTP wall-clock time
If a common NTP clock is used for multiple streams, a receiver can synchronize them
Also allows receivers to estimate data/packet rate and possibly clock skew
Many uses:
Loss rate can be used to select amount of FEC to employ Jitter gives estimate of playout buffer delay at receiver
Membership Management
RTCP provides a canonical name, mapping SSRC to a persistent identifier
Used to associate streams for synchronisation
RTP: Summary
Flexible and extensible media transfer protocol
Supports a range of codecs Allows detection of network problems Allows recovery of media timing
Associated, low rate, reporting of reception quality, time-base, and presence information
Talk Outline
Use of IP for real-time traffic Protocols for real-time multimedia over IP
Outline of signalling protocols Outline of media transport protocols
Robustness
Playout and timing correction Error correction Congestion control
Security Conclusions
Robustness
RTP operates over UDP/IP
Best effort delivery Packets can be lost, delayed, reordered, duplicated, etc.
Timing Recovery
The network can seriously disrupt media timing Receivers must include a jitter compensation buffer to reconstruct the media for playout
Router
Constant inter-packet spacing
Sender
Internet
Router
Variable inter-packet spacing
Receiver
Transmission
Jitter affects inter-packet timing
Playout
Packet discarded due to late arrival
Typical design:
Streaming applications use large delay (10+ seconds) Interactive applications try to keep delay low (tens of milliseconds)
Error Correction
Limited Retransmission
RTCP feedback profile
Unreliable Timely
TCP
UDP RTP
Retransmission in RTP
Why must retransmission be limited?
Timely versus reliable We don't want to re-invent TCP
Immediate FB mode
Group size
Alternative:
Sender adds redundant data to the media stream Receivers use this to correct errors, without contacting sender
RTP
RTP
RTP
FEC
RTP
RTP
RTP
FEC
RTP
Original Stream
Reconstructed stream
Compress
Source image
Packetize
RTP RTP RTP RTP FEC
FEC
Copyright 2002 Colin Perkins
Parity FEC
Parity codes to protect serial communication well known Can apply same technique to packet networks
Generate parity packet
Bit stream A
Bit stream B
Copyright 2002 Colin Perkins
Transmission loses B
Interleaving
Packet loss concealment and correction work best when loss is isolated
Single packet losses
10 11 12 13 14
15 16
Original stream
13
10
14
11 15
12 16
Interleaved stream
13
10
14
12 16
Packet loss
1
Copyright 2002 Colin Perkins
10
12 13 14
16
Reconstructed stream
Congestion Control
But the preceding assumed traffic is well behaved assumed flows respond to congestion in the network or, QoS and flow admission control employed
Congestion Control
An IP network provides a best-effort packet-switched service.
No admission control The network accepts all packets and tries to deliver them.
Normal operation Packets delivered Congestion Collapse
Packets sent
The transport protocol must detect loss, and reduce its rate to allow the congestion to clear
TCP does this automatically RTP does not
Copyright 2002 Colin Perkins
Congestion Control
Adaptation must be done by the application
Possible rate changes depend on the codec Complex feedback loop between codec and network
Normal operation Packets delivered Congestion Collapse
Packets sent
The "best current practice" congestion control scheme for multimedia flows With appropriate parameters:
Fair to TCP on average Slowly changing rate
Derive throughput in terms of observed loss rate, RTT and packet size
Measurable qualities in RTP
Talk Outline
Use of IP for real-time traffic Protocols for real-time multimedia over IP
Outline of signalling protocols Outline of media transport protocols
Robustness
Playout and timing correction Error correction Congestion control
Security Conclusions
Security
Several aspects to multimedia security Confidentiality of the media RTP can help here Authentication of the sender Watermarking Storage
Security in RTP
Basic RTP provides limited security
Packets may be encrypted
DES is specified; algorithm may be negotiated during session setup
Sender authentication
Adds a trailer to each packet, containing authentication code HMAC-SHA1
Implications:
Receivers may store the payload
RTP cannot influence the process The copy may not be perfect
Talk Outline
Use of IP for real-time traffic Protocols for real-time multimedia over IP
Outline of signalling protocols Outline of media transport protocols
Robustness
Playout and timing correction Error correction Congestion control
Security Conclusions
Conclusions
IP provides a non-optimal service for video transport
Careful network engineering can solve many problems
RTP provides:
Robust, flexible and extensible media transport Range of error correction schemes Range of security solutions
Limitations:
Congestion control