Rate-Distortion Optimized Video
Peer-to-Peer Multicast Streaming
Invited Paper
Eric Setton Jeonghun Noh
esetton@stanford.edu jhnoh@stanford.edu
Information Systems Laboratory, Department of Electrical Engineering
Stanford University, Stanford, CA94305-9510, USA
ABSTRACT
We study peer-to-peer multicast streaming, where a source
distributes real-time video to a large population of hosts
by making use of their forwarding capacity rather than re-
lying on dedicated media servers. Hosts which may dis-
connect at any time, therefore a robust control protocol is
needed to maintain connectivity among peers. This work
presents a new peer-to-peer multicast protocol and analyzes
the gains that video coding and prioritized packet scheduling
at the application layer can bring to the overall streaming
performance. A rate-distortion model which predicts end-
to-end video quality in throughput limited environments is
presented and used to determine the over-provisioning nec-
essary to avoid self-inflicted congestion. The video stream
transmitted by the source contains H.264 SP and SI frames,
which are used to adaptively stop error propagation due to
packet loss. Distortion-optimized retransmission requests
are issued by receiving hosts to recover the most impor-
tant missing packets while limiting the induced congestion.
Experiments for several hundred hosts simulated in NS-2
illustrate the benefits of our system. We achieve typical
end-to-end delays of 1 sec, and a stable video quality with
less than 2.5% of frames lost to playout interruptions.
Categories and Subject Descriptors
C.2 [Computer-communication networks]: Distributed
systems.
General Terms
Design, performance.
Keywords
Peer-to-peer, video streaming, multicast.
∗This work was supported, in part, by a gift from Hewlett-
Packard Laboratories, Palo Alto, CA.
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
P2PMMS’05, November 11, 2005, Singapore.
Copyright 2005 ACM 1-59593-248-8/05/0011 ...$5.00.
39
∗
Bernd Girod
bgirod@stanford.edu
1. INTRODUCTION
As IP multicast is not universally supported, distribution
of media streams in the public Internet to a large audience
(“multicasting”) is typically realized by a large number of
unicast connections. If the maximum number of streams of
an individual media server (typically between a few hundred
and a few thousand) is exceeded, additional server capacity
must be provided by a suitable content-delivery infrastruc-
ture, e.g., in the form of a network of replication servers.
Peer-to-peer (P2P) multicasting is an elegant alternative
in which each end-host may act as a potential server for other
clients. This avoids dedicated replication servers altogether.
The approach is self-scaling, as the number of peer “servers”
and peer clients increases at the same rate, hence it avoids
the bottleneck of a central server (or dedicated replication
server). The approach, in principle, would allow a highly
dynamic support of changing multicast demand at very low
cost. The major challenge, however, is the complete lack
of performance guarantees in the P2P network. Peer nodes
might be turned off or disconnected at any time without
prior notice, while other nodes join or re-join. Such a highly
unreliable network fabric poses a major difficulty for media
streaming.
A recent study based on statistics collected over the Inter-
net reveals that although the uplink throughput of peers is a
limiting factor, there is enough bandwidth for P2P stream-
ing, even on a large scale [21]. The stability of the system
which is necessary to provide a satisfactory user experience
largely depends on the design of the protocol. To gain ro-
bustness and possibly aggregate data rate, path diversity
should be attained by distributing streams across a suffi-
ciently large number of complementary multicast trees. In-
dividual nodes are important nodes near the top in some
multicast tree, but near the bottom and less important in
others, thus avoiding single points of failure in the node de-
pendency graph.
Although the control protocol is essential to provide effi-
cient means of building and maintaining multiple multicast
trees, it needs to be combined with efficient multimedia cod-
ing and streaming solutions at the application layer. State-
of-the art compression which achieves better rate-distortion
performance alleviates the bandwidth requirements, while
error-resilient streaming techniques may improve the received
media quality.
In this paper, we present a P2P multicast protocol and
show the benefits of video coding, adaptive streaming and
optimized packet scheduling for such a system. The purpose
of this work is to make the following contributions:
• the design and a analysis of a new distributed P2P
multicast protocol, targeted for low-latency streaming,
• a video distortion model analyzing the streaming per-
formance in a throughput-limited P2P network,
• an adaptive video streaming technique, suitable for a
P2P network, to mitigate error-propagation through
the use of H.264 SP and SI picture types,
• a distortion-optimized retransmission scheduler which
maximizes video quality while limiting the impact on
congestion.
Although the gains reported in this paper for streaming
with SP and SI frames and for the retransmission scheduler
are shown for our implementation of a P2P multicast control
protocol, we believe the results are more general and could
be applied to most implementations of video P2P streaming.
In the next section, we describe the control protocol which
builds and maintains multiple multicast trees to broadcast
a video stream from a source to a set of peers. The per-
formance of the control protocol is assessed through exper-
iments carried out over a simulated network in NS-2 [6].
In Sec. 4, we make use of a video distortion model, previ-
ously proposed in [20], to predict the received video qual-
ity when a video stream is sent from multiple throughput-
limited senders to a receiver at different rates. The model
is used to determine the amount of over-provisioning of net-
work resources required to limit the congestion created be-
tween peers. The video stream multicast over the P2P net-
work contains the new picture types “switching-P” (SP)
and “switching-I” (SI), introduced in the latest video cod-
ing standard H.264 [12]. These switching picture types can
be sent adaptively to stop error propagation in the case of
transmission errors. In Sec. 5, we characterize the bit rate
savings and performance improvement achieved by using SP
and SI frames compared to traditional video transmission
based on I and P frames. In the last part of the paper, we
focus on retransmission scheduling from a receiving host to
its parents. Different from network-level multicast, the in-
corporation of application level retransmission requests into
P2P multicast is possible without feedback implosion, since
the fan-out of each individual node is small. We describe
in Sec.6, how to schedule retransmissions, from the receiver,
to maximize its decoded video quality while limiting the in-
curred network congestion.
2. RELATED WORK
We address the problem of reducing the cost of large-scale
live media delivery and focus on IP networks such as the
Internet. Commercial solutions deployed today mostly rely
on server overlays which act as Content Delivery Networks
(CDN). When a user wishes to access multimedia content,
he is re-directed towards one of the servers of the CDN.
This server is usually located closer to the user and has
enough available bandwidth to support the media streaming
session. Such networks have been deployed, e.g., by Akamai
or Cisco Systems. The design of these large scale systems is
a problem of itself which has been studied, notably in [13]
and [7].
40
Live media multicast over P2P networks proposes to shift
the burden of the media delivery from a dedicated infras-
tructure to the users. As P2P networks do not require any
special servers or routers, the cost of such solutions is ap-
pealing. However, they only offer an attractive alternative
if their robustness can overcome the dynamic behavior of
the peers used for forwarding. Some functioning solutions
based on P2P multicast are already available. Coolstream-
ing which offers Chinese and American cable channels re-
ports an ever-growing user base [1, 23]. Other implemen-
tations exist such as PPLive [5], which multicasts mostly
sports channels, and ESM [3] which has been used to broad-
cast conferences in the scientific community. Although these
implementations are very exciting advances, they all suffer
from long startup delays usually on the order of 3-5 minutes,
instability, and their streaming quality is not yet a threat
to standard definition TV. This motivates our work which
places an emphasis on the role of congestion in P2P video
streaming and strives for stability and low startup latencies
of a few seconds, comparable to cable or satellite television
systems.
In most P2P multicast protocol proposed so far peers
build and maintain one or several multicast trees along which
video content is distributed [8, 10, 17, 22]. The construc-
tion of the trees usually proceeds in a distributed fashion,
which allows the protocol to scale without overwhelming the
source. The control protocol we propose is similar to these
previous systems. Our protocol establishes multiple multi-
cast trees as proposed in [11, 17]. In these systems, multiple
trees are used to mitigate the impact of peer disconnection.
Through the use of multiple description coding or of for-
ward error correction (FEC) they also maintain acceptable
video quality even when a node is not connected to all the
trees. In our system, we also make use of optimized encod-
ing and application layer solutions. The overall performance
of P2P video multicast systems can be greatly improved
by using efficient single-description coding techniques and
achieving the required robustness through over-provisioning
and distortion-optimized scheduling of transmissions and re-
transmissions.
3. P2P MULTICAST PROTOCOL
The control protocol enables a source to distribute a video
stream to a population of peers via P2P multicast. The
video source peer and other peers are connected via multi-
ple trees which are constructed dynamically by the protocol.
The source is the root of all trees and the trees are built and
maintained mostly independently. The branches of each tree
connect a host to its descendants. These links are virtual
tunnels which hide the underlying physical network topol-
ogy. The video stream is distributed evenly over the different
trees. Hence, peers need to join each of the multiple trees
to decode and play out the video successfully.
Our simulations are based on a moderate size network of
a few hundred nodes, which resembles a large private in-
tranet or a campus network. An example is shown in Fig. 1.
We make the following simplifying assumptions: the control
and transmission protocol is implemented over the UDP/IP
protocol stack and we ignore any Network Address Trans-
lator or firewall issue which may limit connectivity; peers
are synchronized and have heterogeneous but fixed upload
bandwidth which they have measured and know accurately.
Although these problems need to be addressed for a real In-

Figure 1: Example of network topology used for sim-
ulation. The setup used in the experiments is an
extended version of this example.
ternet implementation (see e.g. [10]), they are not directly
relevant to the scope of this paper.
Our control protocol is completely distributed, except for
an approximate list of connected peers maintained by the
source. Besides accepting new peers, the protocol maintains
the multicast trees when peers leave or are disconnected
during the session. A peer may leave ungracefully due to
network shutdown or system failure. To stay connected,
peers need to monitor the state of their ancestors and may
decide to reconnect if they detect traffic interruptions.
3.1 Joining
Each joining host discovers the address of the source of the
video stream by a directory service such as a website and
contacts this peer to obtain a list of hosts randomly chosen
among connected members. The joining peer contacts all
the members of the list and waits for replies. It then de-
termines a candidate parent for each tree. If the latter has
enough available bandwidth at the time of the request, it
accepts the host and starts forwarding video packets. This
process is called “6-way handshaking” and consists of one
message exchange with the source and 2 message exchanges
with the candidate parent.
3.1.1 Partial list of connected peers
To reduce control overhead, the source adjusts the size of
the list sent to joining peers according to the current group
size and the number of multicast trees. At the beginning of
the session, the group size is small and the list size increases
linearly. When the group size reaches a certain point, the
list size increases logarithmically. For multiple trees, the
source sends a larger list to allow a joining peer to exploit
path diversity.
3.1.2 Parent selection algorithm
Parents which have enough throughput to support an ad-
ditional peer are selected based on their proximity to the
source. Several other criteria can also be considered in this
process. For example, the amount of available throughput,
the round-trip time (RTT) or geographical proximity. Simi-
larly to the results reported in [21], we found that the prox-
imity to the source gave good performance as will be ex-
plained in more detail below. To make use of path diversity
41
and reduce multiple tree failures due to a common parent,
different parents are chosen as often as possible. Once the
selection of candidates for each tree is done, attachment re-
quests are sent out and each tree will operate mostly inde-
pendently.
3.1.3 Periodic message exchange
Once a peer is connected, it will inform its parents of its
presence by transmitting periodic “hello” messages. These
messages are also used to propagate topology information
such as the subtree size of a peer. Reception of a “hello”
message generates an immediate response intended to con-
firm the parent’s presence.
3.2 Node disconnection
Ungraceful leaves occur when a peer leaves the group with-
out notice which may cause disconnection of the peer’s de-
scendants from the group. When a host leaves, it stops for-
warding video packets and is unresponsive to probing. Once
a peer detects a parent disconnection it will try to reconnect
to the tree by choosing one of its other parents. If this fast
recovery mechanism fails, the peer has to contact the source
to get a new list of candidate parents. While the host re-
connects, retransmission requests are issued over the other
multicast trees to recover missing video packets. We will
explain this retransmission procedure in detail in Sec. 6.
A list of extra links to other potential parents can also be
maintained by each peer. This list is initialized during the
joining process. After a host joins the different multicast
trees, it keeps the list of remaining available hosts to con-
struct a pool of extra links. If the number of hosts in the
pool falls below a certain threshold the list is updated using
a gossip algorithm. Extra links are not reserved resources
but do however indicate which hosts are available for po-
tential reconnection. The performance of this other rejoin
procedure is not presented in this paper but shows that the
procedure should be lightweight in order to avoid creating
additional control overhead.
We observed that the time threshold necessary to detect
parent disconnections may be large (approximately 1-2 sec-
onds), as it is important to avoid false detections, and since
losses will be compensated by retransmissions requested dur-
ing reconnection. Detecting the disconnection of a child is
less influential to the overall performance as it only results in
a temporary waste of the parent’s network resources. Since
the penalty of a false child leave detection is high, a longer
time interval is used. When a child leave is detected, parents
will remove it from their forwarding table.
3.3 Loop avoidance
Once a peer looses its connection to one of the trees, it
will try to rejoin this tree and will send out new attachment
requests. During this process it may try to attach to one of
its descendants. This will create a loop which will eventu-
ally starve the whole subtree. To prevent this kind of event,
each peer keeps the list of its ancestors, which are peers in
the path from the peer to the source. Attachment requests
issued by an ancestor are rejected. Note that this happens
independently over each tree and that the hierarchy between
peers can and should be different over different trees. This
loop avoidance mechanism does not require global knowl-
edge of the tree but only of a small subset of its nodes, hence
additional memory and processing power are negligible.
3.4 Simulation setup
To evaluate the performance of the protocol we carry out
experiments over a network simulated in NS-2 [6]. Figure 1
illustrates the star-shape topology of the network support-
ing the hosts. Simulations are run over a similar topology
with 1000 nodes, 750 of which are placed at the extremity
of the network. The actual number of peers participating in
each simulation is 300, placed evenly among the edge-nodes.
The backbone links are sufficiently provisioned so that con-
gestion only occurs on the links connecting the peers to the
network. The latency of each link is 5 ms, and the diameter
of the network is 10 hops. Losses are only due to conges-
tion and overflowing queues, and transmission errors due to
the presence of ISP boundaries or potential wireless last-hop
links are ignored.
The bandwidth of the peers reflects today’s available ADSL
network access technology. The bandwidth distribution is
given in Tab. 1. The degree indicates the number of chil-
dren a parent transmitting a video stream encoded at 220
kbps can potentially support. This calculation includes a
bit-rate reserve of 33% to account for instantaneous rate
fluctuations, retransmissions and control overhead. We re-
fer to this bit-rate reserve as ”overprovisioning” throughout
this paper. The overprovisioning factor will be justified in
more detail in Sec. 4. The distribution is similar to the find-
ings reported in [21]. Note that more than half of the peers
do not have enough bandwidth to forward the video stream.
This makes them free-riders in a system with only 1 distri-
bution tree to forward data. When more trees are used, the
video stream is divided into several smaller sub-streams and
these peers may forward data to one or several peers.
Table 1: Distribution of peer bandwidth. The de-
gree is computed according to a video stream en-
coded at 220 kbps with an over-provisioning factor
of 33%.
Downlink Uplink Degree Percentage
512 kbps 256 kbps 0 56%
3 Mbps 384 kbps 1 21%
1.5 Mbps 896 kbps 3 9%
20 Mbps 2 Mbps 6 3%
20 Mbps 5 Mbps 17 11%
In the experiments, the dynamic behavior of peers is mod-
elled as follows. A flash crowd is simulated by letting all the
peers request the video during the first minute of the video
session. During the remaining 14 minutes, peers join and
leave the session ungracefully, following a random Poisson
process. Peers remain “on” for 4.5 minutes and “off” for 30
seconds, on average.
The 10s video sequence Mother and Daughter is transmit-
ted from the source to the peers. It is looped enough times
to cover the whole session. The video stream is encoded
with H.264 [12] at a constant quality and the encoding rate
is approximately 220 kbps. We use the freely available ver-
sion of the codec JM 9.2 maintained by the Video Coding
Expert Group (VCEG)[4]. The encoding structure will be
described in detail in Sec.5. Each video frame is packetized
into UDP packets. Frames exceeding the maximum trans-
mission unit size are fragmented before packetization. When
the simulation begins, the source peer multicasts the video
42
by sending UDP packets over the different multicast trees
in round-robin fashion.
3.5 Protocol evaluation
We begin by analyzing the performance of the protocol
connection and re-connection process, then compare the re-
sults obtained for control overhead and video quality over
different numbers of multicast trees.
1 1
0.9 0.9
Cumulative distribution function
Cumulative distribution function
0.8 0.8
0.7 0.7
0.6 0.6
0.5 0.5
0.4 0.4
0.3 0.3
0.2 0.2
0.1 minimum hop 0.1 minimum hop
minimum rtt minimum rtt
0 0
0 2 4 0 2 4
Join time (s) Rejoin time (s)
Figure 2: Distribution of the time needed to join
the multicast trees (left) and to rejoin a tree after a
parent disconnection (right).
Figure 2 shows the cumulative distribution function (cdf)
of the join time, i.e. the time necessary for a node to join
a multicast tree; the cdf of the rejoin time is also plotted.
This is the time a node takes to rejoin a tree once the discon-
nection of its parent has been detected. Results are shown
for two different parent selection algorithms, the first one
is based on the proximity to the source and denoted by
“minimum hop”, the second is based on RTT. The join-
ing procedure takes an average of 0.63 second, parent leaves
are detected in less than 2 seconds and it takes an addi-
tional 0.70 second to rejoin, on average. In terms of latency,
both algorithms show little difference. However, this does
not translate to equivalent overall performance. The “mini-
mum hop” parent selection leads to more stable trees which
translates into higher video quality over time. In our ex-
periments, 80% more rejoin events were observed when the
minimum RTT method was chosen. This justifies the use of
“minimum hop” as our parent selection algorithm.
In Fig. 3, the control traffic overhead and the average
video quality is shown when video is multicast over differ-
ent number of trees. The percentage of overhead obviously
depends on the encoding rate of the video. The video qual-
ity is measured by collecting the mean square error (MSE)
between the luminance of the decoded video signal and the
original video signal. The MSE is translated into Peak Sig-
nal to Noise Ratio (PSNR), and represented in dB. This
metric will be used throughout the paper. A difference in
PSNR exceeding 1 dB is considered visually significant. In
Fig. 3, PSNR is averaged both over time and over the dif-
ferent peers.
As illustrated, as long as 4 multiple trees or less are main-
tained, the overhead stays between 2 and 4.5% of the total
 8%
7%
Percentage of control overhead
6%
5%
4%
3%
2%
Average quality
1%
Control overhead
0%
1 2 3 4 5 6
Number of trees
Figure 3: Percentage of control protocol overhead
and average video quality for different numbers of
multicast trees.
traffic observed on the downlink of the peers. When more
trees are used, the control traffic increases, this is due a
higher frequency of parent leave generating more traffic ex-
change between the peers. When only 1 tree is maintained,
the control traffic is higher than expected. In this case, a
parent leave cannot be accommodated by one of the other
remaining parents, as it happens when more trees are used.
This creates additional control traffic.
When video is distributed over a larger number of multiple
trees, the effect of an ungraceful leave is less important as
children have several parents which they can use for rejoin-
ing. On the other hand, maintaining more trees increases
the probability of a parent disconnection and requires more
control traffic. In Fig. 3, this tradeoff is shown in terms of
the average video quality, as a function of the number of
multiple multicast trees maintained by the protocol. In this
environment, the optimal tradeoff between robustness and
congestion is obtained when 2 multiple trees are used, and
the performance observed for 3 and 4 trees is close to the
optimal. Depending on the rate of the video, on whether or
not retransmissions are requested and on the dynamic peer
behavior, this optimal could occur for a different number of
trees. However, we believe a small number of distribution
trees is enough to guarantee good performance. In most
of the experiments presented in the following video will be
transmitted over 4 multiple trees.
4. VIDEO DISTORTION MODEL
For live video streaming applications, video packets are
transmitted over the network and need to meet a playout
deadline. Decoded video quality at the receiver is there-
fore affected by two factors: distortion introduced by the
encoder, denoted by Denc , and distortion due to packet loss
or late arrivals, denoted by Dloss . Assuming an additive
relation of these two independent factors, a video distortion
model was derived in [20]. The decoded video distortion,
40 Ddec , is given by:
Ddec = Denc + Dloss , (1)
39
Denc = D0 + θ/(R − R0 ), (2)
Dloss = κ(Pr + (1 − Pr )e−(C−R)T /L ), (3)
38

n
C = Ci , (4)
PSNR (dB)
37 i=1

n
R = Ri (5)
36 i=1
In (2), R is the total rate of the video stream, and the pa-
35 rameters D0 , θ and R0 are estimated from empirical rate-
distortion curves via regression techniques.
The second distortion term, Dloss , depends linearly on the
34
7 8 packet loss rate. The scaling factor κ indicates the sensitiv-
ity of the stream to losses. It depends on the encoding struc-
ture and on the amount of motion present in the sequence.
The other factor reflects the combined rate of random losses
and late arrivals. Pr is the random packet loss rate and T is
the time within which each packet should reach the receiver
(typically a few hundred milliseconds). C is the aggregate
available throughput of the network paths over which the
video is transmitted and L is the average packet size.
In (4), the throughput, C, is expressed as the sum of the
available throughput between the receiver and its parents
on the n different multicast trees. Likewise, in (5), Ri is the
fraction of the video stream transmitted over the ith multi-
cast tree. Typically, the same throughput will be reserved
for a receiver on each tree and the video will be divided
equally among the trees: Ri = R/n and Ci = C/n.
This model reflects the impact of the rate on video distor-
tion. At lower rates, reconstructed video quality is limited
by coarse quantization, whereas at high rates, more packets
are delayed beyond their playout deadline due to network
congestion. For live video steaming in a bandwidth-limited
environment, we therefore expect to achieve maximum de-
coded quality for some intermediate rate. As an illustration,
we collect the decoded video quality of a sequences trans-
mitted to a peer using different numbers of multicast trees.
The parents of the peer each have a total available uplink
throughput of 660 kbps. A fourth of this capacity (165 kbps)
is reserved by each parent to the peer to carry the video traf-
fic, the rest being used to serve other peers which share the
same parent. Figure 4 illustrates the decoded video quality
for the sequence Mother and Daughter encoded at different
rates. The fitted model is shown along with experimen-
tal points. In this experiment the playout deadline is 500
ms. The decrease in quality due to congestion is illustrated
by the bell shape of the curve representing decoded video
quality. In this experiment the decrease occurs when the
rate of the video reaches approximately 85% of the available
throughput.
For P2P streaming, it is essential to limit the amount
of self-inflicted congestion created by the media streams.
Indeed, as there might be a large number of intermediate
end-hosts separating a peer from the source, any increase in
network congestion, may be reflected multiplicatively in the
total end-to-end delay. Combined with physical link latency
this delay may cause some packets to miss their playout
deadline, resulting in decoding errors and a decrease in de-
coded quality. Hence, for fine tuning, congestion may be
reduced by the following methods: increasing the amount of
43
 43
42
41
40
PSNR (dB)
39
38
37 1 tree
2 trees
36 3 trees
35
150 200 250 300 350 400 450
Rate kbps
Figure 4: Streaming performance for the sequence
Mother and Daughter transmitted to a receiver over
a different number of multicast trees. The experi-
mental results are shown together with the model,
represented by a solid line. The parameters of the
model are D0 = 0.49, θ = 1222, R0 = 10.39kbps, κ = 185,
T = 0.5s and L = 1.5kbit.
over-provisioning by reserving a larger capacity Ci on each
of the multicast trees between a parent and a child; de-
creasing the encoding rate of the video R. When video is
pre-encoded, as is often the case, it is not always possible
to reduce the streaming rate. Thus, it may be necessary to
employ more over-provisioning, this in turn may decrease
the number of hosts supported by the P2P system. The
model can be used to determine the right amount of over-
provisioning. For example, given the rate of the video, the
number of trees, and the playout deadline, we can deter-
mine the throughput necessary on each tree to keep Dloss
under a given threshold. For video encoding rates of 100
kbps or more, and playout deadlines of no less than 300 ms,
Ci = 1.33 ∗ Ri is enough to avoid any noticeable congestion.
5. ERROR-RESILIENT STREAMING
To avoid interruptions in a streaming application, decod-
ing errors due to packet loss or to late arrivals are concealed
by freezing the previous frame. Because of the predictive
nature of compressed video, this decoding error propagates
to subsequent frames until an independently encoded frame
(e.g. an I frame) is received. In this section, we describe
how error propagation can be stopped adaptively between
peers by using SP and SI pictures.
5.1 SP and SI frames
SP and SI pictures types are part of the latest video coding
standard H.264 [12]. These new types of predictively/intra
coded pictures were initially proposed in 2001 by Karczewicz
and Kurceren, as a solution for error resilience, bitstream
switching and random access [15, 16]. The main advantage
of SP and SI pictures is that they can be used interchange-
ably in a video stream without creating any decoding er-
ror or drift. This leads to interesting applications, such as
stopping error propagation adaptively by refreseshing a pre-
Figure 5: SI frames share the instant refresh prop-
erties of I frames but are only sent after a frame is
lost.
500
Figure 6: GOP structures used for streaming with
SP and SI frames and for periodic I frame insertion.
diction chain, as depicted in Fig. 5. In such a scheme, a
pair of SP and SI pictures are periodically generated dur-
ing the encoding of a sequence. For streaming, the sender
may choose which picture of the pair to transmit. Because
of their larger size, SI frames should, ideally, only be sent
when their instant refresh properties are needed. Therefore,
SI frames are transmitted instead of SP frames only when
a decoding error at the receiver is detected, to stop error
propagation. This differs from traditional video streaming
where large I frames are transmitted periodically whether
decoding errors occur or not.
5.2 Compression efficiency
The video encoding structure shown at the top of Fig. 6
was chosen for the streaming experiments presented in this
paper. The number of frames in a Groups of Pictures (GOP)
is 16, with one SP frame (and its corresponding SI frame)
per GOP and 3 B frames between P frames. This ensures
good error resilience properties and allows to easily scale
down the frame rate by 2 or even 4 if needed. Links to
video sequences compressed with this coding structure at
different rates, as well as rate-distortion preambles charac-
terizing the size and quality of the frames are made publicly
available [2]1 . We also evaluated the performance of tradi-
tional streaming with periodic I frames. The structure of
the GOP used in this case is similar and is illustrated at the
bottom of Fig. 6. As the size of SP frames is less than that
of I frames, for limited loss rates, the bit rate savings ob-
tained by not transmitting unnecessary I frames may reach
up to 25%. This is illustrated in Fig. 7, which shows the
compression efficiency of GOPs containing SP or SI frames
compared to GOPs with periodic I frames. When loss rates
are high, the rate-distortion performance decreases and may
be worse than streaming with I frames, as the size of SI
frames exceeds that of I frames.
1
The preambles also indicate the distortion values obtained
by concealing a frame with any other frame of the stream,
allowing to simulate realistically video streaming without
the overhead of encoding and decoding.
44
 44
43
42
41
PSNR (dB)
40
39
38
periodic I frames
37
periodic SP frames
periodic SI frames
36
0 100 200 300 400 500 600 700
Rate (Kbps)
Figure 7: Rate-distortion performance with periodic
I frame, SP frame or SI frame insertion, for Mother
and Daughter.
5.3 Extension to the P2P case
In Sec. 5.1, we explained how a sender possessing a pair of
corresponding SP and SI pictures could achieve better rate-
distortion performance by choosing which one to transmit
adaptively. We propose to extend this technique to P2P
streaming where a sender has not necessarily received both
the SP and the SI pictures from its ancestors.
The complete details of the encoding/decoding process of
SP and SI frames are beyond the scope of this paper. They
can be found in [16]. One interesting aspect of the decoding
of SP frames is that it allows the corresponding SI picture
to be regenerated with very limited additional complexity2 .
As a consequence, in the P2P streaming scenario, each peer
which receives and decodes an SP frame correctly may also
create the corresponding SI frame. Therefore, it can subse-
quently use the resulting SP and SI picture pair adaptively
to either reduce the bit-rate or stop the error propagation
its descendant might be experiencing. This technique al-
lows the adaptive streaming to take place not only between
the source and its direct descendants but also further down
in the tree, as long as SP frames are received and decoded
correctly by peers.
As larger video frames usually span several transport layer
packets, different parts of SP and/or SI frames will usually
be transmitted by different parents, when different multi-
cast trees are maintained. As all the packets of one of the
pictures are needed to continue decoding, it is necessary for
these different peers to coordinate and choose to transmit
the same picture to their child. This is done by signalling
from the child to the parents the periods when error propa-
gation occurs. During these periods SI frames will be trans-
mitted.
2
This requires the use of an additional entropy encoder.
Note, however, that the reverse is not true: the SP frame
cannot be recovered from the SI frame without a complete
video encoder.
45
5.4 Experimental results
We analyze the benefits of SP and SI frames for video
streaming with low latency over the network described in
Sec. 3. The video stream is encoded at approximately 220
kbps and the maximum tolerated delay between the time
a frame is available at the source and the time it is played
by the peers is 2 seconds. As a comparison, results are also
collected for the same sequence of events for a video stream
with periodic I frames, also encoded at the same rate. For
the bandwidth distribution indicated in Tab. 1, the dynamic
behavior of the hosts and this tight delay constraint, the
two video streams represent the highest supportable video
quality for the two different encoding structures considered.
1
periodic I frames
SP/SI frames
0.9
0.8
Cumulative distribution function
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
38.5 39 39.5 40 40.5 41
PSNR (dB)
Figure 8: Video quality performance for the 300
peers for different encoding structures.
Figure 8 shows the cdf of the peers’ decoded video quality.
The average quality for the SP and SI frames is 40.3 dB, 0.8
dB higher than for streaming with periodic I frames. The
distributions show that 99% of the hosts benefit from this
novel encoding structure. Furthermore, for more than 25%
of the hosts the performance gain exceeds 1 dB. This gain is
due to the better rate-distortion-congestion performance of
the video stream containing SP/SI frames. As the number of
node disconnections is limited, most of the time, SP frames
are transmitted in lieu of SI frames. This reduces the overall
bit-rate and allows for a higher sustainable quality. More-
over, instantaneous bit-rate fluctuation are reduced when
SP frames are used instead of I frames. This results in less
congestion and better end-to-end delay performance. More
detailed experimental results describing similar effects were
reported in our prior work [19].
6. OPTIMIZED RETRANSMISSIONS
The dynamic behavior of peers is one of the most challeng-
ing aspects of peer-to-peer live streaming. In particular, a
peer must continuously monitor the state of its parent(s)
and immediately regain connectivity via another peer when
a leave is detected. As analyzed in Sec. 3, this process can
take a few seconds, a time during which none of the pack-
ets transmitted over the affected tree will reach the peer.
The corresponding loss statistics are obviously time-variant
as there might be some long periods during which a host
is fully connected and experiences no loss, and other times
when a large portion of packets are missing. Hence, FEC or
multiple description coding which seek to protect a user for
a specific loss level may not be appropriate as they will cre-
ate superfluous redundancy most of the time and might be
overwhelmed when losses occur. However, as reconnections
are not instantaneous, in order to maintain high decoded
quality, missing portions of the video need to be recovered
while the peer is rejoining. In this section, we describe how
a peer can mitigate the video quality degradation during the
rejoining process by requesting retransmissions to its other
parents.
Retransmission requests will place an additional burden
on the uplink of peers already forwarding a portion of the
video to 1 or possibly many children. This increase in con-
gestion is more important when there are few multicast trees
as a larger portion of the video will be requested from fewer
parents. Regardless of the number of multicast trees, op-
timal quality will be achieved when enough packets are re-
quested to maintain high video quality while avoiding a too
severe increase in terms of end-to-end delay. Our goal is
to determine an optimal retransmission schedule for miss-
ing packets of a video stream. This schedule would indicate
which packets of the stream will be requested to maximize
the decoded video quality at the receiver while limiting the
congestion created on the network.
6.1 Congestion-distortion optimized scheduler
In [18], we presented and analyzed the performance of a
sender-driven congestion-distortion-optimized scheduler (Co-
DiO) which determines how to minimize video distortion for
a given level of congestion. The benefits of using congestion
and distortion as metrics rather than rate and distortion,
as for example in [9], are two-fold. End-to-end delay (i.e.
congestion) is inherently adaptive to time-varying network
conditions. In addition, it reflects better the impact of a
user operating on a bandwidth-limited network. To find
an optimized schedule, CoDiO selects the most important
packets in terms of video distortion reduction, and requests
them in an order which minimizes the congestion created
on the network. For example, I or SI frames are requested
in priority whereas B frames might not be retransmitted at
all. In addition, CoDiO avoids requesting packets in large
bursts as this has the worse effect on the queuing delay.
In the following, we describe how to extend CoDiO to the
P2P receiver-driven scenario. We present a low-complexity
CoDiO scheduler which performs retransmission scheduling.
We show how to select the most important packets for re-
transmission, how to limit congestion and analyze the per-
formance of the CoDiO retransmission scheduler.
6.2 Computing the importance of packets
After detecting a parent disconnection, a peer can deter-
mine a list of missing packets and iteratively select the most
important ones to request. This choice should depend on the
time at which packets are due, and on the contribution of
each packet to the overall video quality. CoDiO proceeds by
discarding packets already past their due time and by com-
puting an approximate sensitivity of the video distortion to
the remaining packets.
Given a set of received frames an approximate video dis-
tortion may be computed in the following manner. Assum-
ing copy error concealment is used, where an undecodable
46
frame is replaced with the nearest correctly decoded frame,
and given the encoding structure of the video, we can find
which frames will be shown as a function of time. Let
D(s, c(s)) denote the approximate distortion resulting from
showing frame c(s) instead of frame s. We assume the ta-
ble D, pre-computed offline for a generic video sequence, is
available at each peer. Each display outcome is associated
with the appropriate pre-computed distortion value and the
resulting approximate video quality is computed over several
frames:


D= D(s, c(s)) (6)
s
It is then straightforward to determine, the sensitivity of
the average distortion to a single frame, and this sensitivity
is extended to each of the packets composing the frame.
Retransmission requests are sent out following the order of
importance.
The difficulty resides in determining which packets will or
will not be received before their playout deadline. Ideally,
probabilities for these events should be computed, based
on delay distributions, and the resulting expected distor-
tion could be computed, as described in [14], by combin-
ing these different probabilities. In our scheduler, we use a
much simpler techniques and only distinguish lost packets
and received packets. We consider all the parents of a peer
will forward successfully the packets that are transmitted
on their respective multicast tree. The packets sent on the
tree(s) of the disconnected parent(s) are considered missing
except if the peer has requested them from another parent
within a short time interval (we arbitrarily chose 200ms in
our experiments). This classification reduces the number of
possible decoding states to one, thus we know which frames
will be displayed over the horizon considered.
6.3 Limiting congestion
In the scenario considered, the available transmission rate
between a sending peer and a receiving peer depends on sev-
eral factors, such as the uplink throughput of the sending
peer, the number of other hosts served by this peer and the
rate of the video stream. Given these parameters, the avail-
able throughput Ci may be computed and the average end-
to-end delay over a certain time horizon can be estimated
for a given retransmission schedule. However, precise esti-
mation would require modelling the delay distribution of the
path between sender and receiver.
As a low-complexity alternative, we limit the number of
unacknowledged retransmission requests from a peer to each
of its parents. As an unacknowledged retransmission request
represents a packet being transmitted or processed between
the two peers, these packets contribute to end-to-end delay
hence to congestion. The tradeoff between the rate of the
retransmissions and the amount of congestion created on the
bottleneck links can be set by determining the optimal num-
ber of unacknowledged packets tolerated between a sender
and a receiver This optimization is carried out experimen-
tally in the following.
6.4 Experimental results
6.4.1 Influence on video quality
We first analyze the influence of retransmissions on the
video quality. Specifically, we would like to know to what ex-
 40 100%

Connected hosts
PSNR(dB)
38 90%

36 80%

34 70%
18 19 20 21 22 23 24 25 26
Time (second)
40
100%

Connected hosts
PSNR(dB)

38

90%
36

34 80%
18 19 20 21 22 23 24 25 26
Time (second)

Figure 9: Video quality (solid line) and percentage of fully connected peers (dashed line) as a function of
time for 299 peers when a host close to the source leaves. Results are shown in the absence of retransmission
(top) and when the maximum number of unacknowledged retransmissions on each tree is 4.

tent retransmissions mitigate the quality degradation which

occurs when the parent of a peer leaves. To maximize the
effect of the disconnection, in the network scenario specified
in Sec. 3, we let the 300 peers join and disable disconnec-
tions. When a steady state is reached, we disconnect a host
close to the source in one of the trees. In this experiment, 4
multicast trees are used to transmit the video and the max-
imum number of unacknowledged retransmissions requests
on each tree is 2. The performance is shown in Fig. 9, as a
function of time, in terms of the video quality. The average
video quality is taken over all 299 peers. Performance is also
shown when no retransmissions are allowed. The host dis-
connection occurs at time 18 s, and takes about 1.5 s to be
detected. This can be deduced from the fact that connectiv-
ity is reported directly by the peers. As illustrated in Fig. 9,
about 30% of the hosts are affected by the disconnection.
At time 21.5 s, all the affected peers have recovered. Simi-
lar behavior is observed with and without retransmissions.
However, the video quality during the rejoin time is very
different in both cases. When the CoDiO retransmission
scheduler is used, the video quality remains almost constant
over time, a large majority of the missing frames are recov-
ered. In the other case, quality drops by approximately 2-3
dB during the reconnection time.
6.4.2 Influence on congestion
We analyze the impact of retransmissions on congestion
and show how to determine the optimal tradeoff between
congestion and the number of retransmission requests.
Tab. (2) indicates the average decoded video quality for
300 peers operating following the scenario described in Sec. 3.
Video is transmitted over four multicast trees. The numbers
reported in the figure show the gain on the total average
quality which can be achieved through the use of retransmis-
Table 2: Average decoded video quality for different
numbers of retransmissions.
Maximum unacknowledged
retransmissions per tree PSNR
0 37.0 dB
1 37.5 dB
2 37.6 dB
3 37.3 dB
8 36.3 dB
sions for this scenario. As illustrated, two simultaneous re-
transmissions requests per tree allow to gain approximately
0.6 dB. Although this gain is modest, we stress that the
impact of visual quality discussed in the previous section
might be much larger. The small performance gap is due
to the fact that during the 15 minutes event, the number of
disconnections affecting each peer might not be very large.
When more than two simultaneous retransmissions are al-
lowed, the performance degrades slightly for 3 retransmis-
sions and more severely for 8 retransmissions. This indicates
congestion is disrupting the video transmissions. Please note
that the optimal number of simultaneous retransmissions in-
creases with the number of multicast trees.
7. CONCLUSIONS
Live P2P multicast streaming is constrained by the dy-
namic behavior of hosts and by limited uplink through-
put. Dynamic control protocols are needed to support het-
erogeneous peers and react rapidly to node disconnections.

47
We present a new multicast control protocol which builds
and maintains multiple trees to transmit live video to a
large population of peers, and demonstrate how video encod-
ing, streaming and scheduling techniques developed recently
further enhance the performance of the system. A rate-
distortion model is proposed to analyze the tradeoff between
self-inflicted congestion and video quality and is used to de-
termine the amount of over-provisioning necessary when low
latency is required. H.264 SP and SI frames are incorporated
into the video stream to provide adaptive error-resiliency ca-
pability and achieve bit-rate saving gains of up to 25% com-
pared to traditional video streams. Last, retransmissions
of missing packets are requested in a congestion-distortion-
optimized fashion which selects the most important packets
in terms of video quality while limiting the effect on end-
to-end delay. For simulations with up to 300 peers, achieve
typical end-to-end delays of 1 sec, and a stable video quality
with less than 2.5% of frames lost to playout interruptions.
8. REFERENCES
[1] Coolstreaming. http://www.coolstreaming.org, seen on
Aug. 28 2005.
[2] Encoded sequences with SP/SI frames.
http://ivms.stanford.edu/˜ esetton/sequences.htm seen
on Aug. 28 2005.
[3] ESM. http://esm.cs.cmu.edu/, seen on Aug. 28 2005.
[4] H.264/AVC Reference Software.
http://iphome.hhi.de/suehring/tml/download/, seen
on Aug. 28 2005.
[5] PPLive. http://www.PPLive.com, seen on Aug. 28
2005.
[6] The Network Simulator - ns-2.
www.isi.edu/nsnam/ns/, seen on Aug. 28 2005.
[7] J. Apostolopoulos, T. Wong, W. Tan, and S. Wee. On
multiple description streaming with content delivery
networks. Proceedings Infocom, New York, USA,
3:1736–1745, June 2002.
[8] S. Banerjee, B. Bhattacharjee, and C. Kommareddy.
Scalable application layer multicast. Proceedings ACM
Sigcomm, Pittsburgh, USA, pages 205–217, Aug. 2002.
[9] P. Chou and Z. Miao. Rate-distortion optimized
streaming of packetized media. Microsoft Research
Technical Report MSR-TR-2001-35, Feb. 2001.
[10] Y. Chu, A. Ganjam, T. Ng, S. Rao,
K. Sripanidkulchai, J. Zhan, and H. Zhang. Early
experience with an internet broadcast system based on
overlay multicast. Proceedings of USENIX’04, page
155170, June 2004.
[11] Y. Chu, S. Gao, and H. Zhang. A case for end system
multicast. Proceedings ACM Sigmetrics, Santa Clara,
USA, June 2000.
[12] ITU-T and ISO/IEC JTC 1.
48
Advanced Video Coding for Generic Audiovisual
services, ITU-T Recommendation H.264 - ISO/IEC
14496-10(AVC), 2003.
[13] J. Jannotti, D. Gifford, K. Johnson, M. Kaashoek,
and J. O. Jr. Overcast: Reliable multicasting with an
overlay network. USENIX Symposium on Operation
Systems Design and Implementation, San Diego, USA,
Oct. 2000.
[14] M. Kalman, P. Ramanathan, and B. Girod.
Rate-distortion optimized streaming with multiple
deadlines. Proc. International Conference on Image
Processing, Barcelona, Spain, Sept. 2003.
[15] M. Karczewicz and R. Kurceren. A Proposal for
SP-Frames. Video Coding Experts Group Meeting, ,
Doc. VCEG-L-27, Eibsee, Germany, Jan. 2001.
[16] M. Karczewicz and R. Kurceren. The SP- and
SI-frames design for H.264/AVC. IEEE Trans. CSVT,
13(7):637–644, July 2003.
[17] V. N. Padmanabhan, H. J. Wang, and P. A. Chou.
Resilient peer-to-peer streaming. IEEE International
Conference on Network Protocols, Atlanta, USA, Nov.
2003.
[18] E. Setton and B. Girod. Congestion-Distortion
Optimized Scheduling of Video. Multimedia Signal
Processing Workshop (MMSP), Siena, Italy, pages
99–102, Oct. 2004.
[19] E. Setton and B. Girod. Video streaming with SP and
SI frames. Proc. Visual Communications and Image
Processing, Beijing, China, July 2005.
[20] E. Setton, X. Zhu, and B. Girod. Minimizing
distortion for multipath video streaming over ad hoc
networks. International Conference on Image
Processing, Singapore, pages 1751–1754, Oct. 2004.
[21] K. Sripanidkulchai, A. Ganjam, B. Maggs, and
H. Zhang. The feasibility of supporting largescale live
streaming applications with dynamic application
endpoints. Proceedings SIGCOMM’04, Portland, USA,
Aug. 2004.
[22] D. Tran, K. Hua, and T. Do. Zigzag: An efficient
peer-to-peer scheme for media streaming. Proceedings
Infocom, San Francisco, USA, 2:1283–1292, Mar.
2003.
[23] X. Zhang, J. Liu, B. Li, and T.-S. P. Yum.
Donet/coolstreaming: A data-driven overlay network
for live media streaming. Proceedings IEEE Infocom,
Miami, USA, Feb. 2005.