You are on page 1of 6

ISPCS 2009 International IEEE Symposium on Precision Clock

Synchronization for Measurement, Control and Communication


Brescia, Italy, October 12-16, 2009

Packet Delay Variation Management


for a better IEEE1588V2 performance

Dinh Thai BUI, Arnaud DUPAS, Michel LE PALLEC


Semantic and Autonomic Technologies (SAT) Department
Alcatel-Lucent Bell Labs France (ALBLF)
Nozay, FRANCE
Dinh_Thai.Bui@alcatel-lucent.com, Arnaud.Dupas@alcatel-lucent.com, Michel.Le_Pallec@alcatel-lucent.com

Abstract— Designed for the distribution of time, IEEE 1588V2 the delay asymmetry is mainly driven by PDVs generated by
also called as Precision Time Protocol V2 (PTPV2) is often communication paths. While a deterministic value of the delay
referred to as a key candidate for synchronization over Packet asymmetry can still allow for a correct computation of the time
Switched Networks (PSNs). Unfortunately, as the IETF's offset, the random component, essentially induced by PDVs,
Network Time Protocol and unlike the ITU-T's Synchronous leads to the time offset uncertainty and error.
Ethernet, the performance of this protocol strongly depends on
Packet Delay Variations (PDVs) produced by the telecom Thus, the overall PTPV2 performance greatly depends on
network environment. Such behavior represents a strong issue, PDVs generated by the PSN in each communication direction.
particularly considering the severe synchronization requirements
of the mobile backhaul segment. This article focuses on different
After a brief description of different PDV components, we
manners to control and to fight out PDVs. After discussing PDV- illustrate some principles which enable the management of
management principles, via the illustration of some existing PDVs. Such methods are presented following the hierarchical
mechanisms, we propose a new distributed ranging scheme which synchronization architecture: emission profiles are firstly
allows for optimizing the overall PTPV2 synchronization discussed followed by a presentation of network-based
performance. Finally, considered as a critical mobile backhaul approaches. Then, Slave clock mechanisms are depicted.
use case, IEEE 1588V2 over Digital Subscriber Line (DSL) is
Following this “prior art” description, a new distributed
investigated regarding PDV concerns.
ranging mechanism combining the functionalities of the three
Keywords: synchronization protocols, 1588V2, mobile aforementioned approaches (Master/Network/Slave-based) is
backhaul, delay, delay variation, DSL presented.
Finally, IEEE1588V2-over-DSL is investigated as an
I. INTRODUCTION illustration of a difficult PDV use case.
IEEE 1588V2 offers the possibility to distribute both
frequency and time. Even intimately intertwined, these two II. PDV COMPONENT DESCRIPTION
functionalities are studied separately at the ITU-T Study Group Delay and thus related delay variation can be divided into
15 Question 13 which aims at defining IEEE1588V2 telecom three different components [2] [3].
profiles.
- The transmission delay is the result of the velocity of the
Taking time distribution as an example, PTPV2 can be signal between two endpoints of a transmission link and the
deployed either in a one-way or in a two-way signaling distance between these two endpoints. The transmission delay
approach. The former approach implies an “on-path” correction variation is due to many factors depending on the transmission
(see Transparent Clock description in section IV.A) of the time technology employed (e.g. wireless, wire-line, etc). For
offset induced by the path delay. Alternatively - and this why it example on a copper link, temperature variations can modify
is often preferred - the two way approach relies on an offset the transmission delay by shrinking or extending the
delay correction based on a mean-delay knowledge (half of the transmission distance.
round-trip delay). This approach is thus conceptually
dependent on the delay asymmetry between the Master-to- - The processing delay is the delay resulting from the
Slave and Slave-to-Master synchronization signalization processing of a timing packet within a Network Element (NE).
directions. It strongly depends on the NE hardware and software. For
example, the processing delay variation can result from a
This delay asymmetry has two components, a fixed and a specific CPU (Central Processing Unit) configuration: a same
variable one. The fixed and deterministic component is mainly CPU on a given port could be in charge of in-going and out-
driven by the link delay. This component can generally be going packets. Consequently the traffic load in one direction
determined by simple computation means (e.g. optical can have a detrimental effect on the other direction.
transmission index for Gigabit Passive Optical Network) or by
using advanced methodologies [1]. The variable character of

978-1-4244-4392-5/09/$25.00 ©2009 IEEE


- The buffering or queuing delay is the total amount of The counterpart of this approach is its impact on the
waiting time of the timing packet within different buffers or bandwidth consumption, which is especially critical for low
queues in a given NE before being processed and finally bit-rate communication paths. It also implies an additional
transmitted to the next NE on the communication path. CPU power consumption at the Master level. Moreover, packet
Buffering delay variation is partially due to the “competition” correlation has to be taken into account for high timing
(see chapter IV.B) between timing packet flows and other message rate [5].
packet flows, and between different timing packet flows
themselves according to their relative arrival time at different 2) Randomization:
waiting queues and the priority policies implemented. As it comes to syntonization, one often makes use of a
periodic PTPV2 stream (i.e. one-way periodic Sync messages).
These three components have different PDV ranges. In this case, it is demonstrated in [8] that PTPV2 streams with
Usually, the first component generates a PDV in the range of similar transmission frequencies compete against each other
sub-microseconds (especially considering temperature leading to additional PDVs. This phenomenon is often referred
dependence) and the second one in the [1-10] micro-second to as a “beating effect”. In order to solve such an issue, a
range. The last component is often referenced as the main PDV simple but powerful method consists in randomizing the
generator with a [10-10000] micro-second range. transmission instant of PTPV2 messages around a given mean
value.
III. EMISSION PROFILES However, this technique breaks out the periodicity of the
This approach consists in defining the “profile” to be PTPV2 stream and could lead to some incompatibility with
implemented at the 1588V2 Master level. Reference [6] gives a certain Slave clocks. Indeed, these latter could make use of the
definition of what is a profile: “The set of allowed Precision periodicity of the PTPV2 stream in their filtering/ranging
Time Protocol features applicable to a device”. Here we process as described in chapter V.B.
particularly focus on the message emission profile which is the
first tool for managing the PDV as described in the following IV. NETWORK-BASED APPROACHES
examples.
1) Timing packet rate A. Specified IEEE 1588 V2 mechanisms
Timing message rate is one of the main parameters The IEEE has specified hardware components to correct
enabling to dampen PDV effects. Indeed, a higher number of PDVs [6].
samples (timing packets) per time unit allow for improving the Boundary Clocks (BCs) allows for segmenting a large
time or frequency accuracy at the Slave level. Figure 1 synchronization network into small areas (within a PTP
(Frequency accuracy vs PDV at different packet rates), based domain) where PDVs can be engineered and/or controlled
on [4] equations in a first order approach, illustrates this within specified bounds. BCs serve as means to recover the
purpose taking a common 5 mHz Phase Locked Loop (PLL) reference time (or frequency) as accurately as possible before
filter. One can observe, for instant, a factor 10 gain in term of distributing the latter into next (per-PDV) areas of the
frequency accuracy with an emission rate increase from 2 synchronization hierarchy.
messages/s to 128 messages/s (note that IEEE 1588V2
specifies a maximum message rate of 128 messages/s). End-to-End Transparent Clocks (E2E TC) allows for
Targeting an often-mentioned 15 ppb frequency accuracy at the correcting transit delays of the network elements whereas Peer-
network level for wireless requirements (e.g. WCDMA, LTE), to-Peer Transparent Clocks (P2P TC) allow for compensating
the [10-100] µs PDV range appears to be the recommended both transit delays and link delays. The first one allows to
limits. record, within the message “correction field”, its (cumulative)
resident time within each NE and allows for the Slave clock to
compensate PDVs produced by queuing and processing
1E-05 processes. The second one measures not only the NE resident
PLL cutoff frequency = 5 mHz time but also link delays. Both information allows the Slave to
1E-06 completely correct uncertainties due to PDV and thus to reach
Frequency accuracy

accurate time/frequency references at the Slave level.


1E-07 It is noted that there are no syntonization requirement on
TCs. Indeed, considering transit and link delays in the order of
1E-08
15 ppb wireless requirement milliseconds, it clearly appears that low cost local clocks with
some ppm (part per million) accuracy are sufficient to meet
2 packets/s
1E-09 16 packets/s measurement requirements (measurement error in the order of
128 packets/s nanoseconds). Moreover in the case of link delay measurement,
1024 packets/s it also appears that P2P TCs cannot solve by themselves the
1E-10
1 10 100 1000 10000 link delay asymmetry and that specific mechanisms are
Packet Delay Variation (µs) necessary for such a purpose (for instance a delay asymmetry
ratio could be provisioned via the synchronization management
Figure 1 - Frequency accuracy according to the timing packet rate for a plane).
5 mHz PLL (Phase Locked Loop) filtering
Whilst TCs represent a very effective solution to fight out However, in this specific scenario (retiming at the bit level),
PDVs, they represent some non negligible challenges for delay equalization requires the distribution of a precise clock
network operators considering a full deployment. The first one across all the NE ports. This pre-requisite supposes a high
is the cost related to the implementation of such devices within accurate, thus high cost, local clock especially if the latter is
the NEs along all PTPV2 signalization paths. The second left into a free-run or holdover mode.
important one is rather a technical challenge linked to the fact
that TCs represent an infringement to the layer separation V. SLAVE CLOCK MECHANISM
principle. Indeed, for precision purpose, TCs require timing
monitoring operations (e.g. hardware time stamping) being A. Packet Clock Models
realized at the physical layer while reporting the monitoring
results (e.g. packet resident time) at the PTPV2 packet layer. Timing packet clock generally has two-stage architecture as
depicted by the Figure 2 below:
Nevertheless, those constraints have to be re-visited in the
particular scenario of “green field” deployments targeting the
distribution of both accurate time and frequency references.

B. Quality Of Service (QoS) approach


The principle here consists in differentiating timing
signalization flows from other packet flows using QoS-build-
in information within the packet header. This allows for
prioritizing synchronization packets with regards to data
packets, and avoiding a direct competition of the latters Figure 2 – Packet Clock Model
against the formers. QoS technique usually relies on parallel
buffers which filling depends on sheduling policies. The first stage is a packet filtering stage designed for
This technique allows to substantially reduce the PDV selecting the most relevant packets while reducing PDV
induced by queuing effect for a given packet flow, especially effects. The packet filtering or selection process is also called
for high traffic load conditions, but without suppressing this as “ranging” process.
component completely [7].
The second stage aims at disciplining a local oscillator or
Indeed, the method does not fully prevent low priority data wall clock according to the filtered “packet signal” resulting
packets from competing with high priority timing packets. from the first stage.
Even with separated queuing buffers, QoS technique cannot
avoid competition at the physical port when it comes to the
serialization of packets onto the line. For instance, the timing B. Ranging Mechanisms
packet waiting times, induced by already departing Best Effort When syntonization is considered, the PTPV2 Slave deals
packets (from another queue) onto the transmission line with periodic one-way PTPV2 Sync messages. The ranging
(physical layer level), can greatly vary depending on the data process in this case often consists in assessing the inter-arrival
packet length and its relative arrival time at the physical output times between successive Sync messages by averaging over a
medium. This phenomenon is often referred to as the (data) large number of samples. Two methods are considered here
“jumbo packet” phenomenon. making the assumption that delays are independent and
identically distributed: the Moving Average (MA) and the
Another phenomenon that QoS technique cannot solve is Exponentially Weighted Moving Average (EWMA) methods
the competition between timing packet flows with a same QoS which respectively computes means or weighted means on a
priority. In case of periodic timing flows having the similar limited set of stored inter-arrival times. Although conceptually
transmission frequencies, such a competition produces the simple with basic components in its implementation, the MA
“beating effect” [8]. method has the disadvantage of requiring a large number of
registers to achieve low jitter levels in the recovered clock
C. Delay Equalization signal (impacting the convergence time). The EWMA partially
Delay equalization is a well-known technique especially solves this issue at the expense of being very sensitive to
implemented within ATM switches for the interconnection of packet delay correlation. Indeed, the aforementioned
different SDH/SONET transmission lines. The idea consists in assumptions on delay distribution are not realistic as this latter
is observed to be more or less correlated with time. And it has
adding a time label in order to track the node delay
been demonstrated that EWMA method becomes rapidly
experienced by ATM cells. At the output port, this makes it inefficient as the correlation between successive packets
possible to “equalize”, via a buffer, node delays originating increases [11][12]. It is also noted that both MA and EWMA
from the dismantling of the same (SDH/SONET) input performance is predicted while assuming a time-invariant
payload. Delay equalization is quite powerful to fight out (stationary) packet delay distribution which is not always the
PDVs related to both processing and queuing, as different case in real network.
packets pertaining to the same flow undergo the same node Recent works are thus more focused on ranging
delay. mechanisms which selects packets experiencing minimum
(alternatively maximum) packet delays [13][14] within an (often based on packets experiencing minimum delays). If the
observed window in order to dampen PDV effects. New related number of these selected packets is lower than a certain
metrics, such as MinTDEV [15], etc, have been defined for this provisioned threshold (e.g. tm1 for filter 1), the filter module
purpose. A performance comparison between these requests the Master for increasing the message emission
mechanisms has been provided through [16], but no decision frequency. In the opposite way, a second threshold can be
has been taken within standardization organization in terms of provisioned so that the filter module can ask the Master for
recommendation. decreasing the message emission frequency when the number
of selected packets is found to be above this second threshold.
It is noted that the ranging process can give different results This method guarantees that the Slave receives sufficient
regarding a time or a frequency distribution purpose. Indeed if relevant packets during its filtering process in order to feed its
‘p’ is the probability to select a packet with a minimum delay second stage and to discipline its local oscillator or wall clock.
for a single transmission direction (such as the one-way
approach frequency distribution) the probability to select One possible way to change the Master message emission
packets experiencing minimum delays in the two transmission frequency is to use a management message with a
directions (such as the two-way signaling approach) is ‘p²’ [9]. PORT_DATA-SET management Type Length Value (TLV)
This implies strong constraints on the local oscillator data field (this latter includes other TLVs defining different
supporting the ranging process (during the ranging time all message emission frequency values, such as logSyncInterval).
packets have to be selected with the same frequency reference). Alternatively, filter modules can make use of the
In such a case the timing packet rate has to be carefully REQUEST_UNICAST_TRANSMISSION TLV to modify the
configured. Master message emission frequency when unicast profile is
deployed.
VI. NEW DISTRIBUTED RANGING MECHANISM In order to optimize the consumed bandwidth by
downstream NEs, non-selected synchronization messages
A. Principle description having their PDV range over a pre-defined bound can be
This new approach relies on a distributed selection/filtering dropped and thus not forwarded towards the Slave.
of relevant (low delay) timing packets along the
Facing heavy traffic load conditions, the presented
synchronization path thanks to a given synchronization
approach enables to tune the individual timing packet filtering
requirement at the receiver level. The idea particularly enables
ratio in order to ensure the best synchronization performance in
to increase the timing message rate with a limited impact on the
a degraded state, for which the synchronization requirement at
consumed bandwidth. In a simple configuration, each filter
the Slave level is not reachable. This “less bad state” approach
(including the receiver) has the ability to trigger an increase of
could be relevant is some congestion scenarios for which the
the timing source message rate according to a provisionable
synchronization topology reconfiguration time plays a
timing message rate threshold. Figure 3 below depicts the main
significant role.
principles of this approach.
From an implementation perspective, the timing packet
filtering functionality could be integrated either into a Network
Element, or into an external module placed onto the path
between two network nodes.
It is to be noted that these distributed filters can take
advantage of an external syntonization for a precise monitoring
of the selection process but similarly to the TC functionality,
this does not appear critical (some ppm frequency accuracy
should be sufficient to cover stringent synchronization
requirements)

B. Discussions on the mechanism benefits


In order to illustrate the benefits of the distributed ranging
mechanism, two representative PDV examples are taken (PDV
here is defined with regards to the minimum delay) with related
Probability Density Functions (PDF) as depicted in Figure 4.
Figure 3 – Ranging Mechanism Distributed along the Synchronization Path These PDF examples, reflecting “high” and “low” traffic load
conditions as defined by the ITU-T G.8261 recommendation
[8], arbitrary follow a Gamma density function already
In such a representation, “transparent nodes” are discussed within the ITU-T [10]. For low and high traffic
intermediate nodes which do not implement any PTPV2 loads, the computed standard deviations are respectively 7.1 µs
features. Filtering modules have the ability to recognize and 20 µs ( α .β 2 with regards to Figure 4 examples).
PTPV2 packets and to proceed to the ranging process.
For each aforementioned filter module, a ranging process is An arbitrarily configured PDV threshold filtering of 10µs
applied in order to select packets demonstrating low PDV per filter module is considered. This scenario particularly
means that, without any filter module, timing packet VII. DSL USE CASE
demonstrating a PDV over this 10 µs reference value have a Attractive DSL technologies are used to backhaul data and
very poor probability to be retained by the first filtering stage signalization of the wireless networks. For such a purpose, they
of the end receiver packet clock. The following discussions have to face some syntonization requirements, such as the
take the first filtering module as an example but one can distribution of a frequency reference within accuracy better
reiterate the arguments through successive filtering modules than 50ppb (base station syntonization requirement). Meeting
downstream. this requirement, SH-DSL has been the first main deployed
DSL technology. Network Time Reference (NTR) has been
specified for accurately distributing a frequency reference at
Gamma Law: the physical layer.
1E+0 Unfortunately, even if it is specified, NTR has not really
Filtered area been deployed in commercial DSLAMs. Moreover, in some
Probability Density Function

1E-1
High traffic load
configurations, such as pseudo-wire based backhaul (Figure 5),
60% Low traffic load
the DSLAM could be considered as transparent node in terms
1E-2 of syntonization and NTR can not be used consequently.

1E-3
98% α =4, β =10
40%
1E-4
2% α =2, β =5
1E-5
0 20 40 60 80 100
10 µs filtering Packet Delay Variation (µs)

Figure 5 - Pseudo-wired based Mobile backhaul scenario


Figure 4: examples of PDV Probability Density Functions for high and low
traffic load conditions
To face this issue (and for lowering the syntonization cost),
Within this configuration, a low traffic load implies a low equipment vendor try to monitor syntonization at the packet
PDV range and a better probability (e.g. 60%) to have an layer by taking advantage of timing protocols such as PTPV2.
arriving timing message selected by the first filter module. In
this case, the number of messages selected should be sufficient Unfortunately, in DSL environments, the “syntonization
(above the provisioned threshold) and thus no feedback noise” in the downstream (Master-to-Slave) direction implies
mechanism towards the sender (e.g. master clock) is needed in very hard constraints on the Slave features. Indeed, as referred
order to increase the transmission rate. In this scenario, one can to by Symmetricom [4], the observed PDV is very high -
observe a 40% timing-message bandwidth saving at the usually in hundreds of microsecond range even under no load
downstream link next to the first filter module. conditions (see [17]).
At high traffic load condition, the probability to have a
timing message selected by the first module is strongly Mean Delay Mean Delay Stddev PDV Stddev PDV
decreased (e.g. down to 2%). Thus the retained messages are Fwd Rev Fwd Rev
not to be of sufficient number with regards to the module
provisioned threshold. As described in the previous chapter, Switch 21.9 µs 21.3 µs 100 ns 100 ns
one possible policy in this case consists in loosing
synchronization constraints (e.g. degraded state) in order to VDSL 1.76 ms 1.54 ms 104 µs 166 µs
avoid traffic load increase. Alternatively it can be decided to SHDSL 3.60 ms 3.55 ms 89 µs 109 µs
trigger the feedback mechanism for an increase of the timing
message transmission rate. This second process has to be ADSL2+ 15.67 ms 14.65 ms 289 µs 196 µs
carefully calibrated according to the congestion state margin.
Indeed such a mechanism can significantly deteriorate the node
state with a final performance improvement lower than TABLE I. DELAY AND PDV MEASUREMENTS OF SYMMETRICOM FOR XDSL
expected; linked for instance to packet loss. Congestion control TECHNOLOGIES IN NO LOAD CONDITIONS [17]
and/or rerouting are consequently other techniques to be
considered for solving such an issue. Nevertheless, they have
to be well engineered by network operators with regards to:
- the priority allocated to the synchronization traffic;
- the policies which drive topology changes (avoiding
congestion points), as they usually involve signal transients
impacting the synchronization performance.
Moreover, due to its periodic time-slotted structure (i.e. 250 ACKNOWLEDGMENT
µs symbol period), a DSL link can lead to a “beating effect” Authors would like to thank Jean Loup Ferrant and Nicolas
with a periodic PTPV2 emission profile as related in [8]. The Le Sauze for their valuable comments and constant support.
“beating effect” imposes a “sawtooth” delay profile onto the
PTPV2 stream as illustrated in Figure 6 below, under no load
condition. REFERENCES
[1] An Enhanced IEEE 1588 Time Synchronization Algorithm for
Asymmetric Communication Link using Block Burst Transmission
[2] IETF RFC5481 - Packet Delay Variation Applicability Statement –
March 2009.
[3] ITU-T SG15/Q13 WD35 - Effect of routers’architecture on timing
distribution - San Jose, March 2009.
[4] K. Shenoi, Symmetricom - Sync for Support of Legacy Services – ITSF
London, October 2005.
[5] Q. Li, D.L. Mills – The Implication of Short-Range Dependency on
Delay Variation Measurement. IEEE Network Computing and
Applications. 2003.
[6] IEEE Standard for a Precision Clock Synchronization Protocol for
Networked Measurement and Control Systems – 24 July 2008.
[7] ITU-T SG15/Q13 WD38 - Additional PDV measurements on Ethernet
Figure 6 – Systematic delay variation caused by DSL timeslots [8] platform, Edinburgh, June 2009
[8] ITU-T G.8261/Y.1361 - Timing and synchronization aspects in packet
networks – April 2008.
The “beating effect” usually implies a strong time [9] Y. Stein et al, Rad Data Communications - Delivering Better Time-of-
dependency which puts more constraints on the Slave clock Day Using Synchronous Ethernet and 1588 – ITSF London, November
(e.g. stability). On the whole, and as discussed in III.1), such 2007.
PDV profiles imply high cost low-pass filters with cut-off [10] ITU-T SG15/Q13 C327 - Modelling and simulation of long-tailed packet
delay variation (PDV) over a multi-switch network, May 2007.
frequencies within the milliHertz and even sub-milliHertz
range or require to go beyond the specified 128-messages-per- [11] J. Aweya, D.Y. Montuno, M. Ouellette, K. Felske – Clock recovery
based on packet inter-arrival time averaging. Computer Communications
second limit for the PTPV2 packet rate. Ranging algorithms 29. 2006.
discussed in V.B should allow for relaxing clock constraints. [12] J. Aweya, D.Y. Montuno, M. Ouellette, K. Felske – Analysis of a clock-
But if one is targeting a low-cost and robust PTPV2-based recovery technique for circuit emulation services over packet networks.
frequency and time distribution, an important work at the DSL International Journal of Communication Systems. 2008.
physical layer seems to be the only way out. [13] S.B. Moon, P. Skelly, D. Towsley – Estimation and Removal of Clock
Skew from Network Delay Measurements. Proc. IEEE Infocom, March
1999.
VIII. CONCLUSION [14] L. Zhang, Z. Liu, C.H. Xia – Clock Synchronization Algorithms for
There are numerous manners to manage PDVs within a Network Measurements, Proc IEEE Infocom, June 2002.
telecom network. A limited set of powerful approaches have [15] ITU-T SG15/Q13 C363 - Definition of Minimum TDEV (minTDEV) –
been presented with their strengths and weaknesses. Stating Geneva, June 2007.
that all improvements performed at the Slave level to manage [16] ITU-T SG15/Q13 WD36 - Packet selection methods for performance
PDV could not meet all deployment scenarios and application metrics- San Jose, March 2009.
requirements, network-based solutions have to be considered [17] ITU-T SG15/Q13 WD45 - xDSL Access PDV Study - San Jose, March
2009.
with careful studies on deployment cost. In this context, a new
distributed mechanism is proposed. This new mechanism
particularly enables to increase PTPV2 message rate - and thus
to improve PTPV2 performance for a given PDV profile - with
a limited impact on the available network bandwidth.
As an illustration to the whole PDV monitoring discussion,
IEEE1588V2 over DSL has been investigated. A simple
analysis of this difficult use case demonstrates the need to
define a set of compatible and optimal procedures for an
efficient use of PTPV2. For such, monitoring the PDV at the
Master side by an appropriate emission profile appears to be an
attractive approach. This latter will likely require to adapt
Slave algorithms in order to manage packet correlation aspects.
Finally, without any obvious packet-based solution, the
need to directly manage the DSL physical layer for a lower
PDV is particularly growing. This work is to be addressed by
the ITU-T.

You might also like