You are on page 1of 5

The Design and Implementation of IEEE

1588v2 Clock Synchronization System by


Generating Hardware Timestamps in MAC
Layer
Meng Dong1,2, Zhiliang Qiu1, Weitao Pan1*, Can Chen1, Junxiang Zhang2, Dong Zhang2
1 State Key Laboratory of the Integrated Services Networks, Xidian University, Xi’an 710071, China
2 Science and Technology on Information Transmission and Dissemination in Communication Networks Laboratory,
The 54th Research Institute of China Electronics Technology Group Corporation, Shijiazhuang 050081, China
*
Email:wtpan@mail.xidian.edu.cn
Abstract—IEEE 1588 is a standard for a precision clock process of sending and receiving PTP messages. PTP is based
synchronization protocol for networked measurement and control on a Master-Slave hierarchy and message exchanging. The
systems. IEEE 1588 protocol can provide sub-microsecond Master imposes the reference clock and sends messages
accuracy. However, most synchronous systems generate containing the timestamps to the Slaves, which synchronize
timestamps using software-generated methods or proprietary with it [5]. Therefore, it can provide sub-microsecond level or
chips. Thus, the accuracy and portability of synchronous systems
even nanosecond time precision [6] without additional hardware
cannot be balanced. Based on the analysis of IEEE 1588 standard
and current Ethernet time synchronization scheme, this study interface with a high cost.
presented a scheme using FPGA and ARM. In order to improve At present, most synchronization systems that implement
the universality and portability of synchronous system, this IEEE 1588 protocol generate hardware timestamps by specific
research adopted the method of generating hardware timestamps PHY chips. For example, in [7], DP38640 was used to generate
in Media Access Control(MAC) layer. Verified by experiment in timestamps at media independent interface (MII). But this
Gigabit Ethernet environments, this design is correct and the method requires a dedicated chip, which is not conductive to the
marking accuracy is better than 20ns. The portability of upgrade of existing network. Based on detailed analysis of the
synchronous system is improved on the premise of ensuring the
IEEE 1588 protocol and the existing synchronization system,
accuracy of synchronization.
this research proposed a method of using FPGA to generate the
Index Terms—IEEE 1588v2; clock synchronization; hardware
timestamps; MAC; FPGA hardware timestamps in MAC layer. The synchronization
system has been designed and verified by experiment, ensuring
I. INTRODUCTION that the system can work normally and meet the requirement of
accuracy.
Along with the wide application of network technology, The rest of this paper is organized as follows: overview about
network equipment (especially the measurement and control, IEEE 1588 protocol will be presented in chapter II, while the
and communication equipment) requires higher clock proposed system's design will be explained in chapter III. In
synchronization accuracy. The high bandwidth and reliability of chapter IV, the verification result and analysis about the
Ethernet technology makes it become the most popular local proposed system is going to be shown, and at last, the
area network (LAN) technology at present. In order to solve the conclusion will be presented in chapter V.
problem of Ethernet timing synchronization, the computer and
network industry developed a software-mode Network Time II. IEEE 1588V2
Protocol (NTP) to improve the timing synchronization between
the network devices. Traditional NTP can only provide In [8], there are clearly defined two types of messages in the
millisecond level time precision [1-2], and still cannot meet the PTP protocol: event messages and normal messages.
accuracy required for modern measuring instruments and PTP synchronization system realizes the synchronization
industrial controls. between master and slave clocks through the interaction of the
To address the issue of precision clock synchronization in following four PTP messages: Sync, Follow_Up, Delay_Req.
measurement and industrial control networks, IEEE 1588 and Delay_Resp. The synchronous cycle can be set as 1s, 2s and
precise time synchronization protocol had been proposed [3]. 4s, normally set as 2s [9]. The interactive process for a basic
The full name of IEEE 1588 is “IEEE Standard for a Precision message synchronization is shown in Fig. 1.
Clock Synchronization Protocol for Networked Measurement According to the message interaction process, the slave clock
and Control Systems”, which is referred to as Precision Timing will get four timestamps: t1 , t2 , t3 and t4 . Assuming that toffset
Protocol (PTP) [4]. This protocol introduces timestamps in the represents the time offset between the master clock and the slave

978-1-5386-4599-4/18/$31.00 ©2018 IEEE


choose to generate timestamp. As shown in Fig. 2, the existing
MAC MAC implementation method usually generates timestamps in four
The timestamps
locations: A, B, C and D.
t1
obtained in Slave (1) Mark the timestamp at A or B: This is the simplest way.
However, since the PTP frame spans the data link layer to the
t t2 t2
2m application layer, the delay jitter caused by the operating system
and the protocol stack is generally in milliseconds. Thus, a great
t1 t 2 time error will be caused. The final synchronization accuracy
t 3m t3 t1 t 2 t3 will not be better than milliseconds.
(2) Mark the timestamp at C: This method eliminates the
t4 stack delay and jitter, and more accurately reflects the actual
sending and receiving time of synchronization message. It can
t1 t 2 t3 t 4
be realized by both software and hardware, and the time
time
precision can achieve to microsecond level.
Fig. 1. Typical synchronous messages interaction model
(3) Mark the timestamp at D: This is the best way, however,
this needs the special PHY chip to support 1588.
A PTP PTP
In order to achieve higher accuracy, while taking into account
the portability of the synchronization system, this study chose
UDP UDP
to generate the timestamp at C.
IP IP
Network latency
B. Hardware Design
B Network Driver and jitter Network Driver
This paper adopted the FPGA+ARM design method.
MAC MAC
C Hardware implementation diagram as shown in Fig.3. The
D PHY Network PHY dashed box in Fig. 3 shows the MAC_IP module implemented
with Verilog HDL code on the FPGA. This module is
Fig. 2. Timestamp mark position responsible for receiving and sending data frames from the
Ethernet PHY chip, and parsing and reporting PTP frames.
The general design idea of PTP module (Hardware assist
clock, d sm represents the link transmission delay from the
circuit) is as follows.
master clock to the slave clock and d ms represents the link (1) Recording frame arrival (delivery) time
transmission delay from the slave clock to master clock. The When a frame arrives (is sent), the PHY pulls up the
following two formulas can be obtained: rgmii_rx_ctl (rgmii_tx_ctl) signal which indicate the frame is
valid. And MAC_IP core according the signal to record the
t1 + toffset + d ms = t2 (1) arrival (delivery) time of frame and store it in the temporary
t3 + toffset + d sm = t4 (2) timestamp register.
(2) Identifying 1588 protocol frame
Supposing the transportation delay between master and slave
In the process of reading a frame, layer-by-layer filter and
is symmetrical, equation (3) can be obtained:
store the header information, until the message header is filtered
out, or the filtered information can determine that the frame
d sm = d ms (3) must not be the PTP frame.
Equations (1) and (2) are subtracted from both sides of the (3) Updating the PTP registers
equal sign, and combined with equation (3), equations (4) can In this research, PTP frame type flag registers and PTP event
be obtained: frame flag registers were set up. The PTP frame type flag

toffset =
( t2 − t1 ) − ( t4 − t3 ) (4)
registers updated in real time according to the information
filtered and requirements of each type of PTP frame. According
2 to the PTP type frame flag registers and filtered information,
The slave clock should be adjusted according to the toffset , so PTP event frame flag registers are refreshed.
(4) Generating timestamp
as to realize synchronization between master clock and slave
Combined with user configuration information and each type
clock.
of PTP event frame flag register information, the module
generates a flag signal that indicates whether or not to product
III. SYSTEM DESIGN
timestamp. If the flag signal is high, it will drive the
A. Timestamp Mark Position corresponding logic to write the arrive time recorded into the
The timestamp mark position determines the time precision timestamp FIFO and update the corresponding flag signal;
[10]. In engineering applications, there are many places to otherwise, arrived time will be discarded.
CPU
Send MAC_IP
MAC Receive Control
Interface

rx_dv

Receive Generate PTP


Timestamps Receive Receive
FIFO Timestamps parse
Host
CPU
AXI Configuration Ethernet
Interface Local Clock PHY
BUS Module Interface

Send Generate PTP


Timestamps Send Send
FIFO Timestamps parse

tx_en
CPU
Receive
MAC Send Control
Interface

Fig. 3. Hardware implementation diagram


The time synchronization process is shown in Fig. 4. Where
tslave position 1) represents the initial value of the slave clock can be
arbitrary; 2) indicates that the master-slave clock time deviation
Mster clock
is large due to the initial value, and may not be easy to adjust in
Slave clock
place by once; 3) and 4) indicate that the clock is faster than
1)
5) master clock at this time, resulting in a smaller deviation; 5)
4) indicates that the slave clock may be slower than the master
3)
clock after correction.
2)
After repeated and from coarse to fine adjustment, the slave
clock’s time is bound to a certain range of the master clock. This
range is synchronous precision.
2) Frequency Synchronization
tmaster
There are three main reasons for the frequency deviation of
Fig. 4. Time synchronization process
the system: the natural frequency deviation between different
C. CPU Processing Design crystal oscillation, the temperature drift of crystal oscillation,
The main function of master clock’s CPU completed is and the false error of the random noise introduced by the
composition and sending of frames. Synchronization algorithm measurement.
is carried in the slave clock. The synchronization algorithms Since the development board used in this article is an ordinary
include time synchronization and frequency synchronization. crystal, it cannot be adjusted directly by the way of voltage-
1) Time Synchronization controlled oscillation. According to Equation (5), the
As mentioned above, a time offset can be obtained when the cumulative value of time is related to the frequency f and the
master-slave clock completes a synchronization process. period N . Therefore, this paper transfers the adjustment of the
(1) The time offset exceeds -128s~127s: CPU first reads the
frequency error to the adjustment of the period N , and
current local time from FPGA, and accumulates the offset value
introduces the first-order Kalman filter algorithm.
to the current time. And then the local time configuration
command is issued to the FPGA local clock module, which t = ( f × N ) (5)
completes the overload of time by FPGA. Due to the bus delay Supposing the time offset after the n-th synchronization is
between the CPU and the FPGA will affect the time precision,
offsetn , the time interval between the two synchronization is
the coarse adjustment of time deviation of time deviation can
only be done in this way. Tn , and the frequency deviation between the master clock and
(2) The offset is between -128~127s: CPU uses the
the slave clock is Δf . The following formula is established.
proportional control algorithm to choose the suitable proportion
coefficient to calculate the necessary adjustment quantity. And 1
then the CPU issued a local time fine-tuning command to the
fs = (6)
Ns
FPGA local clock module, completed fine-tuning of time by
FPGA. In this case, the CPU does not need to read the current 1
fM = (7)
local time and does not have the effect of bus latency on the NM
precision, so the system time can be adjusted at a resolution of
1 nanosecond. Tn = t4 n − t4( n−1) (8)
t Slave
8.00102

7.98090

7.99901

8.10013

8.00001

7.98999

7.99999 Ideal Time

Real Time Fig. 6. Master-slave clock verification scheme


8.00000
between the two consecutive synchronization , that is, the
tMaster change of N s between adjacent two synchronization. Create a
Fig. 5. Frequency synchronization process Kalman filter model as follows:
Where f M and f S respectively represent the master and slave x ( n | n − 1) = x ( n − 1) (12)
clock frequency, N M and N S respectively represent the
P ( n | n − 1) = P ( n − 1) + Q (13)
master and slave clock cycle, t4n and t4( n−1) represent the n-th
P ( n | n − 1)
t4 timestamp. Since the generation of the t4 Kg ( n ) = (14)
and (n-1)-th P ( n | n − 1) + R
timestamp marks the end of the synchronization process, the
difference between two adjacent t4 can accurately reflect the x ( n ) = x ( n | n − 1) + Kg ×
(15)
synchronization interval Tn . ( period ( n ) − x ( n | n − 1) )
Assuming that the main clock has a nominal frequency of P ( n ) = (1 − Kg ( n ) ) × P ( n | n − 1) (16)
125MHz, it is easy to find the following equation between the
Where the Q and R are random noise, subject to the Gaussian
frequency f S , the period N S and the calculated offset . distribution of variance. The characteristics of the Kalman filter
offsetn − ( offsetn −1 + adjn −1 ) = show that the initial state parameter x ( 0 ) and P ( 0 ) can be
(9)
Tn × ( f s − f m ) × N s randomly given. After many calculations, the filter
convergence, x ( n ) is the best estimate of the current N s .
Where offsetn and offsetn −1 denote the time deviations
The frequency synchronization process is shown in Fig. 5.
obtained after this synchronization and the last synchronization,
The nominal clock frequency of the system is 125MHz, so each
and the adjn−1 denotes the correction amount of the local clock clock interval corresponds to a step length of 8 nanoseconds. As
after the last synchronization completed. The physical meaning the actual clock frequency fluctuates around the nominal value,
of this formula is as follows: The time deviation (the the corresponding step size N s also varies accordingly.
left side of the equation) from the completion of the previous
synchronization to the completion of this synchronization is
IV. FUNCTION VERIFICATION
generated cumulatively by the master-slave clock during this
period of time (equation right). Substituting equations (6), (7) According to the precise time stamping scheme of the message
and (8) into equation (9) yields equations (10) and (11). at the PHY and MAC layer interfaces described above, this
NS = Nm × paper uses two identical Xilinx Zedboard series FPGA
development board as the master clock and slave clock. The
 1  (10) main chip of the board is XC7Z020-CLG484-1 FPGA in the
1 − ( offsetn − ( offsetn −1 + adjn −1 ) )  Xilinx Zynq®-7000 family, which has the dual-core Cortex-A9
 TN  MPcore and the 100MHz crystal. A clock synchronization
N system is established by connecting a master-slave clock
− m ( offsetn − ( offsetn −1 + adjn −1 ) ) terminal through a twisted pair of Gigabit Ethernet, and
Tn (11)
communicates with a host through a serial port. In this paper,
= period n self-developed real-time monitoring software shows the
synchronization accuracy. The verification scheme is shown in
As a result, period n represents the frequency change
Fig. 6.
chip contributes a significant portion of the transmission delay
and jitter is on the order of 10 nanoseconds.

V. CONCLUSION
This paper presented a hardware and software
implementation using FPGA+ARM verification platform to
improve the synchronization of Ethernet master-slave clock
system by using improved MAC IP core capable of handling
IEEE 1588 protocol. Test shows that the system can work at
10/100/1000M Ethernet rate. The system uses a common
Ethernet PHY with RGMII interface, and has the advantages of
Fig. 7. Time deviation diagram before system synchronization fast synchronization, high precision, low requirements on the
peripheral chip, with good portability and scalability.

ACKNOWLEDGMENTS
This work is supported in part by Science and Technology on
Information Transmission and Dissemination in
Communication Networks Laboratory Project (KX152600020/
ITD-U15011), Fundamental Research Funds for the Central
Universities (JB140112), National Natural Science Foundation
(61306047).

REFERENCE
Fig. 8. Time deviation diagram after system synchronization [1] F.-Steinhauser, C.-Riesch, M.-Rudigier, “IEEE 1588 for time
In order to more intuitively display the time deviation synchronization of devices in the electric power industry”, in Proceedings
between the master clock and the slave clock, this paper uses the of International IEEE Symposium on Precision Clock Synchronization for
Measurement Control and Communication, Oct. 2010, pp.1-6
serial port to send the deviation calculated by the CPU to the PC
[2] Y. Peng, Q.-H. Luo, and Z.-Q. Liu, “An automatic evaluation system for
in real time when setting up the verification platform. Using IEEE1588 synchronization clock unit,” in Proceedings of the Ninth
self-developed real-time monitoring software, the time International Conference on Electronic Measurement and Instruments,
deviation between master and slave clock can be displayed in Aug. 2009, pp.408-413.
[3] J. Eidson and Kang Lee, “IEEE 1588 standard for a precision clock
real time after each synchronization. The actual test results are
synchronization protocol for networked measurement and control
shown in Fig. 7 and Fig. 8. The test results are as follows: systems,” in Proc. 2nd ISA/IEEE Sensors for Industry Conference, Nov.
(1) With 30cm network cable or 3m cable, the path delay time 2002, pp. 98-105.
is always around 340us; [4] Zhao, Hongkun, X. Wang, and Z. Yang. "Design and implementation of
precision time synchronization system based on IEEE1588." In
(2) Since the master and slave clocks have different power-
International Conference on Electric Utility Deregulation and
on times, a large deviation will exist at the beginning. But the Restructuring and Power Technologies, 2011, Vol.47, pp, 610-613.
slave clock can quickly lock the master clock and adjust to a [5] Moreira, N., Astarloa, A., Lazaro, J., Garcia, A., & Ormaetxea, E. “IEEE
steady state; 1588 Transparent Clock architecture for FPGA-based network devices.”
in Industrial Electronics (ISIE), 2013 IEEE International Symposium,
(3) After the system is stable, the time deviation between the
May, 2013, pp. 1-6.
master clock and the slave clock remain within ±20ns. [6] C Kutschera, C Veigl, R Höller, et al, “Background IEEE 1588 Clock
(4) Since the time deviation calculation program runs on the Synchronization over IEEE 802.3/Ethernet,” in Proc. ISPCS, Sep. 2008,
CPU, it is possible for the upper layer to use a more advanced pp. 22-26
[7] B. Zhao and N. Wang, “The implementation of IEEE 1588 clock
synchronization algorithm. synchronization system based on FPGA,” in Proc. ICICIP, Aug. 2014, pp.
There are three main reasons for time offset: 216-220.
(1) The difference of synchronization cycle can make [8] IEEE Standard for a Precision Clock Synchronization Protocol for
different time offset. The shorter the synchronization period, the Networked Measurement and Control Systems, IEEE Standard 1588,
2008.
higher the synchronization accuracy. However, in practical [9] K Correll, N Barendt, M Branicky, “Design considerations for software
applications, the performance of the system decreases with the only implementations of the IEEE 1588 precision time protocol,” in Proc.
decreases of clock synchronization cycle. Conference on IEEE 1588 Standard for A Precision Clock
(2) There is a frequency difference between the master clock Synchronization Protocol for Networked Measurement and Control
Systems, Oct. 2005, pp. 11-15.
and the slave clock crystal oscillation that affects the [10] M Ouellette, K Ji, S Liu, et al, “Using IEEE 1588 and boundary clocks for
synchronization precision. clock synchronization in telecom networks,” IEEE Communications
(3) The temperature drift of the crystal oscillation itself has a Magazine, vol. 49, no. 2, pp. 164-171, Feb. 2011.
great influence on the synchronization accuracy. And the PHY

You might also like