2019 IEEE 3rd Information Technology,Networking,Electronic and Automation Control Conference (ITNEC 2019)

Research on Clock Synchronization Mechanism of

Distributed Simulation System based on DDS
Wu Yongliang1 , Luo Peng2, Chen Xiaoping2, Xiong Lichuan1,2
1.Chengdu Aircraft Design & Research Institute of AVIC, Chengdu,China
2. College of Aeronautics and Astronautics,University of Electronic Science and Technology of China,Chengdu, China ;
wuyongliang@buaa.edu.cn, marconi1217@163.com, xpchen@uestc.edu.cn, 611@avic.com

Abstract—Flight control system simulation test is a key
process in aircraft design stage. Distributed system
architecture is often used in flight control simulation test.
Clock synchronization has always been a research hotspot of
distributed simulation system. Data distribution service (DDS)
is widely used in distributed simulation system for its good
real-time performance, high efficiency and low coupling.
However DDS does not provide a excellent clock
synchronization mechanism. Aiming at the demand of clock
synchronization of distributed simulation test system, an
improved PTP clock synchronization algorithm is proposed in
this paper, and a virtual clock of distributed simulation system
is constructed on the basis of the propulsion timing model.
Experimental data show that the model can be applied to
simulation nodes stably, and the improved algorithm can also
achieve clock synchronization between nodes of distributed
system.
Keywords—Data distribution services; Clock synchronization;
Virtual clock
I. INTRODUCTION
In the flight control system simulation test for near space
UAV[1], in order to make the simulation structure more flexible
and the operation of each subsystem more efficient, the test
mostly adopts the distributed architecture. Each node has its
own operating environment. There are four nodes in the present
system: flight control computer, AC flight package, ground
station and airborne system. The simulation structure for flight
control simulation test is as Fig.1.
)OLJKWFRQWURO $&
FRPSXWHU SDFNHWV
''69LUWXDO%86
0DLQFRQWURO
V\VWHP
*URXQG $LUERUQH
VWDWLRQ V\VWHPV
Fig.1. Structure of flight control simulation system
In the distributed system of flight control system simulation
test, the efficient and timely data interaction between each node
is a key point of the system design. For distributed simulation
test system, DDS is an excellent distributed message
middleware [2]. It set up a system composed of release node and
subscription node structure, providing a very efficient data
transmission mechanism and rich and practical QOS
management functions. It is applied to resources field which
requiring high performance and predictability and effective use
of the key tasks.
However DDS lacks the implementation and management
of time synchronization mechanism of each node. What’s more,
clock synchronization between nodes is more important
between distributed systems,. It needs to make sure that all the
nodes in distributed simulation system of the time difference in
a certain range. This paper studied on the basis of the DDS
clock synchronization mechanism of distributed systems.
II. DATA DISTRIBUTION SERVICE
DDS is a software design specification and standard
developed by OMG (object management organization) [3],
which can achieve publish/subscribe communication between
transport layer and application layer while meeting the real-
time requirements. This specification standardizes the interface
and behavior of data publishing, transmission and subscribing
in distributed real-time system. It can enhance the system
dynamics, and improving communication efficiency and
system flexibility.
RTI DDS(Formerly known as NDDS[4]) is developed and
implemented by Real-time Innovation Company, which was
the first commercial product to support the DDS specification.
DDS is designed for data publishing/subscription requirements
of real-time systems[5]. DDS supports multiple data distribution
patterns and data distribution strategies, and supports QoS
configuration in the form of XML. It is possible to change the
communication mode and performance of the system without
changing the source code of the nodes, thus reducing the
coupling between nodes [6] .
DDS data publish-subscribe model establishes the concept
of "global data space", as in Fig.2. When data is exchanged
between nodes, Publisher creates the DataWriter with the
specified Topic and then writes the data to the global data
space through the DataWriter. The Subscriber creates the
DataReader of the corresponding Topic, which retrieves data

978-1-5386-6243-4/19/$31.00 ©2019 IEEE 2224

orized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on January 22,2024 at 09:38:27 UTC from IEEE Xplore. Restrictions ap
 from the global data space. And Topic is the corresponding count runs at the set frequency and the virtual clock starts
relationship between publisher and subscriber. running. The functions listed in the table above are also
provided.
TABLE 1.INTERFACE TO THE VIRTUAL CLOCK
1RGHB$ 1RGHB%
&ODVV 9LUWXD&ORFN
* OREDO'DWD6SDFH ,QLWLDOL]H 
1RGHB& 1RGHB' *HW7LPH%HDW 
6HW)UHTXHQF\ 
,QWHUIDFH
&DOLEUDWH7LPH 
Fig.2.DDS communication model 5HVHW%HDW 

DDS also provide a request reply mode based on publish-

subscribe mode for different application scenarios. The
request-reply pattern has two roles: Requester sends the request
message and waits for the reply message. Replier receives the
request message and responds with a reply message.
Requesters and Repliers manage all interactions on behalf of
the application. Requester and Replier discover each other
automatically based on the service name specified by the
application. When the application invokes the function,
Requester sends a message to the Replier informing the
receiving application. The application uses the Replier to
receive the request and send a reply message in turn.
III. PROPULSION TIME MODEL
In the process of the correction of clock synchronization,
the local clock is generally modified with the calibration results.
However, the change of node local time will also affect other
processes running in the system, and the periodic modification
IV. IMPROVED PTP CLOCK SYNCHRONIZATION ALGORITHM
A. PTP clock synchronization protocol
PTP clock synchronization protocol [9] is a kind of clock
synchronization protocol applied to distributed measurement
and control system through network communication, local
computation and distributed objects. It’s also called ieee1588.
PTP protocol has two assumptions as follows:
1) The network delay has symmetry. That means the
transmission network delay of sending and receiving between
the same two nodes is equal.
2) The sending processing time of each packet is ignored.
PTP protocol clock synchronization model implementation
process is as in Fig.3.
0DVWHU 6ODYH

of the local clock will also make the system run abnormal. To 7P
avoid this situation, this paper constructs a virtual timing clock
referencing computer physical clock timing model. The clock
frequency of it even can be adjusted. 76
The virtual clock is implemented with the method of the
software by a separate higher-priority thread. The clock is 7P

maintained by each node of the distributed system according to 76

the internal synchronization mechanism of the simulation
system, isolating with physical environment of each node.
7P
Achieve the effect of software clock synchronization [6]. This
is called the propulsive clock model [8].
ܲ݇ǡ݊൅ͳ ൌ ܲ݇ǡ݊ ൅ ‫ ݇ܨ‬ሺܶ‫ݏ‬൅ͳ െ ܶ‫ ݏ‬ሻ  
7P
In the formula:
ܲ௞ǡ௡ is the calculation value of the propulsion timing
clock model.
k = 1, 2, 3,... , represents the node number of the Fig.3. PTP clock synchronization model
distributed system.
n =1,2,3... , represents the number of cycles. 1) The master clock sends out sync message and records the
exact time(Tm1) when the message leaves the master node;
‫ܨ‬௞ǡ௡ represents the clock frequency of the propulsion
timing clock model. 2) The master clock encapsulates the accurate sending time
ܶ௦ାଵ െ ܶ௦ represents the logical time interval. Tm1 into follow_up message and send it to the slave node;
In this model, logical time synchronization means that the 3) The slave node record the exact arrival time (Ts1) of
calculation values of each node are the same. If the calculation sync message from the master node;
values of each node at a certain time are all P, it can be
considered that they are at the same time of logical time T. 4) The slave node send delay_req message and record the
The implementation interface of the virtual clock is as in exact sending time(Ts2);
Tabale1. When the clock is initialized, a new thread with the 5) The master node records the exact arrival time of
highest priority is opened to run the clock. The virtual clock delay_req message to the master clock(Tm2);

2225
orized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on January 22,2024 at 09:38:27 UTC from IEEE Xplore. Restrictions ap
 6) The master node sends delay_resp messages with
accurate timestamp information(Tm2)to the slave node;
Offset of master and slave clocks can be calculated by Tm1,
Tm2, Ts1 and Ts2:
ሺ் ି் ሻିሺ் ି் ሻ
‫ ݐ݁ݏ݂݂݋‬ൌ  ೞభ ೘భ ೘మ ౩మ   
ଶ
After getting the offset from the node, the slave node can
synchronize the time by modifying the local virtual clock.
B. Improved PTP protocol based on virtual clock
(1) Initial alignment time
The PTP protocol has its own clock for master and slave
nodes, and the clock is always running. This paper has
presented a propulsive timing model in which each node
maintains a virtual clock that can be invoked by command.
Therefore, there is no clock offset in the improved clock
synchronization process. The master node wakes up the slave
node's clock through the first packet. After receiving the
message from the clock, it is logically separated from the
physical environment where each node is located. The
implementation process is shown in the Fig.4.
0DVWHU 6ODYH
'HOD\
76 
76
'HOD\
76
7P
76Q
7P
7PQ
&DOLEUDWH
Fig.4. Virtual clock synchronization initialization
Synchronization process:
1) The master node send the start command (StartCmd) to
the slave node for requesting clock synchronization, and
records the sending time Tm1;
2) After a certain network delay, the slave node receives the
startup command of the master node and starts the virtual clock
of the node;
3) The slave node send multiple replies to the request of the
master node, and add their own timestamp TS2 in the reply
packet;
2226
orized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on January 22,2024 at 09:38:27 UTC from IEEE Xplore. Restrictions ap
4) After receiving the reply form the slave node, the master
node calculates the network delay and send the calculated
delay to the slave node as the virtual time calibration command;
5) After receiving the calibration command from the master
node , the slave node calibrate it’s clock by making a
backward adjustment(Δ T);
The virtual clock of slave clock get more accurate time
correction by the synchronization process .
୘ ି୘ ି୘
‫ ݐ݁ݏ݂݂݋‬ൌ ೘మ ೘భ ೞభ   
ଶ
The calibration result is the slave clock adjusting ‘ˆˆ•‡– ൅
௠ଵ forward of the virtual clock of this node. In this way, the
time of receiving message from the master node in the above
process can be understand as ‘ˆˆ•‡– ൅ ௠ଵ , which achieves the
clock synchronization in the virtual time domain.
PTP protocol is a clock synchronization based on the
assumption that the sending processing time is ignored. After
the improvement, the last calibration command (CalibrateCmd)
only needs to tell the slave node to adjust how much time. It
does not need to be calculated into the synchronization model.
So, only two packages are used for computing the sending time
of data. In some way, it reduces the error causing by ignoring
processing time.
In the above synchronization process, as in Fig.5, only one
set of data is taken for clock synchronization time correction.
The big problem is the contingency of data because of the
fluctuation of the collected data. In order to ensure that the
uncertainty error has the least impact on the calibration effect
of clock synchronization, the slave clock returns multiple sets
of clock synchronization data when it make the reply to the
master node.
&ORFNVWDUW
FRPPDQGDQG
V\QFKURQRXV V\QFKURQRXV &DOLEUDWLRQ
UHSO\  UHSO\ Q FRPPDQG
V\QFUHTXHVW
1RGHQ



1RGH
1RGH
1RGH
7
Fig.5. Multi-node clock initialization process
When the master node has received the clock
synchronization data, it continues estimating of network delay
by Least Square Method:
› ൌ σ௡௜ୀଵሺʹ‫ ݐ݁ݏ݂݂݋‬െ ௠ଶ௜ ൅ ௠ଵ ൅ ௦ଵ௜ ሻଶ   
That is,
σ೙ ் ିσ೙ ் ்
‫ ݐ݁ݏ݂݂݋‬ൌ  ೔సభ ೘మ೔ ೔సభ ೞభ೔ െ ೘భ  
ଶ௡ ଶ
i represents the ID number of each node of the distributed
system.
 At the end of period of clock synchronization initialization,
the clock master node calculates the time correction of each
node according to the response packet of each slave node to
clock synchronization request. And the master node sends the
clock modification amount of each node ο ൌ ‘ˆˆ•‡–݅ ൅ ݉ͳ
to each slave node in the later period of clock synchronization
Initialization phase.
When the Master Node makes a clock synchronization
request, it creates a synchronous requester. Materializing
requests by using the data types of requests and replies, and
initialize the requester with requester_params. Setting the
node ID of the master node of the synchronization system is 0,
and the startup control command to the macro StartCmd. And
set the timestamp of the packet. The requester's send_request()
function is called to send the synchronization request. Then it
enters the ready receiving state of the reply packet.
The packet structure of the master node publishing for
requesting clock synchronization is as follows:
struct TimeSyncRequester{
unsigned long NodeID;
unsigned long StartCmd;
TimeStamp MasterSendTime;
};
Each Slave Node creates a replier at startup and materialize
requests by using the data types of requests and replies . Then
initializing the replier with replier_params.
When the slave node receive the master node clock
synchronization request, it parse the packet and judge whether
the packet is master clock node (NodeID = 0), and ensure
whether it is the start command for clock (StartCmd). If the
clock synchronization request packet and request command is
from master clock node, each slave node invoke the Initialize ()
command of SoftClock to start a local virtual clock. At the
same time, creating the reply packet, and set the local node ID
in the reply packet, take out the master node publishing time
from the request packet and set it to the reply packet. Before
sending the reply packet, add the time stamp of the slave node's
virtual timestamp to the packet.
Reply packet structure from slave clock:
struct TimeSyncRequester{
unsigned long NodeID;
TimeStamp
MasterSendTime;
TimeStamp SlaveSendTime;
};
After the master node receives the reply from each node,
the synchronization algorithm designed above is used to
calculate the network delay of the synchronization process for
each packet. And the modified results of each slave node
calculated are sent to each node through the write() function of
DDS's publish and subscribe mode. After each slave node
receives the correction time. packet, it modifies the local
virtual clock. So as to complete the slave clock startup and the
2227
orized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on January 22,2024 at 09:38:27 UTC from IEEE Xplore. Restrictions ap
process of simulating the master and slave clock initialization
of the system.
Initial alignment time is complete. The virtual clock
running of each node is affected by the physical characteristics
of the node itself. So in order to improve the stability of clock
synchronization, it have to do periodic correction after
initialization process.
(2) Periodic time calibration
The timing process in the previous section has been
completed. The soft clock of each slave node has been started
by the master node and preliminarily adjusted. In order to
maintain the stability of clock synchronization, it is necessary
to calibrate each clock periodically. Therefore, in each
simulation cycle of the distributed simulation system, the
master clock also needs to publish its own clock information,
and each slave node uses the received master node clock
information to calibrate the clock of the slave node, for
maintaining the stability of clock synchronization effect.
In previous studies, the periodic frequency correction of the
clock mostly calculate by the way of one node i to collect all
other nodes j= 1,2... i-1, i+1... , the clock difference of n, and
the local clock frequency calibration can be obtained after
average or weighted average processing[10]. However this
method has a disadvantage, that is the local virtual clock
frequency is the physical environment and the local is
concerned, if the local clock frequency correction introduced
other nodes of the clock signal, it can hardly optain very good
correction effect. And each node of the physical environment
to virtual clock clock frequency influence still exists, and it
gets worse and worse as the virtual clock runs. According to
the analysis, in the design process of clock frequency
calibration module in the paper, it is done separately by each
node . The node calibrates its virtual clock count and frequency
based on its clock run offset.
The calibration process is as in Fig.6. The master node
publishes the virtual clock of this node to each slave node
through the write() interface of DDS. In order to save network
bandwidth, DDS transmission mode is configured to broadcast
int the paper.
&\FOH  &\FOH  &\FOH  &\FOH Q
0DVWHU 7 0 70 70 70 70Q
,QLWLDOL]DWLRQ
Time calibra tion
Time calibra tion
Time calibra tion
6ODYH 76 76 76 76 76 76Q
Fig.6. All processes of clock synchronizes
In the Fig.6, the master node send the virtual clock
information of slave node (TMn) at the beginning of each
synchronization cycle. And the slave node also records the
local time 76Q after receiving a clock message from the master
node each cycle . Assuming that each time the machine virtual
clock messages sent from the master node to node of one-way
network latency is equal. This hypothesis has less influence on
 the error than the previous bidirectional network delay . It is and the maximum error is 3%. So the slave clock frequency is
closer to the actual situation. relatively stable.
The time information in figure 5 has the following 1.04
x 10
4

relationship: 1.03

1.02

ܶெ௜ െ ܶெሺ௜ିଵሻ ൌ ܶௌ௜ െ ܶௌሺ௜ିଵሻ ൅ ߠ௜ 㸪݅ ൌ ͳǡʹ ǥ ݊ ǥ  

Frequency of slave clock

1.01

Where,ߠ݅ is the clock offset of the slave node to the virtual 0.99

clock of the master node that takes place in the cycle ݅ .Then 0.98

each time the frequency correction is carried out, the 0.97

proportional relation of clock frequency is: 0.96

0 20 40 60 80 100 120 140 160 180
Simulation cycle

ிೖǡ೔ ்ೄ೔ ି்ೄሺ೔షభሻ

ൌ ˈ݅ ൌ ͳǡʹ ǥ ݊ ǥ   Fig.7. Frequency of the slave virtual clock
ிబǡ೔ ்ಾ೔ ି்ಾሺ೔షభሻ
In this model, the clock frequency of the master node is On the other hand, clock comparison before and after
always set to a constant value. That is, ‫Ͳܨ‬ǡ݅ ൌ  ‫ Ͳܨ‬. algorithm improvement is carried out on one node. The
simulation results are as in Fig.8.
So the new correction frequency is:
8
்ಾ೔ ି்ಾሺ೔షభሻ The improved

‫ܨ‬௞ǡ௜ାଵ ൌ ‫ܨ‬଴ ൈ ˈ݅ ൌ ͳǡʹ ǥ ݊ ǥ   7

No improvement

The clock difference between slave and master nodes(ms)

்ೄ೔ ି்ೄሺ೔షభሻ 6

Therefore, after the slave node receives the periodic clock 5

release of the master node, it can calculate the frequency to be 4

corrected by the above formula. By modifying the clock 3

frequency, the master and slave nodes can reach the same 1

speed. Then the offset in the above formula is calculated and 0

corrected. -1
0 100 200 300 400 500 600 700 800
Simulation time(s)

In each time calibration cycle, the offset of the period

length is still caused by the offset of the clock frequency, so the
mathematical relationship involved is as follows:
‫ Ͳܨ‬ൈ ο‫ ݐ‬െ ‫݇ܨ‬ǡ݅ ൈ ο‫ ݐ‬ൌ  ߠ݅   
Meanwhile, the proportionality between ο‫ ݐ‬and the
previous clock frequency is:
்ೄ೔ ି்ೄሺ೔షభሻ ்ಾ೔ ି்ಾሺ೔షభሻ
ο‫ ݐ‬ൌ  ൌ   
ிೖǡ೔ ிబ
The clock offset from the slave node to the master node
during each clock synchronization period can be derived from
formula(9) and formula(10).
‫݇ܨ‬ǡ݅
ߠ݅ ൌ ሺܶ‫ ݅ܯ‬െ ܶ‫ܯ‬ሺ݅െͳሻ ሻሺͳ െ
‫Ͳܨ‬
‫Ͳܨ‬
ߠ݅ ൌ ሺܶܵ݅ െ ܶܵሺ݅െͳሻ ሻ ൈ ሺ െ ͳሻ  
Fig.8. Clock difference between slave-master nodes
The simulation results show that before the algorithm
improves, due to the periodic calibration of slave clock, the
average clock difference between master and slave clock is not
even large, but it's very volatile. After the algorithm is
improved, the clock frequency of slave node is adjusted
periodically. In this way, slave nodes can adjust their clock
speed according to the clock difference with the master node.
Figure 8 shows, this improvement is very effective, clock
difference between the master and slave nodes is very stable.
Deploy the designed clock model and synchronization
algorithm to multiple nodes for verification. The results of time
ሻ  data collection are as in Fig.9.
5
Node-1

‫݇ܨ‬ǡ݅ 4.5
Node-2
Node-3
Node-4

After the above analysis, after the slave node receiving the 4

periodic synchronization time packet from the master node, the

Clock difference with the master node(ms)

3.5

clock offset of it’s local virtual clock is calculated. The slave 2.5

node uses the calculated results to calibrate the local virtual 2

clock. The frequency calibration and offset calibration of 1.5

periodic calibration are completed. 0.5

V. THE SIMULATION VERIFICATION -0.5

0 5 10 15 20 25 30
Simulation time(min)

This part is the simulation and verification of virtual clock

model and synchronization algorithm. The master node clock
frequency is stable to 10 KHz, which means for every 0.1ms,
virtual clock count plus 1. Virtual clock frequency of slave
node calibrates itself according to the difference form the
master clock each cycle. The clock frequency of the slave node
can be obtained as in Fig.7, sampling every second. As can be
seen from the Fig.7, most of the time the error is within 1.5%
Fig.9. Clock difference with the master node
According to the simulation results, when the clock model
and synchronization algorithm are deployed to each node, the
clock error of each node is within milliseconds and mainly
within 3ms. It has good clock synchronization accuracy.
After the previous research and design, the internal time
domain of the simulation system was established on the basis

2228
orized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on January 22,2024 at 09:38:27 UTC from IEEE Xplore. Restrictions ap
 of the propulsion timing model. The clock synchronization
algorithm is used to carry out initialization process and
periodic calibration of the clock of the master and slave nodes.
The calibration process also included frequency correction and
clock offset calibration, so that each simulation node achieves
good clock synchronization accuracy. In this way, even if the
gap between each node and the local physical clock is larger
and larger, the clock synchronization of the entire simulation
time domain can be guaranteed to be within a certain accuracy.
VI. CONCLUSIONS
In the distributed simulation test system, each functional
module runs on different terminals respectively. In order to
solve the practical need of clock synchronization, the virtual
clock which promotes the timing model is adopted to avoid
frequent modification of the local clock of the terminal. Then,
on the basis of distributed simulation test system
communication middleware DDS, the PTP clock
synchronization algorithm is improved and implemented.
Through the analysis of test data results, the virtual clock
model and synchronization algorithm in this paper have a high
accuracy, which meets the requirements of the existing system
for time synchronization accuracy.
2229
orized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on January 22,2024 at 09:38:27 UTC from IEEE Xplore. Restrictions ap
REFERENCES
[1] Y.L Wu,Y.L Cai,H.X Baoyin,L.C Xiong,et al,”Design of redundant
fault-tolerant navigation system for near space UAV”.Acta Aeronautica
et Astronautica Sinica.2016,37(S1):pp91-98.
[2] PardoCastellote,Gerardo.OMGDataDistributionService(DDS):Architect
ural Overview. Real-Time Innovations, Inc. :REAL-TIME
INNOVATIONS INC SUNNYVALE CA, 2011, 1.
[3] OMG, Data distribution service for real-time systems
specification.Version 1.2. http://www.omg.org/spec/DDS/ Jan.2007.
[4] Pardo-Castellote G, Schneider S, Hamilton M. “NDDS: The real-time
publish-subscribe middleware”.Real-Time Innovations, Inc, 1999.
[5] Gerardo PC. “OMG Data-Distribution Service (DDS): Architectural
Update”. 2004 IEEE Military Communications Conference, 2004,
pp961-967.
[6] J Han,Y.S Wang,” Design and Implementation of Sensor Data
Distribution System Based on DDS”.Ship Electronic Engineering,
2018,Vol.38 No.10,pp130-133.
[7] N Sun,W Xiong,Y.Z Ding,”Study and Application of Clock
Synchronization” , Computer Engineering and Applications | Comput
Eng Appl.2003.27,pp177-185.
[8] Y.X.Ling,L.J Fan,T.Wang,G.H Zhang,”ock Synchronization Algorithm
Based on Parametric Difference for Distributed Node Clusters,” ACTA
ARMAMENTARII, V0l.34 No.1,pp.51-58,2013
[9] IEEE Std 1588-2008.IEEE standard for a precision clock
synchronization protocol for networked measurement and control
systems. New York:Institute of Electrical and Electronic
Engineers ,2008.
[10] Y.N Ren,B Zheng,H.L Guo,” Research and development of IEEE 1588
in distributed wireless network”, Information Technology.
2018.11,pp103-110.