AADC 3 : Active-Active Distributed Controller with

3-in-1 Asynchronous Heartbeat Synchronization

Method in Software-Defined Networks
Muhammad Arief Nugroho1, Vera Suryani2
School of Computing
Telkom University
Bandung, Indonesia
arif.nugroho@telkomuniversity.ac.id1, verasuryani@telkomuniversity.ac.id2

Abstract—The problem of synchronous message exchange on

and then the receiver will then respond with one acknowledgment.
active-active distributed SDN controllers is that every message Using this method can reduce the workload of each controller and
sent by controller A (sender) will be responded to by controller B a faster controller-to-controller communication process.
(receiver). Therefore, sending the following message after
receiving each response, causing on performance degradation and II. RELATED W ORKS
a higher controller’s workload because every message expects an A. Related Works
immediate process to generate acknowledgments messages.
Therefore, we developed a method to improve the message [2] Flat architecture provides more resistance to the network's
exchange mechanism and reduce the message resources burden in failures than the hierarchical controller because it has several
sending synchronization message information between con- layers with different responsibilities, so it is more vulnerable to
trollers. This study proposes using asynchronous messages as failure on the network. [3] describe that flat architecture using
distributed SDN controller message exchanges with the 3-in-1 the active-active method to make all active controllers work
method, which is the mechanism carried out by sending three
messages and producing a reply in one acknowledgment. Our together to manage a network simultaneously and more
experiments show that the 3-in-1 method gives lower CPU and frequently to ensure network consistency. Another study shows
memory usage and has a higher throughput than the traditional that the active-passive distributed SDN controller [4] shows that
message exchange in Distributed SDN Controller. the backup controller's existence serves to monitor and do
Index Terms—active-active, distributed SDN controller, asyn- backups if there is a network failure on the main controller. [12]
chronous messages, 3-in-1 conducts the design and implementation of controller-to-
controller communication in- interface. This study also explains
I. INTRODUCTION
the types of synchronous communication and asynchronous
A distributed Controller is a multiple-controller that is used to messages. The results of the study indicate that the use of
increase scalability in the network. When designing a network, asynchronous messages increases network flexibility which
one must consider scalability in a system, network, and process significantly increases network performance and reduces the
that shows its ability to handle several potentially significant workload on the controller
performance improvements. A distributed controller is divided B. Software Defined Networks
into two architectures, flat architecture and hierarchical Software-Defined Network (SDN) is a new paradigm that can
architecture. Flat architecture is a horizontal scaling so that the be programmed and controlled centrally by separating the
control plane consists of only one layer, and each controller has control plane and the data plane. The Control plane is a
the same responsibility at the same time [1]. The flat architecture component on the network that functions to control the network
has more flexibility on the network's failures than hierarchical by performing a system configuration, network management,
architecture [2]. Flat architecture Distributed Controller divided determining the routing table information, and forwarding
into active-active and active-passive. Distributed controller table. Data plane is the part that is responsible for forwarding the
(active-active) using flat architecture makes all controllers active packet. SDN architecture enables a network to increase network
and works together to manage a network simultaneously and scalability, which is realized through a simple implementation of
expand the control plane's capabilities. In the previous research, network components and services. SDN introduces a method for
Distributed Controller (active-active) analysis using synchronous increasing the level of abstraction in network configurations.
message exchange, compared to the active-backup method, the SDN's main objective is to achieve better network management
research results in a lower CPU performance resulting in a faster with great complexity and to ensure that all decisions of the
failover process [3]. In general, synchronous messages have the control system are made from the central point (controller) [6].
drawback of requiring more time while waiting for a receiver's
C. Distributed Controller
response because the communication is real-time, and higher
workload on each controller. Different from asynchronous A distributed controller is a multiple-controller on a Software
messages, several messages can be sent directly, and then the Defined Network (SDN) network used to increase scalability in
receiver will execute. This paper presents a new mechanism on a network. In this controller, SDN has two or more controllers
controller-to-controller communication, AADC 3: 3-in-1 where each controller is responsible for providing information
asynchronous message, which works by sending three messages, or messages to other controllers [7]. A distributed controller is
divided into two architectures, flat architecture and hierarchical
architecture. Both architectures can increase switch or
controller latency. However, flat architecture is more resistant
to failures in the network compared to hierarchical architecture.
Flat architecture has the same responsibilities, whereas
hierarchical architecture has different responsibilities because
devided by multiple levels [2].
affects the speed of data transmission continues to increase
linearly [13]. Therefore, this study uses a 3-in-1 method on the
active-active Distributed Controller architecture by sending
three messages and responding to one acknowledgment in each
message exchange process between controllers to reduce the
burden of message resources in exchanging messages so that
they run faster.

D. Reliability In this research, the message exchange process is done by

Reliability is a network capability that has a high availability sending three messages and then responding by controller B
of network services. This capability considers network traffic with one acknowledgment (ACK). Then, sending messages
such as overload, latency, and also throughput [9]. Reliability works for a maximum of 1 second, so the message sending
has a network technique called a failover mechanism, which process lasts for less than 1 second. Suppose the sending time
provides two or more connection lines when one goes down. is more than 1 second and the message sending is less than three
The connection is still running by diverting to the other line. messages. When the controller is declared down, the controller
This mechanism designed to act as soon as possible when migration process starts from controller A to controller B.
downtime occurs [3] [10]. Timeout between each controller is 1 second, specified as the
maximum time limit for sending synchronization messages.
E. Synchronous vs Asynchronous Message Exchange
The message exchange system is a set of techniques that
allows objects to communicate in sending and receiving
messages over the network. A message contains information
such as a request, response, or unresponded delivery messages.
The message exchange is divided into two communication
models called synchronous messages and asynchronous
messages. The synchronous message has a synchronization
point for each migration interval when communication occurs.
All processes must wait and block until all processes reach the
same point. This approach works in real-time, which means that
the communication process runs directly. When the sender
sends a message, the receiver will be responsible, affecting the
next message’s execution. The sender cannot send the message Fig. 1. 3-in-1 Message exchange under normal condition.
before receiving a response from the receiver. A longer time is
needed while waiting for the response. Unlike the In Figure 1, the asynchronous message exchange between
asynchronous message, this communication method has a controllers using RabbitMQ as message broker. Controller A
queuing scheme to not require a response to continue the next sends heartbeat messages to controller B via message broker.
process. In asynchronous messages, the controller can directly Likewise, controller B conveys the message through the message
send several synchronization messages, and then the receiver broker then forwards the message with an acknowl- edgment
will execute them. This approach has many advantages (ACK) that replies three messages to controller A. The
compared to synchronous messages. Asynchronous messages parameters used in testing with this scenario is CPU Usage to see
(nonblocking i/o) demonstrate higher performance results in the controller workload.
controller-to-controller communication compared to
synchronous messages (blocking i/o) [12].
III. RESEARCH M ETODOLOGY
A. 3-in-1 Active-Active Distributed Controller Message Ex-
change
The 3-in-1 Asynchronous Heartbeat is a new method on
distributed SDN controller message synchronization in this
study using the active-active distributed controller. The
message exchange process will performed after each
controller has mutually detected that the controller status is
mutually active or called a heartbeat. Then, the process of
exchanging messages between controllers using the 3-in-1
method works by way of controller A as a sender sending three
messages and controller B as the receiver will respond with
one acknowledgment. The 3-in-1 method introduces the Fast
Recovery Fast Retransmit in TCP Reno, where the technique
sends three duplicate messages to speed up the lost message’s
delivery time. Then, research explains that in the third
duplicate transmission, the threshold value decreases, which

Figure 2. 3-in-1 Message exchange failure controller
In Figure 2, an asynchronous message exchange has a
controller A down because the message sent is less than three
messages, and the sending time is more than 1 second. So that
the switch migration process from controller A to controller B.
This scenario simulated using a controller failover mechanism.
B. Network Topology
This study's design is a development method from previous
studies, i.e., Distributed Controller (active-active) topology [3].
The communication model as a message exchange in the paper
using asynchronous messages with the 3-in-1 method. The
controller used in this paper using POX
Figure. 3. Network Topology
In figure 3, two controllers are mutually active to work
together, namely Controller A and Controller B. Each
controller manages two switches, where the controller manages
switch one and switch two. Controller A directly connected to
controller B. Switch three and switch four are managed by
controller B and connected by controller A. When controller A
down, the switch will migrate to controller B. In the above
topology, using the same network and each host can
communicate with other hosts.
Table 1 Total Host and Switch
Switch 4 8 16 32
Host 8 16 32 64
Table 1 shows the number of nodes containing the number of
switches and the number of hosts used for evaluation in this paper.
IV. E VALUATION
In this chapter, we demonstrate the result of our experiments
related to the results of our simulation. Our simulation consist
of two scenarios: normal controller operations and failover
controller operations. The parameters used in our scenario are
failover time, CPU Usage, and throughput to see the
performance comparison of the two methods, 3-in-1 message
exchange, and 1-in-1 message exchange
A. Throughput
Figure 4 shows that throughput performance in our
proposed method shows higher throughput on handling
numerous clients shown in table 1 compared to the previous
method (normal message exchange). When more devices
attached to the network, the throughput performance gets
lower. Our proposed method gives higher throughput than the
previous method because the controller has lower workloads on
processing heartbeat messages in each controller. In the
previous method, every message sent from controller A,
controller B, will send ACK to controller A, resulting in a
message processing bottleneck on each controller.
Figure. 4. Network Topology
B. Failover Time
To demonstrate the failover time experiment in our paper is done
by shutting down Controller A. The failover time compared between
3-in-1 message exchange with normal message exchange (1-in-1
message exchange) between controller A and controller B. Figure 5
shows a graph showing the difference in failover time to switch
migration when controller A goes down. The difference obtained from
the time the switch conditions have been disconnected to controller A
until all the switches are connected to controller B is durable. The
failover time is directly proportional to the number of switches
controlled by the controller. The graph explains that the more
switches, the higher the failover time will be. Our experiment results
show the failover time on the 3-in-1 method is more efficient because
it is faster to be responded to by controller B.

Figure. 5. Network Topology
Our proposed frameworks affect the controller workload,
resulting in faster message synchronization processing because
the 3-in-1 message delivery process is faster by directly sending
three messages and replying to one acknowledgment.
Figure 6. Network Topology
Figure 6 shows that the result obtained when pinging from
host one to host eight during the failover process. The figure
explains that after sequence number 20, the switch
disconnected from controller A, so the ping process has a delay,
and at that time, the switch tries to connect to controller B. This
happens because RabbitMQ as a message broker needs time to
make the switch migration from controller A to controller B.
C. CPU Usage
Figure 7. CPU Usage Graph
CPU Usage testing is done based on two ways the first is
without background traffic, and the second test uses
background traffic. Testing the value of CPU usage aims to
represent network scalability that affects the workload on the
controller. The average comparison result of CPU Usage values
in the two methods is the 3-in-1 method with the one-message
one acknowledgment method. The graph is quite representative
of 4 switches to 32 switches which are increasing. Tests were
carried out ten times to get these results with a testing time of
10 seconds for each switch. The highest percentage value
obtained in the 3-in-1 method is 7.90%, while the highest value
in the 1-in-1 method is 9.57%. The 3-in-1 method obtained a
difference value of 1.67% lower than the 1-in-1 message
acknowledgment method based on these results. So that the
workload on the 3- in-1 method also has a lower workload
compared to the one message, one acknowledgment method.
Sending messages directly is sent three messages and is replied
with one acknowledgment containing these three messages, so
the workload on the controller is reduced. Then, the more
number of switches used, the higher CPU Usage will be due to
the number of switches connected to the controller.
V. C ONCLUSION AND F UTURE W ORKS
Our experiments show that AADC 3 provides higher
throughput, lower CPU usage, and quicker failover time.
AADC3 proposes a 3-in-1 message exchange mechanism
between controller A and controller B which means that
controller A will send three synchronization messages to
controller B, controller B will reply with one acknowledgment
message to controller A. This proposed message exchange on
the distributed SDN controller able to reduce CPU load on
each controller. Suggestions for next research, we plan to
develop adaptive AADC 3. When controller A goes down,
controller B will act as a master controller. When controller A
goes alive again, controller B begins to send three messages to
the neighbor controller (controller A)
REFERENCES
[1]H. G. Ahmed and R. Ramalakshmi, ”Performance Analysis of Cen-
tralized and Distributed SDN Controllers for Load Balancing Appli-
cation,” 2018 2nd International Conference on Trends in Electron- ics
and Informatics (ICOEI), Tirunelveli, 2018, pp. 758-764, doi:
10.1109/ICOEI.2018.8553946.
[2]Othmane Blial, Mouad Ben Mamoun, and Redouane Benaini. 2016.
An Overview on SDN Architectures with Multiple Controllers. J.
Comput. Netw. Commun. 2016, Article 4 (April 2016), 8 pages.
DOI:https://doi.org/10.1155/2016/9396525
[3]M.N.A. Syaehoni, “Desain Distributed Controller dengan Metode Active-
Active pada Jaringan Software Define Network,” Program Studi Sarjana
Informatika Fakultas Informatika Universitas Telkom Bandung, 2019.
[4]E. S. Spalla et al., ”AR2C2: Actively replicated controllers for SDN
resilient control plane,” NOMS 2016 - 2016 IEEE/IFIP Network Oper-
ations and Management Symposium, Istanbul, 2016, pp. 189-196, doi:
10.1109/NOMS.2016.7502812.
 [5]Kshira Sagar Sahoo, Sagarika Mohanty, Mayank Tiwary, Brojo Kishore
Mishra, and Bibhudatta Sahoo. 2016. A Comprehensive Tutorial on
Software Defined Network: The Driving Force for the Future Internet
Technology. In Proceedings of the International Conference on Advances
in Information Communication Technology Computing (AICTC ’16).
Association for Computing Machinery, New York, NY, USA, Article 114,
1–6. DOI:https://doi.org/10.1145/2979779.2983928
[6]Y. Yorozu, M. Hirano, K. Oka, and Y. Tagawa, “Electron spectroscopy
studies on magneto-optical media and plastic substrate interface,” IEEE
Transl. J. Magn. Japan, vol. 2, pp. 740–741, August 1987 [Digests 9th
Annual Conf. Magnetics Japan, p. 301, 1982].
[7]T. F. Oliveira and L. F. Q. Silveria, ”Distributed SDN controllers
optimization for energy saving,” 2019 Fourth International Conference
on Fog and Mobile Edge Computing (FMEC), Rome, Italy, 2019, pp. 86-
89, doi: 10.1109/FMEC.2019.8795343.
[8]Akerele, James. (2018). Performance Analysis of
Openflow-Enabled Network Topologies: Openflow SDN
VS Traditional Network. 10.13140/RG.2.2.33523.84008.
DOI:https://doi.org/10.1155/2016/9396525
[9]T. Kim, T. Koo and E. Paik, ”SDN and NFV benchmarking for performance
and reliability,” 2015 17th Asia-Pacific Network Operations and
Management Symposium (APNOMS), Busan, 2015, pp. 600-603, doi:
10.1109/APNOMS.2015.7275403.
[10]F. Fatturrahman, “SDN Controller Robustness and Distribution Frame-
work,” 2017.
[11]A. Kondel and A. Ganpati, ”Evaluating System Performance for han- dling
scalability challenge in SDN,” 2015 International Conference on Green
Computing and Internet of Things (ICGCIoT), Noida, 2015, pp. 594-597,
doi: 10.1109/ICGCIoT.2015.7380534.
[12]Benamrane, Fouad Ben Mamoun, Mouad Redouane, Benaini. (2017).
New method for controller-to-controller communication in distributed
SDN architecture. International Journal of Communication Networks
and Distributed Systems. 19. 357. 10.1504/IJCNDS.2017.086493.
[13]O. Ait-Hellal and E. Altman, ”Analysis of TCP Vegas and TCP Reno,”
Proceedings of ICC’97 - International Conference on Commu- nications,
Montreal, Quebec, Canada, 1997, pp. 495-499 vol.1, doi:
10.1109/ICC.1997.605357.