Proceedings of the 23rd International Conference on
Automation & Computing, University of Huddersfield,
Huddersfield, UK, 7-8 September 2017
PID Controller Tuning Using Multi-Objective
Ant Colony Optimization applied to TCP/AQM
Networks
Samira Chebli1, Ahmed Elakkary2, Nacer Sefiani3
1,2,3
Laboratory of Systems Analysis, Information Processing and Integrated Management
Mohammadia School of Engineers, Mohammed V University of Rabat
Sale, Morocco
1
samira.chebli@gmail.com, 2aelakkary@gmail.com, 3nasefiani@gmail.com
Abstract— In this paper, we introduce a novel
active queue management scheme based on a Proportional-
Integral-Derivative controller with ant colony optimization
for the Internet system. The ACO algorithm searching
technique is applied to search for the best gain parameters of
the PID controller among a stability area. This stability
region is determined using an extension of Hermite-Biehler
theorem applied to delayed systems. The step responses of
closed loop system of the PID controller with the ACO
algorithm are compared with Genetic Algorithm (GA) and
Ziegler-Nichols (ZN) methods. Simulation results show the
performance and effectiveness of using ACO algorithm for
PID tuning.
Keywords —TCP; AQM; MultiObjective Optimization; Ant
Colony Optimization; PID; Delay Systems;
I. INTRODUCTION
The public internet is a computer network, that is, a
network that interconnects computers around the world
using a common software protocol for transmitting and
receiving data. This protocol is known as TCP/IP. TCP’s
congestion control algorithm has been described as the
largest human made feedback controlled system in the
world [1].
Congestion is one of the most important problems
encountered in communication networks. It occurs when a
link or node carries so much data that its quality of service
deteriorates. This results in queue delay, packet loss or
blocking of new connections, resulting in low throughput
and possibly congestion collapse. Most research in this area
try to avoid these situations of congestion. The introduction
of the concept of AQM provides a solution to the problem
in which routers actively drop packets from queues which
in used to keep the packet queue size correct. Recently,
various AQM schemes have been proposed in the literature
to increase network utilization by regulating queues at the
bottleneck links in TCP/IP networks [2], [3].
The automatic control technique that has received more
attention is the PID controller. It is widely used in various
engineering applications. Despite its simplicity, it can
successfully solve a variety of complex problems. But it is
often difficult to find optimal PID parameters with
traditional techniques. Several new methods have been
developed recently to tune the parameters of the PID
controllers such as Genetic Algorithms and ant colony
optimization.
The paper is organized as follows: the second section is
dedicated to the presentation of the TCP/AQM problem
addressed in this study. In the third section we explain the
PID tuning approach. The fourth section is about the Ant
Colony Optimization (ACO) technique. Section five
tackles the Multi-objective Optimization concept and the
design of a multi-objective ant colony optimization
implemented to the tuned PID controller. Finally, we
provide some simulations to show the performance of the
proposed method.
II. PROBLEM STATEMENT
Internet is comprised by the interconnection of highly
heterogeneous, complex and dynamic modules and the
dominant transport protocol for the Internet is TCP, It
adopts the end-to-end windows-based flow control to avoid
congestion.
The main property of TCP is to guarantee reliable
communication on the Internet. Congestion occurs when a
link or node is carrying so much data that its quality of
service deteriorates. To correct the problems with tail-drop
buffer management, Floyd and Jacobsen introduced the
concept of AQM, in which the routing nodes use a more
sophisticated algorithm called RED to manage their queues
[4]. Different algorithms was proposed and introduced into
the Internet routers such as BLUE, REM or AVQ [2], [5].
We design a congestion control mechanism based on
Active Queue Management (AQM) techniques by using
control theory. In order to use control theory, we propose
to introduce a mathematic model of the TCP behavior
which has been developed by Misra et al, and which has
been simplified by Hollot et al [6], [7].
This dynamic model of TCP/AQM is based on fluid-
flow and stochastic differential equations while ignoring
the TCP timeout and the slow-start mechanism. The non-
linear model used for the study is given by the following
two coupled non-linear differential equations:
1 ( ). − ( )
( )= − − ( ) (1)
( ) 2 − ( )
( )
( )= ( ) − (2)
( )
 where w(t) denotes the average of TCP windows size
(packets), q(t) the average of queue length
( )
(packets), ( ) = + the round-trip time (sec),
the propagation delay, C the router’s transmission capacity,
N(t) the number of TCP sessions. ( ) and ( ) denote
respectively the time-derivative of ( ) and ( ).
Most systems in the real world are nonlinear, then the
analysis of certain required properties and design
controllers become difficult, if not impossible. For this
reason the linearization of those models using some
approximation become a relevant step. Taking (w, q) as the
state and p as input, the equations linearization around an
operating point gives a two-order model with delay that is
given by the following transfer function [8-10]:
( )= (3)
2
+ + +2+
Generally, industrial process are modelled as first order
system, which is comparatively easy to analysis:
G( ) = (4)
+1
where
= is the state gain,
= + − + 2 is the constant time,
and
= +2 − is the time delay of the plant.
III. PID CONTROLLER APPROACH
The PID controller is the most widely used controller in
the industry, covering more than 90% of the industrial need
in control. This predominance is due to its simplicity and
efficiency. The system introduced in the previous section
forms part of the delay systems. This described delay
presents both a physical reality and makes it possible to
describe a part of the dynamics of the system in a simpler
way than by using a higher order. The analysis of the
stability of delayed systems is not a trivial task, this
complexity is due to the infinite number of roots of the
characteristic equation of the system also known as quasi-
polynomial.
Recently, an extension of the Hermite-Biehler theorem
applied to quasi-polynomials adopted by Silva et al [11-13]
helped to solve this problem. So we chose it as the basis of
the approach used in this article to tune the PID controller.
This is an analytical approach used to find Hurwitz stable
PID gains. Based on the root interlacing property, this
extension of the Hermite-Biehler theorem consists
essentially in finding a number of roots of the quasi-
polynomial having a negative real part. It should be noted
that in this study the exponential term in the characteristic
equation of the transfer function of the system has not been
approximated, which preserves to the system a number of
its properties. In this manuscript, we address the
determination of the whole stability region of the variables
of the PID regulator for the first-order delay system. This
step is essential for any design and tuning of PID
controllers. In the course of numerous research studies
dedicated to the PID regulator, several configurations have
been assigned to it. The most common of these is the closed
loop SISO [14] which is shown in Figure 1.
( ) ( )
C(s): PID
u( ) ( )
+ G(s): Plant
- Controller
Fig.1. Feedback control system.
where , , and are respectively the desired, error,
command and output signals, C(s) is the PID controller and
G(s) is the plant to be controlled.
The control law of the PID regulator can be written in
Laplace transform as:
( )
( )= = + + (5)
( )
where , and are respectively the proportional,
integral and derivative gains of the PID controller.
IV. ANT COLONY OPTIMIZATION
The Ant Colony Optimization (ACO) came into being
as a result of the ant behavior in nature when searching for
food. These social insects use, to communicate, a volatile
substance called pheromone, which they use to mark their
route between the nest and the food.
Fig.2. Ants converge to shortest path from their nest to the food.
In fact, as illustrated in Figure 2, the ants performing the
minimum time on the way back and forth between the nest
and the food are the ones that have taken the shortest route.
This path has a higher concentration of pheromone and is
more attractive to ants and therefore has a higher
probability of being borrowed. This same track will be
more reinforced than the others, and ultimately will be
chosen by the vast majority of ants. From this observation
[15], the ACO was formalized; it is a meta-heuristic
optimization method. In fact, the Travelling Salesman
Problem (TSP) was the subject of the first implementation
of ACO: Ant System (AS) [16]. The basic algorithm is
divided into three essential phases.
• Initialization
• Constructing ant solution
• Updating pheromone
A. Initialization:
The problem is to find the shortest Hamiltonian cycle in
a graph, where each vertex of the graph represents a city.
The distance between cities and is represented by ,
and the pair ( , ) represents the edge between these two
cities. We first initialize the quantity of pheromone on
edges with , each ant traverses the graph and constructs
a complete path (a solution).
B. Constructing ant solution
At each stage of the construction of the solutions, the
ant must decide at which point it will move, this decision is
taken in a probabilistic way based on the values of
pheromone and a statistical information which allows in
particular to find a good solution.
The probability that an ant moves from vertex to
vertex , which belongs to a set of vertices that are not yet
visited by the ant denoted by , is [17]:
( ( )) . ( )
= (6)
∑ ∈ ( ( )) . ( )
and are two parameters that influence the importance
of the pheromone intensity , and the statistical
information called visibility . This value guides the
choice of ants to nearby towns and avoids those that are
too far away ( = ). For = 0, one takes into account
just the visibility, that is to say that the choice will have
fallen each time on the nearest city. If = 0, only the
pheromone tracks play on the choice. To avoid too fast
selection of a path, a suitable compromise between these
two parameters is mandatory.
C. Updating pheromone
When all the ants have constructed a solution, an
amount of pheromones ∆ is deposited by each ant on
its path. For any iteration , if the path ( , ) is in the round
of the ant the quantity of pheromones deposited on this
path is:
( )=
∆ (7)
( )
where ( ) is the total length of the ant tour , and is a
constant.
So the addition of the amount of pheromones certainly
depends on the quality of the solution obtained, i.e. the
smaller the number of pheromones, the greater the number
of pheromones.
In order not to neglect all the bad solutions obtained,
and thus avoid convergence towards local optima of poor
quality, the concept of evaporation of pheromone tracks is
simulated through a parameter called the evaporation rate
0 < < 1 as follows:
( + 1) ← (1 − ) ( ) + ∆ ( ) (8)
where ∆ ( ) = ∑ ∆ ( ), represents a given
iteration and the number of ants.
Then, many variants have been proposed [15], [18-20]
The first is characterized by the introduction of elitist ants.
In this approach, the ant that has made the shortest route
deposits a larger amount of pheromone in order to increase
the likelihood of other ants to explore the most promising
solution.
V. MULTI-OBJECTIVE ACO OPTIMIZATION
Once the gain stability region of the PID controller is
determined, it will be pertinent to search within this region
for the optimal values of these parameters. However, the
problem to optimize in this manuscript is not reduced to a
single cost function, but to several.
The dynamic performance of a closed-loop system is
generally assessed in terms of stability, precision,
sensitivity and transient response, to name but a few.
Recent control systems require more sophisticated
performance criteria. This second category of criteria takes
into account simultaneously the error and the time during
which it occurred. The importance of this approach has
been amply demonstrated in the literature.
To cover broad aspects of the dynamics of the system,
we will consider several performance criteria that are useful
in designing linear control systems. These criteria are
usually conflicting, which reflects the multi-objectivity of
the problem announced at the beginning of the section.
* The IAE criterion (Integrated-Absolute Error) is a
performance criterion of a common utility. Indeed, when
the absolute value of the error increases, I1 increases which
results in an underdamped system:
1 = | ( )| (9)
* The ISE criterion (Integrated-Squared Error), on the
other hand, penalizes the positive and negative values of the
error by considering the square of the error:
2 = ( ) (10)
* The ITSE criterion (Integrated-Time-Squared Error)
also considers the square of the error but weighted by time,
it gives less importance to the initial errors than the
previous indices:
3 = ( ) (11)
* The ITAE criterion (Integrated-Time-Absolute Error)
is generally considered to be more selective than other
performance indices, it acts on long-duration transients by
penalizing it, and focuses more on the error present in the
system than on the initial error:
4 = | ( )| (12)
0
Since ACO algorithm is a metaheuristic approach, that
is to say it is dedicated to problems of difficult optimization
[21], it therefore makes a very judicious choice to solve the
problem described above. The PID-MOACO controller is
therefore proposed to optimize the cost functions and to
deduce the optimum gains of the PID regulator derived
from the region of stability that will be determined
beforehand. This optimization method applied to the AQM
congestion problem is shown in Figure 3.
 Fig.3. Flowchart of Ant Colony Optimization for proportional-integral-derivative tuning.

VI. SIMULATION RESULTS AND ANALYSIS

In this section, we test the performance of the closed-
loop system with the ACO-PID controller by simulation,
and we compare the results with those obtained by the
application of Ziegler-Nichols (ZN) method and Genetic
Algorithm (GA) Optimization.
The ZN tuning technique is a classical method based on
a step response experiment combined with a table that
relates the controller parameters to the characteristics of
the step response. It has been opted for this method to be
compared with for its simplicity and efficiency [22]. As for
the GA optimization method, it is considered as a useful
optimization method employing the principles of natural
genetic systems [13] to seek a global solution of the
optimization problem. GAs are stochastic optimization
methods that sweep the entire admissible space to search
the optimal solution. It has been chosen in this comparison Fig.4. The stabilizing region of (K , K , K ) for the PID controller in the
for its similarity to the ACO technique. congestion control
For a TCP/AQM network modelled by (4), it is assumed The stability region in 3-dimensional plot for the
that N = 60 homogeneous TCP connections and shares one variables , and of the PID controller is given in
bottleneck link with a link capacity C = 1250 Figure 4. PID Controller optimum parameters supplied by
(packets/second). Furthermore the round trip time R = ACO, GA optimization and Ziegler-Nichols are summarized in
400s, the desired queue size = 150 packets, and the Table I.
buffer size = 200 packets. Then we deduce the values
To evaluate the performance of the studied process in
of the state gain K, the constant time T and the time delay
the time domain, a signal test is used which can be pulse,
L of the plant:
ramp or step. But the latter is the most widely used because
= 234375; = 5.0319 and = 0.7681 it provides very useful information about the dynamics of
the system. The Figures 58 present the step response for
the different optimal PID controllers. From Table II, it
appears that the ZN method gives poor results compared
to other random search methods for the different objective
functions, especially for the overshoot (91.3046%), which
induces strong oscillations and thus affects the stability of
the system. The performance criteria IAE, ISE and ITSE
optimized by ACO gives good results but are very close to
those given by GA. However, the ITAE, which is the most
sensitive criterion of all four, gives better values when
optimized by ACO than by GA. It should be noted that the
ITAE is very useful for practical applications which
require a sensitive criterion as is the case for the AQM
congestion control delay system presented in its
manuscript.
TABLE I.

OPTIMUM PARAMETERS OF PID CONTROLLER

Fig.6. ISE Step Response
Controller Performances
Criteria ( ) ( ) ( )
IAE 0.1892 0.0193 0.0801

ISE 0.2623 0.1115 0.1094

ACO-PID
ITAE 0.2126 0.0397 0.0748

ITSE 0.2828 0.0411 0.0463

IAE 0.2269 0.0422 0.0747

ISECONCLUSION
0.2382 0.0551 0.1051
GA-PID
ITAE 0.2210 0.0415 0.0618

ITSE 0.2396 0.0475 0.0907

ZN-PID - 0.3354 0.2183 0.1288

Fig.7. ITSE Step Response

Fig.5. IAE Step Response Fig.8. ITAE Step Response

 TABLE II. [5] V. Misra, W. Gong, and D Towsley. Fluid-based analysis of
a network of AQM routers supporting TCP flows with an
Numerical values of standard performance measures
application to RED. In ACM SIGCOMM, pages 151–160,
August 2000.
Controller Performances Rise Settling Overshoot [6] [6] V. Misra, W. Gong, and D. Towsley. Stochastic
Criteria Time (s) Time (s) (%) differential equation modeling and analysis of TCP
windowsize behavior. Technical Report ECE-TR-CCS-99-
IAE 0.6719 18.7284 0 10-01, University of Massachusetts, October 1999.
[7] Hollot, C. V., Misra, V., Towsley, D., & Gong, W. B.
ISE 0.334 6.2594 45.3385 (2001). A control theoretic analysis of RED. In INFOCOM
ACO- 2001. Twentieth Annual Joint Conference of the IEEE
PID ITAE 0.6732 2.0865 2.1571 Computer and Communications Societies. Proceedings.
IEEE (Vol. 3, pp. 1510-1519). IEEE.
ITSE 0.5440 6.9084 27.3051 [8] S. Chebli et al, New PI Stabilization of First-Order
Congestion Control of Active Queue Management Routers,
IAE 0.5589 3.1935 10.6662 in Proceeding of Electrical and Information Technologies
(ICEIT) Conference, pp.12 – 217, Marrakech, Morocco,
March, 2015.
ISE 0.3983 5.8501 28.4493
[9] S. Chebli et al, “PI Stabilization for Congestion Control of
GA-PID AQM routers with Tuning Parameter Optimization”
ITAE 0.6297 2.1873 4.1763 International Journal of Interactive Multimedia and
Artificial Intelligence, Vol. 4, Nº1 (2016), p. 52-55.
ITSE 0.4611 3.7296 21.9698 [10] S. Chebli et al, Optimisation par Algorithme Génétique
d’un Régulateur PI pour le Contrôle de Congestion d’un
ZN-PID - 0.1981 6.9528 91.3046 Routeur”. La 3ème Edition de la Journée Scientifique
d’Analyse des Systèmes et Traitement de l’Information
(JASTI 3) (2016), p. 25-30.
[11] G.J. Silva, A. Datta, S.P. Bhattacharyya, PID controllers for
CONCLUSION time delay systems, Springer, London, 2005.
In this work, an efficient and effective tuning approach [12] G. J. Silva, A. Datta and S. P. Bhattacharyya, Stabilization
of First-order Systems with Time Delay using the PID
based on the multi-objective ant colony optimization is controller, Proceedings of the American Control
developed for getting good performances and optimizing Conference, pp. 4650-4655, 2001.
the PID parameters. [13] S. Chebli et al, “Multi-Objective Genetic Algorithm
Optimization using PID Controller for AQM-TCP
The results of the simulations show that the proposed Networks” International Review of Automatic Control
method was more flexible and efficient in terms of dynamic (I.RE.A.CO.), Vol. 10, N. 1, (2017), pp. 33-39.
performance such as reduction of maximum overshoot, rise [14] M. Milosawlewitsch-Aliaga et al, “Model-based iterative
feedback tuning for industrial PID controllers” in Journal of
time, and settling time. The MOACO was capable of Scientific and Industrial Research, (2010) 69 (12), pp. 930
undertaking local research with a rapid rate of convergence. 936.
Thus, the proposed approach has proved its effectiveness [15] M. DORIGO et al, “The Ant System : Optimization by a
through simulation results. To highlight more the efficiency Colony of Cooperating Agents”, IEEE Transactions on
of the MOACO method implemented in PID controller, the Systems, Man, and Cybernetics – Part B, vol. 26, no 1, 1996,
p. 29–41.
comparative study conducted between the proposed
[16] A. Colorni et al, "Distributed optimization by ant colonies."
approach and Genetic Algorithm and Ziegler-Nichols In Toward a Practice of Autonomous Systems: Proceedings
techniques reinforces the relevance of the method of the First European Conference on Artificial Life, p. 134.
addressed in this manuscript. Mit Press, 1992.
[17] M. Abolhasan, T. Wysocki, E. Dutkiewicz, A review of
routing protocols for mobile ad hoc networks, Ad Hoc
Networks 2 (1) (2004) 1_22.
REFERENCES [18] Dorigo, Marco, and L. M. Gambardella. "Ant-Q: A
[1] V. Jacobson, “Congestion avoidance and control,” in reinforcement learning approach to the traveling salesman
Proceedings of ACM SIGCOMM, Stanford, CA, August problem." Proceedings of ML-95, Twelfth Intern. Conf. on
1988. Machine Learning. 2016.
[2] D. Chiu and R. Jain, “Analysis of the increase/decrease [19] Dorigo, Marco, and Luca Maria Gambardella. "Ant colony
algorithms for congestion avoidance in computer system: a cooperative learning approach to the traveling
networks,” Journal of Computer Networks and ISDN, vol. salesman problem." IEEE Transactions on evolutionary
17, no. 1, pp. 1–14, June 1989. computation 1.1 (1997): 53-66.
[3] Ohsaki H., Sugiyama K. and Imase M., Congestion [20] Stützle, Thomas, and Holger H. Hoos. "MAX–MIN ant
Propagation among Routers with TCP Flows, International system." Future generation computer systems 16.8 (2000):
Journal of Computer Networks & Communications, vol. 1, 889-914.
no. 2,pp. 112-127. 2009 [21] Saıt, S., and Habib Youssef. "Iterative computer algorithms
[4] H. I. Alshimaa, A. EL-Sayed & Z. Elsaghir, I. Z. Morsi, with applications in engineering." IEEE Comput. Soc.
“Enhanced Random Early Detection (ENRED)” Press (1999).
International Journal of Computer Applications (0975 – [22] Åström, Karl Johan, and Tore Hägglund. "PID controllers:
8887) Vol. 92 – No.9, April 2014. theory, design, and tuning." (1995).