Intelligent Traffic Light Scheduling for Traffic
Control
Rishabh Jain
Computer Science and Engineering
Indian Institute of Technology, Ropar
2019csb1286@iitrpr.ac.in
Abstract—This is a study of various researches in traffic
light scheduling for reducing congestion in a road network. We
evaluate three researches on traffic light scheduling in this paper.
I. I NTRODUCTION
A. Motivation
Traffic lights are road safety devices used to regulate traffic
across road intersections. Often, traffic lights cause significant
delays and congestion in modern cities. The growing number
of traffic lights necessitates a complex scheduling algorithm
for both- the traffic lights and the cars, which is a hard
problem; and hence, automatic systems are indispensable
nowadays for optimally tackling this task.
There is little research on traffic routing algorithms which
counter the issue of scalability, coordination and feasiblity of
acquiring data.
Scalability here means the extensibility of the the solution
to several cities without major degradation in performance.
Coordination here means information sharing between differ-
ent intersection signals, as if all signals work independently,
decision of one may negatively impact the other. A paper by
C.Chen et. al., MPLight [1], ensures all these requirements;
and is discussed in the section.
B. Problem Definition
The problem definition is to develop a reinforcement learn-
ing based solution for intelligent traffic light scheduling in
order to minimize vehicle wait times.
II. R ELATED W ORKS
A. Clustering based Approach
V.Pattanaik et. al [2] Our initial aim was to develop an
algorithm for vehicles, so that they can find optimal routes
in a dynamic and partially observable environment. A crucial
part of our approach was the congestion prediction algorithm.
For this we came across this paper. The algorithm uses K-
Means Clustering on the local surroundings of the vehicle,
to find traffic clusters . To quantify different clusters, traffic
density is calculated with help of convex hull algorithm.
Ashish Sharma
Mathematics and Computing
Indian Institute of Technology, Ropar
2019mcb1213@iitrpr.ac.in
B. Mean Field Game Approach
Theophile Cabannes et. al [3] propose a Mean Field Game
approach for the N vehicle dynamic routing game. First
they model the traffic flow as a mesoscopic model, meaning
individual vehicles are considered but their interactions are
considered with the remaining traffic flow as a whole. The
game is reformulated as a Mean Field Game. The paper
demonstrates the MFG approach efficiently approximates the
N-player dynamic routing game. It is proved in both ap-
proaches Nash Equilibrium(it is characterized by no change in
optimal payoff value of a player irrespective of it changing its
strategy when all other player strategies are fixed) is achieved.
This approach was not taken further due to it not allowing
vehicle flow in FIFO(first in first out) order.
C. MPLight Architecture
Chacha Chen et. al [1] have suggested a scalable
reinforcement learning solution to traffic light scheduling. In
this work, the authors assign observers to each intersection,
which communicate observations to a centralised RL agent
with a Deep-Q network through parameter sharing. The goal
of the RL agent is to minimize the pressure(Section III.B),
or, make vehicle flow more uniform, and hence increase the
throughput of vehicles. This will in turn minimise the traffic
congestion at every intersection. Maintaining independent
observers for all intersections increases the scalability of the
solution.
DQN Agent Description:
The agent observes the pressure across the traffic segments
coming in and going out of the intersections (12 in total),
and the current phase (the traffic light states).
Actions: The agent (traffic signal controller at an intersection)
chooses its actions from the 8 possible phases, denoting what
next state it should go to.
Reward: The reward for each agent (at each intersection say
i) is the negative of the Intersection Pressure:
r = −P i (1)
The DQN outputs the Q-value based on the Bellman Equation:
Q(st , at ) = R(st , at ) + γmax.Q(st+1 , at+1 ) (2)
The coordination happening here can be seen in Fig.1, where
parameters of single network are shared among all agents.

Fig. 3. Signal Phases

IV. F UTURE W ORK

Fig. 1. MPLight Architecture
III. P RELIMINARIES
Following are the definitions of a few terms that will be
frequently discussed in this paper.
A. Model of an intersection
A road intersection is modelled as a way road intersection,
with each road having two lanes, in opposite direction. A
traffic light is present at a lane which directs traffic into the
intersection. Refer Figure 2.
In the future, integration with the CityFlow simulation
platform has to be implemented. Along with that, possible
improvements in DQ Network will be explored.
Further, as per feedback, methods will be implemented to
make way for Emergency Vehicles in intelligent traffic light
routing environment.
R EFERENCES
[1] C. Chen, H. Wei, N. Xu, G. Zheng, M. Yang, Y. Xiong,
K. Xu, and Z. Li, “Toward a thousand lights: Decentralized
deep reinforcement learning for large-scale traffic signal control,”
Proceedings of the AAAI Conference on Artificial Intelligence,
vol. 34, no. 04, pp. 3414–3421, Apr. 2020. [Online]. Available:
https://ojs.aaai.org/index.php/AAAI/article/view/5744
[2] V. Pattanaik, M. Singh, P. Gupta, and S. Singh, “Smart real-time traffic
congestion estimation and clustering technique for urban vehicular roads,”
in 2016 IEEE Region 10 Conference (TENCON), 2016, pp. 3420–3423.
[3] T. Cabannes, M. Lauriere, J. Perolat, R. Marinier, S. Girgin, S. Perrin,
O. Pietquin, A. M. Bayen, E. Goubault, and R. Elie, “Solving n-player
dynamic routing games with congestion: a mean field approach,” 2021.
[Online]. Available: https://arxiv.org/abs/2110.11943

Fig. 2. Intersection

B. Pressure at an intersection
The definition of pressure, given by C. Chen et. al. [1] is
described as follows. Let x(l, m) denote the difference of the
number of vehicles entering into road l and exiting road m at
intersection i. The pressure of an intersection is defined as
X
Pi = x(l, m) (3)
∀(l,m)∈s

where s in the set of valid signal phases

C. Signal Phases
A traffic signal phase s is defined as a set of permissible
traffic movements. There are 8 possible phases for a 4 way
intersection. Refer Figure 3.