Renewable and Sustainable Energy Reviews 188 (2023) 113873

Contents lists available at ScienceDirect

Renewable and Sustainable Energy Reviews

journal homepage: www.elsevier.com/locate/rser

Review article

Electric vehicle charging service operations: A review of machine learning

applications for infrastructure planning, control, pricing and routing
Nilgun Fescioglu-Unver ∗, Melike Yıldız Aktaş
Department of Industrial Engineering, TOBB University of Economics and Technology, Ankara, Turkey

ARTICLE INFO ABSTRACT

Keywords: The majority of global road transportation emissions come from passenger and freight vehicles. Electric vehicles
Electric vehicle (EV) provide a sustainable transportation way, but customers’ charging service related concerns affect the
Charging service operations EV adoption rate. Effective infrastructure planning, charge scheduling, charge pricing, and electric vehicle
Machine learning
routing strategies can help relieve customer perceived risks. The number of studies using machine learning
Infrastructure planning
algorithms to solve these problems is increasing daily. Forecasting, clustering, and reinforcement based models
Charge scheduling
Pricing
are frequently the main solution methods or provide valuable inputs to other solution procedures. This study
Routing reviews the studies that apply machine learning models to improve EV charging service operations and provides
future research directions.

1. Introduction
Road transportation carbon emissions accounted for 16% of global
emissions in 2022 [1]. Passenger and freight vehicles are responsible
for the majority of this emission. Electric vehicles (EV) provide a
sustainable transportation way, and ensuring a widespread deployment
requires overcoming the adoption barriers.
Various factors affect EV adoption, including economic factors such
as EV purchase/operation/maintenance and charging cost, charging
infrastructure readiness (i.e., station availability, range anxiety), con-
sumer psychological aspects, government policies, vehicle performance,
and environmental perspectives [2]. Especially charging related con-
cerns are among the highest perceived risks of EV adoption [3]. Charg-
ing takes place at homes, offices, publicly accessible places like malls,
and charging stations on roads. Surveys show that home charging and
office charging are the most used places for charging, and around a
total of 95% of charging events occur at these locations. Consumers are
mostly concerned about charging location and charging costs. Public
charging is the least used infrastructure type; however, it is essential
for long journeys and is seen as a safety net. [4].
Charging duration and charging station density influence
consumers’ public charging infrastructure evaluation significantly [5].
The time spent searching for a public charging station and waiting
at the station is one of the highest perceived risks of electric vehicle
adoption [3]. Fast charging can effectively reduce range anxiety [6].
Fast charging infrastructure expansion is required for the widespread
adoption of electric vehicles [7–9] and is a more cost efficient solution
when compared with increasing battery capacities [10]. In the case of
fast charging stations, while users are accepting to make detours, they
strongly reject waiting times at the station [11].
When charging is completed at home or the office, the most critical
concerns are the charging price related to the time of charge and
getting the required state of charge (SoC) before the departure time.
In this case, charge control and scheduling models aid EV users charge
their vehicles at the right time and price. When consumers use public
charging infrastructure, the most critical concerns become locating a
nearby available station, time spent while driving to and waiting at
the station, and the charging price. Adequate infrastructure planning,
dynamic pricing, and routing models improve public charging service
operations.
There are several review studies on electric vehicle charging stations
and charging services. Review topics include charging infrastructure
control and impact on grid [12,13], smart charging [14], station lo-
cation and sizing optimization [15,16], battery swapping stations [17],
pricing [18], electric vehicle routing [19] and the optimization studies
on electric vehicle service operations [20].
Machine learning models are frequently applied to electric vehicle
charging related problems. For the smart grid, machine learning helps
grid power quality and condition monitoring, load forecasting and bal-
ancing [21,22], renewable energy, energy storage system integration,
and security [23]. Hybrid electric vehicle (HEV) energy management
strategies also use machine learning to control HEV’s different en-
ergy sources [24]. Data repositories provide machine learning models
with real-world data about EV market growth, infrastructure, user
behavior [25], and electric vehicle load [26].

∗ Corresponding author.
E-mail addresses: nfunver@etu.edu.tr (N. Fescioglu-Unver), melikeyildiz95@gmail.com (M. Yıldız Aktaş).

https://doi.org/10.1016/j.rser.2023.113873
Received 10 March 2023; Received in revised form 2 October 2023; Accepted 9 October 2023
Available online 17 October 2023
1364-0321/© 2023 Elsevier Ltd. All rights reserved.
N. Fescioglu-Unver and M. Yıldız Aktaş Renewable and Sustainable Energy Reviews 188 (2023) 113873

optimization problems. Several review studies investigate the related

Abbreviations studies from different perspectives. Al-Ogaili et al. [27] review the
AC Alternate Current EV charge control strategies and focus on the studies using schedul-
ANN Artificial Neural Networks ing, clustering, and forecasting methods. They cover mathematical
programming and metaheuristic charge scheduling approaches, clus-
BEV Battery Electric Vehicles
tering methods for load profiling, and forecast studies for short/long
BPNN Back Propagation Neural Network
term load forecasting, published before 2020. Abdullah et al.’s review
CART Classification and Regression Trees study [28] concentrates on reinforcement learning (RL) based EV charg-
CFSFDP Clustering by Fast Search-and-Find of Den- ing coordination literature. They list the reinforcement learning specific
sity Peaks parameters such as the state, reward (objective), and constraints, and
CHAID Chi-square Automatic Interaction Detector the specific RL algorithm type each study uses. Deb [29] review the
CS Charging Station studies that use machine learning approaches for charging demand
DBSCAN Density-Based Spatial Clustering of Appli- prediction, scheduling, charging station placement, and charger uti-
cations with Noise lization prediction. In another review, Shahriar et al. [30] focus on
DC Direct Current predicting charging behavior such as EV arrival/departure times, charg-
DDPG Deep Deterministic Policy Gradient ing duration, and energy consumption and review the studies using
DDQN Double Deep Q-Network supervised/unsupervised machine learning methods. To the best of
our knowledge, there are no review studies investigating the machine
DEVRP Dynamic Electric Vehicle Routing Problem
learning applications in dynamic pricing and EV routing.
DNN Deep Neural Network
EV charging service related topics are interrelated with each other.
DQL Deep Q-Learning The infrastructure plan provides inputs to charge control/scheduling
DQN Deep Q-Network models, charge coordination requires dynamic pricing information and
DRL Deep Reinforcement Learning the outputs of these layers are inputs to the routing algorithms. Gov-
ER-EV Extended Range EVs ernment policies on electric vehicle support and renewable energy
EV Electric Vehicle generation affect the energy demand and supply. Policymakers and
EVCS Electric Vehicle Charging Station Selection engineers can use machine learning algorithms to obtain valuable in-
Problem formation in all EV charging service operations. This study differs from
EVEC Electric Vehicle Energy Consumption Prob- the existing ones with its focus and coverage. The key contributions of
lem this study are:
EVRP Electric Vehicle Routing Problem • Focusing on charging service related topics that mostly influence
EVSC EV Smart Charging the EV user’s EV adoption decisions.
FCEV Fuel Cell Electric Vehicles • Covering a broad range of EV charging service related areas,
FCLM Flow Capturing Location Model including; Infrastructure planning, charge control and scheduling,
GB Gradient Boosting pricing, and routing.
GRU Gated Recurrent Unit • Covering studies that use machine learning algorithms both as
HEV Hybrid Electric Vehicle the primary solution method and as an aid providing input to the
main method.
kNN k-Nearest Neighbor
• Including the most recent literature in its coverage area.
LSTM Long Short Term Memory
• Highlighting the findings and future research directions for each
MCLM Maximal Covering Location Model
charging service topic.
NN Neural Networks
PCA Principal Component Analysis This paper is organized as follows. Section 2 introduces the electric
PHEV Plug-in Hybrid Electric Vehicles vehicle types and charging modes. Section 3 starts with infrastructure
planning and covers the studies that use machine learning for station
PPO Proximal Policy Optimization
placement and demand/charging station (CS) utilization forecasting.
QoS Quality of Service
Next, Section 4 reviews the charge control and schedule literature from
RF Random Forest
the aggregator and user perspectives. Sections 5 and 6 review the ma-
RL Reinforcement Learning chine learning based dynamic pricing and routing studies, respectively.
SAC Soft Actor Critic Finally Section 7 summarizes the findings and Section 8 concludes the
SARSA State Action Reward State Action study.
SoC State of Charge
SVR Support Vector Regressions 2. Electric vehicle charging
TOU Time of Use
V2B Vehicle to Building Electric vehicles can be classified as battery electric vehicles (BEV),
plug-in hybrid electric vehicles (PHEV), hybrid electric vehicles (HEV),
V2G Vehicle to Grid
fuel cell electric vehicles (FCEVs), and extended range EVs (ER-EV),
V2H Vehicle to Home
which are essentially BEVs equipped with a combustion engine to
V2V Vehicle to Vehicle charge the battery when needed. PHEVs and HEVs include both an
V2X Vehicle to Everything electric engine and an internal combustion engine. PHEVs’ electric
VRP Vehicle Routing Problem engine can be charged by an external electric source (by plugging in the
main grid) whereas the HEVs are charged by the internal combustion
engine’s energy [31,32].
Electric vehicle batteries can be charged by conductive and wireless
Looking from the EV users’ perspective, charging service location, charging methods. Conductive charging is the most widely used method
duration, and cost are important factors in EV adoption decisions. Ma- and is classified into three types according to the power transfer levels
chine learning algorithms also aid in service operations planning and — Level 1, Level 2, and Level 3. Level 1 charging is often called slow

2
N. Fescioglu-Unver and M. Yıldız Aktaş
charging and is primarily used at homes, where the vehicles connect
directly to the electrical outlet. Since the power transfer rate is low
(AC single-phase 3.3 kW), the battery charging time required for the
vehicle’s 100 km journey reaches an average of 6-8 h. Level 2 charging
enables faster charging through a charging unit. Level-2 charging is
primarily available at public long term parking locations like shopping
centers, hotels, airports, and workplace parking lots, and EV owners
can also install these chargers in their homes. Depending on the power
level (AC single-phase 7.4 kW, AC three-phase 22 kW), the charging
time for 100 km varies from 4 h to 1 h [33].
Level 3 charging (also called fast or DC charging) decreases the
charging time for a 100 km road to 20–30 min with a 50 kW power
DC charging unit. With DC Rapid (150 kW) and DC Ultra-Fast (350 kW)
charging modes, the charging time will decrease to 10 min at dedicated
charging stations. Level 3 fast charging stations generally meet the
charge needs on long highway corridors. Fast charging is the model
that most closely resembles the refueling process and suits the current
behavioral pattern of vehicle owners. Therefore, locating fast charging
stations in urban areas to meet the daily charging needs of EV users
can improve the EV transition rate [33].
Wireless charging allows the vehicle to be charged without a direct
connection between the vehicle and the grid. During wireless charging,
electricity transfers from the transmitter pads placed on the ground to
the receiver pads placed under the vehicle. Wireless charging technolo-
gies can be classified according to electricity transfer methods or the
transmitted distance (near field–far field). Near-field charging methods
include inductive, magnetic resonance, and capacitative charging. Far
field methods include microwave, radio wave, and laser technologies,
and are still in the prototype stage [34]. In addition, wireless charging
can be static or dynamic. In the static case, the vehicle is stationary,
whereas in the dynamic case, the vehicle is charged en-route while
in motion [35]. Vehicle to grid (V2G) technology is also applicable to
wireless charging [36].
Other electric vehicle charging related technologies include vehicle
to everything (V2X), battery swapping, and mobile charging. Vehicle
to everything (V2X) concept, which connects vehicles to the grid, to
homes, and other vehicles, enables using the energy in EV batteries
outside the vehicles [37,38]. The control of charging can be coordi-
nated by unidirectional charging, such as smart charging, including
V1G, or bidirectional power flows like vehicle-to-grid (V2G), vehicle-
to-home (V2H), vehicle-to-building (V2B) or vehicle-to-vehicle (V2V)
as in Fig. 1. In a V2G setting, the grid operators benefit from frequency
regulation, voltage support, and spinning reserves in case of unplanned
events. The EV owner benefits from the energy arbitrage by buying
energy at low priced times and selling it back to the grid at high priced
peak usage times. In a V2H setting, the EV acts as a backup power
storage during blackouts and aids household load management during
day time. V2B is a setting similar to V2H where the vehicle acts as a
storage, but in this case, the vehicle owners can receive a payment from
the building operators. In a V2V setting, the vehicle acts as a mobile
charging unit, mainly to rescue vehicles on highways [38].
Battery swapping enables drivers to replace the discharged batteries
with a full battery within a few minutes in the battery swap stations
and eliminates the charging wait time [17]. Mobile charging enables
EV users to charge their vehicles at their preferred time and location.
A mobile charging station can be a truck that carries a charging unit to
connect the EVs to the main grid, can have a separate battery to charge
the EVs, or can act as a mobile battery swap station. Vehicle to vehicle
(V2V) power transfer is another type of mobile charging [39].
3. Infrastructure planning
The electric vehicle adoption rate growth highly depends on the
electric vehicle charging infrastructure expansion. Building an ade-
quate infrastructure is an expensive process with several constraints.
Infrastructure planning problem includes selecting the location and
3
Renewable and Sustainable Energy Reviews 188 (2023) 113873
sizing of charging stations. Deb et al. [40] classify the charging sta-
tion placement studies into three groups according to their modeling
considerations: Models considering the distribution network, trans-
portation network, or both. In the distribution network approach,
the main concern is the power system’s benefits and costs. In the
transportation network approach, the main concern is optimal CS dis-
tribution in the road network. The transportation network approach
is divided into node-based, tour-based, and path-based approaches.
Node-based approaches model the CS placement problem as a facility
location problem where the facilities are located at nodes to satisfy
the nodes’ demand. Node-based solution methods include P-median
based approaches and set covering models. Path-based approaches
concentrate on the vehicle flow along the paths and require traffic
flow data. Tour-based approaches include the round-trip possibilities
to the problem. Path-based solutions frequently use flow capturing
location model (FCLM) and maximal covering location model (MCLM)
variations. Metais et al. [41] further classify the infrastructure planning
studies with respect to their coverage as location, sizing, or both. Sizing
studies can optimize the number of vehicles that can be accommodated
or the station’s power level.
Accurate charging demand and station availability/utilization fore-
casting improve the infrastructure planning process outcomes. Machine
learning models help the infrastructure planning problem by improving
the charging station placement and sizing heuristics and the prediction
of long/short term charging demand and charger utilization. Fig. 2
summarizes the infrastructure planning sub-problems machine learning
algorithms are frequently applied.
This section investigates the infrastructure related studies in two
sub-sections. Section 3.1 includes studies using machine learning as
part of the station placement/sizing problem solution. Section 3.2
investigates long/short term charging demand and station utilization
forecasting studies.
3.1. Charging station placement and sizing optimization
Charging station placement and sizing studies determine optimal
locations and capacities for charging stations. Several studies use clus-
tering algorithms to determine the candidate locations and select the
station locations/sizes according to an EV user specific, or CS/grid op-
erator specific objective, or a combination. Examples of EV user specific
objectives include minimizing the distance to the station, minimizing
queuing time, or maximizing the demand coverage. CS or grid operator
specific objectives include maximizing profit, minimizing power loss,
voltage fluctuation, load fluctuation, and minimizing cost. Cost items
include operation, construction, and maintenance costs. Some studies
combine EV user and CS operator benefits in a single objective or use
multiple objectives. Infrastructure planning studies frequently use K-
means and hierarchical clustering algorithms. K-means is a partition
based clustering algorithm that iteratively update the cluster centers
until convergence. It has high computing efficiency but is sensitive
to the preset number of clusters and outliers. Hierarchical clustering
is based on forming a hierarchical relationship between datasets. It
has higher time complexity but is suitable for data sets with arbitrary
shapes [42].
K-means algorithm and its variations are frequently used as a clus-
tering model for location determination. Andrenacci et al. [43] develop
a fuzzy K-means clustering approach to determine the candidate loca-
tions and estimate the energy demand of each cluster through vehicle
trajectory analysis. Karezmi et al. [44] integrate K-means clustering
with Genetic Algorithm to determine the zone borders while maximiz-
ing the distributor’s profit and considering EV users’ welfare. Wang
et al. [45] plan charging stations and scattered piles for electric vehicle
sharing systems. They use K-means clustering to determine candidate
station locations and mean shift clustering algorithm for planning scat-
tered pile locations. Their objective is to minimize the total cost, which
includes the costs related to queueing time, operation, construction,
N. Fescioglu-Unver and M. Yıldız Aktaş
Fig. 1. Vehicle to everything.
Fig. 2. Infrastructure planning.
and maintenance. Marino and Marufuzzaman [46] determine potential
CS locations by principal component analysis (PCA) guided K-means
clustering. Bae et al. [47] propose the K-means based clustering al-
gorithm k-GRAPE (K-means group agent partitioning and placement
event) and maximize the EV users’ individual utilities (including dis-
tance and brand preferences) with a game theoretical approach. Liu
et al. [48] consider renewable energy usage and use the K-means
method to improve user satisfaction in clusters. Sánchez et al. [49]
use the K-means algorithm to solve the combined charging station
placement and electric vehicle routing problem.
Hierarchical clustering algorithm has the benefit of working without
a pre-defined cluster number. Zeng et al. [50] determine the battery
swap station locations using K-means and hierarchical clustering meth-
ods. They map GPS data to the road network by hidden Markov models,
estimate the electricity consumption and the location of swap demand,
characterize the spatial demand distribution by clustering methods, and
select the centroid of the clusters as the station locations. Bitencourt
et al. [51] employ hierarchical clustering to determine semi-fast charg-
ing stations’ zones and determine each zone’s CS location/capacity
using an evolutionary search algorithm.
There are several other clustering algorithms used for solving the
infrastructure planning problem. Yao and Luo [52] use clustering by
fast search-and-find of density peaks (CFSFDP) algorithm and consider
4
Renewable and Sustainable Energy Reviews 188 (2023) 113873
population density. Momtazpour et al. [53] propose a coordinated
clustering approach to cluster locations with low electricity load and
high-income people. After determining the candidate locations, they
assign EVs to stations to determine the demand at each location and
compute the required storage capacity. Wang et al. [54] plan the
charging facilities of electric buses and use AP clustering to plan the
midway supply stations. Clio and Babacan [55] use Density-based
Spatial Clustering of Applications with Noise (DBSCAN) to identify the
smallest number of CS locations sufficient for a large EV taxi fleet,
considering the distances from the historic breakpoints of the drivers.
Other machine learning algorithms such as reinforcement learning are
rarely used for infrastructure planning. As an example Ko [56] combine
genetic algorithm and reinforcement learning to optimize electric tram
wireless charging infrastructure.
In some studies, machine learning algorithms provide demand
prediction information to the main algorithm that solves the capac-
ity/sizing problem. Hu et al. [57] propose a multiple periods location
and capacity expansion model. They employ a bayesian regularization
back propagation neural network (Bayesian BPNN) model for electric
vehicle quantity prediction and solve the model with greedy algorithm.
Jiang et al. [58] evaluate charging station location options for electric
taxis. They use a multi agent simulation framework, employ Q-Learning
to train the agents, and evaluate different charging station location
N. Fescioglu-Unver and M. Yıldız Aktaş Renewable and Sustainable Energy Reviews 188 (2023) 113873

Table 1
Infrastructure planning.
Ref Year Journal Type Objective ML method
[50] 2019 IEEE Access Location Min. distance K-means , Hierarchical clust.
[43] 2016 Applied Energy Location Min. distance Fuzzy K-means
[44] 2016 Applied energy Loc. & Cap. Max. profit K-means
[45] 2020 IEEE Transactions on Industry Applications Loc. & Cap. Min. cost K-means
[46] 2020 Journal of Cleaner Production Loc. & Cap. Min. cost PCA K-means
[48] 2018 Energy Loc. & Cap. Min voltage/load fluctuation, cost K-means
[47] 2020 IEEE Transactions on Intelligent Transportation Systems Location Max. EV utility k-GRAPE
[49] 2022 Energies Location K-means
[54] 2021 IEEE Transactions on Intelligent Transportation Systems Loc. & Cap. Min. cost AP clustering
[51] 2021 Electric Power Systems Research Loc. & Cap. Min. electrical losses, Min. zone center deviation Hierarchical clust.
[53] 2014 ACM Transactions on Intelligent Systems and Technology (TIST Loc. & Cap. Min. cost, distance Coordinated clust.
[52] 2018 Kybernetes Location Max. demand coverage CFSFDP clust.
[55] 2021 Applied Energy Location Min. number of CS locations DBSCAN
[57] 2020 Neurocomputing Loc. & Cap. Max. demand coverage Bayesian BPNN
[58] 2018 IET Generation, Transmission & Distribution Location Min. cost, Min. power loss/voltage deviation Multi-step Q-Learning

scenarios through evidential reasoning to optimize the benefits of
both the taxi owners and the power grid. Table 1 summarizes the
studies which determine the infrastructure location (type:Location),
and both location and capacity (type: Loc. & Cap.). Table 1 shows
that studies have various objectives. Most frequent objectives are
minimizing distance [43,43,50], minimizing cost [45,45,46,48,54,58],
minimizing cost and distance together [53] and maximizing demand
coverage [52,57].
3.2. Charging demand and charging station utilization forecasting
Forecasting studies in this area can be grouped as long term
(monthly, yearly) and short term (daily, weekly). Short term fore-
casting helps EV users and grip operators’ daily planning process.
Charging station utilization prediction is generally handled as a short
term forecasting problem. Month, day, time interval, temperature, and
precipitation are some of the features that machine learning algorithms
use. Studies frequently use LSTM, random forest and kNN algorithms
for classification and/or regression. LSTM algorithm is based on re-
current neural networks and frequently used in time-series prediction.
Random forest (RF) combines multiple decision trees and it is easier to
evaluate the importance of different variables in prediction. K-nearest
neighbor (kNN) algorithm is a non parametric method which uses
a selected distance measure to determine the nearest neighbors. The
algorithm performances vary by the problem type. Most studies select
several forecasting algorithms and compare their performances on the
same data set.
Hecht et al. [59] use gradient boosting (GB) and random forest
(RF) classification algorithms to predict the next day CS occupation in
five classes from very low to very high (features: Temperature, pre-
cipitation, weekday, holiday, traffic). Ostermann et al. [60] compared
different forecasting algorithms to predict the short term occupancy
of charging points (features: Month, day, weekday, interval, holiday,
bringing day, time). Friese et al. [61] use agglomerative clustering to
predict charging point occupancy and random forest classification to
analyze CS usage patterns (features: CP occupancy — weekday and
time, CS pattern — the number of detached or row houses, commercial
buildings, households with a monthly net income of 900 EUR, and the
buying power per inhabitant). Aduama et al. [62] use LSTM to fore-
cast charging station energy demand (features: Temperature, humidity,
wind speed).
Charging demand long term forecasting helps the infrastructure
and expansion planning. Yi et al. [63] forecast monthly commercial
EV charging demand on a state/city scale with sequence to sequence
(Seq2Seq) deep learning and long short term memory (LSTM) models
(features: Number of monthly check-ins, charging site rating, number
and type of plugs, opening hours, parking spots, neighbor point of
interest). Verma et al. [64] compare several classification algorithms,
including classification and regression trees (CART), Random forest
algorithm (RF), k-nearest neighbor (kNN), and Chi-square automatic in-
teraction detector (CHAID), and predict EV charging within households
through energy usage data. Kim and Kim [65] compare LSTM and ANN
with time-series models in power consumption forecasts for short and
long term EV power consumption (features: Geographical data, special
day indicators, temperature). Table 2 lists the forecasting studies with
the machine learning algorithms they use.
4. Charge control and coordination
Uncoordinated EV charging can increase the EV owners’ electricity
purchase price and bring significant loads to the power grid. EV Smart
Charging (EVSC) uses information and communication technologies to
coordinate the EV charging system. The stakeholders in an EV charging
service system are the power grid operator, the aggregator, and the EV
owners.
The grid operator collects energy from power plants and renew-
able energy sources. The aggregators are responsible for managing
the interaction between a group of EV owners and the grid operator.
Aggregators collect the charging demand and connection times from
the EVs via smart meters and make energy buy/sell decisions and
negotiations with the power grid [13]. An aggregator can manage the
charge/discharge schedule of connected EVs such that the power grid,
EV owners, and the aggregator benefit from this operation. The grid
operator can benefit from services such as peak load reduction/shifting,
frequency and voltage regulation [14,66]. Meanwhile, EV owners ben-
efit from reduced charging costs. Fig. 3 shows the interactions between
the grid operator, aggregator, and EVs.
Charge scheduling studies frequently use linear, non linear, mixed
integer, dynamic programming, game theory and metaheuristics ap-
proaches [67]. Nimalsiri et al. [68] analyze the charge control architec-
tures in three categories: Centralized, decentralized, and hierarchical.
In centralized architectures, a central decision maker controls all EV
charging schedules. Decentralized architectures include cases where the
EVs communicate with each other or use an aggregator as a communi-
cation bridge while determining their charging schedules. Hierarchical
architectures use multiple aggregators, each coordinating a group of
EVs. In this case, the aggregator may act as a decision maker for the
connected EVs, communicate with a central aggregator and control the
charging schedule of EVs directly or indirectly (e.g., through electricity
price).
Fig. 4 depicts an architecture with different aggregator
types. Aggregator-1 acts as a main aggregator for the sub aggregators
(Aggregators 5 & 6), which are the charging stations. Aggregator-2 is a
utility company sending/collecting information to/from smart chargers
at homes. Aggregator-3 is a parking lot manager. Aggregator-4 is a
charging station directly connected to the grid operator. In addition,
aggregators may have their own renewable energy sources and storage
units.

5
N. Fescioglu-Unver and M. Yıldız Aktaş Renewable and Sustainable Energy Reviews 188 (2023) 113873

Table 2
Charging demand and CS utilization forecasting.
Ref Year Journal Type Term ML method
[59] 2021 Energies CS utilization Short term GB, RF
[60] 2022 Energies CS utilization Short term Linear and logistic regression, kNN, XGBoost, RF
[61] 2021 Sustainability CS utilization, CS usage patterns Short term Agglomerative Clust., RF
[62] 2023 Energies CS energy demand Short term LSTM
[63] 2022 Journal of Intelligent Transportation Systems Demand Long term Seq2Seq, LSTM, XGBoost, ARIMA, Prophet
[64] 2019 Computers & Industrial Engineering Demand Long term CART, RF, kNN, CHAID
[65] 2021 Energies Demand Long/short term ARIMA, ANN, LSTM

Fig. 3. Interactions between EV, Aggregator, and Grid Operator.

Fig. 4. EV, Aggregator, Grid Network Architecture.

6
N. Fescioglu-Unver and M. Yıldız Aktaş
Wang et al. [66] classify the smart charging algorithms from the
perspectives of the grid operator, aggregator, and user (customer).
The grid operator objectives include grid stability and power quality.
Comprehensive reviews [69,70] investigating electric vehicle charging
station effects on the grid exist in the literature. This review study
mainly concentrates on the end user’s charging experience and handle
the user and aggregator perspectives. For user type scheduling algo-
rithms, the objective is minimizing the total cost while satisfying the
charging demand of the vehicles on time — before the vehicle departs.
The disadvantage of user type algorithms is that when the number of
vehicles using the same scheduling algorithm increase, it may create
new peak points in low price regions. The users can be fleet owners act-
ing as aggregators who schedule their vehicles’ charge times with cost
minimization objectives. Electric vehicle fleet charging problems show
different energy consumption requirement characteristics depending on
the type (ultra-light, light, heavy), number of vehicles, and drivers [71].
Aggregators can directly control charging schedules or indirectly
affect the schedules through dynamic pricing (pricing models inves-
tigated in Section 5). Aggregator type scheduling algorithms aim to
minimize the operating cost or maximize the profit of the aggregator.
In both the user and aggregator perspectives, the primary cost item is
the electricity purchasing cost, while battery degradation cost is also
included in several studies. Meeting the user demand on time can be a
constraint or can be part of the cost with an associated penalty.
Uncertainties in the charge control problem include EV arrival/
departure times, charging time and quantity preferences, and elec-
tricity prices. The energy quantity available is also uncertain if the
energy source is partly or fully photovoltaic or wind. Machine learning
algorithms provide effective online solutions under these uncertain-
ties. Charge/discharge scheduling models frequently use reinforcement
learning (RL) based algorithms. Studies employ different RL tech-
niques, such as Q-Learning, deep Q-networks, and deep deterministic
policy gradient methods, or propose new RL based scheduling algo-
rithms and test their effectiveness. Q-Learning is a model free, value
based reinforcement learning (RL) model that uses the Q-table to track
the value of taking an action in a given state. Q-Learning requires
discrete state variables and discrete actions. When the state variables
take continuous values, a Q-table is not sufficient. Deep Q-Learning
algorithm uses the deep Q-network (DQN) instead of the Q-table and
can handle continuous state space. Deep deterministic policy gradient
(DDPG) is a model free actor-critic model and enables continuous state
and action space. Both DQN and DDPG models belong to the Deep re-
inforcement learning (DRL) algorithms class. LSTM method frequently
accompanies the scheduling models by electricity price forecasting.
Looking from the user’s perspective, Li et al. [72] propose a con-
strained deep learning algorithm to guarantee a fully charged battery
before departure. Yan et al. [73] consider drivers’ range anxiety and
combines supervised learning with soft actor critic reinforcement learn-
ing for charge scheduling. Several studies employ LSTM to predict
future electricity prices and make EV owners’ charge/discharge sched-
ules with deep reinforcement learning. The main objective of these
studies is cost minimization while avoiding on trip battery depletion.
Some studies use cost minimization as the single performance met-
ric [74–76], while others evaluate user satisfaction [77] and consider
range anxiety factor [78]. Zhang et al. [79] combine the user and grid
operator perspectives and propose a multi agent deep reinforcement
model and LSTM model to minimize the cost for users and avoid
grid transformer overload. Chiş et al. [80] predict electricity prices
through a bayesian neural network and uses fitted Q-iteration model
for charge scheduling. While reinforcement learning is frequently used
for scheduling, Lopez et al. [81] propose different machine learn-
ing methods (kNN, shallow neural network-SNN and a deep neural
network-DNN) to determine the charge schedule of a connected EV
with a cost minimization goal. During training the DNN, they use op-
timal solutions obtained with dynamic programming. Liang et al. [82]
solve the joint charge scheduling, order dispatching, and rebalancing
7
Renewable and Sustainable Energy Reviews 188 (2023) 113873
problem of a shared electric vehicle fleet using reinforcement learning.
In this case, the model’s user is the fleet operator, and the objective is
to maximize the profit by maximizing the fleet revenue and minimizing
the charging costs. Vandael et al. [83] plan day ahead charge schedule
of an EV fleet using a fitted Q-iteration based reinforcement learning.
Looking from the aggregators’ perspective, Arif et al. [84] solve
the scheduling problem of a parking lot aggregator with interruptable
charging through Q-Learning. Cao et al. [85] develop an actor-critic
learning based scheduling model with continuous charging amount
action space for charge scheduling at a CS. In another study, they work
with shared autonomous EVs, which can more easily be scheduled to
participate in V2G services [86].
Lee et al. [87] handle a single public charger, model its usage
pattern through kernel density estimation, and use this pattern for
training the DQN (deep Q-network) for charge/discharge scheduling
with cost minimization objective in a dynamic pricing environment.
Wan et al. [88] predict energy prices using neural networks and min-
imize the total cost of a group of EVs using multiagent DRL. Ye
et al. [89] propose a deep Q-Learning algorithm for charge/discharge
scheduling and centrally allocate chargers to vehicles. In this study,
the chargers make their charge/discharge decisions individually to
maximize the charging station’s profit. Aljafari et al. [90] propose
a charge/discharge scheduling model using multi agent deep neural
networks. Wang et al. [91] propose a reinforcement learning based
charge scheduling model for a photovoltaic and energy storage inte-
grated charging station. Shin et al. [92] develop a multiagent deep
reinforcement model for distributed charging stations with photovoltaic
energy and storage systems. Jin and Xu [93] introduce a nodal multi
target policy for scheduling a large scale EV fleet under renewable en-
ergy and price uncertainties, and determine the target levels and apply
the policy through a soft actor critic RL model (SAC). Liu et al. [94]
use neural networks for demand forecasting and proposes a mixed-
integer nonlinear programming model and a heuristic model for charge
scheduling. Yan et al. [95] propose a multi agent DRL based soft actor
critic (SAC) model to minimize the charging cost and range anxiety
while avoiding the transformer overload in a residential area. Ming
et al. [96] propose a two level coordinated charge/discharge scheduling
model that decides the total charge/discharge energy to be distributed
by a constrained DRL (constrained SAC and PPO) and distributed the
energy with a priority rule based on the energy demand amount.
Aggregators selling electricity to the local market should determine
their bidding volume and price in addition to the charging schedule.
Tao et al. [97] develop a deep reinforcement based bidding strategy
for EV aggregators and determine the quantity and price to purchase
and sell electricity at each time slot in the local electricity market under
information asymmetry. Lin et al. [98] use an LSTM model to predict
the EV departure/arrival times, which provides the EV load in each
period to the mixed integer linear programming model that determines
the optimal bidding volume at the electricity market.
Battery swap stations also act as aggregators in charge control
systems. Asadi and Pinkley [99] handle a battery swap station where
battery replacement cost comes in addition to the electricity purchase
and battery degradation costs. They propose a reinforcement learning
based algorithm to generate a charge, discharge, and battery replace-
ment schedule. Gao et al. [100] propose a deep deterministic policy
gradient (DDPG) algorithm to serve batteries to electric buses while
minimizing the operating cost in a charge/discharge environment.
Wang et al. [101] use battery swap stations as fast frequency regu-
lation services and use deep Q-Network for charge scheduling. Liang
et al. [102] handle a system where a central battery changing station
charges the batteries and send them to swap stations via trucks. They
combine multi agent deep reinforcement learning (DRL) and binary
integer programming for joint scheduling of charging, swapping, and
truck routing operations. Tan et al. [103] combine LSTM, DRL, and
mixed integer linear programming model to schedule swap operations,
where LSTM predicts the electricity prices and demand, DRL allocates
N. Fescioglu-Unver and M. Yıldız Aktaş Renewable and Sustainable Energy Reviews 188 (2023) 113873

Table 3
Charge control and scheduling.
Ref Year Journal Type Objective ML method
[72] 2019 IEEE Transactions on Smart Grid User Min. cost Safe DRL
[73] 2021 IEEE Transactions on Smart Grid User Min. cost and anxiety SAC
[77] 2020 IEEE Internet of Things Journal User Min. cost DDPG, LSTM
[74] 2021 Journal of Modern Power Systems User Min. cost DDPG, LSTM
and Clean Energy
[78] 2018 IEEE Transactions on Smart Grid User Min. cost DQN, LSTM
[76] 2020 IEEE Transactions on Vehicular User Min. cost Dyna-Q RL, LSTM
Technology
[75] 2020 Computers and Electrical Engineering User Min. cost Q-Learning, LSTM
[79] 2022 IEEE Transactions on Intelligent User and grid Min. cost and power load Multiagent DRL, LSTM
Transportation Systems
[80] 2016 IEEE Transactions on Vehicular User Min. cost Fitted Q-iteration, Bayesian NN
Technology
[81] 2018 IEEE Transactions on Smart Grid User Min. cost KNN, SNN, DNN
[82] 2020 IEEE Transactions on Smart Grid User Max. profit DRL
[83] 2015 IEEE Transactions on Smart Grid User Min. cost Fitted Q-iteration
[84] 2016 Sustainable Energy, Grids and Aggregator Min. cost Q-learning
Networks
[86] 2020 IEEE Internet of Things Journal Aggregator Min. cost Q-Learning
[85] 2021 IEEE Internet of Things Journal Aggregator Min. cost Actor-critic learning
[87] 2020 Energies Aggregator Min cost DQN, Kernel density estimation
[88] 2022 Journal of the Franklin Institute Aggregator Min. cost Multiagent DRL, NN
[90] 2023 Computers and Electrical Engineering Aggregator Max. profit Multiagent DNN
[89] 2022 IEEE Transactions on Smart Grid Aggregator Max. Profit DQN
[91] 2022 Sustainability Aggregator Min. cost and improve clean Double DQN, Dueling DQN
energy consumption
[92] 2020 IEEE Transactions of Industrial Aggregator Min. cost DQN
Informatics
[93] 2020 IEEE Transactions on Smart Grid Aggregator Min. cost SAC
[94] 2022 International Journal of Electrical Aggregator Min. cost NN
Power and Energy Systems
[95] 2022 IEEE Transactions on Industrial Aggregator Min. cost and anxiety Multi agent DRL, SAC
Informatics
[96] 2023 Applied Energy Aggregator Min. cost Const. SAC, PPO
[97] 2021 IEEE Transactions on Industrial Aggregator Max. profit Async. DRL, DDPG
Informatics
[98] 2020 IEEE Access Aggregator Max profit LSTM
[99] 2021 Transportation Research Part E: Aggregator Max. profit RL
Logistics and Transportation Review
[100] 2020 IEEE Transactions on Industry Aggregator Min. cost DDPG
Applications
[101] 2020 IEEE Transactions on Industrial Aggregator Max. profit DQN
Informatics
[102] 2022 IEEE Transactions on Smart Grid Aggregator Max. profit Multi-agent DRL
[103] 2023 Engineering Applications of Artificial Aggregator Min. cost LSTM, DRL
Intelligence
[104] 2023 IEEE Transactions on Smart Grid Aggregator Min. cost PPO

power to chargers, and MILP model allocates charger power to batter-
ies. Jin et al. [104] determine the total charge amount bu proximal
policy optimization (PPO) and allocate this amount among waiting bat-
teries using the least demand first priority rule, under renewable energy
and electricity price uncertainties. Table 3 summarizes the machine
learning methods and objectives of the charge control and scheduling
studies. The objectives are frequently minimizing cost [72–79,81,83,
85–88,91–94,100,103] or maximizing profit [82,84,89,90,97–99,101,
102], sometimes with additional objectives such as improving clean
energy consumption [91] or minimizing user anxiety [73,95].
5. Dynamic pricing
An efficient time varying pricing strategy creates benefits for service
providers and customers. Service providers’ benefits include grid load
balancing and increased profits. Customers benefit from decreased
waiting times or decreased charging costs. Time varying charging pric-
ing models include time of use (TOU) pricing and dynamic pricing. TOU
pricing depends on time and does not change dynamically according
to the operating conditions. Dynamic pricing allows service operators
dynamically vary the price EV users pay for charging [105]. Charging
service dynamic pricing techniques frequently used in the literature
include evolutionary algorithms, mixed integer linear/nonlinear pro-
gramming, control theory, and dynamic programming [18]. Machine
learning, especially reinforcement learning use, is increasing in this
area.
The dynamic pricing problem requires decision making in a stochas-
tic and continuously changing environment with uncertain EV ar-
rival/departure times, charge demand, and utility company electric
sales prices changing in real time. The charging location can be a
parking lot where the vehicles are parked for a long duration or a fast
charging station where the owners wait alongside their vehicles. The

8
N. Fescioglu-Unver and M. Yıldız Aktaş
same price can be valid for all vehicles, or stations can charge different
prices with a differentiated service.
Charging parked electric vehicles requires joint determination of
charging price and charge schedule. Parked EVs’ demand should be
satisfied before they leave, and EVs will pay from the price rate valid at
the time they enter the parking area. Wang et al. [106] and Paraskevas
et al. [107] determine the charging price and the total charging rate at
each slot through state–action-reward-state–action (SARSA) and deep
Q-Learning models respectively. The total charging rate is later dis-
tributed among EVs using the least laxity first algorithm. Moghaddam
et al. [108] introduce an EV load-shifting mechanism with fast adaptive
heuristic critic (Fast-AHC) based reinforcement learning and aim to
increase the station’s profit while maintaining the utility’s stability.
In the fast charging stations case, EV owners have a limited time,
and they can quit waiting for charging after a while in the queue.
Fang et al. [109] propose a Q-Learning model for dynamic pricing
at a fast charging station and show that as the backlog rate (ratio of
people accepting to wait) increases, the revenue benefit of dynamic
pricing wrt. stationary pricing also increases. Quian et al. [110] handle
multiple charging stations in a competitive environment and propose
a multi agent deep reinforcement learning model for determining the
optimal pricing strategy for each charging station.
In the differentiated service case, Quality of Service (QoS) level is
part of the objective or a constraint. QoS metric may include items
like EV waiting time, service rejection rate, and EV owner’s satisfaction
with charging price. Abdalrahman and Zhuang [111] investigate a
charging service infrastructure with multiple stations with multiple
service types (AC charging, fast DC charging, wireless charging) and
develop a twin delayed DDPG based differentiated pricing model that
maximizes station utilization while maintaining the target QoS level.
Zhao and Lee [112] consider a charging station with multiple charging
service levels that share the same waiting area and propose a deep rein-
forcement based dynamic pricing framework considering differentiated
service requirement levels and maximizing the cumulative QoS.
Looking from the EV owners’ perspective, Zhang et al. [113] pro-
pose a cooperative multi agent-DQN based bidding strategy for an EV
fleet. The EV users determine their charging quantity and unit price
offer through this strategy to maximize their utilities, and charging
stations determine the winning bids with an auction mechanism. Kim
et al. [114] propose two Q-Learning algorithms. The first is a dynamic
pricing model for the service provider, and the second is a charge
scheduling model for the EV owners. Simultaneous use of these models
decreases costs for both parties.
Using the V2G technology, EV aggregators and service providers can
act as prosumers and both buy and sell electricity at the local energy
market. Qui et al. [115] combine the prioritized experience replay
method with deep deterministic policy gradient (DDPG) to determine
the prices the aggregator offers to a population of EVs with and without
V2G flexibility. Liu et al. [116] increase the price change frequency and
establish a quarter-hourly pricing strategy using a DDPG based rein-
forcement model. Table 4 summarizes these studies. The main objective
of most studies is maximizing profit or minimizing cost [106–110,114–
116], while maximizing stability [108], quality of service [111,112],
and EV user utility [113] are additional objectives in some studies.
6. Electric vehicle routing, navigation and charging station selec-
tion
Electric vehicle routing studies can be divided into two main classes:
Electric vehicle routing problems (EVRP) and electric vehicle naviga-
tion problems. EVRP can be described as the set of problems that route
one or more electric vehicles to serve one or more customer nodes.
Electric vehicle navigation problems do not include customer stops
but they route the electric vehicles from an origin to a destination
point. Some of these studies include charging stops to the plan, while
others focus on arriving at the destination point with minimum energy
9
Renewable and Sustainable Energy Reviews 188 (2023) 113873
consumption. A realistic energy consumption computation requires
taking the environmental conditions, road conditions, driver behavior
and vehicle type/load into account [117,118].
The total travel time of a vehicle includes the driving time, waiting
time at the station, and charging time (if charging is part of the plan).
Uncertainties include traffic and road conditions, energy consumption,
changing driver behavior, and waiting times at the stations. Fig. 5
summarizes an electric vehicle navigation problem where vehicles
use different routes and charging stations depending on the traffic
condition, charging price, and congestion at the stations.
Studies that use machine learning as a part of the solution approach
employ it either for the primary path planning tasks or as an aid for pro-
viding input forecasts to the path planning models. This section divides
the EV routing/navigation studies into three subsections. Sections 6.1
and 6.2 investigate the EVRP and EV navigation studies that use ML
as the main path planning algorithm, respectively. Section 6.3 lists the
studies that use ML to provide an input forecast to the path planning
model.
6.1. Electric vehicle routing problem and variations
Electric vehicle routing problem (EVRP) is finding a set of vehicle
routes that service a set of customer nodes and depart/end at a given
depot node. EVRP is a generalization of the VRP (Vehicle routing
problem) and is considered NP-hard in the strong sense [117]. Solution
approaches for EVRP include exact algorithms like branch-price-and-
cut and heuristic algorithms like genetic algorithm, tabu search, and
iterated local search. Solving the EVRP problem in real time brings
additional uncertainties, such as dynamic customer requests. Machine
learning solutions can anticipate and prepare for future demand and
changing conditions and accelerate solving large scale problems.
There are several variants of the EVRP. Machine learning algorithms
are especially used in online routing problems with dynamic customer
request arrivals (DEVRP — Dynamic Electric Vehicle Routing Problem).
Depending on the problem type, the models determine when to relocate
the EVs, when to assign them to which customers, and when to send
them to charging. The charging location can be the depot or a nearby
charging station, and the charging level can be full or partial. James
et al. [119] propose a computationally fast vehicle routing model for
online transportation in large networks where requests arrive/get can-
celed dynamically. They use neural combinatorial optimization for tour
planning and deep reinforcement learning for determining the neural
network model parameters. Basso et al. [120] handle dynamic customer
requests and use Q-Learning for route planning with minimum energy
consumption and battery depletion risk. Shi et al. [121] solves a com-
munity operated fleet’s ride hailing problem using deep reinforcement
learning. Kullman et al. [122] use a deep Q-Learning variant - dueling
double DQN (D3QN) with attention mechanism for the ride hailing
problem. Lin et al. [123] use RL model with an attention mechanism
to solve the EVRP with time windows. For this static problem type, RL
helps solve very large problem instances.
6.2. Electric vehicle navigation with charging station selection
Electric vehicle navigation problems route electric vehicles from an
origin to a destination point. Unlike the EVRP type problems, there
are no customer nodes to stop and serve at. Navigating with charging
station selection (EVCS) studies, route the vehicles so that each ve-
hicle can charge at a charging station. The charging station selection
decision may increase the total distance traveled depending on the
location of the CS or may increase the total travel time depending
on the queuing time at the selected station. Charging station selec-
tion brings additional uncertainties to the routing problem: charging
prices and random arrivals/waiting times at CSs. Studies in this area
frequently use reinforcement learning based algorithms to overcome
these uncertainties. Zhang et al. [124] use the Q-Learning algorithm to
N. Fescioglu-Unver and M. Yıldız Aktaş Renewable and Sustainable Energy Reviews 188 (2023) 113873

Table 4
Dynamic pricing.
Ref Year Journal Type Objective ML method
[106] 2019 IEEE Transactions on Industrial Informatics Price and schedule Max. profit SARSA
[107] 2022 Energies Price and schedule Max. profit DNN, DQN
[108] 2022 IEEE Access Price Max. profit and stability Actor-critic, Fast-AHC
[109] 2020 IEEE Transactions on Intelligent Transportation Systems Price Max. profit Q-Learning
[110] 2021 IEEE Transactions on Power Systems Price Max. profit Multi agent DRL
[111] 2020 IEEE Transactions on Intelligent Transportation Systems Differentiated service Max. demand w. QoS DRL
[112] 2021 IEEE Transactions on Transportation Electrification Differentiated service Max. QoS DNN, DRL
[114] 2015 IEEE Transactions on smart grid Price and schedule Min. cost Q-Learning, PDS learning
[113] 2020 Neurocomputing Price and schedule Max. EV util. Multi agent DQN
[115] 2020 IEEE Transactions on Industry Applications Price and schedule Max. profit DDPG
[116] 2021 IEEE Access Price Max. profit DDPG

Fig. 5. Electric vehicle navigation and charging station selection.

allocate EVs to charging stations while minimizing the station queueing
times and the travel time, including road congestion. Qian et al. [125]
included the varying charging costs at CSs and proposed a deep Q-
Learning based model to minimize the total travel time (road and
station waiting times) and charging costs. Lee et al. [126] introduce
a central management system that takes charging requests from EVs,
current state information (i.e., charging/waiting EVs) from CSs, and
recommends/reserves CSs to minimize the total travel time of multiple
EVs, using deep Q-network (DQN) based model. Xu et al. [127] propose
a graph reinforcement learning based fast charging station recommen-
dation system that minimizes the total travel and station waiting times
while considering the stability of nodal voltages. Tao et al. [128]
handles a multi-type charging service recommendation system with
fast charging, slow charging, and battery swap options and uses deep
Q-Learning with mixed integer linear programming to dispatch the
charging/swapping orders. Suanpang et al. [129] propose a multi agent
deep reinforcement learning model for charging station selection, with
waiting time, charging price, and charging failure (incomplete charge)
rate lowering objective. Jin and Xu [130] develop an advantage actor-
critic (A2C) algorithm for CS selection and path planning. They use the
shortest path algorithm to determine the route to the CSs and select a
CS through the actor-critic model.
6.3. Forecasting for path planning
In several studies, path planning is completed by a linear program-
ming or heuristic approach. Machine learning algorithms help the path
planning process by energy consumption, charging price, speed, and
travel time forecasts.
Thibault et al. [131] plans the EV path with energy consumption
and travel time minimization objective and uses ANN to approximate
the optimal vehicle speed for real time online usage. Shukla et al. [132]
use DNN for traffic volume and station wait time prediction and allo-
cate CSs to EVs to minimize travel time. Huang et al. [133,134] propose
a central route and charge planner for EVs to prevent traffic congestion.
The first study [133] estimates the travel time by Support Vector
Regressions, using weather, day of the week, traffic density, light wait
time, and passenger pick up/drop off time information. The second
study [134] uses support vector regressions (SVR) to compute the EV
charging times and travel times and propose algorithms to avoid rush
hour congestion. Basso et al. [135] use linear programming for EVRP
path planning while forecasting energy usage with Bayesian Regression
techniques. Lu et al. [136] use A-star algorithm for path planning with
charging station selection of EV users with different travel objectives
(travel distance, travel time, wait time, charge cost) and integrates
gated recurrent unit neural network (GRU NN) for charging price
forecasts.
Another group of studies navigate EVs while minimizing their en-
ergy consumption (EVEC). In these studies, vehicle charging either does
not exist or is done using on-route charging stations with no waiting
time. They consider energy consumption factors such as acceleration,
deceleration, hill climbing, rolling resistance, and aerodynamic drag
force. Although these studies do not explicitly handle the charging
service, they provide valuable insight into how machine learning meth-
ods aid in minimizing energy consumption by road planning or by

10
N. Fescioglu-Unver and M. Yıldız Aktaş Renewable and Sustainable Energy Reviews 188 (2023) 113873

Table 5
Electric vehicle routing, navigation, and charging station selection.
Ref Year Journal Type Objective ML method
[119] 2019 IEEE Transactions on Intelligent Transportation Systems EVRP Min. distance, Max. delivered requests DRL, DNN
[120] 2022 Transportation Research Part E EVRP Min. energy, Min. battery depletion risk Q-Learning
[121] 2019 IEEE Transactions on Intelligent Transportation Systems EVRP Min. distance, waiting time DRL
[122] 2022 Transportation Science EVRP Max. profit D3QN
[123] 2021 IEEE Transactions on Intelligent Transportation Systems EVRP Min. distance DRL
[124] 2019 Sustainability EVCS Min. total time Q-Learning
[125] 2019 IEEE Transactions on Smart Grid EVCS Min. total time, cost DQN
[126] 2020 Energies EVCS Min. total time DQN
[127] 2022 International Journal of Electrical Power and Energy Systems EVCS Min. total time, Balance power system Graph RL
[128] 2022 Applied Energy EVCS Min. total time, Max. revenue DQN
[129] 2023 Sustainability EVCS Min wait time, price, failure Multi agent DRL
[130] 2022 IEEE Internet of Things Journal EVCS Min total time and cost A2C
[131] 2018 IEEE Transactions on Intelligent Vehicles Forecast Min. energy ANN
[133] 2021 Information Technology and Control Forecast Min. travel time SVR
[134] 2021 International Journal of Computational Intelligence Systems Forecast Min. distance, total time, cost SVR
[132] 2020 Internet of Things Forecast Min. total time DNN
[135] 2021 Transportation Research Part B: Methodological Forecast Min. energy Bayesian Regression
[136] 2019 IEEE Access Forecast Min. total time, cost GRU NN
[137] 2020 IEEE Transactions on Vehicular Technology EVEC Min. energy Q-Learning
[138] 2021 Electric Power Systems Research EVEC Min. energy DDQN
[140] 2021 Frontiers in Big Data EVEC Min. energy Q-Learning
[141] 2021 Applied Energy EVEC Min. total time, energy RNN
[139] 2022 Vehicles EVEC Min. energy SARSA
[142] 2017 IEEE Transactions on Intelligent Vehicles Forecast Min. total time, energy K-means, Naive Bayes
[143] 2021 ACM Transactions on Internet Technology Forecast Min. energy DNN, Linear regression, Decision tree

energy consumption forecasting. Zhang et al. [137] use the Q-Learning

method to minimize the total energy (fuel + electricity) spent by a
PHEV considering the distance traveled, road slope, and vehicle speed.
Aljohani et al. utilize double deep Q-Learning (DDQN) [138] and
SARSA [139] to develop routing models that minimize EV’s energy
utilization. Dorokhova et al. [140] use Q-Learning approach to generate
energy feasible paths while including energy recuperation of EVs and
allowing charging on the route (station availability-wait times not
considered). Bi et al. [141] propose a central controller model which
uses a bilevel optimization algorithm to minimize travel time and
energy consumption of fuel and electric vehicles on the roads. They
use a random neural network (RNN) model for route planning and a
nonlinear optimization model for generating incentives that motivate
road users. Bozorgi et al. [142] employ data mining techniques to
extract the desired speed profiles that minimize travel time or energy
consumption and use this information to select a route among the
candidate shortest time routes. Ayman et al. [143] employs DNN,
linear regression, and decision tree for energy consumption forecast
and assigns mixed fleets to fixed routes by Genetic algorithm. Detailed
reviews exist on energy consumption [144] and range prediction [145]
studies.
Table 5 summarizes the studies with their problem types, methods,
and objectives. The main objectives can be categorized as minimizing
distance [119,121,123,134], minimizing time (travel, waiting or total
time) [124–130,132–134,136,141,142], minimizing energy [120,131,
135,137–143] and minimizing cost/maximizing profit [122,125,128,
130,134,136].
7. Findings and research directions
Electric vehicle charging service optimization research is growing
continuously. Machine learning methods have an increasing share in
the literature. This study reviews the machine learning applications
related to charging service operations. The study covers journal pub-
lications indexed in Web of Science database, published between 2014
and 2023. The search was conducted separately for each section subject
- Infrastructure planning, charge control/coordination/scheduling, dy-
namic pricing, and routing, using different keyword combinations. This
study has several limitations. First of all, although the search covered a
broad range, there may still be relevant articles uncovered. In addition,
Fig. 6. Frequently used machine learning techniques.
conference proceedings are not included in the search. Lastly, studies
in languages other than English are not covered in this review.
Most of the studies covered (90%) belong to the years 2018–2022.
Fig. 6 provides a visual insight into frequently used machine learning
techniques for charging service area.
Machine learning algorithms aid different planning/service opera-
tions in multiple ways. The first step in charging service planning is
building a cost-efficient infrastructure with sufficient area coverage.
Forecasting models provide valuable input to infrastructure planning
algorithms through long term and short term demand prediction. Clus-
tering algorithms aid in determining the location and capacity of the
charging stations.
Once the infrastructure is planned, the problem is how to control
and coordinate the dynamic charging requests of electric vehicles.
There are several uncertainties in the charge control problem, including
the electric vehicle arrival/departure times, vehicles’ state of charge
and charge amounts requested, and the grid electricity prices. In addi-
tion, when the primary energy source is solar or wind, the available
energy is also uncertain. Forecasting algorithms provide vehicle de-
parture/arrival time, charge demand, electricity price, and renewable
energy amount predictions. For charge control and coordination, rein-
forcement learning based models provide effective scheduling solutions
in real time under uncertainty.

11
N. Fescioglu-Unver and M. Yıldız Aktaş
Dynamic pricing models determine the aggregators’ electricity sell-
ing price in real time under the same uncertainties. Reinforcement
learning based multi agent and single agent models determine the
aggregators’ selling price in real time while minimizing the cost, max-
imizing the profit, or maximizing the quality of service. Finally, nav-
igating an electric vehicle requires energy consumption, travel time,
and speed forecasts. Forecasting algorithms provide these inputs to
the navigation models. Navigation plans may include stops at charging
stations or direct the vehicle from departure to destination points with
the available charge. Reinforcement learning based models navigate the
vehicles successfully under several constraints.
Charging service optimization studies covered in this study in-
clude stand alone machine learning applications and machine learn-
ing/mathematical optimization hybrid models. Machine learning mod-
els provide fast but not necessarily optimal responses in real time under
uncertainty. On the other hand, mathematical optimization models
(like mixed integer or dynamic programming) can provide optimal
solutions to many problem types. However, depending on the type
and scale of the problem, their response time may not be fast enough
to use in real time decision making. Combining machine learning
algorithms with mathematical optimization models and creating hybrid
models enables capturing the advantages of both worlds. Hybridiza-
tion methods include; using machine learning models to provide in-
puts/constraints to mathematical models, separating the main problem
into sub-problems and solving each problem with the most appropriate
model type, and training machine learning models with mathematical
model solutions. Charging service problems like scheduling and rout-
ing are combinatorial optimization problems where machine learning
can improve the already available optimization algorithms in several
ways [146]. Therefore, increasing hybrid studies can speed up EV
charging service planning research advancements.
Although they look separated, all charging service related research
areas are connected with each other. Current studies take each prob-
lem as a separate optimization problem to be solved with several
assumptions related to the other parts of the bigger picture. Better
solutions are possible with a holistic approach that interconnects in-
frastructure planning with coordination, pricing, and navigation. A top
down or bottom up iterative design framework can reduce the number
of assumptions required for each problem type and improve the real
world resemblance of the studies and the reliability of the solutions.
Simulation is a useful tool for determining model outcomes under
different conditions. Each step’s output/emergent requirements can be
feedback to the previous solution step. Fig. 7 proposes a framework for
this type of interconnected study.
New technologies and business models come with their own un-
certainties and response time requirements. Some research areas are
in their development phase, and the number of studies in these areas
needs to be expanded. Some of the areas that can benefit from machine
learning applications are as follows:
1. Infrastructure planning
• Wireless charging is considered an emergent technology
that will accelerate EV adoption [147]. Especially dynamic
charging, charging when the vehicle is en route, is a game
changing technology that will relieve the users’ range anx-
iety. Dynamic charging technology brings its own uncer-
tainties and requires additional infrastructure plans and
dynamic routing algorithms. However, studies in this area
are still limited.
• The number and capacity of renewable energy facilities are
increasing continuously. Including predicted regional re-
newable resource availability and charging stations’ ability
to deploy and use their renewable resources could improve
the infrastructure plans.
12
Renewable and Sustainable Energy Reviews 188 (2023) 113873
• Long term demand forecasting studies provide essential
inputs to infrastructure planning problems. Including new
features, such as government policy predictions, may im-
prove these forecasts’ accuracy.
• Infrastructure resilience during natural disasters such as
earthquakes or floods is another important research area.
The infrastructure requirements differ according to the
disaster type [148]. EVs both act as a resilience resource
for infrastructures, and require additional energy storage
systems/renewable energy for usage during power out-
ages [149]. Along with mixed integer [150] and multi
objective programming [151] based infrastructure opti-
mization studies, capacity simulation [152], EV prioritiza-
tion [153] and evacuation routing [154,155] studies exist
in the literature. Scheduling and routing the EVs and
mobile charging units are some of the problems that ma-
chine learning algorithms can help with under this highly
uncertain environment that requires a very fast response in
real time.
2. Charge control and coordination
• Scheduling and control algorithms generally aim to max-
imize the profit or minimize the costs of the aggrega-
tors/users. As the number of renewable energy facilities
and individual homes integrating solar energy grows, the
studies focusing on minimizing the carbon footprint of EVs
through renewable energy usage should also increase.
• When considering the V2G setting, another uncertainty
that arises is the willingness of EVs to participate in the
V2G system [14].
• Different business models like police and military have
different requirements and uncertainties than regular EV
fleets. Machine learning models can aid in this area.
3. Dynamic pricing
• Dynamic pricing models can promote renewable energy
usage and maximize renewable energy utilization.
• Integrating quality of service based dynamic pricing to fast
charging stations is another important research area with
additional uncertainties.
• Priority service at fast charging stations requires different
resource management models [156,157]. Dynamic pric-
ing models can improve their performance by balancing
different service type requests.
4. Routing - Navigation
• Fast charging stations are mostly preferred for the shorter
charge time they offer. However, when congestion oc-
curs, an accurate waiting time prediction is important to
determine the expected total time at each station.
• Wireless charging infrastructure brings additional uncer-
tainties to the routing problem.
8. Conclusion
Consumer perceived risks regarding the availability and quality of
electric vehicle charging services are important barriers in front of EV
adoption. Machine learning algorithms have a continuously growing
application area, including planning and service operations for electric
vehicle charging.
This study reviews electric vehicle charging infrastructure plan-
ning, charge control and coordination, dynamic pricing, and naviga-
tion/routing studies that use machine learning algorithms as a part
of the solution procedure. The study has several limitations. The
search was limited to the journal publications listed in Web of science
N. Fescioglu-Unver and M. Yıldız Aktaş Renewable and Sustainable Energy Reviews 188 (2023) 113873

Fig. 7. Iterative design framework connecting the research areas.

database, and it includes only the publications in English language. This The future research areas that can benefit from machine learning
study improves the current review studies literature in several ways. applications can be summarized as follows:
The primary focus is on charging service operations that effect the
• Infrastructure planning: Renewable energy usage comes with its
users’ EV adoption decisions. Therefore this study has a broad coverage
own uncertainties. Using governments’ renewable energy policies
area that includes several topics interrelated with each other. It covers
to forecast charging stations’ renewable energy storage/usage
the studies that use machine learning as the main problem solving
potential can effect infrastructure plans. Government policies on
method or as an aid to the main method. This study also proposes an
promoting EV deployment is an important factor in predicting
iterative design framework that connects these research topics.
long term demand. Infrastructure planning for resilience is an-
This review summarizes how machine learning aids problem solving
other topic that has limited studies but very important for pro-
in the current literature and provides future research directions. The
viding mobility during times that require immediate response.
key findings of this study shows that depending on the problem type,
• Charge control and coordination: Along with the rise in renewable
machine learning algorithms serve as the main solution method or
energy deployment, studies focusing on EV charge control for car-
provide inputs/constraints to the main method. When the problem
bon footprint reduction should also increase. In the V2G setting,
to be solved requires fast decision making in an environment with
forecasting the EV’s that will accept participation will improve
uncertainty, reinforcement learning class models provide successful
solution accuracy.
solutions. When an input to the main method has uncertainty and
• Dynamic pricing: Studies promoting renewable energy usage
depends on several other features, forecasting algorithms aid in pre-
through pricing, considering quality of service and different ser-
dicting it. When the problem scale gets bigger and it is not possible
vice level requirements for fast charging stations are important.
to solve with mathematical models within a reasonable time limit,
• Routing/Navigation: Fast charging station waiting time predic-
machine learning algorithms provide satisfactory solutions. Creating
tion is an important but frequently ignored topic. Wireless charg-
hybrid models with machine learning and mathematical models cap-
ing is another emergent research area with its additional uncer-
tures the advantages of both methodologies. Although infrastructure
tainties.
planning, charge control, pricing, and routing studies are interrelated,
they are treated separately with additional assumptions regarding other This study shows that machine learning is a powerful tool that
layers. This study proposes an iterative design framework that con- aids in all EV charging service operation levels. Government policies
nects these areas through simulation. Simulation is a valuable tool on EV support, renewable energy promotions, electricity pricing reg-
to determine the new requests and requirements that will emerge ulations are important factors that affect the demand and resource
under different conditions. When the EV charging service operations’s inputs/constraints of all stages. A holistic approach is required to
interrelated research layers give each other feedback, it will be possible enable EV widespread adoption. Policymakers should use an iterative
to create a holistic design with less assumptions and more resemblance design framework and follow a top down or bottom up approach
to the real-life. to evaluate the effects of their decisions before implementing them.

13
N. Fescioglu-Unver and M. Yıldız Aktaş
Researchers can include policy forecasts to obtain more realistic results.
Engineers can follow the advances in machine learning algorithms and
use hybrid models for successful designs.
Declaration of competing interest
The authors declare that they have no known competing finan-
cial interests or personal relationships that could have appeared to
influence the work reported in this paper.
Data availability
No data was used for the research described in the article.
References
[1] Electric vehicles report. Paris: International Energy Agency IEA; 2022, https:
//www.iea.org/reports/electric-vehicles (accessed Feb, 2023).
[2] Kumar RR, Alok K. Adoption of electric vehicle: A literature review and
prospects for sustainability. J Clean Prod 2020;253:119911.
[3] Featherman M, Jia SJ, Califf CB, Hajli N. The impact of new technologies on
consumers beliefs: Reducing the perceived risks of electric vehicle adoption.
Technol Forecast Soc Change 2021;169:120847.
[4] Hardman S, Jenn A, Tal G, Axsen J, Beard G, Daina N, Figenbaum E,
Jakobsson N, Jochem P, Kinnear N, et al. A review of consumer preferences
of and interactions with electric vehicle charging infrastructure. Transp Res D
2018;62:508–23.
[5] Globisch J, Plötz P, Dütschke E, Wietschel M. Consumer preferences for public
charging infrastructure for electric vehicles. Transp Policy 2019;81:54–63.
[6] Zhang B, Niu N, Li H, Wang Z, He W. Could fast battery charging effectively
mitigate range anxiety in electric vehicle usage? Evidence from large-scale data
on travel and charging in Beijing. Transp Res D 2021;95:102840.
[7] Baumgarte F, Kaiser M, Keller R. Policy support measures for widespread
expansion of fast charging infrastructure for electric vehicles. Energy Policy
2021;156:112372.
[8] Neaimeh M, Salisbury SD, Hill GA, Blythe PT, Scoffield DR, Francfort JE.
Analysing the usage and evidencing the importance of fast chargers for the
adoption of battery electric vehicles. Energy Policy 2017;108:474–86.
[9] Burnham A, Dufek EJ, Stephens T, Francfort J, Michelbacher C, Carl-
son RB, Zhang J, Vijayagopal R, Dias F, Mohanpurkar M, et al. Enabling
fast charging–Infrastructure and economic considerations. J Power Sources
2017;367:237–49.
[10] Funke SÁ, Plötz P, Wietschel M. Invest in fast-charging infrastructure or in
longer battery ranges? A cost-efficiency comparison for Germany. Appl Energy
2019;235:888–99.
[11] Philipsen R, Schmidt T, Van Heek J, Ziefle M. Fast-charging station here,
please! user criteria for electric vehicle fast-charging locations. Transp Res F
2016;40:119–29.
[12] Solanke TU, Khatua PK, Ramachandaramurthy VK, Yong JY, Tan KM. Control
and management of a multilevel electric vehicles infrastructure integrated with
distributed resources: A comprehensive review. Renew Sustain Energy Rev
2021;144:111020.
[13] Das HS, Rahman MM, Li S, Tan C. Electric vehicles standards, charging
infrastructure, and impact on grid integration: A technological review. Renew
Sustain Energy Rev 2020;120:109618.
[14] Sadeghian O, Oshnoei A, Mohammadi-Ivatloo B, Vahidinasab V, Anvari-
Moghaddam A. A comprehensive review on electric vehicles smart charg-
ing: Solutions, strategies, technologies, and challenges. J Energy Storage
2022;54:105241.
[15] Rahman I, Vasant PM, Singh BSM, Abdullah-Al-Wadud M, Adnan N. Review of
recent trends in optimization techniques for plug-in hybrid, and electric vehicle
charging infrastructures. Renew Sustain Energy Rev 2016;58:1039–47.
[16] Bilal M, Rizwan M. Electric vehicles in a smart grid: a comprehensive survey
on optimal location of charging station. IET Smart Grid 2020;3(3):267–79.
[17] Ahmad F, Saad Alam M, Saad Alsaidan I, Shariff SM. Battery swapping
station for electric vehicles: opportunities and challenges. IET Smart Grid
2020;3(3):280–6.
[18] Saharan S, Bawa S, Kumar N. Dynamic pricing techniques for Intelligent
Transportation System in smart cities: A systematic review. Comput Commun
2020;150:603–25.
[19] Dündar H, Ömürgönülşen M, Soysal M. A review on sustainable urban vehicle
routing. J Clean Prod 2021;285:125444.
[20] Shen Z-JM, Feng B, Mao C, Ran L. Optimization models for electric vehicle
service operations: A literature review. Transp Res B 2019;128:462–77.
[21] Elbouchikhi E, Zia MF, Benbouzid M, El Hani S. Overview of signal pro-
cessing and machine learning for smart grid condition monitoring. Electronics
2021;10(21):2725.
14
Renewable and Sustainable Energy Reviews 188 (2023) 113873
[22] Kumar NM, Chand AA, Malvoni M, Prasad KA, Mamun KA, Islam F, Chopra SS.
Distributed energy resources and the application of AI, IoT, and blockchain in
smart grids. Energies 2020;13(21):5739.
[23] Ali SS, Choi BJ. State-of-the-art artificial intelligence techniques for distributed
smart grids: A review. Electronics 2020;9(6):1030.
[24] Song C, Kim K, Sung D, Kim K, Yang H, Lee H, Cho GY, Cha SW. A review of
optimal energy management strategies using machine learning techniques for
hybrid electric vehicles. Int J Automot Technol 2021;22:1437–52.
[25] Pevec D, Babic J, Podobnik V. Electric vehicles: A data science perspective
review. Electronics 2019;8(10):1190.
[26] Amara-Ouali Y, Goude Y, Massart P, Poggi J-M, Yan H. A review of electric
vehicle load open data and models. Energies 2021;14(8):2233.
[27] Al-Ogaili AS, Hashim TJT, Rahmat NA, Ramasamy AK, Marsadek MB, Faisal M,
Hannan MA. Review on scheduling, clustering, and forecasting strategies for
controlling electric vehicle charging: Challenges and recommendations. IEEE
Access 2019;7:128353–71.
[28] Abdullah HM, Gastli A, Ben-Brahim L. Reinforcement learning based EV
charging management systems–a review. IEEE Access 2021;9:41506–31.
[29] Deb S. Machine learning for solving charging infrastructure planning problems:
A comprehensive review. Energies 2021;14(23):7833.
[30] Shahriar S, Al-Ali A-R, Osman AH, Dhou S, Nijim M. Machine learning
approaches for EV charging behavior: A review. IEEE Access 2020;8:168980–93.
[31] Sanguesa JA, Torres-Sanz V, Garrido P, Martinez FJ, Marquez-Barja JM.
A review on electric vehicles: Technologies and challenges. Smart Cities
2021;4(1):372–404.
[32] Leijon J, Boström C. Charging electric vehicles today and in the future. World
Electr Veh J 2022;13(8):139.
[33] LaMonaca S, Ryan L. The state of play in electric vehicle charging services–A
review of infrastructure provision, players, and policies. Renew Sustain Energy
Rev 2022;154:111733.
[34] Mohammed SAQ, Jung J-W. A comprehensive state-of-the-art review of
wired/wireless charging technologies for battery electric vehicles: Classifica-
tion/common topologies/future research issues. IEEE Access 2021;9:19572–85.
[35] Ahmad A, Alam MS, Chabaan R. A comprehensive review of wireless charging
technologies for electric vehicles. IEEE Trans Transp Electrif 2017;4(1):38–63.
[36] Triviño A, Gonzalez-Gonzalez JM, Castilla M. Review on control techniques for
EV bidirectional wireless chargers. Electronics 2021;10(16):1905.
[37] Wang J, Shao Y, Ge Y, Yu R. A survey of vehicle to everything (V2X) testing.
Sensors 2019;19(2):334.
[38] Pearre NS, Ribberink H. Review of research on V2X technologies, strategies,
and operations. Renew Sustain Energy Rev 2019;105:61–70.
[39] Afshar S, Macedo P, Mohamed F, Disfani V. Mobile charging stations for electric
vehicles—A review. Renew Sustain Energy Rev 2021;152:111654.
[40] Deb S, Tammi K, Kalita K, Mahanta P. Review of recent trends in charging
infrastructure planning for electric vehicles. Wiley Interdiscip Rev: Energy
Environ 2018;7(6):e306.
[41] Metais M-O, Jouini O, Perez Y, Berrada J, Suomalainen E. Too much or not
enough? Planning electric vehicle charging infrastructure: A review of modeling
options. Renew Sustain Energy Rev 2022;153:111719.
[42] Xu D, Tian Y. A comprehensive survey of clustering algorithms. Ann Data Sci
2015;2:165–93.
[43] Andrenacci N, Ragona R, Valenti G. A demand-side approach to the optimal
deployment of electric vehicle charging stations in metropolitan areas. Appl
Energy 2016;182:39–46.
[44] Kazemi MA, Sedighizadeh M, Mirzaei MJ, Homaee O. Optimal siting and
sizing of distribution system operator owned EV parking lots. Appl Energy
2016;179:1176–84.
[45] Wang N, Wang C, Niu Y, Yang M, Yu Y. A two-stage charging facilities
planning method for electric vehicle sharing systems. IEEE Trans Ind Appl
2020;57(1):149–57.
[46] Marino CA, Marufuzzaman M. Unsupervised learning for deploying smart
charging public infrastructure for electric vehicles in sprawling cities. J Clean
Prod 2020;266:121926.
[47] Bae S, Jang I, Gros S, Kulcsár B, Hellgren J. A game approach for charging
station placement based on user preferences and crowdedness. IEEE Trans Intell
Transp Syst 2020;23(4):3654–69.
[48] Liu J-p, Zhang T-x, Zhu J, Ma T-n. Allocation optimization of electric vehicle
charging station (EVCS) considering with charging satisfaction and distributed
renewables integration. Energy 2018;164:560–74.
[49] Sánchez DG, Tabares A, Faria LT, Rivera JC, Franco JF. A clustering approach
for the optimal siting of recharging stations in the electric vehicle routing
problem with time windows. Energies 2022;15(7):2372.
[50] Zeng M, Pan Y, Zhang D, Lu Z, Li Y. Data-driven location selection for battery
swapping stations. IEEE Access 2019;7:133760–71.
[51] Bitencourt L, Abud TP, Dias BH, Borba BS, Maciel RS, Quirós-Tortós J. Optimal
location of EV charging stations in a neighborhood considering a multi-objective
approach. Electr Power Syst Res 2021;199:107391.
[52] Yao CZ, Luo J. Deployment of a charging network based on a cost-benefit model
and the CFSFDP algorithm. Kybernetes 2018.
N. Fescioglu-Unver and M. Yıldız Aktaş
[53] Momtazpour M, Butler P, Ramakrishnan N, Hossain MS, Bozchalui MC,
Sharma R. Charging and storage infrastructure design for electric vehicles. ACM
Trans Intell Syst Technol 2014;5(3):1–27.
[54] Wang C, Xie D, Gu C, Fan Y. Planning of regional urban bus charging
facility: A case study of Fengxian, Shanghai. IEEE Trans Intell Transp Syst
2021;23(8):13592–603.
[55] Cilio L, Babacan O. Allocation optimisation of rapid charging stations in large
urban areas to support fully electric taxi fleets. Appl Energy 2021;295:117072.
[56] Ko YD. An efficient integration of the genetic algorithm and the reinforcement
learning for optimal deployment of the wireless charging electric tram system.
Comput Ind Eng 2019;128:851–60.
[57] Hu D, Zhang J, Liu Z-W. Charging stations expansion planning under govern-
ment policy driven based on Bayesian regularization backpropagation learning.
Neurocomputing 2020;416:47–58.
[58] Jiang C, Jing Z, Ji T, Wu Q. Optimal location of PEVCSs using MAS and ER
approach. IET Gener Transm Distrib 2018;12(20):4377–87.
[59] Hecht C, Figgener J, Sauer DU. Predicting electric vehicle charging station
availability using ensemble machine learning. Energies 2021;14(23):7834.
[60] Ostermann A, Fabel Y, Ouan K, Koo H. Forecasting charging point occupancy
using supervised learning algorithms. Energies 2022;15(9):3409.
[61] Friese PA, Michalk W, Fischer M, Hardt C, Bogenberger K. Charging point
usage in germany—automated retrieval, analysis, and usage types explained.
Sustainability 2021;13(23):13046.
[62] Aduama P, Zhang Z, Al-Sumaiti AS. Multi-feature data fusion-based load
forecasting of electric vehicle charging stations using a deep learning model.
Energies 2023;16(3):1309.
[63] Yi Z, Liu XC, Wei R, Chen X, Dai J. Electric vehicle charging demand forecasting
using deep learning model. J Intell Transp Syst 2022;26(6):690–703.
[64] Verma A, Asadi A, Yang K, Maitra A, Asgeirsson H. Analyzing household
charging patterns of Plug-in electric vehicles (PEVs): A data mining approach.
Comput Ind Eng 2019;128:964–73.
[65] Kim Y, Kim S. Forecasting charging demand of electric vehicles using time-series
models. Energies 2021;14(5):1487.
[66] Wang Q, Liu X, Du J, Kong F. Smart charging for electric vehicles:
A survey from the algorithmic perspective. IEEE Commun Surv Tutor
2016;18(2):1500–17.
[67] Yang Z, Li K, Foley A. Computational scheduling methods for integrating plug-
in electric vehicles with power systems: A review. Renew Sustain Energy Rev
2015;51:396–416.
[68] Nimalsiri NI, Mediwaththe CP, Ratnam EL, Shaw M, Smith DB, Halgamuge SK.
A survey of algorithms for distributed charging control of electric vehicles in
smart grid. IEEE Trans Intell Transp Syst 2019;21(11):4497–515.
[69] Khalid MR, Khan IA, Hameed S, Asghar MSJ, Ro J-S. A comprehensive review
on structural topologies, power levels, energy storage systems, and standards
for electric vehicle charging stations and their impacts on grid. IEEE Access
2021;9:128069–94.
[70] Shaukat N, Khan B, Ali S, Mehmood C, Khan J, Farid U, Majid M, Anwar S,
Jawad M, Ullah Z. A survey on electric vehicle transportation within smart grid
system. Renew Sustain Energy Rev 2018;81:1329–49.
[71] Gonçalves F, de Abreu Borges L, Batista R. Electric vehicle charging data
analytics of corporate fleets. World Electr Veh J 2022;13(12):237.
[72] Li H, Wan Z, He H. Constrained EV charging scheduling based on safe deep
reinforcement learning. IEEE Trans Smart Grid 2019;11(3):2427–39.
[73] Yan L, Chen X, Zhou J, Chen Y, Wen J. Deep reinforcement learning for
continuous electric vehicles charging control with dynamic user behaviors. IEEE
Trans Smart Grid 2021;12(6):5124–34.
[74] Li S, Hu W, Cao D, Dragičević T, Huang Q, Chen Z, Blaabjerg F. Electric vehicle
charging management based on deep reinforcement learning. J Mod Power Syst
Clean Energy 2021;10(3):719–30.
[75] Chang F, Chen T, Su W, Alsafasfeh Q. Control of battery charging based on
reinforcement learning and long short-term memory networks. Comput Electr
Eng 2020;85:106670.
[76] Wang F, Gao J, Li M, Zhao L. Autonomous PEV charging scheduling using
Dyna-Q reinforcement learning. IEEE Trans Veh Technol 2020;69(11):12609–
20.
[77] Zhang F, Yang Q, An D. CDDPG: A deep-reinforcement-learning-based approach
for electric vehicle charging control. IEEE Internet Things J 2020;8(5):3075–87.
[78] Wan Z, Li H, He H, Prokhorov D. Model-free real-time EV charging
scheduling based on deep reinforcement learning. IEEE Trans Smart Grid
2018;10(5):5246–57.
[79] Zhang Z, Wan Y, Qin J, Fu W, Kang Y. A deep rl-based algorithm for
coordinated charging of electric vehicles. IEEE Trans Intell Transp Syst
2022;23(10):18774–84.
[80] Chiş A, Lundén J, Koivunen V. Reinforcement learning-based plug-in elec-
tric vehicle charging with forecasted price. IEEE Trans Veh Technol
2016;66(5):3674–84.
[81] López KL, Gagné C, Gardner M-A. Demand-side management using deep
learning for smart charging of electric vehicles. IEEE Trans Smart Grid
2018;10(3):2683–91.
15
Renewable and Sustainable Energy Reviews 188 (2023) 113873
[82] Liang Y, Ding Z, Ding T, Lee W-J. Mobility-aware charging scheduling for shared
on-demand electric vehicle fleet using deep reinforcement learning. IEEE Trans
Smart Grid 2020;12(2):1380–93.
[83] Vandael S, Claessens B, Ernst D, Holvoet T, Deconinck G. Reinforcement
learning of heuristic EV fleet charging in a day-ahead electricity market. IEEE
Trans Smart Grid 2015;6(4):1795–805.
[84] Arif A, Babar M, Ahamed TI, Al-Ammar E, Nguyen P, Kamphuis IR, Malik N.
Online scheduling of plug-in vehicles in dynamic pricing schemes. Sustain
Energy Grids Netw 2016;7:25–36.
[85] Cao Y, Wang H, Li D, Zhang G. Smart online charging algorithm for elec-
tric vehicles via customized actor–critic learning. IEEE Internet Things J
2021;9(1):684–94.
[86] Cao Y, Li D, Zhang Y, Chen X. Joint optimization of delay-tolerant autonomous
electric vehicles charge scheduling and station battery degradation. IEEE
Internet Things J 2020;7(9):8590–9.
[87] Lee J, Lee E, Kim J. Electric vehicle charging and discharging algorithm
based on reinforcement learning with data-driven approach in dynamic pricing
scheme. Energies 2020;13(8):1950.
[88] Wan Y, Qin J, Ma Q, Fu W, Wang S. Multi-agent DRL-based data-driven
approach for PEVs charging/discharging scheduling in smart grid. J Franklin
Inst B 2022;359(4):1747–67.
[89] Ye Z, Gao Y, Yu N. Learning to operate an electric vehicle charg-
ing station considering vehicle-grid integration. IEEE Trans Smart Grid
2022;13(4):3038–48.
[90] Aljafari B, Jeyaraj PR, Kathiresan AC, Thanikanti SB. Electric vehicle optimum
charging-discharging scheduling with dynamic pricing employing multi agent
deep neural network. Comput Electr Eng 2023;105:108555.
[91] Wang R, Chen Z, Xing Q, Zhang Z, Zhang T. A modified rainbow-based deep
reinforcement learning method for optimal scheduling of charging station.
Sustainability 2022;14(3):1884.
[92] Shin M, Choi D-H, Kim J. Cooperative management for PV/ESS-enabled electric
vehicle charging stations: A multiagent deep reinforcement learning approach.
IEEE Trans Ind Inf 2019;16(5):3493–503.
[93] Jin J, Xu Y. Optimal policy characterization enhanced actor-critic approach
for electric vehicle charging scheduling in a power distribution network. IEEE
Trans Smart Grid 2020;12(2):1416–28.
[94] Liu J, Lin G, Rehtanz C, Huang S, Zhou Y, Li Y. Data-driven intelligent EV
charging operating with limited chargers considering the charging demand
forecasting. Int J Electr Power Energy Syst 2022;141:108218.
[95] Yan L, Chen X, Chen Y, Wen J. A cooperative charging control strategy for
electric vehicles based on multiagent deep reinforcement learning. IEEE Trans
Ind Inf 2022;18(12):8765–75.
[96] Ming F, Gao F, Liu K, Li X. A constrained DRL-based bi-level coordinated
method for large-scale EVs charging. Appl Energy 2023;331:120381.
[97] Tao Y, Qiu J, Lai S. Deep reinforcement learning based bidding strategy for
EVAs in local energy market considering information asymmetry. IEEE Trans
Ind Inf 2021;18(6):3831–42.
[98] Lin J, Dong P, Liu M, Huang X, Deng W. Research on demand response of
electric vehicle agents based on multi-layer machine learning algorithm. IEEE
Access 2020;8:224224–34.
[99] Asadi A, Pinkley SN. A stochastic scheduling, allocation, and inven-
tory replenishment problem for battery swap stations. Transp Res E
2021;146:102212.
[100] Gao Y, Yang J, Yang M, Li Z. Deep reinforcement learning based optimal
schedule for a battery swapping station considering uncertainties. IEEE Trans
Ind Appl 2020;56(5):5775–84.
[101] Wang X, Wang J, Liu J. Vehicle to grid frequency regulation capacity optimal
scheduling for battery swapping station using deep Q-network. IEEE Trans Ind
Inf 2020;17(2):1342–51.
[102] Liang Y, Ding Z, Zhao T, Lee W-J. Real-time operation management for battery
swapping-charging system via multi-agent deep reinforcement learning. IEEE
Trans Smart Grid 2022;14(1):559–71.
[103] Tan M, Dai Z, Su Y, Chen C, Wang L, Chen J. Bi-level optimization of charging
scheduling of a battery swap station based on deep reinforcement learning. Eng
Appl Artif Intell 2023;118:105557.
[104] Jin J, Mao S, Xu Y. Optimal priority rule enhanced deep reinforcement learning
for charging scheduling in an electric vehicle battery swapping station. IEEE
Trans Smart Grid 2023.
[105] Limmer S. Dynamic pricing for electric vehicle charging—A literature review.
Energies 2019;12(18):3574.
[106] Wang S, Bi S, Zhang YA. Reinforcement learning for real-time pric-
ing and scheduling control in EV charging stations. IEEE Trans Ind Inf
2021;17(2):849–59.
[107] Paraskevas A, Aletras D, Chrysopoulos A, Marinopoulos A, Doukas DI. Opti-
mal management for EV charging stations: A win–win strategy for different
stakeholders using constrained deep Q-learning. Energies 2022;15(7):2323.
[108] Moghaddam V, Yazdani A, Wang H, Parlevliet D, Shahnia F. An online rein-
forcement learning approach for dynamic pricing of electric vehicle charging
stations. IEEE Access 2020;8:130305–13.
N. Fescioglu-Unver and M. Yıldız Aktaş
[109] Fang C, Lu H, Hong Y, Liu S, Chang J. Dynamic pricing for electric vehicle
extreme fast charging. IEEE Trans Intell Transp Syst 2020;22(1):531–41.
[110] Qian T, Shao C, Li X, Wang X, Chen Z, Shahidehpour M. Multi-agent deep
reinforcement learning method for EV charging station game. IEEE Trans Power
Syst 2021;37(3):1682–94.
[111] Abdalrahman A, Zhuang W. Dynamic pricing for differentiated PEV charging
services using deep reinforcement learning. IEEE Trans Intell Transp Syst
2020;23(2):1415–27.
[112] Zhao Z, Lee CK. Dynamic pricing for EV charging stations: A deep reinforcement
learning approach. IEEE Trans Transp Electrif 2021;8(2):2456–68.
[113] Zhang Y, Zhang Z, Yang Q, An D, Li D, Li C. EV charging bidding by multi-
DQN reinforcement learning in electricity auction market. Neurocomputing
2020;397:404–14.
[114] Kim B-G, Zhang Y, Van Der Schaar M, Lee J-W. Dynamic pricing and energy
consumption scheduling with reinforcement learning. IEEE Trans Smart Grid
2015;7(5):2187–98.
[115] Qiu D, Ye Y, Papadaskalopoulos D, Strbac G. A deep reinforcement learning
method for pricing electric vehicles with discrete charging levels. IEEE Trans
Ind Appl 2020;56(5):5901–12.
[116] Liu D, Wang W, Wang L, Jia H, Shi M. Dynamic pricing strategy of electric
vehicle aggregators based on DDPG reinforcement learning algorithm. IEEE
Access 2021;9:21556–66.
[117] Kucukoglu I, Dewil R, Cattrysse D. The electric vehicle routing problem and its
variations: A literature review. Comput Ind Eng 2021;161:107650.
[118] Xiao Y, Zhang Y, Kaku I, Kang R, Pan X. Electric vehicle routing problem:
A systematic review and a new comprehensive model with nonlinear energy
recharging and consumption. Renew Sustain Energy Rev 2021;151:111567.
[119] James J, Yu W, Gu J. Online vehicle routing with neural combinatorial
optimization and deep reinforcement learning. IEEE Trans Intell Transp Syst
2019;20(10):3806–17.
[120] Basso R, Kulcsár B, Sanchez-Diaz I, Qu X. Dynamic stochastic electric vehicle
routing with safe reinforcement learning. Transp Res E 2022;157:102496.
[121] Shi J, Gao Y, Wang W, Yu N, Ioannou PA. Operating electric vehicle fleet for
ride-hailing services with reinforcement learning. IEEE Trans Intell Transp Syst
2019;21(11):4822–34.
[122] Kullman ND, Cousineau M, Goodson JC, Mendoza JE. Dynamic ride-hailing with
electric vehicles. Transp Sci 2022;56(3):775–94.
[123] Lin B, Ghaddar B, Nathwani J. Deep reinforcement learning for the electric
vehicle routing problem with time windows. IEEE Trans Intell Transp Syst
2021;23(8):11528–38.
[124] Zhang L, Gong K, Xu M. Congestion control in charging stations allocation with
Q-learning. Sustainability 2019;11(14):3900.
[125] Qian T, Shao C, Wang X, Shahidehpour M. Deep reinforcement learning for EV
charging navigation by coordinating smart grid and intelligent transportation
system. IEEE Trans Smart Grid 2019;11(2):1714–23.
[126] Lee K-B, A. Ahmed M, Kang D-K, Kim Y-C. Deep reinforcement learning based
optimal route and charging station selection. Energies 2020;13(23):6255.
[127] Xu P, Zhang J, Gao T, Chen S, Wang X, Jiang H, Gao W. Real-time fast charging
station recommendation for electric vehicles in coupled power-transportation
networks: A graph reinforcement learning method. Int J Electr Power Energy
Syst 2022;141:108030.
[128] Tao Y, Qiu J, Lai S, Sun X, Zhao J, Zhou B, Cheng L. Data-driven
on-demand energy supplement planning for electric vehicles considering
multi-charging/swapping services. Appl Energy 2022;311:118632.
[129] Suanpang P, Jamjuntr P, Kaewyong P, Niamsorn C, Jermsittiparsert K. An intel-
ligent recommendation for intelligently accessible charging stations: Electronic
vehicle charging to support a sustainable smart tourism city. Sustainability
2023;15(1):455.
[130] Jin J, Xu Y. Shortest-path-based deep reinforcement learning for EV charging
routing under stochastic traffic condition and electricity prices. IEEE Internet
Things J 2022;9(22):22571–81.
[131] Thibault L, De Nunzio G, Sciarretta A. A unified approach for electric vehicles
range maximization via eco-routing, eco-driving, and energy consumption
prediction. IEEE Trans Intell Veh 2018;3(4):463–75.
[132] Shukla RM, Sengupta S. Cop: An integrated communication, optimization, and
prediction unit for smart plug-in electric vehicle charging. Internet Things
2020;9:100148.
16
Renewable and Sustainable Energy Reviews 188 (2023) 113873
[133] Huang C-J, Hu K-W, Ho H-Y, Chuang H-W. Congestion-preventing routing and
charging scheduling mechanism for electric vehicles in dense urban areas. Inf
Technol Control 2021;50(2):284–307.
[134] Huang C-J, Hu K-W, Ho HY, Xie BZ, Feng C-C, Chuang H-W. A distributed urban
traffic congestion prevention mechanism for mixed flow of human-driven and
autonomous electric vehicles. Int J Comput Intell Syst 2021;14(1):1714–27.
[135] Basso R, Kulcsár B, Sanchez-Diaz I. Electric vehicle routing problem with
machine learning for energy prediction. Transp Res B 2021;145:24–55.
[136] Lu Y, Gu J, Xie D, Li Y. Integrated route planning algorithm based on spot price
and classified travel objectives for EV users. IEEE Access 2019;7:122238–50.
[137] Zhang Q, Wu K, Shi Y. Route planning and power management for PHEVs with
reinforcement learning. IEEE Trans Veh Technol 2020;69(5):4751–62.
[138] Aljohani TM, Ebrahim A, Mohammed O. Real-time metadata-driven routing
optimization for electric vehicle energy consumption minimization using deep
reinforcement learning and Markov chain model. Electr Power Syst Res
2021;192:106962.
[139] Aljohani TM, Mohammed O. A real-time energy consumption minimiza-
tion framework for electric vehicles routing optimization based on SARSA
reinforcement learning. Vehicles 2022;4(4):1176–94.
[140] Dorokhova M, Ballif C, Wyrsch N. Routing of electric vehicles with interme-
diary charging stations: A reinforcement learning approach. Front Big Data
2021;4:586481.
[141] Bi H, Shang W-L, Chen Y, Wang K, Yu Q, Sui Y. GIS aided sustainable urban
road management with a unifying queueing and neural network model. Appl
Energy 2021;291:116818.
[142] Bozorgi AM, Farasat M, Mahmoud A. A time and energy efficient routing
algorithm for electric vehicles based on historical driving data. IEEE Trans Intell
Veh 2017;2(4):308–20.
[143] Ayman A, Sivagnanam A, Wilbur M, Pugliese P, Dubey A, Laszka A. Data-driven
prediction and optimization of energy use for transit fleets of electric and ICE
vehicles. ACM Trans Internet Technol (TOIT) 2021;22(1):1–29.
[144] Di Martino A, Miraftabzadeh SM, Longo M. Strategies for the modelisation of
electric vehicle energy consumption: A review. Energies 2022;15(21):8115.
[145] Varga BO, Sagoian A, Mariasiu F. Prediction of electric vehicle range:
A comprehensive review of current issues and challenges. Energies
2019;12(5):946.
[146] Bengio Y, Lodi A, Prouvost A. Machine learning for combinatorial optimization:
a methodological tour d’horizon. European J Oper Res 2021;290(2):405–21.
[147] Taiebat M, Xu M. Synergies of four emerging technologies for accelerated
adoption of electric vehicles: Shared mobility, wireless charging, vehicle-to-grid,
and vehicle automation. J Clean Prod 2019;230:794–7.
[148] Adderly SA, Manukian D, Sullivan TD, Son M. Electric vehicles and natural
disaster policy implications. Energy Policy 2018;112:437–48.
[149] Hussain A, Musilek P. Resilience enhancement strategies for and through
electric vehicles. Sustainable Cities Soc 2022;80:103788.
[150] Wang H, Fang Y-P, Zio E. Resilience-oriented optimal post-disruption
reconfiguration for coupled traffic-power systems. Reliab Eng Syst Saf
2022;222:108408.
[151] Ebadat-Parast M, Nazari MH, Hosseinian SH. Distribution system resilience
enhancement through resilience-oriented optimal scheduling of multi-microgrids
considering normal and emergency conditions interlink utilizing multi-objective
programming. Sustainable Cities Soc 2022;76:103467.
[152] MacDonald CD, Kattan L, Layzell D. Modelling electric vehicle charging network
capacity and performance during short-notice evacuations. Int J Disaster Risk
Reduct 2021;56:102093.
[153] Hussain A, Kim H-M. EV prioritization and power allocation during outages: A
lexicographic method-based multiobjective optimization approach. IEEE Trans
Transp Electrif 2021;7(4):2474–87.
[154] Li Q, Soleimaniamiri S, Li X. Optimal mass evacuation planning for electric
vehicles before natural disasters. Transp Res D 2022;107:103292.
[155] Zhang J, Zhang X. A multi-trip electric bus routing model considering equity
during short-notice evacuations. Transp Res D 2022;110:103397.
[156] Kakillioglu EA, Aktaş MY, Fescioglu-Unver N. Self-controlling resource man-
agement model for electric vehicle fast charging stations with priority service.
Energy 2022;239:122276.
[157] Fescioglu-Unver N, Aktaş MY, Kasnakoğlu C. Feedback controlled resource
management model for express service in electric vehicle charging stations.
J Clean Prod 2021;311:127629.