This article has been accepted for publication in IEEE Transactions on Industrial Informatics.
This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TII.2023.3268760
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. XX, NO. XX, XXXX
A Cooperative Evolutionary Computation
Algorithm for Dynamic Multiobjective Multi-AUV
Path Planning
Xiao-Fang Liu, Member, IEEE, Yongchun Fang, Senior Member, IEEE, Zhi-Hui Zhan, Senior Member,
IEEE, Yun-Liang Jiang, Jun Zhang, Fellow, IEEE
Abstract— Multiple autonomous underwater vehicles
(AUVs) are popular for executing submarine missions,
which involve multiple targets distributed in a large and
complex underwater environment. The path planning of
multiple AUVs is a significant and challenging problem,
which determines the location of surface points for AUV
launch and plans the paths of AUVs for target travel-
ing. Most existing works model the problem as a single-
objective static optimization problem. However, the target
missions may change over time, and multiple optimiza-
tion objectives are usually expected for decision making.
Thus, this paper models the problem as a dynamic multiob-
jective optimization problem and proposes a cooperative
evolutionary computation algorithm to provide diverse and
high-quality solutions for decision makers. In the proposed
method, solutions are represented using a bi-layer encode
scheme, in which the first layer indicates the surface lo-
cation points and the second layer represents the trav-
eling sequences of target missions. Multiple populations
for multiple objectives (MPMO) framework is adopted to
efficiently solve the dynamic multiobjective AUV optimiza-
tion problem. In addition, a recombination-based sampling
strategy is developed to improve convergence by fusing
the information of multiple populations. Once a change
occurs, an incremental response strategy is adopted to
generate high-quality solutions for population evolution.
Based on the dataset of New Zealand bathymetry, six
complex underwater scenarios are constructed with a size
of 50km×50km×10km and 400 target missions for tests.
Experimental results show that the proposed method out-
performs the state-of-the-art algorithms in terms of solution
Manuscript . . . This work was supported in part by the National
Natural Science Foundation of China (NSFC) under Grants 62103202,
61903139, and U22A20102, and in part by the Natural Science Founda-
tion of Tianjin under Grant 21JCQNJC00140. (Corresponding authors:
Zhi-Hui Zhan and Jun Zhang)
Xiao-Fang Liu and Yongchun Fang are with the Institute of Robotics
and Automatic Information System, College of Artificial Intelligence,
Nankai University, and also with the Tianjin Key Laboratory of Intelli-
gent Robotics, Nankai University, Tianjin 300350, China (email: liuxiao-
fang@nankai.edu.cn; fangyc@nankai.edu.cn).
Zhi-Hui Zhan is with the College of Artificial Intelligence, Nankai
University, Tianjin 300350, China, and also with the School of Com-
puter Science and Engineering, South China University of Technology,
Guangzhou 510006, China (e-mail: zhanapollo@163.com).
Yun-Liang Jiang is with the School of Computer Science and Tech-
nology, Zhejiang Normal University, Jinhua 321004, Zhejiang, China,
and also with the School of Information Engineering, Huzhou University,
Huzhou 313000, Zhejiang, China (email: jyl2022@zjnu.cn).
Jun Zhang is with Hanyang University, Ansan 15588, South Korea,
and also with Zhejiang Normal University, Jinhua 321004, China (e-mail:
junzhang@ieee.org).
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: SSEN. Downloaded on May 06,2023 at 09:13:06 UTC from IEEE Xplore. Restrictions apply.
1
diversity and optimality.
Index Terms— Dynamic multiobjective; multi-AUV path
planning; differential evolution; ant colony system; evo-
lutionary computation; multiple populations for multiple
objectives
I. I NTRODUCTION
Autonomous underwater vehicles (AUVs) are qualified for
a wide range of subsea missions [1], such as deep surveys of
hydrothermal vent fields [2] and underwater communication
[3]. Due to the limitation of a single AUV in terms of
energy and capacities, multiple AUVs usually cooperatively
work to complete complex missions in a large underwater
environment [4]. The target missions can be decomposed into
multiple subsets and each subset is assigned to one AUV for
execution. For example, in monitoring scenarios, a group of
AUVs are used to visit particular targets in parallel for quickly
investigating the whole area [5]; in tactical edge networks,
AUVs work together to achieve a high degree of coordination
and communications [6]. To save energy, AUVs are usually
launched from movable platforms, e.g., unmanned surface
vehicle, on the ocean rather than a coast. The movable surface
vehicles and AUVs form a multi-AUV system to fulfill target
missions. To build such a system, the path planning of multiple
AUVs is one of challenging problems, especially in a large
underwater environment with complex bathymetry, irregular
obstacles, and risky areas.
The multi-AUV path planning problem aims to locate the
surface points and find the optimal paths of AUVs to travel
target missions. In the literature, many research efforts are
devoted to developing various methods [7]. Most of them
do not consider the location of surface points and solve the
mission assignment and path planning of AUVs separately.
Particularly, the mission assignment problem is NP-hard, and
the methods for mission assignment can be roughly classified
into four categories, i.e., mixed-integer linear programming
[8], market-based methods [9], evolutionary computation [10],
and learning-based methods [11]. Among these methods,
mixed-integer linear programming usually requires a long
time for instances with a large number of target missions.
Market-based methods use auction mechanisms to determine
the AUV assigned to each target mission. They are simple
 This article has been accepted for publication in IEEE Transactions on Industrial Informatics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TII.2023.3268760
2 IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. XX, NO. XX, XXXX
but may not find the optimal solutions since they use local
information only. Evolutionary computation models the mis-
sion assignment as a combinatorial optimization problem and
adopts multiple strategies for population evolution to approx-
imate the optimal solutions, such as genetic algorithm. These
methods have shown good performance especially on large-
scale instances. Learning-based methods use machine learning
techniques to assign missions, such as Q-learning [12] and
self-organizing map [13]. The performance of these algorithms
depends greatly on the training data and the learning strategy.
Given the assigned missions, the paths of multiple AUVs are
planned using a multidimensional random tree star algorithm
[14]. However, the method is task-related. A genetic algorithm
is employed to plan paths of AUVs for sensor placement [15].
Differential evolution is adopted to plan the path of each AUV
according to the estimated environmental scalar field [16],
[17]. These methods are designed for small-scale missions.
In addition, the Q-learning method is applied to AUV path
planning for reducing the energy consumption and travel
distance of AUVs [11]. An on-policy reinforcement learning
approach plans paths to target missions without any collisions
by gaining information and rewards from the environment
[18]. However, the trained network may not work well in new
environments that are quite different from the trained ones.
There are also some works deal with the mission assignment
and path planning simultaneously. For example, a clustering
method is adopted for task assignment and ant colony opti-
mization is used to construct paths of multiple AUVs [19]. A
biologically inspired neural network is adopted to select one
AUV and plan the path to each target mission according to
the active values in the grid maps of underwater environments
[20]. However, the method is only applied to a simple un-
derwater scenario with a size of 10m×10m×10m. The risky
areas and other features of complex underwater environments
are not concerned. In addition, reinforcement learning provides
rewards of allocation strategies and waypoints to guide particle
swarm optimization to find the optimal strategy and paths in
a real-time rescue system [21]. However, the training samples
of reinforcement learning are rare and only small-scale target
missions are concerned.
To our best knowledge, most of existing works consider
one optimization objective only, such as energy consumption,
load balance, and mission running time. However, multiple
optimization objectives are usually expected by decision mak-
ers. For example, energy consumption and load balance are
usually concerned in complex scenarios for efficiently utilizing
the AUVs. In addition, these works often assume that the
target missions are invariant. Indeed, new target missions
can be discovered constantly, such as in rescue scenarios,
and they may also change due to environmental factors.
Thus, this paper studies the multi-AUV path planning with
multiple optimization objectives and changing target missions
in complex underwater environments.
This paper models the multi-AUV path planning as a
dynamic multiobjective optimization problem and proposes a
cooperative evolutionary computation algorithm, named CEA,
to solve it. In the proposed method, the underwater envi-
ronment is converted into a search graph for surface point
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: SSEN. Downloaded on May 06,2023 at 09:13:06 UTC from IEEE Xplore. Restrictions apply.
v1 S v2S v3 S v5 S
v1 S v2S v3 S v5 S
v1 T
v2T v3T
v1 T v2T v3T
(a) G (b) G0
Fig. 1. An example of environment modeling: (a) the search graph G,
(b) the cost map G0 , where VS = {v1S , v2S , v3S , v5S } is the set of can-
didate points for AUV launch and recovery, and VT = {v1T , v2T , v3T }
is the set of target vertices.
location and path planning. Multiple populations are adopted
to find the optimal solutions for maintaining population di-
versity since the multiple populations for multiple objectives
(MPMO) framework has shown good performance on multiob-
jective optimization [22]. The MPMO framework creates one
population for every objective and the multiple populations
coevolve towards the Pareto fronts [23]. Particularly, solutions
are represented using a bi-level encode scheme, which includes
the surface location points in the first layer and the voyages
of AUVs in the second layer. In addition, a recombination-
based sampling strategy is developed to fuse information from
multiple populations for improving algorithm convergence.
Once a change occurs, an incremental response strategy is
adopted to update the outdated populations using historical
information. Experiments are performed on six complex sce-
narios, which are constructed with irregular obstacles and
risky areas based on the dataset of New Zealand bathymetry.
The experimental results show the advantage of the proposed
method over existing methods in terms of solution diversity
and optimality.
The contributions of the work are summarized as follows:
• A new dynamic multiobjective optimization model is
developed for multi-AUV path planning.
• A new cooperative evolutionary computation algorithm is
proposed to plan the paths of multiple AUVs, which visit
a large number of changing target missions in a complex
environment.
The paper is organized as follows. Section II gives the
background. Section III presents the details of the proposed
method. Experiments are performed in Section IV and sum-
maries are concluded in Section V.
II. B ACKGROUND
A. Problem Formulation
Given multiple target traveling missions M =
{m1 , ..., mN } distributed in a large and complex underwater
environment, a multi-AUV system determines the location of
movable surface vehicles and launches AUVs from them to
complete missions in parallel. Every AUV can be released
into water multiple times and recovered before the energy is
exhausted, resulting in multiple voyages. Among all voyages,
each target mission is required to be visited by one voyage
exactly once. The total payload of a voyage is the sum of
the payload requirement li of each visiting target mission mi
 This article has been accepted for publication in IEEE Transactions on Industrial Informatics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TII.2023.3268760
AUTHOR et al.: TITLE 3

and should not be higher than the payload capacity Lmax of successive waypoints wji,k and wji,k+1 as
an AUV. To obtain the optimal voyages in terms of resource n−1
X m−1
utilization, this paper considers two common objectives, i.e.,
X
ht(pj ) = height(wji,k , wji,k+1 ) (4)
minimize the total cost f1 of all voyages and minimize the i=0 k=0
maximum ending time f2 of all voyages. Particularly, f1 is
The turning cost of pj is the sum of the turning cost
related to the energy consumption and fixed costs of AUVs
and f2 is about the load balance among voyages. Since turn(wji,k ) of all waypoints wji,k
the target missions may be changed due to ocean flows or si,k−1 si,k
j j
other environmental factors, the multi-AUV path planning is turn(wji,k ) = 1 − cos(θji,k ) = 1 − (5)
formulated as a dynamic multiobjective problem |si,k−1
j ||si,k
j |

minimize F (X, t) = (f1 (X, t), f2 (X, t)), where si,k−1

j is the line segment connecting wji,k−1 and wji,k ,
X and sj is the one connecting wji,k and wji,k+1 . Similarly, the
i,k
f1 (X, t) = gj + wc · nv ,
risk cost of a voyage is the sum of the risk intensity of each
1≤j≤nv
line segment between waypoints in the voyage as
f2 (X, t) = max hj ,
1≤j≤nv n−1
XX m
(1) risk(pj ) = rt(wj0,0 ) + rt(wji,k ),
X
s.t. xij = 1, ∀1 ≤ i ≤ N,
i=0 k=1
1≤j≤nv
nr
(6)
X X
li · xij ≤ Lmax , ∀1 ≤ j ≤ nv , rt(wji,k ) = RI × max{0, Rr − ||wji,k − rcr ||)}/Rr .
1≤i≤N r=1
gj ≤ Emax , ∀1 ≤ j ≤ nv . where r is the index of a risk area, nr is the number of risk
areas, RI is the maximal risk intensity, Rr is the maximal risk
where F (X, t) is the fitness function vector, X is a solution
radius, and rcr is the center of r-th risk area.
including multiple voyages to travel all target missions, t is
the time, f1 (X, t) is the traveling cost of all voyages, gj is
the energy consumption of j-th voyage, wc is the fixed vehicle B. Dynamic Multiobjective Optimization
cost for the launch and recovery of an AUV, nv is the number To solve dynamic multiobjective optimization problems, an
of voyages, f2 (X, t) is the maximum completion time among algorithm is expected to track the changing Pareto optimal
all voyages, hj is the completion time of j-th voyage, xij solutions. To deal with the multiobjective issue, existing
represents whether a target mi is involved by voyage pj , li methods can be roughly classified into three categories, i.e.,
is the payload requirement of the target mi , Lmax and Emax Pareto local search, metaheuristics, and learning-based meth-
are the maximum payload capacity and the maximum available ods. First, Pareto local search locates a large number of initial
energy of an AUV, respectively. solutions and searches the neighboring areas [24]. Exploration
Particularly, the energy cost gj of a voyage pj is calculated and exploitation are usually balanced to improve algorithm
following [19] as performance [25]. To reduce the computational cost, ND-tree
is developed for archive update [26]. Collaborative mecha-
gj = w1 · len(pj ) + w2 · ht(pj ) nisms are proposed to facilitate the search of neighboring
(2)
+ w3 · turn(pj ) + w4 · risk(pj ) areas [27]. Second, metaheuristics use multiple strategies to
enhance solution diversity and improve algorithm convergence
where w1 , w2 , w3 , and w4 are the weights of different energy
[28], such as nature-inspired genetic algorithm [29], particle
costs, i.e., traveling length cost len(pj ), height cost ht(pj ),
swarm optimization [30], [31], and ant colony system [32].
turning cost turn(pj ), and risk cost risk(pj ). To calculate
Metaheuristics have successfully solved many multiobjective
the energy costs of a voyage, the trajectory between each pair
combinatorial optimization problems. Third, learning-based
of successive points in a voyage is defined first. Denote a
methods use machine learning techniques to extract the pattern
voyage as pj = (vj0 , ..., vjn = vj0 ), where vj0 is the surface
for solution construction or evaluation [33]. For example,
location point to launch the AUV, vji (1 ≤ i ≤ n − 1) is the
Gaussian process models are adopted to approximate the
i-th target mission visited, and the trajectory tij between vji
objective functions and search for Pareto optimal solutions
and vji+1 is a sequence tij = (wji,0 = vji , ...., wji,m = vji+1 ) within trust regions [34]. Features are extracted to characterize
of waypoints (turning points to avoid obstacles). The length the landscape for performance prediction [35]. Graph neural
cost of pj is the sum of Euclidean distances between any two networks are trained to generate solutions for large-scale com-
successive waypoints as binatorial optimization problems [36]. Attention-based models
n−1
X n−1
X m−1
X are trained to learn the heuristic for routing problems [37].
len(pj ) = dist(vji , vji+1 ) = ||wji,k − wji,k+1 || In response to changes, multiple methods are developed
i=0 i=0 k=0 to generate new high-quality solutions, i.e., solution reuse,
(3) prediction, transfer learning, and reinforcement learning. Par-
where ||wji,k − wji,k+1 || is the Euclidean distance between the ticularly, solution reuse methods generate new solutions based
waypoints wji,k and wji,k+1 . The height cost of pj is the sum of on historical ones [38], [39]. Prediction methods learn the
the height differences height(wji,k , wji,k+1 ) between any two changing pattern from historical information for prediction.

© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: SSEN. Downloaded on May 06,2023 at 09:13:06 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for publication in IEEE Transactions on Industrial Informatics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TII.2023.3268760
4 IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. XX, NO. XX, XXXX
For example, an interaction-based prediction method is pro-
posed to predict task scheduling for multiple robots in a coop-
erative multirobot system [40]. Prediction methods may fail if
the changes do not follow a stable pattern. In contrast, transfer
learning shifts historical data to adapt to new environments.
For example, historical data is shifted to new environments
for training a surrogate using transfer learning techniques [41].
Reinforcement learning trains a policy network to give actions
when interacting with environments [42]. It requires a large
training set to get good performance.
III. T HE P ROPOSED C OOPERATIVE E VOLUTIONARY
C OMPUTATION A LGORITHM
A. Underwater Environment Modeling
The 3D underwater environment can be represented by a
graph. To set up this correspondence, the underwater envi-
ronment is evenly divided into multiple cubes, which are
represented by vertices. Edges connect two vertices if the
cubes represented by these vertices are adjacent. The resultant
graph G = (V, E) is called a search graph, where V is the set
of cubes and E is the edge set.
Denote the length, width, and height of the underwater
environment as L, W , and H, and those of a cube as LC ,
WC , and HC . Each cube is indexed by a triple-tuple (i, j, k)
and its center point is denoted as cijk = (x, y, z)
x = (i + 0.5)LC , y = (j + 0.5)WC , z = (k + 0.5)HC .
(7)
where 1 ≤ i ≤ L/LC , 1 ≤ j ≤ W/WC , and 1 ≤ k ≤
H/HC . Every line segment connecting the center points of two
adjacent cubes represents an edge. Since each cube is adjacent
to 26 cubes surrounding it, |E| ≤ 13|V |. Specifically, a cube
is considered as infeasible if it intersects with an obstacle
or the bathymetry and is then removed from G to ensure
the connectivity of vertices. A cube is risky if it intersects
with a risky area, and the risky value is defined according
to its distance to the risky area. The cubes interacting with
water surface make up a subset VS of V , which contains all
candidate points for AUV launch and recovery. The cubes in
which targets locate are called target vertices and make up a
subset VT of V . An example of G is illustrated in Fig. 1(a).
The problem of multi-AUV path planning is equivalent to
the problem of selecting vertices from VS and finding the
“shortest” paths that visit every target vertex in VT exactly
once. A problem involving energy cost and traveling time
can be modeled by assigning energy costs and traveling time
between vertices to the edges. The resultant graph that has a
number assigned to each edge is called a weighted graph G.
In G, two vertices u and v are connected by a path if there is
a sequence of vertices starting with u and ending with v such
that the endpoints of each edge in the path are vertices which
are adjacent. Let the energy or time cost of a path between
vertices be the sum of the weights of the edges of this path. A
path is shortest if it takes the lowest energy consumption and
the shortest time. Specifically, if two paths are nondominated
on energy consumption and time cost, then the one with a
lower energy cost is considered shorter since energy cost is a
critical constraint. The shortest path between two vertices can
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: SSEN. Downloaded on May 06,2023 at 09:13:06 UTC from IEEE Xplore. Restrictions apply.
be calculated using the shortest path faster algorithm [19],
which is adopted due to its popularity and low computational
complexity on spare graphs.
Since the voyage planning involves the surface location
points and target missions only, we induce a subgraph G0 =
(VS ∪ VT , E 0 ) of G by a subset VS ∪ VT of the vertex set V ,
where E 0 contains an edge in E if and only if both endpoints
of this edge are in VT or one endpoint in VS and one endpoint
in VT . We also add an edge e between vertices from VS ∪ VT
to G0 if there is a shortest path between them in G. Note that
edges between vertices from VS are not added to G0 since they
are not used for voyage planning. The energy and time costs
of the shortest path are assigned to the edges. The resulting
G0 provides a cost map for voyage planning as illustrated in
Fig. 1(b).
B. Cooperative Evolutionary Computation Algorithm
Based on the cost map G0 , CEA searches for the surface
location points and optimal voyages to travel all targets first.
Then, between any two successive vertices in voyages, a
trajectory is planned based on G to form complete paths of
AUVs.
1) Solution Representation: Denote the number of surface
points to locate as K. A solution X is represented by a bi-level
encode scheme
X = (S, P ),
S = (s1 , ..., si , ..., sK ) = (a1 , ..., a2K ), (8)
P = (p1 , ..., pi , ..., pK ).
where S is the surface location points of movable surface
vehicles, si is i-th surface point selected from VS . A surface
location point can be represented by a 2-tuple cube index,
i.e., (m, n), since the height is definite. Thus, S is a 2K-
dimensional vector and ai is a number related to a special
surface point. In addition, P is the optimal voyages to travel
all targets and includes K vertex sequences. Particularly, pi is
a sequence, e.g., x0 = si , ..., xj , ..., xn−1 , xn = si (xj ∈ VT ,
∀1 ≤ j ≤ n−1), of vertices for a voyage or the combination of
multiple voyages starting from si . Two voyages are separated
by the surface point si . For example, if two voyages start
from si , pi is a sequence, e.g., x0 = si , ..., xj , ..., xn =
si , xn+1 , ..., xm+n = si (xj ∈ VT , ∀1 ≤ j ≤ n − 1, n + 1 ≤
j ≤ m + n − 1), of vertices.
2) Evolution of Multiple Populations: Two populations are
adopted to optimize different objective functions. One popu-
lation finds the optimal solution of f1 and another one for f2 .
In both populations, the surface location points of individuals
are updated using differential evolution [43] and the voyages
are constructed using the ant colony system [44].
In i-th (1 ≤ i ≤ 2) population, NP individuals are initialized
first. Since Sij of an individual Xij = (Sij , Pij ) is real
vectors, it is randomly generated in the search space or set
as the surface points closest to the cluster centers of VT by
performing K-means [45]. For the voyage planning of Pij ,
a N × N pheromone matrix τi is used to record the search
experience. Each τi (u, v) represents the preference of selecting
v after u and is initialized as τi0 = 1/(N fi (X 0 )), where N
 This article has been accepted for publication in IEEE Transactions on Industrial Informatics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TII.2023.3268760
AUTHOR et al.: TITLE 5

is the number of target missions, X 0 is a solution constructed C1:{1,3,5,6}

using a greedy mechanism, in which the target missions are s1 s2 a

divided into multiple groups using K-means clustering and the P1
3 2 6 5 4 7 1 8 3 6 5 1
voyages of each group of target missions are constructed using
the record-to-record travel algorithm [46]. In X 0 , the surface
location points are the cubes closest to the cluster centers of s1 s2 b
P2
VT . The best solution in the population is denoted as Xi,best = 5 4 8 2 3 1 6 7 5 3 1 6
(Si,best , Pi,best ). In every generation, every individual Xij
0 0
is updated using a new solution Xij = (Sij , Pij0 ), where
0 0 0 0 0 0
Sij = (sij1 , ..., sij2K ) and Pij = (pij1 , ..., pijK ).
0 0 1 3 5 6 1 3 5 6
Surface Point Location: The Sij of Xij is generated using
the “DE/best/1” mutation and crossover operators. A mutant voyages Splitting
0
vector Sij is first generated by
Fig. 2. An example of the voyage planning for a set C1 = {1, 3, 5, 6}
0 of target missions in the recombination-based sampling strategy.
Sij = Si,best + F (Sir1 − Sir2 ) (9)

where F is the scale factor, r1 and r2 are two distant random where pro(v) is the selection probability of v as
integers in the range of [1, NP]. Then the crossover operator
is performed to generate a trial vector τi (xt−1 , v)ηi (xt−1 , v)β
pro(v) = P β
,
( u∈T τi (xt−1 , u)ηi (xt−1 , u) (13)
0
0
Sijd , if r < CR or d = drand , 1
Sijd = (10) ηi (xt−1 , v) = .
Sijd , otherwise. ci (xt−1 , v)
r is a random number in the range of [0,1], and q0 is a
where 1 ≤ d ≤ 2K is the dimension index, r is a random
predefined parameter. If r ≤ q0 , the target vertex with the
number in the range of [0,1], CR is the crossover rate, and
highest probability is selected; otherwise, roulette selection is
drand is a random integer in the range of [1, 2K]. Specifically,
performed to choose v 0 based on the probability distribution. In
if some target missions are unreachable by all surface points in
0 0 addition, τi (xt−1 , v) is the pheromone value of v, ηi (xt−1 , v)
Sij , then Sij is readjusted as follows. The closest reachable
0 is the heuristic value of v, and ci (xt−1 , v) is the traveling
surface points to these targets and the surface points in Sij
cost between xt−1 and v according to fi . Specifically, if T is
are collected to perform K-means clustering. The surface
0 empty, then the voyage p is added to p0ijk and a new one is
points closest to the cluster centers are used to update Sij .
constructed until Ck is empty. After finishing the construction
Specifically, if some target missions are still unreachable after
of voyages of all Ck , the local update of the pheromone is
five tries, the fitness value of the solution is set as a very large
performed
value. (
Multi-AUV Voyage Planning: Based on Sij 0
, the targets are τi0 , if (u, v) ∈ Pij0 ,
assigned to the nearest surface location point according to the τi (u, v) = (1−ρ)τi (u, v)+ρ∆τ, ∆τ =
0, otherwise.
optimization objective, resulting in multiple subsets. Denote (14)
the set of the targets assigned to k-th surface point s0k as Ck = where ρ is the evaporation parameter.
{v1 , ..., v|Ck | }. Every Ck is used to construct voyages starting Then Xij0
is evaluated and used to update Xij and Xi,best
from s0k . To achieve this, targets are selected from Ck step by if it is better on fi . After the update of all individuals, the
step. In the first step, one target x1 is randomly selected from global update of the pheromone is performed
Ck and a voyage is initialized as p = (s0k , x1 ). The selected
target x1 is then removed from Ck . At t-th step, the set of τi (u, v) = (1 − α)τi (u, v) + α∆τ, (15)
available target vertices to select is where
(
T = {v | c0 (p) + c0 (xt−1 , v) + c0 (v, s0k ) ≤ Emax & 1/fi (Xi,best ), if (u, v) ∈ Pi,best ,
(11) ∆τ = (16)
l(p) + l(v) ≤ Lmax , ∀v ∈ Ck }. 0, otherwise.

where c0 (p) is the energy cost of the voyage to the last target α is the pheromone decay parameter, and fi (Xi,best ) is the
xt−1 in the voyage p, c0 (xt−1 , v) is the energy cost between fitness value of the best solution. After both two populations
xt−1 and v, c0 (v, s0k ) is the energy cost between v and s0k , l(p) are updated, their individuals are collected together with the
is the total payload of the current voyage p before visiting v, archived solutions and the nondominated ones are stored in
and l(v) is the payload requirement of v. Then one target xt the archive.
is selected from T according to the state transfer rule 3) Recombination-based Sampling Strategy: Since each
 population optimizes one objective only, they mainly gather
 arg max pro(v), if r ≤ q0 , around the margins of the Pareto front. To reach the areas
xt = v∈T (12) missed by the populations, a recombination-based sampling
v 0 , otherwise. strategy is developed to generate new solutions by fusing

© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: SSEN. Downloaded on May 06,2023 at 09:13:06 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for publication in IEEE Transactions on Industrial Informatics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TII.2023.3268760
6 IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. XX, NO. XX, XXXX
information of different populations. First, two solutions X1 =
(S1 , P1 ) and X2 = (S2 , P2 ) are randomly selected from the
archive as parents. Then the first part S 0 of surface location
points of a new solution X 0 = (S 0 , P 0 ) is generated using
simulated binary crossover and polynomial mutation [47],
and P 0 is constructed by recombining the vertices in parents
P1 = (p11 , ..., pK 1 K
1 ) and P2 = (p2 , ..., p2 ). The construction
0
procedure of P is described as follows. To reduce the traveling
cost, each target vertex is assigned to the nearest surface
location point in S 0 , resulting in one cluster Ck for every
s0k (1 ≤ k ≤ K). For the vertices in Ck , two sequences a
and b are constructed according to their orders in P1 and P2 ,
respectively. The two sequences represent the traveling pref-
erence of parents and can be used to generate new sequences
using crossover. Two crossover points are randomly selected
to split a into three segments, i.e., a1 , a2 , and a3 . Similarly, b
is split into b1 , b2 , and b3 . Then a new sequence is obtained by
combining a3 , a1 , and b2 in order. Notably, repeat targets are
removed. Specially, the unvisited targets in Ck are appended
into the new sequence according to their orders in b. The new
sequence is then used to construct voyages using the splitting
method [48], which splits the sequence into (near) optimal
voyages.
An example of the recombination-based sampling is illus-
trated in Fig. 2. There are two surface points s1 and s2 .
The parent P1 includes two voyages p1 = {s1 , 3, 2, 6, 5, s1 }
and p2 = {s2 , 4, 7, 1, 8, s2 }. The voyages in P2 are p1 =
{s1 , 5, 4, 8, 2, s1 } and p2 = {s2 , 3, 1, 6, 7, s2 }. Given a set
of target missions C1 = {1, 3, 5, 6}, we rearrange the tar-
get missions in C1 based on their order in p1 and p2 of
P1 , resulting in a sequence a = {3, 6, 5, 1}. Similarly, a
sequence b is obtained from P2 , i.e., b = {5, 3, 1, 6}. Then
two random points are adopted to cut a and b into three
subsequences, i.e., a1 = {3}, a2 = {6, 5}, a3 = {1}, and
b1 = {5}, b2 = {3, 1}, b3 = {6}. A new sequence is generated
by concatenating a3 , a1 , and b2 , i.e., c = {1, 3}. The remaining
target missions, i.e., 5 and 6, are appended to c in their order
in b, resulting in c = {1, 3, 5, 6}. Finally, the vertex sequence
c is split to generate multiple new voyages.
In each generation, |A|/2 solutions are generated using the
recombination-based sampling, where |A| is the size of the
archive. The newly generated solutions are used to update the
archive and only nondominated ones are preserved. Through
the evolution of multiple generations, the algorithm converge
towards the Pareto front. The archive provides multiple high-
quality solutions to decision makers.
4) Incremental Response Strategy to Changes: Due to the
ocean flows or other factors, the target missions may change
with time. For example, some target missions are canceled
and some are changed to other locations due to the risk; new
target missions arrive in search and rescue. In response to
such changes, the paths of AUVs should be replanned based
on a newly constructed cost map G0 . Since targets are still
distributed in a large underwater environment, the surface
location points and voyages found before may be useful for the
planning of new target missions. Thus, to utilize the historical
search experience, the surface location points of all individuals
in the populations remain the same and their voyages are
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: SSEN. Downloaded on May 06,2023 at 09:13:06 UTC from IEEE Xplore. Restrictions apply.
reconstructed using the ant colony system. Similarly, for the
archived solutions, the surface location points keep the same
and the voyages are reconstructed using the ant colony system
based on the pheromone and heuristic information of a random
population. The populations and the archived solutions are
evaluated and the nondominated ones are stored in the archive.
The populations then continue to evolve until meeting new
changes or achieving termination conditions.
5) Trajectory Planning: The trajectory planning is performed
at the end of each time step. After finishing the voyage
planning, the trajectory of AUV between tow adjacent vertices
(u, v) in the voyages is planned using A* search [49]. Indeed,
the trajectory between u and v can be recovered from the
shortest path searching in G during the environment modeling
procedure, but it may require a large memory. Thus, if the
shortest path is not stored, A* search is used for finding the
optimal path between u and v in G. To ensure the optimality of
A* search, the heuristic value (the estimated cost) of a vertex
x to the endpoint v should be not higher than the optimal
traveling cost between them [19]. We assume that there is a
direct line segment between x and v in G and set its traveling
cost as the heuristic value of x.
C. Worst Computational Complexity of CEA in a Time
Step
The computational complexity of CEA includes those of
four parts, i.e., cost map building, initialization, evolution of
populations, and incremental response strategy. Denote the size
of a population as NP, the number of target missions as N , the
number of surface points as K, the number of optimization
objectives as M , the number of iterations as I, the number
of vertices in the cost map G0 as |V 0 |, the average times of a
cube entering the queue in the shortest path faster algorithm
as k, and the number of vertices in the graph G as |V 0 |.
First, the computational complexity of the cost map building
is O(k × N × |V 0 |).
Second, in the initialization, the setting of initial surface
points requires O(N Knc ) computations for an individual,
where nc is the number of iterations for k-means clustering
and is usually a small value. The initialization of voyages
requires O((nd nR nU nL N + N + N 2 )nP ) computations using
the record-to-record travel algorithm, where nd = 30 is the
number of loops in the diversification phase, nR = 30 is the
size of the neighbor list, nU = 3 is the number of heuristic
used, nL = 5 is the number of local minima to reach, nP is
the number of perturbations. Thus, the initialization of M ×NP
individuals requires O(M × N × K × NP + M × N 2 × NP)
computations.
Third, in every iteration, the update of surface points in
an individual requires O(K) computations. The clustering
of target missions based on surface points requires O(N K)
computations. Voyage planning requires O(N 2 ) computations.
The local update of a pheromone matrix requires O(N 2 )
computations. The global pheromone update of the pheromone
matrix requires O(N 2 ) computations. Thus, the update of M
populations requires O(((K + N K + N 2 + N 2 ) × NP +
N 2 ) × M ) = O(N × M × K × NP + N 2 × M × NP)
 This article has been accepted for publication in IEEE Transactions on Industrial Informatics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TII.2023.3268760
AUTHOR et al.: TITLE
computations, which can be reduced to O(M × N 2 × NP)
since we have K  N . The recombination-based sampling
strategy requires O(N K +N 2 ) = O(N 2 ) computations. Then
generating solutions using the recombination-based sampling
strategy requires O(N 2 nA ) computations, where nA is the
number of solutions in the archive. The archive update re-
quires O((M × NP + nA )2 × M ) computations. Thus, the
computational complexity of I iterations is O((M × N 2 ×
NP + N 2 nA + (M × NP + nA )2 × M ) × I), which can be
reduced to O(M 3 × NP2 × I + M × N 2 × NP × I) since we
have nA ≈ M × NP in the experiments. At the end of the
algorithm, trajectory planning is performed for the archived
solutions. The worst complexity of the trajectory planning is
O((|V | + |E|)N nA ) = O(M × N × |V | × NP).
Fourth, in the incremental response strategy, the update of
populations and the archive requires O(M × N 2 × NP + M 3 ×
NP2 ) computations.
Therefore, the worst complexity of CEA in a time step is
O(k × N × |V 0 | + M 3 × NP2 × I + M × N 2 × NP × I + M ×
N × |V | × NP).
IV. E XPERIMENTS
A. Experimental Settings
1) Underwater Environment Scenarios: To simulate the un-
derwater environments, the dataset of 250m resolution grid-
ded New Zealand bathymetry1 [19] is used to construct six
complex underwater scenarios, named I1 − I6 , with complex
bathymetry, irregular and branched obstacles, and global risks,
as illustrated in Fig. 3, where the 3D surface is the bathymetry,
red blocks are obstacles, blue balls are risky areas, and black
points are target missions. The length, width, and depth of the
underwater environment is 50km, 50km, and 10km, respec-
tively. The underwater environment is modeled as a search
graph based on cubes with a size of 250m×250m×50m.
Each scenario instance includes 400 target traveling missions
randomly distributed underwater. Each target mission requires
1 unit payload. Each instance experiences nine changes, in
which the positions of 10% target missions are randomly
changed. The change frequency is set as 20,000 function
evaluations. In the following, we call the environment between
two changes as a time step. The available energy of an AUV
is 1200, the payload capacity is 15, and the speed is 2m/s. The
weights in cost functions w1 , w2 , w3 , w4 , and wc are mission-
related and set according to the AUV operators provided by
decision makers. In the experiments, w1 = 1, w2 = 10,
w3 = 50, w4 = 50, and wc = 200 following [19]. The
maximum number of surface points is set as 5.
2) Competing Algorithms: Since there are few works dealing
with the dynamic multiobjective path planning of multiple
AUVs, three typical algorithms are adopted for comparisons,
i.e., the vehicle routing heuristics with K-means (K-VRPH)
[46], non-dominated sorting-based genetic mission planner
(NSGMP) [50], and multiobjective evolutionary algorithm
based on decomposition with DE-C-ACO (MOEA/D-DE-C-
ACO) [19]. Particularly, K-VRPH uses K-means to cluster
1 https://niwa.co.nz/oceans/resources/bathymetry/download-the-data
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: SSEN. Downloaded on May 06,2023 at 09:13:06 UTC from IEEE Xplore. Restrictions apply.
7
(a) I1 (b) I2 (c) I3
(d) I4 (e) I5 (f) I6
Fig. 3. The six underwater environment scenarios (a)-(f) I1 − I6 .
the target missions and adopts local search heuristics (i.e.,
record-to-record travel algorithm) to generate voyages for each
cluster [46]. NSGMP uses edge recombination crossover and
swap mutation to generate new solutions, which are further
refined by a local greedy search [50]. Nondominated solutions
are selected using the crowding comparison procedure [51].
MOEA/D-DE-C-ACO decomposes the problem into multiple
single-objective subproblems using weight vectors, and every
subproblem is optimized using DE-C-ACO for multi-AUV
path planning [19]. Once a change occurs, half of the pop-
ulations are reinitialized.
The parameters of the competing algorithms are set the same
as in the references. In K-VRPH, the size of the main loop
is nL = 20, δ = 0.01 to control the deterioration degree of
the objective function, the size of the neighbor list is nN =
30, and the number of times that the solution is perturbed
once the search is stuck in a local optimum nP = 2. In both
NSGMP and MOEA/D-DE-C-ACO, the population size is set
as NP=100. In NSGMP, the crossover rate is cr = 1.0, and
the mutation rate is mr = 1/D, where D is the problem
dimension. In MOEA/D-DE-C-ACO, F = 0.75, CR = 0.95,
β = 2, q0 = 0.9, α = ρ = 0.1. In CEA, NP = 10, F = 0.75
and CR = 0.95 following [19], and the parameters in the ant
colony system is set following [44], i.e., β = 2, q0 = 0.9, α =
ρ = 0.1. The distribution indexes for crossover and mutation
in the sampling strategy are set as 20.
3) Performance Metrics: Two widely used performance met-
rics, mean inverted generational distance (MIGD) and mean
hypervolume (MHV), are adopted to evaluate the algorithm
performance. The MIGD is calculated as
P ∗
1≤t≤Nchange +1 IGD(P Ft , Pt )
M IGD = ,
Nchange + 1
P ∗ (17)
p∗ ∈P Ft∗ minp∈Pt d(p , p)
∗
IGD(P Ft , Pt ) = .
|P Ft∗ |
where t is the index of the time step, Nchange is the number
of changes, P Ft∗ is the Pareto front of the instance at t
and approximated using the nondominated solutions of all
algorithms, Pt is the set of nondominated solutions found by
an algorithm at time t, and |P Ft∗ | is the number of points in
 This article has been accepted for publication in IEEE Transactions on Industrial Informatics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TII.2023.3268760
8 IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. XX, NO. XX, XXXX

TABLE I  &($

&($
PARAMETER SETTINGS OF ALL ALGORITHMS QR56
QR56
 QR,6
Algorithm Parameter values QR,6


f2
f2
K-VRPH nL = 20, δ = 0.01, nN = 30, nP = 2
NSGMP NP=100, cr = 1, mr = 1/D 
MOEA/D-DE-C-ACO NP=100, F = 0.75, CR = 0.95, β = 2, q0 = 0.9, α = ρ = 0.1
CEA NP=10, F = 0.75, CR = 0.95, β = 2, q0 = 0.9, α = ρ = 0.1

    
TABLE II f1 f1
MHV RESULTS OF CEA AND ITS TWO VARIANTS NO RS AND NO IS (a) 2nd (b) 3rd
Ins noRS noIS CEA
I1 3.98e-02±4.46e-03 + 5.40e-02±2.36e-03 + 5.87e-02±5.88e-03 &($ &($
I2 1.02e-01±1.41e-02 + 1.18e-01±7.36e-03 + 1.36e-01±1.17e-02 QR56
 QR56
I3 7.65e-02±3.86e-03 + 9.43e-02±3.58e-03 = 9.57e-02±4.74e-03 QR,6 QR,6


f2
I4 4.79e-02±3.99e-03 + 6.41e-02±4.22e-03 + 7.28e-02±1.08e-02

f2
I5 6.09e-02±3.85e-03 + 7.45e-02±3.82e-03 = 7.60e-02±5.82e-03 
I6 5.15e-02±5.36e-03 + 6.38e-02±4.36e-03 = 6.72e-02±6.25e-03
Best 0 0 6
+/=/- 6/0/0 3/3/0 
    
f1 f1
(c) 4th (d) 5th
P Ft∗ . Since the PF is unknown, the nondominated solutions
obtained by all algorithms in all runs are used as reference &($
 QR56
&($
points to calculate IGD. The mean hypervolume is calculated QR56
QR,6  QR,6
as

f2

f2
P
1≤t≤Nchange +1 HVt 
M HV = (18) 
Nchange + 1
where HVt is the hypervolume of the nondominated solutions 
    
obtained by an algorithm based on a reference point (1, 1). The f1 f1

solutions are normalized using (1.1f1,max , 1.1f2,max ), where (e) 6th (f) 7th
f1,max and f2,max are the maximum fitness values in P Ft∗ . Fig. 4. Nondominated solutions obtained by noRS, noIS, and CEA in
Each algorithm performs 20 runs, and the median values 2nd -7th time steps of I2 .
and the standard deviations of the performance metrics are
reported. Wilcoxon rank-sum test is performed between the TABLE III
proposed method and every competing algorithm at a 5% MIGD RESULTS OF ALL ALGORITHMS
significance level. The results are signed as “+”, “=”, or “- Ins K-VRPH NSGMP MOEA/D-DE-C-ACO CEA
I1 1.22e+04±3.13e+02 + 3.86e+04±6.31e+02 + 8.57e+03±1.28e+02 + 8.17e+03±2.57e+02
” to represent that the proposed method is significantly better, I2 2.52e+04±1.28e+02 + 3.64e+04±1.32e+03 + 2.36e+04±6.39e+01 + 2.33e+04±1.27e+02
I3 9.90e+03±2.41e+02 + 4.08e+04±5.06e+02 + 7.12e+03±1.95e+02 + 6.96e+03±1.63e+02
equal to, or worse than the competing algorithm, respectively. I4 1.42e+04±2.25e+02 + 4.89e+04±4.46e+02 + 1.13e+04±9.77e+01 + 1.09e+04±2.42e+02
I5 1.76e+04±3.32e+02 + 4.43e+04±3.42e+02 + 1.53e+04±7.99e+01 + 1.47e+04±2.50e+02
I6 1.93e+04±2.17e+02 + 3.96e+04±3.98e+02 + 1.70e+04±1.37e+02 + 1.60e+04±3.21e+02
Best 0 0 0 6
B. Effects of Components in CEA +/=/- 6/0/0 6/0/0 6/0/0

This subsection tests the effects of the recombination-based

sampling strategy and the incremental response strategy. Two
To further observe the solution quality, taking I2 as an
CEA variants are constructed for test, i.e., noRS that does
example, Fig. 4 illustrates the nondominated solutions ob-
not use the recombination-based sampling strategy and noIS
tained by noRS, noIS, and CEA in 2nd to 7th time steps. One
that does not adopt the incremental response strategy. The
unit of f2 represents one minute. Note that the incremental
MHV results of noRS, noIS, and CEA are reported in Table II.
response strategy works after 2nd time step. From Fig. 4, the
From Table II, CEA performs significantly better than noRS on
solutions of CEA are better convergent and distributed than
all instances, showing that the recombination-based sampling
that of two variants in each time step. In contrast, the solutions
strategy can improve algorithm performance. On the one hand,
of noRS are far away from the Pareto front, showing poor
the recombination-based sampling strategy can generate high-
convergence. Similarly, though a few solutions of noIS are
quality solutions for enhancing solution diversity. On the
nondominated with that of CEA in 2nd time step, the solutions
other hand, the sampling strategy takes the advantages of
lack enough diversity and are less convergent in other time
multiple archived solutions to improve solution convergence
steps. This shows that the incremental response strategy can
towards the Pareto front. Compared with noIS, CEA performs
well utilize the historical solutions to improve solution quality
significantly better on 3 out of 6 instances, whereas there is no
and enhance diversity. In addition, the effect of the response
significant difference between them on other instances. This
strategy becomes more significant as time progresses due to
shows that the historical information can assist the algorithm
the accumulation of experience.
to quickly find new high-quality solutions after changes. In
addition, CEA obtains the best results among three algorithms
on all instances. Thus, both the recombination-based sampling C. Compared with Other Algorithms
strategy and the incremental response strategy play important The MIGD and MHV results of all algorithms on the
roles in the proposed method. six instances are reported in Tables III and IV. From Table

© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: SSEN. Downloaded on May 06,2023 at 09:13:06 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for publication in IEEE Transactions on Industrial Informatics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TII.2023.3268760
AUTHOR et al.: TITLE 9

TABLE IV
&($  &($
MHV RESULTS OF ALL ALGORITHMS
.953+ .953+

Ins K-VRPH NSGMP MOEA/D-DE-C-ACO CEA 16*03 16*03
I1 7.34e-03±3.59e-03 + 2.41e-02±3.06e-03 + 5.22e-02±1.71e-03 + 5.87e-02±5.88e-03 

f2

f2
02($''(&$&2 02($''(&$&2
I2 1.44e-03±2.61e-03 + 8.56e-02±4.63e-03 + 1.12e-01±4.28e-03 + 1.36e-01±1.17e-02
I3 1.14e-02±5.17e-03 + 3.74e-02±2.18e-03 + 8.81e-02±6.97e-03 + 9.57e-02±4.74e-03
I4 0.00e+00±2.60e-03 + 1.89e-02±8.16e-04 + 6.92e-02±4.82e-03 = 7.28e-02±1.08e-02
I5 2.90e-03±4.20e-03 + 2.55e-02±1.88e-03 + 6.69e-02±5.38e-03 + 7.60e-02±5.82e-03  
I6 7.72e-03±5.16e-03 + 2.10e-02±2.71e-03 + 5.86e-02±9.21e-03 + 6.72e-02±6.25e-03
Best 0 0 0 6
+/=/- 6/0/0 6/0/0 5/1/0    
f1 f1
(a) 2nd (b) 3rd
&($ &($

.953+ .953+
&($  &($

 16*03 16*03

.953+
 .953+
f2

f2
02($''(&$&2 02($''(&$&2
16*03 16*03



f2
f2
02($''(&$&2 02($''(&$&2


 
    

f1 f1    

(a) 2nd (b) 3rd f1 f1
(c) 4th (d) 5th
 &($ &($

.953+ .953+ 

&($ &($
 16*03  16*03
.953+ .953+

f2

f2

02($''(&$&2 02($''(&$&2
16*03 16*03


f2

f2
02($''(&$&2 02($''(&$&2
 

 
   
f1 f1     
(c) 4th (d) 5th f1 f1
(e) 6th (f) 7th
&($ &($
 .953+
 .953+ Fig. 6. Nondominated solutions obtained by K-VRPH, NSGMP,
16*03 16*03 MOEA/D-DE-C-ACO, and CEA in 2nd -7th time steps of I5 .

f2

f2

02($''(&$&2 02($''(&$&2



 
f1
(e) 6th
Fig. 5. Nondominated solutions obtained by K-VRPH, NSGMP,
MOEA/D-DE-C-ACO, and CEA in 2nd -7th time steps of I2 .
III, CEA performs significantly better than all competing
algorithms on all instances in terms of MIGD. From Table
IV, CEA works significantly better than K-VRPH, NSGMP,
and MOEA/D-DE-C-ACO on 6, 6, and 5 out of 6 instances.
This shows that CEA can find solution sets with higher
quality and stronger diversity. The good performance of CEA
benefits from the multipopulation mechanism, recombination-
based sampling strategy, and the incremental response strategy.
Particularly, the multiple populations help locate multiple
high-quality solutions in the search space, which are the data
source for the recombination-based sampling strategy to gen-
erate better solutions. The cooperation of multiple populations
and the sampling strategy enables the algorithm to search
well-distributed and high-quality solutions. In addition, the
incremental response strategy helps the algorithm quickly react
to changes by efficiently utilizing historical information.
Taking I2 and I5 as examples, the nondominated solutions
obtained by each algorithm in 2nd - 7th time steps are illustrated
in Figs. 5 and 6. The solutions of CEA dominate those of other
algorithms in almost all time steps. Particularly, the solutions
 
of NSGMP take a higher energy cost (f1 ) since NSGMP uses
f1 neighbor information only from past solutions and tends to
(f) 7th assign the target missions into more voyages. In contrast, the
solutions of K-VRPH require a longer traveling time since K-
VRPH constructs solutions using local heuristic information
and is easily trapped in local optima. Compared with NSGMP
and K-VRPH, both CEA and MOEA/D-DE-C-ACO obtain
better solutions. This shows that differential evolution can
locate good surface points and ant colony system can well
plan voyages using the historical experience and heuristic
information in a global scheme. Specifically, the number of
solutions of MOEA/D-DE-C-ACO is smaller than CEA and
the solutions are worse. This shows that the multipopulation
mechanism and the recombination-based sampling strategy can
help CEA better maintain population diversity and approxi-
mate the Pareto front.
Thus, CEA has a strong global search ability to find high-
quality and diverse solutions in complex underwater environ-
ments with dynamic target missions.
V. C ONCLUSION
This paper tracks the dynamic multiobjective multi-AUV
path planning problem in large underwater environments with
complex obstacles and risky areas. A cooperative evolutionary
computation algorithm is proposed to locate the surface points
for AUV launch and plan the paths of AUVs for traveling

© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: SSEN. Downloaded on May 06,2023 at 09:13:06 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for publication in IEEE Transactions on Industrial Informatics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TII.2023.3268760
10
target missions. Experimental results on multiple complex un-
derwater environments show that the proposed method can find
better solution sets than the state-of-the-art algorithms in dy-
namic scenarios. Particularly, the multipopulation mechanism
helps maintain population diversity and the recombination-
based sampling strategy benefits to improving convergence by
fusing information from multiple populations. In addition, the
incremental response strategy improves algorithm performance
by using historical information after changes. The proposed
method is able to provide diverse and high-quality solutions
for decision makers.
Several avenues are open for further investigations. First,
since the proposed method uses historical solutions to update
the populations after changes, it may fail if problems become
substantially different after changes. Transfer learning methods
could be developed to adapt to random changes. Second,
moving obstacles and changing ocean flows are not considered
in the problem model. In future works, new methods could
be developed for the multi-AUV path planning in complex
underwater environments with moving obstacles and changing
ocean flows.
R EFERENCES
[1] Y. Wang, X. Ma, J. Wang, S. Hou, J. Dai, D. Gu, and H. Wang, “Robust
auv visual loop-closure detection based on variational autoencoder
network,” IEEE Trans. Ind. Informat., vol. 18, no. 12, pp. 8829–8838,
2022.
[2] D. Roper, C. A. Harris, G. Salavasidis, M. Pebody, R. Templeton,
T. Prampart, M. Kingsland, R. Morrison, M. Furlong, A. B. Phillips, and
S. McPhail, “Autosub long range 6000: A multiple-month endurance auv
for deep-ocean monitoring and survey,” IEEE J. Ocean. Eng., vol. 46,
no. 4, pp. 1179–1191, 2021.
[3] I. Jawhar, N. Mohamed, J. Al-Jaroodi, and S. Zhang, “An architecture
for using autonomous underwater vehicles in wireless sensor networks
for underwater pipeline monitoring,” IEEE Trans. Ind. Informat., vol. 15,
no. 3, pp. 1329–1340, 2019.
[4] M. Katwe, K. Singh, P. K. Sharma, C.-P. Li, and Z. Ding, “Dynamic
user clustering and optimal power allocation in uav-assisted full-duplex
hybrid noma system,” IEEE Wireless Commun., vol. 21, no. 4, pp. 2573–
2590, 2022.
[5] M. Chen and D. Zhu, “A workload balanced algorithm for task as-
signment and path planning of inhomogeneous autonomous underwater
vehicle system,” IEEE Trans. Cogn. Develop. Syst., vol. 11, no. 4, pp.
483–493, 2019.
[6] G. Han, Z. Tang, Y. He, J. Jiang, and J. A. Ansere, “District partition-
based data collection algorithm with event dynamic competition in
underwater acoustic sensor networks,” IEEE Trans. Ind. Informat.,
vol. 15, no. 10, pp. 5755–5764, 2019.
[7] Y. Rizk, M. Awad, and E. W. Tunstel, “Cooperative heterogeneous multi-
robot systems: A survey,” ACM Comput. Surv., vol. 52, no. 2, apr 2019.
[8] J. Lee and V. Friderikos, “Interference-aware path planning optimization
for multiple uavs in beyond 5g networks,” J. Commun. Netw., vol. 24,
no. 2, pp. 125–138, 2022.
[9] Z. Zhang, J. Wang, D. Xu, and Y. Meng, “Task allocation of multi-
auvs based on innovative auction algorithm,” in ISCID, vol. 2, 2017,
pp. 83–88.
[10] S. Mahmoudzadeh, D. M. Powers, K. Sammut, A. M. Yazdani, and
A. Atyabi, “Hybrid motion planning task allocation model for auv’s
safe maneuvering in a realistic ocean environment,” J. Intell. Robotics
Syst., vol. 94, no. 1, p. 265–282, apr 2019.
[11] G. Han, A. Gong, H. Wang, M. Martı́nez-Garcı́a, and Y. Peng, “Multi-
auv collaborative data collection algorithm based on q-learning in un-
derwater acoustic sensor networks,” IEEE Trans. Veh. Technol., vol. 70,
no. 9, pp. 9294–9305, 2021.
[12] F. Zitouni and R. Maamri, “Cooperative learning-agents for task al-
location problem,” in Interactive Mobile Communication Technologies
and Learning, M. E. Auer and T. Tsiatsos, Eds. Cham: Springer
International Publishing, 2018, pp. 952–968.
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: SSEN. Downloaded on May 06,2023 at 09:13:06 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. XX, NO. XX, XXXX
[13] A. Atyabi, S. MahmoudZadeh, and S. Nefti-Meziani, “Current advance-
ments on autonomous mission planning and management systems: An
auv and uav perspective,” Annu. Rev. Control, vol. 46, pp. 196–215,
2018.
[14] R. Cui, Y. Li, and W. Yan, “Mutual information-based multi-auv path
planning for scalar field sampling using multidimensional rrt*,” IEEE
Trans. Syst., Man, Cybern., Syst., vol. 46, no. 7, pp. 993–1004, 2016.
[15] J.-V. Sørli, O. H. Graven, and J. D. Bjerknes, “Multi-uav cooperative
path planning for sensor placement using cooperative coevolving genetic
strategy,” in Advances in Swarm Intelligence, Y. Tan, H. Takagi, Y. Shi,
and B. Niu, Eds. Cham: Springer International Publishing, 2017, pp.
433–444.
[16] J. Zhang, M. Liu, S. Zhang, R. Zheng, and S. Dong, “Multi-auv
adaptive path planning and cooperative sampling for ocean scalar field
estimation,” IEEE Trans. Instrum. Meas., vol. 71, pp. 1–14, 2022.
[17] D. Wei, H. Ma, H. Yu, X. Dai, G. Wang, and B. Peng, “A hyperheuristic
algorithm based on evolutionary strategy for complex mission planning
of auvs in marine environment,” IEEE J. Ocean. Eng., vol. 47, no. 4,
pp. 936–949, 2022.
[18] W. Luo, Q. Tang, C. Fu, and P. Eberhard, “Deep-sarsa based multi-
uav path planning and obstacle avoidance in a dynamic environment,”
in Advances in Swarm Intelligence, Y. Tan, Y. Shi, and Q. Tang, Eds.
Cham: Springer International Publishing, 2018, pp. 102–111.
[19] X. Yu, W.-N. Chen, X.-M. Hu, T. Gu, H. Yuan, Y. Zhou, and J. Zhang,
“Path planning in multiple-auv systems for difficult target traveling mis-
sions: A hybrid metaheuristic approach,” IEEE Trans. Cogn. Develop.
Syst., vol. 12, no. 3, pp. 561–574, 2020.
[20] D. Zhu, B. Zhou, and S. X. Yang, “A novel algorithm of multi-auvs
task assignment and path planning based on biologically inspired neural
network map,” IEEE Trans. Intell. Veh., vol. 6, no. 2, pp. 333–342, 2021.
[21] J. Wu, C. Song, J. Ma, J. Wu, and G. Han, “Reinforcement learning and
particle swarm optimization supporting real-time rescue assignments for
multiple autonomous underwater vehicles,” IEEE Trans. Intell. Transp.
Syst., vol. 23, no. 7, pp. 6807–6820, 2022.
[22] Z.-H. Zhan, J. Li, J. Cao, J. Zhang, H. S.-H. Chung, and Y.-H. Shi,
“Multiple populations for multiple objectives: A coevolutionary tech-
nique for solving multiobjective optimization problems,” IEEE Trans.
Cybern., vol. 43, no. 2, pp. 445–463, 2013.
[23] X.-F. Liu, Z.-H. Zhan, Y. Gao, J. Zhang, S. Kwong, and J. Zhang,
“Coevolutionary particle swarm optimization with bottleneck objective
learning strategy for many-objective optimization,” IEEE Trans. Evol.
Comput., vol. 23, no. 4, pp. 587–602, 2019.
[24] J. Shi, J. Sun, Q. Zhang, H. Zhang, and Y. Fan, “Improving pareto
local search using cooperative parallelism strategies for multiobjective
combinatorial optimization,” IEEE Trans. Cybern., pp. 1–14, 2022.
[25] A. Jaszkiewicz and T. Lust, “Proper balance between search towards
and along Pareto front: biobjective TSP case study,” Ann. Oper. Res.,
vol. 254, no. 1, pp. 111–130, July 2017.
[26] A. Jaszkiewicz, “Many-objective pareto local search,” Eur. J. Oper. Res.,
vol. 271, no. 3, pp. 1001–1013, 2018.
[27] X. Cai, C. Xia, Q. Zhang, Z. Mei, H. Hu, L. Wang, and J. Hu, “The
collaborative local search based on dynamic-constrained decomposition
with grids for combinatorial multiobjective optimization,” IEEE Trans.
Evol. Comput., vol. 51, no. 5, pp. 2639–2650, 2021.
[28] F. Yang, X. Lei, J. Le, N. Mu, and X. Liao, “Minable data publication
based on sensitive association rule hiding,” IEEE Trans. Emerg. Topics
Comput., vol. 6, no. 5, pp. 1247–1257, 2022.
[29] J. M. Sanchez-Gomez, M. A. Vega-Rodrı́guez, and C. J. Pérez, “Auto-
matic update summarization by a multiobjective number-one-selection
genetic approach,” IEEE Trans. Cybern., pp. 1–0, 2022.
[30] X.-F. Liu, Y. Fang, Z.-H. Zhan, and J. Zhang, “Strength learning particle
swarm optimization for multiobjective multirobot task scheduling,” IEEE
Trans. Syst., Man, Cybern., Syst., pp. 1–12, 2023.
[31] R. Lan, Y. Zhu, H. Lu, Z. Liu, and X. Luo, “A two-phase learning-based
swarm optimizer for large-scale optimization,” IEEE Trans. Cybern.,
vol. 51, no. 12, pp. 6284–6293, 2021.
[32] M. Z. Kreishan and A. F. Zobaa, “Allocation of dump load in islanded
microgrid using the mixed-integer distributed ant colony optimization,”
IEEE Syst. J., vol. 16, no. 2, pp. 2568–2579, 2022.
[33] Y. Bengio, A. Lodi, and A. Prouvost, “Machine learning for combinato-
rial optimization: A methodological tour d’horizon,” Eur. J. Oper. Res.,
vol. 290, no. 2, pp. 405–421, 2021.
[34] K. Touloupas and P. P. Sotiriadis, “Locomobo: A local constrained
multiobjective bayesian optimization for analog circuit sizing,” IEEE
Trans. Comput.-Aided Design Integr. Circuits Syst., vol. 41, no. 9, pp.
2780–2793, 2022.
 This article has been accepted for publication in IEEE Transactions on Industrial Informatics. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/TII.2023.3268760
AUTHOR et al.: TITLE
[35] A. Liefooghe, F. Daolio, S. Verel, B. Derbel, H. Aguirre, and K. Tanaka,
“Landscape-aware performance prediction for evolutionary multiobjec-
tive optimization,” IEEE Trans. Evol. Comput., vol. 24, no. 6, pp. 1063–
1077, 2020.
[36] M. J. A. Schuetz, J. K. Brubaker, and H. G. Katzgraber, “Combinatorial
optimization with physics-inspired graph neural networks,” Nat. Mach.
Intell., vol. 4, pp. 367 – 377, 2021.
[37] W. Kool, H. van Hoof, and M. Welling, “Attention, Learn to Solve
Routing Problems!” ICLR, pp. 1–7, 2019.
[38] D. Sidoti, G. V. Avvari, M. Mishra, L. Zhang, B. K. Nadella, J. E.
Peak, J. A. Hansen, and K. R. Pattipati, “A multiobjective path-planning
algorithm with time windows for asset routing in a dynamic weather-
impacted environment,” IEEE Trans. Syst., Man, Cybern., Syst., vol. 47,
no. 12, pp. 3256–3271, 2017.
[39] J. Cheng, M. Ju, M. Zhou, C. Liu, S. Gao, A. Abusorrah, and C. Jiang,
“A dynamic evolution method for autonomous vehicle groups in a
highway scene,” IEEE Internet Things J., vol. 9, no. 2, pp. 1445–1457,
2022.
[40] X.-F. Liu, X.-X. Xu, Z.-H. Zhan, Y. Fang, and J. Zhang, “Interaction-
based prediction for dynamic multiobjective optimization,” IEEE Trans.
Evol. Comput., pp. 1–1, 2023.
[41] J. Li, T. Sun, Q. Lin, M. Jiang, and K. C. Tan, “Reducing negative
transfer learning via clustering for dynamic multiobjective optimization,”
IEEE Trans. Evol. Comput., vol. 26, no. 5, pp. 1102–1116, 2022.
[42] R. Zhang, J. Hao, R. Wang, H. Deng, and H. Wang, “Multi-objective
global path planning for uav-assisted sensor data collection using drl
and transformer,” in Web and Big Data, B. Li, L. Yue, C. Tao,
X. Han, D. Calvanese, and T. Amagasa, Eds. Cham: Springer Nature
Switzerland, 2023, pp. 492–500.
[43] X.-F. Liu, Z.-H. Zhan, Y. Lin, W.-N. Chen, Y.-J. Gong, T.-L. Gu, H.-Q.
Yuan, and J. Zhang, “Historical and heuristic-based adaptive differential
evolution,” IEEE Trans. Syst., Man, Cybern., Syst., vol. 49, no. 12, pp.
2623–2635, 2019.
[44] M. Dorigo and L. Gambardella, “Ant colony system: a cooperative
learning approach to the traveling salesman problem,” IEEE Trans. Evol.
Comput., vol. 1, no. 1, pp. 53–66, 1997.
[45] A. K. Jain, “Data clustering: 50 years beyond k-means,” Pattern Recog-
nit. Lett., vol. 31, no. 8, pp. 651–666, 2010.
[46] C. S. Groer, B. Golden, and W. Edward, “A library of local search
heuristics for the vehicle routing problem,” Math. Prog. Comp., vol. 2,
no. 2, p. 79–101, 2010.
[47] K. Deb and H. Jain, “An evolutionary many-objective optimization
algorithm using reference-point-based nondominated sorting approach,
part i: Solving problems with box constraints,” IEEE Trans. Evol.
Comput., vol. 18, no. 4, pp. 577–601, 2014.
[48] C. Prins, “A simple and effective evolutionary algorithm for the vehicle
routing problem,” Comput. Oper. Res., vol. 31, no. 12, pp. 1985–2002,
2004.
[49] B. Garau, A. Alvarez, and G. Oliver, “Path planning of autonomous
underwater vehicles in current fields with complex spatial variability:
an a* approach,” in ICRA, 2005, pp. 194–198.
[50] B. Miloradović, B. Çürüklü, M. Ekström, and A. V. Papadopoulos,
“Gmp: A genetic mission planner for heterogeneous multirobot system
applications,” IEEE Trans. Cybern., vol. 52, no. 10, pp. 10 627–10 638,
2022.
[51] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist
multiobjective genetic algorithm: Nsga-ii,” IEEE Trans. Evol. Comput.,
vol. 6, no. 2, pp. 182–197, 2002.
Xiao-Fang Liu (Member, IEEE) received the
B.S. degree and Ph.D degree in computer sci-
ence from Sun Yat-Sen University, Guangzhou,
China, in 2015 and 2020, respectively.
She is currently an Assistant Professor with
College of Artificial Intelligence, Nankai Uni-
versity. Her current research interests include
artificial intelligence, evolutionary computation,
swarm intelligence, and their applications in de-
sign and optimization, such as cloud computing
resources scheduling, multirobot systems.
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: SSEN. Downloaded on May 06,2023 at 09:13:06 UTC from IEEE Xplore. Restrictions apply.
11
Yongchun Fang (Senior Member, IEEE) re-
ceived the B.S. degree in electrical engineering
and the M.S. degree in control theory and ap-
plications from Zhejiang University, Hangzhou,
China, in 1996 and 1999, respectively, and
the Ph.D. degree in electrical engineering from
Clemson University, Clemson, SC, USA, in
2002.
He is currently a Professor with the Institute
of Robotics and Automatic Information System
(IRAIS), Nankai University, Tianjin, China. His
research interests include nonlinear control, visual servoing, AFM-based
nano-systems, and control of underactuated systems.
Zhi-Hui Zhan (Senior Member, IEEE) received
the Bachelor’s degree and the Ph. D. degree in
Computer Science from the Sun Yat-Sen Uni-
versity, Guangzhou China, in 2007 and 2013,
respectively.
He is currently the Changjiang Scholar Young
Professor with the School of Computer Sci-
ence and Engineering, South China University
of Technology, Guangzhou, China. His current
research interests include evolutionary compu-
tation, swarm intelligence, and their applications
in real-world problems and in environments of cloud computing and big
data.
Dr. Zhan was a recipient of the IEEE Computational Intelligence
Society (CIS) Outstanding Early Career Award in 2021, the Outstanding
Youth Science Foundation from National Natural Science Foundations of
China (NSFC) in 2018, and the Wu Wen-Jun Artificial Intelligence Excel-
lent Youth from the Chinese Association for Artificial Intelligence in 2017.
His doctoral dissertation was awarded the IEEE CIS Outstanding Ph.
D. Dissertation and the China Computer Federation Outstanding Ph. D.
Dissertation. He is one of the World’s Top 2% Scientists for both Career-
Long Impact and Year Impact in Artificial Intelligence and one of the
Highly Cited Chinese Researchers in Computer Science. He is currently
the Chair of Membership Development Committee in IEEE Guangzhou
Section and the Vice-Chair of IEEE CIS Guangzhou Chapter. He is
currently an Associate Editor of the IEEE Transactions on Evolutionary
Computation, the Neurocomputing, the Memetic Computing, and the
Machine Intelligence Research.
Yun-Liang Jiang received the Ph.D. degree in
computer science and technology from Zhejiang
University, Hangzhou, China, in 2006. He is
currently a Professor with the School of Com-
puter Science and Technology, Zhejiang Nor-
mal University, Jinhua, China, and also with the
School of Information Engineering, Huzhou Uni-
versity, Huzhou, China. His research interests
include intelligent information processing and
geographic information systems.
Jun Zhang (Fellow, IEEE) received the Ph.D.
degree from the City University of Hong Kong,
Kowloon, Hong Kong, in 2002.
He is currently an Honorary Professer with
Zhejiang Normal University, China and also a
BP professor with the Hanyang University, ER-
ICA, South Korea. He has published over 180
IEEE Transactions papers in his research area.
His research interests include computational in-
telligence, cloud computing, data mining, and
power-electronic circuits. Professor Zhang was a
recipient of the China National Funds for Distinguished Young Scientists
from the National Natural Science Foundation of China in 2011 and was
appointed as a Cheung Kong Chair Professor in 2013 from the Ministry
of Education, China, in 2009. He is currently an Associate Editor of the
IEEE Transactions on Evolutionary Computation and IEEE Transactions
on Cybernetics.