VIET NAM NATIONAL UNIVERSITY HO CHI MINH CITY

UNIVERSITY OF TECHNOLOGY

NGUYEN VAN DONG

SWIMMING GAIT CONTROL OF ELONGATED UNDULATING FINS

BASED ON THE CENTRAL PATTERN GENERATOR

Major: Mechanical Engineering

Major code: 62520103

SUMMARY OF PhD DISSERTATION

HO CHI MINH CITY - 2022

This dissertation is completed at the University of Technology – VNU-HCM

Science advisor : Assoc. Prof. Nguyen Tan Tien, PhD

Independent reviewer 1:
Independent reviewer 2:

Reviewer 1:
Reviewer 2:
Reviewer 3:

The dissertation will be defended in front of the board of examiners at

...............................................................................................................................
...............................................................................................................................

This dissertation can be found in the libraries:

- The library of the University of Technology – VNU-HCM
- General science library - Vietnam National University – Ho Chi Minh
City
- General Science Library - Ho Chi Minh City
CHƯƠNG 1 INTRODUCTION

This dissertation describes the research work done in determining the scientific
basis for modeling and selecting the appropriate number of fin rays per
wavelength for the propulsion module using a biomimetic swimming
mechanism; the use of a reinforcement learning algorithm in determining the
optimal coefficient for the time to change swimming posture while minimizing
swimming form error. At the same time, research to find the optimal swimming
shape for maximum thrust at a specific frequency to create the best moving
dynamics while keeping a fixed undulating frequency to minimize the risk of
detonation. Underwater sound mines. In addition, research motivation and
outline are discussed in this chapter.
1.1 Background
Compared to screw propellers, the movement of fish fins or bodies can give
underwater robots more maneuverability, which can be used to fine-tune their
positions. These abilities inspire new designs that make it easier for artificial
systems to operate in and interact with water. The underwater ecosystem is also
an essential part of the study of BIUVs, especially since marine life has been
getting worse because of how often propellers, which make loud noises in the
wake, have been used. Fish move without making noise because of the way they
swim. Because of this, engineers are also forced to develop new ways to make
vehicles that haven't rotary propellers.

The goal of the research on fish robots in robotics is simple: to turn the idea
behind biomimetic fish into new underwater vehicles that can help people. To
achieve this goal, researchers need to study many things, such as the mechanical
design of fish robots, the materials of biomimetic propellers, the methods of
actuation and actuators for underwater environments, the sensors and electronic
systems for underwater measurements, the control of swimming for highly
efficient locomotion, intelligent control strategies for autonomous
manipulations. This dissertation focuses on exploiting the motion controller
aspect of the propulsion module using the swimming mechanism of the
Gymnotiform fish class.
1.2 Motivation

a new inadequacy arises from the characteristics of the environment where the
mines are located, which is often a mossy environment with a lot of ocean
garbage. The robots using propellers are all stuck and not working effectively,
so a solution is needed and solutions to these problems.

The above situation, concerning the operating mechanism of fish robots in the
world, has motivated me to conduct a research-oriented approach to underwater
robots with high stability and a rigid body to install and place the devices. The
contributions in the thesis are the foundation for the orientation of building a
complete underwater robot for surveying and clearing mines left on the seabed.
1.3 Literature review
This section summarizes the studies on the locomotion of fish classes, and
references the works on biological anatomy explaining the mechanism of fish
activity. In addition, the works closest to the research area mentioned for
reference and inheritance are also stated with the main ideas.
1.4 Discussion

By referencing the research combined with the interpretation of statements, in

the limited awareness of the individual, I have discovered two main issues that
guide my personal research, specifically as follows:

1) The above-aforementioned studies regarding CPG-based bio-fish robots

have not conducted optimization for the convergence rate (Characteristic
coefficient for the time to change swimming form). Inspired from the
studies concerned with applying RL for CPG, this dissertation proposes
 a reinforcement learning-based optimization of locomotion controller
using CPG network for an elongated undulating fin.

2) For each swim frequency, the set of profile parameters of the fin rays
determine the swimming shape and thrust. To get the best thrust it is
necessary to find out this set of parameters. This dissertation investigates
a new ideal for differential particle swarm optimization (D-PSO) to
improve optimization problems. The amplitude values of the CPG
network increase the average propulsive force of the undulating fin
robot to make a faster movement.
1.5 Outline of the Dissertation
The dissertation is presented, including six chapters:

Chapter 1 focuses on researching scientific publications in the same field to find

out the contribution orientation of the dissertation.

Chapter 2: Building a Motion Controller for a Specific Fish Robot Propulsion

Module Model Available on the CPG Platform Simultaneously, model the
propulsion mentioned above system module.

Chapter 3: Research on optimizing the specificity coefficient for the stroke

switching speed of the locomotor controller built by a reinforcement learning
algorithm

Chapter 4: Research on selecting the optimal set of amplitude parameters for the
motion controller with the criterion of keeping the frequency unchanged and
achieving the maximum thrust by the swarm optimization algorithm

Chapter 5: Testing the ability to change swimming posture flexibly, optimizing

the speed characteristic coefficient of the change of swimming form found in
Chapter 3, and measuring the thrust caused by the best set of parameters found
in Chapter 4.

Chapter 6: Conclusions and Future Research

CHƯƠNG 2 DESIGN SWIMMING GAIT CONTROLLER AND
THRUST MODELING

This Chapter proposes a locomotion controller inspired by black Knifefish for

undulating elongated fin robot. The proposed controller is built by a modified
CPG network using sixteen coupled Hopf oscillators with the feedback of the
angle of each fin-ray. By employing the proposed controller, the undulating
elongated fin robot can realize swimming pattern transformations naturally.
Additionally, the proposed controller enables for configuration of the swimming
pattern parameters are known as the amplitude envelope, oscillatory frequency
to perform various swimming patterns.

2.1 Elongated undulating fin description

The elongated undulating fin comprises sixteen oblique adjacent fin-rays
interconnected with a flexible membrane. Each fin-ray is driven by an RC servo
motor that enables the fin-ray to sway around a rotary joint fixed to a supporting
frame .Accordingly, each fin-ray reacts as a shaker bar with a limited angle, and
the phase difference between two adjacent fin-rays is regarded as a phase lag
angle. By changing one of the kinematic parameters, such as amplitude envelope,
oscillatory frequency, and swimming pattern, the magnitude of the propulsive
force can be adjustable. To perform forwarding/reversing motion, the elongated
undulating fin might change the sign of the phase lag angle. Additionally, to
avoid the counter-torque of the elongated undulating fin, the number of
oscillation wavelengths should be an even number.
2.2 Swimming gait controller for elongated undulating fin base on CPGs
the suitable CPGs model for Elongated Undulating Fin Robot is determined as
follows:
• Oscillating:
With references and analysis of oscillators commonly applied to CPG, in this
case (motion of fins along the body). Hopf oscillator was chosen due to its high
stability and good self-healing in the disturbing underwater environment.

Coupling Schemes

Because of the biology of fish with caudal fins along the body, each fin ray is
equal, but there is a mutual interaction with the fin rays immediately before and
after fin rays. so Multiple Coupling is best suited to make realistic fish-like
movements.
• Configurations of Oscillators

According to the biological structure of Elongated Undulating Fin, the fin rays
are arranged in a long row, natural movement is operated by chain coupling
structure. The design in the following sections will comply with the chain
coupling structure.
By these options, the locomotion controller for the studied fin model is
presented Figure 2-17.

Figure 2-17. chain coupling structure CPGs model for Elongated Undulating
Fin

According to this structure, the fin will be in the state of self-propelled control
with the parameters f, A1…A16 from the higher-order controller.

For the first oscillator (𝑖 = 1), there is only perturbation from the second
oscillator (𝑖 + 1), thus the perturbation of the first oscillator is given by:
0
𝑃! = ( 3
𝛽(𝑣" 𝑐𝑜𝑠𝜑# − 𝑢" 𝑠𝑖𝑛𝜑#
where 𝛽 is the coupling strength; 𝜑# is the phase lag angle of two adjacent
oscillator.

In the same manner, the sixteenth oscillator is only affected by the perturbation
from the fifteenth oscillators:
0
𝑃!$ = ( 3
𝛽(𝑢!% 𝑠𝑖𝑛𝜑# + 𝑣!% 𝑐𝑜𝑠𝜑#

For 𝑖-th (1=2,3…14,15) oscillators, the perturbation vector is given by the

following:
0
𝑃& = ( 3
𝛽(𝑢&'! 𝑠𝑖𝑛𝜑# + 𝑣&'! 𝑐𝑜𝑠𝜑# − 𝑢&(! 𝑠𝑖𝑛𝜑# + 𝑣&(! 𝑐𝑜𝑠𝜑#

Corresponding to various amplitudes 𝐴& , the modified CPG network can provide
different swimming patterns for the elongated undulating fin, it thus can produce
different propulsive forces.
2.3 modeling of elongated undulating fin
The diaphragm is the primary and most important part of the fin module because
it is the direct agent that generates thrust. The diaphragm modeling was done to
investigate the factors affecting the motion and identify the design elements. In
the thesis, the model is adjusted from the model introduced by Sfakiotakis.
 N

θN
7
5 6
4
3
2
1

θ1 L

Figure 2-18. Fin Discrete Model

Consider the point 𝑞 located on the fin membrane between two ith and i+1th fin
ray. Set the inertial coordinate system {𝑃} and the local coordinate system of the
fin {𝑂} with the origin set at the head of the first fin ray see Figure 2-19 .

D
x
{O}

x
{P} y h
r q(i+1)
r q
rq(i) z
y w

Figure 2-19. Representation of coordinate systems

the component of force produced by the diaphragm at point 𝑞 is:

1 1 "
𝑓⃗ = <<<<<⃗
𝑓)* = − 𝜌𝐶) @𝑉 <<<⃗ <<<⃗ <<<⃗ <⃗+ 𝑠𝑔𝑛C𝑉
) @𝑉) = − 𝜌𝐶) @𝑉) @ 𝑛
<<<⃗
)D
2 2
By integrating the force 𝑓⃗ over the entire diaphragm with ∈ [0 15], ℎ ∈
[ℎ,&) ℎ,-. ], 𝑤 ∈ [𝑤,&) 𝑤,-. ]. Equation of force produced by the entire
diaphragm:
 1'! 0!"# /!"#

𝐹⃗ = 𝐹
<<<⃗
) = L M
̇ 𝑑𝑤𝑑ℎ
M 𝐹)*
&2+ 0!$% +

2.4 Simulator of swimming gait control base on CPGs

In this section, simulate the swimming postures of fish fins using a CPG-based
locomotion controller. in turn, the traditional fish swimming postures are
produced by different sets of amplitudes of the fin rays. simulated swim poses
including linear envelope, quadratic envelope and elliptic envelope.
Simultaneously the activities change the swimming strokes (thanks to the
changing set of fin ray amplitudes), change the frequency of the fin rays then
witness the flexible and smooth transition of the locomotor controller base on
CPG.

CHƯƠNG 3 OPTIMIZING CONVERGENCE SPEED OF SWIMMING

GAIT CONTROLLER BASE ON CPG BY REINFORCEMENT LEARNING

3.1 Problem stamen

The studies regarding CPG-based bio-fish robots have not conducted
optimization for the convergence speed. Inspired from these, this paper proposes
a CPG-based locomotion controller for an elongated undulating fin. The
elongated undulating fin comprises sixteen oblique fin-rays interconnected with
a membrane known as a flexible surface that is controlled by the proposed
locomotion controller coupled with sixteen neural oscillators to generate the
locomotor corresponding to sixteen fin-rays of the elongated undulating fin. The
advantages of this control method in comparison to the sinusoidal kinematic
equation are discussed. To optimize the convergence rate for the processing time
and the oscillatory error of the CPG output, Q-learning is adopted. The proposed
locomotion controller and reinforcement learning can be implemented on a
microcontroller. The simulation and experimental results are carried out to
evaluate the performance and effectiveness of the proposed control method.
3.2 Theoretical foundations of reinforcement learning
In this section, we briefly review information about reinforcement learning
algorithms. The concepts are raised and discussed in unison as the basis for
establishing a self-learning system by reinforcement learning.
3.3 Reinforcement learning based optimization convergence speed.

The interaction of the agent and the environment of RL is shown in Figure 3-2

Figure 3-2: Interaction of agent and environment

The reward function is designed to minimize the cost in the trade-off between
the transient-state time and the oscillatory error that the mathematical proposed
reward function is given the following:

-
𝑟3 &→3' = 𝐿5 𝑟! (𝑠6! ) + 𝐿7 𝑟" (𝑠6" )
& &

where 𝑠68 is the next state variable, and 𝐿5 , 𝐿7 are reward constants set arbitrarily
such that the condition holds 𝐿5 ≫ 𝐿7 to emphasis that the minimization of the
oscillatory error is more significant than that of the transient-state time. Thus,
𝐿5 , 𝐿7 are respectively set to 100 and 10 in this case. The reward subfunctions
𝑟& C𝑠6& D with 𝑖 = 1, 2 are given by the following:
𝑅,-. V𝑠6& V < minC𝑆 & D
𝑟& C𝑠6& D = T 𝑅,&) V𝑠6& V = minC𝑆 & D
0 V𝑠6& V > minC𝑆 & D
where 𝑅,-. , 𝑅,&) are the maximum reward and the minimum reward set to 1
and 0.1, respectively.

As well, the terminal state 𝑠9 known as the condition for complete an episode
holds the constraint 𝑠9 ≔ ^𝑠6 ∈ 𝑆_𝛿 ≜ C𝐿5 |𝑠6! | + 𝐿7 𝑠6" D ≤ min(Δ; )e with Δ; is
:
the compact set of 𝛿 of each episode.

The Q-value (action-value) function is updated by the simple Temporal

Difference (TD) method:

-
𝑄6 (𝑠6 , 𝑎6 ) = 𝑄6'! (𝑠6 , 𝑎6 ) + 𝛼 i𝑟3 &→3' + 𝛾 max
'
𝑄6'! (𝑠68 , 𝑎68 ) − 𝑄6'! (𝑠6 , 𝑎6 )m
& & -&

where 𝛼 is the learning rate (0 ≤ 𝛼 < 1); 𝛾 is the discount factor (0 ≤ 𝛾 < 1);
𝑎68 is the next action variable; 𝑄6'! (∎) denotes the current Q-value; 𝑄6 (∎)
denotes the new Q-value;
The next policy 𝜋 8 (𝑎6 , 𝑠6 )is implemented by 𝜀-Greedy strategy which is
given by:
argmax 𝑄6'! (𝑠6 , 𝑎6 ) 𝑞 < 1−𝜀
𝜋 8 (𝑠6 , 𝑎6 ) = q -&
randC𝑄6'! (𝑠6 , 𝑎6 )D otherwise

where 𝑞 is the uniform random number.

The optimal convergence speed can be determined by the optimal action value:
𝑎6∗ = argmax 𝑄(𝑠6 , 𝑎6 )
-&

The pseudo-code of the Q-learning optimization for the convergence

speed is illustrated in Table 3-1. The impact of the transient-state time and the
oscillatory error on the convergence speed is depicted in Figure 3-a). As well,
the distribution of Q-value on the state variable and the action variable is
illustrated in Figure 3-b).
 Table 3-1. Pseudo-code of the Q-learning optimization

Algorithm: Q-learning based optimization of the convergence speed

1. Initialize 𝛼, 𝛾, 𝜀
2. Initialize 𝑄6'! (𝑠6 , 𝑎6 ) = [0], 𝑠6 = 𝑟𝑎𝑛𝑑(𝑆), and episode 𝑛
3. Repeat for each step of episode:
4. Choose 𝑎6 = argmax 𝑄(𝑠6 , 𝑎6 ) if uniform random number < 1 − 𝜀
-&
5. Choose 𝑎6 = rand(𝑄(𝑠6 , 𝑎6 ) if otherwise
6. Take the action 𝑎6 (traveling the convergence speed 𝑘 to the modified
CPG network)
-
7. Observe 𝑠68 , 𝑟3 &→3' (perceiving the oscillatory error and the transient-
& &
state time, calculating the reward value
8. Update Q-value
9. The next state is assigned as the next state (𝑠6 ← 𝑠68 )
10. Until the current state is the terminal state (𝑠6 ≡ 𝑠9 )
11. Take the optimal action 𝑎6∗ = argmax 𝑄(𝑠6 , 𝑎6 )
-&

According to the implement of the Q-learning based optimization for the

convergence speed with the discount factor 𝛾 = 0.75, the learning rate 𝛼 = 0.95,
the 𝜀 – greedy of 0.2, and the episode number 𝑛 = 1000, the optimal Q-value
achieved the approximately value 𝑄∗ (𝑠6 , 𝑎6 ) = 195.66 with respect to the
optimal action of 𝑎6∗ = 96. The convergence speed of the modified CPG network
thereby is chosen as 𝑘 = 96 for both simulation studies and experimental studies
in next section.
 Figure 3-7: a) Impact of transient-state time and oscillatory error on the
convergence speed. b) Distribution of Q-value on state variable and action
variable

3.4 simulation and discussion

In this Chapter, the simulation study of the modified CPG network is conducted
through MATLAB with the aim that is to evaluate the flexible transition gait of
the elongated undulating fin relevant to the swimming pattern, intrinsic
amplitude, oscillatory frequency, and the number of waveforms. The swimming
patterns utilized in this research are illustrated in Figure 3-. The simulation
results also demonstrate the affection of the convergence rate on the transient-
state time and the oscillator error of the modified CPG network.

Figure 3-8. Swimming patterns of elongated undulating fin propulsion

Characteristic of convergence rate

 Figure 3-9. The relative convergence rate concerning transient-state time and
oscillatory error.

The modified CPG parameters are given for this study as 𝐴& = 1(with 𝑖 = 1 ÷
16) , 𝑓 = 1, 𝜑# = −𝜋⁄3 , 𝛽 = 0.8 to allow the fin-rays to perform the
cuttlefish-like swimming pattern. Depicts the output of a single oscillator with 𝑘
chosen arbitrarily around the optimal value of 96 for comparison. As can be seen,
with 𝑘 = 86, the transient-state time is nearly obtained as 1.45 seconds, whereas
that of the case 𝑘 = 96 is approximately value of 1.41 seconds compared to the
case of 𝑘 = 106 as 1.36 seconds. It is easy to note that the larger amount of 𝑘
will result in the reducing of the transient-state time due to the modified CPG
output converged to the limit cycle. Nevertheless, increasing the convergence
rate 𝑘 will cause the larger oscillatory error of the modified CPG output
illustrated in Figure 3-, which might affect the performance of the actuators
powered for fin-rays. Therefore, the oscillator error is recognized as the more
significant factor than the transient-state time.
3.5 Conclusions
This Chapter has presented the modified CPG network for generating the rhythm
for the elongated undulating fin with sixteen fin-rays to mimic the fish’s
swimming patterns. Accordingly, the modified CPG network is composed by
chain coupling sixteen oscillators with bidirectional perturbation because each
fin-ray is only affected by its two adjacent oscillators. Both simulation and
experimental results show that the modified CPG network seems to be very
promising to perform the rhythm for a fish robot. It allows changing the
kinematic parameters abruptly with no jerk of oscillation. Additionally, this
paper has also investigated the intrinsic parameter of the CPG known as the
convergence rate, which has not been considered before, usually using the trial-
and-error method for this issue. The simulation results have revealed that the
large convergence rate can reduce the transient-state time; however, it might
cause the oscillator error worse. Therefore, the tunning of the convergence rate
is to trade-off between the transient-state time and the oscillatory error. To deal
with this issue, the Q-learning algorithm is appropriate to find the optimal
convergence rate. To obtain smooth oscillation avoiding damage to the RC servo
motor, the reward function of the Q-learning is defined with more significant
oscillatory error than the transient-state time. The optimal convergence rate
found by the Q-learning can provide the short transient-state time and the
appropriate oscillatory error in the simulation/experimental results with the
abrupt change of kinematic parameters such as amplitude envelope, oscillatory
frequency, and waveform number. Especially, we have found that the transient-
state time is longer in the case of using the large convergence rate when the phase
lag angle is changed into the opposite value for reverse swimming. However, a
change of the convergence rate while the limit cycle of the CPG is obtained does
not affect the CPG output. Thus, this might raise a piece-wise switching function
to change the convergence rate according to the swimming operation.
Consequently, the convergence rate should be changed from the optimal value
into a smaller appropriate value before the phase lag angle is changed to switch
forwarding swimming into backward swimming and vice versa. Afterward, the
convergence rate is again changed into the optimal value to obtain the short
transient-state time.

CHƯƠNG 4 FORCE OPTIMIZATION OF ELONGATED

UNDULATING FIN ROBOT USING IMPROVED PSO BASED CPG

4.1 problem statement

In research that we have published, four locomotion patterns are carried out to
evaluate the locomotion patterns and thrust force influence. The first three
locomotion patterns are similar to nature species (cuttlefish, knife fish, stingray)
respectively, and the fourth locomotion pattern is selected for comparison
without referring to natural aquatic.

Figure 4-1. Four locomotion patterns

The main idea of this chapter is to use the commonly used optimization
algorithms for the selection of control parameters to find the set of Gi parameters
in the numerical equation 2.30, with the objective function being the maximum
thrust for each set of parameters swimming frequency. At first, a CPG using the
chain topology of sixteen coupled Hopf oscillators is offered to generate fishlike
rhythmic movements. Following, the improved D-PSO is exploited to optimize
the amplitude values of the CPG network to increase the average propulsive force
of the undulating fin robot to make a faster movement. Finally, the obtained
result of CPG parameter synthesis using the different optimization methods,
including DPSO, PSO and GA, is implemented to proves the superiority of the
proposed D-PSO algorithm.
4.2 Theory of Particle Swarm Optimization (PSO)
Particle Swarm Optimization (PSO) is an algorithm to solve optimization
problems on intelligent population or swarm intelligence (SI). PSO algorithm
was first introduced and proposed by Kennedy and Eberhat in 1995. About these
authors, in spite of developing simple methods, it works effectively in optimizing
nonlinear continuous functions, extremum of a functional and some other multi-
objective optimization problems. This algorithm is inspired by the observation
of biological populations in nature, the cooperative behavior displayed by
various species, the organization and activities in population in order to give a
fastest optimal solution. Thus far, the idea of PSO is to simulate the method of
searching food from animal swarms such as bird and fish, that is, finding food
by its and neighbor’s information.

PSO is one of evolutionary computation techniques, however, PSO also has a

few differences compared to other evolutionary techniques. Similar to other
techniques, PSO also creates population initialization by using the random
distribution method. The appropriateness of PSO is this algorithm is based on the
evaluation and the selection of alternatives by informations about the experience
of each individual and swarm. This selection is better than classical evolutionary
methods such as GA.

In recent years, with high efficiency and benefit, PSO algorithm has been
developed rapidly, appearing in many researches of ameliorating parameters. All
the results have been proven by many popular functions, viz. Shaffer, Sphere,
Rosenbrock, Rastrigin and Griewank. As a consequence, PSO has applied for
solving some problems: Unconstrained Optimization, Constrained Optimization,
Multi Objective Optimization, Dynamic Optimization Problem, etc.
4.3 Developed PSO-based CPG Optimization
The performance of CPG model is susceptible to the value of amplitude, and
hence it is necessary to seek the best parametric values for purpose of improving
the propulsive performance. In this sub-section, a novel D-PSO is applied to
obtain a set of optimum amplitudes by maximizing the average propulsive force
called the objective function. The optimization problem of CPG model can be
posed in the following way:
Dt
1
Dt ò0
Maximizing: F= F (t )dt (8)
ì Amin £ A £ Amax
Subject to: í (9)
î Ai < Ai +1 , i = 1 ÷ 16
 Installing parameters of D-PSO

Initializing the oscillation amplitudes of CPG

within their ranges

Calling the Hopf oscillator-based CPG

Evaluating the initial fitness function of each

individual and choosing Pbest and Gbest

Set ite = 1

Updating the velocity and the position of each

individual

Evaluating the fitness function of each

individual and calculating Pbest and Gbest

Yes
ite=ite+1 ite <= Maxite ?

No

The optimum amplitude of CPG as G best

Best Avarage Thrust

Figure 4-5: Flowchart of the proposed approach

The developed D-PSO-based CPG optimization problem is performed as

follows:

Select some parameters of D-PSO, including w, c1, c2 and c3

The initial positions and velocities of each individual in swarm are selected by
random values

Initialize oscillation amplitudes of CPG model within their ranges

Call the Hopf oscillator-based CPG model

The fitness function of each individual is evaluated by:

FN i ite = f ( X i ite ) ,∀i

The particle having the best position is indexed as p, and hence the personal
experience and the overall experience are selected as follows:

Pbesti ite = X i ite , and Gbest ite = X p ite

Initialize the iteration at ite = 1

The velocity and position of each individual are updated

The updated fitness function of each particle is re-evaluated:

FN i ite +1 = f ( X i ite +1 ) ,
∀i and indexing for the particle with the best position as q
The personal experience and the overall experience of swarm are updated:

If FN i ite +1 < FN i ite then Pbest i ite +1 = X i ite +1 else Pbest i ite +1 = Pbest i ite ,

If FN q ite +1 < Fb k then Gbest ite +1 < Pbest q ite +1 and select p = q else Gbest ite +1 <
If ite < Maxite then ite = ite +1 and goto the step 7 else goto the step 10

The optimal parameters of CPG are obtained as Gbest ite , and hence the maximum
thrust force is defined.

A detailed flowchart of proposed D-PSO-based CPG model considering the

above steps is shown in Figure 4-9.
4.4 Test Results and Discussion
The proposed D-PSO-based CPG optimization method is performed both on the
simulation model in MATLAB and during an experiment with the real elongated
undulating fin. An actual sixteen fin-ray robot with a size of 775 mm long, 90 m
wide and 290 m deep was designed for the experimental purpose. The actual
testing result of the output signals of CPG is illustrated in Figure 4-

Figure 4-6. The output of the real CPG model

The average thrust is 3.60 N in the case of using the proposed D-PSO, while the
values of force only reach 3.58 N and 3.57 N by PSO and GA, respectively. Also,
the proposed D-PSO takes less convergence time than that of PSO and GA, 2.3
iterations compared to 3.9 iterations and 10.5 iterations, respectively. It can be
further seen from Figure 4-11. The convergence characteristic of some CPG
optimization techniques that there are two steps in the convergence characteristic
of PSO and GA, so it is easy for these two optimization methods to get trapping
the local maxima. This problem has been solved by using the proposed D-PSO,
whose optimization result only shows one step. It means the D-PSO is the ability
to achieve the global best position better than the others.
 Figure 4-11. The convergence characteristic of some CPG optimization
techniques

4.5 Conclusions
In this Chapter, the locomotion control and optimization task has been
successfully performed on the elongated undulating fin robot inspired by the
black knife fish. A bi-directionally coupled sixteen Hopf oscillator-based CPG
network is investigated to create rhythmic movement as a perfect paradigm for
robotic locomotion. In addition, a novel variant of PSO, namely D-PSO, has been
developed to obtain a set of optimum amplitudes of the modified CPG network
for the purpose of maximizing the average propulsive force. In brief, the new
contributions in our study can be drawn as follows:
1) In the term of motion control scheme, the CPG based on coupled Hopf
oscillators not only generates the desired rhythm for the undulating fin robot but
also removes the effect of radial interference on the oscillatory amplitude.

2) The proposed D-PSO algorithm is the capacity to successfully solving the

basic math functions and effectively improving the average thrust of the untested
undulating fin fish obtained from the output signals of modified CPG.

3) The novel D-PSO algorithm gives the fastest propulsive force as compared to
PSO and GA. Moreover, the superiority of the proposed method is presented in
the ability to avoid the local maxima in order to increase the accuracy of the
optimization process.

CHƯƠNG 5 EXPERIMENT

The elongated undulating fin comprises sixteen oblique adjacent fin-rays

interconnected with a flexible membrane. Each fin-ray is driven by an RC servo
motor that enables the fin-ray to sway around a rotary joint fixed to a supporting
frame illustrated in Figure 5-1.

Specific parameters are presented in the Table 5-1 To produce the propulsive
force, the elongated undulating fin performs the sinusoidal oscillatory
propagation along to the fin from the anterior to the posterior.

Figure 5-1: Overview of elongated undulating fin

 Table 5-1. Specific parameters of elongated undulating fin
Paramater Value Unit

length 775 mm

Width 90 mm

Height 290 mm

Number of finrays 16 Pcs

Servo number in model selection is 16; this is the result of our own research that
has been published [40]. The distance between the two rays is limited by the
mechanical structure and is selected to be 30mm. The selected servo motor is a
commercially available type with specific parameters in Table 5-2.
Table 5-2. Servo RC specific

Paramater Value

Operating Voltage Range 6V-7.4V

Idle current (at stopped) 6mA@7.4V

No load speed: 0. 192sec/60°@7.4V

Runnig current (at no load): 180mA@7.4V

Peak stall torque: 35.5kg.cm@7.4V

Rated torque: 11.8kg.cm@7.4V
Stall current: 3.9A@7.4V

Experimentation is a very important part of research, especially with topics

related to application direction. In this chapter, experiments are arranged to
clarify the following factors:
• Ability to flexibly switch the CPG-based motion controller when changing
frequency, amplitude
• Verify the results of convergence rate of some random convergence values K
compared with the optimal value found in chapter 3
• Compare the thrust generated at a given frequency, a certain convergence
factor K of the optimal set of amplitude parameters found in chapter 4 and a
few sets of random parameters.

The experimental results show the accuracy in the simulations. Although

the errors and disturbances cause the graph to differ slightly from the
simulation, the nature of the problem has been proven experimentally.
CHƯƠNG 6 CONCLUSIONS

6.1 Dissertation contributions

In the era of the development of biomimetic robotics and ocean science,
Underwater robots will continue to play an important role in helping us better
understand the working mechanisms of underwater nature. Especially with the
learning characteristics of the fish robot swimming mechanism, it will certainly
have a certain influence on the underwater military field and can create a
revolution in underwater military equipment.

The thesis refers to a new approach to optimizing the parameters of the motion
controller of the robot based on the motion method of a fish with longitudinal
fins.

First, the thesis makes a new scientific contribution by showing the scientific
gap in the convergence coefficient characteristic of the robot's swimming gait
switching speed. The reinforcement learning method for determining the optimal
convergence coefficient is a new and modern method that, like the mathematical
model developed in this thesis, can be applied to all sizes.

The second new contribution the thesis wants to mention is determining

swimming posture by the swarm optimization method. Simulations and
experiments prove the correctness of this approach.
6.2 Future work
From the results obtained in the thesis. Further work will be done to perfect the
locomotion controller with optimal parameters. Specifically, we have now
prepared new machine learning algorithms to help optimize energy consumption
for each specific movement need. In the future, the CPG-based locomotion
controller still has many scientific gaps for further research.
 LIST OF PUBLICATIONS
International journals/Book series (ISI-Scopus)
J1. V. D. Nguyen, “Force Optimization of Elongated Undulating Fin Robot
Using Improved PSO-Based CPG.” Computational Intelligence and
Neuroscience, vol. 2022, pp. 1-11, 2022, doi: 10.1155/2022/2763865. Impact
Factor: 3.633; Journal Rank: 3615; SJR: 0.605; Quartile: Q1
J2. V. D. Nguyen, D. Q. Vo, V. T. Duong, H. H. Nguyen, and T. T. Nguyen,
“Reinforcement learning-based optimization of locomotion controller using
multiple coupled CPG oscillators for elongated undulating fin
propulsion.” Mathematical Biosciences and Engineering, vol. 19, no. 1, pp. 738-
758, 2021, doi: 10.3934/mbe.2022033. Impact Factor: 1.285; H Index: 45; SJR:
0.451; Quartile: Q2
J3. V. H. Nguyen, V. T. Duong, H. H. Nguyen, V. D. Nguyen, V. S. Le, and T.
T. Nguyen, “Kinematic Study on Generated Thrust of Bioinspired Robotic with
Undulating Propulsion.” 2020 International Conference on Advanced Mecha-
tronic Systems (ICAMechS), 2020, doi: 10.1109/icamechs49982.2020.9310158.
Impact Factor: 0.29 ; H Index: 13; SJR: 0.148; Quartile: Q4

National journals
Proceedings of the international conferences
C1. V. D. Nguyen, C. A. T. Pham, V. H. Nguyen, T. P. Tran, and T. T. Nguyen,
“Modular Design of Gymnotiform Undulating Fin.” Lecture Notes in Electrical
Engineering, pp. 924-931, 2019, doi: 10.1007/978-3-030-14907-9_89.
C2. V. D. Nguyen, D. K. Phan, C. A. T. Pham, D. H. Kim, V. T. Dinh, and T. T.
Nguyen, “Study on Determining the Number of Fin-Rays of a Gymnotiform
Undulating Fin Robot.” Lecture Notes in Electrical Engineering, pp. 745-752,
2017, doi: 10.1007/978-3-319-69814-4_72.
C3. V. H. Nguyen, C. A. T. Pham, V. D. Nguyen, D. H. Kim, and T. T. Nguyen,
“A Study on Force Generated by Gymnotiform Undulating Fin.” 2018 15th
International Conference on Ubiquitous Robots (UR), 2018, doi:
10.1109/urai.2018.8441812.
C4. V. H. Nguyen, C. A. T. Pham, V. D. Nguyen, and T. T. Nguyen, “Study on
Velocity Control of Gymnotiform Undulating Fin Module.” Lecture Notes in
Electrical Engineering, pp. 714-722, 2020, doi: 10.1007/978-3-030-53021-1_71.
C5. V. H. Nguyen, C. A. T. Pham, V. D. Nguyen, H. L. Phan, and T. T. Nguyen,
“Computational Study on Upward Force Generation of Gymnotiform Undulating
Fin.” Lecture Notes in Electrical Engineering, pp. 914-923, 2019, doi:
10.1007/978-3-030-14907-9_88.
Proceedings of the national conferences
Other related publications
C1. A. S. Nguyen, V. D. Nguyen, H. H. Nguyen, and T. T. Nguyen, “A Novel
Approach for Determining a Hit Point Based on Estimating Target Movement
and Ballistic Table.” Lecture Notes in Electrical Engineering, pp. 703-713, 2020,
doi: 10.1007/978-3-030-53021-1_70
C2 C. N. Nguyen, T. N. L. Le, V. D. Nguyen, T. P. Tran, and T. T. Nguyen, “A
Study on Localization of Floating Aquaculture Sludge Collecting
Robot.” Advances in Asian Mechanism and Machine Science, pp. 512-518, 2021,
doi: 10.1007/978-3-030-91892-7_48.
Science research projects