Editor-in-Chief

Dr. Xinggang Yan

University of Kent, United Kingdom

Editorial Board Members

Chong Huang, United States Mohamed Riahi, UAE
Yoshifumi Manabe, Japan Yu Tao, China
Hao Xu, United States Neelamadhab Padhy, India
Shicheng Guo, United States Baofeng Ji, China
Diego Real Mañez, Spain Maxim A. Dulebenets, United States
Santhan Kumar Cherukuri, India Jafar Ramadhan Mohammed, Iraq
Asit Kumar Gain, Australia Shitharth Selvarajan, India
Sedigheh Ghofrani, Iran Schekeb Fateh, Switzerland
Saleh Mobayen, Iran Dadmehr Rahbari, Iran
Youqiao Ma, Canada Jun Shen, China
Sasmita Mohapatra, India Yuan Tian, China
Akram Sheikhi, Iran Abdollah Doosti-Aref, Iran
Soltani Sharif Abadi Ali, Poland Fei Wang, China
Tibor Krenicky, Slovakia Xiaofeng Yuan, China
Mohamed Wiem Mkaouer, United States Tajudeen Olawale Olasupo, United States
Marco Bonopera, Taiwan, China Md. Mahbubur Rahman, South Korea
Xinqing Xiao, China Héctor F. Migallón, Spain
Jong-Hwan Kim, South Korea
 Volume
Volume5 Issue 2 •1October
5 Issue • April 2023 • ISSN 2661-3204 (Online)

Journal of
Electronic & Information
Systems
Editor-in-Chief
Editor-in-Chief
Prof.
Dr.Xinggang
XinggangYan
Yan
 Volume 5 ｜ Issue 2 ｜ October 2023 ｜ Page1-66
Journal of Electronic & Information Systems

Contents
Articles

1 Stressed Coral Reef Identification Using Deep Learning CNN Techniques

S.P. Aruna, M. Thamarai
10 The Application of Information Systems to Improve Ambulance Response Times in the UK
Alan Slater
25 Practical Considerations for Implementing Adaptive Acoustic Noise Cancellation in Commercial
Earbuds
Agustinus Oey
35 Control and Treatment of Bone Cancer: A Novel Theoretical Study
Ali Soltani Sharif Abadi, Mansour Rafeeyan, Vahid Abootalebi
45 Enhancing Semantic Segmentation through Reinforced Active Learning: Combating Dataset Imbalances
and Bolstering Annotation Efficiency
Dong Han, Huong Pham, Samuel Cheng
61 Development of Technology and Equipment for Non-destructive Testing of Defects in Sewing Mandrels
of a Three-roll Screw Mill 30-80
Shatalov Roman Lvovich, Zagoskin Egor Evgenievich
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

Journal of Electronic & Information Systems

https://journals.bilpubgroup.com/index.php/jeis

ARTICLE

Stressed Coral Reef Identification Using Deep Learning CNN Techniques

S.P. Aruna1, M. Thamarai2*
1
Skilltroniks Technologies, Tadepalligudem, Andhra Pradesh, 534101, India
2
ECE Department, Sri Vasavi Engineering College, Tadepalligudem, Andhra Pradesh, 534102, India

ABSTRACT
Deep learning is a machine learning technique that allows the computer to process things that occur naturally to
humans. Today, deep learning techniques are commonly used in computer vision to classify images and videos. As a
result, for challenging computer vision problems, deep learning provides state of the art solutions to it. Coral reefs are
an essential resource of the earth. A new study finds the planet has lost half of its coral reefs since 1950. It is necessary
to restore and prevent damage to coral reefs as they play an important role in maintaining a balance in the marine
ecosystem. This proposed work helps to prevent the corals from bleaching and restore them to a healthy condition by
identifying the root cause of the threats. In the proposed work, using deep learning CNN techniques, the images are
classified into Healthy and Stressed coral reefs. Stressed coral reefs are an intermediate state of coral reef between
healthy and bleached coral reefs. The pre-trained models Resnet50 and Inception V3 are used in this study to classify
the images. Also, a proposed CNN model is built and tested for the same. The results of Inception V3 and Resnet50
are improved to 70% and 55% by tuning the hypermeters such as dropouts and batch normalisation. Similarly, the
proposed model is tuned as required and obtains a maximum of up to 90% accuracy. With large datasets, the optimum
amount of neural networks and tuning it as required brings higher accuracy than other methods.
Keywords: Stressed coral reef; Deep learning; Convolutional neural network; Pre-trained models

1. Introduction fights climate crises, and is home to large biodiver-

sity. Unlike landforms, oceans are one continuous
The ocean is a large resource that supports life, body that connects every corner of our planet. Ocean

*CORRESPONDING AUTHOR:
M. Thamarai, ECE Department, Sri Vasavi Engineering College, Tadepalligudem, Andhra Pradesh, 534102, India; Email: mthamarai2014@
gmail.com
ARTICLE INFO
Received: 27 June 2023 | Revised: 29 August 2023 | Accepted: 30 August 2023 | Published Online: 8 September 2023
DOI: https://doi.org/10.30564/jeis.v5i2.5808
CITATION
Aruna, S.P., Thamarai, M., 2023. Stressed Coral Reef Identification Using Deep Learning CNN Techniques. Journal of Electronic & Information
Systems. 5(2): 1-9. DOI: https://doi.org/10.30564/jeis.v5i2.5808
COPYRIGHT
Copyright © 2023 by the author(s). Published by Bilingual Publishing Group. This is an open access article under the Creative Commons Attribu-
tion-NonCommercial 4.0 International (CC BY-NC 4.0) License. (https://creativecommons.org/licenses/by-nc/4.0/).

1
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
ecosystems are the largest of Earth’s aquatic eco-
systems which support many lives and also benefit
human beings in many ways. Coral ecosystems are
part of marine ecosystems formed up of coral reefs.
These coral reefs provide shelter to millions of marine
species and protect the land coastline from storms
and erosions. It provides rich aquatic life for fishing
and other purposes. Increasing ocean temperature
and rapid global warming are some of the risks of
coral reef bleaching. Human activities such as water
pollution also affect the quality and life of the corals.
So, it is necessary to prevent the coral from damage
detect the early signs of coral damage and restore it.
In this method, we are detecting whether the coral
is stressed or not using a CNN deep learning algo-
rithmThe ocean is a large resource that supports life,
fights climate crises, and is home to large biodiver-
sity. Unlike landforms, oceans are one continuous
body that connects every corner of our planet. Ocean
ecosystems are the largest of Earth’s aquatic eco-
systems which support many lives and also benefit
human beings in many ways. Coral ecosystems are
part of marine ecosystems formed up of coral reefs.
These coral reefs provide shelter to millions of marine
species and protect the land coastline from storms
and erosions. It provides rich aquatic life for fishing
and other purposes. Increasing ocean temperature
and rapid global warming are some of the risks of
coral reef bleaching. Human activities such as water
pollution also affect the quality and life of the corals.
So, it is necessary to prevent the coral from damage
detect the early signs of coral damage and restore it.
In this method, we are detecting whether the coral is
stressed or not using a CNN deep learning algorithm.
1.1 Coral reef biodiversity
Coral reefs are formed of colonies of hundreds to
thousands of tiny individual corals commonly known
as polyps. These marine invertebrate animals have
hard exoskeletons made of calcium carbonate and
are sessile. The large coral reef (Great Barrier Reef)
is found in Queensland, Australia. These corals are
colourless in nature. The microscopic algae named
Zooxanthellae that live in it, adds different colours.
2
Corals and algae have a mutual relationship. The
coral offers the zooxanthellae a safe environment as
well as the substances essential for photosynthesis.
The zooxanthellae create oxygen and aid the coral
in removing trash as payment. Millions of zooxan-
thellae produce pigments that add colours to coral.
Depending on the colour of the coral image, we can
predict using CNN whether the coral is stressed or
not. It is an advantage for us, at this point the dam-
age made is minimal we can prevent it from bleach-
ing.
1.2 Coral bleaching
The corals are classified into the following stages.
Healthy Coral: Healthy coral reefs are bright cor-
als filled with microscopic algae called zooxanthel-
lae that live in their tissues. These algae give bright
colors to coral reefs.
Stressed Coral: Stressed coral reefs are less bright
compared to healthy coral reefs. Due to the temper-
ature rise, it loses its color as algae tend to leave the
coral reefs.
Bleached Coral: Bleached coral reefs are empty
coral reefs with any algae present in them. It is white
because only the coral skeleton is present.
The Healthy Coral, Stressed Coral and Bleached
Coral are shown in Figures 1a. and 1b. and 1c. re-
spectively.
a. Healthy coral b. Stressed coral c. Bleached coral
Figure 1. Coral reefs.
Causes for bleaching
Climate change causes a rise in the ocean temper-
ature that results in coral bleaching. Industrial pol-
lutants and fertilizer wastes that drain into the ocean
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
also change its environment and cause coral bleach-
ing. Direct exposure to sunlight in shallow waters
also causes coral bleaching. Low tides and exposure
to air may also cause bleaching. When water gets
too warm for corals, corals release their colorful mi-
croalgae, turning skeletal white. If bleaching events
are prolonged or happen too frequently with not
enough time to recover in between, significant coral
mortality can occur, sealing the fate of coral reefs.
The Global Coral Reef Monitoring Network
(GCRMN) is a network of the International Coral
Reef Initiative (ICRI) that has published a report
titled The Status of Coral Reefs of the World: 2020.
Its findings illustrate that, between 2009 and 2018,
there was a progressive loss of 14% of the corals
brought on by frequent, massive bleaching events.
More hard coral was destroyed than is currently
present on Australia’s coral reefs, totalling around
11,700 square kilometres. Similar to the decline in
hard coral throughout this time period, the amount of
algae on coral reefs around the world has increased
by roughly 20% since 2010. The report depicts four
decades of falling coral abundance, increased bleach-
ing, and rising algae levels, which are indicators of
deteriorating reef health. The paper also highlights
the remarkable capacity of coral reefs to recover in
the absence of local and global threats. Both conclu-
sions ought to spur immediate action. Coral reefs are
largely invisible, but environmental protection must
prioritize their health. Global Coral Reef Monitoring
Network (GCRMN) is a network that maintains and
operates the international coral reef and provides
information about it [1]. GCRMN divides it into ten
regional nodes. They are Australia, the Caribbean,
ETP, etc, GCRMN collects the database, monitors it
from time and time, and publishes its status. It states
that almost all the coral reefs are extinct due to glob-
al warming and local human activities. Also, some
parts of coral reefs proved to remain resilient and can
recover from damage by taking appropriate meas-
ures. There are more than 600 subspecies in just the
Great Barrier Reef in Australia. They come in a wide
range of sizes, shapes, and colours, making them a
diversified species. Hard corals and soft corals are
3
the two main types of corals. Soft corals are flexible
and frequently mistaken for plants because they lack
the Skelton, which hard corals have. The best coral
reef health indicator is hard coral. The most crucial
factor in quantifying the coral population is the hard
coral cover percentage.
Coral reefs that are dominated by algae lose some
of their architectural complexity and structural integ-
rity. The goal of NOAA’s Coral Health and Monitor-
ing Programme (CHAMP), which was established
to help protect and preserve the health of coral reefs
around the world, is to offer resources and services
to researchers and the general public.
The proposed work comprises the following:
· Pre-trained models such as InceptionV3 and
Resnet50 are fine-tuned by adding batch normaliza-
tion and dropout layers to eliminate overfitting pa-
rameter problems and improve the stressed coral reef
identification accuracy.
· An efficient low-complexity CNN architecture
is proposed for stressed coral reef identification.
· The performance of the pre-trained models and
proposed CNN model are analysed using perfor-
mance metrics such as classification accuracy, F1
score, precision and Recall.
The paper is organized as follows. Section 1 de-
scribes the introduction to coral reefs and the impor-
tance of coral reef identification. Section 2 reviews
deep learning and previous work on coral reef classi-
fication using convolution neural networks. Section
3 describes the proposed work for stressed coral reef
identification using pre-trained models and proposed
low-complexity CNN architecture. Section 4 dis-
cusses the results and Section 5 concludes the paper.
2. Deep learning
Deep learning networks, which are advanced neu-
ral networks with a lower error rate, are crucial for
solving prediction and classification problems. Deep
learning networks, a subtype of artificial intelligence,
are employed for a number of tasks, including image
identification, speech to text conversion, scene de-
scription, drug discovery, face detection and recogni-
tion, weather forecasting, and more [2,3].
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
2.1 Convolutional neural network
A convolutional neural network is a network ar-
chitecture of deep learning algorithms [4]. It has high
performance and accuracy in image classifications
by finding or recognizing required patterns in the
images. It breaks the pixels of the image into smaller
pixels and detects the required data in them. A typi-
cal convolutional neural network has the following
layers and the architecture is shown in Figure 2.
Figure 2. Typical convolution neural network architecture.
Convolutional layer
The convolution layer is the mathematical layer
that processes the image segment by segment and fil-
ters it as given. The input image is divided into three
for an RGB image and one for a grey image. The
filter filters the input image with the provided kernel
size. Using the Dot product, the value of the input
image and filter is multiplied and stored in them.
Pooling layer
The pooling layer is next to the convolution layer.
There are two types of pooling, max pooling, and av-
erage pooling. Max pooling returns the max value in
the feature map. Average pooling returns the average
value of the feature map. For example, the kernel
size is 2 × 2, and its values are 34, 56, 78, and 23. If
max pooling is selected, it returns the value 78 from
the feature map. If the average pooling is selected,
it returns the value 48. Between the convolutional
layer and the fully linked layer, the pooling layer fre-
quently acts as an intermediate.
Fully connected layer
The fully connected layer is the layer that con-
nects one neutral network to the output neutral net-
work. It connects the weights and biases of previous
4
neurons to the weights and biases of the next neu-
rons. The final layers of CNN are used to summarize
the network by flattening and connecting it to the
output layer. The flattening layer reduces the dimen-
sion of the network to one. Here, based on the type
of problem, the output layer is provided. For exam-
ple, if it is classification, then it is Sigmoid.
Activation functions
The activation function is an essential function of
the CNN model. It identifies and transfers the vari-
ables from one neuron to another neuron. It decides
what should be transferred and what should be not.
This network is non-linear. Activation functions like
ReLu, Sigmoid, Tanh, and Softmax are used in it.
ReLu connects the value of the variable from zero to
infinity. If any negative value is present in the ReLu,
it is considered zero. The tanh connects from a neg-
ative one to a positive one. Sigmoid is for binary
classification problems and Softmax is for multiclass
classification problems.
2.2 Deep learning in coral reef classification
This section describes the previous work done in
the coral reef classification using deep learning.
Automated underwater vehicles (AUVs) have
been successfully used to monitor these reefs in
recent years. These computers can be improved to
categorise many coral species, though, by integrating
neural networks [5,6].
Additionally, poor picture quality is a constant
problem, resulting in hazy and blurry images. This is
because the sediments in the water make it difficult
to take high-quality pictures, even though the coral
photos transect uses this technique for image capture
and processing. Then, it generates photos of excel-
lent quality and makes it noticeably simpler to see
the characteristics of corals [7].
Elawadyetal. developed a machine vision algo-
rithm to enable underwater robots to locate coral
reefs and pick them up using CNN [8]. The shape and
texture features are added as supplementary channels
along with basic spatial color channels of coral input
images and used for classification. Raphael et al. de-
veloped an automated DL classification scheme for
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

11 species of corals present in the Eilat Gulf regio [9]. nificantly faster.
Chindapol et al.used advection diffusion equations to
model the effects of flow on coral reef colony growth
and shape [10]. Mahmood et al. used neural networks
and deep learning to distinguish various coral species
and live corals from bleached corals. The author used
VGGNet with a 2-layer MLP classifier for the clas-
sification of corals in their work [11]. Mahmood et al.
discussed the power of deep learning for monitoring
coral reefs in their survey work [12]. Mahmood et al.
combined CNN images and handcrafted features for
coral classification, but this approach is computation- Figure 3. Resnet50 architecture.
ally expensive and not suitable for large datasets [12].
Mahmood et al. used VGGNet for the classification
3.2 Inception V3 architecture
of unlabeled coral mosaics [13].
Gómez-Ríos et al. used three various CNN archi- The important milestone in CNN development is
tectures—Inception V3, ResNet, and Densenet and the inception network which is shown in Figure 4.
data augmentation techniques to obtain high accu- Convolutional neural network model Inception V3
racy in coral texture image classification [14]. Mary has 48 layers and was pre-trained. It is an Inception
et al. used improved local derivative patterns for network variant that has been trained on more than
submarine coral reef image classification [15]. Lumini a million images from the image net collection. It is
et.al.developed ensemble-based different convolu- the third iteration of Google’s Inception CNN model.
tion neural network models for underwater imagery A popular image recognition algorithm called In-
analysis [16]. ception V3 has demonstrated improved accuracy on
the ImageNet dataset.
3. Proposed work
The proposed work pre-trained models have been
trained using the transfer learning concept and test-
ed. The pre-trained CNN models Inception V3 and
Resnet50 are fine-tuned to improve the performance in
stressed coral reef identification. In addition to it, a new
CNN model was also built and studied and compared
its performance with the pre-trained models.

Figure 4. Inception network.

3.1 Resnet50 architecture

The pre-trained model, Resnet50 architecture is

shown in Figure 3. The architecture has 50 layers
and is specially designed to eliminate training errors.
The network has a special block called residual block
which is shown in Figure 3. A residual block also
referred to as a “bottleneck”, uses 11 convolutions
to cut down on the number of parameters and matrix
multiplications. This makes each layer’s training sig-
3.3 Fine-tuning the pre-trained models
For the models, the pre-trained models of Incep-
tionV3 and Resnet50 are inherited using the keras
library. For InceptionV3, the base model is loaded
and then fine-tuned with the required parameters.
Using the GlobalAveragePooling3D() function, the
dimensions of the inputs given are adjusted cause

5
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

channels_last corresponds to inputs with shape separately and created a sequential model using
(batch, height, width, channels) while channels_first keras. Inside the model, we added two convolu-
corresponds to inputs with shape (batch, channels, tion2D layers with the required parameters. RELU
height, width). To extend the base model, GlobalAv- activation is used in each neutral network as shown
eragePooling3D is used and also to avoid dimension in Figure 5. Also, dropouts and Batchmormalisation
conflicts. In the case of InceptionV3, the model is hypermeters and adjusted as required to improve
trained with more accuracy and it is in an overfitting the model efficiency. It is connected to output layers
condition. To avoid this condition, a Dropout(0.1) of with dense = 1 and activation = Sigmoid since it is
weights is added to improve the model efficiency and the binary class classification. The model size is very
avoid overfitting. Dropout is the layer of keras that small when compared to the Resnet50 and Inception
is used to remove the weights that are overtrained V3 pre-trained models. The proposed model is mem-
based on the percentage given. Then, it is connected ory efficient with less number of training parameters.
to the intermediate dense layer with “Relu” activa- The model is compiled and fitted with the validation
tion and connected to the output layer with softmax data as 20% and batch size as 4.
activation. After the changes are made, the model
is trained with train and test datasets. It is compiled
using Adam optimizers and then fitted with the vali-
dation data as 20% and batch size as 32.
Similarly, the works are done for the Resnet50
model. The base model is loaded and then fine-tuned
with the required parameters. Using the GlobalAv-
eragePooling3D() function, the dimensions of the
inputs given are adjusted cause channels last cor-
respond to inputs with shape (batch, height, width,
channels) while channels first correspond to inputs
with shape (batch, channels, height, width). To ex-
tend the base model, GlobalAveragePooling3D is
used and also to avoid dimension conflicts. In the
case of Restnet50, our model is learning too slowly
and also reaches a steady learning rate. Once the
model reaches a steady learning rate, adjusting the
learning rate may not give significant results. So, the
Batchnormalisation function is added to the neutral
network to improve the model efficiency. Like In-
ceptionV3, it is also connected to an intermediate
dense layer with “Relu” activation and connected
to the output layer as Softmax activation. After the
changes are made, the model is trained with train and
test datasets. It is compiled using Adam optimizers
and then fitted with the validation data as 20% and
batch size as 32.

3.4 Proposedlow complexity CNN model

In the proposed CNN model, we trained the data Figure 5. Proposed CNN model.

6
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

4. Results and metrics correctly.

Accuracy Formula = (True Positive +True Neg-
The pre-trained models and proposed CNN mod- ative)/(True Positive + False Positive + False Nega-
el are trained and tested with our dataset collected tive + True Negative)
from internet sources. The dataset contains 120 im-
Table 1. Confusion matrix for the coral reef classification (pro-
ages. The dataset has an equal number of healthy and posed) model.
stressed coral images(healthy coral reef images: 60,
False positive—The image of
stressed coral reef images). All the models are com- True positive—The image
stressed coral reef is identified
of stressed coral reefs are
piled and fitted with the validation data as 20% and identified correctly.
as healthy coral reef. (Type II
error)
different batch sizes.
False Negative—The image of
The sample input images in the two categories True Negative—The image
healthy coral reef is identified
(healthy and stressed coral reef images) are shown of coral reefs are found not
as stressed coral reef. (Type I
stressed.
in Figure 6. The healthy coral images used in the error)

training dataset are shown in Figures 6a and 6b and
stressed coral reefs used in the dataset are shown in
Figures 6c and 6d respectively.
Figure 6. Input coral reef images. (a) and (b) represent the
healthy coral reef images used in the dataset. (c) and (d) repre-
sent the stressed coral reef images used in the dataset.
The confusion matrix of each model is extracted
for the test images. Then, the precision, recall, and
Accuracy gives us the number of correct predic-
tions that the trained model achieves.
Comparing the precision, recall, and accuracy for
the three models, based on the correct predictions of
stressed coral reefs and healthy coral reefs, the val-
ues are calculated and given in Table 2. The dataset
is collected from coral website images in Google.
Totally 120 images are in the dataset. The train-
ing images are 100 (Healthy: 50 images; stressed:
50 images). The sample test images given are 20
images, 10—healthy coral reef images and 10—
stressed coral reef images. The performance of the
proposed CNN in classifying stressed coral is 90%,
which is the highest accuracy when compared to the
fine tuned pre-trained models. The accuracy is 15%
higher than the Inception V3 model and 35% higher
than the Resnet50 model. The proposed model CNN
model predicts the healthy and stressed coral with
the same accuracy (90%).
Table 2. Performance comparison of the proposed CNN model
with pre-trained models.

accuracy of each model are calculated. CNN model Test data Precision Recall
F1
score
The confusion matrix for our model is represent-
Inception V3 Healthy 0.67 0.80 0.73
ed in Table 1. (Pretrained fine tuned
Precision = True Positive/(True Positive + False model using transfer Stressed 0.75 0.60 0.67
Positive) learning concept)

Precision gives us the quality of positive predictions. Resnet50 (Pretrained Healthy 0.56 0.50 0.53
finetuned model using
Recall Formula = True Positive/(True Positive + transfer learning Stressed 0.55 0.60 0.57
False Negative) concept )
Recall, also known as sensitivity gives us the Healthy 0.90 0.90 0.90
Proposed CNN model
proportion of actual positives that were identified Stressed 0.90 0.90 0.90

7
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
The classification accuracy is computed as fol-
lows for sample 20 input test images.
Accuracy of Inception V3 model is(8+6)/
(8+2+4+6)= 0.7
Accuracy of Resnet50 model is (5+6)/(5+5+4+6)=
0.55
Accuracy of proposed CNN model is (9+9)/
(9+1+9+1)= 0.9
5. Conclusions
Action needs to be taken after identifying stressed
corals.
Once the stressed coral reef is identified in a
particular area. Scientists can analyse the reasons
behind its decay and its root causes. If the coral is
stressed, it tends to lose its pigmentation causing the
algae that are living together to start to move away
from it, due to some natural causes or man-made
pollution. Natural causes involve ocean temperature
rise, climate change, etc. Man-made causes involve
water pollution, global warming, etc. Once the sci-
entists analysed and found its root cause, it would
be easy to restore the coral reef to a healthy stressed
state. It would take a lot of effort to restore it from a
bleached state. Which is why, we are identifying it in
the stressed state itself.
Prevention is better than cure. The coral reef
forms a major economic source of ocean resources
for a country or a state. It is necessary to maintain
it in its state of health. Because a healthy coral reef
enhances fish production, healthy aquatic life such
as seaweeds has economic value. That’s why it is
significant to prevent the coral from bleaching. In
our models, we are identifying in our early bleaching
stage (i.e. stressed stage) and helping them to restore
it to a healthy state.
The proposed work describes an efficient low
computational complexity CNN model to predict the
stressed corals and the model classification accuracy
is 90% which is 15% higher than the Inception V3
model accuracy and 35% higher than the Resnet50
model. In the future, the pre-processing techniques
will be applied to overcome the underwater image
blur and colour balancing and then the proposed
8
method will be suitable for real-time applications.
Author Contributions
Author 1: Concept, writing, execution and results.
Author 2: Writingand result analysis.
Conflict of Interest
There is no conflict of interest.
References
[1] Status of Coral Reefs of the World: 2020 [Inter-
net]. Available from: https://gcrmn.net/2020-re-
port/
[2] Chandrasekar, V., Sureshkumar, V., Kumar, T.S.,
et al., 2020. Disease prediction based on micro
array classification using deep learning tech-
niques. Microprocessors and Microsystems. 77,
103189.
DOI: https://doi.org/10.1016/j. micpro.2020.103189
[3] Chen, K.C.J., Ebrahimi, M., Wang, T.Y., et al.,
2020. A NoC-based simulator for design and
evaluation of deep neural networks. Micropro-
cessors and Microsystems. 77, 103145.
DOI: https://doi.org/10.1016/j.micpro.2020.103145
[4] O’Shea, K., Nash, R., 2015. An introduction to
convolutional neural networks. arXiv preprint
arXiv: 1511.08458.
[5] Moniruzzaman, M., Islam, S.M.S., Bennamoun,
M., et al. (editors), 2017. Deep learning on un-
derwater marine object detection: A survey. Ad-
vanced Concepts for Intelligent Vision Systems:
18th International Conference, ACIVS 2017;
2017 Sep 18-21; Antwerp, Belgium. p. 150-160.
[6] Borbon, J., Javier, J., Llamado, J., et al. (editors),
2021. Coral health identification using image
classification and convolutional neural networks.
2021 IEEE 13th International Conference on
Humanoid, Nanotechnology, Information Tech-
nology, Communication and Control, Environ-
ment, and Management (HNICEM); 2021 Nov
28-30; Manila, Philippines. New York: IEEE. p.
1-6.
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
[7] Marks, K., 2013. PVC Paparazzi [Internet].
Living Oceans Foundation [cited 2014 Feb 25].
Available from: https://www.livingoceansfoun-
dation.org/pvc-paparazzi/
[8] Elawady, M., 2015. Sparse coral classification
using deep convolutional neural networks. arXiv
preprint arXiv: 1511.09067.
[9] Raphael, A., Dubinsky, Z., Iluz, D., et al., 2020.
Deep neural network recognition of shallow wa-
ter corals in the Gulf of Eilat (Aqaba). Scientific
Reports. 10(1), 12959.
[10] Chindapol, N., Kaandorp, J.A., Cronemberger,
C., et al., 2013. Modelling growth and form of
the scleractinian coral Pocillopora verrucosa and
the influence of hydrodynamics. PLoS Compu-
tational Biology. 9(1), e1002849.
[11] Mahmood, A., Bennamoun, M., An, S., et al.
(editors), 2016. Coral classification with hybrid
feature representations. 2016 IEEE Interna-
tional Conference on Image Processing (ICIP);
2016 Sep 25-28; Phoenix, AZ, USA. New York:
IEEE. p. 519-523.
9
[12] Mahmood, A., Bennamoun, M., An, S., et al.,
2017. Deep learning for coral classification.
Handbook of neural computation. Academic
Press: Cambridge. pp. 383-401.
[13] Mahmood, A., Bennamoun, M., An, S., et al.
(editors), 2016. Automatic annotation of coral
reefs using deep learning. Oceans 2016 MTS/
IEEE Monterey; 2016 Sep 19-23; Monterey,
CA, USA. New York: IEEE. p. 1-5.
[14] Gómez-Ríos, A., Tabik, S., Luengo, J., et al.,
2019. Towards highly accurate coral texture
images classification using deep convolutional
neural networks and data augmentation. Expert
Systems with Applications. 118, 315-328.
[15] Mary, N.A.B., Dharma, D., 2017. Coral reef im-
age classification employing improved LDP for
feature extraction. Journal of Visual Communi-
cation and Image Representation. 49, 225-242.
[16] Lumini, A., Nanni, L., Maguolo, G., 2020. Deep
learning for plankton and coral classification.
Applied Computing and Informatics. (ahead-of-
print).
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

Journal of Electronic & Information Systems

https://journals.bilpubgroup.com/index.php/jeis

ARTICLE

The Application of Information Systems to Improve Ambulance

Response Times in the UK
Alan Slater

University of Huddersfield, Huddersfield, HD13DH, UK

ABSTRACT
Emergency ambulance services in the UK are tasked with providing pre-hospital patient care and clinical services
with a target response time between call connect to on-scene attendance. In 2017, NHS England introduced four new
response time categories based on patient needs. The most challenging is to be on-scene for a life-threatening situation
within seven minutes of the call being connected when such calls are random in terms of time and place throughout
a large territory. Recent evidence indicates emergency ambulance services regularly fall short of achieving the target
ambulance response times set by the National Health Service (NHS). To achieve these targets, they need to undertake
transformational change and apply statistical, operations research and artificial intelligence techniques in the form of
five separate modules covering demand forecasting, plus locate, allocate, dispatch, monitoring and re-deployment of
resources. These modules should be linked in real-time employing a data warehouse to minimise computational data
and generate accurate, meaningful and timely decisions ensuring patients receive an appropriate and timely response.
A simulation covering a limited geographical area, time and operational data concluded that this form of integration of
the five modules provides accurate and timely data upon which to make decisions that effectively improve ambulance
response times.
Keywords: Ambulance response times; Demand forecasting; Geo-location models; Simulation

1. Introduction accepted the challenge to develop the ability to col-

lect and apply information innovatively. However,
Management in the UK Ambulance Service has the focus has been on rapidly changing technology

*CORRESPONDING AUTHOR:
Alan Slater, University of Huddersfield, Huddersfield, HD13DH, UK; Email: alan@contactslater.co.uk
ARTICLE INFO
Received: 5 August 2023 | Revised: 28 August 2023 | Accepted: 29 August 2023 | Published Online: 21 September 2023
DOI: https://doi.org/10.30564/jeis.v5i2.5881
CITATION
Slater, A., 2023. The Application of Information Systems to Improve Ambulance Response Times in the UK. Journal of Electronic & Information
Systems. 5(2): 10-24. DOI: https://doi.org/10.30564/jeis.v5i2.5881
COPYRIGHT
Copyright © 2023 by the author(s). Published by Bilingual Publishing Group. This is an open access article under the Creative Commons Attribu-
tion-NonCommercial 4.0 International (CC BY-NC 4.0) License. (https://creativecommons.org/licenses/by-nc/4.0/).

10
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
and solving stand-alone issues such as crew schedul-
ing rotas rather than linking opportunities together to
improve overall efficiency and productivity. The cen-
tre of attention for management since the COVID-19
pandemic has been arresting a significant decline
in ambulance response time (also known as ‘clock-
time’). This paper addresses the need to understand
how identifying, upgrading and linking together sev-
eral focused micro-information modules will create
a macro information system that offers a foundation
for improved ambulance response times.
The ambulance response time process is seen by
management as the most significant issue in their
service to critically ill patients because response time
is both challenging and the key performance indica-
tor by which the service is judged by both politicians
and the public. This paper focuses upon an alterna-
tive approach to improving the ambulance dispatch
process by utilising the combination of several re-
al-time information systems which the ambulance
service control directly on a day-to-day basis. There
are, however, other influences on response times that
cannot be controlled by the ambulance service on a
day-to-day basis including vehicle design, vehicle
maintenance, driver training, traffic management and
public awareness. Attention to such issues may be
combined with dispatch information to improve am-
bulance response times.
Attention to the information systems which sup-
port the ambulance response system could be the first
step in the creation of a ‘smart ambulance operation’
which uses a set of rules incorporated in an ‘artificial
intelligence’ system to recommend appropriate ac-
tions to call-handlers and dispatchers.
2. Operating environment
The objective of the ambulance service is to pro-
vide ‘out-of-hospital’ early medical assistance to ‘save
lives’ by causing ‘no further harm’ before and during
transporting a patient to a relevant hospital facility.
Cooke [1] speculated that delays in pre-hospital care
could lead to poorer patient outcomes and patient sat-
isfaction increased when response was rapid.
In England, the public believes there is equality
11
of access to appropriate pre-hospital care from the
ambulance service based upon national ambulance
response time targets—which are the time taken in
minutes from an emergency call being connected
to the ambulance service and the on-scene attend-
ance by appropriate staff. The UK public has been
encouraged by both the NHS management and the
press to judge the ambulance service by its response
to ‘life-threatening’ calls. Indicating that when they
make a call for an ambulance one will be dispatched
immediately (with ‘blue lights and sirens’) implying
that resources are always available to answer every
call and that each call is equally important and taken
in priority of receipt.
In reality, ambulance managers prioritise calls
using various response times according to patient
needs. This means that calls classified as ‘life-threat-
ening’ have a short response time target, whereas
calls classified as ‘urgent’ have a longer response
time target. This practice creates a tiered system of
requirements to respond which allows resources to
be allocated throughout the territory as ‘ready to
respond’ thus maximising the response time target
achievement with limited resources. However, short-
term peaks in demand may lead to a lack of resourc-
es to respond to patient calls. In the event of a short-
age of resources, an outbound crew on less urgent
calls may be diverted to a life-threatening call.
The public may be unaware and do not recognise
that there is a variation in response times although
call handlers normally quote a forecast on-scene
arrival time either when a despatcher has allocated
a resource or alternatively they quoted a forecast
waiting time before a resource will be allocated. The
public is not made aware of the contributory factors
behind any change in response time performance.
Only since COVID-19 has such factors as traffic de-
lays, hand-over delays at Accident and Emergency,
or staff and vehicle availability been cited as causes
for response time delays.
Unfortunately, recent press reporting concentrates
upon failures to attend ‘life-threatening emergencies’
within the 7-minute response time target even though
such situations represent less than 10% of all calls.
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
In reality, emergency ambulance services are tasked
with reaching patients, by category of clinical need,
as defined by the Advanced Medical Priority System
(AMPDS), with suitably qualified staff. The re-
sponse time targets, rather than the patient outcome,
have become the key performance measure by which
the press and public judge the success of ambulance
services. However, Heath and Radcliffe [2,3] have crit-
icised the NHS for concentrating only on ambulance
service response times when the ambulance service
offers a greater range of skills.
The NHS has over recent years reviewed the AM-
PDS and adjusted the provisional patient diagnosis
categories, which are accessed by the call-handlers,
such that the target response time is more appro-
priate to the patient pathway measured in terms of
potential further harm and probability of a successful
outcome. The NHS is currently developing response
time targets for some specific conditions for the full
clinical pathway from initial call to treatment in hos-
pital and discharge to community care. These chang-
es are based on data gathered by the NHS on patient
outcomes from particular clinical pathways. Such
applied information systems are available to the am-
bulance service to target and improve the manage-
ment of ambulance response times.
When the NHS was first formed in 1947 the
ambulance service was tasked with providing
out-of-hospital care by attending on-scene emergen-
cies providing patients with first-aid and transport to
a suitable medical facility. In recent years the ambu-
lance service provision has been expanded into other
areas of NHS social care provision including situa-
tions that arise from mental health, alcohol or drugs
and homelessness. These situations have arisen from
a combination of changes in primary care strategy
and shortages in the provision of social care. Private-
ly, ambulance service managers indicate that many
of their calls now relate to emergencies derived from
a lack of social care where the ambulance service is
the patient’s last port of call.
This increase in workload placed pressure on
the service by expanding demand, increasing on-
scene time and increasing patient hand-over time at
12
medical facilities. In addition, Gething, University
of Wales, Health Board [4] reported a substantial
increase in emergency calls mainly due to an ex-
panding elderly population and the widespread mis-
understanding amongst the general public of the am-
bulance service offering that all calls would obtain
an immediate response.
The nature of a medical emergency indicates that
demand for the ambulance service will be somewhat
random in terms of ‘time’, ‘place’, and the ‘required
response’. The operational complexity of the am-
bulance response time problem is one of the most
complex logistics tasks. The main difficulty is that
in certain life-threatening situations there is an ex-
tremely short time before a patient may suffer further
harm if no first-aid is administered. Figure 1 outlines
a process flow chart and the related urgency of re-
sponses required for certain conditions. With average
demand for the busiest ambulance services at over
one call per minute with demand emphasised by sea-
sonal variations and event escalations the logistics
requirements to achieve the response time targets are
particularly difficult to manage. The challenge for
operations managers is to recognise the urgency of
the need, allocate appropriate resources, and achieve
the target response time 24 hours/7 days a week/52
weeks a year at any location in the territory served
whatever the traffic or weather conditions.
Figure 1. Response time flowchart compared to time to potential
harm to patients.
3. Response time targets
[5] [6]
Price and Wankhade indicated that ambu-
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
lance services could not keep pace with demand and
were managing the ‘response time’ target by dis-
patching more than one resource to life-threatening
calls rather than considering overall patient needs.
Cooke [1] concluded that response time targets had
been the main motivator for service restructuring
although ambulance service management recognised
that targets themselves do not save lives, more sig-
nificantly ambulance availability, staff training and
communications throughout the patient pathway
would improve service and save lives.
Following a study into ambulance response times
by Sheffield University, NHS England [7,8] estab-
lished the Ambulance Response Programme with
new integrated performance level and response time
targets for all emergency calls based upon a combi-
nation of triage time by call handlers and ambulance
response time by suitable crews for four categories
of patient need:
Category 1—defined as ‘life threatening’ cases
where the triage time is the earliest of 30 seconds
from the call being connected, an ambulance being
dispatched, or the patient needing to be identified.
These cases require a 7-minute mean response with
90% of the calls responded to within 15 minutes or
less. In addition, suitable transport must be available
to convey the patient within 15 minutes of the call
being connected.
Category 2—defined as ‘emergency cases’ where
the triage time is the earliest of 240 seconds from
the call being connected, an ambulance being dis-
patched, or the patient needing to be identified.
These cases require an 18-minute mean response
time with 90% of the calls responded to within 40
minutes or less. The ‘clock’ is only stopped by the
arrival of a suitable vehicle to convey the patient, or
if a vehicle is not required the first staff on the scene.
Category 3—defined as ‘urgent cases’ where the
triage time is the earliest of 240 seconds from the
call being connected or an ambulance being dis-
patched. These cases require a 90% response within
120 minutes by suitable transport, but if the patient
does not require transport then the first staff on-scene
stops the ‘clock’.
13
Category 4—defined as ‘non-urgent’ cases where
the triage time is 240 seconds from the call being
connected, an ambulance being dispatched, or the
patient needs to be identified. 90% of these cases
should be responded to within 180 minutes and the
‘clock’ is only stopped by the arrival of a vehicle
suitable to transport the patient, but if the patient
does not require transport then the arrival of the first
staff is on-scene.
Further integrated targets were also defined. For
example, by 2022 90% of eligible heart attack pa-
tients should receive definitive treatment at a special-
ist heart centre within 150 minutes of the call being
connected. These targets imply that ambulances take
patients with certain defined conditions directly to
the relevant hospital facility rather than Accident and
Emergency (A & E).
4. Earlier theoretical solutions
In the past, operations research algorithms have
offered a range of potential solutions addressing
specific topics such as ambulance numbers and lo-
cations. Such studies have included Toregas et al. [9]
who used optimisation to minimise the number of
ambulances used to cover a defined geographical
area. Whereas, Church and ReVelle [10] and Repede
and Bernardo [11] indicated how fixed ambulance
numbers could be located to obtain maximum geo-
graphical coverage. Gendreau et al. [12] and Dorner et
al. [13] combined these two ideas to provide the min-
imum number of ambulances over a maximum geo-
graphical coverage. Daskin [14,15] modelled the impact
of ambulance unavailability and Carter et al. [16]
used queuing theory with fixed locations to tackle a
similar problem. Larson [17] developed a ‘hypercube’
queuing model to select ambulances to respond from
a fixed fleet. Lubicz and Meielczarek [18], Savas [19],
Fitzsimmons [20], Swoveland et al. [21], Erkut et al. [22]
and Inakawa et al. [23] have all used queuing and
simulation techniques to predict ambulance response
times in specific cities or geographical areas. Brot-
corne et al. [24] provide a comprehensive review of
ambulance location models. Similar specific parame-
ters have been modelled by Fitch et al. [25], Blackwell
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
and Kaufman [26] and Shane et al. [27]. These opera-
tions research techniques were employed to tackle
location and historic deployment issues rather than
address the end-to-end tasks of managing a short
lead time 24/7 emergency ambulance service sup-
porting populations in a substantial territory
In addition, early operation research models
were weak on input data and underlying operating
assumptions and failed to provide credible solutions
that represented an improvement upon existing op-
erations. When these models were compared with
real-time ambulance deployment planning there
were several significant weaknesses. Furthermore,
these models failed to consider significant real-time
opportunities such as the re-deployment of returning
ambulances to alternative locations, staggered crew
shift times, diverting ambulances from less non-ur-
gent tasks to life-threatening tasks, or the short-term
use of alternative or third-party ambulance services
for non-urgent cases.
Brotcorne et al. [24] indicated that several com-
ments had been made by both users and reviewers of
these techniques. Firstly, the data sets employed are
historic, time-limited and do not account for a dy-
namic starting position, or peaks and troughs in de-
mand on a seasonal, monthly, weekly, daily or hourly
basis. Secondly, as shown by Carson and Batta [28],
the travel time differences throughout the day are
ignored and the travel time is either calculated on a
straight-line distance at a constant speed or distance
is calculated on the ‘square root law’ devised by
Kolesar [29], leading to significant inaccuracies when
compared with actual ambulance travel times. Third-
ly, each model limited its scope by addressing only
the significant issues of either location or fleet size,
although Naoum-Sawaya and Elhedhli [30] considered
a continuous-time chain to redeploy ambulances to a
location upon completion of a task.
None of the models quoted coped with dynamic
operational practices such as the use of deployment
locations only on specific days or times of day; var-
iable road speeds on different roads and the same
road throughout the day. In addition, variations in
on-site timings at different medical facilities includ-
14
ing handover delays were not considered. Further-
more, they do not consider what has now become
the operational practice in terms of dispatching more
than one ambulance from different locations to a sin-
gle call; the use of hubs (main ambulance stations)
with satellites and despatch points (temporary stand-
by ambulance locations), and the use of crews when
returning from a task or extending crew shift times.
5. Operational issues
The shortfall in the operations research models
indicates that problems facing the ambulance service
are extremely complex and not those which could be
solved by models which combine location selection
with vehicle routing and scheduling because they in-
volve a combination of several real-time operational
issues including:
· Call pick-up delays at the call-centre when busy
· Insufficient or incorrect information to catego-
rise patient needs correctly
· The immediate or subsequent availability of
suitable crews
· Outbound (and inbound) traffic conditions
· Ability to obtain relevant patient information
· Selection or diversion to a suitable medical facility
· Waiting time and handover delays at hospital
facilities
· Make ready and/or crew break before returning
to a standby location
· Knowledge of current resources deployed on
which category of task
· Progress of each deployment and potential crew
reassignment time
· Currently available resources by type and skill level
· Number of patients on-scene at each location
· Data on the changing condition of the patient(s)
· Location of the incident and access to the pa-
tient(s)
· Time of day, traffic conditions, weather conditions
These issues delay the process flow of attending,
on-site treatment and delivery of a patient to a suit-
able medical facility and may risk failing to achieve
the target response time for the complete task. Lord
Carter [31] suggested the reality of ambulance service
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

operations is currently very different from the op- for operational reasons. On-scene staff need to know
timum process due to the combination of increased if they have multiple patients to consider at one site
demand and a need to substantially improve produc- and the nature of any hazard that may impede staff
tivity. In addition to response times the ambulance either traveling to the site or at the site plus whether
service has a wider set of financial and operating key any other emergency service has been called out to
performance measures to report to NHS England assist and is a rendezvous arranged with anyone.
monthly. However, in practice, there is real opera- Despatchers will monitor both the progress and re-
tional concentration on response times because both quirement for those resources where they have des-
the general public and the national press highlight patched more than one resource to enable them to
timing failures regularly. achieve the response time; so that they could stand
Following receipt of an emergency call current down duplicated resources at the first opportunity.
practice is to follow one of three pathways—see In the event of severe life-threatening trauma be-
Figure 1 (where patient pathways are numbered 1, ing recognised by staff they may in addition despatch
2 and 3). Initially, the patient’s need is categorised support in the form of a mobile critical care team con-
using AMPDS then call-centre staff will either de- sisting of a paramedic and/or doctor or the helicopter
ploy a suitable resource or provide telephone advice emergency medical service if suitable and available.
(known as ‘hear and treat’ 1). Some emergency and
urgent calls may be concluded on scene employing 6. Alternative approach
treatment or advice from the crew (known as ‘see
The key issues to facilitate improving ambulance
and treat’ 2). For life-threatening and emergency
response times and staff productivity are the collec-
calls (known as ‘see and convey’ 3). one or more
tion, analysis and availability of relevant timely in-
resources may be despatched immediately (if avail-
formation. To overcome the deficiencies encountered
able) or existing outbound crews servicing urgent or
when using operations research techniques to opti-
less urgent calls may be diverted.
mise specific issues an alternative approach would
To assist with achieving target response times, be the combination of solutions from five real-time
call-centre staff have the electronic mapping of the modules all of which could be integrated using a da-
territory which is combined with the use of full post- ta-warehouse—see Figure 2.
codes, or the compass on a caller’s mobile phone, to
give an accurate location of the patient. They have
vehicle tracking and tracing to locate vehicles and
coding to determine if vehicles are available and the
category of the task they are undertaking. Workforce
planning indicates the remaining shift time available
to staff on duty and the potential availability of crews
about to start a shift. On the local map, they have the
locations of all medical facilities, on-call staff, com-
munity first responders, members of the hazardous
area response team, the helicopter emergency medi-
cal service (HEMS), mountain rescue teams (MRT)
and the location of static public access defibrillators
Figure 2. Integrated in formation system.
and bleed control packs.
Call-centre staff also know about handover de- In this instance, a module is represented by one
lays at each medical facility, vehicles out of service or more algorithms that obtain real-time or historic
and crews on shift potentially taken out of service data from a directly linked data-warehouse to either

15
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
answer a specific question or add further processed
data to the data warehouse.
6.1 Demand forecasting from historic data
Traditionally, the ambulance service has collected a
large quantity of real-time data reflecting demand pa-
rameters including location of demand, timing of calls,
classification of requirement, age of patients, operation-
al conditions and short-term outcome for patients. This
data indicates that calls are in essence random both
in location and nature but analysis over an extended
period of 3 years shows similar macro patterns if one-
off factors such as mass-casualty incidents, pandemics,
events or severe weather are discounted.
By combining ambulance service data with data
from public services such as postcodes [32] and popu-
lation data (see Office of National Statistics 2022) it
is possible to determine some basic demand parame-
ters. Pidd et al. [33] showed that mapping volume data
by location and time using a geographic information
system (GIS) would highlight demand ‘hot spots’.
Such analysis of both ambulance service and GIS
data shows demand by postcode sector (defining
area), showing potential forecasting variations, and
defining demand by category of call, time of day and
age of the population—see Table 1.
The profile may differ in each selected territory,
however, data will be more accurate for Functional
Urban Areas (FUA) which are densely inhabited
areas in cities (urban and sub-urban) and slightly
less populated commuter zones (semi-rural and ru-
ral). With data, collected over 3 years and weighted
to the current situation by employing exponential
smoothing, it is possible to plot forecast demand at
postcode sector level and time of day. Best forecasts
are obtained for categories 1 and 2 calls for a limited
period (say 12 weeks) which account for seasonality
and key conditions such as holiday peaks in demand.
Using such demand data by day of the week and
considering weather details, it is possible to forecast
ambulance requirements to potential ‘hot spots’ by
the time of day which allow staff to locate unallocat-
ed resources throughout the territory to areas with a
high probability of demand and therefore maximise
16
the opportunity to achieve the target response times.
6.2 Resource allocation
Previously resources were located at local ambu-
lance stations and hospitals but with the development
of direct-to-vehicle communications plus vehicle
tracking and tracing the control centre has real-time
knowledge of each crew’s location (even while they
are traveling). The current thinking is to allocate re-
sources to ambulance stations and temporary deploy-
ment points in FUAs. Deployment points are parking
places for a single front line double manned ambu-
lance awaiting a job, often located at fire or police
stations, shopping centres or garages where there are
suitable ‘comfort’ facilities for the crew.
However, based upon significantly improved
analysis of demand data it is now more appropri-
ate to allocate resources based on forecast demand.
‘Hot spots’ could be geo-fenced providing a forecast
of potentially successful travel time to a call based
upon travel time from the centre point of a hot-spot.
However, the target distance to potentially travel
in a defined time from the centre of a ‘hot-spot’ is
known as a ‘geo-fence’ which will vary by factors
such as the resource available, the road layout and
the forecast demand. by season, day of the week and
hour of the day—see Figure 3. Ambulance stations
and deployment points could be provided in areas of
potential ‘hot spots’. Significant gaps, not covered
by ‘geo-fences’ could be filled by activating back-
up resources including shift overlaps, on-call staff,
Volunteer Ambulance Crews (VAC) or Community
First Responders (CFRs). Once a crew is despatched
it may be necessary to backfill the location with
another crew at the earliest opportunity. This form
of resource allocation improves the probability of
achieving ambulance response times for category
one and two calls.
Recognising that ‘hot spots’ alter throughout the
day implies relocating operating and backup resourc-
es throughout the day as both demand patterns and
travel times vary. Relocation is best undertaken at
the beginning of a shift, as resources become free
from a completed task or at the end of a crew break.
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

Table 1. Demand parameters: Data sheet.

GEOGRAPHIC AND POPULATION CHARACTERISTICS

Parameters Post Code Split Area Population Daytime Population Night
% % % %
URBAN Below 0.5 square miles per post code 40 2 30 25
Between 0.5 and 2.0 square miles per post
SUB-URBAN 30 8 36 40
code
Between 2.0 and 10.0 square miles per
SEMI-RURAL 20 20 26 25
post code
RURAL Over 10 square miles per post code 10 70 8 10
100 100 100
(Data averaged and rounded up)

FORECASTABLE VARIATIONS IN DEMAND

Season: Spring - Summer - Autumn - Winter
Day of the Week: Mon - Tues - Wed - Thu - Fri - Sat - Sun - Bank Holiday
Location Issues: Tourist Holiday Season - School Holidays
Special Events: Sporting Fixtures - County Shows - University Freshers Week
Zone: Travel Time Dependant by Time of Day
Temperature: Ice - Black Ice - Below Average - Above Average - Hot - Very Hot
Weather: Snow - Sleet - Rain - Heavy Rain - Dry

FORECASTABLE LOCATION OF DEMAND

Urban - Sub-Urban - Semi-Rural - Rural
Population Demographics including Income Levels
Industry/Work Related - Ethnic Origin/Background Related - Housing/Social Related
Availability of Health and Social Services

CALL VOLUME BY TYPE AND TIME OF DAY CALL VOLUME BY POPULATION AGE
Life Threatening Demand by
Time of Day (hours) Emergency Calls Age Group Total Population Split
Emergencies Age Group
% % % %
00.00 to 02.00 7 9 Under 10 12 9
02.00 to 04.00 5 6 11 to 20 13 5
04.00 to 06.00 2 3 21 to 30 12 7
06.00 to 08.00 7 6 31 to 40 15 5
08.00 to 10.00 9 7 41 to 50 14 8
10.00 to 12.00 11 9 51 to 60 13 10
12.00 to 14.00 11 10 61 to 70 10 14
14.00 to 16.00 10 10 71 to 80 7 18
16.00 to 18.00 10 10 81 to 90 3 16
18.00 to 20.00 10 11 Over 90 1 8
20.00 to 22.00 9 10 100 100
22.00 to 24.00 9 9
100 100

17
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
CALL VOLUME BY TYPE AND TIME OF DAY
Life Threatening
Time of Day (hours) Emergency Calls
Emergencies
UN-FORECASTABLE VARIATIONS IN DEMAND
Severe Weather - Pandemic
Mass Casualty Incident - Short Term Localised Issues
Local Events
Requirement for Mutual Aid
Figure 3. Critical variables determining ambulance response
time.
6.3 Resource deployment
Traditionally the ambulance service has deployed
the nearest available resource to a task. They have
even redirected crews on-route to category 3 and 4
calls to a category 1 or 2 call if they were the near-
est. However, if dispatchers adopt real-time vehicle
tracking and tracing combined with geo-fencing
methods they will be able to determine all potential
options to attend the scene. Eglese et al. [34] indicated
that the travel distance from any one point and each
geo-fence structure is determined by the nature of
the road speed and each vehicle geo-fence will be a
different shape offering different ground coverage—
see Figure 3. Eglese et al. [34] also showed that road
speed differs between nodal points (road junctions
on all except minor roads and tracks) on a specific
road and that a ‘road timetable’ may be developed
showing the estimated speed between each nodal
point for each vehicle type with variations for the
time of day, day of the week and season of the year.
It is also practical to impose speed reductions for
specific short-term events which are known to reduce
18
Table 1 continued
CALL VOLUME BY POPULATION AGE
Demand by
Age Group Total Population Split
Age Group
vehicle travel speeds such as long-term road works.
With the use of vehicle tracking, geo-fencing
based upon a relevant ‘road timetable’, it is possible
to calculate the realistic area that may be covered in
7 minutes (for category 1 calls) or 18 minutes (for
category 2 calls) from the vehicle’s current location
(whether static or mobile). Such an analysis of alter-
native vehicles and alternative routes may show the
despatcher that a vehicle with the shortest travel dis-
tance could be slower than a vehicle with the short-
est travel time.
It should be noted that urban and rural clusters
will look different and there will be available re-
sources traveling through each demand cluster when
returning to a standby location and when outbound
to a task—see Figures 4 and 5. The ambulance ser-
vice could also use the combination of critical path
programs, electronic mapping and vehicle tracking
and tracing software to communicate in real-time
with a caller who has a smart mobile the status and
location of any outbound resource responding to
their call—thus establishing a realistic expectation
of an on-scene time and avoiding repeat requests for
immediate ambulance services.
Figure 4. Conceptual view of demand and resource zones (Ur-
ban).
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
Figure 5. Conceptual view of demand and resource zones
(Semi-rural).
6.4 Task review
One of the most significant benefits to a patient
is the backup service provided by call-centre staff
to both on-scene public before a crew arrives and
the ambulance crew as they undertake their obser-
vations, conclude what actions are necessary, which
pathway the patient should take and which medical
facility the patient should be taken.
On-scene staff may also request assistance from
senior clinicians within the call-centre or external
supporting resources (for example Helicopter Emer-
gency Medical Service—HEMS). In addition, staff
may request the public on-scene to assist with imme-
diate first-aid before an ambulance response arrives.
This may take the form of advice by telephone, ad-
vice upon where to collect the nearest static defibril-
lator/bleed pack and how to use these resources.
Staff should be aware of all resources currently
tasked, their location and when they are likely to
become available (upon completion of a task or after
any break) or when their shift finishes and whether
it could be extended if necessary. Constant monitor-
ing may indicate potential issues, for example, long
ambulance handover times at a particular accident
and emergency (A & E) department which may re-
quire further action to divert inbound ambulances to
alternative accident and emergency departments or a
local urgent care centre if practical.
19
6.5 Operating environment
To be able to improve the accuracy of vehicle
running time and on-scene crew operating time any
standard times for each activity must be amended to
reflect the actual operating conditions at the time of
the call. Such variations are achieved by monitoring,
recording and updating specific operating tasks; for
example:
The ‘road timetable’ is a database that forecasts
the speed of a vehicle (operating with blue lights and
sirens) between two nodal points at hourly intervals,
by day of the week, under specific weather and traf-
fic conditions during a specific season—an outbound
travel forecast being the sum of the times between
each nodal point on the route. The ‘road timetable’
is populated and updated from data gathered using
the vehicle tracking and tracing system averaged to
account for differed driver behaviours.
The ‘on scene operating time’ may be estimated
for each AMPDS code, for each type and age of the
patient, whether indoor or outdoor, at what time of
day and in which geographic zone—collection of ac-
tual data over time could generate a ‘look-up’ table.
Real-time monitoring against estimated will provide
a clue as to whether the crew needs telephone or on-
scene assistance.
Direct access (on a portable tablet) to patient re-
cords also provides the crew with a patient history,
medication history and particularly which ‘primary’
and ‘secondary’ medical facilities they have visited.
This may influence both questions asked during the
crew’s initial observations and the decision upon an
onward pathway to a medical facility. Previous pa-
tient calls to the ambulance service should be availa-
ble on a database for crews to access while traveling
outbound.
6.6 Data warehouse
A ‘data warehouse’ could be generated and updat-
ed from relevant data from each module 1 to 5 which
allows relevant staff access 24/7/365. The data ware-
house could be used to identify trends, recognise
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
resource deficiencies and highlight both existing and
potential operating issues. Data could also represent
the basis for comparisons between operations in dif-
ferent geographical zones supporting any necessary
operational changes.
7. Simulation
To test the feasibility and practicality of devel-
oping and using the five modules a simulation was
undertaken based on 24/7 operation, in a limited
but representative geographical area, over the most
difficult season of the year. A honeycomb pattern
of adjusted hexagons was created based on a daily
demand forecast at the response category level, cor-
rected by exponential smoothing of historic demand
patterns and broken down into six-hour periods. The
honeycomb pattern covered an area with urban, sub-
urban, semi-rural and rural populations. ‘Hub’ am-
bulance station locations were selected in functional
urban areas, and ‘deployment points’ were selected
in suitable facilities where there was up to 20 hours
demand for 6 days each week. ‘Spokes’ (parking lo-
cations for Double Manned Front Line Ambulances
DMFLA) were selected, throughout the rest of the
territory, at other suitable facilities to cover forecast
demand which was limited to specific periods in the
day and days of the week.
Before the simulation was attempted a small data
warehouse was established for the geographical area
selected comprising three years of demand data,
resources potentially available (based upon use),
a deployment of resources at the starting position,
data on local medical facilities, a high on-scene time
by AMPDS code and a road time table (adjusted
for ‘blue-light’ vehicles). Several basic decision
rules were adopted, for example, road traffic acci-
dents with two casualties would warrant two double
manned front line vehicles (DMFLV). Additional de-
mand was covered where known, for example, when
the fire service dispatched three or more tenders to
an incident one DMFLV would be dispatched to sup-
port the fire crews. No mass casualty incidents were
included.
The simulation was undertaken based upon a ge-
20
ographically and time-limited data set broken into
three years ‘historic’ data to produce a three-month
statistical forecast of demand. Several operating rules
were tested which achieved the required ambulance
response times for the daily demand. These calcula-
tions were targeted to determine the maximum and
minimum number and type of resources required at
each hub, deployment point and spoke in the defined
area. The simulation was used to determine how re-
sources could be relocated throughout a shift pattern
to meet forecast demand from the hub or deployment
point to spokes and vice-versa to meet forecast de-
mand within and between each hexagon.
The simulation was based on the defragmentation
of the problem and split into four separate but inter-
connected principles:
Firstly, the forecast of short-term demand (ex-
pressed as potential calls by patient category and
time of day) to determine the potential number of
calls in each hexagon for every six hours in a roll-
ing 24 hours. Calls were allocated an AMPDS code,
day of the week and time of day based on weighted
historic data. How many of these calls would fall
into the ‘hear and treat’ and ‘see and treat’ groupings
were estimated based on AMPDS coding. The AM-
PDS code would be translated into a category of pa-
tient need. A reduction factor was applied based on
an estimate of how many would be duplicates or re-
peat calls for the same incident. Frequent caller data
was identified but maintained in the dataset at cate-
gory three because a response would be required. A
separate forecast was calculated for calls transferred
from the 111 service all of which were allocated to
category two demand.
Secondly, based upon demand the selection of
‘hubs’, ‘deployment points’, ‘spokes’ and resources
of all types would be initially allocated throughout
the area. This would include a forecast of operational
tasks by patient category in progress by location and
percentage completion, so resources were limited at
the start through a carry-over situation.
Thirdly, appropriate resources would be allocated
to each call as it was forecast to occur (location, pa-
tient category by AMPDS code and time of day)—
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
the rules would be based upon achieving the target
response time while maintaining maximum avail-
ability throughout the area. Resources that became
available would be redeployed and their transit mon-
itored.
Fourthly, all tasks would be monitored using a
scheduling system with estimated task times and
vehicle routing techniques to obtain operational data
which would be used to determine when a task is
likely to be completed and the crew available for the
next task.
The simulation covered three months in the win-
ter but excluded support for known local events,
pandemics and any mass casualty occurrences. Each
simulation was reviewed and run again with alterna-
tive operational rules.
The best results from several simulations indi-
cated that 96% of the required ambulance response
times would be achievable with 6% fewer resources
than currently allocated in the area. However, this
could only be achieved if and when resources be-
came available they were immediately either allocat-
ed a break period or redeployed to areas of forecast
demand. It was particularly important to ensure that
‘hubs’ would always have resources available or
inbound following the completion of a task. The six
most significant results include:
More than 15% of calls in FUAs were satisfied by
crews being redeployed before reaching less urgent
tasks or available in transit to a hub, deployment
point or spoke.
Achievement for categories 1 and 2 patients was
above the target emergency ambulance response
time in FUAs but slightly below the target in other
areas except where patients were initially attended
by Community First Responders (CFRs) or on-call
staff.
At periods of unusual peak demand times it
would be necessary to convey category 4 patients
using third-party ambulance services.
Over 20% of category 1 and 2 patients which
achieved the target response time at peak times were
responded to by crews from deployment points.
Less than 3% of category 1 and 2 calls required
21
two or more crews
Spacing shift time starts, varying shift length and
having a split shift for ‘hub’ based staff minimised
the requirement to extend crew shifts to complete an
allocated task.
The sensitivity of the simulation results showed
that more than an 8% change in demand levels influ-
enced whether the response time was achieved for
both category 1 and category 2 even if such meas-
ures as extending crew shift time were employed.
After undertaking various sensitivity analysis
runs, as stress tests based on the best simulation re-
sult several other significant conclusions arose, the
six most significant were:
If hand-over times for category 2 and 3 patients at
accident and emergency facilities doubled then sig-
nificant local gaps appeared in resource availability
at both hubs and deployment points leading to a need
to convey category 4 patients employing third-party
ambulance services.
The best results were obtained by having a para-
medic as part of every crew because the paramedic
would after observation treat certain AMPDS codes
as ‘see and treat’.
Utilising ‘retained staff’ in Rapid Response Vehi-
cles (RRVs) from rural Primary Care surgeries and
Volunteer Ambulance Crews (VAC) from rural fire,
police or coastguard stations significantly improved
the response time for category 1 and 2 patients in
both semi-rural and rural areas.
Overall response time targets improved if crews
were allocated category 3 and 4 patients at the end
of their shift and shift starters were deployed to ‘de-
ployment points’ and ‘spokes’ where they would in-
itially be tasked with responding to category 1 and 2
patients.
Redirecting patients to either specialist medical
facilities or alternative accident and emergency units
(rather than the nearest one if hand-over times there
exceeded the target) had little impact on the rede-
ployment of crews or overall achievement of the tar-
get ambulance response times except where a patient
could be grouped as ‘hear and treat’ and directed by
call-centre staff to attend alternative medical facili-
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
ties or a pharmacy.
Following Hamet and Tremblay [35] the appli-
cation of operational rules targeted at testing an
Artificial Intelligence (AI) response towards testing
call-centre staff efficiency by limiting resource op-
tions to those which could either achieve the target
response time or were at the nearest ‘hub’. These
limited computing calculation time and offered the
dispatcher a choice of the best results.
8. Conclusions
It has been widely understood that ambulance
services in England have through their Computer
Aided Dispatch (CAD), AMPDS and patient record
systems collected a large amount of patient and
operations data. However, updating, validating and
processing this information in real-time has not been
recognised as the backbone to achieving national
emergency ambulance response times.
To address this problem the ambulance service
could defragment the problem and using several cur-
rently available information technologies consider
five key decisions in real time.
Where are the demand clusters for life-threaten-
ing and emergency calls?
How many of each type of resource are required
and where to locate these resources?
How many, of which resources and when to allo-
cate resources to each call?
To which facility (if further treatment is required)
the patient should be conveyed?
Which location an ambulance should be deployed
to upon completion of the allocated task?
It has been recognised that these questions may
be answered by separate modules in the form of:
GIS analysis and forecasting modules to deter-
mine demand ‘hot spots’ and demand clusters by
location and time.
A resource coverage and availability module
matched to statistical forecasting of demand over a
short-term time, based upon hexagons each with a
radius set by travel distance over the response time
requirements.
A recommended call resource allocation module
22
based upon interpreting both a set of basic rules and
past dispatch behaviour to maintain maximum avail-
ability of resources covering the territory.
A task scheduling module to provide the basis
for a resource allocation and redeployment module
based upon matching forecast short-term demand
with shift time remaining for each operating crew
and alternative resources that may be available.
A queuing theory module that monitors availabil-
ity at each secondary medical facility (also defining
specialisations) servicing the territory to deliver the
patient to the most appropriate facility, minimise pa-
tient queuing and maximise crew turn-round.
Simulation and sensitivity analysis have proved
that linking the technologies in these modules to-
gether, by utilising a data warehouse in real-time,
provides an opportunity to understand short-term
demand and be able to resource enough calls within
the national emergency ambulance service target re-
sponse times.
These developments represent the first step to-
wards ‘smart ambulance operations’ by establishing
the groundwork upon which information systems
may be used in the decision-making process Subse-
quently, artificial intelligence will use a combination
of activities to locate ‘stand-by’ crews, dispatch
crews or alternative immediate assistance, select a
receiving hospital, sending on-scene real-time inter-
net video recordings to medical teams and to com-
municate patient observations directly from equip-
ment in the ambulance to the hospital based medical
team. Such activities are targeted to improve the
medical assistance available throughout the patients’
pathway and thereby improve the patient experience
and outcome.
Potential progress in the application of these
techniques nationwide may be limited by the ability
to allocate sufficient financial support and the ability
to attract staff with relevant information technology
skills.
Author Contributions
This article has been researched and written by a
single author.
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
Conflict of Interest
There is no conflict of interest.
References
[1] Cook, M., 2011. An introduction to the new am-
bulance clinical quality indicators. Ambulance
Today. 7(5), 35-36.
[2] Heath, G., Radcliffe, J., 2007. Performance
measurement and the English ambulance ser-
vice. Public Money and Management. 27(3),
223-228.
[3] Heath, G., Radcliffe, J., 2010. Exploring the
utility of current performance measures for
changing roles and practices of ambulance para-
medics. Public Money & Management. 30(3),
151-158.
[4] Gething,V (2015) .Written Statement – Clinical
Review of Ambulance Response Time Tar-
gets’ Cabinet Statement, Welsh Government,
Health Board. https://www.gov.wales/writ-
ten-statement-clinical-review-ambulance-re-
sponse-time-targets
[5] Price, L., 2006. Treating the clock and not the
patient: Ambulance response times and risk.
BMJ Quality & Safety. 15(2), 127-130.
[6] Wankhade, P., 2011. Performance measurement
and the UK emergency ambulance service:
Unintended consequences of the ambulance
response time targets. International Journal of
Public Sector Management. 24(5), 384-402.
[7] Turner J (2017) ‘How we remodelled Ambu-
lance Services in England’ Research Features,
University of Sheffield, England. https://
www.sheffield.ac.uk/research/features/remod-
elled-ambulance-service
[8] On the Day Briefing: Ambulance Response Time
Programme [Internet]. NHS Providers. Available
from: https://nhsproviders.org/resources/brief-
ings/on-the-day-briefing-ambulance-response-
programme#
[9] Toregas, C., Swain, R., ReVelle, C., et al., 1971.
The location of emergency service facilities.
Operations Research. 19(6), 1363-1373.
23
[10] Church, R., ReVelle, C., 1974. The maximal
covering location problem. Regional Science.
32(1), 101-118.
[11] Repede, J.F., Bernardo, J.J., 1994. Developing
and validating a decision support system for
locating emergency medical vehicles in Louis-
ville, Kentucky. European Journal of Operation-
al Research. 75(3), 567-581.
[12] Gendreau, M., Laporte, G., Semet, F., 1997.
Solving an ambulance location model by tabu
search. Location Science. 5(2), 75-88.
[13] Doerner, K.F., Gutjahr, W.J., Hartl, R.F., et al.,
2005. Heuristic solution of an extended dou-
ble-coverage ambulance location problem for
Austria. Central European Journal of Operations
Research. 13(4), 325-340.
[14] Daskin, M.S., 1983. A maximum expected cov-
ering location model: Formulation, properties
and heuristic solution. Transportation Science.
17(1), 48-70.
[15] Daskin, M., 1987. Location, Dispatching and
Routing Model for Emergency Services with
Stochastic Travel Times [Internet]. Avail-
able from: https://www.semanticscholar.
o rg / p ap er/ L O C AT ION %2 C -D IS PAT C H -
ING-AND-ROUTING-MOD-
ELS-FOR-WITH-Daskin/5bc91cb1642f023f-
c1958eaf31c228b291f4cde4
[16] Carter, G.M., Chaiken, J.M., Ignall, E., 1972.
Response areas for two emergency units. Opera-
tions Research. 20(3), 571-594.
[17] Larson, R.C., 1974. A hypercube queuing model
for facility location and redistricting in urban
emergency services. Computers & Operations
Research. 1(1), 67-95.
[18] Lubicz, M., Mielczarek, B., 1987. Simulation
modelling of emergency medical services. Eu-
ropean Journal of Operational Research. 29(2),
178-185.
[19] Savas, E.S., 1969. Simulation and cost-effec-
tiveness analysis of New York’s emergency am-
bulance service. Management Science. 15(12),
B-608.
[20] Fitzsimmons, J.A., 1973. A methodology for
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
emergency ambulance deployment. Manage-
ment Science. 19(6), 627-636.
[21] Swoveland, C., Uyeno, D., Vertinsky, I., et al.,
1973. Ambulance location: A probabilistic enu-
meration approach. Management Science. 20,
686-698.
[22] Erkut, E., Ingolfsson, A., Budge, S., 2007.
Maximum Availability Models for Selecting
Ambulance Station and Vehicle Locations:
A Critique [Internet]. Available from: https://
citeseerx.ist.psu.edu/document?repid=rep1&-
type=pdf&doi=965f00e8ec5caa3d08ab916b6c-
71528f5908a311
[23] Inakawa, K., Furuta, T., Suzuki, A. (editors),
2010. Effect of ambulance station locations
and number of ambulances to the quality of
the emergency service. The Ninth Internation-
al Symposium on Operations Research and Its
Applications (ISORA’10); 2010 Aug 19-23;
Chengdu, China. p. 340-347.
[24] Brotcorne, L., Laporte, G., Semet, F., 2003.
Ambulance location and relocation models. Eu-
ropean Journal of Operational Research. 147(3),
451-463.
[25] Fitch, J., 2005. Response times: Myths, mea-
surement and management. Journal of Emergen-
cy Medical Services. 30, 46-56.
[26] Blackwell, T.H., Kaufman, J.S., 2002. Response
time effectiveness: Comparison of response time
and survival in an urban emergency medical ser-
vices system. Academic Emergency Medicine.
9(4), 288-295.
[27] Henderson, S.G., Mason, A.J., 2004. Ambulance
service planning: Simulation and data visuali-
24
sation. Operations research and health care: A
handbook of methods and applications. Spring-
er: Berlin. pp. 77-102.
[28] Carson, Y.M., Batta, R., 1990. Locating an am-
bulance on the Amherst campus of the State
University of New York at Buffalo. Interfaces.
20(5), 43-49.
[29] Kolesar, P., 1975. A model for predicting av-
erage fire engine travel times. Operations Re-
search. 23(4), 603-613.
[30] Naoum-Sawaya, J., Elhedhli, S., 2013. A sto-
chastic optimization model for real-time ambu-
lance redeployment. Computers & Operations
Research. 40(8), 1972-1978.
[31] Carter, L., 2018. Operational Productivity and
Performance in English Ambulance Trusts: Un-
wanted Variations [Internet]. Available from:
https://www.england.nhs.uk/wp-content/up-
loads/2019/09/Operational_productivity_and_
performance_NHS_Ambulance_Trusts_final.
pdf
[32] Royal Mail (2015) ‘History of UK Postcodes’
Royal Mail Group, London. https://www.pow-
eredbypaf.com/the-history-of-uk-postcodes/
[33] Pidd, M., De Silva, F.N., Eglese, R.W., 1996. A
simulation model for emergency evacuation. Eu-
ropean Journal of Operational Research. 90(3),
413-419.
[34] Eglese, R., Maden, W., Slater, A., 2006. A road
timetableTM to aid vehicle routing and schedul-
ing. Computers & Operations Research. 33(12),
3508-3519.
[35] Hamet, P., Tremblay, J., 2017. Artificial intelli-
gence in medicine. Metabolism. 69, S36-S40.
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

Journal of Electronic & Information Systems

https://journals.bilpubgroup.com/index.php/jeis

ARTICLE

Practical Considerations for Implementing Adaptive Acoustic Noise

Cancellation in Commercial Earbuds
Agustinus Oey

VizWave Inc., Seongnam, 13595, Korea

ABSTRACT
Active noise cancellation has become a prominent feature in contemporary in-ear personal audio devices.
However, due to constraints related to component arrangement, power consumption, and manufacturing costs, most
commercial products utilize fixed-type controller systems as the basis for their active noise control algorithms. These
systems offer robust performance and a straightforward structure, which is achievable with cost-effective digital signal
processors. Nonetheless, a major drawback of fixed-type controllers is their inability to adapt to changes in acoustic
transfer paths, such as variations in earpiece fitting conditions. Therefore, adaptive-type active noise control systems
that employ adaptive digital filters are considered as the alternative. To address the increasing system complexity,
design concepts and implementation strategies are discussed with respect to actual hardware limitations. To illustrate
these considerations, a case study showcasing the implementation of a filtered-x least mean square-based active noise
control algorithm is presented. A commercial evaluation board accommodating a low-cost, fixed-point digital signal
processor is used to simplify operation and provide programming access. The earbuds are obtained from a commercial
product designed for noise cancellation. This study underscores the importance of addressing hardware constraints
when implementing adaptive active noise cancellation, providing valuable insights for real-world applications.
Keywords: Active noise cancellation; Adaptive filter; DSP implementation

1. Introduction to reduce ambient noise in the earbuds has achieved

The use of the active noise control (ANC) method significant commercial success, demonstrating the

*CORRESPONDING AUTHOR:
Agustinus Oey, VizWave Inc., Seongnam, 13595, Korea; Email: oeyaugust@vizwave.net
ARTICLE INFO
Received: 7 October 2023 | Revised: 26 October 2023 | Accepted: 30 October 2023 | Published Online: 6 November 2023
DOI: https://doi.org/10.30564/jeis.v5i2.5998
CITATION
Oey, A., 2023. Practical Considerations for Implementing Adaptive Acoustic Noise Cancellation in Commercial Earbuds. Journal of Electronic &
Information Systems. 5(2): 25-34. DOI: https://doi.org/10.30564/jeis.v5i2.5998
COPYRIGHT
Copyright © 2023 by the author(s). Published by Bilingual Publishing Group. This is an open access article under the Creative Commons Attribu-
tion-NonCommercial 4.0 International (CC BY-NC 4.0) License. (https://creativecommons.org/licenses/by-nc/4.0/).

25
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
effectiveness of the method in the frequency range
where the spatial control region is small in compar-
ison to the wavelength. The problem itself can be
simply considered as a one-dimensional problem—
that is, the cancellation of plane waves. However,
incorporating ANC into earbuds presents specific
challenges, primarily due to the compact form factor
essential for earbud design. These challenges include
fitting all the required components into a tiny en-
closure and dealing with tight spacing between the
sensor and actuator, which limits the response time
of an ANC system. This necessitates low-latency
audio processing devices and computationally effi-
cient control algorithms. Achieving significant noise
reduction across a broad bandwidth depends on min-
imizing latency [1].
Another important factor to be considered for
a well-performing ANC system is the variation in
acoustic leakage within the acoustic control region.
This variability is mainly associated with user pref-
erences for diverse earbud fit conditions to ensure
comfortable wear. On some occasions, the fit may
also loosen as the user moves. Obviously, a strong-
er anti-noise signal is required to compensate for
the increase in acoustic leakage. However, even a
minor alteration in the leakage causes variability in
the way ambient noise propagates into the ear. This
introduces uncertainty in how a controller would
respond. Consequently, controller stability and noise
reduction performance must be assured within the
assumed perturbed range.
One of the earliest approaches proposed to ad-
dress uncertainty in active headsets is the utilization
of a robust controller [2]. This method employs an
algorithm based on the two Riccati equations to de-
termine appropriate parameters for a controller with
guaranteed operating margins. The resulting control-
ler is implemented using operational amplifier cir-
cuitry. Another noteworthy alternative approach in-
volves employing a set of stable feedback controllers
with various preset gains [3]. In response to specific
fitness conditions, a comparator switches to the ap-
propriate operational controller. These studies shed
light on the idea that a certain degree of controller
26
adaptiveness is generally necessary to accommodate
varying acoustic environments.
In the field of ANC applications, adaptive con-
trollers incorporating digital filters have gained
popularity for their ability to efficiently adapt to dy-
namic and complex acoustic environments. Howev-
er, practical implementation within earbuds presents
challenges due to the limited choice of computing
hardware, which compromises the system’s ability to
perform complex calculations. High-performing pro-
cessor is essential for executing sophisticated control
algorithms requiring intensive and intricate digital
filtering tasks [4].
Achieving effective ANC in resource-constrained
earbuds demands a careful balance of design factors
to ensure satisfactory noise cancellation performance.
This article investigates practical considerations for
implementing adaptive filters in noise cancellation,
specifically utilizing a low-cost digital signal proces-
sor (DSP) suitable for earbud integration. The adap-
tive controller employs the well-known filtered-x
least mean square algorithm (FxLMS). Details about
the specifications of the speakers and microphone, as
well as their spatial arrangement, are derived from
a popular commercial earbud product. The overall
configurations are determined based on computa-
tional resources and the functional features available
within the chosen DSP. Finally, the article presents
experimentation using broadband random noise to
validate the proposed approach.
2. Adaptive noise control system
Figure 1(a) illustrates a simplified ANC system
within earbuds. The interface unit performs signal
conditioning and audio codec, as well as signal
mixing and amplification. To reduce ambient noise,
typically assumed to be random and broadband, a
feed-forward controller is commonly adopted. This
setup involves one speaker to generate the anti-noise
and two microphones: one positioned upstream and
another downstream to measure the reference input,
x(n), and the noise residue, e(n), respectively. When
the speaker and error microphone are placed within a
sealed enclosure, two propagation paths must be con-
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

sidered. The first is referred to as the primary path, coefficient memories of the same length. The output
P(z), through which noise travels from the reference of the digital filter is the sum product of the values in
microphone to the error microphone. This path is the corresponding rows. Alongside an algorithm for
purely acoustical in nature. The other is the second- adjusting filter coefficients, an adaptive controller is
ary path, S(z), which exists between the controller’s constructed.
output and input ports. It is a propagation medium
that comprises both electrical components within the 2.1 A review of the FxLMS algorithm
circuitry of the signal interface unit, speaker, and er-
In this work, the least mean square algorithm
ror microphone, as well as acoustic elements within
is considered. The algorithm iteratively adjusts the
the space between the speaker and error microphone.same length. The output of the digital filter is the sum product of the
filter coefficients in a way that minimizes noise
rows. Alongside an algorithm for adjusting filter coefficients,
residue by following the negative direction of the
constructed.
error gradient. However, the presence of a secondary
2.1 A path review causes of the a phaseFxLMS mismatchalgorithm in the arrival of theon April 30, 20
Updated
anti-noiseIn thissignal, work, impeding the least mean the correct squarefilter algorithm
update.is considered
adjusts the filter coefficients
Therefore, an auxiliary filter must be introduced in a way that minimizesUpdated intonoise on April residue30, 2
same length. The output direction of theofdigital the filter
error isgradient.
the sum product However, of the the values
presence in the of correspond
a secon
rows. Alongside an the control
algorithm forloop to compensate
adjusting filter for the alteration
coefficients, of
mismatch in the arrival of the anti-noise signal,animpeding
[6]
adaptive the controller
correct
constructed.
Figure 1. A system overview: (a) ANC integration in earbuds. (b) the anti-noise signal by the secondary path . This
same length. The auxiliary
output offilter the digitalmust filter be introduced
is the sum product into theofcontrol the values loop in theto compensa
correspon
FxLMS based controller. rows. Alongside anauxiliary
anti-noise algorithmsignal filter, byoften
for adjusting
the referred
secondary filtertopath as a[6]secondary
coefficients, an path
. This auxiliary adaptive filter,controlle
often
2.1 Aconstructed.
review of the FxLMS algorithm
estimate, �() , can
pathestimate, can be be obtained
obtainedthrough throughtransfer transferfunction mode
In practical implementation, there may In be thisanwork, au- the least mean square algorithm is considered. The algorithm iterativ
approach function is known as the FxLMS algorithm.
adjusts
dio signal feed from the connected device, such of
2.1 the
A filter
review coefficients
asthe FxLMS amodeling
in algorithmway that or measurement.
minimizes noise residueThis approach Updated
by following onisApril the 30, 2023
negat
direction ofInthe known The block
asHowever,
the diagram
FxLMS in
thealgorithm. Figure 1(b) illustrates an adaptive afeed
thiserror work, gradient.
the least mean square presence of
algorithm a secondaryThe
is considered. path causes iterati
algorithm ph
a phone or audio player, that is sent to the speaker.which is implemented as a non-recursive filter with a length of . In r
mismatch
adjusts inthe
same the arrival
filter
length. ofoutput
coefficients
The the
Theanti-noise
ofblock in adigital
the way signal,
diagram impeding
the Figure
in
that isminimizes
filter sum product the 1(b)
noise correct
ofresidue
the filter
illustrates
values byinupdate. Therefore,
an corresponding
following
the the nega
This audio signal could potentially mixfilter
auxiliary
direction with
rows. ofmust theprovided
the
Alongside be
error introduced by theinto
angradient.
adaptive algorithm reference
fortheadjusting
However,
feed-forward controlmicrophone,
the loop coefficients,
presence
filter
controller,
theacontroller
to compensate
of W(z), secondary for generates
an adaptive
which the
path
is alteration
causes
controller theais anp
of
noise residue measured by the anti-noise
errormismatch constructed.
microphone.
signal in the bycan the be
arrival calculated
secondary
of the anti-noise path using[6] the following
. This
signal, auxiliary
impeding relationship,
filter, often referred
the correct filter update. to as aTherefore
second
path auxiliary
estimate, � ()must
filter
 , can implemented
bebe introduced
obtained as
into
through a the
non-recursive
control
transfer −1
loop
functionfilter
to with a length
compensate
modeling for
or the alteration
measurement. of
T
However, because information about the audio 2.1 A review sig- of the FxLMS algorithm[6]
anti-noise
approach is known signal as bythe of
theFxLMS In response
𝐿.secondary path
algorithm. to.theThis input,
auxiliary x(n), provided
filter, often by
referred the  to as a secon
nal is known, it becomes practical to separate Inthe this work, the least mean square algorithm = isconsidered.
   − The algorithm
=  iteratively ()
path estimate, �diagram
() ,reference
caninbeFigure obtained through transfer function modeling or controller,
measurement.
Theadjustsblock the filter coefficients in a1(b)
microphone,way illustrates the
that minimizes an=0adaptive
controller
noise feed-forward
generates
residue by following the the negative (
noise residue. Furthermore, the audio which signal
approach can
is
is implemented knownbe
direction of the anti-noise as the
a FxLMS
non-recursive
error gradient. algorithm.filter
However, with
the a length
presence of .
of acalculatedIn response
secondary path to the input,
phase(
Here, signal,
 denotes
y(n),the which-th can be
coefficient of usingcauses aThe
the update.
controller. ant
effectively utilized for both online and offline mod-
provided by The
the
mismatch block
reference
in the diagrammicrophone,
arrival ofin theFigure the
anti-noise 1(b) illustrates
controller
signal, an
generates
impeding adaptive
the the
correct feed-forward
anti-noise
filter signal, controller,
Therefore,
(), wh
an 
which
canfunction is implemented
auxiliary
be calculated
through
filter the
must as the
following
bea speaker
non-recursive
introduced to
into produce,
relationship,filter
the with
control a
looplength
to of .
compensate In response
for the to the
alteration input,
of the 
eling of the secondary path transfer [5]
anti-noise . Itusingis the following relationship,
by thesignal by themicrophone,
secondary path the .controller
This auxiliary filter, often referred to signal, as a secondary
[6]
provided reference −1 generates the anti-noise (), w
important to note that the scope of this can be work
pathcalculateddoes using
estimate, �() ,the canfollowing
be obtained through
 = transfer
relationship,   +function '  modeling =   or measurement. +   This ()
approach is known  
as the= FxLMS  
algorithm.
   −  =   () (1) (
not encompass a discussion of the algorithm for sep- The block diagram=0
−1
in Figure 1(b) illustrates an adaptive feed-forward controller, (),
arating the noise residue from the audio signal. which is the =
which is implementedasa non-recursive
noise  residue  measured
  filter −
with  a=lengthbyofthe () error microphone. This eq
. In response to the input, (),
Here,  denotes the -th
Here, coefficient
wl=0 denotes of the
the 𝑙-th controller.
coefficient The anti-noise
of the con-signal is transmit
provided byto thesignify
reference the superposition
microphone, the controller of generates
sound waves the anti-noisebetween signal, the (),unknow
which
The heart of the adaptive controller
through is athedigital
speaker fil- to  produce,
   , and the anti-noise,  '
 =  
 (), as they reach the c
canHere,
be calculated troller.
using the The anti-noise
following relationship, signal
 denotes the -th coefficient of the controller. The anti-noise signal is transm is transmitted through
ter, generally classified into two majorthrough
categories: the speaker
re- The gradient −1
of the error surface can be obtained by differ
 to the =produce,
speaker
   +=to' produce, =  − = +  ()  () (
cursive and non-recursive. The former reuses a part  = 2 () with respect to the filter coefficients. By applying (1) the
=0 '
of its output as the input, creating a feedback
which is the noise loop  measured
the iterative
residue = process  + the
for = microphone.
 error
adjusting  the + This
 controller  () (2)
coefficients ispositive
expresse
Here,  denotes the -thby coefficient of the controller. The equation
anti-noise usesis atransmitted
signal s
to signify
that results in a very long impulse response. the
While
through superposition
theaspeaker which of
to produce, sound
is the noise waves between
residue the
measuredThis unknownby equation ambient
the erroruses a positive noise,  
 which is the noise residue measured ' by
 the error asmicrophone.
they+ 1reach = the control −  'point.  ()

recursive filter has the potential to reduce   , and
the com- the anti-noise,
microphone.   = 
This  (),
equation uses athe
to signify the superposition of sound ' waves  between  positive
unknown sign to
ambient noise,(2) 
The gradient of the  error =  
surface +  
can = 
be  
obtained  +  by  ()
differentiating the cost funct
putational burden, it comes with inherent  drawbacks,
2   , and the anti-noise, signify the ' superposition
 =   (),ofas' sound they reach waves
 the control
between point.
  =  which () The with therespect
isgradient noise of toHere,
the
residue themeasured
error
notations
filter coefficients.
surfaceby thecan
 andBy
errorbemicrophone.
 = � the
𝄗 applying
obtained Tby
 ()gradient
Thisdifferentiating
equation
represent algorith
usesdescent
a the
the ite
costsign
positive func
such as response instability and local minimum
the iterative
  =
solu-
process
2
to signify
() with
reference
for the
adjusting
the superposition input,
unknown
respect to theoffilter the respectively.
ambient
controller
soundcoefficients. noise,The
coefficients
waves between d(n)
iteration
=
is p step,
(n)x(n),
expressed
the unknown
By applying
which
and
as the
follows,
ambient noise,
the gradient
is associated
descent  algori =w
tions. In this work, the preference is for the aiterative
non-recur-

   process positive
 , and the coefficient
anti-noise,
foranti-noise,
adjustingthe
' that  can
=   (),
 controller be selected
, as
asthey
coefficients they reachfrom
reach
is a range
the control
the control
expressed of
point.
as values
follows, that is n
The gradient of  the  +error 1 =surface   can − be ' obtained
 ()by differentiating 1 the cost function (
sive digital filter to facilitate convergence during   = 2 () with
ad- point. respecttothe filter coefficients. By applying the gradient descent algorithm,
+ 1 =   −  '.   ()= 2
aptation. It consists of one row of unit delays, Here, where
the iterative
notations process  for The 𝄗gradient
andadjusting '
 = the ofthe()
�controller error surface
coefficients
represent '()
can
is expressed
the as(
be obtained
iteration +
follows,
step )and the filte
a segment of input data is stored, reference
and another input, rowrespectively.
Here, of
notations  The
by differentiating
The iteration
andnotations '
𝄗   + 1=the
step,

�.
= which
cost −function
()and isassociated
 ' represent
 ()J(n) =the
represent e with
2
(n) convergence
with
the
iteration step and speed,
re-
maximum applic
the (3)filt
i
positive coefficient
reference that
input, inherent can be
respectively. selected
The from
iteration a range
step, of
which values
is
delay in the secondary path, respectively. Accordingly, it is e that
associated is notwithlarger than,
convergence speed,
positive coefficient Here, that canpath
notations
secondary beand selected
𝄗'  =from
imposes a1range
�a constraint of values
() represent that
the
on convergence is not step
iteration larger
speed and than,
the. A
[7] filtered
longe
27
reference input, respectively. . The= iteration step, which is associated with convergence speed, is a (
convergence.
positive coefficient that can beselected '() 2 ( 1+  )
= from a range of values that is not larger than,
. 2
The notations . and  represent the1(
'() +  ) applicable iteration step and
maximum
the2.2 The.
importance of the
'()secondary ) path step(4)and
 =
inherent delay
Theinnotations
secondary path,
and respectively.
 represent
.
Accordingly,
2the
( +maximum itapplicable
is evident iteration
that the delay in
 s using
implemented
dpeaker to as a non-recursive
theproduce,
following relationship, filter with a length of . In response to the input, (),
Here,  denotes the -th coefficient of the controller. The anti-noise signal is transmitted
d by the reference−1through the speakercontroller
microphone, the generates the anti-noise signal, (), which
to produce,
calculated using
  = the following
' relationship,

+   () (2)
=   +   =
 −1 −  =   () '   (1)
  =   +   =     +   () (2)
=0
noise residue measured   = by the  error −  = of Electronic
  microphone.
Journal This() equation uses a positive
& Information Systems |(1)
sign Volume
denotes the -th coefficient
which is the ofnoise
the controller.
residue The anti-noise
measured by the error signal is transmitted
microphone. This equation uses 05 | Issue 02
a positive sign| October 2023
e superposition of sound =0 waves between the unknown ambient noise,  =
ker to produce, to' signify the superposition of sound waves between the unknown ambient noise,   =
ndHere,the anti-noise,
 denotes the  -th , and
 =coefficient  (), of as
thethey
the anti-noise, reach
controller.
'
 =the  control
The (), point.
anti-noise signal
as they reach is the
transmitted
control point.
gradient
the speakerof the
to spect
error
produce, ' to the
surface filter
 can coefficients.
be obtained
 By
by applying
differentiating the
surface can be obtained by (2)
gradient
the cost functionby any ANC system to allow a broadband noise can-
 =   +   The =  gradient
  of+the  error
 () differentiating the cost function
with respect todescent the =algorithm,
 filter 2coefficients.
() with respect the iterative
By toapplying
the filter processthe for
gradient
coefficients. adjusting
Bydescent the
applying the cellation,
algorithm, gradient descent is met when the time required for the an-
algorithm,
'
rocess
se residue   =
for measured
adjusting thethe
by the +error
process
controller
iterative  adjusting
 microphone.
=coefficients
for   This is +the

equation
expressed ()
controller as follows,
coefficients
uses a positiveis expressed
sign as(2)
follows,
controller coefficients is expressed as follows, ti-noise to be generated and delivered to the control
superposition of sound waves between the unknown ambient noise,   = Updated on April 30, 2023
sthetheanti-noise,
noise residue
' = measured
+ 1 = (),by
  as the error
− they microphone.
 ' reach 
 ()  +
the(3) 1 = This
 
control point. equation
−  '  uses
() a positive signis faster than (3)
point
(3) the noise propagation time across
ify the superposition of sound waves between the unknown ambient noise,   = Updated o
dient of the error surface can be obtained by 𝄗differentiating the represent
cost function the primary path. In short, it is fulfilled when the
,respect
and theto
notations anti-noise,
andfilterHere, '
= notations
Here,
 �= notations
 μ  and
and '

   (), as they reach the control point.
𝄗' coefficients.  () represent
=
the
�
 

iteration
() represent
step and
the iteration
the filtered
step and the filtered
ith the reference By applying
input, respectively. The the gradient
iteration step,descent
which isalgorithm,
associated with convergence speed, is a path is smaller than that in the
The
ut,
cess gradient
respectively. ofthe the
The error
iteration
iteration surfacestepcan
step, and
which be isthe
obtained filtered
isbeexpressed
associated by asdifferentiating
reference
with convergence the cost
input, speed, delay
functiona inthan,
islarger the electrical ()
2 for adjusting the controller
positive coefficients
coefficient that can selected from follows,
a range of values that is not   =− () (5)
icient withcan
() that respect
be to the
selected filter
from coefficients.
a range of By
values applying
that is the
not gradient
larger descent
than, algorithm,
acoustical  path.
forrespectively. The iteration step, which is 1asassociated ()   =−
tive process  adjusting the controller
+ 1 =   −  ' 1 () . = coefficients is expressed follows, (3) (4) ()
with. convergence
= speed, is a positive '()
To coefficient
2 ( +  )
effectively that
 attenuate The primary
broadband
(4) To effectivelyrandom contributors
noise, the to electrical
transfer function delay thein-
of the controller,
  + 1 '()
=  2 (− + ' )
 () (3) attenuate broadband random noise, transfer function o
otations  and 𝄗 '
can be
 The
= �selected notations
 () represent
from . and
the
a range(),  ofrepresent
iterationshould stepthe
values andmaximum
closely
that theis notapplicable
filtered
match the clude
impulse the
iteration time
step
response required
and ofthe
the for signal
optimal conditioning,
controller. This
(), should closely match the impulse response of the optimal controller. is data
achievable if This
respectively.
notations .The and inherent
iteration delay in thethe
step, which
 ' represent secondary
is associated
maximum path,with respectively.
convergence
applicable Accordingly,
iterationspeed, step isit and
aisTherefore,
evident
the that the delay in the
Here, notations larger than,
andfrom
 secondary 𝄗 path 
�imposes the
represent inverse
the of
iteration() exists.
step [7]and conversion,
the
the filtered inverse it
of isand
() desirable to have
Therefore, a itsecondary
computation process. Signal condi-
exists. is desirablepath
to that
have acan be
secondary p
ent
y in that
thecan be selected
secondary path, a= range
respectively.
of ()a constraint
values that
Accordingly, ison
represented not
itconvergence
larger
is evident
bythan,a
speed
that the
causal
. A longer
delay
and in path delay leads
the
minimum-phase
represented by a
to slower
transfer
causal and function.
minimum-phaseIn practice,
transferthe delay
function. in
In the
practice, t
ce input, respectively. The iteration step, which is
convergence. associated with convergence speed, is a in the anti-aliasing and reconstruction filters
tioning
thcoefficient
imposes athat constraint on convergence
1 speed [7]
.
secondaryA longer path
path delay
should leads
be to
shorter slower
secondary
than path
that inshould
the be shorter
primary than
path. that in the
Additionally, primary
when path. Additionally,
using a non- wh
can
. =
be selected from a range (4) of values that is not larger than, (4)
2.2 The'() ( +of
importance the 2
) secondary pathfilter
recursive introduces
recursive filter aaslatency
as the controller, it should have sufficient length to accommodate the rational part that is
the controller, proportional
it should to thelength
have sufficient filterto accommodate
1
ations . and The . =thenoise
If perfect cancellationof is achievable,
the equation. meaning   the
Fortunately, 0oforder
→the asthe(4)equation.
the
control andplant
adaptation Fortunately,
inversely
converges,
associated the control plant associated with this setup is expec
proportional
with this setupto the filter corner
is expected to be of low
 represent
notations maximum
μmax.
'() and
2 ( +  applicable
represent
 )be
iteration
the step and
maximum
ortance of the secondary
the path
optimal controller,   , can derived from Equation (1) and order
Equation due to
(2) the
as: relatively small acoustical volume within the earbuds.
n the secondary path, respectively. Accordingly,  order due to that
it is evident the the
relatively
delay insmall acoustical volume
the frequency. within the earbuds.
The time for signal conversion between
The noise
fect
mposes notations applicable
.onand
cancellation
a constraint  iteration
represent
is achievable,
convergence step
the
meaning
speed . Aand  the
[7] maximum
longer 0inherent
applicable
→path asdelay delay
iteration
the adaptation
leads in
step and the
converges,
to slower
tontroller,
delay in 
the secondary path,
, cansecondary respectively.
be derived from Accordingly, it is evident that the delay 2.3
Equation (2) as:it is analog and digitalinPerformance
the factors in noise reduction
domains depends on the type
the path,Equation
respectively. (1) and Accordingly,
2.3 [7]
Performance
ry path imposes a constraint on convergence speed . A longer pathfactors in noise
delay leads to slower reduction
The causality condition, which must be satisfied by any ANC sys
evident that the delay in the secondary
ence.of the secondary The causality
path imposes condition, of converter
whichnoise
broadband must and be the
cancellation, number
satisfiedis metby of
whenanybits
theANC involved.
time systemfor
required For
totheallow a to b
anti-noise
ance path
a constraint
t noise cancellation on meaning
is achievable, convergence broadband
  →speed . A longer
0 as the adaptation path example, given the same resolution, a Delta Sigma
[7]noise cancellation, is
converges,
met
delivered whento the
the time
control required
point is for
faster the
thananti-noise
the noise to be generated
propagation time and
across the
importance of the secondary path delivered to the control point is short,
faster it isthan
fulfilled
the when the
noise delay in thetime
propagation electrical
across path
the isprimary
smaller than
path.that
Inin the
roller,   , can be derived
leadsisfrom Equation (1) and Equation (2) as: ADC The is typically slower thantoa electricalSAR ADC. Compu-
If perfect noise delay
cancellation toachievable,
slower convergence.
meaning short,  0 as the adaptation
it is→fulfilled when the delay converges, primary path
in the electrical contributors
is smaller than that indelay include thepath.
the acoustical time requ
mal controller,   , can be derived from Equation (1) and TheEquationprimary(2)contributors
as: tation
conditioning, in the
datacontroller
conversion, consumes
and one
computation sample
to electrical delay include the time required for signal in process. period
Signal conditioning
and reconstruction
during which filtersthe introduces a executes
processor latency that an is proportional
adaptive to the filter ord
2.2 The importance of the secondary
conditioning, datapath
conversion,proportional
and computation process. Signal conditioning in the anti-aliasing
to the filter corner frequency. The time for signal conversion betw
and reconstruction filters introduces control
digital adomains
latency that is on
algorithm.
depends proportional
The type of to
thelatency ofthethe
converter andorder
filter
speaker and
andinversely
the number of bits involve
proportional
If perfect noise cancellation is achievable, mean- given to the filter corner frequency.
the same The time
resolution, for asignal
microphones also contributes to electrical delay.Delta conversion
Sigma ADC between
is analog
typically and than
slower
digital domains depends on theComputation type of converter and the number of bits involved. For
in the controller consumes one sample period during which the pro example,
ing e(n)→0 as the adaptation converges, the optimal The Sigma delay
given the same
Updated on Aprilresolution,
30, 2023 aan Delta
adaptive controlof ADCthe is
algorithm.acoustic
typically primary
The latency of thepath
slower than
speakercan
a andbemicrophones
SAR ADC. als
controller, Wo(z), can be derived from Equation
Computation in the controller electrical (1) influenced
consumes onebysample
delay. various factors.
period In earbuds,
during which thedevice processor fit- executes
and Equation (2) as: an adaptive control algorithm. ting The and The delay
latency of of
thethespeaker
acoustic primary path can bealso influenced by various to fac
enclosure provideand microphones
passive isolation thatcontributes
device fitting and enclosure provide passive isolation that can increase the delay
can
electrical delay. [8]
() increase
path [8] the delay
. Intuitively, in the
acoustic primary
leakage reducespath . Intuitively,
the delay and the lowest estimate o
  =− (5) The delay of the (5) acoustic primary path can be influenced by various factors. In earbuds,
()   , can be leakage
acoustic calculatedreduces
from the the direct sound propagation
delay and the lowest as follows:
es-
device fitting and enclosure provide passive isolation that can increase the delay in the primary
y attenuate broadbandTo random
effectively
noise, attenuate broadband
the transfer
path function
[8] random
of
. Intuitively, noise, leakage
the controller,
acoustic timatereduces
of acoustic
the delaydelay,   ,=
andthe can
(∆be
lowest calculated
estimate
− from delay,
∆ ) /of 0acoustic
match the impulse response of the optimal controller.

the transfer function of the controller,
 , can This
be is achievable
calculated from if
the direct sound propagation
W(z), should the direct sound propagation as follows: as follows:
exists. Therefore, it is desirable to have a secondary path that can be Here, ∆ represents the distance between the two microphones, ∆
closely match
usal and minimum-phase thefunction.
transfer impulseInresponse
practice, of the the
delayoptimal
in the distance
  =between
(∆ − the∆speaker and the error microphone, and 0 is the sound (6) speed
 ) / 0 (6)
d be shorter than controller.
that in theThis
primaryis achievable if the when
path. Additionally, inverseusingofa S(z)
non- 343 m/s. However, when the path of arriving noise is closer to the error micr
controller, it should have sufficient it
length to accommodate Here,
part reference Δxbetween
re represents
microphone, the delay
thetheacoustic distance
may between the to ensure ca
not∆be sufficient
exists. Therefore, is desirable havethe
toHere, a rational
∆  represents
secondary the distance
condition is purely physical
two microphones,
and is less related to  represents the
the allocated
nately, the control plant associated with this distance
setup is expected
between tothe
bespeaker
of low and twothemicrophones,
error microphone, Δxyeand represents
0 isthe
the distance
the sound be- filter length in
speed, approximately
path that can be represented
vely small acoustical volume within the earbuds. by a causal and mini- Practical remedies include increasing acoustic path delay by improving passi
343 m/s. However, when the path tweenof the
arriving
speakernoiseand is closer
the errorto the and c than the
error microphone
microphone,
enhancing spatial information using the multi-reference microphone 0 method. Th
mum-phase transfer function. In practice,
reference the delaythein acoustic
microphone, delay may not be sufficient to can
ensure causality [9]
. This of
tors in noise reduction is the
through sound
the speed,
earbuds approximately
enclosure and ear-tip 343 m/s.
improve How-
the performance
the secondary path should conditionbe shorteristhan thatphysical
purely in the and is less
system related to
by increasing the the allocated
delay between the filter
twolength in the[10]controller.
microphones . The use of add
y condition, which must be satisfied by any ANC system to allow a ever, when the path of arriving noise is closer to the
primary path. Additionally, Practical
when remedies
using a include increasing
non-recur- the acoustic
microphones provides path delay by improving
a comprehensive passive
representation isolation
of noise coming and
from va
ellation, is met when the time required for the anti-noise to be generated and [11] error
. microphone than the referencemethod.
microphone, the passing
enhancing spatial information using the multi-reference microphone The sound
ol point is fastersive
thanfilter as the
the noise controller,
propagation timeitacross
should thehave sufficient
primary path. In Coherence
hen the delay inlength
the electrical path is smaller
through
than that in
the
the
earbuds
acoustical
enclosure and ear-tip
acoustic
path. because delaycanin
maymeasurements
improve defines
not bethesufficient the noiseofreduction
performanceto ensure thecau- performance of
feedforward
to accommodate thesystem rational part of the equa- the adaptation of controller
[10]coefficients relies on two correlated pieces
contributors to electrical delay include the timebyrequired increasingfor the delay between
signal sality [9]by
provided .the
Thistwocondition
microphones
the error is purely
microphone
. The use of additional
physical
and reference and is less
microphone.
reference
Based on the ana
tion. Fortunately, the control plant
microphonesassociated with
provides a
nversion, and computation process. Signal conditioning in the anti-aliasing processes,this
comprehensive representation of noise coming from various directions
related to thethe
noise reduction,filter
allocated , at alength
given frequency  can be estimated as:
in the controller.
ters introduces setup is expected
a latency that is proportional
[11]
to be oftolow . order
the due and
filter order to the rela-
inversely   = 10 10 1 − 2  
lter corner frequency. The time
tively small for signal
acoustical conversion
volume Coherence
within between analog
the earbuds. and Practical
in measurements defines remedies
the noiseinclude
reduction increasing
performancetheofacoustic
an ANC system
nds on the type of converter and the number of because the adaptation
bits involved. of controller
For example, coefficients
path delay relies on two
by improving passivecorrelated pieces
isolation andof en-
information
olution, a Delta Sigma ADC is typically provided slower thanby thea error
SAR microphone
ADC. and reference microphone. Based on the analysis of random
hancing spatial information using the multi-reference
ontroller consumes2.3 Performance
one sample periodfactors in noise
duringprocesses,
which the thereduction
noise reduction,
processor executes , at a given frequency  can be estimated as:
lgorithm. The latency of the speaker and microphones also contributes  to  microphone method. The sound passing through
= 10 10 1 − 2  
The causality condition, which must be satisfied the earbuds enclosure and ear-tip can improve the
the acoustic primary path can be influenced by various factors. In earbuds,
closure provide passive isolation that can increase the delay in the primary28
coustic leakage reduces the delay and the lowest estimate of acoustic delay,
d from the direct sound propagation as follows:

 = (∆ − ∆ ) / (6)
 ]
. Intuitively, acoustic leakage reduces the delay and the lowest estimate of acoustic delay,
an be calculated from the direct sound propagation as follows:

  = Journal
(∆ − of ∆  ) / &
Electronic (6)| October 2023
0 Information Systems | Volume 05 | Issue 02

Here, ∆ represents the distance between the two microphones, ∆ represents the
performance of the feedforward system by increas- phone, the ideal location is near the speaker, where
ce between the speaker and the error microphone, and 0 is[10]the sound speed, approximately
m/s. However,ing the the
when delay between
path the two
of arriving microphones
noise . The
is closer to the errorthe noise residue
microphone thanis the
present with a high correlation to
use of
nce microphone, theadditional
acoustic reference
delay maymicrophones provides
not be sufficient the reference
to aensure causality noise.
[9]
. This
ion is purelycomprehensive
physical and isrepresentation
less related to the allocated
of noise from lengthThe
coming filter performance
in the controller.of the controller is intricately
cal remedies various
include directions
increasing[11]
the
. acoustic path delay by improvinglinked passive
to isolation
numericaland accuracy, which, in turn, is de-
cing spatial information using the multi-reference microphone
Coherence in measurements defines the noise method. The
pendent sound
on thepassing
choice between floating-point and
gh the earbuds enclosure and ear-tip can improve the performance of the feedforward [13]
reduction performance of an ANC system because fixed-point systems in the processing hardware .
m by increasing the delay between the two microphones [10]. The use of additional reference
the adaptation
phones provides of controller
a comprehensive coefficients
representation of relies
noise on two from
coming Floating-point systems offer the advantage of high
various directions
correlated pieces of information provided by the er- precision, making them well-suited for applications
Coherence rorin measurements
microphone and defines the noise
reference reduction
microphone. on where
performance
Based numerical
of an ANC system accuracy is critical. However, typ-
se the adaptation of controller
the analysis coefficients
of random reliesthe
processes, onnoise
two correlated piecesdemanding
reduc- ically of informationmore computational resources can
ded by the error microphone and reference microphone. Based
tion, 0, at a given frequency ω can be estimated as: on the analysis of random
strain the limited processing capabilities of devices
sses, the noise reduction, , at a given frequency  can be estimated as: like earbuds. On the other hand, fixed-point systems,
  = 10 10 1 − 2   while more resource-efficient, may introduce numer-
where represents the coherence function be- ical errors due to lower precision, which can poten-
tween the output of the reference microphone and tially impact the accuracy of controller operations.
the output of the error microphone when the ANC The causes of numerical error in fixed-point systems
system is inactive [12]. The coherence function has are primarily related to the limited number of bits
a range of values between 0 and 1, indicating the available for representing numbers and rounding
quality of signal coherence, from poor to excel- during arithmetic operations. To address these errors,
lent. This can be interpreted as follows: The higher careful consideration of scaling, quantization, and
the coherence between the signals provided by the rounding techniques can be employed in fixed-point
microphones, the greater the noise reduction to be implementations to enhance the accuracy of control-
expected. In practice, signal coherence is primarily ler calculations.
affected by the quality of the instrumentation system,
such as microphone dynamic range, sensitivity, and 3. Implementation of adaptive ANC
directivity, as well as distortion in the signal amplifi- Developing high-performance noise-canceling
cation unit. Additionally, cable shielding and circuit earbuds involves considering numerous aspects [14],
isolation can help minimize input contamination such as the comprehensive acoustic design to pro-
from signal interference. duce favorable characteristics in the earbuds, which
In addition to its impact on system latency, the significantly impacts noise cancellation performance.
proper positioning of the speaker and microphones is Earbuds for ANC applications require careful in-
crucial for controllability. While the enclosure of ear- tegration of components such as speakers, micro-
buds is the obvious place for a speaker, the opening phones, controllers, and partitions. The arrangement
of the ear tip can extend beyond the ear canal open of these components within a compact, well-shaped
end, creating an impedance mismatch where sound enclosure defines the interaction between noise and
waves are transmitted and reflected. Conceptually, anti-noise. Additionally, it is crucial to incorporate
the optimal position for the speaker is at the domi- low-latency electroacoustic components into the sys-
nant anti-node. In terms of the reference microphone tem.
position, controller complexity can be reduced if While the design of the earbuds is a critical factor,
the microphone picks up a negligible amount of the this study does not cover every aspect of creating the
anti-noise radiated from the speaker. Therefore, it is perfect earbuds. It primarily focuses on realizing the
desirable to block the acoustic feedback path through adaptive controller. Therefore, for the experimenta-
proper enclosure design. As for the error micro- tion, earbuds sourced from the Bose QC30 are used.

29
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
Originally, these components were connected to the
main board in the neckband via cables. Each unit of
the earbuds has approximate dimensions of 28 × 30 ×
20 mm and is equipped with two 4 mm electret mi-
crophones and one 15 mm speaker with an imped-
ance of 32 Ω. To gain access to the internal parts, a
few modifications were made: the earbuds need to
be separated from the main board and the connecting
cables must be traced to identify the internal parts.
Two compartments can be found within the ear-
buds. The first one houses the error microphone and
the speaker, facing inward toward the ear tunnel. It is
isolated from the second compartment where the ref-
erence microphone is set. The reference microphone
is facing outward to pick up ambient noise entering
the ear. The distance between the reference micro-
phone and the error microphone and the distance
between the speaker and the error microphone is 8.5
mm and 2 mm, respectively, resulting in an acoustic
delay of approximately 19 μs in this setup.
High-performance DSP is essential for controller
development in ANC applications, particularly for
earbuds. However, implementing ANC in earbuds
can have a significant impact on device operation
time. In short, the DSP should be compact, require
minimal components to support its operation, ener-
gy-efficient, and cost-effective. These qualifications
hold especially true when considering processor
choices for mid and upper-range commercial ear-
buds. For example, options like the Qualcomm S5
Sound Platform and the Apple H2 BT5.3 Audio-
SoC offer functionality for audio processing, voice
services, and device connectivity in a single SoC
component, all while operating with low power con-
sumption.
For performing general experimentation, several
commercial DSP options with the corresponding
development boards are available, including the
CS47L85, i.MX RT1020, and TMS320C5517. Con-
cerning the computational speed, it must ensure that
the delay in the electrical path is shorter than in the
acoustical path. Theoretically, assuming a 19 μs
acoustic delay, any DSP working at a sample rate of
96 kHz or higher is sufficient. A minimum of two in-
30
put channels and one output channel are required. In
this study, the evaluation board EVAL-ADAU1787Z
from Analog Devices, depicted in Figure 2(a), is
employed. This choice aligns with the study objec-
tives as it addresses the challenge posed by the lim-
itation of computational resources. Additionally, the
small DSP footprint makes it a practical choice for
integration into earbuds.
Figure 2. Device selection: (a) DSP board and earbuds. (b) Pas-
sive noise isolation in earbuds.
The evaluation board is equipped with four ADC
channels, two DAC channels, a low-power audio co-
dec, and two fixed-point DSP cores. The first core is
the FastDSP audio processing engine, offering built-
in features such as biquad filters, signal limiters,
mixers, and volume controls. When passing a signal
from the ADC input to the DAC output at a sampling
frequency of 768 kHz, a group delay of 5 μs can be
expected [15]. This particular core can be programmed
to perform specific tasks using no more than 64 in-
struction cycles. Here, group delay represents the
time shift of a packet of oscillating waves centered
around one frequency that travels together. An in-
struction cycle denotes a discrete step a processor
takes to execute a single machine-level instruction.
The second core in the processor is the 28-bit
SigmaDSP audio processing core, offering additional
built-in functions, including FIR filters and many
custom algorithms. For this particular core, the max-
imum number of instruction cycles varies depending
on the sampling frequency, ranging from 32 in-
structions at 768 kHz to 512 instructions at 48 kHz.
Moreover, the core supports a high-performance
mode achieved by overclocking, which doubles the
number of available instruction cycles. It is worth
to note the aforementioned active noise algorithm is
implemented here using one DSP core, which is, the
SigmaDSP core at the normal clock mode.
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
3.1 Software development tool
For programming the SigmaDSP product lineup,
Analog Devices provides SigmaStudio, which is a
graphical programming environment for creating and
deploying signal processing programs on the eval-
uation board. It comprises two fundamental frame-
works, the first one offers access at the DSP register
level to assign built-in operational features, including
power management, signal conditioning filters, data
interpolation and decimation, and channel routing
between input and output (I/O) ports and the DSP
cores.
The second framework involves schematic tools
that allow the assembly of functional blocks to per-
form signal manipulation and control. These com-
posed schematics establish operational flow in the
device, which is called repeatedly at the start of each
sampling period. Each schematic block consumes
computational resources in terms of instruction cy-
cles and memory usage. Given the limited instruction
capacity at high sampling frequencies, it is crucial to
know the minimum feasible sampling frequency and
utilize computationally efficient blocks accordingly.
Fortunately, memory resources are relatively abun-
dant.
There are scenarios where essential schematic
blocks for ANC applications cannot be used due to
their high instruction cycle demands. For instance,
the built-in L-tap FIR filter function consumes 13 in-
struction cycles for function overhead and L + 8 in-
struction cycles for sub-routine overhead. Moreover,
modifying the filter coefficients during program ex-
ecution is also not feasible. Therefore, to implement
an efficient ANC program, it becomes necessary to
employ custom code that grants access to low-level
functionality such as shift registers, multiply and ac-
cumulate (MAC) operations, as well as the memory
read and the memory write operations. With custom
code, an L-tap FIR filtering can be accomplished in
L cycles of MAC instructions and 1 cycle of memory
transfer. The shifting of filter data is automatically
managed in the shift register at the start of each sam-
pling period. Detailed discussions of low-level pro-
gramming are beyond the scope of this article.
31
3.2 Design of the controller
A reasonable target performance of the controller
must be defined, such as the desired minimum noise
attenuation within a given bandwidth. One way to
determine these parameters is by assessing the pas-
sive noise isolation provided by the earbuds. To eval-
uate this, measurement was conducted in a controlled
listening environment using an artificial head and
torso (HEAD Acoustic HTB V) with a loudspeaker
as the sound source. The loudspeaker was positioned
one meter in front of the ear. A broadband, uniform-
ly distributed random noise served as the excitation
signal. By comparing the internal microphone re-
sponses with and without the earbuds attached to
the ears, the passive noise isolation plot displayed in
Figure 2(b) was obtained. From the data, one may
say the passive noise attenuation below 750 Hz is
poor. Therefore, the target to achieve in this work is
a significant noise reduction of at least 10 dB in the
operating bandwidth extending up to 1 kHz.
Considering that the earbuds are sourced from an
external commercial product and cannot be modified,
it is crucial to ensure the system is causal. To do this,
initial data about acoustic paths in the earbuds was
collected through measurements using the mentioned
test equipment. A sine-sweep signal served as the
excitation source. In the primary path measurement,
the signal was directed towards an external loud-
speaker, and the impulse response function between
the two microphones was recorded. In the case of the
secondary path, the signal was directed to the speak-
er in the earbuds, and the path response was captured
from the DSP output port, which is directly connect-
ed to the earbuds error microphone.
Taking into account the assumed 19 μs acoustic
delay, the DSP sampling rate option that meets the
minimum requirement is 96 kHz. The measurement
results at this specific sampling frequency are depict-
ed in Figure 3(a). One can notice that the leading
peak in the impulse response function of the primary
path exhibits a lower amplitude compared to the
secondary path, which is expected due to higher path
attenuation. It is worth noting that the first peak in
the primary path impulse response follows that of
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
the secondary path, indicating the system is causal.
In conclusion, the arrangement of microphones and
the speaker within the earbuds, along with the choice
selection of a 96 kHz DSP sampling rate, confirms
system causality.
Based on the initial assessment, the programming
work for creating the adaptive feedforward control-
ler can begin. Running at 96 kHz, the SigmaDSP
core at its normal clock rate can handle a maximum
of 256 instruction cycles. Approximately 43 instruc-
tion cycles are reserved for core housekeeping. This
means the entire algorithm, along with I/O port as-
signments, must fit within a total of 213 instruction
cycles. The bare minimum noise cancellation pro-
gram comprises built-in functions and custom code.
The built-in functions handle essential tasks such as
ADC input, signal generator, two-state logic switch-
ing, and DAC output. On the other hand, the custom
functions are responsible for signal conditioning
in the upstream path, reference input filtering, and
adaptive control. After allocating computational re-
sources to the built-in functions, 200 instruction cy-
cles remain available for reference input filtering and
adaptive control. It is important to optimize the filter
lengths for these functions to ensure efficient use of
this allocated resource.
There are three functional blocks in the FxLMS
algorithm that handle: anti-noise calculation, as rep-
resented by Equation (1), reference input filtering,
and coefficient adaptation following Equation (3).
The first two blocks, essentially filters, consume one
Table 1. Basic computational requirements for various programming schemes.
Computation cost (instruction cycle)
Computation process
FxLMS (a) FxLMS (b)
Core housekeeping 43 43
ADC: Reference signal 2 2
ADC: Error signal 2 2
DAC: Control signal 2 2
Controller output LWB + 1 LWB + 1
Filtering of input SWB + 1 (SWB + SWA )+ 1
Filtering of output - -
Controller adaptation 2 + 3 LWB 2 + 3 LW
32
instruction cycle per filter coefficient, while the last
block consumes three instruction cycles per filter
coefficient. Table 1 provides a summary of the dis-
tribution of instruction allocations for three potential
design scenarios. In the first column, the controller
and the secondary path estimates are non-recursive
filters. Referring to Figure 3(a), it is evident that
a substantial filter length is required when using a
non-recursive filter to represent the secondary path
estimate, approximately 100 taps to cover the first
microsecond. This condition restricts the length of
the controller to no more than 25 coefficients.
The second scenario involves using a recursive
filter to model the secondary path estimate, which
has proven advantageous as it significantly reduc-
es the required filter length without compromising
controller stability. The optimal filter length for the
secondary path estimate is determined using the
line search method, which is 16 coefficients in the
feed-forward part and 19 coefficients in the feedback
part. Figure 3(b) provides a comparison between the
impulse response functions of the measured the sec-
ondary path and secondary path estimate. The length
of the controller is significantly expanded to 40 coef-
ficients.
The optimal controller can be used to determine
an appropriate controller length. Figure 4(a) dis-
plays an estimate of the impulse response function
of the optimal controller. This calculation, performed
without regularization, utilizes the measured acous-
tic paths. Although not entirely precise, this estimate
Notes
FuLMS (c)
43 (a). W(z) and are non
2 recursive filters
(b). W(z) is non recursive
2 filter, is recursive filter
2 (c). W(z) and are
(LWB + LWA )+ 1 Recursive filters
(SWB + SWA )+ 1 Notations WA and WB
(SWB + SWA )+ 1 in the table indicate filter
association to the feedback
4 + 3(SWB + SWA ) and the feed-forward parts,
respectively
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
serves as an initial approximation for determining
the required length of the controller filter. It becomes
evident that the first microsecond of the impulse
response function carries over 90% of the signal
power. While it is conceivable to set the controller
length near 100 to mimic the dominant optimal re-
sponse, this proves unrealistic because the remaining
resources support only 40 coefficients.
Figure 3. The modeling of secondary paths: (a) The measured
impulse response functions of the earbud’s primary path (red)
and secondary path (blue). (b) Comparison between impulse
response functions of the measured (blue) and the modeled (red,
dashed) secondary paths.
Figure 4. Design of the controller: (a) A non-regularized impulse
response function of the optimal controller estimate. (b) The
performance of the controller in terms of noise reduction at a given
multi-rate factor Nd = n (green), Nd = 2n (red), Nd = 4n (blue).
While one may suggest an alternative approach
involving a recursive filter as the controller, it is es-
sential to note that, as detailed in the third column
of Table 1, additional resources must be allocated
for output filtering during the adaptation of the feed-
back filter. The feedback loop may potentially put
controller stability at risk during adaptation. Never-
theless, there is some potential, considering recent
developments in alternative algorithms that aim to
address stability [16]. Another suggestion, applied
here, is based on multi-rate signal processing [17]. It is
a straightforward approach that involves decimating
the signals. The processes for filter adaptation and
anti-noise generation are performed at different rates.
In summary, the final configuration is as follows:
a 40-tap non-recursive filter for the controller and a
33
35-tap recursive filter as the secondary path estimate,
which corresponds to a total of 248 instruction cy-
cles. A few more instruction cycles were allocated
for a second-order recursive filter, inserted upstream
for input signal treatment. The performance of the
controller to cancel broadband random noise is de-
picted in Figure 4(b). Interestingly, the slopes of
the plots at different multi-rate factors, denoted as
Nd, show variations around 450 Hz. It is believed
that with a smaller multi-rate factor, the observation
time in the controller becomes shorter, making it
challenging for the controller to regulate the low-fre-
quency components, and vice versa. When Nd is set
to 2n, an average noise reduction exceeding 10 dB is
achieved in the frequency range up to 1 kHz, satis-
fies the given design target.
The experimentation showcases adaptive control-
ler design by limiting DSP capability. Greater noise
reduction over a wider frequency range becomes at-
tainable with additional resources. For instance, run-
ning the DSP board in overclock mode doubles the
total instruction cycles to 512. Furthermore, utilizing
Biquad filters in the FDSP core to model the second-
ary path estimate frees up more instruction cycles.
Some signal processing functions, such as multi-rate
signal processing, are performable by the hardware.
These additional resources offer the feasibility of ex-
panding controller length and implementing sophis-
ticated algorithms for improved noise cancellation.
4. Conclusions
The study discussing the development of active
noise control systems for earbuds has been present-
ed, emphasizing the significance of maintaining low
latency to ensure causality for effective broadband
noise cancellation. Nevertheless, when working
within the constraints of earbuds, the available hard-
ware resources are limited, hindering the utilization
of sophisticated adaptive control algorithms. To
overcome these limitations, optimization of filter
configuration is employed, along with implementing
a computationally efficient program. Moreover, mul-
ti-rate signal processing techniques provide practical
solutions for achieving the desired noise reduction.
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
Conflict of Interest
I declare that there is no conflict of interest re-
garding the publication of this paper.
Funding
This research received no external funding.
Acknowledgement
The author extends gratitude to Dr. Ji-Ho Chang
of the Korea Research Institute of Standards and Sci-
ence, Daejeon, and Prof. Sung-Hwan Shin of Kook-
min University, Seoul, for providing access to the
test facility and experimental equipment.
References
[1] Liebich, S., Fabry, J., Jax, P., et al. (editors),
2018. Signal processing challenges for active
noise cancellation headphones. Speech Commu-
nication; 13th ITG-Symposium; 2018 Oct 10-
12; Oldenburg. Berlin: VDE. p. 1-5.
[2] Bai, M., Lee, D., 1997. Implementation of an
active headset by using the H∞ robust control
theory. The Journal of the Acoustical Society of
America. 102(4), 2184-2190.
[3] Goldstein, A.L. (inventor), 2016. Adaptive feed-
back control for earbuds headphones and hand-
set. Patent Application Publication. US patent.
2016/0300562 A1.
[4] Kim, Y., Park, Y., 2019. CPU-GPU architecture
for active noise control. Applied Acoustics. 153,
1-13.
[5] Gan, W.S., Kuo, S.M., 2002. An integrated
audio and active noise control headset. IEEE
Transactions on Consumer Electronics. 48(2),
242-247.
[6] Morgan, D., 1980. An analysis of multiple cor-
relation cancellation loops with a filter in the
auxiliary path. IEEE Transactions on Acoustics,
Speech, and Signal Processing. 28(4), 454-467.
[7] Kuo, S.M., Morgan, D.R. (editors), 2000. Re-
view of DSP algorithms for active noise control.
IEEE International Conference on Control Ap-
plications. Conference Proceedings (Cat. No.
00CH37162); 2000 Sep 27; Anchorage. New
34
York: IEEE. p. 243-248.
[8] Zhong, X., Zhang, D., 2021. Latency prediction
of earmuff using a lumped parameter model.
Applied Acoustics. 176, 107870.
[9] Zhang, L., Qiu, X., 2014. Causality study on a
feedforward active noise control headset with
different noise coming directions in free field.
Applied Acoustics. 80, 36-44.
[10] Rafaely, B., Jones, M., 2002. Combined feed-
back—feedforward active noise-reducing head-
set—The effect of the acoustics on broadband
performance. The Journal of the Acoustical So-
ciety of America. 112(3), 981-989.
[11] Iwai, K., Kinoshita, S., Kajikawa, Y., 2019.
Multichannel feedforward active noise control
system combined with noise source separation
by microphone arrays. Journal of Sound and Vi-
bration. 453, 151-173.
[12] Hansen, C., Snyder, S., Qiu, X., et al., 2012. Ac-
tive control of noise and vibration. CRC Press:
Boca Raton.
[13] Chang, C.Y., 2009. Efficient active noise con-
troller using a fixed-point DSP. Signal Process-
ing. 89(5), 843-850.
[14] Huang, C.H., Pawar, S.J., Hong, Z.J., et al.,
2012. Earbud-type earphone modeling and mea-
surement by head and torso simulator. Applied
Acoustics. 73(5), 461-469.
[15] Four ADC, Two DAC, Low Power Codec with
Audio DSPs ADAU1787 [Internet]. Analog
Devices; 2020. Available from: https://www.
analog.com/media/en/technical-documentation/
data-sheets/adau1787.pdf
[16] Mohammed, J.R., Singh, G. (editors), 2007. An
efficient RLS algorithm for output-error adap-
tive IIR filtering and its application to acoustic
echo cancellation. 2007 IEEE Symposium on
Computational Intelligence in Image and Signal
Processing; 2007 Apr 1-5; Honolulu. New York:
IEEE. p. 139-145.
[17] Graf, J., Reithmeier, E., 2009. Computationally
efficient active noise reduction in headsets. Jour-
nal of Computer and Systems Sciences Interna-
tional. 48, 567-573.
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

Journal of Electronic & Information Systems

https://journals.bilpubgroup.com/index.php/jeis

ARTICLE

Control and Treatment of Bone Cancer: A Novel Theoretical Study

Ali Soltani Sharif Abadi1,2* , Mansour Rafeeyan3 , Vahid Abootalebi4
1
Institute of Automatic Control and Robotics, Faculty of Mechatronics, Warsaw University of Technology, Warsaw, 00-
664, Poland
2
Institute of Electronic Systems, Faculty of Electronics and Information Technology, Warsaw University of Technology,
Warsaw, 00-664, Poland
3
Faculty of Mechanics, Yazd University, Yazd, 8915818411, Iran
4
Faculty of Electrical Engineering, Yazd University, Yazd, 8915818411, Iran

ABSTRACT
The human body has symmetric bones. This paper uses control engineering concepts to design a suitable controller
to synchronize two symmetric bones of the human body to control and treat bone cancer. A Nonsingular Terminal
Sliding Mode Control (NTSMC) method will be employed to design the proposed control inputs. The control inputs
can be the chemical drugs that can be used to treat bone cancer. The dynamical equations of bone cancer will be used
to apply the designed control method and test it. For testing the designed controller, Simulink/MATLAB software will
be used. The proposed controller is chattering-free, robust against uncertainties and external disturbances, and finite-
time stable in the control engineering view. Bone cancer will be treated for almost one year using the proposed control
method.
Keywords: Bone cancer; Synchronization; Finite-time stability; Biomedical engineering

of cells: Osteoblast (OB) and Osteoclast (OC).

1. Introduction This collection is called a Basic Multicellular Unit
The human bones are composed of two types (BMU) [1,2]. Bone diseases are diverse, one of them is

*CORRESPONDING AUTHOR:
Ali Soltani Sharif Abadi, Institute of Automatic Control and Robotics, Faculty of Mechatronics, Warsaw University of Technology, Warsaw, 00-
664, Poland; Institute of Electronic Systems, Faculty of Electronics and Information Technology, Warsaw University of Technology, Warsaw, 00-
664, Poland; Email: ali.soltani_sharif_abadi.dokt@pw.edu.pl
ARTICLE INFO
Received: 9 October 2023 | Revised: 4 November 2023 | Accepted: 7 November 2023 | Published Online: 17 November 2023
DOI: https://doi.org/10.30564/jeis.v5i2.6004
CITATION
Abadi, A.S.S., Rafeeyan, M., Abootalebi, V., 2023. Control and Treatment of Bone Cancer: A Novel Theoretical Study. Journal of Electronic &
Information Systems. 5(2): 35-44. DOI: https://doi.org/10.30564/jeis.v5i2.6004
COPYRIGHT
Copyright © 2023 by the author(s). Published by Bilingual Publishing Group. This is an open access article under the Creative Commons Attribu-
tion-NonCommercial 4.0 International (CC BY-NC 4.0) License. (https://creativecommons.org/licenses/by-nc/4.0/).

35
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
bone cancer. Bone cancer happens when the growth
rate order of the bone cells (OB or OC) is disrupted,
which grows into cancer cells (CCs) [1-3]. Osteosar-
coma (OS) is a type of bone disease. When the OS
happens in the bone, the discipline of the growth of
the bone cells disorganizes. OS is more likely to hap-
pen at 13-16 years old and after 55 years old. This
sickness more occurs in boy children [3,4]. OB cells
are responsible for the remodeling of bone, and OC
cells are for bone growth. If the order of growth and
reproduction of these cells is lost, OB cells will grow
more and cause CCs [5,6]. In the healthy bone (bone
without cancer), OB and OC cells multiply clearly
and periodically. However, in sick bone (cancerous
bone), there is no systematic growth and reproduc-
tion [1].
Bones are the skeleton of the human body, and
almost all of them are symmetrical. If one of the
human bones becomes cancerous, the closest value
of the parameters is its symmetric bone. Therefore,
symmetric bone parameters can be used for the re-
construction and treatment of cancerous bone. This
fact can be used to model, design, and control can-
cerous bones. This concept in control engineering
is called “synchronization”. In the synchronization
problem, the variables and parameters of the “slave”
system will be the same as the variables and parame-
ters of the “master” system [7]. In the synchronization
of two human bones, the cancerous bone (slave)
will be the same as the healthy bone (master). For
synchronization, the systems need to apply control
inputs to the system. These control inputs in the syn-
chronization of the human bones can be considered
as the effect of the dose of the chemical drugs. Re-
cently, many control efforts have been made using
the synchronization concept in different fields, such
as synchronizing communication systems [8,9], chaot-
ic systems [10,11], and chemical systems [12,13].
The Nonsingular Terminal Sliding Mode Control
(NTSMC) method is a robust finite-time control
strategy that guarantees that the system states reach
zero at a finite time. The NTSMC is an extended ver-
sion of Finite-time Sliding Mode Control methods
that have been used in controlling different applica-
36
tions. This method has been used to solve the stability
and tracking problems of rigid manipulators, high-order
nonlinear systems, and robotic surgery [14-16]. It is used
for controlling some practical systems such as ma-
nipulator robots [17], perturbed nonlinear systems [18],
DC-DC buck converters [19], Quadrotor unmanned
aerial vehicles [20], underactuated underwater robots [21],
acute Leukemia therapy [22]. Recently control engi-
neering methods have been used to increase biomed-
ical applications such as drug delivery in cancerous
tumors [23], tumor treatment immunity [24], cancer
chemotherapy [25], control the tumor growth [26], and
angiogenic inhibition therapy [27]. An extended adap-
tive NTSMC using fractional disturbance observer
has been presented to accelerate system response
without resulting in chattering [28]. Also, the NTSMC
has been used to deal with the time delay for con-
trolling the integrating processes [29].
One of the challenges in the designed controller
by NTSMC is the chattering phenomenon. The chat-
tering phenomenon is because of the high frequency
switching gain in the controller. Chattering is a very
harmful phenomenon in control applications. It can
reduce the actuators’ age and add unwanted noise to
the system. In biomedical applications, especially
cancer treatment, the chattering causes to control in-
puts will be uncreatable. It means that the chattered
control inputs cannot be created in the practical tests.
Some types of control methods are developed to
remove, eliminate or reduce the chattering from the
control input signals [30-33].
This paper proposes three control signals to syn-
chronize two symmetrical human bones to control
bone cancer. It is assumed that one of the human
bones (arm or leg bones) is cancerous with can-
cer, and it will be treated by applying the proposed
control inputs, which are the effect of the chemical
drugs. The proposed control inputs will be designed
by the NTSMC control method. The control inputs
are designed using the chattering-free concepts. Be-
low are the most important features of the proposed
control method:
● Robustness against model uncertainties and ex-
ternal disturbances,
 � ()
� ()≤− 44 55 This paper aims to synchronize the OBs and OCs cells of the c
yunbounded
unboundedfunction functiontherefore
thereforeifif ≤−1  1   −− 2
2  () ()sosothetheequilibrium
equilibrium
of this system ==00 will be globally finite-time stable, and state variables
symmetrical healthy2.bone and destroy
Considering scalars theCCs. The model of choosin
and bone O
globally
lobally finite-time
finite-timestable,stable,and andstatestatevariables
variablesofofthis thissystem
systemconverge
convergefrom from each
condition
healthy bones
≥Lemma
eachinitial
initial
to zero, andsame,
are the the upper bound
and only its1 ,settling
theofparameter
2 , …, ∈ℜ
time
values isare
fordifferent
 ≤ 3
−1−1[36]
eupper
upperbound
boundofofits itssettling
settlingtimetimeisisfor for≤≤ 3 3 1  122 ..
[36]
of its parameters for healthy and cancerous bone are published  [1,2,38
2 2 2
diseases. 1 +
≥Lemma
These
 2. Considering
arethe
2

+
most … +common  scalars

1+
≥cancerous 1 , 22 ,+
bone ∈ ℜ andThis
…,…+diseases. 2 [37] .choosin
mo
idering
sideringscalars
scalars1 ,1  , …, ∈∈ℜℜ and
, 2 ,2 …, andJournal
choosing of Electronic
choosing 00<<<<&22Information
then
thenwill Systems
willhave
have  1 1| Volume

++ 2in 05
  | Issue 02 | October 2023
++

presented 2 Equation (2).

+……++2 2 2 2[37] ..
[37]
11  + 2  + … +   ≥ 21 + 3. Explanation
2
+ … +  2 2of[37]
the. purp
● Chattering-free control of the bone cancer, � = 1  −   +  
This paper aims to synchronize the OBs and OCs cells of the
1 1
2

●3.3.
Accurate
Explanation
Explanationtracking ofofthe
the ofpurpose
the healthy cells,
purpose � = 2 2 healthy
symmetrical − 2  +bone 2  and destroy the CCs.(2) The modelofofthebone O
3. Explanation purp
stotosynchronize
synchronizethe ●
theOBsSmooth
OBsand control
andOCs
OCscells of the
cellsofofthe system,
thecancerous
cancerousbone bonetotoOBs OBsand andOCs healthy
OCscells � =  bones
cells3ofThis1
ofthe − are
thepaper the+ same,
  2and only
+   1the parameter values are different
− 
foraims
 3 to synchronize4 the OBs3 andareOCs cells of [1,2,3
the
ne
e and anddestroy
destroythe the●CCs.
CCs. of its parameters healthy and cancerous bone published
FastThe Themodel
model
tracking ofofof bone
the bone OBs,
masterOBs, OCs
OCscells,
states, cells,and andCCs CCsfor forcancerous
cancerous
symmetrical and
and healthy bone and destroy the CCs. The model of bone O
me,
me,and andonly
onlythe theparameter
parameter values
values areare different
different .
[1][1]
.
TheThe provided
provided bone
bone model
model where
diseases.
andand the u,values
the v and are
These
values ω arethe the mostdensity common of cancerous
OC, OB and boneCC diseases. This mo
● Implementable results in control signals. healthy
where , bones
 and are the
are same,
the density and only
of OC, the OBparameter
and CC values
cells, are different
respectively. 
ealthy
althy and andcancerous
cancerousbone boneare arepublished
published[1,2,38] [1,2,38]
for
forMixedMixedLesionLesionand presented
cells,
andOsteolytic
Osteolytic Lesionin Equationα (2).
respectively.
Lesion βi, i parameters
,healthy = and
(1, 2, 3)andmultiplication
OBof its and parameters
CCs and are for ifixed cancerous bone are
positive.  ,published
 = (1, 2, [1,2,3
3, 4
most
ost common
commoncancerouscancerousbone bonediseases.
diseases.This Thismodel
modelisiscalled calledthe theKomarova
Komarovamodel, model,
diseases. which
which These isis are the most common cancerous bone
and diseases. This m
rate � =ofOC,
between OC, OB and
1OB
1  − 1  + 1 
and CCsCCs that and1 ,are
3 are fixed parameters
positive 2 , 4 are negativ
.). 2. Mathematics presentedOBs
between
and � = positive.
2  −
in2Equation
and OCs
σj,j2 =+(1,
(2).that
22, are coefficient
 3, 4)[2]coefficients constant and  1 < 0,  2 > 0 and 
model of bone mass is as follows :>
1 1 Definition 1. Function siɡa(x) with the relation for� the == relationship
131 −− 1 +
1
132 +OC,
+between 4 1OB −and 3 CCs that
2 2 between absolute function |x| and symbol-function σ1, σ3 are  � = 
(2)
(2)  2
−   +  
2 positive2 and σ2 , σ are negative or positive.
2
3  22 11 where
� =− , 1 max − are�,the
0 density
+ 4 max { − �, and
0} CC cells, respectively.
3  ++ 4 4  for −−siɡ
3 3a(x) = |x|asiɡn(x) is defined. Function siɡn(x) is 1and 3 22 +betweenof4 OC, 1 OB
γ1, γ� 2=are 3the −rate
 of +signaling −
OBs 3 and OCs
[34,35] OB and CCs and are fixed parameters and positive.  ,  = (1, 2, 3, 4
defined as follows : that are γthat γ2are and K and
where
between iscoefficient
 of the bone massand and 1 < 0,, 
1 ,1
> 0positive
3 arenormalized
isactivities
the of bone f
edensity
densityofofOC, OC,OB OBand andCC CCcells,
cells,respectively.
respectively.,  , , , ==(1, (1,2,2,3) 3)multiplication
multiplicationwhere rate ,OC,
rate of
OC,OC,
and OB and
are the CCsdensity of2OC, OB and CC cells, 4 are negativ
2 , respectively.
xed xedparameters
parametersand andpositive.
positive. ,, ==(1, 4) coefficients
coefficientsconstant
constantfor are
ability
forthebetween
the steady-state
to carry
OBs of
and the
CCs.
OCs OB As and
that wellOC
are ascells
the
coefficient that are
modeland presented
of
 bone
< 0, as follows:
> 0 and 
siɡn(x) = (1)
(1,2,2,3,3,4) OBrelationship
relationship
and CCs and are fixed parameters and positive. 1  , 2 = (1, 2, 3, 4
Css that
that1 ,1, 3 3are
arepositive
positiveand and2 ,2 
, 4 4arearenegative
negativeororpositive.positive.1 ,1 , 2 2are
arethe model
mass
therate
rate of of isofasbone
signaling
signaling followsmass[2]is: as follows [2]:>
between 1OC, OB and CCs that 1 , 3 are positive and 2 , 4 are negativ
hat
thatare arecoefficient
coefficientand and1 1<<0,0,2 2>>00 and and isisthe theability
abilitytotocarry
carryCCs.
CCs.As As =well
between well
2 asas
OBs the
theand OCs that are coefficient and 1 < 0, 2 > 0 and 
Definition 2. The relation between absolute and 
� 2
follows[2][2]: :
sfollows model
� =− of
2
bone mass  is, 0as follows [2]
:>{ − �, 0} (3)
signum function is as |x| = xsiɡn(x) . [34] 1 1 max  − � + 2 max
 
Lemma 1. For a nonlinear system = f(x), f(0) = � = 1 1
where 1
z is
is the
thebone
bone mass and , k1, are
k2 are normalized
++2 2 max
max{{−−��, 0}
0, x D
, 0}
, x(0) = x0 by assuming the constants ρ1 � =− (3)
where (3)
1 max  − mass and
, 0 +
� 21max 2 { −normalized
�, 0} activities of bone f
activities For healthy bone, the values of parameters areand
of
are steady-state bone
of formation
the OB and OCthat are
cells constant
that are presented as follows:
presented as follo
sand
and1 ,1  to
, 2 2are ρ as ρ
arenormalized
normalized > 0, ρ >
activities0,
activities ρ > 1,
ofof3bone ρ = 1 –
boneformation
formation ,
thatρ
thatare= 1 +
5areconstant positive.
constantand
and positive.
positive.
where  is 
�,
,
� 
,
� 
� are steady-state of the OB and OC
1 the bone mass and 1 , 2 are normalized activities of bone f
4 1 2 4
BBand
andOC
OCcells cellsand
that
thatare
arepresented
presented asasfollows:
follows: are =0.3,
2 2 2 =
1 steady-state of0.1, 1and
the OB = 0.2,
OCcells
2 =that
0.02,
arepresented
1 =− 0.3, asfollows:
2 = 0.5,
Lyapunov function , as a sca- cells
� = that are presented as follows:
0.045,2 3 = 0.05, 1 = 0.001, 2 =− 0.00005, 3 = 0.005, 4
lar continuous radially unbounded function therefore 11
1 1
if – ρ1Vρ4(x) – ρ2Vρ5(x) so the equilibrium x = 0 � =
� = 1 addition, for the Fixed Lesion disease, these parameters have
In
2 2

(4)
2(4)
1 (4)
of this system will be globally finite-time stable, and
1 1
state variables of this system converge from each � = For healthy bone, the values of parameters are presented as foll
1
initial condition to zero, and the upper bound of its 1For = 0.3,  =bone,0.1, the
healthy 1 = 0.2, 2
values = 0.02, 1 =− 0.3, 2 = 0.5,
ne,
e, the
thevalues
valuesofofparameters
parametersarearepresented
presentedasasfollows:
settling time is for T
follows: [36]
. For 2
healthy bone, the valuesofof parameters
parameters areare
presented as foll
0.045, 3 = 0.05, 1 = 0.001, 2 =− 0.00005, 3 = 0.005, 4
presented as follows:
1==0.2,
0.2,2 Lemma
2==0.02,
0.02,1 2.=−
1=−
Considering
0.3,
0.3,2 scalars
2==0.5,
0.5,1 a10.07,
1==
a2,...,
, 0.07, a2
2 and 3
n ==0.0022,
0.0022, 3 ==
 1 = 0.3, 2 = 0.1, 1 = 0.2, 2 = 0.02, 1 =− 0.3, 2 = 0.5,
In
α1maddition, for the Fixed
β2m = Lesion
0.02, disease, these parameters have
==0.001,
0.001,2 choosing
2=−
=−0.00005,
0.00005, q3<==20.005,
0 <3 then will
0.005, 4
4= have q
|a1=|=
=0,0, |a2|q + ... + α0.045,
+300
300 1m = 0.3,
(5)
(5)
3 0.1, β
== 0.05, 1m = 0.2,
1 = 0.001, 2 =− 0.00005, 3 = 0.005, 4
q [37]
γ1m = -0.3, γ2m = 0.5, k1m = 0.07,k2m = 0.0022, (5)
he
theFixed |andisease,
FixedLesion
Lesion |disease,
≥ these
theseparameters . values
parametershave
have valuesasasfollows:
follows: α3m = 0.045,
In βaddition, σ1mthe σ2m = -0.00005,
3m = 0.05,
for = Fixed
0.001, Lesion 
disease, these parameters have
σ3m = 0.005, σ4m = 0.005, σ4m = 0, Km = 300
3. Explanation of the purpose In addition, for the Fixed Lesion disease, these
parameters have values as follows:
This paper aims to synchronize the OBs and OCs
cells of the cancerous bone to OBs and OCs cells of α1s = 0.3, α2s = 0.1, β1s = 0.2, β2s = 0.02, γ1s = -0.3,
γ2s = 0.5, k1s = 0.023, k2s = 0.0023, α3s = 0.055,  (6)
the symmetrical healthy bone and destroy the CCs.
β3s = 0.05, σ1s = σ2s = -0.005, σ3s = 0.001, σ4s = 0, Ks = 3
The model of bone OBs, OCs cells, and CCs for can-
For the synchronization of two healthy and can-
cerous and healthy bones are the same, and only the
cerous bones, the synchronization errors are defined
parameter values are different [1]. The provided bone
model and the values of its parameters for healthy as e1 = us – um, e2 = vs – vm, e3 = ωs – ωm where m is
and cancerous bone are published [1,2,38] for Mixed the abbreviation of the master system (healthy bone),
Lesion and Osteolytic Lesion diseases. These are the also s is the abbreviation of the slave system (cancer-
most common cancerous bone diseases. This model ous bone). This paper aims to reach these errors to
is called the Komarova model, which is presented in zero at a finite time.
Equation (2). The error dynamic will be as follows:

37
 1 For2the synchronization 1 of2two healthy 1 and cancerous 2 bones,1the synchronization 2 errors are 3 defined as  3 1 =
 − 1For
0.05, ,For
=2the thesynchronization
0.001,
= synchronization
 −2  ,=− 3 =0.005, of − of
two two
3
healthy
= where healthy
0.001, and  and cancerous
is4the cancerous
= 0, bones,
abbreviation  =3
bones, theof the the synchronization
synchronization master system errors(healthy errors
are defined arebone), defined
as also 1 = as (6) is 1the =
abbreviation
 − −, ,2 2==of the −−   ,  ,  = =   − −  where where
slave system (cancerous bone). This paper aims to reach these errors to zero at a finite time.
 3 3      is the is abbreviation
the abbreviation of the of masterthe master system system (healthy (healthybone), also
bone),  is the
also  is the
abbreviation
abbreviation Forofof the the the slave slave
synchronization system system (cancerous (cancerous
of two healthy bone). bone). Thisand Thispaper cancerous paperaims aims tobones, reach to reach thesesynchronization
the errors errors
these to zerotoatzero a finite
errors atare atime. finite
defined time. as 1 =
 −  , The 2 =error  −dynamic  , 3 =will
Journal
 be
ofbeElectronic
− as  where
follows:  is the abbreviation
& Information Systems | Volume 05 | Issue 02 | October 2023
of the master system (healthy bone), also  is the
The Theerror error dynamic will as follows:
abbreviation of thedynamic slave system will be(cancerous as follows: bone). This paper aims to reach these errors to zero at a finite time.
1� 1� == 1 −−
1 − 1
1 ++
+ 1+1 + 11
In these control inputs ki1, ki2 are positive
1� � =
2� 2= 22− 22 =  1The − error 1 +dynamic
+ 2 +2 22

 1+ + 1will be as follows:
(7) (7) con-
2� � =  −  +  +  (7) stants and β , β are positive and smaller (7) than one
3� 3== 2
33− 33
−  ++ 2
3+3 + 2
33 2 i1 i2
3�1� = = 3 −− 
 ++ 
 ++ 
1
1 3
1 3 1 31 constant.
where
where 2� ==0.3, 2 −22 = 0.1, + 2 1 += 20.2, 2 = 0.02, 1 =− 0.3, 2 = 0.5, 1 = 0.023, 2 = 0.0023, 3 = 0.055,(7) 3 =
= 0.02, 1 =−where where
0.05,
0.3, 
3� =23=−0.5, 1 = 0.001,
31 +=  30.023,
2 =− 0.005,
+ 3 2 = 0.0023, 3 = 0.055, 3 =  3 = 0.001,  4 = 0,   = 3 Proof: It has been shown that sliding (6) surfaces
= 0.02,  =− 0.3,  ==30.5,11
 = 0.001,14 = 0, 2
= 0.023, 2 = 0.0023, 3 = 0.055,(6) 3 Equation = (11) have finite-time stability, provid-
1=For =1 31 − −11
the
1
synchronization  ++ 1 1
of two healthy and cancerous (6) bones, the synchronization errors are defined as 1 =2
3 = 0.001, 4 = 0,   =
1
 = 1
where 1 =2     
 −   −   +    ed that the c i1, master ci2 are chosen so polynomial of p +
1 += 1
2
2, = 21 −  
=30.2, +
2
wo healthy and cancerous 1− =2 0.3, 2bones, =2= −
0.1,
the  2 ,2 
synchronization 2 
 2− =0.02,  errorswhere 1 are  is
=− defined
0.3, the abbreviation
2 as = 0.5, 1 =1 =of0.023, 2 =system 0.0023,(healthy 3(8) = 0.055, bone), (8)3also =  is the
 
=  
  
 2
− 1
  + 
2
  (8)
two healthy andabbreviation cancerous
0.05, 2
=13 =10.001, bones,
−of
2   the
  the
the slave
+ =− synchronization
3system
2  2
0.005,
 +
 2
24
+(cancerous
 3
1 
4=(healthy 1errors
−
0.001, 3− are
bone).43defined
bone), = This 0,  paper as
= 3 is1cthe =aims i2p +to ci1reach =0 is these Hurwitz errors to zero [39]
. Foratprooving (8)
a finite (6) time. the reach-
 where  is the abbreviation of  master system  also

3 3 = 3 111− − 1 2 + 3  
=1 3=aims
 −  +2 1+
  
 where  isThis the3 abbreviation
cerous of

the master
1 system (healthy
1
bone), also  is the
bone). paper to 


reach  + these 3   errors  4 
to  zero   −
at  a 3 
finite time. ing phase (second part), consider the Lyapunov
ncerous bone). This 2For
paper =aims
The the 2 synchronization
error

to2 reach
dynamic− 2these  +will errors
ofbe
2 two as  tohealthy
zero atand
follows: a finite cancerous time. bones, the synchronization errors are defined(8) as 11 =
And<>
and 3 1 3
And<> −  ,  =   −  ,  =   −  where
 consider  is thethe Lyapunov
abbreviation function
consider of function the masterthe V  ==
Lyapunov system 
(healthy  which
which
function bone), has has
 = conditions
also conditions
 is2the  which of the of has Lyapunov
the conditions functio of
s follows: And<>  = 2 1 −
3 3

 +3 3 + 4  − 3 2 1 =1 2 =1
as follows: abbreviation of the  slave system (cancerous bone). This paper aims Lyapunov to reach these function errors to � of zero =Lemmaat a 3 finite time. 1
 1. Then will have
 � 3 3
1 1� = = 1 −  1−+11+
1  1
 1
+
1 1   will have  consider = =1 the will�  and
consider Lyapunovhave with the applying � = Lyapunovfunction the and control with inputs =
applying which to
3 the
3 1
1 has
the system conditions
control alsoinputs putting
oftothe theup Lyapuof
syste� i
consider the Lyapunov =1   function function =1 2 3 211 which
which has
has conditions
conditions of
 � 1==  1 − 
consider 21+  1
− the +  + 1  function
Lyapunov 33 11 3  = 3
1
 whichconsider has the conditions Lyapunov of function
the Lyapunov
= =
3 1 function
=1
=133 21  1(7) whichof Lemma has conditions
1. Then of
consider
consider the
2the Lyapunov function  =  � =  which
 ( � has+  �
conditions
) consider
as � consider follows: 3 of the the the � Lyapunov
Lyapunov
Lyapunov � function
function
function of = = Lemma
3 1. which
whichThen has
has conditions
conditions of o
Lyapunov −21 function   which  (has 2 conditions of the Lyapunov asapplying function
follows: of Lemma 1. Then

And<> 21 = 
2 =  2  
error 2dynamic
2 −  2  + 
+2be
2      =   will =1 have
9)
  consider
consider
will = have the the  �� = (LyapunovLyapunov
� Lyapunov and
and +3 
with with ) function applying
function  = the
 the
= control 
2 control
=1
3
(9) which
1
2 2inputs whichhas
inputs conditions
to the has tosystem conditions of the
also Lya pu
of
consider =1 the  �� �  and function with applying the  control inputs to the syste
which has conditions of
3� 2==3 The will as follows:  3   = 2
 =1 =122 =1  3 =1
=1
 1 1− + −32+�  + 31 (7) will
will3 1have have   � = = =1 and
and with
with applying
=1
applying
=1
=1 the
2  control
2
the (9)
control � inputs
inputs to
to the
the syste
syste
punov function will
2 will
� � 1=−which 233 have has consider = 2
the+applying  �
Lyapunov
and  with applying
 function
the control inputs
which � � = � asto 3
has the
=1 3 3 system
conditions also of putting
the � � Lyapunov up  � in 
function so can be
of Lemma written 1.
sysT
conditions ofthe the Lyapunov function =of Lemma 1.
 = the
3
will have 3Then
�� will have andand withwith applying
applying the
 = 3 3 3
3� =inputs � 2system �  also=�= putting � and � � and up in so can the bebe control
controlwritten inputs
inputs totothe thethealsosyst
2have and 3with −control willto the have 3system
� � also putting
3 ==1 32  
�+ = follows:   withup � �  in
applying sothecan control be written
inputs to the system
2= − � 
will =have and + 
 2 
with 2
4
applying 1

the control inputs willto ( the have + 
system )�� ( = also =1
3 putting
�� �  )) �� as up
follows: with in
applying so can the written
control inputs to syste
=1 3  +  (9)
=1 (7)
1� 3 = − 1� � =
 =1
−+ =1
1 3 
2
+ + 
2
+ 3 � 113 −−   will=1
1� � � =
have
 3
3=1 33 ( =1
+ =1 =1 =1
  and
as follows: with applying 2�
the control inputs to the syste
�   = =
331 3   
− �  +   ( � + 
3 � � )  as 
follows:
4 = 3  
  � ==
=� == � − 3
=1  
 ( (

be�
( − � +� +
 2

 =1
 � 
  ) �
as )� − 1 as 
follows:
asas  follows:
 −
follows:
follows:   +  −    � +  �
� == � − � ) as  follows: = ( ++   ))as
 will have and with up applying the control inputs
 � and withwhere applying 1 =  (
the control + + )inputs ��  )) to the system also putting 2up � in  so can be wr
to 1
 3the  = system also
� =1 putting � in so can written
3   
(  as +follows: �� = �� 
1 1 1 1  2  2  
=1  
2� U =, =1 i=1
1 1  =1 1 2
+ =1  − 3 3  
=1=1
3
=1
=1
=1  ( (  +
 1
 
+ as
as follows:
follows:
follows:
2 
2=− , (1, 22, +3) are the 3models of4the the control � =in-that 
3 3  +are
= (7)
2 =1
=1 1 
� ) as follows: � 2=i = 2  (1,  2, −  23)  �   + models
2 the
2  � of � control
as follows: inputs
 will � − be 3 designed
  in the
−   next  2
section  and
−     are the  + �  + �� 
 = ( +  ) =1 � � = 3  −    −   2  − (9)

2    3
1   1
2   2
2  
−,  + and 33 +3) 3are =1  1 by simplifying: 
�  = 3=1 −are −− � 1  1 
1
available  1 1
− − − 
2 
2 2−
2section
 22and  2− − 2 
 �  + � 
the models of the control inputs that will be designed in the next   +
� be = 2are  the
puts ==3that 33 will = (1,  2,
designed in the next section and D by
 = simplifying: 3  111as     +
++
models 3of unknowns uncertainties. Assuming that 
the 2 upper bounds for  follows:
� � 
3
  − 3
= 1 
 1
 − −   
2
 − 
−      − +    � � � = = 
3
=1
 +  − −  
1   
  −
2 − 
2    − − (13)
 
2   (13)  � �
��   
� 3
 ==1 =
 
3 
 ,   −1 
= −(1,
11  −
=1
2, 
 3)1 are
1


+   +  −1
the   
1 models
3 2 2
2
 +  2
4   −
1
of the control
 
2  2
2
 
−  3
 

inputs
 i
  + 2  
that �  =
=bounds

=1
will
=1
3 
=1befor
=1
=1 

  −
designed
 −

 1 
1


1    −  22
1
in the   next −  
2section


2    − 
2
and
 −  (13)
 
are the 
22
 
 +
+ 
models 1 =1 of unknowns 1   and uncertainties.
 
  Assuming  thatby the
simplifying: upper =1
 1 1 are available  as 2 follows: 
 2
 2
1 are the
 models of
2 unknowns

� = 3
� 2and − uncertainties.  1  − As- 
upper  by simplifying: (13)

� (13) (13)
� − −32   1+ +available
  − 2  2  2  
models
where
|  | ≤
of = −22
 unknowns  − and 2 ++
uncertainties.  1Assuming
� = 3=1 −that 1 2the  1+ by
by by bounds
simplifying:
simplifying:
simplifying:
simplifying:
 − 1+ for 2−
 are 2 | 1 |− � as1+ follows: (8) 2 | | + � 
 = 1  + 2 
2 −
21 =1  1
byby simplifying:
simplifying: by simplifying: by by simplifying:
2=1 (10)
 suming
|| � | |≤≤  that the
1, −=1(1,2,+3)are 2  upper bounds  the models for
41i of
D
 are available
the control  � =inputs
as by
by that simplifying:
simplifying:
will be designed in the next1+ section � and  are the
3 = 3  +  − 3(8)
23 1 3 by simplifying: 3 1+1 −  1+
1of   +
(8)� that − � �� =  1 333 −− 1+ 1+ − 1 − 2  −1+ as |− |2 +− | +|� |(10) + ��  
| � || ≤  =  
 models = 1 unknowns 1   − 31 andby simplifying: 1 
uncertainties. Assuming the upper  bounds for  2 are1+ 1 
available 2 22 1+
follows: |2
1
 −   follows:
| ≤   � = −   1+ 12− since   the 1+  2  − ≤ ||
=1
� ||
| +
 
since | �  �
=
=also� �  the
=1=1
33 � − − � ≤
1
 1
1 ≤ 
  | so:1+
�
1
|| 1
|
−
− 2
also

 2
  � 1+
≤
2
2 2

−
− so: 2 ||||  ||(10)
+
+ 
�   (14)
| | + �� � �  
� 33 2 1+ 1+ � =1 1+ 1+ 1+
1+
1  3 
  − 3 | | ≤2=1
   � = =   = −−   
 
2
− 1+=1  1

1 − − +  
1   1+ 2 − − 4. 2  
2 ||

Designing | | ++ 
 � 
 
 the
2
 control
 
� = �  =inputs = =  3
=1
3 
− −  − 1 
 
1
1 1 2
     1+
1+
  1 1 1
− − − 
2 
2
2 2    
 1+2
1+
 2 2 2
2
−− − 
2 
2
2
2
 |(14)
  | +
(14) +
 − 1  − 2 (8) − 2 | | +   (14)
=1 2 2 1   1  2  22  
2  2     =1
=1 1
=1
And<> 3 1  1+1 −  since  1+the since �  2 ≤the =1
|two �  � || �� � parts. ≤|≤ also �  || ��  ||  � |≤also part�  so: ≤�� is≤
� � 3
 � since the  | also so:
1+1
− 2  1+ | 3 | =
2 ≤ −3 Designing1| 1− | +  the
+
  controller =2=1 + − using  1  the
 � −NTSMC 3 4. Designing 2method the
2 −
consists control
since  | of |3 +
the 
inputs 
� � � (14)
≤ || The first also 2  2 so:
designing so: the (14)
2≤ | � � || 1||−  ≤ 2
2
2
(10)
since the   ≤ | �
4
|| | also  ≤  � 3
≤ −   so:  1+

1 since � − ≤  the   − 1+
 | 1+ ||
� also    � � 1+
� ≤ 2 so: (10)
since
since
sliding
| �  | ≤the the
surfaces
2Designing  � �
  ≤  
≤ |
1|proof
and � � ||
  ||  | of | also
also
  �
theirstability,�  ≤ ≤ 22 so:
  so: 4. Designing
and the second part issince
=1 2 1  the since
control since
since
since proof 2the
the
the
the
the
=1inputs
of 
  �
reaching
  two �  
  1
 ≤ ≤ ≤
≤ |parts. | | | � � the ||
 ||
 || |
sliding
|
  |
alsoalso
also
also
also 1+
2 
surface.
   � �   ≤ ≤ ≤  
2
Since so:
so:
2isso:
22 so:
so: this
 − the controller using  both� the NTSMC method consists 1||1+  | The 2 ≤ 2
� || | also � paper ≤ 12aims
 = 1for finite-time 1stability,
since  + 1
thetheir �must these parts
also � prove
≤� method
3 
at2− aso: � � finite ≤
3 1+
333time
 −
− of to 1
 −  1+
ensure 1+ the − − finite-time
22  first  1+
part
stability.
designing the
so: Designing the controller   using ≤ |the  ||NTSMC |1+ ≤=1 consists ≤ of 1  
1two parts. The
 1 2 first part is designing the
2
And<>
sliding surfaces and proof of  stability,
12− due and the second part
 � � ≤ 1 =1
is proof −  of  2
reaching 1+
1+ 1 − the sliding 1+
1+ (15)
2surface. Since this
 � � ≤ =33surfaces 2 −   � ≤  2
 −1+
3
proof 1− −+
− their

1+
1+  2 2 4.
to the Designing Lemma the
2 2: control due� ≤ � ≤ ≤ to =1
33 −
the
=1
inputs Lemma
− − 1 1
 1+ 2: 1+ 1+11− 2  1+
1
− − 
2
 

 1+ 2
1+22
(15) (15)
  sliding

paper aims ≤
2
=1 − for1finite-time
1 and
 
1+=1 1
2 
1of
stability,
22  
2
1+ stability,
must both and these the second
parts prove part � ≤
 � ≤
attoasurfaces is 3
3proof
=1
finite
=1
=1 −
− 
time of1 1 1 reaching
2: to ensure
 1+
 1
1 − − the the2 2 sliding
2 finite-time
  1+
 2
2 surface. stability. Since (9)
(15) this
=1 Theorem 1: Consider system Equation (7), defined thedue due
sliding the Lemma 1Equation (11), 2 and control inputs
atofinite the Lemma 2: The
=1
1+1 − 2  4. Designing −− the +control + inputs
paper aims for1finite-time 
stability,  2 must 1+  1 these due to Lemma =1 2:Lemma
1+ 1 2 =
3 = 3 
Designing
1
1 
the1� ≤ +
controller
3
13−  using  4both  1
the − in
NTSMC −parts 31+  prove
(9)
method
(9)
2
 time. due
due 1 +1to
at to
consists the
the Lemmaof 1+1two
time (15)toparts. 2:
2: ensure +1 first
the finite-time
1+1
2 part is stability. designing the2+1 (15)
Equation (12). So
due2 the  states
to the Lemma of this =1 system
2:
1 reach � zero 3 a finite
2 
due � to the Lemma 32 2: 1 2+1 +1 2 +1
1
++4 1  −− due
sliding
due 32to to = the the
 surfaces Lemma
Theorem
 Lemma  and
− 2: 2:
1: proof
Consider +  of their
  system stability,  
Equation ≤ and the (7), −
second defined 2 
part
due due due  sliding to
is
to to
≤ 
the
the
proof
the
1the  Lemma Lemma
surfaces Lemma of − − reaching 2: 2:
2 2
2:
Equation +1 the1  
2sliding
1(11), and
 22 − (9)
surface. 2+1 Since
control 2+1
inputs  2
this
3  Theorem 2   2  2   =1 due
3� sliding to≤ =1 3 Lemma =1
− 1 +1 2 Equation
2: 1
1 +1  2  −+1
 +1 2
2 control 2+1  inputs
2
(16)
4   1: due Consider 2system Equation (7), defined 2surfaces
 (11),  and +1 2
1Designing the controller using the NTSMC meth- 2+1
2: paper
Equation aims (12). ,for finite-time
So  11the (states ) +are stability,
to ofthe
the
this 12system
Lemma
models must ) of both
2: reach
the these zero
control  parts
� in a≤
inputs prove
finite that �� at time. awill
− finite be 3 time
3
designed to12ensure in

1 2+1
+1
the
1 the
−next finite-time 1
2 +12
section 2 andstability.
 2
2+1
2 +1
are the
2+1
3==�1 +3  11
1=  (1, 2, 3) 312  (
−  +   +   1
 1− +1    ≤2+1 − 1 +1  1 12+1 − 2 2 +1   22+1
Equation (12). � So  the 1
states
11 3+1 of this
  11system+1 
+1 1 4 1
+1
with reach the

22+1 zero 3
selection in

a
22+1
of
+1 
finite 2 +1 
parameters

with
=1
� 
time.  the≤
≤ selection values 3
=1 − − as of 22follows: parameters
1
1+1 +1 +1 

1 +1 1 values
1 
 


112 +1
1 +1 −− as 22
follows:
2
2+1  +1 +1 2 
 
 22 2+1
2+1 +1
22 (16)
  ≤ +1 − 2 22 ( ) 1  +1 − 2   1 1 +1 2 2
els of the control od
models
  �  �  consists

2inputs= ≤
≤ � of
2 that + 3
unknowns
 3 of
21will − − two

21
be 2 (
2 parts.
and 2 =1 
designed ) + 
 The
uncertainties.
22 
 
in

first
the
2 2 next − −2 partAssuming 2 2 is
section designing
2
that

and   the 2 upper
2 are with with
the 
 sliding � �  � �  bounds

2  thethe
≤ ≤ ≤ 2
selection
selection
=1
for
3
=1
3
3 3
 −
− 2
− −of are parameters of
2 2 
available
2  parameters
1
   values
11 as


 
12+1
2 values
follows:
2
as − − −
(11)
follows: 2
2
2as
2 
 2
2follows:
(16)
+1
(16)   
2
2 
2 
2 
 

22+1

≤ 1 − control 2
1  2   2 2
dels of the control inputs that
� 1+Theorem
=1 will 31be 1: 11designedConsider+1) +  in 1  the12 next section
1 +1 and
2  are the =1 =1
12 system Equation (7), defined surfaces
=1 2+1 Equation
 +1 1 (11), 2
and inputs 2
1 +1 2 3the
1+1
1== � 3 +=1  +1  ( 
)( 1 )follows: with
1
2 +1 =1
2the −selection of parameters values as follows:
2 ( )2� are +1  32  332 (
2
s.2Assuming
es. Assuming 1 thatthe
that 1the sliding
−= upper
upper
 � +, 31
2(12). =11
surfaces
bounds
bounds (1,
 22,  for3)
for and
3
( 
 ) are proof
the
are
+ ≤  available
models
available  ofsystem
− their
3of
( as
as the stability,
control
2)follows: 11inputs +1 and that with
will2 be
with the
the selection
designed
selection +1 2(16) +1 in2 of theparameters
of next
parameters section values
valuesand as  are
as follows:
the
follows: (16)
Equation
| = of 1 1 + with
� 2selection 11
21 
So the
the
11 21 states
(uncertainties.
selection 1
2 ) + 22
of 12 this
ofvalues 
=1
parameters
12 22 ( 1 reach
) values zero
2 theasupper in a
follows: finite with time. the selection 1+1  of
2
1 parameters values 2+1
+1= 1 as 1
follows: (11) 1
1=1+,1 −=1, 5==1
 |2 ≤ as 12= 1 >bounds 0, = 2 =
2 > 0, follows:
4 =
2 − , 0, =4  =
with
models
with the
the unknowns
selection ofofand parameters
parameters values Assuming as follows:
follows: that  withwith with = thethe
forthe
2 2  selection
selection selection are  available > of
2 >
of of
0, 0,
parameters
parameters
 
parameters
as
= = 2 2 values
values
values
1  > 
asas as
0, >
follows:
follows:
follows:
 = 5   = = 2 1 −
the� 23second =
= �� 2 2+ part
21 
3 + 31  (3 ) + 32  (3 )
is31 proof
21 (2 ) + ofreaching22 32 (the
22
2 ) sliding sur- with
11+1the selection
1  1
1 of parameters
1 = 2 2+1 values
2 2
2 as follows:  3 4 (10) (11)1 1
1 3

53 3 2
| | ≤ 10,=41=−(17)
1+1 2+1 1
of parameters values as follows: with the selection of parameters 1are =
values 2 as1 follows: = 12>0, 1+1 21 = > 0, 22 and =  2 >(0,1).
0,
2+1 42 => ,= 5 1=−2 1 ,=
1, 1+
 =  +   11( ) +   12( ) 5 =
where  � ,  are positive 31 control parameters 32 and  ,  positive constants as ∈
face.
| |1 ≤Since  3 = 1 
11 11this
3 � +
2 31 11 paper 1aims  ( 
3 1) +
+1 3212for  finite-time ( 3 1 ) 1stability,
 2 +1 2 (10)
(10) 1 = 2  1

+1
+1 +1   11 > = 0,  = 2 2 2 +1
122+1 +1 > 0,  =  1
1 = 1 − =
+1
 +1 = 2  > 
0, 22+1
 +1
where = 2 > 1:  > 0, where 1− 11===, ,353522= =1 1  2=
 >
2  11:
1 1
+1 −   ,
31111>> >  >0, 0,= 1 1
0,0,22=2===
2 = = 1 2 + 
22 3 (10) +1  2 >
22>>>0,0, 0, 0,  4 =  3 = ==111−−−3 , ,,555=
1
(17)−  311, 5
(17) 4444= == ==
2 1
1�1= =≤=2� 22 + 21 1 11 >  > 210, 0,22)2 =
( + =1222 2 22 (  2 222)> >0, 0,3 44 = = 2 11 ==11where − 1
4
=
1
2
> = 22 = 
1: 1
1 +1

 =1
1
> ++ 5 2−

0, 
2 2
=  2
2
 2
2 +1


2
> 0, (11)(17) 4
= 
1
11= = 1 −
13
, 
5
=
=
must
| |
 2+1 2
both 2 these parts prove at 1a 1+1 finite 4.
time to ensureDesigning 1 the
2+1 control 1  3 3 inputs 1  
3 1
3 1 2 2
2 1
1
1 
33

3 3 5
> 0,  = 2 where  =    � ,+>  0, are  positive
= 31
 
( = = ) control
+
1 − 2  , parameters
 32
= (  > ) 0,
= and
1 + =  where , 
1  2 3 where2 are >positive
 1: > 0, > constants
 =1: (17)  =as 1  − 1 = , 
  = and
 
= ∈
1 +(0,1). (17)
3=the where 5 ≤r− >4of1 1:
1 2  1
2 3 +231  4inputs 113 32 5 1 32 2 where � � consists 2 > 1: 4 1 25The 3= first 5 2
4.4.Designing
Designingthe
where finite-time
the 1Designing
control
control
, 2 where are stability.
positive
inputs the controller
control 3 parameters using� the  ≤ NTSMC
and−113 , method 4 − are 
2positive where inputs ≤333constants
3− > 1: two
 −4 − parts.
as2 25  and partis∈ designing (0,1). 3 the
where
where   > > 1:1: 3 > 1: 4. Designing the 2 control where
where where   > > > 11:
1: 1:  = 2−
sliding  = surfaces 3 − and
3 − 1proof  oftheir 1 − 2  and the second part
stability, 2
where is 
proof 3
3 > 1:3
of reaching 2
the sliding (12)
surface. Since this
ing the
sing the NTSMC
NTSMC method Theorem
method consists
consists 1:1Consider of
of where two parts. system The Equation
firstNTSMC part is
part is(7), designing
� de-≤consists
designing −thethe ��1 of  3
4 −   54 
first 5 (18)
1 =part
2−
 is designing the
ability,
tability,and and the paper
second  �  =aims − Designing
part 1 for+is � finite-time
proof
the  − controller
−5reaching
of 43 >
stability,
 2using
1: must − the 2
sliding both  surface.these method
surface. parts Since prove 
this
due � � at  to a≤ ≤
≤finite
the
−
two
−− Lemma
21parts.
time 1
44 to
 1,
−
−−
The
the
2 
ensure
 2 
system 55 the
finite-time
Equation (2)∈isstability.stable for
the where

fined
sliding � second� =≤ 
−
sliding
≤surfaces 1
,
part
− + 11
2  isare
surfaces

 
 −
and
4  positive
proof
4 −proof ≤ 22Equation 5of control
2  −
1 their stability,  parameters
the
2(11),
5sliding
due and
and to the
2 the
and
control Lemma
second 1 , 
Since
in-2 are
part1, the positive
this
isdue
 �� proof � system
 to≤≤ ≤
the constants
of− −Equation
−Lemma
reaching  111
1  
44 −

4 4 −as
−
1, the (2)
 2the
2 is
22sliding   
stable
255system
 5 5
= surface. for and
Equation 
a finiteSince(2)
(0,1).
time,
this is(18) anda the
stable
(18)
finite
forsettling
time,
a finite(18)and time the se
time,isan
ust both
must both these
these parts
parts  prove =
prove

 at at −a a finite

finite − time
  to 1
ensure  − the  finite-time

finite-time  stability.
stability. due
Theorem  to ≤ 1−
the is  proved.
1 
Lemma − ∎ 2  1, the 2− system Equation (12) (2) is
−  2  5 paper  aims for−finite-time
  1 stability,
�   must 4both
2
Theorem these 2 5 parts

1 is prove
proved. at ∎ aLemma finite time to (18) ensure the finite-time stability.
 the≤1states − 2  − Theorem 1,1Lemma is proved. −1 (18)

puts � Equation =InTheorem  these (12).−
control 1: So Consider
inputs 1 ,system  −
1 of are this Equation
positive 2system (7),
constants due
reach toand
defined the due 1 ,sliding to 2 the are surfaces
positive the system and
1, Equation the ∎ Equation
smaller system (11),
than (2) one
Equation isand stable control
constant. (2) for is (12) a inputs
stablefinite for time, aa finite and the time, sett an
  =

=

−  + due
1 
 to
1 1 −  
the  Lemma 2 1,
2 2 −  
the system
 2 Equation   (2) is stable
duedue
stable
to
to for
the
thefor a
Lemma
Lemma finite a finite time,
1,
1, the
the time,and
system
system the and settling
Equation
Equationthe settling
time
(2)
(2) is
is
is 
stable
stable
≤ time 
−1−1 for
for is a 
finite
finite

time,
time, .
an
aa
m Equation Equation
due
(7), due
zerodefined � to

defined
to the the (12).
Lemma
Lemma sliding So the1, 1,surfaces thestates
the system
system of this
Equation Equation2system (2)
Equation (11), reach
(2) is
andis zero
stable Theorem
control
stable in for
fora finite
a finite
inputs
a 1
finite due due
is time. to
time,
proved. to
time, the
the and and LemmaLemma
∎ the the settling
settling 1, 1, the the system
system
time
time is is Equation
 Equation
≤  (2)
(2)  is is
5. stable
stable
Simulation . for
. for a a finite
finite time,
time,
 in aTheorem finite 1 time.
em Equation (7),   = −   sliding   1: surfaces
1 
Consider −  
2 system Equation  −  2and  (7), control 
defined inputs due
Theorem
sliding due to
to the
the
surfaces 1 Lemma
Lemma is proved.Equation 1,
1, the
the ∎ system
system
(11), and 
Equation
Equation ≤
control  (2)
(2)
33inputs is
is  
stable
11stable
22 3 for
for a
a
1 2
finite
finite time,
time, aa
   Theorem
Theorem 1
111isaim is proved.
proved. −1 ∎ 5. Simulation 5. Simulation
stem
1, thereach
ystem reach
system zero
zero Theorem
Equation
Equation
Theorem in 
in aafinite=Proof:
finite 11time.
(2)
(12). isisisTheorem
−
time. proved.
Soproved.
Ithas
stable  −states
the
been ∎
for ∎1 
due is
a
shown
to

finite
of
1 that
proved.the
this Lemma
time,
system − ∎sliding and 
2reach 1, the
surfaces

the
2 
system
settling
zero  in a finite
Equation (11)
Equation
time istime.  Theorem Theorem ≤ (2)
haveThe
 is
finite-time
stable is is proved.
 proved.
for of athis
stability,
finite .∎ ∎ Theorem
∎
paper time,
provided
was and
to 1 is
the
show
that proved. 
settling
that
,
1 OB  2(12) time
and is
OC  cells≤ of cancerous  
are1� chosen Insothese polynomial control 1
inputs 2
+ ,12 + are positive constants [39]TheoremTheorem
.and Forprooving 1 , 12isare
3 1 is 1 proved.
proved. 2positive ∎
∎that andphase smaller than one constant. 3 1 2
= =�1In −+these 11  
111 (1inputs )of +12 1 2 2(
2 11) = 0 is The Hurwitz
aim of this andpaper
healthy 1 ,bone was
The asaim tothe well show ofreaching as this eliminatepaper OB and was the(secondOCto show
CCs ofpart),
cells 5.the ofcancerous
Simulation
that cancerous
OB5.and OC
bone.bone cells The track of
simul the
ca
1 control Theorem
 − 2  1is , proved. are − positive 2  constants 2 are positive and smaller than one constant. Simulation
2
ved. ∎   1 ∎  5.
5. Simulation
Simulation

( (11)) 12 = = � 1� 2++1121 11 ((
 21 ) +    12 ( ( 22 healthy ) (11) bone as well as
software.
healthy 5.
eliminate Simulation
The
bone control the
as wellCCs parameters asof the
eliminate cancerous
are selectedthe CCsasbone. follows: of (11) Thethe simulation
cancerous was
bone. conTh
) +   ) 5. 5.Simulation 5. 5.Simulation
Simulation
1212
Proof: Ithas 31 been shown
2 22 2 Simulation The aim (11)ofThe thisaim paper waspaper to show that OB and OC 1 , 5.
cells
2Simulation
of cancerous b
22that
1 sliding surfaces Equation have finite-time stability, provided that OB
1 12
32 software. The control parameters The are aim selected of
of this
this aspaper follows: was
was to
to show
show that
that OB 5. and
and SimulationOC
OC cells
cells of
of ca
ca

2222( (22))
are chosen
32 = � 2� 3Proof:
�3 +The
++these
In
Thesoaim
2131
aim

It has
of
21
control
polynomial 5.
of(this
((
this
2 )3 +
been
The
Simulation
)inputs
+
paper
paper
32
shown
aim 
of was
22 of 
 2 this
2was 1 + that
, (

to to
2(
2
2
)sliding
paperare
show
show
3 ) positive
+ 1that was surfaces
that= 0OB to
OB show
healthy
constants
isand (11)
(11)
Equation
Hurwitz
and that
OC bone5.
software.
and
healthy
OChealthy cells
1cells
OB [39] Simulation
(11)
= .0.02,as
 and
of
1 well
,
For The
have

bone
ofbone
The
OC
The
cancerous
The
The
The
prooving
cancerous
2
2 =
as
are aim
5.
aim
aim
control
finite-time
cells
aim aim
as0.0001,
as
eliminate
positive
Simulation well
well
of
ofbone
of of of
of this
bonethe
this
this
parameters
cancerous
this
this
as
as andpaper
stability,
the
paper
paper
track
1reaching
eliminate
track
paper
eliminate
= 0.01,
paper
CCs
smaller the was
was
was
are
bone
was
was
thephase of
OB
OB
to
the
to
the
selected
provided
than
to to show
(11)
track
the
to=follows:
show
and show
CCs
and
show
CCs
0.01,
show
one OC
(second that
OC the
cancerous as
that constant.
that
that
 of
that
that
of=cells
follows:

OB
theOB
cells
the1
OB
0.9,
OB
OB OB,
part), 
and
ofand
bone.
of
2 and
and and
cancerous
3 = OC
cancerous
and
OC OC OCOC
0.5 cells
OC
cells
cells
The cells
cells
cells
bone.
bone.
of
simulat of
of
of of
of Th
ca
Th
ca
ca
cc
232
((33)) are
healthy
3 = chosen bone
so 
31healthy
as polynomial31
well 3 ) + of
bone as 32 
as
eliminate well
 + 32 (
as

the 3)eliminate +
CCs  of = 0 the
the is
software. CCs Hurwitz
cancerous of Thehealthy
the
healthy
software. healthy
[39]
bone. .
controlFor
cancerous bonebone
The bone
The prooving parameters as as bone.
as
simulation
control wellwell well the The as are
as
parameters as eliminate
reaching
was simulation
selected
eliminate
eliminate conducted are
2
phase
asthe
thethewas
selected CCs(second
CCsCCs
in conducted of
MATLAB
as of of the part),
the the
follows: cancerous
in
cancerous
cancerous MATLAB bone.
bone.
bone. Th
T T
of this paper was healthy
where to show ci1bone , cthat are asOB well
positive andasOC eliminatecells
control Theofaim the
2
of
cancerous
parameters CCs thisofpaper
1 the
boneand 2cancerous was , toas
αi10.0001,
track the healthy
show
software.
healthy OB bone.1that and =bone
bone The OB OC simulation
as
and
control
as cells well
well OC =ofpaper as
cells
parameters
as was
eliminate
eliminateof= =conducted
cancerous are the
selected
the CCs
CCs in
bone MATLAB
of
as
of the
track
follows:
the cancerous
the0.9,
cancerous OB3and =bone.bone. OC cel Th
Th
i2software. The control parameters 1 = 0.02, are selected = software.
follows: =
The 0.02, aim 0.01,
The 2of =this
control 0.0001,
2 0.01,
parameters =1  was 0.9,
0.01,are
to selected
3 2=
show = 0.5 0.01,
that as , =
follows:
OB and 0.5
software.
software. Proof:
The
Theare control It
control has been
parameters
parameters
healthy shown bone are that
areasThe selected sliding
selected as surfaces
follows:
aseliminate
follows: Equation software.
software.
1 software. (11) have
Figure TheThe The finite-time
control
1 control
control shows parameters
the
parameters stability,
parameters
1
curves of provided
are
are
the
aresimulation selected
selected
OC
selected cells, that as
aswas asand follows:
follows:
follows: 
Figure 2 shows the OB
well as the CCs ofconstants the cancerous bone. The
well as eliminate the CCs 1 conducted in MAT
where 1 , 2of the positive cancerous control bone. parameters simulationand  , was 2 are conducted
positive
software. in The MATLAB control 1as=  parameters andare selected as follows:
 ∈ (0,1). 2
are chosen so polynomial of  211+=2 +and parameters
1= 01is1are =
Hurwitz 0.02, bones.
OC
 [39] = =
. 0.02,
cells 0.0001,
For
Figure of prooving
 3cancerous = 1
shows 0.0001, =the 0.01,
the 2 reaching=
curve
 bone 2= of = 0.01,
the
track
0.01, phase CCs  (second
the =0.01,
for
= OB 0.9,
cancerous 
and 3 =
part), = OC0.5
bone.
0.9,  As = 0.5 as F
well
trol parameters
rametersand
ameters and1α are
where
,are
1,i2
selected
2 are
positive
are 1 , 
positive
positive2 as
are follows:
positive
constants
constants
constants software. control asas==0.0001, The
parameters control and and  1 ,
NFigure
∈∈ (0,1). (1,
(0,1).
2 2).arepositive
1 0.01,
shows selected
= 1the
1 constants
2
== as
0.02,
0.02,curves follows:
Figure  as
32 2 of == 10.5 0.0001,
the shows
0.0001, =OC
 and
cells,
the 1
1 2− =
curves
==  and ∈
0.01,
0.01, (0,1).
Figure 
ofthe 2
the = OC 0.01,
20.01,
=initial shows cells, 
the =
==and 0.9,
OB
0.9, Figure 3 = 0.5
cells
3 = of
20.5
0.5 the h
show
112= 0.02,
= 0.02, 22 = 0.0001, 11 = 2   = = 0.0001,0.02,    0.01,
0.01,2− =  bones. = = 0.01,0.01,   = =0.9,  designed 
1 0.5 === 0.9,
0.02,control
0.02,  
2 = = =
inputs. 2 0.0001, 0.0001, In this 
1 simulation, = 0.01,
0.01,
2

2 = = 0.01,
0.01, (19) conditions
(19) = 0.9,
0.9, of (19)
healthy

3 = = 0.5 bone
2 = 0.01,  =3 0.9, 3cells = =0.5
1 2 1 2 0.02,
2 2−
2 Figure shows 1
bones.
3
the
11=
the of
0.02,
curve
cancerousness healthy
Figure of 222=
3=
the 0.0001,
bone
0.0001,
shows
happens, CCs 2−
as
for
the the
well
 1 11= = as
cancerous
curve
distances
0.01,
0.01, ofeliminate bone.

the
of 222=
the
= 0.01,
CCs OC
0.01, As the
forand well 
CCs

cancerous
CCs
=as0.9,
= 0.9,
of Figure
cells are
333=
bone.= 40.5
more
0.5 illustr
As than w
Figure 1 shows the curves of curves the OCof cells, and cells, Figure 2 shows the OB
1 2 1 2 3
0.0001, 1 = 0.01,  =  =+ +  = 0.9,= 0.02,
0.01, 3 = 0.5 2 =designed 0.0001, control 1 = 0.01, inputs. 2 In =OB 0.01,
this Figure
Figure
simulation, is = (19) 0.9, 1
11inputs. shows
shows the =In the
0.5
the
initial curves conditions of the
the OC
OC
of healthy cells, and
and
bone Figure
Figure
are  2
22, show(19
show , 
designed
the
the cells control
Figure less than, shows 3 so the
the this curves
initial simulation, of
conditions the the
OCof initial
cells,
cancerous conditions
and bone Figure are of
selected healt
show a
2cancerous bone. The simulation was conducted
 = 2 1
Figure  Figure 1 shows the curves of bones. the OC Figure cells, and
23the shows Figure the 2
1curve shows ofthe thethe OB CCs cells for of the
cancerous healthy bone. and
andAs cancerous
well Fig
2as
0 0
 = Figure 
 − 1 −
 1 shows the
shows 1  the 1curves
 − of
curves of the OC
thecancerousness
 2 cells,
OC cells, and happens,
and Figure Figure bones.
bones. Figure
shows
shows
Figure Figure
Figure Figure the
the 3
33 1 1OB shows
OB
shows1shows shows shows cells
cells the
the the
of the
of the
curve
curve the curves
curves curves healthy
of
healthy
of of ofofthe
the
the thethe
and
CCs
and
CCs OCOC OCcancerous
for
cancerous
for
(12) cells,
cells,
cells, cancerous
cancerous and
and Figure
FigureFigure
bone.
bone. 22As show
sho
show
As w
w
bones. Figure 1 3 shows 2the
the 2curve of the CCs for thebones. cancerous cancerousness FigureFigure
distances bone. shows shows
happens, As of the well the the
OC the
curve
as curves
andFigure CCs
distances
of of the 4 the
cells of
CCs OC
illustrates are
the for cells,
more
OC and
cancerous
the andthan
curvesCCsFigurea healthy
cells
bone.of 2
the show
are
As bon mw
bones.  =
 Figure  −  −
33 shows  1 the thecurve  − of 2 the  CCs   for designed control
cancerous bones. bones. inputs.
Figure
Figure In 33inputs. thisshows
Figureshows simulation, the
4the curve
curve the ofofinitial CCs conditions
CCs forfor cancerous
cancerousof healthy bone.
bone. bone As
Asw aw
shows 22  curves
22 the
bones. � � of =the Figure −OC

cells, shows
1 and Figure curve

Figure 22shows of the1the CCs
shows
the OB for
the (12)
cells cancerous
curves of the of bone. designed
in bone.
bones.
the
healthy
designed
bones. MATLABOC As AsFigure
cells,
and
Figure well
control
well
control isand
cancerous as3software.
3as Figure
shows
inputs.
shows In
the
In
the so42of
illustrates
this
The illustrates
curve
shows
this
curve simulation,
control
simulation, of
ofthe the
the
the
(12)
the
OB curves
CCs the
parameters
curves
cells
the for initial
of
initialofof the the
canceroustheconditions
are
healthy
conditions bone. and
0 , of
of ,healt
As
cance
healt
designed
designed   = −control 
1designed
control 

inputs.
1
inputs. 
 control
−
−
In
 In
2 
this

this inputs.
2
simulation,
simulation, 
In
− 
− this OB
 the
the
 cells
2simulation,
initial
initial theis less (12)
(12)
conditionsthethan,
cancerousness
conditions
the
designed
initial
the designed designed
soOB
of of
the cells
conditions
healthy
cancerousness healthy
initial
control
happens, control control bonebone
conditions
less of inputs.
the
inputs.
inputs.
happens,
than,
healthy
are are distances In
InIn , ,
the
this
bone
this
the
 
cancerous
this , , 
initial
of simulation,
 are the
simulation,
simulation,
distances = =  OC
(10,, CCs
bone
conditions
(10,
0 of and
0
the
,5,
5, the  are
the
for
1).
1).
0
of
initial
CCs =
initial
OC
cancerous
selected
initial
Since
Since
cancerous
(10,
cells
and conditions5,
are
conditions
conditions
CCs
as1). bone.
bone
more
cellsSince 0than
ofofof
As
are
0w
are
healt
heal
hea m
sa
hows the curveof  the CCs for cancerous  1  bones.bone. 2 Figure As 3well
 shows 2 as Figure the curve
 of the
4 illustrates designed
the
selected
designed CCs theforcurves
cancerousness as control
control cancerous follows: inputs.
ofinputs. happens, thebone. In0this
In 0 this
As
the 0 0 well simulation,
0 as Figure
distances
simulation,
0 of the
the the initial
4OC
initial illustratesand conditions
conditionsCCs thecells ofare
curves
of healt
healt m o
  −−22 the the cancerousness happens, the distances the of
OB the cells the
OC the theOB is cancerousness
andless
cancerousnessCCs
cancerousness than, cells so the happens, are
happens, happens, more
initial the
than
conditions
the
the distances
a
distanceshealthy
distances of of bone,
cancerous
of the
ofdistancethe the OC and
OC and
bonethe CCs
distance
are cells
selected ofare as m
nputs. In this simulation, the cancerousness cancerousness
Inthe initial happens,
happens, designed
conditions the
thecontrol distances
of
distances healthy inputs. of ofbone thethe OC
InareOC this andand
 CCs
simulation,CCsthe
0 , 0 , and thecells
0 =
the cells OB are
are
cancerousness
the (10,
cancerousness cells more
initial
,cells
more
5,are is
is
1). than
lessthan
conditions
less Since happens,
happens, than,
than, aa healthy healthy so
of the
so the
healthy
the
the bone,
bone,initial
initialdistances
distances and
and
bone the
conditions
the
conditions of
are thedistance
0OC , OC
of
of and
,of
0cancerous
andandCCs
of
cancerous
 CCs
=CCs cells
bone
cells
bone
(10, cells 5,are arem
are
are
1). ssm
sSs
the
the OB
OB cells In
cells these thesethe
isis less control
OB
control
less than, cells
than, inputsinputs
so is the
so less
the initial
, than,
initial
1  , are so are
2conditions
positive
conditions thepositiveinitial of
constants
of
constants
conditions
cancerous
cancerous and 
the the of
bone the
bone , 
OB 
cancerous
OBOB 1are cells
are

cells
cells 2selected
positive selected
isisbone is less
positive
less
less as
and as
than,
are than,than, 
smaller

and
selected,so
,so
 so

the
, smaller
the
,the
than

initialas
initial
initial
=one
=
than
(40,
(40,
conditions
01,
constant. ,of
one
,conditions
conditions
1,5). 5).
the
0 constant. =OC of(40,
of of
and 1,0 CCs
cancerous
cancerous
cancerous 5). cells bone
bone
bone
are
are
are
are
m
happens,
2are arepositive the distances
positiveconstants
constants and of the
and 1,, 2 are OC and the
are positive CCs cancerousness
cells
positive and
1 are
2 more
smaller happens, than than a the
healthy
one distances
constant. bone, the
the 1 of andOB
OB the
2 cells
the
cellsOC distance is
and
is less
less CCs than,
than, of cells 0 so
0so are 0 the
0the more 0 initial
initial
0
0conditions
than
conditions a healthy of
of cancerous
bone,
cancerous and bone
the
bone are
distan
are ss
s than, so the initial conditions of cancerous bone are selected as 0 0 0
2 1 2 the OB cells and is smaller
less than, thanso one
the  ,  constant.
initial ,  38 conditions = (40, 1, of 5). cancerous bone are selected as  ,  ,  = (40, 1, 5).
Proof:ItIthas
Proof: hasbeen beenshown shown thatthat sliding sliding surfaces surfaces Equation Equation (11) have (11) have finite-time finite-time stability, stability,
provided provided
that 1 , that
0 0
2 1 , 2
0

hat sliding
at sliding surfaces surfaces are
are Equation Equation
chosen soso(11)
chosen (11)
polynomial polynomialhave
have finite-time finite-time
ofof + +22 2 stability, provided that  [39]
2++
stability, 1 =
provided
10=is0 Hurwitz is that Hurwitz , 
11 , .22 For. prooving
[39]
For prooving the reaching the reaching phase (second phase (second part), part),
2++1==00 isis Hurwitz Hurwitz [39] [39]
. . For For prooving prooving the the reaching reaching phase
phase (second
(second part),
part),
2 1
 3 1 2
∎ 5. Simulation
The aim of this paper was to show that OB and OC cells of cancerous bone track the OB and OC cells of
5. Simulation
healthy bone as well as eliminate the CCs of the cancerous bone. The simulation was conducted in MATLAB
s paper was to software.
show thatThe
OB control
and OC Journal of Electronic
cells of cancerous &bone
Information
theSystems | Volume 05 | of
Issue 02 | October 2023
parameters are selected astrack
follows: OB and OC cells
s eliminate the CCs of the cancerous bone. The simulation was conducted in MATLAB
arameters are selected as follows:
 = 0.02,  = 0.0001,  = 0.01,  = 0.01,  = 0.9,  the = curves
0.5 of the designed control inputs. In(19)
this sim-
1 2 1 2 3
(19) ulation, the initial conditions of healthy bone are (u0,
1, 1 = 0.01, 2 = 0.01,  = 0.9,
Figure 3 = the
1 shows 0.5 curves
 of the OC cells, and Figure 2 shows (19)
v0, ω0) = (10,the 5, OB cells of the healthy and cancerous
1). Since the cancerousness happens,
bones. Figure 3 shows the curve of the CCs for cancerous bone. As well as Figure 4 illustrates the curves of the
theFigure
ws the curves of designed 1 shows
OC cells, and the curves
Figure 2 shows ofthethe
OBOC cells,
cells of the
control inputs. In this simulation, the initial conditions
and
healthy and
the of cancerous
distances of theareOC0and
healthy bone , 0 ,CCs
0 =cells
(10,are more
5, 1). than
Since
the curve of the theCCs
Figure 2forshows
cancerous
the bone.
OB As of
cells well Figure
theashealthy 4OC
of the and illustrates
can-CCsthe curves of thethan a healthy bone, and the distance of
cancerousness happens, the distances and a healthy bone, and the distance of the OB cells is
cells are more
. In this simulation,
the OBthe cells
initial
is conditions ofthe
less than,3so healthy
initialbone are 0of
conditions , cancerous
0 , 0 = (10,bone 5,are Since as  ,  ,  = (40, 1, 5).
1).selected
cerous bones. Figure shows the curve of
ens, the distances of the OC and CCs cells are more than a healthy bone, and the CCs less
thethan, so the
distance of initial conditions of cancerous bone
0 0 0

, so the initial for cancerous

conditions bone. As
of cancerous well
bone are as Figure
selected as 40illustrates
, 0 , 0 = (40, are1,selected
5). as (u0, v0, ω0) = (40, 1, 5).
40
Cancerous bone Healthy bone

35

30

25

20
Density

15

10

-5
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time [day]

Figure 1. The curves of the OC cells of cancerous and healthy bone.

800
Cancerous bone Healthy bone

700

600

500
Density

400

300

200

100

0
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time [day]

Figure 2. The curves of the OB cells of cancerous and healthy bone.

39
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

5
Cancer cells
4.5

3.5

3
Density

2.5

1.5

0.5

0
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time [day]

Figure 3. The curves of the CCs of cancerous bone.

0
Amplitude

-2

-4

0 200 400 600 800 1000 1200 1400 1600 1800 2000

-100
Amplitude

-200

0 200 400 600 800 1000 1200 1400 1600 1800 2000

-0.05
Amplitude

-0.1

0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time [day]

Figure 4. The curves of the control inputs.

40
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
6. Discussion
As the figures are precise, after about 300 days
(almost one year), the OC cells of cancerous bone
have tracked the OC cells of healthy bone until the
CCs have disappeared. In the OB cells, because the
initial conditions of cancerous bone and healthy
bone are close and the amplitude of the figure is big,
the result is not clear correctly. Figure 5 shows the
curve of the OB cells in 30 days (zoomed in). The
period of OS treatment is almost five years in the
600
Cancerous bone
500
400
Density
300
200
100
0
0 5 10
Time [day]
Figure 5. The curve of the OB cells of cancerous and healthy bone in 30 days.
7. Conclusions
In this paper, the NTSMC control method is em-
ployed to synchronize two human body bones. One
of these bones was cancerous bone, and the other
bone was healthy. This paper was a theoretical study
that controlled and treated bone cancer with a theo-
retical method. Three designed control inputs have
the features of chattering-free, finite-time stability
41
real world. This is the reason for selecting the final
time of the simulation as 2000 days. The control in-
puts are smooth. They are possible to implement in
real tests. The smoothness happened because of the
chattering-free design.
In brief, all the required features of a controller
for controlling bone cancer have been considered
in designing the proposed controller. The controller
is robust, chatter-free, accurate, smooth, fast, and
implementable. The results presented in Figures 1-5
show the power of the proposed controller.
Healthy bone
15 20 25 30
and robustness against unknowns and uncertainties,
which can be used in practical tests. These control
inputs can be the effects of doses of medicines or the
power of X-rays. After about a year, it was shown
that the CCs had disappeared, and the cancerous
bone looked like symmetrical healthy bone. For the
subsequent studies, it is suggested to work on im-
plementing these types of studies in real tests for the
treatment of some animals’ bone cancer.
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
Authors’ Contributions
Ali Soltani Sharif Abadi: conceptualization,
methodology, software, validation, formal analysis,
investigation, resources, data curation, writing, re-
view and editing, visualization, and funding.
Mansour Rafeeyan: conceptualization, meth-
odology, validation, formal analysis, investigation,
resources, data curation, writing, review and editing,
visualization, supervision.
Vahid Abootalebi: conceptualization, method-
ology, validation, formal analysis, investigation,
resources, data curation, writing, review and editing,
visualization, supervision.
Conflict of Interest
There is no conflict of interest.
Data Availability Statement
Not applicable.
Funding
Ali Soltani Sharif Abadi acknowledges support
from Warsaw University of Technology (WUT),
grant No: 504440200003.
Acknowledgement
Ali Soltani Sharif Abadi acknowledges support
from the WUT.
References
[1] Jerez, S., Camacho, A., 2018. Bone metastasis
modeling based on the interactions between the
BMU and tumor cells. Journal of Computational
and Applied Mathematics. 330, 866-876.
DOI: https://doi.org/10.1016/j.cam.2016.12.026
[2] Buenzli, P.R., Pivonka, P., Gardiner, B.S., et al.,
2012. Modelling the anabolic response of bone
using a cell population model. Journal of Theo-
retical Biology. 307, 42-52.
DOI: https://doi.org/10.1016/j.jtbi.2012.04.019
42
[3] Han, Y., Kang, Y., 2022. Bone niche and bone
metastases. Bone sarcomas and bone metas-
tases-from bench to bedside. Academic Press:
Cambridge. pp. 107-119.
[4] Ottaviani, G., Jaffe, N., 2010. The epidemiology
of osteosarcoma. Pediatric and adolescent osteo-
sarcoma. Springer: Berlin. pp. 3-13.
[5] Pivonka, P., Zimak, J., Smith, D.W., et al., 2008.
Model structure and control of bone remodeling:
A theoretical study. Bone. 43(2), 249-263.
DOI: https://doi.org/10.1016/j.bone.2008.03.025
[6] Chen-Charpentier, B.M., Diakite, I., 2016. A
mathematical model of bone remodeling with
delays. Journal of Computational and Applied
Mathematics. 291, 76-84.
DOI: https://doi.org/10.1016/j.cam.2014.11.025
[7] Abadi, A.S.S., Hosseinabadi, P.A., Mekhilef, S.,
et al., 2020. Chattering‐free fixed‐time sliding
mode control for bilateral teleoperation under
unknown time-varying delay via disturbance
and state observers. Advanced Control for Ap-
plications: Engineering and Industrial Systems.
2(4), e52.
DOI: https://doi.org/10.1002/adc2.52
[8] D’Amico, A.A., Colavolpe, G., Foggi, T., et
al., 2022. Timing synchronization and channel
estimation in free-space optical OOK communi-
cation systems. IEEE Transactions on Commu-
nications. 70(3), 1901-1912.
DOI: https://doi.org/10.1109/TCOMM.2022.
3142134
[9] Mohadeszadeh, M., Pariz, N., 2022. An appli-
cation of adaptive synchronization of uncertain
chaotic system in secure communication sys-
tems. International Journal of Modelling and
Simulation. 42(1), 143-152.
DOI: https://doi.org/10.1080/02286203.2020.18
48281
[10] Chantawat, C., Botmart, T., 2022. Finite-time
H∞ synchronization control for coronary artery
chaos system with input and state time-varying
delays. Plos One. 17(4), e0266706.
DOI: https://doi.org/10.1371/journal.pone.0266706
[11] Izadbakhsh, A., Gholizade‐Narm, H., Deylami,
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
A., 2022. Observer‐based adaptive controller
design for chaos synchronization using Bern-
stein-type operators. International Journal of
Robust and Nonlinear Control. 32(7), 4318-
4335.
DOI: https://doi.org/10.1002/rnc.6026
[12] Khan, A., Khan, T., Chaudhary, H., 2022. Chaos
controllability in chemical reactor system via
active controlled hybrid projective synchroni-
zation method. AIP Conference Proceedings.
2435(1).
DOI: https://doi.org/10.1063/5.0084689
[13] Vaidyanathan, S., 2015. Adaptive synchroniza-
tion of novel 3-D chemical chaotic reactor sys-
tems. Parameters. 1, 4.
[14] Feng, Y., Yu, X., Han, F., 2013. On nonsingular
terminal sliding-mode control of nonlinear sys-
tems. Automatica. 49(6), 1715-1722.
DOI: https://doi.org/10.1016/j.automatica.2013.01.051
[15] Feng, Y., Yu, X., Man, Z., 2002. Non-singular
terminal sliding mode control of rigid manipula-
tors. Automatica. 38(12), 2159-2167.
DOI: https://doi.org/10.1016/S0005-1098(02)00147-4
[16] Soltani Sharif Abadi, A., Ordys, A., Kukielka,
K., et al., 2023. Review on challenges for robot-
ic eye surgery; surgical systems, technologies,
cost-effectiveness, and controllers. The Interna-
tional Journal of Medical Robotics and Comput-
er Assisted Surgery. e2524.
DOI: https://doi.org/10.1002/rcs.2524
[17] Cruz-Ortiz, D., Chairez, I., Poznyak, A., 2022.
Non-singular terminal sliding-mode control for
a manipulator robot using a barrier Lyapunov
function. ISA Transactions. 121, 268-283.
DOI: https://doi.org/10.1016/j.isatra.2021.04.001
[18] Alattas, K.A., Vu, M.T., Mofid, O., et al., 2022.
Adaptive nonsingular terminal sliding mode
control for performance improvement of per-
turbed nonlinear systems. Mathematics. 10(7),
1064.
DOI: https://doi.org/10.3390/math10071064
[19] Wang, S., Li, S., Su, J., et al., 2023. Extended
state observer-based nonsingular terminal slid-
ing mode controller for a DC-DC buck convert-
43
er with disturbances: Theoretical analysis and
experimental verification. International Journal
of Control. 96(7), 1661-1671.
DOI: https://doi.org/10.1080/00207179.2022.20
63192
[20] Alattas, K.A., Mofid, O., Alanazi, A.K., et al.,
2022. Barrier function adaptive nonsingular
terminal sliding mode control approach for
quad-rotor unmanned aerial vehicles. Sensors.
22(3), 909.
DOI: https://doi.org/10.3390/s22030909
[21] Guo, L., Liu, W., Li, L., et al., 2022. Neural net-
work non-singular terminal sliding mode control
for target tracking of underactuated underwater
robots with prescribed performance. Journal of
Marine Science and Engineering. 10(2), 252.
DOI: https://doi.org/10.3390/jmse10020252
[22] Islam, Y., Ahmad, I., Zubair, M., et al., 2022.
Adaptive terminal and supertwisting sliding
mode controllers for acute Leukemia therapy.
Biomedical Signal Processing and Control. 71,
103121.
DOI: https://doi.org/10.1016/j.bspc.2021.103121
[23] Khalili, P., Vatankhah, R., Taghvaei, S., 2018.
Optimal sliding mode control of drug delivery
in cancerous tumour chemotherapy considering
the obesity effects. IET Systems Biology. 12(4),
185-189.
DOI: https://doi.org/10.1049/iet-syb.2017.0094
[24] Sarhaddi, M., Yaghoobi, M., 2020. A new ap-
proach in cancer treatment regimen using adap-
tive fuzzy back-stepping sliding mode control
and tumor-immunity fractional order model.
Biocybernetics and Biomedical Engineering.
40(4), 1654-1665.
DOI: https://doi.org/10.1016/j.bbe.2020.09.003
[25] Dey, B.S., Bera, M.K., & Roy, B.K. (editors),
2018. Super twisting sliding mode control of
cancer chemotherapy. 2018 15th International
Workshop on Variable Structure Systems (VSS);
2018 Jul 9-11; Graz, Austria. New York: IEEE.
p. 343-348.
DOI: https://doi.org/10.1109/VSS.2018.8460228
[26] Shahri, A.P., Haghighatnia, S., Moghaddam,
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
R.K., et al., 2017. Control the tumour growth
via sliding mode control. International Journal
of Medical Engineering and Informatics. 9(2),
101-109.
DOI: https://doi.org/10.1504/IJMEI.2017.083093
[27] Doruk, R.Ö., 2020. Angiogenic inhibition thera-
py, a sliding mode control adventure. Computer
Methods and Programs in Biomedicine. 190,
105358.
DOI: https://doi.org/10.1016/j.cmpb.2020.105358
[28] Alyoussef, F., Kaya, I., 2023. Improved adaptive
dynamic non-singular terminal sliding mode
controller with fractional disturbance observer.
Information Sciences. 641, 119110.
DOI: https://doi.org/10.1016/j.ins.2023.119110
[29] Alyoussef, F., Kaya, I., 2023. A new dynamic
sliding mode controller with disturbance observ-
er for controlling integrating processes with time
delay. International Journal of Control. 1-21.
DOI: https://doi.org/10.1080/00207179.2023.2201649
[30] Cai, G., Ding, Y., Chen, Q., 2019. SMC Chaos
control of a novel hyperchaotic finance system
using a new chatter free sliding mode control.
Journal of Physics: Conference Series. 1187(3),
032103.
DOI: https://doi.org/10.1088/1742-6596/1187/
3/032103
[31] Adamiak, K., 2020. Chattering-free reference
sliding variable-based SMC. Mathematical
Problems in Engineering. 3454090.
DOI: https://doi.org/10.1155/2020/3454090
[32] Abadi, A.S.S., 2023. A novel control system for
synchronizing chaotic systems in the presence of
communication channel time delay; case study
of Genesio-Tesi and Coullet systems. Nonlinear
Analysis: Hybrid Systems. 50, 101408.
DOI: https://doi.org/10.1016/j.nahs.2023.101408
[33] Abadi, A.S.S., Ordys, A., Pierscionek, B. (edi-
tors), 2023. Controlling a teleoperated robotic
eye surgical system under a communication
channel’s unknown time delay. 2023 27th Inter-
national Conference on Methods and Models in
44
Automation and Robotics (MMAR); 2023 Aug
22-25; Międzyzdroje, Poland. New York: IEEE.
p. 211-215.
DOI: https://doi.org/10.1109/MMAR58394.
2023.10242556
[34] Alinaghi Hosseinabadi, P., Soltani Sharif Abadi,
A., Mekhilef, S., et al., 2020. Chattering-free
trajectory tracking robust predefined-time slid-
ing mode control for a remotely operated vehi-
cle. Journal of Control, Automation and Electri-
cal Systems. 31(5), 1177-1195.
DOI: https://doi.org/10.1007/s40313-020-00599-4
[35] Abadi, A.S.S., Ordys, A., Pierscionek, B., 2023.
Novel off-line self-tuning controller with guar-
anteed stability. International Journal of Auto-
motive Technology. 24(3), 851-862.
DOI: https://doi.org/10.1007/s12239-023-0069-7
[36] Abadi, A.S.S., Hosseinabadi, P.A., Mekhilef,
S., 2018. Two novel approaches of NTSMC and
ANTSMC synchronization for smart grid chaot-
ic systems. Technology and Economics of Smart
Grids and Sustainable Energy. 3, 1-14.
DOI: https://doi.org/10.1007/s40866-018-0050-0
[37] Abadi, A.S.S., Hosseinabadi, P.A., Mekhilef, S.,
2020. Fuzzy adaptive fixed-time sliding mode
control with state observer for a class of high-or-
der mismatched uncertain systems. International
Journal of Control, Automation and Systems.
18, 2492-2508.
DOI: https://doi.org/10.1007/s12555-019-0650-z
[38] Komarova, S.V., Smith, R.J., Dixon, S.J., et al.,
2003. Mathematical model predicts a critical
role for osteoclast autocrine regulation in the
control of bone remodeling. Bone. 33(2), 206-
215.
DOI: https://doi.org/10.1016/S8756-3282(03)00157-1
[39] Feng, Y., Han, F., Yu, X., 2014. Chattering free
full-order sliding-mode control. Automatica.
50(4), 1310-1314.
DOI: https://doi.org/10.1016/j.automatica.2014.
01.004
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

Journal of Electronic & Information Systems

https://journals.bilpubgroup.com/index.php/jeis

ARTICLE

Enhancing Semantic Segmentation through Reinforced Active

Learning: Combating Dataset Imbalances and Bolstering Annotation
Efficiency
Dong Han, Huong Pham, Samuel Cheng*

School of Electrical and Computer Engineering, University of Oklahoma, Tulsa, OK, 74135, USA

ABSTRACT
This research addresses the challenges of training large semantic segmentation models for image analysis,
focusing on expediting the annotation process and mitigating imbalanced datasets. In the context of imbalanced
datasets, biases related to age and gender in clinical contexts and skewed representation in natural images can affect
model performance. Strategies to mitigate these biases are explored to enhance efficiency and accuracy in semantic
segmentation analysis. An in-depth exploration of various reinforced active learning methodologies for image
segmentation is conducted, optimizing precision and efficiency across diverse domains. The proposed framework
integrates Dueling Deep Q-Networks (DQN), Prioritized Experience Replay, Noisy Networks, and Emphasizing
Recent Experience. Extensive experimentation and evaluation of diverse datasets reveal both improvements and
limitations associated with various approaches in terms of overall accuracy and efficiency. This research contributes
to the expansion of reinforced active learning methodologies for image segmentation, paving the way for more
sophisticated and precise segmentation algorithms across diverse domains. The findings emphasize the need for
a careful balance between exploration and exploitation strategies in reinforcement learning for effective image
segmentation.
Keywords: Semantic segmentation; Active learning; Reinforcement learning

1. Introduction label to each pixel within an image, effectively di-

Semantic segmentation involves assigning a class viding the image into segments that carry semantic

*CORRESPONDING AUTHOR:
Samuel Cheng, School of Electrical and Computer Engineering, University of Oklahoma, Tulsa, OK, 74135, USA; Email: samuel.cheng@ou.edu
ARTICLE INFO
Received: 1 November 2023 | Revised: 25 November 2023 | Accepted: 27 November 2023 | Published Online: 4 December 2023
DOI: https://doi.org/10.30564/jeis.v5i2.6063
CITATION
Han, D., Pham, H., Cheng, S., 2023. Enhancing Semantic Segmentation through Reinforced Active Learning: Combating Dataset Imbalances and
Bolstering Annotation Efficiency. Journal of Electronic & Information Systems. 5(2): 45-60. DOI: https://doi.org/10.30564/jeis.v5i2.6063
COPYRIGHT
Copyright © 2023 by the author(s). Published by Bilingual Publishing Group. This is an open access article under the Creative Commons Attribu-
tion-NonCommercial 4.0 International (CC BY-NC 4.0) License. (https://creativecommons.org/licenses/by-nc/4.0/).

45
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
meaning. Unlike image classification, which assigns
a single class label to the entire image, semantic
segmentation is a more granular task, amounting to
pixel-level classification [1]. Over the past few years,
the computer vision community has heavily relied
on effective deep neural networks (DNNs) designed
for semantic segmentation, as evidenced by recent
research [2-11]. These efficient DNNs are characterized
by their low computational demands and quick in-
ference times [12], and their widespread adoption has
significantly influenced applications in various fields
such as autonomous driving [13,14], semantic segmen-
tation enables precise scene understanding, allowing
the vehicle to identify and differentiate between
various objects on the road, such as pedestrians, ve-
hicles, traffic signs, and obstacles. This technology
aids in real-time decision-making, helping the vehi-
cle navigate complex environments and ensure the
safety of passengers and pedestrians; robot manip-
ulation [15,16], for robots to interact intelligently with
their environment, they require a comprehensive un-
derstanding of the objects and structures in their sur-
roundings. Semantic segmentation facilitates this by
enabling robots to identify and differentiate between
different objects and their corresponding spatial re-
lationships. This capability is crucial for tasks such
as object manipulation, navigation in dynamic envi-
ronments, and human-robot collaboration, enhancing
the overall efficiency and safety of robotic systems;
and biomedical image analysis [17,18], this technology
assists healthcare professionals in diagnosing dis-
eases, monitoring the progression of conditions, and
planning effective treatment strategies. By providing
detailed insights into complex biological structures,
semantic segmentation contributes to advancements
in medical research, patient care, and disease man-
agement. where advanced computer vision systems
are paramount. For these models to operate effec-
tively, however, they typically rely on a substantial
volume of pixel-level annotations, a process that of-
ten necessitates expensive human labor.
Semantic segmentation datasets, with their pix-
el-wise annotations for each image, have been instru-
mental in advancing computer vision tasks. How-
46
ever, they are not without their limitations. Here are
some of the key limitations associated with semantic
segmentation datasets: 1) Extensive pixel-level
annotations. Semantic segmentation models often
require a large volume of accurately labeled train-
ing data, where each pixel in the image is assigned
a corresponding class label. This process demands
meticulous and precise annotations, which can be
time-consuming and resource-intensive. Obtaining
such detailed annotations for diverse datasets can be
challenging, particularly for complex scenes with
numerous objects and intricate boundaries. 2) La-
bor-intensive annotation process. The pixel-wise an-
notation process for semantic segmentation datasets
is a labor-intensive task, often requiring significant
human effort and time [19]. This manual labeling pro-
cess is labor-intensive, time-consuming, and can be
prone to human error, especially when dealing with
large datasets. As a result, the creation of high-qual-
ity annotated datasets requires significant human re-
sources and can be a bottleneck in the development
of accurate and robust semantic segmentation mod-
els. 3) Data imbalance and variability. Semantic seg-
mentation datasets may suffer from data imbalance
and variability in the distribution of classes within
the dataset. Certain classes may be underrepresent-
ed, leading to biased model predictions and reduced
performance in specific classes. Handling such data
imbalance and variability is crucial to ensure that
the model can generalize effectively across different
scenarios and accurately segment diverse objects in
various contexts. 4) Generalization and robustness.
Semantic segmentation models must be capable of
generalizing well to unseen data and diverse en-
vironments. Achieving robust performance across
different lighting conditions, viewpoints, and envi-
ronmental changes remains a significant challenge.
Ensuring that the model can accurately segment
objects in various real-world scenarios is essential
for its practical deployment in applications such as
autonomous driving, robotics, and biomedical image
analysis.
This aspect gains prominence during the process
of collecting annotated data under human supervi-
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
sion for the creation of either a novel dataset or the
supplementation of an existing one. Mitigating the
challenges entails the systematic and efficient selec-
tion of image regions warranting annotation. Active
learning (AL) represents a well-established research
discipline explicitly focused on this area. Its primary
objective is the identification of the most informative
samples for annotation, with the overarching goal of
enhancing the performance of learning algorithms
with a minimized data requirement, in contrast to
a non-selective approach where the entire dataset
undergoes indiscriminate labeling. Active learning
methodologies can be broadly categorized into two
main groups: (i) methodologies that integrate vari-
ous manually crafted active learning strategies [20-22],
and (ii) data-centric active learning approaches [23-25].
Notwithstanding the heightened cost and time asso-
ciated with acquiring labels for semantic segmenta-
tion in comparison to image classification, the realm
of active learning for semantic segmentation has
garnered relatively less attention [26-28], primarily em-
phasizing the development of manually engineered
strategies.
How can reinforced active learning be effective-
ly employed to enhance semantic segmentation,
specifically addressing challenges posed by dataset
imbalances and improving annotation efficiency?
The latest active learning techniques leveraging re-
inforcement learning primarily concentrate on anno-
tating one sample at each step [29-31], progressing until
a predetermined label budget is fulfilled. Inspired
by the AL-RL model by Casanova et al. [32], the pro-
posed approach expedites the annotation process by
selectively choosing informative and representative
images to accelerate model learning. Additionally,
we tackle the issue of imbalanced datasets. For in-
stance, in clinical contexts, biases related to age and
gender can arise due to constraints on the diversity
of medical image contributors. In natural images,
certain categories may be significantly more abun-
dant than others, potentially skewing the model’s
performance towards the most frequently represented
category. We investigate strategies to mitigate these
biases with the aim of enhancing efficiency and ac-
47
curacy in semantic segmentation analysis.
Furthermore, we conduct an in-depth exploration
of various Reinforced Active Learning methodol-
ogies for image segmentation to optimize the pre-
cision and efficiency of segmentation tasks across
diverse domains. To achieve this, we implement a
robust framework that integrates various Reinforce-
ment Learning (RL) techniques, including Dueling
Deep Q-Networks (DQN) [33], Prioritized Experience
Replay [34], Noisy Networks [35], Emphasizing Recent
Experience [36], Soft Update Target Network [37], and
Adaptive Epsilon Greedy [38]. We test the proposed
method in the CamVid [39] dataset. Our results illus-
trate both improvements and limitations associated
with various approaches in terms of overall accuracy
and efficiency in image segmentation tasks.
2. Related work
Active learning serves as a dedicated methodol-
ogy focused on optimizing performance gains with
a minimal number of labeled samples. Its primary
goal is to identify the most informative samples
from the unlabeled dataset, subsequently presented
to an oracle, such as a human annotator, for labe-
ling. This process effectively minimizes labeling
costs while ensuring sustained performance. Active
learning approaches can be classified into member-
ship query synthesis [40,41], stream-based selective
sampling [42,43], and pool-based [44] strategies, each
derived from diverse application scenarios [45]. Cer-
tain methodologies amalgamate various techniques
to enhance the overall performance of active learn-
ing. For example, Shui et al. [46] take into account the
diversity and uncertainty of query samples, and try
to discover a balance between those two approaches.
Further investigation into traditional query strategies
is undertaken [47]. Despite the considerable volume
of existing research on active learning, it continues
to grapple with the challenge of extending its appli-
cability to high-dimensional data, such as images,
text, and videos [48]. Consequently, the majority of
active learning studies tend to focus on low-dimen-
sional problems [49]. Several methodologies integrate
various techniques to enhance the performance of
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
artificial intelligence, such as leveraging the explo-
ration-exploitation trade-off [50], on a bandit formula-
tion [21] and reinforcement learning [51].
In recent times, there has been a growing inter-
est in reinforcement learning as an approach to ac-
quiring a labeling policy that directly optimizes the
performance of the active learning algorithm. For
example, Dhiman et al. [29] proposed an automated
annotation model for Multimedia Streaming Appli-
cations (MAS) to address the existing challenges of
slow speeds and inefficiencies in accessing multi-
media content. By leveraging Multi-modal Active
Learning (MAL) and Convolutional Recurrent
Neural Network (CRNN) in tandem with Deep Rein-
forcement Learning (DRL), the model demonstrates
superior retrieval accuracy and performance metrics.
Gong et al. [52] proposed Meta Agent Teaming Ac-
tive Learning (MATAL) framework that effectively
minimizes the laborious efforts involved in pose an-
notations. Sadigh et al. [53] present an active learning
method for Inverse Reinforcement Learning (IRL)
that relies on human-provided preferences between
two sample trajectories. In a similar vein, Kunapu-
li et al. [54] incorporate human expert information
through preference elicitation for actions in a des-
ignated state. Ezzeddine et al. [55] integrate feedback
from a human trainer, particularly in cases where the
provided demonstrations are less than optimal. Liu
et al. [56] utilize expert knowledge derived from ora-
cle policies to develop a labeling policy. In contrast,
Pang et al. [57] employ policy gradient methods to
acquire knowledge for the function. In an alternative
strategy, certain techniques aggregate all labeled data
in a single comprehensive step. Contardo et al. [58]
employ a bi-directional RNN to select all samples
simultaneously, particularly for the task of one-shot
learning. Meanwhile, Sener et al. [59] suggest choos-
ing a batch of representative samples that maximize
coverage across the entire unlabeled set.
Recent active learning work has also looked at
semantic segmentation [60]. Uncertainty-driven active
learning identifies data samples with elevated alea-
toric uncertainty. Entropy [61], which estimates un-
certainty, serves as a commonly employed baseline
48
in active learning selection. This function calculates
per-pixel entropy for the predicted output and utiliz-
es the averaged entropy as the final score. BALD [62]
frequently serves as a baseline in previous studies.
It is applied in segmentation by integrating dropout
layers into the decoder module of the segmentation
model and subsequently computing pixel-wise mu-
tual information through multiple forwards passes.
Kampffmeyer et al. [63] strive to optimize the average
standard deviation of the predicted probabilities.
Jain et al. [64] integrate metrics, defined by manually
engineered heuristics, to promote the diversity and
representativeness of labeled samples. Certain meth-
odologies leverage unsupervised super pixel-based
over segmentation [65,66], relying heavily on the pre-
cision of the super-pixel segmentation. Others con-
centrate on foreground-background segmentation of
biomedical images [67,68], employing similarly crafted
heuristics. Golestaneh et al. [69] focus on self-con-
sistency that uses simple transformations should not
change the observation in active learning for seman-
tic segmentation. Mackowiak et al. [70] concentrate on
cost-effective strategies, emphasizing that the labe-
ling cost for an image is not uniformly treated across
all images.
The advent of DQN marked a significant mile-
stone; however, numerous constraints associated
with this algorithm have surfaced, leading to the
proposal of various extensions. Double DQN [71] mit-
igates the overestimation bias of Q-learning [72] by
separating the selection and evaluation of the boot-
strap action. Prioritized experience replay [34] enhanc-
es data efficiency by prioritizing more frequent re-
play of informative transitions. The dueling network
architecture [33] aids in action generalization by in-
dependently representing state values and action ad-
vantages. Learning from multi-step bootstrap targets,
as seen in A3C [73], adjusts the bias-variance trade-off
and accelerates the propagation of newly observed
rewards to earlier visited states. Noisy DQN [35]
introduces stochastic network layers to facilitate
exploration. To the best of our knowledge, our work
is the first to examine an agent that integrates all the
aforementioned components to the problem of active
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

learning for semantic segmentation. Emphasizing generate the reward signal, which involves evaluat-
recent experience [36] typically refers to assigning ing the segmentation network’s performance on this
greater importance to recent observations and actions set. Additionally, the set DS (with DS being not larger
when making decisions or updating the learning than DT) is utilized for constructing the representa-
model. ERE is often driven by the recognition that tion of the current state.
the environment is non-stationary, implying that the A Markov Decision Process (MDP) is defined
optimal policy might evolve over time. as a tuple (S, A, r, T, γ) where S stands for a set of
states; A for actions, the composed action, consisting
3. Methods of K sub-actions, relies on the segmentation network,
along with the labeled and unlabeled sets. Each
3.1 Active learning with reinforcement learn- sub-action entails requesting the labeling of a par-
ing for semantic segmentation ticular region; r, S × A → R, for the function based
on improvement in mean IoU per class of taking an
Followed by Casanova et al. [32], we use their ar-
action in a state; T, S × A × S → R, for the state-tran-
chitecture for training segmentation networks. We
sition function; and r, for the discount factor imply-
frame the active learning problem as a Markov de-
ing that a reward obtained in the future is worth a
cision process (MDP). The proposed process entails
smaller amount than an immediate reward. Figure
an iterative active learning strategy for enhancing
1 describes this training workflow. In our approach,
the performance of a segmentation network, denoted
the episode concludes upon reaching the designated
as f and parameterized by θ, within a limited labeled
budget B for labeled regions. Post-episode termi-
sample budget, B. At each iteration, a query network,
nation, we reset the weights of the segmentation
represented by π and parameterized by φ, selective-
network, denoted as f, to the initial weights θ0, and
ly picks K regions from the large unlabeled set, Ut.
initiate a new episode. The training process for the
These regions are then submitted to an oracle for
query policy π involves the simulation of multiple
labeling, subsequently augmenting the labeled set,
episodes, with weight updates occurring at each time
t. The segmentation network f is trained using the
step through the sampling of transitions {(st, at, rt+1,
enriched t, and its performance is evaluated based
st+1)}from the experience replay buffer ɛ.
on the Intersection-over-Union (IoU) metric. This
iterative procedure continues until the designated
budget B is attained. By strategically selecting in-
formative regions for labeling, this process optimizes
the performance of the segmentation network, thus
efficiently leveraging a limited labeled dataset to
achieve superior segmentation results. This data-cen-
tric approach enables the model to acquire selection
strategies purely from past active learning encoun-
ters.
In the setting, we employ four distinct data parti-
tions. For training the query network π, we designate
a portion of labeled data DT, utilizing it for multiple Figure 1. The overall workflow of active learning with rein-
iterations of the active learning process to acquire forcement learning in semantic segmentation.
an effective acquisition function that optimizes per-
formance within a B region budget. The evaluation
3.2 Extensions to DQN
of the query network takes place on a separate data
split DV. Moreover, we utilize a distinct subset DR to The evolution of Deep Q-Networks (DQN) has

49
 Dueling deep Q-Learning
The Dueling DQN algorithm introduced by Wang et al. [33] seeks to improve upon
traditional DQN by decomposing the Q-values into two separate components: the value function,
  , Journal
and theofadvantage
Electronic &function, Information  ,Systems
 . The | Volume
value05function
| Issue 02represents
| October 2023the expected reward for a
given state,  , while the advantage function reflects the relative advantage of taking a particular
given rise to several action, significant
, compared extensions, to other actions. eachBy combining
This modificationthese two allows the network
functions, it is possibleto learn to more
compute
addressing specific the fulllimitations
Q-values for and each enhancing
state-action thepair. efficiently in situations where the exact values of in-
algorithm’s overall performance. To implement this decomposition,
Individually, each dividual the Dueling
actions are DQNnot asalgorithm
important, introduces a neural
as it can focus
network with two separate output layers,
of these algorithms leads to significant performance on learning the value function for the state. one for the value function and one for the advantage
function. These outputs are then combined to produce the final Q-values. This modification
enhancements. Given that they tackle fundamental- Prioritized experience replay
allows the network to learn more efficiently in situations where the exact values [34]
of individual
ly different issues and share a common framework,
actions are not as important, as it can focus on learning the value function for the state.2015 in-
The proposition by Schaul et al. in
there is a plausible opportunity for their integration. troduces a resolution termed prioritized experience
We suggest sixPrioritized extensions, each designed
experience replay to over- replay (PER). This approach involves the utilization
come a specific limitation, contributing by The proposition to anSchaul
overall et al.of[34]aninadded
2015data introduces
structure a resolution
that maintains termed the prioritized
priori-
improvement inexperience performance. replay (PER). This
To maintain approach ty
a manage- of each transition. Subsequently, experiences arethat
involves the utilization of an added data structure
maintains the priority of each transition. Subsequently, experiences are sampled based on their
able selection size, we have chosen extensions that sampled based on their respective priorities.
respective priorities.
address distinct concerns. 
  =  (2) (2)
Double deep Q-Learning  
[32]
Following the prior work , we set double DQN The hyperparameter alpha determines the degree
as our baseline architecture. The hyperparameter
One issue with the alpha DQN determines the degree of sampling bias desired. The priorities
correspond to the temporal difference error of the agent of sampling biasduring
desired. itsThe
mostpriorities
recent trainingcorrespondon that
algorithm is that it tends to overestimate the true re- to the temporal difference error of the agent during
particular experience. This strategy enables agent to emphasize learning from its less accurate
wards, leading predictions, to inflated Q-values. thereby refining To address its weakthis, areas and significantly
its most recent training improving sample efficiency.
on that particular experience.New
[71]
the Double DQN algorithm
transitions introduces ainto
are incorporated modifi-
the replay buffer with the highest priority,
This strategy enables the agent to emphasize learning introducing a bias
cation to the Bellman toward equation recent transitions.
used in DQN. It is Instead
essential to recognize that stochastic transitions
from its less accurate predictions, thereby refining may also receive
of using the same preference,
equation, eventhewhen actionthere is limited
selection and remaining knowledge to be gained from them.
its weak areas and significantly improving sample
Updated on April
action evaluation are decoupled in the following efficiency. New transitions are incorporated into the 30, 2023
Emphasizing recent experience
way: Updated on April 30, 2023
Primarily conceived to expedite replay buffer withspeed
the convergence the highest
of Softpriority,
Actor Critic introducing
(SAC)a [36],
 , ;  =  +this  methodology
' ,   '  can ' , 'conceivably
;  ; ' (1) be extended (1)to arecent
bias toward wide transitions.
array of algorithms
It is essential and to tasks
rec-that
main neural network,  , determines
inherently the best
profit next
from action,
accelerated
θ, determines the '
, while the
learningognizetarget
of recent
that experiences,
stochastic particularly
transitions may those
also involving
receive
+  ' ,   'Here,  ' , the'
;  main
; ' neural network,(1)
evaluate this action and compute
multiple itscomponents.
Q-value. This The simple change has
fundamental been entails, during the parameter update phase,
concept
ermines best next action,  , while
the best '
while the
the target
target network is used preference, even when there is limited remaining
er-estimations and lead to better sampling final policies.
the initial mini batch from the entire dataset within the replay buffer. Subsequently, for
ompute its Q-value. to evaluateThisthis simpleaction change
and computehas been its Q-value. This knowledge to be gained from them.
each subsequent mini batch, the sampling range is gradually narrowed, enabling a more
etter
arning final policies.
simple change pronounced has been shown focus to onreduce
recent data over-esti-
points. This Emphasizing
scheme revolvesrecent experience
around two fundamental aspects: (i)
ng DQN algorithm mationsintroduced
and lead to bybetter
a heightened Wang final etpolicies.
samplingal. frequency
[33]
seeks to for morePrimarily
improve upon
recent data, conceived
and (ii)to aexpedite
systematic the convergence
arrangement of
decomposingDueling the Q-values into
updates
deep Q-Learning two separate
ensuring components:
that older datathe value
does notfunction,
overwrite
speed of Soft the more
Actor recent
Critic ones.
(SAC) [36]
The
, this introduction
methodol- of
ced by
ntage Wang et,al.
function, .
[33]
The seeks
value to improve
function upon
represents the expected reward for a
separateThe
 DuelingExperience DQN algorithm Replay Emphasis
introduced (ERE) establishes
by Wang a straightforward
ogy can conceivably be extended yetto effective
a wide array sampling
of
einto
thetwo advantage components:
function reflects thethe
technique value thatfunction,
relative advantage
enables the agent of taking a particular
to prioritize recent transitions without disregarding previously
[33]
etrepresents
al. seeks to improve upon for traditional
a it isDQN by algorithms and tasks that inherently profit from ac-
dalue function
to other actions. By combining the learned
expected these reward
two functions,
policies. possible to compute
flects
r eachthe relative advantage
decomposing
state-action pair. of Q-values
the taking a particular into two separate com- celerated learning of recent experiences, particularly
ning these two functions,
ponents: the value
ment this decomposition, it is
the possible
Adaptive function,
Dueling toDQN
epsilon compute
V(s), and the advantage
algorithm
greedy introduces those a neural involving multiple components. The funda-
separate output function,
layers, A(s, one a). for The value function
the value function and represents
one forthe mental
the advantage concept entails, during the parameter update
he Dueling
utputs are then DQN algorithm
combined introduces
to forproduce a
thestate,neural
finals, Q-values. This modification
expected reward a given while the advan- phase, sampling the initial mini batch from the entire
kforto the
learn value
more function
tage efficiently
function
and in onesituations
reflects
for the advantage
where
themodification the exact values of individual
relative advantage of taking dataset within the replay buffer. Subsequently, for
o produceasthe
mportant, final
it can Q-values.
focus on learning This the value function for the state. each subsequent mini batch, the sampling range is
n situations where a particular
the exact action, values a, compared
of individual to other actions. By
earning
nce replay combining
the value function these
for thetwostate.functions, it is possible to com- gradually narrowed, enabling a more pronounced
sition by Schaul et al.pute the full[34]Q-values for each state-action pair.
in 2015 introduces a resolution termed prioritized focus on recent data points. This scheme revolves
(PER). This approach To implement
involves this decomposition,
the utilization of an added the Dueling
data structure around thattwo fundamental aspects: (i) a heightened
nity2015 introduces
of each transition. a resolution
Subsequently, termed prioritized
experiences
DQN algorithm introduces a neuralare sampled
network basedsampling
with on their frequency for more recent data, and (ii) a
ves the utilization of an added data structure that
s.
equently, experiences two separate are output
sampled layers,
basedone onfor theirthe value function systematic arrangement of updates ensuring that old-

 
and one for the advantage =  function. These outputs er data(2) does not overwrite the more recent ones. The
  
= 
 are then combined to produce the (2)final Q-values. introduction of Experience Replay Emphasis (ERE)
parameter
  alpha determines the degree of sampling bias desired. The priorities
emporal difference error of the agent during its most recent training 50 on that
ce.he This
degree of sampling
strategy enablesbias the desired.
agent to The priorities
emphasize learning from its less accurate
y the agent its
refining during
weakits mostand
areas recent training on
significantly that
improving sample efficiency. New
agent to emphasize
orporated into the replaylearning bufferfromwith its lessthe accurate
highest priority, introducing a bias
 epsilon is set to 0.3, Thisthe output approach
adaptable action is concludes randomly with chosen a notablyfrom the lowactionexploration spaceprobability,
with a probability  , after
 ← init − end  + end ← init − end  + ←endinit − end(4)  + end (4) end
of 0.3, This and with , thereby
a probability facilitating of 0.7, a transition
the towards
output concludesaction an
is selectedincreased emphasis
greedily on exploitation (i.e., a
lowbased on argmax (Q).

This adaptable approach step
concludes
adaptable with aThis
approach notably adaptable approach
low exploration
concludes with a notably probability, lowwith enda, notably
exploration after probability, exploration
end , afterprobability, end ,
greedier
A refined approach)
version during
of the the latter
epsilon-greedy stages of
methodthe training
is referred process. Despite this shift, a minimal
step , thereby facilitating step a, transition
thereby towards
facilitating
step , thereby
an increased
a transition facilitating emphasis
towards a transition
anonincreased towards
exploitation emphasisan increased
(i.e., aontoexploitation
as emphasis
the Adaptive-epsilon-
on exploitation
(i.e., a (i
greedy exploration
approach [38] probability persists, ensuring the ability to explore even as the policy nears
greedier approach) during greedier the latter
approach) stages of. In
greedier
during thethis training
the approach,
approach)
latter during
process.
stages for instance,
thethelatter
ofDespite trainingthis theshift,
stages policy
process.ofa the isDespite
minimal trainedthis
training over
process. N epochs/episodes,
shift, Despite
a minimal this shift,aa mi
Journal
value convergence.
of Electronic
contingent & Information
upon the specific Systemsproblem. | Volume 05 | Issue 02 the
Initially, | October 2023 sets epsilon to 
exploration probabilityexploration persists, ensuring
probability exploration
the persists,
ability probability
to explore
ensuring persists,
theeven ability ensuring
as the thealgorithm
policy
to explore ability
even to
nears as explore
the policy evennears asinitthe(e.g.,
policy
convergence. init = 0.6 ), gradually
convergence. convergence. reducing it to reach  = end (e.g., end = 0.1 ) over a designated number of
establishes a straightforward Noisy network sampling stochasticity directly into the network’s parameters,
yet effective
training epochs/episodes ( step ). Primarily, during the initial training phase, the model is granted
Noisy networks are often utilized instead of the epsilon-greedy method to promote more
Noisy networkthat enables
technique Noisy
increased
thenetwork
agent exploration
to Noisy
prioritize network
freedom recent withenabling a higher aprobability
more nuanced (e.g., and = 0.6 ), followed
initcontinuous exploration by a gradual
effective and dynamic exploration during training. Unlike the epsilon-greedy approach, which
Noisy networks are
epsilon oftenNoisy utilized
decrease networks instead
at a are
rate Noisy
of the
often
r over networks
epsilon-greedy
utilized
the are
instead
training oftenmethod
of utilized
the
epochs/episodes, to instead
promote
epsilon-greedy of
more
adhering the
[35] epsilon-greedy
method to to
the promote
subsequent method
more to promote
formula:
transitions without disregarding only previously
adjusts the exploration learned process. probability, Thenoisy Noisy networksNetwork introduce introduces
stochasticity a noveldirectly into the
effective and dynamic effective exploration andduring dynamic effective
training.
explorationandUnlikedynamic exploration
the epsilon-greedy
during training.− during
Unlikestepapproach,
thetraining. which
epsilon-greedyUnlike the epsilon-greedy
approach, which approach, w
network’s parameters, enabling  =
concept a more  nuanced
of a noisy , and
0 continuous
linearnetworks layer, exploration process. The Noisy
(3)
policies.
only adjusts the exploration only adjusts probability, only adjusts
noisy
the exploration networks the introduce
probability, exploration noisy probability,
stochasticity
networks noisy
directly
 introduce into the integrating
stochasticity introduce directly bothintode-
stochasticity the directly int
Network [35] introduces a novel concept of a noisy linear layer, integrating both deterministic and
Adaptiveparameters,
network’s epsilon greedy enabling a parameters,
network’s more nuanced network’senablingandparameters,a moreterministic
continuous enabling
exploration
nuanced andaand more noisy
process.
continuousnuanced components.
The and
Noisy
exploration continuous process. exploration
The Noisyprocess. The N
noisy components. aa← initconcept
− enddeterministic
 +a  (4)
Network [35]
introducesNetwork
The epsilon-greedy a novel concept
technique
[35]
ofNetwork
introduces
serves a noisyaasnovel [35]
alinear introduces
means layer, of
concept novel
integrating
noisy both
linear of
layer, noisy
end linear
integrating and both layer, integrating both
deterministic and deterministi
 =  +  + noisy ⊙  + noisy ⊙   (5) (5)
noisy components. noisy components. This adaptable noisy approach
components. concludes with a notably low exploration probability,  end , after
to strike a balance between exploration where  and andexploita- are random variables, and ⊙ denotes the element-wise product. With time, the
step  , = thereby + facilitating + noisy ⊙ a=transition
+
 +  where +⊙ towards
 = and
⊙  an+ 
+are  random
increased+noisynoisy variables,
  + and ⊙ 
⊙ emphasis
⊙(5) on denotes
noisy exploitation

 (5) (i.e., a
 network can gradually disregard the noisy
noisy stream, albeit at varying rates across distinct regions of
noisy
tion in
 the process
where  and  are random  of
where training
greedier variables,
 and reinforcement
approach)  where
 and are random
⊙ during learning
 and
denotes the
variables,
the 
are
 latter random
element-wise
and
stages ⊙ variables,
denotes
the state-specific of product.
the
element-wiseexploration the and
trainingWith
element-wise
⊙ denotes
time,
process.
product. with the the
product.
Despite
Witha time, element-wise
With
this time,
shift,product.
the
a minimal With tim
the state space, thereby enabling form ofthe network
intrinsic self-annealing.
policies.
network canFor instance,
gradually network when
disregard
exploration can epsilon
the noisy
gradually network
is set
stream, to
disregard can
0.3,
albeit gradually
the
at
the varying
noisy disregard
rates
stream, the
across
albeit noisy
distinct
at stream,
varying regions albeit
rates of
acrossat varying
distinct rates
regionsacross of distinct
nearsregio
Thisprobability
dynamic exploration persists, strategy ensuring
can gradually allowsthe for ability
disregard
a more to fine-grained
explore
the noisyeven stream,
balanceas the albeit
between policy
at exploration
theoutput
state space, thereby
action is randomly enabling
the state
convergence. state-specific
space,
chosen the
thereby state
enabling
from the action space,
exploration thereby
with
state-specific a enabling
form of state-specific
intrinsic
exploration exploration
self-annealing.
with a form of with
intrinsic a form of
self-annealing. intrinsic self-anne
and exploitation, facilitating varying improved ratesbetween learning
acrossfor efficiencyregions
distinct and adaptability
of exploration
the in complex
statebetween
This dynamic exploration Thisstrategydynamicallows This
exploration for dynamic
a more strategy exploration
fine-grained
allows strategy
forbalance
a more allows
fine-grained a more
explorationbalance fine-grained
between balance explor
space with a probability of 0.3, environments.
and with aexploitation,
probabili-
and exploitation, facilitating
and exploitation, improved and learning
facilitating efficiency
improved space,
facilitating
and thereby
learning improved
adaptability enabling
efficiency learning
in state-specific
complex
and efficiency exploration
adaptability and adaptability in com
in complex
ty of 0.7, the outputenvironments. Noisy
actionnetwork is selectedenvironments. greedily based with a form of intrinsic self-annealing. This dynamic
environments.
Noisy
Soft update networks for target are often network utilized instead of the epsilon-greedy method to promote more
on argmax (Q). effective and The softexploration
dynamic update target exploration
network strategy is a key allows conceptfor in athe more fieldfine-grained
of deep reinforcement
Soft update for target network
Softofupdate for target Soft update for targetduring
[37]network networktraining. Unlike the epsilon-greedy approach, which
A refined version only the
adjusts epsilon-greedy
learning the exploration method
. It refersprobability, to a techniquebalance noisy usednetworks
between to stabilize
exploration and improve
introduce and stochasticitythe training
exploitation, directlyof deep
facil- neural
into reinforce
the
The soft update targetThe network soft update is a key The
target soft
conceptnetwork update
in the is target
afield
key of network is
in athe
deep reinforcement
concept keyfield conceptof deep in the field
reinforcement of deep
is referred to as the networks
Adaptive-epsilon-greedy
network’s Updated
parameters, in on reinforcement
April
enabling ap- 30, a 2023
morelearning nuanced tasks.and Unlikecontinuous hard updates, exploration whichprocess. involveThe periodically
Noisy
learning . It refers to
[37]
a technique
learning [37]
. It usedreferslearning
totostabilize [37]
a technique . Itand refers itating
improve
used to to improved
a technique
the
stabilizetraining and learning
used to
ofimprove
deep efficiency
stabilize
neural
the training andofadaptability
and improve deep theneural
training of deep n
Network copying
[35]
introduces the parametersa novel of
conceptthe main of anetwork
noisy to the layer,
linear target integrating
network, soft bothupdates gradually and
deterministic blend
proach [38]
networks in . reinforcement
In this approach,
networks learning for
in instance,
tasks.networks
reinforcement Unlikethe policy
in reinforcement
hard
learning updates,
tasks.
in complex learning
which
Unlike involve
hard tasks.
environments. Unlike
periodically
updates, which hard updates,
involve which
periodically involve period
noisy the parameters
Updated on of
April the
30, target
2023 network towards those of the main network. This process helps to
copying
is trained the parameters of thecomponents.
copying
over N epochs/episodes, main network
the parameters copying
a value toof the thetarget
contingent parameters
main network,
network of soft
the
to the main
updates
target network
gradually
network, to the blend
soft target
updates network,gradually soft updates
blend gradually
edy technique serves as a means to strike mitigate
a balance thebetween
issue of exploration
drastic changes Soft update in thefor target target network,network which can lead to instability during the
the parameters of the target the parametersnetwork towards of the parameters thosenetwork
target of the of
= mainthe +target
towards network. network
those
+ This
noisyof the towards
⊙ 
process
main + those noisyof
helps
network.
 to⊙the  main
This
process network. helpsThis to process (5) hel
process of mitigate upon the
training specific problem.
reinforcement learning Initially,
learning
policies. the
process.
mitigateFor algorithm
The valuewhen
instance,
the issue of
of  The
drastic
is used. soft
changes
Inupdate
the paper,
in the target
target
it proposed
network
network,
anisalgorithm
which a key
canduring
lead
called DPG. They
concept tothe
instability durin
the issue of drastic mitigate
where changes
the and in the
issue of target
drastic
are network,
random changes which
in the
variables, can
target
and lead ⊙ todenotes
network, instability
which the during
can lead
element-wisetheto instability product. With time, the
greedy
output technique
action is serves
sets epsilon
randomly as a to means tofrom
pinit value
chosen (e.g.,strike p the a= used
balance
0.6),
action 0.001. The
between
 =gradually
space with target
exploration
reduc-
a probability network is updated as follows. [37]
learning process. The learning of is
process.used. learning
In
The the
value paper,process.
of it The
proposed
is used. invalue
In the
an
the field
of
algorithm
paper,
 isof itdeep
used. calledIn
proposed reinforcement
the
DPG. paper,
an They it
algorithm learning
proposed called an DPG. . It
algorithm re-
They called DPG.
he process of training reinforcement network learning can gradually disregard the noisy stream, = albeit
 ×  +atvarying × 1rates −  across distinct regions of(6)

init 
policies. For instance, when     
bility of 0.7,used the
ing output
 it= to
0.001 action
reach . Theistarget
= selected
p
used
the  =greedily
(e.g.,
network
state p
0.001
space, = The based
0.1)
is .updated used
thereby over
as
target on= aargmax
follows.
networkdesig-
0.001
enabling . The (Q). target
isstate-specific
updated
fers network
to asa follows.
technique is updated
exploration used astofollows.
with stabilize
a form ofand improve
intrinsic the towards
self-annealing.
he output action is randomly chosen from the action space end end
 Due with to athe probability
small
   value  of the parameter   , the target network smoothly adjusts
on of the epsilon-greedy
nated number method is referred
This
ofistraining dynamic to as
epochs/episodes =the
exploration ×Adaptive-epsilon-
 + ).strategy
(n Pri- × 1allows
 −=   ×for  +amore   × =1 −×  + (6) balance
fine-grained

× 1 −between  (6)
exploration
bability of 0.7, the output action selected greedily based
the Q-network’sstep on argmax (Q). training of deep
value. To ensure the noticeable impact of this adjustment, frequent neural networks in reinforcement updates are
his approach, Due
for instance, to the
the small
policy value of
Due
istraining
trained the
to parameter
the
over small Due
value
Nfacilitating , the
epochs/episodes,
 to
of the
target
the small
network
parameter value
aupdatelearning  of
smoothly
, thethe parameter
adjusts
target network ,
towards
 the target
smoothly network
adjusts smoothly
towards adjusts tow
rsion of the epsilon-greedy
marily, during method
the isand
initial referredexploitation,
to asphase,
required. the By Adaptive-epsilon-
the employing
model is aimproved
softlearning strategy,
tasks. efficiency
the target
Unlike hard and adaptability
network
updates, can moreinvolve
which in complex
smoothly track the
he
n thisspecific
approach, the Q-network’s
problem.
for instance, value.
Initially, the
the
the policy To ensure
Q-network’s
algorithm
is trained theover the
noticeable
value.
setsN in Q-network’s
epsilon To impact
epochs/episodes,ensure
to init of value.
the this To
noticeable ensure
adjustment, impact the noticeable
frequent
of this updates impact
adjustment,
a(e.g., enabling a more stable and effective learning process. This are of this
frequent adjustment,
updates frequent
are update
granted increased environments.
exploration changes
freedom the amain
with higher network, periodically copying the parameters of the main net-
ducing required.
it to reach By employing a
required.soft update
By employing required.
strategy, athe By
soft targetemploying
update networkstrategy, a soft
can update
more
the target strategy,
smoothly
network the
trackcan target
the
more network
smoothly cantrackmore the smoothly trac
n the specific problem. end (e.g.,the
 = Initially, end = 0.1 ) over
algorithm a designated
sets epsilon to init number (e.g., of
changes
it to probability in the
= end(e.g., main network,
changes
pinit ==training
0.6), in enabling
the
followed mainchanges
a more
network,
by model in
a gradual the
stable main
and
enabling network,
effective
a more enabling
learning
stable and
of work to the target network, soft updates gradually a
process.more
effective stable
This and
learning effective
process. learning
This process.
( step ). Primarily,
reducing reach during the
(e.g.,initial
Soft
end 0.1
update) over phase,
fora designated
target the network number is granted
des ( ). Primarily,
epsilon during
decrease
eedom with a higher probability (e.g., initThe
 the initial
at a training
rate r over phase,
= 0.6 the the model
training
), followed is
epochs/granted
by a gradual blend is theaparameters of the target network towards
step
soft update target network key concept in the field of deep reinforcement
freedom with
r over the training a higher probability
epochs/episodes, (e.g.,  = 0.6 ), followed by a gradual
episodes, adhering tolearning theadhering [37] to the
subsequent subsequent
formula: formula: those of the main network. This process helps to neural
adhering to. the It refers to aformula: technique used to stabilize and improve the training of deep
init
rate r over the training epochs/episodes, −step subsequent
 =   −step , 0 networks in reinforcement learning (3)tasks. the
mitigate Unlike issuehard updates,
of drastic changeswhich ininvolve the target periodically
=  , 0 (3)(3)
 copying the parameters of the mainnetwork, network which to the target can lead to instability during the blend
network, soft updates gradually
 ←← init − end ++end the parameters
end (4)(4) learning
of the target network (4) towards process. Thethe
those of valuemain of network.
τ is used. This In theprocesspaper, helps to
init − end
pproach
e approach concludes
concludes with
with aa notably
notably mitigate
lowexploration
low the issue
exploration of drastic
probability,
probability, changes
end ,afterend , after
in the target network, which can lead to instability during the
This adaptable approach concludes withvalue a notably it proposed an algorithm called DPG. They used τ =
g a transition
ating a transitiontowards towards an increased learning
increasedemphasis emphasisprocess. on on The exploitation
exploitation of  is
(i.e., (i.e.,
a used. a InThe the paper, it proposed an algorithm called DPG. They
low exploration probability,
used  = p
0.001 , after
. The ntarget , thereby
network 0.001.
is updated astarget
follows. network is updated as follows.
ng
uring thethe latter
latterstages
stages of of the trainingprocess.
the training process.Despite Despite
end thisthis shift,
step
shift,a minimala minimal
ty persists,ensuring
persists, facilitating
ensuring the aability
the transition
ability towardseven
toto explore
explore an
even increased
as as the the empha-
policy policy nearsnears    
=  ×  +    × 1 −  (6) (6)
sis on exploitation (i.e., a greedier Due to approach) the small value duringof the parameter Due to the ,small the target valuenetwork smoothly adjusts
of the parameter τ, the towards
the Q-network’s value. To ensure
the latter stages of the training process. Despite this target network smoothly adjusts towards the Q-net- the noticeable impact of this adjustment, frequent updates are
required. By employing a soft update strategy, the target network can more smoothly track the
shift, instead
a minimal of theexploration probabilitytopersists, en- work’s avalue.
ks are often utilized changes in method
epsilon-greedy themethod mainpromote network,
to promote
more enabling more To stableensure andthe noticeable
effective learningimpact of
process. This
are often utilized
c exploration during instead of Unlike
suring training.
the epsilon-greedy
the ability to explore the epsilon-greedy
even as theapproach, policy nears whichmore this adjustment, frequent updates are required. By
exploration
oration probability, duringnoisy training. networks Unlike introduce the epsilon-greedy
stochasticity directly approach, into the which
tion probability, convergence.
noisy networks introduce stochasticity intoemploying a soft update strategy, the target network
, enabling a more nuanced and continuous exploration process.directly The Noisy the
nabling
es a novela concept Noisy
more nuanced network
of a noisyand continuous
linear layer, integrating exploration process. The
both deterministic andNoisy can more smoothly track the changes in the main
a novel concept of Noisy
a noisy networks
linear layer,are often utilizedboth
integrating instead of the network,
deterministic and enabling a more stable and effective learn-
 =  +  + noisy ⊙  + noisy ⊙  
epsilon-greedy method to promote more effective ing process. This technique has proven to be particu- (5)
random  =variables,
 +andand + ⊙noisy denotes ⊙ the
 element-wise product. With time, the
+ noisy ⊙  training. Unlike larly (5) useful in complex reinforcement learning tasks
y disregard the noisydynamic stream, albeit exploration
at varying during rates across distinct regions of
dom variables, and ⊙ denotes the element-wise product. With time, the
the epsilon-greedy
by enabling state-specific explorationapproach, with a formwhich of intrinsic only self-annealing.
adjusts the where maintaining stability during the training phase
sregard
ation the noisy
strategy allows stream,
for a albeitfine-grained
more at varyingbalance rates across between distinct
exploration regions of
enabling state-specific exploration probability,
exploration with noisy a formnetworks of intrinsic introduce
self-annealing. is crucial for achieving optimal performance.
cilitating improved learning efficiency and adaptability in complex
on strategy allows for a more fine-grained balance between exploration
itating improved learning efficiency and adaptability in complex
network 51
ate target network is a key concept in the field of deep reinforcement
to a technique used to stabilize and improve the training of deep neural
etwork
ement learning tasks. Unlike hard updates, which involve periodically
target network is a key concept in the field of deep reinforcement
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

4. Results
This section describes the setup of our experi-
ments, including the dataset, evaluation methods,
and the baselines for comparison.

4.1 Experimental setup

Data collections
Figure 2. Labeled frames from the video at 1 Hz.
The dataset we primarily used in our experiments
is the Cambridge-driving Labeled Video Database Table 1. Statistics for each class used in this study: “%” shows
or CamVid [34] with samples shown in Figure 2. This the ratio of pixels and “Occurrence” shows the number of occur-
public dataset comprises 360 × 480 street scene color rences over all images.
images or frames, each annotated with ground truth Class name Percentage Occurrence
semantic labels for pixels across 32 classes. The Road 27.3 701
images were captured from a moving automobile Building 22.7 687
using high-resolution cameras placed on the streets, Sky 15.8 699
allowing for the observation of various objects. In Tree 10.4 636
this study, our focus is on the segmentation of 11 key Sidewalk 6.33 672
classes. Given the urban context, the Road, Building, Car 3.4 643
and Sky classes collectively constitute most frame Column_Pole 0.98 698
pixels, accounting for approximately 15.81% and Fence 1.43 363
27.35%, and appearing in almost all frames. Other Pedestrian 0.64 640

significant classes, such as Car and Pedestrian, are Bicyclist 0.53 365

consistently present throughout the frame sequence Sign_Symbol 0.12 416

but occupy smaller portions, approximately 3.93% Data collection instruments

and 0.64%, respectively. Additionally, our analysis
includes other classes depicted in Table 1. The ac-
companying table illustrates a significant imbalance
among the different classes, a challenge that we
address in our study. The video sequences were shot
during the daytime and at dusk, where the objects in
the scene can still be recognized but appear darker
than in other sequences. Daylight sequences were
captured in sunny weather conditions, featuring
mixed urban and residential surroundings.
For a fair comparison between different methods,
the segmentation networks of all methods have been
pre-trained on the GTA dataset [74], which comprises
extensive synthetic images with pixel-level semantic
annotations. These images are generated through the
open-world video game Grand Theft Auto 5, depict-
ing scenes from a car perspective within virtual cit-
ies designed in an American style, like our primary
testing dataset introduced earlier.
The CamVid dataset was captured using a digi-
tal film camera under fixed conditions without auto
zooming, focus, or adjustments during the collecting
process. The camera’s focus was adjusted to infinity,
and both gain and shutter speed were fixed. At the
outset, the aperture was widened to its maximum
extent while ensuring that white objects in the scene
did not become overexposed.
Data analysis
In a real-world scenario where we have unlabeled
data, it’s possible to involve a human annotator to
label the necessary dataset based on active learning
recommendations. Nevertheless, in this paper, as a
proof of concept, we opted to work with fully la-
belled data and selectively concealed portions of it
to assess the active learning algorithm’s performance
on the segmentation task.
The dataset we primarily used in our experiments
is CamVid [39] discussed earlier. The training, val-

52
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

idation, and test sets consist of 370, 104, and 234 regions. The x-axis, labeled as “Budget”, represents
images, respectively. In the training set, we used 100 the additional number of regions in thousands and
labelled images for building DT to train the DQN the percentage of utilized unlabeled data. The plots
network for several episodes and learn a good acqui- include means and standard deviations of 5 runs.
sition function that maximizes performance with a The segmentation network utilized in these meth-
budget of B regions. A set of 10 images or DS is used ods has been pre-trained with the GTA dataset and
to construct the state representation. For the baseline part of their respective target datasets. The dashed
evaluation set DV, we utilized 260 images. DS has a line represents 96% of the best performance (Inter-
similar class distribution to DT to represent it. The section Over Union) achieved by the segmentation
original validation set was used to build DR. The network trained with all available labels. Given that
dataset’s test set was employed to train the model the performance of the preceding work [32] surpasses
to obtain the final segmentation results. Each image that of the other baseline models, we will adopt it as
was divided into K regions (in this case, K = 24) the new baseline model for comparisons with other
with a resolution of 80 × 90. For implementation, we methods.
executed 5 different runs with random seeds to cal-
culate the mean and standard deviation. Horizontal
flips and random crops of 224 × 224 were applied
for data augmentation.
Evaluation
We trained the active learning agent on DT with
approximately 0.5 k regions to learn the selection
of regions that would improve performance in da-
ta-scarce scenarios. Subsequently, we evaluated the
model using DV, where the model could access an
increasing number of images within different fixed
budgets. Once the fixed budget was reached, the Figure 3. Comparisons of various active learning methods.
segmentation network was trained with LT until it
In detail, Figure 3 illustrates the performance
met the early stopping condition in DR. The segmen-
of various methods, including Prioritized Expe-
tation network f for all algorithms was pre-trained
rience Replay (PRIO) [34], the reproduced DQN
with the GTA dataset [74], a synthetic dataset, and DT.
baseline (BASELINE) [32], Dueling Deep Q-net-
Finally, we measured the segmentation model’s per-
work (MDQN) [33], Emphasizing Recent Experi-
formance on the CamVid test set using the Intersec-
ences (ERE) [36], and Noisy Network (NOISY) [35],
tion over Union (IoU) score.
Adaptive Epsilon Greedy (ESP) [38], Soft Update
Hardware usage
for Target Network [37]. It’s important to note that
The models were trained using a single NVIDIA
at 1.5 k regions, the performance of some methods
RTX A5000 GPU with 24 GB of VRAM. Training
exceeds 96% of the maximum achieved with fully
the active learning agents took approximately 18
supervised training (having access to all labels). In
hours for 5 runs, and training the segmentation mod-
these experiments, the NOISY model performs the
els to test the active learning algorithm required a
worst, suggesting that acquiring new labels does
total of 8 hours for 5 runs at each of the 6 budgets.
not provide significant additional information to the
model. PRIO and SOFT outperform the other meth-
4.2 Experiment results
ods, including the baseline, in all budget scenarios,
In Figures 3 and 4, we compare various methods except for the 1K case for the PRIO method. They
across increasing budgets of labeled 128 × 128 pixel achieve this without overfitting the training model,

53
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
while the other methods yield similar results. This
suggests that effective active learning, through se-
lective labeling or additional information, can assist
the segmentation model in avoiding local minima
and achieving better performance.
Figure 4. Compare the PRIO method with varying replay buffer
sizes.
In Figure 4, a comparison is made between the
baseline and PRIOR methods, considering different
replay buffer pool sizes: 600, 30,000, 60,000, and
120,000. Performance remains relatively stable for
both 60,000 and 120,000. Interestingly, PRIOR,
despite having a smaller replay buffer (around 600
compared to 30,000, 60,000, and 120,000), outper-
forms the others by a significant margin.
4.3 Discussion
Implications and suggestions
In this section, we examine the primary findings
from the experiments. As the experiment results
show, implementing a soft target network update can
improve performance. This is due to the smoother
tracking of changes in the main network by the tar-
get network through the use of soft updates. Conse-
quently, a more stable learning process is achieved,
and the algorithm is able to converge to a better pol-
icy, resulting in improved performance reflected in
better results and more reliable Q-value estimations.
The Dueling DQN algorithm improves upon earlier
models by decoupling value and advantage func-
tions in Q-value estimation. This allows for better
recognition of action importance in varying states,
54
thereby enhancing learning and generalization. The
resulting architecture enables more effective com-
prehension of state value and action advantage,
leading to improved action selection and overall per-
formance. Moreover, this division reduces the vari-
ability in learned action values, stabilizes learning,
and provides more precise estimates. This solves the
problems of overestimation or underestimation of
Q-values. Prioritized Experience Replay (PER) im-
proves reinforcement learning by enhancing sample
efficiency, stabilizing the learning process, and pro-
moting effective exploration of the state space. PER
prioritizes experiences based on higher probabilities
for transitions with greater TD errors, accelerating
convergence and fostering efficient learning. Empha-
sizing rare events also aids agents in handling critical
scenarios adeptly. The learning process stability is
ensured by policy updates that efficiently lead to rap-
id learning and enhanced convergence. Emphasiz-
ing transitions with high learning potential aids the
thorough exploration of the state space, resulting in
improved overall performance and decision-making
by the agent. PER usage reduces bias from uniform
sampling and mitigates high variance issues, result-
ing in more accurate and stable Q-value updates.
This enhances the learning process and improves
performance in various reinforcement learning tasks.
Our experiments showed that certain techniques,
like incorporating noisy networks to encourage di-
verse segmentation strategies, had a negative impact
on segmentation performance by introducing insta-
bility in the learning process, resulting in decreased
accuracy in certain scenarios. In intricate settings
with sparse rewards or high-dimensional state spac-
es, challenges arise where adjusting the exploration
rate alone may prove ineffective. Poor adjustment
of the exploration rate adaptation and disregard for
specific learning dynamics can disturb the balance
between exploration and exploitation, despite the
use of adaptable epsilon-greedy approaches, creating
difficulties in obtaining the ideal exploration-ex-
ploitation equilibrium, especially in some settings.
Overemphasizing recent experiences in a DQN can
hinder reinforcement learning performance by re-
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
ducing sample efficiency, impeding generalization,
introducing increased variance, and destabilizing the
learning process. It also limits exploration across
the state space, restricting the discovery of critical,
infrequently encountered states and thereby compro-
mising the agent’s convergence to an optimal policy.
Thus, in order to improve performance, it is crucial
to achieve a balance between prioritizing recent ex-
periences and maintaining a varied set of samples
that support efficient learning and exploration across
the entire state space.
Limitations and future work
We have illustrated the successful integration of
various enhancements into the DQN, enhancing the
semantic segmentation model to achieve state-of-
the-art performance. Furthermore, our findings in-
dicate that certain components within the integrated
algorithm yield distinct performance advantages.
While numerous algorithmic components couldn’t
be incorporated in this study, they stand as promising
candidates for future experiments involving integrat-
ed agents. Below, we discuss several of these poten-
tial candidates.
Policy-based methods directly parameterize
the policy, allowing for more flexible and complex
policies compared to value-based approaches like
DQN. Investigating the application of policy-based
methods to active learning in semantic segmentation
could provide valuable insights into optimizing de-
cision-making strategies. Actor-critic methods com-
bine the strengths of both policy and value-based
approaches by maintaining separate networks for
policy and value estimation. Exploring the integra-
tion of actor-critic methods in our active learning
framework may offer advantages in terms of stability
and efficiency. N-step methods extend the traditional
DQN by incorporating multiple consecutive states
and actions. Evaluating the impact of N-step methods
on active learning performance in semantic segmen-
tation could enhance our understanding of the tem-
poral dynamics involved. Distributional RL models
the distribution of returns rather than focusing solely
on expected values. Introducing distributional RL
techniques into our framework may contribute to a
55
more nuanced understanding of uncertainty and risk
management in the active learning process. Imitation
learning leverages expert demonstrations to guide
the learning process. Integrating imitation learning
into active learning for semantic segmentation could
offer a valuable mechanism for initializing the model
and accelerating the learning curve.
The exploration of these alternative RL tech-
niques represents a promising direction for future
research. Investigating how those methods can be
tailored to the specific challenges of active learning
in semantic segmentation is essential. The strengths
of different RL paradigms could be harnessed by
exploring the combination of these techniques in
hybrid models or ensembles. Moreover, the transfer-
ability and generalization of learned policies across
diverse datasets and domains require attention in fu-
ture investigations.
5. Conclusions
In conclusion, our study provides a compre-
hensive comparison of different DQN extensions
designed to improve active learning in semantic
segmentation through reinforcement learning. Our
primary objective is to mitigate the labour-intensive
task of obtaining pixel-wise labels with human in-
tervention. Our results demonstrate that the NOISY
model performs the worst, showing rapid overfitting,
therefore implying that acquiring new labels does not
significantly enhance the model’s information. No-
tably, prioritized experience replay and soft update
outperform all other methods, including the baseline,
in all budget scenarios. Importantly, these methods
achieve superior performance without overfitting,
while the other techniques yield similar results. Ad-
ditionally, the comparison of the baseline and PER
methods, considering different replay buffer pool
sizes, indicates that PRIOR outperforms others de-
spite having a smaller replay buffer. This emphasizes
the importance of utilizing information effectively to
improve the segmentation process. This highlights
the effectiveness of selective labelling or including
additional information to help the segmentation
model avoid local minimum and achieve better per-
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
formance. These findings highlight the prospect of
utilizing sophisticated DQN extensions to enhance
active learning in semantic segmentation, resulting
in streamlined label acquisition and improved model
performance.
Author Contributions
Conceptualization, D.H. and S.C.; methodology,
D.H. and S.C.; investigation, D.H.; validation, D.H.
and P.H.; data curation, D.H. and P.H.; writing, D.H.
and P.H.; review and editing, S.C.; supervision, S.C.
Conflict of Interest
There is no conflict of interest.
Funding
This work is partially supported by the Vice Pres-
ident for Research and Partnerships of the University
of Oklahoma, the Data Institute for Societal Chal-
lenges, and the Stephenson Cancer Center through
DISC/SCC Seed Grant Award.
References
[1] Szeliski, R., 2022. Computer vision: Algo-
rithms and applications. Springer Nature: Ber-
lin.
[2] Li, H., Xiong, P., Fan, H., et al., 2019. DFANet:
Deep Feature Aggregation for Real-Time
Semantic Segmentation [Internet]. Available
from: https://openaccess.thecvf.com/content_
CVPR_2019/papers/Li_DFANet_Deep_Fea-
ture_Aggregation_for_Real-Time_Semantic_
Segmentation_CVPR_2019_paper.pdf
[3] Liu, M., Yin, H., 2019. Feature pyramid encod-
ing network for real-time semantic segmenta-
tion. arXiv preprint arXiv:1909.08599.
DOI: https://doi.org/10.48550/arXiv.1909.08599
[4] Li, X., You, A., Zhu, Z., et al. (editors), 2020.
Semantic flow for fast and accurate scene pars-
ing. Computer Vision-ECCV 2020: 16th Euro-
pean Conference; 2020 Aug 23-38; Glasgow,
UK. Cham: Springer International Publishing.
56
p. 775-793.
DOI: https://doi.org/10.1007/978-3-030-58452-8_45
[5] Yang, X., Wu, Y., Zhao, J., et al., 2020.
Dense Dual-Path Network for Real-time Se-
mantic Segmentation [Internet]. Available
from: https://openaccess.thecvf.com/content/
ACCV2020/papers/Yang_Dense_Dual-Path_
Network_for_Real-time_Semantic_Segmenta-
tion_ACCV_2020_paper.pdf
[6] Orsic, M., Kreso, I., Bevandic, P., et al., 2019.
In Defense of Pre-Trained ImageNet Architec-
tures for Real-Time Semantic Segmentation
of Road-Driving Images [Internet]. Available
from: https://openaccess.thecvf.com/content_
CVPR_2019/papers/Orsic_In_Defense_of_
Pre-Trained_ImageNet_Architectures_for_Re-
al-Time_Semantic_Segmentation_CVPR_2019_
paper.pdf
[7] Zhang, H., Tang, W., Na, W., et al., 2020.
Implementation of generative adversarial net-
work-CLS combined with bidirectional long
short-term memory for lithium-ion battery
state prediction. Journal of Energy Storage. 31,
101489.
DOI: https://doi.org/10.1016/j.est.2020.101489
[8] Zhang, H., Na, W., Kim, J. (editors), 2018.
State-of-charge estimation of the lithium-ion
battery using neural network based on an im-
proved thevenin circuit model. 2018 IEEE
Transportation Electrification Conference and
Expo (ITEC); 2018 Jun 13-15; Long Beach,
CA, USA. New York: IEEE. p. 342-346.
DOI: https://doi.org/10.1109/ITEC.2018.8450162
[9] Zhang, H., Cheng, S., El Amm, C., et al., 2023.
Efficient pooling operator for 3D morphable
models. IEEE Transactions on Visualization
and Computer Graphics. 1-9.
DOI: https://doi.org/10.1109/TVCG.2023.3255820
[10] Han, D., Wang, S., Jiang, C., et al., 2015.
Trends in biomedical informatics: Automated
topic analysis of JAMIA articles. Journal of the
American Medical Informatics Association.
22(6), 1153-1163.
DOI: https://doi.org/10.1093/jamia/ocv157
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
[11] Han, D., Mulyana, B., Stankovic, V., et al.,
2023. A survey on deep reinforcement learning
algorithms for robotic manipulation. Sensors.
23(7), 3762.
DOI: https://doi.org/10.3390/s23073762
[12] Sze, V., Chen, Y.H., Yang, T.J., et al., 2017.
Efficient processing of deep neural networks:
A tutorial and survey. Proceedings of the IEEE.
105(12), 2295-2329.
DOI: https://doi.org/10.1109/JPROC.2017.2761740
[13] Wang, W., Fu, Y., Pan, Z., et al., 2020. Re-
al-time driving scene semantic segmentation.
IEEE Access. 8, 36776-36788.
DOI: https://doi.org/10.1109/ACCESS.2020.2975640
[14] Papadeas, I., Tsochatzidis, L., Amanatiadis, A.,
et al., 2021. Real-time semantic image segmen-
tation with deep learning for autonomous driv-
ing: A survey. Applied Sciences. 11(19), 8802.
DOI: https://doi.org/10.3390/app11198802
[15] Mahe, H., Marraud, D., Comport, A.I. (editors),
2019. Real-time rgb-d semantic keyframe slam
based on image segmentation learning from
industrial cad models. 2019 19th International
Conference on Advanced Robotics (ICAR);
2019 Dec 2-6; Belo Horizonte, Brazil. New
York: IEEE. p. 147-154.
DOI: https://doi.org/10.1109/ICAR46387.2019.
8981549
[16] Bruce, J., Balch, T., Veloso, M. (editors), 2000.
Fast and inexpensive color image segmentation
for interactive robots. Proceedings. 2000 IEEE/
RSJ International Conference on Intelligent
Robots and Systems (IROS 2000) (Cat. No.
00CH37113); 2000 Oct 31-Nov 5; Takamatsu,
Japan. New York: IEEE. p. 2061-2066.
DOI: https://doi.org/10.1109/IROS.2000.895274
[17] Du, X., Nie, Y., Wang, F., et al., 2022. AL-Net:
Asymmetric lightweight network for medical
image segmentation. Frontiers in Signal Pro-
cessing. 2, 842925.
DOI: https://doi.org/10.3389/frsip.2022.842925
[18] Lou, A., Guan, S., Loew, M., 2023. Cfpnet-m:
A light-weight encoder-decoder based network
for multimodal biomedical image real-time
57
segmentation. Computers in Biology and Med-
icine. 154, 106579.
DOI: https://doi.org/10.1016/j.compbiomed.2023.
106579
[19] Cordts, M., Omran, M., Ramos, S., et al., 2016.
The Cityscapes Dataset for Semantic Urban
Scene Understanding [Internet]. Available
from: https://openaccess.thecvf.com/content_
cvpr_2016/papers/Cordts_The_Cityscapes_
Dataset_CVPR_2016_paper.pdf
[20] Gal, Y., Islam, R., Ghahramani, Z., 2017. Deep
Bayesian Active Learning with Image Data
[Internet]. Available from: https://proceedings.
mlr.press/v70/gal17a/gal17a.pdf
[21] Chu, H.M., Lin, H.T. (editors), 2016. Can ac-
tive learning experience be transferred? 2016
IEEE 16th International Conference on Data
Mining (ICDM); 2016 Dec 12-15; Barcelona,
Spain. New York: IEEE. p. 841-846.
DOI: https://doi.org/10.1109/ICDM.2016.0100
[22] Hsu, W.N., Lin, H.T., 2015. Active learning by
learning. Proceedings of the AAAI Conference
on Artificial Intelligence. 29(1).
DOI: https://doi.org/10.1609/aaai.v29i1.9597
[23] Yoo, D., Kweon, I.S., 2019. Learning Loss
for Active Learning [Internet]. Available
from: https://openaccess.thecvf.com/content_
CVPR_2019/papers/Yoo_Learning_Loss_for_
Active_Learning_CVPR_2019_paper.pdf
[24] Lookman, T., Balachandran, P.V., Xue, D.,
et al., 2019. Active learning in materials sci-
ence with emphasis on adaptive sampling using
uncertainties for targeted design. npj Computa-
tional Materials. 5(1), 21.
DOI: https://doi.org/10.1038/s41524-019-0153-8
[25] Fasel, U., Kutz, J.N., Brunton, B.W., et al.,
2022. Ensemble-SINDy: Robust sparse model
discovery in the low-data, high-noise limit,
with active learning and control. Proceedings
of the Royal Society A. 478(2260), 20210904.
DOI: https://doi.org/10.1098/rspa.2021.0904
[26] Hu, Z., Bai, X., Zhang, R., et al., 2022. Lidal:
Inter-frame uncertainty based active learning
for 3d lidar semantic segmentation. European
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
Conference on Computer Vision. 13687, 248-
265.
DOI: https://doi.org/10.1007/978-3-031-19812-0_15
[27] Lenczner, G., Chan-Hon-Tong, A., Le Saux, B.,
et al., 2022. Dial: Deep interactive and active
learning for semantic segmentation in remote
sensing. IEEE Journal of Selected Topics in
Applied Earth Observations and Remote Sens-
ing. 15, 3376-3389.
DOI: https://doi.org/10.1109/JSTARS.2022.3166551
[28] Xie, B., Yuan, L., Li, S., et al., 2022. Towards
Fewer Annotations: Active Learning via Re-
gion Impurity and Prediction Uncertainty for
Domain Adaptive Semantic Segmentation
[Internet]. Available from: https://openaccess.
thecvf.com/content/CVPR2022/papers/Xie_
Towards_Fewer_Annotations_Active_Learn-
ing_via_Region_Impurity_and_Prediction_
CVPR_2022_paper.pdf
[29] Dhiman, G., Kumar, A.V., Nirmalan, R., et al.,
2023. Multi-modal active learning with deep
reinforcement learning for target feature ex-
traction in multi-media image processing ap-
plications. Multimedia Tools and Applications.
82(4), 5343-5367.
DOI: https://doi.org/10.1007/s11042-022-12178-7
[30] Zhou, W., Li, J., Zhang, Q., 2022. Joint com-
munication and action learning in multi-target
tracking of UAV swarms with deep reinforce-
ment learning. Drones. 6(11), 339.
DOI: https://doi.org/10.3390/drones6110339
[31] Hu, M., Zhang, J., Matkovic, L., et al., 2023.
Reinforcement learning in medical image anal-
ysis: Concepts, applications, challenges, and
future directions. Journal of Applied Clinical
Medical Physics. 24(2), e13898.
DOI: https://doi.org/10.1002/acm2.13898
[32] Casanova, A., Pinheiro, P.O., Rostamzadeh,
N., et al., 2020. Reinforced active learning
for image segmentation. arXiv preprint arX-
iv:2002.06583.
DOI: https://doi.org/10.48550/arXiv.2002.06583
[33] Wang, Z., Schaul, T., Hessel, M., et al., 2016.
Dueling Network Architectures for Deep Rein-
58
forcement Learning [Internet]. Available from:
https://proceedings.mlr.press/v48/wangf16.pdf
[34] Schaul, T., Quan, J., Antonoglou, I., et al.,
2015. Prioritized experience replay. arXiv pre-
print arXiv:1511.05952.
DOI: https://doi.org/10.48550/arXiv.1511.05952
[35] Fortunato, M., Azar, M.G., Piot, B., et al.,
2017. Noisy networks for exploration. arXiv
preprint arXiv:1706.10295.
DOI: https://doi.org/10.48550/arXiv.1706.10295
[36] Wang, C., Ross, K., 2019. Boosting soft ac-
tor-critic: Emphasizing recent experience
without forgetting the past. arXiv preprint arX-
iv:1906.04009.
DOI: https://doi.org/10.48550/arXiv.1906.04009
[37] Lillicrap, T.P., Hunt, J.J., Pritzel, A., et al.,
2015. Continuous control with deep rein-
forcement learning. arXiv preprint arX-
iv:1509.02971.
DOI: https://doi.org/10.48550/arXiv.1509.02971
[38] Tokic, M., Palm, G., 2011. Value-difference
based exploration: Adaptive control between
epsilon-greedy and softmax. Annual confer-
ence on artificial intelligence. Springer: Berlin.
pp. 335-346.
DOI: https://doi.org/10.1007/978-3-642-24455-1_33
[39] Brostow, G.J., Shotton, J., Fauqueur, J., et al.
(editors), 2008. Segmentation and recogni-
tion using structure from motion point clouds.
Computer Vision-ECCV 2008: 10th European
Conference on Computer Vision; 2008 Oct 12-
18; Marseille, France. Berlin: Springer. p. 44-
57.
DOI: https://doi.org/10.1007/978-3-540-88682-2_5
[40] Angluin, D., 1988. Queries and concept learn-
ing. Machine Learning. 2, 319-342.
DOI: https://doi.org/10.1023/A:1022821128753
[41] King, R.D., Whelan, K.E., Jones, F.M., et al.,
2004. Functional genomic hypothesis genera-
tion and experimentation by a robot scientist.
Nature. 427(6971), 247-252.
DOI: https://doi.org/10.1038/nature02236
[42] Dagan, I., Engelson, S.P. (editors), 1995. Com-
mittee-based sampling for training probabilis-
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
tic classifiers. Machine Learning Proceedings
1995; 1995 Jul 9-12; Tahoe, California. p. 150-
157.
DOI: https://doi.org/10.1016/B978-1-55860-
377-6.50027-X
[43] Krishnamurthy, V., 2002. Algorithms for op-
timal scheduling and management of hidden
Markov model sensors. IEEE Transactions on
Signal Processing. 50(6), 1382-1397.
DOI: https://doi.org/10.1109/TSP.2002.1003062
[44] Lewis, D.D., 1995. A Sequential Algorithm for
Training Text Classifiers: Corrigendum and Ad-
ditional Data [Internet]. Available from: https://
dl.acm.org/doi/pdf/10.1145/219587.219592
[45] Ren, P., Xiao, Y., Chang, X., et al., 2021. A sur-
vey of deep active learning. ACM Computing
Surveys (CSUR). 54(9), 1-40.
DOI: https://doi.org/10.1145/3472291
[46] Shui, C., Zhou, F., Gagné, C., et al., 2020.
Deep Active Learning: Unified and Principled
Method for Query and Training [Internet].
Available from: https://proceedings.mlr.press/
v108/shui20a/shui20a.pdf
[47] Settles, B., 2012. Active learning, volume 6 of
synthesis lectures on artificial intelligence and
machine learning. Morgan & Claypool.
[48] Settles, B., 2011. From Theories to Queries:
Active Learning in Practice [Internet]. Avail-
able from: https://proceedings.mlr.press/v16/
settles11a/settles11a.pdf
[49] Hernández-Lobato, J.M., Adams, R., 2015.
Probabilistic Backpropagation for Scalable
Learning of Bayesian Neural Networks [Inter-
net]. Available from: http://proceedings.mlr.
press/v37/hernandez-lobatoc15.pdf
[50] Osugi, T., Kim, D., Scott, S. (editors), 2005.
Balancing exploration and exploitation: A new
algorithm for active machine learning. Fifth
IEEE International Conference on Data Mining
(ICDM'05); 2005 Nov 27-30; Houston, TX,
USA. New York: IEEE.
DOI: https://doi.org/10.1109/ICDM.2005.33
[51] Long, C., Hua, G., 2015. Multi-Class
Multi-Annotator Active Learning with Robust
59
Gaussian Process for Visual Recognition [Inter-
net]. Available from: https://openaccess.thecvf.
com/content_iccv_2015/papers/Long_Multi-
Class_Multi-Annotator_Active_ICCV_2015_
paper.pdf
[52] Gong, J., Fan, Z., Ke, Q., et al., 2022. Meta
Agent Teaming Active Learning for Pose Es-
timation [Internet]. Available from: https://
openaccess.thecvf.com/content/CVPR2022/
papers/Gong_Meta_Agent_Teaming_Active_
Learning_for_Pose_Estimation_CVPR_2022_
paper.pdf
[53] Sadigh, D., Dragan, A.D., Sastry, S., et al.,
2017. Active preference-based learning of re-
ward functions. UC Berkeley: Berkeley.
DOI: https://doi.org/10.15607/rss.2017.xiii.053
[54] Kunapuli, G., Odom, P., Shavlik, J.W., et al.
(editors), 2013. Guiding autonomous agents to
better behaviors through human advice. 2013
IEEE 13th International Conference on Data
Mining; 2013 Dec 7-10; Dallas, TX, USA.
New York: IEEE. p. 409-418.
DOI: https://doi.org/10.1109/ICDM.2013.79
[55] Ezzeddine, A., Mourad, N., Araabi, B.N., et al.,
2018. Combination of learning from non-op-
timal demonstrations and feedbacks using
inverse reinforcement learning and Bayesian
policy improvement. Expert Systems with Ap-
plications. 112, 331-341.
DOI: https://doi.org/10.1016/j.eswa.2018.06.035
[56] Liu, M., Buntine, W., Haffari, G. (editors),
2018. Learning how to actively learn: A deep
imitation learning approach. Proceedings of
the 56th Annual Meeting of the Association for
Computational Linguistics (Volume 1: Long
Papers); 2018 Jul 15-20; Melbourne, Australia.
p. 1874-1883.
DOI: https://doi.org/10.18653/v1/P18-1174
[57] Pang, K., Dong, M., Wu, Y., et al., 2018. Me-
ta-learning transferable active learning policies
by deep reinforcement learning. arXiv preprint
arXiv:1806.04798.
DOI: https://doi.org/10.48550/arXiv.1806.04798
[58] Contardo, G., Denoyer, L., Artières, T., 2017.
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
A meta-learning approach to one-step active
learning. arXiv preprint arXiv:1706.08334.
DOI: https://doi.org/10.48550/arXiv.1706.08334
[59] Sener, O., Savarese, S., 2017. Active learning
for convolutional neural networks: A core-set
approach. arXiv preprint arXiv:1708.00489.
DOI: https://doi.org/10.48550/arXiv.1708.00489
[60] Mittal, S., Niemeijer, J., Schäfer, J.P., et al.,
2023. Revisiting deep active learning for
semantic segmentation. arXiv preprint arX-
iv:2302.04075.
DOI: https://doi.org/10.48550/arXiv.2302.04075
[61] Shannon, C.E., 1948. A mathematical theory
of communication. The Bell System Technical
Journal. 27(3), 379-423.
DOI: https://doi.org/10.1002/j.1538-7305.1948.
tb01338.x
[62] Houlsby, N., Huszár, F., Ghahramani, Z., et al.,
2011. Bayesian active learning for classifica-
tion and preference learning. arXiv preprint
arXiv:1112.5745.
DOI: https://doi.org/10.48550/arXiv.1112.5745
[63] Kampffmeyer, M., Salberg, A.B., Jenssen, R.,
2016. Semantic Segmentation of Small Objects
and Modeling of Uncertainty in Urban Remote
Sensing Images Using Deep Convolutional
Neural Networks [Internet]. Available from:
https://www.cv-foundation.org//openaccess/
content_cvpr_2016_workshops/w19/papers/
Kampffmeyer_Semantic_Segmentation_of_
CVPR_2016_paper.pdf
[64] Jain, S.D., Grauman, K., 2016. Active Image
Segmentation Propagation [Internet]. Available
from: https://openaccess.thecvf.com/content_
cvpr_2016/papers/Jain_Active_Image_Seg-
mentation_CVPR_2016_paper.pdf
[65] Vezhnevets, A., Buhmann, J.M., Ferrari, V. (ed-
itors), 2012. Active learning for semantic seg-
mentation with expected change. 2012 IEEE
Conference on Computer Vision and Pattern
Recognition; 2012 Jun 16-21; Providence, RI,
USA. New York: IEEE. p. 3162-3169.
DOI: https://doi.org/10.1109/CVPR.2012.6248050
[66] Konyushkova, K., Sznitman, R., Fua, P., 2015.
60
Introducing Geometry in Active Learning
for Image Segmentation [Internet]. Available
from: https://openaccess.thecvf.com/content_
iccv_2015/papers/Konyushkova_Introducing_
Geometry_in_ICCV_2015_paper.pdf
[67] Aklilu, J., Yeung, S., 2022. ALGES: Active
Learning with Gradient Embeddings for Se-
mantic Segmentation of Laparoscopic Surgical
Images [Internet]. Available from: https://pro-
ceedings.mlr.press/v182/aklilu22a
[68] Shu, X., Yang, Y., Xie, R., et al., 2022. ALS:
Active learning-based image segmentation
model for skin lesion.
DOI: http://dx.doi.org/10.2139/ssrn.4141765
[69] Golestaneh, S.A., Kitani, K.M., 2020. Impor-
tance of self-consistency in active learning for
semantic segmentation. arXiv preprint arX-
iv:2008.01860.
DOI: https://doi.org/10.48550/arXiv.2008.01860
[70] Mackowiak, R., Lenz, P., Ghori, O., et al.,
2018. Cereals-cost-effective region-based ac-
tive learning for semantic segmentation. arXiv
preprint arXiv:1810.09726.
DOI: https://doi.org/10.48550/arXiv.1810.09726
[71] Van Hasselt, H., Guez, A., Silver, D., 2016.
Deep reinforcement learning with double
q-learning. Proceedings of the AAAI Confer-
ence on Artificial Intelligence. 30(1).
DOI: https://doi.org/10.1609/aaai.v30i1.10295
[72] Hasselt, H., 2010. Double Q-learning. Advanc-
es in Neural Information Processing Systems.
23, 2613-2621.
[73] Babaeizadeh, M., Frosio, I., Tyree, S., et al.,
2016. Reinforcement learning through asyn-
chronous advantage actor-critic on a gpu. arXiv
preprint arXiv:1611.06256.
DOI: https://doi.org/10.48550/arXiv.1611.06256
[74] Richter, S.R., Vineet, V., Roth, S., et al. (ed-
itors), 2016. Playing for data: Ground truth
from computer games. Computer Vision-ECCV
2016: 14th European Conference; 2016 Oct
11-14; Amsterdam, The Netherlands. Cham:
Springer International Publishing. p. 102-118.
DOI: https://doi.org/10.1007/978-3-319-46475-6_7
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

Journal of Electronic & Information Systems

https://journals.bilpubgroup.com/index.php/jeis

ARTICLE

Development of Technology and Equipment for Non-destructive Testing

of Defects in Sewing Mandrels of a Three-roll Screw Mill 30-80
Shatalov Roman Lvovich* , Zagoskin Egor Evgenievich

Moscow Polytechnic University, Moscow, 115280, Russian Federation

ABSTRACT
The conditions of heating and cooling of piercing mandrels made of 4X5MFS steel of a three-roll screw mill 30-
80 in the production of a closed cavity of steel vessels of small volume are determined. It is established that multiple
cycles of heating up to 600 °C and cooling with water up to 80 °C for about 7 seconds/1 cycle lead to the formation
of ridges, shells and cracks on the surface and in the volume of the tool. The loss of structural strength of the material
leads to the breakdown of the mandrel during the stitching process. The technique and equipment of magnetic powder
control have been developed to establish the dynamics of the growth of internal and external defects of mandrels. An
equation is obtained that allows determining the increase in the number of defects in the sewing tool of a screw rolling
mill. The technology of non-destructive testing made it possible to develop a rational plan for replacing the sewing
mandrels, which allows for predicting the appearance of defects leading to a complex breakdown of the deforming tool
at the NPO Pribor machine-building enterprise.
Keywords: Screw rolling mill 30-80; Piercing mandrel made of 4X5MFS steel; Vessel made of 50 steel; Temperature;
Crack; Magnetic powder control of hidden defects

1. Introduction shaped rolled products [1,2]. One of the most popular

types of rolled products for quality control today
Automation of rolling production in mechanical is steel vessels of the responsible purpose of small
engineering and metallurgy makes it possible to internal volume. That is why for the hot production
ensure high quality and low cost of flat, round and of semi-finished products, lines consisting of screw

*CORRESPONDING AUTHOR:
Shatalov Roman Lvovich, Moscow Polytechnic University, Moscow, 115280, Russian Federation; Email: mmomd@mail.ru
ARTICLE INFO
Received: 7 October 2023 | Revised: 4 November 2023 | Accepted: 6 December 2023 | Published Online: 12 December 2023
DOI: https://doi.org/10.30564/jeis.v5i2.6000
CITATION
Lvovich, S.R., Evgenievich, Z.E., 2023. Development of Technology and Equipment for Non-destructive Testing of Defects in Sewing Mandrels
of a Three-roll Screw Mill 30-80. Journal of Electronic & Information Systems. 5(2): 61-66. DOI: https://doi.org/10.30564/jeis.v5i2.6000
COPYRIGHT
Copyright © 2023 by the author(s). Published by Bilingual Publishing Group. This is an open access article under the Creative Commons Attribu-
tion-NonCommercial 4.0 International (CC BY-NC 4.0) License. (https://creativecommons.org/licenses/by-nc/4.0/).

61
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

rolling mills with a sewing mandrel installed along cludes heating equipment, a three-roll screw mill
the axis are increasingly used [3-5]. 30-80 with a rotating mandrel installed along the
The technology provides for significant heating axis with the possibility of moving along the axis of
temperatures of the initial bars-blanks to ensure the the firmware. The vessel was extracted with a “P”
continuity of the process of screw rolling, stitching shaped stop. The initial rod-blank had dimensions
and final heat treatment of products. The high temper- of a length of 70 mm diameter of 40 mm. The roll-
ature of about 2/3 of the melting makes it possible to ing speed was 1.2 m/s. The rhythm of rolling and
ensure the high plasticity of the deformed material [5,6]. stitching, including the extraction of the vessel from
The heating of semi-finished products affects the the mandrel, is about 7 seconds. The deformed ves-
deforming tool by heat transfer of the contacting sel was diverted along the stream and fell into the
surfaces [7,8]. The outer surface of the heated metal quenching tank. Figure 1 shows the main line of the
comes into contact with massive rolling rolls, which screw rolling mill 30-80.
are continuously cooled by water. For this reason,
their heating rarely exceeds critical values that re-
duce mechanical properties and lead to the formation
of thermal stresses and defects on the surface and in
the volume of the rolls [9,10]. The inner surface of a
small volume vessel is formed by a mandrel with a
low mass, without the possibility of cooling during
flashing [11-13]. The time of the piercing cycle and the
subsequent extraction of the mandrel from the vessel
cavity is not infrequently sufficient to warm up the Figure 1. The main line of the screw rolling mill is 30-80.
material to the temperature of the beginning of phase
The rolling mill line consisted of 1—asynchro-
rearrangements [14]. Intensive cooling with water al-
nous motor with a power of 50 kW each with a
lows us to exclude the influence of temperature on
rotation speed of 1250 revolutions per minute, 2—
the mandrel at the moment of inactivity. However, in
gimbals transmitting the moment of rotation, 3—
this case, there is a risk of the formation of thermal
gearboxes, 4—guide chutes, 5—pusher rod, 6—roll-
stresses due to the reverse rearrangement of the crys-
ing cage, 7—rotating mandrel, 8—mechanism fixing
tal lattice leading to the formation of cracks in the
the position of the mandrel relative to rolling axes,
volume of the tool [15,16]. Multiple cycles of intensive
9—stop removing the vessel from the mandrel, 10—
heating and cooling increase the risk of tool break-
diverting stream.
age and failure of the process line [17,18]. Therefore,
The scheme of production of steel vessels on the
this work was aimed at developing technology and
production line: The initial rod-billet was heated in
equipment for non-destructive testing of unaccept-
an induction and resistance furnace to a temperature
able defects in sewing mandrels.
of about 1160 °C, after which it was rolled and not
The purpose of the work was the development
completely stitched with a rotating mandrel, after
of technology and equipment for non-destructive
which it was removed with a U-shaped stop and di-
testing of defects in sewing mandrels of a three-roll
verted to the quenching tank along the gutter. Water
screw mill 30-80.
cooling of the rolling rolls was carried out continu-
ously, and the mandrels were at the time of extrac-
2. The methodology of the experiment tion from the cavity of the vessel.
The experiment was carried out during the hot Figure 2 shows the geometric dimensions of the
production of vessels made of steel 50 on the pro- deforming tool and steel vessels at the outlet of the
duction line of the BF JSC NPO Pribor, which in- processing line.

62
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
a mandrel magnetization equipment was developed.
c
Figure 2. Geometric dimensions: a—rolling rolls, b—sewing
mandrel, c—steel vessel at the outlet of the processing line.
Figure 2a shows the permissible geometric di-
mensions of 3 pieces of identical rolling rolls made
of 35 HGSA steel that are part of the equipment of
the rolling mill 30-80, Figure 2b shows the geomet-
ric dimensions of the working part of the piercing
mandrel made of 4X5MFS steel, Figure 2c shows
the geometric dimensions of the completed product
in the form of a vessel made of ST50 material. Per-
missible deviations from the dimensions in Figure 2
are not more than ± 0.1 mm.
The heating temperature of the mandrel was de-
termined using an optical pyrometer SEM 1600 after
its extraction from the vessel cavity.
The magnetic particle inspection method is a uni-
versal way of detecting defects in a deforming tool.
The method is based on the occurrence of magnetic
63
field inhomogeneity in places of discontinuity of fer-
romagnetic material, including steel and iron-based
alloys.
The formation of internal defects of the sewing
mandrel was investigated in accordance with GOST
56512-2015 “Non-destructive testing. Magnetic
powder method”. For this purpose, the design of the
Figure 3 shows the developed magnetization setup
of the sewing mandrels.
Figure 3. Installation of magnetization of sewing mandrels.
MD-I was used as a direct current source. The
magnetization parameters were a current of 2 V, and
a voltage of 160 A. Iron powder with a particle size
not exceeding 0.001 mm was used as an indicator.
The amount of powder was about 1/8 of the carrier
liquid, which included: 1/3 kerosene, 1/3 soap solu-
tion, and the rest of industrial oil and 20. Defects on
the surface of the mandrel were determined after the
release of a batch of vessels from 250 to 1200 piec-
es.
The sequence of technological operations when
checking the sewing mandrel for hidden defects:
● mechanical cleaning of the surface from dirt
and scale layer with sandpaper grain size 80.
● preparation of the suspension consists of inten-
sive mixing in the bathroom.
● the mandrel was installed on the lower elec-
trode, the upper electrode was moved along the
screw of the press all the way to the opposite part of
the mandrel.
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
● magnetization took place in 2 seconds between
the electrodes immersed in the suspension, under the
action of the applied field, the particles moved in the
area of magnetic disturbances, which indicate exter-
nal and internal defects of the material.
● defects of mandrels were determined visually.
Measurements of traces of accumulations of iron
powder were carried out with an electronic calliper.
3. The results of the experiment and
their discussion
The heating temperature of the sewing mandrel
was studied at an interval of 10 mm from the toe of
the mandrel. Its bow is taken as the beginning of the
countdown. Figure 4 shows the results of measure-
ments of the heating of the mandrel of the screw mill
30-80.
Figure 4. The results of measurements of the heating tempera-
ture of the screw machine mandrel 30-80.
Based on the temperature measurements carried
out, it can be assumed that the main formation of man-
drel defects will occur in the nasal region and further
decrease due to the cyclical heating modes shown in
Figure 4 and cooling with water up to 80 °C.
The stitching mandrel of a three-roll screw mill
30-80 was examined after the formation of the vessel
cavity. The check interval was every 250 pieces of
vessels. Figure 5 shows the development of defects
in the outer surface and inner volume of the sewing
mandrel after the formation of the vessel cavity.
The defects shown in Figure 5 indicate the influ-
ence of heating cycles up to 600 °C and cooling to
80 °C at intervals of 7 seconds. After the release of
250 pieces. there are no accumulations of powder on
64
the surface of the mandrel, its surface is smooth. Af-
ter the release of 500 pieces in the transition area of
the diameter from 16 to 18 mm, a longitudinal risk
is visible with a length of about 9 mm and a depth
of up to 1.2 mm. After the release of 750 pieces of
semi-finished products, external shells with a depth
of up to 1.1 mm and a diameter of about 3 mm are
added to the longitudinal risks in the same area. In
the nasal part, there is a pitted surface in small and
frequent shells. With the release of 1000 pieces, a
pronounced accumulation of magnetized powder is
observed in the nasal vessels at a distance of about
10 mm, which suggests the formation of through
fistulas. On the side surface of the mandrel, external
defects do not grow in length, but their depth in-
creases by about 30%. After the release of 1200 piec-
es, the mandrel toe collapsed at a distance of about
5 mm and a crack formed at a distance of about 20
mm. The experiment was stopped.
Figure 5. The development of defects in the outer surface and
inner volume of the sewing mandrel after the formation of the
vessel cavity: 1—250 pieces, 2—500 pieces, 3—750 pieces,
4—1000 pieces, and 5—1200 pieces.
The evolution of the growth of defects in a
4X5MFS steel sewing mandrel during the release of
steel vessels, with an interval of 250 pieces. The area
of defect formation was measured, as well as deter-
mined in the area of their formation at a distance of
50 mm from the toe of the mandrel. The total surface
area at this length was 2811 mm2. The sum of the
outer area of powder accumulations indicating the
formation of internal defects was taken into account.
Figure 6 shows the effect of the number of rolled
vessels on the area of defects on the surface of the
sewing mandrel.
Based on the measurements carried out, it can
be concluded that with an increase in the firmware
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
cycles, the number of defects increases and reaches
maximum values of about 12% of the studied outer
surface with the release of 1000 pieces of vessels.
The increase in the number of defects is subject to
the developed second-order polynomial equation
with a determination coefficient equal to 99%.
Figure 6. The effect of the number of rolled vessels on the area
of defects on the surface of the sewing mandrel.
4. Conclusions
1) Testing of the design and technology of mag-
netic powder inspection of defects of the piercing
mandrel made of 4X5MFS steel of a three-roll screw
mill 30-80 was carried out. It has been established
that with an increase in the number of rolled prod-
ucts, there is an increase in the number of external
and internal defects. A rationally planned interval of
replacement of mandrels for the production of steel
vessels of about 1000 pieces has been determined.
2) A significant influence of the number of rolled
vessels on the growth of the area of shells and cracks
on the sewing mandrel has been established. It is
shown that the critical area of defects at which the
destruction of the deforming tool can occur is about
12% of the total area of the mandrel under study. A
second-order polynomial equation is obtained that
allows determining the increase in the number of
defects of the mandrel surface with a determination
coefficient equal to 99%.
3) The development of a system and technolo-
gy for non-destructive quality control of the screw
mill mandrel allowed us to develop a rational plan
for replacing the piercing mandrels, which allows
us to predict the appearance of defects leading to
65
complex breakage of the deforming tool at the 30-
80 mill, which ensured an increase in the efficiency
of production of steel vessels at the NPO Pribor ma-
chine-building enterprise.
Author Contributions
The contribution of the authors consists of the
developed methodology and equipment for checking
the quality of the mandrel surface. An equation has
been developed to determine the growth of defects
in the mandrel surface from the number of vessel
cavities formed. The technology of non-destructive
testing of the presence of mandrel defects made it
possible to develop a forecast for the replacement of
the deforming tool.
Conflict of Interest
There is no conflict of interest.
References
[1] Muhin, U., Belskij, S., Makarov, E., et al., 2016.
Simulation of accelerated strip cooling on the
hot rolling mill run-out roller table. Frattura ed
Integrita Strutturale. 10(37), 305-311.
DOI: https://doi.org/10.3221/IGF-ESIS.37.40
[2] Belskiy, S., Mazur, I., Lezhnev, S., et al., 2016.
Distribution of linear pressure of thin-sheet
rolling across strip width. Journal of Chemical
Technology and Metallurgy. 51(4), 371-378.
[3] Orlov, D.A., Gamin, Y.V., Goncharuk, A.V.,
et al., 2021. Development and investigation of
piercing process using cooled guide shoes. Met-
allurgist. 65(3-4), 389-399.
DOI: https://doi.org/10.1007/s11015-021-01168-z
[4] Лубе, И.И., Трутнев, Н.В., Тумашев, С.В.,
et al., 2020. Повышение стойкости оправок
п р о ш и в н о го с т а н а п р и п р о и з в од с т в е
бесшовных труб из нержавеющей стали
мартенситного класса марки типа 13Сr в
линии ТПА. ЧЕРНАЯ МЕТАЛЛУРГИЯ.
(Russian) [Mastering of production of seamless
pipes of type 13 Cr martensite class stainless
steel at TPA 159-426 of JSC VTZ.] Бюллетень
 Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023
научно-техниче ской и экономиче ской
информации. 76(12), 1259-1264.
DOI: https://doi.org/10.32339/0135-5910-2020-
12-1259-1264
[5] Zorin, I.V., Dubtsov, Y.N., Sokolov, G.N., et al.,
2017. Piercing mandrel strengthening by sur-
facing with nickel aluminide-based alloy. IOP
Conference Series: Materials Science and Engi-
neering. 177(1), 012101.
[6] Shatalov, R.L., Zagoskin, E.E., Medvedev, V.A.,
2023. Influence of temperature unevenness on
hardness, structures and defects of the stitching
mandrel of a three-roll screw rolling mill 30-80.
Ferrous Metals. 3, 46-52.
DOI: https://doi.org/10.17580/chm.2023.03.08
[7] Vydrin, A.V., Neroznikov, V.L., Zvonarev,
D.Yu., et al., 2022. АНАЛИЗ ВЛИЯНИЯ
Н А С Т Р О Е Ч Н Ы Х П А РА М Е Т Р О В
ПРОШИВНОГО СТАНА НА ИЗМЕНЕНИЕ
Д И А М Е Т РА И Т ОЛ Щ И Н Ы С Т Е Н К И
ГИЛЬЗЫ (Russian) [Analysis of the influence
of setting parameters washing machine on the
change of case wall diameter and thickness].
Bulletin of the South Ural State University. Ser.
Metallurgy. 22(1), 42-52.
DOI: http://dx.doi.org/10.14529/met220105
[8] Tsubouchi, K., Akiyama, M., Okuyama, T.,
1997. Development and optimization of car-
bide-reinforced tools and application to hot
rolling of stainless steel. Journal of Tribology.
119(4), 687-693.
DOI: https://doi.org/10.1115/1.2833870
[9] Shatalov, R.L., Medvedev, V.A., 2019. Effect of
deformed workpiece temperature inhomogene-
ity on mechanical properties of thin-walled steel
vessels during treatment in a rolling and press-
ing line. Metallurgist. 63(1-2), 176-182.
DOI: https://doi.org/10.1007/s11015-019-00807-w
[10] Shatalov, R.L., Shelest, A.E., Medvedev, V.A.,
2020. Electromagnetic device for nondestructive
control of the mechanical properties of thin-
walled steel vessels. Russian Metallurgy (Metal-
ly). (3), 259-264.
DOI: https://doi.org/10.1134/S003602952003012X
[11] Romantsev, B.A., Aleshchenko, A.S., Tsyutsyu-
66
ra, V.Y., et al., 2017. Features of piercing mill
TPA 50-200 working roll wear during rolling
continuously-cast and hot-rolled billets. Metal-
lurgist. 60, 1062-1069.
DOI: https://doi.org/10.1007/s11015-017-0408-x
[12] Shatalov, R.L., Medvedev, V.A., 2022. Defor-
mation temperature conditions providing pre-
scribed property uniformity for steel vessels and
a device for non-destructive control. Metallur-
gist. 65(9-10), 1117-1124.
DOI: https://doi.org/10.1007/s11015-022-01254-w
[13] Shatalov, R.L., Medvedev, V.A., Bogdanov, S.V.,
2022. Development and use of an electromag-
netic unit for controlling the mechanical proper-
ties along the steel vessel height at the exit from
a rolling—pressing line. Russian Metallurgy
(Metally). (6), 628-633.
DOI: https://doi.org/10.1134/S0036029522060210
[14] Gamin, Y.V., Skripalenko, M.M., Romantsev,
B.A., et al., 2021. Prediction of billet fracture at
two-high screw rolling piercing. Metallurgist.
64, 1020-1028.
DOI: https://doi.org/10.1007/s11015-021-01093-1
[15] Goncharuk, A.V., Gamin, Y.V., Sharafanenko,
I.K., et al., 2020. Piercing of a billet in a mill
with guide disks. Russian Metallurgy (Metally).
1637-1642.
DOI: https://doi.org/10.1134/S003602952013011X
[16] Romantsev, B.A., Gamin, Y.V., Goncharuk, A.V.,
et al., 2017. Innovative equipment for producing
cost-effective hollow billets for mechanical-en-
gineering parts of small diameter. Metallurgist.
61, 217-222.
DOI: https://doi.org/10.1007/s11015-017-0480-2
[17] Aleshchenko, A.S., Quang, N., 2022. Studying
the wear resistance of the mandrels of a two-
high screw rolling mill. Metallurgist. 66(5-6),
650-656.
DOI: https://doi.org/10.1007/s11015-022-01372-5
[18] Iskhakov, R.V., Gamin, Y.V., Kadach, M.V.,
et al., 2020. Development of radial-shear rolling
mill special stands for continuous cast billets
deformation. IOP Conference Series: Materials
Science and Engineering. 966(1), 012074.
DOI: https://doi.org/10.1088/1757-899X/966/1/012074
Journal of Electronic & Information Systems | Volume 05 | Issue 02 | October 2023

67