Expert Systems With Applications 238 (2024) 122010

Contents lists available at ScienceDirect

Expert Systems With Applications

journal homepage: www.elsevier.com/locate/eswa

A decision-making tool for the determination of the distribution center

location in a humanitarian logistics network
Xenofon Taouktsis, Christos Zikopoulos *
Department of Business Administration, School of Economics, Aristotle University of Thessaloniki, Greece

A R T I C L E I N F O A B S T R A C T

Keywords: The distribution of humanitarian aid is a vital issue for humanity’s future. In recent years, the management of
Humanitarian logistics humanitarian crises has become more crucial than it was a decade ago. Due to the volatility and urgency that
Network science characterize such situations, one of the most important challenges globally is the optimization of decisions
Centrality index
regarding the timely distribution of aid during humanitarian operations. Our main goal is to develop an inno­
Traveling salesman problem
Binary classification problem
vative decision-making tool, essential for non-profit organizations and governments that aims at the prompt
Deep neural network selection of the location of the distribution center of humanitarian aid, in cases of natural or human-made di­
sasters. The proposed tool is based on network science principles and can be used for selecting a suitable node for
the installation of a distribution center during the beginning of a humanitarian crisis, considering that networks
have a volatile nature and require quick decisions. For the configuration of the proposed tool we use a combi­
nation of a classical heuristic algorithm and predictive models based on a binary classification problem with the
support of a supervised deep neural network. It is developed using the R programming language with the
contribution of the “Shiny” package (web application framework for R) along with other packages for network
analysis, data manipulation and visualization.

1. Introduction
The delivery of humanitarian aid is extremely important for the
future of humanity. In recent years, coping with humanitarian crises has
become a more crucial issue than it was a decade ago. The impact of
disasters due to physical phenomena, such as earthquakes, hurricanes,
floodings, conflagrations, epidemics or armed conflicts is huge, causing
several negative effects, ranging from financial problems to the loss of
people’s lives. Governmental together with non-governmental organi­
zations play a major role as far as the attempt to face these disasters and
deliver humanitarian aid are concerned (UN OCHA, 2018). According to
recent data of the Swiss Re Institute, in 2018, 304 disasters occurred,
with 181 of them being natural disasters and the remaining human-
made. Annually, 350 million people, on average, suffer because of di­
sasters and it has been observed that the number of emergency cases
requiring humanitarian aid has increased (Bevere et al., 2019).
One of the most important challenges worldwide, concerns the ac­
celeration of providing support through the timely distribution of hu­
manitarian aid (medicine, food, water and more) to affected areas (UN
OCHA, 2021). In Humanitarian Logistics (HL), the critical time window
concerns the first 72 h after the occurrence of a disaster. At the end of
that period, the possibility of survival is decreased dramatically. Thus,
human suffering is alleviated when emergency supplies are delivered in
the right amount to the right place in the right time (Beamon & Balcik,
2008; Dangi et al., 2012; Sreedharan et al., 2020).
In order to cope with a humanitarian crisis and offer humanitarian

Abbreviations: ACC, Accuracy; ANOVA, Analysis of Variance; AI, Artificial Intelligence; ANN, Artificial Neural Network; BC, Betweenness Centrality; BD, Big Data;
BCP, Binary Classification Problem; CIs, Centrality Indices; CC, Closeness Centrality; CM, Confusion Matrix; CNRA, Cumulative Node Ranking Accuracy; CTN,
Cumulative Total Networks; DSS, Decision Support System; DT, Decision Tree; DMT, Decision-Making Tool; DL, Deep Learning; DNN, Deep Neural Network; DGC,
Degree Centrality; DM, Disaster Management; DROs, Disaster Relief Operations; DCs, Distribution Centers; EC, Eigenvector Centrality; F1, F1-score; FL, Facility
Location; FLP, Facility Location Problem; FI, Farthest Insertion; F, F-value; HC, Harmonic Centrality; HA, Heuristic Algorithm; HN, Hidden Neuron; HL, Humanitarian
Logistics; HOs, Humanitarian Operations; HRC, Humanitarian Relief Chain; LC, Laplacian Centrality; LLRP, Latency Location-Routing Problem; ML, Machine
Learning; MCC, Matthews Correlation Coefficient; MS, Mean Square; NA, Network Analysis; NR, Node Rank; PPV, Precision; ReLU, Rectified Linear Unit; TPR,
Sensitivity; SC, Strength Centrality; SGC, Subgraph Centrality; SS, Sum of Squares; SCM, Supply Chain Management; TSP, Traveling Salesman Problem.
* Corresponding author.
E-mail addresses: xenofont@econ.auth.gr (X. Taouktsis), cziko@auth.gr (C. Zikopoulos).

https://doi.org/10.1016/j.eswa.2023.122010
Received 22 May 2023; Received in revised form 19 September 2023; Accepted 2 October 2023
Available online 9 October 2023
0957-4174/© 2023 Elsevier Ltd. All rights reserved.
X. Taouktsis and C. Zikopoulos
aid, Supply Chain Management (SCM) constitutes the core of Humani­
tarian Operations (HOs) and the related decision-making processes,
according to relevant studies such as Besiou and Van Wassenhov (2020),
Day et al. (2012), Lewin et al. (2018), Patil et al. (2022), Van Wassen­
hove (2006). Moreover, whilst no disaster can be avoided, their after­
math can be mitigated through adequate preparation before and
response during the crisis. Thus, as Negi (2022) argues, managing lo­
gistics operations properly and efficiently is crucial for the Disaster
Relief Operations (DROs).
Disaster Management (DM) is one of the leading fields involved in
the humanitarian dimensions of emergencies, necessitating the research
and development of newer and more effective ways of managing disaster
situations with the aim to reduce human suffering (Guha et al., 2022).
DROs can be managed by exploiting scientific knowledge from the
operation management field. A class of problems under the DM context
is Facility Location (FL), an essential strategic decision with broad
impact on pre- and post-disaster operations effectiveness and efficiency.
FL decisions involve the location and selection of Distribution Centers
(DCs), warehouses, medical centers and others (Boonmee et al., 2017).
The solution of complex routing or location-allocation problems can be
facilitated by the contribution of Big Data (BD), Artificial Intelligence
(AI) and intelligent optimization techniques (Dubey et al., 2019). Recent
advances in AI and in particular in Machine Learning (ML) and Deep
Learning (DL) improved our ability to address the severe and often
devastating impact of disasters. In particular, there are examples of
different ML and DL techniques applied in DM, in cases of several types
of disasters such as earthquakes, floods, hurricanes, landslides and
others (J.Z. Zhang et al., 2021; Linardos et al., 2022).
In this research paper, we develop a Decision-Making Tool (DMT) for
the selection of a suitable node for the installation of the DC during the
beginning of a humanitarian crisis. An important characteristic of our
work is that is considers networks that, as a result of a destruction, have
volatile form and require quick decisions regarding the location of the
DC and the configuration of the distribution plan, so as to achieve im­
mediate response to network requirements. The proposed DMT is
developed in R programming language (R Core Team, 2019), and its
operation is based on a combination of processes that include the prin­
ciples of network science along with the Centrality Indices (CIs), a
classical Heuristic Algorithm (HA) to solve the Traveling Salesman
Problem (TSP), and predictive models based on a Binary Classification
Problem (BCP) with the support of a supervised Deep Neural Network
(DNN).
At this point it is important to highlight the reasons that dictated the
specific combination of methods. In terms of the CIs, each of them is
designed to exploit a specific attribute of the network under question.
Therefore, these indices are customized to a particular application and
are useful for the identification of key nodes within the network. Using
multiple CIs together provides a better understanding of a node’s
importance and reduces the chance of bias that could occur if only one
CI is used. Consequently, this leads to an improvement in the quality of
the information, thereby facilitating a comprehensive and holistic
approach to decision-making regarding the issue at hand (Oldham et al.,
2019; Singh et al., 2020; Brandes & Erlebach, 2005).
When it comes to addressing optimization problems, such as the
classical TSP, HAs provide a viable alternative to the complexity
involved in mathematical programming models. In particular, their
distinct advantage is that they can rapidly traverse complex problems
and generate feasible solutions even if the problem size increases.
Consequently, these algorithms provide an effective trade-off between
optimality and the computational efficiency of the solution. Therefore,
heuristics are well-suited for practical application in real-world sce­
narios to facilitate more flexible and quick decision-making when
optimal solution quality is not a top priority (Abbatecola et al., 2016;
Kool et al., 2018; Rodríguez et al., 2018; Snyder & Shen, 2019).
Regarding DNNs, they are used in a range of real-world applications,
such as DM, and are utilized in a variety of ML problems, such as
2
Expert Systems With Applications 238 (2024) 122010
classification. It is worth mentioning that DNNs possess the ability to
analyze large amounts of complex data, which may include non-linear
relationships, and to identify different patterns within a short time­
frame with little human attention. These characteristics are critical in
real-time disaster scenarios because of the information they provide.
However, in various disaster situations, researchers and practitioners
also use traditional classification models, which serve their purpose in
decision-making but fall short in handling large volumes of data, data
complexity, and noise in the data (Abdolrasol et al., 2021; Aggarwal,
2018; Guha et al., 2022; Lecun et al., 2015; Sengupta et al., 2020; Sun
et al., 2020). It is worth noting that, according to Guha et al. (2022) a
future research direction is the exploration of newer ANN-based models
of post-disaster activities and the investigation of the potential of ANNs
in HL to enhance the effectiveness and speed of operations during crit­
ical situations. In the same direction, according to a recent study by
Farazmehr and Wu (2023), DNNs are commonly used for prediction and
only rarely used for relief distribution issues. Thus, future studies should
focus on the use of DNNs in humanitarian aid distribution. The contri­
bution of the current research can be summarized as follows:
• A hybrid approach is proposed that uses a DNN model for pattern
recognition by exploiting CIs that indicate which nodes are likely to
be good starting points for a classical HA to provide successful short
distance tours for aid distribution.
• A new approach for selecting the optimal location of the DC is
introduced, to achieve timely distribution to the affected areas of a
HL network.
• A combination of methods and techniques from different fields is
used, providing an innovative approach with the perspective of
improving the effectiveness and efficiency of HL decision-making
processes.
• It provides HL practitioners an alternative DMT based on the new
hybrid approach to apply in real-world HL network scenarios to
develop a robust distribution infrastructure.
• A user-friendly web-based application for timely decisions of the HOs
stakeholders.
The remainder of the research paper is organized as follows. In
Section 2 we provide the literature review on research and trends in HL.
Next, in Section 3 we present the steps of the research methodology used
in our paper. Subsequently, we perform the analysis using the synthetic
data in Section 4. In Section 5, we demonstrate the basic features of a
decision-making R Shiny web-based application from the user perspec­
tive. Finally, in Section 6, we summarize the paper, discuss the most
important conclusions and limitations of our study and provide some
directions for future research.
2. Literature review
2.1. Theoretical and empirical research in humanitarian logistics
Humanitarian aid is an area of growing concern due to the increase
of disasters and their impact in the coming decades. The field of HL
necessitates both conceptual and empirical studies. In addition, there is
a need to understand the challenges posed by different types of disasters,
identify the phases of disaster relief and configure the contribution of
humanitarian organizations (Kovács & Spens, 2009). Furthermore, it
was observed that research on certain phases of the disaster lifecycle,
namely mitigation, preparation and response, is more prevalent
compared to research on the recovery phase. Applied research utilizing
knowledge and experience from real-life scenarios and improved
collaboration between academia and humanitarian organizations are
necessary for successfully addressing the challenges related to HL (Leiras
et al., 2014).
Özdamar and Ertem (2015) conducted a survey on the response and
recovery planning phases of the disaster lifecycle. Emphasis was also
X. Taouktsis and C. Zikopoulos
placed on the use of technology, which is essential for successful HL
operations, especially when combined with a user-friendly interface and
appropriate mathematical models. However, the authors mentioned that
the use of technology has not yet prevailed in HL. Also, they recognize as
obstacles the nature of humanitarian organizations, which are charac­
terized by the lack of commonly accepted interoperability standards, as
well as the lack of real-time data that hinders the proper exploitation of
technology in HL.
Daud et al. (2016) highlighted the contribution of HL on DM,
focusing on the distribution of the essential materials and the optimi­
zation of HL processes. Τhere is also reference to the importance of the
research in HL and the support that it needs by organizations, govern­
ments and the society. Kunz et al. (2017) argue that, despite an
increasing trend of research publications on HL, there is a lack of im­
provements at a practical level. Also, they noted that limited collabo­
ration between academia and practitioners exists, and therefore access
to data is a major challenge in the humanitarian field. In particular, due
to the unpredictable nature of disasters it is rather difficult to collect
data during a disaster response operation. Thus, researchers often have
to rely on hypothetical scenarios or to collect data themselves.
Furthermore, they proposed ten rules to address the gap between HL
research and implementation in practical level. Negi (2022) identifies
several HL-related challenges for managing DROs. These challenges are
related to infrastructure, warehousing, procurement, transportation,
cooperation between stakeholders (humanitarian organizations, gov­
ernments, policymakers) and a number of socio-economic issues.
Furthermore, in this study the importance of the preparedness and
response phases in DM is also highlighted.
In a recent study, Rojas Trejos et al. (2023) highlight the need for
further research in the field of HL in terms of preparedness, response,
recovery operation systems and mitigation. Research focused on these
topics can lead to the emergence of novel strategies or technologies that
could improve HL efficiency. While mathematical modeling is a valuable
tool for developing decision-making models for HL, it is important to
examine alternative techniques and tools, such as social network anal­
ysis and discrete event simulation. Also, it is important to foster
collaboration between academia and humanitarian organizations for
more effective and efficient HL efforts that can meet the needs of
affected communities.
2.2. Quantitative methods and tools for decision-making in humanitarian
logistics
2.2.1. Network analysis and facility location problems
Network Analysis (NA) has its foundations in both graph theory and
matrix algebra. The main goal of NA is the quantification of the
contribution to the network of each node or each link of the network
according to certain indicators, termed as CIs. This type of analysis has
applications in a multiplicity of fields, such as, sociology, finance,
biology, medicine, transportation network management, SCM, emer­
gency response management, HL management, to mention a few. More
extensive discussion on the fundamental characteristics of networks, the
applications of NA, as well as the implementation of CIs can be found in
various research contributions, such as (Barabási & Pósfai, 2016; Bonchi
et al., 2016; Grando et al., 2019; Hua et al., 2019; Kumar & Jana, 2021;
Lee, 2012; Lujak & Giordani, 2018; Rodrigues, 2019; Tacheva & Simp­
son, 2019; Wandelt et al., 2020; Y. Zhang et al., 2011).
In spite of the fact that the theoretical foundation of networks is
widespread in many fields, there is scarce application in HOs. In
particular, network theory has been mainly used to study the impact of
social networks on community responses to disasters and rarely in the
context of delivery of humanitarian aid (Charles et al., 2022).
In emergency HL literature, the Facility Location Problem (FLP) has a
central role. FLPs have applications in a broad range of tactical and
operational decisions including relief distribution logistics management,
transportation routes configuration, vehicle movements planning,
3
Expert Systems With Applications 238 (2024) 122010
warehouse locations determination and more. Certain FLP models focus
on minimizing time, distance and transportation costs in response or
evacuation operations. Other FLP models are used for the determination
of the number of DCs that can serve all available demand points, taking
into account fixed or variable costs. Although there is a large stream of
research on the FLP, here we focus on contributions that examine
emergency FL decisions, employing NA methods for the study of the
related transportation networks (Boonmee et al., 2017; Y. Liu et al.,
2021). Novak and Sullivan (2014) proposed a new index based on
closeness centrality from complex network theory for the evaluation of
accessibility to emergency services via roadway network. The index is
called Critical Closeness Accessibility, a link-focused index unlike the
traditional node-focused accessibility indices and it is used to support FL
decisions and emergency route planning.
Moshref-Javadi and Lee (2016) introduced the Latency Location-
Routing Problem (LLRP), a customer oriented technique suitable for
the distribution planning of commodities from the DCs to affected areas,
as part of the relief activities in the aftermath of a disaster. The goal of
LLRP is to minimize latency through simultaneous optimization of
location, allocation and routing decisions. To deal with LLRP
complexity, they proposed and tested two algorithms with promising
solutions. The best solutions are generated by the algorithm that uses a
centrality-based heuristic approach. Gavião et al. (2020) proposed a
decision-making model to identify the most important nodes in a supply
chain for DM. The model uses CIs and a multiple-criteria decision-
making method. The aim of the model is to identify the optimal locations
for the installation of DCs and other logistics facilities to support HL
operations. Maghfiroh and Hanaoka (2020) proposed a multi-modal
relief distribution model for decision-making during the period imme­
diately after a disaster, when transport resources are limited. They used
degree centrality index to estimate whether a node is important and it is
connected to another node by different modes of transportation.
In terms of the extent of the network under examination, according
to the literature, for the humanitarian aid distribution a single vehicle
can cover a maximum of 20 to 40 affected areas (Anuar et al., 2021;
Rivera et al., 2016). Of course the number of areas included in the
network varies in different research contributions. For example, there
are ten earthquake-affected districts of Tehran, Iran for the application
of a bi-objective stochastic model for medical distribution (Mohamadi &
Yaghoubi, 2017), thirteen affected areas in Yogyakarta, Indonesia
(Maghfiroh & Hanaoka, 2020), twelve hypothetical affected areas for
the application of a DC selection model (Gavião et al., 2020), 22
earthquake-affected districts of Tehran for the application of a robust
fuzzy stochastic programming model for designing a relief commodity
supply chain network (Kebriyaii et al., 2021), 96 flood-affected areas in
Surat Thani, Thailand (Praneetpholkrang & Kanjanawattana, 2021),
fifteen earthquake-affected areas in China (L. Zhang et al., 2022), 20
hypothetical affected areas for a multi-period FLP in disaster relief with
demand uncertainty (K. Liu et al., 2022), twelve hypothetical affected
areas for the application of a bi-objective mathematical model for pro­
duction and routing of inventory during the aftermath of a disaster
(Zargary & Samouei, 2022).
2.2.2. Artificial neural networks
With recent developments, many studies explore AI, ML, and DL
techniques for analyzing and processing various types of BD into
actionable information for effective DM (Nunavath & Goodwin, 2019).
In addition, recent studies show that Artificial Neural Network (ANN)
applications are more efficient due to their ability to handle complex
processes in a restricted time frame. Thus, ANN techniques in DM
models have gained popularity since they achieve better accuracy than
the existing traditional ML techniques (Guha et al., 2022). Grando et al.
(2019) and Wandelt et al. (2020) worked on the combination of network
science and ML with the implementation of ANNs for predicting the
most important nodes using CIs. With these ANNs prediction models, the
classification of the nodes according to their importance was carried out
X. Taouktsis and C. Zikopoulos
without using the mathematical formulas of each index separately. They
used synthetic and actual networks for modeling with encouraging re­
sults concerning prediction ability. Xu et al. (2020) worked on the
combination of dynamic programming and ANNs to solve combinatorial
optimization problems, such as the TSP, and evaluated the performance
and computational time of the proposed framework. In particular, they
observed some performance issues regarding the optimal solution, but
noted improved computational time in optimization problems.
Praneetpholkrang and Kanjanawattana (2021) studied the issue of
making quick decisions in shelter location-allocation problems, which is
a critical part of the disaster response phase for humanitarian relief lo­
gistics. They minimize total cost and evacuation time simultaneously as
a multi-objective optimization problem. They combined the Epsilon
Constraint method, which generated the optimal solutions, but with
high computational time, and a supervised ANN based on the optimal
solutions as input to facilitate quick decision-making by reducing the
computational time. The proposed methodology was applied in a case
study of 96 flood-affected areas in Surat Thani, Thailand. Kim et al.
(2022) proposed a model that combines complex network and DL
models to predict water levels during the pre-flood period. They utilized
degree centrality index to analyze the complex network along with DNN
and Long Short-Term Memory models. The combined model demon­
strated excellent performance in predicting the daily water level vari­
ability. The authors argue that their model has adequate potential to
improve disaster response systems and mitigate the impact of natural
disasters, especially in regions where disaster response systems are not
in place.
2.2.3. Decision-making tools
A related research stream that has appeared in recent years is that
consisting of research contributions which propose DMTs in the field of
HL and disaster response. In particular, Fikar et al. (2016) developed a
Decision Support System (DSS) for the coordination of a critical
component of DROs, the distribution of disaster relief. The DSS is based
on a combination of a Mixed-integer Problem modeling, an Agent-based
simulation of the behavior of different stakeholders, a Heuristic-based
Scheduling and Routing procedure, as well as a Tabu Search meta-heu­
ristic, which is used to search for optimal solutions. As a result, the
combination of these techniques in a DSS can play a vital role in sup­
porting the decision-making by analyzing complex disaster scenarios for
the timely delivery of goods needed in the affected areas.
In a similar direction, Sahebjamnia et al. (2017) developed a hybrid
DSS for the configuration of Humanitarian Relief Chain (HRC) based on
a simulator which calculates performance measures in different disaster
scenarios. Also, there is a rule-based inference engine that checks the
necessary conditions and suggests possible improvements for the
optimal configuration with respect to facilities location, relief items
allocation and distribution planning. In addition, there is a knowledge-
based system to store the optimal configuration, so as the HRC managers
are able to retrieve it. The DMT developed by Cavdur and Sebatli (2019)
can be used for allocating temporary disaster response facilities for the
distribution of relief supplies. This is similar to an open-source spread­
sheet-based DSS by Erdoğan et al. (2019) for FLPs that can be used in the
field of HL.
Another decision-making information system for logistics planning is
proposed by Warnier et al. (2020) and can be used for determining the
access point to affected areas after a disaster. The information system
combines OpenStreetMap, the most widely used geographic information
system, to generate an initial map of the affected areas, and open hu­
manitarian data to provide a near real-time overview of the accessibility
of affected areas. The modeling of the information system is based on a
set of CIs, which are used to identify the most critical nodes and edges.
The DSS is applied in a case study concerning the 2018 earthquake in
Papua New Guinea. Chang et al. (2022) configure a simulation-based
DMT for dynamic post-disaster pedestrian evacuation. This tool can be
used to improve evacuation strategies by exploiting real data from
4
Expert Systems With Applications 238 (2024) 122010
streets and emergency shelters. In addition, it can predict the flow and
number of pedestrians to be evacuated, and can provide details on the
impact of earthquakes on urban infrastructure.
Our work differs from the existing body of knowledge by offering an
innovative and creative approach, combining several techniques and
methods from different fields for the first time, to address FLP in HL
networks. Specifically, by using CIs and a classical HA with the support
of a DNN, our approach provides a unique way of decision-making for
the selection of DC installations for the distribution of aid to affected
areas.
3. Methodology
3.1. Problem description and research hypotheses
The objective of our study is the development of an application
which offers to HOs stakeholders the ability of immediate response for
the distribution of humanitarian aid to the affected areas directly after
the occurrence of a disaster. Specifically, we need to address a FL routing
problem to enable stakeholders to easily identify the suitable node for
the installation of the DC to be used as both the starting and return point
of the vehicle for the distribution of humanitarian aid. In such situations,
it is crucial to avoid the time-consuming process of modeling the volatile
problem in each disaster event with classical mathematical program­
ming models. Instead, it is more effective to use CIs designed to exploit
specific structural characteristics of the network under question in
combination with a HA that can quickly traverse a complex network and
provide feasible routing solutions even as its size increases. Combining
the CI results with these routing solutions under the umbrella of a DNN,
we are able to develop a predictive model for DC location selection with
minimal human intervention. An important characteristic of this model
is the reduction of the total number of alternative solutions examined,
by properly selecting only a proportion of the total number of nodes as
candidate nodes for DC location. The development of a user-friendly
web-based application which incorporates all of the above further fa­
cilitates timely decision-making by the stakeholders. Following are the
research hypotheses of our study.
The humanitarian network examined consists of fifteen affected
areas (represented as nodes of the network). This number is similar to
the number of nodes that are usually used in HL network modeling in
similar contributions, as explained in Section 2. In this network, there is
one vehicle to perform the distribution starting from the DC and
returning back to it. We are interested in the selection of the appropriate
node for the location of the DC among the fifteen available. However, by
training and using a DNN we can reduce the number of candidate DC
location to just a proportion of the total number of nodes in the network.
According to Moshref-Javadi and Lee (2016) in disaster relief logistics
the cost for DCs is negligible considering the importance of saving
humans’ lives, contrary to the commercial logistics distribution systems
were the respective cost is important. However, in HL, the distance
traveled for executing the distribution task is important, not in terms of
cost but it terms of distribution time. Since small delays are not
important compared to other issues that may occur, we consider
acceptable not only the first but also the second lowest distribution
distance, as estimated by applying a TSP approach using a HA. Although
there are many categories, variations of methods and algorithms used
for solving a TSP, here, we focus on a НА, known as the Farthest
Insertion (FI). The idea of all insertion algorithms is to start a tour that
includes an arbitrary node and at each step to choose a node that is not
yet in the existing tour. After that, the node is inserted into the tour
between two other nodes that are consecutive so that the insertion cost is
minimized. When all the nodes are included in the tour the algorithm
stops. The main idea of FI algorithm is to create an early outline of the
entire tour by linking remote nodes into the tour at first (Anbuudaya­
sankar et al., 2014; Bresson & Laurent, 2021; Matai et al., 2010; Rose­
nkrantz et al., 1977).
X. Taouktsis and C. Zikopoulos
It is important to mention that we chose this particular algorithm
because according to the related literature (e.g. Cook et al., 1997;
Hahsler & Hornik, 2007; D. S. Johnson, 1990; Kruithof, 2012; Righini,
2000), has time complexity O(n2 ) and it provides better results
compared to other HAs. In particular, in our study the use of a HA with a
relatively small optimality gap in the solution and reasonable compu­
tational time was considered preferable compared to time-consuming
methods such as mathematical programming models that could hinder
decision-making in emergency situations. In the classic example of
“CAR54” instance where the best TSP tour of 33 cities had to be found,
the FI tour solution was 1.3% worse than the optimal distance and the
best option among other heuristics such as the Nearest Neighbor,
Nearest Insertion, Cheapest Insertion, Convex Hull, Minimum Spanning
Tree, and Christofides (Snyder & Shen, 2019). In addition, in another
study with TSP instances using networks with 20, 50 and 100 nodes, the
gap between the FI solution and the optimal solution was 2.64%, 5.62%
and 7.71%, respectively, in reasonable computational time (Ouyang
et al., 2021). In our case, where we are going to address a crisis by
distributing humanitarian aid in a small and near-realistic network of
fifteen areas with one vehicle, the FI is expected to provide the best
results among the aforementioned heuristics.
Our modeling approach also includes a supervised DNN, which is a
type of ANN that consists of a more complex structure and includes
multiple Hidden Neuron (HN) layers compared to a classical neural
network that usually has up to two layers (Ravi et al., 2017). In
particular, we use a Multi-Layer Perceptron-based model (Candel et al.,
2022). It is worth noting that DNN models can be effectively used in
various real-world cases of binary and multi-class classification prob­
lems, such as ransomware classification, fake news classification, jam­
ming attack detection, and diabetic disease prediction. In many cases,
DNNs can outperform conventional shallow classifiers, such as Decision
Trees and Random Forest, showing greater generalization ability and
achieving higher levels of accuracy when the test dataset is not part of
the original training dataset (Ahuja et al., 2019; Greco et al., 2021;
Ranjeeth et al., 2020; Rusli et al., 2020; Vinayakumar et al., 2017). It is
worth highlighting that for better training and improving the predictive
ability of DNNs, data size plays an important role, as reported by Cho
et al. (2015) and confirmed by Orimoloye et al. (2020). In terms of
handling imbalance data, according to Johnson and Khoshgoftaar
(2019), ML techniques can be applied to DL models. It is also worth
noting that almost all research in this area has concentrated on Con­
volutional Neural Networks for computer vision tasks. They report that
DL applications to address class imbalances in non-image datasets are
quite rare. Therefore, there is a need for further extension of this area of
research. For these reasons, we made the choice to adopt a DNN model
in our study. However, in order to demonstrate the suitability of our
selection, after developing and selecting the appropriate DNN model
Fig. 1. A schematic description of the proposed methodology’s steps.
5
Expert Systems With Applications 238 (2024) 122010
incorporated in the proposed DMT, in Section 4.3 we compare the per­
formance of the DNN model with that of Decision Tree models. This
comparison sets the stage for future research and discussion. In what
follows, we present and describe the methodology followed for the
configuration of the DMT for the location of the DC.
3.2. Steps of methodology
In order to clarify the proposed methodology and to enhance the
overall understanding of the research approach’s structure, a schematic
description of its steps is illustrated in Fig. 1, along with its compre­
hensive description.
In the first step, we generate 70,000 synthetic random undirected
networks with fifteen nodes per network and density, i.e., the ratio of
actual over potential links in a network, between 0.18 and 0.91. We set
the distance (sij) between pairs of connected nodes equal to a random
number uniformly distributed between 5 and 520 km. Since the objec­
tive of distribution planning is the minimization of travel time or dis­
tance, we define the weights of the links connecting nodes i and j, wij , as
the reciprocal of their distance, wij = 1/sij .
In the second step, for each node of the generated networks, we
calculate certain CIs, such as Degree, Strength, Closeness, Harmonic,
Betweenness, Eigenvector, Laplacian and Subgraph. For compactness of
the exposition, we include the presentation of the CIs used in our anal­
ysis and their respective mathematical expressions in Appendix A.
Extensive discussion of these indices can be found in various sources (e.
g., Das et al., 2018; Estrada & Rodríguez-Velázquez, 2005; Koschützki
et al., 2005; Lü et al., 2016; Opsahl et al., 2010; Porta et al., 2006; Qi
et al., 2012; Wan et al., 2021).
In the third step, we aim to examine the suitability of each node of
the generated networks as a potential DC location and distribution
starting point with respect to the total distance. For this purpose, we
consider the TSP approach, which is a widely used routing problem with
the goal to minimize the distance travelled for visiting all available
nodes of a network only once and then returning to the initial node. In
our approach, we apply the FI HA to modified networks by converting
the initial networks to complete ones to ensure the existence of the
Hamiltonian Cycle, hence the existence of TSP solutions. The modifi­
cation process of the initial networks into complete networks is
described next. In particular, we calculate the length of the shortest path
between each pair of nodes that are not connected directly, and we use
the results of the shortest path matrix as a weighted adjacency matrix to
build a complete network using virtual links between nodes. The virtual
links have lengths equal to the shortest paths between nodes in the
initial network. For example, consider the case where we need to move
from node A to node C and there is no direct link. If there are links
connecting A and C via another node, e.g., B, we introduce a virtual link
X. Taouktsis and C. Zikopoulos
Table 1
Confusion matrix of actual and predicted values of two classes.
Actual Positive (P) Actual Negative(N)
Predicted Positive (p) True Positive (TP) False Positive (FP)
Predicted Negative(n) False Negative (FN) True Negative (TN)
A-C with length equal to the route A-B-C in the initial network. The
resulting complete network enables the application of the FI algorithm
and, hence, the calculation of different TSP solutions using each node
separately as a starting point. We then decode the proposed TSP paths
into the initial network, which may involve repeated visits to some
nodes that were part of the shortest path in the initial network. This
process is applied to all generated networks in order to build a dataset
that includes the TSP solutions for each node and the decision class label
for Selecting (SL) or Rejecting (RJ) a node as DC in each network. It
should be noted that a node is classified as SL if it yields the first or the
second lowest distribution distance when selected as a DC. The resulting
dataset is imbalanced, with SL nodes being the minority.
In the fourth step, we combined the data results from steps two and
three into a single dataset. Specifically, we merged the results of the CIs
data of each node of the 70,000 networks, corresponding to 1,050,000
nodes, with the TSP solution data of the distances from each node as a
possible DC and distribution starting point. In fact, at this point, we start
working with a dataset of individual nodes and their results without
considering the network as a whole.
In the fifth step, we developed a DNN-based prediction model for
the selection of the installation node of the DC of the HL network. The
modeling is based on the BCP, and for the development of predictive
data models, aim to SL class according to the dataset created during the
fourth step. The input data includes the CIs and the decision classes SL or
RJ for each node of each network. The former constitute eight inde­
pendent variables and the latter the dependent variable representing the
decision output for the DC selection. We split the dataset into 756,000
nodes for model training, 189,000 for validation, and 105,000 for
testing. The prediction model is trained with stochastic gradient descent
using back-propagation. We developed different trial predictive models
with a range of features to identify the best model for the optimal so­
lution of the DC selection. Specifically, we used eleven different HN
scenarios (9, 18, 27, 36, 45, 54, 63, 80, 100, 150 and 200). Each of the
eleven scenarios had three hidden layers and epochs ranging from 1 to
1,000, which play an important role in the training process of a model.
We conducted 100 experiments per scenario to configure the DNN
models, resulting in 1,100 experiments in total. The Rectified Linear
Unit (ReLU) was used as activation function for DNN training. The ReLU
is widely used because of its simplicity and the superior training per­
formance compared to other activation functions. Also, the Early Stop­
ping technique was used to train the DNN more efficiently, avoiding
long model training that can cause overfitting. Specifically, Log-loss,
known as cross-entropy loss, was used, which is suitable for BCP and
acts as a regularizer to reduce the risk of overfitting. Extensive discus­
sion about DNN model configuration can be found in various sources (e.
g., Abdullah & Kanaya, 2019; Afaq & Rao, 2020; Candel et al., 2022;
Hossen et al., 2017; Rasamoelina et al., 2020; Wang et al., 2019; You­
sefzadeh et al., 2021).
In the sixth step, after training several trial prediction models, we
evaluated the performance of the prediction models on the validation
dataset. In particular, we used the Confusion Matrix (CM), which pro­
vides information about the prediction performance in conjunction with
five well-known significant metrics, according to Chicco and Jurman
(2020), Liu et al. (2018) and Tharwat (2021), such as Accuracy (ACC),
F1-score (F1), Matthews Correlation Coefficient (MCC), Precision (PPV),
and Sensitivity (TPR). The interpretation of a CM according to the actual
observed and predicted values is presented in Table 1. A brief overview
of the model performance metrics used in our analysis is presented in
Appendix B.
6
Expert Systems With Applications 238 (2024) 122010
In the seventh step, the process of selecting the candidate DNN
prediction models is performed. Specifically, it involves the calculation
of the mean values for each scenario of each experiment of HNs in
combination with a statistical evaluation by using the one-way Analysis
of Variance (ANOVA) to examine if there are statistically significant
differences between the alternative scenarios of HNs examined with
respect to the values of each performance metric used to evaluate the
overall performance quality of the candidate DNN models. In addition,
utilizing each metric, we decided to examine those experiments of the
HN scenarios where the mean values of each metric are maximized. To
avoid an exhaustive analysis of the huge number of experiments, we
limited the selection to the two best experiments as potential DNN
models per metric. However, since we did not observe large differences
between the suitability of the alternative metrics examined, we decided
to use the MCC metric for the final selection of potential models to focus
on. This process resulted in the selection of four models, corresponding
to the 50% of the available candidates DNN models selected. The MCC
metric was selected because, according to Boughorbel et al. (2017) it is
more robust to imbalanced data. Also, Chicco and Jurman (2020) and
Chicco et al. (2021) report that the MCC is more reliable and informative
compared to the aforementioned metrics, and therefore it produces
more explanatory results in the evaluation of BCPs. In Section 4, there is
a detailed presentation and analysis of this step.
In the eighth step, we evaluate the predictive DNN models on the
validation dataset, where we address each network as a system of nodes
rather than as a dataset of individual nodes, as was the case in the
previous steps. In particular, we aim to evaluate whether these models
can predict the lowest possible distribution distance using only a third of
the total number of nodes in each network. In our study, we have 12,600
HL networks in the validation dataset and we evaluate the cumulative
prediction accuracy on these networks. Specifically, we examine in how
many networks we were able to identify the appropriate DC location
using one-third of the nodes proposed by our prediction model as po­
tential DCs.
In the ninth step, the best DNN model selected according to its
performance on the validation dataset is now applied to a test dataset
with unseen data that was neither used in training nor for validation.
The goal is to observe the performance of the model on unseen network
data to draw a conclusion about its prediction ability. In practice, during
this ninth step, we implement the processes included in the sixth and
eighth steps, but this time we use the test dataset. If the evaluation shows
that the predictive ability of the model is close to the same level as the
performance in the validation dataset, then we conclude. If there are
large outliers, we re-examine steps 6 to 9 until we identify the best
possible model.
3.3. Modeling and development tools
Several tools have contributed to the implementation of the meth­
odology and the development of DMT. In particular, the R programming
language (R Core Team, 2019) in combination with the user-friendly
environment RStudio (RStudio Team, 2021) for coding and the
“Shiny” (W. Chang et al., 2021) package for the development of a web-
based decision-making application. We used various R packages for data
generation, analysis and visualization. Specifically, we calculated the
essential CIs with “igraph” (Gabor & Tamas, 2006) along with the
“CINNA” (Ashtiani et al., 2019) and “centiserve” (Jalili et al., 2015)
during NA. Also, we used the “TSP” (Hahsler & Hornik, 2007) in order to
use a suitable HA for obtaining the TSP solutions. Moreover, we used the
“h2o” (H2O.ai, 2020) for the DNN model development and analysis,
“dplyr” (Wickham et al., 2021) for data manipulation, “ggplot2”
(Wickham, 2016) for data visualization and “visNetwork” (Almende
et al., 2019) for network visualization.
X. Taouktsis and C. Zikopoulos
4. Data analysis
4.1. Deep Neural network model selection
In our study, as explained in Section 3, we train different DNN pre­
diction models for different HN scenarios and we evaluate the predictive
ability of each model for specific BCP-related metrics. Then, for the final
model selection we conduct several experiments and calculate the mean
values of the selected performance metrics with respect to the number of
the HN used, as illustrated in Fig. 2. In addition, we perform a one-way
ANOVA to obtain more information on the DNN models’ performance.
Specifically, Fig. 2 (A) illustrates that ACC performs better with 200
HNs, with the difference between the minimum and maximum values
being quite small, ranging from about 0.7010 to 0.7025. However, there
is evidence of a slight upward trend as the number of HN increases with
the improvement in the performance of the models. According to Fig. 2
(B), the F1 ranges from about 0.400 to 0.406. The best performance of
the model is obtained at 18 HNs. Again, the difference between the
minimum and maximum values is relatively small, but with a slight
downward trend as the number of HN increases. Given the data pre­
sented in Fig. 2 (C), our visualization highlights that MCC performs
better, with 54 HNs. Once again, the difference between the minimum
and maximum values is relatively small, ranging from about 0.2425 to
0.2445. However, in this case, there is no clear overall trend (upward or
downward). According to the data shown in Fig. 2 (D), the PPV ranges
from about 0.5600 to 0.5675. The best performance of the models was
achieved at 200 HNs, and we observe a slight upward trend as the
number of HN increases. The difference between the minimum and
maximum values is relatively small. Based upon the findings shown in
Fig. 2 (E), the TPR performs better, with 9 HNs. The difference between
the minimum and maximum values is also quite small, ranging from
about 0.308 to 0.320, but with a slight downward trend as the number of
HN increases.
Overall, the plots show that the performance of the DNN models is
not very sensitive to the number of HNs used. In particular, for all the
metrics we consider, the performance is relatively stable with some
Fig. 2. Mean results of the five metrics for each scenario of every experiment of hidden neurons.
Table 2
The one-way ANOVA results for each metric.
Hidden Neurons / Residuals
METRIC SS
ACC 2.10e-04 / 1.92e-03
F1 5.17e-03 / 1.54e-01
MCC 3.37e-04 / 1.20e-02
PPV 4.63e-03 / 5.60e-02
TPR 1.11e-02 / 3.13e-01
Expert Systems With Applications 238 (2024) 122010
slight variations across the different HN scenarios.
Next, in order to better explore the importance of HNs for DDN
prediction models, we examine their behavior using one-way ANOVA. In
particular, we want to determine whether there are statistically signif­
icant differences in the mean values of each of the five metrics consid­
ered with respect to the different HNs scenarios used. Table 2
summarizes the results of the one-way ANOVA. Specifically, the table
includes the Sum of Squares (SS), the Mean Square (MS), the F-value (F),
and the p-value for each metric. The degrees of freedom are equal to 10
for the factors (HNs) and 1,089 for the residuals. Also, the pre-defined
significance level is set at 0.05.
As we observe in Table 2 all five metrics have a very low p-value,
which means that there is a significant difference in performance be­
tween the different scenarios of НN, showing a statistically significant
effect of HNs on the prediction evaluation metrics of the models.
Therefore, we conclude that changes in the number of HNs affect the
performance of the models. However, it is not clear that the performance
will be improved by increasing or decreasing the HNs in the predictive
DNN models since the improvement is relatively small and not always
consistent for all metrics. Therefore, in our study, we choose to use the
experiments of the HN scenarios where the mean values of each metric
are maximized according to the presented plots. Specifically, with 9, 18,
54 and 200 HNs. From the 400 in total experiments of the four НN
scenarios, we decided to choose the top two results for each metric,
resulting in eight models, presented in Table 3 in a descending order by
the MCC metric. Each result is a potential prediction DNN model.
As we mentioned in Section 3.2, according to literature, the MCC
metric is more suitable and reliable for imbalanced data. Model 1, cor­
responding to Experiment 58 of 200 HNs, holds the highest value of the
MCC metric. Therefore, for the validation of the models, we will focus on
the first four models of Table 3. That is, we then apply these four models
to a number of 12,600 networks, examine each network as a system
rather than as a dataset of individual nodes, as we did previously. The
aim of this process is the prediction of the lowest possible distribution
distance to support each HL network for DC installation according to the
initial node proposals from the DNN models. Then, the HA uses the
MS F p-value
2.10e-05 / 1.70e-06 11.93 <2.00e-16
5.17e-04 / 1.42e-04 3.646 8.45e-05
3.37e-05 / 1.11e-05 3.052 7.91e-04
4.63e-04 / 5.14e-05 9.009 2.23e-14
1.11e-03 / 2.87e-04 3.868 3.58e-05
7
X. Taouktsis and C. Zikopoulos Expert Systems With Applications 238 (2024) 122010

Table 3
Metric results from experiments with 9, 18, 54 and 200 hidden neurons.
MODEL ACC F1 MCC PPV TPR EXPERIMENT HNs

1 0.69846 0.44552 0.25676 0.54841 0.37514 58 200

2 0.69934 0.43987 0.25558 0.55209 0.36557 36 200
3 0.69558 0.45247 0.25556 0.53971 0.38950 52 9
4 0.69575 0.44973 0.25441 0.54060 0.38501 64 18
5 0.70396 0.40128 0.24858 0.57838 0.30721 56 200
6 0.70399 0.39535 0.24657 0.58078 0.29967 93 54
7 0.70307 0.37270 0.23712 0.58645 0.27314 30 200
8 0.70288 0.35956 0.23272 0.59152 0.25828 74 200

Table 4
Accuracy of the Models 1 to 4 on validation dataset of networks.
Model 1 Model 2 Model 3 Model 4

NR CTN CNRA CTN CNRA CTN CNRA CTN CNRA

1 3,906 31.00% 3,988 31.65% 4,083 32.40% 4,064 32.25%

2 6,191 49.13% 6,242 49.54% 6,236 49.49% 6,294 49.95%
3 8,055 63.93% 8,073 64.07% 8,002 63.51% 8,046 63.86%
4 9,430 74.84% 9,394 74.56% 9,286 73.70% 9,425 74.80%
5 10,508 83.40% 10,460 83.02% 10,213 81.06% 10,447 82.91%

proposed nodes to solve the TSP by using only a proportion of the total
number of nodes. Specifically, Table 4 presents the results of DNN
models of the prediction accuracy in accordance of the proposed nodes
ranking in the validation dataset of networks. The table includes the
Node Rank (NR), the Cumulative Total Networks (CTN) and the Cu­
mulative Node Ranking Accuracy (CNRA).
As we observe, it is important to highlight that Model 1 outperforms
the other three models in terms of finding the optimal solution under our
hypotheses, using the first five proposed nodes out of a total of fifteen
nodes in each network. In particular, Model 1 has the highest percentage
of optimal solutions, satisfying 83.40% of the total networks. The results
show that we can identify the optimal location of the DC in 10,508 out of
the total 12,600 networks, using only one-third of the total nodes of a
network at a time.
We select Model 1 (Experiment 58 of the 200 HNs) because it seems
to be the best choice for identifying suitable nodes as DC candidates to
minimize distribution delivery times, according to the analysis we per­
formed on the validation dataset of 12,600 networks, and we will pro­
ceed to test it on a test dataset of new networks in the following section.
4.2. Deep Neural network model implementation
Once the model is selected, it is applied to a new dataset of 7,000
networks that were not used in training or validation. In particular, in
Table 5 we generate the CM for the predictions concerning the 105,000
nodes of the 7,000 networks as they were determined according to the
Model 1. We have 33,842 data points which concerns the selection of a
node and 71,158 data points for the rejection of a node, resulting to an
imbalanced dataset.
Also, the CM results are evaluated in terms of five metrics, the results
of which were, ACC (0.69853), F1 (0.44775), MCC (0.25754), PPV
(0.54660) and TPR (0.37917). Therefore, comparing the results of
Model 1 on the validation dataset (Table 3) with those of the test dataset,
we observe that they have small differences in the metric results. This is
an indication that the selected model is not overfitting and can gener­
alize to account for new data.
Table 5
Confusion matrix of selected and rejected nodes of the test dataset.
Actual “Select” Actual “Reject”
Predicted “Select” 12,832 10,644
Predicted “Reject” 21,010 60,514
Table 6
Accuracy results of Model 1 on test dataset of networks.
NR CTN CNRA
1 2,244 32.06%
2 3,515 50.21%
3 4,491 64.16%
4 5,263 75.19%
5 5,839 83.41%
We then apply the DNN of Model 1 to the test dataset of 7,000 net­
works not as separate nodes but as a network consisting of fifteen nodes
each. Specifically, Table 6 presents the results of the prediction CNRA in
accordance of the proposed nodes ranking in the test dataset of networks
by using only in one-third of the total nodes of a network at a time.
Using Model 1 we observe that we satisfied 83.41% of the total
networks in the test dataset of networks. In particular, the results show
that we found a solution for the optimal installation of the DC with the
lowest possible distribution distance in 5,839 out of the total 7,000
networks, using only the first five proposed nodes from each network at
a time.
Overall, based on the results of both the validation and test dataset of
networks, the DNN Model 1 performed quite well, achieving an accuracy
of 83.40% and 83.41%, respectively, in predicting the optimal DC
installation node with the lowest possible distribution distance. Thus,
the DNN model can be a useful DMT for predicting the optimal instal­
lation of DCs for aid distribution in a HL network.
4.3. Comparison of deep neural network model with decision tree models
Having select the most suitable DNN model (Model 1, Experiment 58
with 200 HN), we proceed to a test of a typical classifier model on the
test dataset to compare the performance of DNN. This test will be carried
out both at the node level and at the network level. Specifically, we will
study the performance in our BCP problem of a well-known classifier
model, the Decision Tree (DT). Generally, DTs are used in supervised
learning as predictive models and are considered powerful and
comprehensible ML models.
For the DT model development, we used again the “h2o” (H2O.ai,
2023) package, which implements an algorithm for building the tree
structure of the model. In particular, a DT model operates through a
systematic process to make predictions, as will be briefly outlined next.

8
X. Taouktsis and C. Zikopoulos Expert Systems With Applications 238 (2024) 122010

Table 7
Comparison of alternative performance metrics of decision tree models with the
maximum depth equal to 5, 10, 15 and 20 with the selected DNN model on the
test dataset of nodes.
MODEL ACC F1 MCC PPV TPR
DEPTH-5 0.68913 0.45039 0.24411 0.52351 0.39519
DEPTH-10 0.69295 0.42064 0.23489 0.53673 0.34584
DEPTH-15 0.68922 0.32104 0.18651 0.54255 0.22797
DEPTH-20 0.67428 0.26574 0.13006 0.48591 0.18288
DNN 0.69853 0.44775 0.25754 0.54660 0.37917
In our BCP, the model can predict whether a node is SL or RJ based on
the features of the input data, namely the CIs. The predictions are made
by traversing the tree structure from the root node to a leaf node along a
path defined by the feature conditions of the dataset. Specifically, this
evaluation compares the value of the feature to a threshold. This
threshold is set during the training phase of the model to minimize en­
tropy, a measure of the impurity or disorder in a set of samples. Based on
the comparison of the CI values to the corresponding thresholds, the
model selects the appropriate branch of the tree and navigates to the
corresponding child node. These operations are repeated on interior
nodes until a leaf node is reached as the terminal node. Then, the pre­
dicted class label is assigned by the DT model as the final output for that
node.
For the comparison of the DT model with the DNN model, we
develop different DT prediction models, differing in the maximum depth
of each. It is important to note that if a DT is permitted to grow unre­
strictedly, it may become excessively complex, potentially overfitting
the training data. Thus, the parameter for defining the maximum depth
is a mechanism for controlling the complexity of the tree. In particular, it
can limit the number of splits that entropy directs the DT to make, to
reduce impurity by acting as a stopping criteria. This makes the model
more generalizable to new data and helps avoid overfitting. Extensive
discussion about DTs configuration can be found in various sources (e.g.,
Blanco-Justicia et al., 2020; Costa & Pedreira, 2023; Greco et al., 2021;
H2O.ai, 2023; Izza et al., 2020).
Specifically, four different DT models with maximum depths of 5, 10,
15, and 20 are tested. Each model is trained with the 756,000 node
dataset that we previously used in training of the DNN model. Table 7
summarizes the results of the various DT models on the test dataset of
nodes, comparing each performance metric to the previously reported
results of the selected DNN model.
As we observe, the DNN model has the highest ACC (0.69853), MCC
(0.25754), and PPV (0.54660) values. Also, the DNN model has the
second highest F1 (0.44775) and TPR (0.37917) values. As we discussed
in Section 3.1, it was expected to observe superior performance from the
DNN model in our BCP, given that it involves an imbalanced dataset.
Focusing on MCC and PPV metrics and given that we have a dataset with
SL as a minority class, we can say that the DNN model can make accurate
positive predictions of the SL class while maintaining a balanced per­
formance in both classes (SL and RJ) compared to the DT models. In
terms of the DT models, we observe that as we increase the tree depth,
the MCC metric decreases, so the ability to maintain balanced
performance in both classes (SL and RJ) appears to weaken. The same
behavior is observed with the F1 metric, that corresponds to the ability
to maintain a balance between precision and recall (PPV and TPR),
which seems to weaken as well. TPR metric, which is related to the
ability to identify the SL class effectively, appears to weaken too. On the
contrary, PPV metric increases in the depth of the DTs for the models
with maximum depth equal to 5, 10 and 15. This behavior suggests
fewer FP predictions, which may contribute to a more accurate predic­
tion of the minority SL class. It is worth noting that for the DT model
with the maximum depth used in our comparison, i.e., equal to 20, we
observe the worst performance. This behavior, can be considered an
indication of overfitting of the model, leading to reduced performance
with inaccurate predictions and therefore it cannot be considered
reliable.
We then apply the DT models to the test dataset of 7,000 networks,
addressing each network as a whole rather than as a dataset of indi­
vidual nodes, as we did previously. In particular, Table 8 presents the
results of the prediction CNRA in accordance to the proposed nodes
ranking in the test dataset of networks by using only one-third of the
total nodes of a network at a time.
We observe that the DT DEPTH-15 model achieves the highest CNRA,
satisfying 82.24% of all networks, compared to other DT models. Recall
that according to Table 6 the selected DNN model is able to satisfy
83.41% of the total networks in the test dataset of networks. We ex­
pected the DEPTH-5 and DEPTH-10 models to have better CNRA results
in terms of total network satisfaction since they had the best performing
values in F1 and MCC metrics, according to Table 7. However, the
DEPTH-15 model, with the second highest depth among the other DT
models, has the highest PPV value, as we observed previously. We as­
sume that this is an indication of a reliable model in comparison to other
DT models because it also has the highest CNRA value when addressing
each network as a system of nodes. According to these results, the PPV
metric shows the ability to indicate the most suitable DT model when
considering each network as a system. What is surprising is that the
DEPTH-20 model, which shows the worst performance and is charac­
terized as an overfitted model, according to Table 8, has the second best
CNRA performance at the network level. This behavior introduces a
future research objective considering the determination of the optimal
depth of DT models compared to performance metrics in various cases
with imbalanced data.
Summarizing, our findings from the aforementioned comparative
analysis indicate that the overall behavior of DT models is rather
instable, especially when they become deeper. In a real-world scenario,
such as a post-disaster crisis, finding a suitable location for the DC and
distributing humanitarian aid in a timely manner requires reliable
decision-making processes to minimize the risk of irreversible mistakes.
Our analysis indicates that the DNN model performs more consistently
on large datasets as well as on smaller datasets, in comparison to the DT
models examined here.
5. Decision-making example with R Shiny application
The main goal is the development of a decision-making application

Table 8
Accuracy of the decision tree models with the maximum depth scenarios of 5, 10, 15 and 20 on the test dataset of networks.
DEPTH-5 DEPTH-10 DEPTH-15 DEPTH-20

NR CTN CNRA CTN CNRA CTN CNRA CTN CNRA

1 2,327 33.24% 2,275 32.50% 2,177 31.10% 2,236 31.94%

2 3,431 49.01% 3,459 49.41% 3,533 50.47% 3,521 50.30%
3 4,350 62.14% 4,356 62.23% 4,497 64.24% 4,493 64.19%
4 5,025 71.79% 5,021 71.73% 5,238 74.83% 5,187 74.10%
5 5,487 78.39% 5,595 79.93% 5,757 82.24% 5,738 81.97%

9
X. Taouktsis and C. Zikopoulos Expert Systems With Applications 238 (2024) 122010

that a user has to follow to get the proposed DCs from the DNN model are
illustrated in Fig. 3.
The following illustrations demonstrate the various features of the
decision support application. In particular, in Figs. 4–8 the essential
aspects of the analysis and prediction from the DNN model are illus­
trated according to Fig. 3 steps.

6. Conclusion, limitations and future work

In the current paper, we configure a hybrid approach to select a

suitable node as a DC installation point to minimize the aid distribution
distance to support HL networks, using only a proportion of the nodes of
the network. Our approach combines methods and techniques from
various fields with the perspective of improving the effectiveness and
efficiency of HL decision-making processes. In particular, using CIs
together with HA to obtain TSP solutions, we generate and evaluate a
number of DNNs after conducting several experiments to achieve the
optimal predictive model. Our DNN models use CIs as input and the
output is a binary SL or RJ decision of the node. Then, the DNN model
suggests only one-third of the total nodes in the network under
consideration as suitable candidates for DC installation. These candidate
nodes are used as input to the FI algorithm for obtaining the lowest
possible distribution distance for humanitarian aid operations.
Moreover, in our study, we evaluate the prediction performance of
DNN models combined with five metrics on a validation dataset with
shuffled data nodes from different networks and using the mean values
and one-way ANOVA results. The selected DNN models was then used on
the validation dataset considering each network as a system rather than
Fig. 3. The simplified steps for proposing DCs from the predictive model by
a group of individual nodes. The best model obtained a CNRA of 83.40%
feeding it with CIs.
when using only the first five proposed nodes. We then used it on a test
dataset consisting of unseen data and achieved a CNRA of 83.41%,
that enables the user to easily select the network under question as a
indicating that it is reliable in predicting the lowest distribution distance
disaster case, conduct the NA, visualize the alternatives and extract the
using only the one-third of the total number of nodes. Also, in order to
results to support the DC location decision-making process. Τhe steps

Fig. 4. The menu and the upload of the dataset with the networks to be analyzed.

Fig. 5. Network selection for analysis using network ID.

10
X. Taouktsis and C. Zikopoulos Expert Systems With Applications 238 (2024) 122010

Fig. 6. Results of the CIs calculation for the selected network.

Fig. 7. The prediction - proposal of DCs from the model.

Fig. 8a. Visualization of a TSP solution using “ND14A” node of the candidate DCs.

11
X. Taouktsis and C. Zikopoulos
Fig. 8b. Visualization of a TSP solution using “ND07A” node of the candidate DCs.
examine the suitability of DNN models in comparison to other classifi­
cation models, we conducted a performance evaluation after selecting
the DNN model used in our DMT. The comparison completed against the
well-known DT models demonstrates that in addressing our problem,
the DNN model demonstrates better performance.
The limitations observed during the analysis to select the optimal
DNN model were based on the results of the mean values of the five
metrics. We concluded that the performance of models is affected by the
changes in the number of HNs, but it remains unclear which number of
HNs leads to better predictive performance. This conclusion is based on
the fact that the improvement is relatively small and not always
consistent across the five metrics we examined. Moreover, it is impor­
tant to highlight that the models have a moderate performance on a
validation and on a test dataset in detecting the SL class according to the
results of the five metrics. In particular, the metrics ACC, F1, MCC, PPV
and TPR show that the DNN models need improvement when applied to
an imbalance dataset of different networks with the nodes shuffled.
Our work provides an innovative DMT by combining various tech­
niques and methods from different fields in a unique way for decision-
making in HL networks. This approach can lead to potentially signifi­
cant reductions in decision-making computational time for DC instal­
lation and reduce the distribution time of humanitarian aid in affected
areas, by providing promising solutions for real-world HL network
configuration problems with a support of a web-based application, using
the R “Shiny” package.
An immediate future extension of our work is the improvement of the
predictive ability of the DNN models with various techniques and
methods to handle more effectively imbalanced datasets. Such tech­
niques include oversampling the minority SL class, undersampling the
majority RJ class, and Synthetic Minority Oversampling Technique to
generate synthetic samples (Johnson & Khoshgoftaar, 2019). In addi­
tion, increasing the complexity of the models by adding new CIs in
12
Expert Systems With Applications 238 (2024) 122010
addition to existing ones, formulating new CIs, and incorporating
additional network information is expected to improve predictive ability
of the model. Another useful extension is to study networks with more
nodes, twice or more times the number of nodes used in our initial
approach. Furthermore, it is necessary to conduct experiments with
different hyper-parameters in DNN models, such as the activation
functions, learning rate, number of hidden layers, and regularization
parameters. Moreover, it is interesting to conduct an extensive com­
parison with DT classifier models, which requires new experiments with
different parameters as well as a focus on optimal tree depth for better
performance, plus a comparison with other classifiers such as Random
Forests. Also, to consider different humanitarian aid distribution as­
sumptions, such as a Vehicle Routing Problem model approach with
multiple DCs installations instead of the classic TSP models with one DC.
Moreover, as an additional feature of the proposed DMT, it could be
considered the presence of extra changes of the networks structure
because of post-disaster incidents, as attacks, on nodes and on links.
Thus, it will be possible to check the vulnerability and resilience of the
networks for improved decisions.
Code availability
The code used for the research is available on reasonable request
from the corresponding author.
Declaration of Competing Interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
X. Taouktsis and C. Zikopoulos Expert Systems With Applications 238 (2024) 122010

Data availability X. Taouktsis is a recipient of financial support in the context of a

The data used in the manuscript and the correcponding results are
available here: Taouktsis & Zikopoulos, 2023.
Acknowledgments
The authors are indebted to two anonymous reviewers whose com­
ments helped the significant improvement of the presentation of the
material in this paper.
doctoral thesis (grant number MIS-5113934). The implementation of the
doctoral thesis was co-financed by Greece and the European Union
(European Social Fund-ESF) through the Operational Pro­
gramme—Human Resources Development, Education and Lifelong
Learning—in the context of the Act—Enhancing Human Resources
Research Potential by undertaking a Doctoral Research—Sub-action 2:
IKY Scholarship Programme for Ph.D. candidates in the Greek
Universities.

Appendix A. Centrality indices

In the following description of CIs, we denote the network as G, the set of nodes in the network as V(G) and the number of nodes in the network as
N. The mathematical expressions provided are suitable for the calculation of the normalized values (values between 0 and 1) of the indices.
Degree Centrality (DGC) index is the most popular and simplest centrality. It measures the number of nodes directly connected to a node. An
extension of DGC is Strength Centrality (SC), which considers a node’s overall participation in the network by measuring the total weight of its links.
High DGC and SC scores are useful in FLP by identifying potential hub nodes that are suitable for facility installation because of their increased
accessibility to other nodes. DGC and SC are calculated using Eq. (A1) and Eq. (A2), respectively.
∑
ndgc(i) = [1/(N − 1)] aij (A1)
j∈V(G)

{ /[ ( )]} ∑
nsc(i) = 1 (N − 1)*max wij wij (A2)
j∈V(G)

aij : The elements of the adjacency matrix, indicating direct connection between i and j.
wij : The elements of weighted adjacency matrix. The value of wij corresponds to the weight of the link which connects the node i with the node j.

Closeness Centrality (CC) index is a path-based centrality, related to the reciprocal of the total geodesic distance (the shortest path between two
nodes) of a node from all the others according to Eq. (A3). Specifically, nodes with high CC score are closer to other nodes in the network and thus they
are suitable candidate locations for the FLP because the travel time is reduced.
/
∑
ncc(i) = (N − 1) d(i, j) (A3)
j∈V(G)

d(i, j): The geodesic distance length from node i to node j.

As CC cannot be used in networks with isolated nodes, i.e., disconnected networks, where the distances between certain pairs of nodes are infinite,
it has been proposed the Harmonic Centrality (HC) index calculated using Eq. (A4).
∑
nhc(i) = [1/(N − 1)] [1/d(j, i)] (A4)
j∈V(G)\{i}

Betweenness Centrality (BC) index is effective in FLP because it is identifies nodes that are important from a spatial point of view, since they are
frequently used for traversing the network. BC considers the set of shortest paths in a network and quantifies how frequently a node serves as an
internal point on the shortest path connecting two other nodes, as shown in Eq. (A5).
/
∑
nbc(i) = {2/[(N − 1)(N − 2)]} σjk (i) σjk (A5)
(j,k)∈{(V(G)\{i})2 |j∕
=k}

σ jk : The total number of geodesic paths from node j to node k.

σ jk (i): The total number of geodesic paths from node j to node k passing through node i.

Eigenvector Centrality (EC) index is based on the eigenvectors of the adjacency matrices and it is an improved version of DGC index. The funda­
mental idea behind the EC is that a node is important if it has many and important (with high-EC scores) neighbors. Therefore, a node with a high EC
score has many connections to other important nodes in the network over relatively short distances and is considered more central to the network.
Locating a facility at a node with high EC facilitates the access to other important nodes of the network. EC is calculated using Eq. (A6).
∑( )
ec(i) = (1/λ) wij ⋅ec(j)
j∈V(G)

13
X. Taouktsis and C. Zikopoulos Expert Systems With Applications 238 (2024) 122010

∑
where ec(i) = 1 (A6)
i∈V(G)

λ: The largest eigenvalue of adjacency matrix.

Laplacian Centrality (LC) index is based on the relative decline of “Laplacian Energy” of a network by removing a node from it and it is calculated by
Eq. (A7). LC is a suitable choice when FLP is considered. This is because a high LC score corresponds to a node which if it was removed from the
network it would considerably reduce the connectivity of the network.
el (i, G) = [el (G) − el (Gi )]/el (G) (A7)
∑
el (G) = λ2i , Laplacian Energy
0≤i≤n

L(G) = D(G) − W(G), Laplacian Matrix

Gi : The new network which obtained from G by removing the node i.

L(G): The “Laplacian” weighted matrix of the network.
D(G): The diagonal degree values matrix.
W(G): The weighted matrix.
λi : The eigenvalues of the “Laplacian” weighted matrix L(G).

Subgraph Centrality (SGC) index is an improved version of the DGC index and is based on the node’s participation in motifs or small subgraph
patterns. SGC measures the participation of a node within all connected subgraphs in a network. A small SGC score attributes more importance to the
node in question and a high SGC score means that the node is more embedded in the network through a range of different subgraphs. With respect to
FLP, SGC can be used to identify the strategically important node in terms of the number of alternative routes to other nodes in the network and
through relatively short distance for a facility installation. SGC mathematically is represented using Eq. (A8).
/
∑ ∑
nsgc(i) = [μk (i)/k! ] [μk (j)/k! ] (A8)
k∈ℕ j∈V(G),k∈ℕ

μk (i): The number of closed loops length k that start and end at the node i.

Appendix B. Model performance metrics

Accuracy (ACC) is one of the most popular metrics for measuring the prediction ability in a classification problem and it is defined as the proportion
of correctly predicted values (True Positive and True Negative) to the total data values, as shown in Eq. (B1).
ACC = (TP + TN)/(TP + FN + FP + TN) (B1)
True Positive Rate (TPR) metric appears with several names in the related literature and many times we refer to it as “Sensitivity”, “Recall”, as well
as “Hit Rate”. It measures the ratio of correctly predicted positive values in a number of actual positive data values and can be characterized as
accuracy of positive values, according to Eq. (B2).
TPR = TP/(TP + FN) (B2)
Positive Prediction Value (PPV) most of the times it is referred with the term “Precision” and it accounts for the ratio of correctly predicted positive
values in the number of values which have been predicted (wrongly or correctly) positive, according to Eq. (B3).
PPV = TP/(TP + FP) (B3)
F1-score (F1) is a well-known metric, implemented in both binary and multi classification problems. It combines the TPR and PPV metrics into a
single metric by calculating their harmonic mean and takes values between 0 and 1. A high value of F1 signifies a high prediction ability of the model
under consideration. F1 is calculated using Eq. (B4).
F1 − score = 2TP/(2TP + FP + FN) (B4)
Matthews Correlation Coefficient (MCC) metric measures the correlation between the actual observed values of positive or negative categories in
relation with the predicted values according to the results of the CM. The MCC estimates a balanced accuracy between two classes in BCP. The range of
MCC values is between − 1 and 1. A value equal to − 1 indicates an entirely opposite relationship between the predicted and the actual data values,
while a value equal to 1 indicates an absolutely correct prediction ability of the model. A value of 0 indicates a model with lack of predictive ability and
the categorization as positive or negative is random. MCC can be calculated by Eq. (B5).
/√̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
MCC = (TP × TN − FP × FN) (TP + FP)(TP + FN)(TN + FP)(TN + FN) (B5)

14
X. Taouktsis and C. Zikopoulos Expert Systems With Applications 238 (2024) 122010

References Cho, J., Lee, K., Shin, E., Choy, G., & Do, S. (2015). How much data is needed to train a
medical image deep learning system to achieve necessary high accuracy? .
Cook, W., Cook, W. J., Cunningham, W. H., Pulleyblank, W. R., & Schrijver, A. (1997).
Abbatecola, L., Fanti, M. P., Mangini, A. M., & Ukovich, W. (2016). A Decision support
Combinatorial Optimization. Wiley.
approach for postal delivery and waste collection services. IEEE Transactions on
Costa, V. G., & Pedreira, C. E. (2023). Recent advances in decision trees: An updated
Automation Science and Engineering, 13(4), 1458–1470. https://doi.org/10.1109/
survey. Artificial Intelligence Review, 56(5), 4765–4800. https://doi.org/10.1007/
TASE.2016.2570121
s10462-022-10275-5
Abdolrasol, M. G. M., Hussain, S. M. S., Ustun, T. S., Sarker, M. R., Hannan, M. A.,
Dangi, H., Bardhan, A. K., & Narag, A. S. (2012). Humanitarian relief logistics: An
Mohamed, R., Ali, J. A., Mekhilef, S., & Milad, A. (2021). Artificial neural networks
exploratory study for need and importance of performance measurement system.
based optimization techniques: A review. Electronics, 10(21), 2689. https://doi.org/
International Journal of Logistics Systems and Management, 13(1), 1–16. https://doi.
10.3390/electronics10212689
org/10.1504/IJLSM.2012.048630
Abdullah, A. A., & Kanaya, S. (2019). Machine Learning Using H2O R Package: An
Das, K., Samanta, S., & Pal, M. (2018). Study on centrality measures in social networks: A
Application in Bioinformatics. In Proceedings of the Third International Conference on
survey. Social Network Analysis and Mining, 8(1), 13. https://doi.org/10.1007/
Computing, Mathematics and Statistics (iCMS2017) (pp. 375–381). Singapore:
s13278-018-0493-2
Springer. https://doi.org/10.1007/978-981-13-7279-7_46.
Daud, M. S. M., Hussein, M. Z. S. M., Nasir, M. E., Abdullah, R., Kassim, R.,
Afaq, S., & Rao, S. (2020). Significance of epochs on training a neural network.
Suliman, M. S., & Saludin, M. R. (2016). Humanitarian logistics and its challenges:
International Journal of Scientific & Technology Research, 9(06), 485–488.
The literature review. International Journal of Supply Chain Management, 5(3),
Aggarwal, C. C. (2018). Neural networks and deep learning. In Neural Networks and Deep
107–110.
Learning (1st ed.). Springer International Publishing. https://doi.org/10.1007/978-
Day, J. M., Melnyk, S. A., Larson, P. D., Davis, E. W., & Whybark, D. C. (2012).
3-319-94463-0.
Humanitarian and disaster relief supply chains: A matter of life and death. Journal of
Ahuja, R., Sharma, S. C., & Ali, M. (2019). A diabetic disease prediction model based on
Supply Chain Management, 48(2), 21–36. https://doi.org/10.1111/j.1745-
classification algorithms. Annals of Emerging Technologies in Computing, 3(3), 44–52.
493X.2012.03267.x
https://doi.org/10.33166/AETiC.2019.03.005
Dubey, R., Gunasekaran, A., & Papadopoulos, T. (2019). Disaster relief operations: Past,
Almende, B. V., Thieurmel, B., & Robert, T. (2019). visNetwork: Network visualization
present and future. Ann. Oper. Res., 283(1–2), 1–8. https://doi.org/10.1007/s10479-
using’vis. js’ library. R Package Version, 2(9). https://github.com/datastorm-open/v
019-03440-7
isNetwork.
Erdoğan, G., Stylianou, N., & Vasilakis, C. (2019). An open source decision support
Anbuudayasankar, S. P., Ganesh, K., & Mohapatra, S. (2014). Models for practical routing
system for facility location analysis. Decision Support Systems, 125, Article 113116.
problems in logistics. Springer International Publishing. https://doi.org/10.100
https://doi.org/10.1016/j.dss.2019.113116
7/978-3-319-05035-5.
Estrada, E., & Rodríguez-Velázquez, J. A. (2005). Subgraph centrality in complex
Anuar, W. K., Lee, L. S., Pickl, S., & Seow, H. V. (2021). Vehicle routing optimisation in
networks. Physical Review E, 71(5), Article 056103. https://doi.org/10.1103/
humanitarian operations: A survey on modelling and optimisation approaches.
PhysRevE.71.056103
Applied Sciences (Switzerland), 11(2), 1–70. https://doi.org/10.3390/app11020667
Farazmehr, S., & Wu, Y. (2023). Locating and deploying essential goods and equipment
Ashtiani, M., Mirzaie, M., & Jafari, M. (2019). CINNA: An R/CRAN package to decipher
in disasters using AI-enabled approaches: A systematic literature review. Progress in
Central Informative Nodes in Network Analysis. Bioinformatics, 35(8), 1436–1437.
Disaster Science, 19, Article 100292. https://doi.org/10.1016/j.pdisas.2023.100292
https://doi.org/10.1093/bioinformatics/bty819
Fikar, C., Gronalt, M., & Hirsch, P. (2016). A decision support system for coordinated
Barabási, A. L., & Pósfai, M. A. (2016). Network Science. Cambridge University Press. htt
disaster relief distribution. Expert Systems with Applications, 57, 104–116. https://doi.
p://networksciencebook.com.
org/10.1016/j.eswa.2016.03.039
Beamon, B. M., & Balcik, B. (2008). Performance measurement in humanitarian relief
Gabor, C., & Tamas, N. (2006). The igraph software package for complex network
chains. International Journal of Public Sector Management, 21(1), 4–25. https://doi.
research. InterJournal, Complex Sy, 1695. https://igraph.org.
org/10.1108/09513550810846087
Gavião, L. O., Sant’Anna, A. P., Lima, G. B. A., de Almada Garcia, P. A., & de Sousa, A. M.
Besiou, M., & Van Wassenhove, L. N. (2020). Humanitarian operations: A world of
(2020). Selecting Distribution Centers in Disaster Management by Network Analysis
opportunity for relevant and impactful research. Manufacturing & Service Operations
and Composition of Probabilistic Preferences. In Industrial Engineering and Operations
Management, 22(1), 135–145. https://doi.org/10.1287/msom.2019.0799
Management (pp. 1–11). https://doi.org/10.1007/978-3-030-56920-4_1.
Bevere, L., Ehrler, A., Kumar, V., Lechner, R., Schelbert, A., Schwartz, M., & Sharan, R.
Grando, F., Granville, L. Z., & Lamb, L. C. (2019). Machine learning in network centrality
(2019). Natural Catastrophes and Man-made Disasters in 2018: “secondary” Perils on
measures. ACM Computing Surveys, 51(5), 1–32. https://doi.org/10.1145/3237192
the Frontline. Swiss Re Institute. https://reliefweb.int/report/world/natural-catastro
Greco, C., Pace, P., Basagni, S., & Fortino, G. (2021). Jamming detection at the edge of
phes-and-man-made-disasters-2018-secondary-perils-frontline.
drone networks using Multi-layer Perceptrons and Decision Trees. Applied Soft
Blanco-Justicia, A., Domingo-Ferrer, J., Martínez, S., & Sánchez, D. (2020). Machine
Computing, 111, Article 107806. https://doi.org/10.1016/j.asoc.2021.107806
learning explainability via microaggregation and shallow decision trees. Knowledge-
Guha, S., Jana, R. K., & Sanyal, M. K. (2022). Artificial neural network approaches for
Based Systems, 194, Article 105532. https://doi.org/10.1016/j.knosys.2020.105532
disaster management: A literature review. International Journal of Disaster Risk
Bonchi, F., De Francisci Morales, G., & Riondato, M. (2016). Centrality Measures on Big
Reduction, 81, Article 103276. https://doi.org/10.1016/j.ijdrr.2022.103276
Graphs. In Proceedings of the 25th International Conference Companion on World Wide
H2O.ai. (2020). R Interface for the “H2O” Scalable Machine Learning Platform (3.28.0.2).
Web - WWW ’16 Companion (pp. 1017–1020). https://doi.org/10.1145/287251
https://github.com/h2oai/h2o-3.
8.2891063.
H2O.ai. (2023). Decision Tree (3.42.0.2). http://h2o-release.s3.amazonaws.com/h2o/re
Boonmee, C., Arimura, M., & Asada, T. (2017). Facility location optimization model for
l-3.42.0/2/docs-website/h2o-docs/data-science/decision-tree.html.
emergency humanitarian logistics. International Journal of Disaster Risk Reduction, 24,
Hahsler, M., & Hornik, K. (2007). TSP - Infrastructure for the traveling salesperson
485–498. https://doi.org/10.1016/j.ijdrr.2017.01.017
problem. Journal of Statistical Software, 23(2), 1–21. https://doi.org/10.18637/jss.
Boughorbel, S., Jarray, F., & El-Anbari, M. (2017). Optimal classifier for imbalanced data
v023.i02
using Matthews Correlation Coefficient metric. PLoS One, 12(6), e0177678.
Hossen, T., Plathottam, S. J., Angamuthu, R. K., Ranganathan, P., & Salehfar, H. (2017).
Brandes, U. & Erlebach, T. (2005). Network Analysis. Methodological Foundations. In
Short-term load forecasting using deep neural networks (DNN). North American
Lecture Notes in Computer Science.
Power Symposium (NAPS), 2017, 1–6. https://doi.org/10.1109/NAPS.2017.8107271
Bresson, X., & Laurent, T. (2021). The Transformer Network for the Traveling Salesman
Hua, J., Huang, M., & Huang, C. (2019). Centrality metrics’ performance comparisons on
Problem. ArXiv Preprint ArXiv:2103.03012. .
stock market datasets. Symmetry, 11(7), 916. https://doi.org/10.3390/
Candel, A., LeDell, E., Parmar, V., & Anisha, A. (2022). Deep learning with H2O. In H2O.
sym11070916
ai Inc (6th ed., Issue August). H2O. ai Inc. https://h2o.ai/resources/booklet/deep-le
Izza, Y., Ignatiev, A., & Marques-Silva, J. (2020). On Explaining Decision Trees. htt
arning-with-h2o/.
p://arxiv.org/abs/2010.11034.
Cavdur, F., & Sebatli, A. (2019). A decision support tool for allocating temporary-
Jalili, M., Salehzadeh-Yazdi, A., Asgari, Y., Arab, S. S., Yaghmaie, M., Ghavamzadeh, A.,
disaster-response facilities. Decision Support Systems, 127, Article 113145. https://
& Alimoghaddam, K. (2015). CentiServer: A comprehensive resource, web-based
doi.org/10.1016/j.dss.2019.113145
application and R package for centrality analysis. PLoS One, 10(11), e0143111.
Chang, K.-H., Wu, Y.-Z., & Ke, S.-S. (2022). A simulation-based decision support tool for
Johnson, D. S. (1990). Local optimization and the Traveling Salesman Problem. In
dynamic post-disaster pedestrian evacuation. Decision Support Systems, 157, Article
Automata, Languages and Programming: Vol. 443 LNCS (pp. 446–461). Springer-
113743. https://doi.org/10.1016/j.dss.2022.113743
Verlag. https://doi.org/10.1007/BFb0032050.
Chang, W., Cheng, J., Allaire, J. J., Sievert, C., Schloerke, B., Xie, Y., Allen, J.,
Johnson, J. M., & Khoshgoftaar, T. M. (2019). Survey on deep learning with class
McPherson, J., Dipert, A., & Borges, B. (2021). shiny: Web Application Framework for
imbalance. Journal of Big Data, 6(1), 27. https://doi.org/10.1186/s40537-019-0192-
R (1.6.0). https://cran.r-project.org/package=shiny.
5
Charles, A., Bouleux, G., & Cherifi, C. (2022). Understanding the situation at a glance:
Kebriyaii, O., Hamzehei, M., & Khalilzadeh, M. (2021). A disaster relief commodity
The powerful value added of complex networks to analyse humanitarian operations.
supply chain network considering emergency relief volunteers: A case study. Journal
International Journal of Disaster Risk Reduction, 75, Article 102932. https://doi.org/
of Humanitarian Logistics and Supply Chain Management, 11(3), 493–521. https://doi.
10.1016/j.ijdrr.2022.102932
org/10.1108/JHLSCM-08-2020-0073
Chicco, D., & Jurman, G. (2020). The advantages of the Matthews correlation coefficient
Kim, D., Han, H., Wang, W., & Kim, H. S. (2022). Improvement of deep learning models
(MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics,
for river water level prediction using complex network method. Water, 14(3), 466.
21(1), 6. https://doi.org/10.1186/s12864-019-6413-7
https://doi.org/10.3390/w14030466
Chicco, D., Tötsch, N., & Jurman, G. (2021). The Matthews correlation coefficient (MCC)
Kool, W., van Hoof, H., & Welling, M. (2018). Attention, Learn to Solve Routing
is more reliable than balanced accuracy, bookmaker informedness, and markedness
Problems! 7th International Conference on Learning Representations, ICLR 2019. htt
in two-class confusion matrix evaluation. BioData Mining, 14(1), 13. https://doi.org/
p://arxiv.org/abs/1803.08475.
10.1186/s13040-021-00244-z

15
X. Taouktsis and C. Zikopoulos
Koschützki, D., Lehmann, K. A., Peeters, L., Richter, S., Tenfelde-Podehl, D., &
Zlotowski, O. (2005). Centrality Indices. In U. Brandes, & T. Erlebach (Eds.), Network
Analysis: Methodological Foundations (pp. 16–61). Berlin Heidelberg: Springer.
https://doi.org/10.1007/978-3-540-31955-9_3.
Kovács, G., & Spens, K. (2009). Identifying challenges in humanitarian logistics.
International Journal of Physical Distribution & Logistics Management, 39(6), 506–528.
https://doi.org/10.1108/09600030910985848
Kruithof, M. (2012). Traveling Salesman Problem Comparisons between heuristics, linear and
semidefinite programming approximations (Issue June) [University of Groningen]. htt
ps://fse.studenttheses.ub.rug.nl/id/eprint/10233.
Kumar, V., & Jana, A. (2021). Coverage or response time: A study of fire station location-
allocation models using network analysis. In Advances in Urban Planning in Developing
Nations (pp. 82–98). Routledge India.
Kunz, N., Van Wassenhove, L. N., Besiou, M., Hambye, C., & Kovács, G. (2017).
Relevance of humanitarian logistics research: Best practices and way forward.
International Journal of Operations & Production Management, 37(11), 1585–1599.
https://doi.org/10.1108/IJOPM-04-2016-0202
Lecun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.
https://doi.org/10.1038/nature14539
Lee, S. (2012). The role of centrality in ambulance dispatching. Decision Support Systems,
54(1), 282–291. https://doi.org/10.1016/j.dss.2012.05.036
Leiras, A., de Brito Jr, I., Queiroz Peres, E., Rejane Bertazzo, T., & Tsugunobu Yoshida
Yoshizaki, H. (2014). Literature review of humanitarian logistics research: Trends
and challenges. Journal of Humanitarian Logistics and Supply Chain Management, 4(1),
95–130. https://doi.org/10.1108/JHLSCM-04-2012-0008
Lewin, R., Besiou, M., Lamarche, J.-B., Cahill, S., & Guerrero-Garcia, S. (2018).
Delivering in a moving world…looking to our supply chains to meet the increasing
scale, cost and complexity of humanitarian needs. Journal of Humanitarian Logistics
and Supply Chain Management, 8(4), 518–532. https://doi.org/10.1108/JHLSCM-10-
2017-0048
Linardos, V., Drakaki, M., Tzionas, P., & Karnavas, Y. L. (2022). Machine Learning in
disaster management: Recent developments in methods and applications. Machine
Learning and Knowledge Extraction, 4(2), 446–473. https://doi.org/10.3390/
make4020020
Liu, K., Zhao, Y., & Zhang, Z.-H. (2022). A Dynamic facility location problem with
random demand and the restoration of transportation networks in disaster relief.
SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4002257
Liu, Y., Jiang, C., & Zhao, H. (2018). Using contextual features and multi-view ensemble
learning in product defect identification from online discussion forums. Decision
Support Systems, 105, 1–12. https://doi.org/10.1016/j.dss.2017.10.009
Liu, Y., Yuan, Y., Shen, J., & Gao, W. (2021). Emergency response facility location in
transportation networks: A literature review. Journal of Traffic and Transportation
Engineering (English Edition), 8(2), 153–169. https://doi.org/10.1016/j.
jtte.2021.03.001
Lü, L., Chen, D., Ren, X.-L., Zhang, Q.-M., Zhang, Y.-C., & Zhou, T. (2016). Vital nodes
identification in complex networks. Physics Reports, 650, 1–63. https://doi.org/
10.1016/j.physrep.2016.06.007
Lujak, M., & Giordani, S. (2018). Centrality measures for evacuation: Finding agile
evacuation routes. Future Generation Computer Systems, 83, 401–412. https://doi.
org/10.1016/j.future.2017.05.014
Maghfiroh, M. F. N., & Hanaoka, S. (2020). Multi-modal relief distribution model for
disaster response operations. Progress in Disaster Science, 6, Article 100095. https://
doi.org/10.1016/j.pdisas.2020.100095
Matai, R., Singh, S., & Lal, M. (2010). Traveling Salesman Problem: an Overview of
Applications, Formulations, and Solution Approaches. In D. Davendra (Ed.),
Traveling Salesman Problem, Theory and Applications. InTech. https://doi.org/10.5
772/12909.
Mohamadi, A., & Yaghoubi, S. (2017). A bi-objective stochastic model for emergency
medical services network design with backup services for disasters under
disruptions: An earthquake case study. International Journal of Disaster Risk
Reduction, 23, 204–217. https://doi.org/10.1016/j.ijdrr.2017.05.003
Moshref-Javadi, M., & Lee, S. (2016). The latency location-routing problem. European
Journal of Operational Research, 255(2), 604–619. https://doi.org/10.1016/j.
ejor.2016.05.048
Negi, S. (2022). Humanitarian logistics challenges in disaster relief operations: A
humanitarian organisations’ perspective. Journal of Transport and Supply Chain
Management, 16. https://doi.org/10.4102/jtscm.v16i0.691
Novak, D. C., & Sullivan, J. L. (2014). A link-focused methodology for evaluating
accessibility to emergency services. Decision Support Systems, 57, 309–319. https://
doi.org/10.1016/j.dss.2013.09.015
Nunavath, V., & Goodwin, M. (2019). The Use of Artificial Intelligence in Disaster
Management - A Systematic Literature Review. 2019 International Conference on
Information and Communication Technologies for Disaster Management (ICT-DM), 1–8.
https://doi.org/10.1109/ICT-DM47966.2019.9032935.
Oldham, S., Fulcher, B., Parkes, L., Arnatkevic̆ iūtė, A., Suo, C., & Fornito, A. (2019).
Consistency and differences between centrality measures across distinct classes of
networks. PLOS ONE, 14(7), e0220061. https://doi.org/10.1371/journal.pon
e.0220061.
Opsahl, T., Agneessens, F., & Skvoretz, J. (2010). Node centrality in weighted networks:
Generalizing degree and shortest paths. Social Networks, 32(3), 245–251. https://doi.
org/10.1016/j.socnet.2010.03.006
Orimoloye, L. O., Sung, M.-C., Ma, T., & Johnson, J. E. V. (2020). Comparing the
effectiveness of deep feedforward neural networks and shallow architectures for
predicting stock price indices. Expert Systems with Applications, 139, Article 112828.
https://doi.org/10.1016/j.eswa.2019.112828
16
Expert Systems With Applications 238 (2024) 122010
Ouyang, W., Wang, Y., Weng, P., & Han, S. (2021). Generalization in Deep RL for TSP
Problems via Equivariance and Local Search. .
Özdamar, L., & Ertem, M. A. (2015). Models, solutions and enabling technologies in
humanitarian logistics. European Journal of Operational Research, 244(1), 55–65.
https://doi.org/10.1016/j.ejor.2014.11.030
Patil, A., Madaan, J., Chan, F. T. S., & Charan, P. (2022). Advancement of performance
measurement system in the humanitarian supply chain. Expert Systems with
Applications, 206, Article 117844. https://doi.org/10.1016/j.eswa.2022.117844
Porta, S., Crucitti, P., & Latora, V. (2006). The network analysis of urban streets: A primal
approach. Environment and Planning B: Planning and Design, 33(5), 705–725. https://
doi.org/10.1068/b32045
Praneetpholkrang, P., & Kanjanawattana, S. (2021). A novel approach for determining
shelter location-allocation in humanitarian relief logistics. International Journal of
Knowledge and Systems Science, 12(2), 52–68. https://doi.org/10.4018/
IJKSS.2021040104
Qi, X., Fuller, E., Wu, Q., Wu, Y., & Zhang, C.-Q. (2012). Laplacian centrality: A new
centrality measure for weighted networks. Information Sciences, 194, 240–253.
https://doi.org/10.1016/j.ins.2011.12.027
R Core Team. (2019). R: A Language and Environment for Statistical Computing (3.5.3). R
Foundation for Statistical Computing. https://www.r-project.org/.
Ranjeeth, S., Kandimalla, V. A. K., & D, G. R. (2020). Predicting Diabetes Using Outlier
Detection and Multilayer Perceptron with Optimal Stochastic Gradient Descent. In
2020 IEEE India Council International Subsections Conference (INDISCON) (pp. 51–56).
https://doi.org/10.1109/INDISCON50162.2020.00023.
Rasamoelina, A. D., Adjailia, F., & Sincak, P. (2020). A Review of Activation Function for
Artificial Neural Network. In 2020 IEEE 18th World Symposium on Applied Machine
Intelligence and Informatics (SAMI) (pp. 281–286). https://doi.org/10.1109/
SAMI48414.2020.9108717
Ravi, D., Wong, C., Deligianni, F., Berthelot, M., Andreu-Perez, J., Lo, B., & Yang, G.-Z.
(2017). Deep learning for health informatics. IEEE Journal of Biomedical and Health
Informatics, 21(1), 4–21. https://doi.org/10.1109/JBHI.2016.2636665
Righini, G. (2000). The largest insertion algorithm for the traveling salesman problem.
Note Del Polo-Ricerca, 29.
Rivera, J. C., Murat Afsar, H., & Prins, C. (2016). Mathematical formulations and exact
algorithm for the multitrip cumulative capacitated single-vehicle routing problem.
European Journal of Operational Research, 249(1), 93–104. https://doi.org/10.1016/
j.ejor.2015.08.067
Rodrigues, F. A. (2019). Network Centrality: An Introduction. In A Mathematical Modeling
Approach from Nonlinear Dynamics to Complex Systems (pp. 177–196). Springer
International Publishing. https://doi.org/10.1007/978-3-319-78512-7_10.
Rodríguez, N., Gupta, A., Zabala, P. L., & Cabrera-Guerrero, G. (2018). Optimization
algorithms combining (meta)heuristics and mathematical programming and its
application in engineering. Mathematical Problems in Engineering, 2018, 1–3. https://
doi.org/10.1155/2018/3967457
Rojas Trejos, C. A., Meisel, J. D., & Adarme Jaimes, W. (2023). Humanitarian aid
distribution logistics with accessibility constraints: A systematic literature review.
Journal of Humanitarian Logistics and Supply Chain Management, 13(1), 26–41.
https://doi.org/10.1108/JHLSCM-05-2021-0041
Rosenkrantz, D. J., Stearns, R. E., & Lewis, P. M., II (1977). An analysis of several
heuristics for the traveling salesman problem. SIAM Journal on Computing, 6(3),
563–581. https://doi.org/10.1137/0206041
RStudio Team. (2021). RStudio: Integrated Development Environment (IDE) for R
(1.4.1106). RStudio, PBC. https://www.rstudio.com/.
Rusli, A., Young, J. C., & Iswari, N. M. S. (2020). Identifying Fake News in Indonesian via
Supervised Binary Text Classification. In 2020 IEEE International Conference on
Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT) (pp.
86–90). https://doi.org/10.1109/IAICT50021.2020.9172020
Sahebjamnia, N., Torabi, S. A., & Mansouri, S. A. (2017). A hybrid decision support
system for managing humanitarian relief chains. Decision Support Systems, 95, 12–26.
https://doi.org/10.1016/j.dss.2016.11.006
Sengupta, S., Basak, S., Saikia, P., Paul, S., Tsalavoutis, V., Atiah, F., Ravi, V., & Peters, A.
(2020). A review of deep learning with special emphasis on architectures,
applications and recent trends. Knowledge-Based Systems, 194, Article 105596.
https://doi.org/10.1016/j.knosys.2020.105596
Singh, A., Singh, R. R., & Iyengar, S. R. S. (2020). Node-weighted centrality: A new way
of centrality hybridization. Computational Social Networks, 7(1), 6. https://doi.org/
10.1186/s40649-020-00081-w
Snyder, L. V., & Shen, Z. M. (2019). Fundamentals of supply chain theory. In
Fundamentals of Supply Chain Theory. Wiley. https://doi.org/10.1002/
9781119584445.
Sreedharan, V. R., Kek, V., Dhanya, M., Anjali, S., & Arunprasad, P. (2020).
Understanding the role of logistics in humanitarian operations: Key findings and
analysis from literatures. International Journal of Logistics Systems and Management,
36(4), 463. https://doi.org/10.1504/IJLSM.2020.108961
Sun, W., Bocchini, P., & Davison, B. D. (2020). Applications of artificial intelligence for
disaster management. Natural Hazards, 103(3), 2631–2689. https://doi.org/
10.1007/s11069-020-04124-3
Tacheva, Z., & Simpson, N. (2019). Social network analysis in humanitarian logistics
research. Journal of Humanitarian Logistics and Supply Chain Management, 9(3),
492–514. https://doi.org/10.1108/JHLSCM-06-2018-0047
Taouktsis, X., & Zikopoulos, C. (2023). Research01-HL_DMT_Data-Results. Mendeley
Data, V1. https://doi.org/10.17632/syhz79dmzv.1
Tharwat, A. (2021). Classification assessment methods. Applied Computing and
Informatics, 17(1), 168–192. https://doi.org/10.1016/j.aci.2018.08.003
UN OCHA. (2018). Global Humanitarian Overview 2019. United Nations. https://doi.
org/10.18356/9789214030768.
X. Taouktsis and C. Zikopoulos
UN OCHA. (2021). Global Humanitarian Overview 2022. United Nations. https://doi.
org/10.18356/9789210012423.
Van Wassenhove, L. N. (2006). Humanitarian aid logistics: Supply chain management in
high gear. Journal of the Operational Research Society, 57(5), 475–489. https://doi.
org/10.1057/palgrave.jors.2602125
Vinayakumar, R., Soman, K. P., Senthil Velan, K. K., & Ganorkar, S. (2017). Evaluating
shallow and deep networks for ransomware detection and classification. In 2017
International Conference on Advances in Computing, Communications and Informatics
(ICACCI) (pp. 259–265). https://doi.org/10.1109/ICACCI.2017.8125850
Wan, Z., Mahajan, Y., Kang, B. W., Moore, T. J., & Cho, J.-H. (2021). A Survey on
centrality metrics and their network resilience analysis. IEEE Access, 9,
104773–104819. https://doi.org/10.1109/ACCESS.2021.3094196
Wandelt, S., Shi, X., & Sun, X. (2020). Complex network metrics: Can deep learning keep
up with tailor-made reference algorithms? IEEE Access, 8, 68114–68123. https://doi.
org/10.1109/ACCESS.2020.2984762
Wang, G., Giannakis, G. B., & Chen, J. (2019). Learning ReLU networks on linearly
separable data: Algorithm, optimality, and generalization. IEEE Transactions on
Signal Processing, 67(9), 2357–2370. https://doi.org/10.1109/TSP.2019.2904921
Warnier, M., Alkema, V., Comes, T., & Van de Walle, B. (2020). Humanitarian access,
interrupted: Dynamic near real-time network analytics and mapping for reaching
communities in disaster-affected countries. OR Spectrum, 42(3), 815–834. https://
doi.org/10.1007/s00291-020-00582-0
Wickham, H. (2016). ggplot2: Elegant Graphics for Data Analysis. New York: Springer-
Verlag. https://ggplot2.tidyverse.org.
17
Expert Systems With Applications 238 (2024) 122010
Wickham, H., François, R., Henry, L., & Müller, K. (2021). dplyr: A Grammar of Data
Manipulation. https://dplyr.tidyverse.org.
Xu, S., Panwar, S. S., Kodialam, M., & Lakshman, T. V. (2020). Deep neural network
approximated dynamic programming for combinatorial optimization. Proceedings of
the AAAI Conference on Artificial Intelligence, 34(02), 1684–1691. https://doi.org/
10.1609/aaai.v34i02.5531
Yousefzadeh, M., Hosseini, S. A., & Farnaghi, M. (2021). Spatiotemporally explicit
earthquake prediction using deep neural network. Soil Dynamics and Earthquake
Engineering, 144, Article 106663. https://doi.org/10.1016/j.soildyn.2021.106663
Zargary, S., & Samouei, P. (2022). Production-routing-inventory in post-disaster
conditions: A multi-objective mathematical model and two algorithms. Process
Integration and Optimization for Sustainability. https://doi.org/10.1007/s41660-022-
00274-y
Zhang, J. Z., Srivastava, P. R., Sharma, D., & Eachempati, P. (2021). Big data analytics
and machine learning: A retrospective overview and bibliometric analysis. Expert
Systems with Applications, 184, Article 115561. https://doi.org/10.1016/j.
eswa.2021.115561
Zhang, L., Ding, Y., & Lin, H. (2022). Optimizing synchronized truck-drone delivery with
priority in disaster relief. Journal of Industrial and Management Optimization. https://
doi.org/10.3934/jimo.2022166
Zhang, Y., Wang, X., Zeng, P., & Chen, X. (2011). Centrality characteristics of road
network patterns of traffic analysis zones. Transportation Research Record: Journal of
the Transportation Research Board, 2256(1), 16–24. https://doi.org/10.3141/2256-
03