applied
sciences
Article
Waste Classification of Spent Refractory Materials to Achieve
Sustainable Development Goals Exploiting Multiple Criteria
Decision Aiding Approach
Athanasios Spyridakos 1 , Dimitrios E. Alexakis 2 , Isaak Vryzidis 2, * , Nikolaos Tsotsolas 1 , George Varelidis 3
and Efthimios Kagiaras 4






Citation: Spyridakos, A.; Alexakis,
D.E.; Vryzidis, I.; Tsotsolas, N.;
Varelidis, G.; Kagiaras, E. Waste
Classification of Spent Refractory
Materials to Achieve Sustainable
Development Goals Exploiting
Multiple Criteria Decision Aiding
Approach. Appl. Sci. 2022, 12, 3016.
https://doi.org/10.3390/app
12063016
Academic Editor: Jorge de Brito
Received: 13 February 2022
Accepted: 11 March 2022
Published: 16 March 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Appl. Sci. 2022, 12, 3016. https://doi.org/10.3390/app12063016
1 Laboratory of Business Informatics and Quantitative Methods, Department of Business Administration,
School of Administrative, Economics and Social Sciences, University of West Attica, 250 Thivon & P.Ralli Str.,
12241 Athens, Greece; tspyr@uniwa.gr (A.S.); ntsotsol@uniwa.gr (N.T.)
2 Laboratory of Geoenvironmental Science and Environmental Quality Assurance, Department of Civil
Engineering, University of West Attica, 250 Thivon & P.Ralli Str., 12241 Athens, Greece; d.alexakis@uniwa.gr
3 Laboratory of Urban Planning and Architecture, Department of Civil Engineering, University of West Attica,
250 Thivon & P.Ralli Str., 12241 Athens, Greece; gvarel@uniwa.gr
4 R&D Department, Mathios Refractories S.A., 5 Epidavrou Str., 18233 Athens, Greece; tkagiaras@mathios.gr
* Correspondence: i.vryzidis@uniwa.gr
Featured Application: The method presented in this paper can be applied in waste management
for the achievement of sustainable development goals.
Abstract: The recycling of used refractory materials in the heavy industry constitutes one of the
significant environmental problems in the industry related to environmental and financial issues. This
study proposes a multicriteria methodological frame to characterize the refractory material waste and
identify the recycling capabilities. Considering the chemical and physical analysis of the refractory
material wastes, the proposed methodological frame progresses into a two-phase procedure. The
first phase includes an on/off approach that allows discretizing the refractory material wastes to
compatible or not compatible as far as their recycling prospects. Then, an additive value model
is utilized, including (a) the marginal value functions used for every criterion related to critical
environmental factors, and (b) the weight vector reflecting the relative importance of the criteria
used. A group of experts concerning the environment and the refractory materials was employed
to estimate the additive value model. The assessment of the marginal value function is achieved
using the module of the Multicriteria Interactive Intelligence Decision Aiding System (MIIDAS),
which is based on a modification of the mid-value split point technique incorporating focused
dialogues, artificial intelligence, and visual techniques. The weight vector was assessed using the
weight assessment through prioritization method (WAP), which concludes with the estimation of the
weights based on the criteria ranking and the pairwise expression of the strength of preferences for
the consecutive criteria according to their ranking. The outcome of this approach is to introduce an
environmental appropriateness index for refractory materials based on their chemical composition
and the judgement of an expert group. The main findings of this research may be useful for engineers,
decision-makers, and scientists in the field of circular economy and waste management.
Keywords: MIIDAS; WAP; additive value model; recycling; sustainability; circular economy
1. Introduction
Through applying alternative use cycles, the circular economy (CE) aims at reducing
dependency on (new) natural resource extraction, while expanding the time that resources
spent within the technology sphere [1]. In essence, this approach of CE imitates Earth’s
naturally circular systems [1]. CE envisions a balance between industry and the environ-
ment, ensuring that human civilization will continue to thrive in the long run. All waste is
https://www.mdpi.com/journal/applsci
Appl. Sci. 2022, 12, 3016 2 of 21

designed to become reusable material through different processes. Following the example
of the ecosystems, material cycles in CE are closed [1]. Many researchers have studied the
different aspects raised by the two main pillars of CE, namely, the technical model [2–7] and
the biological model [8–10]. Fereira et al. [2], Horckmans et al. [3], Fang et al. [4], Hanagiri
et al. [5], Poirer et al. [6], and Ariannpour et al. [7] dealt with refractory characterization
and recycling. Pantazopoulou et al. [8] and Kasina et al. [9] concluded on issues related to
the stabilization and potential reuse of sewage sludge and municipal waste incineration
ashes. Moreover, Vlachokostas et al. [10] suggested a framework supporting the decision
making for biodegrabable waste management. On the other hand, Kyriakopoulos et al. [11]
concluded that there are still many issues leading to the negative environmental impacts
that arise from the fundamental assumptions of CE.
Environmental characterization is crucial throughout all stages of refractory material
life as it assists in evaluating, mitigating, and controlling risks. Refractory solid materials
are used in industries (steel, cement, aluminum, glass, ceramics, etc.) where there are
requirements to withstand very high temperatures. Refractory materials are produced from
abundant industrial minerals, including chromite, dolomite, SiO2 , MgO, and Al2 O3 [2].
Refractories are the primary materials applied as the internal lining of kilns, incinerators,
and furnaces. Because these materials are exposed to elevated temperatures and corrosive
liquids or gases, they have to meet different challenging specifications, such as thermal
expansion, mechanical strength, and corrosion resistance [2]. Refractories are produced
in shape form (bricks) for lining subsystems of the industry or in powder form, which is
shaped on the installation site.
Refractories are classified into acidic, basic, or neutral according to the interaction of
water with the primary raw material. Acid refractories, including silica, zircon, and alumina
silicates are typically applied for lower operating temperatures [3]. Basic refractories,
such as doloma, spinel, and magnesia, are usually combined with graphite and carbon
and applied in highly basic environments [3]. Neutral refractories, including alumina
and chromia refractories, are applied broadly by the metal industries because of their
moderate price, elevated melting temperature, and ability to be applied in basic and acidic
environments [3,4].
The industries (metal, cement, glass, etc.) from time to time (depending on the type of
production) replace the refractories into the frame of maintenance of the industrial units.
It is estimated that, worldwide, 35–40 million tons of refractories are produced and used
yearly, and an amount of about 28 million tons of spent refractories are produced [3]. Even
in this vast amount of spent refractory material, it is worth mentioning that, in the last
20 years, the recycling processes have not seen a massive utilization since (a) the production
cost of the raw material was low, and (b) the cost of disposal of the used refractories was
not high.
Many of the abovementioned elements were changed recently, while many envi-
ronmental issues were raised concerning the handling of the refractory wastes and the
increased cost of landfilling. Furthermore, in the EU, new regulations and directives
were posed concerning the environment, such as the waste frame Directive 2008/98/EC,
which poses the priorities for handling the wastes, ordered as follows [12]: (a) prevention,
(b) preparation for reuse, (c) recycling, (d) recovery, and (e) disposal. Even the increasing
interests in the waste disposal of recycling spent refractories are a common approach world-
wide for the management of spent refractories. Therefore, reuse and recycling become of
higher importance and influence handling of the spent refractories.
The decision to reuse or recycle the refractory waste depends on a set of parameters,
the most important of which are (a) the quality of the spent refractories, (b) the previous use
of the refractories and the substances deposited on their surface, (c) the type of refractory,
(d) the composition of refractory materials which could contain potentially toxic elements
posing a threat to human and ecosystems, and (e) the leachability of potentially toxic
elements from refractories.
Appl. Sci. 2022, 12, 3016 3 of 21

Another significant parameter to be considered is the type of recycling process. The

closed-loop process concerns the use of the spent refractories producing new refractories.
This is the most attractive since the recycling of the spent refractories is focused on added-
value products with high production cost as far as the required energy is concerned.
The interest in reusing refractory spent materials with better environmental charac-
teristics is expected to grow in the upcoming years. Therefore, it is crucial to evaluate the
ecotoxicological potential of the refractory spent materials. The evaluation of potential
ecotoxicity of products/materials has been applied in several works [13,14]. The approach
for evaluating the potential environmental risk of granular solid materials for ecotoxi-
cological potential and landfilling was based on the Council Decision 2003/33/EC [15].
The appropriateness of the spent refractories for recycling is strongly dependent on the
quality of the spent refractory and the clearness as far as the environmental impacts and
health issues are concerned. The Council Decision 2003/33/EC discretized 15 components
and their limits with respect to the content of the material used for which these can be
characterized as inert, nonhazardous, and hazardous. In general, the proposed thresholds
of concentration for the 15 components provide a guide for the acceptance for recycling of
the wastes. A question is raised concerning the appropriateness of the spent refractories
and the types of products of the recycling, and this decision is complicated related to (a) the
kind of mineral of the refractory (chromite, dolomite, SiO2 , MgO, Al2 O3 , etc.), (b) the use
of the recycling products in the industries whereby different qualities of refractories are
used for different industries, and (c) the chemical and physical characteristics of the spent
materials and, most importantly, their environmental appropriateness.
However, there is no commonly accepted approach in the literature that evaluates
and quantifies the environmental appropriateness of waste materials by considering the
concentration of the 15 components together with expert judgment. This study aims
to fill this gap and is focused on the development of a methodological approach for
(a) the discretization of the spent refractories to those that can be recycled, and (b) the
development of a measure for the identification of the environmental appropriateness
of the spent refractories which can be used for recycling. The measure is based on a
structured multicriteria decision aid (MCDA) approach described later on, which will
support decisions concerning the kind of recycling that can be used for any type of waste.
The paper is organized as follows: Section 1 includes the introduction which provides
an overview of the current literature related to the specific topic. Section 2 provides a review
of the management options for refractory spent materials, the main types of recycling, and
the guidelines for the characterization of raw wastes based on the assessment of chemical
toxicity. This section also includes an overview of MCDA methods and their applications
in CE. Section 3 presents the proposed methodology for evaluating the appropriateness
of recycling for refractory spent materials, while Section 4 describes the application of the
proposed MCDA approach to spent refractories. Section 5 concludes the main findings of
the study and proposes new avenues for future research.

2. Background
2.1. Refractory Spent Materials and Recycling
Landfilling is a standard management option for refractories, while the recycling
process is an alternative one. The recycling of refractory spent material has emerged in the
last few years. The main types of recycling are (a) open-loop and (b) closed-loop.
The main advantages of closed-loop recycling are the evolution toward CE and the
potential to generate higher economic benefits. In closed-loop recycling, the intrinsic
properties of the virgin material are not significantly different from those of the recycled
material. Therefore, the recycled material can replace the virgin material and be applied in
products as before. In open-loop recycling, the intrinsic properties of the virgin material
differ from those of the recycled material. Therefore, the recycled material can replace other
materials for other product applications [16].
Appl. Sci. 2022, 12, 3016 4 of 21

The classical recycling process for refractory materials involves the following stages [5]:
sorting and classification, crushing, sieving, and drying.
Currently, the majority of spent refractories are employed in low-grade applications
such as roadbed construction or disposed of in landfills. Using the spent refractories for
producing other types (open loop) of refractories is considered good practice but cannot be
considered the best. The parameters controlling the recycling of refractories are, among
others, the (a) economic cost of the virgin raw material, (b) type of usage, (c) composition
of substances deposited on its surface, and (d) leachability of potentially toxic substances.
The studied refractory materials were used at high temperatures, particularly in
metallurgical processes. According to Poirer et al. [6], refractories are not inert materials. In
the aluminum and iron making processes at steelworks, various refractories are applied
for the furnace linings for refining, smelting, and conveying operations. The degraded
elements and impurities of refractories mainly include slag, base metal, and iron, while,
technically speaking, eradicating such impurities is impossible [5]. Arianpour et al. [7]
reported that about 70% of spent refractory bricks could be recycled successfully; while
some refractories are reused as decoration in landscaping.

2.2. Characterization of the Spent Refractory Materials

The assessment of chemical toxicity of raw wastes, as well as of spent refractory
materials, was conducted according to the standard leaching protocol EN 12457.04, which
provides information on the mobility behavior of refractory materials of a particle size below
4 mm and applying a liquid to solid ratio of 10 L/Kg dry matter [17]. The characterization
of the spent refractory materials is based on the Council Decision 2003/33/EC, regarding
the acceptance of spent refractories at landfills [15]. The compliance of the testing results of
the examined spent refractories with the Council Decision 2003/33/EC is a prerequisite in
order for a spent refractory material to be accepted for dumping in a specific landfill [8].
Values of pH and concentration of arsenic (As), barium (Ba), cadmium (Cd), chromium total
(Crtotal ), copper (Cu), mercury (Hg), molybdenum (Mo), nickel (Ni), lead (Pb), antimony
(Sb), selenium (Se), zinc (Zn), fluoride (F), chloride (Cl), and sulfate (SO4 2− ) determined
in the leachates of the spent refractory materials are compared against values given by
Council Decision 2003/33/EC. According to this comparison, the spent refractory materials
are classified as inert, non-hazardous, and hazardous (Table 1).

Table 1. Leaching limit values established by Council Decision 2003/33/EC [15].

Leaching Limit Values (L/S = 10 L/Kg)

Component Inert Non-Hazardous Hazardous (Li )
pH minimum 6
mg/Kg
As 0.5 2 25
Ba 20 100 300
Cd 0.04 1 5
Crtotal 0.5 10 70
Cu 2 50 100
Hg 0.01 0.2 2
Mo 0.5 10 30
Ni 0.4 10 40
Pb 0.5 10 50
Sb 0.06 0.7 5
Se 0.1 0.5 7
Zn 4 50 200
F 10 150 500
Cl− 800 15,000 25,000
SO4 2− 1000 20,000 50,000
Appl. Sci. 2022, 12, 3016 5 of 21

For the implementation of the proposed methodological approach, specific laboratory

work and sample treatments have to be implemented.
The leachability of potentially hazardous substances from waste and spent refractory
materials depends on the grain size distribution, solid-to-liquid ratio, mineral composition
of the virgin material, content of amorphous components, and structure [9]. The As, Ba, Cd,
Cr, Sb, Se, Hg, Mo, Ni, and Pb content of the leachates was obtained applying inductively
plasma mass spectrometry (ICP-MS) and inductively coupled plasma optical emission
spectrometry (ICP-OES). The Cu and Zn concentration of the leachate was determined
using atomic absorption spectroscopy (AAS). Fluoride, Cl− , and SO4 2− concentrations
in leachates were measured applying titrating and gravimetric methods, respectively.
X-ray diffraction (XRD) was applied for mineralogical characterization of spent refractory
materials. As a complementary technique with XRD analysis, scanning electron microscopic
(SEM) analysis was applied. In general, the mineralogical and chemical composition results
are discussed to explain further the findings of the leachability tests and mobility of
inorganic substances.

2.3. Multicriteria Decision Aid Methodologies and Circular Economy

According to Siskos and Spyridakos [18] and Pardalos et al. [19], MCDA procedures
can be organized into four main theoretical trends: (a) value system approaches (analytic
hierarchy/network processes—AHP/ANP [20–22], multi-attribute value/utility theory—
MAVT/MAUT [23–26]), (b) outranking relations approaches [27–31], (c) disaggregation–
aggregation approach [32–34], and (d) the multi-objective optimization approach [35–38].
Over the last decades, many studies [39–41] have been conducted to determine the
most appropriate MCDA method for various research topics. These topics consider a wide
variety of features, e.g., the type of the decision problem, the information required by
the decision-maker, the dimensions of the performance table (i.e., number of criteria), the
treatment of uncertainty, and the available robustness analysis techniques. Moreover, a
decision support system was recently developed by Cinelli et al. [42] for recommending
multicriteria decision analysis methods.
Many researchers have applied MCDA methods on issues related to CE [43–50].
According to Vlachokostas et al. [51], the AHP is mainly applied to real-life cases, while
the MCDA approach is adopted to study issues related to incineration process, which
present many similarities to the refractory material studied in the current research. More
specifically, this study deals with the environmental appropriateness of the refractory
materials which are involved, among others, in the incineration process.

3. Proposed Methodology for Appropriateness of Recycling

The proposed methodology for evaluating the appropriateness of recycling for refrac-
tory materials is a two-stage process. The first stage in an on/off process tests the refractory
spent material in fulfilling the appropriateness for recycling. The second stage is based on
a multicriteria methodological approach (MCDA) for the estimation of appropriateness
for recycling on a scale of 0–1 [28,52]. The proposed MCDA methodology is mainly based
on the estimation of additive value functions, which represent preferences of the decision
making by assessing an overall score for each alternative [53,54]. The abovementioned
index is further exploited to identify the use of the recycled refractories, considering the
granulometric and mineral content. Three samples were received from the REFRACT
project and nine virtual samples were used for the validation of the MCDA methodology.

3.1. The On/Off Criteria Process

The on/off criteria process is triggered by the chemical analysis implemented to
representative samples of the spent refractories. The acceptance or not for recycling of the
spent refractories is based on a set of rules related to the limits suggested by the Council
Decision 2003/33/EC [15] concerning the metals and the undesirable substance (Table 1).
Appl. Sci. 2022, 12, x FOR PEER REVIEW 6 of 19

Appl. Sci. 2022, 12, 3016 6 of 21

spent refractories is based on a set of rules related to the limits suggested by the Council
Decision 2003/33/EC [15] concerning the metals and the undesirable substance (Table 1).
Figure 11includes
Figure includesthe steps
the for for
steps the proposed MCDA
the proposed procedure
MCDA according
procedure to the Council
according to the
Decision 2003/33/EC.
Council Decision 2003/33/EC.

Figure 1. The proposed MCDA (multicriteria decision analysis) procedure and adopted EU frame-
Figure 1. The proposed MCDA (multicriteria decision analysis) procedure and adopted EU frame-
work [15] in the investigation of achievement SDGs (sustainable development goals) for the refrac-
work [15] inmaterial
tory spent the investigation of achievement
(EAI—environmental SDGs (sustainable
appropriateness development goals) for the refractory
index).
spent material (EAI—environmental appropriateness index).
The on/off criteria process examines the content of the refractory samples and the
The on/off criteria process examines the content of the refractory samples and the
existence of a substance that exceeds the hazardous limits of Table 1, which can character-
existence of a substance that exceeds the hazardous limits of Table 1, which can characterize
ize the spent refractories as hazardous:
the spent refractories as hazardous:
∃ i: Tdi ≥ Li the refractories are hazardous, (1)
∃ i: Tdi ≥ Li the refractories are hazardous, (1)
∀ i: T < L , then the refractories are evaluated for environmental appropriatenesss, (2)
∀ i : Tdi < Li , then the refractories are evaluated for environmental appropriatenesss,
where di is component i, Tdi represents the content of component di, and Li is the hazard- (2)
ous limit.
where di is component i, Tdi represents the content of component di , and Li is the hazardous
limit.
Appl. Sci. 2022, 12, 3016 7 of 21

If all the component contents fall within the permissible limits for inert and nonhaz-
ardous substances, we can proceed to the next step to identify the degree of the appropriate-
ness of the refractories for recycling (Figure 1). For example, a refractory sample with more
than 70 mg·Kg−1 Cr content but all the other components lower than the hazardous limits
is characterized as hazardous and has to be secondarily examined as far as the granularity
and the mineral content are concerned.
In cases where the component content falls within the permissible limits for the
examined components, the MCDA approach is applied to identify the environmental
appropriateness index (EAI).

3.2. Assessment of the Nonhazardous Spent Refractory Recycling Compatibility

The proposed multicriteria decision aid approach aims to assess the additive value
model described below. Given a sample of refractory material d evaluated on n preference
monotone or nonmonotone attributes G = {g1 ,g2 , . . . , gn }, a global value [32,52] is assigned
through the utilization of the following formulae:
n
U(d) = ∑ pi ui (d(gi )), for i = 1, 2, .., n, (3)
i=1

n
∑ pi = 1, pi ≥ 0, for i = 1, 2, .., n, (4)
i=1

where n is the number of attributes, d(g) = (d(g1 ), . . . , d(gn )) is the evaluation vector of
the refractory material attributes, gi* and gi * are the least and most preferable levels of the
attribute gi , respectively, and ui (gi ) and pi are the value function and the relative weight of
the i-th attribute; u(gi* ) = 0, u(gi *) = 1.
For every pair of refractory material samples dk and dr , the additive value function
satisfies the following conditions: (a) if U(dk (g)) > U(dr (g)), then dk is more environmen-
tally compatible than dr ; (b) if U(dk (g)) = U(dr (g)), then ak has the same environmental
compatibility as ar .
The utilization of the marginal value functions supports the measurement of the
relative impacts of every attribute, the importance of which concerns the environmental
impacts.

3.2.1. Development of the Marginal Value Functions

The marginal value functions [52,55] denote the degree of environmental impacts of
a factor di in the examined sample on a scale of 0–1, where 1 corresponds to totally inert
materials and 0 corresponds to the level of concentration of hazardous materials (Li ). It
is worth mentioning that the marginal value functions are usually not linear and can be
upward, downward, concave, or s-shaped (Figure 2).
The curvature of the value function corresponds to the behavior of the factor as
far as the environmental issues are concerned. A concave upward function means that
significant environmental impacts are raised for low values of the attribute, while, in
concave downward functions, only higher values of the attributes are significant for the
environmental impacts. Siskos et al. [55] used a family of 12 general parametric forms f (a, b,
c; g) to cover all types of monotone marginal value functions (concave, cave, and s-shaped
general), as well as non-monotone attributes with ∪-shaped and ∩-shaped functions for
the construction of the marginal values. There are two forms of marginal value functions:
increasing preferences assigned to cases where the environmental impact is decreasing for
higher values, and decreasing preferences where the environmental impact is increased
for the higher values. In this study, the decreasing preference value functions are suitable
for the parameters used, where a higher metal concentration leads to lower preferences
as far as the environmental impacts are concerned. The exact estimation of the a, b, and
c parameters of the value function is achieved utilizing a modified mid-value split point
 ative impacts of every attribute, the importance of which concerns the environmental im-
pacts.

3.2.1. Development of the Marginal Value Functions

Appl. Sci. 2022, 12, 3016 The marginal value functions [52,55] denote the degree of environmental impacts of
8 of 21
a factor di in the examined sample on a scale of 0–1, where 1 corresponds to totally inert
materials and 0 corresponds to the level of concentration of hazardous materials (Li). It is
worth mentioning
technique that with
and dialogue the marginal value
experts to functions
conclude are usually value
the appropriate not linear and can
function be up-
for one of
ward, downward,
the attributes. concave, or s-shaped (Figure 2).

(a) (b)

Figure 2. Examples of marginal value functions: (a) downward concave value function for min value
functions for min preference attributes; (b) upward concave value function for min value functions
for min preference attributes.

Actually, in this case, six consecutive ranges of the criterion range were used, and the
experts ranked them according to their importance for changes from the lower value to the
higher ones. In this way, the curvature of the marginal value functions was assessed, as well
as the general form of the marginal value function. Then, utilizing the visual capabilities
and artificial intelligence capabilities of the Multicriteria Interactive Intelligence Decision
Aiding System (MIIDAS), the exact parameters of a, b, and c were calculated, and the
precise form of the marginal value function was determined. Furthermore, interactive
and visual processes can be used, where the shape of the value function is changed in
controlled steps providing to the experts the marginal values to specific samples with
known concentrations. In this way, experts are able to identify the function closer to their
attitudes.

3.2.2. Weight Assessment

There are a large number of proposed methods in the literature for the estimation of cri-
teria weights [56], such as AHP [22], the Simos technique [57–59], the MAUT compensatory
technique [52], MACBETH (measuring attractiveness through a categorical-based evalua-
tion technique) [60], and the weight assessment through prioritization (WAP) method [61].
In the proposed methodology, the weights of each attribute were assessed using the
WAP method and the relative software developed to support the implementation of the
method [61]. For every attribute ai , a weight pi is estimated. The method is proceeds in the
following steps:
(a) The experts rank order the attributes and the ex aequo sets of attributes in a pairwise
manner, concerning their importance in environmental issues. The attributes are
sorted according to their ranking from the most important to the least important and
arranged in m classes (m ≤ n). Each class includes one attribute or a set of ex aequo
criteria.
(b) The key point of the WAP method is the utilization of the z indices for every pair of
successive attributes or sets of ex aequo attributes that satisfy the following formula:

pr
= zr , for r = 1, . . . , m– − 1, (5)
pr+1
where m is the number of importance classes for the criteria, z ≥ 1.
The WAP method does not require the precise identification of the z indices but a
range [zminr , zmaxr ] where the value of zr may vary. For two successive attributes or sets of
ex aequo attributes (i.e., gr , gr+1 ), the range [zminr , zmaxr ] is identified such that zminr ≤ zr
Appl. Sci. 2022, 12, 3016 9 of 21

≤ zmaxr . In order to facilitate the experts to identify this value system, visual interactive
techniques are used. Through this approach, the DM is asked to sort the n attributes into
m classes (m ≤ n) and to identify 2(m–− 1) values for the zminr and zmaxr indices (r = 1, 2,
. . . , m–− 1). Scroll bars are used to assist the visualization of the difference of the relative
importance between two successive attributes or ex aequo sets of criteria, ordered by their
ranking. The zminr and zmaxr values are automatically calculated and presented with bars
and unique labels.
Having identified the zmin and zmax for all the pairs of the successive classes, lin-
ear programming (LP) techniques are employed in order to estimate the minimum and
maximum values of the weights. The following 2n optimization problem describes the
estimation of the weight vectors for the maximization and minimization of the criteria
weights:
Min pi and Max pi , for i = 1, 2, . . . , n, (6)
subject to pi –− pi+1 = 0; if gi+1 is followed by gi and gi+1 belongs to the same importance
class r) as gi or–pi − pi+1 * zminr ≥ 0,–pi − pi+1 * zmaxr ≤ 0; if gi is followed by gi+1 , gi
belongs to the most important class (r) and gi+1 belongs to class (r+1), p1 + p2 + . . . + pn =
1, p1 ≥ 0, p2 ≥ 0, . . . , pn ≥ 0.
Actually, the solutions emerging from the above LPs will lead to the identification
of the minimum and maximum values of the criteria weights into the hyper-polyhedron
(infinite solutions).
(c) The robustness of the estimated hyper-polyhedron is calculated through the utilization
of two indices. The first type of index used is the range between the maximum
and minimum values of the criteria weights for every criterion, as these values are
estimated for each vertex of the hyper-polyhedron. This gives a picture, at first glance,
of the extent of robustness in each criterion. For instance, this index for the i-the
attribute is calculated as follows:
    
µi = max pij − min pij , forI1, 2, . . . , n I j = 1, 2, . . . , m, (7)

where n is the number of attributes, m is the number of vertices of the hyper-polyhedron,

and pij is the weight of the i attribute of the j vertex.
The second index used represents the normalized standard deviation of the different
solutions corresponding to the hyper-polyhedron vertices, where the value 1 corresponds
to the total robustness of the preferred models. This normalized index is called the average
stability index (ASI) [62–64].
s 
 2    
n m j m j 2
∑ i=1 m ∑ j = 1 pi − ∑ j = 1 pi
ASI = 1 − p , (8)
m (n − 1)

where n is the number of attributes, and m is the number of vertices of the hyper-
polyhedron.
Furthermore, for the set of 2n weight vectors, the barycenter is estimated. If the
values of the barycenter are considered as satisfactory by the DM along with the level of
robustness, then it can be used as a working vector of weights. Otherwise, there are two
main capabilities provided by the proposed approach. The first focuses on updating or
altering particular initial preferences, such as the values of the zmin and zmax or the criteria
ranking. The second capability concerns the feedback with which the robustness can be
analyzed, utilizing the tomographical approach [65] and eliciting additional preference
information.
Appl. Sci. 2022, 12, 3016 10 of 21

4. Results
4.1. Evaluation Model for Spent Refractories
The additive value model was assessed to fulfil the main objectives of REFRACT
project. A further analysis was required concerning the appropriateness of the spent
refractories for different types of recycling (open- or closed-loop). Table 2 tabulates the
components content of the examined refractory samples used for the validation of the
method. S2051, S2052, and S2053 were received from the REFRACT project, while REF01–
REF09 were virtual samples.

4.1.1. Estimation of Marginal Value Functions Using the MIIDAS System

The experts utilized the techniques and visual capabilities of the MIIDAS system
for the construction of the marginal value function [55]. For the components As, Cd, Cr,
Sb, and Hg, upward concave functions were selected by the experts, whereby, for these
components, lower concentrations were considered significant for environmental impacts.
For the other attributes, downward concave functions were used since their impact was
only considered significant for higher concentrations. Every marginal value function was
refined through the process described above to identify the exact functions corresponding
to every attribute.
Table 3 includes the graphs of the marginal value function for the 15 attributes derived
from the screen layout of the MIIDAS system used for the construction.
An example for developing the value function of As is demonstrated (Table 3). Since
the leaching limit value for the inert class for As is 0.5 mg·Kg−1 As, it is categorized in
Group I. It is well known that As is a highly toxic element in its inorganic form and is
widely recognized as carcinogenic, even at low concentrations [66,67]. The symptoms of As
toxicity in humans include skin cancer, gangrene, leucomelanosis, keratosis, melanosis, and
cancer of lungs, prostate and liver [66,67]. On the basis of the new scientific data correlated
to the adverse health effects of As, the European Council and World Health Organization
have established 10 µg As·L−1 as the new parametric value for As in potable water, in
contrast to the initial parametric value of 50 µg As·L−1 .
A hazardous material containing a content of As higher than the leaching limit value
of 0.5 mg·Kg−1 (Table 3) may release this element in the environment, contaminate ground-
water, and pose a severe threat to human health. In this case, the slope of the proposed
curve demonstrating the value function of As is high, suggesting that the value function of
the MCDA approach for this element is rigorous considering the element’s toxicity.
Appl. Sci. 2022, 12, 3016 11 of 21

Table 2. Samples (n = 12) of spent refractories, along with inert and hazardous limits.

Components S2051 S2052 S2053 REF01 REF02 REF03 REF04 REF05 REF06 REF07 REF08 REF09 Inert Non-
Hazardous
As 0.050 0.050 0.050 0.400 0.200 0.270 1.000 0.700 1.900 22.000 13.000 19.000 0.5 2
Ba 0.500 0.500 0.500 17.000 7.000 9.000 80.000 71.000 22.000 278.000 101.000 144.000 20 100
Cd 0.002 0.002 0.002 0.020 0.010 0.030 0.700 0.180 0.310 3.000 2.000 3.000 0.04 1
Crtotal 0.500 0.500 0.500 0.100 0.250 0.400 4.000 0.600 0.600 11.000 55.000 61.000 0.5 10
Cu 1.000 1.000 1.000 0.200 0.160 0.170 26.000 2.100 39.000 100.000 55.000 77.000 2 50
Hg 0.010 0.010 0.010 0.001 0.007 0.002 0.140 0.070 0.160 0.330 0.570 0.900 0.01 0.2
Mo 0.500 0.500 0.500 0.050 0.010 0.400 2.000 9.000 0.700 29.000 17.000 22.000 0.5 10
Ni 0.040 0.040 0.040 0.140 0.110 0.270 3.000 3.000 7.000 14.000 39.000 11.000 0.4 10
Pb 0.500 0.500 0.500 0.050 0.340 0.300 5.000 0.610 8.000 38.000 11.000 48.000 0.5 10
Sb 0.050 0.050 0.050 0.010 0.020 0.050 0.210 0.440 0.570 0.800 3.900 0.800 0.06 0.7
Se 0.050 0.050 0.050 0.040 0.070 0.007 0.390 0.370 0.400 6.000 6.000 0.600 0.1 0.5
Zn 0.620 0.620 0.620 1.000 2.000 3.000 19.000 37.000 34.000 199.000 144.000 147.000 4 50
F 0.900 1.000 1.000 2.000 7.000 5.000 102.000 11.000 74.000 354.000 322.000 222.000 10 150
Cl− 71.000 71.000 71.000 457.000 91.000 770.000 4.000 1087.00 14,700.00 5.000 24,711.00 16,000.00 800 15,000
SO4 2− 148.000 123.000 288.000 222.000 124.000 7.000 11.000 14,777.00 5478.00 547.000 49,780.00 41,547.00 1000 20,000
 Appl.
Appl. Appl. Sci. 2022,
Sci. 2022,
Sci. 2022, 12, xx FOR
12, 3016
12, FOR PEER
PEER REVIEW
REVIEW 1121of
12 of11 of 19
19
Appl.
Appl.
Appl. Sci.2022,
Sci.
Sci. 2022,
2022, 12,xxxFOR
12,
12, FORPEER
FOR PEERREVIEW
PEER REVIEW
REVIEW 11 of
11
11 of 19
of 19
19

Table 3. Marginal value functions for the 15 examined components—screen layouts of the MIIDAS
Table
Table
Table3. 3.
3. Marginal
Marginal
3.Marginal value
value
Marginalvalue functions
functions
valuefunctions for
for
functionsfor the
the
forthe 15
15
the15 examined
examined
15examined components—screen
components—screen
examinedcomponents—screen layouts
layouts
components—screenlayouts of
of
layoutsof the
the
ofthe MIIDAS
MIIDAS
theMIIDAS
MIIDAS
Table
Table
system. 3. Marginal value functions for the 15 examined components—screen layouts of the MIIDAS
system.
system.
system.
system.
system.
U
U ==
U===UU
U U(a,
(a,
U(a, b,
= b,
(a,b, c;
U (a,
c;
b,c; di)/[min,
di)/[min,
c;di)/[min,
di)/[min,max]max]
b, c; di)/[min,
max]
max]
max]
No.No
No Component
Component U
U U (a, b, c; di)/[min, max] Marginal Value
Marginal Value
Value Function
Function
No Component
Component
No Component
No Component a, b,
a, b, c Real
a, b,
c Real Marginal
MarginalValue
Marginal
Marginal Value Function
ValueFunction
Function
Function
a,
a, b,ccccReal
a,b,
b, Real
Real
Real

𝑢𝑢 𝑑𝑑 = =
u(𝑎𝑎 − 𝑏a𝑒− b ec(1+,,, 25𝑑𝜖
d)− )𝑑𝜖
9d
0, 25
𝑢𝑢𝑢 𝑑𝑑𝑑 = == 𝑎𝑎−
𝑎 −−=𝑏𝑏 𝑏𝑏𝑒𝑒𝑒𝑒 ,, 𝑑𝜖
𝑑𝜖
, 0,
𝑑𝜖 de25
0,
0,
0,
[0, 25]
25
25
25
1 11 As
AsAs a = a = −
−0.0034999, 0.0034999,
b = b = − 1.881829,
−1.881829,
11 As
As a = −0.0034999,
−0.0034999, bb == −1.881829,
−1.881829,
1 As aaa===−0.0034999,
−0.0034999,bb −1.881829,
c = −==0.628751
−1.881829,
cccc ==== −0.628751
−0.628751
−0.628751
−0.628751
c = −0.628751

𝑢𝑢 𝑑𝑑 = = 𝑎(d−
− 𝑏𝑒 ,,𝑑𝜖
𝑑𝜖 0, [300
9d
=u𝑎𝑎 𝑏𝑒 0, 300
c(1+ 300 ) de
2 Ba 𝑢𝑢𝑢 𝑑𝑑𝑑 = = 𝑎𝑎 −
−
−
)= 𝑏𝑒
𝑏𝑒𝑏𝑒a − be ,,,𝑑𝜖
𝑑𝜖, 0,
𝑑𝜖 0,300
0, 300
0, 300]
300
2 2 Ba
BaBa aaaa ==== −0.3045179,
−0.3045179,− b = −1.533395,
−
222 Ba
Ba a = 0.3045179,
b =
−0.3045179,bbb===−1.533395,
−0.3045179,
a = −0.3045179,
b = 1.533395,
−1.533395,
−1.533395,
−1.533395,
c = −0.161651
cccc ==== −0.161651
−0.161651
−0.161651
−0.161651
c = −0.161651

𝑢𝑢 𝑑𝑑 = =
=u𝑎𝑎 𝑎− − 𝑏𝑒 𝑏𝑒a − bec(1+,,,,,9d5 )𝑑𝜖
𝑏𝑒 𝑑𝜖𝑑𝜖 0, 0,
0,1[11
𝑢𝑢𝑢 𝑑𝑑𝑑 == 𝑎(𝑎d−− )− = 𝑏𝑒
𝑏𝑒 ,𝑑𝜖
𝑑𝜖 0,
0,
de 110, 1]
aaaa ==== 0.5390968,
0.5390968,
0.5390968, bb == −0.9881042,
−0.9881042, cc == −0.7626
−0.7626
0.5390968,
a = 0.5390968, a = 0.5390968, bbb===−0.9881042,
−0.9881042,
−0.9881042,
b = −0.9881042, ccc===−0.7626
−0.7626
−0.7626
c = −0.7626
3 333 Cd
Cd c ( 1 + 9d
)
33 CdCd
Cd
Cd 𝑢
𝑢 𝑑 =𝑑 =
= u 𝑎
( d
𝑎𝑎 𝑏𝑒 ) = 𝑏𝑒
𝑏𝑒 a + be ,
,, 𝑑𝜖
5
𝑑𝜖
,
𝑑𝜖 1,5
de
1,5 [1, 5]
𝑢𝑢𝑢 𝑑𝑑𝑑 ==𝑎
a =𝑎
𝑏𝑒
𝑏𝑒
0.64202844, b =,, −𝑑𝜖
𝑑𝜖 1,51,5
1,5
0.009505888,
a = 0.64202844,
a = 0.64202844,
0.64202844,cbb b = −0.009505888,
aaa===0.64202844,
0.64202844, bb=====0.4210001
−0.009505888,
−0.009505888,
−0.009505888,
−0.009505888,
cccc ==== 0.4210001
0.4210001
0.4210001
0.4210001
c = 0.4210001

𝑢𝑢 𝑑𝑑 = =
=u𝑎(𝑎𝑎− − 𝑏𝑏 𝑒𝑒 ,,𝑑𝜖
𝑑𝜖
9d 0, 0, 70]
𝑢𝑢𝑢 𝑑𝑑𝑑 = 𝑎𝑎d− )−=𝑏𝑏𝑏𝑒a𝑒𝑒− b ec(1,,+,𝑑𝜖
𝑑𝜖 0,
= − 𝑑𝜖 0,
70 ) ,0,
70]
70]
70]
[0, 70]
de70]
4 Cr a = −0.00727097, b = −1.742195,
4 44 Cr total
44 CrCr
Cr
Cr
total
total
total
total
total
aa === −0.00727097,
a −0.00727097,
= −
−0.00727097,
a 0.00727097,bb === −1.742195,
b
a = −0.00727097, b = −1.742195,
−1.742195,
b = −
−1.742195,1.742195,
cc == −0.547901
−0.547901
c = −0.547901
ccc===−0.547901
−0.547901
−0.547901

𝑢𝑢 𝑑𝑑 = =
=𝑎𝑎 𝑎− − 𝑏𝑒
𝑏𝑒 ,, 𝑑𝜖 𝑑𝜖 0,0, 50
50
𝑢𝑢𝑢 𝑑𝑑𝑑 = = 𝑎𝑎− −−𝑏𝑒 𝑏𝑒
𝑏𝑒 ,,9d, 𝑑𝜖𝑑𝜖 0,
𝑑𝜖 0,0,50
50
50
aaaa ==== 0.5483023,
0.5483023,
0.5483023,
0.5483023, u ( d ) b
b
b
b
= ====a−0.9684657,
−0.9684657,
− c(1+ 100 ) , cc
−0.9684657,
−0.9684657,
be c
c ====de−0.7627
−0.7627
−0.7627
−0.7627
[ 0, 50]
a = 0.5483023, b = −0.9684657, c = −0.7627
a = 0.5483023, b = −0.9684657, c = −0.7627
55 Cu
Cu 𝑢𝑢 𝑑𝑑 = =
=u𝑎𝑎𝑎 𝑏𝑒 𝑏𝑒
𝑏𝑒a + bec(1+,,,100 ,,9d𝑑𝜖
𝑑𝜖 50,100
50,100
5 555 CuCu
Cu
Cu 𝑢𝑢𝑢 𝑑𝑑𝑑 = = 𝑎𝑎(d) 𝑏𝑒 𝑏𝑒
= 𝑑𝜖
𝑑𝜖𝑑𝜖 50,100
50,100
) , 50,100
de[50, 100]
a = 0.8111737,
aa === 0.8111737,
0.8111737,
0.8111737, b = −0.05840966,
bb === −0.05840966,
−0.05840966,
−0.05840966,
a a = 0.8111737,b
a = 0.8111737, b = −0.05840966, b = − 0.05840966,
cc == 0.2631001
0.2631001
c = 0.2631001
ccc===0.2631001
0.2631001
0.2631001

𝑢𝑢 𝑑𝑑 ==
=𝑎𝑎𝑎−
− 𝑏𝑒
𝑏𝑒 ,, 𝑑𝜖
𝑑𝜖 0,
0, 0.2
0.2
𝑢𝑢𝑢 𝑑𝑑𝑑 =
= 𝑎𝑎−
−𝑏𝑒
− 𝑏𝑒
𝑏𝑒 ,,, 𝑑𝜖
𝑑𝜖 0,
𝑑𝜖 0,0.2
0, 0.2
0.2
a = 0.5864021,
aaa === 0.5864021,
0.5864021, b
b
b =
=
= −2.104839,
−2.104839,
−2.104839, cc
c =
=
= −1.6271
−1.6271
−1.6271
0.5864021, bb == −2.104839,
a = 0.5864021, −2.104839, cc == −1.6271
−1.6271
66 Hg
Hg 𝑢𝑢 𝑑𝑑 = = 𝑎 𝑏𝑒 𝑏𝑒 ,,𝑑𝜖
𝑑𝜖 0.2, 2
666 Hg
Hg
Hg 𝑢𝑢𝑢 𝑑𝑑𝑑 = ==𝑎𝑎
𝑎𝑎 𝑏𝑒 𝑏𝑒
𝑏𝑒 ,,,𝑑𝜖
0.2,
0.2, 22
𝑑𝜖 0.2,
𝑑𝜖 0.2, 22
aaaa ==== 0.6454874,
0.6454874,
0.6454874,
b
b =
= −0.009583134,
−0.009583134,
0.6454874,bbb===−0.009583134,
−0.009583134,
−0.009583134,
a = 0.6454874,
cccc ==== 0.4210001
0.4210001
0.4210001
0.4210001
c = 0.4210001
Appl. Sci. 2022, 12, 3016 𝑢 𝑑 = 𝑎 − 𝑏𝑒 , 𝑑𝜖 0, 50 13 of 21
a = 0.5483023, b = −0.9684657, c = −0.7627
5 Cu 𝑢 𝑑 = 𝑎 𝑏𝑒 , 𝑑𝜖 50,100
a =Table 3. Cont. b = −0.05840966,
0.8111737,
c = 0.2631001
U = U (a, b, c; di)/[min, max]
No. Component Marginal Value Function
a, b, c Real

𝑢 𝑑 = 𝑎 − 𝑏𝑒 ,9d 𝑑𝜖 0, 0.2
u(d) = a − bec(1+ 2 ) , de[0, 0.2]
a = 0.5864021, b = −2.104839, c = −1.6271
a = 0.5864021, b = −2.104839, c = −1.6271
9d
6 6 HgHg (d) =𝑏𝑒a + bec(1+, 2𝑑𝜖
) , de [0.2, 2]
𝑢 𝑑 =u𝑎 0.2, 2
a = 0.6454874, b = −0.009583134,
Appl. Sci. 2022, 12, x FOR a =REVIEW
PEER 0.6454874, b = −0.009583134, 12 of
of 19
19
Appl. Sci. 2022, 12, x FOR PEER REVIEW c = 0.4210001 12
c = 0.4210001

9d
𝑢𝑑
𝑢 𝑑 = =u𝑎𝑎
(d−−
)= 𝑏𝑒a − bec(1+,,30𝑑𝜖
𝑏𝑒 𝑑𝜖 0, 30
0, 30
) , de [0, 30]
7
7 7 Mo
Mo
Mo a = a = − 0.2018096,
−0.2018096, b = b = −1.465328,
−1.465328,
a = −0.2018096, b = −1.465328,
c = −0.198251
cc == −0.198251
−0.198251

𝑢𝑑
𝑢 𝑑 = =u𝑎𝑎
(d−−
)= 𝑏𝑒a − bec(1+,,9d40𝑑𝜖
𝑏𝑒 𝑑𝜖 0, 40
0,
) , de [40
0, 40]
8
8 8 NiNi
Ni a = −0.2032343,
a = − b =
0.2032343, −1.466112,
b = − 1.466112,
a = −0.2032343, b = −1.466112,
c = −0.197601
cc == −0.197601
−0.197601

𝑢𝑑
𝑢 𝑑 ==u𝑎𝑎
(d−−
)= 𝑏𝑒a − bec(1+,,9d50𝑑𝜖
𝑏𝑒 𝑑𝜖 0, 50
0,
) , de [50
0, 50]
9
9 9 PbPb
Pb a = −0.08888087,
a = − b
0.08888087,= −1.438429,
b = − 1.438429,
a = −0.08888087, b = −1.438429,
c = −0.278401
cc == −0.278401
−0.278401

𝑢=
𝑢 =𝑎 𝑎 u−
−=𝑏𝑒𝑏𝑒a − bec(1,,+𝑑𝜖
𝑑𝜖
9d
0,de5]
5 ) , 0, 5]
[0, 5]
10
10 10 SbSb a = −0.04104724, b = −1.491046,
Sb aa == −0.04104724,
−0.04104724, b == −1.491046,
b −1.491046,
c = −0.359251
cc == −0.359251
−0.359251

𝑢𝑑
𝑢 𝑑 ==𝑎 𝑎− − 𝑏𝑒𝑏𝑒 ,, 𝑑𝜖
𝑑𝜖 0,
0, 7]
7]
11
11 Se
Se a = −0.0270333, b = −1.538527,
a = −0.0270333, b = −1.538527,
cc == −0.404151
−0.404151
Appl. Sci. 2022, 12, 3016 14 of 21

𝑢𝑢 == 𝑎𝑎 −
− 𝑏𝑒
𝑏𝑒 ,,𝑑𝜖
𝑑𝜖 0,
0, 5]
5]
10
10 Sb
Sb aa == −0.04104724,
−0.04104724, bb == −1.491046,
−1.491046,
Table 3.ccCont.
== −0.359251
−0.359251

U = U (a, b, c; di)/[min, max]

No. Component Marginal Value Function
a, b, c Real

𝑢𝑢 𝑑𝑑 = =u(𝑎𝑎d−)−=𝑏𝑒 ,𝑑𝜖
𝑏𝑒a − bec(1,+𝑑𝜖 0,
7 ) , 0,
9d
7]
de7]
[0, 7]
11
1111 SeSe
Se a = −
aa == −0.0270333,0.0270333, b =
−0.0270333, bb == −1.538527, −
−1.538527, 1.538527,
c = −0.404151
cc == −0.404151
−0.404151

𝑢𝑢 𝑑𝑑 = =u(𝑎𝑎d)−
−=𝑏𝑒
𝑏𝑒
a − bec(1+,,𝑑𝜖
𝑑𝜖
9d
200 0,de200
) ,0, 200
[0, 200]
12
1212 ZnZn
Zn aa == −0.2032343,
−0.2032343, bb == −1.466112,
a = −0.2032343, −1.466112,
b = −1.466112,
Appl. Sci.
Sci. 2022,
2022, 12,
12, xx FOR
FOR PEER
PEER REVIEW
REVIEWc = −0.197601
c = −0.197601 13 of
of 19
19
Appl. Sci. 2022, 12, x FOR PEER REVIEWc = −0.197601
Appl. 13
13 of 19

𝑢 𝑑𝑑 =
𝑢𝑢 = 𝑎𝑎 −− 𝑏𝑒 𝑏𝑒 ,,9d,𝑑𝜖
𝑑𝜖 0, 500
) , 0,
13 F 𝑑a ==−0.2242262,
u𝑎(d−) = 𝑏𝑒a − bebc=(1+−1.478325,
500 𝑑𝜖 0, 500
de [500
0, 500]
1313
13 FF F aa == −0.2242262,
a = − b =
0.2242262, −1.478325,
b
−0.2242262, b = −1.478325, = − 1.478325,
ccc === −0.188601
−0.188601
c = −0.188601
−0.188601

𝑢 𝑑𝑑
𝑢𝑢 == 𝑎𝑎 − − 𝑏𝑒
𝑏𝑒 ,
,,,𝑑𝜖
𝑑𝜖 0,0, 15,000
0, 15,000
,
𝑑 =a =𝑎 − 𝑏𝑒
0.562557, b
,
c=
𝑑𝜖
−0.8913583,
(1+ 9d
)
15,000
aa ==u0.562557,
(d) = a − be
0.562557, bb == −0.8913583,
25,000 , de [0, 15, 000]
−0.8913583,
a =cc0.562557,
= −0.7118 b = −0.8913583,
c == −0.7118
−0.7118
c = −0.7118
14 Cl−− − 𝑢 𝑑𝑑 = =u𝑎𝑎(d) 𝑏𝑒 𝑏𝑒a + bec,,(1+ 25,000 ,,, ,
9d
1414
14 Cl
ClCl
− 𝑢𝑢 = )
𝑑𝑑𝜖=15,000,
𝑎 𝑏𝑒 25,000 ,
𝑑𝜖 15,000,
15,000, 25,000
de [ 15, 000, 25, 000 ]
a = a𝑑𝜖
= 0.6260664,
0.6260664,
25,000
b = −0.001916413,
b == −0.001916413,
−0.001916413,
aa == 0.6260664, b
0.6260664, bc == 0.5789001 −0.001916413,
ccc === 0.5789001
0.5789001
0.5789001

𝑢 𝑑𝑑 =
𝑢𝑢 = 𝑎𝑎 −− 𝑏𝑒𝑏𝑒 ,
,9d,,𝑑𝜖
𝑑𝜖 0,20,00020,000
𝑑𝜖) , 0,
,
a = 𝑑 = u𝑎(
0.4392703,−
d ) 𝑏𝑒
= b a=− be ,c(1+ 50,000
−0.84213393, 0,c [20,000
de 0,−0.4067
20, 000]
aa == 0.4392703,
0.4392703, bb == −0.84213393,
a = 0.4392703, b = −0.84213393,
−0.84213393, cc === −0.4067
c = −0.4067
−0.4067
15
1515 SO
SO 2− 2− c (1+ 9d
)
SO 42−4 𝑢 𝑑𝑑 =
𝑢𝑢 =u𝑎(d) = ,,,
𝑏𝑒a + be ,, 50,000 ,
15 SO442− = 𝑎𝑎000,𝑏𝑒
𝑑 [20,
de 𝑏𝑒50, 000],, a = 0.6567029,
𝑑𝜖 20,000,
𝑑𝜖 50,000 , a = c0.6567029,
𝑑𝜖 20,000,
20,000, 50,000
b=− 0.01649786, = 0.3684001
50,000 ,, aa == 0.6567029,
0.6567029,
b = −.01649786,
bb == −.01649786, c = 0.3684001
−.01649786, cc == 0.3684001
0.3684001

4.1.2.
4.1.2. Estimation
Estimation
4.1.2. of Criteria
of Criteria Weights
Weights Using Using
the WAPthe WAP Technique
Technique
4.1.2. Estimation
Estimation of of Criteria
Criteria Weights
Weights UsingUsing thethe WAP
WAP Technique
Technique
TableTable 444 provides
4 provides
Table provides the
thethe assessed
assessed
assessed ranking.There
ranking.
ranking. Thereare
There arevarious
are variousapproaches
various approaches suggesting
suggesting cri- cri-
teria Table
related provides
to potential thetoxicity,
assessedhuman
ranking. Thereand
health, various approaches
are ecological approaches
risk [66–68].suggesting
For the cri-
esti-
criteria
teriarelated
related to
topotential
potential toxicity,
toxicity, human
human health,
health, and ecological risk risk [66–68].
[66–68]. ForFor theesti-
teria
mationrelated thetocriteria
of the potential
criteria toxicity,
weights, the human
experts health,
ranked and
thetheecological
15 elements
elements risk
and [66–68]. For the
substances the esti-
accord-
estimation
mation of criteriaweights,
weights,the theexperts
experts ranked
ranked the 15 15 elements and and substances
substances accord-
mation
ing to of thepotential
to their
their criteria weights,
toxicity. the expertsthis
Through ranked the 15
process, fiveelements
different and substances
ranking classes accord-
(1–5)
according
ing to their potential
potential toxicity.
toxicity. Through
Through thisthis process,
process, five
five different
different ranking
ranking classes
classes (1–5)
ing
were to their
identified potential
for toxicity.
chemical Through
components, this process,
where 1 five
correspondsdifferent
to ranking
high classes
toxicity and 5(1–5)
cor-
(1–5)were
were identified
identified forfor chemical
chemical components,
components, where
where 1 corresponds
11 corresponds to to high toxicity and
were identified
responds to low for chemical
toxicity. Thecomponents,
term “expert where
opinion” corresponds
is used in to high
high toxicity
contrast toxicity
to
and
“expert
55 cor-
andjudge- cor-
responds
responds to
to lowlow toxicity.
toxicity. The term
The state
term of “expert
“expert opinion”
opinion” is used
used inin contrast to
to “expert judge-
ment” to
ment” to emphasize
emphasize an initial
an initial state of knowledge.
knowledge. Theis
The determination
contrast
determination “expert
of expertise
of expertise judge-
included
included
ment” to emphasize
professionals such such as an initial
as scientistsstate
scientists and of knowledge.
and managers. Experts The determination
Experts replied to to the of expertise
the following included
following research
research
professionals
professionals such asthe
scientists and managers.
managers. Experts replied
replied tothe
theranking
following research
question:
question: “What
“What are
are environmental quality expert’s views on of chemical
question:
components “What are the
concerning the environmental
environmental quality expert’s
their toxicity?”quality expert’s views
If the leaching
on
on the
viewsvalue
limit the ranking of
ofranking of chemical
a chemical chemical
compo-
Appl. Sci. 2022, 12, 3016 15 of 21

5 corresponds to low toxicity. The term “expert opinion” is used in contrast to “expert
judgement” to emphasize an initial state of knowledge. The determination of expertise
included professionals such as scientists and managers. Experts replied to the following
research question: “What are the environmental quality expert’s views on the ranking of
chemical components concerning their toxicity?” If the leaching limit value of a chemical
component is 4, 3, 2, 1, and 0 orders of magnitude (rounded to the nearest power of 10)
greater than the maximum leaching limit value for the inert category (1000 mg SO4 2− ·L−1 ),
it belongs to Groups I, II, III, IV, and V, respectively. This classification is also in line with the
parametric values for chemical components provided by the Council Directive 98/83/EC
on the quality of water intended for human consumption [69].

Table 4. Components and their ranking applied in this study.

Component Ranking
Cd 1
Hg 1
Appl. Sci. 2022, 12, x FOR PEER REVIEW Sb 1 14 of 19
Se 1
As 2
Crtotal 2
MoAs 22
Cr
Nitotal 22
Mo
Pb 22
CuNi 32
ZnPb 32
BaCu 43
FZn 43
−
ClBa 54
SO4F2− 54
Cl− 5
SO42− 5
The leaching limit values (L/S = 10 L/Kg) of chemical components for characterization
as inert,
Thenon-hazardous,
next step of the ormethodology
hazardous waste were sorted
involved in decreasing
the relation of each order
groupandto agrouped
rank, as
in five classes as
shown in Table 4. follows (Table 4): (a) Group I: Cd, Hg, Sb, Se; (b) Group II: As, Cr total , Mo,
Ni, Pb; (c) Group III: Cu, Zn; (d) Group IV: Ba, F; (e) Group V: Cl − , SO 2− .
Subsequently, the experts provided the relative importance (zmin4and zmax) of the five
Theutilizing
classes next stepthe
of the methodology
visual techniquesinvolved
of the WAP the relation
methodofand each group
the to a software.
relative rank, as
shown
Screenin Table 4.
layouts are presented in Figure 3 for a comparison of the first and second classes
Subsequently,
(Figure 3). the experts provided the relative importance (zmin and zmax ) of the five
classesTable
utilizing the visual
5 presents thetechniques of the
results of the WAP method
relative importanceand the relativeby
provided software. Screen
the experts, as
layouts are presented in Figure 3 for a comparison of the first and
well as the weights of the classes estimated by the linear program solution. second classes (Figure 3).

Figure3.3.Assessment
Assessmentof
ofthe
theZZmin and
and Z
Zmax between the first and second class.
Figure min max between the first and second class.

Table 5. Estimation
Table of the
5 presents weights.
results of the relative importance provided by the experts, as well
as the weights of the classes estimated
Class/Order by the linear program
Component solution.
[Zmin , Zmax] Pi μi
1 Cd, Hg, Sb, Se [1.3256, 1.5641] 0.1045 0.0186
2 As, Cr, Mo, Pb, Ni [1.439, 1.667] 0.0725 0.0885
3 Cu, Zn [1.299, 1.5] 0.0469 0.0094
4 Ba, F [1.083, 1.2472] 0.03375 0.0105
5 Cl, SO42− 0.0292 0.0127
Average stability index (ASI) 0.989
Appl. Sci. 2022, 12, 3016 16 of 21

Table 5. Estimation of weights.

Class/Order Component [Zmin , Zmax ] Pi µi

1 Cd, Hg, Sb, Se [1.3256, 1.5641] 0.1045 0.0186
2 As, Cr, Mo, Pb, Ni [1.439, 1.667] 0.0725 0.0885
3 Cu, Zn [1.299, 1.5] 0.0469 0.0094
4 Ba, F [1.083, 1.2472] 0.03375 0.0105
5 Cl, SO4 2− 0.0292 0.0127
Average stability index (ASI) 0.989

The above-presented methodological approach was utilized for the estimation char-
acterization of the samples of spent refractories in the frame of the REFRACT project
co-financed by the European Union and Greek national funds.

4.2. Classification of Refractory Spent Materials

The above additive value model was employed to estimate the environmental appro-
priateness index (EAI) for a set of samples with differentiations of the concentration of the
components. Table 6 includes the results of the additive value model for the calculation of
the environmental appropriateness index (EAI).
The EAI values for S2051, S2052, S2053, REF01, REF02, and REF03 samples were higher
than the EAI value of the inert sample. The EAI value of the inert sample was estimated
using the leaching limit values for the inert material (Table 6).
The component concentrations for samples S2051, S2052 and S2053 fell within the
inert range (Table 2). Their EAI values were also higher than the corresponding inert
limit for virtual samples REF01–REF03. The virtual samples (REF01–REF09) were used
in order to validate the EAI, and the results can be considered satisfactory, since the EAI
of samples REF04–REF06 varied between 0.75179 and 0.81631, which can be attributed to
the concentrations exceeding the inert limit for most of the components (Tables 2 and 6).
Moreover, sample REF07 presented a low value of EAI (0.31926), attributed to the high
content of the first-class components (Cd, Hg, Sb, Se), second-class components (As, Crtotal ,
Mo, Pb, Ni), third-class components (Cu, Zn), and fourth-class components (Ba, F) ex-
ceeding the upper nonhazardous limit. Furthermore, the lowest EAI value (0.2456) was
observed for the REF08 sample which presented high concentrations of Ni, Sb, Se, Cl− , and
SO4 2− , while REF09 (0.36595) presented high concentrations for all examined components
(Tables 2 and 6). Lastly, the above analysis reveals that the proposed index for measuring
the appropriateness of the spent material can provide a rational measure for this case study.
Appl. Sci. 2022, 12, 3016 17 of 21

Table 6. Results of the EAI for the studied refractory samples and inert sample.

Weight Components S2051 S2052 S2053 REF01 REF02 REF03 REF04 REF05 REF06 REF07 REF08 REF09 Inert Hazardous
0.07248 As 0.98871 0.98871 0.98871 0.91313 0.95558 0.94051 0.79673 0.85296 0.64924 0.00340 0.04942 0.01011 0.89262 0.63463
0.03375 Ba 0.99684 0.99684 0.99684 0.89677 0.95646 0.94429 0.58052 0.62000 0.86799 0.03429 0.49482 0.34437 0.87942 0.49871
0.10447 Cd 0.99874 0.99874 0.99874 0.98752 0.99372 0.98141 0.71542 0.89910 0.84026 0.49963 0.57436 0.49963 0.97538 0.65590
0.07248 Crtotal 0.96514 0.96514 0.96514 0.99293 0.98242 0.97201 0.75266 0.95831 0.95831 0.45683 0.01365 0.00644 0.96514 0.49071
0.04680 Cu 0.97003 0.97003 0.97003 0.99384 0.99507 0.99476 0.62412 0.93936 0.57936 0.00000 0.53169 0.34064 0.94206 0.56290
0.10447 Hg 0.97080 0.97080 0.97080 0.99698 0.97934 0.99399 0.73479 0.83414 0.71458 0.62332 0.60250 0.56516 0.97080 0.68203
0.07248 Mo 0.96479 0.96479 0.96479 0.99643 0.99929 0.97175 0.86522 0.50186 0.95099 0.01237 0.23544 0.12296 0.96479 0.46123
0.07248 Ni 0.99786 0.99786 0.99786 0.99253 0.99413 0.98564 0.84975 0.84975 0.67820 0.44246 0.00924 0.53459 0.97879 0.56813
0.07248 Pb 0.97306 0.97306 0.97306 0.99728 0.98160 0.98375 0.75866 0.96722 0.64036 0.03289 0.53857 0.00937 0.97306 0.57082
0.10447 Sb 0.96688 0.96688 0.96688 0.99329 0.98662 0.96688 0.86781 0.74221 0.67905 0.57954 0.04255 0.57954 0.96038 0.62099
0.10447 Se 0.97366 0.97366 0.97366 0.97887 0.96331 0.99627 0.81160 0.82036 0.80726 0.08420 0.01842 0.72491 0.94800 0.76501
0.04680 Zn 0.99338 0.99338 0.99338 0.98935 0.97879 0.96833 0.81296 0.66266 0.68607 0.00182 0.12240 0.12236 0.95796 0.56813
0.03375 F 0.99627 0.99585 0.99585 0.99172 0.97125 0.97940 0.64169 0.95513 0.72804 0.14385 0.18609 0.35195 0.95914 0.51149
0.02923 Cl− 0.99211 0.99211 0.99211 0.95166 0.98992 0.92167 0.99955 0.89365 0.55289 0.99944 0.03659 0.53012 0.91892 0.54817
0.02923 SO4 2− 0.99396 0.99497 0.98830 0.99096 0.99493 0.99971 0.99955 0.62936 0.81475 0.97799 0.00951 0.28179 0.96042 0.56896
EAI 0.98009 0.98010 0.97991 0.98128 0.98138 0.97621 0.78548 0.81631 0.75179 0.31926 0.24560 0.36595 0.95464 0.60180
Appl. Sci. 2022, 12, 3016 18 of 21

5. Conclusions
This study introduced a multicriteria approach for the characterization of refractory
material wastes concerning their recycling capabilities. For the above, a two-phase method-
ological approach was introduced. The first step of the proposed approach is to characterize
the refractory material as inert or hazardous according to the composition of its leachate.
Further analysis is performed in the second phase, where a multi-attribute index is intro-
duced for the estimation of the environmental appropriateness of refractory spent materials.
The critical features of the proposed methodological approach are (a) the utilization of a
weight vector for the 15 critical substances, representing their relative importance related to
the environmental impacts, and (b) a nonlinear value function encapsulating the variation
of the environmental impacts on the scale of the acceptable concentration of the components
in the refractory material waste.
The application of the proposed approach to a limited number of samples presented
satisfactory results in measuring the environmental impacts of wastes that can be recy-
cled. Engineers, decision-makers, and scientists may apply the proposed methodology
by evaluating the different qualities of refractory material to specific parts of industrial
systems since specific qualities of spent refractories can be used in several parts of kilns
and furnaces exposed in various temperatures. This constitutes a new direction of this
research and will be implemented in the next phase of the project, along with industrial
experiments.
Lastly, the EAI value of a sample classified as nonhazardous may exceed the EAI
value (0.95464) for inert limits. This corresponds to spent refractories with very good
environmental appropriateness in terms of most components; thus, the application of
further treatment should be investigated.
It is important to conduct further work on investigating the influence of uncertainties
on the estimated parameters (e.g., the parameters of the nonlinear function) in the decision
problem and to explore the suitability of other MCDA methods for this specific issue. The
elicitation of the preference structure from other experts will be beneficial for the investiga-
tion of these parameters. The application of robustness analysis techniques, evaluating the
preference structure and providing tools to improve it, will also be considered in future
work.

Author Contributions: Conceptualization and methodology, A.S., D.E.A. and I.V.; software, A.S. and
N.T.; validation, formal analysis, and investigation, A.S., D.E.A., I.V. and N.T.; resources and data
curation, A.S., D.E.A. and E.K.; writing—original draft preparation, A.S., D.E.A. and I.V.; writing—
review and editing, N.T., G.V. and E.K.; visualization and supervision, G.V.; project administration,
A.S. and E.K. All authors have read and agreed to the published version of the manuscript.
Funding: This research was supported by the European Regional Development Fund of the European
Union and Greek national funds (Greek Secretariat for Research and Innovation–GSRI) through the
Operational Program Competitiveness, Entrepreneurship, and Innovation, (EPAnEK 2014–2020)
under the call RESEARCH—CREATE—INNOVATE. Project: “Recycling of used refractories from
various industries to produce alumino-silicate refractories, ceramics, and masses”(project code:
T1EDK05442).
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Data were obtained from Mathios Refractories SA and are contained
within this article.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Tan, E.C.D.; Lamers, P. Circular Bioeconomy Concepts—A Perspective. Front. Sustain. 2021, 2, 701509. [CrossRef]
2. Ferreira, G.; López-Sabirón, A.M.; Aranda, J.; Mainar-Toledo, M.D.; Aranda-Usón, A. Environmental Analysis for Identifying
Challenges to Recover Used Reinforced Refractories in Industrial Furnaces. J. Clean. Prod. 2015, 88, 242–253. [CrossRef]
Appl. Sci. 2022, 12, 3016 19 of 21

3. Horckmans, L.; Nielsen, P.; Dierckx, P.; Ducastel, A. Recycling of Refractory Bricks Used in Basic Steelmaking: A Review. Resour.
Conserv. Recycl. 2019, 140, 297–304. [CrossRef]
4. Fang, H.; Smith, J.D.; Peaslee, K.D. Study of Spent Refractory Waste Recycling from Metal Manufacturers in Missouri. Resour.
Conserv. Recycl. 1999, 25, 111–124. [CrossRef]
5. Hanagiri, S.; Matsui, T.; Shimpo, A.; Aso, S.; Inuzuka, T.; Matsuda, T.; Sakaki, S.; Nakagawa, H. Recent Improvement of Recycling
Technology for Refractories. Spec. Issue Refract. Technol. Steel Ind. Nippon Steel Tech. Rep. 2008, 98, 93–98.
6. Poirier, J.; Blond, E.; de Bilbao, E.; Michel, R.; Coulon, A.; Gillibert, J.; Boussuge, M.; Zhang, Y.; Ryckelynk, D.; Dusserre, G.; et al.
New Advances in the Laboratory Characterization of Refractories: Testing and Modelling. Metall. Res. Technol. 2017, 114, 610.
[CrossRef]
7. Arianpour, F.; Kazemi, F.; Golestani-Fard, F.; Rasti, M. Characterisation of spent MgO-C refractory bricks with emphasise on
recycling. In Proceedings of the Unified International Technical Conference on Refractorie, Dresden, Germany, 18–21 September
2007; pp. 598–601.
8. Pantazopoulou, E.; Zebiliadou, O.; Mitrakas, M.; Zouboulis, A. Stabilization of Tannery Sludge by Co-Treatment with Aluminum
Anodizing Sludge and Phytotoxicity of End-Products. Waste Manag. 2017, 61, 327–336. [CrossRef]
9. Kasina, M.; Kajdas, B.; Michalik, M. The Leaching Potential of Sewage Sludge and Municipal Waste Incineration Ashes in Terms
of Landfill Safety and Potential Reuse. Sci. Total Environ. 2021, 791, 148313. [CrossRef] [PubMed]
10. Vlachokostas, C.; Achillas, C.; Diamantis, V.; Michailidou, A.V.; Baginetas, K.; Aidonis, D. Supporting Decision Making to Achieve
Circularity via a Biodegradable Waste-to-Bioenergy and Compost Facility. J. Environ. Manag. 2021, 285, 112215. [CrossRef]
[PubMed]
11. Kyriakopoulos, G.L.; Kapsalis, V.C.; Aravossis, K.G.; Zamparas, M.; Mitsikas, A. Evaluating Circular Economy under a Multi-
Parametric Approach: A Technological Review. Sustainability 2019, 11, 6139. [CrossRef]
12. European Parliament. Council of the European Union Directive 2008/98/EC of the European Parliament and of the Council of 19
November 2008 on Waste and Repealing Certain Directives. Off. J. Eur. Union L312 2008, 51, 3–30.
13. Rodrigues, P.; Silvestre, J.D.; Flores-Colen, I.; Viegas, C.A.; De Brito, J.; Kurad, R.; Demertzi, M. Methodology for the Assessment
of the Ecotoxicological Potential of Construction Materials. Materials 2017, 10, 649. [CrossRef] [PubMed]
14. Rodrigues, P.; Silvestre, J.D.; Flores-Colen, I.; Viegas, C.A.; Ahmed, H.H.; Kurda, R.; de Brito, J. Evaluation of the Ecotoxicological
Potential of Fly Ash and Recycled Concrete Aggregates Use in Concrete. Appl. Sci. 2020, 10, 351. [CrossRef]
15. Council of the European Union. Council of the European Union 2003/33/EC: Council Decision of 19 December 2002 Establishing
Criteria and Procedures for the Acceptance of Waste at Landfills Pursuant to Article 16 of and Annex II to Directive 1999/31/EC.
Off. J. Eur. Communities L11 2003, 46, 27–49.
16. Huysman, S.; Debaveye, S.; Schaubroeck, T.; Meester, S.D.; Ardente, F.; Mathieux, F.; Dewulf, J. The Recyclability Benefit Rate of
Closed-Loop and Open-Loop Systems: A Case Study on Plastic Recycling in Flanders. Resour. Conserv. Recycl. 2015, 101, 53–60.
[CrossRef]
17. EN 12457-2:2002; Characterisation of Waste-Leaching-Compliance Test for Leaching of Granular Waste Materials and Sludges.
One Stage Batch Test at a Liquid to Solid Ratio of 10 l/Kg for Materials with Particle Size below 4 Mm (without or with Size
Reduction). European Committee for Standardization: Brussels, Belgium, 2002.
18. Siskos, Y.; Spyridakos, A. Intelligent Multicriteria Decision Support: Overview and Perspectives. Eur. J. Oper. Res. 1999, 113,
236–246. [CrossRef]
19. Pardalos, P.M.; Siskos, Y.; Zopounidis, C. Editorial: Advances in Multicriteria Analysis; Kluwer Academic Publishers: Dordrecht,
The Netherlands, 1995.
20. Saaty, T.L.; Sodenkamp, M. The analytic hierarchy and analytic network measurement processes: The measurement of intangibles.
In Handbook of Multicriteria Analysis; Pardalos, P.M., Zopounidis, C., Eds.; Springer: Berlin/Heidelberg, Germany, 2010; pp.
91–166.
21. Saaty, R.W. The Analytic Hierarchy Process—What It Is and How It Is Used. Math. Model. 1987, 9, 161–176. [CrossRef]
22. Saaty, T.L. The Analytic Hierarchy Process; McGraw-Hill International: New York, NY, USA, 1980.
23. Dyer, J.S. Multiattribute utility and value theories: MAUT—Multiattribute utility theory. In Multiple Criteria Decision Analysis-State
of the Art Surveys; Figueira, J., Greco, S., Ehrgott, M., Eds.; Springer: New York, NY, USA, 2005; pp. 265–298.
24. Fishburn, P.C. The Foundation of Expected Utility; Reidel: Dordrecht, The Netherlands, 1982.
25. Fishburn, P.C. Utility Theory for Decision Making; Wiley: New York, NY, USA, 1970.
26. Keeney, R.L. Value-Focused Thinking: A Path to Creative Decisionmaking, Revised edition; Harvard University Press: Cambridge,
MA, USA, 1996; ISBN 978-0-674-93198-5.
27. Roy, B. From Optimization to multicriteria decision aid: Three main operational attitutes. In Multiple Criteria Decission Making;
Springer: Belin/Heidelberg, Germany, 1976; Volume 130, pp. 1–32.
28. Roy, B. Méthodologie Multicritere d’Aide à La Decision; Economica: Paris, France, 1985.
29. Vanderpooten, D. The construction of prescriptions in outranking methods. In Reading in Multiple Criteria Decision Aid; e Costa, B.,
Ed.; Springer: Berlin/Heidelberg, Germany, 1990; pp. 184–215.
30. Brans, J.-P.; Mareschal, B. Outranking methods: Promethee methods. In Multiple Criteria Decision Analysis-State of the Art Surveys;
Figueira, J., Greco, S., Ehrgott, M., Eds.; Springer: Berlin/Heidelberg, Germany, 2005; pp. 163–198.
Appl. Sci. 2022, 12, 3016 20 of 21

31. Martel, J.-M.; Matarazzo, B. Outranking methods: Other outranking approaches. In Multiple Criteria Decision Analysis-State of the
Art Surveys; Figueira, J., Greco, S., Ehrgott, M., Eds.; Springer: New York, NY, USA, 2005; pp. 197–264.
32. Jacquet-Lagreze, E.; Siskos, J. Assessing a Set of Additive Utility Functions for Multicriteria Decision-Making, the UTA Method.
Eur. J. Oper. Res. 1982, 10, 151–164. [CrossRef]
33. Siskos, Y. Comment Modeliser Les Preferences Au Moyen de Fonctions d’utilite Additives. RAIRO Rech. Oper. 1980, 14, 53–82.
[CrossRef]
34. Siskos, Y.; Grigoroudis, E.; Matsatsinis, N.F. UTA Methods. In Multiple Criteria Decision Analysis-State of the Art Surveys; Figueira,
J., Greco, S., Ehrgott, M., Eds.; Springer: New York, NY, USA, 2005; pp. 297–344.
35. Ehrgott, M.; Wiecek, M.M. Multiobjective Programming. In Multiple Criteria Decision Analysis-State of the Art Surveys; Figueira, J.,
Greco, S., Ehrgott, M., Eds.; Springer: New York, NY, USA, 2005; pp. 667–722.
36. Evans, J.P.; Steuer, R.E. A Revised Simplex Method for Linear Multiple Objective Programs. Math. Program. 1973, 5, 54–72.
[CrossRef]
37. Korhonen, P.; Wallenius, J. A Multiple Objective Linear Programming Decision Support System. Decis. Support Syst. 1990, 6,
243–251. [CrossRef]
38. Zeleny, M. Linear Multiobjective Programming; Springer: New York, NY, USA, 1974.
39. Stewart, T. A Critical Survey on the Status of Multiple Criteria Decision Making Theory and Practice. Omega 1992, 20, 569–586.
[CrossRef]
40. Guarini, M.R.; Battisti, F.; Chiovitti, A. Public Initiatives of Settlement Transformation: A Theoretical-Methodological Approach
to Selecting Tools of Multi-Criteria Decision Analysis. Buildings 2018, 8, 1. [CrossRef]
41. Cinelli, M.; Kadziński, M.; Gonzalez, M.; Słowiński, R. How to Support the Application of Multiple Criteria Decision Analysis?
Let Us Start with a Comprehensive Taxonomy. Omega 2020, 96, 102261. [CrossRef] [PubMed]
42. Cinelli, M.; Kadziński, M.; Miebs, G.; Gonzalez, M.; Słowiński, R. Recommending Multiple Criteria Decision Analysis Methods
with a New Taxonomy-Based Decision Support System. Eur. J. Oper. Res. 2022. [CrossRef]
43. Alamerew, Y.A.; Kambanou, M.L.; Sakao, T.; Brissaud, D. A Multi-Criteria Evaluation Method of Product-Level Circularity
Strategies. Sustainability 2020, 12, 5129. [CrossRef]
44. Bachér, J.; Pihkola, H.; Kujanpää, L.; Mroueh, U.-M. Advancing the Circular Economy through Group Decision-Making and
Stakeholder Involvement. Detritus 2018, 4, 22. [CrossRef]
45. Stankevičienė, J.; Nikanorova, M.; Gentjan, Ç. Analysis of Green Economy Dimension in the Context of Circular Economy: The
Case of Baltic Sea Region. E + M 2020, 23, 4–18. [CrossRef]
46. Pieratti, E.; Paletto, A.; Meo, I.D.; Fagarazzi, C.; Migliorini, M.G.R. Assessing the Forest-Wood Chain at Local Level: A Multi-
Criteria Decision Analysis (MCDA) Based on the Circular Bioeconomy Principles. Ann. For. Res. 2019, 62, 123–138. [CrossRef]
47. Niero, M.; Kalbar, P.P. Coupling Material Circularity Indicators and Life Cycle Based Indicators: A Proposal to Advance the
Assessment of Circular Economy Strategies at the Product Level. Resour. Conserv. Recycl. 2019, 140, 305–312. [CrossRef]
48. Zhao, H.; Zhao, H.; Guo, S. Evaluating the Comprehensive Benefit of Eco-Industrial Parks by Employing Multi-Criteria Decision
Making Approach for Circular Economy. J. Clean. Prod. 2017, 142, 2262–2276. [CrossRef]
49. Parchomenko, A.; Nelen, D.; Gillabel, J.; Rechberger, H. Measuring the Circular Economy—A Multiple Correspondence Analysis
of 63 Metrics. J. Clean. Prod. 2019, 210, 200–216. [CrossRef]
50. Deshpande, P.C.; Skaar, C.; Brattebø, H.; Fet, A.M. Multi-Criteria Decision Analysis (MCDA) Method for Assessing the Sustain-
ability of End-of-Life Alternatives for Waste Plastics: A Case Study of Norway. Sci. Total Environ. 2020, 719, 137353. [CrossRef]
[PubMed]
51. Vlachokostas, C.; Michailidou, A.V.; Achillas, C. Multi-Criteria Decision Analysis towards Promoting Waste-to-Energy Manage-
ment Strategies: A Critical Review. Renew. Sust. Energ. Rev. 2021, 138, 110563. [CrossRef]
52. Keeney, R.L.; Raiffa, H. Decisions with Multiple Objectives: Preferences and Value Tradeoffs; John Wiley & Sons: New York, NY, USA,
1976.
53. Vetschera, R.; Sarabando, P.; Dias, L. Levels of Incomplete Information in Group Decision Models–A Comprehensive Simulation
Study. Comput. Oper. Res. 2014, 51, 160–171. [CrossRef]
54. Kadziński, M.; Ghaderi, M.; Wasikowski,
˛ J.; Agell, N. Expressiveness and Robustness Measures for the Evaluation of an Additive
Value Function in Multiple Criteria Preference Disaggregation Methods: An Experimental Analysis. Comput. Oper. Res. 2017, 87,
146–164. [CrossRef]
55. Siskos, Y.; Spyridakos, A.; Yannacopoulos, D. Using Artificial Intelligence and Visual Techniques into Preference Disaggregation
Analysis: The MUDAS System. Eur. J. Oper. Res. 1999, 113, 281–299. [CrossRef]
56. Siskos, E.; Tsotsolas, N. Elicitation of Criteria Importance Weights through the Simos Method: A Robustness Concern. Eur. J. Oper.
Res. 2015, 246, 543–553. [CrossRef]
57. Simos, J. L’Évaluation Environnementale: Un Processus Cognitif Négocié. Ph.D. Thesis, École Polytechnique Fédérale de
Lausanne, Lausanne, Switzerland, 1990.
58. Figueira, J.; Roy, B. Determining the Weights of Criteria in the ELECTRE Type Methods with a Revised Simos’ Procedure. Eur. J.
Oper. Res. 2002, 139, 317–326. [CrossRef]
59. Simos, J. Evaluer l’Impact Sur l’Environnement: Une Approche Originale Par l’Analyse Multicritère et La Négociation; Presses Polytech-
niques et Universitaires Romandes: Lausanne, Switzerland, 1990.
Appl. Sci. 2022, 12, 3016 21 of 21

60. Bana, E.; Costa, C.A.; De Corte, J.-M.; Vansnick, J.-C. Macbeth. Int. J. Inf. Tech. Dec. Mak. 2012, 11, 359–387. [CrossRef]
61. Tsotsolas, N.; Spyridakos, A.; Siskos, E.; Salmon, I. Criteria Weights Assessment through Prioritizations (WAP) Using Linear
Programming Techniques and Visualizations. Oper. Res. Int. J. 2019, 19, 135–150. [CrossRef]
62. Hurson, C.; Siskos, Y. A Synergy of Multicriteria Techniques to Assess Additive Value Models. Eur. J. Oper. Res. 2014, 238, 540–551.
[CrossRef]
63. Grigoroudis, E.; Siskos, Y. Preference Disaggregation for Measuring and Analysing Customer Satisfaction: The MUSA Method.
Eur. J. Oper. Res. 2002, 143, 148–170. [CrossRef]
64. Grigoroudis, E.; Siskos, Y. Customer Satisfaction Evaluation; Springer: New York, NY, USA, 2010.
65. Spyridakos, A.; Tsotsolas, N.; Siskos, Y.; Yannakopoulos, D.; Vryzidis, I. A Visualization Approach for Robustness Analysis in
Multicriteria Disaggregation–Aggregation Approaches. Oper. Res. Int. J. 2020, 20, 1841–1861. [CrossRef]
66. Mandal, B.K.; Suzuki, K.T. Arsenic Round the World: A Review. Talanta 2002, 58, 201–235. [CrossRef]
67. Golfinopoulos, S.K.; Varnavas, S.P.; Alexakis, D.E. The Status of Arsenic Pollution in the Greek and Cyprus Environment: An
Overview. Water 2021, 13, 224. [CrossRef]
68. United States Environmental Protection Agency (US EPA). Cleanup Laws, Regulations and Guidance; United States Environmental
Protection Agency (US EPA): Washington, DC, USA, 2011.
69. Council of the European Union. Council of the European Union 98/83/EC: Council Directive of 3 November 1998 on the Quality
of Water Intended for Human Consumption. Off. J. Eur. Communities L330 1998, 41, 32–54.