Fuzzy-Delphi Method

The Fuzzy-Delphi Method (FDM) is a widely used and accepted method for achieving convergence of
opinion concerning real-world knowledge solicited from experts within certain topic areas (Hsu, 2010).
This method was proposed by Ishikawa et al. (1993), and it was derived from the traditional Delphi
technique and fuzzy set theory (Hsu, 2010). It is a means and method for consensus-building by using a
questionnaire to collect data from a panel of selected subjects. This method has been applied in various
fields such as program planning, needs assessment, policy determination, and resource utilization
(Manakandan, 2017). Aziz (2015) used this method in examining the factors that influence the safety
culture in companies of the oil and gas industry in Malaysia.

The FDM uses fuzzy set numbers or fuzzy set theory whereby each set will have a value from 0 to 1.
Compared to the Delphi Method, this method reduces cost and time during evaluating each item in a
questionnaire. It also reduces the survey rounds and increases items recovery rate, allows the experts to
express their opinions without any ambiguity biases, which enhances the completeness and consistency
of opinion (Mohd et. al, 2014), and to get the consensus from the experts without jeopardizing their
original opinion and by giving their real reaction towards the questions (Noh et. al, 2013).

Data Analysis

The analysis of the data that will be used in this study is replicated from Mohd et. al (2014) published
material which discusses two important concepts of FDM, particularly Triangular Fuzzy Numbers and
Defuzzification process (refer Figure 1).

Triangular Fuzzy Numbers

Triangular Fuzzy Numbers (TFN) provides an opportunity for each recorded response made by an expert
in the form of Likert scale scoring to be translated into fuzzy scoring (Refer Table I). Each recorded
response had three values to consider, namely the average minimum value (n1), most reasonable value
(n2), and the maximum value (n3). The rationale of TFN was to show the fuzziness or inexactness in the
opinion made by an expert. Every opinion had a certain amount of ambiguity which can't be addressed
by using a Likert scale because it is a fixed score. Let us say an item “Strictly implementing that the
contractors obtain work permits before the start of any operation” was scored 5 (highly agree) by an
expert. The score is converted into minimum, most reasonable, and the maximum value of 0.6, 0.8 and
1.0 fuzzy scores, respectively. It indicated the expert agreeable to the item is 60%, 80%, and 100%,
respectively. The fuzzy scores were averaged as indicated by m1, m2 and m3 values for further
Defuzzification process.
Defuzzification process

Defuzzification process (Amax) is a ranking process of each item to identify the importance level of each
item. This ranking process was very helpful to determine whether to keep or discard certain items based
on the following formula:

Amax = 1/3 * (m1 + m2 + m3)

Determination of item acceptability

There were three prerequisites to be fulfilled to determine the acceptability of the constructs and its
respective items. The prerequisites were (1) threshold value, d-construct (Cheng, 2002), (2) experts
agreement on evaluated items ≥75% (Chu, 2008) and (3) ranking of the item. The threshold value, d-
construct indicates the selection of certain construct based on the consensus of the experts for each
construct. However, prior to that, a threshold value (d) for each item will be find, by calculating the
difference between average fuzzy number and each expert fuzzy number using the formula below:

Once the value was obtained, a threshold value (d-construct) was calculated by using the formula below:

Based on the value, the acceptability of the construct was determined, whereby a construct was
accepted if the Threshold value (d-construct) ≤ 0.2. Expert agreement on each evaluated item was also
based on threshold value (d) for each item, whereby (d) ≤ 0.2 are accepted. The frequency of accepted
values was presented as percentage as shown in Figure 3. Items with expert agreement of less than 75%
were discarded. The rank of an item within a similar construct was determined after Defuzzification
process as mentioned earlier (refer Figure 1). All respondents data were entered and analysed using
Microsoft excel version 2013.
Subject Selection

Choosing the appropriate subjects for a Fuzzy-Delphi study is the most important step in the entire
process because it directly relates to the quality of the results generated (Judd, 1972; Taylor & Judd,
1989; Jacobs, 1996). Since the Delphi technique focuses on eliciting expert opinions over a short period
of time, the selection of Delphi subjects is generally dependent upon the disciplinary areas of expertise
required by the specific issue (Hsu, 2007).

Regarding any set standards of selecting Delphi subjects, there is, in fact, no exact criterion currently
listed in the literature concerning the selection of Delphi participants. That is, “throughout the Delphi
literature, the definition of [Delphi subjects] has remained ambiguous” (Kaplan, 1971, p. 24). Regarding
the criteria used to guide the selection of Delphi subjects, individuals are considered eligible to be
invited to participate in a Delphi study if they have somewhat related backgrounds and experiences
concerning the target issue, are capable of contributing helpful inputs, and are willing to revise their
initial or previous judgments for the purpose of reaching or attaining consensus (Pill, 1971; Oh, 1974).
Helmer and Rescher (1959), Klee (1972), and Oh (1974) concur that choosing individuals who are simply
knowledgeable concerning the target issue is not sufficient nor recommended. Considering the
necessity of selecting the most qualified individuals, Delbecq, Van de Ven, and Gustafson (1975)
specifically state that three groups of people are well qualified to be subjects of a Delphi study.

The authors recommend:

“(1) the top management decision makers who will utilize the outcomes of the Delphi study;

(2) the professional staff members together with their support team; and

(3) the respondents to the Delphi questionnaire whose judgments are being sought” (p. 85).

Delphi subjects should be highly trained and competent within the specialized area of knowledge
related to the target issue. Investigators need to closely examine and seriously consider the
qualifications of Delphi subjects. Oh (1974) indicates that choosing appropriate subjects is generally
based on the judgment and discretion of the principal investigators. Jones and Twiss (1978) state that
the principal investigators of a Delphi study should identify and select the most appropriate individuals
through a nomination process. Ludwig (1994) also states that, “solicitation of nominations of well-
known and respected individuals from the members within the target groups of experts was
recommended” (p. 52). Generally, the pool of selecting possible Delphi subjects is likely to use positional
leaders (Kaplan, 1971; Ludwig, 1994), to follow a review of authors of publications in the literature
(Meyer, 1992; Miller, 2001), and/or to make contacts with those who have firsthand relationships with a
particular issue (Jones, 1975; Anderson & Schneider, 1993). The latter basically consists of individuals
who are primary stakeholders with various interests related to the target issue or research effort.

Concerning the appropriate number of subjects to involve in a Delphi study, Delbecq, Van de Ven, and
Gustafson (1975) recommend that researchers should use the minimally sufficient number of subjects
and should seek to verify the results through follow-up explorations. Ludwig (1994) notes that the
number of experts used in a Delphi study is "generally determined by the number required to constitute
a representative pooling of judgments and the information processing capability of the research team”
(p. 52). However, what constitutes an optimal number of subjects in a Delphi study never reaches a
consensus in the literature. Delbecq, Van de Ven, and Gustafson (1975) suggest that ten to fifteen
subjects could be sufficient if the background of the Delphi subjects is homogeneous. In contrast, if
various reference groups are involved in a Delphi study, more subjects are anticipated to be needed.
Witkin and Altschuld (1995) note that the approximate size of a Delphi panel is generally under 50,

but more have been employed. Ludwig (1997) documents that, “the majority of Delphi studies have
used between 15 and 20 respondents” (p. 2). In sum, the size of Delphi subjects is variable (Delbecq, Van
de Ven, & Gustafson, 1975). If the sample size of a Delphi study is too small, these subjects may not be
considered as having provided a representative pooling of judgments regarding the target issue. If the
sample size is too large, the drawbacks inherent within the Delphi technique such as potentially low
response rates and the obligation of large blocks of time by the respondents and the researcher(s) can
be the result.
Other source

Panel of experts

A panel of experts is defined as a group of persons who are

skilful in the scope of a study area. They are selected based

on leading position in public health care system with a

significant practical knowledge in their field of practice.16

They also should represent his/her circle of professional

Occupational Health group as suggested by the previous

scholar.17 In this study, the inclusion criteria for the experts

were occupational health related specialization, familiarity

with the working zone, authority in the field, and the number

of years of experience. Each chosen experts was at least one

of the following; 1 a public health physician that has

published an article related pesticide applicators, 2 an

administrator who manages the pesticide applicators at

district/state/national level, 3 had previously worked or

experienced in pesticide application related job,4 minimum

five years of experience in the related field of noise and

pesticide exposure,5 an academician or tutor in the

occupational health related field. A total of sixteen experts

were recruited as the panel of experts via non-probable,

purposive sampling method. The number was considered

optimum and complied with previous suggestions which

required 10 to 50 experts. Lesser amount of experts is

required, i.e. 10 to 15, if they are homogenous experts.