MULTI-CRITERIA ANALYSIS IN DEVELOPMENT OF ACADEMIC STUDY PROGRAMS

FOR EXPERTISE ON HAZARDOUS MATERIALS SAFETY AND PROTECTION

JOVICA BOGDANOV1, ZORAN BAJIĆ1, ZLATE VELIČKOVIĆ1, MIHAEL BUČKO 1
1
University of Defense in Belgrade, Military Academy, jovica.bogdanov@va.mod.gov.rs, zoran.bajic@va.mod.gov.rs,
zlate.velickovic@va.mod.gov.rs, mihael.bucko@va.mod.gov.rs

Abstract: Very significant part of military materiel is classified as hazardous material. Modern regulations
stipulate specialized expertise for personnel that are supposed to work with hazardous materials, which
requires appropriate education and training. According to its traditions in higher education of military officers
and engineers for work with different hazardous materials, University of Defense in Belgrade has developed
and accredited study programs “Chemical Engineering of Materials and Protection” for education of experts
that fulfill the requirements of Ministry of Defense and Army of Serbia. These programs can provide
appropriate expertise for other organizations also, primarily from defense industry and security sectors. The
method of multi-criteria analysis using Analytical Hierarchy Process (AHP), that was developed for analysis
of previous bachelor and master academic study programs in the field of chemical engineering since 1993,
was used during the development of new study programs in order to achieve better quality of education and
expertise of personnel in the future. The results of the analysis are presented in this paper, where new study
programs are best ranked.
Keywords: multi-criteria analysis, academic study program, hazardous materials, chemical engineering.

1. INTRODUCTION
Serbia has long tradition of military institutions of higher education. Since 1850 these institutions have been
educating not only military officers for military units but also engineers for domestic defense industry. Such
combined educational goal was established from its early beginnings, where military was tasked to be the main
driving force of domestic industrialization [1–2]. The best example is the first Statute of Artillery School,
where cadets were supposed to be educated for artillery officers, but with engineering curriculum, which
provided “work in cannon and ammunition production workshops” [1].
Such approach proved to be successful. New educational institutions and curriculums were established,
accompanied by many organizational reforms according to social and political changes. Modern education of
military officers and engineers for defense industry is concentrated in Military Academy in Belgrade and
University of Defense in Belgrade since 2000 and 2012, respectively. Different study programs are developed
and accredited according to domestic legislation. Some programs were developed in cooperation with other
universities, in order to improve quality of education and cooperation between institutions. Current study
programs cover three educational levels (bachelor, master and doctorate) within several different scientific
fields and branches.
All military activities include hazardous materials, either their storage, transport and usage, protection from
them or their safe and environmentally acceptable disposal. Education for work with hazardous materials has
its own specific characteristics. First, there are significantly different hazardous materials thus many different
methods and techniques are used with them. Also, quality of education is a key factor of safety considerations.
In current perspective, academic studies in the field of chemical engineering offer appropriate expertize for
work with hazardous materials. Such higher education has its traditions in Serbian military educational
institutions for more than 50 years. Similar approach can be seen in foreign military universities and academies
as well. For the last accreditation cycle, new bachelor, master and doctorate study programs “Chemical
Engineering of Materials and Protection” were developed by Military Academy in Belgrade and were
successfully accredited in 2021. Primarily, purpose of programs is education of military personnel for three
different groups of hazardous materials: explosive ordnance, chemical-biological-radiological-nuclear
(CBRN) protection and flammable substances (fuels, lubricants etc.).
Multi-criteria decision making can be useful tool in evaluation of a curriculum [3–4]. Development of a
modern study program is stepwise and complex process, since many limitations are imposed by legislature and
accreditation procedures. Other issue is the need for quality of offered knowledge, practical skills and other
expertize for work with hazardous materials. Also, it is worth mentioning that engineering curriculums
constantly need more changes because of their nature [3].
During the development phase, it was possible to do multi-criteria analysis of bachelor and master study
programs “Chemical Engineering of Materials and Protection” and compare the results with previous study
programs with same or similar purpose [5]. Based on the results, the factors can be adjusted in order to achieve
better performance of a study program.

2. METHODS OF ANALYSIS
Development of study programs “Chemical Engineering of Materials and Protection” was based on general
requirements of Army of Serbia, which constituted framework for further development of curriculums. Also,
study programs must fulfill all regulations on higher education, in order to be accredited. Four different study
programs for education of engineers of chemical engineering in the period 1993–2018 were previously
analyzed [5] using Analytic Hierarchy Process (AHP) method [6]:
 Study program “Technical Service”, specialty “Ammunition and Energetic Materials” (Military
Technical Academy, Belgrade, 1993);
 Study program “Technical Service”, specialties “Explosive Ordnance” and “Propulsion and
Protection” (Military Technical Academy, Belgrade, 1997);
 Study programs of bachelor and master academic studies “Military Chemical Engineering”,
specialties “Explosive Ordnance” and “CBRN Protection” (Military Academy, Belgrade, 2009);
 Study programs of bachelor and master academic studies “Military Chemical Engineering”,
specialties “Explosive Ordnance”, “Propulsion and Protection” and “CBRN Protection” (Military
Academy, Belgrade, 2009);
Since study programs from 1993 and 1997 were based on previous legislative, where higher education in
engineering lasted five years, study programs after 2009 are comparable only if considered integrally (bachelor
and master).
Following criteria were chosen for analysis [5]:
 hours of scientific vocational courses (C1);
 hours of applied vocational courses (C2);
 hours of scientific and applied vocational exercises and practice (C3);
 hours of military education (C4);
 hours of vocational and military practice (C5);
 hours of other technical and natural science courses (C6);
 hours of foreign language courses (C7);
 hours of other social science courses (C8);
 elective courses factor (C9);
 hours of elective courses (C10);
 average hours of active lessons in a semester (C11);
 average number of exams in a semester (C12).
Each criterion was assigned a number on a Saaty’s comparison scale [6] and the eigenvalue vector for
weights wi was determined, presented in Table 1.

Table 1: Comparison matrix and relative weights of criterions

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 wi
C1 1 1/3 1/3 1 1/3 1 3 3 1/3 1/3 1/3 1/5 0,0434
C2 3 1 1 1 1 3 5 9 1 5 3 3 0,1439
C3 3 1 1 3 1 3 7 7 1 3 5 5 0,1588
C4 1 1 1/3 1 1/3 1 5 9 1/2 1 1 3 0,0791
C5 3 1 1 3 1 3 7 9 1 1 5 5 0,1517
C6 1 1/3 1/3 1 1/3 1 5 7 1/2 1 3 3 0,0737
C7 1/3 1/5 1/5 1/5 1/7 1/5 1 3 1/3 1/3 1/3 1/3 0,0235
C8 1/3 1/9 1/7 1/9 1/9 1/7 1/3 1 1/5 1/3 1/5 1/5 0,0143
C9 3 1 1 2 1 2 3 5 1 3 5 3 0,1313
C10 3 1/5 1/3 1 1 1 3 3 1/3 1 3 1 0,0692
C11 3 1/3 1/5 1 1/5 1/3 3 5 1/5 1/3 1 1/5 0,0453
C12 5 1/3 1/5 1/3 1/5 1/3 3 5 1/3 1 5 1 0,0657
 Such judgement has following consistency parameters: λmax = 13,6303, consistency index CI = 0,1482 and
consistency ratio CR = 9,62%.
Using this method, the performance of study programs of bachelor and master study programs “Chemical
Engineering of Materials and Protection” was constantly monitored and curriculum structure and hours of
appropriate courses were stepwise modified during the development phase. Criterion values xi for previous
and final version of new study programs are presented in Table 2.

Table 2: Criterion values for considered study programs

Chemical
Military Military Engineering
Technical Technical Chemical Chemical of Materials
Criterion
Service, 1993 Service, 1997 Engineering, Engineering, and
2009 2017 Protection,
2021
C1 705 735 435 555 892
C2 825 885 840 975 915
C3 630 705 555 615 757
C4 1050 915 705 495 555
C5 910 910 652 1034 1034
C6 1410 1335 825 1055 802
C7 120 120 240 360 240
C8 195 90 90 120 45
C9 0 2 2 2,72 3,42
C10 0 990 905 1620 1665
C11 460 432 342 410 398
C12 5,5 6,1 4,8 5,6 5,7

In order to compare study program, ratio of individual criterion values Xi was calculated using following
expression:
𝑥
𝑋𝑖 = (𝑥 ) 𝑖 (1)
𝑖 𝑚𝑎𝑥

Using these results and AHP weights, values of Weighted Linear Combination (WLC) were calculated for
all considered study programs (Table 3):
𝑆𝑗 = ∑(𝑤𝑖 ∙ 𝑋𝑖 ) (2)

Table 3: Analysis results for considered study programs

Military Military Chemical
Technical Technical Chemical Chemical Engineering of
Service, 1993 Service, 1997 Engineering, Engineering, Materials and
2009 2017 Protection, 2021
70,12% 82,72% 67,56% 84,89% 89,28%

After the development, new study programs were approved by Ministry of Defense and Army of Serbia
and during the accreditation process their quality was highly graded by external evaluation committee.

3. CONCLUSION
The previously developed method was used for performance evaluation of new study programs during their
development phase. It proved to be a valuable tool during the development phase, where curriculums and
courses were stepwise modified in order to achieve better performance. The analysis results presented in this
paper show that new study programs of bachelor and master study programs “Chemical Engineering of
Materials and Protection” from 2021 have better performance than all previous study programs with similar
purpose. It can be concluded that new study programs offer high quality of specialized expertise for work with
hazardous materials, which can positively contribute to the safety of individuals and whole social environment.
Quantitative methods are considered to have many limitations and there can be noticed strong skepticism
around their use in the field of higher education [7]. It can be said that presented method can be used for
comparison and performance evaluation of study programs with similar purpose or in the same field or branch
of science.

Acknowledgement
The paper is part of scientific research project “VA-TT/1/22-24” of University of Defense in Belgrade and
funded by Ministry of Defense of Republic of Serbia.

REFERENCES
[1] Maksimović, V. (1925). Artillery School and Military Academy (in Serbian). In Memorial of 75th
Anniversary of Military Academy 1850–1925. Belgrade: Štamparska radionica Ministarstva Vojske i
Mornarice.
[2] Pršić, M. (2000). Military schools in Serbia (in Serbian). In B. Ljušić, S. Bojković , M. Pršić, & B. Jovović,
Officers in Serbian higher education 1804–1918, 201-340. Belgrade: Vojnoizdavački zavod.
[3] Erkan, T. E., & Rouyendegh, B. D. (2013). Curriculum change parameters determined by multi criteria
decision making (MCDM). Procedia – Social and Behavioral Sciences, 116, 1744–1747.
[4] Kawser, M. A. (2014). Curriculum Development in Higher Education. University of Oslo, Oslo:
Reprosentralen.
[5] Bogdanov, J., Bajić, Z., & Bučko, M. (2018). Analysis of study programs for education of chemical
technology engineers in military academies between 1993 and 2018 (in Serbian). In Proceeding of.
XLV Symposium on Operational Research “SYM-OP-IS 2018”, Zlatibor, September 16-19, 325-330.
[6] Saaty, T. (1980). The Analytical Hierarchy Process. Pittsburgh: RWS Publications.
[7] Katharaki, M., & Katharakis, G. (2010). A comparative assessment of Greek universities’ efficiency using
quantitative analysis. International Journal of Educational Research, 49, 115–128.