2017 4th International Conference on New Media Studies
Yogyakarta, Indonesia, November 08-10, 2017
Algorithm Comparison Performance in Assessing
the Quality of University Websites
Tenia Wahyuningrum Nur Rokhman
Department of Informatics Department of Computer Science &
Institut Teknologi Telkom Electronics,
Purwokerto, Indonesia Universitas Gadjah Mada
tenia@ittelkom-pwt.ac.id Yogyakarta, Indonesia
nurrokhman@ugm.ac.id
Abstract— The rapid use of the internet in education field
encourages website developers to make improvements in
university website quality. University websites are also part of the
criteria of a higher education institution performance appraisal.
Thus, a good university is considered to have a good website. The
website reflects the popularity and prestige of a college. There
are at least three university ranking agencies of the world with
different ratings on the quality of the website. Due to several
elaborated and numerous criteria, the quality rating of the
website is often associated with Multi-Criteria Decision-Making
issues. Linear Weightage Model (LWM) and Simple Additive
Weighting (SAW) are the simplest algorithm for data rank. This
paper aims to compare LWM and SAW algorithm performance
to determine the best method on ranking University websites.
This research uses five testing of usability criteria: load time,
page rank, traffic, stickiness, and backlink. The results of LWM
algorithm of the experiment using the language of C++ have
more rapid execution time in comparison to SAW. Based on the
results of Pair sample t test, both algorithms have significant
different effects on ranking results because of the different of
normalization process.
Keywords—Linear Weightage Model, Simple Additive
Weighting, website quality.
I. INTRODUCTION
Quality assurance of higher education institutions adopt
World Class University (WCU) terminologies, which include
academic and teaching excellence, research quality,
knowledge development and dissemination, and activities that
contribute to culture, science, and society [1]. These criteria
encourage universities to make the best efforts at the world
level [2], [3]. However, there are some obstacles in achieving
WCU that are considered less objective, due to the many
different assessment criteria [4]. The quality of university
websites is one of the criteria to measure universities
performance [5]. The criterion in line with the rapid use of the
internet to access educational information. At least, there are
millions of people visiting the university web to access
information every year [6]. Good university generally have
high qualified websites regarding the appearance and the
performance. The high qualified website show the credibility
and popularity of higher education institutions.
978-1-5090-6284-3/17/$31.00 ©2017 IEEE 19
Aina Musdholifah
Department of Computer Science &
Electronics
Universitas Gadjah Mada
Yogyakarta, Indonesia
aina_m@ugm.ac.id
Webometrics, uniRank and eduroute are the most popular
academic rankings for the higher institutions
[7]–[9]. In fact, each ranking method comes with weaknes.
The disadvantage of the webometrics method is the influence
of bad practice in the university web name, having two or
more domains or changing the URL that would reduce the
rating. UniRank has a weakness in its methodology that
emphasizes only on the popularity of the website. Mean while,
eduroute has no clarity in ranking methods, as well as tools
used for data retrieval.
Based on the weakness of ranking mentioned above, it is
necessary to do further assessment about the criterion and the
method of university website quality rating. A research on 71
Bangladeshi college websites reveals the important criteria of
university website quality that involves the number of web
pages, simple web impact factor, self-link web impact factor
and external link web impact factor [10]. Another study on
website quality shows that website quality evaluation criteria
focus on usability [11]. Usability is considered as an
assessment that represents the convenience and satisfaction of
the users, thus forces the website enterprise to build a better
web [12]. Therefore, developers do not only concentrate on
the design of the user interface but also pay attention to the
fulfillment of the users needs with certainty. If the website is
difficult to use or it fails to state clearly what would be
offered, then the user would leave it [13]. Therefore, it is
important to assess usability. In other words, usability is the
determining factor of the whole system's success [14]. In this
case, usability is a relevant factor in business success. Hence,
this paper propose five testing of usability criteria: load time,
page rank, traffic, stickiness and backlink.
The experiments performed by Dominic and Hanim (2013)
conducted by combining Linear Weigtage Model (LWM) and
Fuzzy AHP methods called New Hybrid Model (NHM) on the
quality of university websites in Malaysia based on usability
criteria. In these experiments, NHM is applied using a
combination of LWM and FAHP to produce better and
preferred criterion weight [11]. The excellence of LWM
methods are related to their simplicity in measurement by
searching the summation of weight from the subsection of
performance in each alternative in all criteria. The result of the
highest summation is the best alternative chosen [15]. The
other simple method on weighting and ranking is Simple
Additive Weighting (SAW). This paper aims to compare
LWM and SAW algorithms to determine the best method in
ranking university website. The comparison of both
algorithms is conducted based on the execution time required,
and the difference of generated mean. This research is
conducted to find out whether the two algorithms give the
same or different effect in ranking.
Based on the test result, the most efficient algorithm could
be used as the reference for further researches. The paper is
organized as follows. Section 2 discusses the literature study;
Section 3 describes the research method; Section 4 explains
the result and discussion, and Section 5 presents the
conclusions and suggests some areas for further investigation.
II. LITERATURE STUDY
A. Website Quality
The quality of website can be evaluated using several
factors such as aesthetic, logics, and technology [11], [16]–
[20]. Based on the perspective of web designer and
administrator, the evaluation of website quality focuses on the
usability [11]. Usability covers the simplicity in learning,
effectiveness and pleasure for the users [21]. The factor of
usability includes page load time, page rank, traffic, stickiness
and backlink.
Loading time is the real median time required to load a
webpage on the website on the browser of the users. The
standard of websites performance shows that webpage loading
time must be less than 30 second. The maximum average
server response time is 0.5 second [22]. The page rank
meanwhile is taken from the data of the site ranking based on
the traffic of the visitors in a month. Stickiness is a capability
to ensure that each user has been stuck on a page of website in
certain time period. A pleasurable website would attract the
users to revisit later emerging a positive impact called traffic
impact. Backlink is measured based on the numbers of links
referring to the main page. The backlink represented the
credibility of a website for being correlated to the prestige and
popularity of a website [11], [19].
Global Rank, SimilarWeb Global Rank, Majestic Referring
Domains and Majestic Trust Flow [5], [25]. Alexa is an
intelligent personal assistant developed by Amazon that
provides data with the criteria mentioned above. Alexa
estimates web traffic based on data from millions of internet
users from the different browser.
Eduroute focuses on the study and evaluation of university
websites, not on the university performance. Eduroute’s
volunteers study new ways based on the information provided
by educational institutions, the internet, and the outsiders. The
route review indicator is the volume of information published
on the website, online university scientific publications,
website link quantity, and link quality and website
content[26].
One of the shortcomings of the webometrics method is the
influence of bad practice in the university web domain name,
such as having two or more domains and changing the
university web URL. Webometrics is closely related to the
university's open-source policy, an institutional commitment is
needed. Unirank has a weakness in the methodology that
emphasizes only on the popularity of the website[5]. Eduroute
release the first ranking in 2011, but it did not continue until
now. Table 1 shows the methodology comparison between
webometrics, eduroute, and uniRank University ranking.
TABLE I. METHODOLOGY
Webometrics uniRank™ eduroute
Visibility (50%) Google Page Rank Volume (20%)
Size (20%) Total Number of Links quantity (30%)
Rich Files (15%) inbound links Quality of links and
Scholar (15%) Alexa Traffic Rank content (40%)
Online Scientific
* Not mentioned Information (10%)
percentage of each criterion
C. Linear Weightage Model
LWM is the simplest model that commonly depend on the
assessment of decision maker. The weight of the criteria is
fixed based on the interest of each criterion in comparison to
other criteria [11], [19], [27].
The steps of LWM algorithm are presented as follows :
Step 1 : Determine the criteria and alternative.

B. University web rank Step 2 : Assign the weights of each criterion, W=[W1, W2, W3,
…Wj]
Webometrics, eduroute, and uniRank are university ranks
based on activities on the internet. Webometrics uses four Step 3 : Determine a threshold for each criterion.
assessment criteria including Visibility (50%), Presence (5%), Step 4 : Transform the measures to make them all positively
Openness (10%), Excellence (35%). The visibility obtained related to scores and normalized into a 0–1 scale.
from the number of external backlinks to the web pages
represents the credibility, quality, and popularity of university ⎧ , ℎ ℎ ℎ
websites. The Webometrics rank used the analytic tools and ⎪
= (1)
scholar database such as google, ahrefs, majesticseo, google ⎨
scholar, and scimago to reach the data [7], [8], [23], [24]. ⎪ , ℎ ℎ ℎ
⎩

UniRank assesses the provided websites by considering the where

popularity of the website with Moz Domain authority, Alexa Xij = a specific website considered at the time.

20
max = the maximum value of particular criteria among
all websites.
min = the minimum value of the same attribute among
the whole websites.
Step 5 : Calculate the total score.
=∑
Step 6 : Sort the total scores in descending order
Step 7 : Identify the rank of university websites
D. Simple Additive Weigthing (SAW)
SAW is known as the method of weight summation. The
basic concept of SAW is to search the summation of weight
from the rate of the performance in each alternative in all
criteria [28]–[31].
Step 1 : Determine the criteria and alternative, and make a
decision for matrix (X)
Step 2 : Assign the weights of each criteria. W=[W1, W2, W3,
…Wj]
Step 3 : Determine the benefit or cost criteria for each
criterion.
Step 4 : Normalize the decision matrix by calculating values
performance rating of alternative on criteria.
⎧ , ℎ
= (3)
⎨
, ℎ
⎩ A. Algorithm comparison of LWM and SAW
Step 5: Calculate the total score by the summation of the
multiplication element row normalized and weights the
preferences corresponding element column.
=∑ (4)
Step 6 : Sort the total scores in descending order
Step 7 : Identify the rank of university websites
III. RESULT AND DISCUSSION
The first step of this research is to determine the objectives
and the scope of the study. The objectives and the scope of the
study are to compare LWM and SAW algorithms in assessing
the quality of university websites. The next step is to define
the indicator of the variables. The criteria of the website
quality include page load time, page rank, traffic, stickiness
and backlink.
The third step is to collect data. This study is an
experimental study involving 50 University websites. The data
of the website quality were taken from 20 to 30 March 2017
using the Alexa web diagnostic tools. Alexa calculates the
median of all page load time on a site to compare with the
numbers of other sites. An individual page load time is defined
as how long it takes for the page structure to load. The time
does not include the time to load all the images and
stylesheets. Alexa global and country traffic ranks show the
popularity or certain site compare to the others. The data
traffic can be seen from the numbers of daily unique
pageviews per visitor on the site. Stickiness is estimated daily
time on site (second) per visitor to the site. The backlink
count shows the numbers of sites Alexa find the link to this
site. The data taken in this research are performed using the
computer with the processor specification of AMD E2-6110
(2)
APU with AMD Radeon R2 Graphics 1,5GHz, and RAM
2048 MB.
After collecting the data, the next step is to normalize the
data using formula (1) and (3). The weighting criteria are
performed by experts and web developers. The sixth step is to
rank the data by descending. The comparisson of two method
analyzed by paired sample t test using SPSS (Statistical
Package for the Social Sciences) software. General overview
of research can be seen in Figure 1.
Fig 1. Research Design
The main difference between the two algorithms is shown
in the process of data normalization. In the LWM
normalization process, the denominator of its division is a
reduction of the maximum and the minimum values. In SAW
algorithm, we use the as the denominator. If the
criteria are benefit, and if the criteria are cost. The weakness
of SAW algorithm is related to the existence of alternative
value 0 that might lead to the error in the measurement caused
by the division by zero. This deficiency also occurs in the
LWM algorithm if the maximum and the minimum values are
equal, resulting in a value of 0.
Insertion sort is used to rank the calculation result. It has
simple steps and good performance for the smallest array [26],
[27]. The following is the insertion sort algorithm :
1 for j= 2 to m
2 key = A[j]
3 i= j-1
4 while i>0 and A[i]>key
5 A[i+1]=A[i]
6 i=i-1
7 A[i+1]=key
The time complexity of insertion sort operation on seventh
step was:
21
t6 = m+(m-1)+(m-1)+(m-1)+(m*(m-1))+(m*(m-1))+(m-1) UI 1.50 11618 3.96 346 6830
= m+4(m-1)+2(m*(m-1)) UB 1.53 12718 3.07 272 4415
IPB 1.78 14205 4.34 268 4954
= m+4m-4+2(m2-m) UNPAD 1.44 30791 3.67 266 3770
= m+4m-4+2m2-2m UNDIP 1.44 15216 2.91 220 4520
= 2m2+3m-4 UNRIAU 1.11 79537 2.46 368 633
UNUD 1.41 31023 2.99 289 1829
For n criteria, m alternatives, and ti time complexity each step, UNAIR 2.21 22208 2.48 248 3679
UNS 2.08 15906 2.55 219 4997
following is the analysis of LWM and SAW algorithm below=
UNSYA 1.65 48743 4.07 340 1105
t = t1+t2+t3+t4+t5+t6+t7 UNHAS 2.60 29096 3.65 306 1558
= (3+nm+n)+1+6+(4n+10nm)+(m+2nm) + (2m2+3m-4) UNILA 1.51 26271 2.36 271 1993
+2m ITS 1.18 26545 3.88 352 2813
= 2m2+13nm+5n+6m+6 UNSOED 1.84 58489 2.91 286 971
If n = m, then UAD 1.82 48683 5.81 496 1635
UNTAD 1.68 86829 3.03 507 1885
= 15n2+11n+6 PETRA 0.56 62542 2.77 253 1389
= an2+bn+c BINUS 1.74 33286 3.43 282 1202
From the calculation above, the time complexity of both UPI 1.51 16441 2.28 175 4209
algorithm is O(n2). UMY 2.61 45586 3.88 274 3011
UDINUS 1.87 54757 2.37 230 705
UBAYA 1.94 103686 3.23 319 496
To measure the execution time, experiments using C++
ATMA 1.10 142302 3.20 392 660
language are conducted to evaluate the algorithm UNNES 1.61 18707 2.52 254 2890
performance. The experiment is conducted 10 times with the NARO 2.60 64124 1.34 103 2645
alternative website number of 5, 10, 15, 20, 25, 30, 35, 40, 45 UNY 2.13 10077 2.68 243 3078
and 50 units. The result measurement is presented in Table 1. GUNA 0.77 37640 3.67 236 23163
UINJKT 1.17 36081 2.59 219 1015
TABLE II. EXECUTION TIME COMPARISON OF LWM AND SAW UNESA 2.77 52872 2.43 163 2222
TEL U 2.88 34585 3.82 401 962
Execution Time UM 2.78 24924 2.72 275 2727
∑ University Websites
LWM (second) SAW (second) UNAND 1.56 36302 3.18 252 1708
5 0.45 0.51 UNSRI 1.39 53563 3.70 319 1930
10 0.50 0.84 UINSBY 1.07 46107 2.62 210 1164
15 0.74 1.04 UINMLG 1.70 48433 3.16 296 1977
20 0.91 1.26 UMM 1.38 44910 3.38 454 2498
25 1.06 1.45 UMS 1.83 20946 2.26 179 1509
30 1.13 1.65 UNPAR 2.27 128150 3.55 607 546
35 1.26 1.81 UNTAN 1.02 79004 3.26 310 660
40 1.36 1.99 UNEJ 2.13 26249 3.43 413 1381
45 1.48 2.19 UNSRAT 2.56 49856 1.54 299 757
50 1.68 2.38 UKSW 2.27 79441 2.30 177 465
χ (mean) 1.057 1.512 UT 2.03 19155 9.91 872 1397
WIDYA 1.55 57954 2.93 206 412
Table 2 shows that the time required to execute the SAW UNJA 3.04 998570 6.07 729 488
algorithm is larger in comparison to the one in LWM UINP 1.48 180007 4.50 352 1069
algorithm at approximately 4.5 seconds. Thus, it can be USU 0.76 11530 2.81 159 9237
UNISU 3.52 123642 4.15 297 505
conducted that the LWM algorithm works faster than SAW.
Table 3 shows the multiplication of values by weights
B. Website quality rank comparison of LWM and SAW generating the total scores and rank of the LWM and SAW
Measurement units for the criterion of load time and algorithm for 50 alternative websites in universities. The
stickiness are second. Meanwhile the criterion of page rank, weight of the five criteria defined are set as follows : load time
traffic and backlink was number. This research use 50 official (0.3); page rank (0.25); traffic (0.2); stickiness (0.15);
websites of universities in Indonesia. Data that have been backlink (0.1).
collected and normalized by the formula (1) and (3) are
displayed in Table 3. The highest rank of LWM is found in UT followed by
Gunadarma, USU, Petra and ITS. The results are different
from the measurement using SAW algorithm. The results
show that there are five universities with the highest web
TABLE III. ORIGINAL DATA quality; UMM, UT, Gunadarma, USU, and Petra. The
difference is caused by the difference in measurement in data
Load Page Back
Criteria Traffic Stickiness normalization process. LWM calculates the division with the
time rank link
UGM 1.74 8837 3.25 256 8653 difference of maximum and minimum value. Meanwhile,
ITB 1.45 23552 3.81 268 4758 SAW calculates the division directly using the maximum

22
value and the value of each criterion without measuring the Here are the hypotheses:
difference. Table 4 shows that UMM is in the first rank in H0: Data is not normally distributed
SAW. However, LWM method doesn’t even put UMM in the H1: Data is normally distributed
top 5 ranks. The LWM normalization process divides the
value by the difference between the maximum and minimum TABLE V. ONE SAMPLE KOLMOGOROV SMIRNOV TEST
values, resulting in different outputs with the SAW method.
LWM SAW
TABLE IV. TOTAL SCORES N 50 50
Normal Parametersa,,b Mean 0.507 0.283
University Websites SAW LWM
UGM 0.37 0.54 Std. Deviation 0.088 0.080
ITB 0.31 0.56 Most Extreme Differences Absolute 0.102 0.164
UI 0.38 0.59 Positive 0.082 0.164
UB 0.33 0.54
Negative -0.102 -0.094
IPB 0.33 0.55
UNPAD 0.29 0.56 Asymp. Sig. (2-tailed) 0.671 0.134
UNDIP 0.31 0.54 a. Test distribution is Normal.
UNRIAU 0.28 0.55 b. Calculated from data.
UNUD 0.27 0.54
UNAIR 0.24 0.45
UNS 0.26 0.46 Rejection and acceptance of hypothesis are seen from
UNSYA 0.27 0.54 Asymp. Sig. (2-tailed) value. If the Sig. value of the
UNHAS 0.24 0.44 Kolmogorov-Smirnov Test is greater than 0.05, the data are
UNILA 0.26 0.51 normal. If it is below 0.05, the data significantly deviate from
ITS 0.34 0.60(5th)
UNSOED 0.22 0.48
a normal distribution. Based on Table 5, the Sig. Value of
UAD 0.32 0.60 LWM method is 0.671 and SAW method is 0.134, thus the
UNTAD 0.27 0.54 alternative hypothesis is accepted. The differences mean
PETRA 0.42(5th) 0.60(4th) between LWM and SAW method can be analyzed by paired
BINUS 0.25 0.51 sample t test. The proposed hypothesis is:
UPI 0.27 0.50
UMY 0.23 0.44
UDINUS 0.20 0.46 H0: There is no significant difference between LWM and
UBAYA 0.22 0.47 SAW in assessing University web ranking.
ATMA 0.30 0.56 H1: There is significant difference between LWM and SAW
UNNES 0.27 0.51 in assessing University web ranking.
NARO 0.14 0.34
UNY 0.30 0.46 TABLE VI. PAIRED SAMPLE STATISTICS
GUNA 0.46 (3rd) 0.70(2nd)
UINJKT 0.27 0.53 Mean N Std. Deviation Std. Error Mean
UNESA 0.17 0.36
Pair 1 LWM 0.507 50 0.088 0.012
TEL U 0.24 0.43
UM 0.22 0.40 SAW 0.283 50 0.080 0.011
UNAND 0.25 0.52
UNSRI 0.28 0.56 Based on 50 samples, Table 6 indicates that the mean of
UINSBY 0.28 0.54
LWM is greater than SAW. The standard deviation of LWM is
UINMLG 0.25 0.51
UMM 0.52 (1st) 0.59 0.088 and SAW is 0.080.
UMS 0.23 0.46
UNPAR 0.26 0.50 TABLE VII. PAIRED SAMPLE T TEST
UNTAN 0.30 0.57 Mean 0.224
UNEJ 0.27 0.50 Std. Deviation 0.056
UNSRAT 0.17 0.38 Paired
95% Lower 0.208
UKSW 0.17 0.39 Differences
Confidence
UT 0.49 (2nd) 0.75 (1st) Upper 0.240
Interval
WIDYA 0.22 0.49 t 28.174
UNJA 0.31 0.28 df 49
UINP 0.28 0.54 Sig. (2-tailed) 0.000
USU 0.44 (4th) 0.61(3rd)
UNISU 0.19 0.32 Table 7 shows that the value of t is 28.174, and t table with
5% significance and degree of freedom/df = 50-1 = 49 is
Before testing the difference between the two methods of 2.021. Because the t table < t value, there is a significant
LWM and SAW, it is necessary to check the normality using difference between the LWM and SAW method. The rules of
Kolmogorov Smirnov test. rejecting and accepting the hypothesis can be seen from the
Sig. value. If the Sig. value (2-tailed) is less than 0.05 then H0
is rejected and if the Sig. value (2-tailed) greater than 0.05

23
then H0 is accepted. Based on Table 7, the Sig. value (2-tailed)
is 0.000, it can be concluded that H0 is rejected. It means that
LWM and SAW have different effects in ranking results.
IV. CONCLUSION
The LWM and SAW algorithms are simple method to rank
the multi-criteria problem of decision-making.
This study compares the performance of two algorithms in
assessing the quality of university websites. Based on the
result, the execution time in the experiment for both
algorithms show that LWM is more rapid around 4.5 seconds.
Both algorithms applied to each group show significant
difference in university rank performance. The LWM and
SAW have different effects in ranking results due to the
differences of normalization process. LWM divides the value
by the difference between the maximum and the minimum
values. However, the normalization process of SAW method
is performed directly by dividing the minimum or maximum
value. Subsequent research can add some additional criteria
such as user satisfaction. Further investigation can be
conducted by adding the number of test alternatives of other
ranking methods.
REFERENCES
[1] H. M. Levin, ‘What is a World Class University? . Introduction II .
Views on World Class Universities’, in Conference of the
Comparative and International Education Society, 2006, pp. 1–49.
[2] S. R. Mohsen Nazarzadeh Zare, Javad Pourkarimi, Gholamreza
Zaker Salehi, ‘In Search of a World-Class University in Iran’, J.
Appl. Res. High. Educ., vol. 8, no. 4, pp. 522–539, 2016.
[3] C. Prudence and C. Chan, ‘Trends in Publication in the Race for
World- Class University : The Case of Taiwan’, High. Educ. Policy,
vol. 29, no. 4, pp. 431–449, 2016.
[4] P. G. Altbach and J. Salmi, The Road to Academic Excellence-The
Making of World-Class Research universities. Washington DC: The
International Bank for Reconstruction and Development, 2011.
[5] H. Jati, ‘Comparison of University Webometrics Ranking Using
Multicriteria Decision Analysis : TOPSIS and VIKOR Method’,
World Acad. Sci. Eng. Technol., vol. 71, pp. 1663–1669, 2012.
[6] M. J. Kargar, ‘University Website Ranking from Usability Criteria
Perspective; A Case Study in IRAN’, Int. J. Adv. Comput. Technol.,
vol. 3, no. 11, pp. 246–251, 2011.
[7] I. F. Aguillo, J. L. Ortega, M. Fernández, and A. M. Utrilla,
‘Indicators for webometrics ranking of open access repositories’,
Scienciometrics, vol. 82, no. 3, p. 477–486 ST–Indicators for
webometrics ranking o, 2010.
[8] S. Nissom and N. Kulathuramaiyer, ‘The study of Webometrics
Ranking of World Universities’, 2012.
[9] M. R. M. El-Hefnawy, A. H. El-bastawissy, and M. A. Kadry,
‘Benchmarking the Higher Education Institutions in Egypt using
Composite Index Model’, Int. J. Adv. Comput. Sci. Appl., no.
Special Issue, pp. 92–103, 2014.
[10] M. A. Islam, ‘Webometrics study of Universities in Bangladesh’,
Ann. Libr. Inf. Stud., vol. 58, no. December, pp. 307–318, 2011.
[11] P. D. D. Dominic and S. Hanim, ‘University website quality
comparison by using non-parametric statistical test : a case study
from Malaysia’, Int. J. Oper. Res., vol. 16, no. 3, pp. 349–374,
2013.
[12] S. Lee and R. J. Koubek, ‘The effects of usability and web design
attributes on user preference for e-commerce web sites’, Comput.
Ind., vol. 61, no. 4, pp. 329–341, 2010.
[13] J. Nielsen, ‘Usability 101: Introduction to Usability’, Nielsen
Norman Group, 2012. [Online]. Available:
https://www.nngroup.com/articles/usability-101-introduction-to-
usability/.
[14] H. Roder, ‘Specifying usability features with patterns and
24
templates’, 2012 1st Int. Work. Usability Access. Focus. Requir.
Eng. UsARE 2012 - Proc., pp. 6–11, 2012.
[15] R. E. Setyani and R. Saputra, ‘Flood-prone Areas Mapping at
Semarang City by Using Simple Additive Weighting Method’,
Procedia - Soc. Behav. Sci., vol. 227, no. November 2015, pp. 378–
386, 2016.
[16] D. D. J. Suwawi, E. Darwiyanto, and M. Rochmani, ‘Evaluation of
academic website using ISO/IEC 9126’, Information and
Communication Technology (ICoICT ), 2015 3rd International
Conference on. pp. 222–227, 2015.
[17] A. R. Nilawati, D. A. R., A. Y. Pratama, D. Adlina, and N. R. Al
Mukarrohmah, ‘Interface on usability testing Indonesia official
tourism website’, Int. J. Hum. Comput. Interact., vol. 3, no. 2, pp.
26–34, 2012.
[18] M. Rinder, S. Writer, and Friserv, ‘The Importance of Website
Usability Testing’, Appl. Inf. Manag., vol. 1277, no. 800, pp. 1–141,
2012.
[19] P. D. D. Dominic, ‘A comparison of Asian airlines websites
quality : using a non-parametric test Handaru Jati’, Int. J. Bus.
Innov. Res., vol. 5, no. 5, 2011.
[20] E. Iryanti and R. Pandiya, ‘Evaluating The Quality of e-Learning
Using Consistent Fuzzy Preference Relations Method’, in
International Conference on System Engineering and Technology
(ICSET), 2016, pp. 61–66.
[21] H. Sharp, Y. Rogers, and J. Preece, Interaction design: beyond
human-computer interaction, First Edit., vol. 11. USA: John Wiley
& Sons, Inc, 2002.
[22] T. Albalushi, S. Ali, R. Ashrafi, and S. Albalushi, ‘Accessibility and
Performance Evaluation of E-Services in Oman Using Web
Diagnostic Tools’, Int. J. u- e- Serv. Sci. Technol., vol. 9, no. 7, pp.
9–24, 2016.
[23] L. Björneborn and P. Ingwersen, ‘Perspective of webometrics’,
Scientometrics, vol. 50, no. 1, pp. 65–82, 2001.
[24] T. Wahyuningrum and R. Pandiya, ‘Logaritmic Fuzzy Preference
Programming Approach for Evaluating University Ranking
Optimization’, Sci. J. Informatics, vol. 4, no. 1, pp. 1–7, 2017.
[25] ‘uniRank Ranking Methodology’, 2005. [Online]. Available:
http://www.4icu.org/about/index.htm#ranking. [Accessed: 18-May-
2017].
[26] Eduroute, ‘Eduroute top ranking system’, 2008. [Online]. Available:
http://www.eduroute.info/University-Ranking-System.aspx.
[Accessed: 18-May-2017].
[27] A. A. Ali and P. D. D. Dominic, ‘A Case Study of Linear
Weightage Model for Supplier Selection Process’, 2008 Int. Symp.
Inf. Technol., vol. 3, pp. 23–26, 2008.
[28] J. Kittur, ‘Optimal generation evaluation using SAW, WP, AHP and
PROMETHEE multi - Criteria decision making techniques’, Proc.
IEEE Int. Conf. Technol. Adv. Power Energy, TAP Energy 2015, pp.
304–309, 2015.
[29] A. Pranolo and S. M. Widyastuti, ‘Simple additive weighting
method on intelligent agent for urban forest health monitoring’,
Proceeding - 2014 Int. Conf. Comput. Control. Informatics Its Appl.
‘New Challenges Oppor. Big Data’, IC3INA 2014, pp. 132–135,
2014.
[30] S. M. Liu, S. Pan, Z. K. Mi, Q. M. Meng, and M. H. Xu, ‘A simple
additive weighting vertical handoff algorithm based on SINR and
AHP for heterogeneous wireless networks’, 2010 Int. Conf. Intell.
Comput. Technol. Autom. ICICTA 2010, vol. 1, pp. 347–350, 2010.
[31] Y. J. Wang, ‘A fuzzy multi-criteria decision-making model based on
simple additive weighting method and relative preference relation’,
Appl. Soft Comput. J., vol. 30, pp. 412–420, 2015.
[32] R. Dwivedi and D. C. Jain, ‘A Comparative Study on Different
Types of Sorting Algorithms ( On the Basis of C and Java )’, Int. J.
Comput. Sci. Eng. Technol., vol. 5, no. 8, pp. 805–808, 2014.
[33] I. M. Alturani, A. Mahmoud, and I. Alturani, ‘Review on Sorting
Algorithms A Comparative Study’, Int. J. Comput. Sci. Secur., vol.
7, no. 3, pp. 120–126, 2013.