Proc. of 20l5 2nd Int.

Conference on Information Technology, Computer and Electrical Engineering (ICITACEE), Indonesia, Oct l6-l8th

Portability Characteristic Evaluation Academic

Information System Assessment Module using AIS
Quality Instrument

Umi Laili Yuhana Istiningdyah Saptarini

Department of Informatics Department of Informatics
lnstitut Teknologi Sepuluh Nopember (ITS) lnstitut Teknologi Sepuluh Nopember (ITS)
Surabaya, Indonesia Surabaya, Indonesia
yuhana@if.its. ac.id
Siti Rochimah
Department of Informatics
Institut Teknologi Sepuluh Nopember (ITS)
Surabaya, Indonesia
siti@its-sby.edu

Abstract- The role of academic information system in an These quality model are used to assess the quality of
educational institution is quite important, especially with software in general. Yuhana et.al. proposed Academic
regard to the management and reporting of student's academic Information System Quality Instrument (AISQI) to assess
grade. To maintain the quality, Academic information system
the quality of academic information system. There is three
assessment module needs to be evaluated. Evaluation can be
perspective with 12 quality factors in AISQI. AISQI
done by using existing quality models. It can be used to test the
ability of a software system to be transferred from one combine quality factors in the ISO/IEC 9126 and completed
environment to another environment that has been by ISO/IEC 9126 25010: 2011, WBAQM, and COBIT 4. 1.
determined. Portability factor from Developer perspective is adapted
This paper discuss an evaluation of the characteristics of from ISO 9126-3[1].
portability on Assessment Module of ITS's Academic
Portability factor in AISQI is a factor that tests the ability
Information System (SIAKAD ITS) using Academic
of the software to be transfered from one environment to
Information System Quality Instrument (AISQI). The
evaluation result shows that AISQI can be used to evaluate another environment. This characteristic tests a software not
academic information system assessment module from only from the software system, but also hardware used, the
developer view. Sub quality factor include adaptability, ease of user organization, as well as rules relating to the system's
installation, and available co-existence metric. These three portability. Portability characteristic chosen because it has a
metrics yield maximum value, namely 1.0. It can be concluded
metric to measure the portability aspect of the SIAKAD ITS
that the portability aspect of the SIAKAD ITS Assessment
Assessment Module both from the hardware resource used
Module is good.
and the user organization. SIAKAD ITS Assessment
Keywords-software evaluation, quality model, academic Module operates side by side with other modules on
information system quality instrument. SIAKAD. It is therefore important to measure the quality of
the hardware resources used when operating side by side
with other systems. In addition, SIAKAD Assessment
I. INTRODUCTION
Module can facilitate the academic process in ITS only if
n general, an academic information system is a system
I which manages information relating to the academic
they can be used by the entire academic community of ITS.
Then measuring the quality from the organizational side also
activities of an educational institution. Academic needs to be done.
information system aims to help users managing academic This study aims to show how the internal metrics of
data, such as study plans, grades, and administration. Institut AISQI, particularly portability characteristic can be used to
Teknologi Sepuluh Nopember (ITS) Surabaya as one of the measure the quality of academic information systems with
universities in Indonesia, has taken advantage of an case studies of SIAKAD ITS Assessment Module. Section
Academic Information System (SIAKAD) to help manage II describes the previous research and the quality model
academic data. Among the features that exist in SIAKAD, used. Section III describes the evaluation process. Section
features that play an important role are the features IV describes the results of the evaluation of SIAKAD ITS
associated with the assessment. Reporting and academic Assessment Module. While the conclusions are described in
assessment are one of the major business processes in a Section V.
University. Therefore, SIAKAD ITS Assessment Module's
quality needs to be guaranteed, so that users can utilize the
system to the fullest.
Several quality models that can be used as a software
quality measuring tool are McCall quality model, Boehm
quality model, Dromey quality model, and ISO/IEC 9126.

978-1-4799-9863-0/15/$31.00 ©2015 IEEE 133

 Proc. of 2015 2nd Int. Conference on Information Technology, Computer and Electrical Engineering (ICITACEE), Indonesia, Oct 16-18th

TABLE I. WEIGHT OF SUB-CHARACTERISTICS ON PORTABILITY

TABLE 2. CALCULATION PARAMETERS OF ADAPTABILITY SUB­
CHARACTERISTICS [7]
CHARACTERISTIC'S METRICS [7]
Sub-characteristics Weight
Inter-
Adaptability High
pretati
Installability Low Metric Formu- on of
Co-existence High No Parameter A Parameter B
name la mea-
Replaceability Medium snred
Portability compliance High value
I Adapta- Number of Total number x the
:.'I.i' =!!
bility of is data structures of data better
data which are structures if its
II. LITERATURE REVIEW structure operable and requiring value
s has no adaptation closer
A. Academic Information System Quality Evaluation limitation after capability to I
adaptation,
Chua, B.B. & Dyson, L.E. [4] have applied ISO/IEC 9126 confirmed in
to measure e-learning quality. The study carried out review
(example:
measurements on the quality of Blackboard application.
number of
However, aspects of which were measured in these studies tables in the
are aspects of functionality, reliability, usability, and database used)
2 Hardwar A Number of Total number x the
efficiency. Blackboard application portability aspect quality J! !!!j
e implemented of functions better
is not measured. That is because the quality measurements envlron- functions with if its
are done by educators. Portability aspect is considered too mental which are hardware value
complex to be tested by evaluators who are not an [T expert. adaptabi capable of environment closer
-lity achieving adaptation to I
The study resulted in the conclusion that [SO/IEC 9[26
required capability
quality model, mainly functionality, reliability, and results in requirements
efficiency characteristic can be used to measure the quality specified
multiple
of an e-Iearning system. While the usability characteristics
hardware
were deemed to be too complex to be applied by educators. environment
Alrawashdeh, Muhairat, & Althunibat [5] adapted the ISO as specified
3 Organisa A number of Total number x the
/ [EC 9126 as a quality model for measuring the quality of :.'I.i'=-
-tional ij implemented of functions better
Enterprise Resource Planning (ERP). The resulting quality environ- functions with if its
model is used to verify whether the ERP implementation can ment which are organizational value
succeed or fail in an educational institution. This study adaptabi capable of environment closer
-lity achieving adaptation to I
compared the quality of existing quality models and required capability
identified the quality characteristics of the ERP system. The results in requirements
results of this study are a derived quality model of ISO / IEC specified
multiple
9[26, which measures the quality of ERP from the user side.
organizational
Quality models adjusted to the quality characteristics of and business
ERP system. environment
as specified
Rochimah, et.al.[6], used [SO/IEC 9[26 to measure the
4 System Number of Total number x the
usability characteristic on Administration Module of software
:.'I.i'-.:!
- implemented of functions better
;a
Academic Information System. envlron- functions with if its
mental which are adaptation value
B. ISO/lEe 9126 Quality Model adaptabi capable of capability closer
-lity achieving requirement to I
ISO/IEC 9126 quality model is an international standard required in specified
that has been established by the International Organization results in software
for Standardization (ISO) and the [nternational specified environment
multiple
Electrotechnical Commission (1EC). ISO/IEC 9126 can be
system
used to evaluate the quality of software. ISO/IEC 9126 has software
four sections. The first part of the [SO/IEC 9126-[ describes environment
as specified
a general overview of the characteristics and existing sub­
5 Porting A Number of Total number x the
characteristics on ISO/IEC 9126. The second part of ::< =-
user ;a implemented of functions better
[SO/IEC 9[26-2 describes the external metric. External friendlin functions with system if its
ess which are software value
metrics used to measure the quality of software while it is in
capable of environment closer
operation. The third part is the ISO/IEC 9126-3. It describes achieving adaptation to I
the internal metric. [nternal metrics can measure the quality required capability
of the software at the stage of any phase of software characteristics. These characteristics include the
development without having to operate the software. While functionality, reliability, usability, efficiency,
the fourth section, which is ISO / IEC 9126-4 describes the maintainability, and portability [2]. Characteristic analyzed
use of software quality metrics for each characteristic in this study is portability [3].
described in ISO / IEC 9126-l.
ISO/IEC 9126 quality model has six characteristics. Each
characteristic IS further divided into several sub-

978-1-4799-9863-0/15/$31.00 ©2015 IEEE 134

 Proc. of 2015 2nd Int. Conference on Information Technology, Computer and Electrical Engineering (ICITACEE), Indonesia, Oct 16-18th

TABLE 4.
CALCULATION PARAMETERS OF CO-EXISTENCE SUB­
TABLE 3 CALCULATION PARAMETERS OF INSTALLABILITY SUB- CHARACTERISTIC'S METRICS [7]
CHARACTERISTIC'S METRICS [7] Interpreta
Metric Formu Parameter Parameter tion of
Interpreta No
Metric Formu Parameter Parameter tion of
name -la A B measnred
No value
name -la A B measnred
I Available Number of Number of x the better
value
co-
:.'It = ,::! entities in if its value
I Ease of A. Total x the better is entities
;!i=-
Number of
Setup number of if its value existence with which production closer to I
B' implemen-
Re-try setup closer to I enVlfonmen
ted retry product can
operations t that
operations co-exist as
required require co-
for setup specified existence
A.
n=-
2 Installa- Number of Number of x the better
tion � automated installation if its value
2. environmental adaptability, Organisational
effort installation steps closer to I
steps required environment adaptability, System software
3 Installa- A. Number of Number of x the better environmental adaptability, and Porting user
=
tion :It j implement- customizable if its value friendliness. This subcharacteristic has a high weight.
flexibilit ted installation closer to I
Detailed metrics on adaptability subcharacteristic can
y customiza- operation
ble required be seen in Table 2.
installation 3. Installability subcharacteristic aims to measure the
operation as ability of a software to be installed in a particular
specified
environment. installability subcharacteristic consists
of Ease of Setup Re-try metnc, mstallatlOn effort, and
installation flexibility. This subcharacteristic has low
C.AIS Quality Instrument and ISO/IEC 9126-3 weight. Table 3 shows the detailed metrics on
Portability Characteristic[l} installability subcharacteristic.
AIS Quality Instruments (AISQI) is a quality measurement 4. co-existence subcharacteristic aims to measure the
tool that is implemented specifically for the AIS. AISQI ability of a software to be able to co-exist with other
combine quality factors in the ISO/IEC 9126 and completed software that share resources in a single environment.
by ISO/IEC 9126 25010: 2011, WBAQM, and COBIT 4. 1. co-existence subcharacteristic consists of only one
The main framework of the model quality is WBAQM. metric, which is Available co-existence metric. This
AISQI divide by WBAQM quality factor, which is split into subcharacteristic has high weight. Table 4 shows the
three perspectives. AISQI is designed for Academic detailed metrics on co-existence subcharacteristic.
Information System. In addition, because WBAQM a
quality model derived from the ISO/IEC 9126, on the AISQI
D. ITS Academic Information System Assessment Module
also conducted evaluations using a quality factor of ISO /
ITS Academic Information Systems (SIAKAD) is a
IEC 25010:2011. At the owner perspective, the
management information system that is used to store
measurement is done by adopting quality measures COBIT
institute's academic data. SIAKAD ITS is capable of storing,
4.1. AISQI perspective is divided into three, namely visitors,
processing, and helps producing academic reports in
developers and institutions. There are 12 quality factors in
accordance with the needs of teaching and learning activities
AISQI. Accesibility, information contained, usefulness,
in ITS. SIAKAD ITS integrates with other information
credibility, security, functionality, intemationalion,
systems such as scholarships Management Information
portability, maintenance ability, reusability, differences
System (MIS), Reporting MIS, Reasoning MIS, SKEM
factor, popularity, and profitability.
MIS, etc. to help the decision-making process. Users of
Portability characteristic is a characteristic which reflects SIAKAD including students, faculty, administrative staff,
the ability of a software to be transferred from one and management. Each user has the access rights of each
environment to another environment. Portability user according to the type.
characteristic ISO / IEC 9126-3 has five subcharacteristics. SIAKAD ITS has several modules that serve the needs of
Each subcharacteristic has different weights. Weighting is specific users. One of the modules that exist at ITS is
done so that the evaluators can focus on the evaluation of SIAKAD assessment module. This module helps users in
important subcharacteristic points. The higher the weight, managing student academic achievement and the Lecturer
the more important a subcharacteristic trait is on a software. value index. Management of academic grades is an
Weighting for each subcharacteristic can be seen in Table 1 important part of academic institutions because it's one of
In the calculation, the weights can be represented by an the main business processes. Almost all SIAKAD ITS user
ordinal scale of 1 to 9. Subcharacteristic available on who have access rights as students, faculty and
portability characteristics are as follows: administrative staff should be able to perform the function
1. adaptability subcharacteristic aims to measure the of these modules according to their individual needs.
software's ability to adapt to various environments SIAKAD ITS's features which belong to the assessment
without any restriction and are able to produce the module is associated with grade reporting feature, whether it
expected output users. This subcharacteristic is is student grade or lecturer Grade. These features were
composed of five metric measurements, the evaluated in this study
adaptability of data structures metric, Hardware

978-1-4799-9863-0/15/$31.00 ©2015 IEEE 135

 Proc. of 2015 2nd Int. Conference on Information Technology, Computer and Electrical Engineering (ICITACEE), Indonesia, Oct 16-ISth

TABLE 5. SIAKAD ITS ASSESSMENT MODULE EVALUATION TABLE 6.

RESULT SUBCHARACTERISTIC CALCULATION RESULT
N Metric Subcharac- Parameter Parameter Re- N Subcharacte Total metrics Calculation Weight
o name teristic A B suit 0 ristic name used result calculation
I Adaptability Adaptability nla nla nla result
of data I Adaptability 2 1.0 3.0
structures 2 Installability 0 nla nla
2 Hardware Adaptability nla nla nla 3 co-existence I 1.0 3.0
environ-
mental
the quality of ITS SIAKAD Assessment Module. This is due
adaptability
3 Organisa- Adaptability 17 17 1.0 to the adjustment of the state of the system with the purpose
tional of each of the existing metrics on portability characteristics
environment of AISQI.
adaptabiIity
The first metric used is the organizational environmental
4 System Adaptability 17 17 1.0
software adaptability metric. ITS AIS Assessment Module has a need
environ- to be able to adapt in a variety of organizational
mental
environments such as administrative structures, chairman of
adaptabiIity
5 Porting user Adaptability nla nla nla the department, faculty, and students. Based on the results of
friendliness the review, 17 features of the ITS SIAKAD Assessment
6 Ease of Installability nla nla nla Modules are able to adapt well to the organizational
Setup Re-try
pengaturan environment that has been determined. So the results of the
7 Installation Installability nla nla nla calculation according to the formula in the Organisational
effort environment adaptability metric generate maximum value is
8 Installation Installability nla nla nla
1. 0.
flexibility
9 Available co-existence 3 3 1.0 The second metric is the system software environmental
co-existence adaptability metric. ITS AIS Assessment Module is a web­
based application that can only be accessed by using a
browser. ITS AIS Assessment Module has the need to be
accessible to the commonly used browsers such as Mozilla
III. PORTABILITY CHARACTERISTIC EVALUATION ON Firefox, Google Chrome, and Internet Explorer. Based on
SIAKAD ITS ASSESSMENT MODULE the results of the review, 17 features of the ITS SIAKAD
Assessment Module can be accessed by the browser that has
The first step of the evaluation using AISQI is gathering
been determined. System software environmental
artifacts of the software under test. The artifacts include
adaptability metric calculation generates the maximum
document requirements, design documents, reviewing
value which is 1. 0.
documents, source code, and other supporting attributes in
The third metric is the availability co-existence metric.
accordance with the needs of the metrics used for the test.
ITS AIS Assessment Module shares the same operational
The artifacts will be used as input parameters for the
resources with other modules of ITS AIS. In this case, they
calculation of the quality of the software. Artifacts obtained
access the same database server. ITS AIS overall uses three
by collecting data from the system developer, maintainer of
database servers. ITS AIS Assessment Module successfully
the system, and system users.
co-exist with other modules so that the measurement of this
Measurement of quality that is based on the AISQI
metric produce maximum value which is 1. 0. The
portability characteristics requires input parameters such as
measurement results can be seen in Table 5.
user requirements and software review report. User's feature
Then from those three metrics, the sub-characteristic
needs are compared with the software feature's that has been
value of ITS AIS Assessment Module is calculated.
implemented. It is also necessary to add supporting
Organizational environmental adaptability and system
documents governing rules or regulations relating to
software environmental adaptability are included in the
portability aspects of the system.
adaptability sub-characteristic. While the availability co­
Once the input parameters obtained, calculation based on
existence metric is included in the co-existence sub­
the formula of the metrics listed in guidelines of AISQI is
characteristic. So the sub-characteristics calculated are
done. Metric calculation will produce the value that
adaptability and co-existence sub-characteristic. The results
represents the quality of the software being tested in
of both sub-characteristic can be seen in Table 6.
accordance with the purpose of each metric.

v. CONCLUSION
IV. EVALUATION RESULT
Evaluation on SIAKAD ITS Assessment Module
Calculation parameter of each metric on AISQI
indicates that the system has good portability characteristics
portability characteristics are used to measure the quality of
and can meet user expectations. Based on the results of the
ITS AIS Assessment Module. ITS AIS Assessment
review and calculation, three metrics that was used produce
Module's Attributes are used as input. The data are obtained
the maximum value which is 1. 0. subcharacteristic values
by extracting the user requirement as well as reviewing ITS
evaluated also generate maximum value. adaptability
AIS Assessment Module directly. Based on requirement
subcharacteristic produces value 3. 0. Co-existence
analysis and review analysis, there are three metrics used to
subcharacteristic also produces value 3. 0.
measure

978-1-4799-9863-0/15/$31.00 ©2015 IEEE 136

 o
Proc. of 2015 2nd Int. Conference on Information Technology, Computer and Electrical Engineering (ICITACEE), Indonesia, Oct 16-18th

Like ISO/IEC 9126-3, AISQI portability characteristic is

capable of measuring the quality of the software's
organizational view and hardware resources of the system.
Organisational environment adaptability metric calculation
results showed that SIAKAD ITS Assessment Module can
adapt well at all levels of the ITS organization. Additionally
available co-existence metric calculation results also
indicate that SIAKAD ITS Assessment Module is able to
share hardware resources with other SIAKAD ITS modules
well. There is a metrics on the AISQI portability
characteristic which is less can be applied to SIAKAD ITS
Assessment Module, the installability subcharacteristic,
because SIAKAD ITS Assessment Module is a web-based
software. So users do not need to do installation before
using the system.

ACKNOWLEDGMENT
This study is supported by Laboratorium Based Education
(LBE) Grant for Software Engineering Laboratory of Institut
Teknologi Sepuluh Nopember with the main topic is
Managing Evolution and Quality of Academic Information
Systems, Case Study: ITS Academic Information Systems.

REFERENCES

[1] Yuhana, u.L., Rahardjo, A.B., Rochimah, S. Academic Information

System Quality Measurement using Quality Instrument: A Proposed
Model, Proceeding of International Conference on Data and Software
Engineering (ICoDSE 2014),Indonesia,page 231-236
[2] ISO/IEC .lTC, "Information Technology - Software Product Quality -
Part I Quality Model," ISO, Geneva, International Standard
ISO/lEC FDIS 9126-l :2000(E),2000.
[3] Syahrul Fahmy, Nurul Haslinda, Wan Roslina, and Ziti Fariha,
"Evaluating the Quality of Software in e-Book Using the ISO 9126
Model," International Journal of Control and Automation, vol. 5, no.
2,pp. 115-122,Juny 2012.
[4 ] Chua, B.B. & Dyson, L.E. (2004). Applying the IS09126 model to
the evaluation of an elearning system. In R. Atkinson,C. McBeath,D.
Jonas-Dwyer & R. Phillips (Eds), Beyond the comfort zone:
Proceedings of the 21st ASCILITE Conference (pp. 184-190). Perth,
5-8 December.
[ 5] Thamer A. Alrawashdeh, Mohammad Muhairat, and Ahmad
Althunibat, "Evaluating the Quality of Software in ERP Systems
Using the ISO 9126 Model," International Journal of Ambient
Systems and Applications (IJASA), vol. I, no. I, pp. 1-9,March 2013.
[6 ] Rochimah, S. Rahmani, I.R., Yuhana, u.L., "Usability Characteristic
Evaluation On Administration Module of Academic Information
System Using ISO/IEC 9126 Quality Model", Proceeding of
International Seminar on Intelligent Technology and Its Application
(ISITIA) 2015 page. 361-366,Indonesia,20-21 May 2015.
[7] ISO/IEC .lTC, "Information Technology - Software Product Quality -
Part 13 Internal Metrics," ISO, Geneva, International Standard
ISO/lEC TR 9126-3,2002.

978-1-4799-9863-0/15/$31.00 ©2015 IEEE 137