2015 International Conference on Data and Software Engineering
A Proposal for a Quality Model for
E-Government Website
Bayu Hendradjaya
School of Electrical Engineering and Informatics
Institut Teknologi Bandung
Bandung, Indonesia
bayu@stei.itb.ac.id
AbstractThe goal of an E-Government system is to allow the
government services via an electronic means by providing online
information for public accesses. In addition, the services should
also focus on improving the quality of public services effectively
and efficiently. The website should not only provide information
and services, but also should have some certain qualities as key
factors for the success of the implementation of the website.
In this paper, we propose a quality model to assess the E-
government website. The model uses an international standard
for quality and equipped with steps to measure it. The quality
model use some internal/external metrics. The users of the
quality model can be helped with specific tools to help calculate
the metrics.
We have applied this model to an e-Government website as a case
study. The results shows that the model can provide information
regarding the the quality of the case study. Thus we belive that
our propose model can become an alternative for evaluating the
the quality of an e-Government website.
Keywords e-Government, website, quality model, quality
metrics
I. INTRODUCTION
E-Government helps the government to service the citizen,
community or industry. Website designs are needed as one of
the implementation of E-Government. Currently many
government bodies have provided websites as a their first
stage in implementing the services. The content of a website
is important to deliver the service provided by a government
body, this it is essential that a certain quality should be
possessed by an E-Government Websites.
E-Government system is considered as part of an information
system [1] where the software applications and their users are
part of an e-Government system. In this research the software,
content and service is the key factors that are used to measure
the quality of e-Government website.
Several researches, such as in [2], [3], [4], [5], [6], [7] have
discussed how the quality of an e-government website is
assessed based on the perspective of the users (e.g. citizen,
community or industry). Their proposals are examined
through questionnaires from the respondents which is actually
the user of an e-Government system. In our research, we
propose a dimensional and quality attributes to acquire the
main characteristics of an e-Government websites. We have
used ISO 9126 as the basis to propose the characteristics. ISO
9126 is a formal international standard for software quality.
978-1-4673-8430-8/15/$31.00 2015 IEEE
Rina Praptini
School of Electrical Engineering and Informatics
Institut Teknologi Bandung
Bandung, Indonesia
r1n4rien@gmail.com
We then use GQM (Goal-Question-Metrics) approach to
determine the required metrics that are related to the proposed
quality attributes.
I. RELATED WORKS
Yi-Shun Wang and Yi-Wen Liao [2] studied how the
implementation of an e-Government was viewed from the
citizen perspectives. Their study adopted Information System
Success Model of DeLone & McLean for the environment of
G2C in e-Government system. The model was validated using
Structural Equation Modeling Technique, using 119 responds
from the user of 6 popular G2C application in Taiwan. The
model has six dimensions: Information Quality, System
Quality, Service Quality, Use, User Satisfaction, dan
Perceived Net Benefit. They suggested that G2C system
should have good information quality, system quality, and
service quality. These qualities should give positive impact to
the users and at the end should increase the net benefit of the
system.
E-GovQual [3] propose four dimensional framework to
evaluate the quality of service of e-Government. The
framework was developed based on classical theory and
model of SERVQUAL model. The model has offered
efficiency, trust, reliability and citizen support as part of the
dimensions.
COBRAS [4] evaluates the e-Government website that is
based on Costs, Opportunities, Benefits and Risks Analysis for
Satisfactions. Their proposal is evaluated through the respond
questionnaire of an e-Government Turksat in Turk.
Another proposal by European Top of the Web [5] focuses on
the final benefit obtained by the users. A survey was
conducted to measure user satisfaction that are based on
Usability, Benefit, dan Overall evaluation of users
satisfaction compare to their actual hopes. An online survey
was launched targeting users of common citizens and industry.
Another approaches by Thailands government [6] focused on
the development of conceptual framework and elicitation
factors such as reliability, linkage, content, ease of use and
self-service. These factors are assessed by e-Services in
19
 2015 International Conference on Data and Software Engineering

several government body of Thailand (thaigov.net). A survey
of 150 responds was conducted.
Another survey ran by Cairo University used online survey to
study the assessment of Egypts government (www.egypt.-
gov.eg) [7]. The questionnaires contains questions to assess
the service quality in 10 dimensions such as Functionality,
responsiveness, navigation, accuracy, availability, security,
assurance/trust, efficiency, currency, performance.
Quirchmayr et al. has developed another E-Government
quality model that is based on ISO 9126. The model consists
of 19 quality characteristics that is categorized into supply
side and demand side.
In this research, we refer to ISO/IEC 9126 [8] as the base of
our proposal. ISO/IEC 9126 is an international standard to
evaluate the quality of software application. The software
product quality can be evaluated using internal attributes (by
applying static analysis of software products, i.e. source code)
and external attributes (evaluating the behaviour during
execution), or by using quality in use attributes (based on
users view). ISO/IEC 9126 categorized the qualities to six
characteristics, namely: functionality, reliability, usability,
efficiency, maintainability dan portability, which is
decomposed to another sub characteristics. Quality in use is
categorized to: effectiveness, productivity, safety and
satisfaction. The ISO/IEC 9126 has been used to assist in
proposing some model in our previous work [9] [10] [11]. The
standard was considered to be useful in defining the quality
models.
II. THE PROPOSAL
Based on our investigation, there are various dimensions and
attributes in quality of e-Government [2], [3], [4], [5], [6], [7].
These can be grouped to three dimensions of quality of an E-
Government website, as the following: Service Quality,
service to the users. In the end, it contributes to cost-
efficient and cost-effective.
2. Information Quality, information is expected to be
accurate and reliable, up to date and easy to understand.
This should increase peoples confidence to the website.
3. System Quality
a. System availability. Users expect the web to always
ready and fast. Installing a web site on bad system
may degrade the use of the web.
b. Access Time. Good system allows faster download,
thus increased users confidence in using the web.
c. The use of standard browser. The compatibility of the
web to many standard browsers are important. Some
website may not render properly on certain browser.
d. Link. Broken link may lead to bad user experience.
e. Usability. The Accessibility, navigational, easy to
use, consistency is criteria used to see the usability of
a website.
To help measuring the quality of a website, we need to
propose some metrics. The metrics are developed based on
ISO9126 and other references [12], [4], [13], [5], [14], [15],
[16], [17], [18]. Other detailed references are shown in Table 1.
We have used GQM approach [19] to develop the metrics.
The GQM model is shown in Table 2. Three quality
dimensions (Service quality, information quality and system
quality) are the basis to develop the goal of the website. Each
goal is followed by several questions to help characterize the
object to be measured. The attributes from quality dimensions
is the basis of the questions. Each question is then mapped to
one or more metrics.
Table 1 Dimensions and Quality attributes of e-Government
Quality dimensions References
Responsiveness [4], [5], [6], [7]
Service Quality

Service Availability [4], [5], [6]

Information Quality, and System Quality. Table I shows the Multi-lingual [4], [5]
current research of dimensions and quality attributes of e- Service Accuracy [4], [6]
government. User Satisfaction [4], [5]
These dimensions are described as follows: Security and Trust [3], [4], [7]
Service Effectiveness and
1. Service Quality Efficiency [4], [5], [7]
a. Responsiveness. Ability to allow fast response on
Information Accuracy [3], [4], [6], [7]
specific services for users.
Information

Relevance [4], [6], [7]

Quality

b. Service Availability. Allows flexibility for users to

access the system. Up to date [3], [4], [6], [7]
c. Multi-lingual. Support for more than one language
Information understandability [3], [4], [5], [6], [7]
may become an advantage.
d. Service Accuracy. The delivery of a service should System availability [3], [4], [5], [6], [7]
System Quality

promote accuracy to let people use the service. Access Time [3], [4], [5]
e. Service usability. An easy to use service allows Standard Browser Usage [3]
people to use the service. Link [4], [6]
f. Security and Trust. A better security system should Usability [3], [4], [5], [6], [7]
increase the trust of users.
g. Service Effectiveness and efficiency. Allowing a
Figure 1 presents the mapping of quality factors to the
good online service may reduce bureaucratic
metrics, which is actually the description of our proposed
procedure which should potentially increase better
model.

20
 2015 International Conference on Data and Software Engineering

Table 2 The mapping of quality dimension of e-Government Website using GQM approach

Goal Quality
Questions (GQM) Quality Characteristics Proposed Metrics
Attributes

To Responsiveness Is the web provide enough service to allow Usability Interactivity Website interactivity
Increase user interaction?
Service
Quality
How quick is the response time to a required Efficiency Time behaviour User service response time
service?
Time to complete the service
Service Is certain functionality are available and Functionality Suitability Functional implementation
Availability properly functions? completeness
Functional Adequacy
Is the service always up? Reliability Maturity Service availability
Multi-lingual How many language are supported in the Usability Multi-lingual Multi-lingual support
website?
Service Does the service provide accurat Functionality accuracy Computational Accuracy
Accuracy information?
User Do the users understand the use of a service Usability Undestandability Function Understandability
satisfaction or feature?

Security and Is the service secure enough? Functionality Security Data Encryption
Trust
Service Is online service actually cheaper? Productivity - Economic productivity
Effectiveness
and Efficiency
Does online service require shorter time?

Is it more effective when users use online

service?
To Information Is the information in the website accurate Fuctionality Accuracy Information accuracy
increase Accuracy enough?
informati Relevance Is relevance information in the website? Information relevancy
on Suitability
quality Up to date Is recent news in the website? Information timeliness

Information Is the user understand the displayed website? Usability Understandabilit Information Understandability
understandabilit y
y
To System Is the system always up? Reliability Maturity Failure density
increase availability
system Access Time Does the web display fast enough? Efficiency time behaviour Page load time
quality
Standard Can a website run in a standard browser? Portability Adaptability System software environmental
Browser Usage adaptability
Link Is link available and no broken link? Reliability Maturity Broken Link
Usability Is it easy to navigate the web? Usability Understandabilit Navigability
y
Learnability
Operability
Are the user interfaces relatively consistent? Operability Operational Consistency In Use

Is the web accessible to all users? Operability Website Accessibility

Is the web pleasant to use? Attractiveness Attractive interaction

21
 2015 International Conference on Data and Software Engineering

Table 3 presents some positive statements that were obtained

from the questionnaires. The actual statements were expressed
in the Indonesian language, however we have summarized
their comments in the list. We think that these comments are
useful to further assess the website the quality.
Table 3 Statements found in the questionnaires

No Statements Metrics
1 The website is interesting Attractive
interaction
2 The website is easy to use
3 The website is easy to learn
4 It is easy for me to find the required
information/service Navigability

5 It is easy to navigate the web

6 The web contents are consistent in

Operational
terms of colors, fonts, buttons
Consistency In Use
7 I know how to use the services in the
Figure 1 Proposed Quality Model for e-Governments Website Function
website
understandability
We propose 6 quality characteristics that can be used to assess 8 The web has provided enough Information
the quality of a e-government website: functionality, information understandability
reliability, usability, efficiency, portability dan productivity 9 The web has presented related
Information
(please see fig. 1). Most of these characteristics were taken information according to the certain
relevancy
from ISO 9126. Productivity comes from Quality in use, the service
other characteristics are from the internal/external quality. We
also add interactivity and multi-lingual to usability
characteristics (these are not in ISO 9126). III. THE APPLICATION OF QUALITY MODEL
Some metrics are measured using tools. The criteria for the We have applied the model to an Indonesian e-Government
tools are: website. The application aims to validate our proposal. The
website is owned by a division of Transportation minister,
1. Be able to calculate total time when failing to connect a which is responsible to monitoring the road transfport and
website (we have used Uptimerobot1) managing the traffic (Road Transport and Traffic Management
2. Be able to calculate broken link (Searchmetric rapid2 and Center (RTTMC)). The web can be reached through
online broken link checker3 has been used). http://www.rttmc-hubdat.com/.
3. Be able to calculate errors in accessibility. The tool
should support WCAG or Web Content Accessibilty The main features of the web site are as follows:
Guidelines[18] (we have used WAVE Web 1. Live CCTV, monitors traffic conditions from several
Accessibility Versatile Evaluator4) locations.
4. Be able to calculate time to display the web pages. 2. Traffic information, describes the traffic and the live
(Gtmetrics5 have been used). weather information based on live CCTV
5. Be able to see how a website can adapt to certain browser 3. News, related to transportation
and operating systems (we have used 4. Route Information, especially some alternative routes for
CrossBrowserTesting6) accidents or other kinds of traffic jams.
Some proposed metrics are evaluated by questionnaires, using 5. Contact Infrmation: direct contact to police or hospital
Likert scale to determine their values. information
6. Arrival-Departure of bus information
7. Sms center, to facilitate interaction between the citizen to
RTTMC information regarding transportation. At this
1
https://uptimerobot.com stage, twitter and facebook have been used as a means of
2
http://rapid.searchmetrics.com/en/tools communication.
3
http://www.brokenlinkcheck.com
4
http://wave.webaim.org
5
http://gtmetrix.com
6
https://crossbrowsertesting.com/

22
 2015 International Conference on Data and Software Engineering

Table 4 Measurement result of RTTMC website Quality Interpretations

Interpretations Metrics Results
Quality Factors
Metrics Results
Factors
1= support more than 1
Functional 0 <= X <= 1 language
Multi-lingual
implementation 1 0
Closer to 1 is better support
completeness 0= only 1 language
Functional 0 <= X <= 1
Smaller is better, or we can
adequacy Closer to 1 is better 36 min compare to the response
0,86
User service 15 sec time for each services (this
Information 1 <= X <= 5 response time is usually different for other
3,95
relevancy Closer to 5 is better organization)

X>=0 Efficiency
Information Time to Currently there is no online
timeliness 12 days Smaller is better, we need to N/A
complete the services for administering
(News) see the standard used by an
Functional service licensing of service
organization.
ity
X>=0 Page load time Smaller is better
9,28 sec
Information 8 hours Smaller is better. In our
timeliness (Live 27 min case study it is 5 minutes System software 0 <= X <= 1
CCTV) 58 sec update. Portability environmental 0,84
adaptability Closer to 1 is better
0 <= X <= 1
Information 0,95
accuracy Closer to 1 is better The results in the table 4 shows that the RTTMC website is
quite good. Most metrics numbers could be interpreted as close
Computational No computation
Accuracy
N/A to the limit of good boundaries. There are 14 metrics out of 23
Data No private data or password metrics results that can be considered as good. 6 metrics results
N/A
Encryption are considered bad to worse, and another 3 are not applicable.
Service 0 <= X <= 1 Thus it was concluded that the web is good enough, however
0,87 some properties of qualities could still be improved.
availability Closer to 1 is better
0 <= X <= 1 The application of the our model to this case study has
Reliability
Failure density 0,11 shown that the model could be proposed as new alternatives to
Closer to 0 is better
measure the web for e-government. However more work on
0 <= X <= 1 new case studies still needs to be performed to validate the
External Broken
Link
results.
0,23 Closer to 0 is better
IV. CONCLUSIONS
Function 1<=X<=5
Understandabili 3,98
In this research, we have proposed a quality model for e-
ty Closer to 5 is better Government website that mostly is referred from ISO 9126 and
equipped with 22 metrics. The metrics are chosen from internal
Information 1<=X<=5
metrics of ISO 9126-3, external metrics ISO 9126-2 and some
Understandabili 3,99 current proposals. The following metrics are proposed to be
ty Closer to 5 is better
used for the model: Information timeliness, Information
1<=X<=5 accuracy, Service availability, Website accessibility, User
Navigability 3,99
Closer to 5 is better service response time, Time to complete the service dan Page
53 Errors load time. We have proposed the steps to evaluate the metrics
96 Alerts X>=0 and some required tools to help the calculations. We also added
Website
Accessibility
17 the threshold values to the metrics to help concluding the
Usability Contrast Smaller is better quality.
Errors
Operational 1<=X<=5 The model is validated to an e-Government website. We
Consistency In 3,96 Closer to 5 is better have applied the proposed steps and calculate the results. Some
Use of the metrics have shown a good quality of the website,
1<=X<=5 however some is not good enough. We have used all the
Attractive
3,86 Closer to 5 is better guidance in the proposal, therefore we are confident that this
interaction proposed quality model can be another alternatives to measure
the quality of the website. However we realize that more case
1=there is interactions
facility
studies should be performed have a better quality model.
Website
1 Moreover, we also realize that this model may not work on
interactivity
0=no interaction facility different e-Government websites in other countries. Each
country usually has their own custom to serve their citizen.

23
 2015 International Conference on Data and Software Engineering
On the other hand, the model can still be improved by
evaluating services with transactions which usually requires
certain identification and authentication, especially when a
citizen uses the service. In this case, other metrics, such as
Computational Accuracy, Data Encryption, Time to complete
the servic, or Security, may become part of the model.
REFERENCES
[1] R. Heeks, Implementing and Managing eGovernment, London: SAGE
Publications Ltd, 2006.
[2] Y.-S. Wang and Y.-W. Liao, "Assessing E-Government Systems
Success: A Validation Of The Delone And Mclean Model Of
Information Systems Success," in Proceedings of the 11th Annual
Conference of Asia Pacific Decision Sciences Institute, Hong Kong,
2006.
[3] X. Papadomichelaki and G. Mentzas, "e-GovQual: A multiple-item scale
for assessing e-government service quality," Government Information
Quarterly, pp. 98-109, 2011.
[4] I. Osman, A. Anouze, Z. Irani, H. Lee, A. Balci, Medeni.T.D and V.
Weerakkody, "A New Cobras Framework To Evaluate E-Government
Services: A Citizen Centric Perspective," West London, 2011.
[5] eGovernment Unit , DG Information Society, European Commission,
"Top of the web: User satisfaction and usage survey of e-government
services," 2004.
[6] N. Sukasame, "The Development of e-Service in Thai Government,"
2004.
[7] Decision Support and Future Studies, Cairo University , Faculty of
Computers and Information, "Egypt E-Government portal for better
services," Giza, 2010.
[8] I. F. 9126-1, "Information technologySoftware product quality Part
1:Quality model," ISO/IEC 2000, 2000.
[9] H. Fauzia, H. Laksmiwati and B. Hendradjaya, "A quality model for
mobile thick client that utilizes web API," in The 2014 International
Conference on Data and Software Engineering, Bandung, 2014.
[10] Y. Fitrisia and B. Hendradjaya, "Implementation of ISO 9126-1 quality
model for asset inventory information system by utilizing object oriented
metrics.," in The 2014 Electrical Engineering and Computer Science
(ICEECS 2014), Denpasar, 2014.
[11] I. Lestari and B. Hendradjaya, "The application model of learning
management system quality in asynchronous blended learning system,"
in The 2014 International Conference on Electrical Engineering and
Computer Science, Denpasar, 2014.
[12] K. C. Bizer, "Quality-Driven Information Filtering in the Context of
Web-Based Information Systems," Berlin, 2007.
[13] L. Mich, M. Franch and a. L. Gaio, "Evaluating and Designing Web Site
Quality," IEEE, 2003.
[14] L. Olsina, G. Covella and G. Rossi, "Web Quality," in Web Engineering,
Springer Berlin Heidelberg, 2006, pp. 109-142.
[15] B. Behkamal, M. Kahani and M. K. Akbari, "Customizing ISO 9126
quality model for evaluation of B2B applications," Information and
Software Technology, pp. 599-609, 2009.
[16] J. Fisher, J. Bentley and A. Craig, "A usability instrument for evaluating
websites - navigation elements," OZCHI, Melbourne, 2004.
[17] Minervagroup, Quality principles for cultural websites, Minerva project,
2005.
[18] A. Parasuraman, V. A. Zeithaml and L. L. Berry, "A Conceptual Model
of Service Quality and Its Implication For Future Research," Journal of
Marketing, pp. 41-50, 1985.
[19] V. Basili, "Software modeling and measurement: the
Goal/Question/Metric paradigm," University of Maryland, 1992.
[20] G. Quirchmayr, S. Funilkul and W. Chutimaskul, "A Quality Model of e-
Government Services Based on the ISO/IEC 9126 Standard," in IRIS
2007, 2007.
[21] W3C, "Web Content Accessibility Guidelines (WCAG) 2.0," 11
December 2008. [Online]. Available: http://www.w3.org/TR/WCAG20/.
[22] P. R. Indonesia, "Instruksi Presiden Republik Indonesia Nomor 3 Tahun
2003 Tentang Kebijakan Dan Strategi Nasional Pengembangan E-
Government," Jakarta, 2003.
24