Make Your Choice: Dimensionality of an Open Integrated
Conceptual Model for Evaluating E-Service Quality,
Usability and User Experience (e-SQUUX) of Web-Based
Applications
Samuel Ssemugabi
School of Computing
University of South Africa (Unisa)
Johannesburg, South Africa
ssemus@unisa.ac.za
ABSTRACT
Developments in internet technology and pervasive computing
have resulted in a variety of web-based applications (WBAs),
including business transactions, information-delivery and social
networking, as well as e-government, e-health and e-learning.
Different websites have varying requirements and users have
differing backgrounds, experiences and cultures. Users require
effective, easy and enjoyable interaction, which is key to
successful use and acceptance of applications. This study
generated a model that can give rise to customized frameworks to
evaluate three related, yet distinct, constructs: e-service quality,
usability and user experience (e-SQUUX) of WBAs in an
integrated manner.
Following a rigorous review of 264 credible literature sources, a
set of unique dimensions was identified, each associated with one
or more e-SQUUX constructs. Through an iterative reduction and
evolution process, an integrated conceptual e-SQUUX model was
derived, comprising 24 categories, 75 main dimensions and 163
associated dimensions. The model is ‘open’ in that practitioners
and researchers can make their own personal ‘choice’ from the
components of the model to synthesize an implementation of the
model to be used as an evaluation framework.
CCS Concepts
• Human-centered computing ➝ Human computer interaction
(HCI) ➝ HCI theory, concepts and models.
Keywords
Conceptual model; dimensions; e-service quality; evaluation;
usability; user experience; web-based application
1. INTRODUCTION
Developments in internet technology and pervasive computing
over the past two decades have resulted in rapid growth and high
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. Copyrights for
components of this work owned by others than ACM must be honored.
Abstracting with credit is permitted. To copy otherwise, or republish, to
post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from Permissions@acm.org.
SAICSIT '16, September 26-28, 2016, Johannesburg, South Africa
© 2016 ACM. ISBN 978-1-4503-4805-8/16/09…$15.00
DOI: http://dx.doi.org/10.1145/2987491.2987527
M.R. (Ruth) de Villiers
School of Computing
University of South Africa (Unisa)
Johannesburg, South Africa
ruth.devilliers1@gmail.com
dvillmr1@unisa.ac.za
diversification in the use of web-based applications. The internet
is used not only to transfer information via the web but is
increasingly used to provide electronic services to a variety of
users with different characteristics, knowledge and profiles [4,
45]. Usability is considered a key factor of software quality since
it determines whether or not a system can be accepted by users
[19, 48]. Consequently, it has been a key aspect of web-based
applications since the inception of the web in the mid-1990s, due
to worldwide use of the web by users with different backgrounds,
experiences and cultures. In the last decade the emphasis in
human computer interaction (HCI) research has tended to
gravitate more towards user experience than usability [53, 54, 68].
However, the present authors believe that usability is just as
important as user experience. The point of departure in this paper
is that all users want a web-based application to be easy to use
(main focus: usability) and enjoyable to use (main focus: user
experience). In addition, users of web-based applications (WBAs)
need to get required services online without making telephone
calls or physically visiting an organization (main focus: e-service
quality). This study constitutes one of the starting journeys
towards achieving these three focus objectives. It attempts to
create a model that can be used as a framework to evaluate e-
service quality, usability and user experience (e-SQUUX) in an
integrated manner. Though e-service quality originates from the
service marketing field, and usability and user experience from
HCI, the conceptualization of the each of the three constructs has
been ongoing. There are varying definitions and dimensionality
for each [3, 20, 53]. However, the three constructs, though
different, are closely related. For example, Al-Shamayleh et al.
[4] identified the core dimensions of e-service quality as
efficiency, reliability, fulfilment and privacy. Studies by Koohang
[33] and Moumane et al. [46] identified the attributes of usability
as including efficiency, reliability, effectiveness, learnability and
satisfaction. The inclusion of efficiency and reliability in both e-
service quality and usability, shows an overlap between the two.
Similarly, it is widely agreed that user experience is an extension
of usability. According to Park et al. [53] and Hassenzahl et al.
[21], usability is the pragmatic part of UX and is related to
performance.
While e-service quality (e-SQ), usability and user experience
(UX) are indeed related, no research could be found that provides
an integrated model of the dimensionality of all three. This paper
is an effort to that end. Through a systematic literature review, a
multi-dimensional conceptual model has been derived that can be
used to evaluate e-SQUUX. In so doing, we take cognisance of
the fact that such models need to be domain- and context-specific
as stressed by Van Wyk and De Villiers [64]. However, the model
is derived from a wide range of domains that we consider it to be
generic or ‘open’ in nature. The model is also ‘open’ in the sense
that practitioners and researchers can customize e-SQUUX by
making their own ‘choice’ of components for evaluations of
websites or for research. The procedure for usage is described in
Section 6. Not all organizations or researchers would use e-
SQUUX in the same manner, and this customizability will be one
of its strengths.
This paper is organized as follows: Section 2 describes what we
consider to be WBAs, provides a literature overview of the
definitions and dimensions of each of the components of e-
SQUUX and the need for this study. This is followed in Section 3
by the research methodology used to derive e-SQUUX from the
literature. In Section 4 results of the searches and qualitative
content analysis are presented and the reduction process is
explained. Furthermore, the main findings of the study are
presented. Thereafter, Section 5 presents the e-SQUUX
conceptual model and discusses the findings. Section 6 describes
how the conceptual model could be applied. Finally, Section 7
outlines the contributions, limitations and future studies, and
concludes this study. Lastly, the motivation for this study is
provided.
2. WEB-BASED APPLICATIONS,
USABILITY, USER EXPERIENCE, E-
SERVICE QUALITY, AND THE NEED FOR
THIS STUDY
We first define what we consider to be web-based applications.
We then address usability and user experience, which are closely
related. This is followed by a discussion of e-service quality. The
meanings of the concepts are explained and some examples are
given of their dimensions, also called attributes or factors.
2.1 Web-based applications
Since, according to Fowdur et al. [18], web-based applications
(WBAs) include those available on mobile devices, in this
research WBAs are viewed as software applications that use web
technologies such as browsers to run, whether on fixed or mobile
computing devices. The definition includes mobile applications
(Apps) that run on mobile operating systems such as Android or
iOS. However, the research excludes the limitations of certain
mobile devices, such as small screen size and low display
resolution as identified by Moumane et al. [46].
2.2 Usability
Usability is one of the most important factors in HCI research
[41]. The ISO/IEC 25010:2011 [29] defines usability as the extent
to which a product can be used by specified users to achieve
specified goals with effectiveness, efficiency and satisfaction in a
specified context. Good usability results in increased user
productivity, and decreased training costs and user support [19,
40]. Although much research has been done with respect to
usability of general software and usability of websites, there is a
lack of consistency on what exactly is entailed in usability or what
its dimensions are [11]. Nielsen [48] defines usability in terms of
five dimensions – learnability, efficiency, memorability, few
errors, and user satisfaction. Nielsen’s definition or framework of
usability is the most widely adopted and cited, since it provides a
detailed model of usability aspects that can be objectively and
empirically verified through different evaluation methods [31,
39]. There is a need to determine what currently constitutes
usability in terms of its components or dimensions and how it can
be measured. Without this understanding, it is difficult to consider
usability during software development or perform appropriate
software usability evaluation [1, 17].
2.3 User experience
User experience (UX) is a relatively new area in HCI [42, 44].
The phrase ‘user experience’ was coined in 1995 by Don Norman
and refers to a multidimensional concept whose definition is not
yet fully established [2, 34]. UX attempts to go beyond traditional
usability by including hedonic aspects such as beauty, fun,
pleasure and personal growth associated with the use of a product.
A key term for UX is Enjoyability [22, 56, 59, 60]. ISO CD 9241-
210 defines UX as “all aspects of the user’s experience when
interacting with the product, service, environment or facility”
[28]. Preece et al. [55] describe UX as the sensations obtained by
users while interacting with a product, as well as the pleasure and
satisfaction (or the reverse) that they acquire during its use.
According to Hassenzahl [20] there is a lack of a widely-accepted,
shared understanding of what UX constitutes. The main reason for
this, put forward by Moczarny et al. [44], is that UX is complex,
context-specific, subtle and subjective in nature.
2.4 e-Service Quality
The use and development of web technologies have led to
numerous innovative ways in which organizations provide
customer service. Service quality is the ability of an organization
to meet the needs, wants and expectations of a customer. It is
therefore dependent on customer perceptions over time, based on
past experience, the service process and service delivery [62].
Parasuraman et al. [51] and Zeithaml et al. [70] define service
quality from a customer’s perspective as the extent of discrepancy
between customers’ expectations or desires and their perceptions.
This definition led to the development of SERVQUAL, the most
used SQ quality model and instrument for measuring service
quality [4]. SERVQUAL is a multi-item scale developed to assess
customer perceptions of service quality in service and retail
businesses [51, 62]. Due to the successful use of SERVQUAL, it
has been adapted in many other sectors, including e-commerce, to
evaluate the quality of service provided [4]. This has led to the
concept of electronic service quality, commonly referred to as e-
service quality (e-SQ).
E-service quality, also known as website-, web-based or online
service quality is defined as the extent to which services based on
web technology facilitate effective and efficient online
communications, purchases and delivery of product or services [8,
36, 60]. Similarly, Parasuraman et al. [52] and Marimon et al. [38]
define e-service quality as the extent to which a website supports
efficient and effective online transaction and delivery of products
and services. For this study, the definition by Ojasalo [49] that e-
service is the provision of services to customers or users through
the internet, is adopted.
In his analysis of the dimensions of e-service quality, Ladhari [37]
identified its key dimensions as reliability, fulfilment,
responsiveness, ease of use, privacy/security, web design and
information quality. In addition, the researcher pointed out the
variability of dimensions of e-SQ by different researchers. Most
of the dimensions of e-SQ have been adopted from the five
dimensions of SERVQUAL, namely reliability, assurance,
tangibles, empathy, and responsiveness [10, 50, 52]. These have
been contextualized to web-based application environments to
derive corresponding e-SQ dimensions, appropriate to the web
environment. For example, Parasuraman et al. [52] listed
efficiency, fulfilment, system quality and privacy. Cebi [10]
identified technical adequacy, content, security, communication,
prestige, ease of use, ease of learning, memorability, layout,
graphics, system availability, speed, accessibility, navigation,
reliability, accuracy, privacy, contact info, online help,
responsiveness, sustainability and currency as e-SQ dimensions.
2.5 Need for an integrated e-SQUUX
conceptual model for evaluation of WBAs
Though it has been shown in Section 1 that the three constructs of
e-SQUUX are related and are all geared towards meeting users’
needs, there is a need for a conceptual model that can be used as a
basis for evaluating WBAs. The reason for this is that evaluation
has been identified as the mainstream activity in HCI [42]. The
most commonly used method for evaluation is employing a set of
criteria for measuring constructs. According to Hornbaek and
Stage [26] and Huang et al. [27], measurement and modelling of
constructs of products or services are closely related. The process
normally begins by a description or definition of the constructs,
which includes a set of attributes, and sometimes sub-attributes,
which can be measured objectively or subjectively. Nonetheless,
while there is agreement on objective measures or metrics such as
task completion times, there is little agreement on subjective
software quality measures such as satisfaction [25]. The focus of
this study is identification of the attributes or dimensions of
subjective quality measures, so that WBAs can be evaluated.
Once these dimensions are established, problems are identified so
that solutions can be found and fixed [69].
While various studies have identified the dimensionality of each
of the constructs of e-SQUUX, little work exists on integrating
two, or all three, of the constructs. There have been efforts to
determine the dimensionality of a model of usability and UX, for
example:
• Hedegaard and Simonsen’s [23] study was limited to six
literature sources and two domains, and derived only seven
dimensions, namely, memorability, learnability, efficiency,
errors/effectiveness, satisfaction, enjoyment and fun.
There is a debate as to whether usability is a subset of UX or not:
• While acknowledging that other views exist, Moczarny et al.
[44], and Bevan [6] take the stance that the two are different
but closely related with certain common attributes.
• In studies of UX of products and services Roto et al. [58] and
Park et al. [53] considered usability to be part of UX.
There are some studies related to the dimensionality of usability
and e-SQ. Most of these, such as Bhattacharya et al. [7] view
usability as a subset of e-service quality.
Some work has been done at the intersection of UX and e-SQ:
• According to Carrasco et al. [9], in their study to evaluate
UX and e-SQ of hotel services, the most appropriate
dimensions they found were based on SERVQUAL, namely,
reliability, assurance, tangibles, empathy, and
responsiveness.
We acknowledge these studies, each on two of the constructs of e-
SQUUX, but did not find any extensive study on the dimensions
of all three constructs. The closest was the research on user
satisfaction and usability of e-commerce websites by Sahi and
Madan [60], which modelled four dimensions related to the three
constructs. The dimensions are system quality, trust, extension
quality, and propriety of content. These researchers describe
extension quality as related to e-service quality, satisfaction as
being overall experience, and view system quality and trust as
attributes associated with usability.
The need for this study stems from the two main issues discussed
in this section. Firstly, dimensionality of constructs is critical to
evaluation of e-SQUUX of WBAs. Secondly, there is no
comprehensive integrated model of the dimensions of the three
constructs of e-SQUUX.
3. METHODOLOGY
3.1 Research Question
Following the issues raised in Sections 1 and 2, the following
research question emerges:
What is the dimensionality of a generic integrated conceptual
model of e-service quality, usability and user experience (UX) that
can be synthesized from literature, and that is appropriate as a
basis for the evaluation of web-based applications?
To answer this research question, the primary source of data used
in this study is the literature.
3.2 Procedure
In order to determine the dimensionality of e-SQUUX, Barbara
Kitchenham’s guidelines for a systematic literature review, were
followed [32]:
• There should be a review protocol that specifies the research
question and methods to be used.
• There should be a defined search strategy, so that readers can
assess its rigour and completeness. Where possible, this
should incorporate what is included and excluded.
• There should be a clear specification of the type of
information that is required.
• Qualitative or qualitative meta-analysis should be performed
in order to make a conclusion.
The research question is in Section 3.1 above. The methods used,
the search strategy applied for textual content analysis, and the
required information are explained in the next paragraphs.
In order to define, understand and model the three constructs,
various literature sources dealing with e-SQUUX in a number of
domains and sectors were consulted. The focus areas and domains
included software quality in general, web engineering, e-
commerce, e-government, e-retail, e-health, web services, web
portals, library services, the banking sector, travel, finance,
marketing, mobile services, telecommunication, manufacturing,
and IT. However, specialized web applications such as e-gaming
and e-learning, were excluded.
A spreadsheet was used to capture the data on the sources and
dimensions. Each source was assigned a unique number, named
Source#; the type of source was called Source Type, which was
Journal (J), Proceeding (P), Book (B), or website (W); the area/s
of e-SQUUX on which it focused was termed Area, which was e-
service quality (e-SQ), user experience (UX), usability (U) or a
combination of these. Further parameters were Year of
publication and surname of the first Author. Furthermore, natural
numbers, in sequence from 1 upwards, were automatically
assigned and captured as counters to each source as it was entered
in the spreadsheet. These details were captured as rows in the
spreadsheet. For each source, every dimension found in it was
captured in a column, and ‘1’ was inserted in the cell at the
intersection of that Row (Source) and column (Dimension). For
example, if row 20 listed Nielsen as a source and column 17
showed that Nielsen identified efficiency as a dimension, then ‘1’ experience model or dimensions’, etc. The second approach,
was inserted at the intersection of row 20 and column 17. Row recommended by Coursaris and Kim [12], was a document search
totals and column totals were calculated. This tallying process process whereby certain sources in the reference lists of the
determined the number of sources for each dimension and the identified sources, were in turn used as sources themselves. This
number of dimensions associated with each source. As with the was done recursively as the newly found sources were also
sources, dimensions were also numbered using natural numbers subjected to the same process. Some of the authors derived
from 1 upwards. dimensions of their own; others provided definitions that were
used as a basis of a specific study, while others adopted existing
Nearly all sources were acquired by using official electronic definitions. For sources that resulted in a set of dimensions using
databases available to the University of South Africa (Unisa) [63]. or referring to other sources, only the final set of dimensions was
Unisa subscribes to over 300 academic databases of electronic considered. As proposed by Ladhari [37], content analysis of
resources [63] for full-text access by its students and staff. They these sources was used to determine the dimensionality of e-
include well-known databases such as SCOPUS, ScienceDirect, SQUUX.
IEEE Xplore, ACM and Emerald. The main database used was
SCOPUS published by ELSEVIER, because “Scopus is the 4. DATA ANALYSIS AND FINDINGS
largest abstract and citation database of peer-reviewed literature: Altogether, 509 sources were identified in the search process.
scientific journals, books and conference proceedings” [17]. They were critically analysed for their potential relevance. Only
Moreover, the major journals which deal with issues in this study, those that had at least two dimensions of one or more of the
are indexed in SCOPUS. SCOPUS provides access to over 1.5 constructs of e-SQUUX were considered. Consequently, many
million subject-specific databases including ACM, ScienceDirect, sources were found to be unsuitable which left 264 deemed
Springer, IEEE Xplore, Emerald and CHI publications. The relevant for this study. Each source was analysed to determine the
databases also link to multi-disciplinary citations enhanced dimensionality of the components of e-SQUUX constructs. This
resources such as Web of Science. Google Scholar was also used, resulted in an initial set of 723 dimensions.
especially as a point of entry to access details regarding known
sources, and sometimes as the point of entry before searching for 4.1 Types, Number and Time Span of Sources
a full text article via the library resources. The books were mainly Table 2 shows that of the 264 sources reviewed, more than half
HCI texts. The websites were specifically from large private (51%), were academic journals, followed by Proceedings (39%),
corporations that had conducted research in any of the e-SQUUX published books (7%) and web-based sources (3%). The fact that
areas. They included Microsoft, Nokia, Samsung, Apple, Oracle 90% (51 + 39) were either academic journals or proceedings
and SAP. Many of these sources included information that had indicates the reputability and high credibility of the sources used.
been published in journals or conference proceedings. For
example, Prof. Virpi Roto and various colleagues have published Table 2: Type of sources reviewed
over 50 articles [57] in refereed scientific journals, and full papers # Source type Frequency %
in refereed conference proceedings on usability and UX, many of 1 Journals (J) 136 51
which were based on research at Nokia Research Centre. 2 Proceedings (P) 103 39
In line with Kitchenham [32], the search strategy is explained. 3 Books (B) 18 7
Two approaches were used for searching. The first was qualitative 4 Websites (W) 7 3
thematic analysis of text [43]. This was done partly with an Total 264 100
inverse approach, searching text for key terms appropriate to the Table 3 displays the number of sources for each area or
research question. From inspection of selected text, further combination of areas, along with the year in which each area was
pertinent themes and terms would emerge. Table 1 lists the main initially considered. Usability, as a single entity, was the most
terms used to search for relevant literature. The search was done dominant construct, occurring in 44% of all sources reviewed,
by using one or more words or themes from two or more fields 1 next was e-SQ with 24% and closely followed by UX with 20%.
to 5, for example, ‘web usability’, ‘user experience definition’, Eight percent (8%) of the sources dealt with a combination of UX
‘user experience and service quality’, ‘user experience and service and usability; 2% with all three areas, i.e. e-SQ, UX and usability;
quality attributes’, ‘web-based service quality’ and ‘user 2% with e-SQ and usability; and 1% with usability and e-SQ.
Table 1: Summary of key terms used to search for literate Table 3: Summary of numbers of sources in relevant areas
sources
Rank Area Number of % Oldest
Field 1 Field 2 Field Field 4 Field 5 sources source
3 1 U 116 44 1988
Web Usability And Model Dimensions 2 e-SQ 63 24 2000
Web- User Or Framework Attributes 3 UX 53 20 2000
based experience 4 UUX 21 8 2000
Internet UX for Definition Factors 5 e-SQUUX 5 2 2009
Internet- Service Standards Sub-factors 6 e-SQU 4 2 2000
based quality 7 e-SQUX 2 1 2008
Online e-service Guidelines Components Total 264 100%
quality It was observed that all the 63 articles that discussed e-SQ also
Electronic eService Heuristics Sub- mentioned usability, although the focus was on e-SQ. Similarly,
quality dimensions the 53 sources that discussed UX all, likewise, referred to
Website Checklist Criteria usability. The dominance of usability can be explained by the fact
Determinants that usability research has been conducted for considerably longer
than research in the other two areas, UX and e-SQ. The last
column of Table 3 shows the year of the earliest source used in
each case, demonstrating that usability originated in 1988 about
12 years before UX and e-SQ emerged in the literature of 2000.
This is due to the fact that usability studies were conducted since
the end of the Second World War, but only popularized in 1988
[26]. In comparison, UX and e-SQ have been intensively
researched since around 2000 [5, 52]. In general, the sources
acquired and reviewed spanned 27 years, from 1988 to the middle
of 2015.
Although the statistics in Table 3 relate to all three constructs, the
table shows that most sources, 88% or 232 (116 + 63+ 53) of 264,
address a single area of e-SQUUX. The focus of the study was in
the three areas of e-SQUUX, but several sources also discussed
other issues such as software quality (33 sources – not in table),
especially in relation to usability, and data quality (3 sources- not
in table). In addition, Djouab et al. [16] considered traditional SQ
as well as e-SQ. Another notable observation is that there were
very few studies, 1% (only 2 of 264), that briefly mentioned UX
and e-SQ, or all three constructs, 2% (5 of 264). This shows that
much more research is needed in integrating these three areas.
4.2 The Dimensions Reduction Cycle
The initial set of 723 dimensions from the 264 sources evolved
through rounds of reduction to filter and consolidate the textual
data. This is in line with Kitchenham’s [32] inclusion and
exclusion.
4.2.1 First round of reduction
In capturing the dimensions, the exact words from the sources
were used. For example, Security, Privacy, and Privacy/Security
were captured as different dimensions and tallied individually.
Likewise, Learnability and Learnable were considered to be
different. The 723 dimension were arranged alphabetically and
scrutinized further. Duplicate dimensions (identical
words/phrases) were merged and this resulted in 697 unique
dimensions.
4.2.2 Second round of reduction
The second round of reduction involved inspection of the 697
dimensions by the researchers, taking those that were deemed to
be similar in meaning, for example, Easy to learn, Ease of
learning, Learnability, Learnable, Time to learn, etc, in Table 4,
and merging them. Such sets of synonyms were consolidated
under one dimension by selecting the term/phrase with the highest
frequency in each set, in this case Learnability. The frequency of
that dimension was recorded as the sum of all the terms/phrases
that had been combined. For Learnability, its frequency was
changed from 43 to 57. The second round of reduction decreased
the total number of dimensions from 697 to 431.
Table 4: Dimensions directly related to the ‘Learnability’, as
originally captured
# Dimension Frequency
1 Ease of learning 6 under consideration, the main theoretical models used in this
2 Easy to learn 1
3 Learnability 43
4 Learnability and memorability 1
5 Learnability in use 2 and Use of Technology, namely, UTAUT2 [67]; and SERVQUAL
6 Learnable 1 [51], along with relevant ISO standards and other classic
7 Suitability of learning 2 literature.
8 Time to learn 1
Total 57
4.2.3 Third round of reduction – emergence of
categories
The third round of reduction reduced the 431 dimensions and
consolidated them into 70 categories.
First, the 431 dimensions were arranged in descending order of
frequency. They were then divided into two groups. The first
group, comprising 111, included the sets with frequencies of five
and above, and the remaining 320 formed the second group.
Although the cut-off point of 5 was subjective, the reason for
creating the first group was to ensure that all dimensions with a
fairly high frequency (five or more) would not be omitted during
the production of the final model.
The first group of 111 dimensions included terms that were
considered inappropriate to view as dimensions of e-SQUUX.
For example, many authors considered that usability and ease of
use were equivalent. Though these two dimensions had high
frequencies, 45 for usability and 32 for ease of use, they were
removed. The reason for removal was that, since usability is one
of the three key constructs at the top level of e-SQUUX, it should
not also appear at the lower level as a dimension. The remaining
dimensions were more critically analysed. Some which were
synonyms or closely related, were combined. In determining
which dimensions should be consolidated to form a new high-
level concept termed ‘categories’, the decision was based on the
primary researcher’s own knowledge in consultation with the
literature sources from which the items were extracted. For
example, ten sources out of 264 discussed security and privacy
together as a dimension of usability, of UX or of e-SQ. In these
cases, Security and Privacy were put in a single category,
Security/Privacy. Following this process Security had a frequency
of 18, Privacy had 11, and Security/Privacy also had 11.
According to the designated naming convention, security was
written before privacy since it was the dimension with the higher
frequency. The integration process described in this paragraph led
to a set of 70 categories, from which the ‘main dimensions’ were
extracted. This will be further explained in Section 4.2.4 and
Section 5.
The second stage of the third round involved taking the 320
dimensions in the second group and allocating them to the 70
categories. Before allocation, as was the case with the 111
dimensions in the first group, the 320 dimensions were refined,
eliminating 130 of them. The remaining 190 were allocated to the
70 categories. These 190 dimensions formed the basis of what
was later named the associated dimensions.
4.2.4 Fourth round of reduction
The fourth round of the reduction process further consolidated the
70 categories of dimensions to 24. In determining which
categories should be merged, the process was, once again, based
on the researchers’ knowledge in consultation with the literature.
To add rigour and, since there were now only 70 dimensions
study were primarily consulted. These were the three versions of
the Technology Acceptance Model, namely (TAM) [13], TAM2
[66] and TAM3 [65]; DeLone and McLean Models of IS Success
[14, 15]; the second version of the Unified Theory of Acceptance
In this way, categories with closely-related dimensions were
grouped together. This led to 24 categories, 75 main dimensions
and 163 associated dimensions. Each associated dimension is Appearance Presentation
related in meaning to one main dimension. At this stage there 10 Appeal and Appeal, Attractiveness, 98 5.1
were 75 main dimensions, although after the mergers in the third Attractiveness Exciting ,Stimulation
reduction round, there had been 70 categories from which main 11 Security and Security, Privacy 87 4.5
Privacy
dimensions were extracted. The increase occurred alongside
12 Assurance and Assurance, Credibility 85 4.4
mergers, due to cases where categories with more than one
Credibility
dimension were subdivided or converted to associated 13 Satisfaction Satisfaction, Attitude 74 3.8
dimensions. Where frequencies were lower than those of other
dimensions in the group, they were listed as associated 14 Accessibility Accessibility, Availability 65 3.4
dimensions. For example, in the category with Errors, Robustness, 15 Navigation Navigation, Searchability, 62 3.2
Recoverability, Failure and Mistakes, the dimensions of Findability
Recoverability, Failure and Mistakes became associated 16 Errors and Errors, Robustness 50 2.6
dimensions, while Errors and Robustness remained main Robustness
dimensions. 17 Maintainability Maintainability, Support 41 2.1
18 Relevance and Relevance, Suitability 41 2.1
5. THE e-SQUUX CONCEPTUAL MODEL Suitability
AND DISCUSSION 19 Consistency and Consistency, Innovativeness, 37 1.9
Innovativeness Novelty
Table 5 shows the 24 categories in descending order of frequency 20 Competence Competence 37 1.9
(f), and the corresponding 75 main dimensions. The frequency is
the sum of all dimensions in the main and associated dimensions. 21 Aesthetics Aesthetics, Colour 27 1.4
The 163 associated dimensions are not listed in this paper. The 22 Sociability and Sociability ,Collaboration, 27 1.4
table presents the model of e-SQUUX that answers the research Collaboration Communication
question in Section 3.1. 23 Timeliness Timeliness, Currency, Up-to- 21 1.1
datedness
The name/s of the main dimension/s in each category were used to 24 Motivating and Motivating, Challengeability 17 0.9
identify the category. Of the 24 categories, 14 of them were Challengeability
named using a combination of two terms. The main reasons for Total 1936 100
this are as follows: Firstly, in some cases the frequencies of the
The # column shows the rank of each category according to f, the
two, in terms of the number of sources in which they appeared,
frequency of the category. This means that the Learnability and
were so close that it was difficult to select one of them as the
Understandability category (f = 181) had the most frequently used
category name. Secondly, their meanings were related, but the use
dimensions in describing/characterising e-service quality,
of only one would not have captured the essence of the second
usability and user experience and that the Motivating and
one. A good case is Security and Privacy which are listed as
Challengeability category (f = 17) had the least.
Category 11 in Table 5. Alternatively, a completely different term
could have been used. However, the two reasons given at the The % column shows the corresponding percentage of each
beginning of this paragraph explain why the present approach was frequency out of the total frequency (f), which is 1936. This
adopted. percentage can be used as the relative weight of each category. It
can be calculated from the table that about 50% of the relative
Table 5: The 24 categories ranked according to frequency,
weight is due to the first seven categories of the model. Similarly,
main dimensions, combined frequency (f) and percentages
about 75% of the weight is due to the first 12 categories. This
(%)
means that the remaining 12 categories, half the total, make up a
# Category Main dimensions f % relative weight of only 25%, indicating that they are far less
1 Learnability Learnability, 181 9.3 important than the first 12 categories. These relative weight
and Understandability, figures can help an evaluator to make a decision on which
Understand- Memorability, Conciseness,
ability Readability, Clarity.
categories to apply in assessing a particular WBA.
2 Flexibility and Flexibility, Personalization, 145 7.5 The main dimensions (column 3) are listed, for each category, in
Personalization Empathy, Controllability, deceasing order of their frequencies. For example, in Category 1,
Adaptability, Portability,
Learnability and Understandability, the main dimensions are listed
3 Efficiency Efficiency, Simplicity, 143 7.4
Performance speed as: Learnability, Understandability, Memorability, Conciseness,
4 Pleasure and Pleasure, Hedonics, Emotion, 131 6.8 Readability and Clarity, where Learnability was the dimension
Hedonics Fun, Enjoyment, with the highest frequency, followed by Understandability, and so
Entertainment. on, down to Clarity which had the lowest frequency.
5 Responsiveness Responsiveness, 120 6.2 In this study, the categories are not assumed to be mutually
and Helpfulness Helpfulness, Interactivity,
Feedback,
exclusive, which differs from the case in certain hierarchical
Engageability, Fulfilment dimensions models of usability, UX or e-SQ, such as Seffah et al.
6 Reliability Reliability, Accuracy, 117 6.0 [61]. The same applies to main dimensions and the associated
Completeness dimensions. Due to space constraints, the associated dimensions,
7 Information Information quality, Data 116 6.0 which provide lower-level yet rich detail, are not presented here,
Quality quality, Content quality but are available from the authors on request.
8 Effectiveness Effectiveness, Usefulness, 107 5.5
and Usefulness Functionality
9 Interfaces Interfaces design, 107 5.5
Design and Appearance, Tangibility,
6. HOW TO APPLY THE e-SQUUX
MODEL IN EVALUATIONS
We believe the e-SQUUX model is ‘open’ in the sense that
stakeholders in both academia and industry can ‘make a choice’
on how best to use it in their personal contexts. Components can
be selected and customized to generate evaluation criteria for
particular domains and web environments. The implementation in
Figure 1 demonstrates how e-SQUUX can be the basis of a
framework for evaluating web-based applications. The
dimensions can also be used in design guidelines.
The aim of this work was to synthesize a model for use as a
framework to evaluate e-service quality, usability and user
experience in an integrated way. The e-SQUUX model comprises
four levels. Figure 1 distinguishes between the e-SQUUX
conceptual model in Levels 0 to 3 and the implementation of the
conceptual model in Level 4.
At Level 4, the implementation level, measurements in form of
metrics (objective measures) or scale items (subjective measures)
can be undertaken, as has been done in studies such as Seffah et
al. [61], Moumane et al. [46] and Kundu and Datta [35]. The
actual implementation at Level 4 can only be undertaken with
reference to Levels 1 to 3, which correspond to the categories,
main dimensions and associated dimensions respectively, hence
the three arrows to Level 4. The xx (see Figure 1) stands for the
varying number of metrics or items that can be generated
depending on the ‘choice’ of the model user and level/s
implemented.
The levels of the conceptual model are now explained:
Level 0: This is the actual conceptualization of the foundation of
e-SQUUX’s dimensionality.
e-Service quality,
usability and User
experience (e-SQUUX) Level 0
Conceptualisation
e-SQUUX Categories
Level 1
[24] categories
e-SQUUX Main
dimensions
Level 2
[75] Main dimensions
e-SQUUX Associated
dimensions Level 3
[163] Associated
dimensions
e-SQUUX measurement metrics/items
Level 4
[xx metrics/items]
Implementation
Figure 1: Implementation framework of the e-SQUUX
conceptual model.
Level 1 comprises categories of the e-SQUUX dimensions.
Twenty four categories were identified, which can be modified,
extended, increased to more categories, or reduced, hence the
square brackets for the flexible number of occurrences. If, for
instance, an organization wished to use some but not all of the 24
categories, the number would decrease. On the other hand, if a
researcher identified further dimensions, the number would
increase. ‘Openness’ is built into e-SQUUX because it has been
found that different industries or organizations have varying sets
of context- and user-specific critical web factors [47].
Level 1 can be applied at Level 4 (note the direct arrows) by
writing criteria in the form of evaluation items/statements for each
dimension, worded appropriately for the respective target group.
For Category 1, for example, the statement could be, “The site is
easy to learn/understand” or “The site is easy to learn and
understand” and for Category 3, it could be “The site
works/operates efficiently”. Using this approach, 24 statements
would be made using the e-SQUUX categories. One could also
apply the varying connotations of a category as they relate to a
website, say, Efficiency (Category 3) and write more than one
evaluation item for the category. In terms of statistical analysis,
factor analysis could be conducted on a related group of
statements. Whatever technique is used, the statements to evaluate
a specific web-based application could be delivered via a
questionnaire.
Level 2 provides the main dimensions related to the categories, 75
of them in this study. This level contains all the main dimensions,
including the one or two that were also used as category names.
As explained in the previous paragraph on categories, the
dimensions can be reduced or increased. There is an arrow from
Level 2 to Level 4 since the 75 main dimensions can similarly
result in 75 statements that can be used in a questionnaire. The
arrow from Level 2 to Level 3 represents the fact that, in some
cases, one or more associated dimensions are related to a
particular main dimension.
Level 3 consists of 163 associated dimensions linked to the main
dimensions. Due to space constraints, the associated dimensions
are not shown in Table 5, nor presented in this paper. Associated
dimensions do not include the 75 main dimensions of Level 2 and
are considered ‘weaker’ than the former when evaluating
websites. Main dimensions carry more weight and overall
meaning than associated dimensions. Nevertheless, associated
dimensions are valuable, since they encapsulate the detailed
meaning and scope in a richer way than the main dimensions. For
example, for the main dimension Understandability, the
associated dimensions are Wording, Language,
Comprehensibility, Interpretability, Terminology use, and Text
meaning, the combination of which can convey rich information.
In the next stage of this research, e-SQUUX has been
investigated, validated, and refined in an empirical study in the
form of an expert review. The resulting adjustments were not
major and are outside the scope of the present paper, which
focuses on the generation procedure of the original Model and
ways in which it can be implemented in the evaluation of WBAs.
7. CONTRIBUTIONS, LIMITATIONS,
FUTURE STUDIES AND CONCLUSION
7.1 Contribution
The model has both practical and theoretical benefits.
• The main contribution of the e-SQUUX model is its potential
use as the basis of a framework for evaluating e-service
quality, usability and user experience of web-based
applications.
• The model can be used by system designers, developers and
managers for building and evaluation of web applications,
such as web sites, in different sectors and domains.
• To our knowledge, this meta-analysis research is innovative
and original, and the first to provide an integrated view of the
constructs of e-service quality, usability and user experience.
• The open, integrated model can be applied in a flexible
manner. Practitioners and researchers can choose how to use
it, as explained in Section 6.
7.2 Limitations
This section relates to limitations of this study:
• Jiang [30] argues that a limitation of this kind of study is that
it does not differentiate between commercial profit-making
business websites and non-profit organizations such as
government enterprises and NGOs. Furthermore, according
to Hoffman and Krauss [24], different websites require
different dimensions depending on the purpose of the site
and context of use. For example, information-delivery sites
have different requirements from sites designed for casual or
social networking purposes.
• The above limitations can be overcome by the way of
implementing e-SQUUX in evaluations. As stated, the model
is generic, hence it not anticipated that all types of
organizations or researchers would use it in the same manner.
Stakeholders in academia and industry can ‘make a choice’
of categories and dimensions, and determine how best to
apply it in their personal contexts.
• Though numerical values were vital in the textual data
analysis, this study remains qualitative in nature. This is
mainly because there was subjectivity in some of the
techniques used. For example, which key words to use in
database searches, which databases to use, and which articles
to consider relevant? All these factors bring bias to the study.
In addition, there was also subjectivity in determining which
dimensions belong together. However, the authors are of the
opinion that the multi-stage approach improved the reliability
of the qualitative results and brought rigour to the findings.
• As mentioned in Section 5, the associated dimensions have
not been shown due to space constraints. These can be
provided by the authors on request.
• While e-SQUUX is suitable for the fixed computing devices
such as desktops and many mobile devices, it does not
include all the dimensions suitable for some of the mobile
devices that have inherent limitations such as small screen
size and low display resolution as discussed in Section 2.1
7.3 Future studies
Possible future studies include the following:
• The dimensions identified from the literature and hence the
structure of e-SQUUX will be reviewed in order to refine the
artefact in a design research approach. The first empirical
study has already been undertaken, namely an expert review
by a set of academic and practitioner experts in the HCI
sector. Second, an implementation of e-SQUUX (as
described in Section 6) will be used in an evaluation exercise
to test it in practice. Third, the final list of dimensions will be
used to generate a battery of items for inclusion in a
measuring scale that will be developed, tested and validated.
• A reduced version of e-SQUUX could be generated with less
categories and/or dimensions, but without diluting quality.
• A researcher could attempt to allocate the dimensions listed
in the model to the three key constructs of e-SQUUX. This
should include listing dimensions that belong to two or all
three constructs.
7.4 Conclusion
This paper set out to answer the following research question:
What is the dimensionality of a generic integrated conceptual
model of e-service quality, usability and user experience (UX) that
can be synthesized from literature, and that is appropriate as a
basis for the evaluation of web-based applications?
To this end, extant literature on the three constructs, namely, e-
service quality, usability and UX, was reviewed. Following a
protocol recommended by Kitchenham [32], 264 relevant sources
out of 509 acquired in the search process, were analysed. This
resulted in 697 unique dimensions. Through an iterative process
of reduction and evolution, an integrated view of the three
constructs was derived in the form of the generic e-SQUUX
model, comprising 24 categories, 75 main dimensions and 163
associated dimensions. This synthesized conceptual model (Table
5 in Section 5) provides the answer to the research question.
Furthermore, a possible implementation framework is shown and
discussed (Figure 1 in Section 6). Due to the open nature of the e-
SQUUX model, users can choose pertinent components and
instantiate them as the basis of a framework customized for
integrated evaluation of the usability, user experience, and e-
service quality of a web-based application in a specific context.
The contributions, limitations, and possible future studies of this
research are provided in Sections 7.1, 7.2 and 7.3, respectively.
8. References
[1] Abran, A., Khelifi, A., Suryn, W. and Seffah, A., 2003.
Usability meanings and interpretations in ISO
standards. Software Quality Journal, 11(4), pp.325-338.
[2] Agarwal, A. and Meyer, A., 2009, April. Beyond usability:
evaluating emotional response as an integral part of the user
experience. In CHI'09 Extended Abstracts on Human Factors
in Computing Systems (pp. 2919-2930). ACM.
[3] Al-Momani, K. and Noor, N.A.M., 2009. E-service quality,
ease of use, usability and enjoyment as antecedents of e-
CRM performance: an empirical investigation in Jordan
mobile phone services. The Asian Journal of Technology
Management, 2(2), pp.50-64.
[4] Al-Shamayleh, H.Z., Aljaafreh, R., Aljaafreh, A.,
Albadayneh, D., Al-Ali, M., Bazin, N.E.N., Shamsuddin,
S.M., Bamatraf, A., Latiff, M.S.B.A., Coulibaly, Y. and
Khasawneh, A.M., 2015. Measuring the quality of E-services
and its impact on students satisfaction at Jordanian
universities. Journal of Theoretical and Applied Information
Technology, 74(2).
[5] Barnes, S. and Vidgen, R.T., 2000. WebQual: an exploration
of website quality.ECIS 2000 Proceedings, p.74.
[6] Bevan, N., 2009, August. What is the difference between the
purpose of usability and user experience evaluation methods.
In Proceedings of the Workshop UXEM (Vol. 9, pp. 1-4).
[7] Bhattacharya, D., Gulla, U. and Gupta, M.P., 2012. E-service
quality model for Indian government portals: citizens'
perspective. Journal of Enterprise Information
Management, 25(3), pp.246-271.
[8] Bressolles, G. and Nantel, J., 2004, May. Electronic service
quality: a comparison of three measurement scales. In Proc.
of the 33 th EMAC Conference, Murcia, Spain.
[9] Carrasco, R.A., Sánchez-Fernández, J., Muñoz-Leiva, F.,
Blasco, M.F. and Herrera-Viedma, E., 2015. Evaluation of
the hotels e-services quality under the user’s experience. Soft
Computing, pp.1-17.
[10] Cebi, S., 2013. A quality evaluation model for the design
quality of online shopping websites. Electronic Commerce
Research and Applications, 12(2), pp.124-135.
[11] Chen, Y.H., Germain, C.A. and Rorissa, A., 2009. An
analysis of formally published usability and Web usability
definitions. Proceedings of the American Society for
Information Science and Technology, 46(1), pp.1-18.
[12] Coursaris, C. and Kim, D., 2006. A qualitative review of
empirical mobile usability studies. AMCIS 2006
Proceedings, p.352.
[13] Davis, F.D., 1989. Perceived usefulness, perceived ease of
use, and user acceptance of information technology. MIS
quarterly, pp.319-340.
[14] DeLone, W.H. and McLean, E.R., 1992. Information systems
success: The quest for the dependent variable. Information
systems research, 3(1), pp.60-95.
[15] Delone, W.H. and McLean, E.R., 2003. The DeLone and
McLean model of information systems success: a ten-year
update. Journal of management information systems, 19(4),
pp.9-30.
[16] Djouab, R., Abran, A. and Seffah, A., 2016. An ASPIRE-
based method for quality requirements identification from
business goals. Requirements Engineering, pp.1-20.
[17] ELSEVIER. 2015. About SCOPUS. Available:
https://www.elsevier.com/solutions/scopus. Accessed on 15
June 2015.
[18] Fowdur, T.P., Hurbungs, V. and Beeharry, Y., 2016, January.
Statistical analysis of energy consumption of mobile phones
for web-based applications in Mauritius. In 2016
International Conference on Computer Communication and
Informatics (ICCCI) (pp. 1-8). IEEE.
[19] Gumussoy, C.A., 2016. Usability guideline for banking
software design.Computers in Human Behavior, 62, pp.277-
285.
[20] Hassenzahl, M., 2008, September. User experience (UX):
towards an experiential perspective on product quality.
In Proceedings of the 20th International Conference of the
Association Francophone d'Interaction Homme-
Machine (pp. 11-15). ACM.
[21] Hassenzahl, M., Diefenbach, S. and Göritz, A., 2010. Needs,
affect, and interactive products–Facets of user
experience. Interacting with computers,22(5), pp.353-362.
[22] Hassenzahl, M. and Tractinsky, N., 2006. User experience-a
research agenda. Behaviour & information technology, 25(2),
pp.91-97.
[23] Hedegaard, S. and Simonsen, J.G., 2013, April. Extracting
usability and user experience information from online user
reviews. In Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems (pp. 2089-2098).
ACM.
[24] Hoffmann, R. and Krauss, K., 2004, October. A critical
evaluation of literature on visual aesthetics for the web.
In Proceedings of the 2004 annual research conference of
the South African institute of computer scientists and
information technologists on IT research in developing
countries (pp. 205-209). South African Institute for
Computer Scientists and Information Technologists.
[25] Holzinger, A., Searle, G., Kleinberger, T., Seffah, A. and
Javahery, H., 2008.Investigating usability metrics for the
design and development of applications for the elderly (pp.
98-105). Springer Berlin Heidelberg.
[26] Hornbaek, K. and Stage, J., 2006. The interplay between
usability evaluation and user interaction design. International
Journal of Human-Computer Interaction, 21(2), pp.117-123.
[27] Huang, S.J., Chen, W.C. and Chiu, P.Y., 2015, December.
Evaluation Process Model of the Software Product Quality
Levels. In Industrial Informatics-Computing Technology,
Intelligent Technology, Industrial Information Integration
(ICIICII), 2015 International Conference on (pp. 55-58).
IEEE.
[28] ISO, 2008. ISO CD 9241-210: Ergonomics of human-system
interaction-Part 210: Human-centred design process for
interactive systems. International Organization for
Standardization, Geneva.
[29] ISO/IEC, 2011. ISO/IEC 25010:2011, Systems and Software
Engineering - Systems and software Quality Requirements
and Evaluation (SQuaRE) - System and Software Quality
Models. International Organization for Standardization,
Geneva.
[30] Jiang, X. and JI, S., 2014, January. E-Government Web
Portal Adoption: A Service Level and Service Quality
Perspective. In System Sciences (HICSS), 2014 47th Hawaii
International Conference on (pp. 2179-2188). IEEE.
[31] Joo, S. and Yeon Lee, J., 2011. Measuring the usability of
academic digital libraries: Instrument development and
validation. The Electronic Library,29(4), pp.523-537.
[32] Kitchenham, B., 2004. Procedures for performing systematic
reviews. Keele, UK, Keele University, 33(2004), pp.1-26.
[33] Koohang, A., 2004. Expanding the concept of
usability. Informing Science, 7, pp.129-141.
[34] Kujala, S., Roto, V., Väänänen-Vainio-Mattila, K.,
Karapanos, E. and Sinnelä, A., 2011. UX Curve: A method
for evaluating long-term user experience. Interacting with
Computers, 23(5), pp.473-483.
[35] Kundu, S. and Datta, S.K., 2014. A Scale for Measuring
Internet Banking Service Quality: Literature Review and
Validation with Indian Public Sector Banks. Journal of
Electronic Commerce in Organizations (JECO), 12(3),
pp.12-39.
[36] Kuo, T., Lu, I.Y., Huang, C.H. and Wu, G.C., 2005.
Measuring users' perceived portal service quality: An
empirical study. Total quality management and business
excellence, 16(3), pp.309-320.
[37] Ladhari, R., 2010. Developing e-service quality scales: A
literature review.Journal of Retailing and Consumer
Services, 17(6), pp.464-477.
[38] Marimon, F., Vidgen, R., Barnes, S. and Cristóbal, E., 2010.
Purchasing behaviour in an online supermarket: The
applicability of ES-QUAL.International Journal of Market
Research, 52(1), pp.111-129.
[39] Matera, Maristella, et al. "The usability dimension in the
development of web applications." Handbook of Research on
Web Information Systems Quality(2008): 234.
[40] Mayhew, D.J., 2005. Cost justification of usability
engineering for international websites. Cost justifying
usability: An update for the Internet age, pp.359-384.
[41] Mazumder, F. and Das, U., 2014. Usability guidelines for
usable user interface. International Journal of Research in
Engineering and Technology,3(9), pp.79-82.
[42] McNamara, N. and Kirakowski, J., 2006. Functionality,
usability, and user experience: three areas of
concern. Interactions, 13(6), pp.26-28.
[43] Miles, M.B. and Huberman, A.M., 1994. Qualitative data
analysis: An expanded sourcebook. Sage Publications.
[44] Moczarny, I.M., De Villiers, M.R. and Van Biljon, J.A.,
2012, October. How can usability contribute to user
experience?: a study in the domain of e-commerce.
In Proceedings of the South African Institute for Computer
Scientists and Information Technologists Conference (pp.
216-225). ACM
[45] Montero, F., Lozano, M.D. and González, P., 2008.
Usability-oriented quality model based on ergonomic
criteria. Handbook of research on web information systems
quality, pp.220-233.
[46] Moumane, K., Idri, A. and Abran, A., 2016. Usability
evaluation of mobile applications using ISO 9241 and ISO
25062 standards. SpringerPlus, 5(1), p.1.
[47] Nathan, R.J. and Yeow, P.H., 2011. Crucial web usability
factors of 36 industries for students: a large-scale empirical
study. Electronic Commerce Research, 11(2), pp.151-180.
[48] Nielsen, J., 1993. Usability Engineering, Academic Press.
[49] Ojasalo, J., 2010. E-service quality: a conceptual
model. International Journal of Arts and Sciences, 7(3),
pp.127-143.
[50] Parasuraman, A., Zeithaml, V.A. and Berry, L.L., 1985. A
conceptual model of service quality and its implications for
future research. The Journal of Marketing, pp.41-50.
[51] Parasuraman, A., Zeithaml, V.A. and Berry, L.L., 1988.
Servqual. Journal of retailing, 64(1), pp.12-40.
[52] Parasuraman, A., Zeithaml, V.A. and Malhotra, A., 2005.
ES-QUAL a multiple-item scale for assessing electronic
service quality. Journal of service research, 7(3), pp.213-
233.
[53] Park, J., Han, S.H., Kim, H.K., Oh, S. and Moon, H., 2013.
Modeling user experience: A case study on a mobile
device. International Journal of Industrial
Ergonomics, 43(2), pp.187-196.
[54] Park, J., Han, S.H., Kim, H.K., Cho, Y. and Park, W., 2013.
Developing elements of user experience for mobile phones
and services: survey, interview, and observation
approaches. Human Factors and Ergonomics in
Manufacturing & Service Industries, 23(4), pp.279-293.
[55] Preece, J.R. and Rogers, Y., 2007. SHARP (2002):
Interaction Design: Beyond Human-Computer
Interaction. Crawfordsville: John Wiley and Sons, Inc.
Answers. com Technology.
[56] Pucillo, F. and Cascini, G., 2014. A framework for user
experience, needs and affordances. Design Studies, 35(2),
pp.160-179.
[57] Roto, V. 2015. All about UX. Available:
http://www.allaboutux.org/virpiroto. Accessed 15 March
2015.
[58] Roto, V., Rantavuo, H. and Väänänen-Vainio-Mattila, K.,
2009, October. Evaluating user experience of early product
concepts. In Proc. DPPI (Vol. 9, pp. 199-208).
[59] Sahi, G., 2015. User Satisfaction and Website Usability:
Exploring the Linkages in B2C E-Commerce Context. In IT
Convergence and Security (ICITCS), 2015 5th International
Conference on (pp. 1-4). IEEE.
[60] Sahi, G. and Madan, S., 2013. Assessing the Differential
Effect of Web Usability Dimensions on Perceived
Usefulness of a B2C E-Commerce Website. In Advances in
Information Technology (pp. 198-211). Springer
International Publishing.
[61] Seffah, A., Donyaee, M., Kline, R.B. and Padda, H.K., 2006.
Usability measurement and metrics: A consolidated
model. Software Quality Journal,14(2), pp.159-178.
[62] Strawderman, L. and Koubek, R., 2008. Human factors and
usability in service quality measurement. Human Factors
and Ergonomics in Manufacturing & Service
Industries, 18(4), pp.454-463.
[63] Unisa. 2015. Search library resources. Available:
http://www.unisa.ac.za/Default.asp?Cmd=ViewContent&Co
ntentID=95093. Accessed on 2 February 2015.
[64] Van Wyk, E. and de Villiers, R., 2008, October. Usability
context analysis for virtual reality training in South African
mines. In Proceedings of the 2008 annual research
conference of the South African Institute of Computer
Scientists and Information Technologists on IT research in
developing countries: riding the wave of technology (pp.
276-285). ACM.
[65] Venkatesh, V. and Bala, H., 2008. Technology acceptance
model 3 and a research agenda on interventions. Decision
sciences, 39(2), pp.273-315.
[66] Venkatesh, V. and Davis, F.D., 2000. A theoretical extension
of the technology acceptance model: Four longitudinal field
studies. Management science, 46(2), pp.186-204.
[67] Venkatesh, V., Thong, J.Y. and Xu, X., 2012. Consumer
acceptance and use of information technology: extending the
unified theory of acceptance and use of technology. MIS
quarterly, 36(1), pp.157-178.
[68] Vermeeren, A.P., Roto, V. and Väänänen, K., 2015. Design-
inclusive UX research: design as a part of doing user
experience research. Behaviour & Information Technology,
pp.1-17.
[69] Winter, S., Wagner, S. and Deissenboeck, F., 2008. A
comprehensive model of usability. In Engineering interactive
systems (pp. 106-122). Springer Berlin Heidelberg.
[70] Zeithaml, V.A., Parasuraman, A. and Malhotra, A., 2002.
Service quality delivery through websites: a critical review of
extant knowledge. Journal of the academy of marketing
science, 30(4), pp. 362-375.