2012 31st International Conference of the Chilean Computer Science Society
Evaluating a Methodology To Establish Usability Heuristics
Cristhy Jiménez1, Cristian Rusu1, Silvana Roncagliolo1, Rodolfo Inostroza1, Virginica Rusu2
1
Pontificia Universidad Católica de Valparaíso
2
Universidad de Playa Ancha
Valparaíso, Chile
cristhy.jimenez.g@mail.pucv.cl, cristian.rusu@ucv.cl, silvana@ucv.cl, info@randomstudio.com,
virginica.rusu@upla.cl
Abstract— Assessing usability in any software product may be
a key factor for predicting its success or fail. Heuristic
evaluation is the most commonly used usability evaluation
method. It uses a set of recognized usability design principles
(heuristics). Until now, the Nielsen’s ten usability heuristics
have been widely used. However, such heuristics are too
general and currently it is necessary to provide new sets of
heuristics for evaluating specific kinds of applications.
Through a survey, applied to a group of researchers, it was
possible to analyze the pertinence of formalizing a
methodology for establishing specific usability heuristics that
could improve usability assessments.
Keywords- heuristics, heuristic evaluation, methodology,
usability
I. INTRODUCTION
Usability is a quality attribute that assesses how easy to use
are user interfaces. The word "usability" also refers to
methods for improving easiness-of-use during the design
process [1]. Usability evaluation is one of the most important
stages in the user-centered design process. There are many
methods for determining the usability level of an interactive
software system; usability heuristic evaluation is the most
commonly used method. The main characteristic of the
heuristic evaluation is that evaluators analyze the interface of a
system based on a set of usability design principles (heuristics).
There are different sets of usability heuristics; some of them are
too general, but others were developed taking into account
specific characteristics of the systems. Literature has showed
that most of those heuristics have been developed as an
extension and/or particularization of Nielsen’s ones. Recently,
a methodology for establishing usability heuristics was
proposed. This paper summarizes the results of an experiment
conducted in order to analyze the utility and easiness of
application of the methodology. Section II presents an
introduction to usability evaluation. Section III summarizes the
proposal of methodology for establishing usability heuristics.
Section IV presents the process of validation of the
methodology, specifying the participants, instrument and the
results of the experiment. Section V contains a discussion
regarding the main obtained results. Finally, section VI
presents some conclusions and the future work.
II. WHAT IS USABILITY?
The standard ISO 9241, describes usability as “the
extent to which a product can be used by specified users to
1522-4902/13 $31.00 © 2013 IEEE 51
DOI 10.1109/SCCC.2012.14
achieve specified goals with effectiveness, efficiency and
satisfaction in a specified context of use” [2]. Other
accepted and related definition of usability comes from the
standard ISO/IEC 9126 that defines usability as “the
capability of the software product to be understood, learned,
used and attractive to the user, when used under specified
conditions” [3]. Nowadays, usability is considered a key
quality attribute which directly influences the product’s
success or failure. Although usability is considered a purely
subjective characteristic, according to Nielsen [4] there are
five attributes which allow quantifying the usability of an
interactive software product: Learnability, Efficiency,
Memorability, Errors and Satisfaction.
Usability evaluations allow detecting and correcting
specific usability problems in order to improve users’
interaction. According to Lorés [5] “The Usability
evaluation includes a set of methodologies and techniques
for analyzing the usability of an interactive system in the
different stages of the life cycle”. There are different
usability evaluation methods that have been proposed and
classified by several authors. Holzinger [6] categorized the
usability evaluation methods in two categories: (1)
inspections and (2) tests.
Heuristic evaluation is one of the most commonly used
usability inspection methods. This method consists of a
group of experts who systematically analyze the interface of
a system, taking into account a set of well-known usability
design principles (heuristics) [4]. Nielsen [7] proposed a set
of 10 general usability heuristics that are widely used in this
kind of evaluation. However, although the application of the
method is simple, sometimes usability problems are not so
easy to be discovered; due to that Nielsen’s heuristics are
too general for evaluating systems with very specific
characteristics.
III. A METHODOLOGY TO ESTABLISH USABILITY
HEURISTICS
Many research works are focused on to provide different
sets of usability heuristics for evaluating specific kinds of
interactive systems. However, there is no evidence that a
formal process or methodology had been used in order to
establish such sets of heuristics. Rusu and others [8] propose
a set of six stages as an approximation to a methodology for
developing specific usability heuristics that could be used
for evaluating particular software systems. The six stages of
the methodology are: (1) Exploratory, (2) Descriptive, (3) application of the methodology in general and also
Correlational, (4) Explicative, (5) Experimental validation regarding the application of its individual stages. Both open
and (6) Refinement. Table I summarizes the stages of the and closed questions were included. Closed questions were
methodology. graded in a five levels Likert scale in order to quantify the
participant experience. The qualitative labels used for closed
TABLE I
questions are showed in Table II. Open questions were
METHODOLOGY FOR DEVELOPING USABILITY HEURISTICS established in order to allow the participants justify the
closed answers, and also for collecting additional
Stage Description
information about the methodology.
For collecting bibliography regarding specific TABLE II
Exploratory topic of study, including general or related VALUE SCALE OF QUALITATIVE LABELS FOR CLOSED QUESTIONS
features. (if there are some)
1 2 3 4 5
For highlighting the most relevant characteristics
Descriptive of the previously collected information, in order Very
Very easy Easy Neutral Difficult
to formalize the main concepts associated with Difficult
the topic of study.
Equal
For identifying the characteristics that heuristics Much more More Less Much less
number of
for specific applications should have, taking into problems problems problems problems
Correlational problems
account traditional heuristics and the analysis of
cases of study.
Very Very
Specific Neutral General
For formally specifying the set of heuristics, specific general
Explicative
using a standard template (See Table II).
Very
Experimental Very useful Useful Neutral Useless
For checking new heuristics against traditional useless
Validation (Nielsen’s) heuristics by experiments.
Moderate Nothing
A lot of Very little
amount of Neutral about the
For refining the heuristics in base of the feedback information information
Refinement information topic
obtained through the validation stage.

The survey was basically divided in two sections. First

IV. EVALUATING THE METHODOLOGY
Until now, the methodology has been applied in some
research works, in order to establish different sets of specific
usability heuristics. Authors of such works have reported the
advantages of using the particular set of heuristics against
traditional Nielsen’s ones. However, in order to assess if the
methodology is easy to apply and to identify aspects that
allow improving it; an experiment using inquiry techniques
was conducted.
A. Participants
The experiment included the participation of five
researchers who previously applied the methodology for
establishing heuristics to: Grid Computing, Interactive
Television, Virtual Worlds, and Touchscreen Mobile
Devices. The previously mentioned works were already
published in [9][10][11][12]. One author of each work
participated in the experiment and, two authors from two
independent researches for the case of Grid Computing.
B. The Survey
In order to analyze the formalization, easiness of
application and utility of the methodology, a survey was
constructed. The survey was focused on to capture the
individual experiences of the participants regarding the
section included questions which were designed for
capturing the perception of the researchers regarding each
individual stage of the methodology. Second section
included questions focused on to collect information
regarding the utility and easiness of application of the
methodology in general, as well its advantages and
disadvantages.
Regarding the first section of the survey, different sets of
questions were established for collecting information about
each individual stage of the methodology. The questions
used in this section are summarized in Table III. The
easiness of application of the stages was inquired using the
scale Very easy to Very difficult (See Table II). As it can be
seen, the researcher was able to complement his perception,
through the open question Why?
Table III also shows other different criteria inquired for
each stage of the methodology. Thus, the amount of
available information related to the topic of study was an
investigated criterion related with the Exploratory stage.
The quantity of cases of study used in the Correlational
stage was also collected in order to analyze the experimental
information available for each topic of study. In Explicative
stage, it was necessary to know the number of usability
heuristics that the researcher was able to define. The

52
Experimental Validation stage was the one which required Second section of the survey also included open and
more diverse information. Thus, it was necessary to know closed questions regarding the methodology in general. A
the number of cases of study that the researcher used for summary of those questions is presented in Table IV. The
validating the new set of heuristics, the types of usability main goal of this section was to capture the perception of
evaluations used in the experiments, and the number and the researchers in regard to the easiness of application and
specificity of the problems identified by using traditional utility of the methodology, and also some perceived
and new heuristics. advantages and disadvantages.
TABLE III
QUESTIONS DESIGNED FOR COLLECTING INFORMATION REGARDING EACH INDIVIDUAL STAGE OF THE METHODOLOGY
Complementary
Stage Inquired Criteria Measure
open question
Exploratory

a) Easiness of application Likert scale [Very easy - Very difficult] Why?

b) Information related to the topic of study Likert scale [A lot of information - Nothing about the topic]
Descriptive

a) Easiness of application Likert scale [Very easy - Very difficult] Why?

Correlational

a) Easiness of application Likert scale [Very easy - Very difficult] Why?

b) Number of cases of study used. Numeric value

Explicative

a) Easiness of application Likert scale [Very easy - Very difficult] Why?

b) Number of defined heuristics. Numeric value

a) Easiness of application Likert scale [Very easy - Very difficult] Why?

b) Number of cases of study used. Numeric value

Experimental validation

c) Types of usability evaluations applied. Heuristic Evaluation / User Tests / Both

d) Comparison of amount of problems using Likert scale [Much more problems - Much less problems]
new heuristics Vs. Traditional heuristics.

e) Number of problems using traditional Numeric Value

heuristics (Nielsen’s)

f) Number of problems using new heuristics Numeric Value

g) Specificity of problems identified through Likert scale [Very specific - Very general]
new heuristics
Refinement

a) Easiness of application Likert scale [Very easy - Very difficult] Why?

53
 TABLE IV At the moment in which the experiment was performed
QUESTIONS DESIGNED FOR COLLECTING INFORMATION REGARDING
THE METHODOLOGY IN GENERAL
all researchers had previously used the methodology or at
least they were still using it.
Inquired Criteria Measure In general terms, the methodology was perceived by the
Easiness of application Likert scale [Very easy - Very difficult] participants with a medium-high level of easiness of
application. However, the individual stages were graded
Utility of the methodology Likert scale [Very useful – Very useless] with a lower level of facility of application. For participants,
the Exploratory stage was especially difficult to be
Advantages Open question
conducted due to the lack of information related to their
Disadvantages Open question emerging and relatively new cases of study.
Table V shows a summary of the quantitative answers
regarding the easiness for applying the individual stages of
V. THE RESULTS the methodology. Table VI presents the quantitative
The experiment included the participation of five answers regarding the easiness of application and utility of
researchers of the following four categories of applications: the methodology in general. Table VII shows the number of
cases of study used in both, the Correlational and
- Grid Computing (GC1, CG2) Experimental Validation stages. Finally, Figure I depicts the
- Interactive Television (IT) quantity of usability problems identified using both,
traditional usability heuristics (Nielsen’s) and specific
- Virtual Worlds (VW) usability heuristics in each case of study.
- Touchscreen Mobile Devices (TMD)

TABLE V
QUANTITATIVE ANSWERS REGARDING THE EASINESS OF APPLICATION OF THE METHODOLOGY INDIVIDUAL STAGES

Standard
VW IT TMD CG 1 CG 2 Average
Deviation

Exploratory 3 4 3 1 2 2,40 0,89

Descriptive 4 3 4 2 3 3,40 0,89

Correlational 4 2 3 3 3 3,40 0,55

Stage

Explicative 4 3 3 1 3 3,00 1,22

Experimental
5 3 2 4 3 3,80 1,30
Validation

Refinement 2 4 4 4 3 3,00 1,00

Average 3,7 3,2 3,2 2,5 2,8 3,2

TABLE VI
QUANTITATIVE ANSWERS REGARDING THE EASINESS OF APPLICATION AND UTILITY OF THE METHODOLOGY

VW IT TMD CG 1 CG 2 Average

Easiness of Application 4 3 4 4 2 3,40

Utility 5 4 5 5 4 4,60

54
 TABLE VII
AMOUNT OF CASES OF STUDY USED DURING CORRELATIONAL AND EXPERIMENTAL VALIDATION STAGES
VW IT
Correlational 10 3 3
Experimental
2 3
Validation
Figure 1. Number Of Usability Problems Identified Using Nielsen’s And Specific Heuristics.
Following sections present a summary of the obtained
answers in each one of the topics of study which were
included into the experiment. Then, an analysis of the
answers will be presented in order to get a general point of
view regarding the methodology.
A. Case 1. Virtual worlds (VW)
1) Easiness of application of the individual stages.
Taking into account the answers of the researcher
regarding the easiness for applying each stage (See Table
VI), it is possible to conclude that the stages of the
methodology were perceived as easy to be applied. In
average, the easiness of application of the stages reached a
value of 3.7 into a 1 to 5 points scale.
For the researcher of this case of study, the stage most
difficult to apply was the Refinement one. The researcher
literally said: “there is a lack of information about how to
conduct the refinement process of heuristics”.
2) Number of cases of study.
The researcher used a total of 10 cases of study during
Correlational stage, and 2 cases of study during
Experimental Validation stage.
3) Quantity of specific heuristics developed through the
methodology.
The researcher achieved to define a set of 16 usability
55
TMD CG 1 CG 2
2 3
1 2 3
heuristics for evaluating virtual-worlds applications.
Furthermore, researcher stated that “for him the heuristics
template was easy to fill because he had used previous
related works as reference”.
4) Kind of usability evaluations used during
Experimental Validation stage.
The researcher used both heuristic evaluations and user
tests for testing the developed heuristics.
5) Number of identified usability problems.
The researcher stated that through the use of the
developed heuristics, it was possible to identify much more
usability problems than using traditional Nielsen’s ones. In
Figure 1 it is possible to see that the number of problems
identified with the specific heuristics is almost double of
those identified with Nielsen’s heuristics. Furthermore, the
researcher highlighted that the problems identified through
the new set of heuristics were much more specific for his
topic of study.
6) Easiness of application and utility of the
methodology.
The researcher stated that in general, the methodology
was easy to be applied and very useful for developing
specific usability heuristics (See Table VII). Comparing this
perception with the average value (3.7) of the easiness of
individual stages application (See Table VI), it is possible to
note that there is no a significant difference between the
individual and global perceptions of the methodology.
7) Advantages and disadvantages of the methodology.
The researcher pointed that “the main advantage of the
methodology is that it provides a systematic workflow that
guides the search of heuristics”. Furthermore, the researcher
remarked as disadvantage “the lack of information about the
iterative process of the methodology and how can it be
documented”.
B. Case 2. Interactive Television (IT)
1) Easiness of application of the individual stages.
Taking into account the answers of the researcher
regarding the easiness of each stage (See Table VI), it is
possible to conclude that in average the stages were
perceived with a neutral easiness of application (3.2 points
into a 1 to 5 points scale).
The stage that presented higher difficulty to be applied
was the Correlational one. The researcher literally stated
that “it is necessary performing the 2 previous stages
correctly in order to successfully complete the Correlational
stage”.
2) Number of cases of study.
The researcher used a total of 3 cases of study for both
Correlational stage and Experimental Validation stage.
3) Quantity of specific heuristics developed through the
methodology.
The researcher achieved to define a set of 14 usability
heuristics for evaluating interactive television applications.
Furthermore, researcher stated that “the specification of
heuristics through the template is a no trivial process,
especially for those heuristics that are not a merely
adaptation of another ones”.
4) Kind of usability evaluations used during
Experimental Validation stage.
The researcher used both heuristic evaluations and user
tests for testing the developed heuristics.
5) Number of identified usability problems.
The researcher stated that through the use of the
developed heuristics, it was possible to identify more
usability problems than using traditional Nielsen’s ones.
Moreover, the researcher emphasized that the problems
identified through the new set of heuristics were more
specific for his topic of study.
6) Easiness of application and utility of the
methodology.
The researcher said that in general, the methodology was
neutrally easy to be applied and very useful for developing
specific usability heuristics (See Table VII). Comparing this
56
perception with the average value (3.2) of the easiness of
individual stages application (See Table VI), it is possible to
see that there is no a significant difference between the
individual and global perceptions of the methodology.
7) Advantages and disadvantages of the methodology.
The researcher pointed the following advantages of the
methodology:
- The methodology provides a set of systematic steps
for developing specific usability heuristics.
- The information that the template of heuristics
contains is quite complete for documenting the
heuristics.
- The methodology is a significant contribution due to
there is no similar or related guide for developing
usability heuristics.
- The methodology does not require complex or costly
resources for conducting it. It is enough to have
internet connection and access of an application as
case of study.
Finally, the researcher did not state any disadvantage of
the methodology.
C. Case 3. Touchscreen Mobile Devices (TMD)
1) Easiness of application of the individual stages.
Considering the answers of the researcher in regard to the
easiness for applying each individual stage (See Table VI), it
is possible to conclude that the methodology was perceived
as neutrally easy to be applied. In average, the easiness of
application of the stages reached a value of 3.2 into a 1 to 5
points scale.
For the researcher of this case of study, the stage most
difficult to apply was the Experimental Validation stage.
The researcher said that “the coordination of the evaluators
and the interpretation of new heuristics is a complicated
task”.
2) Number of cases of study.
The researcher used 3 cases of study during Correlational
stage and only 1 case of study during Experimental
Validation stage.
3) Quantity of specific heuristics developed through the
methodology.
The researcher was able to define a set of 11 usability
heuristics for evaluating touchscreen mobile devices
applications. Moreover, the researcher stated that “although
the template is quite good specified and is simple to use; the
redaction of the information is a non-trivial task, especially
for new usability heuristics”.
 4) Kind of usability evaluations used during
Experimental Validation stage.
The researcher said that only heuristic evaluations were
used in the Experimental Validation stage. However, he
remarked that Nielsen’s protocol for heuristic evaluation
was not completely followed. Technically, the applied
evaluations were guided usability inspections.
5) Number of identified usability problems.
The researcher stated that through the use of the
developed heuristics, it was possible to identify more
usability problems than using traditional Nielsen’s ones (See
Figure 1). Furthermore, the researcher highlighted that the
problems identified through the new set of heuristics were
specific for his particular topic of study.
6) Easiness of application and utility of the
methodology.
The researcher said that in general, the methodology was
easy to be applied and very useful for developing specific
usability heuristics (See Table VII). Comparing this
perception with the average value (3.2) of the easiness for
applying the individual stages (See Table VI), it is possible
to note that there is no a significant difference between the
individual and global perceptions of the methodology.
7) Advantages and disadvantages of the methodology.
The researcher pointed as main advantages of the
methodology the following features: clarity, specificity, and
flexibility. Additionally, the researcher remarked as
disadvantage “the lack of validation of the methodology”.
D. Case 4. Grid Computing (GC 1)
1) Easiness of application of the individual stages.
Taking into account the answers of the researcher
regarding the easiness of each stage (See Table VI), it is
possible to conclude that in average the stages were
perceived as difficult to be applied (2.5 points into a 1 to 5
points scale).
The stages that presented greater difficulty to be applied
were the Exploratory and Explicative. On one hand, the
researcher stated that “the lack of information regarding his
topic of study hindered the exploratory work”. On the other
hand, it was especially difficult for the researcher the
process of filling the heuristics template.
2) Number of cases of study.
The researcher used a total of 2 cases of study for both
Correlational stage and Experimental Validation stage.
3) Quantity of specific heuristics developed through the
methodology.
The researcher was able to define a set of 12 usability
heuristics for evaluating grid computing applications.
Additionally, the researcher stated that “the process of
filling the template of heuristics was a complex task”.
57
4) Kind of usability evaluations used during
Experimental Validation stage.
The researcher used both heuristic evaluations and user
tests for testing the developed heuristics.
5) Number of identified usability problems.
The researcher said that through the use of the developed
heuristics, it was possible to identify more usability
problems than using traditional Nielsen’s ones. Moreover,
the researcher emphasized that the problems identified
through the new set of heuristics were more specific for his
topic of study.
6) Easiness of application and utility of the
methodology.
The researcher commented that in general, the
methodology was easy to be applied and very useful for
developing specific usability heuristics (See Table VII).
Comparing this perception with the average value (2.5) of
the easiness of individual stages application (See Table VI),
it is possible to see that there is a significant difference
between the individual and global perceptions of the
methodology. On one hand, the researcher considers the
methodology as easy to be applied; however the values
assigned to each stage allow concluding that for this
researcher, the methodology was difficult to be applied.
This fact can be justified taking into account that this
researcher was the pioneer in apply the methodology, and he
had no related information about how to use it.
7) Advantages and disadvantages of the methodology.
The researcher pointed the following advantages of the
methodology:
- The methodology provides a set of systematic steps
for conducting an organized work.
- The methodology takes advantage of iterative
processes.
- The methodology provides flexibility.
Finally, the researcher stated the following disadvantages
of the methodology:
- It is difficult to predict the needed time for applying
each stage.
- The first stage is very general and it needs to be more
specific.
E. Case 5. Grid Computing (GC 2)
1) Easiness of application of the individual stages.
Taking into account the answers of the researcher
regarding the easiness of each stage (See Table VI), it is
possible to conclude that in average the stages were
perceived with a neutral easiness of application (2.8 points
into a 1 to 5 points scale).
The stage that presented higher difficulty to be applied was
the Exploratory one. The researcher literally stated that “the
main difficult was the lack of information regarding the
topic of study”.
2) Number of cases of study.
The researcher used a total of 3 cases of study for both
Correlational stage and Experimental Validation stage.
3) Quantity of specific heuristics developed through the
methodology.
The researcher was able to define a set of 13 usability
heuristics for evaluating grid computing applications.
Furthermore, researcher stated that “the specification of
heuristics through the template was an easy task due to the
previous existent work of the same topic of study”.
4) Kind of usability evaluations used during
Experimental Validation stage.
The researcher used only heuristic evaluations for testing
the developed heuristics.
5) Number of identified usability problems.
The researcher pointed that through the use of the
developed heuristics, it was possible to identify a similar
quantity of usability problems than using traditional
Nielsen’s ones. However, in Figure 1 it is possible to see
that much more problems were discovered using the new
set of heuristics. Moreover, the researcher emphasized that
the problems identified through the new set of heuristics
were more specific for the topic of study.
6) Easiness of application and utility of the
methodology.
The researcher said that in general, the methodology was
difficult to be applied but useful for developing specific
usability heuristics (See Table VII). Comparing this
perception with the average value (2.5) of the easiness of
individual stages application (See Table VI), it is possible to
see that there is no a significant difference between the
individual and global perceptions of the methodology.
7) Advantages and disadvantages of the methodology.
The researcher pointed as main advantage that “the
methodology allows getting an effective set of usability
heuristics for improving the results of heuristics
evaluations”. Finally, the researcher stated as disadvantage
“the amount of time that its application implies and the lack
of automation of its stages”.
VI. DISCUSISON OF THE RESULTS
After to analyze and summarize the collected information
through the surveys, this section presents a discussion of the
main results obtained in this experiment.
Figure 2 depicts that most of participants perceived the
methodology with a neutral level of easiness of application.
58
Figure 3 shows that individual stages were also perceived as
easy to apply for most of researchers. The Exploratory
stage presented greater difficulty to be applied. In fact, one
researcher graded exploratory stage as very difficult to be
applied. The main reason for assigning this value was
justified in the lack of information regarding of the topic of
study.
As it can see in Figure 2, the researchers who studied
Grid Computing applications (GC1 and GC2) were those
who perceived the methodology with a lower level of
easiness of application. On one hand, in the case of CG1,
the researcher did not apply the methodology as such, but
his work was the starting point for the development of the
methodology. On the other hand, due the complexity of the
topic of study and the lack of information regarding it, both
CG1 and CG2 researchers perceived more difficulty for
applying certain stages of the methodology and hence for
developing the new set of heuristics.
Regarding the utility of the methodology, the results
demonstrated that new specific heuristics are more effective
than Nielsen’s ones, for discovering usability problems. The
100% of participants evidenced a high increment in the
quantity of discovered usability problems when they used
the set of specific heuristics.
From the analysis of individual commentaries of the
participants, it was possible to identify some aspects to be
taking into account in order to improve the methodology.
According to the participants of the experiment, it is
necessary to complement the methodology through more
specificity in topics such as: needed time, minimum number
of cases of study, documentation process and criteria for
going back to previous stages. Finally, the use of a standard
template for describing the established heuristics was
positively remarked.
Figure 2. Easiness of Application of the Methodology.

Figure 3. Easiness of Application of Each Stage of the Methodology.
VII. CONCLUSIONS
Through this work it was possible to validate the
pertinence for formalizing and applying a methodology for
establishing usability heuristics. The validation experiment
included the design of a survey that was applied to a group
of 5 researchers who had previously used the methodology.
The analysis of the results allowed establishing that in
general the methodology was perceived as neutrally easy to
apply. However, it was possible to detect the lack of
specificity in some stages of the methodology. The future
work will be focused on to apply the methodology in others
cases of study, and progressively analyze the pertinence of
adding more specificity in each one of the stages.
59
REFERENCES
[1] Otaiza, R., Rusu, C. and Roncagliolo, S. Evaluating the usability of
transactional Web Sites. In Proceedings of the Advances in
Computer-Human Interactions, 2010.
[2] International Standard ISO 9241, “Ergonomic requirements for office
work with visual display terminals (parts 1 to 17),” 1993.
[3] International Standard ISO/IEC 9126. Software engineering-Product
Quality. 2001.
[4] J. Nielsen, “Usability engineering,” Academic Press, San Diego, CA
92101-4495 USA, 2003.
[5] J. Lorés, T. Granollers, and S. Lana, “Introducción a la Interacción
Persona Ordenador.” Universidad de Lleida, 2002.
[6] A. Holzinger, “Usability Engineering Methods for Software
Developers,” Commun. ACM48, 2005.
[7] Nielsen, J. “Ten Usability Heuristics”. 2005. DOI=
http://www.useit.com/papers/heuristic/heuristic_list.html
[8] C. Rusu, S. Roncagliolo, V. Rusu, and C. Collazos, “A Methodology
to Establish Usability Heuristics,” Proc. 4th International Conferences
on Advances in Computer-Human Interactions (ACHI 2011), 2011.
[9] Roncagliolo, S., Rusu, V., Rusu, C., Tapia, G., Hayvar, D. and
Gorgan, D. “Grid Computing Usability Heuristics in Practice," In
Proceedings of Information Technology: New Generations (ITNG),
2011.
[10] Muñoz, R. and Rusu,. “Virtual Worlds: Real Usability?”. In
proceedings of The V Latin American Conference on Human
Computer Interaction 2011 – CLIHC 2011.
[11] Inostroza, R. and Rusu, C. and Roncagliolo, S. and Jimenez, C. and
Rusu, V. 2012. Usability Heuristics for Touchscreen-based Mobile
Devices. Proc. 9th International Conference on Information
Technology: New Generations (ITNG 2012), Las Vegas, Nevada,
USA. IEEE Computer Society, pp. 662–667.
[12] Solano, A., Rusu, C., Collazos, C., Roncagliolo, S., Arciniegas, J. L.
and Rusu, V. “Usability Heuristics for Interactive Digital Television,”
In Proceedings of The Third International Conference on Advances in
Future Internet, Saint Laurent du Var, France, 2011.