2014 International Conference on Information Technology and Multimedia (ICIMU), November 18 – 20, 2014, Putrajaya, Malaysia
Evaluating Mobile Banking Application: Usability
Dimensions and Measurements
Azham Hussain Hamisu Ibrahim Abubakar
School of computing School of computing
University Utara Malaysia University Utara Malaysia
06010 UUM Sintok, Kedah. 06010 UUM Sintok, Kedah.
azham.h@uum.edu.my haibal06@yahoo.com
Abstract - Usability has greatly been considered as one of the
significant quality attributes to determine the success of mobile
application. Mobile banking application is increasingly
recognized as an emergent m-commerce application which
dignified to become the giant killer mobile application arena.
However, prominent usability evaluation models for mobile
applications are too general and do not adequately capture the
complexities of interacting with m-banking application platform.
Similarly, there are no sufficient descriptions concerning the
relationship between phases and appropriate usability measures
for a specific application. Some banks do not offer m-banking
application, while those that offer have inadequate functionalities
and this shows that their interfaces are still insufficient and not
user friendly. To date, usability and measurements for m-
banking application in particular is very limited or even isolated
and this makes usability evaluation of m-banking more of
challenging. Consequently, this report proposes to address this
matter by proposing a suitable and an appropriate set of usability
dimensions and measurements for m-banking evaluation. The
systematic literature review was employed to review relevant
journals and conference proceedings. Seven hundred and eight
papers were downloaded but merely forty nine papers have been
selected and fully reviewed/analysed. Five usability dimensions
and twelve relevant criteria (sub-dimensions) have been created
that can be used to evaluate m-banking application.
Keywords: Mobile banking, usability evaluation, usability
dimensions, evaluation
I. INTRODUCTION
Usability of a product is normally demonstrated through its
interfaces [3]. However, usability is referred to as the ease of
use and suitability of a system for a specific class of users
carrying out particular tasks in a precise environment. Ease of
use affects the user’s performance and their satisfaction, while
acceptability determines whether the product is used by the
customers [1] [2] [14]. In this regard, acceptability to use any
product depends largely upon the full satisfaction of customers
and this can only be attained based on the simplicity and ease
of use of such product. Usability evaluations comprise of a
number of activities, which includes; (1) planning a task
which mirror the future use of the system [15 [16], (2)
978-1-4799-5423-0/14/$31.00 ©2014 IEEE 136
Norlaily Binti Hashim
School of computing
University Utara Malaysia
06010 UUM Sintok, Kedah.
laily@uum.edu.my
determining a method or process to be used for the evaluation
[22] and (3) deciding what is the nature of data to collect and
how to collect it [17] [22]. To this extend, measuring the
interface usability is important and essential which will surely
capture the efficiency, effectiveness and accuracy of the
application interface and will give customers full confidence
easy of mind and satisfaction.
The mobile application needs to be designed in such a
way that will allow users to use the application with effective
interaction. Ability to use to use application with such small
device effectively with good interaction is one of critical
success factors of such application. Therefore, in order to
know whether the application is usable and user can achieve
his objective with minimal effort, the usability of such
application need to be measured with appropriate dimensions
and criteria. Usability can be referred as “the extent to which a
product can be used by specified users to achieve specified
goals with effectiveness, efficiency and satisfaction in a
specified context of use” (ISO 9241−11, 1998). This definition
is too broad; it needs further clarification as only three
variables were identified. There are number of quality
dimensions defined by various researchers that can be used to
measure the actual usability of a mobile application either in
general or specific such as Quim model [21] [3] [14] [12] [5].
Even thought, all these models are more of general but can
still be modified to suit a single application. However,
usability is referred to as the ease of use and suitability of a
system for a specific class of users carrying out particular
tasks in a precise environment. Ease of use affects the user’s
performance and their satisfaction, while acceptability
determines whether the product is used by the customers.
M-banking could still be referred to as an application of
mobile commerce that allows customers to bank essentially at
any convenient time and any place irrespective of
geographical location [18]. M-banking has been seen as one of
the most value-added application and vital mobile service
available. In the study conducted by [9] the main business
drivers that contributed towards the evolution of mobile
banking were identified and these drivers include the
following:
 2014 International Conference on Information Technology and Multimedia (ICIMU), November 18 – 20, 2014, Putrajaya, Malaysia
Customer experience: Customers are getting familiarity
for having access to information at their convenience anytime
and anywhere. This penetrated into access to banking
information which customers may need at locations of their
conveniences. Mobile banking advanced to offer and enhance
customer experience by adopting the latest technology for real
time for data exchange.
Cost savings: Customers walk into bank branches and
ATMs station point for simple transactions which of course
added to the cost of operations for both customers and bank
management of this channel. Use of cheques for mega value
funds transfers, bill payments becomes minimal. All these
contributed immensely to the operational cost and banks
instigated eyeing for other solutions.
Security: The m-banking delivery channels depend on
communication layer supported by the mobile phone. An
initial system of mobile banking uses SMS as the
communicative approach which provide less security against
hackers and message interceptors. Later, m-banking advanced
to WAP and browser-based banking, which really enhanced
the security on sophisticated hand phone to some level.
However, the introduction of downloadable mobile
applications avoided lots of the previous issues, since
applications could be used on many hand phone devices and
made m-banking more standard and comprehensive.
II. RESEARCH MOTIVATION
A study on the usability measurement of m-banking
application is quite inadequate and has not been addressed
extensively particularly in the interface usability evaluation.
Much research concentrated on usability measurement in
mobile application and generally may not be suitable to apply
in specific application areas. Nowadays, most of the banking
industries anticipating effective service distribution via mobile
technology with a good user friendly interface which may
permit clients to access and conduct transaction so easy and
commodious. Thus, to achieve two good interactions and
client satisfaction, usability of such application must be given
equal consideration and user centred issues must be at the
centre of the operation. Close to banks that offer m-banking
services have inadequate functionalities and this proves that
their interfaces are still insufficient and non-user friendly [4].
Usability is an indispensable feature of quality application and
user satisfaction, as it delivers a beneficial impact on m-
banking users’ perception of products and services. On the
basis of the above, this paper is aim of producing quality
dimensions and measurements that can be applied for
measuring the usability of m-banking application.
III. RELATED WORK
Inquiry on the usability of banking services applications is
very limited more especially with regards to downloadable m-
banking application. Most of the literature concentrated on
SMS and WAP applications while others discussed on m-
commerce and e-banking which seems to be overly
cosmopolitan. Revendran, MacColl and Docherty [19] who’s
conducted a study on tag-based interaction in online and m-
banking with an emphasis on the effect on the usability,
suggested that usability needs to be improved with tag-based
137
interface compared to conventional interface for effective user
satisfaction. A similar subject was also conducted by
Revendran, et al. [18] [20], where efficiency, effectiveness and
user satisfaction were considered in the evaluation of tag-
based and conventional. From the results there is no
significant differences observed in terms of actual
effectiveness and efficiency of the two interfaces apart from
significant levels of user satisfaction particularly among the
inexperienced users. A four-cell matrix was presented as the
proportions of nature with the degree of interactivity [1] [24].
These dimensions comprises of transactive, transpassive,
informactive and infopassive. Condos, et al., [4] introduced
ten usability principles for the development of effective WAP
and m-commerce. The ten principles were tested through an
experiment of two UK WAPs and the finding revealed that
WAP usability remains poor and argued that this could have a
negative effect on the user side and of course on the future of
m-commerce and WAP. A substantial modification is required
in such a way that m-commerce and WAP sites are effective
and the demonstration of the contents to the users.
All the same, in another study by [24] on the evaluation of
m-banking portals, six criteria were applied in the evaluation
process in order to identify the strengths and weaknesses of
the m-banking portals. [24] argued that previous research
concentrated only on few criteria which will not give usable
and effective guides in terms of m-banking evaluation. [25]
conducted a study on Chinese m-banking service evaluation; a
model was established for the evaluation of m-banking
services which involves system quality, interface design
quality, information quality, brand quality and fees/benefits.
Thus, viewing the dimensions and measurement used in
the above studies indicated that there is a strong requirement
to come upward with fresh and appropriate usability
dimensions and measurements for evaluating m-banking
application to subdue the heart rate of the ware from the
customers and help the banking industries to improve the
calibre of their product towards usability issue.
IV. METHODOLOGY
In club to achieve an efficient and sufficient study in this
paper that will give guide in creating relevant and appropriate
usability dimensions and measurements for m-banking
application, a systematic literature review (SLR) was used.
The SLR is an approach mainly used to repeat the existing
evidence regarding treatment of data that can be utilized to
summarize the empirical evidence of the benefits and
limitation of a peculiar method. [13]. This method functions as
a template to discover current usability dimensions and
measurements from the literature of both human computer
interaction (HCI) and mobile human interaction (MHI)
respectively. Likewise, this attack provides an idea of
generating relevant and appropriate usability dimensions and
measurement for usability evaluation of m-banking
application.
To achieve the objective of this paper, five main HCI
journals and three conference proceedings have been selected
from 2007 ahead and this method of selection has a basis from
[6] [21]. These journals and conference proceedings that have
 2014 International Conference on Information Technology and Multimedia (ICIMU), November 18 – 20, 2014, Putrajaya, Malaysia

been selected are among the topmost in HCI field. Table1 IJCSE - - 2 2 2 1 - 7
illustrates the selected journals and conference proceedings. IJMHCI - 1 1 2 2 1 - 7
ICHFCS 1 - 1 2 1 1 - 6
The method used for the selection of relevant papers are ICHCIMDS - 2 - 2 1 - 2 7
mainly on the area of usability study, evaluation Hawaii ICSS 2 1 - - - - - 3
method/design and of course papers that provides usability Total 49
principles and dimensions/design guidelines. All the relevant
papers selected were carefully reviewed so as to gather quality
information for creating the dimensions and measurements. The main objective of this paper is to provide usability
TABLE 1: DESIGNATED JOURNALS AND CONFERENCE PROCEEDINGS dimensions and measurement for evaluating m-banking
application; therefore five dimensions with twelve
Journal/Conference Proceeding Publisher measurement criteria (sub-dimension) have been identified.
International Journal of Human Computer Taylor and Francis Group
Interaction (IJHCI) A. EFFICIENCY
Software Quality Journal Springer Science
Journal of usability Studies Usability Professional
• Compatibility: This refers to the extent to which
Association user interface is compatible with different mobile
International Journal of Computer Science Elsevier devices and human ability.
and Engineering • Loading time: This measure the time required for
International Journal of Mobile Human IGI Global
Computer Interaction
the apps to load in to mobile device and
International Conference on Human Factors Proceeding paper appear in login/logout time, i.e. how fast it responds to the
in Computer Science ACM Digital Library user when loading and using the application.
International Conference on Human ACM Annual conference • Accuracy: It is the performance measure of apps
Computer Interaction with Mobile Devices
and Services
to complete specified task, successfully,
Hawaii International Conference on System Proceeding papers appear accurately and within a time. Therefore,
Science in ACM Digital Library integrating the three measurements will
determined the efficiency of m-banking
The abstracts of the downloaded papers were read to application interface.
determine its relevancy before the actual review. Similarly,
based on the quality and relevancy of the paper, sixty three B. EFFECTIVENESS
(708) papers were selected for review. Table2 below indicates • Presentation: This refers the logical presentation
the journal/conference proceeding names, year of publication of the menu buttons (visibility of important option
and number of papers downloaded: which can easily be understood by user), graphic,
TABLE 2. PAPERS DOWNLOADED FOR REVIEW FROM JOURNAL AND interface layout and the readability of the output.
CONFERENCE PROCEEDINGS • Navigation: The degree to which navigation is
Journals/Confe- Year Total logically structured and make sense to the user by
rence Proceedings 07 08 09 10 11 12 13 allowing the user to move around easily between
IJHCI 06 28 30 19 22 10 03 118
related to find contents. However, to measure the
SQJ 04 06 11 07 19 22 07 76
JUS - 41 39 16 31 02 129
effectiveness of m-banking application, the
IJCSE - 10 20 05 02 - 37 navigability and presentation should be integrated
IJMHCI - 13 10 36 13 03 74 to mirror out the actual effectiveness of user
ICHFCS 11 - 17 22 41 09 01 101 interface
ICHCIMDS 20 14 37 08 22 11 05 117
Hawaii ICSS 10 05 16 26 - - - 57
Total 708 C. TRUSTFULNESS
• Privacy: It measures whether the user personal
information is appropriately protected, i.e. user
The main goal is to select relevant and suitable papers that personal data should be confidential.
mainly focus on usability dimensions and measurements, • Reliability: It measures the error rate and the
therefore after cautious, studied and analysis, the relevant and extent to which the interface is absolutely
suitable papers, and then reduced to forty nine (49) as it can be
dependable, trusted and reach user mind stability.
seen in the table 3 below:
Therefore, the mapping of these measurements
TABLE 3. FINAL REVIEW PAPERS will determine the reliability and confidence of
the m-banking application from the customer side.
Journals/Confere Year Total
nce Proceedings 07 08 09 10 11 12 13
D. LEARNABILITY
IJHCI 1 - 1 - 2 2 1 7
SQ Journal - 1 - 1 2 - - 4
• Simplicity: This measures the simplicity level for
JUS - 2 - 2 3 1 - 8 the interface to allow user to carry out
transactions so easily with a minimal effort/action

138
 2014 International Conference on Information Technology and Multimedia (ICIMU), November 18 – 20, 2014, Putrajaya, Malaysia
and communicate with concise and simple user’s
own language.
• Familiarity: It measures the extent to which the
interface provides recognizable elements to the
user through interactions so as to aid easy
understanding. Therefore, simplicity and
familiarity can be integrated together to measure
the degree of effort required for the user to learn
the interface with a high level of satisfaction.
E. USER SATISFACTION
• Content: It measures the degree to which the
content addresses the user’s objectives or goals
and whether the content has value to the user. [15,
17] mentioned that content can be used as one of
the criteria to measure user satisfaction more
especially the degree to which the content
addresses the user’s objectives or goals and
whether the content has value to the user.
• Structured task: Information should be well
organised in the interface so as to provide the user
with a selection of options and interact so easily
when carrying out a transaction or task.
• User guide: This measures the extent to which
the interface provides context sensitive help and
meaningful feedback. Therefore, there three
measurements can be integrated to measure the
user satisfaction for m-banking application
interface.
The categorization of the measurement for each dimension
are as follows: Compatibility, loading time and accuracy are
categorised under “Efficiency” dimension, measurement that
focus on presentation and navigation are categorised under
“Effectiveness” dimension, measurement centred to privacy
and reliability are placed under “Trustfulness” dimension,
other measurement that focus on simplicity and familiarity are
categorised under “Learnability” dimension while
measurement that centred to contents, structured task and user
guide are placed under “user satisfaction” dimension.
However, the justification of the proposed usability
dimensions and criteria above for the evaluation of m-banking
application are linked to the related area from the previous
studies on mobile application usability evaluation.
V. COMPARISON OF OTHER USABILITY
EVALUATION MODELS
There are many usability evaluation models introduced in
the literature but most of them did not provide overall
descriptions on how to select criteria or metrics to each
corresponding usability dimension. Despite that most of the
usability models are general application but it will be difficult
to apply for a single application especially mobile financial
application due to complexity and clarification of the
dimensions used. However, [21] provide little opportunity for
selecting metrics to each corresponding criterion then to
usability dimension but it is too general and not for specific
139
application. Table 4 below describes the comparison of other
usability evaluation models.
TABLE 4. COMPARISON OF OTHER USABILITY MODELS
Usability Evaluation Usability factors used
Models
ISO 9241-11 Efficiency, Effectiveness and Satisfaction
Nielsen (1994) Efficiency, Learnability, Memorability,
Errors/safety, Satisfaction
Performance speed, Time to learn, Retention
Shniederman (2005) over time, Rate of Error by user and
Subjective satisfaction.
QUIM Model (Seffah, et Efficiency, Effectiveness, Productivity,
al. 2006) Satisfaction, Learnability, Safety,
Trustfulness, Accessibility, Universality and
Usefulness
Navigation, Contents, Information
Condos, et al. (2002) architecture, Error prevention, Presentation,
Input rate and Menu visualization
mGQM (Azham, 2011) Simplicity, Accuracy, Time taken, Features,
Safety and Attractiveness
Effectiveness, Efficiency, Satisfaction,
Baharuddin, et al (2013) Usefulness, Aesthetic, Learnability,
Simplicity, Intuitiveness, Understandable
and Attractiveness
Most of the models are based on design and evaluating
usability of mobile applications either in user context, task
support or device type. Unfortunately, many of them lack
adequate descriptions for the selection of criteria and metrics
that correspond to dimensions defined.
VI CONCLUSION & FUTURE WORK
Usability dimensions and measurement believed to be the
key success factors for many emergent software product that
allows users to perform common task very easily to achieve
the specify goal without any hindrance. This paper has
presented a set of usability dimensions and measurement that
can be used to evaluate the usability of mobile banking
application. The dimension and measurement have been
generated from the relevant previous studies through
systematic literature review. Relevant journals and conference
proceedings were carefully considered and reviewed.
In the next stage, metrics for each of the criterion will be
created. Therefore, the derived usability dimensions, criteria
and the metrics will undergo verification process through the
usability experts’ reviewers from both academic and
practitioners to maintain originality and relevancy of their
purpose. Furthermore, the designed measurements will be
validated through the usability test session and post-usability
test questionnaire will equally be used to determine the
 2014 International Conference on Information Technology and Multimedia (ICIMU), November 18 – 20, 2014, Putrajaya, Malaysia
effectiveness of the proposed measurements. Both objective
and subjective metrics are to be used.
REFERENCES
[1] Aleksy, M., Stieger, B., & Vollmar, G., “Case Study on Utilizing Mobile
Applications in Industrial Field Service”, Paper presented at the
Commerce and Enterprise Computing, 2009 (CEC'09. IEEE Conference
on).
[2] Azuma, M., “Software products evaluation system: quality models,
metrics and processes”, International Standards and Japanese practice,
Information and Software Technology, vol 38(3), pp 145-154, 1996.
[3] Baharuddin, R., Singh, D., & Razali, R., “ Usability Dimensions for
Mobile Applications:A Review”, Research Journal of Applied Sciences,
vol 5(6), pp 2225-2231, February, 2013.
[4] Condos, C., James, A., Every, P., & Simpson, T., “Ten usability
principles for the development of effective WAP and m-commerce
services”, Paper presented at the Aslib proceedings, 2002.
[5] Coursaris, C. K., & Kim, D. J., “A qualitative review of empirical
mobile usability studies”, Paper presented at the presentation, Twelfth
Americas Conference on Information Systems, Acapulco, Mexico, 2006.
[6] Coursaris, C. K., & Kim, D. J., “A meta-analytical review of empirical
mobile usability studies”, Journal of Usability Studies, vol 6(3), pp 117-
171, 2011
[7] Dubey, S. K., Gulati, A., & Rana, A., “Integrated Model for Software
Usability”, International Journal on Computer Science and Engineering,
Vol 4, pp 429-437, 2012a.
[8] Dubey, S. K., Gulati, A., & Rana, A., “Integrated Model for Software
Usability”, International Journal on Computer Science & Engineering,
4(3), pp 437-452, 2012b.
[9] Dubey, S. K., & Rana, A., “Assessment of usability metrics for object-
oriented software system”, ACM SIGSOFT Software Engineering
Notes, vol 35(6), pp1-4, 2010.
[10] Hornbæk, K., “Current practice in measuring usability: Challenges to
usability studies and research”, International Journal of Human-
Computer Studies, vol. 64(2), pp 79-102, 2006.
[11] Hussain, A., “Metric based evaluation of mobile devices: mobile goal
question metric (mGQM)” University of Salford, 2012.
[12] Hussain, A., & Kutar, M, “Usability Metric Framework for Mobile
Phone Application”, PGNet, ISBN, 978-971, 2009.
140
[13] Kitchenham, B., “Procedures for performing systematic reviews”, Keele,
UK, Keele University, 33, 2004.
[14] Nielsen, J., “Heuristic evaluation”, Journal, Usability inspection
methods, vol. 24, pp 413, 1994.
[15] Nielsen, J., “Durability of usability guidelines”, Retrieved, vol 11(15),
2006.
[16] Nielsen, J., & Hackos, J. T., “Usability engineering”, (Vol. 125184069):
Academic press Boston, 1993.
[17] Preece, J., & Benyon, D., “A guide to usability: Human factors in
computing”, Addison-Wesley Longman Publishing Co., Inc. 1993.
[18] Ravendran, R., MacColl, I., & Docherty, M., “A comparative usability
study of a tag-based interface in the mobile banking context”, Paper
presented at the Proceedings of the 13th International Conference of the
NZ Chapter of the ACM's Special Interest Group on Human-Computer
Interaction, 2012a.
[19] Ravendran, R., MacColl, I., & Docherty, M., “Tag-based interaction in
online and mobile banking: A preliminary study of the effect on
usability”, Paper presented at the Proceedings of the 10th asia pacific
conference on Computer human interaction, 2012b.
[20] Ravendran, R., MacColl, I., & Docherty, M., “Usability Evaluation of a
Tag-Based Interface”, Journal of Usability Studies, vol. 7(4), pp 143-
160, 2012c.
[21] Seffah, A., Donyaee, M., Kline, R. B., & Padda, H. K.., “Usability
measurement and metrics: A consolidated model”, Software Quality
Journal, vol. 14(2), pp 159-178, 2006.
[22] Shneiderman, B., “Designing the user interface”, Pearson Education
India, 1998.
[23] Singh, A., & Wesson, J., “Evaluation criteria for assessing the usability
of ERP systems”, Paper presented at the Proceedings of the annual
research conference of the South African Institute of Computer
Scientists and Information Technologists, 2009.
[24] Zarifopoulos, M., & Economides, A. A., “Evaluating mobile banking
portals”, International Journal of Mobile Communications, vol. 7(1), pp
66-90, 2009.
[25] Zhuo, Q., & Li, Y., “Chinese Mobile Banking Service Evaluation Based
on AHP Method”, Paper presented at the E-Product E-Service and E-
Entertainment (ICEEE), International Conference, 2010.