You are on page 1of 8

Using the Balanced Scorecard to Evaluate ICT Investments

in Non profit Organisations


Renata Paola Dameri
Facoltà di Economia, Università di Genova, Italy
dameri@economia.unige.it

Abstract: For nonprofit organizations (NPO’s), ICT is crucial to fulfil their social objectives. However, it is rare that
ICT investments have monetary returns; ICT also has indirect impact on the social activity of NPO’s. So it is very
difficult for them both to decide about ICT investments and to evaluate their contribution to performance. NPO’s
should therefore define an appropriate evaluation framework, to understand if, where, what and how much to
invest in ICT, to better achieve their mission. The evaluation framework described in this paper is based on the
peculiar characteristics of nonprofit organizations, on the multidimensional evaluation criteria and on the balanced
scorecard, adapted to the specific nature of nonprofit activities.

Keywords: nonprofit –investment decisions – balanced scorecard – multidimensional evaluation

1. Information technology and The first group is concerned with the ICT
nonprofit organizations investment decisions, that is, the ex-ante
phase in automation processes.
Information technology is a key issue in ƒ The main problem is related to prioritising
organizations of all types, nonprofit as well as technology investments, with respects to
for-profit. Nonprofits spend approximately 4% other investments. The scarcity of financial
of their resources on IT investments on resources, typical in nonprofits, is a crucial
average (Computer Science Corporation, aspect in investment decisions. Nonprofits
2004). There are several reasons for the tend to concentrate their investments in
diffusion of ICT in nonprofits. social actions and programs, that is,
ƒ The main concern is the progressive activities directly related with the social
reduction of ICT costs, which increases the impact of their work. So they tend to neglect
convenience of ICT investments also for investments in support activities, such as
nonprofits, which traditionally have scarce information technology.
financial resources. ƒ There is also a great difficult in identifying
ƒ Another reason for the increased use of the more appropriate ICT solutions for
ICT is the bigger dimension of nonprofit specific nonprofits needs. The lack of best
organizations; indeed they need more and practices concerning ICT in nonprofits and
more ICT to overcome some typical big the specific nature of these organizations
corporations problems, such as need the design of ad hoc decision support
management, coordination, communication, instruments.
administrative expenses, and so on. They ƒ The third concern is identifying the
need ICT in administration processes, to appropriate level of ICT investments. The
support decisions, to integrate their lack of a direct relation between the
geographically dispersed units. investments amount and the financial
ƒ Several nonprofits are also discovering the returns makes it impossible to evaluate the
usefulness of Internet; they understand that resources allocation in ICT.
this technology is a very effective The second group is concerned with the ex-
instrument to extend their communication post ICT evaluation, that is the measurement
power. Communication is a core process of ICT performance. The goal is to determine if
for nonprofits, which are based on social the ICT project has been useful in supporting
consensus and need to be known all over specific action and fulfilling social objectives.
the environment in which they work. ƒ The main problem is related to the indirect
However, there are also several problems relations between the ICT and the nonprofit
about ICT in nonprofit organizations, these mission. Generally, nonprofit actions
problems can reduce the use of ICT and the consist in furnishing social services not
investments amount, and cause the failure of including technology. ICT is instead used
ICT projects. We classify these problems in primary for support and administration
two groups (Mason 2003). activities. Therefore there is no direct
impact of ICT on social effectiveness of
nonprofit actions.

ISSN: 1566-6379 107 ©Academic Conferences Limited


The correct reference for this paper is:
Dameri R P (2005) “Using the Balanced Scorecard to Evaluate ICT Investments in Non profit
Organisations” The Electronic Journal of Information Systems Evaluation, Vol. 8, Iss. 2, pp 107-114,
available online at www.ejise.com
Electronic Journal of Information Systems Evaluation Volume 8 Issue 2 (2005) 107-114

ƒ Nonprofit organizations have no profit goal, Resources are used in processes. The
so a common evaluation language is organization resources portfolio provides both
missing. Whereas profit serves such a an instrument and a constraint for nonprofits:
simple common language for instrument, because resources are used to
communication, delegation and perform the processes towards the social
coordination within for-profit firms, decisions goals to be fulfilled; constraint, because the
in nonprofits have to be made with amount of raised resources is the real
reference to the mission. measure of their financial budgets and of their
These concerns reveal that nonprofit action opportunity.
organizations need evaluation systems, to
facilitate automation analysis and decision- Processes create actions and initiatives that
making and to understand and improve ICT nonprofits perform in a defined lapse of time,
performance. These instruments have to be oriented to the short and long term subgoals
designed ad hoc for nonprofits, using their defined from the mission.
specific characteristics and implementing
performance measurement focused on Actions are carried out from people involved in
achieving milestones on their initiatives. nonprofit, both employers and volunteers.
They close the virtual cycle from resources to
actions.
2. Performance evaluation in
nonprofit organizations
Before to analyse the role of ICT in nonprofit offer
organizations and to design an evaluation
framework to support investment decisions
and to measure the impact of technology, it is RESOURCES PEOPLE
useful to describe the peculiar nature of

are performed by
performance in nonprofits and their evaluation.
are used in

MISSION
The core of nonprofit organizations is the
primacy of their mission. This is also the base PROCESSES ACTIONS
of an evaluation system. We can describe the
nonprofits scheme as in Figure 1.

At the core of nonprofit organization is its create


mission, which is the reason for which the
organization has been settled. The mission
defines at a general level the goals of the Figure. 1: The mission is the primacy of
organization, but it is necessary to define nonprofit organization scheme
strategic subgoals and objectives: both short However, employers and volunteers are not
term and long-term objectives. These the only types of people involved in nonprofit
objectives system is the start point for the organizations. We can identify:
action of the nonprofit organization, to which all ƒ the recipients of nonprofits services, which
activities, processes, actions have to be are the “customer” of their actions, even if
coordinated with. Resources allocation – both they don’t pay any price for these services;
financial and human resources – have also to they express their satisfaction relating to
be allocated in respect with the mission and its quantity, quality and timeliness of benefits
subgoals (Sheehan 1996). received from nonprofit organization;
ƒ the constituencies who make specific
Human resources are the core competences investments, which express their
for nonprofit organizations, because they are consensus about the appropriate use of
mainly volunteers and the real engine of the financial resources made from nonprofit
organization. They are not only informed, they organization;
share the organization goals and values and
take part in its activity. ƒ the public opinion, which values the
organization whole activity in terms of the fit
People offers financial and work resources: between the declared mission and the
work is either voluntary or paid, financial developed projects.
resources are loans or donations. These scheme needs of several evaluation
phase and instruments (Forbes 1998).

www.ejise.com 108 ISSN: 1566-6379


Renata Paola Dameri

ƒ One of the main evaluation activities


concerns the process efficiency, related Disclosure and Satisfaction,
offer
with the use of available resources. The control consensus, trust
resources portfolio in nonprofits is both the
instruments and the constraint to their
action. The use of scarce resources has to RESOURCES PEOPLE
be monitored, in order to maintain a correct

are performed by
are used in
relation between inputs and outputs, in
MISSION
respects of economic efficiency. However,
the qualitative nature of inputs and outputs
PROCESSES ACTIONS
and the absence of market prices make
evaluating processes efficiency more
complex than in for-profit firms; financial
reports are to be replaced with quality
Efficiency create Effectiveness
performance indicators, built around
measurable but non monetary objectives.
ƒ The actions planned and executed from
nonprofits need to be oriented to the Figure 2: Nonprofit organization scheme and
fulfilment of social goals. Processes evaluation activities
efficiency and actions effectiveness have to
be coordinated with respect to These documents have a double role:
organizational goals, both in short and in ƒ they put the nonprofit organization in touch
long terms. Because of the lack of financial with the environment and its stakeholders,
measure of organizational success, also and make it possible to know, control and
effectiveness should be measured by sanction the organization work when it is
qualitative indicators, related to the quality not suitable with the mission and the main
and timeliness of offered services and the values declared from the organization;
fit between these services and the ƒ they contemporary are an internal
expectations of recipients (Atkinson et. al. disclosure instrument, useful to offer
1997). guidance about how the organization
ƒ Actions effectiveness is not sufficient for should select and run actions and
defining the fulfilment of social goals. initiatives.
Effectiveness monitoring is useful to These evaluation activities are exposed in
coordinate the resources allocation and the Figure 2 and can be used also to support ICT
fit between processes and actions, but investment decision and to evaluate ICT
success should be measured by how impact, with appropriate adjustments.
effectively and efficiently nonprofits meet
the expectations of their stakeholders, that 3. Non profit organizations and ICT
is, recipients, volunteers, donors, the public investment decisions
opinion. Each of these aspects needs a
specific evaluation instrument, right to The same activities exposed in Figure 2 are
control the fit between the action of useful to evaluate the role of ICT in nonprofit
nonprofit organizations and the opinion of organizations, both to decide about
stakeholders above them. investments and to understand the contribute
ƒ The complex relation between the nonprofit of technology to the performance of each
organization and its heterogeneous organization. Therefore, it is possible to define
stakeholders takes also the definition of an evaluation framework, arranged to be
specific disclosure programs and focused on ICT, instead of on the general
instruments, both financial and social, that management of nonprofits.
allow verifying the fulfilment of social goals
and the correct use of resources. The main concern of this evaluation framework
is that ICT usually is not directly related with
the actions that nonprofits carry out to fulfil
their mission (Barton 2002); therefore direct
relations between ICT and social performance
are not easy to note. In addiction, ICT absorbs
part of the scarce financial resources of
nonprofit organizations, that are felt disguised
from primary social goals and spent in projects

www.ejise.com 109 ©Academic Conferences Limited


Electronic Journal of Information Systems Evaluation Volume 8 Issue 2 (2005) 107-114

that are difficult to justify, because returns are investments in terms of indirect but real
not proved (Cilek et al. 2001). impact on the social goals and finally on the
fulfilment of the mission. In this way,
Activities Objectives Impact
Processes Activities Efficiency Identify
nonprofit organizations aim not only at the
carried out objectives one or pursuit of their objectives, but also at the
to introduce related to the more obtainment of social consensus essential
ICT into the ICT project process for their survival.
nonprofit efficient All these evaluation activities could be
organization dimension gathered in an evaluation system described in
processes on which
a descriptive diagram (Figure 3). This diagram
ICT is
expected exposes:
to impact ƒ in lines, all the dimensions of the
Action Social Effectivenes Identify organizational scheme of nonprofit
actions s objectives one or organizations, that is: processes involved in
interested expected more automation, social actions impacted from
from ICT from social action ICT, resources used for the ICT project,
project and action and effective obtained services;
expected impacted dimension
improvemen from ICT on which ƒ in columns, the link between automation
t ICT is activities, social objectives and the impact
expected of ICT.
to impact The cells of the table in Figure 3 should be
Resource Resources Objectives Identify filled in with the description of the specific
s used to related to relations aspects involved with automation. To
carry out the rational and between
ICT project, economic use of
understand the process of analysis of ICT in
in quantity use of resources nonprofits, we can use an example, that is, a
and quality resources; in ICT health organization which offers physiotherapic
(when eventual projects treatments at home. This organization wants to
possible to specific and saves realize a web site to timely inform all its
quantify financial aids of recipients about service schedule and the
hours of obtained resources person who will perform the service.
labour and from grace to
expenses government automatio The first, we have to individuate which
amount) or other n of
institutions processes
processes are connected with the new ICT
Services Social Objectives Identify application: in our case study, the
services connected positive communication process about the service to
involved in with ICT impact of be performed. Using the web, we want to
the ICT project in ICT on communicate to the greatest number of
project terms of quantity, recipients in one time, all the information about
offered quality and the expected treatment. To introduce ICT into
services timeliness this communication process, we have to
of services perform several activities: to design the web
Figure 3: Ex-ante evaluation diagram site, to test it, to advertise it to our patients. We
Ex-ante support for ICT investments evaluation have also to train our personnel in use of this
should therefore comes up to several web site and to favourite the passage from the
expectations. traditional communication way to the new
ƒ It should individuate main processes in instrument. The objectives of this project are
which ICT would be used and expected related with a more efficient communication,
benefits due to improved efficiency of their especially in terms of timeliness and
processes. quickness. So we can identify timeliness and
quickness as the dimension of ICT impact.
ƒ It should link the increased processes
efficiency to an effectiveness improvement
The second, we have to link the ICT project
of specific social actions.
implementation with social actions and to
ƒ It should demonstrate that effectiveness understand if this efficiency improvement is
improvement is relevant for core services useful to our social goals. In our case, we have
quantity, quality and timeliness. to understand if to improve timeliness of
ƒ It should also define disclosure able to communication by our web site is useful and
explain to stakeholders this positive impact appreciate from our customers. We can
of ICT, so that they can perceive the describe the objectives of the improvement of
appropriate use of scarce resources in ICT

www.ejise.com 110 ISSN: 1566-6379


Renata Paola Dameri

our action in terms of more satisfaction of our of the web site from satisfactio
recipients, because they are better communicati our patients n about
on process to obtain informatio
from information n on
telephone about the health
informed about the time and the person who
contact to health services
will perform the service. We can identify the web services
increasing satisfaction as the impact of ICT on publication
social actions. Resource Money spent Saved Resource
s to realize the volunteers s spent to
The third, we have to identify the resources to web site time not realize
be allocated to the project. In this case, we Time spent to more used to the
should count both the persons involved and train the phone to the project
the time spent for the project, and also the personnel patients, but compared
and the used to offer with
money spent to realize the web site. We
patients in health resources
should not forget to count also the time spent the use of services saved
to training the personnel and the efforts spent web site thanks to
to win their resistance to change. We should the
also evaluate if money and persons allocated automatio
to the ICT project are disguised from the n
organization usual activity and if there is a Services Description Services Positive
damage or worse health service for our of the improvement impact of
percipient. We should compare the resources improvement compared the web
absorbed from the project with the savings in in with patients site on
communicati expectation the
money and persons, obtained from the ICT
on about the quality,
project. In our case, to publish information in health quantity
our web site saves to phone to each patient to services and
inform him; there is obviously a save of time of timeliness
volunteers, who can be devoted to other of our
activities. health
service
Finally, all these aspects should be gathered to Figure 4: Case study for the ex-ante
describe if our social service is better, with a evaluation diagram
better use of scarce resources and increased The drawing up of a qualitative and descriptive
satisfaction of recipients. We have to connect diagram that explains the virtuous cycle of ICT
the effectiveness of the communication service in nonprofit activity is useful to answer to
with the quality of our main service, that is, the several questions about ICT investment
physiotherapic treatments. Only if the ICT decisions.
impact is effective on our mission, we can say ƒ Prioritising technology investments. The
that automation is useful for our organization. lack of a direct relation between ICT and
social goals of nonprofits often forces to
Figure 4 shows the ex-ante evaluation diagram attribute low priority to ICT investments,
for the Information web site project. This respect to other investments more related
diagram report the result of our analysis, as to the specific organization’s mission. To
previously explained. describe the positive impact of ICT on
processes, actions and – indirectly – on
4. Information web site project – offered services to recipients makes it
ex-ante evaluation diagram possible to compare ICT projects with other
projects and perform a more informed
Activities Objectives Impact
choice, because ICT also reveals its
Processe Creation of a Timeliness Reduced
s web site and time importance.
Training of quickness of spent to ƒ Selecting the more appropriate ICT
personnel communicati phone to solutions. Defining specific ICT objectives
Advertising on about each and ICT impact on nonprofit organizations
among the health patient goals is useful to support automation
patients services Timelines decisions also in terms of selection of the
s of
informatio
most suitable ICT solutions and
n about applications. Indeed, the descriptive
health diagram identifies the relations between
service processes efficiency, actions effectiveness,
Action Changement Use of the Patients offered services and therefore it is possible

www.ejise.com 111 ©Academic Conferences Limited


Electronic Journal of Information Systems Evaluation Volume 8 Issue 2 (2005) 107-114

to understand the more urgent and define the metrics and indicators to be used in
strategic ICT projects just for the specific the balanced scorecard. The model here
nonprofit aims. It is not necessary having designed put together all the specificity of
quantitative indicators in this phase, nonprofit organizations:
because improvement in services and in ƒ the mission is at the core of the nonprofit;
stakeholders satisfaction are expected, not ƒ the stakeholders satisfaction and
increase in financial or economic measures. consensus are the first objective related to
Qualitative indicators are therefore able to the mission and the organization success;
furnish the right information to support the
ƒ actions, processes and resources are the
ICT investment decision.
instruments to be used to fulfil the social
ƒ The descriptive diagram is also useful as objectives that define the mission.
internal and external disclosure instrument.
As we can see in Figure 5, the balanced
Indeed, this diagram describes all the
scorecard puts the mission and the
automation objectives and ICT impact on
stakeholders’ satisfaction at the top, resources,
processes, actions, services, used
actions and processes at the base of the
resources. Stakeholders receive all the
scheme. The elements that form the balanced
necessary information to judge the
scorecard and should be used to evaluate the
organization choices.
ICT projects are therefore the same defined as
structural for nonprofits functioning.
5. The evaluation of ICT impact in
nonprofit organizations The balanced scorecard is then built
associating objectives to measures, as
If it is sufficient a descriptive diagram to
explained in Figure 6.
support ex-ante decisions process for ICT
investments in nonprofits, it is necessary a Objectives Metrics
quantitative evaluation system to evaluate ICT Stakeholders Percipients Number of satisfied
satisfaction recipients
ex-post impact. This evaluation framework Donors Level of percipients
should not be based on financial consensus satisfaction
measurement, but be able to demonstrate the Volunteers Level of donors
organization performance due to the use of participation consensus
Increasing in fund
ICT. The better instrument to be used is a raising
balanced scorecard, based on the specific Increasing in
nature of both nonprofits and ICT (Bannister – volunteers
Remenyi 1999). Level of volunteers
consensus
Actions/services Quantity Increasing in the
Quality quantity of services
Timeliness Increasing in the
number of percipients
Increasing in the
value of services
Timeliness in
services
Frequency in services
Services innovation
New services
Processes Efficiency Support activities
costs reduction
Reduction in work
used in non-social
activities
Better integration
between organization
units
Resources Effectiveness in Balance between
resources costs for primary
allocation activities and for
Figure 5: The balanced scorecard general Resources as a support activities
scheme to evaluate ICT constraint Ad hoc fund raising
for ICT
investments in nonprofits Shift of resources
To build a balanced scorecard of this type, it is from support activities
to primary activities,
useful to start form the general scheme
due to ICT
showed in figure 4 and related to the
functioning of a nonprofit organization (Kaplan Figure 6: The balanced scorecard to evaluate
2000). These characteristics are important to ICT impact in nonprofits

www.ejise.com 112 ISSN: 1566-6379


Renata Paola Dameri

ƒ Resources are considered as a constraint;


therefore, performance in resources The models here proposed is built upon a
allocation is measured with the ratio functioning scheme of nonprofit organizations;
between resources used in the ICT project it puts the mission and its short and terms
(financial and human resources) and saved objectives at the core of the organization;
resources (for example, volunteers not around the mission, people, resources,
more employed in support activities and processes, actions are disposed because they
now employed in social activities); this contribute at the objectives achievement. ICT
measure should be also related to contributes at this virtuous cycle too, by means
resources allocated to social actions, to of its role in processes automation and
resources allocated to other support services effectiveness, absorbing resources
activities, to the total amount of resources but also releasing qualitative benefits.
owned by the organization.
ƒ Processes should be evaluated in terms of In this framework, several performance
increased efficiency, for example time indicators are identified, that allow
reduction, costs reduction, labour reduction, individuating the direct or mediate impact of
owing to processes automation. ICT on the mission. This impact is measured
by increasing in processes efficiency, actions
ƒ Actions should be evaluated measuring the
effectiveness, services quality, stakeholders’
impact of automation on the social services;
consensus, recipients satisfaction (Dameri
this impact should be related to the quantity
2001).
of services, their quality and the timeliness
and frequency in answering to the requests
This model is therefore designed just for
of recipients, but also to the innovation in
nonprofit organizations; however, it could be
old services or to the offer of new services
used also in for-profit firms, when they need to
dues to ICT. .
evaluated ICT projects non directly concerned
ƒ Stakeholders’ satisfaction and consensus with economic or financial impact, but focused
should be evaluated with ad hoc on improving some qualitative business
instruments, such as interviews, etc. but conditions, such as customers and employers
could also be understood from an satisfaction, products and services quality, the
increasing in donations, volunteers and so social and environment dimension of the
on. production impact (Drucker 1989). In these
cases, indeed, it is impossible to apply
6. Conclusions financial evaluation systems, but it is possible
to link ICT projects to the qualitative aspects of
The ICT investments evaluation in nonprofit
business strategies and mission, identifying
organizations could not be carried out using
appropriate metrics for the impact of ICT on
traditional instruments, based on economic
the fulfilment of business goals.
and financial methods. Indeed, the mission of
nonprofits is not profit, nor revenue or cash
flows maximisation, but to furnish social References
services to several recipients, with the aim of AA.VV., “New Study Reveals Key Trends in IT
better satisfying their expectations and at Nonprofit” in Nonprofit Business
obtaining consensus from their different Advisor, February 2004
stakeholders. This particular nature of AA.VV., “Nonprofits Reveal 2004 Budget Plans
nonprofit organizations needs ad hoc for IT Spending” in Nonprofit Business
evaluation instruments, focused on the specific Advisor, Novembre 2003
links between ICT investments and the Anthony R. – Young D., Management
organization mission fulfilment. Controlin Nonprofit Organizations,
To solve this problem, it is useful to design an IRWIN, Homewood Illinois, 1988
evaluation system not based on financial or Atkinson A. – Waterhouse J. – Wells R., “A
economic metrics, but focused on the relations Stakeholders Approach to Strategic
between the allocation of scarce financial and Performance Measurement” in Sloan
human resources and social actions. That is a Management Review 1997, 38
balanced scorecard built contemporary on the Bannister F. – Remenyi D., “Value perception
subject of the evaluation – the nonprofit in IT investment decisions”, Electronic
organization – and the object – the ICT Journal of Information System
projects. It requests the definition of new Evaluation n° 3 September 1999
evaluation dimensions, new metrics, and new Barton N., “Measuring I.T. Performance –
methods for the analysis of ICT investments From Functional Models to Service
and related objectives and expected returns. Models”, Ninth European Conference

www.ejise.com 113 ©Academic Conferences Limited


Electronic Journal of Information Systems Evaluation Volume 8 Issue 2 (2005) 107-114

on Information Technology Evaluation, Kaplan R., “Strategic Performance


Paris 15-16 July 2002 Measurement and Management in
Cilek P. et al., “The Evaluation of IT- Nonprofit Organizations”. Nonprofit
Investments in Public Sector Management and Leadership 2000, 11
Organisations”, Eight European Kaplan R. – Norton D., The Balanced
Conference on Information Scorecard: Translating Strategy into
Technology Evaluation, Oxford 17-18 Action, Harvard Business School
September 2001 Press, Boston 2001
Computer Science Corporation, Fifth Annual Mason N., “Why have a special issue on the
Technology Issue for Financial application of data and new
Executives, 2004, at www.csc.com technologies?” in International Journal
Dameri R.P., “Classification and Evaluation of of Nonprofit and Voluntary Sector
Business Information systems – Marketing, Vol.6 n° 4
Analysis and Definition of a Norreklit H., “The Balance on the Balanced
Taxonomy”, Eight European Scorecard: A Critical Analysis of Some
Conference on Information of Its Assumptions”. Management
Technology Evaluation, Oxford 17-18 Accounting Review 2000, 11
September 2001 Remenyi, D., “The Value Scorecard – Beyond
Drucker P., “What Business Can Learn from the business case”, Ninth European
Nonprofits” in Harvard Business Conference on Information
Review, Jul-Agu 1989 Technology Evaluation, Paris 15-16
Farbey B. - Land F. – Target D., “Evaluating July 2002
investments in IT”, Journal of Sheehan R., “Mission Accomplishment as
Information Technology n° 7/999 Philanthropic Organization
Forbes D., “Measuring the Unmeasurable: Effectiveness: Key Findings from the
Empirical Studies of Nonprofit Excellence in Philanthropy Projects”.
Organization Effectiveness from 1977 Nonprofit and Voluntary Sector
to 1997”. Nonprofit and Voluntary Quarterly, 1996, 25
Sector Quarterly, 1998, 27 Speckbacher G., “The Economiccs of
Haine S., “Measuring the Mission: Using a Performance Management in Nonprofit
Scorecard Approach in Not-For-Profit Organizations” in Nonprofit
Organizations”. Journal of Strategic Management and Leadership, vol. 13
Performance Measurement 1999, 3 n° 3, Spring 2003
Harris I. – Mainelli M., Information Technology Willcocks L., “Evaluating Information
for the Not-For-Profit Sector, ICSA Technology Investments: Research
Publishing, London 2001 Finding and Appraisal”, Journal of
Information Systems Vol. 2 n° 3, 1992

www.ejise.com 114 ISSN: 1566-6379