Professional Documents
Culture Documents
net/publication/252019625
CITATIONS READS
21 2,065
3 authors:
Alain Renault
Luxembourg Institute of Science and Technology (LIST)
49 PUBLICATIONS 572 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Marion Lepmets on 09 September 2015.
Abstract— Our research aims to evaluate the impact of process improvements are not implemented or their benefits not
improvement on IT service quality by conducting a series of quantified.
studies in industry. In order to do that we first establish a The main research question of our research work is to
framework for IT service quality measures and indicators. evaluate the impact of process improvement on IT service
Stemming from the fact that you cannot improve what you quality. Our research extends on various works describing
cannot measure, this paper proposes IT service quality different approaches to measure process improvement and
measures. We adapt two widely known quality measurement innovation and their impact on organizational performance.
frameworks of software engineering to IT services. We apply The study is carried out in the area of IT Service
the software product quality model described in ISO/IEC
Management, and focuses on IT service management
25010 and the quality measurement framework of Practical
models, IT service process improvement and innovation and
Software and Systems Measurement to the IT service quality
attributes, which are derived from ITIL, ISO/IEC 20000 and
their impact on IT service quality. In order to address the
SERVQUAL. As a result of the study, we will have IT service research question, we will first identify the IT service quality
quality measures that industry can collect in order to evaluate measures.
their progress. Little has been published on measuring the quality of IT
services although there is an overall agreement on the
Keywords-IT service quality; quality measurement importance of this topic. This paper describes the first phase
framework; IT service quality measures; SQuaRE; PSM; ITIL; of our study proposing a theoretical framework for
ISO/IEC 20000; SERVQUAL measuring IT service quality. The framework is based on the
relevant related literature to the subject at hand from IT
I. INTRODUCTION service management and software engineering areas.
Software engineering is a domain where measurement and
Service Science is the emerging integration of multiple process improvement have played a key role in research as
methods and approaches from different disciplines related to well as in practice and we aim to transfer this insight into IT
understanding, designing, improving, and scaling service service management domain.
systems >1@. Although many IT organizations have The paper is structured as follows: we will first provide
understood the importance of IT Service Management, most the overview of the related literature on IT service quality.
of them are still far from mastering their IT service Related works also include the description of measurement
processes at the same level as their engineering processes. frameworks from software engineering that we aim to apply
While services have become the largest part of most to measuring the quality of IT services. We then map IT
industrialized nations’ economies, there is still no widely service quality characteristics to the measurement framework
accepted definition of service; and service productivity, in order to propose IT service quality measures that will be
quality, and innovation remain hard to measure. used in our further studies.
Process improvement is about improving the current state II. RELATED LITERATURE ON IT SERVICE QUALITY AND
of practices and is based on the evaluation of these practices,
QUALITY MEASUREMENT
also referred to as process assessment. Process improvement
aims to increase efficiency and performance of This paper relates to two different topics: On the one hand it
organizations. Successful process improvement is one that is related to IT service quality and quality attributes and on
supports the achievement of organizational and operational the other hand it is related to quality measurement
goals. Process assessment helps organizations improve frameworks that illustrate software and systems quality
themselves by identifying their critical problems and concepts and measurement approaches.
establishing improvement priorities. There is a large number
of process assessment and improvement models available but A. IT Service Quality
there is little research conducted to evaluate the impact of Quality of service refers to the extent to which the service
these models on organizational performance in industry. fulfills the requirements and expectations of the customer
There is also an extensive amount of literature about the key (including users) >2@. To be able to provide quality, the
success factors of process improvement but still a lot of
754
778
786
785
SQuaRE is an international standard and it describes the Goal/Question/Metric approach >15@, still popular among
relationship between a quality model, its associated quality researchers. PSM has rapidly become the most widely
characteristics and software product attributes with the practiced approach to the management of software projects
corresponding software quality measures, measurement >13@.
functions, quality measure elements, and measurement The goal of PSM is to provide project and technical
methods >11@. SQuaRE >12@ describes eight software quality managers with the quantitative information required to make
attributes either for a software product or a computer system: informed decisions that impact project cost, schedule, and
x Functional suitability technical performance objectives. The PSM describes
x Reliability measurement as a systematic, but flexible process that can be
x Usability applied to both software and system engineering, as well as
x Performance efficiency management activities. The PSM measurement process is
x Maintainability based on a proven set of principles derived from actual
x Portability experience on government and industry projects.
x Security The PSM organizes issues the managers can use for
decision-making into seven classes or categories that are
x Compatibility
further decomposed into measurable concepts:
Software product quality can be evaluated by measuring x Schedule and progress
internal attributes (typically static measures or intermediate x Resources and cost
products), external attributes (typically by measuring the x Product size and stability
behavior of the code when executed), or by measuring x Product quality
quality in use attributes (when the product is in real or x Process performance
simulated use) >12@. The quality in use of a system x Technology effectiveness
characterizes the impact that the product has on the x Customer satisfaction
stakeholders: effectiveness, efficiency, satisfaction, safety,
and context comprehensiveness. Project success depends on achieving the defined
Improving process quality contributes to improving objectives. Measurements provide insight for making
product quality and product quality contributes to improving decisions that help to achieve these objectives and to assure
system quality in use >12@. Therefore, assessing and project success. An effective measurement process helps
improving a process is a means to improve product quality. project managers to identify and manage problems or risks
Evaluating and improving product quality is one of the that could adversely impact the project. The PSM is an issue-
means to improve system quality in use. The relationships driven measurement process, where these obstacles are
between the SQuaRE quality attributes are illustrated below referred to as issues that must be managed to achieve project
on Fig. 1. success (Fig. 2). The PSM measurement specification
guidance is designed to simplify the mapping of project
Process Software product Effect of software product
The Practical Software and Systems Measurement (PSM) issues to applicable measures. A project manager requires a
initiative began as an effort by the US Department of lot of information from the project to make informed
Defense to improve the management of large software decisions and effective communication throughout the
acquisition projects. PSM was developed as the result of an project organization. The PSM process is adapted to meet
unusual collaborative effort among industry, government and these information needs. Issue areas are derived from
academic organizations >13@. It represents the best management information needs of a project. Issues are areas
measurement practices used within the software and system of concern that may impact the achievement of a project
acquisition and engineering communities >14@. Many of the objective. An indicator is a measure or combination of
basic concepts of PSM have been formalized in ISO/IEC measures that provides insight into an issue or concept. Most
Standard 15939 – Software Measurement Process and are indicators compare actual values with baselines.
closely related to other measurement approaches, like Measurement categories define groups of related measures.
755
779
787
786
A measure is the quantification of a characteristic of a groups of measures: process measures, internal IT service
process or a product.
p measures, external IT service measures, and customer
satisfaction measures. Process measures are related to
process performance evaluated with the process assessment
standards. Internal quality measures are related to the quality
of the information system that the IT service is built upon.
The external quality measures are related to the quality of the
IT service provided to the customer. Customer satisfaction is
related to the perception and expectations of the customer
about the IT service and is based on the customer feedback
to the IT service. The adopted SQuaRE approach to IT
service quality has been illustrated below on Fig. 3.
We now present the mapping of IT service quality
attributes to the PSM framework by common issue areas in
the tables. Common issue areas are derived from SQuaRe
and the PSM and target process, internal, external, and
Figure 2. PSM measurement process elements customer satisfaction measures. Indicators for the measures
are derived primarily from ITIL.
Each indicator in the PSM is related to a common issue We first describe the external measures of IT service
area, measurement category, and measure. An indicator is a quality in (Table 3) that support the stability of the IT
combination of measures that provide insight into one or service. These measures are directly related to the customer
more issues or concepts. Table 2 (for better readability, the experience of the IT service but are measured and improved
tables are in the end of the paper) illustrates a fragment of the
by the IT service provider. Based on ITIL >16@, IT service
indicator examples in the PSM.
stability is related to attributes that aim to provide IT service
III. MAPPING IT SERVICE QUALITY ATTRIBUTES TO continuously in the way agreed with the customer.
QUALITY MEASUREMENT FRAMEWORKS Availability means that the delivered IT service availability
matches or exceeds current and future agreed needs of
Based on >16@, a service is a means of delivering value to business. IT service continuity addresses the IT service
customers by facilitating outcomes customers want to continuity and recovery in agreed time, and the management
achieve without the ownership of specific costs and risks. An of risks of the IT service continuity. IT service capacity
IT service, according to >2@, is an output of an IT relates to the performance and capacity of the operational IT
organization. Both customers and users use the service but services. The speed of information processing is very critical
the users don’t necessarily buy it. In our study, we talk only for the IT service customers and is addressed in IT service
about customers combining all the users as we aim to performance measurement category. Information security
measure the quality based on the feedback of all who use the means that information is available to those who have the
service.
IT Service processes Information System IT Service Effect of IT Service
influences IT service
influences influences
Process IS quality quality Customer
quality attributes attributes satisfaction
depends on depends on depends on
756
780
788
787
IS Quality relates to the system’s ability to support the IT applied to IT service quality attributes described in ITIL,
services delivered. Based on the PSM, ISO/IEC 20000 and ISO/IEC 20000 and SERVQUAL.
ITIL, the internal measurement categories are: functional As a result of the study we have identified IT service
correctness, usability, availability/reliability/maintainability/, quality measures and indicators grouped into four common
component availability, component capacity, scalability, issue areas: IT service quality, IS quality, process
adjustability and portability. Functional correctness is performance and customer satisfaction measures. These
related to the functional defects of the system. Portability indicators could assist industry to evaluate their progress of
stands for the system’s compliance with the standards. providing better IT service and providing the IT service
Usability is related to the problem prevention and resolution better. These indicators will also be used in our further
of the problems. The availability of components at agreed studies for data gathering from industry as we aim to
time and the time it takes to have the system working after evaluate the impact of process performance on the IT service
the failure both support the reliability of the system. and IS quality.
Component capacity is the power to store and process data,
and absorb changes in the required time. Scalability focuses ACKNOWLEDGMENT
on the provider’s ability to ensure growth at the required The present project is supported by the National
speed, without disturbing the business. Adjustability of an Research Fund, Luxembourg and cofounded under the Marie
information system relates to the development method, Curie Actions of the European Commission (FP7-
architecture and infrastructure that can support adjustments COFUND).
according to the needs of the customer organization.
We illustrate the process performance measures in (Table REFERENCES
5) that can be collected from process assessment or [1] J. Spohrer, P. P. Maglio, et al. "Steps toward a science of service
measuring the effectiveness or efficiency of the processes. systems." Computer 40(1): 71pp. (2007).
The performance of IT service processes can be measured in [2] S. Polter, et al., ISO/IEC 20000 - An Introduction, First Edition ed.
three ways: process assessment is an evaluation of process Zaltbommel: Van Haren Publishing, 2008.
performance against process reference model requirements. [3] L. Shu-Ping, et al., "What is the valuable service quality gap for
ISO/IEC 15504 conformant IT service process assessment improving customer satisfaction?," in 6th International Conference on
Service Systems and Service Management, 2009, pp. 242-247.
can be conducted based on TIPA methodology >17@. Process
[4] I. MacDonald, “A simple, low cost but innovative approach to end-to-
effectiveness in software engineering is measured by the end service reporting”, itSMF Whitepaper, 2009
absence of defects in its outputs and process efficiency by the [5] T. Barroero, et al., "Aligning IT Service Levels and Business
increase of software productivity. IT service process Performance: A Case Study," in IEEE International Conference on
effectiveness is measured through the defects discovered and Services Computing (SCC), 2010, pp. 570-577.
rework required after the IT service design phase. [6] P. Brooks, Metrics for IT Service Management, First edition ed.
Finally, we illustrate the measures about the perceptions Zaltbommel: Van Haren Publishing, 2006.
of the IT service from the customer viewpoint in (Table 6). [7] A. Parasuraman, et al., "Refinement and Reassessment of the
SERVQUAL method suggests measuring the gap between SERVQUAL Scale," Journal of Retailing, vol. 67, pp. 420-450, 1991.
the expectation and perception of the IT service user. [8] R. T. Watson, et al., "Measuring information systems service quality:
Customer satisfaction survey results provide the most Lessons from two longitudinal case studies," Mis Quarterly, vol. 22,
pp. 61-79, Mar 1998
important insight into how well the IT services have been
[9] L. F. Pitt, et al., "Service Quality - A Measure of Information Systems
provided and received by the customer. A customer Effectiveness," MIS Quarterly, vol. 19, pp. 173-187, Jun 1995
satisfaction survey should include SERVQUAL quality
[10] V. Zeithaml, et al., Delivering quality service: balancing customer
indicators about the service provider’s attitude and empathy perceptions and expectations: Free Press, 1990, Chapter 2
from the customer’s viewpoint. Customer satisfaction is in [11] ISO/IEC JTC1 SC7, "ISO/IEC 25020, Software engineering -
fact the sum of how good the IT service is and how well the Software product Quality Requirements and Evaluation (SQuaRE) -
IT service is provided. We should therefore gather customer Measurement reference model and guide," 2007
feedback also on IT service quality from the users and IS [12] ISO/IEC JTC1 SC7, "ISO/IEC FCD 25010, Systems and software
quality from the customers who are internal to the IT service engineering - System and software product Quality Requirements and
providing organization. Customer support is the point of Evaluation (SQuaRE) - System and software models," 2009
contact for IT service users on a day-to-day basis known as [13] D. N. Card and C. L. Jones. “Status report: practical software
measurement.” Proceedings of Third International Conference on
the service desk. Quality Software, 2003.
IV. CONCLUSIONS [14] "Practical Software and Systems Measurement: A Foundation for
Objective Project Management, v. 4.0b1”, www.psmsc.com, 2000
This paper describes the first step in our research work [15] R. V. Basili and G. Caldiera, et al. “The Goal Question Metric
on the impact of process improvement in IT service Approach”. Encyclopedia of Software Engineering. Ed: J. J.
management. The aim of this paper was to identify IT Marciniak. New York, John Wiley & Sons, Inc., 1994, pp. 528-532.
service quality measures with the help of quality [16] V. Lloyd and C. Rudd, "ITIL Service Design," OGC, London, 2007
measurement frameworks of SQuaRE and PSM from [17] B. Barafort, et al., ITSM Process Assessment Supporting ITIL.
software engineering domain. These frameworks were Amersfoort: Van Haren Publishing, 2009
757
781
789
788
TABLE II. INDICATOR EXAMPLES OF PSM (PSM)
758
782
790
789
TABLE III. IT SERVICE QUALITY ATTRIBTUES – EXTERNAL MEASURES
759
783
791
790
TABLE V. IT SERVICE PROCESS PERFORMANCE
760
784
792
791