Software Quality Attributes and Measures for Software Product Evaluation

*Muhammad Garba
Department of Computer Science
Kebbi State University of Science and Technology, Aliero-Nigeria
garbamga@gmail.com
**Ganiyu Abiodun Salihu
Department of Computer Science
Umaru Ali Shinkafi Polytechnic, Sokoto-Nigeria
salihu_ganiyu@yahoo.com

ABSTRACT
Software measures provide a scientific basis for monitoring, evaluating, controlling, and
predicting the quality of software products. In the last decade, a great number of quality
attributes and measures for evaluating the quality of software products have been discussed
in literatures but preliminary study reflects that no single study has articulated the current
knowledge of software attributes and measures necessary to evaluate software products. This
paper employs a systematic literature review with the objectives of exploring, identifying and
interpreting some selected primary studies from 2008 to 2017 that present quality attributes
and/or measures for software product. Ninety six (96) measures relating to thirty-two (32)
quality attributes were found as a result of the systematic review. The results of the review
indicated that 74% of the measures evaluate attributes that are related to maintainability
followed by performance efficiency with 13%. 77% of the quality attributes measured,
discussed in selected papers, were internal attributes while the remaining 23% were external
attributes. The study also determined the phases of the software development cycle in which
the measures were applied and it was discovered that the measures were mostly applied at
evaluation phase and least applied at maintenance phase. However, only 33% of the studies
have been empirically validated while 68% have been theoretically validated. The results
have summarily provided a global conceptualisation of the state of art in software measures
and attributes necessary to evaluate software products. This paper will help researchers,
practitioners and software developers in academia and industry to identify weaknesses and
strengths of attributes and measures necessary for software quality evaluation.

Keywords: Software Quality, Software Measures, Quality Attributes, Software Product,

Systematic Review, Software Development Life Cycle
1.0 INTRODUCTION

One of the most difficult tasks during software product derivation is for the software

developers to meet the required quality attributes expected by the software users and other

stakeholders. Montagud and Insfran [1] point out that “quality attribute is a measurable

physical or abstract property of an artefact produced during the software product

development”. Quality of software can mean different things to different stakeholders. For

example, for the software engineer, quality represents the system’s correspondence to

requirements; while for the end-user it also means the usability of the system.

Over some years, a great number of measures for evaluating the quality of Software have

been proposed and several relevant quality attributes have also been identified. However,

despite the existence of these measures, the state of industrial practice is still in its infancy.

Software measurement process must be a good oriented methodical process that measures,

evaluates, adjusts, and finally improves the software development process. Software

measurement has become a key aspect of good software engineering practice. Software

metrics deals with the measurement of software product and software product development

process and it guides evaluating models and tools [2]. Tahir, Rasool and Gencel [3] argue that

in order to assess quality characteristics (both general characteristics and key characteristics

for software), software measures are a good means to monitor, understand, control, predict,

and test software development and maintenance projects. In order to explore the state of art

in software attributes and measures, a systematic review was employed in this research.

Systematic Reviews has been helpful in identifying, measuring, and interpreting the quality

attributes and measures that are essential for software product evaluation. The remainder of

this paper is organized as follows: Section 2 presents material and methods, Section 3

explains the discussions of results, Section 4 discusses the implications of the results for

research and practice. Finally, Section 5 concluded the paper.

2.0 MATERIALS AND METHODS

The main research method used in this paper is systematic literature review. Kitchenham and

Charters [4] define Systematic Literature Review (also known as Systematic Review) as a

“means of identifying, evaluating and interpreting all available researches relevant to a

particular research question, topic area, or phenomenon of interest”. The systematic review is

done in three main steps as recommended by the guidelines provided by [4]. These steps are:

 Planning the review: the necessity for the review is identified, the research questions are

presented, and the review protocol properly defined.

 Conducting the review: selection of the primary studies will be carried out, the criteria

used to include studies are defined, the data extraction and monitoring are performed, and

the obtained data are synthesized.

 Reporting the review: the dissemination mechanisms are specified, and the review reports

are presented.

Fig. 1: Activities in the systematic literature review by [4]

2.1 Data Sources

The following digital libraries were used to get the primary studies on journals, articles and

conference proceedings: IEEEXplore, ACM Digital Library, Science Direct, SpringerLink,

and INSPEC. The search covered the period from 2008 to 2017.
Several search strings were tested, and the following retrieved the greatest number of relevant
papers: ((software! AND (evalu* OR measu* OR quality*)) AND (met* OR improve* OR
goal OR challenge OR problem OR issue OR lessons OR experience OR success OR fail*
OR tool)) . The search was carried out by considering the titles and abstracts of the articles.
Search string notation for each digital library was also adapted, since they vary in syntax. A
gray literature review was also carried out to widen the scope of the selected studies. This
was done by including studies that have been known in advance and other studies that were
found by using search engines e.g. Google and Yahoo.

2.2 Primary Studies Selection Criteria

Both the syntactic and semantic search methods were applied. Considering the fact that titles
and abstracts might not adequately reflect extensively what an article is about, some of the
papers were fully scanned in order to ensure the data satisfy the inclusion and exclusion
criteria. Table1 presents the criteria for inclusion and exclusion.

Table 1: Criteria for Inclusion and Exclusion

Inclusion Criteria Exclusion Criteria

Papers that show software
evaluation technique
Papers that present measures
for quality
evaluation/assessment in
software product
Papers that highlight attributes that
are desirable measures for assessing
quality in software product
Papers written in non-English language
Papers presenting measures which do not
have direct relationship with quality (e.g.,
cost models)
Papers that present measures without
explaining how to apply the measures
Papers proposing measures for assessing
other types of quality different from software
product quality (process quality, quality in
use, etc.)

Table 2 presents a summarised view for classifying the collected quality attributes and

measures for software evaluation.

Table 2: Data Extraction Form

Criteria Measure Quality Attributes

Criterion 1 Quality  Functional Suitability
Characteristics  Performance Efficiency
 Evaluated  Security
 Maintainability
 Reliability
 Operability
 Compatibility
 Transferability

Criterion 2 Type of Quality  Internal

Attribute  External
Criterion 3 Type of  Base Measure
Measures  Derived Measure
Criterion 4 Result of the measure
Type of  Qualitative
Evaluation  Quantitative
 Hybrid
Precision  Exact
 Probabilistic
Criterion 5 Phases of Life  Requirements Analysis
Cycle in which  Design
Measure is  Development
applied  Software Testing
 Implementation
 Evaluation
 Maintenance
 Software Use
Criterion 6 Validation Procedure
Theoretical  Property-based Approach
Validation  Measurement-theory
Approach
Empirical  Case Studies
Validation  Survey
 Experiment
Criterion 7 Tool Support  Manual
 Automatic
Criterion 8 Current Usage  Academic
 Industrial

2.3 Conducting the Systematic Review

A total number of 27 papers were selected as potential papers because their titles and

abstracts contain the supplied search criteria. The papers were selected in accordance with

inclusion and exclusion criteria described in section 2.2. The most significant aspect of this

selection was that many of the papers from digital libraries were included as potential papers

because their titles and abstracts contained the search string. However, once the titles and
abstracts had been read (and this sometimes also required scanning the full paper), it was

discovered that the topic of the paper was different to that of the study. Finally, 27 papers

were selected for the study of which 23 papers were found from the digital libraries and four

from gray literatures.

Table 3: Summary of papers found and selected by source

NO OF NO OF NO OF FINAL
POTENTIAL SELECTED SELECTED
SOURCE PAPER PAPERS PAPERS (NO
(DUPLICATES) DUPLICATE)
DIGITAL
LIBRARIES
ACM 63 11 3
IEEExplore 78 19 10
Inspec 80 13 1
Science Direct 28 7 8
Springer link 18 8 1

GRAY LITERATURE
4
TOTAL 267 58 27

Table 4: Studies included in the final review

Study Paper Title Year of Author(s)

ID Publicati
on
[SP1] Measurement Model of 2012 Tochot, P.,
Evaluation Utilization: Junpeng, P. &
External Evaluation Makmee, P.
[SP2] Towards supporting the 2012 Weinreich, R. &
software architecture life Buchgeher, G.
cycle
[SP3] Evaluating the Quality of 2009 Spinellis, D.
Open Source Software Gousios, G,
Karakoidas, V. &
Louridas, P.
[SP4] A new method for in situ 2014 Pfeifer, P. & Plivia,
measurement of parameters Z.
and degradation processes in
 modern nanoscale
programmable devices

[SP5] Evaluating and selecting 2009 Jadhav, A. &

software packages: A review Sonar, R.

[SP6] An empirical approach 2013 Condori-Fernández,

for evaluating the usability of N., Panach, J.,
model-driven tools Baars, A., Vosc, T
& Pastor, O.
[SP7] A mapping study to 2013 Abdellatief, M.
investigate component based
software system metrics
[SP8] Firms’ involvement in Open 2011 Capra, E.,
Source projects: A trade-off Francalanci, C.,
between software structural Merlo, F. & Rossi-
quality and popularity Lamastra, C.
[SP9] A Systematic Literature 2016 Touseef, T. & Ali,
Review on Software J.
Measurement Programs
[SP10] Complementary methods of 2010 Horsky et al
system usability evaluation:
Surveys and observations
during software design and
development cycles
[SP11] A review of Software Quality 2014 Miguel, J.P.,
Models for the Evaluation of Mauricio, D. &
Software Products Rodriguez, G
[SP12] Development and Evaluation 2013 Glogger,I. et al
of a Computer-Based
Learning Environment for
Teachers: Assessment of
Learning Strategies in
Learning Journals
[SP13] Decision Support for moving 2010 Ullah, M.I., Ruhe,
a single product to a product R. & Garousi, V.
portfolio in evolving software
systems
[SP14] Qualitative cost-benefit 2009 Rogers, P &
evaluation of complex, Stevens, K.
emergent programs
[SP15] A systematic review of 2012 Montagud, S. &
quality attributes and Insfran, E.
measures for software product
lines
[SP16] Software test-code 2015 Yusifoğlu, V.G.,
engineering: A systematic Amanneja, Y. &
mapping Can, A.B.
[SP17] How have we evaluated 2015 Riaz et al
software pattern application?
 A systematic mapping study
of research design practices

[SP18] An approach for concurrent 2009 Vavpotic, D. &

evaluation of technical and Bajec, M.
social aspects of software
development methodologies
[SP19] Evaluation of reusable 2015 Duran, M.B., Pina,
concern oriented goal models A.N. Mussbacher,
G.
[SP20] SDMF: Systematic Decision- 2016 Upandhyay, N.
making Framework for
Evaluation of Software
Architecture
[SP21] A maintenance evaluation 2017 Yi, J,X. et al
method for complex systems
with standby structure based
on goal oriented method
[SP22] Assessing integrated 2011 Olsina, et al
measurement and evaluation
strategies: A case study
[SP23] Evaluation and Measurement 2012 Unterkalmsteiner,
of Software Process M.,
Improvement - A Systematic Gorschek, T. &
Literature Review Feldt, R.
[SP24] A comprehensive empirical 2008 Hulse, J.V. &
evaluation of missing value Khoshgoftaar,
imputation in noisy software T.M..
measurement data
[SP25] Comparison of Software 2012 Dubey, S., et al
Quality Models : An
Analytical Approach
[SP26] A Framework for the 2012 Minhas, S.S.,
Evaluation of Mashup Tools Bampal, P. &
Mehandjiev, N.
[SP27] Software Evaluation Criteria 2012 Shariatzadeh, N.
for Rapid Factory Layout Sivard, G. & Chen,
Planning, Design and D.
Simulation

3.0 RESULTS OF THE SYSTEMATIC REVIEW

The distribution of the final selected primary studies over a period of ten years from 2008-

2017 is presented in figure 2.

 DISTRIBUTION OF PRIMARY STUDIES OVER 10 YEARS
7
6
5
NO OF PAPERS

4
3
2
1
0
2008 2009 2010 2011 2012 2013 2014 2015 2016 2017
YEAR OF PUBLICATION

Fig. 2: Distribution of Primary Studies over a period

A result of systematic review based on empirical method employed is summarily presented in

the figure 3:

Figure 3: Distribution of Studies based on Empirical Method

In the 27 studies selected, 11 of them were published in scientific journals, 3 in industry
oriented journals, and 10 in conferences and 2 in workshop proceedings and 1 technical
report. However, 20 of the papers revealed measures and quality attributes. In these papers,
eight criteria were broadly classified and 96 measures that evaluate 32 different quality
attributes were discovered. The remainder, seven (7) papers, only discuss quality attributes
but do not provide explanation of what measure(s) can be used. The table below presents the
extracted data on software evaluation measures and quality attributes.

Table 5: Data extraction criteria results

No of
Criteria strategy Possible Answer Measures Percentage
Criterion 1: Quality measures evaluated
 Functional Suitability 3 3
Performance Efficiency 12 13
Compatibility 1 1
Usability 2 2
Reliability 2 2
Security 2 2
Maintainability 74 77
Portability 0 0
Criterion 2: Quality attributes evaluated by the measure
Internal attribute 74 77
External attribute 22 23
Criterion 3: Type of measure
Base measure 16 17
Derived measure 80 83
Criterion 4: Result of the measure

Type of Evaluation
Qualitative 4 4
Quantitative 90 94
Hybrid 2 2
Precision
Exact 92 96
Probabilistic 4 4
Criterion 5: Phase of the life cycle in which the measure is applied
Requirements Analysis 4 4
Design 25 26
Development 3 3
Software Testing 9 9
Implementation 4 4
Evaluation 37 39
Maintenance 2 2
Software Use 12 13
Criterion 6: Validation procedure

Theoretical
validation
Yes 28 29
No 68 71

Empirical
validation
Yes 33 34
No 63 66
Criterion 7: Tool support
Manual 45 47
Automatic 51 53
Criterion 8: Current usage
Academic 80 83
 Industrial 16 17

The table 5 summarises all the attributes found in the conducted systematic review and the
number of measures. Each quality attribute may appear in several papers. This quantity is
shown in the number of repetitions column. An attempt has been made to classify the
attributes without measures in several criteria (characteristic of quality, type of attribute,
phase of life cycle), but the majority of authors do not correctly provide all the information
necessary for the classification. The attributes with measures are classified in criterion 2 of
table 5 together with their measures. However, the table 6 shows the quality attributes
measured by each measure in criterion 2 of table 5.

Table 6: List of quality attributes for software product found in the systematic review.

S/N Quality Attributes No of Repetition No of Measures

1. Accessibility 1 0

2. Accountability 0 0

3. Adaptability 0 0

4. Analysability 5 10

5. Appropriateness Recognisability 3 1

6. Authenticity 1 0

7. Availability 0 0

8. Capacity 1 0

9. Co-existence 1 0

10. Complexity 10 6

11. Confidentiality 1 1

12. Fault Tolerance 8 2

13. Functional Appropriateness 6 0

14. Functional Completeness 3 1

15. Functional Correctness 5 2

16. Installability 1 0

17. Integrity 1 1
 18. Interoperability 3 1

19. Learnability 4 0

20. Maturity 1 0

21. Modifiability 2 33

22. Modularity 1 15

23. Non-repudiation 1 0

24. Operability 1 0

25. Recoverability 0 0

26. Replaceability 1 0

27. Resource Utilisation 3 2

28. Reusability 2 1

29. Testability 13 15

30. Time Behaviour 9 4

31. User Error Protection 1 0

32. User Interface Aesthetics 3 1

4.0 DISCUSSION OF RESULTS OF SYSTEMATIC REVIEW

The systematic review revealed that an average number of publications over 10 years stand

between 2-3 publications. This is a surprising observation as software evaluation have been in

use for quite a while and it was expected to have found more empirical studies over a decade.

The disappointing result might be as a result of limited number of researchers/interest in this

topical area or as a result of limited access to companies where software products are being

evaluated. The situation is not encouraging as it is limiting imparting of knowledge and

reducing acquisition of experience in the software evaluation areas.

The systematic review also revealed that case study was majorly used method with (63%)
followed by Report from industry with (22%) while the other methods share the remaining
(15%). This result along with few numbers of empirical studies published on software
evaluations form a major threat to knowledge acquisition in software evaluation area. This
might have accounted for the reasons behind lack of standard framework, tools and practices
as supported in [5]. Case study and industry report were considered based on whether the
empirical study was planned and executed with clear research goals and questions.

4.1 Measures for evaluating Software Product

Criterion 1: Quality characteristics evaluated by the measure. The results indicate that

most of the measures evaluate attributes relating to maintainability. 77% of the measures,

from the result of systematic review actually measure quality attributes relating to

maintainability. In general, these measures are related to modularity, reusability,

analysability, modifiability, testability [6]. Additionally, the measures have also been related

to changeability and stability [7].

Of the total number of measures, 13% measure performance efficiency; such as the platform
efficiency measure, time behaviour, resource utilisation, capacity and the efficiency measure
for the software product [7].

Of the total number of measures, 3% measures quality attributes that are related to the
functional suitability characteristic. These measures are related to functional completeness,
function correctness and functional appropriateness [8]

Of the total number of measures, 1% measures compatibility. The measures that are related to
compatibility are co-existence and interoperability [7]

Of the total number of measures, 2% measures reliability. Mens and Demeyer [9] present the
probability of an error that arises in one component being propagated to other components,
using the error propagation measures. The measures that are related to reliability are fault
tolerance, maturity and recoverability [8].

Of the total number of measures, 2% measures security. [9] propose a measure with which to
compare the safety represented by the structure and composition of software fault trees with
the same root hazard. [8] identifies measures that are related to security as confidentiality,
integrity, non-repudiation, authenticity and accountability.
Of the total number of measures, 2% measures usability. The measures that are related to
usability are appropriateness recognisability, learnability, operability, user error protection,
user interface aesthetics and accessibility [8].

Unfortunately no measure was found for portability which concerns adaptability,

installability and replaceability.

Criterion 2: Quality attributes evaluated by the measures. While considering the type of
attribute under quality evaluated by the measure, most of the measures evaluate internal
attributes (77%). An internal attribute of an entity depends only on the entity. For instance,
the size of software may be considered an internal attribute, since it depends only on the
software. Internal product attributes are mostly measured because they are believed to
influence the external attributes or process attributes (e.g., program complexity is believed to
influence cost). Internal product attributes are therefore easier to measure, but their
measurement is seldom interesting per se, at least from a practical point of view [8]

The external attributes are evaluated in 23% of the cases. The external attribute of software
entity depend on the entity and its context. For instance, the reliability of software depends on
both the program and the environment in which the software is used. External attributes are
usually the ones whose measures have industrial interest. For instance, it is important to
measure the reliability of software during its operational use, so as to assess whether its
quality is sufficiently high. However, one of the factors that accounted for low percentage of
external attributes evaluated was supported by Morasca [6] that external attributes are usually
difficult to measure directly, since they depend on the environment of the software entity.

Criterion 3: Type of measure. The systematic review results show that most of the measures
are derived measures with 83% measures. The remaining 17% measures can be calculated
without employing other measures and that is what is regarded as base measures. This is as a
result of the fact that the attributes are complex and cannot be calculated directly. Base
measures are often used to count the number of elements (e.g., number of class alternative
inclusive variants, number of use case alternative exclusive variants, number of variabilities
in a class diagram, etc). The base measure measures task time, number of tasks, number of
errors made by the user, and proportional value of each missing or incorrect component [10].
Some measures presented here were found to be 17% measures and formed the basis for
calculating other derived measures.
Criterion 4: Result of the measure: With respect to the type of evaluation, mostly, the
measures were calculated in a quantitative manner (94%) while measures that were
determined using qualitative approach was 4% and about 2% combined both quantitative and
qualitative methods. Quantitative and qualitative methods are both rigorous in their own way.
The use of qualitative approach is increasing in the evaluation of informatics. The use of
qualitative approach can potentially enhance user acceptance and ideally avoid system failure
[11]. It is, therefore, believed that having most of the measures with quantitative results is
more acceptable as it will make analysis easier.

Criterion 5: Phase of the life cycle in which the measure is obtained. Most of the measures
were obtained during the Evaluation stage of software development life cycle (39%). The
next measure of the total number of measures is 26% that were related to the Design stage of
the Software Development Life Cycle. The remaining 35% was shared among the other
phases of software development life cycle. It has been unequivocally asserted by Farooq et al
[12] that Software measures are applicable to the whole software development life cycle from
initiation, when cost must be estimated to monitoring the reliability of the end software
product in the field, and the way that product changes overtime with enhancement. Software
metrics hold importance in testing phase, as software testing metrics acts as indicators of
software quality and fault proneness. The design stage is known to be a crucial phase because
it is widely accepted that a good design improves the final software and decreases future
errors. Maybe for this reason, a significant percentage of the measures are focused on the
design stage of the development life cycle. However, the quality assessment in evaluation
phase takes the lion share as it is widely acknowledged that most measures are carried out
during the evaluation stage. However, the other phases of software development life cycle are
equally important since the quality in these phases could be modified when new requirements
are introduced.

Criterion 6: Validation procedure. The result shows that 71% of the measures were not
theoretically validated while 29% of the measures were theoretically validated. With regard
to the empirical validation, 34% measures were empirically validated while 66% were not
empirically validated.

Theoretical Validation involves formalizing intuitive ideas around which there is limited
consensus. To this end, one can use Measurement Theory, which has been developed in the
social sciences, or property-based approaches, which have been used in Mathematics for a
long time. Empirical validation of software measures, i.e., show that a measure is useful
towards some parameter of industrial interest (as opposed to the so-called theoretical
validation of measures which entails providing sufficient evidence that a measure really
quantifies the attribute it purports to measure). This is obtained through the building of
models, based on a number of techniques [6].
The validation confirms that the measure is measuring what it intends to measure and the
results are as expected. Thus, although the systematic review has found many measures, there
is a need to validate them in order to verify them. Moreover, we want to comment that the
significance and relevance of the methods for software validation are often confusing. For
example, many authors claim to perform a case study when in fact they are making a proof of
concept of their proposal [1].

Criterion 7: Tool Support for the measure. The results show that many of the measures do

not have a tool to support their evaluation (47%) while most have tool support (53%). In

some cases, the tool offers a semi-automatic evaluation. In these cases, the measures will be

classified as being automatic because there is a tool to assist the user. There are tools that

extract metrics from software that are widely used as part of static code analysis which acts as

a feedback mechanism for the managers, developers and other stakeholders to improve the

software quality. The software industry and academic research have confirmed the necessity

of such tools and the impact they have on ensuring quality software. Some of these tools

include SD Metrics, Eclipse plug-in, JMetric, JHawk and JDepend [12].

Criterion 8: Current usage. The results show that the authors of the articles commonly

belong to academic institutions mostly colleges or universities. The use of the measure is

therefore often academic (83%). However, as a result of the result of the systematic review,

17% of the measures have been identified to have been employed for industrial use (16%)

4.2 Conceptual Analysis of Software Measures Evaluated

With reference to the data presented in table 5, some of the key attributes will be mapped

with existing knowledge. Out of 96 measures, 3% measures a quality attribute known as

functional suitability. The functional suitability measures include functional completeness,

function correctness and functional appropriateness [7]. Rodríguez et al [13] opine that the

environment where evaluation of functional suitability will be conducted is independent of

the programming language and technology used in developing the software. In addition, they

identified work products required to conduct evaluation of functional suitability as:

requirements specification, functional test cases, soft code and software product run.

Out of the total number of measures, 3% measures quality attributes that are associated with
the performance efficiency characteristic. Performance efficiency is a measure that
determines the degree to which the software product provides appropriate performance,
relative to the amount of resources used, under stated conditions [14]. ISO/IEC 25010 (as
cited in [2] identified that in order to accurately evaluate software, performance efficiency
should be measured with Time behaviour & Resource utilization. Load testing is also useful
for measuring performance efficiency in software evaluation. Stress testing is an important
variation on load testing used to determine the maximum operating capacity of an
application.

Out of the total number of measures, 1% measures compatibility. The compatibility attribute
measures the ability of two or more software components to exchange information and/or to
perform their required functions while sharing the same hardware or software (ISO 25010 as
cited in [14]). Compatibility testing should select a set of configurations (field environments),
where each configuration is assemble of component versions that respects known
dependencies. However a large number of possible configurations and lack of automated
testing support have limited developers to perform compatibility testing on a set of popular
configurations [15]

Moreover, 1% of the total number of measures measure usability. Usability measure aims at
finding the degree to which the software product makes it easy for users to operate and
control it [14]. Software should possess software quality, usability, to the extent that it is
reliable, efficient and human-engineered [16]..
Additionally, 2% of the total number of measures measure reliability. This quality attribute
measures the degree to which the software product can maintain a specified level of
performance when used under specified conditions [15]. The activities under this quality
include evaluating maturity, recoverability and fault tolerance. Software should, therefore,
possesses the characteristic “reliability” to the extent that it can be expected to perform its
intended functions satisfactorily [15].

Furthermore, 2% of the measures are concerned with security quality attribute. Security
quality has measured the protection of system items from accidental or malicious access, use,
modification, destruction, or disclosure [15]. Security issues actually deal with attributes of
software that relate to its ability to prevent unauthorized access, whether accidental or
deliberate, to programs and data [15].

Lastly, 77% of the measures measure maintainability attributes. The highest percentage is not
surprising. Maintainability according to Chandrasekar et al [17] has been described as the
ease with which a software product or component can be modified to correct faults, improve
performance or other attributes, or adapt to a changed environment. They further assert that
maintainability is a multifaceted quality requirement. It incorporates aspects such as
readability and understandability of the source code. Maintainability is also concerned with
testability to some extent, as the system has to be re-validated during the maintenance [17].

5.0 CONCLUSION

The results indicated that most of the measures evaluate attributes relating to maintainability
while other quality attributes such as Functional Suitability, Performance Efficiency,
Compatibility, Usability, Reliability, Security are also important for software measures. It is
therefore grossly evident that a final result of a large number of measures that measure
internal attributes is a good result, since these attributes are easier to measure and can be
measured at design time. Internal attributes should be related to external attributes, since
internal attributes may impact on external attributes Evaluating quality in an efficient manner
throughout the whole life cycle of software is a major challenge, and the existence of
measures to measure internal attributes would greatly help to achieve this goal. Majority of
quality attributes are complex and cannot be calculated directly and may therefore need
derived measures while some measures can be calculated without employing other measures
(base measures). Moreover, both Quantitative and qualitative methods are both rigorous in
their own ways but the use of qualitative approach is increasing in the evaluation of
informatics. Measures have been found to be applied in all stages of software development
life cycle but mostly applied during Evaluation stage and tools support for measures should
be encourage both in academia and industry. It is also important that software measures to be
employed should be validated both theoretically and empirically because the validation
confirms that the measure is measuring what it intends to measure and the results are as
expected.

REFERENCES

[1] S. Montagud and E. Insfran, “A systematic review of quality attributes and measures for
software product lines,”.Software Qual J., (May 2014). [Online]: Available:
https://doi.org/10.1007/s11219-011-9146-7
[2] M. Lee, “Software Quality Factors and Software Quality Metrics to Enhance Software
Quality Assurance”. British Journal of Applied Science & Technology, 4(21), 3069–
3095. (2014). [Online]. Accessed from
http://www.journalrepository.org/media/journals/BJAST_5/2014/Jun/Lee4212014BJAS
T10548_1.pdf
[3] T. Tahir, G. Rasool, and C. Gencel, “A systematic literature review on software
measurement programs” Information and Software Technology, 2016, pp 101–121).
[Online]. Available: https://doi.org/10.1016/j.infsof.2016.01.014.
[4] B. A. Kitchenham and S. Charters, (2007). “Guidelines for performing Systematic
Literature Reviews in Software Engineering”. EBSE :Keele University, 01.
[5] K. Sethi, , Y. Cai, S. Wong, A. Garcia, and C. Sant’Anna. “From Retrospect to
Prospect : Assessing Modularity and Stability from Software Architecture” In Joint
Working IEEE/IFIP Conference on Software Architecture & European Conference on
Software Architecture, 2009, pp. 269–272. [Online]. Available:
https://booksc.org/book/33823172/bbc9f2.
[6] S. Morasca, (2001). “Handbook of Software Engineering and Knowledge Engineering”,
University of Pittsburgh, USA & Knowledge Systems Institute, Ed.(1) [Online].
Available:
https://www.uio.no/studier/emner/matnat/ifi/INF5181/h11/undervisningsmateriale/readi
ng-materials/Lecture-06/morasca-handbook.pdf
[7] R. E. Al-qutaish, & K. T. Sarayreh,. Software Process and Product ISO Standards : A
Comprehensive Survey, November (2016). [Online]. Available:
https://www.researchgate.net/publication/237104465
[8] ISO/IEC: INTERNATIONAL STANDARD ISO/IEC: PRODUCT EVALUATION,
 2011. [Online]. Available: https://www.iso.org/standard/24907.html
[9] T. Mens and S. Demeyer,). “Software Evolution”. New York, USA: Springer-Verlag
Berlin Heidelberg, 2008.
[10] A. Abran and R. E. Al-qutaish, “ ISO 9126 : Analysis of Quality Models and Measures”
in A. Abran (Ed.), Software Metrics and Software Metrology. New York, USA: Wiley-
IEEE Computer Society, 2010. [Online]. Available:
https://doi.org/10.1002/9780470606834.ch10
[11] M. M. Yusof, A. Papazafeiropoulou, R.J. Paul, and L. K. Stergioulas Investigating
evaluation frameworks for health information systems. International Journal of Medical
Informatics, 2008. Vol. 7, pp 377–385.[Online]:
https://doi.org/10.1016/j.ijmedinf.2007.08.004.
[12] N. Kayarvizhy, J. Tessier, F. Sauer, and L. Walton, “Systematic Review of Object
Oriented Metric Tools,” International Journal of Computer Applications, 135(2), 8–13.
(2016). [Online]. Available:
https://pdfs.semanticscholar.org/73c9/df066e77c6bda57f1427b2287bd8688b1b97.pdf
[13] M. Rodríguez, J. R. Oviedo, and M. Piattini, “Evaluation of Software Product
Functional Suitability : A Case Study,” Software Quality Professional Magazine, 18(3),
2016. [Online]. Available:
http://www.aqclab.es/images/AQCLab/Noticias/SQP/software-quality-management-
evaluation-of-software-product-functional-suitability-a-case-study.pdf
[14] J. P. Miguel, D. Mauricio, and G. Rodríguez, “A Review of Software Quality Models
for the Evaluation of Software Products,” International J. of Software Engineering &
Applications (IJSEA), 2014, 5(6), 31–53. [Online]. Available:
https://arxiv.org/ftp/arxiv/papers/1412/1412.2977.pdf
[15] I . Yoon, A.Sussman, A. Memon,, & A. Porter, “Effective and Scalable Software
Compatibility Testing” in Proceedings of the ACM/SIGSOFT International Symposium
on Software Testing and Analysis. Seattle, WA, USA: ISSTA, 2008. [Online].
Available: https://doi.org/10.1145/1390630.1390640
[16] P. Berander et al, Software quality attributes and trade-offs. Blekinge Institute of
Technology, (June), 100. 2005. [Online]. Available:
https://www.researchgate.net/publication/238700270_Software_quality_attributes_and_t
rade-offs_Authors
[17] A. Chandrasekar , M. SudhaRajesh, and P. Rajesh, “A Research Study on Software
Quality Attributes,”. International Journal of Scientific and Research Publications, 4(1),
1–4. 2014. [Online]. Available: http://www.ijsrp.org/research-paper-0114/ijsrp-
p2535.pdf