QUALITY OF SOFTWARE PROCESS OR QUALITY OF

SOFTWARE PRODUCT ?

Mary Lucy S’antana

UNICAMP – Universidade Estadual de Campinas
Departamento de Engenharia Mecanica
Campinas, São Paulo Brazil

Ana Cervigni Guerra

CenPRA – Centro de Pesquisas Renato Archer
Campinas, São Paulo Brazil

ABSTRACT
The field of Information Technology research grows substantially around
the world. In this context, Software plays a fundamental role and its
commercialization goes beyond the territorial limits of the countries that
develop them. In view of its increasing complexity and wide-ranging
action, besides the financial importance for its original country, the quality
of software is a concern that has mobilized in such a way both
governmental efforts as well as private concerns. The norms and models
used by the software organizations always aim at the quality of software
process or software product. The objective of this work is to present an
®
integration proposal of these two approaches: the CMM process model
and the NBR 13596, a Brazilian equivalent of ISO/IEC 9126 that are
concerned about quality characteristics of software product. Activities,
®
goals and work products, of the CMM process areas, were analysed
searching for cases where the definitions, guide lines and quality
characteristics of software product, are embodied in ISO/IEC 9126. The
contributions of this work envolve the increasing efforts that are being
carried out in the software quality, so as to motivate the use of Software
Engineering providing an integrated vision of process and product quality,
besides demonstrating a practical integration possibility. The integration
®
proposed is the simultaneous use of ISO/IEC 9126 and the CMM
process model. The application could be in enterprise which wanted
improve process and product at the same time.
 Introduction

The quality of software is a real concern and efforts have been carried out to
increase the quality of the processes involved in its development and maintenance. The
same concern exists with the software product developed where tests, norms and
methods are used to check for its quality.

However, software quality is still a great concern. A process quality does not
guarantee the quality of the product. A gap is perceived in the efforts that have being
carried out to improve the software quality. The process, which will end up in the
software product, concentrates its efforts in development and maintenance, whereas
the software product quality is almost always focused on the final phase of the process
through quality evaluation of the final software product developed.

As described above, the initiatives to improve software quality are of great value
and deal with its objectives in exemplary way but they act in isolated manners.

The software product quality needs to be a part of the development and

maintenance process concerns in an intense and formal way. The characteristics of
software product quality need to be located and checked in the intermediate software
products during the software process. This procedure minimizes the lack of quality
trends on the final software product and, that a deviation detours from this objective is
detected during its development and maintenance process, thus providing the
necessary changes in the process that will develop a quality software product. A simple
solution to bridge this gap is the simultaneous practice of these two approaches for
®
software quality: process and product. This work uses, the CMM - Capability Maturity
Model for Software (SEI, 2000) as a subsidy for these integration, , that focuses on the
quality of software process, and NBR 13596 (ABNT, 1996) which is a Brazilian
translation Norm of ISO/IEC 9126 (ISO, 1991) dealing with quality trends of a software
product.

CMM® Process Model

®
The CMM - Capability Maturity Model (SEI, 2000) was developed by SEI -
Software Engineering Institute (SEI, 2002) as requested by DoD - Department of
Defense of the USA. This model describes the key elements of an efficient software
process besides indicating an increasing way to changean immature software process,
where it works in an ad hoc form, into a mature, i.e. disciplined software process. In that
way, this model can be used in such a way as to define the Maturity Level of the
organization, taking into account its software process, as also to guide an improvement
process work.

There are five Maturity Levels: 1 - Initial, 2- Repeatable, 3 - Defined, 4 -

Managed and 5 - Optimizing. Each one of these levels acts as a foundation for the
following level and indicates the capability of the organization software process.

In maturity level number 1, the success of the organization depends on the

efforts of the people. Level 2 already possesses a plan for new projects based on its
specifications and experiments of similar latter projects. The existence of a standard

2 ASQ Software Division 12ICSQ

software process for all the organization is the main feature of Level 3 organizations. In
Level 4, the organization keeps a quantitative control of the software process quality and
of the software products produced by it. A Level 5 organization is one that provides a
continuous improvement of its software process in an organized way and without
hindering the good development of the projects that are being carried out.
®
The CMM incorporates planning practices, management, development and
software maintenance. When these practices are carried out, the organization increases
its possibility to attain cost goals , schedule (workcharts), efficiency and quality of final
software product. An immature software organization can change into a mature
organization through carring out the practices of this model.

According to (Paulk, 1999), in an immature software organization, the process is

generally improvised, even when the process is specified, it is not always followed. This
organization is reactive and the managers are usually trying to solve constant problems.
When rigid deadlines are set for the software delivery, normally the functionality, as well
as the product quality as a whole, is engaged.

On the other hand, a mature organization owns its software process

clearlydefined and institutionalized by the organization as a whole. When necessary, the
process is brought up to date and improvements are developed through pilot tests
and/or cost x benefit analyses.

The structure of the CMM is based on the definition of the 5 (five) "Maturity
Levels", previously described, where KPA "Key Process Areas" pose as essential for an
organization in that given Maturity Level. See anexed table

The KPA possesses a set of Practices considered as a Key so that its goal is
attained in the organization. These "Key Practices" are grouped in accordance with its
KPA objective, which can be: "Responsibilities", "Abilities", "Activities", "Measurements
and Analyses" and, "Verifications".

NBR 13596 Norm

The NBR 13596 (ABNT, 1996) edited in 1998 by ABNT (ABNT, 2002) as a
Brazilian equivalent of the ISO/IEC 9126 "Information technology - Software product
evaluation: Quality characteristics and guidelines for their use" (ISO, 1991).

The elaboration of NBR 13596 involved around twenty specialized professionals

from concerns, universities and research centers, who had already applied the concepts
in their professional activities before the final exactly text was approved as a norm
design.

One of the reasons why the Software Quality Study Committee elaborated the
NBR 13596 through the translation of the base text of ISO/IEC 9126, and not to create a
new norm, was the increasing integration of the world-wide economy that called for
quality concepts unification. It could offer a common normative text being used by all
countries.

12 ICSQ ASQ Software Division 3

 NBR 13596 establishes 6 quality trends that a software product must have; it
defines a software product quality model and provides the occasion when the Norm can
be applied.

The model of software product quality was defined to support the objective to
apply the characteristics of quality in a software product evaluation and, it is based on
the definition of the software quality characteristics through its subcharacteristics. Its
use is not a must. The important one, according to this Norm, is the necessity of
following a model in an evaluation.

This Norm can be applied on the following occasions:

Definition of the quality requirements of a software product;

Evaluation of the software specification to verify whether it will meet the

quality requirements during the development;

Description of particulars and implemented software qualities, for

example, in user manuals;

Software product evaluation, before delivery;

Software product evaluation, before acceptance.

The 6 (six) characteristics and the subcharacteristics that define them, according
to NBR 13596, are the following ones:

Functionality

Suitability: The capability of the software product to provide an appropriate set of

functions for specified tasks and user objectives.

Accuracy: To provide the right or accepted results or effects with the required
rate of precision.

Interoperability: To interact with one or more specified systems.

Security: To protect information and data so that unauthorised persons or

systems cannot read or change them and authorised persons or systems are not denied
access thereto.

Functionality Compliance: To adhere to standards, conventions or regulations in

laws and similar prescriptions.

Reliability

Maturity: The capability of the software product to avoid failure as a result of

faults in the software

Fault Tolerance: Capacity to keep performance level in case of failure or

interface violation.

4 ASQ Software Division 12ICSQ

 Recoverability: Capacity to keep the previous performance and data restored
after failures.

Usability

Understandability: Software qualities that show the user´s effort to recognize the
logical concept and its applicability.

Learning ability: Software qualities that indicate user´s effort to learn its
application.

Operability: Software qualities that show the user´s effort for its operation and
operation control.

Efficiency

Time Behavior: Time of answer, processing and functions execution speed.

Resource Behavior: Number of resources used.

Maintainability

Analyzability: Effort necessary to diagnose defficiency and reasons for failures.

Changeability: Effort necessary to carry out modifications and removal of

defects.

Stability: Unexpected absence risks of effect caused by modifications.

Testability: Easily tested.

Portability

Adaptability: Capacity to be adapted to different environments.

Install ability: Effort necessary for installation.

Conformity: In accorddance with standards or portability conventions.

Replacement ability: Capacity and effort necessary to replace another software.

CMM® and NBR 13596 Integration

The contribution, provided for the simultaneous use of a process model and a
Norm that describes the characteristics of a quality software product, is to offer a
practical alternative for software organization. So that those software development and
maintenance will field a quality product, as per internationally recognized parameters.

A strategy that can be adopted is to analyze examples of a software process

model, that has some potential for the use of NBR 13596 concepts.

12 ICSQ ASQ Software Division 5

 ®
Despite CMM model´s v 1.1 being replaced, probably early this year, by
SM
CMMI - Capability Maturity Model Integration (Master, 2000), it is even better indicated
for a work of this nature because it is rather well known and has been worldwide used
since 1993. That causes, a practical application in software organizations to be
immediately feasible thus providing short-range results.
®
The "Activities" group suggested by CMM , that encompasses key practices, is
of great importance in this analysis because they describe 'What' must be done to
establish the software process capability. The "capability of the software process"
defines the limits of expected results when the software process in use is applied.

Upon introducing NBR 13596 in the "Key Practices" of the "Activities", it´s aims
at expanding its objective so that they can also define 'What' must be done to establish
the software product capability.
® ®
CMM CMM +NBR 13596
What must be done Software Process Software Process and Software Product
to establish ... Capability Capability
Focus foreseeing Process Process and Product
results of ...
®
The way of using the NBR 13596 concepts in some examples of the CMM
process model, must be simple and easily implemented. In a Master´s Degree Thesis,
®
(Sant’Ana, 2002) displays more than 30 (thirty) chances of CMM application points of
NBR 13596 concepts use. Some examples of these instances are:

KPA: Requirements Management - a key process area for Level 2: Repeatable

The purpose of Management Requirements is to establish a common

understanding between the customer and the software project the customer's
requirements that will be implemented by the software project.

Considering the use of NBR 13596 in conditions of this KPA, its intention is to
®
add value to the CMM by changing its purpose into: Establishing a mutual agreement
between customer and the software project team of the customer's requirements,
including the software product quality trends , that will be implemented by the software
project".

An instance in this KPA is the "Activity" number 1 that says: "The software
engineering group revises the requirements to be allocated before they are incorporated
into the software project". The "Software Requirements Specification" document must
describe the behavior of the software as expected by the customer.

Considering the use of NBR 13596 in this activity, the software engineering
group revises the requirements to be prior to including them in the software project and
trying to warrat its inclusion in the documment "Software Requirements Specification"
about quality characteristcs of the software product.

6 ASQ Software Division 12ICSQ

KPA: Organization Process Aim - a key process area for Level 3: Defined
®
The purpose of CMM for this KPA is to "establish the organizational
responsibility for software process activities that will improve the organization's overall
software process capability".

The use of NBR 13596 conceptions in accepted situations in this KPA, it

purpose can be described as being "establish the organizational responsibility for
software process activities that improve organization's overall software process and
product capability".

“Activity” number 5 of this KPA states: "New processes, methods, and tools in
limited use in the organization are monitored, appraised and, wherenever appropriate,
transferred to other parts of the organization".

When the tool in na experimental use in the organization is a software, NBR

13596 will contribute contribute by offering a model for evaluating the quality trnds of
this software product. This evaluation can be one of the criteria to check the tool is
appropriate, or not, so as to be used by the organization.

KPA: Defect Prevention - a key process area for Level 5: Optimizing

®
The purpose of Defect Prevention provided by CMM is "to identify the cause of
defects and prevent them from happening".

The aggregate value to this purpose, with the incorporation of NBR 13596 into
software process, is the identification of problems in reaching of goals related to the
software product quality characteristics that is being developed or maintained. It can
prevent the recurrences of problems with software product quality characteristics.

On the “Activity” number 2, the KPA describes that "at the beginning of a
software task, the members of the team performing the task meet to prepare for the
activities of that task and the associatedprevention activities". Considering the use of
NBR 13596 in the team preparation, the quality goals for the software products must be
approached based on rule, the quality characteristics defined by this Norm. In that way,
these goals can be monitored, at moments considered important, during the software
process.

Conclusion
®
According to CMM , the organization must work in partnership with the customer
by understanding his necessities and providing a development or keeping of software
products that establish the customer requirements. It has to be registered and warrant
the allocation of these requirements to the software components.

In accordance with NBR 13596, in order to manage the software quality the
technology is very important to specify and to evaluate both the software product quality
and the software process quality, in na objective and quantitative manner

12 ICSQ ASQ Software Division 7

 ®
Therefore, the CMM and NBR 13596 recognize that the software quality is
highly influenced by the quality of its process and any initiative toward improving the
quality of software product must be applied in the software process improvement.
®
According to (Sant’Ana, 2002), all the 18 KPA, defined for CMM , have instances
that can be indicated as having a potential chance to involve and use the concepts of
NBR 13596 or ISO/IEC 9126 norm. This reflects that, regardless of the organization's
Maturity Level, the chance to use a norm about software product quality in the software
process is always possible, applicable and recommended.

With this simple initiative, the emphasis will be on software quality and no
longer in the process or final product. That is, the best practices in the process must
be carried out to develop and maintain a product that has definite quality trends, and are
widly recognized toward promoting the improvement of the software quality as a whole.

References

Master, S. (2000). Anais of SIMPROS 2000 – Simpósio Internacional de Melhoria de

SM
Software. An Overview of Capability Maturity Model Integration (CMMI ):
version 1.0. (2 ed.). São Paulo. 82 p.

ISO. (1991). ISO/IEC 9126 Information technology – Software product evaluation:

Quality characteristics and guidelines for their use. Genebra. 13 p.

ABNT. (1996). NBR 13596 Tecnologia de informação – Avaliaçao de produto de

software: Caracteristicas de qualidade e diretrizes para o seu uso. Rio de
Janeiro. 10 p.

Paulk, M. C., et al. (1999). The Capability Maturity Model for Software. Pittsburgh: SEI –
Software Engineering Institute. 26 p.

SEI. (2000). The Capability Maturity Model: Guidelines for Improving the Software
Process (14 ed.). EUA: Addison Wesley Longman. 441 p.

CMM. CARNEGIE MELLON UNIVERSITY. SEI – Software Engineering Institute. The

Capability Maturity Model: Guidelines for Improving the Software Process. 14.ed.
EUA: Addison Wesley Longman, 2000. 441 p.

Sant’Ana, M. L. (2002). A M.Ae.S. Thesis. Uma Proposta para Qualidade de Software

Através da Aplicação Integrada do Modelo CMM e da Norma NBR 13596.
Campinas: Faculdade de Engenharia Mecânica, UNICAMP - Universidade
Estadual de Campinas. 124 p.

Annex

LEVEL 2: REPEATABLE
PURPOSE
KPA

Requirements To establish a common understanding between the customer and the

Management software project of the customer’s requirements that will be

8 ASQ Software Division 12ICSQ

 addressed by the software project.
Software Project To establish reasonable plans for performing the software engineering
Planning and for managing the software project.
Software Project To provide adequate visibility into actual progress so that
Tracking and management can take effective actions when the software project’s
Oversight performance deviates significantly from the software plan.
Software Subcontract To select qualified software subcontractors and manage them
Management effectively.
Software Quality To provide management with appropriate visibility into the process
Assurance being used by the software project and of the products being built.
Software To establish and maintain the integrity of the products of the software
Configuration project throughout the project’s software life cycle.
Management

LEVEL 3: DEFINED

PURPOSE
KPA

Organization Process To establish the organizational responsibility for software process

Focus activities that improves the organization’s overall software process
capability.
Organization Process To develop and maintain a usable set of software process assets that
Definition improve process performance across the projects and provide a
basis for cumulative, long-term benefits to the organization.
Training Program To develop the skills and knowledge of individuals so they can
perform their roles effectively and efficiently.
Integrated Software To integrate the software engineering and management activities into
Management a coherent, defined software process that is tailored from the
organization’s standard software process and related process assets,
which are described in Organization Process Definition.
Software Product To consistently perform a well-defined engineering process that
Engineering integrates all the software engineering activities to produce correct,
consistent software products effectively and efficiently.
Intergroup To establish a means for the software engineering group to participate
Coordination actively with the other engineering groups so the project is better
able to satisfy the customer’s needs effectively and efficiently.
Peer Reviews To remove defects from the software work products early and
efficiently.

LEVEL 4: MANAGED
PURPOSE
KPA

Quality Process To control the process performance of the software project

Management quantitatively.
Software Quality To develop a quantitative understanding of the quality of the project’s
Management software products and achieve specific quality goals.

12 ICSQ ASQ Software Division 9

LEVEL 5: OPTIMIZING
PURPOSE
KPA

Defect Prevention To identify the cause of defects and prevent them from recurring.
Technology Change To identify new technologies and transition them into the organization
Management in an orderly manner.
Process Change To continually improve the software processes used in the
Management organization with the intent of improving software quality,
increasing productivity, and decreasing the cycle time for product
development.

10 ASQ Software Division 12ICSQ