Project Management QA in Brief

1
Project Management Project Management

Week 9 Week 9
Quality Assurance Quality Assurance
Lecture Overview Lecture Overview
What is Quality Assurance?
Do we need a quality management system for
software?
Impact of ISO9000
ISO9001 & ISO9000.3
TickIT
The Capability Maturity Model
ISO9000 Vs CMM
WHAT IS QUALITY ASSURANCE?
Quality assurance is the term used to describe those activities
which ensure that a contractually acceptable product is delivered
by a project - In other words, a quality piece of software is one
which, above all else, meets the customer's requirements.
Quality assurance includes many of the good practices which
we have already described, such as holding reviews, producing a
project plan and documenting activities. It is important to point
out that quality assurance is as much concerned with the
processes used to create a software system as the system itself.
2
The production of almost any piece of software requires a
certain amount of communication between people, from the
smallest program to the largest system.
For example, imagine how much communication would be
involved when a company's entire finance department and
board of directors have to communicate their ideas to a software
developer's project team consisting of maybe 15, 20 or even
more people - Not only is there communication between the
customer's staff and the developer's, but there is a great deal of
communication needed within the developer's organisation
between members of the project team.
In a high-quality product, it is imperative that communication is
effective, or the developer could end up producing a product the
customer does not want, or different members of the project team
could end up producing incompatible program units!
One of the aims of quality assurance, therefore, is to ensure that
this communication is effective, i.e. to ensure that the developer, in
particular the developer's project team, has understood correctly the
customer's requirements.
High-quality software is characterised by many other attributes
which are not directly related to the customer's requirements, such
as:
Efficiency
Reliability
Testability
Maintainability
Usability
3
Efficiency - refers to the behaviour of a system in
relation to the resources of the computer system on
which it executes. Most often the efficiency of a system
is described in terms of its execution speed and storage
use. If a system executes as fast as was specified but
consumes twice as much storage as was expected,
maybe in order to achieve the performance requirement,
it is likely to incur the customer's displeasure.
Reliability - relates to the number of errors in a piece
of software, and hence is a measure of the number of
times a software system fails to perform correctly - One
might expect a system containing 10 errors to meet a
customer's requirements more frequently than one
containing 100. (However, this might not be the case;
just one of the 10 may be more serious than all of the
100.)
Testability - refers to the ease with which a software
system can be tested - For example, if a system contains
program units with a large amount of tortuous logic then
it will be difficult to test; if it is difficult to test its
design or implementation may be poor and it may be
less likely to meet customer requirements.
4
Maintainability - refers to the ease with which a
system can be changed once it is in operation - For
example, it should be straightforward to replace a
function in a system if its system design exhibits loose
coupling and high cohesion.
Usability - is the ease with which the system can
be used; if a system is easy to use, if it has a
consistent and clear model of how it should be used,
then it is less likely to be misused.
This characteristic has much to do with the
human-computer interface which is a topic in itself.
It is not just a matter of aesthetics or taste since
misuse could result in financial loss, environmental
damage, or loss of life.
Quality Management Systems Quality Management Systems
For Software For Software
Quality System: the organisational structure,
responsibilities, procedures, processes and resources for
implementing quality management;
Quality management: that aspect of the overall
management function that determines and implements
the quality policy.
Quality: the totality of features and characteristics of
a product or service that bear on its ability to satisfy
stated and implied needs.
5
A quality system is the organisational structure and so
on that controls and influences the quality of a suppliers
products and services.
Quality is what makes your customer happy - virtually
everything in a software development organisation
influences quality, so in practice the quality system is
the means for managing the software development.
Some examples of what may be parts of a quality system
for software:
Schedule and agenda for executive meetings;
Assignments of authorities and responsibilities in the
company;
Procedures for project management / Templates for
documents;
Procedures for reviews and tests / Procedures for handling
customer complaints / Records of employee training;
Procedures for internal audits;
Procedures for handling changes to specifications and
program / The central product library for software.
The basic requirement of a quality system is that it
works.
If an eternal editor can see that your quality system
works, he or she would probably only be able to find
minor things to criticise.
6
Do We Need A Quality Management Do We Need A Quality Management
Systems For Software Systems For Software
So why bother with a quality system? Well, there are a
few possible reasons:
You may want to modify the software;
Your customers may require that your software works;
You may have to convince your customer beforehand
that you will be able to deliver a suitable product;
You may have to hire programmers who have families
and private lives, and who are not prepared to work day
and night for three years;
The original whiz kids may quit and start a business of
their own;
You may even have product liabilities.
If you want to know what you'll get and when you
get it, you had better have a quality system of some
sort.
An intelligent application of the requirements in ISO
9001 will make a software supplier better, the
operative word here being intelligent.
Many of the principles of quality management can
be usefully applied to software development, provided
the particular features of software quality problems are
borne in mind.
The problems of software are not unique. User
requirements are often highlighted as the worst problem
area.
Juran highlighted this area in manufacturing 40 years
ago - complexity requires careful management in all
contexts, but software cannot claim a monopoly here as
quality problems of software development represent a
particular blend of problems, rather than something
completely different.
7
It is suggested that there are four principal aspects to a QMS
for software development:
Development procedures.
This includes the use of design and development
methodologies and tools, testing and associated staff
training.
Quality control.
This includes many activities for the monitoring of quality
during development, e.g. planning, progress meetings, user
sign-off, configuration management, change control,
documentation control, design reviews, code walkthroughs,
error reporting, system testing and acceptance testing.
Quality improvement.
This includes all activities aimed at establishing a
human quality culture amongst the staff, such as
quality improvement teams, quality circles and so
on.
Quality assurance.
Where a quality system is in place, QA becomes the
monitoring of the system itself to ensure that it is
being carried out correctly.
A quality system is designed to move maintenance to
earlier in the lifecycle - this will lead to a reduction in both
time and effort.
All these benefits must be shown to happen in practice -
Once quantified in financial terms, the benefits must be
weighed against the cost, which may be considered in two
stages.
First there is the cost of introducing a quality
management system - Once established there are specific
costs associated with certification.
8
In 1988 the cost of introducing a QMS to a typical
supplier (a company with 50-100 employees and
turnover of 3,000,000) was estimated at 230,000 a
year - In addition, set-up costs were estimated at
120,000.
Standards are generally defined in terms of a model of best
practice, against which all others may be compared.
The role of standards is not to build the proverbial better
mousetrap, but to ensure conformance to the standard.
The standard never improves quality directly nor ensures
perfection - It should, however, ensure that the correct procedures
are in place and being carried out.
The standard provides a model, and the accreditation (audit)
procedure the incentive to ensure that things are done directly.
An audit is an evaluation of your quality system and
documentation - Your organisation may undergo several types
of audits:
First-party audits
Second-party audits
Third-party audits
The first-party audit is an internal quality system audit
performed by the supplier (your organisation) on its own quality
system.
The second-party audit is a quality system audit performed by
your customer on the (supplier & your organisation).
Quality Audits Quality Audits
9
The third-party audit is a quality system audit
performed by an auditor on the supplier in order to
achieve certification for one of the ISO 9000 Standards -
The third-party auditor must be independent of both the
customer and the supplier - as Third-party audits cannot
be performed by the customer or the supplier.
What doestheauditorlookfor whenperformingathird-partyaudit?
Doyour documentsconform
totherequirementsoftheStandard?
Doyouroperationsconform
tothe documents?
Doyourrecordsshowpast conformance
toyourdocuments?
Standard
On-going
operation
Qsuality
records
Controlled
Documents
The accreditation (audit) process provides the number of
potential benefits to the supplier:
it provides external validation to see whether the
investment made in the QMS is being effective;
it gives the supplier and their quality system
external credibility;
it allows the supplier to sell to those customers
who insist on accreditation as a condition of tender;
it qualifies the supplier to be included in buyers
guides compiled by the accreditation bodies and
circulated to potential customers.
10
The cost of creating a satisfactory QMS to one of the
ISO 9000 series standards is small in relation to the cost
of setting up the QMS in the first place. The figures for
a typical supplier in 1988 were estimated at 10,500
initially, with a further annual cost of 4,500.
The advantage of third-party over second-party
accreditation (audits) is that the supplier only has to
satisfy one accreditor/auditor (rather than, say, to have to
justify ones quality practices to six different customers).
The Impact of ISO9000 The Impact of ISO9000
The ISO 9000 standard with which we are concerned
is ISO 9001, since it applies to quality assurance in
design, development, production, installation and
servicing.
This standard was written for the manufacturing
industry, and this poses some problems when applying it
to the development and maintenance of software.
ISO 9001 and ISO 9000.3 ISO 9001 and ISO 9000.3
In manufacturing (for example, kettles), design is a
relatively minor activity - Instead, the cost for each
manufactured item is notable, so when a few items have been
produced, production is by far the major part of the activity.
Therefore, when we talk about quality or productivity
problems and improvements in manufacturing, we tend to focus
on production.
Software development, however, is nearly 100% design.
Production means to copy executable code to diskettes, tapes, or
ROMS, and is performed and checked automatically. So, when
talking about quality & productivity, we focus on design.
11
The functionality and complexity of software and complex
electronics are many orders of magnitude greater than those
of ordinary appliances.
Thus the need for control is greater in software
development than in those of other appliances; at the same
time, that control is more difficult to define and apply.
ISO 9001 covers design, but it focuses on production - even
for a production expert the text in the standard is brief and
needs explanation - to apply it to software development the
standard must be interpreted and explained still further.
In 1991 ISO published a guide for this purpose ISO
9000.3, Quality Management Systems Part 3 Guideline
for the application of ISO 9001 to the development, supply
and maintenance of software.
The guideline is organised into three main groups:
General requirements on the company and its
management;
Requirements on projects and the maintenance
phase;
Requirements on supporting activities (those
activities independent of phases);
Software engineers complain that ISO does not tell us how
to develop quality software. - It was never intended to.
The standard is aimed solely at being a tool for the
customer.
Basically ISO 9001 makes the supplier implement the
basic management of software development, and the
standard then enforces visibility, so that the customer
can see what the developers are doing and judge it.
In practice, ISO9001 and 9000.3 can be used as guides
for the suppliers management, helping them to control
development and gain insight into what is really going
on.
12
By the end of the 80s the ISO standards had become
quite popular in Europe - Manufacturers were certified to the
ISO standards in increasing numbers.
Some had considerable computer departments developing
software for internal use and the certification of these
departments varied depending on the auditors competence
and the attitude of the certification body.
Companies with software as part of their products started
to join in, and soon pure software houses joined in. Industry
was becoming apprehensive about ISO 9001 certification of
software development and maintenance.
It was feared that different certificates might have
very different values that the standard was non-
standard.
The British software industry, together with the
British Department of Trade and Industry, launched an
initiative to amend the situation and called it TickIT.
The goal was to establish effective and unified
certification of software development and maintenance.
TickIT is a system for certifying software development
organisations to ISO 9001.
It comprises of 6 elements:
An interpretation of ISO 9001 for software;
A standard set of requirements on the competence and
behaviour of certification auditors;
A standardised training course for certification auditors;
A registration scheme for certified auditors;
A system for accrediting certification bodies for
conducting TickIT certifications;
A logo to be used on certificates to show TickIT
certification.
TickIT TickIT
13
The British National Accreditation Council for
Certification Bodies is the only national authority issuing
accreditation to certification bodies.
Even companies in Sweden and the USA are
accredited by NACCB.
TickIT TickIT
What's really special about TickIT certification is the
auditors.
TickIT auditors are registered by the International
Register of Certificated Auditors (IRCA) in London.
TickIT auditors come in three levels:
Provisional 'TickIT Auditor,
Senior TickIT Auditor,
Lead TickIT Auditor.
TickIT Auditors TickIT Auditors
In order to become one, you have to fulfil several
requirements:
You must yourself have worked for at least three years in
software development, including all different types of work.
You must have successfully concluded an approved one-
week TickIT auditor's course ending with a formal
examination.
You must have experience as a manager.
To become Senior or Lead TickIT auditor, you must have
experience in conducting and leading, respectively, TickIT
certifications.
There are further requirements regarding your personal
attributes.
14
IRCA shows 0 signs of taking the requirements for
TickIT auditors seriously.
In 1993, it was reported that 15% of the applicants for
registration as TickiT auditors were not called for an
interview, and of those interviewed, 25% failed.
All this means is that when you apply for TickIT
certification you know that your software development
and maintenance procedures will be judged by well-
trained auditors with personal experience in software
development.
The certificate you just received and put up on the
boardroom wall is valid for a certain period of time,
usually three years.
After that time, you will have to apply for certification
and do it all over again - Hopefully, the process will be
much simpler this time.
So when you have received your certificate the
auditors will be back soon - In your contract with the
certification body, it is stated that they will do regular
audits twice a year during the time of the certificate.
Maintaining The Certificate Maintaining The Certificate
Often an organisation is kept on its toes until the
certification is done, but then the reaction comes, and the
disciplined of working is replaced with the usual sloppiness.
This is the reason why the certification body will be back to
check that your company is not one of these organisations.
These regular surveillance audits are less comprehensive
than the certification audit, and usually the auditors
concentrate on different areas of your quality system each
time.
15
During the term of the certificate, you can still lose it
- In some situations, the certification body has the
obligation or authority to withdraw your certificate.
Examples include:
If you fail to correct within the prescribed time
a non-conformance found in a surveillance audit;
If you use the certificate improperly in your
marketing (e.g. indicating that this is a certificate
of the quality of your products);
If you dont pay the bills from the certification
body.
Summary of ISO 9000:
It is concerned with what you do in software
development, not how you do it;
It provides some direction, however, not necessarily
sufficient direction;
It is predominantly a tool for software buyers, not
builders.
One of the most promising current methods of
managing the development of high quality software
systems can be found in the work of the American
Software Engineering Institute (SEI) at Carnegie Mellon
University;
The SEI has worked for a number of years to set a
standard for managing American software development
from all over the world to American industry.
The Capability Maturity Model The Capability Maturity Model
16
The Capability Maturity Model(CMM) provides a
graduated set of software development process goals, a
scale for assessing the maturity levels of existing
software processes, and a programme to drive process
improvement.
The underlying principles of the CMM stand behind
all current approaches to software development process
assessment and improvement, especially ISO.
The Capability Maturity Model for Software
The Capability Maturity Model for Software describes the
principles and practices underlying software process maturity
and is intended to help software organizations improve the
maturity of their software processes in terms of an evolutionary
path from ad hoc, chaotic processes to mature, disciplined
software processes.
The CMM is organized into five maturity levels - A maturity
level is a well-defined evolutionary plateau toward achieving a
mature software process. Each maturity level provides a layer in
the foundation for continuous process improvement.
The Five Maturity Levels
The following characterizations of the five maturity levels
highlight the primary process changes made at each level:
1) Initial The software process is characterized as ad hoc, and
occasionally even chaotic. Few processes are defined, and
success depends on individual effort and heroics.
2) Repeatable Basic project management processes are
established to track cost, schedule, and functionality. The
necessary process discipline is in place to repeat earlier
successes on projects with similar applications.
17
The Five Maturity Levels
3) Defined The software process for both management and
engineering activities is documented, standardized, and
integrated into a standard software process for the organization
- All projects use an approved, tailored version of the
organization's standard software process for developing and
maintaining software.
4) Managed Detailed measures of the software process and
product quality are collected. Both the software process and
products are quantitatively understood and controlled.
5) Optimizing Continuous process improvement is enabled by
quantitative feedback from the process and from piloting
innovative ideas and technologies.
Key Process Areas
Except for level 1, each maturity level is decomposed into
several key process areas that indicate the areas an organization
should focus on to improve its software process.
Key process areas identify the issues that must be addressed
to achieve a maturity level - Each key process area identifies a
cluster of related activities that, when performed collectively,
achieve a set of goals considered important for enhancing
process capability.
The key process areas and their purposes are listed below -
The name of each key process area is followed by its two-letter
abbreviation.
By definition there are no key process areas for level 1.
The key process areas at level 2 focus on the software
project's concerns related to establishing basic project
management controls, as summarized below:
Requirements Management (RM)
Establish a common understanding between the customer and
the software project of the customer's requirements that will
be addressed by the software project.
Software Project Planning (PP)
Establish reasonable plans for performing the software
engineering and for managing the software project.
18
Software Project Tracking and Oversight (PT)
Establish adequate visibility into actual progress so that
management can take effective actions when the
software project's performance deviates significantly
from the software plans.
Software Subcontract Management (SM)
Select qualified software subcontractors and manage
them effectively.
Software Quality Assurance (QA)
Provide management with appropriate visibility into the
process being used by the software project and of the products
being built.
Software Configuration Management (CM)
Establish and maintain the integrity of the products of the
software project throughout the project's software life cycle.
The key process areas at level 3 address both project and
organizational issues, as the organization establishes an
infrastructure that institutionalizes effective software
engineering and management processes across all projects, as
summarized below:
The key process areas at level 3 address both project
and organizational issues, as the organization establishes
an infrastructure that institutionalizes effective software
engineering and management processes across all
projects, as summarized below:
19
Organization Process Focus (PF)
Establish the organizational responsibility for software
process activities that improve the organization's overall
software process capability.
Organization Process Definition (PD)
Develop and maintain a usable set of software process
assets that improve process performance across the
projects and provide a basis for cumulative, long-term
benefits to the organization.
Training Program (TP)
Develop the skills and knowledge of individuals so they
can perform their roles effectively and efficiently.
Integrated Software Management (IM)
Integrate the software engineering and management
activities into a coherent, defined software process that is
tailored from the organization's standard software
process and related process assets.
Software Product Engineering (PE)
Consistently perform a well-defined engineering process
that integrates all the software engineering activities to
produce correct, consistent software products effectively
and efficiently.
Inter-group Coordination (IC)
Establish a means for the software engineering group to
participate actively with the other engineering groups so
the project is better able to satisfy the customer's needs
effectively and efficiently.
20
Peer Reviews (PR)
Remove defects from the software work products early
and efficiently- An important corollary effect is to
develop a better understanding of the software work
products and of the defects that can be prevented.
The key process areas at level 4 focus on establishing
a quantitative understanding of both the software
process and the software work products being built, as
summarized below:
Quantitative Process Management (QP)
Control the process performance of the software project
quantitatively.
Software Quality Management (QM)
Develop a quantitative understanding of the quality of the project's
software products and achieve specific quality goals.
The key process areas at level 5 cover the issues that both the
organization and the projects must address to implement
continuous and measurable software process improvement, as
summarized below:
Defect Prevention (DP)
Identify the causes of defects and prevent them from recurring.
Technology Change Management (TM)
Identify beneficial new technologies (i.e., tools, methods, and
processes) and transfer them into the organization in an orderly
manner.
Process Change Management (PC)
Continually improve the software processes used in the
organization with the intent of improving software quality,
increasing productivity, decreasing the cycle time for PD.
21
Common Features
For convenience, each of the key process areas is
organized by common features - The common features
are attributes that indicate whether the implementation
and institutionalization of a key process area is
effective, repeatable, and lasting.
The five common features, followed by their two-
letter abbreviations, are listed below:
Common Features
Commitment to Perform (CO)
Describes the actions the organization must take to
ensure that the process is established and will endure -
Includes practices on policy and leadership.
Ability to Perform (AB)
Describes the preconditions that must exist in the
project or organization to implement the software
process competently - Includes practices on resources,
organizational structure, training, and tools.
Common Features
Activities Performed (AC)
Describes the roles and procedures necessary to
implement a key process area. Includes practices on
plans, procedures, work performed, tracking, and
corrective action.
Measurement and Analysis (ME)
Describes the need to measure the process and analyze
the measurements. Includes examples of measurements.
22
Common Features
Verifying Implementation (VE)
Describes the steps to ensure that the activities are
performed in compliance with the process that has
been established. Includes practices on management
reviews and audits.
Clearly there is a strong correlation between ISO 9001
and the CMM, although some issues in ISO 9001 are not
covered in the CMM, and some issues in the CMM are
not addressed in ISO 9001.
The levels of detail differ significantly: chapter 4 in
ISO 9001 is about five pages long; sections 5, 6, and 7 in
ISO 9000-3 comprise about 11 pages; and the CMM is
over 500 pages long.
There is some judgment involved in deciding the
exact correspondence, given the different levels of
abstraction.
ISO 9001 Vs CMM ISO 9001 Vs CMM
The clauses in ISO 9001 with no strong relationships
to the CMM key process areas, and which are not well
addressed in the CMM, are purchaser-supplied product
(4.7) and handling, storage, packaging and delivery
(4.15).
The clause in ISO 9001 that is addressed in the CMM
in a completely distributed fashion is servicing (4.19) -
The clauses in ISO 9001 for which the exact
relationship to the CMM is subject to significant debate
are corrective action (4.14) and statistical techniques
(4.20).
23
The biggest difference, however, between these two
documents is the emphasis of the CMM on continuous
process improvement. ISO 9001 addresses the minimum
criteria for an acceptable quality system.
It should also be noted that the CMM focuses strictly
on software, while ISO 9001 has a much broader scope:
hardware, software, processed materials, and services
[Marquardt91].
The biggest similarity is that for both the CMM and
ISO 9001, the bottom line is Say what you do; do what
you say. The fundamental premise of ISO 9001 is that
every important process should be documented and
every deliverable should have its quality checked
through a quality control activity.
ISO 9001 requires documentation that contains
instructions or guidance on what should be done or how
it should be done.
The CMM shares this emphasis on processes that are
documented and practiced as documented - Phrases such
as conducted according to a documented procedure
and following a written organizational policy
characterize the key process areas in the CMM.
The CMM also emphasizes the need to record
information for later use in the process and for
improvement of the process - This is equivalent to the
quality records of ISO 9001 that document whether or
not the required quality is achieved and whether or not
the quality system operates effectively.
24
This statement is controversial in itself. Some
members of the international standards community
maintain that if you read ISO 9001 with insight (between
the lines so to speak), it does address continuous process
improvement.
There is faith that weaknesses will improve over time,
especially given regular surveillance audits - Corrective
action can be interpreted in this way, although that may
not be consistently done today - This will undoubtedly
be one of the major topics for the next revision
cycle for ISO 9001.
Lecture Overview Lecture Overview
What is Quality Assurance?
Do we need a quality management system for
software?
Impact of ISO9000
ISO9001 & ISO9000.3
TickIT
The Capability Maturity Model
ISO9000 Vs CMM

Project Management QA in Brief

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Project Management QA in Brief

Uploaded by

Copyright:

Available Formats

1

Project Management Project Management

You might also like