Quality by definition calls for requirements/specifications in enough

detail so that the

products produced can be quantitatively measured against those
specifications. Few organizations
are willing to expend the effort to produce requirements/specifications
at the
level of detail required for quantitative measurement.

Lack of knowledge of the principles of quality

No categorization scheme for defects (i.e., naming of defects by type)

Nominal Group Technique

The nominal group technique is a structured, facilitated technique

where all team members
participate by individually ranking ideas, issues, concerns, and
solutions, and then achieve
consensus by combining the individual rankings. Sample ideas that
could be ranked with the
nominal group technique are:
• Which defect is the greatest?
• Who are our customers?
• What are our impediments to quality improvement?
• What new standards are needed?
• What are our key indicators?
Quality Assurance
4-27
• What tool is not being used effectively and how can we increase
usage?
• How do we get a quality tool used?
Nominal grouping uses a round table (verbal) or index card (written)
method for equal participation
of teams or groups. It is a good technique to gather large amounts of
information. The steps for the
nominal group technique are:
1. Generate a list of ideas, issues, concerns, or solutions to prioritize.
Brainstorming can
be used if the list is not readily available.
2. If the list contains more than about 35 items, it may be desirable to
shorten it using
Pareto analysis to make it more manageable.
3. As shown in Table 4-3, record the remaining listed items in a
location visible to the
team, prefacing each item with a letter of the alphabet, such as:
A – list item 1
B – list item 2
C – list item 3
Table 4-3 Results from Nominal Grouping
4. Team members individually rank the list by assigning a number to
each line item. One
represents the lowest (least important) ranking, and higher numbers
signify increasing
importance.
5. Total the rankings of all team members. In this example, item “C” is
the most
important.

RTMs

In-Process Reviews
In-Process reviews are used to examine a product during a specific
time period of its life cycle,
such as during the design activity. They are usually limited to a
segment of a project, with the goal
of identifying defects as work progresses, rather than at the close of a
phase or even later, when
they are more costly to correct. These reviews may use an informal,
semiformal or formal review
format.
Checkpoint Reviews
These are facilitated reviews held at predetermined points in the
development process. The
objective is to evaluate a system as it is being specified, designed,
implemented, and tested.
Checkpoint reviews focus on ensuring that critical success factors are
being adequately addressed
during system development. The participants are subject matter
experts on the specific factors to be
reviewed against, and could include customer representatives,
analysts, programmers, vendors,
auditors, etc. For example, if system performance was identified as a
critical requirement, three
checkpoint reviews might be set up at the end of the requirements,
design, and coding phases to
ensure there were no performance issues before proceeding to the
next phase. Instead of walking
team members through a general checklist (as would be done in a
phase-end review), a designated
performance expert would look specifically at whether performance
requirements were being met.
Phase-End Reviews
Phase-end reviews (also called Decision-Point or Gate reviews) look at
the product for the main
purpose of determining whether to continue with planned activities. In
contrast to the checkpoint
reviews, which focus on critical success factors, phase-end reviews are
more general in nature.
Phase-end reviews are held at the end of each phase, in a formal
review format. Defects found are
tracked through resolution, usually through a defect-tracking system.
Although there may be more,
Guide to the 2006 CSQA CBOK
7-12
the most common phase-end reviews are listed below. Project status,
risks, and non-technical
issues are also reviewed.
Software Requirements Review

Post-Implementation Reviews
Post-implementation reviews (also known as "postmortems") are
conducted in a formal format up
to six months after implementation is complete, in order to audit the
process based on actual results.
They are held to assess the success of the overall process after
release, and to identify any
opportunities for process improvement.
These reviews focus on questions such as: “Is the quality what was
expected?” “Did the process
work?” “Would buying a tool have improved the process?” or “Would
automation have sped up
the process?” Post-implementation reviews are of value only if some
use is made of the findings.
The quality assurance practitioner draws significant insight into the
processes used and their
behaviors.

Using Defects for Process Improvement

Using defects to improve processes is not done by many organizations
today, but it offers one of
the greatest areas of payback. NASA emphasizes the point that any
defect represents a weakness in
the process. Seemingly unimportant defects are, from a process
perspective, no different from
critical defects. It is only the developer’s good luck that prevents a
defect from causing a major
failure. Even minor defects, therefore, represent an opportunity to
learn how to improve the process
and prevent potentially major failures. While the defect itself may not
be a big deal, the fact that
there was a defect is a big deal.
Based on the research team findings, this activity should include the
following:
• Go back to the process that originated the defect to understand what
caused the defect
• Go back to the verification and validation process, which should have
caught the defect
earlier
Not only can valuable insight be gained as to how to strengthen the
review process, these steps
make everyone involved in these activities take them more seriously.
This human factor dimension
alone, according to some of the people the research team interviewed,
can have a very large impact
on the effectiveness of the review process.
NASA takes an additional step of asking the question: “If this defect
could have gotten this far into
the process before it was captured, what other defects may be present
that have not been
discovered?” Thus, not only is the process strengthened to prevent
defects, it is strengthened to find
defects which have been created but not yet discovered. This
aggressiveness should be mandatory
on life-critical systems.

In order to achieve maximum efficiency in software product QA, the center of attention should be on
reusability and maintainability of QA artifacts and resources.