When Does Quality Assurance Take Formal Technical Reviews

Place? The following are guidelines for successful technical reviews

Conducted at different stages in development (filters)
Traditionally, quality assurance was the final step of software
Include 3 to 5 people (1 should serve as leader and another as
development, just before release of the product (“quality checking”)
recorder)
Advance preparation (no more than 2 hrs however)
Quality has to be planned in from the beginning and continuously
monitored Limit meeting’s length to 2 hours
Review the product, not the producer
The later bugs are found, the costlier they are to fix
Limit debate and rebuttal
Defect amplification model Identify problems; do not try to solve them
Take written notes
Quality cannot be retrofitted after the fact – it must be built-in
Use an agenda and checklist
Software quality assurance is an umbrella activity that spans the
Review earlier reviews if applicable
entire product development
Result = no change, minor change, major change

David Hedley 2 COMSM1401 David Hedley 4 COMSM1401

What Is Quality? How Is Quality Assured?

Difficult to define precisely. However, worth striving for as it may
Give the company an edge over competitors
Save the company from costly errors (e.g., late delivery,
customers’ law suits)
In the context of software development, quality has to do with
Meeting requirements
Degree of excellence and refinement of a product
Planning for quality (set of methods and tools)
Monitoring and controlling quality (formal technical reviews)
Testing (ensures effective error detection)
Setting of standards and procedures
Managing and controlling change
Developing and collecting relevant metrics

This latter aspect may be quantified by a variety of quality factors

correctness, reliability, efficiency, integrity, usability, maintainability,
flexibility, testability, portability, reusability, and interoperability

David Hedley 1 COMSM1401 David Hedley 3 COMSM1401

Quality Standards and Procedures
The BSI operates a scheme of registering companies that are
assessed as being capable of working to specification
The relevant BSI standard is BS 5750. It is equivalent to ISO 9001
and other similar European norms
BS 5750 covers all products and services. It strongly emphasises
management responsibilities for ensuring
Definition of company policy and general standards
Enforcement of company standards
Project-oriented work organisation, with quality control built into
project planning
Management reviews of project procedures
Control of contract design, documentation, purchasing,
development and testing methods, ...
David Hedley 6 COMSM1401
Formal Technical Reviews
Record kept of the meeting serves as a basis for implementing
changes
Review process is completed once all changes have been made
to satisfaction (i.e., a no change consensus has been reached)
A completed formal technical review on some part of the development
(e.g., the design of a particular module) is like a quality seal placed
on it
It means that, as far as those involved are concerned, there are no
problems with that part
David Hedley 5 COMSM1401
Software Quality Metrics
Attempt to quantify the manifestations of quality
Halstead’s Software Science
– Aims to answer question: “Is there any measurable difference
between code written by an “expert” and by a “novice”?”
– Number of operators and operands, and their occurrences
– Measures program length, program volume (num. bits
necessary to specify a program), program level (complexity)
Cyclomatic complexity
– Number of regions in planar graph
– Complexity is edges nodes + 1
– Complexity of 10-15 desired to minimise module changes
– Measures testing difficulty
David Hedley 8 COMSM1401
Quality Standards and Procedures
Many UK software houses are now registered under BS 5750.
The principles adopted by these companies include
Employment of intelligent, qualified staff with skills and motivation
All work and organisation focused in projects
Agreeing and documenting a specific quality strategy for each
project
Thorough planning of all stages
Formal definition of requirements, to be agreed with client
Formal reviews, to trap errors as soon as possible
Evolution of methods and tools to improve production process and
assist quality control
David Hedley 7 COMSM1401
Proof of Correctness Risk
Programs are mathematical objects
1. The program is viewed as a sequence of instructions that
implements some specified function
Risk identification
2. At selected statements s1, ..., sn, the designer can assert that
specific conditions c1, ..., cn are true Risk projection
Risk assessment
Risk management
Proof of correctness runs as follows
Risk monitoring
Program statements between si and si+1 are shown to cause the
asserted condition ci to be transformed into ci+1
The above step is performed incrementally

Proofs of correctness improve the quality of the product

They also have a preventive role in software quality assurance

David Hedley 10 COMSM1401 David Hedley 12 COMSM1401

Software Quality Metrics Statistical Quality Assurance

Software Maturity Index from IEEE Standard 982.1-1988
– Number of modules added, deleted or modified in each release
– Measures the stability of a product (how much it changes)
Others
– Maintainability. Measured by MTTC (time needed to process a
change request, implement it and redistribute)
– Reliability (probability of failure free operation in a specified
environment for a specified time). Measured by MTBF = MTTF
+ MTTC
– Availability (probability of operating according requirements at
a given point in time). Measured by MT TMT TF
F+MT TC 100%
Statistical quality assurance reflects a growing trend throughout
industry to become more quantitative about quality
1. Collect and categorise information about software defects
2. Trace each defect back to its underlying cause (e.g., design error)
3. Identify the vital few (80/20 rule: 80% of the defects can be traced
to 20% of the possible causes)
4. Correct the problems that have caused the vital defects
Spend your time focusing on things that really matter
But first be sure that you understand what really matters
Statistical quality assurance improves the quality of the process
It provides a statistical certification of the software’s quality

David Hedley 9 COMSM1401 David Hedley 11 COMSM1401

Risk projection Risk management
Estimate likelihood
Gauge consequences Develop strategy for each risk to either:
– Nature: what type of problems will arise – Avoid risk entirely
– Scope: severity – Minimise impact of risk
– Timing: when and for how long
Incorporate risks into project plan
Weight risks by their impact on the success of the project.
Benefit of risk management must outweigh cost
Risk projection is a set of triples [ri li xi] where
r = risk, l = likelihood, x = perceived impact

David Hedley 14 COMSM1401 David Hedley 16 COMSM1401

Risk identification Risk assessment

Project: Staffing / schedule / budget etc
– Barbara is trying for a baby
– John and Mary are away on holiday together Compute the likelihood of project success.
– Rumours of strikes in the hardware division P(pro jectsuccess pro jectedrisklevel)

– Building catches fire
– World War 3 starts Examine accuracy of estimates
Technical: tools / design / maintenance Prioritise risks
– Project complexity underestimated (in debugging hell?) Start thinking of ways to control/avoid risk
– Technology obsolete Form break points: go/no-go decisions
– Requirements change mid-project Attempt to form relationships between risk triples
Business: marketing / demand etc
– Competitors developing similar product
– Poorly educated sales force
– Company strategy changes
David Hedley 13 COMSM1401 David Hedley 15 COMSM1401
Monitoring the Project Coping with Slippage
Assuming that effective monitoring detects slippage from the plan,
1. Take the plan seriously! early reaction is essential.
What choices are there?
2. Plan for regular reviews
Stick to plan, but
at well defined milestones rearrange workload
3. Record the outcome of each review get more resources or effort
Alter the plan
note tasks that have been completed
Move milestone date
define actions to catch up where progress has fallen behind plan
lower aims
Planning cannot be perfect - assume that you will have to re-plan
Unacceptable responses are:
from time to time as the project progresses.
Wait and see if there is really a problem
Hope the problem will go away

David Hedley 18 COMSM1401 David Hedley 20 COMSM1401

Risk monitoring Slippage

Question: How do slippages occur?

Assess whether risk did actually occur Answer: Gradually.
Adjust risk of overall project success
Collect information that can be used in the future A series of small delays
illness, machines crash, missing manuals, unexpected technical
Attempt to allocate blame (which risks caused which problems)
difficulties, delay in arrival of software/hardware, laziness,
unexpected interruptions, ...)
suddenly, the problem is acute

Anticipate delays before they become serious, and pick up the pace.

David Hedley 17 COMSM1401 David Hedley 19 COMSM1401

Re-planning Options (2)
Rearrange workplan
possible more efficient organisation of existing resources. Avoid
bottlenecks etc.
Move (delay) a milestone:
Delay non-critical tasks first.
If project has to be delayed, concentrate on retaining quality.
Lower ambitions
If required changes cannot be achieved, or delay would cost too
much, the project goals have to be reduced.
Try to do so without losing quality.

David Hedley 22 COMSM1401

Re-planning Options (1)

Extra effort:
Overtime
Increase work rate
Danger with these options is that quality may suffer if people are
overworked.
More manpower: Dangerous! Extra manpower means:
Training new members
Extra communication time
More partitioning of work, implies more interfaces, more testing
etc.
(Brooks’ Law: Adding extra manpower to a late project makes it later.)

David Hedley 21 COMSM1401