You are on page 1of 6

When Does Quality Assurance Take Formal Technical Reviews

Place? The following are guidelines for successful technical reviews

Conducted at different stages in development (filters)
Traditionally, quality assurance was the final step of software
Include 3 to 5 people (1 should serve as leader and another as
development, just before release of the product (“quality checking”)
Advance preparation (no more than 2 hrs however)
Quality has to be planned in from the beginning and continuously
monitored Limit meeting’s length to 2 hours
Review the product, not the producer
The later bugs are found, the costlier they are to fix
Limit debate and rebuttal
Defect amplification model Identify problems; do not try to solve them
Take written notes
Quality cannot be retrofitted after the fact – it must be built-in
Use an agenda and checklist
Software quality assurance is an umbrella activity that spans the
Review earlier reviews if applicable
entire product development
Result = no change, minor change, major change

David Hedley 2 COMSM1401 David Hedley 4 COMSM1401

What Is Quality? How Is Quality Assured?

Difficult to define precisely. However, worth striving for as it may
Give the company an edge over competitors
Save the company from costly errors (e.g., late delivery, Planning for quality (set of methods and tools)
customers’ law suits) Monitoring and controlling quality (formal technical reviews)
Testing (ensures effective error detection)
In the context of software development, quality has to do with Setting of standards and procedures
Meeting requirements Managing and controlling change
Degree of excellence and refinement of a product Developing and collecting relevant metrics

This latter aspect may be quantified by a variety of quality factors

correctness, reliability, efficiency, integrity, usability, maintainability,
flexibility, testability, portability, reusability, and interoperability

David Hedley 1 COMSM1401 David Hedley 3 COMSM1401

Quality Standards and Procedures Software Quality Metrics
The BSI operates a scheme of registering companies that are
Attempt to quantify the manifestations of quality
assessed as being capable of working to specification
Halstead’s Software Science
The relevant BSI standard is BS 5750. It is equivalent to ISO 9001
and other similar European norms – Aims to answer question: “Is there any measurable difference
between code written by an “expert” and by a “novice”?”
BS 5750 covers all products and services. It strongly emphasises – Number of operators and operands, and their occurrences
management responsibilities for ensuring – Measures program length, program volume (num. bits
Definition of company policy and general standards necessary to specify a program), program level (complexity)
Enforcement of company standards Cyclomatic complexity
Project-oriented work organisation, with quality control built into – Number of regions in planar graph
project planning – Complexity is edges nodes + 1
Management reviews of project procedures – Complexity of 10-15 desired to minimise module changes
Control of contract design, documentation, purchasing, – Measures testing difficulty
development and testing methods, ...

David Hedley 6 COMSM1401 David Hedley 8 COMSM1401

Formal Technical Reviews Quality Standards and Procedures

Many UK software houses are now registered under BS 5750.
Record kept of the meeting serves as a basis for implementing
The principles adopted by these companies include
Review process is completed once all changes have been made Employment of intelligent, qualified staff with skills and motivation
to satisfaction (i.e., a no change consensus has been reached) All work and organisation focused in projects
Agreeing and documenting a specific quality strategy for each
A completed formal technical review on some part of the development project
(e.g., the design of a particular module) is like a quality seal placed Thorough planning of all stages
on it Formal definition of requirements, to be agreed with client
It means that, as far as those involved are concerned, there are no Formal reviews, to trap errors as soon as possible
problems with that part Evolution of methods and tools to improve production process and
assist quality control

David Hedley 5 COMSM1401 David Hedley 7 COMSM1401

Proof of Correctness Risk
Programs are mathematical objects
1. The program is viewed as a sequence of instructions that
implements some specified function
Risk identification
2. At selected statements s1, ..., sn, the designer can assert that
specific conditions c1, ..., cn are true Risk projection
Risk assessment
Risk management
Proof of correctness runs as follows
Risk monitoring
Program statements between si and si+1 are shown to cause the
asserted condition ci to be transformed into ci+1
The above step is performed incrementally

Proofs of correctness improve the quality of the product

They also have a preventive role in software quality assurance

David Hedley 10 COMSM1401 David Hedley 12 COMSM1401

Software Quality Metrics Statistical Quality Assurance

Statistical quality assurance reflects a growing trend throughout
Software Maturity Index from IEEE Standard 982.1-1988 industry to become more quantitative about quality
– Number of modules added, deleted or modified in each release 1. Collect and categorise information about software defects
– Measures the stability of a product (how much it changes) 2. Trace each defect back to its underlying cause (e.g., design error)
3. Identify the vital few (80/20 rule: 80% of the defects can be traced
– Maintainability. Measured by MTTC (time needed to process a
to 20% of the possible causes)
change request, implement it and redistribute)
– Reliability (probability of failure free operation in a specified 4. Correct the problems that have caused the vital defects
environment for a specified time). Measured by MTBF = MTTF
+ MTTC Spend your time focusing on things that really matter
– Availability (probability of operating according requirements at But first be sure that you understand what really matters
a given point in time). Measured by MT TMT TF
F+MT TC 100%
Statistical quality assurance improves the quality of the process
It provides a statistical certification of the software’s quality

David Hedley 9 COMSM1401 David Hedley 11 COMSM1401

Risk projection Risk management
Estimate likelihood
Gauge consequences Develop strategy for each risk to either:
– Nature: what type of problems will arise – Avoid risk entirely
– Scope: severity – Minimise impact of risk
– Timing: when and for how long
Incorporate risks into project plan
Weight risks by their impact on the success of the project.
Benefit of risk management must outweigh cost
Risk projection is a set of triples [ri li xi] where
r = risk, l = likelihood, x = perceived impact

David Hedley 14 COMSM1401 David Hedley 16 COMSM1401

Risk identification Risk assessment

Project: Staffing / schedule / budget etc
– Barbara is trying for a baby
– John and Mary are away on holiday together Compute the likelihood of project success.
– Rumours of strikes in the hardware division P(pro jectsuccess pro jectedrisklevel)

– Building catches fire
– World War 3 starts Examine accuracy of estimates
Technical: tools / design / maintenance Prioritise risks
– Project complexity underestimated (in debugging hell?) Start thinking of ways to control/avoid risk
– Technology obsolete Form break points: go/no-go decisions
– Requirements change mid-project Attempt to form relationships between risk triples
Business: marketing / demand etc
– Competitors developing similar product
– Poorly educated sales force
– Company strategy changes
David Hedley 13 COMSM1401 David Hedley 15 COMSM1401
Monitoring the Project Coping with Slippage
Assuming that effective monitoring detects slippage from the plan,
1. Take the plan seriously! early reaction is essential.
What choices are there?
2. Plan for regular reviews
Stick to plan, but
at well defined milestones rearrange workload
3. Record the outcome of each review get more resources or effort
Alter the plan
note tasks that have been completed
Move milestone date
define actions to catch up where progress has fallen behind plan
lower aims
Planning cannot be perfect - assume that you will have to re-plan
Unacceptable responses are:
from time to time as the project progresses.
Wait and see if there is really a problem
Hope the problem will go away

David Hedley 18 COMSM1401 David Hedley 20 COMSM1401

Risk monitoring Slippage

Question: How do slippages occur?

Assess whether risk did actually occur Answer: Gradually.
Adjust risk of overall project success
Collect information that can be used in the future A series of small delays
illness, machines crash, missing manuals, unexpected technical
Attempt to allocate blame (which risks caused which problems)
difficulties, delay in arrival of software/hardware, laziness,
unexpected interruptions, ...)
suddenly, the problem is acute

Anticipate delays before they become serious, and pick up the pace.

David Hedley 17 COMSM1401 David Hedley 19 COMSM1401

Re-planning Options (2)
Rearrange workplan
possible more efficient organisation of existing resources. Avoid
bottlenecks etc.
Move (delay) a milestone:
Delay non-critical tasks first.
If project has to be delayed, concentrate on retaining quality.
Lower ambitions
If required changes cannot be achieved, or delay would cost too
much, the project goals have to be reduced.
Try to do so without losing quality.

David Hedley 22 COMSM1401

Re-planning Options (1)

Extra effort:
Increase work rate
Danger with these options is that quality may suffer if people are
More manpower: Dangerous! Extra manpower means:
Training new members
Extra communication time
More partitioning of work, implies more interfaces, more testing
(Brooks’ Law: Adding extra manpower to a late project makes it later.)

David Hedley 21 COMSM1401