You are on page 1of 8

testers to report to management in terms of the

remaining level of risk at this point

management to decide whether to:

extend testing

transfer the remaining risk onto the users, customers,

help desk/technical support and/or operational staff

During test execution the most sophisticated risk-based

testing allows project participants, project and product

managers, executives, senior managers, and project

stakeholders to monitor and control the software

development lifecycle.

This requires the Test Manager to report test results in

terms of risk in a way each test stakeholder can

understand.

Risk-Based
Testing Techniques
There are a number of techniques for risk-based
testing for both formal and informal. Such approaches
are subjective and dependent on the skills,
experience and preferences of the tester.
Attempting to capture the benefits of risk-based testing

while minimizing the costs, many practitioners employ

lightweight risk-based approaches:

Pragmatic Risk Analysis and Management (PRAM)

Systematic Software Testing (SST)

Product Risk Management (PRisMa)

In addition to the usual attributes of risk-based testing,

these techniques typically have the following attributes:

Evolved over time based on industry experience

(especially in industries where questions of efficiency

are important)

Extensive involvement of a cross-functional team of

stakeholders during the initial risk identification and

assessment (both business and technical)

Optimized when introduced in the earliest phases of

a project:

options to mitigate quality risks are maximized

risk analysis can help to influence the product

specification and implementation in a way that

minimizes risk

Use the generated output (risk analysis table) as the

basis for the test plan and the test conditions and all

test management and analysis activities

Support the reporting of test results in terms of

residual risk to all levels of testing stakeholders


Systematic Software Testing (SST)

require requirements specifications as input into the

risk analysis

cannot be used except when requirements specifications

are provided

PRisMa and PRAM

Encourage using a blended risk and requirements based

strategy

Can function entirely based on stakeholder input

Requirements and/or other specifications as an input to

the risk analysis

The use of requirements as an input helps ensure good

coverage of the requirements

Test Manager must ensure that important risks which

are not suggested by the requirements (like non-

functional) are not missed

When good, prioritized requirements are provided as an

input, one typically sees a strong correlation between

risk levels and requirement priority

Many of these techniques encourage the use of risk

identification and assessment process as a way to create

consensus across stakeholders on testing.

Insufficient stakeholder participation leads to gaps in

the risk analysis. Stakeholders do not always agree on the

level of risk, so the person leading a quality risk analysis

effort must work creatively and positively with

stakeholders to achieve the best possible degree of

agreement
Lightweight techniques allow the use of weighting of

the likelihood and impact factors to emphasize business

or technical risks.

Lightweight techniques use:

only two factors: likelihood and impact

simple, qualitative judgments and scales

and these approaches provide:

flexibility

applicability in a range of domains

accessibility across teams of all experience and skill

levels

RISK PRIORITY Extensive Testing, 1st

RISK PRIORITY Extensive Testing, 2nd

RISK PRIORITY Broad Testing, 3rd

RISK PRIORITY Normal Testing, 4th

RISK PRIORITY Normal Testing, 5th

Very Low

Low
doohilekiL

Medium

High

Very High

Impact
At the formal, heavy-weight end of the scale, the Test

Manager has a number of options available:

Hazard analysis

extends the analytical process upstream

attempting to identify the hazards that underlie each

risk

Cost of exposure

involves determining three factors

a. likelihood (expressed as %) of a failure related to

the risk item

b. cost of a loss (expressed as a financial quantity)

associated with a typical failure related to the risk

item, should it occur in production

c. cost of testing for such failures

Probability Cost of Testing, Cost of Loss


40
Risk 1
30

Risk 2
20

Risk 3
10

Risk 4
0
1

0 2,500 5,000 7,500 10,000 1


k

k
is

is

is

is
R

Risk 1: Impact 750 vs Test 500 => must TEST

Risk 2: Impact 800 vs Test 1000 => Do Not TEST

Risk 3: Impact 750 vs Test 750 => Either option is OK

Risk 4: Impact 4000 vs Test 300 => must TEST


Failure Mode and Effect Analysis (FMEA)

identify quality risks, their potential causes and their

likely effects

then severity, occurrence and detection ratings are

assigned in order to establish priority

strength on precision and meticulousness

document heavy and hard to learn

should be considered for high-risk or conservative

projects

Occurrence

RISK
8
Priority Number:
240 (6 x 5 x 8) 5

2,8,4
RISK

Priority Number:
2
64 (2 x 8 x 4) 4
6
6,5,8 8

Severity Detection

Severity = Severity impact of failure. Scale of 1 to 10

Occurrence = Occurrence of failure event. Scale of 1 to 10

Detection = Ability to detect the failure. Scale of 1 to 10

Risk Priority number = Overall risk scale of an event. Calculated by

multiplying Severity x Occurrence x Detection. Highest are first.


Quality Function Deployment (QFD)

a quality risk management technique with testing

implications

concerned with quality risks that arise from an

incorrect or insufficient understanding of the

customers’ or users’ requirements

focus on the function of execution of a quality plan

Fault Tree Analysis (FTA)

observed failures (from testing or from production) or

potential failures (quality risks)

then failures are subjected to a root cause analysis

starting with defects that could cause the failure

then with errors or defects that could cause those

defects

continuing on until the various root causes are

identified
The specific techniques that should be used for risk-based

testing, and the degree of formality of each technique

depend on project, process, and product considerations.

In Agile projects, the quality risk analysis is fully

integrated into the early period of each sprint and the

documentation of risks is blended into the user story

tracking.

Systems of systems require risk analysis on each system

as well as on the overall system of systems and safety-

critical and mission-critical projects require higher

levels of formality and documentation.

The inputs, processes, and outputs for risk-based testing

will tend to be determined by the selected technique.

Inputs

stakeholder insights

specifications

historical data

Processes

risk identification

risk assessment

risk control

Outputs

list of quality risks

defects discovered in input documents

questions or issues related to the risk items

project risks affecting testing or the project as a whole