You are on page 1of 75

CHAPTER 9

RISK
MANAGEMENT
Risk Management is a
Continuous Process

Manage Risks

Understand Define Generate


the Need the Approach Detailed Plans

Monitor Execution
Another Model of the
Risk Management Process

Understand
Needs

Evaluate Risk Define


& Re-plan Management Approach

Generate
Detailed
Plans
Outline of This Chapter

1) Overview - The Risk Management Process


2) Risk Assessment
3) Risk Control
4) Risk Examples
5) The Risk Management Plan
9.1 - The Risk Management Process
Two Phases
1 Risk Assessment - done as part of the planning
-- Risk Identification - what are the risks?
-- Risk Analysis - what is the likelihood &
impact?
-- Risk Prioritization - which are most serious?
-- Risk Planning & Mitigation - minimizing impact
2 Risk Control - done as part of the execution
-- Risk Monitoring - watching to see if risks
happen
-- Risk Abatement - counteracting /
contingencies
Risk Management Methods

• There are several recommended methods in


the literature
• Here we will discuss two:
-- Barry Boehm’s Method
-- widely known
-- very pragmatic approach
-- combines assessment with control
-- Dennis Frailey’s Method
-- somewhat more comprehensive
-- derived from DoD Standards for
defense system software development
Boehm’s Method of
Risk Management (5 steps)

Risk Assessment (steps 1-2):


Top 10 Risks on
Project Alpha
1) Identify the top 10 risk items
(identification, analysis and1. Staffing
prioritization)
2. Late Hardware
3. Requirements
2) Present a plan to resolve each Definition
of the top 10 items 4. Real-time perf-
(mitigation; planning for
control)
Boehm’s Method (continued)

Risk Control (steps 3-5):

3) Update the list and the plan monthly


(monitoring)

4) Highlight the risk items at monthly project


reviews (monitoring)

5) Initiate corrective action for risks that occur


(abatement)

[6) Follow-up until the issue is resolved]


Frailey’s Method of Risk
Management (11 steps)

Risk Assessment (steps 1-7)

Risk Control (steps 8-11)


1) Identify all Risk Elements
(risk identification)

• Risk Elements consist of:


– things that can go wrong
– patterns of risk change over the lifecycle
» for example, cost estimating risks occur early,
whereas risks of staff burnout occur later
• Note that most risk identification is performed
as a part of other planning processes (see the
previous chapters in these notes)
• But it is also good to have a pro-active
attempt to identify all risks in case something
“fell through the cracks”
Recall How All Planning
Activities Identify Risks

Manage Risks

Understand Define Generate


the Need the Approach Detailed Plans

Monitor Execution
2) Partition into Categories
(analysis)
• Sample Categories:
-- cost risks
-- schedule risks
-- other management risks
-- technical risks
-- other risks specific to the situation, such as
safety or security risks
• One Risk may have multiple categories
– Estimating inaccuracies can lead to cost and
schedule risks
Why Partition Into Categories?

• on a given program, some types of risks may


be considered more important than others
– cost vs schedule vs technical capability issues
– “continue at any cost” vs. “only do if low cost”
• different people may be concerned about
different risks
– technical lead vs. “bean counter” vs. end user
waiting for delivery
• different people may be responsible for
different risks
• different risk types may require different
mitigation approaches
3) Identify Contributing Factors
(analysis)
• Many risks can occur in several ways
• If you aren’t careful, you will only be looking for
one of the ways
• Example:
Risk: not enough memory to hold the software
Contributing Factors:
Size of computer memory
Expertise of programming staff
Efficiency of compiler
Choice of algorithms
Operating system
Using a Hierarchy of
Contributing Factors
• Each risk can be seen as a contributing factor
to a larger risk
• Top level risk is that the project will fail
• Sometimes it helps to use a hierarchy to
organize risks and contributing factors
• (See next page)
A Sample Risk Hierarchy

Project
Failure

Performance
Staffing Funding ...
Failure

Memory Processor
...
Too Small Too Slow

Size of Programming Compiler Choice of


Memory Experience Efficiency Algorithms
4) Identify Potential Risk
Monitoring & Mitigation Plans
(analysis)
• This must be done for each contributing factor
• Examples:
Risk: memory size inadequate
– Factor: compiler produces bloated code
» Potential mitigation: choose a more efficient compiler;
negotiate improvements with vendor
– Factor: inexperienced programmers
» Potential mitigation: training program; use more
experienced programming staff
– Potential mitigation that applies to multiple factors:
» select a larger memory size
– Potential monitoring that applies to multiple factors:
» track size estimates monthly; update at each major
milestone
5) Rank and Prioritize Each Risk
(prioritization)
Prioritize on the basis of probability (how
likely) and impact
-- life is full of risks
-- you need to “pick your battles”

Risk Likelihood Cost Weighted Cost

Late 0.75 $100,000 $75,000


Hardware
Subcontractor 0.20 $250,000 $50,000
Failure
Memory Size 0.50 $50,000 $25,000

Test Equip. 0.30 $40,000 $12,000


Delay
Requirements 0.99 $5,000 $4,990
Changes
Building Hit 0.0000001 $50,000,000 $5
by Airplane
6) Identify Monitoring
Procedures for Each Risk
(planning for control)
• Determine how to tell if it is a problem; how
frequently to monitor; etc.
• Example: monitor projected size vs. memory
limits on a monthly basis

150

100
Limit
50 Threshold
Estimate
0
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
7) Develop a Backup or
Contingency Plan
(planning)
Identify what to do if the risk occurs despite
your mitigation efforts
Example:
Risk: memory size exceeded
Contingency Plan:
– switch to a slower but smaller algorithm
– use a more efficient compiler
– use a smaller operating system
– use larger memory size
Frailey’s Method - Risk Control
(4 steps)
8) Review status of risks at periodic reviews
(monitor)
– Metrics
– Changes in impact analysis
9) Take appropriate action when called for
(abatement)
– Closer monitoring
– Contingency activities
10) Track all actions to closure (monitoring)
– Don’t forget about them
11) Update the plan (planning)
– Keep it consistent with current knowledge and status
Beware of Subcontractors and
Co-contractors
• Risk management applies to these as well
• Include risk management elements in
contracts

We want to
Just trust us monitor your
risks
9.2 - Risk Assessment

• Goal: understand what the risks are, what


they mean, and how best to manage them
• Method: develop a risk management plan
-- documents your risks
-- documents your plans for managing them
-- communicates responsibilities to all
affected parties
• The plan must be updated as new risks are
identified and risk likelihoods and impacts
change over time
9.2.1: Risk Identification
The First Step of Risk
Assessment
• We have seen how all management activities,
especially planning activities, serve to
identify risks
• It helps if you know what to look for
• Several authors have talked about risks
associated with software development
Boehm’s Top Ten Software
Risks (and management
techniques for mitigating them)
• This is Boehm’s list, based on experience with
many projects in the 1960-1980 time period
• Your project must define its own list
• But this is a good place to start looking
Possible Exam Question

 Consider a particular situation (provided


with the exam). What risks apply?
 (careful analysis shows that these three
risks are of greatest concern . . .)
 List of Boehm’s top ten risks
Boehm 1: Personnel Shortfalls

Mitigation Techniques:
• Use top talent
• Use people who are well matched to the
problem (i.e., they know the application,
tools, etc.)
• Pre-schedule the key people
• Cross training -- so you don’t depend on
heroes
• Team building
Boehm 2: Unrealistic Schedules
and Budgets
Mitigation Techniques:
• Good estimating techniques
– Know before you commit
• Incremental development cycles
• Detailed milestones within each phase
– Spot trouble early
– Provide evidence of progress
• Software reuse
– Don’t build what was built before
• Requirements scrubbing
– Don’t build what is not required
Boehm 3: Developing the
Wrong Software Functions
Mitigation Techniques:
• Mission analysis
– Understand what is supposed to happen
• User surveys and communication
– Know what the user needs and is expecting
• Prototyping
– Try it out - a prototype is worth a million bytes
• Write end-user documentation early in the
project and use as requirements documents
Boehm 4: Developing the
Wrong User Interface
Mitigation Techniques:
• Prototyping
• Task analysis
• Scenario models
• etc,
Boehm 5: Gold Plating

Mitigation Techniques:
• Requirements scrubbing
– Don’t build what is not required
• Cost-benefit analysis
– Determine what counts the most
• Design-to-cost
– Development costs include debugging, error
correction
• Design to life-cycle-cost
– Life cycle costs include maintenance, debugging,
etc.
Boehm 6: Continuing Stream of
Requirements Changes
Mitigation Techniques:
• Establish a high change threshold
– Make it costly to make a change
• Information hiding
– Minimize the impact of change
• Incremental development
– Do the stable requirements first
• Establish bounds for requirements
– Limit the scope of changes
• Monitor requirements stability and completion
– Know the risk of proceeding
Information Hiding

• A programming technique whereby


information is made available only where it is
required
• This minimizes the impact of change
Display
.. Student Data Base
Examine
.. Student
Access Records
Update
.. Program (details
known only
Compute to access
program)
..
Requirements Bounds

Device
Fo Target

g
When you know the
Requirement: “device must find
range of the “TBD”
target in fog of density TBD.”
requirement, you can
Bounded Requirement: “TBD will make many software
range from .02 to 1.7 densograms” design decisions.
Requirements Stability
Monitoring
Requirements Stability

160 PDR CDR


140
120 Total
100 TBD
Changes
80
60
40
20
0
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Boehm 7: Shortfalls in
Purchased Software/Hardware
Mitigation Techniques:
• Benchmarking
– Learn what is the best
• Inspections
– Make sure it is in good shape before you buy it
• Reference Checking
– Are they able to do the work?
– Do they deliver?
• Compatibility Analysis
– Make sure it fits with your standards, tools,
documentation requirements, etc.
Boehm 8: Shortfalls in
Externally-performed Tasks
Mitigation Techniques:
• Subcontract management (a KPA for Level 2
in the SEI CMM)
• Reference checking
• Pre-award audits and capability evaluations
• Award fee contracts
– Reward or bonus if satisfied
• Competitive design or prototyping
• Team building
– Make them want you to succeed
Boehm 9: Real-time
Performance Shortfalls
Mitigation Techniques:
• Simulation
• Benchmarking
• Modeling
• Prototyping
• Instrumentation
• Tuning
Boehm 10: Straining the Limits
of Computer Science
Mitigation Techniques:
• Technical analysis
• Cost-benefit analysis
• Prototyping
• Reference Checking
9.2.2: Risk Analysis &
Prioritization

Which Ones are Important?


• A popular method is to compute two numbers
for each risk:
– How likely is it to happen (a probability from 0 to 1)
– How much will it cost if it happens (dollars)
• Then make a table showing risks, likelihood,
cost, and weighted cost (likelihood * cost)
Risk Analysis Table

Risk Likelihood Cost Weighted Cost

Memory Size 0.50 $50,000 $25,000

Subcontractor 0.20 $250,000 $50,000


Failure
Late 0.75 $100,000 $75,000
Hardware
Test Equip. 0.30 $40,000 $12,000
Delay
Building Hit 0.0000001 $50,000,000 $5
by Airplane
Requirements 0.99 $5,000 $4,990
Changes
Prioritized Risk Analysis Table

Risk Likelihood Cost Weighted Cost

Late 0.75 $100,000 $75,000


Hardware
Subcontractor 0.20 $250,000 $50,000
Failure
Memory Size 0.50 $50,000 $25,000

Test Equip. 0.30 $40,000 $12,000


Delay
Requirements 0.99 $5,000 $4,990
Changes
Building Hit 0.0000001 $50,000,000 $5
by Airplane
WARNING

This method of risk prioritization has many


risks itself!
• Not all risks can be quantified in terms of
dollar impact
• Estimates of impact are highly subjective
• Impacts change over time -- must be revisited
• Risk mitigation techniques may have risks of
their own
9.2.3: Risk Mitigation

Minimizing the Impact

• Taking actions to reduce the liklihood or net


impact of a risk
• We saw many such actions in reviewing
Boehm’s list of risks
• Each potential action must be evaluated in
terms of its cost vs. the potential impact
This is the Step where you
Impact the Planning Process

Manage Risks

Understand Define Generate


the Need the Approach Detailed Plans

Monitor Execution
Example

Risk:
Subcontractor will fail to deliver
Weighted Cost:
$50,000
Mitigation Options:
Do it in-house -- Costs $100,000 extra
Send staff to live with subcontractor - $50,000
Monthly visits -- $12,000
Risk Table After Mitigation

Mitigated
Risk Likelihood Cost
Weighted Cost
Late 0.75 $100,000 $75,000
Hardware
Memory Size 0.50 $50,000 $25,000
Subcontractor $12,500 +
Failure 0.05 $250,000 $12,000
Test Equip. 0.30 $40,000 $12,000
Delay
Requirements 0.99 $5,000 $4,990
Changes
Building Hit 0.0000001 $50,000,000 $5
by Airplane
Risk Plan May Include Tables of
Mitigation, Contingency, etc.

Methods Proto Bench Extra Back etc.


Risks type mark Hard up
ware
Late Hardware X X

Memory Size X

Subcontractor X
Delayed

Late Test Equip. X

etc. X
9.3: Risk Control

• Monitoring
– Watch what is happening
– Watch for signs of danger
• Risk Abatement
– Applying contingency plans
– Minimizing impact
9.3.1: Risk Monitoring

Methods of monitoring:
• Reviews - periodic status reports
– Must be honest reviews, not “dog and pony shows”
• Metrics - data to compare actuals with plans
and past performance
What to monitor:
• All high priority risk items
• Consider cost of monitoring - only monitor
what is worthwhile
How Often to Monitor

• It depends on priority - you must plan


For example:
– Critical items daily or weekly
– Normal items weekly or monthly
– Minor items quarterly
• Known danger points should be watched
– “We always have a flurry of changes prior to CDR”
• Monitor status at key milestones or progress
points
– Are we where we should be by now?
Don’t Monitor Too Often or in
Too Much Detail
• Too often
– Demotivates people
– Costs a lot
– Robs resources from productive activities
• Too much detail (overly precise)
– Costs a lot
– Does not give significant additional information
– May generate a lot of misleading numbers
Late by 2.7321 weeks is not much different
from late by 3 weeks
Risk Monitoring
Things to Watch Out For
• Hiding the facts to save face
– The purpose of monitoring is to manage properly, not to
find fault with individuals
– Individuals must trust that you will use the metrics
properly to fix the process, not to punish the
messengers
– One solution is self-measurement -- have the individuals
measure and monitor themselves without
communicating higher except on an aggregate scale or
with big problems that they cannot handle
• Hiding the facts to save the project
– Egos and jobs of technical staff vs. judgment of
management vs. pocketbook of sponsor
– This can get very political and very complex
Things to Watch Out For
(continued)
• Failing to get the data because of high
overhead
– Automate collection
– Consider using a separate metrics
collection/analysis staff to minimize impact on
development staff
» but not if it destroys teamwork
• Failure to get the data because of fear,
overconfidence, or other psychological
factors
– Be careful of human nature
– Focus on teaming, responsibility, and professional
behavior
Additional Things to
Watch Out For
• Misinterpretation of data
– Any number can be interpreted in many ways
– Be prepared to ask lots of questions
– Define standard measures and methods of
graphing data to minimize unintentional
misinterpretation
– Educate everyone in statistics
• Failure to trust past performance
– It is your most reliable indicator
The Learning Process

Information is Information is
Communicated Received

Information is
Action Analyzed

Each step offers many opportunities for error


Example of Misinterpretation

Not Enough
Productivity is Work Being
Low Done

More Not Working


Overtime Hard Enough

Perhaps the Real Problem is Too Much Overtime Already


Rate Chart
Actual vs. Plan
• A Rate Chart shows actual vs. planned
progress for some artifact being produced
– Coded units
Units Tested
– Tested units
80
– Inspected units 70
60

– Specifications 50
40
Plan
– Requirements defined 30
20
Actual
History

– Requirements tested
10
0
1 2 3 4 5 6 7 8 9 10 11
– etc.
– etc.
• It works for anything that has discrete,
measurable units
Testing Rate Chart

Units Tested

80
70
60
50
40
30 Plan
20 History
10
0
1 2 3 4 5 6 7 8 9 10 11
Testing Rate Chart

Units Tested

80
70
60
50
40
Plan
30
Actual
20 History
10
0
1 2 3 4 5 6 7 8 9 10 11
Testing Rate Chart

Units Tested

80
70
60
50
40 Plan
30 Actual
20 History
10 Projected
0
1 2 3 4 5 6 7 8 9 10 11
Testing Rate Chart

Units Tested

80
70
60
50
Plan
40
Actual
30
History
20 Projected
10 Likely
0
1 2 3 4 5 6 7 8 9 10 11
Another Rate Chart

30 Historical Average
Plan
25
Actual
20 Projected

15

10

0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
9.3.2: Risk Abatement

• Establish thresholds so you know when to


act
– Beware of the “frog in the water” problem
– Historical precedent is a good basis for knowing
when things are getting out of hand
– We will discuss this more in a later chapter

• Act promptly when necessary


– Historical data are helpful here
– Establish action plans before they are needed
– Learn the plan (fire drills) -- there may be no time in
an emergency
Risk Abatement (continued)

• Let the team participate in the planning


– Their cooperation is necessary for success
– Their ownership of the plan will improve chances of
cooperation
• Use backup plans if needed
– Develop them during planning phase
– Practice them too!
9.4: Risk Examples

Software Needed: “Getting Started on the PC”

To be Subcontracted

Need: something that will make our PC stand


out to new users -- must have pizzazz

Risk Identification:
• Vendor A: best product, positive attitude, but
in financial jeopardy
• Vendor B: mediocre product, sound finances,
blasé attitude
Analysis and Prioritization

• Only vendor A can do the job - mediocre


product will not fill the bill
Risks and Mitigation:
• Need for flash and dash
– End user tests, focus groups
• Deadline is very short
– Incentive clause
– Incremental deliveries
• Company Stability
– Ownership of key development tool resources if
they fail
– Weekly deliveries of in-progress work
– Monthly visits to site
Example 2: Embedded Flight
Control Software
Need: high performance, embedded, real-time
software for flight control

Risk Identification:
• Memory size very limited - might not be
enough
– Compiler is new
– Requirements are unstable
• Processor may be too slow
• Shortage of programmers for this processor
• Schedule very tight
Risk Mitigation

• Memory Size
– Compiler Performance
» evaluation
» backup vendor (different host system)
» assembly language option
– Requirements Stability
» evolutionary lifecycle with prototype phase
» strong requirements control
– Other
» Design hardware to accommodate larger memory
chips
» Use a smaller operating system
Risk Mitigation (continued)

• Speed
– Prototype of key algorithms
– Design to allow faster clock
– Plan to monitor performance regularly, starting with
simulation of design model
– Consider alternative processor design
– Consider use of multiple processors
Risk Mitigation (continued)

• Shortage of Experienced Staff


– List of programs we can steal people from
– Training plan
– Prototype on target hardware, for experience
• Schedule
– Incremental delivery negotiated with customer
– Do operating system first
» relatively stable
» can start before requirements for application
are complete
» development work provides target processor
experience for staff
9.5 - Risk Management Plan

• The plan should record the results of each step


of the process:
– Risks identified
– Priorities
– Mitigation
– Monitoring / Thresholds
– Contingency Plans
• A documented plan serves to communicate the
above to everyone concerned
• It also helps you know what to do during a crisis
• The plan should be reviewed and updated from
time to time
Summary of Chapter

1 Risk Assessment
-- done as part of the planning
-- continues indefinitely
-- includes planning for risk control
2 Risk Control
-- done as part of the execution
-- important to respond promptly when monitoring
indicates a problem

A risk management plan is an important part of


planning
Possible Exam Questions

 Explain the difference between risk


mitigation and risk abatement (risk
contingency)
 Explain why a risk management plan
should be documented
 Explain why a risk management plan
should be periodically revisited
 Explain why one would go to all the
trouble to write a risk management plan
when it tells your manager and your
customer that you have a risky project
End Of
CHAPTER
9

You might also like