Welcome to the October 9th, 2008 Meeting of the Colorado Front Range Chapter, International Council On Systems Engineering

Networking 6:30 7:20 Introductions/Announcements 7:20 7:30 Presentation 7:30 8:30

Practical Application of the Department of Defenses Guide for Risk Management in Acquisition
Mr. Carl Terry CSEP-ACQ FlightSafety Systems Engineering Manager & Past President Colorado Front Range INCOSE Chapter

9 Oct 2008

OUTLINE
Definitions, Objective, and Introductory Material Risk Management Process Overview
Identification Assessment/Analysis Mitigation Planning Implementation of Mitigation Plan Monitoring and Tracking

Risk Handling Plan Risk Management Plan Example

9 Oct 2008 2

Bad things can ruin your day.

9 Oct 2008

DOD RISK DEFINITION

Risk is a measure of future uncertainties in achieving program performance goals and objectives within defined cost, schedule, and performance constraints
DAU Risk Management Guide, top of page 1
Note that this documents intended audience is Government program office personnel There is a lot of similarity between Government and Contractor processes the differences are in the details

Also, Defense Acquisition Guidebook (the DAG), 2006 edition, Section 4.2.3.5 -essentially outlines the DAU guide Note that performance means satisfying the program formal requirements as set forth in the specifications. This is also the quality aspect of the system in that quality here is the measure of how well the requirements are satisfied.
9 Oct 2008 4

HUMOR (Sort of)

You can get your product with 3 attributes: Good, Fast and Cheap. Pick two.
You can get it: Good and Fast, but bring money Good and Cheap, but be very patient Fast and Cheap, but dont put your family in it

Risk Management is a tool for avoiding the Pick 2 syndrome.

9 Oct 2008 5

RISK COMPONENTS
Trigger: A future root cause (yet to happen), which, if eliminated or corrected, would prevent a potential consequence from occurring, A probability (or likelihood) assessed at the present time of that future root cause occurring, and The consequence (or effect) of that future occurrence. Impact Rating: The combination of Probability and Consequence that defines the seriousness
9 Oct 2008 6

RISK vs ISSUES
Risks are future, issues are present
The difference between issue management and risk management is that issue management applies resources to address and resolve current issues or problems, while risk management applies resources to mitigate future potential root causes and their consequences.

A risk realized usually becomes an issue

9 Oct 2008 7

OBJECTIVE
The objective of a well-managed risk management program is to provide a repeatable process for balancing cost, schedule, and performance goals within program funding, especially on programs with designs that approach or exceed the state-of-the-art or have tightly constrained or optimistic cost, schedule, and performance goals.
9 Oct 2008 8

MANAGEMENT CONCEPT
Risk Management performed IAW defined processes simply formalizes and makes visible a set of actions that all humans have been doing naturally for most of their lives
Identify Assess/Analyze Plan for mitigation Implement the plan Track the plan effectiveness

9 Oct 2008

TOP LEVEL GUIDELINES

Assess root causes of risk Use grey beards to identify and analyze many risks are hidden beneath benign appearing initial design and process plans Involve program managers they may eventually have to fund fixes or eat the consequences Be proactive
Real risks do not fix themselves

Establish realistic re-assessment points

Risk attributes can change in response to external and other program factors
9 Oct 2008 10

RISK MGMT PROCESS

9 Oct 2008

11

IDENTIFICATION
Break down program elements to a level where subject matter experts (SMEs same as grey beards!) can perform valid identification
risks are (usually) identified based on prior experience, brainstorming, or lessons learned from similar programs

9 Oct 2008

12

ROOT CAUSES
Root causes are those potential events (occurrences, factors?) that evaluators . determine would adversely affect the program at any time in its life cycle An approach for identifying and compiling a list of root causes is to:
List WBS product or process elements, Examine each in terms of risk sources or areas, Determine what could go wrong, and Ask why multiple times until the source(s) is/are discovered.
13

9 Oct 2008

TYPICAL RISK SOURCES

Threats military or environmental Uncertain, poorly stated, or challenging (bleeding edge) requirements Technology you bid technology that will be available (??) in 3 years when you build the prototype, but is not currently procurable T&E program e.g.- impossible to fully test SW Logistics ability to support once fielded Cost underestimations, component price escalations, unexpected inflation rates Schedule optimistic delivery expectations, supplier failures
9 Oct 2008 14

RISK ASSESSMENT
PROBABILITY/LIKELIHOOD Level 5 4 3 2 1 Likelihood Near Certainty Highly Likely Likely Low Liklihood Not Likely
Risk-04

ASSESSMENT GUIDE

a
H RISK IMPACT RATING HIGH: Significant impact, high priority management attention required MEDIUM: Some impact, special action may be required. Additional management attention indicated. LOW: Minimum impact. Normal oversight needed to ensure risk stays low

5 4 3 2 1

M L L L L A

M M L L L B

H M M L L C

H H M M L D

H H H M M E

Level Technical Performance Minimal or none A Acceptable with some B reduction in margin Acceptable with C moderate reduction in margin D Performance degraded, no remaining margin Unacceptable, severe E degradation

CONSEQUENCE Schedule Cost Impact on Other Teams Minimal or none Min None Additional resources required 1 5% Some able to meet need dates Minor slip in key milestones Moderate not able to meet need dates 5 -10% Critical path impacted 1020% Major Unacceptable

Cannot achieve key team or >20% major project milestone

9 Oct 2008

15

RISK ASSESSMENT
Risk is the combination of the probability/likelihood of a trigger event occurring combined with the consequences resulting from that occurrence Risk Impact Rating is assessed as a combination of probability/likelihood and consequences
9 Oct 2008 16

RISK ASSESSMENT
Risks are usually assigned to cost, schedule or performance/quality classifications Definitions of consequences are developed independently for each class of risk
Examples shown below

9 Oct 2008

17

ANALYSIS-Likelihood
The table on the next slide defines Likelihood/Probability choices The % column is rough guidance only Many probabilities are expressed in terms of occurrences per ensemble of events
Occurrences per thousands of flight hours Occurrences per units produced Occurrences per customer served Occurrences per life of the contract

Likelihood definitions are domain and program unique

9 Oct 2008 18

ANALYSIS-Likelihood
Level 1 2 3 4 5 Likelihood Not Likely Low Likelihood Likely Highly Likely Near Certainty Probability of Occurrence 10% 30% 50% 70% 90%

9 Oct 2008

19

Some Math-be careful!

Impact = Cf + Pf (Cf)*(Pf)

System Engineering Management, Blanchard Cf and Pf expressed numerically as a range from 0.0 to 1.0

Note that if Pf = 1.0, Impact = 1.0 for all Cf including Cf = 0.0! Trivial consequences must be eliminated subjectively
May not be looking at a root cause

Most analyses are done subjectively or have a subjective aspect

9 Oct 2008 20

ANALYSIS - Consequences
The table on the next slide defines consequences choices Consequences are usually defined for performance, schedule and cost
Can affect more than one area Consequence levels are relative to program overall scale and management reserve levels Schedule and cost may have objective definitions, performance usually does not
Performance risk status is often tracked via Technical Performance Measures (TPMs)

9 Oct 2008

21

ANALYSIS - Consequences
Level A Technical Performance Minimal or no consequence to technical performance Minor reduction in technical performance or supportability, can be tolerated with little or no impact on program Schedule Minimal or no impact Cost Minimal or no impact Budget increase or unit production cost increases. < ** (1% of Budget) Budget increase or unit production cost increase < ** (5% of Budget)

Able to meet key dates. Slip < * month(s) Minor schedule slip. Able to meet key milestones with no schedule float. Slip < * month(s) Sub-system slip > * month(s) plus available float.

Moderate reduction in technical performance or supportability with limited impact on program objectives Significant degradation in technical performance or major shortfall in supportability; may jeopardize program success Severe degradation in technical performance; Cannot meet KPP or key technical/supportability threshold; will jeopardize program success

Program critical path affected. Slip < * months

Budget increase or unit production cost increase < ** (10% of Budget)

Cannot meet key program milestones. Slip > * months

Exceeds APB threshold > ** (10% of Budget)

9 Oct 2008

22

ANALYSIS- Risk Impact

Rating
For purposes of sorting, risks are rated as being Low, Medium or High The color chart on the following slide is used to determine risk rating subjectively All risk ratings (particularly high lows and all mediums) should also be subject to review for accuracy do they feel right?
9 Oct 2008 23

ANALYSIS- Risk Impact

Rating
IMPACT RATINGS

5
HIGH

Likelihood/Probability

MEDIUM

2
LOW

A
Risk-03

C D Consequence

9 Oct 2008

24

RISK HANDLING PLANNING

Avoiding risk by eliminating the root cause and/or the consequence
by reducing requirements, or adjusting budget and/or schedule, or by adding robustness Note that you have no control over the likelihood of external risks

Controlling the cause (if not external) and/or consequence Transferring the risk Assuming the level of risk and continuing on the current program plan.
9 Oct 2008 25

HANDLING PLAN Objectives

The intent of risk mitigation plan execution will be to ensure that successful risk mitigation occurs. It answers the question How can the planned risk mitigation be implemented? It:
Determines what planning, budget, requirements or contractual changes may be needed, Provides a coordination vehicle with management and other stakeholders, Directs the teams to execute the defined and approved risk mitigation plans, Outlines the risk reporting requirements for on-going monitoring, and Provides for maintenance of the status history.
9 Oct 2008 26

HANDLING PLAN
The Risk Handling Plan needs to be realistic, achievable, measurable, and documented. Include in Plan Documentation
A short description of the risk (including a summary of the performance, schedule, and resource impacts, likelihood of occurrence, consequence, whether the risk is within the control of the program); Why the risk exists (root causes leading to the risk); The options for mitigation (possible alternatives to reduce the probability for the risk); Who is responsible for making the plan and its elements happen
9 Oct 2008 27

HANDLING PLAN
Include in Plan Documentation (Contd)
Definition of events and activities intended to reduce the risk, success criteria for each plan event, and subsequent risk level if successful values; Risk status (discuss briefly); The fallback (contingency) approach for reducing the impact of the risk (describe the approach and the trigger event(s) that initiate implementation); A management recommendation (whether budget or time is to be allocated, and whether or not the risk mitigation is incorporated in the estimate at completion or in other program plans)
9 Oct 2008 28

IMPLEMENTATION AND TRACKING

Do it! Monitor the status and progress
Risk Tracking Sheet suggested data follows

Report the results at:

Requirements and Design Reviews Program Management Reviews System Progress Reviews Etc.
9 Oct 2008 29

Risk Tracking Sheet For

Engineers & Managers

The intent of risk tracking is to ensure successful risk management. It answers the question How are things going? by:
Communicating risk status to all affected stakeholders, Monitoring risk mitigation plans, Providing for regular progress updates, Displaying risk management dynamics by tracking risk status changes, and Alerting management as to when risk mitigation plans should be implemented or adjusted.
9 Oct 2008 30

Risk Tracking Sheet For

Engineers & Managers

Typically includes:
Identifier & description of the risk Assigned owner Initial analysis results
Including likelihood and consequence

Context (where & when in the program the risk is expected to arise) Mitigation Strategy
Contingency Plans (or references to) and triggers for same

Status and Activities Summaries

9 Oct 2008 31

BRIEFING TEMPLATE
Cliff Notes For Flag Officers
5
RISK IDENT/TITLE CAUSAL FACTOR MITIGATION APPROACH

Likelihood/Probability

Additional Information

A
Risk-05

C D Consequence

9 Oct 2008

32

RISK MGMT PLAN

An example RMP format may include:
Introduction Program Summary Risk Management Strategy and Process Responsible/Executing Organization Risk Management Process and Procedures Risk Identification Risk Analysis Risk Mitigation Planning Risk Mitigation Implementation Risk Tracking
33

9 Oct 2008

EXAMPLE RISK
In early 1996 we chose to propose $5,000 single board Pentium computer systems in PCI cages rather than the (then) usual $35,000 to $85,000 IBM RISC or specialty real time computer systems from Concurrent or GouldEncore. Risk: Needed an estimated 500 MHz CPU speed by late 1999, market best was ~ 100 MHz IN 1996 Mitigation factors:
1. Moores Law 2. System redesign 3. PCI cage gave us the flexibility to board-swap Pentium class CPU single board computers

9 Oct 2008

34

EXAMPLE Contd
Contingency: Use more CPUs to divide up the computing load More HW cost, more SW development cost to meet performance, possible schedule impact
Trigger 1: Not within Moores law range at CDR Trigger 2: No 500 MHZ announced or available by start of hardware/software integration (HSI) Trigger 3: Not within spec (60 Hz iteration rate with 50% unused processing time) at start of DT&E

9 Oct 2008

35

EXAMPLE Contd
Status results:
300 MHz available at CDR - Moores law says that we are ~ in range
Game market is demanding faster CPUs Moores law will be conservative

500 MHz installed during HSI Home Free!!! Oops! Less than 50% spare available in 500 MHz CPU at DT&E
Replaced with newer plug and runtime code compatible 700 MHz Pentium board, got 62% spare time

9 Oct 2008

36

SUMMARY
Risk is a real factor in competitive program management
No-risk proposals usually loose

Current DoD practice places a large risk management responsibility on the System Program Offices
Not all SPOs have picked up the flag

Risk plans have been required as separate plans submitted with the proposal on all of our RFP responses in 2007 and 2008
Reflects current DoD focus Used to be a section in the SEMP
9 Oct 2008 37

REFERENCES
Risk Management Guide for DoD Acquisition; 6th Edition Ver 1; August 2006, Published by DOD; (the DAU Risk Mgmt Guide); http://www.dau.mil/pubs/gdbks/risk_management.asp Defense Acquisition Guidebook, 2006 edition, Published by DoD; https://akss.dau.mil/dag

Handbook of Systems Engineering and Management; Sage, Andrew P. and William R. Rouse editors; John Wiley & Sons, 1999; IBSN 0471-15405-9 - See Chapter 3
Systems Engineering and Analysis; Blanchard, Benjamin S. and Wolter J. Fabrycky; Pearson Prentice Hall; 4th Edition 2006; IBSN 013-186977-9 Various Chapters and Subchapters

9 Oct 2008

38