Basic Techniques for Analyzing and Presentation of Cost Risk Analysis

Randal B. Lorance P.E. and Robert V. Wendling

1999 Annual Meeting AACE International

Monte Carlo Risk Analysis

Steps in process
Develop deterministic model of the estimate Identify the uncertainty in the estimate - Variables Analyze the estimate with Monte Carlo simulation Make decisions based upon results

This presentation examines the last step

What decisions can be made How to present them to the project stakeholders

Estimate Quality Categories

Historical
Conceptual (factored) Preliminary Budget Definitive

Recent Publications by AACE

Recommended Practices

Monte Carlo is a tool to determine which level of Project Definition you have achieved.

AACE - Process Industry

Preliminary Characteristic LEVEL OF PROJECT DEFINITION
Expressed as % of complete definition

Secondary Characteristic EXPECTED ACCURACY RANGE

Typical variation in low and high ranges [a]

ESTIMATE CLASS

END USAGE
Typical purpose of estimate

METHODOLOGY
Typical estimating method

PREPARATION EFFORT
Typical degree of effort relative to least costs index of 1 (b)

Class 5

0% to 2%

Concept Screening

Capacity Factored, Parametric Models, Judgment, or Analogy Equipment Factored or Parametric Models Semi-Detailed Unit Costs with Assembly Level Line Items Detailed Unit Cost with Forced Detailed Take-off Detailed Unit Cost with Detailed TakeOff

L: -20% to - 50% H: +30% to +100%

Class 4

1% to 15%

Study or Feasibility

L: -15% to -30% H: +20% to +50%

2 to 4

Class 3

10% to 40%

Budget, Authorization, or Control

L: -10% to -20% H: +10% to +30%

3 to 10

Class 2

30% to 70%

Control or Bid / Tender

L: -5% to -15% H +5% to +20%

4 to 20

Class 1

50% to 100%

Check Estimate or Bid / Tender

L: -3% to -10% H: +3% to +15%

5 to 100

Notes: [a] The state of process technology and availability of applicable reference cost data affect the range markedly. The +/- value represents typical percentage variation of actual costs from the cost estimate after application of contingency (typically at a 50% level of confidence) for given scope. [b] If the range index value of "1" represents 0.005% of project costs, then an index value of 100 represents 0.5%. Estimate preparation efforts is highly dependent upon the size of the project and the quality of estimating data and tools.

TYPICAL PROJECT SEQUENCE OF ACTIVITIES

PROGRAM AFC
Screening Phase Conceptual Phase Definition Phase Design & Construction Phase
Proje ct AFC Project Control Approval Esti m ate & Schedule Preliminary Engineering otes Write & Recieve budget qu Approve Major Equip. & long l ead items. AFC Commitment for long l ead ite ms Write & Approve AFC Vendor Engr. Equipment & bulks procurement & manufacture Detail Engineering Comme rcial Sale Mechanical Completi on Conceptu al Phase AFC Ap proval Screen Esti m ate & Schedule Feasib ility Stud ies for Screen ing Estimates Write & Approve AFC Exploration & Reservoir Eng.
Initial New Business reservoir engr . Ventures assessment

Definiti on Phase AFC Ap proval Conceptu al Esti m ate & Schedule

Conceptual Studies & Engineering

Composition Prod. rates Final over time reservoi & produ ct r Specs. engr. in put
Cont inued Reservoir Engr.

Oper. Input freeze Opera tions inpu ts to P&ID's, equip ment layou ts, & maj . equip. purchase specs.

Construction

Commissioning

Marketi ng / Contracts Negoti ations

Finalize Negotiation High e nd of accura cy range +60%

80% Co nfidence Interval

+100%

T rending
Trending from coneptual est imate & schedule t o reflect dec isions made during prelimi nary engineer ing (Screeni ng +30% or bet ter quality estimates.)

Exec ution Tr ending

Trending f rom the control esti mate & sc hedule

+15%

Current Estimate
-20% -40% -50% Low end of accura cy range -13%

+8% -8%

RBL 12/97

Statistics and Confidence Intervals

P opula tion of Da ta P oints
100 90 80

Histogra m
35 30 25 20 15 10 5 0 Bin

M inu te s

60 50 40 30 20 10 0 1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96

Fr e q u e n cy

70

Frequenc y 35 30
Fr e q u e n c 120% 100% 80% 60% 40% 20% 0%

Cu m ula tive

25 20 15 10 5 0 40 45 50 55 60 65 70 75 80 85 90 More

70

75

65

60

40

45

50

55

80

85

90

Bin

or

90 Mo re

40

45

50

55

60

65

70

75

80

85

Concepts, Terms, & Definitions Probability Distribution - Skewed to Right

Probability Distribution - Skewed to Right
10% 50% 90%

'

Median Mode Mean

Probability
 

!

!&

#

80% Confidence Interval

Standard Deviation ---

= Square Root Of Variance
0.030

+/ 1.28= 80 %

0.025

0.020

Probability

0.015

0.010

0.005

+/ 1= 68.27 % +/ 2= 95.45 % +/ 3= 99.73 %

Skewness
0.0350 0.0300

0.0250

0.0200

0.0150

0.0100

0.0050

2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0

Cumulative Probability Distribution

100% 90% 80%

70%

Cumulative Probability

60%

50%

40%

30%

20%

10%

0% 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0

More Definitions
Standard Deviation
Measure of average variance from mean. Measure of how widely dispersed the values are in the distribution.

Skewness
Asymmetric distribution. Usually right skewed for cost estimates. Usually left skewed for revenue estimates.

Output from Risk Analysis Model

Graphical
Forecast: Total estimate 2,000 Trials
.033

Simulation Statistics Date: 5/19/93 at 21:11 Iterations: 1500 Simulations: 1 Worksheet: HA_ONSHR.XL Output Range: Total Hainan Is Cell: $M$29 72,128
84,864 13,892 192,990,082 0.451398462 3.2461517

Tabular

Base Estimate= Mean= Std Deviation= Variance= Skewness= Kurtosis=

Frequency Chart

10 Outliers
65

Percentile Values Minimum= 5Perc=

51,160 63,816

10Perc=
15Perc= 20Perc= 25Perc= 30Perc= 35Perc= 40Perc= 45Perc=

67,844
70,343 72,820 75,174 76,653 78,550 80,017 82,028

.024

48.75

.016

32.5

.008

16.25

50Perc=
55Perc= 60Perc= 65Perc= 70Perc= 75Perc= 80Perc= 85Perc=

84,084
85,901 87,579 89,324 91,381 93,759 95,977 99,174

.000 40,000 62,500 85,000 $ in Thousands 107,500 130,000

90Perc=
95Perc= Maximum=

102,964
109,962 144,382

Alternative graphics output

Contingency
General Definition: An Assessment, made by a team of professionals, of the amount of funds which, when added to the line item estimate will reflect an equal chance of underrun or overrun given relative stability of the scope and assumptions upon which the estimate is based. DOE, AACE, & PMI discuss uncertainty and statistical analysis in their definitions in addition to experience. DOE adds: Contingency is not to be used to avoid making an accurate assessment of expected costs. AACE adds: Usually excludes changes in scope or unforeseeable major events such as strike, earthquakes, etc. PMI adds: Contingency reserves are normally included in the projects cost and schedule baselines.

Contingency Analysis
Median - 50/50 point
Equal Chance of overrun / underrun Target

Overrun / Underrun Limits - Range Specified Accuracy versus Achieved Accuracy

Outputs - Tabular - Each Project

Simulation Statistics Date: 5/19/93 at 21:32 Iterations: 1500 Simulations: 1 Worksheet: DRILL.XLS Output Range: Total Drilling. Cell: $M$61 Base Estimate= Mean= Std Deviation= Variance= Skewness= Kurtosis= 75,152
84,718 10,149 102,992,737 0.38689996 2.966930395 Contingency

Contingency Amount

Percentile Values Minimum= 5Perc=

$M 57,313 68,893 -17,839 -6,259

% -23.7% -8.3%

10Perc=
15Perc= 20Perc= 25Perc= 30Perc= 35Perc= 40Perc= 45Perc=

72,189
74,229 75,923 77,547 78,744 80,131 81,401 82,638

-2,963
-923 771 2,395 3,592 4,979 6,249 7,486

-3.9%
-1.2% 1.0% 3.2% 4.8% 6.6% 8.3% 10.0%

86.2%

-13.8%

50Perc=
55Perc= 60Perc= 65Perc= 70Perc= 75Perc= 80Perc= 85Perc=

83,726
85,142 86,723 88,111 89,664 91,139 93,247 95,528

8,574
9,990 11,571 12,959 14,512 15,987 18,095 20,376

11.4%
13.3% 15.4% 17.2% 19.3% 21.3% 24.1% 27.1%

100.0%

80% Confidence Interval

90Perc=
95Perc= Maximum=

98,433
102,352 121,119

23,281
27,200 45,967

31.0%
36.2% 61.2%

117.6%

17.6%

Output Shown as Cumulative

Typical Cost Risk Analysis Presentation
50 45 40 35 30 Add to Base + Escalation (%) 25 20 15% 15 10 5 -13% 0 -5 -10 -15 -20 -25 Specified maximum accuracy range 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Probability of Overrun (%) Cost in Millions

299.3 279.3 259.3 239.3 219.3 199.3 179.3 159.3

248.7
10.40% -7.50%

225.3 Contingency 6%

208.4

Specified confidence interval

Sensitive Analysis
Which inputs are most significant How do you reduce the risk (Improve Accuracy Range) Quantification of Subjective Impressions

Sensitivity Graphics

Conclusions
Monte Carlo uses:
Establish contingency Establish overrun/underrun limits Determine if estimate accuracy meets expectations

Graphical presentation
Cumulative showing Probability of Overrun