Professional Documents
Culture Documents
113r-20 Risk and Cont Determination NOT NB
113r-20 Risk and Cont Determination NOT NB
20
INTEGRATEDCOSTANDSCHEDULE
RISKANAL YSI
SANDCONTI
NGENCY
DETERMI NATI
ONUSI
NG
COMBI NEDPARAMETRI
CAND
EXPECTEDVALUE
AACE® International Recommended Practice No. 113R-20
Any terms found in AACE Recommended Practice 10S-90, Cost Engineering Terminology, supersede terms defined in
other AACE work products, including but not limited to, other recommended practices, the Total Cost Management
Framework, and Skills & Knowledge of Cost Engineering.
Contributors:
Disclaimer: The content provided by the contributors to this recommended practice is their own and does not necessarily
reflect that of their employers, unless otherwise stated.
John K. Hollmann, PE CCP CEP DRMP FAACE Hon. Life Francisco Cruz, PE
(Primary Contributor) Dr. David T. Hulett, FAACE
Colin H. Cropley Martin Snyman, DRMP
This document is copyrighted by AACE International and may not be reproduced without permission. Organizations may obtain permission
to reproduce a limited number of copies by entering into a license agreement. For information please contact editor@aacei.org
AACE® International Recommended Practice No. 113R-20
INTEGRATED COST AND SCHEDULE RISK ANALYSIS
AND CONTINGENCY DETERMINATION USING
COMBINED PARAMETRIC AND EXPECTED VALUE
TCM Framework: 7.6 – Risk Management
TABLE OF CONTENTS
1. INTRODUCTION
1.1. Scope
This recommended practice (RP) of AACE International (AACE) defines general practices and considerations for
integrated cost and schedule risk analysis and estimating contingency using a combination or hybrid of parametric
risk modeling and integrated cost and schedule expected value analysis with Monte Carlo Simulation methods. P+EV
is used as a shorthand designation for the combination. In this RP, EV always refers to expected value (not earned
value). The base methods are covered separately in RP 42R-08 (Risk Analysis and Contingency Determination Using
Parametric Estimating), RP 44R-08 (Risk Analysis and Contingency Determination Using Expected Value) and RP 65R-
11 (Integrated Cost and Schedule Risk Analysis and Contingency Determination using Expected Value) respectively
[1,2,3]. Those RPs should be reviewed for details of the respective methods; this RP is focused on how to use them
in combination. Descriptions of other recommended risk quantification practices can be found in AACE Professional
Guidance Document PGD-02, Guide to Quantitative Risk Analysis [4].
The P+EV method is a fit-for-use, practical, risk-driven method intended to support management’s need for
integrated distributions of potential project cost and schedule outcomes to support investment decision making and
from which contingency and reserves can be established. It is focused on the bottom-line cost and completion date.
This is not an ideal method for understanding risk at a more detailed level such as what the risk impact might be on
an intermediate schedule milestone, or an estimate WBS element. For such in-depth analysis and understanding,
the hybrid combination of the parametric method with the risk-driven critical path schedule method is
recommended (i.e., P+CPM) (117R-21 [5]).
1.2. Purpose
This RP is intended to provide guidelines, not a standard, for contingency estimating that most practitioners would
consider to be good practices that can be relied on and that they would recommend be considered for use where
applicable. There is a range of useful risk analysis and contingency estimating methodologies; this RP will help guide
practitioners in developing or selecting appropriate methods for their situation.
It is an AACE recommendation that whenever the term risk is used, that the term’s meaning be clearly defined for
the purpose of the practice. The parametric modeling method described herein quantifies the impact of systemic
risks (defined in this RP) which include both threats and opportunities, and, from a quantitative perspective, are
uncertainties; i.e., as measures of attributes of or facts about a project system, the probability of occurrence is 100%.
The expected value method is focused on critical project-specific risks that includes both threats and opportunities
and may either be uncertainties (as described above) or risk events (i.e., contingent risks) for which the probability
of occurrence is less than 100%.
1.3. Background
The integrated, hybrid cost and schedule risk quantification method covered by this RP combines parametric
modeling of systemic risks (parametric) and expected value with Monte-Carlo simulation (EV w/MCS) modeling of
project-specific risks. P+EV is used as a shorthand designation for the combination. The component methods are
addressed in RPs 42R-08, 44R-08 and 65R-11 respectively. Two methods are combined because no single method is
optimal for quantifying both systemic and project-specific risks. The parametric model offers empirically-based input
to risk quantification that is largely missing with pure MCS. MCS is required in this hybrid combination to model the
EV portion and to integrate the analyses results. Only the mean values of method outputs are additive (e.g., the
overall cost or duration at say p70 confidence level is not the sum of the separate analyses p70 values). Figure 1
illustrates the hybrid concept:
Systemic risks are those that have systematically predictable relationships to overall project cost and schedule
growth outcomes [6]. The term systemic implies that the risk is an artifact of the project system, i.e., culture,
capabilities, competencies, decision making, complexity, level of technology, and so on. This includes estimating and
scheduling process uncertainties as well (e.g., rates, quantity takeoff, productivities, production rates, etc.). Some
would call these inherent or background risk or general uncertainty. Systemic risks are dominant for poorly defined
projects and/or weak project systems; hence the parametric method is ideal for early phase estimates. Systemic risk
impacts are best modeled based on empirical research of past projects; i.e., multi-variable linear regression (MLR)
or similar analysis (e.g., machine learning) of historical data on risk drivers and their cost and schedule impacts (RP
42R-08 and by extension this RP only address MLR).
Project-specific (P-S) risks are those that do not have predictable, systematic relationships with outcomes, i.e., they
are specific to the project. These are most commonly risk events or conditions although they may also include project
specific uncertainties (e.g., weather variability). Project-Specific risk assessment is amenable to traditional
brainstorming workshops or interviews and can be realistically modeled using EV methods (i.e., probability times
impact with MCS). In the EV method, only critical project-specific risks are quantified, i.e., those with the potential
of material impact on project success. Most risks in a risk register will not meet this criterion. For these critical risks,
the quantitative analysis will first assure that the nature of the risk is well understood (e.g., is the cause understood,
has too much credit been taken for mitigation efficacy, etc.?), and the probability of occurrence and their impact
will be reviewed; i.e., the information in a risk register is never accepted or used verbatim.
It should be noted that a project system attribute (i.e., a systemic risk) can change, or be changed, after the risk
analysis. However, in the P+EV method, uncertain future states of the system are not rated or credited. Any planned
or anticipated change in the project system (i.e., a mitigation) is analyzed as a project-specific risk (an event) that
may or may not occur and may or may not be effective. For example, the project control system at the time of
analysis may be poor, but a new system is planned for implementation during the project. In this case, the systemic
risk is the rating of the current system state, and the new control system implementation is an opportunity with
some probability of it being implemented and being effective with some assumed improvement of cost and/or
schedule (however, such changes are unlikely to be critical).
2. RECOMMENDED PRACTICE
As discussed, RPs 42R-08, 44R-08 and 65R-11 must be reviewed for background and details of each of the underlying
methods. This is not a stand-alone RP. The steps of applying these methods in a combined or P+EV hybrid approach
have been summarized in several AACE International papers, but also a comprehensive text on the method [7,8,9].
The steps of this process assume that tools are in place for 1) parametric analysis of systemic risks and 2) for expected
value analysis with MCS for project-specific risks. Most often, the parametric tool will be based upon either the
Hackney or Rand models that are documented (and provided in Excel® form) in RP 43R-08, Risk Analysis and
Contingency Determination Using Parametric Estimating – Example Models as Applied for the Process Industries [10].
The models in RP 43R-08 generate predicted mean cost growth and, for the Rand model, the schedule slip. RP 42R-
08 describes how to derive a probabilistic distribution from these mean outcomes. The EV tool with MCS is a fairly
basic spreadsheet (no complex or unusual math or functions or macros necessary) with MCS applied. The only
connection between the tools is the need to copy the parametric tool’s cost and schedule output distributions into
the EV tool as will be discussed.
With the tools in place, the steps in applying them as a hybrid application are as follows:
Step 2: Per RP 44R-08; Screen the Risk Register and Identify Critical Risks:
Optimally, the risk register will already have categorized each risk by quantification method type to be applied (i.e.,
create a column in the risk register to identify if the risk is systemic, project-specific, escalation or currency). This
categorization can be a challenge because risks are often not well titled or described as to their nature and cause. In
general, the more ambiguous, or the more the risk is in the nature of a worry or an issue, the greater the likelihood
a risk is systemic.
Further screen the project-specific risks to develop a list of those that are critical and refine the descriptions of their
nature and cause. The definition of critical risks is included in RP 44R-08, but in general these are risks that have a
material impact on the project economics. Risks are selected based on their post treatment, residual status. Check
for any risks that were critical pre-treatment, but non-critical after mitigation; assure that the risk reduction credited
to the mitigation is realistic. Post treatment, there should typically be no more than 5 to 15 critical risks, keeping in
mind that by definition any one critical risk will put the project success at risk. Having too large a number of critical
risks implies that the project may not be viable. Note that escalation and currency risks are not covered in this RP
(see RP 68R-11, Escalation Estimating Using Indices and Monte Carlo Simulation [12].
For schedule, the duration impact, considering the risk response, is to the completion milestone. As described in RP
65R-11, team knowledge of what is on or near the critical path and the network’s general, dynamic behavior is
needed. If confidence is low in understanding of the impact to the completion date, this should be reflected in the
range of the 3-point delay estimate.
For cost, the impact is a combination of time-dependent costs for schedule delays plus the non-time driven cost
considering the risk response. The time-dependent cost is the schedule delay times the applicable burn rate from
Step 4. The non-time dependent cost is the range of potential expenditures considering the risk response.
The EV tool will be set up to perform the calculation of the EV of the cost and of the schedule duration impact of
each risk (probability times impact). In MCS, the simulation results for each risk, or subtotal of several risks, can be
captured independently if desired. This subtotal result can be used to assess the impact on intermediate milestones
if one or more of the risks drive that milestone (see RP 65R-11).
Step 7: This RP; Integrate the Parametric Model Outputs into the EV Model (this creates the hybrid):
To integrate the parametric and EV results, include the parametric model output distributions (re: Step 1) as the first
critical risk in the EV w/MCS tool. The probability of occurrence of the systemic risk is 100 percent (i.e., it is a fact or
attribute of the project system). Systemic risks are treated as independent of the project-specific risks for this
method. The cost and schedule outcome distributions are defined as custom PDFs; this can be as simple as using 3-
point trigen distributions for an 80 percent confidence interval and entering the parametric p10, p50 and p90 values.
Using the MCS software, a correlation coefficient should be established between the parametric model cost and
schedule PDFs. A coefficient of 0.5 is suggested as a reasonable rule of thumb1; systemic risks by nature are positively
correlated to a moderate extent (e.g., poor scope definition will always result in increased cost growth and schedule
slip). However, a project such as offshore drilling for which cost (i.e., based on day-rates) is highly schedule-driven
1
The Rand Corporation research referenced in RP 43R-11 found a correlation coefficient of 0.41 between cost growth and schedule slip
outcomes driven by systemic risks [11].
may have a higher coefficient, and a schedule-driven project for which money is no object (see risk response
discussion), may have somewhat less correlation. Excessive focus on this correlation is not value-adding.
Step 7 is highlighted in bold because that step ties the parametric and EV methods together into a single model for
MCS. The following summarizes the hybrid P+EV application in flow chart format. Note the input of parametric
modeling of systemic risks into the EV w/MCS model is shown in the last row. Escalation is not included in the P+EV
method but is shown here to illustrate that the cost and schedule distributions can be used as inputs to a probabilistic
escalation tool.
Figure 2 – Hybrid Parametric and EV w/MCS Method Flow Chart [9; with permission]
This section provides an example of a P+EV toolset and illustrates the analysis steps applied in a tool. The toolset
shown in the example figures are simplified, but functional tools used in training on the P+EV method [9].
Figure 3 is a snapshot of a parametric model worksheet in Excel®. In the upper part, the parameters for the project
are defined; the values are selected from picklists. In a fully developed tool, each question here would instead be a
set of questions to ensure an objective rating viewed from various perspectives. For example, the PLANNING rating
would be built up from ratings of the status of many deliverables (e.g., is a resource-loaded CPM schedule
complete?). Note that there is a schedule section at the bottom, but only a few more systemic attributes must be
defined (most attributes drive both cost and schedule).
Once the parameters are rated, this tool is complete (i.e., Step 1 in the step-by-step discussion). The distribution
(simplified to a 3-point basis for the exercise) for the impact of systemic risk is shown for the cost and for the
schedule. If this were a Class 5 estimate, this may be the complete analysis. However, in the example, the Class on
average is 3.3 (i.e., the planning is lagging the desired Class 3 status at full funds authorization). In the example, the
p50 cost growth and schedule slips are 9% and 13% respectively.
COST GROWTH
RISK DRIVER ENTER PARAMETER (a) COEFFICIENT (b) a*b
CONSTANT -30.5
SCOPE 3
PLANNING 4
ENGINEERING 3
SCOPE DEFINITION 3.3 9.8 32.7
NEW TECHNOLOGY 0% 0.12 0.0
PROCESS SEVERITY 1 1 1.0
COMPLEXITY 3 1.2 3.6
SUBTOTAL BASE (prior to adjustments) 6.8
ADJUSTMENTS Complex Exec Strategy? > No
Team Development Fair 0
Project Control Fair 0
Estimate Basis Fair 0
Equipment 0% 4
Fixed Price 10% 0
TOTAL BASE (prior to bias adjustment; rounded to whole number) 11
Bias Typical 0
SYSTEMIC COST CONTINGENCY (at shown chance of underrun)
MEAN 11%
10% -10%
50% 9%
70% < enter funding p-level (increments of 10%) 19%
90% 35%
EXECUTION SCHEDULE DURATION SLIP
RISK DRIVER ENTER PARAMETER (a) COEFFICIENT (b) a*b
CONSTANT -23.5
SCOPE DEFINITION 3.3 9.6 32
NEW TECHNOLOGY 0% 0.1 0
PROCESS SEVERITY 1 0.5 0.5
COMPLEXITY 3 0.5 1.5
SUBTOTAL BASE (prior to adjustments) 10.5
ADJUSTMENTS
Schedule Basis Fair 0
TOTAL BASE (prior to bias adjustment; rounded to whole number) 11
Bias Low 3
SYSTEMIC EXECUTION SCHEDULE DURATION CONTINGENCY
MEAN 14%
10% 1%
50% 13%
80% < enter funding p-level (increments of 10%) 22%
90% 28%
Figure 3 – Example Parametric Model for Systemic Risks [9; with permission]
Figure 4 is a snapshot of a simplified P+EV tool model worksheet in Excel®. It has been set up to use PDFs and an
MCS add-on. In a fully developed tool, there would be more thorough entries, enhanced features such as 3-point
probabilities and burn rates, etc. The various user input sections are numbered and described as follows:
After running the MCS simulation, the outcome distribution tables at the bottom are populated from which
management may determine contingency and reserves values.
Figure 4 – Example P+EV Model Integrating Systemic and Project-Specific Risks [9; with permission]
The following are considerations for assuring that all critical uncertainties and risks are covered by the combined
methods while also assuring there is no redundancy:
Non-linearity:
Another consideration for the hybrid method regards the assumption in Step 7 of the application that systemic and
project-specific risks are independent (i.e., no correlation). For most projects, that is a reasonable assumption.
However, research suggests that when a project system is weak and/or complexity is high and the project is impacted
by extreme and/or multiple risk events, risk responses are less likely to be effective. If there is high systemic risk, the
analyst can address this by increasing the high impact value of the critical project specific risks. Correlation could be
added to the EV MCS model; however, correlation may only apply at the extremes. More explicit methods that
address non-linear risk-to-impact relationships resulting from complexity and/or compounding of risk are excluded
from this RP.
In 2009, the National Aeronautics and Space Administration (NASA) instituted a policy that certain project budgets
were to be based on a “joint cost and schedule probabilistic analysis” with budgets to reflect a “percent probability
that the project will be completed at or lower than the estimated amount AND at or before the projected
schedule.”[13] NASA calls this the joint confidence level or JCL. In NASA practice, the JCL is based on the cost-loaded
CPM-based risk analysis method. However, CPM modeling is not required for JCL; the hybrid P+EV method integrates
cost and schedule and supports JCL determination. Figure 5 is an example cost and schedule MCS output scatterplot
(in this case using Palisade @Risk® software) from the P+EV hybrid method. Note that RP 65R-11 also has an example
JCL plot; in that case resulting solely from the project-specific risks.
Figure 5 – Example JCL Graph from a P+EV Hybrid Model [using Palisade @Risk® software]
RP 40R-08, Contingency Estimating – General Principles, provides objective principles against which one can assess
the suitability of a contingency estimating method [14]. The following are the RP’s general principles that any
methodology developed or selected for quantifying risk impact should address:
• Provides probabilistic estimating results in a way the supports effective decision making and risk
management.
Table 1 summarizes how the P+EV hybrid method performs in respect to the principles.
Strengths Weaknesses
• Explicit risk-impact linkage • Fewer users than CPM-based methods
• Inherently empirically based • Not being CPM model based, requires more
• Universal applicability (any quality of input, estimate skilled/intuitive scheduling assessment
class, project size or type) • Does not readily support evaluation of the risks to
• Parametric method supports project system quality intermediate schedule milestones.
assurance/improvement • The EV method does not encourage the use of quality
• Parametric method addresses bias directly planning and schedule methods (but the parametric
• Addresses risk response (cost/schedule trading) method does)
• Allows changes to schedule logic due to risks to be
included without the complexity of branching in CPM
Table 2 – P+EV Hybrid Method Strength and Weaknesses
5. SUMMARY
This RP provides guidance to practitioners in developing or selecting appropriate methods for their situation with
the understanding that no one method is best for quantifying all types of risk. This RP integrates cost and schedule
risk analysis (and supports JCL) using a hybrid approach; and documents the steps to combine the parametric and
expected value methods covered by other RPs in detail. It provides an example using a demonstration toolset. It also
documents situations where a hybrid approach requires or may benefit from special considerations.
REFERENCES
1. AACE International Recommended Practice No. 42R-08, Risk Analysis and Contingency Determination Using
Parametric Estimating, AACE International, Morgantown, WV, (latest revision).
2. AACE International Recommended Practice No. 44R-08, Risk Analysis and Contingency Determination Using
Expected Value, AACE International, Morgantown, WV, (latest revision).
3. AACE International Recommended Practice No. 65R-11, Integrated Cost and Schedule Risk Analysis and
Contingency Determination using Expected Value, AACE International, Morgantown, WV, (latest revision).
4. AACE International, Professional Guidance Document PGD-02, Guide to Quantitative Risk Analysis, online
reference at www.aacei.org.
5. AACE International, Recommended Practice No. 117R-21, Integrated Cost and Schedule Risk Analysis and
Contingency Determination Using a Hybrid Parametric and CPM Method, AACE International, Morgantown,
WV (latest revision).
6. AACE International Recommended Practice No. 10S-90, Cost Engineering Terminology, AACE International,
Morgantown, WV, (latest revision).
7. Hollmann, John K., Alternate Methods for Integrated Cost and Schedule Contingency Estimating, 2010 AACE
International Transactions, AACE International, Morgantown WV, 2010.
8. Hollmann, John K., Realistic and Practical Project Risk Quantification (without CPM), 2017 AACE International
Transactions, AACE International, Morgantown WV, 2017.
9. Hollmann, John, K., Project Risk Quantification, Probabilistic Publishing, Sugarland TX, 2016.
10. AACE International Recommended Practice No. 43R-08, Risk Analysis and Contingency Determination Using
Parametric Estimating – Example Models as Applied for the Process Industries, AACE International,
Morgantown, WV, (latest revision)
11. Myers, Christopher W., and Ralph F. Shangraw, Understanding Process Plant Schedule Slippage and Startup
Costs, (R‐3215‐PSSP/RC), RAND Corporation, 1986.
12. AACE International, Recommended Practice No. 68R-11, Escalation Estimating Using Indices and Monte Carlo
Simulation, AACE International, Morgantown, WV (latest revision).
13. NASA, NPD 1000.5A; “Policy for NASA Acquisition”, National Aeronautics and Space Administration,
Washington, DC, 2010.
14. AACE International Recommended Practice No. 40R-08, Contingency Estimating – General Principles, AACE
International, Morgantown, WV, (latest revision).
CONTRIBUTORS
John K. Hollmann, PE CCP CEP DRMP FAACE Hon. Life (Primary Contributor)
Colin H. Cropley
Francisco Cruz, PE
Dr. David T. Hulett, FAACE
Martin Snyman, DRMP