T H E J O U R N A L O F A A C E I N T E R N AT I O N A L | T H E A U T H O R I T Y F O R T O TA L C O S T M A N A G E M E N T

JA NUA RY / FE BR UARY 2023

Proactive Cost
Engineering
Techniques ALSO:

to Avoid Risk Analysis and

Contingency Estimating
Common for Class 10 Estimates

Project Failures False Estimate Accuracy


CORE
VALUES
QUALITY Demonstrated by our
commitment to excellence and
continuous improvement
INTEGRITY Demonstrated by
transparent and ethical policies
and practices
PROFESSIONALISM Demonstrated
by our leadership and knowledge
TRUSTWORTHY Demonstrated
by our dedication to accuracy
and reliability
Learn more,
or join today!
EXPLORE OUR WEBSITE
ABOUT US
Established in 1956, AACE International
is the Association for the Advancement
of Cost Engineering. Together with the
AACE International Certification Institute,
it serves over 10,000 members and
certificants in 100 countries.
WHY WE EXIST
AACE empowers its members to perform with confidence. Together,
we make an impact and influence outcomes.
WHAT WE DO
We set industry trends. AACE advocates for its Body of Knowledge
and the people who employ it through iteration and innovation of
trusted technical guidance and meaningful collaboration.
WHO WE ARE
AACE is a community of professionals from various practice areas
including project controls, cost & commercial management, change
management, planning & scheduling, estimating, decision & risk
management, asset management, and dispute resolution. Members
include owners, contractors, and consultants from many industries,
such as construction, manufacturing, engineering, infrastructure,
natural resources, power generation & utilities, and government.
CONTENTS
JA N UARY/ FEBR UARY 2023

TECHNICAL ARTICLES

9 Proactive Cost Engineering

Techniques to Avoid Common
Project Failures
CHRISTOPHER W. CARSON, CEP DRMP PSP FAACE

24 Risk Analysis and Contingency

9 Estimating for Class 10 Estimates
JOHN K. HOLLMANN, PE CCP CEP DRMP FAACE HON. LIFE

37 False Estimate Accuracy

DAVE KYLE, CCP CEP

ALSO IN THIS ISSUE

3 AACE International Board of Directors

3 Cost Engineering Journal Information

24 4 Letter from the Editor

8 Upcoming Webinar: Spotlight on RP 27R-03: Schedule
Classification System AND Spotlight on RP 37R-06:
Schedule Levels of Detail
36 Conference & Expo 2023: Chicago - Save the Date!
42 PSP Exam Review Workshop Scheduled

37 For additional industry news and updates,

you can always visit us at web.aacei.org.

JA NUA RY/ FEBRUA RY 2 02 3 1

The Top 10 Reasons
To Join AACE International
Ready to advance your career and begin enjoying the advantages
that our members enjoy? Whether you are an experienced cost
engineer or a student, we have a membership ready for you.

1 Time 6 Technical Development

Gain access to a wealth of resources that will save you time
and money! You’ll stay informed about the complexities of the
cost and management profession - plus you’ll have access to
discounts on educational programs, publications, and more!
2 Information
Locate thousands of technical papers and publications in the
Virtual Library. AACE’s database is keyword searchable for
quickly locating appropriate reference articles.
3 Career
Members can post resumes at no additional cost in our Career
Center and keep your career on track through information
sources such as our annual Salary and Demographic Survey of
Project and Cost Professionals.
Increase your knowledge and expertise by joining one
of AACE International’s many technical subcommittees,
subcommittees, and Special Interest Groups (SIG’s) at no
additional cost to members. Discuss industry problems
with your peers or help experts develop new and improved
techniques and practices for the profession.
7 Networking
By attending a local section or our annual Conference & Expo
for interesting speakers, informational tours, social dinners and
much more. The online Membership Directory is an excellent
source for a list of contact information on thousands of members.
Join one of our many technical subcommittees and participate
in the AACE Communities - a great way to tap into the collective
wisdom and experience of our worldwide membership.

8 Excellence
4 Learning Our certification programs are independently accredited by
the Council of Engineering & Scientific Specialty Boards. AACE
We offer numerous online learning courses on estimating and
certifications are a recognized credible standard in the cost
project management. The Approved Educational Provider
management field. A recent study shows that individuals with
program helps maintain high quality development courses and
an AACE Certification earn 17.4% more than their counterpart
providers. AACE also holds many seminars throughout the year.
without a certificate.

5 Resources 9 Discounts
Starting with the TCM Framework and Recommended
On products and services ranging from AACE International
Practices that are available for free only to members to our
Conference & Expo registration fees, archived webinars and
bi-monthly publication Cost Engineering featuring articles
presentations, certification examination registrations, and more!
for cost professionals around the world. Through the AACE
International website, the Cost Engineering journal is a great
current resource for members and as a member, you gain
access to an archive of past issues.
10 You!
We are your professional partner bringing you information
and support you can trust. Join and become part of a unique
network of individuals who are dedicated to improving the cost
JOIN TODAY! web.aacei.org and management profession.

2 JAN UA RY/FE B R UA RY 20 23
 COSTENGINEERING
AACE INTERNATIONAL
BOARD OF DIRECTORS
PRESIDENT
Shoshanna Fraizinger, CCP
president@aacei.org ESTABLISHED 1958 | Vol. 65, No. 1 January/February 2023

PRESIDENT-ELECT
Patrick M. Kelly, PE PSP MANAGING EDITOR Marvin Gelhausen
preselect@aacei.org mgelhausen@aacei.org

PAST PRESIDENT GRAPHIC DESIGN Little Fish Design Company

James E. Krebs, PE CCP FAACE info@littlefishdesigncompany.com
pastpres@aacei.org
ADVERTISING SALES Business Development Coordinator
SECRETARY Joanna Boggs
Mark C. Sanders, PE CCP CFCC PSP +1.304.296.8444 x1122
secretary@aacei.org jboggs@aacei.org
TREASURER
Cindy L. Hands, CCP
+1.304.296.8444
treasurer@aacei.org
web.aacei.org
DIRECTOR TECHNICAL
Dr. Stephen P. Warhoe, PE CCP CFCC
TechDirector@aacei.org

DIRECTOR EDUCATION
Jeffrey Milo, PSP
EdDirector@aacei.org
DIRECTOR CERTIFICATION
Scott A. Galbraith, PE CFCC
CertDirector@aacei.org
DIRECTORS-AT-LARGE
Husain Al-Omani, CCP CEP DRMP EVP PSP
DirectorAtLarge1@aacei.org
Michael J. Bennink, PE CCP PSP
DirectorAtLarge2@aacei.org
Hannah E. Schumacher, PSP FAACE
DirectorAtLarge3@aacei.org
EXECUTIVE DIRECTOR/CEO
Debra L. Lally, CAE
dlally@aacei.org
AACE® International - The Authority for Total Cost Management®
OUR VISION - To be the recognized technical authority in cost and schedule
management for programs, projects, products, assets, and services.
OUR MISSION - The members of AACE® enable organizations around the
world to achieve their investment expectations by managing and controlling
projects, programs, and portfolios; we create value by advancing technical
knowledge and professional development.
Cost Engineering (ISSN: 0274-9696/23) is published digitally on a bi-monthly production schedule by AACE International,
Inc, 726 Park Avenue #180, Fairmont, WV 26554 USA. Copyright © 2023 by AACE International, Inc., All rights reserved.
This publication or any part thereof may not be reproduced in any form without written permission from the publisher.
Access to the bi-monthly Cost Engineering journal digital files is a benefit of AACE International membership and
requires a member login and password. There is no subscription service for the Cost Engineering journal other than AACE
membership. Digital access is on an individual use basis and not available on any group access basis. Cost Engineering is
a refereed journal. All technical articles are subject to a review by the AACE International Cost Engineering Journal Review
Committee. Abstracts are only accepted in our annual AACE “Call for Papers” for our Conference & Expo. Accepted abstracts
must be followed up with a full approved manuscript that is presented and attendee evaluated at one of our Conference
& Expo events. Top rated manuscripts will be considered for publication in the Cost Engineering journal. Any unsolicited
abstracts received at other times throughout a year will receive e-mail notice to submit in our next “Call for Papers.”
Copying without written permission of AACE is prohibited. E-mail requests for photocopy permission to editor@aacei.org.
ADVERTISING COPY: Contact AACE International, Inc, 726 Park Avenue #180, Fairmont, WV 26554. Telephone: 304.296.8444,
extension 1122. E-mail: marketing@aacei.com for rates. Advertisers and advertising agencies assume liability for all
content (including text, representation, and illustrations) of advertisements published and also assume responsibility
for any claims arising and made against the publisher. The publisher reserves the right to reject any advertising that is
not considered in keeping with the publication’s mission and standards. The publisher reserves the right to place the
words “advertisement” with copy which, in the publisher’s opinion, resembles editorial matter. All advertising accepted for
publication in Cost Engineering is limited to subjects that directly relate to the cost management profession. Current rate
card available on request. COST ENGINEERING DEADLINES: Submissions for Cost Engineering must be received at least 30
days in advance of the issue date. Send to: Business Development Coordinator, AACE International, Inc, 726 Park Avenue #180,
Fairmont, WV 26554 USA. Deadlines do not apply to technical papers.

Policy Concerning Published Columns, Features, and Articles

Viewpoints expressed in columns, features, and articles published in Cost Engineering journal are solely those of the
authors and do not represent an official position of AACE International. AACE International is not endorsing or sponsoring
LEARN MORE AT: the author’s work. All content is presented solely for informational purposes. Columns, features, and articles not
web.aacei.org/about-aace/structure designated as Technical Articles are not subject to the peer-review process.

JA NUA RY/ FEBRUA RY 2 02 3 3

LETTER FROM THE
EDITOR
BY MANAGING EDITOR MARVIN GELHAUSEN
Building a
Technical Article
Happy New Year! We ring
in 2023 with the posting of
three of the top papers from
the 2022 AACE International
Conference and Expo. To get
the new year rolling, check
out papers discussing total
cost management, risk,
and estimating.
4 JAN UA RY/FE B R UA RY 20 23
This issue of the Cost Engineering journal is featuring the following:
TECHNICAL ARTICLE 1
“Proactive Cost Engineering Techniques to Avoid Common Project Failures,”
is authored by Christopher W. Carson, CEP DRMP PSP FAACE. This
Total Cost Management themed paper was first presented as TCM-3813.
TECHNICAL ARTICLE 2
“Risk Analysis and Contingency Estimating for Class 10 Estimates,” is
authored by John K. Hollmann, PE CCP DRMP FAACE Hon. Life. This
risk themed paper was first presented as RISK-3908.
TECHNICAL ARTICLE 3
“False Estimate Accuracy,” is authored by Dave Kyle, CCP CEP. This
estimating themed paper was first presented as EST-3855.
We asked each author to elaborate on how they selected their topic and
what they want you the reader to get as takeaways from reading their
articles. We hope this additional background insights will give you a better
understanding of each article.
 COMPETITOR INTRODUCED CHRIS TO AACE
TA 1. Proactive Cost Engineering As an established author and presenter, Chris outlined how he learned
of AACE and why he became a member and remains a member. He says,
Techniques to Avoid Common “Working for contractors, I started in commercial, light industrial, non-
process work as a project controls estimator, and when I started getting
Project Failures involved in process work, I discovered the Recommended Practice for
Classes of Estimates and was starting to use them. Then when I shifted
BY CHRISTOPHER W. CARSON, CEP DRMP PSP FAACE over to consulting, I became friends with a competitor of mine, Bob Kelly
(Robert D. Kelly), and he introduced me to AACE and recommended the
BEST PRACTICES REDUCE RISK annual meeting. As I realized that all the disciplines that I called project
Author Chris Carson, in explaining how he developed this article, says, “I controls were covered by cost engineering in AACE, I decided to join. At
take lessons learned from industry studies, forensic analysis symptoms of my first AACE annual meeting in 2000, I went to Bob’s session on “Labor
failure, and success stories, and try to capture those to improve my best Productivity Impacts: A Case Study”, and Bob introduced me around and
practices. I realized that there are many areas of risk during the lifecycle of that convinced me to come back.”
a project and cost engineering best practices
reduce the risk in each of those areas, so
I thought that this paper would be able to
help map out a future complete process
for integrated project controls or cost “I thought that this paper would be able
engineering to avoid claims and improve
the probability of success.” He adds, “I to help map out a future complete
also found that my Arcadis best practices
were not as thorough as I’d like in this process for integrated project controls
area of integrated cost engineering/project
controls; we have a PMCM Standards of or cost engineering to avoid claims and
Practice website where I drafted the initial
cost and schedule best practices, but I want improve the probability of success.”
to expand it so a construction manager or
cost engineer/project controls practitioner CHRISTOPHER W. CARSON, CEP DRMP PSP FAACE
can go to the site, choose the project phase,
and immediately see the range of services
and support we provide. I’d been working on that concept, to identify the
services and tasks needed in each project phase and this paper helped me AACE RECOMMENDED PRACTICES MOST USEFUL RESOURCE
work out the areas of major concern.” He notes, “My next step in this effort Asked what he would you list as the most beneficial and useful resource
is the paper that I am co-authoring with my son, Leo Carson, for the 2023 that he has received from being an AACE member, Chris responded, “The
Conference.” Recommended Practices have been the most useful, supported by the
virtual library and all the great papers available with a search. I’ve used
YEARS OF LESSONS LEARNED the RPs as the basis of all my standardization of processes over the years;
Further explaining his article, Chris says, “This is a culmination of aligning with a known and respected set of publications improves the
years of lessons learned about symptoms of failure to produce the right quality of my processes and procedures.”
practices to avoid those symptoms. It’s based on recognizing the risk of
failure and ensuring that we perform the appropriate cost engineering AACE CERTIFICATIONS TO NETWORKING GIVE
techniques to improve our practices.” The takeaways he wants you the HUGE CAREER BENEFITS
reader to digest are: Commenting on how AACE membership has benefited his career, Chris
responded: “This is not a simple answer because it’s been huge for me in
• First—“Lessons learned from claims and other failures are the everything from certifications to networking. I was pretty experienced
best way to develop proactive techniques to reduce the risk of when I discovered AACE and adding another project did nothing for my
those failures.” resume but adding my first AACE certification made a great difference.
• Second—“An integration of cost engineering or project controls with Then I slowly got involved in review of RPs and writing papers, and that
the design process yields a massive opportunity to take control of a enabled me to improve my resume with references to presentations I’d
project with the right processes and improve likelihood of success. given and references to peer-reviewed publications. As an expert witness
And last, that the TCM Framework lays out the best approach to for construction claims, it made quite a difference, and even more so in
implementing good cost engineering techniques to meet owner goals.” writing expert reports. As part of a scientific analysis, I needed to fulfill
Daubert requirements. To do that, we have to demonstrate an industry
IMPROVED PERFORMANCE FOR CONSTRUCTION INDUSTRIES accepted methodology that has been peer reviewed and can be replicated.
Chris believes, “All construction industries will find improved performance Aligning with AACE has made all that possible and easy to achieve.”
by following these suggested best practices. Any project controls or cost Chris continues, “The networking with other experts has improved
engineering practitioner, from estimators to schedulers to risk managers, my own best practices.” He says from debating about AACE products
will improve project performance if they embrace these techniques.” to making connections in the industry, these are things he could never

JA NUA RY/ FEBRUA RY 2 02 3 5

have attained otherwise. “When I knew I needed to leave my previous
company, it was AACE connections that provided multiple options, and
I chose Arcadis in a large part due to the AACE relationships I had with
senior staff. People like Lee Schumacher, John Livengood, Patrick Kelly,
Mark Guevara, and Joe Seibold were here at Arcadis, all people for whom I
had a lot of respect.”
He adds, “My business development efforts have been enhanced as
well by this involvement. I picked up a large federal government delay and
disruption claim solely by taking a casual call from an attorney to discuss
a case, introducing the Forensic Schedule Analysis RP, and explaining
it. After a few calls, they decided to engage me to lead the effort with no
proposal process. That helped me realize that AACE industry leading
publications are seen as contributing to success and our position in the
industry as an industry leader.”
THE PATH FORWARD FOR AACE
Asked if there is one thing could AACE do to better meet member needs,
Chris says, “AACE is already doing much better in the one weakness that
I believed we had, publicizing the great work that we do and the industry
leading publications. The one remaining area would be to engage with
the federal government to improve the acceptance of all the AACE
certifications, similar to the way that some other industry associations
have done.”
AUTHOR’S VOLUNTEER INVOLVEMENT WITH AACE
Chris is an experienced AACE author and presenter. He says, “I’ve
presented about 80 conference papers, with 58 of them published, 37 in
AACE National Conference Proceedings, and 21 published in journals.”
He adds, “I attended AACE annual meetings from 2000 until 2005, and
then in 2006 I wrote my first AACE paper and presented it in Las Vegas. I
have written, presented, and had published at least one paper every year
since 2006.”
Continuing this tradition of AACE participation and volunteering,
Chris notes that he is “Co-authoring six papers and moderating one
session for the 2023 Conference.” These will include the following:
• Assigning Credit for Planned Mitigation in MIP 3.4, with Kenji Hoshino,
CFCC PSP; John Ciccarelli, PE CCP PSP; Mark Nagata, PSP; and Eric
Schatz, CFCC PSP (Arcadis).
• Implementation of an Integrated Phase-Gate Project Controls Process,
with my son, Leo Carson.
• Linear Scheduling for Claims, with Santosh Bhat, PSP.
• Productivity Analysis to Extend Delay Analysis, with Glen Palmer,
CFCC PSP; Steve Warhoe, PE CCP CFCC; and Justin Ovson.
• Scheduling With the Lights On – Mitigation and Recovery, with Robin
Haller, CCP PSP (Arcadis).
• Use of Duration-day Earned Value to Monitor Performance, with
Andrew Dick, PSP (Arcadis).
• Moderating the Truth or Lie Fellows Edition Panel Discussion, with Ron
Winter, PSP; Kenji Hoshino, CFCC PSP; John Ciccarelli, PE CCP PSP;
and Glen Palmer, CFCC PSP.
AN OBLIGATION TO GIVE BACK
Potential authors who may never have submitted for an AACE Conference
& Expo can learn how to get involved and how to volunteer by talking to
any of the experienced authors and presenters from prior AACE events.
Chris says, “Every competent practitioner of cost engineering or project
controls disciplines has some component of their everyday craft that they
do exceptionally well that others would love to hear about. We all have
6 JAN UA RY/FE B R UA RY 20 23
an obligation to give back to the industry and this is the ideal way; their
techniques will be memorialized in the paper. The process of writing,
responding to review comments, and presenting will serve to help improve
the paper and once they have laid out their beliefs, they get constructive
feedback from AACE members to help improve their processes.”
He says, “Writers and presenters get comfortable with writing a
coherent and valuable manuscript, meeting writing quality standards,
researching industry practices, addressing groups of people, and
responding to questions and potentially challenges. All of this makes
them much stronger in their everyday roles, whether it’s debating a
contractor over a change request, or defending their schedule or expert
report from the owner or opposition. It also exposes them to a lot of
high-end talent, learning techniques for their job and role, and makes
networking fun.”
HIS GRANDCHILDREN ARE IMPORTANT TO CHRIS
Something you may not know about Chris is that he still enjoys
woodworking as a hobby, originating from his carpentry days. He notes,
“I just installed a bookcase on a closet for my 9-year-old granddaughter,
creating a hidden door for her Narnia fixation. This year was also an
opportunity to pass along history; I gave my 12-year-old grandson the
Winchester semi-automatic 22 caliber long rifle that my granddad gave me
on my 12th birthday.”
TA 2. Risk Analysis and
Contingency Estimating for
Class 10 Estimates
BY JOHN K. HOLLMANN, PE CCP DRMP FAACE HON. LIFE
ANALYTIC CHALLENGES
Author John Hollmann says, “I have been continually learning about risk
quantification my whole career. But when you tackle one analytic challenge,
you always seem to find another. A great challenge on my "to solve" list has
been the huge range of cost uncertainty observed in research around Class
5/10 estimates. This paper was just my latest notch in the "solved" category
(well maybe not completely solved).”
Further explaining his work, John says, “The paper was not a case
study, but the culmination of having worked on risk analysis of early
estimates for a couple decades. It is based on a prevailing need, not any
one actual project.” With that said, there are takeaways that John wants
readers to get. He says these are:
• To raise awareness that at Class 5 and 10 there is never really just one
scope; there are always options.
• Accuracy ranges on those estimates is often two times what AACE
Class RPs show.
• We need to use a risk quantification method that considers the range
of all the viable options.
Addressing the target article for this article, John says, “All industries will
benefit, but it is owners who do all the work at Class 5/10. If you are with a
contractor in execution phase or small projects, it will not be of much interest.”
 established AACE author and
“When you tackle one analytic challenge, you presenter, John is willing to share
tips he thinks can benefit new
always seem to find another. A great challenge talent that may be considering
writing and then presenting their
on my 'to solve' list has been the huge range first AACE technical paper. He
says, “It is a great way to learn and
of cost uncertainty observed in research develop your writing and speaking
skills. Pick a topic you are not an
around Class 5/10 estimates. This paper was expert in but where there is a hole
in AACE's technical library. Study
just my latest notch in the 'solved' category it inside and out, then share your
learnings. Everyone will benefit.”
(well maybe not completely solved).”
JOHN’S EVOLVING HOBBY
JOHN K. HOLLMANN, PE CCP DRMP FAACE HON. LIFE EXPERIENCES
Something you may not know
about John is that he likes to try
his hand at new things. He says, “I
consider myself a "craftsman". I
John is an established AACE author and presenter. Outlining how he am always creating something. Lately it has been funky pendulum clocks;
learned of AACE and why he became a member and remains a member, I did 5 in the last year. Before that it was 3D kaleidoscopes, before that
John says, “I transitioned from mining engineering to cost engineer in mid- stained glass, and so on.”
career and in a very short time span. I needed to learn everything about
cost engineering pronto. My de facto mentor directed me to look into
AACE to give my learning a boost.”

AACE IS AN OPEN INVITATION TO CONTRIBUTE AND

MAKE A DIFFERENCE
John says what many list as AACE resources, he sees as opportunities. “I
would not call it a resource, but an opportunity. RPs are the best resource,
but when I joined there were very few beyond 10S-90 (terminology). I
soon discovered that AACE is an open invitation to contribute and make
a difference; I ended up writing a lot of RPs; a benefit to the contributor as
well as a resource for users.”
AACE MEMBERSHIP GIVES VALUABLE CAREER BENEFITS
Looking at how AACE membership has benefited his career, John says
“AACE got me on the right foot technically at the start. And then offered
an opportunity to become known via writing, speaking, contributing, and
leading. It is invaluable to a consultant (as I have been for almost 20 years).” ARTICLE TAKEAWAYS
TA 3. False Estimate Accuracy
BY DAVE KYLE, CCP CEP
A GUIDE TO DISCERNING ESTIMATE QUALITY
Author Dave Kyle, CCP CEP, says that his thoughts in selecting and
developing this article are based upon his prior work experience. He
explains by saying, “I have been subjected to many mediocre estimates
that were accompanied by bold claims of excellence. I wanted to offer a
guide to truly discerning estimate quality.” He notes that he has used
“Many past examples” in pulling together the content for this article.
The takeaways he wants readers to get, include the following:

MEMBERSHIP GIVES AUTHOR VOLUNTEER OPPORTUNITIES
Hollmann flipped the question of what one thing could AACE do to better
meet his needs. His response is, “It’s hard for me to say because it has
met my needs fine. I was never looking for AACE to "do" anything for
me. I was looking for a place where I could "do"; and it offered, and still
offers, great venues to jump in and contribute. Reviewing his volunteer
contributions, John notes, “My CV lists 27 papers, most at AACE. I think
7 were published in Cost Engineering. I went to my first annual meeting in
1989, where I presented a paper along with Larry Dysert when we both
worked at Kodak.”
EVERYONE BENEFITS FROM NEW TECHNICAL
PRESENTATIONS
John is not presenting at this year’s Conference & Expo. He says,
“Unfortunately, I will not be traveling to conferences in 2023, and
attendance is required to present AACE papers.” However, as an
• The estimate recipient must have a good level of competence in
estimating.
• Claims of excellence must be supported by detailed evidence.
• Estimate reviews must be thorough, as they will greatly impact the
many parties that use the data.
Dave believes any industry can benefit from the information he is
sharing in his article. He says, “The principles in this article can be applied
in every estimate review situation.”
DAVE SAYS TO JUMP IN AND SEEK ADVICE
Dave also is an experienced AACE author and presenter. He says, “I have
presented or co-presented 8 papers, led one workshop, and rewritten
three RP’s. This is the second one to be published in the journal, and I
have been involved in the AACE Conference & Expo’s since 2010. He
will not be presenting at the 2023 Conference & Expo. However, his

JA NUA RY/ FEBRUA RY 2 02 3 7

 advice for less experienced
authors and presenters is to,
“Simply jump in and seek advice
from other senior AACE members.
Select a topic in an area you are “[My AACE membership] has allowed me to
weak in (or see as a weak area
in your work sphere) and would build a world class estimating department for
benefit your career. Writing
a paper will give you a deep my employer from scratch in a much shorter
understanding of the subject.”
time and with better results than if I had tried
DAVE SAYS AACE GAVE HIM
INCREDIBLE MENTORING to do this on my own.”
How and why did Dave become
an AACE member? He says, “I DAVE KYLE, CCP CEP
was introduced to AACE by
a fellow employee who was a
member of my local section
(Calgary-Chinook). I was
interested in better understanding my new profession.” Asked what my own.” He says one thing AACE can do to better meet member needs is
AACE resources have been most beneficial to him, Dave responded, “Two simply, “Continue to improve the organization and searchability of the vast
things, the incredible mentoring I have received from fellow members, collection of resources.”
and the professionally written AACE publications.” He believes his AACE
membership has also benefited his career. He says, “It has allowed me to RAILROADING AND THE OCEAN
build a world class estimating department for my employer from scratch in A couple things you may not know about Dave is that he is, “a fan of “N”
a much shorter time and with better results than if I had tried to do this on scale model railroading and loves being on the ocean.”

JOIN US FOR THIS

UPCOMING AACE
LIVE

WEBINAR
Spotlight on RP 27R-03:
Schedule Classification System
AND Spotlight on RP 37R-06:
Schedule Levels of Detail
Jan. 25, 2023 • 11 AM ET (US)
Presenter: Host:
H. Lance Stephenson, CCP Mirela Tsagkari, CEP

REGISTER TODAY!

8 JAN UA RY/FE B R UA RY 20 23
Proactive Cost Engineering
Techniques to Avoid
Common Project Failures
BY CHRISTOPHER W. CARSON, CEP DRMP PSP FAACE

Editor’s Note: This article was peer-reviewed and presented as TCM-3813 at the 2022 AACE Conference and Expo. It scored high
by the Technical Board’s Review Committee using the Technical Paper Evaluation Criteria (TPEC) spreadsheet. It also scored high on
attendee review evaluations. Successful completion of these pre-steps resulted in this article being selected for publication in the Cost
Engineering journal.

ABSTRACT
Success in project delivery is a function of anticipating the areas of potential failure and managing those risks across
the full project lifecycle, from the inception for a project to finally turning the completed asset over to operations.
Failures in projects almost always relate to cost and time overruns that are directly caused by these risks.

Maximum and most effective proactive resolution of these failures requires a risk-oriented planning and
monitoring culture integral to the cost engineering disciplines of cost and time management. There is no
simple one-step process, but rather it requires a proactive and planned effort, and includes looking for
opportunities to improve, as well as threats to performance. The author uses lessons learned from disputes,
resolution, recovery, studies, and projects—to build into his cost engineering practices the best proactive
approach to identify the symptoms, recognize the potential failures, protect from the negative risks (threats), and
exploit the positive risks (opportunities) to achieve successful projects and programs.

This approach addresses specific traditional areas of construction and program management that commonly
fail, such as scope definition and change management, and offers proven solutions. This article provides
recommendations for resolution of the potential failure points in each of the phases of the project lifecycle
gleaned from lessons-learned from the author’s over-45-year career.

JA NUA RY/ FEBRUA RY 2 02 3 9


Introduction
Owner risk tolerance is a necessary discussion as an integral part of
planning a project or program. Understanding the level of tolerance is
vital to several of the tasks in the planning effort. Risk can and should be
addressed in the scope definition, in the contract type, in the contractual
language, in the choice of project delivery method, in the change
management process, in the quality and experience of the management
staff, in the procurement process, in the integrated risk/schedule, in the
implementation of the plan, and when executing the project. Types of
risk run the gamut from cost to schedule to political to performance risk,
and each can be dealt with differently in the planning effort to ensure the
appropriate risk assignment and acceptance for each owner.
AACE International has a number of recommended practices that
address risk management, from implementation of risk management to
use in determination of contingency.
The Construction Management Association of America (CMAA) notes,
“In considering risk allocation, the owner should strive to assign risks
to those parties that can best exercise control over those aspects.”1 This
requires a constant evaluation of the ability of each party to manage the
risk, and alignment with the risk under review.
It is also vital to recognize that risk includes both threats and
opportunities. A robust risk management effort identifies performance
improvement opportunities and supports an open collaborative culture on
projects, such that the entire team is oriented toward meeting the owner’s
goals and helping the contractors make money. That is true success and is
achieved at the lowest stress and greatest satisfaction to all stakeholders.
An integrated approach to the entire cost engineering/project controls
disciplines is clearly shown in the Total Cost Management (TCM)
Framework2 and has been proven to be highly successful, however,
there are opportunities to expand this approach to other practitioners
. This would help to reduce the occasions where it is not uncommon for
cost estimating to be separated from cost management, planning to be
separated from scheduling, shallow risk management efforts failing to
provide a full risk management program, and analysis of absorbed delays
performed without consideration for the quality of the source documents.
Sometimes this is a result of an owner who thinks that a cost engineer,
project controls expert, or construction manager is only needed during the
construction process, and sometimes budget is not allocated to support
this functional requirement.
The author has developed cost engineering processes used during
the project lifecycle, from pre-project to post-construction, and has
organized those processes into the major areas of cost engineering
services. This helps align implementation of cost engineering services with
client needs and wishes and minimizes the risks that come from partial
implementation of cost engineering disciplines. It also helps the owner
client understand the importance of implementing cost engineering at the
appropriate phases in the appropriate places.
Background
This article will use cost engineering and project controls interchangeably
since much of the non-process world uses project controls rather than
cost engineering. The techniques and recommendations in this article
apply to planning and scheduling, cost estimating and management, risk
management, document controls/contract administration, and claims
10 JAN UA RY/FE B R UA RY 20 23
avoidance/forensic analysis/dispute resolution, which are typically rolled
up into project controls.
The author’s perspective on the terms cost engineering and project
controls comes from experiences early in his career. When working for
construction managers and/or general contractors, a typical scenario was
that an owner would reach out and state that they had an opportunity
to build a shopping center, an apartment building, or build a small
manufacturing plant for an end user, and needed to know the extent of
the investment and whether it made sense to risk credit and funds for this
project or just leave their funds in current investments. The author’s role
was to pull together all the information and develop what was often called
a “proforma” by the accountants or financial staff. This proforma provided a
conceptual cost estimate, AACE Class 5, with an accompanying AACE Class
5 schedule, and was used to generate a business plan with financial ratios
and cash flows showing potential income and expenses run over multi-year
scenarios, also reviewing project and asset lifecycle costs with an assessment
of the cash flows generated during asset management as well as the project.
This is consistent with the example in the TCM Framework in Section
1.1.1, “Total Cost Management Framework and Related Terminology”3
where it states, “For example, a real estate developer may build, maintain,
renovate, and then demolish an office building during its life cycle—at
each phase of the building life cycle the developer makes significant
investments. To manage these investments, the building developer
monitors building operating costs and profitability; evaluates alternative
investment opportunities; and initiates, plans, and controls improvement
projects. These activities are all within the scope of the TCM process.”
But the author performed these roles acting as the project controls
expert within the construction management project delivery method. Later
as Mr. Carson started providing more services to the process side of the
industrial sector and was introduced to AACE, he discovered that those
same services were called cost engineering. So, the author’s perspective
is that the term project controls is synonymous with cost engineering, it
depends more on where in the industrial or commercial world the work is
being performed. And that seems to be consistent with how these services
are labelled in much of the construction world.
Scope Definition and
Design Maturity
The quality of scope definition is the first decision that should be made
by any owner but is routinely ignored or minimized, partly because it is
driven by the investment in design services. The level of scope definition
directly affects the level of risk for any given project. There is a wide
range of levels of scope definition based on the contract with the designer,
and again with the quality of the final design as disseminated to the
contractors at tender/bid and procurement stages.
The levels of scope definition and maturity of project deliverables are
vital to the range of accuracy of the estimates and schedules produced based
on the scope definition. It is not reasonable, for example, to expect that an
estimate based on 15% scope definition or drawings will be accurate within
+/-5% of the final bid pricing. But if the range of accuracy of the estimate
1
“An Owner’s Guide to Project Delivery Methods”, CMAA, Copyright 2012
2
TCMF [34]
3
TCM Framework [34]
and schedule is considered, then it is understood
that the estimate is not a deterministic single
value price, but rather a range within which the
accurate estimate should fall once the scope is
fully defined. Therefore, a Class 3 estimate based
on 15% scope definition would be accurate
within the range of -10% (low) to +40% (high).
Since the risk of overrun is always greater than
the risk of underrun of budgets and schedules,
the industry would be better served by reference
to the potential overrun budget, the high end
of the range of accuracy, rather than the range
itself, and significantly better than reference to
the deterministic middle range price. That would
result in budgets that are able to be reduced as the
scope is better defined and ensure that original
budgets would sanction sufficient funding.
AACE notes the maturity of scope definition is
a determinant for the range of accuracy of the cost
estimate, as well as the appropriate usage of the
schedule based on the degree of project definition. FIGURE 1 Cost Estimate Classification Matrix for Construction from AACE RP No. 17R-975
One way to look at this process is that during
early phase estimates, perhaps an AACE Class 5
conceptual estimate, there would be a direct cost
element of the estimate with a large contingency
based on low confidence in the scope definition,
but at highly detailed estimates, perhaps a Class
2 line-item estimate, the fixed price element
would be much larger since the scope is well
defined and a much smaller contingency from
fewer unknowns. But, with a good estimating
process that takes into account market conditions,
escalation, and other factors, with no scope,
contract, or delivery change, those two estimates
should balance out, yielding a Class 2 estimate
total that is similar to the Class 5 estimate total.
In addition, maintaining the schedule for the
design process can be challenging, often due to
a lack of planning for the design. Once maturity FIGURE 2 Cost Estimate Classification Matrix - From AACE RP No. 18R-97 6
milestones are established, the scope of work
definition on a discipline-by-discipline basis Figure 1 shows the suggested estimate classes and the associated
can provide both total hours and costs expected to be expended at those maturity level of project definition deliverables aligned with the
milestones. Without the detailed review of the milestone maturity levels, methodology and expected accuracy range.
the architect/engineer (A/E) often does not recognize cost or time overruns Figure 2 addresses these same issues, but specifically for the process
because they either delay publication of the milestone documents or industry. The accuracy range speaks directly to the risk associated with
publish a less-than-specified-maturity document that is not easy to evaluate. the cost estimate; the tighter the accuracy range, the lower the risk of
The ramifications of false publication of scope definition level submission meeting that cost. The accuracy range also demonstrates the benefits for
include cost and schedule evaluations that contain a wider range of probabilistic risk assessment that help ensure better understanding of the
accuracy than planned. This limits the ability to influence design based on potential consequences of the decisions.
scope creep, scope gap, and gold-plating, and may result in cost overruns When the construction industry seems to recognize that an estimate
that will require late-term cost reductions. These cost reductions are provided at 30% scope definition is unlikely to be accurate to within 5%
commonly mis-labeled as value engineering (VE), when they are not true of a deterministic estimate, the owner is better protected recognizing
VE but rather simply removal of scope or quality to save money, defeating that generally the budget estimate at 30% scope definition is more
the goal of “design-to-budget” that can be achieved with the right process. appropriately considered as a -10% / +40% range of accuracy. This
As noted in the paper, “Successful Design Scheduling” 4, there is
significant value in early conceptual design schedules, “Conceptual 4
“Successful Design Scheduling”, Carson, Fletcher, Jones, Carson,
project schedules will show time allowed for phases of design, producing 2020 AACE International Transactions.
increasing levels of scope definition or design maturity, until final 5
RP17R-97 [32]
construction documents are produced.” 6
RP18R-97 [31]

JA NUA RY/ FEBRUA RY 2 02 3 11

ensures that the estimate aligns with the scope maturity and the ability
of the estimator to use appropriate tools to estimate the work. This is not
to say that a knowledgeable estimator developing an estimate from 30%
scope definition might not be able to produce an estimate that is much
more accurate than this range, but it would necessarily depend on good
historical data and clear understanding of the scope.
Capturing the full scope definition requires an appropriate and
reasonable schedule and allows adequate time to support the building-in
of quality instead of inspecting quality. Finishing a project on time using
the right schedule will help minimize delays and reduce cost overruns.
This is part of AACE’s Total Cost Management philosophy, and classifying
schedules like cost estimates aligned with the typical phases and stage-
gates defines the project life cycle.
AACE Recommended Practice 27R-03, “Schedule Classification System”7,
provides these schedule class designations and shows how they align with
project scope definition maturity. From this RP, Figure 3 addresses how
the scheduling methods achieve reasonable project duration and planning
dates while covering scope definition maturity.
FIGURE 3 Schedule Classification Matrix - From AACE RP No. 27R-03
Lessons learned from forensic analysis of disputes indicates that
constructability reviews are ranked third “most effective claims avoidance
techniques”8, and industry studies show that the most serious risks to
project success lie in the scheduling effort, the cost estimating effort, and
the failure to use risk management processes to ensure appropriate budgets
and project durations. In fact, risks from schedule, cost, and risk are twice
as serious to project success as technical, design, and engineering issues.9
In addition, in order to ensure the maturity and appropriateness of
the design documents, a good constructability and biddability review
performed by the stakeholders on the project will help improve the scope
definition. Good schedule constructability helps to identify problems with
scope definition such as design errors and ambiguous specifications.10
Studies have indicated that constructability reviews, evaluated by
percentage of change orders, showed “monetary savings associated with
constructability reviews resulting in a conservative estimate of 1.25% of the
project budget.”11 The author has performed constructability reviews that
often yield between 5:1 and 20:1 return on the investment in the review,
12 JAN UA RY/FE B R UA RY 20 23
and believes they should be performed in a number of project control
areas; schedule constructability (sequencing), biddability (attractiveness
to local bidders), and reasonableness of the schedule (quality control and
aggressive schedule). In addition, of course, engineering constructability
is vital as well. Done well, the review will identify uncertainties in the
bidding documents that need clarification, identify missing scope from the
drawings, and determine appropriate ways to install the work.
Contract Type
Contracts are primarily either fixed price (stipulated sum) or cost
reimbursable formats. There are some variations such as guaranteed
maximum price (GMP), cost plus fee, unit price, fixed price with incentives,
or combinations of these alternates. Allocation of risk is determined by the
type of contract, with fixed price contracts shifting performance risk to the
contractor and cost reimbursable contracts accepting risk by the owner.
With fixed price contracts, there must be adequate competition in
order to make the proposals effective in supporting the business case. The
contractor in a fixed price contract will accept a price which represents
assumptions of a reasonable apportionment of risk. This means that the
contractor must be able to estimate uncertainties in contract performance,
as well as fully understand the contract scope and price accordingly.
Less than fully mature scope definition in contract documents likely will
increase costs at bid and increase change management efforts, resulting in
higher change and total project costs.
Cost reimbursable contracts are often used when the uncertainties of
performance do not allow accurate costs to be developed and use of a fixed
price contract would yield very high bids. These contracts place the bulk of
the risk on the owner and should only be used in specific cases, especially
since the primary incentive for the contractor to control costs is reputation
and relationship with the owner. When used, the owner should recognize
that minimizing cost and time overruns require careful documentation of
actual cost and time, daily if possible. This approach will limit the ability
of contractors to confuse time spent on original contract work with time
spent on the additional time and material (T&M) scope.
One of the places where owners assume unintended risk is in the
change management process during design and construction. The goal
of change management should be to place the owner and contractor back
in the same risk profile as the original contract dictated, however, when
change management is not handled in a timely and effective way, the owner
often assumes additional risk, even if the contractor did not comply with the
contract requirements. The solution to this is to prepare accurate estimates and
time impact analyses that can be used to negotiate change orders, including
legitimate time extensions, as early as possible. This timely approach to change
management reduces the owner assumption of performance risk, avoids
claims such as constructive acceleration, and keeps the schedule as a good
model of project status capable of valuable use in analysis of delays.
7
RP27R-03 [35]
8
“2021 Global Construction Disputes Report – The Road to Early Resolution”,
Arcadis, 2021
9
SmartMarket Report, “Mitigation of Risk in Construction: Strategies for Reducing
Risk and Maximizing Profitability”, McGraw-Hill Construction, Bedford, MA, 2011
10
RP30R-03 [37]
11
“Benefits from Constructability Reviews”, Stamatiadis, Sturgill, Amiridis,
University of Kentucky, Transportation Research Procedia 25 (2017) 2889-2897,
World Conference on Transport Research, Shanghai, July 2016
 and time claims associated with the issue. If the goal is to maintain the
Contractual Language assignment of performance risk to the contractor in the original contract,
reservation of rights can move the risk of performance over to the owner
There are a variety of approaches to limit or shift risk in the contract, during change order negotiations and resolution. The defense to this
regardless of the project delivery method or contract type. These approach is to require all previous delays, change orders, and claims to
approaches are defined in the contract language and can affect risk for time be resolved with any time impact analysis submissions as well as prior
and costs, and require good legal consultation to ensure the appropriate to request for payment on each invoice. This language could include
language is included. specific risk protection such as requiring change orders include all direct
Time risk assignments occur with language to limit or assign and indirect costs including but not limited to impact, disruption, loss of
ownership of project float, which is generally total float. Delays that would efficiency, ripple effects, and other extraordinary or consequential costs.
be legally compensable to the contractor must occur on the critical path of Another case of risk shifting language is that of exculpatory contract
activities that control the project duration, so these are typically zero float clauses, sometimes called disclaimers, that attempt to absolve responsibility
activities. Since it is possible to assign the ownership of float, the owner for damages from future or unknown circumstances. This is a way to shift
can take this ownership and limit the ability of the contractor to earn unplanned and unidentified risk to the contractor from issues like third-party
extensions of time; however, this can place too much risk on the contractor uncontrolled risks. It also occurs in existing conditions such as geotechnical
to the detriment of the project. The quality of the schedule is a significant reports and owner limitations for information only or differing site conditions.
factor in managing the float and requires a high level of technical schedule These can also be “pay when paid” or indemnity clauses, all of which require
review in the baseline and all updates. experienced legal support to provide maximum value in the use.
When the contract is silent with respect to float ownership, it has The last set of risk shifting language is that of the “no damages for
been observed by the author that in most states in the U.S., project float delay”, and this limits delay entitlement to time only. It is important when
is owned by the project and shared by owner and contractor, which is using this type of language to ensure that no exemptions to no damages
consistent with AACE Recommended Practices.12 This is generally the fairest for delay are created by interference by the owner, bad faith, or delays that
allocation of risk and provides the best overall final project cost. The owner just were not contemplated. No damages for delay clauses shift risk to the
must manage this issue by protecting against a contractor inappropriately contractors who do not have the ability to control that risk, so the use of
sequestering or using up all available float. If the owner is not diligent, they this approach tends to increase costs and detracts from the collaborative
may discover that there is a change order needed that would then include construction team effort that is most effective.
extended general conditions for the extension of time required. Careful
schedule review and monitoring to ensure that float is accurately calculated
and reported is essential in protecting against this risk.
Another place where owners can protect themselves against performance Choice of Project
risks is by using language to limit or prevent the possibility of a contractor
pursuing a compensable extension of time based on an early completion Delivery Method
schedule. Case law suggests that a contractor has a right to finish early. If the
contractor bids a project that reduces project costs by planning to finish in There are four basic project delivery methods according to the Construction
less time than the contractual completion date (CCD) provides, they could Management Association of America (CMAA)13: design-bid-build (DBB),
earn extended general conditions if the owner causes a delay beyond the design-build (DB), construction management at risk (CMAR), and alternate
contractor’s early completion date and before the contractual CCD. There project delivery (APD), which includes integrated project delivery (IPD) and
are a number of possible clauses that could protect against the contractor’s multi-prime as well as several variations of these methods. Each method
early completion schedule and leave flexibility in the schedule for owner carries a different level of risk for the owner, and this is related to the amount
needs. This is especially important if the owner cannot take occupancy of of control that the owner accepts over the project. Risk and control are
the project earlier than the CCD, which can often be the case. inversely related so one way to reduce risk is to choose a project delivery
The subject of notice from the contractor to the owner about alleged method that lowers owner’s risk but also gives up more owner’s control.
delays is another place where risk can be controlled. Contract language
requiring the contractor to provide formal, written notice of any delay will
limit the risk of large change orders that come as a surprise to the owner 12
RP29R-03 [28]
with the late discovery limiting the ability of the owner to participate in 13
“An Owner’s Guide to Project Delivery Methods”, CMAA, Copyright 2012
mitigation decisions and actions. This language often defines
failure to provide sufficient or timely notice as a waiver of
rights to make a claim. Waivers can show up in change
order requests as contractual language related to required
processes to perfect a change request. If the contractor
breaches those requirements, they can lose entitlement to
the additional costs and time involved in the change.
A risk shifting approach that contractors often use is a
reservation of rights provided with change orders. This is an
attempt to keep options open for future claims of indirect,
consequential, and/or cumulative disruption costs and time
impacts. This approach can alter change order language
that otherwise notes that the change order settles all cost FIGURE 4 Project Delivery Methods - Risk and Control

JA NUA RY/ FEBRUA RY 2 02 3 13

 Choosing a Project Delivery Method Owner Needs - Factors to Consider (Developed by Chris Carson)
0 Worst Fit 10 Best Fit

Level Needs Wants

Owner Needs Low No
of Early Within Project On Single Lowest Shortest Professional High Reduced Availability Scope Needing Experienced Coordination Reduced Contract Best Ease of
Quality Cost Adversarial Flexibility Familiarity Innovation
Contro Estimate Budget Importance Schedule Contra Risk Duration Input Complexity Changes of Funding Definition Financing Team Required Claims Terms Value Procurement
Method Goal Relationship
l s ctor

PPP/P3 3 1 4 8 3 8 10 10 5 6 10 2 2 2 9 6 2 5 10 8 9 7 9 1 5 6

DBOM 4 0 3 9 2 8 10 8 6 7 10 3 3 2 9 10 0 6 10 9 10 9 10 2 5 8
Integrated
10 8 10 8 8 7 4 8 6 0 7 10 10 10 10 8 6 10 4 8 8 10 10 3 8 1
Project Delivery
EPCM 8 4 6 7 8 6 10 8 7 8 7 6 8 8 8 8 9 9 6 6 7 8 8 5 7 7
Bridging 7 4 3 7 6 6 10 8 7 7 7 5 6 5 8 2 6 8 8 8 6 7 7 4 7 5
DB 4 3 7 8 5 8 10 7 6 6 9 4 4 4 7 0 4 6 7 7 8 5 9 4 4 6
CM (Single
8 8 10 10 10 9 10 6 8 10 9 8 7 8 8 8 10 8 9 8 7 8 6 8 10 8
Contractor)
Agency CM
(Multiple 8 7 8 9 10 10 6 6 10 8 8 10 8 7 7 7 10 9 10 10 8 8 5 10 8 6
Contractors)
Multiple-DB 0 2 5 6 4 6 4 5 5 5 7 3 3 3 5 0 2 4 6 6 5 4 9 2 5 4
Design-Bid-Build 6 10 0 4 8 4 10 5 6 0 0 0 8 2 0 4 10 9 8 7 6 6 0 10 3 2
Program
8 8 8 10 10 8 7 4 9 8 8 10 8 8 6 9 8 8 8 10 10 6 6 8 8 10
Management
CM at Risk 6 6 7 8 6 6 10 3 7 6 8 8 6 6 5 6 6 8 8 7 6 6 7 6 9 5
Multi-Prime 6 9 0 4 5 4 0 1 7 2 4 6 6 4 3 6 6 5 7 8 2 4 2 9 5 4

FIGURE 5 Factors to Consider in Choosing a Project Delivery Method

This risk profile is illustrated in Figure 4 which lists the project delivery
methods from low to high risk. They are public-private-partnerships The Role of Risk-Based
(P3) (this is a similar delivery method to DB except for financing and
operations, which is done by the contractor), DB (which is similar to Contingencies and
engineering-procurement-construction or EPC), DBB, CMAR, and multiple
prime contracts (which places the risk of contract coordination onto the Management Reserves
owner). Other methods such as design-build-operate-maintain (DBOM),
which often includes financing, would likely rank to the left of P3 as less In addition to the range in estimate accuracy, there are risks related to
owner risk and control. a number of issues that should be identified in a risk workshop, from
The choice of project delivery method also depends on the level of incomplete scope definition to unforeseen conditions that must be
scope definition along with time availability for the design and construction. accommodated for a realistic estimate. These risks exist at any stage of
A DBB project cannot be used if the scope definition is not very mature or scope definition.
change management will exceed contingencies for time and budget. On Therefore, professional and experienced estimators recommend
the other end of the scale, attempting to provide too complete of a scope contingencies that are developed based on either good historical data or
definition for a PPP project will reduce flexibility and limit the innovation a risk-based determination. These contingencies can be reduced as the
freedom to control risks that is at the very heart of this type of delivery. maturity of the scope definition improves. Weak contingency estimating
Each type of project delivery method has risks that must be managed and misuse can account for a significant percentage of claims, which
to ensure success. For example, in the CMAR delivery, embracing good are often failures in properly assigning and managing project risks that
practices supports the “design-to-cost” goal. These good practices include should have been considered during the contingency determination. The
establishing a detailed preliminary budget, adhering to a formal stage-gate reasons for these failures often relate to failure to understand the level
review approach to cost/schedule/risk during design development, and of scope definition at the time of procurement, which affects the range of
correlating each evolving budget and award letters to the preliminary accuracy. Other reasons for failure can also be related to lack of risk-
budget. These allow for a reasonable and achievable final guaranteed based contingency determinations. Unless the owner organization has
maximum price when the CMAR becomes a general contractor and historical data about cost and time overruns, it is naive to assume some
takes on full performance risk. Without serious controls in place to arbitrary percentage of overrun for the contingencies. The determination
evaluate the CMAR budgets and schedule, and without ensuring the of contingency should start with a risk register of historical known and
competitiveness and accuracy of the award of subcontracts, the project can unknown risks (and which risks came to fruition); this also provides the
drain the owner’s contingencies early in execution, only to discover that ability to manage a structured contingency draw-down table during the
there are huge savings that might be split after final audits. Allowing this project, releasing contingency monies no longer needed back into the
contingency shift ties up contingency monies that should have been drawn project in a timely manner.
down for the owner’s benefit and returns the money too late for use in the If legitimate change happens and there is a fund set up to
project. This can happen when the contractual fee is low and the shared accommodate the change, there will likely be minimal, if any, impact to the
savings provide additional fee for the contractor. project. Once the categories of change are established (and many contracts
The author takes a detailed evaluation approach to recommending as well as AACE RPs offer definitions14), it is possible to plan for how to
the appropriate project delivery method, based on historical factors to be fund the changes when they occur. Outside of the US federal agency’s
considered. Owner needs, or factors for the owner to consider are ranked in use of standard EIA-748, there are two methods for funding changes;
order of importance for each project delivery method and allows an objective contingency and management reserves., The difference is that contingency
consideration of the best project delivery method to be chosen based on those is intended to be used for changes within the scope of the project that are
owner needs. These factors have been compiled from the author’s experience
and observations, as shown in Figure 5 and reproduced as Appendix A. 14
RP100R-19 [38]

14 JAN UA RY/FE B R UA RY 20 23
expected to happen even if the extent is not known (and is funded under Contingency and management reserves cover the risks that can
the authorized project amount), and management reserves are intended to be categorized, but a robust change management effort during design,
fund requests that are not included in the original scope description (this procurement, and construction is important to control and minimize these
money is withheld from the project team and is only released if the project risks. Use of a formal stage-gate process during the project lifecycle, but
team can demonstrate its need). especially during the design phase, is vital to supporting “design-to-budget”
AACE defines contingency in RP 10S-90, Cost Engineering Terminology, efforts. Use of a thorough review and evaluation of the procurement
as “An amount added to an estimate to allow for items, conditions, or process improves the selection of contractors and suppliers. Correlating
events for which the state, occurrence, or effect is uncertain and that the procurement basis to the budget and schedule helps ensure adequate
experience shows will likely result, in aggregate, in additional costs.”15 time and money is allocated. Use of a robust change management effort
Contingency should not include major scope changes, force majeure events, during design and construction ensures that original contract scope is
management reserves, escalation, and currency exchange rates. provided, that contingency is drawn down appropriately and according to
Contingency can be carried in the original budget, and during the the relief of risks, and management reserves are used appropriately.
integrated stage-gate process of project controls and design, can be When it comes to change management for a project, providing
subdivided into specific categories such as design contingency, estimating accurate AACE Class 2 or Class 1 estimates for changed conditions is vital
contingency, procurement contingency, construction contingency. Note to evaluate the costs. Without the ability to discuss specific quantities
that not all contingency funding is due to specific risk events, some is and unit costs for changes, the owner is at a huge disadvantage when
needed for accommodating the standard of care in the construction negotiating change orders. It is common to find the subcontract portion of
process, from design to estimating to construction. There is some amount a general contractor’s estimate that is poorly documented will be reduced
of design errors and omissions that happens but still does not exceed as a result of a detailed check estimate.
the industry standard of care which recognizes that scope definition in In addition, when there is a time impact from a changed condition or
the way of plans and specifications cannot be perfect. This is part of the delay, the costs for the extended general conditions when the project is
purpose for carrying contingency. truly prolonged can be a large part of the total change order. This makes
AACE defines management reserves, in the Cost Engineering Terminology it imperative that a good process to develop independent time impact
RP, as “An amount added to an estimate to allow for discretionary analyses (TIA) in order to evaluate the contractor’s TIAs. Armed with this
management purposes outside of the defined scope of the project, as independent evaluation, the negotiations are typically quicker and easier.
otherwise estimated.” This is where an owner would normally fund the items Once a predicted delay or impact event has been identified, prior to the
not included in contingency, such as scope change. Management reserves delay getting absorbed into the schedule and project, the goal should be to
would typically be carried outside the project, and managed by the program quickly move the owner back to the original risk allocation strategy from the
manager or owner, not the project CM team. The better the definition of these contract. For fixed price contracts, usually risk of performance was achieved
terms, the easier it is to manage and account for change orders. by assigning the cost and time performance to the contractor. Maintaining
Estimating management reserves is more difficult than contingency this position requires quickly and timely negotiating any extensions of time
because this fund is designed to cover unknowns such as improvements (EoT) that the contractor is entitled to receive after careful analysis to validate
in technology that might interest the end user to upgrade equipment that the request or need. Issuing the proper EoT in a timely fashion fulfills the need
was specified in the original scope definition, is still sufficient, but perhaps to allocate the risk of performance of the changed condition work properly
is no longer the most desired technology. However, the author believes that and eliminates the risk of constructive acceleration to the project. Owners
there should still be a risk-based determination of reserves, which takes are at risk of turning non-compensable time extensions into compensable
advantage of lessons learned and the experience of the team involved with acceleration efforts simply by not awarding legitimate EoTs as they are earned.
reserves determination. Control of risks is dependent on a full change management process
being implemented competently in order to ensure scope is defined and
the increasing maturity of scope definition is monitored to enable the
ability to “design-to-budget”.
Change Management Process CII (Construction Industry Institute) developed a research project “to
evaluate the level of engineering maturity needed at project authorization,
A planned and well-managed change management process is very
important to managing and minimizing risk for a successful project.
Planning for change management starts with a careful definition of
changes, establishing the types of change so appropriate funding
planning can be provided. Some changes are issues that occur in
most projects, such as unforeseen conditions, and some are issues
that cannot be easily anticipated, such as scope changes by end-
users. Planning for defined categories of changes allows alignment
between categories and funding.
If legitimate change happens and there is a fund set up to
accommodate the change, there will likely be minimal, if any, impact
to the project. Once the categories of change are established (and
again, many contracts as well as AACE RPs offer definitions), it is
possible to plan for how to fund the changes when they occur.

FIGURE 6 Front End Engineering Design Process (CII)

15
RP10S-90 [33]
JA NUA RY/ FEBRUA RY 2 02 3 15
but also the accuracy of these engineering deliverables.”16
This front-end engineering design process is shown in
figure 6, which indicates the Gate 3 that cannot be opened
to release further design development until the process
yields the appropriate maturity and accuracy of the design.
With maturity addressing the degree of completeness
and accuracy addressing the degree of confidence in
the measure of maturity, the research project developed
a tool to be used to assess the maturity vs. accuracy.
The research found a 24% cost difference between high
maturity high accuracy and low maturity low accuracy
front end engineering design as shown in Figure 7.
The tool was used to assess 11 projects of over $5.1B
in total construction value in the survey, ranging from
chemical plants to a storage facility, and yielded the 24%
cost difference in the summary shown in figure 7. FIGURE 7 Study about Summary Cost Difference in Maturity and Accuracy - CII17
Quality and Experience
of the Construction
Management Team
Managing risk starts with pre-planning and must be at the forefront of
management throughout the construction project. Weak efforts to develop
initial risk management plans without a very experienced team to support
and implement the plan will result in risk management plans sitting on
shelves providing no value.
While risk is a common buzzword, far too often stakeholders in
projects have a limited depth of understanding of risk and the risk
processes. Risk must be integrated into the construction management
processes, and the CM staff should be well versed in risk principles and
implementation.
Experience in risk management is very important to anticipate the
typical problems that occur and bring the lessons learned from previous
projects to the planning of each new project. Lessons learned can come
from project experience but also from change orders and claims and
dispute resolution experience. In fact, since some claims can result
from failures in risk management, these lessons are often more valuable
than project lessons. Engaging in forensic schedule and cost analysis
requires a deep understanding of CPM scheduling, forensic analysis
methodologies, negotiations skills, and cost and time legal principles. The
experiences of reviewing schedules and documentation to determine
what happened to cause the delays, determining the quantum of delay,
examining entitlement and liability, and placing responsibility for delays,
all contribute to a much better understanding of project risk and how to
control it. This means that CM team members who have forensic analysis
and dispute resolution experience tend to be more competent to manage
risk during the project lifecycle. Involvement in Industry association
publications such as the AACE’s Recommended Practice 29R-03, Forensic
Schedule Analysis, is a valuable reference that provides a thorough
explanation and taxonomy of methodologies used to analysis and resolve
disputes. Lessons learned during development of these types of industry
recommended practices are invaluable in predicting risks and mitigating
to avoid cost and time overruns.
While the author is not aware of very many academic programs that
seem to include detailed knowledge of risk management as a prerequisite
16 JAN UA RY/FE B R UA RY 20 23
for a professional degree, there are professional certifications that support
understanding of risk. At the construction/project management level,
the Project Management Institute (PMI) includes risk management
as one of the knowledge areas, so a Project Management Professional
(PMP) certification would indicate exposure to risk on project work, not
specifically construction projects, but still project risk. The Construction
Management Association of America (CMAA) offers the Certified
Construction Manager (CCM) certification and the CMAA Standards of
Practice as taught for the CCM fully integrate risk into the CM processes,
and these are specifically for construction projects.
When it comes to specialization in risk management, there are
three primary industry risk certifications in North America: the
Risk Management Professional (PMI-RMP) by PMI, the Project Risk
Management Professional (PRMP) by AACE, and the Decision and Risk
Management Professional (DRMP) by AACE International. Once again, the
PMI-RMP is not specifically designed for construction as are the PRMP and
DRMP, but the general risk processes are the same regardless of industry.
AACE believes that it is not possible to separate decision and risk analysis,
so both need to be taught and certified. Recommended Practice No. 71R-
12 notes, “In the asset and project management arena making decisions
and managing risks are largely inseparable topics.”18
Since control of risk to the owner involves cost and time, it is
important that an integrated effort of cost and schedule risk management
is undertaken, which elevates the value of the cost and schedule
certifications. For project management, the PMP certification is useful,
but for construction projects, the CCM is invaluable as it addresses these
areas. Specialization in cost and time certifications is important for CM
staff to support risk control for owners, and AACE International is the
best of the industry associations that issue these certifications. The AACE
Certified Cost Professional (CCP) is a cost engineering certification
which provides a good overview background in time, cost, and risk.
Cost estimators can earn the Cost Estimating Professional (CEP) and
schedulers can earn the Planning & Scheduling Professional (PSP)
certifications, both of which demonstrate a detailed understanding and
experience in cost and time.
16
“Front End Engineering Design Maturity and Accuracy Total Rating System
(FEED MATRS)”, Research Team 331, Assessing the Maturity and Accuracy of
Front End Engineering Design to Support Phase Gate Approvals, 2017
17
CII 2007 Annual Conference, Steve Cabano, Presentation by the Research Team 331
18
RP71R-12 [39]
 The largest risks to project
success are related to cost, time,
and failure to plan for financial
risks per a McGraw Hill study from
2011.19 This study is noteworthy to
recognize that the technical and
engineering risks are among the
least serious. This is in keeping with FIGURE 8 Most Common Dispute Causes in North America
the annual 2020 Global Construction
Disputes Report20 summarized in Figure 8 that shows the top two reasons
for claims in North America to be “errors and/or omissions in the contract organizations, methodologies such as advanced work packaging flows
document” and “contractor/subcontractor failing to understand and/or
comply with its contractual obligation”. This report shows that technical
and engineering risks are not in the top three dispute causes.
While part of the value of industry professional associations includes
CM professionals earning industry certifications, a greater part of the
value is the engagement in these associations by writing and presenting
papers on various cost, scheduling, and risk topics. This engagement
takes a CM professional from a local or regional expert in these fields
to an industry thought leader. At this level, the skills and knowledge of
these professionals has also provided innovative approaches to managing
risk. This further provides the advantage of embracing comments and/
or defending approaches from industry constructive criticism, which
serves to improve the approach and implementation. Lessons learned,
from successful and not-so-successful projects, industry studies, and
true failures as evinced in forensic analysis and dispute resolution, are
irreplaceable in developing and implementing proactive project controls
systems that work.
Procurement Process
Once the contract type and project delivery methods are chosen, and the
appropriate risk assignment language has been selected, it is vital that
the procurement process is managed with an eye to limiting risk. The
procurement process includes a number of steps, from shop drawing
preparation to the supply chain resiliency. Many disputes start with a
breakdown in procurement.
A quality check on the procurement process is to evaluate the
number of questions or requests for information that result from bidding
contractors during development of their cost estimates. Large numbers
of questions during the bid process may indicate that the documents do
not convey the appropriate scope definition and, if the project contingency reasonable level of detail and full scope, and preferably repeated in a
is too low, the likely result will be an increase in change requests that
increase project costs. After pre-qualification of bidders from suppliers
to contractors, careful evaluation of the bids, including trade and general
conditions comparisons, is vital to ensure appropriate awards. Lessons
learned from claims show that a frequent problem with projects that had
cost and time overruns was an inappropriate award to the apparent “low”
bidder. This can be due to insufficient planned general conditions/field
management, unbalanced subcontract trade bids, inappropriate project
duration estimate, missing scope, or inadequate or lack of contingency.
These lessons support the contention that best value is likely to be a better
approach than low bid, but the evaluation and process can mitigate some
of the risks in the low bid procurement effort.
Procurement drives the start of construction trade work, and properly
sequenced procurement may reduce project duration and improve quality
of the work. This requires a careful look at sequencing, both priority
and order, and is enabled by good schedule sequencing. In more mature
from good schedule sequencing to a full process that improves returns on
the project.
Constructability reviews, value planning and engineering, and project
controls review of evolving scope, along with better designer quality
control of documents, are valuable mechanisms to reduce risk to the
owner. Without this rigorous effort, owner risk is enhanced since these
same defects in scope definition will generally raise the bids from the
contractors attempting to limit their risk.
Integrated Cost, Schedule, and
Risk Management
Risk treatment and control attempts to limit or avoid cost and time losses;
and while these are discussed separately, they should be managed in
an integrated approach within the risk management domain. Early risk
assessment identifies project (project specific) or organizational/process
(systemic) risk issues that can then be monitored and controlled. This
can start with identifying cost, schedule, and risk drivers during value
planning and monitoring those drivers throughout the stages of cost and
schedule development in conjunction with scope definition development.
Risk-based approaches to determine appropriate contingency and
management reserve are probabilistic and deterministic and support more
valuable risk control for an owner. The author and company personnel
have been actively involved in developing industry best practices after
testing them on projects. One of those industry associations is AACE, with
many excellent recommended practices for determination of cost and time
contingency, from range estimating to expected value approaches, as well
as those for integrated cost and schedule risk analysis.21
As soon as a preliminary schedule is developed that shows a
fully detailed schedule, an integrated cost and schedule risk management
effort can be facilitated. Risk maturity is a path along which risk engineers
march by introducing an increasingly more complex qualitative and
quantitative risk efforts as the organization or project staff gains in
knowledge. From simple qualitative risk assessment of risk drivers to
comprehensive quantitative risk assessment, looking at risk drivers
19
“Mitigation of Risk in Construction: Strategies for Reducing Risk and Maximizing
Profitability”, SmartMarket Report, McGraw Hill Construction, Bedford, MA, 2011
20
"Tenth Annual Arcadis Global Construction Disputes Report”, Cooper, Arcadis
Contract Solutions Team, Middletown CT, 2020
21
AACE has over 100 formal peer-reviewed published Recommended Practices that
are aligned with the Total Cost Management Framework and provide technical
guidance for the industry.
JA NUA RY/ FEBRUA RY 2 02 3 17
as well as uncertain durations
yields improvements in schedule
predictions of completion. Other
risk practices such as what-if
scenarios for conditional branching
risks (acceptance of one risk can
cause new conditions that branch
out into new risk directions) bring
value to the process of managing
owner’s risk.
While it is possible to provide
schedule risk management as a
stand-alone effort, it is not useful
enough to attempt to provide
cost risk management without
considering the schedule because the
schedule is a significant risk driver
for cost. The integrated cost-schedule
approach to risk assessment provides
the most valuable results.

Use of a Stage-
Gate Process
for Design
Planning &
Monitoring
Three of the top causes of overall
uncertainty in construction FIGURE 9 Estimate Input Checklist Noting Maturity Level of Project Definition Deliverables - RP56R-08 [14]
projects, which often result in cost
overruns and schedule delays,
are related to the design process:
owner program or design changes, design errors, and design omissions.
According to the McGraw Hill study identified earlier, the fourth cause on
that list is unforeseen conditions, which usually occur during the design
and planning stage. Design planning and monitoring become difficult to
implement as well; designers necessarily are artists as well as designers
so it can be very difficult to benchmark their scope of work, and harder
to develop a schedule against which performance can be measured. The
actual maturity of a deliverable labelled as 30% complete is hard to
assess and so might only be 20% complete, because the components that
aggregate to achieve the 30% complete maturity will include a wide range
of discipline completions, possibly 80% structural down to something like important evaluations that an estimator can make to ensure that the range
5% commissioning design complete.
AACE has provided significant good practice development in this
area, primarily through the lens of identifying requirements necessary to
produce quality estimates.22
Given the difficulty in determining the amount of work necessary to
meet any partial design deliverable, and the complexity of the disciplines,
it is vital that a detailed schedule is developed to plan and manage the
design process. With a schedule in hand, validating the design process
on a discipline-by-discipline basis is best performed by identifying the
total hours and cost for each discipline at each stage of the deliverable
development. With that information, the design schedule can be loaded
with costs and hours which allows an earned value management system
to be implemented for monitoring. This will ensure that the appropriate
resources are planned and available when needed.
AACE Recommended Practice No. 56R-08, “Cost Estimate Classification
System – As Applied for the Building and General Construction Industries”23
provides good insight into the level of maturity of various scope definition
documents necessary to perform the cost estimate at each class, as
shown by the estimate input checklist in Figure 9. This is one of the most
of accuracy of the estimate is commensurate with the level of completion of
the scope documents.
A reporting tool should be linked to the schedule to provide real-time
(as timely as updates are performed) management data, e.g.; planned and
consumed hours and costs of design disciplines, estimated completion
percentages, as shown in Figure 10.
22
RP105R-19 [40]
23
RP56R-08 [14]

18 JAN UA RY/FE B R UA RY 20 23
FIGURE 10 Design Resources and
Deliverables Reporting

FIGURE 11 Seven Phases in Stage-Gate Process

Once there is a design schedule in use, a formal

stage-gate process is the best approach to ensure that
design performance is aligned with the deliverables
at required levels of scope definition. The author uses
seven phases of the project planning and construction
lifecycle as shown in Figure 11; one phase is prior to
design, four phases are during design, and two are
during construction/post-construction.
During each phase or stage, once the required
deliverable is achieved, design should be halted at that
gate until the stakeholders determine that the design is
proceeding in the correct direction and at the correct
level of scope definition. This is why the process is called
a stage-gate or phase-gate process and it is important
to monitoring the design to proactively reduce risks of
scope creep and gold plating. There is a natural tendency
for the owner and designer, while working together as
design matures, to attempt to improve upon the design
and, without monitoring and controls, this will rapidly
get out of hand. The risk is even greater in projects where FIGURE 12 Mapping Owner Percent Complete Deliverables to Stage-Gate Process
there is an individual or a board involved in decisions
about design because a world-famous researcher or doctor, or a museum required level of design completion, although each project may be different,
board of visitors, will see only the quality of the design and have desires that and the names or descriptions may be different. In the beginning of every
may not have been accommodated in the original budget and schedule. project, using the estimate input checklist shown in Figure 9, the standard
The next section discusses how the planning, monitoring, and control phases should be mapped to the owner’s required per cent complete
to maintain budget and schedule is provided. Each phase involves some deliverables, similar to Figure 12.

JA NUA RY/ FEBRUA RY 2 02 3 19

Integrating
Project
Controls
with the
Design Effort
to Ensure
Minimal
Benchmark
FIGURE 13 Integrated Project Controls Stage-Gate Process
Deviations
Once the stage-gate process is established with a good design schedule,
loaded with hours and costs, the integrated cost/schedule/risk/claims
avoidance project controls effort is used to engage with each design
deliverable. Satisfaction from the stakeholders comes from minimizing
surprises more than it does meeting some arbitrary budget and schedule;
therefore, the process is designed to keep the stakeholders informed so
they can make necessary decisions.
At the predetermined gates between each phase or stage, the full
set of integrated project controls services is applied to measure design
performance and alignment with the original budget and schedule. These
predetermined gates may be the same as all seven stages as recommended
earlier, or several of the seven may be combined into fewer stages. In any
case, each stage upon completion of the deliverable, halts work, and the
gate does not open to allow design to proceed until the scope of work,
budget, and schedule are validated, and the risk and claims avoidance
issues are confirmed. That provides an iterative process that looks
something like the graphic in figure 13.
Within each gate, there is a list of four primary project controls
disciplines that are integrated to assume coordination; cost, schedule, risk,
and claims avoidance. The review and verification work provided in these
four disciplines combines with the stage-gate process to ensure a project
is “designed-to-budget”, so there are no huge cost or time overruns. This
does not mean that the preliminary budget is necessarily the same as the
final budget, but it means the owner makes the decision, prior to design
evolution, to change scope.
As an example, during the schematic design stage, when the scope
definition is in the range of 1% to about 15% of the final maturity, the
service checks provided should be those shown in the following example
list, discipline by discipline (all maturity and accuracy ranges are
examples only):
1. Estimating and Cost Management
1.1 Review/confirm scope of work definition in drawings,
1.2 Anticipated maturity ~1% to 15% of final scope development
1.3 Develop concept study/feasibility estimate – Class 4 based on
minimal scope definition
1.4 Factored estimate – range of accuracy ~ +60%/-36%
1.5 Provide initial lifecycle cost estimate
1.6 Authorize preliminary budget, support preliminary funding
1.7 Refine and monitor cost drivers
20 JAN UA RY/FE B R UA RY 20 23
1.8 Verify general constructability
1.9 Continue preliminary value planning
1.10 First stage-gate estimate, ensure design is on budget
1.11 Systemic risk assessment for project cost contingencies
(unknowns)
1.12 Determine need for cost management reserves (scope changes)
2. Planning and Scheduling
2.1 Develop preliminary work breakdown structure (WBS) aligned
with cost estimate
2.2 Develop Class 4 milestone schedule – feasibility study
2.3 Identify major equipment, long lead items
2.4 Develop comprehensive Class 4 master schedule, ensure project
is on schedule
2.5 Develop design schedule
2.6 Provide preliminary schedule constructability review
2.7 Evaluate need for quantitative assessment of time contingency
2.8 Evaluate need for quantitative assessment of time management
reserves
3. Risk Management
3.1 Develop contract requirements for owner risk assignment
3.2 Plan for risk management
3.3 Update systemic risk assessment
3.4 Finalize go/no-go decision for sanction of project
4. Claims Avoidance
4.1 Review proposed project controls practices
4.2 Develop time management and changes language for contracts
4.3 Develop claims avoidance, analysis, and dispute resolution
language for contracts
4.4 Discuss need for incentives/disincentives
Each stage has similar tasks that must be performed in these
disciplines to ensure alignment with the business goals, measure
performance, determine variances, and provide feedback and action
to bring the design back into compliance with the benchmarks. When
variances are identified on a detailed basis, analysis is performed to
determine the cost and schedule drivers and the results are discussed
with the designer and owner. If the stakeholders determine that the design
enhancements causing variances are necessary, the benchmark or control
budget and schedule are adjusted to match the needs.
It is important to note that the project delivery method choice does not
change the need for these tasks, it just changes which stakeholder in the
process will perform them. For
example, in Figure 14, for design
build delivery, the design builder will
take over design development usually
during the detailed design phase.
This process has proven to
enable the budget to be maintained
throughout the design process and
results in satisfied stakeholders.

Use of Risk
Workshops to
Identify and
FIGURE 14 Ownership of Design Documents Based on Project Delivery Method
Manage Risk
be spent on corrective actions when the field operations deviate from
Risk workshops range from simple one-day efforts to multi-day, multi- the plan. When there is no detailed planning, additional time must be
meeting workshops, backed by questionnaires and interviews, and all spent in crisis management on-the-spot planning, and the result is less
efforts add value to the process, improving the control of risk. A qualitative efficient construction, raising the risks of delay and disruption. This takes
integrated cost-schedule risk workshop designed to identify and manage risk a commitment from the entire CM team to put the time into this early
drivers will capture the combined experience and lessons learned of all the detailed planning, and there are no excuses for failure to plan.
participants in the workshop. Facilitated properly, this workshop will allow The best way to manage owner risk is to develop the risk management plan
the participants to identify all risks, prioritize the risks based on probability early in the pre-project phase looking at systemic risks and major risk drivers,
and consequence, and write response plans that have the effect of removing update it during design phases while developing and monitoring project
the highest priority risks from the schedule and project. The author has risks, allow it to evolve into the full integrated cost-schedule risk management
written a paper specifically on how to plan and run a risk workshop.24 plan, and use the output or deliverables from each stage to manage the next
These risk removal efforts include time-based practical steps developed stage. Accurate cost estimates with appropriate contingencies, developed at
by the CM team based on their experience. The deliverables from the the appropriate level of accuracy, integrated with the evolving schedules, starts
workshop also link the risks to specific activities in the schedule and start the project with the right benchmarks to monitor. With preliminary schedules
the risk monitoring effort which keeps risks and risk monitoring at the established, a strong risk workshop enables the CM team to identify the likely
forefront of project discussions. Awareness of potential risks and review risks, eliminate the highest priority risks by the risk response plan, and then
of them at the time of inception will allow proactive actions to minimize or monitor the ongoing risks to avoid or mitigate those risks during the project.
mitigate the risk impacts. This effort has been shown to reduce cost and This approach takes advantage of the combined experience of the CM
time overruns by minimizing the effects of the risks on the project. team and embraces risk as an integral part of the CM process such that it
Risk workshops should be performed starting in the pre-design phase to informs the team and helps shape the approach to managing owner risk.
assess systemic risks and enable management of those risks, and then continue The project controls discipline tasks, described previously in this paper,
with project-specific risk workshops as the start of the risk management are represented in a summary as shown in Figure 15. (on next page)
effort in the design phases, as well as when the contractor is procured. This
continual risk effort will help create a proactive environment and improve
determination of risk-oriented systemic and project-specific analysis.
In addition to the value from the risk management effort, these Conclusion
workshops help to establish a partnering or collaborative approach to
construction management, which has also proven to drastically improve An integrated cost engineering or project controls approach to planning
performance and reduce claims. and implementing the TCM Framework is vital to success. Using these
suggested approaches to various project risk areas of failure will be helpful
to achieving the owner’s goal of “design-to-budget”.
Control of owner risks is not a universal one-step panacea, but
Implementation of the Plan rather an integrated program of cost and schedule risk planning and
management that starts pre-project and does not end until all outstanding
The age-old comment, “plan the work, and work the plan”, has been an issues are resolved with the project completion. For the most effective
important concept for decades of successful projects, and failure to follow control of Owner risk, the cost engineering project controls support
this simple adage most commonly results in delays and disruptions. process cannot be a one-time effort or a casual approach, and especially
Detailed planning of the project provides direction so more time can an approach embedded solely in one project lifecycle phase but must
be an integrated cost/schedule/risk culture embedded in the entire cost
24
“Planning and Facilitating a Schedule Risk Workshop”, Carson, 2013 AACE engineering and construction management process.
International Transactions.
JA NUA RY/ FEBRUA RY 2 02 3 21
FIGURE 15 Cost Engineering or Project Controls Discipline Stage-Gate Services Per Phase
References
1. AACE International, Recommended Practice 96R-18, Cost
Estimate Classification System – As Applied in Power Transmission
Line Infrastructure Projects, Morgantown, WV: AACE International,
Latest revision.
2. AACE International, Recommended Practice 90R-17, Statusing the
CPM Schedule - As Applied in Construction, Morgantown, WV: AACE
International, Latest revision.
3. AACE International, Recommended Practice 84R-13, Planning
and Accounting for Adverse Weather, Morgantown, WV: AACE
International, Latest revision.
4. AACE International, Recommended Practice 82R-13, Earned
Value Management (EVM) Overview and Recommended Practices
Consistent with EIA-748-C, Morgantown, WV: AACE International,
Latest revision.
5. AACE International, Recommended Practice 80R-13, Estimate
at Completion (EAC), Morgantown, WV: AACE International,
Latest revision.
6. AACE International, Recommended Practice 68R-11, Escalation
Estimating Using Indices and Monte Carlo Simulation, Morgantown,
WV: AACE International, Latest revision.
7. AACE International, Recommended Practice 65R-11, Integrated Cost
and Schedule Risk Analysis and Contingency Determination Using
Expected Value, Morgantown: AACE International, Latest revision.
8. AACE International, Recommended Practice 64R-11, CPM Schedule
Risk Modeling and Analysis: Special Considerations, Morgantown,
WV: AACE International, Latest revision.
9. AACE International, Recommended Practice 63R-11, Risk Treatment,
Morgantown, WV: AACE International, Latest revision.
10. AACE International, Recommended Practice 62R-11, Risk
Assessment: Identification and Qualitative Analysis, Morgantown, WV:
AACE International, Latest revision.
11. AACE International, Recommended Practice 60R-10, Developing
the Project Controls Plan, Morgantown, WV: AACE International,
Latest revision.
12. AACE International, Recommended Practice 58R-10, Escalation
Estimating Principles and Methods Using Indices, Morgantown, WV:
AACE International, Latest revision.
22 JAN UA RY/FE B R UA RY 20 23
13. AACE International, Recommended Practice 57R-09, Integrated Cost
and Schedule Risk Analysis Using Monte Carlo Simulation of a CPM
Model, Morgantown, WV: AACE International, Latest revision.
14. AACE International, Recommended Practice 56R-08, Cost Estimate
Classification System – As Applied for the Building and General Construction
Industries, Morgantown, WV: AACE International, Latest revision.
15. AACE International, “Recommended Practice 53R-06, Schedule
Update Review - As Applied in Engineering, Procurement, and
Construction,” AACE International, Morgantown, Latest revision.
16. AACE International, Recommended Practice 47R-11, Cost Estimate
Classification System - As Applied in the Mining and Mineral Processing
Industries, Morgantown, WV: AACE International, Latest revision.
17. AACE International, Recommended Practice 46R-11, Required Skills
and Knowledge of Project Cost Estimating, Morgantown, WV: AACE
International, Latest revision.
18. AACE International, Recommended Practice 44R-08, Risk Analysis
and Contingency Determination Using Expected Value, Morgantown,
WV: AACE International, Latest revision.
19. AACE International, Recommended Practice 42R-08, Risk Analysis
and Contingency Determination Using Parametric Estimating,
Morgantown, WV: AACE International, Latest revision.
20. AACE International, Recommended Practice 41R-08, Risk Analysis
and Contingency Determination Using Range Estimating, Morgantown,
WV: AACE International, Latest revision.
21. AACE International, Recommended Practice 40R-08, Contingency
Estimating – General Principles, Morgantown, WV: AACE
International, Latest revision.
22. AACE International, Recommended Practice 38R-06, Documenting the
Schedule Basis, Morgantown, WV: AACE International, Latest revision.
23. AACE International, Recommended Practice 36R-08, Development
of Cost Estimate Plans – As Applied in Engineering, Procurement, and
Construction for the Process Industries, Morgantown, WV: AACE
International, Latest revision.
24. AACE International, Recommended Practice 34R-05, Basis of
Estimate, Morgantown, WV: AACE International, Latest revision.
25. AACE International, Recommended Practice 33R-15, Developing
the Project Work Breakdown Structure, Morgantown, WV: AACE
International, Latest revision.
 26. AACE International, Recommended Practice 32R-04,
Determining Activity Durations, Morgantown, WV: AACE
International, Latest revision.
27. AACE International, Recommended Practice 31R-03, Reviewing,
Validating, and Documenting the Estimate, Morgantown, WV: AACE
International, Latest revision.
28. AACE International, Recommended Practice 29R-03, Forensic Schedule
Analysis, Morgantown, WV: AACE International, Latest revision.
29. AACE International, Recommended Practice 21R-98, Project Code of
Accounts - As Applied in Engineering, Procurement, and Construction,
Morgantown, WV: AACE International, Latest revision.
30. AACE International, Recommended Practice 20R-98, Project Code of
Accounts, Morgantown, WV: AACE International, Latest revision.
31. AACE International, Recommended Practice 18R-97, Cost Estimate
Classification System – As Applied in Engineering, Procurement, and
Construction for the Process Industries, Morgantown, WV: AACE
International, Latest revision.
32. AACE International, Recommended Practice 17R-97, Cost Estimate
Classification System, Morgantown, WV: AACE International, Latest
revision.
33. AACE International, Recommended Practice 10S-90, Cost
Engineering Terminology, Morgantown, WV: AACE International,
Latest revision.
34. Stephenson, H.L., Ed., Total Cost Management Framework:
An Integrated Approach to Portfolio, Program and Project
Management, 2nd ed., Morgantown, WV: AACE International,
Latest revision.
35. AACE International, Recommended Practice 27R-03, Schedule
Classification Matrix, Morgantown, WV: AACE International,
Latest revision.
Appendix A
SAMPLE OF MATRIX FOR CHOOSING A PROJECT DELIVERY METHOD
36. AACE International, Recommended Practice 10R-19, Estimate
Requirements Document – As Applied in Engineering, Procurement,
and Construction for the Process Industries, Morgantown, WV: AACE
International, Latest revision.
37. AACE International, Recommended Practice 30R-03, Implementing
Project Constructability, Morgantown, WV: AACE International,
Latest revision.
38. AACE International, Recommended Practice 100R-19, Contract
Change Management – As Applied in Engineering, Procurement, and
Construction, Morgantown, WV: AACE International, Latest revision.
39. AACE International, Recommended Practice 71R-12, Required Skills
and Knowledge of Decision and Risk Management, Morgantown, WV:
AACE International, Latest revision
40. AACE International, Recommended Practice 105R-19, Estimate
Requirements Document – As Applied in Engineering, Procurement,
and Construction for the Process Industries, Morgantown, WV: AACE
International, Latest revision
ABOUT THE AUTHOR
Christopher W. Carson, CEP DRMP PSP FAACE,
is with Arcadis and can be contacted by sending
email to: chris.carson@arcadis.com
JA NUA RY/ FEBRUA RY 2 02 3 23
 Risk Analysis and
Contingency Estimating
for Class 10 Estimates
BY JOHN K. HOLLMANN, PE CCP CEP DRMP FAACE HON. LIFE

Editor’s Note: This article was peer-reviewed and presented as RISK-3908 at the 2022 AACE Conference
and Expo. It scored high by the Technical Board’s Review Committee using the Technical Paper Evaluation
Criteria (TPEC) spreadsheet. It also scored high on attendee review evaluations. Successful completion of
these pre-steps resulted in this article being selected for publication in the Cost Engineering journal.

ABSTRACT
The AACE® International (AACE) cost estimate classification system recommended
practice (RP) series is likely its most recognized RPs. In 2021, a new Unclassified/Class 10
estimate type was introduced in AACE RP 111R-20, Estimating for Long-Range Planning – As
Applied for the Public Sector. However, there is no AACE RP for quantitative risk analysis
(QRA) methods for estimating contingency or management reserve allowances for
Unclassified/Class 10 estimates.

A goal of this article is to lay the groundwork for a potential QRA RP for Unclassified/
Class 10 QRA recommended practice. It starts by reviewing the concepts of estimate
classification in general and Unclassified/Class 10 estimates in particular. Next, it outlines
various uses of these estimates such as for asset life cycle cost (LCC) estimating and
analysis as part of strategic portfolio management or for surety (bonding) valuation. This
article also reviews scenario analysis and other decision analysis methods to identify
potentially useful QRA concepts. It also reviews current contingency determination
practices for long-range estimates and the limited research on long-range estimate cost
growth. Finally, several proposed Unclassified/Class 10 QRA and contingency/reserve
allowance determination methods, aligned with AACE QRA principles, are presented.

24 JAN UA RY/FE B R UA RY 20 23

Introduction
The AACE® International (AACE) cost estimate classification system series
of recommended practices (RP) is outlined in the AACE Professional
Guidance Document PGD-01, Guide to Cost Estimate Classification Systems
[1]. These are likely the most widely used AACE RPs. Since the first such
RP was published 25 years ago (RP 17R-97, Cost Estimate Classification
System [2]), the classification system has included five classes that
represent levels of project scope definition numbered from 1 to 5 with
Class 5 being the least well defined.
In 2021, a new Unclassified/Class 10 estimate was added as
documented in RP 111R-20, Estimating for Long-Range Planning – As
Applied for the Public Sector [3]. The need for a pre-Class 5 designation
for long-range planning uses (i.e., estimate prepared 10 or more years
before project execution) was first described by Taylor, et.al. in 2018 [4].
This paper summarizes the purpose and definition of this new estimate
classification and proposes fit-for-use quantitative risk analysis (QRA)
and contingency/reserve allowance estimating practices that align with
Unclassified/Class 10 estimating and AACE QRA principles.
The 2018 Taylor paper [4] was by a water treatment utility; an industry
where long-range capital portfolio planning for a growing and evolving
utility system has always been required. In recent years, low carbon
initiatives have increased the stakes. For example, most energy firms have
published 10 and 20 years plans for achieving lower carbon emissions. A
specific example is Consumers Energy’s Clean Energy Plan which includes
retiring coal units and replacing power supply with wind and solar
renewables (renewables are planned to increase from the current 11%
to 42% in 2030 and 56% in 2040 [5]). From conception through start-
up, it is not unusual for a major project to take 10 years to complete, so
most projects in 10-year plans have already entered a company’s phase-
gate estimating and funding approval process. It is for the asset life cycle
and portfolio or system management plans that extend beyond 10 years
(outside of traditional, shorter-term, phase-gate processes) where the
Unclassified/Class 10 classification comes into play.
Background/Basis
This section discusses a number of definitions, concepts and practices
related to Unclassified/Class 10 estimates that will help establish the basis
of proposed QRA methods. It starts with explaining the general concept
of estimate classification, moving on to the concepts of asset life cycle
and life cycle cost (LCC) in long-range planning, and then to the use of
Unclassified/Class 10 estimates in various long-range analysis methods.
ESTIMATE CLASSIFICATION, PHASE-GATE AND QUANTITATIVE
RISK ANALYSIS (QRA)
The estimate classification system was developed to tie cost estimating
practice to the phase-gate project scope development processes that have
become ubiquitous in most industries. Phase-gate is a project system
governance and risk management process wherein project funds are
approved at decision “gates” or milestones in a stepped fashion. Each
definition phase adds more scope definition detail, thus reducing the
definition-driven risk. And each gate approves additional funds (or
recycles project for further definition) until full funding can be prudently
sanctioned, typically at AACE Class 3 or 2 [6]. The risk is usually
communicated as a cost estimate accuracy range which is determined
through QRA (refer to RP 104R-19, Communicating Expected Estimate
Accuracy for more information on accuracy ranges [7]).
Cost estimate classification practices are related to QRA methods in
that fit-for-use QRA methods vary with the level of project scope definition.
For example, at Class 5, having little scope definition, QRA is often done
using parametric risk analysis [8], while at Class 3, with more detailed
scope definition, a critical path schedule method with Monte Carlo
simulation (MCS) is often used for major projects [9]. The AACE PGD-
02, Guide to Quantitative Risk Analysis describes these and other QRA
methods and their relationship to estimate classification [10]. What has
not been addressed in AACE RPs or PGDs to date are QRA methods for
Unclassified/Class 10 estimates: the topic of this article.
BASE ESTIMATES, UNCLASSIFIED/CLASS 10/CLASS 5, AND
CONTINGENCY/RESERVE ALLOWANCE
The focus of this article is on the QRA and contingency/reserve allowance,
not the “base” cost estimate (defined in RP 10S-90 Cost Engineering
Terminology, as the “estimate excluding escalation, foreign currency exchange,
contingency, and management reserves” [11]). Class 10 base estimates are
prepared in the same way as Class 5 base estimates; the difference between
Class 10 and Class 5 is mainly in the use of the estimates (e.g., Class 10 for
life cycle costs estimates (LCCE) or determining surety requirements) and
the greater risk of Class 10 due to potential scope change and increasing
uncertainty over time.
Per RP 111R-20, the use of the Unclassified or the Class 10 designation
is determined by the rigor and requirements of the entity’s asset
management and LCCE processes and practices as follows:
• Unclassified: performed as part of a process where scope change is
not expected to be addressed in planned estimate updates over time
(i.e., one-time, or more ad-hoc).
• Class 10: same as unclassified, but where a classification designation
is required to meet organizational procedures (but also implies more
documented requirements).
RP 111R-20 retains the use of the “Class 5” designation when a long-
term estimate is part of a controlled, documented asset management
process involving planned estimate updates to address scope and cost
changes over time (e.g., often required for nuclear decommissioning, mine
closure, and other regulated asset type projects). The difference between
long-term and phase-gate Class 5 estimates is that the long-term estimate
is done repeatedly over time (e.g., to assess current surety requirements),
whereas the Class 5 estimate, as part of a phase-gate process, is expected
to progress to Class 3 or 2 and an investment decision in a relatively short
time (i.e., <10 years).
For Unclassified/Class 10 estimates, RP 111R-20 states that they “are
not associated with indicated expected accuracy ranges”. In respect to QRA,
the RP further acknowledges that the “estimate is unlikely to be accurate,
and the original scope may not be representative of the final solution and
associated costs. Unpredictable scope and risk challenges over the extended
long-range planning timeframe include economics, technology, availability
of resources, critical infrastructure, population dynamics, regulations,
organizational and asset resiliency, climate, energy, and natural influences.”
In summary, the attribute of “unpredictable scope and risk challenges”
is the driver of the need for specialized QRA and contingency/reserve
allowance determination methods for Unclassified/Class 10 estimates.
However, a terminology problem arises because “contingency,” per RP
10S-90, specifically excludes scope change. Therefore, the expression
“contingency/reserve allowance” is used for the remainder of this article
JA NUA RY/ FEBRUA RY 2 02 3 25
 to describe the QRA end result [11]. One of the more complete treatments
of long-range estimates found (in nuclear decommissioning) took this
definition approach; “in order to reduce possible ambiguity and confusion,
instead of the term ’contingency‘, the terms ’estimating uncertainty‘ and
’funded risk‘ are used with funded risk being for ’out of scope‘ cost”. [12] In
any case, this points out the need for those performing QRA to carefully
define what risks are covered by any particular QRA analysis and any risk
funding account.
The original Taylor paper that led to RP 111R-20 highlighted the
words “cost communication” in its title, recognizing that the concepts of
uncertainty/risk, accuracy, and contingency/reserves are more difficult
to communicate for long-range planning estimating that is outside of
a structured phase-gate process [4]. Other good references regarding
communication are RP 104R-19 and the aforementioned nuclear
decommissioning reference [12].
THE ASSET LIFE CYCLE AND LIFE CYCLE COSTS (LCC)
The scope of RP 111R-20 covers “estimating as well as communicating needs
and concepts that may pre-date the creation, existence, or emergence of an
asset by many years, such as planning for future capacity/infrastructure.
These estimates are prepared as planning-level predictions for facilities that
may be constructed 10-50+ years in the future.” [3]. These estimates are often
prepared as part of an asset life cycle cost (LCC) estimating or analysis
(LCCE/A) process in which multiple projects are planned to be executed
at various phases or points of time during the asset life cycle. This long-
range process is often referred to as a capital portfolio management or
long-term capital budgeting process [13] [14]. This is part of the strategic
asset management sub-process of the AACE Total Cost Management (TCM)
Framework [15] and is also the subject of the International Standards
Organization (ISO) 55000 series standards [16].
As a specific example of a long-range capital budget, the San Francisco
Municipal Transportation Agency published its 20-year (2023 to 2042)
capital plan costing $31.3 billion in which every project and program is
itemized including a brief basis statement such as “estimate based on past
similar work” [17].
These estimates have become increasingly important as society
focuses on the need for sustainability and low carbon. An example of its
importance to cost engineering is the November 2021 3rd edition of the
International Cost Management Standard (ICMS) that now covers not only
life cycle cost as shown in Figure 1, but also carbon emissions [18].
For utilities, transportation and similar industries, long-range planning
1. Surety
also covers ongoing system management which, unlike for a given asset In surety use, traditional Class 5 estimates and QRA methods are typically
or project, has no identified end. System planning typically starts with used assuming fixed scope (albeit the defined scope attempts to address
FIGURE 1 The relationship between ICMS, Life Cycle Costs and Whole Life Costs [18]
26 JAN UA RY/FE B R UA RY 20 23
forecasting long-range demand and other requirements, and then develops
strategic plans for investments to meet those needs.
The concepts of Class 10 and of LCCE/A tend to go hand-in-hand. The
literature on LCCE/A is extensive including in AACE Transactions (such as
in Harbuck [19] and Gransberg et.al. [20]), and in government guidelines
(such as in US Department of Energy [21], and US General Accounting
Office guidelines [22]). These sources typically focus on either analyzing
the cost (LCCAs) of alternative potential features within in one investment
[23] or supporting alternative analysis using discounted LCCEs to derive
alternative net present values (NPV) [19]. The focus in both cases is on
the overall LCC, not the risk-adjusted cost of the various individual long-
term projects within the total LCC (e.g., the individual input estimates for
upgrades, expansions, replacements, closure and so on). For example, the
sources often show each estimate within the LCC being assigned rule-of-
thumb contingencies or uplifts without much regard for the estimate’s
long-term nature. The focus of this article is on QRA methods to quantify
the risk of each project within the overall asset LCC (or stand-alone long-
term projects) in a fit-for-use and principled way.
USE CASES FOR UNCLASSIFIED/CLASS 10
There are two typical high-level long-range planning use cases for
Unclassified/Class 10 or repetitively updated Class 5 estimates. These
include (but are not limited to):
1. Surety (Class 5): determine amounts for insurance reinstatement,
bonding, escrow or other financial liability or assurance instruments
to cover the future cost of required investments (e.g., typically asset
or facility closure/decommissioning regulatory requirements) with
estimates updated repeatedly over the asset life cycle.
2. Long Range Cost and Economic Studies (Unclassified/
Class 10) including:
◦ General Studies: to understand the cost and risk of potential
future investments
◦ Investment Economics and Sustainability (LCCEs/LCCAs):
to support a current investment funding, loan, rate, or other
decision, or for portfolio or systems management, which
consider current and future investments (e.g., additions,
expansions, systems growth, technology evolution,
rehabilitation, replacement, restoration, closure, etc.).
foreseeable conditions) with any scope change
(e.g., regulation changes) addressed in later
surety estimate updates as the asset life cycle
progresses. In this use there is no expectation
that a given estimate will cover the cost of
project scope change (even though such change
is expected in the long-range). These estimates
are not used as the basis to approve or commit
to an overall investment amount, but to support
a narrower surety need, and in general to assure
the current enterprise financial condition is
sound until the next review (periodically or
upon experiencing a major change).
There is a question as to whether the periodic
surety Class 5 estimate updates are subject to
greater risk than traditional Class 5 estimates.
There is some evidence that they are not. For
example, in 2016 the OECD published a study
of the cost of decommissioning nuclear power
plants that included case studies of periodic
cost estimate updates. Most countries require
operators to update their decommissioning
plans and estimates periodically (e.g., every 5 to
10 years) over the plant life (e.g., 35 to 50 years)
[16]. For example, a case study for a Finland
nuclear plant included 6 repeated estimates
from 1987 to 2012; there was a normalized cost
increase of 17% over the 25-year period (mostly
resulting from a regulation change addressed in
the 2008 estimate). A case study for a multi-unit
Swiss plant reviewed an estimate prepared in
2006 and updated in 2011; the normalized cost
for decommissioning each of its four reactor
units increased by 10 to 28% (much of the
increase was for an increase in requirements to
maintain operations during dismantling). Given
the nuclear power industry’s regulatory and
environmental sensitivity, these example cost
increases are quite sedate compared to general
industry research of cost growth for Class 5 FIGURE 2 Typical Class 10 Estimate Studies and Economics Analysis Use
estimates, especially considering the relatively
low contingency values used1 [17].
These generalized study types will be referred to later in this article in
2. General studies and investment economics more specific discussions of analysis methods.
General and economic studies are discussed because the estimates within
the LCC or the unique future project estimate must cover all the future EMPIRICAL STUDIES OF UNCLASSIFIED/CLASS 10 COST
risks including potential project scope change (i.e., Unclassified/Class 10). GROWTH: LOW EXPECTATIONS FOR ESTIMATES
For example, the Taylor paper describes in the following paragraph how a The discussion of Class 5 estimates for periodic surety use described
public wastewater treatment agency has to address scope change resulting studies that showed estimate-to-estimate cost growth was comparable to
from systems management and rate determination needs: estimate-to-actual cost growth for traditional project Class 5 estimates.
There have also been studies of cost growth for very long duration projects;
“Planning has determined that several decades in the future the for example, a 2020 study of 67 major US transport projects showed cost
volume of wastewater flows produced by the region is projected to be growth for projects of greater than 10 years duration exceeded that of
significantly greater than the utility’s existing capacity. Expanding projects of 4 to 10 years duration by about 20%2 [24]. Another study of
services to accommodate the increase in capacity will require significant US transport projects showed that soft costs3 as a percent of construction
public works projects. Despite the fact that these future capacity needs costs have increased by about 0.5% per year for the past 40 years. Such
are decades away and the agency can only speculate as to what future studies point to added risk in long-range estimates [25]. However, there
solutions might ultimately entail, the agency must develop and publish were no references found that described empirical research of estimate-
scope and cost information to address the problem and place probable to-actual cost growth for estimates used in long-range (outside of
programs and projects into its capital improvement portfolio and rate.” phase-gate processes) studies and analyses. This was a surprise given
the several decades of publications and guides on long-range closure/
Figure 2 illustrates three typical cost and investment economic study decommissioning projects (mining and nuclear in particular) that included
types (stripped of their revenue, operating cost, and other non-project development of standard cost models. [26]
input arrows) as follows: One reason for the dearth of long-range cost growth studies may be
that relatively few asset life cycles have actually been formally “closed,”
• Single project: one-off study of a future project; not LCCE.
• Multiple projects, single investment LCCE across a given asset life cycle. 1
The Finnish estimates applied 10% contingency for each periodic estimate. After
• Alternative selection with multiple project LCCEs. the 2011 Swiss estimate, the Swiss regulators imposed a requirement to apply 30%
contingency because the 2006 and 2011 contingency allowances (not stated) were
inadequate. Note that in both cases, contingency percentages were predetermined.
Figure 2 shows the example project investments (vertical arrows) over
the asset life cycle (time: blue arrow). The focus of this article is on the 2
The study did not normalize for escalation, however, the study authors believed
QRA and contingency/reserve allowance determination (the red currency that the funded amount (estimate) included escalation so no correction was
necessary.
symbol) including potential scope change and strategic risk for each
investment. This article’s focus is not on the overall analysis outcome (e.g., 3
Soft costs in that case included not only project management and engineering, but
NPV) except for the single project case which has a project cost outcome. studies and permitting.

JA NUA RY/ FEBRUA RY 2 02 3 27

particularly where there are
potential long-term hazards. One
report on mine closures showed
that only 4 out of 57 mines being
closed had actually relinquished
control. The report stated “Not
only has successful relinquishment
been unattainable for most, but the
financial cost of closure is often
many times higher than was ever
anticipated” [27]. Another report
stated that of 147 shutdown nuclear FIGURE 3 Risk Estimation Methodologies in UK Cost Estimating Guidance [32]
reactors there was “only limited
experience of fully completed decommissioning projects”. [28] The same is another example, a United Kingdom (UK) guideline recommended that
reportedly true for decommissioning offshore oil facilities [29]. 56% uplift be added to the base cost of new build rail investments at their
Another reason is that project scope change and major risks in long- earliest definition phase [31]. Further, as shown in Figure 3, an associated
range studies are often assessed as economic scenarios, not as elements of UK guideline for cost estimating, including risk analysis, suggests using
continency/reserve allowance from the estimator viewpoint. Business and deterministic (which may include uplifts) rather than probabilistic risk
regulatory stakeholders have little expectation that study input estimates estimation methods at the earliest stages of scope definition4 [32].
will be used directly for funding or other individual commitments and hence Given that the purpose of this article is to advance QRA methods
there is no perceived need to include cost for scope change and strategic in a fit-for-use way that aligns with AACE QRA principles, the use of
risk in the LCC input estimates. However, this lack of clarity in expectations non-probabilistic methods (e.g., expert judgment, pre-determined ranges
regarding cost growth can lead to communication confusion later. Key or contingencies, uplifts, etc.) is not further considered. Unfortunately,
stakeholders often remember or see the recorded input cost numbers used the predominate use of deterministic cost risk valuation for long-range
in studies and this can lead to anchoring bias on their part, i.e., stakeholders estimating has likely not resulted from assessment of information needs,
may unreasonably resist later estimates that differ from the original number. but from the fact that typical industry QRA practices are not fit-for-use (e.g.,
Anchoring bias will make communication a key issue for any proposed QRA overly complex, purely subjective, etc.) and/or are widely perceived (with
method RP. A key communication tool is the basis of estimate documentation; good reason) as unreliable and/or unrealistic. This unfortunate situation
documenting where and how scope change and strategic risks are accounted needs to be rectified.
for needs to be very clear for Unclassified/Class 10 estimates. The most common probabilistic approach found in literature search
in respect to contingency/reserve allowances for projects was estimate
ranging with Monte Carlo simulation (MCS) [12]. That method is
described in AACE RP 118R-21, Cost Risk Analysis and Contingency
Review of Published QRA Determination Using Estimate Ranging for Inherent Risk with Monte Carlo
Simulation [33]. However, that RP limits its use to projects with minimal
Methods Relevant to systemic risks. Systemic risks are uncertainties that are artifacts of the
nature of the project system and that includes uncertainties arising from
Unclassified/Class 10 Estimates minimal scope definition [11]; i.e., that RP excludes ranging use on
Unclassified/Class 10 or Class 5 estimates. That exclusion is based upon
The literature on investment economics and LCCE/A and other long- research that has shown that ranging is a “disaster” when used alone on
range estimating practices was reviewed to identify probabilistic QRA projects with significant systemic risks (re: the prior paragraph about why
methods proposed or in use that would be amenable to regular use for deterministic methods are often used for long-range estimates.) [34].
contingency/reserve allowance determination. In summary, the literature The use of pre-determined contingency or pure ranging methods
search did not find any new QRA methods not already covered by AACE may also reflect the fact that many sources focus on the end result of an
RPs. Unfortunately, references often describe outdated methods that are economics or NPV study (e.g., outcomes on the left of Figure 2). The most
not aligned with the principles in RP 40R-08, Contingency Estimating- common references are about sensitivity analysis of LCC cost or NPV that
General Principles [30]. However, some describe methods roughly aligned study the impact of changes to various cost model inputs (e.g., discount
with existing AACE QRA RPs that, with some modification, could be rates, labor rates, etc.) other than the risks addressed by project contingency
recommended for Unclassified/Class 10 estimate use. [23]. In short, contingency/reserve allowances of the LCC project cost
inputs are often treated as more or less irrelevant to the economics or
CONTINGENCY/RESERVE ALLOWANCE METHODS NOT portfolio level decision. This may also reflect the fact that NPV analyses
ALIGNED WITH QRA PRINCIPLES use compounded discounting of future costs (with understated escalation),
In respect to contingency/reserve allowances for projects, many often to the point of making investments later in the asset life cycle
references described LCCE or other approaches that applied deterministic irrelevant. In any case, this common minimal regard for contingency/
approaches (i.e., not probabilistic QRA methods) to putting a cost value reserve allowances is disappointing for estimating practice purposes and
on uncertainty/risk. For example, the Finnish and Swiss nuclear power may result in later communication problems related to anchoring bias.
plant decommissioning estimates applied predetermined 10 or 30 percent
contingencies [28]. Another nuclear reference describes nationally defined
or data-based “factors” (e.g., 20 or 30% uplift) for out-of-scope cost [12]. As 4 The HM Treasury “Strategic Outline Case” in Figure 4 aligns with AACE Class 5
estimates.
28 JAN UA RY/FE B R UA RY 20 23
CONTINGENCY/RESERVE ALLOWANCE METHODS ALIGNED
WITH QRA PRINCIPLES AND AACE RPS
The aforementioned nuclear decommissioning reference listed several
probabilistic QRA methods for consideration [12]. It suggests using the
expected value approach (probability times impact) for both scope
changes and for “strategic” risks [12]. This would be generally consistent
with AACE RP 113R-20 Integrated Cost and Schedule Risk Analysis and
Contingency Determination Using Combined Parametric and Expected
Value [35] or RP RM-34 Integrated Cost and Schedule Risk Analysis and
Contingency Determination Using Estimate Ranging with Expected Value
and Monte Carlo Simulation [36]. However, for long-range estimates, the
expected value method would be extended to considering potential scope
changes and strategic risks rather than what the RPs call critical risks
within the project scope [35].
The nuclear decommissioning source also suggests a method to “apply
a factor tied to past experience” to a base contingency, but it goes on say
this approach is not yet viable for that industry given the lack of historical
data. However, as a general approach, this is consistent with AACE RP
42R-08, Risk Analysis and Contingency Determination Using Parametric
Estimating which includes provision for calibrating a general parametric
model to align with specific experience; in this case potential scope change
and strategic risks [8].
While some sources apply the critical path schedule (CPM) with MCS
QRA method in the context of sensitivity or scenario analysis on near term
projects [37], none suggested that method’s use on long-range estimates
given that CPM-models are generally not prepared for long-range planning.
The parametric and expected value methods will be explored further in
this article for potential use on Unclassified/Class 10 estimates. However,
probabilistic methods used for LCCE/A studies also present possibilities as
discussed in the next section.
PROBABILISTIC ANALYSIS METHODS (BUT NOT FOR
CONTINGENCY/RESERVE ALLOWANCES)
Several common probabilistic LCCE/A methods were found in the
literature. They are widely used in the context of overall investment
economics and decision making considering the asset life cycle. These
methods have long-range project estimates as inputs but these various
input estimates (e.g., expansions, refurbishments, etc.) are not usually the
focus of the methods. Most of the sources addressed the multi-input, multi-
alternative case in Figure 2 with a focus on the ultimate NPV. Many public
development and finance agencies provide guidelines that include these
methods [38]. The most common methods include5:
• sensitivity analysis with MCS.
• scenario analysis with MCS;
• decision tree analysis with MCS.
These general LCCE/A methods are summarized in the following
sections, including select references and ideas as to how the method
concepts could be leveraged for a QRA method to determine Unclassified/
Class 10 project estimate contingency/reserve allowance. In general,
this article seeks to pull the attention back from being exclusively on
the overall NPV output, to being more on getting scope and risk issues
addressed in the estimate inputs.
Sensitivity Analysis with MCS
The definitions of the terms sensitivity and sensitivity analysis in RP 10S-
90 [11] are:
• Sensitivity: the degree to which a change in an element of a model
affects the outcome.
• Sensitivity Analysis: a test of the outcome of an analysis by altering
one or more parameters from an initially assumed value(s).
Altering an input to a model to see the impact to the output is a simple
concept. It is most often applied to a current, often detailed estimate.
For cost, an example is to vary labor rates to see how they impact the
project cost overall. For an NPV study, an example is to vary the discount
rate. Figure 4 illustrates the concept that starts with a base cost estimate
for a future project. Then the inputs to the estimate and/or to the NPV
model (e.g., discount rate) are varied, and the respective NPV outputs are
compared. This can be done probabilistically using MCS by entering the
inputs as probability distributions rather than discretely, thus obtaining an
NPV output distribution rather than a discrete outcome.
FIGURE 4 Conceptual Illustration of a Sensitivity Analysis
As discussed, papers on this method are usually focused on relative
NPV outcomes. Relatively little attention is given to the cost uncertainty
of projects in the model. For example, in a reference on using sensitivity
analysis for the levelized cost of electricity (LCOE) for a long-range
power plant investment, the plant input construction cost variation was
represented by a relatively arbitrary, narrow +/-15% range [39].
However, it does not take much imagination to picture using this
LCCA method for the LCOE output and applying RP RM-34 (combined
estimate ranging and expected value [EV]) as a QRA model for the
construction costs range including scope changes and strategic risks as
EV inputs, i.e., putting some deserved focus on the cost input. Running
the MCS would then support both contingency/reserve allowance for the
project (i.e., a better number to anchor should anyone be paying attention
to it) and the LCOE output distribution. Similarly, RP 113R-20 (combined
parametric and expected value) could be used which provides the added
ability to do sensitivity analysis using the estimate parametric model input
variables as well as EV for the scope changes and strategic risks.
Scenario Analysis with MCS
The definitions of the terms scenario and scenario analysis in RP 10S-90 [11] are:
• Scenario: a description of specific events and conditions and their
probable outcomes. Usually limited to likely or probable scenarios
versus all possible ones. Frequently, most likely, best case, and work
case scenarios are used to define the most probable outcome and the
range of outcomes.
• Scenario Analysis: methods to assess a range of events, conditions
and outcomes employing specific scenarios. An alternative to
simulation methods for assessing ranges.
5
Note that an RP 40R-08 QRA principle is that methods should be fit-for-use, so
complex or esoteric methods that were from the research or academic domains are
not included.
JA NUA RY/ FEBRUA RY 2 02 3 29
 Applying a model multiple times, with each run using inputs that
together reflect artifacts of the selected scenario, to see the range of impacts The definition of the term decision tree in RP 10S-90 [11] is:
on the output is a simple concept. This is more of a big picture or strategic
approach than sensitivity which makes scenario analysis more suited to
long-range planning, i.e., the scenarios can reflect a wide range of future
states. Often, expert input is obtained to help define the scenarios.
A common use today is to consider different climate change scenarios
which may affect many inputs to an NPV or economic cost model including
variations in facility design and construction cost. Figure 5 illustrates the
concept that starts with multiple cost estimates for the respective scenarios,
defining appropriate inputs for the scenario, and comparing the respective
outputs. This can be done probabilistically at two levels. One is to use MCS basic method [40]. An AACE Transactions paper describes a decision tree
by entering the inputs to each scenario model as probability distributions
rather than discretely, thus obtaining an output distribution for each
scenario rather than discrete outcomes. At the second level, a simple
decision tree can be set up wherein each scenario is assigned a probability
for its occurrence (usually with stakeholder and expert input) and after
running MCS, the output will be a single distribution.
FIGURE 5 Conceptual Illustration of a Scenario Analysis
As discussed, papers on this method are focused on the NPV outcome
and to devising scenarios and how the numerous inputs and their
correlations would vary for these scenarios. Relatively less attention is
given to the input project costs to the model.
Again, it does not take much imagination to picture using this method
for the LCOE output and RP RM-34 (combined estimate ranging and EV)
or RP 113R-20 (combined parametric and expected value) as the QRA
model for the construction costs in long-range planning including scope
changes and strategic risks as EV inputs.
One useful reference was found that used scenario analysis with
MCS for QRA of a project ready for funding albeit as a check on the base
case, not for funding the project contingency or reserve allowance per se
[37]. It addresses the situation where a complex project faces a dynamic,
uncertain external environment for which traditional risk registers and
QRA methods tend to fall short. In essence, it is the same long-range
planning situation faced with potential scope change and strategic risks
but compressed into a short-term project duration. The paper describes
identifying scenarios for how things may change during project execution.
It then applies the risk-driven CPM with MCS method (i.e., RP 57R-09
[9] given that the project planning is detailed at the sanction gate) for
each scenario resulting in multiple outputs to compare to the base case
that management is considering for funding. The paper suggests using
the scenario analysis method as a risk communication tool rather than
for funding. However, it is a small step to applying the probability of
each scenario occurring and pulling these separate analyses into one
probabilistic output. That approach will be discussed later as a possible
Unclassified/Class 10 approach (albeit using a QRA method suitable for
lesser defined projects without CPM schedules).
30 JAN UA RY/FE B R UA RY 20 23
Decision Tree with MCS
• Decision tree: A graphical representation of the decision process.
Sequential decisions are drawn in the form of branches of a tree,
stemming from an initial decision point and extending all the way to
final outcomes. Each path through branches of the tree represents a
separate series of decisions and probabilistic events.
This is one of the most common decision analysis methods. AACE
offers RP 85R-14, Use of Decision Trees in Decision Making covering the
for dispute resolution evaluating the expected value of different claim
strategies, but no Transactions papers were found for use in QRA [41]. At
its simplest, a decision tree may have only the initial decision node and
two chance nodes for which there are various discrete outcomes with
various probabilities of occurring. The valuation of the chance node is
simply it’s expected value (for cost, the expected monetary value of EMV);
i.e., the sum of the products of the discrete probabilities x outcomes. For
a cost-based decision, the decision maker would choose the chance node
with the lowest EMV as shown in Figure 6. MCS can readily be applied by
replacing fixed outcome values with distributions.
Scope Variation or Scenario
Low 10%
$1,000
Option A Most Likely 65%
$2,000
$2,400
High 25%
$4,000
Alternative?
$2,400 Low 10%
$1,000
Option B Most Likely 75%
$2,500
$2,500
High 15%
$3,500
FIGURE 6 Simple Example of a Decision Tree
There is an opportunity to use this concept for Unclassified/Class
10 estimate contingency/reserve allowance determination. For example,
a scenario analysis example for a complex, high external risk project
was discussed previously [37]. If that project were Option A in Figure 6,
and the scenarios were the Option A branches with relative likelihoods
assigned to each, then the EV of Option A is a QRA outcome for the risks
represented in the scenarios. If MCS were applied, the cost distribution
of Option A could be used to determine a contingency/reserve allowance.
The branches could also represent the cost impact of variations of design
scope, impacts of strategic risks, and/or various scenarios.
A challenge with decision trees is to limit the discrete outcomes to
consider. A practical approach is to quantify low/most likely/high scenario
branches and their respective probabilities as was done in Figure 5. The
use of this concept as an Unclassified/Class 10 QRA method will be
discussed further in the next section.
Preface to Scope Change? Yes
Scope Change,
but cost in No
Unclassified/
Class 10 QRA
alternatives?
Unclassified/Class
10 Estimate QRA No Yes

Method Proposals
Wide Variation Single number to
Class 5 QRA No No Yes
within Scope? be reported?
Based on the background and methods
review above, several QRA and contingency/
reserve allowance determination methods are Yes
proposed for long-range Unclassified/Class
10 estimates, but also phase-gate, short-range FIGURE 7 Chart for Determining the Need for Special QRA Methods
Class 5 project estimates subject to exceptional
scope uncertainty (but within the bounds of
the business scope) and/or strategic risks. However, there are several However, even if there is likely scope change but the decision analysis
questions discussed below that need to be answered prior to considering method addresses the change (and/or strategic risks) in the definition
the methods in this paper, i.e., will an existing QRA RP suffice, and what is of the unique alternatives, and there is no single cost value reported (i.e.,
the objective? multi-project/multi-alternative case in Figure 2), then, existing RPs may
suffice for each alternative. Otherwise, the Unclassified/Class 10 QRA
THE TROUBLE WITH CLASS 5 methods to be identified in this RP should be considered.
In 2021, the author made several presentations on the topic of “the trouble
with Class 5 ranges” [42] [43]. The presentations discussed a 2020 study WHAT IS THE QRA OBJECTIVE?
of historical cost overruns for North American power projects [44]. That As discussed, stakeholders often have little expectation that strategic
study looked at actual cost growth data for Class 5 estimates (and other long-range LCCE/A or other study input estimates will ever be used
classes) in a phase-gate system. It also included data from prior related directly for funding or other individual commitments. Hence, they may not
studies. It found that the Class 5 p90 value (i.e., 90% had less cost perceive a need to include cost for scope change/variation and strategic
overrun) of the combined study dataset was +162% over the funded risk in the LCC input estimates as a contingency/reserve allowance.
amount. However, the AACE estimate classification RP for hydropower After all, their thinking is nobody will remember or use the project input
projects (RP 69R-12, Cost Estimate Classification Systems – as Applied in number later in its own right. In some cases, that will be true. However,
Engineering, Construction and Procurement for the Hydropower Industry) in ongoing portfolio or systems management, these LCCE/A results will
suggests that the highest Class 5 p90 would typically be +100% [45]. be revisited, projects will enter the phase-gate queue, and inevitably past
Many would consider +50% to be a more typical p90 at Class 5 (e.g., project cost values will be recalled. Cost estimators are all too familiar with
as was seen in the nuclear decommissioning estimate report [12]). The the push-back challenges created by stakeholder anchoring bias as well as
immediate question for a project in a short-range phase-gate system was losing credibility and trust when costs change from estimate to estimate
“how can the high range be more than 3X what is considered the typical AACE (usually increases) and the difference cannot be clearly explained. The cost
estimate classification range?” The answer is that estimating and QRA engineering objective should be to use the best, fit-for-use, practical QRA
processes fail to recognize that at Class 5 there are almost always multiple methods available to produce reliable results and avoid that situation.
scope options and strategic risks still on the table for more complex In any case, the estimator/analyst should start any analysis by assuring
projects such as the hydropower facilities studied (i.e., it is not until the there is clarity as to stakeholder expectations and objectives for the
Class 4 gate that a single alternative is selected). This situation was also QRA and what is to be communicated about risk. Optimally, portfolio
reported in the scenario example for complex upstream oil projects [27]. In management and LCC study processes with appropriate estimate and QRA
short, phase-gate Class 5 estimates for complex projects will benefit from methods, quality requirements, documentation (i.e., basis of estimate),
any Unclassified/Class 10 QRA methods to be identified here. historical data capture, and other elements will be established.
Figure 8 illustrates typical Class-to-Class estimate evolution and QRA
WILL AN EXISTING QRA RP SUFFICE? challenges, and the objective of Unclassified/Class 10 QRA methods to
As a starting point of method evaluation, the only current RPs recommended address the challenges. It shows that for complex projects with viable
for estimates with significant systemic risks such as poorly defined scope scope variations within the overall business scope and/or strategic
(i.e., for Class 5 or Unclassified/Class 10 estimates) are those incorporating risks, Class 5 estimate contingency is often significantly underestimated.
parametric risk analysis, but not CPM which is not practical at early phases; Research indicates that actual cost growth may be 2 to 3 times the
those are RPs 42R-08 and 113R-20 [8] [35]. Figure 7 provides a flow chart estimated contingency using traditional methods (often deterministic) [46].
of how to determine if these existing RPs will suffice. The first question is The same is true for Unclassified/Class 10 estimates where time adds to
whether the project is subject to likely scope change as is expected with a the evolution challenge no matter how complex the project. The objective
greater than 10-year time horizon. If not, the next question is whether there of the methods in this article then is to realistically quantify the risks as
is exceptional scope uncertainty (and/or strategic risks) within the general they are, i.e., to address scope variation and strategic risks inherent to long-
business scope. If no, then existing RPs will suffice, but if yes (even if Class range planning and estimates.
5), this RP’s proposed methods should be considered.

JA NUA RY/ FEBRUA RY 2 02 3 31

 1. MODIFIED RP 42R-08:
CALIBRATED PARAMETRIC
This approach is the same as RP
42R-08 which already addresses the
need to calibrate a base parametric
model to align with a company’s
experience6 [8]. The factoring of a
base model was also suggested in
the OCED uncertainties report [12].
The existing RP provides guidelines
for performing calibration. It is
FIGURE 8 Typical Class-to-Class Estimate Evolution and QRA Challenges recognized that few companies
will have life cycle data with which
to develop a parametric model
from scratch using regression
analysis. However, it is a common
and expected practice to calibrate
existing models where there is some
nominal level of actual/estimate
cost data available to do so.
For long-range planning, it is
most likely that data will be available
for utilities and others who have
decades of water, power or other
system planning experience with
relatively repetitive project types and
FIGURE 9 Calibration Factors for a Parametric Model (image from ValidRisk® software [47]) using technology that improves with
time, but usually not dramatically
so. The data needed, as discussed
Proposed QRA Methods for in RP 42R-08, is actual/estimate cost data that has been normalized to
remove the effect of escalation. Figure 9 shows a calibration input screen
Unclassified/Class 10 Estimates from a commercial parametric risk model; the factors shown for project size
attributes could readily be applied for short versus long-range estimates [47].
The following summarizes the viable QRA methods for project cost The calibrated parametric model will address the typical systemic risks
identified based on the preceding background and methods review. Again, for Class 5 estimates as well as the long-range scope variation and strategic
these are for QRA and contingency/reserve allowance determination for risk impacts covered by the calibration factors. The model will produce an
long-range Unclassified/Class 10 estimates, but also for phase-gate, short- overall project cost distribution from which a contingency/reserve allowance
range Class 5 project estimates subject to exceptional scope uncertainty value can be selected recognizing that this Class 10 value includes more
(but within the bounds of the business scope) and/or strategic risks. The scope definition and variation risk than for a Class 5 estimate.
following methods have been identified along with typical uses cases,
roughly in the order of increasing risks on the project: 2. MODIFIED RP 113R-20: HYBRID PARAMETRIC PLUS
EXPECTED VALUE (EV)
1. Modified RP 42R-08: Calibrated Parametric; primarily for more For Unclassified/Class 10 estimates, this method is essentially the same
repetitive portfolio projects (e.g., water system growth) where life as the RP with the exception that the definition of “critical risks” be
cycle data has been captured. interpreted to also include long-range scope variation and strategic risks
2. Modified RP 113R-20: Hybrid Parametric plus Expected Value with and their cost impacts. The parametric model will address the typical
inclusion of scope variation and strategic risks along with critical systemic risks for Class 5 estimates, and the other enhanced critical risks
risks; primarily for more unique projects, but with relatively stable are addressed in the EV model. The long-range risks would be identified
foreseen futures. using a workshop approach including appropriate business and technical
3. Decision Tree with Scope and/or Scenario Variation Branches (no staff involved in the associated portfolio or other long-range planning.
RP yet); primarily for projects with higher complexity and more This approach would be appropriate for projects that had more unique
dynamic external risk situations. long-range variation and risks than is typically involved in more repetitive system
planning where the parametric method alone would suffice. However, the projects
The following sections provide some detail for each proposed method should have a relatively stable foreseen future, otherwise the number of discrete
or variation. These should be considered as scoping statements for “critical” risks would become unwieldy and the quantification less reliable.
potential RPs to be developed after additional industry discussion and
hopefully more published papers on the topics. Given that there is no
existing RP for the decision tree QRA method, a more detailed treatment is 6 Obtain the latest revision of this RP (as one should do for all RPs) which includes
provided in the Appendix. the calibration guidelines.

32 JAN UA RY/FE B R UA RY 20 23
 An advantage of the hybrid model is that it helps provide more
understanding of the risks, i.e., systemic versus project-specific, but also
risk-by-risk for the critical risks. It can also be used in sensitivity analysis
by varying the parametric model inputs and/or the specific risk inputs in
the EV model.
3. DECISION TREE WITH SCOPE AND/OR SCENARIO
VARIATION BRANCHES
There is no AACE RP for a decision tree QRA method. However, the generic
decision tree RP 85R-14 provides the basics of the general method [40]. A
QRA RP would need to expand on that RP to address more specifically
using a tree with scope and/or scenario variation branches. By leveraging
scenario analysis (as per source [37]), the decision tree approach is more
flexible in addressing complex projects with evolving technology and/
or more dynamic, wider ranging, external risk situations. To provide
a reference for a potential decision tree QRA RP, the method is further
described in the Appendix including several examples.
If used for scenario modeling, the approach requires a scenario
identification process. The paper by Meads et.al, describes a 3-step process
[37]. In the first step, the study identifies external factors that could affect
project execution. The second step is similar but focuses on internal
uncertainties or trends. The third step defines 2-4 plausible situational
scenarios (it suggests avoiding low probability scenarios). It also suggests
grouping external factors and/or key uncertainties into a narrative or
storyboard to better communicate and get consensus on the analysis.
This is far from a perfect process, but a much more robust one than just
analyzing a base case.
Conclusions
This article reviews the concepts of estimate classification in general and
Unclassified/Class 10 estimates in particular. Based on a literature review,
it reviewed the concept of asset life cycle cost estimating and analysis. It
also reviewed various decision analysis methods such as scenario analysis
and decision trees to identify potentially useful QRA concepts. With this
basis, proposed QRA and contingency/reserve allowance determination
methods for long-range estimates were presented. Scoping statements
were provided for potential Unclassified/Class 10 QRA RPs (or revisions
to existing RPs to extend their application to long-range estimates). In
particular, a new RP for decision tree based QRA method was outlined. It
is hoped that this article will encourage others to publish on the topic and
share other potential QRA methods.
References
1. AACE International, “Professional Guidance Document No. PDG-01,
Guide to Cost Estimate Classification Systems,” AACE International,
Morgantown WV, Latest revision.
2. AACE International, “Recommended Practice 17R-97, Cost Estimate
Classification System,” AACE International, Morgantown WV,
latest revision.
3. AACE International, “Recommended Practice 111R-20, Estimating
for Long-Range Planning – As Applied for the Public Sector,” AACE
International, Morgantown WV, latest revision.
4. Taylor, L., G. Brink, P. Galeno, D. Leo, “Pre-Class 5 Estimating Cost
Communication for Long-Term System Planning,” AACE International
Transactions, 2018.
5. Consumers Energy, “Consumers Energy 2019 Clean Energy Plan:
Executive Summary,” Michigan, 2019.
6. Dysert, L. and T. Pickett, “Supporting Estimates with Effective
Scope of Work Definition,” in AACE International Transactions,
Morgantown, WV, 2020.
7. AACE International, “Recommended Practice 104R-19, Communicating
Expected Estimate Accuracy,” AACE International, Morgantown WV,
latest revision.
8. AACE International, Recommended Practice 42R-08, Risk Analysis
and Contingency Determination Using Parametric Estimating,
Morgantown, WV: AACE International, Latest revision.
9. AACE International, “Recommended Practice 57R-09: Integrated
Cost and Schedule Risk Analysis Using Risk Drivers and Monte Carlo
Simulation of a CPM Model,” AACE International, Morgantown WV,
latest revision.
10. AACE International, “Professional Guidance Document No. PGD-
02: Guide to Quantitative Risk Analysis,” AACE International,
Morgantown WV, Latest revision.
11. AACE International, “Recommended Practice 10S-90; Cost Engineering
Terminology,” AACE International, Morgantown WV, Latest revision.
12. OECD, “Addressing Uncertainties in Cost Estimates for Decommissioning
Nuclear Facilities,” A Joint Report by the OECD Nuclear Energy
Agency and the International Atomic Energy Agency, 2017.
13. Stephenson, H.L. “Strategic Portfolio Management: Asset Management
Model,” in AACE International, Morgantown, WV, 2021.
14. Stephenson, H.L. and R. Gerber, “Strategic Portfolio Management:
Funding and Finance Methodologies,” in AACE International
Transactions, Morgantown, WV, 2020.
15. Stephenson, Lance, editor, Total Cost Management Framework,
Morgantown WV: AACE International, 2015 (2nd edition).
16. International Standards Organization (ISO), “ISO 55000:2014, Asset
management — Overview, principles and terminology,” ISO, Geneva,
Switzerland, 2014.
17. SFMTA , “2021 20-Year Capital Plan,” San Francisco Municipal
Transportation Agency , San Francisco, CA, 2021.
18. ICMS Coalition, “ICMS: Global Consistency in Presenting
Construction Life Cycle Costs and Carbon Emissions,” International
Cost Management Standard Coalition, November 2021.
19. Harbuck, R., “Life Cycle Cost Analysis for Transportation Projects,”
AACE International Transactions, 2009.
20. Gransberg, D., and E. Scheepbouwer, “Infrastructure Asset Life Cycle
Cost Analysis Issues,” AACE International Transactions, 2010.
21. U.S. Department of Energy, “Cost Estimating Guide,” U.S.
Department of Energy, Washington DC, 2018.
22. GAO, “Cost Estimating and Assessment Guide,” U.S. General
Accounting Office, Washington DC, 2020.
23. Alborzfard, N., “A Framework for Life Cycle Cost Analysis
of Sustainability Features in Buildings,” AACE International
Transactions, 2012.
24. Gao N. and A. Touran, “Cost Overruns and Formal Risk Assessment
Program in US Rail Transit Projects,” Journal of Construction
Engineering Management, 2020.
25. Zhang, Y. and A Touran, “Are Soft Costs Increasing In Transit Projects?,”
in Construction Research Congress, 2012.
26. Brock, D., “ICMM guidance and resources for integrating closure
into business decision making processes,” in International Council on
Mining and Metals; Mine Closure 2021, United Kingdom, 2021.
27. Pearce, L., “ERM outlines the true cost of mine closure,” Northern
Miner, 4 March 2021.
JA NUA RY/ FEBRUA RY 2 02 3 33
 28. OECD, “Costs of Decommissioning Nuclear Power Plants,” Nuclear
Energy Agency (NEA), Boulogne-Billancourt, France, 2016.
29. Ahiaga-Dagbui, D., P. Love, A. While, P. Boateng, “Costing and
Technological Challenges of Offshore Oil and Gas Decommissioning in
the U.K. North Sea,” Journal of Construction Engineering Management,
2017.
30. AACE International, “Recommended Practice 40R-08, Contingency
Estimating-General Principles,” AACE International, Morgantown,
WV, Latest revision.
31. UK Department for Transport, “Transport Analysis Guideline TAG
Unit A5.3: Rail Appraisal,” UK Department for Transport: Transport
Appraisal and Strategic Modelling Division, London, UK, 2021.
32. Infrastructure and Projects Authority, “Cost Estimating Guidance,”
Infrastructure and Projects Authority, London, UK, 2021.
33. AACE International, “Recommended Practice 118R-21: Cost Risk
Analysis and Contingency Determination Using Estimate Ranging for
Inherent Risk With Monte Carlo Simulation,” AACE International,
Morgantown WV, latest revision.
34. Burroughs, S. and G. Juntima, “Exploring Techniques for Contingency
Setting,” in AACE International Transactions, Morgantown WV, 2004.
35. AACE International, “Recommended Practice 113R-20: Integrated Cost
and Schedule Risk Analysis and Contingency Determination Using
Combined Parametric and Expected Value,” AACE International,
Morgantown WV, Latest revision.
36. AACE International, “Recommended Practice RM-34: Integrated Cost
and Schedule Risk Analysis and Contingency Determination Using
Estimate Ranging with Expected Value and Monte Carlo Simulation,”
AACE International, Morgantown WV, latest revision.
37. Meads, M. and A. Shakourifar, “Utilizing Scenario Planning on
the Risk Assessment Exercise for High Complexity Projects,” AACE
International Transactions, 2020.
38. Department of Treasure and Finance, “Economic Evaluation for
Business Cases Technical Guidelines,” State of Victoria, Australia, 2013.
39. Chillaz, A. and B. Rodriques, “Uncertainty Assessment of Business
Plans Using Monte Carlo Simulations,” AACE International
Transactions, 2014.
40. AACE International, “Recommended Practice 85R-14: Use of Decision
Trees in Decision Making,” AACE International, Morgantown WV,
latest revision.
41. Hoogenboom, J. and W. Dale, “Dispute Resolution Strategy and
Decision Analysis,” AACE International Transactions, Morgantown,
WV, 2005.
42. Hollmann, J., “The Trouble with Class 5 Cost Range,” LinkedIn Article,
4 April 2021.
43. Hollmann, J., “E06 The Trouble with Class 5 (and 10) Estimate
Accuracy with John Hollmann,” YouTube: Project Speaker’s Podcase, pp.
https://www.youtube.com/watch?v=plhZhnH2S-s, 19 June 2021.
44. Hollmann, J., et.al., “Variability in Accuracy Ranges: A Case Study
in the US and Canadian Power Industry,” AACE International
Transactions, Morgantown, WV, 2020.
45. AACE International, “Recommended Practice 69R-12: Cost Estimate
Classification Systems – as Applied in Engineering, Construction
and Procurement for the Hydropower Industry,” AACE International,
Morgantown, WV, latest revision.
46. Hollmann, J., “Chapter 4, Accuracy: Confidence and Credibility,” in
Project Risk Quantification, Sugarland TX, Probabilistic Publishing,
2016, pp. 56-57.
47. ValidRisk, “ValidRisk: Learn About,” in https://validrisk.com/Pub/
Resources, Accessed January 2022.
34 JAN UA RY/FE B R UA RY 20 23
ABOUT THE AUTHOR
John K. Hollmann, PE CCP CEP DRMP FAACE
Hon. Life, is with Validation Estimating, LLC.,
and can be contacted by sending email to:
jhollmann@validest.com
Appendix
ADDRESSING MULTIPLE SCOPE OPTIONS USING A
DECISION TREE
As discussed, a problem with most Class 5 QRA analyses as typically
conducted is that project teams and estimators fail to recognize that in
phase-gate systems there are still multiple, feasible options or “features”
within a very broad, conceptual scope definition at Class 5. In most
phase-gate systems, a single option that will be advanced through basic
engineering or front-end engineering and design (FEED) is not chosen
until the Class 4 gate. So, the p90 of a Class 5 estimate for a single option
will not be the p90 considering the range of cost for all the active scope
and feature options. The p90 considering all of the options will likely be
higher in absolute cost terms than for a single middle-of-the-road option as
is often chosen as the basis for the Class 5 budget number.
As an example, consider a hydropower dam project. At Class 5,
the scope may have been defined as a dam to support a given power
generation capacity at a general river location. Within that broad scope
statement, assume the business has identified three primary optional
dam geometries as to how the structure might cross the river. The final
geometry choice at Class 4 will depend on geotechnical study that
will be done after the Class 5 gate decision. For a project that has any
complexity, this situation of multiple options or features is the norm
at Class 5. In addition to geotechnical studies as in this example, the
phase following the Class 5 gate includes increasingly comprehensive
studies and consultations regarding environmental, community and other
sustainability considerations. However, for this example, only the three
geometry options will be considered.
Assume the base estimates excluding contingency for the 3 options
are $20, $20 and $40M. Based on the company’s “ranging” QRA, assume
that each had a p90/p10 range of +100/-50% (in an actual risk analysis,
not using predetermined ranges, each would have a unique range). Now,
assume the business decided to use as a base estimate for its long-term
budget what they assumed was a mostly likely or middle-of-the-road
value of $20M because 2 of the 3 options cost that much. What should
the p50 and p90 be for a traditional QRA approach using one option as
the favorite? If a Monte Carlo simulation (MCS) was used on this favored
option, with a base estimate or most likely value of $20M and @Risk®
trigen[10,90] distribution of -50/+100%, Figure A-1 is the result. The p50
is about $23M or a contingency of 15% on the $20M base estimate (recall
that in a nuclear decommissioning reference, one agency only allowed 10%
[12]). The p90 value is the +100% of the trigen distribution or about $40M.
In summary, their budget using a single option ranging approach would be
$23M at p50 with a p90 of $40M.
The better way proposed here is to use a method to evaluate all the
viable scope variations, not just the “base case.” The following is simple
example of using a decision tree approach with
Monte Carlo simulation (MCS) for a Class 5
valuation with the 3 options.
The proposed method starts with the
premise of multiple options and for each option
there is a probability of being selected at the
next gate after further study. In the example,
the options are numbered 1, 2 and 3, and the
probabilities of selection (i.e., the favorability
ratings) are assumed equal, i.e., 1/3 or 33.3%
for each. In MCS, the option choice can be set
up as a “discrete” function such that in each
MCS iteration, one of the three options will
be selected in accordance with the respective
selection probabilities.
Next, assume for each of the three options,
a separate QRA ranging analysis has been
done. For the example, this is represented by FIGURE A-1 Example MCS Outcome for a Single Option (using Palisade @Risk software)
three trigen distributions with a p10/p90 range
around the most likely value of -50%/+100%. Figure A-2 shows the Class
5 multi-option analysis worksheet in Excel® using Palisade @Risk®
functions. The trigen distributions include an “if” function that refers to
the discrete sampling function that picks either option 1, 2 or 3 in each
MCS iteration. The three options are summed with the summation being
identified as the @Risk® output for plotting.
The MCS output of the total cost distribution for the above model is
shown in Figure A-3. Note the p50 of this distribution is about $27M or a
contingency of 35% on the “base” of $20M. The p90 of this distribution is
$53M. This is +165% more than the “base” of $20M (20 x 1.65 =$33M).
By design of the example, this roughly compares to the previously
referenced 2020 power project study that reported a p90 of +162% around
the estimates with underestimated contingency [44].
If this simple model had been used as a basis for reporting a Class
5 budget, the p50 value would be $27M with a p90/p10 range of
$53M/$17M or about +96/-37%; again, similar to the 2020 study with a
corrected contingency.
The trouble with Class 5 estimate accuracy then is not that the
indicative AACE classification RP range-of-ranges are incorrect. It is that FIGURE A-2 Example 3-Option Decision Tree MCS Model
contingency and range are being underestimated for individual options and (using Palisade @Risk for Excel)
for the potential scope as a whole. The simplistic
single-option view in the example had a p90
cost of $40M while the multi-option view p90
was $57M. The contingency on the “most likely”
single option was 15% while the multi-option
contingency (on the $20M base estimate) is 35%.
The decision tree approach can become
cumbersome if there are too many options or
branches; however, a more complex branched
model than the example using the same concept
is certainly practical. Also, the example used
simplistic 3-point cost range inputs for each
option; the inputs to this model could have used
the distribution outputs from either the modified
parametric or the modified hybrid parametric
plus EV method described in the paper. In other
words, this method adds a layer of analysis
for more complex project situations, but with
flexibility to keep it fit-for-use making it practical FIGURE A-3 Example 3-Option MCS Model Total Cost Output (Using Palisade @Risk)
for a wide range of uses.

JA NUA RY/ FEBRUA RY 2 02 3 35

ADDRESSING MULTIPLE SCENARIOS
USING A DECISION TREE
The method described in the prior section can
also be used to analyze the cost of various
scenarios rather than discrete scope options.
For example, a low-cost scenario may reflect an
opportunity to introduce new, lower capital cost
technology, while the high-cost scenario may
reflect existing technology with increasing cost
for environmental and other mitigations (i.e.,
strategic risks) to address ever more rigorous
regulations.
Figure A-4 shows the same multi-option
worksheet as for Class 5 in Figure A-2, but with
low cost, base case, and high-cost scenarios
of $10, $20 and $50M respectively (excluding
escalation). Figure A-5 shows the MCS outcome
of the model. It exhibits a long tail with a p90 of
$76M which is 3.8X the base case of $20M. In
the example, the analyst would need to decide FIGURE A-4 Example Multi-Scenario Decision Tree MCS Model (using
what number” to report given the highly skewed Palisade @Risk for Excel)
outcome. However, in all cases, the p90 is very
important to life cycle cost analyses which
usually apply a net present value (NPV) model.
A key output of the NPV approach is usually the
tornado diagram that ranks the importance of
each NPV driver. Revenue is usually the most
important driver. However, the higher the p90
on capital cost, the more management attention
will be drawn to the capital cost tornado bar
(as it should be). A realistic high p90 will also
incentive the owner to invest in new technology
to reduce the cost.
The example used simplistic 3-point cost
range inputs for each scenario; the inputs to
this model could have used the distribution
outputs from the other suggested QRA methods.
In addition, the branching of scenarios could
be much more complex, but still practical. This
example could be applied in a new QRA RP for
Unclassified/Class 10 estimates. FIGURE A-5 Example 3-Scenario MCS Model Total Cost Output (Using Palisade @Risk)

Save the Date! Social and Networking Events | Technical Sessions

CEU Learning Opportunities | and much more!

36 JAN UA RY/FE B R UA RY 20 23
False Estimate Accuracy
BY DAVE KYLE, CCP CEP

Editor’s Note: This article was peer-reviewed and presented as EST-3855 at the 2022 AACE Conference and Expo. It scored high by
the Technical Board’s Review Committee using the Technical Paper Evaluation Criteria (TPEC) spreadsheet. It also scored high on
attendee review evaluations. Successful completion of these pre-steps resulted in this article being selected for publication in the Cost
Engineering journal.

ABSTRACT
This article is intended to provide an introduction to understanding estimate quality and its relationship
to accuracy. It is recommended that the reader continue the learning process through study of the many
supporting AACE documents available (see the References section and AACE website).

Understanding true estimate accuracy is critical in determining if the estimate meets client expectations and is
suitable for the intended purpose. This article examines and discusses the foundations of estimate accuracy, and
common misrepresentations and misconceptions of it. A strong understanding of the foundations of estimate
accuracy enables the estimate reviewer or recipient to better understand an estimates true accuracy.

Some incorrect practices to be examined are creation of deliverables to estimating based on conjecture
instead of sound engineering design; incorrect use and presentation of historical data; assumptions used
in place of a sound execution plan; incorrect use of factors and allowances; improper understanding of
uncertainty and contingency; and a lack of supporting benchmarking.

This article will also examine key contributors to desired estimate accuracy. A proper understanding of the key
drivers of estimate accuracy guides the estimate developer to successful practices that will result in the best
possible (accurate) outcomes. For the estimate recipient, the proper understanding of the foundations of good
estimate accuracy allows early (and comparatively inexpensive) identification and correction of deviations from
best practices that contribute to poor accuracy.

The reader will understand that while a few minor inaccuracies in an estimate may not have a significant
impact on the overall accuracy, a wholesale disregard of the need to reduce uncertainty in the estimate by the
many parties providing input will lead to grossly inaccurate project cost estimates.

The author is drawing from previous work experiences.

JA NUA RY/ FEBRUA RY 2 02 3 37


Introduction
Some of the key goals of an estimator are to minimize estimate
uncertainty, provide a cost estimate that meets the stated requirements
of the estimate requester, and ensure alignment with industry best
practices. The estimate reviews, presentations, and basis document
address cost basis, contingency, and range of accuracy. Many terms are
used within the presentation and estimate documentation that should
help to provide insight into the overall accuracy of the estimate (see
the common terms section). The burden is on the estimate recipient
(or their designate) to properly understand the presented material and
be able to query the estimating process at a level that allows proper
interpretation and understanding of the presented information. This
supports the determination of the estimate provider’s success in meeting
the expectations outlined in the scope of work for the project and in
following industry best estimating practices. An inadequate understanding
of the estimating process leaves the recipient at the mercy of the provider
and may leave them unsure of the quality of the estimate. This article will
provide the reader with a better understanding of common estimating
terminology used to express estimate quality and accuracy. This
understanding will help the estimate receiver to review and drill down into
the estimate basis to help satisfy their understanding of the cost estimate
in order to identify and mitigate unacceptable uncertainty (accuracy).
Common Terms Related to
Estimate Accuracy
The following terms are commonly used in estimate presentations.
The associated definitions (or paraphrases) are excerpts from AACE
Recommended Practice 10S-90 [1]. Each closely relates to understanding
estimate accuracy:
• Scope – The sum of all that is to be or has been invested in and
delivered by the performance of an activity or project.
• Project Definition – The process of exploring thoroughly all
aspects of a proposed project scope and to explore relations between
required performance, development time and cost.
• Historical data – records accumulating past project experience
stored as data for use in planning, estimating, forecasting, and
predicting future events. This includes:
◦ Labor cost
◦ Labor productivity
◦ Key quantities
◦ Bulk material costs
◦ Equipment costs
◦ Indirect costs - costs not directly attributable to the completion
of an activity, which are typically allocated or spread across all
activities on a predetermined basis.
◦ Engineering Procurement Construction Management (EPCM) costs
◦ Owner’s costs
• Execution Strategy – the planned methods for executing the project.
• Cost Estimating Relationship (CER) – In estimating, an algorithm
or formula that is used to perform the costing operation.
• Factors – a cost derived using a CER (cost estimating relationship)
which is related to another cost.
38 JAN UA RY/FE B R UA RY 20 23
• Allowances - resources included in estimates to cover the cost of
known but undefined requirements for an individual activity, work
item, account, or sub-account.
• Base Estimate - Estimate excluding escalation, foreign currency
exchange, contingency and management reserves. Syn.: POINT
ESTIMATE; SINGLE POINT ESTIMATE. See also: DETERMINISTIC
ESTIMATE. (December 2011)
• Uncertainty - A synonym for all risk, including all events both
positive and negative whose probabilities of occurrence are neither
0% nor 100%. See also: RISK.
(2) The total range of events that may happen and produce risks
(including both threats and opportunities) affecting a project.
(Uncertainty = threats + opportunities.)
◦ Systemic Risks - are uncertainties (threats or opportunities)
that are an artifact of an industry, company or project system,
culture, strategy, complexity, technology, or similar over-
arching characteristics.
◦ Project Specific Risks - are uncertainties (threats or
opportunities) related to events, actions, and other conditions
that are specific to the scope of a project. (e.g., weather, soil
conditions, etc.).
• Contingency - An amount added to an estimate to allow for items,
conditions, or events for which the state, occurrence, or effect is
uncertain and that experience shows will likely result, in aggregate, in
additional costs.
• Estimate Confidence Interval - (1) The probability that a result will
be within a range.
(2) In the context of estimate accuracy, the confidence interval
is typically expressed as the probability that the estimate may
be between a low probability of underrun and a high value of
underrun. For example, an 80% confidence interval for an estimate
accuracy range defines the range between the estimate value at a
10% probability of underrun (P10 value) and a 90% probability of
underrun (P90 value). See also: CONFIDENCE LEVEL; EXPECTED
ACCURACY RANGE; P-VALUE. (February 2021)
• Confidence Level – The probability: 1) That results will be equal to
or more favorable than the amount estimated or quoted; or 2) That
the decision made will achieve the desired results; or 3) That the
stated conclusion is true. For example, a 50% confidence level for
an estimate value (often noted as a P50 value) indicates an expected
50% probability that the final result will be less than (more favorable)
or equal to the P50 value. Note: Confidence level may also be
expressed as “equal to or less favorable”. If that is the case, it should
so be noted. Without such a note, the definition shown is assumed.
See also: CONFIDENCE INTERVAL; EXPECTED ACCURACY
RANGE; P-VALUE. (February 2021)
• Benchmarking - A measurement and analysis process that compares
practices, processes, data and/or relevant measures to those of a
selected basis of comparison (i.e., the benchmark) with the goal of
improving performance.
Areas of Examination
This section of the article looks at key areas the estimate recipient
should examine during the review process. The content of this section
supports a detailed examination, such as one that may be conducted on
a deterministic estimate. A conceptual estimate cannot be reviewed at
this level of detail, but the basis of each of these categories and the way
each was adjusted for in the conceptual estimate (if applicable) must be
understood. These assumptions must be well documented in the basis of
estimate document.
SCOPE DEFINITION
The owner’s requirements document and the estimate basis must clearly
define the scope of both the project and all aspects of the cost estimate.
The document should identify estimate exclusions – items that the reader
might expect to be included in the scope for a given type of facility but are
not included in the estimate. Attention should also be paid to the inclusion
of scope in the estimate that is actually the responsibility of another party,
or duplication of costs when scope is captured by more than one party. An
example of this would be inclusion in the estimate of an external power
line constructed by the utility at their expense, with the cost being typically
recovered through ongoing toll charges to the facility. It such as case, the
power line cost should not be included in the capital cost estimate and be
explicitly excluded from the estimate.
PROJECT DEFINITION
The level of project definition required for a specific class of estimate is
well defined in AACE recommended practices and is typically expressed
for specific types of deliverables reflecting the design, and the stage of
development of each deliverable. Adequate project definition is critical
in attaining the required level of estimate accuracy as no amount of
estimating work can compensate for an inadequate technical design.
Verifying that the engineering and project deliverables satisfy the
requirements of the specified estimate class will ensure that the input to
the estimate is adequate.
One common misconception is that the level of detail of deliverables
provided to estimating are indicative of the level of accuracy. For instance,
detailed MTO’s do not support estimate accuracy well if they are based
on educated guesswork by engineering, instead of the appropriate level of
design work (project definition).
Another common misconception is that the use of historical data from
a similar project will provide a high level of accuracy. Even compensating
for major scope, execution, time period, and location other potential
differences will likely provide no more than a conceptual level of estimate.
HISTORICAL DATA
Historical data is a term commonly used in estimate presentations to lend
support to the validity and accuracy of the estimate. However, the term is
vague and does not leave the recipient with a clear understanding of the
basis of the estimate being reviewed. Historical data such as placement
rates (labor hours per unit) must be recent, relevant to the project at hand
(i.e., from a similar project), and properly adjusted to capture the current
projects specific conditions. Query the sources of information and the
adjustment methods of data carefully. Listed next are common areas of
concern for both conceptual and deterministic estimates:
Data Sources
Data may be obtained from a variety of sources including government
agencies, commercial publications, vendor quotes, construction
contractors, and previous projects. Data may not always be current,
nor completely similar to the current project. A critical activity in the
estimating process is adjusting the data to capture the conditions,
specifications, location, and time frame of the current project. A detailed
explanation of the quality of the source data and the adjustment methods
used should be sought and thoroughly examined to determine if the
resulting normalized data is acceptable.
Quantities
The basis of the quantities should be derived from the engineering
design. Any assumed, factored, or forced detail quantities (such as from
an algorithm in estimating software) should be identified along with any
assumptions they are based on. These should be neither overly optimistic
nor pessimistic but reflect a P50 expected outcome (equal probability of
being low or high) as the design continues to develop.
Labor Cost
Labor cost must reflect the current hourly cost for the workers to be
employed at the site. Costs must include all union and government
benefits and burdens, adjustments for planned overtime, and any
additional costs resulting from the chosen shift and rotation schedule.
Labor Productivity
Labor productivity can have a significant cost impact on the outcome of
the estimate. Although the estimator may use a well-known commercially
available source for the unit hours of installation of bulk material
and equipment, the data must be properly adjusted to reflect the site
conditions for the current project conditions. The total hours may be
misstated by a factor of 2-3 times or more if this is not properly done.
Complete a detailed examination of the source and methodology
used to develop the labor hours, and the suitability of data used. Checks
include determining if source hours have been properly calibrated to
reflect projects that are similar to the one being estimated, and that scopes
of work are correctly assigned in accordance with the execution plan (i.e.,
that welds being performed in a heated sheltered fab shop are correctly
assigned appropriate hours reflecting this optimum environment).
Bulk Material Costs
Bulk material costs must reflect the total cost for materials specified for
the current job, including freight to the site, the volume of material to be
purchased (volume will likely impact supplier unit costs), and any current
market conditions such as limited supply, a highly competitive market, or
the ability of a specific vendor to supply a large volume. Global supply-chain
constraints may also need to be considered. Purchase quantities may be
higher than installed quantities as a result of cutting waste or other issues.
Equipment Costs
Equipment costs must reflect the engineering design specifications, freight
costs (including customs and storage), and any special freight charges
such as those encountered for an oversize shipment load. A thorough
bidding or procurement process including a competitive bidding process,
proper technical and commercial evaluation of the quotes or bids, and
detailed recommendations support the reduction of uncertainty in
equipment costs.
Indirect Field Costs
Indirect field costs are frequently underestimated. This portion of the
estimate must be broken into sufficient detail to properly assess the
completeness of the category and reflect content in alignment with
the direct costs. If factored, the basis of the factor must be clearly
defined. It should include documentation supporting the staff required
(i.e., a staffing plan), current unit pricing for labor and all supporting
construction support equipment and facilities, and meet government or
labor organization requirements (i.e., ratio of supervisory staff to workers,
or safety staff to worker ratio). Pricing must reflect the site location and
conditions and be correctly aligned with the project execution plan. If
factored, the basis of the factor must be clearly defined.
JA NUA RY/ FEBRUA RY 2 02 3 39
Engineering, Procurement, and Construction Management (EPCM) Costs
EPCM services costs must be broken into sufficient detail to properly
assess the completeness of the category, including documentation
supporting the staff required and unit pricing for labor and all supporting
equipment and facilities. Comparison between staff hours and quantity of
deliverables is a useful method of assessment. If factored, the basis of the
factor must be clearly defined.
ESCALATION
The method for escalation calculation must be clearly defined, based
on current information from a suitable source. For instance, using a
government consumer price index projection for escalation on a heavy
industrial project would not be suitable when better industry related
data is available. The escalation rate per year and assumed cash flow
assumptions should be stated in the basis of estimate.

Owners’ Costs FOREIGN EXCHANGE

This must clearly reflect the expected costs that will be charged to the
project by the owner and must be in alignment with the owner’s strategy,
capabilities, and direction. This may include staff, land purchases,
regulatory requirements, stakeholder alignment work, legal costs, and
facilities supplied by the owner to the contractors. If factored, the basis of
the factor must be clearly defined.
The basis of foreign exchange must be clearly identified, along with the
percentage of equipment or material sourced from a foreign location. This
identifies the amount of exposure the recipient has to fluctuating exchange
rates. Ensure that domestically sourced material that is obtained by the
vendor from foreign sources is included in the compilation of costs, as this
increases the recipient’s exposure to changes in exchange rates.

EXECUTION STRATEGY BASE ESTIMATE

The documented execution strategy is critical to the accuracy of the The base estimate is the completed estimate excluding escalation, foreign
estimate. Broad assumptions used in place of a proper execution plan currency exchange, contingency, and management reserves. History
for the project will introduce uncertainty into the final cost that can shows that the base estimate tends to be on the low side of final actual
be significant. Improper assumptions around the execution of the cost due to uncertainty and inherent variability of the estimating process.
project such as the schedule, contracting strategy, logistics plan, pre- This should be corrected by a risk (or uncertainty) analysis, including the
commissioning or commissioning plan can significantly reduce certainty in proper development of contingency and escalation values to be added to
the estimate. An example of this is executing a weld in a heated, sheltered the estimate.
shop as opposed to in the middle of an operating plant in winter. This
difference in location for the same type of work can impact labor hours by UNCERTAINTY
a factor of three to four times or more. Another example is the engineering The term uncertainty refers to all threats or opportunities present in the
deliverables sent to estimating must properly assign the breakdown by project at the time of the estimate. This should only include residual threats
contract and location (i.e., the project coding structure) to each item in or opportunities that may impact the costs included in the current estimate.
order to categorize and properly adjust costs in the estimate. This will It is composed of two types of risk, systemic risks, and project specific risks.
create control budgets that can be successfully used to manage activities Each of these risks are dealt with in a significantly different manner and
such as screening and reviewing bids for the proposed scope of work. should be reviewed separately. The estimate provider should ensure that
all threats or opportunities have been recorded in the project team’s risk
COST ESTIMATING RELATIONSHIPS (CERS) register, including a ranking of the likelihood and magnitude of each item.
Conceptual estimating practices (and some minor deterministic estimating
functions) rely heavily on historical data and CER’s derived from it. Systemic Risks
Examination of historical data which CER’s are based on should be Systemic risks are risks that are inherent within the process or company
supported by a well-documented basis for the information. This allows the and are present in most or all estimates. They are not necessarily specific
estimator to properly identify and adjust for the differences between the to a given project. Most systemic risks have a probability of 100% (i.e.,
historical data and the current project. The adjustments should also be they always exist); however, the potential impact of a systemic risk
well documented and make logical sense. may vary depending on the particular project. These risks may include
potential turnover of project team personnel, shortcomings within the
FACTORS corporation in the management of a project (particularly related to
Factors may be used to estimate costs for currently un-designed scope. change management), project complexity (including both technical scope
They are derived from an established relationship with a known cost. For complexity and execution complexity), or issues with the maturity level of
example, small bore pipe may be estimated as 22-24% of the length of the technical and project deliverables that should be available to support
large bore piping for a similar project. Clearly understanding the basis for a estimate preparation. Systemic risk is often assessed qualitatively or via
factored cost is critical in determining the associated uncertainty. assessments made from analysis of historical projects (such as the use of a
parametric model to address the systemic risk impact).
ALLOWANCES
Allowances are an estimated cost for undesigned scope that cannot be Project Specific Risks
estimated using a cost estimating relationship and may be based on Project specific risks are unique to a project, and the associated probability
nothing more than the estimator’s judgment. Allowances typically have a of the risk occurring may be less than 100%. These may include site specific
high amount of uncertainty and should represent a very small portion of conditions, weather constraints, local labor factors etc. Generally, analysis of
the estimate. The allowance should be for specific scope items only and the project specific risks should address both the probability that the risk occurs,
basis clearly documented in the basis of estimate. as well as the potential range of impact that may result if the risk event
actually happens. Project specific risk is typically assessed quantitatively.

40 JAN UA RY/FE B R UA RY 20 23
Contingency
Contingency is frequently misunderstood or misused. Senior executives
may seem somewhat cynical when presented with a valid contingency cost. indicate an error but highlights that area for greater scrutiny. Significant
It may be seen by them as a method to simply overstate the estimated cost differences that are not resolved should always be accompanied by a clear,
to ensure that that the project team does not overrun the budget.
However, ignoring the need for an appropriate contingency value
based on project uncertainty and client risk tolerance may lead to chronic
cost overruns as history shows that base estimates (without consideration
for uncertainty) are most often lower than the final actual cost. Ignoring
or frequently understating contingency may result in project teams
compensating by overstating quantities or costs within the estimate to
prevent cost overruns. Overstating contingency is equally damaging as the (and flushing out false representations of estimate accuracy) is an
client unnecessarily reserves an excessive amount of funds that could be
used for other investments.
Contingency is best determined by a risk analysis which provides
the contingency value based on the desired probability of overrun or
underrun specified by the recipient. A client with a low risk tolerance
(perhaps a smaller client with only one project in their portfolio) may
desire a higher level of contingency thereby reducing the likelihood of a
cost overrun, whereas a larger client with many projects may desire a P50
probability with the understanding that roughly 50% of their projects may
overrun, and 50% underrun, but the overall portfolio will stay close to the
total budget across a portfolio of projects.
The contingency value should reflect the assessment of the uncertainty
within the project including both risks and opportunities. A high-level
comparison between contingency and the project risk register should
intuitively make sense. For example, high levels of uncertainty may
increase contingency, and will most certainly create a broader accuracy
range. The estimate provider should provide a very clear description of
the method used to develop the proposed contingency, including any
supporting documentation.
Confidence Level
The desired confidence level for the total value of the estimate is generally
specified by the estimate recipient and guides the risk analyst in
determining the amount of contingency to be added to the base estimate
to bring the final estimate value to the desired confidence level. This value
should directly reflect the results of the risk analysis and a comparison
between the risk register and the contingency value should again
intuitively make sense. The confidence level defines the probability of
underrun associated with the estimate value including contingency.
Estimate Confidence Interval
The desired estimate confidence interval is generally specified by the
estimate recipient and defines the upper and lower data points (accuracy
range) to be reported from the risk analysis probability distribution. For
example, if the desired estimate accuracy confidence level is 70%, the
stated estimate accuracy range will reflect the range between the P15 value
(the value associated with a 15% probability of underrun) and the P85
value (the value associated with an 85% probability of underrun). These
probabilities are determined from the project cost probability distribution
resulting from a risk (uncertainty) assessment. Using a 70% confidence
level in determination of the stated estimate range implies that the final
cost of the project should be with this range 70% of the time (i.e., 70 out of
100 times of repeating the same project).
BENCHMARKING
Benchmarking analysis should identify a comparison between the current
project estimate and historical data adjusted to reflect the current projects
execution strategies and work conditions etc. Significant differences
between the benchmark values and the estimate values do not necessarily
logical, detailed explanation.
Conclusion
In conclusion, the key to thoroughly understanding estimate accuracy
estimate provider and recipient who both understand the key drivers
behind proper accuracy, and endeavor to attain the best scenario for
each driver, resulting in clearly understood and satisfactory estimate
accuracy. Proper documentation of the estimate is critical in aiding the
understanding of the estimate.
References
1. AACE International, Recommended Practice No. 10S-90, Cost
Engineering Terminology, Morgantown, WV: AACE International,
Latest revision.
2. AACE International, Recommended Practice 105R-19, Cost Estimate
Classification System - As Applied in Engineering, Procurement, and
Construction for the Process Industries, Morgantown, WV: AACE
International, Latest revision.
3. AACE International, Recommended Practice 18R-97, Estimate
Requirements Document - - As Applied in Engineering, Procurement,
and Construction for the Process Industries, Morgantown, WV: AACE
International, Latest revision.
4. AACE International, Recommended Practice 36R-08, Development
of Cost Estimate Plans - As Applied in Engineering, Procurement, and
Construction for the Process Industries, Morgantown, WV: AACE
International, Latest revision.
5. AACE International, Recommended Practice 106R-19, Development
of the Cost Estimate Basis - As Applied in Engineering, Procurement,
and Construction for the Process Industries, Morgantown, WV: AACE
International, Latest revision.
6. AACE International, Recommended Practice 106R-19, Development
of the Cost Estimate Basis - As Applied in Engineering, Procurement,
and Construction for the Process Industries, Morgantown, WV: AACE
International, Latest revision.
7. AACE International, Recommended Practice 59R-10, Development
of Factored Cost Estimates - As Applied in Engineering, Procurement,
and Construction for the Process Industries, Morgantown, WV: AACE
International, Latest revision.
8. AACE International, Recommended Practice 62R-11, Risk Assessment
- Identification and Quantitative Analysis, Morgantown, WV: AACE
International, Latest revision.
9. AACE International, Recommended Practice 63R-11, Risk Treatment,
Morgantown, WV: AACE International, Latest revision.
10. AACE International, Recommended Practice 73R-13, Establishing
Labor Productivity Norms, Morgantown, WV: AACE International,
Latest revision.
11. AACE International, Recommended Practice 31R-03, Reviewing,
Validating, and Documenting the Estimate, Morgantown, WV: AACE
International, Latest revision.
JA NUA RY/ FEBRUA RY 2 02 3 41
 12. AACE International, Recommended Practice 104R-19, Communicating
Expected Estimate Accuracy, Morgantown, WV: AACE International,
Latest revision.
13. AACE International, Recommended Practice 110R-20, Cost Estimate
Validation, Morgantown, WV: AACE International, Latest revision.
14. AACE International, Recommended Practice 114R-20, Project
Historical Database Management, Morgantown, WV: AACE
International, Latest revision.
15. Stephenson, H.L., Ed., Total Cost Management Framework: An
Integrated Approach to Portfolio, Program and Project Management,
2nd ed., Morgantown, WV: AACE International, Latest revision.
PSP Exam Review
Workshop Scheduled
JANUARY 18, 20, 25, AND 27 (4 DAYS TOTAL) || 1.2 CEUS
$650 member / $800 non-member*(*fee includes a digital copy of the PSP Study Guide)
This live, instructor-led seminar will be held virtually from 12:00
PM - 3:00 PM Eastern (US) each day. It will provide a good
review of planning and scheduling principles/techniques, and is
intended as a refresher for individuals preparing to sit for the PSP
certification exam. As such, it is not meant to replace years of
scheduling experience in the field, but rather should be considered
as a component of the preparation recommended to successfully
achieve the PSP certification. The seminar can also be helpful to
those individuals not necessarily looking to sit for the certification
exam, but rather as a good overview of planning and scheduling
that can be built upon by additional project-based experience.
LEARNING OBJECTIVES
• Understand the basic definition and use of planning
• Understand the basic definition and use of scheduling
• Understand how the concepts of planning and scheduling
should be integrated in support of effective project
management
• Understand the different scheduling techniques available and
potential uses for each
• Understand the critical path method (CPM) scheduling, its
basis, and its benefits
• Understand how to effectively communicate schedule status
and potential issues
• Understand advance scheduling techniques/considerations, to
include resource scheduling, earned value management, and
schedule compression/acceleration methods
KNOWLEDGE AREAS
Planning, scheduling, CPM scheduling, schedule communication
methods, advanced scheduling techniques/considerations
42 JAN UA RY/FE B R UA RY 20 23
ABOUT THE AUTHOR
Dave Kyle, CCP CEP, is with CE Optimize
and can be contacted by sending email to:
dave@CEOptimize.net
INSTRUCTOR
Michael R. Nosbisch, CCP PSP FAACE
Michael has over 32 years of experience in
project/program management and controls
within both the government and private
sectors. He has worked for several of the
leading engineering, procurement, and
construction (EPC) firms in the industry, to
include URS Corporation, Jacobs Engineering, Kiewit, Parsons,
and Hill International. From 2009 to 2015, Michael managed
successful EVM consulting practices for SM&A, Project Time &
Cost, and Management Solutions, LLC. He is currently Director
of Project Controls with Strategic Management Solutions LLC
(SMSI), functionally responsible for more than one hundred
project control professionals supporting Department of Energy
(DOE)/National Nuclear Security Administration (NNSA) sites and
projects across the United States.
Michael has been an active member of AACE International since
1999, earning numerous professional certifications as well as
serving as President of the Association from 2011 to 2012, and
is currently the President of the Orange County (CA) Post of the
Society of American Military Engineers (SAME). Michael is also an
adjunct professor at the University of Southern California, having
taught graduate level courses in heavy construction estimating
and scheduling.
PURCHASE TODAY!
 ENERGIZE
Your
Career
Becoming a Certified Cost Professional
is a proven way to enhance your value to
employers and clients by providing an impartial
endorsement of your knowledge and expertise.

Earning your Certified Cost Professional (CCP) Spark change, become a

Certification shows your professional commitment
and ability to your peers, supervisors, and clients,
and may give you that competitive “edge” when
CERTIFIED COST
being considered for a promotion or future
opportunity. In addition, AACE International salary
PROFESSIONAL
surveys reveal that Certified Cost Professionals For more information
earn more at all steps throughout their career than visit web.aacei.org
those without certification.