Cost Assessment Guidelines:

Best Practices for Estimating and Managing

Program Costs

Chapter 1
Characteristics of Credible Cost Estimates
 History
 GAO Review of Program Cost Estimates
– Requirements
 Cost estimates for specific systems required documentation
showing data sources, assumptions, methods, and decisions basic
to the estimates.
– Findings
 Documentation supplying required information was inaccurate or
lacking.
 Known costs had been excluded without valid justification
 Historical cost data used as a basis for computing estimates were
sometimes invalid, unreliable, or unrepresentative.
 Inflation was not always included or uniformly treated if included.
 Understanding the proper use of the estimates was hindered.
 Readily retrievable cost data that could serve as a basis of cost
estimates for new systems were generally lacking.
 Conclusion of Study

 Without realism and objectivity in the cost estimating

process, bias and over optimism creep into estimates
prepared by advocates of systems, thus the estimates tend
to be too low.
 Characteristics of Credible
Cost Estimates
Characterstic Description
Clear identification of Estimator must be provided with the system description, ground
task rules and assumptions, and technical and performance
characteristics.

The estimate’s constraints and conditions must be clearly

identified to ensure the preparation of a well-documented estimate.
Broad participation in All players should be involved in deciding mission need and
preparing estimates requirements and in defining parameters and other system
characteristics.

Data should be independently verified for accuracy, completeness,

and reliability.
Availability of valid Numerous sources of suitable, relevant, and available data should
data be used.

Relevent, historical data should be used from similar systems to

project costs of new systems. The historical data should be
directly related to the systems performance characteristics.
 More Characteristics
Characterstic Description
Standardize structure A standard WBS, as detailed as possible, should be used, refining
for the estimate it as the cost estimate matures and the system becomes more
defined.

The WBS ensures that no portions of the estimate are omitted and
makes it easier to make comparisons to similar systems and
programs.
Provision for program Uncertainties should be identified and allowance developed to
uncertainties cover the cost effect.

Known costs should be included and unknown costs should be

allowed for.
Recognition of The estimator should insure that economic changes, such as
inflation inflation, are properly and realistically reflected in the LCCE.
Recognition of All costs associated with a system should be included; if any cost
excluded costs has been excluded, it should be disclosed and given a rationale.
 … And Finally

Characterstic Description
Independent review Conducting an independent review of an estimate is critical to
of estimates establishing confidence in the estimate. The independent reviewer
should verify, modify, and correct an estimate to ensure realism,
completeness, and consistency

The estimate’s constraints and conditions must be clearly

identified to ensure the preparation of a well-documented estimate.
Revision of estimates Estimates should be updated to reflect changes in a system’s
for significant design requirements. Large changes that affect costs can
program changes significantly influence program decisions.
 A 12 Step Process for Developing
Credible Cost Estimates
Initiation & Research Assessment Analysis Presentation
Your Audience, what you Cost assessment steps The confidence in the Documentation and
are estimating, and why are iterative and can be point or range of the presentation make or
you are estimating it are of accomplished in varying estimate is crucial to break a cost estimating
the utmost importance order or concurrently. the decision maker decision outcome.

Determine Document Present

Define the Define the Conduct
the the estimate to
estimate’s program Sensitivity
estimating estimate management
purpose analysis
approach for approval

Identify Conduct a
Develop the Obtain Update the
ground risk and
estimating the data estimate to
rules and uncertainty
plan reflect actual
assumptions analysis
cost changes
Develop
the point
estimate
 Twelve Steps of a High-Quality
Cost Estimating Process
Step Description Discussed in Manual
1 Define the estimate’s purpose Chapter 5
2 Develop estimating plan Chapters 5 & 6
3 Define program characteristics Chapter 7
4 Determine estimating approach Chapter 8
5 Identify ground rules and assumptions Chapter 9
6 Obtain data Chapter 10
7 Develop point estimate Chapters 11, 12, & 15
8 Conduct sensitivity analysis Chapter 13
9 Conduct risk and uncertainty analysis Chapter 14
10 Document the estimate Chapter 16
11 Present estimate to management for approval Chapter 17
12 Update the estimate to reflect actual costs and changes Chapter 16 & 18
 Step 1 – Define the Estimate’s Purpose
 Determine
– The estimate’s purpose
– The level of detail required
– Who will receive the estimate
– The overall scope of the estimate
– See Chapter 5

Step 2 – Develop the Estimating Plan

 To include the following
– Determine the cost estimating team
– Outline the cost estimating approach
– Develop the estimate timeline
– Determine who will do the independent cost estimate
– Develop the team’s master schedule
– See Chapter 5
 Step 3 – Define Program Characteristics
 Identify in a technical baseline description document
– The program’s purpose
– Its system and performance characteristics
– Any technology implications
– All system configurations
– Program acquisition schedule
– Acquisition strategy
– Relationship to other existing systems
– Support (manpower, training, etc. ) and security needs
– Risk items
– System quantities for development, testing and production
– Deployment and maintenance plans
– Predecessor or similar legacy systems
– See Chapter 7
 Step 4 – Determine Estimating Approach

 With the following

– Define work breakdown structure (WBS) and describe each element
in a WBS dictionary; a major automated information system may have
only a cost element structure
– Choose the estimating method best suited for each WBS element
– Identify potential cross-checks for likely cost and schedule drivers
– Develop a cost estimating checklist
– See Chapter 8
Step 5 – Identify Ground Rules & Assumptions
 Clearly define what is included and excluded from the estimate.
Identify global and program specific assumptions such as
– The estimate’s base year including, including time-phasing and life cycle
– Program schedule information by phase
– Program acquisition strategy
– Any schedule or budget constraints
– Inflation assumptions
– Travel costs
– Equipment the government is to furnish
– Prime contractor and major subcontractors
– Use of existing facilities or new modifications or development
– Technology refresh cycles
– Technology assumptions and new technology to be developed
– Commonality with legacy systems and assumed heritage savings
– Effects of new ways of doing business
– See Chapter 9
 Step 6 – Obtain Data
 With the following activities
– Create a data collection plan with emphasis on collecting current and
relevant technical, programmatic, cost and risk data
– Investigate possible data sources
– Collect data and normalize them for cost accounting, inflation, learning,
and quantity adjustments
– Analyze the data to look for cost drivers, trends, and outliers; compare
results against rules of thumb and standard factors derived from historical
data
– Interview data sources and document all pertinent information, including
an assessment of data reliability and accuracy.
– Store data for future estimates
– See Chapter 10
 Step 7 – Develop Point Estimate
 With the following process
– Develop the cost model by estimating each WBS element, using the best
methodology from the data collected
– Include all estimating assumptions in the cost model
– Express costs in constant year dollars
– Time-phase the results by spreading costs in the years they are expected to
occur, based on the program schedule
– Sum the WBS elements to develop the overall point estimate
– Validate the estimate by looking for errors like double counting and
omitting costs
– Compare estimate against the independent cost estimate and examine
where and why there are differences
– Perform cross-check on cost drivers to see if results are similar
– Update the model as more data become available or as changes occur;
compare results against previous estimates
– See Chapter 11, 12, & 15
 Step 8 – Conduct Sensitivity Analysis

 With the following process

– Test the sensitivity of cost elements to changes in estimating input values
and key assumptions
– Identify effects of changing the program schedule or quantities on the
overall estimate
– On the basis of this analysis, determine which assumptions are key cost
drivers and which cost elements are affected most by changes
– See Chapter 13
Step 9 – Conduct Risk and Uncertainty Analysis
 With the following process
– Determine the level of cost, technical, and schedule risk associated with
each WBS element and discuss with technical experts
– Analyze each risk for its severity and probability of occurrence
– Develop minimum, most likely, and maximum ranges for each element of
risk
– Use an acceptable statistical analysis methodology (e.g. Monte Carlo
simulation) to develop a confidence interval around the point estimate
– Determine the type of risk distributions and reasons for their use
– Identify the confidence level of the point estimate
– Identify the amount of contingency funding and add this to the point
estimate to determine the risk-adjusted cost estimate
– Recommend that the project or program office develop a risk management
plan to track and mitigate risk
– See Chapter 14
 Step 10 – Document the Estimate
• With the following process
– Document all steps used to develop the estimate so that it can be recreated quickly
by a cost analyst unfamiliar with the program and produce the same results
– Document the purpose of the estimate, the team that prepared it, and who approved
the estimate and on what date
– Describe the program, including the schedule and technical baseline used to create
the estimate
– Present the time-phased life-cycle cost of the program
– Discuss all ground rules and assumptions
– Include auditable and traceable data sources for each cost element
– Document for all data sources how the data were normalized
– Describe in detail the estimating methodology and rationale used to derive each
WBS element’s cost (more detail rather than too little is preferred)
– Describe the results of the risk, uncertainty, and sensitivity analyses and whether
any contingency funds were identified
– Document how the estimate compares to the funding profile
– Track how this estimate compares to previous estimates, if applicable
– See Chapter 16
Step 11 – Present Estimate to Management
for Approval
– Develop a briefing that documents the life-cycle cost estimate for
management approval, including
 An explanation of the programmatic and technical baseline and any
uncertainties
 A comparison to an Independent Cost Estimate (ICE) with explanations of any
differences
 A comparison of the estimate (LCCE or ICE) to the budget
 Enough detail so the presenter can easily defend the estimate by showing how it
is accurate, complete, and high in quality
– Focus the briefing, in a logical manner, on the largest cost elements and
drivers of cost
– Make the content crisp and complete so that those who are unfamiliar with
it can easily comprehend the competence that underlies the estimate results
– Make backup slides available for more probing questions
– Act on and document feedback from management
– The cost estimating team should request acceptance of the estimate
– Chapter 17
Step 12 – Update the Estimate to Reflect
Actual Costs and Changes
– Update the estimate to
 Reflect any changes in technical or program assumptions, or
 Keep it current as the program passes through
– Replace estimates with EVM EAC and Independent Estimates at
Completion (EAC) from the integrated EVM system
– Report progress on meeting cost and schedule estimates
– Perform a post mortem and document lessons learned for elements whose
actual costs or schedules differ from the estimate
– Document all changes to the program and how they affect the cost
estimate
– See Chapters 16 & 18
 Challenges Cost Estimators Typically Face

Inexperi
en
analysts ced

Detailed Unreliab Unrealis

le tic
docume Adequa data assumpt
ntation te ion s
availabl cost Historic
e al
Well
reserve data bas cost
es Data no
defined
Risk availabl not t Unreaso
nable
e normali
Analysi Stable zed program
s baseline
conduct
ed program New s
Adequa Historic processe First-tim
te al W s e Cutting
budget data ell train integrati
e on edge
availabl experien d technolo
Obtainin
g
ce
e analysts d Program gy data
instabili Comple
ty x Diminis
technolo hing
gy technical Unrealis
tic
base projecte
d
savings
Disciplines/Concepts for Cost Assessment Team
Economics
Breakeven analysis
Foreign exchange rates
Industrial base analysis
Inflation
Labor agreements
Present value analysis
Budgeting
Defense budget appropriations
Internal company (industry)
Engineering
Design
Materials
Performance parameters
Production engineering
Program development tests
Scheduling
System integration
Cost
Analysis
Public affairs
Appropriations process
Auditors
Legislative issues/
Outside factors
Computer Science/Math
Analysis of commercial models Salesmanship
Analysis of proposals Approach
Development of CER’s Estimate
Model development Analysis
Programming
Statistics Accounting
Forecating Cost data analysis
Learning curve applications Financial analysis
Regression analysis Overhead analysis
Risk/uncertainty analysis Proposal analysis
Sensitivity analysis