Cost Estimation PDF

Training on
Cost Estimation & Analysis

Karen Richey
Jennifer Echard
Madhav Panwar
Outline
Introduction to Cost Estimating

Life Cycle Costs
Data Collection
Data Analysis
Cost Estimating Methodologies
Software Cost Modeling
Cross-checks and Validation
Risk and Sensitivity Analysis
Documentation Requirements
Cost Estimating Challenges
Cost Estimating Auditor Checklists
2

The National Estimating Society has defined Cost Estimating1as:
The art of approximating the probable cost of something based on information
available at the time.
Cost estimating cannot:

Be applied with cookbook precision, but must be tailored to a particular system,
Substitute for sound judgment, management, or control,

Produce results that are better than input data, or
Make the final decisions.
Despite these limitations, cost estimating is a powerful tool because it:

Leads to a better understanding of the problem,
Improves management insight into resource allocation problems, and
Provides an objective baseline to measure progress.
1
The NES Dictionary, National Estimating Society, July 1982

The reliability of cost estimates varies over time.
The closer you get to the actual completion of a project, the estimate
becomes more accurate.
Four types of cost estimates represent various levels of reliability .

Conceptual Estimate: Rough order of magnitude or back of the envelope.
Often inaccurate because there are too many unknowns.
Preliminary Estimate: Used to develop initial budget, more precise.

Detailed Estimate: Serves as a basis for daily project control.
Definitive Estimate: Accuracy should be within 10% of final cost.
Important to repeat estimating process (i.e., re-estimate) on a regular

basis as more information becomes available
This will keep estimate current as well as increase the accuracy
4

(contd)
All cost estimates are constructed by the following tasks:

Identifying the purpose and scope of the new system.
New software development, software reuse, COTS integration, etc.
Choosing an estimate type.

Conceptual, preliminary, detailed, or definitive type estimate
Identifying system performance and/or technical goals.

Laying out a program schedule.
Selecting a cost element structure (CES).
Collecting, evaluating, and verifying data.
Choosing, applying, cross-checking estimating methods to develop the cost
estimate.
Performing risk and sensitivity analysis
Time-phasing the cost estimate by fiscal year for cash flow purposes.
Example: 4 years to develop and 10 years operations and support beginning in FY2003
Providing full documentation.

5
Life Cycle Costs

Most cost estimates in the federal government represent total Life
Cycle Costs (LCC).
LCC estimates include all costs to develop, produce, operate, support, and
dispose of a new system.
Important to look beyond the immediate cost of developing and producing
a system and consider all costs of a systems life cycle.
What may appear to be an expensive alternative among competing systems
may be the least expensive to operate and support
Life Cycle Cost Estimates can be used to:

Compare various alternatives before committing funds to a project,
Support Estimate-to-Budget transition after time-phasing to account for
when funds will be spent.
Life Cycle Costs (contd)

A LCC estimate is summarized using a detailed cost
element structure (CES) that
Identifies the activities required to complete project development
and the effort, loading, and duration of each task,
Provides a framework against which the cost estimate is organized.
Enhances cost data collection and estimate reporting.
Facilitates comparing estimates when a standard CES is used.
Life Cycle Costs

Cost Element Structure (CES)
A CES provides a standard vocabulary for the identification and
classification of cost elements to be used in cost estimating.
Helps to identify costs that may be initially overlooked.
Should be tailored for each project.
The CES should be reviewed to ensure that there is no double counting

of costs that could be allocated to more than one element.
For example, logistics support costs could be included in the investment or
operations and support phase.
The CES is hierarchical in nature to accommodate early development

(when relatively little data is available) through deployment, when more
detailed data is available.
Life Cycle Costs

Example of CES Elements (DOD)
1.0 Investment Phase
2.0 System Operations & Support Phase
1.1 Program Management
2.1 System Management
1.2 Concept Exploration

1.3 System Development
2.2 Annual Operations (supplies/spares)

2.3 Hardware Maintenance
2.4 Software Maintenance
2.5 Outsource Provider Support
2.6 Data Maintenance
2.7 Site Operations (personnel, training,etc.)
1.3.1 System Design & Specification

1.3.2 Development Prototype and Test Site Investment
1.3.3 Software Development
1.3.4Training
1.3.6 Facilities
1.4 System Procurement

1.4.1 Deployment Hardware
1.4.2 Deployment Software
1.4.3 Initial Documentation
1.4.4Logistics Support Equipment
1.4.5 Initial Spares
1.4.6 Warranties
1.5 Outsource Provider Investment

1.6 System Initiation, Implementation & Fielding
1.7 Upgrade (Pre-Planned Product Improvement (P3I))
1.8 Disposal Costs
Data Collection
Data Required
All CES elements will need data to support the estimate
Historical cost and non-cost data need to be collected to
support various estimating techniques
Technical non-cost data describes the physical,
performance, and engineering characteristics of a system
Examples: weight, # of design drawings, SLOC, function points,
# of integrated circuit boards, square footage, etc.
Important to pick data that is a predictor of future cost
Important to have technical and schedule data because they

act as cost drivers
10
Data Collection
Data Required(contd)
Identify both direct and indirect costs
Direct costs are called touch labor and include direct manufacturing,
engineering, quality assurance, material, etc. costs which have a direct
bearing on the production of a good.
Also included are direct non-wage costs such as training, supplies, and travel.
Indirect costs are considered overhead and include such things as

general & administrative support, rent, utilities, insurance, network
charges, and fringe benefits. These expenses are typically charged to a
company as a whole.
Example: sick/annual leave, retirement pay, health insurance, etc.
11
Data Collection
Data Required (contd)
Some direct costs may be burdened with indirect costs and
some may not
Need to know to avoid double-counting or, worse yet,
underestimating
Important to ask when collecting data whether costs are burdened
with indirect costs
12
Data Collection
Data Required (contd)
Data can be collected in a variety of ways
Contractor site visits
Data requests for all relevant CES elements
Documented cost estimates, if available for earlier versions of the
current system
Can save valuable research time for statistical analysis
Published cost studies
Data collection is a critical and time consuming step in the

cost estimating process!
13
Data Collection
Typical data sources
Two types of data sources:
Primary & Secondary
Primary data is found at the original source (e.g., contractor
reports, actual program data, etc.)
Preferred source of data
Secondary data is derived from primary data of another similar

system such as documented cost estimates, cost studies/research,
proposal data, etc.
Second choice to primary data due to data gaps
May be best alternative when time constraints or data availability
limit primary data collection
14
Data Collection
Typical data sources
Current Program Milestone schedule (for phasing program costs over
time)
Current Program System Description
Important to have a technical understanding of how the system will
work
Approved program funding (Compare to proposal estimate)
Contractor actual cost reports (from internal Management Information
System)
Current program estimate documentation (if available)
Contractor proposals (Compare to program funding profile)
Commercial Off-the-Shelf (COTS) catalogs
Manufacturer websites (e.g., Dell, Microsoft, etc.)
15
Data Collection
Typical Data Sources (contd)
Forward Pricing Rate Agreements (FPRA)
Similar program historical actual costs and estimate
documentation
Engineering drawings/specifications,
Interviews with technical and program management
personnel
Surveys
Professional journals and publications
Industry guides and standards
Technical Manuals
16
Data Analysis
Data Validity/Integrity
Important to ensure that the cost data collected is
applicable to the estimate.
Identifying limitations in the historical data is imperative for
capturing uncertainty
When using historical cost data for a similar system, appropriate
adjustments need to be made to account for differences in the new
system being estimated.
Data may need to be mapped from the contractors accounting
system to the Cost Element Structure (CES)
17
Data Analysis
Data Validity/Integrity (contd)
Proposal data should be validated to ensure that contractor
motivations to buy-in or low-bid their estimates are not
occurring.
Compare previous contractor proposal bids and actual costs for
similar programs.
Look for trends in underbidding.
Participate in a fact-finding trip to discuss contractor proposal
estimates and gather supporting data/evidence.
18
Data Analysis
Normalization
Involves making adjustments to the data so that it can
account for differences in
Inflation rates,
Direct/indirect costs,
Recurring and non-recurring costs,
Production rate changes or breaks in production,
Anomalies such as strikes, major test failures, or natural disasters
causing data to fluctuate, and
Learning curve (cost improvement) effects due to efficiencies
gained from continually repeating a process
Analysis of the data may indicate the need for more

suitable data to add credibility to the estimate.
19
Data Analysis
Normalization (contd)
Accounting for Economic Changes (e.g., inflation)
Lack of cost data uniformity due to upward movement in prices
and services over time.
Index numbers are used to deflate or inflate prices to facilitate
comparison analysis.
Wrong to compare costs from 1980s to today without accounting for
inflation over the past 20 years.
Cost estimators use inflation indices to convert costs to a constant
year dollar basis to eliminate distortion that would otherwise be
caused by price-level changes.
Constant dollar estimates represent the cost of the resources required
to meet each years workload using resource prices from one
reference year (e.g., constant 2003 dollars).
Constant dollars reflect the reference year prices for all time periods
allowing analysts to determine the true cost of changes for an item
20
Data Analysis
For the United States, the Office of Management and
Budget (OMB) is responsible for developing inflation
guidance by appropriation for government estimates.
Most common indices used by United States Cost
Estimators are published by the U.S. Department of Labor,
Bureau of Labor Statistics.
Producers Price Index (http://www.bls.gov/ppi/) for goods
Employment and Earnings Index
(http://www.bls.gov/ces/home.htm#data) for services
Important to pick the appropriate market basket of goods

index that most closely matches the costs to be estimated.
21
Data Analysis
For budgetary purposes, however, data expressed in
constant dollars should be inflated to represent current year
(or Then-year) dollars.
Current year estimates calculate the cost of the resources using the
estimated prices for the year in which the resources will be
purchased.
Current year estimates reflect inflation
Necessary to express estimates in current year dollars when

requesting funding to avoid budget shortfalls.
22
Data Analysis
Example: Assume 5% escalation rate compounded annually
Base-year cost
Then-year cost
Current year
$273,100 * 1.0000 multiplier = $ 273,100
Current year + 1 $1,911,700 * 1.0500 multiplier = $ 2,007,285
Total Estimated Labor cost (budget quality)
$25,144,167
23
Cost Estimating Methodologies

Once data has been collected and normalized to constant
dollars, there are five methodologies available for
estimating costs:
Expert Opinion,
Analogy,
Parametric,
Engineering, and
Actual.
24
Estimating Methodology
Considerations
Choice of methodology is dependent upon
Type of system
Software, hardware, etc
Phase of program
Development, Production, Support
Available data
Historical data points from earlier system versions or similar system
Technical parameters of system
25
Methodology
Expert Opinion
Often called Delphi method, proposed by Dr. Barry Boehm
in his book, Software Engineering Economics.
Useful in assessing differences between past projects and
new ones for which no historical precedent exists.
26
Methodology
Expert Opinion (contd)
Pros:
Little or no historical data needed.
Suitable for new or unique projects.
Cons:
Very subjective.
Experts may introduce bias.
Larger number of experts will help to reduce bias
Qualification of experts may be questioned.
27
Methodologies
Expert Opinion - Steps
Gather a group of experts together,
Describe overall program in enough detail so experts can provide an
estimate,
Each member of the expert group then does an independent of the
resources needed,
Estimates are gathered anonymously and compared,
If there exists significant divergence among the estimates, the
estimates will be returned to the expert group,
The expert group then discusses the estimates and the divergence and
works to resolve differences, and
The expert group once again submits anonymous, independent
estimates which continues until a stable estimate results.
28
Methodology
Expert Opinion - Example
Three software engineers are renown in the ERP software
development.
You hold interviews which each explaining the requirements,
sizing level, and development process for your new system.
Each member of the group submits their opinion of the final
cost and the estimate converges to $50M.
29
Methodologies
Analogy
Estimates costs by comparing proposed programs with
similar, previously completed programs for which
historical data is available.
Actual costs of similar existing system are adjusted for complexity,
technical, or physical differences to derive new cost estimates
Analogies are used early in a program cycle when there is
insufficient actual cost data to use as a detailed approach
Compares similarities and differences
Focus is on main cost drivers.
30
Methodologies
Analogy (contd)
Often use single historical data point.
May have several analogy estimates of sub elements to
make up the total cost.
Comparison process is key to success.
May have to add or delete functionality from historical costs to
match new program
Consult technical experts for advice on complexity factors,

technical, performance or physical differences.
(not to be confused with expert opinion method)
Impact of differences on cost is subjective.

31
Methodologies
Analogy (contd)
Good choice for:
A new system that is derived from an existing subsystem.
Make sure actual cost data is available
A system where technology/programmatic assumptions have

advanced beyond any existing cost estimating relationships (CER).
Secondary methodology/cross check
Provides link between technical assumptions and cost.
32
Methodologies
Analogy (contd)
Pros:
Inexpensive
Easily changed
Based on actual experience (of the analogous system)
Cons:
Very Subjective
Large amount of uncertainty
Truly similar projects must exist and can be hard to find
Must have detailed technical knowledge of program and analogous
system
33
Methodologies
Analogy - Steps
Determine estimate needs/ground rules,

Define new system,
Breakout new system into subcomponents for analogy estimating,
Assess data availability of historical similar systems,
Collect historical system component technical and cost data,
Process/normalize data into constant year dollars (e.g., constant FY2003$),
Develop factors based on prior system,
Example: Program Management is 10% of total development cost
Develop new system component costs.

Obtain complexity (and other translation) factors
Example: Historical database is for 4 million records and new database will need
to house 24 million records => complexity factor of 6 (4*6 = 24) times the
historical database cost
Apply factors to historical costs to obtain new system costs
34
Methodologies
Analogy - Example
Your company is developing a new IT payroll system
serving 5,000 people and containing 100,000 lines of C++
code. Another company developed a similar 100,000 lines
of code system for $20M for only 2,000 people. Your
software engineers tell you that the new system is 25%
more complicated than the other system.
Other system development cost was $20M.
Estimated cost for new system = 1.25 * $20M = $25M
Plus 5,000/2,000, or 2.5 * $25M = $62.5M total cost
35
Methodologies
Parametric
Utilizes statistical techniques called Cost Estimating
Relationships (CER).
Relates a dependent variable (cost) to one or more independent
variables
Based on specific factors that have a high correlation to total cost
Number of software lines of code (SLOC) or function points,

Square feet for office floor space,
Number of floors in a high rise building for cabling estimates,
Database size, etc.
Can be used prior to development.

Typically employed at a higher CES level as details are not
known.
Most cases will require in-house development of CER.
36
Methodologies
Parametric (contd)
Pros:
Can be excellent predictors when implemented correctly

Once created, CERs are fast and simple to use
Easily changed
Useful early on in a program
Objective
Cons:
Often lack of data on software intensive systems for statistically
significant CER
Does not provide access to subtle changes
Top level; lower level may be not visible
Need to be properly validated and relevant to system
37
Methodologies
Parametric Example
You have a previously developed CER to estimate a new IT system based on
SLOC.
Cost = SLOC * 25 $/SLOC
The CER is based on systems ranging from 1,000,000 to 3,000,000 SLOC.
You have estimated 2,600,000 SLOC for new system
Cost = 2,600,000 * $25 = $65M
38
Methodologies
Parametric (contd)
Cost estimators can develop their own CERs or they can use
existing commercial cost models.
Various Software cost estimating models will be discussed next
Learning curves specialized type of CER.

CERs can be cost to cost or cost to non-cost.
Cost to Cost: e.g., Manufacturing costs are 1.5 times Quality Assurance costs
Cost to Non-Cost: $/pound, or engineering hours/# of engineering drawings
yields hours/drawing metric that can be applied to new program
Factors and ratios are also examples of parametric estimating.
39
Methodologies
Parametric CERs
CERs are defined as a technique used to estimate a
particular cost by using an established relationship with an
independent variable.
Can be as simple as a one variable ratio to a multi-variable
equation.
CERs use quantitative techniques to develop a
mathematical relationship between an independent variable
and a specific cost.
40
Methodologies
Parametric CERs (contd)
Reliable, normalized data is most important for CER development.

Must determine range of data for which the CER is valid.
Useful at any stage in a program.
Typically CERs are the main cost estimating methodology in early stages of a program.
Must be logically sound as well as statistically sound.
In later stages of a program, CERs serve as a cross check to other methods

High correlation (r2 = 0.75 or higher) for goodness of fit test
Different statistical techniques may be used to judge the quality of the CER.
Least squares best fit (regression analysis, or the ability to predict one variable on the basis of
the knowledge of another variable)
Multiple regression (a change in the dependent variable can be explained by more than one
independent variable)
Curvilinear regression (relationship between dependent and independent variable is not liner,
but based on a curve)
Learning curve (describe how costs decrease as the quantity of an item increases)
41
Methodologies
Parametric CERs (contd)
Statistics may be used to evaluate how well the CER will produce a reliable
estimate.
Coefficient of determination, R2:
Percent of the variation in the Y-data explained by the X-data, (ie. How close the points
are to the line)
Standard Error, SE:

Average estimating error when using the equation as the estimating rule
Coefficient of Variation, CV:

SE divided by mean of the Y-data, relative measure of estimating error
t-stat
Tests whether the individual X-variable(s) is/are valid
F-stat
Tests whether the entire equation, as a whole, is valid
No single statistic may validate or invalidate a CER, quality of the input data is
just as important.
Best to use a statistical software package like SAS or SPSS to quickly evaluate
alternative CERs.
42
Methodologies
Parametric CERs - Steps
Define the dependent variable (e.g., cost dollars, hours, etc.) and what the CER will
estimate,
Select independent variables to be tested for developing estimates of the dependent
variable,
Collect data concerning the relationship between the dependent and independent
variables,
Use statistical analysis to judge strength of relationship and validity of equation
Select the relationship that best predicts the dependent variable, and
Most difficult and time consuming step, but essential that all data is checked to ensure that all
observations are relevant, comparable, and free of unusual costs
Explore the relationship between the dependent and independent variables,
Variables should be quantitatively measurable and available

If there is a choice between developing a CER based on performance or physical
characteristics, performance characteristics are generally the better choice because they are
known early on
A high correlation often indicates that the independent variable will be a good predictive tool
Document your findings.
Identify independent variables tested, data gathered along with sources, time period
(normalization for inflation effects), and any adjustments made to the data
43
Methodologies
Parametric Learning Curves
Basic premise:
Repetitive tasks should result in productivity for subsequent, similar
tasks
This improvement is usually quantified at a rate Y = AXb
In simplest terms, the cost of manufacturing or installing a

unit should decrease as the number of units involved
increases.
As the number of units produced doubles, the cost per unit decreases
by a fixed percentage
The concept of learning curves is not new, originated in the

mid-1930s with T.P. Wright in the Journal of Aeronautical
Sciences.
44
Methodologies
Parametric Learning Curve - Example
Say that the first 100 tasks of an installation took 10 hours
per task and the next 100 averaged 8 hours per task. Thus,
the learning curve would be calculated as follows:
Learning curve = 8 hours per task/10 hours per task = 0.8
Implies an 80% learning curve meaning an improvement of
20% occurred between the first 100 tasks and next 200
tasks
45
Methodologies
Rate Impact
Basic premise: The number of units produced in a single lot effects the
overall cost of producing that lot.
Costco theory that buying in bulk makes the unit cost less
Mathematical equation showing rate impact along with learning curve

Y = AXbQr
Both rate and learning curves can be impacted by the following:

Operator turnover rate (new employees do not meet expected productivity
standards immediately)
Production reworks
Material handling and downtime (learning curves assume material is ready
when needed)
Engineering rework
Rework of vendor supplied parts
46
Methodologies
Engineering
Also referred to as bottoms up or detailed method.
Underlying assumption is that future costs for a system can be
predicted with a great deal of accuracy from historical costs of that
system.
Involves examining separate work segments in detail and then
synthesizing these detailed estimates along with any integration
costs into a total program estimate.
Estimate is built up from the lowest level of system costs.

Uses detailed cost element structure (CES).
Must include all components and functions.
Can be used during development and production.
47
Methodologies
Engineering (contd)
Most useful for systems in production.
Design configuration has stabilized
Test results are available
Development cost actuals are available
Typically broken into functional labor categories e.g.

engineering, manufacturing, quality control, etc.
48
Methodologies
Engineering
Pros:
Objective
Reduced uncertainty
Cons:
Expensive
Time Consuming
Not useful early on
May leave out software integration efforts
49
Methodologies
Engineering Steps
Understand program requirements,

Prepare program baseline definition,
Define ground rules and assumptions,
Develop detailed cost element structure,
Develop functional estimates, and
Use other program history

Compile estimate data
Develop rates and factors
Incorporate supplier/subcontractor prices
Include integration costs to glue the separate components into an
integrated system (may need to use a CER for this estimate)
Summarize estimate.
50
Methodologies
Engineering Example
New IT system can be broken down into Labor and Material
costs.
Labor: Estimated number of hours
250,000 hrs * $200/hr = $50M
Material: From vendor quotes
Server = $10M
COTS = $3M
Desktops (50 @ $100k) = $5M
Cost = $50M+$10M+$3M+$5M = $68M
51
Methodologies
Actual
Bases future costs on recent historical costs of same
system.
Used later in development or production.
Preferred method.
Costs are calibrated to actual development or production
productivity for your organization
52
Methodologies
Actual
Pros:
Most accurate
Most objective of the five methodologies
Cons:
Data not available early on
Time consuming
Labor intensive to collect all the data necessary
53
Methodologies
Actual - Example
The development process is nearing completion. The material
has all been procured at a cost of $20M.
The labor cost to date is $30M. According to earned value
cost performance reports (CPRs) the estimate at complete for
the remainder of the labor is another $20M.
Cost = $20M + $30M + $20M = $70M
54
Methodologies
Summary of the Five Cost Estimate Results
Expert - $50M
Analogy - $62M
Parametric $65M
Engineering $68M
Actual - $70M
55
Software Cost Models

Software costs as a percentage of total system costs continues to
increase while associated hardware costs decrease.
Accurately capturing software costs can be difficult and cost
overruns often occur as a result of software requirements being
difficult to estimate and track.
Software estimating problems occur most often because of the:
Inability to accurately size a software project,
Inability to accurately specify a software development and support
environment,
Improper assessment of staffing levels and skills, and
Lack of well-defined requirements for the specific software activity being
estimated
56
Software Cost Models (contd)

The requirement to develop cost estimates for systems at a time
when few details are known has led to the development of cost
models.
The need to generate estimates quickly and for many alternatives can
make the analogy and engineering methods impractical due to lack of
detailed requirements.
In the early stages of software development, few details other than
performance requirements and some physical constraints are available.
Software lines of code (SLOC) are often used to estimate

software size which is the most significant driver in determining
software costs.
Programming languages also have a significant effect on overall cost.
57

Software cost models are based on statistically derived cost
estimating relationships (CERs) and various estimating
methodologies used to predict the cost of a system.
Models approximate the real world through a combination of
Statistical analysis of historical data,
Informal/Intuitive analysis of rules of thumb based on experience,
Standard procedures such as learning curves, inflation computation, and
labor rate/overhead applications.
Models may contain the following features:

Support costs calculations, time-phasing to spread development costs by
fiscal year, & inflation calculations to convert base year to then-year dollars.
58

Regardless of how software is programmed, it must proceed through
certain steps or phases and must be supported after development.
The combination of software development and support is known as
the software life cycle.
The Cost Element Structure (CES) discussed earlier accounts for the
software life cycle using the following sub-elements for CES 1.3
Software Development:
Requirements Definition
Analysis, Design, and Integration
Quality Assurance Program
Configuration Management
Software Design and Development
HW/SW Integration, Assembly, Test and Checkout
System Operational Test and Evaluation
System Independent Software Verification and Validation
Site Acceptance Testing
Independent Operational Test and Evaluation
Program Management
Installation and Checkout
Corrective Maintenance
59

Examples of Commercially Available Models
COCOMO
COSTXPERT
SLIM
SEER
Costar, REVIC, etc.
60

COCOMO
Constructive Cost Model (COCOMO) is one of the earliest
cost models widely used by the cost estimating
community.
COCOMO was originally published in Software
Engineering Economics by Dr. Barry Boehm in 1981.
COCOMO is a regression-based model that considers
various historical programs software size and multipliers.
61

COCOMO
COCOMOs most fundamental calculation is the use of the
Effort Equation to estimate the number of Person-Months
required to develop a project.
Example: # of person months * loaded labor rate = Estimated Cost
Most of the other estimates (requirements, maintenance,

etc) are derived from this quantity.
62

COCOMO
COCOMO requires as input the projects estimated size in
Source Lines of Code (SLOC).
In addition to SLOC, COCOMO has scale factors which
allow you to tailor conditions to your project.
Scale factors are used to calibrate or adjust the model that is based
on data which is not necessarily representative of the system being
estimated
Intent of calibrating is to allow for general application of a model
developed from a specific database
Based on the assumption that any differences between the predicted
cost and the historical cost not explained by the model are due to:
Differences in the type of systems and acquisition environment
represented in the two data sets
63

COCOMO
Scale factors include:
Development flexibility, architecture, risk, team cohesion, and
process maturity among others.
Additionally COCOMO uses Cost Drivers grouped in

broad categories such as personnel, product, platform,
project etc.
Product Cost drivers include:
required reliability and product complexity.
Personnel cost drivers include:

language, platform, and tool experience
64

COCOMO
COCOMO defaults to a nominal value for all factors when
model is first used.
Cost estimators must calibrate the model to the program being
estimated to their specific development environment and
productivity level of staff.
COCOMO uses analogy and parametric methods based pm

a historical, proprietary database of data submitted by
consortium members.
Allows COCOMO to be compatible with current design methods
and evolving higher order languages.
65

COCOMO
The model makes its estimates of required effort
(measured in Person-Months) based primarily on the
projects estimate of the software size (as measured in
thousands of SLOC, KSLOC)):
Effort = 2.94 * EAF * (KSLOC)**E
Where EAF is the Effort Adjustment Factor derived from

the Cost Drivers and E is an exponent derived from the
five Scale Drivers.
66

COCOMO
For example assuming a project with all nominal cost and
scale drivers, EAF would have a value of 1.00 and exponent,
E, of 1.0997.
Assuming that the project consists of 15,000 source lines of
code, COCOMO estimates that 57.77 Person-Months of effort
is required to complete it.
Effort = 2.94 * (1.0) * (15)**1.0997 = 57.77 Person-Months.
Person-months multiplied by a loaded labor rate (includes direct and
overhead costs) would yield the cost estimate:
57.77 person-months * $15K per person-month = $866,000
67

COCOMO Model output example
68

COCOMO
Example of how model
defaults to nominal
scale factors.
These should be
adjusted for every
estimate to approximate
the new, unique
development effort.
69

COCOMO outputs by phase
70

COCOMO outputs by phase
71

COCOMO overall estimate by phase
72

COCOMO Summary
COCOMO provides the following information:
Software development, maintenance cost and schedule estimates.
Cost, schedule distribution by phase, activity, and increment.
A free updated version of COCOMO is available at:

http://sunset.usc.edu/research/COCOMOII/
73
COCOMO
COSTXPERT
SLIM
SEER
REVIC
Costar, Price S, etc.
74

CostXpert
CostXpert builds on COCOMO by having a database of
actual projects that is constantly being updated for various
types of systems (commercial, military, etc.).
When you enter project specific information, CostXpert
maps it to the closest projects in its database and gives you
results tailored to your project.
CostXpert uses both parametric and analogy estimating methods.
CostXpert can accommodate a variety of software sizing

elements such as SLOC, function points, object metrics,
GUI metrics, and feature points.
75

CostXpert
CostXpert also allows for customization specifically to your work
environment.
In addition, CostXpert creates a detailed cost element structure (CES),
conducts what if scenarios, risk analysis, and provides a wide variety of
reports to include:
Cost allocations by task list,

Labor loading by phase and functional categories,
Maintenance and life-cycle costs,
Software defects estimates, and
Documentation deliverables
GAO uses CostXpert during various cost analysis assessments to

check the reasonableness of both the software cost estimate and
accompanying schedule for delivery.
Evaluation copies can be downloaded at
http://www.costxpert.com/products/downloads.html
76

CostXpert example of input screen
77
COCOMO
COSTXPERT
SLIM
SEER
REVIC
78

SLIM
The Software Life Cycle Model (SLIM) is marketed by
Quantitative Software (QSM).
Set of tools include SLIM-Estimator, SLIM-Data Manager,
SLIM-Control, SLIM-Metrics and SLIM-Master Plan.
Input metrics include SLOC, Function Points, Objects, Windows,
Screens, or Diagrams
Historical productivity and complexity are also inputs to the model
Quick start wizard enable rapid generation of estimates and

details which can then be tailored for the specific project.
79

SLIM
Results are presented at the 50% probability and can be
adjusted by using sliders for various parameters.
Estimates can readily be compared to historical data and
model allows for analysis of what if scenarios.
A Productivity Index (PI) is calculated for your project and
updates as new data is available.
80

SLIM
As described in the users manual, the PI is a macro measure of the
total development environment. It incorporates many factors in
software development, including:
Management influence, development methods, tool, techniques, skill and
experience.
Values for PI range from .1 to 40.

Low values generally are associated with poor environments, process, tools and
complex systems.
High values are associated with mature processes, and capable management and
well-understood, straightforward projects.
Outputs of the model include: project description, estimation analysis

views, schedule, risk analysis, staffing and skill breakout, effort and
cost, reliability estimates, and documentation sections.
Demo copies of SLIM can be downloaded at http://www.qsm.com/ .
81
COCOMO
COSTXPERT
SLIM
SEER
REVIC
82

SEER
This set of tools provides both software and hardware
estimation capabilities.
Software Project Control
Software Estimation, Planning and Project Control
Hardware Project Control
Hardware Estimation, Planning, Project Control and LifeCycle Cost Analysis, Including
Operations and Support
Design for Manufacturability
Cost Designer for Parts, Process and Assembly
83

SEER
The toolset includes:
SEER-SEM for evaluating software development.
SEER-H for estimating hardware and hardware Operations and Support
costs.
SEER-DFM for manufacturability analysis.
SEER-IC for estimating foundry costs.
SEER-SSM for software sizing.
Outputs of the model include

Labor estimates by function,
A quick estimate providing a snapshot of size, effort, and schedule, and
Staffing by month, cost by activity, person-months by activity, delivered
defects, and SEI maturity rating.
Demo copies of SER-SEM and other tools can be downloaded at

http://www.galorath.com/tools_overvw.shtm
84
COCOMO
COSTXPERT
SLIM
SEER
REVIC
85

REVIC
The Revised Enhanced Version of Intermediate COCOMO (REVIC)
model was developed by Mr. Raymond L. Kile formerly of Hughes
Aerospace.
The main difference between REVIC and COCOMO is the
coefficients used in the effort equations. REVIC changed the
coefficients based on a database of recently completed DOD projects.
REVIC's schedule estimates are often considered lengthy because it
assumes that a project's documentation and reviews comply with the
full requirements of DOD-STD-2167A/498.
86
COCOMO
COSTXPERT
SLIM
SEER
REVIC
87

There are many other models available.

Most are based on COCOMO.
Almost all need to be calibrated with project specific data.
Using multiple models and comparing results provides
better confidence.
Crosschecking estimates leads to better cost estimate validation
The International Council on Systems Engineering

(INCOSE) has a list of software tools that may be of use.
http://www.incose.org/tools/tooltax/costest_tools.html
88

After an estimate has been created, the next step involves validating the
estimate by cross-checking.
Cross-checking means using a different approach to create the estimate.
If both estimates are close, the target estimate has some validity.
If both estimates are very different.
This increases the level of uncertainty which must be reflected in a risk analysis.
This may lead to another estimating method to increase cost estimate confidence.
It is a good practice to cross-check major cost drivers.

If time is available, cross-checking other cost elements can further validate
the entire estimate.
Factors tend to be the most common approach for top level checks.
Software cost models play an important role during later stages of
development phase.
Can be used to cross-check the reasonableness of costs forecasted using
either the engineering or actual cost methods.
89

(contd)
Validation is the process of demonstrating that either a
CER or a cost model accurately predicts past history.
Validation also includes a demonstration that:
The data relationships are logical,

The data used are credible,
Strong data relationships exist, and
The CER or cost model is a good predictor of costs.
This can be done using independent test data (not included in models
calibration).
The models cost output would then be compared to the test points
known value to provide an accuracy benchmark.
90

(contd)
Validation also includes a demonstration that:
Model users have sufficient experience and training,

Calibration processes are thoroughly documented,
Formal estimating policies and procedures are established, and
When applicable, information system controls are maintained to
ensure the integrity of the models being used.
91

Risk Analysis Definition
Risk analysis is a process that uses qualitative and quantitative
techniques for analyzing, quantifying and reducing uncertainty
associated with cost goals.
By nature, all cost estimates have some uncertainty.
Earlier in development that uncertainty is higher.
As the project matures these uncertainties decrease due to greater design
definition, actual experience, and less opportunity for change.
Errors can also occur from historical data inconsistencies.

Cost risk analysis aims to achieve the following objectives:
Identify program level confidence for development schedule,
Provide credibility to the target estimate, and
Identify technical, schedule, and cost estimating risk drivers for use in risk
management.
92

Risk Analysis Definition (contd)
Risk is defined as a situation in which the outcome is
subject to an uncontrollable random event stemming from
a known probability distribution.
Roll of two dice is an example since the roll can result in one of 11
possible outcomes.
Uncertainty is defined as a situation in which the outcome

is subject to an uncontrollable random event stemming
from an unknown probability distribution.
Example would be will it rain two weeks from today?
Cost estimating falls more into the range of uncertainty

than risk, but most managers use the term risk analysis.
93

Sensitivity Analysis Definition (contd)
Sensitivity analysis answers the question:
What happens if the assumptions change?
Changes to some assumptions can have profound impact, while huge
changes to other assumptions have little effect on results.
Sensitivity examines the effect of primary cost estimating outputs to
changes on individual CES inputs.
Sensitivity analysis highlights the factors that have the

strongest impact on the overall cost estimate.
Points to management which factors deserve the most attention.
Narrows down the number of lower level cost elements that should
be examined using risk analysis techniques.
94

Sources of Cost Risks
Schedule and Technical risks
Unexpected design changes

Project team experience
Number of business units impacted
Requirements changes
Integration considerations
Technical difficulties or maturity issues
Revised project or acquisition plans
Quantity changes
New labor rates
Higher inflation
Cost estimating risks

Imprecision associated with the estimating techniques used, errors, or
oversights
95

Risk Analysis Techniques
Risk factor approach
Adds cost to the overall point estimate (most likely estimate) to
cover risks by way of a factor.
This factor is usually a percentage derived from past data and
experience.
Often applied to the estimate as a whole versus lower level cost
elements.
Monte Carlo Simulation

Automatically analyzes the effect of varying inputs on outputs of a
cost model using spreadsheet risk analysis.
Randomly generates values for uncertain variables over and over to
simulate a model.
96

Risk Analysis Techniques (contd)
Monte Carlo Simulation Steps
Analyst obtains cost distribution for each element identified as a major
cost driver either through experience or sensitivity analysis.
Cost distribution is usually triangular in the form of optimistic, most likely,
and pessimistic cost estimates.
These cost ranges are usually obtained through expert judgment (engineer or
technical specialist interviews) or using a Delphi technique
After all cost elements have been identified by a distribution, the

simulation is run many times (1,000 10,000 times).
Simulation calculates multiple scenarios of the cost model by repeatedly
sampling values from the probability distributions assigned to the various cost
elements.
While the simulation runs, the forecasts stabilize towards a smooth frequency
distribution called a cumulative frequency distribution or CDF
After thousands of trials, statistics of the results and the certainty of any
outcome can be obtained from the CDF.
97

Risk Analysis Techniques (contd)
Monte Carlo Simulation Results
Reveal not only the result values for each forecast, but also the
probability of any value occurring.
This is helpful to management because it can show the level of
certainty of achieving a cost objective.
For example, a simulation can show there is a 10% chance of the project
finishing for $50 million, a 50% chance of it costing $ 70 million, and a
90% chance of developing the project for $100 million or less.
Decision makers can use these results to decide which projects will
continue funding based on quantifiable risk parameters.
98

Types of Risk Simulation Tools
US GAO uses Crystal Ball to perform Monte Carlo
Simulations
Crystal Ball is an simulation program that helps you
analyze the risks and uncertainties associated with
Microsoft Excel spreadsheet based models.
http://www.decisioneering.com/crystal_ball/
Other tools are available, such as

@Risk (http://www.palisade.com/html/risk.html)
RiskEase (http://www.riskease.com/)
Pertmaster (http://www.pertmaster.com/)
99
Documentation
After cross-checking the estimate major cost drivers, the next step, and
most important one of all, involves documenting the entire estimate
process.
Although this is a difficult and time-consuming procedure, the level of
detail and attention will pay big dividends when it comes time to reestimate
May also help data collection of actual analogous costs.
Important to document the estimate as it is being developed (i.e.,

document as you go).
Hard to remember rationale and judgments for adjusting data months later
when it needs to be documented.
Documenting as you prepare the estimate leads to a better quality estimate
and requires minimum effort at the end.
100
Documentation (contd)
Best to provide more vs. less information
Assume reader knows nothing about the system being
estimated.
Include step-by-step instructions for how the estimate
was developed.
Aim to provide enough information for the estimate to be recreated by
a cost analyst independent of the team.
Most users of the documentation will either be updating the estimate
at a later date or will relay on data for estimating an analogous
system.
101
Documentation should include at a minimum:
Introduction
Purpose of estimate
Estimate team members, POC information, and area of responsibility
Program background and system description
Program schedule (necessary for phasing costs over time)
Scope of estimate (including what was omitted and why)
Ground rules and assumptions (all technical and program specific
assumptions that bound the estimate e.g., constant 2003 dollars using xx
inflation rates)
Summary of estimate by year to show phasing by CES
Overview of main body that describes how each CES element was
estimated
102
Summary of estimate by year showing phasing by CES element
(In constant 2002 millions of dollars)
Investment Costs
Systems Eng/Program Management
Development/Installation/Training
Initial Biometric Hardware Costs
Initial Biometric Software Costs
Network Infrastructure
Consulate Facility Renovation
Hardware Infrastructure Upgrades
Total Investment Costs
FY2002
$
$
$
$
$
$
$
$
540
7,900
6,045
4,600
100
570
19,755
FY2003
$
$
2,523
2,523
$
$
2,523
2,523
$
$
2,523
2,523
$
$
2,523
2,523
FY2003
540
926
6,400
100
50,715
10,540
1,000
152
70,373
$
$
$
$
$
$
$
$
$
$
FY2004
540
926
6,400
100
50,715
10,540
1,000
152
70,373
$
$
$
$
$
$
$
$
$
$
FY2005
540
926
6,400
100
50,715
10,540
1,000
152
70,373
$
$
$
$
$
$
$
$
$
$
FY2006
540
926
6,400
100
50,715
10,540
1,000
152
70,373
$
$
$
$
$
$
$
$
$
$
FY2007
540
926
6,400
100
50,715
10,540
1,000
152
70,373
$
$
$
$
$
$
$
$
$
$
FY2008
540
926
6,400
100
50,715
10,540
1,000
152
70,373
$
$
$
$
$
$
$
$
$
$
FY2009
540
926
6,400
100
50,715
10,540
1,000
152
70,373
$
$
$
$
$
$
$
$
$
1,619.44
2,779
19,200
300
152,145
31,619
3,000
456
211,118
72,896
72,896
72,896
72,896
72,896
72,896
72,896
218,687
33,075
Total Life Cycle Costs
52,830
INS Operating Personnel

Communications Costs
Recurring Training
Consulate Facility Maintenance
FY20010-2012
2,523
2,523
Visa Operating Personnel
FY2009
$
$
Total Operations & Support Costs
Network Infrastructure Maint.
FY2008
2,523
2,523
33,075
Software/System Maintenance
FY2007
$
$
Biometric Hardware Maintenance
FY2006
2,523
2,523
$
$
$
$
$
$
$
$
$
$
Program Management
FY2005
$
$
FY2002
Operations & Support
FY2004
$
$
7,569
7,569
Total
$
$
$
$
$
$
$
$
540
7,900
6,045
4,600
100
570
25,229
44,983
$
$
$
$
$
$
$
$
$
$
Total
5,398
9,263
64,000
1,000
540,225
105,396
10,000
1,520
736,802
781,785
FY20010-2012
103
Main Body documentation should include:
Enough detail for each CES element that would allow for another analyst
to replicate the cost estimate.
Each CES element should provide a description of the methods (with rationale
for choosing it), techniques, and data used to generate the estimate as well as
any cross-checks done to add rigor to the estimate.
Include information such as data sources, direct/indirect labor rates, labor
hours, material/subcontracts, learning curve data and methodology, factors,
cost estimating relationships with corresponding statistics and data ranges,
cost models used, inflation indices, time phasing (e.g., development lasts 3
years and 10 years operations and support), estimator judgments, cross-checks,
risk and confidence levels.
Documentation flow should follow the same CES structure shown in the
estimate summary by year.
If the estimate is an update to a prior estimate, then provide an explanation
for any differences.
104
Be sure to spell out all acronyms when first introduced.
Cross-reference the reader to where data is used multiple times.
Show data at the lowest levels possible.
Include copies of vendor quotes, studies used, statistical analysis printouts,
cost model input and output reports, etc.
Include disclaimers to show the level of risk and uncertainty

surrounding estimate.
Try to quantify the uncertainty by using a simulation model that will
express the summary estimate in terms of level of confidence:
Estimate is presented at the 90% confidence level, or
Point estimate of $1 million is bounded by a range of $750,000 to $1,250,000
105
Cost Estimating Challenges
Access to historical data
Need to invest in database capture of historical costs and technical data for proper CER
development
Validity and uncertainty of data
Garbage in = Garbage out
Limited time to develop estimates
Costly, time consuming, and usually not funded

Development costs for IT systems can quickly become outdated by new programming languages
Maintenance costs are even more difficult to capture because they are seen as on-going support or
overhead and not as metrics
System architecture change effects on cost estimates can be hard to determine
Can result in rough-order magnitude costs being used as budget quality estimates
Cause important steps like validation and Monte Carlo simulation to be omitted
Resources
Lack of trained people is a problem
106
Cost Estimating Auditor Checklists

Various auditor checklists can be found in the many source documents
used to create this briefing. A few of these are shown below:
The Software Engineering Institute (SEI) has created "A Manager's
Checklist for Validating Software Cost and Schedule Estimates."
This checklist is designed to help managers access the credibility of software cost and
schedule estimates.
It identifies seven issues to address and questions to ask when determining your
willingness to accept and use a software estimate.
Each question is associated with elements of evidence that, if present, support the
credibility of the estimate.
http://www.sei.cmu.edu/publications/documents/95.reports/95.sr.004.html
Joint Industry/Government Parametric Estimating Handbook (Second

Edition, Spring 1999)
http://www.jsc.nasa.gov/bu2/PCEHHTML/pceh.htm
Chapter 6, Auditing and Evaluating a CER & Auditing Parametric Estimates
Appendix G, Parametric Estimating System Checklist
107
Cost Estimating
Auditor Checklists (contd)
Department of the Army Economic Analysis Manual (U.S. Army Cost and Economic
Analysis Center, Feb 2001)
http://www.asafm.army.mil/pubs/cdfs/manual/economic.pdf
Appendix M, Economic Analysis Checklist
Department of the Army Cost Analysis Manual (U.S. Army Cost and Economic
Analysis Center, May 2002)
http://www.ceac.army.mil/pubs/default.asp
Link to a zipped MS Word file for downloading or printing
Checklist on page 42, Figure 3-4 Validation Considerations
TASC (The Analytic Sciences Corp.), The AFSC Cost Estimating Handbook, Reading,
MA, prepared for the U.S. AirForce Systems Command, 1986.
Chapter 8, Cost Models pages 8-6 through 8-12
Appendix D, Staff Review Cost Estimate Checklist pages D-1 through D-8
108
Cost Estimating
Auditor Checklists (contd)
Navy Post Graduate School Economic Analysis Handbook
http://www.nps.navy.mil/drmi/chapter6.htm
Chapter 6, Guide for Reviewers
National Research Council Canada (NRC-CNRC) Software Cost Estimation

and Control (1994)
http://seg.iit.nrc.ca/English/abstracts/NRC37116.html
Appendix A, pages 59-67, Software Cost Estimation Questionnaire
109
Contact Information
For further information please contact us:
Karen Richey (202) 512 4784 or richeyk@gao.gov
Madhav Panwar (202) 512- 6228 or panwarm@gao.gov
Jennifer Echard (202) 512 3875 or echardj@gao.gov
110

Cost Estimation PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cost Estimation PDF

Uploaded by

Copyright:

Available Formats

Training on

Cost Estimation & Analysis

Introduction to Cost Estimating

Introduction to Cost Estimating

Cost estimating cannot:

Substitute for sound judgment, management, or control,

Despite these limitations, cost estimating is a powerful tool because it:

The NES Dictionary, National Estimating Society, July 1982

Introduction to Cost Estimating

Four types of cost estimates represent various levels of reliability .

Preliminary Estimate: Used to develop initial budget, more precise.

Important to repeat estimating process (i.e., re-estimate) on a regular

Introduction to Cost Estimating

All cost estimates are constructed by the following tasks:

Choosing an estimate type.

Identifying system performance and/or technical goals.

Providing full documentation.

Life Cycle Costs

Life Cycle Cost Estimates can be used to:

Life Cycle Costs (contd)

Life Cycle Costs

The CES should be reviewed to ensure that there is no double counting

The CES is hierarchical in nature to accommodate early development

Life Cycle Costs

2.0 System Operations & Support Phase

1.1 Program Management

2.1 System Management

1.2 Concept Exploration

2.2 Annual Operations (supplies/spares)

1.3.1 System Design & Specification

1.4 System Procurement

1.5 Outsource Provider Investment

Important to have technical and schedule data because they

Indirect costs are considered overhead and include such things as

Published cost studies

Data collection is a critical and time consuming step in the

Secondary data is derived from primary data of another similar

Analysis of the data may indicate the need for more

Important to pick the appropriate market basket of goods

Necessary to express estimates in current year dollars when

Cost Estimating Methodologies

Qualification of experts may be questioned.

Focus is on main cost drivers.

Consult technical experts for advice on complexity factors,

Impact of differences on cost is subjective.

A system where technology/programmatic assumptions have

Provides link between technical assumptions and cost.

Determine estimate needs/ground rules,

Develop new system component costs.

Number of software lines of code (SLOC) or function points,

Can be used prior to development.

Can be excellent predictors when implemented correctly

Learning curves specialized type of CER.

Factors and ratios are also examples of parametric estimating.

Reliable, normalized data is most important for CER development.

Must be logically sound as well as statistically sound.

In later stages of a program, CERs serve as a cross check to other methods

Standard Error, SE:

Coefficient of Variation, CV:

Use statistical analysis to judge strength of relationship and validity of equation

Explore the relationship between the dependent and independent variables,

Variables should be quantitatively measurable and available

Document your findings.