You are on page 1of 26

Integrity - Service - Excellence

Whats New In Air Force Source Selection


Session #711

Edward C. Martin & Daniel C. Fulmer ASC Acquisition Center of Excellence (937) 255-5423 or (937) 255-8584 Edward.Martin2@wpafb.af.mil or Daniel.Fulmer@wpafb.af.mil

Overview
Background and Basis of the Changes Effectivity Description of the changes
Factors and Ratings Definitions Planning Use of Non-Government Personnel Notification Requirement Other

Source Selection Training Team

Defense Acquisition Performance Assessment (DAPA) Report


Guiding Principles Adopt a risk-based source selection process. For development contracts, proposal cost should be replaced by industry and government agreement on a high confidence cost estimate.

Base SSA competitive range determinations upon high confidence costs which are considered affordable.
Create acquisition strategies that reflect restructuring source selection competitions for ACAT I and II programs to significantly shorten their length and base their results on system risk and management performance instead of cost.

DAPA Report Source Selection Recommendations


1. 2. 3. Create an environment of open communication to insure that industry understands government requirements and government understands industry capabilities and limitations. Include industry in development of program acquisition strategies for each phase of the process, and the acquisition and source selection plans for each competitive source selection. Ensure traceability of requirements from program to the acquisition strategy to the acquisition plan, to the instruction to offerors, to the evaluation factors for award, to the contract incentive provisions and program control and to the performance evaluation metric selection. Standardize the content of the Concept Development and Demonstration phase competitive prototype contracts to include conducting initial baseline review and preliminary design review for the contractors proposed system design and development program. Eliminate the requirement to share all questions or information submitted and eliminate answers provided to a single competitor with all competitors prior to issuance of the final request for proposal. Incorporate existing scheduled contractor technical or program reviews as proposal elements, to the maximum extent possible.

4.

5. 6.

DAPA Report Source Selection Recommendations


7. 8. Require oral presentations of proposals during source selection and encourage open exchanges between evaluators and industry, not limited to clarification only, during these presentations. Use an affordability assessment based upon industry and government-agreed high confidence cost as the principal factor in competitive range determination during source selection. Once a competitors government and industry agreed-to development cost is determined to be affordable, and thus the competitor is determined to be within the competitive range, no other consideration will be given to the development cost, other than cost realism, during subsequent competitive source selection evaluations for Concept Development and Demonstration and System Design and development contracts. Stress the critical nature of risk mitigation and completeness of data supporting offerors claims as a heavily weighted evaluation factor for award. Establish performance and schedule confidence as well as management confidence including subcontractor selection and management as primary evaluation factors for award of Concept Development and Demonstration and System Design and Development contracts. Set target cost for cost-type concept development and system design and development contracts at the Cost Analysis Improvement Group estimate, identifying the difference between proposed and target cost as management reserve, aggressively incentivizing cost performance, and penalizing cost growth over target.

9. 10.

11.

Basis of the Source Selection Changes


Based on an Air Force IPT Source Selection Improvement Team Study
Team incorporated the Defense Acquisition Performance Assessment (DAPA) Jan 2006 Report recommendations, as appropriate SAF/AQC sent an e-mail to all AF acquisition organizations
Requested to identify improvements beyond those in DAPA Received 158 suggested changes and recommended 91 for implementation

Categories
- Evaluation Factors
- Source Selection Advisory Council
- Source Selection Evaluation Team - Source Selection Plan - Definitions - Color Ratings - Past Performance - Pre-Source Selection Process

- Cost & Cost Price Risk Assessment


- Proposal Risk - Documentation

- LPTA/PPT
- Discussion Process - Training - DAPA Associated - Miscellaneous

Effectivity of the Changes


31 Mar 08
All source selection plans approved on or after 31 Mar 08 must comply with the revised procedures Compliance with the revised procedures is optional for source selection plans approved before 31 Mar 08

Factors Mission Capability


Current: Two factors
Mission Capability (MC) - assessed offerors capability to meet Government requirements, at the subfactor level if applicable Proposal Risk -assessed the risk of the offerors proposed approach, using any MC subfactors

New: One factor with two assessments, at the subfactor level if applicable [AFFARS MP5315.3, Para 4.4.1.1 and 5.5.1]
Mission Capability Technical -assesses offerors capability to meet the governments requirements Mission Capability Risk -assesses the risk of the offerors proposed approach Two ratings are presented together and have equal impact Reflects the current practice of looking at the risk associated with the technical approach of the item or service being proposed Additionally, incorporated the option to use plus + ratings for additional stratification within the risk ratings

Mission Capability Technical Ratings


CURRENT
TABLE 1 - MISSION CAPABILITY RATINGS

NEW
TABLE 1 - MISSION CAPABILITY TECHNICAL RATINGS Color Blue Rating Exceptional Description Exceeds specified minimum performance or capability requirements in a way beneficial to the government. A proposal must have one or more strengths and no deficiencies to receive a blue. Meets specified minimum performance or capability requirements. A proposal must have no deficiencies to receive a green but may have one or more strengths.

Color
Blue

Rating
Exceptional

Description
Exceeds specified minimum performance or capability requirements in a way beneficial to the government; proposal must have one or more strengths and no deficiencies to receive a blue. Meets specified minimum performance or capability requirements delineated in the Request for Proposal; proposal rated green must have no deficiencies but may have one or more strengths. Does not clearly meet some specified minimum performance or capability requirements delineated in the Request for Proposal, but any such uncertainty is correctable. Fails to meet specified minimum performance or capability requirements; proposal has one or more deficiencies. Proposals with an unacceptable rating are not awardable.

Green

Acceptable

Green

Acceptable

Yellow *

Marginal

Yellow *

Marginal

There is doubt regarding whether an aspect of the proposal meets a specified minimum performance or capability requirements, but any such uncertainty is correctable.

Red

Unacceptable

Red

Unacceptable

Fails to meet specified minimum performance or capability requirements. The proposal has one or more deficiencies and is not awardable.

* Through discussions, the government evaluators should obtain the necessary information from offerors with interim Yellow/Marginal ratings to resolve outstanding issues within the offer. Yellow/Marginal ratings should be rare by the time of the final evaluation.

10

Mission Capability Risk Ratings


CURRENT
TABLE 2 - PROPOSAL RISK RATINGS Rating High Description Likely to cause significant disruption of schedule, increased cost or degradation of performance. Risk may be unacceptable even with special contractor emphasis and close government monitoring. Can potentially cause disruption of schedule, increased cost, or degradation of performance. Special contractor emphasis and close government monitoring will likely be able to overcome difficulties. Has little potential to cause disruption of schedule, increased cost or degradation of performance. Normal contractor effort and normal government monitoring will likely be able to overcome any difficulties. Rating Low*

NEW
TABLE 2 MISSION CAPABILITY RISK RATINGS Description Has little potential to cause disruption of schedule, increased cost or degradation of performance. Normal contractor effort and normal government monitoring will likely be able to overcome any difficulties. Can potentially cause disruption of schedule, increased cost or degradation of performance. Special contractor emphasis and close government monitoring will likely be able to overcome difficulties. Likely to cause significant disruption of schedule, increased cost or degradation of performance. Extraordinary contractor emphasis and rigorous government monitoring may be able to overcome difficulties. The existence of a significant weakness (or combination of weaknesses) that is very likely to cause unmitigated disruption of schedule, drastically increased cost or severely degraded performance. Proposals with an unacceptable rating are not awardable.

Moderate

Moderate*

Low

High*

Unacceptable * + Used when risk is in the upper boundaries of a Mission Capabilities Risk Rating but not enough to merit the next inferior rating

11

Factors Past Performance


Reduce Performance Confidence ratings from six to five [AFFARS
MP5315.3, Table 3]

Combined significant and high ratings into Substantial rating as the most advantageous
Removes the duplication and confusion with the High risk rating (less advantageous)

Adapted a Limited rating


Removes the duplication from Low risk (more advantageous)

Separated Unknown confidence rating from the other ratings


Broadened the description to include an offeror with a sparse record can receive an unknown/neutral rating

Removed the requirement for Past Performance to be as important as the most important non-cost factor
Teams can now determine the relative importance of each of the evaluation factors based on the specifics of their source selection
12

Past Performance Ratings


CURRENT
TABLE 3- PERFORMANCE CONFIDENCE ASSESSMENTS Rating HIGH CONFIDENCE Description Based on the offerors performance record, the government has high confidence the offeror will successfully perform the required effort Based on the offerors performance record, the government has significant confidence the offeror will successfully perform the required effort Based on the offerors performance record, the government has confidence the offeror will successfully perform the required effort. Normal contractor emphasis should preclude any problems. No performance record is identifiable. Based on the offerors performance record, substantial doubt exists that the offeror will successfully perform the required effort. Based on the offerors performance record, extreme doubt exists that the offeror will successfully perform the required effort. Rating SUBSTANTIAL CONFIDENCE

NEW
TABLE 3- PERFORMANCE CONFIDENCE ASSESSMENTS Description Based on the offerors performance record, the government has a high expectation that the offeror will successfully perform the required effort. Based on the offerors performance record, the government has an expectation that the offeror will successfully perform the required effort. Based on the offerors performance record, the government has a low expectation that the offeror will successfully perform the required effort. Based on the offerors performance record, the government has no expectation that the offeror will be able to successfully perform the required effort. No performance record is identifiable or the offerors performance record is so sparse that no confidence assessment rating can be reasonably assigned.

SIGNIFICANT CONFIDENCE

SATISFACTORY CONFIDENCE

SATISFACTORY CONFIDENCE

LIMITED CONFIDENCE

UNKNOWN CONFIDENCE LITTLE CONFIDENCE NO CONFIDENCE

NO CONFIDENCE

UNKNOWN CONFIDENCE

13

Factors Cost/Price Risk


Elevated Cost/Price Risk as a separate evaluation factor (moves Cost/Price risk from within the Cost/Price evaluation factor) [AFFARS MP5315.3, Para 4.4.1.4, 5.5.4 and 5.6.5.1]
Only applies to ACAT programs in a System Development and Demonstration (SDD) phase utilizing Most Probable Cost (Cost Reimbursement or Fixed-Price Incentive contract) Mandatory discussions of cost model prior to RFP release

Requires robust discussions with each offeror on how the governments best estimate of Probable Costs is calculated
Each offeror will have its own MPC based on their unique approach

When used, Cost/Price Risk shall be a significant factor

14

Cost / Price Risk Ratings


CURRENT
TABLE 2 - PROPOSAL RISK RATINGS Rating High Description Likely to cause significant disruption of schedule, increased cost or degradation of performance. Risk may be unacceptable even with special contractor emphasis and close government monitoring. Can potentially cause disruption of schedule, increased cost, or degradation of performance. Special contractor emphasis and close government monitoring will likely be able to overcome difficulties. Has little potential to cause disruption of schedule, increased cost or degradation of performance. Normal contractor effort and normal government monitoring will likely be able to overcome any difficulties. Rating Low

NEW
TABLE 4 COST/PRICE RISK RATINGS Description Little difference exists between the offerors proposed cost/price and the government most probable cost which may cause minor or even negligible impact to programs probability of success. Cost growth is unlikely to occur as indicated by this difference and/or other anomalies related to cost/price, but the impact is manageable. Some difference exists between the offerors proposed cost/price and the government most probable cost that may impact the programs probability of success. Cost growth is possible to occur as indicated by this difference and/or other anomalies related to cost/price. Significant difference exists between the offerors proposed cost/price and the government most probable cost which may substantially impact the programs probability of success. Cost growth is likely to occur as indicated by this difference and/or other anomalies related to cost/price.

Moderate

Moderate

Low

High

15

Source Selection Evaluation Matrix


CURRENT
Mission Capability
Subfactor 1 Subfactor 2 Subfactor 3

NEW
Cost/Price Risk *
Mission Capability
Subfactor 1
Risk Rating Technical Rating

Subfactor 2
Risk Rating Technical Rating

Subfactor 3
Risk Rating Technical Rating

Proposal Risk Past Performance Cost/Price *


* This factor may require a risk assessment as described in Paragraph 5.5.4.

Past Performance Cost/Price


* For use on cost reimbursement or fixed-price
incentive contracts where Cost/Price Risk is an Evaluation Factor; Most Probable Cost is utilized

16

More about the Cost/Price Risk Factor


This evaluation factor shall be used for Acquisition Category (ACAT), SDD phase programs that use a Cost Reimbursement or Fixed-Price Incentive type contract structure
The Cost/Price Risk rating assesses the degree to which an offerors cost proposal compares with the governments computed Most Probable Costs (MPC) Cost/Price Risk shall be a significant evaluation factor
For non-ACAT acquisitions when a government MPC is computed, a risk rating may be assigned with SSA approval

The purpose of this risk rating is to provide information to the SSA that allows selection of an offeror who proposed a rational and realistic cost for the work to be accomplished
The Government should state in the RFP the general what methods and/or tools that will be utilized when evaluating MPC during the source selection
The discussions should have already occurred during market research, industry days, and have been part of the draft RFP

17

Uncertainty Definition
Added definition: Uncertainty is a doubt regarding whether an aspect of the proposal meets a material performance or capability requirement
[AFFARS MP5315.3, Para 8.16]

Reference Yellow Mission Capability description It requires additional information from the offeror to further explain the proposal before the evaluator can complete his/her review and analysis and should generate the issuance of an EN

18

Withdrawal of AFFARS Deviation for the Definition of Deficiency


Current AFFARS 5315.001 definition of Deficiency
A material failure of a proposal to meet a Government requirement

AFAC deletes the above definition

FAR 15.001 definition of Deficiency


A material failure of a proposal to meet a Government requirement or a combination of significant weaknesses in a proposal that increases the risk of unsuccessful contract performance to an unacceptable level

AFFARS MP now repeats, for ease of reference, the FAR definitions of Deficiency and Weakness
[AFFARS MP5315.3, Para 8]
19

Planning
Deviations to Part 15.3 (FAR, DFAR, AFFARS, and MPs) can now only be approved by the SCO (Individual) and SAF/AQC (Class)
[AFFARS 5301.4 and MP5301.4]

New requirement to address the use of e-mails during the Source Selection process [AFFARS MP5315.3, Para 4.3.3.5]
If used, e-mails shall be encrypted and include in the subject line Source Selection Information See FAR 2.101 and 3.104

Now optional to include Section L with the Source Selection Plan (SSP) [AFFARS MP5315.3, Para 4.3.3.6]
Personnel changes to the Source Selection Evaluation Team, after SSP approval can be made with SSET Chairperson approval in an addendum to the SSP
[AFFARS MP5315.3, Para 4.1.3.3]

20

Use of Non-Government Personnel


Added further discussions concerning the use of NonGovernment Personnel (NGP) as part of the Source Selection AFFARS 5315.305(c)(2) [AFFARS MP5315.3, Para
4.3.3.8]

NGP cannot have any financial interests nor be the SSA, Chairperson of the SSET/SSAC or an advisor to or a member of the PCAG

Attachment 1, Source Selection Information Briefing and Debriefing Certificate was changed to Source Selection Non-Disclosure Agreement [AFFARS MP5315.3, Atch 1]
Area for Non-government personnel was added to acknowledge non-disclosure and non-financial interest

21

Use of Non-Government Personnel in Source Selections


SAF/AQ Policy Memo dated 11 Jul 07 The Memo
States that the use of non-governmental personnel is permissible, with restrictions; Restrictions include written approval by the HCA (or Center Commanders) and offeror agreement; A contractor that gains access to the proprietary information of other companies during source selection must agree with other companies to protect their information; the contracting officer must obtain a copy of this agreement(s); Non-governmental personnel shall not serve as an SSA, PCAG member, Chairman of a SSE/SSAC; and Shall not have any financial interests with any of the proposed offerors

22

Notification Requirement
Moved the reporting notification for source selections >$100M from AFFARS MP5315.3 to AFFARS and expanded the definition to require notification of all competitive actions >$100M
[AFFARS 5315.002 and MP5315.002]

Now required for acquisitions using


Performance Price Tradeoff (PPT) Lowest Price Technically Acceptable (LPTA) Fair Opportunity/Multiple Award Contracts

23

Other Changes
Several other clarifications and updates to the MP and IG were made. For example:
SSA designations were updated to include a reference to Direct Reporting Units [AFFARS MP5315.3, Para 4.1 Table] All Tables now reflect the most advantageous rating as the one on top [AFFARS MP5315.3, Tables 2 and 4]
Additional clarifications to guidance were added

Oral Presentations consideration can be given to declaring a competitive range up front and entering into discussions with offerors during their presentation
[AFFARS MP5315.3, Para 5.6.5.2]
24

Source Selection Training Team


SAF/ACE (acquisitions >$100M) or local ACE offices conducting standardized source selection training
Current training guides are being updated to reflect these changes
ACE Offices available for source selection questions and support

Training modules already developed - https://ace.hq.af.mil/sstt//


Source Selection Training Source Selection Authority (SSA) Source Selection Advisory Council (SSAC) Chairperson Source Selection Evaluation Team (SSET) Chairperson Development of Sections L & M Evaluation of Mission Capability and Proposal Risk Past Performance Cost/Price Planning Cost/Price Evaluation: Firm-Fixed-Price Cost/Price Evaluation: Cost Reimbursable Debriefing A-76 posting soon

SAF/ACE modules will be updated by the end of February


25

Source Selection Improvement Team (SSIT) Recommended Future Actions


Establish Past Performance Alternative Methods Team
Develop streamlined approaches other than the resource-intensive Performance Confidence Assessment Group & publish in Air Force Past Performance Evaluation Guide. Status: First team meeting February 2008. Status: Separate team has selected EZ Source Tool for AF-wide use. Air Force developing implementation plan and allocating funding. Status: Developing one top-level, electronic source selection process guide based upon SSIT recommended standard Source Selection Process Flow
Creating links to sub-process guides/chapters on specific steps (e.g., SSP, Sec L&M, PAR, SSDD, etc.)

Establish separate team to select a standard AF Electronic Source Selection Tool

Consolidate the many disparate source selection guides

26