You are on page 1of 38
Session 9 CPS Edits, Verification Selection, and FSA’s Quality Analysis Tool Session 9- 2

Session 9

CPS Edits, Verification Selection, and FSA’s Quality Analysis Tool

Session 9 CPS Edits, Verification Selection, and FSA’s Quality Analysis Tool Session 9- 2

Session 9- 2

Session 9 CPS Edits, Verification Selection, and FSA’s Quality Analysis Tool Session 9- 2
Introduction n Presenters: – Jeanne Saunders, FSA – Teri Hunt, ORC Macro/CPS – Francine Reeves, FSA

Introduction

  • n Presenters:

Jeanne Saunders, FSA Teri Hunt, ORC Macro/CPS Francine Reeves, FSA

  • n Who is this session for?

All levels of users

  • n Evaluations

  • n Questions

Introduction n Presenters: – Jeanne Saunders, FSA – Teri Hunt, ORC Macro/CPS – Francine Reeves, FSA

Session 9- 3

Introduction n Presenters: – Jeanne Saunders, FSA – Teri Hunt, ORC Macro/CPS – Francine Reeves, FSA
Agenda n CPS Data Integrity Edits n CPS Verification Selection Process n FSA’s Quality Analysis Tool

Agenda

  • n CPS Data Integrity Edits

  • n CPS Verification Selection Process

  • n FSA’s Quality Analysis Tool

Agenda n CPS Data Integrity Edits n CPS Verification Selection Process n FSA’s Quality Analysis Tool

Session 9- 4

Agenda n CPS Data Integrity Edits n CPS Verification Selection Process n FSA’s Quality Analysis Tool
CPS Data Edits n FAFSA data is subjected to several types of data edits during processing

CPS Data Edits

  • n FAFSA data is subjected to several types of data edits during processing to ensure data integrity

  • n Edits initially developed for paper filers but have been expanded and revised to include electronic filers

CPS Data Edits n FAFSA data is subjected to several types of data edits during processing

Session 9- 5

CPS Data Edits n FAFSA data is subjected to several types of data edits during processing
CPS Data Edits cont. n Edits developed based on eligibility requirements, consistent responses to application questions,

CPS Data Edits cont.

  • n Edits developed based on eligibility requirements, consistent responses to application questions, statistical data, correction behavior, policy issues, results of verification, etc.

  • n Re-evaluated and revised annually

CPS Data Edits cont. n Edits developed based on eligibility requirements, consistent responses to application questions,

Session 9- 6

CPS Data Edits cont. n Edits developed based on eligibility requirements, consistent responses to application questions,
CPS Data Edits cont. n Three basic types of edits performed: – Assumptions – Rejects –

CPS Data Edits cont.

  • n Three basic types of edits performed:

Assumptions Rejects Warnings

CPS Data Edits cont. n Three basic types of edits performed: – Assumptions – Rejects –

Session 9- 7

CPS Data Edits cont. n Three basic types of edits performed: – Assumptions – Rejects –
CPS Data Edits cont. n Assumptions – Use available data to make reasonable assumptions for missing

CPS Data Edits cont.

  • n Assumptions

Use available data to make reasonable assumptions for missing or inconsistent answers

Allow processing to continue and EFC to be calculated for more applicants

Only requires a correction if assumption is incorrect

CPS Data Edits cont. n Assumptions – Use available data to make reasonable assumptions for missing

Session 9- 8

CPS Data Edits cont. n Assumptions – Use available data to make reasonable assumptions for missing
CPS Data Edits cont. n Rejects – Critical data is missing or obvious errors are identified

CPS Data Edits cont.

  • n Rejects

Critical data is missing or obvious errors are identified

Requires correction to add or change data Non-verifiable – data must be changed (numeric codes)

Verifiable – data may be verified as correct or changed (alpha codes)

CPS Data Edits cont. n Rejects – Critical data is missing or obvious errors are identified

Session 9- 9

CPS Data Edits cont. n Rejects – Critical data is missing or obvious errors are identified
CPS Data Edits cont. n Rejects (cont.) – Examples: • Reject 5 – Date of Birth

CPS Data Edits cont.

  • n Rejects (cont.)

Examples:

Reject 5 – Date of Birth is missing; must be provided to continue processing and calculate EFC

Reject C – Taxes Paid is greater than or equal to 40% of AGI; data can be verified as correct or corrected to new value to continue processing and calculate EFC

CPS Data Edits cont. n Rejects (cont.) – Examples: • Reject 5 – Date of Birth

Session 9-10

CPS Data Edits cont. n Rejects (cont.) – Examples: • Reject 5 – Date of Birth
CPS Data Edits cont. n Warnings – Reported data is unusual but not severe enough to

CPS Data Edits cont.

  • n Warnings

Reported data is unusual but not severe enough to be rejected

Data highlighted for applicant review; corrected if necessary

Example: Independent student reports “No” to Legal dependents/Do you have children and number of family members is greater than 2

CPS Data Edits cont. n Warnings – Reported data is unusual but not severe enough to

Session 9-11

CPS Data Edits cont. n Warnings – Reported data is unusual but not severe enough to
CPS Data Edits cont. n Assumptions, rejects, and some warning edits are performed by electronic products

CPS Data Edits cont.

  • n Assumptions, rejects, and some warning edits are performed by electronic products (EDExpress and Web applications) to increase accuracy of data and reduce need for corrections

CPS Data Edits cont. n Assumptions, rejects, and some warning edits are performed by electronic products

Session 9-12

CPS Data Edits cont. n Assumptions, rejects, and some warning edits are performed by electronic products
CPS Verification Selection Process n Quality Control studies conducted in the early 1980’s showed significant applicant

CPS Verification Selection Process

  • n Quality Control studies conducted in the early 1980’s showed significant applicant error

  • n To minimize applicant error, in 1986-87 ED implemented an integrated verification process for Title IV programs

  • n CPS edits developed to select applications with potential errors

  • n Current verification process not perfect

CPS Verification Selection Process n Quality Control studies conducted in the early 1980’s showed significant applicant

Session 9-13

CPS Verification Selection Process n Quality Control studies conducted in the early 1980’s showed significant applicant
CPS Verification Selection Process cont. IRS Match n Provision added to 1998 Higher Education Amendments authorizing

CPS Verification Selection Process cont.

IRS Match

  • n Provision added to 1998 Higher Education Amendments authorizing ED to confirm with IRS AGI, taxes paid, number of exemptions, and tax filing status reported on FAFSA

  • n Treasury Department has determined that Internal Revenue Code (IRC) must be amended before IRS match with ED can be implemented

CPS Verification Selection Process cont. IRS Match n Provision added to 1998 Higher Education Amendments authorizing

Session 9-14

CPS Verification Selection Process cont. IRS Match n Provision added to 1998 Higher Education Amendments authorizing
CPS Verification Selection Process cont. IRS Match n IRS income match is one of ED’s top

CPS Verification Selection Process cont.

IRS Match

  • n IRS income match is one of ED’s top priorities

  • n ED and Treasury worked together to draft revised IRC language to forward to Congress; passage is pending

CPS Verification Selection Process cont. IRS Match n IRS income match is one of ED’s top

Session 9-15

CPS Verification Selection Process cont. IRS Match n IRS income match is one of ED’s top
CPS Verification Selection Process cont. ED/IRS Statistical Study n In 2001 and 2002, ED and the

CPS Verification Selection Process cont.

ED/IRS Statistical Study

  • n In 2001 and 2002, ED and the IRS conducted joint statistical studies to --

Determine frequency of income misreporting on the FAFSA

Calculate Pell Grant awards that would have been awarded had IRS income been used in need analysis

Estimate extent of Pell Grant award error

CPS Verification Selection Process cont. ED/IRS Statistical Study n In 2001 and 2002, ED and the

Session 9-16

CPS Verification Selection Process cont. ED/IRS Statistical Study n In 2001 and 2002, ED and the
CPS Verification Selection Process cont. ED/IRS Statistical Study n Study compared both students’ and parents’ FAFSA

CPS Verification Selection Process cont.

ED/IRS Statistical Study

  • n Study compared both students’ and parents’ FAFSA AGI and taxes paid information to IRS

  • n IRS provided aggregated statistical tables to ED

CPS Verification Selection Process cont. ED/IRS Statistical Study n Study compared both students’ and parents’ FAFSA

Session 9-17

CPS Verification Selection Process cont. ED/IRS Statistical Study n Study compared both students’ and parents’ FAFSA
CPS Verification Selection Process cont. ED/IRS Statistical Study n Results show significant -- – Discrepancies between

CPS Verification Selection Process cont.

ED/IRS Statistical Study

  • n Results show significant --

Discrepancies between FAFSA and IRS income

Erroneous payments in the Federal Pell Grant Program

CPS Verification Selection Process cont. ED/IRS Statistical Study n Results show significant -- – Discrepancies between

Session 9-18

CPS Verification Selection Process cont. ED/IRS Statistical Study n Results show significant -- – Discrepancies between
CPS Verification Selection Process cont. Development of Verification Selection Criteria n Annual Process – New selection

CPS Verification Selection Process cont.

Development of Verification Selection Criteria

  • n Annual Process

New selection criteria developed

Designed to evaluate and strengthen the verification approach

Thorough statistical and empirical analysis performed

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Annual Process – New selection

Session 9-19

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Annual Process – New selection
CPS Verification Selection Process cont. Development of Verification Selection Criteria n Objectives – Goal of selecting

CPS Verification Selection Process cont.

Development of Verification Selection Criteria

  • n Objectives

Goal of selecting minimum effective number of applicants who have the most error

Total misallocated payments considered

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Objectives – Goal of selecting

Session 9-20

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Objectives – Goal of selecting
CPS Verification Selection Process cont. Development of Verification Selection Criteria n Data Used – Minimum of

CPS Verification Selection Process cont.

Development of Verification Selection Criteria

  • n Data Used

Minimum of 2 years applicant and payment data used in analysis

Use of IRS data from tax year 2000 added for development of 2003-2004 selection criteria

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Data Used – Minimum of

Session 9-21

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Data Used – Minimum of
CPS Verification Selection Process cont. Development of Verification Selection Criteria n Data Used – Income-related variables

CPS Verification Selection Process cont.

Development of Verification Selection Criteria

  • n Data Used

Income-related variables used as main predictors

Includes calculation of estimated taxes and comparison to reported taxes

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Data Used – Income-related variables

Session 9-22

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Data Used – Income-related variables
CPS Verification Selection Process cont. Development of Verification Selection Criteria n Use of Random and Immune

CPS Verification Selection Process cont.

Development of Verification Selection Criteria

  • n Use of Random and Immune groups

2.5% of applicants selected randomly

2.5% of applicants not selected even if criteria are met (held immune from selection)

Used as control groups to measure effectiveness of targeted selection

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Use of Random and Immune

Session 9-23

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Use of Random and Immune
CPS Verification Selection Process cont. Development of Verification Selection Criteria n Applicants excluded from analysis –

CPS Verification Selection Process cont.

Development of Verification Selection Criteria

  • n Applicants excluded from analysis

QA schools Schools who do 100% verification

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Applicants excluded from analysis –

Session 9-24

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Applicants excluded from analysis –
CPS Verification Selection Process cont. Development of Verification Selection Criteria n Analysis Approach – Changed statistical

CPS Verification Selection Process cont.

Development of Verification Selection Criteria

  • n Analysis Approach

Changed statistical procedure to Stepwise Linear Regression to produce error equations

Separate equations produced using IRS data and applicant/payment data

Equations then integrated to define final criteria

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Analysis Approach – Changed statistical

Session 9-25

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Analysis Approach – Changed statistical
CPS Verification Selection Process cont. Development of Verification Selection Criteria n Criteria defined for each dependency

CPS Verification Selection Process cont.

Development of Verification Selection Criteria

  • n Criteria defined for each dependency status

63 for dependents 61 for independents

  • n Score for meeting each criteria determined by regression equation

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Criteria defined for each dependency

Session 9-26

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Criteria defined for each dependency
CPS Verification Selection Process cont. Development of Verification Selection Criteria n Selection based on score cut-off

CPS Verification Selection Process cont.

Development of Verification Selection Criteria

  • n Selection based on score cut-off that selects 30% of applicants with the most potential error, and that error would result in significant change in grant award

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Selection based on score cut-off

Session 9-27

CPS Verification Selection Process cont. Development of Verification Selection Criteria n Selection based on score cut-off
CPS Verification Selection Process cont. 30% Verification Limit Option n If CPS selects fewer than 30%

CPS Verification Selection Process cont.

30% Verification Limit Option

  • n If CPS selects fewer than 30% of school’s applications for federal aid, school must verify all selected applicants

  • n If CPS selects more than 30%, school has options --

Verify all applicants selected, or

Limit verification to 30% of total number of federal aid applicants at the school

CPS Verification Selection Process cont. 30% Verification Limit Option n If CPS selects fewer than 30%

Session 9-28

CPS Verification Selection Process cont. 30% Verification Limit Option n If CPS selects fewer than 30%
CPS Verification Selection Process cont. 30% Verification Limit Option n ED strongly encourages verification of all

CPS Verification Selection Process cont.

30% Verification Limit Option

  • n ED strongly encourages verification of all selected students

  • n Schools choosing to verify more than 30% can --

Follow verification requirements outlined in the Application and Verification Guide for all selected applicants (see chapter 3), or

Follow federal verification requirements for 30% of applicants, but use own procedures for those beyond 30%

CPS Verification Selection Process cont. 30% Verification Limit Option n ED strongly encourages verification of all

Session 9-29

CPS Verification Selection Process cont. 30% Verification Limit Option n ED strongly encourages verification of all
Quality Analysis Verification Tool n Any school can use the Tool to analyze FAFSA elements to

Quality Analysis Verification Tool

  • n Any school can use the Tool to analyze FAFSA elements to determine the effectiveness of localized verification

Quality Analysis Verification Tool n Any school can use the Tool to analyze FAFSA elements to

Session 9-30

Quality Analysis Verification Tool n Any school can use the Tool to analyze FAFSA elements to
What is the Quality Analysis Tool? n Part of the EDE Suite of Software n Available

What is the Quality Analysis Tool?

  • n Part of the EDE Suite of Software

  • n Available for all schools in ’02-03

  • n Analyze FAFSA application/ISIR record information

  • n Generate reports showing effect of changes in data on EFC/Pell eligibility

  • n Improve/customize institutional verification procedures

  • n Inform the Federal Verification process

What is the Quality Analysis Tool? n Part of the EDE Suite of Software n Available

Session 9-31

What is the Quality Analysis Tool? n Part of the EDE Suite of Software n Available
What is Locally Designed Verification? n Focus on local sources of error n Focus on the

What is Locally Designed Verification?

  • n Focus on local sources of error

  • n Focus on the unique population of an individual school or campus

  • n Target resources on error prone populations where impact on EFC/Pell Grant eligibility is greatest

  • n Stop “pestering” applicants that report accurately or have changes that have zero impact

What is Locally Designed Verification? n Focus on local sources of error n Focus on the

Session 9-32

What is Locally Designed Verification? n Focus on local sources of error n Focus on the
The Power of Data n Schools can conduct in-depth analysis of applicant data n A series

The Power of Data

  • n Schools can conduct in-depth analysis of applicant data

  • n A series of reports can be constructed using custom formats, selected fields, and queries to answer questions about your applicant population

The Power of Data n Schools can conduct in-depth analysis of applicant data n A series

Session 9-33

The Power of Data n Schools can conduct in-depth analysis of applicant data n A series
What’s In It For Me? n As schools identify common FAFSA errors, they can educate students

What’s In It For Me?

  • n As schools identify common FAFSA errors, they can educate students and parents

  • n Schools can determine if CPS edits are working for their “targeted” population

  • n If schools choose to verify “above and beyond” 30% the tool can help determine if the additional effort is worth the investment of time, resources, and impact on awards

What’s In It For Me? n As schools identify common FAFSA errors, they can educate students

Session 9-34

What’s In It For Me? n As schools identify common FAFSA errors, they can educate students
Field Change Report n Provides a panoramic view of ISIR Fields that have a direct effect

Field Change Report

  • n Provides a panoramic view of ISIR Fields that have a direct effect on the EFC

  • n Illustrates how corrections to ISIR records impact the EFC

  • n A good report for schools to use to begin analysis

Field Change Report n Provides a panoramic view of ISIR Fields that have a direct effect

Session 9-35

Field Change Report n Provides a panoramic view of ISIR Fields that have a direct effect
Related Sessions To learn more about the Quality Analysis Tool: n Session #40: Take the Quality

Related Sessions

To learn more about the Quality Analysis Tool:

  • n Session #40: Take the Quality Analysis Tool for a Test Drive. Hands On.

  • n Session #12, Which, Who, How? And So What?: Using the Quality Analysis Tool to Answer Your Institutional Verification Questions.

  • n Visit the PC lab.

  • n Attend QA Focus Group for New Schools and New QA Coordinators

Related Sessions To learn more about the Quality Analysis Tool: n Session #40: Take the Quality

Session 9-36

Related Sessions To learn more about the Quality Analysis Tool: n Session #40: Take the Quality
We appreciate your feedback! Send your questions and suggestions to: Email: Email: fafsaweb@ncs.com fafsaweb@ncs.com Phone: Phone:

We appreciate your feedback! Send your questions and suggestions to:

Email: Email: fafsaweb@ncs.com fafsaweb@ncs.com Phone: Phone: 11--800 800--433 433--3243 3243

We appreciate your feedback! Send your questions and suggestions to: Email: Email: fafsaweb@ncs.com fafsaweb@ncs.com Phone: Phone:

Session 9-37

We appreciate your feedback! Send your questions and suggestions to: Email: Email: fafsaweb@ncs.com fafsaweb@ncs.com Phone: Phone:
Questions Session 9-38

Questions

Questions Session 9-38
Questions Session 9-38

Session 9-38

Questions Session 9-38